Marionetronics for Animation, Gaming and Installation Art

In this research, I will look at new paradigms in regard to animation, animatronics and gaming. I will postulate that the entertainment media of tomorrow may not only bring about hybridized cinematographic art forms, but move away from two-dimensional screen projections—and the strictly passive role of the onlooker—into more interactive and immersive manifestations, akin to gaming and installation art.

It should be first necessary to comment on the historical roots of animation, and allude to some of its intellectual, stylistic and cultural underpinnings. In so doing, I will look at how animation has evolved to become formally heterogeneous and prone to experimentation, and how it readily integrated computational technologies to create a new hybridized cinema, the evolution of which is ongoing.

Consequently, in reference to photorealistic 3D computer animation, I should briefly analyze some of the tensions and considerations brought forth by this technological evolution. One of the problems being the psychological turn-off of seeing nearly perfect animation of humans—what has been called “the uncanny valley.” Another issue is the need to define the role of animation as a distinct art form, if such hyperrealist trends prevail.
My purpose is not to focus on these problems as much as to gain a more complete vision of the range of aesthetic implications they presuppose and the new creative paradigms they generate. I will mostly endeavor to delineate two premises.

The first one is concerned with the use of light animatronic tools, or what I like to call “marionetronics.” The method of marionetronics constitutes a stop-motion technique that combines puppetry with automation. It may still utilize a human operator for some of its movements but can be augmented with motorized limbs or complex motion-tracking systems. It can be coupled with 3D animation for the more intricate details such as facial expressions, hand movements and landscapes and it can be integrated within live action movies.

This technique can save on rendering time, help to cut the production costs, insure a homogeneous look and give a stylistic uniqueness. Many of my marionettes are made of molded paper-mache to insure lightness and they are covered with nylon or latex for a smooth-looking skin; but they do not require heavy pneumatics or hydraulics to be powered since the controls, motors and solenoid are all used to pull the strings. They can also be programmed and operated remotely. Since they work by a pulling/releasing string tension mechanism— and from above, as it were—these props are much more versatile and have greater horizontal and vertical mobility on the scene.

Secondly, we will be leaving the filmic applications and explore the real time, real space animation of 3D objects: it is the notion of projecting moving images onto props—or mannequins—as a means to lend them texture, visual definition and presence. This is a departure from the engrained convention of projecting imagery on a flat screen or watching monitors; it not only postulates new ways of interacting with animated pictures but shatters the distinction between actor/spectator since the controller may be sitting with the audience or even be communicating remotely while dictating the marionette’s parts—an instance of merging the director, actor and spectator roles!

Such a situation brings forth the possibility of a fully interactive, game-like theatre where audience members can share the direction, or exclusively direct the play from their portable—wearable—devices or remotely. Only certain parameters could be pre-programmed, such as the length of time each player may be in control of a character and the physical limits of the latter’s movements. Likewise, other spectators could design objects and backdrop sceneries in real time, forcing the actors on stage – human or mannequins, or both – to continuously adapt to the unexpected outcomes.

This approach could also be used for installation artwork, where participants could remote-control the work from their home or in the gallery; or for marketing purposes (to say nothing of educative tool applications), where the programme contents can be uploaded directly from one’s office or wireless device!
This process implies an intricate motion-tracking system with simultaneous multi-angled projection, capable of following a moving prop with pinpoint accuracy. But it could be as simple as a static object being projected upon from three angles, as shown with the piece Moving Head on the portfolio DVD—or online at: http://www.4shared.com/video/fU8UlU1s/Shark_studio_one.html.

Or it could be interactive, using basic infrared motion-tracking, with 4 projectors, as shown with the piece Shark on the portfolio DVD—or online at:
http://www.4shared.com/video/fU8UlU1s/Shark_studio_one.html.

Of course, where pre-rendered movies, simultaneously shot from multiple camera viewpoints are needed—Like in the above examples, the versatility and capability of 3D animation software is put to the task. Once the projector set-up is in place, another difficulty is to follow a prop, which is in movement on a set. Much experimentation remains to be done in this area, as exemplified with the clip Kunqu on the portfolio DVD—or online at:
http://www.4shared.com/video/1U0zwXKf/Kunqu_studiotwo.html.

Nonetheless, the theoretical consideration should constitute an important element of myresearch. Considering that, in order to lay out the premises with an acceptable degree of accuracy, a thorough understanding of the paradigms prevalent in the field of animation, cinematography and media arts (and perhaps marketing) is critical. Through an in-depth analysis of the current trends, it should be possible to assert to what degree, and in what way, this hybridization of disciplines may provide pertinent answers or generate new questions.

Hence, I should assemble evidences from the existing body of literature and draw from those involved in the practice. Among others, I will look at the works of Jan Svankmajer, Royal Deluxe Theatre, Robert Lepage’s Ex Machina, Lemieux.pilon 4d art, Cirque du Soleil, La Fura Dels Baus and Chunky Move. Furthermore, having been involved in this endeavor for more than 16 years, I have gathered evidences of my own. Consequently, It will be appropriate to pursue experimentations in the field of 3D animation, marionette making and motorization of control, infrared and optical motion-tracking techniques, multiple projection synchronization and tracking, speech recognition and synthesized speech.

Excitingly, many recent technological advances have made such a task easier: I’m alluding to new and powerful animation software, breakthroughs in high definition video projection equipment, affordable computer control technology for simple robotics and speech recognition as well as better material for moldable structures (resins, fabric, paper), etc.