Robotic Marionette

[sunithaya]

OK, here it goes. I have been working on this baby for almost two years now, and as expected gone through various iterations, failed solutions (the best for learning! etc. )I think I will start from the beginning because the process to me is more interesting than any final product.

I suspect most robotic projects don't go deep into symbolism, ethics and such, so my project might at times seem more than a little weird, but bear with me, it will all make (some) sense at the end.

The original concept stem from, what else, laziness. My wife an I had been performing a theatre piece she created where I manipulated a 15 feet puppet on stage. It was a lot of fun for the audience but sort of a bummer to me because I could never see the performance. So I decided that I would roboticize the puppet and make it autonomous so I could just sit and relax watching the show Yeah, right!

This is the original stage design for the robotic option. The picture of the big puppet is actually from a performance, as is the human performer (my wife Lorena) so you can see the scale.

This was presented at an international theatre festival in 2003 http://teatronuestro.com/html/taller_suenos.html

I wanted to keep all the mechanisms and controls visible and not hide them as it is usual in theatre or film.
Since I wanted this robot to be autonomous I had to restrict the environment or "world" where this could happen. So in this case my design started with the environment or spatial enclosure where the character would have a fair degree of autonomy.Since I am interested in symbols and how they are used to control our actions, I decided to use a Metatron (מיטטרון) as the floor plan.


As the Performer moves about the Metatron and intersects or interferes with certain points, colors, directions of travel etc. the Creator (that is the name of the marionette) interprets it and (hopefully) reacts accordingly.
The geometry of the Metatron is used as a computable control surface to determine the position of the Creature at all times. This positional information is used among other things to control the gaze of the Creator and respond to the actions of the Performer.
There is a primary and secondary color coding that on the one hand simbolizes the nature of communication between the Creator and its Creature, that of the wavelenghts of light, which act as channels of information.
The external circles represent the limits of the world both physical and as a symbolic construct. There is nothing beyond those limits but the void, the forbidden territory.

For those interested in "sacred geometry", the Metatron is like a two-dimensional expression of a Tesseract http://en.wikipedia.org/wiki/Tesseract

Think of this geometry a little bit like the cell tower structure that enables mobile communications by triangulating your location, except in my case, the visual representation of the Metatron itself is what I used to define "areas of influence" that would trigger certain behaviors of the marionette depending on where the human performer was at the moment.

I will stop here for now. I am actually not sure if the images are visible since the "preview" function does not display them so let's see.

[kankatee]

Interesting.

You mentioned you have been at this for over two years. Do you have a working prototype yet? What do you have so far? What challenges are you facing?

[sunithaya]

Yes, I actually have a finished marionette with which I already performed once. I will get to it later. I just want to show the entire process:-)

(For some reason, links to pictures on my blog, flickr or picasa do not show, so I am forced to upload them to my server which is a pain and makes things slow, unless someone tells me otherwise

Next step in my process was the research on types of joints, movements, levers etc. My main source of inspiration for this are the studies that Leonardo, specially his "preliminary automation studies" he did for his own automatons as well as drawings by other contemporaries like Borelli who did a number of mechanical studies based on human and animal limbs. Here are some examples which in some cases I applied almost exactly to my marionette.




This is a replica based on the previous drawing that was part of ''Leonardo da Vinci. Mensch - Erfinder - Genie'' exhibit, Berlin 2005. Photo by Erik Möller.

This other study by Borelli is very interesting because he then applied the mechanics to increase leverage by offsetting the fulcrum as you can see in the 2nd drawing.





Since at the beginning I intended to build a 5+ meter marionette I investigated different materials that could be light/strong enough.

In a Nuts and Volts article, September 2006, p.78 I read about a very inexpensive, light and extremely strong construction technique, used for BalloonSATs, these are Sounding balloons filled with helium that rise to altitudes over 100,000 feet.

I immediately tested this technique to build the limbs. The composite material consists of 3/4” extruded polystyrene foam sandwiched between either 1/34” triply birch plywood, and/or 1/4” balsa wood. Styrofoam, as is commonly known, has a high compressive strength foam very resistant to water penetration which was a consideration in a high humidity environment such as Florida where we currently live and work (the swamp!).

Weight-strength-cost were important considerations. Given the size of the puppet (approximately 5.5 meters) and the fact that it would be controlled via servo motors, it was imperative that the structure be as light as possible and at the same time be able to withstand the forces applied to the different joints.
The puppet had to be easily assembled and disassembled for transportation and performance and be able to survive the abuse associated with these events.


The foam and plywood are glued with 5 min epoxy and held together between 2x4’s to distribute the pressure evenly. I use yogurt plastic tops to mix the epoxy because as you probably know epoxy does not stick to this kind of plastic so you can reuse forever




I use Jorgensen’s wooden pressure clamps and can’t say enough about them. These ingenious clamps can vary the angle of pressure precisely. Using almost any other type of clamp will result in the sandwiched elements sliding out of alignment due to the viscosity of the glue while drying.


I found that both the birch plywood and the balsa wood, when epoxied to the styrofoam have an enormous resistance to fracture, you cannot break them with your hands alone. The difference is in terms of surface, since the plywood is smooth and hard and the balsa is very soft and easily damaged.
In terms of weight they end up the same due to the different thickness-weight ratio. At every joint an additional element like a plastic or metal tube or bushing that will support the axis, bearing and constrain the joint must be securely attached.

'nuf for tonite

[4mem8]

This is really interesting for me as I also use this type of technique with some of my modeling, i am always building molds for various types of models and weight is crucial to some aircraft that I build, so your robot using these material is very interesting to me, I look on with great interest as you progress with this project, Thanks for posting this. In my profession I use Carbon, Kevlar, carbon/kevler hybrid materials, also Blue foam and a whole range of epoxy's and fillers.

[sunithaya]

Hi 4m4m8, thanks for your comment. Like I said earlier, although I initially intended to build a 5+mt. marionette, at one point the cost of rigging and the required servos etc. became extremely expensive, so I completely redesigned the project scale wise but keeping all the initial requirements intact. Ultimately I was very happy with the result, not only from a practical point of view but also conceptually, as you will see later on.

My original 5mt.puppet, which was the seed for this idea was constructed with carbon fiber, but once dressed up, masked etc. it was way too heavy for other than very expensive servos. The other main problem was of course the rigging, since as a marionette it had to be controlled from above and not many venues could provide that support.

Here is a pic of the theatre performance where you can see the size of the original puppet, the three other characters are human performers.



p.s. I love your work. Last night I saw wally-e the movie again

[4mem8]

M'mmm Looks so cool Sunithaya, I love this site for it's diverse versatility of ppl with their projects. Pleased you liked Wall-E, I have a greater mission with T-1 at present.

[metaform3d]

Fantastic project. I'm looking forward to hearing more about the mechanical as well as the conceptual aspects of this build.

[Adrenalynn]

Ditto!

As a technical aside:

If you click the "Go Advanced" button to go into the advanced editing mode, you'll see under the post box a button labled "Manage attachments". From there you can select images on your local machine and upload them to the server here.

You can then insert them anywhere you want in the text by using the paper-clip icon at the top of the editing window. When you click on it, it will drop-down with a list of the attachments you uploaded, select the one you want and it will put a bbcode link in the post.

This is also nice because it auto-thumbnails the images, protecting the page formatting. When someone is interested, they can click on the thumbnail and it will overlay a larger version right over the post.

Let me know if you need more explanation of how that works!

[sunithaya]

Thanks so much for the tip Adrenalynn, that will make posting smoother and faster.I'll try it now.

Next challenge on the project was to understand, among other things, the relationship and computation between the number of strings required and the degrees of freedom (DOF) desired.

On the robo-puppet, the wires that would normally go to the handheld device, (sometimes called airplane control due to its cross shape) are driven by a series of servo motors, which wind or unwind the strings to achieve the needed posture. We will call the strings the actuators.

There are three different functions the strings perform.
a) support or constrain/reference.
b) movement control
c) special effects.

Since the puppet is basically stationary within the world (it does not go anywhere except your mind!) the support strings have a simple function and will probably only have an up-down movement. The reference string, will allow the puppet to bow and stand straight. As it name implies all other strings reference this string. This is taken from traditional Chinese marionettes and is called the backbone.

The motion control strings control the head and the limbs. Because the character makes ample and sweeping movements with his arms I originally considered controls mounted on swinging devices, which would function as the plucking of the strings do in traditional marionette control.
However, I opted, at the beginning, for a linear extension that could be easily detached and did not need additional motors for swinging. I thought an added benefit of that solution was to provides faster response. The servos at the end of the arms would control a wire loop that simulates the string plucking of a puppeteer.
Here are some rough 3D models I created in Max to simulate the system:

This was my first quick prototype with some Vex parts and popsicle sticks.

Once the system is in place, the puppet, the control hardware and the software, it is necessary to train the puppet by establishing neutral, initial and target postures that can express meanings and range of emotions. This is similar to the numerous cell or keyframe elements that a conventional cel animator or “claymator” would have to prepare to follow a script. These groups of keyframes (one for every joint) would be called upon according to the response required by the sensors and their interpretation by the control program.

The Input /Output would be handled by a simple neural net.

The input to the system consist primarily on tracking (image and pattern recognition), on the information provided by the ambient sensors, such as movement, light, sound, proximity, etc. This input is correlated with the matrix representing the mindset of the puppet; his “vision of the world”
The output affects the actuators (motors, sounds, lights), ambient displays and control systems.

Some people argued that following this system the puppet would not really be "autonomous" since it had pre-set positions to represent emotions, etc. But I think they failed to see themselves in the mirror and all the pre-programmed body-language postures, gestures and other cues of which we are all unaware but are triggered by specific situations. And we consider ourselves (at least some people do) "autonomous". I am deterministic myself , so there

Some of the considerations that must be taken into account to perform the initial set of calculations include:

- The number of links in the system
- The link length
- The mass
- The center of mass
- The inertia moment (which in the case of a string operated marionette is quite substantial!)
- The joint viscosity

For next installment I will come down to earth and explain what happened on the way to the forum. Basically the 5+ mt marionette became implausible in my constrained economy so after much deliberation, soul searching and checkbook-gazing I decided to radically alter the size of the marionette to a more manageable 45cm. Mind you that all the other requirements had to remaine the same, namely autonomy, response to human performer/s etc. This created an entirely new set of challenges to be explained...

[sunithaya]

OK, so then, after careful consideration and encouragement from my mentor, Deborah Ascheim the robo-puppet went from 5.5 meters to 45cm, quite a change in every respect. But I liked it more and more. (and so did my wife and my wallet!)

I re-designed the body to take advantage of the superb Püterschein system described by W.A. Dwiggins in his 1939 handbook "Marionette in Motion." published and authored by Dwiggins under the pseudonym "Hermann Püterschein"

The basic and extraordinarily simple idea is that when designing the mechanics of a marionette, you consider gravity as the only force operating on it, and that "...your function with the string is simply to control the motion that results from the pull of gravity.

I find that Dwiggins work, considering the scope, quality and influence on the design world, is grossly underrepresented on the web. The link I offered above which is one of the more complete (at least it mentions that he was a puppeteer in addition to an influential book and typography designer) attests to that.


Based on one of Dwiggins models as a starting point I proceeded to modify the proportions, center of gravity etc. to fit the Creator's body.

This sequence shows the progression from the original proportions of a female marionette body based on the Püterschein system and the modifications it went through to achieve the desired proportions. Figures 1-3 show the position of lead weights in the shin and the pelvis (grey markers). In figure #4 you can see (red dots) the alignment of the different parts of the body. Once the parts are constructed but before they are completely finished each one needs to be counterbalanced so that with minimum effort and resources (servos, number of strings) the marionette can achieve a maximum of expressivity.

This of course is a gross simplification but it is the starting template from which the basic blocks will be cut and shaped. After that a careful balance needs to be achieved between all the parts so that it actually let's gravity do most of the work.

This is Lilith, one of the many characters built and rejected ! by W.A. Dwiggins, although it is...perfect.

From: "a complete experimental Theatre in Miniature; the Dwiggins Marionettes" by Dorothy Abbe
the most beautiful and comprehensive book on the master's work. I highly recommend this book to anyone interested in perfection in craft. You might be able to find it at a major library (you can still find it in Amazon where it ranges from $55 to $125, worth every hard-earned penny)

Gone to eat some pasta, hmmm