View Full Version : [Contest Entry] Robotic Marionette

11-30-2008, 06:25 PM
OK, here it goes. I have been working on this baby for almost two years now, and as expected gone through various iterations, failed solutions (the best for learning!:robotsurprised: etc. )I think I will start from the beginning because the process to me is more interesting than any final product.

I suspect most robotic projects don't go deep into symbolism, ethics and such, so my project might at times seem more than a little weird, but bear with me, it will all make (some) sense at the end.:wink:

The original concept stem from, what else, laziness. My wife an I had been performing a theatre piece she created where I manipulated a 15 feet puppet on stage. It was a lot of fun for the audience but sort of a bummer to me because I could never see the performance. So I decided that I would roboticize the puppet and make it autonomous so I could just sit and relax watching the show:tongue: Yeah, right!

This is the original stage design for the robotic option. The picture of the big puppet is actually from a performance, as is the human performer (my wife Lorena) so you can see the scale.
This was presented at an international theatre festival in 2003 http://teatronuestro.com/html/taller_suenos.html

I wanted to keep all the mechanisms and controls visible and not hide them as it is usual in theatre or film.
Since I wanted this robot to be autonomous I had to restrict the environment or "world" where this could happen. So in this case my design started with the environment or spatial enclosure where the character would have a fair degree of autonomy.Since I am interested in symbols and how they are used to control our actions, I decided to use a Metatron (מיטטרון) as the floor plan.

As the Performer moves about the Metatron and intersects or interferes with certain points, colors, directions of travel etc. the Creator (that is the name of the marionette) interprets it and (hopefully) reacts accordingly.
The geometry of the Metatron is used as a computable control surface to determine the position of the Creature at all times. This positional information is used among other things to control the gaze of the Creator and respond to the actions of the Performer.
There is a primary and secondary color coding that on the one hand simbolizes the nature of communication between the Creator and its Creature, that of the wavelenghts of light, which act as channels of information.
The external circles represent the limits of the world both physical and as a symbolic construct. There is nothing beyond those limits but the void, the forbidden territory.

For those interested in "sacred geometry", the Metatron is like a two-dimensional expression of a Tesseract http://en.wikipedia.org/wiki/Tesseract

Think of this geometry a little bit like the cell tower structure that enables mobile communications by triangulating your location, except in my case, the visual representation of the Metatron itself is what I used to define "areas of influence" that would trigger certain behaviors of the marionette depending on where the human performer was at the moment.

I will stop here for now. I am actually not sure if the images are visible since the "preview" function does not display them so let's see.

11-30-2008, 09:01 PM

You mentioned you have been at this for over two years. Do you have a working prototype yet? What do you have so far? What challenges are you facing?

11-30-2008, 09:51 PM
Yes, I actually have a finished marionette with which I already performed once. I will get to it later. I just want to show the entire process:-)

(For some reason, links to pictures on my blog, flickr or picasa do not show, so I am forced to upload them to my server which is a pain and makes things slow, unless someone tells me otherwise:confused:

Next step in my process was the research on types of joints, movements, levers etc. My main source of inspiration for this are the studies that Leonardo, specially his "preliminary automation studies" he did for his own automatons as well as drawings by other contemporaries like Borelli who did a number of mechanical studies based on human and animal limbs. Here are some examples which in some cases I applied almost exactly to my marionette.

http://arturosinclair.com/Images/cable sytem arms.jpg

This is a replica based on the previous drawing that was part of ''Leonardo da Vinci. Mensch - Erfinder - Genie'' exhibit, Berlin 2005. Photo by Erik Möller.

This other study by Borelli is very interesting because he then applied the mechanics to increase leverage by offsetting the fulcrum as you can see in the 2nd drawing.



Since at the beginning I intended to build a 5+ meter marionette I investigated different materials that could be light/strong enough.

In a Nuts and Volts article, September 2006, p.78 I read about a very inexpensive, light and extremely strong construction technique, used for BalloonSATs, these are Sounding balloons filled with helium that rise to altitudes over 100,000 feet.

I immediately tested this technique to build the limbs. The composite material consists of 3/4” extruded polystyrene foam sandwiched between either 1/34” triply birch plywood, and/or 1/4” balsa wood. Styrofoam, as is commonly known, has a high compressive strength foam very resistant to water penetration which was a consideration in a high humidity environment such as Florida where we currently live and work (the swamp!).

Weight-strength-cost were important considerations. Given the size of the puppet (approximately 5.5 meters) and the fact that it would be controlled via servo motors, it was imperative that the structure be as light as possible and at the same time be able to withstand the forces applied to the different joints.
The puppet had to be easily assembled and disassembled for transportation and performance and be able to survive the abuse associated with these events.

The foam and plywood are glued with 5 min epoxy and held together between 2x4’s to distribute the pressure evenly. I use yogurt plastic tops to mix the epoxy because as you probably know epoxy does not stick to this kind of plastic so you can reuse forever


I use Jorgensen’s wooden pressure clamps and can’t say enough about them. These ingenious clamps can vary the angle of pressure precisely. Using almost any other type of clamp will result in the sandwiched elements sliding out of alignment due to the viscosity of the glue while drying.

I found that both the birch plywood and the balsa wood, when epoxied to the styrofoam have an enormous resistance to fracture, you cannot break them with your hands alone. The difference is in terms of surface, since the plywood is smooth and hard and the balsa is very soft and easily damaged.
In terms of weight they end up the same due to the different thickness-weight ratio. At every joint an additional element like a plastic or metal tube or bushing that will support the axis, bearing and constrain the joint must be securely attached.

'nuf for tonite:)

11-30-2008, 10:51 PM
This is really interesting for me as I also use this type of technique with some of my modeling, i am always building molds for various types of models and weight is crucial to some aircraft that I build, so your robot using these material is very interesting to me, I look on with great interest as you progress with this project, Thanks for posting this. In my profession I use Carbon, Kevlar, carbon/kevler hybrid materials, also Blue foam and a whole range of epoxy's and fillers.

12-01-2008, 12:00 AM
Hi 4m4m8, thanks for your comment. Like I said earlier, although I initially intended to build a 5+mt. marionette, at one point the cost of rigging and the required servos etc. became extremely expensive, so I completely redesigned the project scale wise but keeping all the initial requirements intact. Ultimately I was very happy with the result, not only from a practical point of view but also conceptually, as you will see later on.

My original 5mt.puppet, which was the seed for this idea was constructed with carbon fiber, but once dressed up, masked etc. it was way too heavy for other than very expensive servos. The other main problem was of course the rigging, since as a marionette it had to be controlled from above and not many venues could provide that support.

Here is a pic of the theatre performance where you can see the size of the original puppet, the three other characters are human performers.


p.s. I love your work. Last night I saw wally-e the movie again:happy:

12-01-2008, 12:34 AM
M'mmm Looks so cool Sunithaya, I love this site for it's diverse versatility of ppl with their projects. Pleased you liked Wall-E, I have a greater mission with T-1 at present.

12-01-2008, 02:56 AM
Fantastic project. I'm looking forward to hearing more about the mechanical as well as the conceptual aspects of this build.

12-01-2008, 01:01 PM

As a technical aside:

If you click the "Go Advanced" button to go into the advanced editing mode, you'll see under the post box a button labled "Manage attachments". From there you can select images on your local machine and upload them to the server here.

You can then insert them anywhere you want in the text by using the paper-clip icon at the top of the editing window. When you click on it, it will drop-down with a list of the attachments you uploaded, select the one you want and it will put a bbcode link in the post.

This is also nice because it auto-thumbnails the images, protecting the page formatting. When someone is interested, they can click on the thumbnail and it will overlay a larger version right over the post.

Let me know if you need more explanation of how that works!

12-01-2008, 07:47 PM
Thanks so much for the tip Adrenalynn, that will make posting smoother and faster.I'll try it now.

Next challenge on the project was to understand, among other things, the relationship and computation between the number of strings required and the degrees of freedom (DOF) desired.:eek:

On the robo-puppet, the wires that would normally go to the handheld device, (sometimes called airplane control due to its cross shape) are driven by a series of servo motors, which wind or unwind the strings to achieve the needed posture. We will call the strings the actuators.


There are three different functions the strings perform.
a) support or constrain/reference.
b) movement control
c) special effects.

Since the puppet is basically stationary within the world (it does not go anywhere except your mind!) the support strings have a simple function and will probably only have an up-down movement. The reference string, will allow the puppet to bow and stand straight. As it name implies all other strings reference this string. This is taken from traditional Chinese marionettes and is called the backbone.

The motion control strings control the head and the limbs. Because the character makes ample and sweeping movements with his arms I originally considered controls mounted on swinging devices, which would function as the plucking of the strings do in traditional marionette control.
However, I opted, at the beginning, for a linear extension that could be easily detached and did not need additional motors for swinging. I thought an added benefit of that solution was to provides faster response. The servos at the end of the arms would control a wire loop that simulates the string plucking of a puppeteer.
Here are some rough 3D models I created in Max to simulate the system:


This was my first quick prototype with some Vex parts and popsicle sticks.


Once the system is in place, the puppet, the control hardware and the software, it is necessary to train the puppet by establishing neutral, initial and target postures that can express meanings and range of emotions. This is similar to the numerous cell or keyframe elements that a conventional cel animator or �claymator� would have to prepare to follow a script. These groups of keyframes (one for every joint) would be called upon according to the response required by the sensors and their interpretation by the control program.

The Input /Output would be handled by a simple neural net.

The input to the system consist primarily on tracking (image and pattern recognition), on the information provided by the ambient sensors, such as movement, light, sound, proximity, etc. This input is correlated with the matrix representing the mindset of the puppet; his �vision of the world�
The output affects the actuators (motors, sounds, lights), ambient displays and control systems.

Some people argued that following this system the puppet would not really be "autonomous" since it had pre-set positions to represent emotions, etc. But I think they failed to see themselves in the mirror and all the pre-programmed body-language postures, gestures and other cues of which we are all unaware but are triggered by specific situations. And we consider ourselves (at least some people do) "autonomous". I am deterministic myself , so there;)

Some of the considerations that must be taken into account to perform the initial set of calculations include:

- The number of links in the system
- The link length
- The mass
- The center of mass
- The inertia moment (which in the case of a string operated marionette is quite substantial!)
- The joint viscosity

For next installment I will come down to earth and explain what happened on the way to the forum. Basically the 5+ mt marionette became implausible in my constrained economy so after much deliberation, soul searching and checkbook-gazing I decided to radically alter the size of the marionette to a more manageable 45cm. Mind you that all the other requirements had to remaine the same, namely autonomy, response to human performer/s etc. This created an entirely new set of challenges to be explained...:p

12-01-2008, 08:18 PM
OK, so then, after careful consideration and encouragement from my mentor, Deborah Ascheim (http://www.deborahaschheim.com/) the robo-puppet went from 5.5 meters to 45cm, quite a change in every respect. But I liked it more and more. (and so did my wife and my wallet!)

I re-designed the body to take advantage of the superb Pterschein system described by W.A. Dwiggins (ttp://www.hinghamhistorical.org/html/april2003.html) in his 1939 handbook "Marionette in Motion." published and authored by Dwiggins under the pseudonym "Hermann Pterschein"

The basic and extraordinarily simple idea is that when designing the mechanics of a marionette, you consider gravity as the only force operating on it, and that "...your function with the string is simply to control the motion that results from the pull of gravity.

I find that Dwiggins work, considering the scope, quality and influence on the design world, is grossly underrepresented on the web. The link I offered above which is one of the more complete (at least it mentions that he was a puppeteer in addition to an influential book and typography designer) attests to that.

Based on one of Dwiggins models as a starting point I proceeded to modify the proportions, center of gravity etc. to fit the Creator's body.


This sequence shows the progression from the original proportions of a female marionette body based on the Pterschein system and the modifications it went through to achieve the desired proportions. Figures 1-3 show the position of lead weights in the shin and the pelvis (grey markers). In figure #4 you can see (red dots) the alignment of the different parts of the body. Once the parts are constructed but before they are completely finished each one needs to be counterbalanced so that with minimum effort and resources (servos, number of strings) the marionette can achieve a maximum of expressivity.


This of course is a gross simplification but it is the starting template from which the basic blocks will be cut and shaped. After that a careful balance needs to be achieved between all the parts so that it actually let's gravity do most of the work.

This is Lilith, one of the many characters built and rejected ! by W.A. Dwiggins, although it is...perfect.

From: "a complete experimental Theatre in Miniature; the Dwiggins Marionettes" by Dorothy Abbe
the most beautiful and comprehensive book on the master's work. I highly recommend this book to anyone interested in perfection in craft. You might be able to find it at a major library (you can still find it in Amazon where it ranges from $55 to $125, worth every hard-earned penny)

Gone to eat some pasta, hmmm:tongue:

12-01-2008, 10:44 PM
Back from a delicious plate of pasta with Reggiano Parmesano:tongue:

Sculpting the model

Although I intended to use traditional wood for most of the body, I wanted to try to make some parts with Apoxie Sculpt and Apoxie Clay, some of my favorite sculpting materials for small models, prototypes or part replacements. The only drawback in this case was the weight, but I though that maybe not in the case of the hip which being the center of gravity could have some extra weight.


I left the ball of clay harden a little too much and so after whittling away for a while I finally gave up because it went past the "leathery" state which is so pleasant to work with.

Now it was up to power tools to pick up. I had to wait a couple of days for it to harden enough and not gum-up the blades or the sandpaper. I like to avoid power tools as much as possible because I really love sculpting, whittling and shaping with my hands and some simple tools.


I really like the line of Apoxie products. I recommend them to my students and I use them frequently. They are easy to work with , extremely durable and have good mechanical resistance. Not to mention shelf-life!.

I found a couple of containers that had a bit left and had been "lost" for over three years. They were on the humid and very hot environment of the Florida swamp. That is just the opposite of what they recommend; 'refrigerated and dry to extend shelf life'

Well, other than being a little harder at first, after kneading it for a few minutes they came back to life and behaved as expected.

Now, if they only gave me a discount for the plug :-)


After cutting most of the extra epoxy with the trusty old Dremel (the new ones are plastic crap!) I proceeded to sand and sand away from 80 to 1200 grit until the shape fit my template.

The hip was now so smooth and polished! It felt great to the touch. I thought I could use it instead of wood since it seemed to have the proper weight. Once assembled though, I knew the total weight was too much for the medium servos that I planned to use.


After finishing this stage of the hip I drilled and installed attachment points for the legs and the waist, which is the next piece.

The waist unit is finished except for the holes and restrains that will attach it to the hip. This piece was so much easier because I did not let the Apoxie harden and I was able to model it with my hands almost to the exact shape, so it only took a bit of sanding and polishing. On to the legs!


There are so many ways to make legs and knee joints!

This was my first pass, I didn't think would be the last until it was finished, polished, assembled (the hard part) and tested. I was using white pine, it had quite a bit of resin, which made it hard to carve and sand.


At least it is not as hard or life-critical as the prosthetic knees that biomedical engineers have to fashion for their patients!

I used some of those knee models to make a sculpture and as we found out, it was the hardest and most exotic material (titanium-vanadium) we could ever work with! It was worth it, and also gave me an insight as to how strong and durable future cyborgs will be.

My friend Brad (left) and myself after sweating it out to create the sculpture from conception to delivery in a record three days! Now, why couldn't I work as fast for this project?

Brad and myself created this sculpture as a recognition gift for Dr. J. Crayton Pruitt, who had just donated 10 million dollars (das ist recht!) to the Biomedical Engineering Department of the University of Florida.

Dr. J. Crayton Pruitt, as pleased with his gift as UF was with his! :veryhappy:

So we thought appropriate to use some of those vanadium knee parts to serve as the pivot point of the sculpture, the rest of which consisted of beautiful alabaster that slowed light as it absorbed it, as only alabaster can. The wood base was made with an exotic Brazilian hardwood and the plaque was made from aluminum scavenged from the Aerospace Engineering bin and obsessively polished.:eek:

These are some examples. Imagine the possibilities!
Assuming you have the tools to make a dent or solder the critters.

He must have liked the sculpture because ever since then we receive (Brad and I) a yearly crate of Florida wine from his own vineyard. Thank you Dr. Pruitt! :-)


So far the hardest part was to drill the holes through the Apoxie and line them up correctly. That sent me back to examine very carefully old unfinished puppets from the Dwiggins archive. I learned a valuable lesson: the holes must be drilled BEFORE you shape the form (DUH!), otherwise lining up and holding the shape correctly in different axis becomes almost impossible. That meant that the use of Apoxie for main body parts was not the best solution, since it would be a real waste to create first a block and then cut and sand it down.

That said I am very pleased with the look and feel of those two parts, which strike me as very "Japanese" (or Brancusian) in some way. The only un-japanese thing about them is the misalignment and lack of precision of some of the through-holes on the back due to the reasons explained above. No excuses, (Julie Taymor would be very upset).

The fondest memory I have after enjoying many days watching the construction of a Japanese Tea House by a master carpenter was how every single piece of assemblage, including those that would never be seen by anybody, were as perfect as the main "public" pieces. The sculptural quality and the loving care and treatment of the most humble of materials left a permanent impression on me. I have a long way to go.

Next in line was to attach the legs which were already temporarily assembled and test the lower body for problems. After that I shaped the legs to give the Creator that sophisticated look that I am sure it craved:)

To be continued...

12-02-2008, 01:18 AM
This is so cool, I love reading every bit of it, Keep it coming sunithaya.

12-02-2008, 08:14 PM
The Power of Tools


I started the Creator's torso by finding an appropriate block of wood. I had intended to use pine since it is soft and easy to carve, but I had none of the right dimensions. My only option was cypress, which I like a lot for its color and its pattern, but is considerably harder and demands a lot of attention to the direction in which you whittle it. That means you have to be really careful to follow the pattern of the rings and watch when it changes direction if you don't want your blade to get caught and possibly damage the wood, specially when you approach the final surface.


A very important thing when you carve, specially at the beginning when you apply a lot more force to approximate the shape you need, is to be able to hold the piece securely. There might be other techniques and of course you can improvise, but if you have access to the magnificent and versatile Jorgensen clamps, use them!

If it had been pine I could have done it with a set of X-Acto knives without much problem. But for any hardwood you need proper carving tools, I have found that Warren Tools are an excellent choice. I have had my "traveling set" for over twenty years or more when I bought them to carve my first "professional" puppet out of basswood. I've kept that set like a treasure through many life changes and moving from continent to continent. I have lost many things but not this little box. True, there are finer and more expensive carving tools out there but here you have a balance between price, quality and workmanship.


I talk about tools often because I think that tools and the art they enable are inextricable linked, "...where form and function are most closely related as a way to understand and influence our most basic aesthetic and ethical decisions". I also think that great tool design can only be achieved by someone who has mastered his or her craft and realizes the importance of the interface between your mind and the object you intend to create.


Whittling is not done much anymore since the advent of power tools, the same way that cooking (and eating!) has pathetically been replaced by TV dinners specially in American households and definitely in student's dorms. TV watching has taken the place of meditation and your organizer has replaced your capacity to memorize. All I can say is that the magic of discovering the shape that lies inside that block of wood as you approach its delicate surface is one of the greatest rewards.


The Beauty of the Fall

The beauty of the Pterschein system is, as you can see in this picture, that the puppet assumes very natural positions with no human intervention. I just cut the string that held it up and it fell on its knees just like that! A true predecessor of inverse kinematics!.

It seems like it is praying for arms, looking intently at the cut rods that will give it the expressiveness it needs.

I cut an old broom stick to use for the arms, thinking that it was pine and it would save me some time if it already had a round shape. Ha! I don't know what kind of wood it was, but it was hard as oak and I went through a whole set of carbon steel blades just cutting it down to shape. It took me three full days to get them ready for testing! So much for saving time...:rolleyes:


You can see the Creator now just standing naturally, again with only spine support. I did not pose it, other than bringing it down to contact a surface. Couldn't wait to have the entire body functional. In case you are wondering the "feet" would be two slim inverted metal cones. As for hands it will have none in this iteration, since it will be all dressed up and have very long sleeves like the human performer you can see here (http://bp2.blogger.com/_XxQVqUVtq2M/R4sI4auAKAI/AAAAAAAAAEQ/mmXQnx999KU/s1600-h/animacion_criatura04.gif)

This is a detail of the hip and waist rigging that holds and constrains those parts as the marionette flexes and moves.


After three days, a few cuts and sore muscles overall, the arms were close to ready. I tested them and they moved gracefully and have the needed range. Under the torso you can see the channel I gouged out to make room for a couple of screw eyes to pass the string that connects it to the waist and hip.

Next task: the HEAD! this will be interesting since it will have a wooden core or "skull" and it will wear a "self animated" mask, a fancy way of saying it will have some springs to react to any movement of the head. If I have time, which I doubt, I would like to put a micro-servo on the latex mask so that it can have a range of expressions in addition to the eyes. Lorena, who designed the original creature does not like the idea. Perhaps she is right. The simplicity of the spring solution, which we have demonstrated on performance gives such a varied range of expression to the Creator's face that adding more might be less. So there!:veryhappy:

12-02-2008, 10:03 PM
Hip Gnosis

Do and learn. The hip and waist proved to be not only very heavy but also out of proportion. Looking back, this happened because I scaled the entire original schematic to simply make it taller without really thinking through the proportion of the individual body parts.


So, here I go again. I redesigned both hip and waist and decided to use pine wood instead, cutting the overall weight considerably by almost 300 g. My servos were happy! :D


Since I was going to start from a wood block this time, I was able to drill the holes in a more controlled fashion, sort of, because some of them traversed the block at extreme angles. But overall this fact alone made the parts fit better. Molding the parts with Apoxy was fun and quite easy and fast, but not the right solution in this case.


Here you can see the difference in size and weight, of the old and new hip-waist combo. Of course the scale change rippled through the entire puppet since I had to also modify the torso and the legs. I knew I had to do the same with the arms, which I put on hold for a while because the wood was so amazingly hard. But it happened.

String Instrument


Here is the Creator waiting to have his nervous system installed. The secret to a balanced marionette lies mostly in the stringing that controls it both internally and externally.

The strings that hold the different parts of the body from the inside act as ligaments as well as nerves, that stabilize and enable the parts to move in relationship to each other. By transferring forces and movement information through the spinal string, which is the case with our body, it keeps us balanced and acts as a conduit through which our brain controls our movement.

In the case of a traditional marionette, that brain or intelligence comes from the learned hand of the player of the strings, the puppeteer.

In this case it will be the computer or rather the microprocessor, together with the sensorial system that feeds it information about its environment, that will make the equivalent of conscious decisions as to what to do and when.

The other part, the involuntary movements, the character, arise from the structure that has evolved from interacting with the laws and forces that constitute the semiotic "liquid" in which the creature evolves.


This evolution has produced the structural support (skeleton) and the relationship between the systems that allow both us and a marionette designed under those principles to act and respond accordingly, with or without consciousness or awareness of that which ultimately controls us.

This sequence was produced simply by lowering
the spinal control until it no longer supports the body.

12-03-2008, 11:33 PM
[CENTER]Gimme Shelter


Time to create the scaffolding for the Creator's shelter. It will hopefully be as functional and form(ally) beautiful as it can be. In trying to find the right tools and materials I bought some Vex kits when Radio Shack decided to chuck them. So in a way I got a good deal. From a tinkerer's perspective though I wish I had my (very) old Meccano, which from an engineering perspective was a lot more versatile I believe.

The problem with the Vex "system" is that it is not really versatile. It seems to have been designed, not as an optimal modular construction set, but rather limited to some preconceived ideas as to what you should do with it rather than let you decide, others might disagree.


I find it very limiting for free form design. Take this example. To construct the very simple structure I needed required me to use ALL 5 kits! and it was not enough!.

When you think of which is the most useful and common structural element in construction, I am sure that the majority of "builders" or engineers would agree that it is the humble angle, strong, simple and functional. Well, guess what, the entire Vex kit comes with a whooping 3 angles! yes, that is 1,2,3! But what were they thinking?


This forced me to construct the main columns with rails (of which there are quite a few) which prevented me to attach the rest of the support elements in a simple way.:genmad:

The kit does come with lots of elements that might have some use in some predetermined project but it is the least useful grouping of parts in any construction kit that I have found. There must be a reason behind this, and it is probably very sound, like ka-ching! Almost any project will require you to buy extra hardware, at a very steep price, and you can bet (and would win) that the "dedicated" hardware kits bring a completely unbalanced set of elements as well, so that you do keep on buying. Bear in mind that I am not building anything exotic as you can see from the pics.

The structure is 80 cm. tall and 33cm wide, making it a harmonic rectangle area. Of course it is not exact, since it never occurred to the designers that anybody might need , want or know about such things as harmonic proportions.

OK, I was a little pissed off, since the kits did not fulfill my expectations. :sad:

Stage Hands

Assembling the motion section and adjusting the elements so they fit.

I love that time when you have to finally begin working on the stage for a play. It means the performance is imminent (unless the producer pulls out!). It is also time to listen carefully to the director, designer, performers, lighting and sound people and stage hands and riggers to make sure all points of view are represented and taken into consideration. (if only it was like that in real life...)

Since the hole pattern in the VeX system is not really well designed (unless someone proves me wrong) I had to notch and cut lots of pieces so that they overlapped correctly in the configuration I needed. As I said before, I think the engineer/s who designed the system probably did it with a preset number of projects in their mind.

I would go back to the drawing board. It seems strange to me that in this age of CAD and relational tools, they could not test at least a few thousand combinations and optimize the design for that. So anyway, the metal is soft enough that it doesn't take that long to adjust.

The Vex control modules, servos and power supply.

Except for Lorena, my wife, who is the human performer, choreographer and costume designer for the project and who makes sure her needs and demands are met:-), the rest of the crew is me with occasional help from my friends, like Philip who is working on a Flash Develop interface to control the microprocessors and sensors. I had my doubts about the speed of the image processing functions in the new Open Source Flash, but it was worth a try.

At first I was using VeX to build the stage support and to prototype the motion and control system. I was trying to decide which microprocessor to use, perhaps Propeller by Parallax which is a multi processor where each one of the 8 cogs that can operate simultaneously giving you the ability to respond in real time to performer actions, which is what I needed to do.

Easier said than done, since programming is not my strenght, but my friend Philip loves coding and we have done great work together. I needed to complete the prototype so that I we could start programming and optimizing. Everything simpler, easier, faster. Like Ars Electronica's 2006 theme said, SIMPLICITY - the art of complexity.

The Creator's new feet

I was also getting so much more familiar with the marionette I had created, since while you are whittling away at blocks of wood and dealing with joints, balance and many other issues, it is difficult to meet the creature on its own terms. Although it was not awake yet I could already see a bit of its character and personality. This is an experience which every puppeteer, or for that matter any artist goes through as it transitions from production to contemplation or analysis and finally to understanding or appreciation.

This process is not so familiar to the engineering folks, since for them, in the majority of cases, everything must be defined or though in toto in advance and there is usually little room for improvisation. I don't mean this as a criticism, simply as a pattern that I have observed although as in everything there are exceptional exceptions, and I think many of those exceptions hang out in this forum!;)

Do Not Read This if You Value Your Time!

On the banal but also symptomatic stage of the general deterioration of ethics and quality in our world and in particular where I live, I received a large package on the mail that I had no clue as to the contents, since I did not remember ordering anything as large...

Since I don't have a lathe or access to one at the moment I thought it would be easy to find the small pulleys that I urgently need for my project. As it happens, one of the most difficult things to find have been the pulleys or sheaves needed to raise and lower the marionette's articulations.

Why this is I don't understand. No robotic store or company, specially at the "hobby" or craft level carries such a useful and logic device. Can someone explain?

After much searching I finally found a place that purportedly had what I needed. Based on their description I bought 6 little sheaves to test. Here is what I got!

A ridiculously large box with 6 little pieces of very crappy malformed plastic. Supposedly this place, whose motto is "Leaders in Education" wants to lead students, in step with the current government mandate to the lemmings paradise.

I imagine, they consider children or students simple ignorant people who do not deserve to touch anything decently made that might, just might, teach them something. That way they learn to accept as a fact of life the terribly poor quality of life that education (with few exceptions) in our country provides to the millions, ground and trained through the system to accept mediocrity as the norm.

Take a close look!

We ship the crafts I or my wife make all over the US and the rest of the world. One pound, Priority Mail costs $US 4.80 ...the shipping of these few ounces of bad plastic which cost US$ 1.50/each, cost US$ 7.oo for a total of US$ 16.00 I guess the huge box accounts for the "handling" fees.:genmad:

I understand this post is a waste of everyone's time. But it is also part of the process which includes frustration from which you must recover. So back to a more positive track I decided to build my own pulleys with wood. I shaped them with a sander given the lack of a lathe, but that was OK.

What was I thinking!!!:o


Just to finish the pulley rant, I created a template to drill the round wood pieces that I bought (99c/bag) at the local craft store. They seemed perfectly round in the bag...


Much to my surprise (am I new? or what!) they were far from round, I found out when I tried to fit them in my template. To make it short, I sanded them enough to fit, drilled and glued them together. As I was venting my frustration for not finding proper pulleys, my wife who works next to me in the studio, asked why I did not use the spools that she uses for her beading or the ones that I use for stringing (fish line) the marionette.:o

I must have been very tired, sleeping 4 hours max for months. Why didn't I think of the spools which were in front of me? So, there, I had a steady supply of the perfect size/weight spools.
I only needed to create bushings to adapt to the servos, real servos that is, not the modified ones by Vex which of course have a "proprietary" square shaft that only fits their kit stuff, I still cannot get over their lack of foresight and integration with the rest of the world. So much for Open Anything! Dean Kamen, you know much better than that!

I created the bushings with an aluminum rod that happened to have the right diameter.
I drilled the shaft hole to attach the Hi-Tech servos I intended to use, notched them to hold the adhesive better and epoxied them to the spools.

Command and Control

The recent integration of the Make controller with the open source Flash Develop, its scripting language and image processing capabilities, solved many of the problems I was dealing with, from the choice of microcontroller to the controlling software.

The program currently reads the position of the performer on stage and even the distance from head to ground, all with only one cheap webcam as an input device. There is so much information that can be acquired through image analysis that effectively eliminates the need for a variety of sensors that I had planned to use.

The only limitation is the amount of servo ports in the controller. I had to use at least two, to handle all the servos I needed.

The servo that supports the weight of the marionette most of the time needs much higher torque than the ones I have. I ordered a Hi-Tech Mega Giant Scale Servo ASAP to start testing the software that Philip and I were designing. I was very excited and happy with the progress so far. At first I had my doubts that the solution he proposed would work, but now I see it can, and in a very simple and elegant way.

The two sizes of spools we currently use (5 and 8 cm diameter) happened to have the perfect circumference to provide most of the travel that I need without using continuous rotation motors instead of standard servos. This simplifies programing. We found out by testing a variety of "identical" modified servo motors that the speed of rotation, given equal parameters, is quite different, which implies writing routines to address each one of them to synchronize, a royal pain!

The fish-line spools with their bushings in place

The only continuous rotation motors I needed were for the arms since they travel a much greater distance.

Snap to Grid
Serious construction began. This had to be, want it or not, the last stretch of the building process for the Creator stage.

But before I begin this section I want to share a tip that I take for granted, but friends that see it are always amused by it. You know that when you work on a project involving dozens or hundreds of small pieces, screws, washers and what not, things like to go amiss. Lost socks pale in comparison, specially when you lose a servo screw that you won't find in any hardware store (unless you live in the Bay Area, how I miss that!). And ordering a screw or two online just does not cut it.

So, I use a big magnet block (which I don't remember where I got) and simply and literally throw all those little pieces at it, even from quite a distance and it just snaps them as they try to go by. Sometimes I am working on an odd position, (upside down ?) and as I disassemble something, which happens more and more often as I try to refine my design, I start collecting all these little pieces in my hand and mouth that are eager to disappear in the cracks of the floor or my stomach. Well, just toss'em up like a chef flipping pancakes towards the magnetic field and , snap! they are not going anywhere.

Now, mind you that this is not a very powerful magnet, like those neodymium or ceramic magnets that you find in hard drives otherwise getting a washer back would probably be impossible, but it is strong enough to hold quite a bit of stuff. Sometimes you cannot even see the magnet anymore, just a weird nut screwy sculpture.

Divided and Conquer

That seems to be the design philosophy of the VEX guys. I promise (well, a weak promise) that I won't waste more time bitching about their "close source" design decisions. But someone has to say something. Of course I used their stuff to prototype my project since I had bought 5 or 6 kits with the hope of actually using them for the final project. Now, to be fair, their nuts and screws are first rate and I have no complaints, as well as their cute chain transports if you use their square shafts of course, but the rest...:rolleyes:


It was obvious pretty soon that the electronic/hardware side would not work due to their very flimsy nature and weak linkage (or lack of it). Take their servo for example. Not only is it far from standard, so that you can hardly mix with any other brand, but it lacks fundamental elements like a horn or attachment plate of some sort so that you can drive something more than their absurd square shafts.

And guess what, you cannot even screw anything to it, you can only use their "clutch" (the small cylinder with a short square stub) to attach another square shaft to drive any square holed device! BTW the clutch and therefore anything attached to it, will simply fall off if you turn it upside down. Am I missing a screw here?

You cannot see it in this next picture, but those arms or extensions that I need to drive the limbs of the marionette with have as much strength and stability as an apple stuck on the tip of a drinking straw, can you picture that?. The main reason being that as I explained, the servos connect to anything else through the wimpy clutch mechanism that cannot stand any side pressure, only direct transmission to a wheel at the most.


To be fair, after a great deal of literally hacking away with a saw, drilling and cutting and bending, I was able to put together a somewhat functional scaffolding for the stage. If I had the money I would simply make everything from scratch out of aluminum and some steel pieces. And, yes, chuck those servos away. I opened one to convert it, I was so shocked by the poor craftsmanship inside, probably built on a Chinese sweatshop (badly soldered by hand), that I just closed it and proceeded to order a set of excellent Hi-Tech servos.

Give me a big enough lever...
The real servos finally arrived! here is the Hi-Tech HS-805BB Mega Giant Scale Servo being tested. I simply attached one of the many great horns and plates that it comes with to my fish-line spool, using its own screw and voil! 19.8 / 24.7 kg*cm (4.8V/6V) of torque! It is temporarily held in place with straps as you can see because as you can imagine the square holes of the Vex plates don't line up with anything but their own stuff (I promised I would stop bitching about it!)

For the rest of the controls I am using the HS-645MG UltraTorque Metal Gear Servos that have 7.7 / 9.6 kg*cm (4.8V/6V) of torque, very much on the safe side.

On the left of the picture you see Vex's red optical encoder which I planned to use to determine the position of the continuous rotation motors that lift the arms, but again, not even the extremely versatile Make controller could recognize the simple signal. I am sure it can be done but why waste time. I ordered a set of high-res 10 turn potentiometers that are not only more precise but can "hold" the last position value even when the power is turned off.

The Exoskeleton

I have been literally immersed in a virtual reality. Working at a small scale forces you to adjust your senses and perception of self. Then you suddenly realize that somehow your fingers are too big, your body too clumsy, your eyesight not good enough to deal with a Lilliputian world. Fortunately our cyborgian self comes to the rescue and we make use of the many attachments and extensions that makes us makers of things, creators of our own destiny. (I might believe that someday...)

All the mechanics are functioning, at least by hand and in individual servo tests. It has been very hard to deal with some problems, most of which have to do with balance and friction, not of the marionette which is very fine, thank you, but of the mechanical elevator that raises and lowers most of the machinery so that the head can turn, the body express and the arms move while at the same time the Creator changes its position , kneel, stand up etc.

This problem is not so evident at first, specially if you are used to manipulate a puppet or marionette. Since your hand has a mind of its own it is hard to consciously realize tha amount of intelligence that it has to deal with all these complicated maneuvers. As you move the "plane" controller, pull the strings etc, you hand raises and lowers automatically and compensates this movement by tilting, rolling or yawing the controller.

Potentiometer as high precision encoder

However, when you have a fixed structure that supports the controllers it becomes evident that you need more than one motor to actuate on any given movement, because each movement influences the rest. Of course, as Dwiggins taught us, you should let gravity do most of the work, since she knows all about the laws of physics, but then there is the issue of correspondance, economy, efficiency and why not, elegance of design. Simpler is better. The complexity of simplicity.

Detail of the servo with its sturdy mounting plate
Assembled arm control with lead counterweight
Lead counterweight and potentiometer encoder

12-05-2008, 11:48 PM
Sense and Sensibility

Visualizing the user interface for servo control

The nervous system is the current step (by step). Working on the sensors and on the beacons has given me a clearer understanding that there is nothing I know. Still very very far from The Docta Ignorantia.

Here is our microprocessor stack composed of two Make boards which were our final choice. The boards are highly integrated, very easy to interface and simply well designed. The only drawback was the limitation of 4 servo ports per board. I decided early on that 8 motors were as many as I wanted to use, both for practical and symbolic reasons. Four is the symbol of man generated by the trilogy which comes from the duality contained in the unity (bear with me, it is a stream of thought). 8 represents the power of the king in chess, and the queen in all her potency and (coming down to earth), I could only afford two Make controllers anyway! So, there. I had to do it with only 8 motors, period.

Then came the IR beacon circuit. That was a lot of fun to build although it took a lot of trial and error to do it with kitchen stuff. DYI instructions stipulate in no uncertain terms that you should use glossy inkjet paper and ONLY print with a laser printer, then iron transfer into the copper plate. Well, after many frustrating trials, botched plates and burnt fingers we (Philip and I) decided to try what you are not supposed to do. Went to our local print shop and had the circuit printed on their glossiest color laser paper with their fancy color printers. That worked beautifully. It transfered absolutely perfect the first time around. The entire coating of the paper stuck to the plate, and I am sure more than one has though that was wrong, but after soaking it in warm water the paper peeled off like a decal and left an impecable mask.


So here is my friend Philip, which was the instigator of the whole DYI circuit idea cooking up the circuit with a coctel of muriatic acid mixed with hydrogen peroxide. In a few minutes our circuit was done. As usual he is multitasking with his totally hacked IPhone.

After dealing with the obvious and easy choices, like the arms (4 motors there) I realized the problem. My monster servo would lower the body, yes. But what about the head?, the shoulder strings? (they allow the marionette to turn the upper part of the body to face in another direction). They had to be lowered at the same time! otherwise I would need many more motors and complex programming to synchronize all these elements.

So the idea of the elevator was born, with all its ancillary problems, weight, friction, balance, stability, randomness, etc. At first I built tracks to guide the (square!) rods up and down. These rods (quads?) support a platform that houses the 3 remaining servos. The problem is that the head has more than one degree of freedom of course. It not only has to turn left and right, it also has to tilt sideways and bow as well. Or pitch, yaw and roll if you will.

All these simple motions are trivial (if difficult to master) in a marionette. Then I had the shoulder controls to turn the body around; how to do all that with 3 servos and control the wanted nuances?

After about two weeks of anguish I stumbled upon the solution, where but in a dream!. It is pretty obvious that we work as hard in our oneiric universe as we do in our waking world.

Next morning I skeptically went to examine the contraption with the suspicion that, like many other "great ideas" conceived in la-la land it would turn to be impractical, impossible or just plain turdy. It actually took me a while to connect my ideas with the reality of the mechanics, something was missing which I had apparently forgotten. Then, all of a sudden I saw it! make one of the servos perform double duty.

I cannot even explain how I visualized it. It was so obvious, simple, and yes, elegant. The only bummer is that I feel it did not come from me but from the collective unconscious debris. I just picked up the pieces.

I will post some pictures to explain how it works. ;)

Here it is:


How to solve it

My apologies to George Plya for quoting the title of his jewel of a book, but "How to solve it" made an indelible impression on me when I read it decades ago. I wish I had had him as my math teacher instead of the monster that expelled me from his class for having the nerve to record his "lectures" (which I did not understand) in what must have been one of the earliest portable recorders (the size of a worker's box lunch) that my dad, a very forward thinking man gave me as a present so I could use technology to help me in my studies. This wicked teacher foamed (he became redder than he was and I remember his veins bulging in his neck) that my intent was to make money with his awful expositions, when my only intention was to listen to it again at home and review the parts that were unintelligible to me. Remember it was the time (1960's) of slide rulers and solving logarithms, square roots etc. with pen and paper! and teachers which threw solid wooden rulers at students if they failed to pay attention.

In any case, this is how I went about it:


The two control strings for the head (1 and 4) allow it to not only turn left and right, but as I said before to tilt sideways and bow as well. In a traditional marionette these strings are played like an instrument, varying the tension, twisting and plucking.

When the marionette needs to kneel or jump etc. you are not really concerned with this since the entire body, head and all is balanced by your hand, which uses your own personal embedded experience as a subject of gravity to effect movements which prove extremely hard to program convincingly. Imagine for example that you want the character to tilt his head slightly in an inquisitive way while bowing and kneeling at the same time. If you try to break that into individual movements performed by individual muscles of your hand, fingers and arm you will picture the complexity of the apparent simplicity. Don't forget that I said tilt his head in an inquisitive way!

So, in my dream I must have followed some of the principles Plya suggests to poor souls in distress, like for example:

Look for a pattern
Draw a picture
Solve a simpler problem
Use a model
Work backward
Use a formula
Be ingenious


The 'head' strings reach the horn which would turn the head left and right and you would think that is where the strings should end. Then I visualized (and this is what comes from the dream) that the strings at that point corresponded to my left hand fingers twisting them while my right hand controlled the other movements in unison.


So I let the strings continue on to the other 'hand' which would pull or let go to let the gravity do its thing. The combination of these elements (including the most important, gravity) give you a very rich and expressive palette of head "language". This of course I cannot convey simply by pictures, but I will eventually post a video of the movements.

The 'other hand'

I continued raiding my wife's sewing supplies and used this perfect and beautiful spool, which I epoxied to the servo horn (you can still see behind the spool the gray irregular shape of the rock hard Apoxie that I used. Need to refine that.)

There was another problem, which because of its apparent simplicity tends to be forgotten until it is too late. Remember the analogy with a string instrument. So it is that you must tune the strings of your marionette to the perfect tension which allows precise control while at the same time giving gravity and inertia room to infuse the movement with the illusion of life.

Easier said than done. First of all the strings are very thin and usually waxed or Teflon coated to prevent fraying. That makes tying and untying knots practically impossible. Then consider the awkward and tight places where the strings are connected. Finally, size. Things that are very easy at a human scale become wickedly difficult in Lilliput.

Bowl of Sushi by Hiroshige (Edo Period)

My love of sushi came to the rescue. Chopsticks! the bamboo sticks have the right diameter, are pleasant to touch and sight (and taste) and its transversal structure provides the friction necessary to prevent the 80 lb. Teflon string from sliding. This was my prototype but although it stopped that string diameter from sliding, my preferred string is .006" Fireline, which is a braided bead thread very strong and resistant to abrasion (it is used to weave the smallest glass beads). This string just went through my tensioner like a water snake.


So I refined my prototype incorporating an opposing path that effectively holds it in place. And what is more important and the reason for this exercise, it is easy to adjust as needed, to tune the strings of the instrument.

Tension is good when needed :veryhappy::veryhappy:

12-06-2008, 05:07 PM
IR Beacon

This is the first finished beacon to be worn by the performer on stage. This will be tracked by an overhead camera to detect the position on a grid. It has 24 IR (infrared) LED's. The idea was that, since the tracking camera must have a wide angle in order to see the entire stage from above, we needed an array big enough that would be easily tracked.

What I did not realize was that the angle of the LED's that I got was extremely narrow, so that when the performer lowers or tilts her head (the beacon is on top of her head) the LED's practically disappear. So we came up with the idea of using a diffusion dome to spread the light. That of course is half a ping-pong ball which happened to have the exact required diameter.

The ball was too thick and did not let enough light through, so I sanded it until it felt like a turtle egg, if you ever touched one. The solution worked, however there was still a considerable loss of light.

So I started to look for new LED's. This time I made sure that the lens of the LED would be wide enough to allow the free movement of the performer's head. This is what I found:


Ultra Wide Angle IR 850nm

These LED's are awesome. We could not believe how wide they emit light and how bright they are. You can almost turn the away from the camera and you can still see them!. So they will definitely do the trick without the need for a diffuser. We will use a set of ten which has the added benefit of resulting in a much smaller device and, because it is IR, it will be invisible to the audience.

I bought them fromthe guys at NaturalPoint (http://www.naturalpoint.com/) which have a wide (yes, intended) variety of optical devices, mostly for motion-tracking. A set of ten cost me US$ 15 which is about half of the price of "cheap" LED's at RadioCrap.

So now we will design and burn another circuit and probably use a coin battery instead, to make everything really compact and unobtrusive to wear.

As soon as the user interface is ready for the tracking and motion-assigning program I will post pics of the UI and the tracking screen which is a lot of fun to see in operation.

Chicken Coop

After thinking for what seemed years, about what the material for the background for the stage should be I decided to use something called "hardware cloth". I must admit I had never heard such term until I described what I wanted to a person I know at my local Loew's store. He immediately said, "you want hardware cloth", I said "no it must be metallic!". So he simply lead me to the garden area and noted that it was just a sort of "chicken wire". So that was exactly what I wanted, but did not know the name, so now you know too.

Hardware cloth

It is pretty sharp and can cut you very nicely if you are not careful. I did not use gloves because I had to bend the ends and the gloves prevented that, so it took quite a while to do it without bleeding to chicken death.
Here it is already cut and tried for a good fit. I like how it seems a natural fit to the rest of the stage. The idea is that not only supports the materials and "stuff" that I will put there, but it also acts as a grid with all the connotations that the word has in terms of media. Once I project from behind (over a translucent substrate) it will look, or so I hope, as a kind of coarsely pixilated environment, very apropos with the theme.

12-06-2008, 09:33 PM
This is extremely interesting mate, very very inspirational! Could you post a video of the mannequin in action?
Edit: you mentioned the Make Controller had only 4 servo ports, you definitly shoudlve gone with the Axon MCU by John and Societyofrobots.com it has 55 I/O ports and is $130, while I think the two Make MCUs were $80 each?

12-06-2008, 11:05 PM
Thanks No0bert, this is what I like so much about this forum, where people actually share their knowledge and information with others. It reminds me of the early computer clubs where everyone would show their latest invention. Of course all that has changed and lawyers make sure you don't spill any beans if you have a potentially profitable idea!

I will post a video for sure, but I want to go through my entire process to get there because I learned quite a few things along the way and I think it might help others. I will look into the controllers you mention. I have controllers that can handle many servos but one of the reasons I chose MAKE is that it works seamlessly with FlashDevelop (http://www.flashdevelop.org/community/) which was our development environment (and if people don't yet use it I can't recommend it enough). You will see later on when I post the program which of course its not and never will be finished! but as I think I said in an earlier post we will release it as open source through CC (Creative Commons) so that everyone can use it. But I am not there yet...few more posts down the line:-)

12-07-2008, 02:33 PM

This is a little digression which however might clarify the weirdness of it all :tongue:

I tell all this (and there is so much more to tell) because my childhood was, among everything else I know and care about, the root and the source of my current work. In that privileged environment where the fruit of our labor and our play (is there any difference?) both fed us and entertained us, I learned to make my first puppets and my first theater. My audience (beside my grandmother) were the three or four kids that survived in the vicinity of our house.

Nowadays it has become fashionable to "go green", like trying to patch a dam when it is about to burst. We have forgotten how, not too long ago, recycling was for many simply a way of life. But perhaps that is just because some of us grew up in a very different circumstance. People today would say that we lived in misery or poverty, therefore we were forced to straighten a bent rusted nail instead of simply buying a new one.

For me that was what made my life extraordinary. My grandmother, who had taken a "poverty vow" (go explain that to someone living in a consumer society!) lived a life where everything around her had a purpose and had to be taken care of. Since I grew up with her it was natural for me to learn and understand how to create with what the world around me, particularly our shelter, provided. To what others was a miserable shack, to me was a magnificent castle, full of adventure and stories, constantly morphing through the tension between entropy and our creative survival.

12-07-2008, 10:09 PM
Re Cycle Beneath the Wheel

The Creator stage in progress

People, myself included, talk about reducing our "carbon footprint", a term about to disappear from overexposure the same way any word repeated over and over loses its meaning as it is replaced by noise. At the same time we blog about it, consuming terawatts of carbon producing power, exacerbated by the inefficiency of bloated operating systems, the mind blowing amount of energy Google needs, by the energy consumed by the avatars in virtual worlds that need to be kept virtually alive by the server nodes and the computers around the world that humm incessantly without any hope of ever being silent.

Our self-made society has plugged a cannula into our collective vein, to transfuse itself into the mind of the machine, which like a life support system we cannot disconnect, or at least we think we can't, that is, until the plug is pulled by the system itself when no longer needs any more information, other than the one it itself generates.

WW II direct interhuman syringe.

That is the story of The Creator with which we are trying to portray, using puppetry as a transference medium, the stark reminder of our self-dependency. Who or what controls who or what?

On to the practicum.

Since I detest plastic (not withstanding its "usefulness") and the many woes and sickness that have come as a result of its indiscriminate use, I try to target it as a primary candidate for recycling, which is just a palliative approach to the real problem. In any case, I took the vacuformed package of the "energy saving" bulbs that we use nowadays, and just as I was about to send it into its recyling path, I noticed a form which reminded me of ancient alchemical apparatuses which I needed to create for the robotic-marionette stage. So here is the process:


The background, built over the "hardware cloth" is semi-translucent, with the most translucent parts being the "genetic" containers which will transmute common information into a panacea.

A digital projector, fed with data generated imagery will fill up the vessels as well as create the fluid digital environment where the Creator lives.


Here is the very first test as seen from behind, where I am trying unsuccessfully to line up, scale and correct the angle distortion. As you can see I missed the mark by a few centimeters. I am now in the process of positioning the elements as close as I can to the target, given the limitations of inexact everything.

12-08-2008, 06:30 PM
The Night Mare

Plasticine, plaster gauze and Papier-mch is the skin of its face

The credit for the Creator's flesh and blood must go to Lorena, who with her hands brings to life the creatures of our nightmares, the future that we have already created even if we ourselves are not quite there yet to suffer the consequences, but getting very close...

Hmmm, definitely some resemblance...

The Creator gets its make-up treatment. It was eerie to see how it resembled the original Creator's head, that you can see in the background. It was like seeing a clone emerge from another dimension. And God said, "Go ahead and multiply!"

The Cyclopean Eye

The tiniest springs I could find in my "this could be useful" parts bin

The eyes are an important means of expression of the "soul" of a person, animal or character. The only external organ that connects directly to the brain through the shortest possible path.
The eyes of the creator move nervously as he scans its surroundings. Its world. They are animated by a simple spring mechanism that responds to the smallest movement. I did not know it was going to be so challenging to do this, except that, as we have found out, scale matters. And the smaller the scale the more critical the elements become.


The springs proved to be too weak to support the weight of the eye,
so I added a coat of latex to strengthen and stiffen it a bit,
but it proved to be a little too much so I ended up burning the latex away.
The residue that it left was just the right amount.

Lorena irritating the eyes

Of course the real "eye" of the Creator is up in the sky, or rather the ceiling in this case. This cyclopean eye that sees and controls everything is more and more a pervasive and accepted fact of life, in our streets, our buildings and even our homes, not to mention the "real" and inaccessible eye in the sky who observes and analyzes our every move, both as a species as well as individually when the need arises.
A very wide angle camera (76 degrees) was the choice after a good deal of research, and it proved to be the right one. I like that it is flat and although the base actually sucks (if you are going to use it on a desk or a monitor) it is perfect to tape to the roof and adjust the angle precisely. I needed a wide angle because the ceilings are usually not very tall and the camera must cover at least a 10x10 mt performing area, the world of the Creator, its Tartarus.

HP 2 megapixel camera, 76 degrees FOV

Related to the eyes or rather vision system is the light. Not the infrared light that the camera detects and processes but the actual "stage lighting". For quite a while I thought I was going to have to finagle a small lighting system form the theatre department at my university but since that was not very promising I started looking around for solutions.

I must say I am very happy with the outcome. I bought some inexpensive 12 LED flashlights at the local supermarket and hacked them (literally) so as to keep only the front portion. The advantage, in this case, of a cheap Chinese product is that it was very easy to solder to it, unlike more sophisticated models. The beam is extremely bright and is easily controlled by the microcontroller. So I have 4 "spots" with different color lights (a color filter on the LED's) to dramatize the scene and automatically create different moods according to the moment.


I got the plastic flexible contraption to where I attached the light base from one of my many "possible useful things" boxes. I wish I knew what it is or where it came from, maybe someone does becasue it is extremely practical for this and I am sure, other, applications. I suspect that it might belong to some fluid dispensing system.

The following were the very first tests with my new lighting system. I love it! absolute and precise positional and intensity control.


12-09-2008, 09:32 PM
A Tale of Two Masters

First napkin sketch of the user interface

The intermediate stage of the motion control system closely followed
the original concept although of course it has been greatly improved, mostly by simplifying.

As some recent studies of the brain suggest, there is not one but two control centers of control and command which work independently and actually incommunicated from each other even though both work towards a common goal.

The software that Philip Forget and myself have designed works as one of those control centers. The other stems from the independent actions of the performer which by interacting with the environment under surveillance by the Creator produces a synergistic feedback loop.

I am very excited by the software part of the project. It is in my biased opinion the best tool that I have seen for robotic puppetry control (not that there are many, mostly in research projects like mine). Although at first some closed systems were briefly considered, like National Instrument's LabView, to control servos based on image processing and other sensor input, the price and steep learning and development curve ruled it out early on, not to mention my dislike of anything that is not open source.

It was Philip who suggested that we use Flash as the controlling software to drive the MAKE microcontroller. I was right off skeptical. Not having used Flash in a couple of years I was not aware of recent developments.


He was of course referring to FlashDevelop, which is an open source ActionScript 2/3 and web development environment which integrates seamlessly with the Adobe product. This has allowed us to implement exactly what we wanted in a record time and in a very efficient and elegant way.


Our system is essentially a motion-sound-image-video-light sequencer/controller, driven in this case by image processing, but could as well be driven by any other sort of input. The MAKE microcontrollers are directly controlled by the software right out of the box.


The screen captures above show the position editor on the left and the corresponding key frame editor on the right. Each position is a collection of expressive movements achieved by "training" the marionette and recording the position, travel and speed of a group of eight servos. The sliders on the top right control or feedback both position and speed for each individual servo. The placeholder marionette images are automatic captures of their position that take the place of thumbnails identifying a sequence.


When played, the timeline runs thorough all the movements, light states and sounds that correspond to such moment of expression. The modules in the left of the screen are the library module and the main GRID which is the area under surveillance by the camera and which triggers actions defined by both timing and position of the performer or other beaming object, or external sensors (the performer wears an infrared beacon).

It is our intention to release the software as an open source project under Creative Commons.

12-09-2008, 11:11 PM
I honestly don't know what to say but "wow".

12-10-2008, 02:49 PM
1. This is amazing and will certainly go in the history books... for sure

2. What is the human integration, I know how you will use the person's movements to base the movements of the puppet, but you mentioned the brain. I have taken AP Psychology so I am really eager on what you have to say about that.

3. It might have been you, or this "Phillip" character, but whoever is should definitely be a computer programmer at least as a side job, what compiler do you use for Flash, FlashDevelop, is that the compiler, or is that what you integrated into the code?? Obviously I am a programming n0ob.
4. If it was you who programmed it, you should definitely assist the people in the software & Programming section, you would be a vital source.

12-11-2008, 08:24 PM
Hi No0bert,

1- I don't think it is history book material, LOL! but thanks for the kudos anyway...
About the brain I think you are referring to "...there is not one but two control centers of control and command... ". If you are interested in that subject you can check: *Washington University School of Medicine (2007, June 21). Brain's Voluntary Chain-of-command Ruled By Not One But Two Captains. ScienceDaily.*.

2- I only mentioned that as a metaphor for the way the marionette and the performer form a feedback loop. This is purely experimental since it is very hard if not impossible to determine who drives who. Normally in theatre that involves robots or puppetry and human performers all parties have pretty much predetermined tasks. Of course there is improvisation in the case of human controlled puppets and performers.

In this case the role of the human performer (who BTW in this scenario is supposed to be the "creature" or creation of the Creator) needs to respond to the Creator's actions, with its body language and by moving to the different areas of the stage which have special symbolic meanings and act as a trigger "weighted" by other factors. So the actor is not constrained by a script of actions, but rather tries to interpret the body language of the Creator (there are a variety of sounds that the creator emits associated with different "states of mind") and by doing so prompts the Creator to react as well. This is what I mean by the "feedback loop" which is the basis of control.

3- The program was developed entirely in FlashDevelop (http://www.flashdevelop.org/community/), which happens to integrate seamlessly with Adobe Flash but it is a self-contained ActionScript 2/3 programming environment.
I designed the functionality, the UI and the architecture of the program. Philip did the actual coding. We have worked together in some complex interactive projects. He was an architecture major about to graduate until he started working on this project. A friend of mine in NY saw this program and offered him a job he could not refuse!LOL! so he moved from little Gainesville to the Big Apple:-)

4-As far as I can see there are excellent coders with ample experience in this community. So I am the one who would need help in any case. One advantage I have is that I work in a research environment where I design my own projects and get to work with computer science graduates who intern with us. And then again, even if they are experts in C++, C# or whatever, about the only thing useful for them is the discipline and organization, since most of the things we do are new to them in terms of language, applications etc. A lot of fun.

p.s. If you are interested in coding for robotics and whatever you can imagine I would suggest to download the Open Source Flash Develop and give it a go. The community, as is this one, is extremely helpful and welcoming.

12-16-2008, 10:07 PM

Here the Creator is kneeling down since the high-torque spine servo is not powered up.

It was quite an ordeal to get to this point. In fact a couple of years to be exact. But finally the creature moves, and not only moves but seems to have a life of its own. I will post video after I digitize all the footage that I got from the performance. Philip and I crack up laughing every time it moves because it is so life-like.


But come to think of it, what is happening (philosophy purists will probably disagree) is that life in fact has been transferred and conserved as energy in a different form. This is not something new to me. I consider any work of our hands to partake of this mysterious force that we call life, otherwise how would it inspire us or elevate our spirit?

When I see a painting (in reality, not a reproduction) by say, Vermeer, the emotion I feel is a product not only of the formal beauty, but of being in the presence of time standing still, a few years of someone's life (which just happens to be a master) condensed and trapped like a genius in a bottle. I, the recipient of this gift, am the vehicle necessary for this alchemical process to occur. I release the energy contained in the matter of the painting.

The order of the particles, held by resinous mediums, crystallized by the passage of time which conform the external appearances of the scene, resonate and I believe connect me in the most direct way with the creator of the work of art, in the same manner that we are in intimate connection with the writer that is sharing her thoughts and life experiences with us.

This is, has to be, an energy transfer that could probably be calculated by an Einstein, but I prefer to simply shed a tear of joy when I find myself in front of the work of art. That is what art means to me.

12-26-2008, 09:59 PM
This is a video clip of the marionette in action. I feel it is anticlimactic because it is totally out of context (the performance) but it will have to do. I chose a short segment where the performer, while trying to escape the marionette's control, triggers some explosion sounds and drops to the floor. The Creator interprets this (according to the position of the performer and her timing on the stage) as something bad that has happened to his "creature".

Later in the performance the creature recovers and comes back "to the fold", in other words to the control of its master:-), therefore a happy ending of sorts within the apocalyptic atmosphere!



12-28-2008, 02:09 AM
That's tremendous. It's enough that you have a servo-driven puppet that's natural and emotive is one thing; the fact that it's autonomously reacting to another actor's performance is an entirely different kind of awesome. Very impressive.

12-28-2008, 08:13 AM
Color me impressed.

I *really* am enjoying the lengths you are going through to document this project with such incredible detail. Great read and great project!

12-28-2008, 06:42 PM
Thanks for the comments, and I am glad you are enjoying my streams of thought!:veryhappy:

I consider documentation as important as the project itself. Among other things, sometimes the documentation is the only thing that survives at the end of the day! No to mention the fact that you can go back and see what have you learned,( if you learned), from your mistakes and it helps others avoid obvious pitfalls...although it is fun (sometimes) to see what is at the bottom of the pit, heheh.

Happy Holidays to all

01-12-2009, 12:47 AM
I was reviewing this post and realized I had missed posting a picture of the "elevator" that takes the place of the puppeteer's arm going up and down. I probably blocked it out of my head because in fact it was the hardest problem to solve and I am still not happy with it by any means. The funny thing is that it was the hardest problem because it was so obvious that you take it for granted, since it is so natural for a human to do such movement without thinking.

I explained all that in post#16 but forgot to upload the pictures, so I just did. Did not want to repeat them here.

Oh, what the hell... here it is:


01-12-2009, 08:30 AM
Sunithaya, nothing says it better than WOW!!!
I read the whole thread just this morning, and once I started I couldn't stop. Great job on the project and the story. Again, WOW:)

02-16-2009, 10:50 PM
I got Dr. Roboto today, very happy, had it dancing in no time, ideas are flying around! Thank you Trossen Robotics!!!:veryhappy:


p.s. opening the package was a bitch (a fun bitch), never seen anything like it, I should have filmed it!

02-16-2009, 11:05 PM
Congratz! Well deserved!

05-10-2009, 09:38 AM
Just simply WOW!!! I just found this article this morning and could not take a break from reading all the posts.. Great Job and congrats on the win. I can't wait to see more from you...