Page 1 of 5 12345 LastLast
Results 1 to 10 of 48

Thread: Robotic Eye-Hand Coordination

  1. #1
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    47

    Robotic Eye-Hand Coordination

    Hello,

    I just finished up some work on using RoboRealm to guide my robot as it reaches toward a target object. The ultimate goal is for the robot to be able to pick up the object from a random location or take it from someone's hands. For now, I simply wanted to work out the coordinate transformations from visual space to arm space to get the two hands to point in the right direction as the target is moved about. The following video shows the results so far:



    I don't have a full write-up yet on how I did this but it basically just uses 3-d coordinate transformations from the head angles and distance to the target (as measured by sonar and IR sensors mounted near the camera lens) to a frame of reference attached to each shoulder joint. The Dynamixel AX-12 servos are nice for this application since they can be queried for their current position info. The distance to the balloon as measured by the sonar and IR sensors is a little hit and miss and I think I'd get better performance using stereo vision instead.

    --patrick

    http://www.pirobot.org

  2. #2

    Re: Robotic Eye-Hand Coordination

    Wow, very good, I have been playing with Roborealm 3d positioning, but no where near as good as that. Kind of spooky to watch, you think it might cry if you don't give it that balloon soon
    .

    www.ukrobotgroup.com

    "Welcome to the Robolution"

  3. #3
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    47

    Re: Robotic Eye-Hand Coordination

    Thanks Orac! Right now I am using RoboRealm only to get the x-y coordinates of the balloon in the camera frame of reference. I then use a sonar and IR sensor mounted near the lens to get the z-coordinate or distance to the balloon. One could also use the width of the balloon's blob in RoboRealm if you assume it has a fixed size, but I wanted to allow for different sized objects. When STeven and company get their Stereo module completed, I think that will be a good way to go, but for now this seems to get the job done.

    And yes, the next step is to actually give the balloon to the robot before it comes after me flailing those arms. The AX-12's can pack quite a punch.

    --patrick
    Last edited by Pi Robot; 03-03-2010 at 12:14 PM.

  4. #4
    Join Date
    Mar 2010
    Location
    Ontario, Canada
    Posts
    105
    Rep Power
    32

    Re: Robotic Eye-Hand Coordination

    Wow, I am looking forward to see you stereo vision logic working. I guess then two hands and the ball are the 3 visual objects in bot's brain to control the it grabbing the ball. Am I right about your plan?
    This is what humand do, isn't it? We don't calculate the xyz of our hand to grab a cup :-D
    CAD Tools: Solidworks 2011. Altium Designer 10
    Micro Controller: ARM7/9, AVR 8bit/16bit, PIC, Nordic nRF, HOLTEK, 51s, NEC
    RF: 2.4G, 433M
    IDE: KEIL MDK3.5, AVR Studio 4, MP LAB, HOLTEK HT-IDE3000, Elipse
    High End: VC, VB, Unix/Linux C/C++, J2ME (For cell phone game), SQL
    Education: EE Master.
    Hobby: RC Helicopter.

  5. #5
    Join Date
    Mar 2010
    Location
    Ontario, Canada
    Posts
    105
    Rep Power
    32

    Re: Robotic Eye-Hand Coordination

    $89 for this software is really good. The steven guy you mentioned is from Roborealm?
    If $89 can buy a stereo vision SDK, that will be a steal :-)
    Thanks
    CAD Tools: Solidworks 2011. Altium Designer 10
    Micro Controller: ARM7/9, AVR 8bit/16bit, PIC, Nordic nRF, HOLTEK, 51s, NEC
    RF: 2.4G, 433M
    IDE: KEIL MDK3.5, AVR Studio 4, MP LAB, HOLTEK HT-IDE3000, Elipse
    High End: VC, VB, Unix/Linux C/C++, J2ME (For cell phone game), SQL
    Education: EE Master.
    Hobby: RC Helicopter.

  6. #6
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    47

    Re: Robotic Eye-Hand Coordination

    Actually, I imagine that the human brain (as well as other animal brains) does do some form of calculation without having to look at the hands. After all, you can reach for a cup with your eyes closed (though you might be slightly off target). Also, whenever you begin to reach for an object, your hands are not necessarily in view until they are part way to the goal. So you have to get the hands moving in the right direction before you see them.

    Having said that, yes, at some point I was thinking of putting some small fiducials on each hand and these could be used to visually guide the hands to the target.

    --patrick

  7. #7
    Join Date
    Mar 2010
    Location
    Ontario, Canada
    Posts
    105
    Rep Power
    32

    Re: Robotic Eye-Hand Coordination

    Agreed. rough calculation can be done to guide the hand to the area, then use vision system to do the "pick up".
    The non linearity of the lenses, servo postion errors, mechanical errors, etc... all add up.
    Calibration will solve just small part of it.

    The "brain calculation" you mentioned is not actual "calculation", but "imagination". With eyes closed, brain still uses the "vision system" to guide your action by the image created by "imagination".
    "Simulation" maybe another word.
    Last edited by bonmot; 03-09-2010 at 10:13 PM.
    CAD Tools: Solidworks 2011. Altium Designer 10
    Micro Controller: ARM7/9, AVR 8bit/16bit, PIC, Nordic nRF, HOLTEK, 51s, NEC
    RF: 2.4G, 433M
    IDE: KEIL MDK3.5, AVR Studio 4, MP LAB, HOLTEK HT-IDE3000, Elipse
    High End: VC, VB, Unix/Linux C/C++, J2ME (For cell phone game), SQL
    Education: EE Master.
    Hobby: RC Helicopter.

  8. #8
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    47

    Re: Robotic Eye-Hand Coordination

    Yes, "simulation" seems like a better word than "calculation". Or perhaps simply "neural computation". As you probably know, much work has been done on this kind of thing in the fields of psychology and neurophysiology. I found this article (abstract) particularly interesting:

    http://www.ncbi.nlm.nih.gov/pubmed/1...ubmed_RVDocSum

  9. #9
    Join Date
    Aug 2008
    Location
    Iowa City, IA
    Posts
    390
    Rep Power
    44

    Re: Robotic Eye-Hand Coordination

    Cool article. Thanks.

  10. #10
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    47

    Re: Robotic Eye-Hand Coordination

    Try Googling something like "head shoulder transformation reaching" and you'll find a bunch more interesting stuff.

    --patrick

    http://www.pirobot.org

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Sir Kitt robotic TV presenter
    By Orac in forum Project Showcase
    Replies: 19
    Last Post: 10-02-2009, 02:13 AM
  2. Contest Entry Robotic Marionette
    By sunithaya in forum Project Showcase
    Replies: 35
    Last Post: 05-10-2009, 09:38 AM
  3. Project robotic arm
    By TCtech in forum Robotics General Discussion
    Replies: 7
    Last Post: 04-08-2009, 11:46 PM
  4. Robotic Fish
    By scorch in forum Robotics General Discussion
    Replies: 27
    Last Post: 07-29-2008, 03:32 PM
  5. Robotic Model Building
    By Rick Badman in forum Robotics General Discussion
    Replies: 2
    Last Post: 08-27-2007, 10:29 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •