Page 2 of 3 FirstFirst 123 LastLast
Results 11 to 20 of 30

Thread: Kinect Teleoperation of KHR1-HV using ROS

  1. #11
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    49

    Re: Kinect Teleoperation of KHR1-HV using ROS

    Great! Thanks to your brilliant code, I was able to very quickly tweak the joint names to control Pi Robot's virtual self in RViz as shown below. Next, to get it to work with the real robot...

    [ame="http://www.youtube.com/watch?v=MiPpGPFF6U8"]YouTube - Veltrop Kinect Teleop of Pi Robot in RViz[/ame]
    The Pi Robot Project
    http://www.pirobot.org

  2. Re: Kinect Teleoperation of KHR1-HV using ROS

    That's awesome! I'm so glad that the code was easily reusable for you!

    May I link to your video on my ROS contest entry page?

    Please let me know if you put anything else on the web about this so I can link to it!


    Next: movement of the legs on the robot. There's 3 methods I want to use interchangeably. I think that all three methods are needed.

    1. Direct. Maybe rarely needed, it doesn't make sense for most situations. Could be useful for experimenting with balancing on the robot, or maybe a karate kick

    2. Gesture based. IE, walk in place to trigger the robots walk forward routine. Or maybe in your case it could rotate the robots wheels. I'm going to first attack this by hacking up trackpad gesture drivers, but I'm not really sure what to do. Any other ideas or libraries?

    3. Goal based. Take the XYZ position and orientation of the human's torso, and set that (with some scaling) as a goal position for the robot using the ROS navigation stack. This has the limitation that you may walk off camera, but worry about that later. I don't have a navigation stack on Veltrobot yet. Would you be interested in working on this?

  3. #13
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    49

    Re: Kinect Teleoperation of KHR1-HV using ROS

    Hi Taylor,

    First of all, I'd be honored to be linked into your ROS contest page--hopefully it will add to your points in the "Most Useful" category! I just now made a link to your contest entry page and the Pi Robot Youtube video on my home page at http://www.pirobot.org.

    And yes, I am keen to work on the gesture stuff as well. If you can get your humanoid to do a karate kick, I see a whole new future in human-robot gladiator competitions where the world champion is a 5 year-old controlling a 10 foot 5000 lb Transformer from his or her living room.

    Regarding using leg gestures: perhaps this is what you meant by trackpad gestures, but I wonder if you could use something like this: step forward (one step) to go forward. Step back to a straight up position to stop. Step backward to go backward. Step right to go right and so on. Also, the NITE API document describes using hand gestures: Click, Wave, Sweep Left, Sweep Right, Raise Hand Candidate, Hand Candidate Moved. I have no idea how to extract these yet.

    I also like the Goal Based idea, especially when it comes to directing the robot to go somewhere rather than having it mimic your behavior. How about a simple pointing scheme where you use the pointing arm as a vector and extrapolate it to the ground plane to indicate the location you want the robot to go? Then use the ROS navigation stack to actually get the robot to that location.

    --patrick
    The Pi Robot Project
    http://www.pirobot.org

  4. Re: Kinect Teleoperation of KHR1-HV using ROS

    Regarding using leg gestures: perhaps this is what you meant by trackpad gestures, but I wonder if you could use something like this: step forward (one step) to go forward. Step back to a straight up position to stop. Step backward to go backward. Step right to go right and so on.
    Hmm, that's a good compromise. Also then I can use simple offset positioning of the operator instead of detecting complex movement.

    Also, the NITE API document describes using hand gestures: Click, Wave, Sweep Left, Sweep Right, Raise Hand Candidate, Hand Candidate Moved. I have no idea how to extract these yet.
    I wish the documentation wasn't so cryptic.
    We could probably leverage this but I don't know about using it with openni or not:
    http://singularityhub.com/2010/12/10...terface-video/


    I also like the Goal Based idea, especially when it comes to directing the robot to go somewhere rather than having it mimic your behavior. How about a simple pointing scheme where you use the pointing arm as a vector and extrapolate it to the ground plane to indicate the location you want the robot to go? Then use the ROS navigation stack to actually get the robot to that location.
    That is an -awesome- idea. It's usability may be limited to situations where the robot is in the same room as the operator, but that's not a problem. That idea is more of a human-robot interaction than a teleoperation.

    Which makes it the perfect thing to implement with my new NAO!
    Starting the NAO ROS cross compile now.

    For the pointing idea we need to match the robot's position relative to the operator's. On Pi Robot's wheeled base you can probably just use a starting point plus odometry, not sure about the NAO though.
    Last edited by veltrop; 01-02-2011 at 12:12 PM. Reason: Oops, I misread something!

  5. Re: Kinect Teleoperation of KHR1-HV using ROS

    There's a fighting robot competition here in Japan on the 9th. Entry is by the 5th. I'm thinking of using my Kinect teleoperation for it.

    Patrick do you think you could get your gesture teleoperation idea working for moving fwd/back/etc by the 5th?

  6. #16
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    49

    Re: Kinect Teleoperation of KHR1-HV using ROS

    Quote Originally Posted by veltrop View Post
    There's a fighting robot competition here in Japan on the 9th. Entry is by the 5th. I'm thinking of using my Kinect teleoperation for it.

    Patrick do you think you could get your gesture teleoperation idea working for moving fwd/back/etc by the 5th?
    I'll definitely be working on it as much as I can--unfortunately I'm back to my day job tomorrow so robot time will be greatly reduced, but I'll post my results back here as I make progress.

    --patrick
    The Pi Robot Project
    http://www.pirobot.org

  7. #17
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    49

    Re: Kinect Teleoperation of KHR1-HV using ROS

    OK, I'm already making some progress on gesture based forward/back/etc movement. I should have something to show by the end of the tomorrow.

    --patrick
    The Pi Robot Project
    http://www.pirobot.org

  8. #18
    Join Date
    Apr 2009
    Location
    Stanford, CA USA
    Posts
    590
    Rep Power
    49

    Re: Kinect Teleoperation of KHR1-HV using ROS

    OK, here is a short demo of basic "navigation" of Pi Robot using Taylor's teleoperation code but modified to send ROS Twist commands to the Serializer PID controller based on the position of the operator's feet. (I had to use a wired USB connection between the robot and my desktop since for some reason my XBee connection to the Serializer has a huge delay no matter what baud rate or channel I use...)

    [ame="http://www.youtube.com/watch?v=F5c2IshtpOw"]YouTube[/ame]

    I've attached my modified version of Taylor's teleop_kinect.cpp file. The basic premise is to compute the vector pointing from the right foot to the left foot, then use the length of the vector for speed (Twist linear x component) and the relative angle of the vector (computed using atan2) to determine the angular velocity (Twist angular z component). At the moment, I am using just four motion commands (five if you include "stop"). If the right foot is in the forward upper half quadrant, go forward; if it is in the forward lower half quadrant rotate left; if it is in the rearward upper half quadrant rotate right; and if it is in the rearward lower quadrant, go backward. I've also put a top speed on the forward/backward movement of 0.2 m/s for safe testing. And since Pi is rather heavy and hard to turn on carpet, I have fixed the rotation speed at 0.4 rad/s.

    Oh, and if both feet are more or less together, the robot stops.

    This is just a start of course, but it seems to work fairly well.

    --patrick
    The Pi Robot Project
    http://www.pirobot.org

  9. Re: Kinect Teleoperation of KHR1-HV using ROS

    Thanks for the awesome work!

    I'll need to setup veltrobot's movement_controller to process twist messages and translate them into the walking and turning motion routines. It'll be limited because as they are now it can only walk or turn at a single speed, and not simultaneously. Veltrobot's motion system needs improvement...

    But when I get this working on the NAO it'll be no problem. It has an omniwalk function that essentially takes a vector as input.

    But in the meantime I am also giving 1:1 leg control a shot!

    I'll report back when I can

  10. Re: Kinect Teleoperation of KHR1-HV using ROS

    Looking at this other (much more expensive) teleoperation system, I think gestures are a perfectly valid solution:

    http://www.youtube.com/watch?v=TJmQqC1nHTU
    http://www.youtube.com/watch?v=5N93QtVsyv8

    ...Scrub to 1:00 of the first vid, see how the arms have almost zero lag but the legs lag massively? Not sure what they're doing, but thet're definitely processing the crap out of leg data.

    The second vid shows they certainly aren't mimicking the legs, just approximating them. If you could splice up the walking motion routines you could probably make gestures for walk_fwd_left_leg, walk_fwd_right_leg, turn_left and turn_right. walk_fwd gesture could just be lift/bend your leg, and turn gestures could be straight leg lift to the side, kinect should be able to pick those motions up cleanly. (although lift/bend might also need to be to the side so kinect gets a profile view of your bent knee... you'd probably look like you're warming up for sumo while you control veltrobot )
    Last edited by RefugeZero; 01-04-2011 at 01:00 AM.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. The Ultimate Kinect thread
    By RobotAtlas in forum Sensors
    Replies: 51
    Last Post: 01-23-2011, 03:24 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •