PDA

View Full Version : Unity Robotic Controller



VR_Dev
02-14-2018, 05:06 PM
https://www.youtube.com/watch?v=qtR280F4g4k

My free time attempt to create a Unity Editor based Robotic controller, which runs simulations via Kerbal Space Program. Similar to the ROS - Gazebo relationship. KSP is a space and flight simulator, but has gazebo like potential as a robotic simulator. You build custom craft out of dozens of individual parts, which are then subjected to real time physics, as well as atmospheric and orbital calculations. The game also has engines for flight, and a pretty extensive weapons mod.

While I do enjoy the more video game esque features of Kerbal, I believe I can also use the controller to send input to Gazebo, which may be a fun addition to my project.

Inverse Kinematics

The IK code is all custom, but utilized a lot of cheats Unity gives me that aren't available in real time. Vector3.distance for example easily gives me the distance between two points. It all works though, and I plan to expand it out to more joints. The neck arm also allows rotational target matching.

PID Controller Servos

Each servo has it's own tunable PID loop which uses the error between the actual and IK servo angles as its process controller. This output sets the speed of the servo on a frame-by-frame basis.

Gait Sequencing & Steering

Right now the only gait sequence that exists is walking at a set speed. Steering is the last thing I still need to do. I wrote some simple stride length adjustment code, which allows setting the desired stride length for both the right and left legs at the beginning of each stride. The actual steering is adjusted by a PID loop which decreases one side's stride length by a percentage of its default length. So my stride length is 2 meters, and the steering PID can shorten that by up to 5%.

Terrain Mapping & Active Balance

The hexapod now has full terrain mapping and navigation capabilities. Instead of using the vessel height to position the ground and foot targets, each foot target is now placed on the ground directly beneath it.

Each hip target is now set according to the ground position of it't foot. There are two ways of setting the hip targets. Right now the hip targets are set at the average height of all the foot ground positions plus the target height. I've realized since recording the video it would be best to just set the hip targets so that the vessel is always the target height above the ground. Or some combination of the two. The first method helps going up hill, while the second is preferable for down hill. Also setting the hip height half way between the highest (foot ground + target height) and the lowest would be the way to insure each foot has the best chance of hitting it's target when going over extremely rugged terrain.

The vessel also knows if it is traversing up a hill, is level, or traversing down a hill, and sets the color of the ground accordingly. Right now nothing changes according to that, but eventually the gait will respond to the different slopes. I tried taking it out to the mountains but I still need to find a way to orient the gyroscope to a level surface, even when the launch point is not level. The triangle begin drawn on the ground represents the leg group that is actually moving the hexapod forward.

Memory Mapped Files

If you're interested in how I'm controlling KSP from Unity, I go into detail in my ksp thread. We have already begun discussing a real life counterpart as well.
https://forum.kerbalspaceprogram.com/index.php?/topic/170110-wip-unity-controlled-robotic-plugins/


https://www.youtube.com/watch?v=N7ZHFMBv7-Q


https://www.youtube.com/watch?v=FeJs5k5g4Tg


I also have plans to implement VR control, like I have here for a Iron Man demo.

https://gfycat.com/gifs/detail/OpenPlushAngelfish

Which will be used to control my bi-pedal mech's arms.

Thanks for checking it out. Would be happy to answer any question you may have.

VR_Dev
02-19-2018, 04:15 PM
https://www.youtube.com/watch?v=qtR280F4g4k

My gf was gone so I worked on this pretty much non stop over the weekend. (Also why the recording actually has sound). The hexapod now has full terrain mapping and navigation capabilities. Instead of using the vessel height to position the ground and foot targets, each foot target is now placed on the ground directly beneath it.


Each hip target is now set according to the ground position of it't foot. There are two ways of setting the hip targets. Right now the hip targets are set at the average height of all the foot ground positions plus the target height. I've realized since recording the video it would be best to just set the hip targets so that the vessel is always the target height above the ground. Or some combination of the two. The first method helps going up hill, while the second is preferable for down hill. Also setting the hip height half way between the highest (foot ground + target height) and the lowest would be the way to insure each foot has the best chance of hitting it's target when going over extremely rugged terrain.

The vessel also knows if it is traversing up a hill, is level, or traversing down a hill, and sets the color of the ground accordingly. Right now nothing changes according to that, but eventually the gait will respond to the different slopes. I tried taking it out to the mountains but I still need to find a way to orient the gyroscope to a level surface, even when the launch point is not level. The triangle begin drawn on the ground represents the leg group that is actually moving the hexapod forward.

Building the terrain mapping box was pretty difficult, as there are 36 vertices being set each frame. It took a while but it was definitely worth it.

Developing this process I found a couple of big mistakes in the gate code, which have now been fixed. The stride is now much smoother and more efficient than it used to be. I haven't done a top speed test yet after the fixes, but average speeds have picked up across the board. Still lots of tuning to be done.

Next up is steering.

VR_Dev
06-26-2018, 01:33 PM
Man you guys are hard to impress. I have completely reworked the memory bridge to make it 100 times faster. But as part of the rework I am building the robotics from scratch.

The previous videos show the custom code that is written specifically for the hexapod. My goal now is autorig any design with IK. Right now I can generate a skeleton for any design. I can autogroup the servos based on the pivot axis. Servos in the same group are the same color.

The last step before IK is normalizing all the rotations. If a servo is 180 degrees off the others in the group, that needs to be accounted for. That is the last step in the groundwork, than I can try to auto generate the IK.

7240

Also I've built a knuckle walker that will also feature bipedal locomotion, as well as short/med range VTOl.

7241
7242

Edit: Some other videos you may enjoy. The flight controller has already been added into the mech platform.


https://youtu.be/HEbhV4tjA8E?t=3

https://www.youtube.com/watch?v=eawcsv3eOZM&t=1s

https://youtu.be/dq5RnllvrPQ?t=19

tician
06-26-2018, 08:04 PM
I expect part of the lack of responses is due to the probable answer to the question: "Can any of this actually be built with existing technology on a reasonable budget?"

If one wants to watch or play with an improbable robot of any desired quality level (mechanical complexity, visual effects, physics accuracy, etc.), there are several decades of movies, shows, and games to choose from. If it is what you want to spend your time on, go for it. Just have reasonable expectations about others' enthusiasm about something that probably cannot be built and does not really hold much entertainment value beyond a few views.


Note: the above is not meant to be mean spirited. Me being mean spirited would be asking how anyone's Alpha 2 (http://forums.trossenrobotics.com/showthread.php?7695-Indiegogo-Alpha-2) is working out, or when we might be able to order our hydraulic exoskeletons (http://forums.trossenrobotics.com/showthread.php?9251-Hydraulic-Exoskeleton-controlled-by-bioelectrics-Questons-and-Answers). Current state of 'temporary layoff' that might become permanent has left me very irritable.

jwatte
06-26-2018, 11:35 PM
People interested in game development typically hang out in engine-specific forums, and people interested in Kerbal mods typically hang out in the Kerbal forum :-)
Also, I think most of us have built software simulations of our robots, and a ROS install comes with Gazebo which is a tool that does exactly that, so ... good for you for climbing the learning curve? (Why would you use a memory mapped file when you can just open a UDP socket?)

Simulating something in a game engine is one thing; actually putting things in physical material with weight, power, and cost limitations, is totally different. This site is more about actual, physical, hardware.
There's some cool hex hardware built, at various price points:
https://www.youtube.com/watch?v=1sRlFQLwg3w -- real life hex! hydraulic. Slow, and expensive.
https://www.youtube.com/watch?v=yn3FWb-vQQ4 -- MX64 size, also rolls, reasonably cheap at probably < $10,000.

VR_Dev
06-27-2018, 05:53 PM
Well I truly meant that as a lighthearted joke, apologies if it came across any other way. I understand this forum is more about physical projects than simulated, and I'd be happy to remove it if it's too out of place.

While I understand the criticisms, I just wanted to point out that I am only using kerbal space program as a gazebo-esque simulator for my Unity based robotic controller concept. KSP is simply a stand in for what could someday (when I'm rich) be a controller for actual machines. The IK and gate sequencing are no different than what you may find in the phoenix code. But instead of applying it to an actual servo, I send the command to a kerbal servo.


If one wants to watch or play with an improbable robot of any desired quality level (mechanical complexity, visual effects, physics accuracy, etc.), there are several decades of movies, shows, and games to choose from. If it is what you want to spend your time on, go for it. Just have reasonable expectations about others' enthusiasm about something that probably cannot be built and does not really hold much entertainment value beyond a few views.

This I think is the main point of misunderstanding. My robots are not at all like the ones in video games. Mechs in video games are moved around by the body, and the legs just follow animations to make it look like they are doing the work. My robots are actually supported and moved by the legs via ground friction, and balance has to be maintained. If the gait is inefficient, it wont be going anywhere.

KSP is a game that gives you a variety of parts, each with their own weight, joint strength, torque, collision strength, electrical draw, and many more. When you build a craft, physics are applied to all the parts, including aerodynamic. When a leg moves, it has kinetic energy that needs to be accounted for. My servos each have there own PID loop that is responsible for slowing it down as it gets closer to its target. If the foot hits the ground with too much force it will explode.


I expect part of the lack of responses is due to the probable answer to the question: "Can any of this actually be built with existing technology on a reasonable budget?"

I think yeah a scaled down version is totally possible. Obviously not the engines or weapons, but the design is no different than an IRL hexapod. The legs only have 3 DOF and are made of pretty basic servos.


People interested in game development typically hang out in engine-specific forums, and people interested in Kerbal mods typically hang out in the Kerbal forum :-)
Also, I think most of us have built software simulations of our robots, and a ROS install comes with Gazebo which is a tool that does exactly that(Why would you use a memory mapped file when you can just open a UDP socket??

Yeah I get that, but again what I'm trying to show off here is a robotic controller. I can't use unity to send commands to Gazebo, (actually I looked into it and gazebo is c++ so I think I can) and I thought maybe someone would be interested in a physics simulator that allows you to make flying craft, and has a pretty fun weapons mod.

The memory mapped files are much faster than going through the ip stack, if I were ever to try and use this to control a real networked robot, I'd have to switch to that.


so ... good for you for climbing the learning curve?

(middle finger emoji)


Simulating something in a game engine is one thing; actually putting things in physical material with weight, power, and cost limitations, is totally different.

Again I stress these things are accounted for, maybe not to the most realistic degree, but Unity runs on nvidias PhysX engine, which isn't too shabby.

So yeah, like I said I'd be happy to remove it, or move it to the software forum. I just figured people who like walking robots would think it's cool.

jwatte
06-27-2018, 11:30 PM
The memory mapped files are much faster than going through the ip stack

Not really, because you have to solve synchronization separately. There is no atomicity built-in, or even ordering, in the memory mapped file system. One side may write first at location A and then location B, and the other side will see an update to location B, followed by an update to location A. And if you "fix" that by forcing fdatasync() or similar all the time, then you end up being slower than UDP.
At least, that's been my experience :-)

Good luck on your project!
Btw: What's your plan for becoming rich enough to build a life-size hexapod? Want to share some secrets? :-D

VR_Dev
06-29-2018, 08:27 AM
Not really, because you have to solve synchronization separately. There is no atomicity built-in, or even ordering, in the memory mapped file system. One side may write first at location A and then location B, and the other side will see an update to location B, followed by an update to location A. And if you "fix" that by forcing fdatasync() or similar all the time, then you end up being slower than UDP.
At least, that's been my experience :-)

Good luck on your project!
Btw: What's your plan for becoming rich enough to build a life-size hexapod? Want to share some secrets? :-D

Not if you don't handle synchronization lol. Right now unity and ksp are just pinging the same file non stop, so all the values are based on the last frame. I don't really have anything that needs to wait for a response, mostly just reading/writing servo values. You are 100% correct though. The hexapod would probably run fine via ports, but the lag is too much when velocity gets too high. aka the F-22s.

I have no plans to make this life size, I just can't even afford little ones. I'm gonna move this over to the software thread if anyone still wants to follow the progress.

jwatte
06-29-2018, 02:31 PM
the lag is too much when velocity gets too high

Then you're doing something wrong! UDP sockets can do millions of round-trips per second, so "lag" would be less than a microsecond.

If you use TCP, and you don't turn on SO_NODELAY, you will see "lag" caused by Nagle's algorithm, though.

VR_Dev
07-06-2018, 10:55 AM
Then you're doing something wrong! UDP sockets can do millions of round-trips per second, so "lag" would be less than a microsecond.

If you use TCP, and you don't turn on SO_NODELAY, you will see "lag" caused by Nagle's algorithm, though.

I actually never tried. There is a mod called kRPC that I tried which allows you to communicate with the game via remote procedure calls. The lag in that is what started me down this path.

Maybe I'll give a try for myself. Like I said I would have to switch techniques to control real bots anyway. Thanks for the hot info jwatte.

tician
07-06-2018, 05:44 PM
Note that jwatte built his own system around ZMQ (mostly UDP) and RPCs (http://forums.trossenrobotics.com/showthread.php?6631-ROS-for-hobby-use&p=62059#post62059) because he couldn't figure out ROS (and also did not want to have to install Ubuntu just for installing ROS successfully). I had been working on a comprehensive motion control system for arbitrary robot designs built around ZMQ with stochastic gait seeking modules/nodes, but mostly gave up a couple years back. The somewhat higher performance of ZMQ was simply not necessary and the amount of work to get the ZMQ system up and running was in no way worth the effort compared to just using ROS for all the inter-node communications.

VR_Dev
10-04-2018, 11:36 AM
Note that jwatte built his own system around ZMQ (mostly UDP) and RPCs (http://forums.trossenrobotics.com/showthread.php?6631-ROS-for-hobby-use&p=62059#post62059) because he couldn't figure out ROS (and also did not want to have to install Ubuntu just for installing ROS successfully). I had been working on a comprehensive motion control system for arbitrary robot designs built around ZMQ with stochastic gait seeking modules/nodes, but mostly gave up a couple years back. The somewhat higher performance of ZMQ was simply not necessary and the amount of work to get the ZMQ system up and running was in no way worth the effort compared to just using ROS for all the inter-node communications.

Interesting. Also the exact same reason I haven't learned ROS yet. This project has been sidelined for a new one. Controlling and tracking my DJI/Ryze Tello in Unity. Eventually the plan is for autonomous flying around a course.

https://tellopilots.com/threads/unity-controller-for-tello-3d-tracking-tello-simulator-flight-stick-support.2288/

https://youtu.be/C8G9U7ioMlo

https://youtu.be/KrBYfsxCI60

KurtEck
10-04-2018, 05:00 PM
Note that jwatte built his own system around ZMQ (mostly UDP) and RPCs (http://forums.trossenrobotics.com/showthread.php?6631-ROS-for-hobby-use&p=62059#post62059) because he couldn't figure out ROS (and also did not want to have to install Ubuntu just for installing ROS successfully). I had been working on a comprehensive motion control system for arbitrary robot designs built around ZMQ with stochastic gait seeking modules/nodes, but mostly gave up a couple years back. The somewhat higher performance of ZMQ was simply not necessary and the amount of work to get the ZMQ system up and running was in no way worth the effort compared to just using ROS for all the inter-node communications.

But maybe now it might be time to learn more about ROS2... It uses more standard lower level communications (DDS) and they are setting it up to be able to be installed on Linux, OSX and Windows: https://github.com/ros2/ros2/wiki/Installation

I know that Robotis is working with ROS2 and has a version of the Turtlebot3 Burger that runs with it. They are also coming out with some new controllers to work with ROS 2: https://xelnetwork.readthedocs.io/en/latest/