PDA

View Full Version : High Frequency I/O for SBC running ROS



RoboTy
11-17-2010, 10:32 PM
Hi Everyone,

I'm looking to have high frequency input and output from/to a single board computer (or something such as a fit-PC2). I'm still trying to nail down the specific requirements I'll need to meet, but I'm thinking somewhere on the order of 1000 Hz real time control.

I was wondering what kind of suggestions some of the more experienced people here have. I'm considering EtherCAT since I'll be running ROS on the single board computer and EtherCAT is what the PR2 uses (the main robot that ROS is developed with). But the EtherCAT slave devices are rather expensive, and it looks like there are no off the shelf solutions for what I need (very small size).

What I/O board would you guys suggest. I've been looking at Roboard, but it would only be used for data acquisition and motor control. And it's also only capable of either the gyro or accel, but not both (I need both). For example, the accel can update up to 3200Hz, but I have no idea if I can transfer that much data + all the other sensors to an additional onboard SBC (I'll need something considerably more powerful than the Roboard).

I had another question. How can I figure out exactly what frequency I need to update my sensors and control my motors?

So in summary I have 3 questions.
1. How can I figure out what a certain protocal is capable of for sensor/motor frequency control
2. How can I figure out what frequency I need to update my sensors and control my motors
3. What are some of your recommendations for I/O boards.

Thanks for the help.
Tyler

darkback2
11-17-2010, 11:08 PM
what specifically are you trying to do? That is probably a better place to start.

Hope this helps.

DB

RoboTy
11-17-2010, 11:25 PM
I'm trying to build a robot that will be statically unstable such as an inverted pendulum. The update rate will need to be fast enough to keep it from falling over, and also to keep it from being "jittery".

I haven't decided what I'd want to do first, such as a balance bot, flying helicopter, etc. I just want to make sure what I'm learning and starting with is fully capable in the future. Which is why I'm thinking of targeting 1000Hz with many degrees of freedom (20 or more).

SK.
11-18-2010, 01:23 AM
I'm not sure it is trivially possible to do EtherCAT stuff yourself, because you need these slave devices, which you can only buy from a few select companies and I think are pretty expensive like you said.
Another option would be to use RTNET (http://www.rtnet.org/), so you could have a microcontroller-equipped interface board for collecting sensor data that then get sent via Ethernet to a FitPC2 or similar for processing.

Reliable 1kHz+ is hard realtime OS territory, so with Linux you´ll be looking at either RT-Preempt or a dual kernel approach like RTAI or Xenomai. I can recommend looking at orocos_toolchain_ros, this already works pretty well for getting realtime control in conjunction with ROS.

Be warned though, all this realtime stuff is used only by a relatively small number of people, so getting help will be much harder than when using "normal" OSes or microcontrollers.

tician
11-18-2010, 02:17 AM
Whenever I post, you must repeat to yourself...

http://forums.trossenrobotics.com/picture.php?albumid=87&pictureid=496


Hopefully there are some useful questions among these that might help narrow down your choices:
1) How many more sensors do you really need? At 20+ DOF, it would appear you are planning far more than just a simple inverted pendulum or autonomous heli(/quad/hex/...)copter as most autopilots I've seen only use 6 DOF for stability control and 9 DOF for AHRS.
2) Are these sensors Analog, Digital, or a mix? If digital, what interface (I2C, SPI, etc)?
3) What scale are we talking, AX-12+ inverted pendulum bot or full size human carrying Segway? You mention it must be small, but then talk about something considerably more powerful than a Roboard which takes you at least to Atom, if not iCore series processors (increases not only size/weight, but power consumption). Sticking an iCore mini-ITX board on a small-ish copter does not seem too feasible to me.


On a side note, I am wondering where you got the impression the Roboard could only handle either the gyro or the accelerometer/magnetometer? The two boards offered by Trossen use I2C for all three sensors (the gyro is analog, but the 8-CH ADC on board with it is I2C), so there is nothing stopping you from connecting them all together on the I2C bus (at a maximum 400kHz clock frequency). An even better choice for a single sensor board might be http://www.sparkfun.com/products/10121 (3-accel + 3-gyro on a board the size of a US quarter). Concerning the power of the Roboard, it is a 1GHz x86-compatible system with 256MB of DDR2 RAM that can run linux or windows xp/ce (so 1/1.1 to 1/2 the CPU clock and 1/4 to 1/8 the ram of a fit-PC2, but with onboard PWM, I2C, SPI, etc.).
In general though, if you do not think the Roboard is powerful enough to run ROS by itself, a smaller microcontroller based board with multiple I2C and/or SPI interfaces with each dedicated to a single sensor might be an option. There are numerous microcontrollers out there that can handle all of that plus ethernet and usb, but would likely be a custom build.
Sorry about the above, kinda cranky when sleep deprived.

RoboTy
11-18-2010, 11:25 AM
I haven't heard of RTnet. That will be worth looking into. I was already leaning towards using OROCOS for RT control since it's already implemented with ROS.

I want to re-iterate that I'm still trying to determine what sensors and controllers I will be using. Of course there will have to be several encoders, 3 axis accel, 3 axis gyro, maybe some sonar/IR, some sort of camera, motor control output, + more for current and voltage measurements, etc.

The point of this is to try to determine how I can pipe all of this to a more powerful SBC at high I/O without running out of inputs.

As far as size goes, it's definitely not going to be a micro copter or anything. I probably shouldn't have mentioned a flying robot because if it flies than like you said, I can't put all that heavy stuff on it. I was just spitballing general ideas I had.

About the roboard, I didn't realize you could connect mutliple I2C devices to a single I2C header? How do you do this, or is there more I don't understand. I think the comments about the roboard pretty much hit the nail on the head about what I'm trying to figure out. I should have mentioned my background is mechanical engineering, not electrical or software. For example, how can I figure out if it's possible to have the Accel and Gyro both sample at 1000 Hz via the I2C, and then how can I determine if the USB connection (or whatever connection to the SBC I chose) can handle the bandwidth (or what the bandwidth is).

So for example, I can go with 1 device such as the roboard, and try to plug everything into it, and then send all of that data to a SBC, or I could go with multiple smaller devices, which communicate through a bus system such as EtherCAT (or RTnet) which in the end can be routed together into a single Ethernet connection (I believe). Could I even get away with just connecting everything together via USB?

tician
11-19-2010, 01:57 AM
About I2C, the accel/mag board has two identically wired headers/connectors (both connect to a single I2C bus). The gyro board may or may not have two headers (tis a copy-paste of the accel/mag board's page), but it does not matter either way. You connect one of the headers on the accel/mag board to the sbc I2C header, and the gyro header gets connected to the second header of the accel/mag board (aka daisy-chaining). The sbc then uses the unique addresses of the three different sensors to communicate with only one sensor at a time.

RoboTy
11-19-2010, 11:39 AM
So I ended up spending all of last night just reading Wikipedia on the different communication protocols. I didn't realize that I2C was actually a bus (I didn't realize that most of the connectors are actually bus's).

So If I daisy chain connectors on an I2C bus, how can I tell how fast they are capable of updating? The spec sheet for the accel says it can do 3200 Hz. But how can I tell if it can communicate that 3200 Hz through the I2C? And then how could I tell of the Roboard (or any other microcontroller) can communicate that 3200 Hz back to a more powerful on board SBC?

darkback2
11-19-2010, 12:27 PM
I think your going about this the wrong way. I would start by finding the simplest way to do what I want, and then try to build up from there. Here is an example of a lego balance bot. Is this what you are trying to do? It uses the lego NXT and seams to work just fine. A search for lego segway turned this up.

I hope this helps.

DB

RoboTy
11-20-2010, 09:30 AM
I'd rather try to do something like this

YouTube - High-Speed Robot Hand

As opposed to something like this...

YouTube - Humanoid Robot Stupid Walk Test 2 [www.hkrobot.com]

I'd rather just start out with a very capable system that can be utilized until I try to do some really complicated stuff. Otherwise I'll waste my time and money on something such as the lego kit.

SK.
11-20-2010, 11:09 AM
I'd rather try to do something like this.
Well then you better start saving money, because the modified Barret WAM arm +custom high speed camera system are probably way above 200.000$. It's nice to have challenging goals, but I'd sure suggest to take some small steps instead of aiming high and achieving nothing.

RoboTy
11-20-2010, 02:27 PM
I'd like to re-frame this topic back to it's original purpose. A discussion of high speed I/O options and bus systems. Including high speed real time communication between a PC and various components. Not specific to a particular robot or purpose. Certainly it's possible to have this kind of discussion.

So far we've EtherCAT and RTnet in some detail. I've found several EtherCAT slave devices which can communicate with microcontrollers easily, and I'm sure there's plenty more.
http://www.beckhoff.com/english.asp?ethercat/fb1111_fb1122_fb1130.htm
http://www.gridconnect.com/unigateicec.html
The one with an actual price point (and not quote only) was around 190 dollars, certainly within the realm of possibilities.

But there are still things I don't understand about rtnet. The home page says "RTnet is an Open Soure hard real-time network protocol stack for Xenomai (http://www.xenomai.org/) and RTAI (http://www.rtai.org/) (real-time Linux extensions)" So does this mean that ROS (which runs on Ubuntu) is not capable of using RTnet? Or maybe it would just be much more difficult to implement since the PR2 uses EtherCAT already. (Searching around I didn't find any existing stacks for ROS using RTnet.) Rtnet might be nice if it can be implemented on anything with a LAN connection.

Other topics which I'm still trying to understand is if there are any less expensive/difficult options for doing RT communication at high speeds with a SBC that doesn't have a million inputs on it. For example, USB, or RS-485.

And lastly, all this high speed real time communication is pointless if the microcontroller, sensors, and motor drivers are not capable of using the high speed. I tried to expand on this with the discussion on I2C between the accels and Roboard. If there are 3 accels, and 1 compas on the same board all outputing 13bit data to I2C, at 1000 Hz that would be 13 + 13 + 13 + 13 bits per .001 seconds, or 52000 bits per second, or 52 kbps + overhead? (how could i figure out overhead? That doesn't seem correct to me btw, but I just don't know how to figure it out). This would seem like a 1 mbps I2C on the roboard (spec says 1 to ~3.3) would easily handle this communication. Is this correct? But then that same amount of data would need passed to the main SBC at 1000 Hz. There would certainly be some lag time in there. How could I estimate that?

It's possible to design the entire system before a single purchase is made. That is my goal. To create a complete robot design before a single purchase is made, and fully understand the system capabilities/ bottlenecks, limiting factors. In the end I plan to have a performance/cost tradeoff analysis for the different options and components I find, and use that to decide what route I want to take.

Maybe I should break this out into several less encompassing forum topics.

lnxfergy
11-20-2010, 05:22 PM
And lastly, all this high speed real time communication is pointless if the microcontroller, sensors, and motor drivers are not capable of using the high speed. I tried to expand on this with the discussion on I2C between the accels and Roboard. If there are 3 accels, and 1 compas on the same board all outputing 13bit data to I2C, at 1000 Hz that would be 13 + 13 + 13 + 13 bits per .001 seconds, or 52000 bits per second, or 52 kbps + overhead? (how could i figure out overhead? That doesn't seem correct to me btw, but I just don't know how to figure it out).

You need to read up on how I2C works -- it's a byte-based bus, where you can read register values from slave devices. So, each time you read the 13 bit data from a sensor, you'll need to send at least a 3 byte packet (7 bit address+read bit, a byte address to read, and a # of bytes registers to read), and then read back at least 2 bytes.

You'll have to see the datasheet for the specific sensors to see how many clock cycles it takes them to compose a response to a request (there's likely at least a few spin cycles).


This would seem like a 1 mbps I2C on the roboard (spec says 1 to ~3.3) would easily handle this communication. Is this correct? But then that same amount of data would need passed to the main SBC at 1000 Hz. There would certainly be some lag time in there. How could I estimate that?I2C is really good for reading a few bytes quickly -- but I'm not sure how well it holds up for reading nearly continuously. I'm not sure anyone has stress tested the Roboard in this capacity and could really tell you what sort of overhead there is in getting information from Roboard core-> I2C -> roboard core. You may just have to try it out.

There's also some electrical considerations to take into account when beating on the I2C bus like this, to make sure that it can rise/fall fast enough to keep up communications without lost/garbled bytes (this mainly pertains to the overall capacitance of your bus).

-Fergs

iBot
11-21-2010, 04:37 AM
When considering timing, you need to address, thoughput, latency and timing accuracy for the communication with the sensors and the drives, and also in the controller OS.

By not having a clear objective of your application, you are burning a lot of cycles of your time and those trying to help. You have chosen a 1000Hz rate which is an order of magnitude higher than most existing controllers, sensors, and actuators. A bit like designing a car for 1000mph because you are not sure if you are going to the mall or the salt flats.

Many remote control devices can be directly operated by humans, so this suggest a rate of substancially less than 100Hz. Devices with gyros and IMU, generally manage with less than 100Hz, unless they move very fast or are very small.

RoboTy
11-21-2010, 11:03 PM
Thanks for the info fergs. Do you know of any textbooks or online references that would be good for this kind of reading? (I can always search amazon, but who knows what I'll end up with).

I'm trying to nail down the additional challenges to go from something standard such as 100 Hz, to 1000Hz. Several studies show dynamic systems perform better at higher frequencies (and why wouldn't they) pretty much regardless of what is being accomplished. Granted, maybe 100 Hz would work just fine for a particular task, but I'm also trying to learn rather than just build something.