Protocol for the TROM

[Matt]

Let's continue the discussion for what kind of communication protocol we want to use for the TROM. As we all know, we so far have been discussing sockets / web services / etc.

[LinuxGuy]

Let's continue the discussion for what kind of communication protocol we want to use for the TROM. As we all know, we so far have been discussing sockets / web services / etc.
We could even layer something new on top of stuff like sockets. Sockets are easy to program with in any lanugage as far as I know. I've even done a bit of socket programming in Python. Whatever protocol is decided on has to be as easy as possible to use, do exactly what we need it to do and no more (if we design a new protocol), and have an API that is easy to deal with. I'd prefer not to get into layering on top something that's broken just to fix problems, but in this case would rather help fix the problem(s) and contribute changes back to the home project.

There may be an existing protocol that is very close to what we want, but it needs additions or changes. Assuming for now it is relatively bug and other problem free, we might want to take it and work with it, contributing back to the existing Open Source project as we work on it. In anycase, I think the chosen protocol has to be object oriented, or interfacing it with today's languages might be a hassle, or worse, not workable.

8-Dale

[Adrenalynn]

Can we define a bit better what we're each thinking of when we say "sockets"? Different things to different folks on different platforms.

[LinuxGuy]

Can we define a bit better what we're each thinking of when we say "sockets"? Different things to different folks on different platforms.
TCP/IP sockets. You get pretty automatic networking and communication this way.

8-Dale

[Sienna]

I want to discuss CORBA.... Long story short, its a GREAT thing for us.... I am going to successively edit this post until I finally get this stupid thing into the forum! (Sometimes technology just ....!)

1) What is CORBA?
Common Object Request Broker Architecture (http://www.omg.org/gettingstarted/corbafaq.htm) (CORBA) is a paradigm for transmitting messages between software applications, that crosses software languages, machines, vendors, all interoperable. The CORBA specificiations are maintained by the Object Model Group (OMG). The standards are published and freely available. Unlike web services, CORBA has had 15 years to mature. This means its not 'bling bling sales material' to sell to upper management, but it does mean it has been vetted by the industry and works well. (e.g., the growing pains are taken care of already.)

2) What is an ORB?
An Object Reference Broker (ORB) is to CORBA what an router is to Ethernet. When a Client wants to talk to an Object, the client hands the request to the ORB, who figures out where the object is (wether remote or local), and the ORB takes care of routing, retransmissions, etc, to get the messages between client and object. There are free ORBs available for the major platforms. (For instance, look at TAU) Typically, there is an ORB on every platform.

3) What is IIOP?
The Internet Inter-ORB Protocol (IIOP) is the binary message format that ORBs use to communicate. E.g., if you really want to get down to a bit bang level, this is the wire format to use.

4) How to the ORBs know what functions are available?
That is where the Interface Definition Language (IDL) comes into play. IDL specifies to the ORB what functions and such are available for other applications to use. Anything not in the IDL, is off limits to remote functions. (like object oriented programming...)

5) But this IDL thing, how do I implement it?
Relax! Standard IDL mappings are already standardized for a lot of various languages already! (see the OMG site for downloads)

So, what does this all mean for the TRS?

Actually, its quite simple. At each boundary, we specify what the IDL looks like. For instance, at the sensor library, we can specify IDLs for ultrasonics, ir, gyros, etc, or on the other side, servos, steppers, galvometers, what have you. You can do the same for the algorithm library, exposing functions for IK, FK, etc.

ORBs handle the message routing and discovery, IDL handles making sure that incoming and outgoing data is consistent across langauges. The bulk of specifying the protocol, interaction methods, etc, have already been built by industry, and vetted. All we have to do is specify what the IDL looks like for our various architecture components, and we have interoperable pieces!

(All that is at least in theory... I have to admit that my total experience with CORBA is ... this past Sunday reading up on it.... But I did talk the problem over with my co-worker who has used it before, and he agrees on it in this application... But it really does look promising, and it looks like it will take a great deal of work from us!)

[LinuxGuy]

1) What is CORBA?
Common Object Request Broker Architecture (http://www.omg.org/gettingstarted/corbafaq.htm) (CORBA) is a paradigm for transmitting messages between software applications, that crosses software languages, machines, vendors, all interoperable.
I love it when you use big words! :veryhappy::veryhappy:

I'll have to look at this once I get OGRE built and working. I can only handle one thing at a time right now. I have heard of CORBA, but never had a reason to look at it. Now I do.

2) What is an ORB?
An Object Reference Broker (ORB) is to CORBA what an router is to Ethernet. When a Client wants to talk to an Object, the client hands the request to the ORB, who figures out where the object is (wether remote or local), and the ORB takes care of routing, retransmissions, etc, to get the messages between client and object. There are free ORBs available for the major platforms. (For instance, look at TAU) Typically, there is an ORB on every platform.
Ah, and Object Router (OR). :veryhappy::veryhappy: Hey, I live in OR. Hmmmm, does that make me an object?

I know CORBA is available for Linux/UNIX. What is the Windows implementation? Or is that TAU? Oh, I can see possibilities here for inter-robot communications too. W.A.L.T.E.R. and his swarm of RSRs.. You know I am going to want to get all this onto Hammer.. :happy:

8-Dale

[Sienna]

You caught me in the middle of edits Dale, now go read the whole post :P

[Matt]

This means its not 'bling bling sales material' to sell to upper management

LOL, you are hilarious :veryhappy::veryhappy:

[Adrenalynn]

>> Hey, I live in OR. Hmmmm, does that make me an object?

I Object!

[metaform3d]

I looked into CORBA about 8-10 years ago for an entirely different project, so I'm probably out of date. It seemed to me the main weakness of the system was that it required everything to be under the same ORB umbrella. Which is probably fine when we're talking about hooking up to the driver for a sensor, but I don't want to have to organize all my motion files into a database accessible to the ORB for anything to work right. I suspect most of this was because -- at the time -- it was designed to build enterprise-wide web apps rather than quick hacks that by design have to be more nimble. Obviously a lot can change over the years so I'd be very curious to hear some detail and examples of how this would work.

To my untrained eye at the time the IDL and the interfaces looked very much like a knock-off of COM, although I'm sure there were some differences. I also imagine it has evolved over the years, which is certainly a point in its favor.

[Sienna]

But my problem with COM is that it doesn't appear to be an industry standard, but a single company standard. As I said earlier... if you could point me to the controlling group (e.g., something bigger then one monopoly), I would read up.

[Alex]

In anycase, I think the chosen protocol has to be object oriented, or interfacing it with today's languages might be a hassle, or worse, not workable.I hope this isn't even up for discussion. I can't even think in terms of non-OO programming anymore:p

but I don't want to have to organize all my motion files into a database accessible to the ORB for anything to work right.

Does CORBA require a database? I certainly hope not because that goes against what I believe we're trying to develop here...

[Adrenalynn]

TCP/IP sockets. You get pretty automatic networking and communication this way.

8-Dale

If we're in agreement that this is what we're looking at, a potential concern is latency. Latency betwixt components seems to grow exponentially. In an instance like that, pipes might be preferred.

Any pipe/socket system gets us to distributed computing, which is cool

[Sienna]

Does CORBA require a database? I certainly hope not because that goes against what I believe we're trying to develop here...

I don't understand what you mean?


And to the 'socket' people... are we seriously going to even consider rewriting the wheel again? Do you all know just how many libraries there are to pass data on network like things? It would be madness to consider writing another one! (oh, and stay away from specifying the transport layer, as we should be free to say my transport layer is SLIP one day, IPsec the next, PPP the third, loopback addresses the fourth, zigbee the fifth, etc.) Honestly, I would be inclined to say that the TRS should only worry about communicating on the application layer of the OSI model, not anything below it. Thats what an operating system is for.

Adrenalynn, what do you mean latency is a concern? I actually have to design networks at work, and any decent switched network will give you <5ms to go across multiple hops (typically sub 1ms on a single hop) (and yes, I actually have devices that measure latency and jitter).
What do you imagine that will impact?

[LinuxGuy]

And to the 'socket' people... are we seriously going to even consider rewriting the wheel again? Do you all know just how many libraries there are to pass data on network like things?
Be calm, awesome Geekette.. :happy:

We are not trying to upset your apple cart. :veryhappy::veryhappy: It was only a brief consideration. I also wanted to point out a possibility of interfacing with a chosen protocol at different levels.

8-Dale

[metaform3d]

But my problem with COM is that it doesn't appear to be an industry standard, but a single company standard. As I said earlier... if you could point me to the controlling group (e.g., something bigger then one monopoly), I would read up.I see. I tend to think of COM as a technology spec, not a protocol standard. It's a little like XML. There's a standards body and you can find the official way to encode midi or something, but as a technology it's mostly just a stream of tokens with angle-brackets and slashes that you can easily understand and use any way you want. One of the great strengths of XML is that you don't have to be a computer scientist to parse it.

That said, given that the direction here seems to be towards some sort of network distributed system I don't think COM would be appropriate. As I already said, it's chief weaknesses are in that area.

I am curious why there seems to be a focus on finding the right network protocol for robotic control. Do people really find that web-enabling their robots is what people are really looking for? For one I would be more interested in using concurrent threads in the same processor for dividing up control and sensing tasks, and I can't imagine that network protocol latency would be great for that.

[Adrenalynn]

Adrenalynn, what do you mean latency is a concern? I actually have to design networks at work, and any decent switched network will give you <5ms to go across multiple hops (typically sub 1ms on a single hop) (and yes, I actually have devices that measure latency and jitter).
What do you imagine that will impact?

In a realtime system? Everything. Go jump in the car and see what 4ms will mean to your airbag deployment. Notice that that isn't a TCP system.

>> I actually have devices that measure latency and jitter

Who doesn't? Since around the time when I was doing the simultaneous multi-OS networking port for Apollo/HP Apollo and having to juggle BSD Sockets vs SysV Streams. Remember those? Been awhile since I even thought about 'em. Which was your favorite back in the early 80's - Streams or Sockets? Were you an Ultrix girl, or an Apollo Domain/HP-UX/Solaris girl? Or did you ignore all that and just stick to VMS?

That's actually a significant questions, because it tells us what color your message passing glasses are. Myself, I was all about Streams.

I could also hang 2500 lbs of QOS routing gear on the 'bot too. It's just a bit excessive to hang 2500 lbs of routing gear on an antweight...

[Adrenalynn]

Do people really find that web-enabling their robots is what people are really looking for? For one I would be more interested in using concurrent threads in the same processor for dividing up control and sensing tasks, and I can't imagine that network protocol latency would be great for that.

Totally onboard with you here. But distributed computing for things like vision, voice, and other things requiring BigMath (high density FFT, for example) is where it's at out there.

I don't much care for being stuck with the latency from a few dozen sensors either, though. Which is why I've put my toe in the water for a multi-interface methodology - whether that's DCOM or something else. I'd sorta rather avoid DCOM as that's really mickeysoft-oriented.

[LinuxGuy]

I am curious why there seems to be a focus on finding the right network protocol for robotic control. Do people really find that web-enabling their robots is what people are really looking for?
Absolutely! You would be surprised at the number of people working towards network enabling their robots. Ummm, hey, that includes me! :happy: This is why I am excited about using Python for the new control software for W.A.L.T.E.R. - everything is already there for me to build on and change as required. You bet I want to be able to hook into TROM and all the potential it offers. :veryhappy: I already have W.A.L.T.E.R. setup for USB WiFi B/G (http://www.edimax.com/en/produce_detail.php?pd_id=3&pl1_id=1&pl2_id=44) and am working on Bluetooth (serial DIL module (http://www.sparkfun.com/commerce/product_info.php?products_id=8461)), all interfaced to Hammer (http://www.tincantools.com/product.php?productid=16133&cat=0&page=1&featured). ZigBee will be the next wireless module I will add, especially for the mesh networking capability. Think swarm robotics (http://swarm-robotics.org/) here. :veryhappy::veryhappy:

For one I would be more interested in using concurrent threads in the same processor for dividing up control and sensing tasks, and I can't imagine that network protocol latency would be great for that.
This would be nice also, but I don't think it should be too high on a priority list. Good multitasking software systems exist that I think would work just as well - Linux, uCLinux (http://www.uclinux.org), Ecos (http://ecos.sourceware.org/), FreeRTOS (http://www.freertos.org), etc. I'm experimenting with FreeRTOS (http://www.freertos.org/) on an Olimex NXP LPC2148 (http://www.sparkfun.com/commerce/product_info.php?products_id=545) board.

8-Dale

[Adrenalynn]

This would be nice also, but I don't think it should be too high on a priority list. Good multitasking software systems exist that I think would work just as well - Linux, uCLinux (http://www.uclinux.org), Ecos (http://ecos.sourceware.org/), FreeRTOS (http://www.freertos.org), etc. I'm experimenting with FreeRTOS (http://www.freertos.org/) on an Olimex NXP LPC2148 (http://www.sparkfun.com/commerce/product_info.php?products_id=545) board.

8-Dale

If your app is some monolithic single thread thing, all the RTOS in the world isn't going to help. You can't divy-up serial code that way...

[LinuxGuy]

If your app is some monolithic single thread thing, all the RTOS in the world isn't going to help. You can't divy-up serial code that way...
No, it's all distributed, which is why I am getting even more excited about TROM as we discuss it. I've got to get OGRE (http://www.ogre3d.org) built so I can tinker with it, then look at CORBA and see what I can get to communicate. I'm real big on interfacing. :veryhappy: I have my eye on the Blackfin Camera Board (http://www.surveyor.com/blackfin/) for vision, and even it can run Linux if I want to go that route. OpenCV (http://opencvlibrary.sourceforge.net/) might work good here. I'm also watching development of the i.MX31 board (http://wiki.virtualcogs.com/index.php?title=VCMX310) that Virtual COGS (http://www.virtualcogs.com) is working on.

8-Dale

[Adrenalynn]

Last time I used CORBA, I quit the job after about six weeks. I'm sure it's grown up a fair bit in the last eleven years or so though. I just remember it was awful, and have avoided looking at it since. (Architect for a company called "Dascom" doing biometrics and encryption stuff - wonder if they're still around)

[LinuxGuy]

Last time I used CORBA, I quit the job after about six weeks. I'm sure it's grown up a fair bit in the last eleven years or so though. I just remember it was awful, and have avoided looking at it since. (Architect for a company called "Dascom" doing biometrics and encryption stuff - wonder if they're still around)
Well, let's keep an open mind and see how things shape up. So far, from what I have read about CORBA, it looks worthwhile. I so love distributed architectures!

8-Dale

[Adrenalynn]

That's the first negative thing I've said about CORBA. I'm pretty open minded about such things, and I realize that time and technology marches on. Just not so open-minded that my brain falls out. ;)

[LinuxGuy]

That's the first negative thing I've said about CORBA. I'm pretty open minded about such things, and I realize that time and technology marches on. Just not so open-minded that my brain falls out. ;)
Ha ha, funny girl! :tongue: The positives and negatives of anything we are considering using in the core of TROM needs to be discussed. What we choose is what we are going to be using for a long time.

8-Dale

[Alex]

Does CORBA require a database? I certainly hope not because that goes against what I believe we're trying to develop here...
I don't understand what you mean?

metaform3d mentioned CORBA and a database in post #10 (http://forums.trossenrobotics.com/showpost.php?p=8390&postcount=10):

I looked into CORBA about 8-10 years ago for an entirely different project, so I'm probably out of date. It seemed to me the main weakness of the system was that it required everything to be under the same ORB umbrella. Which is probably fine when we're talking about hooking up to the driver for a sensor, but I don't want to have to organize all my motion files into a database accessible to the ORB for anything to work right. I suspect most of this was because -- at the time -- it was designed to build enterprise-wide web apps rather than quick hacks that by design have to be more nimble. Obviously a lot can change over the years so I'd be very curious to hear some detail and examples of how this would work.

I was probably misunderstanding and looking a little too deeply into this though.

[metaform3d]

I was probably misunderstanding and looking a little too deeply into this though.I was speaking somewhat imprecisely, but an ORB *is* a database of a kind. It has to maintain an index of all the objects in your system and matches requests with available services. When you need a new object, how does the ORB find it? From my previous experience with it (and recall this was 8-10 years ago) it seemed to me that this formalism was imposed at a lot of levels. I'm having a hard time saying this exactly, but it's kind of like once you put a little bit of stuff into the system you had to put it all in there.

[metaform3d]

Have to disagree about the importance of concurrency -- it should be one of the primary goals of the design. You can't design a system as linear and add support for threading later. In addition to any blocking API we should support non-blocking message passing or a shared message boards as well. We should also support the full range from closed-loop real-time control to asynchronous BigMath (tm) computations.

[Sienna]

Would you like the moon and sun as well?

At some point I think we are going to have to bound the problem we are trying to solve.

The needs of a hardcore realtime engine and the needs of a remote processing engine are completely different. I think it entirely unfeasable to support every possible architecture you can think of. That is, unless you plan to spend millions of dollars?

[Matt]

I think that while we analyze different protocols we should keep in mind that a lot of weight should be given to what is known and used widely. While some technologies are superior to others the fact that only 3% of the technical population know and use it can be a hurdle too high too choose it.

CORBA sounds awesome, but if it is the protocol of elite 250K a year technicians while web services is the "every man" version the decision is obvious to me. To be clear, I'm not advocating one over the other right now, I want the debate to go on. I'm only asking that we keep careful consideration on this part of the decision. I think it's great that Sienna laid out all this info on CORBA, we need to do this exercise to make an informed choice.

I found this article (http://www2002.org/CDROM/alternate/395/) that does a good comparison of the two.

[Alex]

You can't design a system as linear and add support for threading later.It can be added later, but not very easily and I believe it'll be worth our time to do it sooner rather than later.

This is an excellent topic you brought up and is something that I forgot to bring up till now. Matt and I had discussions of adding multi-threading capabilities to the TROM back when I was developing it. I don't want to hijack this thread on this topic, so please see this thread (http://forums.trossenrobotics.com/showthread.php?t=1691).

[Sienna]

I think that while we analyze different protocols we should keep in mind that a lot of weight should be given to what is known and used widely. While some technologies are superior to others the fact that only 3% of the technical population know and use it can be a hurdle too high too choose it.

CORBA sounds awesome, but if it is the protocol of elite 250K a year technicians while web services is the "every man" version the decision is obvious to me. To be clear, I'm not advocating one over the other right now, I want the debate to go on. I'm only asking that we keep careful consideration on this part of the decision. I think it's great that Sienna laid out all this info on CORBA, we need to do this exercise to make an informed choice.

I found this article (http://www2002.org/CDROM/alternate/395/) that does a good comparison of the two.

Not to derail this thread, I made a topic on 'everyman (http://forums.trossenrobotics.com/showthread.php?p=8545#post8545)'

[Sienna]

Matt, the article you linked too, it seems to suggest that the right thing to do is merge the two.

And you know what, I think that may work. If we can implement a CORBA like architecture between core lower level components, and then tie a SOAP layer in a high level (whatever that might be), it might make the protocol both efficient at a low level and accesable at a higher level.

[metaform3d]

Would you like the moon and sun as well?Sure, as long as you don't forget to support galaxies and some of the smaller asteroids.

I agree we need to bound the problem. Having re-read the initial "battle cry" for this project, it's clear that the TR people are not interested in programming microcontrollers, so we can rule that out. Like them, I would *far* prefer to do everything on the main processor, especially if it's multi-core. I'm much less interested in trying to distribute my robot's brain over the internet. The key thing in my mind is replacing microcontrollers with software, which to me means dedicated closed-loop threads.

[Matt]

Matt, the article you linked too, it seems to suggest that the right thing to do is merge the two.

And you know what, I think that may work. If we can implement a CORBA like architecture between core lower level components, and then tie a SOAP layer in a high level (whatever that might be), it might make the protocol both efficient at a low level and accesable at a higher level.

If we can do both then I love that answer. But if we can't I have to say that web services is really in the lead as our choice. It's just won too much of the market and there are too many benefits of that we would be walking away from.

Adrenalynn also mentioned that she thinks we can implement both sockets and pipes which pretty much ends that debate nicely :)

I think we are getting close here on some decisions, yay team.

[LinuxGuy]

And you know what, I think that may work. If we can implement a CORBA like architecture between core lower level components, and then tie a SOAP layer in a high level (whatever that might be), it might make the protocol both efficient at a low level and accesable at a higher level.
Can we have both efficiency and accessibility at the lower and higher levels?

With SOAP, don't we need WATER too? :tongue:

8-Dale

[LinuxGuy]

I agree we need to bound the problem. Having re-read the initial "battle cry" for this project, it's clear that the TR people are not interested in programming microcontrollers, so we can rule that out. Like them, I would *far* prefer to do everything on the main processor, especially if it's multi-core.
I think you are going backwards a bit here. Much has happened since Matt and Alex opened the door to the bunch of us. There is an ever graying line between what is called a "microcontroller" and what is called a "PC" now. I believe the two will eventually converge at some point. TROM needs to be able to scale in both directions - down to system on chip processors as well as up to PCs and even larger systems that may include clusters of small, large, or mixtures of small and large systems.

I'm much less interested in trying to distribute my robot's brain over the internet. The key thing in my mind is replacing microcontrollers with software, which to me means dedicated closed-loop threads.
This may be true for you, but it is not true for everyone. There are many people out there doing many different things with robotics. If TROM can not scale in both directions, it will never become the standard we hope it will eventually become. TROM needs to allow you to work with it and use the parts of it that are useful to you in your projects, but it also has to allow me and others to work with it and use the parts that are useful to use for our projects. I think you are looking at TROM as a monolithic solution, but this does not embrace the distributed architecture, which must be embraced as well. Done right, TROM can handle both equally well, at all levels. We aim to do TROM RIGHT. :veryhappy::veryhappy:

Swarm Robotics is and will continue to be a rapidly growing area of robotics research and development. TROM must embrace this at some level, just as it must embrace the single robot projects you, I, and others work on. TROM is BIG, bigger than any one of us or the group of us or any one company.

8-Dale

[Alex]

Very well said Dale, and I agree with you. If we do this right, the TROM should be scalable to run directly on even microcontrollers. It's just a matter of figuring out how the hell to do it:D

[metaform3d]

Very well said Dale, and I agree with you. If we do this right, the TROM should be scalable to run directly on even microcontrollers. It's just a matter of figuring out how the hell to do it:DOK, fair enough. That's beyond my area of expertise so I'll let others hash it out. I'm curious to hear if CORBA is flexible enough to talk to objects running on a SPIN, for example.

[Alex]

Hey meta, the link that Matt gave us in a previous post in this thread REALLY helped me to understand CORBA and Web Servies from a top level perspective. It doesn't get into the deep geeky details and does a nice comparison between the two:

http://www2002.org/CDROM/alternate/395/

[LinuxGuy]

I'm curious to hear if CORBA is flexible enough to talk to objects running on a SPIN, for example.
Are you refering to the Parallax Propeller (http://www.parallax.com/Default.aspx?tabid=407)?

8-Dale

[Matt]

Okay, we have a drawing on the wall now. Our dream scenario is a TROM that can do both web services & CORBA using sockets & pipes or some combination thereof.

I believe it's time to start working on a theory of architecture. I also think it's safe to begin thinking through this in C++ since no other language is surviving the community gauntlet.

The question is: How do we do it?

Dale asked this>> Can we have both efficiency and accessibility at the lower and higher levels?

Good question, so let's begin to think about ways to approach integrating multiple protocol choices. Can it be done in one package or should we be thinking about having a few different flavors of the TROM? All identical except for the protocol?

We have 4 components that need to communicate: TROM, SCL, Rendering Tool, & Algorithm Library (AL). It will probably help to imagine the data that needs to be passed around between these components.

What other early points of consideration should be on the table? Let's not over engineer right away though. We need to just think through how we might be able to put these 4 protocol puzzle pieces together first. If we can't then we start to break up or whittle. We can worry about XML standards and such things later.

[LinuxGuy]

I believe it's time to start working on a theory of architecture. I also think it's safe to begin thinking through this in C++ since no other language is surviving the community gauntlet.
Agreed. The question now is are we actually going to do the port from C# to C++ now or wait? I would much prefer sooner to later.

Good question, so let's begin to think about ways to approach integrating multiple protocol choices. Can it be done in one package or should we be thinking about having a few different flavors of the TROM? All identical except for the protocol?
First, we have to define what protocols (such as SOAP, CORBA, etc) and/or access methods (Sockets, Pipes, etc) we want to use. I think each has a particular level it would be best positioned at. I'd say Sockets and Pipes should be available at all levels, but they are lower level data and communication conduits. Sockets and Pipes are pretty easy to use. It's really the data available at each level that is the difference. Sockets and Pipes are really just connectors between a client and a data source (which can be a client or server). I use the term 'server' here to mean a provider of data or service(s).

We have 4 components that need to communicate: TROM, SCL, Rendering Tool, & Algorithm Library (AL). It will probably help to imagine the data that needs to be passed around between these components.
I am seeing it this way: TROM --> Algorithm Library --> SCL. The Rendering Tool really doesn't belong in the chain. It will be used mostly by those who add on to TROM, but it will also be used in the APIs we and others define. We shouldn't have to really worry too much about the Rendering Tool yet. Have we officially adopted OGRE or are there other rendering tools we should look at before deciding? The Rendering Engine will interact with the rest at many different levels.

What other early points of consideration should be on the table? Let's not over engineer right away though. We need to just think through how we might be able to put these 4 protocol puzzle pieces together first. If we can't then we start to break up or whittle. We can worry about XML standards and such things later.
I think we can't for further until we have defined our components completely. We need to adopt protocol(s) we want to use, how we will use them, and at what level they will be used. It would be nice to see some samples of how CORBA and SOAP are used.

I can't do this right now because I'm working with OGRE. I want to see if I can produce some sort of result and how difficult it is to do it.

We are almost at a point where we can start working with what we have defined producing some results. :veryhappy::veryhappy:

Matt and Alex: Have you considered an interactive white board tool? Sometimes it's easier to draw a picture or diagram than to try explaining something. I think SKYPE has this capability, but am not sure. Can the huge image of the TROM diagram be made readable as a smaller image or perhaps more than one image? Slide format would be great if this is doable. I'd love to take something like that to the PARTS (http://www.portlandrobotics.org) meeting.

8-Dale

[Adrenalynn]

I propose we mock-up that architecture. Do a simple client-server state engine, just a simple server that returns something predictable and a simple client that sends something predictable. I'd have to relearn some CORBA myself, and could do so Friday night unless someone else is getting in line. Sockets and Unnamed Pipes are trivial (from my perspective).

Any other takers, or should I run with mocking something up as a proof-of-concept? I think it'd go a long way towards demonstrating that it's either feasible or not so feasible...

I stand ready for the group's take on it...

[metaform3d]

Any other takers, or should I run with mocking something up as a proof-of-concept? I think it'd go a long way towards demonstrating that it's either feasible or not so feasible...I for one would love to see a working example of what's being discussed here. Web apps are not my forte so I'm not sure I'd understand it, but I'd understand it a lot better in practical form.

And yes, I meant the Propeller chip. Spin was the high-level language. Just misremembered terms from my last ROBOT magazine.

[LinuxGuy]

And yes, I meant the Propeller chip. Spin was the high-level language. Just misremembered terms from my last ROBOT magazine.
I have not looked at the Propeller or SPIN for awhile, but do have a Propeller here somewhere. It should be possible to get SPIN to communicate with TROM in some way, even if it is via a standard serial port. You would have to write some code for an interface between SPIN and TROM though. Dang, there I go trying to create a new interface again.

8-Dale

[LinuxGuy]

I propose we mock-up that architecture. Do a simple client-server state engine, just a simple server that returns something predictable and a simple client that sends something predictable. I'd have to relearn some CORBA myself, and could do so Friday night unless someone else is getting in line. Sockets and Unnamed Pipes are trivial (from my perspective).
I think this is a great idea. However, I think we should review the master diagram Matt and Alex have and make any necessary changes there first. Yes, sockets and pipes should not be difficult at any level of TROM.

8-Dale

[Sienna]

I have been doing some more reading on CORBA vs Web Services these past few nights. Here is the general impression I am getting:

CORBA: Declining in industry as a whole as the industry moves on to 'shinier' things. Growing in certain markets such as real time systems and corporate intranets. ORBs are an all or nothing approach, so there is no such thing yet as a 'lightweight' ORB. CORBA's 'death' seemed to be caused by standard body bickering and politics.

Web Services: The Shiny is starting to wear off, and people are starting to transfer the shiny to "REST". A lot of the promises still haven't panned out (like interoperability). It seems the biggest reason people like Web Services is because of 'XML' and port 80 traversal of firewalls (which is a naive assumption, as it puts more load on firewalls, and firewalls are increasingly doing deep packet inspection and filtering)

Personally, WSDL seems like a hard to read language, so if we did go web services I would ask that we consider really narrowing/simplyfying the syntax.

Then there is sockets/pipes discussion. I am not quite sure the distinction between the two. Is it just one of them has random access, the other has sequential access?

Then we might need to take a step back, and say which parts of the TRS we expect to be RPC based, and which parts we expect to be message based.

[Adrenalynn]

CORBA = dying. I think most in the thread have made that observation too.

Sockets = network. Unnamed Pipes = local low-latency IPC. (That can be tricked, but just go with it for the moment)

>> as it puts more load on firewalls

Tell little Jimmy to lighten-up on the Pr0n or mommy's robot is going to unplug his computer. We're not talking about NASA's public-facing firewall here, and figuring out whether or not carrying the rover's traffic over the public Internet is a good idea... We already have an answer to that...

>> WSDL seems like a hard to read language

IPC over sockets has existed since the 70's, at least. We don't need to make it complicated...

[Matt]

I am seeing it this way: TROM --> Algorithm Library --> SCL. The Rendering Tool really doesn't belong in the chain. It will be used mostly by those who add on to TROM, but it will also be used in the APIs we and others define.

The AL isn't between the TROM and the SCL. The TROM is pretty much sitting ontop of the SCL in the chain of com. I have Jen working on a new chart graphic today which will help the discussion. I'll post as soon as it's ready :)

[LinuxGuy]

CORBA: Declining in industry as a whole as the industry moves on to 'shinier' things. Growing in certain markets such as real time systems and corporate intranets. ORBs are an all or nothing approach, so there is no such thing yet as a 'lightweight' ORB. CORBA's 'death' seemed to be caused by standard body bickering and politics.
This still sounds like it could be useful for robotics and TROM. What kind of overhead are we talking about to use CORBA?

Personally, WSDL seems like a hard to read language, so if we did go web services I would ask that we consider really narrowing/simplyfying the syntax.
I am not liking the sound of this at all. I would prefer to not have to deal with something that is still a moving target..

Then there is sockets/pipes discussion. I am not quite sure the distinction between the two. Is it just one of them has random access, the other has sequential access?
Sockets and Pipes can be used pretty much anywhere. Pipes are connectors between components and work pretty much the same as piping the output from one command to another (ps -ax| more). Sockets are connectors between components on a network, which can be expanded easily to include the entire internet. I can create a socket to connect a Python (http://www.python.org) script to RoboRealm (http://www.roborealm.com) running on a PC anywhere on the internet for video processing of images or video a robot might gather from an on board camera. A robot would send messages through the socket to RoboRealm to tell it what data it needs.

8-Dale

[JonHylands]

IPC over sockets has existed since the 70's, at least. We don't need to make it complicated...

I use this technique in BrainBot, for talking between the "brains" PC and the gumstix that is onboard. The socket basically ends up acting like a fast bidirectional serial port, and I send commands in one direction, and get responses back again. The nice thing about using sockets is it doesn't matter if the two processes are on the same processor, or two machines on the same network. In my case, its two machines, and since the gumstix supports wifi, you get a high-speed wireless link. When I run completely on the PC, the socket is on the same machine, and it still works exactly the same regardless.

We also use this to talk between the "brain" process and RoboRealm, which is a nice piece of vision processing software. Once again, RoboRealm can be running on the same machine, or we can run it on a hopped-up quad core box that is blistering fast, and connected via gigabit ethernet.

Sockets give you a lot of freedom to choose (and change, at runtime) your machine configuration. Since almost every language out there that runs on a real OS has socket support, you don't constrain what language people can write their code in so they can connect to your system.

Everyone here knows I'm a Smalltalk fanatic - if RoboRealm worked like OpenCV, I'd never be able to use it, at least not without writing an enormous amount of glue code in the middle.

- Jon

[LinuxGuy]

We've been tossing around CORBA as a possible protocol to use for TROM, but are there any other options we should be considering? I've been reading a bit about CORBA, and so far I really like what I see and think it would work perfectly for TROM.

Let's discuss the pros and cons of using CORBA, as well as any alternatives there may be. I would much rather use a well documenented and standardized protocol if possible, because this will increase the ability of TROM to connect to resources we might not even think we need right now.

For instance, I can see connecting to a remote computing resource over the LAN or internet. We might even have a robot connect to this resource for a larger data store, raw computing power for vision and image processing, etc. This resource could be anywhere on the internet.

8-Dale

[Alex]

The Shiny is starting to wear off, and people are starting to transfer the shiny to "REST"

Are you referring to REST services, ie. what I mentioned in this thread (http://forums.trossenrobotics.com/showpost.php?p=8234&postcount=10)? If so, could you break it down a bit so that others can understand it better (specifically me:))? Probably better in a new thread. From what I read about them, they almost seem spot on with our needs.

[Matt]

I propose we mock-up that architecture. Do a simple client-server state engine, just a simple server that returns something predictable and a simple client that sends something predictable. I'd have to relearn some CORBA myself, and could do so Friday night unless someone else is getting in line. Sockets and Unnamed Pipes are trivial (from my perspective).

Any other takers, or should I run with mocking something up as a proof-of-concept? I think it'd go a long way towards demonstrating that it's either feasible or not so feasible...

I stand ready for the group's take on it...

Mock up is a good idea. Let's get our hands a little dirty in order to get a feel for the protocols. What I'm curious about is how to have both in the same model if we can. Should they be layered? Side by side? Incompatible that close? Etc.

[metaform3d]

This is not my area of expertise, so feel free to ignore me if I'm making no sense. But having tried to understand what you are doing in other threads my question is this: do you need a protocol?

As I understand it, the idea of a protocol is to allow modules written for the system to interoperate. Obviously this is essential inasmuch as different things have to work together, although not so much by the modules talking to each other as the system as a whole talking to the different modules. You're already creating your objects dynamically based on an XML description -- an approach I very much respect. It seems to me that you have picked the XML format as your lingua franca; why do you need to pick another protocol? In other words, the TRS XML is your protocol.

Just speaking generally, suppose I wanted to add some hardware H to the system. I would have to start by writing the XML file for H, which the system would look up and load when a user's configuration included a reference to H. Suppose that XML file also included a description of the protocol? If it was your built-in protocol P then you would just use it directly. But if it was some external protocol X you would simply look up and load its XML description. That description would ultimately contain a reference to a module that would translate between P and X.

I guess I'm not saying that protocols aren't important, just that they are fungible. You have to define an intrinsic protocol P, but it can be essentially arbitrary. Pick your favorite, or the easiest to code, it doesn't matter. Ultimately the XML which describes the TRS component will facilitate interoperability.

[LinuxGuy]

Mock up is a good idea. Let's get our hands a little dirty in order to get a feel for the protocols. What I'm curious about is how to have both in the same model if we can. Should they be layered? Side by side? Incompatible that close? Etc.
I will help with this if I can. I'm starting to feel like I am drowning and am way out of my league with this stuff lately, and I do not swim. Somebody care to toss me a life preserver?

8-Dale

[Adrenalynn]

At least 95% of life is showing-up. Just dive in, you're a smart guy, you'll figure it out. I think all of us have been open here that much of this is outside of our experience, and we're making it up as we go... If it were easy, someone else would have already done it.

Don't sweat it, Robo'Guy. We'll all learn it/figure it out together

[LinuxGuy]

Don't sweat it, Robo'Guy. We'll all learn it/figure it out together
I just sort of seem to have fallen into this whole thing somehow. I'm still trying to figure out if I'm dreaming or not. Pinch me and see if I am awake. :happy:

I just realized we are on the same coast. We could almost do lunch.

8-Dale

[LinuxGuy]

Are there other protocols that need to be discussed, besides CORBA? It gives us a lot that we won't have to write ourselves, even if it is more than we actually need. I'd rather have more than we need than find out something we are using is just not enough and have to rewrite.

If we can decide on the protocol, we can start looking at implementations.

8-Dale

[Matt]

Hi,

We have decided on the protocol, I just haven't official announced it yet. So I guess this is it. :P Since the goal of the TRS is to provide easy to use tools and methods for your "average" developer/programmer to be able to build robotics it is logical to begin with web services. We will implement both sockets and pipes.

I've talked to too many engineers and developers that say CORBA is too much to deal with and unnecessary for our needs. I have to agree. We will be careful to build the TROM in such a way that the protocol layer is a "cap" that can be swapped out and replaced later by those that wish to do so.

We will build web services that aren't "MS web services" but rather the lighter more flexible and nimble version of web services. (IE: this is what roborealm did)

[Adrenalynn]

Cheer!

Have we looked at what RoboRealm did for passing around variables? Really clever repurpose... They're using OSC (Open Sound Control) protocol to toss variables around over UDP to other less aware apps. Pretty darned spiffy for broad-ranging support - especially of proprietary systems.

[LinuxGuy]

We have decided on the protocol, I just haven't official announced it yet. So I guess this is it. :P Since the goal of the TRS is to provide easy to use tools and methods for your "average" developer/programmer to be able to build robotics it is logical to begin with web services. We will implement both sockets and pipes.
I must have missed something. Sockets are cool to work with and quite easy to use. Even I can use them! :happy:

I've talked to too many engineers and developers that say CORBA is too much to deal with and unnecessary for our needs. I have to agree. We will be careful to build the TROM in such a way that the protocol layer is a "cap" that can be swapped out and replaced later by those that wish to do so.
If the calls are done right, that should not be difficult. These "web service" calls can be replaces with identical calls that call other protocol(s) that might be needed later. I don't really like adding extra layers like that though and prefer to work directly with the protocols themselves.

We will build web services that aren't "MS web services" but rather the lighter more flexible and nimble version of web services. (IE: this is what roborealm did)
RoboRealm (http://www.roborealm.com) works great too, no doubt about that. I wish there were a Linux version of it.

8-Dale