Autonomous robot's navigation

[EDV]

Hello,

Now I work at associative video memory. The method still in developing (now it version 0.5)
but it gives good results already today.

I am dealing with research of computer vision in parallel with my main job
at "Impulse" more than three years (it is my hobby).

About me: http://edv-detail.narod.ru/EDV_resume.html

In the beginning my achievements were insignificant and little part of ideas has worked properly.
But I did not surrender. I generated big quantity of hypotheses and then test it.

The most ideas did not work indeed but those that worked were similar to particles of gold
in huge quantity of dross. My associative video memory method is working indeed.

============================- Common information -==========================

Algorithm AVM uses a principle of multilevel decomposition of recognition matrices,
it is steady against noise of the camera and well scaled, simply and quickly
for training, also it shows acceptable quick-action on a greater image resolution
of entrance video (960x720 and more). The algorithm works with grayscale images.

The detailed information about AVM algorithm can be looked here:
http://edv-detail.narod.ru/AVM_main.html

AVM SDK v0.5 with examples of using and tests for comparison
of characteristics of the previous and new versions:
http://edv-detail.narod.ru/AVM_SDK_v0-5.zip

Demonstration video how to train AVM:
http://edv-detail.narod.ru/Face_training_demo.avi

AVM demo with the user interface (GUI), installation for Windows:
http://edv-detail.narod.ru/Recognition.zip

Connect the web-camera and start AVM demo after installation of "Recognition.exe".
After starting the program will inform that there is not stored previously data
of training AVM and then will propose to establish the key size of the image
for creation of new copy AVM. Further train AVM using as an example Face_training_demo.avi (http://edv-detail.narod.ru/Face_training_demo.avi).

========================- Robot's navigation -=========================

I also want to introduce my first experience in robot's navigation powered by AVM.

Briefly, the navigation algorithm do attempts to align position of a tower
and the body of robot on the center of the first recognized object in the list
of tracking and if the object is far will come nearer and if it is too close it
will be rolled away back.

See video below:
EcXl0mQcPxs
AvQMi_DdF94

I have made changes in algorithm of the robot's control (Navigator.cpp)
also I have used low resolution of entrance images 320x240 pixels.
And it gave good result (see "Follow me"):

HTxNlOpm11U

Source code of "Navigator" program:
http://edv-detail.narod.ru/Navigator_src.zip

During experiments I noted significant reserve of quick-action
when AVM v0.5 is working with resolution 320x240.

And it give me good conditions for starting of new experiments
in navigation field (algorithm "Location tree"), of course I also
will try to make AVM faster with resolution 640x480 in the future.

Also I finished some experiments with "Natural beacons":
I4NXCsCCg3A

Source code of demo (project "Beacons test"):
http://edv-detail.narod.ru/Navigator_Tool_Kit.zip

I hope that this information can help you in questions:
How we can use AVM algorithm?

If somebody can read in Russian then welcome: (Russian thread: "Autonomous robot's navigation" (http://roboforum.ru/viewtopic.php?f=35&t=6021)).

[ROBOTMAN]

Wow this works great so far! How easy would it be to interface it with python?

[nagmier]

WOW Nice I'll be taking a more indepth look at this over the weekend

[Adrenalynn]

Dear Dmitriy, first - welcome to the Trossen Robotics Forum!

Thank you most sincerely for sharing such a comprehensive result of your investigations into computer vision/recognition and vision-based navigation. A tremendous amount of information to absorb here - digital video is my specialty - and I look forward to learning from your explorations and documentation.

I do hope you will find yourself visiting us regularly!

In response to questions about "ease of integration" - this is a research project, and its intent is to advance a field and body of knowledge. IMHO: If that's the first question that comes to mind, this probably isn't the best place to start.

[EDV]

Wow this works great so far! How easy would it be to interface it with python?

I don't plan a development of the interface for python at the moment but it is possible in the future.

But AVM v0.5 SDK (http://edv-detail.narod.ru/AVM_SDK_v0-5.zip) with examples of using is available.

The compilation is possible by one of compiler:
AVM_SDK\samples\vc6.prj\AVM_prj.dsw for Microsoft Visual C++ 6.0
AVM_SDK\samples\vc9.prj\AVM_prj.sln for Microsoft Visual Studio 2008
Also library OpenCV_1.0 is needed for compilation.

The algorithm "Associative video memory" is commercial project but you can use AVM SDK in your researches and in non-commercial projects.

I will try to prepare the detailed documentation about using of AVM v0.5 SDK.

Also I will share with you results of my future researches in robot's navigation.

[Adrenalynn]

Congrats too on spinning AVM off into a commercial project!

[EDV]

Source code of "Navigator" program (http://edv-detail.narod.ru/Navigator_src.zip) and "Navigator tool kit" (http://edv-detail.narod.ru/Navigator_Tool_Kit.zip) was prepared for Russian community (some comments in source code in Russian).

But if you would show interest to this topic then I could adapt it to English community too.

[Adrenalynn]

[Mod Note: Thread-Jack Relocation: http://forums.trossenrobotics.com/showthread.php?t=3600 (http://forums.trossenrobotics.com/showthread.php?t=3600)Nikhil Thiruvengadam's mower project moved]

[siempre.aprendiendo]

Awesome! I can't read russian "directly", but Google is pretty useful (http://translate.google.com/translate?hl=ns&sl=ru&tl=en&u=http%3A%2F%2Froboforum.ru%2Fviewtopic.php%3Ff%3D 35%26t%3D6021) :)

[EDV]

Robot navigation by gate from point "A" to "B"

R4Ogq2YUfrU

1w2bMlTsLUI

First an user must set the visual beacons (gates) that will show direction where robot has to go.
Robot will walk from gate to gate. If the robot recognize "target" then he come nearer and stop walking.

Navigation application (installation for Windows):
http://edv-detail.narod.ru/Recognition.zip

Installation also contains source code of robot control driver (InstDir\RobotController_src).
So you can adapt it to your robots and repeat my navigation experiments by yourself.


Also available source code of navigation application (but all comments in Russian):
http://edv-detail.narod.ru/Navigator_src.zip

[nagmier]

Thanks again for posting some great stuff!!!!

[EDV]

Navigator package description

The package consists of three parts: the robot control driver, the pattern recognition application (GUI), and a dynamic link library "Navigator".
Compilation of pattern recognition application will need wxWidgets-2.8.x (http://sourceforge.net/projects/wxwindows/files/wxMSW/2.8.10/wxMSW-2.8.10-Setup.exe/download) and OpenCV_1.0 (http://sourceforge.net/projects/opencvlibrary/files/opencv-win/1.0/OpenCV_1.0.exe/download). If someone has no desire to deal with the GUI, then the project already has compiled recognizer (as EXE) and you will be enough to compile Navigator.dll, which contains the navigation algorithm. Compilation of Navigator.dll needed only library OpenCV_1.0 (http://sourceforge.net/projects/opencvlibrary/files/opencv-win/1.0/OpenCV_1.0.exe/download). You can build project by compiler Microsoft Visual C ++ 6.0 (folder vc6.prj) and by compiler Microsoft Visual Studio 2008 (folder vc9.prj).

After installation (and compilation) libraries wxWidgets-2.8.x and OpenCV_1.0 need to specify additional folders for the compiler:

Options / Directories / Include files:
<Install_Dir> \ OPENCV \ CV \ INCLUDE
<Install_Dir> \ OPENCV \ CVAUX \ INCLUDE
<Install_Dir> \ OPENCV \ CXCORE \ INCLUDE
<Install_Dir> \ OPENCV \ OTHERLIBS \ HIGHGUI
<Install_Dir> \ OPENCV \ OTHERLIBS \ CVCAM \ INCLUDE
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB \ VC_LIB \ MSW
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB \ VC_LIB \ MSWD
<Install_Dir> \ WXWIDGETS-2.8.10 \ INCLUDE
<Install_Dir> \ WXWIDGETS-2.8.10 \ INCLUDE \ MSVC

Options / Directories / Library files:
<Install_Dir> \ OPENCV \ LIB
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB \ VC_LIB

Source code of the "Navigator" (for English community) can be downloaded here:
http://edv-detail.narod.ru/Navigator_src_en.zip

For compilation of the source code you can use the "MS Visual C ++ 2008 Express Edition" (http://www.microsoft.com/express/download/). It is all official and free.


For connection of robot to Navigator program you have to adapt the control driver (.\src\RobotController) to your robot.

It's simple: the application Recognition.exe interacts with the robot driver "through shared memory (gpKeyArray). And all you need to do - it is a timer (method CMainWnd:: OnTimer) to send commands from the "gpKeyArray" to your robot.

The chain of start commands will be transmitted to robot for "power on" (cmFIRE, cmPOWER) when you start navigation mode. Respectively command "power off" (cmPOWER) will be transmitted when navigation mode will be disabled.

And most importantly: the commands cmLEFT and cmRIGHT should not activate motion in itself but only in combination with the commands "forward", "back" (cmFORWARD, cmBACKWARDS).

If you have adapted control driver to your robot then you are ready join to navigation experiments.

So, let's have a fun together :)

HTxNlOpm11U

R4Ogq2YUfrU

[Adrenalynn]

wow! +Rep for Massive Code Share! As soon as I'm moved, I'll definitely have to look at your work. Thanks for sharing!

[badcommandorfilename]

This looks very interesting, EDV. My thesis was on autonomous navigation - specifically on perception and landmark sensing, so I'll have to give this a try out.

Are the lib dependancies open source libraries? It would be very cool to get this to run on an embedded system so that the robot wouldn't need to be tethered.

[Adrenalynn]

given the load such video processing generates, a 1.8Ghz tiny little Windbook would be the ideal "embedded system" - and this would do fine.

[EDV]

>> Are the lib dependencies open source libraries?

In "Autonomous robot's navigation" project (http://edv-detail.narod.ru/Navigator_src_en.zip) AVM algorithm (http://edv-detail.narod.ru/AVM_main.html) is a closed technology (presented as library "rcg.lib") but other part (that is involved in navigation) is open source.

What is AVM v0.5?
- It is obtainable, accurate and fast pattern recognition algorithm already today.

What AVM algorithm will able give for us? How it can be useful for us?
- You can use AVM algorithm (http://edv-detail.narod.ru/AVM_main.html) in your researches for development of efficient navigation solution for robotics (as recognizer). You can test your hypothesis concerning robot navigation based on landmark beacons with AVM. And if successful navigation solution will be achieved then you will have two ways:

1. You can develop your own pattern recognition algorithm and then replace AVM algorithm in your finished project.
2. Or you can use commercial version of AVM algorithm in your finished project.

Anyway now you can use AVM v0.5 SDK (http://edv-detail.narod.ru/AVM_SDK_v0-5.zip) for free in your robotics hobby.

>> It would be very cool to get this to run on an embedded system so that the robot wouldn't need to be tethered.

You can compile "Navigator" program (http://edv-detail.narod.ru/Navigator_src_en.zip) by "Microsoft Visual C ++ 6.0" for example on "Windows 98" and then install it on "Roboard RB 100" (http://www.robotshop.us/roboard-starter-kit-3.html).

[EDV]

New Year's weekend has not gone in vain and I want to share my experiment results with you:

NMjv6skiumM

So, what we see in video
First the robot was set at checkpoint p[1] and then robot has received command: "walking to the point p[0]".
The robot has started looking around and when he has seen gates which correspond to the route [0] then began move to checkpoint p[0] by gates.
In checkpoint p[0] robot has stopped. Then the robot was instructed to follow to checkpoint p[1] and robot again has started looking around
and found the appropriate route gates and has come to checkpoint p[1].

I plan to update "Navigator" package soon by new navigation algorithm.

[Suicidal.Banana]

thats awesome man :eek:

[EDV]

Navigation algorithm update and yet another video:

xbCpthKrL0o

Changes
In the new version of navigation algorithm the gate data contains weights for the seven routes that indicate importance of this gateway for each route. At the bottom of the screen was added indicator "horizon" which shows direction for adjust the robot's motion for further movement on the route. Field of gates is painted blue if the gates do not participate in this route (weight rate 0), and warmer colors (ending in yellow) show a gradation of "importance" of the gate in the current route.

The procedure of training on route
For training of the route you have to indicate actual route (button “Walking by way”) in "Nova gate" mode and then you must drive the robot manually by route (the gates will be installed automatically). In the end of the route you must click on the button "Set checkpoint" and then robot will turn several times on one spot and mark his current location as a checkpoint.

So, I updated "Navigator" package:
http://edv-detail.narod.ru/Navigator_src_en.zip

[EDV]

I have remade AVM v0.5 SDK and now algorithm AVM presented as DLL library.

AVM v0.5 SDK package was updated:
http://edv-detail.narod.ru/AVM_SDK_v0-5.zip

Implementation of AVM algorithm as DLL:
.\AVM_SDK\samples\bin\avm057.dll

Program interface:
.\AVM_SDK\include\Ver0.5\AssociativeMemory.h
.\AVM_SDK\lib\Ver0.5\rcg.lib

Also I updated "Navigator" package:
http://edv-detail.narod.ru/Navigator_src_en.zip

Implementation of AVM algorithm as DLL:
.\Navigator_en\bin\avm057.dll

Program interface:
.\Navigator_en\include\AssociativeMemory.h
.\Navigator_en\lib\rcg.lib

[4mem8]

Awesome work EDV so interesting. keep up the good work.

[EDV]

Thanks, I will try to go further and once maybe machines will see the light and T-1000 will cease to be a fantasy :)

[EDV]

The first iteration of AVM v0.5 SDK porting to C# is done:
http://edv-detail.narod.ru/AVM_SDK_v0-5-net.zip


This package is using C# porting of OpenCV v2.0 by library: "Emgu CV" (http://www.emgu.com/wiki/index.php/Main_Page).

Emgu CV download (http://sourceforge.net/projects/emgucv/files/emgucv/2.0.1.0/Emgu.CV.x86_2.0.1.0_windows_installer.zip/download)

About OpenCV v2.0 (http://www.willowgarage.com/pages/software/opencv)

OpenCV v2.0 download (http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.0/OpenCV-2.0.0a-win32.exe/download)


The test video ".\AVM_SDK.net\bin\RcgTraining4s.avi" is using XviD codec (http://www.codecguide.com/download_k-lite_codec_pack_full.htm).
Make sure that it installed on your PC.

[nagmier]

Awesome I love seeing this type of work this may help me understand your prev work better! Thanks! +rep for sharing!