PDA

View Full Version : [Contest Entry] Sentient Turret



ekoos7
12-05-2007, 01:50 PM
Hi all this is my first project that I have done with equipment from Trossen Robotics. Basically how it all started is I live in South Africa probably one of the worst countries in the world right now when it comes to crime! One of my friends is a distributer of PepperBall self-defense systems. So he said he wanted to build some sort of automated gun. And well so I said I would have a look and then I found Trossen online and it all started.

The idea was for a gun turret that could operate in two modes one manual where you would move your mouse over the video feed and click to shoot. So the gun would get its target coordinates from your cursor. And the other to track the object it self and if the object either went into a restricted area or got too close it will shoot.

So I got some servos and the Phidgets 4 x servo controller board, bend and twisted some aluminum developed some software all in C# and there you go an automated gun turret!

I will update this soon with more pictures and some video of it actually working.

http://ultradev.kicks-ass.net/sg/GunDigS.jpg

Here is a very quick video of it in a bit of action:
http://www.youtube.com/watch?v=gaAHz8ajtKs

http://www.youtube.com/watch?v=gaAHz8ajtKs

This is a quick test to calibrate the firing servo there is no ammunition in the gun and the cylinder was used and was empty by shot two. But it worked so that part is okay. But I will update the fire mechanism with a solenoid soon.

http://www.youtube.com/watch?v=8Gs5ybqygk0


Hope you guys will like it. Gerhard

fish123456
12-06-2007, 07:52 AM
Did you use code that you wrote to handle the video or is that just another application? If it's some other app, how did you get the coordinates of the mouse for that window?

Cool project!

ekoos7
12-06-2007, 08:24 AM
Did you use code that you wrote to...

No I developed all the code myself. The turret has two modes manual and automatic. The video below is of the manual control I will update the video soon with a better example and maybe enlarge the cursor to give a better view. I will also get a video up of it in auto mode where it will track moving objects and shoot under certain conditions. The manual mode works quite simple to calculate the coordinates but the object tracker has quite a bit more math.

Calibration: First the turret has to be calibrated via a lazer pointer placed inside the mussel of the gun. The calibration application lets you move the turret up and down and left and right. The turret must also be level to get the best result for accuracy. You then have to place the lazer center top, bottom, left and right and each time select position then the coordinates will be saved in an Xml file. Thus your video camera and turret are then linked in their current respected positions. You will note on the video that is placed on the back camera bracket. So if the camera is fastened the turret can be moved and the calibration will stay valid. Also the width and height of the video must stay the same for the calibration to say valid.

Manual mode: The calibration file can then be imported into the manual turret control application (Obviously later all of this will be integrated into a single application). So now when the relative cursor position to the video is retrieved the coordinates for each servo can be calculated and send to the servo control. And then the gun targets the actual target in real time.

If you did have a video feed that for some reason could only be accessed via some type of proprietary software. It would be quite possible to still use the turret with the video feed as one could use a global mouse hook and plot the position of the video window onscreen and use similar calculations to control the turret.

Thanks

kdwyer
12-06-2007, 09:27 AM
I rated this post 'good', although I have my doubts about using technology for the suppresion of fellow primates. Yes, an excellent development of resources to solve a problem, but since meat-based analog devices (MBADs) already possess the skill to track and aim, and underemployment of said MBADs is also a problem, enabling a bot to take over disciplinary duties might just add to the problem of crime. The possibility of exchanging the pepper-balls with something more sinister is troubling.
I know, the genie is out of the bottle anyway. But I will not equip Otto the Robot with anything other than intelligence-gathering devices. That is more than enough to get him into trouble as it is.
That said, you did a really ingenious job with simple off-the-shelf components and raw materials. As a proof-of-concept, it is a tour de force. The flexibility of manual and auto modes makes this truly innovative and open ended. I just wish it would hurl insults instead.
And that those insults would hurt.
And they would be enough.

ekoos7
12-06-2007, 10:00 AM
I rated this post 'good', although I have....
Thanks for the post Kdwyer much appreciated and I do understand your dilemma in having the intellect to perceive the consequence that such a development might bring to any society. But I must stress that I do live in Africa and by that many rules in European societies no longer are valid as the majority do not necessary perceive reality in the same manner as say your average European. I am myself of European decent but I am South African by more than ten generations. Hence myself and my parents and theirs and so on all are literally made of Africa’s minerals and water. So by this alone I am as African as any other African.

So first you will note that this is a PepperBall gun and not lethal to most living things larger than rats. I did once shot a rat in the house with my PepperBall gun and it was a bit messy. I could post the image but it might upset sensitive viewers. And well the goal is not to replace any one outside of your house patrolling you lawn with a PepperBall gun. But more to give you that functionality to deter some individual or leopard from trying to enter your property unlawfully without casing serious injury to the intruder. Secondly the system does actually give a verbal warning before action is taken. And perhaps that would disencourage most, even leopards.

Alex
12-06-2007, 11:00 AM
Hey fish,

I wasn't too sure if this was what you were asking, but:

In C#, any event that passes the MouseEventArgs (http://msdn2.microsoft.com/en-us/library/system.windows.forms.mouseeventargs%28VS.80%29.asp x) object, has the X and Y Properties.

For example (if you already know what I'm talking about, feel free to skip over this), if you create a new C# Form Application, select the form that was just created, and click on the lightening bolt in the Properties Grid on the right, you'll see MouseClick, MouseDoubleClick, MouseDown, MouseMove & MouseUp. If you double click on any of these, Visual Studio will automatically create the event handler for you and take you to the code editor. In the code editor, you'll see that each one of these handlers have the MouseEventArgs object as the second parameter passed. Go into any of these handlers and type "e" (no quotes) and you'll see that each one has the Property X and Y, which will give you the coordinates of your mouse on the form:)

fish123456
12-06-2007, 11:17 AM
Hey fish,

I wasn't too sure if this was what you were asking, but:

In C#, any event that passes the MouseEventArgs (http://msdn2.microsoft.com/en-us/library/system.windows.forms.mouseeventargs%28VS.80%29.asp x) object, has the X and Y Properties.

For example (if you already know what I'm talking about, feel free to skip over this), if you create a new C# Form Application, select the form that was just created, and click on the lightening bolt in the Properties Grid on the right, you'll see MouseClick, MouseDoubleClick, MouseDown, MouseMove & MouseUp. If you double click on any of these, Visual Studio will automatically create the event handler for you and take you to the code editor. In the code editor, you'll see that each one of these handlers have the MouseEventArgs object as the second parameter passed. Go into any of these handlers and type "e" (no quotes) and you'll see that each one has the Property X and Y, which will give you the coordinates of your mouse on the form:)

Hi Alex. Yes, I'm aware of the MouseEventArgs and pointer handling in C#. I was more concerned with whether the application being used was an external application, like, say, a web cam application from Logitech. In that case, I was trying to understand whether or not he required a full screen window, because then the mouse coordinates would be equal to the window coordinates, thus simplifying the application, but decreasing its utility. It appears, from his response, that he understands how to hook into Windows DLLs to extract the global mouse coordinates, and one could simply use the location of the window to translate that into local coordinates.

Alex
12-06-2007, 11:56 AM
right on fish, just wanted to check.

ekoos7, I hear ya. If there was a freakin leopard in my yard, I'd want this pepper gun to shoot at it too!! It wouldn't kill the animal, but I think it'd scare it enough to get off my damn lawn!

We only have to worry about raccoons and possums getting in our garbage in the states;)

ekoos7
12-06-2007, 02:13 PM
A quick vid showing the tracking software:

http://www.youtube.com/watch?v=b4dfDHVUi-w

Here are some images to explain a bit more:

http://forums.trossenrobotics.com/gallery/files/1/4/6/6/Track01_thumb.jpg (http://forums.trossenrobotics.com/gallery/showimage.php?i=38&c=newimages)

http://forums.trossenrobotics.com/gallery/files/1/4/6/6/Track01_thumb.jpg (http://forums.trossenrobotics.com/gallery/showimage.php?i=38&c=newimages)

http://forums.trossenrobotics.com/gallery/files/1/4/6/6/Track03_thumb.png (http://forums.trossenrobotics.com/gallery/showimage.php?i=38&c=newimages)

fish123456
12-06-2007, 07:53 PM
That's a pretty neat piece of software. The video is a bit difficult to see, however. Maybe you could record your screen? Just a thought... I wrote some code in C many years ago that would do a histogram of a black and white image that averaged the "hotspots" (where the color was above a normalized value of 250) and drew a bounding box around the area. The idea was that the camera had no IR filter on it, so the IR emitter I was tracking always looked the brightest. Anyway, that was with an expensive Matrox framegrabber. Are you using a USB camera or a lipstick camera? How are you grabbing the video? Lastly, what sort of code are you using to display the video?

ekoos7
12-07-2007, 01:32 AM
That's a pretty neat piece of software....

You will note the images at the bottom are saved frames from the video. Here I used a standard webcam via USB I am using DirectShow to capture the video and then handling the frames. Then I draw the frames onto a picturebox’s graphics object. Many other tv cards and frame grabbers can also be used via DirectShow as long as they support WDM (Windows driver module). I am also using some other hardware that provides its own drivers and methods for accessing frames or video. One of the frame grabbers I used are from a company called Euresys (http://www.euresys.com) their frame grabbers support both WDM and their own Multicam which I prefer to use with their products. I use their frame grabbers for normal analogue cameras. And just before I forget it is obviously quite easy to use IP cameras as well.

fish123456
12-07-2007, 07:23 AM
You will note the images at the bottom are saved frames from the video. Here I used a standard webcam via USB I am using DirectShow to capture the video and then handling the frames. Then I draw the frames onto a picturebox’s graphics object. Many other tv cards and frame grabbers can also be used via DirectShow as long as they support WDM (Windows driver module). I am also using some other hardware that provides its own drivers and methods for accessing frames or video. One of the frame grabbers I used are from a company called Euresys (http://www.euresys.com) their frame grabbers support both WDM and their own Multicam which I prefer to use with their products. I use their frame grabbers for normal analogue cameras. And just before I forget it is obviously quite easy to use IP cameras as well.

Good to know. I have not interfaced with DirectShow at all in programming, but since it's a microsoft product, I would imagine there must be some good tutorials on how to set up capturing video to a C# Win Forms app via a WDM compatible source. Thanks for the heads up. I may go buy a webcam just for this purpose! I enjoyed doing the video capturing and then performing analysis on it at my old job.

Alex
12-07-2007, 09:03 AM
I just checked Microsoft's site and saw this:

http://msdn2.microsoft.com/en-us/library/ms783323.aspx

Microsoft didn't go into this much depth with DirectShow way back when I tried working with it (back in mid 05), so I'm guessing it's been added after the fact. It looks way less intimidating now with all of that documentation!

ekoos, do you have any "favorite" tutorials on DirectShow bookmarked?

Dankie;)

EDIT: Of course, after I went through all the headache of learning DirectShow when I did, it looks like shortly after there were a ton of helpful tutorials popping up. I just found another one (dated three months after I learned it, haha!):

http://www.codeproject.com/KB/directx/PrgmngDirectShowappsCS.aspx

ekoos7
12-07-2007, 02:13 PM
Well as DirectShow is not supported natively in C# only C++ and some higher level functions in VB it is never to straight forward to work with DirectShow in C# or J# or VB.Net. And it is still not easy to use in C++! The whole concept of DirectShow is very abstract and if you don’t work with it often it becomes Greek even for a geek. So to use DirectShow in C# you have to pinvoke all its functions. But there is hope this is the best thing for any C# developer if you are planning on using DirectShow in your .Net applications the open source project is called DirectShow.Net and here is the link, http://sourceforge.net/projects/directshownet/ (http://sourceforge.net/projects/directshownet/). There are many informative articles all over the web and on the CodeProject. Hope this will point you guys in the right direction. I do not really have a favorite article Alex but the DirectShow.Net library saved my bacon on many occasions.

ekoos7
12-07-2007, 02:16 PM
Alex your Afrikaans is quite good!

ekoos7
12-07-2007, 04:21 PM
This is directly captured from the tracking application.

http://www.youtube.com/watch?v=gKIlylI2z3E

fish123456
12-07-2007, 09:19 PM
Well you've inspired me and I picked up a Logitech Pro 9000 webcam this afternoon. I'm going to try to get some DirectShow stuff going over the next few weeks. Hopefully the camera is compatible. It can capture in HD, but I'm sure I will use a lower setting, as it's probably overkill. Maybe you will feel generous and share some code (assuming you aren't planning on selling it or something).

ekoos7
12-07-2007, 11:51 PM
Well you've inspired me and I picked up a Logitech Pro 9000 webcam this afternoon. I'm going to try to get some DirectShow stuff going over the next few weeks. Hopefully the camera is compatible. It can capture in HD, but I'm sure I will use a lower setting, as it's probably overkill. Maybe you will feel generous and share some code (assuming you aren't planning on selling it or something).

Okay I will post a sample with a download link soon. I don't know if your camera is supported. If you look in My Computer and you see your camera there and you can accessed it in Windows directly it should work fine.

Here is the link to some code, hope it helps.
http://ultradev.kicks-ass.net/download/DirectShowEKOOS7.zip

Alex
12-12-2007, 11:54 AM
Hey fish, keep us updated on your progress with the video cam and DirectShow! Probably create a new thread in the Software and Programming forums (http://forums.trossenrobotics.com/forumdisplay.php?f=21). Hopefully ekoos7 can give ya a hand, but I'll definitely try to help out the best I can:)