View Full Version : HUD creation with real time graphics overlay over webcam stream

04-24-2013, 09:55 AM

I've seen one example in a mechwar prequal video, however I wonder if anyone can point me in the right direction with how to achieve overlaying of data such as speed, heading and voltage over a live webcam stream within a browser. Getting the raw data is not an issue but rather how to mix and transpose that over a web stream in a browser.

Im hoping this may be achieved with use of HTML5/Javascript for example, if anyone has links to working examples, githubs or other that would be much appreciated.

Cheers :veryhappy:

04-24-2013, 11:35 AM
I assume i can create a series of HTML5 layers now that video is a HTML5 tag. And use javascript overlay.

Here's an example I found to get me started, if anyone can suggest better HUD examples using video stream rather than 'static' video source much appreciated:


04-24-2013, 01:21 PM
As far as I know, you just layer a transparent DIV on top of your VIDEO tag (or even EMBED/OBJECT tag) and that "should work." With modern, compositing browser renderers, it even "does work" in many cases.
Case in point: https://withme.com/
(https://withme.com/)We're using WebGL with HTML UI blended in, both on top and behind the 3D. HTML5 is great for interactive web apps!

That being said, my control UI is written in C++ using FLTK. I also don't use the TCP-based webcam stream, but instead simple UDP packets with MJPEG frames in them.

04-24-2013, 03:05 PM
Thanks for reminding me about WebGL, with looking at so much other stuff recently I had forgotten about that. Seems to be plenty of examples on the web to get me started, for example having some projected wireframes overlayed over video that can be animated according to robot movement.

I thought this was a nice introduction to WebGl:


04-25-2013, 02:09 AM
Indeed the web technologies are the current rave. Even new games are using webkit to draw gui elements on top of their 3D engine output. Another option would be Qt. They have several demos of doing all sorts of stuff on top of live video feed, including modifying the video stream itself.

04-25-2013, 02:35 AM
HD2 is running ROS which makes these kind of things really easy once you master the concepts. Even the dynamic walking corridor which is projected on the floor plane. In essence it's a two-liner in ROS which does all the heavy math behind the scenes and the end result is surprisingly accurate. That is given your robot model is precise enough. (Mastering ROS can take some time even for seasoned programmers, though.)

04-25-2013, 03:12 AM
Janis it was your HD2 bot that I was impressed both with the mechanics and the HUD, just superb, so thanks for your input.

As my intention for rover control and feedback will utilise web technologies it makes sense to utilise as much HTML5 and WebGL as possible and having seen several WebGL demos I think HUD implementation with WebGL should be perfectly reasonable and interesting. Thanks.