View Full Version : [Contest Entry] Tactile Aural Simulated Kinetics (TASK) Remotely Oeprated Vehicle (ROV) Control

02-27-2008, 09:05 AM
Here is a description of a project I am developing for my PhD Dissertation. The project is titled Tactile Aural Simulated Kinetics (TASK), which will be used to increase the Situational Awareness (SA) of UAV/ROV/UGV operators teleoperating the vehicles through increased telepresence using simulation techniques. The following describes the project in further detail:

Current Identified Problem
There are an increasing number of Unmanned UAV mishaps and accidents. To address these accidents, the USAF has identified a a need for a "thorough human factors evaulation to develop UAV-focused Human-Computer interface design rules and principles." This need for the development of new design rules and principles comes fromt he utilization of technology and designs that provide limited visual and aural representations of the UAV operating environment. Missing from the current generation of pilot interfaces are familiar forms of "Feedback" (ie. the Look, sound, and feel normally experienced in the operation of an aircraft).

Proposed Solution to Problem
The purpose of the TASK system is to mitigate the Factors that reduce Situational Awareness, by incorporating simulated motion/force feedback, aural/sound cues, and increased visual perception. The incorporation of these concepts is expected to increase an operators performance and abilities.

The TASK system was designed to replicate the operating environment of an ROV using simulation
technology and techniques. The use of these techniques can be transferred/translated to other types of tele-operated vehicles such as UAVs, Unmanned Ground Vehicles (UGV)s, and Automated Underwater Vehicles (AUV)s.

Force Feedback Component
Force Feedback represents the first aspect of Tactile simulation of the ROV operating environment. It provides the user with the Feel of what the vehicle is experiencing through the use of commanded joystick position changes and vibration effects. The movement effects occur in the Joystick X and Y axis, while the vibration effect occur in the handgrip (along both axis).

Simulated Aural Feedback
This system utilizes simulated sounds that replicate actions and events in the ROV operating environment (vehicle state) and the creation of vibrations that model movement of the ROV through its environment (Tactile/Seat Shaker). The aural feedback portrays the vehicle state through the use of simulated engine revving, engine failure, crash/impacts, wind, and subsystem activation (i.e. turret sound for camera movement). In addition, any Caution/Warning Advisory lights on the GUI that have associated beeping or steady tones are portrayed through this system. Additionally, this system represents the second form of tactile feedback as it portrays the vehicle's movement/orientation state in specific environments through the use of custom created bass-heavy sound cues. These cues provide slight motion events based on the oscillation of the pitch/roll/yaw accelermeters and the measurement of impacts.

Increased Visual Display System
This system increases the visual display of a GCS display by implementing a 32" LCD monitor with 16:9 aspect ratio. Three (3) LCD screens display vehicle telemetry, state, and control data to the user via a custom created GUI. In addition a head tracking visor vision component has been incorporated to enhance the user's incorporation into the simulation. As the user moves their head, the camera mimics the movement providing an identical visual perception.

The TASK Prototype System replicates the operating environment of a custom built ROV, the TASK Information Gathering Environmental Rover (TIGER), using simulation technology and techniques. The TASK Prototype System is comprised of two main subsystems:

TASK Ground Control Station(GCS)
The user interface that simulates the Look, Feel and Sound of the ROV operating environment. This subsystem provides control instructions and data presentation for the ROV. (See First Attached Image Below).

TIGER Vehicle
The TIGER tele-operated vehicle captures visual, aural, and motion data and relays it back to the TASK GCS. This subsystem accepts commands from the TASK GCS. (See Second Attached Image Below).

Hardware/Software Components
The TASK Prototype was built using the following Hardware/Software:
Ground Control Station (GCS)
32" LCD Monitor
17" LCD Monitors
Windows XP PC (with 4 video outputs)
Force Feedback Joystick with Throttle
Head Tracker
Video Visor
Surround Sound Speaker System
Bluetooth USB Adapter
Serial rs232 Wireless Modem
Battery Charger

Remotely Operated Vehicle (ROV)
Wireless Camera with Audio
Bluetooth RS232 Adapter
Serial rs232 Wireless Modem
Acroname Brainstem Microcontroller
RC Truck Chassis (with Engine/Servos/Speed Controller)
Camera Pan/Tilt Assembly (with Servos)
7.2v Batteries (4)
Temperature Sensor (Analog)
Voltage Sensor (Analog)
Accelerometer (XY Analog)
Infrared (IR) Sensor (Analog)
Two (2) Hall Effect Sensor (Digital)
5v Laser (Digital -output)

Windows XP Operating System (OS)
Visual Studios 2005 (Pro Edition)
.Net Framework (v2.0)
Bespoke Software Brainstem .Net Library

Additional Materials
A Series of images have been attached to this posting to further describe the project. Two of these are zip files that contain multiple .PNG files that represent slides from a Power Point File that describe the project in greater detail. Additionally, two images have been directly included to provide clarity to the description above.