Task acquisition and instruction presentation using augmented reality

 

The new NASA mandate calls for missions of unprecedented remoteness and duration while human capabilities remain relatively fixed. The RIVET team (Penn, Orbitec, and NASA JSC) is to develop computer based integrated training and instruction tools that are visually intuitive, easily authorable, and adaptable to user skill level and context. The goal is to improve reliability in executing instructions by a crew with limited training, especially for critical tasks in nominal and emergency situations.

The RIVET prototype is an extensible (scalable) embedded training system using graphical and speech interfaces. RIVET is designed for multiple application environments (e.g., operations, maintenance, medical) and provides user interaction tools appropriate to both training and operational environments (e.g., portable systems or hands-free voice actuation). Currently RIVET implements a flexible instruction delivery interface for Advanced Cardiac Life Support (ACLS).

Current work is focused on authoring crew instructions from expert examples using multi-modal information streams (motion capture, handshape capture, video, and audio).

Primary funding: NASA “ RIVET: Rapid Interactive Visualization for Extensible Training ” (N. Badler, PI).

•  N. Badler, J. Allbeck, A. Megahed and M. Whitmore. “RIVET: Rapid Interactive Visualization for Extensible Training.” Abstract, Proc. Habitation, Orlando , FL , 2006.

Complexity, customization, and packaging of military platforms and systems increase maintenance difficulty at the same time as the available pool of skilled technical personnel may be shrinking. In this environment maintenance training, technical order presentation, and flight-line operational practice may need to adopt “just-in-time” procedural aids. Moreover, the realities of real-world maintenance may not permit the hardware indulgences and rigid controls of laboratory settings for visualization and training systems, and at the same time the actual activities of maintainers will challenge requirements for portable or wearable devices. This project investigates technologies that may be used in the maintenance of Air Force equipment.

Primary funding: Air Force “AVIS-MS Advanced Visual and Instruction Systems for Maintenance Support” (N. Badler, PI).

•  N. Badler, C. Erignac and Y. Liu. “Virtual humans for validating maintenance procedures.” Communications of the ACM , July 2002, pp. 57-63.