LiveActor
LiveActor combines an optical motion capture system with a stereo 3-D projection environment that allows users to experience a truly immersive virtual reality environment. The combination of these systems provides functionality beyond their individual, traditional uses. For example, motion capture is traditionally used for animation, game development, and human performance analysis, but with LiveActor, users will be able to directly interact with characters embedded within virtual worlds. Traditional VR experiences provided through CAVEs(TM) or head mounted displays usually offer limited simulations and interactions (i.e. head tracking and wand tracking), but the LiveActor system will allow for whole body tracking as well as GSR (Galvanic Skin Response) feedback. This allows for a more realistic experience and more data for analysis.
Traditional uses for stereo projection systems include architectural walk-throughs, entertainment, media walls, medical and product visualizations, and games. However, the experience is limited to mainly observation and navigation. Through whole body motion capture input, LiveActor affords a richer set of interactions with both characters and objects in the virtual environment.
LiveActor is an ideal environment for a variety of training purposes. Using our virtual characters, you can simulate any environment or circumstance, chart user reactions, and train the user to behave in new and more appropriate ways. Unlike training with actual humans, the virtual characters can be scripted to behave in variety of ways depending upon your needs. By creating a feedback system that provides motion capture data to the virtual characters, Live Actor allows for real-time reactions to your actions.
This LiveActor project has the overall objective of providing real-time, 3D, multi-modal interaction between live and virtual embodied agents. Crucial to this interaction is the computational modeling of agents with empirical attributes based on known testing instruments, physiological performance models, and psychosocial behaviors. Agent construction methodologies are under-studied, and we address this issue through both graphical and language user interfaces. We hypothesize that realistic agent behaviors are even more important than visual appearance in experiential veracity of a live simulation, and this can be tested in the LiveActor environment.
LiveActor was made possible through grants and support furnished by the National Science Foundation , University of Pennsylvania's School of Engineering and Applied Science , Ascension Technology Corporation , and EON Reality .Scientific contributions of this project include methodologies and systems to:
- Allow flexible construction of agent models and their human performance and psychosocial behaviors, using natural language for behavior rules, graphical interfaces for interactively entering data, and current psychosocial testing instruments for agent attributes.
- Observe and analyze multi-channel inputs from a live participant, including gesture and face motion qualities, speech and language, and physiological parameters.
- Use this real-time data to affect the choices, responses, and actions of virtual agents to the participant's state.
- Select virtual agent actions based on deep agent models, a parameterized action dictionary, and plot lines to create an ongoing, unfolding story.
- Build a real-time virtual environment presentation of the virtual agents, including appropriate and consistent body gestures, facial expression, and speech.
- Demonstrate the application of this architecture in several application domains.
- Promote the LiveActor architecture as a foundation for other research groups and applications
Press Release from Ascension Technology Corporation
Media:
For more information on LiveActor refer to our fact sheet ( pdf ).
ReActor Construction Images (June 2002)