HMS : Research

Parameterized Action Representation

Natural Language often describes actions at a high level, leaving out many of the details that have to be specified for animation. The PAR bridges the gap between natural language and animations. PAR gives a description of an action. The PAR has to specify the agent of the action as well as any relevant objects and information about path, location, manner, and purpose for a particular action. There are linguistic constraints on how this information can be conveyed by the language; agents and objects tend to be verb arguments, path is often a prepositional phrase, and manner and purpose might be in additional clauses. A parser and translator can map the components of an instruction into the parameters or variables of the PAR, which is then linked directly to PaT-Nets (finite state machines) and ultimately to the specified movement generators.

We use the example “Walk to the door and turn the handle slowly,” to illustrate the function of the PAR. Whether or not the PAR system processes this instruction, there is nothing explicit in the linguistic representation about grasping the handle or which direction it will have to be turned, yet this information is necessary to the action's actual visible performance. The PAR has to include information about applicability, preparatory, and termination conditions in order to fill in these gaps. It also has to be parameterized, because other details of the action depend on the PAR's participants, including agents, objects, and other attributes. The representation of the “handle” object lists the actions that the object can perform and what state changes they cause. The number of steps it will take to get to the door depends on the agent's size and starting location.

Primary funding: Completed projects sponsored by Air Force Research Laboratory, NASA, and NSF. On going project sponsored by Army SUBTLE MURI. and Air Force.

Another resource for information on verbs is the Unified Verb Index, which merges links and webpages from a few different natural language processing projects. WordNet contains valuable information on both verbs and nouns. WordsEye is an interesting system that uses natural language to construct virtual scenes.

PAR Related Publications

* most recommended reading

J. Allbeck and N. Badler. "Automated Analysis of Human Factors Requirements." Proceedings of the 2006 Digital Human Modeling for Design and Engineering Conference , July 2006, Document 2001-01-2366, 7 pages.

N. Badler, J. Allbeck, A. Megahed, M. Whitmore. "RIVET: Rapid Interactive Visualization for Extensible Training." Habitation 2006, Orlando, FL, February 2006.

N. Badler, J. Allbeck, S. Lee, R.Rabbitz, T. Broderick, and K. Mulkern. "New Behavioral Paradigms for Virtual Human Models." Proceedings of SAE International Digital Human Modeling for Design and Engineering , 2005.

N. Badler, C. Erignac, and Y. Liu. "Virtual Humans for Validating Maintenance Procedures." Communications of the ACM , Vol. 45, Issue 7, pages 56-63, July 2002.

J. Allbeck, K. Kipper, C. Adams, W. Schuler, E. Zoubanova, N. Badler, M. Palmer, and A. Joshi. "ACUMEN: Amplifying Control and Understanding of Multiple Entities." Proceedings of the International Conference on Autonomous Agents, pages 191-198.

N. Badler, J. Allbeck, L. Zhao, and M. Byun. "Representing and Parameterizing Agent Behaviors." Proceedings of Computer Animation 2002 (CA 2002) , IEEE Computer Society, 2002, Geneva, Switzerland, pages 133-143.

J. Allbeck and N, Badler. "Embodied Autonomous Agents." Handbook of Virtual Environments, K. Stanney (Ed.), Lawrence Erlbaum Associates, 2002, pages 313-332.

L. Zhao, K. Kipper, W. Schuler and N. Badler. "A Machine Translation System from English to American Sign Language." Conference AMTA-2000: Envisioning Machine Translation in the Information Future, Mexico, 2000.

* R. Bindiganavale, W. Schuler, J. Allbeck, N. Badler, A. Joshi, and M. Palmer. "Dynamically Altering Agent Behaviors Using Natural Language Instructions." Proceedings of the 4th International Conference on Autonomous Agents 2000 (AGENTS 2000) , pages 293-300.

Y. Xu and N. Badler. "Algorithms for Generating Motion Trajectories Described by Prepositions." Proceedings of the Computer Animation 2000 Conference , May 2000, pages 30-35.

N. Badler, R. Bindiganavale, J. Allbeck, W. Schuler, L. Zhao, M. Palmer. "Parameterized Action Representation for Virtual Human Agents." In Embodied Conversational Agents , J. Cassell, J. Sullivan, S. Prevost and E. Churchill (Eds.), MIT Press, 2000, pages 256-284.

K. Kipper and M. Palmer. "Representation of Actions as an Interlingua." Workshop on Applied Interlinguas: Practical Applications of Interlingual Approaches to NLP , in conjunction with ANLP-NAACL 2000. 6 pages, Seattle, WA, April 2000.

N. Badler, R. Bindiganavale, J. Allbeck, W. Schuler, L. Zhao, S. Lee, H. Shin, and M. Palmer. "Parameterized Action Representation and Natural Language Instructions for Dynamic Behavior Modification of Embodied Agents." AAAI Spring Symposium , 2000.

N. Badler, M. Palmer and R. Bindiganavale. "Animation Control for Real-Time Virtual Humans." Communications of the ACM , 42(8), August 1999, pages 64-73.

N. Badler, R. Bindiganavale, J. Bourne, J. Allbeck, J. Shi and M. Palmer. "Real Time Virtual Humans." Proceedings of the International Conference on Digital Media Futures , British Computer Society, Bradford, UK, April 1999.

J. Shi, T. J. Smith, J. Granieri, and N. Badler. "Smart Avatars in JackMOO." IEEE Proceedings of Virtual Reality 1999 , March 1999, pages 156-163.

N. Badler, R. Bindiganavale, J. Bourne, M. Palmer, J. Shi and W. Schuler. "A Parameterized Action Representation for Virtual Human Agents." Workshop on Embodied Conversational Characters, Lake Tahoe, California, October 1998.