Attention, eye, and head motion modeling

 

Our objective is to develop a computational model of multiple influences on eye gaze behavior for an embodied agent in a dynamic environment. An embodied agent should possess human attention attributes so that its eyes and resultant body movements convey appropriate attending behaviors. Suppressed or inappropriate eye movements can damage the effectiveness of an embodied agent. Visual attention models may be the key to leading animated agents out of the “uncanny valley” where increasing visual appearance accuracy results in a ghoulish appearance when animated. Visual perceptual capability, human-like imperfect cognitive ability, as well as some aspects of internal cognitive state influence attention and eye gaze behaviors. Visual perceptual capability starts with an early vision process that exhibits changes in visual sensitivity such as night vision and flash blindness under changing scene illumination. Visual attention directs the limited gaze resource to resolve visual competition with the cooperation of top-down attention and conspicuous bottom-up guidance.

We are building an architecture to capture and model many of these attention and eye gaze features. The attention architecture first operates at the image level to automatically generate fixation sequences, and then extends to dynamic image sequences. This model has four factors -- conspicuity, mental workload, expectation and capacity -- that determine successful attention allocation. The attention model replicates many aspects of normal human function as well as some of its imperfect behaviors, such as inattentional blindness. Gaze role is not only modeled based on the cognitive task, but also driven by peripheral events, some with abrupt onset. Experiments with an agent-human collaborative system consisting of virtual agents and real subjects will help evaluate the plausibility of these models. This research is thus to develop a computational attention model, quantify the inattention factors, add them to a general eye gaze model, apply the completed model to animated agents, and empirically evaluate the resulting naturalness and effectiveness of the agents. This comprehensive model should be portable across applications and should have demonstrable impact on improving human realism of game characters and virtual training agents.

Primary funding: ONR VIRTE “Virtual Technologies and Environments” (N. Badler, PI) [completed] and N SF American Sign Language Natural Language Generation and Machine Translation (N. Badler and M. Marcus, co-PIs).

•  E. Gu, C. Stocker and N. Badler. “Do you see what eyes see? Implementing inattentional blindness.” Intelligent Virtual Agents (IVA) 2005, LNCR 3661, Spring-Verlag, pp. 178-190. [Ms. Stocker was an undergrad when this was written; she presented the paper in Kos, Greece .]

•  S. Lee, J. Badler and N. Badler. “Eyes alive.” ACM Transactions on Graphics - Special Issue, Proceedings of SIGGRAPH 2002, San Antonio, TX, pp. 637-644.

S. Chopra-Khullar and N. Badler. “Where to look? Automating attending behaviors of virtual human characters.” Autonomous Agents and Multi-agent Systems 4(1/2), 2001, pp. 9-23.