The Virtual Technologies and Environments (VIRTE) project from the Office of Naval Research is developing a virtual reality based training system for Marine Corps fire teams in close quarters battle (CQB). HMS has the task of creating smarter, real-time, reactive computer generated forces (CGFs). As in many games, current virtual opponents simply replay stored motions (key-framed or motion captured). This means their actions are not tailored to context often resulting in large motion datasets or unacceptable behaviors, such as limbs passing through walls or stereotyped (un)emotional reactions. Virtual opponents also often lack appropriate responses to environmental stimuli, such as low illumination, obstacles, light flashes, other agents, and explosions. HMS is beginning work on this project by investigating extending the capabilities of the underlying game engine, Gamebryo, by adding inverse kinematics. Other areas of focus include synthetic vision and attention models, which will allow smart agents with specific cognitive tasks and internal states to generate appropriate eye, face, head, and body behaviors in dynamically changing environments. Current effort has focused on dark adaptation and light adaptation.

Dark adaptation: Think about walking indoors from outside on a bright sunny day. At first you can hardly see anything. After being inside for a while, you adapt to the low light, and easily see your way around. This is called dark adaptation. The actual time course takes more than ten minutes for dark adaptation. We have scaled time to seconds for simulation.

Light adaptation: Intensely bright light or flashes of bright light can cause flash blindness, or dazzle. (That's why oncoming bright automobile headlights are both annoying and dangerous.)  The human eye reacts quickly to such an abrupt increase in illumination, but not instantaneously. Recovery from temporary loss of visual function takes about three to eight seconds.

Our inverse kinematics, IKAN, is used to keep the soldier's hands on the weapon as the weapon is aimed.

These demonstrations show the effects of visual adaptation on agent behavior. The bottom left window in each panel shows the gradually changing visual image of the agent under the conditions of dark adaptation and flash blindness. Once a threshold of visibility is reached, the essential perceptual capability of the agent is restored.  The agent sees the opponent and raises the gun in reaction.  Thus perception of the opponent is not instantaneous, but rather is highly dependent on the lighting conditions and the agent's synthetic vision.

Previous work on visual attention: Where to Look? Automating Visual Attending Behaviors of Characters