Sig Lab Model

Summer Research

Our usual CG summer research program in the SIGLab will not be held this year due to COVID and a lack of onsite supervisory personnel. We hope to be back in 2022.

One of the reasons students attend Penn is for access to faculty and research opportunities, yet few undergraduates take advantage of this opportunity. While classroom experience is essential, so is the opportunity to create new knowledge while examining the unknown. The Digital Media Design program is closely related to the Center for Human Modeling and Simulation and the ViDi Center for Digital Visualization. Research projects are undertaken by heterogeneous teams of graduate and undergraduate students and visitors. Undergraduates who contribute in a substantial way become co-authors in publications. In the past few years, summer researchers in HMS and ViDi have had papers published in notable computer graphics academic conferences and journals.

From year to year the internship topics vary depending on funding, research needs, and student interests. The projects have overall faculty guidance, but students are expected to learn new software systems, do extensive programming, contribute to archival materials (software, documentation and written papers), and orally present their work to others. Posters and participation in department-wide research demonstrations is strongly encouraged.

Many of the participating undergraduate students from previous summers are recognized through authorship in published papers.

  • The UG students include 10 women (Emiliya Al Yafei, Mabel Ogiriki, Elissa Wolf, Jennie Shapira, Fannie Liu, Teresa Fan, Nicole Nelson, Samantha Raja, Rebecca Fletcher, Nancy Yu, Desiree Velázquez-Rios) and 18 men (Josh Nadel, Youssef Victor, Charles Wang, Kai Ninomiya, Francisco Garcia, Nathan Marshak, Yu Wang, Max Gilbert, Daniel Garcia, Matthew Jones, Robert Mead, Dan Markowitz, Ian Perera, Matthew Croop, Jeremy Cytryn, Jonathan McCaffrey, Matt Kuruc, Vijay Nair).
  • Collectively they account for 25 publications: 6 in journals, 18 in conference proceedings, and 1 conference poster. UG students are first authors in 5 of these. Elissa Wolf was first author in a Presence, MIT Press journal paper.
  • Francisco Garcia co-authored 5 papers, Jennie Shapira co-authored 3, and Kai Ninomaya, 2.
  • Francisco Garcia, Jennie Shapira, Max Gilbert, and Nathan Marshak contributed to the book 'Virtual Crowds: Steps Toward Behavioral Realism' (Morgan & Claypool, 2015).
  • 7 students, Fannie Liu, Samantha Raja, Kai Ninomiya, Francisco Garcia, Yu Wang, Max Gilbert, and Ian Perera, continued with computer science graduate studies, and of these, 3 students, Francisco Garcia, Yu Wang, and Ian Perera, embarked on PhD degrees in Computer Science.
  • Summer 2019 undergraduate students Nicholas Magarino, Saranya Sampath, Thy Tran, Enoch Solano, Sang Lee and Shenqi Hu won the third place of the CIS 2019 Summer Undergraduate Research for their work E.A.T.S. Extraction of Abdominal Topology and Simulation.
  • Summer 2018 undergraduate student Yi Gu won the first place of the CIS 2018 Summer Undergraduate Research for her work Robotics Control with Physics-based Fluid Simulation in Virtual Environment.
  • Summer 2018 undergraduate student Jacob Snipes won the second place of the CIS 2018 Summer Undergraduate Research for his work H.O.G.W.A.R.T.S (Handy Offline Grid-based Wind Advection Reverse Time Solver).
  • Summer 2018 undergraduate students Liangzhen Fei and Chen Li won the third place of the CIS 2018 Summer Undergraduate Research for their work Frame Interpolation of Smoke Simulation with Neural Network.
  • Summer 2017 undergraduate students Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu won the honorable mention of the CIS 2017 Summer Undergraduate Research for their work Enriching and accelerating the Material Point Method.

2020 Research Topics

Norman Badler

Project: SPACES -Spatialized Performance And Ceremonial Event Simulations

Keywords: Computer graphics and animation, human figures, archaeological site reconstruction.

Although computerized crowd simulations exist, most effort has been directed toward low-level navigation, collision avoidance, and trajectory realism. “Higher-level” organization is left to user discretion, artistic decisions, or creative goals. Crowds are often behaviorally homogeneous with only vague overall purpose. SPACES will center on processional environments: what activities occur where, about how long they last, what objects agents carry/use/play, sound and motion coordination, and interpersonal interactions. We will develop a user interface to control such parameters. SPACES will use the UnReal game engine for graphic visual experiences. SPACES will allow a user to embed herself as a crowd participant and active performer. SPACES requires highly realistic graphic environments, and responsive behaviors in the other characters to cement the sense of cultural presence: “the feeling of being and making sense there together.” What better way to experience the ethos of a bygone culture than by being embedded in its public ceremonial practices? The exemplar we will be using is the Inca site of Pachacamac (Peru). Extensive in architecture and clearly designed for large-scale events, Pachacamac serves as our test-bed for immersive exploration of hypothetical cultural experiences.

Requirements: Computer programming, graphics, and 3D modeling experience necessary.

Project: Sign Language Movement, Energetics, and Evolution Analysis

This NSF REU (Research Experience for Undergraduates) will involve learning and using a software toolkit to compute physical and biomechanically appropriate measures of human movement during sign language performances.These measures will be used to assess the evolution of signing performance in several available datasets, in particular, to see if and how energetics and muscular workloads affect sign evolution.The suite of tools has been written by Dr. Aline Normoyle, whose PhD Dissertation is a primary resource, but will also involve reading and understanding the background biomechanics and physics of human movement.The software computes a number of measures such as torques, accelerations, and work.We do not know which measures are operative in sign language evolution, so much of the analytics are oriented to finding meaningful measures in the given datasets. You will take the joint position data from analyzed video sequences, format the data for the analytic tools, run the analyses, and visualize the results with suitable graphs and diagrams.You will also assist Drs. Badler and Normoyle with writing up the results of this study for publication or submission to a scholarly meeting.

Chenfanfu Jiang

Project: Building and Improving Physics-based Simulation Infrustracture

There exists strong value of augmenting algorithms for use with high-performance platforms, especially multi-GPU and distributed memory architecture, for scalable simulations. Large-scale simulations are becoming more demanding and popular than ever. There are many efforts invested in algorithms that scale with supercomputers or cloud servers. The biggest advantage of HPC over cloud computing is that supercomputer nodes are usually more tightly coupled through faster and more stable connections (e.g. infiniband). For physics-based simulations, the frequent communications among its subdomains heavily relies on these interconnections. On the other hand, there has been a trend towards the deployment of general-purpose GPU (GPGPU) as an accelerator in modern high-end HPC systems. This internship position focuses on building and improving large scale, massively parallel physics-based simulation infrustracture on super computers, multi-GPU platforms, and distributed servers.

Requirements: Candidate should be familiar with modern C++ programming. Experience with HPC such as single node multi-threading, CUDA, OpenCL, or MPI is strongly recommended.

Project: Numerical Modeling of Soft Interactions for Robots

The goal of this project is to bridge the gap between the soft robots currently fabricated in GRASP lab with the fast simulation tools developed in SIG lab. Soft and semi-rigid robots are a new and active area in robotics research because they promise safer, more robust interactions with the environment and with humans. However, soft robot design suffers from limitations in computational models of these interactions. Existing approaches to simulation are slow and computationally expensive, making it cumbersome to iterate over designs in a virtual environment. The field would benefit greatly from a fast, accurate, and preferably differentiable simulation for soft body interactions.
At the same time, researchers in computer graphics have excelled at creating real-time and high-resolution simulations of soft bodies ranging from foams to fabrics to hair to fluids like water and mud. However, the focus of this work has been on creating visually reasonable animations, rather than conducting quantitative comparisons to real physical systems. As a result, the physical accuracy of these simulations has never been evaluated.
This project will explore the applicability of simulations originating from the computer graphics community to real-world physical robots. The work will require close collaboration between Cynthia Sung (mechanical engineering) and Chenfanfu Jiang (computer science) to merge ideas from both groups. Cynthia Sung’s expertise is in rapid fabrication and robot design, and she will contribute experimental instrumentation and robotics intuition necessary to understand the robotic implications of the work. Chenfanfu Jiang’s expertise is in material point methods for high performance simulation, and he will contribute algorithmic and physics intuition needed to pinpoint gaps between the experiments and reality. Given the disciplinary language barriers and traditional separation of the two fields, it would be unlikely for either group to be able to explore this project without a consolidated and concentrated effort.

Position 1: focusing on Physics-based Simulation:
The student will work with existing physics-based simulation code. At the beginning of the summer, some time will be necessary for the student to understand the code base and how to use the system. The student will coordinate with robotics team members, who are planning the physical experimental setup, to create the same experimental conditions within the simulation and output virtual results. As the experiments progress, the students will collectively work to understand the difference between the simulation results and the physical results produced, and this student will work to integrate those insights into the simulation code base.
Needed background: The student will need to be proficient in C++.
Learning opportunities: The student will learn state of the art computer graphics techniques and will also see how well those techniques generalize to real, physical systems.

Position 2: focusing on 3D Modeling:
This student will be responsible for creating 3D models for the soft robots and their test environments. These models will be used both in the simulations and in the experimental tests. The student will work directly with robotics students in printing out the 3D models verifying mechanical properties (e.g., elasticity), which need to be inputted into the simulation. As new insights about the simulation and physical systems are understood, the student will refine the 3D models to take experimental constraints into account.
Needed background: The student will need to be familiar with 3D modeling tools such as Blender or Maya. These tools are commonly used by students in digital media design or others interested in computer animations.
Learning opportunities: The student will learn how to create complex 3D models and will gain intuition on when 3D models are physically (not just virtually) feasible.

Steve Lane

Project: Interactive Authoring of Augmented Reality Task Content

The goal of this project is to develop an innovative in situ authoring system that transforms the way augmented reality content is created for many training and education tasks. This will be accomplished by capturing the movements, actions and verbal descriptions of an instructor, trainer, designer, etc. as they actually perform a task. The system will then automatically segment the captured task data into goal-directed subtasks and adapt it for use in a “Helping Hands” application to automatically prompt and guide users while they perform the task themselves, taking into account the specifics of the task environment (physical layout, location of parts and objects, etc.).

Project: Augmented Reality Enhancement of Medical Simulation and Training Applications

The goal of this project is to develop a prototype Augmented Reality (AR) surgical navigation system for orthopedic applications (insertion of pins, rods, plates, etc.) using the Microsoft Hololens and Magic Leap devices. Knowledge of the Unity game engine required.

Project: Deep Learning of high-resolution 3D volumetric views given-low resolution 2D image inputs

This project will investigate the configuration and training of Convolutional Neural Network models that allow low resolution camera inputs to generate both camera position/orientation data and high-resolution volumetric views as neural network outputs.

Project: Development of Augmented Reality Applications for Large Screen Displays

The goal of this project is to create a compelling Unity demo of a user holding/wearing an android-based smartphone interacting with content shown on a large screen display. Both a handheld mode and a head-mounted display mode (using the GearVR HMD) will be developed using the Google ARcore SDK and the Vuforia7 computer vision plugin. The project involves estimating the position and orientation of the smartphone/HMD with respect to the display screen (and optionally the user’s head with respect to the handheld smartphone) in order to create new multi-user game experiences and crowd-sourced augmented reality application content.

Background Image: Jeremy Newlin (DMD 2013, CGGT 2014)