Researchers dress virtual avatars with digitally captured clothing

4D Movies Capture People in Clothing, Creating Realistic Virtual Try-on

Researchers at the Max Planck Institute for Intelligent Systems (MPI-IS) have developed technology to digitally capture clothing on moving people, turn it into a 3-D digital form, and dress virtual avatars with it. This new technology makes virtual clothing try-on practical.

Ever wonder how you might look in a beautiful dress worn by Michelle Obama? Imagine trying it on and seeing how the fabric moves and fits before buying. This technology could make it possible.

Traditional virtual clothing try-on involves getting the 2-D clothing pattern from the manufacturer, sizing it to an individual, and simulating how the clothing drapes on the body. The new technique replaces garment simulation with garment capture. Capturing and transferring existing garments greatly simplifies the process of virtual try-on.

“Our approach is to scan a person wearing the garment, separate the clothing from the person, and then rendering it on top of a new person,” says Dr. Gerard Pons-Moll, research scientist at MPI-IS and principal investigator of the project. “This process captures all the detail present in real clothing, including how it moves, which is hard to replicate with simulation,” says Pons-Moll. The ClothCap (ClothCapture) results exceed the realism of existing approaches (Figure 1).

ClothCap uses 4-D movies of people recorded with a unique 4-D high-resolution scanner. The system uses 66 cameras and projectors to illuminate the person being scanned. “This scanner captures every wrinkle of clothing at high resolution. It is like having 66 eyes looking at a person from every possible angle,” says Michael Black, director at MPI-IS. “This allows us to study humans in motion like never before.”

ClothCap computes the body shape and motion under clothing while separating and tracking the garments on the body as it moves. “The software turns the captured scans into separate meshes corresponding to the clothing and the body,” says Dr. Sergi Pujades, postdoctoral researcher at MPI and one of the main authors of this work.