Virtual 3D video instead of Avatar

New technology creates holographic-like human images in 3D

Being in the thick of things with VR glasses: A new camera system represents people virtually and realistically virtually. © Fraunhofer HHI
Read out

Holography with VR glasses: Researchers have developed a technology that makes images of real people seem like a holographic in the middle of the room. Equipped with VR glasses, you can go around this image and get everywhere a realistic three-dimensional image. This is achieved with special camera technology and special software.

So far, it has not been possible to represent the image of a real person as a virtual 3D image. Although there is already the possibility avatars through a VR glasses to look at, but these are often artificial. That's because they are not based on a person's real video. As a result, details are missing, such as wrinkles in the shirt that make a picture appear real to humans.

Another possibility was to record the video of a person in the VR data glasses. As a result, humans are depicted realistically but only in two dimensions. So you can not go around it. This also works artificially when you move in a virtual reality.

Stereo with 20 cameras

However, Oliver Schreer and his colleagues from the Fraunhofer Institute for Telecommunications in Berlin have now developed a technology that combines realism and 3D impression. For the system "3D Human Body Reconstruction" they use a new stereo camera system. Like the human being with his two eyes, she takes the person with two lenses and thus from a slightly different angle. The combination of these two pictures results in the three-dimensional impression.

With these cameras, the researchers took the person from all directions, so that we can also circumnavigate them in the virtual world. A camera is not enough for that. "We currently use more than 20 stereo cameras to image a human being, " says Schreer. As each camera captures only a part of the person, Schreer claims that the challenge is to put the frames together in such a way as to create a realistic overall picture. display

More depth information through algorithm

In addition, the researchers have also developed a new algorithm. This calculation method can very quickly extract depth information from the camera images. The program then calculates the 3D shape of the recorded person from this information. A computer then generates the human virtual image that we later experience through the VR glasses.

The combination of stereo camera and the new algorithm now makes it possible to view a real image of a person in the virtual world. Since the researchers use video recordings as a basis, the human being in the virtual world is very real. Details in the picture, such as wrinkles in the clothing, the surface structure and the facial features, can thus also be seen by users in the virtual world.

In addition, the movements are natural: When developing the algorithms, we made sure that they work very efficiently and thus fast, so that the movements of the conversation partners very quickly into a dynamic model Schreer says.

Immerse yourself in the movie

For the future, however, the team has even more plans: Our goal is that a realistic image of a human being can interact directly with the virtual world in the future - for example, by intervening virtual objects Says Schreer. Researchers can imagine other areas of application in the area of ​​virtual video conferencing or in television: Spectators could then walk directly into a feature film scene with VR glasses.

What's more, Schreer can also imagine using the system for film production. "Film producers could use it to simplify the movement of actors in film scenes, " said the researcher. For other methods, it would be very costly for the graphic artist to represent the facial expressions in real terms. On the other hand, he would like to dissolve and portray the movement and emotions more finely. Whether the team will provide their system of camera and software in the future as a service or whether they license it to production companies, they do not yet know.

(Fraunhofer-Gesellschaft, 07.09.2016 - HDI)