To main content

Spherical stereo for virtual presence

The goal of the project has been to develop the next generation spherical 3D camera for video and sound. Playback in VR goggles will provide the user with an immersive realistic presence.

Contact person

Screen capture of a rendered stereo (VR) view of a recorded 360 3D video. Photo: Petter Risholm/SINTEF

An ecosystem for recording and playback of 360 3D (3D60) video and sound was developed in this project. The primary playback platform is VR, where the goal is to provide the user with a realistic 3D audio-visual experience. Contrary to state-of-the-art 360 video systems which only allow for head rotations, 3D60 provides full freedom for head movements (translation and rotation) while preserving the visual 3D effect of the video. In combination with the 3D sound-field, this provides an unparalleled realistic immersive experience – it feels like you are there!

The research in this project was necessary because there do not exist any good solutions for joint recording and realistic display of 3D audio and video in VR goggles which preserves the depth perception without artifacts.

A 3D60 prototype and an initial demo showcasing the potential of the technology was developed in the last reporting period. The 3D60 camera is based on multiple RGB-D cameras which are configured to have minimal overlapping field of views. The automatic post-processing software stitches the RGB-D video from the respective cameras onto a uniform 360 3D representation of the space around the camera. A real-time VR player has been developed for the Oculus Rift head set.

Key Factors

Project duration

2015 - 2018

Financing

The Research Council of Norway

Cooperation Partners

MakingView AS, SquareHead Technology AS, Smart Sensor Systems, and SINTEF Digital

Project Type

IPN