3D VIDEO PROCESSING FOR IMMERSIVE 3D VIDEOCONFERENCING

Dr. Oliver Schreer
Fraunhofer Institut for Telecommunications, Heinrich-Hertz-Institut
Einsteinufer 37, 10587 Berlin, Germany

Abstract

This tutorial gives a detailed overview on the current state of immersive videoconferencing and derives the existing drawbacks for natural and convincing immersive video communication. A new concept for 3D immersive videoconferencing is presented which targets the existing challenges beyond the state of the art. The geometrical system design of the display configuration and multicamera setup fulfilling multi-user and multi-party requirements is discussed. Based on this example application of immersive 3D videoconferencing a complete real-time 3D video processing chain is introduced. The complete set of modules ranging from multi-view capture, multi-view 3D analysis, encoding, transmission, rendering and display are considered. The specific challenges of such a system in terms of real-time capability, low delay and low latency are tackled.

Outline

Due to the advances in computer vision, hardware development and display technologies, 3D video processing has achieved a lot of attention in the last few years. Many research activities have been started on European level in order to investigate 3D video processing for different domains such as 3DTV, 3D cinema, 3D on mobile devices and 3D videoconferencing. Even the commercial video conferencing sellers recognized that immersive experience for realistic multi-user and multi party videoconferencing is a demanding feature. Nevertheless the introduction of 3D is still in its infancy in this domain.

The tutorial gives a detailed overview on the current state of immersive videoconferencing and derives the existing drawbacks for natural and convincing immersive video communication. A new concept for 3D immersive videoconferencing is presented which targets the existing challenges beyond the state of the art. A geometrical system design of the display configuration and multicamera setup fulfilling multi-user and multi-party requirements is introduced. Based on the application example of immersive 3D videoconferencing, all the modules of a real-time, low delay and low latency 3D video processing chain are discussed in a step by step approach, The presentation starts with multi-view capturing and 3D video analysis, continues with multi-view encoding and transmission and ends up with decoding, rendering and multi-perspective 3D display.

Background and Potential Target Audience

The tutorial targets researchers and scientist from academia and industry working in the area of 3D video processing. Due to the application oriented nature of the tutorial, no specific knowledge is required except from some fundamentals on computer vision and 3D video processing in particular.

Biography

Oliver Schreer is working as scientific project manager of the Immersive Media & 3D Video- Group in the Image Processing Department. He received his Dr.-Ing. degree in electrical engineering from the Technical University of Berlin in 1999. Since August 1998, he is with the IP department, where he is engaged in research for 3D analysis, novel view synthesis, vision-based HCI, real-time 3D video conferencing systems and immersive TV applications. He was involved in different European research projects like ACTS PANORAMA and IST VIRTUE. Since autumn 2001, Oliver Schreer is Adjunct Professor at the Faculty of Electrical Engineering and Computer Science, Technical University Berlin and since autumn 2006, he has the position of an Associated Professor at the same faculty. Since February 2007, Oliver Schreer is coordinating the FP6 project RUSHES. From January 2008 onwards, Oliver Schreer is involved in the European FP7 research project 3DPresence.