A framework for realistic 3D tele-immersion

Meeting, socializing and conversing online with a group of people using teleconferencing systems is still quite different from the experience of meeting face to face. We are abruptly aware that we are online and that the people we are engaging with are not in close proximity. Analogous to how talking on the telephone does not replicate the experience of talking in person. Several causes for these differences have been identified and we propose inspiring and innovative solutions to these hurdles in attempt to provide a more realistic, believable and engaging online conversational experience. We present the distributed and scalable framework REVERIE that provides a balanced mix of these solutions. Applications build on top of the REVERIE framework will be able to provide interactive, immersive, photo-realistic experiences to a multitude of users that for them will feel much more similar to having face to face meetings than the experience offered by conventional teleconferencing systems.

[1]  Klara Nahrstedt,et al.  Towards the understanding of human perceptual quality in tele-immersive shared activity , 2012, MMSys '12.

[2]  Thomas Stockhammer,et al.  Raptor Forward Error Correction Scheme for Object Delivery , 2007, RFC.

[3]  Russell L. Martin,et al.  Construction of 3-D Audio Systems: Background, Research, and General Requirements , 2008 .

[4]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[5]  Petros Daras,et al.  Real-Time, Full 3-D Reconstruction of Moving Foreground Objects From Multiple Consumer Depth Cameras , 2013, IEEE Transactions on Multimedia.

[6]  R. M. Aguilar,et al.  Control Application Using Fuzzy Logic: Design of a Fuzzy Temperature Controller , 2012 .

[7]  Titus B. Zaharia,et al.  TFAN: A low complexity 3D mesh compression algorithm , 2009, Comput. Animat. Virtual Worlds.

[8]  Henry Fuchs,et al.  Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[9]  Anna Hilsmann,et al.  Kinematic ICP for Articulated Template Fitting , 2012, VMV.

[10]  Wei Tsang Ooi,et al.  Modeling progressive mesh streaming: Does data dependency matter? , 2011, TOMCCAP.

[11]  Daniel Herrera C,et al.  Joint depth and color camera calibration with distortion correction. , 2012, IEEE transactions on pattern analysis and machine intelligence.

[12]  W. Yost.  Fundamentals of hearing: An introduction , 1977 .

[13]  Ruzena Bajcsy,et al.  High-Quality Visualization for Geographically Distributed 3-D Teleimmersive Applications , 2011, IEEE Transactions on Multimedia.

[14]  Jean Ponce,et al.  Carved Visual Hulls for Image-Based Modeling , 2006, International Journal of Computer Vision.

[15]  Hui Li,et al.  Middleware for streaming 3D progressive meshes over lossy networks , 2006, TOMCCAP.

[16]  Marcin Korytkowski,et al.  Merging Ensemble of Neuro-Fuzzy Systems , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[17]  Marco Gillies,et al.  Integrating autonomous behavior and user control for believable agents , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[18]  Françoise J. Prêteux,et al.  MPEG-4 Part 25: A graphics compression framework for XML-based scene graph formats , 2009, Signal Process. Image Commun..

[19]  Anna Hilsmann,et al.  Pose Space Image Based Rendering , 2013, Comput. Graph. Forum.

[20]  Michele Sanna,et al.  A 3D tele-immersion system based on live captured mesh geometry , 2013, MMSys.

[21]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[22]  Mohammad Fazle Azeem,et al.  Fuzzy Inference System - Theory and Applications , 2012 .

[23]  Mel Slater,et al.  Building Expression into Virtual Characters , 2006, Eurographics.

[24]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[25]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[26]  Stacy Marsella,et al.  A domain-independent framework for modeling emotion , 2004, Cognitive Systems Research.

[27]  Elliot B. Werner.  Manual of Visual Fields , 1991 .

[28]  Yücel Altunbasak,et al.  3TP: an application-layer protocol for streaming 3-D graphics , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[29]  Yücel Altunbasak,et al.  3TP: an application-Layer protocol for streaming 3-D models , 2005, IEEE Transactions on Multimedia.