Depth camera indoor mapping for 3D virtual radio play

By combining game engines with indoor mapping, it is possible to create interactive virtual environments that represent the real world. In this paper a virtual version of an audio installation in a historic building is produced, where the user freely explores the building and experiences a set of audio clips, creating a virtual radio play. A depth camera indoor mapping system, Matterport, captured a set of staged interiors. The aim was to evaluate the quality and usability of the indoor models and to demonstrate applying them in 3D application development. The quality

[1]  C. Ogleby,et al.  From Rubble to Virtual Reality: Photogrammetry and the Virtual World of Ancient Ayutthaya, Thailand , 1999 .

[2]  Frank Boochs,et al.  3D‐Sutra—Interactive Analysis Tool for a Web Atlas of Scanned Sutra Inscriptions in China , 2011 .

[3]  Alessandro De Gloria,et al.  A serious game model for cultural heritage , 2012, JOCCH.

[4]  Howard Rheingold,et al.  Virtual Reality , 1991 .

[5]  Petri Rönnholm,et al.  MOTORIZED PANORAMIC CAMERA MOUNT – CALIBRATION AND IMAGE CAPTURE , 2016 .

[6]  Young Min Kim,et al.  Design and calibration of a multi-view TOF sensor fusion system , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[7]  Zhenfeng Shao,et al.  3D RECONSTRUCTION OF CULTURAL TOURISM ATTRACTIONS FROM INDOOR TO OUTDOOR BASED ON PORTABLE FOUR-CAMERA STEREO VISION SYSTEM , 2015 .

[8]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  J. McGann,et al.  Associative learning and sensory neuroplasticity: how does it happen and what is it good for? , 2015, Learning & memory.

[10]  Antero Kukko,et al.  Photogrammetric application of spherical imaging , 2004 .

[11]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[12]  Timo Ojala,et al.  Props: 3D-game-like mediator for improvisational storytelling , 2014, Entertain. Comput..

[13]  Maren Lindstaedt,et al.  COMPARATIVE GEOMETRICAL INVESTIGATIONS OF HAND-HELD SCANNING SYSTEMS , 2016 .

[14]  Hannu Hyyppä,et al.  AUTOMATED IMAGE-BASED RECONSTRUCTION OF BUILDING INTERIORS - A CASE STUDY , 2014 .

[15]  Tomás Pajdla,et al.  Multi-view reconstruction preserving weakly-supported surfaces , 2011, CVPR 2011.

[16]  Maria Roussou.  The Interplay between Form, Story, and History : The Use of Narrative in Cultural and Educational Virtual Reality , 2001, International Conference on Virtual Storytelling.

[17]  Shamus P. Smith,et al.  Computer game engines for developing first-person virtual environments , 2008, Virtual Reality.

[18]  Frederick P. Brooks.  What's Real About Virtual Reality? , 1999, IEEE Computer Graphics and Applications.

[19]  Halim Setan,et al.  3D DATA ACQUISITION FOR INDOOR ASSETS USING TERRESTRIAL LASER SCANNING , 2013 .

[20]  Fotis Liarokapis,et al.  Developing serious games for cultural heritage: a state-of-the-art review , 2010, Virtual Reality.

[21]  Shamus P. Smith,et al.  Extending Building Information Models into Game Engines , 2014, IE.

[22]  Jianxiong Xiao,et al.  Reconstructing the World’s Museums , 2012, International Journal of Computer Vision.

[23]  Timo Ojala,et al.  VirtualOulu: collaborative, immersive and extensible 3D city model on the web , 2016, Web3D.

[24]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[25]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[26]  Tony Parisi,et al.  Learning Virtual Reality: Developing Immersive Experiences and Applications for Desktop, Web, and Mobile , 2015 .

[27]  R. Kaijaluoto,et al.  Precise indoor localization for mobile laser scanner , 2015 .

[28]  Bo Liu,et al.  3 D RECONSTRUCTION OF CULTURAL TOURISM ATTRACTIONS FROM INDOOR TO OUTDOOR BASED ON PORTABLE FOUR-CAMERA STEREO VISION SYSTEM , 2015 .

[29]  Leena Arhippainen,et al.  User experiences of 3D virtual learning environment , 2011, MindTrek.

[30]  R. Reulke,et al.  Remote Sensing and Spatial Information Sciences , 2005 .

[31]  Daniel Boland,et al.  Lost in the rift , 2015, XRDS.

[32]  Tomasz Mazuryk,et al.  Virtual Reality History , Applications , Technology and Future , 1999 .

[33]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[34]  Juha Hyyppä,et al.  Photorealistic Building Reconstruction from Mobile Laser Scanning Data , 2011, Remote. Sens..

[35]  Adrienne Grêt-Regamey,et al.  Developing a GIS-Based Visual-Acoustic 3D Simulation for Wind Farm Assessment , 2014, ISPRS Int. J. Geo Inf..

[36]  Alessandro Rizzi,et al.  3D ICHNOLOGY—RECOVERING DIGITAL 3D MODELS OF DINOSAUR FOOTPRINTS , 2010 .

[37]  Jan Boehm,et al.  MOBILE LASER SCANNING FOR INDOOR MODELLING , 2013 .

[38]  Didier Stricker,et al.  Structure from Motion using full spherical panoramic cameras , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[39]  José Luis Lerma,et al.  Photogrammetry and augmented reality for cultural heritage applications , 2009 .

[40]  Didier Stricker,et al.  A Generalized Structure from Motion Framework for Central Projection Cameras , 2015, VISIGRAPP.

[41]  P A Hancock,et al.  The perception of spatial layout in real and virtual worlds. , 1997, Ergonomics.

[42]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[43]  Sisi Zlatanova,et al.  FIRST EXPERIMENTS WITH THE TANGO TABLET FOR INDOOR SCANNING , 2016 .

[44]  Thomas J. Madden,et al.  A Closer Look at Classical Conditioning , 1985 .

[45]  Harri Kaartinen,et al.  Localization of mobile laser scanner using classical mechanics , 2015 .

[46]  Pedro Arias,et al.  Terrestrial laser scanning for geometry extraction and change monitoring of rubble mound breakwaters , 2014 .

[47]  Philip Greenwood,et al.  Using game engine technology to create real-time interactive environments to assist in planning and visual assessment for infrastructure , 2009 .

[48]  Walid Y. Thabet,et al.  Utilizing a 3D game engine to develop a virtual design review system , 2011, J. Inf. Technol. Constr..

[49]  Mathieu Brédif,et al.  AN ACCURACY ASSESSMENT OF AUTOMATED PHOTOGRAMMETRIC TECHNIQUES FOR 3D MODELING OF COMPLEX INTERIORS , 2012 .

[50]  Dieter Fritsch,et al.  VISUALISATION USING GAME ENGINES , 2004 .

[51]  Phillip M. Harter,et al.  Simulation for Team Training and Assessment: Case Studies of Online Training with Virtual Worlds , 2008, World Journal of Surgery.

[52]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[53]  Juha Hyyppä,et al.  Intelligent Open Data 3D Maps in a Collaborative Virtual World , 2015, ISPRS Int. J. Geo Inf..

[54]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[55]  C. Fraser,et al.  Image-based modelling of architectural sites: The iWitness approach , 2007 .

[56]  F. Fassi,et al.  Surveying and modelling the main spire of Milan Cathedral using multiple data sources , 2011 .

[57]  Adrian Herwig,et al.  Game Engines: Tools for Landscape Visualization and Planning? , 2004 .

[58]  Giuseppe Riva,et al.  The Role of Immersion and Narrative in Mediated Presence: The Virtual Hospital Experience , 2011, Cyberpsychology Behav. Soc. Netw..