Projector-Camera Systems for Immersive Training

Abstract : Real time computer graphics are limited in that they can only be displayed on projection screens and monitors. Monitors and projection screens cannot be used in live fire training or scenarios in which the displays could be physically damaged by trainees. To address this issue, we have developed projection systems using computer vision based color correction and image processing to project onto non-ideal surfaces such as painted walls, cinder blocks, and concrete floors. These projector-camera systems effectively paint the real world with digital light. Any surface can become an interactive projection screen allowing unprepared spaces to be transformed into an immersive environment. Virtual bullet holes, charring, and cracks can be added to real doors, walls, tables, chairs, cabinets, and windows. Distortion correction algorithms allow positioning of projection devices out of the field of view of trainees and their weapons. This paper describes our motivation and approach for implementing projector-camera systems for use within the FlatWorld wide area mixed reality system.

[1]  Aditi Majumder,et al.  Perceptual photometric seamlessness in projection-based tiled displays , 2005, TOGS.

[2]  Gordon Wetzstein,et al.  Radiometric compensation of global illumination effects with projector-camera systems , 2006, SIGGRAPH '06.

[3]  Anselmo Lastra,et al.  Life-sized projector-based dioramas , 2001, VRST '01.

[4]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[5]  Ramesh Raskar,et al.  Oblique projector rendering on planar surfaces for a tracked user , 1999, SIGGRAPH '99.

[6]  Shree K. Nayar,et al.  Making one object look like another: controlling appearance using a projector-camera system , 2004, CVPR 2004.

[7]  Taku Komura,et al.  Computing inverse kinematics with linear programming , 2005, VRST '05.

[8]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[9]  Ulrich Neumann,et al.  FlatWorld: Combining Hollywood Set-Design Techniques with VR , 2003, IEEE Computer Graphics and Applications.

[10]  Oliver Bimber,et al.  Compensating Indirect Scattering for Immersive and Semi-Immersive Projection Displays , 2006, IEEE Virtual Reality Conference (VR 2006).

[11]  Shree K. Nayar,et al.  A Projection System with Radiometric Compensation for Screen Imperfections , 2003 .

[12]  Ramesh Raskar,et al.  Immersive planar display using roughly aligned projectors , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[13]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[14]  Shree K. Nayar,et al.  A projector-camera system with real-time photometric adaptation for dynamic environments , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Anton Treskunov,et al.  The Flatworld Simulation Control Architecture (FSCA): A Framework for Scalable Immersive Visualization Systems , 2006 .

[16]  Oliver Bimber,et al.  Real-Time Adaptive Radiometric Compensation , 2006, IEEE Transactions on Visualization and Computer Graphics.

[17]  James M. Rehg,et al.  Shadow elimination and occluder light suppression for multi-projector displays , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Oliver Bimber,et al.  Multifocal projection: a multiprojector technique for increasing focal depth , 2006, IEEE Transactions on Visualization and Computer Graphics.

[19]  Oliver Bimber,et al.  Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces , 2007, J. Virtual Real. Broadcast..

[20]  Marc Levoy,et al.  Dual photography , 2005, SIGGRAPH 2005.