Geometric and Color Calibration of Multiview Panoramic Cameras for Life-Size 3D Immersive Video

In this paper we address calibration of camera arrays for life-size 3D video acquisition and display where mosaicking and multiviewpoint stereo are combined to provide an immersive experience which, by its size, resolution, and three-dimensionality, is meant to rival being there. This coupling of multiview and mosaicking requires integration of numerous synchronized video streams in a single presentation, aligned for both panoramic blending and epipolar rectification. The calibration framework we have developed extends the classical checkerboard approach through a modular multi-stage pipeline performing global optimization across intrinsics, extrinsics, panoramas, multiview epipolar alignments, and color correction. We demonstrate the methodology on several multiview camera arrays with various configurations aimed at mosaicking and epipolar light-field analysis. The results of this analysis have driven real-time life-sized panoramic 3D displays of captured events such as a concert, a fashion show, and sports activity.

[1]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[2]  Vincent Nozick,et al.  Multiple view image rectification , 2011, 2011 1st International Symposium on Access Spaces (ISAS).

[3]  Minh N. Do,et al.  Symmetric multi-view stereo reconstruction from planar camera arrays , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  A. Verri,et al.  A compact algorithm for rectification of stereo pairs , 2000 .

[5]  An Ping,et al.  A rectification algorithm for un-calibrated multi-view images based on SIFT features , 2010, 2010 International Conference on Audio, Language and Image Processing.

[6]  Peter Kauff,et al.  Multi-camera rectification using linearized trifocal tensor , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[7]  Niranjan Damera-Venkata,et al.  A Unified Paradigm For Scalable Multi-Projector Displays , 2007, IEEE Transactions on Visualization and Computer Graphics.

[8]  Yo-Sung Ho,et al.  An Efficient Rectification Algorithm for Multi-View Images in Parallel Camera Array , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[9]  Jiang Li,et al.  A real-time interactive multi-view video system , 2005, MULTIMEDIA '05.

[10]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[11]  Fatih Murat Porikli,et al.  Inter-camera color calibration by correlation model function , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[12]  M. Levoy,et al.  Automatic Color Calibration for Large Camera Arrays , 2005 .

[13]  Greg Welch,et al.  Ensuring color consistency across multiple cameras , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[14]  Zeyu Li,et al.  Capture considerations for multiview panoramic cameras , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  H. Harlyn Baker,et al.  A Multi-imager Camera for Variable-Definition Video (XDTV) , 2006, MRCS.