Computational stereo camera system with programmable control loop

Stereoscopic 3D has gained significant importance in the entertainment industry. However, production of high quality stereoscopic content is still a challenging art that requires mastering the complex interplay of human perception, 3D display properties, and artistic intent. In this paper, we present a computational stereo camera system that closes the control loop from capture and analysis to automatic adjustment of physical parameters. Intuitive interaction metaphors are developed that replace cumbersome handling of rig parameters using a touch screen interface with 3D visualization. Our system is designed to make stereoscopic 3D production as easy, intuitive, flexible, and reliable as possible. Captured signals are processed and analyzed in real-time on a stream processor. Stereoscopy and user settings define programmable control functionalities, which are executed in real-time on a control processor. Computational power and flexibility is enabled by a dedicated software and hardware architecture. We show that even traditionally difficult shots can be easily captured using our system.

[1]  Henrique S. Malvar,et al.  High-quality linear interpolation for demosaicing of Bayer-patterned color images , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  Marc Levoy,et al.  The Frankencamera: an experimental platform for computational photography , 2010, ACM Trans. Graph..

[3]  Takashi Shibata,et al.  Development of software for editing of stereoscopic 3D movies , 2002, IS&T/SPIE Electronic Imaging.

[4]  O. Braddick,et al.  Seeing in Depth , 2008 .

[5]  M. Gross,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, ACM Trans. Graph..

[6]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[7]  Alexander A. Sawchuk,et al.  Disparity manipulation for stereo images and video , 2008, Electronic Imaging.

[8]  Paul F. Whelan,et al.  Projective rectification from the fundamental matrix , 2005, Image Vis. Comput..

[9]  Peter Eisert,et al.  The Stereoscopic Analyzer — An image-based assistance tool for stereo shooting and 3D production , 2010, 2010 IEEE International Conference on Image Processing.

[10]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[11]  Masaki Emoto,et al.  Spatial distortion prediction system for stereoscopic images , 2006, J. Electronic Imaging.

[12]  Ming-Hsuan Yang,et al.  Visual tracking with online Multiple Instance Learning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[14]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[15]  Geng Sun,et al.  Evaluating methods for controlling depth perception in stereoscopic cinematography , 2009, Electronic Imaging.

[16]  Clive S. Fraser,et al.  Zoom-dependent camera calibration in digital close-range photogrammetry , 2006 .

[17]  Horst Bischof,et al.  Hierarchical Disparity Estimation with Programmable 3D Hardware , 2004 .

[18]  Peter Eisert,et al.  REAL-TIME ANALYSIS AND CORRECTION OF STEREOSCOPIC HDTV SEQUENCES , 2009 .

[19]  Graham R Jones,et al.  Controlling perceived depth in stereoscopic images , 2001, IS&T/SPIE Electronic Imaging.

[20]  Bernard Mendiburu,et al.  3D Movie Making: Stereoscopic Digital Cinema from Script to Screen , 2009 .

[21]  Robert Mueller,et al.  A systematized WYSIWYG pipeline for digital stereoscopic 3D filmmaking , 2008, Electronic Imaging.

[22]  P. Eisert,et al.  Joint Estimation of Epipolar Geometry and Rectification Parameters using Point Correspondences for Stereoscopic TV Sequences , 2010 .

[23]  Michael F. Cohen,et al.  A Viewer-Centric Editor for 3D Movies , 2011, IEEE Computer Graphics and Applications.

[24]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..

[25]  W.E. Snyder,et al.  Color image processing pipeline , 2005, IEEE Signal Processing Magazine.