Axial-cones: modeling spherical catadioptric cameras for wide-angle light field rendering

Catadioptric imaging systems are commonly used for wide-angle imaging, but lead to multi-perspective images which do not allow algorithms designed for perspective cameras to be used. Efficient use of such systems requires accurate geometric ray modeling as well as fast algorithms. We present accurate geometric modeling of the multi-perspective photo captured with a spherical catadioptric imaging system using axial-cone cameras: multiple perspective cameras lying on an axis each with a different viewpoint and a different cone of rays. This modeling avoids geometric approximations and allows several algorithms developed for perspective cameras to be applied to multi-perspective catadioptric cameras. We demonstrate axial-cone modeling in the context of rendering wide-angle light fields, captured using a spherical mirror array. We present several applications such as spherical distortion correction, digital refocusing for artistic depth of field effects in wide-angle scenes, and wide-angle dense depth estimation. Our GPU implementation using axial-cone modeling achieves up to three orders of magnitude speed up over ray tracing for these applications.

[1]  Rahul Swaminathan.  Focus in Catadioptric Imaging Systems , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Peter F. Sturm,et al.  Theory and Calibration for Axial Cameras , 2006, ACCV.

[3]  Shree K. Nayar,et al.  A general imaging model and a method for finding its parameters , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  David Salesin,et al.  Spatio-angular resolution tradeoffs in integral photography , 2006, EGSR '06.

[5]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[6]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Shree K. Nayar,et al.  Multiview radial catadioptric imaging for scene capture , 2006, SIGGRAPH 2006.

[8]  Yuichi Taguchi,et al.  Analytical Forward Projection for Axial Non-central Dioptric and Catadioptric Cameras , 2010, ECCV.

[9]  Yasushi YAGI,et al.  Calibration and Performance Evaluation of Omnidirectional Sensor with Compound Spherical Mirrors , 2005 .

[10]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[11]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[12]  Peter F. Sturm,et al.  A Generic Concept for Camera Calibration , 2004, ECCV.

[13]  Ruigang Yang,et al.  Real‐Time Consensus‐Based Scene Reconstruction Using Commodity Graphics Hardware † , 2003, Comput. Graph. Forum.

[14]  Donald S. Fussell,et al.  Uniformly Sampled Light Fields , 1998, Rendering Techniques.

[15]  Tomás Pajdla,et al.  Autocalibration & 3D reconstruction with non-central catadioptric cameras , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[16]  Leonard McMillan,et al.  General Linear Cameras , 2004, ECCV.

[17]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  Shree K. Nayar,et al.  Caustics of catadioptric cameras , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  Andrew Gardner,et al.  Capturing and Rendering with Incident Light Fields , 2003, Rendering Techniques.

[20]  Ashok Veeraraghavan,et al.  Axial light field for curved mirrors: Reflect your perspective, widen your view , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[23]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[24]  Marc Levoy,et al.  Synthetic aperture confocal imaging , 2004, SIGGRAPH 2004.

[25]  Peter F. Sturm,et al.  Calibration of Cameras with Radially Symmetric Distortion , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Kanbara Masayuki,et al.  3D Scene Reconstruction from Reflection Images in a Spherical Mirror , 2006 .

[27]  Yuanyuan Ding,et al.  Multiperspective stereo matching and volumetric reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[28]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[29]  Shree K. Nayar,et al.  A Theory of Single-Viewpoint Catadioptric Image Formation , 1999, International Journal of Computer Vision.

[30]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[31]  Shree K. Nayar,et al.  Stereo with mirrors , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  Douglas Lanman,et al.  Spherical Catadioptric Arrays: Construction, Multi-View Geometry, and Calibration , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[33]  Wolfgang Heidrich,et al.  High-quality Shading and Lighting for Hardware-accelerated Rendering , 1999 .