Single Camera Stereo System Using Prism and Mirrors

Stereo and 3D reconstruction are used by many applications such as object modeling, facial expression studies and human motion analysis. But synchronizing multiple high frame rate cameras require special hardware or sophisticated software solutions. In this paper, we propose a single camera stereo system by using a setup made of prism and mirrors. Our setup is cost effective, portable and can be calibrated similar to two camera stereo to obtain high quality 3D reconstruction.We demonstrate the application of the proposed system in dynamic 3D face expression capture, depth super-resolution in stereo video and general depth estimation.

[1]  Hung-Tat Tsui,et al.  3D reconstruction from a single view of an object and its image in a plane mirror , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[2]  Narendra Ahuja,et al.  Single camera stereo using planar parallel plate , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[3]  Jufu Feng,et al.  A Novel Facial Appearance Descriptor Based on Local Binary Pattern , 2008, 2008 Chinese Conference on Pattern Recognition.

[4]  Dimitris N. Metaxas,et al.  D - Clutter: Building object model library from unsupervised segmentation of cluttered scenes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Naokazu Yokoya,et al.  Generation of high-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidal mirrors , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[7]  François Cabestaing,et al.  SINGLE-CAMERA STEREOVISION SETUP with Orientable Optical Axes , 2006 .

[8]  Mohan M. Trivedi,et al.  Real-time target localization and tracking by N-ocular stereo , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[9]  Klaus-Dieter Kuhnert,et al.  Fusion of Stereo-Camera and PMD-Camera Data for Real-Time Suited Precise 3D Environment Reconstruction , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[11]  Keith J. Hanna,et al.  Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences , 2001, SIGGRAPH.

[12]  Radim Sára,et al.  Efficient Sampling of Disparity Space for Fast And Accurate Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Ruigang Yang,et al.  Spatial-Temporal Fusion for High Accuracy Depth Maps Using Dynamic MRFs , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Shree K. Nayar,et al.  Rectified catadioptric stereo sensors , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Shree K. Nayar,et al.  Stereo with mirrors , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[17]  Shree K. Nayar,et al.  Catadioptric Stereo Using Planar Mirrors , 2001, International Journal of Computer Vision.

[18]  Ruigang Yang,et al.  3D Tele-Collaboration Over Internet2 , 2002 .

[19]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Thomas Lu,et al.  A single-camera system captures high-resolution 3D images in one shot , 2006 .

[21]  Shree K. Nayar,et al.  Planar catadioptric stereo: geometry and calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[22]  Yong Xiao,et al.  A prism-based single-lens stereovision system: From trinocular to multi-ocular , 2007, Image Vis. Comput..

[23]  Shree K. Nayar,et al.  Real-Time Omnidirectional and Panoramic Stereo , 1998 .

[24]  Fabio Remondino,et al.  The potential of 3D techniques for cultural heritage object documentation , 2007, Electronic Imaging.

[25]  Seong-Whan Lee,et al.  View-independent human action recognition with Volume Motion Template on single stereo camera , 2010, Pattern Recognit. Lett..

[26]  Andrea Fusiello,et al.  Quasi-Euclidean uncalibrated epipolar rectification , 2008, 2008 19th International Conference on Pattern Recognition.

[27]  Jun Wang,et al.  3D Facial Expression Recognition Based on Primitive Surface Feature Distribution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, CVPR.

[29]  In-So Kweon,et al.  A biprism-stereo camera system , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[30]  Jim Austin,et al.  Three-dimensional face recognition using combinations of surface feature map subspace components , 2008, Image Vis. Comput..

[31]  Feng Li,et al.  A hybrid camera for motion deblurring and depth map super-resolution , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Luca Iocchi,et al.  Human Posture Tracking and Classification through Stereo Vision and 3D Model Matching , 2008, EURASIP J. Image Video Process..

[33]  Andrew Blake,et al.  High Precision Multi-touch Sensing on Surfaces using Overhead Cameras , 2007, Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07).

[34]  Yutaka Fukui,et al.  3-D Reconstruction Using Mirror Images Based on a Plane Symmetry Recovering Method , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Hiroyuki Miyamoto,et al.  A gesture recognition system using stereo vision and arm model fitting , 2007 .

[36]  Don Ray Murray,et al.  Using Real-Time Stereo Vision for Mobile Robot Navigation , 2000, Auton. Robots.