A Framework for Augmented Reality using Non-Central Catadioptric Cameras

This paper addresses the problem of augmented reality on images acquired from non-central catadioptric systems. We propose a solution which allows the projection of textured objects to images of these type of systems and, depending on the complexity of the objects, can run up to 20 fps, using a 1328×1048 image resolution. The main contributions are related with the image formation of the non-central catadioptric cameras: projection of the 3D segments onto the image of non-central catadioptric cameras; occlusions; and illumination/shading. To validate the proposed solution, we used a non-central catadioptric camera formed with a perspective camera and a spherical mirror. Also, to test the robustness of the proposed method, we used a regular object (a parallelepiped) and three well known irregular objects in computer graphics: “bunny”, “happy buddha” and “dragon”, from Stanford database.

[1]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[2]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[3]  Wen-Yan Chang,et al.  On pose recovery for generalized visual sensors , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shree K. Nayar,et al.  Caustics of catadioptric cameras , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Yuichi Taguchi,et al.  Beyond Alhazen's problem: Analytical projection model for non-central catadioptric cameras with quadric mirrors , 2011, CVPR 2011.

[6]  Axel Pinz,et al.  Globally Optimal O(n) Solution to the PnP Problem for General Camera Models , 2008, BMVC.

[7]  Abhilash Pandya,et al.  Improved Telemanipulator Navigation During Display-Control Misalignments Using Augmented Reality Cues , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[8]  Vishvjit S. Nalwa,et al.  A True Omni-Directional Viewer † , 1996 .

[9]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[10]  Tomás Pajdla,et al.  Autocalibration & 3D reconstruction with non-central catadioptric cameras , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  S. Nayar,et al.  Catadioptric Image Formation , 1997 .

[12]  Lois F. Lunin.  Imaging Systems: What, When, How. , 1992 .

[13]  Helder Araújo,et al.  Pose estimation for non-central cameras using planes , 2014, 2014 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[14]  Shree K. Nayar,et al.  A Theory of Single-Viewpoint Catadioptric Image Formation , 1999, International Journal of Computer Vision.

[15]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[16]  Arthur Appel,et al.  Some techniques for shading machine renderings of solids , 1968, AFIPS Spring Joint Computing Conference.

[17]  Helder Araújo,et al.  Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems , 2013, Comput. Vis. Image Underst..

[18]  Nuno Gonçalves.  On the reflection point where light reflects to a known destination on quadratic surfaces. , 2010, Optics letters.

[19]  H. Gouraud.  Continuous Shading of Curved Surfaces , 1971, IEEE Transactions on Computers.

[20]  Steven K. Feiner,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 1990 .

[21]  Tomas Pajdla,et al.  Autocalibration & 3D reconstruction with non-central catadioptric cameras , 2004, CVPR 2004.

[22]  Shree K. Nayar,et al.  A perspective on distortions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Veronica Teichrieb,et al.  Real Time Ray Tracing for Augmented Reality , 2012, 2012 14th Symposium on Virtual and Augmented Reality.

[24]  Helder Araújo,et al.  Planar pose estimation for general cameras using known 3D lines , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  A. Fournier,et al.  Common Illumination between Real and Computer Generated Scenes , 1992 .

[26]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[27]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[28]  Loren C. Carpenter,et al.  The A -buffer, an antialiased hidden surface method , 1984, SIGGRAPH.

[29]  Shree K. Nayar,et al.  A general imaging model and a method for finding its parameters , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[32]  Amit K. Agrawal,et al.  Single Image Calibration of Multi-axial Imaging Systems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Nuno Goncalves.  Noncentral catadioptric systems with quadric mirrors : geometry and calibration , 2008 .

[34]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[35]  Burkhard Wünsche,et al.  Mixed reality simulation for mobile robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[36]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.