A Framework for Augmented Reality Using Non-Central Catadioptric Cameras

In this article we propose a framework for the application of augmented reality to non-central catadioptric imaging devices. Considering a virtual object in the world with known 3D coordinates, the goal is to project this object into the image of a non-central catadioptric camera. We propose a solution to this problem which allows us to project texturized objects to the image in realtime, up to 20 fps: projection of 3D segments to the image, occlusions, illumination and shading. To the best of our knowledge this is the first time that this problem is addressed (all state-of-the-art methods are derived for central camera systems). In our experiments, we used a non-central catadioptric camera formed with a perspective camera and a spherical mirror. To test the proposed approach, we define a cube with texturized faces where each of the main steps of the framework is evaluated. To conclude, we used the proposed framework to project to the image the Stanford "bunny" object.

[1]  S. Nayar,et al.  Catadioptric Image Formation , 1997 .

[2]  Dave Shreiner.  OpenGL programming guide , 2013 .

[3]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[4]  Yuichi Taguchi,et al.  Beyond Alhazen's problem: Analytical projection model for non-central catadioptric cameras with quadric mirrors , 2011, CVPR 2011.

[5]  HENRI GOURAUD,et al.  Continuous Shading of Curved Surfaces , 1971, IEEE Transactions on Computers.

[6]  Burkhard Wünsche,et al.  Mixed reality simulation for mobile robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[7]  Shree K. Nayar,et al.  A Theory of Single-Viewpoint Catadioptric Image Formation , 1999, International Journal of Computer Vision.

[8]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[9]  Helder Araújo,et al.  Pose estimation for non-central cameras using planes , 2014, 2014 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[10]  Wen-Yan Chang,et al.  On pose recovery for generalized visual sensors , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[12]  Shree K. Nayar,et al.  Caustics of catadioptric cameras , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Veronica Teichrieb,et al.  Real Time Ray Tracing for Augmented Reality , 2012, 2012 14th Symposium on Virtual and Augmented Reality.

[14]  Katsushi Ikeuchi,et al.  Acquiring a Radiance Distribution to Superimpose Virtual Objects onto Real Scene , 2001, MVA.

[15]  Shree K. Nayar,et al.  A general imaging model and a method for finding its parameters , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Amit K. Agrawal,et al.  Single Image Calibration of Multi-axial Imaging Systems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Nuno Goncalves.  Noncentral catadioptric systems with quadric mirrors : geometry and calibration , 2008 .

[18]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[19]  Axel Pinz,et al.  Globally Optimal O(n) Solution to the PnP Problem for General Camera Models , 2008, BMVC.

[20]  Abhilash Pandya,et al.  Improved Telemanipulator Navigation During Display-Control Misalignments Using Augmented Reality Cues , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[21]  Vishvjit S. Nalwa,et al.  A True Omni-Directional Viewer † , 1996 .

[22]  Shree K. Nayar,et al.  A perspective on distortions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Helder Araújo,et al.  Planar pose estimation for general cameras using known 3D lines , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Loren C. Carpenter,et al.  The A -buffer, an antialiased hidden surface method , 1984, SIGGRAPH.

[25]  A. Fournier,et al.  Common Illumination between Real and Computer Generated Scenes , 1992 .

[26]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[27]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[28]  Tomás Pajdla,et al.  Autocalibration & 3D reconstruction with non-central catadioptric cameras , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[29]  Arthur Appel,et al.  Some techniques for shading machine renderings of solids , 1968, AFIPS Spring Joint Computing Conference.

[30]  Helder Araújo,et al.  Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems , 2013, Comput. Vis. Image Underst..

[31]  Nuno Gonçalves.  On the reflection point where light reflects to a known destination on quadratic surfaces. , 2010, Optics letters.

[32]  M. Carter.  Computer graphics: Principles and practice , 1997 .