From 2D Images to 3D Tangible Models: Autostereoscopic and Haptic Visualization of Martian Rocks in Virtual Environments

A planetary rover acquires a large collection of images while exploring its surrounding environment. For example, 2D stereo images of the Martian surface captured by the lander and the Sojourner rover during the Mars Pathfinder mission in 1997 were transmitted to Earth for scientific analysis and navigation planning. Due to the limited memory and computational power of the Sojourner rover, most of the images were captured by the lander and then transmitted to Earth directly for processing. If these images were merged together at the rover site to reconstruct a 3D representation of the rover's environment using its on-board resources, more information could potentially be transmitted to Earth in a compact manner. However, construction of a 3D model from multiple views is a highly challenging task to accomplish even for the new generation rovers (Spirit and Opportunity) running on the Mars surface at the time this article was written. Moreover, low transmission rates and communication intervals between Earth and Mars make the transmission of any data more difficult. We propose a robust and computationally efficient method for progressive transmission of multi-resolution 3D models of Martian rocks and soil reconstructed from a series of stereo images. For visualization of these models on Earth, we have developed a new multimodal visualization setup that integrates vision and touch. Our scheme for 3D reconstruction of Martian rocks from 2D images for visualization on Earth involves four main steps: a) acquisition of scans: depth maps are generated from stereo images, b) integration of scans: the scans are correctly positioned and oriented with respect to each other and fused to construct a 3D volumetric representation of the rocks using an octree, c) transmission: the volumetric data is encoded and progressively transmitted to Earth, d) visualization: a surface model is reconstructed from the transmitted data on Earth and displayed to a user through a new autostereoscopic visualization table and a haptic device for providing touch feedback. To test the practical utility of our approach, we first captured a sequence of stereo images of a rock surface from various viewpoints in JPL MarsYard using a mobile cart and then performed a series of 3D reconstruction experiments. In this paper, we discuss the steps of our reconstruction process, our multimodal visualization system, and the tradeoffs that have to be made to transmit multiresolution 3D models to Earth in an efficient manner under the constraints of limited computational resources, low transmission rate, and communication interval between Earth and Mars.

[1]  Thomas S. Huang,et al.  A survey of construction and manipulation of octrees , 1988, Comput. Vis. Graph. Image Process..

[2]  Ken Perlin,et al.  An autostereoscopic display , 2000, SIGGRAPH.

[3]  Gregg Podnar,et al.  Geometry of binocular imaging , 1994, Electronic Imaging.

[4]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[5]  Kari Pulli,et al.  Multiview registration for large data sets , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[6]  HoppeHugues,et al.  Surface Reconstruction from Unorganized Points , 1992 .

[7]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using orthonormal matrices , 1988 .

[9]  M. Halle.  Autostereoscopic displays and computer graphics , 1997, COMG.

[10]  Touradj Ebrahimi,et al.  MESH: measuring errors between surfaces using the Hausdorff distance , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[11]  David John Trayner,et al.  Autostereoscopic display using holographic optical elements , 1996, Electronic Imaging.

[12]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[13]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[14]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[15]  Cagatay Basdogan,et al.  Haptic Rendering in Virtual Environments , 2002 .

[16]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[17]  James Arvo,et al.  Graphics Gems II , 1994 .

[18]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[19]  Holly E. Rushmeier,et al.  The 3D Model Acquisition Pipeline , 2002, Comput. Graph. Forum.

[20]  David R. Anderson,et al.  Development of a stereoscopic haptic acoustic real-time computer (SHARC) , 1998, Electronic Imaging.

[21]  HalleMichael.  Autostereoscopic displays and computer graphics , 1997 .

[22]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[23]  Michael Halle,et al.  Autostereoscopic displays and computer graphics , 1997, SIGGRAPH Courses.

[24]  Maarten Vergauwen,et al.  A stereo-vision system for support of planetary surface exploration , 2003, Machine Vision and Applications.

[25]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[26]  Cagatay Basdogan,et al.  Haptics in virtual environments: taxonomy, research status, and challenges , 1997, Comput. Graph..

[27]  L. Hodges,et al.  Rotation algorithm artifacts in stereoscopic images , 1990 .

[28]  Warren Robinett,et al.  The Visual Display Transformation for Virtual Reality , 1995, Presence: Teleoperators & Virtual Environments.

[29]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[30]  E LorensenWilliam,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987 .

[31]  Cagatay Basdogan,et al.  Efficient Point-Based Rendering Techniques for Haptic Display of Virtual Objects , 1999, Presence.

[32]  J. Dean Brederson,et al.  The Visual Haptic Workbench , 2005, The Visualization Handbook.

[33]  Cagatay Basdogan,et al.  Autostereoscopic and haptic visualization for space exploration and mission design , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[34]  Francis J. M. Schmitt,et al.  Multilevel Representation and Transmission of Real Objects with Progressive Octree Particles , 2003, IEEE Trans. Vis. Comput. Graph..

[35]  Doug A. Bowman,et al.  The Simple Virtual Environment Library: An Extensible Framework for Building VE Applications , 2000, Presence: Teleoperators & Virtual Environments.