Image-based ghostings for single layer occlusions in augmented reality

In augmented reality displays, X-Ray visualization techniques make hidden objects visible through combining the physical view with an artificial rendering of the hidden information. An important step in X-Ray visualization is to decide which parts of the physical scene should be kept and which should be replaced by overlays. The combination should provide users with essential perceptual cues to understand the relationship of depth between hidden information and the physical scene. In this paper we present an approach that addresses this decision in unknown environments by analyzing camera images of the physical scene and using the extracted information for occlusion management. Pixels are grouped into perceptually coherent image regions and a set of parameters is determined for each region. The parameters change the X-Ray visualization for either preserving existing structures or generating synthetic structures. Finally, users can customize the overall opacity of foreground regions to adapt the visualization.

[1]  Pat Hanrahan,et al.  Designing effective step-by-step assembly instructions , 2003, ACM Trans. Graph..

[2]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[3]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[4]  Victoria Interrante,et al.  Illustrating transparent surfaces with curvature-directed strokes , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[5]  Dieter Schmalstieg,et al.  Importance masks for revealing occluded objects in augmented reality , 2009, VRST '09.

[6]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Neil W. Bergmann,et al.  Perceptually based quantization technique for MPEG encoding , 1998, Electronic Imaging.

[8]  Steven K. Feiner,et al.  Automated generation of intent-based 3D Illustrations , 1991, SIGGRAPH.

[9]  Stefan Bruckner,et al.  Illustrative Context-Preserving Exploration of Volume Data , 2006, IEEE Transactions on Visualization and Computer Graphics.

[10]  Thomas Ertl,et al.  Transparency in Interactive Technical Illustrations , 2002, Comput. Graph. Forum.

[11]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jens Schneider,et al.  ClearView: An Interactive Context Preserving Hotspot Visualization Technique , 2006, IEEE Transactions on Visualization and Computer Graphics.

[13]  Guang-Zhong Yang,et al.  pq-space Based Non-Photorealistic Rendering for Augmented Reality , 2007, MICCAI.

[14]  J. Moran,et al.  Sensation and perception , 1980 .

[15]  Dieter Schmalstieg,et al.  Comprehensible Visualization for Augmented Reality , 2009, IEEE Transactions on Visualization and Computer Graphics.

[16]  Tobias Höllerer,et al.  Resolving multiple occluded layers in augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[17]  Ronald Azuma,et al.  Augmented-reality visualizations guided by cognition: perceptual heuristics for combining visible and obscured information , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[18]  Tobias Höllerer,et al.  Interactive tools for virtual x-ray vision in mobile augmented reality , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[19]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[20]  S. Griffis.  EDITOR , 1997, Journal of Navigation.

[21]  David Salesin,et al.  Automated generation of interactive 3D exploded view diagrams , 2008, SIGGRAPH 2008.

[22]  Victoria Interrante,et al.  Conveying the 3D Shape of Smoothly Curving Transparent Surfaces via Texture , 1997, IEEE Trans. Vis. Comput. Graph..

[23]  Dieter Schmalstieg,et al.  Real-time panoramic mapping and tracking on mobile phones , 2010, 2010 IEEE Virtual Reality Conference (VR).

[24]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Ekkehard Euler,et al.  A Practical Approach for Intraoperative Contextual In-Situ Visualization , 2009 .

[26]  Dieter Schmalstieg,et al.  Explosion Diagrams in Augmented Reality , 2009, 2009 IEEE Virtual Reality Conference.

[27]  Adam Finkelstein,et al.  Real-time hatching , 2001, SIGGRAPH.

[28]  Steven K. Feiner,et al.  Cutaways and ghosting: satisfying visibility constraints in dynamic 3D illustrations , 1992, The Visual Computer.

[29]  H. Reeves,et al.  The Guild handbook of scientific illustration , 1991 .

[30]  Mark Billinghurst,et al.  Interaction With Partially Transparent Hands And Objects , 2005, AUIC.

[31]  Nassir Navab,et al.  Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.