Augmented-reality visualizations guided by cognition: perceptual heuristics for combining visible and obscured information

One unique feature of mixed and augmented reality (MR/AR) systems is that hidden and occluded objects an be readily visualized. We call this specialized use of MR/AR, obscured information visualization (OIV). In this paper, we describe the beginning of a research program designed to develop such visualizations through the use of principles derived from perceptual psychology and cognitive science. In this paper we surveyed the cognitive science literature as it applies to such visualization tasks, described experimental questions derived from these cognitive principles, and generated general guidelines that can be used in designing future OIV systems (as well improving AR displays more generally). We also report the results from an experiment that utilized a functioning AR-OIV system: we found that in relative depth judgment, subjects reported rendered objects as being in front of real-world objects, except when additional occlusion and motion cues were presented together.

[1]  Victoria Interrante,et al.  Illustrating surface shape in volume data via principal direction-driven 3D line integral convolution , 1997, SIGGRAPH.

[2]  Blair MacIntyre,et al.  Adapting to dynamic registration errors using level of error (LOE) filtering , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[3]  Shimon Ullman,et al.  Structural Saliency: The Detection Of Globally Salient Structures using A Locally Connected Network , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[4]  James E. Cutting,et al.  HIGH-PERFORMANCE COMPUTING AND HUMAN VISION I , 2002 .

[5]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[6]  Geoffrey S. Hubona,et al.  The relative contributions of stereo, lighting, and background scenes in promoting 3D depth visualization , 1999, TCHI.

[7]  B. S. Manjunath,et al.  Color image segmentation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[9]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[10]  Stephen R. Ellis,et al.  Localization of Virtual Objects in the Near Visual Field , 1998, Hum. Factors.

[11]  Steve Mann,et al.  Humanistic computing: "WearComp" as a new framework and application for intelligent signal processing , 1998, Proc. IEEE.

[12]  Shumin Zhai,et al.  Applications of augmented reality for human-robot communication , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[13]  Ulrich Neumann,et al.  Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[14]  Paul Milgram,et al.  Perceptual issues in augmented reality , 1996, Electronic Imaging.

[15]  M. Braunstein,et al.  Recovering viewer-centered depth from disparity, occlusion, and velocity gradients , 1986, Perception & psychophysics.

[16]  Simon Haykin,et al.  Humanistic Intelligence: WearComp as a new framework and application for intelligent signal processing , 2001 .

[17]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[18]  Larry F. Hodges,et al.  The Perception of Distance in Simulated Visual Displays:A Comparison of the Effectiveness and Accuracy of Multiple Depth Cues Across Viewing Distances , 1997, Presence: Teleoperators & Virtual Environments.

[19]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[20]  Steven K. Feiner,et al.  View management for virtual and augmented reality , 2001, UIST '01.

[21]  M. D. Vernon,et al.  THE PERCEPTION OF DISTANCE , 1937 .

[22]  Deborah Hix,et al.  Usability engineering: domain analysis activities for augmented-reality systems , 2002, IS&T/SPIE Electronic Imaging.

[23]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[24]  Steven K. Feiner,et al.  Architectural Anatomy , 1995, Presence: Teleoperators & Virtual Environments.

[25]  M. Jagersand.  Saliency maps and attention selection in scale and spatial coordinates: an information theoretic approach , 1995, Proceedings of IEEE International Conference on Computer Vision.

[26]  Hong Chen,et al.  Observing a volume rendered fetus within a pregnant patient , 1994, Proceedings Visualization '94.

[27]  Andrew W. Fitzgibbon,et al.  An Experimental Comparison of Range Image Segmentation Algorithms , 1996, IEEE Trans. Pattern Anal. Mach. Intell..