Visual feature extraction via eye tracking for saliency driven 2D/3D registration

This paper presents a new technique for extracting visual saliency from experimental eye tracking data. An eye-tracking system is employed to determine which features that a group of human observers considered to be salient when viewing a set of video images. With this information, a biologically inspired saliency map is derived by transforming each observed video image into a feature space representation. By using a feature normalisation process based on the relative abundance of visual features within the background image and those dwelled on eye tracking scan paths, features related to visual attention are determined. These features are then back projected to the image domain to determine spatial areas of interest for unseen video images. The strengths and weaknesses of the method are demonstrated with feature correspondence for 2D to 3D image registration of endoscopy videos with computed tomography data. The biologically derived saliency map is employed to provide an image similarity measure that forms the heart of the 2D/3D registration method. It is shown that by only processing selective regions of interest as determined by the saliency map, rendering overhead can be greatly reduced. Significant improvements in pose estimation efficiency can be achieved without apparent reduction in registration accuracy when compared to that of using a non-saliency based similarity measure.

[1]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[2]  William H. Press,et al.  Numerical Recipes in Fortran 77: The Art of Scientific Computing 2nd Editionn - Volume 1 of Fortran Numerical Recipes , 1992 .

[3]  Terry M. Peters,et al.  Mixed Reality Merging of Endoscopic Images and 3-D Surfaces , 2000, MICCAI.

[4]  Guang-Zhong Yang,et al.  Visual search: psychophysical models, practical applications , 2002, Image Vis. Comput..

[5]  Nicholas Ayache,et al.  The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration , 1998, MICCAI.

[6]  K F Moos,et al.  A vision-based three-dimensional capture system for maxillofacial assessment and surgical planning. , 1998, British Journal of Oral and Maxillofacial Surgery.

[7]  Jun Sugiyama,et al.  Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images , 2002, Medical Image Anal..

[8]  William E. Higgins,et al.  Technique for registering 3D virtual CT images to endoscopic video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  William H. Press,et al.  Numerical recipes in C , 2002 .

[10]  Nicolai Petkov,et al.  Computational models of visual neurons specialised in the detection of periodic and aperiodic oriented visual stimuli: bar and grating cells , 1997, Biological Cybernetics.

[11]  W. Eric L. Grimson,et al.  Multi-modal Volume Registration Using Joint Intensity Distributions , 1998, MICCAI.

[12]  Guang-Zhong Yang,et al.  Hot spot detection based on feature space representation of visual search , 2003, IEEE Transactions on Medical Imaging.

[13]  Kevin Cleary.  Final Report of the Technical Requirements for Image-Guided Spine Procedures Workshop , 2000, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[14]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[15]  Ali Shokoufandeh,et al.  View-based object recognition using saliency maps , 1999, Image Vis. Comput..

[16]  Eduard Gröller,et al.  EndoView: A Phantom Study of a Tracked Virtual Bronchoscopy , 2002, WSCG.

[17]  Jürgen Weese,et al.  A Comparison of Simularity Measures for use in 2D-3D Medical Image Registration , 1998, MICCAI.

[18]  Kensaku Mori,et al.  Method for tracking camera motion of real endoscope by using virtual endoscopy system , 2000, Medical Imaging.

[19]  Christof Koch,et al.  Comparison of feature combination strategies for saliency-based visual attention systems , 1999, Electronic Imaging.

[20]  Terry M. Peters,et al.  Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas , 2002, IEEE Transactions on Medical Imaging.

[21]  D. Hill,et al.  Medical image registration , 2001, Physics in medicine and biology.

[22]  Turner Whitted,et al.  An improved illumination model for shaded display , 1979, CACM.

[23]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[24]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[25]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[26]  Martin Jägersand,et al.  Saliency Maps and Attention Selection in Scale and Spatial Coordinates: An Information Theoretic Approach , 1995, ICCV.

[27]  Jürgen Weese,et al.  A comparison of similarity measures for use in 2-D-3-D medical image registration , 1998, IEEE Transactions on Medical Imaging.