Spotlight: Automated Confidence-Based User Guidance for Increasing Efficiency in Interactive 3D Image Segmentation

We present Spotlight, an automated user guidance technique for improving quality and efficiency of interactive segmentation tasks. Spotlight augments interactive segmentation algorithms by automatically highlighting areas in need of attention to the user during the interaction phase. We employ a 3D Livewire algorithm as our base segmentation method where the user quickly provides a minimal initial contour seeding. The quality of the initial segmentation is then evaluated based on three different metrics that probe the contour edge strength, contour stability and object connectivity. The result of this evaluation is fed into a novel algorithm that autonomously suggests regions that require user intervention. Essentially, Spotlight flags potentially problematic image regions in a prioritized fashion based on an optimization process for improving the final 3D segmentation. We present a variety of qualitative and quantitative examples demonstrating Spotlight's intuitive use and proven utility in reducing user input by increasing automation.

[1]  Uncertainty-Aware Guided Volume Segmentation , 2010, IEEE Transactions on Visualization and Computer Graphics.

[2]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[3]  Ross T. Whitaker,et al.  Interactive, GPU-Based Level Sets for 3D Segmentation , 2003, MICCAI.

[4]  Ghassan Hamarneh,et al.  Efficient interactive 3D Livewire segmentation of complex objects with arbitrary topology , 2008, Comput. Medical Imaging Graph..

[5]  Rüdiger Westermann,et al.  Random Walks for Interactive Organ Segmentation in Two and Three Dimensions: Implementation and Validation , 2005, MICCAI.

[6]  P. J. Narayanan,et al.  CUDA cuts: Fast graph cuts on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[7]  Noel E. O'Connor,et al.  A comparative evaluation of interactive segmentation algorithms , 2010, Pattern Recognit..

[8]  J. Y. Yen,et al.  Finding the K Shortest Loopless Paths in a Network , 2007 .

[9]  Leo Grady,et al.  Fast approximate Random Walker segmentation using eigenvector precomputation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  William A. Barrett,et al.  Interactive segmentation of image volumes with Live Surface , 2007, Comput. Graph..

[11]  William A. Barrett,et al.  Interactive Segmentation with Intelligent Scissors , 1998, Graph. Model. Image Process..

[12]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[13]  William A. Barrett,et al.  Interactive live-wire boundary extraction , 1997, Medical Image Anal..

[14]  Ghassan Hamarneh,et al.  Hands-free interactive image segmentation using eyegaze , 2009, Medical Imaging.

[15]  Ghassan Hamarneh,et al.  Active Learning for Interactive 3D Image Segmentation , 2011, MICCAI.

[16]  Leo Grady,et al.  Multilabel random walker image segmentation using prior models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[19]  Leo Grady,et al.  Minimal Surfaces Extend Shortest Path Segmentation Methods to 3D , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[21]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Filip Malmberg,et al.  A 3D Live-Wire Segmentation Method for Volume Images Using Haptic Interaction , 2006, DGCI.

[23]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[24]  Ghassan Hamarneh,et al.  Exploration and Visualization of Segmentation Uncertainty using Shape and Appearance Prior Information , 2010, IEEE Transactions on Visualization and Computer Graphics.

[25]  Ross T. Whitaker,et al.  Markov surfaces: A probabilistic framework for user-assisted three-dimensional image segmentation , 2011, Comput. Vis. Image Underst..

[26]  Axel Pinz,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[27]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[28]  Derek Hoiem,et al.  Learning CRFs Using Graph Cuts , 2008, ECCV.

[29]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[30]  Lasse Riis Østergaard,et al.  Active Surface Approach for Extraction of the Human Cerebral Cortex from MRI , 2006, MICCAI.

[31]  Ghassan Hamarneh,et al.  ProbExplorer: Uncertainty‐guided Exploration and Editing of Probabilistic Medical Image Segmentation , 2010, Comput. Graph. Forum.

[32]  Ghassan Hamarneh,et al.  Quaternion Color Curvature , 2008, Color Imaging Conference.

[33]  Arnold W. M. Smeulders,et al.  Interaction in the segmentation of medical images: A survey , 2001, Medical Image Anal..

[34]  Leo Grady,et al.  A Seeded Image Segmentation Framework Unifying Graph Cuts And Random Walker Which Yields A New Algorithm , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[35]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[36]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[37]  Camille Couprie,et al.  Power watersheds: A new image segmentation framework extending graph cuts, random walker and optimal spanning forest , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[39]  Pushmeet Kohli,et al.  Measuring Uncertainty in Graph Cut Solutions - Efficiently Computing Min-marginal Energies Using Dynamic Graph Cuts , 2006, ECCV.

[40]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[41]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Joachim Denzler,et al.  GPU-Based Volume Segmentation , 2005 .

[43]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[44]  Ghassan Hamarneh,et al.  Fast Random Walker with Priors Using Precomputation for Interactive Medical Image Segmentation , 2010, MICCAI.

[45]  Yan Kang,et al.  Interactive 3D editing tools for image segmentation , 2004, Medical Image Anal..

[46]  Gareth Funka-Lea,et al.  An Energy Minimization Approach to the Data Driven Editing of Presegmented Images/Volumes , 2006, MICCAI.

[47]  M. Stella Atkins,et al.  Fully automatic segmentation of the brain in MRI , 1998, IEEE Transactions on Medical Imaging.

[48]  Antonio J. Plaza,et al.  Supervised hyperspectral image segmentation using active learning , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.