Improving 2D-3D Registration by Mutual Information using Gradient Maps

In this paper we propose an extension for the algorithms of image-to-geometry registration by Mutual Information(MI) to improve the performance and the quality of the alignment. Proposed for the registration of multi modal medical images, in the last years MI has been adapted to align a 3D model to a given image by using different renderings of the model and a gray-scale version of the input image. A key aspect is the choice of the rendering process to correlate the 3D model to the image without taking into account the texture data and the lighting conditions. Even if several rendering types for the 3D model have been analyzed, in some cases the alignment fails for two main reasons: the peculiar reflection behavior of the object that we are not able to reproduce in the rendering of the 3D model without knowing the material characteristics of the object and the lighting conditions of the acquisition environment; the characteristics of the image background, especially non uniform background, that can degrade the convergence of the registration. To improve the quality of the registration in these cases we propose to compute the MI between the gradient map of the 3D rendering and the gradient map of the image in order to maximize the shared data between them.

[1]  Dietmar Saupe,et al.  Highly-Automatic MI Based Multiple 2D/3D Image Registration Using Self-initialized Geodesic Feature Correspondences , 2009, ACCV.

[2]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[3]  Paul Suetens,et al.  Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information , 1999, Medical Image Anal..

[4]  Max A. Viergever,et al.  Image registration by maximization of combined mutual information and gradient information , 2000, IEEE Transactions on Medical Imaging.

[5]  Alois Knoll,et al.  Mutual Information-Based 3D Object Tracking , 2008, International Journal of Computer Vision.

[6]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[7]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[8]  Paolo Cignoni,et al.  Minimizing user intervention in registering 2D images to 3D models , 2005, The Visual Computer.

[9]  M. Powell.  Developments of NEWUOA for minimization without derivatives , 2008 .

[10]  Dietmar Saupe,et al.  Stochastic Optimization of Multiple Texture Registration Using Mutual Information , 2007, DAGM-Symposium.

[11]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[12]  Katsushi Ikeuchi,et al.  Appearance compression and synthesis based on 3D model for mixed reality , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[14]  Roberto Scopigno,et al.  Image‐to‐Geometry Registration: a Mutual Information Method exploiting Illumination‐related Geometric Properties , 2009, Comput. Graph. Forum.