Hidden Markov model-based multi-modal image fusion with efficient training

Automated spatial alignment of images from different modalities is an important problem, particularly in bio-medical image analysis. We propose a novel probabilistic framework, based on a variant of the 2D hidden Markov model (2D HMM), to capture the deformation between multi-modal images. Smoothness is ensured via transition probabilities of the 2D HMM and cross-modality similarity via class-conditional, modality-specific emission probabilities. The method is derived for general multi-modal settings, and its performance is demonstrated for an application in cellular microscopy. We also present an efficient algorithm for parameter estimation. Experiments on synthetic and real biological data show improvement over state-of-the-art multi-modal image fusion techniques.

[1]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[2]  JR ANDERSON,et al.  The Viking viewer for connectomics: scalable multi-user annotation and summarization of large volume data sets , 2011, Journal of microscopy.

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[5]  Oscar E. Agazzi,et al.  Keyword Spotting in Poorly Printed Documents using Pseudo 2-D Hidden Markov Models , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Robert M. Gray,et al.  Multiresolution image classification by hierarchical modeling with two-dimensional hidden Markov models , 2000, IEEE Trans. Inf. Theory.

[7]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[8]  Paul Suetens,et al.  Nonrigid Image Registration Using Conditional Mutual Information , 2007, IPMI.

[9]  Max A. Viergever,et al.  Registration of Cervical MRI Using Multifeature Mutual Information , 2009, IEEE Transactions on Medical Imaging.

[10]  Kenneth Rose,et al.  Iterative decoding of two-dimensional hidden Markov models , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[11]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[12]  D. Mastronarde,et al.  Exploring the retinal connectome , 2011, Molecular vision.

[13]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[14]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[15]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[16]  Robert M. Gray,et al.  Image classification by a two dimensional hidden Markov model , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[17]  Sébastien Ourselin,et al.  Fast free-form deformation using graphics processing units , 2010, Comput. Methods Programs Biomed..