Distributed high dimensional information theoretical image registration via random projections

Information theoretical measures, such as entropy, mutual information, and various divergences, exhibit robust characteristics in image registration applications. However, the estimation of these quantities is computationally intensive in high dimensions. On the other hand, consistent estimation from pairwise distances of the sample points is possible, which suits random projection (RP) based low dimensional embeddings. We adapt the RP technique to this task by means of a simple ensemble method. To the best of our knowledge, this is the first distributed, RP based information theoretical image registration approach. The efficiency of the method is demonstrated through numerical examples.

[1]  Sanjoy Dasgupta,et al.  Experiments with Random Projection , 2000, UAI.

[2]  Jan Kybic,et al.  Regional image similarity criteria based on the Kozachenko-Leonenko entropy estimator , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Barnabás Póczos,et al.  Estimation of Renyi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs , 2010, NIPS.

[4]  Alfred O. Hero,et al.  Applications of entropic spanning graphs , 2002, IEEE Signal Process. Mag..

[5]  Santosh S. Vempala,et al.  An algorithmic theory of learning: Robust concepts and random projection , 1999, Machine Learning.

[6]  Chinmay Hegde,et al.  Random Projections for Manifold Learning , 2007, NIPS.

[7]  Carla E. Brodley,et al.  Random Projection for High Dimensional Data Clustering: A Cluster Ensemble Approach , 2003, ICML.

[8]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[9]  Jan Kybic.  High-dimensional mutual information estimation for image registration , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[10]  Kenneth Ward Church,et al.  Nonlinear Estimators and Tail Bounds for Dimension Reduction in l1 Using Cauchy Random Projections , 2006, J. Mach. Learn. Res..

[11]  Dan Stowell,et al.  Fast Multidimensional Entropy Estimation by $k$-d Partitioning , 2009, IEEE Signal Processing Letters.

[12]  Kenneth Ward Church,et al.  Very sparse random projections , 2006, KDD '06.

[13]  Simon K. Warfield,et al.  Accelerating Feature Based Registration Using the Johnson-Lindenstrauss Lemma , 2009, MICCAI.

[14]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[15]  Heikki Mannila,et al.  Random projection in dimensionality reduction: applications to image and text data , 2001, KDD '01.

[16]  George Bebis,et al.  Face recognition experiments with random projection , 2005, SPIE Defense + Commercial Sensing.

[17]  András Lörincz,et al.  Fast Parallel Estimation of High Dimensional Information Theoretical Quantities with Low Dimensional Random Projection Ensembles , 2009, ICA.

[18]  Vincent Lepetit,et al.  Fast Keypoint Recognition Using Random Ferns , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Mateu Sbert,et al.  High-Dimensional Normalized Mutual Information for Image Registration Using Random Lines , 2006, WBIR.

[20]  Jirí Matousek,et al.  On variants of the Johnson–Lindenstrauss lemma , 2008, Random Struct. Algorithms.

[21]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[22]  Pascal Frossard,et al.  Optimal image alignment with random measurements , 2009, 2009 17th European Signal Processing Conference.

[23]  Dennis M. Healy,et al.  FAST GLOBAL IMAGE REGISTRATION USING RANDOM PROJECTIONS , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[24]  Alfred O. Hero,et al.  Image registration methods in high‐dimensional space , 2006, Int. J. Imaging Syst. Technol..

[25]  Peter Frankl,et al.  The Johnson-Lindenstrauss lemma and the sphericity of some graphs , 1987, J. Comb. Theory, Ser. B.

[26]  J. Matousek,et al.  On variants of the Johnson–Lindenstrauss lemma , 2008 .

[27]  Piotr Indyk,et al.  Approximate Nearest Neighbor: Towards Removing the Curse of Dimensionality , 2012, Theory Comput..

[28]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[29]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[30]  Santosh S. Vempala,et al.  The Random Projection Method , 2005, DIMACS Series in Discrete Mathematics and Theoretical Computer Science.

[31]  Bernard Chazelle,et al.  Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform , 2006, STOC '06.

[32]  Henrik Boström,et al.  Reducing High-Dimensional Data by Principal Component Analysis vs. Random Projection for Nearest Neighbor Classification , 2006, 2006 5th International Conference on Machine Learning and Applications (ICMLA'06).

[33]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[36]  Alfred O. Hero,et al.  Weighted k-NN graphs for Rényi entropy estimation in high dimensions , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[37]  Dmitriy Fradkin,et al.  Experiments with random projections for machine learning , 2003, KDD '03.

[38]  J. Yukich.  Probability theory of classical Euclidean optimization problems , 1998 .

[39]  Pierre Vandergheynst,et al.  On the estimation of geodesic paths on sampled manifolds under random projections , 2008, 2008 15th IEEE International Conference on Image Processing.

[40]  Sanjay Chawla,et al.  An incremental data-stream sketch using sparse random projections , 2007, SDM.

[41]  Barnabás Póczos,et al.  Undercomplete Blind Subspace Deconvolution , 2007, J. Mach. Learn. Res..