A Linear Optimal Transportation Framework for Quantifying and Visualizing Variations in Sets of Images

Transportation-based metrics for comparing images have long been applied to analyze images, especially where one can interpret the pixel intensities (or derived quantities) as a distribution of ‘mass’ that can be transported without strict geometric constraints. Here we describe a new transportation-based framework for analyzing sets of images. More specifically, we describe a new transportation-related distance between pairs of images, which we denote as linear optimal transportation (LOT). The LOT can be used directly on pixel intensities, and is based on a linearized version of the Kantorovich-Wasserstein metric (an optimal transportation distance, as is the earth mover’s distance). The new framework is especially well suited for computing all pairwise distances for a large database of images efficiently, and thus it can be used for pattern recognition in sets of images. In addition, the new LOT framework also allows for an isometric linear embedding, greatly facilitating the ability to visualize discriminant information in different classes of images. We demonstrate the application of the framework to several tasks such as discriminating nuclear chromatin patterns in cancer cells, decoding differences in facial expressions, galaxy morphologies, as well as sub cellular protein distributions.

[1]  L. Wallrath,et al.  Connections between epigenetic gene silencing and human disease. , 2007, Mutation research.

[2]  G. Burton.  TOPICS IN OPTIMAL TRANSPORTATION (Graduate Studies in Mathematics 58) By CÉDRIC VILLANI: 370 pp., US$59.00, ISBN 0-8218-3312-X (American Mathematical Society, Providence, RI, 2003) , 2004 .

[3]  Z Pincus,et al.  Comparison of quantitative methods for cell‐shape analysis , 2007, Journal of microscopy.

[4]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[5]  Robert F Murphy,et al.  Deformation‐based nuclear morphometry: Capturing nuclear shape variation in HeLa cells , 2008, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[6]  E. Bengtsson.  Fifty years of attempts to automate screening for cervical cancer , 1999 .

[7]  Haibin Ling,et al.  An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Yann Brenier,et al.  A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem , 2000, Numerische Mathematik.

[9]  Bjarne K. Ersbøll,et al.  FAME-a flexible appearance modeling environment , 2003, IEEE Transactions on Medical Imaging.

[10]  Trevor Darrell,et al.  Fast contour matching using approximate earth mover's distance , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  Lani F. Wu,et al.  Image-based multivariate profiling of drug responses from single cells , 2007, Nature Methods.

[12]  J. Barrett,et al.  Partial L1 Monge–Kantorovich problem: variational formulation and numerical approximation , 2009 .

[13]  Wei Wang,et al.  Detection and classification of thyroid follicular lesions based on nuclear structure from histopathology images , 2010, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[14]  Christophe Chefd'Hotel,et al.  Intensity-based image registration using Earth Mover's Distance , 2007, SPIE Medical Imaging.

[15]  David W. Jacobs,et al.  Approximate earth mover’s distance in linear time , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  James B. Orlin,et al.  A faster strongly polynomial minimum cost flow algorithm , 1993, STOC '88.

[17]  Allen R. Tannenbaum,et al.  An Efficient Numerical Method for the Solution of the L2 Optimal Mass Transfer Problem , 2010, SIAM J. Sci. Comput..

[18]  Lior Shamir,et al.  Automatic morphological classification of galaxy images. , 2009, Monthly notices of the Royal Astronomical Society.

[19]  Gustavo K. Rohde,et al.  An Optimal Transportation Approach for Nuclear Structure-Based Pathology , 2011, IEEE Transactions on Medical Imaging.

[20]  Kim L. Boyer,et al.  Computer-aided evaluation of neuroblastoma on whole-slide histology images: Classifying grade of neuroblastic differentiation , 2009, Pattern Recognit..

[21]  L. Younes,et al.  Statistics on diffeomorphisms via tangent space representations , 2004, NeuroImage.

[22]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[23]  Steven Haker,et al.  Minimizing Flows for the Monge-Kantorovich Problem , 2003, SIAM J. Math. Anal..

[24]  Josef Stoer,et al.  Numerische Mathematik 1 , 1989 .

[25]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[26]  Hemant K. Sawant,et al.  Detection and classification of EEG waves , 2010 .

[27]  Michael Werman,et al.  A Linear Time Histogram Metric for Improved SIFT Matching , 2008, ECCV.

[28]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[29]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[30]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[31]  Lin Yang,et al.  Virtual Microscopy and Grid-Enabled Decision Support for Large-Scale Analysis of Imaged Pathology Specimens , 2009, IEEE Transactions on Information Technology in Biomedicine.

[32]  C. Villani.  Topics in Optimal Transportation , 2003 .

[33]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[34]  R. Brubaker.  Models for the perception of speech and visual form: Weiant Wathen-Dunn, ed.: Cambridge, Mass., The M.I.T. Press, I–X, 470 pages , 1968 .

[35]  Gian Luca Delzanno,et al.  Generalized Monge-Kantorovich Optimization for Grid Generation and Adaptation in Lp , 2010, SIAM J. Sci. Comput..

[36]  L. Wallrath,et al.  Linking Heterochromatin Protein 1 (HP1) to cancer progression. , 2008, Mutation research.

[37]  David J. Odde,et al.  Model Convolution: A Computational Approach to Digital Image Interpretation , 2010, Cellular and molecular bioengineering.

[38]  LingHaibin,et al.  An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison , 2007 .

[39]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[40]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[41]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[42]  Alejandro F Frangi,et al.  Automatic construction of 3-D statistical deformation models of the brain using nonrigid registration , 2003, IEEE Transactions on Medical Imaging.

[43]  James B. Orlin.  A Faster Strongly Polynomial Minimum Cost Flow Algorithm , 1993, Oper. Res..

[44]  Robert F. Murphy,et al.  A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells , 2001, Bioinform..

[45]  Yilin Mo,et al.  Penalized Fisher discriminant analysis and its application to image-based morphometry , 2011, Pattern Recognit. Lett..

[46]  Lei Zhu,et al.  Optimal Mass Transport for Registration and Warping , 2004, International Journal of Computer Vision.

[47]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[48]  Carey E. Priebe,et al.  Collaborative computational anatomy: An MRI morphometry study of the human brain via diffeomorphic metric mapping , 2009, Human Brain Mapping.

[49]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.