Specificity: A Graph-Based Estimator of Divergence

In statistical modeling, there are various techniques used to build models from training data. Quantitative comparison of modeling techniques requires a method for evaluating the quality of the fit between the model probability density function (pdf) and the training data. One graph-based measure that has been used for this purpose is the specificity. We consider the large-numbers limit of the specificity, and derive expressions which show that it can be considered as an estimator of the divergence between the unknown pdf from which the training data was drawn and the model pdf built from the training data. Experiments using artificial data enable us to show that these limiting large-number relations enable us to obtain good quantitative and qualitative predictions of the behavior of the measured specificity, even for small numbers of training examples and in some extreme cases. We demonstrate that specificity can provide a more sensitive measure of difference between various modeling methods than some previous graph-based techniques. Key points are illustrated using real data sets. We thus establish a proper theoretical basis for the previously ad hoc concept of specificity, and obtain useful insights into the application of specificity in the analysis of real data.

[1]  Ian Craw,et al.  Parameterising Images for Recognition and Reconstruction , 1991, BMVC.

[2]  J. Beardwood,et al.  The shortest path through many points , 1959, Mathematical Proceedings of the Cambridge Philosophical Society.

[3]  J. D. Gorman,et al.  Alpha-Divergence for Classification, Indexing and Retrieval (Revised 2) , 2002 .

[4]  Erik G. Miller.  A new class of entropy estimators for multi-dimensional densities , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[5]  Timothy F. Cootes,et al.  Assessing the accuracy of non-rigid registration with and without ground truth , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[6]  E. Hellinger,et al.  Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. , 1909 .

[7]  Tien-Tsin Wong,et al.  Landmark Correspondence Optimization for Coupled Surfaces , 2007, MICCAI.

[8]  A. Wade.  Explicit laws of large numbers for random nearest-neighbour-type graphs , 2006, Advances in Applied Probability.

[9]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[10]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[11]  Timothy F. Cootes,et al.  Training Models of Shape from Sets of Examples , 1992, BMVC.

[12]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[13]  Michel,et al.  2 - Graphes de représentation minimaux, entropies et divergences : applications , 2000 .

[14]  Timothy F. Cootes,et al.  Diffeomorphic Statistical Shape Models , 2008, BMVC.

[15]  Timothy F. Cootes,et al.  Automatic tracking, coding and reconstruction of human faces, using flexible appearance models , 1994 .

[16]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[17]  Christopher J. Taylor,et al.  Statistical models of shape - optimisation and evaluation , 2008 .

[18]  K. Matusita.  Decision Rules, Based on the Distance, for Problems of Fit, Two Samples, and Estimation , 1955 .

[19]  Kaleem Siddiqi,et al.  Medial Representations: Mathematics, Algorithms and Applications , 2008 .

[20]  Yili Fu,et al.  A framework for automatic construction of 3D PDM from segmented volumetric neuroradiological data sets , 2010, Comput. Methods Programs Biomed..

[21]  J. Steele.  Probability theory and combinatorial optimization , 1987 .

[22]  Guido Gerig,et al.  Parametrization of Closed Surfaces for 3-D Shape Description , 1995, Comput. Vis. Image Underst..

[23]  Timothy F. Cootes,et al.  Shape Discrimination in the Hippocampus Using an MDL Model , 2003, IPMI.

[24]  Alfred O. Hero,et al.  Geodesic entropic graphs for dimension and entropy estimation in manifold learning , 2004, IEEE Transactions on Signal Processing.

[25]  Qing Wang,et al.  Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances , 2009, IEEE Transactions on Information Theory.

[26]  Martin Styner,et al.  Evaluation of 3D Correspondence Methods for Model Building , 2003, IPMI.

[27]  Christopher J. Taylor,et al.  Specificity as a Graph-Based Estimator of Cross-Entropy and KL Divergence , 2006, BMVC.

[28]  Alfred O. Hero,et al.  Applications of entropic spanning graphs , 2002, IEEE Signal Process. Mag..

[29]  J. Steele.  Growth Rates of Euclidean Minimal Spanning Trees With Power Weighted Edges , 1988 .

[30]  Alejandro F. Frangi,et al.  Automatic Construction of 3D Statistical Deformation Models of the Brain using Non-Rigid Registration , 2003, IEEE Trans. Medical Imaging.

[31]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[32]  David H. Eberly,et al.  Zoom-Invariant Vision of Figural Shape: The Mathematics of Cores , 1996, Comput. Vis. Image Underst..

[33]  Kenneth A. Brakke,et al.  Statistics of Random Plane Voronoi Tessellations , 2006 .

[34]  A. Hero,et al.  Estimation of Renyi information divergence via pruned minimal spanning trees , 1999, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics. SPW-HOS '99.

[35]  S. M. Ali,et al.  A General Class of Coefficients of Divergence of One Distribution from Another , 1966 .

[36]  Alejandro F Frangi,et al.  Automatic construction of 3-D statistical deformation models of the brain using nonrigid registration , 2003, IEEE Transactions on Medical Imaging.