Algebraic Geometric Comparison of Probability Distributions

We propose a novel algebraic algorithmic framework for dealing with probability distributions represented by their cumulants such as the mean and covariance matrix. As an example, we consider the unsupervised learning problem of finding the subspace on which several probability distributions agree. Instead of minimizing an objective function involving the estimated cumulants, we show that by treating the cumulants as elements of the polynomial ring we can directly solve the problem, at a lower computational cost and with higher accuracy. Moreover, the algebraic viewpoint on probability distributions allows us to invoke the theory of algebraic geometry, which we demonstrate in a compact proof for an identifiability criterion.

[1]  Franz J. Király,et al.  The Stationary Subspace Analysis Toolbox , 2011, J. Mach. Learn. Res..

[2]  Henry P. Wynn,et al.  Algebraic and geometric methods in statistics , 2009 .

[3]  Yoshinobu Kawahara,et al.  Stationary Subspace Analysis as a Generalized Eigenvalue Problem , 2010, ICONIP.

[4]  K. Müller,et al.  Finding stationary subspaces in multivariate time series. , 2009, Physical review letters.

[5]  Seth Sullivant,et al.  Lectures on Algebraic Statistics , 2008 .

[6]  Sebastian Pokutta,et al.  Approximate computation of zero-dimensional polynomial ideals , 2009, J. Symb. Comput..

[7]  Anthony Iarrobino,et al.  Compressed algebras: Artin algebras having given socle degrees and maximal length , 1984 .

[8]  David A. Cox,et al.  Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics) , 2007 .

[9]  Santiago Laplagne,et al.  An algorithm for the computation of the radical of an ideal , 2006, ISSAC '06.

[10]  Hans J. Stetter,et al.  Numerical polynomial algebra , 2004 .

[11]  Stephen M. Watt,et al.  The singular value decomposition for polynomial systems , 1995, ISSAC '95.

[12]  Teresa Krick,et al.  An Algorithm for the Computation of the Radical of an Ideal in the Ring of Polynomials , 1991, AAECC.

[13]  Motoaki Kawanabe,et al.  Learning invariances with Stationary Subspace Analysis , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[14]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[15]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[16]  渡邊 澄夫 Algebraic geometry and statistical learning theory , 2009 .

[17]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  D. Eisenbud,et al.  Direct methods for primary decomposition , 1992 .

[19]  Kari Torkkola,et al.  Feature Extraction by Non-Parametric Mutual Information Maximization , 2003, J. Mach. Learn. Res..

[20]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[21]  青木 敏,et al.  Lectures on Algebraic Statistics (Oberwolfach Seminars Vol.39), Mathias Drton, Bernd Sturmfels and Seth Sullivant 著, Birkhauser, Basel, Boston, Berlin, 2009年3月, 171+viii pp., 価格 24.90i, ISBN 978-3-7643-8904-8 , 2012 .

[22]  R. Kondor The skew spectrum of functions on finite groups and their homogeneous spaces , 2007, 0712.4259.

[23]  Karsten M. Borgwardt,et al.  The skew spectrum of graphs , 2008, ICML '08.

[24]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[25]  B. Sturmfels SOLVING SYSTEMS OF POLYNOMIAL EQUATIONS , 2002 .

[26]  Grete Hermann,et al.  Die Frage der endlich vielen Schritte in der Theorie der Polynomideale , 1926 .

[27]  Klaus-Robert Müller,et al.  Feature Extraction for Change-Point Detection Using Stationary Subspace Analysis , 2011, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Keith Pardue,et al.  Generic sequences of polynomials , 2010 .

[29]  Patrizia M. Gianni,et al.  Gröbner Bases and Primary Decomposition of Polynomial Ideals , 1988, J. Symb. Comput..

[30]  Carlo Traverso,et al.  Yet Another Ideal Decomposition Algorithm , 1997, AAECC.

[31]  Martin Kreuzer,et al.  From Oil Fields to Hilbert Schemes , 2009 .

[32]  Klaus-Robert Muller,et al.  Finding stationary brain sources in EEG data , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[33]  H. Hotelling The Generalization of Student’s Ratio , 1931 .