Machine learning the real discriminant locus

Parameterized systems of polynomial equations arise in many applications in science and engineering with the real solutions describing, for example, equilibria of a dynamical system, linkages satisfying design constraints, and scene reconstruction in computer vision. Since different parameter values can have a different number of real solutions, the parameter space is decomposed into regions whose boundary forms the real discriminant locus. This article views locating the real discriminant locus as a supervised classification problem in machine learning where the goal is to determine classification boundaries over the parameter space, with the classes being the number of real solutions. For multidimensional parameter spaces, this article presents a novel sampling method which carefully samples the parameter space. At each sample point, homotopy continuation is used to obtain the number of real solutions to the corresponding polynomial system. Machine learning techniques including nearest neighbor and deep learning are used to efficiently approximate the real discriminant locus. One application of having learned the real discriminant locus is to develop a real homotopy method that only tracks the real solution paths unlike traditional methods which track all~complex~solution~paths. Examples show that the proposed approach can efficiently approximate complicated solution boundaries such as those arising from the equilibria of the Kuramoto model.

[1]  Stavros J. Perantonis,et al.  Constrained Learning in Neural Networks: Application to Stable Factorization of 2-D Polynomials , 2004, Neural Processing Letters.

[2]  Kagan Tumer,et al.  Estimating the Bayes error rate through classifier combining , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[3]  David A. Cox,et al.  Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics) , 2007 .

[4]  Yang-Hui He,et al.  Exploring the potential energy landscape over a large parameter-space , 2013, 1301.0946.

[5]  C. Aggarwal Chapter 22 Active Learning : A Survey , 2014 .

[6]  Xin Xin,et al.  Analytical solutions of equilibrium points of the standard Kuramoto model: 3 and 4 oscillators , 2016, 2016 American Control Conference (ACC).

[7]  Frank Sottile,et al.  ALGORITHM XXX: ALPHACERTIFIED: CERTIFYING SOLUTIONS TO POLYNOMIAL SYSTEMS , 2011 .

[8]  Marina Weber,et al.  Using Algebraic Geometry , 2016 .

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  Alicia Dickenstein,et al.  Regions of multistationarity in cascades of Goldbeter–Koshland loops , 2018, Journal of Mathematical Biology.

[11]  Helen M. Byrne,et al.  Decomposing the Parameter Space of Biological Networks via a Numerical Discriminant Approach , 2016, MC.

[12]  D. S. Arnon,et al.  Algorithms in real algebraic geometry , 1988 .

[13]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[14]  Bernd Sturmfels,et al.  Learning algebraic varieties from samples , 2018, Revista Matemática Complutense.

[15]  Jonathan D. Hauenstein,et al.  Real monodromy action , 2019, Appl. Math. Comput..

[16]  Dhagash Mehta,et al.  Tumbling through a landscape: Evidence of instabilities in high-dimensional moduli spaces , 2013, 1303.4428.

[17]  Dhagash Mehta,et al.  Finding all flux vacua in an explicit example , 2012, 1212.4530.

[18]  R. Spigler,et al.  The Kuramoto model: A simple paradigm for synchronization phenomena , 2005 .

[19]  Andrew J. Sommese,et al.  The numerical solution of systems of polynomials - arising in engineering and science , 2005 .

[20]  Bican Xia,et al.  DISCOVERER: a tool for solving semi-algebraic systems , 2007, ACCA.

[21]  E. J. Doedel,et al.  AUTO: a program for the automatic bifurcation analysis of autonomous systems , 1980 .

[22]  Richard H. Middleton,et al.  Towards Modeling HIV Long Term Behavior , 2011 .

[23]  De-Shuang Huang,et al.  A Neural Root Finder of Polynomials Based on Root Moments , 2004, Neural Computation.

[24]  Edgar A. Bernal,et al.  The Loss Surface of XOR Artificial Neural Networks , 2018, Physical review. E.

[25]  Jonathan D. Hauenstein,et al.  Locating and Counting Equilibria of the Kuramoto Model with Rank-One Coupling , 2017, SIAM J. Appl. Algebra Geom..

[26]  M. Das,et al.  Polynomial real roots finding using Feed forward Neural Network: A simple approach , 2012, 2012 NATIONAL CONFERENCE ON COMPUTING AND COMMUNICATION SYSTEMS.

[27]  Daniel A. Brake,et al.  Paramotopy: Parameter Homotopies in Parallel , 2018, ICMS.

[28]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[29]  Michael Lindenbaum,et al.  Selective Sampling for Nearest Neighbor Classifiers , 1999, Machine Learning.

[30]  Guillaume Moroz,et al.  Stability and Bifurcation Analysis of Coupled Fitzhugh-Nagumo Oscillators , 2010, ArXiv.

[31]  Marc Moreno Maza,et al.  On Solving Parametric Polynomial Systems , 2012, Mathematics in Computer Science.

[32]  Hans-Jörg Schek,et al.  A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces , 1998, VLDB.

[33]  Alexandr Andoni,et al.  Learning Polynomials with Neural Networks , 2014, ICML.

[34]  Jonathan D. Hauenstein,et al.  Numerical Decomposition of the Rank-Deficiency Set of a Matrix of Multivariate Polynomials , 2009 .

[35]  Jeremy Gunawardena,et al.  Robustness and parameter geography in post-translational modification systems , 2019, bioRxiv.

[36]  A. Morgan,et al.  Coefficient-parameter polynomial continuation , 1989 .

[37]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[38]  De-Shuang Huang,et al.  A constructive approach for finding arbitrary roots of polynomials by neural networks , 2004, IEEE Transactions on Neural Networks.

[39]  Alicia Dickenstein,et al.  Lower bounds for positive roots and regions of multistationarity in chemical reaction networks , 2018, Journal of Algebra.

[40]  Willy Govaerts,et al.  MATCONT: A MATLAB package for numerical bifurcation analysis of ODEs , 2003, TOMS.

[41]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[42]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[43]  S. Strogatz From Kuramoto to Crawford: exploring the onset of synchronization in populations of coupled oscillators , 2000 .

[44]  Volker Weispfenning,et al.  Comprehensive Gröbner Bases , 1992, J. Symb. Comput..

[45]  H. Sebastian Seung,et al.  Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks , 2003, Neural Computation.

[46]  Alexandr Andoni,et al.  Learning Sparse Polynomial Functions , 2014, SODA.

[47]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[48]  Jonathan D. Hauenstein,et al.  Numerically Solving Polynomial Systems with Bertini , 2013, Software, environments, tools.

[49]  De-Shuang Huang Constrained learning algorithms for finding the roots of polynomials: a case study , 2002, 2002 IEEE Region 10 Conference on Computers, Communications, Control and Power Engineering. TENCOM '02. Proceedings..

[50]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[51]  Zheru Chi,et al.  Neural networks with problem decomposition for finding real roots of polynomials , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[52]  Jonathan D. Hauenstein,et al.  Adaptive strategies for solving parameterized systems using homotopy continuation , 2017, Appl. Math. Comput..

[53]  Jonathan D. Hauenstein,et al.  Membership tests for images of algebraic sets by linear projections , 2013, Appl. Math. Comput..

[54]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[55]  Sanjoy Dasgupta Consistency of Nearest Neighbor Classification under Selective Sampling , 2012, COLT.

[56]  Florian Dörfler,et al.  Synchronization in complex networks of phase oscillators: A survey , 2014, Autom..

[57]  Jonathan D. Hauenstein,et al.  Witness sets of projections , 2010, Appl. Math. Comput..

[58]  Matthew England,et al.  Using Machine Learning to Improve Cylindrical Algebraic Decomposition , 2018, Mathematics in Computer Science.

[59]  Eugene L. Allgower,et al.  Numerical continuation methods - an introduction , 1990, Springer series in computational mathematics.

[60]  Jonathan D. Hauenstein,et al.  Smooth points on semi-algebraic sets , 2020, ACM Commun. Comput. Algebra.

[61]  Elisenda Feliu,et al.  Identifying parameter regions for multistationarity , 2016, PLoS Comput. Biol..

[62]  Aranya Chakrabortty,et al.  Locating Power Flow Solution Space Boundaries: A Numerical Polynomial Homotopy Approach , 2017, ArXiv.

[63]  佐藤 洋祐,et al.  特集 Comprehensive Grobner Bases , 2007 .

[64]  Dimitris K. Tasoulis,et al.  Determining the number of real roots of polynomials through neural networks , 2006, Comput. Math. Appl..