Efficient Simplex-Like Methods for Equilibria of Nonsymmetric Analog Networks

What is the complexity of computing equilibria for physically implementable analog networks (Hopfield 1984; Sejnowski 1981) with arbitrary connectivity? We show that if the amplifiers are piecewise-linear, then such networks are instances of a game-theoretic model known as polymatrix games. In contrast with the usual gradient descent methods for symmetric networks, equilibria for polymatrix games may be computed by vertex pivoting algorithms similar to the simplex method for linear programming. Like the simplex method, these algorithms have characteristic low order polynomial behavior in virtually all practical cases, though not certain theoretical ones. While these algorithms cannot be applied to models requiring evolution from an initial point, they are applicable to clamping models whose input is expressed purely as a bias. Thus we have an a priori indication that such models are computationally tractable.

[1]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[2]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[3]  G. Dantzig,et al.  COMPLEMENTARY PIVOT THEORY OF MATHEMATICAL PROGRAMMING , 1968 .

[4]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[6]  Tj Sejnowski,et al.  Skeleton filters in the brain , 2014 .

[7]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[8]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Richard W. Cottle.  Observations on a class of nasty linear complementarity problems , 1980, Discret. Appl. Math..

[10]  M. B. Pour-El,et al.  The wave equation with computable initial data such that its unique solution is not computable , 1981 .

[11]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Katta G. Murty,et al.  Linear complementarity, linear and nonlinear programming , 1988 .

[13]  J. Nash,et al.  NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[14]  B. Dickinson,et al.  The complexity of analog computation , 1986 .

[15]  C. E. Lemke,et al.  Bimatrix Equilibrium Points and Mathematical Programming , 1965 .

[16]  B. Curtis Eaves,et al.  Polymatrix Games with Joint Constraints , 1973 .

[17]  Christos H. Papadimitriou,et al.  The complexity of recognizing polyhedral scenes , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[18]  Michael J. Todd,et al.  Polynomial expected behavior of a pivoting algorithm for linear complementarity and linear programming problems , 1986, Math. Program..

[19]  Mihalis Yannakakis,et al.  How easy is local search? , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[20]  Steven W. Zucker,et al.  Two Stages of Curve Detection Suggest Two Styles of Visual Computation , 1989, Neural Computation.

[21]  Steven W. Zucker,et al.  Copositive-plus Lemke algorithm solves polymatrix games , 1991, Oper. Res. Lett..

[22]  Mihalis Yannakakis,et al.  On the complexity of local search , 1990, STOC '90.

[23]  N. Megiddo,et al.  New results on the average behavior of simplex algorithms , 1984 .

[24]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .