On the Computational Power of Max-Min Propagation Neural Networks

We investigate the computational power of max-min propagation (MMP) neural networks, composed of neurons with maximum (Max) or minimum (Min) activation functions, applied over the weighted sums of inputs. The main results presented are that a single-layer MMP network can represent exactly any pseudo-Boolean function F:{0,1}n → [0,1], and that two-layer MMP neural networks are universal approximators. In addition, it is shown that several well-known fuzzy min-max (FMM) neural networks, such as Simpson's FMM, are representable by MMP neural networks.

[1]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks - Part 2: Clustering , 1993, IEEE Trans. Fuzzy Syst..

[2]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[3]  Pablo Antonio Estévez.  Max-min propagation neural networks : representation capabilities, learning algorithms and evolutionary structuring , 1995 .

[4]  E. Cheney.  Introduction to approximation theory , 1966 .

[5]  Wolfgang Maass,et al.  Neural Computation with Winner-Take-All as the Only Nonlinear Operation , 1999, NIPS.

[6]  T. Kailath,et al.  Discrete Neural Computation: A Theoretical Foundation , 1995 .

[7]  Antonello Rizzi,et al.  Adaptive resolution min-max classifiers , 2002, IEEE Trans. Neural Networks.

[8]  P. K. Simpson.  Fuzzy Min-Max Neural Networks-Part 1 : Classification , 1992 .

[9]  Pablo A. Estévez,et al.  Hierarchical mixture of experts and Max-Min propagation neural networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Wolfgang Maass,et al.  Bounds for the computational power and learning complexity of analog neural nets , 1993, SIAM J. Comput..

[11]  Mohamad H. Hassoun,et al.  Implementation of O(n) complexity max/min circuits for fuzzy and connectionist computing , 1993, IEEE International Conference on Neural Networks.

[12]  Kiichi Urahama,et al.  K-winners-take-all circuit with O(N) complexity , 1995, IEEE Trans. Neural Networks.

[13]  P. K. Simpson,et al.  Fuzzy min-max neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[14]  Iluminada Baturone,et al.  Current-mode multiple-input Max circuit , 1994 .

[15]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[16]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[17]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[18]  Y. Okabe,et al.  Max-min propagation nets: learning by delta rule for the Chebyshev norm , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[19]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.

[20]  Andrzej Bargiela,et al.  General fuzzy min-max neural network for clustering and classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[21]  Stephen I. Gallant,et al.  Neural network learning and expert systems , 1993 .

[22]  Valmir Carneiro Barbosa,et al.  Learning in the combinatorial neural model , 1998, IEEE Trans. Neural Networks.

[23]  Thomas Kailath,et al.  Depth-Size Tradeoffs for Neural Computation , 1991, IEEE Trans. Computers.

[24]  Loo-Nin Teow,et al.  Effective learning in recurrent max-min neural networks , 1998, Neural Networks.

[25]  Martin A. Giese,et al.  Biophysiologically Plausible Implementations of the Maximum Operation , 2002, Neural Computation.

[26]  Chang Chieh Hang,et al.  The min-max function differentiation and training of fuzzy neural networks , 1996, IEEE Trans. Neural Networks.

[27]  Aristidis Likas.  Reinforcement Learning Using the Stochastic Fuzzy Min–Max Neural Network , 2004, Neural Processing Letters.

[28]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.