On topology, size and generalization of non-linear feed-forward neural networks

Abstract The use of similarity transforms in the design and the interpretation of feed-forward neural networks is proposed. The method is based on the so-called Buckingham Theorem or Pi Theorem and is valid for all neural network function approximation problems which belong to the class of dimensionally homogeneous equations. The new design method allows the a priori determination of a minimal topology size of the first and last network layer. Finally, the correct and unique pointwise generalization capability of the new so-called similarity network topology is proved and illustrated using two examples.

[1]  Juri Pawlowski,et al.  Die Ähnlichkeitstheorie in der physikalisch-technischen Forschung , 1971 .

[2]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[3]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[4]  J. R. Radbill,et al.  Similitude and Approximation Theory , 1986 .

[5]  Stephan Rudolph,et al.  A Methodology for the Systematic Evaluation of Engineering Design Objects , 1995 .

[6]  A statistical procedure for model building in dimensional analysis , 1990 .

[7]  Stephan Rudolph.  On A Genetic Algorithm for the Selection of Optimally Generalizing Neural Network Topologies , 1996 .

[8]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[9]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[10]  L. E. Malvern.  Introduction to the mechanics of a continuous medium , 1969 .

[11]  A. Roli.  Artificial Neural Networks , 2012, Lecture Notes in Computer Science.

[12]  J. Cole,et al.  Similarity methods for differential equations , 1974 .

[13]  John S. Gero,et al.  Effect of Representation on the Performance of Neural Networks in Structural Engineering Applications , 1994 .

[14]  Gerhard Paass,et al.  Künstliche Neuronale Netze: eine Bestandsaufnahme , 1994, Künstliche Intell..

[15]  S. Godtfredsen,et al.  Ullmann ' s Encyclopedia of Industrial Chemistry , 2017 .

[16]  Thomas Kailath,et al.  Rational approximation techniques for analysis of neural networks , 1994, IEEE Trans. Inf. Theory.

[17]  Timothy Masters,et al.  Multilayer Feedforward Networks , 1993 .

[18]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[19]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[20]  E. Buckingham.  The Principle of Similitude , 1915, Nature.

[21]  J. Meinhardt,et al.  Symmetries and differential equations , 1981 .

[22]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[23]  Juma Yousuf Alaydi,et al.  Heat Transfer , 2018, A Concise Manual of Engineering Thermodynamics.

[24]  Gerhard Paass,et al.  Assessing and Improving Neural Network Predictions by the Bootstrap Algorithm , 1992, NIPS.

[25]  F. Kneule,et al.  Die Ähnlichkeitstheorie in der physikalisch-technischen Forschung. Von J. Pawlowski. Springer-Verlag, Berlin-Heidelberg-New York 1971. 1. Aufl., VIII, 172 S., 45 Abb., geb. DM 49, –. , 1972 .

[26]  William A Quade,et al.  Dimensional analysis for economists , 1968 .

[27]  N. C..  Dimensional Analysis , 1932, Nature.

[28]  C. Wieselsberger,et al.  Ergebnisse der aerodynamischen Versuchsanstalt zu Göttingen , 1921 .

[29]  William A Quade,et al.  Dimensional analysis for economists , 1967 .

[30]  Edward K. Blum,et al.  Approximation theory and feedforward networks , 1991, Neural Networks.

[31]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .