Numeric Law Discovery Using Neural Networks

This paper proposes a new connectionist approach to numeric law discovery; i.e., neural networks (law-candidates) are trained by using a newly invented second-order learning algorithm based on a quasi-Newton method, called BPQ, and the MDL criterion selects the most suitable from law-candidates. The main advantage of our method over previous work of symbolic or connectionist approach is that it can efficiently discover numeric laws whose power values are not restricted to integers. Experiments showed that the proposed method works well in discovering such laws even from data containing irrelevant variables or a small amount of noise.

[1]  C. Lee Giles,et al.  Learning with Product Units , 1994, NIPS.

[2]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[3]  Richard S. Sutton,et al.  Learning Polynomial Functions by Feature Construction , 1991, ML.

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  M. Moller,et al.  Supervised learning on large redundant training sets , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[6]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[7]  Brian Falkenhainer,et al.  6 – INTEGRATING QUANTITATIVE AND QUALITATIVE DISCOVERY IN THE ABACUS SYSTEM , 1990 .

[8]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[9]  Philip E. Gill,et al.  Practical optimization , 1981 .

[10]  Terence D. Sanger,et al.  Basis-Function Trees as a Generalization of Local Variable Selection Methods , 1990, NIPS.

[11]  Kazumi Saito,et al.  Partial BFGS Update and Efficient Step-Length Calculation for Three-Layer Neural Networks , 1997, Neural Computation.

[12]  Herbert A. Simon,et al.  Scientific discovery: compulalional explorations of the creative process , 1987 .

[13]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[14]  Jan M. Zytkow,et al.  Data-Driven Approaches to Empirical Discovery , 1989, Artif. Intell..

[15]  Pat Langley,et al.  A Robust Approach to Numeric Discovery , 1990, ML.