Knowledge Compilation to Speed Up Numerical Optimisation

Many important application problems can be formalized as constrained non-linear optimization tasks. However, numerical methods for solving such problems are brittle and do not scale well. This paper describes a method to speed up and increase the reliability of numerical optimization by (a) optimizing the computation of the objective function, and (b) splitting the objective function into special cases that possess differentiable closed forms. This allows us to replace a single inefficient non-gradient-based optimization by a set of efficient numerical gradient-directed optimizations that can be performed in parallel. In the domain of 2-dimensional structural design, this procedure yields a 95% speedup over traditional optimization methods and decreases the dependence of the numerical methods on having a good starting point.

[1]  Thomas G. Dietterich,et al.  Knowledge Compilation to Speed Up Numerical Optimization , 1991, ML.

[2]  Thomas G. Dietterich,et al.  Readings in Machine Learning , 1991 .

[3]  Thomas G. Dietterich,et al.  A model of the mechanical design process based on empirical data , 1988, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[4]  Thomas G. Dietterich.  The EG project: recent progress , 1986 .

[5]  Thomas G. Dietterich,et al.  The Test Incorporation Theory of Problem Solving , 1986 .

[6]  Thomas G. Dietterich,et al.  A data representation for collaborative mechanical design , 1992 .

[7]  Thomas Ellman,et al.  Explanation-based learning: a survey of programs and perspectives , 1989, CSUR.

[8]  Chu-Kia Wang,et al.  Introductory Structural Analysis , 1983 .

[9]  Thomas G. Dietterich,et al.  Forward Chaining Logic Programming with the ATMS , 1987, AAAI.

[10]  Thomas G. Dietterich.  The Test Incorporation Hypothesis and the Weak Methods , 1986 .

[11]  Thomas G. Dietterich,et al.  Learning and Inductive Inference , 1982 .

[12]  Thomas G. Dietterich,et al.  Improving the Performance of Radial Basis Function Networks by Learning Center Locations , 1991, NIPS.

[13]  W. H. Mac Williams.  Keynote address , 2006, AIEE-IRE '51.

[14]  Thomas G. Dietterich,et al.  What Good Are Experiments? , 1989, ML.

[15]  Ac Palmer,et al.  Optimizing the shape of pin-jointed structures , 1970 .

[16]  Thomas G. Dietterich,et al.  Learning to Predict Sequences , 1985 .

[17]  Caroline Nan Koff.  A specialized ATMS for equivalence relations , 1988 .

[18]  Thomas G. Dietterich,et al.  Exploiting functional vocabularies to learn structural descriptions , 1986 .

[19]  Steven Minton,et al.  Quantitative Results Concerning the Utility of Explanation-based Learning , 1988, Artif. Intell..

[20]  Thomas G. Dietterich.  Learning About Systems That Contain State Variables , 1984, AAAI.

[21]  Thomas G. Dietterich,et al.  A Comparative Review of Selected Methods for Learning from Examples , 1983 .

[22]  Thomas G. Dietterich.  Limitations on Inductive Learning , 1989, ML.

[23]  Thomas G. Dietterich,et al.  FORLOG : A Logic-based Architecture for Design , 1987 .

[24]  Thomas G. Dietterich,et al.  Performance Comparison between Human Engineered & Machine Learned Letter-to-sound Rules for English: a Machine Learning Success Story , 1993 .

[25]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[26]  Thomas G. Dietterich,et al.  Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs , 1991, AAAI.

[27]  Garret N. Vanderplaats,et al.  Numerical Optimization Techniques for Engineering Design: With Applications , 1984 .

[28]  Thomas G. Dietterich,et al.  Learning and Generalization of Characteristic Descriptions: Evaluation Criteria and Comparative Review of Selected Methods , 1979, IJCAI.

[29]  Thomas G. Dietterich,et al.  On Learning More Concepts , 1992, ML.

[30]  Thomas G. Dietterich,et al.  A Comparison of Dynamic Reposing and Tangent Distance for Drug Activity Prediction , 1993, NIPS.

[31]  Thomas G. Dietterich,et al.  Locally Adaptive Nearest Neighbor Algorithms , 1993, NIPS.

[32]  Thomas G. Dietterich,et al.  A Comparative Study of ID3 and Backpropagation for English Text-to-Speech Mapping , 1990, ML.

[33]  J. E. Gordon,et al.  Structures: Or Why Things Don't Fall Down , 1978 .

[34]  Thomas G. Dietterich.  Applying General Induction Methods to the Card Game Eleusis , 1980, AAAI.

[35]  Thomas G. Dietterich,et al.  Selecting Appropriate Representations for Learning from Examples , 1986, AAAI.

[36]  Thomas G. Dietterich,et al.  Efficient Algorithms for Identifying Relevant Features , 1992 .

[37]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[38]  Thomas G. Dietterich,et al.  The Role of the Critic in Learning Systems , 1984 .