Models of Learning Systems.

Abstract : The terms adaptation, learning, concept-formation, induction, self-organization, and self-repair have all been used in the context of learning system (LS) research. In this article, three distinct approaches to machine learning and adaptation are considered: (i) the adaptive control approach, (ii) the pattern recognition approach, and (iii) the artificial intelligence approach. Progress in each of these areas is summarized in the first part of the article. In the next part a general model for learning systems is presented that allows characterization and comparison of individual algorithms and programs in all of these areas. The model details the functional components felt to be essential for any learning system, independent of the techniques used for its construction, and the specific environment in which it operates. Specific examples of learning systems are described in terms of the model. (Author)

[1]  Leonard Uhr,et al.  A Pattern Recognition Program That Generates, Evaluates and Adjusts its Own Operators , 1899 .

[2]  A. Oettinger.  CXXIV. Programming a digital computer to learn , 1952 .

[3]  M. Mead,et al.  Cybernetics , 1953, The Yale Journal of Biology and Medicine.

[4]  William M. Smith,et al.  A Study of Thinking , 1956 .

[5]  Richard M. Friedberg,et al.  A Learning Machine: Part I , 1958, IBM J. Res. Dev..

[6]  R. Kálmán.  Design of a Self-Optimizing Control System , 1958, Journal of Fluids Engineering.

[7]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[8]  John McCarthy,et al.  Programs with common sense , 1960 .

[9]  Oliver G. Selfridge,et al.  Pattern recognition by machine , 1960 .

[10]  E. A. Feigenbaum,et al.  The simulation of verbal learning behavior , 1899, IRE-AIEE-ACM '61 (Western).

[11]  Leonard Uhr,et al.  A pattern recognition program that generates, evaluates, and adjusts its own operators , 1961, IRE-AIEE-ACM '61 (Western).

[12]  Earl B. Hunt,et al.  Programming a model of human concept formulation , 1899, IRE-AIEE-ACM '61 (Western).

[13]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[14]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[15]  Allen Newell,et al.  Learning, Generality and Problem Solving , 1962, IFIP Congress.

[16]  John H. Holland,et al.  Outline for a Logical Theory of Adaptive Systems , 1962, JACM.

[17]  Arthur L. Samuel,et al.  Some studies in machine learning using the game of checkers" in computers and thought eds , 1995 .

[18]  H A SIMON,et al.  HUMAN ACQUISITION OF CONCEPTS FOR SEQUENTIAL PATTERNS. , 1963, Psychological review.

[19]  Edward S. Johnson.  An information-processing model of one kind of problem solving. , 1964 .

[20]  J. Sklansky,et al.  Correspondence: Adaptation, learning, self-repair, and feedback , 1964 .

[21]  T. G. Evans.  A program for the solution of a class of geometric-analogy intelligence-test questions , 1964 .

[22]  L. Uhr,et al.  Computers in the study of learning: BOGART—A discovery and induction program for games , 1965, ACM '65.

[23]  B. Shackcloth,et al.  Synthesis of Model Reference Adaptive Systems by Liapunov's Second Method , 1965 .

[24]  D. L. Johnson,et al.  Computer learning in theorem proving , 1966 .

[25]  Viktor Mikhaĭlovich Glushkov,et al.  An Introduction to Cybernetics , 1957, The Mathematical Gazette.

[26]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[27]  J. S. Koford,et al.  The use of an adaptive threshold element to design a linear optimal pattern classifier , 1966, IEEE Trans. Inf. Theory.

[28]  J. Agassi.  The Structure of Scientific Revolutions (review) , 1966 .

[29]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[30]  A. L. Samuel,et al.  Some studies in machine learning using the game of checkers. II: recent progress , 1967 .

[31]  Ya Tsypkin,et al.  Self-learning--What is it? , 1968 .

[32]  Elliot B. Koffman.  Learning Games through Pattern Recognition , 1968, IEEE Trans. Syst. Sci. Cybern..

[33]  Laurent Siklossy,et al.  Natural language learning by computer , 1968 .

[34]  N V Findler,et al.  Computer simulation of a self-preserving and learning organism. , 1969, The Bulletin of mathematical biophysics.

[35]  J. Andel.  Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[36]  Satosi Watanabe,et al.  Methodologies of Pattern Recognition , 1969 .

[37]  C. Churchman.  The Role of Weltanschauung in Problem Solving and Inquiry , 1970 .

[38]  Jerry M. Mendel,et al.  Adaptive, learning, and pattern recognition systems : theory and applications , 1970 .

[39]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[40]  George M. White.  Machine learning through signature trees: applications to human speech , 1970 .

[41]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[42]  Bruce G. Buchanan,et al.  On generality and problem solving: a case study using the DENDRAL program , 1970 .

[43]  Donald A. Waterman,et al.  Generalization Learning Techniques for Automating the Learning of Heuristics , 1970, Artif. Intell..

[44]  Karl Johan Åström,et al.  BOOK REVIEW SYSTEM IDENTIFICATION , 1994, Econometric Theory.

[45]  JOHN F. Young.  Machine Intelligence , 1971, Nature.

[46]  Yaakov Bar-Shalom,et al.  An actively adaptive control for linear systems with random parameters via the dual control approach , 1972, CDC 1972.

[47]  B. Mark.  On Self Tuning Regulators , 1972 .

[48]  Richard Fikes,et al.  Learning and Executing Generalized Robot Plans , 1993, Artif. Intell..

[49]  G. Saridis,et al.  Parameter identification and control of linear discrete-time systems , 1972 .

[50]  J. Feldman,et al.  A SURVEY OF RESULTS IN GRAMMATICAL INFERENCE , 1972 .

[51]  Kumpati S. Narendra,et al.  Adaptive, learning, and pattern recognition systems: Theory and applications , 1972 .

[52]  Michael H. Smith.  A learning program which plays partnership dominoes , 1973, CACM.

[53]  Gerald J. Sussman,et al.  A Computational Model of Skill Acquisition , 1973 .

[54]  Bernard Widrow,et al.  The "Rubber-Mask" Technique II. Pattern Storage and Recognition , 1973 .

[55]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[56]  Lee D. Erman,et al.  A model and a system for machine recognition of speech , 1973 .

[57]  R. Carroll,et al.  Survey of adaptive control using Liapunov design , 1973 .

[58]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[59]  I. D. Landau,et al.  Design of multivariable adaptive model following control systems , 1973, Autom..

[60]  H. Simon,et al.  Problem solving and rule induction: A unified view. , 1974 .

[61]  I. D. Landau,et al.  A survey of model reference adaptive techniques - Theory and applications , 1973, Autom..

[62]  Arnold K. Griffith,et al.  A Comparison and Evaluation of Three Machine Learning Procedures as Applied to the Game of Checkers , 1974, Artif. Intell..

[63]  Laveen N. Kanal,et al.  Patterns in pattern recognition: 1968-1974 , 1974, IEEE Trans. Inf. Theory.

[64]  R. Monopoli.  Model reference adaptive control with an augmented error signal , 1974 .

[65]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[66]  P. Wortman,et al.  Pattern Recognition, Learning, and Thought , 1974 .

[67]  Kumpati S. Narendra,et al.  Learning Automata - A Survey , 1974, IEEE Trans. Syst. Man Cybern..

[68]  William A. Woods,et al.  Control concepts in a speech understanding system , 1975 .

[69]  Donald A. Waterman,et al.  Adaptive Production Systems , 1975, IJCAI.

[70]  Steve Hardy,et al.  Synthesis Of LISP Functions From Examples , 1975, IJCAI.

[71]  Frederick Hayes-Roth,et al.  An Automatically Compilable Recognition Network For Structured Patterns , 1975, IJCAI.

[72]  Petr Hájek.  On Logics of Discovery , 1975, MFCS.

[73]  M. Bauer,et al.  A Basis For The Acquisition Of Procedures From Protocols , 1975, IJCAI.

[74]  Björn Wittenmark,et al.  Stochastic adaptive control methods: a survey , 1975 .

[75]  Taylor L. Booth,et al.  Grammatical Inference: Introduction and Survey - Part I , 1975, IEEE Trans. Syst. Man Cybern..

[76]  R. Solomonott.  Inductive Inference Theory - A Unified Approach to Problems in Pattern Recognition and Artificial Intelligence , 1975, IJCAI.

[77]  Taylor L. Booth,et al.  Grammatical Inference: Introduction and Survey-Part I , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  David E. Shaw,et al.  Inferring LISP Programs From Examples , 1975, IJCAI.

[79]  Victor Lesser,et al.  Organization of the Hearsay II speech understanding system , 1975 .

[80]  Douglas B. Lenat,et al.  AM, an artificial intelligence approach to discovery in mathematics as heuristic search , 1976 .

[81]  Azriel Rosenfeld,et al.  Some Experiments in Grammatical Inference , 1976 .

[82]  C. Cordell Green.  The design of the PSI program synthesis system , 1976, ICSE '76.

[83]  Jacques Pitrat,et al.  Realization of a Program Learning to find Combinations at Chess , 1976 .

[84]  L. Siklóssy,et al.  Procedural Learning in Worlds of Robots , 1976 .

[85]  L. Hasdorff.  Gradient Optimization and Nonlinear Control , 1976 .

[86]  R. B. Asher,et al.  Bibliography on adaptive control systems , 1976 .

[87]  Patrick Henry Winston,et al.  The psychology of computer vision , 1976, Pattern Recognit..

[88]  Charles L. Hedrick,et al.  Learning Production Systems from Examples , 1976, Artif. Intell..

[89]  R. Banerji.  Learning to Solve Games and Puzzles , 1976 .

[90]  E. Tse,et al.  Actively adaptive control for nonlinear stochastic systems , 1976, Proceedings of the IEEE.

[91]  Bruce G. Buchanan,et al.  Scientific Theory Formation by Computer , 1976 .

[92]  George Charles Stockman,et al.  A problem-reduction approach to the linguistic analysis of waveforms. , 1977 .

[93]  Tom M. Mitchell,et al.  Version Spaces: A Candidate Elimination Approach to Rule Learning , 1977, IJCAI.

[94]  Nicholas V. Findler,et al.  Studies in machine cognition using the game of poker , 1977, CACM.

[95]  H. Simon.  Scientific Discovery and the Psychology of Problem Solving , 1977 .

[96]  C. R. Johnson.  Input Matching: A New Procedure For Adaptive Control Design , 1977 .

[97]  Tom Michael Mitchell,et al.  Model-directed learning of production rules , 1977, SGAR.

[98]  Edward M. Riseman,et al.  Levels of Pattern Description in Learning , 1977, IJCAI.

[99]  Pat Langley,et al.  BACON: A Production System That Discovers Empirical Laws , 1977, IJCAI.

[100]  Raymond Aubin,et al.  Strategies for Mechanizing Structural Induction , 1977, IJCAI.

[101]  Ryszard S. Michalski,et al.  Inductive inference of VL decision rules , 1977, SGAR.

[102]  Allen Newell,et al.  An instructable production system: basic design issues , 1977, SGAR.

[103]  Edward M. Riseman,et al.  Knowledge-directed learning , 1977, SGAR.

[104]  Frederick Hayes-Roth,et al.  Knowledge acquisition from structural descriptions , 1977, IJCAI 1977.

[105]  D. A. Waterman.  EXEMPLARY PROGRAMMING IN RITA , 1978 .

[106]  Carl,et al.  PROGRAMMING A MODEL OF HUMAN CONCEPT , 2022 .