Pattern Recognition and Machine Learning

An accelerated stochastic approximation algorithm is developed for the identification of weighting function associated with boundary control in one-dimensional linear distributed-parameter systems. The weighting function is assumed to be variable separable, and each variable is approximated by a finite number of orthonormal polynomials. In the absence of noise, this algorithm will converge in a finite number of steps. For adaptive control, on-line weighting function estimators are developed which use the optimal control function as input. These estimators are functional gradient algorithms based on least square approach. They can be used for estimating weighting function associated with either boundary or distributed control.

[1]  Nils J. Nilsson,et al.  Learning Machines: Foundations of Trainable Pattern-Classifying Systems , 1965 .

[2]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[3]  R. J. Solomonoff,et al.  Some recent work in artificial intelligence , 1966 .

[4]  C. Rattray.  Learning Systems , 1968, Nature.

[5]  Kumpati S. Narendra,et al.  Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria , 1969, IEEE Trans. Syst. Sci. Cybern..

[6]  D. Meeter.  Stochastic Approximation and Nonlinear Regression , 1969 .

[7]  Takao Kurokawa,et al.  Adaptive Pattern Classifiers Applied to Time Optimal Control , 1968 .

[8]  J. Nagumo,et al.  A learning method for system identification , 1967, IEEE Transactions on Automatic Control.

[9]  Henry Cox,et al.  On the estimation of state variables and parameters for noisy dynamic systems , 1964 .

[10]  Y. T. Chien.  The threshold effect of a nonlinear learning algorithm for pattern recognition , 1970, Inf. Sci..

[11]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[12]  A. E. Bryson,et al.  Estimation using sampled data containing sequentially correlated noise. , 1967 .

[13]  M. Aoki,et al.  On a priori error estimates of some identification methods , 1970 .

[14]  J. T. Tou,et al.  Steps toward learning control , 1966 .

[15]  Moriya Oda.  Heuristic method of searching an optimum point of a two-dimensional multimodal criterion function , 1969 .

[16]  Nils J Nilsson,et al.  Research on Intelligent Automata , 1969 .

[17]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[18]  Kahei Nakamura.  Sequential vector space description of discrete control system , 1969 .

[19]  K. Narendra,et al.  Multiparameter self-optimizing systems using correlation techniques , 1964 .

[20]  G. Saridis,et al.  Stochastic approximation algorithms for linear discrete-time system identification , 1967 .

[21]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[22]  Anthony N. Mucciardi,et al.  Evolutionary Pattern Recognition in Incomplete Nonlinear Multithreshold Networks , 1966, IEEE Trans. Electron. Comput..

[23]  Daniel G. Keehn,et al.  A note on learning for Gaussian properties , 1965, IEEE Trans. Inf. Theory.

[24]  King-Sun Fu,et al.  A variable structure automaton used as a multi-modal searching technique , 1965 .

[25]  Jerry M. Mendel.  Gradient, error-correction identification algorithms , 1968, Inf. Sci..

[26]  Thomas B. Sheridan,et al.  Human Control of Remote Computer-Manipulators , 1969, IJCAI.

[27]  V. A. Kovalevsky,et al.  Pattern Recognition: Heuristics or Science? , 1970 .

[28]  F. Downton.  Stochastic Approximation , 1969, Nature.

[29]  M. Aoki.  On some convergence questions in bayesian optimization problems , 1965 .

[30]  H. Sorenson,et al.  Stochastic optimal linear estimation and control , 1972, IEEE Transactions on Automatic Control.

[31]  Jack Holmes.  Two stochastic approximation procedures for identifying linear systems , 1968 .

[32]  George N. Saridis,et al.  Self-Organizing Approach to the Stochastic Fuel Regulator Problem , 1970, IEEE Trans. Syst. Sci. Cybern..

[33]  George N. Saridis,et al.  On a Class of Performance-Adaptive Self-Organizing Control Systems , 1971 .

[34]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[35]  T. Kitagawa.  SUCCESSIVE PROCESSES OF STATISTICAL CONTROLS, (1) , 1952 .

[36]  J. Lamperti,et al.  Some asymptotic properties of Luce's beta learning model , 1960 .

[37]  V. V. Fedyurko,et al.  An algorithm for proving theorems in group theory , 1966 .

[38]  Atsuhiko Noda.  An Inconsistency between the Rate and the Accuracy of the Learning Method for System Identification and its Tracking Characteristics , 1971 .

[39]  R. E. Larson,et al.  Control of a robot in a partially unknown environment , 1970 .

[40]  Richard L. Grimsdale,et al.  A system for the automatic recognition of patterns , 1959 .

[41]  H. K. Hartline,et al.  SPATIAL SUMMATION OF INHIBITORY INFLUENCES IN THE EYE OF LIMULUS, AND THE MUTUAL INTERACTION OF RECEPTOR UNITS , 1958, The Journal of general physiology.

[42]  B. Chandrasekaran,et al.  On Expediency and Convergence in Variable-Structure Automata , 1968, IEEE Trans. Syst. Sci. Cybern..

[43]  H. J. Scudder,et al.  Probability of error of some adaptive pattern-recognition machines , 1965, IEEE Trans. Inf. Theory.

[44]  Stephen S. Yau,et al.  Bayesian Approach to the Optimization of Adaptive Systems , 1967, IEEE Trans. Syst. Sci. Cybern..

[45]  F. Powell,et al.  Predictive adaptive control , 1968 .

[46]  E. Axelband.  The structure of the optimal tracking problem for distributed-parameter systems , 1968 .

[47]  H. C. Khatri,et al.  Identification of Distributed Parameter Systems Using Finite Differences , 1969 .

[48]  M. L. Tsetlin.  On the Behavior of Finite Automata in Random Media , 1961 .

[49]  Julius T. Tou,et al.  Evolution of Heuristics by Human Operators in Control Systems , 1968, IEEE Trans. Syst. Sci. Cybern..

[50]  King-Sun Fu,et al.  A Formulation of Fuzzy Automata and Its Application as a Model of Learning Systems , 1969, IEEE Trans. Syst. Sci. Cybern..

[51]  Kiyoji Asai,et al.  A method for optimizing control of multimodal systems using fuzzy automata , 1971, Inf. Sci..

[52]  K. Krickeberg.  Convergence of martingales with a directed index set , 1956 .

[53]  Bernard Berelson,et al.  Human behavior: An inventory of scientific findings. , 1964 .

[54]  Jin-ichi Nagumo.  Real-Time Display System of Response Characteristics of Manual Control Systems , 1971 .

[55]  B. Widrow,et al.  Generalization and information storage in network of adaline 'neurons' , 1962 .

[56]  Samuel H. Brooks.  A Discussion of Random Methods for Seeking Maxima , 1958 .

[57]  Allen Newell,et al.  Chess-Playing Programs and the Problem of Complexity , 1958, IBM J. Res. Dev..

[58]  G. Saridis,et al.  Identifying solutions of distributed parameter systems by stochastic approximation , 1970, IEEE Transactions on Automatic Control.

[59]  Daniel E. Whitney,et al.  State Space Models of Remote Manipulation Tasks , 1969, IJCAI.

[60]  Abraham Wald,et al.  Statistical Decision Functions , 1951 .

[61]  James R. Slagle,et al.  A Heuristic Program that Solves Symbolic Integration Problems in Freshman Calculus , 1963, JACM.

[62]  Y. T. CHIEN,et al.  On Bayesian Learning and Stochastic Approximation , 1967, IEEE Trans. Syst. Sci. Cybern..

[63]  M. Levin.  Estimation of a system pulse transfer function in the presence of noise , 1964 .

[64]  Kiyoji Asai,et al.  Fuzzy Automaton and its Application to Learnig Control Systems , 1968 .

[65]  Jr. J. Deyst,et al.  A derivation of the optimum continuous linear estimator for systems with correlated measurement noise , 1969 .

[66]  H. Gelernter,et al.  Realization of a geometry theorem proving machine , 1995, IFIP Congress.

[67]  H. Kushner.  Approximations to optimal nonlinear filters , 1967, IEEE Transactions on Automatic Control.

[68]  Katsuhiko Fujii,et al.  Recognition System for Handwritten Letters Simulating Visual Nervous System , 1971 .

[69]  L. Young.  On adaptive manual control. , 1969, Ergonomics.

[70]  Karl K. Pingle,et al.  Computer control of a mechanical arm through visual input , 1968, IFIP Congress.

[71]  Marvin Minsky,et al.  A Selected Descriptor-Indexed Bibliography to the Literature on Artificial Intelligence , 1961 .

[72]  S. W. Kuffler.  Discharge patterns and functional organization of mammalian retina. , 1953, Journal of neurophysiology.

[73]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[74]  A. Bryson,et al.  Linear filtering for time-varying systems using measurements containing colored noise , 1965 .

[75]  Norman M. Abramson,et al.  Learning to recognize patterns in a random environment , 1962, IRE Trans. Inf. Theory.

[76]  George Samuel Levy.  Accelerated gradient pattern recognition : an application to system identification. , 1968 .

[77]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[78]  King-Sun Fu,et al.  On search techniques in switching environments , 1970 .

[79]  J. S. Riordon.  An adaptive automaton controller for discrete-time Markov processes , 1969 .

[80]  T. Bohlin,et al.  On the maximum likelihood method of identification , 1970 .

[81]  M. Norman.  Some convergence theorems for stochastic learning models with distance diminishing operators , 1968 .

[82]  G. Saridis,et al.  A new algorithm for linear system identification , 1968 .

[83]  Anthony Ralston,et al.  Mathematical Methods for Digital Computers , 1960 .

[84]  R. E. Mortensen,et al.  Filtering for stochastic processes with applications to guidance , 1972 .

[85]  Moriya Oda,et al.  A pattern recognizing study of palm reading , 1971 .

[86]  R. E. Goodson,et al.  Identification of Systems Described by Partial Differential Equations , 1966 .

[87]  Richard S. Bucy,et al.  Optimal filtering for correlated noise , 1967 .

[88]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[89]  George Nagy,et al.  An Experimental Study of Machine Recognition of Hand-Printed Numerals , 1968, IEEE Trans. Syst. Sci. Cybern..

[90]  A. L. Samuel,et al.  Some studies in machine learning using the game of checkers. II: recent progress , 1967 .

[91]  Dean C. Karnopp,et al.  Random search techniques for optimization problems , 1963, at - Automatisierungstechnik.

[92]  J. S. Koford,et al.  The use of an adaptive threshold element to design a linear optimal pattern classifier , 1966, IEEE Trans. Inf. Theory.

[93]  George N. Saridis,et al.  A parameter-adaptive control technique , 1969, Autom..

[94]  Sergio Telles Ribeiro,et al.  Random-Pulse Machines , 1967, IEEE Trans. Electron. Comput..

[95]  Bernard Widrow,et al.  Adaptive sampled-data systems , 1960 .

[96]  中村 嘉平.  Fundamental principle and behavior of learntrols , 1966 .