A Theory ofAdaptive Pattern Classifiers

This paper describes error-correction adjustment pro-needsaparametric treatment, thatis,thedistributions cedures fordetermining theweight vector oflinear pattern classifiers mustbelimited tothoseofacertain knownkindwhose undergeneral pattern distribution. Itismainly aimedatclarifying distributions can bespecified bya finite numberof theoretically theperformance ofadaptive pattern classifiers. Inthe case where theloss depends onthedistance between apattern vectorparameters. Ioreover, thediscriminant functions thus andadecision boundary andwheretheaverage risk function is obtained depend directly on allofthepastpatterns so unimodal, itisproved that, bytheprocedures proposed here, the thattheyarenotabletoquicklyfollowthesudden weight vector converges totheoptimal oneevenunder nonseparable changeofthedistributions. In ordertoavoidthese pattern distributions. Thespeedandtheaccuracy ofconvergence areanalyzed, anditisshownthat there isanimportant tradeoff be- shocrt s,bw shapro sennonprametric lanion tween speed andaccuracy ofconvergence. Dynamical behaviors, procedures, bywhichthepresent discriminant function whentheprobability distributions ofpatterns arechanging, arealsoismodified according onlytothepresentmisclassified shown. Thetheory isgeneralized andmadeapplicable tothecase pattern. withgeneral discriminant functions, including piecewise-linear dis- Thesteepest-descent methodisoften usedinorder to criminant functions. minimize a knowvn function. How^ever, inourlearning Index Terms-Accuracy oflearning, adaptive pattern classifier, situation, wecannotobtainthedescending directions of convergence oflearning, learning undernonseparable pattern dis-theaverageriskwhichwe intendtominimize, because tribution, linear decision function, piecewise-linear decision function, theprobability distributions ofthepatterns are un- rapidity oflearning. known.Whatwe can utilize isthepresent pattern only, I.INTRODUCTION whichobeystheunknownprobability distribution. We N ADAPTIVE pattern classifier systemisone shall associate a correction vectortoeachpattern in \ r1of themost typical learning or * suchamannerthattheaverage ofthecorrection vectors /\\O tnemosttyplal larnlgorself-organizing .. )t ~~~~~~ .l 1 isi n 1 nadescending direction. Bytheabovecorrection, it