Use of Stochastic AutomataforParameter Self- Optimization withMultimodalPerformanceCriteria

The application ofstochastic automata toadaptive parameter optimization problems isconsidered. Thefundamental problem isthatofrelating theconcepts ofautomata theory and mathematical psychology leaming theory totheusual notion ofaper- formance index inacontrol system. Consideration isgiven toanum- berofpossible automata structures, linear andnonlinear. Onepar- ticular linear modelisderived withoptimal rather thanexpedient properties ofconvergence. Abasic feature ofthis modelisthatitis basedonasystem response setofrewards andinactions, thelatter being substituted forthemorecommonpenalty responses. This choice ofresponse setisdirectly related totheachievement ofthe desired behavior. Simulations aredescribed forthemaximization of multimodal performance functions intentionally constructed to demonstrate theuseofthemethodinsituations whererelative ex- tremaoccur. Anexample isalsogiven oftheautomaton asadirect adaptive controller forathird order control system.