A learning machine with monologue

The learning machine STeLLA has been developed considerably since it was first described in 1962. Although it has not been built in entirety, it has been simulated on computers in many different forms and with many different problems; special circuits have been developed for its construction. The need to give the machine a “monologue” ability arose because the machine cannot learn to solve problems for which the input to the machine is inadequate to distinguish successive steps made by the machine. A simple example of this is the problem of learning to perform a sequence of actions when the sequence is not related to information received by the machine from its problem environment. In this paper an introductory description of the STeLLA machine is given with the help of a particular problem which is then used to illustrate the generation of control policies by a dual machine. The dual STeLLA comprises two interdependent STeLLA machines or STeLLAments; one interacts directly with the problem environment, while the second STeLLAment interacts with an auxiliary “vocal” environment to provide monologue. Monologue is used to supplement the information from the problem invironment with information from the vocal environment. The STeLLAment interacting directly with the vocal environment has its input supplemented by information from the control policy of the other machine. The two machines are co-ordinated further by giving reward to the dual machine as a whole. The procedure of counting is the kind of monologue which can be used to distinguish a sequence of steps that cannot be distinguished by successive inputs from the problem environment. This is illustrated in the paper by considering a problem that requires STeLLA “to walk across a dark courtyard”. Monologue is not restricted to such counting of steps, but can take the form of more sophisticated “symbol” sequences. By allowing an operator to inject sequences into the monologue of the machine, an elementary form of dialogue could be set up. It would be a special feature of the dialogue that no predetermined language was imposed on the participants. Examples of the use of monologue have been worked out and more complicated situations are being programmed for computer simulation both in the monologue and dialogue form.