A Formal Framework for Learning in Embedded Systems

Publisher Summary This chapter presents a few problems in creating agents that learn about their world, focusing on methods for measuring the performance of learning algorithms for embedded agents. An embedded agent acting in the world can be seen as continually executing a function that maps its perceptual inputs to its effector outputs. It operates in a cycle, receiving an input from the world, doing some computation, and then generating an output that affects the world. The mapping that it uses may have state or memory, allowing its action at any time to depend on the entire stream of inputs that it has received until that time. The problem of programming an agent to behave correctly in an environment is to choose some behavior given that the rest of the parameters of the agent and environment are fixed. One of the most difficult problems that a learning agent must contend with is noise. A learning behavior is a computational object that learns an appropriate behavior for an agent in an environment. Before comparing algorithms for learning behaviors, the criteria on which they are to be judged must be fixed. There are three major considerations that need to be kept in mind in this regard: (1) correctness, (2) convergence, and (3) time–space complexity.