Integration and Control of Reactive Visual Processes 1

This paper describes a new approach to the integration and control of continuously operating visual processes. Visual processes are expressed as transformations which map signals from virtual sensors into commands for devices. These transformations define reactive processes which tightly couple perception and action. Such transformations may be used to control robotic devices, including fixation an active binocular head, as well as the to select and control the processes which interpret visual data. This method takes inspiration from so-called "behavioural" approaches to mobility and manipulation. However, unlike most previous work, we define reactive transformations at the level of virtual sensors and device controllers. This permits a system to integrate a large number of perceptual processes and to dynamically compose sequences of such processes to perform visual tasks. The transition between visual processes is mediated by signals from a supervisory controller as well as signals obtained from perception. This method offers the possibility of constructing vision systems with large numbers of visual abilities in a manner which is both scalable and learnable. After a review of related work in mobility and manipulation, we adapt the reactive process framework to computer vision. We define reactive visual processes which map information from virtual sensors to device commands. We discuss the selection and control of reactive visual processes to accomplish visual tasks. We then illustrate this approach with a system which detects and fixates on different classes of moving objects.