Sensory Interactive Robots * *Contribution of the U.S. National Bureau of Standards, not subject to copyright.

For robots to operate effectively in the partially unconstrained environment of manufacturing, they must be equipped with control systems that have sensory capabilities. This paper describes a control system that consists of three parallel cross coupled hierarchies. First is a control hierarchy which decomposes high level tasks into primitive actions. Second is a sensory processing hierarchy that analyses data from the environment. Third is a world model hierarchy which generates expectations. These are compared against the sensory data at each level of the sensory processing hierarchy. Deviations between expected and observed data is used by the control hierarchy to modify its task decomposition strategies so as to generate sensory-interaction goal-directed behavior. This system has been implemented on a research robot, using a network of microcomputers and real-time vision system mounted on the robot wrist.