Planning, acting, and sensor fusion

Abstract An intelligent agent must understand its sur-roundings by integrating sensory data from many sources over time. This integration typi-cally consists of processing raw data into ab- stract models that fuse the data from manysensors into a consistent interpretation. This process is often quite complex when attemptedwith raw data because noise, uncertainty, and missing information create ambiguities that cannot be resolved until after an interpretationis chosen. The very same problems exist in gen-erating a consistent interpretation of data overtime, in particular, identifying an object as hay- ing been "seen" before. This paper suggeststhat sensor interpretation and model building are active processes driven by an agent's goals, and that many sensor fusion issues are really issues in planning and acting. An object identi-fication system based on consciously gatheringappropriate new data is presented as an exam-pie of doing active task directed data fusion. 1 Introduction