Decision theoretic resource management for intelligent environments

Motivation: Natural human-computer interaction in an intelligent environment requires a rich multi-modal interface. Ideally, the environment should allow user interaction via speech, gesture, movement, and other means. Similarly, the environment should speak and give visual responses, at the very least. Achieving such interaction abilities is complicated, involving many distinct hardware and software components. For example, in order to hear and respond to a user’s question, the environment must have a microphone, audio input controller, natural language interpreter, textual question answering system, speech synthesizer, audio output controller, and speakers. Increasing the interaction complexity increases the number of components required and possible interactions among them. Clearly, without an effective means of managing these components, their use quickly becomes infeasible.