The design of a model-based multimedia interaction manager

We describe here the conceptual design ofCicero, an application-independent human-computer interaction manager that performs run-time media coordination and allocation, so as to adapt dynamically to the presentation context; knows what it is presenting, so as to maintain coherent extended human-machine dialogues; and is plug-in compatible with host information resources such as “briefing associate” workstations, expert systems, databases, etc., as well as with multiple media such as natural language, graphics, etc. The system design calls for two linked reactive planners that coordinate the actions of the system's media and information sources. To enable presentational flexibility, the capabilities of each medium and the nature of the contents of each information source are semantically modeled as Virtual Devices — abstract descriptions of device I/O capabilities — and abstract information types respectively in a single uniform knowledge representation framework. These models facilitate extensibility by supporting the specification of new interaction behaviors and the inclusion of new media and information sources.