Speech Recognition Architectures for Multimedia Environments

1 Abstract Computer workstations have recently become powerful enough to support speech recognition entirely in software, but speech recognizers still vary in their functionality, and each vendor offers their own programmatic interface. Developing recognition applications currently means writing to non-portable protocols. As new improved recognizers become available, such applications will need to be rewritten for new protocols. A recognition server can abstract such differences from client applications, while supporting the use of different classes of recognizers available today. This paper describes the design and implementation of an asynchronous recognition server; the asynchronous server allows a client application to continue operation and possibly attend to other input and output events while waiting for recognition to complete. Its internal architecture is based on an object-oriented engine application programming interface (API). The server is designed to support speaker-independent connectedspeech recognition and speaker-dependent, isolatedword recognition paradigms, while offering applications a consistent programmatic interface for accessing this functionality. As recognition engines with better performance arrive, they can be incorporated into to the server, via a standardized engine API, and automatically become available to all client applications without modification. 2 Motivations and Applications Today, sophisticated multimedia environments include not only graphical input but also audio input, and applications employing the use of speech recognition are beginning to appear in such environments. Unfortunately, while many of the state-of-the-art recognition systems offer similar functionality, they also have non