Integration of audio/visual information for use in human-computer intelligent interaction

Human-computer intelligent interaction (HCII) in virtual environments is a rapidly developing field. Natural human communication is multi-modal, however, most modern computer interfaces rely exclusively on one mode of interaction. We employ a novel approach to integrating multiple modes of human-computer communication. By using auditory and visual features at different levels of integration we explore optimal ways of combining these modalities.

[2]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  John R. Kender,et al.  Toward the use of gesture in traditional user interfaces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[4]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[5]  Vladimir Pavlovic,et al.  Gestural interface to a visual computing environment for molecular biologists , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[6]  David McNeill,et al.  Speech, Gesture, and Discourse. , 1992 .

[7]  Ali Adjoudani,et al.  Audio-visual speech recognition compared across two architectures , 1995, EUROSPEECH.

[8]  Vladimir Pavlovic,et al.  Speech/gesture interface to a visual computing environment for molecular biologists , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[9]  David G. Stork,et al.  Visionary Speech: Looking Ahead to Practical Speechreading Systems , 1996 .

[10]  Michael I. Jordan,et al.  Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.

[11]  Alex Pentland,et al.  Invariant features for 3-D gesture recognition , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[12]  Alexander G. Hauptmann,et al.  Gestures with Speech for Graphic Manipulation , 1993, Int. J. Man Mach. Stud..

[13]  James Llinas,et al.  An introduction to multisensor data fusion , 1997, Proc. IEEE.