M Ami a Taxonomy of Multimodal Interaction in the Human Information Processing System

Editors Abstract This document has been prepared in the ESPRIT BRA No. 8579, Multimodal Integration for Advanced Multimedia Interfaces | in the following referred to as MIAMI | in order to serve as a basis for future work. The basic terms which will be used in MIAMI will be deened and an overview on man-machine-interfaces will be given. The term \taxonomy" is used in the following sense, adapted from 217]: \1: the study of the general principles of scientiic classiication: SYSTEMATICS; 2: CLASSIFICATION; specif: orderly clas-siication of plants and animals according to their presumed natural relationships"; but instead of plants and animals, we attempt to classify input and output modalities.

[1]  Joel S. Warm,et al.  Psychology of Perception , 1957 .

[2]  W. Lindemann.  Extension of a binaural cross-correlation model by contralateral inhibition. I. Simulation of lateralization for stationary signals. , 1986, The Journal of the Acoustical Society of America.

[3]  Michael M. Cohen,et al.  Modeling Coarticulation in Synthetic Visual Speech , 1993 .

[4]  Alexander H. Waibel,et al.  Improving connected letter recognition by lipreading , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Parke,et al.  Parameterized Models for Facial Animation , 1982, IEEE Computer Graphics and Applications.

[6]  W. Gaik,et al.  Combined evaluation of interaural time and intensity differences: psychoacoustic results and computer modeling. , 1993, The Journal of the Acoustical Society of America.

[7]  Maurizio Gentilucci,et al.  On gesture and speech , 2015 .

[8]  E. Reed.  The Ecological Approach to Visual Perception , 1989 .

[9]  R. Plamondon,et al.  The relation between pen force and pen-point kinematics in handwriting , 1990, Biological Cybernetics.

[10]  Mark T. Maybury,et al.  Intelligent multimedia interfaces , 1994, CHI Conference Companion.

[11]  D H Warren,et al.  Spatial Localization under Conflict Conditions: Is There a Single Explanation? , 1979, Perception.

[12]  P. Ekman,et al.  Facial action coding system , 2019 .

[13]  P. Ladefoged.  WHAT ARE LINGUISTIC SOUNDS MADE OF , 1980 .

[14]  S. McAdams,et al.  Auditory Cognition. (Book Reviews: Thinking in Sound. The Cognitive Psychology of Human Audition.) , 1993 .

[15]  Stuart K. Card,et al.  Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys, for text selection on a CRT , 1987 .

[16]  Lindsay William Macdonald,et al.  Interacting with Virtual Environments , 1993 .

[17]  S. S. Stevens.  On the psychophysical law. , 1957, Psychological review.

[18]  Rae A. Earnshaw,et al.  Virtual Reality Systems , 1993 .

[19]  Palmer Morrel-Samuels,et al.  Clarifying the Distinction Between Lexical and Gestural Commands , 1990, Int. J. Man Mach. Stud..

[20]  Ken Pimentel,et al.  Virtual reality - through the new looking glass , 1993 .

[21]  Louis D. Braida,et al.  Evaluating the articulation index for auditory-visual input. , 1987, The Journal of the Acoustical Society of America.

[22]  Innes A. Ferguson.  TouringMachines: an architecture for dynamic, rational, mobile agents , 1992 .

[23]  Joseph S. Perkell,et al.  On the Use of Feedback in Speech Production , 1981 .

[24]  Monique Nahas,et al.  Animation of a B-Spline figure , 1988, The Visual Computer.

[25]  M. Halle,et al.  Preliminaries to Speech Analysis: The Distinctive Features and Their Correlates , 1961 .

[26]  Giancarlo Ferrigno,et al.  Automatic analysis of lips and jaw kinematics in VCV sequences , 1989, EUROSPEECH.

[27]  H. Hudde,et al.  Estimation of the area function of human ear canals by sound pressure measurements. , 1983, The Journal of the Acoustical Society of America.

[28]  Abigail Sellen,et al.  A comparison of input devices in element pointing and dragging tasks , 1991, CHI.

[29]  C. Pelachaud.  Communication and coarticulation in facial animation , 1992 .

[30]  J. Vroomen,et al.  Hearing Voices and Seeing Lips. Investigations in the Psychology of Lipreading , 1992 .

[31]  Karun B. Shimoga,et al.  A survey of perceptual feedback issues in dexterous telemanipulation. I. Finger force feedback , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[32]  B.M. Jau,et al.  Anthropomorhic Exoskeleton dual arm/hand telerobot controller , 2002, IEEE International Workshop on Intelligent Robots.

[33]  David Wessel,et al.  Improvisation with Highly Interactive Real-Time Performance Systems , 1991, ICMC.

[34]  Abigail Sellen,et al.  A study in interactive 3-D rotation using 2-D control devices , 1988, SIGGRAPH.

[35]  Dominic W. Massaro,et al.  Synthesis of visible speech , 1990 .

[36]  Noam Chomsky,et al.  The Sound Pattern of English , 1968 .

[37]  Hiroshi Harashima,et al.  A Media Conversion from Speech to Facial Image for Intelligent Man-Machine Interface , 1991, IEEE J. Sel. Areas Commun..

[38]  Wolfgang Felger,et al.  Die Virtuelle Umgebung - Eine neue Epoche in der Mensch-Maschine-Kommunikation, Teil I: Einordnung, Begriffe und Geräte , 1994, Inform. Spektrum.

[39]  Stephen Brewster,et al.  A Detailed Investigation into the Effectiveness of Earcons , 1997 .

[40]  P Bertelson,et al.  Auditory-visual interaction and the timing of inputs , 1987, Psychological research.

[41]  E. Bizzi,et al.  Mechanisms underlying achievement of final head position. , 1976, Journal of neurophysiology.

[42]  Allen Newell,et al.  The keystroke-level model for user performance time with interactive systems , 1980, CACM.

[43]  Hewitt D. Crane,et al.  Pen and voice unite , 1993 .

[44]  Demetri Terzopoulos,et al.  Techniques for Realistic Facial Modeling and Animation , 1991 .

[45]  M. Bodden.  Modeling human sound-source localization and the cocktail-party-effect , 1993 .

[46]  L. Braida.  Crossmodal Integration in the Identification of Consonant Segments , 1991, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[47]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[48]  Klaus Genuit,et al.  Evaluating sound environments with binaural technology-Some basic consideration , 1993 .

[49]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[50]  A. King,et al.  Auditory function: Neurobiological bases of hearing G.M. Edelman W.E. , 1990, Neuroscience.

[51]  Raymond D. Kent,et al.  Coarticulation in recent speech production models , 1977 .

[52]  Christian Abry,et al.  Plateaus, catastrophes and the structuring of vowel systems , 1989 .

[53]  Gary M. Olson,et al.  The growth of cognitive modeling in human-computer interaction since GOMS , 1990 .

[54]  Daniel Thalmann,et al.  Abstract muscle action procedures for human face animation , 1988, The Visual Computer.

[55]  Pietro Morasso,et al.  Self-organizing topographic maps and motor planning , 1994 .

[56]  Joëlle Coutaz,et al.  A design space for multimodal systems: concurrent processing and data fusion , 1993, INTERCHI.

[57]  Giancarlo Ferrigno,et al.  Articulatory dynamics of lips in Italian /'vpv/ and /'vbv/ sequences , 1993, EUROSPEECH.

[58]  P. Ladefoged.  A course in phonetics , 1975 .

[59]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[60]  Abigail Sellen,et al.  Two-handed input in a compound task , 1994, CHI 1994.

[61]  N. Badler,et al.  Linguistic Issues in Facial Animation , 1991 .

[62]  K. Lashley.  The problem of serial order in behavior , 1951 .

[63]  D. H. Warren,et al.  The role of visual-auditory “compellingness” in the ventriloquism effect: Implications for transitivity among the spatial senses , 1981, Perception & psychophysics.

[64]  Sieb G. Nooteboom,et al.  The target theory of speech production , 1970 .

[65]  Tayeb Mohamadi,et al.  Synthèse à partir du texte de visages parlants : réalisation d'un prototype et mesures d'intelligibilité bimodale , 1993 .

[66]  Gillian Rhodes,et al.  Cross-modal effects on visual and auditory object perception , 1984, Perception & psychophysics.

[67]  Michael Good,et al.  Participatory design of a portable torque-feedback device , 1992, CHI.

[68]  C D Marsden,et al.  Latency measurements compatible with a cortical pathway for the stretch reflex in man. , 1973, The Journal of physiology.

[69]  Garner Wr.  An informational analysis of absolute judgments of loudness. , 1953 .

[70]  Axel Mulder.  Virtual Musical Instruments: Accessing the Sound Synthesis Universe as a Performer , 2007 .

[71]  Ravin Balakrishnan,et al.  Virtual hand tool with force feedback , 1994, CHI '94.

[72]  James H. Abbs,et al.  chapter 5 – Peripheral Mechanisms of Speech Motor Control , 1976 .

[73]  Keith Waters,et al.  A muscle model for animation three-dimensional facial expression , 1987, SIGGRAPH.

[74]  Wayne A. Wickelgran.  Context-sensitive coding, associative memory, and serial order in (speech) behavior. , 1969 .

[75]  Makoto Shimojo,et al.  Edge tracing of virtual shape using input device with force feedback , 1992, Systems and Computers in Japan.

[76]  K. K. Neely.  Effect of Visual Factors on the Intelligibility of Speech , 1956 .

[77]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[78]  Caroline Henton,et al.  Saying and seeing it with feeling: techniques for synthesizing visible, emotional speech , 1994, SSW.

[79]  A. Mlcoch,et al.  Speech Production Models as Related to the Concept of Apraxia of Speech , 1980 .

[80]  A A Montgomery,et al.  Auditory and visual contributions to the perception of consonants. , 1974, Journal of speech and hearing research.

[81]  A.,et al.  Cognitive Engineering , 2008, Encyclopedia of GIS.

[82]  F. Lavagetto,et al.  Lipreadable frame animation driven by speech parameters , 1994, Proceedings of ICSIPNN '94. International Conference on Speech, Image Processing and Neural Networks.

[83]  Mohamed Tahar Lallouache,et al.  Un poste "visage-parole" couleur : acquisition et traitement automatique des contours des lèvres , 1991 .

[84]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[85]  N. P. Erber,et al.  Auditory, visual, and auditory-visual recognition of consonants by children with normal and impaired hearing. , 1972, Journal of speech and hearing research.

[86]  A. Benguerel,et al.  Coarticulation of Upper Lip Protrusion in French , 1974, Phonetica.

[87]  David Frohlich,et al.  The Design Space of Interfaces , 1992 .

[88]  M. Radeau.  Auditory-visual spatial interaction and modularity , 1994, Current psychology of cognition = Cahiers de psychologie cognitive : CPC.

[89]  Ning Xiang,et al.  A miniature dummy head for binaural evaluation of tenth-scale acoustic models , 1991 .

[90]  Jean R. Ward,et al.  Digitizer Technology: Performance Characteristics and the Effects on the User Interface , 1987, IEEE Computer Graphics and Applications.

[91]  E. H. Dooijes.  Analysis of handwriting movements , 1983 .

[92]  P. Falzon,et al.  Architecture of a Multimodal Dialogue Interface for Knowledge-Based Systems , 1990 .

[93]  Kerry P. Green,et al.  Exploring the basis of the “McGurk effect”: Can perceivers combine information from a female face and a male voice? , 1990 .

[94]  R. Fox.  Modularity and the Motor Theory of Speech Perception , 1994 .

[95]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[96]  Ruth Campbell,et al.  Tracing Lip Movements: Making Speech Visible , 1988 .

[97]  Shuji Hashimoto,et al.  A computer music system that follows a human conductor , 1991, Computer.

[98]  Wolfgang Felger,et al.  How interactive visualization can benefit from multidimensional input devices , 1992, Electronic Imaging.

[99]  P. Johnson-Laird.  Mental models , 1989 .

[100]  Ching Y. Suen,et al.  The State of the Art in Online Handwriting Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[101]  Leonello Tarabella.  Guest Editor's introduction to the special Issue on Man‐Machine interaction in live performance , 1993 .

[102]  Kristinn R. Thórisson,et al.  Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[103]  Marc Leman,et al.  AI-based Music Signal Applications - A Hybrid Approach , 1997 .

[104]  Q Summerfield,et al.  Use of Visual Information for Phonetic Perception , 1979, Phonetica.

[105]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[106]  M. Pichora-Fuller,et al.  Coarticulation effects in lipreading. , 1982, Journal of speech and hearing research.

[107]  William Colgrove.  Intelligent user interfaces and the Internet , 1995 .

[108]  Gregory J. Wolff,et al.  Neural network lipreading system for improved speech recognition , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[109]  Joëlle Coutaz,et al.  Applying the Wizard of Oz Technique to the Study of Multimodal Systems , 1993, EWHCI.

[110]  Pattie Maes,et al.  Designing autonomous agents: Theory and practice from biology to engineering and back , 1990, Robotics Auton. Syst..

[111]  Joseph T. Chung,et al.  Hyperinstruments: Musically Intelligent and Interactive Performance and Creativity Systems , 1989, ICMC.

[112]  David A. Wroblewski,et al.  Architectural qualities and principles for multimodal and multimedia interfaces , 1992 .

[113]  S. Nishida.  Speech recognition enhancement by lip information , 1986, CHI '86.

[114]  D. Massaro,et al.  Models of integration given multiple sources of information. , 1990, Psychological review.

[115]  Christian Benoît,et al.  A 3-d model of the lips for visual speech synthesis , 1994, SSW.

[116]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[117]  Lance Williams,et al.  Performance-driven facial animation , 1990, SIGGRAPH Courses.

[118]  Hans-Leo Teulings,et al.  Digital recording and processing of handwriting movements , 1984 .

[119]  N. P. Erber.  Interaction of audition and vision in the recognition of oral speech stimuli. , 1969, Journal of speech and hearing research.

[120]  Robert J. Beaton,et al.  User Evaluation Of Cursor-Positioning Devices For 3-D Display Workstations , 1988, Photonics West - Lasers and Applications in Science and Engineering.

[121]  Michael A. Gigante,et al.  Virtual Reality: Definitions, History and Applications , 1993, Virtual Reality Systems.

[122]  John S. Lew,et al.  Optimal Accelerometer Layouts for Data Recovery in Signature Verification , 1980, IBM J. Res. Dev..

[123]  Thecla Schiphorst,et al.  Tools for interaction with the creative process of composition , 1990, CHI '90.

[124]  LAMBERT R. B. SCHOMAKER,et al.  A computational model of cursive handwriting , 1987 .

[125]  J. Hannerz,et al.  Contraction time and voluntary discharge properties of individual short toe extensor motor units in man. , 1979, The Journal of physiology.

[126]  L. E. Murphy.  Absolute judgments of duration. , 1966 .

[127]  H. W. Campbell,et al.  Phoneme recognition by ear and by eye: a distinctive feature analysis , 1974 .

[128]  J Schroeter.  The use of acoustical test fixtures for the measurement of hearing protector attenuation. Part I: Review of previous work and the design of an improved test fixture. , 1986, The Journal of the Acoustical Society of America.

[129]  D W Massaro,et al.  American Psychological Association, Inc. Evaluation and Integration of Visual and Auditory Information in Speech Perception , 2022 .

[130]  I. Scott MacKenzie,et al.  Extending Fitts' law to two-dimensional tasks , 1992, CHI.

[131]  Osamu Fujimura.  Elementary Gestures and Temporal Organization — What Does an Articulatory Constraint Mean? , 1981 .

[132]  A. Meltzoff,et al.  The bimodal perception of speech in infancy. , 1982, Science.

[133]  Motoyuki Akamatsu,et al.  A multi-modal mouse with tactile and force feedback , 1994, Int. J. Hum. Comput. Stud..

[134]  J. Bortz.  Lehrbuch der Statistik , 1979 .

[135]  B.P. Yuhas,et al.  Integration of acoustic and visual speech signals using neural networks , 1989, IEEE Communications Magazine.

[136]  C. Benoît,et al.  A set of French visemes for visual speech synthesis , 1994 .

[137]  Peter Ladefoged,et al.  PHONETIC PREREQUISITES FOR A DISTINCTIVE FEATURE THEORY , 1972 .

[138]  D Goodman,et al.  On the nature of human interlimb coordination. , 1979, Science.

[139]  Wulf Pompetzki.  Psychoakustische Verifikation von Computermodellen zur binauralen Raumsimulation , 1993 .

[140]  R. Campbell,et al.  Hearing by Eye , 1980, The Quarterly journal of experimental psychology.

[141]  Roger B. Dannenberg,et al.  Panel I-computer-generated music and multimedia computing , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[142]  Neil P. McAngus Todd,et al.  The auditory “Primal Sketch”: A multiscale model of rhythmic grouping , 1994 .

[143]  Yves Demazeau,et al.  Principles and techniques for sensor data fusion , 1993, Signal Process..

[144]  Tsuneya Kurihara,et al.  A Transformation Method for Modeling and Animation of the Human Face from Photographs , 1991 .

[145]  N. P. Erber.  Auditory-visual perception of speech. , 1975, The Journal of speech and hearing disorders.

[146]  Peter C. Litwinowicz,et al.  Facial Animation by Spatial Mapping , 1991 .

[147]  S. R. Ellis.  Nature and origins of virtual environments: a bibliographical essay , 1991 .

[148]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[149]  C M Reed,et al.  Research on the Tadoma method of speech communication. , 1983, The Journal of the Acoustical Society of America.

[150]  Kiyoharu Aizawa,et al.  Model-based analysis synthesis image coding (MBASIC) system for a person's face , 1989, Signal Process. Image Commun..

[151]  P. Viviani,et al.  Motor-perceptual interactions , 1992 .

[152]  Marc Leman.  Schema-based tone center recognition of musical signals , 1994 .

[153]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[154]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[155]  D. Papadias,et al.  Computational Imagery , 1992, Cogn. Sci..

[156]  W. Lindemann.  Extension of a binaural cross-correlation model by contralateral inhibition. II. The law of the first wave front. , 1986, The Journal of the Acoustical Society of America.

[157]  P. MacNeilage.  Motor control of serial ordering of speech. , 1970, Psychological review.

[158]  E. Owens,et al.  Visemes observed by hearing-impaired and normal-hearing adult viewers. , 1985, Journal of speech and hearing research.

[159]  Lambert Schomaker,et al.  Using stroke- or character-based self-organizing maps in the recognition of on-line, connected cursive script , 1993, Pattern Recognit..

[160]  Hans J. Charwat.  Lexikon der Mensch-Maschine-Kommunikation , 1992 .

[161]  Luc Steels,et al.  Emergent Frame Recognition and Its Use in Artificial Creatures , 1991, IJCAI.

[162]  R. Wurtz,et al.  Organization of monkey superior colliculus: enhanced visual response of superficial layer cells. , 1976, Journal of neurophysiology.

[163]  Jock D. Mackinlay,et al.  The cognitive coprocessor architecture for interactive user interfaces , 1989, UIST '89.

[164]  Ivan E. Sutherland,et al.  The Ultimate Display , 1965 .

[165]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[166]  Allen A. Montgomery,et al.  ANIMAT: A set of programs to generate, edit, and display sequences of vector-based images , 1982 .

[167]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[168]  R. M. Halsey,et al.  Absolute Judgments of Spectrum Colors , 1956 .

[169]  James Lubker.  Representation and Context Sensitivity , 1981 .

[170]  Paula M. T. Smeele,et al.  The contribution of vision to speech perception , 1991, EUROSPEECH.

[171]  Réjean Plamondon,et al.  An evaluation of motor models of handwriting , 1989, IEEE Trans. Syst. Man Cybern..

[172]  P. L. Adams.  THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[173]  E. Robinson.  Cybernetics, or Control and Communication in the Animal and the Machine , 1963 .

[174]  Georg Geiser.  Mensch-Maschine-Kommunikation , 1990 .

[175]  Paul O'Rorke,et al.  Explaining Emotions , 1994, Cogn. Sci..

[176]  Gerd Hirzinger.  Multisensory shared autonomy and tele-sensor programming - Key issues in space robotics , 1993, Robotics Auton. Syst..

[177]  D C Shepherd,et al.  Visual-neural correlate of speechreading ability in normal-hearing adults: reliability. , 1982, Journal of speech and hearing research.

[178]  Marc Leman,et al.  Introduction to auditory models in music research , 1994 .

[179]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[180]  Denis Baggi,et al.  Readings in computer-generated music , 1992 .

[181]  Stephen Michael Platt,et al.  A structural model of the human face (graphics, animation, object representation) , 1985 .

[182]  H W HAKE,et al.  Multidimensional stimulus differences and accuracy of discrimination. , 1955, Journal of experimental psychology.

[183]  Michael M. Cohen,et al.  Real-time analysis-synthesis and intelligibility of talking faces , 1994, SSW.

[184]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[185]  Tamas Ungvary,et al.  NUNTIUS: A Computer System for the Interactive Composition and Analysis of Music and Dance , 1992 .

[186]  A. Risberg,et al.  Speech , Music and Hearing Quarterly Progress and Status Report Prosody and speechreading , 2007 .

[187]  Brenda Laurel,et al.  PLACEHOLDER: landscape and narrative in virtual environments , 1994, MULTIMEDIA '94.

[188]  Ronald J. Brachman,et al.  An overview of the KL-ONE Knowledge Representation System , 1985 .

[189]  Daniel Thalmann,et al.  The Direction of Synthetic Actors in the Film Rendez-Vous a Montreal , 1987, IEEE Computer Graphics and Applications.

[190]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[191]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[192]  Novia Weiman,et al.  An Evaluation Of Input Devices For 3-D Computer Display Workstations , 1987, Photonics West - Lasers and Applications in Science and Engineering.

[193]  Carol A. Fowler,et al.  Coarticulation and theories of extrinsic timing , 1980 .

[194]  J. Abbs,et al.  Lip and Jaw Motor Control during Speech: Motor Reorganization Responses to External Interference , 1974 .

[195]  David Taylor.  Hearing by Eye: The Psychology of Lip-Reading , 1988 .

[196]  D. Massaro,et al.  Perception of Synthesized Audible and Visible Speech , 1990 .

[197]  R. Hammarberg.  The metaphysics of coarticulation , 1976 .

[198]  M. L. Meeks,et al.  MEASUREMENT OF DYNAMIC DIGITIZER PERFORMANCE , 1990 .

[199]  Y. Hatwell,et al.  Toucher l'espace : la main et la perception tactile de l'espace , 1986 .

[200]  Eric David Petajan,et al.  Automatic Lipreading to Enhance Speech Recognition (Speech Reading) , 1984 .

[201]  N Magnenat Thalmann,et al.  Creating Realistic Three-Dimensional Human Shape Characters for Computer-Generated Films , 1991 .

[202]  B. Wyvill,et al.  Animating speech: an automated approach using speech synthesised by rules , 1988, The Visual Computer.

[203]  Dennis H. Klatt,et al.  Software for a cascade/parallel formant synthesizer , 1980 .

[204]  I. Pollack.  The Information of Elementary Auditory Displays , 1952 .

[205]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[206]  J. P. Lewis,et al.  Automated lip-synch and speech synthesis for character animation , 1987, CHI '87.

[207]  Doug Riecken.  Intelligent agents , 1994, CACM.

[208]  R. L. Elliott.  Computer-generated movies as an analytic tool , 1978 .

[209]  E.R. Brocklehurst.  The NPL Electronic Paper Project , 1991, Int. J. Man Mach. Stud..

[210]  Harry Hollien,et al.  A Neural Model for Language and Speech. , 1978 .

[211]  A..  Temporal Backpropagation for FIR Neural Networks , 2004 .

[212]  David Wessel,et al.  Real-Time Neural Network Processing of Gestural and Acoustic Signals , 1991, ICMC.

[213]  Shen Lin,et al.  A Advanced Telerobotic Control System for a Mobile Robot with Multisensor Feedback , 1995 .

[214]  Hiroo Iwata,et al.  Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator , 1990, SIGGRAPH.

[215]  Ning Xiang,et al.  Binaural scale modelling for auralisation and prediction of acoustics in auditoria , 1993 .

[216]  Jens Blauert,et al.  Principles of binaural room simulation , 1992 .

[217]  Catherine G. Wolf,et al.  The Use of Hand-Drawn Gestures for Text Editing , 1987, Int. J. Man Mach. Stud..

[218]  C. Faure.  Pen and voice interface for incremental design of graphic documents , 1994 .

[219]  Kurzfassung,et al.  Calibrating the Active Stereo Vision System Kastor 1 for Real-time Robot Navigation Karlsruhe Active Stereo Real-time Vision System , 1994 .

[220]  David Goldberg,et al.  Touch-typing with a stylus , 1993, INTERCHI.

[221]  Kim J. Vicente,et al.  Ecological interface design: theoretical foundations , 1992, IEEE Trans. Syst. Man Cybern..

[222]  C. Fowler.  An event approach to the study of speech perception from a direct realist perspective , 1986 .

[223]  David E. Kieras,et al.  An Approach to the Formal Analysis of User Complexity , 1999, Int. J. Man Mach. Stud..

[224]  J. Schroeter,et al.  The use of acoustical test fixtures for the measurement of hearing protector attenuation. Part II: Modeling the external ear, simulating bone conduction, and comparing test fixture and real-ear data. , 1986, The Journal of the Acoustical Society of America.

[225]  Brian Wyvill,et al.  Speech and expression: a computer solution to face animation , 1986 .

[226]  V J Samar,et al.  Visual evoked-response components related to speechreading and spatial skills in hearing and hearing-impaired adults. , 1984, Journal of speech and hearing research.

[227]  Norman I. Badler,et al.  Animating facial expressions , 1981, SIGGRAPH '81.

[228]  Jens Rasmussen,et al.  Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[229]  Antonio Camurri,et al.  Music and Multimedia Knowledge Representation and Reasoning: The HARP System , 1995 .

[230]  Alan Jeffrey Goldschen,et al.  Continuous automatic speech recognition by lipreading , 1993 .

[231]  Roger B. Dannenberg,et al.  Multimedia interface design , 1992 .

[232]  P L Olson,et al.  Perception-Response Time to Unexpected Roadway Hazards , 1986, Human factors.

[233]  David Gradwell,et al.  Human Information Processing , 2017 .

[234]  Dennis H. Klatt,et al.  Speech perception: a model of acoustic–phonetic analysis and lexical access , 1979 .

[235]  J Rhyne.  Dialogue management for gestural interfaces , 1987, COMG.

[236]  L. Steels.  The Artiicial Life Roots of Artiicial Intelligence , 1993 .

[237]  P. N. Kugler,et al.  1 On the Concept of Coordinative Structures as Dissipative Structures: I. Theoretical Lines of Convergence* , 1980 .

[238]  E. Bizzi.  7 Central and Peripheral Mechanisms in Motor Control , 1980 .

[239]  Marie-Luce Viaud.  Animation faciale avec rides d'expression vieillissement et parole , 1992 .

[240]  Dominic W. Massaro,et al.  The motor theory of speech perception revisited , 2008, Psychonomic bulletin & review.

[241]  Joachim Grollmann,et al.  On the Software Structure of User Interface Management Systems , 1989, Eurographics.

[242]  Kiyoharu Aizawa,et al.  Real-time facial action image synthesis system driven by speech and text , 1990, Other Conferences.

[243]  D C Donderi,et al.  The effect of sound on visual apparent movement. , 1983, The American journal of psychology.

[244]  Ben Shneiderman,et al.  Designing the user interface - strategies for effective human-computer interaction, 2nd Edition , 1992 .

[245]  N. P. Erber,et al.  Voice/mouth synthesis and tactual/visual perception of /pa, ba, ma/. , 1978, The Journal of the Acoustical Society of America.

[246]  Cynthia H. Null,et al.  A white paper: NASA virtual environment research, applications, and technology , 1993 .

[247]  H. Hudde,et al.  Measurement of the eardrum impedance of human ears. , 1983, The Journal of the Acoustical Society of America.

[248]  F. I. Parke June,et al.  Computer Generated Animation of Faces , 1972 .

[249]  Kaisa Väänänen,et al.  Gesture Driven Interaction as a Human Factor in Virtual Environments - An Approach with Neural Networks , 1993, Virtual Reality Systems.

[250]  Ben Pinkowski.  LPC spectral moments for clustering acoustic transients , 1993, IEEE Trans. Speech Audio Process..