E-learning and machine perception: in pursuit of human-like interaction in computer-based teaching systems

More and more technological advances offer new paradigms for training, allowing novel forms of teaching and learning to be devised. Unfortunately, however, most e-learning systems are still based on complex procedures and little usable interfaces, which may result in frustrating learning experiences, because of users' lack of computer skills. To reduce such risk it would be helpful to go beyond the standard interface usability, towards more natural ways of providing input to the machine. In this paper we consider the possibility of exploiting vision and speech as intuitive communication channels, to enhance the quality of the interaction between users and e-learning platforms. Through an analysis of current research in the fields of vision-based and speech-based perceptive interfaces, we discuss several application scenarios for e-learning, stressing the important role that such natural communication forms could play in improving the interaction process.

[1]  Arun Ross,et al.  Multibiometric systems , 2004, CACM.

[2]  Shumin Zhai,et al.  Manual and gaze input cascaded (MAGIC) pointing , 1999, CHI '99.

[3]  Jayfus T. Doswell Pedagogical embodied conversational agent , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[4]  François Bérard The Perceptual Window: Head Motion as a New Input Stream , 1999, INTERACT.

[5]  The Virtual Human Interface : A Photorealistic Digital , 2022 .

[6]  A. Pentland Social Dynamics: Signals and Behavior , 2004 .

[7]  Maomi Ueno Animated pedagogical agent based on decision tree for e-learning , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[8]  Christine L. Lisetti,et al.  Emotion recognition from physiological signals using wireless sensors for presence technologies , 2004, Cognition, Technology & Work.

[9]  Ashish Kapoor,et al.  Multimodal affect recognition in learning environments , 2005, ACM Multimedia.

[10]  Kentaro Go,et al.  A gaze and speech multimodal interface , 2004, 24th International Conference on Distributed Computing Systems Workshops, 2004. Proceedings..

[11]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[12]  Alex Pentland,et al.  Looking at People: Sensing for Ubiquitous and Wearable Computing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Matthew Turk,et al.  Perceptual user interfaces , 2000 .

[14]  Thomas Rist,et al.  Adding animated presentation agents to the interface , 1997, IUI '97.

[15]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[16]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[17]  Rosalind W. Picard Building HAL: computers that sense, recognize, and respond to human emotion , 2001, IS&T/SPIE Electronic Imaging.

[18]  Alex Pentland,et al.  Perceptual Intelligence , 1999, HUC.

[19]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  D. Goleman Emotional Intelligence: Why It Can Matter More Than IQ , 1995 .

[21]  Marco Porta,et al.  Vision-based user interfaces: methods and applications , 2002, Int. J. Hum. Comput. Stud..

[22]  Christian Gütl,et al.  AdeLE (Adaptive e-Learning with Eye-Tracking): Theoretical Background, System Architecture and Application Scenarios , 2005 .

[23]  Joseph J. Tecce Psychophysiology: human behavior and physiological response (4th ed.) , 2001 .

[24]  Jonathan Klein,et al.  Computers that recognise and respond to user emotion: theoretical and practical implications , 2002, Interact. Comput..

[25]  Fatma Nasoz,et al.  Emotion Recognition from Physiological Signals for Presence Technologies , 2004 .

[26]  Virginio Cantoni,et al.  Perspectives and challenges in e-learning: towards natural interaction paradigms , 2004, J. Vis. Lang. Comput..

[27]  Ronald A. Cole,et al.  Perceptive animated interfaces: first steps toward a new paradigm for human-computer interaction , 2003, Proc. IEEE.

[28]  Kentaro Toyama,et al.  “Look, Ma – No Hands!” Hands-Free Cursor Control with Real-Time 3D Face Tracking , 1998 .

[29]  J. Andreassi Psychophysiology: Human Behavior and Physiological Response , 1980 .

[30]  Nicu Sebe,et al.  MULTIMODAL EMOTION RECOGNITION , 2005 .

[31]  Otman A. Basir,et al.  Adaptive User Interfaces for Intelligent E-Learning: Issues and Trends , 2004, ICEB.

[32]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[33]  Cynthia Breazeal,et al.  Affective Learning — A Manifesto , 2004 .

[34]  Pat Langley,et al.  User modeling in adaptive interfaces , 1999 .

[35]  Rosalind W. Picard,et al.  Automated Posture Analysis for Detecting Learner's Interest Level , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[36]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[37]  Ross Buck,et al.  The communication of emotion , 1984 .

[38]  Andreas Stolcke,et al.  Prosody-based automatic detection of annoyance and frustration in human-computer dialog , 2002, INTERSPEECH.

[39]  Minoru Nakayama,et al.  Frequency analysis of task evoked pupillary response and eye-movement , 2004, ETRA.

[40]  Roel Vertegaal,et al.  Attentive User Interfaces , 2003 .

[41]  Paul P. Maglio,et al.  Attentive agents , 2003, Commun. ACM.

[42]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[43]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Surendra Ranganath,et al.  Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Matthew Turk,et al.  Moving from GUIs to PUIs , 1998 .

[46]  Rosalind W. Picard Computers That Recognize and Respond to User Emotion , 2003, User Modeling.

[47]  Martha E. Crosby,et al.  Assessing Cognitive Load with Physiological Sensors , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[48]  Clark N. Quinn,et al.  Making It Matter to the Learner: e-Motional e-Learning , 2006 .

[49]  Jennifer C. Lai,et al.  Conversational interfaces , 2000, CACM.

[50]  Spike Cramphorn Blink: The Power of Thinking without Thinking / Strangers to Ourselves: Discovering the Adaptive Unconscious , 2006, Journal of Advertising Research.