An Open Sensing and Acting Platform for Context-Aware Affective Support in Ambient Intelligent Educational Settings

Supporting learners affectively while carrying out stressful educational activities is an open research issue. It requires the appropriate infrastructure for recognizing emotional states and reacting accordingly in runtime. In this paper, we describe the open platform that we have implemented (named AICARP v2) to detect changes in physiological signals that can be associated with stressful situations, and when this happens, it recommends the learner to relax by delivering modulated sensorial support in terms of light, sound, or vibration at a relaxation breath rate. In this way, by taking advantage of ambient intelligence, the learner can perceive the recommended action without interrupting the learning activity (in this case, practicing the oral exam of a second language). The signal acquisition of the system (which combines sensors from Libellium e-Health platform with others integrated ad hoc) has been compared with a commercial system (J&J Engineering I-330-C2), obtaining similar outcomes as to identifying significant changes in the physiological signals when the learner experiments an emotional reaction. However, the cost of AICARP v2 is much lower, and at the same time, it is open hardware and flexible and, thus, has the advantage of providing runtime data processing. User studies have served to evaluate participants' perception of the sensorial support, as well as to calibrate the delivery rule and to evaluate the effectiveness of the support provided to them.

[1]  Emre Ertin,et al.  Continuous inference of psychological stress from sensory measurements collected in the natural environment , 2011, Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks.

[2]  Miguel Arevalillo-Herráez,et al.  Eigenexpressions: Emotion Recognition Using Multiple Eigenspaces , 2013, IbPRIA.

[3]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[4]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[5]  Andreas Dunser,et al.  Combining EEG with Pupillometry to Improve Cognitive Workload Detection , 2015, Computer.

[6]  Olga C. Santos,et al.  Toward interactive context-aware affective educational recommendations in computer-assisted language learning , 2016, New Rev. Hypermedia Multim..

[7]  Juan Carlos Augusto,et al.  Learning patterns in ambient intelligence environments: a survey , 2010, Artificial Intelligence Review.

[8]  Sergio Salmeron-Majadas,et al.  Filtering of Spontaneous and Low Intensity Emotions in Educational Contexts , 2015, AIED.

[9]  Angel Torrado-Carvajal.  A Cross-Platform EEG Monitoring and Processing Application for Mobile Devices , 2013 .

[10]  Sergio Salmeron-Majadas,et al.  Towards Emotion Detection in Educational Scenarios from Facial Expressions and Body Movements through Multimodal Approaches , 2014, TheScientificWorldJournal.

[11]  Juan Antonio Hernández Tamames,et al.  Objective Assessment of Olfactory Function Using Functional Magnetic Resonance Imaging (fMRI) , 2010, IEEE Transactions on Instrumentation and Measurement.

[12]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[13]  Jean-Marc Dewaele,et al.  Effects of Trait Emotional Intelligence and Sociobiographical Variables on Communicative Anxiety and Foreign Language Anxiety among Adult Multilinguals: A Review and Empirical Investigation , 2008 .

[14]  Egon L. van den Broek,et al.  Affective Man-Machine Interface: Unveiling Human Emotions through Biosignals , 2009, BIOSTEC.

[15]  Marco Porta,et al.  e5Learning, an E-Learning Environment Based on Eye Tracking , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[16]  Cynthia Breazeal,et al.  Affective Learning — A Manifesto , 2004 .

[17]  Arne Jönsson,et al.  Wizard of Oz studies: why and how , 1993, IUI '93.

[18]  O. John,et al.  Big Five Inventory , 2012, Encyclopedia of Personality and Individual Differences.

[19]  Yvonne Rogers,et al.  Buzzing to play: lessons learned from an in the wild study of real-time vibrotactile feedback , 2011, CHI.

[20]  Sergio Salmeron-Majadas,et al.  A Methodological Approach to Eliciting Affective Educational Recommendations , 2014, 2014 IEEE 14th International Conference on Advanced Learning Technologies.

[21]  Luca Chittaro,et al.  Stress Detection Using Physiological Sensors , 2015, Computer.

[22]  Heye Zhang,et al.  Assessment of Biofeedback Training for Emotion Management Through Wearable Textile Physiological Monitoring System , 2015, IEEE Sensors Journal.

[23]  Matthias Steinbauer,et al.  Using Big Data for Emotionally Intelligent Mobile Services through Multi-Modal Emotion Recognition , 2015, ICOST.

[24]  Pedro C. Santana-Mancilla,et al.  Towards Smart Education: Ambient Intelligence in the Mexican Classrooms , 2013 .

[25]  Javier Macías Guarasa,et al.  Acoustic emotion recognition using dynamic Bayesian networks and multi-space distributions , 2009, INTERSPEECH.

[26]  Juan Antonio Hernández Tamames,et al.  Changing communications within hospital and home health care , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[27]  O. John,et al.  Los Cinco Grandes across cultures and ethnic groups: multitrait multimethod analyses of the Big Five in Spanish and English. , 1998, Journal of personality and social psychology.