Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation

Explicitly alerting users is not always an optimal intervention, especially when they are not motivated to obey. For example, in video-based learning, learners who are distracted from the video would not follow an alert asking them to pay attention. Inspired by the concept of Mindless Computing, we propose a novel intervention approach, Mindless Attractor, that leverages the nature of human speech communication to help learners refocus their attention without relying on their motivation. Specifically, it perturbs the voice in the video to direct their attention without consuming their conscious awareness. Our experiments not only confirmed the validity of the proposed approach but also emphasized its advantages in combination with a machine learning-based sensing module. Namely, it would not frustrate users even though the intervention is activated by false-positive detection of their attentive state. Our intervention approach can be a reliable way to induce behavioral change in human–AI symbiosis.

[1]  G. L. Trager Paralanguage : A first approximation , 1958 .

[2]  Thad Starner,et al.  BuzzWear: alert perception in wearable tactile displays on the wrist , 2010, CHI.

[3]  K. Goldschmidt The COVID-19 Pandemic: Technology use to Support the Wellbeing of Children , 2020, Journal of Pediatric Nursing.

[4]  Dinesh Babu Jayagopi,et al.  Predicting student engagement in classrooms using facial behavioral cues , 2017, MIE@ICMI.

[5]  Robert J Zatorre,et al.  Neural specializations for speech and pitch: moving beyond the dichotomies , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[6]  F. D. Horowitz,et al.  The effects of intonation on infant attention: the role of the rising intonation contour , 1983, Journal of Child Language.

[7]  Anastasia Kuzminykh,et al.  Classification of Functional Attention in Video Meetings , 2020, CHI.

[8]  Cary Stothart,et al.  The attentional cost of receiving a cell phone notification. , 2015, Journal of experimental psychology. Human perception and performance.

[9]  Meltem Huri Baturay An Overview of the World of MOOCs , 2015 .

[10]  Alexandra Chouldechova,et al.  A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores , 2020, CHI.

[11]  John Seely Brown,et al.  The coming age of calm technolgy , 1997 .

[12]  P. Brennan,et al.  Human factors recognition at virtual meetings and video conferencing: how to get the best performance from yourself and others , 2020, British Journal of Oral and Maxillofacial Surgery.

[13]  Fred G. Martin,et al.  Will massive open online courses change how we teach? , 2012, CACM.

[14]  Xiang Xiao,et al.  Context and cognitive state triggered interventions for mobile MOOC learning , 2016, ICMI.

[15]  Vassilis Athitsos,et al.  A Realistic Dataset and Baseline Temporal Model for Early Drowsiness Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Cewu Lu,et al.  RMPE: Regional Multi-person Pose Estimation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  M. Kerres Against All Odds: Education in Germany Coping with Covid-19 , 2020, Postdigital Science and Education.

[18]  Patrick Jermann,et al.  A gaze-based learning analytics model: in-video visual feedback to improve learner's attention in MOOCs , 2016, LAK.

[19]  Crispin R. Coombs,et al.  Will COVID-19 be the tipping point for the Intelligent Automation of work? A review of the debate and implications for research , 2020, International Journal of Information Management.

[20]  Shinnosuke Takamichi,et al.  Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device , 2019, 10th ISCA Workshop on Speech Synthesis (SSW 10).

[21]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[22]  Ó. Gonçalves,et al.  Paying attention to my voice or yours: An ERP study with words , 2015, Biological Psychology.

[23]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[24]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[25]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[26]  Susanne Boll,et al.  Towards reducing alarm fatigue: peripheral light pattern design for critical care alarms , 2018, NordiCHI.

[27]  Andrew Olney,et al.  Gaze tutor: A gaze-reactive intelligent tutoring system , 2012, Int. J. Hum. Comput. Stud..

[28]  Anastasia Kuzminykh,et al.  Low Engagement As a Deliberate Practice of Remote Participants in Video Meetings , 2020, CHI Extended Abstracts.

[29]  Yi Xu,et al.  Speech melody as articulatorily implemented communicative functions , 2005, Speech Commun..

[30]  Fang Chen,et al.  Effects of Uncertainty and Cognitive Load on User Trust in Predictive Decision Making , 2017, INTERACT.

[31]  Ronald E. Rice,et al.  Evaluating video as a technology for informal communication , 1992, CHI.

[32]  Fernando Poyatos Paralanguage: A linguistic and interdisciplinary approach to interactive speech and sounds , 1993 .

[33]  Jean-Paul Imbert,et al.  Red Alert: A Cognitive Countermeasure to Mitigate Attentional Tunneling , 2020, CHI.

[34]  Andrej Kosir,et al.  Predicting students’ attention in the classroom from Kinect facial and body features , 2017, EURASIP J. Image Video Process..

[35]  P. Belin,et al.  Electrophysiological markers of voice familiarity , 2006, The European journal of neuroscience.

[36]  B. Wansink,et al.  Portion Size Me: Downsizing Our Consumption Norms , 2007, Journal of the American Dietetic Association.

[37]  Sidney K. D'Mello,et al.  "Out of the Fr-Eye-ing Pan": Towards Gaze-Based Models of Attention during Learning with Technology in the Classroom , 2017, UMAP.

[38]  Philip J. Guo,et al.  How video production affects student engagement: an empirical study of MOOC videos , 2014, L@S.

[39]  Keita Higuchi,et al.  BBeep: A Sonic Collision Avoidance System for Blind Travellers and Nearby Pedestrians , 2019, CHI.

[40]  Stephen C. Nettelhorst,et al.  The effect of advertisement choice, sex, and need for cognition on attention , 2012, Comput. Hum. Behav..

[41]  Charles B. Hodges,et al.  Modeling Students' Attention in the Classroom using Eyetrackers , 2019, ACM Southeast Regional Conference.

[42]  Mohammad Rafayet Ali,et al.  CoCo , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[43]  Berkeley J. Dietvorst,et al.  Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err , 2014, Journal of experimental psychology. General.

[44]  Steve Whittaker,et al.  Rethinking video as a technology for interpersonal communications: theory and design implications , 1995, Int. J. Hum. Comput. Stud..

[45]  Mark Weiser,et al.  Designing Calm Technology , 2004 .

[46]  Massimo Zancanaro,et al.  Overt or subtle? Supporting group conversations with automatically targeted directives , 2014, IUI.

[47]  L Guo,et al.  Agricultural machinery safety alert system using ultrasonic sensors. , 2002, Journal of agricultural safety and health.

[48]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[49]  Mitsuru Kodama,et al.  Digitally transforming work styles in an era of infectious disease , 2020, International Journal of Information Management.

[50]  Irene Kotsia,et al.  RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Susan G. Hill,et al.  Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In A , 1989 .

[52]  Alexander Travis Adams,et al.  Mindless computing: designing technologies to subtly influence behavior , 2015, UbiComp.

[53]  Tomoki Toda,et al.  Implementation of Computationally Efficient Real-Time Voice Conversion , 2012, INTERSPEECH.

[54]  Fang Chen,et al.  User Trust Dynamics: An Investigation Driven by Differences in System Performance , 2017, IUI.