Sensing Technologies for Autism Spectrum Disorder Screening and Intervention

This paper reviews the state-of-the-art in sensing technologies that are relevant for Autism Spectrum Disorder (ASD) screening and therapy. This disorder is characterized by difficulties in social communication, social interactions, and repetitive behaviors. It is diagnosed during the first three years of life. Early and intensive interventions have been shown to improve the developmental trajectory of the affected children. The earlier the diagnosis, the sooner the intervention therapy can begin, thus, making early diagnosis an important research goal. Technological innovations have tremendous potential to assist with early diagnosis and improve intervention programs. The need for careful and methodological evaluation of such emerging technologies becomes important in order to assist not only the therapists and clinicians in their selection of suitable tools, but to also guide the developers of the technologies in improving hardware and software. In this paper, we survey the literatures on sensing technologies for ASD and we categorize them into eye trackers, movement trackers, electrodermal activity monitors, tactile sensors, vocal prosody and speech detectors, and sleep quality assessment devices. We assess their effectiveness and study their limitations. We also examine the challenges faced by this growing field that need to be addressed before these technologies can perform up to their theoretical potential.

[1]  R. Epstein,et al.  Sleep Patterns in Autistic Children , 1999, Journal of autism and developmental disorders.

[2]  David J. Grelotti Clinical Manual for the Treatment of Autism , 2008 .

[3]  H. Nagaraja,et al.  How accurately does wrist actigraphy identify the states of sleep and wakefulness? , 2001, Sleep.

[4]  S. Blakemore,et al.  The application of eye‐tracking technology in the study of autism , 2007, The Journal of physiology.

[5]  M Toussaint,et al.  First-night effect in normal subjects and psychiatric inpatients. , 1995, Sleep.

[6]  Hoda Eydgahi,et al.  Design and evaluation of iCalm : a novel, wrist-worn, low-power, low-cost, wireless physiological sensor module , 2008 .

[7]  T. Dijkstra,et al.  Emotional language processing in autism spectrum disorders: a systematic review , 2015, Front. Hum. Neurosci..

[8]  Shuzhi Sam Ge,et al.  Force and motion analyses of the human patting gesture for robotic social touching , 2011, 2011 IEEE 5th International Conference on Cybernetics and Intelligent Systems (CIS).

[9]  A. Sadeh,et al.  Estimating sleep patterns with activity monitoring in children and adolescents: how many nights are necessary for reliable measures? , 1999, Sleep.

[10]  John-John Cabibihan,et al.  Cute and soft: baby steps in designing robots for children with autism , 2012, WASA '12.

[11]  Charles A. Nelson,et al.  Eye-Tracking, Autonomic, and Electrophysiological Correlates of Emotional Face Processing in Adolescents with Autism Spectrum Disorder , 2013, Journal of autism and developmental disorders.

[12]  Ahmed H. Tewfik,et al.  Automatic characterization and detection of behavioral patterns using linear predictive coding of accelerometer sensor data , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[13]  Shuzhi Sam Ge,et al.  Towards Humanlike Social Touch for Prosthetics and Sociable Robotics: Three-Dimensional Finite Element Simulations of Synthetic Finger Phalanges , 2009, FIRA RoboWorld Congress.

[14]  John-John Cabibihan,et al.  Illusory Sense of Human Touch From a Warm and Soft Artificial Hand , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Ming-Zher Poh,et al.  Wearable sensors: Opportunities and challenges for low-cost health care , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[16]  Mark H. Johnson,et al.  Precursors to Social and Communication Difficulties in Infants At-Risk for Autism: Gaze Following and Attentional Engagement , 2012, Journal of autism and developmental disorders.

[17]  F. Volkmar,et al.  Speech and prosody characteristics of adolescents and adults with high-functioning autism and Asperger syndrome. , 2001, Journal of speech, language, and hearing research : JSLHR.

[18]  Matthew S. Goodwin,et al.  Recognizing stereotypical motor movements in the laboratory and classroom: a case study with children on the autism spectrum , 2009, UbiComp.

[19]  G. Stores,et al.  Sleep patterns and sleep disorders in children with autistic spectrum disorders: insights using parent report and actigraphy , 2004, Developmental medicine and child neurology.

[20]  Fabio Placidi,et al.  Sleep abnormalities in mentally retarded autistic subjects: Down's syndrome with mental retardation and normal subjects , 1999, Brain and Development.

[21]  John-John Cabibihan,et al.  Effects of the Artificial Skin's Thickness on the Subsurface Pressure Profiles of Flat, Curved, and Braille Surfaces , 2014, IEEE Sensors Journal.

[22]  Patricia A. Keating,et al.  Voicesauce: A Program for Voice Analysis , 2009, ICPhS.

[23]  B. Goodlin-Jones,et al.  The use of actigraphy to study sleep disorders in preschoolers: some concerns about detection of nighttime awakenings. , 2008, Sleep.

[24]  Rosalind W. Picard,et al.  Empatica E3 — A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition , 2014, 2014 4th International Conference on Wireless Mobile Communication and Healthcare - Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH).

[25]  Aude Billard,et al.  Measuring gaze of children with autism spectrum disorders in naturalistic interactions , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[26]  John-John Cabibihan,et al.  Using robot animation to promote gestural skills in children with autism spectrum disorders , 2016, J. Comput. Assist. Learn..

[27]  Paolo Dario,et al.  The Uncanny Valley and the Search for Human Skin-Like Materials for a Prosthetic Fingertip , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[28]  John-John Cabibihan,et al.  Affective Tele-touch , 2012, ICSR.

[29]  James M. Rehg,et al.  Detecting bids for eye contact using a wearable camera , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[30]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[31]  Matthew S. Goodwin,et al.  Automated Detection of Stereotypical Motor Movements , 2011, Journal of autism and developmental disorders.

[32]  S. Harris,et al.  Changes in language development among autistic and peer children in segregated and integrated preschool settings , 1990, Journal of autism and developmental disorders.

[33]  E. Saemundsen,et al.  Children diagnosed with autism spectrum disorder before or after the age of 6 years , 2011 .

[34]  Loong Fah Cheong,et al.  Modeling the Temporality of Saliency , 2014, ACCV.

[35]  John-John Cabibihan The influence of gazing and gestures of a storytelling robot on its persuasive power , 2011 .

[36]  M. Prior,et al.  The sleep/wake rhythm in children with autism , 1995, European Child & Adolescent Psychiatry.

[37]  Sachiko Shimizu,et al.  Body movement analysis during sleep for children with ADHD using video image processing , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[38]  H. Critchley Electrodermal responses: what happens in the brain. , 2002, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[39]  P. Hobson,et al.  The Cradle of Thought , 2002 .

[40]  Shuzhi Sam Ge,et al.  Towards Humanlike Social Touch for Prosthetics and Sociable Robotics: Handshake Experiments and Finger Phalange Indentations , 2009, FIRA RoboWorld Congress.

[41]  S. Steinhauer,et al.  Respiratory sinus arrhythmia as an index of emotional response in young adults. , 2004, Psychophysiology.

[42]  D. Muir,et al.  The effect of manipulating maternal behavior during an interaction on three- and six-month-olds' affect and attention. , 1988, Child development.

[43]  Rosalind W. Picard Future affective technology for autism and emotion communication , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[44]  Tracy Vaillancourt,et al.  A prospective study of autistic-like traits in unaffected siblings of probands with autism spectrum disorder. , 2013, JAMA psychiatry.

[45]  John-John Cabibihan,et al.  Object shape discrimination using sensorized glove , 2013, 2013 10th IEEE International Conference on Control and Automation (ICCA).

[46]  F. Volkmar,et al.  Defining and quantifying the social phenotype in autism. , 2002, The American journal of psychiatry.

[47]  Katarzyna Chawarska,et al.  Gaze Response to Dyadic Bids at 2 Years Related to Outcomes at 3 Years in Autism Spectrum Disorders: A Subtyping Analysis , 2013, Journal of Autism and Developmental Disorders.

[48]  Haizhou Li,et al.  Towards an Effective Design of Social Robots , 2011, Int. J. Soc. Robotics.

[49]  Joseph Piven,et al.  Abnormal Use of Facial Information in High-Functioning Autism , 2007, Journal of autism and developmental disorders.

[50]  Tzyy-Ping Jung,et al.  Assisting autistic children with wireless EOG technology , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[51]  Connor Kerns,et al.  Sleep in Children with Autism Spectrum Disorder , 2017, Current Psychiatry Reports.

[52]  P. Linkowski,et al.  The first-night effect may last more than one night. , 2001, Journal of psychiatric research.

[53]  S. Rogers,et al.  Visual Fixation Patterns during Reciprocal Social Interaction Distinguish a Subgroup of 6-Month-Old Infants At-Risk for Autism from Comparison Infants , 2007, Journal of autism and developmental disorders.

[54]  Shuzhi Sam Ge,et al.  Synthetic Skins with Humanlike Warmth , 2010, ICSR.

[55]  Giorgio Metta,et al.  Touch sensors for humanoid hands , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[56]  Tiago H. Falk,et al.  Very early detection of Autism Spectrum Disorders based on acoustic analysis of pre-verbal vocalizations of 18-month old toddlers , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[57]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[58]  W. Stone,et al.  Same but Different: 9‐Month‐Old Infants at Average and High Risk for Autism Look at the Same Facial Features but Process Them Using Different Brain Mechanisms , 2012, Autism research : official journal of the International Society for Autism Research.

[59]  S. Rogers,et al.  Gaze behavior and affect at 6 months: predicting clinical outcomes and language development in typically developing infants and infants at risk for autism. , 2009, Developmental science.

[60]  Eric Schopler,et al.  The research basis for autism intervention , 2001 .

[61]  A. Schuler,et al.  Emerging language in autistic children , 1980 .

[62]  S. Warren,et al.  What Automated Vocal Analysis Reveals About the Vocal Production and Language Learning Environment of Young Children with Autism , 2010, Journal of autism and developmental disorders.

[63]  Matthew S. Goodwin,et al.  iCalm: Wearable Sensor and Network Architecture for Wirelessly Communicating and Logging Autonomic Activity , 2010, IEEE Transactions on Information Technology in Biomedicine.

[64]  C. Gillberg,et al.  Gaze Performance in Children with Autism Spectrum Disorder when Observing Communicative Actions , 2012, Journal of autism and developmental disorders.

[65]  C. Pollak,et al.  The role of actigraphy in the study of sleep and circadian rhythms. , 2003, Sleep.

[66]  Geoffrey Bird,et al.  Attention does not modulate neural responses to social stimuli in autism spectrum disorders , 2006, NeuroImage.

[67]  L. Kanner Autistic disturbances of affective contact. , 1968, Acta paedopsychiatrica.

[68]  Shuzhi Sam Ge,et al.  Artificial Skin Ridges Enhance Local Tactile Shape Discrimination , 2011, Sensors.

[69]  Chrystopher L. Nehaniv,et al.  Title of paper : KASPAR – A Minimally Expressive Humanoid Robot for Human-Robot Interaction Research , 2009 .

[70]  Charis Eng,et al.  Development of an Objective Autism Risk Index Using Remote Eye Tracking. , 2016, Journal of the American Academy of Child and Adolescent Psychiatry.

[71]  Filomena O. Soares,et al.  Automatic detection of stereotyped hand flapping movements: Two different approaches , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[72]  Brian Scassellati,et al.  How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism , 2005, ISRR.

[73]  Medi Nazar,et al.  Pointing Gestures for a Robot Mediated Communication Interface , 2009, ICIRA.

[74]  John-John Cabibihan,et al.  Toward Soft, Robust Robots for Children with Autism Spectrum Disorder , 2015, FinE-R@IROS.

[75]  Kerstin Dautenhahn,et al.  Using a Humanoid Robot to Elicit Body Awareness and Appropriate Physical Interaction in Children with Autism , 2015, Int. J. Soc. Robotics.

[76]  R. Ferri,et al.  Sleep in subjects with autistic disorder: a neurophysiological and psychological study , 2000, Brain and Development.

[77]  Steve Warren,et al.  Sensors and instrumentation for unobtrusive sleep quality assessment in autistic children , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[78]  José Lozada,et al.  Tactile Emotions: A Vibrotactile Tactile Gamepad for Transmitting Emotional Messages to Children with Autism , 2012, EuroHaptics.

[79]  Ann C. Halbower,et al.  Sleep disorders in children. , 2003, Current opinion in pulmonary medicine.

[80]  Antonio Cisternino,et al.  The FACE of autism , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[81]  Christopher Gillberg,et al.  Visualization and Analysis of Eye Movement Data from Children with Typical and Atypical Development , 2013, Journal of Autism and Developmental Disorders.

[82]  Tony Charman,et al.  The development of face orienting mechanisms in infants at-risk for autism , 2013, Behavioural Brain Research.

[83]  John H. L. Hansen,et al.  Discrete-Time Processing of Speech Signals , 1993 .

[84]  F. Shic,et al.  Decreased Spontaneous Attention to Social Scenes in 6-Month-Old Infants Later Diagnosed with Autism Spectrum Disorders , 2013, Biological Psychiatry.

[85]  Bjørn Bjorvatn,et al.  Sleep disturbances in adolescents and young adults with autism and Asperger syndrome , 2005, Autism : the international journal of research and practice.

[86]  C. Guilleminault,et al.  Comparison of actigraphic, polysomnographic, and subjective assessment of sleep parameters in sleep-disordered patients. , 2001, Sleep medicine.

[87]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[88]  Tony Charman,et al.  What you see is what you get: contextual modulation of face scanning in typical and atypical development , 2013, Social cognitive and affective neuroscience.

[89]  P. Hancock,et al.  Do Faces Capture the Attention of Individuals with Williams Syndrome or Autism? Evidence from Tracking Eye Movements , 2009, Journal of autism and developmental disorders.

[90]  G. Dawson Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder , 2008, Development and Psychopathology.

[91]  Rosalind W. Picard,et al.  Affective Computing and Autism , 2006, Annals of the New York Academy of Sciences.

[92]  Kent Larson,et al.  The Design of a Portable Kit of Wireless Sensors for Naturalistic Data Collection , 2006, Pervasive.

[93]  L. Wing,et al.  Asperger's syndrome: a clinical account , 1981, Psychological Medicine.

[94]  Frans W Cornelissen,et al.  The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[95]  Sven Bölte,et al.  Lack of Visual Orienting to Biological Motion and Audiovisual Synchrony in 3-Year-Olds with Autism , 2013, PloS one.

[96]  Aude Billard,et al.  WearCam: A head mounted wireless camera for monitoring gaze attention and for the diagnosis of developmental disorders in young children , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[97]  R. Joseph,et al.  Holistic and part-based face recognition in children with autism. , 2003, Journal of child psychology and psychiatry, and allied disciplines.

[98]  Max Hirshkowitz,et al.  Practice parameters for the role of actigraphy in the study of sleep and circadian rhythms: an update for 2002. , 2003, Sleep.

[99]  Katarzyna Chawarska,et al.  Context modulates attention to social scenes in toddlers with autism. , 2012, Journal of child psychology and psychiatry, and allied disciplines.

[100]  Marcelo H. Ang,et al.  Why Robots? A Survey on the Roles and Benefits of Social Robots in the Therapy of Children with Autism , 2013, International Journal of Social Robotics.

[101]  A. Richdale,et al.  Sleep problems in autism spectrum disorders: prevalence, nature, & possible biopsychosocial aetiologies. , 2009, Sleep medicine reviews.

[102]  Shuzhi Sam Ge,et al.  Design and development of Nancy, a social robot , 2011, 2011 8th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[103]  J. Piven,et al.  Visual Scanning of Faces in Autism , 2002, Journal of autism and developmental disorders.

[104]  P. Dario,et al.  The Conformance Test for Robotic/Prosthetic Fingertip Skins , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[105]  Simon Baron-Cohen,et al.  A test of central coherence theory: Can adults with high-functioning autism or Asperger syndrome integrate objects in context? , 2001 .

[106]  Zhengchen Zhang,et al.  Telerobotic Pointing Gestures Shape Human Spatial Cognition , 2012, Int. J. Soc. Robotics.

[107]  Michael Morales,et al.  Responding to Joint Attention Across the 6- Through 24-Month Age Period and Early Language Acquisition , 2000 .

[108]  Shanti S. Thirumalai,et al.  Rapid Eye Movement Sleep Behavior Disorder in Children With Autism , 2002, Journal of child neurology.

[109]  Laurent Mottron,et al.  Atypical sleep architecture and the autism phenotype. , 2005, Brain : a journal of neurology.

[110]  Andy Adler,et al.  Validation of the Empatica E4 wristband , 2016, 2016 IEEE EMBS International Student Conference (ISC).

[111]  Tony Charman,et al.  Infant Neural Sensitivity to Dynamic Eye Gaze Is Associated with Later Emerging Autism , 2012, Current Biology.

[112]  J. Lagopoulos Electrodermal activity , 2007, Acta Neuropsychiatrica.

[113]  Sue Peppé,et al.  Prosody in autism spectrum disorders: a critical review. , 2003, International journal of language & communication disorders.

[114]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[115]  A. Billard,et al.  Investigating Gaze of Children with ASD in Naturalistic Settings , 2012, PloS one.

[116]  N. V. Thakor,et al.  Bio-mimetic strategies for tactile sensing , 2013, 2013 IEEE SENSORS.

[117]  David S Mandell,et al.  Explaining differences in age at autism spectrum disorder diagnosis: A critical review , 2014, Autism : the international journal of research and practice.

[118]  Kerstin Dautenhahn,et al.  Investigating tactile event recognition in child-robot interaction for use in autism therapy , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[119]  Brian Scassellati,et al.  Prosody recognition in male infant-directed speech , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[120]  Philip Barker,et al.  The triad of impairment in autism revisited. , 2009, Journal of child and adolescent psychiatric nursing : official publication of the Association of Child and Adolescent Psychiatric Nurses, Inc.

[121]  Nobumasa Kato,et al.  Atypical gaze patterns in children and adults with autism spectrum disorders dissociated from developmental changes in gaze behaviour , 2010, Proceedings of the Royal Society B: Biological Sciences.

[122]  Yanna Song,et al.  A Pilot Study of Shoulder Placement for Actigraphy in Children , 2012, Behavioral sleep medicine.

[123]  Aung Aung Phyo Wai,et al.  Sleeping patterns observation for bedsores and bed-side falls prevention , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[124]  Johnj Mceachln,et al.  Long-term Outcome for Children with Autism Who Received Early Intensive Behavioral Treatment , 2022 .

[125]  Mark H. Johnson,et al.  Neural Correlates of Eye-Gaze Detection in Young Children with Autism , 2005, Cortex.

[126]  R. Jordan Autistic Spectrum Disorders: An Introductory Handbook for Practitioners , 1999 .

[127]  Kerstin Dautenhahn,et al.  Tactile Interactions with a Humanoid Robot: Novel Play Scenario Implementations with Children with Autism , 2014, Int. J. Soc. Robotics.

[128]  Agata Rozga,et al.  Automatic assessment of problem behavior in individuals with developmental disabilities , 2012, UbiComp.

[129]  Hiroshi Ishii,et al.  Design of haptic interfaces for therapy , 2009, CHI.

[130]  A. E. Cook,et al.  Face processing in children with autism , 2007, Autism : the international journal of research and practice.

[131]  John-John Cabibihan,et al.  Physiological Responses to Affective Tele-Touch during Induced Emotional Stimuli , 2017, IEEE Transactions on Affective Computing.