DAVE: Detecting Agitated Vocal Events

DAVE is a comprehensive set of event detection techniques to monitor and detect 5 important verbal agitations: asking for help, verbal sexual advances, questions, cursing, and talking with repetitive sentences. The novelty of DAVE includes combining acoustic signal processing with three different text mining paradigms to detect verbal events (asking for help, verbal sexual advances, and questions) which need both lexical content and acoustic variations to produce accurate results. To detect cursing and talking with repetitive sentences we extend word sense disambiguation and sequential pattern mining algorithms. The solutions have applicability to monitoring dementia patients, for online video sharing applications, human computer interaction (HCI) systems, home safety, and other health care applications. A comprehensive performance evaluation across multiple domains includes audio clips collected from 34 real dementia patients, audio data from controlled environments, movies and Youtube clips, online data repositories, and healthy residents in real homes. The results show significant improvement over baselines and high accuracy for all 5 vocal events.

[1]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[2]  V. Patel,et al.  A rating scale for aggressive behaviour in the elderly – the RAGE , 1992, Psychological Medicine.

[3]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[4]  J. Cohen-Mansfield,et al.  Conceptualization of Agitation: Results Based on the Cohen-Mansfield Agitation Inventory and the Agitation Behavior Mapping Instrument , 1997, International Psychogeriatrics.

[5]  A. Stolcke,et al.  Automatic detection of discourse structure for speech recognition and understanding , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[6]  G. Alexopoulos,et al.  A short‐term inpatient program for agitated demented nursing home residents , 2001, International Journal of Geriatric Psychiatry.

[7]  Ted Pedersen,et al.  An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet , 2002, CICLing.

[8]  Torin Monahan,et al.  Somatic Surveillance: Corporeal Control through Information Networks , 2002 .

[9]  Jianyong Wang,et al.  Mining sequential patterns by pattern-growth: the PrefixSpan approach , 2004, IEEE Transactions on Knowledge and Data Engineering.

[10]  Eric Castelli,et al.  A Decision Tree-Based Method for Speech Processing: Question Sentence Detection , 2006, FSKD.

[11]  James R Bettman,et al.  The effects of nonconsciously priming emotion concepts on behavior. , 2007, Journal of personality and social psychology.

[12]  Laurent Besacier,et al.  Automatic question detection: prosodic-lexical features and crosslingual experiments , 2007, INTERSPEECH.

[13]  G. Bachur,et al.  1 Separation of Voiced and Unvoiced using Zero crossing rate and Energy of the Speech Signal , 2008 .

[14]  Dilek Z. Hakkani-Tür,et al.  Any questions? Automatic question detection in meetings , 2009, 2009 IEEE Workshop on Automatic Speech Recognition & Understanding.

[15]  Mari Ostendorf,et al.  Question Detection in Spoken Conversations Using Textual Conversations , 2011, ACL.

[16]  Xiaodong Luo,et al.  Detection of mild cognitive impairment and early-stage dementia with an audio-recorded cognitive scale , 2013, Alzheimer's & Dementia.

[17]  Hee-Jung Yoon,et al.  Kintense: A robust, accurate, real-time and evolving system for detecting aggressive actions from streaming 3D skeleton data , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[18]  Amit P. Sheth,et al.  Cursing in English on twitter , 2014, CSCW.

[19]  Denis Jouvet,et al.  Combining Lexical and Prosodic Features for Automatic Detection of Sentence Modality in French , 2015, SLSP.

[20]  Lianhong Cai,et al.  Question detection from acoustic features using recurrent neural network with gated recurrent unit , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Yelena Perkhounkova,et al.  A Communication Intervention to Reduce Resistiveness in Dementia Care: A Cluster Randomized Controlled Trial , 2016, The Gerontologist.