A smart dialogue-competent monitoring framework supporting people in rehabilitation

In this paper, we present work in progress on the development of a smart monitoring framework to support people with motor disabilities and their caregivers in clinical and non-clinical rehabilitation and care environments. The innovation of the platforms lies in the combination of smart monitoring solutions, such as activity recognition and lifestyle tracking, with an intelligent virtual agent that aims to empower and motivate people in need through personalized feedback and responses, as well as to assist caregivers and clinicians to easily collect information about the patients. The proposed system exploits and combines state-of-the-art technologies in speech recognition and synthesis, knowledge representation and reasoning, dialogue management and sensor data analysis, infusing clinical knowledge and patient history. Aiming for a practical, acceptable solution, the proposed system takes into account aspects of integration, security and privacy.

[1]  Robert Dale,et al.  Building Natural Language Generation Systems: Figures , 2000 .

[2]  Boris Motik,et al.  OWL 2: The next step for OWL , 2008, J. Web Semant..

[3]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[4]  Juliana Miehle,et al.  Challenges for adaptive dialogue management in the KRISTINA project , 2017, ISIAA@ICMI.

[5]  Israel Gannot,et al.  A Method for Automatic Fall Detection of Elderly People Using Floor Vibrations and Sound—Proof of Concept on Human Mimicking Doll Falls , 2009, IEEE Transactions on Biomedical Engineering.

[6]  Kalina Bontcheva,et al.  Improving habitability of natural language interfaces for querying ontologies with feedback and clarification dialogues , 2013, J. Web Semant..

[7]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[8]  Patrick Nguyen,et al.  Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis , 2018, NeurIPS.

[9]  Jerome R. Bellegarda,et al.  State of the art in statistical methods for language and speech processing , 2016, Comput. Speech Lang..

[10]  Nader Karimi,et al.  Automatic Monocular System for Human Fall Detection Based on Variations in Silhouette Area , 2013, IEEE Transactions on Biomedical Engineering.

[11]  D. Luxton.  An Introduction to Artificial Intelligence in Behavioral and Mental Health Care , 2016 .

[12]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[13]  Raphaël Troncy,et al.  LODE: Linking Open Descriptions of Events , 2009, ASWC.

[14]  C. Rougier,et al.  Monocular 3D Head Tracking to Detect Falls of Elderly People , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[15]  Vincent Pollet,et al.  Unit Selection with Hierarchical Cascaded Long Short Term Memory Bidirectional Recurrent Neural Nets , 2017, INTERSPEECH.

[16]  Georgios Meditskos,et al.  Pervasive and Mobile Computing ( ) – Pervasive and Mobile Computing Semantic Web Technologies in Pervasive Computing: a Survey and Research Roadmap , 2022 .

[17]  Diego Calvanese,et al.  The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[18]  Julian Hough.  Incremental Semantics Driven Natural Language Generation with Self-Repairing Capability , 2011, RANLP Student Research Workshop.

[19]  Yiannis Kompatsiaris,et al.  Recognition of Activities of Daily Living for Smart Home Environments , 2013, 2013 9th International Conference on Intelligent Environments.

[20]  Akane Sano,et al.  Intelligent Mobile, Wearable, and Ambient Technologies for Behavioral Health Care , 2016 .

[21]  Michael Wessel,et al.  OntoVPA - An Ontology-Based Dialogue Management System for Virtual Personal Assistants , 2017, IWSDS.

[22]  Heiga Zen,et al.  Statistical parametric speech synthesis using deep neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Cardona Alzate,et al.  Predicción y selección de variables con bosques aleatorios en presencia de variables correlacionadas , 2020 .

[24]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[25]  Heiga Zen,et al.  Speech Synthesis Based on Hidden Markov Models , 2013, Proceedings of the IEEE.

[26]  Emiel Krahmer,et al.  Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation , 2017, J. Artif. Intell. Res..

[27]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[28]  Aldo Gangemi,et al.  Understanding the Semantic Web through Descriptions and Situations , 2003, OTM.

[29]  Amar Ramdane-Cherif,et al.  Multi levels semantic architecture for multimodal interaction , 2013, Applied Intelligence.

[30]  Parma Nand,et al.  GENERATION : A SURVEY AND CLASSIFICATION OF THE EMPIRICAL LITERATURE , 2017 .

[31]  Tara N. Sainath,et al.  A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition , 2018, 2018 IEEE Spoken Language Technology Workshop (SLT).

[32]  Navdeep Jaitly,et al.  Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Duygu Altinok,et al.  An Ontology-Based Dialogue Management System for Banking and Finance Dialogue Systems , 2018, ArXiv.

[34]  Pierre Lison,et al.  Structured Probabilistic Modelling for Dialogue Management , 2014 .

[35]  J. Pilcher,et al.  Sleep quality versus sleep quantity: relationships between sleep and measures of health, well-being and sleepiness in college students. , 1997, Journal of psychosomatic research.

[36]  Yiannis Kompatsiaris,et al.  A Hybrid Approach for Biomedical Relation Extraction Using Finite State Automata and Random Forest-Weighted Fusion , 2017, CICLing.

[37]  Georgios Meditskos,et al.  MetaQ: A knowledge-driven framework for context-aware activity recognition combining SPARQL and OWL 2 activity patterns , 2016, Pervasive Mob. Comput..