Context-aware decision making under uncertainty for voice-based control of smart home

This paper presents a framework to build home automation systems reactive to voice for improved comfort and autonomy at home. The focus of this paper is on the context-aware decision process which must reason from uncertain facts inferred from real sensor data. This framework for building context aware systems uses a hierarchical knowledge model so that different inference modules can communicate and reason with same concepts and relations. The context-aware decision module is based on a Markov Logic Network, a recent approach which make it possible to benefit from formal logical representation and to model uncertainty of this knowledge. In this work, uncertainty of the decision model has been learned from data. Although some expert systems are able to deal with uncertainty, the Markov Logic Network approach brings a unified theory for dealing with logical entailment, uncertainty and missing data. Moreover, the ability to use a priori knowledge and to learn weights and structure from data make this model appealing to address the challenge of adaptation of expert systems to new applications. Finally, the framework has been implemented in an on-line system which has been evaluated in a real smart home with real naive users. Results of the experiment show the interest of context-aware decision making and the advantages of a statistical relational model for the framework.

[1]  Pedro M. Domingos,et al.  Efficient Belief Propagation for Utility Maximization and Repeated Inference , 2010, AAAI.

[2]  C. Aitken,et al.  The logic of decision , 2014 .

[3]  Manuel P. Cuéllar,et al.  A fuzzy ontology for semantic modelling and recognition of human behaviour , 2014, Knowl. Based Syst..

[4]  Alex Mihailidis,et al.  Development of an automated speech recognition interface for personal emergency response systems , 2009, Journal of NeuroEngineering and Rehabilitation.

[5]  Tom J. Moir,et al.  From science fiction to science fact: A Smart-House interface using speech technology and a photo-realistic avatar , 2008, 2008 15th International Conference on Mechatronics and Machine Vision in Practice.

[6]  Michel Vacher,et al.  Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions , 2011, INTERSPEECH.

[7]  Atta Badii,et al.  CompanionAble: integrated cognitive assistive and domotic companion robotic systems for ability and security , 2009 .

[8]  Manuel P. Cuéllar,et al.  A survey on ontologies for human behavior recognition , 2014, ACM Comput. Surv..

[9]  Ilias Maglogiannis,et al.  Enabling human status awareness in assistive environments based on advanced sound and motion data classification , 2008, PETRA '08.

[10]  A. Schmidt,et al.  Ontology-Centred Design of an Ambient Middleware for Assisted Living : The Case of SOPRANO * , 2007 .

[11]  Alexander Artikis,et al.  Event Recognition for Unobtrusive Assisted Living , 2014, SETN.

[12]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[13]  Brigitte Meillon,et al.  Evaluation of a Context-Aware Voice Interface for Ambient Assisted Living , 2015, ACM Trans. Access. Comput..

[14]  Petros Maragos,et al.  The DIRHA simulated corpus , 2014, LREC.

[15]  Francis Jambon,et al.  Une plateforme usage pour l'intégration de l'informatique ambiante dans l'habitat. L'appartement Domus , 2013, Tech. Sci. Informatiques.

[16]  Brigitte Meillon,et al.  Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects , 2011, Personal and Ubiquitous Computing.

[17]  Heiner Stuckenschmidt,et al.  Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships , 2011, Pervasive Mob. Comput..

[18]  Brigitte Meillon,et al.  The Sweet-Home speech and multimodal corpus for home automation interaction , 2014, LREC.

[19]  Arkady B. Zaslavsky,et al.  A probabilistic context-aware approach for quality of experience measurement in pervasive systems , 2011, SAC.

[20]  Michel Vacher,et al.  Embedded Implementation of Distress Situation Identification through Sound Analysis , 2008 .

[21]  Ian Horrocks,et al.  Handbook of Knowledge Representation Edited Description Logics 3.1 Introduction , 2022 .

[22]  Abdul Rahman Ramli,et al.  A rule-based framework for heterogeneous subsystems management in smart home environment , 2009, IEEE Transactions on Consumer Electronics.

[23]  Chris D. Nugent,et al.  A Logical Framework for Behaviour Reasoning and Assistance in a Smart Home , 2008 .

[24]  Michel Vacher,et al.  Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges , 2011, Int. J. E Health Medical Commun..

[25]  Yun Peng,et al.  Belief Update in Bayesian Networks Using Uncertain Evidence , 2006, 2006 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'06).

[26]  Roy H. Campbell,et al.  An infrastructure for context-awareness based on first order logic , 2003, Personal and Ubiquitous Computing.

[27]  Sung-Bae Cho,et al.  Fusion of Modular Bayesian Networks for Context-Aware Decision Making , 2012, HAIS.

[28]  Heidi Christensen,et al.  homeService: Voice-enabled assistive technology in the home using cloud-based automatic speech recognition , 2013, SLPAT.

[29]  Bin Hu,et al.  Rule Strategies for Intelligent Context-Aware Systems: The Application of Conditional Relationships in Decision-Support , 2011, 2011 International Conference on Complex, Intelligent, and Software Intensive Systems.

[30]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[31]  Alessandra Mileo,et al.  Reasoning support for risk prediction and prevention in independent living , 2010, Theory and Practice of Logic Programming.

[32]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[33]  Stephen S. Yau,et al.  Hierarchical situation modeling and reasoning for pervasive computing , 2006, The Fourth IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems, and the Second International Workshop on Collaborative Computing, Integration, and Assurance (SEUS-WCCIA'06).

[34]  Alessandra Mileo,et al.  Support for context-aware monitoring in home healthcare , 2009, J. Ambient Intell. Smart Environ..

[35]  Michel Vacher,et al.  Sound Environment Analysis in Smart Home , 2012, AmI.

[36]  Berardina De Carolis,et al.  C@sa: Intelligent Home Control and Simulation , 2004, International Conference on Computational Intelligence.

[37]  Siegfried Handschuh,et al.  Ontology-based situation recognition for context-aware systems , 2013, I-SEMANTICS '13.

[38]  Pedro M. Domingos,et al.  Efficient Weight Learning for Markov Logic Networks , 2007, PKDD.

[39]  Michel Vacher,et al.  Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach , 2013, AmI.

[40]  Andreas P. Schmidt,et al.  SOPRANO – An extensible , open AAL platform for elderly people based on semantical contracts 1 , 2008 .

[41]  Takashi Nishiyama,et al.  Development of agent system based on decision model for creating an ambient space , 2010, AI & SOCIETY.

[42]  Heiner Stuckenschmidt,et al.  A probabilistic ontological framework for the recognition of multilevel human activities , 2013, UbiComp.

[43]  Ian H. Witten,et al.  Making Better Use of Global Discretization , 1999, ICML.

[44]  Ronald A. Howard,et al.  Readings on the Principles and Applications of Decision Analysis , 1989 .

[45]  Doruk Coskun,et al.  On-line Context Aware Physical Activity Recognition from the Accelerometer and Audio Sensors of Smartphones , 2014, AmI.

[46]  Seng Wai Loke Representing and reasoning with situations for context-aware pervasive computing: a logic programming perspective , 2004, Knowl. Eng. Rev..

[47]  Michel Vacher,et al.  Fusion of Audio and Temporal Multimodal Data by Spreading Activation for Dweller Localisation in a Smart Home , 2011, AMI 2011.

[48]  Pascal Hitzler,et al.  Ontologies and Rules , 2009, Handbook on Ontologies.

[49]  Hsien-Chou Liao,et al.  A RDF and OWL-Based Temporal Context Reasoning Model for Smart Home , 2007 .

[50]  Pedro M. Domingos,et al.  A Language for Relational Decision Theory , 2009 .

[51]  Wolfgang Kastner,et al.  A semantic representation of energy-related information in future smart homes , 2012 .

[52]  Riza Cenk Erdur,et al.  iConAwa - An intelligent context-aware system , 2012, Expert Syst. Appl..

[53]  Bart Vanrumste,et al.  Self-taught assistive vocal interfaces: an overview of the ALADIN project , 2013, INTERSPEECH.

[54]  Gerard Lacey,et al.  Context-Aware Shared Control of a Robot Mobility Aid for the Elderly Blind , 2000, Int. J. Robotics Res..

[55]  Michel Vacher,et al.  Experimental Evaluation of Speech Recognition Technologies for Voice-based Home Automation Control in a Smart Home , 2013, SLPAT.

[56]  Miguel A. Patricio,et al.  Context-based scene recognition from visual data in smart homes: an Information Fusion approach , 2012, Personal and Ubiquitous Computing.

[57]  Alexander Artikis,et al.  Logic-based event recognition , 2012, The Knowledge Engineering Review.

[58]  Michel Vacher,et al.  On-line human activity recognition from audio and home automation sensors: Comparison of sequential and non-sequential models in realistic Smart Homes , 2016, J. Ambient Intell. Smart Environ..

[59]  Bill N. Schilit,et al.  Context-aware computing applications , 1994, Workshop on Mobile Computing Systems and Applications.

[60]  Marjorie Skubic,et al.  An acoustic fall detector system that uses sound height information to reduce the false alarm rate , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.