Decision making in assistive environments using multimodal observations

An assistive environment is a smart domestic space based on pervasive computing to support the elderly and disabled. Unlike sensors, which can only provide passive monitoring, a robot can be an active element to improve the quality of life for the human. In this paper, we propose an active service of the robot in assistive environments, to help human in the case of emergency situation. It works on a hierarchical partially observable Markov decision process (POMDP). The multimodal observation series are used in the decision and evaluation process. An active robot is a kind of robot that can provide a preferable and necessary active service to the human. This is used in our emergency response system (ERS) to deal with the emergency situations, such as an older adult falls down or emergency diseases. The purpose of multimodal observations is to guarantee the precision of report for the emergency situations. Four observation sources are introduced in this paper: the vision recognition, the voice recognition, the physical input devices and the foreign systems. For each observation source, there are two observation series. Multiple information sources give the agent more opportunities to learn from the real world, so as to make more reasonable predictions, evaluations and decisions.