Toward a Companion Agent for the Elderly - The Methods to Estimate At-Home and Outside-Home Daily Life Activities of the Elderly Who Live Alone

With advances in medical technology, people's life have been extended, and there are more and more older adults isolated. If they do not maintain social life with others, they may feel loneliness and anxiety. For their mental health, it is reported effective to keep their social relationship with others, for example, the conversation with their caregivers or other elderly people. Active listening is a communication technique that the volunteer listener listens to the speaker the elderly carefully and attentively by confirming or asking for more details about what they heard. This helps to make the elderly feel cared and to relieve their anxiety and loneliness. This paper presents our in-progress project aiming to develop a framework of a virtual companion agent who is always with the user and can engage active listening to maintain a long-term relationship with elderly users. In order to achieve the agent's companionship with the user for a longer period, we believe that it is essential to make the agent to understand the user as best as it can. This kind of user-fitted conversation is not addressed in previous companion agent work. The proposed approach is the acquisition of the "memory" of the user's daily life in two situations, at-home and outside-home. In the former one, multiple Microsoft Kinect depth sensors were adopted. The depth information is integrated to detect the user's position and posture and then to estimate the user's daily activity. In the outside-home configuration, the prototype application is an Android smartphone application that recognizes the user's moving status with the information from the on-board three-axis accelerometer as well as the location of the user from GPS information. These data are then used to estimate the user's outside-home activity. All estimated daily activities are recorded in an activity history database. Both the at-home and outside-home activity estimation methods have been developed and have been evaluated in a laboratory environment with student subjects at a moderate accuracy. The interface of the companion agent is being designed with the results from human-human and human-agent driven by the data from the human listener condition subject experiments. After the technologies are more matured, we would like to conduct real-world experiment with elderly subjects in near future.