An automatic approach to virtual living based on environmental sound cues

This paper presents a novel indoor and outdoor monitoring system based on sound cues that can be used for the automatic creation of a Life-Log, health care monitoring and/or ambient communication with virtual worlds. Basically, the system detects daily life activities (e.g., laughing, talking, traveling, cooking, sleeping, etc.) and situational references (e.g., inside a train, at a park, at home, at school, etc.) by processing environmental sounds, creates a Life-Log and recreates those activities into a virtual-world. It is easily extensible, portable, feasible to implement and reveals advantages and originality compared with other life-sensing systems. The results of the perceptual tests are encouraging and the system performed satisfactorily in a noisy environment, attracting the attention and curiosity of the subjects.

[1]  Alex Pentland,et al.  InSense: Interest-Based Life Logging , 2006, IEEE MultiMedia.

[2]  Andrey Temko,et al.  Classification of meeting-room acoustic events with support vector machines and variable-feature-set clustering , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[3]  A. Fitzgibbon,et al.  Memories for life: Managing information over a human lifetime , 2004 .

[4]  Karen Zita Haigh,et al.  Learning Models of Human Behaviour with Sequential Patterns , 2002 .

[5]  Walter Bender,et al.  Next-Generation Personal Memory Aids , 2004 .

[6]  Mirco Musolesi,et al.  The Second Life of a Sensor Integrating Real-world Experience in Virtual Worlds using Mobile Phones , 2008 .

[7]  Michael C. Mozer,et al.  The Neural Network House: An Environment that Adapts to its Inhabitants , 1998 .

[8]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[9]  J. Barbenel,et al.  The Use of Artificial Intelligence in the Design of an Intelligent Cognitive Orthosis for People with Dementia , 2001, Assistive technology : the official journal of RESNA.

[10]  Ning Liu,et al.  Bathroom Activity Monitoring Based on Sound , 2005, Pervasive.

[11]  Gordon Bell,et al.  MyLifeBits: fulfilling the Memex vision , 2002, MULTIMEDIA '02.

[12]  A. Glascock,et al.  Behavioral Telemedicine: A New Approach to the Continuous Nonintrusive Monitoring of Activities of Daily Living , 2000 .

[13]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[14]  Dadong Wan,et al.  Magic Medicine Cabinet: A Situated Portal for Consumer Healthcare , 1999, HUC.

[15]  Gregory Grefenstette,et al.  Validating the Coverage of Lexical Resources for Affect Analysis and Automatically Classifying New Words along Semantic Axes , 2006, Computing Attitude and Affect in Text.

[16]  Vasileios Hatzivassiloglou,et al.  Predicting the Semantic Orientation of Adjectives , 1997, ACL.

[17]  Alex Pentland,et al.  The familiar: a living diary and companion , 2001, CHI Extended Abstracts.

[18]  Juha T. Tuomi,et al.  Audio-based context awareness - acoustic modeling and perceptual evaluation , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[19]  H.G. Okuno,et al.  Computational Auditory Scene Analysis and its Application to Robot Audition , 2004, 2008 Hands-Free Speech Communication and Microphone Arrays.

[20]  Thad Starner,et al.  Remembrance Agent: A Continuously Running Automated Information Retrieval System , 1996, PAAM.

[21]  Mark Lentczner,et al.  Second Life , 2007, USENIX Annual Technical Conference.

[22]  Alex Pentland,et al.  Unsupervised clustering of ambulatory audio and video , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).