inSensed: Wearable Collection of Multimedia based on Interest

We present a wearable data collection system that allows users to collect their experiences into a continually growing and adapting multimedia diary. The system, called iSensed, uses the patterns in sensor readings from a camera, microphone, and accelerometers, to classify the user's activities and automatically collect multimedia clips when the user is in aninterest- ing' situation. `Interest' is estimated by a user-trained rule relating the sequence of user ac- tivities/situations to their concept ofinterest'. This allows the multimedia information to be structured in a manner similar to the user's episodic memory, e.g., unusual and potentially important events are recorded chronologically, allowing rapid browsing and automatic struc- turing of the multimedia diary. The system is based on commercially available mobile devices, either a PDA or a camera phones, together with two small wireless accelerometers worn on the wrist and belt.

[1]  Jeffrey S. Shell,et al.  Augmenting and sharing memory with eyeBlog , 2004, CARPE'04.

[2]  Alex Pentland,et al.  MIThril 2003: applications and architecture , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[3]  A. Fitzgibbon,et al.  Memories for life: Managing information over a human lifetime , 2004 .

[4]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[5]  Mark Lawrence Blum,et al.  Real-time Context Recognition by , 2005 .

[6]  Kent Larson,et al.  MITes: Wireless Portable Sensors for Studying Behavior , 2004 .

[7]  Walter Bender,et al.  Next-Generation Personal Memory Aids , 2004 .

[8]  Alex Pentland,et al.  Unsupervised clustering of ambulatory audio and video , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[9]  Thad Starner,et al.  Remembrance Agent: A Continuously Running Automated Information Retrieval System , 1996, PAAM.

[10]  A. Pentland,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence, Manuscript Id 1 Life Patterns , 2022 .

[11]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.

[12]  Gordon Bell,et al.  MyLifeBits: fulfilling the Memex vision , 2002, MULTIMEDIA '02.

[13]  Alex Pentland,et al.  The familiar: a living diary and companion , 2001, CHI Extended Abstracts.

[14]  Sumit Basu,et al.  Conversational scene analysis , 2002 .