Sit straight (and tell me what I did today): a human posture alarm and activity summarization system

In this paper we present a novel system for monitoring a computer user's posture and activities in front of the computer (e.g., reading, speaking on the phone, etc.) for self-reporting. In our system, a camera and a microphone are placed in front of a computer work area (e.g., on top of the computer screen). The system monitors the computer user's postures and summarizes his or her activities. The system gives the user real time feedback on the goodness of his current posture, triggers alarms if the postures are not good postures, and generates summaries of postures and activities over a specified period of time (e.g., hours, days, months, etc.). All elements of the system are highly customizable: the user decides what "good" postures are, what alarms are triggered, if any, and what activity and posture summaries are generated. We present novel algorithms for posture measurement (using geometric features of the user's silhouette), and activity classification (using machine learning). Finally, we present experiments that show the feasibility of our approach, and discuss privacy issues and applications of the techniques presented (health monitoring, productivity analysis, and others).

[1]  Roel Vertegaal,et al.  Attentive User Interfaces , 2003 .

[2]  Hirotsugu Kashimura,et al.  Classification of human actions using face and hands detection , 2004, MULTIMEDIA '04.

[3]  Shih-Fu Chang,et al.  Conceptual structures and computational methods for indexing and organization of visual information , 2003 .

[4]  Alan Hedge.  Effects of Ergonomic Management Software on Employee Performance , 1999 .

[5]  Emil M. Petriu,et al.  Real-time 2(1/2)-D head pose recovery for model-based video-coding , 2001, IEEE Trans. Instrum. Meas..

[6]  Guy J. Brown,et al.  Interactive explorations in speech and hearing , 1999 .

[7]  Mathias Kölsch,et al.  The Postural Comfort Zone for Reaching Gestures , 2003 .

[8]  Brian P. Bailey,et al.  A method, system, and tools for intelligent interruption management , 2005, TAMODIA '05.

[9]  Pascal Fua,et al.  Implicit surfaces make for better silhouettes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[11]  Larry S. Davis,et al.  Gesture recognition using a probabilistic framework for pose matching , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[12]  Rita Cucchiara,et al.  Probabilistic posture classification for Human-behavior analysis , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[13]  Noel E. O'Connor,et al.  An integrated approach for object shape registration and modeling , 2005 .

[14]  Larry S. Davis,et al.  Ghost: a human body part labeling system using silhouettes , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[15]  Monique Thonnat,et al.  Human Posture Recognition in Video Sequence , 2003 .

[16]  Ian Witten,et al.  Data Mining , 2000 .

[17]  Michael D. Abràmoff,et al.  Image processing with ImageJ , 2004 .

[18]  Cristian Sminchisescu,et al.  Human Pose Estimation from Silhouettes - A Consistent Approach Using Distance Level Sets , 2002, WSCG.

[19]  Trevor Darrell,et al.  Fast pose estimation with parameter-sensitive hashing , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[21]  Steve R. Gunn,et al.  Snake head boundary extraction using global and local energy minimisation , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[22]  Nicu Sebe,et al.  Affective Meeting Video Analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[23]  Rómer Rosales,et al.  Inferring body pose without tracking body parts , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[24]  Raghu Machiraju,et al.  Finding optimal views for 3D face shape modeling , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[25]  Gordon Bell,et al.  The MyLifeBits lifetime store , 2003, ETP '03.

[26]  Mun Wai Lee,et al.  Human Upper Body Pose Estimation in Static Images , 2004, ECCV.

[27]  Lawrence K. Saul,et al.  Real-Time Pitch Determination of One or More Voices by Nonnegative Matrix Factorization , 2004, NIPS.