Situated Observation of Human Activity

Many human activities follow a loosely defined script in which individuals assume roles. Encoding such scripts in a formal representation makes it possible to build systems that observe and understand human activity. In this paper, we first present a conceptual framework in which scripts for human activity are described as scenarios composed of actors and objects within a network of situations. We provide formal definitions for the underlying concepts for situation models, and then propose a layered, component-based, software architecture model for constructings systems to observe human activity. Both the conceptual framework and architectural model are illustrated with a system for real-time composition of a synchronized audio-video streams for recording activity within a meeting or lecture.

[1]  Oliver Brdiczka,et al.  Automatic detection of interaction groups , 2005, ICMI '05.

[2]  James L. Crowley,et al.  Event-based Activity Analysis in Live Video Using a Generic Object Tracker , 2002 .

[3]  James L. Crowley,et al.  Robust face tracking using color , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  James L. Crowley,et al.  Perceptual user interfaces: things that see , 2000, CACM.

[5]  David Garlan,et al.  Context is key , 2005, CACM.

[6]  James L. Crowley,et al.  Multi-modal tracking of faces for video communications , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  John Rasure,et al.  The Khoros Application Development Environment , 1994 .

[8]  Mary Shaw,et al.  Software architecture - perspectives on an emerging discipline , 1996 .

[9]  Augustin Lux,et al.  The Imalab method for vision systems , 2003, Machine Vision and Applications.

[10]  James L. Crowley,et al.  Perceptual Components for Context Aware Computing , 2002, UbiComp.

[11]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[12]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[13]  James L. Crowley,et al.  Integration and control of reactive visual processes , 1994, Robotics Auton. Syst..