Implementation and evaluation of wearable sensor devices for recording and detecting conversational interaction

We have developed a wearable device that records the activities of human-human and human-artifact interactions. The device records personal and social everyday-life experiences in multiple modalities such as voice and visible scene, using microphone and camera to imitate human perceptibility. These sensors are used to record the perceived experiences continuously as well as to detect and index interactions from nonverbal behaviors. We employ an infrared LED ID tag system to detect the interactions in terms of the ID and its relative positions of the objects in the visual field of the camera. In this paper, we propose the "Interaction Scope" which is defined as the range of relative human/object positions that have a high probability of occurring in conversational interactions. We confirm that the interaction scope exists and can represent the interactions naturally through the analysis of experimental conversational sessions. We also demonstrate that our tag system can be effectively used as a sensor that detects and measures the proposed interaction scope.

[1]  J. Gibson,et al.  Perception of another person's looking behavior. , 1963, The American journal of psychology.

[2]  E. Hall,et al.  The Hidden Dimension , 1970 .

[3]  Anthony Davids,et al.  Ecological Psychology: Concepts and Methods for Studying the Environment of Human Behavior. , 1969 .

[4]  Steve Mann,et al.  Humanistic computing: "WearComp" as a new framework and application for intelligent signal processing , 1998, Proc. IEEE.

[5]  Mitchel Resnick,et al.  Meme tags and community mirrors: moving from conferences to collaboration , 1998, CSCW '98.

[6]  Gregory D. Abowd,et al.  The Conference Assistant: combining context-awareness with wearable computing , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[7]  Lynn Wilcox,et al.  Meeting Capture in a Media Enriched Conference Room , 1999, CoBuild.

[8]  Alexander H. Waibel,et al.  Modeling focus of attention for meeting indexing based on multiple cues , 2002, IEEE Trans. Neural Networks.

[9]  Tanzeem Choudhury.  Modeling Face-to-Face Communication using the Sociometer , 2003 .

[10]  Masashi Toda,et al.  Structual Representation of Personal Events , 2003 .

[11]  Sidney Fels,et al.  Collaborative Capturing and Interpretation of Interactions , 2004 .

[12]  Toyoaki Nishida,et al.  A layered interpretation of human interactions captured by ubiquitous sensors , 2004, CARPE'04.

[13]  Norihiro Hagita,et al.  Interaction Corpus for Experience Sharing using Ubiquitous Experience Media , 2004 .