Situation Aware Computing with Wearable Computers

For most computer systems even virtual reality systems sensing techniques are a means of getting input directly from the user However wearable sensors and computers o er a unique opportunity to re direct sensing technology towards recovering more general user context Wearable computers have the potential to see as the user sees hear as the user hears and experience the life of the user in a rst person sense This increase in contextual and user information may lead to more intelligent and uid interfaces that use the physical world as part of the interface Wearable computers are excellent platforms for contextually aware applications but these applications are also necessary to use wearables to their fullest Wearables are more than just highly portable computers they perform useful work even while the wearer isn t directly interacting with the system In such environments the user needs to concentrate on his environment not on the computer interface so the wearable needs to use information from the wearer s context to be the least distracting For example imagine an interface which is aware of the user s location while being in the subway the system might alert him with a spoken summary of an e mail However during a conversation the wearable computer may present the name of a potential caller unobtrusively in the user s head up display or simply forward the call to voicemail The importance of context in communication and interface can not be overstated Phys ical environment time of day mental state and the model each conversant has of the other participants can be critical in conveying necessary information and mood An anecdote from Nicholas Negroponte s book Being Digital Negroponte illustrates this point

[1]  Andy Hopper,et al.  Active badges and personal interactive computing objects , 1992 .

[2]  Alex Pentland,et al.  Extracting context from environmental audio , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[3]  Bernt Schiele,et al.  Probabilistic object recognition using multidimensional receptive field histograms , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[4]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[5]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[6]  T. Humphries,et al.  A Basic Course in American Sign Language , 1981 .

[7]  Thad Starner,et al.  The locust swarm: an environmentally-powered, networkless location and messaging system , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[8]  Lawrence J. Najjar,et al.  Wearable computers for performance support: initial feasibility study , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[9]  Katashi Nagao,et al.  The world through the computer: computer augmented interaction with real world environments , 1995, UIST '95.

[10]  Bradley J. Rhodes,et al.  The wearable remembrance agent: A system for augmented memory , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[11]  Lawrence J. Najjar,et al.  A wearable computer for quality assurance inspectors in a food processing plant , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[12]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[13]  Gregory D. Abowd,et al.  Rapid prototyping of mobile context-aware applications: the Cyberguide case study , 1996, MobiCom '96.

[14]  Alex Pentland,et al.  An Interactive Computer Vision System DyPERS: Dynamic Personal Enhanced Reality System , 1999, ICVS.

[15]  William Noah Schilit,et al.  A system architecture for context-aware mobile computing , 1995 .

[16]  Trevor Darrell,et al.  A novel environment for situated vision and behavior , 1994 .

[17]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[18]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[20]  Takeo Kanade,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995, CVRMed.

[21]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[22]  Thad Starner,et al.  Remembrance Agent: A Continuously Running Automated Information Retrieval System , 1996, PAAM.

[23]  Daniel P. Siewiorek,et al.  The CMU mobile computers: a new generation of computer systems , 1994, Proceedings of COMPCON '94.

[24]  Jeffrey M. Levin.  Real-time target and pose recognition for 3-D graphical overlay , 1997 .

[25]  Alex Pentland,et al.  Stochasticks: augmenting the billiards experience with probabilistic vision and wearable computers , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[26]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[27]  Robert E. Kraut,et al.  Collaboration in performance of physical tasks: effects on outcomes and communication , 1996, CSCW '96.

[28]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[29]  M. Lamming,et al.  "Forget-me-not" Intimate Computing in Support of Human Memory , 1994 .

[30]  Ulrich Neumann,et al.  Fast color fiducial detection and dynamic workspace extension in video see-through self-tracking augmented reality , 1997, Proceedings The Fifth Pacific Conference on Computer Graphics and Applications.

[31]  Asim Smailagic,et al.  Metronaut: a wearable computer with sensing and global communication capabilities , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[32]  Bernt Schiele,et al.  Object Recognition Using Multidimensional Receptive Field Histograms , 1996, ECCV.

[33]  Chris Schmandt.  Voice communication with computers , 1993 .

[34]  Tomas Lozano-Perez,et al.  An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization , 1996 .

[35]  Jon Orwant,et al.  Doppelgänger goes to school : machine learning for user modeling , 1993 .

[36]  Alex Pentland,et al.  Unsupervised clustering of ambulatory audio and video , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[37]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[38]  Christopher Schmandt.  Voice Communications with Computers , 1994 .

[39]  .. McGlone,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995 .

[40]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[41]  Chris Schmandt,et al.  Speaking and listening on the run: design for wearable audio computing , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[42]  Rajeev Sharma,et al.  Computer Vision-Based Augmented Reality for Guiding Manual Assembly , 1997, Presence: Teleoperators & Virtual Environments.

[43]  Katashi Nagao,et al.  Ubiquitous Talker: Spoken Language Interaction with Real World Objects , 1995, IJCAI.

[44]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[45]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..