DyPERS: Dynamic Personal Enhanced Reality System

DyPERS, 'Dynamic Personal Enhanced Reality System', is a wearable system which uses augmented reality and computer vision to autonomously retrieve 'media memories' based on associations with real objects the user encounters. These are evoked as audio and video clips taken by the user and overlayed on top of real objects the user looks at. The user's visual and auditory scene is stored in real-time by the system (upon request) and is then associated (by user input) with a snap shot of a visual object. The object acts as a key which is detected by a real-time vision system when it is in view, triggering DyPERS to play back the appropriate audio-visual sequence. The vision system is a probabilistic algorithm which is capable of discriminating between hundreds of everyday objects under varying viewing conditions (lighting, pose, etc.). The record-andassociate paradigm of the system has many potential applications. Results on the use of the system in a museum tour scenario are described.

[1]  Ron Frederick,et al.  Audio aura: light-weight audio augmented reality , 1997, UIST '97.

[2]  Blair MacIntyre,et al.  Annotating the real world with knowledge-based graphics on a see-through head-mounted display , 1992 .

[3]  Jeffrey M. Levin.  Real-time target and pose recognition for 3-D graphical overlay , 1997 .

[4]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[5]  John J. Leggett,et al.  Interaction styles and input/output devices , 1993, Behav. Inf. Technol..

[6]  M. Lamming,et al.  "Forget-me-not" Intimate Computing in Support of Human Memory , 1994 .

[7]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[8]  Bernt Schiele,et al.  Probabilistic object recognition using multidimensional receptive field histograms , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[9]  Benjamin B. Bederson,et al.  Audio augmented reality: a prototype automated tour guide , 1995, CHI 95 Conference Companion.

[10]  J. Davenport.  Editor , 1960 .

[11]  Thad Starner,et al.  Remembrance Agent: A Continuously Running Automated Information Retrieval System , 1996, PAAM.

[12]  Alex Pentland,et al.  Stochasticks: augmenting the billiards experience with probabilistic vision and wearable computers , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[13]  Vania Conan,et al.  Virtually documented environments , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[14]  Katashi Nagao,et al.  The world through the computer: computer augmented interaction with real world environments , 1995, UIST '95.