Personal long-term memory aids

The prevalence and affordability of personal and environmental recording apparatuses are leading to increased documentation of our daily lives. This trend is bound to continue and it follows that academic, industry, and government groups are showing an increased interest in such endeavors for various purposes. In the present case, I assert that such documentation can be used to help remedy common memory problems. Assuming a long-term personal archive exists, when confronted with a memory problem, one faces a new challenge, that of finding relevant memory triggers. This dissertation examines the use of information-retrieval technologies on long-term archives of personal experiences towards remedying certain types of long-tern forgetting. The approach focuses on capturing audio for the content. Research on Spoken Document Retrieval examines the pitfalls of information-retrieval techniques on error-prone speech-recognizer-generated transcripts and these challenges carry over to the present task. However, “memory retrieval” can benefit from the person's familiarity of the recorded data and the context in which it was recorded to help guide their effort. To study this, I constructed memory-retrieval tools designed to leverage a person's familiarity of their past to optimize their search task. To evaluate the utility of these towards solving long-term memory problems, I (1) recorded public events and evaluated witnesses' memory-retrieval approaches using these tools; and (2) conducted a longer-term memory-retrieval study based on recordings of several years of my personal and research-related conversations. Subjects succeeded with memory-retrieval tasks in both studies, typically finding answers within minutes. This is far less time than the alternate of re-listening to hours of recordings. Subjects' memories of the past events, in particular their ability to narrow the window of time in which past events occurred, improved their ability to find answers. In addition to results from the memory-retrieval studies, I present a technique called “speed listening.” By using a transcript (even one with many errors), it allows people to reduce listening time while maintaining comprehension. Finally, I report on my experiences recording events in my life over 2.5 years. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

[1]  P. Barnard,et al.  Autobiographical memory and daily schemas at work. , 1994, Memory.

[2]  L. Curtin,et al.  Privacy in the information age? , 1999, Health management technology.

[3]  M. Conway,et al.  VERY LONG TERM RETENTION OF KNOWLEDGE , 1992 .

[4]  Joseph E LeDoux Emotion, memory and the brain. , 1994, Scientific American.

[5]  Abigail Sellen,et al.  Memory Problems at Work: Their Range, Frequency and Severity , 1992 .

[6]  Barry Arons,et al.  Techniques, Perception, and Applications of Time-Compressed Speech , 2009 .

[7]  Paul Over,et al.  The TREC-2002 Video Track Report , 2002, TREC.

[8]  Frank M. Shipman,et al.  Finding and using implicit structure in human-organized spatial layouts of information , 1995, CHI '95.

[9]  Victor Zue,et al.  Phonetic recognition for spoken document retrieval , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[10]  D. Schacter The seven sins of memory. Insights from psychology and cognitive neuroscience. , 1999, The American psychologist.

[11]  E F Loftus,et al.  Since the eruption of Mt. St. Helens, has anyone beaten you up? Improving the accuracy of retrospective reports with landmarkevents , 1983, Memory & cognition.

[12]  Jon Orwant,et al.  Heterogeneous learning in the Doppelgänger user modeling system , 2005, User Modeling and User-Adapted Interaction.

[13]  Susan T. Dumais,et al.  Milestones in Time: The Value of Landmarks in Retrieving Information from Personal Stores , 2003, INTERACT.

[14]  Martín Hadis,et al.  From generation to generation : family stories, computers and genealogy , 2002 .

[15]  Michael G. Rogers,et al.  Bodily Intrusion in Search of Evidence: A Study in Fourth Amendment Decisionmaking , 1987 .

[16]  Steve Whittaker,et al.  Semantic speech editing , 2004, CHI.

[17]  K. Emnett,et al.  Synthetic News Radio , 2000, IBM Syst. J..

[18]  Gerard Salton,et al.  Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer , 1989 .

[19]  Jennifer Healey,et al.  StartleCam: a cybernetic wearable camera , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[20]  Gordon Bell,et al.  MyLifeBits: fulfilling the Memex vision , 2002, MULTIMEDIA '02.

[21]  Robert G. Schwab,et al.  The temporal structure of cooperative activity , 1990, CSCW '90.

[22]  Thad Starner,et al.  Towards Conversational Speech Recognition for a Wearable Computer Based Appointment Scheduling Agent , 2002 .

[23]  Steve Mann,et al.  Wearable Tetherless Computer-Mediated Reality: WearCam as a wearable face-recognizer, and other applications for the disabled , 1996 .

[24]  Alexander G. Hauptmann,et al.  Experiments in Spoken Document Retrieval at CMU , 1997, TREC.

[25]  Trevor Kletz,et al.  Lessons from Disaster: How Organizations Have No Memory and Accidents Recur , 1993 .

[26]  M. Lamming,et al.  The Design of a Human Memory Prosthesis , 1994 .

[27]  James F. Allen,et al.  Identifying Discourse Markers in Spoken Dialog , 1998, ArXiv.

[28]  Nevenka Dimitrova,et al.  Multimedia Content Analysis and Indexing for Filtering and Retrieval Applications , 1999, Informing Sci. Int. J. an Emerg. Transdiscipl..

[29]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[30]  Roger S. Brown,et al.  The "Tip of the Tongue" Phenomenon , 1966 .

[31]  Robert Spence,et al.  Rapid, Serial and Visual: A Presentation Technique with Potential , 2002, Inf. Vis..

[32]  Michael J. Witbrock,et al.  Informedia News-On Demand: Using Speech Recognition to Create a Digital Video Library , 1998 .

[33]  Daniel E. Rose,et al.  Content awareness in a file system interface: implementing the “pile” metaphor for organizing information , 1993, SIGIR.

[34]  F. Craik,et al.  Levels of Pro-cessing: A Framework for Memory Research , 1975 .

[35]  Chris Schmandt Voice communication with computers: conversational systems , 1994 .

[36]  Gerald E. Loeb,et al.  Development of BION/spl trade/ technology for functional electrical stimulation: hermetic packaging , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[37]  P. Philippot,et al.  Flashbulb memories and the underlying mechanisms of their formation: Toward an emotional-integrative model , 1998, Memory & cognition.

[38]  David A. Balota,et al.  Memory changes in healthy older adults. , 2000 .

[39]  A. Baddeley,et al.  When does context influence recognition memory , 1980 .

[40]  Jr. J.P. Campbell,et al.  Speaker recognition: a tutorial , 1997, Proc. IEEE.

[41]  Push Singh,et al.  Common Sense Conversations: Understanding Casual Conversation using a Common Sense Database , 2003 .

[42]  Donald A. Norman,et al.  Issues for supporting notetaking and note using in the computer environment , 1990 .

[43]  Walter Bender,et al.  Improving speech playback using time-compression and speech recognition , 2004, CHI.

[44]  Barry Arons,et al.  Interactively skimming recorded speech , 1994 .

[45]  Peter Weibel,et al.  CTRL [space] : rhetorics of surveillance from bentham to big brother , 2002 .

[46]  E. Tulving,et al.  Availability versus accessibility of information in memory for words , 1966 .

[47]  D. Hynes,et al.  Rhetorics of Surveillance from Bentham to Big Brother , 2005 .

[48]  Joseph Nathaniel Kaye,et al.  Symbolic olfactory display , 2001 .

[49]  P. Werner,et al.  Anxiety disorder and accompanying subjective memory loss in the elderly as a predictor of future cognitive decline , 2003, International journal of geriatric psychiatry.

[50]  Timothy D. Lee,et al.  Motor Control and Learning: A Behavioral Emphasis , 1982 .

[51]  P. H. Blaney Affect and memory: a review. , 1986, Psychological bulletin.

[52]  Peter Schäuble,et al.  New techniques for open-vocabulary spoken document retrieval , 1998, SIGIR '98.

[53]  T. Sticht,et al.  Review of research on the intelligibility and comprehension of accelerated speech. , 1969, Psychological bulletin.

[54]  Barry Arons,et al.  The audio notebook: paper and pen interaction with structured speech , 2001, CHI.

[55]  Kristian J. Hammond,et al.  Watson: Anticipating and Contextualizing Information Needs , 1999 .

[56]  Debby Hindus,et al.  Ubiquitous audio: capturing spontaneous collaboration , 1992, CSCW '92.

[57]  Ernest A. Edmonds,et al.  Using Memory for Events in the Design of Personal Filing Systems , 1992, Int. J. Man Mach. Stud..

[58]  Daniel S. Beasley,et al.  chapter 12 – Time- and Frequency-Altered Speech , 1976 .

[59]  Julia Hirschberg,et al.  SCAN: designing and evaluating user interfaces to support retrieval from speech archives , 1999, SIGIR '99.

[60]  Candace L. Sidner,et al.  Email overload: exploring personal information management of email , 1996, CHI.

[61]  G. E. Loeb,et al.  DEVELOPMENT OF BION TECHNOLOGY FOR FUNCTIONAL ELECTRICAL STIMULATION : HERMETIC PACKAGING , 2001 .

[62]  B M Ross,et al.  Long-term memory of odors with and without verbal descriptions. , 1973, Journal of experimental psychology.

[63]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[64]  Leysia Palen,et al.  “I'll get that off the audio”: a case study of salvaging multimedia meeting records , 1997, CHI.

[65]  M. Weiser The Computer for the Twenty-First Century , 1991 .

[66]  Rosalind W. Picard,et al.  A computational model for the automatic recognition of affect in speech , 2004 .

[67]  Gregory D. Abowd,et al.  The Personal Audio Loop: Designing a Ubiquitous Audio-Based Memory Aid , 2004, Mobile HCI.

[68]  Thomas G. Sticht Comprehension of Repeated Time-Compressed Recordings , 1969 .

[69]  Julia Hirschberg,et al.  ASR satisficing: the effects of ASR accuracy on speech retrieval , 2000, INTERSPEECH.

[70]  K. Bartholomew,et al.  Do you remember?: Recollections of adult attachment patterns , 1998 .

[71]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[72]  H. P. Bahrick,et al.  Fifty years of memory for names and faces: A cross-sectional approach. , 1975 .

[73]  Pattie Maes,et al.  Just-in-time information retrieval , 2000 .

[74]  W. Terry Everyday Forgetting: Data from a Diary Study , 1988 .

[75]  Gregory D. Abowd,et al.  Classroom 2000: An Experiment with the Instrumentation of a Living Educational Environment , 1999, IBM Syst. J..

[76]  Andreas Stolcke,et al.  The Meeting Project at ICSI , 2001, HLT.

[77]  W. Wagenaar My memory: A study of autobiographical memory over six years , 1986, Cognitive Psychology.

[78]  D. B. Orr,et al.  TRAINABILITY OF LISTENING COMPREHENSION OF SPEEDED DISCOURSE. , 1965, Journal of educational psychology.

[79]  E. Warrington,et al.  The Fate of Old Memories , 1971 .

[80]  Barry Arons,et al.  SpeechSkimmer: interactively skimming recorded speech , 1993, UIST '93.

[81]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[82]  Alisa Marshall Signal-Based Location Sensing Using 802.11b , 2003 .

[83]  Endel Tulving,et al.  The effects of presentation and recall of material in free-recall learning , 1967 .

[84]  Richard M. Stern,et al.  Efficient Cepstral Normalization for Robust Speech Recognition , 1993, HLT.

[85]  Michael D. Rugg,et al.  An event-related potential study of memory for words spoken aloud or heard , 1997, Neuropsychologia.

[86]  V. Michael Bove,et al.  P‐125: Personal Projection, or How to Put a Large Screen in a Small Device , 2003 .

[87]  Elizabeth D. Liddy,et al.  Advances in Automatic Text Summarization , 2001, Information Retrieval.

[88]  Wei-Hao Lin,et al.  A wearable digital library of personal conversations , 2002, JCDL '02.

[89]  Walter Bender,et al.  Every sign of life , 2003 .

[90]  Aaron E. Rosenberg,et al.  SCANMail: a voicemail interface that makes speech browsable, readable and searchable , 2002, CHI.

[91]  Richard Mander,et al.  Working with audio: integrating personal tape recorders and desktop computers , 1992, CHI '92.

[92]  Elizabeth A. Olson,et al.  Eyewitness testimony. , 2003, Annual review of psychology.

[93]  Ellen M. Voorhees,et al.  The TREC Spoken Document Retrieval Track: A Success Story , 2000, TREC.

[94]  Stefanie Tellex,et al.  An Audio-Based Personal Memory Aid , 2004, UbiComp.

[95]  David Gelernter,et al.  Lifestreams: an alternative to the desktop metaphor , 1996, CHI Conference Companion.

[96]  Gerhard Schwabe Providing for Organizational Memory in Computer-Supported Meetings , 1994, J. Organ. Comput. Electron. Commer..

[97]  G. Vattimo The Transparent Society , 1992 .

[98]  M. Lamming,et al.  "Forget-me-not" Intimate Computing in Support of Human Memory , 1994 .