Audio-enhanced paper photos: encouraging social interaction at age 105

Photographs are powerful and pervasive media that facilitate communication and support reminiscence. Adding audio narrations to traditional paper photographs combines the familiarity of paper photos with the nuanced richness of voice. We enable creation of and interaction with audio-enhanced paper photographs through custom software deployed on a digital pen. This paper reports on use of a paper-digital photo album by an older adult (Ethel, age 105), her extended family, and nursing staff over a five-month field deployment. The interactive photo album was found to be easy to use and accessible, effectively engaged Ethel's and family members' interest, and provided a focal point for communication and social interaction. Family and caregivers report improvements in Ethel's social interaction. We discuss the properties of our audio-enhanced paper approach that make it a promising medium for engaging older adults and distill general design considerations for paper-digital photo albums.

[1]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[2]  D. Blazer.  Social support and mortality in an elderly community population. , 1982, American journal of epidemiology.

[3]  H Brownell,et al.  The getting of wisdom: theory of mind in old age. , 1998, Developmental psychology.

[4]  Allan Kuchinsky,et al.  Requirements for photoware , 2002, CSCW '02.

[5]  Judy Kay,et al.  MEMENTO: a digital-physical scrapbook for memory sharing , 2007, Personal and Ubiquitous Computing.

[6]  Christian Heath,et al.  Tasks-in-interaction: paper and screen based documentation in collaborative activity , 1992, CSCW '92.

[7]  Max Mühlhäuser,et al.  CoScribe: Integrating Paper and Digital Documents for Collaborative Knowledge Work , 2009, IEEE Transactions on Learning Technologies.

[8]  Edwin Hutchins,et al.  The uses of paper in commercial airline flight operations , 2006, CSCW '06.

[9]  Lynn Wilcox,et al.  A dynamic grouping technique for ink and audio notes , 1998, UIST '98.

[10]  Alex S. Taylor,et al.  Editorial: Collocated social practices surrounding photos , 2009 .

[11]  Daniela Petrelli,et al.  Sonic souvenirs: exploring the paradoxes of recorded sound for family remembering , 2010, CSCW '10.

[12]  Pamela K. Smith,et al.  Models of visuospatial and verbal memory across the adult life span. , 2002, Psychology and aging.

[13]  Abigail Sellen,et al.  The myth of the paperless office , 2001 .

[14]  Gemma Webster,et al.  Multimedia profiles as external personalities to support people with dementia and their carers , 2011 .

[15]  R. Baecker,et al.  Multimedia Biographies: A Reminiscence and Social Stimulus Tool for Persons with Cognitive Impairment , 2009 .

[16]  Anne Marie Piper,et al.  Introducing multimodal paper-digital interfaces for speech-language therapy , 2010, ASSETS '10.

[17]  Ella Tallyn,et al.  Audiophotography: practice and prospects , 1999, CHI EA '99.

[18]  T. Salthouse.  Effects of age and skill in typing. , 1984, Journal of experimental psychology. General.

[19]  Jacques Grégoire,et al.  Effect of age on forward and backward digit spans , 1997 .

[20]  Scott R. Klemmer,et al.  Books with voices: paper transcripts as a physical interface to oral histories , 2003, CHI '03.

[21]  François Guimbretière,et al.  Paper augmented digital documents , 2003, UIST '03.

[22]  M. Hirsch.  Family Frames: Photography, Narrative and Postmemory , 1997 .

[23]  Anne Marie Piper,et al.  TAP & PLAY: an end-user toolkit for authoring interactive pen and paper language activities , 2012, CHI.

[24]  Paul B. Baltes,et al.  Intellectual Development in Adulthood: The Seattle Longitudinal Study , 1996 .

[25]  L. Carstensen,et al.  Socioemotional Selectivity Theory and the Regulation of Emotion in the Second Half of Life , 2003 .

[26]  Dave Levin,et al.  ProofRite: A Paper-Augmented Word Processor , 2004 .

[27]  Wendy E. Mackay,et al.  The missing link: augmenting biology laboratory notebooks , 2002, UIST '02.

[28]  David M. Frohlich,et al.  Audiophotography: Bringing photos to life with sounds (The Computer Supported Cooperative Work Series) , 2004 .

[29]  Margaret E. Morris,et al.  Social networks as health feedback displays , 2005, IEEE Internet Computing.

[30]  James D. Hollan,et al.  Proceedings of the 38th Hawaii International Conference on System Sciences- 2005 Digital Photos as Conversational Anchors , 2022 .

[31]  Gregory D. Abowd,et al.  Getting into the Living Memory Box: Family archives & holistic design , 2003, Personal and Ubiquitous Computing.

[32]  Michael Rohs,et al.  Bridging the gap between the Kodak and the Flickr generations: A novel interaction technique for collocated photo sharing , 2009, Int. J. Hum. Comput. Stud..

[33]  Norman Alm,et al.  Designing a multimedia conversation aid for reminiscence therapy in dementia care environments , 2004, CHI EA '04.

[34]  L L Light,et al.  Adult age differences in repetition priming: a meta-analysis. , 1994, Psychology and aging.

[35]  Barry Arons,et al.  The audio notebook: paper and pen interaction with structured speech , 2001, CHI.

[36]  E. Hutchins.  Cognition in the wild , 1995 .

[37]  Alex S. Taylor,et al.  Photo displays and intergenerational relationships in the family home , 2009, BCS HCI.

[38]  Tom Rodden,et al.  Collaborating around collections: informing the continued development of photoware , 2004, CSCW.

[39]  D. Rubin,et al.  Life-narrative and word-cued autobiographical memories in centenarians: Comparisons with 80-year-old control, depressed, and dementia groups , 2003, Memory.

[40]  Kiyoshi Yasuda,et al.  Networked reminiscence therapy for individuals with dementia by using photo and video sharing , 2006, Assets '06.

[41]  Wendy E. Mackay,et al.  Is paper safer? The role of paper flight strips in air traffic control , 1999, TCHI.

[42]  David M. Frohlich,et al.  Sound, paper and memorabilia: resources for a simpler digital photography , 2007, Personal and Ubiquitous Computing.

[43]  David Frohlich,et al.  The Memory Box , 2000 .

[44]  R. Chalfen.  Snapshot versions of life , 1987 .

[45]  Scott R. Klemmer,et al.  ButterflyNet: a mobile capture and access system for field biology research , 2006, CHI.

[46]  J. Lubben.  Centrality of social ties to the health and well-being of older adults , 2003 .