Wicked Problems and Gnarly Results: Reflecting on Design and Evaluation Methods for Idiosyncratic Personal Information Management Tasks

This paper is a case study of an artifact design and evaluation process; it is a reflection on how right thinking about design methods may at times result in sub-optimal results. Our goal has been to assess our decision making process throughout the design and evaluation stages for a software prototype in order to consider where design methodology may need to be tuned to be more sensitive to the domain of practice, in this case software evaluation in personal information management. In particular, we reflect on design methods around (1) scale of prototype, (2) prototyping and design process, (3) study design, and (4) study population.

[1]  Jane Fulton Suri,et al.  Experience prototyping , 2000, DIS '00.

[2]  Abigail Sellen,et al.  Do life-logging technologies support memory for the past?: an experimental study using sensecam , 2007, CHI.

[3]  Paul P. Maglio,et al.  Supporting notable information in office work , 2003, CHI Extended Abstracts.

[4]  Jonathan Grudin,et al.  Human Computer Interaction: The Year 2000 and Beyond , 1995, HCI.

[5]  Daniel G. Bobrow,et al.  What a to-do: studies of task management towards the design of a personal task list manager , 2004, CHI.

[6]  Michael S. Bernstein,et al.  Information ! Scraps : ! ! How ! and ! Why ! Information ! Eludes ! our ! Personal ! Information ! Management , 2022 .

[7]  Ann Blandford,et al.  Group and Individual Time Management Tools: What You Get is Not What You Need , 2001, Personal and Ubiquitous Computing.

[8]  Gregory D. Abowd,et al.  Practices for capturing short important thoughts , 2003, CHI Extended Abstracts.

[9]  William E. Jones,et al.  Context in information behavior research , 2007 .

[10]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[11]  Wayne G. Lutters,et al.  Understanding the micronote lifecycle: improving mobile support for informal note taking , 2004, CHI.

[12]  Monica M. C. Schraefel,et al.  Hunter gatherer: interaction support for the creation and management of within-web-page collections , 2002, WWW.

[13]  Mary Czerwinski,et al.  GroupBar: The TaskBar Evolved , 2003 .

[14]  H. Rittel,et al.  Dilemmas in a general theory of planning , 1973 .

[15]  Vaiva Kalnikaité,et al.  Software or wetware?: discovering when and why people use digital prosthetic memory , 2007, CHI.

[16]  Michael S. Bernstein,et al.  Management of personal information scraps , 2007, CHI Extended Abstracts.

[17]  Louise Barkhuus,et al.  From Mice to Men - 24 Years of Evaluation in CHI , 2007, CHI 2007.

[18]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[19]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[20]  Marc Rettig,et al.  Prototyping for tiny fingers , 1994, CACM.

[21]  J. McGrath.  Methodology matters: doing research in the behavioral and social sciences , 1995 .

[22]  Ian Smith,et al.  Taking email to task: the design and evaluation of a task management centered email tool , 2003, CHI '03.

[23]  Michael S. Bernstein,et al.  Gui --- phooey!: the case for text input , 2007, UIST.

[24]  Diane Kelly,et al.  Understanding what works: Evaluating PIM tools , 2007 .

[25]  Mary Czerwinski,et al.  An Investigation of Memory for Daily Computing Events , 2002 .