Toward a Framework for Evaluating Ubiquitous Computing Applications

Although they routinely evaluate pervasive or ubiquitous computing applications, researchers have difficulty comparing results rigorously and quantitatively. The lack of a widely accepted framework for user evaluations of ubiquitous applications hampers their efforts. By proposing such a framework, we hope to help researchers compare results, create ubiquitous computing design guidelines, develop effective discount evaluation techniques, understand the appropriateness of different evaluation techniques, and develop a more complete structure so they can avoid overlooking key areas of evaluation.

[1]  Anind K. Dey,et al.  Heuristic evaluation of ambient displays , 2003, CHI '03.

[2]  Alan Borning,et al.  Value Sensitive Design: Theory and Methods , 2002 .

[3]  R.A. Brooks,et al.  The Intelligent Room project , 1997, Proceedings Second International Conference on Cognitive Technology Humanizing the Information Age.

[4]  R. John Brockmann,et al.  Handbook of screen format design , 1984, IEEE Transactions on Professional Communication.

[5]  Gaetano Borriello,et al.  Labscape: A Smart Environment for the Cell Biology Laboratory , 2002, IEEE Pervasive Comput..

[6]  Gregory D. Abowd,et al.  Cyberguide: A mobile context‐aware tour guide , 1997, Wirel. Networks.

[7]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[8]  Paul Dourish,et al.  Awareness and coordination in shared workspaces , 1992, CSCW '92.

[9]  S. A. Bly,et al.  A comparison of tiled and overlapping windows , 1986, CHI '86.

[10]  Adam L. Janin,et al.  Several devils in the details: making an AR application work in the airplane factory , 1999 .

[11]  A. Jameson Adaptive interfaces and agents , 2002 .

[12]  Paul Dourish,et al.  Introduction to This Special Issue on Context-Aware Computing , 2001, Hum. Comput. Interact..

[13]  Stuart Bennett,et al.  The use of Expert-EASE's 'EASE+' package to develop user-interfaces for mathematical modelling and simulation , 1990 .

[14]  Sidney L. Smith,et al.  Guidelines for Designing User Interface Software , 1986 .

[15]  Gregory D. Abowd,et al.  A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications , 2001, Hum. Comput. Interact..

[16]  Austin Henderson,et al.  Making sense of sensing systems: five questions for designers and researchers , 2002, CHI.

[17]  Jenna Burrell,et al.  Context-Aware Computing: A Test Case , 2002, UbiComp.

[18]  Mark Weiser The computer for the 21st century , 1991 .

[19]  D AbowdGregory,et al.  Lessons learned from eClass , 2004 .

[20]  Mirjana Spasojevic,et al.  Rememberer: A Tool for Capturing Museum Visits , 2002, UbiComp.

[21]  Gregory D. Abowd,et al.  Lessons learned from eClass: Assessing automated capture and access in the classroom , 2004, TCHI.

[22]  R. Fairlie Unleashing the Killer App , 1999 .

[23]  Armando Fox,et al.  System Software for Ubiquitous Computing , 2022 .

[24]  Jonathan Trevor,et al.  Issues in Personalizing Shared Ubiquitous Devices , 2002, UbiComp.

[25]  Jonathan Grudin,et al.  Why CSCW applications fail: problems in the design and evaluationof organizational interfaces , 1988, CSCW '88.

[26]  W. Keith Edwards,et al.  Intelligibility and Accountability: Human Considerations in Context-Aware Systems , 2001, Hum. Comput. Interact..

[27]  Mark Billinghurst,et al.  Crossing the Chasm , 2001 .

[28]  Thorstein Lunde,et al.  KISS the Tram: Exploring the PDA as Support for Everyday Activities , 2001, UbiComp.

[29]  Jonathan J. Cadiz,et al.  Interaction Issues in Context-Aware Intelligent Environments , 2001, Hum. Comput. Interact..

[30]  P. H. Kahn,et al.  Human values, ethics, and design , 2002 .

[31]  Robert W. Proctor,et al.  Human information processing: an overview for human-computer interaction , 2002 .