The paradox of the guided user: assistance can be counter-effective

This project investigates the conditions under which externalizing interface information by interface controls influences users’ performance in solving problems requiring planning. Our main research question was: in tasks where planning is required, which interface style leads to more plan-based behavior, better strategy, and consequently better task performance? And besides immediate performance, which interface style causes better knowledge of the tasks and solutions afterwards in a transfer situation (with altered task/interface circumstances), or when a severe task interruption occurs? To answer our questions, two series of controlled experiments were conducted using two interface styles: one version in which certain information is externalized onto the interface (Externalization) and another version where this is not done (Internalization). In the Externalization version the operators in the interface conveyed information, in the Internalization version this was not the case. The first series of experiments used a computerized isomorphic version of the well-known “Missionaries & Cannibals” problem, called “Ball & Boxes”. The second series of experiments used a more realistic office-like application called “Conference Planner”. Immediate and delayed performance when using the Externalization interface, was worse than when performance use the Internalization interface. Also transfer of skill was worse for users of the Externalization interface, both to another task, and to another interface. These users were characterized by display-based behavior. Subjects that used the Internalization interface imprinted relevant task and rule knowledge better and were not affected by a severe interruption in the workflow, whereas Externalization subjects were. We conclude that users who internalize information themselves behave more plan-based, are more proactive and ready to make inferences. This in turn results in more focus, more direct and economical solutions, better strategies, and better imprinting of knowledge. This knowledge is easier to recall at a future point in time, and is better transferable to transfer situations where the interface, the task, or both were different, less vulnerable to a severe interruption, and better applicable to transfer situations. Human-computer interaction designers can take advantage from considerations that go beyond plain usability, even when they go against common sense. In designing interfaces we have to take care with providing interface cues that give away too much information, and must design in such a manner that the way users (should) think is optimally supported, which in turn could help the software to achieve its specific goal. Examples are situations where risky and complex tasks are performed, and where a user suddenly is confronted with a new situation. One can also think of situations in which interruptions are commonplace, or where operations come with a cost and direct solutions without deviations are the aim. Based on our findings an interaction framework is proposed that can guide decisions regarding interface design.

[1]  Rosalind W. Picard,et al.  External Representation of Learning Process and Domain Knowledge: Affective State as a Determinate of its Structure and Function , 2001 .

[2]  John Sweller,et al.  Instructional Design in Technical Areas , 1999 .

[3]  Paul A. Kirschner,et al.  Cognitive load theory: implications of cognitive load theory on the design of learning , 2002 .

[4]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[5]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[6]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[7]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[8]  Kenton O'Hara Cost of operations affects planfulness of problem-solving behaviour , 1994, CHI '94.

[9]  Eshaa M. Alkhalifa Cognitively Informed Systems: Utilizing Practical Approaches to Enrich Information Presentation and Transfer , 2006 .

[10]  H. Simon,et al.  What makes some problems really hard: Explorations in the problem space of difficulty , 1990, Cognitive Psychology.

[11]  Herre van Oostendorp,et al.  Can more help be worse?: the over-assisting interface , 2004 .

[12]  Brianna Scott,et al.  Navigational spatial displays: The role of metacognition as cognitive load * , 2007 .

[13]  Antti Oulasvirta,et al.  Surviving task interruptions: Investigating the implications of long-term working memory theory , 2006, Int. J. Hum. Comput. Stud..

[14]  Richard Catrambone,et al.  Designing Instructional Examples to Reduce Intrinsic Cognitive Load: Molar versus Modular Presentation of Solution Procedures , 2004 .

[15]  Morten Kyng,et al.  Design at Work , 1992 .

[16]  J. Cacioppo,et al.  DISPOSITIONAL DIFFERENCES IN COGNITIVE MOTIVATION : THE LIFE AND TIMES OF INDIVIDUALS VARYING IN NEED FOR COGNITION , 1996 .

[17]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[18]  E. Hutchins Cognition in the wild , 1995 .

[19]  D. Norman The psychology of everyday things , 1990 .

[20]  Jeffrey Rubin,et al.  Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests , 1994 .

[21]  P M Todd,et al.  Précis of Simple heuristics that make us smart , 2000, Behavioral and Brain Sciences.

[22]  Martin Valcke,et al.  Cognitive load: updating the theory? , 2002 .

[23]  H. van Oostendorp,et al.  Does context sensitivity in the interface help , 2004 .

[24]  Lucy A. Suchman,et al.  Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, , 1987 .

[25]  George W. Ernst,et al.  GPS : a case study in generality and problem solving , 1971 .

[26]  Daniel Burgos,et al.  Performance in a planning task: The (ir)relevance of interface style and user's cognitive style , 2005 .

[27]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987 .

[28]  Herre van Oostendorp,et al.  Guidance in the Interface: Effects of Externalizing Information During Problem Solving , 2006 .

[29]  C. F. Kao,et al.  The efficient assessment of need for cognition. , 1984, Journal of personality assessment.

[30]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[31]  Richard E. Mayer,et al.  Meaningful Design for Meaningful Learning: Applying Cognitive Theory to Multimedia Explanations , 2000 .

[32]  Ernesto Arroyo,et al.  Usability tool for analysis of web designs using mouse tracks , 2006, CHI Extended Abstracts.

[33]  R. Mayer Thinking, Problem Solving, Cognition , 1983 .

[34]  Jocelyn Wishart,et al.  Cognitive factors related to user involvement with computers and their effects upon learning from an educational computer game , 1990 .

[35]  Jakob Nielsen,et al.  The Anti-Mac interface , 1996, CACM.

[36]  Peter Brusilovsky,et al.  Adaptive Hypermedia , 2001, User Modeling and User-Adapted Interaction.

[37]  W. Schnotz,et al.  A Reconsideration of Cognitive Load Theory , 2007 .

[38]  Jiajie Zhang,et al.  External representation: An issue for cognition , 1993, Behavioral and Brain Sciences.

[39]  Jiajie Zhang,et al.  The Nature of External Representations in Problem Solving , 1997, Cogn. Sci..

[40]  Eero Sormunen,et al.  Learning Information Retrieval through an Educational Game. Is Gaming sufficient for learning? , 2000, Educ. Inf..

[41]  Herre van Oostendorp,et al.  The paradox of the assisted user: guidance can be counterproductive , 2006, CHI.

[42]  BurgosD.,et al.  Game-based learning and the role of feedback , 2007 .

[43]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[44]  Heiner Deubel,et al.  The mind's eye : cognitive and applied aspects of eye movement research , 2003 .

[45]  Brian Shackel,et al.  Usability - Context, framework, definition, design and evaluation , 1991, Interact. Comput..

[46]  Cliff McKnight,et al.  Navigation in Hypertext: Structural Cues and Mental Maps , 1989, UK Hypertext.

[47]  D. Berlyne A theory of human curiosity. , 1954, British journal of psychology.

[48]  H. van Oostendorp,et al.  The Role of Interface Style in Planning during Problem Solving , 2005 .

[49]  Nicole Yankelovich,et al.  Context and orientation in hypermedia networks , 1989, TOIS.

[50]  M A Neerincx,et al.  Cognitive task analysis: harmonizing tasks to human capacities. , 1996, Ergonomics.

[51]  Eero Sormunen,et al.  Learning information retrieval through an educational game , 2000 .

[52]  A. Localio,et al.  Role of computerized physician order entry systems in facilitating medication errors. , 2005 .

[53]  Herre van Oostendorp,et al.  Guidance in the interface and transfer of task performance , 2007, ECCE '07.

[54]  A. Inhoff,et al.  Chapter 2 – Definition and Computation of Oculomotor Measures in the Study of Cognitive Processes , 1998 .

[55]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[56]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[57]  R. Jeffries,et al.  A process model for Missionaries-Cannibals and other river-crossing problems , 1977, Cognitive Psychology.

[58]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.

[59]  H. van Oostendorp,et al.  The over-assisting interface: The role of support in knowledge acquisition , 2004 .

[60]  D. Broadbent,et al.  Two modes of learning for interactive tasks , 1988, Cognition.

[61]  K. Holyoak,et al.  Schema induction and analogical transfer , 1983, Cognitive Psychology.

[62]  Stephen J. Payne,et al.  Reflection and goal management in exploratory learning , 1995, Int. J. Hum. Comput. Stud..

[63]  Sharon L. Oviatt,et al.  Quiet interfaces that help students think , 2006, UIST.

[64]  C. van Nimwegen,et al.  Externalization vs. internalization: the influence on problem solving performance , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[65]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[66]  Rob Koper,et al.  Does an Interface with Less Assistance Provoke More Thoughtful Behavior? , 2006, ICLS.

[67]  Gunnvald B. Svendsen,et al.  The Influence of Interface Style on Problem Solving , 1991, Int. J. Man Mach. Stud..

[68]  F. Hesse,et al.  When are powerful learning environments effective? The role of learner activities and of students’ conceptions of educational technology , 2004 .

[69]  Jiajie Zhang,et al.  Representations in Distributed Cognitive Tasks , 1994, Cogn. Sci..

[70]  Jean-Michel Boucheix,et al.  Multimedia simulation and comprehension aid for complex technical documents: a course for crane drivers , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[71]  K. Kiili Evaluations of an experiential gaming model , 2006 .

[72]  Robert J. K. Jacob,et al.  Eye tracking in human-computer interaction and usability research : Ready to deliver the promises , 2002 .

[73]  Hunter Heath Making It Easier , 1992 .

[74]  イントランス Human interface guidelines : the apple desktop interface , 1989 .

[75]  G. Loewenstein The psychology of curiosity: A review and reinterpretation. , 1994 .

[76]  Stephen J. Payne,et al.  Display-Based Action at the User Interface , 1991, Int. J. Man Mach. Stud..

[77]  Amy Wohl,et al.  Apple computer, Inc. , 2003 .

[78]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[79]  John Millar Carroll The Nurnberg Funnel: Designing Minimalist Instruction for Practical Computer Skill , 1990 .

[80]  Rik Pieters,et al.  "Neiging tot nadenken": Samenhang met beredeneerd gedrag. , 1987 .

[81]  Wolfgang Schnotz,et al.  Enabling, facilitating, and inhibiting effects of animations in multimedia learning: Why reduction of cognitive load can have negative results on learning , 2005 .

[82]  Henk G. Schmidt,et al.  Cognitive Load Theory as a Tool for Expertise Development , 2004 .

[83]  Roy Rada,et al.  Interacting With Hypertext: A Meta-Analysis of Experimental Studies , 1996, Hum. Comput. Interact..

[84]  J. H. Larkin,et al.  Display-based problem solving , 1989 .

[85]  Stephen J. Payne,et al.  Self-monitoring during exploration of an interactive device , 1996, Int. J. Hum. Comput. Stud..

[86]  Neville A. Stanton,et al.  Maps as Navigational Aids in Hypertext Environments: An Empirical Evaluation. , 1992 .

[87]  C. Weinstein,et al.  Self-regulated learning: The interactive influence of metacognitive awareness and goal-setting , 1992 .

[88]  Wai-Tat Fu,et al.  Resolving the paradox of the active user: stable suboptimal performance in interactive tasks , 2004, Cogn. Sci..

[89]  Wai-Tat Fu,et al.  Ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2001, CHI.

[90]  H. Simon,et al.  Why are some problems hard? Evidence from Tower of Hanoi , 1985, Cognitive Psychology.

[91]  Kenton O'Hara,et al.  Planning and the user interface: the effects of lockout time and error recovery cost , 1999, Int. J. Hum. Comput. Stud..

[92]  J. Cacioppo,et al.  The need for cognition. , 1982 .

[93]  Jan Noyes,et al.  Effect of experience and mode of presentation on problem solving , 2007, Comput. Hum. Behav..

[94]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[95]  S. Feinberg,et al.  Applying cognitive load theory to the design of Web-based instruction , 2000, 18th Annual Conference on Computer Documentation. ipcc sigdoc 2000. Technology and Teamwork. Proceedings. IEEE Professional Communication Society International Professional Communication Conference an.

[96]  Phillip L. Morgan,et al.  Influencing Cognitive Strategy by Manipulating Information Access , 2007, Comput. J..

[97]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[98]  Brian Shackel,et al.  Human factors for informatics usability , 1991 .

[99]  C. Bray Transfer of learning. , 1928 .

[100]  S. Kaul,et al.  Trial and error. How to avoid commonly encountered limitations of published clinical trials. , 2010, Journal of the American College of Cardiology.

[101]  Jacob Lenting Informed gambling : conception and analysis of a multi-agent mechanism for discrete reallocation , 1999 .

[102]  Keith Oatley,et al.  Information Flow in a User Interface: the Effect of Experience and Context on the Recall of MacWrite Screens , 1988, BCS HCI.

[103]  Herbert A. Simon,et al.  Situated Action: A Symbolic Interpretation , 1993, Cogn. Sci..

[104]  Herre van Oostendorp,et al.  The questionable impact of an assisting interface on performance in transfer situations. , 2009 .

[105]  John R. Anderson The Adaptive Character of Thought , 1990 .