An evaluation of how small user interface changes can improve scientists' analytic strategies

Subtle changes in analysis system interfaces can be used purposely to alter users' analytic behaviors. In a controlled study subjects completed three analyses at one-week intervals using an analysis support system. Control subjects used one interface in all sessions. Test subjects used modified versions in the last two sessions: a first set of changes aimed at increasing subjects' use of the system and their consideration of alternative hypotheses; a second set of changes aimed at increasing the amount of evidence collected. Results show that in the second session test subjects used the interface 39% more and switched between hypotheses 19% more than in the first session. They then collected 26% more evidence in the third than in the second session. These increases differ significantly (p<0.05) from near constant control rates. We hypothesize that this approach can be used in many real applications to guide analysts unobtrusively towards improved analytic strategies.

[1]  M. Deutsch,et al.  A study of normative and informational social influences upon individual judgement. , 1955, Journal of abnormal psychology.

[2]  S. Asch.  Studies of independence and conformity: I. A minority of one against a unanimous majority. , 1956 .

[3]  M. Potter,et al.  Interference in Visual Recognition , 1964, Science.

[4]  P C Wason,et al.  Reasoning about a Rule , 1968, The Quarterly journal of experimental psychology.

[5]  Du Charme,et al.  A response bias explanation of conservative human inference , 1969 .

[6]  H. Simon,et al.  Rationality as Process and as Product of Thought , 1978 .

[7]  J. Klayman,et al.  Confirmation, Disconfirmation, and Informa-tion in Hypothesis Testing , 1987 .

[8]  J. Kagan,et al.  Rational choice in an uncertain world , 1988 .

[9]  G. Klein,et al.  A recognition-primed decision (RPD) model of rapid decision making. , 1993 .

[10]  Kevin Dunbar,et al.  Concept Discovery in a Scientific Domain , 1993, Cogn. Sci..

[11]  John E. Hunter,et al.  The Discard of Study Evidence by Literature Reviewers , 1994 .

[12]  B. J. Fogg,et al.  Persuasive computers: perspectives and research directions , 1998, CHI.

[13]  Gershon Ben-Shakhar,et al.  Seek and Ye Shall Find: Test Results Are What You Hypothesize They Are , 1998 .

[14]  T. Chartrand,et al.  The chameleon effect: the perception-behavior link and social interaction. , 1999, Journal of personality and social psychology.

[15]  Kevin Dunbar,et al.  What Scientific Thinking Reveals About the Nature of Cognition , 2000 .

[16]  G. Loftus.  OF EXPERIMENTAL DATA , 2001 .

[17]  K. Crowley,et al.  Designing for Science: Implications from Everyday, Classroom, and Professional Settings. , 2001 .

[18]  Charles Stangor,et al.  Social influence and intergroup beliefs: The role of perceived social consensus. , 2001 .

[19]  Jon K. Maner,et al.  Dynamical Evolutionary Psychology: Mapping the Domains of the New Interactionist Paradigm , 2002 .

[20]  A. Elstein,et al.  Clinical problem solving and diagnostic decision making: selective review of the cognitive literature , 2002, BMJ : British Medical Journal.

[21]  R. Thaler,et al.  Libertarian Paternalism is Not an Oxymoron , 2003 .

[22]  R. Thaler,et al.  Save More Tomorrow™: Using Behavioral Economics to Increase Employee Saving , 2004, Journal of Political Economy.

[23]  Noah J. Goldstein,et al.  Social influence: compliance and conformity. , 2004, Annual review of psychology.

[24]  Wai-Tat Fu,et al.  Soft constraints in interactive behavior: the case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2004, Cogn. Sci..

[25]  Ernesto Arroyo,et al.  Waterbot: exploring feedback and persuasive techniques at the sink , 2005, CHI.

[26]  Taemie Jung Kim,et al.  Dynamic speedometer: dashboard redesign to discourage drivers from speeding , 2005, CHI EA '05.

[27]  Marco Arguedas,et al.  Concept Maps: Integrating Knowledge and Information Visualization , 2005, Knowledge and Information Visualization.

[28]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[29]  Michelle X. Zhou,et al.  Interactive Visual Synthesis of Analytic Knowledge , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[30]  Alexander W. Skaburskis,et al.  The Sandbox for analysis: concepts and methods , 2006, CHI.

[31]  Oded Nov,et al.  What motivates Wikipedians? , 2007, CACM.

[32]  Matthew O. Ward,et al.  Nugget discovery in visual exploration environments by query consolidation , 2007, CIKM '07.

[33]  Robert Harper,et al.  Stories in GeoTime , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[34]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[35]  William Ribarsky,et al.  Visual analytics for complex concepts using a human cognition model , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[36]  John T. Stasko,et al.  Jigsaw: Supporting Investigative Analysis through Interactive Visualization , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[37]  Harri Oinas-Kukkonen,et al.  Towards Deeper Understanding of Persuasion in Software and Information Systems , 2008, First International Conference on Advances in Computer-Human Interaction.

[38]  David S. Ebert,et al.  Applied visual analytics for economic decision-making , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[39]  Stuart K. Card,et al.  Entity-based collaboration tools for intelligence analysis , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[40]  William Ribarsky,et al.  Building and Applying a Human Cognition Model for Visual Analytics , 2009, Inf. Vis..

[41]  John T. Stasko,et al.  Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[42]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .