How Deceptive are Deceptive Visualizations?: An Empirical Analysis of Common Distortion Techniques

In this paper, we present an empirical analysis of deceptive visualizations. We start with an in-depth analysis of what deception means in the context of data visualization, and categorize deceptive visualizations based on the type of deception they lead to. We identify popular distortion techniques and the type of visualizations those distortions can be applied to, and formalize why deception occurs with those distortions. We create four deceptive visualizations using the selected distortion techniques, and run a crowdsourced user study to identify the deceptiveness of those visualizations. We then present the findings of our study and show how deceptive each of these visual distortion techniques are, and for what kind of questions the misinterpretation occurs. We also analyze individual differences among participants and present the effect of some of those variables on participants' responses. This paper presents a first step in empirically studying deceptive visualizations, and will pave the way for more research in this direction.

[1]  Bernice E. Rogowitz,et al.  How not to lie with visualization , 1996 .

[2]  Pierre Dragicevic,et al.  Assessing the Effect of Visualizations on Bayesian Reasoning through Crowdsourcing , 2012, IEEE Transactions on Visualization and Computer Graphics.

[3]  Michelle A. Borkin,et al.  What Makes a Visualization Memorable? , 2013, IEEE Transactions on Visualization and Computer Graphics.

[4]  Eytan Adar,et al.  The impact of social information on visual judgments , 2011, CHI.

[5]  Steven Pinker,et al.  A theory of graph comprehension. , 1990 .

[6]  P. Fayers,et al.  The Visual Display of Quantitative Information , 1990 .

[7]  Jeffrey Heer,et al.  Perceptual Guidelines for Creating Rectangular Treemaps , 2010, IEEE Transactions on Visualization and Computer Graphics.

[8]  Haim Levkowitz,et al.  Color scales for image data , 1992, IEEE Computer Graphics and Applications.

[9]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[10]  Jing Li,et al.  Judging Correlation from Scatterplots and Parallel Coordinate Plots , 2010, Inf. Vis..

[11]  Juliane Junker,et al.  Artificial Intelligence And The Future Of Testing , 2016 .

[12]  Desney S. Tan,et al.  Benevolent deception in human computer interaction , 2013, CHI.

[13]  Gerald E. Jones.  How to Lie with Charts , 1995 .

[14]  L M Martinez,et al.  Media matters. , 2005, Women in action.

[15]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[16]  Jef I. Richards.  Deceptive Advertising: Behavioral Study of A Legal Concept , 1990 .

[17]  Jean-Daniel Fekete,et al.  A Principled Way of Assessing Visualization Literacy , 2014, IEEE Transactions on Visualization and Computer Graphics.

[18]  Jeffrey Heer,et al.  Narrative Visualization: Telling Stories with Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[19]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[20]  Wg.Cdr. Pongphet Congpuong.  How to lie With Statistics , 2013 .

[21]  Franziska Marquart,et al.  Communication and persuasion : central and peripheral routes to attitude change , 1988 .

[22]  Oded Nov,et al.  Personality-targeted design: theory, experimental procedure, and preliminary results , 2013, CSCW.

[23]  Oded Nov,et al.  Exploring personality-targeted UI design in online social participation systems , 2013, CHI.

[24]  Cynthia A. Brewer.  Spectral Schemes: Controversial Color Use on Maps , 1997 .

[25]  Webster.  Webster's Dictionary , 1986 .

[26]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[27]  W. Hays.  Semiology of Graphics: Diagrams Networks Maps. , 1985 .

[28]  P. Shah,et al.  Review of Graph Comprehension Research: Implications for Instruction , 2002 .

[29]  Oded Nov,et al.  The Persuasive Power of Data Visualization , 2014, IEEE Transactions on Visualization and Computer Graphics.

[30]  Edward R. Tufte,et al.  The Visual Display of Quantitative Information , 1986 .

[31]  John J. Bertin,et al.  The semiology of graphics , 1983 .

[32]  Aner Tal,et al.  Blinded with science: Trivial graphs and formulas increase ad persuasiveness and belief in product efficacy , 2016, Public understanding of science.

[33]  Richard Rubinstein,et al.  The Human Factor: Designing Computer Systems for People , 1984 .

[34]  Ronald A. Rensink,et al.  The Perception of Correlation in Scatterplots , 2010, Comput. Graph. Forum.

[35]  M. Monmonier.  How to Lie with Maps , 1991 .

[36]  David Borland,et al.  Rainbow Color Map (Still) Considered Harmful , 2007, IEEE Computer Graphics and Applications.

[37]  Carl Gutwin,et al.  Useful junk?: the effects of visual embellishment on comprehension and memorability of charts , 2010, CHI.

[38]  Ralph Lengler.  Identifying the Competencies of 'Visual Literacy' - a Prerequisite for Knowledge Visualization , 2006, Tenth International Conference on Information Visualisation (IV'06).