The Communicability of Graphical Alternatives to Tabular Displays of Statistical Simulation Studies

Simulation studies are often used to assess the frequency properties and optimality of statistical methods. They are typically reported in tables, which may contain hundreds of figures to be contrasted over multiple dimensions. To assess the degree to which these tables are fit for purpose, we performed a randomised cross-over experiment in which statisticians were asked to extract information from (i) such a table sourced from the literature and (ii) a graphical adaptation designed by the authors, and were timed and assessed for accuracy. We developed hierarchical models accounting for differences between individuals of different experience levels (under- and post-graduate), within experience levels, and between different table-graph pairs. In our experiment, information could be extracted quicker and, for less experienced participants, more accurately from graphical presentations than tabular displays. We also performed a literature review to assess the prevalence of hard-to-interpret design features in tables of simulation studies in three popular statistics journals, finding that many are presented innumerately. We recommend simulation studies be presented in graphical form.

[1]  Joachim Meyer,et al.  Multiple Factors that Determine Performance with Tables and Graphs , 1997, Hum. Factors.

[2]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[3]  William Remus,et al.  An Empirical Investigation of the Impact of Graphical and Tabular Data Presentations on Decision Making , 1984 .

[4]  Wayne A. Fuller,et al.  Testing for Trend in the Presence of Autoregressive Error , 2004 .

[5]  P. Fayers,et al.  The Visual Display of Quantitative Information , 1990 .

[6]  Peter L. Brooks,et al.  Visualizing data , 1997 .

[7]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[8]  B. H. Mahon,et al.  Statistics and Decisions : The Importance of Communication and the Power of Graphical Presentation , 1977 .

[9]  Edward R. Tufte,et al.  The Visual Display of Quantitative Information , 1986 .

[10]  A. Brix Bayesian Data Analysis, 2nd edn , 2005 .

[11]  Howard Wainer,et al.  Extracting Sunbeams From Cucumbers , 2011 .

[12]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[13]  M. Wells,et al.  GENERALIZED THRESHOLDING ESTIMATORS FOR HIGH-DIMENSIONAL LOCATION PARAMETERS , 2010 .

[14]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[15]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[16]  Andrew Ehrenberg,et al.  Rudiments of Numeracy , 1977 .

[17]  Issei Fujishiro,et al.  The elements of graphing data , 2005, The Visual Computer.

[18]  Myunghee C. Paik,et al.  Efficiencies of methods dealing with missing covariates in regression analysis , 2006 .

[19]  Paul Murrell,et al.  R Graphics , 2006, Computer science and data analysis series.

[20]  S. Lewandowsky,et al.  Displaying proportions and percentages , 1991 .

[21]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[22]  Andrew Gelman,et al.  Let's Practice What We Preach , 2002 .

[23]  Howard Wainer,et al.  Improving Tabular Displays, With NAEP Tables as Examples and Inspirations , 1997 .

[24]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[25]  Han Su Pictures at an Exhibition , 2007 .

[26]  Joachim Meyer,et al.  Information Structure and the Relative Efficacy of Tables and Graphs , 1999, Hum. Factors.

[27]  Livia De Giovanni,et al.  Confidence intervals for the long memory parameter based on wavelets and resampling , 2008 .

[28]  Andrew Gelman,et al.  Why Tables Are Really Much Better Than Graphs , 2011 .

[29]  Howard Wainer,et al.  Visual Revelations , 2009 .

[30]  Stephen M. S. Lee,et al.  Iterated smoothed bootstrap confidence intervals for population quantiles , 2005, math/0504516.

[31]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .