Cognitive control in media multitaskers: Two replication studies and a meta-Analysis

Ophir, Nass, and Wagner (2009, Proceedings of the National Academy of Sciences of the United States of America, 106(37), 15583–15587) found that people with high scores on the media-use questionnaire—a questionnaire that measures the proportion of media-usage time during which one uses more than one medium at the same time—show impaired performance on various tests of distractor filtering. Subsequent studies, however, did not all show this association between media multitasking and distractibility, thus casting doubt on the reliability of the initial findings. Here, we report the results of two replication studies and a meta-analysis that included the results from all published studies into the relationship between distractor filtering and media multitasking. Our replication studies included a total of 14 tests that had an average replication power of 0.81. Of these 14 tests, only five yielded a statistically significant effect in the direction of increased distractibility for people with higher scores on the media-use questionnaire, and only two of these effects held in a more conservative Bayesian analysis. Supplementing these outcomes, our meta-analysis on a total of 39 effect sizes yielded a weak but significant association between media multitasking and distractibility that turned nonsignificant after correction for small-study effects. Taken together, these findings lead us to question the existence of an association between media multitasking and distractibility in laboratory tasks of information processing.

[1]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[2]  Mark W. Becker,et al.  Media Multitasking Is Associated with Symptoms of Depression and Social Anxiety , 2013, Cyberpsychology Behav. Soc. Netw..

[3]  J. Sterne,et al.  Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. , 2001, Journal of clinical epidemiology.

[4]  D. Strayer,et al.  Who Multi-Tasks and Why? Multi-Tasking Ability, Perceived Multi-Tasking Ability, Impulsivity, and Sensation Seeking , 2013, PloS one.

[5]  Walter Schneider,et al.  STEP—A System for Teaching Experimental Psychology using E-Prime , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[6]  Jonathan S. A. Carriere,et al.  Media multitasking and behavioral measures of sustained attention , 2014, Attention, Perception, & Psychophysics.

[7]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[8]  Alex J Sutton,et al.  Performance of the trim and fill method in the presence of publication bias and between‐study heterogeneity , 2007, Statistics in medicine.

[9]  Peter C Gøtzsche,et al.  ' s response to reviews Title : Spontaneous improvement in randomised clinical trials : meta-analysis of three-armed trials comparing no-treatment , placebo and active intervention , 2008 .

[10]  Evan C. Carter,et al.  Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated? , 2014, Front. Psychol..

[11]  Kelvin F H Lui,et al.  Does media multitasking always hurt? A positive correlation between multitasking and multisensory integration , 2012, Psychonomic bulletin & review.

[12]  Jeffrey R. Spies,et al.  The Replication Recipe: What Makes for a Convincing Replication? , 2014 .

[13]  P. Jolicoeur,et al.  A Solution to the Effect of Sample Size on Outlier Elimination , 1994 .

[14]  Jelte M. Wicherts,et al.  On making the right choice: A meta-analysis and large-scale replication attempt of the unconscious thought advantage , 2015, Judgment and Decision Making.

[15]  Anthony D. Wagner,et al.  Media multitasking and memory: Differences in working memory and long-term memory , 2016, Psychonomic bulletin & review.

[16]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[17]  A. Lenhart.  Teens, Social Media & Technology Overview 2015 , 2015 .

[18]  M. Lee,et al.  Statistical Evidence in Experimental Psychology , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[19]  Matthew S Cain,et al.  Distractor filtering in media multitaskers. , 2010, Perception.

[20]  J. Ioannidis,et al.  Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials , 2011, BMJ : British Medical Journal.

[21]  Jennifer Wiley,et al.  What Are the Odds? A Practical Guide to Computing and Reporting Bayes Factors , 2014, J. Probl. Solving.

[22]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[23]  Santiago G. Moreno,et al.  Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study , 2009, BMC medical research methodology.

[24]  J P Vandenbroucke,et al.  Bias in meta-analysis detected by a simple, graphical test , 1998 .

[25]  Mark W. Becker,et al.  The association between media multitasking, task-switching, and dual-task performance. , 2013, Journal of experimental psychology. Human perception and performance.

[26]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[27]  E. Wagenmakers.  A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[28]  Meredith Minear,et al.  Working memory, fluid intelligence, and impulsiveness in heavy media multitaskers , 2013, Psychonomic Bulletin & Review.

[29]  Michael King,et al.  Trap of trends to statistical significance: likelihood of near significant P value becoming more significant with extra data , 2014, BMJ : British Medical Journal.

[30]  C. S. Green,et al.  Technology consumption and cognitive control: Contrasting action video game experience with media multitasking , 2015, Attention, perception & psychophysics.

[31]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[32]  M. S. Patel,et al.  An introduction to meta-analysis. , 1989, Health Policy.

[33]  Ryota Kanai,et al.  Higher Media Multi-Tasking Activity Is Associated with Smaller Gray-Matter Density in the Anterior Cingulate Cortex , 2014, PloS one.

[34]  Brandon C. W. Ralph,et al.  Media multitasking and failures of attention in everyday life , 2014, Psychological research.

[35]  Elizabeth Gilbert,et al.  Reproducibility Project: Results (Part of symposium called "The Reproducibility Project: Estimating the Reproducibility of Psychological Science") , 2014 .

[36]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1997, Statistics in medicine.

[37]  Jeffrey N. Rouder,et al.  Computation of Bayes Factors for Common Designs , 2015 .

[38]  Greg Roberts,et al.  Meta-Analysis With Complex Research Designs , 2014, Review of educational research.

[39]  Hakwan Lau,et al.  Does response interference depend on the subjective visibility of flanker distractors? , 2012, Attention, Perception, & Psychophysics.

[40]  Daniel Smilek,et al.  Individual differences in media multitasking and performance on the n-back , 2017, Attention, perception & psychophysics.

[41]  Clifford Nass,et al.  Cognitive control in media multitaskers , 2009, Proceedings of the National Academy of Sciences.

[42]  Gregor E. Kennedy,et al.  Measurement and evidence of computer-based task switching and multitasking by 'Net Generation' students , 2011, Comput. Educ..

[43]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[44]  Matthew S Cain,et al.  Media multitasking in adolescence , 2016, Psychonomic bulletin & review.

[45]  J. Everett,et al.  Response Styles in a Cross-Cultural Managerial Study , 1984 .

[46]  James Gips,et al.  Media Multitasking Behavior: Concurrent Television and Computer Usage , 2011, Cyberpsychology Behav. Soc. Netw..

[47]  Daniël Lakens,et al.  Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs , 2013, Front. Psychol..

[48]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1999 .

[49]  Jeffrey N. Rouder,et al.  A Bayes factor meta-analysis of Bem’s ESP claim , 2011, Psychonomic bulletin & review.

[50]  B. Becker,et al.  How meta-analysis increases statistical power. , 2003, Psychological methods.

[51]  Susanne E. Baumgartner,et al.  Measuring Media Multitasking: Development of a Short Measure of Media Multitasking for Adolescents , 2017, J. Media Psychol. Theor. Methods Appl..

[52]  J. Palfrey,et al.  Born digital: understanding the first generation of digital natives , 2009, Choice Reviews Online.

[53]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[54]  Nienke Meulman,et al.  An ERP study on L2 syntax processing: When do learners fail? , 2014, Front. Psychol..

[55]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[56]  Maro G. Machizawa,et al.  Neural measures reveal individual differences in controlling access to working memory , 2005, Nature.

[57]  M. Moisala,et al.  Media multitasking is associated with distractibility and increased prefrontal activity in adolescents and young adults , 2016, NeuroImage.

[58]  C Shawn Green,et al.  Short-term mindfulness intervention reduces the negative attentional effects associated with heavy media multitasking , 2016, Scientific Reports.

[59]  J. Sterne,et al.  Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. , 2000, Journal of clinical epidemiology.

[60]  N. Cowan.  The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[61]  Susanne E. Baumgartner,et al.  The Relationship Between Media Multitasking and Executive Function in Early Adolescents , 2014 .

[62]  Stephen Wee Hun Lim,et al.  Media multitasking predicts unitary versus splitting visual focal attention , 2013 .