Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning

Psychology has historically been concerned, first and foremost, with explaining the causal mechanisms that give rise to behavior. Randomized, tightly controlled experiments are enshrined as the gold standard of psychological research, and there are endless investigations of the various mediating and moderating variables that govern various behaviors. We argue that psychology’s near-total focus on explaining the causes of behavior has led much of the field to be populated by research programs that provide intricate theories of psychological mechanism but that have little (or unknown) ability to predict future behaviors with any appreciable accuracy. We propose that principles and techniques from the field of machine learning can help psychology become a more predictive science. We review some of the fundamental concepts and tools of machine learning and point out examples where these concepts have been used to conduct interesting and important psychological research that focuses on predictive research questions. We suggest that an increased focus on prediction, rather than explanation, can ultimately lead us to greater understanding of behavior.

[1]  A. K. Kurtz A Research Test of the Rorschach Test , 1948 .

[2]  R. Wherry IV. Comparison of Cross-Validation with Statistical Inference of Betas and Multiple R From a Single Sample , 1951 .

[3]  C. I. Mosier I. Problems and Designs of Cross-Validation 1 , 1951 .

[4]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[5]  R. Wherry Underprediction from Overfitting: 45 years of Shrinkage. , 1975 .

[6]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[7]  Howard Wainer,et al.  Estimating Coefficients in Linear Models: It Don't Make No Nevermind , 1976 .

[8]  Neal Schmitt,et al.  A Monte Carlo evaluation of three formula estimates of cross-validated multiple correlation. , 1977 .

[9]  R. Dawes Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[10]  P. Meehl Why Summaries of Research on Psychological Theories are Often Uninterpretable , 1990 .

[11]  Michael R. Hagerty,et al.  Comparing the predictive powers of alternative multiple regression models , 1991 .

[12]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[13]  P. Ekman An argument for basic emotions , 1992 .

[14]  J. Shao Linear Model Selection by Cross-validation , 1993 .

[15]  K. Lesch,et al.  Association of Anxiety-Related Traits with a Polymorphism in the Serotonin Transporter Gene Regulatory Region , 1996, Science.

[16]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  R. Ebstein,et al.  Dopamine D4 receptor (D4DR) exon III polymorphism associated with the human personality trait of Novelty Seeking , 1996, Nature Genetics.

[18]  D. Rawlings,et al.  Music Preference and the Five-Factor Model of the NEO Personality Inventory , 1997 .

[19]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[20]  C. Apte,et al.  Data mining with decision trees and decision rules , 1997, Future Gener. Comput. Syst..

[21]  J. Desmond,et al.  Making memories: brain activity that predicts how well visual experience will be remembered. , 1998, Science.

[22]  J. Pennebaker,et al.  Linguistic styles: language use as an individual difference. , 1999, Journal of personality and social psychology.

[23]  L. Arseneault,et al.  Impulsivity predicts problem gambling in low SES adolescent males. , 1999, Addiction.

[24]  Browne,et al.  Cross-Validation Methods. , 2000, Journal of mathematical psychology.

[25]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[26]  V. Carey,et al.  Mixed-Effects Models in S and S-Plus , 2001 .

[27]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001 .

[28]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001, Statistical Science.

[29]  R. L. Holbert,et al.  Structural Equation Modeling in the Communication Sciences, 1995–2000 , 2002 .

[30]  R. MacCallum,et al.  When fit indices and residuals are incompatible. , 2002, Psychological methods.

[31]  S. Gosling,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES The Do Re Mi’s of Everyday Life: The Structure and Personality Correlates of Music Preferences , 2003 .

[32]  Elizabeth A. Olson,et al.  Eyewitness testimony. , 2003, Annual review of psychology.

[33]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[34]  Gerd Gigerenzer,et al.  Do Studies of Statistical Power Have an Effect on the Power of Studies? , 2004 .

[35]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[36]  S. Gosling,et al.  Personality Processes and Individual Differences E-perceptions: Personality Impressions Based on Personal Websites , 2022 .

[37]  Marc Brysbaert,et al.  Lexique 2 : A new French lexical database , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[38]  David R. Anderson,et al.  Understanding AIC and BIC in Model Selection , 2004 .

[39]  Richard Simon,et al.  Bias in error estimation when using cross-validation for model selection , 2006, BMC Bioinformatics.

[40]  Michael J Strube,et al.  SNOOP: A program for demonstrating the consequences of premature and repeated null hypothesis testing , 2006, Behavior research methods.

[41]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[42]  Rebecca Treiman,et al.  The English Lexicon Project , 2007, Behavior research methods.

[43]  J. Ioannidis Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[44]  D. Funder,et al.  Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. , 2008, Journal of personality and social psychology.

[45]  D. Balota,et al.  Moving beyond Coltheart’s N: A new measure of orthographic similarity , 2008, Psychonomic bulletin & review.

[46]  J. Ioannidis,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias , 2008, PloS one.

[47]  Thomas J. Harris,et al.  The use of simplified or misspecified models : Linear case , 2008 .

[48]  Marc Brysbaert,et al.  Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English , 2009, Behavior research methods.

[49]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[50]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[51]  A. Poropat A meta-analysis of the five-factor model of personality and academic performance. , 2009, Psychological bulletin.

[52]  Susan Goldin-Meadow,et al.  Early gesture selectively predicts later language learning. , 2009, Developmental science.

[53]  T. Yarkoni Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power—Commentary on Vul et al. (2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[54]  A. Gelman,et al.  Of Beauty, Sex and Power , 2009 .

[55]  Marco Zorzi,et al.  Beyond single syllables: Large-scale modeling of reading aloud with the Connectionist Dual Process (CDP++) model , 2010, Cognitive Psychology.

[56]  Anthony D. Wagner,et al.  Detecting individual memories through the neural decoding of memory states and past experience , 2010, Proceedings of the National Academy of Sciences.

[57]  Christian Windischberger,et al.  Toward discovery science of human brain function , 2010, Proceedings of the National Academy of Sciences.

[58]  Tal Yarkoni Personality in 100,000 Words: A large-scale analysis of personality and word use among bloggers. , 2010, Journal of research in personality.

[59]  Juliane M. Stopfer,et al.  Facebook Profiles Reflect Actual Personality, Not Self-Idealization , 2010, Psychological science.

[60]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[61]  Marcus R. Munafò,et al.  Dissecting the genetic architecture of human personality , 2011, Trends in Cognitive Sciences.

[62]  Dušica Filipović Đurđević,et al.  An amorphous model for morphological processing in visual comprehension based on naive discriminative learning. , 2011, Psychological review.

[63]  Samuel D. Gosling,et al.  Manifestations of Personality in Online Social Networks: Self-Reported Facebook-Related Behaviors and Observable Profile Information , 2011, Cyberpsychology Behav. Soc. Netw..

[64]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[65]  Galit Shmueli,et al.  To Explain or To Predict? , 2010, 1101.0891.

[66]  Hernando Ombao,et al.  Penalized least squares regression methods and applications to neuroimaging , 2011, NeuroImage.

[67]  Han L. J. van der Maas,et al.  Science Perspectives on Psychological an Agenda for Purely Confirmatory Research on Behalf Of: Association for Psychological Science , 2022 .

[68]  Cameron Marlow,et al.  A 61-million-person experiment in social influence and political mobilization , 2012, Nature.

[69]  Pedro M. Domingos A few useful things to know about machine learning , 2012, Commun. ACM.

[70]  Daragh E. Sibley,et al.  Individual Differences in Visual Word Recognition: Insights from the English Lexicon Project , 2012 .

[71]  G. Loewenstein,et al.  Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling , 2012, Psychological science.

[72]  Jeffrey R. Spies,et al.  Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability , 2012, 1205.4251.

[73]  H. Beek F1000Prime recommendation of False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. , 2012 .

[74]  A. Greenwald PSS 7210 . 1177 / 1745691611434210 GreenwaldMethod – Theory Synergy There Is Nothing So Theoretical as a Good Method , 2012 .

[75]  T. Yarkoni Psychoinformatics: New Horizons at the Interface of the Psychological and Computing Sciences , 2012 .

[76]  J. Wicherts,et al.  The Rules of the Game Called Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[77]  S. Vrieze Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). , 2012, Psychological methods.

[78]  J. Ioannidis Why Science Is Not Necessarily Self-Correcting , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[79]  G. Miller,et al.  Science Perspectives on Psychological the Smartphone Psychology Manifesto on Behalf Of: Association for Psychological Science the Smartphone Psychology Manifesto Previous Research Using Mobile Electronic Devices What Smartphones Can Do Now and Will Be Able to Do in the near Future , 2022 .

[80]  C. Gamble,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review , 2013, PloS one.

[81]  Arthur W. Toga,et al.  Human neuroimaging as a “Big Data” science , 2013, Brain Imaging and Behavior.

[82]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[83]  T. Graepel,et al.  Private traits and attributes are predictable from digital records of human behavior , 2013, Proceedings of the National Academy of Sciences.

[84]  N. Wray,et al.  A mega-analysis of genome-wide association studies for major depressive disorder , 2013, Molecular Psychiatry.

[85]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[86]  Brian A. Nosek,et al.  Registered Reports A Method to Increase the Credibility of Published Results , 2014 .

[87]  Yong Tao,et al.  Compound facial expressions of emotion , 2014, Proceedings of the National Academy of Sciences.

[88]  David J. Fleet,et al.  Computer Vision – ECCV 2014 , 2014, Lecture Notes in Computer Science.

[89]  Anthony G. Greenwald,et al.  Psychology data from the Race Implicit Association Test on the Project Implicit Demo website , 2014 .

[90]  M. J. O’Brien,et al.  Mapping collective behavior in the big-data era , 2014, Behavioral and Brain Sciences.

[91]  D. A. Kenny,et al.  Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. , 2014, Journal of experimental psychology. General.

[92]  M. Rietschel,et al.  Neuropsychosocial profiles of current and future adolescent alcohol misusers , 2014, Nature.

[93]  Ljubomir J. Buturovic,et al.  Cross-validation pitfalls when selecting and assessing regression and classification models , 2014, Journal of Cheminformatics.

[94]  Ross M. Fraser,et al.  Defining the role of common variation in the genomic and biological architecture of adult human height , 2014, Nature Genetics.

[95]  Reginald B. Adams,et al.  Investigating Variation in Replicability: A “Many Labs” Replication Project , 2014 .

[96]  J. Dana,et al.  Comparing the accuracy of experimental estimates to guessing: a new perspective on replication and the “Crisis of Confidence” in psychology , 2013, Behavior Research Methods.

[97]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[98]  Yoni K. Ashar,et al.  Interactions between donor Agreeableness and recipient characteristics in predicting charitable donation and positive social evaluation , 2015, PeerJ.

[99]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[100]  Daniel M McNeish,et al.  Using Lasso for Predictor Selection and to Assuage Overfitting: A Method Long Overlooked in Behavioral Sciences , 2015, Multivariate behavioral research.

[101]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[102]  Tal Yarkoni,et al.  Statistically Controlling for Confounding Constructs Is Harder than You Think , 2016, PloS one.

[103]  Kai J. Jonas,et al.  How can preregistration contribute to research in our field , 2016 .

[104]  Stuart J. Ritchie,et al.  Genome-wide analysis of over 106 000 individuals identifies 9 neuroticism-associated loci , 2015, Molecular Psychiatry.

[105]  J. Pell,et al.  Erratum: Genome-wide analysis of over 106 000 individuals identifies 9 neuroticism-associated loci , 2016, Molecular psychiatry.

[106]  Brian A. Nosek,et al.  Many Labs 3: Evaluating participant pool quality across the academic semester via replication , 2016 .

[107]  D. A. Kenny,et al.  Experiments with More Than One Random Factor: Designs, Analytic Models, and Statistical Power , 2017, Annual review of psychology.

[108]  D. Donoho 50 Years of Data Science , 2017 .

[109]  Y. Watanabe,et al.  IV , 2018, Out of the Shadow.

[110]  A. Gelman,et al.  The garden of forking paths : Why multiple comparisons can be a problem , even when there is no “ fishing expedition ” or “ p-hacking ” and the research hypothesis was posited ahead of time ∗ , 2019 .

[111]  A. Gelman,et al.  Of Beauty , Sex and Power Too little attention has been paid to the statistical challenges in estimating small effects , 2022 .