Testing the accuracy of the retrospective recall method used in expertise research

Expertise typically develops slowly over years, and controlled experiments to study its development may be impractical. Researchers often use a correlational, retrospective recall method in which participants recall career data, sometimes over many years before. However, recall accuracy is uncertain. The present study investigated the accuracy of recalled career data for up to 38 years, in over 600 international chess players. Participants’ estimates of their entry year into international chess, total career games played, and number of games in a typical year were compared with the known true values. Entry year typically was recalled fairly accurately, and accuracy did not diminish systematically with time since list entry from 10 years earlier to 25 or more years earlier. On average, games-count estimates were reasonably accurate. However, some participants were very inaccurate, and some were more inaccurate in their total-games counts and entry-year estimates. The retrospective recall method yields usable data but may have some accuracy problems. Possible remedies are outlined.

[1]  Daniel Elmore,et al.  How good is it , 1998 .

[2]  D. Collins.  Pretesting survey instruments: An overview of cognitive methods , 2003, Quality of Life Research.

[3]  D. Wright.  Recall of the Hillsborough disaster over time: Systematic biases of ‘flashbulb’ memories , 1993 .

[4]  Richard Potts,et al.  Young Adults' Retrospective Reports of Childhood Television Viewing , 2008, Commun. Res..

[5]  Robert W Howard.  A complete database of international chess players and chess performance ratings for varied longitudinal studies , 2006, Behavior research methods.

[6]  Guy Starkey,et al.  Estimating audiences: sampling in television and radio audience research , 2004 .

[7]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[8]  Lance J Rips,et al.  The subjective dates of natural events in very-long-term memory , 1985, Cognitive Psychology.

[9]  F. Gobet,et al.  The role of domain-specific practice, handedness, and starting age in chess. , 2007, Developmental psychology.

[10]  Paul J. Feltovich,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[11]  Neil Charness,et al.  The role of deliberate practice in chess expertise , 2005 .

[12]  G. Willis,et al.  Research Synthesis: The Practice of Cognitive Interviewing , 2007 .

[13]  A. Tekcan,et al.  Autobiographical and event memory for 9/11: changes across one year , 2003 .

[14]  J. Starkes,et al.  Wrestling with the nature expertise: a sport specific test of Ericsson, Krampe and Tesch-Römer's (1993) theory of "deliberate practice". , 1996 .

[15]  J. Ronald Milavsky.  HOW GOOD IS THE A. C. NIELSEN PEOPLE-METER SYSTEM? A REVIEW OF THE REPORT BY THE COMMITTEE ON NATIONWIDE TELEVISION AUDIENCE MEASUREMENT , 1992 .

[16]  P. Ross,et al.  The expert mind. , 2006, Scientific American.

[17]  F. Gobet,et al.  LEFT LATERALIZATION IN AUTOBIOGRAPHICAL MEMORY: AN fMRI STUDY USING THE EXPERT ARCHIVAL PARADIGM , 2008, The International journal of neuroscience.

[18]  J. Cȏté,et al.  The Cambridge Handbook of Expertise and Expert Performance: Time Budgets, Diaries, and Analyses of Concurrent Practice Activities , 2006 .

[19]  A. Elo.  The rating of chessplayers, past and present , 1978 .

[20]  M. Birnbaum.  Human research and data collection via the internet. , 2004, Annual review of psychology.

[21]  Anique B. H. de Bruin,et al.  Deliberate practice predicts performance over time in adolescent chess players and drop-outs: a linear mixed models analysis. , 2008, British journal of psychology.

[22]  M. Conway,et al.  Memory for true and false autobiographical event descriptions , 2004, Memory.

[23]  G. Colvin,et al.  Talent Is Overrated: What Really Separates World-Class Performers from Everybody Else , 2008 .

[24]  Agnieszka Niedźwieńska,et al.  Distortion of autobiographical memories , 2003 .

[25]  K. S. Lewis.  Talent is Overrated , 2011 .

[26]  Janet L. Starkes,et al.  Deliberate Imagery Practice , 2005, Research quarterly for exercise and sport.

[27]  S. Christianson.  Flashbulb memories: Special, but not so special , 1989, Memory & cognition.

[28]  S. Kemp.  Dating recent and historical events , 1988 .

[29]  False memories and individual variations: The role of Field Dependence–Independence , 2009 .

[30]  G. Cohen.  Memory In The Real World , 1989 .

[31]  K. Davids.  Skill acquisition and the theory of deliberate practice: It ain't what your do it's the way that you do it! Commentary on Starkes, L. "The road to expertise: Is practice the only determinant?" , 2000 .

[32]  R. Howard.  Individual differences in expertise development over decades in a complex intellectual domain , 2009, Memory & cognition.

[33]  K. A. Ericsson,et al.  Expert Performance in Sports: Advances in Research on Sport Expertise , 2003 .

[34]  M. Conway,et al.  Reconstructing Dates of True and False Autobiographical Memories , 1997 .

[35]  K. A. Ericsson,et al.  The Influence of Experience and Deliberate Practice on the Development of Superior Expert Performance , 2006 .

[36]  H. P. Bahrick.  Semantic memory content in permastore: fifty years of memory for Spanish learned in school. , 1984, Journal of experimental psychology. General.

[37]  W. Friedman.  Memory for the time of past events. , 1993 .

[38]  K. Malmberg.  Toward an Understanding of Individual Differences in Episodic Memory: Modeling the Dynamics Of Recognition Memory , 2007 .

[39]  Brenda A. Kirchhoff,et al.  Individual Differences in Episodic Memory: The Role of Self-initiated Encoding Strategies , 2009, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[40]  B. Matthes,et al.  Collecting Event Histories with TrueTales:Techniques to Improve Autobiographical Recall Problems in Standardized Interviews , 2007 .