The Think-Aloud Controversy in Second Language Research

Chapter 1: The use of verbal reports in language research. Chapter 2: Controversy over the use of think-alouds. Chapter 3: Features that Make a Task Amenable to Think-Aloud: A Meta-analysis of Studies Investigating the Validity of Think-Alouds on Verbal Tasks. Chapter 4: Data Collection Considerations. Chapter 5: Data Analysis Considerations. Chapter 6: Conclusion. Appendix A: Studies Included in the Meta-Analysis. Appendix B: Summary of Unique Sample Studies.

[1]  P. David Pearson,et al.  The reading strategies of bilingual Latina/o students who are successful English readers: Opportunities and obstacles , 1996 .

[2]  E. Rosa,et al.  EXPLICITNESS, INTAKE, AND THE ISSUE OF AWARENESS , 1999, Studies in Second Language Acquisition.

[3]  Liz Murphy,et al.  A Temporal Analysis of Formulation Processes in L1 and L2 Writing. , 2001 .

[4]  G. Rijlaarsdam,et al.  Relations Between Writing Processes and Text Quality: When and How? , 1994 .

[5]  P. Sleight,et al.  Effects of controlled breathing, mental activity and mental stress with or without verbalization on heart rate variability. , 2000, Journal of the American College of Cardiology.

[6]  P. Robinson.  Attention, Memory, and the “Noticing” Hypothesis , 1995 .

[7]  Suzanne E. Wade,et al.  Using Think Alouds to Assess Comprehension. , 1990 .

[8]  Helen P Woodfield.  Problematising Discourse Completion Tasks: Voices from Verbal Report , 2008 .

[9]  Andrew D. Cohen,et al.  Strategies in learning and using a second language , 1998 .

[10]  R. Ellis.  MEASURING IMPLICIT AND EXPLICIT KNOWLEDGE OF A SECOND LANGUAGE: A Psychometric Study , 2005, Studies in Second Language Acquisition.

[11]  R. Mathews,et al.  Role of Implicit and Explicit Processes in Learning From Examples: A Synergistic Effect , 1989 .

[12]  Andrew J. Rosman,et al.  Methodological issues in judgment and decision-making research: Concurrent verbal protocol validity and simultaneous traces of process. , 1993 .

[13]  Margaret Thomas,et al.  8. Research synthesis and historiography: The case of assessment of second language proficiency , 2006 .

[14]  Sang-Ki Lee,et al.  VISUAL INPUT ENHANCEMENT AND GRAMMAR LEARNING: A Meta-Analytic Review , 2008, Studies in Second Language Acquisition.

[15]  S. Gass,et al.  Stimulated Recall Methodology in Second Language Research , 2000 .

[16]  Jim McLennan,et al.  Effects of Concurrent Verbalization on a Time-Critical, Dynamic Decision-Making Task , 2000, The Journal of general psychology.

[17]  Timothy D. Wilson,et al.  Thinking too much: introspection can reduce the quality of preferences and decisions. , 1991, Journal of personality and social psychology.

[18]  R. Gagne,et al.  A study of the effects of verbalization on problem solving. , 1962, Journal of experimental psychology.

[19]  Eduard J. Fidler.  The Reliability and Validity of Concurrent, Retrospective, and Interpretive Verbal Reports: An Experimental Study , 1983 .

[20]  S. Gass,et al.  USING STIMULATED RECALL TO INVESTIGATE NATIVE SPEAKER PERCEPTIONS IN NATIVE-NONNATIVE SPEAKER INTERACTION , 2006, Studies in Second Language Acquisition.

[21]  Janet Fraser.  Public Accounts: Using Verbal Protocols to Investigate Community Translation , 1993 .

[22]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[23]  Yu-Fen Yang.  Reading Strategies or Comprehension Monitoring Strategies? , 2006 .

[24]  R. Leow.  A STUDY OF THE ROLE OF AWARENESS IN FOREIGN LANGUAGE BEHAVIOR , 2000, Studies in Second Language Acquisition.

[25]  Stephan M. Koplan.  The Fugitive Literature , 1979 .

[26]  Alison Mackey,et al.  LEARNERS' INTERPRETATIONS OF RECASTS , 2006, Studies in Second Language Acquisition.

[27]  Michael H. Long.  The Role of the Linguistic Environment in Second Language Acquisition , 1996 .

[28]  Kyle Perkins,et al.  An exploratory study into the construct validity of a reading comprehension test: triangulation of data sources , 1991 .

[29]  Ronald P. Leow,et al.  Attention and awareness in SLA , 2005 .

[30]  K. A. Ericsson,et al.  Protocol analysis: Verbal reports as data, Rev. ed. , 1993 .

[31]  R. Weber.  Basic Content Analysis , 1986 .

[32]  Nava Nevo.  Test-taking strategies on a multiple-choice test of reading comprehension , 1989 .

[33]  J. Payne,et al.  Exploring predecisional behavior: An alternative approach to decision research , 1978 .

[34]  D. B. Tarr,et al.  Verbalization, experimenter presence, and problem solving. , 1968, Journal of personality and social psychology.

[35]  M. Ko.  GLOSSES, COMPREHENSION, AND STRATEGY USE , 2005 .

[36]  On the relation between justification of solution method and correctness of solution in statistical problem solving , 1990 .

[37]  Riitta Jääskeläinen,et al.  Focus on Methodology in Think-aloud Studies on Translating , 2000 .

[38]  Clare Wood,et al.  The Role of Lexical Analogies in Beginning Reading: Insights from Children's Self-Reports. , 2007 .

[39]  A. Cohen,et al.  The Production of Speech Acts by EFL Learners , 1993 .

[40]  Marilyn L. Abbott.  ESL Reading Strategies: Differences in Arabic and Mandarin Speaker Test Performance , 2006 .

[41]  B. Rosenshine,et al.  The Use of Scaffolds for Teaching Higher-Level Cognitive Strategies. , 1992 .

[42]  Renée Jourdenais.  Cognition and Second Language Instruction: Cognition, instruction and protocol analysis , 2001 .

[43]  Melissa A. Bowles.  L2 glossing: To CALL or not to CALL , 2004 .

[44]  L. Fowler.  Clinical Reasoning Strategies Used during Care Planning , 1997, Clinical nursing research.

[45]  Janis Harmon.  Assessing and Supporting Independent Word Learning Strategies of Middle School Students. , 2000 .

[46]  Sherrie L. Nist,et al.  Teaching comprehension and study strategies through modeling and thinking aloud , 1986 .

[47]  R. Ellis.  The Definition and Measurement of L2 Explicit Knowledge , 2004 .

[48]  B. Brehmer.  Hypotheses about relations between scaled variables in the learning of probabilistic inference tasks , 1974 .

[49]  M A Just,et al.  From the SelectedWorks of Marcel Adam Just 1990 What one intelligence test measures : A theoretical account of the processing in the Raven Progressive Matrices Test , 2016 .

[50]  William F. Wright,et al.  Expertise and the explanation effect , 1988 .

[51]  Wenyu Wang,et al.  L1 Use in the L2 Composing Process: An Exploratory Study of 16 Chinese EFL Writers. , 2002 .

[52]  C. Yewchuk,et al.  Use of Metacognitive Reading Strategies by Gifted Learning Disabled Students: An Exploratory Study , 1996 .

[53]  Alison Mackey,et al.  Interaction research in SLA : A meta-analysis and research synthesis , 2007 .

[54]  Jerome B. Zutell,et al.  Thinking Aloud during Spelling Word Sorts. , 1998 .

[55]  Kevin M. Brooks,et al.  Thoughts beyond words : When language overshadows insight , 1993 .

[56]  K. Morgan‐Short,et al.  TO THINK ALOUD OR NOT TO THINK ALOUD: The Issue of Reactivity in SLA Research Methodology , 2004, Studies in Second Language Acquisition.

[57]  Gabriele Kasper,et al.  Data Collection in Pragmatics Research , 2000, Culturally Speaking Second Edition.

[58]  R. Hoosain,et al.  Talking While Performing a Task: A Better Attentional Performance in Patients With Closed Head Injury? , 2002, Journal of clinical and experimental neuropsychology.

[59]  Comprehension monitoring strategies of nonproficient college readers , 1991 .

[60]  R. Leow.  The Effects of Amount and Type of Exposure on Adult Learners' L2 Development in SLA , 1998 .

[61]  Unkyoung Maeng.  A Comparative Study of Reading Strategies in L1 and L2 , 2005 .

[62]  A. Hafner.  Influence of Verbalization on Problem Solving , 1957 .

[63]  K. Anders Ericsson,et al.  Towards a procedure for eliciting verbal expression of non-verbal experience without reactivity: interpreting the verbal overshadowing effect within the theoretical framework for protocol analysis , 2002 .

[64]  Peter Robinson,et al.  Consciousness, rules and instructed second language acquisition , 1996 .

[65]  Sophie. Beare.  Differences in content generating and planning processes of adult L1 and L2 proficient writers. , 2000 .

[66]  Numa Markee.  Conversation Analysis for Second Language Acquisition , 2005 .

[67]  Kozue Uzawa,et al.  Second language learners' processes of L1 writing, L2 writing, and translation from L1 into L2 , 1996 .

[68]  Lourdes Ortega,et al.  Synthesizing research on language learning and teaching , 2006 .

[69]  Michal Zellermayer,et al.  Varying paths for learning to revise , 1996 .

[70]  Melissa A. Bowles,et al.  REACTIVITY AND TYPE OF VERBAL REPORT IN SLA RESEARCH METHODOLOGY: Expanding the Scope of Investigation , 2005, Studies in Second Language Acquisition.

[71]  Candace Séguinot,et al.  Some Thoughts About Think-Aloud Protocols , 1996 .

[72]  Peter Smagorinsky,et al.  Rethinking protocol analysis from a cultural perspective , 2001, Annual Review of Applied Linguistics.

[73]  Doug Cushman.  From Scribbles to Stories. , 2002 .

[74]  Mami Yoshida,et al.  Think-Aloud Protocols and Type ofReading Task: The Issue of Reactivity in L2 Reading Research , 2008 .

[75]  P. Ewert,et al.  Part II: The Effect of Verbal Instructions upon the Formation of a Concept , 1932 .

[76]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[77]  Lara Lomicka,et al.  TO GLOSS OR NOT TO GLOSS: AN INVESTIGATION OF READING COMPREHENSION ONLINE , 1998 .

[78]  Robert Pritchard,et al.  The Effects of Cultural Schemata on Reading Processing Strategies. , 1990 .

[79]  J A Brinkman,et al.  Verbal protocol accuracy in fault diagnosis. , 1993, Ergonomics.

[80]  F. Scott Walters,et al.  A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence , 2007 .

[81]  F. Hashim,et al.  Use of L1 in L2 Reading Comprehension among Tertiary ESL Learners. , 2006 .

[82]  Anna Herwig.  Chapter 7. Plurilingual Lexical Organisation: Evidence from Lexical Processing in L1–L2–L3–L4 Translation , 2001 .

[83]  Katherine M. Robinson,et al.  The validity of verbal reports in children's subtraction , 2001 .

[84]  Thomas A. Upton,et al.  THE ROLE OF THE FIRST LANGUAGE IN SECOND LANGUAGE READING , 2001, Studies in Second Language Acquisition.

[85]  Neil J. Anderson.  Individual differences in strategy use in second language reading and testing. , 1991 .

[86]  Steven W. Evans,et al.  The effect of thinking aloud on the problem-solving performance of bright, average, learning disabled, and developmentally handicapped students , 1991 .

[87]  J. César Félix-Brasdefer,et al.  Interlanguage Refusals: Linguistic Politeness and Length of Residence in the Target Community. , 2004 .

[88]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[89]  R. Schmidt.  The role of consciousness in second language learning , 1990 .

[90]  Li Jianda.  Measuring interlanguage pragmatic knowledge of Chinese EFL learners , 2004 .

[91]  K. Mitchell,et al.  Verbal Protocol Patterns of College Dormitory Counselors. , 1970 .

[92]  Thomas C. Taveggia,et al.  The teaching-learning paradox : a comparative analysis of college teaching methods , 1968 .

[93]  Peter McGeorge,et al.  The effects of concurrent verbalization on performance in a dynamic systems task , 1989 .

[94]  R. S. Tomlin,et al.  Attention in Cognitive Science and Second Language Acquisition , 1994, Studies in Second Language Acquisition.

[95]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[96]  Mary E. Ahlum-Heath,et al.  The effect of conscious controlled verbalization cognitive strategy on transfer in problem solving , 1986, Memory & cognition.

[97]  R. Durst,et al.  Cognitive and Linguistic Demands of Analytic Writing. , 1987 .

[98]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[99]  Donald S. Qi,et al.  Exploring the Role of Noticing in a Three-Stage Second Language Writing Task. , 2001 .

[100]  G. Hu.  PSYCHOLOGICAL CONSTRAINTS ON THE UTILITY OF METALINGUISTIC KNOWLEDGE IN SECOND LANGUAGE PRODUCTION , 2002, Studies in Second Language Acquisition.

[101]  L. Midanik,et al.  'Unstandard' ways of answering standard questions: protocol analysis in alcohol survey research. , 1991, Drug and alcohol dependence.

[102]  Dianne C. Berry,et al.  Metacognitive Experience and Transfer of Logical Reasoning , 1983 .

[103]  Nina Spada,et al.  4. The effectiveness of corrective feedback for the acquisition of L2 grammar: A meta-analysis of the research , 2006 .

[104]  Junko Yamashita,et al.  Reading Strategies in L1 and L2: Comparison of Four Groups of Readers with Different Reading Ability in L1 and L2 , 2002 .

[105]  Matthew E. Poehner,et al.  Beyond the test: L2 dynamic assessment and the transcendence of mediated learning , 2007 .

[106]  Problem solving as a function of the situation. , 1951, Journal of experimental psychology.

[107]  J. Wilhelm.  Getting Kids into the Reading Game: You Gotta Know the Rules. , 2001 .

[108]  Susanne Rott,et al.  Processing Glosses: A Qualitative Exploration of How Form-Meaning Connections Are Established and Strengthened. , 2005 .

[109]  A. M. Williams,et al.  Assessing cue usage in performance contexts: A comparison between eye-movement and concurrent verbal report methods , 1997 .

[110]  James F. Baumann,et al.  Using Think Alouds to Enhance Children's Comprehension Monitoring Abilities. , 1993 .

[111]  S. Norris.  Effect of Eliciting Verbal Reports of Thinking on Critical Thinking Test Performance. , 1990 .

[112]  A. Cohen.  Mentalistic Measures in Reading Strategy Research: Some Recent Findings. , 1986 .

[113]  G. Kasper,et al.  Pragmatic Development in a Second Language , 2003 .

[114]  Steve Parkes,et al.  Trends in the use of verbal protocol analysis in software engineering research , 2003, Behav. Inf. Technol..

[115]  R. Ellis.  Introduction: Investigating Form‐Focused Instruction , 2001 .

[116]  R. Leow.  Toward operationalizing the process of attention in SLA: Evidence for Tomlin and Villa's (1994) finegrained analysis of attention , 1998, Applied Psycholinguistics.

[117]  P. Friedman,et al.  Relationship of clinician feedback to child initiated verbalization during language training. , 1976, Journal of communication disorders.

[118]  D. Broadbent,et al.  On the Relationship between Task Performance and Associated Verbalizable Knowledge , 1984 .

[119]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[120]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[121]  C. D. Castro.  The role of Tagalog in ESL writing : Clues from students' think-aloud protocols , 2004 .

[122]  J. Norris,et al.  Effectiveness of L2 Instruction: A Research Synthesis and Quantitative Meta‐analysis , 2000 .

[123]  Andrew D. Cohen,et al.  SOME USES OF MENTALISTIC DATA IN SECOND LANGUAGE RESEARCH1 , 1981 .

[124]  Carol Hosenfeld,et al.  A preliminary investigation of the reading strategies of successful and nonsuccessful second language learners , 1977 .

[125]  Carolyn R. Fehrenbach.  Gifted/Average Readers , 1991 .

[126]  James N. Davis,et al.  How do L1 and L2 reading differ? Evidence from Think Aloud protocols , 1993 .

[127]  D. Alvermann.  Second Graders’ Strategic Preferences While Reading Basal Stories , 1984 .

[128]  P. Pollard,et al.  On the conflict between logic and belief in syllogistic reasoning , 1983, Memory & cognition.

[129]  Ronald P. Leow.  The Effects of Input Enhancement and Text Length on Adult L2 Readers' Comprehension and Intake in Second Language Acquisition. , 1997 .

[130]  The effect of number of irrelevant stimulus dimensions, verbalization, and sex on learning biconditional classification rules , 1967 .

[131]  Peter Storey,et al.  Examining the test-taking process: a cognitive perspective on the discourse cloze test , 1997 .

[132]  THE ROLE OF LEXICAL TEMPORAL INDICATORS AND TEXT INTERACTION FORMAT IN THE INCIDENTAL ACQUISITION OF THE SPANISH FUTURE TENSE , 2007, Studies in Second Language Acquisition.

[133]  Jukka Hyönä,et al.  Do adult readers know how they read? Evidence from eye movement patterns and verbal reports. , 2006, British journal of psychology.

[134]  Gabriel J. Biehal,et al.  The Effects of Concurrent Verbalization on Choice Processing , 1989 .

[135]  Jeffrey J. Walczyk,et al.  Overcoming inefficient reading skills , 2001 .

[136]  Merrill Swain,et al.  VERBAL PROTOCOLS: WHAT DOES IT MEAN FOR RESEARCH TO USE SPEAKING AS A DATA COLLECTION TOOL? , 2006 .

[137]  Nicole Tracy-Ventura,et al.  Investigating the empirical link between task-based interaction and acquisition: a quantitative meta-analysis , 2006 .

[138]  Berndt Brehmer,et al.  Does having to justify one's judgments change the nature of the judgment process? , 1983 .

[139]  T. Carr,et al.  Cognitive Factors in Learning about Structured Sequences , 1994, Studies in Second Language Acquisition.

[140]  Peter Afflerbach,et al.  Verbal Protocols of Reading: The Nature of Constructively Responsive Reading , 1996 .

[141]  Naoko Taguchi.  Pragmatic Comprehension in Japanese as a Foreign Language , 2008 .

[142]  Charlene Polio,et al.  LEARNERS' USES OF TWO TYPES OF WRITTEN FEEDBACK ON A L2 WRITING REVISION TASK , 2007, Studies in Second Language Acquisition.

[143]  Andrew D. Cohen,et al.  `I want to go back to the text': Response strategies on the reading subtest of the new TOEFL® , 2007 .

[144]  J. C. Félix-Brasdefer.  Perceptions of Refusals to Invitations: Exploring the Minds of Foreign Language Learners , 2008 .

[145]  Richard G. Kern.  The Role of Mental Translation in Second Language Reading , 1994, Studies in Second Language Acquisition.

[146]  Ronald P. Leow,et al.  Computerized Task-Based Exposure, Explicitness, Type of Feedback, and Spanish L2 Development , 2004 .

[147]  Melissa A. Bowles.  TASK TYPE AND REACTIVITY OF VERBAL REPORTS IN SLA: A First Look at a L2 Task Other Than Reading , 2008, Studies in Second Language Acquisition.

[148]  Toshiyo Nabei,et al.  Learner Awareness of Recasts in Classroom Interaction: A Case Study of an Adult EFL Student's Second Language Learning , 2002 .

[149]  Ellen L. Block.  The Comprehension Strategies of Second Language Readers , 1986 .

[150]  Patricia L. Carrell,et al.  METACOGNITIVE AWARENESS AND SECOND LANGUAGE READING , 1989 .

[151]  B. Brehmer,et al.  Effects of function form and cue validity on the subjects' hypotheses in probabilistic inference tasks , 1974 .

[152]  J. Hayes,et al.  Fluency in Writing , 2001 .

[153]  Hossein Nassaji.  The Relationship between Depth of Vocabulary Knowledge and L2 Learners' Lexical Inferencing Strategy Use and Success , 2004 .

[154]  Matthew J. Anderson,et al.  Some Evidence On The Effect Of Verbalization On Process - A Methodological Note , 1985 .

[155]  L. Karsenty.  Adapting verbal protocol methods to investigate speech systems use. , 2001, Applied ergonomics.

[156]  J. Greenwood,et al.  Some surprising similarities in the clinical reasoning of 'expert' and 'novice' orthopaedic nurses: report of a study using verbal protocols and protocol analyses. , 2006, Journal of advanced nursing.

[157]  R. Mathews,et al.  Insight without Awareness: On the Interaction of Verbalization, Instruction and Practice in a Simulated Process Control Task , 1989 .

[158]  Ronald P. Leow,et al.  Do Learners Notice Enhanced Forms While Interacting with the L2?: An Online and Offline Study of the Role of Written Input Enhancement in L2 Reading , 2001 .

[159]  George Katona,et al.  Organizing and memorizing , 1940 .

[160]  P. B. El-Dinary,et al.  Children's Learning Strategies in Language Immersion Classrooms , 1999 .

[161]  W. Poulisse,et al.  The Use of Compensatory Strategies by Dutch Learners of English , 1990 .

[162]  R. Leow.  Attention, Awareness, and Foreign Language Behavior , 1997 .

[163]  R. Schmidt.  Awareness and Second Language Acquisition , 1992, Annual Review of Applied Linguistics.

[164]  Suzanne Graham.  The Think-Aloud Controversy in Second Language Research , 2011 .

[165]  Elise Ann Earthman.  Creating the Virtual Work: Readers' Processes in Understanding Literary Texts. , 1992 .

[166]  Gerhard P. Deffner,et al.  Evaluation of Concurrent Thinking Aloud Using Eye-tracking Data , 1990 .

[167]  Carol Hosenfeld.  Learning about Learning: Discovering Our Students' Strategies* , 1976 .

[168]  Alison Mackey,et al.  HOW DO LEARNERS PERCEIVE INTERACTIONAL FEEDBACK? , 2000, Studies in Second Language Acquisition.

[169]  Leif Stinessen,et al.  The influence of verbalization on problem‐solving , 1985 .

[170]  M. Nissen,et al.  Attentional requirements of learning: Evidence from performance measures , 1987, Cognitive Psychology.

[171]  Harriet Wood Bowden,et al.  Concurrent Verbalizations, Pedagogical Conditions, and Reactivity: Two CALL Studies , 2009 .

[172]  E. Ronowicz,et al.  Translator’s Frequent Lexis Store and Dictionary Use as Factors in SLT Comprehension and Translation Speed – A Comparative Study of Professional, Paraprofessional and Novice Translators , 2005 .