Instructional Interventions Affecting Critical Thinking Skills and Dispositions: A Stage 1 Meta-Analysis

Critical thinking (CT), or the ability to engage in purposeful, self-regulatory judgment, is widely recognized as an important, even essential, skill. This article describes an ongoing meta-analysis that summarizes the available empirical evidence on the impact of instruction on the development and enhancement of critical thinking skills and dispositions. We found 117 studies based on 20,698 participants, which yielded 161 effects with an average effect size (g+) of 0.341 and a standard deviation of 0.610. The distribution was highly heterogeneous (Q T = 1,767.86, p < .001). There was, however, little variation due to research design, so we neither separated studies according to their methodological quality nor used any statistical adjustment for the corresponding effect sizes. Type of CT intervention and pedagogical grounding were substantially related to fluctuations in CT effects sizes, together accounting for 32% of the variance. These findings make it clear that improvement in students’ CT skills and dispositions cannot be a matter of implicit expectation. As important as the development of CT skills is considered to be, educators must take steps to make CT objectives explicit in courses and also to include them in both preservice and in-service training and faculty development.

[1]  An Historical vs. Contemporary Problem Solving use of the College Physical Science Laboratory Period for General Education , 1953 .

[2]  J. C. Bledsoe.  A comparative study of values and critical thinking skills of a group of educational workers. , 1955 .

[3]  AN EXPERIMENTAL STUDY OF THE DEVELOPMENT OF CRITICAL THINKING SKILLS OF HIGH SCHOOL ENGLISH TEACHERS ENROLLED IN A METHODS COURSE. INTERIM REPORT. , 1968 .

[4]  Raymond Arlo.  The relative effectiveness of inductive and expository teaching of principles of general semantics upon the critical reading ability of 9th-grade students , 1969 .

[5]  Alvin Bean.  Some Comparisons between Conventional College Teaching Methods and a Composite of Procedures Involving Large Lecture Groups, Seminars, and Reduced Class Time , 1969 .

[6]  J. Hook.  Illinois State-Wide Curriculum Study Center in the Preparation of Secondary School English Teachers. Final Report. , 1969 .

[7]  Development of a More Flexible Physical Science Laboratory Program for Non-Science Majors with Superior High School Science Backgrounds. Final Report. , 1969 .

[8]  Conditional logic and children (Cornell Critical Thinking Readiness Project, Phase IIC) , 1969 .

[9]  E. Glaser.  An Experiment in Development of Critical Thinking , 1942 .

[10]  Structured Oral Inquiry Improves Thinking , 1972 .

[11]  A Focus on Problems of National Interest in the College General Chemistry Laboratory: The Effects of the Problem-Oriented Method Compared with Those of the Traditional Approach , 1972 .

[12]  I. Scheffler.  Reason and teaching , 1973 .

[13]  The Effects of Instruction on the Critical Thinking Abilities of Middle-School-Age Children. , 1975 .

[14]  Design and Implementation of an Evaluation Procedure for an Innovative Undergraduate Program. , 1977 .

[15]  D. Annis,et al.  The impact of philosophy on students' critical thinking ability , 1979 .

[16]  The Role of Programmed Instruction for Sequential Skill Development in Higher Education. , 1979 .

[17]  Developing Critical Thinking Skills in Nonacademic Social Studies Classes. , 1980 .

[18]  Larry E. Wood,et al.  An “intelligent” program to teach logical thinking skills , 1980 .

[19]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[20]  D. Annis.  Critical Thinking and Education , 1982 .

[21]  An Analysis of the Research Concerning Debate Training and Critical Thinking Improvements. , 1983 .

[22]  Critical thinking and content acquisition using a modified guided design process for large course sections. , 1983 .

[23]  P. Klassen.  Changes in personal orientation and critical thinking among adults returning to school through weekend college: An alternative evaluation , 1983 .

[24]  Gloria M. Boodt.  Critical listeners become critical readers in remedial reading class , 1984 .

[25]  Edward L. Vockell,et al.  Computerized Science Simulations Stimulus to Generalized Problem Solving Capabilities. , 1984 .

[26]  E. Neimark.  A Cognitive Style Change Approach to the Modification of Thinking in College Students. , 1984 .

[27]  Cognitive and Attitudinal Effects of Reasoning Message Strategies. , 1984 .

[28]  Ingram Olkin,et al.  Estimation of a Single Effect Size: Parametric and Nonparametric Methods , 1985 .

[29]  The Effects of Teaching Logical Reasoning Upon Students' Critical Thinking and Science Achievement. , 1985 .

[30]  S. Norris.  SYNTHESIS OF RESEARCH ON CRITICAL THINKING , 1985 .

[31]  S. Norton.  The Effects of an Independent Laboratory Investigation on the Critical Thinking Ability and Scientific Attitude of Students in a General Microbiology Class. , 1985 .

[32]  Gender Differences in the Instruction and Intelligent Behavior of Fifth-Graders. Summary Paper. , 1986 .

[33]  Academic Challenge Program: Evaluation Report. , 1986 .

[34]  J. H. McMillan,et al.  Enhancing college students' critical thinking: A review of studies , 1987 .

[35]  Kent R. Colbert.  The Effects of CEDA and NDT Debate Training on Critical Thinking Ability. , 1987 .

[36]  Robert H. Ennis.  A taxonomy of critical thinking dispositions and abilitities , 1987 .

[37]  Walter Pierce.  Effects of a Secondary Curriculum for Moral Development and Critical Thinking. , 1988 .

[38]  Bryce B. Hudgins,et al.  Chlldren’s Self-Directed Critical Thinking , 1988 .

[39]  Stimulating Critical Thinking through Faculty Development: Design, Evaluation, and Problems. , 1988 .

[40]  H. Siegel.  Educating Reason: Rationality, Critical Thinking and Education , 1988 .

[41]  What Junior High School Students Do, Can, and Should Know about the Nature of Science and Scientific Inquiry. Technical Report 88-2. , 1988 .

[42]  J. Kurfiss.  Critical Thinking: Theory, Research, Practice, and Possibilities. ASHE-ERIC Higher Education Report No. 2, 1988. , 1988 .

[43]  Robert H. Ennis.  Critical Thinking and Subject Specificity: Clarification and Needed Research , 1989 .

[44]  Larry V. Hedges,et al.  A Practical Guide to Modern Methods of Meta-Analysis. , 1989 .

[45]  Doris B. Matthews,et al.  The Effect of a Thinking-Skills Program on the Cognitive Abilities of Middle School Students. , 1989 .

[46]  Evaluations of the "Tactics for Thinking" Program: Summary Report. , 1989 .

[47]  Paul J. Germann,et al.  Directed‐inquiry approach to learning science process skills: treatment effects and aptitude‐treatment interactions , 1989 .

[48]  Peter A. Facione,et al.  The California Critical Thinking Skills Test--College Level. Technical Report #4. Interpreting the CCTST, Group Norms, and Sub-Scores. , 1990 .

[49]  Peter A. Facione,et al.  The California Critical Thinking Skills Test--College Level. Technical Report #2. Factors Predictive of CT Skills. , 1990 .

[50]  Peter A. Facione,et al.  The California Critical Thinking Skills Test--College Level. Technical Report #1. Experimental Validation and Content Validity. , 1990 .

[51]  An Investigation of Paideia Program Effects on Students' Critical Thinking Skills. , 1990 .

[52]  J. Kurfiss.  Critical thinking : theory, research, practice, and possibilities , 1990 .

[53]  Robert L. Bangert-Drowns,et al.  Meta-Analysis of Effects of Explicit Instruction for Critical Thinking. , 1990 .

[54]  P. Facione.  Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Research Findings and Recommendations. , 1990 .

[55]  C. Green,et al.  Teaching Critical Thinking and Writing through Debates: An Experimental Evaluation. , 1990 .

[56]  Bryce B. Hudgins,et al.  Retention and Transfer of Children's Self-Directed Critical Thinking Skills , 1991 .

[57]  An Analysis of the Relationship between Student-Invented Hypotheses and the Development of Reflective Thinking Strategies. , 1991 .

[58]  Karen C. Stoiber,et al.  The Effect of Technical and Reflective Preservice Instruction on Pedagogical Reasoning and Problem Solving , 1991 .

[59]  Norman J. Unrau,et al.  The Effects of Explicit Instruction on Critical Reading and Argumentative Writing: The TASK of Reading and Writing. , 1991 .

[60]  S. Frost.  Fostering the Critical Thinking of College Women through Academic Advising and Faculty Contact. , 1991 .

[61]  R. Paul,et al.  Critical Thinking: What Every Person Needs To Survive in a Changing World , 1991 .

[62]  Mark Sadoski,et al.  The Effects of Instruction in Forming Generalizations on High School Students' Critical Thinking in World History. , 1991 .

[63]  Assessment of General Education Knowledge and Competencies in PSY 101, Introductory Psychology. , 1991 .

[64]  Karol K. Walters,et al.  Two Studies of the Effects of Annotation Training on College Students. , 1991 .

[65]  Teresa Garcia,et al.  Critical Thinking and Its Relationship to Motivation, Learning Strategies, and Classroom Experience. , 1992 .

[66]  James L. Poirot,et al.  The Effect of Computer Science Instruction on Critical Thinking Skills and Mental Alertness , 1992 .

[67]  J. V. Faryniarz,et al.  Effectiveness of Microcomputer Simulations in Stimulating Environmental Problem Solving by Community College Students. , 1992 .

[68]  M. Pritchard.  Thinking in Education , 1993 .

[69]  Robyn Quin,et al.  Monitoring Standards in Media Studies: Problems and Strategies , 1993 .

[70]  Pinchas Tamir,et al.  Incorporating Critical Thinking Into a Regular High School Biology Curriculum , 1993 .

[71]  Assessment of Critical Thinking Courses , 1993 .

[72]  Robert H. Gass,et al.  Does Teaching Argumentation Facilitate Critical Thinking , 1994 .

[73]  P. B. Horton,et al.  The Effects of an Interdisciplinary Curriculum Unit on the Environmental Decision-Making of Secondary School Students. , 1994 .

[74]  Pinchas Tamir,et al.  The Effect of the Biology Critical Thinking Project on the Development of Critical Thinking. , 1994 .

[75]  J. Royalty,et al.  The Generalizability of Critical Thinking: Paranormal Beliefs Versus Statistical Reasoning , 1995 .

[76]  Christine L. Allegretti,et al.  A Model for Thinking Critically about Ethical Issues , 1995 .

[77]  D. A. Bensley,et al.  The Acquisition of General Purpose Strategic Knowledge for Argumentation , 1995 .

[78]  Ron E. McBride,et al.  Teacher and at-risk students' cognitions during open-ended activities: Structuring the learning environment for critical thinking☆ , 1995 .

[79]  Bernadette M. Gadzella,et al.  Teaching and Learning Critical Thinking Skills. , 1996 .

[80]  Measuring the Impact of Forensics and Communication Education on Critical Thinking: A Meta-Analytic Summary. , 1997 .

[81]  Keith E. Stanovich,et al.  Structuring Reflection: Teaching Argument Concepts and Strategies Enhances Critical Thinking , 1997 .

[82]  Ann Thompson,et al.  Guided Instruction in Logo Programming and the Development of Cognitive Monitoring Strategies among College Students , 1997 .

[83]  J. Ayoub,et al.  The effect of classroom use of an electronic group support system on student critical thinking, performance and satisfaction , 1997 .

[84]  Widening the Lens of Diversity: Motivating Reflective Journal Writing. , 1998 .

[85]  Paul M. Lewanski.  The effect of computer assisted library research on critical thinking skills acquisition of ninth grade students , 1998 .

[86]  Teaching Logic to Adolescents to Improve Thinking Skills , 1998 .

[87]  Evaluating Supplemental Program Implementation through Measurements of Students' Critical Thinking Skills and Writing Ability. , 1998 .

[88]  Lisa Tsui.  A Review of Research on Critical Thinking , 1998 .

[89]  Theresa M. Arburn.  Assisting at-risk community college students' acquisition of critical thinking learning strategies in human anatomy and physiology , 1999 .

[90]  Beyond the Sponge Model: Encouraging Students' Questioning Skills in Abnormal Psychology , 1998 .

[91]  John Dwayne English.  Improvement in drafting instruction: Using graphic organizers and paired problem solving to enhance critical thinking and content knowledge , 1998 .

[92]  R. Wesp,et al.  Developing Critical Thinking through the Study of Paranormal Phenomena , 1998 .

[93]  L. Rebeschi,et al.  Critical thinking skills of baccalaureate nursing students at program entry and exit. , 1999, Nursing and health care perspectives.

[94]  Barbara L. Adams,et al.  Nursing education for critical thinking: an integrative review. , 1999, The Journal of nursing education.

[95]  Min Liu,et al.  The effect of being hypermedia designers on elementary school students' motivation and learning of design knowledge , 1999 .

[96]  F. Soukup.  Assessment of Critical Thinking Skills in Associate Degree Nursing Students. , 1999 .

[97]  M. Allen,et al.  A Meta-Analysis of the Impact of Forensics and Communication Education on Critical Thinking. , 1999 .

[98]  Skill-Building Exercises and Generalizing Psychological Concepts to Daily Life , 1999 .

[99]  K. Stanovich,et al.  The domain specificity and generality of belief bias: Searching for a generalizable critical thinking skill. , 1999 .

[100]  B. Daley,et al.  Concept maps: a strategy to teach and evaluate critical thinking. , 1999, The Journal of nursing education.

[101]  Marie Thielke Huff,et al.  A Comparison Study of Live Instruction versus Interactive Television for Teaching MSW Students Critical Thinking Skills , 2000 .

[102]  Lola Hill,et al.  What does it Take to Change Minds? Intellectual Development of Preservice Teachers , 2000 .

[103]  J. Itano,et al.  The impact of the use of inquiry-based learning as a teaching methodology on the development of critical thinking. , 2000, The Journal of nursing education.

[104]  Cheryl Hilgenberg,et al.  Student perceptions of satisfaction and opportunities for critical thinking in distance education by interactive video , 2000 .

[105]  M. Morley,et al.  The application of the structure of intellect programme , 2000 .

[106]  R. Soden,et al.  Critical thinking in education: a review , 2000 .

[107]  Epistemological Beliefs of Community-College Students. , 2000 .

[108]  J. Halliday.  Critical thinking and the academic vocational divide , 2000 .

[109]  K. Stevens,et al.  Critical Thinking Outcomes of Computer-Assisted Instruction versus Written Nursing Process , 2000 .

[110]  M. Duffey,et al.  An evidence-based project for evaluating strategies to improve knowledge acquisition and critical-thinking performance in nursing students. , 2000, The Journal of nursing education.

[111]  Abigail Housen,et al.  Methods for Assessing Transfer from an Art-Viewing Program. , 2001 .

[112]  Curtis J. Bonk,et al.  Holy COW: Scaffolding Case Based Conferencing on the Web with Preservice Teachers. , 2001 .

[113]  Anna Craft,et al.  The Impact of Cognitive Strategy Instruction on Deaf Learners: An International Comparative Study , 2001, American annals of the deaf.

[114]  Timothy J. Ellis,et al.  Multimedia enhanced educational products as a tool to promote critical thinking in adult students , 2001 .

[115]  Don R. Gelven,et al.  Developing Critical Thinking Skills of Tech Prep Students Using Applied Communications. , 2001 .

[116]  Karla J. Oty,et al.  The effect of an interdisciplinary algebra/science course on students' problem solving skills, critical thinking skills and attitudes towards mathematics , 2001 .

[117]  Mark Hawkes,et al.  Variables of Interest in Exploring the Reflective Outcomes of Network-Based Communication , 2001 .

[118]  Jeffrey D. Kromrey,et al.  Teaching Critical Thinking in a Community College History Course: Empirical Evidence from Infusing Paul's Model , 2001 .

[119]  Weimin Wang.  The Relative Effectiveness of Structured Questions and Summarizing on Near and Far Transfer Tasks. , 2001 .

[120]  Improving Critical Thinking in an Introductory Psychology Course. , 2001 .

[121]  D. Garrison,et al.  Critical thinking, cognitive presence, and computer conferencing in distance education , 2001 .

[122]  C. Howe,et al.  Peer interaction and the learning of critical thinking skills in further education students , 2001 .

[123]  Ruth Drake,et al.  Teaching for Transfer. , 2001 .

[124]  Lisa Tsui.  Fostering Critical Thinking Through Effective Pedagogy: Evidence from Four Institutional Case Studies , 2002 .

[125]  Gerald F. Smith.  Are There Domain–Specific Thinking Skills? , 2002 .

[126]  D. Bartlett,et al.  Measuring change in students' critical thinking ability: implications for health care education. , 2002, Journal of allied health.

[127]  Linda D. Avery,et al.  A Curriculum Study of Gifted-Student Learning in the Language Arts , 2002 .

[128]  Camille J. Bunting,et al.  The Influence of Challenge Course Participation on Moral and Ethical Reasoning , 2002 .

[129]  Michael G. Parkinson,et al.  The Socratic method in the introductory PR course: An alternative pedagogy , 2002 .

[130]  Patricia S. O'Sullivan,et al.  Does Project L.I.V.E. Case Modality Impact Critical Thinking in PBL Groups , 2002 .

[131]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[132]  M. J. W. Thomas,et al.  Learning within incoherent structures: the space of online discussion forums , 2002, J. Comput. Assist. Learn..

[133]  S. Hannabuss.  The Blackwell Guide to the Philosophy of Education , 2003 .

[134]  Nicholas Athanassiou,et al.  Critical Thinking in the Management Classroom: Bloom's Taxonomy as a Learning Tool , 2003 .

[135]  Paul Black,et al.  A Systematic Review of the Impact on Students and Teachers of the Use of ICT for Assessment of Creative and Critical Thinking Skills , 2003 .

[136]  Robert L. Williams.  Critical Thinking as a Predictor and Outcome Measure in a Large Undergraduate Educational Psychology Course. , 2003 .

[137]  The Failure of Critical Thinking: Considering Virtue Epistemology as a Pedagogical Adventure , 2003 .

[138]  The Delphi Technique: A Research Strategy for Career and Technical Education , 2004 .

[139]  Robert M. Bernard,et al.  How Does Distance Education Compare With Classroom Instruction? A Meta-Analysis of the Empirical Literature , 2004 .

[140]  E. Hyslop-Margison,et al.  CRITICAL THINKING IN CAREER EDUCATION: THE DEMOCRATIC IMPORTANCE OF FOUNDATIONAL RATIONALITY , 2004 .

[141]  Patrick J. Fahy.  Two methods for assessing critical thinking in computer-mediated communications (CMC) transcripts. , 2005 .

[142]  Elizabeth Murphy,et al.  Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study , 2006, J. Educ. Technol. Soc..

[143]  Tania B. Huedo-Medina,et al.  Assessing heterogeneity in meta-analysis: Q statistic or I2 index? , 2006, Psychological methods.

[144]  W. O'Neil,et al.  THE NATURE OF SCIENCE AND SCIENTIFIC INQUIRY , 2007 .

[145]  Robert M. Bernard,et al.  Exploring the structure of the Watson–Glaser Critical Thinking Appraisal: One scale or many subscales? , 2008 .

[146]  E. Glaser.  Watson-Glaser Critical Thinking Appraisal ® , 2008 .

[147]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[148]  Mira Feuerstein,et al.  Media Literacy in Support of Critical Thinking , 2010 .