Numbers and Narrative: Combining Their Strengths in Research Reviews

When several independent research studies examine the same program or treatment, conflicting findings often result, making it difficult to draw overall conclusions. Recent methodological work has created procedures, sometimes called meta-analysis, for combining quantitative results across studies. In this article, Richard Light and David Pillemer argue that qualitative information is equally important for explaining conflicting or puzzling outcomes. They discuss six ways in which qualitative information is essential to the process of literature review. The authors outline three broad strategies for combining different types of information in a review: quantifying descriptive reports, presenting quantitative outcomes narratively, and allying statistical and descriptive evidence while maintaining the integrity of each. They suggest that reviews organized to ally both forms of information will ultimately maximize our knowledge about the complexities of program success.

[1]  Benjamin Kleinmuntz,et al.  Statistical versus clinical lie detection. , 1981 .

[2]  Laura C. Leviton,et al.  Reviewing the literature: A comparison of traditional methods with meta-analysis. , 1980 .

[3]  Gregg B. Jackson,et al.  Methods for Integrative Reviews , 1980 .

[4]  L. Hedges,et al.  Vote-counting methods in research synthesis. , 1980 .

[5]  R. Rosenthal,et al.  Statistical versus traditional procedures for summarizing research findings. , 1980, Psychological bulletin.

[6]  M. Miles.  Qualitative Data as an Attractive Nuisance: The Problem of Analysis , 1979 .

[7]  Charles McClintock,et al.  Applying the logic of sample surveys to qualitative case studies: The case cluster method , 1979 .

[8]  Michael R. Gottfredson,et al.  Beyond Probation: Juvenile Corrections and the Chronic Delinquent , 1979 .

[9]  Mary M. Kennedy,et al.  Generalizing From Single Case Studies , 1979 .

[10]  L. Sechrest,et al.  The Rehabilitation of Criminal Offenders: Problems and Prospects , 1979 .

[11]  Chen-Lin C. Kulik,et al.  A Meta-Analysis of Outcome Studies of Keller's Personalized System of Instruction. , 1979 .

[12]  J. Shaver.  The Productivity of Educational Research and the Applied-Basic Research Distinction1 , 1979 .

[13]  J. Durlak.  Comparative effectiveness of paraprofessional and professional helpers. , 1979, Psychological bulletin.

[14]  L. Steinberg,et al.  The effects of day care: A critical review. , 1978 .

[15]  Donald B. Rubin,et al.  Interpersonal expectancy effects: the first 345 studies , 1978, Behavioral and Brain Sciences.

[16]  Mary Lee Smith,et al.  Meta-Analysis of Research on the Relationship of Class-Size and Achievement. The Class Size and Instruction Project. , 1978 .

[17]  Edward Zigler,et al.  IQ, Social Competence, and Evaluation of Early Childhood Intervention Programs. , 1978 .

[18]  R. R. Jones,et al.  Effects of serial dependency on the agreement between visual and statistical inference. , 1978, Journal of applied behavior analysis.

[19]  Susan Presby,et al.  Overly broad categories obscure important differences between therapies. , 1978 .

[20]  H. Eysenck.  An exercise in mega-silliness. , 1978 .

[21]  R. Stake.  The Case Study Method in Social Inquiry1 , 1978 .

[22]  R. Rosenthal.  Combining results of independent studies. , 1978 .

[23]  L. Kamin.  Comment on Munsinger's review of adoption studies. , 1978 .

[24]  H. Munsinger.  Reply to Kamin. , 1978 .

[25]  John M. Gottman,et al.  4 – Analysis of Interrupted Time-Series Experiments , 1978 .

[26]  Judith A. Hall.  Gender Effects in Decoding Nonverbal Cues , 1978 .

[27]  T. Kratochwill.  N=1: An alternative research strategy for school psychologists☆ , 1977 .

[28]  M. L. Smith,et al.  Meta-analysis of psychotherapy outcome studies. , 1977, The American psychologist.

[29]  Dale Carlson,et al.  Deep Dark Deficits of the Adversary Evaluation Model1 , 1977 .

[30]  G. Glass.  9: Integrating Findings: The Meta-Analysis of Research , 1977 .

[31]  Robert K. Yin,et al.  The Difference That Quality Makes , 1976 .

[32]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[33]  F Mosteller,et al.  Swine flu: quantifying the "possibility". , 1976, Science.

[34]  P. Thompson,et al.  Energy conservation and credibility. , 1976, Science.

[35]  D. Barlow,et al.  Single Case Experimental Designs: Strategies for Studying Behavior Change , 1976 .

[36]  Harry Munsinger,et al.  The adopted child's IQ: A critical review. , 1975 .

[37]  R. Yin,et al.  Using the Case Survey Method To Analyze Policy Studies. , 1975 .

[38]  D. Campbell.  III. “Degrees of Freedom” and the Case Study , 1975 .

[39]  Frederick Mosteller,et al.  2 – Assessing Social Innovations: An Empirical Base for Policy , 1975 .

[40]  M. A. Lewis,et al.  An evaluation of the impact of school nurse practitioners. , 1974, The Journal of school health.

[41]  K. M. Barker,et al.  The Use of Nurse Practitioners in a General Practice , 1974, Medical care.

[42]  R. Light,et al.  Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies. , 1971 .

[43]  R. Hood,et al.  Key issues in criminology , 1970 .

[44]  Walter C. Bailey,et al.  Correctional Outcome: An Evaluation of 100 Reports , 1966 .

[45]  Paul E. Meehl,et al.  Seer over sign: The first good example. , 1965 .

[46]  J. Tukey.  The Future of Data Analysis , 1962 .

[47]  A. W. Astin,et al.  Glutamic acid and human intelligence. , 1960, Psychological bulletin.

[48]  E D KANE.  Differential indications for the use of glutamic acid. , 1953, The American journal of psychiatry.

[49]  G. S. Chambers,et al.  An evaluation of glutamic acid in mental deficiency. , 1952, The American journal of psychiatry.

[50]  McCulloch Tl.  The effect of glutamic acid feeding on cognitive abilities of institutionalized mental defectives. , 1950 .

[51]  P. Lazarsfeld,et al.  The quantification of case studies. , 1940 .

[52]  F. Freeman,et al.  The Influence of Environment on the Intelligence, School Achievement, and Conduct of Foster Children , 1928, Teachers College Record: The Voice of Scholarship in Education.