Is Originality in the Eye of the Beholder? Comparison of Scoring Techniques in the Assessment of Divergent Thinking

Divergent thinking (DT) tests are among the most popular techniques for measuring creativity. However, the validity evidence for DT tests, as applied in educational settings, is inconsistent partly due to different scoring methods. This study explored the reliability and validity issues of various techniques for administering and scoring two DT tests. Results show distinct differences among several methods for scoring these DT tests and suggest that the percentage scoring method (i.e., dividing originality scores by fluency scores) may be the most appropriate scoring strategy. The potential impact on educational research and practice is discussed in detail.

[1]  D. Hocevar.  The unidimensional nature of creative thinking in fifth grade children. , 1979 .

[2]  D. Hocevar,et al.  A Taxonomy and Critique of Measurements Used in the Study of Creativity , 1989 .

[3]  M. Runco.  Teachers' Judgments of Creativity and Social Validation of Divergent Thinking Tests , 1984 .

[4]  Teresa M. Amabile,et al.  Creativity In Context: Update To The Social Psychology Of Creativity , 1996 .

[5]  S. Messick.  Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[6]  T. M. Amabile.  Social psychology of creativity: A consensual assessment technique. , 1982 .

[7]  THE MEASUREMENT AND PROPERTIES OF DIVERGENT THINKING ABILITY AS A SINGLE COMPOUND ENTITY , 1983 .

[8]  R. Sternberg.  The Nature of creativity : contemporary psychological perspectives , 1988 .

[9]  R. Sternberg.  Handbook of Creativity: Subject Index , 1998 .

[10]  John Baer,et al.  Essentials of Creativity Assessment , 2008 .

[11]  M. Runco,et al.  The Psychometric Properties of Four Systems for Scoring Divergent Thinking Tests , 1987 .

[12]  Gary A. Davis,et al.  Testing for Creative Potential , 1989 .

[13]  John T. Willse,et al.  Assessing Creativity With Divergent Thinking Tasks: Exploring the Reliability and Validity of New Subjective Scoring Methods , 2008 .

[14]  Patrick C. Kyllonen,et al.  Noncognitive Constructs and Their Assessment in Graduate Education: A Review , 2005 .

[15]  M. Runco.  Children's divergent thinking and creative ideation , 1992 .

[16]  M. Runco,et al.  Creativity and the Finding and Solving of Real-World Problems , 1991 .

[17]  Michael A. Wallach,et al.  Modes of thinking in young children: A study of the creativity-intelligence distinction. , 1965 .

[18]  M. Runco.  Interrater Agreement on a Socially Valid Measure of Students' Creativity , 1987 .

[19]  D. Hocevar,et al.  Measurement of creativity: review and critique. , 1981, Journal of personality assessment.

[20]  H. Mirels,et al.  FLUENCY AS A PERVASIVE ELEMENT IN THE MEASUREMENT OF CREATIVITY1 , 1970 .

[21]  M. Runco.  Reliability and Convergent Validity of Ideational Flexibility as a Function of Academic Achievement , 1985 .

[22]  Mark A. Runco,et al.  Scoring Divergent Thinking Tests Using Total Ideational Output and a Creativity Index , 1992 .

[23]  Mark A. Runco,et al.  Commentary: Divergent Thinking Is Not Synonymous With Creativity , 2008 .

[24]  E. Hong,et al.  Original thinking as a predictor of creative performance in young children , 1995 .

[25]  W. R. Smith,et al.  Interpersonal and intrapersonal evaluations of creative ideas , 1992 .

[26]  Eunsook Hong,et al.  Creative thinking and creative performance in adolescents as predictors of creative attainments in adults: A follow‐up study after 18 years , 1993 .

[27]  Jonathan A. Plucker,et al.  Is the Proof in the Pudding? Reanalyses of Torrance's (1958 to Present) Longitudinal Data , 1999 .

[28]  D. Hocevar.  A comparison of statistical infrequency and subjective judgment as criteria in the measurement of originality. , 1979, Journal of personality assessment.

[29]  Nancy C. Canestaro,et al.  Creativity and achievement in design coursework , 1989 .

[30]  Mark A. Runco,et al.  The Reliability and Validity of Ideational Originality in the Divergent Thinking of Academically Gifted and Nongifted Children , 1985 .

[31]  E. Torrance.  The nature of creativity as manifest in its testing. , 1988 .

[32]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[33]  D. Hocevar,et al.  Ideational fluency as a confounding factor in the measurement of originality. , 1979 .

[34]  MaryBeth Walpole,et al.  SELECTING SUCCESSFUL GRADUATE STUDENTS: IN‐DEPTH INTERVIEWS WITH GRE® USERS , 2002 .

[35]  Carolyn M. Callahan,et al.  Creativity and Giftedness: Published Instrument Uses and Abuses , 1995 .

[36]  Jonathan A. Plucker,et al.  The death of creativity measurement has been greatly exaggerated: Current issues, recent advances, and future directions in creativity assessment , 1998 .

[37]  J. Plucker.  Student Responses to Creativity Checklists: Evidence of Content Generality. , 1996 .

[38]  William B. Michael,et al.  The Effects of Scoring Formulas on the Discriminant Validity of Tests of Divergent Thinking , 1979 .

[39]  Teresa M. Amabile,et al.  Story-Telling: A Method for Assessing Children's Creativity. , 1988 .

[40]  Paul J. Silvia,et al.  Creativity and Intelligence Revisited: A Latent Variable Analysis of Wallach and Kogan (1965) , 2008 .