Furious activity vs. understanding: How much expertise is needed to evaluate creative work?

What is the role of expertise in evaluating creative products? Novices and experts do not assess creativity similarly, indicating domain-specific knowledge’s role in judging creativity. We describe two studies that examined how “quasi-experts” (people who have more experience in a domain than novices but also lack recognized standing as experts) compared with novices and experts in rating creative work. In Study 1, we compared different types of quasi-experts with novices and experts in rating short stories. In Study 2, we compared experts, quasi-experts, and novices in evaluating an engineering product (a mousetrap design). Quasi-experts (regardless of type) seemed to be appropriate raters for short stories, yet results were mixed for the engineer quasi-experts. Some domains may require more expertise than others to properly evaluate creative work.

[1]  T. Tebbs,et al.  Creativity 101 , 2011 .

[2]  Planck ' s Principle Do younger scientists accept new scientific ideas with greater alacrity than older scientists ? , 2005 .

[3]  Steven M. Smith,et al.  The creative cognition approach. , 1995 .

[4]  J. Kaufman,et al.  Do experts and novices evaluate movies the same way , 2009 .

[5]  J. W. Getzels,et al.  The Creative Vision: A Longitudinal Study of Problem Finding in Art , 1977 .

[6]  S. Dollinger,et al.  Note on Consensual Assessment Technique in Creativity Research , 2005, Perceptual and motor skills.

[7]  James C. Kaufman,et al.  Dissecting the Golden Goose: Components of Studying Creative Writers , 2002 .

[8]  P. McLeod,et al.  Inflexibility of experts—Reality or myth? Quantifying the Einstellung effect in chess masters , 2008, Cognitive Psychology.

[9]  J. Kaufman,et al.  Beyond Big and Little: The Four C Model of Creativity , 2009 .

[10]  Dean Keith Simonton,et al.  Creative productivity: A predictive and explanatory model of career trajectories and landmarks. , 1997 .

[11]  Adrian Furnham.  The criterion problem , 2009 .

[12]  James C. Kaufman,et al.  Captions, consistency, creativity, and the consensual assessment technique: New evidence of reliability , 2007 .

[13]  C. Darwin.  The Origin of Species by Means of Natural Selection, Or, The Preservation of Favoured Races in the Struggle for Life , 1859 .

[14]  James C. Kaufman,et al.  Comprar Essentials of Creativity Assessment | James C. Kaufman | 9780470137420 | Wiley , 2008 .

[15]  Lisa F. Smith,et al.  The Influence of Presentation Format and Viewer Training in the Visual Arts on the Perception of Pictorial and Aesthetic Qualities of Paintings , 2001, Perception.

[16]  John Baer,et al.  The American Idol Effect: Are Students Good Judges of Their Creativity across Domains? , 2010 .

[17]  James C. Kaufman,et al.  Expertise, Domains, and the Consensual Assessment Technique. , 2009 .

[18]  Adrian Furnham,et al.  Personality and aesthetic preference in Spain and England: two studies relating sensation seeking and openness to experience to liking for paintings and music , 2000 .

[19]  I L CHILD,et al.  Personal preferences as an expression of aesthetic sensitivity. , 1962, Journal of personality.

[20]  P. Silvia,et al.  Must interesting things be pleasant? A test of competing appraisal structures. , 2006, Emotion.

[21]  T. Jacobsen,et al.  Aesthetic judgments of music in experts and laypersons--an ERP study. , 2010, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[22]  Paul Hekkert,et al.  Beauty in the eye of expert and nonexpert beholders : A study in the appraisal of art , 1996 .

[23]  David H. Cropley,et al.  Correlates of creativity among visual art students. , 2010 .

[24]  Donald W. Fiske,et al.  Criterion Problem , 2022, The SAGE Encyclopedia of Research Design.

[25]  John Baer,et al.  A Comparison of Expert and Nonexpert Raters Using the Consensual Assessment Technique , 2008 .

[26]  John Baer,et al.  Essentials of Creativity Assessment , 2008 .

[27]  Lucille Daniels.  Stimulating Creativity. Vol. 1, Individual Procedures , 1976 .

[28]  Jocelyn E. Holden,et al.  The criterion problem and creativity in film: Psychometric characteristics of various measures. , 2008 .

[29]  David H. Cropley,et al.  Measuring Functional Creativity: Non‐Expert Raters and the Creative Solution Diagnosis Scale , 2012 .

[30]  M. F. Washburn,et al.  Experiments on a Possible Test of Aesthetic Judgment of Pictures , 1918 .

[31]  R. Sternberg,et al.  Complex Problem Solving : Principles and Mechanisms , 1992 .

[32]  A. Cropley,et al.  Elements of a universal aesthetic of creativity. , 2008 .

[33]  Paul J. Silvia,et al.  Discernment and creativity: How well can people identify their most creative ideas? , 2008 .

[34]  R. W. Roring,et al.  Giftedness and evidence for reproducibly superior performance: an account based on the expert performance framework , 2007 .

[35]  Ying-Yao Cheng,et al.  Effects of Association Instruction on Fourth Graders' Poetic Creativity in Taiwan , 2010 .

[36]  Dean Keith Simonton,et al.  Psychology's Status as a Scientific Discipline: Its Empirical Placement within an Implicit Hierarchy of the Sciences , 2004 .

[37]  D. Simonton.  Varieties of (Scientific) Creativity: A Hierarchical Model of Domain-Specific Disposition, Development, and Achievement , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[38]  M. Hickey,et al.  An Application of Amabile's Consensual Assessment Technique for Rating the Creativity of Children's Musical Compositions , 2001 .

[39]  T. M. Amabile.  Social psychology of creativity: A consensual assessment technique. , 1982 .

[40]  H. Leder,et al.  How art is appreciated. , 2012 .

[41]  Kevin W. Boyack,et al.  Toward a consensus map of science , 2009 .

[42]  S. Dollinger,et al.  Creativity and conservatism , 2007 .

[43]  J. Kaufman,et al.  In the Eye of the Beholder: Differences Across Ethnicity and Gender in Evaluating Creative Work , 2010 .

[44]  John Baer,et al.  Divergent thinking is not a general trait: A multidomain training experiment , 1994 .

[45]  James C. Kaufman,et al.  Do Gifted Student Writers and Creative Writing Experts Rate Creativity the Same Way? , 2005 .

[46]  Randi A. Engle,et al.  From representation to decision: An analysis of problem solving in international relations. , 1991 .

[47]  James C. Kaufman,et al.  Beyond New and Appropriate: Who Decides What Is Creative? , 2012 .

[48]  Arthur J. Kover,et al.  Stimulating Creativity, Volume 1: Individual Procedures. , 1974 .

[49]  Keith K. Millis,et al.  Making meaning brings pleasure: the influence of titles on aesthetic experiences. , 2001, Emotion.

[50]  Teresa M. Amabile,et al.  Creativity In Context: Update To The Social Psychology Of Creativity , 1996 .

[51]  Kevin W. Boyack,et al.  Toward a consensus map of science , 2009, J. Assoc. Inf. Sci. Technol..

[52]  John Baer,et al.  Extension of the Consensual Assessment Technique to Nonparallel Creative Products , 2004 .

[53]  David H. Cropley,et al.  Engineering Creativity: A Systems Concept of Functional Creativity , 2005 .

[54]  Joohyun Lee,et al.  A Variation of CAT for Measuring Creativity in Business Products , 2005 .

[55]  Paula E. Stephan,et al.  Planck's Principle Revisited: A Note , 1995 .

[56]  Dean Keith Simonton,et al.  What is a creative idea? Little-c versus Big-C creativity , 2013 .

[57]  J. Chan,et al.  Handbook of Research on Creativity , 2014 .

[58]  J. Schooler,et al.  The ineffability of insight. , 1995 .

[59]  Su Weiwei,et al.  A Multi-cultural Application of the Consensual Assessment Technique , 2008 .

[60]  Aaron Kozbelt.  A quantitative analysis of Beethoven as self-critic: implications for psychological theories of musical creativity , 2007 .

[61]  John Baer,et al.  Gender Differences in Creativity , 2008 .

[62]  Paul J. Silvia,et al.  Artistic Training and Interest in Visual Art: Applying the Appraisal Model of Aesthetic Emotions , 2006 .

[63]  M. Runco,et al.  Problem Construction and Creativity: The Role of Ability, Cue Consistency, and Active Processing , 1997 .