Evaluation of idea generation methods for conceptual design: Effectiveness metrics and design of experiments

A wide range of formal methods have been devised and used for idea generation in conceptual design. Experimental evidence is needed to support claims regarding the effectiveness of these methods in promoting idea generation in engineering design. Towards that goal this paper presents a set of effectiveness metrics experimental methods, data collection and analysis techniques. Statistically based Design of Experiments (DOE) principles were used in developing the guidelines. Four classes of operating variables were considered to characterize the design problem and the environment. The effectiveness metrics proposed are based on outcome and consists of the quantity, quality, novelty, and variety of ideas generated. Two experimental approaches have been developed. In the Direct Method, the influence of the type of design problem and various parameters related to the procedure of an idea generation method is measured by using the method in its entirety. In the Indirect Method, each idea generation method is decomposed into key components and its overall effectiveness is predicted by experimentally studying the effectiveness of its components and their mutual interactions.

[1]  Mark D. Gross,et al.  The Electronic Cocktail Napkin—a computational environment for working with design diagrams , 1996 .

[2]  Ömer Akin,et al.  Design protocol data and novel design decisions , 1995 .

[3]  Robin Roy,et al.  Case studies of creativity in innovative product development , 1993 .

[4]  Gordon R. Pennock,et al.  Theory of Machines and Mechanisms , 1965 .

[5]  Ömer Akin,et al.  Frames of reference in architectural design: analysing the hyperacclamation (A-h-a-!) , 1996 .

[6]  E. P. Torrance,et al.  ROLE OF EVALUATION IN CREATIVE THINKING. , 1964 .

[7]  Wade C. Rowatt,et al.  Perceptions of Brainstorming in Groups: The Quality Over Quantity Hypothesis , 1997 .

[8]  J. McGrath.  Groups: Interaction and Performance , 1984 .

[9]  Ray Hyman,et al.  On Prior Information and Creativity , 1961 .

[10]  Edward Sickafus.  Unified structured inventive thinking : how to invent , 1997 .

[11]  E. Paul Torrance,et al.  Guiding creative talent , 1962 .

[12]  Raanan Lipshitz,et al.  Getting out of Ruts: A Laboratory Study of a Cognitive Model of Reframing. , 1995 .

[13]  Richard B. Frost.  A Suggested Taxonomy for Engineering Design Problems , 1994 .

[14]  David G. Ullman,et al.  A taxonomy for mechanical design , 1992 .

[15]  W. Gordon.  Synectics: The Development of Creative Capacity , 1961 .

[16]  B. Lawson.  Design in mind , 1994 .

[17]  C. Martindale.  Creativity and connectionism. , 1995 .

[18]  Willemien Visser,et al.  Two functions of analogical reasoning in design: a cognitive-psychology approach , 1996 .

[19]  John Fabian,et al.  Creative Thinking And Problem Solving , 1990 .

[20]  Stephen R. Grossman,et al.  Seven Operating Principles for Enhanced Creative Problem Solving Training , 1993 .

[21]  Janet L. Kolodner,et al.  Powers of observation in creative design , 1996 .

[22]  Michael Tovey,et al.  Thinking styles and modelling systems , 1986 .

[23]  Willemien Visser,et al.  Use of episodic knowledge and information in design problem solving , 1995 .

[24]  Paula Phillips Carson,et al.  Managing Creativity Enhancement Through Goal-Setting and Feedback† , 1993 .

[25]  V. K. Kumar,et al.  Creativity Styles of Freshmen Students , 1991 .

[26]  Wolfgang Beitz,et al.  Engineering Design: A Systematic Approach , 1984 .

[27]  Barbara Hirst.  How Artists Overcome Creative Blocks. , 1992 .

[28]  John C. Donovan.  Creative Success: Twenty Years on the Back of an Envelope , 1985 .

[29]  David Cohen,et al.  Effect of group cohesiveness and training upon creative thinking. , 1960 .

[30]  F. Barron.  Putting creativity to work. , 1988 .

[31]  S. Parnes,et al.  Visioneering — State of the Art , 1987 .

[32]  T. Rickards,et al.  Procedures For Managers In Idea‐Deficient Situations: An Examination Of Brainstorming Approaches , 1978 .

[33]  S. Parnes.  Effects of extended effort in creative problem solving. , 1961 .

[34]  R. Sternberg,et al.  Intelligence, Information Processing and Analogical Reasoning : The Componential Analysis of Human Abilities , 1977 .

[35]  Nigel Cross,et al.  Winning by design: the methods of Gordon Murray, racing car designer , 1996 .

[36]  George Bugliarello.  Developing Creativity in an Engineering Science Course. Socrates Revisited. , 1969 .

[37]  I. B. Myers.  Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator , 1985 .

[38]  H. Scott Fogler,et al.  Strategies for Creative Problem Solving , 1994 .

[39]  T. Poze.  Analogical connections: the essence of creativity , 1983 .

[40]  D. Perkins.  The Mind's Best Work , 1981 .

[41]  R. Hogarth.  Judgement and choice: The psychology of decision , 1982 .

[42]  Lynne Schwab,et al.  No Static In Your Attic: Tapping Into Your Creative and Intuitive Abilities , 1991 .

[43]  G. Graen,et al.  Training in creative problem solving: Effects on ideation and problem finding and solving in an industrial research organization , 1982 .

[44]  J P GUILFORD,et al.  Relations of creative responses to working time and instructions. , 1957, Journal of experimental psychology.

[45]  S. Parnes,et al.  Effects of "brainstorming" instructions on creative problem solving by trained and untrained subjects. , 1959 .

[46]  Scott G. Isaksen,et al.  Curriculum Planning for Creative Thinking and Problem Solving. , 1985 .

[47]  John S. Gero,et al.  Design and other types of fixation , 1996 .

[48]  Randolph M. Jones,et al.  A computational model of scientific insight , 1986 .

[49]  Justin C. D. Savage,et al.  The interaction of time and cost constraints on the design process , 1998 .

[50]  Linda Candy,et al.  Creative design of the Lotus bicycle: implications for knowledge support systems research , 1996 .

[51]  Ron Thompson,et al.  Usefulness of the Ideation Principle of Extended Effort in Real World Professional and Managerial Creative Problem Solving , 1986 .

[52]  R. Hamel,et al.  Sketching and creative discovery , 1998 .