A comparison of designer activity using core design situations in the laboratory and practice

In 2011 one quarter of all articles published in Design Studies and the Journal of Engineering Design used experimental studies. However, there is little work exploring the relationship between laboratory and practice. This paper addresses this by detailing an analysis of designer activity in three situations commonly studied by design researchers: information seeking, ideation and design review. This comparison is instantiated through three complementary studies: an observational study of practice and two experimental studies. These reveal a range of similarities and differences that are described using a mixed methods approach. Based on this it is concluded that laboratory studies are important research tools and that clear and definable relationships do exist between design activity in practice and the laboratory.

[1]  Alice M. Agogino,et al.  Insights on Designers’ Sketching Activities in New Product Design Teams , 2004 .

[2]  Gary E. Bolton,et al.  Does Laboratory Trading Mirror Behavior in Real World Markets? Fair Bargaining and Competitive Bidding on Ebay , 2008, SSRN Electronic Journal.

[3]  K. Leonard,et al.  Outpatient process quality evaluation and the Hawthorne Effect. , 2006, Social science & medicine.

[4]  John P. Forsyth,et al.  Moving from the laboratory to the real world and back again : Increasing the relevance of laboratory examinations of anxiety sensitivity , 1999 .

[5]  Carolyn Watters,et al.  A field study characterizing Web-based information-seeking tasks , 2007 .

[6]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[7]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[8]  Stephen Culley,et al.  Analysing creative behaviour in the later stage design process , 2013 .

[9]  Stephen Culley,et al.  The management of digital sketches through PLM solutions , 2009 .

[10]  Herbert Birkhofer,et al.  Imparting Design Methods With the Strategies of Experts , 2007 .

[11]  T. Mechlinski,et al.  Product Data Management , 1999, PRO-VE.

[12]  Jan A.M. Corremans,et al.  Measuring the effectiveness of a design method to generate form alternatives: an experiment performed with freshmen students product development , 2011 .

[13]  Loran F. Nordgren,et al.  The Scope-Severity Paradox , 2011 .

[14]  Donald W. King,et al.  Communication by Engineers: A Literature Review of Engineers' Information Needs, Seeking Processes, and Use. , 1994 .

[15]  Duncan Sanderson,et al.  Coordinating joint design work: the role of communication and artefacts , 1998 .

[16]  Ken Anderson,et al.  Numbers Have Qualities Too: Experiences with Ethno‐Mining , 2009 .

[17]  Ben J Hicks,et al.  An information requirement strategy for capturing and analysisng design activity and behaviour , 2010 .

[18]  Mattias Bergström,et al.  Lessons learned and future challenges for design observatory research , 2009 .

[19]  Nick Steen,et al.  Applying psychological theories to evidence-based clinical practice: identifying factors predictive of placing preventive fissure sealants , 2010, Implementation science : IS.

[20]  Linden J. Ball,et al.  Putting ethnography to work: the case for a cognitive ethnography of design , 2000, Int. J. Hum. Comput. Stud..

[21]  J. G. Adair,et al.  The Hawthorne effect: A reconsideration of the methodological artifact. , 1984 .

[22]  Cynthia J. Atman,et al.  A comparison of freshman and senior engineering design processes , 1999 .

[23]  G. Button.  The ethnographic tradition and design , 2000 .

[24]  Kathleen M. Eisenhardt,et al.  Theory Building From Cases: Opportunities And Challenges , 2007 .

[25]  Emma Marsden,et al.  Can educational experiments both test a theory and inform practice , 2007 .

[26]  D. Torgerson,et al.  Avoiding Bias in Randomised Controlled Trials in Educational Research , 2003 .

[27]  Petra Badke-Schaub,et al.  Analysis of design projects , 1999 .

[28]  Christoph Hölscher,et al.  Web search behavior of Internet experts and newbies , 2000, Comput. Networks.

[29]  Ernest A. Edmonds,et al.  The studio as laboratory: Combining creative practice and digital technology research , 2005, Int. J. Hum. Comput. Stud..

[30]  Steven D. Levitt,et al.  What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World , 2007 .

[31]  Elizabeth Wilkinson,et al.  The Task Force on Statistical Inference , 1999 .

[32]  Chris A. McMahon,et al.  Making sense of engineering design review activities , 2007, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[33]  John S. Gero,et al.  The structure of concurrent cognitive actions: a case study on novice and expert designers , 2002 .

[34]  Mark Robinson.  An empirical analysis of engineers' information behaviors , 2010 .

[35]  Helen Sharp,et al.  Ethnographically-informed empirical studies of software practice , 2007, Inf. Softw. Technol..

[36]  Karrie Karahalios,et al.  VCode and VData: illustrating a new framework for supporting the video annotation workflow , 2008, AVI '08.

[37]  Ben J Hicks,et al.  Designer behaviour and activity: an industrial observation method , 2011 .

[38]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[39]  Nigel Cross,et al.  Expertise in Design: an overview , 2004 .

[40]  Suzie Allard,et al.  Design engineers and technical professionals at work: Observing information usage in the workplace , 2009, J. Assoc. Inf. Sci. Technol..

[41]  Denis Newman,et al.  Can Scientific Research From the Laboratory be of Any Use to Teachers? , 2004 .

[42]  S. Reed.  Structure and Behaviour: Extending Duverger's Law to the Japanese Case , 1990, British Journal of Political Science.

[43]  Thomas J. Howard,et al.  Reuse of ideas and concepts for creative stimuli in engineering design , 2011 .

[44]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[45]  Edward Elias,et al.  Methodological insights from a rigorous small scale design experiment , 2012 .

[46]  Pirita Seitamaa-Hakkarainen,et al.  Composition and construction in experts' and novices' weaving design , 2001 .

[47]  Ken Friedman,et al.  Creating design knowledge: from research into practice , 2000 .

[48]  L. S. Vygotskiĭ,et al.  Mind in society : the development of higher psychological processes , 1978 .

[49]  Mark A. Robinson,et al.  An empirical analysis of engineers' information behaviors , 2010, J. Assoc. Inf. Sci. Technol..

[50]  Lucienne Blessing,et al.  Understanding the differences between how novice and experienced designers approach design tasks , 2003 .

[51]  William R. Shadish,et al.  Propensity Scores and Quasi-Experiments: A Testimony to the Practical Side of Lee Sechrest. , 2006 .