Getting the whole story: an experience report on analyzing data elicited using the war stories procedure

When analyzing data elicited using the “war stories” technique, previously introduced by Lutters and Seaman (Inf Softw Technol 49(6):576–587, 2007), we encountered unexpected challenges in applying standard qualitative analysis techniques. After reviewing the literature on stories and storytelling, we realized that a richer analysis would be possible if we accorded more respect to the data’s structure and nature as stories, rather than treating our participants’ utterances simply as textual data. We report on five lessons learned regarding how we can better analyze war stories as stories: 1) war stories tend to be about exceptional situations; 2) war stories tend to be diverse and resistant to being combined into a single grand narrative; 3) the humanities can be a valuable resource for analyzing war stories; 4) war stories are not just text, they are also performances; and 5) war stories are not just data, they are also instructive and evocative.

[1]  Hilary Rose,et al.  Modest_Witness@Second_Millenium. Femaleman. copyright Meets_OncoMouse trademark: Feminism and Technoscience. Donna J. Haraway , 1998 .

[2]  E. Guba,et al.  Fourth Generation Evaluation , 1989 .

[3]  Robert L. Campbell,et al.  The programmer's burden: developing expertise in programming , 1992 .

[4]  Galal H. Galal-Edeen,et al.  Stakeholder identification in the requirements engineering process , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[5]  Marc Eisenstadt,et al.  My hairiest bug war stories , 1997, CACM.

[6]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[7]  Alfonso Fuggetta,et al.  Software process: a roadmap , 2000, ICSE '00.

[8]  Cristina V. Lopes,et al.  Archetypal Internet-Scale Source Code Searching , 2008, OSS.

[9]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[10]  Charles L. A. Clarke,et al.  Archetypal source code searches: a survey of software developers and maintainers , 1998, Proceedings. 6th International Workshop on Program Comprehension. IWPC'98 (Cat. No.98TB100242).

[11]  James A. Holstein,et al.  The active interview , 1995 .

[12]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[13]  Richard Baskerville,et al.  Amethodical systems development: the deferred meaning of systems development methods , 2000 .

[14]  B. Massumi,et al.  The postmodern condition : a report on knowledge , 1979 .

[15]  Dwayne D. Gremler.  The Critical Incident Technique in Service Research , 2004 .

[16]  Marsha Chechik,et al.  Multi-valued symbolic model-checking , 2003, TSEM.

[17]  D. Haraway,et al.  Modest_Witness@Second_Millennium. FemaleMan_Meets_OncoMouse , 1997 .

[18]  R. Schank,et al.  Knowledge and Memory: The Real Story , 1995 .

[19]  A. Strauss,et al.  Basics of qualitative research: Grounded theory procedures and techniques. , 1992 .

[20]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[21]  Juliet M. Corbin,et al.  Basics of Qualitative Research (3rd ed.): Techniques and Procedures for Developing Grounded Theory , 2008 .

[22]  A. Strauss,et al.  Basics of qualitative research: Grounded theory procedures and techniques. , 1993 .

[23]  P. Kay,et al.  What is the Sapir-Whorf hypothesis? , 1983 .

[24]  D. Boje.  The storytelling organization: A study of story performance in an office-supply firm. , 1991 .

[25]  Kerstin Dautenhahn,et al.  Chapter 4. Stories of lemurs and robots: The social origin of story-telling , 2003 .

[26]  Ian Sommerville,et al.  Social analysis in the requirements engineering process: from ethnography to method , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).

[27]  Bashar Nuseibeh,et al.  Security Requirements Engineering: A Framework for Representation and Analysis , 2008, IEEE Transactions on Software Engineering.

[28]  Kathleen Lennon,et al.  Modest_ Witness@Second__Millennium.FemaleMan.© Meets_Oncomouse™: Feminism and Technoscience , 1998 .

[29]  Don Hellriegei,et al.  The Lessons of Experience: How Successful Executives Develop on the Job , 1989 .

[30]  John Seely Brown,et al.  Book Reviews : The Social Life of Information By John Seely Brown and Paul Duguid. Boston: Harvard Business School Press, 2000. 320 pages , 2000 .

[31]  Mehrdad Sabetzadeh,et al.  View merging in the presence of incompleteness and inconsistency , 2006, Requirements Engineering.

[32]  Barbara Czarniawska.  Narrating the Organization: Dramas of Institutional Identity , 1997 .

[33]  Vladimir Propp,et al.  Morphology of the folktale , 1959 .

[34]  William M. K. Trochim,et al.  Research methods knowledge base , 2001 .

[35]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[36]  Ban Al-Ani,et al.  Marginal Notes on Amethodical Requirements Engineering:  What Experts Learned from Experience , 2008, 2008 16th IEEE International Requirements Engineering Conference.

[37]  Wayne G. Lutters,et al.  Revealing actual documentation usage in software maintenance through war stories , 2007, Inf. Softw. Technol..

[38]  J. Bruner.  The Narrative Construction of Reality , 1991, Critical Inquiry.

[39]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[40]  C. Geertz.  Thick Description: Towards an Interpretive Theory of Culture , 1973 .

[41]  Forrest Shull,et al.  The empirical investigation of Perspective-Based Reading , 1995, Empirical Software Engineering.

[42]  B. Pentland.  Building Process Theory with Narrative: from Description to Explanation , 1999 .

[43]  Neil A. M. Maiden,et al.  Can Requirements Be Creative? Experiences with an Enhanced Air Space Management System , 2007, 29th International Conference on Software Engineering (ICSE'07).

[44]  D. Wellman,et al.  Talking About Machines: An Ethnography of a Modern Job. , 1997 .

[45]  Joseph A. Goguen,et al.  Techniques for requirements elicitation , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[46]  I. Her.  Thick description: toward an interpretive theory of culture, in: The interpretation of cultures: selected essays . New-York/N.Y./USA etc , 2022 .

[47]  Wu Qing-lin.  The Role of Deliberate Practice in the Acquisition of AOC E-sports Expertise , 2005 .

[48]  J. Orr,et al.  Talking About Machines: An Ethnography of a Modern Job. , 1997 .

[49]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .