Post-deployment Data Collection in Software-Intensive Embedded Products

Companies collect terabytes of data about their products in the field, but research shows that R&D makes little use of this data, i.e., it is an untapped resource. In this paper, we explore collection and usage of post-deployment product data. We highlight the existing limitations in post-deployment data usage and the untapped resource that post-deployment product data remains. Based on a multiple case study at three software development companies, we collected three main findings. First, post-deployment data is used as input to the next pre-development phase, but not for improvement of existing product versions. Second, post-deployment data is used for troubleshooting and support activities, but not for innovation of new features. Third, post-deployment data provides a system-level understanding of operation and performance, but does not provide insight in individual feature usage. Finally, we propose a framework in which we outline what development practices and organizational mechanisms that need to be in place for advancing the usage of post-deployment data and advance the development of software-intensive embedded products.

[1]  Craig Larman,et al.  Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum , 2008 .

[2]  Jeffrey Y. Kim,et al.  Customer-Driven Innovation , 2008 .

[3]  Tony Gorschek,et al.  The impact of agile principles on market-driven software product development , 2010, J. Softw. Maintenance Res. Pract..

[4]  Jan Bosch,et al.  Towards Data-Driven Product Development: A Multiple Case Study on Post-deployment Data Usage in Software-Intensive Embedded Systems , 2013, LESS.

[5]  Kaj Grønbæk,et al.  CSCW challenges: cooperative design in engineering projects , 1993, CACM.

[6]  Denise Silver,et al.  Joint Application Development , 1995 .

[7]  Pekka Abrahamsson,et al.  ‘Lots done, more to do’: the current state of agile systems development research , 2009, Eur. J. Inf. Syst..

[8]  Acm Sigsoft,et al.  Proceedings of the 2000 International Conference on Software Engineering, June 4-11, 2000, Limerick, Ireland , 2000 .

[9]  Keith H. Bennett,et al.  Software maintenance and evolution: a roadmap , 2000, ICSE '00.

[10]  Geoff Walsham,et al.  Interpretive case studies in IS research: nature and method , 1995 .

[11]  Malcolm Munro,et al.  Software Maintenance and Evolution , 2003 .

[12]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[13]  Ron Kohavi,et al.  Controlled experiments on the web: survey and practical guide , 2009, Data Mining and Knowledge Discovery.

[14]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[15]  M. Lecompte,et al.  Ethnography and Qualitative Design in Educational Research , 1984 .

[16]  Jan Bosch,et al.  Climbing the "Stairway to Heaven" -- A Mulitiple-Case Study Exploring Barriers in the Transition from Agile Development towards Continuous Deployment of Software , 2012, 2012 38th Euromicro Conference on Software Engineering and Advanced Applications.

[17]  Ken Schwaber,et al.  Agile Software Development with SCRUM , 2001 .

[18]  Nicholas Berente,et al.  Emerging principles for requirements processes in organizational contexts , 2008, Ingénierie des Systèmes d Inf..

[19]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[20]  A. Strauss,et al.  Basics of qualitative research: Grounded theory procedures and techniques. , 1992 .

[21]  Alok Mishra,et al.  Journal of Software Maintenance and Evolution: Research and Practice Complex Software Project Development: Agile Methods Adoption , 2022 .

[22]  Raees Ahmad Khan,et al.  Software Engineering: A Practitioners Approach , 2014 .

[23]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[24]  Alistair Cockburn,et al.  Agile Software Development: The Business of Innovation , 2001, Computer.

[25]  Bashar Nuseibeh,et al.  Requirements engineering: a roadmap , 2000, ICSE '00.

[26]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach (McGraw-Hill Series in Computer Science) , 2004 .

[27]  Jan Bosch,et al.  Building Products as Innovation Experiment Systems , 2012, ICSOB.

[28]  Tony Gorschek,et al.  The impact of agile principles on market-driven software product development , 2010, J. Softw. Maintenance Res. Pract..

[29]  Jan Bosch,et al.  Eternal Embedded Software: Towards Innovation Experiment Systems , 2012, ISoLA.

[30]  Kent L. Beck,et al.  Embracing Change with Extreme Programming , 1999, Computer.

[31]  Craig Larman,et al.  Agile and Iterative Development: A Manager's Guide , 2003 .

[32]  E. Hippel Democratizing innovation: The evolving phenomenon of user innovation , 2009 .

[33]  Douglas Schuler,et al.  Participatory Design: Principles and Practices , 1993 .

[34]  A. Strauss,et al.  Basics of qualitative research: Grounded theory procedures and techniques. , 1993 .

[35]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[36]  Thomas H. Davenport,et al.  How to design smart business experiments , 2009 .