Towards Continuous Customer Validation: A Conceptual Model for Combining Qualitative Customer Feedback with Quantitative Customer Observation

Software-intensive product companies are becoming increasingly data-driven as can be witnessed by the big data and Internet of Things trends. However, optimally prioritizing customer needs in a mass-market context is notoriously difficult. While most companies use product owners or managers to represent the customer, research shows that the prioritization made is far from optimal. In earlier research, we have coined the term ‘the open loop problem’ to characterize this challenge. For instance, research shows that up to half of all the features in products are never used. This paper presents a conceptual model that emphasizes the need for combining qualitative feedback in early stages of development with quantitative customer observation in later stages of development. Our model is inductively derived from an 18 months close collaboration with six large global software-intensive companies.

[1]  Franz Lehner,et al.  Requirements Engineering as a Success Factor in Software Projects , 2001, IEEE Softw..

[2]  Craig Larman,et al.  Agile and Iterative Development: A Manager's Guide , 2003 .

[3]  Ken Schwaber,et al.  Agile Software Development with SCRUM , 2001 .

[4]  Fabian Fagerholm,et al.  Building blocks for continuous experimentation , 2014, RCoSE 2014.

[5]  Juhani Iivari,et al.  The user – the great unknown of systems development: reasons, forms, challenges, experiences and intellectual contributions of user involvement , 2010, Inf. Syst. J..

[6]  Eric Ries,et al.  The lean startup : how constant innovation creates radically successful businesses , 2011 .

[7]  Jan Bosch,et al.  Building Products as Innovation Experiment Systems , 2012, ICSOB.

[8]  Mika Westerlund,et al.  Designing Business Models for the Internet of Things , 2014 .

[9]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[10]  Slinger Jansen,et al.  Customer Involvement in Requirements Management: Lessons from Mass Market Software Development , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[11]  Geoff Walsham,et al.  Interpretive case studies in IS research: nature and method , 1995 .

[12]  Rongqiu Chen,et al.  Customer Participation: Co-Creating Knowledge with Customers , 2008, 2008 4th International Conference on Wireless Communications, Networking and Mobile Computing.

[13]  A. Strauss,et al.  Basics of Qualitative Research , 1992 .

[14]  Vassilis Agouridas,et al.  Value creation and capture mechanisms in innovation ecosystems: a comparative case study , 2013, Int. J. Technol. Manag..

[15]  Jan Bosch,et al.  From Opinions to Data-Driven Software R&D: A Multi-case Study on How to Close the 'Open Loop' Problem , 2014, 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications.

[16]  N. Mack,et al.  Qualitative research methods: a data collectors field guide. , 2005 .

[17]  Jan Bosch,et al.  The Early Stage Software Startup Development Model: A Framework for Operationalizing Lean Principles in Software Startups , 2013, LESS.

[18]  Jan Bosch,et al.  Towards Data-Driven Product Development: A Multiple Case Study on Post-deployment Data Usage in Software-Intensive Embedded Systems , 2013, LESS.

[19]  Veda C. Storey,et al.  Business Intelligence and Analytics: From Big Data to Big Impact , 2012, MIS Q..

[20]  Steve Blank.  The Four Steps to the Epiphany: Successful Strategies for Products that Win , 2013 .

[21]  Jan Bosch,et al.  Post-deployment Data Collection in Software-Intensive Embedded Products , 2013, ICSOB.

[22]  Jan Bosch,et al.  User involvement throughout the innovation process in high-tech industries , 2015 .

[23]  Jan Bosch,et al.  Eternal Embedded Software: Towards Innovation Experiment Systems , 2012, ISoLA.

[24]  R. Stake.  The art of case study research , 1995 .

[25]  Jan Bosch,et al.  Automated User Interaction Analysis for Workflow-Based Web Portals , 2014, ICSOB.

[26]  M. Lecompte,et al.  Ethnography and Qualitative Design in Educational Research , 1984 .

[27]  Jan Bosch,et al.  Scale and Responsiveness in Large-Scale Software Development , 2014, IEEE Software.

[28]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[29]  Alistair Cockburn,et al.  Agile Software Development: The Business of Innovation , 2001, Computer.

[30]  Ron Kohavi,et al.  Responsible editor: R. Bayardo. , 2022 .