From Opinions to Data-Driven Software R&D: A Multi-case Study on How to Close the 'Open Loop' Problem

In most software development companies the road mapping and requirements prioritization process is a complex process in which product management experiences difficulties in getting timely and accurate customer feedback. The feedback loop from customers is slow and often there is a lack of mechanisms that allow for efficient customer data collection and analysis. As a result, there is the risk that requirements prioritization becomes opinion-based rather than data-driven, and that R&D investments are made without an accurate way of continuously validating whether they correspond to customer needs. We call this phenomenon the 'open loop' problem, referring to the challenges for product management to get accurate and timely feedback from customers. To address this problem, we develop the HYPEX model (Hypothesis Experiment Data-Driven Development) that supports companies in running feature experiments to shorten customer feedback loops. We evaluate the model in three software development companies and observe how feature experiments increase the opportunity for data-driven software development.