Software Development in Startup Companies

Context: Software startups are newly created companies with no operating history and are extremely fast in producing cutting-edge technologies. These companies develop software un- der highly uncertain conditions, tackling fast growing markets with severe lack of resources. Startups present an unique combination of characteristics which pose several challenges to the software development activities, creating interesting problems for software engineers. However, despite the increasing economical importance and the high failure rate, there are only a few sci- entific studies attempting to address software engineering (SE) issues, especially for early-stages startups. In a context where a wrong decision can easily lead the entire business to failure, the support of SE can contribute to foster performances of startups and making a big impact on a large number of companies. Objective: In view of a lack of primary studies, the first step to attending the software development strategies with scientific and engineering approaches is by an understanding of the startups’ behavior. For this reason this research aims to understand how software development strategies are engineered by practitioners, in the period of time that goes from idea conception to the first open beta release of the software product. Methods: This research combines a systematic review of the state-of-the-art with a cross- sectional case study conducted in 13 web startups recently founded and distributed in di↵erent geographic areas and market sectors. A grounded theory approach guided the execution of a sys- tematic mapping study, integrated with semi-structured interviews and follow-up questionnaires to explore the state-of-practice. Results: We selected, classified and evaluated 37 relevant studies. The systematic review revealed that the studies which constitute the body of knowledge are generally not rigorously designed and executed, make use of inconsistent terminology, and cover only small samples of startups. Moreover, we extrapolated concepts from the case study to form a theoretical model, explaining the underlying phenomenon of software development in early-stage startups. The model is grounded in the empirical data and its explanatory power was further validated through a systematic procedure. Finally we proposed a multi-faceted evolutionary model to describe the dynamics of the software development after the first product release. Conclusions: The research provided a wide set of evidences fostering the understanding of how software development is structured and executed, from idea conception to the first release. The results revealed the urgent priority of startups of releasing the product as quickly as possi- ble to verify the product/market fit and to adjust the business and product trajectory according to the early collected user feedbacks. Nevertheless, the initial gain obtained in speeding-up the development by low-precision and product-centric engineering activities is counterbalanced by the need of restructuring the product and the workflows before setting o↵ for further grow. In fact, when user requests and company’s size start to increase startups face an initial and temporary drop-down in productivity, creating the need of mitigation strategies to find a sweet spot between being fast enough to enter the market early while controlling the amount of accu- mulated technical debt. The conclusions can be generalized with a good degree of confidence to the majority of early-stage software startups involved in the production of innovative products, especially for web and mobile applications.

[1]  Tefko Saracevic,et al.  Evaluation of evaluation in information retrieval , 1995, SIGIR '95.

[2]  E.J. de Bruijn,et al.  Innovation and technology based economic development: Are there short-cuts? , 2008, 2008 4th IEEE International Conference on Management of Innovation and Technology.

[3]  Steve Hansen,et al.  Web Engineering: Creating a Discipline among Disciplines , 2001, IEEE Multim..

[4]  BernsteinLarry Characterizing people as non-linear, first-order components in software development, is written by Alistair A.R. Cockburn and published in Humans and Technology, HaT Technical Report 1999.03, Oct 21, 1999. , 2010 .

[5]  Gerry Coleman,et al.  eXtreme Programming (XP) as a 'minimum' software process: a grounded theory , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[6]  A. Strauss,et al.  Grounded theory , 2017 .

[7]  Mohamed E. Fayad,et al.  Software Engineering in the Small , 2000 .

[8]  Robert L. Nord,et al.  Managing technical debt in software-reliant systems , 2010, FoSER '10.

[9]  Christophe Midler,et al.  Managing robust development process for high-tech startups through multi-project learning: The case of two European start-ups , 2008 .

[10]  S. Jamieson Likert scales: how to (ab)use them , 2004, Medical education.

[11]  D. Ross Jeffery,et al.  An exploratory study of why organizations do not adopt CMMI , 2007, J. Syst. Softw..

[12]  Athula Ginige,et al.  Guest Editors' Introduction: Web Engineering - An Introduction , 2001, IEEE Multim..

[13]  Tony Gorschek,et al.  Empirical evidence in global software engineering: a systematic review , 2010, Empirical Software Engineering.

[14]  S. Brinkkemper,et al.  Understanding the dynamics of product software development using the concept of coevolution , 2006, 2006 Second International IEEE Workshop on Software Evolvability (SE'06).

[15]  O.W. Bertelsen Toward A Unified Field Of SE Research And Practice , 1997, IEEE Software.

[16]  C. Chan,et al.  The Satir Model and Cultural Sensitivity: A Hong Kong Reflection , 2002 .

[17]  Rory O'Connor,et al.  Investigating software process in practice: A grounded theory perspective , 2008, J. Syst. Softw..

[18]  M. Kakati Success criteria in high-tech new ventures , 2003 .

[19]  James Mater Solving the Software Quality Management Problem , 2001 .

[20]  Jessica Livingston,et al.  Founders at Work: Stories of Startups' Early Days , 2007 .

[21]  Gerhard Chroust What is a software process? , 1996, J. Syst. Archit..

[22]  Ward Cunningham,et al.  The WyCash portfolio management system , 1992, OOPSLA '92.

[23]  Frank Maurer,et al.  LIPE: A Lightweight Process for E-business Startup Companies Based on Extreme Programming , 2001, PROFES.

[24]  Ian Sommerville,et al.  Improving Market-Driven RE Processes , 1999 .

[25]  A. Grandi,et al.  Business incubators and new venture creation: an assessment of incubating models , 2005 .

[26]  Mauri Laitinen,et al.  Process assessment considered wasteful , 1997, CACM.

[27]  M Slatkin,et al.  Fisher's fundamental theorem of natural selection. , 1992, Trends in ecology & evolution.

[28]  Fergal McCaffery,et al.  Adept: A Unified Assessment Method for Small Software Companies , 2007, IEEE Software.

[29]  A. M. M. Sharif Ullah,et al.  Analysis of Kano‐model‐based customer needs for product development , 2011, Syst. Eng..

[30]  Eric Deakins,et al.  A helical model for managing innovative product and service initiatives in volatile commercial environments , 2005 .

[31]  Daniel M. Berry,et al.  Academic Legitimacy of the Software Engineering Discipline , 1992 .

[32]  Barbara Igel,et al.  The factors of making the first successful customer reference to leverage the business of start-up software company - multiple case study in Thai software industry , 2004 .

[33]  Mary Shaw,et al.  Writing good software engineering research papers , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[34]  Thierry Burger-Helmchen,et al.  Capabilities in small high‐tech firms: a case of plural‐entrepreneurship , 2009 .

[35]  Vanda Broughton How to Find Information: A Guide for Researchers , 2005 .

[36]  Slinger Jansen,et al.  Pragmatic and Opportunistic Reuse in Innovative Start-up Companies , 2008, IEEE Software.

[37]  Joost Visser,et al.  An empirical model of technical debt and interest , 2011, MTD '11.

[38]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[39]  Tony Gorschek,et al.  Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review , 2012, IEEE Transactions on Software Engineering.

[40]  Tony Gorschek,et al.  A method for evaluating rigor and industrial relevance of technology evaluations , 2011, Empirical Software Engineering.

[41]  Erran Carmel,et al.  Time-to-completion in software package startups , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[42]  J. Pearl Why there is no statistical test for confounding, why many think there is, and why they are almost right , 1998 .

[43]  Carlo Ghezzi,et al.  Software processes: a retrospective and a path to the future , 1998, Software Process: Improvement and Practice.

[44]  Stanley M. Sutton The Role of Process in a Software Start-up , 2000, IEEE Softw..

[45]  Colin Potts,et al.  Invented requirements and imagined customers: requirements engineering for off-the-shelf software , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[46]  Philippe Kruchten,et al.  Reconciling Perspectives: How People Manage the Process of Software Development , 2011, 2011 AGILE Conference.

[47]  Karl E. Wiegers Software Process Improvement in Web Time , 1999, IEEE Softw..

[48]  John W. Daly,et al.  Multi-method research: An empirical investigation of object-oriented technology , 1999, J. Syst. Softw..

[49]  Bart Baesens,et al.  Identifying financially successful start-up profiles with data mining , 2011, Expert Syst. Appl..

[50]  Edgar Possani,et al.  Keep or return? Managing ordering and return policies in start-up companies , 2007, Eur. J. Oper. Res..

[51]  LuAnn Bean,et al.  Wiki: A speedy new tool to manage projects , 2005 .

[52]  Johan Natt och Dag,et al.  Elicitation and management of user requirements in market-driven software development , 2002 .

[53]  Linda Rising,et al.  The Scrum Software Development Process for Small Teams , 2000, IEEE Softw..

[54]  Marvin V. Zelkowitz,et al.  Software process improvement in small organizations: a case study , 2005, IEEE Software.

[55]  Frederick P. Brooks,et al.  No Silver Bullet: Essence and Accidents of Software Engineering , 1987 .

[56]  Christian W. Dawson,et al.  Projects in Computing and Information Systems: A Student's Guide , 2009 .

[57]  Barbara Mirel,et al.  Product, process, and profit: the politics of usability in a software venture , 2000, AJCD.

[58]  Muthu Ramachandran,et al.  PRISMS: an approach to software process improvement for small to medium enterprises , 2003, Third International Conference on Quality Software, 2003. Proceedings..

[59]  Alistair Cockburn,et al.  Surviving Object-Oriented Projects , 1998 .

[60]  Yasuhiro Mashiko,et al.  Using the GQM Paradigm to Investigate Influential Factors for Software Process Improvement , 1997, J. Syst. Softw..

[61]  Michael A. Cusumano,et al.  Building a Company on Internet Time: Lessons from Netscape , 1999 .

[62]  Roberto Deias,et al.  Introducing XP in a start-up , 2002 .

[63]  Suresh Tadisina,et al.  Factors Impacting Customers' Initial Trust in E-Businesses: An Empirical Study , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[64]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[65]  James A. Highsmith,et al.  Adaptive Software Development: A Collaborative Approach to Managing Complex Systems , 1999 .

[66]  G. Norman Likert scales, levels of measurement and the “laws” of statistics , 2010, Advances in health sciences education : theory and practice.

[67]  Nanua Singh,et al.  Lean product development: Maximizing the customer perceived value through design change (redesign) , 2008 .

[68]  Colin Robson,et al.  Real World Research: A Resource for Social Scientists and Practitioner-Researchers , 1993 .

[69]  Pearl Brereton,et al.  Using Mapping Studies in Software Engineering , 2008, PPIG.

[70]  Jan Pries-Heje,et al.  How Internet software companies negotiate quality , 2001, Computer.

[71]  Mira Kajko-Mattsson,et al.  From Knowing Nothing to Knowing a Little: Experiences Gained from Process Improvement in a Start-Up Company , 2008, 2008 International Conference on Computer Science and Software Engineering.

[72]  Rita Gunther McGrath,et al.  The Entrepreneurial Mindset: Strategies for Continuously Creating Opportunity in an Age of Uncertainty , 2000 .

[73]  Tim J. Kane,et al.  The Importance of Startups in Job Creation and Job Destruction , 2010 .

[74]  Victor R. Basili,et al.  Iterative enhancement: A practical technique for software development , 1975, IEEE Transactions on Software Engineering.

[75]  Olli-Pekka Hilmola,et al.  The value of product development lead time in software startup , 2003 .

[76]  Erran Carmel,et al.  A process model for packaged software development , 1995 .

[77]  Mark Zuckerberg,et al.  Why Software Is Eating the World , 2011 .

[78]  Tore Dybå,et al.  Evidence-Based Software Engineering for Practitioners , 2005, IEEE Softw..

[79]  V. Basili Software modeling and measurement: the Goal/Question/Metric paradigm , 1992 .

[80]  P. Banerjee,et al.  Some indicators of dynamic technological competencies: understanding of Indian software managers , 2003 .

[81]  Rory O'Connor,et al.  Software Process in Practice: A Grounded Theory of the Irish Software Industry , 2006, EuroSPI.

[82]  Bob Dorf,et al.  The step-by-step guide for building a great company , 2012 .

[83]  A. Jefferson Offutt,et al.  Quality Attributes of Web Software Applications , 2002, IEEE Softw..

[84]  Tobias Kollmann,et al.  IT Competence in Internet Founder Teams , 2010, Bus. Inf. Syst. Eng..

[85]  Christiane Gresse von Wangenheim,et al.  Helping small companies assess software processes , 2006, IEEE Software.

[86]  J. Creswell,et al.  Determining Validity in Qualitative Inquiry , 2000 .

[87]  Marko Taipale Huitale - A Story of a Finnish Lean Startup , 2010, LESS.

[88]  Rory V. O'Connor,et al.  Using grounded theory to understand software process improvement: A study of Irish software product companies , 2007, Inf. Softw. Technol..

[89]  Jonathan Grudin,et al.  Interactive systems: bridging the gaps between developers and users , 1991, Computer.

[90]  Akbar Saeed,et al.  Extreme Programming in Action: A Longitudinal Case Study , 2007, HCI.

[91]  Clayton M. Christensen The Innovator's Dilemma , 1997 .

[92]  Jarkko Pyysiäinen,et al.  Challenges of software product companies: results of a national survey in Finland , 2004, 10th International Symposium on Software Metrics, 2004. Proceedings..

[93]  S. Brinkkemper,et al.  A Tentative Technique for the Study and Planning of Co-Evolution in Product , 2007, Third International IEEE Workshop on Software Evolvability 2007.

[94]  Tore Dybå,et al.  A systematic review of statistical power in software engineering experiments , 2006, Inf. Softw. Technol..

[95]  R. Cooper,et al.  0 0 0 0 An Investigation into the New Product Process : Steps , Deficiencies , and Impact , 1986 .

[96]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[97]  P. Brudlo Entrepreneurship in Internet business , 2008, 2008 1st International Conference on Information Technology.

[98]  James Bach Microdynamics of Process Evolution , 1998, Computer.

[99]  Michael A. Cusumano,et al.  Software Development on Internet Time , 1999, Computer.

[100]  Mira Kajko-Mattsson,et al.  From scrum to scrumban: A case study of a process transition , 2012, 2012 International Conference on Software and System Process (ICSSP).

[101]  Shuib Basri,et al.  Towards an Understanding of Software Development Process Knowledge in Very Small Companies , 2011 .

[102]  M.M. Tanabian,et al.  Building high-performance team through effective job design for an early stage software start-up , 2005, Proceedings. 2005 IEEE International Engineering Management Conference, 2005..

[103]  Scott W. Ambler,et al.  Lessons in Agility From Internet-Based Development , 2002, IEEE Softw..

[104]  Noam Wasserman,et al.  The Founder's Dilemmas: Anticipating and Avoiding the Pitfalls That Can Sink a Startup , 2012 .

[105]  R. Keith Stanfill,et al.  AC 2007-2797: IMPROVING ENTREPRENEURSHIP TEAM PERFORMANCE THROUGH MARKET FEASIBILITY ANALYSIS, EARLY IDENTIFICATION OF TECHNICAL REQUIREMENTS, AND INTELLECTUAL PROPERTY SUPPORT , 2007 .

[106]  Jianfeng Xu,et al.  Web Quality of Agile Web Development , 2009, SSME.

[107]  Alistair R. Anderson,et al.  Success in Israeli high-tech start-ups; Critical factors and process , 2006 .

[108]  David J. Storey,et al.  Entrepreneurship and New Firm , 2016 .

[109]  Anthony B. Perkins,et al.  The Internet Bubble: Inside the Overvalued World of High-Tech Stocks--- And What You Need to Know to Avoid the Coming Shakeout , 1999 .

[110]  Wanda J. Orlikowski,et al.  CASE Tools as Organizational Change: Investigating Incremental and Radical Changes in Systems Development , 1993, MIS Q..

[111]  Emilia Mendes,et al.  Software process improvement success factors for small and medium Web companies: A qualitative study , 2012, Inf. Softw. Technol..

[112]  Helen M. Edwards,et al.  The repertory grid technique: Its place in empirical software engineering research , 2009, Inf. Softw. Technol..

[113]  Jarmo J. Ahonen,et al.  Big Improvements with Small Changes: Improving the Processes of a Small Software Company , 2008, PROFES.

[114]  M. Crowne,et al.  Why software product startups fail and what to do about it. Evolution of software product development in startup companies , 2002, IEEE International Engineering Management Conference.

[115]  Fabio Kon,et al.  XP South of the Equator: An eXPerience Implementing XP in Brazil , 2005, XP.

[116]  Yuanfang Cai,et al.  Organizing the technical debt landscape , 2012, 2012 Third International Workshop on Managing Technical Debt (MTD).

[117]  Roel Wieringa,et al.  Requirements engineering paper classification and evaluation criteria: a proposal and a discussion , 2005, Requirements Engineering.

[118]  L. Lai,et al.  Chinese Entrepreneurship in the Internet Age: Lessons from Alibaba.com , 2010 .

[119]  Gerry Coleman,et al.  An Empirical Study of Software Process in Practice , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[120]  David A. E. Wall Software Technologies - Using Open Source for a Profitable Startup , 2001, Computer.

[121]  Olli Pekka Hilmola Question of software start-up finance: system dynamics simulation analysis , 2009 .

[122]  Lee J. Krajewski,et al.  Kanban, MRP, and Shaping the Manufacturing Environment , 1987 .

[123]  Donald J. Reifer,et al.  Web Development: Estimating Quick-to-Market Software , 2000, IEEE Softw..

[124]  Jari Ruokolainen,et al.  Gear-up your software start-up company by the first reference customer—nomothetic research study in the Thai software industry , 2005 .

[125]  Randy K. Smith,et al.  Value-Oriented Requirements Prioritization in a Small Development Organization , 2007, IEEE Software.

[126]  Tugrul U. Daim,et al.  Information technology acquisition in the service sector , 2010 .

[127]  Tony Gorschek,et al.  Technology transfer decision support in requirements engineering research: a systematic review of REj , 2009, Requirements Engineering.

[128]  Jaelson Brelaz de Castro,et al.  A Study in Market-Driven Requirements Engineering , 2006, WER.

[129]  Gordon B. Davis,et al.  Software Development Practices, Software Complexity, and Software Maintenance Performance: a Field Study , 1998 .

[130]  Kent Beck,et al.  Extreme Programming Explained: Embrace Change (2nd Edition) , 2004 .

[131]  Mark Kasunic,et al.  Designing an Effective Survey , 2005 .

[132]  Tore Dybå,et al.  Empirical studies of agile software development: A systematic review , 2008, Inf. Softw. Technol..

[133]  S. Yogendra,et al.  Aligning business and technology strategies: a comparison of established and start-up business contexts , 2002, IEEE International Engineering Management Conference.

[134]  Susan Woldenberg Butler Tips and tricks in performing a systematic review , 2007 .

[135]  Erran Carmel,et al.  Customer-developer links in software development , 1995, CACM.

[136]  K. Eisenhardt Building theories from case study research , 1989, STUDI ORGANIZZATIVI.

[137]  Suu-Chuang Li,et al.  The Role of Value Proposition and Value Co-Production in New Internet Startups: How New Venture e-Businesses Achieve Competitive Advantage , 2007, PICMET '07 - 2007 Portland International Conference on Management of Engineering & Technology.

[138]  Steve Blank The Four Steps to the Epiphany: Successful Strategies for Products that Win , 2013 .

[139]  B. Glaser Theoretical Sensitivity: Advances in the Methodology of Grounded Theory , 1978 .

[140]  Judith G. Brodman,et al.  What small business and small organizations say about the CMM: experience report , 1994, ICSE '94.

[141]  Ita Richardson,et al.  Why are small software organisations different , 2007 .

[142]  Soly Mathew Biju Agile Software Development Methods and its Advantages , 2010 .

[143]  B. Turner,et al.  Grounded Theory and Organizational Research , 1986 .

[144]  Rory O'Connor,et al.  An investigation into software development process formation in software start-ups , 2008, J. Enterp. Inf. Manag..

[145]  David Smagalla,et al.  The Truth About Software Startups , 2004 .

[146]  Newton M. Campos The Lean Startup: How today's entrepreneurs use continuous innovation to create radically successful businesses , 2014 .

[147]  K. Eisenhardt,et al.  Time pacing: competing in markets that won't stand still. , 1998, Harvard business review.

[148]  Hindy Lauer Schachter,et al.  The role played by Frederick Taylor in the rise of the academic management fields , 2010 .

[149]  G. Kelly A theory of personality : the psychology of personal constructs , 1963 .

[150]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[151]  L Ramer,et al.  Quantitative versus qualitative research? , 1989, Journal of obstetric, gynecologic, and neonatal nursing : JOGNN.

[152]  Björn Regnell,et al.  Challenges in Market-Driven Requirements Engineering - an Industrial Interview Study , 2002 .

[153]  Charles Ashbacher,et al.  Peopleware: Productive Projects and Teams (Second Edition) , 2000 .

[154]  Roger Tourangeau,et al.  Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines. , 1987 .

[155]  Nauman Bin Ali,et al.  Towards innovation measurement in the software industry , 2013, J. Syst. Softw..

[156]  Ken Martin,et al.  An Open Source Approach to Developing Software in a Small Organization , 2007, IEEE Software.

[157]  M.M. Lehman,et al.  Programs, life cycles, and laws of software evolution , 1980, Proceedings of the IEEE.

[158]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[159]  Cynthia F. Kurtz,et al.  The new dynamics of strategy: sense-making in a complex and complicated world , 2003, IEEE Engineering Management Review.