Finding and solving problems in software new product development

New product development is notoriously difficult, and software new product development particularly so. Although a great deal of research has investigated new product development, projects developing new software products continue to have problems meeting their goals. In fact, one line of research proposes new product development is difficult because it must solve an ongoing stream of complex problems. I integrate this line of research with two others to develop a conceptual framework of new product development as a process of finding and solving problems. From this framework, I develop four hypotheses that predict the probability projects developing new products will attain their development schedule and product quality goals. More specifically, I hypothesize that projects that generate access to, and integrate, large quantities of creative ideas, in-depth knowledge, and accurate information, should increase their probability of attaining schedule and product quality goals. Projects developing new products should both generate and integrate this “knowledge” to solve the problems that stand between them and their goals. However, how projects find problems also matters. Projects that search to identify problems earlier, rather than later, should also increase their probability of meeting schedule and product quality goals. To test these hypotheses I gathered data on 33 projects that tried to develop new software products from 23 firms, through interviews and questionnaires. Results from regression analyses support three out of four hypotheses. The projects that had high levels of both knowledge generation and integration had a significantly higher probability of attaining their product quality goals, but not their schedule goals. In contrast, projects that merely searched to find problems had a higher probability of attaining both goals. Moreover, projects that not only generated and integrated knowledge to solve problems, but also searched to find them, had the highest probability of attaining their product quality goals. This study illustrates the usefulness of modeling new product development as a bundle of problems to be found and solved. These results suggest that projects that combine practices to implement high levels of both knowledge generation and integration—not just one or the other—increase their chances of meeting product quality goals. This in turn suggests that focus on any single process or practice may be misplaced. Moreover, proactive search for problems may increase projects’ chances of meeting both schedule and product quality goals. In fact, search for problems was highly significant in this study, which suggests the way projects identify problems deserves further study. Although these prescriptions are preliminary, this study suggests they can help projects—and their managers—embody their visions in products and deliver those products to market.

[1]  J. B. Quinn,et al.  Managing Innovation: Controlled Chaos , 1985 .

[2]  Edward Yourdon Decline and Fall of the American Programmer , 1992 .

[3]  Kasra Ferdows,et al.  Managing international manufacturing , 1989 .

[4]  Tom R. Burns,et al.  The Management of Innovation. , 1963 .

[5]  Marshall W. Meyer,et al.  Power in Organizations. , 1982 .

[6]  P. Lawrence,et al.  Organization and Environment: Managing Differentiation and Integration , 1967 .

[7]  Michael A. Cusumano,et al.  Alternative designs for product component integration , 1998 .

[8]  K. Clark,et al.  The power of product integrity. , 1990, Harvard business review.

[9]  N. Draper,et al.  Applied Regression Analysis , 1967 .

[10]  K. Clark,et al.  Organizing and Leading “Heavyweight” Development Teams , 1992 .

[11]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[12]  C. Smart,et al.  Designs for crisis decision units. , 1977, Administrative science quarterly.

[13]  Stefan H. Thomke,et al.  The Effect of 'Front-Loading' Problem-Solving on Product Development Performance , 2000 .

[14]  S. Kiesler,et al.  Managerial Response to Changing Environments: Perspectives on Problem Sensing from Social Cognition. , 1982 .

[15]  A. V. D. Ven,et al.  Determinants of Coordination Modes within Organizations , 1976 .

[16]  S. Chatterjee,et al.  Regression Analysis by Example , 1979 .

[17]  J. Jaccard,et al.  Interaction effects in multiple regression , 1992 .

[18]  Thomas J. Allen,et al.  Managing the flow of technology: technology transfer and the dissemination of technological informat , 1977 .

[19]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[20]  Deborah G. . Ancona,et al.  Bridging the Boundary: External Activity and Performance in Organizational Teams. , 1992 .

[21]  I. Nonaka A Dynamic Theory of Organizational Knowledge Creation , 1994 .

[22]  E. Hippel Sticky Information and the Locus of Problem Solving: Implications for Innovation , 1994 .

[23]  M. Tushman Work Characteristics and Subunit Communication Structure: A Contingency Analysis. , 1979 .

[24]  Tom DeMarco,et al.  Controlling Software Projects: Management, Measurement, and Estimates , 1986 .

[25]  James D. Thompson Organizations in Action , 1967 .

[26]  A. Griffin The Effect of Project and Process Characteristics on Product Development Cycle Time , 1997 .

[27]  Jane M. Howell,et al.  Champions of Technological Innovation. , 1990 .

[28]  A. Pettigrew Information Control as a Power Resource , 1972 .

[29]  C. Perrow A FRAMEWORK FOR THE COMPARATIVE ANALYSIS OF ORGANIZATIONS , 1967 .

[30]  Deborah Dougherty,et al.  Organizing for Innovation , 1999 .

[31]  C. Judd,et al.  Statistical difficulties of detecting interactions and moderator effects. , 1993, Psychological bulletin.

[32]  W. Nord,et al.  Implementing Routine and Radical Inno-vation , 1986 .

[33]  Vijay Mahajan,et al.  Issues and Opportunities in New Product Development: An Introduction to the Special Issue , 1997 .

[34]  D. Dougherty,et al.  Sustained product innovation in large, mature organizations: Overcoming innovation-to-organization problems. , 1996 .

[35]  Marco Iansiti,et al.  Rapid Learning and Adaptation in Product Development: An Empirical Study of the Internet Software Industry , 1998 .

[36]  R. Bales,et al.  Phases in group problem-solving. , 1951, Journal of abnormal and social psychology.

[37]  Robert E. Kraut,et al.  Coordination in software development , 1995, CACM.

[38]  Michael Aiken,et al.  Organization Structure and Communications , 1971 .

[39]  Graham K. Rand,et al.  Quantitative Applications in the Social Sciences , 1983 .

[40]  David H. Gobeli,et al.  Analyzing Product Innovations , 1987 .

[41]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[42]  A. Chakrabarti,et al.  Innovation Speed: A Conceptual Model of Context, Antecedents, and Outcomes , 1996 .

[43]  J. R. French,et al.  The bases of social power. , 1959 .

[44]  S. West,et al.  Multiple Regression: Testing and Interpreting Interactions. , 1994 .

[45]  M. Tushman,et al.  Readings in the Management of Innovation , 1988 .

[46]  Michael L. Tushman,et al.  Information Processing as an Integrating Concept in Organizational Design. , 1978 .

[47]  C. Gersick Pacing Strategic Change: The Case of a New Venture , 1994 .

[48]  Donald R. Lehmann,et al.  Models of Cooperative Group Decision-Making and Relative Influence: An Experimental Investigation of Family Purchase Decisions , 1987 .

[49]  Henry Mintzberg,et al.  The Structure of "Unstructured" Decision Processes , 1976 .

[50]  Stewart Clegg,et al.  Handbook of organization studies , 1997 .

[51]  R. Emerson Power-Dependence Relations , 1962, Power in Modern Societies.

[52]  Jay R. Galbraith Designing Complex Organizations , 1973 .

[53]  R. Calantone,et al.  Organisational, Technical and Marketing Antecedents for Successful New Product Development , 1993 .

[54]  W. G. Astley,et al.  Structural Sources of Intraorganizational: Power: A Theoretical Synthesis , 1984 .

[55]  A. V. D. Ven,et al.  Central problems in the management of innovation , 1986 .

[56]  K. Eisenhardt,et al.  Accelerating Adaptive Processes: Product Innovation in the Global Computer Industry , 1995 .

[57]  M. Tushman Special Boundary Roles in the Innovation Process. , 1977 .

[58]  Rosabeth Moss Kanter,et al.  When a Thousand Flowers Bloom: Structural, Collective, and Social Conditions for Innovation in Organization , 2000 .

[59]  E. Witte Field Research on Complex Decision-Making Processes – The Phase Theorem , 1972 .

[60]  Michael A. Cusumano,et al.  How Microsoft builds software , 1997, CACM.

[61]  K. Eisenhardt,et al.  PRODUCT DEVELOPMENT: PAST RESEARCH, PRESENT FINDINGS, AND FUTURE DIRECTIONS , 1995 .

[62]  Willow A. Sheremata Centrifugal and Centripetal Forces in Radical New Product Development Under Time Pressure , 2000 .

[63]  Gary D. Scudder,et al.  Improving Speed and Productivity of Software Development: A Global Survey of Software Developers , 1996, IEEE Trans. Software Eng..

[64]  M. Iansiti,et al.  Developing products on Internet time. , 1997, Harvard business review.

[65]  Kim P. Corfman,et al.  Perceptions of Relative Influence: Formation and Measurement , 1991 .