Technology readiness levels at 40: A study of state-of-the-art use, challenges, and opportunities

The technology readiness level (TRL) scale was introduced by NASA in the 1970s as a tool for assessing the maturity of technologies during complex system development. TRL data have been used to make multi-million dollar technology management decisions in programs such as NASA's Mars Curiosity Rover. This scale is now a de facto standard used for technology assessment and oversight in many industries, from power systems to consumer electronics. Low TRLs have been associated with significantly reduced timeliness and increased costs across a portfolio of US Department of Defense programs. However, anecdotal evidence raises concerns about many of the practices related to TRLs. We study TRL implementations based on semi-structured interviews with employees from seven different organizations and examine documentation collected from industry standards and organizational guidelines related to technology development and demonstration. Our findings consist of 15 challenges observed in TRL implementations that fall into three different categories: system complexity, planning and review, and validity of assessment. We explore research opportunities for these challenges and posit that addressing these opportunities, either singly or in groups, could improve decision processes and performance outcomes in complex engineering projects.

[1]  Robert Phaal,et al.  Technology roadmapping—A planning framework for evolution and revolution , 2004 .

[2]  P. Clarkson,et al.  Predicting change propagation in complex design , 2004 .

[3]  Shahram Sarkani,et al.  The Relationship of Technology and Design Maturity to DoD Weapon System Cost Change and Schedule Change During Engineering and Manufacturing Development, , 2015, Syst. Eng..

[4]  Ronald N. Kostoff,et al.  Science and technology roadmaps , 2001, IEEE Trans. Engineering Management.

[5]  Robert Shishko,et al.  NASA technology assessment using real options valuation , 2004, Syst. Eng..

[6]  Devanandham Henry,et al.  A system maturity index for the systems engineering life cycle , 2008 .

[7]  Robert Rosen,et al.  The NASA technology push towards future space mission systems , 1989 .

[8]  Michael W. Berry,et al.  Document classification techniques for automated technology readiness level analysis , 2008 .

[9]  Shahram Sarkani,et al.  A Comprehensive Review and Analysis of Maturity Assessment Approaches for Improved Decision Support to Achieve Efficient Defense Acquisition , 2009 .

[10]  Christopher Ittner,et al.  An Empirical Examination of Dynamic Quality-Based Learning Models , 2001, Manag. Sci..

[11]  Brian J. Sauser,et al.  Integration maturity metrics: Development of an integration readiness level , 2010, Inf. Knowl. Syst. Manag..

[12]  Timothy K. Brady.  Utilization of Dependency Structure Matrix Analysis to Assess Complex Project Designs , 2002 .

[13]  Marvin V. Zelkowitz,et al.  Software engineering technology infusion within NASA , 1996 .

[14]  Shahram Sarkani,et al.  A framework for evaluating technology readiness, system quality, and program performance of U.S. DoD acquisitions , 2011, Syst. Eng..

[15]  Tyson R. Browning,et al.  Design Structure Matrix Methods and Applications , 2012 .

[16]  S. Argentati,et al.  Horizon 2020: the EU framework programme for research and innovation , 2015 .

[17]  John C. Mankins,et al.  Technology readiness assessments: A retrospective , 2009 .

[18]  Nelson P. Repenning,et al.  A Simulation-Based Approach to Understanding the Dynamics of Innovation Implementation , 2002, Organ. Sci..

[19]  W. Zangwill,et al.  Toward a Theory of Continuous Improvement and the Learning Curve , 1998 .

[20]  S.L. Cornford,et al.  Quantitative methods for maturing and infusing advanced spacecraft technology , 2004, 2004 IEEE Aerospace Conference Proceedings (IEEE Cat. No.04TH8720).

[21]  nasa,et al.  NASA Systems Engineering Handbook , 2007 .

[22]  Dimitri N. Mavris,et al.  Characterization of Technology Integration Based on Technology Readiness Levels , 2014 .

[23]  John C. Mankins,et al.  Approaches to strategic research and technology (R&T) analysis and road mapping , 2002 .

[24]  Olivier L. de Weck,et al.  Assessing risks and opportunities of technology infusion in system design , 2007, Syst. Eng..

[25]  Scott Cunningham.  Big data and technology readiness levels , 2014, IEEE Engineering Management Review.

[26]  Sirous Yasseri.  Subsea system readiness level assessment , 2013 .

[27]  A. Edmondson,et al.  METHODOLOGICAL FIT IN MANAGEMENT FIELD RESEARCH. , 2007 .