Discovering taxonomic structure in design archives with application to risk-mitigating actions in a large engineering organisation

ABSTRACT This paper demonstrates a general iterative technique for discovering a taxonomy that categorises information contained within a weakly structured design project archive. It is common for large engineering organisations to maintain archives about past design projects. In principle, engineering organisations could mine these archives to extract useful lessons and relationships that would improve future design projects. However, this is difficult in practice, since archives often consist of weakly structured data formats, such as plaintext documentation. This restricts the use of many useful analysis tools. The taxonomy-discovery process presented here is a critical first step towards unlocking the value of such archives. The technique is based on methods from the qualitative research literature. We demonstrate this process by creating a taxonomy of risk-mitigating actions in design projects based on a design project archive from a large engineering organisation. We discuss practical considerations such as missing contextual information as part of the case study. The taxonomy is sufficiently generic to be of use to other organisations. Furthermore, individual organisations can use the iterative technique introduced in this paper to tailor the taxonomy to their own project archives. Thus, this research provides an important foundation for unlocking the value of archived design project information.

[1]  Asit P. Basu,et al.  Probabilistic Risk Analysis , 2002 .

[2]  Daniel A. McAdams,et al.  A Component Taxonomy as a Framework for Computational Design Synthesis , 2009, J. Comput. Inf. Sci. Eng..

[3]  Roger Miller,et al.  Understanding and Managing Risks in Large Engineering Projects , 2001 .

[4]  B. Boehm.  Software risk management: principles and practices , 1991, IEEE Software.

[5]  L B Lave,et al.  Risk analysis and risk management. , 1990, The Science of the total environment.

[6]  Orkunt Sabuncu,et al.  An ontology-based retrieval system using semantic indexing , 2010, 2010 IEEE 26th International Conference on Data Engineering Workshops (ICDEW 2010).

[7]  Steven L. Salzberg,et al.  Managing information for concurrent engineering: Challenges and barriers , 1990 .

[8]  P. Brennan,et al.  The kappa statistic for establishing interrater reliability in the secondary analysis of qualitative clinical data. , 1992, Research in nursing & health.

[9]  Bamfa Ceesay,et al.  NTNU: An Unsupervised Knowledge Approach for Taxonomy Extraction , 2015, SemEval@NAACL-HLT.

[10]  Pieter E. Vermaas,et al.  The coexistence of engineering meanings of function: Four responses and their methodological implications , 2013, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[11]  W E Vesely,et al.  Fault Tree Handbook , 1987 .

[12]  D. Coghlan,et al.  Action research for operations management , 2002 .

[13]  Yan Liang,et al.  Learning the "Whys": Discovering design rationale using text mining - An algorithm perspective , 2012, Comput. Aided Des..

[14]  K. B. Johnson,et al.  Quantifying the Literature of Computer‐aided Instruction in Medical Education , 2000, Academic medicine : journal of the Association of American Medical Colleges.

[15]  William C. Regli,et al.  On the long-term retention of geometry-centric digital engineering artifacts , 2011, Comput. Aided Des..

[16]  Michael zur Muehlen,et al.  Risk Management in the BPM Lifecycle , 2005, Business Process Management Workshops.

[17]  Carl Auerbach,et al.  Qualitative Data: An Introduction to Coding and Analysis , 2003 .

[18]  Jack C. Wileden,et al.  A Semantic Information Model for Capturing and Communicating Design Decisions , 2010, J. Comput. Inf. Sci. Eng..

[19]  Y. Wang,et al.  A multi-facet taxonomy system with applications in unstructured knowledge management , 2005, J. Knowl. Manag..

[20]  Flavius Frasincar,et al.  Domain taxonomy learning from text: The subsumption method versus hierarchical clustering , 2013, Data Knowl. Eng..

[21]  Jianzhong Cha,et al.  Design synthesis approach based on process decomposition to design reuse , 2012 .

[22]  Suresh L. Konda,et al.  Taxonomy-Based Risk Identification , 1993 .

[23]  Morten Hertzum,et al.  The information-seeking practices of engineers: searching for documents as well as for people , 2000, Inf. Process. Manag..

[24]  Chi Fai Cheung,et al.  Knowledge-based extraction of intellectual capital-related information from unstructured data , 2014, Expert Syst. Appl..

[25]  Ying Liu,et al.  An approach for design rationale retrieval using ontology-aided indexing , 2014 .

[26]  Laks V. S. Lakshmanan,et al.  Efficient extraction of ontologies from domain specific text corpora , 2012, CIKM '12.

[27]  Johan Malmqvist,et al.  Effective method for creating engineering checklists , 2013 .

[28]  A. Wilcox-Herzog,et al.  Actions speak louder than words: How experience and education relate to teachers’ behaviors , 2004 .

[29]  P. Mielke,et al.  A Generalization of Cohen's Kappa Agreement Measure to Interval Measurement and Multiple Raters , 1988 .

[30]  R. Bakeman,et al.  Detecting Sequential Patterns and Determining Their Reliability With Fallible Observers , 2001 .

[31]  P. John Clarkson,et al.  Core information categories for engineering design – contrasting empirical studies with a review of integrated models , 2014 .

[32]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[33]  David Clark-Carter,et al.  Doing Quantitative Psychological Research: From Design To Report , 1997 .

[34]  Y. Wang,et al.  A multi-faceted and automatic knowledge elicitation system (MAKES) for managing unstructured information , 2011, Expert Syst. Appl..

[35]  Daniel P. Thunnissen,et al.  Uncertainty Classification for the Design and Development of Complex Systems , 2003 .

[36]  A. Cantor,et al.  Sample-size calculations for Cohen's kappa. , 1996 .

[37]  Colin Robson,et al.  Real World Research: A Resource for Social Scientists and Practitioner-Researchers , 1993 .

[38]  Steve Culley,et al.  An analysis of the content of technical information used by engineering designers , 2000 .

[39]  Timo Honkela,et al.  Learning a taxonomy from a set of text documents , 2012, Appl. Soft Comput..

[40]  Elena Paslaru Bontas Simperl,et al.  ONTOCOM: A reliable cost estimation method for ontology development projects , 2012, J. Web Semant..

[41]  Martin Hepp,et al.  Ontologies: State of the Art, Business Potential, and Grand Challenges , 2008, Ontology Management.

[42]  Ian H. Witten,et al.  Automatic construction of lexicons, taxonomies, ontologies, and other knowledge structures , 2013, WIREs Data Mining Knowl. Discov..

[43]  Alan Gilchrist,et al.  Thesauri, taxonomies and ontologies - an etymological note , 2003, J. Documentation.

[44]  H. Schneider.  Failure mode and effect analysis : FMEA from theory to execution , 1996 .

[45]  Dave Stewart,et al.  Waypoint: An Integrated Search and Retrieval System for Engineering Documents , 2004, J. Comput. Inf. Sci. Eng..

[46]  Pieter Spooren,et al.  Student evaluation of teaching quality in higher education: development of an instrument based on 10 Likert‐scales , 2007 .

[47]  Ken M. Wallace,et al.  A Methodology for Creating Ontologies for Engineering Design , 2007, J. Comput. Inf. Sci. Eng..

[48]  Davide Aloini,et al.  Risk management in ERP project introduction: Review of the literature , 2007, Inf. Manag..

[49]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[50]  Santosh Jagtap,et al.  In-service information required by engineering designers , 2011 .

[51]  Homayoon Dezfuli,et al.  Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners (Second Edition) , 2011 .