Extraction of System States from Natural Language Requirements

In recent years, simulations have proven to be an important means to verify the behavior of complex software systems. The different states of a system are monitored in the simulations and are compared against the requirements specification. So far, system states in natural language requirements cannot be automatically linked to signals from the simulation. However, the manual mapping between requirements and simulation is a time-consuming task. Named-entity Recognition is a sub-task from the field of automated information retrieval and is used to classify parts of natural language texts into categories. In this paper, we use a self-trained Named-entity Recognition model with Bidirectional LSTMs and CNNs to extract states from requirements specifications. We present an almost entirely automated approach and an iterative semi-automated approach to train our model. The automated and iterative approach are compared and discussed with respect to the usual manual extraction. We show that the manual extraction of states in 2,000 requirements takes nine hours. Our automated approach achieves an F1-score of 0.51 with 15 minutes of manual work and the iterative approach achieves an F1-score of 0.62 with 100 minutes of work.

[1]  Luisa Mich,et al.  Market research for requirements analysis using linguistic tools , 2004, Requirements Engineering.

[2]  Ellen Riloff,et al.  Learning subjective nouns using extraction pattern bootstrapping , 2003, CoNLL.

[3]  Birgit Penzenstadler,et al.  From Requirements to Models: Feedback Generation as a Result of Formalization , 2011, CAiSE.

[4]  Nan Ye,et al.  Domain adaptive bootstrapping for named entity recognition , 2009, EMNLP.

[5]  Christopher D. Manning,et al.  Improved Pattern Learning for Bootstrapped Entity Extraction , 2014, CoNLL.

[6]  Andreas Vlachos,et al.  Bootstrapping and Evaluating Named Entity Recognition in the Biomedical Domain , 2006, BioNLP@NAACL-HLT.

[7]  Nelly Bencomo,et al.  Models@run.time , 2014, Lecture Notes in Computer Science.

[8]  Marcelo R. Campo,et al.  Identi(cid:28)cation of Non-Functional Requirements in Textual Speci(cid:28)cations: A Semi-Supervised Learning Approach , 2009 .

[9]  Ellen Riloff,et al.  A Bootstrapping Method for Learning Semantic Lexicons using Extraction Pattern Contexts , 2002, EMNLP.

[10]  Jonathan Schmidt,et al.  Executable State Machines Derived from Structured Textual Requirements - Connecting Requirements and Formal System Design. , 2019, MODELSWARD 2019.

[11]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[12]  Eric Nichols,et al.  Named Entity Recognition with Bidirectional LSTM-CNNs , 2015, TACL.

[13]  Leonid Kof.  Requirements Analysis: Concept Extraction and Translation of Textual Specifications to Executable Models , 2009, NLDB.

[14]  Eugénio C. Oliveira,et al.  A Bootstrapping Approach for Training a NER with Conditional Random Fields , 2011, EPIA.

[15]  Maria Pampaka,et al.  Simulation-based requirements discovery for smart driver assistive technologies , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[16]  Alex Dekhtyar,et al.  RE Data Challenge: Requirements Identification with Word2Vec and TensorFlow , 2017, 2017 IEEE 25th International Requirements Engineering Conference (RE).

[17]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[18]  Zornitsa Kozareva.  Bootstrapping Named Entity Recognition with Automatically Generated Gazetteer Lists , 2006, EACL.

[19]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[20]  Robert Feldt,et al.  Alignment of Requirements Specification and Testing: A Systematic Mapping Study , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[21]  Shubhashis Sengupta,et al.  Automatic extraction of glossary terms from natural language requirements , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[22]  Andreas Vogelsang,et al.  Automatic Classification of Requirements Based on Convolutional Neural Networks , 2016, 2016 IEEE 24th International Requirements Engineering Conference Workshops (REW).

[23]  Andreas Vogelsang,et al.  Take Care of Your Modes! An Investigation of Defects in Automotive Requirements , 2016, REFSQ.

[24]  Colin J. Neill,et al.  State of practice in requirements engineering: contemporary data , 2014, Innovations in Systems and Software Engineering.

[25]  Lionel C. Briand,et al.  Automated Transition from Use Cases to UML State Machines to Support State-Based Testing , 2011, ECMFA.

[26]  Steffen Staab,et al.  Bootstrapping an Ontology-Based Information Extraction System , 2003, Intelligent Exploration of the Web.

[27]  Andreas Vogelsang,et al.  Systematic elicitation of mode models for multifunctional systems , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[28]  David Dietrich,et al.  A mode-based pattern for feature requirements, and a generic feature interface , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[29]  Shige Wang,et al.  A feature-oriented requirements modelling language , 2012, 2012 20th IEEE International Requirements Engineering Conference (RE).

[30]  Constance L. Heitmeyer,et al.  The SCR Method for Formally Specifying, Verifying, and Validating Requirements: Tool Support , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[31]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[32]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[33]  Paul Rook,et al.  Controlling software projects , 1986, Softw. Eng. J..

[34]  Walter F. Tichy,et al.  Transferring research into the real world: How to improve RE with AI in the automotive industry , 2014, 2014 IEEE 1st International Workshop on Artificial Intelligence for Requirements Engineering (AIRE).

[35]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[36]  Ellen Riloff,et al.  Automatically Generating Extraction Patterns from Untagged Text , 1996, AAAI/IAAI, Vol. 2.

[37]  Marti A. Hearst.  Automatic Acquisition of Hyponyms from Large Text Corpora , 1992, COLING.

[38]  Andreas Vogelsang,et al.  A Lightweight Multilevel Markup Language for Connecting Software Requirements and Simulations , 2019, REFSQ.

[39]  Manfred Broy,et al.  Multifunctional software systems: Structured modeling and specification of functional requirements , 2010, Sci. Comput. Program..

[40]  J. Banks,et al.  Discrete-Event System Simulation , 1995 .