Automatic Classification of Requirements Based on Convolutional Neural Networks

The results of the requirements engineering process are predominantly documented in natural language requirements specifications. Besides the actual requirements, these documents contain additional content such as explanations, summaries, and figures. For the later use of requirements specifications, it is important to be able to differentiate between legally relevant requirements and other auxiliary content. Therefore, one of our industry partners demands the requirements engineers to manually label each content element of a requirements specification as "requirement" or "information". However, this manual labeling task is time-consuming and error-prone. In this paper, we present an approach to automatically classify content elements of a natural language requirements specification as "requirement" or "information". Our approach uses convolutional neural networks. In an initial evaluation on a real-world automotive requirements specification, our approach was able to detect requirements with a precision of 0.73 and a recall of 0.89. The approach increases the quality of requirements specifications in the sense that it discriminates important content for following activities (e.g., which parts of the specification do I need to test?).

[1]  Daniel Méndez Fernández,et al.  It's the Activities, Stupid! A New Perspective on RE Quality , 2015, 2015 IEEE/ACM 2nd International Workshop on Requirements Engineering and Testing.

[2]  Lior Rokach,et al.  Introduction to Knowledge Discovery and Data Mining , 2010, Data Mining and Knowledge Discovery Handbook.

[3]  Jonas Winkler.  Automatische Klassifikation von Anforderungen zur Unterstützung von Qualitätssicherungsprozessen , 2016, GI-Jahrestagung.

[4]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[7]  Daniel Ott,et al.  Automatic Requirement Categorization of Large Natural Language Specifications at Mercedes-Benz for Review Improvements , 2013, REFSQ.

[8]  Anna Perini,et al.  A Machine Learning Approach to Software Requirements Prioritization , 2013, IEEE Transactions on Software Engineering.

[9]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[10]  Charu C. Aggarwal,et al.  A Survey of Text Clustering Algorithms , 2012, Mining Text Data.

[11]  Charu C. Aggarwal,et al.  A Survey of Text Classification Algorithms , 2012, Mining Text Data.

[12]  James T. Kwok.  Automated Text Categorization Using Support Vector Machine , 1998, ICONIP.

[13]  Wenbin Li,et al.  Measuring Requirement Quality to Predict Testability , 2015, 2015 IEEE Second International Workshop on Artificial Intelligence for Requirements Engineering (AIRE).

[14]  Jane Cleland-Huang,et al.  Automated classification of non-functional requirements , 2007, Requirements Engineering.

[15]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[16]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[17]  José Miguel Mantas,et al.  Extraction of similarity based fuzzy rules from artificial neural networks , 2006, Int. J. Approx. Reason..

[18]  Nitesh V. Chawla,et al.  Data Mining for Imbalanced Datasets: An Overview , 2005, The Data Mining and Knowledge Discovery Handbook.

[19]  Vivek Narayanan,et al.  Fast and Accurate Sentiment Classification Using an Enhanced Naive Bayes Model , 2013, IDEAL.

[20]  Wenbin Li,et al.  Weka meets TraceLab: Toward convenient classification: Machine learning for requirements engineering problems: A position paper , 2014, 2014 IEEE 1st International Workshop on Artificial Intelligence for Requirements Engineering (AIRE).

[21]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[22]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[23]  Manfred Broy,et al.  A meta model for artefact-orientation: fundamentals and lessons learned in requirements engineering , 2010, MODELS'10.

[24]  Ignacio Requena,et al.  Are artificial neural networks black boxes? , 1997, IEEE Trans. Neural Networks.

[25]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.