On comparing manual and automatic generated textual descriptions of business process models

Several organizations maintain textual process descriptions alongside graphical process descriptions to make them usable for all stakeholders. Maintaining textual process descriptions in the presence of continuously changing processes is a labor‐intensive task. Therefore, the automatic generation of textual descriptions is desirable. However, the trade‐offs between the manual and automatic generation of descriptions are yet to be investigated. To that end, this paper aims to answer two vital questions. How similar are the descriptions generated by the two approaches? What is the impact of using the two types of descriptions on process matching? To answer these specific questions, we have generated textual descriptions of 552 process models using the two approaches. To answer the first question, we have applied six text‐matching techniques and established that the descriptions overlap significantly; however, the formulation of sentences is substantially different. For answering the second question, we have used 11 text‐matching techniques to evaluate the impact of both descriptions on process matching. Results show (a) the choice of matching technique, and the type of description, have an impact on the matching performance and (b) vector space model (VSM) is the most appropriate matching technique whereas 5 gram is the worst performing technique.

[1]  Ralf Laue,et al.  A comparative survey of business process similarity measures , 2012, Comput. Ind..

[2]  Jan Mendling,et al.  Towards the Automated Annotation of Process Models , 2015, CAiSE.

[3]  Jeffrey Parsons,et al.  What do the pictures mean? Guidelines for experimental evaluation of representation fidelity in diagrammatical conceptual modeling techniques , 2005, Data Knowl. Eng..

[4]  Wil M. P. van der Aalst,et al.  On the Suitability of BPMN for Business Process Modelling , 2006, Business Process Management.

[5]  Flávia Maria Santoro,et al.  Business process mining from group stories , 2009, 2009 13th International Conference on Computer Supported Cooperative Work in Design.

[6]  Remco M. Dijkman,et al.  Report: The Process Model Matching Contest 2013 , 2013, Business Process Management Workshops.

[7]  Jan Mendling,et al.  A Study Into the Factors That Influence the Understandability of Business Process Models , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[8]  Jan Mendling,et al.  The Impact of Secondary Notation on Process Model Understanding , 2009, PoEM.

[9]  Jan Mendling,et al.  Activity labeling in process modeling: Empirical insights and recommendations , 2010, Inf. Syst..

[10]  Adnan Abid,et al.  A Process Model Collection and Gold Standard Correspondences for Process Model Matching , 2019, IEEE Access.

[11]  Jan Recker,et al.  Detecting approximate clones in business process model repositories , 2015, Inf. Syst..

[12]  Marlon Dumas,et al.  Fast detection of exact clones in business process model repositories , 2013, Inf. Syst..

[13]  Jan Mendling,et al.  Increasing Recall of Process Model Matching by Improved Activity Label Matching , 2013, BPM.

[14]  Paul W. P. J. Grefen,et al.  Generating process model collections , 2015, Software & Systems Modeling.

[15]  Remco M. Dijkman,et al.  Similarity of business process models: Metrics and evaluation , 2011, Inf. Syst..

[16]  Jan Mendling,et al.  Making sense of business process descriptions: An experimental comparison of graphical and textual notations , 2012, J. Syst. Softw..

[17]  Hector Garcia-Molina,et al.  SCAM: A Copy Detection Mechanism for Digital Documents , 1995, DL.

[18]  Heiner Stuckenschmidt,et al.  Probabilistic Evaluation of Process Model Matching Techniques , 2016, ER.

[19]  Mathias Weske,et al.  Behavioral Similarity - A Proper Metric , 2011, BPM.

[20]  Remco M. Dijkman,et al.  APROMORE: An advanced process model repository , 2011, Expert Syst. Appl..

[21]  Andrew Gemino,et al.  A framework for empirical evaluation of conceptual modeling techniques , 2004, Requirements Engineering.

[22]  Jan Mendling,et al.  On Measuring the Understandability of Process Models , 2009, Business Process Management Workshops.

[23]  Jan Mendling,et al.  Supporting Process Model Validation through Natural Language Generation , 2014, IEEE Transactions on Software Engineering.

[24]  Jan Mendling,et al.  Integrating Textual and Model-Based Process Descriptions for Comprehensive Process Search , 2016, BMMDS/EMMSAD.

[25]  A ReijersHajo,et al.  A visual analysis of the process of process modeling , 2015 .

[26]  Imran Sarwar Bajwa,et al.  From Natural Language Software Specifications to UML Class Models , 2011, ICEIS.

[27]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.

[28]  Beate List,et al.  An evaluation of conceptual business process modelling languages , 2006, SAC.

[29]  August-Wilhelm Scheer,et al.  Enterprise resource planning: making ERP a success , 2000, CACM.

[30]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[31]  Peter Loos,et al.  The Process Model Matching Contest 2015 , 2013, EMISA.

[32]  Geert Poels,et al.  A visual analysis of the process of process modeling , 2014, Information Systems and e-Business Management.

[33]  Wil M. P. van der Aalst,et al.  Workflow Patterns , 2004, Distributed and Parallel Databases.

[34]  Keng Siau,et al.  Evaluation techniques for systems analysis and design modelling methods – a review and comparative analysis , 2011, Inf. Syst. J..

[35]  Yahiko Kambayashi,et al.  A longest common subsequence algorithm suitable for similar text strings , 1982, Acta Informatica.

[36]  Rao Muhammad Adeel Nawab,et al.  Mono-lingual Paraphrased Text Reuse and Plagiarism Detection , 2012 .

[37]  Henrik Leopold,et al.  A Textual Description Based Approach to Process Matching , 2016, PoEM.

[38]  Eva Söderström,et al.  Towards a Framework for Comparing Process Modelling Languages , 2002, CAiSE.

[39]  CloughPaul,et al.  Developing a corpus of plagiarised short answers , 2011 .

[40]  Mathias Weske,et al.  Business Process Management: Concepts, Languages, Architectures , 2007 .

[41]  Christus,et al.  A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins , 2022 .

[42]  Martin Porter,et al.  Snowball: A language for stemming algorithms , 2001 .

[43]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[44]  Jan Mendling,et al.  Metrics for Process Models: Empirical Foundations of Verification, Error Prediction, and Guidelines for Correctness , 2008, Lecture Notes in Business Information Processing.

[45]  Yorick Wilks,et al.  Measuring Text Reuse , 2002, ACL.

[46]  Jan Mendling,et al.  Seven process modeling guidelines (7PMG) , 2010, Inf. Softw. Technol..

[47]  Hajo A. Reijers,et al.  Dealing with Behavioral Ambiguity in Textual Process Descriptions , 2016, BPM.

[48]  Monique Snoeck,et al.  Testing a Selection of BPMN Tools for Their Support of Modelling Guidelines , 2015, PoEM.

[49]  Mark Stevenson,et al.  Developing a corpus of plagiarised short answers , 2011, Lang. Resour. Evaluation.

[50]  Student,et al.  THE PROBABLE ERROR OF A MEAN , 1908 .

[51]  Saonee Sarker,et al.  An Exploration into the Process of Requirements Elicitation: A Grounded Approach , 2010, J. Assoc. Inf. Syst..

[52]  Wil M. P. van der Aalst,et al.  Workflow Patterns , 2003, Distributed and Parallel Databases.

[53]  Khurram Shahzad,et al.  Comparing manual- and auto-generated textual descriptions of business process models , 2016, 2016 Sixth International Conference on Innovative Computing Technology (INTECH).