Comparing manual- and auto-generated textual descriptions of business process models

Textual description of business process models can be generated either manually or automatically. Both ways of generating descriptions have their strengths and weaknesses. However, it is not clear, how similar or different are the two descriptions (manually generated and automatic generated). Once the answer to this question is known, it may lead to several conclusions, such as, the two descriptions can be used as an alternative to each other or not. To answer that question, in this study we have generated textual descriptions of 552 process models using two approaches (1) manual textual description approach and (2) automatic textual description approach. Subsequently, we apply three similarity estimation models including η-gram overlap, Longest Common Subsequence, and Vector Space Model to compute the degree of similarity between manual-automatic textual description pairs. Results show that the two types of textual descriptions are significantly different from each other and thus there is a space for improving the automatic technique in order to generate a textual description comparable with manually generated description.