Evaluating the Quality of Process Models : Empirical Analysis of a Quality Framework

This paper conducts an empirical analysis of the quality framework proposed by Lindland et al (1994) for evaluating the quality of conceptual models. 194 participants were trained in the concepts of the quality framework, and then used it to evaluate models in an extended ER language. A randomised, double-blind design was used, and the results evaluated by a combination of quantitative and qualitative analysis. Finally, an analysis was conducted of the framework’s perceived ease of use, usefulness and likelihood of adoption in practice. The study provides strong support for the validity of the framework and suggests that it is likely to be adopted in practice, but raises questions about its reliability. The research findings provide clear direction for further refinements of the framework to improve its reliability.

[1]  S. Carlsen.  Action Port Model: a mixed paradigm conceptual workflow modeling language , 1998, Proceedings. 3rd IFCIS International Conference on Cooperative Information Systems (Cat. No.98EX122).

[2]  Michael Rosemann,et al.  Testing a Framework for the Quality of Process Models - A Case Study , 2001, PACIS.

[3]  Patrick Y. K. Chau,et al.  An Empirical Assessment of a Modified Technology Acceptance Model , 1996, J. Manag. Inf. Syst..

[4]  Daniel L. Moody,et al.  Building links between IS research and professional practice: improving the relevance and impact of IS research , 2000, ICIS.

[5]  Ian G. MacDonald,et al.  Information Engineering , 2019, Information Systems Design Methodologies: Improving the Practice.

[6]  Peretz Shoval,et al.  Entity-Relationship and Object-Oriented Data Modeling-an Experimental Comparison of Design Quality , 1997, Data Knowl. Eng..

[7]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[8]  John Krogstie,et al.  Towards a Deeper Understanding of Quality in Requirements Engineering , 2013, Seminal Contributions to Information Systems Engineering.

[9]  Bill C. Hardgrave,et al.  Comparing Object-Oriented and Extended-Entity-Relationship Data Models , 1995 .

[10]  Arne Sølvberg,et al.  Understanding quality in conceptual modeling , 1994, IEEE Software.

[11]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[12]  Robert D. Galliers,et al.  Relevance and rigour in Information Systems Research: some personal reflections on issues facing the Information Systems research community , 1994, Business Process Re-Engineering.

[13]  Jörg Becker,et al.  Guidelines of Business Process Modeling , 2000, Business Process Management.

[14]  Christopher Alexander.  Notes on the Synthesis of Form , 1964 .

[15]  S. Lauesen,et al.  Preventing Requirement Defects: An Experiment in Process Improvement , 2001, Requirements Engineering.

[16]  Martha E. Crosby,et al.  The effect of graphic style on data model interpretation , 1999, Inf. Syst. J..

[17]  John Krogstie,et al.  Integrating the understanding of quality in requirements specification and conceptual modeling , 1998, SOEN.

[18]  Graeme G. Shanks,et al.  What Makes a Good Data Model? Evaluating the Quality of Entity Relationship Models , 1994, ER.

[19]  Susan Greener,et al.  Business Research Methods , 2008 .

[20]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[21]  N. Rescher.  Methodological Pragmatism: A Systems-Theoretic Approach to the Theory of Knowledge , 1977 .