Evaluating quality of conceptual modelling scripts based on user perceptions

This paper presents the development of a user evaluations based quality model for conceptual modelling scripts applying Seddon's variant of the well-known model of DeLone and McLean [W.H. DeLone, E.R. McLean, Information systems success: the quest for the dependent variable, Information Systems Journal 3(1) (1992) 60-95] for evaluating the success of information systems. Given the growing awareness among researchers and practitioners about the importance of high-quality conceptual modelling scripts, it is surprising that there is no practical evaluation framework that considers the quality of conceptual modelling scripts from the user perspective. A first research goal is therefore to determine what the appropriate dimensions are for evaluating the success or quality of conceptual modelling scripts from the user point of view. A second goal is to investigate the relationships between these quality dimensions. The paper also presents an empirical test of the proposed model of quality dimensions and their relationships. Results are presented of two experiments with 187 and 124 business students respectively, that were designed to test a set of hypotheses generated from the proposed model. The results largely support the proposed model and have implications for both theory and practice of quality evaluation of conceptual modelling scripts.

[1]  Daniel L. Moody,et al.  Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions , 2005, Data Knowl. Eng..

[2]  Ron Weber,et al.  On the ontological expressiveness of information systems analysis and design grammars , 1993, Inf. Syst. J..

[3]  Marta Indulska,et al.  How do practitioners use conceptual modeling in practice? , 2006, Data Knowl. Eng..

[4]  John Hulland,et al.  Use of partial least squares (PLS) in strategic management research: a review of four recent studies , 1999 .

[5]  Ron Weber,et al.  An Ontological Model of an Information System , 1990, IEEE Trans. Software Eng..

[6]  Dale E. Zand,et al.  Theory of Change and the Effective Use of Management Science. , 1975 .

[7]  Ron Weber,et al.  Properties do not have Properties: Investigating a Questionable Conceptual Modelling Practice , 2003 .

[8]  D. Sandy Staples,et al.  Dimensions of Information Systems Success , 1999, Commun. Assoc. Inf. Syst..

[9]  Arne Sølvberg,et al.  Understanding quality in conceptual modeling , 1994, IEEE Software.

[10]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[11]  Izak Benbasat,et al.  The Relative Importance of Structural Constraints and Surface Semantics in Information Modeling , 1997, Inf. Syst..

[12]  Venkataraman Ramesh,et al.  Human Factors Research on Data Modeling: A Review of Prior Research, An Extended Framework and Future Research Directions , 2002, J. Database Manag..

[13]  I. Ajzen.  The theory of planned behavior , 1991 .

[14]  Susanne Patig,et al.  Zur Ausdrucksstärke der Stammdaten des Advanced Planning and Scheduling , 2004, Wirtschaftsinf..

[15]  Mario Piattini,et al.  Quality in Conceptual Modeling - New Research Directions , 2002, ER.

[16]  Ron Weber,et al.  On the deep structure of information systems , 1995, Inf. Syst. J..

[17]  Michael Rosemann,et al.  Measuring Process Modelling Success , 2002, ECIS.

[18]  Geert Poels,et al.  Measuring the Perceived Semantic Quality of Information Models , 2005, ER.

[19]  Reinhard Schütte,et al.  The Guidelines of Modeling - An Approach to Enhance the Quality in Information Models , 1998, ER.

[20]  Cheryl L. Dunn,et al.  Auditor efficiency and effectiveness with diagrammatic and linguistic conceptual model representations , 2001, Int. J. Account. Inf. Syst..

[21]  Peter Meso,et al.  Conceptualizing Systems for Understanding: An Empirical Test of Decomposition Principles in Object-Oriented Analysis , 2006, Inf. Syst. Res..

[22]  H. Winklhofer,et al.  Index Construction with Formative Indicators: An Alternative to Scale Development , 2001 .

[23]  Kenneth A. Bollen,et al.  Structural Equations with Latent Variables , 1989 .

[24]  Young-Gul Kim,et al.  Comparing data modeling formalisms , 1995, CACM.

[25]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[26]  E. Tansley,et al.  Using ontology to validate conceptual models , 2003, CACM.

[27]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[28]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[29]  Peter B. Seddon.  A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[30]  Peter B. Seddon,et al.  A Partial Test and Development of the DeLone and McLean Model of IS Success , 1994, ICIS.

[31]  Cheryl L. Dunn,et al.  Perceived semantic expressiveness of accounting systems and task accuracy effects , 2000, Int. J. Account. Inf. Syst..

[32]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[33]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[34]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[35]  Fred D. Davis.  A technology acceptance model for empirically testing new end-user information systems : theory and results , 1985 .

[36]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[37]  Antoni Olivé,et al.  Conceptual Schema-Centric Development: A Grand Challenge for Information Systems Research , 2005, CAiSE.

[38]  John Krogstie,et al.  Quality of Interactive Models , 2002, ER.

[39]  Diane M. Strong,et al.  Beyond Accuracy: What Data Quality Means to Data Consumers , 1996, J. Manag. Inf. Syst..

[40]  Jeffrey Parsons,et al.  What do the pictures mean? Guidelines for experimental evaluation of representation fidelity in diagrammatical conceptual modeling techniques , 2005, Data Knowl. Eng..

[41]  Maarten Gelderman,et al.  The relation between user satisfaction, usage of information systems and performance , 1998, Inf. Manag..

[42]  Guttorm Sindre,et al.  Evaluating the Quality of Process Models: Empirical Testing of a Quality Framework , 2002, ER.

[43]  Barbara Wixom,et al.  Antecedents of Information and System Quality: An Empirical Examination Within the Context of Data Warehousing , 2005, J. Manag. Inf. Syst..

[44]  K. Bollen.  Multiple indicators: Internal consistency or no necessary relationship? , 1984 .

[45]  Cheryl L. Dunn,et al.  An Investigation of Localization as an Element of Cognitive Fit in Accounting Model Representations , 2001, Decis. Sci..

[46]  Ron Weber,et al.  Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests , 2001, Inf. Syst. Res..

[47]  Graeme G. Shanks,et al.  Improving the quality of data models: empirical validation of a quality management framework , 2003, Inf. Syst..

[48]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[49]  Andrew Gemino,et al.  Complexity and clarity in conceptual modeling: Comparison of mandatory and optional properties , 2005, Data Knowl. Eng..

[50]  Keng Siau,et al.  Informational and Computational Equivalence in Comparing Information Modeling Methods , 2004, J. Database Manag..

[51]  Wynne W. Chin.  The partial least squares approach for structural equation modeling. , 1998 .

[52]  George A. Marcoulides,et al.  Modern methods for business research , 1998 .

[53]  Daniel L. Moody,et al.  Evaluating the quality of information models: empirical testing of a conceptual model quality framework , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[54]  Ron Weber,et al.  Understanding relationships with attributes in entity-relationship diagrams , 1999, ICIS.

[55]  Izak Benbasat,et al.  Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation , 1991, Inf. Syst. Res..

[56]  Albert Endres,et al.  A handbook of software and systems engineering - empirical observations, laws and theories , 2003, The Fraunhofer IESE series on software engineering.

[57]  Terry A. Halpin.  Comparing Metamodels for ER, ORM and UML Data Models , 2004, Advanced Topics in Database Research, Vol. 3.

[58]  John Krogstie,et al.  Process models representing knowledge for action: a revised quality framework , 2006, Eur. J. Inf. Syst..

[59]  Samira Si-Said Cherfi,et al.  Conceptual Modeling Quality - From EER to UML Schemas Evaluation , 2002, ER.

[60]  Ron Weber,et al.  Research Commentary: Information Systems and Conceptual Modeling - A Research Agenda , 2002, Inf. Syst. Res..

[61]  Gerhard O. Mensch,et al.  TIMS Studies in the management sciences : Burton V. Dean and Joel L. Goldhar (eds.), Management of Research and Innovation, North-Holland Publishing Company, Amsterdam, Vol. 15, 1980, 300 pp. , 1983 .

[62]  Walter F. Tichy,et al.  Proceedings 25th International Conference on Software Engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[63]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[64]  Barbara D. Klein,et al.  User evaluations of IS as surrogates for objective performance , 2000, Inf. Manag..

[65]  John Krogstie,et al.  Defining quality aspects for conceptual models , 1995, ISCO.

[66]  Someswar Kesh,et al.  Evaluating the quality of entity relationship models , 1995, Inf. Softw. Technol..

[67]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..