Do Ontological Guidelines Improve Understandability of Conceptual Models? A Meta-analysis of Empirical Work

Conceptual models are used to understand information systems application domains and to communicate about them in order to better understand system requirements. Conceptual models are required to represent relevant aspects of the modeled domain faithfully, and be understandable. The use of ontological theories has been proposed to guide the creation of effective models. Specifically, Bunge's ontology has been applied to guide the use of the ERM, the UML, and business process grammars. However, because the choice of ontology reflects beliefs, and model understanding involves human cognition, whether or not ontological guidance results in better models is an empirical question. Several empirical works studied this issue, differing in grammars, modeling aspects, empirical tasks, and measures. We report a meta-analysis of published empirical research about the impact of ontological guidance based on Bunge's model, and user understanding of conceptual models. The results support the proposition that ontological guidance can improve model understandability.

[1]  Salvatore T. March,et al.  The Effects of State-Based and Event-Based Data Representation on User Performance in Query Formulation Tasks , 2006, MIS Q..

[2]  Andrew Gemino,et al.  A framework for empirical evaluation of conceptual modeling techniques , 2004, Requirements Engineering.

[3]  Ron Weber,et al.  On the deep structure of information systems , 1995, Inf. Syst. J..

[4]  Peter Meso,et al.  Conceptualizing Systems for Understanding: An Empirical Test of Decomposition Principles in Object-Oriented Analysis , 2006, Inf. Syst. Res..

[5]  Frederico T. Fonseca,et al.  The double role of ontologies in information science research , 2007, J. Assoc. Inf. Sci. Technol..

[6]  Veda C. Storey,et al.  An ontological analysis of the relationship construct in conceptual modeling , 1999, TODS.

[7]  Ron Weber,et al.  Research Commentary: Information Systems and Conceptual Modeling - A Research Agenda , 2002, Inf. Syst. Res..

[8]  Andreas Rosenblad,et al.  Introduction to Meta‐Analysis by Michael Borenstein, Larry V. Hedges, Julian P.T. Higgins, Hannah R. Rothstein , 2009 .

[9]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[10]  R. Orwin.  A fail-safe N for effect size in meta-analysis. , 1983 .

[11]  Feng Li,et al.  An Introduction to Metaanalysis , 2005 .

[12]  Michael Rosemann,et al.  Integrated Process Modeling: An Ontological Evaluation , 2000, Inf. Syst..

[13]  Andrew Gemino,et al.  Complexity and clarity in conceptual modeling: Comparison of mandatory and optional properties , 2005, Data Knowl. Eng..

[14]  Marta Indulska,et al.  Do Ontological Deficiencies in Modeling Grammars Matter? , 2011, MIS Q..

[15]  Paul L. Bowen,et al.  Analysis of Competing Data Structures: Does Ontological Clarity Produce Better End User Query Performance , 2006, J. Assoc. Inf. Syst..

[16]  Iris Vessey,et al.  The Role of Cognitive Fit in the Relationship Between Software Comprehension and Modification , 2006, MIS Q..

[17]  R. Mayer.  Models for Understanding , 1989 .

[18]  Ron Weber,et al.  An Ontological Model of an Information System , 1990, IEEE Trans. Software Eng..

[19]  Iris Vessey,et al.  Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature* , 1991 .

[20]  Paul L. Bowen,et al.  An Empirical Investigation of End-User Query Development: The Effects of Improved Model Expressiveness vs. Complexity , 2009, Inf. Syst. Res..

[21]  Ron Weber,et al.  Is Use of Optional Attributes and Associations in Conceptual Modeling Always Problematic? Theory and Empirical Tests , 2012, ICIS.

[22]  Yair Wand,et al.  Theoretical foundations for conceptual modelling in information systems development , 1995, Decis. Support Syst..

[23]  Yair Wand,et al.  Guidelines for Designing Visual Ontologies to Support Knowledge Identification , 2011, MIS Q..

[24]  Bashar Nuseibeh,et al.  Requirements engineering: a roadmap , 2000, ICSE '00.

[25]  Peter Meso,et al.  The Effects of Decomposition Quality and Multiple Forms of Information on Novices' Understanding of a Domain from a Conceptual Model , 2008, J. Assoc. Inf. Syst..

[26]  Graeme G. Shanks,et al.  Representing part-whole relations in conceptual modeling: an empirical evaluation , 2008 .

[27]  Daniel L. Moody,et al.  Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions , 2005, Data Knowl. Eng..

[28]  Ron Weber,et al.  On the ontological expressiveness of information systems analysis and design grammars , 1993, Inf. Syst. J..

[29]  Joerg Evermann,et al.  Ontological Modeling Rules for UML: An Empirical Assessment , 2006, J. Comput. Inf. Syst..

[30]  R. Rosenthal,et al.  Meta-analysis: recent developments in quantitative methods for literature reviews. , 2001, Annual review of psychology.

[31]  Jeffrey Parsons,et al.  What do the pictures mean? Guidelines for experimental evaluation of representation fidelity in diagrammatical conceptual modeling techniques , 2005, Data Knowl. Eng..

[32]  M. Ross Quillian,et al.  Retrieval time from semantic memory , 1969 .

[33]  Andrew C. Gemino.  Empirical comparisons of system analysis modeling techniques , 1999 .

[34]  Ron Weber,et al.  Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests , 2001, Inf. Syst. Res..

[35]  Marlene Schommer.  Effects of beliefs about the nature of knowledge on comprehension. , 1990 .