How Does Your Model Grow? An Empirical Investigation of the Effects of Ontological Clarity and Application Domain Size on Query Performance

Semantic data models provide a map of the components of an information system. The characteristics of these models affect their usefulness for various tasks (e.g., information retrieval). The quality of information retrieval has obvious important consequences, both economic and otherwise. Traditionally, data base designers have produced parsimonious logical data models. In spite of their increased size, ontologically clearer conceptual models have been shown to facilitate better performance for both problem solving and information retrieval tasks in experimental settings. The experiments producing evidence of enhanced performance for ontologically clearer models have, however, used application domains of modest size. Data models in organizational settings are likely to be substantially larger than those used in these experiments. This research used an experiment to investigate whether the benefits of improved information retrieval performance associated with ontologically clearer models are robust as the size of the application domains increase. The experiment used an application domain of approximately twice the size as tested in prior experiments. The results indicate that, relative to the users of the parsimonious implementation, end users of the ontologically clearer implementation made significantly more semantic errors, took significantly more time to compose their queries, and were significantly less confident in the accuracy of their queries.

[1]  E. Tansley,et al.  Using ontology to validate conceptual models , 2003, CACM.

[2]  Keng Siau,et al.  Effects of query complexity and learning on novice user query performance with conceptual and logical database interfaces , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[3]  Keng Siau,et al.  User-Database Interface: The Effect of Abstraction Levels on Query Performance , 1993, MIS Q..

[4]  H. Simon,et al.  Models of Man. , 1957 .

[5]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[6]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[7]  H. Akaike A new look at the statistical model identification , 1974 .

[8]  Sangkyu Rho,et al.  An analysis of semantic overload in database access systems using multi-table query formulation , 1997 .

[9]  R. Wood Task complexity: Definition of the construct , 1986 .

[10]  David A. Bradbard,et al.  The Effects of Relational and Entity-Relationship Data Models on Query Performance of End Users , 1989, Int. J. Man Mach. Stud..

[11]  Ron Weber,et al.  Conceptual Modeling and Ontology: Possibilities and Pitfalls , 2003, J. Database Manag..

[12]  Norman E. Fenton,et al.  A Critique of Software Defect Prediction Models , 1999, IEEE Trans. Software Eng..

[13]  Micaela Serra,et al.  An object-oriented layered approach to interfaces for hardware/software codesign of embedded systems , 1998, Proceedings of the Thirty-First Hawaii International Conference on System Sciences.

[14]  Ron Weber,et al.  Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests , 2001, Inf. Syst. Res..

[15]  Ron Weber,et al.  On the ontological expressiveness of information systems analysis and design grammars , 1993, Inf. Syst. J..

[16]  A. Faye Borthick,et al.  A Model for and the Effects of Information Request Ambiguity on End-user Query Performance , 2001, ICIS.

[17]  Bernard C. Y. Tan,et al.  Three important determinants of user performance for database retrieval , 1999, Int. J. Hum. Comput. Stud..

[18]  Brenda L. Killingsworth,et al.  Is a map more than a picture? the role of SDSS technology, subject characteristics, and problem complexity on map reading and problem solving 1 , 2000 .

[19]  P. Kline Models of man , 1986, Nature.

[20]  M. Lipow,et al.  Number of Faults per Line of Code , 1982, IEEE Transactions on Software Engineering.

[21]  Ron Weber,et al.  Understanding relationships with attributes in entity-relationship diagrams , 1999, ICIS.

[22]  Donald R. Jones,et al.  The effects of information request ambiguity and construct incongruence on query development , 2001, Decis. Support Syst..

[23]  Robert Balzer An Implementation Methodology for Semantic Data Base Models , 1979, ER.

[24]  Bin Yu,et al.  Model Selection and the Principle of Minimum Description Length , 2001 .

[25]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[26]  Paul L. Bowen,et al.  Analysis of Competing Data Structures: Does Ontological Clarity Produce Better End User Query Performance , 2006, J. Assoc. Inf. Syst..

[27]  Yanchun Zhang,et al.  An analytical evaluation of NIAM'S grammar for conceptual schema diagrams , 1996, Inf. Syst. J..

[28]  D. Campbell Task Complexity: A Review and Analysis , 1988 .

[29]  Kil-Soo Suh,et al.  A Comparison of Linear Keyword and Restricted Natural Language Data Base Interfaces for Novice Users , 1992, Inf. Syst. Res..

[30]  William C. Perkins,et al.  Spatial decision support systems: An overview of technology and a test of efficacy , 1995, Decis. Support Syst..

[31]  Young-Gul Kim,et al.  Comparing data modeling formalisms , 1995, CACM.

[32]  Veda C. Storey,et al.  An ontological analysis of the relationship construct in conceptual modeling , 1999, TODS.

[33]  Ron Weber,et al.  Ontological foundations of information systems , 1997 .

[34]  Shamkant B. Navathe,et al.  An abbreviated concept-based query language and its exploratory evaluation , 2002, J. Syst. Softw..

[35]  Fumio Akiyama,et al.  An Example of Software System Debugging , 1971, IFIP Congress.