EMPIRICAL COMPARISON OF THE MULTIDIMENSIONAL MODELS OF ITEM RESPONSE THEORY IN E-COMMERCE

The measurement of latent traits within the organizational field such as quality, effectiveness and learning has been conducted in several formats using a wide variety of quantitative methods, including Item Response Theory that consistently increased in organizational studies. The purpose of this article is to compare the hierarchical and non-hierarchical structures of three multidimensional models of Item Response Theory, based on the interface quality measurement in e-commerce sites. We compared the multiple unidimensional, compensatory multidimensional and bifactorial models, and also elaborated and applied 75 items in a sample of 441 e-commerce websites. As a result, we conducted a discussion of the latent construct, the quality in e-commerce and its multidimensional configuration to adjust and compare three multidimensional models.

[1]  Charles A. McMellon,et al.  Exploring the determinants of retail service quality on the Internet , 2004 .

[2]  F. Drasgow,et al.  Theoretical, Statistical, and Substantive Issues in the Assessment of Construct Dimensionality , 2012 .

[3]  Veda C. Storey,et al.  A Framework for Analysis of Data Quality Research , 1995, IEEE Trans. Knowl. Data Eng..

[4]  A. Rupp,et al.  Performance of the S − χ2 Statistic for Full-Information Bifactor Models , 2011 .

[5]  Lucila Maria de Souza Campos,et al.  Measuring environmental management disclosure in industries in Brazil with Item Response Theory , 2013 .

[6]  Gregg C. Vanderheiden,et al.  Web Content Accessibility Guidelines (WCAG) 2.0 , 2008 .

[7]  William R. King,et al.  ASSESSING INFORMATION SYSTEM VALUE: AN EXPERIMENTAL STUDY , 1983 .

[8]  Daniel A. Newman,et al.  Never Say ‘‘Always’’? Extreme Item Wording Effects on Scalar Invariance and Item Response Curves , 2010 .

[9]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[10]  Terry A. Ackerman.  A Didactic Explanation of Item Bias, Item Impact, and Item Validity from a Multidimensional Perspective , 1992 .

[11]  Richard Vidgen,et al.  European Conference on Information Systems ( ECIS ) 2000 WebQual : An Exploration of Website Quality , 2017 .

[12]  William J. Kettinger,et al.  Perceived Service Quality and User Satisfaction with the Information Services Function , 1994 .

[13]  Robert D Gibbons,et al.  On the psychometric validity of the domains of the PDSQ: an illustration of the bi-factor item response theory model. , 2009, Journal of psychiatric research.

[14]  Mark D. Reckase,et al.  The Discriminating Power of Items That Measure More Than One Dimension , 1991 .

[15]  Prashant C. Palvia,et al.  Developing and validating an instrument for measuring user-perceived web quality , 2002, Inf. Manag..

[16]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[17]  Francesco Bartolucci,et al.  Dimensionality of the Latent Structure and Item Selection Via Latent Class Multidimensional IRT Models , 2012, Psychometrika.

[18]  Abby Day,et al.  A model for monitoring Web site effectiveness , 1997, Internet Res..

[19]  Geoffrey J. McLachlan,et al.  A mixture model-based approach to the clustering of microarray expression data , 2002, Bioinform..

[20]  R. Hambleton,et al.  Item Response Theory: Principles and Applications , 1984 .

[21]  Kazimierz Worwa,et al.  Quality of Web-based information systems , 2010 .

[22]  Kevin Ibeh,et al.  Customers' Perception of Electronic Service Delivery in the UK Retail Banking Sector , 2006 .

[23]  C. Bernini,et al.  Investigating heterogeneity in residents’ attitudes toward tourism with an IRT multidimensional approach , 2015 .

[24]  Blanca Hernández Ortega,et al.  International Journal of Information Management , 2022 .

[25]  Terry A. Ackerman.  Using multidimensional item response theory to understand what items and tests are measuring , 1994 .

[26]  Gillian Rice,et al.  Multidimensional latent traits of perceived organizational innovation: Differences between Thai and Egyptian employees , 2008 .

[27]  A. Meade,et al.  Examining Question and Context Effects in Organization Survey Data Using Item Response Theory , 2009 .

[28]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[29]  Klara Nelson.  Global information systems quality: key issues and challenges , 1996 .

[30]  Oreste Signore,et al.  A comprehensive model for Web sites quality , 2005, Seventh IEEE International Symposium on Web Site Evolution.

[31]  Peter M. Bentler,et al.  Exploratory Bi-Factor Analysis , 2011, Psychometrika.

[32]  C. Flavián,et al.  Perceived e‐service quality (PeSQ) , 2007 .

[33]  Dalton Francisco de Andrade,et al.  Measuring web usability using item response theory: Principles, features and opportunities , 2011, Interact. Comput..

[34]  Susan E. Embretson,et al.  A multidimensional latent trait model for measuring learning and change , 1991 .

[35]  Steven M Downing,et al.  Item response theory: applications of modern test theory in medical education , 2003, Medical education.

[36]  C. Flavián,et al.  Perceived e-service quality ( PeSQ ) Measurement validation and effects on consumer satisfaction and web site loyalty , 2012 .

[37]  V. Zeithaml,et al.  E-S-QUAL A Multiple-Item Scale for Assessing Electronic Service Quality , 2004 .

[38]  Thong Ngee Goh,et al.  Quality dimensions of Internet search engines , 1998, J. Inf. Sci..

[39]  José G. Dias,et al.  Model Selection for the Binary Latent Class Model: A Monte Carlo Simulation , 2006, Data Science and Classification.

[40]  Klaas Sijtsma,et al.  Introduction to the measurement of psychological attributes , 2011 .

[41]  Gustavo Rossi,et al.  Specifying Quality Characteristics and Attributes for Websites , 2001, Web Engineering.

[42]  Minjoon Jun,et al.  Internet users' perceptions of online service quality: a comparison of online buyers and information searchers , 2003 .

[43]  Johannes Hartig,et al.  Multidimensional IRT models for the assessment of competencies , 2009 .

[44]  K. Holzinger,et al.  The Bi-factor method , 1937 .

[45]  Klaus Opwis,et al.  Mental models for web objects: Where do users expect to find the most frequent objects in online shops, news portals, and company web pages? , 2010, Interact. Comput..

[46]  Kicka Lindroos,et al.  Use quality and the World Wide Web , 1997, Inf. Softw. Technol..

[47]  Mei Wang,et al.  Evaluating the usability of Web-based learning tools , 2002, J. Educ. Technol. Soc..

[48]  Naveen Donthu,et al.  Developing a Scale to Measure the Perceived Quality of An Internet Shopping Site ( SITEQUAL ) , 2005 .

[49]  Nathan T. Carter,et al.  Using Mixed-Model Item Response Theory to Analyze Organizational Survey Responses: An Illustration Using the Job Descriptive Index , 2011 .

[50]  Robert A. Forsyth,et al.  An Examination of the Characteristics of Unidimensional IRT Parameter Estimates Derived From Two-Dimensional Data , 1985 .

[51]  Adel M. Aladwani.  Implications of some of the recent improvement philosophies for the management of the information systems organization , 1999 .

[52]  Rian van der Merwe,et al.  A framework and methodology for evaluating e-commerce Web sites , 2003, Internet Res..

[53]  Utilização da teoria da resposta ao item na produção de indicadores sócio-econômicos , 2005 .

[54]  Dominique P. Rauch,et al.  Multiple-choice versus open-ended response formats of reading test items: A two-dimensional IRT analysis , 2010 .

[55]  Olivia R. Liu Sheng,et al.  e-SELFQUAL: A scale for measuring online self-service quality , 2011 .

[56]  Michael F. Green,et al.  Bifactor and item response theory analyses of interviewer report scales of cognitive impairment in schizophrenia. , 2011, Psychological assessment.

[57]  David M. Lahuis,et al.  An Examination of Item Response Theory Item Fit Indices for the Graded Response Model , 2011 .

[58]  W. Revelle.  psych: Procedures for Personality and Psychological Research , 2017 .

[59]  E. Muraki,et al.  Full-Information Item Factor Analysis , 1988 .

[60]  T. C. Oshima,et al.  Multidimensionality and Item Bias in Item Response Theory , 1992 .

[61]  Mark Wilson,et al.  Using the concept of a measurement system to characterize measurement models used in psychometrics , 2013 .

[62]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[63]  R. Philip Chalmers,et al.  mirt: A Multidimensional Item Response Theory Package for the R Environment , 2012 .

[64]  H. Akaike.  Maximum likelihood identification of Gaussian autoregressive moving average models , 1973 .

[65]  M. Reckase.  Multidimensional Item Response Theory , 2009 .

[66]  Shailey Minocha,et al.  Usability beyond the website: An empirically-grounded e-commerce evaluation instrument for the total customer experience , 2006, Behav. Inf. Technol..

[67]  Timon C. Du,et al.  A study of the service quality of general portals , 2009, Inf. Manag..

[68]  Frederic Marimon Viadiu,et al.  Functional quality and hedonic quality: A study of the dimensions of e-service quality in online travel agencies , 2012, Inf. Manag..

[69]  Moshe Zviran,et al.  Information systems effectiveness: research designs for causal inference , 1994, Inf. Syst. J..

[70]  Mark D. Reckase,et al.  The Difficulty of Test Items That Measure More Than One Ability , 1985 .

[71]  Kori Inkpen Quinn,et al.  Left‐handed scrolling for pen‐based devices , 2006, Int. J. Hum. Comput. Interact..

[72]  M. Fassnacht,et al.  Quality of Electronic Services , 2006 .

[73]  Rafael Tezza,et al.  Relevance and advantages of using the item response theory , 2013 .

[75]  Donald Hedeker,et al.  Full-Information Item Bifactor Analysis of Graded Response Data , 2007 .

[76]  Shari Lawrence Pfleeger,et al.  Software Quality: The Elusive Target , 1996, IEEE Softw..

[77]  Terry A. Ackerman.  The Use of Unidimensional Parameter Estimates of Multidimensional Items in Adaptive Testing , 1991 .

[78]  Representation of Competencies in Multidimensional IRT Models with Within-Item and Between-Item Multidimensionality , 2008 .

[79]  Renata Zilse,et al.  Ergodesign e arquitetura da informação: trabalhando com o usuário , 2010 .

[80]  Robert J. McQueen,et al.  Developing an Evaluation Instrument for e-Commerce Web Sites from the First-Time Buyer’s Viewpoint , 2004 .

[81]  Roy Levy,et al.  Posterior Predictive Model Checking for Conjunctive Multidimensionality in Item Response Theory , 2011 .

[82]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[83]  B. Muthén,et al.  Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study , 2007 .

[84]  Norman F. Schneidewind,et al.  Methodology For Validating Software Metrics , 1992, IEEE Trans. Software Eng..

[85]  Ramayya Krishnan,et al.  Assessing data quality in accounting information systems , 1998, CACM.

[86]  Steven P. Reise,et al.  The role of the bifactor model in resolving dimensionality issues in health outcomes measures , 2007, Quality of Life Research.

[87]  Daniel A. Newman,et al.  Using Mixed-Measurement Item Response Theory With Covariates (MM-IRT-C) to Ascertain Observed and Unobserved Measurement Equivalence , 2011 .

[88]  D. Seo.  Application of the Bifactor Model to Computerized Adaptive Testing. , 2011 .

[89]  H. R. Rao,et al.  A multidimensional trust formation model in B-to-C e-commerce: a conceptual framework and content analyses of academia/practitioner perspectives , 2005, Decision Support Systems.

[90]  Gregory Camilli,et al.  A Conceptual Analysis of Differential Item Functioning in Terms of a Multidimensional Item Response Model , 1992 .

[91]  A. Parasuraman,et al.  Service quality delivery through web sites: A critical review of extant knowledge , 2002, Journal of the Academy of Marketing Science.