An empirical and comparative analysis of E-government performance measurement models: Model selection via explanation, prediction, and parsimony

Abstract Driven by the growing importance of the digital provision of government services (e-government), recent research has sought to develop and test conceptual models of citizen satisfaction and trust with these services. Yet, there remains little agreement on how to optimally model these relationships with regards to the somewhat divergent goals of explanation and prediction of citizen trust. In this paper, we test two prominent modeling paradigms of the e-government satisfaction-trust relationship: the “service quality” model and the “expectancy-disconfirmation” model. We compare several variations of these models for their in-sample explanatory abilities, out-of-sample predictive abilities, and parsimony. To test the models, we examine a pooled, cross-agency sample of survey data measuring citizens' experiences with and perceptions of three important and widely accessed U.S. federal e-government services—the webpages of the Social Security Administration, the Internal Revenue Service, and the U.S. Census Bureau. Our findings suggest that while the expectancy-disconfirmation paradigm performs well in explanation, a parsimonious model with an “overall quality-satisfaction-trust” link is best suited for predicting trust. In addition, the service quality paradigm offers the best compromise between predictive accuracy and explanatory power. These findings offer new insights for academic researchers, government agencies, and practitioners, especially those deciding upon an empirical model to adopt to measure e-government satisfaction and its impact upon citizen trust.

[1]  William Rand,et al.  Building Agent-Based Decision Support Systems for Word-of-Mouth Programs: A Freemium Application , 2016 .

[2]  D. Hambrick THE FIELD OF MANAGEMENT'S DEVOTION TO THEORY: TOO MUCH OF A GOOD THING? , 2007 .

[3]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[4]  France Bélanger,et al.  Digitizing Government Interactions with Constituents: An Historical Review of E-Government Research in Information Systems , 2012, J. Assoc. Inf. Syst..

[5]  A. Woodside Moving beyond multiple regression analysis to algorithms: Calling for adoption of a paradigm shift from symmetric to asymmetric thinking in data analysis and crafting theory , 2013 .

[6]  Dianne Cyr,et al.  Modeling Web Site Design Across Cultures: Relationships to Trust, Satisfaction, and E-Loyalty , 2008, J. Manag. Inf. Syst..

[7]  Shan Ling Pan,et al.  Advancing Public Trust Relationships in Electronic Government: The Singapore E-Filing Journey , 2012, Inf. Syst. Res..

[8]  Pratyush Nidhi Sharma,et al.  Model Selection in Information Systems Research Using Partial Least Squares Based Structural Equation Modeling , 2012, ICIS.

[9]  D. G. Morrison,et al.  Do We Really Need Multiple-Item Measures in Service Research? , 2001 .

[10]  Herbert A. Simon,et al.  Simplicity, Inference and Modelling: Science seeks parsimony, not simplicity: searching for pattern in phenomena , 2002 .

[11]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[12]  P. M. Podsakoff,et al.  Self-Reports in Organizational Research: Problems and Prospects , 1986 .

[13]  Benito E. Flores,et al.  A pragmatic view of accuracy measurement in forecasting , 1986 .

[14]  Li Jiang,et al.  Trust and Electronic Government Success: An Empirical Study , 2008, J. Manag. Inf. Syst..

[15]  Detmar W. Straub,et al.  Examining Trust in Information Technology Artifacts: The Effects of System Quality and Culture , 2008, J. Manag. Inf. Syst..

[16]  Galit Shmueli,et al.  Research Commentary - Too Big to Fail: Large Samples and the p-Value Problem , 2013, Inf. Syst. Res..

[17]  Detmar W. Straub,et al.  Structural Equation Modeling and Regression: Guidelines for Research Practice , 2000, Commun. Assoc. Inf. Syst..

[18]  Spyros Makridakis,et al.  Accuracy measures: theoretical and practical concerns☆ , 1993 .

[19]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[20]  A. Parasuraman,et al.  SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. , 1988 .

[21]  Hock-Hai Teo,et al.  Conceptualizing and Testing a Social Cognitive Model of the Digital Divide , 2011, Inf. Syst. Res..

[22]  Jay I. Myung,et al.  Model Comparison Methods , 2004, Numerical Computer Methods, Part D.

[23]  H. Chernoff,et al.  Why significant variables aren’t automatically good predictors , 2015, Proceedings of the National Academy of Sciences.

[24]  A. Vedlitz,et al.  Explaining Popular Trust in the Department of Homeland Security , 2013 .

[25]  Oliver James,et al.  Evaluating the Expectations Disconfirmation and Expectations Anchoring Approaches to Citizen Satisfaction with Local Public Services , 2007 .

[26]  Mayuram S. Krishnan,et al.  Research Note - Information Technology, Customer Satisfaction, and Profit: Theory and Evidence , 2016, Inf. Syst. Res..

[27]  Forrest V. Morgeson Comparing determinants of website satisfaction and loyalty across the e-government and e-business domains , 2011, Electron. Gov. an Int. J..

[28]  Van Ryzin,et al.  Expectations, performance, and citizen satisfaction with urban services , 2004 .

[29]  Gregory A. Porumbescu,et al.  Linking public sector social media and e-government website use to trust in government , 2016, Gov. Inf. Q..

[30]  C. Mitchell Dayton,et al.  Journal of Modern Applied Statistical Methods Invited Articles Model Comparisons Using Information Measures , 2022 .

[31]  Lars Bergkvist,et al.  Appropriate use of single-item measures is here to stay , 2015 .

[32]  Kar Yan Tam,et al.  The Effects of Post-Adoption Beliefs on the Expectation-Confirmation Model for Information Technology Continuance , 2006, Int. J. Hum. Comput. Stud..

[33]  I. J. Myung,et al.  The Importance of Complexity in Model Selection. , 2000, Journal of mathematical psychology.

[34]  J. Kampen,et al.  Potential for comparative public opinion research in public administration , 2005 .

[35]  Narayan Ramasubbu,et al.  Designing Web Sites for Customer Loyalty Across Business Domains: A Multilevel Analysis , 2006, J. Manag. Inf. Syst..

[36]  Vishanth Weerakkody,et al.  The University of Bradford Institutional Repository , 2022 .

[37]  Kristin M. Bakke,et al.  The perils of policy by p-value: Predicting civil conflicts , 2010 .

[38]  Richard T. Vidgen,et al.  Interactive E-Government: Evaluating the Web Site of the UK Inland Revenue , 2004, J. Electron. Commer. Organ..

[39]  Ken Aho,et al.  Model selection for ecologists: the worldviews of AIC and BIC. , 2014, Ecology.

[40]  Mark Vandenbosch,et al.  Research Report: Richness Versus Parsimony in Modeling Technology Adoption Decisions - Understanding Merchant Adoption of a Smart Card-Based Payment System , 2001, Inf. Syst. Res..

[41]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[42]  Michael D. Smith,et al.  Gaining Trust Through Online Privacy Protection: Self-Regulation, Mandatory Standards, or Caveat Emptor , 2007, J. Manag. Inf. Syst..

[43]  Gregory A. Porumbescu,et al.  Internet, Trust in Government, and Citizen Compliance , 2014 .

[44]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[45]  M. J. Moon,et al.  Linking Citizen Satisfaction with E-Government and Trust in Government , 2004 .

[46]  Shaun Pather,et al.  The e-Government evaluation challenge: A South African Batho Pele-aligned service quality approach , 2011, Gov. Inf. Q..

[47]  R. Rosenthal The file drawer problem and tolerance for null results , 1979 .

[48]  Pratyush Nidhi Sharma,et al.  PLS-Based Model Selection: The Role of Alternative Explanations in Information Systems Research , 2019, J. Assoc. Inf. Syst..

[49]  Arun Rai,et al.  Predictive Validity and Formative Measurement in Structural Equation Modeling: Embracing Practical Relevance , 2013, ICIS.

[50]  A. Parasuraman,et al.  A Conceptual Model of Service Quality and Its Implications for Future Research , 1985 .

[51]  J. Rossiter,et al.  The Predictive Validity of Multiple-Item versus Single-Item Measures of the Same Constructs , 2007 .

[52]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[53]  Jeffrey R. Edwards,et al.  Reconsidering Theoretical Progress in Organizational and Management Research , 2010 .

[54]  Qing Hu,et al.  Assimilation of Enterprise Systems: The Effect of Institutional Pressures and the Mediating Role of Top Management , 2007, MIS Q..

[55]  Forrest V. Morgeson Expectations, Disconfirmation, and Citizen Satisfaction with the US Federal Government: Testing and Expanding the Model , 2013 .

[56]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[57]  H. G. Gauch,et al.  Predictive and postdictive success of statistical analyses of yield trials , 1988, Theoretical and Applied Genetics.

[58]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[59]  A. Bowling,et al.  Just One Question: If One Question Works, Why Ask Several? Single Compared with Multi- Item Measures , 2005 .

[60]  Sunil Mithas,et al.  How Information Technology Strategy and Investments Influence Firm Performance: Conjecture and Empirical Evidence , 2016, MIS Q..

[61]  Marko Sarstedt,et al.  Partial least squares structural equation modeling (PLS-SEM): An emerging tool in business research , 2014 .

[62]  J. C. Thomas,et al.  The Effect of Expectations and Expectancy Confirmation/Disconfirmation on Motorists’ Satisfaction with State Highways , 2011 .

[63]  Wynne W. Chin The partial least squares approach for structural equation modeling. , 1998 .

[64]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[65]  Ryan T. Wright,et al.  Controlling for Common Method Variance in PLS Analysis: The Measured Latent Marker Variable Approach , 2013 .

[66]  P. Goodwin,et al.  On the asymmetry of the symmetric MAPE , 1999 .

[67]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[68]  William J. Baumol,et al.  The Cost Disease: Why Computers Get Cheaper and Health Care Doesn't , 2012 .

[69]  Kar Yan Tam,et al.  Understanding Continued Information Technology Usage Behavior: A Comparison of Three Models in the Context of Mobile Internet , 2006, Decis. Support Syst..

[70]  Forrest V. Morgeson,et al.  Customer Satisfaction and Stock Prices: High Returns, Low Risk: , 2006 .

[71]  Jukka Ylitalo,et al.  PLS marker variable approach to diagnosing and controlling for method variance , 2011, ICIS.

[72]  J. Colquitt,et al.  TRENDS IN THEORY BUILDING AND THEORY TESTING: A FIVE-DECADE STUDY OF THE ACADEMY OF MANAGEMENT JOURNAL , 2007 .

[73]  P M Bentler,et al.  Choice of structural model via parsimony: a rationale based on precision. , 1989, Psychological bulletin.

[74]  Gregg G. Van Ryzin,et al.  Pieces of a Puzzle: Linking Government Performance, Citizen Satisfaction, and Trust , 2007 .

[75]  Sunil Mithas,et al.  Commentary - The Economic and Statistical Significance of Stock Returns on Customer Satisfaction , 2009, Mark. Sci..

[76]  Sunil Mithas,et al.  Modeling Heterogeneity in the Satisfaction, Loyalty Intention, and Shareholder Value Linkage: A Cross-Industry Analysis at the Customer and Firm Levels , 2016 .

[77]  Jinwei Cao,et al.  From knowledge sharing to firm performance: A predictive model comparison , 2016 .

[78]  Jake M. Hofman,et al.  Prediction and explanation in social systems , 2017, Science.

[79]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[80]  J. P. Wanous,et al.  Overall job satisfaction: how good are single-item measures? , 1997, The Journal of applied psychology.

[81]  G. V. Ryzin Testing the Expectancy Disconfirmation Model of Citizen Satisfaction with Local Government , 2006 .

[82]  The nature of doubly concrete constructs and how to identify them , 2016 .

[83]  Jooho Lee,et al.  E‐Participation, Transparency, and Trust in Local Government , 2012 .

[84]  Mary Tate,et al.  Comparing Out-of-Sample Predictive Ability of PLS, Covariance, and Regression Models , 2014, ICIS.

[85]  Galit Shmueli,et al.  To Explain or To Predict? , 2010, 1101.0891.

[86]  Gregoris Mentzas,et al.  Classification and synthesis of quality approaches in e-government services , 2007, Internet Res..

[87]  Jaeki Song,et al.  Dynamics of Trust Revision: Using Health Infomediaries , 2008, J. Manag. Inf. Syst..

[88]  Mark Keil,et al.  Why Software Projects Escalate: An Empirical Analysis and Test of Four Theoretical Models , 2000, MIS Q..

[89]  G. Kitagawa,et al.  Information Criteria and Statistical Modeling , 2007 .

[90]  Eric K. Clemons,et al.  Global Differences in Online Shopping Behavior: Understanding Factors Leading to Trust , 2016, J. Manag. Inf. Syst..

[91]  Michael Parent,et al.  Building Citizen Trust Through E-government , 2005, Gov. Inf. Q..

[92]  Fany Yuval,et al.  Managerial quality, administrative performance and trust in governance revisited: A follow‐up study of causality , 2003 .

[93]  T. Poister,et al.  Citizens, Accountability, and Service Satisfaction , 2006 .

[94]  Forrest V. Morgeson,et al.  Do they all perform alike? An examination of perceived performance, citizen satisfaction and trust with US federal agencies , 2011 .

[95]  Viswanath Venkatesh,et al.  Technology adoption decisions in the household: A seven‐model comparison , 2014, J. Assoc. Inf. Sci. Technol..

[96]  Sunil Mithas,et al.  Misplaced Trust? Exploring the Structure of the E-Government-Citizen Trust Relationship , 2011 .

[97]  Henry C. Lucas,et al.  How Channel Choice and Service Failure Influence Customer Satisfaction: The Case of Public Services , 2013, ICIS.

[98]  Mayuram S. Krishnan,et al.  Information Technology, Customer Satisfaction, and Profit: Theory and Evidence , 2016 .

[99]  Paul Jen-Hwa Hu,et al.  Information Technology Acceptance by Individual Professionals: A Model Comparison Approach , 2001, Decis. Sci..

[100]  Marc J. Hetherington,et al.  The Political Relevance of Political Trust , 1998, American Political Science Review.

[101]  Paul P. Tallon Does IT pay to focus? An analysis of IT business value under single and multi-focused business strategies , 2007, J. Strateg. Inf. Syst..

[102]  Eric W. K. Tsang Commentary - Assumptions, Explanation, and Prediction in Marketing Science: "It's the Findings, Stupid, Not the Assumptions" , 2009, Mark. Sci..

[103]  Jeff Peterson,et al.  Theory Pruning: Strategies to Reduce Our Dense Theoretical Landscape , 2010 .

[104]  Caroline J. Tolbert,et al.  The Effects of E-Government on Trust and Confidence in Government , 2003, DG.O.

[105]  Adamantios Diamantopoulos,et al.  Selecting single items to measure doubly concrete constructs: A cautionary tale , 2016 .

[106]  D. Ketchen A Primer on Partial Least Squares Structural Equation Modeling , 2013 .

[107]  Yogesh Kumar Dwivedi,et al.  An empirical validation of a unified model of electronic government adoption (UMEGA) , 2017, Gov. Inf. Q..

[108]  Kevin A. Clarke The Necessity of Being Comparative:Theory Confirmation in Quantitative Political Science , 2007 .

[109]  Galit Shmueli,et al.  The elephant in the room: Predictive performance of PLS models , 2016 .

[110]  Jing Wang,et al.  Continued usage intention of multifunctional friend networking services: A test of a dual-process model using Facebook , 2014, Decis. Support Syst..

[111]  Rudolf R. Sinkovics,et al.  The Use of Partial Least Squares Path Modeling in International Marketing , 2009 .

[112]  Paul Jen-Hwa Hu,et al.  Extending the two‐stage information systems continuance model: incorporating UTAUT predictors and the role of context , 2011, Inf. Syst. J..

[113]  R. Nuzzo How scientists fool themselves – and how they can stop , 2015, Nature.

[114]  Gerald F. Davis Editorial Essay , 2015 .

[115]  R. Loo A caveat on using single‐item versus multiple‐item scales , 2002 .

[116]  Heather Douglas Reintroducing Prediction to Explanation , 2009, Philosophy of Science.

[117]  Galit Shmueli,et al.  What is Predictive about Partial Least Squares ? , 2011 .

[118]  Andrea Everard,et al.  How Presentation Flaws Affect Perceived Site Quality, Trust, and Intention to Purchase from an Online Store , 2005, J. Manag. Inf. Syst..

[119]  Paul Benjamin Lowry,et al.  Explaining and Predicting the Impact of Branding Alliances and Web Site Quality on Initial Consumer Trust of E-Commerce Web Sites , 2007, J. Manag. Inf. Syst..

[120]  Paul P. Tallon A Process-Oriented Perspective on the Alignment of Information Technology and Business Strategy , 2007, J. Manag. Inf. Syst..

[121]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[122]  John R. Rossiter,et al.  The C-OAR-SE procedure for scale development in marketing , 2002 .

[123]  Gregoris Mentzas,et al.  e-GovQual: A multiple-item scale for assessing e-government service quality , 2012, Gov. Inf. Q..

[124]  Galit Shmueli,et al.  Predictive Analytics in Information Systems Research , 2010, MIS Q..