Data Triangulation in Action: Using Comment Analysis to Refine Web Quality Metrics

This paper focuses on the evaluation of quality perceptions from users of an electronic government website. As government organizations have begun to enhance transparency, communicate and interact with citizens via the Web, developing appropriate online services has demanded heightened understanding of the requirements of users and appropriate tailoring of solutions. The site examined in the paper enables the online submission of self-assessed tax returns in the UK. Survey data collected via the eQual instrument were used to statistically model the perceptions of site users. However, in addition to quantitative data, we also collected open comments from respondents, and it is these comments that provide the crux of this paper. Such comments, via data triangulation, provided much more insight into the perceptions of site quality than the statistical data alone. The results of the comment analysis both support the instrument and point to additional factors determining the perceptions of quality of such e-government services requiring attention in the instrument development. The paper rounds off with a conclusions and an agenda for future research in this area.

[1]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[2]  Richard T. Watson,et al.  Service Quality: A Measure of Information System Effectiveness , 1995, MIS Q..

[3]  Fred D. Davis.  User Acceptance of Information Technology: System Characteristics, User Perceptions and Behavioral Impacts , 1993, Int. J. Man Mach. Stud..

[4]  A. Parasuraman,et al.  The nature and determinants of customer expectations of service , 1993 .

[5]  Steven Cohen,et al.  The future of e-government: a project of potential trends and issues , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[6]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[7]  Richard T. Vidgen,et al.  Interactive E-Government: Evaluating the Web Site of the UK Inland Revenue , 2007, Int. J. Electron. Gov. Res..

[8]  Jakob Nielsen,et al.  User interface directions for the Web , 1999, CACM.

[9]  V. Grover,et al.  An assessment of survey research in POM: from constructs to theory , 1998 .

[10]  Richard T. Vidgen,et al.  Interactive E-Government: Evaluating the Web Site of the UK Inland Revenue , 2004, J. Electron. Commer. Organ..

[11]  A. Parasuraman,et al.  Delivering quality service : balancing customer perceptions and expectations , 1990 .

[12]  Douglas Holmes,et al.  eGov: E-Business Strategies for Government , 2001 .

[13]  Richard Y. Wang,et al.  A product perspective on total data quality management , 1998, CACM.

[14]  William J. Kettinger,et al.  Pragmatic Perspectives on the Measurement of Information Systems Service Quality Analysis with LISREL : An Appendix to Pragmatic Perspectives on the Measurement of Information Systems Service Quality , 2002 .

[15]  Leon A. Kappelman,et al.  Measuring Information Systems Service Quality: Concerns on the Use of the SERVQUAL Questionnaire , 1997, MIS Q..

[16]  Richard T. Vidgen,et al.  Measuring Web site quality improvements: a case study of the forum on strategic management knowledge exchange , 2003, Ind. Manag. Data Syst..

[17]  Richard T. Vidgen,et al.  Interactive e-government services: modelling user perceptions with eQual , 2004, Electron. Gov. an Int. J..

[18]  Abdullah Mat Zaid.  Measuring and monitoring service quality at Malaysia Airlines , 1995 .

[19]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[20]  William J. McIver,et al.  Guest Editors' Introduction: The Ongoing March Toward Digital Government , 2001 .

[21]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[22]  Carolyn Snyder,et al.  Web Site Usability: A Designer's Guide , 1997 .

[23]  Mary Jo Bitner,et al.  Evaluating service encounters: The effects of physical surroundings and employee responses. , 1990 .

[24]  Sean B. Eom,et al.  Designing effective cyber store user interface , 2002, Ind. Manag. Data Syst..

[25]  Stuart J. Barnes,et al.  Evaluating Alcohol Advice Web Sites: A Cross-Country Perspective , 2003 .

[26]  Jakob Nielsen,et al.  Designing web usability , 1999 .

[27]  Donald F. Norris,et al.  Electronic government at the grass roots: contemporary evidence and future trends , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[28]  Diane M. Strong,et al.  Data quality in context , 1997, CACM.

[29]  Richard T. Vidgen,et al.  An Integrative Approach to the Assessment of E-Commerce Quality , 2002, J. Electron. Commer. Res..

[30]  Richard T. Watson,et al.  Measuring Information Systems Service Quality: Concerns for a Complete Canvas , 1997, MIS Q..