Measuring validity and reliability of website credibility factors in influencing user engagement questionnaire

Purpose The purpose of this paper is to show the importance of validity process in survey questionnaire instrument development. It is to confirm all items chosen are valid in the context of study, especially in assessing municipal website Web user behaviours. The quality of measurement also can be assured. Design/methodology/approach Several methods used to assess validity of the survey instrument in this study, such as face validity, content validity index (CVI) and construct validity. CVI assessment is widely used to examine each item in the survey instrument exact measures what is supposed to be measured. Prolific items used to evaluate web aesthetic, usability, navigation, content organization, interactivity and user engagement. Findings Several items were rephrased and simplified during face validity assessment, while three items from user engagement construct were excluded after the CVI assessment. Reliability and redundancy analysis had shown good results for each item and construct for both dependent and independent variables. None of the items in any construct need to be eliminated. Research limitations/implications The face validity, content validity and construct validity assessments are sufficient to assure the consistency of the conceptual framework, the content of items and overall measurement approach reflecting the setting or context of study which are web users. Practical implications The validated survey instrument provides practical guidelines for government municipal website owners on how to stimulate website user to keep engaged on the government municipal website. Identified website content that acts as credibility cues can be as object attention when planning the strategy of Web content management. Through proper design and attracting touch up, the credibility cues can attract website user and force the user to engage deeper on the government municipal website content, believe it as the main source of information and recognize it as authoritative organization. Originality/value The measurement model of this study that consists of formative construct of surface credibility, and user engagement is proposed to be tested in a different context of this study.

[1]  OliveiraMauro José de,et al.  Factors driving young users' engagement with Facebook , 2016 .

[2]  Anol Bhattacherjee,et al.  Influence Processes for Information Technology Acceptance: An Elaboration Likelihood Model , 2006, MIS Q..

[3]  C. I. Hovland,et al.  The Influence of Source Credibility on Communication Effectiveness , 1951 .

[4]  Allen I. Huffcutt,et al.  A Review and Evaluation of Exploratory Factor Analysis Practices in Organizational Research , 2003 .

[5]  Jacquelyn A. Burkell,et al.  Believe it or not: Factors influencing credibility on the Web , 2002, J. Assoc. Inf. Sci. Technol..

[6]  Hemant K. Jain,et al.  Success of Data Resource Management in Distributed Environments: An Empirical Investigation , 1998, MIS Q..

[7]  M. Lynn.  Determination and quantification of content validity. , 1986, Nursing research.

[8]  E. Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[9]  Scott B. MacKenzie,et al.  Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques , 2011, MIS Q..

[10]  J. Oluwatayo.  Validity and Reliability Issues in Educational Research , 2012 .

[11]  Zhibin Lin,et al.  Factors driving young users' engagement with Facebook: Evidence from Brazil , 2016, Comput. Hum. Behav..

[12]  Peter Cabrera-Nguyen.  Author Guidelines for Reporting Scale Development and Validation Results in the Journal of the Society for Social Work and Research , 2010, Journal of the Society for Social Work and Research.

[13]  Theresa A. Pardo,et al.  Benchmarking e-Government: A comparison of frameworks for computing e-Government index and ranking , 2011, Gov. Inf. Q..

[14]  Detmar W. Straub,et al.  Reconceptualizing System Usage: An Approach and Empirical Test , 2006, Inf. Syst. Res..

[15]  Andreas Monnier,et al.  Content Validity Index and Intra- and Inter-Rater Reliability of a New Muscle Strength/Endurance Test Battery for Swedish Soldiers , 2015, PloS one.

[16]  B. Gualano,et al.  Development of a Specific Anaerobic Field Test for Aerobic Gymnastics , 2015, PloS one.

[17]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[18]  Paul A. Cairns,et al.  An empirical evaluation of the User Engagement Scale (UES) in online news environments , 2015, Inf. Process. Manag..

[19]  Gabriella Kazai,et al.  Towards a science of user engagement (Position Paper) , 2011 .

[20]  Gilbert A. Churchill.  A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[21]  F Yaghmaie,et al.  Content validity and its estimation , 2003 .

[22]  Cheryl Tatano Beck,et al.  The content validity index: are you sure you know what's being reported? Critique and recommendations. , 2006, Research in nursing & health.

[23]  Paul Benjamin Lowry,et al.  Partial Least Squares (PLS) Structural Equation Modeling (SEM) for Building and Testing Behavioral Causal Theory: When to Choose It and How to Use It , 2014, IEEE Transactions on Professional Communication.