An Assessment of Measurement Invariance between Online and Mail Surveys

One of the latest trends in marketing research is the increasing use of online surveys, which offer lower costs and faster responses. Yet, critics question whether data collected via online surveys are equivalent to data collected via traditional mail surveys. Since existing evidence from the comparison of Web-based and paper-and-pencil surveys is inconclusive, we empirically examine the equivalence of online and traditional mail surveys in a marketing context.

[1]  Barbara M. Byrne,et al.  Structural equation modeling with EQS : basic concepts, applications, and programming , 2000 .

[2]  Herschel Knapp,et al.  Using pencil and paper, Internet and touch-tone phones for self-administered surveys: does methodology matter? , 2003, Comput. Hum. Behav..

[3]  J. Stanton.  AN EMPIRICAL ASSESSMENT OF DATA COLLECTION USING THE INTERNET , 1998 .

[4]  John R. Hauser,et al.  Polyhedral Methods for Adaptive Choice-Based Conjoint Analysis , 2004 .

[5]  Mary Jo Bitner,et al.  Self-Service Technologies: Understanding Customer Satisfaction with Technology-Based Service Encounters , 2000 .

[6]  Rosalyn M. Metz.  Conducting Online Research: Undergraduate Preferences of Sources , 2006 .

[7]  B. Byrne,et al.  Structural Equation Modeling with EQS and EQS/Windows. , 1996 .

[8]  Paul E. Green,et al.  The Journal of Marketing Research: Its Initiation, Growth, and Knowledge Dissemination , 2003 .

[9]  Donna L. Hoffman,et al.  Measuring the Customer Experience in Online Environments: A Structural Modeling Approach , 2000 .

[10]  L. T. DeCarlo.  On the meaning and use of kurtosis. , 1997 .

[11]  John L. Smith,et al.  Using the Internet for psychological research: personality testing on the World Wide Web. , 1999, British journal of psychology.

[12]  Anthony G. Greenwald,et al.  Applications of Covariance Structure Modeling in Psychology : Cause for Concern ? , 2001 .

[13]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[14]  P. Bentler.  Comparative Fit Indices in Structural Models , 1990 .

[15]  S. Breckler.  Applications of covariance structure modeling in psychology: cause for concern? , 1990, Psychological bulletin.

[16]  W. Aquilino,et al.  A Comparison of Computer-Assisted and Paper-and-Pencil Self-Administered Questionnaires in a Survey on Smoking, Alcohol, and Drug Use , 1998 .

[17]  D. Dillman.  Mail and internet surveys , 1999 .

[18]  M Birgelenvan,et al.  Customer evaluations of after-sales service contact modes : an empirical analysis of national culture's consequences , 2001 .

[19]  K. Mardia.  Measures of multivariate skewness and kurtosis with applications , 1970 .

[20]  R. Vandenberg.  Toward a Further Understanding of and Improvement in Measurement Invariance Methods and Procedures , 2002 .

[21]  Frederick Wiseman,et al.  METHODOLOGICAL BIAS IN PUBLIC OPINION SURVEYS , 1972 .

[22]  A. Parasuraman,et al.  SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. , 1988 .

[23]  S. Baron,et al.  Online Surveys in Marketing Research , 2002 .

[24]  T. Micceri.  The unicorn, the normal curve, and other improbable creatures. , 1989 .

[25]  Eric J. Johnson,et al.  When Web Pages Influence Choice: Effects of Visual Primes on Experts and Novices , 2002 .

[26]  J. Epstein,et al.  Insuring sample equivalence across internet and paper-and-pencil assessments , 2001, Comput. Hum. Behav..

[27]  S. Kiesler,et al.  Response Effects in the Electronic Survey , 1986 .

[28]  S. West,et al.  The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. , 1996 .

[29]  R. Klassen,et al.  Experimental comparison of Web, electronic and mail survey technologies in operations management , 2001 .

[30]  Rohit Verma,et al.  Understanding Consumer Choices and Preferences in Transaction-Based e-Services , 2003 .

[31]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[32]  A. Satorra,et al.  Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: a Monte Carlo study. , 1991, The British journal of mathematical and statistical psychology.

[33]  S. Baron,et al.  ONLINE SURVEYS IN MARKETING RESEARCH: PROS AND CONS , 2002 .

[34]  Gilbert A. Churchill,et al.  Improving the measurement of service quality , 1993 .

[35]  Gerald Albaum,et al.  A Comparison of Response Characteristics from Web and Telephone Surveys , 2004 .

[36]  John G. Lynch,et al.  Wine Online: Search Costs Affect Competition on Price, Quality, and Distribution , 2000 .

[37]  Albert Satorra,et al.  A scaled difference chi-square test statistic for moment structure analysis , 1999 .

[38]  Mark D. Shermis,et al.  A Comparison of Survey Data Collected by Regular Mail and Electronic Mail Questionnaires , 1999 .

[39]  B. Byrne.  Book Review: Structural Equation Modeling with EQS and EQS/Windows: Basic Concepts, Applications, and Programming , 1994 .

[40]  R. Peterson,et al.  Measuring customer satisfaction: Fact and artifact , 1992 .

[41]  M. Couper.  A REVIEW OF ISSUES AND APPROACHES , 2000 .

[42]  Don A. Dillman,et al.  Development of a Standard E-Mail Methodology: Results of an Experiment , 1998 .

[43]  Milam W. Aiken,et al.  Flaming in electronic communication , 2004, Decis. Support Syst..

[44]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[45]  George R. Franke,et al.  Structural Equation Modeling with EQS and EQS/Windows , 1995 .

[46]  M. Traugott,et al.  Web survey design and administration. , 2001, Public opinion quarterly.

[47]  D. Aaker,et al.  Chapter 2 – Marketing research , 2004 .

[48]  P. Chisnall.  Mail and Internet Surveys: The Tailored Design Method , 2007, Journal of Advertising Research.

[49]  A. Satorra,et al.  Corrections to test statistics and standard errors in covariance structure analysis. , 1994 .

[50]  Y Kano,et al.  Can test statistics in covariance structure analysis be trusted? , 1992, Psychological bulletin.

[51]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[52]  Zeki Simsek,et al.  A Primer on Internet Organizational Surveys , 2001 .

[53]  Martin Wetzels,et al.  Customer evaluations of after-sales service contact modes : an empirical analysis of national culture's consequences , 2002 .

[54]  Muni S. Srivastava,et al.  A measure of skewness and kurtosis and a graphical method for assessing multivariate normality , 1984 .

[55]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .