Web Surveys Optimized for Smartphones: Are there Differences Between Computer and Smartphone Users?

This paper shows that computer users and smartphone users taking part in a web survey optimized for smartphones give responses of almost the same quality. Combining a design of one question in each page and innovative page navigation methods, we can get high quality data by both computer and smartphone users. The two groups of users are also compared with regard to their precisely measured item response times. The analysis shows that using a smartphone instead of a computer increases about 20% the geometric mean of item response times. The data analyzed in this paper were collected by a smartphone-friendly web survey. All question texts are short and the response buttons are large and easy to use. As a result, there are no significant interactions between smartphone use and either the length of the question or the age of the respondent. Thus, the longer response times among smartphone users should be explored in other causes, such as the likelihood of smartphone users being distracted by their environment.

[1]  Mick P. Couper,et al.  Mobile Web Survey Design: Scrolling versus Paging, SMS versus E-mail Invitations , 2014 .

[2]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[3]  S M Case,et al.  Relationships among Item Characteristics, Examine Characteristics, and Response Times on USMLE Step 1 , 2001, Academic medicine : journal of the Association of American Medical Colleges.

[4]  Ioannis Andreadis,et al.  To Clean or not to Clean? Improving the Quality of VAA Data , 2012 .

[5]  Aigul Mavletova,et al.  Data Quality in PC and Mobile Web Surveys , 2013 .

[6]  R. Tourangeau,et al.  Fast times and easy questions: the effects of age, experience and question complexity on web survey response times , 2008 .

[7]  Marika de Bruijne,et al.  Comparing Survey Results Obtained via Mobile Devices and Computers , 2013 .

[8]  J. Tukey,et al.  Variations of Box Plots , 1978 .

[9]  Carey E. Stapleton.  The Smart(Phone) Way to Collect Survey Data , 2013 .

[10]  Willem J. van der Linden,et al.  Using Response Times for Item Selection in Adaptive Testing , 2008 .

[11]  P. Schmidt,et al.  Limited-Dependent and Qualitative Variables in Econometrics. , 1984 .

[12]  Marek Fuchs,et al.  The Coverage Bias of Mobile Web Surveys Across European Countries , 2009 .

[13]  W. Greene,et al.  计量经济分析 = Econometric analysis , 2009 .

[14]  Ioannis Andreadis,et al.  Data quality and data cleaning , 2014 .

[15]  Ronald P. Carver,et al.  Reading Rate: Theory, Research, and Practical Implications. , 1992 .

[16]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[17]  Daniel J. Bauer.  Estimating Multilevel Linear Models as Structural Equation Models , 2003 .

[18]  Jeffrey M. Woodbridge.  Econometric Analysis of Cross Section and Panel Data , 2002 .

[19]  Michael W. Link,et al.  Comparison of Smartphone and Online Computer Survey Administration , 2014 .

[20]  F. Mosteller,et al.  Understanding robust and exploratory data analysis , 1985 .

[21]  Vera Toepoel,et al.  What Happens if You Offer a Mobile Option to Your Web Panel? Evidence From a Probability-Based Panel of Internet Users , 2014 .

[22]  Tobias Gummer,et al.  Explaining Interview Duration in Web Surveys , 2015 .

[23]  G. A. Marcoulides.  Multilevel Analysis Techniques and Applications , 2002 .

[24]  L. Corrado.  Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models , 2005 .

[25]  Annamaria Bianchi,et al.  Innovation and quality in web-based data collection , 2014 .

[26]  Patrick J Curran,et al.  Have Multilevel Models Been Structural Equation Models All Along? , 2003, Multivariate behavioral research.

[27]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[28]  J. Heckman.  Dummy Endogenous Variables in a Simultaneous Equation System , 1977 .

[29]  Ioannis Andreadis.  Comparison of Response Times between Desktop and Smartphone Users , 2015 .

[30]  Frauke Kreuter,et al.  Using paradata to explore item level response times in surveys , 2013 .

[31]  Ioannis Andreadis,et al.  Voting advice applications: a successful nexus between informatics and political science , 2013, BCI '13.

[32]  J. N. Bassili,et al.  Response-time measurement in survey research: A method for CATI and a new look at nonattitudes. , 1991 .