Spatial maps in voting advice applications: The case for dynamic scale validation

Low-dimensional spatial representations of political preferences are a widespread feature of voting advice applications (VAAs). Currently, VAA spatial maps tend to be defined on the basis of a priori reasoning. This article argues that VAA spatial maps should be empirically validated to safeguard fundamental psychometric properties – in particular, unidimensionality and reliability. We suggest dynamic scale validation as a pragmatic method for improving measurement quality in VAA spatial maps. The basic logic of dynamic scale validation is to exploit early user data as a benchmark against which ex-ante defined maps can be evaluated. We draw on data from one of the most institutionalised VAA settings, Switzerland, to illustrate this dynamic approach to scale validation.

[1]  David W. Gerbing,et al.  An Updated Paradigm for Scale Development Incorporating Unidimensionality and Its Assessment , 1988 .

[2]  Klaas Sijtsma,et al.  Selection of Unidimensional Scales From a Multidimensional Item Bank in the Polytomous Mokken I RT Model , 1995 .

[3]  James Mitchell,et al.  The dimensionality of the Scottish political space , 2014 .

[4]  van der Ark,et al.  Mokken Scale Analysis in R , 2007 .

[5]  D. Apter,et al.  Ideology and discontent , 1966 .

[6]  Hanspeter Kriesi,et al.  Globalization and the transformation of the national political space: Six European countries compared , 2006 .

[7]  D. Bochsler.  Who Gains from Apparentments Under D'Hondt? , 2010 .

[8]  M. Germann,et al.  Assessing the psychometric properties of latent measures of ideology in VAA spatial maps , 2016 .

[9]  R. J. Mokken,et al.  A Theory and Procedure of Scale Analysis: With Applications in Political Research , 1971 .

[10]  Tom Louwerse,et al.  Design challenges in cross-national VAAs: the case of the EU Profiler , 2012 .

[11]  Klaas Sijtsma,et al.  Introduction to Nonparametric Item Response Theory , 2002 .

[12]  Drew A. Linzer,et al.  poLCA: An R Package for Polytomous Variable Latent Class Analysis , 2011 .

[13]  Jonathan.,et al.  Exploiting Smartvote Data for the Ideological Mapping of Swiss Political Parties , 2012 .

[14]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[15]  Stefaan Walgrave,et al.  Voting Aid Applications and the Effect of Statement Selection , 2009 .

[16]  M. Germann,et al.  Dynamic scale validation reloaded , 2016 .

[17]  W. H. Schuur,et al.  Mokken Scale Analysis: Between the Guttman Scale and Parametric Item Response Theory , 2003, Political Analysis.

[18]  Klaas Sijtsma,et al.  A Comparative Study of Test Data Dimensionality Assessment Procedures Under Nonparametric IRT Models , 2004 .

[19]  Michael Hermann,et al.  Atlas der politischen Landschaften : ein weltanschauliches Porträt der Schweiz , 2003 .

[20]  Fernando Mendez,et al.  Matching voters with political parties and candidates: an empirical test of four algorithms , 2012 .

[21]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[22]  Jan Fivaz,et al.  Impact of Voting Advice Applications (VAAs) on Voter Turnout and Their Potential Use for Civic Education , 2010 .

[23]  John Hattie,et al.  Methodology Review: Assessing Unidimensionality of Tests and ltenls , 1985 .

[24]  Klaas Sijtsma,et al.  On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha , 2008, Psychometrika.

[25]  Ioannis Andreadis,et al.  To Clean or not to Clean? Improving the Quality of VAA Data , 2012 .

[26]  Michael Boss.  Economic theory of democracy , 1974 .

[27]  L. Hooghe,et al.  Measuring party positions in Europe , 2015 .

[28]  L. Hooghe,et al.  Party Competition and European Integration in the East and West , 2006 .

[29]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[30]  Stephen Ansolabehere,et al.  The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting , 2008, American Political Science Review.

[31]  Dominik Hangartner,et al.  Comparing Candidates and Citizens in the Ideological Space , 2010 .

[32]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[33]  Klaas Sijtsma,et al.  A Latent Class Approach to Estimating Test-Score Reliability , 2011 .

[34]  Vasiliki Triga,et al.  'Neither agree, nor disagree': a critical analysis of the middle answer category in Voting Advice Applications , 2012 .

[35]  J. Spanje,et al.  Immigration, Europe and the ‘new’ cultural dimension , 2009 .

[36]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[37]  Kenneth Benoit,et al.  The dimensionality of political space: Epistemological and methodological considerations , 2012 .

[38]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[39]  Tom Louwerse,et al.  The design effects of voting advice applications: Comparing methods of calculating matches , 2013, Acta Politica.

[40]  van der Ark,et al.  New Developments in Mokken Scale Analysis in R , 2012 .

[41]  K. Gemenis.  Estimating parties’ policy positions through voting advice applications: Some methodological considerations , 2013 .

[42]  P. Converse.  The Nature of Belief Systems in Mass Publics , 2004 .

[43]  Tom Louwerse,et al.  The Design Effects of Voting Advice Applications: Comparing Methods of Calculating Results , 2011 .

[44]  D. Watson,et al.  Constructing validity: Basic issues in objective scale development , 1995 .

[45]  Christopher H. Achen.  Mass Political Attitudes and the Survey Response , 1975, American Political Science Review.

[46]  Jonathan Wheatley,et al.  Using VAAs to explore the dimensionality of the policy space: experiments from Brazil, Peru, Scotland and Cyprus , 2012 .

[47]  E. Davidov.  Measurement Equivalence of Nationalism and Constructive Patriotism in the ISSP: 34 Countries in a Comparative Perspective , 2009, Political Analysis.