Measuring agreement in ordered rating scales.

Ordered rating scales are one of the most frequently used question formats in large-scale surveys. Analysts of the responses to such questions often find themselves in need of describing the degree of agreement (concentration, consensus) of the answers to such questions. For that purpose they commonly use standard deviations of the response distributions, or measures based on these (such as the coefficient of consensus defined by Granberg and Holmberg, 1988), or the coefficient of variability, etc. This paper demonstrates that such measures are inappropriate for this purpose because they misrepresent what they are supposed to measure: the `peakedness' of a distribution. As an alternative a measure of agreement A is proposed. This measure is a weighted average of the degree of agreement that exists in the simple component parts – layers – into which any frequency distribution can be disaggregated, and for which agreement can be expressed in a straightforward and unequivocal way.

[1]  H. McClosky,et al.  Issue Conflict and Consensus among Party Leaders and Followers , 1960, American Political Science Review.

[2]  C. Eijk,et al.  Choosing Europe?: The European Electorate and National Politics in the Face of Union , 1996 .

[3]  Hendrik van Schuur.  Sructure in political beliefs. A new model for Stochastic Unfolding with Application to European Party Activists , 1984 .

[4]  P. Converse,et al.  Political Representation in France , 1986 .

[5]  A. Downs.  An Economic Theory of Democracy , 1957 .

[6]  C. Coombs.  A theory of data. , 1965, Psychology Review.

[7]  Eric R. A. N. Smith,et al.  PUBLIC OPINION AND CONGRESSIONAL REPRESENTATION , 1992 .

[8]  W. Brug.  Where's the party? : voters' perceptions of party positions , 1997 .

[9]  John D. Huber,et al.  Congruence between Citizens and Policymakers in Two Visions of Liberal Democracy , 1994, World Politics.

[10]  H. McClosky,et al.  Consensus and Ideology in American Politics , 1964, American Political Science Review.

[11]  D. Granberg,et al.  The Political System Matters: Social Psychology and Voting Behavior in Sweden and the United States , 1988 .

[12]  Cees van der Eijk,et al.  Policy preferences and party choice , 1999 .

[13]  D. Apter,et al.  Ideology and discontent , 1966 .

[14]  G. Pomper.  From Confusion to Clarity: Issues and American Voters, 1956–1968 , 1972, American Political Science Review.

[15]  Dutch Parliamentary Election Study 1998 , 1999 .

[16]  W. H. Schuur,et al.  Nonparametric Unidimensional Unfolding for Multicategory Data , 1992 .

[17]  J. Thomassen,et al.  Partisan Structures in the European Parliament , 1999 .