Separating Controversy from Noise: Comparison and Normalization of Structural Polarization Measures

Quantifying the amount of polarization is crucial for understanding and studying political polarization in political and social systems. Several methods are used commonly to measure polarization in social networks by purely inspecting their structure. We analyse eight of such methods and show that all of them yield high polarization scores even for random networks with similar density and degree distributions to typical real-world networks. Further, some of the methods are sensitive to degree distributions and relative sizes of the polarized groups. We propose normalization to the existing scores and a minimal set of tests that a score should pass in order for it to be suitable for separating polarized networks from random noise. The performance of the scores increased by 38%-220% after normalization in a classification task of 203 networks. Further, we find that the choice of method is not as important as normalization, after which most of the methods have better performance than the best-performing method before normalization. This work opens up the possibility to critically assess and compare the features and performance of structural polarization methods.

[1]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[2]  S. Fortunato,et al.  Resolution limit in community detection , 2006, Proceedings of the National Academy of Sciences.

[3]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Manfred Schoch,et al.  Quantification of echo Chambers: a Methodological Framework considering Multi-Party Systems , 2019, ECIS.

[5]  Chiara Orsini,et al.  Quantifying randomness in real networks , 2015, Nature Communications.

[6]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[7]  Anna Rumshisky,et al.  Combining Network and Language Indicators for Tracking Conflict Intensity , 2017, SocInfo.

[8]  S. Levin,et al.  The dynamics of political polarization , 2021, Proceedings of the National Academy of Sciences.

[9]  Venkata Rama Kiran Garimella.  Polarization on Social Media , 2018 .

[10]  Tiago P Peixoto,et al.  Parsimonious module inference in large networks. , 2012, Physical review letters.

[11]  A. Maritan,et al.  True scale-free networks hidden by finite size effects , 2020, Proceedings of the National Academy of Sciences.

[12]  C. Makridis,et al.  The Real Cost of Political Polarization: Evidence from the COVID-19 Pandemic , 2020, SSRN Electronic Journal.

[13]  Marco Brambilla,et al.  Measuring Controversy in Social Networks Through NLP , 2020, SPIRE.

[14]  Venkata Rama Kiran Garimella,et al.  Secular vs. Islamist polarization in Egypt on Twitter , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[15]  David G. Myers,et al.  Polarizing effects of social comparison , 1978 .

[16]  Elizabeth Suhay,et al.  Explaining Group Influence: The Role of Identity and Emotion in Political Conformity and Polarization , 2015 .

[17]  Bruce A. Reed,et al.  A Critical Point for Random Graphs with a Given Degree Sequence , 1995, Random Struct. Algorithms.

[18]  Dario Fasino,et al.  Generating large scale-free networks with the Chung-Lu random graph model , 2019, ArXiv.

[19]  R. Guimerà,et al.  Modularity from fluctuations in random graphs and complex networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[20]  M. Fiorina,et al.  Political Polarization in the American Public , 2008 .

[21]  Yilin Zhang,et al.  Understanding Regularized Spectral Clustering via Graph Conductance , 2018, NeurIPS.

[22]  Yasushi Kawase,et al.  Z-Score-Based Modularity for Community Detection in Networks , 2015, PloS one.

[23]  David R. Jones.  Party Polarization and Legislative Gridlock , 2001 .

[24]  Igor M. Sokolov,et al.  Modeling echo chambers and polarization dynamics in social networks , 2019, Physical review letters.

[25]  Mucahid Kutlu,et al.  Embeddings-Based Clustering for Target Specific Stances: The Case of a Polarized Turkey , 2020, ArXiv.

[26]  Claire Cardie,et al.  A Measure of Polarization on Social Media Networks Based on Community Boundaries , 2013, ICWSM.

[27]  Justin H. Gross,et al.  Measurement and theory in legislative networks: The evolving topology of Congressional collaboration , 2014, Soc. Networks.

[28]  A. J. Morales,et al.  Measuring Political Polarization: Twitter shows the two sides of Venezuela , 2015, Chaos.

[29]  Mir Saman Tajbakhsh,et al.  A framework for quantifying controversy of social network debates using attributed networks: biased random walk (BRW) , 2020, Social Network Analysis and Mining.

[30]  Jonathan Bright,et al.  Explaining the Emergence of Political Fragmentation on Social Media: The Role of Ideology and Extremism , 2018, J. Comput. Mediat. Commun..

[31]  L. Akoglu.  Quantifying Political Polarity Based on Bipartite Opinion Networks , 2014, ICWSM.

[32]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[33]  Aristides Gionis,et al.  Quantifying Controversy on Social Media , 2018, ACM Trans. Soc. Comput..

[34]  Jack Zhou.  Boomerangs versus Javelins: How Polarization Constrains Communication on Climate Change , 2016 .

[35]  ANDRES REILJAN,et al.  ‘Fear and loathing across party lines’ (also) in Europe: Affective polarisation in European party systems , 2019, European Journal of Political Research.

[36]  Peter J. Mucha,et al.  Portrait of Political Party Polarization1 , 2013, Network Science.

[37]  Cristopher Moore,et al.  Scalable detection of statistically significant communities and hierarchies, using message passing for modularity , 2014, Proceedings of the National Academy of Sciences.

[38]  Zachary P. Neal,et al.  A sign of the times? Weak and strong polarization in the U.S. Congress, 1973-2016 , 2020, Soc. Networks.

[39]  E. Hargittai,et al.  Cross-ideological discussions among conservative and liberal bloggers , 2007 .

[40]  P. Mucha,et al.  Party Polarization in Congress: A Social Networks Approach , 2009, 0907.3509.

[41]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[42]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[43]  Aristides Gionis,et al.  Political Discourse on Social Media: Echo Chambers, Gatekeepers, and the Price of Bipartisanship , 2018, WWW.

[44]  David Chavalarias,et al.  Reconstruction of the socio-semantic dynamics of political activist Twitter networks—Method and application to the 2017 French presidential election , 2018, PloS one.

[45]  Mahmoud Al-Ayyoub,et al.  Measuring the controversy level of Arabic trending topics on Twitter , 2016, 2016 7th International Conference on Information and Communication Systems (ICICS).

[46]  Romualdo Pastor-Satorras,et al.  Quantifying echo chamber effects in information spreading over political communication networks , 2019, EPJ Data Science.

[47]  Robert N. Stern,et al.  Informal Networks and Organizational Crises: An Experimental Simulation , 1988 .

[48]  Colin McDiarmid,et al.  Modularity of regular and treelike graphs , 2018, J. Complex Networks.

[49]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[50]  Mikko Kivelä,et al.  Polarization of Climate Politics Results from Partisan Sorting: Evidence from Finnish Twittersphere , 2020, Global Environmental Change.

[51]  F. Radicchi,et al.  Statistical significance of communities in networks. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  1959 , 2018, Documents on Irish Foreign Policy vol. XI.

[53]  Clara Pizzuti,et al.  Analyzing Voting Behavior in Italian Parliament: Group Cohesion and Evolution , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[54]  Colin McDiarmid,et al.  Modularity of Erdős‐Rényi random graphs , 2018, AofA.

[55]  A. Gelman,et al.  Partisans without Constraint: Political Polarization and Trends in American Public Opinion. , 2008, AJS; American journal of sociology.

[56]  Joel Nishimura,et al.  Configuring Random Graph Models with Fixed Degree Sequences , 2016, SIAM Rev..

[57]  Kareem Darwish,et al.  Quantifying Polarization on Twitter: The Kavanaugh Nomination , 2019, SocInfo.

[58]  Jennifer McCoy,et al.  Polarization and the Global Crisis of Democracy: Common Patterns, Dynamics, and Pernicious Consequences for Democratic Polities , 2018 .

[59]  Jacob Ratkiewicz,et al.  Political Polarization on Twitter , 2011, ICWSM.

[60]  Lilliana Mason,et al.  “I Disrespectfully Agree”: The Differential Effects of Partisan Sorting on Social and Issue Polarization , 2015 .

[61]  Kyriaki Kalimeri,et al.  Falling into the Echo Chamber: the Italian Vaccination Debate on Twitter , 2020, ICWSM.

[62]  James P. Bagrow,et al.  Communities and bottlenecks: trees and treelike networks have high modularity. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  Taro Takaguchi,et al.  Randomized reference models for temporal networks , 2018, SIAM Rev..

[64]  Priya Mahadevan,et al.  Systematic topology analysis and generation using degree correlations , 2006, SIGCOMM.

[65]  John H. Evans,et al.  Have American's Social Attitudes Become More Polarized? , 1996, American Journal of Sociology.

[66]  Michelangelo Puliga,et al.  Voting Behavior, Coalitions and Government Strength through a Complex Network Analysis , 2014, PloS one.