Understanding selection bias, time-lags and measurement bias in secondary data sources: Putting the Encyclopedia of Associations database in broader context.

Secondary data gathered for purposes other than research play an important role in the social sciences. A recent data release has made an important source of publicly available data on associational interests, the Encyclopedia of Associations (EA), readily accessible to scholars (www.policyagendas.org). In this paper we introduce these new data and systematically investigate issues of lag between events and subsequent reporting in the EA, as these have important but under-appreciated effects on time-series statistical models. We further analyze the accuracy and coverage of the database in numerous ways. Our study serves as a guide to potential users of this database, but we also reflect upon a number of issues that should concern all researchers who use secondary data such as newspaper records, IRS reports and FBI Uniform Crime Reports.

[1]  Gregory D. Saxton,et al.  Social Capital and the Growth of the Nonprofit Sector , 2005 .

[2]  J. D. McCarthy,et al.  Measuring association populations using the Encyclopedia of Associations: Evidence from the field of labor unions ☆ , 2006 .

[3]  John Boli,et al.  World culture in the world polity : A century of international non-governmental organization , 1997 .

[4]  J. Castelli,et al.  The Necessity for Studying Organizational Advocacy Comparatively , 2002 .

[5]  Naomi Rosenthal,et al.  Social Movements and Network Analysis: A Case Study of Nineteenth-Century Women's Reform in New York State , 1985, American Journal of Sociology.

[6]  Donald E. Abelson,et al.  Following Suit or Falling Behind? A Comparative Analysis of Think Tanks in Canada and the United States , 1998, Canadian Journal of Political Science.

[7]  John D. McCarthy,et al.  Images of protest: Dimensions of selection bias in media coverage of Washington demonstrations, 1982 and 1991 , 1996 .

[8]  V. Hodgkinson,et al.  Measuring the Impact of the Nonprofit Sector , 2002 .

[9]  Pamela Oliver,et al.  Political Processes and Local Newspaper Coverage of Protest Events: From Selection Bias to Triadic Interactions1 , 2000, American Journal of Sociology.

[10]  Debra C. Minkoff.  The sequencing of social movements , 1997 .

[11]  D. Lowery.  Interest Organization Populations: The Demands of the Scale of Analysis and the Theoretical Purposes of Counting , 2012 .

[12]  W. Gove,et al.  ARE UNIFORM CRIME REPORTS A VALID INDICATOR OF THE INDEX CRIMES? AN AFFIRMATIVE ANSWER WITH MINOR QUALIFICATIONS* , 1985 .

[13]  Martin Ruef,et al.  Organizations evolving, second edition , 2006 .

[14]  Robert J. Brulle,et al.  Building Environmentalism: The Founding of Environmental Movement Organizations in the United States, 1900–2000 , 2012 .

[15]  Steven D. Levitt,et al.  The Relationship Between Crime Reporting and Police: Implications for the Use of Uniform Crime Reports , 1998 .

[16]  Mark A. Hager,et al.  Response Rates for Mail Surveys of Nonprofit Organizations: A Review and Empirical Test , 2003 .

[17]  Ross L. Matsueda,et al.  コントロ-ル理論と分化的接触理論の検証--因果モデルを用いて(American Sociological Review.,Vol.47,1982) , 1989 .

[18]  Hua Zhong,et al.  TRENDS IN THE GENDER GAP IN VIOLENCE: REEVALUATING NCVS AND OTHER EVIDENCE* , 2009 .

[19]  Gerald F. Davis,et al.  Social movements and organization theory , 2005 .

[20]  M. Olson,et al.  The Logic of Collective Action: Public Goods and the Theory of Groups. , 1973 .

[21]  S. Staggenborg,et al.  Methods of Social Movement Research , 2002 .

[22]  Bob Edwards.  Semiformal Organizational Structure Among Social Movement Organizations: An Analysis of the U.S. Peace Movement , 1994 .

[23]  Richard McCleary,et al.  Uniform Crime Reports as Organizational Outcomes: Three Time Series Experiments , 1982 .

[24]  Anthony J. Nownes.  The Population Ecology of Interest Group Formation: Mobilizing for Gay and Lesbian Rights in the United States, 1950–98 , 2003, British Journal of Political Science.

[25]  Debra C. Minkoff,et al.  Reinvigorating the study of organizational processes in social movements , 2005 .

[26]  David Knoke,et al.  ASSOCIATIONS AND INTEREST GROUPS , 1986 .

[27]  Jack Quarter,et al.  The Interchangeability of Paid Staff and Volunteers in Nonprofit Organizations , 2008 .

[28]  C. Perrow.  A society of organizations , 1991 .

[29]  Philip A. Schrodt.  Precedents, Progress, and Prospects in Political Event Data , 2012 .

[30]  Local Environmental Groups: A Systematic Enumeration in Two Geographical Areas* , 2009 .

[31]  Jason Beckfield,et al.  The Social Structure of the World Polity1 , 2010, American Journal of Sociology.

[32]  Erik W. Johnson,et al.  Ecological Threat and the Founding of U.S. National Environmental Movement Organizations, 1962–1998 , 2011 .

[33]  K. L. Schlozman,et al.  Organized interests and American democracy , 1986 .

[34]  Organizing for Equality: The Evolution of Women's and Racial-Ethnic Organizations in America, 1955-1985 , 1996 .

[35]  Dennis Chong.  Collective action and the civil rights movement , 1991 .

[36]  Grant Jordan,et al.  The scale of interest organization in democratic politics : data and research methods , 2012 .

[37]  Bob Edwards,et al.  The organizational structure of local environmentalism , 2005 .

[38]  Jackie Smith,et al.  Social Movements in the World-System: The Politics of Crisis and Transformation , 2012 .

[39]  Karen A. Froelich,et al.  Financial Measures in Nonprofit Organization Research: Comparing IRS 990 Return and Audited Financial Statement Data , 2000 .

[40]  J. D. McCarthy,et al.  Replacing Members with Managers? Mutualism among Membership and Nonmembership Advocacy Organizations in the United States1 , 2011, American Journal of Sociology.

[41]  Jack L. Walker.  The Origins and Maintenance of Interest Groups in America , 1983, American Political Science Review.

[42]  Erik W. Johnson.  Changing issue representation among major United States environmental movement organizations , 2006 .

[43]  J. Sousa,et al.  Comparing Member-Based Organizations within a Social Economy Framework , 2001 .

[44]  Pamela Oliver,et al.  How Events Enter the Public Sphere: Conflict, Location, and Sponsorship in Local Newspaper Coverage of Public Events1 , 1999, American Journal of Sociology.

[45]  J. D. McCarthy,et al.  Movement Organizations, Synergistic Tactics and Environmental Public Policy , 2010 .

[46]  S. Bevan.  Continuing the Collective Action Dilemma , 2013 .

[47]  John Mark Hansen,et al.  Gaining Access: Congress and the Farm Lobby, 1919-1981 , 1991 .

[48]  G. King,et al.  What to Do about Missing Values in Time‐Series Cross‐Section Data , 2010 .

[49]  John I. Kitsuse,et al.  A Note on the Uses of Official Statistics , 1963 .

[50]  J. D. McCarthy,et al.  The use of newspaper data in the study of collective action , 2003 .

[51]  Jack L. Walker.  Mobilizing Interest Groups in America , 1991 .

[52]  Dretha Phillips,et al.  The Mismeasure of Crime , 2002 .

[53]  Michael G. Maxfield,et al.  The National Incident-Based Reporting System: Research and Policy Applications , 1999 .

[54]  John D. McCarthy,et al.  Who Counts and How: Estimating the Size of Protests , 2004 .

[55]  J. D. McCarthy,et al.  Resource Mobilization and Social Movements: A Partial Theory , 1977, American Journal of Sociology.

[56]  Ann Hironaka,et al.  The Nation-State and the Natural Environment over the Twentieth Century , 2000, American Sociological Review.

[57]  Debra C. Minkoff.  Bending with the Wind: Strategic Change and Adaptation by Women's and Racial Minority Organizations1 , 1999, American Journal of Sociology.

[58]  M. Olson,et al.  The Logic of Collective Action: Public Goods and the Theory of Groups , 1969 .

[59]  R. Molz,et al.  The Big Short. , 2011 .

[60]  Michelle H. Yetman,et al.  Calibrating the Reliability of Publicly Available Nonprofit Taxable Activity Disclosures , 2009 .

[61]  Neal Caren,et al.  Making the News , 2010 .

[62]  Donald Black,et al.  Production of Crime Rates , 1970 .

[63]  Jackie Smith.  Nonresponse Bias in Organizational Surveys: Evidence from a Survey of Groups and Organizations Working for Peace , 1997 .

[64]  Brayden G. King,et al.  Competition and Resource Partitioning in Three Social Movement Industries1 , 2008, American Journal of Sociology.

[65]  Sidney C. Sufrin,et al.  The Logic of Collective Action: Public Goods and the Theory of Groups. , 1966 .

[66]  M. Laver,et al.  Extracting Policy Positions from Political Texts Using Words as Data , 2003, American Political Science Review.

[67]  Virginia Gray,et al.  The Population Ecology of Interest Representation: Lobbying Communities in the American States , 1996 .

[68]  Steven P. Croley,et al.  Gaining Access: Congress and the Farm Lobby, 1919-1981 , 1992 .

[69]  J. D. McCarthy,et al.  Tracking Interest Group Populations in the US and the UK , 2012 .