Natural experiments in leadership research: An introduction, review, and guidelines

Abstract Endogeneity is a serious challenge for leadership research. To overcome the problem, researchers increasingly rely upon experimental designs, such as laboratory and field experiments. In this paper, we argue that natural experiments — in the form of standard natural experiments, instrumental variable, and regression discontinuity designs — offer additional opportunities to infer causal relationships. We conduct a systematic, cross-disciplinary review of 87 studies that leverage natural experimental designs to inquire into a leadership topic. We introduce the standard natural experiment, instrumental variable, and regression discontinuity design and use topic modeling to analyze which leadership topics have been investigated using natural experimental designs. Based on the review, we provide guidelines that we hope will assist scholars in discovering natural exogenous variations, selecting the most suitable form of natural experiment and by mobilizing appropriate statistical techniques and robustness checks. The paper is addressed to leadership and management scholars who aim to use natural experiments to infer causal relationships.

[1]  David S. Lee,et al.  Regression Discontinuity Designs in Economics , 2009 .

[2]  L. Koh,et al.  The impact of environmental, social, and governance disclosure on firm value: The role of CEO power , 2017 .

[3]  O. Tosun The Effect of CEO Option Compensation on the Capital Structure: A Natural Experiment , 2011 .

[4]  Claudia C. Cogliser,et al.  Scholarly leadership of the study of leadership: A review of The Leadership Quarterly's second decade, 2000-2009. , 2010 .

[5]  Jianlei Han,et al.  Politically Connected Boards, Value or Cost: Evidence from a Natural Experiment in China , 2018 .

[6]  Roy Mersland,et al.  The influence of the CEO’s business education on the performance of hybrid organizations: the case of the global microfinance industry , 2017 .

[7]  Reinout E. de Vries,et al.  Personality predictors of leadership styles and the self-other agreement problem , 2012 .

[8]  Matias D. Cattaneo,et al.  A Practical Introduction to Regression Discontinuity Designs , 2019, 1911.09511.

[9]  R. Galbiati,et al.  Earthquakes, Religion, and Transition to Self-Government in Italian Cities , 2016, SSRN Electronic Journal.

[10]  D. Campbell,et al.  Regression-Discontinuity Analysis: An Alternative to the Ex-Post Facto Experiment , 1960 .

[11]  Erica Frantz,et al.  Countering Coups: Leadership Succession Rules in Dictatorships , 2017 .

[12]  Coady Wing,et al.  Designing Difference in Difference Studies: Best Practices for Public Health Policy Research. , 2018, Annual review of public health.

[13]  G. Harrison,et al.  Field experiments , 1924, The Journal of Agricultural Science.

[14]  M. Turner,et al.  Capturing hearts and minds: The influence of relational identification with the leader on followers' mobilization and cardiovascular reactivity , 2017, The Leadership Quarterly.

[15]  M. Sabatier,et al.  A women’s boom in the boardroom: effects on performance? , 2015 .

[16]  Pierre Azoulay,et al.  Social Influence Given (Partially) Deliberate Matching: Career Imprints in the Creation of Academic Entrepreneurs1 , 2009, American Journal of Sociology.

[17]  David A. Jaeger,et al.  Problems with Instrumental Variables Estimation when the Correlation between the Instruments and the Endogenous Explanatory Variable is Weak , 1995 .

[18]  Philippe Aghion,et al.  Innovation and Institutional Ownership , 2009 .

[19]  M. Conyon,et al.  Firm Performance and Boardroom Gender Diversity: A Quantile Regression Approach , 2017 .

[20]  Maretno A. Harjoto,et al.  Religiosity, female directors, and corporate social responsibility for Italian listed companies , 2019, Journal of Business Research.

[21]  Andrew McCallum,et al.  Optimizing Semantic Coherence in Topic Models , 2011, EMNLP.

[22]  Aviv Nevo,et al.  Identification With Imperfect Instruments , 2008, Review of Economics and Statistics.

[23]  P. Jiraporn,et al.  Exploring the Effect of Religious Piety on Corporate Governance: Evidence from Anti-takeover Defenses and Historical Religious Identification , 2017 .

[24]  G. Bascle Controlling for endogeneity with instrumental variables in strategic management research , 2008 .

[25]  Matthew E. Kahn,et al.  Water Pollution Progress at Borders: The Role of Changes in China's Political Promotion Incentives , 2015 .

[26]  Francisco Pérez-Gónzalez,et al.  Inside the Family Firm: The Role of Families in Succession Decisions and Performance , 2006 .

[27]  Wen-Chin Wu When do Dictators Decide to Liberalize Trade Regimes? Inequality and Trade Openness in Authoritarian Countries , 2014 .

[28]  Ali C. Akyol,et al.  Who Chooses Board Members? , 2013 .

[29]  Ronald H. Heck,et al.  Examining relationships among elementary schools' contexts, leadership, instructional practices, and added-year outcomes: a regression discontinuity approach , 2010 .

[30]  Isabelle Le Breton-Miller,et al.  For love and money: Marital leadership in family firms , 2017 .

[31]  Madhav V. Rajan,et al.  Identifying Control Motives in Managerial Ownership: Evidence from Antitakeover Legislation , 2005 .

[32]  Y. Kuang,et al.  Does obfuscating excessive CEO pay work? The influence of remuneration report readability on say-on-pay votes , 2017 .

[33]  Philip M. Podsakoff,et al.  Experimental designs in management and leadership research: Strengths, limitations, and recommendations for improving publishability , 2019, The Leadership Quarterly.

[34]  Vincenzo Scoppa,et al.  Gender Discrimination and Evaluators’ Gender: Evidence from Italian Academia , 2015 .

[35]  V. Scoppa,et al.  Can Gender Quotas Break Down Negative Stereotypes? Evidence From Changes In Electoral Rules , 2010 .

[36]  Justin McCrary,et al.  Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test , 2007 .

[37]  Donald P. Green,et al.  Instrumental Variables Estimation in Political Science: A Readers’ Guide , 2011 .

[38]  Andrew J. Oswald,et al.  Boss Competence and Worker Well-Being , 2017 .

[39]  P. Brockman,et al.  CEO Compensation Risk and Timely Loss Recognition , 2015 .

[40]  M. Gangl Causal Inference in Sociological Research , 2010 .

[41]  Matias D. Cattaneo,et al.  Econometric Methods for Program Evaluation , 2018, Annual Review of Economics.

[42]  Robert G. Lord,et al.  Leadership theory and research in the new millennium: Current theoretical trends and changing perspectives , 2014 .

[43]  Todd T. Milbourn,et al.  CEO Compensation and Corporate Risk: Evidence from a Natural Experiment , 2013 .

[44]  Michael Wyrwich Entrepreneurship and the intergenerational transmission of values , 2015 .

[45]  Kenneth F. Scheve,et al.  Estimating the Effect of Elite Communications on Public Opinion Using Instrumental Variables , 2007 .

[46]  Robin Martin,et al.  The role of self-regulation in developing leaders: a longitudinal field experiment , 2013 .

[47]  I. Filatotchev,et al.  Founder retention as CEO at IPO in emerging economies: The role of private equity owners and national institutions , 2019, Journal of Business Venturing.

[48]  Do Co‐Opted Directors Influence Dividend Policy? , 2018 .

[49]  M. Hovey,et al.  The Endogeneity of Executive Compensation and its Impact on Management Discretionary Behavior Over Financial Reporting , 2012 .

[50]  F. D. Hidalgo,et al.  Can Politicians Police Themselves? Natural Experimental Evidence From Brazil’s Audit Courts , 2016 .

[51]  P. Jiraporn,et al.  How do independent directors influence innovation productivity? A quasi-natural experiment , 2018 .

[52]  E. Coman Local elites, electoral reform and the distribution of central government funds: Evidence from Romania , 2018, Electoral Studies.

[53]  T. Breda,et al.  Professors in Core Science Fields Are Not Always Biased against Women: Evidence from France , 2015 .

[54]  J. Snow On the Mode of Communication of Cholera , 1856, Edinburgh medical journal.

[55]  Edward Lee,et al.  CEO Accountability for Corporate Fraud: Evidence from the Split Share Structure Reform in China , 2011, Journal of Business Ethics.

[56]  Fabian Homberg,et al.  Adaptation to Externally Driven Change : Th e Impact of Political Change on Job Satisfaction in the Public Sector , 2014 .

[57]  Daniel Ferreira,et al.  Understanding the Relationship between Founder-CEOs and Firm Performance , 2008 .

[58]  B. Adhikari Female executives and corporate cash holdings , 2017 .

[59]  Punit Arora Financially Linked Independent Directors and Bankruptcy Reemergence: The Role of Director Effort , 2018 .

[60]  Adam S. Yore,et al.  'I Do': Does Marital Status Affect How Much CEOs 'Do'? , 2014 .

[61]  Jonathan A. Solis,et al.  Leaders, Tenure, and the Politics of Sovereign Credit , 2018 .

[62]  M. Petersen,et al.  Perceived Conflict and Leader Dominance: Individual and Contextual Factors Behind Preferences for Dominant Leaders , 2017 .

[63]  Rikhil R Bhavnani Do the Effects of Temporary Ethnic Group Quotas Persist? Evidence from India , 2017 .

[64]  Petko Bogdanov,et al.  Introduction—Topic models: What they are and why they matter , 2013 .

[65]  Jie Chen,et al.  The Impact of Board Gender Composition on Dividend Payouts , 2016 .

[66]  Jason Poulos Land lotteries, long-term wealth, and political selection , 2018, Public Choice.

[67]  K. Rouse The Impact of High School Leadership on Subsequent Educational Attainment , 2012 .

[68]  J. Antonakis On doing better science: From thrill of discovery to policy implications , 2017 .

[69]  Lachlan McNamee,et al.  Indirect Colonial Rule Undermines Support for Democracy: Evidence From a Natural Experiment in Namibia , 2018 .

[70]  Matthew Semadeni,et al.  The perils of endogeneity and instrumental variables in strategy research: : Understanding through simulations , 2014 .

[71]  Roberto Weber,et al.  The Leadership Quarterly: State of the journal , 2019, The Leadership Quarterly.

[72]  Simon Munzert,et al.  Examining a Most Likely Case for Strong Campaign Effects: Hitler’s Speeches and the Rise of the Nazi Party, 1927–1933 , 2018, American Political Science Review.

[73]  John J. McConnell,et al.  Outside directors and corporate board decisions , 2005 .

[74]  Alessandro Minichilli,et al.  Gender Interactions Within the Family Firm , 2014, Manag. Sci..

[75]  Jeffrey M. Wooldridge,et al.  Introductory Econometrics: A Modern Approach , 1999 .

[76]  Taylor C. Boas,et al.  Controlling the Airwaves: Incumbency Advantage and Community Radio in Brazil , 2011 .

[77]  T. Dunning,et al.  Ethnic Quotas and Political Mobilization: Caste, Parties, and Distribution in Indian Village Councils , 2013, American Political Science Review.

[78]  G. Cox,et al.  Electoral Rules, Career Ambitions and Party Structure: Comparing Factions in Japan's Upper and Lower Houses , 2000 .

[79]  A. Khwaja,et al.  Can good projects succeed in bad communities , 2009 .

[80]  Jasjeet S. Sekhon,et al.  When Natural Experiments Are Neither Natural nor Experiments , 2012, American Political Science Review.

[81]  Esther Duflo,et al.  Female Leadership Raises Aspirations and Educational Attainment for Girls: A Policy Experiment in India , 2012, Science.

[82]  Technological Change and Political Turnover: The Democratizing Effects of the Green Revolution in India , 2017 .

[83]  Alma Cohen,et al.  How Do Staggered Boards Affect Shareholder Value? Evidence from a Natural Experiment , 2013 .

[84]  J. McNulty,et al.  Observing the Counterfactual? The Search for Political Experiments in Nature , 2009, Political Analysis.

[85]  Thad Dunning,et al.  Natural Experiments in the Social Sciences , 2012 .

[86]  Daniel M. Butler A regression discontinuity design analysis of the incumbency advantage and tenure in the U.S. House , 2009 .

[87]  D. S. Lee,et al.  Are Two Heads Better Than One? Evidence from the Thrift Crisis , 2011 .

[88]  Matias D. Cattaneo,et al.  The Choice of Neighborhood in Regression Discontinuity Designs , 2021 .

[89]  Scott E. Yonker,et al.  Board Diversity, Firm Risk, and Corporate Policies , 2017 .

[90]  Robert Dur,et al.  Team incentives, task assignment, and performance: A field experiment , 2020 .

[91]  The Impact of the Short-Short Rule Repeal on the Timing Ability of Mutual Funds , 2008 .

[92]  P. Bansal,et al.  Does a Long-Term Orientation Create Value? Evidence from a Regression Discontinuity , 2016 .

[93]  Chen Lin,et al.  Managerial Incentives, CEO Characteristics and Corporate Innovation in China’s Private Sector , 2009 .

[94]  Julie Wulf,et al.  The Flattening Firm and Product Market Competition: The Effect of Trade Liberalization on Corporate Hierarchies , 2010 .

[95]  W. Simpson,et al.  Managerial Incentives and the Use of Foreign-Exchange Derivatives by Banks , 2004 .

[96]  J. Canil,et al.  CEO pay disparity: Efficient contracting or managerial power? , 2016, Journal of Corporate Finance.

[97]  D. Rubin Causal Inference Using Potential Outcomes , 2005 .

[98]  Maria Guadalupe,et al.  Executive Compensation and Competition in the Banking and Financial Sectors , 2004, SSRN Electronic Journal.

[99]  Erik Grönqvist,et al.  The Making of a Manager: Evidence from Military Officer Training , 2015, Journal of Labor Economics.

[100]  J. Stoker,et al.  Tightening the leash after a threat: A multi-level event study on leadership behavior following the financial crisis , 2019, The Leadership Quarterly.

[101]  N. Rickman,et al.  Favoritism and Financial Incentives: A Natural Experiment , 2005 .

[102]  R. Lund Identification and Inference for Econometric Models: Essays in Honor of Thomas Rothenberg , 2006 .

[103]  Betsy Sinclair Cambridge Handbook of Experimental Political Science: Design and Analysis of Experiments in Multilevel Populations , 2011 .

[104]  K. Røed,et al.  The gender gap in entrepreneurship – The role of peer effects , 2017 .

[105]  C. Barros,et al.  Governance and CEO pay and performance in non-profit organizations , 2007 .

[106]  Martin Baekgaard THE IMPACT OF FORMAL ORGANIZATIONAL STRUCTURE ON POLITICO-ADMINISTRATIVE INTERACTION: EVIDENCE FROM A NATURAL EXPERIMENT , 2011 .

[107]  S. Weiler,et al.  Does gender matter for job creation? Business ownership and employment growth , 2016 .

[108]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[109]  Todd Milbourn,et al.  CEO Equity Incentives and Financial Misreporting: The Role of Auditor Expertise , 2014 .

[110]  Amalia R. Miller,et al.  A Female Style in Corporate Leadership? Evidence from Quotas , 2012 .

[111]  Delia Baldassarri,et al.  Field experiments across the social sciences , 2017 .

[112]  Janaki Gooty,et al.  Construct redundancy in leader behaviors: A review and agenda for the future , 2017 .

[113]  Richard R. Townsend,et al.  How Do Quasi-Random Option Grants Affect CEO Risk-Taking? , 2017 .

[114]  Christopher Winship,et al.  Counterfactuals and Causal Inference: Methods and Principles for Social Research , 2007 .

[115]  C. Kuzey,et al.  The effect of board gender diversity on firm performance: evidence from Turkey , 2016 .

[116]  Philippe Jacquart,et al.  On making causal claims: A review and recommendations , 2010 .

[117]  Iftekhar Hasan,et al.  The Effect of Board Directors from Countries with Different Genetic Diversity Levels on Corporate Performance , 2015, Manag. Sci..

[118]  Jody Hoffer Gittell,et al.  Is the doctor in? A relational approach to job design and the coordination of work , 2008 .

[119]  J. Sieweke,et al.  The Legitimacy of Inequality: Integrating the Perspectives of System Justification and Social Judgment , 2018 .

[120]  Linus Siming Orders of Merit and CEO Compensation: Evidence from a Natural Experiment , 2016 .

[121]  John P. Haisken-DeNew,et al.  The causal effect of income on health: evidence from German reunification. , 2005, Journal of health economics.

[122]  Carol Tavris,et al.  The queen bee syndrome , 1974 .

[123]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[124]  Simon Chauchard Can Descriptive Representation Change Beliefs about a Stigmatized Group? Evidence from Rural India , 2014, American Political Science Review.

[125]  A. Young Mostly Harmless Econometrics , 2012 .

[126]  Benjamin F. Jones,et al.  Do Leaders Matter? National Leadership and Growth Since World War Ii , 2004 .

[127]  Joshua D. Angrist,et al.  Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records , 1990 .

[128]  D. Rubin,et al.  Causal Inference for Statistics, Social, and Biomedical Sciences: Causality: The Basic Framework , 2015 .

[129]  Nitesh V. Chawla,et al.  A network’s gender composition and communication pattern predict women’s leadership success , 2019, Proceedings of the National Academy of Sciences.

[130]  Guido Imbens,et al.  Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs , 2014 .

[131]  Paulo Arvate,et al.  The queen bee: A myth? The effect of top-level female leadership on subordinate females , 2018, The Leadership Quarterly.

[132]  S. Norton Nonrandom Risk: The 1970 Draft Lottery , 1997 .

[133]  M. Huber,et al.  Including Covariates in the Regression Discontinuity Design , 2018, Journal of Business & Economic Statistics.

[134]  C. Driver,et al.  R&D and CEO departure date: do financial incentives make CEOs more opportunistic? , 2017 .

[135]  J. L. M. Olea,et al.  A Robust Test for Weak Instruments , 2013 .

[136]  Melody E. Valdini A deterrent to diversity: The conditional effect of electoral rules on the nomination of women candidates , 2012 .