Is There Selection Bias in Laboratory Experiments?

Do the social and risk preferences of participants in laboratory experiments represent the preferences of the population from which they are recruited? To answer this question, we conducted a classroom experiment with a population of 1,173 students using a trust game and a lottery choice task to measure individual preferences. Separately, all 1,173 students were invited to participate in a laboratory experiment. To determine whether selection bias exists, we compare the preferences of the individuals who eventually participated in a laboratory experiment to those in the population. We find that the social and risk preferences of the students participating in the laboratory experiment are not significantly different from the preferences of the population from which they were recruited. We further show that participation decisions across most subgroups (e.g., men vs. women) do not differ significantly. We therefore fail to find selection bias based on social and risk preferences.

[1]  E. Rosen.  Differences between volunteers and non-volunteers for psychological studies. , 1951 .

[2]  M. Orne.  On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. , 1962 .

[3]  Robert Rosenthal,et al.  The Volunteer Subject , 1965 .

[4]  Robert Rosenthal,et al.  The Volunteer Subject , 1976 .

[5]  R. Rosenthal,et al.  The Volunteer Subject , 1965 .

[6]  J. Heckman.  Sample selection bias as a specification error , 1979 .

[7]  H. P. Binswanger.  Attitudes toward risk: Experimental measurement in rural india , 1980 .

[8]  H. P. Binswanger.  Attitudes toward Risk: Theoretical Implications of an Experiment in Rural India , 1981 .

[9]  W. Greene.  Sample Selection Bias as a Specification Error: Comment , 1981 .

[10]  S. Zamir,et al.  Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study , 1991 .

[11]  M. Rabin.  Published by: American , 2022 .

[12]  Thomas Gilovich,et al.  Does Studying Economics Inhibit Cooperation , 1993 .

[13]  Daniel Friedman,et al.  Experimental Methods: A Primer for Economists , 1994 .

[14]  J. Kagel,et al.  Handbook of Experimental Economics , 1997 .

[15]  Joyce E. Berg,et al.  Trust, Reciprocity, and Social History , 1995 .

[16]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[17]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[18]  John H. Kagel,et al.  Gaming against Managers in Incentive Systems: Experimental Results with Chinese Students and Chinese Managers , 1999 .

[19]  Edward,et al.  Measuring Trust , 2000 .

[20]  E. Fehr,et al.  Fairness and Retaliation: The Economics of Reciprocity , 2000, SSRN Electronic Journal.

[21]  Catherine C. Eckel,et al.  Volunteers and Pseudo-Volunteers: The Effect of Recruitment Method in Dictator Experiments , 2000 .

[22]  Colin Camerer,et al.  In search of homo economicus: Experiments in 15 small-scale societies , 2001 .

[23]  John H. Miller,et al.  NOTES AND COMMENTS GIVING ACCORDING TO GARP: AN EXPERIMENTAL TEST OF THE CONSISTENCY OF PREFERENCES FOR ALTRUISM , 2002 .

[24]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[25]  Colin Camerer,et al.  Measuring Social Norms and Preferences Using Experimental Games: A Guide for Social Scientists , 2002 .

[26]  R. Hardin.  Trust and Trustworthiness , 2002 .

[27]  U. Fischbacher,et al.  Why Social Preferences Matter - the Impact of Non-Selfish Motives on Competition, Cooperation and Incentives , 2002 .

[28]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[29]  John H. Kagel,et al.  Partial Gift Exchange in an Experimental Labor Market: Impact of Subject Population Differences, Productivity Differences, and Effort Requests on Behavior* , 2002, Journal of Labor Economics.

[30]  Lise Vesterlund,et al.  Trust in Children , 2002 .

[31]  Charles Bellemare,et al.  On Representative Trust , 2003 .

[32]  R. Zeckhauser,et al.  Trust, Risk and Betrayal , 2003 .

[33]  E. Fehr,et al.  The Hidden Costs and Returns of Incentives - Trust and Trustworthiness Among CEOS , 2004 .

[34]  J. Tirole,et al.  Incentives and Prosocial Behavior , 2005 .

[35]  Ben Greiner,et al.  The Online Recruitment System ORSEE 2.0 - A Guide for the Organization of Experiments in Economics , 2004 .

[36]  Iris Bohnet,et al.  Trust, Risk and Betrayal , 2004 .

[37]  Charles Bellemare,et al.  On Representative Social Capital , 2004, SSRN Electronic Journal.

[38]  James C. Cox,et al.  How to identify trust and reciprocity , 2004, Games Econ. Behav..

[39]  Klaus M. Schmidt,et al.  The Economics of Fairness, Reciprocity and Altruism--Experimental Evidence and New Theories. , 2005 .

[40]  Dean S. Karlan,et al.  Using Experimental Economics to Measure Social Capital and Predict Financial Decisions , 2005 .

[41]  Glenn W. Harrison,et al.  Risk Attitudes, Randomization to Treatment, and Self-Selection into Experiments , 2005 .

[42]  Joel Sobel,et al.  INTERDEPENDENT PREFERENCES AND RECIPROCITY , 2005 .

[43]  I. Bohnet,et al.  Decomposing trust and trustworthiness , 2006 .

[44]  J. List.  The Behavioralist Meets the Market: Measuring Social Preferences and Reputation Effects in Actual Transactions , 2005 .

[45]  D. Fudenberg,et al.  A Dual Self Model of Impulse Control , 2004, The American economic review.

[46]  Steven D. Levitt,et al.  Viewpoint: On the Generalizability of Lab Behaviour to the Field , 2007 .

[47]  John C. Ham,et al.  Selection Bias, Demographic Effects and Ability Effects in Common Value Auction Experiments , 2005 .

[48]  Steven D. Levitt,et al.  What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World , 2007 .

[49]  Stephan Meier,et al.  Do People Behave in Experiments as in the Field? Evidence from Donations , 2006 .

[50]  J. Carpenter,et al.  Altruistic behavior in a representative dictator experiment , 2008 .

[51]  Hawaii,et al.  Supporting Online Material Materials and Methods Figs. S1 to S6 Tables S1 and S2 Database S1 Antisocial Punishment across Societies , 2022 .

[52]  Catherine C. Eckel,et al.  Forecasting Risk Attitudes: An Experimental Study Using Actual and Forecast Gamble Choices , 2008 .

[53]  S. Gächter.  Behavioral Game Theory , 2008, Encyclopedia of Evolutionary Psychological Science.

[54]  Charles Bellemare,et al.  MEASURING INEQUITY AVERSION IN A HETEROGENEOUS POPULATION USING EXPERIMENTAL DECISIONS AND SUBJECTIVE PROBABILITIES , 2008 .

[55]  C. Plott,et al.  Handbook of Experimental Economics Results , 2008 .

[56]  James C. Cox,et al.  Risk aversion in experiments , 2008 .

[57]  Glenn W. Harrison,et al.  Preference Heterogeneity in Experiments: Comparing the Field and Laboratory , 2007 .

[58]  Rachel T. A. Croson,et al.  Gender Differences in Preferences , 2009 .

[59]  Robin P. Cubitt,et al.  Experimental Economics: Rethinking the Rules , 2009 .

[60]  Robert Slonim,et al.  The robustness of trust and reciprocity across a heterogeneous U.S. population , 2009 .

[61]  Ananish Chaudhuri,et al.  Propensities to engage in and punish corrupt behavior: Experimental evidence from Australia, India, Indonesia and Singapore , 2009 .

[62]  B. Roe,et al.  Risk-attitude selection bias in subject pools for experiments involving neuroimaging and blood samples , 2009 .

[63]  Paola Sapienza,et al.  Can We Infer Social Preferences from the Lab? Evidence from the Trust Game , 2010 .

[64]  D. Friedman.  Preferences, beliefs and equilibrium: What have experiments taught us? , 2010 .

[65]  C. Carter.  Freefall: America, Free Markets, and the Sinking of the World Economy , 2011 .