Theory-Testing, Generalization, and the Problem of External Validity*

External validity refers to the generalization of research findings, either from a sample to a larger population or to settings and populations other than those studied. While definitions vary, discussions generally agree that experiments are lower in external validity than other methodological approaches. Further, external validity is widely treated as an issue to be addressed through methodological procedures. When testing theories, all measures are indirect indicators of theoretical constructs, and no methodological procedures taken alone can produce external validity. External validity can be assessed through determining (1) the extent to which empirical measures accurately reflect theoretical constructs, (2) whether the research setting conforms to the scope of the theory under test, (3) our confidence that findings will repeat under identical conditions, (4) whether findings support the theory being tested, and (5) the confirmatory status of the theory under test. In these ways, external validity is foremost a theoretical issue and can only be addressed by an examination of the interplay between theory and methods.

[1]  William R. Shadish,et al.  Revisiting field experimentation: field notes for the future. , 2002, Psychological methods.

[2]  Kristin G. Esterberg,et al.  Qualitative Methods in Social Research , 2001 .

[3]  Donald G. McTavish,et al.  Social Research: An Evolving Process , 2001 .

[4]  Shane R. Thye.  Reliability in Experimental Sociology , 2000 .

[5]  M. Dunn,et al.  Conducting Marketing Science: The Role of Replication in the Research Process , 2000 .

[6]  Kim J. Vicente,et al.  A Framework for Epistemological Analysis in Empirical (Laboratory and Field) Studies , 2000, Hum. Factors.

[7]  D. Guthrie,et al.  The State, Courts, and Equal Opportunities for Female CEOs in U.S. Organizations: Specifying Institutional Mechanisms , 1999 .

[8]  Odd Nordhaug,et al.  Institutional and Rational Determinants of Organizational Practices: Human Resource Management in European Firms , 1999 .

[9]  Gary L. Wells,et al.  Stimulus Sampling and Social Psychological Experimentation , 1999 .

[10]  J. Lynch.  Theory and external validity , 1999 .

[11]  Russell S. Winer,et al.  Experimentation in the 21st century: The importance of external validity , 1999 .

[12]  P McElroy,et al.  An institutional analysis of HIV prevention efforts by the nation's outpatient drug abuse treatment units. , 1999, Journal of health and social behavior.

[13]  D. Wayne Osgood,et al.  Gang Resistance Education and Training (Great): Results from the National Evaluation , 1999 .

[14]  Catherine E. Ross,et al.  Disorder and Decay , 1999 .

[15]  Michael J. Lovaglia,et al.  Status Processes and Mental Ability Test Scores , 1998, American Journal of Sociology.

[16]  K. Bollen,et al.  Detection and determinants of bias in subjective measures , 1998 .

[17]  Timothy D. Wilson,et al.  Experimentation in social psychology. , 1998 .

[18]  Kwok Sai Tricia Cheuk-Wong,et al.  Stress preparation, coping style, and nurses' experience of being spurned by patients , 1997 .

[19]  M. Foschi.  On Scope Conditions , 1997 .

[20]  James D. Wright,et al.  Evaluating an alcohol and drug treatment program for the homeless: An econometric approach , 1997 .

[21]  T. Nayak.  Statistical Significance: Rationale, Validity and Utility , 1997 .

[22]  Tabbye M. Chavous,et al.  Multidimensional Inventory of Black Identity: A Preliminary Investigation of Reliability and Construct Validity , 1997 .

[23]  T. Allen,et al.  Hierarchy Theory: A Vision, Vocabulary, and Epistemology , 1996 .

[24]  Kevin M. Thompson,et al.  Specialization patterns of gang and nongang offending: A latent structure analysis , 1996 .

[25]  Gordon Bear,et al.  Computationally intensive methods warrant reconsideration of pedagogy in statistics , 1995 .

[26]  K. Raman.  Inductive Inference and Replications: A Bayesian Perspective , 1994 .

[27]  James W. Neuliep,et al.  Reviewer bias against replication research. , 1993 .

[28]  Frank L. Schmidt,et al.  What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. , 1992 .

[29]  Richard J. Shavelson,et al.  Generalizability Theory: A Primer , 1991 .

[30]  T. J. Top.  Sex bias in the evaluation of performance in the scientific, artistic, and literary professions: A review , 1991 .

[31]  B. Berg.  Qualitative Research Methods for the Social Sciences , 1989 .

[32]  T. Pettigrew.  Influencing Policy with Social Psychology , 1988 .

[33]  K. Hopkins,et al.  Mail Surveys of Professional Populations: The Effects of Monetary Gratuities on Return Rates. , 1988 .

[34]  Irving M. Lane,et al.  A note on the role of laboratory methodologies in applied behavioural research: Don't throw out the baby with the bath water , 1988 .

[35]  J J Louviere,et al.  CONJOINT ANALYSIS MODELLING OF STATE OF PREFERENCES. A REVIEW OF THEORY, METHODS, RECENT DEVELOPMENTS AND EXTERNAL VALIDITY , 1988 .

[36]  E. A. Locke,et al.  Generalizing From Laboratory to Field Settings. , 1987 .

[37]  David Willer,et al.  Theory and the Experimental Investigation of Social Structures , 1987 .

[38]  Robin M. Hogarth,et al.  Generalization in Decision Research: The Role of Formal Models , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  M. Gordon,et al.  The “Science of the Sophomore” Revisited: from Conjecture to Empiricism , 1986 .

[40]  Henry A. Walker,et al.  Scope Statements: Imperatives for Evaluating Theory , 1985 .

[41]  The Determinants of Marital Instability: Some Methodological Issues. , 1984 .

[42]  J. A. Wiggins.  Family Violence as a Case of Interpersonal Aggression: A Situational Analysis , 1983 .

[43]  J. Lynch.  The Role of External Validity in Theoretical Research , 1983 .

[44]  J. McGrath,et al.  External Validity and the Research Process: A Comment on the Calder/Lynch Dialogue , 1983 .

[45]  D. Mook,et al.  In defense of external invalidity. , 1983 .

[46]  J. Lynch.  On the External Validity of Experiments in Consumer Research , 1982 .

[47]  Alice M. Tybout,et al.  The Concept of External Validity , 1982 .

[48]  John D. Greenwood,et al.  On the Relation Between Laboratory Experiments and Social Behaviour: Causal Explanation and Generalization , 1982 .

[49]  L. Berkowitz,et al.  External validity is more than skin deep: Some answers to criticisms of laboratory experiments. , 1982 .

[50]  M. W. Taylor.  The Noninfluence of "Family Background" On Intellectual Attainment , 1980 .

[51]  Ira J. Firestone,et al.  Implications of representativeness for generalizability of field and laboratory research findings. , 1980 .

[52]  E. P. Willems,et al.  The external validity of papers on external validity. , 1980 .

[53]  L. Delbeke.  Quasi-experimentation - design and analysis issues for field settings - cook,td, campbell,dt , 1980 .

[54]  H. Blalock.  The Presidential Address: Measurement and Conceptualization Problems: The Major Obstacle to Integrating Theory and Research , 1979 .

[55]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[56]  J. Sell,et al.  The Role of the Experiment in the Social Sciences , 1979 .

[57]  J. S. Shrauger,et al.  Symbolic interactionist view of self-concept: Through the looking glass darkly. , 1979 .

[58]  R. L. Dipboye,et al.  Research settings in industrial and organizational psychology: Are findings in the field more generalizable than in the laboratory? , 1979 .

[59]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[60]  Kenneth J. Gergen,et al.  Experimentation in social psychology: A reappraisal , 1978 .

[61]  K. Bailey.  Methods of Social Research , 1978 .

[62]  R. Ferber.  Research By Convenience , 1977 .

[63]  R. Swinburne.  OBJECTIVE KNOWLEDGE: AN EVOLUTIONARY APPROACH , 1973 .

[64]  David Willer,et al.  Systematic empiricism: critique of a pseudoscience , 1973 .

[65]  Joseph Berger,et al.  Status Characteristics and Social Interaction , 1972 .

[66]  K. Popper.  Objective Knowledge: An Evolutionary Approach , 1972 .

[67]  M. Webster,et al.  Artificiality in Experimental Sociology , 1971 .

[68]  David Willer,et al.  Theoretical Concepts and Observables , 1970 .

[69]  Earl R. Babbie,et al.  The practice of social research , 1969 .

[70]  Ernest Nagel,et al.  The Structure of Science , 1962 .

[71]  L. Kattsoff,et al.  The Structure of Theories , 1957 .