Consideration of the Incomplete Block Design for Policy-Capturing Research

This article introduces and urges consideration of the incomplete block design for policy-capturing research. The authors compare incomplete block designs with full factorial and fractional factorial designs along several dimensions, including survey length, analysis limitations, and impact on study participants. An initial empirical investigation of an incomplete block design and a full factorial design is performed in a policy-capturing study of 183 job seekers. As predicted, the two designs generated equivalent results on the main effects and most interactions; however, the designs yielded divergent results on some interactions. Study participants who received the incomplete block design were more positive about the survey length and experienced less stress and exhaustion than those receiving the full design. The authors discuss how methodologists might examine and compare policy-capturing designs in future research.

[1]  Robert J. Wherry,et al.  The Use of Simulated Stimuli and the "Jan" Technique to Capture and Cluster the Policies of Raters , 1965 .

[2]  Sara L. Rynes,et al.  ACADEMIC RESEARCH INSIDE ORGANIZATIONS: INPUTS, PROCESSES, AND OUTCOMES , 1999 .

[3]  Howard E. Miller,et al.  Recruiter and Job Influences on Candidates for Employment , 1983 .

[4]  D. Baker,et al.  Exploring Contrasting Formulations of Expectancy Theory , 1989 .

[5]  Timothy A. Judge,et al.  Effects of Work Values on Job Choice Decisions , 1992 .

[6]  R J Wherry,et al.  Generating multiple samples of multivariate data with arbitrary population parameters , 1965, Psychometrika.

[7]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[8]  M. Hagen,et al.  Ethical principles of psychologists and code of conduct. , 2002, The American psychologist.

[9]  David B. Balkin,et al.  Determinants of faculty pay: An agency theory perspective. , 1992 .

[10]  P. Hoffman.  The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[11]  M. Hardy.  Regression with dummy variables , 1993 .

[12]  Amy L. Pablo.  Determinants of acquisition integration level: A decision-making perspective. , 1994 .

[13]  T. Judge,et al.  PAY PREFERENCES AND JOB SEARCH DECISIONS: A PERSON-ORGANIZATION FIT PERSPECTIVE , 1994 .

[14]  H. N. Wheeler,et al.  MANAGERIAL DECISION MAKING ABOUT EMPLOYEE DISCIPLINE: A POLICY‐CAPTURING APPROACH , 1990 .

[15]  Mark V. Roehling,et al.  “EXTRACTING” POLICY FROM JUDICIAL OPINIONS: THE DANGERS OF POLICY CAPTURING IN A FIELD SETTING , 1993 .

[16]  C. Fombrun,et al.  What's in a Name? Reputation Building and Corporate Strategy , 1990 .

[17]  F. W. Gibson,et al.  Policy capturing as an approach to understanding and improving performance appraisal: a review of the literature. , 1983, Academy of management review. Academy of Management.

[18]  Stephen A. Stumpf,et al.  Capturing Rater Policies in Evaluating Candidates for Promotion , 1981 .

[19]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[20]  S. M. Lewis,et al.  Orthogonal Fractional Factorial Designs , 1986 .

[21]  Thomas W. Zimmerer,et al.  Modeling Strategic Acquisition Policies: A Simulation of Executives' Acquisition Decisions , 1984 .

[22]  Paul Slovic,et al.  Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment. , 1971 .

[23]  Michael J. Stahl,et al.  Using decision modeling to measure second level valences in expectancy theory , 1983 .

[24]  A. Perl,et al.  Code of Ethics , 1963, Science.

[25]  J. Neter,et al.  Applied Linear Statistical Models (3rd ed.). , 1992 .

[26]  F. Yates.  Incomplete randomized blocks , 1936 .

[27]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[28]  John J. Lawler,et al.  A policy-capturing investigation of the role of expectancies in decisions to pursue job alternatives. , 1983 .

[29]  Paul M. Muchinsky,et al.  Intrasubject predictions of vocational preference: Convergent validation via the decision theoretic paradigm , 1985 .

[30]  Timothy A. Judge,et al.  When We Don’t See Eye to Eye: Discrepancies between Supervisors and Subordinates in Absence Disciplinary Decisions , 1995 .

[31]  Michael H. Birnbaum,et al.  How to show that 9 > 221 : Collect judgments in a between-subjects design , 1999 .

[32]  S. Zedeck,et al.  Capturing rater policies for processing evaluation data. , 1977, Organizational behavior and human performance.