Measuring the importance of cues in policy capturing

Abstract Regression analysis has often been used to “capture” the decision policies of raters or judges. The usefulness of policy capturing generally has been thought to be limited to the case in which the evaluative cues are uncorrelated; otherwise, the various indices of cue importance often do not agree. We argue that the raw-score regression weight is the most appropriate index of importance in that it represents the expected change in overall rating per unit change on a given cue and is invariant across changes in the cue intercorrelations. To test the proposition that raw-score regression weights do not change as a function of cue structure, the decision policies of 14 subjects were captured under three cue intercorrelation structures. The raw-score regression weights did not change significantly as a function of cue structure, whereas the simple correlations, the semipartial correlations, and the standardized regression weights did.

[1]  R. Dawes.  A case study of graduate admissions: Application of three principles of human decision making. , 1971 .

[2]  Arthur L. Dudycha,et al.  The Effect of Variations in the Cue R Matrix Upon the Obtained Policy Equation of Judges , 1966 .

[3]  Todd E. Marques,et al.  Toward the development of a system for instructional evaluation: Is there consensus regarding what constitutes effective teaching? , 1979 .

[4]  R. Darlington,et al.  Multiple regression in psychological research and practice. , 1968, Psychological bulletin.

[5]  R. B. Darlington,et al.  Assessing the Importance of Independent Variables in Nonlinear Causal Laws1 , 1972 .

[6]  J. H. Ward,et al.  Comments on "the paramorphic representation of clinical judgment". , 1962, Psychological bulletin.

[7]  H. J. Einhorn,et al.  Linear regression and process-tracing models of judgment. , 1979 .

[8]  J. Shanteau,et al.  Livestock judges: How much information can an expert use? , 1978 .

[9]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[10]  R. D. Bock,et al.  Multivariate Statistical Methods in Behavioral Research , 1978 .

[11]  P. Hoffman.  The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[12]  N. Schmitt,et al.  Statistical and subjective weights. Some problems and proposals , 1977 .

[13]  B. J. Winer.  Statistical Principles in Experimental Design , 1992 .

[14]  Robert J. Wherry,et al.  The Use of Simulated Stimuli and the "Jan" Technique to Capture and Cluster the Policies of Raters , 1965 .

[15]  J. C. Naylor,et al.  A Cautionary Note Concerning the Use of Regression Analysis for Capturing the Strategies of People , 1968 .