Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies

The mathematical representation of Brunswik's lens model has been used extensively to study human judgment and provides a unique opportunity to conduct a meta-analysis of studies that covers roughly five decades. Specifically, we analyze statistics of the lens model equation (Tucker, 1964) associated with 259 different task environments obtained from 78 papers. In short, we find - on average - fairly high levels of judgmental achievement and note that people can achieve similar levels of cognitive performance in both noisy and predictable environments. Although overall performance varies little between laboratory and field studies, both differ in terms of components of performance and types of environments (numbers of cues and redundancy). An analysis of learning studies reveals that the most effective form of feedback is information about the task. We also analyze empirically when bootstrapping is more likely to occur. We conclude by indicating shortcomings of the kinds of studies conducted to date, limitations in the lens model methodology, and possibilities for future research.

[1]  P. Freebody,et al.  Assessment as Judgment-in-Context: Analysing how teachers evaluate students' writing 1 , 2007 .

[2]  Frada Burstein,et al.  Decision support in an uncertain and complex world , 2007, Decis. Support Syst..

[3]  Anna Cuxart,et al.  What Risks Do People Perceive in Everyday Life? A Perspective Gained from the Experience Sampling Method (ESM) , 2007, Risk analysis : an official publication of the Society for Risk Analysis.

[4]  M. Mehl,et al.  The lay assessment of subclinical depression in daily life. , 2006, Psychological assessment.

[5]  Natalia Karelaia,et al.  On Heuristic and Linear Models of Judgment: Mapping the Demand for Knowledge , 2006 .

[6]  S. Gosling,et al.  Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life. , 2006, Journal of personality and social psychology.

[7]  R. Youmans,et al.  To thy own self be true: finding the utility of cognitive information feedback , 2005 .

[8]  Kai H. Lim,et al.  Improving judgmental forecasts with judgmental bootstrapping and task feedback support , 2005 .

[9]  Reid Hastie,et al.  The robust beauty of majority rules in group decisions. , 2005, Psychological review.

[10]  Robin M. Hogarth,et al.  Information Sampling and Adaptive Cognition: Is Confidence in Decisions Related to Feedback? Evidence from Random Samples of Real-World Behavior , 2005 .

[11]  Mandeep K. Dhami,et al.  The role of representative design in an ecological approach to cognition. , 2004, Psychological bulletin.

[12]  E. Mullet,et al.  Functional learning among children, adolescents, and young adults. , 2004, Journal of experimental child psychology.

[13]  S. Gosling,et al.  e-Perceptions : Personality Impressions Based on Personal Websites , 2004 .

[14]  Ann M. Bisantz,et al.  Measuring the Fit between Human Judgments and Automated Alerting Algorithms: A Study of Collision Detection , 2003, Hum. Factors.

[15]  Alex Kirlik,et al.  A Systems Perspective on Situation Awareness II: Experimental Evaluation of a Modeling and Measurement Technique , 2003 .

[16]  K. Gilhooly,et al.  Factors influencing prescribing decisions in the treatment of depression: a social judgement theory approach , 2003 .

[17]  Alex Kirlik,et al.  Inferring rule-based strategies in dynamic judgment tasks: toward a noncompensatory formulation of the lens model , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[18]  S. Gosling,et al.  A room with a cue: personality judgments based on offices and bedrooms. , 2002, Journal of personality and social psychology.

[19]  U. Hoffrage,et al.  Fast, frugal, and fit: Simple heuristics for paired comparison , 2002 .

[20]  Michael D. Shields,et al.  Why Does Fixation Persist? Experimental Evidence on the Judgment Performance Effects of Expensing Intangibles , 2001 .

[21]  R. Cooksey,et al.  Judgment of Factors Influencing Interest: An Australian Study , 2001 .

[22]  Arthur D. Fisk,et al.  Modeling and analysis of a dynamic judgment task using a lens model approach , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[23]  Robert P. Mahan,et al.  Cognitive adaptation and its consequences: a test of cognitive continuum theory , 2000 .

[24]  E. Mullet,et al.  How well do elderly people cope with uncertainty in a learning task? , 1999, Acta psychologica.

[25]  Ulrich Hoffrage,et al.  Why does one-reason decision making work? A case study in ecological rationality , 1999 .

[26]  R. Tweney,et al.  The Role of Data and Feedback Error in Inference and Prediction , 1998 .

[27]  T. R. Stewart,et al.  Aging and multiple cue probability learning: the case of inverse relationships. , 1997, Acta psychologica.

[28]  T. R. Stewart,et al.  The Importance of the Task in Analyzing Expert Judgment , 1997 .

[29]  W. Remus,et al.  Does feedback improve the accuracy of recurrent judgmental forecasts? , 1996, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[30]  D Kahneman,et al.  On the reality of cognitive illusions. , 1996, Psychological review.

[31]  G. Gigerenzer On Narrow Norms and Vague Heuristics: A Reply to Kahneman and Tversky (1996) , 1996 .

[32]  Ray W. Cooksey,et al.  Judgment analysis : theory, methods, and applications , 1996 .

[33]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[34]  B. Brehmer The psychology of linear judgement models , 1994 .

[35]  Robert Gifford,et al.  A lens-mapping framework for understanding the encoding and decoding of interpersonal dispositions in nonverbal behavior. , 1994 .

[36]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[37]  Michael E. Doherty,et al.  The assessment of self-insight in judgment policies , 1992 .

[38]  Leslie B. Hammer,et al.  Task information, cognitive information, or functional validity information: Which components of cognitive feedback affect performance?☆ , 1992 .

[39]  T G Tape,et al.  Comparing Methods of Learning Clinical Prediction from Case Simulations , 1992, Medical decision making : an international journal of the Society for Medical Decision Making.

[40]  N. Castellan,et al.  Relations between linear models: Implications for the lens model , 1992 .

[41]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[42]  J. Frank Yates,et al.  How Quantity Judgment Changes as the Number of Cues Increases: An Analytical Framework and Review , 1992 .

[43]  J P Ornato,et al.  Use of Clinical Judgment Analysis to Explain Regional Variations in Physicians' Accuracies in Diagnosing Pneumonia , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[44]  P. Werner,et al.  Predicting Violence in Newly Admitted Inmates , 1990 .

[45]  Peter Freebody,et al.  The Ecology of Spelling: A Lens Model Analysis of Spelling Errors and Student Judgments of Spelling Difficulty. , 1990 .

[46]  R S Wigton,et al.  Teaching old dogs new tricks: using cognitive feedback to improve physicians' diagnostic judgments on simulated cases , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[47]  M. Doherty,et al.  Effects of cognitive feedback on performance. , 1989 .

[48]  B. Brehmer,et al.  Human judgment : the SJT view , 1988 .

[49]  Paul Slovic,et al.  Graphic Representation of Judgmental Information , 1986, SGCH.

[50]  Berndt Brehmer,et al.  Use of experts in complex decision making: A paradigm for the study of staff work , 1986 .

[51]  Peter Freebody,et al.  Teachers’ Predictions of Children’s Early Reading Achievement: An Application of Social Judgment Theory , 1986 .

[52]  Ronald J. Ebert,et al.  Policy capturing in the employment interview. , 1986 .

[53]  H. Rothstein The effects of time pressure on judgment in multiple cue probability learning , 1986 .

[54]  Mary W. Downton,et al.  Judgments of photographs vs. field observations in studies of perception and judgment of the visual environment , 1984 .

[55]  A. H. Ashton,et al.  An Empirical-Study Of Budget-Related Predictions Of Corporate-Executives , 1982 .

[56]  L. Cohen Can human irrationality be experimentally demonstrated? , 1981, Behavioral and Brain Sciences.

[57]  Colin F. Camerer,et al.  General conditions for the success of bootstrapping models , 1981 .

[58]  L. Adelman The influence of formal, substantive, and contextual task properties on the relative effectiveness of different forms of feedback in multiple-cue probability learning tasks☆ , 1981 .

[59]  Paul Slovic,et al.  Multidimensional functional learning (MFL) and some new conceptions of feedback , 1981 .

[60]  R. Ashton A Descriptive Study Of Information Evaluation , 1981 .

[61]  R. Ashton,et al.  Feedback And Prediction Achievement In Financial Analysis , 1981 .

[62]  B. Brehmer In one word: Not from experience. , 1980 .

[63]  P. Koele The influence of labeled stimuli on nonlinear multiple-cue probability learning , 1980 .

[64]  Berndt Brehmer,et al.  Content and consistency in probabilistic inference tasks , 1980 .

[65]  H. J. Einhorn,et al.  Linear regression and process-tracing models of judgment. , 1979 .

[66]  William F. Wright Properties of judgment models in a financial setting , 1979 .

[67]  William H. Clover,et al.  Can we learn anything about interviewing real people from “interviews” of paper people? Two studies of the external validity of a paradigm☆ , 1978 .

[68]  Terry Connolly,et al.  Some Effects of Feedback Error BV Diagnostic Decision Tasks , 1978 .

[69]  Ronald J. Ebert,et al.  Bootstrapping the security analyst. , 1978 .

[70]  Neal Schmitt,et al.  Types of task information feedback in multiple-cue probability learning. , 1977 .

[71]  K. Rudestam,et al.  Effects of Amounts and Units of Information on the Judgmental Process , 1976 .

[72]  M. Lindell Cognitive and Outcome Feedback in Multiple-Cue Probability Learning Tasks. , 1976 .

[73]  B. Brehmer Note on clinical judgment and the formal characteristics of clinical tasks. , 1976 .

[74]  Michael E. Doherty,et al.  Judgment theory applied to the selection of life insurance salesmen , 1976 .

[75]  Larry King,et al.  Feedback and task predictability as determinants of performance in multiple cue probability learning tasks , 1976 .

[76]  Michael E. Doherty,et al.  Feedback effects in a metric multiple-cue probability learning task. , 1976 .

[77]  Robert Libby,et al.  Man versus model of man: some conflicting evidence , 1976 .

[78]  Derick O. Steinmann The effects of cognitive feedback and task complexity in multiple-cue probability learning , 1976 .

[79]  P. Muchinsky,et al.  Human inference behavior in abstract and meaningful environments , 1975 .

[80]  R. C. Sherman,et al.  Effect of Lens-Model and Outcome Feedback in a Social Judgment Analogue , 1974 .

[81]  B. Brehmer The effect of cue intercorrelation on interpersonal learning of probabilistic inference tasks , 1974 .

[82]  B. Armelius,et al.  The use of redundancy in multiple-cue judgments: Data from a suppressor-variable task. , 1974 .

[83]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[84]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[85]  Berndt Brehmer,et al.  Effects of task predictability and cue validity on interpersonal learning of inference tasks involving both linear and nonlinear relations , 1973 .

[86]  B. Brehmer Effects of Cue Validity on Interpersonal Learning of Inference Tasks with Linear and Nonlinear Cues , 1973 .

[87]  N. John CastellanJr.,et al.  Comments on the “lens model” equation and the analysis of multiple-cue judgment tasks , 1973 .

[88]  D. Magnusson,et al.  Cue relevance and feedback in a clinical prediction task , 1973 .

[89]  Kenneth R. Hammond,et al.  Negative effects of outcome-feedback in multiple-cue probability learning. , 1973 .

[90]  Derick O. Steinmann,et al.  A lens model analysis of a bookbag and poker chip experiment: A methodological note , 1972 .

[91]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[92]  Hillel J. Einhorn,et al.  Expert measurement and mechanical combination , 1972 .

[93]  Donald H. Deane,et al.  Acquisition and application of knowledge in complex inference tasks. , 1972 .

[94]  N. Wiggins,et al.  Man versus model of man revisited: The forecasting of graduate school success. , 1971 .

[95]  R. Dawes A case study of graduate admissions: Application of three principles of human decision making. , 1971 .

[96]  H. J. Einhorn Use of nonlinear, noncompensatory models as a function of task and amount of information , 1971 .

[97]  Lewis R. Goldberg,et al.  Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences. , 1970 .

[98]  Berndt Brehmer,et al.  Cognitive Dependence on Additive and Configural Cue-Criterion Relations , 1969 .

[99]  Rita C. Summers,et al.  Judgments Based on Different Functional Relationships between Interacting Cues and a Criterion , 1969 .

[100]  J. C. Naylor,et al.  The influence of cue redundancy upon the human inference process for tasks of varying degrees of predictability , 1968 .

[101]  C. N. Uhl Effects of multiple stimulus validity and criterion dispersion on learning of interval concepts. , 1966, Journal of experimental psychology.

[102]  J. C. Naylor,et al.  Characteristics of the human inference process in complex choice behavior situations , 1966 .

[103]  K. R. Hammond,et al.  A research paradigm for the study of interpersonal learning. , 1966, Psychological bulletin.

[104]  K. R. Hammond,et al.  COGNITIVE DEPENDENCE ON LINEAR AND NONLINEAR CUES. , 1965, Psychological review.

[105]  J. R. Newton JUDGMENT AND FEEDBACK IN A QUASI-CLINICAL SITUATION. , 1965, Journal of personality and social psychology.

[106]  L. Tucker A SUGGESTED ALTERNATIVE FORMULATION IN THE DEVELOPMENTS BY HURSCH, HAMMOND, AND HURSCH, AND BY HAMMOND, HURSCH, AND TODD. , 1964, Psychological review.

[107]  K. R. Hammond,et al.  ANALYZING THE COMPONENTS OF CLINICAL INFERENCE. , 1964, Psychological review.

[108]  K. R. Hammond,et al.  SOME METHODOLOGICAL CONSIDERATIONS IN MULTIPLE-CUE PROBABILITY STUDIES. , 1964, Psychological review.

[109]  L. Grebstein Relative accuracy of actuarial prediction, experienced clinicians, and graduate students in a clinical judgment task. , 1963, Journal of consulting psychology.

[110]  D. B. Yntema,et al.  Man-Computer Cooperation in Decisions Requiring Common Sense , 1961 .

[111]  P. Hoffman The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[112]  E. Brunswik,et al.  The Conceptual Framework of Psychology , 1954 .

[113]  E. Brunswik Organismic achievement and environmental probability. , 1943 .