The Incremental and Predictive Validity of the Rorschach Test in Personality Assessments of Normal, Neurotic and Psychotic Subjects.

The purpose of this study was to determine the optimal amount of Rorschach protocol data required for maximal accuracy in person­ ality assessment when experience level of judges and degree of pathology of the subjects are taken into account. Confidence in judgment as it relates to the same three factors was also examined. One hundred forty-four judges, consisting of 36 untrained undergraduates, 36 senior undergraduate psychology majors, 36 graduate clinical psychology students, and 36 Ph.D.’s who were Fellows of the Society of Projective Techniques, or had indicated a special interest in projective techniques in the APA Directory, were used. Six Rorschach tests were administered to persons represent­ ing Normal, Neurotic and Psychotic states of mental health. They were selected on the basis of their life situations, treatment condi­ tion, and results on the MMPI. Case data was then divided into the following separate increments: 1 -Basic Identifying Data (BID): age, sex, level of educa­ tion, marital status, past and present treatment status, and occupation. 2 BID plus Free Association and the Location Sheet (FA and Loc). 3 BID, FA, Loc, and Scoring based upon FA (Sc). 4 BID, FA, Loc and Inquiry (Inq). 5 BID. FA, Inq, Loc and Scoring based upon FA and Inq (Scor). 6 BID, FA, Inq, Loc, Scor and Testing of the Limits. Each judge received a packet containing only one of the foregoing information conditions on a single case. After consider­ ing the data, a judge was required to delineate the protocol subject, utilizing the California Q-sort as a descriptive device. The . criterion for each case consisted of a consensual CQ-sort obtained from persons well acquainted with the subjects. Confidence in judgment was expressed quantitatively In the form of "percentage right," and subjectively in the form of selfevaluation of performance on a .5 point scale ranging from 5 for "Very Well" to 1 for "Very Poorly." It was found that; 1 There was an inverse relationship between level of train­ ing and judgmental accuracy, regardless of type of proto­ col or amount of information available. 2 Increased amounts of data resulted in decreased levels of accuracy. 3 BID used by itself was productive of more accuracy than any of the Rorschach data conditions. It exceeded Total Rorschach Data at the .01 level, and the FA/Inq combina­ tion at the .05 level of significance. 4 The best Rorschach data source for use with Normals and Neurotics was the FA/Sc combination. Verbalization was more crucial in the assessment of Psychotics and the FA condition by itself was most productive for that level of pathology. 5 Psychotics could be discriminated from the Normals at the .01 level, and they could be discriminated from the Neurotics at the .05 level of significance. In general, there was a positive relationship between accuracy and the degree of pathology of the subject. 6 Less trained judges were most accurate with Normal re­ cords, and more trained judges were most accurate with Psychotic records. 7 There was positive relationship between training and the tendency to over-estimate pathology of subjects. 8 Confidence did not increase as the amount of data in­ creased, and it did not alter with regard to type of protocol. 9 Most of the judges estimated their performances as being Fair or average, and thus no difference in subjective estimates of confidence was found. There was an inverse relationship between the quantitative estimate of confid­ ence and training, however. This difference appeared to be a reflection of degree of sophistication in statistical matters rather than differential confidence levels. 10 Rorschach accuracy and Social Desirability of the subject were related positively, 11 Genotypic as well as phenotypic characteristics were missed by the more trained judges.

[1]  G. Stricker Actuarial, naive clinical, and sophisticated clinical prediction of pathology from figure drawings. , 1967, Journal of consulting psychology.

[2]  Sidney E. Zimbalist Clinical and Social Judgment: The Discrimination of Behavioral Information. By James Bieri, Alvin L. Atkins, Scott Briar, Robin Lobeck Leaman, Henry Miller, and Tony Tripodi. New York: John Wiley & Sons, 1966. 271 pp. $7.95 , 1967 .

[3]  R Johnston,et al.  Statistical versus clinical prediction: length of neuropsychiatric hospital stay. , 1967, Journal of abnormal psychology.

[4]  T. Tripodi,et al.  Information Accrual and Clinical Judgment , 1967 .

[5]  R. W. Wildman The practice of clinical psychology in the United States. , 1967, Journal of clinical psychology.

[6]  R. Fancher Accuracy versus validity in person perception. , 1967, Journal of consulting psychology.

[7]  B Kleinmuntz,et al.  Sign and seer: another example. , 1967, Journal of abnormal psychology.

[8]  J. Rotter Can the clinician learn from experience , 1967 .

[9]  J. R. Tomlinson Situational and personality correlates of predictive accuracy. , 1967 .

[10]  K. Little Research etiquette in the study of clinician's behavior. , 1967 .

[11]  J. Sawyer,et al.  Measurement and prediction, clinical and statistical. , 1966, Psychological bulletin.

[12]  R. Fancher Explicit personality theories and accuracy in person perception. , 1966, Journal of personality.

[13]  D. Turner Predictive efficiency as a function of amount of information and level of professional experience. , 1966, Journal of projective techniques & personality assessment.

[14]  S. Oskamp OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[15]  J. R. Newton JUDGMENT AND FEEDBACK IN A QUASI-CLINICAL SITUATION. , 1965, Journal of personality and social psychology.

[16]  R. Leonard,et al.  Foundations of Behavioral Research; Educational and Psychological Inquiry , 1965 .

[17]  R. McHugh,et al.  CONTRIBUTION OF STEREOTYPED AND INDIVIDUALIZED INFORMATION TO PREDICTIVE ACCURACY. , 1964, Journal of consulting psychology.

[18]  T. Tripodi,et al.  INFORMATION TRANSMISSION IN CLINICAL JUDGMENTS AS A FUNCTION OF STIMULUS DIMENSIONALITY AND COGNITIVE COMPLEXITY. , 1964, Journal of personality.

[19]  Jack Block,et al.  The Q-sort method in personality assessment and psychiatric research , 1964 .

[20]  K. Howard,et al.  Ratings of Projective Test Protocols as a Function of Degree of Inference , 1963 .

[21]  R. Stoller,et al.  THE CONSISTENCY OF PSYCHIATRISTS' CLINICAL JUDGMENTS , 1963, The Journal of nervous and mental disease.

[22]  J. D. Pauker Relationship of Rorschach content categories to intelligence. , 1963, Journal of projective techniques.

[23]  J. Weiss Effect of professional training and amount and accuracy of information on behavioral prediction. , 1963, Journal of consulting psychology.

[24]  L. Sechrest Incremental Validity : a Recommendation , 1963 .

[25]  M. Horowitz A study of clinicians' judgments from projective test protocols. , 1962, Journal of consulting psychology.

[26]  K. Howard,et al.  The convergent and discriminant validation of ipsative ratings from three projective instruments. , 1962, Journal of clinical psychology.

[27]  James C. Mcgreevey Interlevel Disparity and Predictive Efficiency , 1962 .

[28]  Z. Sperber,et al.  The accuracy of clinical psychologists' estimates of interviewees' intelligence. , 1961, Journal of consulting psychology.

[29]  Edward Zigler,et al.  Psychiatric diagnosis: A critique. , 1961 .

[30]  W. A. Hunt,et al.  THE EXPERIMENTAL INVESTIGATION OF CLINICAL JUDGMENT , 1961 .

[31]  F. Attneave Applications of information theory to psychology: A summary of basic concepts, methods, and results. , 1961 .

[32]  N. Sundberg The practice of psychological testing in clinical services in the United States. , 1961 .

[33]  H. Stone,et al.  An exploratory investigation into the levels hypothesis. , 1960, Journal of projective techniques.

[34]  F. Shontz,et al.  Diagnostic prototypes and diagnostic processes of clinical psychologists. , 1960, Journal of consulting psychology.

[35]  R. Stoller,et al.  The Objective Assessment of Clinical Judgment in Psychiatry , 1960 .

[36]  P. Hoffman The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[37]  L. Sines The relative contribution of four kinds of data to accuracy in personality assessment. , 1959 .

[38]  Arne Trankell,et al.  The psychologist as an instrument of prediction. , 1959 .

[39]  A. Montagu A Comprehensive Dictionary of Psychological and Psychoanalytical Terms , 1959 .

[40]  N. Tallent On individualizing the psychologist's clinical evaluation. , 1958, Journal of clinical psychology.

[41]  S. J. Beck,et al.  Further explorations of the six schizophrenias: type S-3. , 1958, The American journal of orthopsychiatry.

[42]  W. T. Powers,et al.  A comparative analysis of deviant Rorschach response characteristics. , 1958, Journal of consulting psychology.

[43]  H. Leventhal,et al.  Cognitive processes and interpersonal predictions. , 1957, Journal of abnormal psychology.

[44]  J K ADAMS,et al.  A confidence scale defined in terms of expected percentages. , 1957, The American journal of psychology.

[45]  W. T. Powers,et al.  The validity, bases, and process of clinical judgment, using a limited amount of projective test data. , 1957, Journal of projective techniques.

[46]  K. R. Hammond,et al.  The generality of accuracy and response sets in interpersonal perception. , 1957, Journal of abnormal psychology.

[47]  W. Kessen The role of experience in judging children's photographs. , 1957, Journal of abnormal psychology.

[48]  D. W. Fiske,et al.  Factors influencing the prediction of behavior from a diagnostic interview. , 1957, Journal of consulting psychology.

[49]  Jack Block,et al.  A Comparison of the Forced and Unforced Q-Sorting Procedures , 1956 .

[50]  D. More,et al.  Does TAT add information to interviews? Statistical analysis of the increment. , 1956, Journal of clinical psychology.

[51]  Donald M. Johnson The psychology of thought and judgment , 1956 .

[52]  A. Jones Distributions of traits in current Q-sort methodology. , 1956, Journal of abnormal psychology.

[53]  F. Shontz Evaluative conceptualizations as the basis for clinical judgments. , 1956, Journal of consulting psychology.

[54]  F. Kerlinger The attitude structure of the individual: a Q-study of the educational attitudes of professors and laymen. , 1956, Genetic psychology monographs.

[55]  N. Livson,et al.  Discrimination and reliability in Q-sort personality descriptions. , 1956, Journal of abnormal psychology.

[56]  S. Hathaway Clinical intuition and inferential accuracy. , 1956, Journal of personality.

[57]  H. O. Schmidt,et al.  The reliability of psychiatric diagnosis; a new look. , 1956, Journal of abnormal psychology.

[58]  A. L. Edwards Social desirability and Q sorts. , 1955, Journal of consulting psychology.

[59]  W. Stephenson The study of behavior : Q-technique and its methodology , 1955 .

[60]  L. Cronbach,et al.  Conceptual and methodological problems in interpersonal perception. , 1955, Psychological review.

[61]  J. Bieri Cognitive complexity-simplicity and predictive behavior. , 1955, Journal of abnormal psychology.

[62]  Paul E. Meehl,et al.  Clinical versus Statistical Prediction. , 1955 .

[63]  J. Regensburg : The Six Schizophrenias: Reaction Patterns in Children and Adults , 1955 .

[64]  L. Cronbach Processes affecting scores on understanding of others and assumed similarity. , 1955, Psychological bulletin.

[65]  W. T. Powers,et al.  Relationship between diagnostic category and deviant verbalizations on the Rorschach. , 1955, Journal of consulting psychology.

[66]  A. B. Monro Psychiatric types: a Q-technique study of 200 patients. , 1955, The Journal of mental science.

[67]  K. Little,et al.  The validity of thematic projective technique interpretations. , 1955, Journal of personality.

[68]  V. B. Cline,et al.  Ability to judge personality assessed with a stress interview and sound-film technique. , 1955, Journal of abnormal psychology.

[69]  S. B. Sells,et al.  Prediction of flying success by clinical analysis of test protocols. , 1954, Journal of abnormal psychology.

[70]  J. Phelan,et al.  The skills of clinicians in analysis of projective tests. , 1954, Journal of clinical psychology.

[71]  R. Hamlin,et al.  The clinician as judge: details of procedure in judging projective material. , 1954, Journal of consulting psychology.

[72]  R. Newton The clinician as judge: total Rorschachs and clinical case material. , 1954, Journal of consulting psychology.

[73]  F. N. Arnhoff Some factors influencing the unreliability of clinical judgments. , 1954, Journal of clinical psychology.

[74]  E. Robins,et al.  Prediction in the clinical method and interrelations of biochemistry, psychiatry, and psychology. , 1954, Journal of abnormal psychology.

[75]  A. Kostlán A method for the empirical study of psychodiagnosis. , 1954, Journal of consulting psychology.

[76]  J. Mosel,et al.  The comparability of measures of profile similarity: an empirical study. , 1954, Journal of consulting psychology.

[77]  R. C. Oldfield,et al.  Memory mechanisms and the theory of schemata. , 1954, The British journal of medical psychology.

[78]  A. L. Edwards,et al.  Social Desirability as a Variable in 2 Technique Studies , 1953 .

[79]  R. Dymond An adjustment score for Q sorts. , 1953, Journal of consulting psychology.

[80]  R. Gibby,et al.  The relation of Rorschach free association to inquiry. , 1953, Journal of consulting psychology.

[81]  R. Dymond Can clinicians predict individual behavior? , 1953, Journal of personality.

[82]  K. Bradway,et al.  The relation between diagnoses and certain types of extreme deviations and content on the Rorschach. , 1953, Journal of projective techniques.

[83]  J. G. Watkins,et al.  An index of pathological thinking in the Rorschach. , 1952, Journal of projective techniques.

[84]  W. Stephenson Some observations on Q technique. , 1952, Psychological bulletin.

[85]  D. Johnson,et al.  Experimental Analysis of the Reliability of Observations of Social Behavior , 1952 .

[86]  A. L. Edwards,et al.  Experimental Design in Psychological Research. , 1951 .

[87]  J. Luft Differences in prediction based on hearing versus reading verbatim clinical interviews. , 1951, Journal of consulting psychology.

[88]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[89]  C. L. Odom A study of the time required to do a Rorschach examination. , 1950, Journal of projective techniques.

[90]  L. Cronbach Statistical methods applied to Rorschach scores; a review. , 1949, Psychological bulletin.

[91]  T. Sarbin The logic of prediction in psychology. , 1944 .

[92]  T. Sarbin A Contribution to the Study of Actuarial and Individual Methods of Prediction , 1943, American Journal of Sociology.

[93]  George A. Lundberg,et al.  Case-Studies Vs. Statistical Methods-An Issue Based on Misunderstanding , 1941 .

[94]  S. G. Estes Judging personality from expressive behavior. , 1938 .

[95]  E. Boring A History of Experimental Psychology. , 1930 .

[96]  D. J. Watley Do counselors know when to use their heads instead of the formula , 1967 .

[97]  D. J. Watley Counselor predictive skill and differential judgments of occupational suitability. , 1967 .

[98]  C. J. Krauskopf,et al.  Clinical prediction and immediate feedback training. , 1967 .

[99]  F. W. Huff,et al.  A study of the effects on agreement among clinicians of redundant and new information, confidence, and time available for assessments. , 1967, The Journal of general psychology.

[100]  R. Atta A method for the study of clinical thinking. , 1966 .

[101]  D. J. Watley Counselor confidence in accuracy of predictions. , 1966 .

[102]  D. J. Watley Counselor variability in making accurate predictions. , 1966 .

[103]  C. Bartlett,et al.  Clinical prediction: Does one sometimes know too much? , 1966 .

[104]  F. Shontz Research methods in personality , 1965 .

[105]  Paul E. Meehl,et al.  Seer over sign: The first good example. , 1965 .

[106]  G. Lindzey,et al.  Seer versus sign. , 1965 .

[107]  F. V. D. Veen The parent's concept of the family unit and child adjustment. , 1965 .

[108]  J. Richards,et al.  Cline and Richards' reply to O'Connor's methodological note. , 1963 .

[109]  W. Neff,et al.  A Q-sort instrument to assess the meaning of work. , 1963 .

[110]  W. J. Mueller The influence of self insight on social perception scores. , 1963 .

[111]  W. F. O'connor A methodological note on the Cline and Richards' studies on accuracy of interpersonal perception. , 1963 .

[112]  Stuart Oskamp,et al.  The relationship of clinical experience and training methods to several criteria of clinical prediction. , 1962 .

[113]  D. Sundland The construction of Q sorts: a criticism. , 1962, Psychological review.

[114]  J. C. Lee,et al.  An investigation of clinical judgment: a study in method. , 1962, Journal of abnormal and social psychology.

[115]  L. Salzman,et al.  Changes in clinical judgment as a function of psychiatric education. , 1961, Journal of medical education.

[116]  P. Meehl The cognitive activity of the clinician. , 1960 .

[117]  T. Sarbin,et al.  Clinical Inference And Cognitive Theory , 1960 .

[118]  J. Richards,et al.  Accuracy of interpersonal perception--a general trait? , 1960, Journal of abnormal and social psychology.

[119]  K. Little,et al.  Congruencies among interpretations of psychological test and anamnestic data. , 1959 .

[120]  W. F. Soskin Influence of four types of data on diagnostic conceptualization in psychological testing. , 1959, Journal of abnormal psychology.

[121]  Paul E. Meehl,et al.  A comparison of clinicians with five statistical methods of identifying psychotic MMPI profiles. , 1959 .

[122]  Paul E. Meehl,et al.  Some ruminations on the validation of clinical procedures. , 1959 .

[123]  D. Brieland An experimental study of the selection of adoptive parents at intake : a study conducted and published in cooperation with the Child Welfare League of America , 1959 .

[124]  R. Holt Clinical and statistical prediction; a reformulation and some new data. , 1958, Journal of abnormal psychology.

[125]  C. Parker As a clinician thinks. , 1958 .

[126]  J. Block,et al.  Accuracy of interpersonal prediction as a function of judge and object characteristics. , 1957, Journal of abnormal psychology.

[127]  Paul E. Meehl,et al.  When shall we use our heads instead of the formula , 1957 .

[128]  J. A. Bath,et al.  Clinical and actuarial predictions of academic survival and attrition. , 1957 .

[129]  J. Block A comparison between ipsative and normative ratings of personality. , 1957, Journal of abnormal psychology.

[130]  W. Crow The effect of training upon accuracy and variability in interpersonal perception. , 1957, Journal of abnormal psychology.

[131]  R. Taft,et al.  The ability to judge people. , 1955, Psychological bulletin.

[132]  S. T. Cummings The clinician as judge: judgments of adjustment from Rorschach single-card performance. , 1954, Journal of consulting psychology.

[133]  R. Schafer Psychoanalytic Interpretation in Rorschach Testing , 1954 .

[134]  W. F. Soskin Bias in postdiction from projective tests. , 1954, Journal of abnormal psychology.

[135]  D. Chaffee Statistical Analysis of , 1954 .

[136]  O. Mowrer Psychotherapy: theory and research , 1953 .

[137]  S. E. Jacobs The Prediction of Performance in Clinical Psychology. , 1953 .

[138]  C. A. Dailey The Effects of Premature Conclusion Upon the Acquisition of Understanding of a Person , 1952 .

[139]  Paul Wallin,et al.  Supplementary study A: The prediction of individual behavior from case studies. , 1941 .

[140]  C. W. Valentine THE RELATIVE RELIABILITY OF MEN AND WOMEN IN INTUITIVE JUDGMENTS OF CHARACTER1 , 1929 .

[141]  L. Beach,et al.  Man as an Intuitive Statistician , 2022 .