Psychological Review Verbal Reports as Data

The central proposal of this article is that verbal reports are data. Accounting for verbal reports, as for other kinds of data, requires explication of the mechanisms by which the reports are generated, and the ways in which they are sensitive to experimental factors (instructions, tasks, etc.). Within the theoretical framework of human information processing, we discuss different types of processes underlying verbalization and present a model of how subjects, in response to an instruction to think aloud, verbalize information that they are attending to in short-term memory (STM). Verbalizing information is shown to affect cognitive processes only if the instructions require verbalization of information that would not otherwise be attended to. From an analysis of what would be in STM at the time of report, the model predicts what can reliably be reported. The inaccurate reports found by other research are shown to result from requesting information that was never directly heeded, thus forcing subjects to infer rather than remember their mental processes.

[1]  N. Maier Reasoning in humans. II. The solution of a problem and its appearance in consciousness. , 1931 .

[2]  Mary Elizabeth Bulbrook An Experimental Inquiry into the Existence and Nature of 'Insight' , 1932 .

[3]  H. Rees,et al.  An investigation of the establishment and operation of mental sets. , 1935 .

[4]  R. Thouless Experimental Psychology , 1939, Nature.

[5]  C. Buxton,et al.  The Psychology of Efficiency. , 1944 .

[6]  G. Hendrix A New Clue to Transfer of Training , 1947, The Elementary School Journal.

[7]  R. D. Walk Effect of discrimination reversal on human discrimination learning. , 1952, Journal of experimental psychology.

[8]  B. R. Bugelski,et al.  An experimental demonstration of unconscious mediated association. , 1952, Journal of experimental psychology.

[9]  J. C. Flanagan Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[10]  J. Greenspoon,et al.  The reinforcing effect of two spoken sounds on the frequency of two responses. , 1955, The American journal of psychology.

[11]  W. Russell,et al.  Implicit verbal chaining in paired--associate learning. , 1955, Journal of experimental psychology.

[12]  A correlational study of two reasoning problems. , 1958, Journal of experimental psychology.

[13]  STAGES IN CONCEPT FORMATION AND LEVELS OF COGNITIVE FUNCTIONING , 1960 .

[14]  Eva I. Shipstone Some variables affecting pattern conception. , 1960 .

[15]  D. L. Horton,et al.  An experimental analysis of associative factors in mediated generalization. , 1961 .

[16]  W. S. Verplanck,et al.  Unaware of where's awareness: some verbal operants--notates, monents, and notants. , 1962, Journal of personality.

[17]  C. Spielberger The role of awareness in verbal conditioning. , 1962, Journal of personality.

[18]  R. Gagne,et al.  A study of the effects of verbalization on problem solving. , 1962, Journal of experimental psychology.

[19]  D. Dulany,et al.  The place of hypotheses and intentions: an analysis of verbal control in verbal conditioning. , 1962, Journal of personality.

[20]  C. P. Duncan Effect of instructions and information on problem solving. , 1963 .

[21]  D. Dulany,et al.  Does partial reinforcement dissociate verbal rules and the behavior they might be presumed to control , 1963 .

[22]  Edward S. Johnson An information-processing model of one kind of problem solving. , 1964 .

[23]  L. DeNike,et al.  THE TEMPORAL RELATIONSHIP BETWEEN AWARENESS AND PERFORMANCE IN VERBAL CONDITIONING. , 1964, Journal of experimental psychology.

[24]  H. A. Witkin,et al.  SOME PRELIMINARY OBSERVATIONS WITH AN EXPERIMENTAL PROCEDURE FOR THE STUDY OF HYPNAGOGIC AND RELATED PHENOMENA. , 1964, Archivio di psicologia, neurologia e psichiatria.

[25]  D. L. Horton The effects of meaningfulness, awareness, and type of design in verbal mediation , 1964 .

[26]  STAGES IN CONCEPT FORMATION.: IV. A temporal analysis of effects of an extra intention to verbalize , 1965 .

[27]  Stages in concept formation : iii. further inquiries into the effects of an extra intention to verbalize , 1965 .

[28]  STAGES IN CONCEPT FORMATION. II. EFFECTS OF AN EXTRA INTENTION TO VERBALIZE THE CONCEPT AND OF STIMULUS PREDIFFERENTIATION. , 1965, Scandinavian journal of psychology.

[29]  CONCEPT LEARNING AND VERBAL CONTROL UNDER PARTIAL REINFORCEMENT AND SUBSEQUENT REVERSAL OR NONREVERSAL SHIFTS. , 1965, Journal of experimental psychology.

[30]  J. Phelan A REPLICATION OF A STUDY ON THE EFFECTS OF ATTEMPTS TO VERBALIZE ON THE PROCESS OF CONCEPT ATTAINMENT. , 1965, The Journal of psychology.

[31]  S. Schwartz Trial-by-trial analysis of processes in simple and disjunctive concept-attainment tasks. , 1966, Journal of experimental psychology.

[32]  John R. Schuck,et al.  A comparison of verbal and non-verbal reports of fragmenting visual Images , 1966 .

[33]  J. E. Carlson,et al.  Effects of reporting associative strategies on the retention of paired-associates , 1966 .

[34]  S. J. Dean,et al.  Reported mediation as a function of degree of learning , 1966 .

[35]  The effect of number of irrelevant stimulus dimensions, verbalization, and sex on learning biconditional classification rules , 1967 .

[36]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[37]  J. Smedslund MENTAL PROCESSES INVOLVED IN RAPID LOGICAL REASONING , 1968 .

[38]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[39]  D. B. Tarr,et al.  Verbalization, experimenter presence, and problem solving. , 1968, Journal of personality and social psychology.

[40]  J Smedslund Psychological diagnostics. , 1969, Psychological bulletin.

[41]  Donald F. Dansereau,et al.  An information processing model of mental multiplication. , 1969 .

[42]  L. R. Peterson,et al.  Concurrent verbal activity. , 1969 .

[43]  M. Levine,et al.  Human discrimination learning: A test of the blank-trials assumption , 1970 .

[44]  T. D. Kennedy Verbal Conditioning without Awareness: The Use of Programmed Reinforcement and Recurring Assessment of Awareness. , 1970 .

[45]  T. D. Kennedy Reinforcement frequency, task characteristics, and interval of awareness assessment as factors in verbal conditioning without awareness. , 1971 .

[46]  Blank-trial probes and introtacts in human discrimination learning , 1971 .

[47]  R. Dawes,et al.  Subjective value of the reinforcer (RSv) and performance: Crux of the S-R versus cognitive mediation controversy , 1971 .

[48]  John Benjafield,et al.  Evidence for a two-process theory of problem solving , 1971 .

[49]  Larry Wilder,et al.  Overt and covert verbalization in problem solving , 1971 .

[50]  E. Klinger Structure and functions of fantasy , 1971 .

[51]  G. Loftus Eye fixations and recognition memory for pictures , 1972 .

[52]  Cognitive process in auditory concept acquisition , 1972 .

[53]  W. Montague Elaborative Strategies in Verbal Learning and Memory , 1972 .

[54]  A. N. Sokolov,et al.  Inner Speech and Thought , 1972 .

[55]  S M Soliday,et al.  HAZARD PERCEPTION IN AUTOMOBILE DRIVERS: AGE DIFFERENCES , 1972 .

[56]  D. Wood,et al.  A Preliminary Study of Distinctive Features in Problem Solving , 1973 .

[57]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[58]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[59]  G. Bower,et al.  Human Associative Memory , 1973 .

[60]  Larry Sowder The Influence of Verbalization of Discovered Numerical- or Sorting-Task Generalizations on Short-Term Retention in Connection with the Hendrix Hypothesis. , 1974 .

[61]  E. Klinger,et al.  Utterances to evaluate steps and control attention distinguish operant from respondent thought while thinking out loud , 1974 .

[62]  D. Godden,et al.  An Investigation of the Relationships between Problem Solving Strategies, Representation and Memory , 1974 .

[63]  B. Brehmer Hypotheses about relations between scaled variables in the learning of probabilistic inference tasks , 1974 .

[64]  W. Brewer There is no convincing evidence for operant or classical conditioning in adult humans. , 1974 .

[65]  B. Fellows,et al.  'Perceptual' strategies in the solving of three term series problems. , 1975 .

[66]  J. Singer The inner world of daydreaming , 1975 .

[67]  G R Loftus,et al.  Two types of information in picture memory. , 1975, Journal of experimental psychology. Human learning and memory.

[68]  The Inference of Covert Hypotheses by Verbal Reports in Concept-Learning Research , 1975 .

[69]  A. Kazdin Assessment of imagery during covert modeling of assertive behavior , 1976 .

[70]  J. Payne,et al.  Judgments about Crime and the Criminal , 1977 .

[71]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[72]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[73]  Stephen K. Reed,et al.  Memory for Problem Solutions , 1977 .

[74]  D. Broadbent Levels, Hierarchies, and the Locus of Control* , 1977 .

[75]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[76]  G Rosolato,et al.  Symbol formation. , 1978, The International journal of psycho-analysis.

[77]  Eliot R. Smith,et al.  Limits on perception of cognitive processes: A reply to Nisbett and Wilson. , 1978 .

[78]  H. Simon,et al.  Models of Thought , 1979 .

[79]  STATE UNIVERSITY OF NEW YORK AT STONY BROOK , 2005 .