Collegial verbalisation – a case study on a new method on information acquisition

The purpose of this paper is to propose and discuss a new method for information acquisition called collegial verbalisation. The method is explored in an empirical case study, and it consists of vehicle operators being videotaped while driving, followed by some of their colleagues making verbal reports while watching the video data. These colleagues have previously experienced exactly the same driving task as the operators, that is, they have travelled the same route, with the same craft, and used the same instrumentation. Thus, they are very familiar with the driving task and the driver environment. The empirical study is carried out on a high-speed ferry. The method is explored in relation to three rather open hypotheses: the amount of information provided in general; the reliability of the data; and how it contributes to the detection of ‘buggy mental models’ within the operators. The method is discussed in relation to the more traditional forms of verbal reports: concurrent and retrospective verbalisations, respectively. From the results of the empirical exploration, it is suggested that the method of collegial verbalisation may have combinatorial advantages that makes it more powerful as an analytical tool than the traditional forms of verbalisation, specifically if one wants to analyse work tasks that are dynamic and where the operators' behaviours are highly automated. However, more elaborate and systematic investigations must be conducted on the topic, and the paper therefore ends with a suggestion on an experimental design for this purpose.

[1]  Lisanne Bainbridge,et al.  Verbal reports as evidence of the process operator's knowledge , 1999, Int. J. Hum. Comput. Stud..

[2]  George Wright,et al.  Eliciting and modelling expert knowledge , 1987, Decision Support Systems.

[3]  Anders Jansson,et al.  Augmented reality as a navigation aid for the manoeuvring of High-Speed Crafts , 2004 .

[4]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[5]  Maarten van Someren,et al.  The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes , 1994 .

[6]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[7]  Austin Henderson,et al.  Interaction Analysis: Foundations and Practice , 1995 .

[8]  Anders Jansson,et al.  Safer Navigation at Sea using Augmented Reality , 2002 .

[9]  J. Shaoul Human Error , 1973, Nature.

[10]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[11]  Ralph H. Turner Psychology: The Science of Mental Life. , 1974 .

[12]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Anders Jansson,et al.  Work on the bridge – studies of officers on high-speed ferries , 2006, Behav. Inf. Technol..

[14]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[15]  Nadine B. Sarter,et al.  How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control , 1995, Hum. Factors.

[16]  Benjamin Kuipers,et al.  How to Discover a Knowledge Representation for Causal Reasoning by Studying an Expert Physician , 1983, IJCAI.

[17]  Jim McLennan,et al.  Effects of Concurrent Verbalization on a Time-Critical, Dynamic Decision-Making Task , 2000, The Journal of general psychology.

[18]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[19]  James T. Reason,et al.  A systems approach to organizational error , 1995 .

[20]  George A. Miller,et al.  Psychology : the science of mental life , 1962 .

[21]  Anders Jansson,et al.  Bridging the gap between analysis and design: improving existing driver interfaces with tools from the framework of cognitive work analysis , 2005, Cognition, Technology & Work.

[22]  Michael Holm Larsen,et al.  Distributed knowledge management based on product state models - the case of decision support in health care administration , 1987, Decis. Support Syst..

[23]  Janni Nielsen,et al.  Mindtape: A Tool for Reflection in Participatory Design , 2000 .