Studying the process of information selection in manuals: A review of four instruments

Research instruments for studying information selection in manuals should meet several goals at the same time. First of all, they should accurately register relevant data, such as selected information, selection moments, and reading times. They should also enable the reader to use both the manual and the accompanying product in a natural way. And finally, both data collection and analysis should not be too time-consuming or expensive. In this paper, four research instruments that may be used for studying selection processes in manuals are described and evaluated with respect to these three requirements. Three of them are well-known from other research domains: recording sentence reading times by button presses, methods of eye movement registration and the thinking aloud technique. The fourth, called the click & read method, was recently developed for studying the use of procedural and declarative information in software manuals. This instrument presents the manual in a paper layout on a large screen. The text is presented in blocks of comparable length and is initially blurred. Non-blurred headings above the blocks enable readers to select a block and sharpen it by clicking on it. These clicks are recorded and thus provide selection and time data.The evaluation shows that none of the four instruments is suitable for all research questions about manual use: instruments should be carefully chosen for specific research questions. Then again, all techniques have their advantages and disadvantages, some of which we are not yet certain about. Therefore, the paper concludes by arguing that a thorough analysis, comparison and further development of available and new research instruments is necessary to further facilitate and improve studies of information selection and processing