IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use

This article describes recent research in subjective usability measurement at IBM, focused on evaluating the psychometric properties of questionnaires designed for use in scenario‐based usability evaluation. The questionnaires address evaluation at both a global overall system level and at a more detailed scenario level. The primary goals of this article are to (a) discuss the psychometric characteristics of IBM questionnaires that measure user satisfaction with computer system usability, and (b) provide the questionnaires, with administration and scoring instructions. For scenario‐level measurement, the 3‐item After‐Scenario Questionnaire (ASQ) has excellent internal consistency, with coefficient alphas across a set of scenarios ranging from .90 to .96. For more global assessment, the Post‐Study System Usability Questionnaire (PSSUQ) also has excellent internal consistency, with an overall coefficient alpha of .97. Preliminary principal factor analysis of 48 PSSUQ questionnaires suggested the presence of...

[1]  METHODOLOGIES EMPLOYED IN THE PSYCHOLOGICAL EVALUATION OF H.C.I. , 1987 .

[2]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[3]  Thomas K. Landauer,et al.  Research Methods in Human-Computer Interaction , 1988 .

[4]  Gary B. Reid.  Current Status of the Development of the Subjective Workload Assessment Technique , 1985 .

[5]  Norman Cliff,et al.  Analyzing Multivariate Data , 1987 .

[6]  Joseph B. Sidowski,et al.  Measurements of computer satisfaction, literacy, and aptitudes: A review , 1990, Int. J. Hum. Comput. Interact..

[7]  James R. Lewis,et al.  Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ , 1991, SGCH.

[8]  Jurek Kirakowski,et al.  What is Usability , 1991 .

[9]  M. Aldenderfer,et al.  Cluster Analysis. Sage University Paper Series On Quantitative Applications in the Social Sciences 07-044 , 1984 .

[10]  D. Gopher,et al.  On the Psychophysics of Workload: Why Bother with Subjective Measures? , 1984 .

[11]  John L. Bennett,et al.  Usability Engineering: Our Experience and Evolution , 1988 .

[12]  Richard J. Harris.  A primer of multivariate statistics , 1975 .

[13]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[14]  James R. Lewis.  Psychometric Evaluation of the Post-Study System Usability Questionnaire: The PSSUQ , 1992 .

[15]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[16]  Michael D. Coovert,et al.  Determining the Number of Common Factors in Factor Analysis: A Review and Program , 1988 .

[17]  George E. Cooper,et al.  The use of pilot rating in the evaluation of aircraft handling qualities , 1969 .

[18]  James R. Lewis,et al.  Integrated office software benchmarks: A case study , 1990, INTERACT.

[19]  James R. Lewis.  AN AFTER-SCENARIO QUESTIONNAIRE FOR USABILITY STUDIES: PSYCHOMETRIC EVALUATION OVER THREE TRIALS , 1991, SGCH.