Finding canonical behaviors in user protocols

While the collection of behavioral protocols has been common practice in human-computer interaction research for many years, the analysis of large protocol data sets is often extremely tedious and time-consuming, and automated analysis methods have been slow to develop. This paper proposes an automated method of protocol analysis to find canonical behaviors --- a small subset of protocols that is most representative of the full data set, providing a reasonable "big picture" view of the data with as few protocols as possible. The automated method takes advantage of recent algorithmic developments in computational vision, modifying them to allow for distance measures between behavioral protocols. The paper includes an application of the method to web-browsing protocols, showing how the canonical behaviors found by the method match well to sets of behaviors identified by expert human coders.

[1]  Fabio Paternò,et al.  Intelligent analysis of user interactions with web applications , 2002, IUI '02.

[2]  Robert W. Reeder,et al.  Information scent as a driver of Web behavior graphs: results of a protocol analysis method for Web usability , 2001, CHI.

[3]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[4]  Julie Chen,et al.  The bloodhound project: automating discovery of web usability issues using the InfoScentπ simulator , 2003, CHI '03.

[5]  Anthony J. Hornof,et al.  Cognitive strategies and eye movements for searching hierarchical computer displays , 2003, CHI '03.

[6]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[7]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[8]  Dana K. Smith,et al.  Automated Protocol Analysis , 1993, Hum. Comput. Interact..

[9]  Wai-Tat Fu,et al.  A user-tracing architecture for modeling interaction with the world wide web , 2002, AVI '02.

[10]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[11]  Frank J. Lee,et al.  Production Compilation: A Simple Mechanism to Model Complex Skill Acquisition , 2003, Hum. Factors.

[12]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[13]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[14]  John R. Anderson,et al.  Automated Eye-Movement Protocol Analysis , 2001, Hum. Comput. Interact..

[15]  Ali Shokoufandeh,et al.  Canonical subsets of image features , 2008, Comput. Vis. Image Underst..

[16]  John R. Anderson,et al.  Eye tracking the visual search of click-down menus , 1999, CHI '99.

[17]  Frank E. Ritter,et al.  Developing Process Models as Summaries of HCI Action Sequences , 1994, Hum. Comput. Interact..

[18]  Tom Johnston,et al.  MacSHAPA and the enterprise of exploratory sequential data analysis (ESDA) , 1994, Int. J. Hum. Comput. Stud..