Finding the right shots: assessing usability and performance of a digital video library interface

The authors developed a system in which visually dense displays of thumbnail imagery in storyboard views are used for shot-based video retrieval. The views allow for effective retrieval, as evidenced by the success achieved by expert users with the system in interactive query for NIST TRECVID 2002 and 2003. This paper demonstrates that novice users also achieve comparatively high retrieval performance with these views using the TRECVID 2003 benchmarks. Through an analysis of the user interaction logs, heuristic evaluation, and think-aloud protocol, the usability of the video information retrieval system is appraised with respect to shot-based retrieval. Design implications are presented based on these TRECVID usability evaluations regarding efficient, effective information retrieval interfaces to locate visual information from video corpora.

[1]  Mark V. Springett,et al.  A comparison of usability techniques for evaluating design , 1997, DIS '97.

[2]  Michael G. Christel,et al.  Exploiting multiple modalities for interactive video retrieval , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[3]  Jean-Luc Gauvain,et al.  The LIMSI Broadcast News transcription system , 2002, Speech Commun..

[4]  Paul Over,et al.  The TREC VIdeo Retrieval Evaluation (TRECVID): A Case Study and Status Report , 2004, RIAO.

[5]  Gary Marchionini,et al.  Key frame preview techniques for video browsing , 1998, DL '98.

[6]  Jakob Nielsen,et al.  Evaluating the thinking-aloud technique for use by computer scientists , 1993 .

[7]  Gregory D. Abowd,et al.  Human-computer interaction (2nd ed.) , 1998 .

[8]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[9]  Alan F. Smeaton,et al.  Designing the User Interface for the Físchlár Digital Video Library , 2006, J. Digit. Inf..

[10]  Janni Nielsen,et al.  Getting access to what goes on in people's heads?: reflections on the think-aloud technique , 2002, NordiCHI '02.

[11]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[12]  Anoop Gupta,et al.  Browsing digital video , 2000, CHI.

[13]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[14]  Dragutin Petkovic,et al.  Key to effective video retrieval: effective cataloging and browsing , 1998, MULTIMEDIA '98.

[15]  Paul Over,et al.  The TREC-2002 Video Track Report , 2002, TREC.

[16]  Ramesh C. Jain,et al.  ACM SIGMM retreat report on future directions in multimedia research , 2005, TOMCCAP.