User interface design for keyframe-based content browsing of digital video

With the technological advancements in computing and networking today, many interesting and potentially useful applications are being developed and turned into products. Digital video is one of those application areas that is getting more and more attention as the consumer market is likely to be huge, including professional video editing and home entertainment. However, most digital video systems so far have been developed based on technologically based ideas and possibilities, thus providing very little usability and poorly designed user interfaces. Having no long-term user base or usage context make it difficult to develop a new system using a user-oriented methodology where a target usage specification is made and a system developed accordingly. In this research, the design of user interface to a digital video system is studied in depth, particularly focusing on keyframe-based video content browsing interfaces. By analysing and identifying important elements in designing keyframe-based browsing interfaces, we construct a welldefined "design space" where a specific user interface can be designed by simply selecting a set of available options. This analytic approach makes the user interface design, which can be difficult to clarify due to its fuzzy nature, much more systematic and manageable. The usefulness of the constructed design space is demonstrated in the context of diverse devices such as desktop PCs and PDAs (Personal Digital Assistants) sharing the same data, and also diverse characteristics of the users. Specifically, video browsers suitable for a PDA are designed, and also various video browsers supporting different user preferences are designed and implemented using the design space. A qualitative evaluation is conducted on the designed browsing interfaces, to improve their usability and to gain further insights into the design of such browsers and refine the underlying design space.

[1]  Nicholas J. Belkin,et al.  A case for interaction: a study of interactive information retrieval behavior and effectiveness , 1996, CHI.

[2]  P. J. Hayes,et al.  Classifying users: a hard look at some controversial issues , 1986, CHI '86.

[3]  Alan F. Smeaton,et al.  The Fischlar Digital Video Recording, Analysis and Browsing System , 2000, RIAO.

[4]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[5]  Rune Hjelsvold,et al.  Integrated video archive tools , 1995, MULTIMEDIA '95.

[6]  Yoshinobu Tonomura,et al.  Multimedia Interfaces - Multimedia Content Indication , 1997, Handbook of Multimedia Information Management.

[7]  Tom Rodden,et al.  At home with the technology: an ethnographic study of a set-top-box trial , 1999, TCHI.

[8]  Ben Shneiderman,et al.  Visual Information Seeking: Tight Coupling of Dynamic Query Filters with Starfield Displays , 1994 .

[9]  Steinar Kristoffersen,et al.  “Making place” to make IT work: empirical explorations of HCI for mobile CSCW , 1999, GROUP.

[10]  Glorianna Davenport,et al.  Video streamer , 1994, CHI Conference Companion.

[11]  Sara Shatford Layne Some issues in the indexing of images , 1994 .

[12]  Mark D. Dunlop,et al.  Supporting different search strategies in video query interface , 2000, RIAO.

[13]  Peter G. B. Enser Pictorial information retrieval , 1995 .

[14]  Alan F. Smeaton,et al.  User Interface Issues for Browsing Digital Video , 1999, BCS-IRSG Annual Colloquium on IR Research.

[15]  Joëlle Coutaz,et al.  Embedding Plasticity in the Development Process of Interactive Systems , 2000 .

[16]  Chris Stary,et al.  A Structured Contextual Approach to Design for All , 2000 .

[17]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[18]  Aaron Marcus,et al.  Baby faces: user-interface design for small displays , 1998, CHI Conference Summary.

[19]  Charles L. Compton,et al.  Internet CNN NEWSROOM: a digital video news magazine and library , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.

[20]  Shingo Uchihashi,et al.  An interactive comic book presentation for exploring video , 2000, CHI.

[21]  B. S. Manjunath,et al.  NeTra-V: toward an object-based video representation , 1997, Electronic Imaging.

[22]  Wolfgang Effelsberg,et al.  Video abstracting , 1997, CACM.

[23]  Anoop Gupta,et al.  Browsing digital video , 2000, CHI.

[24]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[25]  Yukinobu Taniguchi,et al.  An intuitive and efficient access interface to real-time incoming video based on automatic indexing , 1995, MULTIMEDIA '95.

[26]  Ramana Rao,et al.  Rich interaction in the digital library , 1995, CACM.

[27]  Stefano Mizzaro,et al.  Evaluating user interfaces to information retrieval systems: a case study on user support , 1996, SIGIR '96.

[28]  Brenda Laurel,et al.  Computers as theatre , 1991 .

[29]  Walter Bender,et al.  Salient video stills: content and context preserved , 1993, MULTIMEDIA '93.

[30]  Ben Shneiderman,et al.  An Exploratory Study of Video Browsing User Interface Designs and Research Methodologies: Effectiveness in Information Seeking Tasks. , 1999 .

[31]  Michael G. Christel,et al.  Multimedia abstractions for a digital video library , 1997, DL '97.

[32]  M. Krause Intellectual problems of indexing picture collections , 1988 .

[33]  Alexander G. Hauptmann,et al.  Adjustable filmstrips and skims as abstractions for a digital video library , 1999, Proceedings IEEE Forum on Research and Technology Advances in Digital Libraries.

[34]  Howard Besser,et al.  Visual Access to Visual Images: The UC Berkeley Image Database Project , 1990 .

[35]  Joemon M. Jose,et al.  Spatial querying for image retrieval: a user-oriented evaluation , 1998, SIGIR '98.

[36]  Robin Jeffries,et al.  Applying cognitive walkthroughs to more complex user interfaces: experiences, issues, and recommendations , 1992, CHI.

[37]  Nevenka Dimitrova,et al.  The myth of semantic video retrieval , 1995, CSUR.

[38]  Gary Marchionini,et al.  Interfaces and Tools for the Library of Congress National Digital Library Program , 1998, Inf. Process. Manag..

[39]  Marti A. Hearst TileBars: visualization of term distribution information in full text information access , 1995, CHI '95.

[40]  Eric Bergman,et al.  Information appliances and beyond : interaction design for consumer products , 2000 .

[41]  Nicholas J. Belkin,et al.  Evaluation of a tool for visualization of information retrieval results , 1996, SIGIR '96.

[42]  Edward R. Tufte,et al.  The Visual Display of Quantitative Information , 1986 .

[43]  Andy Smith,et al.  Human-Computer Factors: A Study of Users and Information Systems , 1997 .

[44]  Constantine Stephanidis,et al.  Encapsulating intelligent interactive behaviour in unified user interface artefacts , 2000, Interact. Comput..

[45]  David B. Paradice,et al.  Prepare your mind for creativity , 2000, CACM.

[46]  Agnes Kukulska-Hulme,et al.  The informedia digital video library system at the open university , 1999 .

[47]  Mark T. Keane,et al.  Cognitive Psychology: A Student's Handbook , 1990 .

[48]  Alan F. Smeaton,et al.  Online television library: organization and content browsing for general users , 2001, IS&T/SPIE Electronic Imaging.

[49]  Donis A. Dondis A primer of visual literacy , 1973 .

[50]  Sara Shatford,et al.  Analyzing the Subject of a Picture: A Theoretical Approach , 1986 .

[51]  Michael Mills,et al.  A magnifier tool for video data , 1992, CHI.

[52]  Shih-Fu Chang,et al.  VideoQ: an automated content based video search system using visual cues , 1997, MULTIMEDIA '97.

[53]  Thomas P. Moran,et al.  Design rationale: the argument behind the artifact , 1989, CHI '89.

[54]  Nevenka Dimitrova,et al.  Video keyframe extraction and filtering: a keyframe is not a keyframe to everyone , 1997, CIKM '97.

[55]  Wayne H. Wolf,et al.  Key frame selection by motion analysis , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[56]  Alan F. Smeaton,et al.  Evaluation of automatic shot boundary detection on a large video test suite , 1999 .

[57]  Philippe Aigrain,et al.  Representation-based user interfaces for the audiovisual library of the year 2000 , 1995, Electronic Imaging.

[58]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[59]  Gary Marchionini,et al.  Key frame preview techniques for video browsing , 1998, DL '98.

[60]  Remi Depommier,et al.  Content-based browsing of video sequences , 1994, MULTIMEDIA '94.

[61]  David Bawden User-Oriented Evaluation of Information Systems and Services , 1990 .

[62]  Duncan Sanderson,et al.  Technology design and mimicry , 1997, DIS '97.

[63]  Otthein Herzog,et al.  Video retrieval by still-image analysis with ImageMiner , 1997, Electronic Imaging.

[64]  A. Grund ICONCLASS. On subject analysis of iconographic representations of works of art , 1993 .

[65]  Alan F. Smeaton,et al.  Fischlar: an on-line system for indexing and browsing broadcast television content , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[66]  Thomas D. C. Little,et al.  Video query formulation , 1995, Electronic Imaging.

[67]  David Pye,et al.  AT_TV: Broadcast Television and Radio Retrieval , 2000, RIAO.

[68]  Alan F. Smeaton,et al.  Fischlar on a PDA: handheld user interface design to a video indexing, browsing and playback system , 2001, HCI.

[69]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[70]  Barry Arons,et al.  SpeechSkimmer: a system for interactively skimming recorded speech , 1997, TCHI.

[71]  Elizabeth D. Mynatt,et al.  Designing auditory interactions for PDAs , 1995, UIST '95.

[72]  Giridharan Iyengar,et al.  Semantically controlled content-based retrieval of video sequences , 1998, Other Conferences.

[73]  Osamu Hori,et al.  A shot classification method of selecting effective key-frames for video browsing , 1997, MULTIMEDIA '96.

[74]  Stephen W. Smoliar,et al.  Video parsing, retrieval and browsing: an integrated and content-based solution , 1997, MULTIMEDIA '95.

[75]  William W. Gaver,et al.  Alternatives: exploring information appliances through conceptual design proposals , 2000, CHI.

[76]  William M. Newman,et al.  Better or just different? On the benefits of designing interactive systems in terms of critical parameters , 1997, DIS '97.

[77]  Dragutin Petkovic,et al.  Automatic and semiautomatic methods for image annotation and retrieval in query by image content (QBIC) , 1995, Electronic Imaging.

[78]  Boon-Lock Yeo,et al.  Retrieving and visualizing video , 1997, CACM.

[79]  Noi Sukaviriya,et al.  Using small screen space more efficiently , 1996, CHI.

[80]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[81]  Pat Molholt,et al.  The role of the ‘Art and Architecture Thesaurus’ in communicating about visual art , 1993 .

[82]  Edward R. Tufte,et al.  Envisioning Information , 1990 .

[83]  Kim J. Vicente,et al.  An experimental evaluation of transparent menu usage , 1996, CHI.

[84]  Pamela Savage,et al.  User interface evaluation in an iterative design process: a comparison of three techniques , 1996, CHI Conference Companion.

[85]  Barry Smyth,et al.  A personalized television listings service , 2000, CACM.

[86]  Brad A. Myers,et al.  Two-handed input using a PDA and a mouse , 2000, CHI.

[87]  D. Norman The psychology of everyday things , 1990 .

[88]  Alan F. Smeaton,et al.  An evaluation of alternative techniques for automatic detection of shot boundaries in digital video , 1999 .

[89]  P. D Harper,et al.  Improving user satisfaction: the questionnaire for user interaction satisfaction version 5 , 1993 .

[90]  Yiqing Liang,et al.  A digital video library on the World Wide Web , 1997, MULTIMEDIA '96.

[91]  Ken Yap,et al.  FRANK: trialing a system for remote navigation of film archives , 1997, Other Conferences.

[92]  Wai On Lee,et al.  Introducing Internet Terminals to the Home: Interaction Between Social, Physical, and Technological Spaces , 2000, BCS HCI.

[93]  Edward Tufte,et al.  Visual Explanations , 1997 .

[94]  Joëlle Coutaz,et al.  A dimension space for the design of interactive systems within their physical environments , 2000, DIS '00.

[95]  Noel E. O'Connor,et al.  Evaluating and combining digital video shot boundary detection algorithms , 2000 .

[96]  Jean Vanderdonckt,et al.  Visual techniques for traditional and multimedia layouts , 1994, AVI '94.

[97]  Tefko Saracevic,et al.  Evaluation of evaluation in information retrieval , 1995, SIGIR '95.

[98]  John M. Gauch,et al.  Vision: a digital video library , 1996, DL '96.

[99]  James M. Turner Indexing “Ordinary” Pictures for Storage and Retrieval , 1994 .

[100]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[101]  Sara Shatford Layne,et al.  Some Issues in the Indexing of Images , 1994, J. Am. Soc. Inf. Sci..

[102]  Jakob Nielsen,et al.  Enhancing the explanatory power of usability heuristics , 1994, CHI '94.

[103]  Takeo Kanade,et al.  Intelligent Access to Digital Video: Informedia Project , 1996, Computer.

[104]  William R. Hersh,et al.  A task-oriented approach to information retrieval evaluation , 1996 .