Video browsing by direct manipulation

We present a method for browsing videos by directly dragging their content. This method brings the benefits of direct manipulation to an activity typically mediated by widgets. We support this new type of interactivity by: 1) automatically extracting motion data from videos; and 2) a new technique called relative flow dragging that lets users control video playback by moving objects of interest along their visual trajectory. We show that this method can outperform the traditional seeker bar in video browsing tasks that focus on visual content rather than time.

[1]  Ennio Mingolla,et al.  Global induced motion and visual stability in an optic flow illusion , 1997, Vision Research.

[2]  Ravin Balakrishnan,et al.  Fluid interaction techniques for the control and annotation of digital video , 2003, UIST '03.

[3]  Shumin Zhai,et al.  Beyond Fitts' law: models for trajectory-based HCI tasks , 1997, CHI Extended Abstracts.

[4]  Frank M. Shipman,et al.  Trailblazing: Video Playback Control by Direct Object Manipulation , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[5]  William Buxton,et al.  There's more to interaction than meets the eye: some issues in manual input , 1987 .

[6]  WeiJian Yang.  An efficient motion estimation method for MPEG-4 video encoder , 2003, IEEE Trans. Consumer Electron..

[7]  Reinhard Hölzl.  How does ‘dragging’ affect the learning of geometry , 1996, Int. J. Comput. Math. Learn..

[8]  Yakup Genc,et al.  GPU-based Video Feature Tracking And Matching , 2006 .

[9]  Michiel van de Panne,et al.  Motion doodles: an interface for sketching character motion , 2004, SIGGRAPH Courses.

[10]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Jae-Woo Chang,et al.  Trajectory-Based Video Retrieval for Multimedia Information Systems , 2004, ADVIS.

[12]  B. Schneirdeman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[13]  Donald A. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[14]  Pierre Dragicevic,et al.  Mnemonic rendering: an image-based approach for exposing hidden changes in dynamic displays , 2006, UIST.

[15]  Ajay Divakaran,et al.  Constant pace skimming and temporal sub-sampling of video using motion activity , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[16]  Kuo-Chin Fan,et al.  A motion-flow-based fast video retrieval system , 2005, MIR '05.

[17]  Reinhard Männer,et al.  Fast and Accurate Closest Point Search on Triangulated Surfaces and its Application to Head Motion Estimation , 2003 .

[18]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[19]  James D. Hollan,et al.  Direct manipulation interfaces (excerpt) , 1987 .

[20]  Michel Beaudouin-Lafon,et al.  Novel interaction techniques for overlapping windows , 2001, UIST '01.

[21]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[22]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[23]  Ari Z Zivotofsky.  The Duncker illusion: intersubject variability, brief exposure, and the role of eye movements in its generation. , 2004, Investigative ophthalmology & visual science.

[24]  J. S. Briggs,et al.  Designing the user interface: strategies for effective human-computer interaction: by Ben Schneiderman. Published by Addison-Wesley. 463pp. £17.95 , 1987 .

[25]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[26]  Bruce Randall Donald,et al.  Accessible animation and customizable graphics via simplicial configuration modeling , 2000, SIGGRAPH.

[27]  Gabriel Taubin,et al.  Curve and surface smoothing without shrinkage , 1995, Proceedings of IEEE International Conference on Computer Vision.

[28]  David Salesin,et al.  Schematic storyboarding for video visualization and editing , 2006, SIGGRAPH 2006.

[29]  Jean-Daniel Fekete,et al.  OrthoZoom scroller: 1D multi-scale navigation , 2006, CHI.

[30]  Jenq-Neng Hwang,et al.  Fast and automatic video object segmentation and tracking for content-based applications , 2002, IEEE Trans. Circuits Syst. Video Technol..

[31]  L Proteau,et al.  Visual Perception Modifies Goal-directed Movement Control: Supporting Evidence from a Visual Perturbation Paradigm , 1997, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[32]  Ben Shneiderman,et al.  Designing the user interface (2nd ed.): strategies for effective human-computer interaction , 1992 .

[33]  Ken Shoemake,et al.  ARCBALL: a user interface for specifying three-dimensional orientation using a mouse , 1992 .

[34]  Terry Winograd,et al.  FlowMenu: combining command, text, and data entry , 2000, UIST '00.

[35]  Anoop Gupta,et al.  Browsing digital video , 2000, CHI.

[36]  Pierre Dragicevic,et al.  SpiraClock: a continuous and non-intrusive display for upcoming events , 2002, CHI Extended Abstracts.

[37]  John F. Hughes,et al.  Navigating documents with the virtual scroll ring , 2004, UIST '04.

[38]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[40]  Michel Beaudouin-Lafon,et al.  Instrumental interaction: an interaction model for designing post-WIMP user interfaces , 2000, CHI.

[41]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..