Human-centered multimedia: representations and challenges

Human has always been a part of the computational loop. So, what do we mean by human-centered computing (HCC)? aren't humans always the focus of computations some how? The goal of this paper is to help answer this question within the context of multimedia applilations. So, what do we mean by human-centered multimedia systems. We discuss some issues and challenges facing developing real humancentered multimedia applications.

[1]  Alex Pentland,et al.  Recognition of Space-Time Gestures using a Distributed Representation , 1993 .

[2]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[3]  Stephanie R. Taylor,et al.  Analysis and recognition of walking movements , 2002, Object recognition supported by user interaction for service robots.

[4]  I H Robertson,et al.  Visual attention: Controlling what we see and do , 1998, Current Biology.

[5]  Larry S. Davis,et al.  Robust periodic motion and motion symmetry detection , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Francis K. H. Quek.  Eyes in the interface , 1995, Image Vis. Comput..

[8]  Lucia Ballerini,et al.  Time-Varying Image Processing and Moving Object Recognition , 1997 .

[9]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Takeo Kanade,et al.  A statistical method for 3D object detection applied to faces and cars , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[11]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[13]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[14]  Theodoros Evgeniou,et al.  A TRAINABLE PEDESTRIAN DETECTION SYSTEM , 1998 .

[15]  D. Koller,et al.  Towards robust automatic traffic scene analysis in real-time , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[16]  Andrew Blake,et al.  Classification of human body motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[18]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[19]  Rama Chellappa,et al.  Discriminant analysis of principal components for face recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[20]  Aaron F. Bobick,et al.  Gait recognition from time-normalized joint-angle trajectories in the walking plane , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[22]  Peter G. B. Enser,et al.  Towards a Comprehensive Survey of the Semantic Gap in Visual Image Retrieval , 2003, CIVR.

[23]  Marco La Cascia,et al.  Fast, reliable head tracking under varying illumination , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[26]  Ambasamudram N. Rajagopalan,et al.  Human Identification Using Gait , 2002 .

[27]  Michael J. Black,et al.  Automatic Detection and Tracking of Human Motion with a View-Based Representation , 2002, ECCV.

[28]  Mark S. Nixon,et al.  Comparing Different Template Features for Recognizing People by their Gait , 1998, BMVC.

[29]  Dan Tsafrir,et al.  Human-Centered Scheduling of Interactive and Multimedia Applications on a Loaded Desktop , 2003 .

[30]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[31]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[33]  Andrew Blake,et al.  Probabilistic tracking in a metric space , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  K. P. Karmann,et al.  Moving object recognition using an adaptive background memory , 1990 .

[35]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[36]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[37]  James M. Rehg,et al.  A multiple hypothesis approach to figure tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[38]  R. Nelson,et al.  Low level recognition of human motion (or how to get your man without finding his body parts) , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[39]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[40]  Larry S. Davis,et al.  Real-time 3D Motion Capture , 1998 .

[41]  S. Sclaroff,et al.  Hand Pose Reconstruction Using Specialized Mappings , 2000 .

[42]  Mark S. Nixon,et al.  Extended Model-Based Automatic Gait Recognition of Walking and Running , 2001, AVBPA.

[43]  Nevenka Dimitrova.  Multimedia Content Analysis: The Next Wave , 2003, CIVR.

[44]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[45]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[46]  Aaron F. Bobick,et al.  Recognition of multi-agent interaction in video surveillance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[47]  Aaron F. Bobick,et al.  Recognition and interpretation of parametric gesture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[48]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[49]  Yee-Hong Yang,et al.  The background primal sketch: An approach for tracking moving objects , 1992, Machine Vision and Applications.

[50]  Stefano Soatto,et al.  Recognition of human gaits , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[51]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[52]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[53]  Alejandro Jaimes,et al.  Human factors in automatic image retrieval system design and evaluation , 2006, Electronic Imaging.

[54]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[55]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[56]  P. Saisan.  Image-based modeling of human gaits with higher-order statistics , 2002 .

[57]  Larry S. Davis,et al.  Ghost: a human body part labeling system using silhouettes , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[58]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[59]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[60]  Randal C. Nelson,et al.  Qualitative recognition of motion using temporal texture , 1992, CVGIP Image Underst..

[61]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[62]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[63]  Hsi-Jian Lee,et al.  Knowledge-guided visual perception of 3-D human gait from a single image sequence , 1992, IEEE Trans. Syst. Man Cybern..

[64]  Tomaso A. Poggio,et al.  Pedestrian detection using wavelet templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[65]  David J. Fleet,et al.  A Layered Motion Representation with Occlusion and Compact Spatial Support , 2002, ECCV.

[66]  David J. Fleet,et al.  Stochastic Tracking of 3D Human Figures Using 2D Image Motion , 2000, ECCV.

[67]  Xiang Gao,et al.  Error analysis of background adaption , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[68]  Roman Goldenberg,et al.  'Dynamism of a Dog on a Leash' or Behavior Classification by Eigen-Decomposition of Periodic Motions , 2002, ECCV.

[69]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[70]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[71]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[72]  Jake K. Aggarwal,et al.  Articulated and elastic non-rigid motion: a review , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[73]  Randal C. Nelson.  Qualitative detection of motion by a moving observer , 2004, International Journal of Computer Vision.

[74]  M. Rosenblum,et al.  Human emotion recognition from motion using a radial basis function network architecture , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[75]  Stan Sclaroff,et al.  3D hand pose reconstruction using specialized mappings , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[76]  Jian-Gang Wang,et al.  Eye gaze estimation from a single image of one eye , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[77]  Ioannis A. Kakadiaris,et al.  Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[78]  Larry S. Davis,et al.  EigenGait: Motion-Based Recognition of People Using Image Self-Similarity , 2001, AVBPA.

[79]  Michael J. Black,et al.  Implicit Probabilistic Models of Human Motion for Synthesis and Tracking , 2002, ECCV.

[80]  Mubarak Shah,et al.  Toward 3-D Gesture Recognition , 1999, Int. J. Pattern Recognit. Artif. Intell..

[81]  J. O'Rourke,et al.  Model-based image analysis of human motion using constraint propagation , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[82]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[83]  A. Yuille.  Deformable Templates for Face Recognition , 1991, Journal of Cognitive Neuroscience.

[84]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[85]  Thomas S. Huang,et al.  Human face detection in a complex background , 1994, Pattern Recognit..

[86]  Stuart J. Russell,et al.  Image Segmentation in Video Sequences: A Probabilistic Approach , 1997, UAI.

[87]  S. Yantis,et al.  Visual attention: control, representation, and time course. , 1997, Annual review of psychology.

[88]  Dariu Gavrila,et al.  Pedestrian Detection from a Moving Vehicle , 2000, ECCV.

[89]  Randal C. Nelson,et al.  Detecting activities , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[90]  Larry S. Davis,et al.  W/sup 4/: Who? When? Where? What? A real time system for detecting and tracking people , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[91]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[92]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[93]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[94]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[95]  Yang Song,et al.  Towards detection of human motion , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[96]  Amnon Shashua,et al.  Principal Component Analysis over Continuous Subspaces and Intersection of Half-Spaces , 2002, ECCV.

[97]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[98]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[99]  Jeff B. Pelz.  Portable eyetracking in natural behavior , 2004 .

[100]  Rómer Rosales,et al.  Inferring body pose without tracking body parts , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[101]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[102]  Aaron F. Bobick,et al.  Gait recognition using static, activity-specific parameters , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[103]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[104]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[105]  William T. Freeman,et al.  Bayesian Reconstruction of 3D Human Motion from Single-Camera Video , 1999, NIPS.

[106]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[107]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..