Image and Video Processing Tools for HCI

Publisher Summary Image and video processing tools for human–computer interaction (HCI) are reviewed in this chapter. Different tools are used in close view applications, such as desktop computer applications or mobile telephone interfaces, and in distant view setups, such as smart-rooms scenarios or augmented-reality games. In the first case, the user can be captured in a close view and some assumptions can be made regarding the location and pose of the user. For instance, in face-oriented applications, a frontal view of the face is generally assumed, whereas for gestural interfaces, the hand is supposed to perform a gesture from a specific dictionary directly in front of the camera. This chapter describes face and hand analysis techniques that can be used in close view interfaces, such as desktop computer applications. Face analysis is used in HCI for recognition of the person and for more advanced interfaces that take into account the user state, analyzing for instance its facial expressions.

[1]  Larry S. Davis,et al.  A probabilistic framework for rigid and non-rigid appearance based tracking and recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[2]  Montse Pardàs,et al.  Motion estimation based tracking of active contours , 2001, Pattern Recognit. Lett..

[3]  Roberto Cipolla,et al.  Feature-based human face detection , 1997, Image Vis. Comput..

[4]  Michael J. Black,et al.  The Digital Office: Overview , 1998 .

[5]  Gwen Littlewort,et al.  Fully Automatic Facial Action Recognition in Spontaneous Behavior , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[6]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[8]  Montse Pardàs,et al.  Human model and motion based 3D action recognition in multiple view scenarios , 2006, 2006 14th European Signal Processing Conference.

[9]  Rainer Stiefelhagen,et al.  Head Pose Estimation in Single- and Multi-view Environments - Results on the CLEAR'07 Benchmarks , 2007, CLEAR.

[10]  Alberto Del Bimbo,et al.  Improving evidential quality of surveillance imagery through active face tracking , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Montse Pardàs,et al.  Edge projections for eye localization , 2008 .

[12]  Jiri Matas,et al.  Feature-based affine-invariant localization of faces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Alexander H. Waibel.  CHIL - Computers in the Human Interaction Loop , 2005, MVA.

[14]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[16]  Michael Isard,et al.  Partitioned Sampling, Articulated Objects, and Interface-Quality Hand Tracking , 2000, ECCV.

[17]  Verónica Vilaplana,et al.  Region-based mean shift tracking: Application to face tracking , 2008, 2008 15th IEEE International Conference on Image Processing.

[18]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[19]  Timothy F. Cootes,et al.  A Multi-Stage Approach to Facial Feature Detection , 2004, BMVC.

[20]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Donald Geman,et al.  Coarse-to-Fine Face Detection , 2004, International Journal of Computer Vision.

[22]  Adrian Hilton,et al.  Simultaneous Pose Estimation of Multiple People using Multiple-View Cues with Hierarchical Sampling , 2003, BMVC.

[23]  Takeo Kanade,et al.  Detection, tracking, and classification of action units in facial expression , 2000, Robotics Auton. Syst..

[24]  Shao Jinyou,et al.  モアレ干渉縞パターンを用いたインプリントリソグラフィのアライメントの測定法 | 文献情報 | J-GLOBAL 科学技術総合リンクセンター , 2008 .

[25]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Bjarne K. Ersbøll,et al.  FAME-a flexible appearance modeling environment , 2003, IEEE Transactions on Medical Imaging.

[29]  Ian R. Fasel,et al.  A generative framework for real time object detection and classification , 2005, Comput. Vis. Image Underst..

[30]  Cordelia Schmid,et al.  Face Detection and Tracking in a Video by Propagating Detection Probabilities , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Maja Pantic,et al.  Fully Automatic Facial Action Unit Detection and Temporal Analysis , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[32]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Yuxiao Hu,et al.  Learning a Person-Independent Representation for Precise 3D Pose Estimation , 2007, CLEAR.

[35]  Carlos Hitoshi Morimoto,et al.  Eye gaze tracking techniques for interactive applications , 2005, Comput. Vis. Image Underst..

[36]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[37]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[38]  Montse Pardàs,et al.  Multimodal real-time focus of attention estimation in SmartRooms , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[39]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Yoshinobu Ebisawa,et al.  Proposal of a zoom and focus control method using an ultrasonic distance-meter for video-based eye-gaze detection under free-head conditions , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[42]  Alexander G. Hauptmann,et al.  Towards robust face recognition from multiple views , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[43]  Rasmus Larsen,et al.  TOF imaging in Smart room environments towards improved people tracking , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[44]  Joachim Hornegger,et al.  3-D gesture-based scene navigation in medical imaging applications using Time-of-Flight cameras , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[45]  Alexander H. Waibel,et al.  Segmenting hands of arbitrary color , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[46]  Montse Pardàs,et al.  Facial animation parameters extraction and expression recognition using Hidden Markov Models , 2002, Signal Process. Image Commun..

[47]  Dario Maio,et al.  Real-time face location on gray-scale static images , 2000, Pattern Recognition.

[48]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[49]  Jake K. Aggarwal,et al.  Human motion analysis: a review , 1997, Proceedings IEEE Nonrigid and Articulated Motion Workshop.

[50]  Montse Pardàs,et al.  Exploiting Structural Hierarchy in Articulated Objects Towards Robust Motion Capture , 2008, AMDO.

[51]  Verónica Vilaplana,et al.  Binary Partition Trees for Object Detection , 2008, IEEE Transactions on Image Processing.

[52]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[53]  Julien Letessier,et al.  Visual tracking of bare fingers for interactive surfaces , 2004, UIST '04.

[54]  Narendra Ahuja,et al.  A SNoW-Based Face Detector , 1999, NIPS.

[55]  Theo Gevers,et al.  Accurate eye center location and tracking using isophote curvature , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Thomas S. Huang,et al.  Joint face and head tracking inside multi-camera smart rooms , 2007, Signal Image Video Process..

[57]  Qiang Ji,et al.  Special issue: eye detection and tracking , 2005, Comput. Vis. Image Underst..

[58]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[60]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Azriel Rosenfeld,et al.  Eye detection and tracking , 1999 .

[62]  Alexander H. Waibel,et al.  Modeling focus of attention for meeting indexing based on multiple cues , 2002, IEEE Trans. Neural Networks.

[63]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[64]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[65]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[66]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[67]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[68]  Henry Schneiderman,et al.  Learning Statistical Structure for Object Detection , 2003, CAIP.

[69]  Georgios Tziritas,et al.  Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis , 1999, IEEE Trans. Multim..

[70]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[71]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[72]  Ian D. Reid,et al.  Articulated Body Motion Capture by Stochastic Search , 2005, International Journal of Computer Vision.

[73]  King Ngi Ngan,et al.  Face segmentation using skin-color map in videophone applications , 1999, IEEE Trans. Circuits Syst. Video Technol..

[74]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[75]  Ferran Marqués,et al.  Bayesian Approach for Morphology-Based 2-D Human Motion Capture , 2007, IEEE Transactions on Multimedia.

[76]  Derek R. Magee,et al.  On-line Face Tracking Using a Feature Driven Level Set , 2003, BMVC.

[77]  Beat Fasel,et al.  Automatic facial expression analysis: a survey , 2003, Pattern Recognit..

[78]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[79]  Baback Moghaddam,et al.  Principal Manifolds and Probabilistic Subspaces for Visual Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  Nicu Sebe,et al.  Authentic facial expression analysis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..