Face and eye rectification in video conference using affine transform

The lack of eye contact in video conference degrades the user's experience. This problem has been known and studied for many years. There are hardware-based solutions to the eye gazing problem. However, these specialized systems are not generally accessible. This paper suggests a software approach that rectifies the face and the eyes in video conference, only utilizing one camera. In the setup phase, the view point and the affine transform matrix are calculated, using five frames each from the front view and the above view, with the aids of face detection and eye detection algorithm. Once the setup is done, we rectify the face with affine transformation, and rectify the eyes using image warping, based on an eye model. This is done in near real time (18 frames per second, for the resolution of 320/spl times/240). The result is not genuine but is significantly better.

[1]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  R. Stokes.  Human Factors and Appearance Design Considerations of the Mod II PICTUREPHONE® Station Set , 1969 .

[3]  Milton Chen,et al.  Leveraging the asymmetric sensitivity of eye contact for videoconference , 2002, CHI.

[4]  Ben Yip.  Face and Eye Rectification in Video Conference Using Artificial Neural Network , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[5]  Jesse S. Jin,et al.  Pose determination and viewpoint determination of human head in video conferencing based on head movement , 2004, 10th International Multimedia Modelling Conference, 2004. Proceedings..

[6]  A. G. Ramakrishnan,et al.  Eye detection using color cues and projection functions , 2002, Proceedings. International Conference on Image Processing.

[7]  Paulo Menezes,et al.  Human-robot interaction based on Haar-like features and eigenfaces , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[8]  Yutaka Matsushita,et al.  Multiparty videoconferencing at virtual social distance: MAJIC design , 1994, CSCW '94.

[9]  Jin Liu,et al.  Eye and gaze tracking for visually controlled interactive stereoscopic displays , 1999, Signal Process. Image Commun..

[10]  Jin Liu,et al.  A COMPUTATIONAL APPROACH TO ESTABLISH EYE-CONTACT IN VIDEOCOMMUNICATION , 1995 .

[11]  Kentaro Toyama,et al.  Manipulation of Video Eye Gaze and Head Orientation for Video Teleconferencing , 1999 .

[12]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Kentaro Toyama,et al.  Gaze-awareness for Videoconferencing: A Software Approach , 2000 .

[14]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[15]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Ruigang Yang,et al.  Eye gaze correction with stereovision for video-teleconferencing , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Kentaro Toyama,et al.  Gaze Awareness for Video-Conferencing: A Software Approach , 2000, IEEE Multim..