Threshold-based outer lip segmentation using support vector regression

Automated lip reading from videos requires lip segmentation. Threshold-based segmentation is straightforward, but it is rarely used. This study proposes a histogram threshold based on the feedback of shape information. Both good and bad lip segmentation examples were used to train an $$\epsilon $$ -support vector regression model to infer the segmentation accuracy from the region shape. The histogram threshold was optimised to minimise the segmentation error. The proposed method was tested on 895 images from 112 subjects using the AR Face Database. The proposed method, implemented in simple segmentation algorithms, reduced segmentation errors by 23.1%.

[1]  Jie Yang,et al.  Facial feature localization based on an improved active shape model , 2008, Inf. Sci..

[2]  V. Torczon,et al.  Direct search methods: then and now , 2000 .

[3]  Alan Wee-Chung Liew,et al.  An Automatic Lipreading System for Spoken Digits With Limited Training Data , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  A. Martínez,et al.  The AR face databasae , 1998 .

[5]  Jean-Luc Dugelay,et al.  Combining Edge Detection and Region Segmentation for Lip Contour Extraction , 2010, AMDO.

[6]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[7]  C. L. Philip Chen,et al.  A Cooperative Learning-Based Clustering Approach to Lip Segmentation Without Knowing Segment Number , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[9]  Gustavo Carneiro,et al.  One Shot Segmentation: Unifying Rigid Detection and Non-Rigid Segmentation Using Elastic Regularization , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Shilin Wang,et al.  Spatio-Temporal Fusion Based Convolutional Sequence Learning for Lip Reading , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Liya Ding,et al.  Features versus Context: An Approach for Precise and Detailed Detection and Delineation of Faces and Facial Features , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Vered Aharonson,et al.  Automatic computation of histogram threshold for lip segmentation using feedback of shape information , 2016, Signal Image Video Process..

[13]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[14]  Ashley Daniel Gritzman.  Adaptive threshold optimisation for colour-based lip segmentation in automatic lip-reading systems , 2016 .

[15]  Alan Wee-Chung Liew,et al.  Lip Image Segmentation Based on a Fuzzy Convolutional Neural Network , 2020, IEEE Transactions on Fuzzy Systems.

[16]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[17]  Alan Wee-Chung Liew,et al.  Lip Image Segmentation in Mobile Devices Based on Alternative Knowledge Distillation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[18]  Panagiota Spyridonos,et al.  Multi-Threshold LIP Contour Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).