Computer vision aided lip movement correction to improve English pronunciation

Dr. Yingjie Chen is an assistant professor in the Department of Computer Graphics Technology of Purdue University. He received his Ph.D. degree in the areas of human-computer interaction, information visualization, and visual analytics from the School of Interaction Arts and Technology at Simon Fraser University (SFU) in Canada. He earned the Bachelor degree of Engineering from the Tsinghua University in China, and a Master of Science degree in Information Technology from SFU. His research covers interdisciplinary domains of information visualization, visual analytics, digital media, and human computer interaction. He seeks to design, model, and construct new forms of interaction in visualization and system design, by which the system can minimize its influence on design and analysis, and become a true free extension of human’s brain and hand.

[1]  Wen-Hsing Luo.  The Cambridge Guide to Teaching English to Speakers of Other Languages , 2003 .

[2]  Oda Mariko,et al.  Development of a Pronunciation Practice CAI System Based on Lip Reading Techniques for Deaf Children , 2007 .

[3]  Douglas W. Coleman,et al.  On Foot in SIM City: Using SIM Copter as the Basis for an ESL Writing Assignment , 2002 .

[4]  Georgy Gimel'farb,et al.  Lip Contour Extraction from Video Sequences under Natural Lighting Conditions , 2009 .

[5]  Masoud Hashemi,et al.  Computer Assisted Language Learning Freedom or Submission to Machines , 2011 .

[6]  Aurora Tatiana Dina,et al.  The Advantages and Disadvantages of Computer Assisted Language Learning and Teaching for Foreign Languages , 2013 .

[7]  Carol A. Chapelle,et al.  A meta-analysis of effectiveness studies on computer technology-supported language learning , 2013, ReCALL.

[8]  Donald M. Lance,et al.  Pronunciation , 1885, Science.

[9]  In-Seok Kim,et al.  Automatic Speech Recognition: Reliability and Pedagogical Implications for Teaching Pronunciation , 2006, J. Educ. Technol. Soc..

[10]  N. Léchopier.  " Experimental and quasi-experimental designs for research on teaching ", de Donald T. Campbell & Julian C. Stanley, (1963). , 2011 .

[11]  Alice Caplier,et al.  New color transformation for lips segmentation , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[12]  Ken Beatty,et al.  Teaching & Researching: Computer-Assisted Language Learning , 2013 .

[13]  Tracey M. Derwing,et al.  Teaching Native Speakers to Listen to Foreign-accented Speech , 2002 .

[14]  Joy Egbert,et al.  CALL environments : research, practice, and critical issues , 1999 .

[15]  Helen Meng,et al.  Enunciate: An internet-accessible computer-aided pronunciation training system and related user evaluations , 2011, 2011 International Conference on Speech Database and Assessments (Oriental COCOSDA).

[16]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Barney Dalton,et al.  Automatic Speechreading using dynamic contours , 1996 .

[18]  Mark Aronoff,et al.  Contemporary linguistics: An introduction , 1989 .

[19]  Michael Dobrovolsky,et al.  three PHONOLOGY : THE FUNCTION AND PATTERNING OF SOUNDS , 2001 .

[20]  Steven H Bayless,et al.  Trends in Computer Vision: An overview of vision-based data acquisition and processing technology and its potential for the transportation sector , 2011 .

[21]  Fatima Zaki Mohammad Al-Qudah,et al.  Improving English Pronunciation through Computer-Assisted Programs in Jordanian Universities. , 2012 .

[22]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[23]  Yong Zhao,et al.  Recent Developments in Technology and Language Learning: A Literature Review and Meta-analysis , 2013 .

[24]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[25]  Farima Talebi,et al.  The Effect of Computer- assisted Language Learning on Improving EFL Learners' Pronunciation Ability , 2013 .

[26]  Alan Wee-Chung Liew,et al.  Visual Speech Recognition: Lip Segmentation and Mapping , 2008 .

[27]  Of references. , 1966, JAMA.

[28]  Kevin Bird.  Within-subjects Designs , 2004 .

[29]  Thomas Hansen.  Computer-Assisted Pronunciation Training:: The Four K's of Freedback , 2006, MM 2006.

[30]  Alice Caplier.  Lip detection and tracking , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[31]  Andrea G. Osburne.  Pronunciation strategies of advanced ESOL learners , 2003 .

[32]  Min Liu,et al.  A Look at the Research on Computer-Based Technology Use in Second Language Learning , 2002 .

[33]  Jean-Luc Dugelay,et al.  Combining Edge Detection and Region Segmentation for Lip Contour Extraction , 2010, AMDO.

[34]  Sheng Yang,et al.  A fast mouth detection algorithm based on face organs , 2009, 2009 2nd International Conference on Power Electronics and Intelligent Transportation System (PEITS).

[35]  B. Planken,et al.  AGE AND ULTIMATE ATTAINMENT IN THE PRONUNCIATION OF A FOREIGN LANGUAGE , 1997, Studies in Second Language Acquisition.

[36]  Lisa Gjedde,et al.  Current developments in technology-assisted education , 2006 .

[37]  Sridha Sridharan,et al.  An approach to statistical lip modelling for speaker identification via chromatic feature extraction , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[38]  Peter L. Silsbee,et al.  Audiovisual Sensory Integration Using Hidden Markov Models , 1996 .

[39]  P. Zepeda,et al.  The teaching of pronunciation , 2010 .

[40]  Kohei Arai,et al.  Effects of Pronunciation Practice System Based on Personalized CG Animations of Mouth Movement Model , 2012 .

[41]  Abdurrahman Ghaleb Almekhlafi,et al.  Employing Reading and Writing Computer-Based Instruction in English as a Second Language in Elementary Schools , 2012 .

[42]  Ronald Carter,et al.  The Cambridge Guide to Teaching English to Speakers of Other Languages: List of abbreviations , 2001 .

[43]  Juergen Luettin,et al.  Active Shape Models for Visual Speech Feature Extraction , 1996 .

[44]  K. Lee,et al.  English teachers' barriers to the use of computer-assisted language learning , 2000 .

[45]  David G. Stork,et al.  Using deformable templates to infer visual speech dynamics , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[46]  Murat Hişmanoğlu,et al.  An Investigation of Pronunciation Learning Strategies of Advanced EFL Learners. , 2012 .

[47]  Gerald P. Delahunty,et al.  The English Language: From Sound to Sense , 2010 .