Quantitative Survey of the State of the Art in Sign Language Recognition

This work presents a meta study covering around 300 published sign language recognition papers with over 400 experimental results. It includes most papers between the start of the field in 1983 and 2020. Additionally, it covers a fine-grained analysis on over 25 studies that have compared their recognition approaches on RWTH-PHOENIX-Weather 2014, the standard benchmark task of the field. Research in the domain of sign language recognition has progressed significantly in the last decade, reaching a point where the task attracts much more attention than ever before. This study compiles the state of the art in a concise way to help advance the field and reveal open questions. Moreover, all of this meta study's source data is made public, easing future work with it and further expansion. The analyzed papers have been manually labeled with a set of categories. The data reveals many insights, such as, among others, shifts in the field from intrusive to non-intrusive capturing, from local to global features and the lack of non-manual parameters included in medium and larger vocabulary recognition systems. Surprisingly, RWTH-PHOENIX-Weather with a vocabulary of 1080 signs represents the only resource for large vocabulary continuous sign language recognition benchmarking world wide.

[1]  Houqiang Li,et al.  Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition , 2020, AAAI.

[2]  Hermann Ney,et al.  Pronunciation Clustering and Modeling of Variability for Appearance-Based Sign Language Recognition , 2005, Gesture Workshop.

[3]  Ruiduo Yang,et al.  Segmentation-robust representations, matching, and modeling for sign language , 2011, CVPR 2011 WORKSHOPS.

[4]  Alan W. C. Tan,et al.  A feature covariance matrix with serial particle filter for isolated sign language recognition , 2016, Expert Syst. Appl..

[5]  Ramamoorthy Srinath,et al.  Time Series Neural Networks for Real Time Sign Language Translation , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[6]  Seong-Whan Lee,et al.  Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine , 2013, Pattern Recognit. Lett..

[7]  Francesco Piazza,et al.  Signer independent isolated Italian sign recognition based on hidden Markov models , 2014, Pattern Analysis and Applications.

[8]  Wen Gao,et al.  Viewpoint invariant sign language recognition , 2007, IEEE International Conference on Image Processing 2005.

[9]  Mariusz Oszust,et al.  Determining Subunits for Sign Language Recognition by Evolutionary Cluster-Based Segmentation of Time Series , 2010, ICAISC.

[10]  Bin Xie,et al.  RGB-D static gesture recognition based on convolutional neural network , 2018 .

[11]  Timothy F. O'Connor,et al.  The Language of Glove: Wireless gesture decoder with low-power and stretchable hybrid electronics , 2017, PloS one.

[12]  Honggang Wang,et al.  American Sign Language Recognition Using Multi-dimensional Hidden Markov Models , 2006, J. Inf. Sci. Eng..

[13]  Boon Giin Lee,et al.  Smart Wearable Hand Device for Sign Language Interpretation System With Sensors Fusion , 2018, IEEE Sensors Journal.

[14]  Wen Gao,et al.  The Recognition of Finger-Spelling for Chinese Sign Language , 2001, Gesture Workshop.

[15]  Hermann Ney,et al.  Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs , 2018, International Journal of Computer Vision.

[16]  Frederico G. Guimarães,et al.  Feature extraction in Brazilian Sign Language Recognition based on phonological structure and using RGB-D sensors , 2014, Expert Syst. Appl..

[17]  Carol Neidle,et al.  A new web interface to facilitate access to corpora: development of the ASLLRP data access interface , 2012 .

[18]  Mieke Van Herreweghe,et al.  Sign classification in sign language Corpora with deep neural networks , 2016, LREC 2016.

[19]  Oscar Koller,et al.  Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Meng Wang,et al.  Connectionist Temporal Fusion for Sign Language Translation , 2018, ACM Multimedia.

[21]  Dimitris N. Metaxas,et al.  Parallel hidden Markov models for American sign language recognition , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Dimitris N. Metaxas,et al.  Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes , 1999, Gesture Workshop.

[23]  Shan Lu,et al.  Recognition of local features for camera-based sign language recognition system , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[24]  Nicolas Pugeault,et al.  Sign language recognition using sub-units , 2012, J. Mach. Learn. Res..

[25]  Houqiang Li,et al.  Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[26]  Lale Akarun,et al.  A multi-class classification strategy for Fisher scores: Application to signer independent sign language recognition , 2010, Pattern Recognit..

[27]  Peter Vamplew.  Recognition of sign language gestures using neural networks , 1996 .

[28]  Bülent Sankur,et al.  SignTutor: An Interactive System for Sign Language Tutoring , 2009, IEEE Multimedia.

[29]  Nicolas Pugeault,et al.  Sign Language Recognition using Sequential Pattern Trees , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Houqiang Li,et al.  Continuous Sign Language Recognition via Reinforcement Learning , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[31]  Helen Cooper,et al.  University of Surrey , 2019, The Grants Register 2022.

[32]  Richard Bowden,et al.  HMM-based Approaches to Model Multichannel Information in Sign Language Inspired from Articulatory Features-based Speech Processing , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[34]  Anara Sandygulova,et al.  Evaluation of Manual and Non-manual Components for Sign Language Recognition , 2020, LREC.

[35]  Paul Lukowicz,et al.  Using multiple sensors for mobile sign language recognition , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[36]  George Awad,et al.  Novel boosting framework for subunit-based sign language recognition , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[37]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Hermann Ney,et al.  Appearance-Based Recognition of Words in American Sign Language , 2005, IbPRIA.

[39]  Ming C. Leu,et al.  Linguistic properties based on American Sign Language isolated word recognition with artificial neural networks using a sensory glove and motion tracker , 2007, Neurocomputing.

[40]  Houqiang Li,et al.  A new system for Chinese sign language recognition , 2015, 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP).

[41]  Hermann Ney,et al.  An Integrated Tracking And Recognition Approach For Video , 2008 .

[42]  Hermann Ney,et al.  Enhanced continuous sign language recognition using PCA and neural network features , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[43]  H. S. Nagendraswamy,et al.  Symbolic Representation of Sign Language at Sentence Level , 2015 .

[44]  Morteza Zahedi,et al.  Robust appearance based sign language recognition , 2007 .

[45]  Pujianto Yugopuspito,et al.  Mobile Sign Language Recognition for Bahasa Indonesia using Convolutional Neural Network , 2018, MoMM.

[46]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[47]  P. V. V. Kishore,et al.  Three-Dimensional Sign Language Recognition With Angular Velocity Maps and Connived Feature ResNet , 2018, IEEE Signal Processing Letters.

[48]  Z. Zenn Bien,et al.  A dynamic gesture recognition system for the Korean sign language (KSL) , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[49]  Hermann Ney,et al.  Visual Modeling and Feature Adaptation in Sign Language Recognition , 2011 .

[50]  M. Deriche,et al.  A signer-independent Arabic Sign Language recognition system using face detection, geometric features, and a Hidden Markov Model , 2012, Comput. Electr. Eng..

[51]  Felipe Trujillo-Romero,et al.  Mexican sign language recognition using kinect and data time warping algorithm , 2017, 2017 International Conference on Electronics, Communications and Computers (CONIELECOMP).

[52]  Daniel Schneider,et al.  Rapid Signer Adaptation for Isolated Sign Language Recognition , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[53]  Vassilis Athitsos,et al.  A database-based framework for gesture recognition , 2010, Personal and Ubiquitous Computing.

[54]  Meng Wang,et al.  Hierarchical LSTM for Sign Language Translation , 2018, AAAI.

[55]  Ali Farhadi,et al.  Aligning ASL for Statistical Translation Using a Discriminative Word Model , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[56]  Robyn A. Owens,et al.  Visual Sign Language Recognition , 2000, Theoretical Foundations of Computer Vision.

[57]  Manoochehr Nahvi,et al.  An algorithm on sign words extraction and recognition of continuous Persian sign language based on motion and shape features of hands , 2018, Pattern Analysis and Applications.

[58]  Petros Daras,et al.  SIGN LANGUAGE RECOGNITION BASED ON HAND AND BODY SKELETAL DATA , 2018, 2018 - 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[59]  Houqiang Li,et al.  A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition , 2014, ICIMCS '14.

[60]  Bodo Rosenhahn,et al.  Real-Time Sign Language Recognition Using a Consumer Depth Camera , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[61]  Wen Gao,et al.  Adaptive Sign Language Recognition With Exemplar Extraction and MAP/IVFS , 2010, IEEE Signal Processing Letters.

[62]  Petros Maragos,et al.  Dynamic-static unsupervised sequentiality, statistical subunits and lexicon for sign language recognition , 2014, Image Vis. Comput..

[63]  Ming C. Leu,et al.  Linguistic Properties Based on American Sign Language Recognition with Artificial Neural Networks Using a Sensory Glove and Motion Tracker , 2005, IWANN.

[64]  Mohamed A. Deriche,et al.  Arabic Sign Language Recognition an Image-Based Approach , 2007, 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07).

[65]  Wen Gao,et al.  An approach based on phonemes to large vocabulary Chinese sign language recognition , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[66]  Wen Gao,et al.  Recognition of strong and weak connection models in continuous sign language , 2002, Object recognition supported by user interaction for service robots.

[67]  Zhaoyang Yang,et al.  Fully Convolutional Networks for Continuous Sign Language Recognition , 2020, ECCV.

[68]  Andrew Zisserman,et al.  Learning sign language by watching TV (using weakly aligned subtitles) , 2009, CVPR.

[69]  Joon Son Chung,et al.  BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues , 2020, ECCV.

[70]  Yoshiaki Shirai,et al.  Extraction of Hand Features for Recognition of Sign Language Words , 2002 .

[71]  Wen Gao,et al.  A Real-Time Large Vocabulary Recognition System for Chinese Sign Language , 2001, Gesture Workshop.

[72]  Anitha Julian,et al.  Tamil Sign Language Translator—An Assistive System for Hearing- and Speech-Impaired People , 2019 .

[73]  Tomasz Kapuscinski,et al.  Recognition of Isolated Words of the Polish Sign Language , 2005, CORES.

[74]  Xilin Chen,et al.  Iterative Reference Driven Metric Learning for Signer Independent Isolated Sign Language Recognition , 2016, ECCV.

[75]  Nazli Ikizler-Cinbis,et al.  Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages? , 2019, BMVC.

[76]  Hermann Ney,et al.  Smoothed Disparity Maps for Continuous American Sign Language Recognition , 2009, IbPRIA.

[77]  Salvatore Gaglio,et al.  A Framework for Sign Language Sentence Recognition by Commonsense Context , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[78]  E. Kiran Kumar,et al.  S3DRGF: Spatial 3-D Relational Geometric Features for 3-D Sign Language Representation and Recognition , 2019, IEEE Signal Processing Letters.

[79]  P. V. V. Kishore,et al.  Deep convolutional neural networks for sign language recognition , 2018, 2018 Conference on Signal Processing And Communication Engineering Systems (SPACES).

[80]  Wen Gao,et al.  Signer-Independent Continuous Sign Language Recognition Based on SRN/HMM , 2001, Gesture Workshop.

[81]  J. Bhattacharya,et al.  Hand Gesture Recognition for Sign Language : A New Hybrid Approach , 2011 .

[82]  Helton Hideraldo Bíscaro,et al.  Hand movement recognition for Brazilian Sign Language: A study using distance-based neural networks , 2009, 2009 International Joint Conference on Neural Networks.

[83]  Yong Hu,et al.  Sign Language Fingerspelling Recognition Using Depth Information and Deep Belief Networks , 2018, Int. J. Pattern Recognit. Artif. Intell..

[84]  Khaled Assaleh,et al.  User-independent recognition of Arabic sign language for facilitating communication with the deaf community , 2011, Digit. Signal Process..

[85]  Petros Daras,et al.  A Comprehensive Study on Sign Language Recognition Methods , 2020, ArXiv.

[86]  Pengwen Xiong,et al.  Dynamic Sign Language Recognition Based on Video Sequence With BLSTM-3D Residual Networks , 2019, IEEE Access.

[87]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[88]  Jaafar Alghazo,et al.  ArASL: Arabic Alphabets Sign Language Dataset , 2019, Data in brief.

[89]  Mieke Van Herreweghe,et al.  Sign Language Recognition with Transformer Networks , 2020, LREC.

[90]  Ahmet Alp Kindiroglu,et al.  BosphorusSign22k Sign Language Recognition Dataset , 2020, SIGNLANG.

[91]  Surendra Ranganath,et al.  Towards subject independent continuous sign language recognition: A segment and merge approach , 2014, Pattern Recognit..

[92]  Xilin Chen,et al.  Fast sign language recognition benefited from low rank approximation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[93]  Yann LeCun,et al.  A Closer Look at Spatiotemporal Convolutions for Action Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[94]  Lihong Zheng,et al.  Sign language recognition using depth images , 2016, 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[95]  SystemBritta,et al.  Towards an Automatic Sign Language Translation , 1999 .

[96]  Vasiliki Kosmidou,et al.  Enhanced Sign Language Recognition Using Weighted Intrinsic-Mode Entropy and Signer's Level of Deafness , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[97]  Stan Sclaroff,et al.  The American Sign Language Lexicon Video Dataset , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[98]  Suharjito,et al.  Sentence Level Indonesian Sign Language Recognition Using 3D Convolutional Neural Network and Bidirectional Recurrent Neural Network , 2018, 2018 Indonesian Association for Pattern Recognition International Conference (INAPR).

[99]  Khaled Assaleh,et al.  Multiple Proposals for Continuous Arabic Sign Language Recognition , 2019 .

[100]  Karl-Friedrich Kraiss,et al.  Video-based sign recognition using self-organizing subunits , 2002, Object recognition supported by user interaction for service robots.

[101]  Lukasz Wandzik,et al.  Hand Shape Recognition Using Very Deep Convolutional Neural Networks , 2018, ICCCV '18.

[102]  Luigi Cinque,et al.  Exploiting Recurrent Neural Networks and Leap Motion Controller for the Recognition of Sign Language and Semaphoric Hand Gestures , 2018, IEEE Transactions on Multimedia.

[103]  Naohiro Fukumura,et al.  Sign language word recognition using via-point information and correlation of they bimanual movements , 2014, 2014 International Conference of Advanced Informatics: Concept, Theory and Application (ICAICTA).

[104]  Fattah Alizadeh,et al.  Kurdish Sign Language Recognition System , 2018, UKH Journal of Science and Engineering.

[105]  Jan Zelinka,et al.  Feature space transforms for Czech sign-language recognition , 2008, INTERSPEECH.

[106]  George Awad,et al.  Boosted subunits: a framework for recognising sign language from videos , 2013, IET Image Process..

[107]  Richard Bowden,et al.  Large Lexicon Detection of Sign Language , 2007, ICCV-HCI.

[108]  Changshui Zhang,et al.  A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training , 2019, IEEE Transactions on Multimedia.

[109]  Chung-Lin Huang,et al.  Sign language recognition using model-based tracking and a 3D Hopfield neural network , 1998, Machine Vision and Applications.

[110]  Nicolas Pugeault,et al.  Spelling it out: Real-time ASL fingerspelling recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[111]  Changsheng Xu,et al.  Discriminative Exemplar Coding for Sign Language Recognition With Kinect , 2013, IEEE Transactions on Cybernetics.

[112]  Ruiduo Yang,et al.  Enhanced Level Building Algorithm for the Movement Epenthesis Problem in Sign Language Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[113]  Yangho Ji,et al.  Sign Language Learning System with Image Sampling and Convolutional Neural Network , 2017, 2017 First IEEE International Conference on Robotic Computing (IRC).

[114]  Zhongfu Ye,et al.  A Novel Chinese Sign Language Recognition Method Based on Keyframe-Centered Clips , 2018, IEEE Signal Processing Letters.

[115]  Seong-Whan Lee,et al.  Combination of manual and non-manual features for sign language recognition based on conditional random field and active appearance model , 2011, 2011 International Conference on Machine Learning and Cybernetics.

[116]  M. Deriche,et al.  Arabic sign language recognition by decisions fusion using Dempster-Shafer theory of evidence , 2013, 2013 Computing, Communications and IT Applications Conference (ComComAp).

[117]  Lale Akarun,et al.  Multi-class classification strategies for Fisher scores of gesture and sign sequences , 2008, 2008 19th International Conference on Pattern Recognition.

[118]  Daniel Kelly,et al.  A framework for continuous multimodal sign language recognition , 2009, ICMI-MLMI '09.

[119]  Murni,et al.  Gesture recognition for Indonesian Sign Language (BISINDO) , 2018, Journal of Physics: Conference Series.

[120]  A E Marble,et al.  Image processing system for interpreting motion in American Sign Language. , 1992, Journal of biomedical engineering.

[121]  Karl-Friedrich Kraiss,et al.  Recent developments in visual sign language recognition , 2008, Universal Access in the Information Society.

[122]  Lars Petersson,et al.  Transferring Cross-Domain Knowledge for Video Sign Language Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[123]  Evgeny Izutov,et al.  ASL Recognition with Metric-Learning based Lightweight Network , 2020, ArXiv.

[124]  Andrew Zisserman,et al.  Employing signed TV broadcasts for automated learning of British Sign Language , 2010 .

[125]  Xia Liu,et al.  Sign recognition using depth image streams , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[126]  Shuo Wang,et al.  Dense Temporal Convolution Network for Sign Language Translation , 2019, IJCAI.

[127]  Xilin Chen,et al.  Semantics constrained dictionary learning for signer-independent sign language recognition , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[128]  Karl-Friedrich Kraiss,et al.  Towards an Automatic Sign Language Recognition System Using Subunits , 2001, Gesture Workshop.

[129]  Xin Yu,et al.  Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[130]  Khaled Assaleh,et al.  Vision-based system for continuous Arabic Sign Language recognition in user dependent mode , 2008, 2008 5th International Symposium on Mechatronics and Its Applications.

[131]  Meng Wang,et al.  Online Early-Late Fusion Based on Adaptive HMM for Sign Language Recognition , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[132]  Changsheng Xu,et al.  Latent support vector machine for sign language recognition with Kinect , 2013, 2013 IEEE International Conference on Image Processing.

[133]  George Awad,et al.  Modelling and segmenting subunits for sign language recognition based on hand motion analysis , 2009, Pattern Recognit. Lett..

[134]  Samir I. Shaheen,et al.  Sign language recognition using a combination of new vision based features , 2011, Pattern Recognit. Lett..

[135]  Wen Gao,et al.  Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[136]  H F Chen,et al.  A fuzzy rule-based approach to recognizing 3-D arm movements. , 2001, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[137]  Richard Bowden,et al.  Learning signs from subtitles: A weakly supervised approach to sign language recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[138]  Luc Van Gool,et al.  Real-time sign language letter and word recognition from depth data , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[139]  Lale Akarun,et al.  A belief-based sequential fusion approach for fusing manual signs and non-manual signals , 2009, Pattern Recognit..

[140]  Lale Akarun,et al.  Sign Language Recognition for Assisting the Deaf in Hospitals , 2016, HBU.

[141]  Mariusz Oszust,et al.  Recognition of signed expressions observed by Kinect Sensor , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[142]  Mohamed F. Tolba,et al.  A proposed PCNN features quality optimization technique for pose-invariant 3D Arabic sign language recognition , 2013, Appl. Soft Comput..

[143]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[144]  Zhongfu Ye,et al.  Continuous sign language recognition using level building based on fast hidden Markov model , 2016, Pattern Recognit. Lett..

[145]  Ayan Banerjee,et al.  Learn2Sign: Explainable AI for Sign Language Learning , 2019, IUI Workshops.

[146]  Shuangquan Wang,et al.  SignFi , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[147]  Dan Guo,et al.  Continuous Sign Language Recognition Based on Pseudo-supervised Learning , 2019, Proceedings of the 2nd Workshop on Multimedia for Accessible Human Computer Interfaces - MAHCI '19.

[148]  Ying Gao,et al.  Real-time sign language recognition based on neural network architecture , 2011, 2011 IEEE 43rd Southeastern Symposium on System Theory.

[149]  H. M. Cooper,et al.  Sign language recognition : Generalising to more complex corpora , 2010 .

[150]  P. V. V. Kishore,et al.  Selfie video based continuous Indian sign language recognition system , 2017, Ain Shams Engineering Journal.

[151]  Hermann Ney,et al.  Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[152]  Petros Maragos,et al.  Model-level data-driven sub-units for signs in videos of continuous Sign Language , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[153]  Charles Markham,et al.  Weakly Supervised Training of a Sign Language Recognition System Using Multiple Instance Learning Density Matrices , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[154]  Ruiduo Yang,et al.  Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign Language Recognition Using Nested Dynamic Programming , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[155]  Francesco Piazza,et al.  A New Italian Sign Language Database , 2012, BICS.

[156]  Stephan Liwicki,et al.  Automatic recognition of fingerspelled words in British Sign Language , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[157]  Yu Zhou,et al.  Unsupervised adaptive sign language recognition based on hypothesis comparison guided cross validation and linguistic prior filtering , 2015, Neurocomputing.

[158]  Xiaojuan Qi,et al.  Self-boosted Gesture Interactive System with ST-Net , 2018, ACM Multimedia.

[159]  Xilin Chen,et al.  Communication tool for the hard of hearings: A large vocabulary sign language recognition system , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[160]  Tamer Shanableh,et al.  Spatio-Temporal Feature-Extraction Techniques for Isolated Gesture Recognition in Arabic Sign Language , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[161]  Nicolas Pugeault,et al.  Sign Spotting Using Hierarchical Sequential Patterns with Temporal Intervals , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[162]  Richard Bowden,et al.  Sign Language Recognition Using Boosted Volumetric Features , 2007, MVA.

[163]  Philippe Dreuw,et al.  Probabilistic sequence models for image sequence processing and recognition , 2012 .

[164]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[165]  Houqiang Li,et al.  Sign Language Recognition using 3D convolutional neural networks , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[166]  Nguyen Duc Dung,et al.  A Comparison of SimpSVM and RVM for Sign Language Recognition , 2017, ICMLSC.

[167]  Oscar Koller,et al.  MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language , 2018, BMVC.

[168]  Murat Saraclar,et al.  Keyword Search for Sign Language , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[169]  Thad Starner,et al.  A novel approach to American Sign Language (ASL) phrase verification using reversed signing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[170]  David Windridge,et al.  A Linguistic Feature Vector for the Visual Interpretation of Sign Language , 2004, ECCV.

[171]  Yuntao Cui,et al.  Appearance-Based Hand Sign Recognition from Intensity Image Sequences , 2000, Comput. Vis. Image Underst..

[172]  Hermann Ney,et al.  RWTH-PHOENIX-Weather: A Large Vocabulary Sign Language Recognition and Translation Corpus , 2012, LREC.

[173]  Houqiang Li,et al.  Deep Grammatical Multi-classifier for Continuous Sign Language Recognition , 2019, 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM).

[174]  M. B. Waldron,et al.  Isolated ASL sign recognition system for deaf persons , 1995 .

[175]  Thad Starner,et al.  American sign language recognition with the kinect , 2011, ICMI '11.

[176]  E. Kiran Kumar,et al.  Training CNNs for 3-D Sign Language Recognition With Color Texture Coded Joint Angular Displacement Maps , 2018, IEEE Signal Processing Letters.

[177]  Aiping Liu,et al.  A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors , 2015, Sensors.

[178]  Moritz Knorr,et al.  The significance of facial features for automatic sign language recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[179]  H. S. Nagendraswamy,et al.  LBPV for Recognition of Sign Language at Sentence Level: An Approach Based on Symbolic Representation , 2017, J. Intell. Syst..

[180]  Yingli Tian,et al.  Recognizing American Sign Language Gestures from Within Continuous Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[181]  Mohamed Mohandes,et al.  Recognition of Two-Handed Arabic Signs Using the CyberGlove , 2013 .

[182]  Chao Xie,et al.  Chinese sign language recognition with adaptive HMM , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[183]  Wen Gao,et al.  Expanding Training Set for Chinese Sign Language Recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[184]  Wen Gao,et al.  Transition movement models for large vocabulary continuous sign language recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[185]  Poonam Sharma,et al.  Sign Language Gesture Recognition using Zernike Moments and DTW , 2018, 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN).

[186]  Changshui Zhang,et al.  Recurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[187]  Mohamed A. Deriche,et al.  Arabic sign language recognition using the leap motion controller , 2014, 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE).

[188]  Wen Gao,et al.  A Fast Sign Word Recognition Method for Chinese Sign Language , 2000, ICMI.

[189]  C.R. Hema,et al.  Extraction of head and hand gesture features for recognition of sign language , 2008, 2008 International Conference on Electronic Design.

[190]  Lale Akarun,et al.  Temporal Accumulative Features for Sign Language Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[191]  Houqiang Li,et al.  Sign language recognition with long short-term memory , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[192]  Suharjito,et al.  Sign Language Recognition Using Modified Convolutional Neural Network Model , 2018, 2018 Indonesian Association for Pattern Recognition International Conference (INAPR).

[193]  Kenneth P. Camilleri,et al.  Phonologically-Meaningful Subunits for Deep Learning-Based Sign Language Recognition , 2020, ECCV Workshops.

[194]  Chin-Hui Lee,et al.  Sign Transition Modeling and a Scalable Solution to Continuous Sign Language Recognition for Real-World Applications , 2016, ACM Trans. Access. Comput..

[195]  Nicolas Pugeault,et al.  Reading the signs: A video based sign dictionary , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[196]  Kartik Shenoy,et al.  Real-time Indian Sign Language (ISL) Recognition , 2018, 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT).

[197]  Robert W. Lindeman,et al.  A new instrumented approach for translating American Sign Language into sound and text , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[198]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[199]  Debi Prosad Dogra,et al.  A position and rotation invariant framework for sign language recognition (SLR) using Kinect , 2018, Multimedia Tools and Applications.

[200]  Hermann Ney,et al.  Using Different Aspects of the Signings for Appearance-based Sign Language Recognition , 2008 .

[201]  Roland Pfau,et al.  Nonmanuals: their grammatical and prosodic roles , 2010 .

[202]  Hermann Ney,et al.  Improving Continuous Sign Language Recognition: Speech Recognition Techniques and System Design , 2013, SLPAT.

[203]  Hermann Ney,et al.  Speech recognition techniques for a sign language recognition system , 2007, INTERSPEECH.

[204]  Fei Wang,et al.  A Bi-directional Interactive System of Sign Language and Visual Speech Based on Portable Devices , 2019, 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[205]  Lale Akarun,et al.  BosphorusSign: A Turkish Sign Language Recognition Corpus in Health and Finance Domains , 2016, LREC.

[206]  Raúl Rojas,et al.  Sign Language Recognition Using Kinect , 2012, ICAISC.

[207]  Dimitris N. Metaxas,et al.  Handshapes and Movements: Multiple-Channel American Sign Language Recognition , 2003, Gesture Workshop.

[208]  Ming Ouhyoung,et al.  A sign language recognition system using hidden markov model and context sensitive search , 1996, VRST.

[209]  Karl-Friedrich Kraiss,et al.  Towards a Video Corpus for Signer-Independent Continuous Sign Language Recognition , 2007 .

[210]  Wen Gao,et al.  A Chinese sign language recognition system based on SOFM/SRN/HMM , 2004, Pattern Recognit..

[211]  Maria Letizia Corradini,et al.  Development of a Wearable Device for Sign Language Translation , 2017, HFR.

[212]  Xu Zhang,et al.  A Component-Based Vocabulary-Extensible Sign Language Gesture Recognition Framework , 2016, Sensors.

[213]  James M. Rehg,et al.  Learning the basic units in American Sign Language using discriminative segmental feature selection , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[214]  P. Maragos,et al.  Data-Driven Sub-Units and Modeling Structure for Continuous Sign Language Recognition with Multiple Cues , 2010 .

[215]  Wen Gao,et al.  Signer-independent sign language recognition based on SOFM/HMM , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[216]  Johannes Wagner,et al.  Bi-channel sensor fusion for automatic sign language recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[217]  Bidyut B. Chaudhuri,et al.  A Modified LSTM Model for Continuous Sign Language Recognition Using Leap Motion , 2019, IEEE Sensors Journal.

[218]  Wen Gao,et al.  Viewpoint invariant sign language recognition , 2007, Comput. Vis. Image Underst..

[219]  Mieke Van Herreweghe,et al.  Gesture and Sign Language Recognition with Temporal Residual Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[220]  Michael Ng,et al.  Self-Attention-Based Fully-Inception Networks for Continuous Sign Language Recognition , 2020, ECAI.

[221]  Shan Lu,et al.  The Recognition Algorithm with Non-contact for Japanese Sign Language Using Morphological Analysis , 1997, Gesture Workshop.

[222]  Guang Li,et al.  Sign Language Recognition and Translation with Kinect , 2013 .

[223]  Bernabé Linares-Barranco,et al.  Introduction and Analysis of an Event-Based Sign Language Dataset , 2020, 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020).

[224]  Yingli Tian,et al.  Multi-modality American Sign Language recognition , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[225]  Anant Agarwal,et al.  Sign language recognition using Microsoft Kinect , 2013, 2013 Sixth International Conference on Contemporary Computing (IC3).

[226]  Masaru Takeuchi,et al.  A method for recognizing a sequence of sign language words represented in a Japanese sign language sentence , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[227]  E. Kiran Kumar,et al.  Motionlets Matching With Adaptive Kernels for 3-D Indian Sign Language Recognition , 2018, IEEE Sensors Journal.

[228]  Mariusz Oszust,et al.  Polish sign language words recognition with Kinect , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[229]  Houqiang Li,et al.  Iterative Alignment Network for Continuous Sign Language Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[230]  W. Kadous.  GRASP: Recognition of Australian Sign Language Using Instrumented Gloves , 1995 .

[231]  Kai Zhao,et al.  Investigation on AQ11, ID3 and the principle of discernibility matrix , 2008, Journal of Computer Science and Technology.

[232]  Sudeep Sarkar,et al.  Unsupervised Modeling of Signs Embedded in Continuous Sentences , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[233]  Houqiang Li,et al.  Sign Language Recognition with Multi-modal Features , 2016, PCM.

[234]  Khaled Assaleh,et al.  Glove-Based Continuous Arabic Sign Language Recognition in User-Dependent Mode , 2015, IEEE Transactions on Human-Machine Systems.

[235]  Jie Huang,et al.  Video-based Sign Language Recognition without Temporal Segmentation , 2018, AAAI.

[236]  Hermann Ney,et al.  Extensions of the Sign Language Recognition and Translation Corpus RWTH-PHOENIX-Weather , 2014, LREC.

[237]  Marly Guimarães Fernandes Costa,et al.  A fully automatic method for recognizing hand configurations of Brazilian sign language , 2017 .

[238]  Hermann Ney,et al.  Neural Sign Language Translation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[239]  David Windridge,et al.  Induced Decision Fusion in Automated Sign Language Interpretation: Using ICA to Isolate the Underlying Components of Sign , 2004, Multiple Classifier Systems.

[240]  Petros Daras,et al.  A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition , 2018, 2018 IEEE International Conference on Imaging Systems and Techniques (IST).

[241]  Yasuo Horiuchi,et al.  Sign Language Recognition Based on Position and Movement Using Multi-Stream HMM , 2008, 2008 Second International Symposium on Universal Communication.

[242]  Daniel Kelly,et al.  Recognizing Spatiotemporal Gestures and Movement Epenthesis in Sign Language , 2009, 2009 13th International Machine Vision and Image Processing Conference.

[243]  Napoleon H. Reyes,et al.  A New 2D Static Hand Gesture Colour Image Dataset for ASL Gestures , 2011 .

[244]  Hermann Ney,et al.  Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[245]  Petros Maragos,et al.  Product-HMMs for automatic sign language recognition , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[246]  Laura Cristina Lanzarini,et al.  A Study of Convolutional Architectures for Handshape Recognition applied to Sign Language , 2017 .

[247]  Stan Sclaroff,et al.  Exploiting phonological constraints for handshape inference in ASL video , 2011, CVPR 2011.

[248]  Heike Brock,et al.  Learning Motion Disfluencies for Automatic Sign Language Segmentation , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[249]  Xilin Chen,et al.  Sparse Observation (SO) Alignment for Sign Language Recognition , 2016, Neurocomputing.

[250]  John K. Tsotsos,et al.  Definition and recovery of kinematic features for recognition of American sign language movements , 2008, Image Vis. Comput..

[251]  Petros Maragos,et al.  Dynamic affine-invariant shape-appearance handshape features and classification in sign language videos , 2013, J. Mach. Learn. Res..

[252]  P. V. V. Kishore,et al.  A Video Based Indian Sign Language Recognition System (INSLR) Using Wavelet Transform and Fuzzy Logic , 2012 .

[253]  Mi Zhang,et al.  DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation , 2017, SenSys.

[254]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[255]  Hermann Ney,et al.  Modality Combination Techniques for Continuous Sign Language Recognition , 2013, IbPRIA.

[256]  Haibo Wang,et al.  Chinese sign language recognition with 3D hand motion trajectories and depth images , 2014, Proceeding of the 11th World Congress on Intelligent Control and Automation.

[257]  Ming Ouhyoung,et al.  A Real‐time Continuous Alphabetic Sign Language to Speech Conversion VR System , 1995, Comput. Graph. Forum.

[258]  Franco Ronchetti,et al.  Handshape recognition for Argentinian Sign Language using ProbSom , 2016 .

[259]  Thad Starner,et al.  Towards a one-way American sign language translator , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[260]  Vadim Kimmelman,et al.  Towards Real-time Sign Language Interpreting Robot: Evaluation of Non-manual Components on Recognition Accuracy , 2019, CVPR Workshops.

[261]  Wen Gao,et al.  A Parallel Multistream Model for Integration of Sign Language Recognition and Lip Motion , 2000, ICMI.

[262]  Houqiang Li,et al.  Attention-Based 3D-CNNs for Large-Vocabulary Sign Language Recognition , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[263]  Dan Guo,et al.  Parallel Temporal Encoder For Sign Language Translation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[264]  Oscar Koller,et al.  SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[265]  Hermann Ney,et al.  Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition , 2016, BMVC.

[266]  Andrew Zisserman,et al.  Learning sign language by watching TV (using weakly aligned subtitles) , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[267]  Houqiang Li,et al.  Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition , 2018, IJCAI.

[268]  Xilin Chen,et al.  Weakly Supervised Metric Learning towards Signer Adaptation for Sign Language Recognition , 2015, BMVC.

[269]  Marek Hrúz,et al.  Sign Language Numeral Gestures Recognition Using Convolutional Neural Network , 2018, ICR.

[270]  Shinichi Tamura,et al.  Recognition of sign language motion images , 1988, Pattern Recognit..

[271]  T. Kobayashi,et al.  Partly-hidden Markov model and its application to gesture recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[272]  Rajkumar Palaniappan,et al.  A phoneme based sign language recognition system using skin color segmentation , 2010, 2010 6th International Colloquium on Signal Processing & its Applications.

[273]  Sarah Ebling,et al.  SMILE Swiss German Sign Language Dataset , 2018, LREC.

[274]  Andrew Zisserman,et al.  Minimal Training, Large Lexicon, Unconstrained Sign Language Recognition , 2004, BMVC.

[275]  Xiaoxu Li,et al.  Chinese Sign Language Recognition Based on SHS Descriptor and Encoder-Decoder LSTM Model , 2017, CCBR.

[276]  Petros Daras,et al.  Continuous Sign Language Recognition Through Cross-Modal Alignment of Video and Text Embeddings in a Joint-Latent Space , 2020, IEEE Access.

[277]  Vassilis Athitsos,et al.  A System for Large Vocabulary Sign Search , 2010, ECCV Workshops.

[278]  Thad Starner,et al.  American sign language recognition in game development for deaf children , 2006, Assets '06.

[279]  Houqiang Li,et al.  Sign Language Recognition Based on Trajectory Modeling with HMMs , 2016, MMM.

[280]  Hermann Ney,et al.  Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[281]  Zeungnam Bien,et al.  Real-time recognition system of Korean sign language based on elementary components , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[282]  Petros Maragos,et al.  Advances in phonetics-based sub-unit modeling for transcription alignment and sign language recognition , 2011, CVPR 2011 WORKSHOPS.

[283]  Hermann Ney,et al.  Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers , 2015, Comput. Vis. Image Underst..

[284]  Lale Akarun,et al.  Automatic sign segmentation from continuous signing via multiple sequence alignment , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[285]  Liya Ding,et al.  Modelling and recognition of the linguistic components in American Sign Language , 2009, Image Vis. Comput..

[286]  Surendra Ranganath,et al.  Sign Language Phoneme Transcription with Rule-based Hand Trajectory Segmentation , 2010, J. Signal Process. Syst..

[287]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[288]  Teresa Bernarda Ludermir,et al.  Extreme Learning Machine for Real Time Recognition of Brazilian Sign Language , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[289]  Zhaoyang Yang,et al.  SF-Net: Structured Feature Network for Continuous Sign Language Recognition , 2019, ArXiv.

[290]  Hermann Ney,et al.  Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition , 2016 .

[291]  Kostas Karpouzis,et al.  Non parametric, self organizing, scalable modeling of spatiotemporal inputs: The sign language paradigm , 2012, Neural Networks.

[292]  Kirsti Grobel,et al.  Video-Based Sign Language Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[293]  A. Karpov,et al.  Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled , 2012, Pattern Recognition and Image Analysis.

[294]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[295]  M. Maraqa,et al.  Recognition of Arabic Sign Language (ArSL) using recurrent neural networks , 2008, 2008 First International Conference on the Applications of Digital Information and Web Technologies (ICADIWT).