'Online recognition of Chinese characters: the state-of-the-art

Online handwriting recognition is gaining renewed interest owing to the increase of pen computing applications and new pen input devices. The recognition of Chinese characters is different from western handwriting recognition and poses a special challenge. To provide an overview of the technical status and inspire future research, this paper reviews the advances in online Chinese character recognition (OLCCR), with emphasis on the research works from the 1990s. Compared to the research in the 1980s, the research efforts in the 1990s aimed to further relax the constraints of handwriting, namely, the adherence to standard stroke orders and stroke numbers and the restriction of recognition to isolated characters only. The target of recognition has shifted from regular script to fluent script in order to better meet the requirements of practical applications. The research works are reviewed in terms of pattern representation, character classification, learning/adaptation, and contextual processing. We compare important results and discuss possible directions of future research.

[1]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Ernesto F. Yhap,et al.  An On-Line Chinese Character Recognition System , 1981, IBM J. Res. Dev..

[3]  M. P. Perrone,et al.  Handwritten document retrieval , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[4]  Shigeki Sagayama,et al.  Substroke approach to HMM-based on-line Kanji handwriting recognition , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[5]  Réjean Plamondon,et al.  Automatic Signature Verification: The State of the Art - 1989-1993 , 1994, Int. J. Pattern Recognit. Artif. Intell..

[6]  Kuo-Chin Fan,et al.  Peripheral and global features for use in coarse classification of Chinese characters , 1997, Pattern Recognit..

[7]  Hang Joon Kim,et al.  On-line Chinese character recognition using ART-based stroke classification , 1996, Pattern Recognit. Lett..

[8]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[9]  Masaki Nakagawa,et al.  The state of the art in Japanese online handwriting recognition compared to techniques in western handwriting recognition , 2003, Document Analysis and Recognition.

[10]  Fumitaka Kimura,et al.  Modified Quadratic Discriminant Functions and the Application to Chinese Character Recognition , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Y.-J. Liu,et al.  A new approach to on-line handwritten Chinese character recognition , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[12]  Zheng Zhang,et al.  Optical Recognition of Chinese Characters , 1989 .

[13]  Chien-Cheng Tseng,et al.  Candidate Selection in On-Line Chinese Character Recognition System Using Voting Scheme , 1999, J. Inf. Sci. Eng..

[14]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[15]  Ching,et al.  The State of the Art in On-Line Handwriting Recognition , 2000 .

[16]  ERKKI OJA,et al.  The ALSM algorithm - an improved subspace method of classification , 1983, Pattern Recognit..

[17]  George Nagy,et al.  Style-consistency in isogenous patterns , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[18]  Xiaolin Li,et al.  Corner detection and shape classification of on-line handprinted kanji strokes , 1993, Pattern Recognit..

[19]  Robert M. Haralick,et al.  The Consistent Labeling Problem: Part I , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Kazuhiko Yamamoto,et al.  On-line handwriting character string separation method using network expression , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[21]  Shigeki Matsuda,et al.  Context-dependent substroke model for HMM-based on-line handwriting recognition , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[22]  Masaki Nakagawa,et al.  Collection and analysis of on-line handwritten Japanese character patterns , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[23]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[24]  Wen-Tsuen Chen,et al.  A hierarchical deformation model for on-line cursive script recognition , 1994, Pattern Recognit..

[25]  Dorothea Blostein.  General Diagram-Recognition Methodologies , 1995, GREC.

[26]  Andrew K. C. Wong,et al.  An algorithm for graph optimal monomorphism , 1990, IEEE Trans. Syst. Man Cybern..

[27]  Suh-Yin Lee,et al.  A Hierarchical Representation for the Reference Database of On-Line Chinese Character Recognition , 1996, SSPR.

[28]  Kazutaka Yamasaki.  Automatic prototype stroke generation based on stroke clustering for on-line handwritten Japanese character recognition , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[29]  KEH-JIANN CHEN,et al.  A System for on-Line Recognition of Chinese Characters , 1988, Int. J. Pattern Recognit. Artif. Intell..

[30]  Anil K. Jain,et al.  On-line signature verification, , 2002, Pattern Recognit..

[31]  Taizo Iijima,et al.  A Theory of Character Recognition by Pattern Matching Method , 1974 .

[32]  K. Yamada,et al.  On-line Japanese character recognition experiments by an off-line method based on normalization-cooperated feature extraction , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[33]  Yuji Matsumoto,et al.  Japanese OCR Error Correction Using Stochastic Morphological Analyzer and Probabilistic Word N-gram Model , 2000, Int. J. Comput. Process. Orient. Lang..

[34]  Wen-Hsiang Tsai,et al.  Attributed String Matching with Merging for Shape Recognition , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Daniel P. Lopresti,et al.  CROSS­DOMAIN SEARCHING USING HANDWRITTEN QUERIES , 2004 .

[36]  Thomas Zimmerman,et al.  Pen computing: challenges and applications , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[37]  A. Tanaka,et al.  Online recognition of freely handwritten Japanese characters using directional feature densities , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[38]  Wen-Tsuen Chen,et al.  A hierarchical deformation model for online cursive script recognition , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[39]  Takahiko Kawatani.  Character recognition performance improvement using personal handwriting characteristics , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[40]  Jianzhuang Liu,et al.  Online Chinese character recognition using attributed relational graph matching , 1996 .

[41]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[42]  J. Tsukumo,et al.  Classification of handprinted Chinese characters using nonlinear normalization and correlation methods , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[43]  Urs Ramer,et al.  An iterative procedure for the polygonal approximation of plane curves , 1972, Comput. Graph. Image Process..

[44]  Jianzhuang Liu,et al.  Stroke order and stroke number free on-line Chinese character recognition using attributed relational graph matching , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[45]  Kazumi Odaka,et al.  On-Line Cursive Kanji Character Recognition Using Stroke-Based Affine Transformation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Masaki Nakagawa.  Non-keyboard input of Japanese text on-line recognition of handwritten characters as the most hopeful approach , 1990 .

[48]  Jinho Kim,et al.  Bayesian network modeling of strokes and their relationships for on-line handwriting recognition , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[49]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[50]  Han Ngee. Tan,et al.  An on-line Chinese character recognition system , 1988 .

[51]  Kohji Fukunaga,et al.  Introduction to Statistical Pattern Recognition-Second Edition , 1990 .

[52]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[53]  K. Takahashi,et al.  A fast HMM algorithm for on-line handwritten character recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[54]  Kuo-Chin Fan,et al.  On-line recognition by deviation-expansion model and dynamic programming matching , 1993, Pattern Recognit..

[55]  R. Casey.  Moment normalization of handprinted characters , 1970 .

[56]  Shigeki Sagayama,et al.  Generation of hierarchical dictionary for stroke-order free Kanji handwriting recognition based on substroke HMM , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[57]  Réjean Plamondon,et al.  On-line handwriting recognition. , 1999 .

[58]  Hiromitsu Yamada,et al.  A nonlinear normalization method for handprinted kanji character recognition - line density equalization , 1990, Pattern Recognit..

[59]  Wentai Liu,et al.  Optical recognition of handwritten Chinese characters: Advances since 1980 , 1993, Pattern Recognit..

[60]  Masaki Nakagawa,et al.  On-line handwritten character pattern database sampled in a sequence of sentences without any writing instructions , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[61]  Shigeaki Watanabe,et al.  Subspace method to pattern recognition , 1973 .

[62]  Sung-Jung Cho,et al.  Bayesian network modeling of strokes and their relationships for on-line handwriting recognition , 2004, Pattern Recognit..

[63]  Seong-Whan Lee,et al.  Nonlinear shape normalization methods for the recognition of large-set handwritten characters , 1994, Pattern Recognit..

[64]  Suh-Yin Lee,et al.  On-Line Chinese Character Recognition via A Representation of Spatial Relationships between Strokes , 1997, Int. J. Pattern Recognit. Artif. Intell..

[65]  Naomi Iwayama,et al.  ADAPTIVE CONTEXT PROCESSING IN ON-LINE HANDWRITTEN CHARACTER RECOGNITION , 2004 .

[66]  Masaki Nakagawa,et al.  A learning algorithm for structured character pattern representation used in online recognition of handwritten Japanese characters , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[67]  Dit-Yan Yeung,et al.  Error detection, error correction and performance evaluation in on-line mathematical expression recognition , 2001, Pattern Recognit..

[68]  Ching Y. Suen,et al.  The State of the Art in Online Handwriting Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Zen Chen,et al.  Preclassification of handwritten Chinese characters based on basic stroke substructures , 1995, Pattern Recognit. Lett..

[70]  Yoshimitsu Komiya,et al.  RAV (reparameterized angle variations) algorithm for online handwriting recognition , 2001, International Journal on Document Analysis and Recognition.

[71]  T. Wakahara.  Online cursive script recognition using local affine transformation , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[72]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[74]  Ruwei Dai,et al.  On-Line Handwritten Chinese Character Recognition Directed Components with Dynamic Templates , 1998, Int. J. Pattern Recognit. Artif. Intell..

[75]  T. Matsumoto,et al.  On-line Hand Writing Recognition by Discrete HMM with Fast Learning , 1999 .

[76]  Tetsu Ohishi,et al.  A Pen Input On-Line Signature Verifier Integrating Position, Pressure and Inclination Trajectories , 2001 .

[77]  Masaki Nakagawa,et al.  Precise Candidate Selection for Large Character Set Recognition by Confidence Evaluation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[78]  Erkki Oja,et al.  Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters , 2001, International Journal on Document Analysis and Recognition.

[79]  Suh-Yin Lee,et al.  On-line handwriting recognition of Chinese characters via a rule-based approach , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[80]  Kuo-Chin Fan,et al.  Radical-based neighboring segment matching method for on-line Chinese character recognition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[81]  Wen-Yen Wu,et al.  Detecting the Dominant Points by the Curvature-Based Polygonal Approximation , 1993, CVGIP Graph. Model. Image Process..

[82]  Kuo-Chin Fan,et al.  Confusion set recognition of on-line Chinese characters by artificial intelligence technique , 1995, Pattern Recognit..

[83]  Masaki Nakagawa,et al.  Structural Learning of Character Patterns for On-Line Recognition of Hand-Written Japanese Characters , 1996, SSPR.

[84]  Wenhao Shu,et al.  A HYBRID POST-PROCESSING SYSTEM FOR HANDWRITTEN CHINESE CHARACTER RECOGNITION , 2002 .

[85]  Fumio Yoda,et al.  Online handwritten Chinese character recognition; comparison and improvement to Japanese Kanji recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[86]  John C. Platt,et al.  QuickStroke: an incremental on-line Chinese handwriting recognition system , 2002, Object recognition supported by user interaction for service robots.

[87]  Kuo-Chin Fan,et al.  Knowledge model based approach in recognition of on-line Chinese characters , 1994, IEEE J. Sel. Areas Commun..

[88]  Kuo-Chin Fan,et al.  Coarse classification of on-line Chinese characters via structure feature-based method , 1994, Pattern Recognit..

[89]  Kazuhiko Yamamoto,et al.  On-line handwritten character recognition method using directional features and clockwise/counterclockwise direction-change features , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[90]  K. Yamada,et al.  ON­LINE CHARACTER RECOGNITION ADAPTIVELY CONTROLLED BY HANDWRITING QUALITY , 2004 .

[91]  Louis Vuurpijl,et al.  New use for the pen: outline-based image queries , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[92]  Chien-Cheng Tseng,et al.  On-line chinese character recognition with effective candidate radical and candidate character selections , 1996, Pattern Recognit..

[93]  Hiroshi Tanaka,et al.  An Adaptation Method Based on Template Cache for Online Character Recognition , 2001 .

[94]  John Bennett,et al.  The effect of large training set sizes on online Japanese Kanji and English cursive recognizers , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[95]  King-Sun Fu,et al.  Subgraph error-correcting isomorphisms for syntactic pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[96]  Jianzhuang Liu,et al.  Two-layer assignment method for online Chinese character recognition , 2000 .

[97]  Wen-Tsuen Chen,et al.  A stochastic representation of cursive Chinese characters for on-line recognition , 1997, Pattern Recognit..

[98]  Richard Zanibbi,et al.  Recognizing Mathematical Expressions Using Tree Transformation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[99]  Masaki Nakagawa,et al.  Online writing-box-free recognition of handwritten Japanese text considering character size variations , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[100]  Mutsuo Sano,et al.  Analysis and evaluation of dictionary learning on a handy type pen-input interface for personal use , 2003, Systems and Computers in Japan.

[101]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[102]  Paul Y. S. Cheung,et al.  Fuzzy-attribute graph with application to Chinese character recognition , 1992, IEEE Trans. Syst. Man Cybern..

[103]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[104]  Gerhard Rigoll,et al.  Multimedia database retrieval using hand-drawn sketches , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[105]  Kazuhiko Yamamoto,et al.  On-line handwriting character recognition using direction-change features that consider imaginary strokes , 1999, Pattern Recognit..

[106]  Hiroshi Sako,et al.  Handwritten Chinese character recognition: alternatives to nonlinear normalization , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[107]  Masaki Nakagawa,et al.  HANDWRITING-BASED USER INTERFACES EMPLOYING ON-LINE HANDWRITING RECOGNITION , 1999 .

[108]  Chen Hong,et al.  Segmentation and Recognition of Continuous Handwriting Chinese Text , 1998, Int. J. Pattern Recognit. Artif. Intell..

[109]  Hang Joon Kim,et al.  On-line recognition of handwritten chinese characters based on hidden markov models , 1997, Pattern Recognit..

[110]  Ai-Jia Hsieh,et al.  Bipartite weighted matching for on-line handwritten Chinese character recognition , 1995, Pattern Recognit..

[111]  Michio Umeda.  Advances in Recognition Methods for Handwritten Kanji Characters (Special issue on Character Recognition and Document Understanding) , 1996 .

[112]  R M Haralick,et al.  The consistent labeling problem: part I. , 1979, IEEE transactions on pattern analysis and machine intelligence.

[113]  Y. J. Liu,et al.  A structural approach to online Chinese character recognition , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[114]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[115]  Sargur N. Srihari,et al.  On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[116]  Ju-Wei Tai,et al.  Some Research Achievements on Chinese Character Recognition in China , 1991, Int. J. Pattern Recognit. Artif. Intell..

[117]  Catherine G. Wolf,et al.  On-Line Run-on Character recognizer: Design and Performance , 1991, Int. J. Pattern Recognit. Artif. Intell..

[118]  C. Myers,et al.  A level building dynamic time warping algorithm for connected word recognition , 1981 .

[119]  Hiroshi Murase.  Online recognition of free-format Japanese handwritings , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[120]  K. Yamada,et al.  A maximum-likelihood approach to segmentation-based recognition of unconstrained handwriting text , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[121]  Masaki Nakagawa,et al.  A new warping technique for normalizing likelihood of multiple classifiers and its effectiveness in combined on-line/off-line japanese character recognition , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[122]  Masaki Nakagawa,et al.  Evaluation of prototype learning algorithms for nearest-neighbor classifier in application to handwritten character recognition , 2001, Pattern Recognit..

[123]  Xiaoqing Ding,et al.  Recognizing on-line handwritten Chinese character via FARG matching , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[124]  Jin Hyung Kim,et al.  Statistical Character Structure Modeling and Its Application to Handwritten Chinese Character Recognition , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[125]  Wen-Hsiang Tsai,et al.  Attributed String Matching by Split-and-Merge for On-Line Chinese Character Recognition , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[126]  Erkki Oja,et al.  Adaptation of Prototype Sets in On-line Recognition of Isolated Handwritten Latin Characters , 1999 .

[127]  Masaki Nakagawa,et al.  Two on-line Japanese character databases in Unipen format , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[128]  Xiaoqing Ding,et al.  Spatio-temporal unified model for on-line handwritten Chinese character recognition , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[129]  K. Ishigaki.  A Top-Down On-line Handwritten Character Recognition Method via the Denotation of Variation , 1988 .

[130]  Shigeki Sagayama,et al.  Pen pressure features for writer-independent on-line handwriting recognition based on substroke HMM , 2002, Object recognition supported by user interaction for service robots.

[131]  Tetsushi Wakabayashi,et al.  Improvement of handwritten Japanese character recognition using weighted direction code histogram , 1997, Pattern Recognit..

[132]  Akira Suzuki,et al.  On-line cursive Kanji character recognition as stroke correspondence problem , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[133]  Azriel Rosenfeld,et al.  Angle Detection on Digital Curves , 1973, IEEE Transactions on Computers.

[134]  Cheng-Lin Liu,et al.  Preprocessing and statistical/structural feature extraction for handwritten numeral recognition , 1997 .

[135]  Chorkin Chan,et al.  Postprocessing statistical language models for handwritten Chinese character recognizer , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[136]  Ching Y. Suen,et al.  n-Gram Statistics for Natural Language Understanding and Text Processing , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[137]  K. Ishigaki,et al.  Hybrid pen-input character recognition system based on integration of online-offline recognition , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[138]  Suh-Yin Lee,et al.  On-line handwritten Chinese character recognition via a fuzzy attribute representation , 1994, Image Vis. Comput..

[139]  Isabelle Guyon,et al.  UNIPEN project of on-line data exchange and recognizer benchmarks , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[140]  Masafumi Hagiwara,et al.  Large scale on-line handwritten Chinese character recognition using successor method based on stochastic regular grammar , 1999, Pattern Recognit..

[141]  NOBUYASU ITOH.  Japanese language model based on bigrams and its application to on-line character recognition , 1995, Pattern Recognit..

[142]  Jungpil Shin,et al.  Optimal stroke-correspondence search method for on-line character recognition , 2002, Pattern Recognit. Lett..

[143]  Masaki Nakagawa,et al.  Robust and highly customizable recognition of online handwritten Japanese characters , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[144]  Zen Chen,et al.  Handwritten Chinese character analysis and preclassification using stroke structural sequence , 1996, Proceedings of 13th International Conference on Pattern Recognition.