Developing Domain-Specific Gesture Recognizers for Smart Diagram Environments

Computer understanding of visual languages in pen-based environments requires a combination of lexical analysis in which the basic tokens are recognized from hand-drawn gestures and syntax analysis in which the structure is recognized. Typically, lexical analysis relies on statistical methods while syntax analysis utilizes grammars. The two stages are not independent: contextual information provided by syntax analysis is required for lexical disambiguation. Previous research into visual language recognition has focussed on syntax analysis while relatively little research has been devoted to lexical analysis and its integration with syntax analysis. This paper describes GestureLab, a tool designed for building domain-specific gesture recognizers, and its integration with Cider, a grammar engine that uses GestureLab recognizers and parses visual languages. Recognizers created with GestureLab perform probabilistic lexical recognition with disambiguation occurring during parsing based on contextual syntactic information. Creating domain-specific gesture recognizers is not a simple task. It requires significant amounts of experimentation and training with large gesture corpora to determine a suitable set of features and classifier algorithm. GestureLab supports such experimentation and facilitates collaboration by allowing corpora to be shared via remote databases.

[1]  Zenon Kulpa,et al.  DIAGRAMMATIC REPRESENTATION AND REASONING , 1994 .

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Gerard Allwein,et al.  Intelligent Diagrammatic Interfaces: State of the Art , 2002, Diagrammatic Representation and Reasoning.

[4]  Chih-Jen Lin,et al.  Errata to "A comparison of methods for multiclass support vector machines" , 2002, IEEE Trans. Neural Networks.

[5]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[6]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Minimum Spanning Trees in Hierarchical Multiclass Support Vector Machines Generation , 2005, IEA/AIE.

[7]  Wenyin Liu,et al.  Advances in Web-Based Learning – ICWL 2004 , 2004, Lecture Notes in Computer Science.

[8]  Bidyut Baran Chaudhuri,et al.  Recognition of online handwritten mathematical expressions , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[10]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[11]  Wenyin Liu.  On-line Graphics Recognition: State-of-the-Art , 2003, GREC.

[12]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[13]  Kim Marriott,et al.  On the Classification of Visual Languages by Grammar Hierarchies , 1997, J. Vis. Lang. Comput..

[14]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[15]  Kim Marriott,et al.  Cider: A Component-Based Toolkit for Creating Smart Diagram Environments , 2004, Diagrams.

[16]  Moonis Ali,et al.  Innovations in Applied Artificial Intelligence , 2005 .