Efficient search strategy in structural analysis for handwritten mathematical expression recognition

Problems with local ambiguities in handwritten mathematical expressions (MEs) are often resolved at global level. Therefore, keeping local ambiguities is desirable for high accuracy, with a hope that they may be resolved by later global analyses. We propose a layered search framework for handwritten ME recognition. From given handwritten input strokes, ME structures are expanded by adding symbol hypotheses one by one, representing ambiguities of symbol identities and spatial relationships as numbers of branches in the expansion. We also propose a novel heuristic predicting how likely the set of remaining input strokes forms valid spatial relationships with the current partially interpreted structure. Further complexity reduction is achieved by delaying the symbol identity decision. The elegance of our approach is that the search result would be unchanged even if we prune out unpromising branches of the search. Therefore, we can examine a much larger number of local hypotheses with a limited amount of computing resource in making global level decisions. The experimental evaluation shows promising results of the efficiency of the proposed approach and the performance of our system, which results from the system's capacity to examine a large number of possibilities.