A Robust Approach to Detecting Text from Images of Whiteboards and Handwritten Notes

Detecting text from the images of whiteboards and handwritten notes is an important yet under-researched topic. In this paper, we present a robust approach to solving this challenging problem as follows. First, given a color image, colorenhanced Contrasting Extremal Regions (CERs) are extracted from its grayscale image as candidate text connected components (CCs). Second, four shallow neural networks are used to preprune efficiently most of unambiguous non-text CCs. Third, a Fast R-CNN based approach is proposed to filter out remaining nontext CCs by leveraging contextual information and to estimate the corresponding text-line orientation in the position of each remaining text CC. Fourth, each pair of the remaining text CCs within a certain distance and orientation constraint are connected to construct a directed graph. Finally, based on the estimated textline orientations, candidate text-lines are generated easily by pruning greedily redundant edges in the graph to make each vertex have at most one direct successor and one direct predecessor, respectively. Our proposed approach has achieved promising results on an in-house testing set consisting of 285 camera-captured images of whiteboards and handwritten notes.