Region Proposal for Pattern Spotting in Historical Document Images

Pattern spotting consists in searching in a document image for the occurrences of a queried graphical object. The main challenge in pattern spotting is that the query image is generally small and the occurrences may be located at any random places in the image. Rather than exhaustively indexing all possible subwindows extracted from the document images, the common way is to rely on a segmentation or a document layout analysis to limit the search space. However, there is no segmentation nor document layout analysis technique reliable enough for historical document images. Region proposal, a technique used to generate a set of regions potentially containing an object, has contributed to many state of the art object detection systems recently. Although it is initially proposed for object detection, we will show that region proposal also offers promising results for document images, particularly in the case of pattern spotting. In this paper, we aim at investigating the use of region proposal to produce high quality subwindows to replace the usual document layout analysis step and the blind sliding windowing step. From experiments conducted on the DocExplore dataset, we show that region proposal generates a comparable number of subwindows while helping the system to achieve significant better results than the system built with commonly used layout analysis techniques.

[1]  Caroline Petitjean,et al.  A scalable pattern spotting system for historical documents , 2016, Pattern Recognit..

[2]  Feng Liu,et al.  A Novel Improved Binarized Normed Gradients Based Objectness Measure Through the Multi-feature Learning , 2015, ICIG.

[3]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Derek Hoiem,et al.  Category-Independent Object Proposals with Diverse Ranking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Eamonn J. Keogh,et al.  Mother Fugger: Mining Historical Manuscripts with Local Color Patches , 2010, 2010 IEEE International Conference on Data Mining.

[6]  Bin Yang,et al.  CRAFT Objects from Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  C. Clausner,et al.  Historical Document Layout Analysis Competition , 2011, 2011 International Conference on Document Analysis and Recognition.

[8]  Josep Lladós,et al.  Efficient segmentation-free keyword spotting in historical document collections , 2015, Pattern Recognit..

[9]  Matthew B. Blaschko,et al.  Learning a category independent object detection cascade , 2011, 2011 International Conference on Computer Vision.

[10]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[11]  Caroline Petitjean,et al.  Segmentation-free pattern spotting in historical document images , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Josep Lladós,et al.  Word and Symbol Spotting Using Spatial Organization of Local Descriptors , 2008, 2008 The Eighth IAPR International Workshop on Document Analysis Systems.

[14]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[15]  Cristian Sminchisescu,et al.  CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Laurent Heutte,et al.  Spot It! Finding Words and Patterns in Historical Documents , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[17]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[18]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[19]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Eamonn J. Keogh,et al.  Establishing the provenance of historical manuscripts with a novel distance measure , 2013, Pattern Analysis and Applications.

[22]  Philip H. S. Torr,et al.  BING: Binarized normed gradients for objectness estimation at 300fps , 2019, Computational Visual Media.

[23]  Jitendra Malik,et al.  DeepBox: Learning Objectness with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[25]  Santiago Manen,et al.  Prime Object Proposals with Randomized Prim's Algorithm , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Bernt Schiele,et al.  How good are detection proposals, really? , 2014, BMVC.