Rectification of figures and photos in document images using bounding box interface

This paper proposes an algorithm for the segmentation and rectification of figures and photos in document images. The algorithm requires just a rough user-provided bounding box for the objects in a single-view image. On receiving the user's bounding box, it takes about 1–2 seconds to segment and rectify mega-pixel sized figures. The main feature of the algorithm is a novel segmentation method that exploits the properties of printed figures. Specifically, a set of boundary candidates is generated using the properties, and the optimal boundary in the set is found by using an alternating optimization scheme. This segmentation result is further refined so that it is well localized to the true boundary. In addition to our segmentation method, we also propose a new boundary interpolation method for the rectification of segmented figures. The method improves the quality of output by largely removing perspective distortions compared to conventional boundary interpolation methods. Experimental results on a variety of images show that the method is efficient, robust, and easy to use.

[1]  Jiri Matas,et al.  Robust Detection of Lines Using the Progressive Probabilistic Hough Transform , 2000, Comput. Vis. Image Underst..

[2]  Atsushi Yamashita,et al.  Shape reconstruction and image restoration for non-flat surfaces of documents with a stereo vision system , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[3]  S. A. Coons.  SURFACES FOR COMPUTER-AIDED DESIGN OF SPACE FORMS , 1967 .

[4]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[5]  Maurizio Pilu,et al.  Extraction of illusory linear clues in perspectively skewed documents , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[7]  David S. Doermann,et al.  Flattening curved documents in images , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Kazuhiro Fukui,et al.  Edge Extraction Method Based on Separability of Image Features , 1995, MVA.

[9]  David S. Doermann,et al.  Camera-based analysis of text and documents: a survey , 2005, International Journal of Document Analysis and Recognition (IJDAR).

[10]  Michael S. Brown,et al.  Geometric and shading correction for images of printed materials using boundary , 2006, IEEE Transactions on Image Processing.

[11]  W. Brent Seales,et al.  Document restoration using 3D shape: a general deskewing algorithm for arbitrarily warped documents , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Majid Mirmehdi,et al.  Estimating the Orientation and Recovery of Text Planes in a Single Image , 2001, BMVC.

[13]  Nam Ik Cho,et al.  Composition of a Dewarped and Enhanced Document Image From Two View Images , 2009, IEEE Transactions on Image Processing.

[14]  Christoph H. Lampert,et al.  Document capture using stereo vision , 2004, DocEng '04.

[15]  Andrew Zisserman,et al.  Metric rectification for perspective images of planes , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[16]  Naokazu Yokoya,et al.  Video Mosaicing Based on Structure from Motion for Distortion-Free Document Digitization , 2007, ACCV.

[17]  Michael S. Brown,et al.  Multi-View Document Rectification using Boundary , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Andy M. Yip,et al.  A Restoration Framework for Correcting Photometric and Geometric Distortions in Camera-based Document Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Changsong Liu,et al.  A cylindrical surface model to rectify the bound document image , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Maurizio Pilu.  Undoing paper curl distortion using applicable surfaces , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.