Multiview Rectification of Folded Documents

Digitally unwrapping images of paper sheets is crucial for accurate document scanning and text recognition. This paper presents a method for automatically rectifying curved or folded paper sheets from a few images captured from multiple viewpoints. Prior methods either need expensive 3D scanners or model deformable surfaces using over-simplified parametric representations. In contrast, our method uses regular images and is based on general developable surface models that can represent a wide variety of paper deformations. Our main contribution is a new robust rectification method based on ridge-aware 3D reconstruction of a paper sheet and unwrapping the reconstructed surface using properties of developable surfaces via <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math> <alternatives><inline-graphic xlink:href="you-ieq1-2675980.gif"/></alternatives></inline-formula> conformal mapping. We present results on several examples including book pages, folded letters and shopping receipts.

[1]  Bruno Lévy,et al.  Least squares conformal maps for automatic texture atlas generation , 2002, ACM Trans. Graph..

[2]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[3]  Changsong Liu,et al.  A cylindrical surface model to rectify the bound document image , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Yu Zhang,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 an Improved Physically-based Method for Geometric Restoration of Distorted Document Images , 2007 .

[5]  Maurizio Pilu.  Undoing paper curl distortion using applicable surfaces , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Michael S. Brown,et al.  Geometric and shading correction for images of printed materials: a unified approach using boundary , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Youjie Zhou,et al.  Distance Transform Based Active Contour Approach for Document Image Rectification , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[9]  Atsushi Yamashita,et al.  Shape reconstruction and image restoration for non-flat surfaces of documents with a stereo vision system , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[10]  W. Brent Seales,et al.  Image restoration of arbitrarily warped documents , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shijian Lu,et al.  A partition approach for the restoration of camera images of planar and curled document , 2006, Image Vis. Comput..

[12]  Wenxin Li,et al.  A Model-based Book Dewarping Method Using Text Line Detection , 2007 .

[13]  Esther Portnoy.  Developable surfaces in hyperbolic space , 1975 .

[14]  Christoph H. Lampert,et al.  Document image dewarping using robust estimation of curled text lines , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[15]  Chew Lim Tan,et al.  Estimation of 3D shape of warped document surface for image restoration , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[16]  Ioannis Pratikakis,et al.  Goal-Oriented Rectification of Camera-Based Document Images , 2011, IEEE Transactions on Image Processing.

[17]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[18]  Michael S. Brown,et al.  A unified framework for document restoration using inpainting and shape-from-shading , 2009, Pattern Recognit..

[19]  Mathieu Perriollat,et al.  A computational model of bounded developable surfaces with application to image‐based three‐dimensional reconstruction , 2013, Comput. Animat. Virtual Worlds.

[20]  Nam Ik Cho,et al.  Document dewarping via text-line based optimization , 2015, Pattern Recognit..

[21]  Daniel Cohen-Or,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010, TOGS.

[22]  Chew Lim Tan,et al.  Restoring Warped Document Images through 3D Shape Modeling , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Takashi Matsuyama,et al.  Shape from Shading with Interreflections Under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book , 1997, International Journal of Computer Vision.

[24]  Markus H. Gross,et al.  Feature Preserving Point Set Surfaces based on Non‐Linear Kernel Regression , 2009, Comput. Graph. Forum.

[25]  Gaofeng Meng,et al.  Metric Rectification of Curved Document Images , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Nam Ik Cho,et al.  Composition of a Dewarped and Enhanced Document Image From Two View Images , 2009, IEEE Transactions on Image Processing.

[27]  Yi Ma,et al.  Unwrapping low-rank textures on generalized cylindrical surfaces , 2011, 2011 International Conference on Computer Vision.

[28]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[29]  Gaofeng Meng,et al.  Active Flattening of Curved Document Images via Two Structured Beams , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[31]  W. B. Seales,et al.  Restoring 2D Content from Distorted Documents , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Seiichi Uchida,et al.  Dewarping of document image by global optimization , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[33]  Michael S. Brown,et al.  Multi-View Document Rectification using Boundary , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Yuandong Tian,et al.  Rectification and 3D reconstruction of curved document images , 2011, CVPR 2011.

[35]  David S. Doermann,et al.  Geometric Rectification of Camera-Captured Document Images , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[37]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[38]  Yu Zhang,et al.  Restoring camera-captured distorted document images , 2014, International Journal on Document Analysis and Recognition (IJDAR).

[39]  Pierre Gurdjos,et al.  Shape from shading for the digitization of curved documents , 2007, Machine Vision and Applications.