Video-based document image scanning using a mobile device

Document scanning is a daily office work. However, traditional devices like flatbed scanners are not easy to carry for mobile work. In this paper, we present a new document image scanning method using a mobile device. With a clip of continuous video capturing, our approach first extracts several key-frames. A mobile GPU based image stitching method is adopted to generate a high resolution document image. Semi-automatic document image dewarping is then applied to rectify the perspective distortion and document page warping. For shadow and uneven environment lighting problem, we use Retinex theory based image enhancement method to remove those artifacts. Experimental results show that our results are outperforming existing commercial mobile applications.

[1]  Dieter Schmalstieg,et al.  Real-time panoramic mapping and tracking on mobile phones , 2010, 2010 IEEE Virtual Reality Conference (VR).

[2]  Raja Bala,et al.  Mobile Video Capture of Multi-page Documents , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Hujun Bao,et al.  Locally Developable Constraint for Document Surface Reconstruction , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[4]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[5]  Changsong Liu,et al.  Rectifying the bound document image captured by the camera: a model based approach , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[6]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[7]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[8]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[9]  Dieter Schmalstieg,et al.  Panoramic mapping on a mobile phone GPU , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[10]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[11]  Joseph R. Cavallaro,et al.  A fast and efficient sift detector using the mobile GPU , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Muhammad Muzzamil Luqman,et al.  Mobile Phone Camera-Based Video Scanning of Paper Documents , 2013, CBDAR.

[13]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[14]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[15]  Jean-Michel Morel,et al.  Multiscale Retinex , 2014, Image Process. Line.

[16]  Richard Szeliski,et al.  Efficiently registering video into panoramic mosaics , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  E. Land.  Recent advances in retinex theory , 1986, Vision Research.