Creating digital libraries: content generation and re-mastering

This article has two main goals: to describe the automatic creation of a digital library and to provide an overview of the metaalgorithmic patterns that can be applied to increase the accuracy of its creation. Automating the creation of useful digital libraries - that is, digital libraries affording searchable text and reusable ("repurposable") output - is a complicated process, whether the original library is paper-based or already available in electronic form. We outline the steps involved in the creation of a deployable digital library (>1.2 /spl times/ 10/sup 6/ pages) for MIT Press, as well as its implications to other aspects of digital library creation, management, use and repurposing. Input, transformation, information extraction, and output processes are considered in light of their utility in creating layers of content. Interestingly, in some aspects, scanning directly from paper offers extra opportunities for error-checking through feedback-feedforward combination. Strategies for quality assurance (QA) at the document, chapter and book level are also discussed. We emphasize the use of metaalgorithmic design patterns for application towards improving the content generation, extraction and remastering. This approach also increases the ease with which modules and algorithms are added to and deprecated from the system.

[1]  Friedrich M. Wahl,et al.  Block segmentation and text extraction in mixed text/image documents , 1982, Comput. Graph. Image Process..

[2]  Xiaofan Lin.  Text-mining based journal splitting , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[3]  Xiaofan Lin,et al.  Using OCR in Run Once, Use Everywhere manner to facilitate re-use in a document processing workflow , 2002 .

[4]  Steven J. Simske,et al.  A ground-truthing engine for proofsetting, publishing, re-purposing and quality assurance , 2003, DocEng '03.

[5]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Atsuhiro Takasu,et al.  An automated generation of an electronic library based on document image understanding , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[7]  Sherif Yacoub.  Quality Assurance for Document Understanding Systems , 2002 .

[8]  Steven J. Simske,et al.  Automatic document navigation for digital content remastering , 2003, IS&T/SPIE Electronic Imaging.

[9]  George R. Thoma,et al.  Automated data entry system: performance issues , 2001, IS&T/SPIE Electronic Imaging.

[10]  Mahesh Viswanathan,et al.  A prototype document image analysis system for technical journals , 1992, Computer.

[11]  Henry S. Baird,et al.  Digital libraries and document image analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[12]  Steven J. Simske,et al.  Performance analysis of pattern classifier combination by plurality voting , 2003, Pattern Recognit. Lett..

[13]  Xiaofan Lin,et al.  Reliable OCR solution for digital content re-mastering , 2001, IS&T/SPIE Electronic Imaging.