From the Digitization of Cultural Artifacts to the Web Publishing of Digital 3D Collections: an Automatic Pipeline for Knowledge Sharing

In this paper, we introduce a novel approach intended to simplify the production of multimedia content from real objects for the purpose of knowledge sharing, which is particularly appropriate to the cultural heritage field. It consists in a pipeline that covers all steps from the digitization of the objects up to the Web publishing of the resulting digital copies. During a first stage, the digitization is performed by a high speed 3D scanner that recovers the object's geometry. A second stage then extracts from the recovered data a color texture as well as a texture of details, in order to enrich the acquired geometry in a more realistic way. Finally, a third stage converts these data so that they are compatible with the recent WebGL paradigm, then providing 3D multimedia content directly exploitable by end-users by means of standard Internet browsers. The pipeline design is centered on automation and speed, so that it can be used by non expert users to produce mul- timedia content from potentially large object's collections, like it may be the case in cultural heritage. The choice of a high speed scanner is particularly adapted for such a design, since this kind of devices has the advantage of being fast and intuitive. Processing stages that follow the digitization are both completely automatic and "seamless", in the sense that it is not incumbent upon the user to perform tasks manually, nor to use external softwares that generally need additional operations to solve compatibility issues.

[1]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[2]  Richard I. Hartley,et al.  Optimised KD-trees for fast image descriptor matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jan Silovský,et al.  Study on Cross-Lingual Adaptation of a Czech LVCSR System towards Slovak , 2010, COST 2102 Conference.

[4]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[5]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[6]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Stephen M. Omohundro,et al.  Efficient Algorithms with Neural Network Behavior , 1987, Complex Syst..

[8]  Karel Blavka,et al.  Automatic segmentation and annotation of audio archive documents , 2011, 2011 10th International Workshop on Electronics, Control, Measurement and Signals.

[9]  Linda G. Shapiro,et al.  Acquisition and visualization of colored 3D objects , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[10]  Jan Silovský,et al.  Czech-to-slovak adapted broadcast news transcription system , 2008, INTERSPEECH.

[11]  Roberto Scopigno,et al.  SpiderGL: a JavaScript 3D graphics library for next-generation WWW , 2010, Web3D '10.

[12]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[13]  Jan Nouza,et al.  System for automatic collection, annotation and indexing of Czech broadcast speech with full-text search , 2010, Melecon 2010 - 2010 15th IEEE Mediterranean Electrotechnical Conference.

[14]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Vincent Nozick,et al.  Pyramidal Normal Map Integration for Real-time Photometric Stereo , 2010 .

[16]  Andrew Gardner,et al.  Capturing and Rendering with Incident Light Fields , 2003, Rendering Techniques.

[17]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Julie Dorsey,et al.  Digital Modeling of Material Appearance , 2007 .

[19]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[20]  Roeland Ordelman,et al.  Exploration of audiovisual heritage using audio indexing technology , 2006 .

[21]  Tim Weyrich,et al.  A system for high-volume acquisition and matching of fresco fragments: reassembling Theran wall paintings , 2008, ACM Trans. Graph..

[22]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[23]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[24]  Gang Lin,et al.  An improved vertex caching scheme for 3D mesh rendering , 2006, IEEE Transactions on Visualization and Computer Graphics.

[25]  Pedro V. Sander,et al.  Fast triangle reordering for vertex locality and reduced overdraw , 2007, SIGGRAPH 2007.

[26]  Luc Van Gool,et al.  In-hand scanning with online loop closure , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[27]  Paolo Cignoni,et al.  Minimizing user intervention in registering 2D images to 3D models , 2005, The Visual Computer.

[28]  Petr Pollák,et al.  Accuracy Analysis of Generalized Pronunciation Variant Selection in ASR Systems , 2008, COST 2102 Conference.

[29]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[30]  Bhuvana Ramabhadran,et al.  Automatic recognition of spontaneous speech for access to multilingual oral history archives , 2004, IEEE Transactions on Speech and Audio Processing.

[31]  Jan Nouza,et al.  Very large vocabulary speech recognition system for automatic transcription of czech broadcast programs , 2004, INTERSPEECH.

[32]  Adam Baumberg,et al.  Blending Images for Texturing 3D Models , 2002, BMVC.

[33]  Gabriel Taubin,et al.  Appying Shape from Lighting Variation to Bump Map Capture , 1997, Rendering Techniques.

[34]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[35]  Jan Silovský,et al.  Voice Technology to Enable Sophisticated Access to Historical Audio Archive of the Czech Radio , 2011, MM4CH.

[36]  Thomas Malzbender,et al.  Surface enhancement using real-time photometric stereo and reflectance transformation , 2006, EGSR '06.

[37]  Robert B. Fisher,et al.  Fusing multiple color images for texturing models , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[38]  Hugues Hoppe,et al.  Optimization of mesh locality for transparent vertex caching , 1999, SIGGRAPH.

[39]  Szymon Rusinkiewicz,et al.  Estimating the Laplace‐Beltrami Operator by Restricting 3D Functions , 2009, Comput. Graph. Forum.

[40]  Yasuyuki Matsushita,et al.  A hand-held photometric stereo camera for 3-D modeling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[41]  Jirí Navrátil,et al.  Spoken language recognition-a step toward multilinguality in speech processing , 2001, IEEE Trans. Speech Audio Process..

[42]  Paolo Cignoni,et al.  Reconstructing Textured Meshes from Multiple Range RGB Maps , 2002, VMV.

[43]  Jean-Michel Dischler,et al.  Automatic Registration and Calibration for Efficient Surface Light Field Acquisition , 2006, VAST.

[44]  Frank Nielsen,et al.  On weighting clustering , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Roberto Scopigno,et al.  Automatic Texturing without Illumination Artifacts from In-Hand Scanning Data Flow , 2011, MM4CH.

[46]  Roeland Ordelman,et al.  Speech-based Annotation of Heterogeneous Multimedia Content Using Automatic Speech Recognition , 2007 .

[47]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[48]  David J. Kriegman,et al.  Passive photometric stereo from motion , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[49]  Daniel Cohen-Or,et al.  Seamless Montage for Texturing Models , 2010, Comput. Graph. Forum.

[50]  Paolo Cignoni,et al.  Improved color acquisition and mapping on 3D models via flash-based photography , 2010, JOCCH.

[51]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Jan Nouza,et al.  A System for Information Retrieval from Large Records of Czech Spoken Data , 2006, TSD.

[53]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[54]  Jean-Luc Gauvain,et al.  Speech Processing for Audio Indexing , 2008, GoTAL.

[55]  Luc Van Gool,et al.  Efficient Non-Maximum Suppression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[56]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[57]  Don Brutzman,et al.  X3D: Extensible 3D Graphics for Web Authors , 2007 .

[58]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[59]  Jan Silovský,et al.  PLDA-Based Clustering for Speaker Diarization of Broadcast Streams , 2011, INTERSPEECH.

[60]  Luc Van Gool,et al.  Content-Based Image Retrieval Based on Local Affinely Invariant Regions , 1999, VISUAL.

[61]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[62]  Yannis Avrithis,et al.  Detecting Regions from Single Scale Edges , 2010, ECCV Workshops.

[63]  Paolo Cignoni,et al.  Flow-Based Local Optimization for Image-to-Geometry Projection , 2012, IEEE Transactions on Visualization and Computer Graphics.

[64]  Roberto Scopigno,et al.  Image‐to‐Geometry Registration: a Mutual Information Method exploiting Illumination‐related Geometric Properties , 2009, Comput. Graph. Forum.

[65]  Jindrich Zdansky,et al.  Comparison of Generative and Discriminative Approaches for Speaker Recognition with Limited Data , 2009 .

[66]  Anita Sellent,et al.  Floating Textures , 2008, Comput. Graph. Forum.

[67]  Borivoj Vojnovic,et al.  An Algorithm for image stitching and blending , 2005, SPIE BiOS.

[68]  Ioannis Stamos,et al.  Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-Scale Scenes , 2008, International Journal of Computer Vision.

[69]  Atsushi Nakazawa,et al.  The Great Buddha Project: Digitally Archiving, Restoring, and Analyzing Cultural Heritage Objects , 2007, International Journal of Computer Vision.

[70]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[71]  Paolo Cignoni,et al.  Masked photo blending: Mapping dense photographic data set on high-resolution sampled 3D models , 2008, Comput. Graph..