An Advanced Pre-Processing Pipeline to Improve Automated Photogrammetric Reconstructions of Architectural Scenes

Automated image-based 3D reconstruction methods are more and more flooding our 3D modeling applications. Fully automated solutions give the impression that from a sample of randomly acquired images we can derive quite impressive visual 3D models. Although the level of automation is reaching very high standards, image quality is a fundamental pre-requisite to produce successful and photo-realistic 3D products, in particular when dealing with large datasets of images. This article presents an efficient pipeline based on color enhancement, image denoising, color-to-gray conversion and image content enrichment. The pipeline stems from an analysis of various state-of-the-art algorithms and aims to adjust the most promising methods, giving solutions to typical failure causes. The assessment evaluation proves how an effective image pre-processing, which considers the entire image dataset, can improve the automated orientation procedure and dense 3D point cloud reconstruction, even in the case of poor texture scenarios.

[1]  Emmanuel P. Baltsavias,et al.  Multiphoto geometrically constrained matching , 1991 .

[2]  Leonid P. Yaroslavsky,et al.  Local adaptive image restoration and enhancement with the use of DFT and DCT in a running window , 1996, Optics & Photonics.

[3]  Richard G. Baraniuk,et al.  Wavelet domain filtering for photon imaging systems , 1997, Optics & Photonics.

[4]  E. Kolaczyk WAVELET SHRINKAGE ESTIMATION OF CERTAIN POISSON INTENSITY SIGNALS USING CORRECTED THRESHOLDS , 1999 .

[5]  M. Ronnier Luo,et al.  Testing Color-Difference Formulae on Complex Images Using a CRT Monitor , 2000, Color Imaging Conference.

[6]  Peter A. Rhodes,et al.  A study of digital camera colorimetric characterisation based on polynomial modelling , 2001 .

[7]  M O Leach,et al.  Pre-processed image reconstruction applied to breast and brain MR imaging , 2001, Physiological measurement.

[8]  E. Baltsavias,et al.  Cloud mapping from the ground: use of photogrammetric methods , 2002 .

[9]  Rakhi C. Motwani,et al.  Survey of Image Denoising Techniques , 2004 .

[10]  Wencheng Wu,et al.  The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations , 2005 .

[11]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[12]  T. Ohdake,et al.  3D MODELING OF HIGH RELIEF SCULPTURE USING IMAGE BASED INTEGRATED MEASUREMENT SYSTEM , 2005 .

[13]  D. Pascale RGB coordinates of the Macbeth ColorChecker , 2006 .

[14]  Charles Kervrann,et al.  Optimal Spatial Adaptation for Patch-Based Image Denoising , 2006, IEEE Transactions on Image Processing.

[15]  Suyash P. Awate,et al.  Unsupervised, information-theoretic, adaptive image filtering for image restoration , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Neil A. Dodgson,et al.  Decolorize: Fast, contrast enhancing, color to grayscale conversion , 2007, Pattern Recognit..

[17]  Ioannis Stamos,et al.  Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-Scale Scenes , 2008, International Journal of Computer Vision.

[18]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[19]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[20]  Raimondo Schettini,et al.  Content Aware Image Enhancement , 2007, AI*IA.

[21]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[22]  Nikolay N. Ponomarenko,et al.  An automatic approach to lossy compression of AVIRIS images , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[23]  Ian D. Reid,et al.  Modeling and generating complex motion blur for real-time tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Li Zhang,et al.  Turning images into 3-D models , 2008, IEEE Signal Processing Magazine.

[25]  Garrett M. Johnson,et al.  Color Imaging: Fundamentals and Applications , 2008 .

[26]  Karol Myszkowski,et al.  Apparent Greyscale: A Simple and Fast Conversion to Perceptually Accurate Images and Video , 2008, Comput. Graph. Forum.

[27]  A Comparative Study of Color and Contrast Enhancement for Still Images and Consumer Video Applications , 2008, Color Imaging Conference.

[28]  David W. Murray,et al.  Improving the Agility of Keyframe-Based SLAM , 2008, ECCV.

[29]  Armin Gruen,et al.  Turning Images into 3-D Models ( Developments and performance analysis of image matching for detailed surface reconstruction of heritage objects ) , 2008 .

[30]  Seungyong Lee,et al.  Robust color-to-gray via nonlinear global mapping , 2009, SIGGRAPH 2009.

[31]  Seungyong Lee,et al.  Robust color-to-gray via nonlinear global mapping , 2009, ACM Trans. Graph..

[32]  Himanshu Aggarwal,et al.  A Comprehensive Review of Image Enhancement Techniques , 2010, ArXiv.

[33]  Fabio Remondino,et al.  Orientation and 3D modelling from markerless terrestrial images: combining accuracy with automation , 2010 .

[34]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[35]  C. Fraser,et al.  Interest operators for feature‐based matching in close range photogrammetry , 2010 .

[36]  Paolo Cignoni,et al.  Machine Vision and Applications Manuscript No , 2022 .

[37]  Alessandro Foi,et al.  Noise estimation and removal in MR imaging: The variance-stabilization approach , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[38]  Kyoung Mu Lee,et al.  Simultaneous localization, mapping and deblurring , 2011, 2011 International Conference on Computer Vision.

[39]  Alessandro Foi,et al.  Optimal Inversion of the Anscombe Transformation in Low-Count Poisson Image Denoising , 2011, IEEE Transactions on Image Processing.

[40]  Livio De Luca,et al.  Automated Image-Based Procedures for Accurate Artifacts 3D Modeling and Orthoimage Generation , 2011 .

[41]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[42]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[43]  Thomas P. Kersten,et al.  Low-Cost and Open-Source Solutions for Automated Image Orientation - A Critical Overview , 2012, EuroMed.

[44]  Jean-Michel Morel,et al.  Secrets of image denoising cuisine* , 2012, Acta Numerica.

[45]  Djamel Merad,et al.  Underwater image preprocessing for automated photogrammetry in high turbidity water: An application on the Arles-Rhone XIII roman wreck in the Rhodano river, France , 2012, 2012 18th International Conference on Virtual Systems and Multimedia.

[46]  Jiaya Jia,et al.  Real-time contrast preserving decolorization , 2012, SA '12.

[47]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[48]  Xiaobin Xu,et al.  Decolorization: is rgb2gray() out? , 2013, SIGGRAPH ASIA Technical Briefs.

[49]  Jyotsna Patil,et al.  A Comparative Study of Image Denoising Techniques , 2013 .

[50]  Cewu Lu,et al.  Contrast Preserving Decolorization with Perception-Based Quality Metrics , 2014, International Journal of Computer Vision.

[51]  Gosuke Ohashi,et al.  Estimation of the Helmholtz–Kohlrausch effect for natural images , 2014 .

[52]  S. Robson,et al.  Modelling the appearance of heritage metallic surfaces , 2014 .

[53]  Gabriele Guidi,et al.  Image pre-processing for optimizing automated photogrammetry performances , 2014 .

[54]  Jean-Michel Morel,et al.  The noise clinic: A universal blind denoising algorithm , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[55]  Jean-Michel Morel,et al.  Nonparametric noise estimation method for raw images. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[56]  Fabio Remondino,et al.  State of the art in high density image matching , 2014 .

[57]  Fabrizio Ivan Apollonio,et al.  Evaluation of feature-based methods for automated network orientation , 2014 .

[58]  Zhengguo Li,et al.  Content Adaptive Image Detail Enhancement , 2015, IEEE Signal Processing Letters.

[59]  Jean-Michel Morel,et al.  Multiscale Image Blind Denoising , 2015, IEEE Transactions on Image Processing.

[60]  Norbert Pfeifer,et al.  Mind your grey tones : examining the influence of decolourization methods on interest point extraction and matching for architectural image-based modelling , 2015 .

[61]  Jean-Michel Morel,et al.  The Noise Clinic: a Blind Image Denoising Algorithm , 2015, Image Process. Line.

[62]  Jan-Michael Frahm,et al.  Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset) , 2015, CVPR 2015.

[63]  Fabio Bellavia,et al.  Fast Adaptive Frame Preprocessing for 3D Reconstruction , 2015, VISAPP.