Dense matching for multi-scale images by propagation

This article puts forward a new algorithm of dense matching between two images of the same scene, not necessarily stereoscopic and which can be of different scales. This algorithm is based on points of interest identified by a multi-scale detector and later matched according to a very high threshold to keep only the most reliable points. Afterwards, these points serve as germs for the next iteration. This propagation process is guided according to geometric constraints, and repeated until all the possible correspondents between the two images are obtained. The results of the experiments obtained on test images and those of the real world are very satisfactory even in difficult cases of great displacements and changes in appearance between the two captures.

[1]  Liu Liu,et al.  Progressive Matching Based On Segmentation for 3D reconstruction , 2005, The Fifth International Conference on Computer and Information Technology (CIT'05).

[2]  Stefano Mattoccia.  Stereo Vision Algorithms for FPGAs , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Urbano Nunes,et al.  Using the GPU for fast symmetry-based dense stereo matching in high resolution images , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Abderrahim Saaidi,et al.  Incremental Multi-view 3D Reconstruction Starting from Two Images Taken by a Stereo Pair of Cameras , 2015 .

[5]  Hongxun Yao,et al.  View-based 3D object retrieval via multi-modal graph learning , 2015, Signal Process..

[6]  Dmitry Chetverikov,et al.  Affine propagation for surface reconstruction in wide baseline stereo , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[7]  Shai Avidan,et al.  Coherency Sensitive Hashing , 2011, ICCV.

[8]  Jesús M. Pérez,et al.  Real-time stereo matching using memory-efficient Belief Propagation for high-definition 3D telepresence systems , 2011, Pattern Recognit. Lett..

[9]  M. Lhuillier,et al.  Reconstruction quasi-dense et modèles 3D à partir d'une séquence d'images , 2004 .

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Y. Ruichek,et al.  A Near Real-Time Color Stereo Matching Method for GPU , 2013 .

[13]  Lihi Zelnik-Manor,et al.  On SIFTs and their scales , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Chong Wang,et al.  A new high resolution depth map estimation system using stereo vision and depth sensing device , 2013, 2013 IEEE 9th International Colloquium on Signal Processing and its Applications.

[15]  Mikael Ekström,et al.  High-speed segmentation-driven high-resolution matching , 2015, Other Conferences.

[16]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[17]  Stefan Lüke,et al.  Real-Time Stereo Vision: Making More Out of Dynamic Programming , 2009, CAIP.

[18]  Yue Gao,et al.  Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing , 2016, IEEE Transactions on Image Processing.

[19]  Long Quan,et al.  Region-based progressive stereo matching , 2004, CVPR 2004.

[20]  Jean-François Nezan,et al.  Real-time GPU-based local stereo matching method , 2013, 2013 Conference on Design and Architectures for Signal and Image Processing.

[21]  Zhengyou Zhang,et al.  A Progressive Scheme for Stereo Matching , 2000, SMILE.

[22]  Yue Gao,et al.  Zero-Shot Learning With Transferred Samples , 2017, IEEE Transactions on Image Processing.

[23]  Guillaume Gales,et al.  Reliability measure for propagation-based stereo matching , 2012, 2012 13th International Workshop on Image Analysis for Multimedia Interactive Services.

[24]  Vamshhi Pavan Kumar Varma Vegeshna,et al.  Stereo Matching with Color-Weighted Correlation, Hierachical Belief Propagation and Occlusion Handling , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  Lu Fang,et al.  Stereo matching by adaptive weighting selection based cost aggregation , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[26]  Greg Brown,et al.  A performance and energy comparison of convolution on GPUs, FPGAs, and multicore processors , 2013, TACO.

[27]  Yutaka Ishibashi,et al.  Efficient edge-awareness propagation via single-map filtering for edge-preserving stereo matching , 2015, Electronic Imaging.

[28]  José Manuel Ferrández,et al.  Evaluation of stereo correspondence algorithms and their implementation on FPGA , 2014, J. Syst. Archit..

[29]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[30]  Yasuyuki Matsushita,et al.  Smoothly varying affine stitching , 2011, CVPR 2011.

[31]  N. Lu,et al.  Matching Cost Filtering for Dense Stereo Correspondence , 2013 .

[32]  Truong Q. Nguyen,et al.  Local Disparity Estimation With Three-Moded Cross Census and Advanced Support Weight , 2013, IEEE Transactions on Multimedia.

[33]  Jie Shen,et al.  Accelerating Cost Aggregation for Real-Time Stereo Matching , 2012, 2012 IEEE 18th International Conference on Parallel and Distributed Systems.

[34]  W. Paul Cockshott,et al.  Acceleration of Stereo-Matching on Multi-core CPU and GPU , 2014, 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS).

[35]  Dongxiao Li,et al.  Fast stereo matching using adaptive guided filtering , 2014, Image Vis. Comput..

[36]  Long Quan,et al.  Match Propagation for Image-Based Modeling and Rendering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Hongxun Yao,et al.  Strategy for dynamic 3D depth data matching towards robust action retrieval , 2015, Neurocomputing.

[38]  Tsutomu Maruyama,et al.  Fast and Accurate Stereo Vision System on FPGA , 2014, TRETS.

[39]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[40]  James K. Archibald,et al.  Hardware-Efficient Design of Real-Time Profile Shape Matching Stereo Vision Algorithm on FPGA , 2014, Int. J. Reconfigurable Comput..

[41]  Iasonas Kokkinos,et al.  Scale invariance without scale selection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Liming Chen,et al.  A Fast Trilateral Filter-Based Adaptive Support Weight Method for Stereo Matching , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[44]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[45]  Antonios Gasteratos,et al.  A stereo matching approach based on particle filters and scattered control landmarks , 2015, Image Vis. Comput..

[46]  Ce Liu,et al.  Deformable Spatial Pyramid Matching for Fast Dense Correspondences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Qian Chen,et al.  A volumetric stereo matching method: application to image-based modeling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[48]  Jae Wook Jeon,et al.  Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Qunsheng Peng,et al.  Depth map enhancement based on color and depth consistency , 2013, The Visual Computer.

[50]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[51]  Hong Liu,et al.  View-based 3D object retrieval with discriminative views , 2017, Neurocomputing.

[52]  Ji Ma,et al.  A Modified Census Transform Based on the Neighborhood Information for Stereo Matching Algorithm , 2013, 2013 Seventh International Conference on Image and Graphics.

[53]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[54]  Jean-Christophe Pesquet,et al.  A Convex Optimization Approach for Depth Estimation Under Illumination Variation , 2009, IEEE Transactions on Image Processing.

[55]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[56]  Eric Psota,et al.  Real-Time Stereo Matching on CUDA Using an Iterative Refinement Method for Adaptive Support-Weight Correspondences , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[57]  Panos Nasiopoulos,et al.  The Effect of Frame Rate on 3D Video Quality and Bitrate , 2015, 1803.04826.

[58]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[59]  Yu Wang,et al.  Real-time high-quality stereo vision system in FPGA , 2013, 2013 International Conference on Field-Programmable Technology (FPT).

[60]  Iasonas Kokkinos,et al.  Dense Segmentation-Aware Descriptors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Cheng-Liang Lin,et al.  Real-time stereo to multi-view conversion system based on adaptive meshing , 2015, Journal of Real-Time Image Processing.

[62]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Neus Sabater,et al.  How Accurate Can Block Matches Be in Stereo Vision? , 2011, SIAM J. Imaging Sci..

[64]  Maziar Loghman,et al.  Real-Time Refinement of Kinect Depth Maps using Multi-Resolution Anisotropic Diffusion , 2014, Mob. Networks Appl..

[65]  John Morris,et al.  Comparison of FPGA and GPU implementations of real-time stereo vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[66]  Alexandrina Rogozan,et al.  A robust cost function for stereo matching of road scenes , 2014, Pattern Recognit. Lett..

[67]  A. Saaidi,et al.  Images Matching Using Voronoï Regions Propagation , 2015 .

[68]  Khalid Satori,et al.  Robust point matching via corresponding circles , 2017, Multimedia Tools and Applications.

[69]  Enhua Wu,et al.  Constant Time Weighted Median Filtering for Stereo Matching and Beyond , 2013, 2013 IEEE International Conference on Computer Vision.

[70]  Jiangbo Lu,et al.  DAISY Filter Flow: A Generalized Discrete Approach to Dense Correspondences , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.