A Hybrid Approach to Wide-Baseline Image Matching

Recent works such as DEEPDESC, DEEPCOMPARE have proposed the learning of robust local image descriptors using a Siamese convolutional neural network directly from images instead of handcrafting them like traditional descriptors such as SIFT and MROGH. Though these algorithms show the state-of-the-art results on the Multi-View Stereo (MVS) dataset, they fail to accomplish many challenging real world tasks such as stitching image panoramas, primarily due to the limited performance of finding correspondence. In this paper, we propose a novel hybrid algorithm with which we are able to harness the power of a learning based approach along with the discriminative advantages that traditional descriptors have to offer. We also propose the PhotoSynth dataset, with size of an order of magnitude more that the traditional MVS dataset in terms of the number of scenes, images, patches along with positive and negative correspondence. Our PhotoSynth dataset also has better coverage of the overall viewpoint, scale, and lighting challenges than the MVS dataset. We evaluate our approach on two data sets which provides images having high viewpoints difference and wide-baselines. One of them is Graffiti scene from the Oxford Affine Covariant Regions Dataset (ACRD) for matching images with 2D affine transformations. The other is the Fountain-P11 dataset for images with 3D projective transformations. We report, to the best of our knowledge, the best results till date on the ACRD Graffiti scene compared to descriptors such as SIFT, MROGH or any other learnt descriptors such as DEEPDESC.