Hierarchical line segment matching for wide-baseline images via exploiting viewpoint robust local structure and geometric constraints

Abstract Line segment matching for wide-baseline images is challenging due to the significant viewpoint differences. In this study, we propose a hierarchical line segment matching method based on viewpoint robust local structure and geometric constraints. In our approach, line segments are paired and classified into three types representing heuristically those with different level of matchability: structured line pairs (S-LPs), unstructured line pairs (U-LPs), and individual line segments (I-LSs). Accordingly, we design a hierarchical matching framework that consists of three matching layers respectively corresponding to the above three types: in the first layer, robust local structures are constructed for S-LPs. We match the S-LPs by measuring local structure similarity (LSS). In the second layer, we build a topological descriptor-based constraint based on the S-LP matches and combine it with epipolar geometry-based constraints to select candidate matches for U-LPs, and then use LSS to further refine the matches. In the third layer, we estimate local homography for I-LSs to build constraints, to extract matches by exploiting the implicit region information of line pair matches. A pair of pre- and postprocessing algorithms, namely line segment merging and match reassignment, are performed before and after the matching procedure to overcome the negative effect of line segment fragmentation on the matching. Experimental results demonstrate that the proposed method performs better than state-of-the-art methods (with the largest improvement of 124.17% in terms of F-Measure over the best one among the compared methods).