Scale invariant representation of imbalanced points

Imbalance oriented candidate selection was introduced as an alternative of non-maximum suppression, aiming to improve the localization accuracy. To distinguish interest points detected via non-maximum suppression, we call interest points detected via imbalance oriented selection imbalanced points. Scale assignment for imbalanced points is not straightforward because of a dilemma of involving non-maximum suppression. The scale space theory, a popular scale assignment scheme, requests non-maximum suppression to detect extreme points from scale spaces, while imbalanced points are expected to be free of non-maximum suppression in order to maintain the localization accuracy. In this paper, we propose a bypass strategy that circumvents the above dilemma by establishing an association between an imbalanced point and a certain interest point with a known scale (e.g., Lowe's keypoints and Hessian-Laplace). Furthermore, we propose a hybrid representation of imbalanced points for a two-layer matching scheme, where the first-layer matching is based on discriminant SIFT-type descriptors of imbalanced points, and the second-layer matching is based on patch-type descriptors. Experiments show the effectiveness of the proposed scale assignment and hybrid representation.