2D-image to 3D-range registration in urban environments via scene categorization and combination of similarity measurements

Statistical similarity measurements, such as mutual information (MI) and normalized mutual information (NMI), show potential in the registration of 2D-image to 3D-range scans collected in urban environments. However 2D-3D registration with these measurements are of limited usage in urban sensing applications because: 1) it relies on the diversity and dependency between pre-defined pair of 2D-3D attributes, such as the intensity from images and reflectivity from range scans, in the urban sensing environment, and 2) it requires high-end range sensors with strong abilities to capture reflectivity in urban scenarios. In this paper, we propose a robust way of estimating statistical similarity measurements for 2D-3D data that are collected in various urban scenes with both low-cost and high-end range sensors. Rather than estimate the similarity of 2D-3D data on specific pair of 2D-3D attributes, we compute similarity measurements between a set of 2D-3D attribute-pairs that could be dominant in the category of sensed urban scene and combine them into a reliable similarity measurement. By applying the combined similarity measurement to the common framework of statistical 2D-3D registration, we get superior results when compared with state-of-art similarity measurements (MI and NMI) in terms of registration accuracy and robustness to initial condition, as indicated by experiments conducted on two datasets that are collected in various urban scenes, with low-cost and high-end sensors.