A quantitative assessment of structural errors in grid maps

Various common error sources affect the quality of a map, e.g., salt and pepper noise and other forms of noise that are more or less uniformly distributed over the map. But there also exist errors that only occur very rarely in the mapping process but that have severe effects on the final result. They influence not only the local accuracy but also the whole spatial layout of the map. Examples of related error sources include bump noise in the robot’s pose or residual errors in Simultaneous Localization and Mapping (SLAM). The concept of brokenness is introduced in this article to capture the notion of structural errors in grid maps. The map is partitioned into regions that are locally consistent with ground truth but “off” relative to each other. Brokenness measures the number of these regions and their spatial relations. A theoretical basis is introduced to derive the concept of brokenness in a formal way. Furthermore, it is shown how brokenness can be computed in an algorithmic way. Experiments with maps from simulated as well as real world data are presented. They show that the metric can indeed be used to automatically determine the structural quality of a map in a quantitative way.