Mapping underwater ship hulls using a model-assisted bundle adjustment framework

Abstract This paper reports on a model-assisted bundle adjustment (BA) framework in which visually-derived features are fused with an underlying three-dimensional (3D) mesh provided a priori. By using an approach inspired by the expectation–maximization (EM) class of algorithms, we introduce a hidden binary label for each visual feature that indicates if that feature is considered part of the nominal model, or if the feature corresponds to 3D structure that is absent from the model. Therefore, in addition to improved estimates of the feature locations, we can identify visual features that correspond to foreign structure on the ship hull. We show that this framework is a special case of the Gaussian max-mixtures framework, which can be efficiently incorporated into state-of-the-art graph-based simultaneous localization and mapping (SLAM) solvers. In addition, the precision of our bundle adjustment framework allows the identification of structural deviations between 3D structure inferred from bundle-adjusted camera imagery and the prior model. These structural deviations are clustered into shapes, which allow us to fuse camera-derived structure back into the 3D mesh. This augmented model can be used within a 3D photomosaicing pipeline, providing a visually intuitive 3D reconstruction of the ship hull. We evaluate our pipeline using the Bluefin Robotics hovering autonomous underwater vehicle (HAUV) surveying the SS Curtiss, where a 3D mesh derived from computer aided design (CAD) drawings serves as the prior model. In addition to more consistent visual reconstructions, we can update the prior mesh with 3D information corresponding to underwater structure, such as biofouling or manually-placed cylindrical shapes with known dimensions.