Direct Bundle Adjustment for 3D Image Fusion with Application to Transesophageal Echocardiography*

In this paper, we propose a novel algorithm for fusing a sequence of 3D images, named as Direct Bundle Adjustment (DBA). This algorithm simultaneously optimizes the global pose parameters of image frames and the intensity values of the fused global image using the 3D image data directly (without extracting features from the images). This one-step 3D image fusion approach is achieved by formulating the problem as an optimization problem to minimize the intensity differences between the global image and the corresponding points in the different local images. The proposed DBA method is particularly useful in the scenarios where distinct features are not available, such as Transesophageal Echocardiography (TEE) images. We validate the proposed method via simulated and in-vivo 3D TEE images. It is shown that the proposed method is robust to intensity noises and much more accurate than the conventional sequential fusion method.