Keyframe selection for robust pose estimation in laparoscopic videos

Motion estimation based on point correspondences in two views is a classic problem in computer vision. In the field of laparoscopic video sequences - even with state of the art algorithms - a stable motion estimation can not be guaranteed generally. Typically, a video from a laparoscopic surgery contains sequences where the surgeon barely moves the endoscope. Such restricted movement causes a small ratio between baseline and distance leading to unstable estimation results. Exploiting the fact that the entire sequence is known a priori, we propose an algorithm for keyframe selection in a sequence of images. The key idea can be expressed as follows: if all combination of frames in a sequence are scored, the optimal solution can be described as a weighted directed graph problem. We adapt the widely known Dijkstras Algorithm to find the best selection of frames.1 The framework for keyframe selection can be used universally to find the best combination of frames for any reliable scoring function. For instance, forward motion ensures the most accurate camera position estimation, whereas sideward motion is preferred in the sense of reconstruction. Based on the distribution and the disparity of point correspondences, we propose a scoring function which is capable of detecting poorly conditioned pairs of frames. We demonstrate the potential of the algorithm focusing on accurate camera positions. A robot system provides ground truth data. The environment in laparoscopic videos is reflected by an industrial endoscope and a phantom.