Towards Detection and Localisation of Instruments in Minimally Invasive Surgery

Methods for detecting and localising surgical instruments in laparoscopic images are an important element of advanced robotic and computer assisted interventions. Robotic joint encoders and sensors integrated or mounted on the instrument can provide information about the tool’s position but this often has inaccuracy when transferred to the surgeon’s point of view. Vision sensors are currently a promising approach for determining the position of instruments in the coordinate frame of the surgical camera. In this study, we propose a vision algorithm for localising the instrument’s pose in 3D leaving only rotation in the axis of the tool’s shaft as an ambiguity. We propose a probabilistic supervised classification method to detect pixels in laparoscopic images that belong to surgical tools. We then use the classifier output to initialise an energy minimisation algorithm for estimating the pose of a prior 3D model of the instrument within a level set framework. We show that the proposed method is robust against noise using simulated data and we perform quantitative validation of the algorithm compared to ground truth obtained using an optical tracker. Finally, we demonstrate the practical application of the technique on in vivo data from MIS with traditional laparoscopic and robotic instruments.