Detect-Focus-Track-Servo (DFTS): A vision-based workflow algorithm for robotic image-guided micromanipulation

Robotic image-guided micromanipulation contributes towards the ease of operation, speed, accuracy, and repeatability in cell manipulation. However, such technology is not fully exploited because of the challenges in the integration of robotic modules with existing microscope systems, and the difficulty in incorporating robot assistance seamlessly into the workflow. In this paper, we propose a vision-based workflow algorithm termed Detect-Focus-Track-Servo (DFTS). It facilitates easy integration of robotic modules. It also supports user interactions while minimizing the need for manual intervention and disruption to workflow through automatic detection, focusing, tracking and servoing. Experimental results suggest satisfactory detection accuracy of 99.0 % at 70 μm tolerance. The robustness test suggests no difference in the accuracy under blurred and cluttered images. The self-focus algorithm is also demonstrated to bring the tip into focus consistently. The track-servo algorithm achieves low sub-pixel uncertainty. By proposing the DFTS workflow algorithm, we hope that the level of autonomy and ease of deployment in robot and vision modules for micromanipulation can be improved so as to open up new possibilities in the development of robotic image-guided cell manipulation.