Learning 2D Surgical Camera Motion From Demonstrations

Automating camera movement during robot-assisted surgery has the potential to reduce burden on surgeons and remove the need to manually move the camera. An important sub-problem is automatic viewpoint selection, proposing camera poses that focus on important anatomical features. We use the 6 DoF Stewart Platform Research Kit (SPRK) to move the environment with a fixed endoscope, as a dual to moving the endoscope itself, to study camera motion in surgical robotics. To provide demonstrations, we link the platform's control directly to the da Vinci Research Kit (dVRK) master control system and allow control of the platform using the same pedals and tools as a clinical movable endoscope. We propose a probabilistic model that identifies image features that “dwell” close to the camera's focal point in expert demonstrations. Our experiments consider a surgical debridement scenario on silicone phantoms with inclusions of varying color and shape. We evaluate the extent to which the system correctly segments candidate debridement targets (box accuracy) and correctly ranks those targets (rank accuracy). For debridement of a single uniquely colored inclusion, the box accuracy is 80% and the rank accuracy is 100% after 100 training data points. For debridement of multiple inclusions of the same color, the box accuracy is 70.8% and the rank accuracy is 100% after 100 training data points. For debridement of inclusions of a particular shape, the box accuracy is 70.5% and the rank accuracy is 90% after 100 training data points. A demonstration video is available at: https://vimeo.com/260362958