Toward Automated Tissue Classification for Markerless Orthopaedic Robotic Assistance

A markerless computer aided orthopaedic platform will require a complex computer vision system to isolate and track rigid bodies used to localize a robot to a patient. Isolating rigid bodies such as bone requires accurate segmentation and this study explores using diffuse laser reflectivity to accurately classify tissue. Lasers (red at 650nm and infrared – IR – at 850nm) intersected four material types; cartilage, ligament, muscle and metal surgical tools within a controlled cadaveric setup. Images were captured with an infrared CMOS sensor, pre-processed to isolate laser centers, and resized to test information requirements. Images for both laser types were scaled from <inline-formula> <tex-math notation="LaTeX">$5\times5$ </tex-math></inline-formula> pixels to <inline-formula> <tex-math notation="LaTeX">$30\times30$ </tex-math></inline-formula> pixels and trained on a convolutional neural network, GoogLeNet. At sizes above <inline-formula> <tex-math notation="LaTeX">$15\times15$ </tex-math></inline-formula> pixels, the IR laser had a higher classification accuracy, reaching 97.8% at <inline-formula> <tex-math notation="LaTeX">$30\times30$ </tex-math></inline-formula> pixels, whereas the red laser peaked at 94.1%. It was shown as not possible to qualitatively identify materials that were not trained in the network based on their probability outputs. Further work will be performed to classify multiple points in a single scene as a step toward segmenting entire surgical views for markerless Computer Assisted Orthopedic Surgery (CAOS) systems.