Intraoperative integration of structured light scanning for automatic tissue classification: a feasibility study

Purpose Structured light scanning is a promising inexpensive and accurate intraoperative imaging modality. Integration of these scanners in surgical workflows has the potential to enable rapid registration and augment preoperative imaging, in a practical and timely manner in the operating theatre. Previously, we have demonstrated the intraoperative feasibility of such scanners to capture anatomical surface information with high accuracy. The purpose of this study was to investigate the feasibility of automatically characterizing anatomical tissues from textural and spatial information captured by such scanners using machine learning. Assisted or automatic identification of relevant components of a captured scan is essential for effective integration of the technology in surgical workflow. Methods During a clinical study, 3D surface scans for seven total knee arthroplasty patients were collected, and textural and spatial features for cartilage, bone, and ligament tissue were collected and annotated. These features were used to train and evaluate machine learning models. As part of our preliminary preparation, three fresh-frozen knee cadaver specimens were also used where 3D surface scans with texture information were collected during different dissection stages. The resulting models were manually segmented to isolate texture information for muscles, tendon, cartilage, and bone. This information, and detailed labels from dissections, provided an in-depth, finely annotated dataset for building machine learning classifiers. Results For characterizing bone, cartilage, and ligament in the intraoperative surface models, random forest and neural network-based models achieved an accuracy of close to 80%, whereas an accuracy of close to 90% was obtained when only characterizing bone and cartilage. Average accuracy of 76–82% was reached for cadaver data in two-, three-, and four-class tissue separation. Conclusions The results of this project demonstrate the feasibility of machine learning methods to accurately classify multiple types of anatomical tissue. The ability to automatically characterize tissues in intraoperatively collected surface models would streamline the surgical workflow of using structured light scanners—paving the way to applications such as 3D documentation of surgery in addition to rapid registration and augmentation of preoperative imaging.