Automated detection and growth tracking of 3D bio-printed organoid clusters using optical coherence tomography with deep convolutional neural networks

Organoids are advancing the development of accurate prediction of drug efficacy and toxicity in vitro. These advancements are attributed to the ability of organoids to recapitulate key structural and functional features of organs and parent tumor. Specifically, organoids are self-organized assembly with a multi-scale structure of 30–800 μm, which exacerbates the difficulty of non-destructive three-dimensional (3D) imaging, tracking and classification analysis for organoid clusters by traditional microscopy techniques. Here, we devise a 3D imaging, segmentation and analysis method based on Optical coherence tomography (OCT) technology and deep convolutional neural networks (CNNs) for printed organoid clusters (Organoid Printing and optical coherence tomography-based analysis, OPO). The results demonstrate that the organoid scale influences the segmentation effect of the neural network. The multi-scale information-guided optimized EGO-Net we designed achieves the best results, especially showing better recognition workout for the biologically significant organoid with diameter ≥50 μm than other neural networks. Moreover, OPO achieves to reconstruct the multiscale structure of organoid clusters within printed microbeads and calibrate the printing errors by segmenting the printed microbeads edges. Overall, the classification, tracking and quantitative analysis based on image reveal that the growth process of organoid undergoes morphological changes such as volume growth, cavity creation and fusion, and quantitative calculation of the volume demonstrates that the growth rate of organoid is associated with the initial scale. The new method we proposed enable the study of growth, structural evolution and heterogeneity for the organoid cluster, which is valuable for drug screening and tumor drug sensitivity detection based on organoids.

[1]  Jeffrey E. Lee,et al.  3D imaging analysis on an organoid-based platform guides personalized treatment in pancreatic ductal adenocarcinoma , 2022, The Journal of clinical investigation.

[2]  Boyang Zhang,et al.  D-CryptO: deep learning-based analysis of colon organoid morphology from brightfield images. , 2022, Lab on a chip.

[3]  Mingen Xu,et al.  Dot extrusion bioprinting of spatially controlled heterogenous tumor models , 2022, Materials & Design.

[4]  K. Anseth,et al.  4D Materials with Photoadaptable Properties Instruct and Enhance Intestinal Organoid Development. , 2022, ACS biomaterials science & engineering.

[5]  P. Tam,et al.  Developing Biliary Atresia-like Model by Treating Human Liver Organoids with Polyinosinic:Polycytidylic Acid (Poly (I:C)) , 2022, Current issues in molecular biology.

[6]  Yixuan Ming,et al.  Longitudinal morphological and functional characterization of human heart organoids using optical coherence tomography , 2022, bioRxiv.

[7]  Bjoern H Menze,et al.  The Medical Segmentation Decathlon , 2021, Nature Communications.

[8]  P. Liberali,et al.  Multiscale light-sheet organoid imaging framework , 2021, Nature Communications.

[9]  Daniel A. Gil,et al.  Volumetric growth tracking of patient-derived cancer organoids using optical coherence tomography. , 2021, Biomedical optics express.

[10]  Cheng Wang,et al.  A deep learning model for detection and tracking in high-throughput images of organoid , 2021, Comput. Biol. Medicine.

[11]  Bitewulign Kassa Mekonnen,et al.  Generation of Augmented Capillary Network Optical Coherence Tomography Image Data of Human Skin for Deep Learning and Capillary Segmentation , 2021, Diagnostics.

[12]  F. Matthäus,et al.  Long-term live imaging and multiscale analysis identify heterogeneity and core principles of epithelial organoid morphogenesis , 2021, BMC biology.

[13]  Wolfgang Drexler,et al.  Ultra-High-Resolution 3D Optical Coherence Tomography Reveals Inner Structures of Human Placenta-Derived Trophoblast Organoids , 2020, IEEE Transactions on Biomedical Engineering.

[14]  S. Du,et al.  A review of object detection based on deep learning , 2020, Multimedia Tools and Applications.

[15]  Martin Jägersand,et al.  U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection , 2020, Pattern Recognit..

[16]  T. R. Fennel,et al.  Improved automated segmentation of human kidney organoids using deep convolutional neural networks , 2020, Medical Imaging: Image Processing.

[17]  Di Zhao,et al.  A review of the application of deep learning in medical image classification and segmentation , 2020, Annals of translational medicine.

[18]  Muhammad Haris,et al.  Application of deep learning for retinal image analysis: A review , 2020, Comput. Sci. Rev..

[19]  Ashley M. Fuller,et al.  Characterizing optical coherence tomography speckle fluctuation spectra of mammary organoids during suppression of intracellular motility. , 2020, Quantitative imaging in medicine and surgery.

[20]  Mathias Fink,et al.  Dynamic full-field optical coherence tomography: 3D live-imaging of retinal organoids , 2019, Light: Science & Applications.

[21]  Jaepyeong Cha,et al.  Deep Neural Network Regression for Automated Retinal Layer Segmentation in Optical Coherence Tomography Images , 2020, IEEE Transactions on Image Processing.

[22]  Timothy Kassis,et al.  OrgaQuant: Human Intestinal Organoid Localization and Quantification Using Deep Convolutional Neural Networks , 2019, Scientific Reports.

[23]  Ming-Ming Cheng,et al.  EGNet: Edge Guidance Network for Salient Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  Heinz Handels,et al.  Segmentation of mouse skin layers in optical coherence tomography image data using deep convolutional neural networks. , 2019, Biomedical optics express.

[25]  T. Takebe,et al.  Organoids by design , 2019, Science.

[26]  Hans Clevers,et al.  Cancer modeling meets human organoid technology , 2019, Science.

[27]  E. Siggia,et al.  Self-organization of stem cells into embryos: A window on early mammalian development , 2019, Science.

[28]  Xiangjian He,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[29]  Xun Xu,et al.  Robust deep learning method for choroidal vessel segmentation on swept source optical coherence tomography images. , 2019, Biomedical optics express.

[30]  Leopold Schmetterer,et al.  Automated segmentation of dermal fillers in OCT images of mice using convolutional neural networks. , 2019, Biomedical optics express.

[31]  Harry Begthel,et al.  Mouse and human urothelial cancer organoids: A tool for bladder cancer research , 2019, Proceedings of the National Academy of Sciences.

[32]  A. Oudenaarden,et al.  Long‐term expanding human airway organoids for disease modeling , 2019, The EMBO journal.

[33]  Mechthild Krause,et al.  Human gastric cancer modelling using organoids , 2018, Gut.

[34]  Delia Cabrera DeBuc,et al.  OCT Segmentation via Deep Learning: A Review of Recent Work , 2018, ACCV Workshops.

[35]  Klaus H. Maier-Hein,et al.  nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2018, Bildverarbeitung für die Medizin.

[36]  Leixin Zhou,et al.  Multiple surface segmentation using convolution neural nets: application to retinal layer segmentation in OCT images , 2018, Biomedical optics express.

[37]  Hayley E. Francies,et al.  Organoid cultures recapitulate esophageal adenocarcinoma heterogeneity providing a model for clonality studies and precision therapeutics , 2018, Nature Communications.

[38]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[39]  Markus H. Heim,et al.  Organoid Models of Human Liver Cancers Derived from Tumor Needle Biopsies , 2018, Cell reports.

[40]  Cyriac Kandoth,et al.  Tumor Evolution and Drug Response in Patient-Derived Organoid Models of Bladder Cancer , 2018, Cell.

[41]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[42]  Bon-Kyoung Koo,et al.  Human Primary Liver Cancer -derived Organoid Cultures for disease modelling and drug screening , 2017, Nature Medicine.

[43]  Raquel Urtasun,et al.  DeepRoadMapper: Extracting Road Topology from Aerial Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[44]  Jacob W. Guggenheim,et al.  A process engineering approach to increase organoid yield , 2017, Development.

[45]  Hans Clevers,et al.  Designer matrices for intestinal stem cell and organoid culture , 2016, Nature.

[46]  Dirk Schumacher,et al.  Assay Establishment and Validation of a High-Throughput Screening Platform for Three-Dimensional Patient-Derived Colon Cancer Organoid Cultures , 2016, Journal of biomolecular screening.

[47]  Russell M Taylor,et al.  Inverse-power-law behavior of cellular motility reveals stromal-epithelial cell interactions in 3D co-culture by OCT fluctuation spectroscopy. , 2015, Optica.

[48]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[49]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[50]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[51]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..