Real‐time video mosaicking to guide handheld in vivo microscopy

Handheld and endoscopic optical-sectioning microscopes are being developed for noninvasive screening and intraoperative consultation. Imaging a large extent of tissue is often desired, but miniature in vivo microscopes tend to suffer from limited fields of view. To extend the imaging field during clinical use, we have developed a real-time video mosaicking method, which allows users to efficiently survey larger areas of tissue. Here we modified a previous post-processing mosaicking method so that real-time mosaicking is possible at >30 frames/sec when using a device that outputs images that are 400 x 400 pixels in size. Unlike other real-time mosaicking methods, our strategy can accommodate image rotations and deformations that often occur during clinical use of a handheld microscope. We perform a feasibility study to demonstrate that the use of real-time mosaicking is necessary to enable efficient sampling of a desired imaging field when using a handheld dual-axis confocal microscope. This article is protected by copyright. All rights reserved.

[1]  R. Webb,et al.  In vivo confocal scanning laser microscopy of human skin: melanin provides strong contrast. , 1995, The Journal of investigative dermatology.

[2]  M. Rajadhyaksha,et al.  Miniature in vivo MEMS-based line-scanned dual-axis confocal microscope for point-of-care pathology. , 2016, Biomedical optics express.

[3]  Miguel Cordova,et al.  Automated video-mosaicking approach for confocal microscopic imaging in vivo: an approach to address challenges in imaging living tissue and extend field of view , 2017, Scientific Reports.

[4]  James G. Fujimoto,et al.  Assessment of breast pathologies using nonlinear microscopy , 2014, Proceedings of the National Academy of Sciences.

[5]  Liang Gao,et al.  Extend the field of view of selective plan illumination microscopy by tiling the excitation light sheet. , 2015, Optics express.

[6]  Tomasz S. Tkaczyk,et al.  Real-time video mosaicing with a high-resolution microendoscope , 2012, Biomedical optics express.

[7]  M. Rajadhyaksha,et al.  Correlation of Handheld Reflectance Confocal Microscopy With Radial Video Mosaicing for Margin Mapping of Lentigo Maligna and Lentigo Maligna Melanoma , 2017, JAMA dermatology.

[8]  Milind Rajadhyaksha,et al.  Rapid confocal imaging of large areas of excised tissue with strip mosaicing. , 2011, Journal of biomedical optics.

[9]  Chengbo Yin,et al.  Dual-Axis Confocal Microscopy for Point-of-Care Pathology , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[10]  N. Sanai,et al.  Trends in fluorescence image-guided surgery for gliomas. , 2014, Neurosurgery.

[11]  Stephan Saalfeld,et al.  Globally optimal stitching of tiled 3D microscopic image acquisitions , 2009, Bioinform..

[12]  M. Rajadhyaksha,et al.  Peri‐operative delineation of non‐melanoma skin cancer margins in vivo with handheld reflectance confocal microscopy and video‐mosaicking , 2019, Journal of the European Academy of Dermatology and Venereology : JEADV.

[13]  M E Martone,et al.  Automated microscopy system for mosaic acquisition and processing , 2006, Journal of microscopy.

[14]  D H Brooks,et al.  Video‐mosaicing of reflectance confocal images for examination of extended areas of skin in vivo , 2014, The British journal of dermatology.

[15]  Shaoqun Zeng,et al.  Rapid imaging of large tissues using high-resolution stage-scanning microscopy. , 2015, Biomedical optics express.

[16]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Zachary M. Eastman,et al.  Confocal mosaicing microscopy in Mohs skin excisions: feasibility of rapid surgical pathology. , 2008, Journal of biomedical optics.

[18]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[19]  R. Webb,et al.  In vivo confocal scanning laser microscopy of human skin II: advances in instrumentation and comparison with histology. , 1999, The Journal of investigative dermatology.

[20]  M. Rajadhyaksha,et al.  Feasibility of a Video‐Mosaicking Approach to Extend the Field‐of‐View For Reflectance Confocal Microscopy in the Oral Cavity In Vivo , 2019, Lasers in surgery and medicine.

[21]  John Kenneth Salisbury,et al.  In Vivo Micro-Image Mosaicing , 2011, IEEE Transactions on Biomedical Engineering.

[22]  F Piccinini,et al.  Automated image mosaics by non‐automated light microscopes: the MicroMos software tool , 2013, Journal of microscopy.

[23]  R S Balaban,et al.  Rapid overlapping‐volume acquisition and reconstruction (ROVAR): automated 3D tiling for high‐resolution, large field‐of‐view optical microscopy , 2011, Journal of microscopy.

[24]  Kristen C. Maitland,et al.  Imaging inflammation in mouse colon using a rapid stage-scanning confocal fluorescence microscope. , 2012, Journal of biomedical optics.

[25]  Jonathan T. C. Liu,et al.  Handheld line-scanned dual-axis confocal microscope with pistoned MEMS actuation for flat-field fluorescence imaging. , 2019, Optics letters.

[26]  Jonathan T. C. Liu,et al.  Toward Quantitative Neurosurgical Guidance With High-Resolution Microscopy of 5-Aminolevulinic Acid-Induced Protoporphyrin IX , 2019, Front. Oncol..

[27]  Miguel Cordova,et al.  Handheld Reflectance Confocal Microscopy for the Detection of Recurrent Extramammary Paget Disease , 2017, JAMA dermatology.

[28]  J. Fujimoto,et al.  Rapid imaging of surgical breast excisions using direct temporal sampling two photon fluorescent lifetime imaging. , 2015, Biomedical optics express.