Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Histological staining is a vital step in diagnosing various diseases and has been used for more than a century to provide contrast in tissue sections, rendering the tissue constituents visible for microscopic analysis by medical experts. However, this process is time consuming, labour intensive, expensive and destructive to the specimen. Recently, the ability to virtually stain unlabelled tissue sections, entirely avoiding the histochemical staining step, has been demonstrated using tissue-stain-specific deep neural networks. Here, we present a new deep-learning-based framework that generates virtually stained images using label-free tissue images, in which different stains are merged following a micro-structure map defined by the user. This approach uses a single deep neural network that receives two different sources of information as its input: (1) autofluorescence images of the label-free tissue sample and (2) a “digital staining matrix”, which represents the desired microscopic map of the different stains to be virtually generated in the same tissue section. This digital staining matrix is also used to virtually blend existing stains, digitally synthesizing new histological stains. We trained and blindly tested this virtual-staining network using unlabelled kidney tissue sections to generate micro-structured combinations of haematoxylin and eosin (H&E), Jones’ silver stain, and Masson’s trichrome stain. Using a single network, this approach multiplexes the virtual staining of label-free tissue images with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created in the same tissue cross section, which is currently not feasible with standard histochemical staining methods. A machine learning approach uses combinations of multiple digital stains to highlight microscopic elements in a tissue sample, avoiding the delays, inconsistences and sometimes the need for multiple biopsies, all characteristic of traditional manual tissue staining techniques. The deep learning-based tissue staining framework was developed by Aydogan Ozcan, Yair Rivenson and colleagues at the University of California, Los Angeles. They used their approach to train neural networks to virtually stain kidney samples with one of three different types of stains or their combinations. Comparisons with manually stained tissue samples demonstrated that the virtual staining was highly accurate. The ability to use multiple digital stains and virtual stain blending on the same tissue sample using a single neural network could allow pathologists to get more relevant information from tissue and thus improve diagnoses.