We proposed a two-branch multitask learning convolutional neural network to solve two different but related tasks at the same time. Our main task is to predict occult invasive disease in biopsy proven Ductal Carcinoma in-situ (DCIS), with an auxiliary task of segmenting microcalcifications (MCs). In this study, we collected digital mammography from 604 patients, 400 of which were DCIS. The model used patches with size of 512×512 extracted within a radiologist masked ROIs as input, with outputs including noisy MC segmentations obtained from our previous algorithms, and classification labels from final diagnosis at patients’ definite surgery. We utilized a deep multitask model by combining both Unet segmentation networks and prediction classification networks, by sharing first several convolutional layers. The model achieved a patch-based ROC-AUC of 0.69, with a case-based ROC-AUC of 0.61. Segmentation results achieved a dice coefficient of 0.49.

Traditional approaches to automatic diagnosis of skin lesions consisted of classifiers working on sets of hand-crafted features, some of which modeled lesion aspects of special importance for dermatologists. Recently, the broad adoption of convolutional neural networks (CNNs) in most computer vision tasks has brought about a great leap forward in terms of performance. Nevertheless, with this performance leap, the CNN-based computer-aided diagnosis (CAD) systems have also brought a notable reduction of the useful insights provided by hand-crafted features. This paper presents DermaKNet, a CAD system based on CNNs that incorporates specific subsystems modeling properties of skin lesions that are of special interest to dermatologists aiming to improve the interpretability of its diagnosis. Our results prove that the incorporation of these subsystems not only improves the performance, but also enhances the diagnosis by providing more interpretable outputs.

To fully define the target objects of interest in clinical diagnosis, many deep convolution neural networks (CNNs) use multimodal paired registered images as inputs for segmentation tasks. However, these paired images are difficult to obtain in some cases. Furthermore, the CNNs trained on one specific modality may fail on others for images acquired with different imaging protocols and scanners. Therefore, developing a unified model that can segment the target objects from unpaired multiple modalities is significant for many clinical applications. In this work, we propose a 3D unified generative adversarial network, which unifies the any-to-any modality translation and multimodal segmentation in a single network. Since the anatomical structure is preserved during modality translation, the auxiliary translation task is used to extract the modality-invariant features and generate the additional training data implicitly. To fully utilize the segmentation-related features, we add a cross-task skip connection with feature recalibration from the translation decoder to the segmentation decoder. Experiments on abdominal organ segmentation and brain tumor segmentation indicate that our method outperforms the existing unified methods.

The development of cameras in smartphones is possible as a point of care for early detection of cancer. Early detection using smartphones is carried out by giving the smartphone the ability to recognize objects with skin cancer characteristics. The convolution neural network (CNN) is often used in disease detection and classification. However, the CNN method requires high computing capability and a large memory that is difficult to perform on smartphones. In this paper, the MobileNet v2 and Faster R-CNN methods are utilized and executed on an Android-based application that can detect skin cancer. Both proposed architectures were trained to recognize actinic keratosis and melanoma skin cancer targets on images. The dataset used was 600 images, divided into two classes, actinic keratosis images and melanoma images with no attention for gender, age, or additional factors. In this study, an android app was developed to utilize the smartphone camera for skin cancer detection. The Faster R-CNN and MobileNet v2 models were implemented as an intelligent system for the screening. Two testing methods were performed in this study, the Jupyter notebook and android camera. Based on the experiment result, Faster R-CNN obtained higher accuracy when testing using the Jupyter, and MobileNet v2 got the same high accuracy when applying on a smartphone.

The American Cancer Society expected to diagnose 276,480 new cases of invasive breast cancer in the USA and 48,530 new cases of noninvasive breast cancer among women in 2020. Early detection of breast cancer, followed by appropriate treatment, can reduce the risk of death from this disease. DL through CNN can assist imaging specialists in classifying the mammograms accurately. Accurate classification of mammograms using CNN needs a well-trained CNN by a large number of labeled mammograms. Unfortunately, a large number of labeled mammograms are not always available. In this study, a novel procedure to aid imaging specialists in detecting normal and abnormal mammograms has been proposed. The procedure supplied the designed CNN with a cheat sheet for some classical attributes extracted from the ROI and an extra number of labeled mammograms through data augmentation. The cheat sheet aided the CNN through encoding easy-to-recognize artificial patterns in the mammogram before passing it to the CNN, and the data augmentation supported the CNN with more labeled data points. Fifteen runs of 4 different modified datasets taken from the MIAS dataset were conducted and analyzed. The results showed that the cheat sheet, along with data augmentation, enhanced CNN's accuracy by at least 12.2% and enhanced the precision of the CNN by at least 2.2. The mean accuracy, sensitivity, and specificity obtained using the proposed procedure were 92.1, 91.4, and 96.8, respectively, while the average area under the ROC curve was 94.9.

State of the art on melanoma screening evolved rapidly in the last two years, with the adoption of deep learning. Those models, however, pose challenges of their own, as they are expensive to train and difficult to parameterize. Objective: We investigate the methodological issues for designing and evaluating deep learning models for melanoma screening, by exploring nine choices often faced to design deep networks: model architecture, training dataset, image resolution, type of data augmentation, input normalization, use of segmentation, duration of training, additional use of SVM, and test data augmentation. Methods: We perform a two-level full factorial experiment, for five different test datasets, resulting in 2560 exhaustive trials, which we analyze using a multi-way ANOVA. Results: The main finding is that the size of training data has a disproportionate influence, explaining almost half the variation in performance. Of the other factors, test data augmentation and input resolution are the most helpful. Deeper models, when combined, with extra data, also help. We show that the costly full factorial design, or the unreliable sequential optimization, are not the only options: ensembles of models provide reliable results with limited resources. Conclusions and Significance: To move research forward on automated melanoma screening, we need to curate larger shared datasets. Optimizing hyperparameters and measuring performance on the same dataset is common, but leads to overoptimistic results. Ensembles of models are a cost-effective alternative to the expensive full-factorial and to the unstable sequential designs.

Skin lesion classification in the dermoscopy images exerts an enormous function on the improvement of diagnostic performance and reduction of melanoma deaths. This skin lesion classification task remains a challenge. Deep learning requires a lot of training data, and the classification algorithms of skin lesions have certain limitations. These two points make the accuracy of the skin lesion classification needs to be further improved. In this paper, a mutual learning model was presented to separate malignant from benign skin lesions using the skin dataset. This model enabled dual deep convolutional neural networks to mutually learn from each other. Experimental results on the ISIC 2016 Skin Lesion Classification dataset indicate that the mutual learning model obtains the most advanced performance.

Skin lesion classification and segmentation are highly correlated tasks. However, their relationship is not fully utilized in previous methods. In this paper, we propose a multi-task deep convolutional neural network architecture to solve the skin lesion classification and segmentation problem simultaneously. To take full advantage of features from different tasks and thus get richer knowledge about the sample, we design a feature passing module to pass messages between segmentation branch and classification branch. Since feature passing module is not always helpful and can be related with individual samples, gate functions are used for controlling messages transmission. Therefore, features from one task are learned and selectively passed to the other task, and vice versa, which effectively improves the performance of both tasks. We have evaluated the proposed method on ISBI-2017 challenge dataset, and the experimental results demonstrate the superiority and effectiveness of the proposed method, compared to our base model and other state-of-art methods.

Skin diseases have been increased in the recent decades and if detected earlier, can be treated effectively. Segmentation of affected skin lesion using dermoscopy or dermatoscopic images for cancer diagnosis has received attention from the community of image processing. This research work proposed a simple yet effective method in Dermoscopy image segmentation for the affected skin lesion. In order to remove undesirable pixels for better segmentation of images, blind deconvolution (BD) is used. After applying BD, the resultant RGB image is transformed into L*a*b colour channel. The 'a' channel from L*a*b is extracted as a feature value for the segmentation of the affected region. In order to yield better segmented skin lesion from the 'a' channel image, Otsu's thresholding with Morphology is used as a post-processing technique. A series of experimental results achieved an accuracy of 95% using 150 dermoscopy images.

Skin cancer, specially melanoma is one of most deadly diseases. In the color images of skin, there is a high similarity between different skin lesion like melanoma and nevus, which increase the difficulty of the detection and diagnosis. A reliable automated system for skin lesion classification is essential for early detection to save effort, time and human life. In this paper, an automated skin lesion classification method is proposed. In this method, a pre-trained deep learning network and transfer learning are utilized. In addition to fine-tuning and data augmentation, the transfer learning is applied to AlexNet by replacing the last layer by a softmax to classify three different lesions (melanoma, common nevus and atypical nevus). The proposed model is trained and tested using the ph2 dataset. The well-known quantative measures, accuracy, sensitivity, specificity, and precision are used in evaluating the performance of the proposed method where the obtained values of these measures are 98.61%, 98.33%, 98.93%, and 97.73%, respectively. The performance of the proposed method is compared with the existing methods where the classification rate of the proposed method outperformed the performance of the existing methods.

Skin cancer is the most common form of cancer affecting humans. Melanoma is the most dangerous type of skin cancer; and early diagnosis is extremely vital in curing the disease. So far, the human knowledge in this field is very limited, thus, developing a mechanism capable of identifying the disease early on can save lives, reduce intervention and cut unnecessary costs. In this paper, the researchers developed a new learning technique to classify skin lesions, with the purpose of observing and identifying the presence of melanoma.  This new technique is based on a convolutional neural network solution with multiple configurations; where the researchers employed an International Skin Imaging Collaboration (ISIC) dataset. Optimal results are achieved through a convolutional neural network composed of 14 layers. This proposed system can successfully and reliably predict the correct classification of dermoscopic lesions with 97.78% accuracy.

Skin cancer is one of the most threatening diseases worldwide. However, diagnosing a skin cancer correctly is challenging. Recently, deep learning algorithms have achieved excellent performance on various tasks. Particularly, they have been also implemented for the tasks of skin disease diagnosis. In this paper, we present a review on deep learning methods and their applications in skin disease diagnosis. We first introduce skin diseases and image acquisition methods in dermatology, and list several publicly available datasets for training and testing algorithms for skin disease diagnosis. Then, we introduce the conception of deep learning and review popular deep learning architectures. Thereafter, popular deep learning frameworks that facilitate the implementation of deep learning algorithms and performance evaluation metrics are presented. As an important part of this article, we then review the literatures involving deep learning methods for skin disease diagnosis from several aspects according to the specific tasks. Additionally, we discuss the challenges faced in the area of skin disease diagnosis with deep learning and suggest possible future research directions. Finally, we summarize the article. The major purpose of this article is to provide a conceptual and systematically review of the recent works on skin disease diagnosis with deep learning. Given the popularity of deep learning, there remains great challenges in the area, as well as opportunities that we can explore in the future.

Semi-supervised learning has recently been attracting attention as an alternative to fully supervised models that require large pools of labeled data. Moreover, optimizing a model for multiple tasks can provide better generalizability than single-task learning. Leveraging self-supervision and adversarial training, we propose a novel general purpose semi-supervised, multiple-task model---namely, self-supervised, semi-supervised, multitask learning (S$^4$MTL)---for accomplishing two important tasks in medical imaging, segmentation and diagnostic classification. Experimental results on chest and spine X-ray datasets suggest that our S$^4$MTL model significantly outperforms semi-supervised single task, semi/fully-supervised multitask, and fully-supervised single task models, even with a 50\% reduction of class and segmentation labels. We hypothesize that our proposed model can be effective in tackling limited annotation problems for joint training, not only in medical imaging domains, but also for general-purpose vision tasks.

Pressure injuries represent a tremendous healthcare challenge in many nations. Elderly and disabled people are the most affected by this fast growing disease. Hence, an accurate diagnosis of pressure injuries is paramount for efficient treatment. The characteristics of these wounds are crucial indicators for the progress of the healing. While invasive methods to retrieve information are not only painful to the patients but may also increase the risk of infections, non-invasive techniques by means of imaging systems provide a better monitoring of the wound healing processes without causing any harm to the patients. These systems should include an accurate segmentation of the wound, the classification of its tissue types, the metrics including the diameter, area and volume, as well as the healing evaluation. Therefore, the aim of this survey is to provide the reader with an overview of imaging techniques for the analysis and monitoring of pressure injuries as an aid to their diagnosis, and proof of the efficiency of Deep Learning to overcome this problem and even outperform the previous methods. In this paper, 114 out of 199 papers retrieved from 8 databases have been analyzed, including also contributions on chronic wounds and skin lesions.

Over the last decade, convolutional neural networks (CNNs) have emerged as the leading algorithms in image classification and segmentation. Recent publication of large medical imaging databases have accelerated their use in the biomedical arena. While training data for photograph classification benefits from aggressive geometric augmentation, medical diagnosis – especially in chest radiographs – depends more strongly on feature location. Diagnosis classification results may be artificially enhanced by reliance on radiographic annotations. This work introduces a general pre-processing step for chest x-ray input into machine learning algorithms. A modified Y-Net architecture based on the VGG11 encoder is used to simultaneously learn geometric orientation (similarity transform parameters) of the chest and segmentation of radiographic annotations. Chest x-rays were obtained from published databases. The algorithm was trained with 1000 manually labeled images with augmentation. Results were evaluated by expert clinicians, with acceptable geometry in 95.8% and annotation mask in 96.2% (n = 500), compared to 27.0% and 34.9% respectively in control images (n = 241). We hypothesize that this pre-processing step will improve robustness in future diagnostic algorithms.Clinical relevance— This work demonstrates a universal pre-processing step for chest radiographs – both normalizing geometry and masking radiographic annotations – for use prior to further analysis.

Melanoma is a type of skin cancer with high mortality rate. Early diagnosis of malignant melanoma usually relies on the judgement of dermatologists. Recently, more and more research paid attention to classify skin lesion automatically by dermoscopy images, which is able to assist clinicians to diagnose quickly and accurately. However, there still remain some challenges on extraction of discriminative feature due to the inter-class similarity and intra-class variation. In this paper, we propose a novel multi-pooling attention learning for skin lesions classification, which consists of multiple types of pooling operations, attention mechanism and feature fusion. The multi-pooling layer with flexible pooling patterns is designed to capture more representative features. The attention mechanism has the ability to help our model focus on the lesion regions. Meanwhile, in order to deal with the issues of inter-class similarity and intra-class variation of skin lesions, we adopt a joint loss function to optimize the final results. Our proposed approach has been validated on ISIC2017 dataset that provided by IEEE International Symposium on Biomedical Imaging (ISBI) 2017 challenge for Skin Lesion Analysis Towards Melanoma Detection. The results show that our proposed network achieves superior performance over six top-ranking approaches in ISBI 2017 challenge leaderboard and five other methods.

Early detection and segmentation of skin lesions is crucial for timely diagnosis and treatment, necessary to improve the survival rate of patients. However, manual delineation is time consuming and subject to intra- and inter-observer variations among dermatologists. This underlines the need for an accurate and automatic approach to skin lesion segmentation. To tackle this issue, we propose a multi-task convolutional neural network (CNN) based, joint detection and segmentation framework, designed to initially localize the lesion and subsequently, segment it. A ‘Faster region-based convolutional neural network’ (Faster-RCNN) which comprises a region proposal network (RPN), is used to generate bounding boxes/region proposals, for lesion localization in each image. The proposed regions are subsequently refined using a softmax classifier and a bounding-box regressor. The refined bounding boxes are finally cropped and segmented using ‘SkinNet’, a modified version of U-Net. We trained and evaluated the performance of our network, using the ISBI 2017 challenge and the PH2 datasets, and compared it with the state-of-the-art, using the official test data released as part of the challenge for the former. Our approach outperformed others in terms of Dice coefficients (\({>}0.93\)), Jaccard index (\({>}0.88\)), accuracy (\({>}0.96\)) and sensitivity (\({>}0.95\)), across five-fold cross validation experiments.

Discriminative models that require full supervision are inefficacious in the medical imaging domain when large labeled datasets are unavailable. By contrast, generative modeling---i.e., learning data generation and classification---facilitates semi-supervised training with limited labeled data. Moreover, generative modeling can be advantageous in accomplishing multiple objectives for better generalization. We propose a novel multi-task learning model for jointly learning a classifier and a segmentor, from chest X-ray images, through semi-supervised learning. In addition, we propose a new loss function that combines absolute KL divergence with Tversky loss (KLTV) to yield faster convergence and better segmentation performance. Based on our experimental results using a novel segmentation model, an Adversarial Pyramid Progressive Attention U-Net (APPAU-Net), we hypothesize that KLTV can be more effective for generalizing multi-tasking models while being competitive in segmentation-only tasks.

Dermoscopy image retrieval can assist dermatologists to make a diagnosis by reference to confirmed cases, which can improve the accuracy of the diagnosis result. This paper proposed a retrieve method based on the combination of color and texture. The proposed method uses the color moments and Gabor wavelet to extract features and implements retrieval function by SKLSH hash code. In the experiments stage, we retrieve dermoscopy images including 4 kinds of skin diseases from the datasets which are pigmented nevus, seborrheic keratosis, psoriasis and eczema. Besides, we compared our methods with other color and texture features, as well as other dermoscopy image retrieval method, and the results show that our method obtains the best retrieval result.

Dermoscopy image analysis (DIA) is a growing field, with works being published every week. This makes it difficult not only to keep track of all the contributions, but also for new researchers to identify relevant information and new directions to be explored. Several surveys have been written in the past decade, but these tend to cover all of the steps of a CAD system, which can be overwhelming. Moreover, in these works, each of the steps is briefly discussed due to lack of space. Among the different blocks of the CAD system, the most relevant is the one devoted to feature extraction. This is also the block where existing works exhibit the most variability. Therefore, we believe that it is important to review the state-of-the-art on this matter. This work thoroughly explores the several types of features that have been used in DIA. A discussion on their relevance and limitations, as well as suggestions for future research are provided.