SEnSeI: A Deep Learning Module for Creating Sensor Independent Cloud Masks

We introduce a novel neural network architecture—spectral encoder for sensor independence (SEnSeI)—by which several multispectral instruments, each with different combinations of spectral bands, can be used to train a generalized deep learning model. We focus on the problem of cloud masking, using several preexisting datasets, and a new, freely available dataset for Sentinel-2. Our model is shown to achieve state-of-the-art performance on the satellites on which it was trained (Sentinel-2 and Landsat 8) and is able to extrapolate to sensors that it has not seen during training, such as Landsat 7, PerúSat-1, and Sentinel-3 Sea and Land Surface Temperature Radiometer (SLSTR). Model performance is shown to improve when multiple satellites are used in training, approaching, or surpassing the performance of specialized, single-sensor models. This work is motivated by the fact that the remote sensing community has access to data taken with a huge variety of sensors. This has inevitably led to labeling efforts being undertaken separately for different sensors, which limits the performance of deep learning models, given their need for huge training sets to perform optimally. Sensor independence can enable deep learning models to utilize multiple datasets for training simultaneously, boosting performance, and making them much more widely applicable. This may lead to deep learning approaches being used more frequently for onboard applications and in ground segment data processing, which generally requires models to be ready at launch or soon afterward.