SPECTRAL SYNTHESIS FOR SATELLITE-TO-SATELLITE TRANSLATION

Abstract

Earth observing satellites carrying multi-spectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the Earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem with partial labels and introduce a novel shared spectral reconstruction loss. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. On a downstream cloud detection task, we show that generating synthetic bands with our model improves segmentation performance beyond our baseline. Our proposed approach enables synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.

1. INTRODUCTION

Climate change and related environmental issues -including the loss of biodiversity and extreme weather -are listed by the World Economic Forum as the most important risks to our planet (7). Monitoring the Earth is critical to mitigating these risks, understanding the effects, and making future predictions (38). Multi-and hyper-spectral satellite-based remote sensing enables global observation of the Earth, allowing scientists to study large-scale system dynamics and inform general circulation models (26). In weather forecasts satellite data initializes the atmospheric state for future predictions. On longer time scales, these data are used to measure the effects of climate change such as land-cover variations, temperature trends, solar radiation levels, and the rate of snow/ice melt. In the coming decades, increased investments from the public and private sectors in satellite-based observations will continue to improve global monitoring, as highlighted in NASA's decadal survey (25). Satellites are designed based on specifications for a given set of applications with fiscal, technological, and physical constraints which limit their temporal, spatial, and spectral resolutions. Geostationary (GEO) satellites rotate with the Earth to stay over a constant position above the equator at a high elevation of 35,786km. This position enables GEO satellites with on-board multi-spectral imagers to take continuous and high-temporal snapshots over large spatial regions and are ideal for monitoring diurnal and fast moving events. Spectral bands measure brightness and radiance intensities of the electromagnetic spectrum at a specified center wavelength and bandwidth. Bands are selected to satisfy defined variables of interest constrained by technological cost and accuracy. Applications of GEO sensors include atmospheric winds measurement (35), tropical cyclone tracking (36), wildfire monitoring (41), and short-term forecasting (24). Multiple GEO satellites are needed to generate global high-temporal resolution datasets to better monitor these events around the world. However, variations in resolutions, sensor uncertainties, and temporal life spans leads to a set of separate datasets which are not consistent, making this process very challenging (26). Developing consistent and homogeneous global datasets would relieve many of these challenges.

Supplementary

Material: https://github.com/anonymous-ai-for-earth/ satellite-to-satellite-translation 

