Masters Thesis Presentation by Debashis Gupta on Monday, July 28 at 2:00 PM
Join us Monday, July 28 at 2:00 PM in Manc 024 for Debashis’s Masters Thesis Presentation!
His thesis titled “MoSAiC: Multi-Label Classification from Sparse Labels in Multi-Modal Remote Sensing Data“, supervised by Dr. Paúl Pauca, explore how MoSAiC, a dual-stream contrastive learning framework specifically designed for multi-modal satellite imagery, addresses key challenges face by multi-label land-cover mapping.
Abstract:
Multi-modal satellite imagery from sensors such as Sentinel-1 SAR and Sentinel-2 multispectral data provides complementary structural and spectral information crucial for effective environmental monitoring. However, harnessing these modalities for multi-label land cover mapping is complicated by limited and noisy annotations, high spectral similarity among classes, and difficulty capturing discriminative local features across radar and optical modalities. Existing contrastive learning (CL) approaches typically employ self-supervised methods separately within modalities but often neglect explicit supervised semantic alignment between modalities, particularly for multi-label scenarios with spectrally similar and spatially ambiguous classes. To address these gaps, this thesis introduces MoSAiC, a dual-stream contrastive learning framework specifically designed for multi-modal satellite imagery. MoSAiC leverages a self-supervised approach within each modality to preserve essential local characteristics—such as SAR backscatter textures and optical spectral signatures—while simultaneously employing a supervised multi-label contrastive loss for explicit semantic alignment between modalities. This combined learning strategy significantly enhances feature discriminability, particularly for classes with high spectral overlap. Evaluations on benchmark datasets BigEarthNet and Sent12MS confirm that MoSAiC achieves state-of-the-art performance, demonstrating notable improvements over conventional single-modality and simple fusion baselines. Notably, using only 10% training data, MoSAiC attains comparable accuracy to supervised, self-supervised & transformer-based models. These findings highlight MoSAiC’s effectiveness in achieving precise inter-modality semantic alignment through supervised contrastive learning, combined with robust self-supervised intra-modality feature extraction, enabling accurate and scalable multi-label land cover classification for Earth Observation applications.