Few-Shot Remote Sensing Image Domain Generalization

Citation Author(s):
Mainak
Singha
Bupendra Singh
Solanki
Ankit
Jha
Shirsha
Bose
Biplab
Banerjee
Submitted by:
Ankit Jha
Last updated:
Wed, 08/07/2024 - 00:53
DOI:
10.21227/71d7-ad49
Data Format:
Research Article Link:
License:
59 Views
Categories:
Keywords:
0
0 ratings - Please login to submit your rating.

Abstract 

In recent years, the success of large-scale visionlanguage models (VLMs) such as CLIP has led to their increased usage in various computer vision tasks. These models enable zero-shot inference through carefully crafted instructional text prompts without task-specific supervision. However, the potential of VLMs for generalization tasks in remote sensing (RS) has not been fully realized. To address this research gap, we propose a novel image-conditioned prompt learning strategy called the Visual Attention Parameterized Prompts Learning Network (APPLeNet). APPLeNet emphasizes the importance of multi-scale feature learning in RS scene classification and disentangles visual style and content primitives for domain generalization tasks. To achieve this, APPLeNet combines visual content features obtained from different layers of the vision encoder and style properties obtained from feature statistics of domain-specific batches. An attention-driven injection module is further introduced to generate visual tokens from this information. We also introduce an anticorrelation regularizer to ensure discrimination among the token embeddings, as this visual information is combined with the textual tokens. To validate APPLeNet, we curated four available RS benchmarks and introduced experimental protocols and datasets for three domain generalization tasks. Our results consistently outperform the relevant literature and code is available at https://github.com/ mainaksingha01/APPLeNet

Instructions: 

We use the online availabe remote sensing dataset i.e., PatternNet (https://sites.google.com/view/zhouwx/dataset), MLRSNet (https://github.com/201528014227051/RSICD_optimal?tab=readme-ov-file), RSICD (https://www.tensorflow.org/datasets/catalog/resisc45), and RESISC45 (https://data.mendeley.com/datasets/7j9bv9vwsx/3) and curated the newer version for domain generalization using prompt learning (which consists of equal number of classes in all the remote sensing datasets).

1. PatternNetv2

2. RSICDv2

3. RESISC45v2

4. MLRSNetv2

Dataset Files

    Files have not been uploaded for this dataset