Track 1: Flood rapid mapping with SAR data

This track focuses on mapping the water surface from Copernicus Sentinel-1 SAR imagery. A set of events will be provided from selected geographical areas and events. The goal is to accurately determine water and non-water pixels in these event areas by fusing data from one or more of the provided data sources. A baseline of machine learning algorithms, like random forest, yield F1-scores that range from 0.70 for the most difficult cases to 0.98 for the simplest ones. The goal of the challenge is to maximize the average F1-score over the test set.

Track 2: Flood rapid mapping with optical data

Track-2 focuses on mapping the water surface from Copernicus Sentinel-2 and Landsat optical imageries. The physics of measurement is very different from track 1. Despite a greater number of spectral bands and a much better signal-to-noise ratio, water reflectance is quite variable in the optical domain. Moreover, the availability of optical data around flooding events is generally poorer due to cloud cover. The latter is well characterized in the available data, but it will be necessary to take it into account in the various training and inference stages. As for track 1, a set of events will be provided from selected geographical areas and events. The goal is to accurately determine water and non-water pixels in these event areas by fusing data from one or more of the provided data sources.

Competition Phases

The contest in both tracks will consist of two phases. In Phase 1, participants will train and validate their algorithms against a common data set. In Phase 2, participants will have a week to run inference against a new test set to determine the winners.

Phase 1: Participants are provided training data and additional validation images (without corresponding reference data) to train and validate their algorithms. Participants can submit results for the validation set to the Codalab competition website to get feedback on their performance. The performance of the best submission from each account will be displayed on the leaderboard. In parallel, participants are expected to submit a short description of the approach used to be eligible to enter Phase 2.

Phase 2: Participants receive the test data set (without the corresponding reference data) and submit their results within seven days from the release of the test data. After evaluation of the results, four winners for each track are announced. Following this, they will have one month to write their manuscript, which will be presented and included in the IGARSS 2024 proceedings. Manuscripts are 4-page IEEE-style formatted. Each manuscript describes the addressed problem, the proposed method, and the experimental results.

Calendar

PHASE 1

• January 8: Contest opening: release of training and validation data
• January 8: Evaluation server begins accepting submissions for the validation data set
• March 1: Participants submit a short description of their approach in 1-2 pages to iadf_chairs@grss-ieee.org (using the IGARSS paper template)

PHASE 2

• March 11: Release of test data; evaluation server begins accepting test submissions
• March 17: Evaluation server stops accepting submissions
• March 22 : Updated and final description of the approach

WINNER ANNOUNCEMENT AND PUBLICATIONS

• March 29: Winner announcement
• April 26: Internal deadline for papers, DFC Committee review process
• May 25: Submission deadline of final papers to be published in the IGARSS 2024 proceedings
• July: presentation at IADF-dedicated IGARSS 2024 Community-Contributed Sessions

The Data

Data from the following datasets are available to participants. These consist of SAR remote sensing data, Optical remote sensing data, Digital Elevation Model, land-use and water occurrence. The datasets are well documented in their respective archives.

• Copernicus/Sentinel-1: C-band synthetic aperture radar, 10m resolution
• Harmonized Landsat Sentinel-2 images: 30m multispectral images in the optical range
• Copernicus DEM (30m): it is a Digital Surface Model (DSM) that represents the surface of the Earth including buildings, infrastructure and vegetation
• MERIT (90m): it is a digital terrain model widely used in the hydrology scientific community
• Global Surface Water Occurrence: maps the location and temporal distribution of water surfaces at the global scale over the past 32 years and provides statistics on the extent and change of those water surfaces
• ESA WorldCover: global land cover product at 10 m resolution

Labeled training data sets will be curated from:

• Flood extent labeled by the Copernicus Emergency Management Service
• OPERA Dynamic Surface Water Extent CalVal database

●     CERFACS simulations of flooded area correspondingto Copernicus Sentinel acquisitions. Reference data will be generated from a detailed CERFACS hydrodynamic model in the FloodDAM Digital Twin (FloodDAM DT). The modeled water extents will be provided on flood events over two physical sites, the Garonne River/France and Ohio River/US.

Submission and Evaluation

In both tracks, the task consists of a binary water / non-water pixel-wise classification. Participants will submit rapid flood maps to the codalab server. Flood maps shall be a TIFF gridded raster product where individual pixels are labeled water (1), or non-water (0). The flood map will have the same grid and resolution as the identified test data file.

Classification accuracy will be evaluated against a test subsample of the reference dataset, which will not be provided to participants. The F1-score metric will be used to rank the results. The algorithm with the highest F1-score on the Phase 2 test set will be the winner.

Results, Awards, and Prizes

The first 4 ranked teams in each track will be declared as winners. Winning teams will:

• Present their approach in an invited session dedicated to the DFC24 at IGARSS 2024
• Publish their manuscripts in the proceedings of IGARSS 2024
• Be awarded IEEE Certificates of Recognition
• Winning teams will be awarded during IGARSS 2024, Athens Greece, in July 2024. The costs for open-access publication in JSTARS will be supported by the GRSS. The first ranked team prize is kindly sponsored by the organizing partners.

Moreover,

• The authors of the first and second-ranked teams of each track will co-author a journal paper which will summarize the outcome of the DFC24 and will be submitted with open access to IEEE JSTARS.
• The first-ranked teams of each track will receive one paid trip (flight and hotel) to IGARSS24 sponsored by Space for Climate Observatory.
• The second and third-ranked teams will be invited to make a presentation at an international event, different from IGARSS, like the annual SCO congress.

The Rules of the Competition

• The dataset can be openly downloaded from IEEE Data Port

• To enter the contest, participants must read and accept the Contest Terms and Conditions.
•  Participants of the contest are intended to submit results as the Submission and Evaluation Section
•  The results will be submitted to the Codalab competition website for evaluation.
•  Ranking between the participants will be based on the metrics as described in the Submission and Evaluation Section.
•  The maximum number of trials of one team is five per day in the test phase.
•  The submission server of the test phase will be opened on March 11 2024 at 23:59 UTC-12 hours.
•  The deadline for result submission and final description is March 17 2024, 23:59 UTC-12 hours (e.g., March 16, 2024, 6:59 in New York City, 12:59 in Paris, or 19:59 in Beijing).
•  Each team needs to submit a short paper of 1–2 pages explaining the approach used, the team members, their Codalab accounts, and one Codalab account to be used for the test phase by March 1, 2024. Please send a paper to iadf_chairs@grss-ieee.org using the IGARSS paper template. Only teams that have submitted the short description complete with all information will be admitted to the test phase.
•  For the winning teams, the internal deadline for full paper submission is April 26, 2024, 23:59 UTC – 12 hours (e.g., April 27, 2024, 07:59 in New York City, 13:59 in Paris, or 19:59 in Beijing).

• Important: Only team members explicitly stated on these documents will be considered for the next steps of the DFC, i.e., being eligible to be awarded as winners and joining the author list of the respective potential publications (IGARSS24 and JSTARS articles). Furthermore, no overlap among teams is allowed, i.e., one person can only be a member of one team. Adding more team members after the end of the development phase, i.e., after submitting these documents is not possible.
• Persons directly involved in the organization of the contest, i.e., the (co-)chairs of IADF as well as the co-organizers are not allowed to enter the contest. Please note that IADF WG leads can enter the contest. They have been excluded from relevant information concerning the content of the DFC to ensure a fair competition.

Failure to follow any of these rules will automatically make the submission invalid, resulting in the manuscript not being evaluated and disqualification from the prize award.

Participants in the Contest are requested not to submit an extended abstract describing their approach to tackle the DFC24 to IGARSS 2024 by the corresponding conference deadline in January 2024. Only contest winners (participants corresponding to the best-ranking submissions) will submit a 4-page paper describing their approach to the Contest by April 26, 2024. The received manuscripts will be reviewed by the Award Committee of the Contest, and reviews will be sent to the winners. Winners will submit the 4-pages full-paper to the Award Committee of the Contest by May 25, who will then take care of the submission to the IGARSS Data Fusion Contest Community Contributed Session by May 31, 2024, for inclusion in the IGARSS Technical Program and Proceedings.

 Terms and Conditions

 Participants of this challenge acknowledge that they have read and agree to the following Contest Terms and Conditions:

• In any scientific publication using the data, the data shall be referenced as follows: “[REF. NO.] 2024 IEEE GRSS Data Fusion Contest. Online: www.grss-ieee.org/technical-committees/image-analysis-and-data-fusion/”.
• Any scientific publication using the data shall include a section “Acknowledgement”. This section shall include the following sentence: “The authors would like to thank the IEEE GRSS Image Analysis and Data Fusion Technical Committee, the Space for Climate Observatory, CNES, NASA, and CERFACS for organizing the Data Fusion Contest”.

Acknowledgements

 The IADF TC chairs thank the Space for Climate Observatory, CNES, NASA and CERFACS for providing the data and the IEEE GRSS for continuously supporting the annual Data Fusion Contest through funding and resources.