Abstract

Synthetic Aperture Radar (SAR) imagery plays a vital role in identifying flooded areas in the aftermath causing loss of life and significant economic and environmental damage, as water surfaces reflect less microwave energy compared to land due to their smooth texture and low surface roughness. In this study, we present a multimodal fusion strategy that enhances the existing model introduced in [1] through an integration of histogram extraction layers designed for SAR data with Active Contour Models (ACMs), which are integrated into fine-tuned Deep Segmentation Neural Networks (DSNNs). The model was tested on a real SAR dataset, with cross-dataset validation using an external cohort, representing a second innovation in our approach. Experimental results demonstrate that our model, with histogram layers + ACM, outperforms previous approaches by up to 10% in internal and 4% in external cohorts as intersection over union (IoU) and provides a comprehensive evaluation through metrics like Accuracy and Loss.

Instructions:

# Enhanced-HisSegNet

## Enhanced-HisSegNet: Improved SAR Image Flood Segmentation with Histogram Layers and Active Contour Model

This repository provides the code and details for the paper submitted to IEEE Geoscience and Remote Sensing Letters titled "Enhanced-HisSegNet: Improved SAR Image Flood Segmentation with Histogram Layers and Active Contour Model." The proposed model integrates histogram extraction layers with Active Contour Models (ACMs) into fine-tuned Deep Segmentation Neural Networks (DSNNs) for enhanced flood detection using Synthetic Aperture Radar (SAR) imagery.

## Description

This study presents a multimodal fusion strategy that enhances a previously introduced model by integrating histogram extraction layers for Synthetic Aperture Radar (SAR) data with Active Contour Models (ACMs) into fine-tuned Deep Segmentation Neural Networks (DSNNs), demonstrating improved performance of up to 10% in internal and 4% in external cohorts for flood identification from SAR imagery, as evaluated by Intersection over Union (IoU) and other metrics.

## Dataset Access

### Sen1Floods11

**NOTE: As of v1.1 of Sen1Floods11 the data has been moved to the `sen1floods11` bucket on GCS. The original v1.0 data is still hosted for now on the original `cnn_chips` bucket. v1.1 involved major restructing of the bucket and more understandable file naming as well as updates to the dataset and data format to comply with COG spec.**

Sen1Floods11: a georeferenced dataset to train and test deep learning flood algorithms for Sentinel-1. This data was generated by Cloud to Street, a Public Benefit Corporation: https://www.cloudtostreet.info/. For questions about this dataset.

info. Please cite this data as:

Bonafilia, D., Tellman, B., Anderson, T., Issenberg, E. 2020. Sen1Floods11: a georeferenced dataset to train and test deep learning flood algorithms for Sentinel-1. The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 210-211.

Available Open access at: http://openaccess.thecvf.com/content_CVPRW_2020/html/w11/Bonafilia_Sen1F...

The dataset is available for access through Google Cloud Storage bucket at: `gs://senfloods11/`

You can access the dataset bucket using the [gsutil](https://cloud.google.com/storage/docs/gsutil) command line tool. If you would like to download the entire dataset (~14 GB) you can use `gsutil rsync` to clone the bucket to a local directory. The `-m` flag is recommended to speed downloads. The `-r` flag will download sub-directories and folder recursively. See the example below.

```bash

$ gsutil -m rsync -r gs://sen1floods11 /YOUR/LOCAL/DIRECTORY/HERE

```

If using an example notebook, you can download the dataset to the folder that notebooks expect it to be in by running

```bash

$ mkdir /home/files

$ gsutil -m rsync -r gs://sen1floods11 /home/files

```

#### Bucket Structure

The `sen1floods11` bucket is split into subfolders containing data, checkpoints, training/testing splits, and a [STAC](https://stacspec.org/) compliant catalog. More detail on each is provided in the docs README.

#### Dataset Information

Each file follows the naming scheme EVENT_CHIPID_LAYER.tif (e.g. `Bolivia_103757_S2Hand.tif`). Chip IDs are unique, and not shared between events. Events are named by country and further information on each event (including dates) can be found in the event metadata below. Each layer has a separate GeoTIFF, and can contain multiple bands in a stacked GeoTIFF. All images are projected to WGS 84 (`EPSG:4326`) at 10 m ground resolution.

| ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| S2 | Raw Sentinel-2 MSI Level-1C imagery Contains all spectral bands (1 - 12) Does not contain QA mask | Unit: TOA reflectance (scaled by 10000) | GeoTIFF 512 x 512 13 bands UInt16 | 0: B1 (Coastal) 1: B2 (Blue) 2: B3 (Green) 3: B4 (Red) 4: B5 (RedEdge-1) 5: B6 (RedEdge-2) 6: B7 (RedEdge-3) 7: B8 (NIR) 8: B8A (Narrow NIR) 9: B9 (Water Vapor) 10: B10 (Cirrus) 11: B11 (SWIR-1) 12: B12 (SWIR-2) |

#### Example images

A sample of the dataset for chip _Spain_7370579_ is provided below:

<div>

</div>

## Methodology

The Enhanced-HisSegNet model is built upon a fusion of techniques to improve the segmentation of SAR flood images:

Histogram Extraction Layers: These layers help enhance feature extraction from SAR images, capturing the distinct characteristics of water and land surfaces.

Active Contour Models (ACMs): Integrated into the segmentation network to refine boundary detection and improve accuracy.

Deep Segmentation Neural Networks (DSNNs): Fine-tuned DSNNs are used to learn and perform the flood segmentation task more effectively.

The model was evaluated using a real SAR dataset with cross-dataset validation, demonstrating significant performance improvements over previous approaches.

Results

The proposed model showed the following improvements in segmentation performance:

1. Internal Dataset: Up to 10% improvement in IoU.

2. External Dataset (Cross-Validation): 4% improvement in IoU.

3. Comprehensive metrics, including Accuracy and Loss, were used to validate the model's performance across different datasets.

## Code Structure

The repository contains six models for training and evaluating SAR flood segmentation:

1. Tuned_ResNet_50: Fine-tuned ResNet50 model.

2. Normal_ResNet_50 (LV): ResNet50 model with (Histogram Normal layer + ACM) combined with FCNN.

3. Log_Normal_ResNet_50: ResNet50 model with (Histogram Log-Normal layer + ACM) combined with FCNN.

4. DeepLab0: Fine-tuned DeepLab model.

5. Normal_DeepLabV3: DeepLabV3 model with (Histogram Normal layer + ACM) combined with FCNN.

6. DeepLab(Log_Normal): DeepLabV3 model with (Histogram Log-Normal layer + ACM) combined with FCNN.

## Event Metadata

Locations of the flood events and metadata is contained in _Sen1Floods11_Metadata.geojson_. The following fields can be found:

| Field | Description |

| --------------- | -------------------------------------------------------------------------------------- |

| ID | Unique ID for each event |

| location | Flood event location (country) |

| ISO_CC | ISO Country Code for flood event location |

| s1_date | Date (YYYY-MM-dd) that Sentinel-1 image was acquired |

| s2_date | Date (YYYY-MM-dd) that Sentinel-2 image was acquired |

| orbit | Orbit (ASCENDING or DESCENDING) that Sentinel-1 image was acquired |

| rel_orbit_num | Relative Orbit Number that Sentinel-1 image was acquired |

| coincident_size | Number of coincident tiles from S2 |

| VH_thresh | Threshold used for Sentinel-1 VH band to classify water in reference S1 classification |

| train_chip | Number of chips used for training |

| val_chip | Number of chips used for validation |

## Citation

Citation link will be added.

## Contact

For any questions regarding the dataset, please email support@cloudtostreet.info. For questions about the code, contact ilani.a.m1990@gmail.com.

Dataset Files

Source Code.zip (492.67 kB)

Documentation

Attachment	Size
README.md	10.01 KB

Datasets

Standard Dataset

Enhanced-HisSegNet: Improved SAR Image Flood Segmentation with Learnable Histogram Layers and Active Contour Model

Abstract

Dataset Files

Documentation

QUESTIONS?