7.0.1. Implementation Details
Since GRD products have two-channel inputs (VV and VH) and because pre-trained models like ResNet require three-channel inputs, the GRD format is incompatible. Instead of converting them to RGB, which could lead to the loss of SAR-specific information, we modify the model forward function to add a zero-filled third channel at runtime and we train the ResNet model architectures from scratch.
To manage outliers effectively, SAR data is clipped to maintain values within a 0-1 range, and we further standardize this data to ensure a zero mean and unit variance, which is common practice before feeding the data into models. As for the loss function, we experimented both with the focal Tversky loss and the custom weighted cross-entropy function as explained in
Section 6.2.1. The results have shown that the latter performs better. Our model adopts an Adam Optimizer with a learning rate of
, a weight decay
and momentum
, which have shown to achieve a stable training after testing various parameters. To have a balance between generalization and memory limitations, we employed a batch size of 8. Our experiments employing different image patch sizes, ranging from 128 to 512, revealed that larger patches significantly enhance performance by capturing more spatial information. However, due to memory constraints, we settled on a consistent patch size of 512x512. Before feeding patches to the model, we use Albumentations [
34] for random augmentations like flips (horizontal and vertical), rotations, and cropping. All with a probability of 0.5.
We tried two distinct settings for image processing. a) The first involves the generation of patches from the original images to ensure full coverage with minimal overlap, avoiding padding. Subsequently, we apply random augmentation. b) Alternatively, our second approach involves padding images to their maximum dimensions within the dataset, followed by random cropping of 512x512 patches from the original images, alongside the previously mentioned augmentations. The second technique, while slightly extending training time, improves model generalization.
7.1. Results on SAR-Optical Dataset
This section analyzes the performance of the top model, Unet + ResNet50, often surpassing the ground truth predictions from the Optical layer, used to train the model itself. During the EMSR442-0-6 flood event, despite cloud cover obscuring the Optical ground truth, the SAR model successfully identified the water regions. Additionally, the model excels in EMSR280-4-17 event by distinguishing snow-covered areas, which typically appear similar to water in SAR images. Here’s a detailed explanation of the labels in the images:
SAR Image – The input SAR image.
Optical (SCL) Ground Truth – Optical ground truth from Sentinel-2 SCL layer, provided by Copernicus.
Model Prediction - The model prediction (presence of water, whether it’s due to flooding or permanent sources).
Figure 10.
Performance Analysis of EMSR258-13-1 Permanent Water in Albania.
Figure 10.
Performance Analysis of EMSR258-13-1 Permanent Water in Albania.
Figure 11.
Performance Analysis of EMSR407-1-4 Flood Event in the United Kingdom. Event Date: November 12, 2019 at 21:15.
Figure 11.
Performance Analysis of EMSR407-1-4 Flood Event in the United Kingdom. Event Date: November 12, 2019 at 21:15.
Figure 12.
Performance Analysis of EMSR277-2-0 Flood Event in Greece. Event Date: March 29, 2018 at 11:27.
Figure 12.
Performance Analysis of EMSR277-2-0 Flood Event in Greece. Event Date: March 29, 2018 at 11:27.
Figure 13.
Performance Analysis of EMSR501-0-3 Flood Event in Albania. Event Date: February 12, 2021 at 16:50.
Figure 13.
Performance Analysis of EMSR501-0-3 Flood Event in Albania. Event Date: February 12, 2021 at 16:50.
Figure 14.
Performance Analysis of EMSR442-0-6 Flood Event in Norway. Event Date: June 13, 2020 at 10:30.
Figure 14.
Performance Analysis of EMSR442-0-6 Flood Event in Norway. Event Date: June 13, 2020 at 10:30.
Figure 15.
Performance Analysis of EMSR280-4-17 Flood Event in Sweden. Event date: April 21, 2018 at 15:15.
Figure 15.
Performance Analysis of EMSR280-4-17 Flood Event in Sweden. Event date: April 21, 2018 at 15:15.
Overall, the model has demonstrated great performance. An example is shown in flood event EMSR442-0-6. Illustrated in
Figure 14, this event highlights a section of water missing from the Optical (SCL) ground truth due to cloud cover obstruction. However, given SAR capacity of penetrating through clouds, the model was able to accurately detect those flood areas.
Figure 16.
RGB image of the EMSR442-0-6 flood event, from the Optical Sentinel-2 Satellite. It demonstrates the presence of cloud cover.
Figure 16.
RGB image of the EMSR442-0-6 flood event, from the Optical Sentinel-2 Satellite. It demonstrates the presence of cloud cover.
Another notable performance is seen in the flood event EMSR280-4-17. As illustrated in
Figure 15, this event demonstrates the model’s capability to identify and exclude snow-covered areas. In SAR images, regions covered in snow closely resemble water, making them indistinguishable to the human eye. However, the model successfully distinguished these areas avoiding the misclassification of snow as water.
Figure 17.
RGB image of the EMSR280-4-17 flood event, from the Optical Sentinel-2 Satellite. It demonstrates the presence of snow
Figure 17.
RGB image of the EMSR280-4-17 flood event, from the Optical Sentinel-2 Satellite. It demonstrates the presence of snow
An example where the model underperforms the Optical, SCL ground truth, is given by the flood event in EMSR258-13-1. As shown in
Figure 10, it’s evident that the model predictions are less accurate along the borders, likely due to the lower resolution of SAR resolution compared to the Optical ground truth.
7.2. Results on MMFlood-CD Dataset
In the following section is shown a visual comparison of the model prediction on flood events included in the MMFlood-CD dataset, using the trained model on SAR-Optical and the ground truth provided by EMS. Here’s a detailed explanation of the labels in the images:
EMS Prediction - Ground Truth EMS to be compared against our model.
SAR Model Prediction - The final flood prediction of our SAR Model.
Pre-Flood Prediction - The pre-flood model predictions of water.
Post-Flood Prediction- The post-flood model prediction of water.
SAR Image Post-Flood - The SAR raw image post-flood.
SAR Image Pre-Flood - The SAR raw image pre-flood.
Note that a lower IoU indicates that the EMS ground truth and the SAR-Optical Model predictions don’t closely match, either due to our SAR model or the EMS ground truth error.
Figure 18.
Comparative Analysis of the EMSR265-14-6 Flood Event in France. Event Date: January 23, 2018, at 19:17. Acquisition Date: January 25, 2018, at 05:59.
Figure 18.
Comparative Analysis of the EMSR265-14-6 Flood Event in France. Event Date: January 23, 2018, at 19:17. Acquisition Date: January 25, 2018, at 05:59.
Figure 19.
Comparative Analysis of the EMSR332-2-1 Flood Event in Italy. Event Date: November 1, 2018, at 21:08. Acquisition Date: November 2, 2018 at 05:18.
Figure 19.
Comparative Analysis of the EMSR332-2-1 Flood Event in Italy. Event Date: November 1, 2018, at 21:08. Acquisition Date: November 2, 2018 at 05:18.
Figure 20.
Comparative Analysis of the EMSR314-2-2 Flood Event in Nigeria. Event Date: September 18, 2018, at 21:27. Acquisition Date: September 22, 2018 at 17:44.
Figure 20.
Comparative Analysis of the EMSR314-2-2 Flood Event in Nigeria. Event Date: September 18, 2018, at 21:27. Acquisition Date: September 22, 2018 at 17:44.
Figure 21.
Comparative Analysis of the EMSR-319-3-4 Flood Event in Tunisia. Event Date: September 29, 2018, at 17:42. Acquisition Date: October 3, 2018 at 17:11.
Figure 21.
Comparative Analysis of the EMSR-319-3-4 Flood Event in Tunisia. Event Date: September 29, 2018, at 17:42. Acquisition Date: October 3, 2018 at 17:11.
Figure 22.
Comparative Analysis of the EMSR-342-6-1 Flood Event in Australia. Event Date: February 1, 2019, at 04:45. Acquisition Date: February 5, 2019 at 19:43.
Figure 22.
Comparative Analysis of the EMSR-342-6-1 Flood Event in Australia. Event Date: February 1, 2019, at 04:45. Acquisition Date: February 5, 2019 at 19:43.
Figure 23.
Comparative Analysis of the EMSR-399-0-0 Flood Event in Vietnam. Event Date: October 28, 2019. Acquisition Date: October 28, 2019 at 22:45.
Figure 23.
Comparative Analysis of the EMSR-399-0-0 Flood Event in Vietnam. Event Date: October 28, 2019. Acquisition Date: October 28, 2019 at 22:45.
The model has demonstrated great performance in this qualitative evaluation. Despite the inability to directly compare its performance with an alternative ground truth, we can easily discern areas where the EMS model might fail by examining both pre- and post-flood SAR images.
Figure 18,
Figure 19, and
Figure 21 are possible examples where the EMS model may have misses certain flood areas, detecting them only partially compared to our model. In
Figure 20, EMS model appears to miss a rapid flood event that our model correctly detects.
Figure 21 is the only instance where both the EMS and our SAR model produce matching predictions.