3.1. Loss Functions

• The Generalized Dice Loss $L_{GD}(Y,T)$ is a multiclass variant of the Dice Loss.
• The Tversky Loss $L_T(Y,T)$ is a weighed version of the Twersky index designed to deal with unbalanced classes.
• The Focal Tversky Loss $L_{FT}(Y,T)$ is a variant of the Twersky loss where a modulating factor is used to ensure that the model focuses on hard samples instead of properly classified examples.
• The Focal Generalized Dice Loss $L_{FGD}(Y,T)$ is the focal version of the Generalized Dice Loss.
• The Log-Cosh Dice Loss $L_{lcGD}(Y,T)$ is a combination of the Dice Loss and the Log-Cosh function, applied with the purpose of smoothing the loss curve.
• The Log-Cosh Focal Tversky Loss $L_{lcFT}(Y,T)$ is based on the same idea of smoothing, here applied to the Focal Tversky Loss.
• The SSIM Loss $L_S(Y,T)$ is obtained from the Structural similarity (SSIM) index, usually adopted to evaluate the quality of an image.
• The MS-SIM Loss $L_{MS}(Y,T)$ is a variant of $L_S(Y,T)$ defined using the Multiscale structural similarity (MS-SSIM) index.
• The losses described above can be combined in different ways:

  –

  $Com{b}_1(Y,T)=L_{FGD}(Y,T)+L_{FT}(Y,T)$,

  –

  $Com{b}_2(Y,T)=L_{lcGD}(Y,T)+L_{FGD}(Y,T)+L_{lcFT}(Y,T)$,

  –

  $Com{b}_3(Y,T)=L_S(Y,T)+L_{GD}(Y,T)$,

  –

  $Com{b}_4(Y,T)=L_{MS}(Y,T)+L_{FGD}(Y,T)$.

• The Boundary Enhancement Loss ($L_{BE}$) explicitly focuses on the boundary areas during training. The Laplacian filter $\mathcal{L}(·)$ is used to generate strong responses around the boundaries and zero everywhere else. We gather Dice Loss, Boundary Enhancement loss and the Structure Loss together, weighted appropriately: $L_{DiceBES}={\lambda }_1{L}_{Dice}+{\lambda }_2{L}_{BE}+L_{Str}$. We set ${\lambda }_1=1$ and ${\lambda }_2=0.01$
• The Structure Loss is a combination of the weighted Intersect over Union ($L_{wIoU}$) and the weighted binary-crossed entropy loss $L_{wbce}$. We refer the reader to [10] for details. The weights in this loss function are determined by the importance of each pixel, which is calculated from the difference between the center pixel and its surrounding pixels. To give more importance to the binary-crossed entropy loss, we use a weight of 2 for it: $L_{STR}=L_{wIoU}+2L_{wbce}$.

3.2. Data Augmentation

• Data Augmentation 1 (DA1) [24] is obtained through horizontal flip, vertical flip, and 90° rotation;
• Data Augmentation 2 (DA2) [24] consists of 13 operations, some changing the color of an image and some changing its shape.
• Data Augmentation 3 (DA3) is a variant of the approach used in [12]. It consists in using multi-scale strategies (i.e., 1.25, 1, 0.75) to alleviate the sensitivity of the network to scale variability. Simultaneously, random perspective technology is adopted to process the input image with a probability of 0.5, together with random color adjustment with a probability of 0.2 for data augmentation. While DA1 and DA2 do not include randomness, DA3 uses a different training set for each network. The application of this data augmentation technique substantially amplifies result variability within the network, consequently fostering greater diversity among ensemble constituents.

3.3. Performance Metrics

3.4. Datasets and Testing Protocols

4.1. Experiments: DeepLabV3+

• initial learning rate = 0.01;
• number of epoch = 10 or 15 (it depends on data augmentation: see below);
• momentum = 0.9;
• L2Regularization = 0.005;
• Learning Rate Drop Period = 5;
• Learning Rate Drop Factor = 0.2;
• shuffle training images at every epoch;
• optimizer = SGD (stochastic gradient descent).

• ERN18(N) is an ensemble of N RN18 networks trained with DA1.
• ERN50(N) is an ensemble of N RN50 networks trained with DA1.
• ERN101(N) is an ensemble of N RN101 networks trained with DA1.
• E101(10) is an ensemble of 10 RN101 models trained with DA1 and five different loss functions. The final fusion is determined by the formula: $2\times L_{GD}+2\times L_T+2\times Comb1+2\times Comb2+2\times Comb3$, where $2\times L_x$ indicates two RN101 models trained using the loss function $L_x$.
• EM(10) is a similar ensemble, but the two networks using the same loss (as in E101(10), the five losses are $L_{GD}$, $L_T$, $Comb1$, $Comb2$, $Comb3$) are trained once using DA1 and once using DA2.
• EM2(10) is similar to the previous ensemble, but $LDiceBES$ is used instead of $L_T$.
• In EM2(5)_DAx, five RN101 networks are trained using the loss of EM2(10). All five networks are trained using data augmentation DAx.
• EM3(10) is similar to the previous ensemble, but $L_{STR}$ is used as a loss function.

• Among stand-alone networks, RN101 obtains the best average performance, but in RIBS (small training set) it works worse than the others. This probably happens because it is a larger network with respect to RN18 and RN50, thus requiring a larger training set for better tuning.
• ERN101(10) always outperforms RN101(1).
• E101(10) outperforms ERN101(10) with a p-value of 0.0078 (Wilcoxon signed rank test) and EM(10) outperforms E101(10) with a p-value of 0.0352. For the sake of space, we have not reported the performance obtained from individual losses. In any case, there is no winner, the various losses lead to similar performance.
• EM3(10) obtains the highest average performance but the p-value is quite high: it outperforms EM(10) with a p-value of 0.1406 and EM2(10) with a p-value of 0.2812.
• There is no statistical difference between the performance of EM2(5)_DA1 and EM2(5)_DA2.

4.2. Experiments: Combining Different Topologies

• LRa: ${10}^{-4}$.
• LRb: $5·{10}^{-4}$ decaying to $5·{10}^{-5}$ after 10 epochs.
• LRc: $5·{10}^{-5}$ decaying to $5·{10}^{-6}$ after 30 epochs.

• Fusion: the combination of all the nets varying DA and LR strategy.
• Baseline Ensemble: fusion between 9 networks (same size of Fusion) obtained retraining DA3-LRc nine times.
• Previous: the best ensemble previously reported among [34,35,36], where SOTA performance was obtained for the given segmentation problem.

• Fusion obtains the best performance, outperforming (on average) the stand-alone approaches and previous ensemble.
• There is no clear winner among the different data augmentation approaches and learning rate strategies.
• The proposed Fusion always improves the Baseline Ensemble except in CAMO. In this dataset there is a significant difference in performance between LRc and the other learning strategy, combining only the 3 networks based on LRc (i.e., using the three data augmentations coupled with LRc) both HS and PVT get a Dice of 0.830, outperforming the Baseline Ensemble.

• Ens1: EM3(10) ⊖ Fusion(FH) ⊖ Fusion(PVT) ⊖ Fusion(HSN).
• Ens2: Fusion(FH) ⊖ Fusion(PVT) ⊖ Fusion(HSN).
• Ens3: Fusion(PVT) ⊖ Fusion(HSN).