1. Introduction

2. Materials and Methods

2.2. U²-Net Retrofit

2.2.1. U²-Net Network and Convolutional Block Attention Module (CBAM)

The U²-Net network has the following advantages: firstly, the network is a two-layer, nested, U-shaped structure that does not use a pre-trained backbone model for image classification, and the model weights are trained from the training set; secondly, the new architecture allows the network to go deeper and obtain high-resolution features without substantially increasing memory and computational costs. In the bottom layer, a new RSU is designed to extract intra-stage, multi-scale features without reducing the feature mapping resolution. In the top layer, there is a U-Net-like structure, where each stage is populated by a ReSidual U-block. The feature map is down-sampled twice after each Encoder and up-sampled before passing through each Decoder twice before passing through each decoder. The ReSidual U-blocks in the network can be divided into two categories: Encoder1–Encoder4 and Decoder1–Decoder4. They use the same modular structure of RSU-L, only the depth (L) is different. Taking L = 7 as an example, the maximum downsampling time was 32 times. When Encoder5–6 and Decoder 6 used the module RSU-F, the main difference between the RSU-L and RSU-F structures was that there was no more downsampling (purple part). Because the image size has been sufficiently downsampled, much contextual information will be lost when the image size is decreased again, which affects image segmentation. Therefore, RSU-4F is used in the deep layer, and the module decreases the downsampling part and uses multi-layer expansion convolution to enlarge the receptive field. The right side shows the output of the fusion of the features of each layer after upsampling to obtain the segmented image as large as the original image, and the feature map of each layer is then convolved by channel number 1 to obtain the final output.

The CBAM contains the Channel Attention Mechanism (Channel Attention Module, CAM) and Spatial Attention Mechanism (Spatial Attention Module, SAM), with two sub-modules, (Figures 1 and 2, respectively). The Channel Attention Mechanism is a one-dimensional vector obtained by compressing a feature map in the spatial dimension. Mean and maximum pooling operations were used to compress the spatial and channel dimensions. The mean and maximum pooling aggregate the spatial information of the feature map, which is then mapped to the weights of each channel through the convolutional layer or fully connected layer. The original features are multiplied with this vector in the channel dimension to obtain the weighted feature map, and the size of the weights reflects the degree of relevance and importance of the features in the layer (channel) for the key information. CBAM contains a spatial attention mechanism and a channel attention mechanism, and the structure is shown in Figure 3. The CBAM is a simple yet effective attention module for feedforward CNNs. The module sequentially infers attention maps along two separate dimensions, channel and spatial, and then multiplies the attention maps with the input feature map for adaptive feature refinement. It can be seamlessly integrated into any CNN architecture with negligible overhead and is end-to-end trainable along with the base CNNs to improve the accuracy of the CNN [25].

Li et al. proposed an improved YOLOv4-based pavement damage detection model. The model improves the saliency of pavement damage by introducing a convolutional block attention module (CBAM) to suppress background noise and explores the influence of the embedding position of the CBAM module in the YOLOv4 model on detection accuracy. The results indicate that embedding CBAM into the neck and head modules can effectively improve the detection accuracy of the YOLOv4 model [26].

Sun et al. proposed an attention-based feature pyramid module (AFPM), which integrates the attention mechanism based on a multi-level feature pyramid network to efficiently and pertinently extract high-level semantic features and low-level spatial structure features, thus improving the accuracy of instance segmentation [27]. According to the above findings, the Convolutional Block Attention Module (CBAM) can independently learn the importance of each channel and space feature, recalibrate the channel and space features, and improve image classification performance [28]. Therefore, in this study, CBAM was added to multiple locations of U²-Net.

2.2.2. SCM-RSU and SC-U²-Net Network

The SC-U²-Net improvement was divided into two parts. First, the SCM-RSU was used instead of the RSU (Figure 4). In the downsampling stage, SCM-RSU used the residual structure, and the output of the upper layer skipped the intermediate convolution as the input of the lower convolution directly to reduce the loss of features in the downsampling process. The U²-Net network used multi-scale expansion convolution only in RSU-4F, which was located in the deep layer of the network, to extract richer features at different scales. The SCM-RSU deep perception module was changed to multi-scale expansion convolution, where the input feature maps were subjected to 1×1 convolution with an expansion factor (d) of 1, 3×3 convolution with a d of 1, and 3×3 expansion with a d of 2, respectively. Subsequently, the outputs of the three convolutions were stitched in the channel dimension, and finally, channel fusion was performed using a 1×1 convolution. In the decoding stage, the output of the same depth encoding and that of the previous decoding depth were processed by the CBAM attention mechanism and inputted.

Secondly, in the overall framework, multiple attention mechanisms were added (Figure 5). The feature map of the encoder output feature map decoder upsampling after using CBAM processing and then input to the corresponding Decoder; the Decoder output feature map enables CBAM processing and then convolution operation to obtain the feature map with channel number 1; each feature map upsampled to the input image. After upsampling each feature map to the size of the input image, stitching was performed to obtain an output with a channel number of 6 and the same size as the original image, and then CBAM was used to process the output. Finally, the output with a channel number of 1 was obtained using 1×1 convolution.

2.3. Sea Ice Image Segmentation Experimental Setup

In order to improve the accuracy of sea ice image segmentation, this paper improves two aspects of data augmentation and the U²-Net network structure. In terms of data augmentation, five data sets were constructed using various data augmentation methods (such as noisy data augmentation), and the model weights were obtained by training the U²-Net network with the five data sets respectively, and the accuracy was checked using the test set. In terms of network structure improvement, a multi-layer CBAM attention mechanism and multi-scale expanded convolution were added to U²-Net to enhance the network feature extraction ability, and the accuracy and generalization ability of the improved model were checked using the test set. The SC-U²-Net network model was constructed by adding a multi-layer, CBAM attention mechanism and multi-scale inflation convolution to enhance the network feature extraction ability. The accuracy and generalization ability of the improved model were tested using test data (Figure 6).

Figure 6. Flow chart of sea ice segmentation.

Figure 6. Flow chart of sea ice segmentation.

The initial learning rate was set to 0.001, and this study used the learning rate warmup. The learning rate warmup [29] and cosine annealing [30] were used to combine the learning rate change strategy. The weights of the neural network were randomly initialized at the beginning of training, and the warmup gradually increased the learning rate from low to high to ensure a good convergence of the network. When the gradient descent algorithm was used to optimize the objective function, cosine annealing reduced the learning rate of the cosine function as it approached the global minimum of the loss value, making the model as close to the optimal value as possible. The loss function is a binary cross-entropy loss function (Equation (7)), where

$\stackrel{\wedge }{y}$ is the result of the model prediction sample and $y$ is the sample label.

The loss function of U²-Net is calculated in Equation 8, which contains two parts:

${\displaystyle \sum _{m=1}^M{w}_{side}^{(m)}{l}_{side}^{(m)}}$ represents the sum of the cross-entropy of the output results of different depth Decoder and GT images, and

$w_{fuse}{l}_{fuse}$ is the cross-entropy loss of the final output and GT images after multichannel fusion.

$$Loss=-(y\cdot \log (\stackrel{\wedge }{y})+(1-y)\cdot \log (1-\stackrel{\wedge }{y})$$

(7)

$$L={\displaystyle \sum _{m=1}^M{w}_{side}^{(m)}{l}_{side}^{(m)}+w_{fuse}{l}_{fuse}}$$

(8)

The validation accuracy of the model increased with an increase in epochs, and the model started to converge when the epoch reached 360 when the U²-Net network was trained using training set 1. To balance model accuracy and training efficiency, the epoch was set to 360 in this study. Five test sets were used for training, in which the epochs of training datasets 1 and 2 were set to 360, the epochs of datasets 3, 4, and 5 were set to 90, and the epochs of datasets 3, 4, and 5 were set to 90.

The accuracy evaluation metrics selected in this study are the Intersection over Union (Iou), F1-Score, and recall. If sea ice is called a positive case (Positive), non-sea ice is called a negative case (Negative), and the classifier predicts correctly is noted as True (True) and incorrectly predicts as False (False), and the four basic terms are combined with each other to form the four basic elements of the confusion matrix, true case (TP), false positive case (FP), false negative case (FN), and true negative case (TN), then Iou, F1-Score, Recall are calculated by Equations (9)–(11).

$$IoU=\frac{TP}{TP+FP+FN}$$

(9)

$$F_1=\frac{2TP}{2TP+FN+FP}$$

(10)

$$\mathrm{Re}call=\frac{TP}{TP+FN}$$

(11)

To distinguish the network model from the dataset used for training, the naming rules for the network and model weights were as follows: The U²-Net network was trained using dataset i, and the names of the model weights were obtained as U²-Net-i.

3. Results

3.1. Data Augmentation Experiments

The average cross-merge ratio of U²-Net-1 tested on the test set was 0.842, the average recall was 0.897, and the average F1-Score was 0.889. U²-Net-1 predicted noise-free sea ice images well, but there was overfitting of the network weights, and weak noise interfered with the segmentation, as shown in Figure 7d. The recall distribution curves of U²-Net-1 predicting noise-free images and weakly noisy test set images are shown in Figure 8.

Figure 7. U²-Net-1 predicted noise image.

Figure 7. U²-Net-1 predicted noise image.

Figure 8. U²-Net-1 recall distribution probability curve.

Figure 8. U²-Net-1 recall distribution probability curve.

To further study the effect of noise, we trained U²-Net using different training sets to obtain the corresponding model weights (U²-Net-1 was the weight obtained by training U²-Net on training set 1). We performed a segmentation of the test set containing different noise levels using U²-net trained from different training sets.The average IoU, average recall, and average F1-Score of the test set with different noise levels were counted (Table 2). The curves of average IoU, average recall average F1-Score with noise level of the test set were made according to Table 2, as shown in Figure 9, Figure 10 and Figure 11 respectively.

Figures 12 and 13 show the results of U²-Net-2 and model U²-Net-3 predicting different levels of test set. Model 2 could predict images with low noise, but the predicted noise limit was approximately 0.15; U²-Net-3 had a stronger generalization ability than model U²-Net-2 and was able to segment images with more severe noise pollution.

To further verify the usefulness of multiple data augmentation methods, the transformations (e.g., rotation and mirroring) were performed on the test set simultaneously to obtain a test set with different geometric and radiometric characteristics from the original test set in this study. The experimental accuracies are shown in Table 3 using the transformed test sets for U²-Net-3, U²-Net-4, and U²-Net-5.

The accuracy evaluation indices of U²-Net-4 were higher than those of U²-Net-1 when predicting the noiseless test set (Figure 14). The accuracy evaluation indices of U²-Net-5 were higher than those of U²-Net-3 when predicting the test set with 0.2 noise level (Figure 15).

3.2. SC-U²-Net Network

The SC-U²-Net-1 and U2-Net-1 network was trained using the same dataset(Training dataset 1). The accuracies of SC-U²-Net-1 and U²-Net-1 were tested using the test set, and the accuracy of SC-U²-Net-1 and U²-Net-1 tests are shown in Table 4.

Figure 16 shows some experimental results, where figures (a) to (h) show the sea ice images of different regions, and (1) to (4) show the original image, labeled image, U²-Net segmentation result, and SC-U²-Net segmentation result, respectively. U²-Net was less effective in segmenting some sea ice, such as the broken ice area at the land edge. SC-U²-Net performed better and could extract the outline and some details of sea ice as shown in (a) of Figure 16.

Figure 16. Comparison of SC-U²-Net and U²-Net segmentation results.

Figure 16. Comparison of SC-U²-Net and U²-Net segmentation results.

The accuracy of SC-U²-Net-5 was examined using an extended test set (1500 images) and compared to U²-Net-5. The IoU, F1-Score and recall of U²-Net-5 and SC-U²-Net-5 on the extended test set are shown in Table 5.

Using the transformed test sets to test U²-Net-1 and SC-U²-Net-5, the statistics of IoU, F1-Score, and recall per image for each test set were calculated (Table 3; Figure 17). SC-U²-Net-5 had a much better segmentation effect than U²-Net-1 on each test set (Table 6). The results in Table 6 show that the simultaneous use of data augmentation and network improvements can improve the accuracy and generalization of the model.

Figure 17. U²-Net-1 and SC-U²-Net-5 accuracy rating box line diagram.

Figure 17. U²-Net-1 and SC-U²-Net-5 accuracy rating box line diagram.

4. Discussion

5. Conclusions

References

1. Liu, M. Research on the Sea Ice Classification and Thickness Detection with High-Resolution and Polarimetric SAR Data. College of Information and Control Engineering China University of Petroleum (EastChina), 2016.
2. Liang, J. Maritime Strategic Game in Arctic Region: China and India. S. Asian Stud. Q. 2019, 24–33. [Google Scholar]
3. Cheng, X.; Chen, Z.; Hui, F. Current Status and Outlook of Space-Based Remote Sensing Observation in Polar Regions of China. Sci. Technol. Foresight 2022, 1, 183–197. [Google Scholar]
4. Wang, L. Research on Sea Ice Inversion Algorithm Based on Satellite Remote Sensing Data. Nanjing University of Information Engineering, 2021.
5. Li, Y. Research on Sea Ice Detection Method Based on the Decomposition of Mixed Pixels; Wuhan University, 2020.
6. Li, P. Combining Active Learning and Semi-supervised Learning for Sea Ice Image Classification; Shanghai Ocean University, 2018.
7. Han, Y.; Li, J.; Zhang, Y.; Hong, Z. Hyperspectral Sea Ice Detection Using Improved Similarity Metric. Remote Sens. Inf. 2018, 33, 76–85. [Google Scholar]
8. Zhou, J.; Lu, P.; Wang, Q.; Xie, F.; Li, R. Research on Automatic Detection Algorithm Based on Video Image Acquisition for Ice Surface Features. Hydro Science Cold Zone Eng. 2021, 4, 60–65. [Google Scholar]
9. Yu, Z. Sea Ice Classification of Remote Sensing Image Based on Neighborhood Relationships; CUPB (East China),2019.
10. Dowden, B.; De Silva, O.; Huang, W.; Oldford, D. Sea Ice Classification via Deep Neural Network Semantic Segmentation. I.E.E.E. Sens. J. 2021, 21(10), 11879–11888. [Google Scholar] [CrossRef]
11. Han, Y.; Cui, P.; Zhang, Y.; Zhou, R.; Yang, S.; Wang, J. Remote Sensing Sea Ice Image Classification Based on Multilevel Feature Fusion and Residual Network. Math. Probl. Eng. 2021, 2021, 1–10. [Google Scholar] [CrossRef]
12. Shi, Q. Homologous and Heterologous Remote Sensing Sea Ice Classification Based on Deep Learning; Shanghai Ocean University, 2022.
13. Fang, Y. Research on Sea Ice Area Identification Based on MODIS Satellite Remote Sensing Images; Qingdao University of Science and Technology, 2021.
14. Cui, Y.; Zou, B.; Han, Z.; Shi, L.; Liu, S. Application of Convolutional Neural Networks in Satellite Remote Sensing Sea Ice Image Classification: A Case Study of Sea Ice in the Bohai Sea. Acta Oceanol. Sin. 2020, 42, 100–109. [Google Scholar] [CrossRef]
15. Han, Y.; Gao, Y.; Zhang, Y.; Wang, J.; Yang, S. Hyperspectral Sea Ice Image Classification Based on the Spectral-Spatial-Joint Feature with Deep Learning. Remote Sens. 2019, 11(18), 2170. [Google Scholar] [CrossRef]
16. Han, Y.; Liu, Y.; Hong, Z.; Zhang, Y.; Yang, S.; Wang, J. Sea Ice Image Classification Based on Heterogeneous Data Fusion and Deep Learning. Remote Sens. 2021, 13(4), 592. [Google Scholar] [CrossRef]
17. Zhang, T.; Yang, Y.; Shokr, M.; Mi, C.; Li, X.; Cheng, X.; Hui, F. Deep Learning Based Sea Ice Classification with Gaofen-3 Fully Polarimetric SAR Data. Remote Sens. 2019, 13(8), 1452. [Google Scholar] [CrossRef]
18. Wen, C.; Zhai, M.; Lei, R.; Xie, T.; Zhu, J. Automated Identification of Landfast Sea Ice in the Laptev Sea from the True-Color MODIS Images Using the Method of Deep Learning. Remote Sens. 2023, 15(6), 1610. [Google Scholar] [CrossRef]
19. Liu, Z.; Zhang, S.; Liu, Y.; Luo, C.; Li, M. Data Augmentation Method Based on Image Gradient. J. Appl. Sci. 2023, 39, 302–311. [Google Scholar]
20. Huang, Z.; Liu, X.; Shi, y.; Lin, C. Small Object Detection in Road Science Base on Data Augmenttation. Journeal Wuhan Univ. Technol. 2022, 44, 79–87. [Google Scholar]
21. Yang, Z.; Yang, Y.; Cang, S.; Li, Y.; H.; Y.; Zhang, F.; Wu, G. Data Augmentation Method of Ship Remote Sensing Images Based on GAN. Appl. Sci. Technol. 2022, 49, 8.
22. Lin, C.; Shan, C.; Zhao, G.; Yang, Z.; Peng, J.; Chen, S.; Huang, R.; Li, Z.; Yi, X.; Du, J.; Li, S.; Luo, H.; Fan, X.; Chen, B. Review of Image Data Augmentation in Computer Vision. J. Front. Comput. Sci. Technol. 2021, 15, 583–611. [Google Scholar]
23. Lee, D.-U.; Villasenor, J. D.; Luk, W.; Leong, P. H. W. A Hardware Gaussian Noise Generator Using the Box-Muller Method and Its Error Analysis. In I.E.E.E. Trans. Comput. 2006, 55(6), 659–671. [Google Scholar] [CrossRef]
24. Qin, X.; Zhang, Z.; Huang, C.; Dehghan, M.; Zaiane, O. R.; Jagersand, M. U²-Net: Going Deeper with Nested U-Structure for Salient Object Detection. Pattern Recognit. 2020, 106, 107404. [Google Scholar] [CrossRef]
25. Woo, S.; Park, J.; Lee, J.; Kweon, I. S. Cbam: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV); 2018; pp. 3–19. [Google Scholar] [CrossRef]
26. Li, L.; Fang, B.; Zhu, J. Performance Analysis of the YOLOv4 Algorithm for Pavement Damage Image Detection with Different Embedding Positions of CBAM Modules. Appl. Sci. 2022, 12(19), 10180. [Google Scholar] [CrossRef]
27. Sun, Y.; Gao, W.; Pan, S.; Zhao, T.; Peng, Y. An Efficient Module for Instance Segmentation Based on Multi-level Features and Attention Mechanisms. Appl. Sci. 2021, 11(3), 968. [Google Scholar] [CrossRef]
28. Zhang, L.; Duan, L. Cross-Scenario Transfer Diagnosis of Reciprocating Compressor Based on CBAM and ResNet. J. Intell. Fuzzy Syst. 2022, 43(5), 5929–5943. [Google Scholar] [CrossRef]
29. Xiong, R.; Yang, Y.; Di He; Zheng, K.; Zheng, S.; Xing, C.; Zhang, H.; Lan, Y.; Wang, L.; Liu, T. On Layer Normalization in the Transformer Architecture. ICML. 2020.
30. Loshchilov, I.; Hutter, F. SGDR: Stochastic Gradient Descent with Warm Restarts. ICLR 2017, 1–16. [Google Scholar]