1. Introduction

2. Related Works

3. Proposed Skin Lesion Segmentation Models: UCM-NetV2 and BNN-UCM-NetV2

• UCM-NetV2 is a hybrid-based architecture that combines Convolutional Neural Networks (CNN) and Multilayer Perceptions (MLP) to enhance feature learning. Utilizing new proposed group loss functions, our method surpasses existing lightweight techniques in skin lesion segmentation. UCM-NetV2 is optimized on top of the previous generation, UCM-NetV2, for hardware-limited computing platforms. By focusing on maximizing the model's prediction performance, we proposed a new loss function for UCM-NetV2.
• The BNN-UCM-NetV2 network extends the UCM-NetV2 model by incorporating binary neural networks (BNNs) for Binary Efficiency for tight resource-constrained devices: In BNNs, weights and activations are binarized into 1-bit values, drastically reducing the model size and computational demands. We especially proposed a new loss function for BNN-UCM-NetV2 to power the learning ability. We are the first to explore combining binary neural networks with U-shape networks for skin cancer segmentation.

3.1. UCM-NetV2 Network Structure

Network Structure Design:Figure 3 provides a detailed overview of the structural framework of UCM-NetV2, an advanced architecture with a distinctive U-shape design. This design builds upon the first generation of UCM-Net, incorporating both a down-sampling encoder and an up-sampling decoder to create a powerful network for skin lesion segmentation. The network is composed of six stages of encoder-decoder units, each with channel capacities of 8, 16, 24, 32, 48, and 64, respectively.

Convolution Block: In the initial encoder-decoder stage, our design uses a standard convolution layer with a 3x3 filter, effectively capturing spatial relationships within the input features. This kernel size is particularly beneficial for maintaining spatial integrity in the network's initial layers. For stages 2 through 6, we utilize a 1x1 convolution layer, significantly reducing the number of learnable parameters and the overall computational load, thus enhancing model efficiency. Each stage combines the 1x1 convolutional layer with UCM-Net blocks, contributing to advanced feature extraction and processing capabilities.

UCM-NetV2 Block: The 2nd-6th stages primarily utilize the UCM-NetV2 block for feature learning. Figure 4 shows the differences between the UCM-Net and UCM-NetV2 blocks. Compared to its predecessor, the UCM-NetV2 block enhances feature learning by incorporating forward and backward paths with normalization, linear operations, concatenation, and additional convolutional layers. This block features twice the number of linear operations and an additional convolutional layer, reducing computational costs. With more residual operations, the UCM-NetV2 block enables more complex feature extraction and improved performance. The PyTorch-style pseudocode in Figure 5 outlines the sequence of operations for feature learning.

Loss Function for the UCM-NetV2 Network: In our solution, we designed a new group loss function similar to those used in TransFuse [24], EGE-UNet [29], and our previous work, UCM-Net [19]. However, unlike these previous works, our proposed base loss function incorporates binary cross-entropy (BCE) (1), Dice Loss (2), and Squared Dice Loss (3) components. These components calculate the loss from the scaled layer masks at different stages than the ground truth masks. Equations (5) and (6) present the stage loss for different layers and the output loss in the output layer, respectively, using BCE and Dice loss components. Additionally, we introduce a dynamic coefficient for the output loss, equation (6), based on the training epoch, adding more weight to the loss during the earlier training stages. This approach ensures that the model focuses more on learning crucial features for the final output segmentation early in the training process. The advantages of using different loss functions are as follows:

• Binary Cross-Entropy Losses (BCE): Widely used in classification tasks, including biomedical image segmentation, these losses are highly effective for pixel-level segmentation.
• Dice Loss: Commonly used in biomedical image segmentation, Dice loss addresses class imbalance by focusing on the overlap between predicted and true regions.
• Squared Dice Loss: Further enhances the Dice loss by emphasizing the contribution of well-predicted regions, thereby improving stability and performance.

$$BCE=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left[y_i\mathit{log}\left(p_i\right)+\left(1-y_i\right)\mathit{log}\left(1-p_i\right)\right]$$

(1)

where N is the total number of pixels (for image segmentation) or elements (for other tasks), ${\mathrm{y}}_{\mathrm{i}}$ is the ground truth value, and ${\mathrm{p}}_{\mathrm{i}}$ is the predicted probability for the i-th element.

$$\mathrm{Dice}\mathrm{Loss}=1-{\displaystyle \frac{2\times {\sum }_{i=1}^N\left(p_i\cdot y_i\right)+\mathrm{smooth}}{{\sum }_{i=1}^N{p}_i+{\sum }_{i=1}^N{y}_i+\mathrm{smooth}}}$$

(2)

where smooth is a small constant added to improve numerical stability.

$$\mathrm{Squared}\mathrm{Dice}\mathrm{Loss}=1-{\displaystyle \frac{2\times {\left({\sum }_{i=1}^N\left(p_i\cdot y_i\right)\right)}^2+\mathrm{smooth}}{{\left({\sum }_{i=1}^N{p}_i\right)}^2+{\left({\sum }_{i=1}^N{y}_i\right)}^2+\mathrm{smooth}}}$$

(3)

Which represents an enhancement over the standard Dice loss by emphasizing the squared terms of intersections and union.

$$\mathrm{Base}\text{\_}\mathrm{loss}=\mathrm{BCE}+\mathrm{Dice}\mathrm{Loss}+\mathrm{Squared}\mathrm{Dice}\mathrm{Loss}$$

(4)

Equations (1), (2), and (3) define the base loss function (4) for our proposed model, incorporating the Dice loss and Squared Dice loss components. To fully understand how different combinations of loss functions affect the trained model's performance, we add an ablation experiment in the next section.

$$Los{s}_{Stage}=\mathrm{Base}\text{\_}\mathrm{loss}\left(StagePred,Target\right)$$

(5)

$${\mathrm{Loss}}_{\mathrm{Output}}=(2-\sin \left({\displaystyle \frac{curren{t}_{epoch}}{tota{l}_{epoch}}}\ast {\displaystyle \frac{Pi}{2}}\right))\ast \mathrm{Base}\text{\_}\mathrm{loss}\left(\mathrm{OutputPred},\mathrm{Target}\right)$$

(6)

$$Group\_Loss=Los{s}_{Output}+\sum _{i=1}^5{\mathsf{\lambda }}_i\times Los{s}_{Stag{e}_i}$$

(7)

where ${\lambda }_i$ is the weight for different stages. In this paper, we set ${\lambda }_i$to 0.1, 0.2, 0.3, 0.4, and 0.5 based on the i-th stage, as illustrated in Figure 4. Equation (7) is our proposed group loss function that calculates the loss from the scaled layer masks in different stages compared to the ground truth masks. Equations (5) and (6) present the stage loss in different layers and the output loss in the output layer.

Figure 5. UCM-NetV2 Block Pseudocode.

Figure 5. UCM-NetV2 Block Pseudocode.

Figure 6. BNN-UCM-NetV2 network.

Figure 6. BNN-UCM-NetV2 network.

3.2. BNN-UCM-NetV2 Network Structure

Deep learning (DL) has revolutionized the development of intelligent systems with numerous real-world applications. However, deploying DL on resource-constrained devices with limited computational power and energy remains challenging. Binary Neural Networks (BNNs) offer a promising solution in this context. BNNs are a type of neural network where activations and weights are binarized to 1-bit values in all hidden layers, providing a highly compact version of a Convolutional Neural Network (CNN) [21]. BNNs' ability to compress 32-bit activations and weights to 1-bit values significantly reduces storage, computation costs, and energy consumption, making them ideal for deployment on tiny, resource-limited devices. Key advantages of BNNs are

• Compression: BNNs compress 32-bit floating-point values into 1-bit representations, leading to significant reductions in model size.
• Computational Efficiency: Binary operations (e.g., XNOR) replace computationally expensive floating-point operations, resulting in faster inference.
• Energy Savings: Lower computational demands translate to reduced energy consumption, vital for battery-powered devices.

To the best of our knowledge, no relevant research work has applied BNNs to skin lesion segmentation. We are the first to explore this topic, which we believe will be a potential direction to reduce computation costs for the segmentation model in the future. Our study focuses on the practical and efficient application of XNOR-Net to UCM-NetV2 to develop BNN-UCM-NetV2. This approach significantly reduces inference computation costs to under 0.02 GFLOPs while maintaining the model's prediction performance. BNNs achieve this by compressing 32-bit float weights and layer inputs into 1-bit numbers. Within each neural network layer's matrices, weights and inputs are represented using [-1, +1] based on the sign of the original value. This allows the substitution of original matrix multiplication with the XNOR operation, a highly efficient approach that drastically reduces computational overhead. XNOR-Net, a classic type of BNN, represents a significant milestone in efficient deep learning. Figure 7 illustrates the BNN-UCM-NetV2 Block, a high-performing block that leverages the efficiency of binary operations to minimize computational complexity and memory usage. The BNN-UCM-NetV2 Block is a specific component of the BNN-UCM-NetV2 Network, and its efficient design is crucial for the network's overall performance. Full-precision operations are replaced with their binary counterparts, labeled as BNNLinear and BNNConv2D in the figure. Using binary weights and activations significantly reduces the memory and computational resources required. BNN layers employ the XNOR-Net approach, performing convolution operations with binary weights to enhance computational efficiency.

Loss Function for BNN-UCM-NetV2: The loss function in BNNs is a crucial component that balances the efficiency gains of binary operations with the need for effective learning. While BNNs offer significant advantages in computational and energy efficiency, they come with challenges related to gradient approximation and potential performance trade-offs. Careful design and tuning of the loss function are essential to unlock the full potential of BNNs for various applications. In traditional neural networks, loss functions like Binary Cross-Entropy Loss or Dice Loss for segmentation are commonplace. However, Binary Neural Networks (BNNs) require a more specialized approach to accommodate their binarized values and ensure smooth gradient flow during backpropagation. For the loss function for BNNs, we adapt UCM-NetV2's loss functions with additional terms or modifications to handle the difference between the binarization output and full-precision matrix output. A proposed approach incorporates the following components:

• Primary Loss Function: This component focuses on the main task, calculating the difference between the ground truth and the model's prediction. We selected our proposed loss function (7).
• Regularization Term: A regularization term, Manhattan norm (L1-norm), is added to each binary layer to learn the difference between the loss function's binarization matrix and the full-precision matrix. We record the difference between binary matrix output and full-precision matrix output in each layer. We calculate the L1-Norm value on those records and add the L1-Norm value to our loss function.
• Gradient Approximation: Binarization from the sign function introduces discontinuities in the loss landscape, challenging direct gradient calculation. The reason is that the derivative value from the sign is always zero, which prevents the model from learning about loss during the training. A common method is gradually using approximation techniques like the straight-through estimator (STE).

The BNN-UCM-NetV2's loss function is:

$$BNNGroup\_Loss=Los{s}_{Output}+\sum _{i=1}^5{\mathsf{\lambda }}_i\times Los{s}_{Stag{e}_i}+L1\_\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}$$

(8)

where L1_Norm represents the Manhattan norm of the recorded difference binary matrix output with full-precision matrix output in each layer.

The loss function in BNNs is designed to balance the efficiency of binary operations with the need for effective learning. While BNNs offer significant advantages in terms of computational and energy efficiency, they come with challenges related to gradient approximation and performance trade-offs.

Figure 8. BNNs STE process (cited from [21]).

Figure 8. BNNs STE process (cited from [21]).

In conclusion, UCM-NetV2 and BNN-UCM-NetV2 represent a significant step forward in skin lesion segmentation. UCM-NetV2 delivers high-precision results on standard platforms, while BNN-UCM-NetV2 extends deep learning capabilities to resource-constrained environments. These models have the potential to revolutionize how dermatologists diagnose and monitor skin conditions, making accurate and efficient segmentation accessible to a broader range of practitioners and patients.

4. Results and Discussions

5. Conclusions

References

1. Rundle, Chandler W., et al. "Epidemiologic burden of skin cancer in the US and worldwide." Current Dermatology Reports 9 (2020): 309-322. ACS, ACS. "Cancer Facts & Figures 2022." American Cancer Society, National Home Office, Atlanta (2022).
2. Stephen L Brown, Peter Fisher, Laura Hope-Stone, Bertil Damato, Heinrich Heimann, Rumana Hussain, and M Gemma Cherry. Fear of cancer recurrence and adverse cancer treatment outcomes: predicting 2-to 5-year fear of recurrence from post-treatment symptoms and functional problems in uveal melanoma survivors. Journal of Cancer Survivorship, 17(1):187–196, 2023. [CrossRef]
3. Rebecca L Siegel, Kimberly D Miller, and Nikita Sandeep Wagle. Cancer statistics, 2023.
4. Robin Marks. An overview of skin cancers. Cancer, 75(S2):607–612, 1995.
5. Thomas Martin Lehmann, Claudia Gonner, and Klaus Spitzer. Survey: Interpolation methods in medical image processing. IEEE transactions on medical imaging, 18(11):1049–1075, 1999. [CrossRef]
6. Muhammad Nasir, Muhammad Attique Khan, Muhammad Sharif, Ikram Ullah Lali, Tanzila Saba, and Tassawar Iqbal. An improved strategy for skin lesion detection and classification using uniform segmentation and feature selection based approach. Microscopy research and technique, 81(6):528–543, 2018. [CrossRef]
7. Catarina Barata, M Emre Celebi, and Jorge S Marques. Explainable skin lesion diagnosis using taxonomies. Pattern Recognition, 110:107413, 2021. [CrossRef]
8. Isaac Sanchez and Sos Agaian. Computer aided diagnosis of lesions extracted from large skin surfaces. In 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 2879–2884. IEEE, 2012.
9. Isaac Sanchez and Sos Agaian. A new system of computer-aided diagnosis of skin lesions. In Image Processing: Algorithms and Systems X; and Parallel Processing for Imaging Applications II, volume 8295, pages 390–401. SPIE, 2012.
10. Tiago M de Carvalho, Eline Noels, Marlies Wakkee, Andreea Udrea, and Tamar Nijsten. Development of smartphone apps for skin cancer risk assessment: progress and promise. JMIR Dermatology, 2(1):e13376, 2019.
11. butterflynetwork.https://www.butterflynetwork.com/iq-ultrasound-individuals.
12. phonemedical.https://blog.google/technology/health/ai-dermatology-preview-io-2021/.
13. Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. A guide to deep learning in healthcare. Nature medicine, 25(1):24–29, 2019. [CrossRef]
14. Chunlei Chen, Peng Zhang, Huixiang Zhang, Jiangyan Dai, Yugen Yi, Huihui Zhang, and Yonghui Zhang. Deep learning on computational-resource-limited platforms: a survey. Mobile Information Systems, 2020:1–19, 2020. [CrossRef]
15. Neil C Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F Manso. The computational limits of deep learning. arXiv preprint arXiv:2007.05558, 2020.
16. Su, Weixing, et al. "AI on the edge: a comprehensive review." Artificial Intelligence Review 55.8 (2022): 6125-6183. [CrossRef]
17. Singh, Raghubir, and Sukhpal Singh Gill. "Edge AI: a survey." Internet of Things and Cyber-Physical Systems 3 (2023): 71-92.
18. Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024.
19. Yuan, Chunyu, Dongfang Zhao, and Sos S. Agaian. "UCM-Net: A lightweight and efficient solution for skin lesion segmentation using MLP and CNN." Biomedical Signal Processing and Control 96 (2024): 106573. [CrossRef]
20. Hyeji Kim, Muhammad Umar Karim Khan, and Chong-Min Kyung. Efficient neural network compression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12569–12577, 2019.
21. Chunyu Yuan and Sos S Agaian. A comprehensive review of binary neural network. Artificial Intelligence Review, 56(11):12949–13013, 2023. [CrossRef]
22. Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.
23. Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.
24. Yundong Zhang, Huiye Liu, and Qiang Hu. Transfuse: Fusing transformers and cnns for medical image segmen-tation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, pages 14–24. Springer, 2021.
25. Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. Swin-unet: Unet-like pure transformer for medical image segmentation. In European conference on computer vision, pages 205–218. Springer, 2022.
26. White, Colin, et al. "Neural architecture search: Insights from 1000 papers." arXiv preprint arXiv:2301.08727 (2023).
27. Jeya Maria Jose Valanarasu and Vishal M Patel. Unext: Mlp-based rapid medical image segmentation network. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 23–33. Springer, 2022.
28. Jiacheng Ruan, Suncheng Xiang, Mingye Xie, Ting Liu, and Yuzhuo Fu. Malunet: A multi-attention and lightweight unet for skin lesion segmentation. In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1150–1156. IEEE, 2022.
29. Jiacheng Ruan, Mingye Xie, Jingsheng Gao, Ting Liu, and Yuzhuo Fu. Ege-unet: an efficient group enhanced unet for skin lesion segmentation. arXiv preprint arXiv:2307.08473, 2023.
30. Jiacheng Ruan and Suncheng Xiang. Vm-unet: Vision mamba unet for medical image segmentation. arXiv preprint arXiv:2402.02491, 2024.
31. Mingya Zhang, Yue Yu, Limei Gu, Tingsheng Lin, and Xianping Tao. Vm-unet-v2 rethinking vision mamba unet for medical image segmentation. arXiv preprint arXiv:2403.09157, 2024.
32. Weibin Liao, Yinghao Zhu, Xinyuan Wang, Cehngwei Pan, Yasha Wang, and Liantao Ma. Lightm-unet: Mamba assists in lightweight unet for medical image segmentation. arXiv preprint arXiv:2403.05246, 2024.
33. Renkai Wu, Yinghao Liu, Pengchen Liang, and Qing Chang. Ultralight vm-unet: Parallel vision mamba significantly reduces parameters for skin lesion segmentation. arXiv preprint arXiv:2403.20035, 2024.
34. Isic 2017 challenge dataset. https://challenge.isic-archive.com/data/#2017.
35. Matt Berseth. Isic 2017-skin lesion analysis towards melanoma detection. arXiv preprint arXiv:1703.00523, 2017.
36. Isic 2018 challenge dataset. https://challenge.isic-archive.com/data/#2018.
37. Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv:1902.03368, 2019.
38. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
39. Lambda cloud gpu. https://cloud.lambdalabs.com/instances.
40. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018.
41. Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
42. Linares, Miguel A., Alan Zakaria, and Parminder Nizran. "Skin cancer." Prim Care 42.4 (2015): 645-59.
43. Armstrong, Bruce K., and Anne Kricker. "Skin cancer." Dermatologic Clinics 13.3 (1995): 583-594.
44. Marks, Robin. "An overview of skin cancers." Cancer 75.S2 (1995): 607-612.
45. Khalid M Hosny, Doaa Elshora, Ehab R Mohamed, Eleni Vrochidou, and George A Papakostas. Deep learning and optimization-based methods for skin lesions segmentation: A review. IEEE Access, 2023.
46. Rastegari, Mohammad, et al. "Xnor-net: Imagenet classification using binary convolutional neural networks." European conference on computer vision. Cham: Springer International Publishing, 2016.
47. Aghdam, Ehsan Khodapanah, et al. "Attention swin u-net: Cross-contextual attention mechanism for skin lesion segmentation." 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). IEEE, 2023.
48. Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 201.
49. Lei, Dajiang, et al. "SCRNet: an efficient spatial channel attention residual network for spatiotemporal fusion." Journal of Applied Remote Sensing 16.3 (2022): 036512-036512.
50. Shishuai Hu, Zehui Liao, and Yong Xia. Devil is in channels: Contrastive single domain generalization for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 14–23. Springer, 2023.Bio-Implants—Global Market Outlook (2017–2026): Expected to Grow at a CAGR of 10.1% ResearchAndMarkets.com. 2018. Available online: https://www.businesswire.com/news/home/20181109005315/en/Bio-Implants---Global-Market-Outlook2017-2026-Expected-to-Grow-at-a-CAGR-of-10.1---ResearchAndMarkets.com (accessed on 6 April 2024).