Prerpints.org logo
Preprint
Article

SkinLiTE: Lightweight Supervised Contrastive Learning Model for Enhanced Skin Lesion Detection and Disease Typification in Dermoscopic Images

Altmetrics

Downloads

316

Views

180

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

02 March 2024

Posted:

05 March 2024

You are already at the latest version

Alerts
Abstract
This study introduces SkinLiTE, a lightweight supervised contrastive learning model, tailored to enhance the detection and typification of skin lesions in dermoscopic images. The core of SkinLiTE lies in its unique integration of supervised and contrastive learning approaches, which leverages labeled data to learn generalizable representations. This approach is particularly adept at handling the complexities and imbalances inherent in skin lesions datasets. The methodology encompasses a two-phase learning process. In the first phase, SkinLiTE utilizes an encoder network and a projection head to transform and project dermoscopic images into a feature space where contrastive loss is applied, focusing on minimizing intra-class variations while maximizing inter-class differences. The second phase freezes the encoder's weights, leveraging the learned representations for classification through a series of dense and dropout layers. The model was evaluated using three datasets from Skin Cancer ISIC 2019-2020, covering a wide range of skin conditions. SkinLiTE demonstrated superior performance across various metrics—accuracy, AUC , and F1 score—particularly when compared with traditional supervised learning models. Notably, SkinLiTE achieved an accuracy of 0.9087 using AugMix augmentation for binary classification of skin lesions. It also showed comparable results with the state-of-the-art approaches of ISIC challenge without relying on external data, underscoring its efficacy and efficiency. The results highlight the potential of SkinLiTE as a significant step forward in the field of dermatological AI, offering a robust, efficient, and accurate tool for skin lesion detection and classification. Its lightweight architecture and ability to handle imbalanced datasets make it particularly suited for integration into Internet of Medical Things environments, paving the way for enhanced remote patient monitoring and diagnostic capabilities. This research contributes to the evolving landscape of AI in healthcare, demonstrating the impact of innovative learning methodologies in medical image analysis.
Keywords: 
Subject: Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Introduction

Skin lesions and diseases encompass a wide range of conditions that affect the skin. In general, a skin lesion is any abnormality in the skin’s appearance, which can be a lump, bump, sore, ulcer, or colored area. These lesions can be benign (non-cancerous) or malignant (cancerous), and their causes vary from infections and genetics to environmental factors [1].
Various studies have provided a comprehensive view of the current state and future directions of artificial intelligence (AI), focusing on skin disease diagnosis and cosmetic dermatology. Research studies by Mohanty, et al. [2], Kumar, et al. [3], and Vatiwutipong, et al. [4] collectively highlighted the diverse and evolving applications of AI in dermatology. Mohanty, Sutherland, Bezbradica and Javidnia [2] focused on overcoming data scarcity in skin disease diagnosis using advanced techniques like generative adversarial networks and meta-learning, addressing a core challenge also pertinent to Kumar, Koul, Singla and Ijaz [3] in diagnosing various diseases, including skin conditions, emphasizing the necessity of diverse data for accurate diagnosis. Meanwhile, Vatiwutipong, Vachmanus, Noraset and Tuarob [4] expanded the scope to cosmetic dermatology, demonstrating AI’s role in areas ranging from product development to treatment prediction. Research challenges in the context of skin disease diagnosis include image preprocessing, and the improvement of methods for tasks like classification, detection, segmentation, multi-task modeling, and dataset characterization [5]. Several studies collectively explored the integration of advanced imaging and machine learning technologies in the medical field, especially in oncology and early disease diagnosis [6,7,8]. For instance, Kaur, Goyal and Daniel [6] focused on machine learning-based medical assistive systems in oncology, highlighting how these algorithms aid healthcare practitioners in cancer detection and treatment planning. Huérfano-Maldonado, Mora, Vilches, Hernández-García, Gutiérrez and Vera [7] delved into extreme learning machines in medical imaging, emphasizing their efficiency and accuracy in diagnosing various diseases through different imaging modalities like CT, MRI, and ultrasound. Mangotra, Srivastava, Jaiswal, Rani and Sharma [8] discussed the applications of hyperspectral imaging (HSI) combined with machine learning in early disease diagnosis, underscoring its non-invasive approach in fast detection and aiding surgical procedures.
Following, several studies have highlighted the significant advancements of deep learning algorithms in medical imaging, particularly in the diagnosis and categorization of various diseases, including cancer and skin conditions [9,10,11,12,13,14,15,16]. While many studies focused on diagnosing autoimmune blistering skin diseases using deep neural networks, emphasizing the need for computerized systems to overcome the limitations of current diagnostic methods [9,12], other studies were directed to advanced algorithms for skin lesion segmentation, a critical step in skin cancer diagnosis [10,14,15]. The challenges and recent developments in multiple-lesion recognition, highlighting the complexity of recognizing different lesions simultaneously was explored [11]. The use of deep learning in detecting various types of cancer, underlining the role of these technologies in early diagnosis and improved patient outcomes was investigated [13,14]. Segmentation of optical coherence tomography images, a challenging task crucial for diagnosing diseases like glaucoma was explored [15]. Lastly, a study surveyed deep learning in human cancer categorization, emphasizing the effectiveness of convolutional neural networks (CNN) in classifying histopathological images [16]. Together, these studies underscore the transformative impact of deep learning in enhancing medical diagnostics across diverse domains, from skin lesions to various cancer types, through improved accuracy and efficiency in image analysis and disease categorization.
Supervised learning and contrastive learning represent distinct methodologies, with supervised learning focusing on mapping inputs to outputs using labeled data for tasks like image classification, and contrastive learning emphasizing the learning of data representations by distinguishing similar from dissimilar data points, useful in scenarios with limited labeled data [17]. Supervised contrastive learning, a novel approach by Google Research [18], merges these techniques by using labeled data to enhance representation learning. This method forms pairs or groups based on class labels, employing a supervised contrastive loss function to group together data points of the same class while separating different classes. Especially beneficial in handling imbalanced datasets and as pre-training for various downstream tasks, supervised contrastive learning capitalizes on the strengths of both approaches, offering improved performance in tasks requiring nuanced understanding of data.
To the best of our knowledge, previous research has not delved into the application of a supervised contrastive learning model specifically for melanoma detection and the categorization of skin diseases. Thus, this study investigates the effective combination of labeled dermoscopic skin data with contrastive learning to optimally utilize label information in the context of complex and imbalanced skin lesion datasets. This work aims to harness the latest advancements in supervised contrastive learning, integrated with deep neural networks, for skin lesions detection. Our proposed model, SkinLiTE, is designed to generate more robust and flexible representations from dermoscopic images, which is vital for identifying subtle characteristics of various skin conditions. Especially beneficial for dealing with datasets characterized by imbalance, such as those prevalent in skin disease studies with limited data, SkinLiTE focuses on discerning the relative similarities and differences across different classes. In a manner akin to unsupervised contrastive learning, the representations developed through this model provide a strong basis for numerous subsequent supervised learning tasks aimed at predicting skin lesions. The key contributions and architectural design of the SkinLiTE model can be summarized as follows:
  • SkinLiTE is designed to leverage the strengths of both supervised learning and contrastive learning for skin lesion detection and disease typification from dermoscopic images. This hybrid approach is particularly potent for tasks that involve complex medical datasets.
  • SkinLiTE addresses the challenge of imbalanced datasets we encountered in skin lesions by focusing on discerning relative similarities and differences among various classes, which is often the case with rare skin diseases.
  • The model uses labeled data to form pairs or groups based on class labels, which is essential for supervised contrastive learning. The representations derived from SkinLiTE are designed to be strong and adaptable, enabling the model to discover features of various skin conditions effectively.
  • The two-phase architecture enhances the learning process in SkinLiTE and yields a light-weight model for the application in the world of the internet of medical things (IoMT). In the first phase, the model utilizes an encoder and a projection head to map the input dermoscopic images into a representation space where contrastive loss is applied. Here, the model is trained to update its weights to minimize this loss, effectively learning to anchor data points of the same class closely together and push different classes apart. The second phase involves the encoder with its frozen weights and a trainable projection head, meaning the representations learned in the earlier phase are kept stable. The focus here is on classification, where a series of layers, including dropout and dense layers, lead to a classification layer.
  • SkinLiTE is trained for multi-tier classification to handle different classification scenarios: Bi-Classifier: Differentiates between benign and malignant lesions., Tri-Classifier: Categorizes lesions into melanoma, nevus, or seborrheic keratosis., and N-Classifier: A more granular classification that includes multiple types of skin conditions like Actinic Keratosis (AK), Basal Cell Carcinoma (BCC), Benign Keratosis-like Lesions (BKL), Dermatofibroma (DF), Melanoma (MEL), Nevus (NV), Squamous Cell Carcinoma (SCC), and Vascular Lesions (VASC).
These contributions and design elements position SkinLiTE as a promising model for the diagnosis and classification of skin lesions, potentially improving the accuracy and reliability of automated dermoscopic image analysis. The rest of this paper is structured as follows. It reviews the related work, discussing the evolution from traditional machine learning to current deep learning paradigms, including the strategic use of pre-trained models and transfer learning techniques. The paper addresses the challenges posed by imbalanced datasets and underscores the importance of attention mechanisms in model architecture. It also contemplates the trends and potential of IoMT while identifying existing gaps in the field. The methodology section delves into the problem formulation, describes the architecture of the SkinLiTE model, elaborates on the employed augmentation techniques, and introduces the concept of supervised contrastive loss. The experimental setup is detailed, covering the datasets used, augmentation results, and evaluation metrics, along with the computing resources deployed. Following this, the paper presents the results and engages in a discussion about the findings. It concludes with reflections on the study’s implications and suggestions for future research avenues, ensuring data availability for reproducibility and further investigation.

2. Related Work

2.1. The Glance of Machine Learning

Multiple studies have highlighted the pivotal role of machine learning and image processing in advancing the precision and efficiency of diagnosing skin diseases, signaling a shift towards the integration of AI in the field of dermatology [19,20,21,22,23,24,25]. For example, AlDera and Ben Othman [19] innovated a diagnostic model for conditions like acne and melanoma, implementing a comprehensive process encompassing image acquisition, preprocessing, segmentation, feature extraction, and classification, and achieved impressive accuracy with algorithms such as SVM, RF, and KNN. Hatem [20] introduced a system based on the KNN algorithm, tailored for skin lesion classification in MATLAB, achieving a remarkable 98% accuracy. Saghir and Hasan [21] concentrated on skin cancer detection, utilizing the differential analyzer algorithm and attaining a notable 96% classification accuracy. Mustafa, Jaffar, Iqbal, Abubakar, Alshahrani and Alghamdi [22] formulated a melanoma classification method using hybrid color texture features and neural network, outperforming existing methods across various datasets. Huong, Khang, Quynh, Thang, Canh and Sang [23] proposed a sophisticated deep and machine learning hybrid model for classifying monkeypox, obtaining high accuracy and F1-score. Meena, Veni, Deepapriya, Vardhini, Kalyani and Sharmila [25] experimented with KNN, SVM, and RF classifiers for skin disease data, emphasizing feature selection and real-time dataset performance. Finally, MunishKhanna, Singh and Garg [24] adopted a nature-inspired computing strategy for predicting various human diseases, including skin cancer, leveraging ant-lion optimization for feature selection and achieving significant accuracy across diverse datasets.

2.2. The Move to Deep Learning

Many studies have collectively highlighted advancements in deep learning for skin disease classification, each adopted varied approaches, from leveraging pre-trained models to developing novel architectures and combining different methods. These themes and the corresponding studies demonstrate the breadth and depth of AI research in dermatology. From integrating complex data sources for more comprehensive analysis to leveraging advanced AI architectures and optimization techniques, researchers have explored various avenues to enhance the accuracy, efficiency, and usability of AI tools in diagnosing and understanding skin diseases. These research trends are explained in the subsections below.

2.2.1. Use of Pre-Trained Models and Transfer Learning

Several studies have utilized pre-trained models like EfficientNet, VGG, MobileNet, ResNet, AlexNet, Xception and Inception, demonstrating the efficacy of transfer learning in medical image analysis [26,27,28,29,30,31,32]. These studies demonstrated the effectiveness of applying transfer learning techniques in dermatology, facilitating the development of diagnostic tools. Transfer learning approach is particularly beneficial for dealing with limited datasets, as shown in Spolaôr, Lee, Mendes, Nogueira, Parmezan, Takaki, Coy, Wu and Fonseca-Pinto [27]’s study, where fine-tuning pre-trained networks on small datasets proved effective. Contrastingly, Tripathi, et al. [33] and Bala, et al. [34] developed specific models named XCNN and MonkeyNet, tailored for their respective tasks, suggesting a preference for custom-built architectures in certain scenarios. Wei, et al. [35] combined DenseNet and ConvNeXt models, incorporating attention modules and pre-training, data augmentation, and fine-tuning processes to classify skin diseases, especially focusing on acne-like diseases. El Gannour, et al. [36] utilized two ensemble learning techniques, integrating image and clinical data inputs for improved skin disease classification accuracy. Aboulmira, et al. [37] explored several deep CNN architectures for skin disease classification using the DermNet dataset, ultimately finding DenseNet most effective. Anand, et al. [38] used a transfer learning-based model with a modified pre-trained Xception model, applying data augmentation techniques for classifying skin diseases. Studies by the same group evaluated four pre-trained transfer learning models on skin dataset images, focusing on preprocessing steps and various simulation parameters and comparing it with different ResNet models [39,40]. Nath, et al. [41] developed a system using CNN architecture and six preset models to classify common skin conditions, applying data augmentation on the dataset. Arora, et al. [42] evaluated the performance of fourteen deep learning networks using transfer learning on the ISIC 2018 dataset for skin lesion classification. Al-Tuwaijari, et al. [43] used three pre-trained deep learning models namely DenseNet121, VGG19, and an improved ResNet152 for classifying skin cancer from dermoscopic images. Tasar [44] developed SkinCancerNet, employing modified CNN frameworks based on transfer learning for classifying skin lesions. Magdy, et al. [45] proposed two methods for detecting and classifying dermoscopic images into benign and malignant tumors, utilizing optimized AlexNet. Pramila and Subhashini [46] presented an automated skin lesion detection and classification method using fused deep convolutional neural networks, focusing on VGG19 and ResNet152 models for feature extraction. Arshed, et al. [47] compared the performance of pre-trained vision transformers with CNN-based models for classifying seven types of skin cancers. Raghavendra, et al. [48] proposed a model for multi-class skin lesion classification, incorporating global average pooling and preprocessing techniques. Kani, et al. [49] developed a computer-aided diagnosis (CAD) system using a modified InceptionV3 model to classify skin lesions, focusing on melanoma detection from dermoscopic images.

2.2.2. Innovative Approaches and Combination Strategies

Aloraini [50] and Riaz, et al. [51] introduced novel combination approaches whereby Aloraini [50] used two-stream CNNs combining RGB and gradient images, while Riaz, Qadir, Ali, Ali, Raza, Jurcut and Ali [51] combined CNN and local binary pattern (LBP), both achieving high accuracy, indicating the potential of multi-input or multi-method approaches. In contrast, Kaur and Kaur [52] focused on feature fusion using multiple networks, which is a different strategy of combining strengths of various architectures. Li, et al. [53] created DIET-AI, which combined dual-channel images and extracted text for diagnosing 31 common skin diseases. This system was based on a large dataset from the Asian population and demonstrated diagnostic performance comparable to senior doctors. Li, et al. [54] introduced SCAN, focusing on few-shot learning. This model identified subclusters within skin disease images and utilized a dual-branch framework for class-wise feature learning and subcluster preservation. Reddy, et al. [55] proposed a framework using lesion segmentation and classification techniques, employing enhanced U-Net for segmentation and an ensemble-convolutional network for classification. Josphineleela, et al. [56] proposed a multi-stage faster RCNN-based iSPLInception model for skin disease classification, utilizing two datasets for performance evaluation. Sreekala, et al. [57] used an enhanced CNN for skin lesion classification, incorporating feature extraction and median filtering. He, et al. [58] proposed a multi-task learning CNN for joint skin lesion segmentation and classification, introducing edge prediction as an auxiliary task. Alam, et al. [59] proposed SC-C-2-DeLeNet, using a segmentation sub-network and classification architecture. Dong, et al. [60] proposed a pipeline for efficient skin lesions’ segmentation and classification, utilizing a novel module for feature exploration. Yadav, et al. [61] implemented a sailfish-based gradient boosting framework for recognizing skin lesions, focusing on high segmentation and classification accuracy. Duman and Tolan [62] combined CNN models in an ensemble approach for diagnosing skin diseases from color lesion images. Kalaivani and Karpagavalli [63] developed a fuzzy transfer learning model as a domain adaptation technique, combined with a bootstrapping of fine-tuned segmentation and classification. Ramamurthy, et al. [64] presented a two-stage approach involving segmentation and classification architectures for effective detection of skin disease, combining the benefits of residual learning and convolutions.

2.2.3. Focus on Specific Challenges

Several studies have addressed specific challenges in skin diseases. Rao, et al. [65] integrated a binary butterfly optimization algorithm for accuracy and efficiency, whereas Manoj, et al. [66] focused on integrating color and texture features for categorization, showing how targeted approaches can address particular issues in medical imaging. Hao, et al. [67] introduced a novel hybrid model, named ConvNeXt-ST-AFF, combining the strengths of different neural network architectures. This innovation aimed to improve diagnostic accuracy and efficiency in dermatological classifications. Additionally, several studies targeted specific dermatological conditions. Kalyani, et al. [68], Seeja and Geetha [69] and Albraikan, et al. [70] focused on melanoma, using deep learning for accurate classification. Meswal, et al. [71] used a weighted ensemble of seven deep-learning neural networks for binary classification of skin lesions, focusing on melanoma and nevus. Asif, et al. [72], Aloraini [73], Alrusaini [32], Ren [74] and Ahsan, et al. [75] concentrated on monkeypox, developing models to detect this emerging disease. Bian, et al. [76] proposed a two-phase classification method for skin lesion images, integrating medical domain knowledge with deep learning, focusing on distinguishing malignant melanomas. Radhika and Chandana [77] proposed the multi-class skin detection model for melanoma, benign keratosis, melanocytic nevi, and basal cell carcinoma.

2.2.4. Advanced Optimization Techniques

Several studies employed advanced optimization and feature selection techniques. Shaheen and Singh [29] used particle swarm optimization, and Ghosh, et al. [78] employed a two-phase evolutionary approach, highlighting a trend towards incorporating advanced optimization algorithms for enhancing model performance. Rajeshwari and Sughasiny [79] used beetle swarm optimization, while Mostafa, et al. [80] introduced a hybrid-mutated differential evolution method to enhance model performance. Zhou, et al. [81] combined data preprocessing, augmentation, and residual networks, focusing on color-based background selection’s influence on classification. Jain, et al. [82] introduced an optimal probability-based deep neural network for skin disease prediction, using whale optimization for optimizing weights in the network. Liu, et al. [83] proposed the anti-coronavirus optimized kernel-based Softplus Machine for predicting different types of skin cancer from high-dimensional datasets. Vidhyalakshmi and Kanchana [84] used a GSO-optimized kernel random forest classifier based on transfer learning for classifying 12 different skin cancer types, leveraging ImageNet and various CNN architectures. Renith and Senthilselvi [85] proposed a model using an improved Adaboost-based Aphid-Ant Mutualism classification model. Lastly, Desale and Patil [86] introduced an optimized vision transformer approach for classifying skin tumors using a self-sparse watershed algorithm.

2.2.5. Dataset Utilization

Many studies used public datasets like HAM10000 and ISIC (2016-2020) [28,29,36,38,42,50,87,88,89], showcasing the reliance on standardized datasets for model training and validation. Bala, Hossain, Hossain, Abdullah, Rahman, Manavalan, Gu, Islam and Huang [34] created a new dataset (MSID) for monkeypox, underlining the importance of dataset development in emerging medical conditions. Part of studies focused on the developed models that integrated diverse data types [53,90]. To elaborate, Li, Zhang, Wei, Qian, Tang, Hu, Huang, Xia, Zhang, Cheng, Yu, Zhang, Dan, Liu, Ye, He, Jiang, Liu, Fan, Song, Zhou, Wang, Zhang and Lv [53] combined images and medical records to diagnose skin diseases, while El Saleh, Chantaf and Nait-ali [90] focused on facial skin diseases, using images captured under various conditions. Bozkurt [91] emphasized data augmentation to address dataset limitations. This approach highlighted the importance of enhancing data quality and quantity for training more robust AI models in dermatology. Pokhrel, et al. [92] employed mask RCNN in conjunction with an augmented dataset from HAM10000 for skin disease classification, focusing on preprocessing, model architecture, and evaluation. Holmes, et al. [93] adapted the StarGAN network for augmenting images of faces and combined it with a transfer learning approach for classifying facial skin diseases.

2.2.6. Performance and Interpretability

There’s a range in accuracy, from 85.2% in Zou, Cheng and Liang [26]’s study to 99.01% in Kaur and Kaur [52]’s research, illustrating varying levels of effectiveness across different methods and conditions. The high accuracy in Kaur and Kaur [52]’s study and in the fusion architecture of Riaz, Qadir, Ali, Ali, Raza, Jurcut and Ali [51] suggests the potential of multi-model or multi-feature approaches in enhancing classification performance. Another focus by Rezk, et al. [94] was given to the interpretability of their AI model in skin cancer classification. They aimed to make the model’s decisions more transparent and clinically relevant.

2.3. The Challenge of Imbalanced Data

Several studies have been conducted significant research in the realm of skin cancer detection and classification, particularly focusing on the challenge of imbalanced datasets in dermatology [89,95,96,97,98,99,100,101,102,103,104]. These studies collectively advanced the field of medical image classification by addressing the critical issue of imbalanced datasets, each contributing novel techniques and frameworks to improve classification accuracy and efficiency in dermatological and other medical imaging applications. Hoang, Vo and Jo [95] addressed this issue by developing a categorical weighting domination method for deep CNNs that boosted the sample weights of minority classes and reduced the impact of majority classes, resulting in higher precision and stability across various models without needing to augment the dataset. Their method outperformed traditional approaches like data augmentation and focal loss, in terms of recall, precision, specificity, and F1 score. Yu, Wang, Zhou, Wu and Wang [96] introduced a residual neural network-assisted one-class classification algorithm tailored for melanoma recognition, effectively handling imbalanced data, and achieving stable training results and consistent performance across complex datasets. This approach was notable for its combination of variational self-encoder and adversarial training techniques. Adepu, Sahayam, Jayaraman and Arramraju [89] proposed a knowledge-distilled lightweight framework for melanoma classification, addressing class imbalance with cost-sensitive learning and focal loss. They also employed novel data augmentation techniques and test time augmentation to enhance the model’s robustness. Their method achieved a significant increase in sensitivity for melanoma classification on the ISIC-2020 dataset, setting a new state-of-the-art result. Chen, Lei, Zhao, Zheng and Wang [97] introduced the progressive class-center triplet framework, which effectively managed class imbalances in medical image datasets through a two-stage process involving class-balanced triplet loss and a class-center involved triplet strategy. This innovative approach led to significant improvements in F1 scores across various datasets. Okuboyejo and Olugbara [98] proposed a novel ensemble algorithm that leveraged residual networks and dual path networks, achieving high sensitivity, specificity, and balanced accuracy in skin lesion classification without needing prior segmentation for extremely imbalanced training datasets. Vidhyalakshmi and Kanchana [99] developed a hybrid flash butterfly optimized CNN, targeting early and accurate prediction of skin diseases from dermoscopic images. Huang, Wu, Wang, Li and Ioannou [100] focused on semi-supervised learning with their class-specific distribution alignment (CSDA) framework. They introduced a new perspective on distribution alignment and a variable condition queue (VCQ) module to maintain a balanced number of unlabeled samples for each class, showing competitive performance in various medical image classification tasks. Zhuang, Cai, Zhang, Zheng and Wang [101] introduced the class attention to regions of the lesion (CARE) framework, which embedded attention into the CNN training process to focus on lesion regions of rare diseases. This approach, including automated bounding box generation variants, effectively improved classification performance on imbalanced skin image datasets. Yao, Shen, Xu, Liu, Zhang, Xing, Shao, Kaffenberger and Xu [102] proposed a single-model deep-CNN strategy for skin lesion classification on small and imbalanced datasets. They introduced regularization techniques, a novel loss function, and a cumulative learning strategy, achieving high classification accuracy while conserving computational resources. Alshawi and Musawi [103] focused on melanoma classification using ensemble classifier models to address misclassification and overfitting, employing transfer learning to tackle imbalanced classification issues. Lastly, Yue, Wei, Zhou, Jiang, Yan and Wang [104] proposed a solution for multi-center skin lesion classification by introducing an adaptively weighted balance (AWB) loss to the classification network. This method, focusing on data imbalance rather than framework improvement, showed flexibility and improved performance in classifying imbalanced datasets.

2.4. Attention to Skin Lesions

Numerous studies have utilized attention mechanisms in their deep learning models for skin cancer classification, although their approaches and implementations varied [105,106,107,108]. Omeroglu, Mohammed, Oral and Aydin [105] and Bao, Han, Huang and Muzahid [108] explicitly used soft attention mechanisms to enhance feature detection in skin lesions, Desale and Patil [107]’employed an attention mechanism in a convolutional capsule network, and Zhang, Wang, Cheng and Song [106]’s approach, while not explicitly stated as an attention mechanism, focused on capturing comprehensive image features, which can be seen as a form of attention to image details. Moreover, To, et al. [109] introduced the MetaAttention model for skin lesion diagnosis. The unique aspect of their attention mechanism was its combination of image features with patient metadata. The attention mechanism was aligned with clinical ABCD (Asymmetry, Border, Color, Diameter) signals, which are critical in identifying melanoma. By focusing on these specific features, the model could more effectively distinguish melanoma cell carcinoma, a significant challenge in skin cancer diagnosis. The squeeze and excitation network (SENet) attention model was used by Surati, et al. [110] for classifying diseases like monkeypox, chickenpox, and measles. SENet works by recalibrating channel-wise feature responses by explicitly modelling interdependencies between channels. This means the model can emphasize informative features and suppress less useful ones, improving the overall performance of the classification task. Damineni, et al. [111] proposed a streamlined attention module for a range of medical imagery, including skin lesions, brain tumors, and COVID-19. The streamlined attention mechanism focused on enhancing the model’s capability to extract detailed features with proportionate attention, enabling it to identify fine-grained visual patterns critical in medical diagnostics. The use of grad-cam heat maps illustrated how this attention mechanism could highlight the most relevant features in the images. In the TS-CNN architecture [112], an attention branch was used to select important regions and discard irrelevant parts in medical images. This approach improved interpretability and classification performance, especially for images with scattered and randomly shaped lesions. The attention branch worked in conjunction with a global branch (which learned patterns from the entire image) and a fusion branch (which integrated knowledge from both global and local branches). Ren, et al. [113] modified the Swin transformer for the automatic classification of dermatological conditions, using a self-attention mechanism and shifting windows. Finally, Zhang, et al. [114] developed CR-Conformer, a dual-branch fusion network integrating transformer branches for classifying clinical skin lesion images.

2.5. The Trends of Internet of Medical Things and Remote Patient Monitoring

Recently, several studies have explored innovative approaches in healthcare technology, focusing on AI for remote patient monitoring (RPM) and skin disease classification [115,116,117,118,119,120,121,122,123,124]. These studies established the synergy between emerging technologies like IoMT devices, and mobile applications in tackling complex skin disease challenges. For example, Shaik, Tao, Higgins, Li, Gururajan, Zhou and Acharya [115] examined the state and challenges of AI-enhanced RPM systems, highlighting AI’s transformative role in early health deterioration detection, personalized monitoring using federated learning, and behavior pattern understanding. They emphasized the integration of IoMT technologies like cloud, fog, edge, and blockchain in RPM. Oztel, Oztel and Sahin [116] and Shahin, Chen, Hosseinzadeh, Koodiani, Shahin and Nafi [120] developed a deep learning-based mobile application for skin disease classification, leveraging smartphone cameras and TensorFlow Lite to assist in preliminary diagnoses and reduce patient stress. Their user-friendly technology achieved notable accuracy in classifying skin diseases. Yang, Xie, Zhang, Wang, Liu, Zhang and Ding [117] introduced a novel skin lesion classification method using a multi-scale fully-shared fusion network, integrating features from clinical images to enhance accuracy. Manzoor, Majeed, Siddique, Meraj, Rauf, El-Meligy, Sharaf and Abd Elgawad [118] presented a lightweight automated system, combining features from deep CNNs and SVMs, demonstrating high accuracy in detecting various skin diseases and emphasizing the importance of early detection in conditions like skin cancer. Vivekananda, Almufti, Suresh, Samsudeen, Devarajan, Srikanth and Jayashree [119] introduced IoT-based model for skin disease detection, focusing on moles, skin tags, and warts using automatic lumen detection. Kumar, Satheesha, Salvador, Mithileysh and Ahmed [121] introduced AuDNN framework for skin cancer classification, leveraging multi-dimensional datasets on industrial IoT standards and achieving a 93.26% accuracy. Shi, Li and Chen [122] developed a federated contrastive learning framework for skin lesion diagnosis in edge computing networks, enhancing diagnostic accuracy by utilizing contrastive learning and dual encoder networks. Gupta, Bhagat and Jain [123] presented a blockchain-enabled system for early monkeypox detection, utilizing transfer learning and achieving a classification accuracy of 98.80%. Lastly, Hossen, Panneerselvam, Koundal, Ahmed, Bui and Ibrahim [124] explored the classification of skin diseases using a custom dataset and CNN, achieving precision rates up to 86% for diseases like acne, eczema, and psoriasis. They enhanced the model’s generalizability through image augmentation and addressed IoMT security by employing a federated learning approach. This method distributed the dataset among various clients, resulting in average accuracies between 81.21% and 94.15%, effectively merging CNN-based disease classification with data privacy preservation.

2.6. The Gap

From the literature surveyed, spanning from 2022 to the present, there exists no precedent of employing a supervised contrastive learning model that simultaneously addresses the dual challenges of managing imbalanced skin lesion datasets with improved precision and creating a rapid, lightweight learning framework suitable for the IoMT and RPM. Our proposed SkinLiTE model fills this gap by introducing a multi-tier classification system using supervised contrastive learning specifically designed to be compatible with IoT devices, edge and mobile platforms, thereby enhancing the capability to confront intricate skin disease diagnostics effectively.

3. Methodology

3.1. Problem Formulation

This study proposed the use of supervised contrastive learning that combines elements of both supervised and contrastive learning approaches for skin lesions classification. The proposed model, named SkinLiTE, aimed at improving the quality of learned representations by leveraging label information in a contrastive learning framework. Unlike traditional self-contrastive learning which typically relies on unlabeled data, supervised contrastive learning uses labeled data. It learns representations by not only bringing similar (positive) examples closer and pushing dissimilar (negative) examples apart, but also by incorporating label information to further guide this process. The goal is to learn representations where examples of the same class (according to their labels) are close together in the feature space, while examples from different classes are far apart, even more so than in standard contrastive learning.

3.2. SkinLiTE Model Architecture

The diagram in Figure 1 illustrates the two-phase framework of SkinLiTE for skin disease classification. In phase (1), the model begins by taking input images of skin lesions. These images are augmented, then passed through an encoder network, which is typically a type of deep neural network that transforms raw image data into a more abstract and useful representation. The encoder is designed to extract features from the images that are essential for the task at hand. Following the encoder, a projection head, which is a neural network layer, projects the encoder’s features into a space where contrastive loss can be applied effectively. For each image (anchor), the model selects a positive example (another image of the same class) and negative examples (images of different classes). The model uses contrastive loss to update its weights. This loss function encourages the model to bring features from the same class closer together by reducing the distance between anchor and positive and push features from different classes farther apart. In phase (2), the weights of the encoder are frozen, meaning they are not updated during this phase. This utilizes the representations learned during Phase 1. However, the projection head here is trainable, which means it is fine-tuned during the classification process to better suit the classification task. The features are passed through one or more dense layers (fully connected neural network layers). Dropout layers are included to prevent overfitting. They work by randomly deactivating some neurons during training, which forces the network to learn more robust features. Finally, the features are passed to a classification (softmax) layer, which is responsible for making the final prediction about the skin disease class. During the training, a classification loss, such as cross-entropy loss, is calculated to measure the difference between the predicted class probabilities and the true class labels. The network weights are updated to minimize this loss.
The framework includes multiple classifiers for different levels of classification granularity. The Bi-Classifier distinguishes between benign and malignant; the Tri-Classifier categorizes lesions into melanoma, nevus, or seborrheic keratosis; and the N-Classifier has a broader range of specific skin disease classes like AK (Actinic Keratosis), BCC (Basal Cell Carcinoma), etc. The model updates its weights through backpropagation in both phases, using the respective loss functions as guides to learn discriminative features for both contrasting and classifying skin diseases. The use of supervised contrastive learning in SkinLiTE potentially results in more meaningful and generalizable features, which can enhance the performance of the classification. The overall architecture is designed to be lightweight and fast, making it suitable for deployment in IoMT environments and for remote patient monitoring applications, where quick and accurate diagnosis is essential. The model’s ability to handle imbalanced datasets is particularly valuable in medical imaging, where some conditions are much rarer than others.

3.3. Augmentation

In the context of supervised contrastive learning for skin lesion analysis, augmentation plays a crucial role in improving the model’s ability to generalize and learn robust features. The goal is to create a diverse set of training examples from the original dataset, encouraging the model to learn invariant features that are not dependent on specific characteristics of the training images. This process is especially important in phase (1) of the framework in Figure 1, where the model is learning to distinguish between different classes of skin lesions in the feature space. By using augmented images as the anchor, positive, and negative examples, the model can learn to group similar lesions together while pushing dissimilar ones apart, regardless of variations introduced by the augmentation processes. This study employed a set of augmentation methods applied to input datasets for the training of a skin lesion analysis model.
RandAug applies a series of random transformations to an image. Each transformation is randomly selected from a predefined set and applied with random magnitude. This includes geometric transformations such as random rotations, flipping (horizontal and vertical), translations, and zooming. For skin lesion images, these transformations simulate variations in the orientation and location of lesions on the skin. Additionally, color adjustments such as the brightness, contrast, saturation, and hue help the model become more robust to variations in lighting conditions and skin tones. Random cropping and then resizing back to the original dimensions can simulate different scales of lesions and help the model focus on various parts of the lesion. Likewise, applying elastic transformations can simulate the natural variability of skin textures and lesions. Randomly removing parts of the image (i.e., zeroing out pixel values) forces the model to focus on less prominent features, which may be crucial for classification. In this study, we used three random augmentations per image sample.
AugMix [125] is an augmentation strategy that mixes multiple augmentation operations. It involves taking an image and creating several augmented versions of it using different augmentation processes. These images are then combined in a weighted manner to create the final augmented image. This process results in augmented images that maintain the content of the original image while including diverse visual features.
MixUp [126] is a data augmentation technique where two images are combined together to create a single new image. The corresponding labels are also mixed proportionally to the amount of each image in the combination. For example, if image A with label A is combined with image B with label B at a ratio of 0.7 to 0.3, then the resulting image will be 70% of image A and 30% of image B, and the label will be a weighted combination of labels A and B. This can lead to a regularization effect, as the model learns to generalize from the linear interpolations of images and their labels.
CutMix [127] is similar to MixUp, but instead of combining the pixel values of two images, it involves cutting and pasting patches from one image to another. A patch from image A is cut out and pasted onto image B, and the labels are mixed in the same way as in MixUp, according to the ratio of the areas of the patches. This encourages the model to focus on different regions of the image, improving localization and leading to better performance on tasks like object detection and classification.
The diagram in Figure 1 also categorizes these methods into two methods. "No Label Mixing" methods like RandAug and AugMix apply transformations to the images without altering the labels. In contrast, "Label Mixing" methods like MixUp and CutMix create new images by blending images and their labels. These techniques are particularly useful in preventing overfitting and helping the model learn more robust features by training on a more diverse set of data points.

3.4. Supervised Contrastive Learning

The extension of contrastive learning to the supervised setting, as explored by Google Research, represents a significant advancement in the realm of deep learning and representation learning [18]. In this phase, SkinLiTE model is trained to learn representations by bringing the embeddings of similar examples closer and pushing the embeddings of dissimilar examples further apart. The contrastive loss is calculated not just on the basis of similarity or dissimilarity but also considering the class labels of the skin lesions data. By using label information, the model learns to group together instances of the same class while separating those of different classes. This leads to more discriminative and robust representations, improving the model’s performance on downstream tasks. The core of this approach is the supervised contrastive loss following to the work by Khosla, Teterwak, Wang, Sarna, Tian, Isola, Maschinot, Liu and Krishnan [18]. This loss encourages examples from the same class to have similar representations, while examples from different classes are pushed apart in the representation space. The details of the algorithm used in this phase are as follows.
  • Input Images: Dermoscopic images are input to the encoder.
  • Encoding: We used ResNet50V2 for the encoder neural network to produce embeddings.
  • Projection: The embeddings are then mapped to a projection space using a projection head, which is typically a shallow neural network.
  • Contrastive Loss Calculation: The model computes the supervised contrastive loss for the anchor, positive, and negative samples. Given an anchor image embedding x , and a set of positive examples P where P constitutes examples from the same label plus augmentations of anchor x , and a set of negative examples N from other classes, the supervised contrastive loss used in this study L s c l was defined as:
L s c l = − 1 P ∑ p ∈ P l o g exp ⁡ s i m x , p τ ∑ n ∈ N exp ⁡ s i m x , n τ + ∑ p ∈ P exp ⁡ s i m x , p τ
where s i m ( x , p ) is the similarity measure (e.g., dot product or cosine similarity) between two embeddings of x and its positive pair p , and s i m ( x , n ) is the similarity between two embeddings of x and its negative pair n , and τ is a temperature scaling parameter that controls the separation margin.
The overall objective in supervised contrastive learning is to minimize the L s c l ​ loss over the entire training dataset. This results in embeddings where instances of the same class are closer together, and instances of different classes are further apart, efficiently utilizing label information in SkinLiTE’ framework.

3.5. Supervised Classification Learning

In the second phase, the SkinLiTE model utilizes the representations learned earlier to classify the skin lesions. The encoder’s weights are frozen to retain the learned features, and only the classification head is updated. The details of the algorithm used in this phase are as follows.
  • Inputs:
    o
    Dermoscopic images are input to the encoder.
    o
    Frozen Encoder which is the encoder from phase (1) with its frozen weights provides the feature embeddings.
  • Trainable Projection Head: A new projection head, which can be trained, is used for fine-tuning to the classification task.
  • Classification Layers: A series of layers including dropout and dense layers process the embeddings.
  • Cross-Entropy Loss Calculation: The model calculates the cross-entropy loss based on the output of the classification layers and the true labels. Given a set of true labels y and the predicted probabilities y ^ by the model, the cross-entropy loss L c e for classification is defined as:
L c e = − ∑ i y i l o g ⁡ ( y ^ i )
where y i is the true label (0 or 1) for class i , and y ^ i is the predicted probability for that class. By optimizing these two loss functions given by equations (1) and (2) in their respective phases, SkinLiTE can learn accurate and reliable feature representations for skin disease diagnosis and typification.

4. Experimental Setup

4.1. Datasets

Table 1 presents three datasets used for skin cancer image classification and melanoma detection. The Skin Cancer ISIC 2019 & 2020 dataset [128], compiled from images provided by the International Society for Digital Imaging of the Skin, includes 11,396 images with a binary classification into benign and malignant tumors, balanced by class weights, and split into 9,117 training and 2,279 validation images. The Melanoma Detection Dataset [129] aims to advance the development of automated melanoma diagnosis tools and contains 2,750 labeled images, classified into three categories—melanoma, nevus, and seborrheic keratosis—with specific class weights, including 2,200 training and 550 validation images. Lastly, the ISIC 2019 dataset [129,130,131], incorporating data from 2017 and 2018, features 25,331 images across nine diagnostic categories for dermoscopic image classification, with a weighted distribution to address class imbalance, divided into 20,265 training and 5,066 validation images. The dermoscopic images are distributed across eight distinct diagnostic categories: The category of Actinic keratosis (AK) includes 867 images. Basal cell carcinoma (BCC) is represented with 3,323 images. The Benign keratosis group, which encompasses solar lentigo, seborrheic keratosis, and lichen planus-like keratosis (BKL), has 2,624 images. There are 239 images for Dermatofibroma (DF). Melanoma (MEL) is well-represented with 4,522 images. A substantial number of 12,875 images are provided for Melanocytic nevus (NV). Squamous cell carcinoma (SCC) accounts for 628 images. The Vascular lesion (VASC) category includes 253 images.

4.2. Augmentation Results

Figure 2 and Figure 3 showcase samples from the dermoscopic image datasets utilized in this study focusing on skin cancer classification and melanoma detection. Figure 2 presents raw images from the dataset prior to any augmentation, providing a baseline for comparison. Figure 3 displays the same or similar images after the application of various augmentation techniques: RandAug illustrates the effects of random augmentations that apply a series of transformations to enhance generalization; AugMix demonstrates a combination of multiple augmentations to create composite images; MixUp portrays the blending of pairs of images and their labels to encourage the model to learn from interpolations; and CutMix shows examples of images where sections have been cut and pasted from one image onto another, along with the mixing of their labels. These augmentation methods are intended to enrich the dataset, improve the robustness of the model, and better simulate the variability encountered in clinical settings.

4.3. Evaluation and Computing Resources

In the context of the SkinLiTE model for skin lesion typification, the metrics of Accuracy, AUC (Area Under the Receiver Operating Characteristic Curve), and F1 score are common ways to evaluate the performance of classification models. Accuracy is the most intuitive performance measure, and it is simply a ratio of correctly predicted observation to the total observations. For instance, in the case of binary classification for benign vs. malignant, it would be the number of true positives and true negatives divided by all predictions. It is formulated as:
A c c u r a c y = N u m b e r   o f   c o r r e c t   p r e d i c t i o n s T o t a l   n u m b e r   o f   p r e d i c t i o n s
The ROC curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. The AUC represents the probability that a random positive (malignant) example is positioned more highly than a random negative (benign) example. A model with perfect discrimination (no overlap in the two distributions) has an AUC of 1.0, while a model with no discriminative power has an AUC of 0.5. The F1 score is the harmonic mean of precision and recall, and is a measure of a test’s accuracy. It considers both the precision and the recall of the test to compute the score. Precision is the number of correct positive results divided by the number of all positive results, and recall is the number of correct positive results divided by the number of positives that should have been returned. The F1 score can be formulated as:
F 1 = 2   P r e c i s i o n × R e c a l l P r e c i s i o + R e c a l l , R e c a l l = T r u e   P o s i t i v e s T r u e   P o s i t i v e s + F a l s e   N e g a t i v e s , P r e c i s i o n = T r u e   P o s i t i v e s T r u e   P o s i t i v e s + F a l s e   P o s i t i v e s
For the SkinLiTE model, which likely has to deal with multi-class classification problems given the different types of skin diseases it identifies, these metrics would be calculated for each class and then averaged (possibly in a weighted manner) to provide a single performance metric for the model. We used Google Colab with an A100 GPU to evaluate the proposed SkinLiTE model and its baselines. The A100 provides massive acceleration for all types of workloads in AI, data analytics, and high-performance computing (HPC). It greatly speeds up model training and evaluation, allowing for more iterations and potentially leading to a more accurate and robust model for skin lesion detection and classification.

5. Results and Discussion

Table 2 presents the results obtained from the traditional supervised learning classifiers and the SkinLiTE classifiers which utilize supervised contrastive learning. For each skin lesion classifier category—Bi-Classifier, Tri-Classifier, and N-Classifier—the table delineates performance metrics namely cross-entropy, accuracy, area AUC, and F1 score under various data augmentation methods, including None, RandAug, AugMix, MixUp, and CutMix. The results highlight the SkinLiTE model’s capacity to maintain higher levels of accuracy and F1 scores consistently across different augmentation strategies, underscoring the model’s robustness and its enhanced ability to handle complex classification tasks in medical imaging domains. Notably, the SkinLiTE model exhibits substantial improvements in model performance yielding 0.9087 accuracy when using AugMix to classify malignant and benign skin lesions. Similarly, the model obtained accuracy of 0.6691, 0.6561 for classifying tri-classifier and n-classifier respectively, demonstrating its effectiveness in leveraging contrastive loss to bolster diagnostic accuracy. A comparative between the baselines approaches in ISIC-2019 challenge compared with the proposed SkinLiTE’s supervised contrastive learning is demonstrated in Table 3. As can be seen, methods named Ensample-24-mcm10 and MinJie models, which utilize external data and have the highest Top-1% accuracies of 0.67 and 0.662 respectively. This suggests that the use of external data may be contributing positively to the performance of these models. The SkinLiTE model, which is proposed in this study, shows very comparable results of 0.6561 on ISIC-2019 without relying on external data. This could be significant in scenarios where the use of external data is restricted or where data privacy is a concern. SkinLiTE’s design is lighter and takes less training time yet with nearly the same performance as its precedence benchmarks.
To further elaborate our findings, Figure 4, Figure 5, and Figure 6 provide a comparative analysis of training and validation loss across different augmentation techniques for the baseline (i.e., traditional supervised classifier) and SkinLiTE classifier, including Bi-Classifier, Tri-Classifier, and N-Classifier. As can be seen from these three figures, the traditional classifiers exhibit more fluctuation in both training and validation loss across epochs for all augmentation techniques, indicating a possible overfitting issue as the model learns the training data too well, which does not generalize to the validation data. Conversely, the SkinLiTE classifiers consistently show a smoother and more stable decline in both training and validation loss, regardless of the augmentation technique used. This stability is a good indicator of the model’s ability to learn generalizable patterns from the data, which is crucial for robust performance on unseen data. The NoAug (i.e., no augmentation) scenario shows the most significant difference in loss reduction between the baseline and SkinLiTE classifiers, suggesting that SkinLiTE is particularly effective when dealing with non-augmented data. However, even with complex augmentation techniques such as RandAug, AugMix, MixUp, and CutMix, SkinLiTE maintains a steady performance, emphasizing its resilience to various data manipulations.
More attention of the accuracy results over epochs for both baseline and SkinLiTE is demonstrated in Figure 7. The left column represents the baseline model’s accuracy, showing considerable variability between training and validation accuracy, indicative of potential overfitting to the training data. This disparity suggests that while the model learns the training data well, it may not generalize effectively to new, unseen data. The right column, representing the SkinLiTE model’s performance, shows a more consistent and closer alignment between training and validation accuracy across all data augmentation methods. This demonstrates the model’s robustness and its ability to generalize from the training data to the validation data. Notably, the SkinLiTE model achieves higher validation accuracy more quickly (i.e., less epochs) and maintains it more consistently across epochs compared to the baseline model. The CutMix augmentation technique, which combines aspects of two different images, shows the most significant improvement in model performance for SkinLiTE, with the training and validation accuracy curves being almost overlapping, which is indicative of excellent model generalization. SkinLiTE N-Classifier demonstrates enhanced stability and improved accuracy across all data augmentation methods compared to the baseline, underscoring the effectiveness of the SkinLiTE architecture in skin lesion classification tasks. This is particularly evident in the model’s ability to maintain high validation accuracy, which is critical for the reliable performance of a model in practical applications.
More attention of the accuracy results over epochs for both baseline and SkinLiTE is given to the ISIC-2019 dataset which offers a total of 25,331 dermoscopic images for classification across eight distinct diagnostic categories. A comparative visualization of performance metrics—between a baseline skin lesion classification model using traditional supervised learning and the SkinLiTE model, which employs contrastive supervised learning is demonstrated in Figure 8. The top portion of the figure consists of three-line graphs, each corresponding to one of the aforementioned metrics and segmented by the type of classifier (Bi-Classifier, Tri-Classifier, N-Classifier). Across all three classifier types and metrics, SkinLiTE demonstrates consistently higher or comparable performance relative to the baseline model. This suggests that the contrastive learning approach employed by SkinLiTE contributes positively to its classification capability. The bottom portion of the figure further elucidates the comparative results between the two models using ISIC-2019. The SkinLiTE model consistently outperforms the baseline model in accuracy and AUC for most augmentation techniques, with the largest observable differences in the NoAug and CutMix scenarios. The F1 score, a measure of a test’s accuracy that considers both the precision and the recall, is notably higher in the SkinLiTE model across all data augmentation techniques except for MixUp, where the models perform similarly. These results underscore the robustness of the SkinLiTE model, particularly in its ability to effectively utilize augmented data to enhance its predictive performance. SkinLiTE model’s enhanced accuracy, AUC, and F1 score across the ISTC-2019 classification tasks demonstrate its potential as a powerful tool for skin disease classification.

6. Conclusion and Future Work

This study presents SkinLiTE, a supervised contrastive learning model designed for enhanced detection and typification of skin lesions in dermoscopic images. The prominence of SkinLiTE lies in its ability to merge the strengths of supervised learning with contrastive learning, forming an innovative approach that significantly improves the quality of learned representations. Through its two-phase learning process, SkinLiTE has demonstrated superior performance in accuracy, AUC, and F1 score across multiple datasets, notably outperforming traditional supervised learning models. The SkinLiTE classifier’s performance across various augmentation methods and classifier types in these figures suggests that it is a robust architecture capable of handling different data distributions and complexities, making it a promising tool for skin lesion classification tasks. Its lightweight architecture, combined with its efficiency and accuracy, makes it an ideal candidate for integration into remote patient monitoring and diagnostic capabilities. The ability of SkinLiTE to effectively handle imbalanced datasets further underscores its utility in medical imaging, where data scarcity and imbalance often pose significant challenges. Looking ahead, there are several areas where future research can build upon the foundations laid by this study. First, the model design can be enhanced by incorporating attention, sequence and excitation net, and feature optimization layers. Second, the use of external data and demographic information to enhance the results is highly recommended. SkinLiTE’s application could be extended to other medical imaging modalities such as MRI, CT scans, and ultrasound images. This would test the model’s adaptability and effectiveness across different types of medical data. Investigating other types of learning such as few shot and ensemble learning with SkinLiTE could open up possibilities for the model to learn from one set of data and apply its knowledge to different but related tasks or datasets, enhancing its versatility. While SkinLiTE handles imbalanced datasets effectively, further research could focus on improving its performance in extremely imbalanced scenarios, a common challenge in medical datasets.

Data Availability Statement

To promote reproducibility, we have made the source code for all the models featured in our study accessible to the public in a GitHub repository at: https://github.com/SalhaAlzahrani/SkinLiTE, allowing the wider research community to utilize and build upon our work.

Acknowledgments

The researchers would like to acknowledge the Deanship of Graduate Studies and Scientific Research, Taif University, Saudi Arabia for funding this work.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  1. Cassidy, B.; Kendrick, C.; Brodzicki, A.; Jaworek-Korjakowska, J.; Yap, M.H. Analysis of the ISIC image datasets: Usage, benchmarks and recommendations. Medical Image Analysis 2022, 75, 102305. [Google Scholar] [CrossRef]
  2. Mohanty, A.; Sutherland, A.; Bezbradica, M.; Javidnia, H. Skin Disease Analysis With Limited Data in Particular Rosacea: A Review and Recommended Framework. IEEE Access 2022, 10, 39045–39068. [Google Scholar] [CrossRef]
  3. Kumar, Y.; Koul, A.; Singla, R.; Ijaz, M.F. Artificial intelligence in disease diagnosis: a systematic literature review, synthesizing framework and future research agenda. Journal of Ambient Intelligence and Humanized Computing 2022. [Google Scholar] [CrossRef]
  4. Vatiwutipong, P.; Vachmanus, S.; Noraset, T.; Tuarob, S. Artificial Intelligence in Cosmetic Dermatology: A Systematic Literature Review. IEEE Access 2023, 11, 71407–71425. [Google Scholar] [CrossRef]
  5. Li, X.H.; Zhao, X.Y.; Ma, H.R.; Xie, B. Image Analysis and Diagnosis of Skin Diseases-A Review. Current Medical Imaging 2023, 19, 199–242. [Google Scholar] [CrossRef]
  6. Kaur, B.; Goyal, B.; Daniel, E. A Survey on Machine Learning Based Medical Assistive Systems in Current Oncological Sciences. Current Medical Imaging 2022, 18, 445–459. [Google Scholar] [CrossRef]
  7. Huérfano-Maldonado, Y.; Mora, M.; Vilches, K.; Hernández-García, R.; Gutiérrez, R.; Vera, M. A comprehensive review of extreme learning machine on medical imaging. Neurocomputing 2023, 556. [Google Scholar] [CrossRef]
  8. Mangotra, H.; Srivastava, S.; Jaiswal, G.; Rani, R.; Sharma, A. Hyperspectral imaging for early diagnosis of diseases: A review. Expert Systems 2023, 40. [Google Scholar] [CrossRef]
  9. Singh, M.; Singh, M.; De, D.; Handa, S.; Mahajan, R.; Chatterjee, D. Towards Diagnosis of Autoimmune Blistering Skin Diseases Using Deep Neural Network. Archives of Computational Methods in Engineering 2023, 30, 3529–3557. [Google Scholar] [CrossRef]
  10. Hosny, K.M.; Elshoura, D.; Mohamed, E.R.; Vrochidou, E.; Papakostas, G.A. Deep Learning and Optimization-Based Methods for Skin Lesions Segmentation: A Review. IEEE Access 2023, 11, 85467–85488. [Google Scholar] [CrossRef]
  11. Jiang, H.Y.; Diao, Z.S.; Shi, T.Y.; Zhou, Y.; Wang, F.Y.; Hu, W.R.; Zhu, X.L.; Luo, S.J.; Tong, G.Y.; Yao, Y.D. A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation. Computers in Biology and Medicine 2023, 157. [Google Scholar] [CrossRef]
  12. Noronha, S.S.; Mehta, M.A.; Garg, D.; Kotecha, K.; Abraham, A. Deep Learning-Based Dermatological Condition Detection: A Systematic Review With Recent Methods, Datasets, Challenges, and Future Directions. IEEE Access 2023, 11, 140348–140381. [Google Scholar] [CrossRef]
  13. Painuli, D.; Bhardwaj, S.; Kose, U. Recent advancement in cancer diagnosis using machine learning and deep learning techniques: A comprehensive review. Computers in Biology and Medicine 2022, 146. [Google Scholar] [CrossRef] [PubMed]
  14. Rai, H.M. Cancer detection and segmentation using machine learning and deep learning techniques: a review. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  15. Nawaz, M.; Uvaliyev, A.; Bibi, K.; Wei, H.; Abaxi, S.M.D.; Masood, A.; Shi, P.L.; Ho, H.P.; Yuan, W. Unraveling the complexity of Optical Coherence Tomography image segmentation using machine and deep learning techniques: A review. Computerized Medical Imaging and Graphics 2023, 108. [Google Scholar] [CrossRef] [PubMed]
  16. Ibrahim, A.; Mohamed, H.K.; Maher, A.; Zhang, B.C. A Survey on Human Cancer Categorization Based on Deep Learning. Frontiers in Artificial Intelligence 2022, 5. [Google Scholar] [CrossRef] [PubMed]
  17. Wang, W.-C.; Ahn, E.; Feng, D.; Kim, J. A Review of Predictive and Contrastive Self-supervised Learning for Medical Images. Machine Intelligence Research 2023, 20, 483–513. [Google Scholar] [CrossRef]
  18. Khosla, P.; Teterwak, P.; Wang, C.; Sarna, A.; Tian, Y.; Isola, P.; Maschinot, A.; Liu, C.; Krishnan, D. Supervised Contrastive Learning. 2020, arXiv:2004.11362. [CrossRef]
  19. AlDera, S.A.; Ben Othman, M.T. A Model for Classification and Diagnosis of Skin Disease using Machine Learning and Image Processing Techniques. International Journal of Advanced Computer Science and Applications 2022, 13, 252–259. [Google Scholar] [CrossRef]
  20. Hatem, M.Q. Skin lesion classification system using a K-nearest neighbor algorithm. Visual Computing for Industry Biomedicine and Art 2022, 5. [Google Scholar] [CrossRef]
  21. Saghir, U.; Hasan, M. Skin cancer detection and classification based on differential analyzer algorithm. Multimedia Tools and Applications 2023, 82, 41129–41157. [Google Scholar] [CrossRef]
  22. Mustafa, S.; Jaffar, A.; Iqbal, M.W.; Abubakar, A.; Alshahrani, A.S.; Alghamdi, A. Hybrid Color Texture Features Classification Through ANN for Melanoma. Intelligent Automation and Soft Computing 2023, 35. [Google Scholar] [CrossRef]
  23. Huong, L.H.; Khang, N.H.; Quynh, L.; Thang, L.; Canh, D.M.; Sang, H.P. A Proposed Approach for Monkeypox Classification. International Journal of Advanced Computer Science and Applications 2023, 14, 643–651. [Google Scholar] [CrossRef]
  24. MunishKhanna; Singh, L.K.; Garg, H. A novel approach for human diseases prediction using nature inspired computing & machine learning approach. Multimedia Tools and Applications 2023. [CrossRef]
  25. Meena, K.; Veni, N.N.K.; Deepapriya, B.S.; Vardhini, P.A.H.; Kalyani, B.J.D.; Sharmila, L. A novel method for prediction of skin disease through supervised classification techniques. Soft Computing 2022, 26, 10527–10533. [Google Scholar] [CrossRef]
  26. Zou, Q.X.; Cheng, J.Y.; Liang, Z.L. Automatic Diagnosis of Melanoma Based on EfficientNet and Patch Strategy. International Journal of Computational Intelligence Systems 2023, 16. [Google Scholar] [CrossRef]
  27. Spolaôr, N.; Lee, H.D.; Mendes, A.I.; Nogueira, C.V.; Parmezan, A.R.S.; Takaki, W.S.R.; Coy, C.S.R.; Wu, F.C.; Fonseca-Pinto, R. Fine-tuning pre-trained neural networks for medical image classification in small clinical datasets. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  28. Patel, M.B. Multi Class Skin Diseases Classification Based On Dermoscopic Skin Images Using Deep Learning. International Journal of Next-Generation Computing 2022, 13, 151–161. [Google Scholar] [CrossRef]
  29. Shaheen, H.; Singh, M.P. Multiclass skin cancer classification using particle swarm optimization and convolutional neural network with information security. Journal of Electronic Imaging 2023, 32. [Google Scholar] [CrossRef]
  30. Alghieth, M. Skin Disease Detection for Kids at School Using Deep Learning Techniques. International Journal of Online and Biomedical Engineering 2022, 18, 114–128. [Google Scholar] [CrossRef]
  31. Sivakumar, M.S.; Leo, L.M.; Gurumekala, T.; Sindhu, V.; Priyadharshini, A.S. Deep learning in skin lesion analysis for malignant melanoma cancer identification. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  32. Alrusaini, O.A. Deep Learning Models for the Detection of Monkeypox Skin Lesion on Digital Skin Images. International Journal of Advanced Computer Science and Applications 2023, 14, 637–644. [Google Scholar] [CrossRef]
  33. Tripathi, A.; Singh, A.K.; Singh, A.; Choudhary, A.; Pareek, K.; Mishra, K.K. Analyzing Skin Disease Using XCNN (eXtended Convolutional Neural Network). International Journal of Software Science and Computational Intelligence-Ijssci 2022, 14. [Google Scholar] [CrossRef]
  34. Bala, D.; Hossain, M.S.; Hossain, M.A.; Abdullah, M.I.; Rahman, M.M.; Manavalan, B.; Gu, N.J.; Islam, M.S.; Huang, Z.J. MonkeyNet: A robust deep convolutional neural network for monkeypox disease detection and classification. Neural Networks 2023, 161, 757–775. [Google Scholar] [CrossRef]
  35. Wei, M.J.; Wu, Q.W.; Ji, H.Y.; Wang, J.K.; Lyu, T.; Liu, J.Y.; Zhao, L. A Skin Disease Classification Model Based on DenseNet and ConvNeXt Fusion. Electronics 2023, 12. [Google Scholar] [CrossRef]
  36. El Gannour, O.; Hamida, S.; Lamalem, Y.; Cherradi, B.; Saleh, S.; Raihani, A. Enhancing Skin Diseases Classification Through Dual Ensemble Learning and Pre-trained CNNs. International Journal of Advanced Computer Science and Applications 2023, 14, 436–445. [Google Scholar] [CrossRef]
  37. Aboulmira, A.; Hrimech, H.; Lachgar, M. Comparative Study of Multiple CNN Models for Classification of 23 Skin Diseases. International Journal of Online and Biomedical Engineering 2022, 18, 127–142. [Google Scholar] [CrossRef]
  38. Anand, V.; Gupta, S.; Koundal, D.; Nayak, S.R.; Nayak, J.; Vimal, S. Multi-class Skin Disease Classification Using Transfer Learning Model. International Journal on Artificial Intelligence Tools 2022, 31. [Google Scholar] [CrossRef]
  39. Anand, V.; Gupta, S.; Nayak, S.R.; Koundal, D.; Prakash, D.; Verma, K.D. An automated deep learning models for classification of skin disease using Dermoscopy images: a comprehensive study. Multimedia Tools and Applications 2022, 81, 37379–37401. [Google Scholar] [CrossRef]
  40. Anand, V.; Gupta, S.; Koundal, D.; Mahajan, S.; Pandit, A.K.; Zaguia, A. Deep Learning Based Automated Diagnosis of Skin Diseases Using Dermoscopy. Cmc-Computers Materials & Continua 2022, 71, 3145–3160. [Google Scholar] [CrossRef]
  41. Nath, S.; Das Gupta, S.; Saha, S. Deep learning-based common skin disease image classification. Journal of Intelligent & Fuzzy Systems 2023, 44, 7483–7499. [Google Scholar] [CrossRef]
  42. Arora, G.; Dubey, A.K.; Jaffery, Z.A.; Rocha, A. A comparative study of fourteen deep learning networks for multi skin lesion classification (MSLC) on unbalanced data. Neural Computing & Applications 2023, 35, 7989–8015. [Google Scholar] [CrossRef]
  43. Al-Tuwaijari, J.M.; Yousir, N.T.; Alhammad, N.A.M.; Mostafa, S. Deep Residual Learning Image Recognition Model for Skin Cancer Disease Detection and Classification. Acta Informatica Pragensia 2023, 12, 19–31. [Google Scholar] [CrossRef]
  44. Tasar, B. SkinCancerNet: Automated Classification of Skin Lesion Using Deep Transfer Learning Method. Traitement Du Signal 2023, 40, 285–295. [Google Scholar] [CrossRef]
  45. Magdy, A.; Hussein, H.; Abdel-Kader, R.F.; Abd El Salam, K. Performance Enhancement of Skin Cancer Classification Using Computer Vision. IEEE Access 2023, 11, 72120–72133. [Google Scholar] [CrossRef]
  46. Pramila, R.P.; Subhashini, R. Automated skin lesion detection and classification using fused deep convolutional neural network on dermoscopic images. Computational Intelligence 2023. [Google Scholar] [CrossRef]
  47. Arshed, M.A.; Mumtaz, S.; Ibrahim, M.; Ahmed, S.; Tahir, M.; Shafi, M. Multi-Class Skin Cancer Classification Using Vision Transformer Networks and Convolutional Neural Network-Based Pre-Trained Models. Information 2023, 14. [Google Scholar] [CrossRef]
  48. Raghavendra, P.; Charitha, C.; Begum, K.G.; Prasath, V.B.S. Deep Learning-Based Skin Lesion Multi-class Classification with Global Average Pooling Improvement. Journal of Digital Imaging 2023, 36, 2227–2248. [Google Scholar] [CrossRef] [PubMed]
  49. Kani, M.; Parvathy, M.S.; Banu, S.M.; Kareem, M.S.A. Classification of skin lesion images using modified Inception V3 model with transfer learning and augmentation techniques. Journal of Intelligent & Fuzzy Systems 2023, 44, 4627–4641. [Google Scholar] [CrossRef]
  50. Aloraini, M. Two-stream convolutional networks for skin cancer classification. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  51. Riaz, L.; Qadir, H.M.; Ali, G.; Ali, M.; Raza, M.A.; Jurcut, A.D.; Ali, J. A Comprehensive Joint Learning System to Detect Skin Cancer. IEEE Access 2023, 11, 79434–79444. [Google Scholar] [CrossRef]
  52. Kaur, R.; Kaur, N. Ti-FCNet: Triple fused convolutional neural network-based automated skin lesion classification. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  53. Li, H.Y.; Zhang, P.; Wei, Z.K.; Qian, T.; Tang, Y.Q.; Hu, K.; Huang, X.Q.; Xia, X.X.; Zhang, Y.S.; Cheng, H.X.; et al. Deep skin diseases diagnostic system with Dual-channel Image and Extracted Text. Frontiers in Artificial Intelligence 2023, 6. [Google Scholar] [CrossRef]
  54. Li, S.H.; Li, X.M.; Xu, X.W.; Cheng, K.T. Dynamic Subcluster-Aware Network for Few-Shot Skin Disease Classification. IEEE Transactions on Neural Networks and Learning Systems 2023. [Google Scholar] [CrossRef]
  55. Reddy, D.A.; Roy, S.; Kumar, S.; Tripathi, R. Enhanced U-Net segmentation with ensemble convolutional neural network for automated skin disease classification. Knowledge and Information Systems 2023, 65, 4111–4156. [Google Scholar] [CrossRef]
  56. Josphineleela, R.; Rao, P.; Shaikh, A.; Sudhakar, K. A Multi-Stage Faster RCNN-Based iSPLInception for Skin Disease Classification Using Novel Optimization. Journal of Digital Imaging 2023, 36, 2210–2226. [Google Scholar] [CrossRef] [PubMed]
  57. Sreekala, K.; Rajkumar, N.; Sugumar, R.; Sagar, K.V.D.; Shobarani, R.; Krishnamoorthy, K.P.; Saini, A.K.; Palivela, H.; Yeshitla, A. Skin Diseases Classification Using Hybrid AI Based Localization Approach. Computational Intelligence and Neuroscience 2022, 2022. [Google Scholar] [CrossRef] [PubMed]
  58. He, X.Y.; Wang, Y.; Zhao, S.; Chen, X. Joint segmentation and classification of skin lesions via a multi-task learning convolutional neural network. Expert Systems with Applications 2023, 230. [Google Scholar] [CrossRef]
  59. Alam, M.J.; Mohammad, M.S.; Hossain, M.A.F.; Showmik, I.A.; Raihan, M.S.; Ahmed, S.; Ibn Mahmud, T. S-2-C-DeLeNet: A parameter transfer based segmentation-classification integration for detecting skin cancer lesions from dermoscopic images. Computers in Biology and Medicine 2022, 150. [Google Scholar] [CrossRef] [PubMed]
  60. Dong, C.X.; Dai, D.W.; Zhang, Y.Z.; Zhang, C.Y.; Li, Z.F.; Xu, S.H. Learning from dermoscopic images in association with clinical metadata for skin lesion segmentation and classification. Computers in Biology and Medicine 2023, 152. [Google Scholar] [CrossRef]
  61. Yadav, A.K.; Mehta, R.; Kumar, V.; Medikondu, N.R. An optimized boosting framework for skin lesion segmentation and classification. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  62. Duman, E.; Tolan, Z. Ensemble the recent architectures of deep convolutional networks for skin diseases diagnosis. International Journal of Imaging Systems and Technology 2023, 33, 1293–1305. [Google Scholar] [CrossRef]
  63. Kalaivani, A.; Karpagavalli, S. Advanced domain adaptation for skin disease segmentation and classification using bootstrapping of fine-tuned deep learner. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  64. Ramamurthy, K.; Muthuswamy, A.; Mathimariappan, N.; Kathiresan, G.S. A novel two-staged network for skin disease detection using atrous residual convolutional networks. Concurrency and Computation-Practice & Experience 2023, 35. [Google Scholar] [CrossRef]
  65. Rao, G.M.; Ramesh, D.; Gantela, P.; Srinivas, K. A hybrid deep learning strategy for image based automated prognosis of skin disease. Soft Computing 2023. [Google Scholar] [CrossRef]
  66. Manoj, S.O.; Abirami, K.R.; Victor, A.; Arya, M. Automatic Detection and Categorization of Skin Lesions for Early Diagnosis of Skin Cancer Using YOLO-v3 - DCNN Architecture. Image Analysis & Stereology 2023, 42, 101–117. [Google Scholar] [CrossRef]
  67. Hao, S.N.; Zhang, L.G.; Jiang, Y.Y.; Wang, J.K.; Ji, Z.L.; Zhao, L.; Ganchev, I. ConvNeXt-ST-AFF: A Novel Skin Disease Classification Model Based on Fusion of ConvNeXt and Swin Transformer. IEEE Access 2023, 11, 117460–117473. [Google Scholar] [CrossRef]
  68. Kalyani, K.; Althubiti, S.A.; Ahmed, M.A.; Lydia, E.L.; Kadry, S.; Han, N.; Nam, Y. Arithmetic Optimization with Ensemble Deep Transfer Learning Based Melanoma Classification. Cmc-Computers Materials & Continua 2023, 75, 149–164. [Google Scholar] [CrossRef]
  69. Seeja, R.D.; Geetha, A. Effective melanoma classification using inter neighbour mean order interleaved pattern on dermoscopy images. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  70. Albraikan, A.A.; Nemri, N.; Alkhonaini, M.A.; Hilal, A.M.; Yaseen, I.; Motwakel, A. Automated Deep Learning Based Melanoma Detection and Classification Using Biomedical Dermoscopic Images. Cmc-Computers Materials & Continua 2023, 74, 2443–2459. [Google Scholar] [CrossRef]
  71. Meswal, H.; Kumar, D.; Gupta, A.; Roy, S. A weighted ensemble transfer learning approach for melanoma classification from skin lesion images. Multimedia Tools and Applications 2023. [Google Scholar] [CrossRef]
  72. Asif, S.; Zhao, M.; Tang, F.X.; Zhu, Y.S.; Zhao, B.K. Metaheuristics optimization-based ensemble of deep neural networks for Mpox disease detection. Neural Networks 2023, 167, 342–359. [Google Scholar] [CrossRef]
  73. Aloraini, M. An effective human monkeypox classification using vision transformer. International Journal of Imaging Systems and Technology 2023. [Google Scholar] [CrossRef]
  74. Ren, G.Y. Monkeypox Disease Detection with Pretrained Deep Learning Models. Information Technology and Control 2023, 52, 288–296. [Google Scholar] [CrossRef]
  75. Ahsan, M.M.; Uddin, M.R.; Ali, M.S.; Islam, M.K.; Farjana, M.; Sakib, A.N.; Al Momin, K.; Luna, S.A. Deep transfer learning approaches for Monkeypox disease diagnosis. Expert Systems with Applications 2023, 216. [Google Scholar] [CrossRef]
  76. Bian, X.F.; Pan, H.W.; Zhang, K.J.; Li, P.Y.; Li, J.B.; Chen, C.L. Skin lesion image classification method based on extension theory and deep learning. Multimedia Tools and Applications 2022, 81, 16389–16409. [Google Scholar] [CrossRef]
  77. Radhika, V.; Chandana, B.S. MSCDNet-based multi-class classification of skin cancer using dermoscopy images. Peerj Computer Science 2023, 9. [Google Scholar] [CrossRef]
  78. Ghosh, A.; Jana, N.D.; Das, S.; Mallipeddi, R. Two-Phase Evolutionary Convolutional Neural Network Architecture Search for Medical Image Classification. IEEE Access 2023, 11, 115280–115305. [Google Scholar] [CrossRef]
  79. Rajeshwari, J.; Sughasiny, M. Modified Filter Based Feature Selection Technique for Dermatology Dataset Using Beetle Swarm Optimization. Eai Endorsed Transactions on Scalable Information Systems 2023, 10. [Google Scholar] [CrossRef]
  80. Mostafa, R.R.; Khedr, A.M.; Al Aghbari, Z.; Afyouni, I.; Kamel, I.; Ahmed, N. An adaptive hybrid mutated differential evolution feature selection method for low and high-dimensional medical datasets. Knowledge-Based Systems 2024, 283. [Google Scholar] [CrossRef]
  81. Zhou, J.C.; Wu, Z.; Jiang, Z.X.; Huang, K.; Guo, K.H.; Zhao, S. Background selection schema on deep learning-based classification of dermatological disease. Computers in Biology and Medicine 2022, 149. [Google Scholar] [CrossRef] [PubMed]
  82. Jain, A.; Rao, A.C.S.; Jain, P.K.; Abraham, A. Multi-type skin diseases classification using OP-DNN based feature extraction approach. Multimedia Tools and Applications 2022, 81, 6451–6476. [Google Scholar] [CrossRef] [PubMed]
  83. Liu, N.N.; Rejeesh, M.R.; Sundararaj, V.; Gunasundari, B. ACO-KELM: Anti Coronavirus Optimized Kernel-based Softplus Extreme Learning Machine for classification of skin cancer. Expert Systems with Applications 2023, 232. [Google Scholar] [CrossRef]
  84. Vidhyalakshmi, A.M.; Kanchana, M. Skin cancer classification using improved transfer learning model-based random forest classifier and golden search optimization. International Journal of Imaging Systems and Technology 2023. [Google Scholar] [CrossRef]
  85. Renith, G.; Senthilselvi, A. An efficient skin cancer detection and classification using Improved Adaboost Aphid-Ant Mutualism model. International Journal of Imaging Systems and Technology 2023, 33, 1957–1972. [Google Scholar] [CrossRef]
  86. Desale, R.P.; Patil, P.S. An efficient multi-class classification of skin cancer using optimized vision transformer. Medical & Biological Engineering & Computing 2023. [Google Scholar] [CrossRef]
  87. Villa-Pulgarin, J.P.; Ruales-Torres, A.A.; Arias-Garzón, D.; Bravo-Ortiz, M.A.; Arteaga-Arteaga, H.B.; Mora-Rubio, A.; Alzate-Grisales, J.A.; Mercado-Ruiz, E.; Hassaballah, M.; Orozco-Arias, S.; et al. Optimized Convolutional Neural Network Models for Skin Lesion Classification. Cmc-Computers Materials & Continua 2022, 70, 2131–2148. [Google Scholar] [CrossRef]
  88. Anand, V.; Gupta, S.; Koundal, D.; Singh, K. Fusion of U-Net and CNN model for segmentation and classification of skin lesion from dermoscopy images. Expert Systems with Applications 2023, 213. [Google Scholar] [CrossRef]
  89. Adepu, A.K.; Sahayam, S.; Jayaraman, U.; Arramraju, R. Melanoma classification from dermatoscopy images using knowledge distillation for highly imbalanced data. Computers in Biology and Medicine 2023, 154. [Google Scholar] [CrossRef] [PubMed]
  90. El Saleh, R.; Chantaf, S.; Nait-ali, A. Identification of facial skin diseases from face phenotypes using FSDNet in uncontrolled environment. Machine Vision and Applications 2022, 33. [Google Scholar] [CrossRef]
  91. Bozkurt, F. Skin lesion classification on dermatoscopic images using effective data augmentation and pre-trained deep learning approach. Multimedia Tools and Applications 2023, 82, 18985–19003. [Google Scholar] [CrossRef]
  92. Pokhrel, K.; Sanin, C.; Sakib, M.K.H.; Islam, M.R.; Szczerbicki, E. Improved Skin Disease Classification with Mask R-CNN and Augmented Dataset. Cybernetics and Systems 2023. [Google Scholar] [CrossRef]
  93. Holmes, K.; Sharma, P.; Fernandes, S. Facial skin disease prediction using StarGAN v2 and transfer learning. Intelligent Decision Technologies-Netherlands 2023, 17, 55–66. [Google Scholar] [CrossRef]
  94. Rezk, E.; Eltorki, M.; El-Dakhakhni, W. Interpretable Skin Cancer Classification based on Incremental Domain Knowledge Learning. Journal of Healthcare Informatics Research 2023, 7, 59–83. [Google Scholar] [CrossRef] [PubMed]
  95. Hoang, V.; Vo, X.T.; Jo, K.H. Categorical Weighting Domination for Imbalanced Classification With Skin Cancer in Intelligent Healthcare Systems. IEEE Access 2023, 11, 105170–105181. [Google Scholar] [CrossRef]
  96. Yu, L.S.; Wang, Y.F.; Zhou, L.Y.; Wu, J.S.; Wang, Z.H. Residual neural network-assisted one-class classification algorithm for melanoma recognition with imbalanced data. Computational Intelligence 2023. [Google Scholar] [CrossRef]
  97. Chen, K.H.; Lei, W.X.; Zhao, S.; Zheng, W.S.; Wang, R.X. PCCT: Progressive Class-Center Triplet Loss for Imbalanced Medical Image Classification. IEEE Journal of Biomedical and Health Informatics 2023, 27, 2026–2036. [Google Scholar] [CrossRef] [PubMed]
  98. Okuboyejo, D.A.; Olugbara, O.O. Classification of Skin Lesions Using Weighted Majority Voting Ensemble Deep Learning. Algorithms 2022, 15. [Google Scholar] [CrossRef]
  99. Vidhyalakshmi, A.M.; Kanchana, M. Classification of skin disease using a novel hybrid flash butterfly optimization from dermoscopic images. Neural Computing & Applications 2023. [Google Scholar] [CrossRef]
  100. Huang, Z.Z.; Wu, J.W.; Wang, T.; Li, Z.Y.; Ioannou, A. Class-Specific Distribution Alignment for semi-supervised medical image classification. Computers in Biology and Medicine 2023, 164. [Google Scholar] [CrossRef]
  101. Zhuang, J.X.; Cai, J.B.; Zhang, J.G.; Zheng, W.S.; Wang, R.X. Class attention to regions of lesion for imbalanced medical image recognition. Neurocomputing 2023, 555. [Google Scholar] [CrossRef]
  102. Yao, P.; Shen, S.W.; Xu, M.J.; Liu, P.; Zhang, F.; Xing, J.Y.; Shao, P.F.; Kaffenberger, B.; Xu, R.X. Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification. IEEE Transactions on Medical Imaging 2022, 41, 1242–1254. [Google Scholar] [CrossRef]
  103. Alshawi, S.A.; Musawi, G. Skin Cancer Image Detection and Classification by CNN based Ensemble Learning. International Journal of Advanced Computer Science and Applications 2023, 14, 710–717. [Google Scholar] [CrossRef]
  104. Yue, G.H.; Wei, P.S.; Zhou, T.W.; Jiang, Q.P.; Yan, W.Q.; Wang, T.F. Toward Multicenter Skin Lesion Classification Using Deep Neural Network With Adaptively Weighted Balance Loss. IEEE Transactions on Medical Imaging 2023, 42, 119–131. [Google Scholar] [CrossRef] [PubMed]
  105. Omeroglu, A.N.; Mohammed, H.M.A.; Oral, E.A.; Aydin, S. A novel soft attention-based multi-modal deep learning framework for multi-label skin lesion classification. Engineering Applications of Artificial Intelligence 2023, 120. [Google Scholar] [CrossRef]
  106. Zhang, R.Z.; Wang, L.J.; Cheng, S.L.; Song, S.J. MLP-based classification of COVID-19 and skin diseases. Expert Systems with Applications 2023, 228. [Google Scholar] [CrossRef]
  107. Desale, R.P.; Patil, P.S. An automated hybrid attention based deep convolutional capsule with weighted autoencoder approach for skin cancer classification. Imaging Science Journal 2023. [Google Scholar] [CrossRef]
  108. Bao, Q.W.; Han, H.; Huang, L.; Muzahid, A.A.M. A Convolutional Neural Network Based on Soft Attention Mechanism and Multi-Scale Fusion for Skin Cancer Classification. International Journal of Pattern Recognition and Artificial Intelligence 2023, 37. [Google Scholar] [CrossRef]
  109. To, H.D.; Nguyen, H.G.; Le, H.T.T.; Le, H.M.; Quan, T.T. MetaAttention model: a new approach for skin lesion diagnosis using AB features and attention mechanism. Biomedical Physics & Engineering Express 2023, 9. [Google Scholar] [CrossRef]
  110. Surati, S.; Trivedi, H.; Shrimali, B.; Bhatt, C.; Travieso-González, C.M. An Enhanced Diagnosis of Monkeypox Disease Using Deep Learning and a Novel Attention Model Senet on Diversified Dataset. Multimodal Technologies and Interaction 2023, 7. [Google Scholar] [CrossRef]
  111. Damineni, D.H.; Sekharamantry, P.K.; Badugu, R. An Adaptable Model for Medical Image Classification Using the Streamlined Attention Mechanism. International Journal of Online and Biomedical Engineering 2023, 19, 93–110. [Google Scholar] [CrossRef]
  112. Ashwath, V.A.; Sikha, O.K.; Benitez, R. TS-CNN: A Three-Tier Self-Interpretable CNN for Multi-Region Medical Image Classification. IEEE Access 2023, 11, 78402–78418. [Google Scholar] [CrossRef]
  113. Ren, Y.G.; Xu, W.Q.; Mao, Y.X.; Wu, Y.C.; Fu, B.; Thanh, D.N.H. Few-shot learning for dermatological conditions with Lesion Area Aware Swin Transformer. International Journal of Imaging Systems and Technology 2023, 33, 1549–1560. [Google Scholar] [CrossRef]
  114. Zhang, D.Z.; Li, A.L.; Wu, W.D.; Yu, L.; Kang, X.J.; Huo, X.Z. CR-Conformer: a fusion network for clinical skin lesion classification. Medical & Biological Engineering & Computing 2023. [Google Scholar] [CrossRef]
  115. Shaik, T.; Tao, X.H.; Higgins, N.; Li, L.; Gururajan, R.; Zhou, X.J.; Acharya, U.R. Remote patient monitoring using artificial intelligence: Current state, applications, and challenges. Wiley Interdisciplinary Reviews-Data Mining and Knowledge Discovery 2023, 13. [Google Scholar] [CrossRef]
  116. Oztel, I.; Oztel, G.Y.; Sahin, V.H. Deep Learning-Based Skin Diseases Classification using Smartphones. Advanced Intelligent Systems 2023. [Google Scholar] [CrossRef]
  117. Yang, Y.G.; Xie, F.Y.; Zhang, H.P.; Wang, J.C.; Liu, J.; Zhang, Y.L.; Ding, H.D. Skin lesion classification based on two-modal images using a multi-scale fully-shared fusion network. Computer Methods and Programs in Biomedicine 2023, 229. [Google Scholar] [CrossRef]
  118. Manzoor, K.; Majeed, F.; Siddique, A.; Meraj, T.; Rauf, H.T.; El-Meligy, M.A.; Sharaf, M.; Abd Elgawad, A.E. A Lightweight Approach for Skin Lesion Detection Through Optimal Features Fusion. Cmc-Computers Materials & Continua 2022, 70, 1617–1630. [Google Scholar] [CrossRef]
  119. Vivekananda, G.N.; Almufti, S.M.; Suresh, C.; Samsudeen, S.; Devarajan, M.V.; Srikanth, R.; Jayashree, S. Retracing-efficient IoT model for identifying the skin-related tags using automatic lumen detection. Intelligent Data Analysis 2023, 27, S161–S180. [Google Scholar] [CrossRef]
  120. Shahin, M.; Chen, F.F.; Hosseinzadeh, A.; Koodiani, H.K.; Shahin, A.; Nafi, O.A. A smartphone-based application for an early skin disease prognosis: Towards a lean healthcare system via computer-based vision. Advanced Engineering Informatics 2023, 57. [Google Scholar] [CrossRef]
  121. Kumar, K.A.; Satheesha, T.Y.; Salvador, B.B.L.; Mithileysh, S.; Ahmed, S.T. Augmented Intelligence enabled Deep Neural Networking (AuDNN) framework for skin cancer classification and prediction using multi-dimensional datasets on industrial IoT standards. Microprocessors and Microsystems 2023, 97. [Google Scholar] [CrossRef]
  122. Shi, Y.H.; Li, X.; Chen, S.G. Skin Lesion Intelligent Diagnosis in Edge Computing Networks: An FCL Approach. Acm Transactions on Intelligent Systems and Technology 2023, 14. [Google Scholar] [CrossRef]
  123. Gupta, A.; Bhagat, M.; Jain, V. Blockchain-enabled healthcare monitoring system for early Monkeypox detection. Journal of Supercomputing 2023, 79, 15675–15699. [Google Scholar] [CrossRef] [PubMed]
  124. Hossen, M.N.; Panneerselvam, V.; Koundal, D.; Ahmed, K.; Bui, F.M.; Ibrahim, S.M. Federated Machine Learning for Detection of Skin Diseases and Enhancement of Internet of Medical Things (IoMT) Security. IEEE Journal of Biomedical and Health Informatics 2023, 27, 835–841. [Google Scholar] [CrossRef] [PubMed]
  125. Hendrycks, D.; Mu, N.; Cubuk, E.D.; Zoph, B.; Gilmer, J.; Lakshminarayanan, B. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty. 2019, arXiv:1912.02781. [CrossRef]
  126. Zhang, H.; Cisse, M.; Yann; Lopez-Paz, D. mixup: Beyond Empirical Risk Minimization. arXiv pre-print server 2018, doi:arxiv:1710.09412. [CrossRef]
  127. Yun, S.; Han, D.; Chun, S.; Oh, S.J.; Yoo, Y.; Choe, J. CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV); 2019; pp. 6022–6031. [Google Scholar]
  128. Rotemberg, V.; Kurtansky, N.; Betz-Stablein, B.; Caffery, L.; Chousakos, E.; Codella, N.; Combalia, M.; Dusza, S.; Guitera, P.; Gutman, D.; et al. A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific Data 2021, 8, 34. [Google Scholar] [CrossRef] [PubMed]
  129. Codella, N.C.F.; Gutman, D.; Emre Celebi, M.; Helba, B.; Marchetti, M.A.; Dusza, S.W.; Kalloo, A.; Liopyris, K.; Mishra, N.; Kittler, H.; Halpern, A. Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC). 2017, arXiv:1710.05006. 0500. [Google Scholar] [CrossRef]
  130. Tschandl, P.; Rosendahl, C.; Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 2018, 5, 180161. [Google Scholar] [CrossRef]
  131. Combalia, M.; Codella, N.C.F.; Rotemberg, V.; Helba, B.; Vilaplana, V.; Reiter, O.; Carrera, C.; Barreiro, A.; Halpern, A.C.; Puig, S.; Malvehy, J. BCN20000: Dermoscopic Lesions in the Wild. 2019, arXiv:1908.02288. [CrossRef]
  132. Singh, P.; Sizikova, E.; Cirrone, J. CASS: Cross Architectural Self-Supervision for Medical Image Analysis. 2022, arXiv:2206.04170. [CrossRef]
Preprints 100413 g001
Figure 1. The overall architecture of the proposed model (SkinLiTE) for skin lesions and diseases detection and typification.
Figure 1. The overall architecture of the proposed model (SkinLiTE) for skin lesions and diseases detection and typification.
Preprints 100413 g001
Preprints 100413 g002
Figure 2. Sample from the dermoscopic image datasets for skin cancer classification and melanoma detection used in this study (before augmentation).
Figure 2. Sample from the dermoscopic image datasets for skin cancer classification and melanoma detection used in this study (before augmentation).
Preprints 100413 g002
Preprints 100413 g003aPreprints 100413 g003b
Figure 3. Sample from the dermoscopic image datasets for skin cancer classification and melanoma detection used in this study (after augmentation).
Figure 3. Sample from the dermoscopic image datasets for skin cancer classification and melanoma detection used in this study (after augmentation).
Preprints 100413 g003aPreprints 100413 g003b
Preprints 100413 g004
Figure 4. Comparative analysis of training and validation loss across different augmentation techniques for Baseline and SkinLiTE Bi-Classifiers.
Figure 4. Comparative analysis of training and validation loss across different augmentation techniques for Baseline and SkinLiTE Bi-Classifiers.
Preprints 100413 g004
Preprints 100413 g005
Figure 5. Comparative analysis of training and validation loss across different augmentation techniques for Baseline and SkinLiTE Tri-Classifiers.
Figure 5. Comparative analysis of training and validation loss across different augmentation techniques for Baseline and SkinLiTE Tri-Classifiers.
Preprints 100413 g005
Preprints 100413 g006
Figure 6. Comparative analysis of training and validation loss across different augmentation techniques for Baseline and SkinLiTE N-Classifier.
Figure 6. Comparative analysis of training and validation loss across different augmentation techniques for Baseline and SkinLiTE N-Classifier.
Preprints 100413 g006
Preprints 100413 g007aPreprints 100413 g007b
Figure 7. Comparative analysis of accuracy results over epochs for baseline and SkinLiTE with ISIC-2019 dataset with various data augmentation techniques. The left column represents the baseline model’s performance, while the right column showcases the SkinLiTE model’s enhanced stability and improved accuracy across all data augmentation methods, demonstrating the effectiveness of the SkinLiTE architecture in skin lesion classification tasks.
Figure 7. Comparative analysis of accuracy results over epochs for baseline and SkinLiTE with ISIC-2019 dataset with various data augmentation techniques. The left column represents the baseline model’s performance, while the right column showcases the SkinLiTE model’s enhanced stability and improved accuracy across all data augmentation methods, demonstrating the effectiveness of the SkinLiTE architecture in skin lesion classification tasks.
Preprints 100413 g007aPreprints 100413 g007b
Preprints 100413 g008
Figure 8. Visualization of the accuracy, AUC and F1 score results obtained from the baseline (traditional supervised classification) SkinLiTE (contrastive supervised classification) with various data augmentation techniques. The bottom part of the figure showcases the SkinLiTE model’s enhanced accuracy, AUC, and F1 using ISIC-2019 classification tasks.
Figure 8. Visualization of the accuracy, AUC and F1 score results obtained from the baseline (traditional supervised classification) SkinLiTE (contrastive supervised classification) with various data augmentation techniques. The bottom part of the figure showcases the SkinLiTE model’s enhanced accuracy, AUC, and F1 using ISIC-2019 classification tasks.
Preprints 100413 g008
Table 1. Details of dermoscopic image datasets for skin cancer classification and melanoma detection used in this study.
Table 1. Details of dermoscopic image datasets for skin cancer classification and melanoma detection used in this study.
Datasets Desc. No. class Weighting per class No. train No. val Total
Skin Cancer ISIC 2019 & 2020 Malignant or Benign [128] The dataset was compiled using images from the ISIC 2019 and ISIC 2020 collections, which were sourced from the International Society for Digital Imaging of the Skin. It comprises a total of 11,396 images, with 5,096 categorized as benign tumors and 6,300 categorized as malignant tumors. 2 {0: 0.9044,
1: 1.1181}		 9,117 2,279 11,396 images
Melanoma Detection Dataset [129] The objective of this dataset is to facilitate the research and creation of automated systems for the diagnosis of melanoma, which is the deadliest form of skin cancer. The collection of labeled images includes 2,750 images in total, of which 521 are identified as melanoma, 1,843 as nevus, and 386 as seborrheic keratosis. 3 {0: 1.7594,
1: 0.4973,
2: 2.3747}		 2200 550 2,750
ISIC 2019 Skin Lesion images for classification
[129,130,131] 		The dataset encompasses the training data for the ISIC 2019 competition, which also incorporates data from the 2018 and 2017 challenges. The ISIC 2019 dataset offers a total of 25,331 dermoscopic images for classification across eight distinct diagnostic categories:
- Actinic keratosis (AK): 867 images.
- Basal cell carcinoma (BCC): 3,323 images.
- Benign keratosis group (BKL): 2,624 images.
- Dermatofibroma (DF).239 images.
- Melanoma (MEL): 4,522 images.
- Melanocytic nevus (NV): 12,875 images.
- Squamous cell carcinoma (SCC): 628 images.
- Vascular lesion (VASC): 253 images.		 8 {0: 3.6521,
1: 0.9528,
2: 1.2066,
3: 13.2484,
4: 0.7002,
5: 0.2459,
6: 5.0419,
7: 12.515}		 20265 5066 25,331 images
Table 2. Comparative performance metrics of traditional supervised learning versus SkinLiTE’s supervised contrastive learning for binary, triple, and multiple skin lesion classification datasets.
Table 2. Comparative performance metrics of traditional supervised learning versus SkinLiTE’s supervised contrastive learning for binary, triple, and multiple skin lesion classification datasets.
Skin Lesions Model Aug. Method Traditional classifier (Supervised Learning) SkinLiTE (Supervised Contrastive Learning)
Cross-entropy Accuracy AUC F1
score		 Contrastive loss Cross-entropy Accuracy AUC F1
score
Bi-Classifier None 0.3430 0.8956 0.9106 0.8926 2.8242 0.4062 0.8416 0.8808 0.8415
RandAug 0.3805 0.8311 0.8624 0.8164 3.2218 0.5130 0.7231 0.6770 0.7171
AugMix 0.2305 0.9096 0.9351 0.9064 3.0800 0.4035 0.9087 0.9264 0.9067
MixUp 0.5453 0.7793 0.8057 0.7508 2.9713 0.3774 0.8771 0.7425 0.8706
CutMix 0.2972 0.8881 0.9127 0.8830 3.2412 0.4894 0.8091 0.5810 0.8089
Tri-Classifier None 1.0733 0.4291 0.5707 0.3786 2.8309 1.0139 0.5200 0.6497 0.3965
RandAug 1.0937 0.3818 0.5898 0.3210 3.4490 1.1055 0.1945 0.3957 0.1086
AugMix 0.9353 0.4891 0.6743 0.4800 3.4023 0.9751 0.5273 0.6469 0.4446
MixUp 0.9863 0.4891 0.6467 0.4385 3.3382 1.1188 0.1945 0.4290 0.1086
CutMix 0.9240 0.6109 0.7104 0.4550 3.4335 1.0548 0.6691 0.5830 0.2672
N-Classifier None 1.3930 0.4974 0.7638 0.3063 2.1985 1.9215 0.5626 0.7693 0.3382
RandAug 1.8438 0.2777 0.6546 0.1265 3.2310 1.7804 0.4345 0.7197 0.1181
AugMix 1.6061 0.4311 0.7260 0.2101 2.8542 1.7088 0.5034 0.6576 0.3443
MixUp 1.8259 0.3121 0.6326 0.1754 2.5672 1.2964 0.5805 0.7900 0.3810
CutMix 1.7430 0.4501 0.7477 0.1588 3.1836 1.2298 0.6561 0.7658 0.4126
The table illustrates the cross-entropy, accuracy, AUC, and F1 score for each model under different augmentation methods: None, RandAug, AugMix, MixUp, and CutMix. The SkinLiTE model demonstrates varying degrees of improvement in accuracy, AUC, and F1 scores across different augmentation techniques, highlighting its effectiveness in skin lesion classification tasks.
Table 3. Rank of benchmark approaches used for ISIC-2019 challenge for skin lesion diagnosis compared with SkinLiTE’s performance.
Table 3. Rank of benchmark approaches used for ISIC-2019 challenge for skin lesion diagnosis compared with SkinLiTE’s performance.
Model Name Manuscript Use External Data Top-1%
Ensample-24-mcm10 (https://challenge.isic-archive.com/leaderboards/live/) No Yes 0.670
MinJie (https://challenge.isic-archive.com/leaderboards/live/) No Yes 0.662
SkinLiTE (proposed) Yes No 0.656
CASS: Cross Architectural Self-Supervision for Medical Image Analysis [132] Yes Yes 0.652
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated