1. Introduction

2. Significance of the Research

• Improved Security and Compliance: The proposed research work provides huge potential considering the privacy and security of medical imaging. The research follows the guidelines as per GDPR and DPA for data security that will be a great revolution in the medical industry.
• Enhanced diagnostic capacity: The research framework that constitutes the hybrid model that will ensure the security of data and the accuracy efficiency of disease detection that will ultimately result in early diagnosis and treatment.
• Facilitating Collaboration: Research will promote an innovative culture that allows the mutual collaboration of hospitals and medical institutes to be joined to achieve the improved advancements.
• Benchmark for Potential Innovation: Based on the study analysis, it will guide the future scope of innovation in medical imaging for researchers. The proposed research can be a benchmark for the future development of the idea that can be mutually beneficial for medical institutes.
• Scalable and Flexible Framework: The research illustrates the use of a CNN-based pre-trained model on the federated learning framework that is highly scalable towards the multiple of type of medical images. Provides a robust and enhanced solution for medical image detection.
• Economic Influence: Research will bring important changes to ease the economic impact such as early detection. It will ultimately bring about early disease detection saving cost and resources in the medical industry.
• Global extent and convenience: Using the FL framework as in the proposed research will ensure data privacy, allowing data from diverse sources to enhance machine learning models' learning capability.

3. Literature Review

4. Methodology

4.1. Research Model Design

The conventional system adopted by medical institutions and hospitals abides by data protection and privacy laws. This are why data is not shared with other institutions or any third party due to GDPR. Therefore, in our FL framework, data privacy is ensured while model training is performed in multiple medical institutes and hospitals without sharing data. In medical image detection, this approach is effective as it helps train the ML model from various medical centres; in this way, a significant amount of heterogeneous real-time data is used, which ultimately helps the ML model for effective training. The framework of using FL with the CNN-based pre-trained models is selected based on previous work on medical image detection. As mentioned in the literature review, the performance of the ML model is enhanced if the model is trained with a larger amount of data. In other words, fewer data produces ineffective performance in ML model training and vice versa. Therefore, a collaborative way of using the dataset's multiple sources is required, which can be possible by mutual collaboration. In this way, the model learns the patterns of the image data and helps to form an effective model for detecting diseases from pool of dataset. The image below illustrates our research model design:

Figure 1. Proposed Framework.

Figure 1. Proposed Framework.

As elaborated in the figure above, our research workflow follows the data collection. The collected dataset was pre-processed by resizing and removing any irregularities. Once the data are pre-processed, it is split into training, validation, and testing with a ratio of 70%, 20%, and 10%, respectively. The training dataset is used for applying the machine learning model after processing which is sent across the participants for model training in the above-mentioned federated learning framework. Once the model is locally trained on the local devices, the individual trained model is shared with the FL server (central server), forming an updated model after aggregation. Secure aggregation is in place in this model design to ensure the security of the trained model. The performance of the model is evaluated with the validation and testing data set. Training the ML model on the individual devices and the FL server is an iterative process to get more updates and correspondence from the dataset on the client side. As our research framework is based on FL, let us explore the fundamental concepts behind FL.

We have used FL architecture in our research to demonstrate the privacy-preserving approach that allows for mutual collaboration between hospitals and medical institutions. A standard approach is required to agree by participating parties that includes the model framework, loss, and activation functions. In general, the FL architecture can be illustrated as follows:

$$\underset{w\in {\mathbb{R}}^d}{min} \mathcal{l}(x,y;w)\mathrm{where}\mathcal{l}(x,y;w)\stackrel{\mathrm{def}}{=}\frac{1}{n}\sum _{i=1}^n \mathcal{l}\left(x_i,y_i;w\right)$$

(1)

To understand the architecture of FL, let us take hospitals ‘C’ comprising the dataset ‘Di = nc’, ‘n’ shows the quantity of data, and the FL architecture in the individual hospital can be illustrated as follows:

$$\mathcal{l}(x,y;w)=\mathrm{where}{L}_c\left(x_c,y_c;w\right)=\frac{1}{n_c}\sum _{i\in D_i} \mathcal{l}\left(x_i^c,y_i^c;w\right)$$

(2)

The FL global server inputs the model that consists of different parameters for medical image detection. During the individual cycle with the global server, random participants are considered. Generate the one-to-one link with the server. The local participant downloads the model from the local server that calculates the average gradient based on the loss ‘fc’ which can be demonstrated based on the weight 'wt' that constitutes the learning rate ‘η’. In this way, local participant keeps updating and at the same time keeps updating the central server. The aggregated model receiving the updates from the participants can be calculated as follows:

$$\begin{array}{c}{w}_{t+1}\leftarrow w_t-\eta \nabla l(x,y;w)\\ w_{t+1}\leftarrow w_t-\eta \sum _{c=1}^C \frac{n_c}{n}\nabla L_c\left(x_c,y_c;w\right)\end{array}$$

(3)

$$w_{t+1}\leftarrow w_t-\eta \frac{n_c}{n}{f}_c$$

(4)

For every hospital c, $w_{t+1}^c\leftarrow w_t-\eta f_c$, then

$$w_{t+1}\leftarrow w_t-\sum _{c=1}^C \frac{n_c}{n}{w}_{t+1}^c$$

(5)

The various sizes of the data set at each round help to improve the model leaning capability in terms of detection $w_{t+1}^c$ which can be quite useful for the skewed data set and can be illustrated in the following equation:

$$w_{t+1}\leftarrow w_t-\sum _{c=1}^C \frac{n_c}{n}{\alpha }_{t+1}^c{w}_{t+1}^c$$

(6)

The concept of increasing the weight parameter can result in better performance. The averaging of the weight from the distinct client tends to improve the learning capabilities of the model for image detection.

4.2. Configuration of Models in Federated Learning

In the configuration setting of our models in federated learning framework, we have used ‘return_df()’ function that is capable of processing the ‘.png’ images file directories based on distinct datasets and spread equally across the participating clients, i.e., client 4,6 and 8. The individual file '.png' keeps track of the file location. The return function is capable of doing an equal distribution of available data labels to the clients. For example, if we have 5 clients and 100 images, then the return function will ensure the distribution of 20 images to individual clients.

In our setting, we have also used custom data loader, which is efficient as it avoids loading the full dataset in RAM instead it loads on demand basis. This is an effective approach, especially on the larger datasets. While using the data loading function, i.e., 'return_dataloader()', the DataLoader object is outputted as below:

Figure 2. Data loader function.

Figure 2. Data loader function.

• TrainLoader: It is the type of DataLoader that determines the training section of the dataset. It is quite useful in iterating on the transformed images as well as in iterating on the labels over the batches. Variability is also ensured, as it is involved in the reshuffling of the training data prior to every epoch.
• valLoader: It shows the validation section of the dataset. Data iteration takes place over batches on validation images, as well as labels. It is not involved in the data reshuffling; therefore, the order stays intact throughout the epochs.
• testLoader: It is the type DataLoader that demonstrates the test part of the data set. The data iteration takes place over the batched over the test images as well as labels. Unlike TrainLoader, it is not involved in data reshuffling.

DataLoaders are depending on the percentage allocated for the training, validation, and the testing data. In general, DataLoader is quite efficient in memory consumption and can be effectively used in the training validation and testing stages of ML models. It eventually helps to make the data ready for PyTorch-based models.

We have also used the 'train_model()' function, which is involved in the following stages:

• Initialise the variable and list: The initial step involved in the tracking of the model state, validation loss, and the accuracy for the individual epoch.
• Epoch loops: The train model function also loops over a certain number of epochs.
• Phase loops: The function also performs the loops over the training as well as the validation stage of individual epochs.
• Set Models Mode: The function also keeps the mode to ‘train’ if the model is in training mode. In this way certain features including dropout as well as batch normalisation are activated. Similarly, during the validation stage, these parameters are disabled.
• Batch loops: The function also loops with the data batches.
• Forward pass: The function transfers the input label to the corresponding device via model and performs loss calculation.
• Backward pass and optimisation: When using the train model function, if the training is zero, then the gradient of the model is calculated by using the optimisers also known as backward pass.
• Statistics calculation: In the train_model function, the prediction is calculated, and the model run loss as well as accuracy is updated.
• Epoch Statistic Calculation: During the training phase, by the end of every epoch, the loss and accuracy are calculated by the end of every epoch. While in the vase of validation stage, the precision, accuracy, F1-score is calculated while showing the confusion matrix.

Several numbers of iterations are performed by using the while loop. The ‘while loop’ in general is involved in higher number of continuous iterations, however, in our case, it runs at one time based on the rounds:

• In our case, we have used clients 4, 6 and 8 to train the model individually on the clients while using the ‘train_model’ function, and in this way, model accuracy and loss are given out.
• The weights from the individual clients are kept in ‘w_local’ lists, similarly, the accuracy and loss of the model are kept on their respective lists.
• In the federated learning framework, the ‘fed_avg’ function is used to take the average weight of all models to form the global model.
• The mean loss and accuracy is also calculated on the participants.
• The mean average of all the weights is referred back towards the model on individual devices. It allows every client to receive the similar model update.
• The weight average of the models is stored as the ‘fed_model_client’.
• The output is displayed with the round phase, loss, as well as the accuracy.
• Alterations in ‘validation_loss’ as well as accuracy from the last round are displayed, too.

In this configuration setup of federated learning, we have demonstrated that ML models are locally trained on participating clients, the weight of individual clients is aggregated to form a global model which is shared acres the clients. In this approach, data stays local to client, and only the weight average model is shared to central server. In these data, data privacy is retained.

10.

Return best model: Once all corresponding epoch rounds are accomplished, the one with the least validation loss loads up the model.

At every epoch, initially the model is trained with the training dataset, and once the training is done, the model performance is observed with the validation data. The confusion matrix including the accuracy and loss function is demonstrated at each stage to understand the performance of the model. Once all the epochs are completed, the function determines the epoch with the least validation loss, which ultimately selects the corresponding model state. It is a quite useful method that helps us to understand the state of the model in best terms of its performance.

4.4. Exploratory Data Analysis (EDA)

To perform the EDA, we have randomly selected the training dataset to visualise the data diversity. We have taken 15 images individually from the training data set and the validation data set with their own labels. The visualisation of the training data set has helped us to understand the data while observing it’s performance during the validation stage. The EDA on the malaria dataset is quite important as it helped to ensure the correct labels on the images, as well as it also aids in understanding the data variation and complexities, so the models can be adjusted according while training.

4.4.1. Number of Available Data Sets

The dataset consititue of following distribution which is class balanced as well:

The individual data set that is available on the training and validation dataset is 13,780. The count of dataset based on the infected and unaffected can be visualised as follows:

Figure 3. Malaria dataset Distribution.

Figure 3. Malaria dataset Distribution.

The total count of 27,560 images is available when combining the training and validation datasets. The large number of images helps our models to perform an effective training from the variety of images. The balance of the data set helps to achieve better model training that is essential for correct disease prediction.

4.4.2. How Parasitised and Nonparasitised Cells Look Like

When observing the malaria dataset, the main element that makes the difference between the two classes is the 'dots' that is visible in the parasitised (infected) class. The dots are actually the malaria parasite. The parasite that causes malaria is known as the plasmodium that directly targets red blood cells of the body. Under the microscope, the infected part of the cell is seen as small dots.

In our dataset, we see the ‘ting dots’ as parasite in the cells, which is one of the visual side of detecting malaria. The tiny dots also make a good difference between the healthy cells in contrast to those affected by the malaria parasite. Healthy cells lack these dots that make up non-malaria cells. The difference between healthy cells and malaria cells can be identified in the following visualisation.

Figure 4. The distinguish between the cell among the two classes help models, i.e., resent50 and densenet, to learn the patterns from the cells. Therefore, the ML models classify cells with the presence of dots as parasitised and those without dots as uninfected.

Figure 4. The distinguish between the cell among the two classes help models, i.e., resent50 and densenet, to learn the patterns from the cells. Therefore, the ML models classify cells with the presence of dots as parasitised and those without dots as uninfected.

5. Data Preparation

• The initial stage involves reading the images from the directory.
• Decoding of the image content that involves converting into grid form as RGB.
• Conversion of images into float point tensor.
• Rescaling the tensors into the form that allows the scale range from 0 and 255 to be 0s and 1s as the CNN models takes in the smaller inputs.

5.1. Visualize the Training Images

We have taken the 15 random images from the training dataset as well as from the validation dataset which can be visualised as follows with respect to their labels:

Figure 5. 15 random images of the sample training images.

Figure 5. 15 random images of the sample training images.

Figure 6. 15 images from the sample validation images.

Figure 6. 15 images from the sample validation images.

6. Malaria Experiments and Results

6.1. FL_Densenet and FL_Resnet50 (4 Clients)

The figure below illustrates the confusion matrix for densenet and resnet50 with 4 clients:

Figure 7. the confusion matrix for densenet and resnet50 with 4 clients.

Figure 7. the confusion matrix for densenet and resnet50 with 4 clients.

In terms of performance of model towards the unseen data, let us consider the test data performance analysis with critical discussion:

• Accuracy: The overall correct malaria predictions out of all predictions, the accuracy can be calculated as follows:
  • DenseNet: (1276 + 1353)/(1276 + 1353 + 39 + 84) = 0.9463 or 94.63%
  • ResNet50: (1254 + 1366)/(1254 + 1366 + 26 + 106) = 0.9486 or 94.86%

ResNet50 has slightly higher accuracy than DenseNet.

2.

Precision: It shows the accuracy of the positive prediction of malaria and can be observed in the following equation:

• DenseNet: 1276/(1276 + 39) = 0.9703 or 97.03%
• ResNet50: 1254/(1254 + 26) = 0.9796 or 97.96%

ResNet50 has slightly higher precision.

3.

Recall (sensitivity): It involves the fraction of positive predictions which is correctly determined:

• DenseNet: 1276/(1276 + 84) = 0.9382 or 93.82%
• ResNet50: 1254/(1254 + 106) = 0.9220 or 92.20%

DenseNet has slightly higher recall.

4.

Specificity: It involves the fractions of negative prediction that are correctly determined:

• DenseNet: 1353/(1353 + 39) = 0.9720 or 97.20%
• ResNet50: 1366/(1366 + 26) = 0.9813 or 98.13%

ResNet has higher specificity.

5.

F1 Score: The weighted average of precision and recall of both models can be calculated as follows:

• DenseNet: 2 * (0.9703 * 0.9382)/(0.9703 + 0.9382) = 0.9540 or 95.40%
• ResNet50: 2 * (0.9796 * 0.9220)/(0.9796 + 0.9220) = 0.9501 or 95.01%

DenseNet has a slightly higher F1 score.

It can be observed from the above analysis that Densenet has higher recall as well as F1-score, however, on the other hand esnet50 stands out with accuracy, precision, and specificity.

To determine the selection of model based on the above results, it depends on the use case in a given situation. In the case, if we need to decrease false negative values, for example classify if someone doesn't have malaria, but actually has it, the selection of densenet is preferred, as it has a higher recall rate. In the other case, if we need to reduce false positive values, in other words, if the model tells someone to have malaria, however in reality they do not, then resnet50 could be a better option due to better precision and specificity. Resnet50 is also leads slightly in accuracy.

It is worthwhile to understand the importance of model selection based on the metric; however, the selection mostly relies on the use case scenario. In real life, it is good to have a model with few false positive values (including some further testing) instead of missing true positive results that can neglect the disease treatment. Therefore, in this case the model with higher recall rate should be prioritised.

6.2. FL_Densenet and FL_Resnet50 (6 Clients)

The following confusion matrix is achieved by using 6 clients in the malaria dataset:

• Accuracy: (TP+TN)/(TP+FP+FN+TN)
  • Densenet: (688+1272)/(688+0+168+1272) = 0.9213 (92.13%)
  • Restnet50: (666+1269)/(666+3+703+1269) = 0.7250 (72.50%)
• Precision: TP/(TP+FP)
  • Densenet: 688/(688+0) = 1 (100%)
  • Restnet50: 666/(666+3) = 0.9955 (99.55%)
• Recall: TP/(TP+FN)
  • Densenet: 688/(688+168) = 0.8037 (80.37%)
  • Restnet50: 666/(666+703) = 0.4864 (48.64%)
• F1-score: 2*(precision*recall)/(precision+recall)
  • Densenet: 2*(1*0.8037)/(1+0.8037) = 0.8911 (89.11%)
  • Restnet50: 2*(0.9955*0.4864)/(0.9955+0.4864) = 0.6530 (65.30%)
• Specificity: TN/(TN+FP)
  • Densenet: 1272/(1272+0) = 1 (100%)
  • Restnet50: 1269/(1269+3) = 0.9976 (99.76%)

The above results shows the better performance of densenet in contrast to resnet50 while comparing accuracy, recall, f1-score and specificity. Resnet50 also constitutes less precision, while the the the difference is quite minor. The overall comparison shows that there are less false negative and non false positive in contrast. The detailed analysis shows that the densenet model stands out in terms of malaria classification; however, it is also important to understand the certain requirement of the use case. In the case where higher false positives are required, the use of densenet could be a good choice, as it constitutes higher precision. Overall, densenet performed well in the malaria case study.

Figure 8. confusion matrix of FL_Densenet and FL_Resnet50 (6 Clients).

Figure 8. confusion matrix of FL_Densenet and FL_Resnet50 (6 Clients).

6.4. Significance Test

The t-test suggests whether the difference between the methods really makes the difference based on the critical differences or it is just a coincidence. Therefore, we have used our CNN models along with FL to conduct pairwise tests. The p-value reflects whether the difference between the models is larger, in case if the values go lower than 5%, it will result in rejection. The p-test conducted on the CNN models individually on the FL framework reflects the performance based on each model. In other words, t-test shows the mean difference of the chosen parameter between the two models. The positive t-values indicates whether the 1^st group have any difference with the 2^nd group and negative values reflects that the 2nd group has major differences with the 1^st one. Similarly, the p-value, as in our case is standard scale of 0.05, so if the results are below this value it indicates the significant difference between the models. The following table shows the t-stats and p-values of our results:

Table 1. T-stats and p-values of different models on malaria dataset.

Table 1. T-stats and p-values of different models on malaria dataset.

Metrics	Model Name	T-Stats	P-Value
Accuracy	FL_DENSENET And FL RESNET50	11.45726	0
Precision	FL_DENSENET And FL RESNET50	−0.09118	0.92746
Re-call	FL_DENSENET And FL RESNET50	0.09637	0.92335
F1-Score	FL_DENSENET And FL RESNET50	0.25665	0.79778

Figure 9. Confusion matrix of FL_Densenet and FL_Resnet50 (8 Clients).

Figure 9. Confusion matrix of FL_Densenet and FL_Resnet50 (8 Clients).

7. Summary

References

1. Ali, S.; Raut, S. Detection of Diabetic Retinopathy from fundus images using Resnet50," 2023 2nd International Conference on Paradigm Shifts in Communications Embedded Systems, Machine Learning and Signal Processing (PCEMS), Nagpur, India, 2023, pp. 1–5. [CrossRef]
2. S. S., M; Rao, M.D.S.; Rani, M.S.; Durga, K.; Kranthi, A. S. S. M; Rao, M.D.S.; Rani, M.S.; Durga, K.; Kranthi, A. Covid-19 X-Ray Image Detection using ResNet50 and VGG16 in Convolution Neural Network," 2022 IEEE Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI), Gwalior, India, 2022; pp. 1–5. [CrossRef]
3. Jing, S.; Kun, H.; Xin, Y.; Juanli, H. Optimization of Deep-Learning Network Using Resnet50 Based Model for Corona Virus Disease (COVID-19) Histopathological Image Classification. In Proceedings of the 2022 IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA), Changchun, China; 2022; pp. 992–997. [Google Scholar] [CrossRef]
4. Huy, V.T.Q.; Lin, C.-M. An Improved Densenet Deep Neural Network Model for Tuberculosis Detection Using Chest X-Ray Images. IEEE Access 2023, 11, 42839–42849. [Google Scholar] [CrossRef]
5. S and A. L., R. P J. Study of Deep Learning Approaches for Diagnosing Covid-19 Disease using Chest CT Images," 2023 7th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 2023, pp. 263–269. [CrossRef]
6. Chowdhury, D.; et al. Detection of Sars-Cov-2 from human chest CT images in Multi-Convolutional Neural Network’s environment," 2023 11th International Conference on Internet of Everything, Microwave Engineering, Communication and Networks (IEMECON), Jaipur, India, 2023, pp. 1–7. [CrossRef]
7. Zhou, Z.; Liu, Y.; Wang, Q.; Toe, T.T. Detection of Pneumonia Based on ResNet Improved by Attention Mechanism," 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA), Shenyang, China, 2023, pp. 859–863. [CrossRef]
8. Tariq, T.; Suhail, Z.; Nawaz, Z. Knee Osteoarthritis Detection and Classification Using X-Rays," in IEEE Access, vol. 11, pp. 48292–48303, 2023. [CrossRef]
9. Amelia Jiménez-Sánchez, Mickael Tardy, Miguel A. González Ballester, Diana Mateus, Gemma Piella, Memory-aware curriculum federated learning for breast cancer classification, Computer Methods and Programs in Biomedicine, Volume 229, 2023, 107318, ISSN 0169-2607. [CrossRef]
10. Jakaite, L.; Schetinin, V.; Maple, C. Bayesian Assessment of Newborn Brain Maturity from Two-Channel Sleep Electroencephalograms. Comput. Math. Methods Med. 2012, 2012, 1–7. [Google Scholar] [CrossRef] [PubMed]
11. Jakaite, L.; Schetinin, V.; Maple, C.; Schult, J. Bayesian Decision Trees for EEG Assessment of newborn brain maturity’, in 2010 UK Workshop on Computational Intelligence (UKCI), Colchester, United Kingdom, Sep. 2010, pp. 1–6. [CrossRef]
12. Schetinin, V.; Jakaite, L.; Nyah, N.; Novakovic, D.; Krzanowski, W. Feature Extraction with GMDH-Type Neural Networks for EEG-Based Person Identification. Int. J. Neur. Syst. 2018, 28, 1750064. [Google Scholar] [CrossRef] [PubMed]
13. Schetinin, V.; Jakaite, L.; Jakaitis, J.; Krzanowski, W. Bayesian Decision Trees for predicting survival of patients: A study on the US National Trauma Data Bank’. Computer Methods and Programs in Biomedicine 2013, 111, 602–612. [Google Scholar] [CrossRef] [PubMed]
14. Swapna, G.; Vinayakumar, R.; Soman, K.P. Diabetes detection using deep learning algorithms. ICT express, 2018, 4, 243–246. [Google Scholar]
15. Available online: https://www.kaggle.com/datasets/iarunava/cell-images-for-detecting-malaria.