Prerpints.org logo
Preprint
Review

XAI-based Clinical Decision Support System: A Systematic Review

Altmetrics

Downloads

170

Views

114

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

11 June 2024

Posted:

11 June 2024

You are already at the latest version

Alerts
Abstract
With increasing electronic medical data and the development of artificial intelligence, Clinical Decision Support Systems (CDSSs) assist clinicians in diagnosis and prescription. Traditional knowledge-based CDSSs follow an accumulated medical knowledgebase and a predefined rule system, which clarifies the decision-making process; however, maintenance cost issues exist in the medical data quality control and standardization process. Non-knowledge-based CDSSs utilize vast amounts of data and algorithms to effectively decide; however, the deep learning black-box problem causes unreliable results. EXplainable Artificial Intelligence (XAI)-based CDSS provides a valid rationale and explainable results. It ensures trustworthiness and transparency by showing the recommendation and prediction results process through explainable techniques. However, existing systems have limitations, such as the scope of data utilization and the lack of explanatory power of AI models. This study proposes a new XAI-based CDSS framework to address these issues; introduce resources, datasets, and models that can be utilized; and provides a foundation model to support decision-making in various disease domains. Finally, we propose future directions for CDSS technology and highlight societal issues that need to be addressed to emphasize the potential of CDSS in the future.
Keywords: 
Subject: Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Introduction

A Clinical Decision Support System(CDSS) [1] supports decision-making for clinicians in diagnosing and treating diseases based on the patient’s clinical information. With the advancing big data analysis and artificial intelligence (AI) lately, the research on these techniques to CDSS has gained considerable attention. Notably, AI-based CDSS is highly valuable because of its effectiveness in supporting clinical diagnosis, prescription, prognosis, and treatment through AI models.
Traditional CDSS is categorized into knowledge-based CDSS, where rules are defined in advance and non-knowledge-based CDSS, where the rules are not predefined [2]. The principle of knowledge-based CDSS is to make decisions using correlations and If–Then rules for accumulated data. Knowledge-based CDSS is beneficial as the decision-making process is transparent because it follows predefined rules. However, it has the limitation that knowledge and rules must be defined in advance for all cases.
Conversely, non-knowledge-based CDSS provides decision-making by learning patterns found in past clinical information through machine learning or AI without rules. Non-knowledge based CDSS has been extensively studied in various medical areas [3,4], dealing with issues such as hypertension, heart failure, and lung disease. It is expected to be a breakthrough methodology that can reduce the cost of knowledge construction and provide personalized treatment. However, the black-box problem [5], in which explaining the process behind the results derived by AI models is not possible, makes it difficult to apply them in healthcare, where transparency is essential. Therefore, to utilize non-knowledge-based CDSS, it is necessary to introduce robust clinical validation and evaluations or provide convincing evidence to support the results.
Explainable AI (XAI) [6] was proposed to explain the process of the results of AI-based systems. The differences in knowledge-, non-knowledge-, and XAI-based CDSS are shown in Figure 1. XAI can explain the process and rationale behind an AI model’s decision in a manner that can be interpreted by the user. XAI technologies can be broadly categorized into feature, model-, complexity, and methodology-based, and XAI technologies can be applied to CDSS to ensure the transparency and reliability of the healthcare system. In this study, we propose an XAI-based CDSS framework and introduce its application. Our main contributions are summarized as follows:
  • We perform a systematic review of explainable AI techniques that can ensure trust-worthiness and transparency in the medical domain and present a forward-looking roadmap for XAI-based CDSS.
  • We categorized various studies from traditional CDSS to state-of-the-art XAI-based CDSS by appropriate criteria and summarize the features and limitations of each CDSS to propose a new XAI-based CDSS framework.
  • We propose a novel CDSS framework using the latest XAI technology and show that potential value by introducing areas that can be utilized most effectively.
This paper is organized as follows. In Section 2, we describe the research trends related to knowledge-based, nonknowledge-based, and XAI-based CDSS and the limitations of existing technologies. In Section 3, we present the XAI-based CDSS framework and introduce the datasets and models required for its construction. Applications of XAI-based CDSS is presented in Section 4, followed by the conclusion in Section 5.

2. Related Work

This section categorizes CDSS into knowledge-based, nonknowledge-based, and XAI-based, and each section describes the main research, technologies, and methodologies, with the overview shown in Figure 2.

2.1. Knowledge-Based CDSS

To manage and utilize big data, an architecture called a knowledge base has emerged, and a methodology has been proposed to incorporate a knowledge base built by correlation of the accumulated data based on the experience of clinicians in a CDSS [7]. This is categorized as a knowledge-based CDSS that supports decision-making by inferring results from a rule-based knowledge base with an inference engine. Accordingly, it is important to design a knowledge base structure and appropriate rule system for each field. The features, functions and applied domain of knowledge-based CDSS are organized in Table 1.
Knowledge-based CDSSs define rules based on the literature, practice, or patient-oriented evidence [8] and are therefore often used in clinical practice based on clinical guidelines or in Evidence Based Medicine (EBM). Rule-base Inference Methodology using the Evidential Reasoning (RIMER) is based on a Belief Rule Base (BRB) system [9,10,11]. BRB systems set belief degrees to represent different types of uncertain knowledge and extend the If–Then rules to represent knowledge. Most BRB-based CDSS frameworks comprise an interfacial layer, an application processing layer and a data management layer [12,13,14]. These frameworks have proven their performance by in various fields, such as COVID-19 [15] and heart failure [16], psychogenic pain [17], tuberculosis [18], acute coronary syndrome (ACS) [19], and lymph node cancer metastasis[20].
However, an effective knowledge representation for CDSS could be Decision Tree, Bayesian Network, or Nearest Neighbors [21]. A study leveraging decision trees proposed a knowledge modeling method in which a clinical model extracted from glaucoma clinical practice guidelines was represented as mind maps and converted into iterative decision trees by clinicians and engineers [22]. In a similar study, mind maps representing the clinical treatment process for thyroid nodules obtained from clinicians is converted into an iterative decision tree model to extract rules [23]. This is followed by the process of representing tacit knowledge to explicit knowledge and finally converting it to executable knowledge. In another study on a decision-tree-based CDSS, a pediatric allergy knowledge base, ONNX inference engine, and tree algorithm were used to provide knowledge of diagnosis and treatment to clinicians [24].
Additionally, Bayesian network-based CDSS, which is used in various medical areas, (such as liver disease [25], breast cancer [26], infectious diseases [27], diabetes [28], angina [29], respiratory diseases [30], and lymph node cancer metastasis [31]) uses a what–if analysis mechanism. In the field of dental hygiene, CNNs have recently been used in conjunction with a Bayesian network framework based on the (Expectation-Maximization (EM) algorithm to detect abnormal oral images [32]. Additionally, a diagnosis of an Hepatitis C Virus (HCV) diagnosis system has been proposed using a fuzzy Bayesian network with a fuzzy ontology to resolve ambiguity and uncertainty in outbreaks [33].
Research using the K-Nearest Neighbor (KNN) algorithm structured medical information by classifying similar clinical cases through ontology extraction methods for Case Based Reasoning (CBR) [34]. Moreover, a Computer-Aided Diagnosis (CAD) system proposed for melanoma diagnosis provides a related ontology based on Asymmetry, Border, Color and Differential (ABCD) structure rules, and classifies similar melanoma cases using the KNN algorithm [35]. Another study that uses similarity of knowledge for decision-making, as in the aforementioned studies, provides an appropriate diagnosis for patients semantically classified by time-series similarity based on the patient’s medical history [36].
Knowledge-based CDSS supports decision-making based on pre-built knowledge base, effective data modeling, and knowledge-based updating for each domain is also ongoing. Recently, in genomics, a clinical genomic data model has been proposed to analyze clinical genomic workflow and extract attributes using genomic data for clinical application of genomic information [37]. Additionally, methodologies have appeared to facilitate knowledge base updating by analyzing newly acquired textual knowledge through natural language processing to generate rules [38].
Knowledge-based CDSS has potential in that the decision-making process is clear and traceable. However, it is limited by maintenance and construction costs because it relies on medical specialists and knowledge engineers for standardization and error correction, as data quality control is essential [39,40].

2.2. Non-Knowledge-Based CDSS

With the explosion of data and specialized knowledge, the amount of information that must be processed to make clinical decisions is growing astronomically. To learn on its own like a human, using massive amounts of data, deep learning, and AI, which is based on artificial neural networks, supports clinical decision-making. These methods analyze patterns in patient data to draw associations between symptoms and diagnoses. Moreover, deep learning and AI can be used to analyze various data, including text, images, videos, audio, and signals, enabling the development of non-knowledge-based CDSS that can understand the overall clinical situation and context. The first step toward a nonknowledge-based CDSS began with analyzing images and using them to make clinical decisions. A convolutional neural network (CNN) [41], which trains image patterns by mimicking the structure of the human optic nerve has been used to diagnose obstructive sleep apnea by learning high-order correlation features between polysomnography images and their labels [42,43], and an automated system has been proposed to optimize patient satisfaction by analyzing patients’ experiences with ambulance transport services with a combined model of CNN and word embeddings [44]. Similarly, a technique for diagnosing melanoma using a single CNN trained on a dataset of clinical images has been introduced [45].
There are also a number of cases of recurrent neural networks (RNNs) that can handle time-series data. EHR data are good candidates for using RNNs [46] because it provide clinical records with temporal information. A previous study [47] applied RNNs to the EHR data of heart failure patients to predict heart failure outperformed machine learning methods such as SVMs [48], MLPs [49], logistic regression [50], and KNNs [51]. Because ECG data also contain temporal information, ECG signals can be analyzed using RNN-based models detect sleep apnea [52].
When dealing with clinical data, owing to its long-term properties, the problem of forgetting previous data and ignoring past information may arise. Therefore, studies using LSTMs [53] to predict future data by considering past data have been proposed. An LSTM was used to learn multiple diagnostic labels to classify diagnoses [54] and oral–nasal thermal airflow, nasal pressure, and abdominal breathing-guided plethysmography data from polysomnography were analyzed with a bidirectional LSTM model to diagnose sleep apnea [55]. Deep learning is frequently applied in medical image analysis. Chest radiographs can be analyzed using deep learning to diagnose chest diseases such as lung nodules [56], lung cancer [57], and pulmonary tuberculosis [58].
Unlike traditional supervised and unsupervised learning, reinforcement learning [59] generated its own training data by observing the current state and selecting future actions. Because existing CDSSs are trained based on evaluations made by different clinicians with different criteria, interrelated symptoms are not considered in some cases. This problem can be solved by applying reinforcement learning, which is used to learn complex environments. A CDSS based on a deep reinforcement learning algorithm has been introduced to determine the initial dose for ICU patients, where an accurate medication prescription is critical, and prevents mis-dosing and complications. [60] Reinforcement learning of secure computations enables the implementation of patient-centered clinical decision-making systems while protecting sensitive clinical data. A privacy-preserving reinforcement learning framework with iterative secure computation was proposed to provide dynamic treatment decisions without leaking sensitive information to unauthorized users [3]. A reinforcement learning-based conversational software for radiotherapy was also studied, where the framework used graph neural networks and reinforcement learning to improve clinical decision-making performance in radiology with many variables, uncertain treatment responses, and inter-patient heterogeneity [61].
BERT [62], a large language model based on a Transformer [63], was used to develop a CDSS with natural language understanding capabilities. To reduce diagnostic errors, a framework for multi-classifying diagnosis codes in EHRs using BERT [64] has been developed to help clinicians predict the most likely diagnosis. However, specialists have raised concerns about the reliability and responsibility of these deep learning and AI models because of their inability to explain their decisions. Therefore, they are often unwilling to use them in diagnosis. To this end, it is necessary to adopt AI, which provides evidence for prediction and an understandable explanation.

2.3. XAI-Based CDSS

EXplainable AI (XAI)[5], has emerged to overcome the black-box[6] problem of deep learning models, which means that deep learning models have the highest perfor-mance compared to other rule-based or machine learning models but have the limitation of lacking interpretability. This can be described as the "performance–interpretability trade-off, " and is shown in Figure 3. Performance is highest for deep learning, followed by machine learning models (Decision Tree, Nearest Neighbors, Bayesian Network), and rule-based models; however, transparency (interpretability) is inversely proportional. In other words, applying XAI to deep learning models is capable of explaining the reason and logic behind the results predicted by the model to ensure transparency and reliability of the results with high-performance deep learning. With attempts to apply XAI in various fields [65], XAI is gained attention as a solution to the uncertainty problem in CDSS systems where accuracy and reliability are important [66].
The description techniques used in XAI can be broadly categorized into scoop-, model-, complexity-, and methodology-based [67]. The most popular XAI methods in recent research include SHapley Additive exPlanations (SHAP) [68], (local interpretable model–agnostic Explanations (LIME) [69], Post hoc Interpretability [70], and (Gradient-weighted Class Activation Mapping (GradCAM) [70].
Scoop-based techniques determine the contribution of data based on the importance features to train the AI model. A prominent example is the local explainers method (LIME) [71] is a method specific to a particular instance or outcome. LIMEs directly explain how the model’s input data change the outcome, and after training the model, it can make guesses about samples that have not appeared before [72]. For COVID-19, LIME and traditional machine learning models were combined to identify the features that had the greatest impact on medical history, time of onset, and patients’ primary symptoms [68]. Similarly, an LSTM model was used in a study on depressive-symptom detection, in conjunction with a LIME approach to identify text suggestive of depressive symptoms [73]. Other applications include the diagnosis of Parkinson disease [74], hip disease [75], Alzheimer’s disease and mild cognitive impairment [76].
By contrast, SHAP is a global explainer method that provides a theoretical interpretation of any dataset uses cooperative game theory concepts [77] to calculate the contributions of biomarkers or clinical features (players) for a specific disease outcome (Reward) [72]. To predict postoperative malnutrition in children with congenital heart disease, the XGBoost and SHAP algorithms were used to calculate the average of five risk factors (weight one month after surgery, weight at discharge, etc.) for all patients [78]. PHysiologicAl Signal Embeddings (PHASE), a method for transforming time-series signals into input features, wasfirst applied to embed body signals with an LSTM model to features extracted using SHAP from EMR/EHR data [79]. In addition, a multi-layer XAI framework utilizing multimodal data, such as MRI and PET images, cognitive scores, and medical histories, have been proposed [80].
SHAP is applied to all layers of the framework, where the first layer performs multiple classification for the early diagnosis of AD. In the second layer, the binary classification score is were used to determine the transition from cognitive impairment to AD [80]. Similarly, SHAP has been widely used in various diseases and clinical domains such as predicting readmission [81,82], COVID-19 [83,84,85,86], liver cancer [87], influenza [88], and malignant cerebral edema [89].
More recently, researchers have utilized LIME and SHAP simultaneously to ensure a convincing description of the system. A hybrid approach combining Vision Transformer (ViT) and a Gated Recurrent Unit) was used to generate LIME heat maps using the top three features from the brain MRI images, and SHAP was used to visualize the model’s predictions to demonstrate the validity of data patterns [90]. In addition, the Department of Chronic Kidney Disease also used LIME and SHAP algorithms simultaneously to represent the importance of features in the best model trained by five machine learning methods methods (Random Forest, Decision Tree, Naïve Bayes, XGBoost, and Logistic Regression) [91].
Model-based techniques can be classified into model-specific and model-agnostic methods. Model-specific methods utilize the unique features of a model to make decisions, indicating that they can only be applied to the internal operations of a specific model. An example is Score-CAM [92], which is based on CNNs and compares output for the given input features, thereby indicating their importance. A previous study proposed a system for classifying images from a clock-drawing test as a tool for diagnosing dementia was trained on API-Net [93] and visualized it using Score-CAM to provide explainability, and transparency[94]. However, model-agnostic methods are model-independent and can be applied to any model or algorithm. As a CDSS tool that reduces the model dependency, a COVID-19 symptom severity classifier that utilizes different machine learning models to identify high-risk patients for COVID-19 has been proposed [95].
Complexity-based techniques make machine learning or deep learning models fully interpretable. Interpretability can be categorized into intrinsic interpretability [96] and post hoc interpretability [72] depending on the viewpoint. In general, intrinsic interpretability indicates that a model with a simple architecture can be explained by the trained model itself, whereas post hoc interpretability means that the trained model has a complex architecture and must be retrained to explain this phenomenon. In a study on brain tumor detection based on MRI images, three pre-trained CNNs, DarkNet53 [92], EfficientNet-B0 [97], and DenseNet201 [98], were used to extract features using a hybrid methodology to explain post-interpretability [99].
Another framework for brain tumor diagnosis, NeuroNet19, combines a 19-layer VGG19 that detects complex hierarchical features in images with an inverted pyramid pooling module (iPPM) model, which refines these features, leveraging post-interpretability [100]. Methodology-based techniques are categorized into Backpropagation-based and Perturbation-based [67], among which Backpropagation-based GradCAM was proposed to describe CNN models with good performance [101]. GradCAM was applied to the convolutional layer at the end of the CNN, and uses the gradient information of the layer to find the features that are highly involved in a particular decision [72,102]. To further improve classification performance, several studies have been proposed to predict oral cancer from oral images using guided attention inference network (GAIN) along with the aforementioned CNN-based VGG19 model GradCAM [103], and also to diagnose glaucoma from fundus images using GradCAM’s heatmap and ResNet-50 model [104]. SBecause CNN models are widely applied in image classification and processing, GradCAM technology is used in several studies utilizing image data [105,106,107,108,109,110,111].
Table 2. Non-Knowledge-based CDSS.
Table 2. Non-Knowledge-based CDSS.
Preprints 108939 i002
Table 3. XAI-based CDSS.
Table 3. XAI-based CDSS.
Preprints 108939 i003a
Preprints 108939 i003b

3. XAI CDSS Framework

3.1. Proposed Architecture

This study proposes an XAI-based CDSS framework that can handle both high-performance and interpretation of the decision-making process. This is achieved by applying explainable AI technology to artificial intelligence models. Details of this framework are shown in Figure 4. The proposed framework handles multimodal data from various medical domains, including text, audio, images, and genomes. It applies the explainable AI methodology to representative deep learning models. Finally, we demonstrate the potential value of the proposed framework through the presentation of application plans, illustrating the circumstances under which they can be utilized effectively.
Most of the existing AI-based CDSSs are limited to text data. Even if the data-usage range is extended, only one additional type of biometric signal or image is used. However, a multimodal data utilization plan is necessary to make clinical decisions considering factors that are difficult to record in formal form, such as the patient’s condition, facial expressions, and behavior that change in real time. Consequently, the proposed framework must be capable of expanding its knowledge through the continuous learning and analysis of information derived from multimodal data. As illustrated in Figure 2, a knowledge graph was constructed by extracting the relationship between features based on multimodal features obtained using models such as large-scale language models, VATT, and audio analysis models derived from multimodal data such as text, images, and signals. Furthermore, through reinforcement learning or continuous learning, knowledge graphs can expand automatically, enabling them to respond flexibly to new knowledge, while retaining previous information.
A multimodal clinical XAI learning model for an explanatory clinical decision-making system requires three elements: a deep explanatory label, an interpretable model, and model inference. First, the features that can explain the prediction results must be identified and labeled, and a model must be created in connection with a decision tree with high explanatory power. The explainable models were inferred from the black boxes, which are the largest problem of existing AI methodologies. Explanations are generated with predictions. All of the aforementioned processes [114] are conducted through interactions for patients, medical professionals, and clinical systems in real time. The predictions and explanations generated by analyzing the data obtained from each participant are provided to the participants once more to facilitate overall clinical decision-making, including monitoring, diagnosis, prescription, warning, and document management.

3.2. Dataset

3.2.1. Clinical Dataset

The first large-scale multimodal clinical dataset is UK Biobank [115]. It has been collected since 2006 and includes several hierarchical data types such as lifestyle habits, body information, biological samples, electrocardiogram data, and EHR data from more than 500,000 participants. In addition to basic biometric data, the dataset provides genomic analysis, exon mutation testing, full-length genome sequencing, brain MRI, cardiac MRI, abdominal MRI, carotid artery ultrasonography and radiographic findings. Similar datasets were also collected from the China Kadoori Biobank [116] and Biobank Japan [117]. The MMIC dataset [118], published by the Massachusetts Institute of Technology, has now been published up to the fourth version. This is an open-source dataset comprising EHR data, including demographic information, diagnostic codes, and medications obtained from ICU patients at the Beth Israel Deaconess Medical Center. MMIC-IV is one of the the most representative datasets of clinical AI models that aim to predict clinical events or readmissions. It contains textual data, such as reports, medical notes, and imaging data, including laboratory and physiological data, and chest radiographs. Furthermore, it is possible to reconstruct the multimodal dataset by utilizing a combination of data from a single modality. This may include, for example, an Alzheimer’s patient’s brain image dataset, the Alzheimer’s Disease Neuroimaging Initiative (ADNI)[119], and exercise activity data from patients with schizophrenia and depression[120]. Table 4 lists the summary of the monomodal and multimodal clinical datasets.

3.2.2. Knowledge Graph

The use of knowledge graphs enables the formal expression of medical expertise and the semantic structuring of unstructured multimodal data. During this period, while learning existing HKG, the knowledge graph can be expanded using new data. In particular, the emergence of large language models has facilitated the construction of more comprehensive and accurate HKG. HKG for CDSS exists in fields such as medicine (prescription) [141,142], genetics [142,143], and disease [144,145], and based on this, an extended knowledge graph can be generated.
To effectively respond to new clinical cases, information collected in real time should be used in conjunction with existing knowledge. As mentioned previously, an automatic scalability of knowledge graphs is required and reinforcement learning can be applied to ensures persuasive power. Numerous knowledge graphs exist in the clinical fields other than CDSS, which are summarized in Table 5 by application range.

3.3. XAI Model

Prior to the analysis of the multimodal data, the process of fusing multimodal data is required. In particular, it is important to select meaningful features to obtain the desired information from the vast amounts of data. To achieve this, deep learning, which trains a neural network composed of multiple layers, can be used to extract features and representations from complex data. Deep learning models[158] are well-suited to the integration and extraction of meaningful information owing to their capacity to learn complexpatterns and generate knowledge for decision-making through the processing of vast amounts of data. Deep learning models[159], such as those pre-trained on large datasets such as ImageNet[160] or natural language corpora can be employed to obtain correlations through the generation of new samples from multiple modalities with generative models from GAN[161] or VAE [162].
The advent of the transformer [62] has made multimodal data more accessible as it enables immediate inference based on an attention network rather than a model using convolutional structure, as previously described. Using the transformer structure, it is possible to learn multiple modalities together because encoding is possible in a consistent structure for all data modalities. Since the proposal of the Vision Transformer (VT) [163], that encodes images in a manner similar to natural language, there have been numerous attempts to apply the Transformer to other modalities in video and voice.
Recently, VATT [164], a framework for learning multimodal representations from unlabeled data, has been developed for the extraction of multimodal representations from unlabeled signals. It is possible to perform a range of tasks, including behavioral recognition, Voice event classification, image classification, and text-video search using video and audio, and text features extracted from Internet images. Based on the VATT model, the patient’s daily, and counseling videos can be analyzed to identify biometric signals, changes over time and nonverbal expressions and use them in clinical decision-making.
XAI techniques can be largely divided based on explanation method, interpretation method, model specificity, and explanation range [165], as shown in Figure 5. The backpropagation-based XAI measures the degree to which each feature affects the result as a gradient value. Class activity map-based XAI visualizes features with a significant impact using the feature map of the uppermost layer, which aggregates the necessary information. Finally, input interference-based XAI provides explanatory power through the process of repeatedly investigating the model while making various changes to the input value of the model.

4. Applications

4.1. Function of CDSS

The objective of CDSS is to facilitate optimal decision-making for patient safety. Consequently, the role of CDSS is to support the decision-making processes involved in Diagnosis, treatment, and prescription, which are directly related to patient safety. Table 6 illustrates the functions of CDSS.
The first function of CDSS is diagnostic support. CDSS provides diagnostic information on suspected diseases by collecting and monitoring figures showing the patient’s conditions, such as biosignals. This function provides efficient management of beds by supporting useful decision-making to inexperienced clinicians or nursing personnel.
The second function is treatment support. This function supports the determination of the optimal treatment by analyzing all applicable treatment methods in consideration of the patient’s current condition. Through treatment method analysis, information analysis such as inconsistencies, errors, omissions, and side effects between the treatment methods is possible. This function is the most commonly studied area, and provides interaction information between prescribed drugs or checks for side effects when multiple drugs are prescribed together. The third function of CDSS is medical image analysis. As the performance of deep learning models using image data develops, it is the most commonly used CDSS functions related to AI in the medical field. By analyzing medical image data such as X-rays, MRIs, and CT scans, which are most commonly used to identify patient diseases through deep learning, more accurate decision support can be provided to clinicians.
Finally, the system serves as a risk notification function. This function is typically activated in patients admitted to the hospital, and immediately alerts medical personel when abnormal symptoms or dangerous levels are identified while collecting and monitoring patient biometric signals, such as pulse rate, blood pressure, and temperature.

4.2. The Potential of XAI-Based CDSS

The most significant features of the proposed XAI-based CDSS framework are generalizations owing to the use of multimodal data, scalability to apply various deep learning SOTA models, and trustworthiness through explainable AI technologies. The Fusion and learning of various types of multimodal data generated in the medical domain can result in the discovery of features and patterns that were not found in models that only address single modalities. This can be achieved by developing a generalized foundation model using deep learning models that achieved SOTA performance in each field. Finally, the decision-making process can be explained and interpreted as various explainable AI technologies, enabling the development of a transparent and reliable clinical decision-making system. Figure 6 illustrates the areas where the XAI-based CDSS. framework proposed in this study is the most efficiently utilized. Disease detection represents the first field in which it can be used. Research related to deep learning-based disease detection has already demonstrated high performance; however, there is a challenge in utilizing it in the actual medical domain because of issues such as the black-box nature of deep learning. To address this challenge, explainable AI technology is employed to elucidate the rationale behind the predictions generated by the deep learning model, thereby facilitating its deployment in the medical domain.
The second area of application is bed decision support for inpatients. In comparison to current inpatients, there is a significant challenge in that the number of professional nursing personnel in charge of patients is insufficient. Furthermore, in the case of an inexperienced nursing personnel, it is challenging to identify and diagnose appropriate symptoms necessary for patients. The XAI-based CDSS framework is capable of simultaneously monitoring multiple patients and efficiently supporting the decision-making of professional nursing personnel by providing the cause and basis when abnormal symptoms are detected.
The third area of application is treatment and prescription. By analyzing the patient’s medical history or medical record, customized treatment and prescription optimized for the current patient are possible. Specifically, when prescribing a drug, it is possible to derive more optimal prescription results for patient safety by analyzing potential side effects that may occur between the components of the drug prescribed together in advance and providing information to professional medical personnel.
The final area in which XAI-based CDSS can be utilized is clinical practice. Unlike the aforementioned applications in hospitals and medical institutions, it is employed in the education of professional personnel. In particular, in the case of professional nursing personnel, clinical practice is conducted in educational institutions on cases that may occur in hospitals. The use of XAI-based CDSS explains the decisions that should be made about the situations that may occur in the ward and the reasons for the decisions.

5. Discussion and Conclusion

This paper addressed the technological trends of the clinical decision-making system CDSS in accordance with the decision-making system (DSS) framework. It begins with a traditional knowledge-based CDSS and then moves on to a non-knowledge-based CDSS incorporating AI and a multimodal-based XAI CDSS that ensures reliability. By organizing the CDSS services introduced above by field and feature, it is possible to grasp the strengths and weaknesses of each CDSS service, and the matters required for CDSS services in the future. However, in the case of existing systems, there are limitations. These include a limited data utilization range, lack of explanatory power for AI models, and the opacity of the decision-making process. Consequently, in the medical field, which demands reliability and transparency, the decision-making process must be clearly represented for users to interpret it. This paper addressed the black-box problem of unknown-based CDSS as a solution to the problem and proposed an XAI-based CDSS framework that provides valid evidence and reasons for the results. Furthermore, it introduced the available datasets, models, and resources.
The proposed framework is designed to construct an automated extended knowledge graph with multimodal features derived from multi-format data. It comprises three key elements: a deep explanatory label, an interpretable model, and model inference, which collectively facilitate explainable AI. The framework has the potential to automate the entire process of medical clinical services, from personalized treatment to real-time patient condition reflection. It distinguishes itself from existing systems in terms of multimodal data management, utilization plan, explainable AI application plan, and CDSS application range. Furthermore, the medical knowledge graph and HKG, which are structured using available medical multimodal data, are summarized to enhance expertise in decision-making. The proposed XAI-based CDSS framework serves as a foundational model that can be flexibly applied to multiple disease domains. This approach enables the development of a medical system with minimal temporal and spatial constraints.
Furthermore, advancements in the medical field are facilitated by the ease with which computerized data can be used for research purposes. However, measures to enhance social awareness are necessary because the current medical data are not readily accessible owing to concerns regarding the protection of sensitive personal information. Additionally, there is a prevailing attitude that humans should be trusted even when explanations of the data sources, transparency regarding decision-making processes, and the results are provided. As non-face-to-face medical systems become increasingly prevalent, it is imperative that relevant legal deregulation be enacted. If social awareness and institutional improvements are guaranteed, it is expected that this will facilitate the development of compelling medical solutions through CDSS research and development that integrate richer medical multimodal data, medical knowledge graphs, and XAI technology.

Author Contributions

“Conceptualization, S.K and O.J.; methodology, D.K.; software, H.K.; validation, D.K., M.K. and H.K.; formal analysis, D.K.; investigation, M.K, H.K.; resources, H.K.; data curation, M.K.; writing—original draft preparation, M.K and H.K.; writing—review and editing, D.K.; visualization, D.K.; supervision, O.J.; project administration, O.J.; funding acquisition, S.K.

Funding

This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government MSIT (Ministry of Science and ICT (No.RS-2023-00273954).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government MSIT (Ministry of Science and ICT (No.RS-2023-00273954).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Khalifa, M. Clinical decision support: Strategies for success. Procedia Computer Science 2014, 37, 422–427. [Google Scholar] [CrossRef]
  2. Simon, S.R.; Smith, D.H.; Feldstein, A.C.; Perrin, N.; Yang, X.; Zhou, Y.; Platt, R.; Soumerai, S.B. Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. Journal of the American Geriatrics Society 2006, 54, 963–968. [Google Scholar] [CrossRef] [PubMed]
  3. Liu, X.; Deng, R.H.; Choo, K.K.R.; Yang, Y. Privacy-preserving reinforcement learning design for patient-centric dynamic treatment regimes. IEEE Transactions on Emerging Topics in Computing 2019, 9, 456–470. [Google Scholar] [CrossRef]
  4. De Fauw, J.; Ledsam, J.R.; Romera-Paredes, B.; Nikolov, S.; Tomasev, N.; Blackwell, S.; Askham, H.; Glorot, X.; O’Donoghue, B.; Visentin, D.; et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature medicine 2018, 24, 1342–1350. [Google Scholar] [CrossRef]
  5. Liang, Y.; Li, S.; Yan, C.; Li, M.; Jiang, C. Explaining the black-box model: A survey of local interpretation methods for deep neural networks. Neurocomputing 2021, 419, 168–182. [Google Scholar] [CrossRef]
  6. Doshi-Velez, F.; Kim, B. Towards a rigorous science of interpretable machine learning. arXiv e-prints 2017, arXiv-1702. [CrossRef]
  7. Syeda-Mahmood, T. Role of big data and machine learning in diagnostic decision support in radiology. Journal of the American College of Radiology 2018, 15(3), 569–576. [Google Scholar] [CrossRef]
  8. Sim, I.; Gorman, P.; Greenes, R.A.; Haynes, R.B.; Kaplan, B.; Lehmann, H.; Tang, P.C. Clinical decision support systems for the practice of evidence-based medicine. Journal of the American Medical Informatics Association. Journal of the American Medical Informatics Association 2001, 8(6), 527–534. [Google Scholar] [CrossRef] [PubMed]
  9. Yang, J.B.; Singh, M.G. An evidential reasoning approach for multiple-attribute decision making with uncertainty. IEEE Transactions on systems, Man, and Cybernetics 1994, 24(1), 1–18. [CrossRef]
  10. Yang, J.B. Rule and utility based evidential reasoning approach for multiattribute decision analysis under uncertainties. European journal of operational research 2001, 131(1), 31–61. [Google Scholar] [CrossRef]
  11. Yang, J.B.; Liu, J.; Wang, J.; Sii, H.S.; Wang, H.W. Belief rule-base inference methodology using the evidential reasoning approach-RIMER. IEEE Transactions on systems, Man, and Cybernetics- part A: Systems and Humans 2006, 36(2), 266–285. [CrossRef]
  12. Rahaman, S.; Islam, M.M.; Hossain, M.S. A belief rule based clinical decision support system framework. In Proceedings of the 2014 17th international conference on computer and informa- tion technology (ICCIT). IEEE; 2014; pp. 165–169. [Google Scholar] [CrossRef]
  13. Kong, G.; Xu, D.L.; Liu, X.; Yang, J.B. Applying a belief rule-base inference methodology to a guideline-based clinical decision support system. Expert Systems 2009, 26(5), 391–408. [Google Scholar] [CrossRef]
  14. Hossain, M.S.; Andersson, K.; Naznin, S. A belief rule based expert system to diagnose measles under uncertainty. In World Congress in Computer Science, Computer Engineering, and Applied Computing (WORLDCOMP’15): The 2015 International Conference on Health Informatics and Medical Systems 2015, pp. 17–23.
  15. Ahmed, F.; Hossain, M.S.; Islam, R.U.; Andersson, K. An evolutionary belief rule-based clinical decision support system to predict COVID-19 severity under uncertainty. Applied Sciences 2021, 11(13), 5810. [Google Scholar] [CrossRef]
  16. Rahaman, S.; Hossain, M.S. A belief rule based clinical decision support system to assess suspicion of heart failure from signs, symptoms and risk factors. 2013 International Conference on Informatics, Electronics and Vision (ICIEV) 2013, pp. 1–6. [CrossRef]
  17. Kong, G.; Xu, D.L.; Body, R.; Yang, J.B.; Mackway-Jones, K.; Carley, S. A belief rule-based decision support system for clinical risk assessment of cardiac chest pain. European Journal of Operational Research 2012, 219(3), 564–573. [Google Scholar] [CrossRef]
  18. Hossain, M.S.; Ahmed, F.; Andersson, K. A belief rule based expert system to assess tuberculosis under uncertainty. Journal of medical systems 2017, 41(3), 43. [Google Scholar] [CrossRef]
  19. Hossain, M.S.; Rahaman, S.; Mustafa, R.; Andersson, K. A belief rule-based expert system to assess suspicion of acute coronary syndrome (ACS) under uncertainty. Soft Computing 2009, 22, 7571–7586. [Google Scholar] [CrossRef]
  20. Zhou, Z.G.; Liu, F.; Jiao, L.C.; Zhou, Z.J.; Yang, J.B.; Gong, M.G.; Zhang, X.P. A bi-level belief rule based decision support system for diagnosis of lymph node metastasis in gastric cancer. Knowledge-based systems 2013, 54, 128–136. [Google Scholar] [CrossRef]
  21. Silva, B.; Hak, F.; Guimaraes, T.; Manuel, M.; Santos, M.F. Rule-based system for effective clinical decision support. Procedia Computer Science 2023, 220, 880–885. [Google Scholar] [CrossRef]
  22. Hyungwon, Y.e.a. Clinical Knowledge Modeling for Thyroid Nodule Surgical Treatment CDSS Using Mind Maps and Iterative Decision Trees. Journal of The Korean Institute of Communication Sciences 2020, 37(9), 28–33. [Google Scholar]
  23. Yu, H.W.; Hussain, M.; Afzal, M.; Ali, T.; Choi, J.Y.; Han, H.S.; Lee, S. Use of mind maps and iterative decision trees to develop a guideline-based clinical decision support system for routine surgical practice: case study in thyroid nodules. Journal of the American Medical Informatics Association 2019, 26(6), 524–536. [Google Scholar] [CrossRef] [PubMed]
  24. Yu, H.W.e.a. Design and Implementation of a Clinical Decision Support System for Supporting Allergy Diagnosis and Treatment Decision Making for Pediatricians. Journal of Knowledge Information Technology and Systems 2023, 18(3), 5250535. [Google Scholar] [CrossRef]
  25. Wasyluk, H.; Onisko, A.; Druzdzel, M. Support of diagnosis of liver disorders based on a causal Bayesian network model. Medical Science Monitor 2001, 7(1), 327–331. [Google Scholar]
  26. Cruz-Ramirez, N.; Acosta-Mesa, H.G.; Carrillo-Calvet, H.; Nava-Fernández, L.A.; Barrientos- Martinez, R.E. Diagnosis of breast cancer using Bayesian networks: A case study. Computers in Biology and Medicine 2007, 37(11), 1553–1564. [Google Scholar] [CrossRef]
  27. Lucas, P.J.; De Bruijn, N.C.; Schurink, K.; Hoepelman, A. A probabilistic and decision-theoretic approach to the management of infectious disease at the ICU. Artificial Intelligence in medicine 2000, 19, 251–279. [Google Scholar] [CrossRef]
  28. Andreassen, S.; Benn, J.J.; Hovorka, R.; Olesen, K.G.; Carson, E.R. A probabilistic approach to glucose prediction and insulin dose adjustment: description of metabolic model and pilot evaluation study. Computer methods and programs in biomedicine 1994, 41(3-4), 153–165. [CrossRef]
  29. Vila-Francés, J.; Sanchis, J.; Soria-Olivas, E.; Serrano, A.J.; Martinez-Sober, M.; Bonanad, C.; Ventura, S. Expert system for predicting unstable angina based on Bayesian networks. Expert systems with applications 2013, 40, 5004–5010. [Google Scholar] [CrossRef]
  30. Edye, E.O.; Kurucz, J.F.; Lois, L.; Paredes, A.; Piria, F.; Rodríguez, J.; Delgado, S.H. Applying Bayesian networks to help physicians diagnose respiratory diseases in the context of COVID-19 pandemic. In 2021 IEEE URUCON 2021, pp. 368–371.
  31. Reijnen, C.; Gogou, E.; Visser, N.C.; Engerud, H.; Ramjith, J.; Van Der Putten, L.J.; Van de Vijver, K.; Santacana, M.; Bronsert, P.; Bulten, J.; et al. Preoperative risk stratification in endometrial cancer (ENDORISK) by a Bayesian network model: A development and validation study. PLoS medicine 2020, 17(5), e1003111. [Google Scholar] [CrossRef]
  32. Thanathornwong, B.; Suebnukarn, S.; Ouivirach, K. Clinical Decision Support System for Geriatric Dental Treatment Using a Bayesian Network and a Convolutional Neural Network. Healthcare Informatics Research 2023, 29(1), 23. [Google Scholar] [CrossRef]
  33. Riali, I.; Fareh, M.; Ibnaissa, M.C.; Bellil, M. A semantic-based approach for hepatitis C virus prediction and diagnosis using a fuzzy ontology and a fuzzy Bayesian network. Journal of Intelligent & Fuzzy Systems 2023, 44(2), 2381–2395. [CrossRef]
  34. Cao, S.; Lingao, W.; Ji, R.; Wang, C.; Yao, L.; Kai, L.; Abdalla, A.N. Clinical Decision Support System Based on KNN/Ontology Extraction Method. In Proceedings of the 2020 3rd International Conference on Signal Processing and Machine Learning 2020; pp. 56–62. [CrossRef]
  35. Abbes, W.; Sellami, D.; Marc-Zwecker, S.; Zanni-Merk, C. Fuzzy decision ontology for melanoma diagnosis using KNN classifier. Multimedia Tools and Applications 2021, 80, 25517–2553. [Google Scholar] [CrossRef]
  36. Comito, C.; Falcone, D.; Forestiero, A. Diagnosis Detection Support based on Time Series Similarity of Patients Physiological Parameters. In 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI) 2021, pp. 1327–1331. [CrossRef]
  37. Kim, H.J.; Kim, H.J.; Park, Y.; Lee, W.S.; Lim, Y.; Kim, J.H. Clinical genome data model (cGDM) provides interactive clinical decision support for precision medicine. Scientific reports 2020, 10(1), 1414. [Google Scholar] [CrossRef] [PubMed]
  38. DongJin, J.e.a. Natural language processing-based evolutionary clinical decision support systems: a case study in glaucoma diagnosis. Journal of The Korean Institute of Communication Sciences 2020, 37(9), 34–39. [Google Scholar]
  39. Musarrat Hussain, e.a. Intelligent Medical Platform: IMP. Journal of The Korean Institute of Communication Sciences 2020, 37, 9. [Google Scholar]
  40. Sutton, R.T.; Pincock, D.; Baumgart, D.C.; Sadowski, D.C.; Fedorak, R.N.; Kroeker, K.I. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ digital medicine 2020, 3(1), 17. [Google Scholar] [CrossRef]
  41. Chua, L.O. CNN: A vision of complexity. International Journal of Bifurcation and Chaos 1997, 7(10), 2219–2425. [Google Scholar] [CrossRef]
  42. Cen, L.; Yu, Z.L.; Kluge, T.; Ser, W. Automatic system for obstructive sleep apnea events detection using convolutional neural network. In Proceedings of the 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE; 2018; pp. 3975–3978. [Google Scholar] [CrossRef]
  43. Dey, D.; Chaudhuri, S.; Munshi, S. Obstructive sleep apnoea detection using convolutional neural network based deep learning framework. Biomedical engineering letters 2018, 8, 95–100. [Google Scholar] [CrossRef] [PubMed]
  44. Cerqueiro-Pequeño, J.; Casal-Guisande, M.; Comesaña-Campos, A.; Bouza-Rodríguez, J.B. Conceptual Design of a New Methodology Based on Intelligent Systems Applied to the Determination of the User Experience in Ambulances. In Proceedings of the Ninth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’21); 2021; pp. 290–296. [Google Scholar] [CrossRef]
  45. Esteva, A.; Kuprel, B.; Novoa, R.A.; Ko, J.; Swetter, S.M.; Blau, H.M.; Thrun, S. Dermatologist- level classification of skin cancer with deep neural networks. nature 2017, 542, 115–118. [Google Scholar] [CrossRef]
  46. Niraula, Dipesh, e.a. Recurrent neural networks. Design and Applications 2001, 5(2), (64–67).
  47. Choi, E.; Schuetz, A.; Stewart, W.F.; Sun, J. Using recurrent neural network models for early detection of heart failure onset. Journal of the American Medical Informatics Association 2017, 24, 361–370. [Google Scholar] [CrossRef] [PubMed]
  48. Suthaharan, S. Machine learning models and algorithms for big data classification. Integr. Ser. Inf. Syst 2016, 36, 1–12. [Google Scholar]
  49. Taud, H.; Mas, J.F. Multilayer perceptron (MLP). Geomatic approaches for modeling land change scenarios 2018, pp. 451–455. [CrossRef]
  50. LaValley, M.P. Logistic regression. Circulation 2020, 117(18), 2395–2399. [Google Scholar] [CrossRef]
  51. Mahesh, B. Machine learning algorithms-a review. nternational Journal of Science and Research (IJSR) 2020, 9(1), 381–386. [Google Scholar] [CrossRef]
  52. Cheng, M.; Sori, W.J.; Jiang, F.; Khan, A.; Liu, S. Recurrent neural network based classification of ECG signal features for obstruction of sleep apnea detection. In Proceedings of the 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC); 2017; Volume 2, pp. 199–202. [Google Scholar] [CrossRef]
  53. DiPietro, R.; Hager, G.D. Deep learning: RNNs and LSTM. In Handbook of medical image computing and computer assisted intervention; Elsevier, 2020; pp. 503–519. [CrossRef]
  54. Lipton, Z.C.; Kale, D.C.; Elkan, C.; Wetzel, R. Learning to diagnose with LSTM recurrent neural networks. arXiv preprint arXiv:1511.03677 2015. [CrossRef]
  55. ElMoaqet, H.; Eid, M.; Glos, M.; Ryalat, M.; Penzel, T. Deep recurrent neural networks for automatic detection of sleep apnea from single channel respiration signals. Sensors 2020, 20, 5037. [Google Scholar] [CrossRef]
  56. Lee, K.H.; Goo, J.M.; Park, C.M.; Lee, H.J.; Jin, K.N. Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers’ performance. Korean journal of radiology 2012, 13, 564. [Google Scholar] [CrossRef]
  57. Mazzone, P.J.; Obuchowski, N.; Phillips, M.; Risius, B.; Bazerbashi, B.; Meziane, M. Lung cancer screening with computer aided detection chest radiography: design and results of a randomized, controlled trial. PloS one 2013, 8, e59650. [Google Scholar] [CrossRef]
  58. Pande, T.; Cohen, C.; Pai, M.; Ahmad Khan, F. Computer-aided detection of pulmonary tuberculosis on digital chest radiographs: a systematic review. The International Journal of Tuberculosis and Lung Disease 2016, 20, 1226–1230. [Google Scholar] [CrossRef]
  59. Wiering, M.A.; Van Otterlo, M. Reinforcement learning. Adaptation, learning, and optimization 2012, 12, 729. [Google Scholar] [CrossRef]
  60. Qiu, X.; Tan, X.; Li, Q.; Chen, S.; Ru, Y.; Jin, Y. A latent batch-constrained deep reinforcement learning approach for precision dosing clinical decision support. Knowledge-based systems 2022, 237, 107689. [Google Scholar] [CrossRef]
  61. Niraula, D.; Sun, W.; Jin, J.; Dinov, I.D.; Cuneo, K.; Jamaluddin, J.; Matuszak, M.M.; Luo, Y.; Lawrence, T.S.; Jolly, S.; et al. A clinical decision support system for AI-assisted decision- making in response-adaptive radiotherapy (ARCliDS). Scientific Reports 2023, 13, 5279. [Google Scholar] [CrossRef] [PubMed]
  62. Vaswani, e.a. Attention is all you need. Advances in neural information processing systems 2017, 30. [Google Scholar]
  63. Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transform- ers for language understanding. arXiv preprint arXiv:1810.04805 2018. [CrossRef]
  64. Tang, R.; Yao, H.; Zhu, Z.; Sun, X.; Hu, G.; Li, Y.; Xie, G. Embedding electronic health records to learn BERT-based models for diagnostic decision support. In Proceedings of the 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI). IEEE; 2021; pp. 311–319. [Google Scholar] [CrossRef]
  65. Gade, K.e.a. Explainable AI in Industry. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA, 2019; KDD ’19. pp. 3203–3204. [Google Scholar] [CrossRef]
  66. Hulsen, T. Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare. AI 2023, 4, 652–666. [Google Scholar] [CrossRef]
  67. Ali, S.e.a. Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence. Information Fusion 2023, 99, 101805. [Google Scholar] [CrossRef]
  68. Gabbay, F.e.a. A LIME-Based Explainable Machine Learning Model for Predicting the Severity Level of COVID-19 Diagnosed Patients. Applied Sciences 2021, 11. [Google Scholar] [CrossRef]
  69. Ribeiro, M.T.; Singh, S.; Guestrin, C. " Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016; pp. 1135–1144. [Google Scholar] [CrossRef]
  70. Zafar, M.R.e.a. Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability. Machine Learning and Knowledge Extraction 2021, 3, 525–541. [Google Scholar] [CrossRef]
  71. Adadi, A.e.a. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 2018, 6, 52138–52160. [Google Scholar] [CrossRef]
  72. Loh, H.W.e.a. Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011–2022). Computer Methods and Programs in Biomedicine 2022, 226, 107161. [Google Scholar] [CrossRef]
  73. Uddin, M.Z.e.a. Deep learning for prediction of depressive symptoms in a large textual dataset. Neural Computing and Applications 2022, 34, 721–744. [Google Scholar] [CrossRef]
  74. Magesh, P.R.e.a. An Explainable Machine Learning Model for Early Detection of Parkinson’s Disease using LIME on DaTSCAN Imagery. Computers in Biology and Medicine 2020, 126, 104041. [Google Scholar] [CrossRef] [PubMed]
  75. Dindorf, C.e.a. Interpretability of Input Representations for Gait Classification in Patients after Total Hip Arthroplasty. Sensors 2020, 20. [Google Scholar] [CrossRef] [PubMed]
  76. Sidulova, M.e.a. Towards Explainable Image Analysis for Alzheimer’s Disease and Mild Cognitive Impairment Diagnosis. In Proceedings of the 2021 IEEE Applied Imagery Pattern Recognition Workshop (AIPR); 2021; pp. 1–6. [Google Scholar] [CrossRef]
  77. Lundberg, S.M.e.a. A unified approach to interpreting model predictions. Advances in neural information processing systems 2017, 30. [Google Scholar] [CrossRef]
  78. SShi, H.e.a. Explainable machine learning model for predicting the occurrence of postoperative malnutrition in children with congenital heart disease. Clinical Nutrition 2022, 41, 202–210. [Google Scholar] [CrossRef]
  79. Chen, H.e.a. Forecasting adverse surgical events using self-supervised transfer learning for physiological signals. NPJ Digital Medicine 2021, 4, 167. [Google Scholar] [CrossRef] [PubMed]
  80. El-Sappagh, S.e.a. A multilayer multimodal detection and prediction model based on ex- plainable artificial intelligence for Alzheimer’s disease. Scientific reports 2021, 11, 2660. [Google Scholar] [CrossRef] [PubMed]
  81. Lo, Y.T.e.a. Predictive modeling for 14-day unplanned hospital readmission risk by using machine learning algorithms. BMC medical informatics and decision making 2021, 21, 1–11. [Google Scholar] [CrossRef] [PubMed]
  82. Chmiel, F.P.e.a. Using explainable machine learning to identify patients at risk of reattendance at discharge from emergency departments. Scientific reports 2021, 11, 21513. [Google Scholar] [CrossRef]
  83. NNguyen, S.e.a. Budget constrained machine learning for early prediction of adverse outcomes for COVID-19 patients. Scientific Reports 2021, 11, 19543. [Google Scholar] [CrossRef]
  84. Lu, J.e.a. An Explainable System for Diagnosis and Prognosis of COVID-19. IEEE Internet of Things Journal 2021, 8, 15839–15846. [Google Scholar] [CrossRef] [PubMed]
  85. Rozenbaum, D.e.a. Personalized prediction of hospital mortality in COVID-19–positive patients. Mayo Clinic Proceedings: Innovations, Quality & Outcomes 2021, 5, 795–801. [Google Scholar] [CrossRef]
  86. Alves, M.A.e.a. Explaining machine learning based diagnosis of COVID-19 from routine blood tests with decision trees and criteria graphs. Computers in Biology and Medicine 2021, 132, 104335. [Google Scholar] [CrossRef] [PubMed]
  87. AAlsinglawi, B.e.a. An explainable machine learning framework for lung cancer hospital length of stay prediction. Scientific reports 2022, 12, 607. [Google Scholar] [CrossRef] [PubMed]
  88. Hu, C.A.e.a. Using a machine learning approach to predict mortality in critically ill influenza patients: a cross-sectional retrospective multicentre study in Taiwan. BMJ Open 2020, 10. Available online: https://bmjopen.bmj.com/content/10/2/e033898.full.pdf. [CrossRef] [PubMed]
  89. FForoushani, H.M.e.a. Accelerating prediction of malignant cerebral edema after ischemic stroke with automated image analysis and explainable neural networks. Neurocritical care 2022, 36, 471–482. [Google Scholar] [CrossRef] [PubMed]
  90. Mahim, S.M.e.a. Unlocking the Potential of XAI for Improved Alzheimer’s Disease Detection and Classification Using a ViT-GRU Model. IEEE Access 2024, 12, 8390–8412. [Google Scholar] [CrossRef]
  91. Ghosh, S.K.e.a. Investigation on explainable machine learning models to predict chronic kidney diseases. Scientific Reports 2024, 14, 3687. [Google Scholar] [CrossRef]
  92. Redmon, J.e.a. Yolov3: An incremental improvement. arXiv preprint 2018. [CrossRef]
  93. Zhuang, P.e.a. Learning Attentive Pairwise Interaction for Fine-Grained Classification. Proceed- ings of the AAAI Conference on Artificial Intelligence 2020, 34, 13130–13137. [Google Scholar] [CrossRef]
  94. Raksasat, R.e.a. Attentive pairwise interaction network for AI-assisted clock drawing test assessment of early visuospatial deficits. Scientific Reports 2023, 13, 18113. [Google Scholar] [CrossRef] [PubMed]
  95. Nambiar, A.e.a. Model-agnostic explainable artificial intelligence tools for severity predic- tion and symptom analysis on Indian COVID-19 data. Front Artif Intell 2023, 6, 1272506 PMC10726049. [Google Scholar] [CrossRef]
  96. Hu, Z.F.; Kuflik, T.; Mocanu, I.G.; Najafian, S.; Shulner Tal, A. Recent studies of xai-review. In Proceedings of the Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, 2021, pp.; pp. 421–431. [CrossRef]
  97. Tan, M.e.a. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International conference on machine learning. PMLR; 2019; pp. 6105–6114. [Google Scholar] [CrossRef]
  98. Huang, e.a. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition; 2017; pp. 4700–4708. [Google Scholar] [CrossRef]
  99. Ozbay, F.A.e.a. Brain tumor detection with mRMR-based multimodal fusion of deep learning from MR images using Grad-CAM. Iran Journal of Computer Science 2023, 6, 245–259. [Google Scholar] [CrossRef]
  100. Haque, R.; Hassan, M.M.; Bairagi, A.K.; Shariful Islam, S.M. NeuroNet19: an explainable deep neural network model for the classification of brain tumors using magnetic resonance imaging data. Scientific Reports 2024, 14, 1524. [Google Scholar] [CrossRef] [PubMed]
  101. Selvaraju, R.R.e.a. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the Proceedings of the IEEE international conference on computer vision; 2017; pp. 618–626. [Google Scholar] [CrossRef]
  102. Jahmunah, V.e.a. Explainable detection of myocardial infarction using deep learning models with Grad-CAM technique on ECG signals. Computers in Biology and Medicine 2022, 146, 105550. [Google Scholar] [CrossRef] [PubMed]
  103. Figueroa, K.C.e.a. Interpretable deep learning approach for oral cancer classification using guided attention inference network. Journal of biomedical optics 2022, 27, 015001–015001. [Google Scholar] [CrossRef] [PubMed]
  104. Chang, J.e.a. Explaining the Rationale of Deep Learning Glaucoma Decisions with Adversarial Examples. Ophthalmology 2021, 128, 78–88. [Google Scholar] [CrossRef] [PubMed]
  105. QQian, X.e.a. Prospective assessment of breast cancer risk from multimodal multiview ul- trasound images via clinically applicable deep learning. Nature biomedical engineering 2021, 5, 522–532. [Google Scholar] [CrossRef]
  106. Singh, R.K.e.a. COVIDScreen: explainable deep learning framework for differential diagnosis of COVID-19 using chest X-rays. Neural Computing and Applications 2021, 33, 8871–8892. [Google Scholar] [CrossRef]
  107. Thakoor, K.A.e.a. Robust and interpretable convolutional neural networks to detect glaucoma in optical coherence tomography images. IEEE Transactions on Biomedical Engineering 2020, 68, 2456–2466. [Google Scholar] [CrossRef]
  108. Xu, F.e.a. The Clinical Value of Explainable Deep Learning for Diagnosing Fungal Keratitis Using in vivo Confocal Microscopy Images. Frontiers in Medicine 2021, 8. [Google Scholar] [CrossRef] [PubMed]
  109. CChetoui, M.e.a. Explainable COVID-19 Detection on Chest X-rays Using an End-to-End Deep Convolutional Neural Network Architecture. Big Data and Cognitive Computing 2021, 5. [Google Scholar] [CrossRef]
  110. Liu, S.C.e.a. Predicting microvascular invasion in hepatocellular carcinoma: a deep learning model validated across hospitals. Cancer Imaging 2021, 21, 1–16. [Google Scholar] [CrossRef] [PubMed]
  111. Hou, J.e.a. Explainable DCNN based chest X-ray image analysis and classification for COVID-19 pneumonia detection. Scientific Reports 2021, 11, 16071. [Google Scholar] [CrossRef] [PubMed]
  112. Lamy, J.B.e.a. Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach. Artificial intelligence in medicine 2019, 94, 42–53. [Google Scholar] [CrossRef] [PubMed]
  113. Jin, W.e.a. Artificial intelligence in glioma imaging: challenges and advances. Journal of neural engineering 2020, 17, 021002. [Google Scholar] [CrossRef] [PubMed]
  114. Gunning, D, e. a. DARPA’s Explainable Artificial Intelligence (XAI) Program. AI Magazine 2019, 40, 44–58. [CrossRef]
  115. Woodfield, R., e. a. Accuracy of electronic health record data for identifying stroke cases in large- scale epidemiological studies: a systematic review from the UK Biobank Stroke Outcomes Group. PloS one 2015, 10, e0140533. [Google Scholar] [CrossRef]
  116. Chen, Z. China Kadoorie Biobank of 0.5 million people: survey methods, baseline characteristics and long-term follow-up. International Journal of Epidemiology 2011, 40, 1652– 1666. 2011, 40, 1652–1666, [https://academicoupcom/ije/article. Available online: https://academic.oup.com/ije/article-pdf/40/6/1652/2396544/dyr120.pdf. [CrossRef]
  117. Nagai, A. Overview of the BioBank Japan Project: Study design and profile. Journal of Epidemiology 2017, 27, S2–S8. [CrossRef]
  118. Johnson, A.E. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data 2019, 6, 317. [CrossRef] [PubMed]
  119. Mueller, S.G. The Alzheimer’s disease neuroimaging initiative. Neuroimaging Clinics of North America 2005, 15, 869. [CrossRef] [PubMed]
  120. Berle, J.O. Actigraphic registration of motor activity reveals a more structured behavioural pattern in schizophrenia than in major depression. BMC research notes 2010, 3, 1–7. [CrossRef]
  121. Bien, N.e.a. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: devel- opment and retrospective validation of MRNet. PLoS medicine 2018, 15, e1002699. [Google Scholar] [CrossRef] [PubMed]
  122. Shih, G.e.a. Augmenting the national institutes of health chest radiograph dataset with expert annotations of possible pneumonia. Radiology: Artificial Intelligence 2019, 1, e180041. [Google Scholar] [CrossRef] [PubMed]
  123. Demner-Fushman, e.a. Preparing a collection of radiology examinations for distribution and retrieval. Journal of the American Medical Informatics Association 2015, 23, 304–310. [Google Scholar] [CrossRef] [PubMed]
  124. Marcus, D.S.e.a. Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults. Journal of Cognitive Neuroscience 2007, 19, 1498–1507. [Google Scholar] [CrossRef] [PubMed]
  125. Wang, X. ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly- Supervised Classification and Localization of Common Thorax Diseases. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA; 2017; pp. 3462–3471. [CrossRef]
  126. Armato III, S.G.e.a. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical physics 2011, 38, 915–931. [Google Scholar] [CrossRef] [PubMed]
  127. Clark, K.e.a. The Cancer Imaging Archive (TCIA): maintaining and operating a public informa- tion repository. Journal of digital imaging 2013, 26, 1045–1057. [Google Scholar] [CrossRef]
  128. Johnson, A.E.e.a. MIMIC-III, a freely accessible critical care database. Scientific data 2016, 3, 1–9. [Google Scholar] [CrossRef]
  129. PPollard, T.J.e.a. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific data 2018, 5, 1–13. [Google Scholar] [CrossRef]
  130. Uzuner, O.e.a. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association 2011, 18, 552–556. [Google Scholar] [CrossRef]
  131. Sun, W.e.a. Evaluating temporal relations in clinical text: 2012 i2b2 Challenge. Journal of the American Medical Informatics Association 2013, 20, 806–813. [Google Scholar] [CrossRef] [PubMed]
  132. Stubbs, A.; Uzuner, Ö. Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus. Journal of biomedical informatics 2015, 58, S20–S29. [Google Scholar] [CrossRef]
  133. Yang, X.e.a. Identifying relations of medications with adverse drug events using recurrent convolutional neural networks and gradient boosting. Journal of the American Medical Informatics Association 2019, 27, 65–72. [Google Scholar] [CrossRef] [PubMed]
  134. Tomczak, K.e.a. Review The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge. Contemporary Oncology/Współczesna Onkologia 2015, 2015, 68–77. [Google Scholar] [CrossRef]
  135. De Herrera, e.a. Overview of the medical tasks in ImageCLEF 2016. CLEF working notes Evora, Portugal 2016.
  136. Demner-Fushman, D.; Antani, S.; Simpson, M.; Thoma, G.R. Design and development of a mul- timodal biomedical information retrieval system. Journal of Computing Science and Engineering 2012, 6, 168–177. [Google Scholar] [CrossRef]
  137. Goldberger, A.L.e.a. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. circulation 2000, 101, e215–e220. [Google Scholar] [CrossRef]
  138. Li, B.; Weng, Y.; Xia, F.; Sun, B.; Li, S. VPAI_LAB at MedVidQA 2022: a two-stage cross-modal fusion method for medical instructional video classification. In Proceedings of the Proceedings of the 21st Workshop on Biomedical Language Processing; 2022; pp. 212–219. [Google Scholar]
  139. Lucey, P.; Cohn, J.F.; Prkachin, K.M.; Solomon, P.E.; Matthews, I. Painful data: The UNBC- McMaster shoulder pain expression archive database. In Proceedings of the 2011 IEEE In- ternational Conference on Automatic Face I& Gesture Recognition (FG); 2011; pp. 57–64. [Google Scholar] [CrossRef]
  140. Gupta, D.; Attal, K.; Demner-Fushman, D. A dataset for medical instructional video classification and question answering. Scientific Data 2023, 10, 158. [Google Scholar] [CrossRef]
  141. Wishart, D.S. DrugBank 5.0: a major update to the DrugBank database for 2018. Nucleic acids research 2018, 46, D1074–D1082. [CrossRef]
  142. Bizon, C., e. a. ROBOKOP KG and KGB: integrated knowledge graphs from federated sources. Journal of chemical information and modeling 2019, 59, 4968–4973. [Google Scholar] [CrossRef] [PubMed]
  143. Ernst, P. KnowLife: A knowledge graph for health and life sciences. In Proceedings of the 2014 IEEE 30th International Conference on Data Engineering; 2014; pp. 1254–1257. [CrossRef]
  144. SSchriml, L.M. Disease Ontology: a backbone for disease semantic integration. Nucleic Acids Research 2011, 40, D940–D946. Available online: https://academicoupcom/nar/ARTICLE. [CrossRef]
  145. Su, C.e.a. Biomedical discovery through the integrative biomedical knowledge hub (iBKH). Iscience 2023, 26. [Google Scholar] [CrossRef]
  146. Ashburner, M.; Ball, C.A.; Blake, J.A.; Botstein, D.; Butler, H.; Cherry, J.M.; Davis, A.P.; Dolinski, K.; Dwight, S.S.; Eppig, J.T.; et al. Gene ontology: tool for the unification of biology. Nature genetics 2000, 25, 25–29. [Google Scholar] [CrossRef]
  147. Ranjan, A.; Shukla, S.; Datta, D.; Misra, R. Generating novel molecule for target protein (SARS- CoV-2) using drug–target interaction based on graph neural network. Network Modeling Analysis in Health Informatics and Bioinformatics 2022, 11, 1–11. [Google Scholar] [CrossRef]
  148. Himmelstein, D.S.; Baranzini, S.E. Heterogeneous network edge prediction: a data integration approach to prioritize disease-associated genes. PLoS computational biology 2015, 11, e1004259. [Google Scholar] [CrossRef] [PubMed]
  149. Li, B.; Chen, H. Prediction of compound synthesis accessibility based on reaction knowledge graph. Molecules 2022, 27, 1039. [Google Scholar] [CrossRef] [PubMed]
  150. Kanehisa, M. The KEGG database. In Proceedings of the ‘In silico’simulation of biological processes: Novartis Foundation Symposium 247. Wiley Online Library, 2002, Vol. 247, pp. 91–103. [CrossRef]
  151. Jeong, J.; Lee, N.; Shin, Y.; Shin, D. Intelligent generation of optimal synthetic pathways based on knowledge graph inference and retrosynthetic predictions using reaction big data. Journal of the Taiwan Institute of Chemical Engineers 2022, 130, 103982. [Google Scholar] [CrossRef]
  152. Zhang, R.; Hristovski, D.; Schutte, D.; Kastrin, A.; Fiszman, M.; Kilicoglu, H. Drug repurposing for COVID-19 via knowledge graph completion. Journal of biomedical informatics 2021, 115, 103696. [Google Scholar] [CrossRef]
  153. Serra, L.M.; Duncan, W.D.; Diehl, A.D. An ontology for representing hematologic malignancies: the cancer cell ontology. BMC bioinformatics 2019, 20, 231–236. [Google Scholar] [CrossRef]
  154. Gao, Z.; Ding, P.; Xu, R. KG-Predict: A knowledge graph computational framework for drug repurposing. Journal of biomedical informatics 2022, 132, 104133. [Google Scholar] [CrossRef] [PubMed]
  155. Chandak, P.; Huang, K.; Zitnik, M. Building a knowledge graph to enable precision medicine. Scientific Data 2023, 10, 67. [Google Scholar] [CrossRef] [PubMed]
  156. Zheng, S.; Rao, J.; Song, Y.; Zhang, J.; Xiao, X.; Fang, E.F.; Yang, Y.; Niu, Z. PharmKG: a dedicated knowledge graph benchmark for bomedical data mining. Briefings in bioinformatics 2021, 22, bbaa344. [Google Scholar] [CrossRef] [PubMed]
  157. Zhang, X.; Che, C. Drug repurposing for Parkinson’s disease by integrating knowledge graph completion model and knowledge fusion of medical literature. Future Internet 2021, 13, 14. [Google Scholar] [CrossRef]
  158. Wang, X.e.a. Large-scale multi-modal pre-trained models: A comprehensive survey. Machine Intelligence Research 2023, 20, 447–482. [Google Scholar] [CrossRef]
  159. Du, C. Multimodal deep generative adversarial models for scalable doubly semi- supervised learning. Information Fusion 2021, 68, 118–130. [Google Scholar] [CrossRef]
  160. Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE conference on computer vision and pattern recognition. Ieee; 2009; pp. 248–255. [Google Scholar] [CrossRef]
  161. Zhou, T.e.a. GAN review: Models and medical image fusion applications. Information Fusion 2023, 91, 134–148. [Google Scholar] [CrossRef]
  162. Kingma, D. P. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. 2013. arXiv:1312.6114 2013. [CrossRef]
  163. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020. [CrossRef]
  164. Akbari, H.e.a. Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text. Advances in Neural Information Processing Systems 2021, 34, 24206–24221. [Google Scholar] [CrossRef]
  165. Lin, Y.S.e.a. What do you see? Evaluation of explainable artificial intelligence (XAI) in- terpretability through neural backdoors. In Proceedings of the Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining; 2021; pp. 1027–1035. [Google Scholar] [CrossRef]
  166. Selvaraju, R.e.a. Grad-CAM: Why did you say that? arXiv:1611.07450 2016. [CrossRef]
  167. Simonyan, K.; Vedaldi, A.; Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034. 2013. arXiv:1312.6034 2013. [CrossRef]
  168. Springenberg, J.T.; Dosovitskiy, A.; Brox, T.; Riedmiller, M. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806. 2014. arXiv:1412.6806.
  169. Shrikumar, A.; Greenside, P.; Kundaje, A. Learning important features through propagating activation differences. In Proceedings of the International conference on machine learning. PMLR; 2017; pp. 3145–3153. [Google Scholar] [CrossRef]
  170. Shrikumar, A.; Greenside, P.; Shcherbina, A.; Kundaje, A. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713. 2016. arXiv:1605.01713 2016. [CrossRef]
  171. Zeiler, M.D.; Fergus, R. Visualizing and understanding convolutional networks. In Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13. Springer, 2014, pp. 818–833. [CrossRef]
  172. Smilkov, D.; Thorat, N.; Kim, B.; Viégas, F.; Wattenberg, M. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 2017. [CrossRef]
  173. Castro, J.; Gómez, D.; Tejada, J. Polynomial calculation of the Shapley value based on sampling. Computers & Operations Research 2009, 36, 1726–1730. [Google Scholar] [CrossRef]
  174. Fisher, A.; Rudin, C.; Dominici, F. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research 2019, 20, 1–81. [Google Scholar] [CrossRef]
Preprints 108939 g001
Figure 1. Difference between knowledge-based, non-knowledge-based, and XAI-based CDSS.
Figure 1. Difference between knowledge-based, non-knowledge-based, and XAI-based CDSS.
Preprints 108939 g001
Preprints 108939 g002
Figure 2. Overview of related work.
Figure 2. Overview of related work.
Preprints 108939 g002
Preprints 108939 g003
Figure 3. Trade-off between Performance and Interpretability.
Figure 3. Trade-off between Performance and Interpretability.
Preprints 108939 g003
Preprints 108939 g004
Figure 4. XAI-based CDSS Framework.
Figure 4. XAI-based CDSS Framework.
Preprints 108939 g004
Preprints 108939 g005
Figure 5. XAI Taxonomy.
Figure 5. XAI Taxonomy.
Preprints 108939 g005
Preprints 108939 g006
Figure 6. Applications of XAI-based CDSS.
Figure 6. Applications of XAI-based CDSS.
Preprints 108939 g006
Table 1. Knowledge-based CDSS.
Table 1. Knowledge-based CDSS.
Preprints 108939 i001
Table 14. Datasets.
Table 14. Datasets.
Category Modality Dataset Features
Single- modality Image MURA[120] Musculoskeletal radiology images
MRNet[121] MRI images
RSNA[122] Chest X-ray images
Demner, F., 2016[123] Chest X-ray images
OASIS[124] MRI images
ADNI[119] CT images
X. Wang, 2017[125] Chest X-ray images
Armato III 2011[126] CT images
TCIA[127] Cancer tumor images
EHR MIMIC-III[128] Demographics, clinical history, diagnoses, prescrip- tions, physical information, etc
eICU[129] Management activities, clinical assessments, treat- ment plans, vital sign measurements, etc.
Text 2010 i2b2/VA[130] Discharge statements, progress reports, radiology reports, pathology reports
2012 i2b2[131] Discharge statements
2014 i2b2/UTHealth[132] Longitudinal medical record
2018 n2c2[133] Discharge statements
Multi-Modality Genome, Image TCGA[134] Genome, medical images
Genome, Image, EHR UK Biobank[115] Clinical information, genomic analysis, exon variant testing, whole genome sequencing testing, MRI, etc.
Image, Text ImageCLEFmed[135] Diagnostic images, visible light images, signals, waves, etc.
Openi[136] Medical articles, images
Multiple signal PhysiNet[137] Biomedical signals
Video MedVidCL[138] Medical instruction images
UNBC-McMaster[139] Patient with shoulder pain
MedVidOA[140] Medical instruction images
Table 5. Clinical Knowledge Graphs.
Table 5. Clinical Knowledge Graphs.
CDSS Bio-informatics Medicine Pharmaceutical Chemistry
HKG DrugBank[141] Gene Ontology[146] GEFA[147] HetioNet[148]
POBOKOP[142] Reaction[149]
KnowLife[143] KEGG[150] ASICS[151] DrKG[152]
Disease Ontology[144] Hetionet[152]
iBKH[145] Cell Ontology[153] GP-KG[154] PrimeKG[155]
PharmKG[156] DRKF[157]
Table 6. XAI Models.
Table 6. XAI Models.
Preprints 108939 i004a
Preprints 108939 i004b
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated