Prerpints.org logo
Preprint
Article

Machine Learning Classifiers for Predictive Biomarkers Combining Clinical and Radiomic Data in Testicular Cancer

Altmetrics

Downloads

195

Views

88

Comments

0

A peer-reviewed article of this preprint also exists.

supplementary.pdf (502.02KB )

Submitted:

18 October 2023

Posted:

18 October 2023

You are already at the latest version

Alerts
Abstract
Radiomics refers to the extraction and analysis of a wide range of medical imaging features in a non-invasive and cost-effective manner to comprehensively characterise tumours. In this study, machine learning models combining radiomics and clinical factors were developed to predict retroperitoneal lymph node metastasis in testicular germ cell tumours (TGCTs), with the aim of reducing unnecessary treatment in this group of young patients. Ninety-one patients with surgically proven testicular germ cell tumours and contrast-enhanced CT were included in this retrospective study. After segmenting 273 retroperitoneal lymph nodes using dedicated radiomics software, we developed machine-learning prediction models using Random Forest (RF), Light Gradient Boosting Machine (LGBM), Support Vector Machine Classifier (SVC), and K-Nearest Neighbours (KNN). For each classifier, we developed a radiomics-only, clinical-only, and combined radiomics-clinical prediction model. The models’ performances were evaluated using the area under the receiver operating characteristic curves (AUCs). The RF-based combined clinical and radiomic model showed the most robust performance in predicting LNM with an area under the curve (AUC) of 0.95 (±0.03; 95% CI), accuracy 87%, precision 89%, recall 86% and F1 score 87%, followed by the LGBM model with an area under the curve (AUC) of 0.93 (±0.05; 95% CI), accuracy 83%, precision 87%, recall 80% and F1 score 82%. Decision curve analysis demonstrated the clinical utility of our proposed RF-based combined clinical–radiomics model. Our study has identified reliable and predictive machine-learning techniques for predicting lymph node metastasis in early-stage testicular cancer. Identifying the most effective machine-learning approaches for predictive analysis based on radiomics integrating clinical risk factors can expand the applicability of radiomics in precision oncology and cancer treatment. Multi-centre validation is required to provide high-quality evidence for clinical application.
Keywords: 
Subject: Medicine and Pharmacology  -   Urology and Nephrology

1. Introduction

Testicular germ cell tumours (TGCTs) are the most common form of cancer in males between 15 and 40 years (1,2). This disease is distinguished from other types of cancer by its unique patient population and high treatment success rates, representing an outstanding achievement in cancer treatment (3,4).
Apart from cure, minimising treatment's immediate and long-term side effects is the main objective. This is of particular importance because of the young age of the patients and their longevity after cure (5–10).
TGCTs are diagnosed by physical examination, ultrasound scans and specific markers such as alpha-fetoprotein (AFP), beta-hCG and lactate dehydrogenase (LDH) (11,12).There are two main histological types: seminoma and non-seminoma, with the latter including both pure non-seminoma and mixed germ cell tumours.
Considering that 95% of metastases from testicular germ cell tumours involve the ipsilateral retroperitoneal lymph nodes, German guidelines recommend adjuvant therapy (one to two cycles of carboplatin or 20 Gy paraaortic radiotherapy) for early-stage seminoma if the tumour is larger than 4 cm in diameter (13).
Although the common tumour markers AFP, beta-HCG and LDH are widely used, they lack specificity and are present in approximately 60% of cases of testicular cancer (12,14). Furthermore, conditions such as liver disease or genetic factors can cause these markers to become falsely elevated (15). Without reliable biomarkers for predicting recurrence, TGCT patients are usually followed with active surveillance (16).
CT scans are effective at detecting enlarged lymph nodes. However, they have difficulty distinguishing between benign and infiltrated nodes, especially smaller ones (17). Lymph nodes are usually considered malignant if their short-axis diameter is between 7-8 mm, but it's important to note that micrometastatic lymph nodes can be present in 30% of cases (18). To date, imaging, tumour markers and clinical nomograms are unreliable in predicting lymph node metastasis (19,20).
Significant advances have been made in medical imaging by integrating high-resolution imaging, advanced computing technologies and artificial intelligence (AI). This integration has paved the way for the emerging field of radiomics (21,22).
Radiomics enables the identification and extraction of specific diagnostic image patterns, which are then transformed into quantifiable and analysable 'big data' through data characterisation algorithms (23,24). In precision medicine, AI image analysis can help identify important image details that human radiologists may miss, offering repeatable and accessible ways to assess challenging lesions in the body to improve the detection, classification and monitoring of both the primary tumour and its associated metastases in various cancers, such as gastric, rectal and bladder cancers (25–28).
Our presented study combines machine learning-based radiomics with clinical predictors to improve the accuracy of predicting lymph node metastasis in early-stage TGCT patients. This advancement has the potential to significantly improve the accuracy of imaging in the clinical oncology setting.

2. Materials and Methods

2.1. Patients

This retrospective study includes retroperitoneal lymph nodes of early-stage testicular cancer patients from January 2006 to December 2016.
A comprehensive review of electronic medical records and the radiology information system was used to collect patient demographic, laboratory and clinical data. Incomplete clinical or imaging records as well as missing histological confirmation following surgery were used as exclusion criteria.
The study's primary objective was to investigate retroperitoneal LN metastases from TGCT using clinical and imaging studies retrieved from electronic medical records. From an initial screening of 167 patients, only 91 patients met the selection criteria and were part of the final cohort.
Preprints 88081 g001
Figure 1. Recruitment pathway.
Figure 1. Recruitment pathway.
Preprints 88081 g001

2.2. Image acquisition and segmentation

All patients underwent contrast-enhanced CT according to standard clinical scanning protocols (detailed scanning parameters are provided in supplemental table S1) The images were acquired as part of the routine staging procedure prior to orchiectomy to determine the status of the disease (+/- 2 weeks, mean time 3 ± 11 days, range 2 to 24 days).
For image segmentation and analysis, all reconstructed images were retrieved from the hospital's picture archiving and communication system (PACS).

2.3. Segmentation and Radiomic Feature Extraction

The evaluation of image features, such as histogram features or features from the co-occurrence matrix, was first introduced by Haralick et al. in 1973 (29) and has shown considerable potential in different types of cancer and for different types of questions (22,30).
Two experienced radiologists (>10 years' experience in interpreting CT scans and with a strong background in texture analysis) contoured three retroperitoneal infrarenal lymph nodes per patient using Mint Lesion software (v3.8.4). Regions of interest (ROIs) were drawn on the CT images along the lesion contour on each successive slice within the boundaries of the retroperitoneal lymph node, excluding adjacent vessels, fat and normal tissue. The flowchart illustrating the ROI segmentation and feature extraction used to develop the model is shown in Figure 2.
Texture feature descriptors were used according to the guidelines of the Image Biomarker Standardisation Initiative (IBSI) (24). A total of 85 image features were extracted from each of the 273 ROIs, covering the size and shape of the lymph node in three dimensions. In addition, the distribution of voxel intensities within the ROI was described using first-order statistics. Texture-based features were calculated from the grey level co-occurrence matrix (GLCM) to capture voxel intensity patterns. A list of all features and parameters is provided in supplemental table S1.

2.4. Development of the Predictive Machine Learning Models

We used four classic machine learning algorithms to identify the best radiomics model for predicting lymph node metastases in testicular cancer: Random Forest (RF), Light Gradient Boosting Machine (LGBM), Support Vector Machine Classifier (SVC), and K-nearest neighbors (KNN) classifiers.
To improve the model's performance and maximise the area under the receiver operating characteristic curve (AUC-ROC), the optimal hyperparameters of the model were determined using a grid search procedure (31). (Details are provided in supplemental table S2).
We constructed a radiomics-only, clinical-only, and combined radiomics-clinical prediction model for each classifier.
Clinical factors included in our analysis were age, pre-orchiectomy serum tumour markers AFP and B-HCG, histotype (seminoma vs. non-seminoma) and body mass index (BMI). These factors were included because they all have shown predictive value in TGCT (32–35).
The predictive performance of each classifier for LN metastases was evaluated using receiver operating characteristic (ROC) curve analysis. The clinical utility of the predictive models was assessed using decision curve analysis (DCA) which evaluates the net benefit of the predictive models at different cut-offs in the training population and compares the performance of the models.
The clinical radiomics prediction models were constructed using custom-developed software implemented using the Python Scikit-learn package (36,37). (Details are provided in supplemental table S2).

3. Results

3.1. Clinicopathological Characteristics

Age, AFP levels, HCG levels, histotype and body mass index (BMI) were not statistically significantly different between LNM-positive and LNM-negative patients.
The baseline clinicopathological characteristics of the patients are listed in supplemental table S3).

3.2. Dataset Characteristics and Preprocessing

Three infrarenal retroperitoneal lymph nodes were segmented per patient, yielding 273 sample instances. We used a group shuffle split to divide the data into training (70%) and test (30%) groups on a patient-by-patient basis. This patient-by-patient split ensured that the lymph nodes of a given patient would remain together in either the training set or the test set. A tenfold cross-validation checked the robustness of the procedure.
There were 33 instances in the category “relapse of disease in terms of lymph node metastases” (minority class) and 240 instances in the category “without relapse of disease” (majority class).
Due to class imbalance in the dataset, we used the SMOTE oversampling technique to balance the data (38). The balanced data were used as input variables for the machine learning modelling.

3.3. Performance Evaluation of the Prediction Models

85 CT-derived radiomic features were fed into the machine-learning models using RF, LGBM, SVC and KNN (a list of all features and parameters is provided in supplemental table S1).
Based on the random forest algorithm, the combined clinical-radiomics model showed the best prediction accuracy with 87% (AUC 0.95±0.03; 95% CI), indicating that adding radiomics features could improve the predictive performance. The second best performer was LGBM with an AUC of 0.93 (±0.05; 95% CI), accuracy 83%, precision 87%, recall 80% and F1 score 82%. Details of the performance of the radiomics-only, clinical-only and combined clinical-radiomics models of all classifiers are shown in supplementary Table S4. See Figure 3 with the merged confusion matrix of the Random Forest-based models using radiomics-only and clinical-only values for classification.
Figure 4 shows the receiver operating characteristic (ROC) curves for the clinical, the radiomics and the combined clinical-radiomics models based on random forest algorithms.
Figure 5 shows the receiver operating characteristic (ROC) curves for the clinical, the radiomics and the combined clinical-radiomics models for the RF, LGBM, SVC and KNN.
To assess the clinical utility of the clinical-radiomics model combination, a decision curve analysis was performed. Figure 6 shows the net utility versus threshold probability trade-offs between true positives and false positives.

4. Discussion

Medical imaging has made great advances in cancer diagnosis and treatment planning with the emergence of "radiomics" - a field involving high-throughput medical image data mining. Radiomics has significant potential to improve clinical decision support in cancer care by providing a non-invasive and cost-effective approach (23,39). As radiomics deals with large amounts of medical image data ("big data"), efficient methods are needed to extract relevant information from these large radiomic datasets (21).
In this study, we analysed the performance of different machine learning methods, namely random forest (RF), light gradient boosting machine (LGBM), support vector machine classifier (SVC) and k-nearest neighbour classifier (KNN), in predicting lymph node metastases in patients with early-stage testicular germ cell tumours (TGCTs).
We constructed radiomics-only, clinical-only and combined predictive models for each classifier, integrating clinical and radiomic features to identify patients who require adjuvant therapy and those who do not. We compared the results with what we found in the previous study, where we used logistic regression analysis to predict lymph node metastases.
Our main findings can be summarised by the following:
The combined radiomics–clinical model based on the random forest algorithm showed the best predictive accuracy of 87% (AUC 0.95 ± 0.03; 95% CI), indicating that the addition of clinical features improved the predictive performance (accuracy radiomics-only model 85% vs. clinical only model 79 %).
The LGBM classifier performed second best with a prediction accuracy of 83% (AUC 0.93 ±0.05; 95% CI). In contrast to the RF, the addition of clinical features to the radiomics prediction model led to a worsening of the predictive performance (accuracy radiomics-only model 85% vs. clinical-only model 73%).
In our analysis, SVC- and KNN-based prediction models performed significantly worse than RF and LGBM. However, it is worth noting that in both cases the combined radiomics-clinical model outperformed the radiomics-only prediction models.
This is in line with the results of our previous study (40) in which we demonstrated that logistic regression analysis is useful for the prediction of lymph node metastasis in TGCT patients, with the best predictive performance being the combined clinical-radiomics model with a predictive accuracy of 87% (AUC 0.95 ± 0.03; 95% CI)
The serum biomarkers AFP, β-HCG and LDH play an important role in the diagnosis and prognosis of TGCTs and are included in the prognostic index of the International Germ Cell Cancer Consensus Group(41). However, their sensitivity is limited, as around 40% of men have 'normal' levels at recurrence (42). Studies have suggested additional prognostic factors such as age and BMI, but their role is unclear and continues to be debated (32–35).
So far, there is no evidence that imaging, preoperative serum tumour markers or clinical nomograms can reliably predict nodal involvement (19,20). Inadequate management of TGCT patients places them at risk of adverse outcomes, as both overtreatment and undertreatment carry equal risks.
There are few studies on discriminating between benign and malignant LNs in testicular cancer. In patients with metastatic non-seminomatous testicular germ cell tumours, Baessler et al. (43) found that a CT radiomics-based machine learning classifier could predict lymph node histopathology after dissection following chemotherapy. They used a support vector machine learning classifier in their single-centre retrospective study of 80 patients and 204 lesions. The model distinguished between benign and malignant histopathologies with an accuracy of 81%.
But in contrast to our study, they didn't include clinical factors in their radiomics approach to improve diagnostic accuracy. They also split their moderate-sized dataset into three subgroups: 63 patients were assigned to train and only 19 to test. The splitting of data for validation purposes is common practice. But doing so reduces statistical power because the sample sizes in both groups are smaller than in the initial sample.
To overcome this problem, we used a cross-validation technique that makes use of multiple splits of the data to avoid overfitting while still providing accurate estimates of the coefficients of the model (44).
We are confident that our combined prediction model will generalise better to novel data due to our tenfold cross-validation approach, the a priori inhomogeneity of our dataset and the integration of clinical risk factors. Therefore, future prospective studies should be conducted to further validate our trained model.
To date, several clinical models have been developed to predict the dignity of LN metastases. But, these models have shown inconsistent results and have not yet been adopted for clinical decision-making (19,45,46).
In summary, the identification and implementation of novel biomarkers may be helpful for early diagnosis and disease recurrence monitoring.
The present study, however, has some limitations that we acknowledge. First, the study's retrospective nature and the small cohort sizes might have led to unavoidable selection bias.
Secondly, two different scanners were used to acquire the CT images. Thirdly, the results of this study were obtained from a single centre. Due to the high cure rate of stage I TGCTs, it is a challenge for power studies to investigate prognostic and predictive factors adequately. Therefore, prospective and multicentre validation is warranted to provide higher-level evidence in the following studies.
Fourth, due to small sample size and relapse events, we could not include the classical prognostic pathohistological factors (primary tumour size and rete testis invasion for seminoma and lymphovascular invasion and presence of embryonal carcinoma for non-seminoma) in our analysis. Their inclusion in the combined radiomics-clinical model may have further improved accuracy and is a promising addition for future validation studies.
Finally, in addition to protein-based tumour markers, non-coding RNAs, especially stem cell-associated microRNAs such as miR-371a-3p and miR-302/367 clusters, show superior sensitivity compared to traditional markers in the detection of newly diagnosed TGCT patients, demonstrating their potential as biomarkers (47,48).

5. Conclusion

In summary, our combined random forest-based radiomics-clinical model represents an exciting tool for better prediction of lymph node involvement in early-stage TGCTs, with the potential to reduce over- and undertreatment in this young patient population. Further validation in larger prospective clinical trials should combine this approach with novel clinical biomarkers.

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org.

Author Contributions

Conceptualisation, C.S.L., F.Z., C.G.L.; methodology, C.S.L., D.W. and M.G.; software, S.M., D.W. and M.G.; formal analysis, C.S.L. and C.G.L.; investigation, C.S.L. and C.G.L.; resources, C.S.L..; data curation, C.S.L. and C.G.L., writing—original draft preparation, C.S.L.; writing—review and editing, W.T., M.B., S.A.S., F.Z., C.B., F.Z.; M.G.; visualisation, C.S.L. and C.G.L., supervision, M.B.; project administration, C.S.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work is funded by ``NUM 2.0'' (FKZ: 01KX2121) as part of the Racoon Project.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the ethics committee of the medical faculty of the University of Ulm (protocol code 155/18, 25 April 2018).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

AFP Alpha-fetoprotein
AUC Area under the curve
CT Computed tomography
hCG Human chorionic gonadotropin
KNN K-nearest neighbors
LGBM Light Gradient Boosting Machine
LNs Lymph nodes
LNM Lymph node metastases
LR Logistic regression
ML Machine learning
RF Random Forest
ROC Receiver operating curve
ROI Region of interest
SVC Support Vector Machine Classifier
TGCT Testicular germ cell tumour

References

  1. Ruf CG, Isbarn H, Wagner W, Fisch M, Matthies C, Dieckmann KP. Changes in epidemiologic features of testicular germ cell cancer: age at diagnosis and relative frequency of seminoma are constantly and significantly increasing. In Elsevier; 2014; p. 33-e1.
  2. Bray F, Richiardi L, Ekbom A, Pukkala E, Cuninkova M, Møller H. Trends in testicular cancer incidence and mortality in 22 European countries: continuing increases in incidence and declines in mortality. International journal of cancer. 2006, 118, 3099–3111. [Google Scholar] [CrossRef]
  3. Einhorn, LH. Treatment of testicular cancer: a new and improved model. Journal of clinical oncology. 1990, 8, 1777–1781. [Google Scholar] [CrossRef]
  4. Kollmannsberger C, Tandstad T, Bedard PL, Cohn-Cedermark G, Chung PW, Jewett MA, et al. Patterns of relapse in patients with clinical stage I testicular cancer managed with active surveillance. J Clin Oncol. 2015, 33, 51–57. [Google Scholar] [CrossRef]
  5. Fung C, Sesso HD, Williams AM, Kerns SL, Monahan P, Zaid MA, et al. Multi-institutional assessment of adverse health outcomes among North American testicular cancer survivors after modern cisplatin-based chemotherapy. Journal of Clinical Oncology. 2017, 35, 1211. [Google Scholar] [CrossRef] [PubMed]
  6. Huddart R, Norman A, Shahidi M, Horwich A, Coward D, Nicholls J, et al. Cardiovascular disease as a long-term complication of treatment for testicular cancer. Journal of Clinical Oncology. 2003, 21, 1513–1523. [Google Scholar] [CrossRef] [PubMed]
  7. Travis LB, Ng AK, Allan JM, Pui CH, Kennedy AR, Xu XG, et al. Second malignant neoplasms and cardiovascular disease following radiotherapy. Journal of the National Cancer Institute. 2012, 104, 357–370. [Google Scholar] [CrossRef] [PubMed]
  8. Kerns SL, Fung C, Monahan PO, Ardeshir-Rouhani-Fard S, Zaid MIA, Williams AM, et al. Cumulative burden of morbidity among testicular cancer survivors after standard cisplatin-based chemotherapy: a multi-institutional study. Journal of Clinical Oncology. 2018, 36, 1505. [Google Scholar] [CrossRef] [PubMed]
  9. Agrawal V, Dinh Jr PC, Fung C, Monahan PO, Althouse SK, Norton K, et al. Adverse health outcomes among US testicular cancer survivors after cisplatin-based chemotherapy vs surgical management. JNCI cancer spectrum. 2020, 4, pkz079. [Google Scholar] [CrossRef]
  10. Tandstad T, Kollmannsberger CK, Roth BJ, Jeldres C, Gillessen S, Fizazi K, et al. Practice makes perfect: the rest of the story in testicular cancer as a model curable neoplasm. Journal of Clinical Oncology. 2017, 35, 3525. [Google Scholar] [CrossRef]
  11. Dieckmann KP, Simonsen-Richter H, Kulejewski M, Anheuser P, Zecha H, Isbarn H, et al. Serum tumour markers in testicular germ cell tumours: frequencies of elevated levels and extents of marker elevation are significantly associated with clinical parameters and with response to treatment. BioMed Research International. 2019, 2019. [Google Scholar]
  12. Gilligan TD, Hayes DF, Seidenfeld J, Temin S. ASCO clinical practice guideline on uses of serum tumor markers in adult males with germ cell tumors. Journal of oncology practice. 2010, 6, 199. [Google Scholar] [CrossRef]
  13. Kliesch S, Schmidt S, Wilborn D, Aigner C, Albrecht W, Bedke J, et al. Management of germ cell tumours of the testis in adult patients. German clinical practice guideline part I: epidemiology, classification, diagnosis, prognosis, fertility preservation, and treatment recommendations for localized stages. Urologia internationalis. 2021, 105, 169–180. [Google Scholar] [CrossRef] [PubMed]
  14. Murray MJ, Huddart RA, Coleman N. The present and future of serum diagnostic tests for testicular germ cell tumours. Nature Reviews Urology. 2016, 13, 715–725. [Google Scholar] [CrossRef]
  15. Albers P, Albrecht W, Algaba F, Bokemeyer C, Cohn-Cedermark G, Fizazi K, et al. Guidelines on testicular cancer: 2015 update. European urology. 2015, 68, 1054–1068. [Google Scholar] [CrossRef]
  16. Nichols CR, Roth B, Albers P, Einhorn LH, Foster R, Daneshmand S, et al. Active surveillance is the preferred approach to clinical stage I testicular cancer. Journal of Clinical Oncology. 2013, 31, 3490–3493. [Google Scholar] [CrossRef] [PubMed]
  17. Hale GR, Teplitsky S, Truong H, Gold SA, Bloom JB, Agarwal PK. Lymph node imaging in testicular cancer. Translational Andrology and Urology. 2018, 7, 864. [Google Scholar] [CrossRef]
  18. Hudolin T, Kastelan Z, Knezevic N, Goluza E, Tomas D, Coric M. Correlation between retroperitoneal lymph node size and presence of metastases in nonseminomatous germ cell tumors. International Journal of Surgical Pathology. 2012, 20, 15–18. [Google Scholar] [CrossRef]
  19. Steyerberg E, Gerl A, Fossa S, Sleijfer D, de Wit R, Kirkels W, et al. Validity of predictions of residual retroperitoneal mass histology in nonseminomatous testicular cancer. Journal of clinical oncology. 1998, 16, 269–274. [Google Scholar] [CrossRef]
  20. Vergouwe Y, STEYERBERG EW, FOSTER RS, HABBEMA JDF, DONOHUE JP. Validation of a prediction model and its predictors for the histology of residual masses in nonseminomatous testicular cancer. The Journal of urology. 2001, 165, 84–88. [Google Scholar] [CrossRef] [PubMed]
  21. Obermeyer Z, Emanuel EJ. Predicting the future—big data, machine learning, and clinical medicine. The New England journal of medicine. 2016, 375, 1216. [Google Scholar] [CrossRef]
  22. Lambin P, Leijenaar RT, Deist TM, Peerlings J, De Jong EE, Van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nature reviews Clinical oncology. 2017, 14, 749–762. [Google Scholar] [CrossRef]
  23. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016, 278, 563. [Google Scholar] [CrossRef] [PubMed]
  24. Zwanenburg A, Vallières M, Abdalah MA, Aerts HJ, Andrearczyk V, Apte A, et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020, 295, 328–338. [Google Scholar] [CrossRef] [PubMed]
  25. Dong D, Tang L, Li ZY, Fang MJ, Gao JB, Shan XH, et al. Development and validation of an individualized nomogram to identify occult peritoneal metastasis in patients with advanced gastric cancer. Annals of Oncology. 2019, 30, 431–438. [Google Scholar] [CrossRef] [PubMed]
  26. Huang Y qi, Liang C hong, He L, Tian J, Liang C shan, Chen X, et al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. Journal of clinical oncology. 2016, 34, 2157–2164. [Google Scholar] [CrossRef] [PubMed]
  27. Wu S, Zheng J, Li Y, Yu H, Shi S, Xie W, et al. A Radiomics Nomogram for the Preoperative Prediction of Lymph Node Metastasis in Bladder CancerA Radiomics Nomogram for Bladder Cancer. Clinical Cancer Research. 2017, 23, 6904–6911. [Google Scholar] [CrossRef] [PubMed]
  28. Gao J, Han F, Jin Y, Wang X, Zhang J. A radiomics nomogram for the preoperative prediction of lymph node metastasis in pancreatic ductal adenocarcinoma. Frontiers in oncology. 2020, 10, 1654. [Google Scholar] [CrossRef]
  29. Haralick RM, Shanmugam K, Dinstein IH. Textural features for image classification. IEEE Transactions on systems, man, and cybernetics. 1973, 6, 610–621. [Google Scholar]
  30. Shen C, Liu Z, Guan M, Song J, Lian Y, Wang S, et al. 2D and 3D CT radiomics features prognostic performance comparison in non-small cell lung cancer. Translational oncology. 2017, 10, 886–894. [Google Scholar] [CrossRef]
  31. Agrawal, T. Hyperparameter Optimization Using Scikit-Learn. In: Hyperparameter Optimization in Machine Learning: Make Your Machine Learning and Deep Learning Models More Efficient [Internet]. Berkeley, CA: Apress; 2021. p. 31–51. [CrossRef]
  32. Fosså SD, Cvancarova M, Chen L, Allan AL, Oldenburg J, Peterson DR, et al. Adverse prognostic factors for testicular cancer–specific survival: a population-based study of 27,948 patients. Journal of Clinical Oncology. 2011, 29, 963–970. [Google Scholar] [CrossRef]
  33. Parker C, Milosevic M, Panzarella T, Banerjee D, Jewett M, Catton C, et al. The prognostic significance of the tumour infiltrating lymphocyte count in stage I testicular seminoma managed by surveillance. European Journal of Cancer. 2002, 38, 2014–2019. [Google Scholar] [CrossRef]
  34. Lerro C, McGlynn K, Cook M. A systematic review and meta-analysis of the relationship between body size and testicular cancer. British journal of cancer. 2010, 103, 1467–1474. [Google Scholar] [CrossRef] [PubMed]
  35. Dieckmann KP, Hartmann JT, Classen J, Diederichs M, Pichlmeier U. Is increased body mass index associated with the incidence of testicular germ cell cancer? Journal of cancer research and clinical oncology. 2009, 135, 731–738. [Google Scholar] [CrossRef] [PubMed]
  36. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine learning in Python. Journal of machine Learning research 2011, 12, 2825–2830. [Google Scholar]
  37. van Rossum G, Drake FL. Python/C API Manual-Python 2.6. 2009.
  38. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research. 2002, 16, 321–357. [Google Scholar] [CrossRef]
  39. Lambin P, Leijenaar RT, Deist TM, Peerlings J, De Jong EE, Van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nature reviews Clinical oncology. 2017, 14, 749–762. [Google Scholar] [CrossRef]
  40. Lisson CS, Manoj S, Wolf D, Schrader J, Schmidt SA, Beer M, et al. CT Radiomics and Clinical Feature Model to Predict Lymph Node Metastases in Early-Stage Testicular Cancer. Onco 2023, 3, 65–80. [Google Scholar] [CrossRef]
  41. Eyben FE von. Laboratory markers and germ cell tumors. Critical reviews in clinical laboratory sciences 2003, 40, 377–427. [Google Scholar] [CrossRef]
  42. Trigo JM, Tabernero JM, Paz-Ares L, García-Llano JL, Mora J, Lianes P, et al. Tumor markers at the time of recurrence in patients with germ cell tumors. Cancer 2000, 88, 162–168. [Google Scholar] [CrossRef]
  43. Baessler B, Nestler T, Pinto dos Santos D, Paffenholz P, Zeuch V, Pfister D, et al. Radiomics allows for detection of benign and malignant histopathology in patients with metastatic testicular germ cell tumors prior to post-chemotherapy retroperitoneal lymph node dissection. European Radiology. 2020, 30, 2334–2345. [Google Scholar] [CrossRef]
  44. Harrell Jr FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in medicine. 1996, 15, 361–387. [Google Scholar] [CrossRef]
  45. Leão R, Nayan M, Punjani N, Jewett MAS, Fadaak K, Garisto J, et al. A New Model to Predict Benign Histology in Residual Retroperitoneal Masses After Chemotherapy in Nonseminoma. European Urology Focus. 2018, 4, 995–1001. [Google Scholar] [CrossRef] [PubMed]
  46. Vergouwe Y, Steyerberg EW, Foster RS, Sleijfer DT, Fosså SD, Gerl A, et al. Predicting Retroperitoneal Histology in Postchemotherapy Testicular Germ Cell Cancer: A Model Update and Multicentre Validation with More Than 1000 Patients. European Urology. 2007, 51, 424–432. [Google Scholar] [CrossRef]
  47. Dieckmann KP, Radtke A, Spiekermann M, Balks T, Matthies C, Becker P, et al. Serum Levels of MicroRNA miR-371a-3p: A Sensitive and Specific New Biomarker for Germ Cell Tumours. European Urology. 2017, 71, 213–220. [Google Scholar] [CrossRef]
  48. Bezan A, Gerger A, Pichler M. MicroRNAs in testicular cancer: implications for pathogenesis, diagnosis, prognosis and therapy. Anticancer research. 2014, 34, 2709–2713. [Google Scholar]
Preprints 88081 g002
Figure 2. illustrates the schematic of the ROI segmentation and feature extraction process that provided the input for the development of the models, with decision analysis as the clinical utility evaluation tool.
Figure 2. illustrates the schematic of the ROI segmentation and feature extraction process that provided the input for the development of the models, with decision analysis as the clinical utility evaluation tool.
Preprints 88081 g002
Preprints 88081 g003
Figure 3. shows that most of the values are aligned along the main diagonal, indicating a high level of agreement between clinical and radiomic results. In particular, 38% are correctly classified as true positives (TP) and 40% as true negatives (TN) by both models. Both models misclassify 2% as false positive (FP). In addition, the radiomics model misclassifies 8% of cases as FP, but these are correctly identified by the clinical model. In contrast, the clinical model misclassified 7% FN and 6% FP.
Figure 3. shows that most of the values are aligned along the main diagonal, indicating a high level of agreement between clinical and radiomic results. In particular, 38% are correctly classified as true positives (TP) and 40% as true negatives (TN) by both models. Both models misclassify 2% as false positive (FP). In addition, the radiomics model misclassifies 8% of cases as FP, but these are correctly identified by the clinical model. In contrast, the clinical model misclassified 7% FN and 6% FP.
Preprints 88081 g003
Preprints 88081 g004
Figure 4. The ROC curves of the random forest-based prediction models show that the combined model outperforms the radiomics-only and the clinical-only model in predicting lymph node metastasis (95% vs 92% and 88%, respectively).
Figure 4. The ROC curves of the random forest-based prediction models show that the combined model outperforms the radiomics-only and the clinical-only model in predicting lymph node metastasis (95% vs 92% and 88%, respectively).
Preprints 88081 g004
Preprints 88081 g005
Figure 5. The ROC curves of the prediction models show that the combined RF-based model outperforms the LGBM-based model and the logistic regression analysis-based model. In our analysis, SVC- and KNN-based prediction models performed considerably worse.
Figure 5. The ROC curves of the prediction models show that the combined RF-based model outperforms the LGBM-based model and the logistic regression analysis-based model. In our analysis, SVC- and KNN-based prediction models performed considerably worse.
Preprints 88081 g005
Preprints 88081 g006
Figure 6. The decision curve shows that when the threshold probability is between 0 and 0.89, the use of the RF-based combined prediction model provides an increased benefit over treating all or none of the patients.
Figure 6. The decision curve shows that when the threshold probability is between 0 and 0.89, the use of the RF-based combined prediction model provides an increased benefit over treating all or none of the patients.
Preprints 88081 g006
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated