preprints.org > computer science and mathematics > artificial intelligence and machine learning > doi: 10.20944/preprints202408.0520.v1 (registering DOI)

Preprint Article Version 1 This version is not peer-reviewed

Version 1 : Received: 7 August 2024 / Approved: 7 August 2024 / Online: 7 August 2024 (14:13:00 CEST)

As the use of AI in medical imaging has increased, so has the need to explain a model’s results. Segmentation models are one technique used to produce explainable results. Due to larger size and sophistication, segmentation models which operate on 2D data can often produce better results than models operating on 3D data. In the real world, imaging is combined with clinical factors for diagnosis. To replicate this, multimodal models are used which combine image and text modalities. I propose a multimodal arrangement that converts 3D scans to 2D and uses a 2D segmentation model (DeepLabV3) to analyze the images. This is combined with clinical biomarkers to achieve a complete confidence score. Using the Medical Segmentation Decathlon Lung (MSDL) dataset and the LUng CAncer Screening dataset (LUCAS), I achieved a testing Dice coefficient of 0.91 on segmentation with a receiver operating characteristic (ROC), average precision (AP), and F-Score of 0.89, 0.91, and 0.85 respectively on the final multimodal results. This approach raises the F-score from the previously achieved value of 0.508 to 0.85, creating a new baseline of what is achievable with multimodal lung cancer diagnostics and possible methods to achieve these accuracies.

multimodal; deep learning; lung cancer; semantic segmentation; CT scan; predictive biomarkers; 2D vs. 3D

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

* All users must log in before leaving a comment