Robinson, K.; Martinez, A.; Turner, E. Enhanced Modal Fusion Learning for Multimodal Sentiment Interpretation. Preprints2024, 2024091887. https://doi.org/10.20944/preprints202409.1887.v1
APA Style
Robinson, K., Martinez, A., & Turner, E. (2024). Enhanced Modal Fusion Learning for Multimodal Sentiment Interpretation. Preprints. https://doi.org/10.20944/preprints202409.1887.v1
Chicago/Turabian Style
Robinson, K., Ava Martinez and Ethan Turner. 2024 "Enhanced Modal Fusion Learning for Multimodal Sentiment Interpretation" Preprints. https://doi.org/10.20944/preprints202409.1887.v1
Abstract
Multimodal sentiment analysis is rapidly gaining traction due to its ability to comprehensively interpret opinions expressed in video content, which is ubiquitous across various digital platforms. Despite its promising potential, the field is hindered by the limited availability of high-quality, annotated datasets, which poses substantial challenges to the generalizability of predictive models. Models trained on such scarce data often inadvertently assign excessive importance to irrelevant features, such as personal attributes (e.g., eyewear), thereby diminishing their accuracy and robustness. To address this issue, we propose an Enhanced Modal Fusion Learning (EMFL) methodology aimed at significantly improving the generalization capabilities of neural networks. EMFL achieves this by optimizing the integration and interpretation processes of multimodal data, ensuring that sentiment-relevant features are prioritized over confounding attributes. Through extensive experiments conducted on multiple benchmark datasets, we demonstrate that EMFL consistently elevates the accuracy of sentiment predictions across verbal, acoustic, and visual modalities. These findings underscore EMFL's efficacy in mitigating the impact of non-relevant features and enhancing the overall performance of multimodal sentiment analysis models.
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
