Article
Version 1
This version is not peer-reviewed
Fusion-Enhanced Multimodal Social Media Analysis
Version 1
: Received: 9 August 2024 / Approved: 12 August 2024 / Online: 12 August 2024 (10:14:37 CEST)
How to cite: Jackson, A.; Patel, R.; Taylor, E.; Anderson, M. Fusion-Enhanced Multimodal Social Media Analysis. Preprints 2024, 2024080779. https://doi.org/10.20944/preprints202408.0779.v1 Jackson, A.; Patel, R.; Taylor, E.; Anderson, M. Fusion-Enhanced Multimodal Social Media Analysis. Preprints 2024, 2024080779. https://doi.org/10.20944/preprints202408.0779.v1
Abstract
The analysis of social media content plays a crucial role in uncovering intricate user behaviors and trends across diverse digital platforms. Social media is inherently multimodal, incorporating texts, images, audios, and videos, each offering unique insights into user engagement and preferences. Traditional classification methods often focus narrowly on the most prominent modality, typically neglecting the synergistic potential of integrating multiple data types. To address this gap, we introduce the Unified Multimodal Classifier (UMC), a suite of streamlined models that adeptly harness and integrate these varied modalities. UMC leverages a novel architecture that combines a pooling layer with auxiliary learning tasks, facilitating the formation of a robust and shared feature space. This integration allows UMC to not only accommodate but also capitalize on the inherent diversity of social media data. The models are designed to be inherently flexible, adjusting to the availability of data modalities and maintaining high classification accuracy under varied conditions. In emotion classification scenarios, UMC has shown exceptional performance, significantly outperforming traditional methods by effectively synthesizing information across modalities. Its robustness is highlighted in its ability to deliver consistent results, even in the absence of one or more modalities. The simplicity and efficiency of UMC make it a potent tool for social media analytics, enabling it to achieve higher accuracies and maintain effectiveness even with limited data inputs. This adaptive capability ensures that UMC can be deployed in real-world applications where data incompleteness is common, thus broadening the scope of its applicability in various analytical contexts.
Keywords
multimodal classification; social media analysis; emotion recognition
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (0)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment
