Dual-Modality Feature Blending: A Channel-Aware Modeling for Multimodal Integration

1. Revisiting the Challenge of Vision-Language Integration

Recent advancements in artificial intelligence have emphasized the growing importance of multimodal learning in developing general-purpose agents. In particular, tasks that involve interpreting visual content based on textual queries—such as Visual Question Answering (VQA) [20,97]—present a challenging yet rewarding frontier. These tasks require models to combine syntactic understanding with perceptual recognition, often involving spatial reasoning, object identification, and complex linguistic parsing. A simple question like “What type of drink is to the right of the soda bottle?” demands a cascade of capabilities, including spatial discrimination, object categorization, and language grounding.

With the need for robust cross-modal reasoning increasing, the focus has shifted to how best to integrate vision and language representations. A conventional approach, often referred to as merged attention, involves concatenating visual and textual tokens and processing them through a multimodal transformer encoder with shared self-attention [16,19,38,43]. Although merged attention offers simplicity and computational efficiency, it often struggles to accurately align relevant information across modalities, especially in more nuanced reasoning tasks.

In contrast, co-attention models adopt a more segregated approach by processing vision and text tokens through separate transformers and facilitating interaction through cross-attention layers [5,29,50]. This method allows each modality to retain its structural integrity while still enabling targeted information exchange. However, it incurs significant computational overhead and does not benefit from global token-level attention.

Empirical studies, such as those by [16], suggest that co-attention mechanisms can outperform merged attention in specific tasks. Nevertheless, the increased parameter count and disjointed attention flow present limitations when scalability and resource constraints are considered. These observations underscore the need for a hybrid strategy—one that retains the precision of co-attention while exploiting the simplicity and coverage of merged attention.

To address this gap, we propose CrossFusionTokens (XFT), a novel mechanism that introduces a dual cross-attention pathway followed by channel-wise fusion. Our strategy begins with using vision tokens to extract text-aligned features, forming compound visual tokens. Next, text tokens are employed to query visual features, producing compound textual tokens. By aligning these features at the channel level—rather than expanding the sequence—we preserve efficiency while maximizing contextual richness.

Unlike models that expand token length, XFT maintains a constant number of tokens, reducing computational load and enhancing model scalability. This design is especially important for generative tasks that rely on decoder-based architectures. Moreover, our channel fusion framework provides a seamless way to combine global and localized information from each modality without the need for repeated attention layers.

Our proposed method stands out by offering an architecture that efficiently balances alignment quality with computational practicality. The XFT framework provides significant performance improvements across several multimodal benchmarks. On SNLI-VE [55], our model achieves a remarkable accuracy of $84.53\%$, surpassing previous baselines like METER [16]. For GQA [20], we obtain a score of $83.24\%$, representing a notable leap over existing state-of-the-art systems. Even in the challenging VQA2.0 [97] dataset, XFT records $72.15\%$, highlighting its generalization capability.

Following the generation-based pipelines established by prior works [10,43,54], we assess the output quality under stringent exact match criteria. This evaluation scenario poses greater challenges than classification settings, emphasizing the robustness of token-level alignment in our design. Importantly, we also test encoder-only models to ensure consistency in low-resource scenarios, adhering to protocols similar to [43].

In conclusion, CrossFusionTokens (XFT) provides a scalable, effective, and theoretically grounded approach for vision-language fusion. It addresses the shortcomings of both merged attention and co-attention strategies by offering a balanced, efficient, and expressive framework. Our comprehensive evaluations across diverse benchmarks solidify XFT as a compelling advancement in the field of multimodal learning.

2. Related Work

The rapid development of vision-language learning has been fueled by the successes of large-scale pretrained models. Drawing inspiration from foundational language models such as T5 [45], BERT [14], and GPT-3 [4], the field has witnessed the emergence of powerful multimodal architectures like VilBERT [36], BEiT-3 [53], SimVLM [54], Flamingo [2], and PaLI [8]. These models are designed to exploit large-scale image-text data, leading to substantial progress across a diverse set of tasks, including visual dialog [7,12,25], visual reasoning [49,57], natural language entailment in the visual domain [9,55], visual question answering [3,21,54,97], automatic caption generation [2,6], and image-text retrieval [22,39].

One of the critical architectural shifts that enabled better scalability and flexibility was the transition from using region-based object detectors—such as Faster-RCNN [46] employed in earlier systems like [31,33,36,50,58]—to more generalized visual backbones like ResNet [18] and Vision Transformers [15]. By removing the reliance on costly, manually annotated datasets like Visual Genome [26], newer models have been able to leverage weakly supervised or web-scale datasets, thereby accelerating training efficiency and generalization to unseen domains.

Parallel to architectural changes, significant attention has been given to pretraining strategies and training objectives. Researchers have proposed a variety of methods for aligning vision and language features, including contrastive objectives [32,56], image-text matching [28,36], caption generation [2], prefix-based language modeling [54], and patch-to-word alignment [23]. Several works have integrated multiple learning signals within unified training pipelines [16,30], while others have built multi-task setups to jointly solve question answering, captioning, and retrieval problems within a shared model framework [37,41,43].

Despite these advancements, the design of effective fusion mechanisms for vision and language representations remains relatively underexplored. Most prior work adopts straightforward strategies such as concatenating visual and textual tokens before applying joint self-attention layers—an approach known as merged attention [16,38,43,54]. These models differ slightly in whether fusion occurs early or late in the pipeline, or whether separate encoders are used before merging. However, simple concatenation often fails to ensure precise token alignment across modalities, especially when dealing with fine-grained semantic dependencies.

An alternative line of work explores co-attention mechanisms, where separate encoders are used for vision and text, and interaction is achieved through specialized cross-attention modules [16,29,50]. Although this setup offers more controlled and interpretable modality interaction, it typically leads to increased model complexity and parameter count due to duplicated transformer layers.

In contrast, our work centers on enhancing the fusion strategy itself. Rather than relying on token-level concatenation or building modality-specific pipelines, we introduce a new approach—CrossFusionTokens (XFT)—that enriches the multimodal fusion process through channel-level integration of representations obtained via bi-directional cross-attention. XFT allows visual tokens to query linguistic context and vice versa, encouraging better semantic alignment without inflating sequence length or sacrificing computational efficiency. This channel-aware fusion introduces a refined and compact joint representation that leverages cross-modal signals while remaining scalable across tasks and data scales.

By emphasizing the fusion step rather than solely focusing on feature extraction or training objectives, XFT contributes a complementary perspective to existing work. Our approach aligns well with the broader goal of constructing unified, general-purpose vision-language systems capable of reasoning over complex, multimodal inputs.

3. Methodology

3.1. Preliminaries and Foundations

To better contextualize our approach, we begin by outlining essential concepts underlying the attention mechanism and multimodal fusion schemes. For clarity, we omit layer normalization, feedforward projections, and residual pathways, which are standard components in modern transformer-based architectures.

Attention Mechanism: Given query vectors $\mathbf{Q}\in {\mathbb{R}}^{N\times d}$ and key vectors $\mathbf{K}\in {\mathbb{R}}^{M\times d}$, the attention function aggregates contextual information from a set of value vectors $\mathbf{V}\in {\mathbb{R}}^{M\times c}$ by computing similarity scores between $\mathbf{Q}$ and $\mathbf{K}$. Specifically, for scaled dot-product attention [51], each output $z_{i,\ell }$ corresponding to the i-th query and ℓ-th output dimension is given by:

$$\begin{array}{cccccc}\hfill a_{i,j}& ={\displaystyle \frac{q_i^T{k}_j}{\sqrt{d}}},\hfill & \hfill {\alpha }_{i,j}& ={\displaystyle \frac{exp\left(a_{i,j}\right)}{{\sum }_{\ell }exp\left(a_{i,\ell }\right)}},\hfill & \hfill z_{i,\ell }& =\sum _j{\alpha }_{i,j}{\mathbf{V}}_{j,\ell }.\hfill \end{array}$$

(1)

This framework generalizes to multi-head attention by allowing multiple learned projections per head. When the queries are identical to the keys (i.e., $\mathbf{Q}=\mathbf{K}$), the attention is referred to as self-attention. Otherwise, it is termed cross-attention.

Multimodal Fusion Paradigms: A prevailing technique for cross-modal modeling involves token-level concatenation, where image embeddings $\mathcal{I}\in {\mathbb{R}}^{N\times d}$ and text embeddings $\mathcal{T}\in {\mathbb{R}}^{M\times d}$ are concatenated to form a unified sequence $\mathcal{O}=[\mathcal{I};\mathcal{T}]\in {\mathbb{R}}^{(N+M)\times d}$ that is then passed to a multimodal transformer. Output representations are typically derived via either classification heads or sequence decoders. Hybrid fusion designs employ self- and cross-attention within distinct layers [29,30,50], encouraging selective cross-modal interaction but often increasing architectural complexity.

3.2. Overview of the CrossFusionTokens Framework

The CrossFusionTokens (XFT) framework is designed to maximize the representational expressiveness of vision-language models while maintaining computational efficiency. It consists of four key components: a visual encoder, a text encoder, a cross-attentive fusion module with channel concatenation, and an autoregressive decoder. Each component is modular and trainable in an end-to-end manner. Figure 1 shows the overall model.

3.2.1. Visual and Textual Embedding Modules

The image encoder transforms raw pixel inputs into a sequence of dense visual tokens $\mathcal{I}=[v_1,v_2,\cdots ,v_N]\in {\mathbb{R}}^{N\times d}$ using a convolutional or transformer-based model. For consistency with prior work, we employ ResNet-50 [18] in our main experiments.

Simultaneously, the text input is tokenized and embedded using a pretrained language encoder such as T5-base [45]. The resulting text token matrix is $\mathcal{T}=[t_1,t_2,\cdots ,t_M]\in {\mathbb{R}}^{M\times d}$. Prior to cross-modal fusion, both $\mathcal{I}$ and $\mathcal{T}$ are linearly projected to a reduced feature dimension:

$$\begin{array}{cc}\hfill \tilde{\mathcal{I}}& =\mathcal{I}{W}_I,\tilde{\mathcal{T}}=\mathcal{T}{W}_T,W_I,W_T\in {\mathbb{R}}^{d\times d/2}.\hfill \end{array}$$

(2)

3.2.2. Cross-Modality Fusion with Channel Concatenation

Cross-attention is applied bidirectionally between visual and textual representations to retrieve cross-modal contextual signals. Specifically, we compute:

$$\begin{array}{cccc}\hfill \widehat{\mathcal{I}}& =\mathcal{A}(\tilde{\mathcal{I}},\tilde{\mathcal{T}},\tilde{\mathcal{T}}),\hfill & \hfill \widehat{\mathcal{T}}& =\mathcal{A}(\tilde{\mathcal{T}},\tilde{\mathcal{I}},\tilde{\mathcal{I}}).\hfill \end{array}$$

(3)

The resulting cross-attended outputs are concatenated with the original projected features to form compound tokens via channel concatenation:

$$\begin{array}{cccc}\hfill {\mathcal{I}}_{cmpd}& =[\tilde{\mathcal{I}};\widehat{\mathcal{I}}],\hfill & \hfill {\mathcal{T}}_{cmpd}& =[\tilde{\mathcal{T}};\widehat{\mathcal{T}}],\hfill \end{array}$$

(4)

which restores the representation dimensionality to d and enriches it with aligned information from the other modality.

3.2.3. Multimodal Token Aggregation and Encoding

After obtaining the compound representations, we concatenate them along the token dimension:

$$\begin{array}{c}\hfill {\mathcal{O}}_{cmpd}=[{\mathcal{I}}_{cmpd};{\mathcal{T}}_{cmpd}]\in {\mathbb{R}}^{(N+M)\times d}.\end{array}$$

(5)

This sequence is then passed through a multimodal transformer encoder, which leverages self-attention to allow global interaction between visual and textual tokens.

3.2.4. Decoder and Training Objective

The output from the encoder is forwarded to a T5-style decoder for generation. Given a target output sequence $Y=[y_1,y_2,...,y_T]$, we model the generation as:

$$\begin{array}{c}\hfill logP\left(Y\right|{\mathcal{O}}_{cmpd})=\sum _{t=1}^{T}logP\left(y_t\right|y_{<t},{\mathcal{O}}_{cmpd}).\end{array}$$

(6)

We optimize the model using a standard autoregressive cross-entropy loss:

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{CE}}=-\sum _{t=1}^{T}logP\left(y_t\right|y_{<t},{\mathcal{O}}_{cmpd}).\end{array}$$

(7)

Label smoothing [47] and learning rate warmup schedules are optionally applied to stabilize training.

3.3. Optimization and Implementation Details

All components of the XFT framework are differentiable and jointly trainable. The encoders and decoder are initialized from pretrained checkpoints to accelerate convergence. We use the AdamW optimizer with ${\beta }_1=0.9$, ${\beta }_2=0.98$, and a learning rate scheduler with linear warmup followed by cosine decay.

During training, input sequences are truncated or padded to fixed lengths N and M for the image and text, respectively. Gradient checkpointing is applied to reduce memory usage, and mixed-precision training is used to accelerate computation.

Overall, CrossFusionTokens provides a modular, extensible, and efficient architecture that facilitates high-quality cross-modal reasoning through carefully designed fusion operations and transformer-based contextualization.

4. Experiments

4.1. Model Configuration and Setup

We adopt the CrossFusionTokens (XFT) framework and instantiate it with ResNet-50 [18] as the visual backbone and T5-base [45] for both the text encoder and decoder components. The visual encoder is initialized from ImageNet-1k [13] pretrained weights, while the language modules are initialized from standard T5 checkpoints. All components are optimized in an end-to-end manner. During inference, the decoder generates free-form text answers conditioned on fused representations.

4.2. Datasets and Tasks

We benchmark our model across three vision-language reasoning tasks:

SNLI-VE [55]: A large-scale visual entailment dataset with 500K image-text pairs, requiring classification into entailment, neutral, or contradiction categories.

VQA2.0 [97]: A widely used question answering benchmark with over 400K samples, where each image-question pair includes multiple annotated answers.

GQA [20]: A compositional VQA dataset derived from Visual Genome [26], featuring 22M question-answer pairs targeting relational and spatial reasoning.

For all datasets, we use open-vocabulary evaluation. Outputs are considered correct only when they exactly match the ground-truth answers. We use accuracy for SNLI-VE and GQA, and the standard VQA metric for VQA2.0.

4.3. Pretraining Strategy

To better initialize our models, we perform multimodal pretraining on CC3M [48] and COCO Captions [34]. The training objectives include:

• Image-text Matching (ITM): Binary classification on whether image-text pairs are aligned.
• Caption Generation: Auto-regressive generation of full captions.
• Masked Caption Completion: Predicting masked tokens in partial captions.
• Masked Language Modeling (MLM): Predicting masked tokens in text, following BERT-style pretraining [14].

4.4. Training Configuration

Models are pretrained for 300K steps with a batch size of 512 and finetuned for 100K steps with batch size 128. During pretraining, images are resized to $224\times 224$ and increased to $384\times 384$ during finetuning. Text inputs and outputs are truncated to length 32 and 8 respectively. Optimization uses AdamW with learning rate $5e^{-5}$, weight decay 0.01, and cosine decay scheduling. We apply label smoothing ($ϵ=0.1$) during classification tasks.

4.5. Effectiveness of Channel Fusion

We compare several fusion techniques for forming compound tokens:

• Channel Concatenation: XFT’s strategy.
• Learned Weighting: $Y=\alpha q+\beta X$ with learnable scalars.
• Element-wise Multiplication: $Y=q\odot X$.
• Simple Summation: $Y=q+X$.

Table 1. Comparison of fusion techniques for SNLI-VE and GQA. Channel concatenation consistently yields the best performance.

Fusion Method	GFlops	SNLI-VE Acc.	GQA Acc.
Channel Concat.	20.71	80.85	80.79
Learned Weighting	20.71	80.63	80.61
Summation	20.71	80.75	80.35
Element-wise Product	20.71	80.81	78.31

Table 1. Comparison of fusion techniques for SNLI-VE and GQA. Channel concatenation consistently yields the best performance.

Fusion Method	GFlops	SNLI-VE Acc.	GQA Acc.
Channel Concat.	20.71	80.85	80.79
Learned Weighting	20.71	80.63	80.61
Summation	20.71	80.75	80.35
Element-wise Product	20.71	80.81	78.31

4.6. Main Results

We evaluate XFT under both pretraining and non-pretraining settings across three standard benchmarks: VQA2.0, SNLI-VE, and GQA. Our evaluation focuses on accuracy as the primary metric, while also reporting computational efficiency in terms of GFlops. The fusion methods compared include our full CrossFusionTokens (XFT), its reduced variant (TAQ), and the widely used merged attention baseline.

The results clearly demonstrate the superiority of XFT across all datasets. With vision-language pretraining, XFT achieves a VQA score of 57.51%, surpassing the merged attention baseline by more than 4.1 percentage points. On SNLI-VE, XFT obtains an accuracy of 81.49%, showing a 0.24% improvement over merged attention. For GQA, the improvement is even more substantial, with XFT reaching 80.45% versus 78.25% for the baseline.

Even the lighter TAQ variant of XFT—where only the text tokens are used as queries—still outperforms merged attention on SNLI-VE and GQA while using fewer floating-point operations. This confirms that even a partial instantiation of our cross-modal alignment strategy improves over conventional concatenation-based fusion.

These results validate that using cross-attention followed by channel-level integration is an effective and scalable strategy for joint visual-language representation learning. Notably, these gains are achieved with only a modest increase in FLOPs, confirming the practical efficiency of our design.

Table 2. Performance comparison with vision-language pretraining. XFT outperforms baseline methods.

Fusion Method	GFlops	VQA	SNLI-VE	GQA
Merged Attention	19.31	53.33	81.25	78.25
XFT (Ours)	19.87	57.51	81.49	80.45
XFT (TAQ variant)	17.34	53.23	81.21	77.74

Table 2. Performance comparison with vision-language pretraining. XFT outperforms baseline methods.

Fusion Method	GFlops	VQA	SNLI-VE	GQA
Merged Attention	19.31	53.33	81.25	78.25
XFT (Ours)	19.87	57.51	81.49	80.45
XFT (TAQ variant)	17.34	53.23	81.21	77.74

4.7. Multimodal Encoder with Transformer Blocks

To explore whether the benefits of XFT persist in deeper architectures, we integrate our fusion method with a multimodal transformer encoder comprising multiple self-attention layers. We compare against strong alternatives: Co-Attention [16], which uses distinct streams for each modality, and Co-Tokenization [44], a more elaborate method that iteratively selects and refines visual tokens using a TokenLearner module.

As shown in the results, XFT continues to perform strongly even in this competitive setup. Our model achieves 80.52% on SNLI-VE and 78.21% on GQA with only 32.90 GFlops and 326M parameters. Although Co-Tokenization yields slightly better scores, it requires significantly more computation—over 57 GFlops—and additional parameters due to multiple rounds of token refinement.

Compared to Co-Attention, XFT achieves higher accuracy on both SNLI-VE (+0.32%) and GQA (+0.46%), while using fewer self-attention blocks (10 vs. 12) and fewer overall parameters. These findings emphasize that our fusion strategy can serve as a plug-in module in both shallow and deep transformer architectures, maintaining robustness and performance efficiency.

Table 3. Fusion methods with 12-layer transformer encoder. XFT is more efficient and competitive with state-of-the-art.

Method	Layers	Params	RES	GFlops	SNLI-VE	GQA
Merged Attention	12	333M	${384}^2$	34.89	79.81	78.07
Co-Attention	12	361M	${384}^2$	29.61	80.20	77.75
Co-Tokenization	12	392M	${384}^2$	57.78	80.79	81.07
XFT (Ours)	10	326M	${384}^2$	32.90	80.52	78.21

Table 3. Fusion methods with 12-layer transformer encoder. XFT is more efficient and competitive with state-of-the-art.

Method	Layers	Params	RES	GFlops	SNLI-VE	GQA
Merged Attention	12	333M	${384}^2$	34.89	79.81	78.07
Co-Attention	12	361M	${384}^2$	29.61	80.20	77.75
Co-Tokenization	12	392M	${384}^2$	57.78	80.79	81.07
XFT (Ours)	10	326M	${384}^2$	32.90	80.52	78.21

4.8. Encoder-Only for VQA Classification

To better understand the relatively lower performance of generative models on VQA, we investigate the impact of decoder design by comparing XFT with and without the decoder. In the encoder-only configuration, the decoder is replaced by a classification head trained to predict the correct answer from a predefined set of 3,130 VQA categories.

Results show a dramatic performance gain: the encoder-only variant achieves 70.39% accuracy, significantly outperforming the encoder-decoder counterpart, which only reaches 58.14%. This 12.25-point gap highlights the challenge of learning a robust decoder for VQA-style short-answer generation, especially when constrained to limited output lengths.

The improvement indicates that much of the performance bottleneck in the generative setting stems not from the fused representation but from the decoder’s capacity to generate discrete labels accurately. Consequently, we use the encoder-only variant in subsequent comparisons with state-of-the-art methods to ensure fair benchmarking.

Table 4. Encoder vs. encoder-decoder performance on VQA.

Fusion Method	Architecture	GFlops	VQA Acc.
XFT (Ours)	Encoder-Decoder	35.50	58.14
XFT (Ours)	Encoder Only	31.86	70.66

Table 4. Encoder vs. encoder-decoder performance on VQA.

Fusion Method	Architecture	GFlops	VQA Acc.
XFT (Ours)	Encoder-Decoder	35.50	58.14
XFT (Ours)	Encoder Only	31.86	70.66

4.9. Comparison with State-of-the-Art

We position XFT alongside a suite of competitive baselines in the literature, including METER [16], ALBEF [29], CFR [40], and the large-scale SimVLM [54]. These models vary significantly in terms of architecture size, pretraining corpus, and computational footprint.

Our XFT model contains 340M parameters, making it comparable to METER and smaller than ALBEF and BLIP. It is pretrained on a moderate mix of COCO and CC3M, unlike SimVLM which leverages a proprietary dataset exceeding 1.5B samples.

Despite these disparities, XFT sets a new bar on SNLI-VE and GQA with scores of 82.87% and 82.43%, respectively—outperforming all publicly available baselines. On VQA, although we do not reach the topmost score, our model still achieves 70.62% using an encoder-only setting with minimal computation.

The overall results highlight that XFT delivers state-of-the-art or highly competitive accuracy on vision-language tasks while using fewer computational resources. This balance of performance and efficiency makes XFT particularly well-suited for real-world deployment where both accuracy and scalability are essential.

Table 5. Comparison with recent SOTA models. SimVLM is trained on private 1.5B dataset.

Method	Params	GFlops	VQA	SNLI-VE	GQA
SimVLMHuge [54]	1.5B	890	80.34	86.32	-
METER [16]	336M	130	77.68	80.61	-
ALBEF [29]	418M	122	75.84	80.91	-
CFR [40]	-	-	69.80	-	73.60
XFT (Ours)	340M	36	70.62	82.64	81.33

Table 5. Comparison with recent SOTA models. SimVLM is trained on private 1.5B dataset.

Method	Params	GFlops	VQA	SNLI-VE	GQA
SimVLMHuge [54]	1.5B	890	80.34	86.32	-
METER [16]	336M	130	77.68	80.61	-
ALBEF [29]	418M	122	75.84	80.91	-
CFR [40]	-	-	69.80	-	73.60
XFT (Ours)	340M	36	70.62	82.64	81.33

5. Conclusions and Future Directions

In this paper, we present CrossFusionTokens (XFT), a novel and effective fusion mechanism designed to enhance the representational alignment and interaction between visual and textual modalities in multimodal learning systems. By leveraging cross-attention to retrieve contextually rich representations and channel-wise concatenation to integrate these features, our approach addresses long-standing limitations in traditional fusion methods such as simple token concatenation or dual-stream co-attention.

Our extensive empirical analysis shows that XFT consistently outperforms baseline fusion strategies—including merged attention and co-attention—on multiple vision-language benchmarks. Across SNLI-VE, GQA, and VQA2.0, XFT demonstrates not only higher accuracy but also superior computational efficiency. For instance, on SNLI-VE, our method outpaces ALBEF and METER by a notable margin of nearly 2 percentage points, and achieves over 8 points of improvement on GQA compared to other leading approaches. This robust performance holds across various evaluation setups, including with and without vision-language pretraining, different image resolutions, and encoder architectures.

Beyond task-specific gains, XFT also proves to be versatile and adaptable. Its design permits seamless integration into both encoder-decoder and encoder-only model paradigms. We find that encoder-only variants of XFT deliver significantly stronger performance on VQA, offering practical implications for tasks requiring classification from predefined answer spaces.

Given its lightweight nature and strong generalization capabilities, we believe XFT provides a promising foundation for future multimodal systems. Although our experiments focus on vision-and-language tasks, the modular structure of XFT readily supports extension to additional modalities such as audio and video. Future research may explore this multimodal generalization and apply XFT to broader tasks like video question answering or audio-visual grounding.

Additionally, scaling XFT to larger model capacities and incorporating more diverse training corpora—such as web-scale noisy datasets or multimodal knowledge bases—may further enhance its applicability and performance. We also plan to evaluate XFT under few-shot and zero-shot learning settings, where robust multimodal reasoning is often most challenging.

In summary, XFT offers an efficient, effective, and extensible alternative to traditional fusion strategies in multimodal learning. We hope this work encourages the community to rethink the architecture of multimodal representation learning and inspires new innovations beyond simple token merging or rigid stream separation.