Language-Driven Image Restoration and Semantic-Aware Quality Assessment: A Survey

1. Introduction

Image restoration (IR), as a fundamental problem in low-level computer vision, aims to recover high-quality images from degraded counterparts. Over the past decades, numerous IR methods have been extensively studied for a wide range of tasks, including denoising [1,2,3], deraining [4,5,6], dehazing [7,8,9], desnowing [10,11,12], deblurring [13,14,15], low-light enhancement (LLIE) [16,17,18], super-resolution [19,20,21]. Beyond these tasks, IR techniques have also been widely applied to domain-specific scenarios, such as underwater enhancement [22,23,24], medical IR [25,26].

IR methodologies have evolved from model-driven approaches based on handcrafted priors to data-driven paradigms powered by deep neural networks, including CNNs [27], Transformers [28], and more recent architectures [29]. In parallel, restoration frameworks have progressed from task-specific designs, where each degradation type is modeled independently [5,9,30], to unified frameworks such as all-in-one (AiO) restoration, which aim to handle multiple degradation types within a single model [31,32,33,34].

Despite these advances, existing methods remain largely constrained by predefined degradation assumptions and are typically optimized with pixel-level supervision, limiting their ability to generalize to complex or user-dependent restoration scenarios. These limitations motivate the exploration of new paradigms beyond conventional frameworks.

Recent breakthroughs in foundation models (FMs), such as vision–language models (VLMs) and multimodal large language models (MLLMs), have opened new opportunities to address these challenges. By encoding rich semantic priors and exhibiting strong reasoning capabilities beyond conventional visual representations, VLMs and MLLMs have been increasingly introduced into IR pipelines [34,35,36,37]. Rather than directly performing pixel-level reconstruction, these models are typically employed as auxiliary components to provide high-level information, such as degradation interpretation, semantic guidance, and adaptive control. This emerging paradigm fundamentally reshapes IR from a purely visual mapping problem into a multimodal, semantically informed, and interactive framework.

This paradigm shift introduces a critical challenge in image quality assessment (IQA). Traditional IQA metrics [38,39,40] are effective for measuring pixel-level fidelity or perceptual similarity. However, they are not designed to assess semantic alignment or instruction consistency, leading to a mismatch between the restoration targets and the evaluation criteria. Recent language-driven IQA methods [41,42,43,44,45,46,47] address this gap by leveraging multimodal representations to assess semantic coherence and cross-modal consistency.

Although integrating language for IR and IQA has developed rapidly [34,37,48], a comprehensive survey of this emerging field remains absent, to the best of our knowledge. Existing surveys on IR primarily focus on architectural designs [49] or task-specific learning strategies [4,31,50], while recent reviews on multimodal models [51,52] seldom address low-level vision problems in depth.

In this survey, we provide the first systematic review of language-driven image restoration and language-driven IQA. Figure 1 summarizes the taxonomy of this survey in a hierarchically structured way. We review advances in language-driven restoration frameworks, summarize representative model designs and training strategies, and discuss open challenges and potential future research directions.

The main contributions of this survey are summarized as follows:

• We introduce a unified conceptual framework that interprets language-driven IR as an interaction-centric paradigm, revealing how language models affect restoration behavior beyond architectural modifications, including feature-level, optimization-level, and execution-level coupling.
• We analyze language-driven IQA, highlighting its conceptual distinctions from conventional fidelity metrics and clarifying the challenges of evaluation reliability, calibration stability, and semantic bias.
• We summarize the restoration datasets used in language-driven frameworks and analyze their limitations and emerging requirements from a language-driven perspective, emphasizing the need for semantically enriched, language-aware benchmarks. We also provide comparisons between conventional frameworks and language-driven methods across different settings.
• We investigate open challenges posed by language-integrated restoration systems and outline promising directions for future research that bridge multimodal reasoning, visual perception, and restoration optimization.

The rest of the work is organized as follows. Section 2 introduces the necessary preliminaries, including IR fundamentals, multimodal language models, and IQA, providing an overview of the core concepts relevant to this work. Section 3 reviews representative language-driven IR approaches, and the analysis is guided by the proposed interaction-centric taxonomy with a focus on their model architectures and key technical innovations. Section 4 summarizes commonly used datasets for different IR tasks and discusses corresponding evaluation metrics, covering both traditional criteria and recent language-driven assessment methods. Finally, Section 5 analyzes open challenges in current studies and outlines potential solutions and future research directions.

2. Background

In this section, we introduce the fundamental concepts and taxonomy of language-driven image restoration, aiming to provide a structured background for understanding recent language-driven restoration frameworks.

2.1. Image Restoration

Image restoration aims to recover a clean image $\mathcal{I}\left(x\right)$ from its degraded observation $\mathcal{D}\left(x\right)$ by mitigating various types of distortions:

$$\mathcal{D}\left(x\right)=H\left(\mathcal{I}\right(x\left)\right)+\mathcal{N},$$

(1)

where $H(·)$ denotes the degradation operator and $\mathcal{N}$ represents additive noise.

Figure 2. Examples of different degradation types and domains. The images correspond to the following open-source datasets: (a) PolyU [53], (b) LHP [54], (c) NH-HAZE [55], (d) RealSnow10K [56], (e) DPDblur [14], (f) LSRW [57], (g) DIV4K50 [58], (h) Squid [59], (i) FastMRI [60], and (j) MiO100 [61].

Figure 2. Examples of different degradation types and domains. The images correspond to the following open-source datasets: (a) PolyU [53], (b) LHP [54], (c) NH-HAZE [55], (d) RealSnow10K [56], (e) DPDblur [14], (f) LSRW [57], (g) DIV4K50 [58], (h) Squid [59], (i) FastMRI [60], and (j) MiO100 [61].

Existing IR methods can be broadly categorized into task-specific models and unified frameworks. Task-specific IR methods [8,17] are trained to handle a single degradation type. While such models often achieve strong performance within their targeted tasks, their specialization inherently restricts generalization to unseen or mixed degradations. This limitation motivates the development of general IR frameworks [62,63], which employ unified architectures capable of addressing multiple restoration tasks. Nevertheless, these approaches typically still rely on task-wise training or fine-tuning procedures. More recently, AiO restoration frameworks [32,33,64,65] have attracted increasing attention. By enabling multiple restoration tasks within a single model without explicit retraining, AiO methods aim to further improve generalization. This is often achieved through degradation-aware representations, prompt-based mechanisms, or multi-branch architectures that explicitly distinguish different degradation types.

Despite these advances, conventional deep learning–based restoration paradigms remain largely constrained by predefined task formulations and limited adaptability to diverse restoration requirements. To address these limitations, recent studies have started to explore the use of language models for IR [34,36,66,67,68]. Leveraging their strong semantic understanding, reasoning capability, and cross-modal alignment, language-driven approaches enable more flexible and interactive restoration pipelines, such as instruction-guided or context-aware restoration. This emerging paradigm marks a shift from task-centric restoration models toward more general, adaptive, and semantically informed IR frameworks.

2.2. Foundation Models

Recent studies on FMs encompass multiple closely related model families, including large language models (LLMs), VLMs, and MLLMs. These models differ in their input modalities and output objectives, yet collectively serve as the primary computational backbone of language-driven IR methods reviewed in this survey. While capability boundaries between VLMs and MLLMs continue to develop, we adopt a functional distinction based on their dominant usage patterns in restoration frameworks.

Large Language Models. LLMs are FMs trained on large-scale text datasets and operate exclusively on language inputs and outputs. Representative models include GPT [69], LLaMA [70], Qwen [71], and DeepSeek [72]. These models have achieved remarkable progress in natural language understanding, reasoning, and generation, demonstrating strong generalization across a wide range of language-centric tasks. LLMs primarily operate on textual inputs and outputs and do not natively process raw visual data.

Vision-Language Models. In a broad sense, VLMs refer to models that jointly process visual and textual signals for cross-modal understanding. This includes both contrastive representation learning frameworks (e.g., CLIP-like [73,74] and ALIGN-like [75]) and more recent architectures that integrate visual encoders with language modeling components. VLMs primarily enable tasks such as semantic grounding, cross-modal retrieval, and image understanding. In language-driven IR, VLMs often serve as semantic extractors or conditioning providers that bridge visual content and high-level language descriptions.

Multimodal Large Language Models. MLLMs extend LLMs by incorporating modality-specific perception modules, allowing them to process visual and other non-textual inputs (e.g., audio and video) while maintaining a language-centric interface. Representative models include GPT-4o [76], Gemini [77], and Qwen-VL [78,79]. In restoration pipelines, MLLMs are particularly suitable for tasks that require high-level reasoning, such as degradation analysis, restoration planning, and adaptive strategy selection.

In this survey, we emphasize functional roles of VLMs and MLLMs in language-driven IR systems. When referring to methods that leverage language as a high-level interaction signal, regardless of the specific model architecture, we adopt the umbrella term language-driven image restoration.

2.3. Image Quality Assessment

IQA plays a critical role in image processing by enabling the evaluation and optimization of visual content quality [80]. Although subjective assessment based on the human visual system (HVS) is generally regarded as the most reliable criterion, it is costly and time-consuming, motivating extensive research into objective metrics.

Depending on the availability of reference images, IQA methods are broadly categorized into full-reference (FR-IQA) and no-reference (NR-IQA). FR-IQA methods compare distorted images with their high-quality references using metrics such as PSNR [39], SSIM [38], and MSE, as well as learning-based perceptual metrics [40,81,82,83,84]. In contrast, NR-IQA predicts image quality directly from distorted inputs without reference images. Early methods rely on handcrafted natural scene statistics, such as BRISQUE [85], NIQE [86], and PIQE [87], while recent approaches adopt deep learning to model complex perceptual patterns [88,89,90,91,92].

Most recently, language-driven IQA methods [41,42,45,46,47,93,94,95] have emerged as a new paradigm. By leveraging multimodal reasoning capabilities and foundational knowledge, these approaches aim to narrow the gap between objective metrics and subjective human assessment. Some methods evaluate image quality using natural languages [95,96,97], while others directly produce quantitative quality scores [45,94,98].

2.4. Relevant Surveys

A number of surveys have reviewed IR from different perspectives. Many of these works organize the literature according to specific degradation types or application domains [1,4,7,13,16,19,22,31,50]. Along the recent trend of AiO restoration, Jiang et al. [31] provided a systematic overview of AiO restoration frameworks. Beyond task-oriented surveys, several studies have examined IR from a model-centric perspective. For example, Su et al. [49] reviewed deep learning architectures widely adopted in IR, while Li et al. [102] offered an in-depth review of diffusion-based IR methods. Despite these efforts, the rapid emergence of language-driven IR methodologies has not yet been systematically reviewed. In contrast, in this work, we present a comprehensive review of language-driven IR from the following three aspects: 1) recent advances in language-driven IR methods, 2) datasets and language-driven evaluation protocols, and 3) benchmarking and comparative evaluation of language-driven approaches. The comparison between previous surveys and our work is summarized in Table 1.

3. Methodology

This section first categorizes language-driven method prototypes and then analyzes each category in detail through representative approaches. Figure 3 summarizes the representative paradigms of existing language-driven IR methods.

3.1. Overview of Language-Driven Restoration and Taxonomy Definition

Recent advances in FMs have significantly influenced the design of IR systems. Rather than solely improving restoration architectures, an increasing number of approaches leverage language-driven semantic priors, cross-modal reasoning, and high-level decision-making to enhance restoration robustness, flexibility, and controllability. As a result, clarifying the roles of language models and how they interact with restoration pipelines becomes critical for understanding existing frameworks and guiding future research.

• Feature-Level Coupling: FM outputs are injected into the forward process to modulate intermediate representations without altering the optimization objective or execution structure. This includes pretrained feature conditioning, degradation-aware conditioning, semantic conditioning, and global context conditioning.
• Optimization-Level Coupling: FM outputs define or reshape the optimization objective by introducing differentiable loss terms or scalar reward functions, thereby altering optimization dynamics.
• Execution-Level Coupling: FM outputs determine the execution logic of the restoration pipeline by generating high-level plans or control signals, enabling dynamic selection, composition, or scheduling of restoration modules beyond a fixed computational graph.

Existing surveys on IR mainly organize the literature from architecture-driven (e.g., CNN or transformer) [49] or task-driven [4,31] perspectives. From the perspective of model architectures, existing approaches involve different types of foundation models, including VLMs and MLLMs. However, these categorizations become insufficient for language-driven restoration systems, where the primary methodological distinctions arise from interaction mechanisms between language models and restoration networks. Specifically, language-driven restoration systems reshape restoration behavior by modifying information flow structures, supervision signals, control mechanisms, and optimization objectives. Therefore, to provide a structured understanding of this rapidly growing body of work, we adopt an interaction-centric taxonomy that categorizes methods according to the functional role of language models within restoration pipelines, while model families are treated as an orthogonal dimension. The interaction-centric taxonomy is described below:

Based on this interface formulation, we distinguish different paradigms according to the dominant interaction type as follows:

In practice, certain tendencies can be observed: VLMs are often used for representation-level conditioning, whereas MLLMs are more frequently involved in higher-level decision-making, such as planning and control. Nevertheless, these associations are not strict, and the same interaction paradigm can be realized with different types of foundation models. To maintain clarity, each method is categorized according to its primary functional influence, although practical systems may exhibit hybrid characteristics across multiple interaction levels.

Figure 4 illustrates the evolution of representative language-driven IR approaches from 2023 to Jan. 2026. In the following subsections, we review each interaction category in detail, summarizing representative methods, core design strategies, as well as their respective strengths and limitations.

3.2. Feature-Level Coupling

Representation Reuse. Representation reuse directly integrates representations from pretrained FMs into restoration backbones, typically by replacing the visual encoding stage. In this paradigm, the pretrained model acts as a fixed feature extractor that provides robust and semantically aligned priors to guide downstream reconstruction. A representative example is CLIP-based feature reuse. During CLIP pretraining, visual representations are optimized to align with text semantics that inherently lack high-frequency details. As a result, these representations tend to overlook noise and are less affected by degradation perturbations, providing more robust guidance for restoration.

For instance, CLIPDenoising [103] incorporates a frozen CLIP [104] visual encoder together with a learnable denoising decoder. By leveraging the distortion-invariant and content-aware properties of CLIP’s dense visual features, the method demonstrates promising out-of-distribution generalization under unseen conditions.

Degradation-Aware Conditioning. Degradation-aware conditioning introduces degradation-specific signals, such as type, severity, or mixture, derived from FMs into the restoration pipeline to guide feature modulation. These signals are projected as conditioning inputs to adapt the restoration process to diverse and potentially unseen degradations.

LDP [105] utilizes text-aligned embeddings to describe blur attributes, allowing restoration models to adjust internal feature representations. GPP-LLIE [106] introduces a generative perceptual prior by extracting high-level attributes from LLaVA [44]. In AiO multi-degradation restoration, UniProcessor [107], VLU-Net [67], and AllRestorer [108] exploit textual priors to identify and handle mixed degradations. Furthermore, InstructIR [34] employs natural-language instructions to unify multiple restoration tasks into a single model. DiffRes [109] leverages BLIP-2 [110] to encode degradation information through feature difference instructions. RectiWeather [111] models degradation conditions by estimating weather-related attributes from images and uses these signals to modulate the restoration process. In hyperspectral restoration, MP-HSIR [112] integrates spectral prompts with language–visual prompts, where spectral representations provide low-rank priors and language signals encode degradation characteristics. PromptHSI [113] further represents degradation factors through prompts to support a universal hyperspectral restoration model.

Beyond feed-forward architectures, degradation-aware conditioning has also been widely adopted in diffusion-based frameworks. DA-CLIP [114] explores controllable adaptation of VLM representations for multi-task restoration. Methods such as AutoDIR [115], SPIRE [116], and MPerceiver [117] encode degradation descriptors into language-aligned embeddings for diffusion conditioning. VLM-SSR [118] introduces CLIP-derived features to guide real-world super-resolution.

Semantic Conditioning. Semantic conditioning incorporates high-level semantic signals generated from FMs into the restoration process to guide content-aware feature modulation. Unlike degradation-aware conditioning, which focuses on characterizing degradation attributes, this paradigm emphasizes the extraction and utilization of semantic information, such as structural layouts, textures, and scene descriptions, to improve perceptual fidelity and detail reconstruction.

CLIP-SR [119] couples linguistic guidance with image feature processing to refine structure and textures for super-resolution. VLM-IMI [120] further explores instruction-based guidance by generating textural descriptions from the input images and user prompts. Additionally, TPGDiff [121] extends this idea by incorporating VLM-generated degradation priors into a hierarchical multi-prior formulation, constraining the diffusion trajectory at multiple stages. Semantic conditioning has also been explored in more structured learning frameworks. LLMRA [68] employs an MLLM [122] to perform a high-level description of image content, which is then mapped into a language-aligned embedding space via a pretrained CLIP text encoder to guide restoration. PromptFix [123] utilizes a combination of LLaVA [44] together with a pretrained CLIP visual encoder to extract semantic cues, which are used as auxiliary prompts to guide an instruction-driven diffusion restoration model. While both LLMRA and PromptFix adopt similar language-guided pipelines, they differ in the role of semantic signals. In LLMRA, language-derived representations are integral to the restoration process, whereas in PromptFix, they act as an auxiliary enhancement that complements a generative prior.

Global Context Conditioning. Global context conditioning introduces high-level signals that summarize the overall state of the input and provide holistic guidance for restoration. These signals are derived from aggregated information, such as degradation categories, scene-level descriptions, or perceptual quality assessments. In practice, these global signals are encoded into embeddings and fused into the restoration network, where they act as conditioning features that guide the recovery process.

ClearAIR [66] exemplifies this paradigm by integrating multiple language-driven modules to construct complementary global guidance signals. A VLM-based task identifier [114] encodes degradation categories into prompt-like embeddings, while an MLLM-based IQA module [45] produces perceptual quality scores that are further transformed into conditioning features. Together with semantic guidance modules, these signals are injected into the restoration backbone to refine features.

3.3. Optimization-Level Coupling

Loss-Driven Optimization. Loss-driven optimization incorporates language-derived signals into the training objective by formulating them as auxiliary loss terms. In this paradigm, FMs provide high-level perceptual or semantic supervision, complementing conventional reconstruction losses and optimizing the restoration process.

In structured medical imaging, LEDA [124] enforces consistency between perceptual representations and semantic tokens through a language-informed codebook, while LangMamba [125] integrates language-aligned features into a Mamba-based backbone for efficient CT denoising. In adversarial settings, methods such as NeRCo [126] and SeD [127] condition discriminators on VLM-derived representations, introducing semantic-aware discriminative signals that complement adversarial objectives. This paradigm is particularly effective in scenarios where paired ground-truth data are limited or unavailable. For instance, in underwater image enhancement [128], a FLIP [129]-pretrained text encoder encodes high-level quality concepts via textual prompts, and the resulting similarity scores are incorporated as perceptual loss terms alongside reconstruction objectives. Furthermore, Perceive-IR [130] enables fine-grained quality control through CLIP-aligned prompt learning and difficulty-adaptive supervision.

Despite these advantages, the supervision provided by FMs is often global, which may limit sensitivity to localized artifacts and make performance dependent on prompt design and model expressiveness. Closely related but conceptually distinct from loss-driven optimization, some recent works employ MLLMs as perceptual evaluators to generate pseudo-label supervision for restoration training, rather than directly incorporating language-derived signals into the optimization objective. For example, SnowMaster [56] leverages MLLM-derived perceptual preference feedback to rank candidate desnowing results and uses the selected outputs as pseudo-labels to support semi-supervised training on real-world snowy images.

Reward-Driven Optimization. Reward-driven optimization treats foundation models as perceptual evaluators that provide feedback signals to guide restoration behavior. Rather than defining explicit reconstruction losses, this paradigm leverages reward signals, scoring mechanisms, or evaluator-based feedback to shape optimization or decision policies, shifting the objective from pixel-level fidelity toward perceptual quality.

MOERL [131] formulates image restoration as a reinforcement learning problem, where a Mixture-of-Experts (MoE) model is optimized using perceptual rewards to adaptively handle complex and mixed degradations without requiring explicit labels. In contrast, SimpleCall [132] employs a lightweight agent that sequentially invokes restoration tools based on MLLM-derived feedback, enabling policy learning in a label-free setting without relying on ground-truth supervision.

3.4. Execution-Level Coupling

Planning-Based Execution. Planning-based execution assigns foundation models the role of high-level controllers that organize the restoration process. Instead of providing auxiliary conditioning signals, FMs structure restoration as a sequential decision procedure, where degradations are identified, tasks are decomposed, and restoration operations are scheduled. In this setting, FMs orchestrate the execution logic, while task-specific restoration networks are responsible for pixel-level reconstruction.

Early works such as RestoreAgent [133] and AgenticIR [37] demonstrate this paradigm by enabling MLLMs to determine restoration tasks and execution sequences. Subsequent studies improve planning stability and efficiency. Q-Agent [134] employs chain-of-thought reasoning [135] to decompose multi-degradation perception and adopts quality-driven greedy planning based on NR-IQA, effectively mitigating unnecessary rollbacks. To enhance scalability and robustness, later works extend this paradigm to multi-agent systems. MAIR [136] introduces a scheduler–expert hierarchy that separates degradation perception from restoration execution, while HybridAgent [137] employs collaborative agents to balance planning accuracy and computational cost. Specialized frameworks such as 4KAgent [58] adapt planning-based execution to ultra-high-definition restoration scenarios. System-level approaches, including Clarity ChatGPT [138] and JarvisIR [36], further emphasize interactive refinement and robustness in real-world settings.

4. Experiments

In this section, we summarize representative datasets widely used in language-driven IR frameworks across different tasks. We then review commonly adopted IQA metrics, covering both classical and recent language-driven evaluation paradigms. Together, these datasets and metrics provide a comprehensive view of current benchmarks and evaluation practices for language-driven IR.

4.1. Datasets

Existing language-driven IR studies commonly adopt subsets of established restoration benchmarks, as summarized in Table 2 and Table 3. These datasets vary substantially in scale, spatial resolution, scene diversity, and data-acquisition protocols, spanning both real-world captures and synthetically generated degradations. While synthetic datasets offer controllability and scalability, they may inadequately capture the ambiguity, complexity, and stochastic characteristics of real-world degradations. This discrepancy becomes particularly critical in language-driven frameworks, where degraded understanding relies on semantic representations that are inherently sensitive to contextual variations.

On the other hand, recent language-driven restoration frameworks increasingly require datasets that explicitly model image-language interactions. However, as shown in Figure 5, most datasets consist primarily of degraded-clean image pairs while rarely providing structured linguistic annotations describing degradation characteristics or perceptual attributes. The absence of structured language annotations may introduce inconsistencies when training or evaluating language-driven restoration models. To bridge this gap, PromptFix [123] constructs a large-scale instruction-following dataset that contains approximately 1.01 million input-goal-instruction triplets across diverse low-level tasks. Furthermore, in safety-critical applications such as autonomous driving, CleanBench [36] defines an instruction sample as a triplet, consisting of a user instruction, a degraded image, and a response. Such datasets reflect a paradigm shift in which restoration is formulated not only as pixel reconstruction but also as a language-conditioned generation problem. Despite these initial explorations, restoration datasets with captions are still underexplored.

4.2. Evaluation Metrics

Existing IQA methods and evaluation protocols are summarized in Table 4 and Table 5, covering both conventional and language-driven assessment paradigms.

Conventional IQA metrics. Image restoration performance is traditionally evaluated using IQA metrics that measure fidelity or perceptual similarity between restored images and references. Full-reference IQA metrics, such as PSNR [39], SSIM [38], FSIM [188], and MAE, quantify pixel-level or structural consistency with ground-truth images. Learning-based metrics (e.g., LPIPS [40], DISTS [81], CKDN [82], and AHIQ [83]) assess perceptual similarity in deep feature spaces and exhibit improved correlation with human judgments. When reference images are unavailable, no-reference IQA metrics are commonly adopted, ranging from hand-crafted statistical measures (e.g., BRISQUE [85], NIQE [86], and PIQE [87]) to learning-based models such as MUSIQ [88] and MANIQA [90]. In addition, distribution-based metrics like FID [190] are often used to evaluate feature distribution alignment between restored and real images, particularly in generative restoration scenarios.

Compared with conventional IQA metrics, recent studies have explored language-driven evaluation paradigms that leverage VLMs and MLLMs. These approaches introduce language as a new modality for modeling perceptual quality, enabling assessment through various mechanisms. Based on the functional role of language, existing methods can be broadly categorized into alignment-based, reasoning-based, and scoring-based IQA.

Alignment-Based IQA. This type of IQA maps images into a language-informed representation space. In this paradigm, textual descriptions, prompts, or attribute embeddings serve as anchors to encode quality semantics, and image quality is inferred through similarity, ranking, or conditional modeling within this shared space.

Early studies, such as CLIP-IQA [41], demonstrate that quality perception can be formulated as a prompt-driven similarity-comparison problem using antonym prompt pairs (e.g., good vs. bad). Subsequent works extend this principle in various directions. QualiClip [43] proposes a self-supervised, opinion-unaware approach via quality-aware image–text ranking to learn distortion-sensitive representations. PromptIQA [196] further enables requirement-adaptive assessment by incorporating a small set of image–score pairs as prompts. In addition, GRMP-IQA [197] improves data efficiency through meta-prompt learning with gradient regularization, and ATTIQA [198] addresses annotation scarcity through attribute-aware pretraining using pseudo-labels generated by VLMs.

Beyond prompt-based modeling, LIQE [194] establishes a unified vision–language correspondence framework to jointly model semantics, distortions, and quality, while UniQA [201] proposes a multimodal pretraining strategy bridging quality assessment and aesthetic evaluation. SFD [200] further explores semantic feature discrimination as a proxy for quality estimation. In domain-specific settings, SCUIA [195] incorporates semantic-aware contrastive learning for underwater IQA. CAP-IQA [199] integrates text priors with image-specific context for task-aware scoring. Finally, RALI [202] aligns images with quality-aware textual representations distilled from RL-based IQA models, and predicts image quality by similarity matching in the learned text space, thereby preserving generalization without requiring LLM reasoning during inference.

Reasoning-Based IQA. Reasoning-based IQA methods formulate quality assessment as an inference process, where perceptual quality is derived through language-driven analysis, explanation, or decision-making. Instead of directly predicting scores, these methods leverage MLLMs to perform structured reasoning, such as descriptive assessment, comparative judgment, or region-level grounding, to approximate human-like evaluation.

Representative frameworks such as DepictQA [95] and DepictQA-Wild [203] formulate IQA as descriptive and comparative language-based reasoning, enabling human-like assessment. IQAGPT [204] adopts a caption-driven pipeline to jointly produce quality scores and natural-language explanations. Beyond global reasoning, SEAGULL [96] introduces region-aware quality reasoning via SAM-guided feature modeling, while Q-Ground [97] enables fine-grained visual grounding for quality analysis. Co-Instruct [205] further explores open-ended comparative reasoning, allowing MLLMs to generate detailed quality comparisons. Moreover, AgenticIQA [48] advances this paradigm by modeling IQA as a structured decision-making process with planning, execution, and aggregation stages, moving beyond single-pass inference toward agent-based evaluation.

Scoring-Based IQA. Scoring-based IQA methods aim to predict scalar quality scores through direct or learned mappings from images to quality ratings, often enhanced by preference learning, distribution modeling, or distillation strategies.

In particular, Q-Align [46] discretizes subjective scores to a one-hot label to emulate the human judgment process. In contrast, DeQA-Score [45] improves quality prediction by modeling score distributions as soft labels, and Q-Scorer [98] explicitly adapts multimodal representations for direct scalar quality prediction. In addition, Dog-IQA [206] introduces a standard-guided discrete scoring mechanism combined with mix-grained global–local quality aggregation. To address label efficiency, LEAF [210] decouples perceptual knowledge from MOS calibration via distillation.

On the other hand, the comparison-derived scoring framework Compare2Score [94] derives continuous scores from adaptive pairwise comparisons. Q-Insight [207] integrates reinforcement learning with preference modeling, while Q-Ponder [208] introduces explicit joint optimization objectives for both scoring and reasoning. Q-Hawkeye [209] further improves robustness through uncertainty-aware policy optimization.

Resources and Benchmarks. In addition to standalone IQA methods, recent efforts have focused on constructing benchmarks and instruction datasets to evaluate and enhance the perceptual reasoning capabilities of multimodal models. These resources provide standardized evaluation protocols and training signals for language-driven IQA.

For instance, Q-Instruct [93] and Q-Bench [211,212] focus on improving and evaluating low-level perceptual reasoning capabilities of FMs, underscoring the importance of instruction tuning and benchmarking.

Comparison: Conventional vs. Language-Driven Evaluation. Conventional IQA metrics and language-driven evaluation reflect different yet complementary assessment paradigms. Classical full-reference measures (e.g., PSNR, SSIM) quantify pixel fidelity and structural consistency, providing deterministic and reproducible criteria. These metrics remain essential for benchmarking reconstruction accuracy and optimization stability.

By contrast, language-driven evaluation estimates quality through multimodal similarity modeling, preference reasoning, and language-conditioned quality interpretation. Instead of directly measuring pixel fidelity, these approaches estimate quality through mechanisms such as prompt-driven image–text alignment, pairwise comparison, distribution-aware score modeling, and reasoning-guided scoring. Such evaluation strategies are particularly relevant for generative restoration, mixed degradations, and no-reference scenarios. Unlike conventional metrics, language-driven approaches may favor semantic plausibility or contextual consistency.

Importantly, language-driven evaluation should not be interpreted as a direct replacement for conventional IQA metrics, but rather as a complementary perceptual assessment mechanism. Conventional metrics ensure numerical stability and comparability, whereas language-driven evaluators capture complementary perceptual and semantic cues. Recent studies [37,116,132] increasingly adopt hybrid evaluation protocols that combine deterministic fidelity measures with language-driven perceptual assessment to achieve more comprehensive performance evaluation.

However, despite the growing interest in language-driven IQA, a unified evaluation benchmark is still lacking. Existing methods are typically evaluated under task-specific settings, with varying datasets and scoring strategies, making direct quantitative comparison across different works difficult. For instance, recent studies [111,132] demonstrate the effectiveness of language-driven evaluation metrics within their customized experimental setups. However, such results are not directly comparable due to differences in evaluation configurations. Overall, current evidence suggests that language-driven metrics provide complementary insights to conventional measures. Nevertheless, establishing standardized benchmarks for language-driven IQA remains an important open problem.

Evaluation Protocols and Criteria. In addition to direct quality prediction metrics, as shown in Table 5, correlation-based criteria such as PLCC, SRCC, and KRCC [213] are widely adopted to measure the consistency between model-generated assessments and human subjective ratings. These statistics do not evaluate image quality directly but instead quantify the reliability of IQA models with respect to human perception. Additionally, text-based metrics [215,216] are often used to evaluate the semantic fidelity of generated descriptions and reference captions, especially in instruction-following or explanation-based restoration frameworks.

Furthermore, several studies adopt task-oriented evaluation paradigms that assess image quality indirectly through downstream performance [97]. In this setting, restored images are treated as inputs to high-level vision systems, such as detection, segmentation, or recognition models, where improvements in task performance serve as proxies for quality enhancement. The underlying assumption is that degradations often impair feature extraction and perception reliability, and thus, higher downstream accuracy may indicate improved visual quality. Nevertheless, this assumption is not universally valid. Sun et al. [219] have shown that perceptual quality and downstream performance are not strictly causally linked. Despite these limitations, task-driven evaluation remains relevant in application-critical domains, including medical imaging and autonomous driving, where restoration quality is ultimately defined by its functional impact on subsequent perception or decision-making processes.

Metrics for Language-Driven IQA Ability. While the above criteria are commonly used to evaluate IQA outputs, they are still insufficient for systematically assessing the intrinsic IQA capabilities of language-driven models. Leveraging the strong semantic understanding capabilities of MLLMs and VLMs for IQA has emerged as a promising and rapidly evolving research direction. 2AFC [220] provides an initial attempt by proposing consistency, accuracy, and correlation metrics to analyze judgment robustness and alignment with human opinion scores. Despite this initial effort, the evaluation of language-driven IQA remains largely underexplored, leaving substantial room for further investigation.

4.3. Experimental Results

To better understand the effectiveness of different design choices, we report experimental results of several language-driven all-in-one restoration frameworks under three commonly used experimental settings. Table 6 presents results on three degradations: deraining, denoising ($\sigma =15,25,50$), and dehazing. Meanwhile, Table 7 extends the evaluation to five degradations by additionally including deblurring and LLIE. Table 8 further reports performance on the single LLIE task. All results are obtained from the corresponding original works and reported in terms of PSNR and SSIM.

Under the three-degradation setting (Table 6), language-driven restoration frameworks exhibit comparable overall performance. For instance, VLU-Net [67] achieves the highest deraining PSNR (38.93 dB), whereas ClearAIR [66] exhibits more consistent cross-task behavior, maintaining stable performance across deraining, denoising, and dehazing. A similar trend is observed in the five-degradation setting (Table 7), where ClearAIR achieves the best average performance compared to other language-driven approaches. In contrast, on the single-task LLIE benchmark (Table 8), task-specific non-language-driven methods such as RetinexFormer [30] (25.16 dB) and RetinexDiff++ [228] (24.67 dB) outperform most language-driven models in terms of PSNR.

These observations suggest that, although language-driven frameworks enhance flexibility and generalization in multi-degradation scenarios, their advantages in specialized tasks remain limited. Moreover, gains in pixel-level fidelity (e.g., PSNR) remain limited. This is because language-driven methods focus more on semantic alignment and user intent than on distortion minimization. This leads to a mismatch between current evaluation protocols and the goals of language-driven restoration. Moreover, language-driven IQA metrics are rarely included in existing benchmarks, highlighting the need for evaluation frameworks that jointly consider fidelity, semantic correctness, and user-oriented restoration quality.

5. Discussion and Open Challenges

Although the integration of VLMs and MLLMs has driven the development of IR frameworks, it has introduced new capabilities while simultaneously giving rise to additional challenges. In this section, we analyze key open challenges in language-driven methods, focusing on generalization, computational efficiency, cross-paradigm trade-offs, evaluation reliability, dataset design, high-dimensional representations, and trustworthiness. We further discuss potential research directions that may help address these challenges and inform future developments in language-driven restoration systems.

5.1. Generalization and Robustness

In real-world scenarios, degradations are often complex, mixed, or poorly defined, making generalization and robustness persistent challenges for IR, even when incorporating semantic priors and degradation-aware guidance via VLMs and MLLMs. Specifically, while VLM- or MLLM-derived representations provide high-level semantic context, they do not fully eliminate sensitivity to degradation variations. Second, language-driven frameworks frequently rely on textual descriptions, prompt formulations, or semantic interpretations to characterize degradation types [68,120,123]. Such dependencies may lead to inconsistent restoration behaviors due to linguistic variability and prompt sensitivity. Moreover, language models themselves may exhibit hallucinations and domain biases inherited from large-scale pretraining, which can affect degradation understanding and guidance reliability.

One potential solution involves prompt-invariant modeling, such as template-based or structured prompts, to reduce instability caused by diverse linguistic formulations. Another direction is uncertainty-aware restoration, where degradation cues are accompanied by confidence estimates to improve robustness against hallucination-induced errors. Despite these emerging strategies, developing principled mechanisms for handling degradation ambiguity, prompt variability, and language-model uncertainty remains an important challenge for future research.

5.2. Computational Efficiency

While language-driven IR frameworks introduce enhanced semantic awareness and perceptual reasoning capabilities, computational efficiency remains a central challenge. The computational burden arises from multiple sources. First, pretrained VLMs and MLLMs typically contain a large number of parameters, making semantic feature extraction and cross-modal reasoning inherently expensive. Additional conditioning and reasoning mechanisms further increase memory consumption. Moreover, agentic paradigms often require multiple reasoning iterations, candidate evaluations, or tool invocation loops, leading to high latency and inference cost.

Recent studies have explored several strategies to mitigate these efficiency bottlenecks. For example, policy optimization [132] and dynamic routing mechanisms [134] have been applied to reduce unnecessary reasoning and model invocation steps. Hybrid designs [137] further improve efficiency by restricting expensive language-driven computation to critical stages. Despite these advances, computational efficiency remains a key challenge for practical deployment. Future research may focus on developing compact multimodal models tailored for restoration tasks as well as reducing redundancy in cross-modal representations through mechanisms such as knowledge distillation or token reduction. In addition, resolution-adaptive architectures present a promising direction, enabling language-driven reasoning to operate at coarse semantic scales while preserving high-resolution reconstruction within lightweight visual backbones.

5.3. Cross-Paradigm Trade-Offs

Existing language-driven approaches are built upon fundamentally different interaction paradigms, leading to inherent trade-offs in system design. A fundamental distinction lies in how VLMs and MLLMs participate in the restoration process, ranging from static guidance to iterative involvement. In static paradigms, language information is incorporated once through fixed embeddings, prompts, or auxiliary objectives, resulting in stable and computationally efficient pipelines. However, such limited involvement restricts adaptability, as the restoration process cannot revise its behavior during inference. In contrast, interactive paradigms repeatedly engage language models during restoration, enabling dynamic reasoning, planning, and execution. This allows better handling of complex or mixed degradations and supports more flexible restoration strategies. Nevertheless, increased interaction also introduces higher computational cost and the risk of error accumulation across multiple steps, potentially leading to unstable outcomes.

These interaction patterns are further reflected in how language influences restoration, either through implicit guidance (e.g., feature modulation or loss design) or explicit decision-making (e.g., planning, tool invocation, or agent-based control). While implicit mechanisms provide stability and end-to-end optimization, explicit interaction offers greater interpretability and controllability at the cost of increased system complexity.

Overall, rather than converging to a single unified framework, future research may benefit from hybrid strategies that balance static and interactive involvement while combining implicit and explicit forms of language guidance.

5.4. Evaluation Reliability

In contrast to traditional evaluation metrics, language-driven IQA models do not always produce numerically stable or strictly consistent scores. Instead, they often operate in a semantic assessment space, where quality judgments are expressed through preferences, rankings, or linguistic interpretations [95,203]. Recently, several studies [45,98] have explored MLLMs as quantitative quality scorers. While such models demonstrate promising alignment with human perception in many scenarios, their evaluation behavior may exhibit variability and uncertainty. In particular, assessment results can be sensitive to prompt formulations, where small variations in phrasing or textual context may lead to inconsistent quality predictions. This prompt sensitivity introduces challenges for reproducibility and comparability across evaluation settings. Moreover, score calibration remains a nontrivial challenge, as language models are not explicitly optimized for metric-level numerical stability. This variability becomes particularly critical when language-based IQA outputs are directly used for model selection or benchmark comparisons.

The coexistence of conventional and language-driven IQA metrics introduces new considerations regarding evaluation reliability. Future research may explore standardized prompting protocols, confidence-aware evaluation mechanisms, and hybrid assessment frameworks that combine deterministic metrics with language-driven quality reasoning. Nevertheless, ensuring stable and comparable evaluation remains an open problem.

5.5. Dataset Design for Language-Driven IR

Numerous datasets have been proposed for various restoration tasks. However, several limitations still remain in existing datasets. Current restoration benchmarks rarely incorporate structured linguistic annotations describing degradation characteristics or perceptual attributes. While VLMs and MLLMs benefit from large-scale pretraining and strong semantic understanding, they may still exhibit failure modes in complex or ambiguous scenarios. Consequently, language-driven restoration frameworks often rely on synthetic prompts [112] or automatically generated descriptions [56,68], which can introduce semantic inconsistencies. This highlights the importance of reliable textual supervision for improving model stability and performance. Moreover, most restoration datasets are constructed under predefined degradation models, which may not adequately capture the complexity or compositional nature of real-world degradations. Such limitations may restrict the robustness and generalization capability of language-integrated restoration systems.

Future dataset construction may benefit from multi-level annotations beyond pixel-level ground truth, including degradation semantics, perceptual quality descriptions, and task-oriented linguistic guidance. Furthermore, developing standardized degradation annotation protocols represents a critical research direction for improving cross-modal consistency while reducing annotation ambiguity. Furthermore, incorporating diverse, composed, and open-world degradations may enhance model robustness under realistic conditions.

5.6. Leveraging Multimodal Data and High-Dimensional Representations

VLMs and MLLMs enable IR frameworks to incorporate textual priors and cross-modal semantics, opening new opportunities to exploit richer data modalities. However, existing studies mainly focus on RGB images, while other sensing modalities remain largely underexplored, including infrared imagery, event-based data, and depth measurements [229,230]. Integrating heterogeneous sensory inputs may improve the interpretation of degradation, structural reasoning, and contextual consistency. Therefore, extending language-driven restoration frameworks to these modalities presents a promising research direction.

Beyond static images, applying language-driven restoration paradigms to high-dimensional data, such as videos, represents another important direction for future research. Video restoration inherently requires not only accurate frame-level reconstruction but also temporal consistency across frames. In this context, language-driven reasoning mechanisms may provide complementary benefits by scene-level coherence. For example, MLLMs may assist in interpreting dynamic degradation patterns [231] or evaluating video quality to support invoking tools.

5.7. Ethics and Trustworthiness

Ethical considerations and trustworthiness have become increasingly relevant in language-driven IR frameworks. Certain approaches rely on online inference services, which require transmitting visual data for non-local processing [138]. Such designs may raise privacy and data security concerns, particularly in sensitive scenarios including medical imaging, unmanned aerial vehicle (UAV), and autonomous driving.

Beyond data privacy, the reliability of language-driven components introduces additional challenges. MLLMs are known to exhibit hallucinations, reasoning inconsistencies, and biases inherited from large-scale training data. When integrated into restoration pipelines, these factors may result in incorrect degradation interpretation, unstable guidance signals, or semantically inconsistent restoration outcomes. Unlike conventional restoration errors, failures caused by MLLMs or VLMs may be less predictable and more difficult to attribute.

Future research may investigate privacy-preserving multimodal restoration frameworks and robustness against hallucination-induced errors. In addition, the development of locally deployable multimodal models [72] represents a promising direction for mitigating privacy concerns while preserving semantic reasoning capabilities.

6. Conclusions

This survey presents a unified view of language-driven IR by systematically analyzing how VLMs and MLLMs are integrated into restoration pipelines. Instead of categorizing methods solely by model architecture, we introduced an interaction-centric taxonomy that characterizes the role of language across feature-level, optimization-level, and execution-level paradigms. This perspective reveals that recent advances in IR are not merely driven by architectural modifications, but by fundamentally new ways of incorporating language priors into low-level vision tasks. Building on this taxonomy, we reviewed representative language-driven IR frameworks and highlighted how language information enables more flexible, controllable, and semantically aware restoration. We further summarized emerging language-driven IQA methods, emphasizing their differences from conventional metrics, particularly in capturing perceptual and contextual quality. In addition, we provided a comprehensive overview of commonly used datasets and compared recent methods across multiple restoration tasks.

Despite these advances, language-driven IR still faces several fundamental challenges. These include the trade-off between semantic guidance and pixel-level fidelity, limited generalization under complex and unseen degradations, high computational cost, and the lack of reliable and standardized evaluation protocols. Moreover, the construction of high-quality image–text paired datasets remains underexplored, and the implementation of online MLLMs raises concerns about robustness and trustworthiness in real-world applications. These challenges suggest that language-driven restoration should be viewed not only as an extension of existing pipelines but as a paradigm shift that requires rethinking both system-level model design and evaluation strategies. We further anticipate future research directions: exploring uncertainty-aware restoration mechanisms, more efficient semantic-guided architectures, and scalable annotation strategies for image–text paired datasets. Extending language-driven restoration paradigms to heterogeneous modalities and high-dimensional data also presents promising opportunities.

In summary, as language and vision models continue to develop, their integration is expected to play an increasingly important role in bridging low-level visual restoration with high-level semantic understanding. We hope this survey provides a clear and structured foundation for understanding the evolving design principles of language-driven IR and encourages further investigation into this rapidly developing research direction.s