1. Introduction

• Comprehensive scope: Unlike research endeavors that restrict their inquiry to a singular temporal scale, our survey provides a comprehensive analysis that amalgamates short-term weather forecasting with medium- and long-term climate predictions. In total, 20 models were surveyed, from which a select subset of eight were chosen for an in-depth scrutiny.These models are discerned as the industry’s avant-garde, thereby serving as invaluable references for researchers. For instance, the PanGu model exhibits a remarkable congruence with actual observational results, thereby illustrating the caliber of models included in our analysis
• In-Depth Analysis: Breaking new ground, this study delves into the intricate operational mechanisms of the eight focal models. We have dissected the operating mechanisms of these eight models, distinguishing the differences in their approaches and summarizing the commonalities in their methods through comparison. This comparison helps readers gain a deeper understanding of the efficacy and applicability of each model and provides a reference for choosing the most appropriate model for a given scenario.
• Identification of Contemporary Challenges and Future Work: The survey identifies pressing challenges currently facing the field, such as limited dataset of chronological seasons and complex climate change effects, and suggests directions for future work, including simulating dataset and physics-Based Constraint model. These recommendations not only add a forward-looking dimension to our research but also act as a catalyst for further research and development in climate prediction.

2. Background

3. Related work

Table 2. Commonly used symbols and definitions.

4. Taxonomy of climate prediction applications

4.1. Climate prediction Milestone based on machine-learning.

In this subsection, we surveyed almost 20 methods of machine learning applications for weather prediction and climate prediction. These methods are representative and common. We listed them in the following timeline. The journey of machine learning applications in climate and weather prediction has undergone significant transformations since its inception.

Climate prediction methods before 2010. The earliest model in this context is the Precipitation Neural Network prediction model published in 1998. This model serves as an archetype of Basic DNN Models, leveraging Artificial Neural Networks to offer short-term forecasts specifically for precipitation in the Middle Atlantic Region. Advancing to the mid-2000s, the realm of medium-to-long-term predictions saw the introduction of ML-Enhanced Non-Deep-Learning Models, exemplified by KNN-Downscaling in 2005 and SVM-Downscaling in 2006. These models employed machine learning techniques like K-Nearest Neighbors and Support Vector Machines, targeting localized precipitation forecasts in the United States and India, respectively. In 2009, the field welcomed another medium-to-long-term model, CRF-Downscaling, which used Conditional Random Fields to predict precipitation in the Mahanadi Basin.

Climate prediction methods from 2010 - 2019. During the period from 2010 to 2019, the field of weather prediction witnessed significant technological advancements and diversification in modeling approaches. Around 2015, a notable shift back to short-term predictions was observed with the introduction of Hybrid DNN Models, exemplified by ConsvLSTM. This model integrated Long Short-Term Memory networks with Convolutional Neural Networks to provide precipitation forecasts specifically for Hong Kong. As the decade progressed, models became increasingly specialized. For instance, the 2017 Precipitation Convolution prediction model leveraged Convolutional Neural Networks to focus on localized precipitation forecasts in Guang Dong, China. The following year saw the emergence of the Stacked-LSTM-Model, which utilized Long Short-Term Memory networks for temperature predictions in Amsterdam and Eindhoven.

Figure 1. Applications of machine-learning on climate prediction milestone.

Figure 1. Applications of machine-learning on climate prediction milestone.

Climate prediction methods from 2020. Fast forward to 2020, the CapsNet model, a Specific Model, leveraged a novel architecture known as Capsule Networks to predict extreme weather events in North America. By 2021, the scope extended to models like RF-bias-correction and the Sea-ice prediction model, focusing on medium-to-long-term predictions. The former employed Random Forests for precipitation forecasts in Iran, while the latter utilized probabilistic deep learning techniques for forecasts in the Arctic region. Recent advancements as of 2022 and 2023 incorporate more complex architectures. Cycle GAN, a 2022 model, utilized Generative Adversarial Networks for global precipitation prediction. PanGu, a 2023 release, employed 3D Neural Networks for predicting extreme weather events globally. Another recent model, FourCastNet, leveraged a technique known as AFNO to predict extreme global events. And in 2022, this year also witnessed the introduction of DeepESD-Downscaling and CNN-Bias-correction models, both utilizing Convolutional Neural Networks to predict local temperature scales and perform global bias correction, respectively.

5. Short-term weather forecast

5.1. Model Design

Numerical Weather Model. Numerical Weather Prediction (NWP) stands as a cornerstone methodology in the realm of meteorological forecasting, fundamentally rooted in the simulation of atmospheric dynamics through intricate physical models. At the core of NWP lies a set of governing physical equations that encapsulate the holistic behaviour of the atmosphere:

• The Navier-Stokes Equations [74]: Serving as the quintessential descriptors of fluid motion, these equations delineate the fundamental mechanics underlying atmospheric flow.

  $$\nabla ·\mathbf{v}=0$$

  (3)

  $$\rho \left(\frac{\partial \mathbf{v}}{\partial t}+\mathbf{v}·\nabla \mathbf{v}\right)=-\nabla p+\mu {\nabla }^2\mathbf{v}+\rho \mathbf{g}$$

  (4)
• The Thermodynamic Equations [75]: These equations intricately interrelate the temperature, pressure, and humidity within the atmospheric matrix, offering insights into the state and transitions of atmospheric energy.

  $$\frac{\partial \rho }{\partial t}+\nabla ·\left(\rho \mathbf{v}\right)=0(\mathrm{Continuity}\mathrm{equation})$$

  (5)

  $$\frac{\partial T}{\partial t}+\mathbf{v}·\nabla T=\frac{q}{c_p}(\mathrm{Energy}\mathrm{equation})$$

  (6)

  $$\frac{Dp}{Dt}=-\rho c_p\nabla ·\mathbf{v}(\mathrm{Pressure}\mathrm{equation})$$

  (7)

The model is fundamentally based on a set of time-dependent partial differential equations, which require sophisticated numerical techniques for solving. The resolution of these equations enables the simulation of the inherently dynamic atmosphere, serving as the cornerstone for accurate and predictive meteorological insights. Within this overarching framework, a suite of integral components is embedded to address specific physical interactions that occur at different resolutions, such as cloud formation, radiation, convection, boundary layers, and surface interactions. Each of these components serves a pivotal role:

• The Cloud Microphysics Parameterization Scheme is instrumental for simulating the life cycles of cloud droplets and ice crystals, thereby affecting [78,79] and the atmospheric energy balance.
• Shortwave and Longwave Radiation Transfer Equations elucidate the absorption, scattering, and emission of both solar and terrestrial radiation, which in turn influence atmospheric temperature and dynamics.
• Empirical or Semi-empirical Convection Parameterization Schemes simulate vertical atmospheric motions initiated by local instabilities, facilitating the capture of weather phenomena like thunderstorms.
• Boundary-Layer Dynamics concentrate on the exchanges of momentum, energy, and matter between the Earth’s surface and the atmosphere, which are crucial for the accurate representation of surface conditions in the model.
• Land Surface and Soil/Ocean Interaction Modules simulate the exchange of energy, moisture, and momentum between the surface and the atmosphere, while also accounting for terrestrial and aquatic influences on atmospheric conditions.

These components are tightly coupled with the core atmospheric dynamics equations, collectively constituting a comprehensive, multi-scale framework. This intricate integration allows for the simulation of the complex dynamical evolution inherent to the atmosphere, contributing to more reliable and precise weather forecasting.

In Numerical Weather Prediction (NWP), a critical tool for atmospheric dynamics forecasting, the process begins with data assimilation, where observational data is integrated into the model to reflect current conditions. This is followed by numerical integration, where governing equations are meticulously solved to simulate atmospheric changes over time. However, certain phenomena, like microphysics of clouds, cannot be directly resolved and are accounted for through parameterization to approximate their aggregate effects. Finally, post-processing methods are used to reconcile potential discrepancies between model predictions and real-world observations, ensuring accurate and reliable forecasts. This comprehensive process captures the complexity of weather systems and serves as a robust method for weather prediction [80]. While the sophistication of NWP allows for detailed simulations of global atmospheric states, one cannot overlook the intensive computational requirements of such models. Even with the formidable processing capabilities of contemporary supercomputers, a ten-day forecast simulation can necessitate several hours of computational engagement.

MetNet. MetNet [47] is a state-of-the-art weather forecasting model that integrates the functionality of CNN, LSTM, and auto-encoder units. The CNN component conducts a multi-scale spatial analysis, extracting and abstracting meteorological patterns across various spatial resolutions. In parallel, the LSTM component captures temporal dependencies within the meteorological data, providing an in-depth understanding of weather transitions over time [43]. Autoencoders are mainly used in weather prediction for data preprocessing, feature engineering and dimensionality reduction to assist more complex prediction models in making more accurate and efficient predictions.This combined architecture permits a dynamic and robust framework that can adaptively focus on key features in both spatial and temporal dimensions, guided by an embedded attention mechanism [81,82].

MetNet is consist of three core components: Spatial Downsampler, Temporal Encoder (ConvLSTM), and Spatial Aggregator. In this architecture, the Spatial Downsampler acts as an efficient encoder that specializes in transforming complex, high-dimensional raw data into a more compact, low-dimensional, information-intensive form. This process helps in feature extraction and data compression. The Temporal Encoder, using the ConvLSTM (Convolutional Long Short-Term Memory) model, is responsible for processing this dimensionality-reduced data in the temporal dimension. One of the major highlights of ConvLSTM is that it combines the advantages of CNNs and LSTM. The advantage of ConvLSTM is that it combines the advantages of CNN and LSTM, and is able to consider the localization of space in time series analysis simultaneously, increasing the model’s ability to perceive complex time and space dependencies. The Spatial Aggregator plays the role of an optimized, high-level decoder. Rather than simply recovering the raw data from its compressed form, it performs deeper aggregation and interpretation of global and local information through a series of axial self-attentive blocks, thus enabling the model to make more accurate weather predictions. These three components work in concert with each other to form a powerful and flexible forecasting model that is particularly well suited to handle meteorological data with a high degree of spatio-temporal complexity.

The operational workflow of MetNet begins with the preprocessing of atmospheric input data, such as satellite imagery and radar information [83]. Spatial features are then discerned through the CNN layers, while temporal correlations are decoded via the LSTM units. This information is synthesized, with the attention mechanism strategically emphasizing critical regions and timeframes, leading to short-term weather forecasts ranging from 2 to 12 hours [82]. MetNet’s strength lies in its precise and adaptive meteorological predictions, blending spatial and temporal intricacies, and thus offers an indispensable tool for refined weather analysis [47].

Figure 2. MetNet Structure.

Figure 2. MetNet Structure.

FourCastNet. In response to the escalating challenges posed by global climate changes and the increasing frequency of extreme weather phenomena, the demand for precise and prompt weather forecasting has surged. High-resolution weather models serve as pivotal instruments in addressing this exigency, offering the ability to capture finer meteorological features, thereby rendering more accurate predictions [84,85]. Against this backdrop, FourCastNet [48] has been conceived, employing ERA5, an atmospheric reanalysis dataset. This dataset is the outcome of a Bayesian estimation process known as data assimilation, fusing observational results with numerical models’ output [87]. FourCastNet leverages the Adaptive Fourier Neural Operator (AFNO), uniquely crafted for high-resolution inputs, incorporating several significant strides within the domain of deep learning.

The essence of AFNO resides in its symbiotic fusion of the Fourier Neural Operator (FNO) learning strategy with the self-attention mechanism intrinsic to Vision Transformers (ViT) [88]. While FNO, through Fourier transforms, adeptly processes periodic data and has proven efficacy in modeling complex systems of partial differential equations, the computational complexity for high-resolution inputs is prohibitive. Consequently, AFNO deploys the Fast Fourier Transform (FFT) in the Fourier domain, facilitating continuous global convolution. This innovation reduces the complexity of spatial mixing to $O(NlogN)$, thus rendering it suitable for high-resolution data [86]. The workflow of AFNO encompasses data preprocessing, feature extraction with FNO, feature processing with ViT, spatial mixing for feature fusion, culminating in prediction output, representing future meteorological conditions such as temperature, pressure, and humidity.

Tailoring AFNO for weather prediction, FourCastNet introduces specific adaptations. Given its distinct application scenario—predicting atmospheric variables utilizing the ERA5 dataset—a dedicated precipitation model is integrated into FourCastNet, predicting six-hour accumulated total precipitation [87]. Moreover, the training paradigm of FourCastNet includes both pre-training and fine-tuning stages. The former learns the mapping from weather state at one time point to the next, while the latter forecasts two consecutive time steps. The advantages of FourCastNet are manifested in its unparalleled speed—approximately 45,000 times swifter than conventional NWP models—and remarkable energy efficiency—consuming about 12,000 times less energy compared to the IFS model [88]. The model’s architectural innovations and its efficient utilization of computational resources position it at the forefront of high-resolution weather modeling.

Figure 3. (a) The multi-layer transformer architecture (b) two-step fine-tuning (c) backbone model (d) forecast model in free-running autoregressive inference mode.

Figure 3. (a) The multi-layer transformer architecture (b) two-step fine-tuning (c) backbone model (d) forecast model in free-running autoregressive inference mode.

GraphCast.GraphCast represents a notable advance in weather forecasting, melding machine learning with complex dynamical system modelling to pave the way for more accurate and efficient predictions. It leverages machine learning to model complex dynamical systems and showcases the potential of machine learning in this domain. It’s an autoregressive model, built upon graph neural networks (GNNs) and a novel multi-scale mesh representation, trained on historical weather data from the European Centre for Medium-Range Weather Forecasts (ECMWF)’s ERA5 reanalysis archive.

The structure of GraphCast employs an "encode-process-decode" configuration utilizing GNNs to autoregressively generate forecast trajectories. In detail:

• Encoder: The encoder component maps the local region of the input data (on the original latitude-longitude grid) onto the nodes of the multigrid graphical representation. It maps two consecutive input frames of the latitude-longitude input grid, with numerous variables per grid point, into a multi-scale internal mesh representation. This mapping process helps the model better capture and understand spatial dependencies in the data, allowing for more accurate predictions of future weather conditions.
• Processor: This part performs several rounds of message-passing on the multi-mesh, where the edges can span short or long ranges, facilitating efficient communication without necessitating an explicit hierarchy. More specifically, the section uses a multi-mesh graph representation. It refers to a special graph structure that is able to represent the spatial structure of the Earth’s surface in an efficient way. In a multi-mesh graph representation, nodes may represent specific regions of the Earth’s surface, while edges may represent spatial relationships between these regions. In this way, models can capture spatial dependencies on a global scale and are able to utilize the power of GNNs to analyze and predict weather changes.
• Decoder: It then maps the multi-mesh representation back to the latitude-longitude grid as a prediction for the next time step.

Figure 4. (a) The encoder component of the GraphCast architecture maps the input local regions (green boxes) to the nodes of the multigrid graph.(b) The processor component uses learned message passing to update each multigrid node. (c) The decoder component maps the processed multigrid features (purple nodes) to the grid representation. (d) A multi-scale grid set.

Figure 4. (a) The encoder component of the GraphCast architecture maps the input local regions (green boxes) to the nodes of the multigrid graph.(b) The processor component uses learned message passing to update each multigrid node. (c) The decoder component maps the processed multigrid features (purple nodes) to the grid representation. (d) A multi-scale grid set.

The workflow of GraphCast begins with the input of weather state(s) defined on a high-resolution latitude-longitude-pressure-levels grid. The encoder processes these inputs into a multi-scale internal mesh representation, which then undergoes many rounds of message-passing in the processor to capture spatio-temporal relationships in the weather data. Finally, the decoder translates the multi-mesh representation back to the latitude-longitude grid to generate predictions for subsequent time steps. It is worth noting that, as shown in the d part, due to the multi-scale mesh mapping property, the model is able to capture both localized weather features on a high-resolution mesh and large-scale weather features on a low-resolution mesh at the same time.

In essence, GraphCast encapsulates a pioneering stride in enhancing weather forecasting accuracy and efficiency through the amalgamation of machine learning and complex dynamical system modelling. It uniquely employs an autoregressive model structure underpinned by graph neural networks and a multi-scale mesh representation. The model’s "encode-process-decode" configuration, executed through a novel multi-mesh graphical representation, adeptly captures spatial dependencies and facilitates global-scale weather prediction. By processing high-resolution weather data inputs through a systematic workflow of encoding, message-passing, and decoding, GraphCast not only generates precise weather predictions for subsequent time intervals but also exemplifies the profound potential of machine learning in advancing meteorological forecasting methodologies.

PanGu. In the rapidly evolving field of meteorological forecasting, PanGu emerges as a pioneering model, predicated on a three-dimensional neural network that transcends traditional boundaries of latitude and longitude. Recognizing the intrinsic relationship between meteorological data and atmospheric pressure, PanGu incorporates a neural network structure that accounts for altitude in addition to latitude and longitude. The initiation of the PanGu model’s process involves Block Embedding, where the dataset is parsed into smaller subsets or blocks. This operation not only mitigates spatial resolution and complexity but also facilitates the subsequent data management within the network.

Following block embedding, the PanGu model integrates the data blocks into a 3D cube through a process known as 3D Cube Fusion, thereby enabling data processing within a tri-dimensional space. Swin Encoding [90], a specialized Transformer encoder utilized in the deep learning spectrum, applies a self-attention mechanism for data comprehension and processing. This encoder, akin to the Autoencoder, excels in extracting and encoding essential information from the dataset. The ensuing phases include Decoding, which strives to unearth salient information, and Output Splitting, which partitions data into atmospheric and surface variables. Finally, Resolution Restoration reinstates the data to its original spatial resolution, making it amenable for further scrutiny and interpretation.

PanGu [50]’s innovative 3D neural network architecture [91] offers a groundbreaking perspective for integrating meteorological data, and its suitability for three-dimensional data is distinctly pronounced. Moreover, PanGu introduces a hierarchical time-aggregation strategy, an advancement that ensures the network with the maximum lead time is consistently invoked, thereby curtailing errors. In juxtaposition with running a model like FourCastNet [48] multiple times, which may accrue errors, this approach exhibits superiority in both speed and precision. Collectively, these novel attributes and methodological advancements position PanGu as a cutting-edge tool in the domain of high-resolution weather modeling, promising transformative potential in weather analysis and forecasting.

Figure 5. Network training and inference strategies. a. 3DEST architecture. b. Hierarchical temporal aggregation. We use FM1, FM3, FM6 and FM24 to indicate the forecast models with lead times being 1 h, 3 h, 6 h or 24 h, respectively.

Figure 5. Network training and inference strategies. a. 3DEST architecture. b. Hierarchical temporal aggregation. We use FM1, FM3, FM6 and FM24 to indicate the forecast models with lead times being 1 h, 3 h, 6 h or 24 h, respectively.

MetNet, FourCastNet, GraphCast and PanGu are state-of-the-art methods in the field of weather prediction, and they share some architectural similarities that can indicate converging trends in this field. All four models initiate the process by embedding or downsampling the input data. FourCastNet uses AFNO, MetNet employs a Spatial Downsampler, and PanGu uses Block Embedding to manage the spatial resolution and complexity of the datasets, while GraphCast maps the input data from the original latitude-longitude grid into a multi-scale internal mesh representation. Spatio-temporal coding is an integral part of all networks; FourCastNet uses pre-training and fine-tuning phases to deal with temporal dependencies, MetNet uses ConvLSTM, and PanGu introduces a hierarchical temporal aggregation strategy to manage temporal correlations in the data, while GraphCast employs GNNs to capture and address spatio-temporal dependencies in weather data. Each model employs a specialized approach to understand the spatial relationships within the data. FourCastNet uses AFNO along with Vision Transformers, MetNet utilizes Spatial Aggregator blocks, and PanGu integrates data into a 3D cube via 3D Cube Fusion, while GraphCast translates data into multi-scale internal mesh. Both FourCastNet and PanGu employ self-attention mechanisms, derived from the Transformer architecture, for better capturing long-range dependencies in the data. FourCastNet combines FNO with ViT, andPanGu uses Swin Encoding.

6. Medium-to-long-term climate prediction

6.1. Model Design

Climate Model. Climate models, consisting of fundamental atmospheric dynamics and thermodynamics equations, focus on simulating Earth’s long-term climate system [93]. Unlike NWP which targets short-term weather patterns, climate models address broader climatic trends. These models encompass Global Climate Models (GCMs), which provide a global perspective but often at a lower resolution, and Regional Climate Models (RCMs), designed for detailed regional analysis [94]. The main emphasis is on the average state and variations rather than transient weather events. The workflow of climate modelling begins with initialization by setting boundary conditions, possibly involving centuries of historical data. Numerical integration follows, using the basic equations to model the long-term evolution of the climate system [95]. Parameterization techniques are employed to represent sub-grid scale processes like cloud formation and vegetation feedback. The model’s performance and uncertainties are then analyzed and validated by comparing them with observational data or other model results [96]. The advantages of climate models lie in their ability to simulate complex climate systems, providing forecasts and insights into future climate changes, thereby informing policy and adaptation strategies. However, they also present challenges such as high computational demands, sensitivity to boundary conditions, and potential uncertainties introduced through parameterization schemes. The distinction between GCMs and RCMs, and their integration in understanding both global and regional climate phenomena, underscores the sophistication and indispensable role of these models in advancing meteorological studies [97].

Conditional Generative Forecasting [62]. In the intricate arena of medium to long-term seasonal climate prediction, the scarcity of substantial datasets since 1979 poses a significant constraint on the rigorous training of complex models like CNNs, thus limiting their predictive efficacy. To navigate this challenge, a pioneering approach of transfer learning has been embraced, leveraging the simulated climate data drawn from CMIP5 (Coupled Model Intercomparison Project Phase 5) [98]to enhance modeling efficiency and accuracy. The process begins with a pre-training phase, where the CNN is enriched with CMIP5 data to comprehend essential climatic patterns and relationships. This foundational insight then transfers seamlessly to observational data without resetting the model parameters, ensuring a continuous learning trajectory that marries simulated wisdom with empirical climate dynamics. The methodology culminates in a fine-tuning phase, during which the model undergoes subtle refinements to align more closely with the real-world intricacies of medium to long-term ENSO forecasting [99]. This innovative strategy demonstrates the transformative power of transfer learning in addressing the formidable challenges associated with limited sample sizes in medium to long-term climate science.

Leveraging 52,201 years of climate simulation data from CMIP5/CMIP6, which serves to increase the sample size, the method for medium-term forecasting employs CNNs and Temporal Convolutional Neural Networks (TCNNs) to extract essential features from high-dimensional geospatial data. This feature extraction lays the foundation for probabilistic deep learning, which determines an approximate distribution of the target variables, capturing the data’s structure and uncertainty [100]. The model’s parameters are optimized through maximizing the Evidence Lower Bound (ELBO) within the variational inference framework. The integration of deep learning techniques with probabilistic modeling ensures accuracy, robustness to sparse data, and flexibility in assumptions, enhancing the precision of forecasts and offering valuable insights into confidence levels and expert knowledge integration.

Leveraging advanced techniques in variational inference and neural networks, the method described seeks to approximate the complex distribution $p(Y\mid X,M)$, where Y is the target variable, and X and M are predictor and GCM index information, respectively. The process is outlined as follows:

• Problem Definition: The goal is to approximate $p(Y\mid X,M)$, a task challenged by high-dimensional geospatial data, data inhomogeneity, and a large dataset.
• Model Specification:
  • Random Variable z: A latent variable with a fixed standard Gaussian distribution.
  • Parametric Functions $p_{\theta },q_{\varphi },p_{\psi }$: Neural networks for transforming z and approximating target and posterior distributions.
  • Objective Function: Maximization of the Evidence Lower Bound (ELBO).
• Training Procedure:
  • Initialize: Define random variable $z\sim N(0,1)$ [101] [102] [103]
    parametric functions $p_{\theta }(z,X,M),q_{\varphi }(z\mid X,Y,M),p_{\psi }(Y\mid X,M,z)$.
  • Training Objective (Maximize ELBO) [104]: The ELBO is defined as:

    $$\mathrm{ELBO}={\mathbb{E}}_{z\sim q_{\varphi }}\left(log{p}_{\psi }(Y\mid X,M,z)\right)-{\mathcal{D}}_{\mathrm{KL}}(q_{\varphi }\parallel p(z\mid X,M))-{\mathcal{D}}_{\mathrm{KL}}(q_{\varphi }\parallel p(z\mid X,Y,M))$$

    (8)

    with terms for reconstruction, regularization, and residual error.
  • Optimization: Utilize variational inference, Monte Carlo reparameterization, and Gaussian assumptions.
• Forecasting: Generate forecasts by sampling $p(z\mid X,M)$, the likelihood of $p_{\psi }$, and using the mean of $p_{\psi }$ for an average estimate.

Figure 6. Conditonal Generative Forecasting (CGF) model.

Figure 6. Conditonal Generative Forecasting (CGF) model.

This method embodies a rigorous approach to approximating complex distributions, bridging deep learning and probabilistic modeling to enhance forecasting accuracy and insights.

$$\mathrm{ELBO}\left(\lambda \right)={\mathbb{E}}_{q\left(\mathbf{z}\right|\mathbf{x})}\left[logp(\mathbf{x},\mathbf{z})-logq\left(\mathbf{z}\right|\mathbf{x})\right](\mathrm{Evidence}\mathrm{Lower}\mathrm{Bound})$$

(9)

In summary, the combination of deep learning and probabilistic insights presents a unique and potent method for spatial predictive analytics. The approach is marked by scalability, flexibility, and an ability to learn complex spatial features, even though challenges persist such as intrinsic complexity in computational modeling and the requirement for profound statistical and computer science background. Its potential in handling large data settings and adapting to varying scenarios highlights its promising applicability in modern spatial predictive analytics, representing an advanced tool in the arena of seasonal climate prediction.

Cycle-Consistent Generative Adversarial Networks. Cycle-Consistent Generative Adversarial Networks (CycleGANs) have been ingeniously applied to the bias correction of high-resolution Earth System Model (ESM) precipitation fields, such as GFDL-ESM4 [105]. This model includes two generators responsible for translating between simulated and real domains, and two discriminators to differentiate between generated and real observations. A key component of this approach is the cycle consistency loss, ensuring a reliable translation between domains, coupled with a constraint to maintain global precipitation values for physical consistency. By framing bias correction as an image-to-image translation task, CycleGANs have significantly improved spatial patterns and distributions in climate projections. The model’s utilization of spatial spectral densities and fractal dimension measurements further emphasizes its spatial context-awareness, making it a groundbreaking technique in the field of climate science. CycleGAN model consists of two generators and two discriminators along with a cycle consistency loss:

• Two Generators: The CycleGAN model includes two generators. Generator G learns the mapping from the simulated domain to the real domain, and generator F learns the mapping from the real domain to the simulated domain [106].
• Two Discriminators: There are two discriminators, one for the real domain and one for the simulated domain. Discriminator $D_x$ encourages generator G to generate samples that look similar to samples in the real domain, and discriminator $D_y$ encourages generator F to generate samples that look similar to samples in the simulated domain.
• Cycle Consistency Loss: To ensure that the mappings are consistent, the model enforces the following condition through a cycle consistency loss: if a sample is mapped from the simulated domain to the real domain and then mapped back to the simulated domain, it should get a sample similar to the original simulated sample. Similarly, if a sample is mapped from the real domain to the simulated domain and then mapped back to the real domain, it should get a sample similar to the original real sample.

  $${\mathcal{L}}_{\mathrm{cyc}}(G,F)={\mathbb{E}}_{x\sim p_{\mathrm{data}}\left(x\right)}\left[\left|\right|F\left(G\left(x\right)\right)-{x\left|\right|}_1\right]+{\mathbb{E}}_{y\sim p_{\mathrm{data}}\left(y\right)}\left[\left|\right|G\left(F\left(y\right)\right)-{y\left|\right|}_1\right]$$

  (10)
• Training Process: The model is trained to learn the mapping between these two domains by minimizing the adversarial loss and cycle consistency loss between the generators and discriminators.

  $${\mathcal{L}}_{\mathrm{Gen}}(G,F)={\mathcal{L}}_{\mathrm{GAN}}(G,D_y,X,Y)+{\mathcal{L}}_{\mathrm{GAN}}(F,D_x,Y,X)+\lambda {\mathcal{L}}_{\mathrm{cyc}}(G,F)$$

  (11)
• Application to Prediction: Once trained, these mappings can be used for various tasks, such as transforming simulated precipitation data into forecasts that resemble observed data.

Figure 7. CycleGAN flow chart.

Figure 7. CycleGAN flow chart.

The bidirectional mapping strategy of Cycle-Consistent Generative Adversarial Networks (CycleGANs) permits the exploration and learning of complex transformation relationships between two domains, without reliance on paired training samples. This attribute holds profound significance, especially in scenarios where only unlabeled data are available for training. In the specific application within climate science, this characteristic of CycleGAN enables precise capturing and modeling of the subtle relationships between real and simulated precipitation data. Through this unique bidirectional mapping, the model not only enhances the understanding of climatic phenomena but also improves the predictive accuracy of future precipitation trends. This provides a novel, data-driven methodology for climate prediction and analysis, contributing to the ever-expanding field of computational climate science.

DeepESD. Traditional GCMs, while proficient in simulating large-scale global climatic dynamics [107,108], exhibit intrinsic limitations in representing finer spatial scales and specific regional characteristics. This inadequacy manifests as a pronounced resolution gap at localized scales, restricting the applicability of GCMs in detailed regional climate studies [109,110].

In stark contrast, the utilization of CNNs symbolizes a significant breakthrough [111]. Structurally characterized by hierarchical convolutional layers, CNNs possess the unique ability to articulate complex multi-scale spatial features across disparate scales, commencing from global coarse-grained characteristics and progressively refining to capture intricate regional details. An exemplar implementation of this approach was demonstrated by Baño-Medina et al. [112], wherein a CNN comprised three convolutional layers with spatial kernels of varying counts (50, 25, and 10 respectively). The transformation process began with the recalibration of ERA-Interim reanalysis data to a 2° regular grid, elevating it to 0.5° [113,114,115]. This configuration allowed the CNN to translate global atmospheric patterns into high-resolution regional specificity [116,117].

The nuanced translation from global to regional scales, achieved through sequential convolutional layers, not only amplifies the spatial resolution but also retains the contextual relevance of climatic variables [118,119]. The first convolutional layer captured global coarse-grained features, with subsequent layers incrementally refining these into nuanced regional characteristics. By the terminal layer, the CNN had effectively distilled complex atmospheric dynamics into a precise high-resolution grid [120,121].

This enhancement fosters a more robust understanding of regional climatic processes, ushering in an era of precision and flexibility in climate modeling. The deployment of this technology affirms a pivotal advancement in the field, opening new possibilities for more granulated, precise, and comprehensive examination of climatic processes and future scenarios [122,123,124]. The introduction of CNNs thus represents a transformative approach to bridging the resolution gap inherent to traditional GCMs, with substantial implications for future climate analysis and scenario planning.

NNCAM.The design and implementation of the Neural Network Community Atmosphere Model (NNCAM) are architected to leverage advancements in machine learning for improved atmospheric simulations. The architecture is a nuanced blend of traditional General Circulation Models (GCMs), specifically the Super-Parameterized Community Atmosphere Model (SPCAM), and cutting-edge machine learning techniques like Residual Deep Neural Networks (ResDNNs).

• Reference Model: SPCAM. SPCAM serves as the foundational GCM and is embedded with Cloud-Resolving Models (CRMs) to simulate microscale atmospheric processes like cloud formation and convection. SPCAM is employed to generate "target simulation data," which serves as the training baseline for the neural networks. The use of CRMs is inspired by recent advancements in data science, demonstrating that machine learning parameterizations can potentially outperform traditional methods in simulating convective and cloud processes.
• Neural Networks: ResDNNs. ResDNNs, a specialized form of deep neural networks, are employed for their ability to approximate complex, nonlinear relationships. The network comprises multiple residual blocks, each containing two fully connected layers with Rectified Linear Unit (ReLU) activations. ResDNNs are designed to address the vanishing and exploding gradient problems in deep networks through residual connections, offering a stable and effective gradient propagation mechanism. This makes them well-suited for capturing the complex and nonlinear nature of atmospheric processes.
• Subgrid-Scale Physical Simulator. Traditional parameterizations often employ simplified equations to model subgrid-scale processes, which might lack accuracy. In contrast, the ResDNNs are organized into a subgrid-scale physical simulator that operates independently within each model grid cell. This simulator takes atmospheric states as inputs and outputs physical quantities at the subgrid scale, such as cloud fraction and precipitation rate.

Figure 8. DeepESD structure.

Figure 8. DeepESD structure.

In the NNCAM model, the core workflow is divided into several key steps to achieve efficient and accurate climate simulations. First, the dynamic core, which serves as the base component of the model, is responsible for solving the underlying hydrodynamic equations and calculating the current climate state, e.g., temperature, pressure, and humidity, as well as the environmental forcings, e.g., wind and solar radiation. These calculations are then transmitted to the NN-GCM coupler. Upon receiving these data, the coupler further passes them to the neural network parameterization module. This module utilizes pre-trained neural networks, specifically ResDNNs, for faster and more accurate parameterization of the climate. Upon completion of the predictions, these results are fed back to the host GCM, i.e., NNCAM.The host GCM then uses the predictions generated by these neural networks to update the climate state in the model, and based on these updates, performs the simulation at the next time step.

Overall, the host GCM, as the core of the whole simulation, is not only responsible for the basic climate simulation, but also efficiently interacts with the dynamic core and neural network parameterization modules to achieve higher simulation accuracy and computational efficiency. This hierarchical architecture ensures both computational efficiency and high simulation fidelity. It allows for seamless integration and synchronization of the model states and predictions, thereby enabling continuous and efficient operation of NNCAM. The proposed framework represents a significant stride in the realm of atmospheric science, offering a harmonious integration of machine learning and physical simulations to achieve unprecedented accuracy and computational efficiency.

CGF, DeepESD, CycleGAN are very different in their uses and implementations, but there are also some levels of similarity. All three approaches focus on mapping from one data distribution to another. And, they focus more on the mechanisms of climate change than previous models for weather forecasting. CycleGAN specifically emphasizes the importance of not only mapping from distribution A to B, but also the inverse mapping capability from B to A, which is to some extent what CGF and DeepESD are concerned with. NNCAM realizes the mapping from physical parameterization to machine learning parameterization. This mapping can be viewed as a functional mapping that replaces parameterized functions in the physical process with functions learned and inferred by the machine learning model.

7. Discussion

8. Conclusion

References

1. Abbe, C. The physical basis of long-range weather forecasts. Mon. Weath. Rev. 1901, 29, 551–561. [Google Scholar] [CrossRef]
2. Zheng, Y.; Capra, L.; Wolfson, O.; Yang, H. Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 2014, 5, 1–55. [Google Scholar] [CrossRef]
3. Gneiting, T.; Raftery, A.E. Weather forecasting with ensemble methods. Science 2005, 310, 248–249. [Google Scholar] [CrossRef]
4. Agapiou, A. Remote sensing heritage in a petabyte-scale: satellite data and heritage Earth Engine applications. Int. J. Digit. Earth 2017, 10, 85–102. [Google Scholar] [CrossRef]
5. Bendre, M. R.; Thool, R. C.; Thool, V. R. Big data in precision agriculture: Weather forecasting for future farming. NGCT. IEEE 2015, 744–750. [Google Scholar]
6. Zavala, V. M.; Constantinescu, E. M.; Krause, T. On-line economic optimization of energy systems using weather forecast information. Journal of Process Control 2009, 1725–1736. [Google Scholar] [CrossRef]
7. Nurmi, Vaino, et al. "Economic value of weather forecasts on transportation–Impacts of weather forecast quality developments to the economic effects of severe weather." EWENT FP7 project, no.490 (2012).
8. Russo Jr, J.A. The economic impact of weather on the construction industry of the United States. Bulletin of the American Meteorological Society 1966, 967–972. [Google Scholar] [CrossRef]
9. Badorf, F.; Kai, H. The impact of daily weather on retail sales: An empirical study in brick-and-mortar stores. Journal of Retailing and Consumer Services 2020, 101921. [Google Scholar] [CrossRef]
10. Smith, K. Environmental hazards: assessing risk and reducing disaster. Routledge 2013. [Google Scholar]
11. Hammer, G. L.; et al. Advances in application of climate prediction in agriculture. Agricultural systems 2001, 515–553. [Google Scholar] [CrossRef]
12. Guedes, Gilvan, Rodrigo Raad, and Lucélia Raad. "Welfare consequences of persistent climate prediction errors on insurance markets against natural hazards." Estudos Econômicos (São Paulo), no.49 (2019): 235-264.
13. McNamara, D.E.; Andrew, K. A coupled physical and economic model of the response of coastal real estate to climate risk. Nature Climate Change 2013, 559–562. [Google Scholar] [CrossRef]
14. Kleerekoper, L.; Van Esch, M.; Salced, T.B. How to make a city climate-proof, addressing the urban heat island effect. Resources, Conservation and Recycling 2012, 30–38. [Google Scholar] [CrossRef]
15. Kaján, Eva, and Jarkko Saarinen. "Tourism, climate change and adaptation: A review." Current Issues in Tourism, no.16.2 (2013): 167-195.
16. de Freitas, C.R. Tourism climatology: evaluating environmental information for decision making and business planning in the recreation and tourism sector. international Journal of Biometeorology 2003, 45–54. [Google Scholar] [CrossRef]
17. Dessai, Suraje, et al. "Climate prediction: a limit to adaptation." Adapting to climate change: thresholds, values, governance, no.64 (2009): 78.
18. Reichstein, M.; Camps-Valls, G.; Stevens, B.; Jung, M.; Denzler, J.; Carvalhais, N. et al.; et al. Deep learning and process understanding for data-driven Earth system science. Nature 2019, 566, 195–195. [Google Scholar] [CrossRef]
19. Howe, L.; Wain, A. Predicting the Future, Vol. V, pp. 1–195. Cambridge Univ. Press (1993).
20. Hantson, S.; et al. The status and challenge of global fire modelling. Biogeosciences 2016, 13, 3359–3375. [Google Scholar] [CrossRef]
21. Racah, E.; Beckham, C.; Maharaj, T.; et al. ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. Advances in Neural Information Processing Systems 2017, 3402–3413. [Google Scholar]
22. Gao, S.; Zhao, P.; Pan, B.; et al. A nowcasting model for the prediction of typhoon tracks based on a long short term memory neural network. Acta Oceanologica Sinica 2018, 37, 8–12. [Google Scholar] [CrossRef]
23. Ham, Y. G.; Kim, J. H.; Luo, J. J. Deep learning for multi-year ENSO forecasts. Nature 2019, 573, 568–572. [Google Scholar] [CrossRef]
24. Ren, X.; Li, X.; Ren, K.; Song, J.; Xu, Z.; Deng, K.; Wang, X. Deep Learning-Based Weather Prediction: A Survey. Big Data Research 2021, 23, 100178. [Google Scholar] [CrossRef]
25. Reichstein, M.; Camps-Valls, G.; Stevens, B.; et al. Deep learning and process understanding for data-driven Earth system science. Nature 2019, 566, 195–204. [Google Scholar] [CrossRef]
26. Stockhause, M.; Lautenschlager, M. CMIP6 data citation of evolving data. Data Sci. J. 2017, 16, 30. [Google Scholar] [CrossRef]
27. Hsieh, W. W. Machine Learning Methods in the Environmental Sciences: Neural Networks and Kernels; Cambridge University Press: Cambridge, 2009. [Google Scholar]
28. Krasnopolsky, V. M.; Fox-Rabinovitz, M. S.; Chalikov, D. V. New Approach to Calculation of Atmospheric Model Physics: Accurate and Fast Neural Network Emulation of Longwave Radiation in a Climate Model. Mon. Weather Rev. 2005, 133, 1370–1383. [Google Scholar] [CrossRef]
29. Krasnopolsky, V. M.; Fox-Rabinovitz, M. S.; Belochitski, A. A. Using ensemble of neural networks to learn stochastic convection parameterizations for climate and numerical weather prediction models from data simulated by a cloud resolving model. Adv. Artif. Neural Syst. 2013, 2013, 1–13. [Google Scholar] [CrossRef]
30. Chevallier, F.; Morcrette, J-J.; Chéruy, F.; Scott, N.A. Use of a neural-network-based long-wave radiative-transfer scheme in the ECMWF atmospheric model. Quarterly Journal of the Royal Meteorological Society 2000, 126, 761–776. [Google Scholar]
31. V.M. Krasnopolsky, M.S. Fox-Rabinovitz, Y.T. Hou, S.J. Lord, and A.A. Belochitski, Accurate and fast neural network emulations of model radiation for the NCEP coupled climate forecast system: Climate simulations and seasonal predictions, Monthly Weather Review, vol. 138, no. 5, pp. 1822–1842, 2010.
32. H.L. Tolman, V.M. Krasnopolsky, and D.V. Chalikov, Neural network approximations for nonlinear interactions in wind wave spectra: direct mapping for wind seas in deep water, Ocean Modelling, vol. 8, no. 3, pp. 253–278, 2005.
33. Markakis, E.; Papadopoulos, A.; Perakakis, P. Spatiotemporal Forecasting: A Survey. arXiv preprint 2018, arXiv:1808.06571. [Google Scholar]
34. Box, G.E.; Jenkins, G.M.; Reinsel, G.C.; Ljung, G.M. Time Series Analysis: Forecasting and Control; John Wiley & Sons: Hoboken, NJ, USA, 2015. [Google Scholar]
35. He, Y.; Kolovos, A. Spatial and Spatio-Temporal Geostatistical Modeling and Kriging; Wiley StatsRef: Statistics Reference Online, 2018. [Google Scholar]
36. Lu, H.; Fan, Z.; Zhu, H. Spatiotemporal Analysis of Air Quality and Its Application in LASG/IAP Climate System Model. Atmospheric and Oceanic Science Letters 2011, 4, 204–210. [Google Scholar]
37. Chatfield, C. The Analysis of Time Series: An Introduction, 7th ed.; CRC Press: Boca Raton, FL, USA, 2016. [Google Scholar]
38. Stull, R. Meteorology for Scientists and Engineers, 3rd ed.; Brooks/Cole: Pacific Grove, CA, USA, 2015. [Google Scholar]
39. Yuval, J.; O’Gorman, P.A. Machine Learning for Parameterization of Moist Convection in the Community Atmosphere Model. Proceedings of the National Academy of Sciences 2020, 117, 12–20. [Google Scholar]
40. Gagne, D.J.; Haupt, S.E.; Nychka, D.W. Machine Learning for Spatial Environmental Data. Meteorological Monographs 2020, 59, 9–1. [Google Scholar]
41. Xu, Z.; Li, Y.; Guo, Q.; Shi, X.; Zhu, Y. A Multi-Model Deep Learning Ensemble Method for Rainfall Prediction. Journal of Hydrology 2020, 584, 124579. [Google Scholar]
42. Kuligowski, R. J.; Barros, A. P. Localized precipitation forecasts from a numerical weather prediction model using artificial neural networks. Weather and Forecasting 1998, 1194–1204. [Google Scholar] [CrossRef]
43. Xingjian, S.; et al. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. arXiv 2015, arXiv:1506.04214. [Google Scholar]
44. Qiu, M.; Zhao, P.; Zhang, K.; Huang, J.; Shi, X.; Wang, X.; Chu, W. A short-term rainfall prediction model using multi-task convolutional neural networks, in: Data Mining (ICDM), IEEE International Conference on, IEEE, 2017, pp. 395–404.
45. Karevan, Z.; Suykens, J.A. Spatio-temporal stacked lstm for temperature prediction in weather forecasting. arXiv arXiv:1811.06341.
46. Chattopadhyay, A.; Nabizadeh, E.; Hassanzadeh, P. Analog Forecasting of extreme-causing weather patterns using deep learning. Journal of Advances in Modeling Earth Systems 2020, 12, e2019MS001958. [Google Scholar] [CrossRef]
47. Jiang, X.; et al. MetNet: A Neural Weather Model for Precipitation Forecasting. arXiv 2020, arXiv:2003.12140. [Google Scholar]
48. J. Pathak et al., "FourCastNet: A Global Data-Driven High-Resolution Weather Model Using Adaptive Fourier Neural Operators," arXiv, February 22, 2022.
49. Lam, R.; Sanchez-Gonzalez, A.; Willson, M.; et al. GraphCast: Learning skillful medium-range global weather forecasting. arXiv 2022, arXiv:2212.12794. [Google Scholar]
50. Bi, K.; Xie, L.; Zhang, H.; Chen, X.; Gu, X.; Tian, Q. Accurate Medium-Range Global Weather Forecasting with 3D Neural Networks. Nature 2023, 619, 533–538. [Google Scholar] [CrossRef] [PubMed]
51. Nguyen, T.; Brandstetter, J.; Kapoor, A.; et al. ClimaX: A foundation model for weather and climate. arXiv 2023, arXiv:2301.10343. [Google Scholar]
52. Gangopadhyay, S.; Clark, M.; Rajagopalan, B. "Statistical downscaling using K-nearest neighbors," Water Resources Research 41, no. 2 (2005). Publisher: Wiley Online Library.
53. Tripathi, S.; Srinivas, V.V.; Nanjundiah, R. S. Downscaling of precipitation for climate change scenarios: a support vector machine approach. Journal of hydrology 2006, 330, 621–640. [Google Scholar] [CrossRef]
54. Krasnopolsky, V. M.; Fox-Rabinovitz, M. S. Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction. Neural Networks 2006, 122–134. [Google Scholar] [CrossRef]
55. Raje, D.; Mujumdar, P.P. "A conditional random field–based downscaling method for assessment of climate change impact on multisite daily precipitation in the Mahanadi basin," Water Resources Research 45, no. 10 (2009). Publisher: Wiley Online Library.
56. Zarei, M.; Najarchi, M.; Mastouri, R. "Bias correction of global ensemble precipitation forecasts by Random Forest method," Earth Science Informatics 14 (2021): 677–689. Publisher: Springer.
57. Andersson, T. R.; Hosking, J. S.; Pérez-Ortiz, M.; Paige, B.; Elliott, A.; Russell, C.; Law, S.; et al. Seasonal Arctic Sea Ice Forecasting with Probabilistic Deep Learning. Nature Communications 2021, 5124. [Google Scholar] [CrossRef]
58. Wang, Xin and Han, Yilun and Xue, Wei and Yang, Guangwen and Zhang, Guang. Stable climate simulations using a realistic general circulation model with neural network parameterizations for atmospheric moist physics and radiation processes. Geoscientific Model Development 2022, 3923–3940.
59. Baño-Medina, J.; Manzanas, R.; Cimadevilla, E.; Fernández, J.; González-Abad, J.; Cofiño, A. S.; Gutiérrez, J. M. Downscaling Multi-Model Climate Projection Ensembles with Deep Learning (DeepESD): Contribution to CORDEX EUR-44. Geoscientific Model Development 2022, 15, 6747–6758. [Google Scholar] [CrossRef]
60. Hess, P.; Lange, S.; Boers, N. Deep Learning for bias-correcting comprehensive high-resolution Earth system models. arXiv 2022, arXiv:2301.01253. [Google Scholar]
61. Wang, F.; Tian, D. "On deep learning-based bias correction and downscaling of multiple climate models simulations," Climate dynamics 59, no. 11-12 (2022): 3451–3468. Publisher: Springer.
62. Pan, B.; Anderson, G. J.; Goncalves, A.; Lucas, D. D.; Bonfils, C. J. W.; Lee, J. Improving Seasonal Forecast Using Probabilistic Deep Learning. Journal of Advances in Modeling Earth Systems 2022, 14, e2021MS002766. [Google Scholar] [CrossRef]
63. Hu, Y.; Chen, L.; Wang, Z.; et al. SwinVRNN: A Data-Driven Ensemble Forecasting Model via Learned Distribution Perturbation. Journal of Advances in Modeling Earth Systems 2023, 15, e2022MS003211. [Google Scholar] [CrossRef]
64. Chen, L.; Zhong, X.; Zhang, F.; et al. FuXi: A cascade machine learning forecasting system for 15-day global weather forecast. arXiv 2023, arXiv:2306.12873. [Google Scholar] [CrossRef]
65. Lin, H.; Gao, Z.; Xu, Y.; et al. Conditional local convolution for spatio-temporal meteorological forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 2022, 7470–7478. [Google Scholar] [CrossRef]
66. Chen, K.; Han, T.; Gong, J.; et al. FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead. arXiv 2023, arXiv:2304.02948. [Google Scholar]
67. C.O. de Burgh-Day and T. Leeuwenburg, Machine Learning for numerical weather and climate modelling: a review, EGUsphere, vol. 2023, pp. 1–48, 2023.
68. LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998, 86, 2278–2324. [Google Scholar] [CrossRef]
69. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 2012, 1097–1105. [Google Scholar] [CrossRef]
70. Scherer, D.; Müller, A.; Behnke, S. Evaluation of pooling operations in convolutional architectures for object recognition. International Conference on Artificial Neural Networks 2010, 92–101. [Google Scholar]
71. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
72. Liu, Y.; Racah, E.; Correa, J.; Khosrowshahi, A.; Lavers, D.; Kunkel, K.; Wehner, M.; Collins, W. Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv preprint 2016, arXiv:1605.01156. [Google Scholar]
73. Goodfellow, I.; Warde-Farley, D.; Mirza, M.; Courville, A.; Bengio, Y. Maxout networks. International Conference on Machine Learning 2013, 1319–1327. [Google Scholar]
74. Marion, Martine, and Temam, Roger. Navier-Stokes equations: Theory and approximation. Handbook of numerical analysis, 6, 503–689 (1998). Elsevier.
75. Iacono, M.J.; Mlawer, E.J.; Clough, S.A.; Jean-Jacques, M. Impact of an improved longwave radiation model, RRTM, on the energy budget and thermodynamic properties of the NCAR community climate model, CCM3. Journal of Geophysical Research: Atmospheres 2000, 105, 14873–14890. [Google Scholar] [CrossRef]
76. Giorgi, F.; Marinucci, M.R.; Bates, G.T. Development of a second-generation regional climate model (RegCM2). Part I: Boundary-layer and radiative transfer processes. Monthly Weather Review 1993, 121, 2794–2813. [Google Scholar] [CrossRef]
77. Mann, G.W.; Carslaw, K.S.; Spracklen, D.V.; Ridley, D.A.; Manktelow, P.T.; Chipperfield, M.P.; Pickering, S.J.; Johnson, C.E. Description and evaluation of GLOMAP-mode: A modal global aerosol microphysics model for the UKCA composition-climate model. Geoscientific Model Development 2010, 3, 519–551. [Google Scholar] [CrossRef]
78. Y. Guo, C. Shao, and A. Su, “Comparative Evaluation of Rainfall Forecasts during the Summer of 2020 over Central East China,” Atmosphere, vol. 14, no. 6, Article 992, 2023. Available online: https://www.mdpi.com/2073-4433/14/6/992.
79. Y. Guo, C. Shao, and A. Su, “Investigation of Land–Atmosphere Coupling during the Extreme Rainstorm of 20 July 2021 over Central East China,” Atmosphere, vol. 14, no. 10, Article 1474, 2023. 20 July. Available online: https://www.mdpi.com/2073-4433/14/10/1474.
80. Bauer, P.; Alan, T.; Gilbert, B. The Quiet Revolution of Numerical Weather Prediction. Nature 2015, 525, 47–55. [Google Scholar] [CrossRef]
81. Vaswani, A. et al., Attention is All You Need, NeurIPS, 2017.
82. Ho, J.; et al. Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation. arXiv 2019, arXiv:2003.07853. [Google Scholar]
83. Schmit, T. J.; et al. A closer look at the ABI on the GOES-R series. Bulletin of the American Meteorological Society 2017, 681–698. [Google Scholar] [CrossRef]
84. Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, "Fourier Neural Operator for Parametric Partial Differential Equations," in International Conference on Learning Representations (ICLR), 2021a.
85. J. Guibas, M. Mardani, Z. Li, A. Tao, A. Anandkumar, and B. Catanzaro, "Adaptive Fourier Neural Operators: Efficient token mixers for transformers," in International Conference on Representation Learning (to appear), 22. 20 April.
86. Rasp, S.; et al. Weatherbench: a benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems 2022, 12, e2020MS002203. [Google Scholar] [CrossRef]
87. Rasp, S.; Thuerey, N. Purely data-driven medium-range weather forecasting achieves comparable skill to physical models at similar resolution. arXiv 2020, arXiv:2008.08626. [Google Scholar]
88. Weyn, J.A.; et al. Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models. arXiv 2021, arXiv:2102.05107. [Google Scholar] [CrossRef]
89. Rasp, S.; et al. Weatherbench: a benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems 2020, 12, e2020MS002203. [Google Scholar] [CrossRef]
90. Z. Liu et al., "Swin transformer: hierarchical vision transformer using shifted windows," in Proc. International Conference on Computer Vision, pp. 10012–10022, IEEE, 2021.
91. A. Dosovitskiy et al., "An image is worth 16x16 words: transformers for image recognition at scale. Available online: https://arxiv.org/abs/2010.11929, 2020.
92. Váňa, F.; Düben, P.; Lang, S.; Palmer, T.; Leutbecher, M.; Salmond, D.; Carver, G. Single precision in weather forecasting models: An evaluation with the IFS. Monthly Weather Review 2017, 145, 495–502. [Google Scholar] [CrossRef]
93. IPCC, 2013: Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.
94. Flato, G., et al. 2013: Evaluation of Climate Models. In: Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.
95. Washington, W. M., & Parkinson, C. L. (2005). An Introduction to Three-Dimensional Climate Modeling. University Science Books.
96. Giorgi, F.; Gutowski, W. J. Regional Dynamical Downscaling and the CORDEX Initiative. Annual Review of Environment and Resources 2015, 40, 467–490. [Google Scholar] [CrossRef]
97. Randall, D. A.; et al. Climate Models and Their Evaluation. In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change; Cambridge University Press: Cambridge, United Kingdom; New York, NY, USA, 2007. [Google Scholar]
98. Taylor, K. E.; Stouffer, R. J.; Meehl, G. A. An overview of CMIP5 and the experiment design. Bulletin of the American Meteorological Society 2012, 93, 485–498. [Google Scholar] [CrossRef]
99. Ham, Y.-G.; Kim, J.-H.; Luo, J.-J. Deep Learning for Multi-Year ENSO Forecasts. Nature 2019, 573, 568–572. [Google Scholar] [CrossRef]
100. Miao, C.; Shen, Y.; Sun, J. Spatial–temporal ensemble forecasting (STEFS) of high-resolution temperature using machine learning models. Journal of Advances in Modeling Earth Systems 2019, 11, 2961–2973. [Google Scholar]
101. Mukkavilli, S., Perone, C. S., Rangapuram, S. S., & Müller, K. R. (2020). Distribution regression forests for probabilistic spatio-temporal forecasting. In International Conference on Machine Learning (ICML).
102. Mylonas, A.; Bontempi, G.; Panagakis, Y. Stochastic Deep Learning for Short-Term Load Forecasting. IEEE Transactions on Power Systems 2021. [Google Scholar]
103. Walker, G.; Charlton-Perez, A.; Lee, R.; Inness, P. Challenges and progress in probabilistic forecasting of convective phenomena: The 2016 GFE/EUMETSAT/NCEP/SPC severe convective weather workshop. Bulletin of the American Meteorological Society 2016, 97, 1829–1835. [Google Scholar]
104. Kingma, D. P.; Welling, M. Auto-encoding variational bayes. arXiv 2013, arXiv:1312.6114. [Google Scholar]
105. Krasting, J. P., John, J. G., Blanton, C., McHugh, C., Nikonov, S., Radhakrishnan, A., ... Zhao, M. (2018). NOAA-GFDL GFDL-ESM4 model output prepared for CMIP6 CMIP. Earth System Grid Federation. [CrossRef]
106. Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223–2232).
107. Brands, S.; Herrera, S.; Fernández, J.; Gutiérrez, J. M. How well do CMIP5 Earth System Models simulate present climate conditions in Europe and Africa? Clim. Dynam. 2013, 41, 803–817. [Google Scholar] [CrossRef]
108. Vautard, R.; Kadygrov, N.; Iles, C.; et al. Evaluation of the large EURO-CORDEX regional climate model ensemble. J. Geophys. Res.-Atmos. 2021, 126, e2019JD032344. [Google Scholar] [CrossRef]
109. Boé, J.; Somot, S.; Corre, L.; Nabat, P. Large discrepancies in summer climate change over Europe as projected by global and regional climate models: causes and consequences. Clim. Dynam. 2020, 54, 2981–3002. [Google Scholar] [CrossRef]
110. Baño-Medina, J.; Manzanas, R.; Gutiérrez, J. M. Configuration and intercomparison of deep learning neural models for statistical downscaling. Geosci. Model Dev. 2020, 13, 2109–2124. [Google Scholar] [CrossRef]
111. LeCun, Y. Title, Journal, Year.
112. Baño-Medina, J.; Manzanas, R.; Gutiérrez, J. M. Configuration and intercomparison of deep learning neural models for statistical downscaling. Geosci. Model Dev. 2020, 13, 2109–2124. [Google Scholar] [CrossRef]
113. Dee, D. P.; Uppala, S. M.; Simmons, A. J.; et al. The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Q. J. Roy. Meteor. Soc. 2011, 137, 553–597. [Google Scholar] [CrossRef]
114. Cornes, R. C.; van der Schrier, G.; van den Besselaar, E. J. M.; Jones, P. D. An Ensemble Version of the E-OBS Temperature and Precipitation Data Sets. J. Geophys. Res.-Atmos. 2018, 123, 9391–9409. [Google Scholar] [CrossRef]
115. Baño-Medina, J.; Manzanas, R.; Gutiérrez, J. M. On the suitability of deep convolutional neural networks for continentalwide downscaling of climate change projections. Clim. Dynam. 2021, 57, 1–11. [Google Scholar] [CrossRef]
116. Maraun, D.; Widmann, M.; Gutiérrez, J. M.; et al. VALUE: A framework to validate downscaling approaches for climate change studies. Earths Future 2015, 3, 1–14. [Google Scholar] [CrossRef]
117. Vrac, M.; Ayar, P. Influence of Bias Correcting Predictors on Statistical Downscaling Models. J. Appl. Meteorol. Clim. 2016, 56, 5–26. [Google Scholar] [CrossRef]
118. Williams, P. M.: Modelling Seasonality and Trends in Daily Rainfall Data, in: Advances in Neural Information Processing Systems 10, MIT Press, Proceedings of the Neural Information Processing Systems (NIPS), 985–991, ISBN 0-262-10076-2, 1998.
119. Cannon, A. J. Probabilistic Multisite Precipitation Downscaling by an Expanded Bernoulli–Gamma Density Network. J. Hydrometeorol. 2008, 9, 1284–1300. [Google Scholar] [CrossRef]
120. Schoof, J. T.; Pryor, S. C. Downscaling temperature and precipitation: a comparison of regression-based methods and artificial neural networks. Int. J. Climatol. 2001, 21, 773–790. [Google Scholar] [CrossRef]
121. Maraun, D.; Widmann, M. Statistical Downscaling and Bias Correction for Climate Research; Cambridge University Press: Online, 2018; ISBN 9781107588783. [Google Scholar]
122. Vrac, M.; Stein, M.; Hayhoe, K.; Liang, X.-Z. A general method for validating statistical downscaling methods under future climate change. Geophys. Res. Lett. 2007, L18701, L18701. [Google Scholar] [CrossRef]
123. San-Martín, D.; Manzanas, R.; Brands, S.; Herrera, S.; Gutiérrez, J. M. Reassessing Model Uncertainty for Regional Projections of Precipitation with an Ensemble of Statistical Downscaling Methods. J. Climate 2017, J. Climate, 203–223. [Google Scholar] [CrossRef]
124. Quesada-Chacón, D.; Barfus, K.; Bernhofer, C. Climate change projections and extremes for Costa Rica using tailored predictors from CORDEX model output through statistical downscaling with artificial neural networks. Int. J. Climatol. 2021, 41, 211–232. [Google Scholar] [CrossRef]