This paper presents machine learning methods for approximate solutions of reaction-diffusion equations with multivalued interaction functions. This approach addresses the challenge of finding all possible solutions for such equations, which often lack uniqueness. The proposed method utilizes physics-informed neural networks (PINNs) to approximate generalized solutions.
Keywords:
Subject: Computer Science and Mathematics - Computational Mathematics
1. Introduction
In this paper we establish machine learning methods for approximate solutions of classes of reaction-diffusion equations with multivalued interaction functions allowing for non-unique solutions of the Cauchy problem. The relevance of this problem is primarily due to the lack of methods for finding all solutions for such mathematical objects. Therefore, there is an expectation that another approximate method will provide us with yet another solution for these problems. In addition, methods for approximate solutions of nonlinear systems with partial derivatives without uniqueness are mostly theoretical and are used primarily in qualitative research [1,2]. The availability of computational power for parallel computations and the creation of open-source software libraries such as PyTorch [3] have stimulated a new wave of development in IT and artificial intelligence methods. Sample-based methods for approximate solutions of such problems were first proposed in [4]. To date, such systems with smooth nonlinearities have been qualitatively and numerically studied. There is a need to develop a methodology for approximating generalized solutions of nonlinear differential-operator systems without uniqueness using recurrent neural networks, sample-based methods, and variations of the Monte Carlo method.
Let and be a sufficiently smooth function. We consider the problem:
with initial conditions:
where
For a fixed let be a bounded domain with sufficiently smooth boundary and According to [1] (see the book and references therein), there exists a weak solution with of Problem (1)–(2) in the following sense:
for all where be a measurable function such that
Such inclusions with multivalued nonlinearities appear in problems of climatology (Budyko-Sellers Model), chemical kinetics (Belousov-Zhabotinsky equations), biology (Lotka–Volterra systems with diffusion), quantum mechanics (FitzHugh–Nagumo system), engineering and medicine (several syntheses and impulse control problems); see [1,2] and references therein.
The main goal of this paper is to develop an algorithm for approximation of solutions for classes of reaction-diffusion equations with multivalued interaction functions allowing for non-unique solutions of the Cauchy problem (1)–(2) via the so-called physics-informed neural networks (PINNs); [5,6,7] and references therein.
2. Methodology of Approximate Solutions for Reaction-Diffusion Equations with Multivalued Interaction Functions
Fix an arbitrary and a sufficiently smooth function We approximate the function f by the following Lipschitz functions:
For a fixed consider the problem:
with initial conditions:
According to [2] and references therein, for each Problem (7)–(8) has an unique solution Moreover, [8] implies that each convergent subsequence of corresponding solutions to Problem (7)–(8) weakly converges to a solution u of Problem (1)–(2) in the space
endowed with the standard graph norm, where is a bounded domain with sufficiently smooth boundary and
Thus, the first step of the algorithm is to replace the function f in Problem (1)–(2) with considering Problem (7)–(8) for sufficiently large
Let us now consider Problem (7)–(8) for sufficiently large Theorem 16.1.1 from [5] allows us to reformulate Problem (7)–(8) as an infinite dimensional stochastic optimization problem over a certain function space. More exactly, let , let be a probability space, let and be independent random variables. Assume for all , that
Note that be Lipschitz continuous, and let satisfy for all that
Theorem 16.1.1 from [5] implies that the following two statements are equivalent:
Thus, the second step of the algorithm is to reduce the regularized Problem (7)–(8) to the infinite dimensional stochastic optimization problem in
However, due to its infinite dimensionality, the optimization problem (10) is not yet suitable for numerical computations. Therefore, we apply the third step, the so-called Deep Galerkin Method (DGM) [9], that is, we transform this infinite dimensional stochastic optimization problem into a finite dimensional one by incorporating artificial neural networks (ANNs); see [5,9] and references therein. Let be differentiable, let satisfy , and let satisfy for all that
where is the d-dimensional version of a function that is,
is the function which satisfies for all , with that
for each satisfying and for a function we denote by the realization function of the fully-connected feedforward artificial neural network associated to with layers with dimensions and activation functions defined as:
for all and for each satisfying the affine function from to associated to is defined as
for all
The final step in the derivation involves approximating the minimizer of using stochastic gradient descent optimization methods [5]. Let , , for each let and be random variables. Let for each
Let is defined as
for each and let satisfy for all that
Ultimately, for sufficiently large the realization is chosen as an approximation:
of the unknown solution u of (1)–(2) in the space W defined in (9).
So, the following theorem is justified.
Theorem1.
Let and Then the sequence of defined in (13)–(14) has an accumulation point in the weak topology of W defined in (9). Moreover, each partial limit of the sequence in hands is weakly converges in W to the solution of Problem (1)–(2) in the sense of (4)–(5).
Proof.
According to Steps 1–4 above, to derive PINNs, we approximate u in the space W defined in (9) by a deep ANN with parameters and minimize the empirical risk associated to over the parameter space . More precisely, we approximate the solution u of (1)–(2) by where
for a suitable choice of training data . Here denotes the number of training samples and the pairs , denote the realizations of the random variables X and T.
Analogously, to derive DGMs, we approximate u by a deep Galerkin method (DGM) with parameters and minimize the empirical risk associated to over the parameter space More precisely, we approximate the solution u of (1)–(2) by where
for a suitable choice of training data . Here denotes the number of training samples and the pairs , denote the realizations of the random variables X and T. □
The empirical risk minimization problems for PINNs and DGMs are typically solved using SGD or variants thereof, such as Adam [5]. The gradients of the empirical risk with respect to the parameters can be computed efficiently using automatic differentiation, which is commonly available in deep learning frameworks such as TensorFlow and PyTorch. We provide implementation details and numerical simulations for PINNs and DGMs in the next section.
3. Numerical Implementation
Let us present a straightforward implementation of the method as detailed in the previous Section for approximating a solution of Problem (1)–(2) with the initial condition where
Let This implementation follows the original proposal by [6], where realizations of the random variable are first chosen. Here, T is uniformly distributed over and X follows a normal distribution in with mean and covariance A fully connected feed-forward ANN with 4 hidden layers, each containing 50 neurons, and employing the Swish activation function is then trained. The training process uses batches of size sampled from the preselected realizations of Optimization is carried out using the Adam SGD method. A plot of the resulting approximation of the solution u after training steps is shown in Figure 1.
4. Conclusions
In this paper, we presented a novel machine learning methodology for approximating solutions to reaction-diffusion equations with multivalued interaction functions, a class of equations characterized by non-unique solutions. The proposed approach leverages the power of physics-informed neural networks (PINNs) to provide approximate solutions, addressing the need for new methods in this domain.
Our methodology consists of four key steps:
Approximation of the Interaction Function: We replaced the multivalued interaction function with a sequence of Lipschitz continuous functions, ensuring the problem becomes well-posed.
Formulation of the Optimization Problem: The regularized problem was reformulated as an infinite-dimensional stochastic optimization problem.
Application of Deep Galerkin Method (DGM): We transformed the infinite-dimensional problem into a finite-dimensional one by incorporating artificial neural networks (ANNs).
Optimization and Approximation: Using stochastic gradient descent (SGD) optimization methods, we approximated the minimizer of the empirical risk, yielding an approximation of the unknown solution.
The numerical implementation demonstrated the effectiveness of the proposed method. We used a fully connected feed-forward ANN to approximate the solution of a reaction-diffusion equation with specific initial conditions. The results showed that the PINN method could approximate solutions accurately, as evidenced by the visual plots.
The key contributions of this paper are as follows:
Development of a Machine Learning Framework: We established a robust framework using PINNs to tackle reaction-diffusion equations with multivalued interaction functions.
Handling Non-Uniqueness: Our method addresses the challenge of non-unique solutions, providing a practical tool for approximating generalized solutions.
Numerical Validation: We provided a detailed implementation and numerical validation, demonstrating the practical applicability of the proposed approach.
Future work could explore the extension of this methodology to other classes of partial differential equations with multivalued interaction functions, as well as further optimization and refinement of the neural network architectures used in the approximation process. The integration of more advanced machine learning techniques and the exploration of their impact on the accuracy and efficiency of the solutions also present promising avenues for research.
Author Contributions
All the authors contributed equally to this work.
Funding
“This research was funded by EIT Manufacturing asbl, 0123U103025, grant: “EuroSpaceHub - increasing the transfer of space innovations and technologies by bringing together the scientific community, industry and startups in the space industry”. The second and the third authors were partially supported by NRFU project No. 2023.03/0074 “Infinite-dimensional evolutionary equations with multivalued and stochastic dynamics”.
Institutional Review Board Statement
The authors have nothing to declare.
Informed Consent Statement
The authors have nothing to declare.
Data Availability Statement
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Conflicts of Interest
The author have no relevant financial or non-financial interests to disclose.
References
Zgurovsky, M.Z.; Mel’nik, V.S.; Kasyanov, P.O. Evolution Inclusions and Variation Inequalities for Earth Data Processing I: Operator Inclusions and Variation Inequalities for Earth Data Processing; Vol. 24, Springer Science & Business Media, 2010.
Zgurovsky, M.Z.; Kasyanov, P.O. Qualitative and quantitative analysis of nonlinear systems; Springer, 2018.
Paszke, A.; Sam, G.; Chintala.; Soumith.; Chanan, G. PyTorch, 2016. Accessed on June 5, 2024.
Rust, J. Using randomization to break the curse of dimensionality. Econometrica: Journal of the Econometric Society1997, pp. 487–516.
Jentzen, A.; Kuckuck, B.; von Wurstemberger, P. Mathematical introduction to deep learning: methods, implementations, and theory. arXiv preprint arXiv:2310.203602023.
Raissi, M.; Perdikaris, P.; Karniadakis, G.E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics2019, 378, 686–707. [Google Scholar] [CrossRef]
Beck, C.; Hutzenthaler, M.; Jentzen, A.; Kuckuck, B. An overview on deep learning-based approximation methods for partial differential equations. Discrete and Continuous Dynamical Systems-B2023, 28, 3697–3746. [Google Scholar] [CrossRef]
Zgurovsky, M.Z.; Kasyanov, P.O.; Kapustyan, O.V.; Valero, J.; Zadoianchuk, N.V. Evolution Inclusions and Variation Inequalities for Earth Data Processing III: Long-Time Behavior of Evolution Inclusions Solutions in Earth Data Analysis; Vol. 27, Springer Science & Business Media, 2012.
Sirignano, J.; Spiliopoulos, K. DGM: A deep learning algorithm for solving partial differential equations. Journal of computational physics2018, 375, 1339–1364. [Google Scholar] [CrossRef]
Figure 1.
Plots for the functions where and is an approximation of the solution u of Problem (1)–(2) with where is defined in (15), computed by means of the PINN method as implemented in Source code 1.
Figure 1.
Plots for the functions where and is an approximation of the solution u of Problem (1)–(2) with where is defined in (15), computed by means of the PINN method as implemented in Source code 1.
Listing 1.
Modified version of sourse code from Section 16.3 of 5.
Listing 1.
Modified version of sourse code from Section 16.3 of 5.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.