Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations. II: Illustrative Application to Heat and Energy Transfer in the Nordheim-Fuchs Phenomenological Model for Reactor Safety

Preprint

Article

Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations. II: Illustrative Application to Heat and Energy Transfer in the Nordheim-Fuchs Phenomenological Model for Reactor Safety

Altmetrics

Downloads

Views

Comments

A peer-reviewed article of this preprint also exists.

Dan Gabriel Cacuci^*

This version is not peer-reviewed

Submitted:

14 October 2024

Posted:

15 October 2024

You are already at the latest version

Alerts

Abstract

This work presents an illustrative application of the newly developed “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (2nd-FASAM-NODE)” methodology to determine most efficiently the exact expressions of the first- and second-order sensitivities of NODE-decoder responses to the neural net’s underlying parameters (weights and initial conditions). The application of the 2nd-FASAM-NODE methodology will be illustrated using the Nordheim-Fuchs phenomenological model for reactor safety, which describes a short-time self-limiting power transient in a nuclear reactor system having a negative temperature coefficient in which a large amount of reactivity is suddenly inserted. The representative model responses that will be analyzed in this work include the model’s the time-dependent total energy released, neutron flux, temperature and thermal conductivity. The 2nd-FASAM-NODE methodology yields the exact expressions of the first-order sensitivities of these decoder responses with respect to the underlying uncertain model parameters and initial conditions, requiring just a single large-scale computation per response. Furthermore, 2nd-FASAM-NODE methodology yields the exact expressions of the second-order sensitivities of a model response requiring as few large-scale computations as there are features/functions of model parameters, thereby demonstrating its unsurpassed efficiency for performing sensitivity analysis of NODE-nets.

Keywords:

Subject: Engineering - Safety, Risk, Reliability and Quality

1. Introduction

The mathematical/theoretical framework underlying the “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (2nd-FASAM-NODE)” has been presented in the accompanying Part 1 [1]. In this work, the application of the newly developed 2nd-FASAM-NODE methodology to determine most efficiently the exact expressions of second-order sensitivities of decoder responses to the neural net’s underlying parameters (weights and initial conditions) will be illustrated using the Nordheim-Fuchs phenomenological model for reactor safety [2,3]. This phenomenological model describes a short-time self-limiting power transient in a nuclear reactor system having a negative temperature coefficient in which a large amount of reactivity is suddenly inserted, either intentionally or by accident. The Nordheim-Fuchs model responses that will be analyzed in this work include model’s state functions/variables (namely: the time-dependent total energy released per cm³, the reactor’s time-dependent temperature and the reactor’s time dependent neutron flux), all of which are subject to uncertainties stemming from the model’s uncertain parameters. The application of the newly developed 2nd-FASAM-NODE methodology will also be illustrated by considering the reactor’s time-dependent thermal conductivity, which is a representative model response involving decoder weights that are also subject to uncertainties.

This work is structured as follows: Section 2 illustrates the application of the 2nd-FASAM-NODE methodology [1] to compute most efficiently the exact expressions of the first-order sensitivities of the model state functions and thermal conductivity. It is shown that the computation of the first-order sensitivities requires just a single large-scale computation per response. It is also shown that the number of large-scale computations can be reduced by identifying subsystems within the NODE-structure which could be solved independently of each other. For example, the Nordheim-Fuchs NODE-model can be decoupled into three equations, one for each of the dependent variables, which can be solved independently of each other. It is shown that the number of large-scale computations for determining the 1st-order (and, subsequently, the 2nd-order) sensitivities is greatly reduced when the re-structured NODE-equations can be solved independently of each other.

Section 3 presents illustrative applications of the 2nd-FASAM-NODE methodology to compute the second-order sensitivities of the Nordheim-Fuchs model’s dependent/state variables with respect to the underlying parameters. The computation of the second-order sensitivities by applying the 2nd-FASAM-NODE to the original NODE-structure is presented in Section 3.1. The computation of the second-order sensitivities by applying the 2nd-FASAM-NODE to the decoupled NODE-structure is presented in Section 3.2, illustrating the significant computational advantages gained by using decoupled subsystems, whenever possible.

Section 4 presents the application of the 2nd-FASAM-NODE methodology to compute second-order sensitivities of the thermal conductivity in the Nordheim-Fuchs model, which is a representative model response involving decoder-weights subject to uncertainties. These second-order sensitivities are computed in Section 4.1 by applying the 2nd-FASAM-NODE methodology to the original NODE-structure, and are subsequently computed alternatively, in Section 4.2, by applying the 2nd-FASAM-NODE methodology to the decoupled NODE-structure. It is shown that using a decoupled structure, whenever possible, is even more advantageous for the computation of second-order sensitivities than for the computation of the first-order ones. The discussion presented in Section 5 concludes this work, highlighting the salient features and unparalleled capabilities of the 2nd-FASAM-NODE methodology for computing first- and second-order sensitivities of decoder response to uncertain NODE-parameters and initial conditions.

2. Illustrative Application of the First-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (1st-FASAM-NODE) to the Nordheim-Fuchs Reactor Dynamics/Safety Model

Section 2.1 presents the Nordheim-Fuchs phenomenological model formulated in NODE-format. Section 2.2 presents a representative application of the 1st-FASAM-NODE to compute most efficiently the exact expressions of the first-order sensitivities of model state functions, while Section 2.3 presents a representative application of the 1st-FASAM-NODE to the exact expressions of first-order sensitivities of a typical response involving decoder-weights. Section 2.4 compares the alternative ways of applying the 1st-FASAM-NODE to minimize the number of computations for evaluating the exact expressions of first-order sensitivities of model responses to model parameters by possibly identifying subsystems, within the NODE-structure, that contain as few dependent variables as possible, and which could be solved independently of each other. For example, the Nordheim-Fuchs model can be decoupled into three equations, one equation for each of the dependent variables, which can be solved independently of each other.

2.1. NODE-Modeling of the Nordheim-Fuchs Reactor Dynamics/Safety Phenomenological Model

The mathematical modeling of a NODE-net was introduced in [4]. This modeling was generalized in [1] by introducing the concept of “features of primary model parameters/weights”, so that the generalized representation of a NODE-network that comprises such “features” is provided by the following system of “augmented” equations:

\frac{d h (t)}{d t} = f [h (t); F (θ); t], t > 0

, (1)

h (t_{0}) = h_{e} (x, w), a t t = t_{0}

, (2)

r (t_{f}) = h_{d} [h (t_{f}); φ], a t t = t_{f}

, (3)

where:

(i): the $T H$ -dimensional vector-valued function $h (t) ≜ {[h_{1} (t), ..., h_{T H} (t)]}^{†}$ represents the hidden/latent neural networks; in this work, all vectors are considered to be column-vectors, and transposition is denoted by using the dagger “ $†$ ” superscript;
(ii): the $T H$ -dimensional vector-valued nonlinear function $f [h (t); F (θ); t] ≜ {[f_{1} (h; F (θ); t), ..., f_{T H} (h; F (θ); t)]}^{†}$ models the dynamics of the latent neurons;
(iii): the components of the vector $θ ≜ {[θ_{1}, ..., θ_{T W}]}^{†}$ represents learnable scalar adjustable weights, which are considered to be the “primary model parameters;” where $T W$ denotes the total number of adjustable weights in all of the latent neural nets;
(iv): the components of the vector-valued function $F (θ) ≜ {[F_{1} (θ), ..., F_{T F} (θ)]}^{†}$ represent the ”feature” functions of the respective weights, which the quantity $T F \leq T W$ denotes the “total number of feature/functions of the primary model parameters” comprised in the NODE;
(v): the $T H$ -dimensional vector-valued function $h_{e} (x, w) ≜ {\{h_{1}^{e} (x, w), ..., h_{T H}^{e} (x, w)\}}^{†}$ represents the “encoder” which is characterized by “inputs” $x ≜ {[x_{1}, ..., x_{T I}]}^{†}$ and “learnable” scalar adjustable weights $w ≜ {[w_{1}, ..., w_{T E W}]}^{†}$ , where $T I$ denotes the total number of “inputs” and $T E W$ denotes the total number of “learnable encoder weights” that define the encoder;
(vi): the $T R$ -dimensional vector-valued function $r (t_{f}) ≜ {\{r_{1} [h (t_{f}); φ], ..., r_{T R} [h (t_{f}); φ]\}}^{†} = h_{d} [h (t_{f}); φ]$ represents the vector of “system responses; (vii) the vector-valued function $h_{d} [h (t_{f}); φ] ≜ {\{h_{1}^{d} [h (t_{f}); φ], ..., h_{T R}^{d} [h (t_{f}); φ]\}}^{†}$ represents the “decoder” with learnable scalar adjustable weights, which are represented by the components of the vector $φ ≜ {[φ_{1}, ..., φ_{T D}]}^{†}$ , where $T D$ denotes the total number of adjustable weights that characterize the decoder.

The Nordheim-Fuchs phenomenological model comprises the following balance/conservation equations:

1.: The time-dependent neutron balance (point kinetics) equation for the neutron flux $ψ (t)$ :

\frac{d ψ (t)}{d t} = \frac{ρ (t) - 1}{l_{p}} ψ (t), t > 0

, (4)

ψ (0) = ψ_{0}, t = 0

, (5)

where

l_{p}

denotes the prompt-neutron lifetime,

ρ (t)

denotes the reactor’s multiplication factor, and

ψ_{0}

denotes the initial (i.e., extant flux) prior to initiating the transient at time

t = 0

2.: The energy production equation:

E (t) = γ Σ_{f} \int_{0}^{t} ψ (x) d x

, (6)

where

γ

denotes the recoverable energy per fission;

Σ_{f} ≜ σ_{f} N_{f}

denotes the reactor’s effective macroscopic fission cross section, where

σ_{f}

denotes the reactor’s equivalent microscopic fission cross section while

N_{f}

denotes the reactor’s equivalent atomic number density.

3.: The energy conservation equation:

c_{p} [T (t) - T_{0}] = E (t)

, (7)

where

E (t)

denotes the total energy released (per cm³) at time

t

in the reactor after the onset of reactivity change;

c_{p}

denotes the specific heat (per cm³) of the reactor.

4.: The reactivity-temperature feedback equation: $ρ (t) = ρ_{0} - α_{T} ρ_{0} [T (t) - T_{0}]$ , where $ρ_{0} ≜ ρ (0) \geq 1$ denotes the changed multiplication factor following the reactivity insertion at $t = 0$ ; $α_{T}$ denotes the magnitude of the negative temperature coefficient; $T (t)$ denotes the reactor’s temperature; $T_{0}$ denotes the reactor’s initial temperature at time $t = 0$ . This work will consider the special case of a “prompt critical transient” which occurs when $ρ_{0} = 1$ and the reactor becomes prompt critical after the reactivity insertion. In this particular case, the reactivity-temperature feedback equation takes on the following particular form:

ρ (t) = 1 - α_{T} [T (t) - T_{0}]

. (8)

Equations (4)‒(8) can be transformed into the following system of nonlinear differential equations written in NODE-format:

\frac{d ψ (t)}{d t} = - \frac{α_{T}}{l_{p} c_{p}} E (t) ψ (t), ψ (0) = ψ_{0},

(9)

\frac{d E (t)}{d t} = γ σ_{f} N_{f} ψ (t), E (0) = 0

, (10)

\frac{d T (t)}{d t} = \frac{γ σ_{f} N_{f}}{c_{p}} ψ (t), T (0) = T_{0}

. (11)

As detailed in [5], the Nordheim-Fuchs model described by Eqs. (9)‒(11) can be solved analytically to obtain the following exact closed-form expression for the state functions

ψ (t)

E (t)

, and

T (t)

, which can be used for subsequent verification of all expressions of sensitivities:

E (t) = K_{1} (α) \tanh [t K_{2} (α)]

, (12)

ψ (t) = ψ_{0} \{1 - \tanh^{2} [t K_{2} (α)]\} = \frac{ψ_{0}}{\cosh^{2} [t K_{2} (α)]}

. (13)

T (t) = T_{0} + \frac{K_{1} (α)}{c_{p}} \tanh [t K_{2} (α)]

. (14)

where:

K_{1} (α) ≜ {[\frac{2 ψ_{0} γ σ_{f} N_{f} l_{p} c_{p}}{α_{T}}]}^{1 / 2}; K_{2} (α) ≜ {[\frac{α_{T} ψ_{0} γ σ_{f} N_{f}}{2 l_{p} c_{p}}]}^{1 / 2}

. (15)

Comparing the structure of the Nordheim-Fuchs model, cf. Eqs. (9)‒(11), to the generic structure of a NODE defined by Eqs. (1)‒(3) indicates the following correspondences for the weights/parameters of the hidden/latent units:

\begin{array}{l} θ ≜ {[θ_{1}, ..., θ_{T W}]}^{†} ≜ {(α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f})}^{†}; ​​ T W = 6; \\ x ≜ {[x_{1}, x_{2}, x_{3}]}^{†} ≜ {(ψ_{0}, 0, T_{0})}^{†}; T I = 3 . \end{array}

(16)

The precise values of the components of the vectors

θ

and

x

remain unknown even after having trained the NODE, since the actual values of the parameters underlying the Nordheim-Fuchs model are experimentally-measured and are thus subject to uncertainties. However, the optimal parameter values obtained after having trained the NODE become the known nominal values of these parameters, and are considered to be exactly reproducible by the “trained” NODE; these nominal values will be denoted using a superscript “zero,” as follows:

θ^{0} ≜ {[θ_{1}^{0}, ..., θ_{6}^{0}]}^{†} ≜ {(α_{T}^{0}, l_{p}^{0}, c_{p}^{0}, γ^{0}, σ_{f}^{0}, N_{f}^{0})}^{†}; x^{0} ≜ {[x_{1}^{0}, x_{2}^{0}, x_{3}^{0}]}^{†} ≜ {[ψ_{0}^{0}, 0; T_{0}^{0}]}^{†}

. (17)

It is apparent from the structure of Eqs. (9)‒(11) that although there are several possibilities for choosing the components of the vector-valued function

F (θ) ≜ {[F_{1} (θ), ..., F_{T F} (θ)]}^{†}

, it is advantageous to define these components so as to minimize their number while incorporating all of the primary model parameters. A choice that satisfies these considerations is as follows:

F_{1} (θ) ≜ - \frac{α_{T}}{l_{p}}; F_{2} (θ) ≜ γ σ_{f} N_{f}; F_{3} (θ) ≜ \frac{1}{c_{p}}

. (18)

Using the definitions provided in Eq. (18), Eqs. (9)‒(11) can be re-written in the following NODE-format:

\frac{d h_{1} (t)}{d t} = F_{1} (θ) F_{3} (θ) h_{1} (t) h_{2} (t), h_{1} (0) = ψ_{0};

(19)

\frac{d h_{2} (t)}{d t} = F_{2} (θ) h_{1} (t), h_{2} (0) = 0;

(20)

\frac{d h_{3} (t)}{d t} = F_{2} (θ) F_{3} (θ) h_{1} (t), h_{3} (0) = T_{0} .

(21)

where:

h (t) ≜ {[h_{1} (t), ..., h_{T H} (t)]}^{†} ≜ {[ψ (t), E (t), T (t)]}^{†}; ​​​​   ​ T H = 3;

(22)

Since the exact values of the model parameters are unknown, the exact values of the functions

h (t) ≜ {[h_{1} (t), h_{2} (t), h_{3} (t)]}^{†} ≜ {[ψ (t), E (t), T (t)]}^{†}

are also unknown. However, the nominal values

h^{0} (t) ≜ {[h_{1}^{0} (t), h_{2}^{0} (t), h_{3}^{0} (t)]}^{†} ≜ {[ψ^{0} (t), E^{0} (t), T^{0} (t)]}^{†}

of these quantities (which are denoted using the superscript “zero”), are known after having solved Eqs. (19)‒(21) at the nominal values

(θ^{0}, x^{0})

The typical results of interest (called “model response”) for the Nordheim-Fuchs model are the values of the state functions at a “final-time” instance, denoted as

t = τ

, after the initiation at

t = 0

of the prompt-critical power transient. The most important response determined using the Nordheim-Fuchs model is the total energy (per cm³) released at the “final time” instance

t = τ

, which can be represented in “decoder” form as follows:

r (h) = E (τ) = \int_{0}^{τ} h_{2} (t) δ (t - τ) d t

, (23)

where

δ (t - τ)

denotes the Dirac-delta functional. The neutron flux

ψ (τ)

and the reactor’s temperature

T (τ)

at a “final time” instance

t = τ

can be represented by similar integrals.

A representative decoder-response that also involves “weights” is provided by the temperature-dependent thermal conductivity, denoted as

K (T; φ)

, of the conceptual reactor’s material. As a specific example, the material’s conductivity is considered to depend quadratically on the reactor’s temperature; the corresponding decoder-response can be represented by the following expression:

K (T; φ) ≜ \int_{0}^{τ} h_{4}^{d} [h (t); φ] δ (t - τ) d t; ​​​​​​ h_{4}^{d} [h (t); φ] ≜ φ_{0} + φ_{1} T (t) + φ_{2} T^{2} (t)

, (24)

where the scalars-valued components of

φ ≜ {[φ_{0}, φ_{1}, φ_{2}]}^{†}

are experimentally-determined quantities and are thus subject to uncertainties. Of course, the temperature

T (τ)

is also subject to uncertainties since it is the solution of Eqs. (19)‒(21), which themselves involve uncertain parameters.

2.2. Representative Application of the 1st-FASAM-NODE to Compute Most Efficiently the Exact Expressions of the First-Order Sensitivities of Model State Functions to Uncertain Parameters

The application of the 1st-FASAM-NODE methodology will be illustrated by considering the total energy (per cm³) released at the “final time” instance

t = τ

, defined in Eq. (23), as the NODE-response. The total first-order sensitivity of this response is provided by the first-order G-differential of Eq. (23), which is obtained, by definition, as follows:

δ E (τ) = {\{\frac{d}{d ε} \int_{0}^{τ} [h_{2}^{0} (t) + ε δ h_{2} (t)] δ (t - τ) d t\}}_{ε = 0} = \int_{0}^{τ} δ h_{2} (t) δ (t - τ) d t

, (25)

where the variation

δ h_{2} (t)

is the solution of the system obtained by taking the first-order Gateaux- (G-) variation of the original system of Eqs. (19)‒(21), which yields:

\frac{d}{d t} δ h_{1} (t) = [(δ F_{1}) F_{3} + F_{1} (δ F_{3})] h_{1} (t) h_{2} (t) + F_{1} F_{3} [δ h_{1} (t) h_{2} (t) + h_{1} (t) δ h_{2} (t)]

, (26)

\frac{d}{d t} δ h_{2} (t) = (δ F_{2}) h_{1} (t) + F_{2} δ h_{1} (t)

, (27)

\frac{d}{d t} δ h_{3} (t) = [(δ F_{2}) F_{3} + F_{2} (δ F_{3})] h_{1} (t) + (F_{2} F_{3}) δ h_{1} (t)

, (28)

δ h_{1} (0) = δ ψ_{0}; δ h_{2} (0) = 0; δ h_{3} (0) = δ T_{0} .

(29)

The system defined by Eqs. (26)‒(29) is called the “First-Level Variational Sensitivity System (1st-LVSS)”and can be conveniently written in matrix/vector form as follows:

\frac{d v^{(1)} (t)}{d t} = J v^{(1)} (t) + \frac{\partial f (h; F)}{\partial F} δ F

, (30)

v^{(1)} (0) ≜ {[δ h_{1} (0), δ h_{2} (0), δ h_{3} (0)]}^{†} = {[δ ψ_{0}, 0, δ T_{0}]}^{†}

. (31)

where the following definitions were used:

J ≜ \frac{\partial f (h; F)}{\partial h} ≜ (\begin{matrix} F_{1} F_{3} h_{2} (t) & F_{1} F_{3} h_{1} (t) & 0 \\ F_{2} & 0 & 0 \\ F_{2} F_{3} & 0 & 0 \end{matrix}); v^{(1)} (t) ≜ (\begin{matrix} δ h_{1} (t) \\ δ h_{2} (t) \\ δ h_{3} (t) \end{matrix});

(32)

\frac{\partial f (h; F)}{\partial F} ≜ (\begin{matrix} F_{3} h_{1} (t) h_{2} (t) & 0 & F_{1} h_{1} (t) h_{2} (t) \\ 0 & h_{1} (t) & 0 \\ 0 & F_{3} h_{1} (t) & F_{2} h_{1} (t) \end{matrix}); δ F ≜ (\begin{matrix} δ F_{1} \\ δ F_{2} \\ δ F_{3} \end{matrix}) .

(33)

It is evident that the 1st-LVSS, defined by Eqs. (30) and (31), would need to be solved repeatedly in order to compute the 1st-level variational function

v^{(1)} (t) ≜ {[δ h_{1} (t), δ h_{2} (t), δ h_{3} (t)]}^{†} ≜ {[δ ψ (t), δ E (t), δ T (t)]}^{†}

for every possible variations

δ θ ≜ {[δ θ_{1}, ..., δ θ_{T W}]}^{†} ≜ {(δ α_{T}, δ l_{p}, δ c_{p}, δ γ, δ σ_{f}, δ N_{f})}^{†}

in the model parameters and variations

v^{(1)} (0) ≜ {[δ h_{1} (0), δ h_{2} (0), δ h_{3} (0)]}^{†} = {[δ ψ_{0}, 0, δ T_{0}]}^{†}

in the “encoder” initial conditions. This computationally expensive path can be avoided by applying the concepts of the 1st- FASAM-NODE, as follows:

Consider that the 1st-level variational function $v^{(1)} (t) ≜ {[δ h_{1} (t), δ h_{2} (t), δ h_{3} (t)]}^{†} \in H_{1} (Ω_{t})$ , $Ω_{t} ≜ (0, τ)$ , is an element in a Hilbert space denoted as $H_{1} (Ω_{t})$ which comprises vector-elements of the form $χ^{(1)} (t) ≜ {[χ_{1}^{(1)} (t), χ_{2}^{(1)} (t), χ_{3}^{(1)} (t)]}^{†}$ and $η^{(1)} (t) ≜ {[η_{1}^{(1)} (t), η_{2}^{(1)} (t), η_{3}^{(1)} (t)]}^{†}$ , being endowed with an inner product ${〈χ^{(1)} (t), η^{(1)} (t)〉}_{1}$ defined as follows:

{〈χ^{(1)} (t), η^{(1)} (t)〉}_{1} ≜ \int_{t_{0}}^{τ} {[χ^{(1)} (t)]}^{†} η^{(1)} (t) d t = \sum_{i = 1}^{3} \int_{0}^{τ} χ_{i}^{(1)} (t) η_{i}^{(1)} (t) d t

. (34)

2.: Use Eq. (34) to form the inner product of Eq. (30) with a yet undefined function $a^{(1)} (t) ≜ {[a_{1}^{(1)} (t), a_{2}^{(1)} (t), a_{3}^{(1)} (t)]}^{†} \in H_{1} (Ω_{t})$ , to obtain the following relation:

\int_{0}^{τ} {[a^{(1)} (t)]}^{†} \frac{d v^{(1)} (t)}{d t} d t - \int_{0}^{τ} {[a^{(1)} (t)]}^{†} J v^{(1)} (t) d t = \int_{0}^{τ} {[a^{(1)} (t)]}^{†} \frac{\partial f (h; F)}{\partial F} δ F d t

. (35)

3.: Integrating by parts the term on the left-side of Eq. (35) and rearranging the terms inside the integrals leads to the following relation:

\begin{array}{l} \int_{0}^{τ} {[a^{(1)} (t)]}^{†} \frac{d v^{(1)} (t)}{d t} d t - \int_{0}^{τ} {[a^{(1)} (t)]}^{†} J v^{(1)} (t) d t \\ = {[a^{(1)} (τ)]}^{†} v^{(1)} (τ) - {[a^{(1)} (0)]}^{†} v^{(1)} (0) + \int_{0}^{t_{f}} {[v^{(1)} (t)]}^{†} A^{(1)} a^{(1)} (t) d t, \end{array}

(36)

where:

A^{(1)} a^{(1)} (t) ≜ - \frac{d a^{(1)} (t)}{d t} - J^{†} a^{(1)} (t)

, (37)

with

J^{†} ≜ {[\frac{\partial f (h; F)}{\partial h}]}^{†} = (\begin{matrix} F_{1} F_{3} h_{2} (t) & F_{2} & F_{2} F_{3} \\ F_{1} F_{3} h_{1} (t) & 0 & 0 \\ 0 & 0 & 0 \end{matrix}) .

(38)

4.: The definition of the function $a^{(1)} (t)$ is now completed by requiring that: (i) the G-differential $δ E (τ)$ defined in Eq. (25) be represented by the integral term on the right-side of Eq. (36); and (ii) the appearance of the unknown values of the components of $v^{(1)} (τ)$ be eliminated from appearing in Eq. (36). These requirements are satisfied by requiring the function $a^{(1)} (t) ≜ {[a_{1}^{(1)} (t), a_{2}^{(1)} (t), a_{3}^{(1)} (t)]}^{†} \in H_{1} (Ω_{t})$ to be the solution of the following “1st-Level Adjoint Sensitivity System (1st-LASS)”:

A^{(1)} a^{(1)} (t) ≜ - \frac{d a^{(1)} (t)}{d t} - J^{†} a^{(1)} (t) = {[0, δ (t - τ), 0]}^{†}

; (39)

a^{(1)} (τ) ≜ {[a_{1}^{(1)} (τ), a_{2}^{(1)} (τ), a_{3}^{(1)} (τ)]}^{†} = {[0, 0, 0]}^{†}

. (40)

In component form, Eq. (39) comprises the following relations written in NODE-format:

\frac{d a_{1}^{(1)} (t)}{d t} = - F_{1} F_{3} h_{2} (t) a_{1}^{(1)} (t) - F_{2} a_{2}^{(1)} (t) - F_{2} F_{3} a_{3}^{(1)} (t);

(41)

\frac{d a_{2}^{(1)} (t)}{d t} = - F_{1} F_{3} h_{1} (t) a_{1}^{(1)} (t) - δ (t - τ);

(42)

\frac{d a_{3}^{(1)} (t)}{d t} = 0.

(43)

It is important to note that the 1st-LASS is linear in the 1st-level adjoint sensitivity function

a^{(1)} (t) ≜ {[a_{1}^{(1)} (t), a_{2}^{(1)} (t), a_{3}^{(1)} (t)]}^{†}

. Furthermore, the 1st-LASS is independent of any parameter variations, so it needs to be solved only once. In particular, Eqs. (40)‒(43) imply that

a_{3}^{(1)} (t) \equiv 0

. The actual/explicit expressions of the components

a_{1}^{(1)} (t)

and

a_{2}^{(1)} (t)

a^{(1)} (t)

are unimportant for the purpose of the present illustrative/conceptual discussion.

5.: Using Eqs. (25) and (35) in Eq. (36) yields the following expression for the first G-differential $δ E (τ)$ :

\begin{array}{l} δ E (τ) = a_{1}^{(1)} (0) δ ψ_{0} + a_{3}^{(1)} (0) δ T_{0} + \int_{0}^{τ} {[a^{(1)} (t)]}^{†} \frac{\partial f (h; F)}{\partial F} δ F d t \\ ≜ \frac{\partial E (τ)}{\partial ψ_{0}} δ ψ_{0} + \frac{\partial E (τ)}{\partial T_{0}} δ T_{0} + \sum_{i = 1}^{3} \frac{\partial E (τ)}{\partial F_{i}} δ F_{i} . \end{array}

(44)

Expanding the integral term on the right-side of Eq. (44) while recalling that

a_{3}^{(1)} (t) \equiv 0

yields the following expressions for the first-order sensitivities of the response

E (τ)

\frac{\partial E (τ)}{\partial ψ_{0}} = a_{1}^{(1)} (0) = \int_{0}^{τ} a_{1}^{(1)} (t) δ (t) d t

; (45)

\frac{\partial E (τ)}{\partial T_{0}} = a_{3}^{(1)} (0) = 0

; (46)

\frac{\partial E (τ)}{\partial F_{1}} = F_{3} (θ) \int_{0}^{τ} a_{1}^{(1)} (t) h_{1} (t) h_{2} (t) d t

; (47)

\frac{\partial E (τ)}{\partial F_{2}} = \int_{0}^{τ} a_{2}^{(1)} (t) h_{1} (t) d t

; (48)

\frac{\partial E (τ)}{\partial F_{3}} = F_{1} (θ) \int_{0}^{τ} a_{1}^{(1)} (t) h_{1} (t) h_{2} (t) d t

. (49)

The expressions of the first-order sensitivities obtained in Eqs. (45)‒(49) are to be evaluated at the nominal values of the respective parameters and functions. The numerical values of these first-order sensitivities can be computed after having solved the 1st-LASS to obtain the components of the 1st-level adjoint sensitivity function

a^{(1)} (t) ≜ {[a_{1}^{(1)} (t), a_{2}^{(1)} (t), a_{3}^{(1)} (t)]}^{†}

The sensitivities of the response

E (τ)

with respect to the primary model parameters are obtained by using Eqs. (45)‒(49) in conjunction with the following “chain-rule” type relations:

\frac{\partial E (τ)}{\partial θ_{i}} = \sum_{j = 1}^{3} \frac{\partial E (τ)}{\partial F_{j}} \frac{\partial F_{j} (θ)}{\partial θ_{i}}; i = 1, .., T W = 6

. (50)

Of course, the first-order sensitivities of

E (τ)

with respect to the primary model parameters

θ ≜ {[θ_{1}, ..., θ_{T W}]}^{†} ≜ {(α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f})}^{†}

could have been computed directly, ab initio, using a single large-scale computation to solve the corresponding 1st-LASS, as presented in [5]. The advantage of using “features/functions of parameters” is minimal for computing 1st-order sensitivities but becomes increasingly important for computing second- and higher-order sensitivities. This fact will be demonstrated in Section 3 and Section 4, when computing second-order sensitivities of model responses to parameters.

It is important to note that for any other model response (“decoder”), only the source-terms on the right-side of Eq. (39) will be different. The left-side of the 1st-LASS defined by Eq. (39) and the “final-time” conditions in Eq. (40) will remain unchanged. Therefore, if several responses are of interest, it would be computationally convenient if the inverse of the operator

A^{(1)} (h; θ)

could be stored in order to be used repeatedly, for computing all of the 1st-level adjoint sensitivity functions that would correspond to the responses of interest. Furthermore, the expressions on the right-side of Eqs. (45)‒(49) will remain formally the same for computing the first-order sensitivities of any other response. Of course, the numerical values of the components of the 1st-level adjoint sensitivity function

a^{(1)} (t) ≜ {[a_{1}^{(1)} (t), a_{2}^{(1)} (t), a_{3}^{(1)} (t)]}^{†}

will be replaced by the numerical values of the 1st-level adjoint sensitivity function that will correspond to the response under consideration. Thus, although the respective 1st-level adjoint sensitivity functions differ according to the response under consideration, the quadrature-schemes needed to evaluate the integrals defining the respective sensitivities remain unchanged. Therefore, the same numerical procedures and/or neural nets can be used for computing the respective integrals that define the 1st-order sensitivities, while using the appropriate/corresponding values for the 1st-level adjoint sensitivity functions. If the response depends on parameters/weights, additional sensitivities will arise from the respective “direct-effect term.”

2.3. Representative Application of the 1st-FASAM-NODE to Compute Most Efficiently the Exact Expressions of First-Order Sensitivities of a Typical Response Involving Decoder-Weights

The application of the 1st-FASAM-NODE methodology to compute efficiently and exactly the first-order sensitivities of a representative NODE-decoder response involving decoder weights will be illustrated in this Subsection by considering the thermal conductivity response defined in Eq. (24). The first-order sensitivities of this response are obtained by applying the definition of the G-differential to Eq. (24), which yields the following expression:

\begin{array}{l} δ K (T^{0}; φ^{0}; δ T; δ φ) ≜ \{\frac{d}{d ε} \int_{0}^{τ} [(φ_{0}^{0} + ε δ φ_{0}) + (φ_{1}^{0} + ε δ φ_{1}) (T^{0} + ε δ T) \\ {+ (φ_{2}^{0} + ε δ φ_{2}) {(T^{0} + ε δ T)}^{2}] δ (t - τ) d t\}}_{ε = 0} \\ = {\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{d i r} + {\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{i n d} . \end{array}

(51)

In Eq. (51), the direct-effect term

{\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{d i r}

comprises variations

δ φ ≜ {[δ φ_{0}, δ φ_{1}, δ φ_{2}]}^{†}

in the weights defining the decoder and is defined as follows:

{\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{d i r} ≜ \int_{0}^{τ} [δ φ_{0} + δ φ_{1} T^{0} (t) + δ φ_{2} {(T^{0})}^{2}] δ (t - τ) d t

, (52)

while the indirect-effect term

{\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{i n d}

comprises the variations

δ T (t)

and is defined as follows:

{\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{i n d} ≜ \int_{0}^{τ} [φ_{1}^{0} + 2 φ_{2}^{0} T^{0} (t)] δ T (t) δ (t - τ) d t

. (53)

The direct-effect term defined in Eq. (52) yields the following first-order sensitivities with respect to the decoder’s weights, which can be computed immediately at the nominal value of the reactor temperature:

\frac{\partial K [T (τ)]}{\partial φ_{0}} = \int_{0}^{τ} δ (t - τ) d t ​ ​ = 1

; (54)

\frac{\partial K [T (τ)]}{\partial φ_{1}} = \int_{0}^{τ} T (t) δ (t - τ) d t = T (τ)

; (55)

\frac{\partial K [T (τ)]}{\partial φ_{2}} = \int_{0}^{τ} T^{2} (t) δ (t - τ) d t = T^{2} (τ)

. (56)

The sensitivities stemming from the indirect-effect term can be obtained by applying the principles underlying the 1st-FASAM-NODE methodology by following the same steps as outlined in items (1)‒(5) in Subsection 2.2, to obtain the following expression:

\begin{array}{l} {\{δ K (T^{0}; φ^{0}; δ T; δ φ)\}}_{i n d} = b_{1}^{(1)} (0) δ ψ_{0} + b_{3}^{(1)} (0) δ T_{0} + \int_{0}^{τ} {[b^{(1)} (t)]}^{†} \frac{\partial f (h; F)}{\partial F} δ F d t \\ ≜ \frac{\partial K (τ)}{\partial ψ_{0}} δ ψ_{0} + \frac{\partial K (τ)}{\partial T_{0}} δ T_{0} + \sum_{i = 1}^{3} \frac{\partial K (τ)}{\partial F_{i}} δ F_{i} . \end{array}

(57)

where the 1st-level adjoint sensitivity function

b^{(1)} (t) ≜ {[b_{1}^{(1)} (t), b_{2}^{(1)} (t), b_{3}^{(1)} (t)]}^{†}

is the solution of the following 1st-Level Adjoint Sensitivity System (1st-LASS):

A^{(1)} b^{(1)} (t) ≜ - \frac{d b^{(1)} (t)}{d t} - J^{†} b^{(1)} (t) = {\{0, 0, [φ_{1} + 2 φ_{2} T (t)] δ (t - τ)\}}^{†}

; (58)

b^{(1)} (τ) ≜ {[b_{1}^{(1)} (τ), b_{2}^{(1)} (τ), b_{3}^{(1)} (τ)]}^{†} = {[0, 0, 0]}^{†}

. (59)

The 1st-LASS defined by Eqs. (58) and (59) is solved at the nominal values for all parameters and original functions, but the superscript “zero” which would have denoted this fact has been omitted in order to simplify the notation. Expanding the vector-matrix product in the integral on the right-side of Eq. (57) and identifying the expressions that multiply the respective variations in the feature functions and initial conditions leads to the following expressions for the sensitivities of the response

K (τ)

\frac{\partial K (τ)}{\partial ψ_{0}} = b_{1}^{(1)} (0) = \int_{0}^{τ} b_{1}^{(1)} (t) δ (t) d t

; (60)

\frac{\partial K (τ)}{\partial T_{0}} = b_{3}^{(1)} (0) = \int_{0}^{τ} b_{3}^{(1)} (t) δ (t) d t

; (61)

\frac{\partial K (τ)}{\partial F_{1}} = F_{3} (θ) \int_{0}^{τ} h_{1} (t) h_{2} (t) b_{1}^{(1)} (t) d t

; (62)

\frac{\partial K (τ)}{\partial F_{2}} = \int_{0}^{τ} [b_{2}^{(1)} (t) + F_{3} (θ) b_{3}^{(1)} (t)] h_{1} (t) d t

; (63)

\frac{\partial K (τ)}{\partial F_{3}} = \int_{0}^{τ} [F_{1} (θ) b_{1}^{(1)} (t) h_{2} (t) + F_{2} (θ) b_{3}^{(1)} (t)] h_{1} (t) d t

. (64)

As indicated in Eqs. (60)‒(64), there are 5 first-order sensitivities of the response

K (τ)

with respect to the two non-zero initial conditions and the three components of the feature function. The first-order sensitivities with respect to the 6 primary parameters

α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f}

can be obtained by using Eqs. (60)‒(64) in conjunction with chain-rule relations similar to those presented in Eqs. (50).

2.4. Discussion: Minimizing the Number of Computations for Evaluating the Exact Expressions of First-Order Sensitivities of Model Responses to Model Parameters

It is evident that the 1st-FASAM-NODE is the most efficient methodology for computing first-order sensitivities whenever it is possible to identify features/functions of the model parameters in the underlying NODE-equations. Additional efficiency can be attained if the original system of NODE-equations could be decoupled into subsystems that contain as few dependent variables as possible, which could then be solved independently of each other. For the Nordheim-Fuchs model, for example, Eqs. (9)‒(11) can be decoupled into three equations, one equation for each of the dependent variables, which can be solved independently of each other. For example, the following equation has been obtained in [5] for the function

E (t)

in NODE-format:

\frac{d E (t)}{d t} = - \frac{α_{T}}{2 l_{p} c_{p}} E^{2} (t) + ψ_{0} γ σ_{f} N_{f}, E (0) = 0

. (65)

The “features/functions of parameters” in Eq. (65) can be conveniently chosen as follows:

\begin{array}{l} Φ_{1} (p) ≜ \frac{α_{T}}{2 l_{p} c_{p}}; Φ_{2} (p) ≜ ψ_{0} γ σ_{f} N_{f}; Φ (p) ≜ {[Φ_{1} (p), Φ_{2} (p)]}^{†}; \\ p ≜ {[p_{1}, ..., p_{7}]}^{†} ≜ {[α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f}, ψ_{0}]}^{†} . \end{array}

(66)

In terms of the “feature function”

Φ (p) ≜ {[Φ_{1} (p), Φ_{2} (p)]}^{†}

, Eq. (65) can alternatively be written as follows:

\frac{d E (t)}{d t} = - Φ_{1} (p) E^{2} (t) + Φ_{2} (p), E (0) = 0

. (67)

Of course, a specific NODE-model would need to be constructed to represent Eq. (67).

In terms of the feature function

Φ (p) ≜ {[Φ_{1} (p), Φ_{2} (p)]}^{†}

, the solution of Eq. (67) has the following form:

E (t) = {[\frac{Φ_{2} (p)}{Φ_{1} (p)}]}^{1 / 2} \tanh [t G (p)]; G (p) ≜ \sqrt{Φ_{1} (p) Φ_{2} (p)}

. (68)

The 1st-Level Variational Sensitivity System (1st-LVSS) for the variational function

δ E (t)

is obtained by applying the definition of the first-order G-differential to Eq. (67), which yields the following 1st-LVSS:

[\frac{d}{d t} + 2 Φ_{1} E (t)] δ E (t) = - δ Φ_{1} E^{2} (t) + δ Φ_{2}, t > 0

, (69)

δ E (0) = 0, t = 0.

(70)

The 1st-LVSS represented by Eq. (69) is to be solved at the nominal values for the parameters and the state function

E (t)

but the superscript “0” (which indicates “nominal values”) has been omitted to simplify the notation. Numerically, the 1st-LVSS would need to be solved anew for the various variations

δ F_{1}

δ F_{2}

, in the components of the feature function

Φ (p)

. This need for repeatedly solving the 1st-LVSS can be avoided by constructing the corresponding 1st-Level Adjoint Sensitivity System (1st-LASS). The Hilbert space appropriate for the construction of the 1st-LASS corresponding to Eq. (69) will be denoted as

H_{1, E} (Ω_{t})

, where the subscript “1” indicates “1st-level”, the subscript “E” indicates that this Hilbert space is constructed exclusively for the “energy-variable” and where

Ω_{t} ≜ (0, τ)

. The inner product appropriate for

H_{1, E} (Ω_{t})

between two functions

ω_{1} (t) \in H_{1, E} (Ω_{t})

and

ω_{2} (t) \in H_{1, E} (Ω_{t})

is denoted as

{〈ω_{1} (t), ω_{2} (t)〉}_{1, E}

and is defined as follows:

{〈ω_{1} (t), ω_{2} (t)〉}_{1, E} = \int_{0}^{τ} ω_{1} (t) ω_{2} (t) d t

. (71)

Using Eq. (71), the 1st-LASS is constructed by applying the same sequence of step as those leading to Eqs. (39)‒(44), to obtain the following expression for the G-differential

δ E (τ)

of the response

E (τ)

δ E (τ) = - (δ Φ_{1}) \int_{0}^{τ} e^{(1)} (t) E^{2} (t) d t + (δ Φ_{2}) \int_{0}^{τ} e^{(1)} (t) d t

, (72)

where the 1st-level adjoint sensitivity function

e^{(1)} (t)

is the solution of the following 1st-Level Adjoint Sensitivity System (1st-LASS):

[- \frac{d}{d t} + 2 Φ_{1} E (t)] e^{(1)} (t) = δ (t - τ), ​​​ t > 0

, (73)

e^{(1)} (τ) = 0, t = τ

. (74)

The 1st-LASS represented by Eqs. (73) and (74) is independent of variations in the feature functions (or parameters) so it would need to be solved only once, numerically. In the present case, the 1st-LASS can be solved analytically to obtain the following closed-form expression for the 1st-level adjoint sensitivity function

e^{(1)} (t)

e^{(1)} (t) = H (τ - t) {\{\frac{\cosh [t G (p)]}{\cosh [τ G (p)]}\}}^{2}

, (75)

where

H (t - τ)

denotes the Heaviside functional. It follows from Eqs. (72), (75) and (68) that the two sensitivities of the response

E (τ)

with respect to the two components of the feature function

Φ ≜ {(Φ_{1}, Φ_{2})}^{†}

have the following expressions:

\frac{\partial E (τ)}{\partial Φ_{1}} = - \int_{0}^{τ} e^{(1)} (t) E^{2} (t) d t = \frac{1}{2} {[\frac{Φ_{2} (p)}{Φ_{1} (p)}]}^{1 / 2} \{\frac{τ}{\cosh^{2} [τ G (p)]} - \frac{\tanh [τ G (p)]}{G (p)}\};

(76)

\frac{\partial E (τ)}{\partial Φ_{2}} = \int_{0}^{τ} e^{(1)} (t) d t = \frac{1}{2 G (p)} \tanh [τ G (p)] + \frac{τ}{2 \cosh^{2} [τ G (p)]}

. (77)

The above expressions are to be evaluated at the nominal parameter values but the superscript “zero” has been omitted, for simplicity. The expressions obtained in Eqs. (76) and (77) can be verified by differentiating the expression provided in Eq. (68), evaluated at a user-chosen time

t = τ

within the interval

0 < τ < \infty

The sensitivities of the response

E (τ)

with respect to the model parameters are obtained by using the general relationship:

\frac{\partial E (τ; Φ_{1}; Φ_{2})}{\partial p_{i}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1} (p)}{\partial p_{i}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2} (p)}{\partial p_{i}}; i = 1, ..., 7.

(78)

The explicit expressions for the specific sensitivities of the response

E (τ)

with respect to the parameters underlying the feature functions are obtained using Eq. (78) in conjunction with Eqs. (76) and (77) while recalling the definitions of the feature functions

Φ_{1} (p)

and

Φ_{2} (p)

defined in Eq. (66). The expressions of these first-order sensitivities are as follows:

\frac{\partial E (τ)}{\partial α_{T}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1}}{\partial α_{T}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial α_{T}} = \frac{1}{2 l_{p} c_{p}} \frac{\partial E (τ)}{\partial Φ_{1}}

; (79)

\frac{\partial E (τ)}{\partial l_{p}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1}}{\partial l_{p}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial l_{p}} = - \frac{α_{T}}{2 {(l_{p})}^{2} c_{p}} \frac{\partial E (τ)}{\partial Φ_{1}}

; (80)

\frac{\partial E (τ)}{\partial c_{p}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1}}{\partial c_{p}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial c_{p}} = - \frac{α_{T}}{2 {(c_{p})}^{2} l_{p}} \frac{\partial E (τ)}{\partial Φ_{1}}

; (81)

\frac{\partial E (τ)}{\partial γ} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1}}{\partial γ} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial γ} = ψ_{0} σ_{f} N_{f} \frac{\partial E (τ)}{\partial Φ_{2}}

; (82)

\frac{\partial E (τ)}{\partial σ_{f}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1}}{\partial σ_{f}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial σ_{f}} = ψ_{0} γ N_{f} \frac{\partial E (τ)}{\partial Φ_{2}}

; (83)

\frac{\partial E (τ)}{\partial N_{f}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1}}{\partial N_{f}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial N_{f}} = ψ_{0} γ σ_{f} \frac{\partial E (τ)}{\partial Φ_{2}}

. (84)

\frac{\partial E (τ)}{\partial ψ_{0}} = \frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{2}}{\partial ψ_{0}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2}}{\partial ψ_{0}} = γ σ_{f} N_{f} \frac{\partial E (τ)}{\partial Φ_{2}}

. (85)

Notably, the application of the 1st-FASAM-N requires just one “large-scale” computation to solve the 1st-LASS, cf. Eq. (73) and (74), which is a single ODE, to obtain the 1st-level adjoint function

e^{(1)} (t)

, which is a scalar-valued function. However, solving the 1st-LASS, comprising Eq. (73) and (74) requires the construction of a separate, albeit simpler, NODE. The 1st-level adjoint function

e^{(1)} (t)

is subsequently used for obtaining the two sensitivities of the response

E (τ)

with respect to the two components

Φ_{1} (p)

and

Φ_{2} (p)

of the feature function

Φ (p) ≜ {(Φ_{1}, Φ_{2})}^{†}

, which requires the evaluation of two integrals using quadrature formulas. Subsequently, all of the response sensitivities with respect to the model’s primary parameters are obtained analytically by using the chain-rule to differentiate the components of the feature function with respect to the underlying model parameters and initial conditions.

On the other hand, the direct computation of the sensitivities of the response with respect to the model parameters and initial conditions, the original NODE can be used to solve (backward in time) the 1st-LASS, which comprises a system of three coupled ODEs for obtaining the 1st-level adjoint function

a^{(1)} (t) ≜ {[a_{1}^{(1)} (t), a_{2}^{(1)} (t), a_{3}^{(1)} (t)]}^{†}

, which is a vector-valued function comprising three components. The respective vector-valued 1st-level adjoint function is subsequently used in performing six (rather than two, if the 1st-FASAM is used) integrals (quadrature) for obtaining the six sensitivities of the respective response with respect to the six model parameters.

After having obtained the 1st-order sensitivities of

E (τ)

, the 1st-order sensitivities of the temperature

T (τ)

at a time instance

t = τ

can be obtained exactly and most effectively by using the expressions provided in Eqs. (79)‒(85) in conjunction with the following relation obtained by differentiating the expression in Eq. (7) with respect to the parameters

p_{i}

i = 1, ..., 7

\frac{\partial T (τ)}{\partial p_{i}} = \frac{\partial (1 / c_{p})}{\partial p_{i}} E (τ) + \frac{1}{c_{p}} \frac{\partial E (τ)}{\partial p_{i}} .

(86)

Using Eqs. (9) and (65) yields the following relation:

ψ (t) = ψ_{0} - \frac{α_{T}}{2 l_{p} c_{p} γ σ_{f} N_{f}} E^{2} (t)

. (87)

The 1st-order sensitivities of the neutron flux

ψ (τ)

at a time instance

t = τ

can be obtained exactly and most effectively by using the 1st-order sensitivities of

E (τ)

, in conjunction with the following relation obtained by differentiating Eq. (87) with respect to the parameters

p_{i}

i = 1, ..., 7

\frac{\partial ψ (τ)}{\partial ψ_{0}} = 1; \frac{\partial ψ (τ)}{\partial p_{i}} = - E^{2} (τ) \frac{\partial}{\partial p_{i}} (\frac{α_{T}}{2 l_{p} c_{p} γ σ_{f} N_{f}}) - \frac{α_{T} E (τ)}{l_{p} c_{p} γ σ_{f} N_{f}} \frac{\partial E (τ)}{\partial p_{i}}

. (88)

After having obtained the 1st-order sensitivities of

T (τ)

, the 1st-order sensitivities of the thermal conductivity

K (τ)

can be obtained exactly, analytically and most efficiently by differentiating Eq. (24) with respect to the parameters

p_{i}

i = 1, ..., 7

. This procedure yields the following expressions:

\begin{array}{l} \frac{\partial K (τ)}{\partial φ_{0}} = 1; \frac{\partial K (τ)}{\partial φ_{1}} = T (τ); \frac{\partial K (τ)}{\partial φ_{2}} = T^{2} (τ); \\ \frac{\partial K (τ)}{\partial p_{i}} = φ_{0} + [φ_{1} + 2 φ_{2} T (τ)] \frac{\partial T (τ)}{\partial p_{i}} \\ = φ_{0} + [φ_{1} + 2 φ_{2} T (τ)] [\frac{\partial (1 / c_{p})}{\partial p_{i}} E (τ) + \frac{1}{c_{p}} \frac{\partial E (τ)}{\partial p_{i}}] . \end{array}

(89)

3. Illustrative Application of the 2nd-FASAM-NODE Methodology to Compute Second-Order Sensitivities of the Nordheim-Fuchs Model’s Dependent/State Variables with Respect to the Underlying Parameters

The computation of the second-order sensitivities of a dependent/state variable will be illustrated by using the response

E (τ)

as a paradigm example. The second-order sensitivities of this response are computed by using the fundamental definition of being “the first-order sensitivities of the first-order sensitivities.” In Subsection 3.1, the second-order sensitivities of

E (τ)

will be obtained from the NODE-equations derived in Subsection 2.2, which led to the expressions presented in Eqs. (45)‒(49). Alternatively, in Subsection 3.2, the second-order sensitivities of

E (τ)

will be obtained most efficiently from the first-order sensitivities with respect to the feature functions derived in Eqs. (76) and (77).

3.1. Computation of Second-Order Sensitivities of $E (τ)$ Using the Coupled NODE-Equations

The representative computation of second-order sensitivities of

E (τ)

can be illustrated by considering the expression obtained in Eq. (49). The G-differential of this expression is obtained, by definition, as follows:

\begin{array}{l} δ [\frac{\partial E (τ)}{\partial F_{3}}] = \{\frac{d}{d ε} (F_{1}^{0} + ε δ F_{1}) \int_{0}^{τ} [a_{1}^{(1, 0)} (t) + ε δ a_{1}^{(1)} (t)] [h_{1}^{0} (t) + ε δ h_{1} (t)] \\ {\times [h_{2}^{0} (t) + ε δ h_{2} (t)] d t\}}_{ε = 0} = δ {[\partial E (τ) / \partial F_{3}]}_{d i r} + δ {[\partial E (τ) / \partial F_{3}]}_{i n d}, \end{array}

(90)

where the direct-effect term

δ {[\partial E (τ) / \partial F_{3}]}_{d i r}

comprises parameter variations and is defined as follows:

δ {[\partial E (τ) / \partial F_{3}]}_{d i r} ≜ (δ F_{1}) \int_{0}^{τ} a_{1}^{(1)} (t) h_{2} (t) h_{1} (t) d t

, (91)

while the indirect-effect term

δ {[\partial E (τ) / \partial F_{3}]}_{i n d}

comprises variations in the state functions and is defined as follows:

\begin{array}{l} δ {[\partial E (τ) / \partial F_{3}]}_{i n d} ≜ F_{1} \int_{0}^{τ} \{[δ a_{1}^{(1)} (t) h_{2} (t) h_{1} (t)] + [δ h_{2} (t) a_{1}^{(1)} (t) h_{1} (t)] \\ + [δ h_{1} (t) a_{1}^{(1)} (t) h_{2} (t)]\} d t . \end{array}

(92)

The expressions of the direct-effect and indirect-effect terms obtained in Eqs. (91) and (92) are to be evaluated at the respective nominal values of parameters and functions but the superscript “zero,” which has been used in Eq. (90) to indicate this fact, has been omitted in order to simplify the notation.

The direct-effect term can be evaluated numerically at this time since all of the functions in Eq. (91) are available. The indirect-effect term can be evaluated only after having determined the variational functions

δ a^{(1)} (t) ≜ {[δ a_{1}^{(1)} (t), δ a_{2}^{(1)} (t), δ a_{3}^{(1)} (t)]}^{†}

and

v^{(1)} (t) ≜ {[δ h_{1} (t), δ h_{2} (t), δ h_{3} (t)]}^{†} ≜ {[δ ψ (t), δ E (t), δ T (t)]}^{†}

. The variational function

v^{(1)} (t)

is the solution of the 1st-LVSS defined by Eqs. (30) and (31). The variational function

δ a^{(1)} (t)

is the solution of the system of equations obtained by G-differentiating the 1st-LASS defined by Eqs. (39) and (40), which yields the following relations:

\begin{array}{l} \frac{d}{d t} δ a_{1}^{(1)} (t) = - δ a_{1}^{(1)} (t) F_{1} F_{3} h_{2} (t) - δ a_{2}^{(1)} (t) F_{2} - δ a_{3}^{(1)} (t) F_{2} F_{3} - δ h_{2} (t) F_{1} F_{3} a_{1}^{(1)} (t) \\ - [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{2} (t) a_{1}^{(1)} (t) - (δ F_{2}) a_{2}^{(1)} (t) - [(δ F_{2}) F_{3} + (δ F_{3}) F_{2}] a_{3}^{(1)} (t); \end{array}

(93)

\begin{array}{l} \frac{d}{d t} δ a_{2}^{(1)} (t) = - δ a_{1}^{(1)} (t) F_{1} F_{3} h_{1} (t) - δ h_{1} (t) F_{1} F_{3} a_{1}^{(1)} (t) \\ ​​ - [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{1} (t) a_{1}^{(1)} (t), \end{array}

(94)

\frac{d}{d t} δ a_{3}^{(1)} (t) = 0

. (95)

δ a^{(1)} (τ) ≜ {[δ a_{1}^{(1)} (τ), δ a_{2}^{(1)} (τ), δ a_{3}^{(1)} (τ)]}^{†} = {[0, 0, 0]}^{†}

(96)

As indicated by Eqs. (93)‒(96), the components of

δ a^{(1)} (t)

are connected to the components of

v^{(1)} (t)

. Therefore, the function

v^{(2)} (t) ≜ {[v^{(1)} (t), δ a^{(1)} (t)]}^{†}

will be the solution of the so-called “2nd-Level Variational Sensitivity System” (2nd-LVSS) obtained by concatenating Eqs. (93)‒(96) with Eqs. (26)‒(29). The function

v^{(2)} (t) ≜ {[v^{(1)} (t), δ a^{(1)} (t)]}^{†}

will be called the “2nd-level variational function.” The superscript “(2)” indicates “2nd-level.”

It is impractical to solve the 2nd-LVSS to compute each 2nd-level variational function

v^{(2)} (t) ≜ {[v^{(1)} (t), δ a^{(1)} (t)]}^{†}

that would correspond to every component of the variations

δ F

and

δ φ

. The need for computing

v^{(2)} (t)

can be circumvented by applying the principles of the 2nd-FASAM-NODE methodology [1], which comprises the following sequence of steps:

1.: Consider that $v^{(2)} (t) ≜ {[v^{(1)} (t), δ a^{(1)} (t)]}^{†}$ is an element in a Hilbert space denoted as $H_{2} (Ω_{t})$ , $Ω_{t} ≜ [0, τ]$ , comprising as elements 2-block vectors having the following structure: $χ^{(2)} ≜ {[χ_{1}^{(2)} (t), χ_{2}^{(2)} (t)]}^{†}$ , with $χ_{1}^{(2)} (t) ≜ {[χ_{1, 1}^{(2)} (t), χ_{1, 2}^{(2)} (t), χ_{1, 3}^{(2)} (t)]}^{†}$ and $χ_{2}^{(2)} (t) ≜ {[χ_{2, 1}^{(2)} (t), χ_{2, 2}^{(2)} (t), χ_{2, 3}^{(2)} (t)]}^{†}$ . The Hilbert space $H_{2} (Ω_{t})$ is considered to be endowed with an inner product denoted as ${〈χ^{(2)}, η^{(2)}〉}_{2}$ and defined as follows:

{〈χ^{(2)}, η^{(2)}〉}_{2} ≜ \sum_{i = 1}^{3} \int_{0}^{τ} χ_{1, i}^{(2)} (t) η_{1, i}^{(2)} (t) d t + \sum_{i = 1}^{3} \int_{0}^{τ} χ_{2, i}^{(2)} (t) η_{2, i}^{(2)} (t) d t

. (97)

2.: Use the definition of the inner product provided in Eq. (97) to form the inner product of Eqs. (26)‒(28) and (93)‒(95) with a vector $a^{(2)} ≜ {[a_{1}^{(2)} (t), a_{2}^{(2)} (t)]}^{†} \in H_{2} (Ω_{t})$ , where $a_{1}^{(2)} (t) ≜ {[a_{1, 1}^{(2)} (t), a_{1, 2}^{(2)} (t), a_{1, 3}^{(2)} (t)]}^{†}$ and $a_{2}^{(2)} (t) ≜ {[a_{2, 1}^{(2)} (t), a_{2, 2}^{(2)} (t), a_{2, 3}^{(2)} (t)]}^{†}$ , to obtain the following relationship:

\begin{array}{l} \int_{0}^{τ} a_{1, 1}^{(2)} (t) \frac{d}{d t} δ h_{1} (t) d t - \int_{0}^{τ} a_{1, 1}^{(2)} (t) F_{1} F_{3} [δ h_{1} (t) h_{2} (t) + h_{1} (t) δ h_{2} (t)] d t \\ + \int_{0}^{τ} a_{1, 2}^{(2)} \frac{d}{d t} δ h_{2} (t) d t - \int_{0}^{τ} a_{1, 2}^{(2)} F_{2} δ h_{1} (t) d t + \int_{0}^{τ} a_{1, 3}^{(2)} \frac{d}{d t} δ h_{3} (t) d t \\ - \int_{0}^{τ} a_{1, 3}^{(2)} (F_{2} F_{3}) δ h_{1} (t) d t + \int_{0}^{τ} a_{2, 1}^{(2)} \frac{d}{d t} δ a_{1}^{(1)} (t) d t \\ + \int_{0}^{τ} a_{2, 1}^{(2)} [δ a_{1}^{(1)} (t) F_{1} F_{3} h_{2} (t) + δ a_{2}^{(1)} (t) F_{2} + δ a_{3}^{(1)} (t) F_{2} F_{3} + δ h_{2} (t) F_{1} F_{3} a_{1}^{(1)} (t)] d t \\ \int_{0}^{τ} a_{2, 2}^{(2)} \frac{d}{d t} δ a_{2}^{(1)} (t) d t + \int_{0}^{τ} a_{2, 2}^{(2)} [δ a_{1}^{(1)} (t) F_{1} F_{3} h_{1} (t) + δ h_{1} (t) F_{1} F_{3} a_{1}^{(1)} (t)] d t \\ + \int_{0}^{τ} a_{2, 3}^{(2)} \frac{d}{d t} δ a_{3}^{(1)} (t) d t = q^{(2)}, \end{array}

(98)

where:

\begin{array}{l} q^{(2)} ≜ \int_{0}^{τ} a_{1, 1}^{(2)} (t) [(δ F_{1}) F_{3} + F_{1} (δ F_{3})] h_{1} (t) h_{2} (t) d t \\ + \int_{0}^{τ} a_{1, 2}^{(2)} (δ F_{2}) h_{1} (t) d t + \int_{0}^{τ} a_{1, 3}^{(2)} [(δ F_{2}) F_{3} + F_{2} (δ F_{3})] h_{1} (t) d t \\ - \int_{0}^{τ} a_{2, 1}^{(2)} \{[(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{2} (t) a_{1}^{(1)} (t) + (δ F_{2}) a_{2}^{(1)} (t) \\ + [(δ F_{2}) F_{3} + (δ F_{3}) F_{2}] a_{3}^{(1)} (t)\} d t - \int_{0}^{τ} a_{2, 2}^{(2)} [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{1} (t) a_{1}^{(1)} (t) d t . \end{array}

(99)

3.: Integrating the left-side of Eq. (98) by parts over the independent variable $t$ yields the following relation:

\begin{array}{l} \int_{0}^{τ} \{- δ h_{1} (t) \frac{d a_{1, 1}^{(2)} (t)}{d t} - a_{1, 1}^{(2)} (t) F_{1} F_{3} [δ h_{1} (t) h_{2} (t) + h_{1} (t) δ h_{2} (t)]\} d t \\ + a_{1, 1}^{(2)} (τ) δ h_{1} (τ) - a_{1, 1}^{(2)} (0) δ h_{1} (0) + a_{1, 2}^{(2)} (τ) δ h_{2} (τ) - a_{1, 2}^{(2)} (0) δ h_{2} (0) \\ - \int_{0}^{τ} \{δ h_{2} (t) \frac{d a_{1, 2}^{(2)}}{d t} + a_{1, 2}^{(2)} F_{2} δ h_{1} (t)\} d t + a_{1, 3}^{(2)} (τ) δ h_{3} (τ) - a_{1, 3}^{(2)} (0) δ h_{3} (0) \\ - \int_{0}^{τ} \{δ h_{3} (t) \frac{d a_{1, 3}^{(2)}}{d t} + a_{1, 3}^{(2)} (F_{2} F_{3}) δ h_{1} (t)\} d t + a_{2, 1}^{(2)} (τ) δ a_{1}^{(1)} (τ) - a_{2, 1}^{(2)} (0) δ a_{1}^{(1)} (0) \\ \int_{0}^{τ} \{- δ a_{1}^{(1)} (t) \frac{d a_{2, 1}^{(2)}}{d t} + a_{2, 1}^{(2)} [δ a_{1}^{(1)} (t) F_{1} F_{3} h_{2} (t) + δ a_{2}^{(1)} (t) F_{2} + δ a_{3}^{(1)} (t) F_{2} F_{3} \\ + δ h_{2} (t) F_{1} F_{3} a_{1}^{(1)} (t)] d t\} + a_{2, 2}^{(2)} (τ) δ a_{2}^{(1)} (τ) - a_{2, 2}^{(2)} (0) δ a_{2}^{(1)} (0) \\ + \int_{0}^{τ} \{- δ a_{2}^{(1)} (t) \frac{d a_{2, 2}^{(2)}}{d t} + a_{2, 2}^{(2)} [δ a_{1}^{(1)} (t) F_{1} F_{3} h_{1} (t) + δ h_{1} (t) F_{1} F_{3} a_{1}^{(1)} (t)]\} d t \\ + a_{2, 3}^{(2)} (τ) δ a_{3}^{(1)} (τ) - a_{2, 3}^{(2)} (0) δ a_{3}^{(1)} (0) - \int_{0}^{τ} δ a_{3}^{(1)} (t) \frac{d a_{2, 3}^{(2)}}{d t} d t = q^{(2)}, \end{array}

(100)

4.: The unknown terms on the left-side of Eq. (100) are eliminated by imposing the following conditions:

a_{1, 1}^{(2)} (τ) = 0; a_{1, 2}^{(2)} (τ) = 0; a_{1, 3}^{(2)} (τ) = 0;

(101)

a_{2, 1}^{(2)} (0) = 0; a_{2, 2}^{(2)} (0) = 0; a_{2, 3}^{(2)} (0) = 0.

(102)

5.: Using the conditions given in Eqs. (29), (96), (101) and (102) on the right-side of Eq. (100) and rearranging the remaining terms yields the following relation:

\begin{array}{l} - a_{1, 1}^{(2)} (0) δ ψ_{0} - a_{1, 3}^{(2)} (0) δ T_{0} \\ + \int_{0}^{τ} δ h_{1} (t) [- \frac{d a_{1, 1}^{(2)} (t)}{d t} - a_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - a_{1, 2}^{(2)} F_{2} - a_{1, 3}^{(2)} (F_{2} F_{3}) + a_{2, 2}^{(2)} F_{1} F_{3} a_{1}^{(1)} (t)] \\ + \int_{0}^{τ} δ h_{2} (t) [- \frac{d a_{1, 2}^{(2)}}{d t} - a_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} a_{1}^{(1)} (t)] d t - \int_{0}^{τ} δ h_{3} (t) \frac{d a_{1, 3}^{(2)}}{d t} d t \\ + \int_{0}^{τ} δ a_{1}^{(1)} (t) [- \frac{d a_{2, 1}^{(2)}}{d t} + a_{2, 1}^{(2)} F_{1} F_{3} h_{2} (t) + a_{2, 2}^{(2)} F_{1} F_{3} h_{1} (t)] \\ + \int_{0}^{τ} δ a_{2}^{(1)} (t) [- \frac{d a_{2, 2}^{(2)}}{d t} + a_{2, 1}^{(2)} F_{2}] d t + \int_{0}^{τ} δ a_{3}^{(1)} (t) [- \frac{d a_{2, 3}^{(2)}}{d t} + a_{2, 1}^{(2)} F_{2} F_{3}] d t = q^{(2)}, \end{array}

(103)

6.: The integral terms on the left-side of Eq. (103) are now required to represent the “indirect-effect” term defined in Eq. (92), which is achieved by imposing the following requirements on the components of the 2nd-level adjoint sensitivity function $a^{(2)} ≜ {[a_{1}^{(2)} (t), a_{2}^{(2)} (t)]}^{†}$ :

\begin{array}{l} - \frac{d a_{1, 1}^{(2)} (t)}{d t} - a_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - a_{1, 2}^{(2)} (t) F_{2} - a_{1, 3}^{(2)} (t) F_{2} F_{3} + a_{2, 2}^{(2)} (t) F_{1} F_{3} a_{1}^{(1)} (t) \\ ​​​ = F_{1} a_{1}^{(1)} (t) h_{2} (t); \end{array}

(104)

- \frac{d a_{1, 2}^{(2)} (t)}{d t} - a_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} a_{1}^{(1)} (t) = F_{1} a_{1}^{(1)} (t) h_{1} (t)

; (105)

\frac{d a_{1, 3}^{(2)} (t)}{d t} = 0

; (106)

- \frac{d a_{2, 1}^{(2)} (t)}{d t} + a_{2, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) + a_{2, 2}^{(2)} (t) F_{1} F_{3} h_{1} (t) = F_{1} h_{2} (t) h_{1} (t)

; (107)

- \frac{d a_{2, 2}^{(2)} (t)}{d t} + a_{2, 1}^{(2)} (t) F_{2} = 0

; (108)

- \frac{d a_{2, 3}^{(2)} (t)}{d t} + a_{2, 1}^{(2)} (t) F_{2} F_{3} = 0

. (109)

The system of equations comprising Eqs. (101), (102), (104)‒(109) constitutes the “2nd-Level Adjoint Sensitivity System (2nd-LASS)” for the 2nd-level adjoint sensitivity function

a^{(2)} ≜ {[a_{1}^{(2)} (t), a_{2}^{(2)} (t)]}^{†}

. Evidently, the 2nd-LASS is linear in

a^{(2)} ≜ {[a_{1}^{(2)} (t), a_{2}^{(2)} (t)]}^{†}

and is independent of parameter variations. Notably, this system of equations does not need to be solved simultaneously, but can be solved sequentially, by first solving Eqs. (107)‒(109) subject to the initial conditions given in Eq. (102) to determine the function

a_{2}^{(2)} (t)

, and subsequently using the function

a_{2}^{(2)} (t)

in Eqs. (104)‒(106) to solve them subject to the “final-time” condition given in Eq. (101) to obtain the function

a_{1}^{(2)} (t)

. The 2nd-LASS is to be solved using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity.

7.: Using the results obtained in Eqs. (104)‒(109) in Eq. (103) yields the following alternative expression for the “indirect-effect” term $δ {[\partial E (τ) / \partial F_{3}]}_{i n d}$ defined in Eq. (92):

δ {[\partial E (τ) / \partial F_{3}]}_{i n d} = q^{(2)} + a_{1, 1}^{(2)} (0) δ ψ_{0} + a_{1, 3}^{(2)} (0) δ T_{0}

. (110)

Notably, the expression on the right-side of Eq. (110) no longer involves the 2nd-level variational sensitivity function

v^{(1)} (t)

but involves the 2nd-level adjoint sensitivity function

a^{(2)} ≜ {[a_{1}^{(2)} (t), a_{2}^{(2)} (t)]}^{†}

8.: Using in Eq. (110) the definition of the function $q^{(2)}$ provided in Eq. (99) and adding the resulting expression for the indirect-effect term $δ {[\partial E (τ) / \partial F_{3}]}_{d i r}$ to the expression for the direct-effect term $δ {(\partial r_{n} / \partial F_{i})}_{d i r}$ provided in Eq. (91) yields the following expression for the total first-order G-differential $δ [\partial E (τ) / \partial F_{3}]$ :

\begin{array}{l} δ [\partial E (τ) / \partial F_{3}] = (δ F_{1}) \int_{0}^{τ} a_{1}^{(1)} (t) h_{2} (t) h_{1} (t) d t + a_{1, 1}^{(2)} (0) δ ψ_{0} + a_{1, 3}^{(2)} (0) δ T_{0} \\ + \int_{0}^{τ} a_{1, 1}^{(2)} (t) [(δ F_{1}) F_{3} + F_{1} (δ F_{3})] h_{1} (t) h_{2} (t) d t + \int_{0}^{τ} a_{1, 2}^{(2)} (δ F_{2}) h_{1} (t) d t \\ + \int_{0}^{τ} a_{1, 3}^{(2)} [(δ F_{2}) F_{3} + F_{2} (δ F_{3})] h_{1} (t) d t - \int_{0}^{τ} a_{2, 1}^{(2)} \{[(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{2} (t) a_{1}^{(1)} (t) \\ + (δ F_{2}) a_{2}^{(1)} (t) + [(δ F_{2}) F_{3} + (δ F_{3}) F_{2}] a_{3}^{(1)} (t)\} d t \\ - \int_{0}^{τ} a_{2, 2}^{(2)} [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{1} (t) a_{1}^{(1)} (t) d t . \end{array}

(111)

The expression shown in Eq. (111) is to be evaluated at the nominal values of all functions and parameters/weights but the superscript “zero” (which has been used to indicate this fact) has been omitted for notational simplicity.

Identifying the expressions that multiply the individual variations in the quantities

δ ψ_{0}

δ T_{0}

and

δ F_{i}

i = 1, 2, 3

, yields the following expressions for the second-order sensitivities that stem from the first-order sensitivity

\partial E (τ) / \partial F_{3}

\frac{\partial^{2} E (τ)}{\partial ψ_{0} \partial F_{3}} = a_{1, 1}^{(2)} (0); \frac{\partial^{2} E (τ)}{\partial T_{0} \partial F_{3}} = a_{1, 3}^{(2)} (0);

(112)

\begin{array}{l} \frac{\partial^{2} E (τ)}{\partial F_{1} \partial F_{3}} = \int_{0}^{τ} a_{1}^{(1)} (t) h_{2} (t) h_{1} (t) d t + F_{3} \int_{0}^{τ} a_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t \\ - F_{3} \int_{0}^{τ} a_{2, 1}^{(2)} h_{2} (t) a_{1}^{(1)} (t) - F_{3} \int_{0}^{τ} a_{2, 2}^{(2)} h_{1} (t) a_{1}^{(1)} (t) d t; \end{array}

(113)

\frac{\partial^{2} E (τ)}{\partial F_{2} \partial F_{3}} = \int_{0}^{τ} a_{1, 2}^{(2)} h_{1} (t) d t + F_{3} \int_{0}^{τ} a_{1, 3}^{(2)} h_{1} (t) d t - \int_{0}^{τ} a_{2, 1}^{(2)} [a_{2}^{(1)} (t) + F_{3} a_{3}^{(1)} (t)] d t;

(114)

\begin{array}{l} \frac{\partial^{2} E (τ)}{\partial F_{3} \partial F_{3}} = F_{1} \int_{0}^{τ} a_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t + F_{2} \int_{0}^{τ} a_{1, 3}^{(2)} h_{1} (t) d t \\ - \int_{0}^{τ} a_{2, 1}^{(2)} [F_{1} h_{2} (t) a_{1}^{(1)} (t) + F_{2} a_{3}^{(1)} (t)] d t - F_{1} \int_{0}^{τ} a_{2, 2}^{(2)} h_{1} (t) a_{1}^{(1)} (t) d t . \end{array}

(115)

The second-order sensitivities that stem from the first-order sensitivities

\partial E (τ) / \partial ψ_{0}

\partial E (τ) / \partial F_{1}

and

\partial E (τ) / \partial F_{2}

are obtained from the G-differentials of the expressions provided in Eqs. (45), (47) and (48), respectively, and by subsequently following the same procedure as was used in this Section for determining the sensitivities stemming from

\partial E (τ) / \partial F_{3}

, which led to the results presented in Eqs. (112)‒(115). The 2nd-LASS that will thus be obtained for the 2nd-level adjoint sensitivity functions that correspond to the indirect-effect terms stemming from the G-differentials of

\partial E (τ) / \partial ψ_{0}

\partial E (τ) / \partial F_{1}

and

\partial E (τ) / \partial F_{2}

, respectively, will have the same operators on their left-sides as the left-side of the 2nd-LASS obtained in Eq. (104)‒(109). Furthermore, the 2nd-level adjoint sensitivity functions that correspond to the indirect-effect terms stemming from

\partial E (τ) / \partial ψ_{0}

\partial E (τ) / \partial F_{1}

and

\partial E (τ) / \partial F_{2}

will satisfy the same initial time and final-time conditions as shown in Eqs. (101) and (102). On the other hand, the source terms on the right-sides of the 2nd-LASS obtained in Eqs. (104)‒(109) will correspond to the “indirect-effect” terms stemming from

\partial E (τ) / \partial ψ_{0}

\partial E (τ) / \partial F_{1}

and

\partial E (τ) / \partial F_{2}

. Therefore, these source terms will all differ from one another and will also differ from the source terms on the right-sides of Eqs. (104)‒(109).

To avoid repetition, the detailed derivation of the second-order sensitivities stemming from

\partial E (τ) / \partial ψ_{0}

\partial E (τ) / \partial F_{1}

and

\partial E (τ) / \partial F_{2}

will be omitted, but it is clear that their derivation will entail the construction and subsequent solving of three distinct 2nd-Level Adjoint Sensitivity Systems. Conceptually, as was shown in the general mathematical formalism underlying the 2nd-FASAM-NODE [1], the computation of the second-order sensitivities entails the construction and solution of as many 2nd-LASS as there are non-zero first-order sensitivities of the response under consideration with respect to the components of the feature functions and encoder’s initial conditions. The second-order sensitivities with respect to the NODE-parameters are obtained from the sensitivities with respect to the components of the feature functions by using the “chain-rule” of differentiating Eq. (50), as shown below:

\frac{\partial^{2} E (τ)}{\partial θ_{j} \partial θ_{i}} = \frac{\partial}{\partial θ_{j}} [\sum_{k = 1}^{3} \frac{\partial E (τ)}{\partial F_{k}} \frac{\partial F_{k} (θ)}{\partial θ_{i}}]; j, i = 1, .., T W = 6

. (116)

Of course, the second-order sensitivities of the response

E (τ)

can be computed directly with respect to the parameters

θ ≜ {[θ_{1}, ..., θ_{T W}]}^{†} ≜ {(α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f})}^{†}

T W = 6

, as demonstrated in [5]. In this case, a total of 6 (as opposed to 3) distinct 2nd-Level Adjoint Sensitivity Systems would have been needed [6], each of these six 2nd-LASS corresponding to the G-differential of each of the six first-order sensitivities that were determined in Eqs. (79)‒(84).

It is important to note that the 2nd-order mixed sensitivities will be determined twice, obtaining two distinct expressions involving two distinct 2nd-level adjoint functions, for each mixed 2nd-order mixed sensitivity. This fact has been demonstrated in detail in [5] for the 2nd-order mixed sensitivities of

E (τ)

with respect to the primary model parameters and will also be demonstrated in Subsection 3.2 and Section 4. Determining the mixed 2nd-order sensitivities twice, using distinct expressions and 2nd-level adjoint sensitivity functions provides a stringent test for verifying the accuracy of the computations of the respective adjoint functions.

3.2. Computation of Second-Order Sensitivities of $E (τ)$ Using the Corresponding Single NODE-Equation

It is expected that the most effective procedure for computing second-order sensitivities of a NODE-response would be to use as few NODE-equations as possible. For the illustrative Nordheim-Fuchs model under consideration, it is expected that the most efficient procedure would be to obtain the second-order sensitivities stemming from the first-order sensitivities

\partial E (τ) / \partial Φ_{1}

and

\partial E (τ) / \partial Φ_{2}

, which were determined in Eqs. (76) and (77), respectively.

3.2.1. Computation of Second-Order Sensitivities of $E (τ)$ Stemming from $\partial E (τ) / \partial Φ_{1}$

The 2nd-order sensitivities which stem from the 1st-order sensitivity

\partial E (τ) / \partial Φ_{1}

defined in Eq. (76) will be obtained by determining the first-order G-differential of

\partial E (τ) / \partial Φ_{1}

, which is obtained by definition as follows:

\begin{array}{l} δ \{\frac{\partial E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{1}}\} = - {\{\frac{d}{d ε} [\int_{0}^{τ} (e^{(1)} + ε δ e^{(1)}) {[E (t) + ε δ E (t)]}^{2} d t]\}}_{ε = 0} \\ = - \int_{0}^{τ} [2 e^{(1)} (t) E (t) δ E (t) + δ e^{(1)} (t) E^{2} (t)] d t \\ = \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{1} \partial Φ_{1}} δ Φ_{1} + \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{2} \partial Φ_{1}} δ Φ_{2} . \end{array}

(117)

The variational function

δ e^{(1)} (t)

is the solution of the system of equations obtained by G-differentiating the 1st-LASS defined in Eqs. (73) and (74). Performing the G-differentiation of this 1st-LASS yields the following equations, which are to be evaluated at the nominal values of the respective parameters and functions:

[- \frac{d}{d t} + 2 Φ_{1} E (t)] δ e^{(1)} (t) + 2 Φ_{1} e^{(1)} (t) [δ E (t)] = - 2 (δ Φ_{1}) e^{(1)} (t) E (t), 0 < t < τ,

(118)

δ e^{(1)} (τ) = 0, t = τ

. (119)

Concatenating Eqs. (118) and (119) with the 1st-LVSS for

δ E (t)

defined in Eqs. (69) and (70) yields the following 2nd-Level Variational Sensitivity System (2nd-LVSS) for the 2nd-level variational function

v^{(2)} (2; t) ≜ {[δ E (t), δ e^{(1)} (t)]}^{†}

, where the superscript “2”denotes “2nd-level”:

(\begin{matrix} d / d t + 2 Φ_{1} E (t) & 0 \\ 2 Φ_{1} e^{(1)} (t) & - d / d t + 2 Φ_{1} E (t) \end{matrix}) (\begin{matrix} δ E (t) \\ δ e^{(1)} (t) \end{matrix}) = (\begin{matrix} - (δ Φ_{1}) E^{2} (t) + (δ Φ_{2}) \\ - 2 (δ Φ_{1}) e^{(1)} (t) E (t) \end{matrix});

(120)

(\begin{matrix} δ E (0) \\ δ e^{(1)} (τ) \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .

(121)

The need for solving repeatedly the 2nd-LVSS defined by Eqs. (120) and (121) to obtain the value of the 2nd-level variational function

v^{(2)} (2; t) ≜ {[δ E (t), δ e^{(1)} (t)]}^{†}

for every possible parameter variation is circumvented by deriving an alternative expression for the first-order G-differential

δ \{\partial E (τ) / \partial Φ_{1}\}

defined in Eq. (117), in which the variational function

v^{(2)} (2; t) ≜ {[δ E (t), δ e^{(1)} (t)]}^{†}

will be replaced by a two-component 2nd-level adjoint function that will be denoted as

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

and which will be independent of parameter variations. The notation used for the two-component vector

e_{1}^{(2)} (t)

is as follows: the superscript “2” indicates “2nd-level” while the subscript “1” indicates that this 2nd-level adjoint sensitivity function will correspond to the first-order derivative

\partial E (τ) / \partial Φ_{1}

with respect to the “first” feature function,

Φ_{1}

. Subsequently, a 2nd-level adjoint sensitivity function

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

will be used for determining the second-order sensitivities stemming from the first-order derivative

\partial E (τ) / \partial Φ_{2}

with respect to the “second” feature function,

Φ_{2}

The 2nd-level adjoint function

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

will be the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS) to be constructed by applying the principles underlying the 2nd-FASAM-NODE methodology [1]. This 2nd-LASS is constructed in a Hilbert space denoted as

H_{2, E} (Ω_{t})

Ω_{t} ≜ [0, τ]

, where the subscript “2”indicates “2nd-level” and which comprises as elements two-component vectors of the same form as

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

. The Hilbert space

H_{2, E} (Ω_{t})

is considered to be endowed with an inner product, denoted as

{〈χ^{(2)}, η^{(2)}〉}_{2, E}

, of two vectors

χ^{(2)} (t) ≜ {[χ_{1}^{(2)} (t), χ_{2}^{(2)} (t)]}^{†}

and

η^{(2)} (t) ≜ {[η_{1}^{(2)} (t), η_{2}^{(2)} (t)]}^{†}

, which is defined as follows:

{〈χ^{(2)}, η^{(2)}〉}_{2, E} ≜ \sum_{i = 1}^{2} \int_{0}^{τ} χ_{i}^{(2)} (t) η_{i}^{(2)} (t) d t

. (122)

Using the definition provided in Eq. (122), form the inner product of

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

with Eq. (120) to obtain the following relation:

\begin{array}{l} \int_{0}^{τ} e_{11}^{(2)} (t) [d / d t + 2 Φ_{1} E (t)] δ E (t) d t + \int_{0}^{τ} e_{12}^{(2)} (t) [2 Φ_{1} ω^{(1)} (t) δ E (t)] d t \\ + \int_{0}^{τ} e_{12}^{(2)} (t) [- d / d t + 2 Φ_{1} E (t)] δ e^{(1)} (t) d t = s_{1}^{(2)} (t), \end{array}

(123)

where:

s_{1}^{(2)} (t) ≜ \int_{0}^{τ} e_{11}^{(2)} (t) [- (δ Φ_{1}) E^{2} (t) + (δ Φ_{2})] d t - 2 (δ Φ_{1}) \int_{0}^{τ} e_{12}^{(2)} (t) e^{(1)} (t) E (t) d t

. (124)

Integrating by parts the terms involving

d / d t

on the left-side of Eq. (123) and rearranging the resulting terms yields the following relation:

\begin{array}{l} e_{11}^{(2)} (τ) δ E (τ) - e_{11}^{(2)} (0) δ E (0) - e_{12}^{(2)} (τ) δ e^{(1)} (τ) + e_{12}^{(2)} (0) δ e^{(1)} (0) \\ + \int_{0}^{τ} δ E (t) \{[- d / d t + 2 Φ_{1} E (t)] e_{11}^{(2)} (t) + 2 Φ_{1} e^{(1)} (t) e_{12}^{(2)} (t)\} d t \\ + \int_{0}^{τ} δ e^{(1)} (t) [d / d t + 2 Φ_{1} E (t)] e_{12}^{(2)} (t) d t = s_{1}^{(2)} (t) . \end{array}

(125)

The integral terms on the left-side of Eq. (125) are required to represent the G-differential

δ \{\partial E (τ) / \partial Φ_{1}\}

defined in Eq. (117), and the unknown terms on the left-side of Eq. (125) are eliminated by requiring the function

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

to satisfy the following 2nd-Level Adjoint System (2nd-LASS):

(\begin{matrix} - d / d t + 2 Φ_{1} (p) E (t) & 2 Φ_{1} (p) e^{(1)} (t) \\ 0 & d / d t + 2 Φ_{1} (p) E (t) \end{matrix}) (\begin{matrix} e_{11}^{(2)} (t) \\ e_{12}^{(2)} (t) \end{matrix}) = (\begin{matrix} - 2 e^{(1)} (t) E (t) \\ - E^{2} (t) \end{matrix});

(126)

(\begin{matrix} e_{11}^{(2)} (τ) \\ e_{12}^{(2)} (0) \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .

(127)

The above 2nd-LASS is to be solved at the nominal values for the 1st-level adjoint function

e^{(1)} (t)

and for the parameters

p ≜ {[α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f}, ψ_{0}]}^{†}

. Notably, the 2nd-LASS is independent of variations in the feature functions (and/or parameter variations), so it needs to be solved just once to obtain the 2nd-level adjoint function

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

. Furthermore, the 2nd-LASS is an upper-triangular system, so the respective equations need not solved simultaneously, but can be solved sequentially, first for the component

e_{12}^{(2)} (t)

and subsequently for the component

e_{11}^{(2)} (t)

. The explicit expressions of the components of

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

are not essential for following the subsequent conceptual derivations.

Using in Eq. (125) the 2nd-LASS defined by Eqs. (126) and (127) together with the conditions provided in Eq. (121) yields the following expression for the variation

δ \{\partial E (τ) / \partial Φ_{1}\}

in terms of the 2nd-level adjoint function

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

\begin{array}{l} δ \{\frac{\partial E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{1}}\} = \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{1} \partial Φ_{1}} δ Φ_{1} + \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{2} \partial Φ_{1}} δ Φ_{2} \\ = \int_{0}^{τ} e_{11}^{(2)} (t) [- (δ Φ_{1}) E^{2} (t) + (δ Φ_{2})] d t - 2 (δ Φ_{1}) \int_{0}^{τ} e_{12}^{(2)} (t) e^{(1)} (t) E (t) d t . \end{array}

(128)

It follows from Eq. (128) that the second-order sensitivities of the response

E (τ)

with respect to the feature functions

Φ_{1}

and

Φ_{2}

have the following expressions:

\frac{\partial^{2} E (τ)}{\partial Φ_{1} \partial Φ_{1}} = - \int_{0}^{τ} [e_{11}^{(2)} (t) E (t) + 2 e_{12}^{(2)} (t) e^{(1)} (t)] E (t) d t;

(129)

\frac{\partial^{2} E (τ)}{\partial Φ_{2} \partial Φ_{1}} = \int_{0}^{τ} e_{11}^{(2)} (t) d t

. (130)

3.2.2. Computation of Second-Order Sensitivities of $E (τ)$ Stemming from $\partial E (τ) / \partial Φ_{2}$

The 2nd-order sensitivities which stem from the 1st-order sensitivity

\partial E (τ) / \partial Φ_{2}

are obtained from the first-order G-differential

δ \{\partial E (τ) / \partial Φ_{2}\}

of the expression provided in Eq. (77), which yields, by definition, the following expression:

\begin{array}{l} δ \{\frac{\partial E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{2}}\} = {\{\frac{d}{d ε} [\int_{0}^{τ} [e^{(1)} (t) + ε δ e^{(1)} (t)] d t]\}}_{ε = 0} \\ = \int_{0}^{τ} δ e^{(1)} (t) d t ≜ \frac{\partial^{2} E (τ; Φ_{1}, Φ_{22})}{\partial Φ_{1} \partial Φ_{2}} δ Φ_{1} + \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{2} \partial Φ_{2}} δ Φ_{2} . \end{array}

(131)

The variational function

δ e^{(1)} (t)

is the solution of Eqs. (118) and (119). Notably, the right-side of Eq. (131) depends only on the variational function

δ e^{(1)} (t)

, but does not depend directly on the variational function

δ E (t)

. Nevertheless, since the variational function

δ e^{(1)} (t)

is related to the variational function

δ E (t)

through Eqs. (118) and (119), the 2nd-level adjoint function that will be constructed in order to eliminate the appearance of

δ e^{(1)} (t)

on the right-side of Eq. (131) will be the solution of a 2nd-LASS which will correspond to the 2nd-LVSS defined by Eqs. (120) and (121). The construction of the 2nd-LASS that will be used to eliminate the appearance of the variational function

δ e^{(1)} (t)

from Eq. (131) is based on the principles underlying the 2nd-FASAM-NODE [1], using the same Hilbert Space,

H_{2, E} (Ω_{t})

, with the inner product defined in Eq. (122), as was used for obtaining the results shown in Eqs. (129) and (130).

The inner product defined in Eq. (122) is now used to construct the 2nd-Level Adjoint Sensitivity System (2nd-LASS) for the 2nd-level adjoint function

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

by following the same sequence of steps as was used to obtain the expressions shown in Eqs. (129) and (130), but using the expression provided in Eq.(131) to determine the right-side (“source”) for the 2nd-LASS to be used for determining the second-order sensitivities stemming from

δ \{\partial E (τ) / \partial Φ_{2}\}

. Thus, using the definition provided in Eq. (122), form the inner product of

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

with Eq. (120) to obtain the following relation:

\begin{array}{l} \int_{0}^{τ} e_{21}^{(2)} (t) [d / d t + 2 Φ_{1} E (t)] δ E (t) d t + \int_{0}^{τ} e_{22}^{(2)} (t) [2 Φ_{1} ω^{(1)} (t) δ E (t)] d t \\ + \int_{0}^{τ} e_{22}^{(2)} (t) [- d / d t + 2 Φ_{1} E (t)] δ e^{(1)} (t) d t = s_{2}^{(2)} (t), \end{array}

(132)

where:

s_{2}^{(2)} (t) ≜ \int_{0}^{τ} e_{21}^{(2)} (t) [- (δ Φ_{1}) E^{2} (t) + (δ Φ_{2})] d t - 2 (δ Φ_{1}) \int_{0}^{τ} e_{22}^{(2)} (t) e^{(1)} (t) E (t) d t

. (133)

Integrating by parts the terms involving

d / d t

on the left-side of Eq. (132) and rearranging the resulting terms yields the following relation:

\begin{array}{l} e_{21}^{(2)} (τ) δ E (τ) - e_{21}^{(2)} (0) δ E (0) - e_{22}^{(2)} (τ) δ e^{(1)} (τ) + e_{22}^{(2)} (0) δ e^{(1)} (0) \\ + \int_{0}^{τ} δ E (t) \{[- d / d t + 2 Φ_{1} E (t)] e_{21}^{(2)} (t) + 2 Φ_{1} e^{(1)} (t) e_{22}^{(2)} (t)\} d t \\ + \int_{0}^{τ} δ e^{(1)} (t) [d / d t + 2 Φ_{1} E (t)] e_{22}^{(2)} (t) d t = s_{2}^{(2)} (t) . \end{array}

(134)

The integral terms on the left-side of Eq. (134) are required to represent the G-differential

δ \{\partial E (τ) / \partial Φ_{2}\}

defined in Eq. (131), and the unknown terms on the left-side of Eq. (134) are eliminated by requiring the function

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

to satisfy the following 2nd-Level Adjoint System (2nd-LASS)

(\begin{matrix} - d / d t + 2 Φ_{1} (p) E (t) & 2 Φ_{1} (p) e^{(1)} (t) \\ 0 & d / d t + 2 Φ_{1} (p) E (t) \end{matrix}) (\begin{matrix} e_{21}^{(2)} (t) \\ e_{22}^{(2)} (t) \end{matrix}) = (\begin{matrix} 0 \\ 1 \end{matrix});

(135)

(\begin{matrix} e_{21}^{(2)} (τ) \\ e_{22}^{(2)} (0) \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .

(136)

The above 2nd-LASS is to be solved at the nominal values for the 1st-level adjoint function

e^{(1)} (t)

and for the parameters

p ≜ {[α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f}, ψ_{0}]}^{†}

. Since the 2nd-LASS is independent of variations in the feature functions (and/or parameter variations), it needs to be solved just once to obtain the 2nd-level adjoint function

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

. Furthermore, the 2nd-LASS is an upper-triangular system, so the respective equations need not solved simultaneously, but can be solved sequentially, first for the component

e_{22}^{(2)} (t)

and subsequently for the component

e_{21}^{(2)} (t)

Using in Eq. (134) the 2nd-LASS defined by Eqs. (135) and (136) together with the conditions provided in Eq. (121) yields the following expression for the variation

δ \{\partial E (τ) / \partial Φ_{2}\}

in terms of the 2nd-level adjoint function

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

\begin{array}{l} δ \{\frac{\partial E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{2}}\} = \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{1} \partial Φ_{2}} δ Φ_{1} + \frac{\partial^{2} E (τ; Φ_{1}, Φ_{2})}{\partial Φ_{2} \partial Φ_{2}} δ Φ_{2} \\ = \int_{0}^{τ} e_{21}^{(2)} (t) [- (δ Φ_{1}) E^{2} (t) + (δ Φ_{2})] d t - 2 (δ Φ_{1}) \int_{0}^{τ} e_{22}^{(2)} (t) e^{(1)} (t) E (t) d t . \end{array}

(137)

It follows from Eq. (137) that:

\frac{\partial^{2} E (τ)}{\partial Φ_{1} \partial Φ_{2}} = - \int_{0}^{τ} [e_{21}^{(2)} (t) E (t) + 2 e_{22}^{(2)} (t) e^{(1)} (t)] E (t) d t;

(138)

\frac{\partial^{2} E (τ)}{\partial Φ_{2} \partial Φ_{2}} = \int_{0}^{τ} e_{21}^{(2)} (t) d t

. (139)

It is important to note that the second-order mixed partial derivative

\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{2} = \partial^{2} E (τ) / \partial Φ_{2} \partial Φ_{1}

can be obtained from either Eq. (138) or Eq. (130). The equivalence between these expressions provides a stringent verification of the accuracy of solving (on the one hand) the 2nd-LASS comprising Eqs. (126) and (127) to obtain

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

and solving (on the other hand) the 2nd-LASS comprising Eqs. (135) and (136) to obtain

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

Notably, only two “large-scale computations” are necessary for solving the two distinct 2nd-LASS for obtaining the 2nd-level adjoint functions

e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}

and

e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}

involved in the computation of the three distinct second-order response sensitivities i.e.,

\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{1}

\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{2}

and

\partial^{2} E (τ) / \partial Φ_{2} \partial Φ_{2}

, with respect to the two feature functions

Φ_{1}

and

Φ_{2}

. The second-order sensitivities of the response

E (τ)

with respect to the primary model parameters are obtained by using the total differential of the expression provided in Eq. (78) in conjunction with the parameter-dependencies of the functions

Φ_{1} (p)

and

Φ_{2} (p)

defined in Eq. (66), and with the expressions obtained in Eqs. (129), (130), (138) and (139).

In summary, computing all of the second-order sensitivities of the response

E (τ)

with respect to the feature functions and initial conditions using the original NODE-equations requires as many large-scale computations as there are component features (3 components) functions and initial conditions (2 initial conditions), for a total of 5 “large-scale” computations to solve the respective 2nd-LASSystems, for a total of 5x6=30 differential equations (since each of the respective 2nd-LASS comprises 6 differential equations). In contradistinction, computing all of the second-order sensitivities of the response

E (τ)

with respect to the primary model parameters using the modified NODE-equation for the single dependent variable

E (t)

requires 2 “large-scale” computations to solve the 2nd-LASS represented by Eq. (126) and, respectively, Eq. (135), each involving two-coupled differential equations, amounting to a total of 4 differential equations, followed by inexpensive computations using the “chain-rule” of differentiation. Hence, it is very advantageous to extract equations involving as few dependent variables as possible from the original NODE-system (like the single equation for

E (t)

in the illustrative example in this subsection), even if this procedure entails modifications to the original NODE system.

After having obtained the second-order sensitivities

\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{1}

\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{2}

and

\partial^{2} E (τ) / \partial Φ_{2} \partial Φ_{2}

, the exact expressions of the second-order sensitivities

\partial^{2} E (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

, of the response

E (τ)

with respect to the components of the vector of primary parameters

p ≜ {[p_{1}, ..., p_{7}]}^{†} ≜ {[α_{T}, l_{p}, c_{p}, γ, σ_{f}, N_{f}, ψ_{0}]}^{†}

can be determined analytically by applying the chain-rule of differentiation to the expression of the first-order sensitivities

\partial E (τ) / \partial p_{i}

i = 1, ..., 7

, shown in Eq. (78), to obtain the following relation:

\frac{\partial^{2} E (τ; Φ_{1}; Φ_{2})}{\partial p_{j} \partial p_{i}} = \frac{\partial}{\partial p_{j}} [\frac{\partial E (τ)}{\partial Φ_{1}} \frac{\partial Φ_{1} (p)}{\partial p_{i}} + \frac{\partial E (τ)}{\partial Φ_{2}} \frac{\partial Φ_{2} (p)}{\partial p_{i}}]; i, j = 1, ..., 7.

(140)

After having obtained the second-order sensitivities

\partial^{2} E (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

, the second-order sensitivities

\partial^{2} T (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

, of the temperature-response

T (τ)

can be determined analytically by applying the chain-rule of differentiation to the expression of the first-order sensitivities

\partial T (τ) / \partial p_{i}

i = 1, ..., 7

, shown in Eq. (86), to obtain the following relation:

\begin{array}{l} \frac{\partial^{2} T (τ)}{\partial p_{j} \partial p_{i}} = \frac{\partial}{\partial p_{j}} [\frac{\partial (1 / c_{p})}{\partial p_{i}} E (τ) + \frac{1}{c_{p}} \frac{\partial E (τ)}{\partial p_{i}}] = \frac{\partial^{2} (1 / c_{p})}{\partial p_{j} \partial p_{i}} E (τ) \\ + \frac{\partial E (τ)}{\partial p_{j}} \frac{\partial (1 / c_{p})}{\partial p_{i}} + \frac{\partial (1 / c_{p})}{\partial p_{j}} \frac{\partial E (τ)}{\partial p_{i}} + \frac{1}{c_{p}} \frac{\partial^{2} E (τ)}{\partial p_{j} \partial p_{i}}; i, j = 1, ..., 7. \end{array}

(141)

After having obtained the second-order sensitivities

\partial^{2} E (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

, the second-order sensitivities

\partial^{2} ψ (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

, of the neutron flux response

ψ (τ)

can be determined analytically by applying the chain-rule of differentiation to the expression of the first-order sensitivities

\partial ψ (τ) / \partial p_{i}

i = 1, ..., 7

, shown in Eq. (88), to obtain the following relations:

\frac{\partial^{2} ψ (τ)}{\partial p_{j} \partial ψ_{0}} = 0; j = 1, ..., 7;

(142)

\frac{\partial^{2} ψ (τ)}{\partial p_{j} \partial p_{i}} = - \frac{\partial}{\partial p_{j}} [E^{2} (τ) \frac{\partial}{\partial p_{i}} (\frac{α_{T}}{2 l_{p} c_{p} γ σ_{f} N_{f}}) + \frac{α_{T} E (τ)}{l_{p} c_{p} γ σ_{f} N_{f}} \frac{\partial E (τ)}{\partial p_{i}}]; i, j = 1, ..., 7.

. (143)

4. Application of the 2nd-FASAM-NODE Methodology to Compute Second-Order Sensitivities of an Illustrative Nordheim-Fuchs Model Response Involving Decoder-Weights

This Section presents the application of the 2nd-FASAM-NODE methodology to compute 2nd-order sensitivities of the thermal conductivity in the Nordheim-Fuchs model, which is a representative model response involving decoder-weights subject to uncertainties. Section 4.1 presents the determination of the second-order sensitivities by applying the 2nd-FASAM-NODE methodology to the original NODE-structure. Section 4.2 presents the alternative computation of these 2nd-order sensitivities by applying the 2nd-FASAM-NODE methodology to the decoupled NODE-structure. It will be shown that using a decoupled structure, whenever possible, is even more advantageous for the computation of second-order sensitivities than for the computation of the first-order ones.

4.1. Second-Order Sensitivities of the Thermal Conductivity Response $K (τ)$ Computed Using the Coupled NODE-Equations

The application of the 2nd-FASAM-NODE methodology to a Nordheim-Fuchs model response involving decoder-weights will be illustrated by considering the thermal conductivity response

K (τ)

. The first-order sensitivities of

K (τ)

were obtained in Section 2.3, as follows:

(i): The first-order sensitivities of $K (τ)$ with respect to the decoder weights arose from the “direct-effect term” defined in Eq. (52), and were obtained in Eqs. (54)‒(56). The computation of the second-order sensitivities stemming from these first-order sensitivities will be illustrated in Subsection 4.1.1.
(ii): The first-order sensitivities of $K (τ)$ with respect to the feature functions were obtained in Eqs. (62)‒(64). The second-order sensitivities stemming from these first-order sensitivities will be illustrated in Subsection 4.1.2.
(iii): The first-order sensitivities of $K (τ)$ with respect to the initial conditions were obtained in Eqs. (60) and (61). The second-order sensitivities stemming from these first-order sensitivities will be illustrated in Subsection 4.1.3.

4.1.1. Second-Order Sensitivities Stemming from the First-Order Sensitivities of $K (τ)$ with Respect to the Decoder Weights $φ_{0}$ , $φ_{1}$ and $φ_{2}$

A.: Second-order sensitivities stemming from $\partial K [T (τ)] / \partial φ_{0}$

It is evident from the expression of

\partial K [T (τ)] / \partial φ_{0}

obtained in Eqs. (54) that all of the second-order sensitivities which stem from this first-order sensitivity are identically zero, i.e.,

\partial^{2} K [T (τ)] / \partial φ_{i} \partial φ_{0} \equiv 0

, for

i = 1, 2, 3

;

\partial^{2} K [T (τ)] / \partial F_{i} \partial φ_{0} \equiv 0

, for

i = 1, 2, 3

;

\partial^{2} K [T (τ)] / \partial x_{i} \partial φ_{0} \equiv 0

, for

i = 1, 2, 3

B.: Second-order sensitivities stemming from $\partial K [T (τ)] / \partial φ_{1}$

The second-order sensitivities which stem from the first-order sensitivity

\partial K [T (τ)] / \partial φ_{1}

are determined from the G-differential of the expression provided in Eq. (55), which is obtained, by definition, as follows:

δ \{\frac{\partial K [T (τ)]}{\partial φ_{1}}\} ≜ {\{\frac{d}{d ε} \int_{0}^{τ} [T^{0} (t) + ε δ T (t)] δ (t - τ) d t\}}_{ε = 0} = \int_{0}^{τ} δ T (t) δ (t - τ) d t

. (144)

As indicated by Eq. (144), the G-differential

δ \{\partial K [T (τ)] / \partial φ_{1}\}

depends only on the variation

δ T (t)

. It therefore follows that

δ \{\partial K [T (τ)] / \partial φ_{1}\}

can be evaluated by following the procedure outlined in Subsection 2.3, to obtain the following expression:

\begin{array}{l} δ \{\partial K [T (τ)] / \partial φ_{1}\} = b_{1}^{(2)} (0) δ ψ_{0} + b_{3}^{(2)} (0) δ T_{0} + \int_{0}^{τ} {[b^{(2)} (t)]}^{†} \frac{\partial f (h; F)}{\partial F} δ F d t \\ ≜ \frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial φ_{1}} δ ψ_{0} + \frac{\partial^{2} K (τ)}{\partial T_{0} \partial φ_{1}} δ T_{0} + \sum_{i = 1}^{3} \frac{\partial^{2} K (τ)}{\partial F_{i} \partial φ_{1}} δ F_{i}, \end{array}

(145)

where the 2nd-level adjoint sensitivity function

b^{(2)} (t) ≜ {[b_{1}^{(2)} (t), b_{2}^{(2)} (t), b_{3}^{(2)} (t)]}^{†}

is the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS), which is to be solved at the nominal values for all parameters and original functions:

- \frac{d b^{(2)} (t)}{d t} - J^{†} b^{(2)} (t) = {(0, 0, 1)}^{†}

; (146)

b^{(2)} (τ) ≜ {[b_{1}^{(2)} (τ), b_{2}^{(2)} (τ), b_{3}^{(2)} (τ)]}^{†} = {(0, 0, 0)}^{†}

. (147)

It follows from Eq. (145) that the respective second-order sensitivities have the following expressions:

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial φ_{1}} = b_{1}^{(2)} (0) = \int_{0}^{τ} b_{1}^{(2)} (t) δ (t) d t

; (148)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial φ_{1}} = b_{3}^{(2)} (0) = \int_{0}^{τ} b_{3}^{(2)} (t) δ (t) d t

; (149)

\frac{\partial^{2} K (τ)}{\partial F_{1} \partial φ_{1}} = F_{3} (θ) \int_{0}^{τ} h_{1} (t) h_{2} (t) b_{1}^{(2)} (t) d t

; (150)

\frac{\partial^{2} K (τ)}{\partial F_{2} \partial φ_{1}} = \int_{0}^{τ} [b_{2}^{(2)} (t) + F_{3} (θ) b_{3}^{(2)} (t)] h_{1} (t) d t

; (151)

\frac{\partial^{2} K (τ)}{\partial F_{3} \partial φ_{1}} = \int_{0}^{τ} [F_{1} (θ) b_{1}^{(2)} (t) h_{2} (t) + F_{2} (θ) b_{3}^{(2)} (t)] h_{1} (t) d t

. (152)

C.: Second-order sensitivities stemming from $\partial K [T (τ)] / \partial φ_{2}$

The second-order sensitivities stemming from the first-order sensitivity

\partial K [T (τ)] / \partial φ_{2}

are determined from the G-differential of the expression provided in Eq. (56), which is obtained, by definition, as follows:

\begin{array}{l} δ \{\frac{\partial K [T (τ)]}{\partial φ_{2}}\} ≜ {\{\frac{d}{d ε} \int_{0}^{τ} {[T^{0} (t) + ε δ T (t)]}^{2} δ (t - τ) d t\}}_{ε = 0} \\ = 2 \int_{0}^{τ} T^{0} (t) δ T (t) δ (t - τ) d t . \end{array}

(153)

The second-order sensitivities stemming from the G-differential

δ \{\partial K [T (τ)] / \partial φ_{2}\}

obtained in Eq. (153) are derived by following the same procedure as outlined in Subsection 2.3, which was also used above to obtain the second-order sensitivities stemming from

\partial K [T (τ)] / \partial φ_{1}

. The final expressions of these second-order sensitivities are as follows:

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial φ_{2}} = c_{1}^{(2)} (0) = \int_{0}^{τ} c_{1}^{(2)} (t) δ (t) d t

; (154)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial φ_{2}} = c_{3}^{(2)} (0) = \int_{0}^{τ} c_{3}^{(2)} (t) δ (t) d t

; (155)

\frac{\partial^{2} K (τ)}{\partial F_{1} \partial φ_{2}} = F_{3} (θ) \int_{0}^{τ} h_{1} (t) h_{2} (t) c_{1}^{(2)} (t) d t

; (156)

\frac{\partial^{2} K (τ)}{\partial F_{2} \partial φ_{2}} = \int_{0}^{τ} [b_{2}^{(2)} (t) + F_{3} (θ) c_{3}^{(2)} (t)] h_{1} (t) d t

; (157)

\frac{\partial^{2} K (τ)}{\partial F_{3} \partial φ_{2}} = \int_{0}^{τ} [F_{1} (θ) c_{1}^{(2)} (t) h_{2} (t) + F_{2} (θ) c_{3}^{(2)} (t)] h_{1} (t) d t

. (158)

where the second-order adjoint sensitivity function

c^{(2)} (t) ≜ {[c_{1}^{(2)} (t), c_{2}^{(2)} (t), c_{3}^{(2)} (t)]}^{†}

is the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):

- \frac{d c^{(2)} (t)}{d t} - J^{†} c^{(2)} (t) = {[0, 0, 2 T (t)]}^{†}

; (159)

c^{(2)} (τ) ≜ {[c_{1}^{(2)} (τ), c_{2}^{(2)} (τ), c_{3}^{(2)} (τ)]}^{†} = {(0, 0, 0)}^{†}

. (160)

4.1.2. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Feature Functions $F_{1}$ , $F_{2}$ and $F_{3}$

A.: Second-order sensitivities stemming from $\partial K (τ) / \partial F_{1}$

The second-order sensitivities stemming from the first-order sensitivity

\partial K (τ) / \partial F_{1}

are obtained from the G-differential

δ \{\partial K [T (τ)] / \partial F_{1}\}

, which is obtained by using Eq. (62), as follows:

\begin{array}{l} δ \{\frac{\partial K (τ)}{\partial F_{1}}\} ≜ \{\frac{d}{d ε} \int_{0}^{τ} [F_{3}^{0} + ε δ F_{3}] [h_{1}^{0} (t) + ε δ h_{1} (t)] \\ {\times [h_{2}^{0} (t) + ε δ h_{2} (t)] [b_{1}^{(1, 0)} (t) + ε δ b_{1}^{(1)} (t)] d t\}}_{ε = 0} = (δ F_{3}) \int_{0}^{τ} h_{1}^{0} (t) h_{2}^{0} (t) b_{1}^{(1, 0)} (t) d t \\ + F_{3}^{0} \int_{0}^{τ} δ h_{1} (t) h_{2}^{0} (t) b_{1}^{(1, 0)} (t) d t + F_{3}^{0} \int_{0}^{τ} δ h_{2} (t) h_{1}^{0} (t) b_{1}^{(1, 0)} (t) d t \\ + F_{3}^{0} \int_{0}^{τ} δ b_{1}^{(1)} (t) h_{1}^{0} (t) h_{2}^{0} (t) d t = δ {\{\partial K (τ) / \partial F_{1}\}}_{d i r} + δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d}, \end{array}

(161)

where the superscript “zero” indicates, as before, that the respective quantities are to be evaluated at their nominal values. The following definitions were used for the “direct-effect” and, respectively, “indirect-effect” terms defined in Eq. (161):

δ {\{\partial K (τ) / \partial F_{1}\}}_{d i r} ≜ (δ F_{3}) \int_{0}^{τ} h_{1}^{0} (t) h_{2}^{0} (t) b_{1}^{(1, 0)} (t) d t

, (162)

\begin{array}{l} δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d} ≜ F_{3}^{0} \int_{0}^{τ} δ h_{1} (t) h_{2}^{0} (t) b_{1}^{(1, 0)} (t) d t + F_{3}^{0} \int_{0}^{τ} δ h_{2} (t) h_{1}^{0} (t) b_{1}^{(1, 0)} (t) d t \\ + F_{3}^{0} \int_{0}^{τ} δ b_{1}^{(1)} (t) h_{1}^{0} (t) h_{2}^{0} (t) d t . \end{array}

(163)

The variational function

δ b^{(1)} (t) ≜ {[δ b_{1}^{(1)} (t), δ b_{2}^{(1)} (t), δ b_{3}^{(1)} (t)]}^{†}

is the solution of the G-differentiated 1st-Level Adjoint Sensitivity System defined by Eqs. (58) and (59), which is obtained by applying the definition of the G-differential to these equations in order to obtain the following relations:

\begin{array}{l} \frac{d}{d t} δ b_{1}^{(1)} (t) = - δ b_{1}^{(1)} (t) F_{1} F_{3} h_{2} (t) - δ b_{2}^{(1)} (t) F_{2} - δ b_{3}^{(1)} (t) F_{2} F_{3} - δ h_{2} (t) F_{1} F_{3} b_{1}^{(1)} (t) \\ - [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{2} (t) b_{1}^{(1)} (t) - (δ F_{2}) b_{2}^{(1)} (t) - [(δ F_{2}) F_{3} + (δ F_{3}) F_{2}] b_{3}^{(1)} (t); \end{array}

(164)

\begin{array}{l} \frac{d}{d t} δ b_{2}^{(1)} (t) = - δ b_{1}^{(1)} (t) F_{1} F_{3} h_{1} (t) - δ h_{1} (t) F_{1} F_{3} b_{1}^{(1)} (t) \\ ​​ - [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{1} (t) b_{1}^{(1)} (t), \end{array}

(165)

\frac{d}{d t} δ b_{3}^{(1)} (t) = - [δ φ_{1} + 2 δ φ_{2} h_{3} (t) + 2 φ_{2} δ h_{3} (t)] δ (t - τ)

. (166)

δ b^{(1)} (τ) ≜ {[δ b_{1}^{(1)} (τ), δ b_{2}^{(1)} (τ), δ b_{3}^{(1)} (τ)]}^{†} = {[0, 0, 0]}^{†}

(167)

The indirect-effect term defined by Eq. (163) is evaluated by following the same procedural steps as described in Subsection 3.1, which comprises the following sequence of steps:

1.: Use the definition of the inner product provided in Eq. (97) to form the inner product of Eqs. (26)‒(28) and (164)‒(166) with the 2nd-level adjoint sensitivity function $u^{(2)} ≜ {[u_{1}^{(2)} (t), u_{2}^{(2)} (t)]}^{†} \in H_{2} (Ω_{t})$ , $u_{1}^{(2)} (t) ≜ {[u_{1, 1}^{(2)} (t), u_{1, 2}^{(2)} (t), u_{1, 3}^{(2)} (t)]}^{†}$ and $u_{2}^{(2)} (t) ≜ {[u_{2, 1}^{(2)} (t), u_{2, 2}^{(2)} (t), u_{2, 3}^{(2)} (t)]}^{†}$ , to obtain the following relationship:

\begin{array}{l} \int_{0}^{τ} u_{1, 1}^{(2)} (t) \frac{d}{d t} δ h_{1} (t) d t - \int_{0}^{τ} u_{1, 1}^{(2)} (t) F_{1} F_{3} [δ h_{1} (t) h_{2} (t) + h_{1} (t) δ h_{2} (t)] d t \\ + \int_{0}^{τ} u_{1, 2}^{(2)} \frac{d}{d t} δ h_{2} (t) d t - \int_{0}^{τ} u_{1, 2}^{(2)} F_{2} δ h_{1} (t) d t + \int_{0}^{τ} u_{1, 3}^{(2)} \frac{d}{d t} δ h_{3} (t) d t \\ - \int_{0}^{τ} u_{1, 3}^{(2)} (F_{2} F_{3}) δ h_{1} (t) d t + \int_{0}^{τ} u_{2, 1}^{(2)} \frac{d}{d t} δ b_{1}^{(1)} (t) d t \\ + \int_{0}^{τ} u_{2, 1}^{(2)} [δ b_{1}^{(1)} (t) F_{1} F_{3} h_{2} (t) + δ b_{2}^{(1)} (t) F_{2} + δ b_{3}^{(1)} (t) F_{2} F_{3} + δ h_{2} (t) F_{1} F_{3} b_{1}^{(1)} (t)] d t \\ \int_{0}^{τ} u_{2, 2}^{(2)} \frac{d}{d t} δ b_{2}^{(1)} (t) d t + \int_{0}^{τ} u_{2, 2}^{(2)} [δ b_{1}^{(1)} (t) F_{1} F_{3} h_{1} (t) + δ h_{1} (t) F_{1} F_{3} b_{1}^{(1)} (t)] d t \\ + \int_{0}^{τ} u_{2, 3}^{(2)} \frac{d}{d t} δ b_{3}^{(1)} (t) d t + 2 φ_{2} \int_{0}^{τ} δ h_{3} (t) u_{2, 3}^{(2)} δ (t - τ) d t = r^{(2)}, \end{array}

(168)

\begin{array}{l} r^{(2)} ≜ \int_{0}^{τ} u_{1, 1}^{(2)} (t) [(δ F_{1}) F_{3} + F_{1} (δ F_{3})] h_{1} (t) h_{2} (t) d t \\ + \int_{0}^{τ} u_{1, 2}^{(2)} (δ F_{2}) h_{1} (t) d t + \int_{0}^{τ} u_{1, 3}^{(2)} [(δ F_{2}) F_{3} + F_{2} (δ F_{3})] h_{1} (t) d t \\ - \int_{0}^{τ} u_{2, 1}^{(2)} \{[(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{2} (t) b_{1}^{(1)} (t) + (δ F_{2}) b_{2}^{(1)} (t) \\ + [(δ F_{2}) F_{3} + (δ F_{3}) F_{2}] b_{3}^{(1)} (t)\} d t - \int_{0}^{τ} u_{2, 2}^{(2)} [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{1} (t) b_{1}^{(1)} (t) d t \\ - \int_{0}^{τ} u_{2, 3}^{(2)} [δ φ_{1} + 2 δ φ_{2} h_{3} (t)] δ (t - τ) d t . \end{array}

(169)

2.: Integrating the left-side of Eq. (168) by parts over the independent variable $t$ yields the following relation:

\begin{array}{l} \int_{0}^{τ} \{- δ h_{1} (t) \frac{d u_{1, 1}^{(2)} (t)}{d t} - u_{1, 1}^{(2)} (t) F_{1} F_{3} [δ h_{1} (t) h_{2} (t) + h_{1} (t) δ h_{2} (t)]\} d t \\ + u_{1, 1}^{(2)} (τ) δ h_{1} (τ) - u_{1, 1}^{(2)} (0) δ h_{1} (0) + u_{1, 2}^{(2)} (τ) δ h_{2} (τ) - u_{1, 2}^{(2)} (0) δ h_{2} (0) \\ - \int_{0}^{τ} \{δ h_{2} (t) \frac{d u_{1, 2}^{(2)}}{d t} + u_{1, 2}^{(2)} F_{2} δ h_{1} (t)\} d t + u_{1, 3}^{(2)} (τ) δ h_{3} (τ) - u_{1, 3}^{(2)} (0) δ h_{3} (0) \\ - \int_{0}^{τ} \{δ h_{3} (t) \frac{d u_{1, 3}^{(2)}}{d t} + u_{1, 3}^{(2)} (F_{2} F_{3}) δ h_{1} (t)\} d t + u_{2, 1}^{(2)} (τ) δ b_{1}^{(1)} (τ) - u_{2, 1}^{(2)} (0) δ b_{1}^{(1)} (0) \\ \int_{0}^{τ} \{- δ b_{1}^{(1)} (t) \frac{d u_{2, 1}^{(2)}}{d t} + u_{2, 1}^{(2)} [δ a_{1}^{(1)} (t) F_{1} F_{3} h_{2} (t) + δ b_{2}^{(1)} (t) F_{2} + δ b_{3}^{(1)} (t) F_{2} F_{3} \\ + δ h_{2} (t) F_{1} F_{3} b_{1}^{(1)} (t)] d t\} + u_{2, 2}^{(2)} (τ) δ b_{2}^{(1)} (τ) - u_{2, 2}^{(2)} (0) δ b_{2}^{(1)} (0) \\ + \int_{0}^{τ} \{- δ b_{2}^{(1)} (t) \frac{d u_{2, 2}^{(2)}}{d t} + u_{2, 2}^{(2)} [δ b_{1}^{(1)} (t) F_{1} F_{3} h_{1} (t) + δ h_{1} (t) F_{1} F_{3} b_{1}^{(1)} (t)]\} d t \\ + u_{2, 3}^{(2)} (τ) δ b_{3}^{(1)} (τ) - u_{2, 3}^{(2)} (0) δ b_{3}^{(1)} (0) - \int_{0}^{τ} δ b_{3}^{(1)} (t) \frac{d u_{2, 3}^{(2)}}{d t} d t \\ + 2 φ_{2} \int_{0}^{τ} δ h_{3} (t) u_{2, 3}^{(2)} (t) δ (t - τ) d t = r^{(2)}, \end{array}

(170)

3.: The unknown terms on the left-side of Eq. (170) are eliminated by imposing the following conditions:

u_{1, 1}^{(2)} (τ) = 0; u_{1, 2}^{(2)} (τ) = 0; u_{1, 3}^{(2)} (τ) = 0;

(171)

u_{2, 1}^{(2)} (0) = 0; u_{2, 2}^{(2)} (0) = 0; u_{2, 3}^{(2)} (0) = 0.

(172)

4.: Using the conditions given in Eqs. (29), (167), (171) and (172) on the right-side of Eq. (170) and rearranging the remaining terms yields the following relation:

\begin{array}{l} - u_{1, 1}^{(2)} (0) δ ψ_{0} - u_{1, 3}^{(2)} (0) δ T_{0} \\ + \int_{0}^{τ} δ h_{1} (t) [- \frac{d u_{1, 1}^{(2)} (t)}{d t} - u_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - u_{1, 2}^{(2)} F_{2} - u_{1, 3}^{(2)} (F_{2} F_{3}) + u_{2, 2}^{(2)} F_{1} F b_{1}^{(1)} (t)] \\ + \int_{0}^{τ} δ h_{2} (t) [- \frac{d u_{1, 2}^{(2)}}{d t} - u_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} u_{1}^{(1)} (t)] d t \\ - \int_{0}^{τ} δ h_{3} (t) [\frac{d u_{1, 3}^{(2)}}{d t} + 2 φ_{2} u_{2, 3}^{(2)} (t) δ (t - τ)] d t \\ + \int_{0}^{τ} δ b_{1}^{(1)} (t) [- \frac{d u_{2, 1}^{(2)}}{d t} + u_{2, 1}^{(2)} F_{1} F_{3} h_{2} (t) + u_{2, 2}^{(2)} F_{1} F_{3} h_{1} (t)] \\ + \int_{0}^{τ} δ b_{2}^{(1)} (t) [- \frac{d u_{2, 2}^{(2)}}{d t} + u_{2, 1}^{(2)} F_{2}] d t + \int_{0}^{τ} δ b_{3}^{(1)} (t) [- \frac{d u_{2, 3}^{(2)}}{d t} + u_{2, 1}^{(2)} F_{2} F_{3}] d t = r^{(2)}, \end{array}

(173)

5.: The integral terms on the left-side of Eq. (173) are now required to represent the “indirect-effect” term defined in Eq. (163), which is achieved by imposing the following requirements on the components of the 2nd-level adjoint sensitivity function $u^{(2)} ≜ {[u_{1}^{(2)} (t), u_{2}^{(2)} (t)]}^{†}$ :

\begin{array}{l} - \frac{d u_{1, 1}^{(2)} (t)}{d t} - u_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - u_{1, 2}^{(2)} (t) F_{2} - u_{1, 3}^{(2)} (t) F_{2} F_{3} + u_{2, 2}^{(2)} (t) F_{1} F_{3} b_{1}^{(1)} (t) \\ ​​​ = F_{3} b_{1}^{(1)} (t) h_{2} (t); \end{array}

(174)

- \frac{d u_{1, 2}^{(2)} (t)}{d t} - u_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} u_{1}^{(1)} (t) = F_{3} b_{1}^{(1)} (t) h_{1} (t)

; (175)

\frac{d u_{1, 3}^{(2)}}{d t} + 2 φ_{2} u_{2, 3}^{(2)} (t) δ (t - τ) = 0

; (176)

- \frac{d u_{2, 1}^{(2)} (t)}{d t} + u_{2, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) + u_{2, 2}^{(2)} (t) F_{1} F_{3} h_{1} (t) = F_{3} h_{1} (t) h_{2} (t)

; (177)

- \frac{d u_{2, 2}^{(2)} (t)}{d t} + u_{2, 1}^{(2)} (t) F_{2} = 0

; (178)

- \frac{d u_{2, 3}^{(2)} (t)}{d t} + u_{2, 1}^{(2)} (t) F_{2} F_{3} = 0

. (179)

The system of equations comprising Eqs. (171), (172), (174)‒(179) constitutes the “2nd-Level Adjoint Sensitivity System (2nd-LASS)” for the 2nd-level adjoint sensitivity function

u^{(2)} ≜ {[u_{1}^{(2)} (t), u_{2}^{(2)} (t)]}^{†}

. Evidently, the 2nd-LASS is linear in

u^{(2)} ≜ {[u_{1}^{(2)} (t), u_{2}^{(2)} (t)]}^{†}

and is independent of parameter variations. Notably, this system of equations does not need to be solved simultaneously, but can be solved sequentially, by first solving Eqs. (177)‒(179) subject to the initial conditions given in Eq. (172) to determine the function

u_{2}^{(2)} (t)

, and subsequently using the function

u_{2}^{(2)} (t)

in Eqs. (174)‒(176) to solve these equations subject to the “final-time” condition given in Eq. (171), to obtain the function

u_{1}^{(2)} (t)

. The 2nd-LASS is to be solved using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity.

6.: Using the results obtained in Eqs. (174)‒(179) in Eq. (173) yields the following alternative expression, in terms of $u^{(2)} ≜ {[u_{1}^{(2)} (t), u_{2}^{(2)} (t)]}^{†}$ , for the “indirect-effect” term $δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d}$ defined in Eq. (163):

δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d} = r^{(2)} + u_{1, 1}^{(2)} (0) δ ψ_{0} + u_{1, 3}^{(2)} (0) δ T_{0}

. (180)

7.: Using in Eq. (180) the definition of the function $r^{(2)}$ provided in Eq. (169) and adding the resulting expression for the indirect-effect term $δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d}$ to the expression for the direct-effect term $δ {\{\partial K (τ) / \partial F_{1}\}}_{d i r}$ provided in Eq. (162) yields the following expression for the total first-order G-differential $δ [\partial K (τ) / \partial F_{1}]$ :

\begin{array}{l} δ [\partial K (τ) / \partial F_{1}] ≜ \frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial F_{1}} δ ψ_{0} + \frac{\partial^{2} K (τ)}{\partial T_{0} \partial F_{1}} δ T_{0} + \sum_{i = 1}^{3} \frac{\partial^{2} K (τ)}{\partial F_{i} \partial F_{1}} δ F_{i}, \\ = (δ F_{3}) \int_{0}^{τ} h_{1} (t) h_{2} (t) b_{1}^{(1)} (t) d t + u_{1, 1}^{(2)} (0) δ ψ_{0} + u_{1, 3}^{(2)} (0) δ T_{0} \\ + \int_{0}^{τ} u_{1, 1}^{(2)} (t) [(δ F_{1}) F_{3} + F_{1} (δ F_{3})] h_{1} (t) h_{2} (t) d t \\ + \int_{0}^{τ} u_{1, 2}^{(2)} (δ F_{2}) h_{1} (t) d t + \int_{0}^{τ} u_{1, 3}^{(2)} [(δ F_{2}) F_{3} + F_{2} (δ F_{3})] h_{1} (t) d t \\ - \int_{0}^{τ} u_{2, 1}^{(2)} \{[(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{2} (t) b_{1}^{(1)} (t) + (δ F_{2}) b_{2}^{(1)} (t) \\ + [(δ F_{2}) F_{3} + (δ F_{3}) F_{2}] b_{3}^{(1)} (t)\} d t - \int_{0}^{τ} u_{2, 2}^{(2)} [(δ F_{1}) F_{3} + (δ F_{3}) F_{1}] h_{1} (t) b_{1}^{(1)} (t) d t \\ - \int_{0}^{τ} u_{2, 3}^{(2)} [δ φ_{1} + 2 δ φ_{2} h_{3} (t)] δ (t - τ) d t . \end{array}

(181)

The expression shown in Eq. (181) is to be evaluated at the nominal values of all functions and parameters/weights but the superscript “zero” (which has been used to indicate this fact) has been omitted for notational simplicity.

Identifying in Eq. (181) the quantities that multiply the respective variations in the initial conditions and feature functions yields the following expressions for the second-order sensitivities stemming from

δ [\partial K (τ) / \partial F_{1}]

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial F_{1}} = u_{1, 1}^{(2)} (0)

; (182)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial F_{1}} = u_{1, 3}^{(2)} (0)

; (183)

\frac{\partial^{2} K (τ)}{\partial F_{1} \partial F_{1}} = F_{3} \int_{0}^{τ} [u_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) - u_{2, 1}^{(2)} (t) h_{2} (t) b_{1}^{(1)} (t) - u_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t)] d t

; (184)

\frac{\partial^{2} K (τ)}{\partial F_{2} \partial F_{1}} = \int_{0}^{τ} u_{1, 2}^{(2)} (t) h_{1} (t) d t + F_{3} \int_{0}^{τ} u_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} u_{2, 1}^{(2)} [b_{2}^{(1)} (t) + F_{3} b_{3}^{(1)} (t)] d t

; (185)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{3} \partial F_{1}} = \int_{0}^{τ} h_{1} (t) h_{2} (t) b_{1}^{(1)} (t) d t + F_{1} \int_{0}^{τ} u_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t \\ + F_{2} \int_{0}^{τ} u_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} u_{2, 1}^{(2)} (t) [F_{1} h_{2} (t) b_{1}^{(1)} (t) + F_{2} b_{3}^{(1)} (t)] d t \\ - F_{1} \int_{0}^{τ} u_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t) d t . \end{array}

. (186)

\frac{\partial^{2} K (τ)}{\partial φ_{1} \partial F_{1}} = - \int_{0}^{τ} u_{2, 3}^{(2)} (t) δ (t - τ) d t = - u_{2, 3}^{(2)} (τ)

; (187)

\frac{\partial^{2} K (τ)}{\partial φ_{2} \partial F_{1}} = - 2 \int_{0}^{τ} u_{2, 3}^{(2)} (t) h_{3} (t) δ (t - τ) d t = - 2 u_{2, 3}^{(2)} (τ) h_{3} (τ)

. (188)

The expressions of the second-order sensitivities represented by Eqs. (182)‒(188) are also to be evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted, for notational simplicity.

B.: Second-order sensitivities stemming from $\partial K (τ) / \partial F_{2}$

The second-order sensitivities stemming from the first-order sensitivity

\partial K (τ) / \partial F_{2}

are obtained from the G-differential

δ \{\partial K [T (τ)] / \partial F_{2}\}

, which is in turn obtained by using Eq. (63), as follows:

\begin{array}{l} δ \{\frac{\partial K (τ)}{\partial F_{2}}\} ≜ {\{\frac{d}{d ε} \int_{0}^{τ} [b_{2}^{(1, 0)} (t) + ε δ b_{2}^{(1)} (t)] [h_{1}^{0} (t) + ε δ h_{1} (t)] d t\}}_{ε = 0} \\ + {\{\frac{d}{d ε} \int_{0}^{τ} [F_{3}^{0} + ε δ F_{3}] [b_{3}^{(1, 0)} (t) + ε δ b_{3}^{(1)} (t)] [h_{1}^{0} (t) + ε δ h_{1} (t)] d t\}}_{ε = 0} \\ = \int_{0}^{τ} δ b_{2}^{(1)} (t) h_{1}^{0} (t) d t + \int_{0}^{τ} δ h_{1} (t) b_{2}^{(1, 0)} (t) d t + (δ F_{3}) \int_{0}^{τ} b_{3}^{(1, 0)} (t) h_{1}^{0} (t) d t \\ + F_{3}^{0} \int_{0}^{τ} δ b_{3}^{(1)} (t) h_{1}^{0} (t) d t + F_{3}^{0} \int_{0}^{τ} δ h_{1} (t) b_{3}^{(1, 0)} (t) d t \\ = δ {\{\partial K (τ) / \partial F_{2}\}}_{d i r} + δ {\{\partial K (τ) / \partial F_{2}\}}_{i n d}, \end{array}

(189)

δ {\{\partial K (τ) / \partial F_{2}\}}_{d i r} ≜ (δ F_{3}) \int_{0}^{τ} b_{3}^{(1, 0)} (t) h_{1}^{0} (t) d t

, (190)

\begin{array}{l} δ {\{\partial K (τ) / \partial F_{2}\}}_{i n d} ≜ \int_{0}^{τ} δ b_{2}^{(1)} (t) h_{1}^{0} (t) d t + \int_{0}^{τ} δ h_{1} (t) b_{2}^{(1, 0)} (t) d t \\ + F_{3}^{0} \int_{0}^{τ} δ b_{3}^{(1)} (t) h_{1}^{0} (t) d t + F_{3}^{0} \int_{0}^{τ} δ h_{1} (t) b_{3}^{(1, 0)} (t) d t . \end{array}

(191)

The indirect-effect term

δ {\{\partial K (τ) / \partial F_{2}\}}_{i n d}

, defined by Eq. (191), is evaluated by following the same procedural steps as used above for determining the second-order sensitivities stemming from

δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d}

. The following expressions are ultimately obtained for the second-order sensitivities stemming from

δ \{\partial K [T (τ)] / \partial F_{2}\}

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial F_{2}} = w_{1, 1}^{(2)} (0)

; (192)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial F_{2}} = w_{1, 3}^{(2)} (0)

; (193)

\frac{\partial^{2} K (τ)}{\partial F_{1} \partial F_{2}} = F_{3} \int_{0}^{τ} [w_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) - w_{2, 1}^{(2)} (t) h_{2} (t) b_{1}^{(1)} (t) - w_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t)] d t

; (194)

\frac{\partial^{2} K (τ)}{\partial F_{2} \partial F_{2}} = \int_{0}^{τ} w_{1, 2}^{(2)} (t) h_{1} (t) d t + F_{3} \int_{0}^{τ} w_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} w_{2, 1}^{(2)} [b_{2}^{(1)} (t) + F_{3} b_{3}^{(1)} (t)] d t

; (195)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{3} \partial F_{2}} = \int_{0}^{τ} b_{3}^{(1)} (t) h_{1} (t) d t + F_{1} \int_{0}^{τ} w_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t \\ + F_{2} \int_{0}^{τ} w_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} w_{2, 1}^{(2)} (t) [F_{1} h_{2} (t) b_{1}^{(1)} (t) + F_{2} b_{3}^{(1)} (t)] d t \\ - F_{1} \int_{0}^{τ} w_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t) d t . \end{array}

. (196)

\frac{\partial^{2} K (τ)}{\partial φ_{1} \partial F_{2}} = - \int_{0}^{τ} w_{2, 3}^{(2)} (t) δ (t - τ) d t = - w_{2, 3}^{(2)} (τ)

; (197)

\frac{\partial^{2} K (τ)}{\partial φ_{2} \partial F_{2}} = - 2 \int_{0}^{τ} w_{2, 3}^{(2)} (t) h_{3} (t) δ (t - τ) d t = - 2 w_{2, 3}^{(2)} (τ) h_{3} (τ)

. (198)

The 2nd-level adjoint sensitivity function

w^{(2)} ≜ {[w_{1}^{(2)} (t), w_{2}^{(2)} (t)]}^{†} \in H_{2} (Ω_{t})

, where

w_{1}^{(2)} (t) ≜ {[w_{1, 1}^{(2)} (t), w_{1, 2}^{(2)} (t), w_{1, 3}^{(2)} (t)]}^{†}

and

w_{2}^{(2)} (t) ≜ {[w_{2, 1}^{(2)} (t), w_{2, 2}^{(2)} (t), w_{2, 3}^{(2)} (t)]}^{†}

, which appears in the expression provided in Eqs. (192)‒(198) is the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):

\begin{array}{l} - \frac{d w_{1, 1}^{(2)} (t)}{d t} - w_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - w_{1, 2}^{(2)} (t) F_{2} - w_{1, 3}^{(2)} (t) F_{2} F_{3} + w_{2, 2}^{(2)} (t) F_{1} F_{3} b_{1}^{(1)} (t) \\ ​​​ = b_{2}^{(1)} (t) + F_{3} b_{3}^{(1)} (t); \end{array}

(199)

- \frac{d w_{1, 2}^{(2)} (t)}{d t} - w_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} b_{1}^{(1)} (t) = 0

; (200)

\frac{d w_{1, 3}^{(2)} (t)}{d t} + 2 φ_{2} w_{2, 3}^{(2)} (t) δ (t - τ) = 0

; (201)

- \frac{d w_{2, 1}^{(2)} (t)}{d t} + w_{2, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) + w_{2, 2}^{(2)} (t) F_{1} F_{3} h_{1} (t) = 0

; (202)

- \frac{d w_{2, 2}^{(2)} (t)}{d t} + w_{2, 1}^{(2)} (t) F_{2} = h_{1} (t)

; (203)

- \frac{d w_{2, 3}^{(2)} (t)}{d t} + w_{2, 1}^{(2)} (t) F_{2} F_{3} = F_{3} h_{1} (t)

; (204)

w_{1, 1}^{(2)} (τ) = 0; w_{1, 2}^{(2)} (τ) = 0; w_{1, 3}^{(2)} (τ) = 0;

(205)

w_{2, 1}^{(2)} (0) = 0; w_{2, 2}^{(2)} (0) = 0; w_{2, 3}^{(2)} (0) = 0.

(206)

The 2nd-LASS represented by Eqs. (199)‒(206) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. Furthermore, the expressions of the second-order sensitivities represented by Eqs. (192)‒(198) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity.

C.: Second-order sensitivities stemming from $\partial K (τ) / \partial F_{3}$

The second-order sensitivities stemming from the first-order sensitivity

\partial K (τ) / \partial F_{3}

are obtained from the G-differential

δ \{\partial K [T (τ)] / \partial F_{3}\}

\partial K (τ) / \partial F_{3}

, which is obtained using Eq. (64) as follows:

\begin{array}{l} δ \{\frac{\partial K (τ)}{\partial F_{3}}\} ≜ {\{\frac{d}{d ε} \int_{0}^{τ} [F_{2}^{0} + ε δ F_{2}] [b_{3}^{(1, 0)} (t) + ε δ b_{3}^{(1)} (t)] [h_{1}^{0} (t) + ε δ h_{1} (t)] d t\}}_{ε = 0} \\ + {\{\frac{d}{d ε} \int_{0}^{τ} [F_{1}^{0} + ε δ F_{1}] [b_{1}^{(1, 0)} (t) + ε δ b_{1}^{(1)} (t)] [h_{2}^{0} (t) + ε δ h_{2} (t)] [h_{1}^{0} (t) + ε δ h_{1} (t)] d t\}}_{ε = 0} \\ = (δ F_{2}) \int_{0}^{τ} b_{3}^{(1, 0)} (t) h_{1}^{0} (t) d t + F_{2}^{0} \int_{0}^{τ} h_{1}^{0} (t) δ b_{3}^{(1)} (t) d t + F_{2}^{0} \int_{0}^{τ} b_{3}^{(1, 0)} (t) δ h_{1} (t) d t \\ + (δ F_{1}) \int_{0}^{τ} b_{1}^{(1, 0)} (t) h_{2}^{0} (t) h_{1}^{0} (t) d t + F_{1}^{0} \int_{0}^{τ} δ b_{1}^{(1)} (t) h_{2}^{0} (t) h_{1}^{0} (t) d t \\ + F_{1}^{0} \int_{0}^{τ} b_{1}^{(1, 0)} (t) δ h_{2} (t) h_{1}^{0} (t) d t + F_{1}^{0} \int_{0}^{τ} b_{1}^{(1, 0)} (t) h_{2}^{0} (t) δ h_{1} (t) d t \\ = δ {\{\partial K (τ) / \partial F_{3}\}}_{d i r} + δ {\{\partial K (τ) / \partial F_{3}\}}_{i n d} . \end{array}

(207)

The superscript “zero” in Eq. (207) indicates, as before, that the respective quantities are to be evaluated at their nominal values. The following definitions were used for the “direct-effect” and, respectively, “indirect-effect” terms in Eq. (207):

δ {\{\partial K (τ) / \partial F_{3}\}}_{d i r} ≜ (δ F_{1}) \int_{0}^{τ} b_{1}^{(1, 0)} (t) h_{1}^{0} (t) h_{2}^{0} (t) d t + (δ F_{2}) \int_{0}^{τ} b_{3}^{(1, 0)} (t) h_{1}^{0} (t) d t

, (208)

\begin{array}{l} δ {\{\partial K (τ) / \partial F_{3}\}}_{i n d} ≜ F_{1}^{0} \int_{0}^{τ} δ b_{1}^{(1)} (t) h_{2}^{0} (t) h_{1}^{0} (t) d t + F_{1}^{0} \int_{0}^{τ} b_{1}^{(1, 0)} (t) δ h_{2} (t) h_{1}^{0} (t) d t \\ + F_{1}^{0} \int_{0}^{τ} b_{1}^{(1, 0)} (t) h_{2}^{0} (t) δ h_{1} (t) d t + F_{2}^{0} \int_{0}^{τ} h_{1}^{0} (t) δ b_{3}^{(1)} (t) d t + F_{2}^{0} \int_{0}^{τ} b_{3}^{(1, 0)} (t) δ h_{1} (t) d t . \end{array}

(209)

The indirect-effect term defined by Eq. (209) is evaluated by following the same procedural steps as previously described when determining the second-order sensitivities stemming from

δ {\{\partial K (τ) / \partial F_{1}\}}_{i n d}

and from

δ {\{\partial K (τ) / \partial F_{2}\}}_{i n d}

. The following expressions are ultimately obtained for the second-order sensitivities stemming from

δ \{\partial K [T (τ)] / \partial F_{3}\}

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial F_{3}} = z_{1, 1}^{(2)} (0)

; (210)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial F_{3}} = z_{1, 3}^{(2)} (0)

; (211)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{1} \partial F_{3}} = \int_{0}^{τ} b_{1}^{(1)} (t) h_{1} (t) h_{2} (t) d t \\ + F_{3} \int_{0}^{τ} [z_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) - z_{2, 1}^{(2)} (t) h_{2} (t) b_{1}^{(1)} (t) - z_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t)] d t; \end{array}

(212)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{2} \partial F_{3}} = \int_{0}^{τ} b_{3}^{(1)} (t) h_{1} (t) d t \\ + \int_{0}^{τ} z_{1, 2}^{(2)} (t) h_{1} (t) d t + F_{3} \int_{0}^{τ} z_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} z_{2, 1}^{(2)} [b_{2}^{(1)} (t) + F_{3} b_{3}^{(1)} (t)] d t \end{array}

(213)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{3} \partial F_{3}} = F_{1} \int_{0}^{τ} z_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t + F_{2} \int_{0}^{τ} z_{1, 3}^{(2)} (t) h_{1} (t) d t \\ - \int_{0}^{τ} z_{2, 1}^{(2)} (t) [F_{1} h_{2} (t) b_{1}^{(1)} (t) + F_{2} b_{3}^{(1)} (t)] d t - F_{1} \int_{0}^{τ} z_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t) d t . \end{array}

(214)

\frac{\partial^{2} K (τ)}{\partial φ_{1} \partial F_{3}} = - \int_{0}^{τ} z_{2, 3}^{(2)} (t) δ (t - τ) d t = - z_{2, 3}^{(2)} (τ)

; (215)

\frac{\partial^{2} K (τ)}{\partial φ_{2} \partial F_{3}} = - 2 \int_{0}^{τ} z_{2, 3}^{(2)} (t) h_{3} (t) δ (t - τ) d t = - 2 z_{2, 3}^{(2)} (τ) h_{3} (τ)

. (216)

The 2nd-level adjoint sensitivity function

z^{(2)} ≜ {[z_{1}^{(2)} (t), z_{2}^{(2)} (t)]}^{†} \in H_{2} (Ω_{t})

, where

z_{1}^{(2)} (t) ≜ {[z_{1, 1}^{(2)} (t), z_{1, 2}^{(2)} (t), z_{1, 3}^{(2)} (t)]}^{†}

and

z_{2}^{(2)} (t) ≜ {[z_{2, 1}^{(2)} (t), z_{2, 2}^{(2)} (t), z_{2, 3}^{(2)} (t)]}^{†}

, which appears in the expressions provided in Eqs. (210)‒(216) is the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):

\begin{array}{l} - \frac{d z_{1, 1}^{(2)} (t)}{d t} - z_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - z_{1, 2}^{(2)} (t) F_{2} - z_{1, 3}^{(2)} (t) F_{2} F_{3} + z_{2, 2}^{(2)} (t) F_{1} F_{3} b_{1}^{(1)} (t) \\ ​​​ = F_{2} b_{3}^{(1)} (t); \end{array}

(217)

- \frac{d z_{1, 2}^{(2)} (t)}{d t} - z_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} b_{1}^{(1)} (t) = F_{1} b_{1}^{(1)} (t) h_{1} (t)

; (218)

\frac{d z_{1, 3}^{(2)} (t)}{d t} + 2 φ_{2} z_{2, 3}^{(2)} (t) δ (t - τ) = 0

; (219)

- \frac{d z_{2, 1}^{(2)} (t)}{d t} + z_{2, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) + z_{2, 2}^{(2)} (t) F_{1} F_{3} h_{1} (t) = F_{1} h_{2} (t) h_{1} (t)

; (220)

- \frac{d z_{2, 2}^{(2)} (t)}{d t} + z_{2, 1}^{(2)} (t) F_{2} = 0

; (221)

- \frac{d z_{2, 3}^{(2)} (t)}{d t} + z_{2, 1}^{(2)} (t) F_{2} F_{3} = F_{2} h_{1} (t)

; (222)

z_{1, 1}^{(2)} (τ) = 0; z_{1, 2}^{(2)} (τ) = 0; z_{1, 3}^{(2)} (τ) = 0;

(223)

z_{2, 1}^{(2)} (0) = 0; z_{2, 2}^{(2)} (0) = 0; z_{2, 3}^{(2)} (0) = 0.

(224)

The 2nd-LASS represented by Eqs. (217)‒(224) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. Furthermore, the expressions of the second-order sensitivities represented by Eqs. (210)‒(216) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity.

4.1.3. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Initial Conditions $ψ_{0}$ and $T_{0}$

A.: Second-order sensitivities stemming from $\partial K (τ) / \partial ψ_{0}$

The second-order sensitivities stemming from the first-order sensitivity

\partial K (τ) / \partial ψ_{0}

are determined from the G-differential

δ \{\partial K [T (τ)] / \partial ψ_{0}\}

\partial K (τ) / \partial ψ_{0}

, which is obtained using Eq. (60) as follows:

δ \{\frac{\partial K (τ)}{\partial ψ_{0}}\} ≜ {\{\frac{d}{d ε} \int_{0}^{τ} [b_{1}^{(1, 0)} (t) + ε δ b_{1}^{(1)} (t)] δ (t) d t\}}_{ε = 0} = \int_{0}^{τ} δ b_{1}^{(1)} (t) δ (t) d t

. (225)

The second-order sensitivities stemming from the expression of

δ \{\partial K [T (τ)] / \partial ψ_{0}\}

obtained in Eq. (225) are determined by following the same procedural steps as described in Subsection 4.1.2. The following expressions are ultimately obtained for the second-order sensitivities stemming from

δ \{\partial K [T (τ)] / \partial ψ_{0}\}

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial ψ_{0}} = ξ_{1, 1}^{(2)} (0)

; (226)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial ψ_{0}} = ξ_{1, 3}^{(2)} (0)

; (227)

\frac{\partial^{2} K (τ)}{\partial F_{1} \partial ψ_{0}} = F_{3} \int_{0}^{τ} [ξ_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) - ξ_{2, 1}^{(2)} (t) h_{2} (t) b_{1}^{(1)} (t) - ξ_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t)] d t;

(228)

\frac{\partial^{2} K (τ)}{\partial F_{2} \partial ψ_{0}} = \int_{0}^{τ} ξ_{1, 2}^{(2)} (t) h_{1} (t) d t + F_{3} \int_{0}^{τ} ξ_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} ξ_{2, 1}^{(2)} [b_{2}^{(1)} (t) + F_{3} b_{3}^{(1)} (t)] d t

; (229)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{3} \partial ψ_{0}} = F_{1} \int_{0}^{τ} ξ_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t + F_{2} \int_{0}^{τ} ξ_{1, 3}^{(2)} (t) h_{1} (t) d t \\ - \int_{0}^{τ} ξ_{2, 1}^{(2)} (t) [F_{1} h_{2} (t) b_{1}^{(1)} (t) + F_{2} b_{3}^{(1)} (t)] d t - F_{1} \int_{0}^{τ} ξ_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t) d t . \end{array}

(230)

\frac{\partial^{2} K (τ)}{\partial φ_{1} \partial ψ_{0}} = - \int_{0}^{τ} ξ_{2, 3}^{(2)} (t) δ (t - τ) d t = - ξ_{2, 3}^{(2)} (τ)

; (231)

\frac{\partial^{2} K (τ)}{\partial φ_{2} \partial ψ_{0}} = - 2 \int_{0}^{τ} ξ_{2, 3}^{(2)} (t) h_{3} (t) δ (t - τ) d t = - 2 ξ_{2, 3}^{(2)} (τ) h_{3} (τ)

. (232)

The 2nd-level adjoint sensitivity function

ξ^{(2)} ≜ {[ξ_{1}^{(2)} (t), ξ_{2}^{(2)} (t)]}^{†} \in H_{2} (Ω_{t})

, where

ξ_{1}^{(2)} (t) ≜ {[ξ_{1, 1}^{(2)} (t), ξ_{1, 2}^{(2)} (t), ξ_{1, 3}^{(2)} (t)]}^{†}

and

ξ_{2}^{(2)} (t) ≜ {[ξ_{2, 1}^{(2)} (t), ξ_{2, 2}^{(2)} (t), ξ_{2, 3}^{(2)} (t)]}^{†}

, which appears in the expressions provided in Eqs. (226)‒(230) is the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):

- \frac{d ξ_{1, 1}^{(2)} (t)}{d t} - ξ_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - ξ_{1, 2}^{(2)} (t) F_{2} - ξ_{1, 3}^{(2)} (t) F_{2} F_{3} + ξ_{2, 2}^{(2)} (t) F_{1} F_{3} b_{1}^{(1)} (t) = 0

; (233)

- \frac{d ξ_{1, 2}^{(2)} (t)}{d t} - ξ_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} b_{1}^{(1)} (t) = 0

; (234)

\frac{d ξ_{1, 3}^{(2)} (t)}{d t} = 0

; (235)

- \frac{d ξ_{2, 1}^{(2)} (t)}{d t} + ξ_{2, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) + ξ_{2, 2}^{(2)} (t) F_{1} F_{3} h_{1} (t) = δ (t)

; (236)

- \frac{d ξ_{2, 2}^{(2)} (t)}{d t} + ξ_{2, 1}^{(2)} (t) F_{2} = 0

; (237)

- \frac{d ξ_{2, 3}^{(2)} (t)}{d t} + ξ_{2, 1}^{(2)} (t) F_{2} F_{3} = 0

; (238)

ξ_{1, 1}^{(2)} (τ) = 0; ξ_{1, 2}^{(2)} (τ) = 0; ξ_{1, 3}^{(2)} (τ) = 0;

(239)

ξ_{2, 1}^{(2)} (0) = 0; ξ_{2, 2}^{(2)} (0) = 0; ξ_{2, 3}^{(2)} (0) = 0.

(240)

The 2nd-LASS represented by Eqs. (233)‒(240) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. The expressions of the second-order sensitivities represented by Eqs. (226)‒(230) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity. In particular, Eqs. (235) and (239) imply that

ξ_{1, 3}^{(2)} (t) \equiv 0

, so that the general expressions shown in Eqs. (227), (229) and (230) simplify accordingly.

B.: Second-order sensitivities stemming from $\partial K (τ) / \partial T_{0}$

The second-order sensitivities stemming from the first-order sensitivity

\partial K (τ) / \partial T_{0}

are obtained from the G-differential

δ \{\partial K [T (τ)] / \partial T_{0}\}

\partial K (τ) / \partial T_{0}

, which is obtained using Eq. (61) as follows:

δ \{\frac{\partial K (τ)}{\partial T_{0}}\} ≜ {\{\frac{d}{d ε} \int_{0}^{τ} [b_{3}^{(1, 0)} (t) + ε δ b_{3}^{(1)} (t)] δ (t) d t\}}_{ε = 0} = \int_{0}^{τ} δ b_{3}^{(1)} (t) δ (t) d t

. (241)

The second-order sensitivities stemming from the expression of

δ \{\partial K [T (τ)] / \partial T_{0}\}

δ \{\partial K [T (τ)] / \partial T_{0}\}

\frac{\partial^{2} K (τ)}{\partial ψ_{0} \partial T_{0}} = ζ_{1, 1}^{(2)} (0)

; (242)

\frac{\partial^{2} K (τ)}{\partial T_{0} \partial T_{0}} = ζ_{1, 3}^{(2)} (0)

; (243)

\frac{\partial^{2} K (τ)}{\partial F_{1} \partial T_{0}} = F_{3} \int_{0}^{τ} [ζ_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) - ζ_{2, 1}^{(2)} (t) h_{2} (t) b_{1}^{(1)} (t) - ζ_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t)] d t;

(244)

\frac{\partial^{2} K (τ)}{\partial F_{2} \partial T_{0}} = \int_{0}^{τ} ζ_{1, 2}^{(2)} (t) h_{1} (t) d t + F_{3} \int_{0}^{τ} ζ_{1, 3}^{(2)} (t) h_{1} (t) d t - \int_{0}^{τ} ζ_{2, 1}^{(2)} [b_{2}^{(1)} (t) + F_{3} b_{3}^{(1)} (t)] d t

; (245)

\begin{array}{l} \frac{\partial^{2} K (τ)}{\partial F_{3} \partial T_{0}} = F_{1} \int_{0}^{τ} ζ_{1, 1}^{(2)} (t) h_{1} (t) h_{2} (t) d t + F_{2} \int_{0}^{τ} ζ_{1, 3}^{(2)} (t) h_{1} (t) d t \\ - \int_{0}^{τ} ζ_{2, 1}^{(2)} (t) [F_{1} h_{2} (t) b_{1}^{(1)} (t) + F_{2} b_{3}^{(1)} (t)] d t - F_{1} \int_{0}^{τ} ζ_{2, 2}^{(2)} (t) h_{1} (t) b_{1}^{(1)} (t) d t . \end{array}

(246)

\frac{\partial^{2} K (τ)}{\partial φ_{1} \partial T_{0}} = - \int_{0}^{τ} ζ_{2, 3}^{(2)} (t) δ (t - τ) d t = - ζ_{2, 3}^{(2)} (τ)

; (247)

\frac{\partial^{2} K (τ)}{\partial φ_{2} \partial T_{0}} = - 2 \int_{0}^{τ} ζ_{2, 3}^{(2)} (t) h_{3} (t) δ (t - τ) d t = - 2 ζ_{2, 3}^{(2)} (τ) h_{3} (τ)

. (248)

The 2nd-level adjoint sensitivity function

ζ^{(2)} ≜ {[ζ_{1}^{(2)} (t), ζ_{2}^{(2)} (t)]}^{†} \in H_{2} (Ω_{t})

, where

ζ_{1}^{(2)} (t) ≜ {[ζ_{1, 1}^{(2)} (t), ζ_{1, 2}^{(2)} (t), ζ_{1, 3}^{(2)} (t)]}^{†}

and

ζ_{2}^{(2)} (t) ≜ {[ζ_{2, 1}^{(2)} (t), ζ_{2, 2}^{(2)} (t), ζ_{2, 3}^{(2)} (t)]}^{†}

, which appears in the expressions provided in Eqs. (226)‒(230) is the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):

- \frac{d ζ_{1, 1}^{(2)} (t)}{d t} - ζ_{1, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) - ζ_{1, 2}^{(2)} (t) F_{2} - ζ_{1, 3}^{(2)} (t) F_{2} F_{3} + ζ_{2, 2}^{(2)} (t) F_{1} F_{3} b_{1}^{(1)} (t) = 0

; (249)

- \frac{d ζ_{1, 2}^{(2)} (t)}{d t} - ζ_{1, 1}^{(2)} (t) F_{1} F_{3} h_{1} (t) + F_{1} F_{3} b_{1}^{(1)} (t) = 0

; (250)

\frac{d ζ_{1, 3}^{(2)} (t)}{d t} = 0

; (251)

- \frac{d ζ_{2, 1}^{(2)} (t)}{d t} + ζ_{2, 1}^{(2)} (t) F_{1} F_{3} h_{2} (t) + ζ_{2, 2}^{(2)} (t) F_{1} F_{3} h_{1} (t) = 0

; (252)

- \frac{d ζ_{2, 2}^{(2)} (t)}{d t} + ζ_{2, 1}^{(2)} (t) F_{2} = 0

; (253)

- \frac{d ζ_{2, 3}^{(2)} (t)}{d t} + ζ_{2, 1}^{(2)} (t) F_{2} F_{3} = δ (t)

; (254)

ζ_{1, 1}^{(2)} (τ) = 0; ζ_{1, 2}^{(2)} (τ) = 0; ζ_{1, 3}^{(2)} (τ) = 0;

(255)

ζ_{2, 1}^{(2)} (0) = 0; ζ_{2, 2}^{(2)} (0) = 0; ζ_{2, 3}^{(2)} (0) = 0.

(256)

ξ_{1, 3}^{(2)} (t) \equiv 0

, so that the general expressions shown in Eqs. (227), (229) and (230) simplify accordingly.

4.2. Second-Order Sensitivities of the Thermal Conductivity Response $K (τ)$ Computed Using the Corresponding Single NODE-Equation

Alternatively, the second-order sensitivities of the response

K (τ)

can be computed by differentiating analytically the single-NODE expressions which were obtained in Eq. (89) for the first-order sensitivities of

K (τ)

. This procedure yields the following expressions for the second-order sensitivities

\partial^{2} K (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

\frac{\partial^{2} K (τ)}{\partial p_{j} \partial φ_{0}} = 0; \frac{\partial^{2} K (τ)}{\partial p_{j} \partial φ_{1}} = \frac{\partial T (τ)}{\partial p_{j}}; \frac{\partial^{2} K (τ)}{\partial p_{j} \partial φ_{2}} = 2 T (τ) \frac{\partial T (τ)}{\partial p_{j}}; j = 1, ..., 7

(257)

\frac{\partial^{2} K (τ)}{\partial p_{j} \partial p_{i}} = \frac{\partial φ_{0}}{\partial p_{j}} + \frac{\partial}{\partial p_{j}} \{[φ_{1} + 2 φ_{2} T (τ)] \frac{\partial T (τ)}{\partial p_{i}}\}; i, j = 1, ..., 7

. (258)

For evaluating the expressions provided in Eqs. (257) and (258), the following expressions, previously obtained, are to be used: (i) the expressions for the first-order sensitivities

\partial T (τ) / \partial p_{i}

i = 1, ..., 7

, provided in Eq. (86); and (ii) the expressions for the second-order sensitivities

\partial^{2} T (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

, provided in Eq. (141).

4.3. Computing the Second-Order Sensitivities of the Thermal Conductivity Response $K (τ)$ : Using the Coupled-NODE vesus the Single-NODE Equations

Computing all of the second-order sensitivities

\partial^{2} K (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

,using the coupled-NODE equations requires solving 39 differential equations (the total number of differential equations included in eight 2nd-LASS), as follows:

(i): 3x3=9 differential equations for obtaining the second-order sensitivities of $K (τ)$ with respect to the decoder weights $φ_{0}$ , $φ_{1}$ and $φ_{2}$ .
(ii): 3x6=18 differential equations for obtaining the second-order sensitivities of $K (τ)$ with respect to the feature functions $F_{1}$ , $F_{2}$ and $F_{3}$ .
(iii): 2x6=12 differential equations for obtaining the second-order sensitivities of $K (τ)$ with respect to the initial conditions $ψ_{0}$ and $T_{0}$ Notably, the same operator appears on the left-side of each of the eight 2nd-LASS, i.e., all of these 2nd-LASS have the following operator form: $A y_{m} = q_{m}$ , $m = 1, ..., 8$ . Only the source terms, symbolically represented by $q_{m}$ , which appear on the right sides of these 8 2nd-LASS, differ from each other. Therefore, if the inverse operator $A^{- 1}$ could be stored, then the respective coupled differential equations could be easily solved to obtain $y_{m} = A^{- 1} q_{m}$ . Storing the operator $A^{- 1}$ may be feasible for small problems/systems but becomes impractical for large systems.

The mixed sensitivities

\partial^{2} K (τ) / \partial p_{j} \partial p_{i}

are computed twice, using distinct 2nd-order adjoint functions, which provides a stringent test for verifying the accuracy of the computations needed for solving the 2nd-LASS involved in determining the respective 2nd-order adjoint functions.

On the other hand, computing all of the second-order sensitivities

\partial^{2} K (τ) / \partial p_{j} \partial p_{i}

i, j = 1, ..., 7

using the single-NODE equations requires solving just 2x2=4 differential equations, namely:

(i): solving Eqs. (126) and (127) for obtaining the 2nd-level adjoint sensitivity function $e_{1}^{(2)} (t) ≜ {[e_{11}^{(2)} (t), e_{12}^{(2)} (t)]}^{†}$ , which is required for computing $\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{1}$ and $\partial^{2} E (τ) / \partial Φ_{1} \partial Φ_{2}$ ; and
(ii): solving Eqs. (135) and (136) for obtaining the 2nd-level adjoint sensitivity function $e_{2}^{(2)} (t) ≜ {[e_{21}^{(2)} (t), e_{22}^{(2)} (t)]}^{†}$ , which is required for computing $\partial^{2} E (τ) / \partial Φ_{2} \partial Φ_{1}$ and $\partial^{2} E (τ) / \partial Φ_{2} \partial Φ_{2}$ .

The above considerations clearly highlight the advantages of decoupling the original NODE system whenever possible, to obtain and solve equations for single dependent variables, such as

E (t)

, rather than solving the entire system of coupled equations simultaneously.

5. Discussion and Conclusions

A new sensitivity analysis methodology, called the “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations” and abbreviated as “2nd-FASAM-NODE” has been developed in [1]. This work has illustrated the application of the 2nd-FASAM-NODE to a paradigm benchmark model, called the Nordheim-Fuchs phenomenological model for reactor safety [2,3]. This phenomenological model describes a short-time self-limiting power transient in a nuclear reactor system having a negative temperature coefficient in which a large amount of reactivity is suddenly inserted, either intentionally or by accident. The Nordheim-Fuchs model responses analyzed in this work included the model’s state functions/variables (namely: the time-dependent total energy released per cm³, the reactor’s time-dependent temperature and the reactor’s time dependent neutron flux),and the reactor’s time-dependent thermal conductivity, which is a representative model response involving decoder weights. All of the parameters underlying the Nordheim-Fuchs phenomenological model are subject to uncertainties.

“Large-scale” computations are those needed to solve systems of equations (algebraic, differential, integral) such as those underlying the original model and the adjoint sensitivity systems of various levels (1st-LASS, 2nd-LASS, etc.). By comparison, the evaluation of integrals, such as those expressing then various sensitivities, by means of quadrature formulas are “small-scale” computations. For large-scale systems involving many parameters, the conventional (e.g., “statistical” or “finite-difference”) methods are impractical for computing response sensitivities higher than first-order. The 2nd-FASAM-NODE methodology provides the most efficient means of computing the exact expressions of the second-order sensitivities of a model-decoder response with respect to the underlying model parameters and initial conditions, requiring at most as many large-scale computations as there are features/functions of model parameters. Importantly, the mixed second-order sensitivities are computed twice, using distinct adjoint functions, thus providing an intrinsic mechanism for verifying the accuracy of the respective first- and second-level adjoint functions. Ongoing research aims at developing similarly efficient methodologies for performing high-order sensitivity analysis of integral and integro-differential equation neural networks.

Funding

This research received no external funding.

Conflicts of Interest

The author declares no conflict of interest.

References

Cacuci, D.G. Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations. I: Mathematical Framework. Submitted to Processes, 2024.
Lamarsh, J.R. Introduction to Nuclear Reactor Theory; Adison-Wesley Publishing Co.: Reading, MA, USA, 1966; pp. 491–492.
Hetrick, D.L. Dynamics of Nuclear Reactors; American Nuclear Society, Inc.: La Grange Park, IL, USA, 1993; pp. 164–174. [Google Scholar]
Chen, R.T.Q.; Rubanova, Y.; Bettencourt, J.; Duvenaud, D.K. Neural ordinary differential equations. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: New York, NY, USA, 2018; Volume 31, pp. 6571–6583. [Google Scholar] [CrossRef]
Cacuci, D.G. Illustrative Application of the nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems to the Nordheim–Fuchs Reactor Dynamics/Safety Model. J. Nucl. Eng. 2022, 3, 191–221. [Google Scholar] [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations. II: Illustrative Application to Heat and Energy Transfer in the Nordheim-Fuchs Phenomenological Model for Reactor Safety

Abstract

1. Introduction

2. Illustrative Application of the First-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (1st-FASAM-NODE) to the Nordheim-Fuchs Reactor Dynamics/Safety Model

2.1. NODE-Modeling of the Nordheim-Fuchs Reactor Dynamics/Safety Phenomenological Model

2.2. Representative Application of the 1st-FASAM-NODE to Compute Most Efficiently the Exact Expressions of the First-Order Sensitivities of Model State Functions to Uncertain Parameters

2.3. Representative Application of the 1st-FASAM-NODE to Compute Most Efficiently the Exact Expressions of First-Order Sensitivities of a Typical Response Involving Decoder-Weights

2.4. Discussion: Minimizing the Number of Computations for Evaluating the Exact Expressions of First-Order Sensitivities of Model Responses to Model Parameters

3. Illustrative Application of the 2nd-FASAM-NODE Methodology to Compute Second-Order Sensitivities of the Nordheim-Fuchs Model’s Dependent/State Variables with Respect to the Underlying Parameters

3.1. Computation of Second-Order Sensitivities of E ( τ ) Using the Coupled NODE-Equations

3.2. Computation of Second-Order Sensitivities of E ( τ ) Using the Corresponding Single NODE-Equation

3.2.1. Computation of Second-Order Sensitivities of E ( τ ) Stemming from ∂ E τ / ∂ Φ 1

3.2.2. Computation of Second-Order Sensitivities of E ( τ ) Stemming from ∂ E τ / ∂ Φ 2

4. Application of the 2nd-FASAM-NODE Methodology to Compute Second-Order Sensitivities of an Illustrative Nordheim-Fuchs Model Response Involving Decoder-Weights

4.1. Second-Order Sensitivities of the Thermal Conductivity Response K τ Computed Using the Coupled NODE-Equations

4.1.1. Second-Order Sensitivities Stemming from the First-Order Sensitivities of K τ with Respect to the Decoder Weights φ 0 , φ 1 and φ 2

4.1.2. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Feature Functions F 1 , F 2 and F 3

4.1.3. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Initial Conditions ψ 0 and T 0

4.2. Second-Order Sensitivities of the Thermal Conductivity Response K τ Computed Using the Corresponding Single NODE-Equation

4.3. Computing the Second-Order Sensitivities of the Thermal Conductivity Response K τ : Using the Coupled-NODE vesus the Single-NODE Equations

5. Discussion and Conclusions

Funding

Conflicts of Interest

References

MDPI Initiatives

Important Links

Subscribe

3.1. Computation of Second-Order Sensitivities of $E (τ)$ Using the Coupled NODE-Equations

3.2. Computation of Second-Order Sensitivities of $E (τ)$ Using the Corresponding Single NODE-Equation

3.2.1. Computation of Second-Order Sensitivities of $E (τ)$ Stemming from $\partial E (τ) / \partial Φ_{1}$

3.2.2. Computation of Second-Order Sensitivities of $E (τ)$ Stemming from $\partial E (τ) / \partial Φ_{2}$

4.1. Second-Order Sensitivities of the Thermal Conductivity Response $K (τ)$ Computed Using the Coupled NODE-Equations

4.1.1. Second-Order Sensitivities Stemming from the First-Order Sensitivities of $K (τ)$ with Respect to the Decoder Weights $φ_{0}$ , $φ_{1}$ and $φ_{2}$

4.1.2. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Feature Functions $F_{1}$ , $F_{2}$ and $F_{3}$

4.1.3. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Initial Conditions $ψ_{0}$ and $T_{0}$

4.2. Second-Order Sensitivities of the Thermal Conductivity Response $K (τ)$ Computed Using the Corresponding Single NODE-Equation

4.3. Computing the Second-Order Sensitivities of the Thermal Conductivity Response $K (τ)$ : Using the Coupled-NODE vesus the Single-NODE Equations