Jeffreys Divergence and Generalized Fisher Information Measures on Fokker-Planck Space-Time Random Field

Preprint

Article

Jeffreys Divergence and Generalized Fisher Information Measures on Fokker-Planck Space-Time Random Field

Altmetrics

Downloads

101

Views

Comments

A peer-reviewed article of this preprint also exists.

Jiaxing Zhang^*

This version is not peer-reviewed

Submitted:

24 August 2023

Posted:

24 August 2023

You are already at the latest version

Alerts

Abstract

In this paper, we derive Jeffreys divergence, generalized Fisher divergence and corresponding De Bruijn identities on space-time random field. First, we determine the relation between Fisher information on the space-time random field in one of the space-time points and the ratio of Jeffreys divergence on a space-time random field at distinct space-time positions to the square of coordinate difference. In addition, we also find identities between the partial derivative of the Jeffreys divergence and the generalized Fisher divergence with respect to space-time variables, i.e. the De Bruijn identities, between two space-time random fields obtained by different parameters under the same Fokker-Planck equations. At the end of this paper, we present three examples of the Fokker-Planck equations on space-time random fields, identify their density functions, and derive the Jeffreys divergence, generalized Fisher information, generalized Fisher divergence, and accompanying De Bruijn identities.

Keywords:

Subject: Computer Science and Mathematics - Applied Mathematics

1. Introduction

Information entropy and Fisher information are quantities to measure random information, and entropy divergence is derived from information entropy to measure the difference between two probability distributions. Formally, we can construct straightforward definitions of entropy divergence and Fisher information for the case of a space-time random field found on classical definitions. The density function, in their definitions, can be obtained in many different ways. In this paper, the density function of a space-time random field is obtained by Fokker-Planck equations. The traditional Fokker-Planck equation is a partial differential equation that describes the probability density function of a random process[1]. It describes the density function’s time-varying change rule. However, Fokker-Planck equations for random fields, particularly space-time random fields, does not have a clear form so far. The classical equation needs to be generalized because the variable varies from time to space-time.

In this paper, we mainly obtain the relation between Jeffreys divergence and generalized Fisher information for space-time random field generated by Fokker-Planck equations. Jeffreys divergence is a symmetric entropy divergence, which is generalized from Kullback-Leibler divergence (KL divergence). In information theory and statistics, Jeffreys divergence is often used to measure the distance between predicted and true distributions, but its drawback is that when there is less overlap between the two distributions, the result is infinity. In order to avoid the divergence results, we consider the relationship between Jeffreys divergence and generalized Fisher information for space-time random field with small differences in space-time parameters.

Moreover, the classical De Bruijn identity describes the relationship between differential entropy and Fisher information of Gaussian channel[2], and it can be generalized other cases[3,4,5,6,7]. Thanks their works and following their ideas, we obtain De Bruijn identities on Jeffreys divergence and generalized Fisher information of two space-time random fields whose density function satisfies Fokker-Planck equations.

1.1. Space-time random field

Random field was first studied by Kolmogorov[8,9,10] and it was gradually improved by Yaglom[11,12,13] in the middle of last century. A random field with

n \in N_{+}

variables can be expressed as

X (t_{1}, t_{2}, \dots, t_{n})

(1)

where

(t_{1}, t_{2}, \dots, t_{n}) \in R^{n}

. We call (1) as a generalized random field, or a multiparameter stochastic process. In some practical applications, we often use the concept of space-time random field. The space-time random field on a d-dimensional space is expressed as

X (t, x)

(2)

where

(t, x) \in R_{+} \times R^{d}

are the space-time variables. It has many applications in statistics, finance, signal processing, stochastic partial differential equations and other fields[14,15,16,17,18,19,20,21,22,23,24,25,26,27].

1.2. Kramers-Moyal expansion and Fokker-Planck equation

In the literature of stochastic processes, Kramers-Moyal expansion refers to a Taylor series of the master equation, named after Kramers and Moyal[28,29]. The Kramers-Moyal expansion is an infinite order partial differential equation

\frac{\partial p (u, t)}{\partial t} = \sum_{n = 1}^{\infty} \frac{{(- 1)}^{n}}{n!} \frac{\partial^{n}}{\partial u^{n}} [K_{n} (u, t) p (u, t)]

(3)

where

p (u, t)

is the density function and

K_{n} (u, t) = \int_{R} {(u^{'} - u)}^{n} W (u^{'} | x, t) d u^{'}

(4)

is the n-order conditional moment. Here

W (u^{'} | u, t)

is the transition probability rate. The Fokker-Planck equation is obtained by keeping only first two terms of the Kramers-Moyal expansion. In statistical mechanics, Fokker-Planck equation is usually describes the time evolution of probability density function of the velocity of a particle under the influence of drag forces and random forces, as in famous Brownian motion. And this equation is often used to find the density function of Itô stochastic differential equation[1].

1.3. Differential entropy and De Bruijn identity

The entropy of a continuous distribution was proposed by Shannon in 1948, known as differential entropy[30]:

h (X) = - \int_{R^{d}} p (x) log p (x) d x

(5)

where

h (\cdot)

represent the differential entropy and

p (\cdot)

is probability density function of X. However, differential entropy is not easy to calculate and seldom exists. There have been related studies on the entropy of stochastic processes and continuous systems[31,32,33,34]. If we consider a classical one-dimensional Gaussian channel model

Y_{t} = X + \sqrt{t} G

(6)

here X is the input signal, G is standard Gaussian noise,

t \geq 0

is the strength and

Y_{t}

is the output, we can obtain the density of

Y_{t}

satisfies the following Fokker-Planck equation

\frac{\partial p (y, t)}{\partial t} = \frac{1}{2} \frac{\partial^{2} p (y, t)}{\partial y^{2}}

(7)

Further, by calculating the differential entropy of

Y_{t}

and obtaining its derivative to t, we can get

\frac{d h_{Y_{t}} (t)}{d t} = \frac{1}{2} F I_{Y_{t}} (t)

(8)

where

F I_{Y_{t}} (t) = \int_{R} {[\frac{\partial log p (y, t)}{\partial y}]}^{2} p (y, t) d y

(9)

is the Fisher information of

Y_{t}

. The equation (8) here is the De Burijn identity. The de Bruijn identity connects the differential entropy

h (\cdot)

and the Fisher information

F I (\cdot)

, which shows that they are different aspects of the concept of “information”.

1.4. Entropy Divergence

In information theory and statistics, an entropy divergence is a statistical distance generated from information entropy to measure the difference between two probability distributions. There are various divergences generated by information entropy, such as Kullback-Leibler divergence[35], Jeffreys divergence[36], Jensen-Shannon divergence[37], Rényi divergence[38], etc. These measures have been applied in a variety of fields such as finance, economics, biology, signal processing, pattern recognition and machine learning[39,40,41,42,43,44,45,46,47,48,49]. In this paper, we mainly focus on the Jeffreys divergence of two distributions, formed as

J D (P, Q) = \int_{R} [p (x) - q (x)] log \frac{p (x)}{q (x)} d μ (x)

(10)

where

μ

is a measure of x.

2. Notations, Definitions and Propositions

2.1. Notations and Assumptions

In this paper, we use the following notations and definitions:

Given a probability space $(Ω, F, P)$ , two real valued space-time random fields are denoted as $X (ω; t, x)$ , $Y (ω; s, y)$ or $X (t, x)$ , $Y (s, y)$ , where $ω \in Ω$ and $(t, x) (s, y) \in R_{+} \times R^{d}$ , $d \in N_{+}$ , are space-time variables.

The probability density functions of P and Q are denoted as p and q. With $\forall u \in R$ , $p (u; t, x)$ is the density value at $(t, x)$ of X and $q (u; s, y)$ is the density value at $(s, y)$ of Y.

Suppose that our density functions $p (u; t, x), q (u, s, y) \in C^{2} (R) \times C^{1} (R_{+} \times R^{d})$ , i.e. $p (u; t, x)$ and $q (u; s, y)$ are derivable twice with respect to u and once with respect to $(t, x)$ or $(s, y)$ , respectively.

In this paper, we denote that only the k-th coordinate differs in the d-dimensional real vectors as ${\tilde{x}}_{k}$ and ${\tilde{y}}_{k}$ , where the k-th coordinates are $x_{k}$ and $y_{k}$ , $k = 1, 2, \dots, d$ .

2.2. Definitions

To obtain the generalized De Bruijn identities between Jeffreys divergence and Fisher divergence, we need to introduce some new definitions and propositions.

The first and important information quantity is the Kullback-Leibler divergence for random fields. Same as the classical Kullback-Leibler divergence, we can easily get the Definition 1.

Definition 1: The Kullback-Leibler divergence between two space-time random fields

X (t, x)

and

Y (s, y)

(t, x), (s, y) \in R_{+} \times R^{d}

, with density functions

p (u; t, x)

and

q (u; s, y)

, is defined as

K L (P (t, x) ∥ Q (s, y)) = \int_{R} p (u; t, x) log \frac{p (u; t, x)}{q (u; s, y)} d u

(11)

Similar to the classical Kullback-Leibler divergence, Kullback-Leibler divergence on random fields is not symmetrical, i.e.

K L (P (t, x) ∥ Q (s, y)) \neq K L (Q (s, y) ∥ P (t, x))

(12)

Following the classical definition of Jeffreys divergence on two random variables, we mainly consider Jeffreys divergence for random fields in this paper.

Definition 2: The Jeffreys divergence divergence between two space-time random fields

X (t, x)

and

Y (s, y)

(t, x), (s, y) \in R_{+} \times R^{d}

, with density function

p (u; t, x)

and

q (u; s, y)

is defined as

J D (P (t, x), Q (s, y)) = K L (P (t, x) ∥ Q (s, y)) + K L (Q (s, y) ∥ P (t, x))

(13)

Here, we replaced || by , in the distortion measure to emphasize the symmetric property.

Another significant measure of information is Fisher information. In this paper, we consider the generalized Fisher information of space-time random field.

Definition 3: The Generalized Fisher information of space-time random field

X (t, x)

(t, x) \in R_{+} \times R^{d}

, with density function

p (u; t, x)

defined by nonnegative function

f (\cdot)

are formed as

F I_{f} (P (t, x)) = \int_{R} f (u) {[\partial_{u} log p (u; t, x)]}^{2} p (u; t, x) d u

(14)

In particular, as

f \equiv 1

F I_{1} (t, x)

is the Fisher information in usual case. In addition to equation (14), we have other similar forms of generalized Fisher information

F I_{f}^{(t)} (P (t, x)) = \int_{R} f (u) {[\partial_{t} log p (u; t, x)]}^{2} p (u; t, x) d u

(15)

and

F I_{f}^{(x_{k})} (P (t, x)) = \int_{R} f (u) {[\partial_{x_{k}} log p (u; t, x)]}^{2} p (u; t, x) d u

(16)

for

k = 1, 2, \dots, d

. Obviously, (15) and (16) are generalized Fisher information on space-time variables. Regarding the generalized Fisher information (14), we can come to a following simple proposition.

Proposition 4: For arbitrary non-negative function

f (\cdot)

, we assume the generalized Fisher information of random variable X

F I_{f} (X) : = \int_{R} f (x) {[\frac{d log p_{X} (x)}{d x}]}^{2} p_{X} (x) d x

(17)

is well defined, where

p_{X} (x)

represents the probability density. Then we have the generalized Fisher information inequality

\frac{1}{F I_{f} (X + Y)} \geq \frac{1}{F I_{f} (X)} + \frac{1}{F I_{f} (Y)}

(18)

When

f \equiv 1

F I_{1} (X)

is the Fisher information in usual case.

Proof Denote

Z = X + Y

p_{X}

p_{Y}

and

p_{Z}

represent densities i.e.

p_{Z} (z) = \int_{R} p_{X} (x) p_{Y} (z - x) d x

(19)

and derivative function

p_{Z}^{'} (z) = \int_{R} p_{X}^{'} (x) p_{Y} (z - x) d x

(20)

p_{X}

p_{Y}

and

p_{Z}

are never vanish,

\begin{matrix} \frac{p_{Z}^{'} (z)}{p_{Z} (z)} \\ = & \int_{R} \frac{p_{X} (x) p_{Y} (z - x)}{p_{Z} (z)} \frac{p_{X}^{'} (x)}{p_{X} (x)} d x \\ = & E [\frac{p_{X}^{'} (x)}{p_{X} (x)} | z] \end{matrix}

(21)

is the conditional expectation of

\frac{p_{X}^{'} (x)}{p_{X} (x)}

for given z. Similarly, we can obtain

\frac{p_{Z}^{'} (z)}{p_{Z} (z)} = E [\frac{p_{Y}^{'} (y)}{p_{Y} (y)} | z]

(22)

and

\forall μ

λ \in R

, we find that

E [μ \frac{p_{X}^{'} (x)}{p_{X} (x)} + λ \frac{p_{Y}^{'} (y)}{p_{Y} (y)} | z] = (μ + λ) \frac{p_{Z}^{'} (z)}{p_{Z} (z)}

(23)

Hence, we have

\begin{matrix} {[(μ + λ) \frac{p_{Z}^{'} (z)}{p_{Z} (z)}]}^{2} \\ = & {\{E [μ \frac{p_{X}^{'} (x)}{p_{X} (x)} + λ \frac{p_{Y}^{'} (y)}{p_{Y} (y)} | z]\}}^{2} \\ \leq & E \{{[μ \frac{p_{X}^{'} (x)}{p_{X} (x)} + y \frac{p_{Y}^{'} (y)}{p_{Y} (y)}]}^{2} | z\} \end{matrix}

(24)

with equality only if

μ \frac{p_{X}^{'} (x)}{p_{X} (x)} + λ \frac{p_{Y}^{'} (y)}{p_{Y} (y)} = (μ + λ) \frac{p_{Z}^{'} (z)}{p_{Z} (z)}

(25)

with probability 1 whenever

z = x + y

and we have

f (z) {[(μ + λ) \frac{p_{Z}^{'} (z)}{p_{Z} (z)}]}^{2} \leq f (z) E [{(μ \frac{p_{X}^{'} (x)}{p_{X} (x)} + y \frac{p_{Y}^{'} (y)}{p_{Y} (y)})}^{2} | z]

(26)

Averaging both sides over the distribution of z

\begin{matrix} E \{f (z) {[(μ + λ) \frac{p_{Z}^{'} (z)}{p_{Z} (z)}]}^{2}\} \\ \leq & E \{f (z) E [{(μ \frac{p_{X}^{'} (x)}{p_{X} (x)} + y \frac{p_{Y}^{'} (y)}{p_{Y} (y)})}^{2} | z]\} \\ = & μ^{2} E \{f (z) E [{(\frac{p_{X}^{'} (x)}{p_{X} (x)})}^{2} | z]\} + λ^{2} E \{E [f (z) {(\frac{p_{Y}^{'} (y)}{p_{Y} (y)})}^{2} | z]\} \end{matrix}

(27)

i.e.

{(μ + λ)}^{2} J_{f} (X + Y) \leq μ^{2} J_{f} (X) + λ^{2} J_{f} (Y)

(28)

Setting

μ = \frac{1}{J_{f} (X)}

and

λ = \frac{1}{J_{f} (Y)}

, we obtain

\frac{1}{F I_{f} (X + Y)} \geq \frac{1}{F I_{f} (X)} + \frac{1}{F I_{f} (Y)}

(29)

According to Definition 3, we can get relevant generalized definitions of generalized Fisher information in general.

Definition 5: The Generalized cross Fisher information for space-time random fields

X (t, x)

and

Y (s, y)

(t, x), (s, y) \in R_{+} \times R^{d}

, with density functions

p (u; t, x)

and

q (u; s, y)

, defined by nonnegative function

f (\cdot)

is formed as

C F I_{f} (P (t, x), Q (s, y)) = \int_{R} f (u) {[\partial_{u} log q (u; s, y)]}^{2} p (u; t, x) d u

(30)

Similar to the concept of cross-entropy, it’s easily to verify that (30) is symmetrical about P and Q.

Definition 6: The Generalized Fisher divergence for space-time random fields

X (t, x)

and

Y (s, y)

, for

(t, x), (s, y) \in R_{+} \times R^{d}

, with density functions

p (u; t, x)

and

q (u; s, y)

, defined by nonnegative function

f (\cdot)

is formed as

F D_{f} (P (t, x) ∥ Q (s, y)) = \int_{R} f (u) {[\partial_{u} log p (u; t, x) - \partial_{u} log q (u; s, y)]}^{2} p (u; t, x) d u

(31)

In particular, as

f \equiv 1

F D_{1} (P (t, x) ∥ Q (s, y))

is the Fisher divergence in usual case.

Obviously, the generalized Fisher divergence of random fields is not a symmetrical divergence. To get the symmetrical formula, we need to generalize (31) in another field.

Definition 7: The Generalized Fisher divergence for space-time random fields

X (t, x)

and

Y (s, y)

, for

(t, x), (s, y) \in R_{+} \times R^{d}

, with density functions

p (u; t, x)

and

q (u; s, y)

, defined by nonnegative functions

f (\cdot)

and

g (\cdot)

is formed as

\begin{matrix} F D_{(f, g)} (P (t, x) ∥ Q (s, y)) = & \int_{R} [f (u; t, x) \partial_{u} log p (u; t, x) - g (u; s, y) \partial_{u} log q (u; s, y)] \\ \times [\partial_{u} log p (u; t, x) - \partial_{u} log q (u; s, y)] [p (u; t, x) + q (u, s, y)] d u \end{matrix}

(32)

In particular, as

f \equiv g

F D_{(f, f)} (P (t, x) ∥ Q (s, y))

is the generalized Fisher divergence for random fields by one function. In general,

F D_{(f, g)} (P (t, x) ∥ Q (s, y))

is asymmetric with respect to P and Q, i.e.

F D_{(f, g)} (P (t, x) ∥ Q (s, y)) \neq F D_{(f, g)} (Q (s, y) ∥ P (t, x))

(33)

If we suppose that f and g are functions only related to P and Q, i.e.

\{\begin{matrix} f (u; t, x) = T [p (t, x)] (u) \\ g (u; s, y) = T [q (s, y)] (u) \end{matrix}

(34)

where

T

is a operator, the generalized Fisher divergence

F D_{(f, g)} (P (t, x) ∥ Q (s, y))

can be rewrite as

\begin{matrix} F D_{(f, g)} (P (t, x) ∥ Q (s, y)) \\ = & \int_{R} \{T [p (t, x)] (u) \partial_{u} log p (u; t, x) - T [q (s, y)] (u) \partial_{u} log q (u; s, y)\} \\ \times [\partial_{u} log p (u; t, x) - \partial_{u} log q (u; s, y)] [p (u; t, x) + q (u, s, y)] d u \end{matrix}

(35)

and we can easily get

F D_{(f, g)} (P (t, x) ∥ Q (s, y)) = F D_{(g, f)} (Q (t, x) ∥ P (s, y))

(36)

In this case, we call (35) symmetric Fisher divergence for random fields generated by operator

T

and denote it as

\begin{matrix} s F D_{T} (P (t, x), Q (s, y)) \\ = & \int_{R} \{T [p (t, x)] (u) \partial_{u} log p (u; t, x) - T [q (s, y)] (u) \partial_{u} log q (u; s, y)\} \\ \times [\partial_{u} log p (u; t, x) - \partial_{u} log q (u; s, y)] [p (u; t, x) + q (u, s, y)] d u \end{matrix}

(37)

Notice that

\begin{matrix} A a - B b \\ = & \frac{1}{2} \times 2 (A a - B b) \\ = & \frac{1}{2} [(A a - A b + A b - B b) + (A a - B a + B a - B b)] \\ = & \frac{1}{2} [(A + B) (a - b) + (A - B) (a + b)] \end{matrix}

(38)

for A, B, a,

b \in R

, then we can rewrite (37) as

\begin{matrix} s F D_{T} (P (t, x), Q (s, y)) \\ = & \frac{1}{2} \int_{R} \{T [p (t, x)] (u) + T [q (s, y)] (u)\} {[\partial_{u} log p (u; t, x) - \partial_{u} log q (u; s, y)]}^{2} \\ \times [p (u; t, x) + q (u, s, y)] d u \\ + \frac{1}{2} \int_{R} \{T [p (t, x)] (u) - T [q (s, y)] (u)\} \{{[\partial_{u} log p (u; t, x)]}^{2} - {[\partial_{u} log q (u; s, y)]}^{2}\} \\ \times [p (u; t, x) + q (u, s, y)] d u \\ = & \frac{1}{2} [F D_{T [p (t, x)] + T [q (s, y)]} (P (t, x) ∥ Q (s, y)) + F D_{T [p (t, x)] + T [q (s, y)]} (Q (s, y) ∥ P (t, x))] \\ + \frac{1}{2} [F I_{T [p (t, x)] - T [q (s, y)]} (P (t, x)) + F I_{T [p (t, x)] - T [q (s, y)]} (Q (s, y))] \\ + \frac{1}{2} [C F I_{T [p (t, x)] - T [q (s, y)]} (Q (s, y), P (t, x)) - C F I_{T [p (t, x)] - T [q (s, y)]} (P (t, x), Q (s, y))] \end{matrix}

(39)

Proposition 8 (Kramers-Moyal expansion)[28,29]: Suppose that the random process

X (t)

has any order moment, then the probability density function

p (u, t)

satisfies the Kramers-Moyal expansion

\frac{\partial p (u, t)}{\partial t} = \sum_{n = 1}^{\infty} \frac{{(- 1)}^{n}}{n!} \frac{\partial^{n}}{\partial u^{n}} [K_{n} (u, t) p (u, t)]

(40)

where

K_{n} (u, t) = \int_{R} {(u^{'} - u)}^{n} W (u^{'} | u, t) d u^{'}

(41)

is the n-order conditional moment. Here

W (x^{'} | x, t)

is the transition probability rate.

Proposition 9 (Pawula theorem)[50,51]: If the limit on conditional moment of random process

X (t)

lim_{Δ t \to 0} \frac{1}{Δ t} E \{{[X (t + Δ t) - X (t)]}^{n} | X (t) = x\}

(42)

exists for all

n \in N_{+}

, and the limit value equals 0 for some even number, then the limit values are 0 for all

n \geq 3

Pawula theorem states that there are only three possible cases in the Kramers-Moyal expansion:

1) The Kramers-Moyal expansion is truncated at

n = 1

, meaning that the process is deterministic;

2) The Kramers-Moyal expansion stops at

n = 2

, with the resulting equation being the Fokker-Planck equation, and describes diffusion processes;

3) The Kramers-Moyal expansion contains all the terms up to

n = \infty

In this paper, we focus on the case of Fokker-Planck equation.

3. Main Results and Proofs

Lemma 1: Suppose f, p, q are continuous derivable functions defined on

R

. Then

\int_{R} f (u) [\frac{d log p (u)}{d u} - \frac{d log q (u)}{d u}] [p (u) + q (u)] d u = - \int_{R} f^{'} (u) [p (u) - q (u)] d u

(43)

holds true if it makes sense.

Proof:

\begin{matrix} \int_{R} f (u) [\frac{d log p (u)}{d u} - \frac{d log q (u)}{d u}] [p (u) + q (u)] d u \\ = & \int_{R} f (u) p (u) \frac{d log p (u)}{d u} d u + \int_{R} f (u) q (u) \frac{d log p (u)}{d u} d u \\ - \int_{R} f (u) p (u) \frac{d log q (u)}{d u} d u - \int_{R} f (u) q (u) \frac{d log q (u)}{d u} d u \\ = & \int_{R} f (u) [p^{'} (u) - q^{'} (u)] d u + \int_{R} f (u) [\frac{q (u)}{p (u)} p^{'} (u) - \frac{p (u)}{q (u)} q^{'} (u)] d u \end{matrix}

(44)

Notice that

\{\begin{matrix} \frac{d}{d u} [\frac{p (u)}{q (u)}] = \frac{1}{q (u)} [p^{'} (u) - \frac{p (u)}{q (u)} q^{'} (u)] \\ \frac{d}{d u} [\frac{q (u)}{p (u)}] = \frac{1}{p (u)} [q^{'} (u) - \frac{q (u)}{p (u)} p^{'} (u)] \end{matrix}

(45)

i.e.

\{\begin{matrix} \frac{p (u)}{q (u)} q^{'} (u) = p^{'} (u) - q (u) \frac{d}{d u} [\frac{p (u)}{q (u)}] \\ \frac{q (u)}{p (u)} p^{'} (u) = q^{'} (u) - p (u) \frac{d}{d u} [\frac{q (u)}{p (u)}] \end{matrix}

(46)

then

\begin{matrix} \int_{R} f (u) [\frac{q (u)}{p (u)} p^{'} (u) - \frac{p (u)}{q (u)} q^{'} (u)] d u \\ = & \int_{R} f (u) [p^{'} (u) - q (u) \frac{d}{d u} [\frac{p (u)}{q (u)}]] d u - \int_{R} f (u) [q^{'} (u) - p (u) \frac{d}{d u} [\frac{q (u)}{p (u)}]] d u \\ = & \int_{R} f (u) [p^{'} (u) - q^{'} (u)] d u - \int_{R} f (u) q (u) d [\frac{p (u)}{q (u)}] + \int_{R} f (u) p (u) d [\frac{q (u)}{p (u)}] \\ = & \int_{R} f (u) [p^{'} (u) - q^{'} (u)] d u + \int_{R} [f^{'} (u) q (u) + f (u) q^{'} (u)] \frac{p (u)}{q (u)} d u \\ - \int_{R} [f^{'} (u) p (u) + f (u) p^{'} (u)] \frac{q (u)}{p (u)} d u \\ = & \int_{R} f (u) [p^{'} (u) - q^{'} (u)] d u + \int_{R} f^{'} (u) [q (u) - p (u)] d u - \int_{R} f (u) [\frac{q (u)}{p (u)} p^{'} (u) - \frac{p (u)}{q (u)} q^{'} (u)] d u \\ = & \int_{R} d \{f (u) [q (u) - p (u)]\} - \int_{R} f (u) [\frac{q (u)}{p (u)} p^{'} (u) - \frac{p (u)}{q (u)} q^{'} (u)] d u \\ = & - \int_{R} f (u) [\frac{q (u)}{p (u)} p^{'} (u) - \frac{p (u)}{q (u)} q^{'} (u)] d u \end{matrix}

(47)

So we can get the result

\int_{R} f (u) [\frac{d log p (u)}{d u} - \frac{d log q (u)}{d u}] [p (u) + q (u)] d u = - \int_{R} f^{'} (u) [p (u) - q (u)] d u

(48)

Lemma 2: Suppose that the Fokker-Planck equation for the density function

p (u, t)

\frac{\partial p (u, t)}{\partial t} = \frac{1}{2} \frac{\partial^{2} [b (u, t) p (u, t)]}{\partial u^{2}} - \frac{\partial [a (u, t) p (u, t)]}{\partial u}

(49)

We use variable substitution

u = u (v)

v = v (u)

to convert the equation to

\frac{\partial p (v, t)}{\partial t} = \frac{E (t)}{2} \frac{\partial^{2} p (v, t)}{\partial u^{2}} - F (t) \frac{\partial p (v, t)}{\partial u} + G (t) p (v, t)

(50)

where

E (t)

F (t)

, and

G (t)

are functions of t and

E (t) > 0

. Then we get the density

p (u, t) = \frac{e^{\int_{0}^{t} G (s) d s}}{\sqrt{2 π \int_{0}^{t} E (s) d s}} e^{- \frac{{[v (u) - v (u_{0}) - \int_{0}^{t} F (s) d s]}^{2}}{2 \int_{0}^{t} E (s) d s}}

(51)

Proof: Using Fourier transform, we can get the solution of Fokker-Planck equation (50) about v

p (v, t) = \frac{e^{\int_{0}^{t} G (s) d s}}{\sqrt{2 π \int_{0}^{t} E (s) d s}} e^{- \frac{{[v - v_{0} (x) - \int_{0}^{t} F (s) d s]}^{2}}{2 \int_{0}^{t} E (s) d s}}

(52)

It is worth noting that while p is a probability density function for u, and p is not a probability density function for v. Our transformation,

u = u (v)

v = v (u)

, is only introduced in the process of solving the equation, which does not mean that v is a random process. Therefore, the integral value of p with respect to v is not 1. But we can obtain

p (u, t) = \frac{e^{\int_{0}^{t} G (s) d s}}{\sqrt{2 π \int_{0}^{t} E (s) d s}} e^{- \frac{{[v (u) - v (u_{0}) - \int_{0}^{t} F (s) d s]}^{2}}{2 \int_{0}^{t} E (s) d s}}

(53)

and

\int_{I_{u}} \frac{e^{\int_{0}^{t} G (s) d s}}{\sqrt{2 π \int_{0}^{t} E (s) d s}} e^{- \frac{{[v (u) - v (u_{0}) - \int_{0}^{t} F (s) d s]}^{2}}{2 \int_{0}^{t} E (s) d s}} d u = 1

(54)

where

I_{u}

is the supporting set of

p (u, t)

with respect to u.

Theorem 3: The probability density function

p (u; t, x)

of space-time random field

X (t, x)

u \in R

(t, x) \in R_{+} \times R^{d}

, satisfies the following Fokker-Planck equations

\{\begin{matrix} \frac{\partial}{\partial t} p (u; t, x) & = \frac{1}{2} \frac{\partial^{2}}{\partial u^{2}} [b_{0} (u; t, x) p (u, t, x)] - \frac{\partial}{\partial u} [a_{0} (u; t, x) p (u; t, x)] \\ \frac{\partial}{\partial x_{k}} p (u; t, x) & = \frac{1}{2} \frac{\partial^{2}}{\partial u^{2}} [b_{k} (u; t, x) p (u, t, x)] - \frac{\partial}{\partial u} [a_{k} (u; t, x) p (u; t, x)] \\ k & = 1, 2, \dots, d \end{matrix}

(55)

where

\{\begin{matrix} a_{0} (u; t, x) & = lim_{Δ t \to 0} \frac{1}{Δ t} M_{1} (u; t, Δ t, x) \\ b_{0} (u; t, x) & = lim_{Δ t \to 0} \frac{1}{Δ t} M_{2} (u; t, Δ t, x) \\ a_{k} (u; t, x) & = lim_{Δ x_{k} \to 0} \frac{1}{Δ x_{k}} M_{1} (u; t, x, Δ x_{k}) \\ b_{k} (u; t, x) & = lim_{Δ x_{k} \to 0} \frac{1}{Δ x_{k}} M_{2} (u; t, x, Δ x_{k}) \\ k & = 1, 2, \dots, d \end{matrix}

(56)

here

\{\begin{matrix} M_{n} (u; t, Δ t, x) = E \{{[X (t + Δ t, x) - X (t, x)]}^{n} | X (t, x) = u\} \\ M_{n} (u; t, x, Δ x_{k}) = E \{{[X (t, x + Δ x_{k}) - X (t, x)]}^{n} | X (t, x) = u\} \end{matrix}

(57)

are n-order conditional moments and

(e_{1}, e_{2}, \dots, e_{d})

is the orthonormal basis of the standard identity of

R^{d}

Proof:

\forall Δ t \neq 0

, we can get the difference of density function in time variable

p (u; t + Δ t, x) - p (u; t, x) = \sum_{n = 1}^{+ \infty} \frac{{(- 1)}^{n}}{n!} \frac{\partial^{n}}{\partial u^{n}} [M_{n} (u; t, Δ t, x) p (u; t, x)]

(58)

where

M_{n} (u; t, Δ t, x) = E \{{[X (t + Δ t, x) - X (t, x)]}^{n} | X (t, x) = u\}

(59)

is the n-order conditional moment. Then the partial derivative of the density function with respect to t is

\frac{\partial}{\partial t} p (u; t, x) = lim_{Δ t \to 0} \frac{1}{Δ t} \sum_{n = 1}^{+ \infty} \frac{{(- 1)}^{n}}{n!} \frac{\partial^{n}}{\partial u^{n}} [M_{n} (u; t, Δ t, x) p (u; t, x)]

(60)

The Pawula theorem implies that if the Kramers-Moyal expansion stops after the second term, we get the Fokker-Planck equation about time variable t

\frac{\partial}{\partial t} p (u; t, x) = \frac{1}{2} \frac{\partial^{2}}{\partial u^{2}} [b_{0} (u; t, x) p (u, t, x)] - \frac{\partial}{\partial u} [a_{0} (u; t, x) p (u; t, x)]

(61)

where

\{\begin{matrix} a_{0} (u; t, x) = lim_{Δ t \to 0} \frac{1}{Δ t} M_{1} (u; t, Δ t, x) \\ b_{0} (u; t, x) = lim_{Δ t \to 0} \frac{1}{Δ t} M_{2} (u; t, Δ t, x) \end{matrix}

(62)

Similarly, we may consider the increment

Δ x_{k}

of the space variable

x_{k}

and we can obtain the Fokker-Planck equations about

x_{k}

\frac{\partial}{\partial x_{k}} p (u; t, x) = \frac{1}{2} \frac{\partial^{2}}{\partial u^{2}} [b_{k} (u; t, x) p (u, t, x)] - \frac{\partial}{\partial u} [a_{k} (u; t, x) p (u; t, x)]

(63)

where

\{\begin{matrix} a_{k} (u; t, x) = lim_{Δ x_{k} \to 0} \frac{1}{Δ x_{k}} M_{1} (u; t, x, Δ x_{k}) \\ b_{k} (u; t, x) = lim_{Δ x_{k} \to 0} \frac{1}{Δ x_{k}} M_{2} (u; t, x, Δ x_{k}) \end{matrix}

(64)

here

M_{n} (u; t, x, Δ x_{k}) = E \{{[X (t, x + Δ x_{k} e_{k}) - X (t, x)]}^{n} | X (t, x) = u\}

(65)

(e_{1}, e_{2}, \dots, e_{d})

is the orthonormal basis of the standard identity of

R^{d}

k = 1, 2, \dots, d

Theorem 4: Suppose that

p (u; t, x)

and

p (u; s, y)

are rapidly decreasing functions of space-time random field

X (t, x)

such that the partial integral terms are 0,

(t, x), (s, y) \in R_{+} \times R^{d}

. If there are two infinitesimal

ε (t, x, s, y) > 0

and

η (t, x, s, y) > 0

satisfied

\{\begin{matrix} lim_{| t - s | \to 0} ε (t, x, s, y) & = lim_{| t - s | \to 0} η (t, x, s, y) = 0 \\ lim_{| x_{k} - y_{k} | \to 0} ε (t, x, s, y) & = lim_{| x_{k} - y_{k} | \to 0} η (t, x, s, y) = 0 \\ k & = 1, 2, \dots, d \end{matrix}

(66)

where

x_{k}

y_{k}

are the k-th coordinate of x,

y \in R

, such that

1 - ε (t, x, s, y) < \frac{p (u; t, x)}{p (u; s, y)} < 1 + η (t, x, s, y)

(67)

is true to all meaningful

u \in R

. Then we have

\{\begin{matrix} \frac{1}{1 + η (t, x, s, y)} + λ (t - s) & < \frac{J D (P (t, x), P (s, x))}{{| t - s |}^{2} F I_{1}^{(s)} (X (s, x))} < \frac{1}{1 - ε (t, x, s, y)} + λ (t - s) \\ \frac{1}{1 + η (t, x, s, y)} + μ_{k} (x_{k} - y_{k}) & < \frac{J D (P (t, {\tilde{x}}_{k}), P (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2} F I_{1}^{(y_{k})} (X (t, {\tilde{y}}_{k}))} < \frac{1}{1 - ε (t, x, s, y)} + μ_{k} (x_{k} - y_{k}) \end{matrix}

(68)

here

F I_{1}^{(s)}

and

F I_{1}^{(y_{k})}

are generalized Fisher information on space-time variables and

\{\begin{matrix} lim_{| t - s | \to 0} λ (t - s) & = 0 \\ lim_{| x_{k} - y_{k} | \to 0} μ_{k} (x_{k} - y_{k}) & = 0 \end{matrix}

(69)

are infinitesimal on space-time variables,

k = 1, 2, \dots, d

Proof: First, we consider

J D (P (t, x), P (s, x)) / {| t - s |}^{2}

. Notice that

\begin{matrix} log p (u; t, x) - log p (u; s, x) \\ = & \frac{1}{γ p (u; t, x) + (1 - γ) p (u; s, x)} [p (u; t, x) - p (u; s, x)] \\ = & \frac{1}{γ p (u; t, x) + (1 - γ) p (u; s, x)} [\partial_{s} p (u; s, x) (t - s) + o (t - s)] \end{matrix}

(70)

then we can get

\begin{matrix} \frac{J D (P (t, x), P (s, x))}{{| t - s |}^{2}} \\ = & \frac{1}{{| t - s |}^{2}} \int_{R} {[\partial_{s} p (u; s, x) (t - s) + o (t - s)]}^{2} \frac{1}{γ p (u; t, x) + (1 - γ) p (u; s, x)} d u \\ = & \frac{1}{{| t - s |}^{2}} \int_{R} {[\partial_{s} [log p (u; s, x)] (t - s) + \frac{1}{p (u; s, x)} o (t - s)]}^{2} p (u; s, x) \\ \times \frac{p (u; s, x)}{γ p (u; t, x) + (1 - γ) p (u; s, x)} d u \\ = & \int_{R} {[\partial_{s} [log p (u; s, x)] + \frac{1}{p (u; s, x)} \frac{o (t - s)}{| t - s |}]}^{2} p (u; s, x) \frac{1}{γ \frac{p (u; t, x)}{p (u; s, x)} + (1 - γ)} d u \end{matrix}

(71)

where

0 < γ < 1

Recall the conditions, there are two infinitesimal

ε (t, x, s, y), η (t, x, s, y) > 0

and

lim_{| t - s | \to 0} ε (t, x, s, y) = lim_{| t - s | \to 0} η (t, x, s, y) = 0

(72)

such that

1 - ε (t, x, s, y) < \frac{p (u; t, x)}{p (u; s, y)} < 1 + η (t, x, s, y)

(73)

is true to all meaningful

u \in R

, then we can get

\begin{matrix} \frac{1}{1 + η (t, x, s, y)} \\ < & \frac{1}{γ (1 + η (t, x, s, y)) + (1 - γ)} \\ < & \frac{1}{γ \frac{p (u; t, x)}{p (u; s, x)} + (1 - γ)} \\ < & \frac{1}{γ (1 - ε (t, x, s, y)) + (1 - γ)} \\ < & \frac{1}{1 - ε (t, x, s, y)} \end{matrix}

(74)

so there exits infinitesimal

λ (t - s)

| t - s | \to 0

such that

\frac{1}{1 + η (t, x, s, y)} + λ (t - s) < \frac{J D (P (t, x), P (s, x))}{{| t - s |}^{2} F I_{1}^{(s)} (X (s, x))} < \frac{1}{1 - ε (t, x, s, y)} + λ (t - s)

(75)

Similarly, we can get the bounds of difference quotient on Jeffreys divergence for space coordinates

\frac{1}{1 + η (t, x, s, y)} + μ_{k} (x_{k} - y_{k}) < \frac{J D (P (t, {\tilde{x}}_{k}), P (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2} F I_{1}^{(y_{k})} (X (t, {\tilde{y}}_{k}))} < \frac{1}{1 - ε (t, x, s, y)} + μ_{k} (x_{k} - y_{k})

(76)

for

k = 1, 2, \dots, d

Theorem 5: Suppose that

p (u; t, x)

and

q (u; t, x)

are rapidly decreasing functions of space-time random fields

X (t, x)

and

Y (t, x)

such that the partial integral terms are 0,

(t, x) \in R_{+} \times R^{d}

. Then the Jeffreys divergence

J D (P (t, x), Q (t, x))

satisfies generalized De Bruijn identities

\{\begin{matrix} \frac{\partial}{\partial t} J D (P (t, x), Q (t, x)) & = - \frac{1}{2} F D_{(b_{0}^{(1)}, b_{0}^{(2)})} (P (t, x) ∥ Q (t, x)) - R_{0} (P (t, x) ∥ Q (t, x)) \\ \frac{\partial}{\partial x_{k}} J D (P (t, x), Q (t, x)) & = - \frac{1}{2} F D_{(b_{k}^{(1)}, b_{k}^{(2)})} (P (t, x) ∥ Q (t, x)) - R_{k} (P (t, x) ∥ Q (t, x)) \\ k & = 1, 2, \dots, d \end{matrix}

(77)

where

a_{0}

b_{0}

a_{k}

b_{k}

are the forms in (56) and (57), and

\{\begin{matrix} R_{0} (P (t, x) ∥ Q (t, x)) & = \int_{R} [\frac{1}{2} \partial_{u u}^{2} (b_{0}^{(1)} - b_{0}^{(2)}) - \partial_{u} (a_{0}^{(1)} - a_{0}^{(2)})] (p + q) d u \\ R_{k} (P (t, x) ∥ Q (t, x)) & = \int_{R} [\frac{1}{2} \partial_{u u}^{2} (b_{k}^{(1)} - b_{k}^{(2)}) - \partial_{u} (a_{k}^{(1)} - a_{k}^{(2)})] (p + q) d u \\ k & = 1, 2, \dots, d \end{matrix}

(78)

here we omit

(u; t, x)

in the integrals.

Proof: Notice that

\begin{matrix} J D (P (t, x), Q (t, x)) \\ = & K L (P (t, x) ∥ Q (t, x)) + K L (Q (t, x) ∥ P (t, x)) \\ = & \int_{R} p log \frac{p}{q} d u + \int_{R} q log \frac{q}{p} d u \\ = & \int_{R} [p log \frac{p}{q} + q log \frac{q}{p}] d u d u \end{matrix}

(79)

where

p : = p (u; t, x)

q : = q (u; t, x)

and we omit the variable

(u; t, x)

. Then

\begin{matrix} \partial_{t} J D (P (t, x), Q (t, x)) \\ = & \int_{R} [\partial_{t} p log \frac{p}{q} + q \partial_{t} (\frac{p}{q}) + \partial_{t} q log \frac{q}{p} + p \partial_{t} (\frac{q}{p})] d u \\ = & \int_{R} [\partial_{t} p log \frac{p}{q} + \partial_{t} p - \frac{p}{q} \partial_{t} q + \partial_{t} q log \frac{q}{p} + \partial_{t} q - \frac{q}{p} \partial_{t} p] d u \\ = & \int_{R} [(log \frac{p}{q} - \frac{q}{p}) \partial_{t} p + (log \frac{q}{p} - \frac{p}{q}) \partial_{t} q] d u \\ = & \int_{R} (log \frac{p}{q} - \frac{q}{p}) [\frac{1}{2} \partial_{u u}^{2} (b_{0}^{(1)} p) - \partial_{u} (a_{0}^{(1)} p)] d u \\ + \int_{R} (log \frac{q}{p} - \frac{p}{q}) [\frac{1}{2} \partial_{u u}^{2} (b_{0}^{(2)} q) - \partial_{u} (a_{0}^{(2)} q)] d u \\ = & - \int_{R} [\frac{1}{2} \partial_{u} (b_{0}^{(1)} p) - a_{0}^{(1)} p] [\frac{q}{p} \partial_{u} (\frac{p}{q}) - \partial_{u} (\frac{q}{p})] d u \\ - \int_{R} [\frac{1}{2} \partial_{u} (b_{0}^{(2)} q) - a_{0}^{(2)} q] [\frac{p}{q} \partial_{u} (\frac{q}{p}) - \partial_{u} (\frac{p}{q})] d u \\ = & - \int_{R} [\frac{1}{2} \partial_{u} (b_{0}^{(1)} p) - a_{0}^{(1)} p] \frac{1}{p} [q \partial_{u} (\frac{p}{q}) - p \partial_{u} (\frac{q}{p})] d u \\ - \int_{R} [\frac{1}{2} \partial_{u} (b_{0}^{(2)} q) - a_{0}^{(2)} q] \frac{1}{q} [p \partial_{u} (\frac{q}{p}) - q \partial_{u} (\frac{p}{q})] d u \\ = & - \int_{R} \{[\frac{1}{2} \partial_{u} (b_{0}^{(1)} p) - a_{0}^{(1)} p] \frac{1}{p} - [\frac{1}{2} \partial_{u} (b_{0}^{(2)} q) - a_{0}^{(2)} q] \frac{1}{q}\} (\partial_{u} log p - \partial_{u} log q) (p + q) d u \\ = & - \int_{R} [\frac{1}{2} b_{0}^{(1)} \partial_{u} log p - \frac{1}{2} b_{0}^{(2)} \partial_{u} log q + \frac{1}{2} \partial_{u} (b_{0}^{(1)} - b_{0}^{(2)}) - (a_{0}^{(1)} - a_{0}^{(2)})] \\ \times (\partial_{u} log p - \partial_{u} log q) (p + q) d u \\ = & - \int_{R} [\frac{1}{2} b_{0}^{(1)} \partial_{u} log p - \frac{1}{2} b_{0}^{(2)} \partial_{u} log q] (\partial_{u} log p - \partial_{u} log q) (p + q) d u \\ - \int_{R} [\frac{1}{2} \partial_{u} (b_{0}^{(1)} - b_{0}^{(2)}) - (a_{0}^{(1)} - a_{0}^{(2)})] (\partial_{u} log p - \partial_{u} log q) (p + q) d u \\ = & - \frac{1}{2} F D_{(b_{0}^{(1)}, b_{0}^{(2)})} (P (t, x) ∥ Q (t, x)) - R_{0} (P (t, x) ∥ Q (t, x)) \end{matrix}

(80)

where

\begin{matrix} F D_{(b_{0}^{(1)}, b_{0}^{(2)})} (P (t, x) ∥ Q (t, x)) \\ = & \int_{R} [b_{0}^{(1)} \partial_{u} log p - b_{0}^{(2)} \partial_{u} log q] (\partial_{u} log p - \partial_{u} log q) (p + q) d u \end{matrix}

(81)

and

R_{0} (P (t, x) ∥ Q (t, x)) = \int_{R} [\frac{1}{2} \partial_{u u}^{2} (b_{0}^{(1)} - b_{0}^{(2)}) - \partial_{u} (a_{0}^{(1)} - a_{0}^{(2)})] (p + q) d u

(82)

Similarly, for

k = 1, 2, \dots, d

, the generalized De Bruijn identities about the space variable

x_{k}

\frac{\partial}{\partial x_{k}} J D (P (t, x), Q (t, x)) = - \frac{1}{2} F D_{(b_{k}^{(1)}, b_{k}^{(2)})} (P (t, x) ∥ Q (t, x)) - R_{k} (P (t, x), Q (t, x))

(83)

where

R_{k} (P (t, x) ∥ Q (t, x)) = \int_{R} [\frac{1}{2} \partial_{u u}^{2} (b_{k}^{(1)} - b_{k}^{(2)}) - \partial_{u} (a_{k}^{(1)} - a_{k}^{(2)})] (p + q) d u

(84)

then we get the conclusion.

4. Three Fokker-Planck Random Fields and Their Corresponding Information Measures

In this section, we list three types Fokker-Planck equations, obtain their density functions and the corresponding information measures, the Jeffreys divergence, generalized Fisher information and Fisher divergence. Starting from these quantities, we get two results illustrated in Theorem 4 and Theorem 5. On the one hand, we obtain the quotient of Jeffreys divergence and the squared of space-time variation on the same Fokker-Planck space-time random field at different space-time points, comparing with the generalized Fisher information. On the other hand, we obtain the De Burijn identities on Jeffreys divergence and generalized Fisher divergence from the Fokker-Planck equations on space-time random field at the same space-time position under the same type and different parameters.

4.1. A Trivial Equation

If we let

\{\begin{matrix} a_{0} (u; t, x) & = a_{0} (t, x) \\ b_{0} (u; t, x) & = b_{0} (t, x) > 0 \\ a_{k} (u; t, x) & = a_{k} (t, x) \\ b_{k} (u; t, x) & = b_{k} (t, x) > 0 \\ k & = 1, 2, \dots, d \end{matrix}

(85)

are continuous derivable functions independent of u and

p (u; 0, x) = δ [u - u_{0} (x)]

is the initial density, the Fokker-Planck equations are simple parabolic equations and the solution can be obtained by Fourier transform

\{\begin{matrix} p (u; t, x) & = \frac{1}{\sqrt{2 π \int_{0}^{t} b_{0} (s, x) d s}} e^{- \frac{{[u - u_{0} (x) - \int_{0}^{t} a_{0} (s, x) d s]}^{2}}{2 \int_{0}^{t} b_{0} (s, x) d s}} \\ p (u; t, x) & = \frac{1}{\sqrt{2 π \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} e^{- \frac{{[u - u_{0} (x) - \int_{0}^{x_{k}} a_{k} (t, x) d x_{k}]}^{2}}{2 \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} \end{matrix}

(86)

In order to write the above results into a unified form, we suppose that

\{\begin{matrix} \int_{0}^{t} a_{0} (s, x) d s = \int_{0}^{x_{k}} a_{k} (t, x) d x_{k} \\ \int_{0}^{t} b_{0} (s, x) d s = \int_{0}^{x_{k}} b_{k} (t, x) d x_{k} \end{matrix}

(87)

hold for

k = 1, 2, \dots, d

, we can get a unified formula. This reminds us that we need to find functions

α (t, x)

and

β (t, x) > 0

subject to the total differential

\{\begin{matrix} d α (t, x) = a_{0} d t + a_{1} d x_{1} + \dots + a_{d} d x_{d} \\ d β (t, x) = b_{0} d t + b_{1} d x_{1} + \dots + b_{d} d x_{d} \end{matrix}

(88)

i.e.

a_{0}, a_{1}, \dots, a_{d}

b_{0}, b_{1}, \dots, b_{d}

correspond to partial derivatives of

α (t, x)

and

β (t, x)

with respect to space-time variables, respectively. In this way, we can get the probability density function

p (u; t, x) = \frac{1}{\sqrt{2 π β (t, x)}} e^{- \frac{{[u - u_{0} (x) - α (t, x)]}^{2}}{2 β (t, x)}}

(89)

Actually, there are many examples whose Fokker-Planck equations fit this form. Let

B (t, x)

(1 + d, 1)

Brownian sheet. That is, a centered continuous Gaussian process which is indexed by

(1 + d)

real, positive parameters and takes its values in

R

[52,53]. Moreover, its covariance structure is given by

E [B (t, x) B (s, y)] = (t \land s) \prod_{k = 1}^{d} (x_{k} \land y_{k})

(90)

for

(t, x_{1}, x_{2}, \dots, x_{d})

(s, y_{1}, y_{2}, \dots, y_{d}) \in R_{+} \times R_{+}^{d}

, where

(\cdot \land \cdot)

represents the minimum of two numbers. We can easily get

E [B^{2} (t, x)] = prod (t, x)

(91)

where

prod (t, x) = t x_{1} x_{2} \dots x_{d}

is the coordinate product of

(t, x)

and the density function is formed as

p^{(1)} (u; t, x) = \frac{1}{\sqrt{2 π prod (t, x)}} e^{- \frac{u^{2}}{2 prod (t, x)}}

(92)

Moreover, the Fokker-Planck equations are

\{\begin{matrix} \frac{\partial}{\partial t} p^{(1)} (u; t, x) & = \frac{prod (x)}{2} \frac{\partial^{2}}{\partial u^{2}} p^{(1)} (u, t, x) \\ \frac{\partial}{\partial x_{k}} p^{(1)} (u; t, x) & = \frac{prod (t, x)}{2 x_{k}} \frac{\partial^{2}}{\partial u^{2}} p^{(1)} (u, t, x) \\ k & = 1, 2, \dots, d \end{matrix}

(93)

with the initial condition

p (u; t, x) = δ (u)

prod (t, x) = 0

Following the construct idea of Brownian bridge on Brownian motion[53], we name

B^{*} (t, x) = B (t, x) - prod (t, x) B (1, 1, \dots, 1)

(94)

Brownian sheet bridge on cube

(t, x) \in [0, 1] \times {[0, 1]}^{d}

where

B (t, x)

is the Brownian sheet. Obviously,

B^{*} (t, x)

is Gaussian,

E [B^{*} (t, x)] = 0

and

E [B^{*} (t, x) B^{*} (s, y)] = E [B (t, x) B (s, y)] - prod (t, x) prod (s, y)

(95)

we can get

E [B^{2} (t, x)] = prod (t, x) [1 - prod (t, x)]

(96)

and density function of

B^{*} (t, x)

p^{(2)} (u; t, x) = \frac{1}{\sqrt{2 π prod (t, x) [1 - prod (t, x)]}} e^{- \frac{u^{2}}{2 prod (t, x) [1 - prod (t, x)]}}

(97)

On the other hand, the Fokker-Planck equations are

\{\begin{matrix} \frac{\partial}{\partial t} p^{(2)} (u; t, x) & = \frac{prod (x) [1 - 2 prod (t, x)]}{2} \frac{\partial^{2}}{\partial u^{2}} p^{(2)} (u, t, x) \\ \frac{\partial}{\partial x_{k}} p^{(2)} (u; t, x) & = \frac{prod (t, x)}{2 x_{k}} [1 - 2 prod (t, x)] \frac{\partial^{2}}{\partial u^{2}} p^{(2)} (u, t, x) \\ k & = 1, 2, \dots, d \end{matrix}

(98)

with the initial condition

p (u; t, x) = δ (u)

prod (t, x) = 0

and we get the solution (97).

Now, we have density functions (92) and (97) to get their respective Jeffreys divergence and the generalized De Burijn identities. Actually, we can easily get the Jeffreys divergence of (89) at different space-time points is formed as

J D (P (t, x), P (s, y)) = \frac{{[α (t, x) - α (s, y)]}^{2} + β (s, y)}{2 β (t, x)} + \frac{{[α (t, x) - α (s, y)]}^{2} + β (t, x)}{2 β (s, y)} - 1

(99)

and the Fisher divergence of

P^{(1)}

and

P^{(2)}

at the same space-time points

\begin{matrix} F D_{(b_{k}^{(1)}, b_{k}^{(2)})} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) \\ = & \frac{1}{β_{1}^{2} (t, x) β_{2}^{2} (t, x)} {{[α_{1} (t, x) - α_{2} (t, x)]}^{2} [b_{k}^{(2)} β_{1}^{2} (t, x) + b_{k}^{(1)} β_{2}^{2} (t, x)] \\ + [β_{1} (t, x) - β_{2} (t, x)] [β_{1} (t, x) + β_{2} (t, x)] [b_{k}^{(2)} β_{1} (t, x) - b_{k}^{(1)} β_{2} (t, x)]} \end{matrix}

(100)

here

k = 0, 1, \dots, d

Bring the density function of Brownian sheet into (99), we can get the Jeffreys divergence of Brownian sheet at different space-time points

J D (P^{(1)} (t, x), P^{(1)} (s, y)) = \frac{prod (s, y)}{2 prod (t, x)} + \frac{prod (t, x)}{2 prod (s, y)} - 1

(101)

and the generalized Fisher information on space-time variables are

\{\begin{matrix} F I_{1}^{(t)} (P^{(1)} (t, x)) & = \frac{1}{2 t^{2}} \\ F I_{1}^{(x_{k})} (P^{(1)} (t, x)) & = \frac{1}{2 x_{k}^{2}} \end{matrix}

(102)

k = 1, 2, \dots, d

Then we can get quotients of the squared difference between Jeffreys divergence and space-time variables

\{\begin{matrix} \frac{J D (P^{(1)} (t, x), P^{(1)} (s, x))}{{| t - s |}^{2}} = \frac{1}{2 s t} \\ \frac{J D (P^{(1)} (t, {\tilde{x}}_{k}), P^{(1)} (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2}} = \frac{1}{2 x_{k} y_{k}} \end{matrix}

(103)

and then we can get the relation between quotients and generalized Fisher information

\{\begin{matrix} \frac{J D (P^{(1)} (t, x), P^{(1)} (s, x))}{{| t - s |}^{2} F I_{1}^{(t)} (P^{(1)} (t, x))} = \frac{t}{s} \\ \frac{J D (P^{(1)} (t, {\tilde{x}}_{k}), P^{(1)} (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2} F I_{1}^{(x_{k})} (P^{(1)} (t, x))} = \frac{x_{k}}{y_{k}} \end{matrix}

(104)

for

k = 1, 2, \dots, d

. If we consider the approximation of spacetime points

(t, x)

and

(s, y)

, the final result (104) satisfies the conclusion of Theorem 4.

Similarly, we can get the Jeffreys divergence of Brownian sheet bridge at different space-time points

J D (P^{(2)} (t, x), P^{(2)} (s, y)) = \frac{prod (s, y) [1 - prod (s, y)]}{2 prod (t, x) [1 - prod (t, x)]} + \frac{prod (t, x) [1 - prod (t, x)]}{2 prod (s, y) [1 - prod (s, y)]} - 1

(105)

and the generalized Fisher information on space-time variables

\{\begin{matrix} F I_{1}^{(t)} (P^{(2)} (t, x)) & = \frac{{[1 - 2 prod (t, x)]}^{2}}{2 t^{2} {[1 - prod (t, x)]}^{2}} \\ F I_{1}^{(x_{k})} (P^{(2)} (t, x)) & = \frac{{[1 - 2 prod (t, x)]}^{2}}{2 x_{k}^{2} {[1 - prod (t, x)]}^{2}} \end{matrix}

(106)

k = 1, 2, \dots, d

. Further, we can easily get the quotients of the squared difference between Jeffreys divergence and space-time variables

\{\begin{matrix} \frac{J D (P^{(2)} (t, x), P^{(2)} (s, x))}{{| t - s |}^{2}} = \frac{{[1 - prod (x) (s + t)]}^{2}}{2 s t [1 - prod (s, x)] [1 - prod (t, x)]} \\ \frac{J D (P^{(2)} (t, {\tilde{x}}_{k}), P^{(2)} (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2}} = \frac{1}{2 x_{k} y_{k} [1 - prod (t, x)] [1 - prod (t, y)]} {[1 - \frac{prod (t, {\tilde{x}}_{k})}{x_{k}} (x_{k} + y_{k})]}^{2} \end{matrix}

(107)

and then we can get the relation between quotients and generalized Fisher information

\{\begin{matrix} \frac{J D (P^{(2)} (t, x), P^{(2)} (s, x))}{{| t - s |}^{2} F I_{1}^{(t)} (P^{(2)} (t, x))} = & \frac{t [1 - prod (t, x)] {[1 - prod (x) (s + t)]}^{2}}{s [1 - prod (s, x)] {[1 - 2 prod (t, x)]}^{2}} \\ \frac{J D (P^{(2)} (t, {\tilde{x}}_{k}), P^{(2)} (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2} F I_{1}^{(x_{k})} (P^{(2)} (t, x))} = & \frac{x_{k} [1 - prod (t, {\tilde{x}}_{k})]}{y_{k} [1 - prod (t, {\tilde{y}}_{k})] {[1 - 2 prod (t, x)]}^{2}} {[1 - \frac{prod (t, {\tilde{x}}_{k})}{x_{k}} (x_{k} + y_{k})]}^{2} \end{matrix}

(108)

for

k = 1, 2, \dots, d

. Without loss of generality, the result (108) also satisfies Theorem 4.

Next, we consider the Jeffreys divergence on (92) and (97) at the same space-time points. Notice that the bounded domain of the Brownian sheet bridge density function, we consider only the space-time area

(t, x) \in [0, 1] \times {[0, 1]}^{d}

We can easily get the Jeffreys divergence

J D (P^{(1)} (t, x), P^{(2)} (t, x)) = \frac{1 - prod (t, x)}{2} + \frac{1}{2 [1 - prod (t, x)]} - 1

(109)

and the Fisher divergence from (100)

\{\begin{matrix} F D_{(b_{0}^{(1)}, b_{0}^{(2)})} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) = prod (x) - \frac{prod (x)}{{[1 - prod (t, x)]}^{2}} \\ F D_{(b_{k}^{(1)}, b_{k}^{(2)})} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) = \frac{prod (t, x)}{x_{k}} - \frac{prod (t, x)}{x_{k} {[1 - prod (t, x)]}^{2}} \end{matrix}

(110)

with the remainder terms

R_{0} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) = R_{k} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) = 0

k = 1, 2, \dots, d

. Furthermore, we can get the generalized De Bruijn identities

\{\begin{matrix} \frac{\partial}{\partial t} J D (P^{(1)} (t, x), P^{(2)} (t, x)) & = - \frac{1}{2} F D_{(b_{0}^{(1)}, b_{0}^{(2)})} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) \\ \frac{\partial}{\partial x_{k}} J D (P^{(1)} (t, x), P^{(2)} (t, x)) & = - \frac{1}{2} F D_{(b_{k}^{(1)}, b_{k}^{(2)})} (P^{(1)} (t, x) ∥ P^{(2)} (t, x)) \\ k & = 1, 2, \dots, d \end{matrix}

(111)

4.2. An Nontrivial Equation

If we let

\{\begin{matrix} a_{k} (u; t, x) & = 0 \\ b_{k} (u; t, x) & = \frac{C_{1} (t, x) C_{2}^{2} (t, x)}{4} {(C_{4} (t, x) - u)}^{2} > 0 \\ k & = 0, 1, 2, \dots, d \end{matrix}

(112)

are continuous derivable functions and

v (u; t, x) = \frac{1}{C_{2} (t, x)} log [\frac{C_{1} (t, x) C_{2}^{2} (t, x)}{4 C_{3} (t, x)} {(C_{4} (t, x) - u)}^{2}]

(113)

is the transformation. Then the Fokker-Planck equations can be rewrite as

\frac{\partial p}{\partial t} = \frac{C_{1} (t, x)}{2} \frac{\partial^{2} p}{\partial v^{2}} + C_{2} (t, x) \frac{\partial p}{\partial v} + \frac{C_{1} (t, x) C_{2}^{2} (t, x)}{4} p

(114)

Using Fourier transform, we can get the solution

\{\begin{matrix} p (v; t, x) & = \frac{e^{\frac{1}{4} \int_{0}^{t} C_{1} (s, x) C_{2}^{2} (s, x) d s}}{\sqrt{2 π \int_{0}^{t} C_{1} (s, x) d s}} e^{- \frac{{[v - v_{0} (x) + \int_{0}^{t} C_{2} (s, x) d s]}^{2}}{2 \int_{0}^{t} C_{1} (s, x) d s}} \\ p (u; t, x) & = \frac{e^{\frac{1}{4} \int_{0}^{x_{k}} C_{1} (t, x) C_{2}^{2} (t, x) d x_{k}}}{\sqrt{2 π \int_{0}^{x_{k}} C_{1} (t, x) d x_{k}}} e^{- \frac{{[v - v_{0} (x) + \int_{0}^{x_{k}} C_{2} (t, x) d x_{k}]}^{2}}{2 \int_{0}^{x_{k}} C_{1} (t, x) d x_{k}}} \\ k & = 1, 2, \dots, d \end{matrix}

(115)

where

v_{0} (x) = \frac{1}{C_{2} (t, x)} log \{\frac{C_{1} (t, x) C_{2}^{2} (t, x)}{4 C_{3} (t, x)} {[C_{4} (t, x) - u_{0} (x)]}^{2}\}

is the logarithm of initial value. Then we need to suppose that there are functions subject to

\{\begin{matrix} \int_{0}^{t} C_{1} (s, x) d t & = \int_{0}^{x_{k}} C_{1} (t, x) d x_{k} \\ \int_{0}^{t} C_{2} (s, x) d s & = \int_{0}^{x_{k}} C_{2} (t, x) d x_{k} \\ \int_{0}^{t} C_{1} (s, x) C_{2}^{2} (s, x) d s & = \int_{0}^{x_{k}} C_{1} (t, x) C_{2}^{2} (t, x) d x_{k} \\ k & = 1, 2, \dots, d \end{matrix}

(116)

and we take

v = v (u; t, x)

into

p (v; t, x)

, we can get a unified density function formula.

As a example, we let

\{\begin{matrix} a_{k} (u; t, x) & \equiv 0 \\ b_{k} (u; t, x) & = b_{k} (t, x) u^{2} > 0 \\ k & = 0, 1, 2, \dots, d \end{matrix}

(117)

and

u_{0} (x) = 1

, the Fokker-Planck equations are

\{\begin{matrix} \frac{\partial p^{(3)}}{\partial t} & = \frac{b_{0} (t, x)}{2} u^{2} \frac{\partial^{2} p^{(3)}}{\partial u^{2}} + 2 b_{0} (t, x) u \frac{\partial p^{(3)}}{\partial u} + b_{0} (t, x) p^{(3)} \\ \frac{\partial p^{(3)}}{\partial x_{k}} & = \frac{b_{k} (t, x)}{2} u^{2} \frac{\partial^{2} p^{(3)}}{\partial u^{2}} + 2 b_{k} (t, x) u \frac{\partial p^{(3)}}{\partial u} + b_{k} (t, x) p^{(3)} \\ k & = 1, 2, \dots, d \end{matrix}

(118)

and

v = log u

i.e.

u = e^{v}

, we can obtain

\{\begin{matrix} \frac{\partial p^{(3)}}{\partial t} & = \frac{b_{0} (t, x)}{2} \frac{\partial^{2} p^{(3)}}{\partial v^{2}} + \frac{3 b_{0} (t, x)}{2} \frac{\partial p^{(3)}}{\partial v} + b_{0} (t, x) p^{(3)} \\ \frac{\partial p^{(3)}}{\partial x_{k}} & = \frac{b_{k} (t, x)}{2} \frac{\partial^{2} p^{(3)}}{\partial v^{2}} + \frac{3 b_{k} (t, x)}{2} \frac{\partial p^{(3)}}{\partial v} + b_{k} (t, x) p^{(3)} \\ k & = 1, 2, \dots, d \end{matrix}

(119)

with the solution

\{\begin{matrix} p^{(3)} (v; t, x) & = \frac{e^{\int_{0}^{t} b_{0} (s, x) d s}}{\sqrt{2 π \int_{0}^{t} b_{0} (s, x) d s}} e^{- \frac{{[v + \frac{3}{2} \int_{0}^{t} b_{0} (s, x) d s]}^{2}}{2 \int_{0}^{t} b_{0} (s, x) d s}} \\ p^{(3)} (v; t, x) & = \frac{e^{\int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}}{\sqrt{2 π \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} e^{- \frac{{[v + \frac{3}{2} \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}]}^{2}}{2 \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} \\ k & = 1, 2, \dots, d \end{matrix}

(120)

and

\{\begin{matrix} p^{(3)} (u; t, x) & = \frac{e^{\int_{0}^{t} b_{0} (s, x) d s}}{\sqrt{2 π \int_{0}^{t} b_{0} (s, x) d s}} e^{- \frac{{[log u + \frac{3}{2} \int_{0}^{t} b_{0} (s, x) d s]}^{2}}{2 \int_{0}^{t} b_{0} (s, x) d s}} \\ p^{(3)} (u; t, x) & = \frac{e^{\int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}}{\sqrt{2 π \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} e^{- \frac{{[log u + \frac{3}{2} \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}]}^{2}}{2 \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} \\ k & = 1, 2, \dots, d \end{matrix}

(121)

Then we suppose that there is a function

β (t, x)

subject to the total differential

d β (t, x) = b_{0} d t + b_{1} d x_{1} + \dots + b_{d} d x_{d}

(122)

and the density function is

p^{(3)} (u; t, x) = \frac{e^{β (t, x)}}{\sqrt{2 π β (t, x)}} e^{- \frac{{[log u + \frac{3}{2} β (t, x)]}^{2}}{2 β (t, x)}}

(123)

Similar to the idea of Section 4.1, if we consider different

β_{3} (t, x)

and

β_{4} (t, x)

in (123), we can get density functions

p^{(3)} (u; t, x)

and

p^{(4)} (u; t, x)

, then we can get the Jeffreys divergence

J D (P^{(3)} (t, x), P^{(3)} (s, y)) = \frac{β_{3} (t, x) + β_{3} (s, t) + 4}{8 β_{3} (t, x) β_{3} (s, y)} {[β_{3} (t, x) - β_{3} (s, y)]}^{2}

(124)

and generalized Fisher information

\{\begin{matrix} F I_{1}^{(t)} (P^{(3)} (t, x)) & = \frac{β_{3} (t, x) + 2}{4 β_{3}^{2} (t, x)} {[b_{0}^{(3)} (t, x)]}^{2} \\ F I_{1}^{(x_{k})} (P^{(3)} (t, x)) & = \frac{β_{3} (t, x) + 2}{4 β_{3}^{2} (t, x)} {[b_{k}^{(3)} (t, x)]}^{2} \end{matrix}

(125)

then the quotients

\{\begin{matrix} \frac{J D (P^{(3)} (t, x), P^{(3)} (s, x))}{{| t - s |}^{2}} & = \frac{β_{3} (t, x) + β_{3} (s, x) + 4}{8 β_{3} (t, x) β_{3} (s, y)} {[\frac{β_{3} (t, x) - β_{3} (s, y)}{t - s}]}^{2} \\ \frac{J D (P^{(3)} (t, {\tilde{x}}_{k}), P^{(3)} (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2}} & = \frac{β_{3} (t, {\tilde{x}}_{k}) + β_{3} (t, {\tilde{y}}_{k}) + 4}{8 β_{3} (t, {\tilde{x}}_{k}) β_{3} (t, {\tilde{y}}_{k})} {[\frac{β_{3} (t, {\tilde{x}}_{k}) - β_{3} (t, {\tilde{x}}_{k})}{x_{k} - y_{k}}]}^{2} \end{matrix}

(126)

and we can easily get the relation

\{\begin{matrix} \frac{J D (P^{(3)} (t, x), P^{(3)} (s, y))}{{| t - s |}^{2} F I_{1}^{(t)} (P^{(3)} (t, x))} & = \frac{β_{3} (t, x)}{β_{3} (s, x)} \frac{β_{3} (t, x) + β_{3} (s, x) + 4}{2 [β_{3} (t, x) + 2]} {[\frac{β_{3} (t, x) - β_{3} (s, y)}{b_{0}^{(3)} (t, x) (t - s)}]}^{2} \\ \frac{J D (P^{(3)} (t, {\tilde{x}}_{k}), P^{(3)} (t, {\tilde{y}}_{k}))}{| x_{k} - y_{k} |^{2} F I_{1}^{(x_{k})} (P^{(3)} (t, {\tilde{x}}_{k}))} & = \frac{β_{3} (t, {\tilde{x}}_{k})}{β_{3} (t, {\tilde{y}}_{k})} \frac{β_{3} (t, {\tilde{x}}_{k}) + β_{3} (t, {\tilde{y}}_{k}) + 4}{2 [β_{3} (t, {\tilde{x}}_{k}) + 2]} {[\frac{β_{3} (t, {\tilde{x}}_{k}) - β_{3} (t, {\tilde{x}}_{k})}{b_{k}^{(3)} (t, x) (x_{k} - y_{k})}]}^{2} \end{matrix}

(127)

for

k = 1, 2, \dots, d

. Without loss of generality, the result (127) also corroborates Theorem 4.

Furthermore, if we consider different

β_{3} (t, x)

and

β_{4} (t, x)

in (123), we can get density functions

p^{(3)} (u; t, x)

and

p^{(4)} (u; t, x)

, then the generalized Fisher divergence at the same space-time points is

\begin{matrix} F D_{(b_{k}^{(3)}, b_{k}^{(4)})} (P^{(3)} (t, x) ∥ P^{(4)} (t, x)) \\ = & \frac{β_{3} (t, x) - β_{4} (t, x)}{4 β_{3}^{2} (t, x) β_{4}^{2} (s, y)} \\ \times \{b_{k}^{(4)} (t, x) β_{3} (t, x) [β_{3}^{2} (t, x) - 2 β_{4}^{2} (s, y)] - b_{k}^{(3)} (t, x) β_{4} (t, x) [β_{4}^{2} (s, y) - 2 β_{3}^{2} (t, x)]\} \\ \times [4 β_{3} (t, x) + 4 β_{4} (t, x) - 3 β_{3} (t, x) β_{4} (t, x)] [b_{k}^{(4)} (t, x) β_{3} (t, x) - b_{k}^{(3)} (t, x) β_{4} (t, x)] \end{matrix}

(128)

with the remainder terms

R_{k} (P^{(3)} (t, x) ∥ P^{(4)} (t, x)) = 2 [b_{k}^{(3)} (t, x) - b_{k}^{(4)} (t, x)]

k = 0, 1, 2, \dots, d

. Then the generalized De Bruijn identities are

\{\begin{matrix} \frac{\partial}{\partial t} J D (P^{(3)} (t, x), P^{(4)} (t, x)) & = - \frac{1}{2} F D_{(b_{0}^{(3)}, b_{0}^{(4)})} (P^{(3)} (t, x) ∥ P^{(4)} (t, x)) - 2 [b_{0}^{(3)} (t, x) - b_{0}^{(4)} (t, x)] \\ \frac{\partial}{\partial x_{k}} J D (P^{(3)} (t, x), P^{(4)} (t, x)) & = - \frac{1}{2} F D_{(b_{k}^{(3)}, b_{k}^{(4)})} (P^{(3)} (t, x) ∥ P^{(4)} (t, x)) - 2 [b_{k}^{(3)} (t, x) - b_{k}^{(4)} (t, x)] \\ k & = 1, 2, \dots, d \end{matrix}

(129)

4.3. An Interesting Equation

If we let

\{\begin{matrix} a_{k} (u) & = - \frac{3 D_{1}^{2} D_{3}}{8} (u - D_{6}) \\ b_{k} (u) & = D_{1}^{2} D_{5}^{2} [1 - \frac{D_{3}}{4 D_{5}^{2}} {(u - D_{6})}^{2}] \\ k & = 0, 1, 2, \dots, d \end{matrix}

(130)

are continuous derivable functions and

u = \frac{2 D_{5}}{\sqrt{D_{3}}} sin (\frac{\sqrt{D_{3}}}{2} v + D_{4}) + D_{6}

(131)

where

D_{1}

D_{6}

are nonnegative functions independent of u and we omit

(t, x)

Thus, we can rewrite the Fokker-Planck equations as

\{\begin{matrix} \frac{\partial p}{\partial t} & = \frac{D_{1}^{2}}{2} \frac{\partial^{2} p}{\partial v^{2}} + \frac{D_{1}^{2} D_{3}}{8} p \\ \frac{\partial p}{\partial x_{k}} & = \frac{D_{1}^{2}}{2} \frac{\partial^{2} p}{\partial v^{2}} + \frac{D_{1}^{2} D_{3}}{8} p \\ k & = 1, 2, \dots, d \end{matrix}

(132)

with the solution

p (v; t, x) = \frac{e^{\frac{1}{8} \int_{0}^{t} D_{1}^{2} D_{3} d s}}{\sqrt{2 π \int_{0}^{t} D_{1}^{2} d s}} e^{- \frac{{(v - v_{0})}^{2}}{2 \int_{0}^{t} D_{1}^{2} d s}}

(133)

Recall the Lemma 2, we obtain

\begin{matrix} \int_{I_{u}} p (u; t, x) d u \\ = \frac{D_{5} e^{\frac{1}{8} \int_{0}^{t} D_{1}^{2} D_{3} d s}}{\sqrt{2 π \int_{0}^{t} D_{1}^{2} d s}} \int_{R} cos (\frac{\sqrt{D_{3}}}{2} \tilde{u} + D_{4}) e^{- \frac{{(\tilde{u} - {\tilde{u}}_{0})}^{2}}{2 \int_{0}^{t} D_{1}^{2} d s}} d \tilde{u} \\ = 1 \end{matrix}

(134)

and use the result

\begin{matrix} \frac{1}{\sqrt{2 π H}} \int_{R} cos (ω x + φ) e^{- \frac{{(x - G)}^{2}}{2 H}} d x = e^{- \frac{ω^{2} H}{2}} cos (ω G + φ) \end{matrix}

(135)

the integral (134) can be write as

D_{5} e^{\frac{1}{8} \int_{0}^{t} D_{1}^{2} D_{3} d s - \frac{D_{3}}{8} \int_{0}^{t} D_{1}^{2} d s} cos (\frac{\sqrt{D_{3}}}{2} {\tilde{u}}_{0} + D_{4}) = 1

(136)

where

{\tilde{u}}_{0} = \frac{2}{\sqrt{D_{3}}} arcsin [\frac{\sqrt{D_{3}}}{2 D_{5}} (u_{0} - D_{6})]

Here we give a simple example. Let

u \in [- 1, 1]

\{\begin{matrix} a_{k} (u; t, x) & = - \frac{3}{2} b_{k} (t, x) u \\ b_{k} (u; t, x) & = b_{k} (t, x) (1 - u^{2}) \\ k & = 0, 1, 2, \dots, d \end{matrix}

(137)

where

b_{k} (t, x) > 0

and we can obtain the Fokker-Planck equations formed as

\{\begin{matrix} \frac{\partial p}{\partial t} & = \frac{b_{0} (t, x)}{2} \frac{\partial^{2}}{\partial u^{2}} [(1 - u^{2}) p] - \frac{3 b_{0} (t, x)}{2} \frac{\partial}{\partial u} (u p) \\ \frac{\partial p}{\partial x_{k}} & = \frac{b_{k} (t, x)}{2} \frac{\partial^{2}}{\partial u^{2}} [(1 - u^{2}) p] - \frac{3 b_{k} (t, x)}{2} \frac{\partial}{\partial u} (u p) \\ k & = 1, 2, \dots, d \end{matrix}

(138)

with initial value

u_{0} (x) = 0

Using the formula

u = sin v

, we can rewrite the Fokker-Planck equations

\{\begin{matrix} \frac{\partial p}{\partial t} & = \frac{b_{0} (t, x)}{2} \frac{\partial^{2} p}{\partial v^{2}} + \frac{b_{0} (t, x)}{2} p \\ \frac{\partial p}{\partial x_{k}} & = \frac{b_{k} (t, x)}{2} \frac{\partial^{2} p}{\partial v^{2}} + \frac{b_{k} (t, x)}{2} p \\ k & = 1, 2, \dots, d \end{matrix}

(139)

with the solution

\{\begin{matrix} p (v; t, x) & = \frac{e^{\frac{1}{2} \int_{0}^{t} b_{0} (s, x) d s}}{\sqrt{2 π \int_{0}^{t} b_{0} (s, x) d s}} e^{- \frac{v^{2}}{2 \int_{0}^{t} b_{0} (s, x) d s}} \\ p (v; t, x) & = \frac{e^{\frac{1}{2} \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}}{\sqrt{2 π \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} e^{- \frac{v^{2}}{2 \int_{0}^{x_{k}} b_{k} (t, x) d x_{k}}} \end{matrix}

(140)

and the variable substitution is

u = sin v

Similarly, we suppose that there is a function

β (t, x)

subject to the total differential

d β (t, x) = b_{0} d t + b_{1} d x_{1} + \dots + b_{d} d x_{d}

(141)

then the density is

\{\begin{matrix} p (v; t, x) & = \frac{e^{\frac{1}{2} β (t, x)}}{\sqrt{2 π β (t, x)}} e^{- \frac{v^{2}}{2 β (t, x)}} \\ u & = sin v \end{matrix}

(142)

From density function (142), if we consider different

β_{5} (t, x)

and

β_{6} (t, x)

in (142), we can get density functions

p^{(5)} (u; t, x)

and

p^{(6)} (u; t, x)

, then we can get the Jeffreys divergence and generalized Fisher information

\begin{matrix} J D (P^{(5)} (t, x), P^{(5)} (s, y)) = \frac{1 - β_{5} (t, x) - β_{5} (s, y)}{2 β_{5} (t, x) β_{5} (s, y)} {[β_{5} (t, x) - β_{5} (s, y)]}^{2} \end{matrix}

(143)

and

\{\begin{matrix} F I_{1}^{(t)} (P^{(5)} (t, x)) & = \frac{1 - 2 β_{5} (t, x)}{2 β_{5}^{2} (t, x)} {[b_{0}^{(5)} (t, x)]}^{2} \\ F I_{1}^{(x_{k})} (P^{(5)} (t, x)) & = \frac{1 - 2 β_{5} (t, x)}{2 β_{5}^{2} (t, x)} {[b_{k}^{(5)} (t, x)]}^{2} \end{matrix}

(144)

then the quotients

\{\begin{matrix} \frac{J D (P^{(5)} (t, x), P^{(5)} (s, x))}{{| t - s |}^{2}} & = \frac{1 - β_{5} (t, x) - β_{5} (s, x)}{2 β_{5} (t, x) β_{5} (s, x)} {[\frac{β_{5} (t, x) - β_{5} (s, x)}{t - s}]}^{2} \\ \frac{J D (P^{(5)} (t, {\tilde{x}}_{k}), P^{(5)} (t, {\tilde{y}}_{k}))}{| {\tilde{x}}_{k} - {\tilde{y}}_{k} |^{2}} & = \frac{1 - β_{5} (t, {\tilde{x}}_{k}) - β_{5} (t, {\tilde{y}}_{k})}{2 β_{5} (t, {\tilde{x}}_{k}) β_{5} (t, {\tilde{y}}_{k})} {[\frac{β_{5} (t, {\tilde{x}}_{k}) - β_{5} (t, {\tilde{y}}_{k})}{{\tilde{x}}_{k} - {\tilde{y}}_{k}}]}^{2} \end{matrix}

(145)

Obviously, we can easily get

\{\begin{matrix} \frac{J D (P^{(5)} (t, x), P^{(5)} (s, x))}{{| t - s |}^{2} F I_{1}^{(t)} (P^{(5)} (t, x))} & = \frac{1 - β_{5} (t, x) - β_{5} (s, x)}{1 - 2 β_{5} (t, x)} \frac{β_{5} (t, x)}{β_{5} (s, x)} {[\frac{β_{5} (t, x) - β_{5} (s, x)}{b_{0}^{(5)} (t, x) (t - s)}]}^{2} \\ \frac{J D (P^{(5)} (t, {\tilde{x}}_{k}), P^{(5)} (t, {\tilde{y}}_{k}))}{| {\tilde{x}}_{k} - {\tilde{y}}_{k} |^{2} F I_{1}^{(x_{k})} (P^{(5)} (t, x))} & = \frac{1 - β_{5} (t, {\tilde{x}}_{k}) - β_{5} (t, {\tilde{y}}_{k})}{1 - 2 β_{5} (t, {\tilde{x}}_{k})} \frac{β_{5} (t, {\tilde{x}}_{k})}{β_{5} (t, {\tilde{y}}_{k})} {[\frac{β_{5} (t, {\tilde{x}}_{k}) - β_{5} (t, {\tilde{y}}_{k})}{b_{k}^{(5)} (t, x) ({\tilde{x}}_{k} - {\tilde{y}}_{k})}]}^{2} \end{matrix}

(146)

for

k = 1, 2, \dots, d

. Without loss of generality, the result (146) corroborates Theorem 4.

Furthermore, if we consider different

β_{5} (t, x)

and

β_{6} (t, x)

in (142), we can get density functions

p^{(5)} (u; t, x)

and

p^{(6)} (u; t, x)

, then the generalized Fisher divergence at the same space-time points is

\begin{matrix} F D_{(b_{k}^{(5)}, b_{k}^{(6)})} (P^{(5)} (t, x) ∥ P^{(6)} (t, x)) \\ = & \frac{[β_{5} (t, x) - β_{6} (t, x)] [b_{k}^{(6)} β_{5} (t, x) - b_{k}^{(5)} β_{6} (t, x)]}{2 β_{5}^{2} (t, x) β_{6}^{2} (t, x)} [β_{5} (t, x) - β_{5}^{2} (t, x) + β_{6} (t, x) - β_{6}^{2} (t, x)] \end{matrix}

(147)

with the remainder terms

R_{k} (P^{(5)} (t, x) ∥ P^{(6)} (t, x)) = b_{k}^{(5)} (t, x) - b_{k}^{(6)} (t, x)

k = 0, 1, 2, \dots, d

. Then the generalized De Bruijn identities are

\{\begin{matrix} \frac{\partial}{\partial t} J D (P^{(5)} (t, x), P^{(6)} (t, x)) & = - \frac{1}{2} F D_{(b_{0}^{(5)}, b_{0}^{(6)})} (P^{(5)} (t, x) ∥ P^{(6)} (t, x)) - [b_{0}^{(5)} (t, x) - b_{0}^{(6)} (t, x)] \\ \frac{\partial}{\partial x_{k}} J D (P^{(5)} (t, x), P^{(6)} (t, x)) & = - \frac{1}{2} F D_{(b_{k}^{(5)}, b_{k}^{(6)})} (P^{(5)} (t, x) ∥ P^{(6)} (t, x)) - [b_{k}^{(5)} (t, x) - b_{k}^{(6)} (t, x)] \\ k & = 1, 2, \dots, d \end{matrix}

(148)

5. Conclusions

In this paper, we generalize the classical definitions of entropy, divergence and Fisher information, obtain these measures on space-time random field. In addition to that, we also get the Fokker-Planck equations (55) for space-time random field and obtain the density functions. Moreover, we obtain the Jeffreys divergence of a space-time random field at different space-time positions, and we get the approximation of the ratio of Jeffreys divergence to the square of space-time coordinate difference to the generalized Fisher information(68). Further, we use the Jeffreys divergence on two space-time random fields from same type but different parameters Fokker-Planck equations, to obtain generalized De Bruijn identities (77), and get the relation between Jeffreys divergence of space-time random field and generalized Fisher divergence. Finally, we give three examples of Fokker-Planck equations, with their solutions, calculate the corresponding Jeffreys divergence, generalized Fisher information and Fisher divergence and obtain the De Bruijn identities. These results encourage further research into the entropy divergence of space-time random fields, which advances the pertinent fields of information entropy, Fisher information, and De Bruijn identities.

Acknowledgments

The author would like to thank Prof. Pingyi Fan for providing relevant references and helpful discussions on topics related to this work.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:

KL	Kullback-Leibler divergence
FI	Fisher information
CFI	Cross Fisher information
FD	Fisher divergence
sFD	symmetric Fisher divergence
JD	Jeffreys divergence

References

Risken, H. The Fokker-Planck equation: methods of solution and applications; Springer Heidelberg: Berlin, 1984. [Google Scholar]
Stam, A. J. Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. Control 1959, 2, 101–112. [Google Scholar] [CrossRef]
Barron, A. R. Entropy and the central limit theorem. Ann. Probab. 1986, 14, 336–342. [Google Scholar] [CrossRef]
Johnson, O. Information theory and the central limit theorem; Imperial college Press: London, U.K., 2004. [Google Scholar]
Guo, D. Relative entropy and score function: New information estimation relationships through arbitrary additive perturbation. Proc. IEEE Int. Symp. Inf. Theory, Seoul, South Korea, Jun./Jul. 2009; pp. 814–818.
Toranzo, I. V.; Zozor, S.; Brossier, J-M. Generalization of the De Bruijn Identity to General ϕ-Entropies and ϕ-Fisher Informations. IEEE Trans. Inform. Theory 2018, 64, 6743–6758. [Google Scholar] [CrossRef]
Kharazmi, O.; Balakrishnan, N. Cumulative residual and relative cumulative residual Fisher information and their properties. IEEE Trans. Inform. Theory 2021, 67, 6306–6312. [Google Scholar] [CrossRef]
Kolmogorov, A. N. The local structure of turbulence in incompressible viscous fluid for very large Reynolds numbers. Dokl. Akad. Nauk SSSR 1941, 30, 299–303. [Google Scholar]
Kolmogorov, A. N. On the degeneration of isotropic turbulence in an incompressible viscous flu. Dokl. Akad. Nauk SSSR 1941, 31, 538–542. [Google Scholar]
Kolmogorov, A. N. Dissipation of energy in isotropic turbulence. Dokl. Akad. Nauk SSSR 1941, 32, 19–21. [Google Scholar]
Yaglom, A. M. Some classes of random fields in n-dimensional space, related to stationary random processes. Theory Probab. Its Appl. 1957, 2, 273–320. [Google Scholar] [CrossRef]
Yaglom, A. M. Correlation theory of stationary and related Random functions. Volume I: basic results; Springer-Verlag: New York, 1987. [Google Scholar]
Yaglom, A. M. Correlation theory of stationary and related random functions. Volume II: supplementary notes and references; Springer-Velag: Berlin, 1987. [Google Scholar]
Bowditch, A.; Sun, R. The two-dimensional continuum random field Ising model. Ann. Probab. 2022, 50, 419–454. [Google Scholar] [CrossRef]
Bailleul, I.; Catellier, R.; Delarue, F. Propagation of chaos for mean field rough differential equations. Ann. Probab. 2021, 49, 944–996. [Google Scholar] [CrossRef]
Wu, L.; Samorodnitsky, G. Regularly varying random fields. Stoch. Process Their. Appl. 2020, 130, 4470–4492. [Google Scholar] [CrossRef]
Koch, E.; Dombry, C.; Robert, C. Y. A central limit theorem for functions of stationary max-stable random fields on R^d. Stoch. Process Their. Appl. 2020, 129, 3406–3430. [Google Scholar] [CrossRef]
Ye, Z. On entropy and ε-entropy of random fields. Ph.D.dissertation, Cornell University, 1989. [Google Scholar]
Ye, Z.; Berger, T. A new method to estimate the critical distortion of random fields. IEEE Trans. Inform. Theory 1992, 38, 152–157. [Google Scholar] [CrossRef]
Ye, Z.; Berger, T. Information Measures for Discrete Random Fields; Science Press: Beijing/New York, 1998. [Google Scholar]
Ye, Z.; Yang, W. Random Field: Network Information Theory and Game Theory; Science Press: Beijing, 2023. (in Chinese) [Google Scholar]
Ma, C. Stationary random fields in space and time with rational spectral densities. IEEE Trans. Inform. Theory 2007, 53, 1019–1029. [Google Scholar] [CrossRef]
Hairer, M. A theory of regularity structures. Invent. Math. 2014, 198, 269–504. [Google Scholar] [CrossRef]
Hairer, M. Solving the KPZ equation. Ann. Math. 2013, 178, 559–664. [Google Scholar] [CrossRef]
Kremp, H.; Perkowski, N. Multidimensional SDE with distributional drift and Lévy noise. Bernoulli 2022, 28, 1757–1783. [Google Scholar] [CrossRef]
Beeson, R.; Namachchivaya, N. S.; Perkowski, N. Approximation of the filter equation for multiple timescale, correlated, nonlinear systems. SIAM J. Math. Anal. 2022, 54(3), 3054–3090. [Google Scholar] [CrossRef]
Song, Z.; Zhang, J. A note for estimation about average differential entropy of continuous bounded space-time random field. Chinese J. Electron 2022, 31, 793–803. [Google Scholar] [CrossRef]
Kramers, H. A. Brownian motion in a field of force and the diffusion model of chemical reactions. Physica 1940, 7, 284–304. [Google Scholar] [CrossRef]
Moyal, J. E. Stochastic processes and statistical physics. J R Stat Soc Series B Stat Methodol 1949, 11, 150–210. [Google Scholar] [CrossRef]
Shannon, C. E. A mathematical theory of communication. Bell System Technical Journal 1948, 27, 379–423, 623–656. [Google Scholar] [CrossRef]
Neeser, F. D.; Massey, J. L. Proper complex random processes with applications to information theory. IEEE Trans. Inform. Theory 1991, 39, 1293–1302. [Google Scholar] [CrossRef]
Ihara, S. Information theory-for continuous systems; World Scientific: Singapore, 1993. [Google Scholar]
Gray, R. M. Entropy and information theory; Springer: Boston, 2011. [Google Scholar]
Bach, F. Information Theory With Kernel Methods. IEEE Trans. Inform. Theory 2023, 69, 752–775. [Google Scholar] [CrossRef]
Kullback, S.; Leibler, R. A. On information and sufficiency, Ann. Math. Stat. 1951, 22, 79–86. [Google Scholar] [CrossRef]
Jeffreys, H. An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. A 1946, 186, 453–461. [Google Scholar]
Fuglede, B.; Topsøe, F. Jensen-Shannon divergence and Hilbert space embedding. In Proceedings of the IEEE International Symposium on Information Theory (ISIT), Chicago, IL, USA, 27 June-2 July 2004; p. 31. [Google Scholar]
Rényi, A. On measures of entropy and information. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, 1961, 1, 547–561. [Google Scholar]
She, R.; Fan, P.; Liu, X.-Y.; Wang, X. Interpretable Generative Adversarial Networks With Exponential Function. IEEE Trans. Signal Process. 2021, 69, 3854–3867. [Google Scholar] [CrossRef]
Liu, S.; She, R.; Zhu, Z.; Fan, P. Storage Space Allocation Strategy for Digital Data with Message Importance. Entropy 2020, 22, 591. [Google Scholar] [CrossRef]
She, R.; Liu, S.; Fan, P. Attention to the Variation of Probabilistic Events: Information Processing with Message Importance Measure. Entropy 2019, 21, 439. [Google Scholar] [CrossRef]
Wan, S.; Lu, J.; Fan, P.; Letaief, K.B. Information Theory in Formation Control: An Error Analysis to Multi-Robot Formation. Entropy 2018, 20, 618. [Google Scholar] [CrossRef]
She, R.; Liu, S.; Fan, P. Recognizing Information Feature Variation: Message Importance Transfer Measure and Its Applications in Big Data. Entropy 2018, 20, 401. [Google Scholar] [CrossRef] [PubMed]
Nielsen, F. An Elementary Introduction to Information Geometry. Entropy 2020, 22, 1100. [Google Scholar] [CrossRef] [PubMed]
Nielsen, F. On the Jensen–Shannon Symmetrization of Distances Relying on Abstract Means. Entropy 2019, 21, 485. [Google Scholar] [CrossRef] [PubMed]
Nielsen, F.; Nock, R. Generalizing skew Jensen divergences and Bregman divergences with comparative convexity. IEEE Signal Process. Lett. 2017, 24, 1123–1127. [Google Scholar] [CrossRef]
Furuichi, S.; Minculete, N. Refined Young Inequality and Its Application to Divergences. Entropy 2021, 23, 514. [Google Scholar] [CrossRef]
Pinele, J.; Strapasson, J.E.; Costa, S.I. The Fisher-Rao Distance between Multivariate Normal Distributions: Special Cases, Bounds and Applications. Entropy 2020, 22, 404. [Google Scholar] [CrossRef]
Reverter, F.; Oller, J.M. Computing the Rao distance for Gamma distributions. J. Comput. Appl. Math. 2003, 157, 155–167. [Google Scholar] [CrossRef]
Pawula, R. F. Generalizations and extensions of the Fokker-Planck-Kolmogorov equations. IEEE Trans. Inform. Theory 1967, 13, 33–41. [Google Scholar] [CrossRef]
Pawula, R. F. Approximation of the linear Boltzmann equation by the Fokker-Planck equation. Phys. Rev. 1967, 162, 186–188. [Google Scholar] [CrossRef]
Khoshnevisan, D.; Shi, Z. Brownian Sheet and Capacity, Ann. Probab. 1999, 27, 3, 1135–1159. [Google Scholar] [CrossRef]
Daniel, R.; Marc, Y. Continuous Martingales and Brownian Motion, 2nd. ed; Springer-Verlag: New York, 1999. [Google Scholar]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Jeffreys Divergence and Generalized Fisher Information Measures on Fokker-Planck Space-Time Random Field

Abstract

1. Introduction

1.1. Space-time random field

1.2. Kramers-Moyal expansion and Fokker-Planck equation

1.3. Differential entropy and De Bruijn identity

1.4. Entropy Divergence

2. Notations, Definitions and Propositions

2.1. Notations and Assumptions

2.2. Definitions

3. Main Results and Proofs

4. Three Fokker-Planck Random Fields and Their Corresponding Information Measures

4.1. A Trivial Equation

4.2. An Nontrivial Equation

4.3. An Interesting Equation

5. Conclusions

Acknowledgments

Conflicts of Interest

Abbreviations

References

MDPI Initiatives

Important Links

Subscribe