From structured data to evolution linear partial differential equations E. Lorina,b a
b
School of Mathematics and Statistics, Carleton University, Ottawa, Canada, K1S 5B6 Centre de Recherches Math´ematiques, Universit´e de Montr´eal, Montr´eal, Canada, H3T 1J4
Abstract This paper is devoted to the derivation of computational methods for constructing partial differential equations from data. Following some recent works [7, 14, 15, 20], we propose a methodology based on symbolic calculus [8, 9, 13], pseudospectral methods [2, 3] and stochastic processes [6], in order to determine non-constant coefficients of linear evolution Partial Differential Equations (PDEs), from a set of structured data constituted by solutions, at given times and positions of an unknown linear PDE. Keywords: Partial differential equations, numerical approximation, operator symbols, pseudospectral methods. 1. Introduction Machine learning is one of the most active research areas in applied mathematics, statistics and computer science. Among fundamental problems in this field is the reconstruction of differential equations (DE) from experimental data [7, 14, 15, 20]. In this work, we propose simple and direct methods for this problem, which however do not directly use machine learning paradigm. More specifically, we assume that a set of “data” corresponding to the solution of an unknown equation, are given, along with the corresponding times and positions. This is typically what “in-lab” experiments would provide, making the proposed approach potentially relevant from a practical point of view. A similar paradigm was used in [7] with techniques distinct, from the ones developed in this paper. This problem is very complex, as the differential equation could be in principle, of any kind, high order, fractional, nonlinear, non-homogeneous, etc. In this work, we will assume that the searched equation is a linear evolution equation, with non-constant coefficients. To a certain extent, quasilinear equations will also be considered. The latter assumption is the main drawback of the presented approach, as the “general form” of the searched model is imposed, which may lead to an inaccurate equation. However, in modeling in classical or quantum physics/chemistry/engineering [4], the general form of the searched equation is often preliminarily known, but not necessarily their coefficients. Email address:
[email protected] (E. Lorin) Preprint submitted to Elsevier
December 19, 2018
From a set of given experimental data: i) v n := {vjn }j∈Zd ,n∈N , ii) at locations D = {xj }j∈Zd , iii) and times {tn }n∈N , we plan to construct a partial differential equation, assuming that the data are approximate solutions to some linear d-dimensional PDE, at {tn }n∈N and at {xj }j∈Zd . We denote unj = u(tn , xj ) (≈ vjn ), xj = xj1 , · · · , xjd and j = (j1 , · · · , jd ), where u is the solution to the searched PDE. We will denote along the paper, Kd (resp. N ) a finite subset of Zd (resp. N), for d ∈ N∗ . 1.1. Random perturbation In practice, we assume that the given data are perturbed by the measurements. In other words, we assume that the given data are perturbations of the exact solution values to the PDE. We will then introduce a stochastic process {ε(t, x, ζ) : (t, x) ∈ R+ × Rd }, with uniform distribution U(0, ε∞ ), for ε∞ > 0 small enough. We define ε(·, ·, ζ) : R+ × Rd 7→ S where S is the state space. We will then assume that, for any n and j ∈ Zd vjn;` := unj 1 + εn;` , j
(1)
where εn;` j = ε(tn , xj , ζ ` ), where ` is the realization index. This random perturbation models the uncertainty in the structure and estimate of the given data. The subsequent analysis is more complex than considering a uniform perturbation εnj on the data, unj + εjn;` , but potentially more realistic for a large of experimental problems. As a consequence for each set of data, the predicted coefficients/equations are dependent on the random process. The proposed algorithm then needs to be repeated and by an average process. Notice that the limK→+∞
1 PK n;` 1 PK n;` = unj limK→+∞ = unj (1 + ε) `=1 vj `=1 1 + εj K K
where ε is the mean of the discrete random variable, which is assumed constant (the mean is (n, j)-independent). This is a crucial property which allows to justify convergence of the proposed reconstruction algorithm. In order to accelerate the convergence, we could also use MultiResolution Analysis (MRI) algorithms [12]. The motivation is that the random noise on the data, a priori have high spatial frequencies and without any particular structure. To this hand, we expect that the frequency of the operator symbol or of the coefficients, do not possess high frequencies as the symbols of the searched (relatively low order) PDE are polynomial functions (of relatively low degree). Notice, direct MRI on the data may be misleading as the solution itself can possess high frequencies coming from the initial data. 1.2. Filtering In order to refine the procedure described in this paper, it may be useful to filter the approximate operator symbols. The idea is based on the fact that i) relevant/practical mathematical models in physics, chemistry, etc, involve relatively low order operators (otherwise, the model may not be relevant for practical purposes, that is numerical or analytical predictions), and as a consequence ii) the associated symbols have a relatively low homogeneity 2
degree [13]. A filtering could then be applied to remove from the approximate symbol the very large spatial frequencies, which do not exist in the exact one. However, by construction, the approximate symbol is itself constructed as the ratio of the Fourier transform of the given data. Although, the data themselves could contain high frequencies (depending again for instance on the initial data), the ratio of the Fourier transform of consecutive data should mainly contain low frequencies. The random perturbation will only pollute the high frequency part of the reconstructed symbol. For small enough wavenumbers, the approximate symbol should then provide a good approximation of the exact symbol. As an example, we report in Fig. 1 a comparison to the exact, approximate and filtered approximate symbols, using the method developed below (see details in Section 2 and Subsection 5.1) for the searched equation ∂t u − 5 × 10−2 ux − 5∂xx u − 10−1 ∂xxx u + 4∂xxxx u − 10−3 ∂xxxxx u = 1 .
(2)
We remove from the approximate symbol the high frequencies which come from the random noise.
2
10
1
10
0
10
Modulus of filtered reconstructed symbol Modulus of reconstructed symbol Modulus of exact symbol −5
−4
−3
−2
−1
0 1 Spatial frequency
2
3
4
5
Figure 1: Spatial spectrum: exact, approximate and filtered approximate symbols, for Equation 2.
1.3. Organization of the paper This paper is organized as follows. Section 2 is devoted to a symbol-based method for reconstructing time-dependent coefficients of homogeneous and non-homogeneous partial differential equations, Subsections 2.1, 2.2. Fractional equations are also considered in Subsection 2.3. In Section 3, we present the methodology for constructing space-dependent coefficients to unknown PDEs from randomly perturbed data. The case of first order in-time 3
equations is first addressed in Subsection 3.1. The general case (N th order equation) is presented in Subsection 3.3. The corresponding methodology is mainly based on pseudospectral methods, but finite differences can also be used, see Subsection 3.2. Extension to quasilinear equations is discussed in Subsection 3.4. One of the most important questions, which is, how to determine the order of the searched equation, is addressed in Section 4. One- and two-dimensional numerical experiments are presented in Section 5, illustrating the derived methodologies. We finally conclude in Section 6. 2. Symbolic method: first order in-time linear equation In this section, we consider the following order N (∈ N) equation, which is assumed in d dimensions, of the form X ∂t u(t, x) + (t, x) ∈ (0, T ) × Rd (3) a(α) (t)Dxα u(t, x) = f (t, x), |α|6N
with N 6 1 and α = (α1 , · · · , αn ) ∈ Nn is a multi-index with n ∈ N∗ . Fractional equations could as well be considered, see Subsection 2.3. We intend to determine the operator P = P (α) (t)Dxα and the function f from the data. We denote by u b the Fourier transform |α|6k a of u / x 1 u b(t, ξ) = (2π)d
Z
u(t, x)e−ix·ξ dx, (t, ξ) ∈ [0, T ] × Rd .
Rd
with u(t, ·) at least L2 (Rd ). We then denote by p the partial /x (polynomial) symbol of P [8, 9], p(t, ξ) = σx P (t, Dx ) , such that Z 1 p(t, ξ)b u(t, ξ) = P (t, Dx )u(t, x)e−ix·ξ dx . d (2π) Rd α P That is, p(t, ξ) = |α|6N a(α) (t) iξ . The equation is Fourier space reads ∂t u b(t, ξ) + p(t, ξ)b u(t, ξ) = fb(t, ξ),
(t, x) ∈ (0, T ) × Rd .
(4)
The objective is i) to construct an approximation pe of p, from which ii) we will deduce an approximate operator Pe. We construct the discrete Fourier transform in space for consecutive times, of {vjn+k }j∈Zd , with k ∈ N∗ . The corresponding DFT are denoted {b vkn+k }k∈Zd , and are defined as follows. X vbkn+k = e−2iπk·xj vjn+k . j∈Zd
4
2.1. Homogeneous equation We assume here that f = 0. That is, we search for a homogeneous equation. Taking the Fourier transform in space of (3), we get Z t p(s, ξ)ds u b0 (ξ) u b(t, ξ) = exp − 0
where u0 ∈ L2 (Rd ). Thus, from two consecutive “times” tn < tn+1 Z tn+1 p(s, ξ)ds u b(tn , ξ) . u b(tn+1 , ξ) = exp − tn
Denoting ∆tn := tn+1 − tn and u b(tn , ξ) 6= 0, this is formally equivalent to Z tn+1 u 1 1 b(tn+1 , ξ) p(s, ξ)ds = − log . ∆tn tn ∆tn u b(tn , ξ) At the discrete level and for ` fixed, we denote by {pkn;` }k the sequence computed from given d data {vjn;` }j , which are assumed of the form vjn;` = u(tn , xj )(1 + εn;` j ) with k ∈ Z , and n;` vbk 6= 0, where pn;` k
vbn+1;` 1 log k n;` . := − ∆tn vbk
This sequence is dependent on the random process (`-dependence). In order to refine the data, we then need to repeat the process, and we define pnk
K 1X pn;` := lim k K→+∞ K `=1
where, the index ` refers to the actual realization. The more data, the better the accuracy of the developed algorithm. The latter result is justified by the linearity of the equation. Assume for the sake of simplicity that εn;` is only time-dependent, and we denote {b unk }k the n;` Fourier transform of {uj }j . We formally have vbn+1;` 1 PK n;` 1 PK k p = log K `=1 k K `=1 vbkn;` u 1 PK bn+1;` +u bn+1;` εn+1;` k k = log K `=1 u bkn;` + u bkn;` εn;` n+1 u 1 + εn+1;` bk 1 PK = log + log . u bnk K `=1 1 + εn;` For |εn;` | 6 ε∞ is small enough, we get K K K u 1X 1X bn+1 1X n;` k n+1;` = pk − log ε − εn;` + O(ε2∞ ) n K `=1 u bk K `=1 K `=1
5
As K K n1X o 1X n+1;` n;` lim ε − ε = 0, K→+∞ K K `=1 `=1
we expect for K large K 1X n;` pk − σx (P (t, Dx ) tn , k) K `=1
to be small. Remark 2.1. An alternative approach, consists in taking the average of the realization before implementing the method presented above. In this case, we benefit from the fact that the unk , and we can define mean is space-time independent (1). Thus vbkn = (1 + ε)b pnk :=
bn+1 vbkn+1 u k = . vbkn u bnk
From the knowledge of pnk for each n in a subset N ⊂ N, and k in a subset Kd ⊂ Zd , we can reconstruct an approximation of the operator P . More specifically, we define by pe : (t, ξ) ∈ [0, T ] × K 7→ pe(t, ξ) a polynomial function, interpolating the sequence pnk k,n such that, for any k ∈ Kd and any n ∈ N pe(tn , ξ k ) = pnk . That is pe(t, ξ) = Π {pnk }(k,n)∈Kd ×N (t, ξ) where Π is a polynomial interpolation operator. From this function, we define the operator −1 e P (t, Dx ) from its symbol σξ pe(t, ·) . Thus Pe is defined by Z −1 e σξ pe(t, ξ)b u(t, ξ) = P (t, Dx )u(t, x) := pe(t, ξ)b u(t, ξ)eix·ξ dξ Rd
and the reconstructed equation reads (t, x) ∈ (0, T ) × Rd ,
∂t u(t, x) + Peu(t, x) = 0,
(5)
which is such that for k ∈ Kd and n ∈ N σx Pe(t, Dx ) (tn , ξ k ) = σx (P (t, Dx ) (tn , ξ k ) . By construction, kp − pek∞ is bounded by a constant depending on CardKd , CardN , the regularity of p and the size of the convex hull of Kd × N . B-splines can as well be employed to further reduce kp − pek∞ , see [10]. Notice that the methodology does not require any assumption of the spatial order of the searched equation. 6
Remark 2.2. If we know the general form of the searched equation and its order, the strategy can be refined. Say we search for b(α) such that X ∂t u(t, x) + b(α) (t)Dxα u(t, x) = 0, (t, x) ∈ (0, T ) × Rd . (6) |α|6N
We denote by t 7→ A(α) (t) the antiderivative of t 7→ a(α) (t) X α (α) (α) u b(tn+1 , ξ) = exp − (iξ) A (tn+1 ) − A (tn ) u b(tn , ξ) . |α|6N
We then expect that {pnk }k will provide an approximation to {p(tn , ξ k )}k . In order to determine t 7→ a(α) (t), we search for t 7→ B (α) (t) approximating t 7→ A(α) (t) X
α
(iξ k )
B
(α)
(tn+1 ) − B
|α|6N
(α)
u bn+1 k , (tn ) = log u bnk
(7)
where we have denoted u bn+1 =u b(tn+1 , xk ), and for u bnk = 6 0. For each n, we then determine k (α) B (tn ), by considering a sufficiently large number of equations (7), for different values of k. Remark 2.3. In the case of equations with constant coefficients a(α) , (7) is solved, by considering again the equation X
α
(α)
(iξ k )
α
|α|6N
u bn+1 1 = log kn ∆tn u bk
for a sufficiently large number values of k (depending on N ). 2.2. Non-homogeneous equation The same approach can still be applied to non-homogeneous equations, as an explicit solution can be explicitly constructed in Fourier space, say for u0 and f in L2 (Rd ) Z t Z Z s t b f (s, ξ) exp p(τ, ξ)dτ ds u b0 (ξ) . u b(t, ξ) = exp − p(s, ξ)ds 0
0
0
Thus, from two consecutive “times” tn < tn+1 Z tn+1 Z Z s tn+1 b u b(tn+1 , ξ) = exp − p(s, ξ)ds f (s, ξ) exp p(τ, ξ)dτ ds u b(tn , ξ) . tn
tn
tn
Unlike the homogeneous case, a second unknown must be determined. Rather than considering fb(t, ξ), for the sake of simplicity of the presentation, let us assume that f is t-independent. We then formally get Z tn+1 Z s u Z tn+1 b(tn+1 , ξ) b − log = log f (ξ) + p(s, ξ)ds + log exp p(τ, ξ)dτ ds u b(tn , ξ) tn tn tn 7
for u b(tn , ξ) 6= 0. A possible algorithm can then be established from pn;` k
vbn+1;` 1 1 ` b − log fk = − log k n;` + log(∆tn ) . ∆tn ∆tn vbk
b` A second equation is needed, as there are now 2 unknowns pn;` k and fk . In practice, we can instead solve a 3-equation system. Assuming p(·, ξ) (that is a(·, ξ)) smooth enough: n;` 2 = pn;` pn±1;` k ± ∆tn pt,k + O((∆tn ) ), where k X = e−2iπk·xj (∂t p)jn;` . pn;` t,k j∈Zd n;` b` The unknowns are now: pn;` k , pt,k and fk . The second and third equations which are proposed are then, for vbkn±1;` 6= 0,
pn;` k
vbn+1±1;` 1 1 ` b ± ∆tn pt,k − log fk = − log k n±1;` + log(∆tn±1 ) . ∆tn ∆tn±1 vbk
As proposed above, an average process allows for the construction of {pnk }k,n , {pnt,k }k,n and {fbk }k K 1X pn;` k , K→+∞ K `=1
pnk := lim
K 1X pn;` t,k , K→+∞ K `=1
pnt,k := lim
K 1X fbk` . K→+∞ K `=1
fbk := lim
We then construct approximate fe, and Pe using the exact same interpolation strategy as the homogeneous case, to get an approximate equation: ∂t u(t, x) + Peu(t, x) = fe(x),
(t, x) ∈ (0, T ) × Rd ,
(8)
such that, for k ∈ Kd ⊂ Zd and n ∈ N ⊂ N: pe(tn , ξ k ) = pnk , fe(ξ k ) = fbk . Notice that the procedure described above, still does not require the a priori knowledge of the order the searched equation. An expression of pe can also be obtained, using sparse regression from a feature library, see [7]. 2.3. Fractional equations The methodology which was presented above remains essentially valid for fractional equations, α = (α1 , · · · , αN ) ∈ Qd and order Q ∈ Q∗ [19], assumed well-posed X ∂t u(t, x) + a(α) (t)Dxα u(t, x) = f (t, x), (t, x) ∈ (0, T ) × Rd . (9) αi ∈Q, |α|6Q
The corresponding symbol is a fractional function in ξ, X α p(t, ξ) = a(α) (t) iξ . αi ∈Q, |α|6Q
8
As in the partial differential case, we take the Fourier transform in space, of the (3) we get for two consecutive “times” tn < tn+1 and denoting ∆tn := tn+1 − tn 1 ∆tn
Z
tn+1
tn
u 1 b(tn+1 , ξ) p(s, ξ)ds = − log . ∆tn u b(tn , ξ)
We denote by t 7→ A(α) (t) the antiderivative of t 7→ a(α) (t) X u b(tn+1 , ξ) = exp − b(tn , ξ) . (iξ)α A(α) (tn+1 ) − A(α) (tn ) u αi ∈Q, |α|6Q
At the discrete level, we proceed as for the partial differential case. We then expect that {pnk }k∈Kd will provide an approximation of the function {p(tn , ξ k )}k∈Kd . From this sequence, fractional interpolation allows for the construction of an approximate symbol (t, ξ) 7→ pe(t, ξ). −1 e Finally, we define P (t, Dx ) = σξ pe(t, ξ) such that pe(tn , ξ k ) = p(tn , ξ k ) for all n ∈ N ⊂ N and k ∈ Kd ⊂ Zd . From the approximate symbol, we can again reconstruct a differential operator. However, unlike the PDE case, if we do not know the exact structure of the searched fractional equation, we can hardly reconstruct an approximate symbol, as a very large number of configurations is possible (depending on α). 3. Linear equations with non-constant coefficients We now consider more general equations. We assume that the searched equation in d > 1 dimensions, reads: o X
α(i) (x)∂ti u(t, x) +
i=1
X
β (α) (x)Dxα u(t, x) = 0,
(t, x) ∈ (0, T ) × Rd .
|α|6N
The coefficients {α(`) 16`6o , and {β (α) 16|α|6N are assumed smooth. This time, it is no more possible to directly apply the Fourier transform, as the coefficients are space-dependent, and the symbol σx (P ) = p(t, ξ) reads X p(x, ξ) = β (α) (x)(iξ)α . |α|6N
3.1. First order equations Data {vjn }j are collected at different times steps tn−1 < tn < tn+1 . We first assume that the searched equation is a first order equation of the form: ∂t u(t, x) +
d X
β (i) (x)∂xi u(t, x) = 0,
i=1
9
(t, x) ∈ (0, T ) × Rd
(10)
where x = (x1 , · · · , xd ). The unknown coefficients, we intend to determine are the functions {β (i) }16i6d . In this goal, we propose the following formal discretization to (10) un+1 j
−
Z
unj
tn+1
+ tn
d X
(i)
βj [[∂xi u(t)]]j dt = 0,
(11)
i=1
where [[∂xi un ]] = [[∂xi un ]]j j is an approximation of the formal pseudodifferential definition of first order differential operators [[∂xi un ]] ≈ Fx−1 Fx (∂xi u) = Fx−1 iξFx (u) (i)
which is defined in Appendix. We intend to determine the sequences βj , for all 1 6 i 6 d, from (11) Z
d X
tn+1
− tn
(i)
βj [[∂xi u(t)]]j dt = un+1 − unj . j
(12)
i=1
The idea consists in mimicking (12) using the data {vjn;` }j approximating {unj }j , and assumed of the form vjn;` := unj 1 + εn;` , where εjn;` is a random perturbation, see Subsection 1.1. For j any j: 1. We numerically compute [[∂xi v n ]]j . 2. We assume that {vjn;` }j satisfies ∆tn
d X
(i);`
βj
[[∂xi v n;` ]]j = vjn+1;` − vjn;` .
(13)
i=1 (1);`
(d);`
For each j ∈ Zd , we have d unknowns βj , · · · βj , we then need d equations. As the coefficients are only x−dependent, we then consider d linear equations for 1 6 k 6 d, ∆tn+k−1
d X
(i);`
βj
[[∂xi v n+k−1;` ]]j = vjn+k;` − vjn+k−1;`
(14)
i=1 n;` ` which are equivalent to An;` j X j = bj where n+k−1;` An;` ]]j j = ∆tn+k−1 [[∂xi v
16k,i6d
bjn;` = vjn+k;` − vjn+k−1;`
∈ Rd×d ,
16k6d
.
The unknown vector is given by (i);`
X `j = βj
16i6d
.
(15)
The computational complexity for solving this system, depends on the dimension d. Although in principle this can be computed once for each j ∈ Zd , for a given n ∈ N, the 10
accuracy can be increased by solving the system for several realizations, and to statistically (i) get a better estimation of βj . Finally, we propose the following average process: 1 ` Xj . K→+∞ K
X j = lim
This is justified by the fact that, by linearity of the equation, we get K 1X lim bjn;` = (1 + ε)bnj K→+∞ K `=1 n+k where bn;` − un+k−1 j = uj j
16k6d
. Similarly
K 1X Ajn;` = (1 + ε)Anj , K→+∞ K `=1
lim
where Anj = ∆tn+k−1 [[∂xi un+k−1 ]]j
16k,i6d
∈ Cd×d .
This comes from the fact that limK→+∞
1 PK 1 PK n;` ∂x v n;` ]]j `=1 [[∂xi v ]]j = [[limK→+∞ K K `=1 i = (1 + ε)[[∂xi un ]]j .
This justifies that, the more realizations, the better the accuracy in the reconstruction of the equation coefficients. 3.2. Finite difference approach The exact same idea can be implemented using a finite difference approximation. That is rather than using the DFT, we can approximate ∂xi u at xj using the following approximation, at (tn , xj ) and ζ` . For r ∈ N, we denote τi±r j := (j1 , · · · , ji−1 , ji ± r, ji+1 , · · · , jd ) . The first order spatial derivative at (tn , xj ) can be approximated by n;` vτn;` 1 j − v −1 τ j i
i
∆xji +1 + ∆xji
.
Then, we consider n;` d vτn;` 1 − v −1 vjn+1;` − vjn;` X τi j (i) i j + βτ −1 j = 0. i ∆tn ∆xji +1 + ∆xji i=1
11
(16)
As above, in order to compute X `j (15) knowing bn;` j for given j, we have to solve a linear n;` n;` ` system of d equations Aj X j = bj , where this time An;` j
= ∆tn+k−1
n;` vτn;` 1 j − v −1 τ j i
i
∆xji +1 + ∆xji
16k,i6d
∈ Cd×d .
Example 3.1. For instance for first order equation in 3d ut +
3 X
β (i) (x)∂xi u = 0 .
i=1
We use the following equations for k = 0, 1, 2 (1);` βj − vjn+k;` 2 vjn+k+1;` − vjn+k;` ∆x1 ∆x2 ∆x3 = ∆tn+k ∆x2 ∆x3 vjn+k;` − + 1 1 (2);` +∆tn+k ∆x1 ∆x3 vjn+k − vjn+k;` βj + − 2 2 (3);` +∆tn+k ∆x1 ∆x2 vjn+k;` − vjn+k;` βj . + − 3
3
(1);`
For each j ∈ Z3 , this 3-equation system has to be solved, to estimate βj
(2);`
, βj
(3);`
and βj
.
3.3. N th order linear equations The generalization to N th order linear equations with non-constant coefficients is straightforward, but naturally requires in principle the solution to larger linear systems. In d > 1 dimensions, we assume that the searched equation reads o X i=1
α(i) (x)∂ti u(t, x) +
X
β (α) (x)Dxα u(t, x) = 0,
(t, x) ∈ (0, T ) × Rd .
|α|6N
In the case of homogeneous equation, the total number of unknown functions is σd (N ) + o, where σd (N ) the number of possible distinct multi-indices α such that |α| 6 N in d dimensions. This is also the total number of equations needed to compute the σd (N ) terms (α) (i) βj and the additional o equations for determining αj , with i ∈ {1, · · · , o}. This is done thanks to approximations. Formally, for v n;` given, we consider the following approximation, for each j ∈ Zd o X
X (α);` (i);` βj [[∂ (α) v n;` ]]j = 0 αj Π vjn;` , · · · , vjn+i;` ; ∆tn , · · · , ∆tn+i−1 +
i=1
|α|6N
where Π vjn;` , · · · , vjn+i;` is a finite difference approximation to ∂ti u(tn+i , xj ). As σd (N ) + o equations are needed, we consider, for k ∈ {σd (N ) + o} o X
X (α;`) (i);` βj [[∂ (α) v n+k ]]j = 0 . αj Π vjn+k;` , · · · , vjn+i+k;` ; ∆tn , · · · , ∆tn+i−1 j +
i=1
|α|6N
12
n;` n;` ` ∈ This can again be rewritten as a linear system of the form An;` j X j = bj , where Aj n;` R(σd (N )+o)×(σd (N )+o) , and bj ∈ Rσd (N )+o read (i);`
(α);`
X `j = αj , βj
16i6o,16k6σd
n;` n;` n+o;` , b = − Π v , · · · , v j j j (N ) 16i6o,16k6σ
d (N )
As proposed above, we will eventually define X j = limK→+∞ larger K, the more accurate the reconstruction.
PK
`=1
X `j /K. In practice the
Discussion on the computational complexity. We now discuss the computational complexity of the proposed methodology in the framework of N th order linear equations, presented at the beginning of this subsection. We assume that M data (approximate solution at given location and time {vjn;` }j;n with 1 6 ` 6 K) are given for each realization ` 6 K. In order to reconstruct the unknown coefficients of the PDE, the following operations are required. • For each multi-index α, the computation of [[∂ α ]] requires O(M log M ) operations, that is a total of O(σd (N )M log M ) operations are necessary for all the distinct multi-indices such that |α| 6 N . • For each 1 6 i 6 o, the construction of Π vjn;` , · · · , vjn+i;` ; ∆tn , · · · , ∆tn+i−1 requires O(i) operations, that is a total O(o2 M ) operations are required. 2 • Assembling each matrix of the M matrices Ajn;` requires O σd (N ) + o operations, 2 that is a total O M σd (N ) + o operations. • for each n, j and `, the σd (N ) + o equation-linear system Ajn;` X `j = bn;` must be solved 3 j n;` to determine X j . This represents a total of O M σd (N ) + o operations. The overall complexity of the algorithm for constructing the coefficients of the PDE is 3 O KM σd (N ) log M + (σd (N ) + o) , assuming d > 3, M large and for any K realizations, as 1 6 ` 6 K. Finally traditional polynomial interpolations, typically requiring O(M 2 ) for each coefficients 1 6 i 6 o and α, can then be implemented to reconstruct the PDE coefficients, that is a O(M 2 (σd (N ) + o)). Notice, that for a very large collection of data it becomes necessary i) to use typical machinelearning techniques, such as artificial neural networks, for solving inverse problems involving the numerical approximation to differential equations [5, 17, 11]. Artificial neural networks can then be used in order to reconstruct approximate symbols of the unknown PDE. This is an approach that we are currently investigating starting with boundary value problems. Example 3.2. Assume that we search for a 2nd order system of the form (t, x) ∈ (0, T )×Rd ∂t2 u(t, x) + α(x)∂t u(t, x) +
d X
β (i) (x)∂xi (t, x) +
i=1
d X d X i=1 j=i
13
γ (i,j) (x)∂xi xj u(t, x) = 0 .
(17)
The proposed formal discretization is (i);` [[∂xi v n;` ]]j i=1 βj
Pd
+
vjn+1;` − 2vjn;` + vjn−1;` = − ∆tn ∆tn−1 n+1;` vj − vjn;` . −αj ∆tn
(i,j);` [[∂xi xj v n;` ]]j j=i γj
Pd Pd i=1
(1);`
(d);`
(18)
(i,j);`
For each j ∈ Zd , we have d unknowns βj , · · · βj and d(d + 1)/2 unknowns γj , we then need σd (2) = d(d + 3)/2 equations. An addition equation is needed to determine αj . As the coefficients are only x−dependent, we proceed as follows, from the knowledge v n−1;` , v n;` , v n+1;` 1. We numerically compute [[∂xi v n;` ]]j , [[∂xi xl v n;` ]]j . 2. We assume that v n;` := {vjn;` }j satisfies vjn+k;` − 2vjn+k−1;` + vjn+k−2;` = ∆tn+k−2 αj` (vjn+k` − vjn+k−1;` ) P (i);` (19) +∆tn+k−2 ∆tn+k−1 di=1 βj [[∂xi v n+k−1;` ]]j Pd Pd (i,j);` [[∂xi xj v n+k−1;` ]]j +∆tn+k−2 ∆tnk −1 i=1 j=i γj We consider 1 6 k 6 d(d + 3)/2 independent linear equations (19) . This system can be n;` ` rewritten in the form An;` j X j = bj , where An;` = ∆tn+k−2 vjn+k;` − vjn+k−1;` , ∆tn+k−2 ∆tn+k−1 [[∂xi v n+k−1;` ]]j , j ∆tn+k−2 ∆tnk −1 [[∂xi xj v n+k−1;` ]]j 16i,k6d(d+3)/2
and n+k;` bn;` − 2vjn+k−1;` + vjn+k−2;` j = vj
16k6d(d+3)/2
(20)
and the unknown vector is given by (1);`
X `j = αj` , βj
(d);`
, · · · , βj
(1,1);`
, γj
(d,d);`
, · · · , γj
∈ Rd(d+3)/2 .
(21)
Remark 3.1. In the case of non-homogeneous systems with source term of the form x 7→ f (x), an additional equation is necessary in order to determine fj` for each j ∈ Zd . 3.4. Quasilinear equations A forthcoming paper will be dedicated to the treatment of time-dependent coefficients and nonlinear equations. However, some techniques developed here can be extended to a class of standard quasilinear PDEs. Assume that the searched equation is of the form (first order in time): ut (t, x) +
d X
β (i) (u)∂xi u(t, x) = 0
i=1
14
(22)
where β (i) are smooth functions. We are given the data v n;` := {vjn;` }j , locations {xj }j , and times {tn }n∈N , and propose the following formal discretization of (22): d vjn+1 − vjn;` X + β (i) (vjn )[[∂xi v n;` ]]j = 0 . ∆tn i=1
In order to reconstruct u 7→ β (i) (u) from vjn;` , [[∂xi v n;` )]]j , we can not use the exact same procedure to determine, as above, as the unknown (1) n;` X n;` (vj ), · · · , β (d) (vjn;` ) j = β is t-dependent. As we have d unknown functions, we need for each at least d equations, for k = 1, · · · , d d vjn+k;` − vjn+k−1;` X + β (i);` (vjn+k−1;` )[[∂xi v n+k−1;` ]]j = 0 . ∆tn+k−1 i=1
It is well-known that nonlinear hyperbolic equations, such as the one which is searched usually have non-differentiable weak solutions, called shock waves [18, 16, 1]. The procedure which is proposed can then be applied for any j ∈ Zd , if β (i) is smooth enough and in regimes avoiding shock-waves. The latter can easily be identified as they correspond to data with spatial discontinuity. In this case: β (i);` (vjn+1;` ) = β (i) (vjn;` ) + β˙ (i);` (vjn;` )(vjn+1;` − vjn;` ) + O (vjn;` − vjn+1;` )2 . As a consequence, for |vjn+1;` − vjn;` | small enough, we need to determine β (i) (vjn;` ) and β˙ (i) (vjn;` ). This can be done by doubling the size of the linear system, compared to the linear case. That is for k, k 0 = 1, · · · , d, we solve the 2d-equation linear system: vjn+k;` − vjn+k−1;` Pd (i);` n+k−1;` (vj )[[∂xi v n+k−1;` ]]j = i=1 β ∆t n+k−1 0 0 vjn+k +1;` − vjn+k ;` Pd 0 0 (i);` n+k −1;` = (vj )[[∂xi v n+k ;` ]]j i=1 β ∆tn+k0 0 0 0 0 +β˙ (i);` (vjn+k −1;` )[[∂xi v n+k ;` ]]j (vjn+k ;` − vjn+k −1;` ) . n;` n;` This system can be equivalently rewritten An;` j X j = bj , where
bn;` = j ` Xj =
vjn+1;` − vjn;` , · · · , vjn+d;` − vjn+d−1;` , vjn+2;` − vjn+1;` , · · · , vjn+d+1;` − vjn+d;` , (23) β (1);` (vjn;` ), · · · , β (d);` (vjn;` ), β˙ (1);` (vjn;` ), · · · , β˙ (d);` (vjn;` ) .
15
4. How to determine the order of the linear equation ? Beyond the construction of the β (α) and α(i) functions, one of the main problems is the evaluation of the overall order of the searched PDE. This is a crucial question, as a wrong order will lead to an inaccurate equation. Assume that the searched equation Eo;N , reads o X
α(i) (x)∂ti u(t, x) +
i=1
X
β (α) (x)Dxα u(t, x) = 0,
(t, x) ∈ (0, T ) × Rd .
|α|6N
In this goal, we consider an equation Eoe;Ne , of the form oe X
α e(i) (x)∂ti u(t, x) +
i=1
X
βe(α) (x)Dxα u(t, x) = 0,
(t, x) ∈ (0, T ) × Rd
e |α|6N
e , o 6= oe, and as a consequence kβ (α) − βe(α) k∞ and kα(i) − α where in principle N 6= N e(i) k∞ e = N, N e < N, N e > N , with oe = o, are expected to be large. Nine situations can occur N oe < o, oe > o. 4.1. Iterative approach In order to reach convergence of the overall algorithm, a natural approach is to proceed e . The algorithm looks as follows for given data v n;` , · · · , v n+k;` along by induction in oe and N with {xj }j and {tn+k }k>0 , assuming that the maximal order in t (resp. x) is oemax (resp. emax ). Start with oe = 0. N e = 1 and oe ← oe + 1. Step 0. (Re)set N n;` ` Step 1. Solve linear system on β (α) , and α(i) : An;` j X j = bj . Step 2. Analysis of the results. e ←N e + 1. • If no solution to linear system: N e o and Nmax > N . That is the searched equation is of the form o X i=1
α
(i)
(x)∂ti u
+
oX max i=o+1
α(i) (x)∂ti u +
X
β (α) (x)Dxα u +
|α|6N
X N o and |α| > N would be found null or inconsistent, and other words the matrix Anj must have a null determinant. In this case, the order of the searched equation must be reduced. This is a crucial criterion in order to determine the order of the searched equation. Alternatively the corresponding coefficients can be found very small suggesting that the contribution of the corresponding differential operators can be neglected. A recursive algorithm can again be established to iteratively, reducing the overall equation order, discriminate the negligible operators. 5. Numerical experiments We propose in this section, several numerical experiments to validate the methodology derived above. 5.1. Symbolic method In this first test, we plan to reconstruct the following equation, using a symbolic approach. ∂t u + a(1) ux + a(2) ∂xx u + a(3) ∂xxx u + a(4) ∂xxxx u + a(5) ∂xxxxx u = a0
(24)
where the coefficients are given by a(0) = −10−1 , a(1) = 1, a(2) = 10−1 , a(3) = 10−2 , a(4) = 5 × 10−3 , a(5) = −10−4 . From a initial data, u0 (x) = exp(−x2 ) cos(10x), we construct an approximate solution {unj }j computed in a domain [−5, 5], on a Nx = 8001-point grid with ∆x = 1/800 and ∆t = 10−3 . The final iteration is Nt = 400. We impose a stochastic process on the approximate solution, with an uniform law U(0, ε∞ ), with ε∞ = 10−8 , where vjn;` = unj (1 + εn;` j ). The perturbed data at final time are reported in Fig. 2 in semilogscale. Case 1. Perturbation is null/cancelled. We first assume that the perturbation is null. This case is also equivalent to perturbations which would be averaged, as discussed in Remark 2.1. In the latter situation, we indeed have vbkn+1 /b vkn = u bn+1 /b unk . We construct the DFT {b vkNt −1 }k , k and {b vkNt }k from {vjNt −1 }j and {vjNt }j . We then construct the sequence, for vbkNt −1 6= 0 , t {pN k }k t pN k
vbNt 1 := − log Nkt −1 . ∆t vbk
We compare in Fig. 13, the real and imaginary partsP of the exact and reconstructed sym5 (`) ` bols to (24). That is we search for ξk 7→ p(ξk ) = e(ξk ). We `=0 a (iξk ) and ξk 7→ p obviously get an excellent agreement. However, we still need to reconstruct an approximate (continuous) from these data. In this goal, we assume that the searched equation is Pomax symbol (l) l ∂t u + l=0 α ∂x u = 0, with omax unknown, and we implement a polynomial interpolation in order determine the unknown coefficients of the PDE. That is, we search pe(t, ξ) such that for all l ∈ {1, · · · omax + 1}, pe(tn , ξk ) = p(tn , ξk ), 17
10 -4
Perturbed data
10 -6
10 -8
10 -10
10 -12 -5
0
5
Figure 2: Perturbed data at final time in semilogscale.
3
10
Real part of the reconstructed symbol Real part of the exact symbol Imaginary part of the reconstructed symbol Imaginary part of the exact symbol
2
10
1
10
0
10
−1
10
0
2
4
6
8
10
12
14
Figure 3: Real and imaginary parts of the exact and reconstructed symbols.
where ξk = k∆ξ, and ∆ξ = π/5, with k ∈ Z. We assume the maximal order is omax = 6. Searching for the coefficients is equivalent to solve a Vandermonde system. We picked 1 different set of omax + 1 = 7 constraints. According to Table 1, we get estimates of the order of the searched equation and of its coefficients. More specifically, the inconsistency of the 18
α(0) α(1) α(2) −1.000081850888496 × 10−1 9.999912470046054 × 10−1 1.000042171075373 × 10−2 α(3) α(4) α(5) 1.000097592645466 × 10−2 5.000141820071519 × 10−3 −1.000096443526068 × 10−4 α(6) 4.257054797261438 × 10−10 Table 1: Numerical values α(0) , · · · , α(6) , where ε is null.
computed α(6) ’s suggest that o = 5. Next, the consistency of the other computed coefficients allow for an accurate estimate of the PDE coefficients. Case 2. Space dependent random perturbation. In this case, we assume that the random process is t-independent, that is vjn;` = unj (1 + ε`j ), with ` denoting the realization index of (`) 1 Nt ;` Nt ;` Nt −1;` the stochastic process. We set pn;` = − log v b /b v , where we have denoted v b k k k k ∆t the DFT of the perturbed data at `th realization. The rest of the algorithm is as in the nonPomax (l);` l ` (`) perturbed case. We assume that ξk → pe (ξk ) is of the form pe (ξk ) = l=1 α (iξk ) where omax = 6, and solve a Vandermonde system to get approximates values of the coefficients. Next, it is necessary to implement an average process in order to get an estimate of the PDE coefficients. We denote by {α(l);` }16l6omax the computed coefficients for each sample `. We then compute, (for K large enough, in practice). K 1X α(l);` . K→+∞ K `=1
α(l) = lim
We report in Fig. 4 (Left) the `2 -norm of the error in logscale, between the exact and approximate coefficients as a function of `, that is {`, log E(`) } E(`) :=
max oX
|α(l);` − a(l) |2
1/2
.
l=1
We also report in Tables 2, the found coefficients ` = 1 (first line), and the converged coefficients (second line). Case 3. Space and time dependent random perturbation. In this case, we assume that the −9 random process is such that vjn;` = unj (1 + εn;` j ), with εmax = 10 . We proceed as above, performing the algorithms several times. We report in Fig. 4 (Right) the `2 -norm of the error in logscale, between the exact and approximate coefficients at stochastic realization `, that is {`, log E(`) }. 5.2. One-dimensional test. Linear equation. In this subsection, the searched equation is ∂t u + c(x)∂x u + ν(x)∂xx u = 0, (t, x) ∈ [0, T ] × [−1, 1] 19
α(0) α(1) α(2) −8.496905353740625 × 10−2 9.914638147888091 × 10−1 9.302849563998560 × 10−2 −1.001897636979343 × 10−1 9.997928029182725 × 10−1 1.000833219501233 × 10−1 α(3) α(4) α(5) −2 −3 1.1290669346466807 × 10 4.787951773996767 × 10 −1.075221456936866 × 10−4 1.001814167944528 × 10−2 5.002455815685041 × 10−3 −1.001455983589550 × 10−4 α(6) 4.648585626177240 × 10−7 6.266263919080856 × 10−9 Table 2: Numerical values α(0);1 , · · · , α(6);1 and α(0) , · · · , α(6) : ε(x, ζ) with εmax = 10−8 .
α(0) α(1) α(2) −1 −1 −3.887008083111291 × 10 6.243784097545035 × 10 1.952699801460228 × 10−1 −9.031650266918659 × 10−2 1.004654230531507 9.554460918355122 × 10−2 α(3) α(4) α(5) 4.678156614586105 × 10−2 7.108540944477762 × 10−3 −4.248146685454826 × 10−4 9.404316598788863 × 10−3 4.863543299705904 × 10−3 −9.337375897290761 × 10−5 (6) α1 9.29794446838190 × 10−6 3.547161290147765 × 10−7 Table 3: Numerical values α(0);1 , · · · , α(6);1 and α(0) , · · · , α(6) : ε(t, x, ζ) with εmax = 10−9 .
20
2
2
l −norm error
l −norm error −2
Error in logscale
Error in logscale
10
−3
10
−1
10
−2
100
200
300
400
500 600 Iteration
700
800
900
10
1000
20
40
60
80 Iteration
100
120
140
Figure 4: `2 -norm error in logscale {`, E(`)} (Left) ε(x, ζ) with εmax = 10−8 . (Right) ε(t, x, ζ) with εmax = 10−9 .
and
c(x) = 10 exp − 10x2 + 0.1 × 1 + 0.2 sin(20x) ν(x) = 10−1 sin2 (8πx) × exp(−10x2 ) .
The initial data is given by u0 (x) = exp(−20x2 ). We accurately solve this equation, which provides the sequences {unj }|j|6J for several n 6 Nt and Nx = 2001, ∆t = ∆x/ supx∈[−1,1] |c(x)|. We implement the method, assuming that we do not know the order in space of the equation, of the searched equation, that is α(x)∂tt u + ∂t u + β (1) (x)∂x u + β (2) (x)∂xx u + β (3) (x)∂xxx u = 0 . Case 1. We assume that for any j, εjn;` = 0, that is there is no perturbation on the data, in this first test. For each |j| 6 J, we propose here a FDS approach, rather than a pseudospectral (2) (2) (1) Nt t one: we search for X j = (αj , βj , βj , βj ) such that AN j X j = bj where t bN = (vjNt − vjNt −1 )/∆x, (vjNt −1 − vjNt −2 )/∆x, (vjNt −2 − vjNt −3 )/∆x, (vjNt −3 − vjNt −4 )/∆x j and ANt ∈ R4×4 t AN j;m =
Nt −m Nt −m − (vjNt +1−m − 2vjNt −m + vjNt −m−1 )/∆t2 , −(vj+1 − vj−1 )/2∆x, Nt −m Nt −m Nt −m Nt −m Nt −m Nt −m Nt −m 2 (vj+1 − 2vj + vj−1 )/∆x − (vj+2 − 2vj+1 + 2vj−1 − vj−2 )/2∆x3
We respectively report the data {vjNt }|j|6J , at final time T , {αj }|j|6J and {cj }|j|6J and in Fig. 5 and in Fig. 6 {β (2) }|j|6J , {νj }|j|6J , and {β (3) }|j|6J , {ηj }|j|6J . A very good agreement for the velocity and diffusion coefficients is again observed. We also obtain a dispersion coefficients close to zero, as expected. More generally, this test suggests that it may be appropriate to directly search for an high order equation, and discriminate afterwards the negligible coefficients. 21
Velocity, β(1)
Double derivative in t coefficient, α
Data at final time T 1
0.01
20
0.9
0.008
18
0.8
0.006
16
0.7
0.004
14
0.6
0.002
12
0.5
0
0.4
−0.002
0.3
−0.004
0.2
−0.006
0.1
−0.008
0
−0.01
Reconstructed velocity Exact velocity
10 8 6 4 2 0
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
Figure 5: (Left) Experimental data (Middle) Coefficients {αj }|j|6J (Right) Reconstructed and exact veloci(1)
ties ({βj }|j|6J , {cj }|j|6J , ). Diffusion coefficient, β(2)
Dispersion coefficient, β(3)
−3
5 Reconstructed diffusion Exact diffusion
x 10
Reconstructed dispersion Exact dispersion
4
0.8
3 2
0.6
1 0.4
0 −1
0.2
−2 −3
0
−4 −0.2
−0.6
−0.4
−0.2
0
0.2
0.4
−5
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
(2)
Figure 6: (Left) Reconstructed and exact diffusion ({βj }|j|6J , {νj }|j|6J ) (Right) Reconstructed dispersion (3)
({βj }|j|6J , {ηj }|j|6J ).
The reconstruction was performed using the data at iterations Nt , · · · Nt − 4, but could have been as well done using earlier iterations, in order to validate the results. Some fluctuations/singularities observed in the reconstruction solutions. These fluctuations are due to the numerical solver of the induced linear systems. For very specific values j that matrix could A;`j could be ill-conditioned. A balance should then found between efficiency of the linear system solver, and the solution accuracy. In practice, we have used the basic GMRES solver from matlab. Case 2. We assume here that the perturbation is of the form ε(x, ζ) (resp. ε(t, x, ζ)) where ζ is the random variable, where ε∞ := kεk∞ = 10−7 (resp. ε∞ := kεk∞ = 10−9 ). We report the reconstructed velocity and diffusion coefficients in Fig. 7. We report in Fig. 8, the `2 -norm, as a function of `, for time independent (resp. dependent) perturbation (Left) (resp. (Right)). s X X |αj` |2 + |β (1);` − cj |2 + |β (2);` − νj |2 + |β (3);` |2 . E(`) = j
j
22
Again a very good agreement is obtained. Velocity, β(1)
Dispersion coefficient, β(3)
−3
5
20 Reconstructed velocity Exact velocity
18
x 10
Reconstructed dispersion Exact dispersion
4
16
3
14
2
12
1
10
0
8
−1
6
−2
4 −3 2 −4 0 −0.6
−0.4
−0.2
0
0.2
0.4
−5
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
(1)
Figure 7: (Left) Reconstructed and exact velocities ({βj }|j|6J , {cj }|j|6J ). (Middle) Reconstructed and (3)
exact dispersion ({βj }|j|6J , {ηj }|j|6J ).
l2−norm error
l2−norm error
1
10 0
l2−norm error
l2−norm error
10
0
50
100
150
200
250
10
Iteration
20
30
40 Iteration
50
60
Figure 8: `2 -norm error in logscale {`, E(`)}. (Left) ε(x, ζ) with εmax = 10−7 . (Right) ε(t, x, ζ) with εmax = 10−9 .
Case 3. In this test, we assume that the searched equation is the following ∂t u + c(x)∂x + ν(x)∂xx u + η(x)∂xxx u = 0, (t, x) ∈ [0, T ] × [−1, 1] with c(x) = exp − 10x2 + 10−1 × 1 + 0.2 sin(20x) , ν(x) = 5 × 10−1 sin2 (2πx) exp(−10x2 ), η(x) = 10−3 exp(−50x2 ) sin2 (8x) . The initial data is given by u0 (x) = exp(−20x2 )| cos(x)|. Numerically the data are computed Nx = 1001-point grid, ∆t = ∆x/ supx∈[−1,1] |c(x)|. We do not impose any perturbation in 23
this test. The convergence is then reached in 1 iteration. We respectively report Fig. 9 (2) (3) {βj }|j|6J , {νj }|j|6J , and {βj }|j|6J , {ηj }|j|6J . We notice again a very good agreement for the velocity and diffusion coefficients. We also obtain a dispersion coefficients close to zero, as expected. This suggests again that it may be appropriate to directly search for an high order equation, and to discriminate afterwards the negligible operators. Diffusion coefficient, β(2)
Dispersion coefficient, β(3)
−4
0.07
15
x 10
Reconstructed diffusion Exact diffusion
0.06
Reconstructed dispersion Exact dispersion
0.05
10
0.04 0.03 5 0.02 0.01 0
0 −0.01 −0.02
−0.6
−0.4
−0.2
0
0.2
0.4
−5
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
(2)
Figure 9: (Left) Reconstructed and exact diffusion ({βj }|j|6J , {νj }|j|6J ) (Right) Reconstructed dispersion (3)
({βj }|j|6J , {ηj }|j|6J ).
5.3. First order quasilinear equation In this test we assume that the searched equation is quasilinear ∂t u + a(u)∂x u + ν(u)∂xx u = 0, (t, x) ∈ [0, T ] × [−a, a] where a(u) = 0.5u2 and ν(u) = u2 | sin(u)|. The initial data, the data are constructed from is given by u0 (x) = exp(−5x2 ). Numerically these data are computed on a Nx = 3001-point grid, ∆t = ∆x/ supx∈[−1,1] |a(u(x))|. We search the equation in the form ∂t u + β (1) (u)∂x u + β (2) (u)∂xx u = 0, (t, x) ∈ [0, T ] × [−a, a] . In order to reconstruct the unknown function, we search for (1) Nt t XN (vj ), β (2) (vjNt ), β˙ (1) (vjNt ), β˙ (2) (vjNt )) , j = (β Nt Nt t solving a 4 equations systems AN j X j = bj where t bN = j
(vjNt − vjNt −1 )/∆tNt , (vjNt −1 − vjNt −2 )/∆tNt −1 , Nt −1 Nt −2 (vjNt −2 − vjNt −3 )/∆tNt −2 , (vj+1 − vj+1 )/∆tNt −1
24
and ANt ∈ R4×4 t AN j;1 =
t AN j;2 =
Nt −1 Nt −1 Nt −1 Nt −1 (vj+1 − vj−1 )/2∆x, (vj+1 − 2vjNt −1 + vj−1 ))/∆x2 Nt −1 Nt −1 Nt −1 (vjNt −1 − vjNt −2 )(vjNt −1 − vj−1 )/∆x, (vjNt −1 − vjNt −2 )(vj+1 − 2vjN t−1 + vj−1 )/∆x2
Nt −2 Nt −2 Nt −2 Nt −2 (vj+1 − vj−1 )/2∆x, (vj+1 − 2vjNt −2 + vj−1 ))/∆x, 0, 0
t AN j;3 =
Nt −3 Nt −3 Nt −3 Nt −3 (vj+1 − vj−1 )/2∆x, (vj+1 − 2vjNt −3 + vj−1 ))/∆x2 Nt −3 Nt −1 Nt −3 Nt −3 −(vjNt −3 − vjNt −4 )(vj+1 − vj−1 )/2∆x, (vjNt −3 − vjNt −4 )(vj+1 − 2vjN t−3 + vj−1 )/∆x2
t AN j;4 =
Nt −2 Nt −2 Nt −2 (vj+2 − vjNt −2 )/2∆x, (vj+2 − 2vj+1 + vjNt −2 ))/∆x2 Nt −2 Nt −2 Nt −2 Nt −2 Nt −2 Nt −2 N t−2 −(vj+1 − vj−1 )(vj+2 − vjNt −2 )/4∆x, (vj+1 − vj−1 )(vj+2 − 2vj+1 + vjNt −2 )/∆x2
In Fig. 10, we respectively report the solution at final time T , β (1) (vjNt ) |j|6J approxi (2) Nt t t mating {a(uN (vj ) |j|6J approximating ν(uN j )}|j|6J and β j )|j|6J . We notice again a very good agreement for the quasilinear velocity and diffusion. This example is however, from a limited interest as the exact order of the equation is known. Nonlinear diffusion, β(2)(u)
Velocity, β(1)(u)
Data at final time T 1 0.9
Reconstructed nonlinear velocity Exact nonlinear velocity
0.6
Reconstructed nonlinear diffusion Exact nonlinear diffusion
0.45 0.4
0.8 0.5
0.35
0.7 0.6
0.4
0.5
0.3
0.3 0.25 0.2
0.4
0.2
0.15
0.3
0.1
0.1 0.2
0.05 0
0.1 0 −0.6
−0.4
−0.2
0
0.2
0.4
−0.1
0.6
0 −0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.05
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
(Left) Experimental data (Middle) Reconstructed and exact velocities ( β (1) (vjNt ) |j|6J , (2) Nt t t {a(uN (vj ) |j|6J , ν(uN j )}|j|6J ) (Right) Reconstructed and exact diffusion ( β j )|j|6J ). Figure 10:
5.4. Fractional equation in 2-d We assume that the searched equation is a first order fractional equation of the form ∂t u +
4 X
l=1 (10) 3/4 1/4 +a ∂x ∂y u
a(l) ∂xl/4 u + a(l+4) ∂yl/4 u + a(9) ∂x1/2 ∂y1/2 u
(25)
+ a(11) ∂x1/4 ∂y3/4 u + a(12) ∂x1/2 ∂y1/2 u = 0 .
(26)
The given data corresponds to the solution to (25), where a(1) = 0.25, a(2) = −0.1, a(3) = 0.5, a(4) = 0.1, a(5) = −0.25, a(6) = 0.2, a(7) = 0.1, a(8) = 0.04, a(9) = 0.0, a(10) = 0.1, a11 = 0.05, a(12) = 0.075 25
(27)
and the initial data u0 (x, y) = exp − (|x|2 + |y|2 )/20 on [−20, 20]2 . The given data are approximations of the sequence {u(tn , xj )}j , with j = (k, l) ∈ Z2 and x = (x, y), where u is solution to the unknown Equation (25). In this benchmark, there are preliminarily numerically computed from (25) on a Nx × Ny -point grid, with Nx = Ny = 401 and ∆t = 10−4 . The data are assumed to be perturbed by a space dependent stochastic process vjn;` = unj (1 + ε`j ) where kεk∞ 6 10−6 . We construct the Fourier transform of {vjNt ;` }j and {vjNt −1;` }j , that is {b vkNt ;` }k and {b vkNt −1 }k . The objective is to estimate the symbol p of (25), P4 (l) l/4 + a(l+4) (iξy )l/4 − a(9) i(ξx ξy )1/2 p(t, ξx , ξy ) = l=1 a (iξx ) +a(10) (iξy )1/4 (iξx )3/4 + a(11) (iξx (i)1/4 ξy )3/4 + a(12) i(ξx ξy )1/2 . t That is, we search for pe(tNt , ·) such that for any k: pe(tNt , ξ k ) = pN k , with
t ;` peN k
vbNt ;` 1 = − log Nkt −1;` , ∆t vbk
where vbkNt −1;` 6= 0. The approximate symbol then reads pe(t, ξx , ξy ) =
P4
(iξx )l/4 + α(l+4) (iξy )l/4 − α(9) i(ξx ξy )1/2 iξy (iξx )1/2 + α(11) iξx (iξy )1/2 − α(12) ξx ξy
l=1 α (10)
+α
(l)
such that pe(tNt , ξ k ) = p(tNt , ξ k ). This leads to a 12-equation system, which can be solved for 12 different values of ξ k of `. We report in Tables 4, the computed values of the {α(l) }16l612 , as well as the relative error (last line) compared to the exact coefficients in (25) and (27). We report in Fig. 11 the `2 -norm of the error in logscale, between the exact and approximate coefficients at stochastic iteration `, that is {`, log E(`) }, where E(`) :=
12 X
|α
(l);`
(l) 2
−a |
1/2
.
l=1
A good agreement is again obtained for a small amplitude perturbation. In particular, the simulations suggest that α(0) should be taken null. 5.5. Linear equation with non-constant coefficients In this second two-dimensional test, the searched equation is the following ∂tt u(t, x, y) + a(t) (x, y)∂t u(t, x, y) = b(x) (x, y)∂x u(t, x, y) + b(xx) (x, y)∂xx u(t, x, y) +b(xy) (x, y)∂xy u(t, x, y) + a(0) (x, y)u(t, x, y) .
26
2
l −norm error
−1
Error in logscale
10
10
20
30
40
50 Iteration
60
70
80
90
Figure 11: `2 -norm error in logscale {`, E(`)} (Left) ε(x, ζ) with εmax = 10−8 .
α(1) 2.545853246443962 × 10−1 α(4) 9.532077960495579 × 10−2 α(7) 1.001893005064210 × 10−1 α(10) 9.559526009066791 × 10−2
α(2) α(3) −1.026353837787546 × 10−1 5.102543087095289 × 10−1 α(5) α(6) −1 −2.543654731841340 × 10 2.064762283831498 × 10−1 α(8) α(9) 3.465727520278510 × 10−2 1.432068478385523 × 10−8 α(11) α(12) 4.139113603023897 × 10−2 7.337416044103000 × 10−2
Table 4: Numerical values α(0);1 , · · · , α(6);1 and α(0) , · · · , α(6) : ε(t, x, ζ) with εmax = 10−9 .
where the coefficients are given by a(0) (x, y) = exp − |x|/2 − 3|y|/4 , a(t) (x, y) = exp − |x|2 /4 − |y|2 /2 1 b(x) (x, y) = exp − |x|2 /10 − |y|2 /10 cos(2x) sin(2y), b(xx) (x, y) = exp − 2|x|2 − |y|2 /2 (28) 2 b(xy) (x, y) = exp − (|x|2 + |y|2 )/10 . We assume that the searched equation is a linear wave equation of the form ∂tt u(t, x, y) + α(t) ∂t u(t, x, y) = β (x) (x, y)∂x u(t, x, y) + β (y) (x, y)∂y u(t, x, y) +β (xx) (x, y)∂xx u(t, x, y) + β (yy) (x, y)∂yy u(t, x, y) +β (xy) (x, y)∂xy u(t, x, y) + α(0) (x, y)u(t, x, y) . We implement the method described in Section 3. The data {vjn }j are given with the corresponding positions, for (at least) 7 consecutive times. In practice, these data were obtained numerically on a finite domain [−5, 5]2 , at final time T = 1.75, with a initial data u0 (x, y) exp(−|x|2 − |y|2 ). The time step is ∆t = 1.75 × 10−2 , and the computation 27
Figure 12: Solution at final time.
was performed on a 101 × 101-point grid. From {v n;` }, with n ∈ {Nt − 11, · · · , Nt }, and Nt = 100, we compute: • [[∂x v n;` ]], [[∂y v n;` ]], [[∂xx v n;` ]], [[∂yy v n;` ]], [[∂xy v n;` ]]. (0);`
(t);`
(x);`
(y);`
(xx);`
(yy);`
• For any j, we have to determine 7 coefficients αj , αj , βj , βj , βj , βj from the equations (n-dependent) (t);` (x);` (y);` 0 = ∆tαj vjn−1;` + vjn−2;` + ∆t2 βj [[∂x v n−1;` ]]j + βj [[∂y v n−1;` ]]j (xx);` (yy);` (xy);` n−1;` n−1;` n−1;` +βj [[∂xx v ]]j + βj [[∂yy v ]]j + βj [[∂xy v ]]j
(xy);`
, βj
+α(0) vjn−1;` − vjn;` − 2vjn−1;`;` + vjn−2;` . As for any j, we then need 7 equations, involving {v Nt −11;` }, · · · , {v Nt ;` }. We then solve a 7×7 : Ajn;` X `j = bjn;` , where 7 × 7 system, with An;` j ∈ C X `j = t ;` bN = j
(0);` (t);` (x);` (y);` (xx);` (yy);` (xy);` αj , αj , βj , βj , βj , βj , βj vjNt −6;` − 2vjNt −7;` + vjNt −8;` , · · · , vjNt ;` − 2vjNt −1;` + vjNt −2;` .
Again, more data, would allow to refine the computed data. We represent in Fig. 13 the reconstructed coefficients, when the perturbation ε is null, or cancelled following the methodology developed in Remark 2.1. The obtained coefficients are consistent with the exact coefficients (28). We also report the reconstructed coefficients, in the case of a perturbative spatial random 28
,
process of the data. That is: vjn;` = u(tn , xj )(1 + ε`j ) with supj |εj | 6 10−4 , see Fig. 14. In particular, the numerical computations suggest, as expected, that β (y) , and β (yy) should be null functions. 6. Conclusion In this paper, we have presented a methodology for constructing linear evolution PDE from data: i) experimental values of the solution of the unknown equation, along with ii) the corresponding positions and times. Using pseudospectral methods and symbolic representation of pseudodifferential operators, we have solved this inverse problem by assuming that the data are randomly weakly perturbed. Although, some assumptions are still required to address this problem (linearity of the equation, order in time), some promising results are obtained, at least for sufficiently small amplitude perturbations. Extension to timedependent and nonlinear equations is currently investigated, using in particular artificial neural networks to consider larger sets of data. Appendix A. Spectral approximation In a bounded domain Πdi=1 [−axi , axi ], we denote the grid-point set by DNx1 ,··· ,Nxd = xj = (x1,j1 , · · · , xd,jd ) j∈O Πd Nxi i=1
where j = (j1 , · · · , jd ) and with OΠdi=1 Nxi = j ∈ Nd , : j1 = 0, · · · , Nx1 − 1; · · · ; jd = 0, · · · , Nxd − 1 . and xki +1 − xki = ∆xi = 2axi /Nxi . The corresponding discrete wavenumbers are defined by ξ i := (ξi,1 , · · · , ξi,d ), where ξp = pπ/axi with p ∈ {−Nxi /2, · · · , Nxi /2 − 1}. We can define the partial Fourier pseudospectral approximations φe with, in the xi -direction PNx1 −1 e b −iξk1 (xk1 +ax1 ) φek1 (t, x2 , · · · , xd )eiξp (x1 +ax1 ) = k1 =0 φk1 (t, x2 , · · · , xd )e ··· ··· ··· ··· ··· ··· PNxd −1 e be −iξkd (xkd +axd ) iξp (x1 +ax1 ) φkd (t, x1 , · · · , xd−1 )e = kd =0 φkd (t, x1 , · · · , xd−1 )e where we have also denoted e x1,k , x2 , · · · , xd ) φek1 (t, x2 , · · · , xd ) = φ(t, 1 ··· ··· ··· ··· ··· ··· e x1 , · · · , xd−1 , xd,k ). φekd (t, x1 , · · · , xd−1 ) = φ(t, d 29
We finally define the approximate first-order partial derivatives, using the pseudospectral approximation ∂xi φ(tn , x), by e := [[∂xi φ]]
1 PNxi /2−1 be iξki (xki +axi ) ki =−Nxi /2 iξp φk1 (t, x1 , · · · , xi−1 , xi+1 , · · · , xd )e Nxi (`)
Approximate higher (`th) order partial derivatives ∂xi φ(tn , x), by (`) e [[∂xi φ]] :=
` b 1 PNxi /2−1 iξ φek1 (t, x1 , · · · , xi−1 , xi+1 , · · · , xd )eiξki (xki +axi ) k i Nxi ki =−Nxi /2
We refer to [3] and [2] for details. [1] F. Alouges, F. De Vuyst, G. Le Coq, and E. Lorin. The reservoir technique: a way to make godunov-type schemes zero or very low diffuse. application to colella-glaz solver. European Journal of Mechanics, B/Fluids, 27(6):643–664, 2008. [2] X. Antoine, C. Besse, and V. Rispoli. High-order IMEX-spectral schemes for computing the dynamics of systems of nonlinear Schr¨odinger/GrossPitaevskii equations. J. Comput. Phys., 327:252–269, 2016. [3] X. Antoine and E. Lorin. Computational performance of simple and efficient sequential and parallel dirac equation solvers. Comput. Phys. Commun., 220:150–172, 2017. [4] A.D. Bandrauk, F. Fillion-Gourdeau, and E. Lorin. Atoms and molecules in intense laser fields: Gauge invariance of theory and models. Journal of Physics B: Atomic, Molecular and Optical Physics, 46(15), 2013. [5] D.P. Bertsekas and J.N. Tsitsiklis. Gradient convergence in gradient methods with errors. SIAM Journal on Optimization, 10(3):627–642, 2000. [6] N. Bouleau and D. L´epingle. Numerical methods for stochastic processes. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley & Sons, Inc., New York, 1994. A Wiley-Interscience Publication. [7] S.L. Brunton, J.L. Proctor, J.N. Kutz, and W. Bialek. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences of the United States of America, 113(15):3932–3937, 2016. [8] L. H¨ormander. Linear partial differential operators. Springer Verlag, Berlin, 1976. [9] L. H¨ormander. The analysis of linear partial differential operators. III. Classics in Mathematics. Springer, Berlin, 2007. Pseudo-differential operators. [10] D. Kincaid and W. Cheney. Numerical analysis. Brooks/Cole Publishing Co., Pacific Grove, CA, second edition, 1996. Mathematics of scientific computing. 30
[11] I.E. Lagaris, A. Likas, and D.I. Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987– 1000, 1998. [12] S.G. Mallat. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7):674–693, 1989. [13] L. Nirenberg. Lectures on linear partial differential equations. American Mathematical Society, Providence, R.I., 1973. [14] M. Raissi and G.E. Karniadakis. Hidden physics models: Machine learning of nonlinear partial differential equations. J. of Comput. Phys., 357:125–141, 2018. [15] M. Raissi, P. Perdikaris, and G.E. Karniadakis. Machine learning of linear differential equations using gaussian processes. J. of Comput. Phys., 348:683–693, 2017. [16] D. Serre. Syst`emes de lois de conservation. I. Fondations. [Foundations]. Diderot Editeur, Paris, 1996. Hyperbolicit´e, entropies, ondes de choc. [Hyperbolicity, entropies, shock waves]. [17] J. Sirignano and K. Spiliopoulos. DGM: A deep learning algorithm for solving partial differential equations. ArXiv e-prints, 2017. [18] J. Smoller. Shock waves and reaction-diffusion equations, volume 258 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Science]. Springer-Verlag, New York-Berlin, 1983. [19] M. E. Taylor. Partial differential equations III. Nonlinear equations, volume 117 of Applied Mathematical Sciences. Springer, New York, second edition, 2011. [20] E Weinan, J. Han, and A. Jentzen. Deep learning-based numerical methods for highdimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat., 5(4), 2017.
31
32
Figure 13: Computed coefficients: (1st row) α(0) , α(t) . (2nd row) β (x) , β (y) (3rd row) β (xx) , β (yy) . (4th row) β (xy) .
33
Figure 14: Computed coefficients: (1st row) α(0) , α(t) . (2nd row) β (x) , β (y) (3rd row) β (xx) , β (yy) . (4th row) β (xy) .
