Wavelet approach to numerical differentiation of noisy functions

Manuscript submitted to AIMS’ Journals Volume X, Number 0X, XX 200X

Website: http://AIMsciences.org pp. X–XX

WAVELET APPROACH TO NUMERICAL DIFFERENTIATION OF NOISY FUNCTIONS

Jianzhong Wang Department of Mathematics and Statistics Sam Houston State University 1901 Avenue J., P.O. Box 2206 Huntsville, TX 77341-2206, USA

Abstract. We apply wavelet transform in the study of numerical differentiation for the functions which are infected by noise. Because of the presence of noise, the observed noisy function is not differentiable. In order to estimate the derivatives of the target function from its observation, a pretreatment of the observation is necessary. The paper introduces differential approximation wavelets (DA-wavelets) so that the DA-wavelet transforms of the observed function approximate the derivatives of the target function. The paper also shows that the derivatives of compactly supported splines lead to a certain type of DA-wavelet transforms, which are difference formulas for computing derivatives. The relation between difference formulas and splines enables us to construct various difference formulas via splines and to estimate the computing errors of difference formulas in the spline framework.

1. Introduction. Let f (t) be a smooth signal (also called a smooth function) and u(t) be an observation (also called a noisy signal) of f (t), which carries on noise n(t) : u(t) = f (t) + n(t), (1) where n(t) is a random time function, (also called a stochastic process). It is well-known that derivatives of the signal f (t) present its important features. Unfortunately, when a signal f is infected by noise as in (1), the noisy signal u(t) is not differentiable so that we cannot directly estimate f (j) (t) from u(t). Therefore, how to estimate f (j) from u becomes an important task in numerical differentiation. In numerical differentiation, for computing derivatives of functions, we often use difference formulas. That is, we compute f (j) (t) using the values of f (t+kh), where h > 0 is the difference step and k ∈ Z. (Sometimes, non-uniform time-step is also used. Then kh is replaced by tk ). Hence, difference formulas are consistent with the discrete model of (1): u(t + hk) = f (t + hk) + n(t + hk),

k ∈ Z.

(2)

No matter what form the function f is of, to eliminate the effect of noise, a pretreatment of the observed signal is necessary. The main techniques to pre-treat the 2000 Mathematics Subject Classification. Primary: 42C40, 65D25; Secondary: 41A15, 65D07, 93E14. Key words and phrases. Wavelets, numerical differentiation, spline, noisy data, data analysis. The research supported by SHSU EGR-2006 Grant.

1

2

JIANZHONG WANG

noisy signals include filter method, regularization method, and polynomial interpolation method (see [1], [13], [15], [22], [23], [27], [30], and their relevant references). The purpose of the paper is to introduce wavelet transform method for estimating f (j) from u. The wavelet transform method enables us to treat the continuous model and the discrete model of signals in a same framework. It also generalizes the methods mentioned above. The notion of wavelet transform can be found in [3], [9], [14], [20], [21], and their references. In the paper we briefly introduce the (continuous) wavelet transform as follows. We abbreviate the notation of a space of functions defined on R, say X(R), by X. We define the space C r by n o C r = f | f (j) ∈ C, 0 ≤ j ≤ r . The space of locally absolutely continuous functions on R is denoted by ACloc , and the Sobolev space W r,p is defined by W r,p = {f | f (j) ∈ ACloc , 0 ≤ j ≤ r − 1, f (r) ∈ Lp },

1 ≤ p ≤ ∞.

When r = 0, the Sobolev space W r,p is reduced to the Lebesgue space Lp . The convolution of two functions (or a distribution and a function) f and g is defined by Z f ∗ g(t) = f (t − τ )g(τ )dτ, R

provided that the integral exists [31]. Particularly, the convolution of a function f ∈ C and the delta function δ, which is a distribution in the dual space of C, is f (t) = δ ∗ f (t). A function ψ ∈ L1 ∩ L2 is called a wavelet if Z ψ(t)dt = 0. R

For a real function f, we denote fs (t) = 1s f ( st ), f¯(t) = f (−t). The (continuous) wavelet transform (derived from ψ) of a function u ∈ L2 is defined by Z 1 τ −t s Wψ u(t) = ψ( )u(τ )dτ = ψ¯s ∗ u(t), s > 0, t ∈ R. (3) s R s We often need the scaled wavelet transform 1 Wψs,j u(t) = j Wψs u(t). s

(4)

For simplicity, in this paper, (4) is also called the wavelet transform of u. In applications, the wavelet ψ is often required to have certain regularity and vanishing moments. A function f is said to be r-regular (r ≥ 0) if f ∈ C r and, for any positive integer m, |f (j) (t)| ≤ Cm (1 + |t|)−m , A function f is said to have the j

th

order vanishing moment (j ∈ Z+ ) if

Z tj f (t)dt = 0. R

0 ≤ j ≤ r.

WAVELET IN NUMERICAL DIFFERENTIATION

3

If a function has the vanishing moments up to the (n − 1)th order, then it is said to have n vanishing moments (VM). If a function has the vanishing moments from 1st order to mth order (m ≥ 1), i.e. Z tj f (t)dt = 0, 1 ≤ j ≤ m, (5) R

then it is said to have m positive vanishing moments (PVM). The idea of approximating the j th derivative of a function f by its wavelet transform (4) comes from the approximation theory of singular integrals. In approximation theory, an r-regular function φ is called an approximative identity (AI-) function if it satisfies the condition Z φ(t)dt = 1. R

An AI-function generates a kernel of approximative identity {φs }s>0 , i.e., the following holds [2]. Proposition 1. Let φ be an r-regular (r ≥ 0) AI-function. Then lim ||φ¯s ∗ f − f ||Lp = 0, ∀f ∈ Lp , s→0+

lim ||φ¯s ∗ f − f ||C = 0,

s→0+

∀f ∈ C.

¡ ¢ By this proposition, φ¯s ∗ f (j) is an approximation of f (j) . Let ψ be the wavelet defined by ψ = (−1)j φ(j) . Then the wavelet transform Wψs,j f approximates f (j) . Note that, although the noisy signal u in (1) is not differentiable, its wavelet transform Wψs,j u exists. Thus, f (j) can be estimated by Wψs,j u. To develop difference formulas for computing Wψs,j u, we need to discretize the wavelet transform Wψs,j u. An alternate way is to adopt distribution-wavelet transforms. Let the Fourier transform of a compactly supported distribution f be denoted by Z fˆ(ω) = f (t)e−itω dt. R

Then fˆ is an analytic function. Thus, we define distribution wavelets as follows. Definition 1. A compactly supported distribution ψ is called a wavelet if ˆ ψ(0) = 0. Particularly, a distribution wavelet ψ in the form of ψ(t) =

N X

di δ(t − ti ),

t1 < t 2 < · · · < t N

(6)

i=1

is called a δ-wavelet. For a δ-wavelet ψ of form (6), we have Wψs,j (t) =

N 1 X di f (t + sti ). sj i=1

(7)

Hence, a δ-wavelet transform of f is a difference formula. Note that a δ-wavelet is the j th derivative of a spline of order j. Hence, representing difference formulas as δ-wavelet transforms enables us to apply the spline theory in their construction.

4

JIANZHONG WANG

It is well-known that when we apply difference formulas to compute derivatives on a computer, the computer introduces round-off errors, which are measured by the machine epsilon ²M . Because of the presence of round-off errors, difference formulas cannot provide arbitrary accuracy for computing derivatives as the difference step s → 0+ . For instance, the minimal error for the difference formula (f (t+s)−f (t))/s √ √ of f 0 (t) is attained at s = c²M , which leads to the optimal error bound d²M for a certain constant d. When noise presents, the accuracy of a difference formula is also impacted by noise. The δ-wavelet transform representations of difference formulas benefit the error analysis for both noise error and round-off error. For the spline theory, we refer readers to [4]–[7], [8], [24], [25], [26], [29], and their references. In this paper, we need the following notions and notations of splines. Let Π : {· · · t−1 < t0 < t1 < · · · }i∈I be a (finite or infinite) partition of R. A function s(x) defined on R is called a spline of order n (on Π) if X s(t) = pn−1 (t) + ci (t − ti )n−1 + , i∈I

where pn−1 is a polynomial of degree ≤ n − 1 and ½ (t − ti )n−1 , t > ti n−1 (t − ti )+ = . 0, t ≤ ti n The space of splines of order n on Π is denoted by SΠ . If Π = Z, then it is abbreviated n n to S , and a spline in S is called a cardinal spline of order n. The space Shn is defined by Shn = {s(t)| s( ht ) ∈ S n }. The subspace of all compactly supported splines in n,0 n SΠ is denoted by SΠ . The cardinal B-spline Nm ∈ S m,0 is defined recursively by

Nm = Nm−1 ∗ N1 ,

N1 = χ[0,1) ,

The Fourier transform of Nm is µ ¶ ¶ µ −iω m sin ω2 m ˆm (ω) = 1 − e N , = e−iωm/2 ω iω 2 and the explicit expression of Nm (t) is m

Nm (t) =

X 1 (−1)k (m − 1)! k=0

µ

m k

(8)

¶ (t − k)m−1 . +

It is known that supp Nm = [0, m], Nm (t) ∈ C m−2 , and {Nm (t − k)}k∈Z is a locally linearly independent basis of S m [8]. The mth derivative of Nm (t) is µ ¶ m X m (m) k (−1) δ(t − k). Nm (t) = k k=0

When m is even, say m = 2n, we define the central (cardinal) B-spline by C2n (t) = N2n (t + n), whose Fourier transform is µ ¶ sin ω2 2n Cˆ2n (ω) = . (9) ω 2

The following notions and notations in statistics will be used in the paper. The expectation of a random variable x is denoted by E(x). The mean of a random function of x(t) is the function E(x(t)), and the autocorrelation function of x(t) is the two-variable function Rx (t1 , t2 ) = E(x(t1 )x(t2 )).

WAVELET IN NUMERICAL DIFFERENTIATION

5

The time average of a random function x(t) is defined by Z T 1 M (x) = lim x(t)dt, T →∞ 2T −T and the time correlation function of x(t) is Z T 1 Rx (t) = lim x(τ )x(t + τ )dτ. T →∞ 2T −T A random function x(t) is called wide-sense stationary if (1) E(x(t)) is independent of t, (hence, we can denote it by E(x)) (2) its autocorrelation function Rx (t1 , t2 ) is only dependent of the difference of two times: Rx (t1 , t2 ) = Rx (t2 − t1 ), (3) its mean square value is finite : E(x2 (t)) < ∞. A wide-sense stationary random function is called ergodic if it also satisfies the following: (4) M (x) is independent of the selection of samples and M (x) = E(x), and (4) its time correlation function is equal to its autocorrelation function: Rx (t) = Rx (t). Thus, for an ergodic random function, its statistical characteristics can be attained by one of its sample functions. Assume x(t) is wide-sense stationary. Let xT (t) = x(t)χ[−T.T ] (t). The power spectrum density of x(t) is defined by 1 2 Px (ω) = lim E(|xc T (ω)| ). T →∞ 2T We have Z 1 2 E(x (t)) = Px (ω)dω. 2π R cx (ω). Hence, if x(t) is ergodic, then By Wiener-Khinchine Theorem, Px (ω) = R cx (ω). Px (ω) = R In this paper, noise is always assumed to be ergodic, and the signal f and the noise n in (1) are assumed to be uncorrelated, i.e., Z n(τ )f (τ + t)dτ = 0 ∀t ∈ R. R

If an ergodic noise n(t) satisfies M (n) = 0 and Pn (ω) = σ 2 , then n(t) is called an (idea) white noise. If a noise n satisfies Pn (ω) = 0, |ω| ≤ M, for a large M > 0, then it is called a high frequency noise. A high frequency noise is said to have a line-spectrum if m X n(t) = Ak sin(ωk t + φk ), k=1

where, for each k, ωk ≥ M, Ak and ωk are regular variables, and φk is a random variable with the uniform distribution on (0, 2π). When time is discrete, random functions become random sequences. Wide-sense stationary random sequences and ergodic random sequences are defined in a similar way. Assume n(k) is a wide-sense stationary random sequence. Let nN (k) = ½ n(k) |k| ≤ N . The power spectrum density of n(k) is defined by 0 |k| > N Pn (ω) = lim

T →∞

1 2 E(|n∧ N (ω)| ), 2N

6

JIANZHONG WANG

where x∧ (ω) = have

P k∈Z

x(k)e−ikω denotes the Fourier series of a sequence (x(k)) . We E(n2 (k)) =

1 2π

Z

2π

Pn (ω)dω 0

and Pn (ω) = R∧ n (ω). If n(k) is ergodic, then Pn (ω) = Rn∧ (ω). The paper is organized as follows. In the next section, we study the approximation of derivatives by wavelet transform. The analysis of the approximation error is included in Section 3. The methods for constructing differential approximation wavelets are introduced in Section 4. In section 5, we construct differential approximation wavelets of Gaussian-type. In Section 6, we discuss how to build the difference formulas using AI-splines. 2. Differential Approximation Based on Wavelet Transforms. In this section, we study the wavelet transform approach to the approximation of derivatives of functions. We first give the following definition. Definition 2. An r-regular wavelet (or a δ-wavelet) ψ is called a differential approximation (DA-)wavelet (or a DA δ-wavelet) of order (j, n) if ψ satisfies the following vanishing moment conditions Z tk ψ(t)dt = j!δk,j , 0 ≤ k ≤ n + j − 1, (10) ½ where δk,j =

R

1, k = j . 0, k = 6 j

A DA-wavelet of order (j, n) can be attained from the j th derivative of an AIfunction φ. In fact, we have the following. Lemma 1. An r-regular wavelet ψ is a DA-wavelet of order (j, n) if and only if there is an (r + j)-regular AI-function φ with n − 1 PVM such that ψ(t) := (−1)j φ(j) (t). A δ-wavelet N X ψ(t) = di δ(t − ti ), t0 < t1 < · · · < tN , i=0

is a DA-wavelet of order (j, n), j ≤ n, if and only if there is a compactly supported j AI-spline φ ∈ SΠ (Π = {t0, t1 , · · · , tN }), which has supp φ = [t0 , tN ] and n − 1 PVM, such that ψ(t) := (−1)j φ(j) (t). Proof. If φ is an (r + j)-regular AI-function which has n − 1 PVM, then ψ(t) := (−1)j φ(j) (t) is r-regular. When k ≤ j, we have Z Z Z tk φ(j) (t)dt = (−1)j−k k! φ(j−k) (t)dt = j!δk,j , tk ψ(t)dt = (−1)j R

R

and, when j + 1 ≤ k ≤ n + j − 1, we have Z Z tk φ(j) (t)dt = (−1)k−j tk ψ(t)dx = (−1)j R

R

R

k! (k − j)!

Z tk−j φ(t)dt = 0. R

WAVELET IN NUMERICAL DIFFERENTIATION

7

Hence, ψ is an r-regular DA-wavelet of order (j, n). We now assume that ψ(t) is an r-regular DA-wavelet of order (j, n). Define Z u2 Z t Z uj φ(t) = (−1)j ··· ψ(u1 )du1 · · · duj . R

−∞

−∞

−∞

k

Since ψ(t) is r-regular and R t ψj (t)dt = 0, 0 ≤ k ≤ j − 1, we have that limt→∞ φ(t) = 0 and φ is (r + j)-regular with φ(j) (t) = (−1)j ψ(t). Finally, the conditions Z tk ψj (t)dt = j!δk,j , j ≤ k ≤ n + j − 1, R

yield, for 0 ≤ k ≤ n − 1, Z Z Z k! k! tk φ(t)dt = (−1)j tk+j φ(j) (t)dt = tk+j ψj (t)dt = δk,0 , (k + j)! R (k + j)! R R i.e., φ is an AI-function with n − 1 PVM. The similar proof can be completed for the case that ψ is a δ-wavelet. The main result of this section is the following. Theorem 1. Let ψ be an r-regular DA-wavelet (or a DA δ-wavelet) of order (j, n), j > 0, and φ be the (j + r)-regular AI-function (or the compactly supported AI-spline of order j) with n − 1 PVM such that (−1)j φ(j) = ψ. Then for f ∈ C j lim Wψs,j (f, t) = f (j) (t),

s→0+

and the approximation order is n. More precisely, for f ∈ C j+n+1 , ¶ µ (j+n) Z ¯ ¯ f (t) ¯ s,j ¯ (j) n n φ(τ )τ dτ + °(sn+1 ), ¯Wψ (f, t) − f (t)¯ = s n! R the approximation error for f ∈ C j+n can be estimated by ¶° µ Z ¯ ° ¯ 1 ¯ s,j ° ° ¯ n (j) n |φ(τ )| |τ | dτ °f (n+j) ° , ¯Wψ (f, t) − f (t)¯ ≤ s n! R C

(11)

(12)

and the approximation error for f ∈ W j+n,p can be estimated by µ Z ¶° ° ° ° 1 ° s,j ° ° ° n |φ(τ )| |τ | dτ °f (n+j) ° p . (13) °Wψ (f ) − f (j) ° 2 ≤ sn n! R L L ¯ ¯ ¯ ¯ In numerical differentiation, ¯Wψs,j (f, t) − f (j) (t)¯ is often called the truncated error. To prove theorem 1, we need the following two lemmas. Lemma 2. Let f ∈ W r,p or C r . Then f (t + v) =

r−1 k (k) X v f (t) k=0

k!

+

vr (r − 1)!

Z

1

ur−1 f (r) (t + v(1 − u))du.

Proof. For f ∈ W r,p or C r , we have the Taylor’s formula Z z r−1 X (z − t)k f (k) (t) 1 f (z) = + (z − τ )r−1 f (r) (τ )dτ. k! (r − 1)! x k=0

Let z = t + v, we have f (t + v) =

r−1 k (k) X v f (t) k=0

k!

+

1 (r − 1)!

(14)

0

Z t

t+v

(t + v − τ )r−1 f (r) (τ )dτ.

(15)

8

JIANZHONG WANG

In the integral, making the variable change τ = t + v(1 − u), we obtain (14). Lemma 3. Let φ be an r-regular (r ≥ 0) AI-function with n − 1 PVM. Then for f ∈ C n+1 , µ Z ¶ Z 1 n n φs (v − t)f (v)dv − f (t) = s φ(τ )τ dτ f (n) (t) + °(sn+1 ), (16) n! R R for f ∈ C n , ¯Z ¯ µ Z ¶° ° ¯ ¯ ° (n) ° n ¯ φs (v − t)f (v)dv − f (t)¯ ≤ sn 1 f |φ(τ )| |τ | dτ ° , ° ¯ ¯ n! C R

(17)

R

and for f ∈ W n,p , °Z ° ° ° ° φs (v − t)f (v)dv − f (t)° ° ° R

µ ≤s

n

L2

1 n!

Z

¶° ° ° ° |φ(τ )| |τ | dτ °f (n) ° n

R

L2

.

(18)

Proof. We first prove (16). Applying Lemma 2 to f ∈ C n+1 , we have Z n X v n+1 1 n (n+1) v k (k) f (t + v) = f (t) + f (t) + u f (t + v(1 − u))du. k! n! 0 k=1

Pn−1

k

v (k) Recall that p(v) := (t) is a polynomial of v of degree ≤ n − 1 and k=1 k! f p(0) = 0. Hence, (5) yields Z Z φs (v − t)p(v)dv = φs (v)p(v + t)dv = 0. R

R

Therefore, Z φs (v − t)f (v)dv − f (t) R Z = φs (v) (f (t + v) − f (t)) dv R

µZ 1 ¶ Z Z f (n) (t) 1 n n+1 n (n+1) = φs (v)v dv + φs (v)v u f (t + v(1 − u))du dv n! n! R 0 µ ZR ¶ 1 = sn φ(τ )τ n dτ f (n) (t) n! R µZ 1 ¶ Z n+1 1 n+1 n (n+1) +s φ(τ )τ u f (t + sτ (1 − u))du dτ. n! R 0 By ¯Z µZ ¯ ¯ φ(τ )τ n+1 ¯ R

0

1

¶ ¯ Z ¯ ||f (n+1) ||C ¯ ¯ ¯φ(τ )τ n+1 ¯ dτ, un f (n+1) (t + sτ (1 − u))du dτ ¯¯ ≤ n+1 R

we get (16). The proofs of (17) and (18) are similar. We now finish the proof of Theorem 1. Proof. Let ψ be an r-regular DA-wavelet of order (r, n). By Lemma 1, there is an (r + j)-regular AI-function φ with n − 1 PVM such that (−1)j φ(j) = ψ. By Lemma 3, for f ∈ C n+j+1 , we have µ Z ¶ Z 1 (j) (j) n n φs (v − t)f (v)dv − f (t) = s φ(τ )τ dτ f (n+j) (t) + °(sn+1 ). (19) n! R R

WAVELET IN NUMERICAL DIFFERENTIATION

9

By mathematical induction, we have Z Z Z 0 φs (v − t)f (j) (v)dv = φs (v)f (j) (v + t)dv = − (φs ) (v)f (j−1) (t + v)dv R R R Z (j) j = (−1) (φs ) (v)f (t + v)dv = Wψs,j (f, t), R

which combining with (19) yields (11). The proof of (12) and (13) are similar. In a similar way, we can prove the theorem for a DA δ-wavelet ψ. 3. DA-Wavelet Transform of Noise. We now return to the model (1) and estimate f (j) by Wψs,j u. From (1), we have Wψs,j u(t) = Wψs,j f (t) + Wψs,j n(t), where both Wψs,j u(t) and Wψs,j n(t) are random time function.We measure the error of the estimate by µ¯ µ¯ ¯2 ¶ ¯ ¯2 ¯2 ¶ ¯ ¯ ¯ ¯ ¯ ¯ E ¯Wψs,j u(t) − f (j) (t)¯ = ¯Wψs,j f (t) − f (j) (t)¯ + E ¯Wψs,j n(t)¯ . (20) ¯ ¯ ¯ ¯ In (20), ¯Wψs,j f (t) − f (j) (t)¯ has been estimated in the previous section. In this µ¯ ¯2 ¶ ¯ ¯ section, we estimate E ¯Wψs,j n(t)¯ . We first consider the wavelet transform derived from an r-regular (r ≥ 0) DAwavelet ψ of order (j, n). Since n(t) is an ergodic random function, we have µ¯ Z Z ¯2 ¶ 1 ¯ ¯ E ¯Wψs,j n(t)¯ = 2j ψs (x)ψs (y)E (n(t + x)n(t + y)) dxdy s ZR ZR 1 = 2j ψs (x)ψs (y)Rn (x − y)) dxdy s ZR R 1 (ψs ∗ Rn ) (x)ψs (x)dx = 2j s R µ ¶ Z ¯ ¯2 1 1 ¯b ¯ = 2j ¯ψ(sω)¯ Pn (ω)dω s 2π R µ ¶ Z ¯ ¯ 1 1 ω ¯ b ¯2 = 2j+1 ¯ψ(ω)¯ Pn ( )dω . s 2π R s Thus, we have proved the following. Theorem 2. In (1), assume f ∈ C j+n and n(t) is an ergodic random function. Let ψ be an r-regular (r ≥ 0) DA-wavelet of order (j, n) and φ be the (r + j)-regular AI-function such that (−1)j φ(j) = ψ. Write ¶° µ Z ° 1 ° ° n |φ(τ )| |τ | dτ °f (n+j) ° . Af = n! R C Then

µ¯ ¯2 ¶ ¯ s,j ¯ (j) E ¯Wψ u(t) − f (t)¯ ≤ A2f s2n +

1 2j+1 s 2π 1

Z ¯ ¯ ω ¯ b ¯2 ¯ψ(ω)¯ Pn ( )dω. s R

(21)

10

JIANZHONG WANG

We now find s which minimizes the error (21). Case of white noise. Assume the noise n(t) in (1) is a white noise with Pn (ω) = σ 2 . Then we have µ¯ ¯2 ¶ σ2 ¯ ¯ 2 E ¯Wψs,j u(t) − f (j) (t)¯ ≤ A2f s2n + 2j+1 kψkL2 . (22) s When

à s=

2

(2j + 1) kψkL2 2nA2f

1 ! 2n+2j+1 2

σ 2n+2j+1 ,

(23)

(22) has the minimal estimate error µ¯ ¯2 ¶ 4n ¯ ¯ E ¯Wψs,j u(t) − f (j) (t)¯ ≤ Mψ,f σ 2n+2j+1 , where Mψ,f =

µ 2n ¡ 2j+1 ¢ 2n+2j+1 2n

³ +

2n 2j+1

¶ 2j+1 ´ 2n+2j+1

4j+2

(24) 4n

Af2n+2j+1 kψkL2n+2j+1 . 2

Case of high frequency noise. Assume the noise in (1) is a high frequency noise with PN (ω) = 0, |ω| ≤ M. Then Z ¯ Z ¯ ¯ ¯ 1 1 ω ω ¯ b ¯2 ¯ b ¯2 ¯ψ(ω)¯ PN ( )dω = ¯ψ(ω)¯ PN ( )dω 2π R s 2π |sM |≥0 s Pm If the noise n(t) has the line-spectrum: n(t) = k=1 Ak sin(ωk t + φk ), we have m X A2

k

Rn (τ ) =

k=1

which yields

2

cos ωk τ,

m

Pn (ω) =

1X 2 Ak (δ(ω − ωk ) + δ(ω + ωk )) . 4 k=1

Hence, 1

1 2j+1 s 2π

Z ¯ m ¯ ¯2 1 X 2 ¯¯ ˆ ω ¯ b ¯2 ¯ A ψ(sω ) ¯ψ(ω)¯ PN ( )dω = ¯ ¯ , k k s 4πs2j R k=1

which yields

µ¯ ¯2 ¶ ¯ ¯ E ¯Wψs,j u(t) − f (j) (t)¯ ≤ A2f s2n +

m ¯2 1 X 2 ¯¯ ˆ ¯ A ψ(sω ) ¯ k ¯ . k 4πs2j

(25)

k=1

We now consider the δ-wavelet transform. Assume the δ-wavelet ψ has the form of ψ(t) =

N X

dk δ(t − tk ),

(26)

k=0

Then the δ-wavelet transform of a function u ∈ C is Wψs,j u(t) =

N 1 X dk u(t + stk ). sj

(27)

k=0

Applying (27) to (1), we have Wψs,j u(t) = Wψs,j f (t) + Wψs,j n(t) =

N N 1 X 1 X d f (t + st ) + dk n(t + stk ). k k sj sj k=0

k=0

WAVELET IN NUMERICAL DIFFERENTIATION

11

Assume the random sequence n ˜ (k) := n(t + stk ) is a white noise with Pn (ω) = σ 2 . Then ¯ ¯2  N N ¯X ¯ ´ 1 1 X 2 ³ σ2 ¯ ¯ 2 2 E(|Wψs,j n(t)|2 ) = 2j E ¯ dk n ˜ (k)¯  = 2j di E |˜ n(k)| dω = 2j kdkl2 . ¯ ¯ s s s k=0

k=0

Hence, we have proved the following. Theorem 3. In (1), assume f ∈ C j+n and n is a white noise ∼ N (0, σ 2 ). Let ψ be a DA δ-wavelet of order (j, n) which has the form (26), and φ be the AI-spline of order j such that (−1)j φ(j) = ψ. Then µ¯ ¯2 ¶ σ2 ¯ s,j ¯ 2 (j) E ¯Wψ u(t) − f (t)¯ ≤ A2f s2n + 2j kdkl2 . (28) s By the theorem, the optimal s is 1 à ! 2n+2j+1 2 (2j + 1) kdkl2 s= 2nA2f

which yields the error estimate µ¯ ¯2 ¶ ¯ ¯ E ¯Wψs,j u(t) − f (j) (t)¯ ≤ mψ,f where mψ,f =

µ 2n ¡ 2j+1 ¢ 2n+2j+1 2n

³ +

2n 2j+1

¶ 2j+1 ´ 2n+2j+1

√

2n+2j+1

σ2 ,

√

2n+2j+1

4j+2

(29)

σ 4n ,

(30) 4n

Af2n+2j+1 kdkl2n+2j+1 is a constant 2

independent of σ. Finally, we briefly discuss how the round-off errors impact the accuracy for computing a DA δ-wavelet transform. We first assume the noise does not exist on the function f. Then we can apply the wavelet transform directly to the sampling data of the function f ∈ C. Thus, by (26), we have Wψs,j f (t) =

N 1 X dk f (t + stk ). sj k=0

As we have proved in the previous section, the truncated error of the DA δ-wavelet transform for f ∈ C n+j is ¯ ¯ ¯ s,j ¯ ¯Wψ f (t) − f (j) (t)¯ ≤ Af sn . Let ²M denote the machine epsilon of the computer. Then the round-off error for computing Wψs,j f (t) is 2 (N + 1) ²M Er = . sj Adding the round-off error and the truncated error together, the total error for estimating f (j) (t) by Wψs,j f (t) is ¯ ¯ 2(N + 1)²M ¯ s,j ¯ . ¯Wψ f (t) − f (j) (t)¯ ≤ Af sn + sj Then the optimal s is ! Ã s √ 2(N + 1)j n+j n+j s= ²M , nAf

12

JIANZHONG WANG

which yields the error estimate ¯ ¯ ¯ s,j ¯ ¯Wψ f (t) − f (j) (t)¯ ≤ Rψ,j where Rψ,j =

à q ¡ j ¢n n+j n

+

r³ ´ ! j

n+j

n j

q n+j

p

n+j

²nM ,

j

n

(Af ) (2(N + 1)) . If the function f

carries on a white noise ∼ N (0, σ 2 ), where σ >> ²M , the round-off error for computing Wψs,j f (t) becomes much smaller than the error caused by noise (see (29) and (30)). Hence, the round-off error can be neglected. 4. Construction of DA-Wavelets. In Section 2, we showed that a DA-wavelet ψ of order (j, n) is the j th derivative of an AI-function with n − 1 PVM. Hence, in order to construct DA-wavelets, we only need to construct AI-function with the conditions (5). In this section, we discuss the methods to construct AI-functions with certain PVM. Recall that many types of functions, such as continuous probability density, Bsplines, continuous low-pass filters, are AI-functions. Hence, we start our construction from a real r-regular AI-function φ (which may have 0 PVM). Recall that ˆ φˆ ∈ C ∞ with φ(0) = 1. Hence, the Maclaurin’s series of φ1ˆ exists. Let 1 ˆ φ(ω)

=1+

∞ X

k

ck (iω) ,

(31)

k=1

ˆ) k dk (1/φ k f (·)(ω), the conditions (5) where ck = (−i) (0) is real. By ik fˆ(k) (ω) = ·\ k! dω k now are equivalent to the conditions

φˆ(k) (0) = δk,0,

0 ≤ k ≤ n − 1.

We introduce three methods to construct AI-functions with the conditions (5) from φ. Method 1 [Derivative method]. We use a linear combination of φ and its derivatives to construct an AI-function with the conditions (5). The construction is described in the following theorem. Theorem 4. Let φ be an r-regular AI-function. Let the Maclaurin’s series of (31). Then the function Φn (t) = φ(t) +

n−1 X

cj φ(j) (t)

1 ˆ φ

is

(32)

j=1

is an (r − n + 1)-regular AI-function with n − 1 PVM. If φ is even and n = 2m, then the function m−1 X Φe2m (t) = φ(t) + c2j φ(2j) (t) j=1

is an (r − 2m + 1)-regular AI-function with 2m − 1 PVM. Pn−1 d . Proof. Let Pn (x) = 1 + k=1 ck xk . Recall that Φn (t) = Pn (D)φ(t), where D = dt r r φ(ω) = (iω) φ(ω), d ˆ ˆ cn (ω) = Pn (iω)φ(ω), By D we have Φ whose Maclaurin’s series

WAVELET IN NUMERICAL DIFFERENTIATION

has the form

à cn (ω) = Φ

1+

It follows that cn Φ

!

1 ˆ φ(ω)

(j)

ˆ φ(ω)

k

ck (iω)

k=1

à =

n−1 X

13

!

ˆ + °(ω ) φ(ω) = 1 + °(ω n ). n

(0) = δ0,j ,

0 ≤ j ≤ n − 1,

i.e., Φn has n − 1 PVM. The first part of theorem is proved. The proof for Φe2m is similar. Method 2 [Translate method]. The smoothness of Φn in Theorem 4 is less than φ unless φ ∈ C ∞ . The translate method will not change the smoothness of the constructed AI-functions. Let the translate operator Sh (h > 0) be defined ihω ˆ f (ω). Since the following lemma is by Sh f (t) = f (t + h). We have Sd h f (ω) = e obvious, we omit its proof. r,r−l

Lemma 4. Let c =[1, c1 , · · · cr ]T be a vector in Rr+1 , let Mhr,l = (mj,k )j=0,k=−l be the (r + 1) × (r + 1) matrix, where mj,k = (kh)j , and let a = [a−l , · · · , ar−l ]T be attained by ³ ´−1 a = Mhr,l c. (33) Then the exponential polynomial Ar (x) =

r−l X

ak ekhx

(34)

k=−l

satisfies A(j) r (0) = cj , Thr

0 ≤ j ≤ r.

(35)

r (tj,k )j,k=0

Similarly, let = be the (r + 1) × (r + 1) matrix, where tj,k = £ ¤ (−1)j (kh)2j . Let the vector b = [b0 , · · · , br ]T be attained by −1

b = (Thr )

c.

(36)

bk cos khω

(37)

Then the cosine polynomial Br (ω) =

r X k=0

satisfies Br(2j) (0) = cj ,

0 ≤ j ≤ r.

(38)

By Lemma 4, we have the following. Theorem 5. Let φ be an r-regular AI-function. Let c =[1, c1 , · · · cn−1 ]T with ck = ³ ´−1 ˆ) Pn−1−l dk (1/φ (−i)k dωk (0), and a = Mhn−1,l c. Write αn (x) = ak xk . Then the k=l AI-function Φn,h (t) = αn (Sh )φ(t) (39)

14

JIANZHONG WANG

has n − 1 PVM. Assume φ is an even function. Let ce = [1, c2 , · · · , c2m−2 ]T with ˆ) ¡ ¢−1 Pm−1 ³ xk +x−k ´ d2k (1/φ ce . Write βm (x) = c2k = dω2k (0), and b = Thm−1 . k=0 bk 2 Then the AI-function Φem,h (t) = βm (Sh )φ(t)

(40)

has 2m − 1 PVM. Proof. We have ihω ˆ Φd )φ(ω) n,h (ω) = αn (e

where αn(j) (0) =

n−1−l X

j

ak (ihk) = ij cj =

³ ´ dj 1/φˆ dω j

k=l

Hence,

(0),

0 ≤ j ≤ n − 1.

³ ´  k ˆ d 1/ φ 1 1 αn (eihω ) = (0) ω k = + °(ω n ), ˆ k! dω k φ(ω) k=0 n−1 X

which yields



à Φd n,h (ω) =

1 ˆ φ(ω)

! ˆ = 1 + °(ω n ), + °(ω n ) φ(ω)

i.e., Φn,h (t) has n − 1 PVM. The proof for Φem,h is similar. In the approximation theory, the operators αn (Sh ) and βm (Sh ) are called quasiinterpolation operators. The detailed discussion of quasi-interpolation operators can refer to [10], [16], and [17]. Method 3 [Dilation method]. The third way to construct an AI-function from φ is to apply dilation operators on φ. The dilation operator ρb (b > 0, b 6= 1) is defined by ρb f (t) = f (bt) . Theorem 6. Let φ be an r-regular AI-function and ρb be a dilation operator. Let Vbn be the Vandermonde matrix   1 1 ··· 1  1 1/b ··· 1/bn−1    Vbn =  .. .. .. .. ,  .  . . . n−1 (n−1)2 1 1/b · · · 1/b T

and en1 = [1, 0, · · · , 0] ∈ Rn be the first vector of the standard basis for Rn . Let Pn−1 d =[d0 , d1 , · · · dn−1 ]T be determined by Vbn d = en1 . Write dn (x) = k=0 dk xk . Then the function Φn,b (t) = dn (bρb )φ(t) has n − 1 PVM. Assume φ is an even function. Then the AI-function ³√ ´ Φe2n,b (t) = dn bρ√b φ(t) has 2n − 1 PVM.

(41)

(42)

WAVELET IN NUMERICAL DIFFERENTIATION

15

Proof. We have d Φ n,b (ω) =

n−1 X

dj φ(b−j ω).

j=0

ˆ Recall that φ(ω) =1+

P∞

ˆ(k) (0) k φ k=1 k! ω .

Hence,   n−1 ∞ n−1 X X X φˆ(k) (0) k d  ω . Φ dj + dj b−jk  n,b (ω) = k! j=0 j=0 k=1

³ ´(k) d Then Φ (0) = δ0,k , 0 ≤ k ≤ n − 1, if and only if Vbn d = en1 . Assume now φ is n,b even. Hence, φ(2k−1) = 0 for k ∈ N. We have   n−1 ∞ n−1 X X X ˆ(2k) (0) −jk  φ e [  ω 2k . Φ (ω) = d + d b j j 2n,b (2k)! j=0 j=0 k=1

³

e [ Hence, Φ 2n,b completed.

´(k)

(0) = δ0,k , 0 ≤ k ≤ 2n − 1, if and only if Vbn d = en1 . The proof is

Note that dn (x) is the first Lagrangian polynomial of degree n − 1 for the point ³ n ´ b 1 n−1 set {1, 1/b, · · · , 1/b }. Hence, dn+1 (x) = bn −1 x − bn −1 dn (x), i.e., we have the following. Lemma 5. The polynomial dn (x) in Theorem 6 satisfies the recursive formula µ n ¶ b 1 d1 (x) = 1, dn+1 (x) = x − dn (x), ∀n ≥ 1. (43) bn − 1 bn − 1 Let Φn,b (t) be the AI-function defined by (41). By Lemma 5, Φn+1,b (t) =

bn+1 1 Φn,b (bt) − n Φn,b (t). bn − 1 b −1 (j)

Since the DA-wavelet of order (j, n) is Ψj,n (t) = (−1)j Φn,b (t), we have the recursive formula bn+j+1 1 Ψj,n+1 (t) = n Ψj,n (bt) − n Ψj,n (t). (44) b −1 b −1 Similarly, let Φe2n,b be an even AI-function defined by (42). Then Ψej,2n (t) = ³ ´(j) (−1)j Φe2n,b (t) is a DA-wavelet of order (j, 2n), which satisfies the recursive formula √ bn+(j+1)/2 c 1 Ψej,2n+2 (t) = Ψj,2n ( bt) − n Ψe (t). (45) n b −1 b − 1 j,2n The readers can see that the formulas (44) and (45) provide the Richardson extrapolation technique. We summary the conclusion into the following theorem. Theorem 7. Let ψj,n be an arbitrary DA-wavelet of order (j, n). Then the DAwavelet ψj,n+1 derived by the recursive formula (44) is a DA-wavelet of order (j, n+ e 1). Let ψj,2n be an arbitrary even DA-wavelet of order (j, 2n). Then the DA-wavelet e ψj,2n+2 derived by the recursive formula (45) is a DA-wavelet of order (j, 2n + 2).

16

JIANZHONG WANG

5. DA-Wavelets of Gaussian Type. In applications, the Gaussian function G(t) = √12π exp(−t2 /2) is an important AI-function. The convolution Gs ∗ f is called the Gauss-Weierstrass singular integral. In this section we discuss how to construct DA-wavelets by using the Gaussian function. If a DA-wavelet is a linear combination of G and its dilations (or its translates, or its derivatives), it is called a DA-wavelet of Gaussian type. The Gaussian function G is even and then has 1 PVM. Hence, (−1)j G(j) (t) is the DA-wavelet of Gaussian type of order (j, 2), among which G00 (t) has the name of Mexican Hat Wavelet. The author of [12] applies the Richardson extrapolation technique to the wavelet transform derived by Mexican Hat Wavelet to approximate the second derivative of functions. In this section we will construct general DA-wavelets of Gaussian type. We start our discussion from 1 ˆ G(ω)

= exp(ω 2 /2) =

∞ X (−1)k (iω)2k k=0

2k k!

.

We first apply the derivative method. By Theorem 4, we have the following result. The AI-function of Gaussian type φG,2m (t) =

m−1 X k=0

(−1)k (2k) G (t) 2k k!

(46)

has 2m − 1 PVM. The following lemma gives the recursive formula to represent D(j) (t) as a product of G(t) and a polynomial of t. Lemma 6. The j th derivative of G(t) can be written as G(j) (t) = Pj (t)G(t),

(47)

where Pj (t) is a polynomial of degree j, which has the following properties: (1) P2m−1 (t) is an odd polynomial and P2m (t) is an even polynomial. (2) Pj (t) satisfies the following recursive formulas: P0 (t) = 1, 0 (t) − tPj−1 (t), Pj (t) = Pj−1

j ≥ 1,

(48)

and Pj0 (t) = −jPj−1 (t).

(49)

(3) The polynomial Pj (t) has exact j distinct real roots. If the roots of Pj (t) are arranged as tj,1 < tj,2 < · · · < tj,j , then tj,k = −tj,j−k . Furthermore, the roots of Pj and the roots of Pj−1 are alternate, i.e. tj,1 < tj−1,1 < tj,2 < tj−1,2 < · · · < tj−1,j−1 < tj,j .

(50)

Proof. The recursive formula (48) can be easily proved by the mathematical induction. Applying (48) and by the mathematical induction, we have 00 0 Pj0 (t) = Pj−1 (t) − tPj−1 (t) − Pj−1 (t) 0 = − (j − 1) Pj−2 (t) + t (j − 1) Pj−2 (t) − Pj−1 (t) = −jPj−1 (t),

which completes the proof of (49). Finally, (50) follows (48) and (49). The lemma is proved. By (46), we have the following construction theorem for DA-wavelets of Gaussian type.

WAVELET IN NUMERICAL DIFFERENTIATION

17

Theorem 8. A DA-wavelet of Gaussian type of order (j, 2m) may have the expression G,1 (t) = (−1)j Q2m−2 (t)Pj (t)G(t), Wj,2m

(51)

where Q2m−2 (t) =

m−1 X k=0

(−1)k P2k (t) 2k k!

and Pn is the polynomial of degree n defined by (48). Besides, the DA-wavelet G,1 Wj,2m (t) has exact 2m + j − 2 cross-zeros. (j)

G Proof. Recall that Wj,2m (t) = (−1)j φG,2m (t). By (46) and (47), we have (51). Q2m−2 (t)Pj (t) is a polynomial of degree 2m + j − 2. Hence, it has at most 2m + j − 2 G has 2m + j VM, it has at lest 2m + j − 2 cross-zeros. The zeros. Because Wj,2m theorem is proved.

By Theorem 8, we obtain µ

¶ 3 1 3 = −Q2 (t)G (t) = t − t G(t), 2 2 µ ¶ 1 3 4 G,1 00 2 W2,4 (t) = Q2 (t)G (t) = − + 2t − t G(t), 2 2 µ ¶ 15 5 3 1 5 G,1 0 W1,6 (t) = −Q4 (t)G (t) = t − t + t G(t), 8 4 8 µ ¶ 15 25 2 11 4 1 6 G,1 00 W2,6 (t) = Q4 (t)G (t) = − + t − t + t G(t), 8 8 8 8 µ ¶ 35 35 3 7 5 1 7 G,1 0 W1,8 (t) = Q6 (t)G (t) = t − t + t − t G(t), 16 16 16 48 µ ¶ 35 35 2 21 4 11 6 1 8 G,1 00 W2,8 (t) = Q6 (t)G (t) = − + t − t + t − t G(t), 16 8 8 24 48 µ ¶ 315 105 3 63 5 3 7 1 9 G,1 0 W1,10 (t) = Q8 (t)G (t) = t− t + t − t + t G(t), 128 32 64 32 384 µ ¶ 37 8 1 10 315 735 2 273 4 69 6 G,1 W2,8 (t) = Q8 (t)G00 (t) = − + t − t + t − t + t G(t). 128 128 64 64 384 384 G,1 W1,4 (t)

0

We now apply the translate method (setting h = 1) to construct DA-wavelets of Gaussian type. Some examples are followings. i (−1)j h (j) G (t + 1) + G(j) (t − 1) , 2 (−1)j (j) G,2 Wj,6 (t) = [G (t + 2) − 7G(j) (t + 1) + 18G(j) (t) − 7G(j) (t − 1) + G(j) (t − 2)], 6 · ¸ (−1)j −17G(j) (t + 3) + 162G(j) (t + 2) − 675G(j) (t + 1) + 1420G(j) (t) G,2 Wj,8 (t) = . −675G(j) (t − 1) + 162G(j) (t − 2) − 17G(j) (t − 3) 360

G,2 Wj,4 (t) =

18

JIANZHONG WANG

By the dilation method (setting b = 4 in (45)), we have the following. µ ¶ 1 (j) 8 j (j) = (−1) − G (t) + 2 G (2t) , 3 3 µ ¶ 1 (j) 40 j (j) 256 j (j) G,3 j Wj,6 (t) = (−1) G (t) − 2 G (2t) + 4 G (4t) , 45 45 45 µ ¶ 1 96 j (j) − 2835 G(j) (t) + 2835 2 G (2t) G,3 j Wj,8 (t) = (−1) . 32768 j (j) j (j) − 5232 2835 4 G (4t) + 2835 8 G (8t)

G,3 Wj,4 (t)

j

6. Construction of DA δ-Wavelets. In Section 2, we has shown that each difference formula for computing the j th derivative is corresponding to a DA δ-wavelet, which is the j th derivative of a compactly supported spline of order j. In this section, we discuss how to construct the spline, whose derivative is a DA δ-wavelet of order (j, n). Note that, for a fixed t ∈ R, if we define g(τ ) = f (t + τ ), then g (j) (0) = f (j) (t). Hence, without loss of generality, we only develop the difference formulas for computing f (j) (0) for an arbitrary function f. Let ψ have the form (6), PN i.e., ψ(t) = k=1 dk δ(t − tk ). Then Wψh,j (0) =

N N 1 X 1 X d f (ht ) = dk f (htk ). k k hj hj k=1

(52)

k=1

In numerical differentiation, since the difference formulas with a uniform difference PN step are very popular, we mainly focus on tk = k in (52), i.e., ψ(t) = k=−M dk δ(t− k) and Wψh,j (0) =

N 1 X dk fk , hj

(53)

k=−M

where fk = f (kh). By (8) and (9), we have 1 ˆm (ω) N

=1+

∞ X am,k k=1

k!

(iω)k ,

(54)

1 1 1 where am,1 = 12 m, am,2 = 41 m2 − 12 m, am,3 = 18 m3 − 18 m2 , am,4 = 120 m + 48 m2 − 1 1 1 5 5 1 13 1 1 3 4 2 3 4 5 3 2 18 m + 16 m , am,5 = 48 m + 96 m − 48 m + 32 m , am,6 = 576 m − 96 m − 252 m + 5 5 1 7 7 1 35 7 1 4 5 6 4 3 2 5 6 7 64 m − 64 m + 64 m , am,7 = 1152 m − 192 m − 72 m + 384 m − 128 m + 128 m , and

1 Cˆm (ω)

=1+

∞ X cm,k 2k ω , (2k)!

(55)

k=1

1 1 1 1 1 where cm,1 = 12 m, cm,2 = 120 m + 48 m2 cm,3 = 252 m + 96 m2 + 1 101 7 35 2 3 4 240 m + 8640 m + 576 m + 6912 m . Particularly, by (54), we have

5 3 576 m ,

1 1 1 1 1 = 1 + (iω) + (iω)2 − (iω)4 + (iω)6 + °(ω 7 ). ˆ 2 12 720 30240 N1 (ω)

cm,4 =

WAVELET IN NUMERICAL DIFFERENTIATION

19

Applying (39), we obtain the following AI-splines in S 1 . φ1,1 (t) = N1 (t), P V M = 0, 1 φ1,2 (t) = [N1 (t + 1) + N1 (t)] , P V M = 1, 2 1 ˜ φ1,2 (t) = [3N1 (t) − N1 (t − 1)] , P V M = 1, 2 1 φ1,4 (t) = [−N1 (t + 2) + 7N1 (t + 1) + 7N1 (t) − N1 (t − 1)] , P V M = 3, 12 · ¸ 1 N1 (t + 3) − 8N1 (t + 2) + 37N1 (t + 1) φ1,6 (t) = , P V M = 5. +37N1 (t) − 8N1 (t − 1) + N1 (t − 2) 60 Let ψ1,n = −φ01,n . The wavelet transforms derived from ψ1,n are the following difference formulas. f 0 (0) = f 0 (0) = f 0 (0) = f 0 (0) = f 0 (0) =

f1 − f0 0 + °(h), from ψ1,1 = − (φ1,1 ) , h f1 − f−1 0 + °(h2 ), from ψ1,2 = − (φ1,2 ) , 2h ³ ´0 −f2 + 4f1 − 3f0 + °(h2 ), from ψ˜1,2 = − φ˜1,2 , 2h −f2 + 8f1 − 8f−1 + f−2 0 + °(h4 ), from ψ1,4 = − (φ1,4 ) , 12h f3 − 9f1 + 45f1 − 45f−1 + 9f−2 − f−3 0 + °(h6 ), from ψ1,6 = − (φ1,6 ) . 60h

Similarly, by (55), we have µ ¶ µ ¶ 1 1 2 1 1 4 5 1 6 =1+ ω + ω + ω + + ° (ω 8 ). ˆ 6 2 10 4! 42 6! C2 (ω) 1

Applying (40), we have the following AI-functions in S 2 . φe2,2 (t) = C2 (t), P V M = 1, 1 φe2,4 (t) = [−C2 (t + 1) + 14C2 (t) − C2 (t − 1)] , P V M = 3, 12 · ¸ 1 2C2 (t + 2) − 23C2 (t + 1) φe2,6 (t) = , P V M = 5. 180 +222C2 (t) − 23C2 (t − 1) + 2C2 (t − 2) They derive the following central difference formulas for computing f 00 (0). f1 − 2f0 + f−1 + °(h2 ), h2 −f2 + 16f1 − 30f0 + 16f−1 − f−2 f 00 (0) = + °(h4 ), 12h2 2f3 − 27f2 + 270f1 − 490f0 + 270f−1 − 27f−2 + 2f−3 + °(h6 ). f 00 (0) = 180h2

f 00 (0) =

20

JIANZHONG WANG

The followings are some even AI-splines in S 3 : φ3,1 (t) = N3 (t + 2), P V M = 0, 1 φ3,2 (t) = [N3 (t + 2) + N3 (t + 1)] , P V M = 1, 2 1 φ3,3 (t) = [−N3 (t + 3) + 4N2 (t + 2) − N3 (t + 1)] , P V M = 2, 4 1 φ3,4 (t) = [−N3 (t + 3) + 5N3 (t + 2) + 5N3 (t + 1) − N3 (t)] , P V M = 3, 8 which derive the following difference formulas: −f1 + 3f0 − 3f−1 + f−2 f 000 (0) = + °(h), h3 f2 − 2f1 + 2f−1 − f−2 f 000 (0) = + °(h2 ), 2h3 f2 + f1 − 10f0 + 14f−1 − 7f−2 + f−3 f 000 (0) = + °(h3 ), 4h3 −f3 + 8f2 − 13f1 + 13f−1 − 8f−2 + f−3 + °(h4 ). f 000 (0) = 8h3 Some AI-splines in S 4 and their corresponding difference formulas are the followings. φe4,2 (t) = N4 (t + 2), P V M = 1, 1 φe4,4 (t) = [−N4 (t + 3) + 8N4 (t + 2) − N4 (t + 1)] , 6

P V M = 3,

and f2 − 4f1 + 6f0 − 4f−1 + f−2 + °(h2 ), h4 −f3 + 12f2 − 39f1 + 56f0 − 39f−1 + 12f−2 − f−3 + °(h4 ). f (4) (0) = 6h4 Remark 1. Taking the advantage of the formulation of the δ-wavelet transform in (52) and applying the conditions (10), we can also directly compute the coefficient vector d = (d1, · · · , dN )T in (52) for a DA δ-wavelet of (j, n) order by solving the following linear system t Vj,N d = j!enj , (56) f (4) (0) =

where enj is the j th vector of the standard basis for Rn , and  1 1 ··· 1  t t · · · t 1 2 N  t Vj,N = .. .. ..  . . ··· . n+j−1 n+j−1 tn+j−1 t · · · t 1 2 N

   . 

It is clear that the linear system (56) has a solution if and only if N ≥ n + j, and the solution is unique if and only if N = n + j. This method is particularly useful when the time-steps are not uniform. Finally, we apply the dilation method to construct AI-splines, which are used for building difference formulas. It is well-known that cardinal B-splines satisfy two-scaling equations [4]: ¶ j µ 1 X j Nj (x) = j−1 Nj (2x − k) (57) k 2 k=0

WAVELET IN NUMERICAL DIFFERENTIATION

and C2m (x) =

m µ X

1 22m−1

k=−m

Let φj,n (t) =

X

2m m+k

21

¶ C2m (2x − k).

al Nj (t − l)

l∈L

be an AI-spline in S j with n − 1 PVM. Then the spline 2n+1 1 φ˜j,n+1 (t) = n φn (2t) − n φn (t) 2 −1 2 −1 is in S2j and has n PVM. Applying (57), we have " # ¶ j µ X 1 X n+1 1 j ˜ 2 al Nj (2t − l) − j−1 al φj,n+1 (t) = n Nj (2 (t − l) − k) k 2 −1 2 l∈L k=0   µ ¶ X X 1 j 2n+1 al − 1  Nj (2t − l). = n ak l − 2k 2 −1 2j−1 l∈L

l−j≤2k≤l

Forcing it back to S j by dilation, we have the required spline X 1 bl Nj (t − l), φj,n+1 (t) = φ˜n+1 (t/2) = 2

(58)

l∈L

where

 1 1  n 2 al − j bl = n 2 −1 2

Similarly, let φe2m,2n (t) =

X

X

µ

l−j≤2k≤l

al C2m (t − l),

j l − 2k



¶

ak  .

al = a−l ,

l∈L

be an even AI-spline in S 2m with 2n − 1 PVM. Then X φe2m,2n+2 (t) = dl C2m (t − l)

(59)

l∈L

is an even AI-spline in S 2m with 2n + 1 PVM, where à µ ¶ ! m 1 1 X 2m n dl = n 4 al − m ak . m + l − 2k 4 −1 4 2k−l=m

Example. The B-spline φ1,1 (t) := N1 (t) ∈ S 1 is an AI-spline with 0 PVM. Recursively applying (58), we have 1 3 φ1,2 (t) = N1 (t) − N1 (t − 1) 2 2 and 21 11 1 1 φ1,3 (t) = N1 (t) − N1 (t − 1) + N1 (t − 2) + N1 (t − 3). 12 12 12 12 Similarly, the central B-spline φe2,1 (t) := C2 (t) ∈ S 2 is an even AI-spline with 1 PVM. Applying (59), we have 14 1 1 (60) φe2,4 (t) = − C2 (t − 1) + C2 (t) − C2 (t + 1) 12 12 12

22

JIANZHONG WANG

and

· ¸ 1 2C2 (t + 2) − 23C2 (t + 1) . (61) 180 +222C2 (t) − 23C2 (t − 1) + 2C2 (t − 2) The expressions in (60) and (61) are same as those obtained by using the translate method. φe2,6 (t) =

7. Comparison of the DA-wavelet Transform Formulas and Difference Formulas. In this section, we give an example to show the efficiency of the DAwavelet transform formulas in the computation of the derivatives of functions with G,3 high frequency noise. The AD-wavelet of Gaussian type, ψ(t) = W2,6 (t), is used in the following example for computing the second derivative of the target function f (t) = cos(t), which is interfered by the high frequency noise n(t) = cos mt + sin mt, where m is a large integer. Hence, the observed function is um (t) = cos t + cos mt + sin mt. The DA-wavelet transform formula for the numerical differentiation is Z ∞ µ ¶ 1 t Wψh,2 u(0) = 3 ψ u(t)dt. h −∞ h

(62)

40 1 G(t) − 45 G(2t) + 256 Recall that ψ(t) = φ00 (t), where φ(t) := 45 45 G(4t). By (25), the error is estimated by µ¯ ¯2 ¶ ¯2 1 ¯¯ ˆ ¯ h,2 ¯ ¯ 00 E ¯Wψ u(0) − y (0)¯ ≤ A2f h12 + ψ(mh) (63) ¯ ¯ , 2πh5

where

µ Af =

1 6!

Z

¶° Z ° 1 ° (8) ° 6 |φ(τ )| |τ | dτ °f ° = |φ(τ )| |τ | dτ 720 C R R 6

and

√ · µ ¶ µ ¶¸ (mh)2 2π mh mh ˆ ψ(mh) =− G(mh) − 20G + 64G . 45 2 4 Setting m = 100, 1000, and 10000 respectively, we obtain the optimal h values for these m values are h100 ' 0.234, h1000 ' 0.031, and h10000 ' 0.0035. Applying them in (62), we have Wψ100,2 u100 (0) = −0.99999999971641, Wψ1000,2 u1000 (0) = −0.99999999999984, Wψ10000,2 u10000 (0) = −1.00000000000000. They give very good approximation of f 00 (0) = −1. To give the comparison of the AD-wavelet transform formula (62) and the difference formula 1 (f (t − h) + f (t − h) − 2f (t)) , (64) h2 we set h = 0.5, 0.25, 0.1, 0.05, 0.03, 0.01, 0.005, 0.003, 0.001, and create 4 tables for m = 0, 100, 1000, 10000 respectively. Table 1 shows the numerical results for m = 0. In this case, the function f (t) is not interfered by noise. Then both formulas (62) and (64) provide approximation of f 00 (0). Table 2 shows the numerical results for m = 100. In this case, the function f (t) is interfered by a noise with a relatively low frequency. Then formulas (62) generates

WAVELET IN NUMERICAL DIFFERENTIATION

h 0.5 0.25 0.1 0.05 0.03 0.01 0.005 0.003 0.001 0.0001

23

Difference Formula (64) DA-Wavelet Transform (62) −0.97933950487702 −0.99999511692696 −0.99480250525937 −0.99999992133666 −0.99916694439486 −0.99999999967501 −0.99979168402697 −0.99999999999492 −0.99992500224990 −0.99999999999976 −0.99999166669473 −1.00000000000000 −0.99999791666860 −1.00000000000000 −0.99999925002376 −1.00000000000000 −0.99999991665101 −1.00000000000000 −0.99999999392253 −1.00000000000000 Table 1. The numerical results for m = 0.

h 0.5 0.25 0.1 0.05 0.03 0.01 0.005 0.003 0.001 0.0001

Difference Formula (64) DA-Wavelet Transform (62) −0.00012596112769 × 104 −0.99999511692696 −0.00012763125256 × 104 −0.99999992602106 −0.03688134727597 × 104 −1.06248642676023 −0.05740700433134 × 104 −1.63161333794088 −0.44232054730032 × 104 −1.92950653725905 −0.91949538743039 × 104 −1.99972265810015 −0.97943950466868 × 104 −1.99999511692696 −0.99262246380042 × 104 −1.99999976616601 −0.99926694438652 × 104 −1.99999999967501 −1.00009166669413 × 104 −2.00000000000000 Table 2. The numerical results for m = 1000.

h 0.5 0.25 0.1 0.05 0.03 0.01 0.005 0.003 0.001 0.0001

Difference Formula (64) DA-Wavelet Transform (62) −0.00016050133692 × 105 −0.99999511692696 −0.00025283176736 × 105 −0.99999992133666 −0.00028535392487 × 105 −0.99999999967501 −0.00029026968890 × 105 −0.99999999999492 −0.01880441147474 × 105 −1.00000000000063 −0.36782430573196 × 105 −1.06248642708521 −0.57308025160859 × 105 −1.63161333794596 −4.42221554799349 × 105 −1.92950653725928 −9.19396388263637 × 105 −1.99972265810015 −9.99167944394852 × 105 −1.99999999967501 Table 3. The numerical results for m = 1000.

small errors for h ∈ [0.25, 0.5], and deterioration occurs when h ≤ 0.3. However, the formula (64) cannot provide an approximation of f 00 (0). Table 3 shows the numerical results for m = 1000. In this case, the function f (t) is interfered by a noise with a medium high frequency. Then formulas (62) generates small errors for h ∈ [0.02, 0.1], and deterioration occurs when h ≤ 0.02. However, the formula (64) cannot provide an approximation of f 00 (0).

24

JIANZHONG WANG

h 0.5 0.25 0.1 0.05 0.03 0.01 0.005 0.003 0.001 0.0001

Difference Formula (64) DA-Wavelet Transform (62) 7 −0.00000077419923 × 10 −0.99999511692696 −0.00000086803989 × 107 −0.99999992133666 −0.00000885233517 × 107 −0.99999999967501 −0.00015080792104 × 107 −0.99999999999492 −0.00022723257456 × 107 −0.99999999999976 −0.00027546225459 × 107 −1.00000000000000 −0.00028037177185 × 107 −1.00000000000000 −0.01879451222465 × 107 −1.00000000000087 −0.36781440581528 × 107 −1.06248642708521 −9.19395398263720 × 107 −1.99972265810015 Table 4. The numerical results for m = 10000.

Table 4 shows the numerical results for m = 10000. In this case, the function f (t) is interfered by a noise with a high frequency. Then formulas (62) generates small errors for h ∈ [0.002, 0.1], and deterioration occurs when h ≤ 0.002. However, the formula (64) cannot provide an approximation of f 00 (0). Acknowledgement. The author would like to thank the referees for their helpful comments. REFERENCES [1] R. S. Anderssen and M.Hegland, For numerical differentiation, dimensionality can be a blessing, Math. Comp., 277 (1999), 1121–1141. [2] P. L. Butzer and R. J. Nessel, Fourier Analysis and Approximation, Vol. 1, Birkh¨ auser, 1971. [3] C. K. Chui, An Introduction to Wavelets, Academic Press, Boston, 1992. [4] C. K. Chui and J. Z. Wang, A cardinal spline approach to wavelets, Proc. Amer. Math. Soc., 113 (1991), 785–793. [5] C., K. Chui and J. Z. Wang, On compactly supported wavelet and a duality principle, Trans. Amer. Math. Soc., 330 (1992), 903–915. [6] C. K. Chui and J. Z. Wang, A general framework of compactly supported splines and wavelets, J. Approx. Theory, 71 (1992), 263–304. [7] C. K. Chui and J. Z. Wang, An analysis of cardinal spline-wavelets, J. Approx. Theory, 72 (1993), 54–68. [8] C. de Boor, A Practical Guide to Splines, Springer-Verlag, New York, 1978. [9] I. Daubechies, Ten Lectures on Wavelets, SIAM Publications, Philadelphia, 1992. [10] C. de Boor and G. J. Fix, Spline approximation by quasi-interpolants, J. Approx. Theory, 8 (1973), 19–45. [11] M. Hanke and O. Scherzer, Inverse problem light: numerical differentiation, Amer. Math. Monthly, 6 (2001), 512–521. [12] M. Hasson, Wavelet-based filters for accurate computation of derivatives, Math. Comput., 75 (2006), 259–280. [13] M. Floater, Error formulas for divided difference expansions and numerical differentiation, J. Approx. Theory, 122 (2003), 1–9. andez and G. Weiss, A First Course on Wavelets, CRC Press, New York, NY, 1996. [14] E. Hern´ [15] P. Hoffman and K. C. Reddy, Numerical differentiation by high order interpolation, SIAM J. Sci. Statist. Comput., 6 (1987), 979–987. [16] R-Q. Jia, A characterization of the approximation order of translation invariant spaces of functions, Proc. Amer. Math. Soc., 111 (1991), 61–70. [17] R-Q. Jia, Shift-invariant spaces on the real line, Proc. Amer. Math. Soc., 125 (1997), 785–793. [18] I. Knowles and R. Wallance, A variational method for numerical differentiation, Numer. Math., 1 (1995), 91–110.

WAVELET IN NUMERICAL DIFFERENTIATION

25

[19] A. Lupac and D. Mache, On the numerical differentiation, Rev. Anal. Numer. Theory Approx., 26 (1997), 109–115. [20] Y. Meyer, Ondelettes et Op´ erateurs I: Ondelettes, Hermann, Paris, France, 1990. [21] Y. Meyer, Wavelets: Algorithms and Applications, (translated and revised by R. D. Ryan,) SIAM Publications, Philadelphia, 1993. [22] D. A. Murio, C. E. Meja, and S. Zhan, Discrete mollification and automatic numerical differentiation, Comput. Math. Appl., 5 (1998), 1–16. [23] A. G. Ramm and A. Smirnova, On stable numerical differentiation, Math. Comput., 70 (2001), 1131-1153. [24] I. J. Scheonberg, On spline functions, in “Inequalities”, Shisha Ed., Academic Press, New York, (1967) 255–291. [25] I. J. Scheonberg, Cardinal Spline Interpolation, CBMS-NSF Series in Applied Math. #12, SIAM Pub., Philadelphia, 1973. [26] L. L. Schumaker, Spline Functions: Basic Theory, Wiley-Interscience, New York, 1981. [27] M. R. Skrzipek, Generalized associated polynomials and their application in numerical differentiation and quadrature, Calcolo, 40 (2003), 131–147. [28] O. L. Vinogradov and V. V. Zhuk, Sharp estimates for errors of numerical differentiation type formulas on trigonometric polynomials function theory and partial differential equations, J. Math. Sci., New York, 105 (2003), 2347–2376. [29] J. Z. Wang, On spline wavelet, in “Wavelets and Splines-Athens 2005”, G. Chen and M-J Lai Ed., Nashboro Press, Nashville, TN, (2006), 456–483. [30] Y. B. Wang, X. Z. Jia, and J. Cheng, A numerical differentiation method and its application to reconstruction of discontinuity, Inverse Problems, 6 (2002), 1461–1476. [31] K. Yosida, Functional Analysis, Fifth Edition, Springer-Verlag, New York, 1978. E-mail address: [email protected]