Preprint of the paper that appeared in: IEEE Transactions on Information Theory. 1994 vol. 40, nr 5, pp 1490-1503
On restoring band-limited signals Miroslaw Pawlak?
?
and
Ewaryst Rafajlowicz??
University of Manitoba, Department of Electrical & Computer Engineering, Winnipeg, Manitoba, Canada, R3T 2N2
??
University of Manitoba, Department of Electrical & Computer Engineering, Winnipeg, Manitoba, Canada, R3T 2N2 on leave from Institute of Engineering Cybernetics, Technical University of Wroclaw, Wybrze˙ze Wyspia´ nskiego 27, 50 370 Wroclaw, Poland
?
This work was supported by the Humboldt Foundation and NSERC grant A8131 ??
This work was partly supported by the grant of Council for Scientific Research of Poland and NSERC grant A8131
Address for Correspondence: M. Pawlak Department of Electrical & Computer Engineering,
The University of Manitoba, Winnipeg, Manitoba, Canada, R3T 2N2 E-mail:
[email protected]
2
Abstract The problem of reconstruction of band-limited signals from discrete and noisy data is studied. The reconstruction schemes employing cardinal expansions are proposed and their asymptotical properties are examined. In particular, the conditions for the convergence of the mean integrated squared errors are found and the rate of convergence is evaluated. The main difference between the proposed reconstruction scheme and the classical one is in treating differently the sampling rate and the reconstruction rate. This distinction is necessary to ensure consistency of the reconstruction scheme in the presence of noise.
Keywords: band-limited functions, cardinal expansion, orthogonal series, noisy data, reconstruction, convergence, nonparametric regression.
1. Introduction and Preliminaries It is common in many problems of communication theory to assume that a signal f (t) has a bounded spectrum, i.e., that its Fourier transform F (ω) vanishes outside of a finite interval. Any signal with such a property is referred to as band-limited. It is well-known, due originally to Whittaker (see [5], [14], [15], for the history and review of this result) that f (t) can be recovered from discrete values f (k Ωπ ), k = 0, ±1, ±2, . . ., since f (t) =
∞ X
π π f (k )sinc Ω t − k Ω Ω k=−∞
,
(1.1)
where sinc(x) = sin(x)/x, and Ω is the bandwidth of f (t), i.e., f (t) =
1 ZΩ F (ω) exp(iωt) dω. 2π −Ω
(1.2)
Here F is the square integrable function on (−Ω, Ω). In the sequel we shall denote the class of functions satisfying (1.2) by BL(Ω) . The expansion in (1.1) is often referred to as the cardinal series or the Whittaker-Shannon interpolation scheme. In practical applications of (1.1) one has to truncate (1.1) and, moreover, often replace f (k Ωπ ) by its noisy observations yk = f (k Ωπ ) + zk , where zk is a zero mean noise process with var zk = σ 2 < ∞. This would lead to the following reconstruction scheme π fn (t) = yk sinc Ω t − k Ω |k|≤n
X
,
(1.3)
where 2n + 1 is the number of observations taken into account. The fundamental question, which arises is whether fn (t) can be a consistent estimate of f (t), i.e., whether %(fn , f ) → 0 as n → ∞, in a certain probabilistic sense, for some distance measure %. Since f is assumed to be square integrable then the natural measure between fn (t) and f (t) is the mean integrated squre error M ISE(fn ) = E
Z ∞ −∞
(fn (t) − f (t))2 dt,
which represents the energy of the error signal fn (t) − f (t). 1
(1.4)
Using Corollary 3 in Section 3 we can get, however, that σ 2 π(2n + 1) π X 2 π M ISE(fn ) = + f k , Ω Ω |k|>n Ω
(1.5)
which tends to infinity as n → ∞. This result is caused by the presence of the noise in the data. The problem of the reduction of the noise level in the representation (1.1) has been discussed in the literature [15], and it has been recommended to employ the oversampling version of (1.1), i.e., f (t) =
∞ X
f (kh)sinc
k=−∞
where h ≤ of f (t)
π Ω
π (t − kh) , h
(1.6)
is the sampling rate. This would yield the following estimate fnh (t) =
π yk sinc( (t − kh)), h |k|≤n X
(1.7)
where yk = f (kh) + zk . The MISE of this estimate, see Corollary 3 in Section 3, is given by σ 2 h(2n + 1) + h
X
f 2 (kh).
(1.8)
|k|>n
The second term, see Lemma 1 in Section 3, is equal to h
X
f 2 (nh) −
Z nh
f 2 (t) dt
−nh
|k|≤n
and it tends to zero if nh → ∞. On the other hand, the first term in (1.8) converges to zero as nh → 0, i.e., the consistency result is again not possible. We refer also to [3] and [15] for a similar discussion of this issue. The aim of this paper is to construct some consistent estimates of f (t) stemming from the cardinal expansion (1.6). This task is carried out by using the multirate idea, i.e., we use different rates corresponding to the signal sampling and recovering process. The conditions for consistency of our estimates are established and their rate of convergence is evaluated. Some numerical examples are also given. 2
Throughout the paper we assume that the data are generated from the following model j = 0, ±1, ±2, . . .
yj = f (jτ ) + zj ,
(1.9)
where τ > 0 is the sampling rate, and zj is uncorrelated noise process with E zj = 0, var(zj ) = σ 2 < ∞. The problem of recovering a nonparametric function in the model (1.9) has been an active area of research in recent years, see, e.g., [9], [12]. Nonparametric curve estimation techniques like kernel, spline, orthogonal series estimates have been extensively studied. Nevertheless, unlike in our case (the band-limited functions are not time-limited) the estimated function has been assumed to be defined on a finite interval. A related problem of estimating band-limited probability density functions has been studied in [8] and [13].
2. The Estimators It has been observed in [11] (for h = π/Ω , see also [20] for a general case) that a cardinal series in (1.6) can be written as the orthogonal expansion f (t) =
∞ X
ck sk (t),
(2.1)
k=−∞
the orthogonal and complete syswhere sk (t) = sinc( πh (t − kh)) defines R∞ tem in BL(Ω) if h ≤ π/Ω , i.e., −∞ sk (t)s` (t) dt is equal to 0 or h as k 6= ` orR k = `, respectively. The Fourier coefficient ck is defined as ∞ ck = h−1 −∞ f (t)sk (t) dt. It is also clear that ck = f (kh). Our estimation technique rely on (2.1) with ck replaced by the piecewise R∞ constant approximation of the integral −∞ f (t)sk (t)dt and with the truncation of the series in (2.1). Hence, let fˆ(t) =
X
cˆk sk (t),
(2.2)
|k|≤N
where cˆk = h−1
X
yj
|j|≤n
and Aj =
2j−1 τ, 2j+1 τ 2 2
. 3
Z Aj
sk (z) dz
(2.3)
The estimate defined in (2.2), (2.3) requires a numerical evaluation of the integral
R (j+ 12 )τ
s (z)dz. This can be (j− 12 )τ k 1 [(j − 2 )τ, (j + 12 )τ ] and then
carried out by a fine partition of
by applying some known numerthe interval ical quadrature techniques. The simplest strategy is to use a middle point R (j+ 1 )τ approximation, i.e., to replace (j− 12)τ sk (z)dz by τ sk (jτ ). 2 This yields the following estimate of f (t) f˜(t) =
X
c˜k sk (t) ,
(2.4)
|k|≤N
c˜k =
τ X yj sk (jτ ) . h |j|≤n
(2.5)
In (2.2) and (2.4) the parameter N defines number of terms in the expansion which are taken into account and 2n + 1 is the sample size. It is also worth to noting that the sampling rate τ is different than the reconstruction rate h. The latter is assumed to be constant and not greater than π/Ω. On the other hand, τ depends on n in such a way that nτn → ∞ and τn → 0 as n → ∞ . In the next section we present conditions on N and τ assuring that MISE(fˆ) and MISE(f˜) tend to zero as n → ∞ . The rate at which MISE(fˆ) and MISE(f˜) converge to zero is also evaluated. It is shown that both estimates attain the same asymptotic rate of convergence. Nevertheless, the estimate fˆ(t) yields a tighter upper bound for the MISE. Note finally that techniques proposed in (2.2) and (2.4) are in the form of orthogonal series estimators, see [7], [9], [15] and [16] for related results in the context of estimation of regression functions.
3. Consistency For further considerations we shall need. Lemma 1 : Let ρ ∈ BL(Ω). If τ ≤ lim
Z nτ
n→∞ −nτ
π Ω
and τ = τn with nτn → ∞, then
ρ2 (u) du − τ
X |j|≤n
4
ρ2 (jτ )
= 0.
The proof of this identity easily results from the orthogonality of {sk (t)} (see also [20]), which also implies M ISE(fˆ) = h
E(ˆ ck − ck )2 + h
X |k|≤N
c2k .
X
(3.1)
|k|>N
The first term in this identity can be further decomposed as h
X |k|≤N
(Eˆ ck − ck )2 .
X
var(ˆ ck ) + h
(3.2)
|k|≤N
Analogous formulas hold for f˜(t) with cˆk replaced by c˜k . The decomposition of the error given in (3.1) and (3.2) allows us to examine separately the estimates variance and bias. This will be carried out in a series of lemmas. A. Variance: Let us denote VAR(fˆ) = h
X
var(ˆ ck ) ,
|k|≤N
and VAR(f˜) in the analogous way. Lemma 2 : Let h ≤ Ωπ . Then for n ≥ 0 VAR(fˆ) ≤ σ 2 (2N + 1)τ. Proof. By the Cauchy-Schwartz inequality we have 2 −2
var(ˆ ck ) = σ h
X |j|≤n Z ∞
≤ σ 2 h−2 τ
−∞
Z Aj
!2
sk (z) dz
≤
s2k (z) dz = σ 2 h−1 τ.
This confirms our claim. • For the sequences an , bn we denote an ' bn if limn→∞ an /bn = 1. The behaviour of VAR(f˜) gives the following lemma. Lemma 3 : Let τ ≤ h and h ≤ π/Ω. If nτ → ∞ then VAR(f˜) ' σ 2 (2N + 1)τ. 5
Proof. It is plain that var(˜ ck ) = σ 2 = σ2
τ2 X 2 s (jτ ) = h2 |j|≤n k τ X τ τ sinc2 jπ − πk π . πh |j|≤n h h
Owing to Lemma 1 we have X |j|≤n
2
sinc
Z nπτ /h τ τ jπ − πk π ' sinc2 (u − πh) du , h h −nπτ /h
∞ where the last integral tends to −∞ sinc2 (u) du = π, if nτ → ∞ and τ ≤ h (sinc(t) is band-limited with the bandwidth equals to 1). • It is also worth noting that VAR(fˆ) ' σ 2 (2N +1)τ under the assumptions as in Lemma 3. Lemma 3 gives the asymptotic expression for VAR(f˜), i.e., as nτ → ∞. A bound valid for a finite n is given in the following corollary.
R
Corollary 1 : Let τ ≤ h ≤ π/Ω. If nτ > hN , then "
#
4 3τ 2 h VAR(f˜) ≤ σ 2 (2N + 1)τ + − 2 . π h π nτ − N h h Let us also observe that π4 + 3τh − π22 nτ −N ≤ π4 + 3. h The proof of this inequality is based on the detail analysis of the leading P term in var(˜ ck ), i.e., |j|≤n sinc2 (jτ πh − πk). This is deferred to the appendix. The next paragraph concerns the bias component of the error.
B. Bias: Owing to (3.1) and (3.2) BIAS(fˆ) = h
X
(Eˆ ck − ck ) 2 + h
|k|≤N
X
c2k .
(3.3)
|k|>N
Let us note that the first term is due to the piecewise approximation of the integral in ck , while the second term in (3.3) is caused by the truncation of the expansion in (2.1). BIAS(f˜) is defined in the analogous way.
6
Lemma 4 : Let f ∈ BL(Ω) . If N < n hτ , then BIAS(fˆ) ≤ where θn = R∞ 2 −∞ f (t) dt.
X 4θn 2N + 1 Ω3 E0 3 τ (2n + 1)(2N + 1) + 2 nτ +h c2k 6π π ( h − N) |k|>N
R R(n,τ )
(3.4)
f 2 (t) dt, and R(n, τ ) = {t : |t| > (n + 1/2)τ }, while E0 =
Proof. It is plain that we only have to evaluate the first term on the right-hand side of (3.3). Let us denote B(fˆ) = h
(Eˆ ck − ck )2 =
X |k|≤N
= h−1
|k|≤N
Let us decompose
R∞
−∞
X Z |j|≤n
Aj
X Z |j|≤n
X
X
Aj
f (jτ )
|j|≤n
Z
sk (t) dt −
Aj
Z ∞ −∞
2
f (t)sk (t) dt .
f (t)sk (t) dt as X Z
f (t)sk (t) dt +
|j|>n
Aj
Z
f (t)sk (t) dt +
R(n,τ )
f (t)sk (t) dt =
f (t)sk (t) dt.
Then, clearly, B(fˆ) is not greater than 2
2 X X Z (f (jτ ) − f (t))sk (t) dt + h |k|≤N |j|≤n Aj !2 2 X Z + f (t)sk (t) dt = B1 + B2 , h |k|≤N R(n,τ )
(3.5)
say. Let us first examine B1 . Using the Cauchy-Schwartz inequality twice, first for integrals and then for sums we obtain. Z 2 X X Z 2 B1 ≤ (f (jτ ) − f (t)) dt s2k (t) dt . h |k|≤N |j|≤n Aj R(n,τ )
7
It is known [17] that if f ∈ BL(Ω) , then f satisfies also the Lipschitz condition, i.e., |f (t) − f (t0 )| ≤ L|t − t0 |, (3.6) where L2 = E0 Ω3 /(3π). R Thus, noting that R(n,τ ) s2k dt ≤ h and that for t ∈ Aj , |jτ − t| ≤ τ /2, we obtain L2 τ 3 (2n + 1)(2N + 1) B1 ≤ . (3.7) 2 Concerning the term B2 in (3.5), we see that due to the Cauchy-Schwartz inequality X Z 2 B2 ≤ θn s2 (t) dt , h |k|≤N R(n,τ ) k where θn = R(n,τ ) f 2 (t) dt. In the end let us consider R
X Z |k|≤N
R(n,τ )
s2k (t)
( ) Z −βk sin2 (z) h X Z ∞ sin2 (z) dz + dz , (3.8) dt = π |k|≤N γk z2 z2 −∞
where γk = πh nτ − kπ, βk = πh nτ + kπ. It is clear that 0 < γN ≤ γr , r = N − 1, . . . , −N , as long as N < hτ n. Hence, X Z ∞ |k|≤N
γk
Z ∞ 2N + 1 2N + 1 sin2 (z) sin2 (z) . dz ≤ (2N + 1) dz ≤ = nτ 2 2 z z γN γN π h −N
Since the second integral in (3.8) can be treated in the same way we can conclude that 4θn 2N + 1 . (3.9) B2 ≤ 2 nτ π −N h
By this and taking (3.5) and (3.7) into account we can conclude the proof of Lemma 4. • Lemma 5 : Let the assumptions of Lemma 4 hold. Then Ω3 E0 3 τ (2n + 1)(2N + 1) + 3π X π 2 ηn τ 3 4θn 2N + 1 +h + (2n + 1)(2N + 1) + 2 nτ c2k , 3 h π −N |k|>N BIAS(f˜) ≤
h
8
(3.10)
where ηn = τ
P
|j|≤n
f 2 (jτ ), and θn is defined as in Lemma 4.
Proof. Similarly as in the proof of Lemma 4 let B(f˜) = h
X
(E˜ ck − ck )2 .
|k|≤N
Then, proceeding as in the proof of Lemma 4 we have
#2
"
Z 2 X X ˜ f (jτ )sk (jτ )τ − B(f ) ≤ f (z)sk (z) dz + h |k|≤N |j|≤n Aj !2 2 X Z + f (z)sk (z) dz . h |k|≤N R(n,τ )
(3.11)
The second term has already been examined and it has the bound given in (3.9). On the other hand the first term on the right-hand side of (3.11) does not exceed
#2
"
Z 4 X X sk (z) dz + f (jτ )sk (jτ )τ − f (jτ ) h |k|≤N |j|≤n Aj
+
"
4 f (jτ ) h |k|≤N |j|≤n X
X
Z Aj
sk (z) dz −
#2
Z Aj
f (z)sk (z) dz =
= C + 2B1 , say. The term B1 has the bound given in (3.7). R P P Since |j|≤n f (jτ )sk (jτ )τ = |j|≤n Aj f (jτ )sk (jτ ) dt, then using arguments like in the proof of Lemma 4 we can find that C is not greater than X X Z 4 ηn (sk (jτ ) − sk (t))2 dt , h |k|≤N |j|≤n Aj
where ηn = τ |j|≤n f 2 (jτ ). Note that sk (t) = sinc( πh t − kh) is the bandlimited function with the bandwith πh and its energy is equal to h. Thus, P
sk (t) satisfies (3.6) with L2 = C ≤
1 3
π 2 ηn 3
2 π h
. Using this we can easily conclude that
3
τ h
(2N + 1)(2n + 1) . 9
The proof of Lemma 5 has been completed. • The aforementioned results reveal that the estimates f˜(t) and fˆ(t) have different bounds for the bias term (note that VAR(fˆ)/VAR(f˜) → 1 as nτ → ∞). In fact, by a quick inspection of (3.4) and (3.10) one can observe that the bound for BIAS(f˜) is greater than one for BIAS(fˆ). Nevertheless, the obtained upper bounds need not be tight. To get further insight into the relative performance of fˆ(t) and f˜(t) let us consider the asymptotic behaviour of E(ˆ ck − ck ) − E(˜ ck − ck ) or equivalently E(ˆ ck − c˜k ). This is exhibited in the following corollary. Corollary 2 : Let τ ≤ h ≤ π/Ω then E(ˆ ck − c˜k ) '
τ 2 (2) f (kh) 24
as τ → 0, nτ → ∞. The proof of this result is in the appendix. ¿From the foregoing, it follows, in particular, that if f (2) (kh) < 0 then E(ˆ ck − ck ) < E(˜ ck − ck ), at least asymptotically, i.e., when nτ → ∞, τ → 0. The proof of Corollary 2 also reveals that E(ˆ ck − c˜k ) '
∞ X s=1
(τ /2)2s (2s) f (kh), (2s + 1)!
(3.12)
as n τ → ∞. The above result shows the difference between the estimates of the Fourier coefficients at a single point t = kh. In order to obtain an average behaviour of the difference E(ˆ ck − c˜k ) let us consider the arithmetic mean µ(T ) =
X 1 E(ˆ ck − c˜k ), (2T + 1) |k| ≤ T
for T ≥ 1. Then by the fact that f (2s) (t) is continuous and (3.12) we can get µ(T ) '
∞ X s=1
(τ /2)2s (2s) f (θ), (2s + 1)!
as n τ → ∞, where θ ∈ [−T h, T h]. 10
Furthermore, by virtue of the Berntein’s theorem [2, p. 206], [17], |f (2s) (t)| ≤ Ω2s M , for all f ∈ BL(Ω), M = (E0 Ω/π)1/2 , we can also obtain that |µ(T )| ≤ M ξ(Ω τ /2), where ξ(t) = ξ(x).
et −e−t 2t
− 1. The latter results from the fact that
x2s s=1 (2s+1)!
P∞
=
C. MISE: Owing to (3.1), (3.2) and Lemma 2, 3, 4, 5 one can easily find the explicit bounds for the M ISE(fˆ) and M ISE(f˜) . Furthermore, the conditions for the convergence M ISE(fˆ), M ISE(f˜) → 0 as n → ∞ can be easily drawn from these results. In fact, let us note that the term θn appearing in (3.4) and (3.10) tends to zero as nτ → ∞. On the other hand ηn in (3.10) goes to E0 as nτ → ∞ due to Lemma 1. The same lemma P also yields h |k|>N c2k → 0 as hN → ∞. All these considerations yield the following theorems. Theorem 1 Let f ∈ BL(Ω) . Let h be constant and let h ≤ π/Ω. Let τ (3.13) N < n h If
N → ∞
Nτ
→ 0 (3.14)
3
nτ → ∞
τ Nn
→ 0
then M ISE(fˆ) → 0
as
n → ∞.
Theorem 2 : Let all the conditions of Theorem 1 hold. Let additionally τ ≤ h then M ISE(f˜) → 0 as n → ∞. Corollary 3 : The techniques used in proving Theorem 1 and Theorem 2 easily yield the formulas in (1.5) and (1.8). In fact, one has to only apply Lemma 1 to get (1.5) and (1.8). If one selects τ = an−α , N = bnβ , α, β, a, b > 0, then the conditions in (3.14) of Theorem 1 are satisfied if α < 1, β < α, 1 + β < 3α. The condition in (3.13) holds if α + β ≤ 1 with a > bh. Figure 1 depicts the convergence region with respect to α and β. Figure 1: Insert about here 11
4. Rate of Convergence The conditions required in Theorem 1, 2 put some restrictions on the parameters τ and N . They suggest that N should increase with n but if too many terms are used in our estimates the convergence property can fail. Similarly, τ should decrease with n, but also not too fast. Moreover, there is an apparent relationship between τ and N . To know how τ and N should be selected, at least asymptoticaly, one has to evaluate the rate of convergence at which M ISE(fˆ), M ISE(f˜) tend to zero as n → ∞. Owing to Lemma 3 and Lemma 4 we have 12θn 3Ω3 E0 3 τ nN + 2 M ISE(fˆ) ≤ 3σ 2 N τ + 2π π
−1
nτ −1 Nh
+h
X
c2k (4.1)
|k|>N
On the other hand, Corollary 1 and Lemma 5 yield Ω3 π 2 + 3 τ 3 nN + π h !
M ISE(f˜) ≤ 12.9σ N τ + 3E0 2
12θn + 2 π
−1
nτ −1 Nh
+h
X
c2k ,
(4.2)
|k|>N
where θn = R(n,τ ) f 2 (t) dt. It is plain that both estimates will have the same rate of convergence, they can have only different asymptotic constants. Therefore, without loss of generality, let us consider the bound in (4.1). To calculate the rate of convergence we need to examine the terms θn and P h |k|>N c2k . The behaviour of both terms are related to the rate at which |f (t)| → 0 as |t| → ∞. It should be noted that if f ∈ BL(Ω) then f (t) → 0 as |t| → ∞, see [2, p. 98]. To get specific rates we shall consider a number of assumptions concerning f (t) or its Fourier transform F (ω). R
Assumption 1 : Let f ∈ BL(Ω) and let F (ω) be a function of bounded variation on [−Ω, Ω] with F (±Ω) = 0. Using now (1.2) and integration by parts we easily get that 1 Z Ω iωt f (t) = − e dF (ω), 2πit −Ω 12
for t 6= 0 .
This yields
v0 −1 |t| , |t| ≥ > 0 , (4.3) 2π where v0 is the total variation of F (ω) on [−Ω, Ω]. Owing to (4.3) one can easily find that v0 2 θn ≤ 2 (nτ )−1 . (4.4) 2π Since ck = f (kh) therefore we have |f (t)| ≤
X
h
c2k ≤
|k|>N
v0 2π
2
h−1
X
k −2 ≤ 2
|k|>N
v0 2π
2
(hN )−1 .
(4.5)
R π/h
1 ikhω It is also worth noting that ck = f (kh) = 2π F (ω)dω for h ≤ π/Ω. −π/h e −1 Hence, ck = h Fk , where Fk is the k-th Fourier coefficient of the expansion of F (ω) into the Fourier series on the the interval (− πh , πh ). This observation P would allow us to examine the term h |k|>N c2k using well-known relationships between the smoothness of F (ω) and its Fourier coefficients, see, e.g., [1]. Combining (4.1) with (4.4) and (4.5) we obtain
3Ω3 E0 3 τ nN + M ISE(fˆ) ≤ 3σ 2 N τ + 2π 2 −1 1 v0 2 v0 nτ (nτ )−1 + +6 2 −1 (hN )−1 .(4.6) π Nh 2 π It is clear that there are optimal values of τ and N minimizing the right hand side of (4.6). The direct minimization of (4.6), however, leads to complicated nonlinear algebraic equations with respect to τ and N . Nevertheless, let τ = an−α , a > 0 and due to Theorem 1 (see also Figure 1), 1/3 < α < 1. This yields the bound for M ISE(fˆ) of the form 3σ 2 a
N 3Ω3 E0 3 N + a 3α−1 + nα 2π n !−1 2 v0 an1−α 1 1 v0 2 1 + 6 2 −1 + , π hN n1−α 2 π Nh
i.e., it can be written as c1
N N 1 1 + c + c + c = γn , 2 3 4 nα n3α−1 n1−α N 13
say,
(4.7)
where positive constants c1 , c2 , c3 , c4 can be written explicitly. Hence, if 1/2 ≤ α < 1 then γn ≤ (c1 + c2 )
N 1 1 . + c + c 3 4 nα n1−α N
This is minimized by N = (c4 /(c1 + c2 ))1/2 n−α/2 , yielding γn ≤ 2 (c4 (c1 + c2 ))1/2 n−α/2 + c3 n−(1−α) . If, furthermore, 1/2 ≤ α ≤ 2/3 then
γn ≤ 2 (c4 (c1 + c2 ))1/2 + c3 n−α/2 . Continuing such analysis to the cases 2/3 ≤ α < 1 and 1/3 < α ≤ 1/2 we can conclude that the value of α giving the best possible rate of convergence is equal to 2/3. Hence, one should select τ = an−2/3 with N = bn1/3 yielding the M ISE(fˆ) of order O(n−1/3 ). Clearly, in order to satisfy the condition n > hτ N we need to have a > bh. Since the similar analysis can be carried out for the bound in (4.2) we have obtained the following result. Theorem 3 : Let Assumption 1 be in force. If τ = an−2/3 , N = bn1/3 , and a > bh then M ISE(fˆ) = O(n−1/3 ) and M ISE(f˜) = O(n−1/3 ). It is clear that the constants a, b and the asymptotic constants in O(n−1/3 ) are different for the estimates fˆ and f˜. The optimal exponents α = 2/3 and β = 1/3 are located on the boundary of the convergence region shown in Figure 1. The conditions in Assumption 1 imply that |f (t)| behaves like 1/|t| for large t. To see how even faster decreasing tails of f (t) influence on the rate at which M ISE(fˆ) and M ISE(f˜) tend to zero let us assume the following. Assumption 2 : Let f ∈ BL(Ω) . Let the r-th derivative F (r) be continuous at ±Ω, and let F (r−1) be absolutely continuous on [−Ω, Ω]. Suppose that F (r) is of the bounded variation on [−Ω, Ω]. 14
Assumption 1, roughly speaking, corresponds to Assumption 2 for the case r = 0. The conditions like in Assumption 1 and Assumption 2 have been considered in [4] in the context calculating the pointwise truncation error in the cardinal expansion (1.1). Under Assumption 2 it has been proved in [4] vr that |f (t)| ≤ 2π |t|−(r+1) , |t| ≥ , > 0, where vr is the total variation of F (r) on [−Ω, Ω]. By this we can easily find that vr θn ≤ 2π
2
2 (nτ )−(2r+1) (2r + 1)
and h
X
c2k ≤ 2
|k|>N
vr 2π
2
(hN )−(2r+1) .
Hence, plugging these bounds into (4.1) and (4.2) and then proceeding as in the proof of Theorem 3 we can obtain Theorem 4 : Let Assumption 2 hold. If τ = an−2(r+1)/(2r+3) , N = bn1/(2r+3) , and a > bh, then M ISE(fˆ) = O(n−(2r+1)/(2r+3) ) and M ISE(f˜) = O(n−(2r+1)/(2r+3) ). It is now clear that if f (t) tends faster to zero as |t| → ∞ then both τ and N should be selected smaller. For example, for r = 1 we have τ = an−4/5 , N = bn1/5 , while for r = 2, τ = an−6/7 , N = bn1/7 . The corresponding order of MISE is O(n−4/5 ) and O(n−5/7 ). An extreme case occurs when the tails of f (t) decay exponentially fast. That is, let us assume that. Assumption 3 : Let f ∈ BL(Ω) and α
|f (t)| ≤ M e−a|t| ,
|t| > ε,
where M, a, ε > 0 and 0 < α < 1. As an example of a function satisfying this assumption let us take F (ω) = e−(1−ω 15
2 )−1
1[−1,1] (ω),
where 1A (ω) stands for the indicator function of A. An explicit formula for f (t) (the inverse of F (ω)) is difficult to obtain. It is known, however, see √ − t [16], that |f (t)| = O(e ), t ≥ ε, some positive ε. Under Assumption 3 it is straightforward to find that
α
θn = O e−c1 (nτ )
and
h
X
α
c2k = O e−c2 (nτ )
,
|k|≤N
c1 , c2 positive constants. Then, using (4.1), (4.2) and after a simple algebra we can get the following result. 1/α
Theorem 5 : Let Assumption 3 be satisfied. If τ = a ln n n , N = ln1/α n and a > bh, then ! 2/α ln n M ISE(fˆ) = O n and
ln2/α n ˜ M ISE(f ) = O n
!
Hence, in this case almost parametric rate of convergence O(n−1 ) can be obtained.
5. Numerical Example Our aim in this section is to present results of simulation studies, directed to get some insight into the behaviour of the estimates f˜ and fˆ for a finite number of samples. As a signal hidden in the noise the following function was chosen √
cos(t/2) f (t) = 4π 2π π 2 − t2
!2
.
The same testing signal was used in [4], where it was shown that f ∈ BL(Ω) with Ω = 1 and it satisfies Assumption 2 with r = 2. Thus, the asymptotic results given in Theorem 4 would suggest that the sampling rate τ and the truncation parameter N should be selected of order n−6/7 and n1/7 , respectively. This yields the mean integrated square error of order O(n−5/7 ) for 16
both estimates. In the following simple simulation studies we want to shed some light on these asymptotic results. The following methodology of simulations was used: (j)
(j)
1. Samples y` = f (τ `) + ` were generated for ` = ±1, ±2, . . . , ±n, and (j) repeated for various realizations of random errors ` , j = 1, 2, . . . , M . 2. Repeated samples (M = 30) were used to calculate estimates f˜(j) , fˆ(j) . During these calculations parameters (N, τ, n, h) were kept constant. Thus, the whole variability of f˜(j) and fˆ(j) , j = 1, 2, . . . , M results from random errors. 3. Then, an empirical counterpart of MISE, further called EMISE, was calculated according to the formula: EMISE = τ ((2n + 1)M )−1
M X h X
i2
f (τ `) − f˜(j) (τ `)
j=1 |`|≤n
and an analogous formula for fˆ. Measurement random errors were generated from the Gaussian distribution with zero mean and variance 0.01. The sample size 2n + 1, n = 100, was used and h = 1 was assumed. Figures 2, 3, 4, 5 reveal some differences between the estimates fˆ and f˜. The first estimate is more sensitive to the selection of τ than the second one. On the other hand, f˜ is more sensitive to the selection of N . The error for fˆ and f˜ has a global minimum at (τ = 6.5, N = 4) and (τ = 0.65, N = 5), respectively. Note that this agrees with the restrictions imposed on h and τ in Theorem 1 and Theorem 2. Table 1 gives numerical values of the error for various τ and N . It is seen that, within the range 0 < τ ≤ h, fˆ can outperform f˜, i.e., M ISE(fˆ) < M ISE(f˜). Nevertheless, as it has already been indicated in Section 1 f˜ is easier to calculate than fˆ. In summary one can conclude that the estimates are not too sensitive to the choice of the parameters τ, N . This seems to be a desirable property since tuning algorithm’s parameters to given data can be relatively easy.
6. Concluding Remarks In this paper we have proposed two algorithms for recovering a band-limited signal observed under noise. Assuming that the signal is a square integrable 17
function the sufficient conditions for the convergence of the mean integrated square error have been established. Putting additional conditions for the signal behaviour at the infinity we have also evaluated the rate of convergence. The latter reveals that the sampling rate τ and truncation parameter N can be optimally selected in the form τ = an−α , N = bnβ , where α and β depend solely on the tail behaviour of the signal, wheras the constant a, b are some functions of the signal energy E0 and bandwidth Ω as well as the reconstruction rate h and noise variance σ 2 . Clearly, all these parameters are unknown in practice and one would like to select τ and N directly from data. For a given data record the sampling rate would be fixed and then the truncation point N remains to be selected. To fix our idea let us consider the estimate defined in (2.4) and (2.5). Clearly, one wishes to get N ? which minimizes M ISE(f˜), or equivalently the quantity I(N ) =
Z ∞
f˜2 (t) dt − 2
Z ∞
f˜(t)f (t) dt.
−∞
−∞
By Parseval’s formula X
I(N ) = h
c˜2k − 2h
|k|≤N
X
c˜k ck .
|k|≤N
The value N ? minimizing I(N ) cannot be obtained since ck ’s are unknown. To circumvent this difficulty we have to estimate c˜k ck . A one possible prescription is of the following form ξ˜k =
2
τ h
(2n(2n + 1))−1
X X
yi yj sk (iτ )sk (jτ ),
|i|≤n |j|≤n
where the summation is carried over all |i| ≤ n, and |j| ≤ n such that i 6= j. Its is relatively straightforward to show that |˜ ck ck − ξ˜k | → 0, in probability, as n → ∞. Hence, one can select N by minimizing the following criterion ˜ )=h I(N
c˜2k − 2h
X |k|≤N
X
ξ˜k .
|k|≤N
The problem whether such selection can produce a consistent estimate of N ? is left for further research, see [12] for an overview of data driven methods for selection of smoothing parameters in nonparametric regression techniques. 18
In our consistency results we assume that the reconstruction rate h is constant and could be chosen as π/Ω. One could also consider the case when h = hn and hn → 0 as n → ∞. Then, a quick inspection of the proofs of Theorem 1 and Theorem 2 yields the following consistency conditions for the estimate fˆ(t): h ≤ π/Ω, N < n hτ , N h → ∞, N τ → 0, nτ → ∞ and τ 3 N n → 0 as n → ∞. On the other hand, the estimate f˜(t) needs τ ≤ h ≤ π/Ω, N < n hτ , N h → ∞, N τ → 0, nτ → ∞, τ 3 N n → 0 and τ 3 N n/h3 → 0. The estimates with variable h would be needed for the problem of recovering not necessary band-limited functions, see [5], [6]. It is also worth noting that in such a case the reconstruction problem can be put into the setting of wavelet theory in which the sinc function plays the role of the scaling function [21]. Furthermore, it is known that not all band-limited functions are of finite energy (the case studied in this paper), e.g., trigonometric polynomials and functions of finite power, see [7], [22]. The problem of reconstructing this class of band-limited signals is left for future studies. Finally, let us comment that the results of this paper can be extended to the d-dimensional case, where the orthogonal system can be obtained in the Q form of the product of sinc functions, i.e., sk (t) = di=1 ski (ti ), where k = (k1 , k2 , . . . , kd ), t = (t1 , t2 , . . . , td ), see [15, Chapter 6], for some discussion of the sampling theorem in higher dimensions. The resconstruction problem in this case will be presented elsewhere.
Acknowledgment Mount-first Ng provided the programming support for the simulations. The authors also thank the reviewers for their useful comments.
Appendix Proof of Corollary 1 P The term |j|≤n sinc2 (jτ πh − πk) can be decomposed in the following way sinc2 (jτ
X
hk − (
h h + τ ) ≤ jτ ≤ hk + ( + τ ) π π 19
π − πk) + h
sinc2 (jτ
X
+
h hk + ( + τ ) < jτ ≤ nτ π
π − πk) h
sinc2 (jτ
X
+
−nτ ≤ jτ < hk − (
π − πk) h
h + τ) π
= M + R1 + R2 ,
+
=
say.
Since |sinc(t)| ≤ 1 therefore M ≤
2h + 3. πτ
Let us now consider the term R1 . It is plain that R1 ≤
1 (τ
h πτ
≤
X
π 2 ) (kh+ h +τ )/τ
