The Shannon sampling theorem, also known as the Whit- ... of Shannon's theorem, known as the Paley–Wiener–Levinson sam- pling theorem. This theorem ...
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 5, SEPTEMBER 1997
1717
whose restriction to the real axis is square-integrable, i.e., F is an
Nonuniform Sampling of Bandlimited Signals with Polynomial Growth on the Real Axis
entire function that satisfies jF (z)j Aejzj ;
Ahmed I. Zayed and Antonio G. Garc´ıa, Member, IEEE
and Abstract— We derive a sampling expansion for bandlimited signals with polynomial growth on the real axis. The sampling expansion uses nonuniformly spaced sampling points. But unlike other known sampling expansions for such signals, ours converge uniformly to the signal on any compact set. An estimate of the truncation error of such a series is also obtained.
1 01
I. INTRODUCTION
for some f 2 L2 (0; ), then it can be reconstructed from its samples at the points tn = n; n 2 ; where is the set of integers, by means of the formula F (t) =
1 n=01
F (tn )
0 tn ) 0 tn ) :
sin (t
(t
(1)
If the sampling points ftn gn2 are not equidistant, still F can be reconstructed from its values at these points by using a generalization of Shannon’s theorem, known as the Paley–Wiener–Levinson sampling theorem. This theorem states that if ftn gn2 is a sequence of real numbers such that
jtn 0 nj D < 14 ; for all n 2 (2) then any signal bandlimited to [0; ] can be reconstructed from its values at the points ftn gn2 by means of the formula F (t) =
1
n=01
where G(t) = (t
0 t0 )
F (tn )
1 n=1
1
G(t)
(t
0 tn )G0 (tn )
0 ttn
1
0 t0tn
(3)
:
(4)
Let us denote the class of signals bandlimited to [0; ] by B2 : It is worth mentioning that n (t) = G(t)=[(t
0 tn )G0 (tn )] 2 B2
for all n: Another famous theorem of Paley and Wiener [7] asserts that if F 2 B2 , it is an entire function of exponential type at most Manuscript received July 1, 1996; revised February 24, 1997. A. I. Zayed is with the Department of Mathematics, University of Central Florida, Orlando, FL 32816 USA. A. G. Garc´ıa is with the Departamento de Matem´aticas, Universidad Carlos III de Madrid, 28911 Legan´es, Spain. Publisher Item Identifier S 0018-9448(97)05712-X.
jF (x)j2 dx < 1;
where x = Re z:
For such a signal F , its energy E is represented by the L2 -norm, e.g., 2
E =
Index Terms— Bandlimited signals, nonuniform sampling, sampling theorems.
The Shannon sampling theorem, also known as the Whittaker–Shannon–Kotel’nikov sampling theorem [12], asserts that if F is a signal bandlimited to [0; ], i.e., it can be written in the form 1 0itw dw F (t) = f (w)e 2 0
for some A > 0
1
01
jF (x)j2 dx:
It is also known that bandlimited signals are bounded on the real axis. But since many signals in practical applications do not have finite energy according to the above definition, such as power signals, which are signals satisfying the condition T 1 lim
T !1 T
0T
jF (t)j2 dt < 1
it becomes evident that the class B2 of bandlimited signals is not large enough to include many signals of practical applications. A number of generalizations of the class B2 have been introduced. For example, the class Bp (1 p < 1) is defined as the class of all entire functions of exponential type at most that belong to Lp ( ) when restricted to the real axis. Likewise, the class B1 is defined as the class of all entire functions of exponential type at most that are bounded on the real axis. It is known that Bp Bq B1 , for 1 p q ; see [12, p. 24]. More generalizations of the class B2 have also been considered. Zakai [11], for example, introduced a class of bandlimited signals F in which the signals are entire functions of exponential type that are not necessarily bounded on the real axis. In fact, they satisfy the relation
1
01
jF (x)j2 dx < 1: 1 + x2
Not only the class of bandlimited signals B2 , but also its associated sampling theorems have been the focus of many generalizations; see [12, ch.2] for details. Unlike the class B2 , the class Bp for p > 2 is not defined in terms of the Fourier transform of its members, unless the Fourier transform is taken in the sense of generalized functions, since functions in Lp ( ), for p > 2, do not in general have Fourier transforms in the classical sense. If we allow the Fourier transform to be taken in the sense of generalized functions or Schwartz distributions, then the class of bandlimited signals can be enlarged tremendously, allowing bandlimited signals to be not only unbounded but also of polynomial growth on the real axis. The constant function, which represents a power signal, can be regarded as a bandlimited signal since its Fourier transform is essentially the Dirac delta function, (x), which is a generalized function with compact support. Sampling theorems for signals that are the Fourier transforms of generalized functions with compact support have been studied by many people. To the best of our knowledge, the first to do so was Campbell [1], who derived sampling expansions for signals that are the Fourier transform of generalized functions with compact support using uniformly distributed sampling points. His expansion converges pointwise, but lacks the familiar appearance of the Shannon sampling expansion (1). Pfaffelhuber [8] obtained a sampling expansion that
0018–9448/97$10.00 1997 IEEE
1718
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 5, SEPTEMBER 1997
restored the form (1), but its convergence was weakened from pointwise to convergence in the sense of tempered distributions. His results were generalized further by Lee [5], and Hoskins and De Sousa Pinto [4]. In [9], Walter introduced a convergence factor in the sampling expansions obtained by Lee and Pfaffelhuber, and also examined the (C; ) summability of these expansions. He showed that if F is a bandlimited signal with polynomial growth on the real axis, then the series (1) is (C; ) summable to F provided that F is oversampled and is chosen sufficiently large. In [10], Walter extended some of his results in [9] to nonuniform sampling expansions. Among other things, he showed that for bandlimited signals with polynomial growth on the real axis, the series (3) converges to F in the sense of ultradistributions, or more precisely in the sense of Z3 , where Z3 is the topological dual space of the space Z : The space Z is the image under the Fourier transformation of the space D consisting of all infinitely differentiable functions with support in (0; ); and provided with its usual topology described in [13]. In this correspondence, we study the nonuniform sampling expansions of bandlimited signals with polynomial growth on the real axis in a way parallel to that of Walter [10]. But unlike Walter and the others mentioned above, we obtain a nonuniform sampling expansion of a signal F that converges uniformly to F on compact sets. The price we pay for obtaining uniform convergence is the lack of the familiar form (3) in our sampling series. Our sampling functions are given in a form of a convolution involving the function G(t) defined in (4). This convolution structure of the sampling functions is not unusual; similar forms have already appeared in the work of Feichtinger and Gr¨ochenig [2], [3], where the sampling expansions of a large class of functions are shown to converge uniformly on compact sets.
To prove our results, we shall need some facts from the theory of nonharmonic analysis. For the reader’s convenience, we have collected in the following theorem the most important results needed for our forthcoming discussion. We call this theorem the Paley–Wiener– Levinson (PWL) theorem. The proofs of these results can be found in [6, ch.IV] and [7, ch.VII]. Theorem 2.1 (PWL Theorem): Let ftn g be a sequence of real numbers such that
jtn 0 nj D < ;
where m;n is the Kronecker symbol. 2) If f 2 L2 [0; ], then the series
1
01
and
1
2
1
01
(f
3 g)(x) =
01
f
and
0
(5)
f (x)hn (x) dx:
1 p 0 D) jfnj jjf jj 2
2
g
is defined as
(7)
For any open interval K , we define the space DK as the space of all infinitely differentiable functions '(x) with compact support in K: The support of ' is defined as the smallest closed set containing the set fx: '(x) 6= 0g: The support of ' will be denoted by supp ': If K = (a; b), then the support of any ' 2 DK is contained in 1 [a + ; b 0 ] for some > 0: A sequence f'n gn=0 in DK is said to converge to zero in DK if there is a compact set K1 K such that ( p) supp 'n K1 for all n and 'n ! 0 uniformly as n ! 1 for all p = 0; 1; 2; 1 1 1 : The topological dual space of DK will be denoted 3 is said to converge to 0 in DK 3 3 : A sequence ffn gn1=0 in DK by DK if for any ' 2 DK ; we have hfn ; 'i ! 0 as n ! 1, where hf; 'i is the number that the functional f assigns to ': In this article we shall be concerned mainly with the space DK where K = (0; ) and to simplify the notation we shall denote this space by D and its dual by D3 :
=
0
jf (x)j dx 2
(8)
(see [7, eqs. (29.04) and (30.07)]).
hn (x) =
1
2
1
G(t) e0ixt dt 01 (t 0 tn )G0 (tn )
where
f (t)g(x 0 t) dt
3 g)(w) = f^(w)^g(w):
2
n=01
3)
hence (f
(1
f^(w)e0iwt dw:
1
x)
converges uniformly to zero over any interval [0 + ; 0 ], for > 0, where f^n = 1 f (x)e0inx dx (6) 2 0
fn =
f (t)eiwt dt
The convolution of two functions
0 fneit
Moreover
so that its inverse transform is given by
f (t) =
inx
(f^n e
n=01
We define the Fourier transform of a function f (t) as
f^(w) =
:
Then the following are true: 1) The set of functions feit x g is complete in L2 [0; ] and possesses a unique, complete biorthonormal set fhn (x)g such that hm (x)eit x dx = m;n 0
II. PRELIMINARIES
1
for all n 2
1 4
G(t) = (t 0 t0 ) 4) If
f
1 n=1
2 L [0; ], then
1
0 ttn
1
0 t0tn :
2
f (x) =
1 n=01
f~n hn (x)
where
f~n =
0
f (x)eit x dx:
(9)
III. THE SAMPLING THEOREM To prove our sampling theorem, the following lemmas will be needed. For simplicity, we shall assume from now on that t0 = 0 so that hm (x) dx = m;0 0 and denote the set of all functions orthogonal to
h0 (x) by H0? :
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 5, SEPTEMBER 1997
Lemma 3.1: Let ' have p continuous derivatives and assume that the support of ' is contained in (0; ): If
2 H ?; i = 0; 1; 1 1 1 ; p 1 then the sequences f'n g1 ~n gn 01 , defined by (7) and n 01 and f' ( i)
'
0
=
Corollary 3.1: Let '(x) be an infinitely differentiable function with support in the open interval (0; ) such that ' and all 1=01 and its derivatives are in H0? . Then the sequences f'n gn 1 f'~n gn=01 are rapidly decreasing, i.e.,
(9), satisfy
'n ; ' ~n = O
as jnj
;
np
! 1:
Proof: We begin with the sequence f' ~n g: By integrating by parts p times, we obtain p (i) (p) it x dx ' ~n = ' (x)e p (tn ) 0
1
'n ; ' ~n = O
=
1
1719
np
as jnj
;
A = f' 2 L [0; ]: f'n gn1 01 is rapidly decreasingg: The set A is nonempty as can be seen from Corollary 3.1, and it is ~ be the set of all functions also closed under differentiation. Let D in D that belong to A: ~ 3 be the topological dual space of the space D ~ Lemma 3.2: Let D 3 3 ~ and B be that subspace of D consisting of all generalized functions with support in (0; ): Then the nonharmonic Fourier series, 2
=
1
j'~n j jtn jp ;
for all n and tn = 6 0:
For tn = 0, we have j' ~0 j C: But since tn n as jnj ! 1, the result follows. As for the sequence f'n g, we have in view of (5) that the series
1
('n
(p)
n=01
0
inx e
(p)
it x ) 'n e
converges uniformly in [0 + ; 0 ], in particular, it converges at (p) (p) (p) x = 0, where f' ^n g and f'n g are the coefficients of ' (x) as given by (6) and (7). Thus the series
1
(p)
(' ^n
n=01
0 'np ) ( )
(p)
hf; 'i =
( )
j'n j j'^n j + ;
hf; 'i =
N:
(p)
p ^ and '(p) = (it )p ' : To But it is easy to see that ' ^n = (in) ' n n n n show the latter, let us note that if '(x) and '(1) (x) are in H0? , then (x) =
(1)
n6=0
it x
'n e
n6=0
it x =
'n e
(1)
'n
n6=0
(itn )
1
f;
'n e
it x
n=01
'n e
1
jtn jp j'n j jnjp j'^nj + ; for all jnj N: Because f' ^n g are the Fourier coefficients of a p-differentiable ^n j A for all n: function with compact support, we have that jnjp j ' Thus since tn n as jnj ! 1, we have p j'n j jtBn jp = jnBjp jjtnnjjp jnCjp ; for all n 6= 0
n=01
h
it x
'n f; e
i
'n f~n n=01 which shows that the series converges. Some kind of a converse of this lemma also exists. Lemma 3.3: Let f be a generalized function with support in it x for all n: [ ; ] and define f~n by f~n = f; e Then the series
h
1
it x + C
(1)
1
=
=
e
which implies that C = 0: This leads to 'n = (itn )'n for n = 6 0, and the result now follows by induction. Therefore,
where A; B; and C are constants.
'n f~n
it x
0
and by integrating this series, we obtain '(x) =
and
~ too. Now by the continuity of f , we obtain of ' converges to ' in D
for all jnj
(p)
(1)
n=01
2 B3
where f~n = hf; eit x i and 'n are defined by (7). Proof: Since f has compact support and eit x is infinitely differentiable for each n, the coefficients f~n are well defined. The ~ converges to the 2 -periodic extenFourier series of any ' 2 D sion, 'per , of ', in the topology of E , the space of all infinitely differentiable functions. But on (0; ); 'per = '; therefore, the ~ : Thus by Fourier series of ' converges to it in the topology of D Theorem 2.1, the nonharmonic Fourier series
and hence for any > 0 there exists N > 0 such that
'
1
n=01
0 '^np ) = 0
'n e
~ , and for any f ~ converges to ' in D of any ' 2 D ~ , we have ' 2 D
1
lim ('n
(p)
it x
n=01
converges, which implies that
jnj!1
for all p 0:
Let
which implies C
!1
n=01
i
f~n hn (x)
~ 3 to f and f~n are of polymonial growth as jnj ! 1: converges in D Proof: As in the proof of Lemma 3.2, the coefficients f~n are well-defined. Since f has compact support, there exist a continuous function g and a nonnegative integer q such that f (x) = Dq g(x) in the sense of generalized functions, where D = d=dx; see [13, p. 93]. Thus it x i = hDq g; eit x i = (0it )q hg; eit x i f~n = hf; e n
and hence
jf~nj C jtn jq C jnjq ; 1
for some constants C1 and C:
for all n 6= 0
1720
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 5, SEPTEMBER 1997
It is evident that
it has an expansion of the form
SN (x) =
jnjN
f~n hn (x)
defines a generalized function with support in [0; ]: Therefore, for ~ , we have any ' 2 D
hSN (x); '(x)i =
jnjN
hSN (x); '(x)i = Nlim !1
lim N !1
jnjN
'n f~n
=
F (t) =
f'n g is of rapid 1 n=01
f;
1
n=01
Therefore,
f (x) = ~3 : in D
'n eit x
1 n=01
1
=
n=01
'n f~n
n=01
'n f~n :
f~n hn (x)
does not, in general, define a generalized function with support in the interval (0; ): Now we are able to state and prove our sampling theorem. Theorem 3.1: Let f 2 B3 and choose an even function (x) such that ^ 2 C 1 (0; ) with supp ^ (0; ); ^(x) = 1 on the support of f; and eitx ^(x) 2 A: Then
F (t) = hf (x); eitx i
n=01
F (tn )Sn (t)
(10)
where
with (t
n=01
F (tn )n (t)
(12)
^(x)hn (x)eitx dx:
0
1
01
n (u) (t
0 u) du = ( n 3 )(t) = Sn (t)
which upon its substitution in (12) yields (10). To show that the series in (10) converges uniformly on compact sets, we recall that F (tn ) = O(jtn jq ) for some q 0: The sampling functions Sn (t) are the coefficients of an infinitely differentiable function (t; x) 2 A, with support in (0; ) when expanded in the nonharmonic Fourier series (11), and therefore they are rapidly decreasing in n for each fixed t: This means that jSn (t)j C (t)=jtn jp for sufficiently large n and each fixed t, for all p 0: To complete the proof, we must show that for all t in some compact set K; jC (t)j CK , independent of t: First, we differentiate (11) p times with respect to x to obtain
@ p (t; x) @xp
1
=
p it x (itn ) n (t)e
n=01
then apply (8) to obtain (1
0
p
D)2
1
@ p (t; x) 0 @xp
jtn j p jn(t)j 2
n=01
G(t)
F (t) = hf (x); eitx i = hf (x); eitx ^(x)i: But since (x; t) = eitx ^(x) is an infinitely differentiable function in x with support in (0; ), it belongs to the space D , and hence
=
2
k=0 p k=0
p k
k itx (p0k) (it) e ^ (x)
p k
jtjk j ^ p0k (x)j 2p (
)
p
2
dx:
(13)
jtjk j ^ p0k (x)j: (
k=0
Let K be a compact set and choose A such that K Then
@ p (t; x) @xp
p
2p Ap
[0A; A]:
j ^ p0k (x)j = 2p Ap (x) (
k=0
)
)
which, upon its substitution in (13), yields
0 tn )G0 (tn )
and the points ftn g are any points satisfying (2). The series converges uniformly on any compact subset of the real axis. Proof: That F is an entire function of exponential type 0 with the prescribed growth rate is a known fact in the theory of generalized functions; see [13, ch.7]. Now
p
@ p (t; x) @xp
Sn (t) = ( n 3 )(t)
n (t) =
(t; x)hn (x) dx =
0
1
n (t)hf; eit x i =
By Leibniz rule, we have
is an entire function of exponential type 0 for some > 0 that grows no faster than a polynomial on the real axis as jtj ! 1: Moreover, F can be reconstructed from its samples at the points ftn g via
F (t) =
n=01
n (t) =
f~n hn (x)
1
(11)
Recognizing n (t) as the Fourier transform of ^(x)hn (x), we can, with the aid of the fact that is even, write n (t) in the form
It should be noted that Lemma 3.3 is not exactly the converse of Lemma 3.2 since the series
1
1
with
n (t) =
exists, but on the other hand, as in Lemma 3.2
hf; 'i =
n=01
n (t)eit x
~ ; see Lemma 3.2. Thus which converges to it in the sense of D
'n f~n :
But since ff~n g is of polynomial growth and decay, we have
1
(x; t) =
(1
0
where C ^ =
1
p
D)2
jjjj , 2
jtn j p jn(t)j 2 p A p C 2
n=01
2
2
2
2 ^
and p k=0
Thus
jSn (t)j = jn (t)j
j ^ p0k (x)j: (
(x) =
p p 2 A C ^
(1
0
p
)
D)jtn jp
;
for all p 0:
Hence, having in mind that tn n as jnj ! 1 and taking p > q + 1, the uniform convergence of the series (10) is proven.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 5, SEPTEMBER 1997
IV. TRUNCATION ERRORS As a byproduct of Theorem 3.1 we can give an estimate of the truncation error which arises if one ignores all the samples outside a finite interval. More precisely, we have the following corollary. Theorem 4.1: Let us define the truncation error EN (t) as follows:
EN (t) = F (t) 0 Then 2p
jEN (t)j
(1
0
p2
jnjN
F (tn )Sn (t):
0q bAp C ^
; D)(p 0 q 0 1)N p0q01 t in the compact set K (14)
where p > q + 1, and q the polynomial order of growth of F (t): Proof: We know from Theorem 3.1 that jF (tn )j bjtn jq for some q 0 and a constant b, and
jSn (t)j
p p 2 A C ^
(1
0
p
CK
; j j jtn jp for all p 0 and t in the compact set K:
D) tn p
=
1721
[4] R. F. Hoskins and J. De Sousa Pinto, “Sampling expansions for functions band-limited in the distributional sense,” SIAM J. Appl. Math., vol. 44, pp. 605–610, 1984. [5] A. J. Lee, “Characterization of band-limited functions and processes,” Inform. Contr., vol. 31, pp. 258–271, 1976. [6] N. Levinson, Gap and Density Theorems (Amer. Math. Soc. Colloq. Pub. Ser.), vol. 26. New York: Amer. Math. Soc., 1940. [7] R. Paley and N. Wiener, Fourier Transforms in the Complex Domain (Amer. Math. Soc. Colloq. Pub. Ser.), vol. 19. Providence, RI: Amer. Math. Soc., 1934. [8] E. Pfaffelhuber, “Sampling series for band-limited generalized functions,” IEEE Trans. Inform. Theory, vol. IT-17, pp. 650–654, 1971. [9] G. G. Walter, “Sampling bandlimited functions of polynomial growth,” SIAM J. Math. Anal., vol. 19, no. 5, pp. 1198–1203, 1988. , “Nonuniform sampling of bandlimited functions of polynomial [10] growth,” SIAM J. Math. Anal., vol. 23, no. 4, pp. 995–1003, 1992. [11] M. Zakai, “Band-limited functions and the sampling theorem,” Inform. Contr., vol. 8, pp. 143–158, 1965. [12] A. I. Zayed, Advances in Shannon’s Sampling Theorem. Boca Raton, FL: CRC Press, 1993. [13] A. H. Zemanian, Distribution Theory and Transform Analysis. New York: MacGraw-Hill, 1965. , Generalized Integral Transformations. New York: Dover, 1987. [14]
Hence, if we take any p > q + 1, we will have
jEN (t)j
jnj>N
bCK bCK
jF (tn )jjSn (t)j
jnj>N j j 0(N +1)
2bCK
Covering Numbers for Real-Valued Function Classes
1 tn p0q
n=01
n+
1
N 0(1=4) (x
0
q 0p
1 4 1
1 +
n=N +1
n0
1
q 0p
4
0q dx:
1 p ) 4
If we use the change of variable, x 0 1=4 = Nt, and note that [N 0 (1=2)]=N; 8N 1, we obtain
1=2
jEN (t)j 2bCK =
1
N 0(1=4) (x 2bCK
2q 0p+1 (p
0
1
2bCK
0q dx N p0q01
1 p ) 4
0 q 0 1)N p0q0
1
P. L. Bartlett, Member, IEEE, S. R. Kulkarni, Senior Member, IEEE, and S. E. Posner
1 dt p0q 1=2 t
Abstract—We find tight upper and lower bounds on the growth rate for the covering numbers of functions of bounded variation in the L1 metric in terms of all the relevant constants. We also find upper and lower bounds on covering numbers for general function classes over the family of L1 (dP ) metrics in terms of a scale-sensitive combinatorial dimension of the function class. Index Terms— Bounded variation, covering numbers, fat-shattering dimension, metric entropy, scale-sensitive dimension, VC dimension.
:
I. INTRODUCTION
Therefore, replacing the constant CK by its value, we obtain the desired result (14). ACKNOWLEDGMENT This work was performed while the first author was visiting the “Universidad Carlos III de Madrid” in Spain and he wishes to take this opportunity to thank the Mathematics Department for its hospitality and stimulating atmosphere. REFERENCES [1] L. L. Campbell, “Sampling theorem for the Fourier transform of a distribution with bounded support,” SIAM J. Appl. Math., vol. 16, no. 3, pp. 626–636, 1968, [2] H. G. Feichtinger and K. Gr¨ochenig, “Irregular sampling theorems and series expansions of band-limited functions,” J. Math. Anal. Appl., vol. 167, pp. 530–556, 1992. , “Iterative reconstructions of multivariate band-limited functions [3] from irregular sampling values,” SIAM J. Math. Anal., vol. 26, pp. 244–261, 1992.
Covering numbers have been studied extensively in a variety of literature dating back to the work of Kolmogorov [10], [12]. They play a central role in a number of areas in information theory and statistics, including density estimation, empirical processes, and machine learning (see, for example, [4], [8], and [16]). Let F be a subset of a metric space (X ; ). For a given > 0, the metric covering number N (; F ; ) is defined as the smallest number of sets of radius whose union contains F . (We omit if the context is clear.) Manuscript received February 12, 1996; revised March 31, 1997. This work was supported in part by the NSF under Grant NYI IRI-9457645. This work was done while S. E. Posner was with the Department of Electrical Engineering, Princeton University, Princeton, NJ, and the Department of Statistics, University of Toronto, Toronto, Ont., Canada. P. L. Bartlett is with the Department of Systems Engineering, Australian National University, Canberra 0200, Australia. S. R. Kulkarni is with the Department of Electrical Engineering, Princeton University, Princeton, NJ 08544 USA. S. E. Posner is with ING Baring Securities, New York, NY 10021 USA. Publisher Item Identifier S 0018-9448(97)05711-8.
0018–9448/97$10.00 1997 IEEE
