Signal interpolation using numerically robust differential operators Aleks Ignjatović School of Computer Science and Engineering University of New South Wales and National ICT Australia (NICTA) Sydney, Australia
[email protected]
Foundations of the technique in: 1. Aleksandar Ignjatovic: Local approximations based on orthogonal differential operators, Journal of Fourier Analysis and Applications, vol. 13, no. 3, 2007. 2. ——"—— : Chromatic derivatives and local approximations, IEEE Transactions on Signal Processing, Vol. 57, issue 8, 2009. 3. ——"—— : Chromatic derivatives and associated function spaces, East Journal on Approximations, vol. 15, no. 3, 2009. Papers and preprints are available at: http://www.cse.unsw.edu.au/˜ignjat/diff/
Problem Formulation ◮ Problem: Transmit a (short) fragment ϕ(t) of duration of T unit intervals of a π band limited signal f (t) within time of N + T + N unit intervals, using minimally the bandwidth outside [−π, π]. 4
4
2
2
10
10
20
30
40
50
-4
20
60
-2
-2
⇒
-4
How to optimally extend the signal over [0, N ] and [N + T , N + T + N ]?
30
40
50
60
Technique motivation Let f ∈ BL(π), i.e. f ∈ L2 , fd (ω) supported on [−π, π]; then: ———————————————————————————— f (t) =
Sampling Expansion:
∞ X
f (n)
n=−∞
(Whittaker–Kotelnikov–Nyquist–Shannon)
sin π(t − n) π(t − n)
◮ global in nature – requires samples f (n) for all n; ◮ fundamental to signal processing; ———————————————————————————— Taylor’s Expansion:
f (t) =
∞ X
n=0
f (n) (0)
tn n!
◮ local in nature – requires values of f (n) at t = 0 only; ◮ very little use in signal processing — WHY??
◮
Numerical evaluation of derivatives of high orders of a sampled signal is extremely noise sensitive.
◮
• Truncations of Nyquist’s expansion of an f ∈ BL(π) belong to BL(π) and converge to f uniformly and in BL(π). • In comparison, truncations of Taylor’s expansion of an f ∈ BL(π) do not belong to BL(π), converge neither uniformly nor in L2 , are unbounded and accumulate error rapidly.
HOW TO FIX ALL OF THESE PROBLEMS???
Numerical differentiation of band limited signals f (n) (t) 1 Let f ∈ BL(π); then = n π 2π
Z
π n
i
−π
n
ω π
fd (ω)e i ωt dω.
0.5
-3
-2
1
-1
2
3
-0.5
n
Figure: (ω/π) for n = 15 − 18 ◮ ◮
◮
Derivatives of high order obliterate the spectrum. Graphs of the transfer functions of the (normalized) derivatives cluster together and are nearly indistinguishable. Is there a better base for the vector space of linear differential operators??
A Better base for the vector space of lin. diff. operators ◮ Start with normalized and re-scaled Legendre polynomials: 1 2π
Z
π
−π
L PnL (ω)Pm (ω)dω = δ(m − n).
◮ Obtain operator polynomials by replacing ω k with ik d k /dt k : Ktn
1 d = n PnL i dt i
Thus, P0L (ω) = 1 √ 3ω L P1 (ω) = π √ 5 (3 ω 2 − π 2 ) L P2 (ω) = 2π 2 √ 3 7 (5 ω − 3 ωπ 2 ) P3L (ω) = 2π 3
7→ 7→ 7→ 7→
K0 [f ](t) = f (t) √ ′ 3 f (t) 1 K [f ](t) = √ π ′′ 5 (3f (t) + π 2 f (t)) K2 [f ](t) = 2π 2 √ ′′′ 7 (5 f + 3 π 2 f ′ (t)) K3 [f ](t) = 2π 3
A Better base for the vector space of lin. diff. operators ◮ Definition of Kn chosen so that Ktn [e i ωt ] = in PnL (ω) e i ωt .
◮ Thus, for f ∈ BL(π), Kn [f ](t) =
1 2π
Z
π
−π
(ω)e i ωt dω. in PnL (ω)fd
◮ Why do Kn form a better base for the vector space dn of linear differential operators than n ??? dt
A Better base for the vector space of diff. operators 0.5
Π
-Π
Π
Π
2
2
-0.5
2 1
Π
-Π
Π
Π
2
2
-1
-2
◮ Compare the graphs of the transfer functions of 1/π n d n /dt n , i.e., (ω/π)n (first graph) and of Kn , i.e., PnL (ω) (second graph). ◮ Transfer functions of Kn form a sequence of well separated comb - like filters which preserve spectral features of the signal, thus we call them the chromatic derivatives.
3 2 1
Π
-
Π
Π
Π
8
4
4
8
-1 -2 -3
◮ Third graph: transfer function of the ideal filter K15 (red) vs. transfer function of a transversal filter (blue), (128 taps, 2× oversampling).
Fixing Taylor’s Expansion: Chromatic Expansion Proposition: Let sinc (t) = analytic function. Then, f (t) = =
∞ X
sin(πt) and let f (t) be any πt
(−1)n Kn [f ](0) Kn [sinc (t)]
n=0 ∞ X n=0
Kn [f ](0)
√
2n + 1 jn (πt)
jn − the spherical Bessel functions
solutions of x 2 y ′′ + 2xy ′ + [x 2 − n(n + 1)]y = 0
◮
The truncations of the series belong to BL(π).
◮
If f ∈ BL(π) the series converges to f (t) both uniformly and in L2 .
Chromatic approximation versus Taylor’s approximation 2.0 1.5 1.0 0.5
-15
-10
5
-5
10
-0.5 -1.0 -1.5
◮
red: the signal; blue: the chromatic approximation of order 15; green: Taylor’s approximation of order 15.
15
Local representation of the scalar product in BL(π) Proposition: Assume that f , g ∈ BL(π); then the sums on the left hand side of the following equations do not depend on the choice of the instant t, and ∞ X
n
2
K [f ](t) =
n=0 ∞ X
K n [f ](t)K n [g](t) =
n=0 ∞ X
n=0
K n [f ](t)Ktn [g(u − t)] =
Z
∞
−∞
Z
∞
−∞
Z
f (t)2 dt = kf k2 f (t)g(t)dt = hf , gi
∞
−∞
f (t)g(u − t)dt = (f ∗ g)(u)
◮ These are the local equivalents of the usual, “globally defined” norm, scalar product and convolution!
Application of chromatic derivatives ◮ If f ∈ BL(π) then Thus,
∞ X
n=0
n
2
K [f ](t) =
Z
∞
−∞
f (t)2 dt = kf k2 .
|K n [f ](t)| ≤ kf k. One can show that for every |a| < π there exists M > 0 such that for all |ω| ≤ a |PnL (ω)| < M . Thus, since Ktn [e i ωt ] = in PnL (ω) e i ωt , we have
|K n [sin(t)]| ≤ |PnL (ω)| < M . For band limited signals and for trigonometric polynomials the values of |K n [f ](t)| are uniformly bounded in t and n. This makes constraints involving chromatic derivatives numerically feasible.
Application of chromatic derivatives
Note that this is NOT the case with the standard derivatives: for f ∈ BL(π) f
(n)
1 (t) = 2π
Z
∞
−∞
in ω n bf (ω)ei ωt dt;
for the trigonometric functions dn [ei ωt ] = in ω n ei ωt ; dt n Note that ω n vanishes for |ω| < 1 and ”explodes" for |ω| > 1 if n is large.
Application of chromatic derivatives: band limited interpolation ◮ Finally, back to our problem: We want to transmit a (short) fragment ϕ(t) of duration of T unit intervals of a π band limited signal f (t). The transmission can last at most N + T + N unit intervals: 4
2
10
20
30
40
50
60
-2
-4
Thus, the transmitted signal cannot be band limited.
Application of chromatic derivatives: band limited interpolation We want to extend the signal from the interval [N , N + T ] to the interval [0, N + T + N ] so that:
◮ the extended signal has minimal fraction of energy outside the bandwidth; [−π, π];
◮ its maximal amplitude over [0, N ] and [N + T , N + T + N ] is also as small as possible.
Application of chromatic derivatives: band limited interpolation How to control out of band content of the extrapolated signal? 4
2
10
20
30
40
50
60
-2
-4
Main idea: extend fragment φ(t) to a function ψ(t) such that: ◮ ψ(t) = 0 for t < 0 and for t > N + T + N ; ◮ ψ(t) is N − 1 times continuously differentiable.
Application of chromatic derivatives: band limited interpolation Why does this work? ◮ Clearly, being finitely supported, ψ (n) (t) are all L1 functions; thus K i.e.,
(N −1)
1 [ψ](t) = 2π
Z
∞
−∞
i ωt ω b iN −1 PN −1 (ω)ψ(ω)e d
b (K (N −1) [ψ]) b(ω) = iN −1 PN −1 (ω)ψ(ω)
and K (N −1) [ψ] b(ω) is continuous and bounded; Thus, for some M > 0 and all ω,
.
|K (N −1) [ψ] b(ω)| < M
Application of chromatic derivatives: band limited interpolation Consequently,
b ≤ |ψ(ω)|
M |PN −1 (ω)|
This implies that outside [−π, π], which contains all the zeros of b PN −1 (ω), |ψ(ω)| rapidly decreases. To minimize the value of the constant M we note that |K
(N −1)
[ψ] b(ω)| = ≤
Z ∞ 1 (N −1) − i ωt K [ψ](t)e dt 2π −∞ Z T +2N 1 (N −1) [ψ](t) dt K
2π
0
Application of chromatic derivatives: band limited interpolation
Thus, we have to minimize the values of |K (N −1) [ψ](t)| over intervals [0, N ] and [N + T , N + T + N ]. Remember that we also want to minimize |ψ(t)| over the same intervals. This is accomplished using two chromatic approximations, one over [0, N ] and one over [N + T , N + T + N ].
N/2
N/2
Km[App1](0) = 0 for all m < N
T
N/2
N/2
Km[App1](N) = Km[f](N) Km[App2](N+T) = Km[f](N+T)
Km[App2](2N+T) = 0
Application of chromatic derivatives: band limited interpolation Let Bn (t) =
√
2n + 1 jn (πt); we set App(t) =
3N X
k=0
Xk Bk (t − N /2),
and impose the following constraints: for all m ≤ N − 1, K m [App](0) = 0;
K m [App](N ) = K m [f ](N ).
~ = hX1 , . . . Xn i which minimizes We now find the value of X
Max {App(i/8) : 0 ≤ i ≤ 8N } ∪ {µK N −1 [App](i/8) : 0 ≤ i ≤ 8N } where µ is a constant whose value can change the "priority" given to minimizing the amplitude versus minimizing the out-of-band content.
Application of chromatic derivatives: band limited interpolation The DFT of the extrapolated signal, sampled at twice the Nyquist rate:
4
3
2
1
-200
-100
100
200
THANK YOU!
