scenario in Fig. 1, in which an AF Hi(z) tries to identify a plant Ho(z), both IIR with the same order M. The desired signal d(i) is given by [3] d(i) = yo(i) + v(i) = xo.
FIR-IIR Adaptive Filters Hybrid Combination Humberto F. Ferro, Luiz F. O. Chamon and Cassio G. Lopes In order to enhance the performance in IIR system identification scenarios, we propose a hybrid combination of FIR and IIR adaptive filters (AFs) via a supervisor that senses which one is performing best. The FIR LMS AF is short, providing fast and robust convergence, while the IIR LMS AF is slow but accurate. The stagnation effect caused by the different convergence rates is tackled through cyclic weights transfers FIR → IIR, which also ensure good tracking properties. A design technique for the transfers cycle length is proposed, providing good convergence while keeping computational cost low.
Introduction: We introduce an FIR-IIR AFs combination in the system identification setup which shows improved performance over an ordinary IIR AF while keeping the computational complexity low by skipping stability checks. To achieve this goal, the FIR AF (or guide) is designed to be fast and robust while the IIR AF has a small step size, thus it is accurate. Also, a cyclic transfer of weights based on the balanced order reduction [1] is performed. The resulting system converges as fast as the FIR guide while achieving the lowest Excess Mean Squared Error (EMSE) of the IIR AF, and can be extended to other applications, as adaptive prediction [2].
where ui = [u(i) u(i − 1) · · · u(i − M1 + 1)] is M1 × 1 in the FIR case. Both update rules of Eqs. 3 and 4 provide unbiased estimates at a low computational cost, but the latter may require stability checks to prevent unstable poles in Hi (z) [5, 7, 8], making the adaptation computationally costly. However, in this work the IIR AF is implemented with a small step-size to provide accuracy in steady-state, which leads to exponential stability [9] and, therefore, makes the checks unnecessary. Also, stability is further enforced via AF combinations, as explored in the sequel. Combinations of adaptive filters: This work proposes a hybrid FIR-IIR combination (FIIR cell) as in Fig. 2, in which a fast and robust FIR-LMS (AF1) is combined with an accurate but slow IIR-NLMS (AF2), generating the overall filter output y(i) = λ(i)y1 (i) + [1 − λ(i)]y2 (i), where yk (i) is 1 the k-th AF output, λ(i) = 1+e−a(i−1) is the convex supervisor, and a(i) is adapted via a gradient-descent rule [10, 11] a(i) = a(i − 1) + µa e(i) [y1 (i) − y2 (i)] λ(i) [1 − λ(i)] ,
(5)
with e(i) = d(i) − y(i) the combination global error and µa is a step size.
Fig. 2. Filters Combination
Fig. 1. System Identification
FIR and IIR Adaptive Filtering: Consider the system identification scenario in Fig. 1, in which an AF Hi (z) tries to identify a plant H o (z), both IIR with the same order M . The desired signal d(i) is given by [3] d(i) = y o (i) + v(i) = xoi wo + v(i),
with the unknown weights
wo
and the plant regressor
xoi
(1) vectors set to
wo = [−ao1 − ao2 · · · − aoM −1 − aoM bo0 bo1 · · · boM −1 boM ]T xoi = [y o (i − 1) y o (i − 2) · · · y o (i − M ) u(i) u(i − 1) · · · u(i − M )],
where u(i) is a Gaussian input signal, {bok } and {aok } are the unknown feedforward and feedback coefficients, v(i) is a Gaussian zero-mean white noise. Similarly, define the AF weights and regressor vectors wi and xi as
Such scheme drives the global filter y(i) to the convergence rate of the fast AF and the steady state error of the accurate filter, although a stagnation effect takes place until the latter outperforms the former one [11]. When both AFs are FIR, this can be mitigated via direct FIR1 → FIR2 weights transfer. When combining FIR with IIR filters, the stagnation effect may be much worse, and weights transfers are not trivial. Transfers IIR → FIR are simple, but not wise, so that robustness is enforced. On the other hand, transfers FIR → IIR are mathematically involving, but may considerably enhance overall stability and convergence. We address this issue by introducing a projection function P : FIR → IIR in the context of AF combinations that maps AF1 into AF2 whenever convenient. There are many ways to define P ; here, we resort to balance order reduction [1], implemented via the efficient eigen-decomposition of the FIR AF weights Hankel matrix [12], at complexity O((M1 + 1)2 log(M1 + 1)), with M1 small (FIR-AF). The transfers AF1 → AF2 are performed cyclicly, every L iterations, whenever AF1 is better than AF2, as tracked by λ(i). Herewith, complexity per cycle is further reduced and non-stationaries are addressed, particularly abrupt changes in wo . The resulting algorithm is then given by
wi = [−a1 (i) − a2 (i) · · · − aM (i) b0 (i) b1 (i) · · · bM (i)]T and
w1,i = w1,i−1 + µ1 uT i e1 (i)
xi = [y(i − 1) y(i − 2) · · · y(i − M ) u(i) · · · u(i − M )],
(
so that the AF output is y(i) = xi wi−1 [3, 4]. In the Output Error LMS (IIR-LMS) algorithm [3, 5], the squared error e2 (i) = (d(i) − y(i))2 is minimized via the stochastic gradient ∇w e2 (i) = −ϕT i e(i), where the P filtered regressor ϕi is defined as ϕi = xi − M a (i) ϕi−k [5] and k k=1 then the IIR-LMS update rule arises as [3, 5] wi = wi−1 + µϕT i e(i)
ϕT i e(i), kϕi k2 +
(3)
in which is a small regularization factor. With {ak = 0}, the standard FIR-LMS is retrieved from the IIR-LMS aforementioned wi = wi−1 + µuT i e(i),
ELECTRONICS LETTERS
22 January 2014
(4)
Vol. 00
δL P(w1,i−1 ) + (1 − δL )w2,i−1 , w2,i−1 ,
λ(i) ≥ β λ(i) < β
(7)
ϕT i e2 (i) kϕi k2 +
(8)
a(i) = a(i − 1) + µa e(i)(y1 (i) − y2 (i))λ(i)(1 − λ(i)),
(9)
w2,i = w2,i−1 + µ2
(2)
IIR-LMS may be employed, but its normalized version, the IIR-NLMS algorithm, is usually preferred as it is less sensitive to underdamped or clustered poles in H o (z) [6] wi = wi−1 + µ
w2,i−1 =
(6)
in which δL = δ(r) is the Kronecker delta with r = i mod L, 0 β < 1 is a constant that determines when AF1 is better than AF2, ek (i) = d(i) − yk (i), y1 (i) = ui w1,i−1 , y2 (i) = xi w2,i−1 . Cycle Length L: We explore the Energy Conservation Relation (ECR) [4] to devise a simple yet efficient design technique for L. First, the FIR-LMS learning is approximated by a straight line (in dB) that captures the AF initial convergence rate (see Fig. 3a). When this curve reaches the noise floor 10 log σv2 at i = L, the AF1 steady state is declared and the transfer P : w1,i−1 → w2,i−1 is performed, if AF1 is better than AF2. Applying ECR over the stand-alone FIR-LMS filter (AF1) leads to the weighted variance relation [4], which provides a model for the AF1
No. 00
0
x 1000
Theory (Eq.10 and 10⋅ log
−10
σ2v)
AF1 MSE (measured)
−20 −30 −40 0
(a)
L
6 4
(b)
Standalone IIR
−10
No−WT FIR Guide
−20 WT −30
2 0 0
750 1500 2250 ITERATIONS
EMSE (dB)
MSE (dB)
0
Convergence Time
8
0
0.5
1
1000 2000 3000 4000 CYCLE LENGTH
1.5
ITERATIONS
2
2.5
3 4
x 10
Fig. 5. Stationary Scenario Fig. 3. Cycle Length Design 0
2
E kw ˜1,i−1 k = γ
i
E kw1o k2
+
2 µ1 σv2 σu (M1
+ 1)
i−1 X
k
γ ,
EMSE (dB)
transient evolution, assuming a real valued Gaussian white input u(i); i.e., (10)
k=0
2 2 σd −σv 2 σu
−30
No−WT
WT −50 0
[4], such that E e21 (i) ≈
Standalone IIR
No−WT
−20
−40
where γ , 1 − 2µσu2 + µ21 σu4 (M1 + 4), w1o is the FIR truncated from the IIR H o (z), w ˜1,i = w1o − w1,i . Eq. (10) can be approximated by E kw ˜1,i−1 k2 ≈ γ i E kw1o k2 . As e1 (i) = ui w ˜1,i−1 + v(i), the Mean Squared Error (MSE) E e21 (i) equals σu2 E kw ˜1,i−1 k2 + σv2 . The quantity kw1o k2 can be estimated from kw1o k2 =
Standalone IIR FIR Guide
−10
1
2
3
4
FIR Guide
WT
5
7
ITERATIONS
6
8 4
x 10
Fig. 6. Non-Stationary Scenario
γ i (σd2 − σv2 ) and the AF1 approximate transient model becomes (in dB) M SE1,dB (i) ∼ = 10i log γ + 10 log(σd2 − σv2 )
the overall complexity of the mapping P . Current work involves FFT-based maps to efficiently replace the function P employed here.
(11)
A good estimate for L is then L = {i|M SE1 (i) = 10 log σv2 } (see Fig. 3(a)). This simple design is quite efficient, and robust as L varies over a wide range, as Fig. 3(b) depicts. Therein, the convergence time of an FIIR cell is depicted as a function of L, and it is defined as the time required for the combination reach 85 % of AF2 steady-state error.
Acknowledgment: The work of the first two authors has been supported
by CNPq and CAPES, Brazil. References
Simulation Results: In the simulations below, the FIIR cell identifies the plants which frequency responses are in Figs. 4(a) and 4(b). Both are particularly hard due their peculiar poles-zeros setup [6, 13]: the first is a Butterworth system with clustered poles and the second is a notch with underdamped poles. In both cases, M = 6, u(i) is a zero mean, Gaussian white signal with power σu2 = 1 and v(i) has a power of σv2 = 1 × 10−3 .
1 B. Beliczynski, I. Kale, and G.D. Cain, “Approximation of fir by iir digital filters: an algorithm based on balanced model reduction,” Signal Processing, IEEE Transactions on, vol. 40, no. 3, pp. 532 –542, mar 1992. 2 C.G. Lopes, E.H. Satorius, P. Estabrook, and A.H. Sayed, “Adaptive carrier tracking for mars to earth communications during entry, descent, and landing,” Aerospace and Electronic Systems, IEEE Transactions on, vol. 46, no. 4, pp. 1865–1879, 2010. 3 Paulo S. R Diniz, Adaptive Filtering: Algorithms and Practical Implementation, Springer, 2008. 4 Ali H. Sayed, Adaptive Filters, Wiley-IEEE Press, 2008. 5 Phillip A. Regalia, Adaptive IIR Filtering in Signal Processing and Control, CRC Press, 1994. 6 P.M.S. Burt and M. Gerken, “A polyphase iir adaptive filter: error surface analysis and application,” in Acoustics, Speech, and Signal Processing, IEEE International Conference on, apr 1997, vol. 3, pp. 2285–2288. 7 H.K. Kwan, “Adaptive iir digital filters with saturation outputs for noise and echo cancellation,” Electronics Letters, vol. 38, pp. 661–663, 2002.
Fig. 4. Frequency response of the simulation scenarios
8 H.C. So, “Sign-sign algorithm for unbiased iir filtering in impulsive noise,” Electronics Letters, vol. 40, no. 11, pp. 704–706, 2004.
The plots that follow show the EMSE of AF1 (“guide”), FIIR cell with weight transfers (“WT”, refer to Eq. 7) and without (“No-WT”), where the superiority of the former is clear. For reference, the EMSE of a stand-alone IIR AF is also shown. In Fig. 5, the combination identifies the Butterworth system with M1 = 10, µ1 = .02 and µ2 = .005. In this case, the error surface of a regular IIR AF exhibits near constant error regions around the global minimum on which gradient based algorithms adapt slowly [6]. The combination considerably improves performance, as AF1 guides AF2 towards the vicinity of the minimum rapidly. Only the first iterations are shown as the stand-alone IIR takes too long to reach the combination’s steady-state. In Fig. 6, the combination identifies the notch with M1 = 15, µ1 = .007 and µ2 = .03. When i = 4 × 104 , the notch frequency changes suddenly (abrupt change in wo ). Although the standard IIR nearly stagnated, the FIIR cell responded fast, with a much superior performance.
9 A. Carini, V.J. Mathews, and G.L. Sicuranza, “Sufficient stability bounds for slowly varying direct-form recursive linear filters and their applications in adaptive iir filters,” Signal Processing, IEEE Transactions on, vol. 47, no. 9, pp. 2561 –2567, sep 1999. 10 J. Arenas-Garcia, A.R. Figueiras-Vidal, and A.H. Sayed, “Mean-square performance of a convex combination of two adaptive filters,” Signal Processing, IEEE Transactions on, vol. 54, no. 3, pp. 1078 – 1090, 2006. 11 M.T.M. Silva, V.H. Nascimento, and J. Arenas-Garcia, “A transient analysis for the convex combination of two adaptive filters with transfer of coefficients,” in Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, march 2010, pp. 3842 –3845. 12 Franklin T. Luk and Sanzheng Qiao, “A fast eigenvalue algorithm for hankel matrices,” Linear Algebra and its Applications, vol. 316, no. 1, pp. 171–182, 2000, Special issue for the 60th birthday of R. J. Plemmons. 13 Li-xin Wang, Wei Xiang, and Jing-yuan Zhang, “The monte carlo framework for adaptive iir filter and arma parameters estimation,” in Wireless, Mobile and Sensor Networks. IET Conference on, 2007, pp. 725–728.
Conclusion: The FIIR cells are robust, have a superior convergence rate and handle abrupt non stationarities better than a standard IIR AF while avoiding the stability checks. In stationary scenarios, a properly designed L minimizes the number of transfers needed to accelerate AF2 and decreases 2
