Efficient Design of Sparse FIR Filters with Optimized Filter Length Aimin Jiang1 , Hon Keung Kwan2 , Yibin Tang3 , Yanping Zhu4 1,3
College of IoT Engineering, Hohai University, Changzhou, China 213022, Emails: {jiangam,tangyb}@hhuc.edu.cn Department of Electrical and Computer Engineering, University of Windsor, Windsor, Canada N9B 3P4, Email:
[email protected] 4 College of Information Science and Technology, Changzhou University, Changzhou, China 213164, Email:
[email protected] 2
Abstract—A large number of experiments have demonstrated that for an FIR filter the sparsity of filter coefficients is highly related to its filter order. However, traditional sparse FIR filter design methods focus on how to increase the number of zerovalued coefficients, but overlook the impact of filter orders on design performance. As an attempt to jointly optimize filter length and sparsity of an FIR filter, a novel method is proposed in this paper to design sparse linear-phase FIR filters. With peak error constraints, the objective function of the design problem is formulated as a combination of the sparsity of filter coefficients and a measure of the effective filter order. Then, the design problem is then recast as a weighted l0 -norm optimization problem, which is solved by an efficient numerical method based on the iterative-reweighted-least-squares (IRLS) algorithms. Experimental results illustrate that the proposed method can efficiently reduce the effective filter order while enhancing the sparsity of an FIR filter. Keywords—Iterative-reweighted-least-squares (IRLS), l0 norm, quadratic programming, linear programming, sparse FIR filter.
I. I NTRODUCTION FIR filters are widely used in a variety of applications on signal processing and communications [1]. Given a set of specifications, traditional design methods try to find an FIR filter whose frequency response (or magnitude response) can best approximate the ideal one under some criterion. Using this design strategy, a large number of FIR filter design problems can be cast as convex optimization problems and then solved by various efficient numerical methods (see [2], [3], and references therein). Another design strategy, which faces more challenges, aims to reduce the implementation complexity of an FIR filter subject to some other constraints. Extensive efforts have been dedicated to designing FIR filters with binary coefficients [4] or power-of-two coefficients [5], or making use of special structures, such as frequency-response masking technique [6], [7] and subexpressions among filter coefficients [8], [9]. Lately, with the advance of sparse representation and its applications in signal processing [10], researchers pay much attention on sparse FIR filter designs. A sparse FIR filter contains as few nonzero-valued coefficients as possible. The principle advantage of using sparse FIR filters is that additions This work was supported in part by the Fundamental Research Funds for the Central Universities of China under grant 2011B11214, the National Nature Science Foundation of China under grant 61101158, and the Natural Science Foundation of Jiangsu Province,China under Grant BK20130238.
978-1-4799-3432-4/14/$31.00 ©2014 IEEE
and multiplications corresponding to zero-valued coefficients are omitted such that the implementation complexity is consequently reduced. For an FIR filter, the sparsity of filter coefficients is generally evaluated by l0 (quasi-)norm. Theoretically speaking, one can find an FIR filter with the minimum number of nonzero-valued coefficients by exhaustive search. However, the computational complexity of this design strategy is too high to design FIR filters of large order. Thereby, a more reasonable design strategy is to greedily determine locations of zero-valued filter coefficients based on certain selection rules. In [11], the orthogonal matching pursuit (OMP) algorithm is employed to tackle sparse FIR filter design problems. Although it is not guaranteed to achieve optimal solutions, its computational complexity is dramatically reduced. Inspired by the basis pursuit algorithm, several design methods [1214] are proposed to achieve sparse FIR filters by replacing the original l0 norm by l1 norm. In [12], two heuristic approaches are presented to design sparse linear-phase FIR filters. Using two index selection rules (i.e., minimum-increase and smallest-coefficient), the first approach gradually nullifies filter coefficients. The second approach combines l1 norm and smallest-coefficient selection rule to greedily eliminate nonzero-valued coefficients. In [13] and [14], l1 norm is also employed to design sparse filters. Both algorithms aim to minimize the maximum approximation error and the l1 norm of filter coefficient vector. The iterative reweighted l1 minimization (IRL1) scheme is employed by [14] to achieve sparse solutions. Another class of design methods [15], [16] are inspired by the iterative thresholding/shrinkage (IST) algorithms. It is proven that these methods can converge to locally optimal solutions under certain conditions. A large number of experiments demonstrate that for an FIR filter the sparsity of filter coefficients is related to its filter order. Generally speaking, higher filter orders may lead to sparser results. However, this does not mean that by increasing filter orders we can always achieve FIR filters with lower implementation complexity, which only depends on the number of nonzero-valued coefficients. On the other hand, for linear-phase FIR filters, higher orders imply that group delays of the corresponding filters become larger, which is unfavorable in many applications. Unfortunately, the aforementioned design methods focus on how to enhance the sparsity of filter coefficients under a set of specifications (e.g., a fixed filter order which is generally arbitrarily selected) and thus
966
overlook the impact of filter orders on final design results. As an attempt to address this issue, we shall develop an efficient sparse FIR filter design method in this paper. A regularization term is combined with the sparsity of filter coefficients, such that a higher filter order will be penalized during the design procedure. The rest of the paper is organized as follows. In Section II, the design problem is formally expressed as an l0 -norm optimization problem. The proposed design method is then developed in Section III. The effectiveness of the proposed method is verified by two sets of experiments in Section IV. Finally, we conclude the paper in Section V.
Suppose that an N th-order linear phase (LP) FIR filter is under consideration and that N is sufficiently large. For ease of presentation, we only describe designs of sparse LP FIR filters of type I in this paper. However, the proposed method can be applied to address design problems of sparse LP FIR filters of other types. Based on the symmetry property of an LP FIR filter, free variables to be optimized are n = 0, . . . , M
(1)
where M = N/2 and hn s are impulse responses of the LP FIR filter. Let D(ω) and ΩI be the ideal magnitude response and the union of frequency bands of interests, respectively. Then, traditional sparse FIR filter design methods consider the following optimization problem min x
s.t.
kxk0 |eMAG (x, ω)| ≤ γ(ω),
(2a) ∀ω ∈ ΩI
(2b)
where x = [x0 x1 . . . xM ]T ∈ RM +1 , and kxk0 represents l0 (quasi-)norm which counts the number of nonzero-valued elements of x. In (2b), eMAG (x, ω) defined below represents the magnitude approximation error, which is upper bounded by a specified γ(ω), T eMAG (x, ω) = vM (ω)x − D(ω),
vM (ω) = [2 cos M ω . . . 2 cos ω 1]T .
Directly taking (5) into (2) yields a more complicated design problem. Thereby, we first rewrite the design problem as a more tractable form, which is then solved by an efficient iterative procedure developed in this section. Note that ρ(x) ≤
=
M X m X
2
xi 0 m=0 i=0 M X
(6)
(M + 1 − m) x2m 0 .
m=0
Using (6) and the identity x2m 0 = kxm k0 , we finally have
II. P ROBLEM F ORMULATION
xn = hn = hN −n ,
III. P ROPOSED D ESIGN M ETHOD
(3) (4)
As mentioned before, the design performance of traditional methods is related to the value of N , which is arbitrarily specified by designers. To achieve an FIR filter whose implementation complexity is maximally reduced, we shall incorporate a regularization term which accounts for the effective filter order. Note that, given N , the filter length of an FIR filter can be reduced by forcing xm = 0 for m = 0, . . . , T . The effective ˆ = N − 2T . However, the order of the resulting FIR filter is N value of T has to be optimized along with the sparsity kxk0 . To this end, the objective function of (2) is further modified as kxk0 +β ·ρ(x) where β is a specified regularization parameter and ρ(x) is defined by
x20
2 2
x0 + x1
ρ(x) = (5)
. ..
.
x2 + x2 + . . . + x2
0 1 M 0
min x
s.t.
M X
am kxm k0 m=0 T vM (ω)x − D(ω)
(7a) ≤ γ(ω),
∀ω ∈ ΩI
(7b)
where am = 1 + β (M + 1 − m) ,
m = 0, 1, . . . , M.
(8)
The design problem above is a weighted l0 -norm optimization problem. The proposed approach to tackle (7) is based on the iterative-reweighted-least-squares (IRLS) algorithm. The basic idea behind the IRLS algorithm is that l0 -norm optimization problems can be robustly resolved by replacing l0 norm of an vector by its convex relaxation form, i.e., l1 norm. Furthermore, by defining X = diag(|x0 |, . . . , |xM |), we have kxk1 = xT X−1 x. However, before solving (7), we cannot obtain X. To overcome this difficulty, we may replace each diagonal element of X by an approximate value which is computed by the solution obtained in the previous iterative step. Specifically, in the lth iterative step, the design problem (7) is modified as min xT W(l) x x T s.t. vM (ω)x − D(ω) ≤ γ(ω),
(9a) ∀ω ∈ ΩI
(9b)
where W(l) is a diagonal matrix whose mth diagonal element is computed by am (l) p , (10) Wm = (l−1) xm + σ (l−1)
(l−1)
(l−1)
x1 . . . xM ]T is the solution obtained x(l−1) = [x0 in the previous iterative step, σ denotes a small number (e.g., 10−6 used in our experiments) to avoid the division by zero in (10), and the power p ≥ 1 is used to accelerate the convergence (l) of the iterative procedure. As Wm s are all positive, (9) is a quadratic programming problem, which can be reliably solved by a variety of efficient numerical solvers (e.g., SeDuMi [17]). The solution obtained from (9) is then taken as x(l) into the next iteration. The iterative procedure continues until the update between x(l−1) and x(l) is negligible, or the maximum number of iterations is reached. Due to the introduction of the regularization term ρ(x), xm s with small indices tend to 0. Thus, after the iterative procedure, ˆ + 1 by removing ˆ of length M we may obtain an FIR filter x zeros from the forefront of x. Finally, we can further refine
967
Table I M AJOR STEPS OF PROPOSED DESIGN Inputs: Outputs: Step 1: Step 2: Step 3: Step 4: Step 5:
METHOD
N , γ(ω), D(ω) ˆ ˆ (or h) x Set l = 0 and initialize x(0) ; Set l = l + 1 and compute W(l) by (10). Then, solve (9) to achieve x(l) ; If x(l−1) − x(l) 2 is sufficiently small or l is larger than the maximum number of iterations, go to Step 4; Otherwise, go to Step 2; Remove zeros from the forefront of x. The resulting filter is ˆ of length M ˆ + 1 (or N ˆ + 1); ˆ (or h) denoted by x Solve (11) to refine the design result. Table II D ESIGN SPECIFICATIONS OF E XAMPLE 1 Passband region Stopband region Filter order Passband magnitude Stopband magnitude
(a) Magnitude response
[0, 0.0436π] [0.0872π, π] 100 Within ±0.5dB of unity Below −35dB
the design result by solving the following linear programming problem min δ x,δ T s.t. vM ˆ (ω)x − D(ω) ≤ γ(ω) + δ, xm = 0,
m∈Z
(11a) ∀ω ∈ ΩI
(11b) (11c)
where Z represents a subset of indices at which the correˆ are equal to 0. Finally, we restore sponding elements of x ˆ = 2M ˆ by impulse responses of a sparse FIR filter of order N (1). For completeness, the major steps of the proposed design method are summarized in Table I.
(b) Magnitude response in passband Fig. 1. Magnitude responses of FIR filters obtained in Example 1. Table III D ESIGN RESULTS OBTAINED IN E XAMPLE 1
IV. S IMULATIONS In this section, two examples are presented to demonstrate the effectiveness of the proposed design approach. For ease of computation, constraint (2b) is imposed on a dense grid of frequency points equally sampled over [0, π]. The number of sampling points is 501 in our designs. The power p used in (10) is chosen equal to 1.5 in each example. At the beginning of the iterative procedure, all the elements of x(0) are set to 1. The maximum number of iterations is chosen as 100 in our designs. Quadratic programming problem (9) and linear programming problem (11) are both solved by SeDuMi [17]. The first example is to design a lowpass LP FIR filter. The design specifications are given in Table II. In this example, the regularization parameter β used in (8) is set to 1. After 40 iterations, the iterative procedure converges. Magnitude responses of the FIR filter obtained by the proposed method are depicted in Fig. 1. Using the same set of specifications, we employ the smallest-coefficient method and the minimum 1-norm method both proposed in [12] to design sparse FIR filters. Design results are also illustrated in Fig. 1. The effective ˆ and the number of nonzero-valued filter coefficients order N denoted by LNZ are summarized in Table III. Note that all the design methods can achieve FIR filters with effective orders lower than N . However, compared to the other two design
Proposed Smallest-coefficient [12] Minimum 1-norm [12]
ˆ N 76 80 78
LNZ 55 57 65
approaches, the proposed method achieves a sparse FIR filter with the fewest number of nonzero-valued coefficients and the lowest effective filter order. The second example is to design a bandpass LP FIR filter. The detailed specifications are given in Table IV. In this design, the regularization parameter β used in (8) is chosen equal to 0.1. The performance of the proposed method is compared to that of the regularized l1 -norm method [13], which tries to resolve the following problem min t + µ kxk1 x,t T s.t. vM (ω)x − D(ω) ≤ t,
(12a) ∀ω ∈ ΩI .
(12b)
After solving the design problem formulated above, hard thresholding is further applied on filter coefficients obtained by (12) to retrieve a sparse FIR filter. Specifically, all the coefficients with magnitudes smaller than a thresholding parameter (e.g., 10−6 used in this example) are forced equal
968
Table IV D ESIGN SPECIFICATIONS OF E XAMPLE 2 Passband region Stopband region Filter order
[0.4π, 0.6π] [0, 0.265π] ∪ [0.73π, π] 70
Table V D ESIGN RESULTS OBTAINED
Proposed Regularized l1 -norm [13]
IN
E XAMPLE 2
ˆ LNZ N (µ=1/0.5/0.1/0.05/0.01) (µ=1/0.5/0.1/0.05/0.01) 44/56/64/64/68 13/15/23/21/41 60/56/68/70/70 19/23/31/57/69
to zeros. In our designs, the regularization parameter µ used in (12) is set to 1, 0.5, 0.1, 0.05, and 0.01, respectively. For a fair comparison, the upper bound γ(ω) of the magnitude approximation error is chosen as that of the sparse FIR filter obtained by the regularized l1 -norm method. All the designs by the proposed method converges in at most 23 iterations. The effective filter order and the number of nonzerovalued coefficients in each design are summarized in Table V. Magnitude responses of FIR filters obtained by both design methods for µ = 0.1 are depicted in Fig. 2. It can be observed that the proposed method can achieve much better designs than the regularized l1 -norm method in terms of both effective filter orders and sparsity of filter coefficients.
(a) Magnitude response
V. C ONCLUSIONS A novel method is developed in this paper to design sparse LP FIR filters. Compared to traditional design methods, the impact of filter orders on final designs is explicitly taken into account. The design problem is cast as a weighted l0 norm optimization problem, which is then tackled by an efficient numerical method based on the IRLS algorithm. Simulation results reveal that, for a given set of specifications, the proposed method is capable of automatically determining an appropriate filter order while improving the sparsity of an FIR filter.
[2] [3]
[4] [5] [6] [7]
Fig. 2. Magnitude responses of FIR filters (µ = 0.1) obtained in Example 2.
[8] [9] [10] [11]
R EFERENCES [1]
(b) Magnitude response in passband
A. Antoniou, Digital Filters: Analysis, Design, and Applications, 2nd ed. New York: McGraw-Hill, 2000. T. N. Davidson, “Enriching the art of FIR filter design via convex optimization,” IEEE Signal Process.Mag., vol. 27, pp. 89–101, May 2010. K. M. Tsui, C. Chan, and K. S. Yeung, “Design of FIR digital filters with prescribed flatness and peak error constraints using second-order cone programming,” IEEE Trans.Circuits Syst. II, vol. 52, pp. 601–605, Sep. 2005. M. Bateman, B. Liu, “An approach to programmable CTD filters using coefficients 0, +1, and −1,” IEEE Trans. Circuits Syst., vol. 27, pp. 451–456, Jun. 1980. D. Li, Y. C. Lim, Y. Lian, and J. Song, “A polynomial-time algorithm for designing FIR filters with power-of-two coefficients,” IEEE Trans. Signal Process., vol. 50, pp. 1935–1941, Aug. 2002. W.-S. Lu and T. Hinamoto, “Optimal design of frequency-response masking filters using semidefinite programming,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 50, no. 4, pp. 557–568, Apr. 2003. Y. C. Lim and R. Lian, “Frequency-response masking approach for digital filter design: complexity reduction via masking filter factorization,” IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 41, no. 8, pp. 518–525, Aug. 1994.
[12] [13] [14] [15] [16] [17]
969
D. Shi and Y. J. Yu, “Design of linear phase FIR filters with high probability of achieving minimum number of adders,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 58, pp. 126–136, Jan. 2011. Y. J. Yu and Y. C. Lim, “Design of linear phase FIR filters in subexpression space using mixed integer linear programming,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 54, pp. 2330–2338, Oct. 2007. M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. NewYork: Springer, 2010. D. Mattera, F. Palmieri, and S. Haykin, “Efficient sparse FIR filter design,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2002, vol. 2, pp. 1537–1540. T. Baran, D. Wei, and A. V. Oppenheim, “Linear programming algorithms for sparse filter design,” IEEE Trans. Signal Process., vol. 58, pp. 1605–1617, Mar. 2010. W.-S. Lu and T. Hinamoto, “Digital filters with sparse coefficients,” in Proc. IEEE Int. Symp. Circuits Syst., Paris, France, May 2010, pp. 169–172. C. Rusu and B. Dumitrescu, “Iterative reweighted l1 design of sparse FIR filters,” Signal Process., vol. 92, pp. 905-911, Apr. 2012. A. Jiang, H. K. Kwan, and Y. Zhu, “Peak-error-constrained sparse FIR filter design using iterative SOCP,” IEEE Trans. Signal Process., vol. 60, pp. 4035–4044, Aug. 2012. A. Jiang and H. K. Kwan, “WLS design of sparse FIR digital filters,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 60, pp. 125–135, Jan. 2013. J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones,” Optim. Methods Softw., vol. 11/12, pp. 625–653, 1999.
