The denominator and numerator estimation problems are theoretically decoupled into separate problems. The decoupled criteria have reduced dimensionalities ...
SIGNAL
PROCESSING Signal Processing 42 (1995) 19l-206
Design of denominator
separable 2-D IIR filters
Arnab K. Shaw*,’ Electrical Engineering Department,
Wright State University, Dayton, OH-45435, USA
Received 14 May 1993; revised 15 August 1994
Abstract Optimal design of an important class of two-dimensional (2-D) digital IIR filters from spatial impulse response data is addressed. The denominator of the desired 2-D filter is assumed to be separable into two 1-D factors. The filter coefficients are estimated by minimizing the /~--norm of the error between the prescribed and the estimated spatial-domain responses. The denominator and numerator estimation problems are theoretically decoupled into separate problems. The decoupled criteria have reduced dimensionalities. The denominator criterion is simultaneously optimized w.r.t. the coefficients in both dimensions using an iterative algorithm. The numerator coefficients are found in a straightforward manner. If the desired response is known to be symmetric, the proposed algorithm can be constrained to produce optimal denominators which are identical in both domains. The performance of the algorithm is demonstrated with simulation examples. Zusammenfassimg Es wird der optimale Entwurf einer wichtigen Klasse von Zweidimensionalen (2-D) digitalen IIR-Filtern iiber die riumliche Impulsantwort betrachtet. Der Nenner des gewiinschten 2-D-Filters wird als separierbar in zwei l-DFaktoren angenommen. Die Filterkoefhzienten werden durch Minimierung der /,-Norm des Fehlers zwischen vorgeschriebener und geschatzter rlumlichen Antwort bestimmt. Die Nenner- und Zahler-Schiitzprobleme werden theoretisch in separate Probleme zerlegt. Die entkoppelten Kriterien sind von reduzierter Dimensionalitlt. Das Nenner-Kriterium wird simultan optimiert, wobei fur die KoeXizienten in beiden Richtungen ein iterativer Algorithmus angewendet wird. Die ZIhlerkoetTizienten werden auf direktem Wege bestimmt. Falls der gewiinschte Frequenzgang symmetrisch ist, kann der vorgeschlagene Algorithmus so spezifiziert werden, dag die Nenner identisch in beiden Richtungen sind. Die Eigenschaften des Algorithmus werden durch Simulationsbeispiele veranschaulicht. Rbumi! Nous nous inttressons dans cet article a la conception optimale dune classe importante de filters IIR numeriques bi-dimensionnels (2-D) a partir de don&es de rtponse impulsionnelle spatiale. Le dtnominateur du filtre 2-D desire est suppose etre separable en deux facteurs 1-D. Les coefficients du filtre sont estimes par une minimisation de la norme e2 de l’erreur entre les rtponses prescrite et estimee dans le domaine spatial. Les problemes d’estimation du denominateur et
*Corresponding author: Tel: (513) 873-5064. Fax: (513) 873-5009. ’ Research partly supported by AFOSR-F49620-90-C-09076 and by AFOSR-F49620-93-l-0014. 0165-1684/95/$9.50 0 1995 Elsevier Science B.V. All rights reserved SSDI 0165-1684(94)00127-8
192
A.K. Shaw 1 Signal Processing 42 (1995) 191-206
du numtrateur sont decouples de man&e thiorique en deux problemes s&pares. Les criteres decouples ont une dimensionalitt rbduite. Le critere pour le denominateur est optimalist simultankrnent par rapport aux coefficients dans les deux dimensions a l’aide dun algorithme iteratif. Les coefficients du numtrateur sont obtenus de man&e immediate. Si la rtponse dtsirke est connue btre symitrique, l’algorithme propose peut etre contraint a produire des dinominateurs optimaux qui sont identiques dans les deux domaines. Les performances de l’algorithme sont mises en lumiere par des exemples de simulation. Keywords:
Two-dimensional digital filter design; Spatial impulse response fit; Separable denominator; Least-squares
method
1. Introduction Two-dimensional IIR filters are used in various applications of 2-D signal processing including, digital image processing, seismic or geophysical signal processing, video communication, etc. [l, 2, 4-7, 9-12, 14-16, 17,211. As in 1-D processing, 2-D IIR filters can match a desired specification with significantly reduced number of coefficients than required for an equivalent 2-D FIR filter. Synthesis of 2-D IIR filters from prescribed spatial domain impulse response data is an important and challenging design problem and has received considerable attention in recent literature [l, 2,4-7,11, 17,211. Spatial-domain design of 2-D IIR filters is analogous to 1-D recursive filter design based on time-domain specifications. The practical advantages of denominator-separable 2-D recursive filters include ease of design and modular implementation, simpler stability test and stabilization. Furthermore, various widely used circularly symmetric 2-D filters inherently conform with denominatorseparable structures. Most 2-D filter design algorithms are primarily extensions of existing 1-D methods. In particular, Shanks et al. [ 171 had extended the work of Shanks [ 161; Cadzow [l] and Shaw and Mersereau [21] utilized many of the general non-linear optimization methods; and Shaw and Mersereau [Zl] also extended the work of Steiglitz and McBride [22]. The 1-D work of Mullis and Roberts [ll] was further extended and applied to the 2-D case in [6]. The approaches noted above do not minimize the true spatial impulse response error, though it may be mentioned that the extension of Steiglitz-McBride method in [22] closely approximates the true fitting error. For the strictly proper case,
i.e., when the numerator order is one less than that of the denominator, Evans and Fischl (EFM) had proposed an optimal method for synthesis of 1-D IIR filters [3]. The 2-D filter synthesis algorithm presented here is a generalization of EFM to 2-D. The proposed solution is optimal in the sense that it minimizes the true and complete spatial error criterion for synthesizing strictly-proper 2-D IIR filters. EFM has been found to be highly accurate for 1-D filter design. A modified and complex version of the EFM with certain symmetry constraints has also been shown to be effective for maximum-likelihood 1-D and 2-D frequency-wave number estimation [S, 18,201. Generalization of EFM for strictly proper 2-D filter design has also been considered previously [4,5], but it appears that the full potential of EFM has not been utilized in the 2-D case. Specifically, it will be shown in this paper that the complete error criterion encompassing the entire subspace orthogonal to the model fitting error was not optimized in [4,5]. Instead, two suboptimal error criteria were formed in each domain and the filter coefficients were optimized in the two dimensions independently. In this paper, a 2-D version of EFM is developed for optimal design of 2-D recursive filters from prescribed spatial domain data. The complete basis space orthogonal to the spatial fitting error will be identified and the corresponding error criterion will be shown to be dependent only on the 2-D filter parameters. Similar to 1-D EFM, the non-linear error criterion will be decoupled into a purely linear and a non-linear subproblem. For the separable denominator case, it is also shown that the error vector possesses a quasi-linear relationship with the denominator coefficients in both domains
193
A.K. Shaw / Signal Processing 42 (1995) 191-206
simultaneously. Unlike several existing 2-D methods [l, 2, 4, 5, 171, the exact fitting error is minimized w.r.t. the filter coefficients in both dimensions simultaneously. Simultaneous optimization is particularly effective for synthesizing 2-D filters with symmetric impulse response which are quite common in practice. In such cases, the error criterion can be appropriately constrained to produce identical denominators in both domains ensuring symmetry in the estimated spatial response. The paper is arranged as follows: In Section 2, the least-squares problem is stated. In Section 3, the preliminaries for the non-separable numerator and separable denominator case is given. In Section 4, the new orthogonal basis spaces are defined, the error criterion is derived and the computational algorithm is summarized. In Section 5, some simulation results are given and finally, in Section 6 some concluding remarks along with directions for future work are included.
2. Problem statement and formulation
Note that for the strictly proper case of EFM, nl = ml - 1 and n2 = m2 - 1. If the kl x k2 first quadrant samples are assumed to be significant, H(zl,z2) can also be written as =
zwz2,
where z1 A[1 z;’ z24[f
z;’
(2)
T 1 3
... zl(-kl-1)
. . . z;(k2-1)]T
h(kl - 1,0) h(kl - 1,l)
...
h(kl - l,kz - 1)
h$
IL hkz_] :I hz .
,
(4)
where hi denotes the ith column of H. Next let the prescribed space-domain impulse response matrix be denoted as
x(0,1) r x(0,0) ...
x(1,1)
x(O,kz- 1)
...
-I
x(kl - 1,0) x(kl - 1,1) ... x(ki - l,kz - 1)
W) and the corresponding vector be formed as
r x17
I .:I. x2
In this paper, the following 2-D least-squares synthesis problem is addressed. Given the 2-D spatial impulse response matrix X, estimate p and q by optimizing the following error criterion: r$i Ile112~Ilx-hl12
withp(O,O)= 1,
(7a)
where !z%I(O90) q(l,O) *‘* q(n1,u2)lT
(7b)
and
and
HP
r4
xe
In general, a 2-D rational function H(zl,z2), with non-decomposable numerator and denominator is described as
Wz,,z2)
Define a vector by stacking the columns of H as follows:
P’CP@,O)
I. (3)
P(40)
a’. P(W,mz)l’.
(7c)
This problem is non-linear in p and standard gradient-based optimization algorithms have been used for 1-D as well as for 2-D designs [l, 2, 211. But these generic algorithms do not make effective use of the matrix structures inherent in this particular problem and they are known to be sensitive to initialization. Several suboptimal algorithms based on linearization of the true non-linear criterion have also been
194
A.K. Shaw / Signal Processing 4.2 (1995) 191-206
proposed [2, 11, 211. In this work, the exact fitting criterion defined in (7) will be theoretically decoupled into a purely linear problem for q and a non-linear problem for p. Furthermore, the non-linear criterion will be shown to possess a quasi-linear relationship to the unknown denominator coefficients. This will lead to the formulation of an iterative algorithm for its minimization.
d(m2)
0
...
0
... .
0
dil)
..:
d(0)
.... ..:
’ d(mA d(m2 - 1)
d(m2 - 1) d(m2) . DA
d(O) 0
. 0
4-J
[Wkzx(kz-mz)
E
(lob)
3
3. Design with separable denominator and non-separablenumerator
c(0)
0
In this case, the 2-D rational transfer function can be written as Cl P
Define &[c(O)
c(1) ‘.. c(m,)]T
(8b)
c(1) c(0)
... ...
c(ml - 1) c(ml - 2)
.
.
(j 0
0 0 .
*.: .‘. .
. 40) 0
0
0
..:
(j
E [Whxw
5
and d&Cd(O) d(1) ... d(m2)lT.
(84
Multiplying both sides of (8a) by C~‘&c(i)z;‘x an d equating the coefficients of the I;:, d(j)z;j same powers of z- ’ and z; ’ [4],
41)
...
0
40)
...
0
0
0
.. .
0
0
..
0
40)
D1p
CDT 8 CTlh= q,
(94
0
[DT @ICT]h = 0,
(9’4
0
[DT @ CT]h = 0,
(9c)
[D’ @ CT]/8= 0,
(9d)
[Zk* @ C’]h
= 0,
(9e)
[DT 0 Z,,]lr = 0,
(9f)
dh dh 40)
ER kzxmz. 9
where
CA
40)
c(1)
0
c(0)
*.* c(mr) ... C(rnl - 1)
uw
Z,, E [Wk’ ’ k1and zkzE [wkzx k2are identity matrices and @ denotes the Kronecker product [13]. If h, c and d are known, the numerator vector q can be calculated using (9a). But in practice, h, c and d need to be estimated from the prescribed response X. If h is replaced by the prescribed x in 9(b)-9(f), the righthand sides will not be equal to zero, but will result in the following equation errors: [D; @ CT> = [D: @ CT] [h + e] = [DT @ CT]eke&,
(114
[D’ fj3 t2:j.x = [DT @ CT] [II + e] = [DT 0 CT]eke&,
(1W
195
A.K. Shaw / Signal Processing 42 (1995) 191-206
[DT Q C’l_x = [DT Q C’][h + e) =
[DT Q CT]eAe&,
Ulc)
[Zkl0 CT* = Vkz0 CT1 P + el = [Zk, Q CT]e ee&,
(114
[DTQZkJx = [DTQZk,l[h + el =
[II’
A 5 @Zk,]e=eeq.
(I le)
In (lla)
(284
-~o~c+%@%lx = [Ikz0 c??c + 9D0 Ik,- 9%0 9Jx. Note that in dependence criterion for cient vectors
CW
where
W)
W(c) e C(CTC)_ l
this final form of the error, there is no on either T or Q. Hence, the error determining the denominator coeffic and d can now be written as
VW
and W(d)%I(DTD)-‘.
(28c)
X1 and X2 are constructed X as [lS, 201,
from the elements in
x’ 2 (XT x; . . . X,‘,)‘,
(29)
where the (1,k)th term of Xi is formed as (26) Eqs. (26) and (9a) represent the desired decoupled criteria for determining the denominator and numerator coefficients, respectively. Optimization of (26) would produce the optimal c and d, denoted as, co and d”, respectively. Letting e“ denote the minimized error corresponding to the optimum denominator coefficients, the optimum spatial-response vector h can be found from ha&, - &’
(27)
This h” can then be used in (9a) to obtain the optimal numerator vector, 4’. Analyzing the criteria in (26) it is apparent that the first two terms are the orthogonal projections of the data x onto the parameter spaces of each of the two spatial dimensions. The third term is the orthogonal projection common to both dimensions but is subtracted once because the common (or, joint) projections have already been included once in each of the first two terms. It is very interesting to note that this criterion is quite analogous to the standard formula of the Probability of Union of two
Xj(l,k)PXi(E - k + ml + 1) for i = 172,a.., k2
(304
and
X2&
X m+l
...
Xl
X,,+2
...
X2 PW
“. xkz - mz ... !1. i: Xkz Eq. (28a) is one of the key results derived in this paper. It clearly shows that both the unknown coefficient vectors c and d appear simultaneously in a ‘quasi-linear’ relationship w.r.t. the true error vector. This quasi-linear relationship can be exploited for simultaneous optimization of the criterion in (26) w.r.t. c and d. The algorithm is similar in flavor to the ones in [3-5, 8, 18, 201, except that both the denominators are optimized and estimated simultaneously. Specifically, the algorithm iteratively minimizes the e2-norm of the error vector formed in (26) in two phases. In Phase-l, the Wmatrices are treated as constants and are formed using the estimates of c and d obtained at the
198
A.K. Show / Signa/ Processing 42 (1995) 191-206
previous iteration. In Phase-2, the estimates are improved upon by setting the gradient of the complete error norm to zero. The iterations are initialized by setting c = [l 0 ... OIT and d=[l 0 ... 0] T. The iterations are continued until the changes in the estimates in successive iterations become very small. It may be mentioned here that extensive simulation experience in 1-D [S, 18, 193 as well as for 2-D cases [ 18, 201 indicate that Phase-l itself produces very good estimates of the filter coefficients and in most cases, there may not be any need for invoking Phase-2 at all. It may be noted here that in [4, 51, the complete error e(c,d) in (26) was not optimized. Symmetric spatial response - A special case. Many 2-D filters used in image processing are symmetrically shaped in the spatial domain. Some notable examples are, Gaussian and circular filters. In designing such spatially symmetric 2-D filters, the methods in [4, 51 sometimes produced slightly different sets of denominator polynomials. Hence, the estimated spatial response may not possess the desired symmetry. This problem may be attributed to separate estimation of the individual denominators. In the proposed approach, both the denominators are optimized simultaneously by minimizing the entire error in (28a). If necessary, the desired symmetry may be imposed by setting, c = d in (28a) at the outset. For this special but very important special case, (28a) would have the following form:
tered 2-D filter design problems are considered. The four design examples addressed are, Gaussian, Laplacian-of-Gaussian, 2-D bandpass and 2-D low-pass filters. The results have been compared with the performance of a 2-D version of Prony’s equation error-based method [9]. The comparisons are given in terms of the deviation from the desired (or true) response when similar number of coefficients were used in designing with either algorithm and also in terms of the number of coefficients required for obtaining equivalent match with both algorithms. For providing a more practical understanding, the optimum coefficient values were quantized while computing the deviations. In all cases, it was found that 16-bit quantization with 4 bits allocated for sign and integer parts are necessary for obtaining reasonable match. In case of the proposed method, each separate 1-D denominators were factored into second and first order (if needed) factors before quantizing. It may be noted that for the equation error method [9], second-order factoring of non-separable 2-D denominator polynomial is not straightforward, hence it has not been implemented.
5.1. Gaussian filter design The spatial response of a quarter plane Gaussian filter defined over the first quadrant is given by H(i,j)
e(c) P [((ZkZ- PC) 0 K,(e))X’ + (@AC) owmc,
(31)
where the subscripts of Wdenote leading dimensions which may be unequal. Minimization of the norm of the error in (31) will result in a single set of optimal coefficients in both dimensions. This is one of the major advantages of the proposed approach over the ones in [4, 51 where separate optimization in each domain does not necessarily guarantee identical denominator coefficients in both domains. 5. Simulation results In order to demonstrate the effectiveness of the proposed algorithm, several commonly encoun-
=
0.256322
e[-~.~~~~~~~(~-~)2+(~-~)*~~,
where, (i,j) E Sf and the support Sr is given by Sr = {(i,j)lO < i < 19; 0
