A new approach to the optimization of envelope ... - IEEE Xplore

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 42, NO. 2, FEBRUARY 1994

-, “Extraction of motor related activity from single trial EEG,” in Proc. 10th Ann. Inr. Conf. Eng. Medicine Biol. Soc. (New Orleans, LA),

Nov. 4-7, 1988. S. Cerutti, G. Chiarenza, D. Liberati, P. Mascellani, and G. Pavesi, “A parametric method of identification of single-trial event-related potentials in the brain,”IEEETruns. Biomed. Eng., vol. 35, pp. 701-71 1, Sept. 1988. H. a1 Nashi, “A maximum likelihood method for estimating EEG evoked potentials,” IEEE Trans. Biomed. Eng., vol. BME-33, pp. 1087-1095, Dec. 1986. K. Yu and C. D. McGillem, “Optimum filters for estimating evoked potential waveforms, in IEEE Trans. Biomed. Eng., vol. BME-30, pp. 730-737, Nov. 1983. C. D. McGillem, I. 1. Aunon, and D. G. Childers, “Signal processing in evoked potential research: Applications of filtering and pattem recognition,” CRC Crir. Rev. Bioeng., vol. 6, pp. 225-265, 1981.

A New Approach to the Optimization of Envelope-Constrained Filters with Uncertain Input K. L. Teo, Antonio Cantoni, and X. G. Lin Abstract- In envelope-constrained filtering, the filter is optimized subject to the constraint that the filter response to a given signal lies within a specified envelope or mask. In a number of signal processing applications, the envelope-constrained filtering problem is more directly relevant than least squares approximation-based approaches. In this correspondence, we develop an efficient method for solving an extended version of the envelope-constrained filtering problem in which the input pulse is not known exactly but is known to lie within a specified mask. This envelope-constrainedproblem with uncertain input (ECUI) has been examined elsewhere, but the algorithm proposed there for its solution has, in general, inferior convergence characteristics.

I. INTRODUCTION In this correspondence, we consider the design of a filter with timeinvariant finite impulse response (FIR)such that its output in response to a class of input signal lies within an output pulse shape envelope and such that its gain with respect to input white noise is minimized. The class of input signals is also defined in terms of a pulse-shaped envelope. Following the terminology in [I], the filtering problem is said to be envelope constrained with uncertain input (ECUI). The design of filters subject to envelope constraints has many applications. For example, intemational communications standards specify the allowable pulse shape in digital data transmission systems in terms of envelope constraints [2], [3], the quality of a channel used for video transmission is assessed in terms of a mask called the I< rating mask [4],and a number of signal processing problems in radar and antenna arrays have been successfully formulated in terms of envelope-constrained filtering [5]-[8]. The ECUI problem considered in this correspondence arises when one introduces robustness requirements in envelope-constrained filtering problems since it allows for Manuscript received October 15, 1991; revised February 15, 1993. The associate editor coordinating the review of this paper and approving it for publication was Dr. Robert A. Gabel. K. L. Teo and X. G. Lin are with the Department of Mathematics, The University of Westem Australia, Nedlands, Australia. A. Cantoni is with the Australian Telecommunications Research Institute, Curtin University of Technology, Perth, Australia. IEEE Log Number 9214184.

Fig. 1. ECUI filter and masks. the fact that in practice, the input pulse may not be precisely the ideal shape. Furthermore, by allowing for uncertainty in the input pulse to the filter, we prevent the high sensitivity to modeling parameters that optimal solutions sometimes exhibit. The ECUI problem we consider was first proposed in [ 11 and was solved by combining the primal-dual algorithm with the GoldsteinLevitin-Poljak gradient projection method [9], [IO]. Although the algorithm was shown to converge for certain step size sequences, numerical results indicated that the convergence rate was quite poor. In this paper, we develop a simpler and more efficient method for solving the ECUI problem. The technique developed for the ECUI problem is more generally applicable to a class of nonsmooth constrained optimization problems [ 1 11. The new method of solution relies on exploiting properties of the ECUI cost function and the constraints to derive an equivalent standard quadratic programming problem that has a unique solution and can be solved efficiently by known techniques (see Chapter 10 of [ 121). This correspondence is organized as follows. In Section 11, we introduce some notation and formulate the ECUI problem. A class of nonsmooth optimization problems representative of the ECUI problem is introduced in Section III. We then define a standard quadratic optimization problem and show how the solution to this problem is related to the solution of the nonsmooth constrained optimization problem and hence the ECUI problem. Finally, in Section IV, we present numerical results to an ECUI problem and compare them with those reported in [l]. 11. PROBLEM FORMULATION The output of a time-invariant finite impulse response filter to an input pulse of a finite support can be described by

9 =su where U = [ u ~ ., . , U,]’ [*I,. . . , U,]’ filter, U

=

=

.

E E

(1)

R” is the impulse response of the R,V is the output response vector,

s [ S I , . . . s,IT E R”‘ is an input pulse vector, and S E R.VXnis the signal matrix given by

-V = n

. . .

Sln

0

. .

L o . . . .

S1

.

. .

.

+ m - 1,

SrnJ

We are concerned with filter designs in which the output of the filter is required to lie within an output pulse shape envelope for input pulses that lie within a given input pulse shape envelope, as illustrated in Fig. 1.

IO53-587X/94$04.00 0 1994 IEEE


Let E+ E R" and E- E R" be given upper and lower output pulse shape constraints, respectively. Then, we define

421

Problem ( P I ) :

Minllul(2 subject to

cu-d -cu d

Thus, the output pulse shape constraint becomes

IQ where for any a

- dJ = JSu - dJ 5

E

> 0.

(3)

= [ a ] ,.... a,IT E R" la( = [la1 I, .... Ian IIT.

(4)

Similarly, the input signal can be shown to satisfy 1s - c( 5 0

>0

(5)

=

where c ;(s+ +s-) and 0 i(s+ - s-), whereas s+ E R" and s- E R" are the input pulse shape upper and lower boundaries. To continue, we define matrix bounds on the signal matrix S such that

IS - CI

50

c T = [

. . . .

1:

. .

cm

0

'

cm

.

1

and 0 E R N X "is given by

0' E

. . .

0"

0 0 1 ' .

.

. . .

0

0

0 . 0 , o .

' .

. .

81

1.

(13)

.

1 0 0 0 . . . O l O O . . .

.

:I

. . o o 0 -1 . . 0 0 - 1 0

00 ~ 0 ~ 0 ( 14) and f E R"/' is the all-zero n/2-dimensional vector. Problem P1 extended with the addition of a linear equality constraint can be written as the following nonsmooth optimization problem.

. Cm 0

(12c)

where for n even, D E Rg X n is given by

0

'

. . . .

c1

0

(12b)

E.

Du = f

D= '

E

In [l], a method is developed for solving the problem (12) by combining the primal-dual algorithm [ 101 with the Goldstein-kvitinPoljak gradient projection method [9]. It is possible to include additional linear equality constraints to the problem formulation without significantly impacting on the complexity of the method of solution developed in this paper. For example, it is possible to impose a linear phase [ 131 condition on the filter by adding a linear constraint of the form

(6)

where C E R N x nis given by

. . . . c1 .

+ Olul 5

+ + Olul 5

0

0

~

~

0

1

-

1

Problem ( P 2 ) :

SI.

MinJIu 11' subject to

Au

0,

+ Blul 5 e Du= f

Thus, the set of all time-invariant linear filters that will take any and

B>O

every s E

S

5 {SE

Rm : IS - CI 5 0 )

(7) where U E R", e E RL is a given vector, and A, B E R L x n , D E RK X n are known matrices. We have used the notation

and map it into any element

B > O * b b , , 2 0 vi=1,2 ..... L j = 1 , 2 ..... n.

{Q E R" : 1 9 -dl

9 E @'

5 E}

In the case of the ECUI problem, L = 2 N and

AT = [C', -C'] BT = [O',O'] eT = [ ( E + d)T, ( E - d l T I.

is given by

U

E {U E

R"

:

J S u - d ( 5 EVS : I S - C ( 5 0}

(9)

and the optimal filter is found by solving

(15) (16) (17)

If a linear phase filter is required, then D and f can be chosen as Minllul(2,

U

EU

E R"

(lo)

described previously.

where

111. EQUIVALENT OFTIMEATION PROBLEM

(1 1)

Consider the following optimization problem. Problem ( P 3 ) :

The use of the min J1u11' cost is motivated by the fact that zero mean additive white noise of variance g 2 at the input of the filter produces a noise component at the output of the filter that is zero mean and has variance 0 ~ 1 1 ~ 1 1 ~ . The above problem is an ECUI. The objective is to find an optimal filter so that any and every waveform in the input envelope evokes a response in the output envelope. Using Lemma 2.1 of [l], we see that the above problem is equivalent to the following nonsmooth optimization problem.

minl(z1I2, z E R2" subject to

H z 517 Gz = f z>o

428


I

i-A

LJ

La

Fig. 2. Input pulse mask for example.

Note that problem P3 involves a convex cost and a convex constraint set. Thus, the problem has a unique solution, provided the constraint set is not empty. Let z* be the solution to (P3). Since Z* E R2",it can be partitioned as follows where z*,y* E R": z* =

Fig. 3. Output pulse mask for example.

[;:I.

It can be shown [14] that * U

*

= x - Y*

(20)

is the solution to problem P2 and, hence, the ECUI problem. The proof of this result relies on setting up a number of equivalent optimization problems and exploiting the fact that B 2 0. Quadratic programming problems subject only to equality constraints are very easy to handle numerically. Methods for solving such quality-constrained quadratic programming problems are discussed in great detail in the first two sections of Chapter 10 of [12]. The problem (P3) is a constrained quadratic programming problem subject to both equality and inequality constraints. It can be solved efficiently by the active set method [15] and section 10.3 of [12]. The method has two distinct phases. In the first phase (linear programming phase), an iterative procedure is carried out to determine a feasible point. The second phase (the quadratic programming phase) generates a sequence of feasible iterates in order to minimize the quadratic objective function. If the Hessian matrix of the objective function is positive semidefinite, we will obtain a global minimum. If it is positive definite, then we will have a unique global minimum. The routine E04NAF in the NAG Library (cf. [16]) is developed based on this approach. It is also the one that we use to solve our numerical examples in the next section. The computational time taken by each iteration of the routine E04NAF is approximately proportional to min(n2, m 2 ) ,where n and m represent, respectively, the number of variables and the number of general linear constraints in the problem (P3).

IV. NUMERICAL EXAMPLES

To demonstrate the effectiveness of the new method of solving the ECUI problem, we use an example studied originally in [I]. The example corresponds to the compression of a 13-b Barkercoded waveform represented by m = 13 input samples. The allowable error in the input was assumed to be 2.5%, as shown in Fig. 2. The ECUI filter was fixed at n = 27 samples, and the output was constrained to be in the mask shown in Fig. 3. The output mask has an allowable sidelobe level of 0.025 and a mainlobe peak of 0.69 and a tolerance of f 0.075. By converting the corresponding problem into the form of (P3) and then using the routine E04NAF, an optimal solution is obtained in 49 iterations. In comparison, the results in [ I ] indicate that after 500

Fig. 4. Optimal filter impulse response in example. 07

I

1

06 0.5 -

0 4 ~ I+J

0.3-

-0.11 0

' 5

10

15

20

25

30

35

1

40

i Fig. 5. Corresponding output pulse for an input pulse specified by c.

iterations, the Goldstein-Levitin-Poljak gradient projection algorithms has not yet converged. Fig. 4 shows the optimal filter obtained using the problem (P3), and Fig. 5 shows the corresponding output pulse for an input pulse specified by c .

V. CONCLUSION In this correspondence, we have introduced a new method for the optimization of envelope-constrained filters with uncertain input (ECUI). The method relies on the formulation of a standard quadratic programming problem whose solution can be solved more efficiently than the original ECUI problem. The more efficient algorithm enables a more comprehensive evaluation of the ECUI approach to be carried out and may also enable efficient adaptive filters to be developed.


429

I. INTRODUCTION

REFERENCES [ I ] R. J. Evans, A. Cantoni, and K. J. Ahmed, “Envelope-constrained filter with uncertain input”, Circuits Sysr. Signal Processing, vol. 2, no. 2, pp.

131-154, 1983. [2] CCITT Recommendations G. 703, vol. III, Fascicle 111.3 of the CCITT Blue Books, Geneva Int. Telecommun. Union, 1988. [3] J. W. Lechleider, “A new interpolation theorem with application to pulse transmission,” IEEE Trans. Commun., vol. 39, no. IO, pp. 1438-1444, Oct. 1991. [4] Z. L. Budrikis, “Visual fidelity criterion and modeling”, Proc. IEEE, vol. 60, pp. 771-889, 1972. [5] R. J. Evans and K. M. Ahmed, “A new array processor with robustness and broadband capabilities,” IEEE Trans. Antennas Propagar., vol. AP32, no. 9, pp. 944-950, 1984. [6] R. J. Evans, A. Cantoni, and T. E. Fortmann, “Envelope-constrained filter, Part 11, Adaptive structures,” IEEE Trans. Info. Theory, vol. IT-23, no. 4, pp. 435-444, July 1977. [7l R. J. Evans, T. E. Fortmann, and A. Cantoni, “Envelope-constrained filter, Part I, Theory and applications,” IEEE Trans. Inform. Theory, vol. IT-23, no. 4, pp. 421-434, July 1977. [8] T. E. Fortmann and R. J. Evans, “Optimal filter design subject to output sidelobe constraints: Computational algorithms and numerical results”, J. Optimization Theory Applications, vol. 14, no. 3, pp. 271-290, Sept. 1974. [9] A. A. Goldstein, Constructive Real Analysis. New York: Harper-Row, 1967. [lo] L. S. Lasdon, Optimization Theory for Large Systems. New York: MacMillan, 1970. [ I l l Y. C. Soh, R. J. Evans, I. R. Petersen, and R. E. Betz, “Robust pole assignment,” Auromatica, vol. 23, no. 5, pp. 607-610, 1987. [12] R. Fletcher, Practical Methods of Optimization. New York: Wiley, 1987, 2nd ed. [I31 L. R. Rabiner and B. Gold, Theory and Application of Digital Signal Processing. Englewood Cliffs, NJ: Prentice Hall, 1975, ch. 3. [I41 K. L. Teo, A. Cantoni, and X. G. Lin, “Non-smooth optimisation problems in envelope constrained filtering”, Tech. Rep. ASPL 1992-6, Adaptive Signal Processing Lab., Australian Telecommun. Res. Inst., Curtin Univ. of Technol., Bentley, Westem Australia. [15] P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization. London: Academic, 1981. [I61 NAG Library, FORTRAN L i b r a r M a r k 14. Oxford, UK: Mayfield House. vol. 4.

The systolic array for computing the residuals of the recursive least squares (RLS) based on the QR decomposition (QRD) by the Givens rotations was proposed by McWhirter [ 11. Recently, there have been several works conceming the weight (tap coefficient) extraction by systolic arrays for RLS. Shepherd et d. [2] first showed that, based on the QRD, i.e., X, = Q , R , with X , (the data matrix), Q , (an orthogonal matrix), and R , (an upper triangular matrix), the weight extraction can be done by updating (R,’ )T = RZT with a lower triangular array, and by combining this array with an upper triangular array for updating R,, the overall rhombic array was obtained. Also, Pan and Plemmons [3] have derived the inverse factorization (IF) algorithm, where the same equation for updating R I T with that of [2] is used, but the rotation angles are computed based on not R, and R k T . Thus, in this algorithm, only the elements of RLT need to be stored. On the other hand, the modified Gram-Schmidt (MGS) method has been used by Kawase et d. [4] and Ling et al. [SI. In this paper, MGS or square-root-free versions of the algorithms in [2] and [3] are presented. Systolic-array implementations of our algorithms are considered on a 2-D rhombic array. Finally, simulation results are presented to compare the finite wordlength effect of our algorithms and existing algorithms. 11. lk? MGS-BASEDRLS ALGORITHMS In this section we review’ the MGS-based RLS algorithms [4], [ 5 ] . Here we mainly follow the notation in [5]. Let us consider the problem of estimating a signal ~ ( nby) a linear combination of N signals y ( n ) = ( y l ( n ) y * ( n ) . . . y . ~ ( in n )the ) ~least-squares sense. That is, we want to determine the weight (tap coefficient) vector w ( t ) = ( w l ( t ) w Z ( t ) .. . u ~ ~ v ( by t ) minimizing )~ the criterion t

€ ( t )=

1Xf--7‘e2(n,t) n=O t

=

Xt-”(z(n) - w r ( t ) y ( n ) ) 2

(2)

n=O

where X is the forgetting factor. Let us define the extended ( t ( A%T 1) data matrix by

Recursive Least-Squares Algorithms of Modified Gram-Schmidt Type for Parallel Weight Extraction

+

where the ( t Hideaki Sakai

+ 1)X

+ 1) x 1 vector Y,(t) is defined by

Y,( t) = (X”2y, ( O ) , X(t-1)’2y,

( 1) . ’ . , y*( t ) ) r ,

( l = l , . . ~ , N + l )(3) Abstract-This paper presents some new algorithms for parallel weight extraction in the recursive least-squares (RLS) estimation based on the modified Gram-Schmidt (MGS) method. These are the counterparts of the algorithms using an inverse QR decomposition based on the Givens rotations and do not contain the square root operation. Systolic-array implementationsof the algorithms are considered on a 2-D rhombic array. Simulation results are also presented to compare the finite word-length effect of our algorithms and existing algorithms.

where we put y . i v + l ( t ) = z ( t ) and X ( t ) = Ylv+l(t). By the Gram-Schmidt orthogonalization method we generate the orthogonal vectors q l ( t ) , . . . , q l v ( t ) , q ~ + l = ( f )e ( t ) % X ( t ) A ( t ) w ( t )such that (4)

+

where k(t ) is an ( X 1) x (Aunit diagonal elements given as Manuscript received January 2, 1992; revised February 24, 1993. The associate editor coordinating the review of this paper and approving it for publication was Prof. John J. Shynk. The author is with the Division of Applied Systems Science, Faculty of Engineering, Kyoto University, Kyoto 606, Japan. IEEE Log Number 9214182.

+ 1) upper triangular matrix with

with (Ii(t))z, = k t J ( t ) ( i< J ) and ( K r ( f ) )=t k z . , v + l ( t ) .From (2) and (5) the minimizer of (1) or equivalently lle(t)1I2satisfies the

1053-587X/94$04.00 0 1994 IEEE