gabor systems with good tf-localization and applications to image

GABOR SYSTEMS WITH GOOD TF-LOCALIZATION AND APPLICATIONS TO IMAGE PROCESSING Hans G. Feichtinger, Peter Prinz and Werner Kozek

NUHAG { Numerical Harmonic Analysis Group University of Vienna, Department of Mathematics Strudlhofgasse 4, A-1090 Vienna, Austria e-mail: [email protected]

ABSTRACT The basic design freedom of a (generalized) Gabor transform is the choice of (i) the time{frequency lattice constants and (ii) the analysis prototype (atom, window). The design of the synthesis prototype is subject to linear constraints (Wexler{Raz condition) depending on the redundancy of the presentation. For multidimensional signals the design freedom can be greatly increased by the consideration of nonseparable situations: i) Nonseparable prototypes and/or ii) Nonseparable position{frequency sampling lattices. We present such a general theory for the Gabor expansion of 2D signals. Our main result is a generalized biorthogonality condition connecting the analysis and synthesis prototype. The theory is illustrated by a simple numerical experiment.

1. INTRODUCTION AND OUTLINE The Gabor expansion [1, 2, 3, 4, 5] is an important linear signal representation. Gabor analysis is one speci c approach to a local Fourier analysis, as such it is closely related to the short{time Fourier transform, generalized (weighted) cosine transform and modulated lter banks. In the one{ dimensional case these parallelism is now well{recognized and it has led to a certain amount of cross{fertilization between the elds [6, 7]. The two{dimensional (2D) case can be traced back to the 1D case whenever the Gabor system is separable, i.e., the prototype can be written as the outer product of two 1D prototype functions and the position{ frequency sampling lattice is a 4D rectangular grid. For nonseparable situations, however, key questions are still unresolved: Given a nonseparable position{frequency sampling lattice which generates a Weyl{Heisenberg (WH) frame for some speci ed analysis prototype, can one guarantee existence of a dual frame with identical structure (as conjectured in [8])? Given a nonseparable 2D analysis prototype, how can one determine the synthesis prototype? The following section provides a review of the underlying de nitions. In section 3 we formulate the main results addressing the above problems. We discuss a simple numerical experiment in section 4, followed by concluding remarks in section 5. This work was partially supported by project S-7001 of the Austrian Science foundation FWF.

2. GABOR EXPANSION AND GROUP THEORY 2.1. 2D Gabor Expansion Our signal space H consists of images x of size N  M with the usual inner product

h x; y i :=

NX1 MX1 n=0 m=0

x(n; m)y(n; m)

and associated norm kxk2 := h x; x i . The (cyclic) translation (position shift) operator is de ned as
(Tu x) (n; m) = x [n u1 ]N ; [m u2 ]M ; (1) where the index u is a vector of position shifts, u = (u1 ; u2 ), and the subscripts M; N indicate the modulus. The modulation (frequency shift) operator acts as nv mv (Mv x)(n; m) := x(n; m)ej2f N1 + M2 g ; with v = (v1 ; v2 ). The modulation operator is the Fourier dual1 of the translation operator in the sense that Mv = F2 Tv F2 , where the Fourier transform is de ned as (F2 x)(k; l) =

NX1 MX1 n=0 m=0

x(n; m)e

j 2f nk N + ml M g:

We de ne the Gabor analysis system (h; ) as a family of functions generated out of the prototype via position and frequency shifts indexed by a position{frequency sampling lattice : h(u;v) := Mv Tu h (u; v) 2 : The Gabor coecients can then be de ned as 

(u; v) 2 : (2) cx;h (u; v) := x; h(u;v) ; For separable position{frequency sampling lattice the index set can be determined as (we assume that the lattice constants a1 ; b1 and a2 ; b2 are divisors of N and M , respectively): n  = (u; v)ju1 = ka1 ; u2 = la2 ; v1 = mb1 ; v2 = nb2 k 2 f0; : : : ; aN1 1g; l 2 f0; : : : ; aM2 1g o m 2 f0; : : : ; bN1 1g; n 2 f0; : : : ; Mb2 1g

2.2. Lattices and Subgroups

Nonseparable position{frequency sampling lattices can be treated with the help of sampling matrices [9]. However, we deviate from the usual matrix characterization of multidimensional sampling geometries, rather we employ the algebraic concept of groups and subgroups since this yields both compact notation and short{cut derivations of our main results. A group is de ned as follows [10]: De nition 1 A group consists of a set and an operation , compactly written as ( ; ) with the following properties (a; b; c 2 ):  is closed under the operation: a b 2

 Validity of the associative law: (a b) c = a (b c)  Existence of a (unique) neutral element e 2 : a e = e a = a  Existence of an inverse element a 1 2 : 1 a a = a a 1 = e When the group operation is commutative as well, i.e. a b = b a for arbitrary a; b 2 , one calls ( ; ) an abelian group. A subset   forms a so{called subgroup of ( ; ) when (; ) is itself a group. Based on these mathematical preliminaries we can de ne a general position{frequency sampling lattice as follows: De nition 2 A (position{frequency sampling) lattice  is a subgroup of the group (ZM  ZM  ZN  ZN ; +). Figure 1 shows an example of separable and nonseparable subgroup of the 2D time{frequency plane. The elements of the group are 4D vectors consisting of two position indices combined to one position vector u = (u1 ; u2 ), and two frequency indices combined to one \frequency vector" v = (v1 ; v2 ). The group operation precisely means pointwise addition of the indices modulo N in the rst coordinate and modulo M in the second, i.e., given two position{frequency points a ; b 2  one has:  = a + b () (u; v) = (ua + ub ; va + vb ) separable lattice

non-separable lattice

2.3. Basics of Weyl{Heisenberg Frames A Gabor system (h; ) generates a Weyl{Heisenberg (WH) frame whenever the following inequality holds for all x 2 H X hx; h(u;v) i 2  B kxk2 ; (3) Akxk2  (u;v)2

where A and B are called lower and upper frame bound (0 < A  B < 1). The frame operator is de ned as Gabor analysis followed by Gabor synthesis using the same prototype:

S x :=

X

hx; h(u;v) ih(u;v) :

(u;v)2

The maximum and minimum eigenvalues of the frame operator are equivalent to the frame bounds. In the discrete setting, it is rather trivial to check whether the Gabor system is a frame or not: The Gabor system (h; ) generates a WH frame, i the linear span of the family h(u;v) (u;v)2 , coincides with the signal space H. It is clear that jj  NM and jj divides (NM )2 . We de ne red := jj=NM  1 as the redundancy of the WH frame.

3. DUALITY AND BIORTHOGONALITY The frame operator commutes with position{frequency shifts on the lattice:

Mv Tu S = SMv Tu ; (u; v) 2 : this commutability property is of fundamental relevance for the theory of WH frames. In fact, applying the (well{ de ned) inverse frame operator both from left and right one gets: Mv Tu S 1 = S 1 Mv Tu ; (u; v) 2 : This proves the identical structure of a general WH frame and its dual (cx;g (u; v) is de ned in (2)): Theorem 1 If (g; ) generates a frame, there exists at least one dual prototype h such that for all x 2 H:

x=

X

cx;g (u; v) h(u;v) =

(u;v)2

Figure 1: Separable and non{separable subgroups with

u1 = [ua;1 + ub;1 ]N u2 = [ua;2 + ub;2 ]M and likewise for v. We denote the number of elements of the subgroup  by jj.

X

cx;h (u; v) g(u;v)

(u;v)2

(4)

We call h and g dual (with respect to ) if (4) holds true. Now, the important question arises how to determine the dual prototype for a given Gabor analysis system. It turns out that the connection between the dual prototypes can be formulated similar to the Wexler/Raz principle for 1D signals [2] by a biorthogonality condition, to this end we need the following de nition: De nition 3 For each lattice  the adjoint lattice (or commutator lattice)  is de ned by   = (r; s) j Ms Tr Mv Tu = Mv Tu Ms Tr ; 8(u; v) 2  :

Note that one has

jjj j = (NM )2 ;

which means that with increasing redundancy of a Gabor system the adjoint lattice gets sparse. For the case of product lattices the adjoint group is just the lattice with complementary lattice constants (with position and frequency variable interchanged). Thus the new general notion of "adjointness" extends the known one for 1D or 2D lattice cases, but also the one in [11] for the multidimensional continuous setting. sampling lattice

adjoint lattice

.

A basis of Ng is easily obtained by calculating the null space of the matrix given by (5). Summing up we obtain the general solution of a dual Gabor window:

hgen = harb + n with n 2 Ng : The usual "frame dual" ~g = S 1g can be characterized

among all dual prototypes by dierent minimality conditions. For the continuous 1D setting such results are due to Daubechies [13] and Janssen [3].

Theorem 3 (Minimality Characterizations [14]) Among all possible \dual prototypes" h, the "frame dual" g~ = S 1 g is optimal in terms of: 1. delivering minimal norm coecients; 2. having itself minimal norm (in l2 {sense); 3. being most similar to the original prototype g, i.e. minimizing kg hk (both normalized); 4. NUMERICAL RESULTS

Figure 2: Lattice and its adjoint lattice One of the fundamental properties of  is the fact that one can characterize the action of the WH frame operator in a canonical way:

S = red

X

(r;s)2

(r;s) Ms Tr ;

where the coecients are given by sampling the ambiguity function of the prototype on the adjoint lattice:

 (r;s) := h; h(r;s) : This characterization of the frame operator is a consistent generalization of known results for the separable case [3, 12]. A necessary condition for perfect reconstruction in the form of (4) can be formulated as Theorem 2 (General Biorthogonality) Two prototypes h; g are mutually dual in the sense of (4) i they satisfy a biorthogonality relation on the adjoint grid:

g ; h = 1
 for (r; s) 2  : (5) (r;s)

red 0;(r;s) The ane space of all possible dual Gabor prototypes h to a given analysis prototype g can be characterized by the following biorthogonality with respect to the adjoint lattice  : Let harb be an arbitrary solution of (5). Then harb + n is also a solution of (5), if and only if h g(r;s) ; n i = 0 for (r; s) 2  : (6) We denote the linear space of all n satisfying (6) by Ng . The dimension of Ng increases as redundancy increases, more precisely, one has
dim(Ng ) = 1 red1 NM 

The basic principle of transform coding is to establish an approximate decorrelation of the signal source prior to the quantization [15]. Ideal decorrelation is obtained by the Karhunen{Loeve (KL) transform which is signal adaptive and unstructured. One promising compromise between the numerically expensive KL transform and xed transforms can be achieved by matching the Gabor analysis prototype to the signal. For the 1D case, a theory for matching the prototype has been obtained in [16]. It is not within the scope of this paper to derive a precise generalization of these results for the 2D case, rather we show that even a coarse matching leads to nonseparable prototypes. We consider a part of the Lena image with 120x120 pixels, see Fig. 4(a). For this image we have computed a (cyclic) autocorrelation estimate as follows:



r~x (k; l) := x; T(k;l) x ; where T(k;l) denotes the cyclic translation operator as de ned in (1). Fig. 4(b) shows a contour plot of the autocorrelation function (ACF) estimate. The ACF clearly indicates a large correlation width in the direction of the diagonal. Based on the knowledge of the ACF we have considered two dierent Gabor analysis prototypes: (i) A separable prototype (see Fig. 4(a)) with Gaussian shape whose essential width was matched to the image by trial and error; (ii) A rotated version of the prototype coarsely matched to the ACF estimate. The position{frequency sampling lattice was separable, determined by the position constants a1 = a2 = 24 and frequency constants b1 = b2 = 4 with a resulting redundancy of 1:56. Figs.4 (b), (d) show contour plots of the synthesis prototypes obtained by minimum norm inversion of the frame operator. For a qualitative comparison of the analysis prototypes we computed the entropy of the (normalized) Gabor coecients: e=

X

(u;v)2

jcx (u; v)j log (jcx (u; v)j)

(Small entropy corresponds to good energy concentration of the coecients, as is desirable for source coding.) The entropy achieved by the separable prototype was 3:81 while that of the rotated, nonseparable prototype was 3:66. Hence, the choice of an (even only coarsely adapted) nonseparable prototype may improve the energy concentration of the Gabor coecients. However, minimum entropy can also be interpreted as optimal decorrelation which correponds to the approximate diagonalization of a covariance kernel. From this point of view it may be expected that nonseparable prototypes are also relevant for image reconstruction or image enhancement [17, 18] applications. (a)

(b) 120

20

100

40

80

60

60

80

40

100

20

120

20

40

60

80 100 120

20

40

60

80 100

Figure 3: (a) Part of Lena and (b) its ACF (a)

(b)

120

120

100

100

80

80

60

60

40

40

20

20 20

40

60

80 100

20

40

(c) 120

100

100

80

80

60

60

40

40

20

20 40

60

80 100

(d)

120

20

60

80 100

20

40

60

80 100

Figure 4: (a) Separable prototype, (b) its dual. (c) Matched nonseparable prototype, (d) its dual.

5. CONCLUSION We have presented a theory for (oversampled) 2D Gabor expansions including the case of nonseparable position{frequency sampling lattices and nonseparable prototypes. This theory provides the basis for a more exible design of two{ dimensional Weyl{Heisenberg frames. By a simple numerical experiment we have shown the practical relevance of nonseparable analysis prototypes.

6. REFERENCES [1] M.J. Bastiaans. Gabor's Expansion of a Signal into Gaussian Elementary Signals. IEEE Proc., 68:538{539, April 1980. [2] J. Wexler and S. Raz. Discrete Gabor Expansions. Signal Processing, 21(3):207{221, November 1990. [3] A.J.E.M Janssen. Duality and biorthogonality for Weyl-Heisenberg frames. J. Four. Anal. Appl., 1(4):403{437, 1995. [4] M. Zibulski and Y.Y. Zeevi. Oversampling in the Gabor scheme. IEEE Trans. SP, 41(8):2679{2687, 1993. [5] H.G. Feichtinger, O. Christensen, and T. Strohmer. A group-theoretical approach to Gabor analysis. Optical Engineering, 34:1697{1704, 1995. [6] H. Bolcskei, F. Hlawatsch, and H.G. Feichtinger. Equivalence of DFT lter banks and the Gabor expansions. In SPIE Proc., Wavelet Applications in Signal and Image Processing III, volume 2569, pages 128{139, San Diego (CA), July 1995. [7] Z. Cvetkovic. Oversampled modulated lter banks and tight Gabor frames in l2 (ZZ). pages 1456{1459, Detroit (MI), May 1995. [8] M. Zibulski and Y.Y. Zeevi. Multiwindow Gabor-type transform for signal representation and analysis. In Proc. SPIE, Wavelet Applications, San Diego, 1995. [9] D.E. Dudgeon and R.M. Merserau. Multidimensional Signal Processing. Prentice Hall, Englewood Clis (NJ), 1984. [10] W. Rudin. Fourier Analysis on Groups. Wiley, New York, 1967. [11] A. Ron and Z. Shen. Weyl-Heisenberg frames and Riesz bases in L2 (Rd). Technical Report 95-03, University of Wisconsin, Madison (WI), 1995. [12] R. Tolimieri and R. Orr. Poisson Summation, the ambiguity function and the theory of Weyl-Heisenberg frames. J. Four. Anal. Appl., 1(3):233{247, 1995. [13] I. Daubechies, H. Landau, and Z. Landau. Gabor time-frequency lattices and the Wexler-Raz identity. J. Four. Anal. Appl., 1(4):437{478, 1995. [14] P. Prinz. Theory and Algorithms for Discrete 1{ Dimensional Gabor Frames. Master's thesis, Univ. Vienna, 1996. [15] T. Ebrahimi and M. Kunt. Image Compression by Gabor Expansion. Opt. Eng., 30/7:873{880, 1991. [16] W. Kozek. Matched generalized Gabor expansion of nonstationary processes. In Proc. IEEE Int. Conf. Signals, Systems, and Computers, pages 499{503, Paci c Grove, CA, Nov. 1993. [17] J.G. Teti. Weyl-Heisenberg and wavelet phase space ltering using waveform signature templates. SPIE Wavelet Applications, 2242:370{381, 1994. [18] G. Cristobal and R. Navarro. Space and frequency variant image enhancement based on a Gabor representation. Pattern Recognition Letters, 15(3):273{277, 1994.