ASTRONOMICAL MULTIBAND IMAGE FUSION AND ... - CiteSeerX

ASTRONOMICAL MULTIBAND IMAGE FUSION AND RESTORATION USING PYRAMIDAL ANALYSIS AND MARKOVIAN SEGMENTATION Farid Flitti1 , Christophe Collet1 and Eric Slezak2 1

Universit´e Strasbourg I, LSIIT - UMR CNRS 7005 - France 3

Observatoire de la Cˆote d’Azur - UMR 6162 - France 1

[email protected], 2 [email protected]

ABSTRACT This paper deals with noised astronomical multiband image fusion and restoration. The wavelet domain is well adapted for such tasks. In fact, intensity fluctuations corresponding to the noise are most important at the finest resolution and related details coefficients decrease quickly as the scale increases. Real structures in the image will therefore lead to larger detail coefficients values. This information can be suitably combined to detect and fuse real structures in all bands. In this work, we present a new fusion-restoration scheme for astronomical multispectral data, operating in four steps. At first, a pyramidal algorithm with one wavelet analyzes all spectral bands leading to a pyramid of details coefficients for each one. Secondly, the detail information for all wavelengths are combined in a unique Multiband-Multiresolution Pyramidal to feed a hierarchical Markovian classifier based on a quadtree topology, providing a multiscale two classes segmentation map masking at each scale small enough coefficients. At third, the selected coefficients are merged leading to a unique pyramidal structure of detail coefficients. Finally, this resulting structure is used with an iterative reconstruction procedure to recover the denoised fused image. The Markovian quadtree models spatial, interscale and interband correlation together allowing a real whole cube information fusion. Key words : Multispectral image fusion, restoration, Markov quadtree, pyramidal algorithm with one wavelet.

eling of wavelets was introduced in [4] capturing interscale and spacial wavelet coefficient correlations. In this paper we use a more general Markovian framework modeling not only spatial and interscale dependencies as the existent models but also interband correlation for multiband image fusion and denoising task. The proposed approach operates as follows (Fig.1). We first carry out a decimated multiresolution transform using pyramidal algorithm with one wavelet [5] for each band (Sect. 2). In a second step, we combine all pyramidal representations obtained in the previous step in a single multibandmultiresolution pyramid (MMP) by considering detail coefficients at same space-scale location as a unique vector. In third step, the MMP are segmented in two-classes using a vectorial hidden Markov quadtree (Sect. 3) to separate significant wavelet coefficients from those associated with the noise. This modeling is very useful since the selection relies now not only on the sole coefficient magnitude [2] but also takes into account its neighbors: in space, in scale and with wavelength. This classification scheme produces a multiresolution binary mask highlighting significant coefficients. In the fourth step the significant coefficients of all pyramids are fused in a single one using different rules detailed in Sect. 4. The final step consists in the reconstruction of the fused image using the iterative Van Cittert’s algorithm [5]. 2. THE PYRAMIDAL ALGORITHM WITH ONE WAVELET

1. INTRODUCTION Fusion of multiband images is of great interest in astronomy, allowing to merge the whole multiband information in a single scene. Generally this task is more difficult for noisy observations. The wavelet domain is well adapted both for fusion [1] and denoising [2] tasks. In fact, intensity fluctuations corresponding to the noise are most important at the finest resolution and related details coefficients decrease quickly as the scale increases. Real structures in the image will therefore lead to larger detail coefficients values along scales [3]. This information can be suitably combined to detect and fuse real structures in all bands. Recently, an efficient Markovian mod-

The most usual tool for multiscale analysis of real images is the orthogonal/bi-orthogonal wavelets transform [6]. This transformation detects singularities across given directions. However, astronomical objects are diffuse and exhibit sharp edges only in very few exceptional cases. This is why isotropic wavelet transforms which do not privilege any direction are preferred [5]. The pyramidal algorithm with one wavelet is an isotropic transform obtained by adapting the Laplacian pyramid of Burt and Adelson [7]. Lets consider a 1D discrete signal f (k). Its successive approximations are obtained by repeatedly applying a low-pass filtering followed by a down-sampling by a fac-

tialized by setting c0 (k) = f (k). The detail information

  
 


456789: ;

cj (k)

-

- cj+1 (k) - 2l ?

h

  

? - l

- wj+1 (k)

?@8A768?B:  !!"# $%&'() *%+!, &"" 7D8BD

cj+1 (k)

- 2l 6

h

- l - cj (k) 6

wj+1 (k) Fig. 2. Pyramidal algorithm with one wavelet: analysis at the top and synthesis at the bottom.

/ 01 23

-. .

EFGHI JHGKLMHI NOPQH

wj (k) lost from resolution j to resolution j + 1 is computed as the difference between the approximation at scale j and the undecimated approximation at scale j as follows : wj+1 (k) = cj (k) − c˜j+1 (k)

(2)

where c˜j+1 (k) is given by : c˜j+1 (k) =

X

h(k − l)cj (l).

(3)

l

Fig. 1. Fusion-restoration algorithm illustrated for a biband image. A new fusion-restoration scheme for multispectral data, operating as follows. A pyramidal wavelet transform analyzes the N = 2 spectral bands (on the top). This leads to a multiresolution pyramid of wavelet coefficients for each band, up to scale M = 4. Then, all wavelet pyramids are combined to carry out two-class multiresolution Markovian segmentation map (on the right). This segmentation map masks small coefficients at different scales. The remainder coefficients are fused using an appropriate rule. The result with the average of coarsest approximations feed an iterative reconstruction procedure to give a unique fused restored image.

tor 2. Thus the approximation cj+1 (k) at scale j + 1 is given by : X cj+1 (k) = h(l − 2k)cj (l), (1) l

where cj (k) is the approximation at scale j and h a normalized, symmetric low-pass filter verifying the equal contribution constraint 1 [7]. This procedure(Fig. 2) is ini1 Burt and Adelson used a filter of length 5. If h(0) = a, h(−1) = h(1) = b, and h(−2) = h(2) = c then the equal contribution requires that a + 2c = 2b.

The reconstruction procedure (Fig. 2) is the same as with the Laplacian pyramid. Given details and approximation at scale j + 1, the approximation at scale j is computed by: cj (k) = wj+1 (k) −

X

h(k − 2l)cj+1 (l).

(4)

l

However, this reconstruction is not exact [5]. The exact reconstruction can be approached iteratively by the Van Citter’s algorithm (Sect. 5). For an image F (i.e.; 2D discrete signal), one operates separately with the same filter h on the rows and then on the columns [5]. Thus we obtain a pyramid W where each plan Wj corresponds to a given resolution. For a multiband image F with N bands, a wavelet decomposition is carried out for each band b separately leading to a multiresolution pyramids W b , b = 1, · · · , N . These N pyramids are combined in unique MultibandMultiresolution pyramid (MMP) W by considering detail coefficients, Wj1 (k), · · · , WjN (k), at the space location k and the scalej as a components of an unique vector Wj (k). The (MMP) W feed a vectorial hidden Markov quadtree to give two classes segmentation map separating significant coefficients from those associated with the noise.

3. HIDDEN MARKOV QUADTREE MODEL Let a quadtree G = (S, L) be a graph composed of a set S of nodes and a set L of edges as illustrated in Fig. 3. Each node s apart from the root r has a unique direct predecessor, its parent s− , on the path to the root. Each node s, apart from the terminal ones, the leaves, has four children s+ . The set of nodes S can be partitioned into scales, S = S 0 ∪S 1 . . .∪S R , according to the path length from the leaves to the root . Thus, S R = {r}, S n involves 4R−n sites, and S 0 is the finest scale formed by the leaves. This graph has the same hierarchical structure as the wavelet pyramid W. Then each space-scale location (k, j) in W can be associated with a given node s in the G. In the sequel we note Wj (k) = Ws . ¤s¡ £s¢ Wr root d xr = xs−

¤s¡ £ds¢

¤s¡ £sd¢

¤s¡ £ds¢

¤s¡ £ds¢

¤s¡ £sd¢

¤s¡ £sd¢

¤s¡ £ds¢

¤s¡ £ds¢ ¤s¡ £sd¢

¤s¡ £ds¢

¤s¡ £ds¢

¤s¡ £ds¢

¤s¡ £sd¢ Ws ¤s¡ xs £sd¢ ¤s¡ £sd¢

¤s¡ £sd¢

¤s¡ £sd¢ ¤s¡ £sd¢

¤s¡ £ds¢

¤s¡ £sd¢ x s+

following product : P (W|x) =

R Y

P (W n |xn ) =

R Y Y

P (Ws |xs ) (6)

n=0 s∈S n

n=0

4

where ∀s ∈ S n , ∀n ∈ {0, ..., R}, P (Ws |xs = ı) = fin (Ws ), captures the likelihood3 of the data Ws . We use fin (Ws ) ∼ N (Σni , µni ). From these assumptions, it can be easily seen that the joint distribution P (x, W) can be expressed as [8]: P (x, W) = P (xr )

Y

P (xs |xs− )

s6=r

Y

P (Ws |xs )

(7)

s∈S

where P (xr ) stands for the a priori and P (xs |xs− ) for the parent to child transition probability. The expression of P (x, W) allows to estimate exactly and efficiently P (xs = i|W) for all nodes s ∈ S by two passes on the quadtree. The segmentation label map is finally given by : x ˆs = arg maxi∈{1,2} P (xs = i|W). The quadtree defined above needs a parameter estimation (i.e.; parameters of the gaussian pdf’s, inter-scale transition probabilities and a priori) procedure to be unsupervised. Such estimation is obtained using available estimation algorithms like Expectation Maximization algorithm [10] and Iterated Conditional Estimation algorithm [11]. In practice, given an initial realization of X, one alternates model parameter estimation and segmentation until convergence. 4. FUSION RULES

Fig. 3. Example of a dependency graph corresponding to a quadtree structure on a 4 × 4 lattice. White circles represent labels and black circles represent multiband observations Ws , s ∈ S. Each node s has a unique parent s− , and four children s+ .

When significant Wavelet coefficients are selected by the classification process, we use one of this three rules: PN 1. ∀s ∈ S n : Wsf used = i=1 xs Wsi 2. ∀s ∈ S n : Wsf used =

PN n i i=1 σi xs Ws P , N nx σ s i=1 i th

σin being the

standard deviation of the i marginal of the likeliLet X = (Xs )s∈S and W = (Ws )s∈S two stochashood associated with class 1 at scale n. tic processes indexed on S corresponding to the hidden states and the MMP respectively. Each Xs takes one of 3. ∀s ∈ S n : Wsf used = maxi Wsi if xs = 1, 0 othertwo values: 0 if Ws is non significant and 1 otherwise. wise. X is supposed to be Markovian in scale, i.e.,2 P (xn |xk , k > n n+1 n) = P (x |x ). Moreover, given its parent, each hidThe performances of these rules will be discussed in Sect. den state is supposed to be independent from all its ascen6. Since The approximations are enough cleaned the fused dants and the probabilities of inter-scale transitions can be image approximation is computed by averaging approxifactorized in the following way [8, 9]: mations of all bands. Y P (xn |xn+1 ) = P (xs |xs− ) (5) 5. ITERATIVE RECONSTRUCTION n s∈S

Each Ws is independent from all the other nodes conditionally to Xs and thus the likelihood is expressed as the 2 To

P (x).

simplify notation, we will denote the probability P (X = x) as

The reconstruction using the fused wavelet coefficients W f used can not be done using the Eq. 4. Indeed, the 3 In this paper the observations are introduced on M scales in the quadtree, For the other scale the likelihood is set to equal 1: for n > M , fin (Ws ) = 1∀i.

structure W f used does not correspond to a smooth image since all non significant coefficients are put to zero. We seek instead a smooth solution Fˆ f used which minimizes k (W f used − O(Fˆ f used )) k where O is the wavelet transform operator. In practice we use the Van Cittert’s algorithm [5] : • Initialisation : p=0 Fˆ [p] = O−1 (W f used ) where O−1 is the reconstruction operator Eq. 4. [p] 1. Wr = (W [p] − O(Fˆ [p] )) ¯ X [p] [p] Fr = O−1 (Wr ) ¯ being the term by term multiplication and X the binary map obtained in Sect. 3. [p] 2. Fˆ [p+1] = Fˆ [p] + Fr [p]

3. if the residual k Wr k is under a given threshold value then stop, else p = p + 1 and goto 1.

8. REFERENCES [1] Z. ZHANG and R. S. BLUM, “A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application,” Proceedings of the IEEE, vol. 87, no. 8, pp. 1315 – 1326, August 1999. [2] D.L.Donoho and I.M. Johnstone, “Ideal spatial adaptation by wavelet shrinkage,” Biometrika, vol. 81, pp. 425–455, September 1994. “A multiscale vision [3] A. Bijaoui and F. Ru´e, model adapted to the astronomical images,” Signal Processing, vol. 46, pp. 345–362, 1995. [4] M.S. Crouse, R.D. Nowak, and R.G. Baraniuk, “Wavelet-based statistical signal processing using hidden markov models,” IEEE Trans. Image Processing, vol. 46, no. 4, April 1998.

When the algorithm converges the restored image is given by Fˆ f used = Fˆ [p+1] .

[5] J.-L. Starck, F. Murtagh, and A. Bijaoui, Image Processing and Data Analysis: The Multiscale Approach, Cambridge University Press, 1998.

6. EXPERIMENTS

[6] S. Mallat, A wavelet tour of signal processing, Academic Press, 1998.

The technique described above was tested on real noised astronomical multiband images of high-z galaxies from the Hubble Deep Field observed with the Hubble Space Telescope at six wavelengths, from the rest-frame FUV to I band (Fig. 4 and 6). We perform a wavelet transform on M = 4 levels for each band using the pyramidal algorithm. The final restored fused images obtained using different fusion rules are presented in Fig. 5 and 7. We can observe in both cases that our approach produce a more cleaned result then the simple averaging. This is due to the fact that fusion is operated only on the significant wavelet coefficients. Thus we see the effectiveness of the Markovian modeling to select real structure. Moreover, rules 1 and 2 seem better than 3 since they produce a more smoothed fused images. Finally, we note that rule 2 produces a slightly better result than rule 1. This can be explained by the fact that with rule 2 the contributions of the different bands are well weighted by the variance of class 1 which represent real structures in the image. 7. CONCLUSION An new scheme for multiband astronomical images fusion and restoration is presented. It uses a quite general Markovian framework, modeling not only spatial and interscale dependencies as the existent models but also interband correlation. The experiment results on real noised astronomical image are quite satisfying especially for rule 2. The Markovian modeling appears very effective for detection of real structures, so they are efficiently summarized in the fusion result. We note finally that all computations are done in unsupervised way.

[7] P. J. Burt and E. H. Adelson, “The laplacian pyramid as a compact image code,” IEEE Trans. Communications, vol. 31, no. 4, pp. 532–540, April 1983. [8] J.-M. Lafert´e, P. P´erez, and F. Heitz, “Discrete markov image modeling and inference on the quadtree,” IEEE Trans. Image Process., vol. 9, no. 3, pp. 390–404, March 2000. [9] M. Luettgen, Image Processing with Multiscale Stochastic Models, Phd thesis, MIT Laboratory of Information and Decision Systems, May 1993. [10] A.P. Dempster, N.M. Laird, and D.B. Rubin, “Maximum likelihood from incomplete data via the em algorithm,” Royal Statistical Society, pp. 1–38, December 1986. [11] W. Pieczynski, “Statistical image segmentation,” Machine Graphics and Vision, vol. 1, no. 2, pp. 261– 268, 1992.

Fig. 4. Original multiband image (in inverse video): hdf4-378. 6 bands corresponding to the wavelengths (in nm): 300, 450, 606, 814, 1100 and 1600.

Fig. 5. Fusion result (in inverse video) of the multiband image of Fig.4 using different rules. Top left : a simple averaging of the bands. Top right : proposed technique with rule 3. Down left : proposed technique with rule 1. Down right : proposed technique with rule 2.

Fig. 6. Original multiband image (in inverse video): hdf4-550. 6 bands corresponding to the wavelengths (in nm): 300, 450, 606, 814, 1100 and 1600.

Fig. 7. Fusion result (in inverse video) of the multiband image of Fig.6 using different rules. Top left : a simple averaging of the bands. Top right : proposed technique with rule 3. Down left : proposed technique with rule 1. Down right : proposed technique with rule 2.