Adaptive DCT-based filtering of images corrupted by spatially correlated noise Nikolay N. Ponomarenkoa,Vladimir V. Lukina, Alexander A. Zelenskya, Jaakko T. Astolab, Karen O. Egiazarianb a National Aerospace University, 61070, Kharkov, Ukraine;1 b Tampere University of Technology, Institute of Signal Processing, P.O. Box-553, FIN-33101, Tampere, Finland ABSTRACT Majority of image filtering techniques are designed under assumption that noise is of special, a priori known type and it is i.i.d., i.e. spatially uncorrelated. However, in many practical situations the latter assumption is not true due to several reasons. Moreover, spatial correlation properties of noise might be rather different and a priori unknown. Then the assumption that noise is i.i.d. under real conditions of spatially correlated noise commonly leads to considerable decrease of a used filter effectiveness in comparison to a case if this spatial correlation is taken into account. Our paper deals with two basic aspects. The first one is how to modify a denoising algorithm, in particular, a discrete cosine transform (DCT) based filter in order to incorporate a priori or preliminarily obtained knowledge of spatial correlation characteristics of noise. The second aspect is how to estimate spatial correlation characteristics of noise for a given image with appropriate accuracy and robustness under condition that there is some a priori information about, at least, noise type and statistics like variance (for additive noise case) or relative variance (for multiplicative noise). We also present simulation results showing the effectiveness (the benefit) of taking into consideration noise correlation properties. Keywords: DCT based filter, spatially correlated noise, robustness, adaptation.
1. INTRODUCTION Noise appearing in obtained images is an inherent phenomenon in different types of imaging systems starting from digital photo cameras and completing by infrared and radar remote sensing complexes1-4. Because of this, image denoising is one of the most typical operations in image processing. Numerous filters have been designed to cope with noise; they can be referred to several basic “families” like nonlinear non-adaptive (order statistic) filters1,3, locally adaptive nonlinear filters5, empirical filters (examples are the ones proposed by J-S. Lee6,7), transform based techniques (see 8 and references therein), combined approaches9,10, etc. (see also a survey in the paper11). Filter design and performance analysis are commonly carried out under certain assumptions concerning noise statistical and spatial spectral (correlation) characteristics. Only a quite small percentage of known filters do not take into account noise type and statistics. Few examples are the simplest techniques of image denoising like the standard mean, median, center weighted median, K-nearest neighbour filters3,5, etc. Other filters which are more efficient and commonly more complicated are intended for removal of noise of particular type and noise statistics. For example, there are filters specially designed for speckle suppression12, strike removal13, etc. When noise type and statistics is supposed known in advance or, at least, noise type is identified14 and its statistics is pre-estimated15, a proper filter is selected and its parameters are adjusted correspondingly16. Thus, noise type and statistics in image filtering are often taken into account. In particular, noise variance or standard deviation in case of removal of additive Gaussian noise is taken into consideration by setting a proper threshold in all transform based filters (see the paper17 and references therein). Similarly this happens in suppression of pure multiplica-
1
Correspondence to Lukin Vladimir, e-mail
[email protected] Image Processing: Algorithms and Systems VI, edited by Jaakko T. Astola, Karen O. Egiazarian, Edward R. Dougherty, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6812, 68120W, © 2008 SPIE-IS&T · 0277-786X/08/$18 SPIE-IS&T Vol. 6812 68120W-1
2008 SPIE Digital Library -- Subscriber Archive Copy
tive and signal dependent noise8,18. However, almost all filters in no way exploit information about spatial correlation properties of noise. Moreover, it is commonly supposed that noise is i.i.d. (i.e. spatially uncorrelated). On one hand, this simplifies filter design and comparison of performance of different filters for some standard, commonly accepted test images and noise models with noise statistics varying in certain limits. On the other hand, if one considers spatially correlated noise case, then a first question is what are spatial correlation characteristics of noise to be considered. Really, they can vary in rather wide limits, and this means that filter comparison should be done for this variety. Note that there have been several attempts to study, at least, some aspects of filtering spatially correlated noise. In this sense, we can mention the papers of Kuan19 and Plotkin20. Particular aspects of noise suppression efficiency in image homogeneous regions are touched in the paper21 where it is shown that the difference in noise reduction between the mean and L-filters with the same scanning window size for spatially correlated noise becomes smaller than for i.i.d. noise. Some researchers consider that in case of spatially correlated noise it is enough to increase filter scanning window size in comparison to a size recommended for i.i.d. case (e.g., to use a 7x7 window instead of 5x5). Sometimes this really partly helps in the sense of improving noise reduction in image homogeneous regions but it is not the best decision since simultaneously this commonly leads to detail and texture blurring. Here we should also stress that an image corrupted by spatially correlated noise is characterized by worse visual quality than if it is corrupted by i.i.d. noise with the same statistics (type of noise and, e.g., its variance)22. One can be also interested whether or not it is important to consider the case of spatially correlated noise. Our answer is “yes, it is important”. One reason is that spatially correlated noise quite often appears in practice due to data incorrect sampling5, influence of imaging system point spread function, raw data interpolation before getting an image for its visualization and further processing23, dependence of imaging system spatial (e.g., azimuth) resolution on range like in radar5 and ultrasound systems24, etc. Another reason is that accounting of spatial correlation characteristics of noise leads (as it will be shown below) to considerable improvement of filtering effectiveness and visual quality of processed images8-11, 24, 25. To start considering spatially correlated noise and effectiveness of its removal, we have selected DCT based filtering. There are several motivations behind this. First, in case of filtering images corrupted by i.i.d. Gaussian additive and speckle noise, transform (wavelet, DCT) based filters have demonstrated effectiveness close to the best among currently existing groups of filters (probably, only PCA and ICA based techniques26,27 can compete with transform based filters but they are rather complex). Second, DCT based filters perform is sliding blocks and, thus, in opposite to wavelet filters, they are “local action” filters. This allows to incorporate knowledge on spatial correlation properties of noise in an easier manner since rather high degree of spatial correlation commonly takes place for neighbouring pixels, i.e. for local areas. Third, DCT approaches to Karhunen-Loeve transform in its decorrelation ability and this is important in image denoising and compression applications17. In this paper, we study a rather simple case of spatially invariant correlation characteristics of noise. At the very beginning, we show how the conventional DCT based filtering can be modified in order to incorporate a priori information on spatial correlation function of noise supposed to be accurately known. After this, we give some examples of filter effectiveness improvement due to this for the case of zero mean additive Gaussian noise. Then we present a method for estimating 2-D spatial correlation function in DCT basis for a given image. The obtained estimates are slightly erroneous and we consider what is the influence of these errors on filter effectiveness. Numerical simulation results are presented that give imagination on filter performance for several methods of evaluation of noise spatial correlation. Some recommendations on selection of the method parameters are given. Finally, the conclusions follow.
2. INITIAL ASSUMPTIONS USED IN MODIFIED DCT BASED FILTER DESIGN There are several basic assumptions that we rely on in design and analysis of the modified DCT based filter for spatially correlated noise case. Let us briefly consider them. A first assumption is that a 2-D spatial correlation function of noise is constant for all fragments of an image to be processed. In other words, below we do not deal with situations that are, e.g., typical for ultrasound imaging where depending upon range spatial correlation of noise varies considerably.
SPIE-IS&T Vol. 6812 68120W-2
Second, we assume that noise type is a priori known, its statistical characteristics are a priori known or pre-estimated with appropriate accuracy, and there is no impulse noise. One can argue that these assumptions are too strict or even unrealistic. As counter-arguments we can state the following. For some types of imaging systems there is a priori information what is noise type. An example is synthetic aperture radar imaging for which it is possible to consider noise present in formed images as pure multiplicative with characteristics that depend upon number of looks (which is also commonly known in advance) 5,28. Similarly, for optical images noise is often considered pure additive; if one deal with color images, then this assumption relates to R, G., and B color components29. Finally, efficient methods for determination of noise type have been designed recently14. If noise can not be referred to one of types considered in the paper14, then more sophisticated methods for determination of dependence of local variance on local mean σ 2 ( I ) can be applied10.
If noise is pure additive or pure multiplicative, its variance can be estimated with quite high accuracy. However, here we should stress that in case of spatially correlated noise not all techniques are worth applying for this purpose. For example, the methods of blind evaluation of noise variance that operate in spectral domain30,31 produce biased estimates. Therefore, it is necessary to use techniques operating in spatial domain (see the paper15 and references therein). These techniques for majority of practical situations produce accuracy characterized by relative root mean square error of about few percent. Such accuracy is appropriate for the considered application. Concerning impulse noise, there are several comments. First, impulse noise is not always present. Second, there are methods for blind evaluation of probability of impulse noise occurrence32, a lot of methods for impulse noise removal (see, e.g., the paper13), techniques for blind evaluation of additive or multiplicative noise variance in case of mixed noise32,33. Thus, below we do not pay attention to possible presence of impulse noise. Third, we assume that DCT is the basis of filtering. In this sense, we would like to briefly recall main principles of transform based filtering. It has several stages of data processing within a given block. The first stage is obtaining a set of DCT coefficients (spectrum), the considered block size is 8x8 pixels. The second stage is coefficient thresholding, i.e. assigning zero values to DCT coefficients with absolute values smaller than a pre-selected threshold (below we consider hard thresholding scheme). The third stage is carrying out inverse DCT with obtaining filtered values for all pixels belonging to the considered block. After obtaining filtered values for a given pixel coming from overlapping blocks (below we consider shift invariant denoising with full overlapping of blocks) they are averaged. We would also like to remind that in theory of linear filtering an operation of pre-whitening is commonly applied in case of correlated noise34. Since DCT based filtering is a local approximation of optimal (Wiener) linear filtering (although transform based filters include nonlinear operation of thresholding), some analog of pre-whitening should be used in it in order to cope with correlated noise. One more assumption concerning spatially correlated noise is that its correlation function has a main lobe with a width (expressed in a number of inter-pixel distances) smaller than block side size. Besides, we assume that outside this main lobe 2-D autocorrelation function has values quite close to zero. In other words, we suppose that there is no longdistance correlation or obvious side lobes of autocorrelation functions. In many practical situations of image denoising this assumption is valid since imaging system designers try to minimize the influence of point spread function (antenna pattern) and selected spatial sampling of data on a formed image quality. One exception are ultrasound imagers for which it is difficult to decrease azimuth correlation of noise for ranges close to maximal.
3. MODIFIED DCT BASED FILTER Consider now what happens with DCT spectrum of noise if it is spatially correlated. For the beginning, let us revisit a case of stationary pure additive noise. Recall first that in case of i.i.d. zero mean noise it “uniformly spreads” over all components of an orthogonal transform used for its spectral representation. An orthogonal transform coefficients then are zero mean random values with symmetric PDF close to Gaussian and equal variances. These properties are well known and widely exploited in wavelet and DCT based filtering and compression18,35. On the contrary, if noise is spatially correlated then 2-D spatial spectrum coefficients remain to be zero mean random values with symmetric PDF close
SPIE-IS&T Vol. 6812 68120W-3
to Gaussian but their variances σ W2 (k , l ) become different although all of them are proportional to variance of noise. Then setting a constant threshold at the second stage of transform based denoising leads to undesirable effects. The coefficients that have rather small σ W2 (k , l ) are “killed” (assigned zero value) with very high probability (close to unity). At the same time, other spectral coefficients that possess relatively large σ W2 (k , l ) remain untouched (unchanged) with large probability. After inverse orthogonal transform they appear in filtered image, i.e. these components of noise are not removed. For the assumed properties of spatially correlated noise the abovementioned effects result in partial suppression of noise and appearing of residual noise that is characterized by even larger degree of spatial correlation than it was in original image. This means that if we would like to “kill” spectral coefficients which correspond to noise with approximately equal probability we have to set a spectrally varying threshold T (k , l ) proportional to σ W (k , l )
( T (k , l ) =β σ W (k , l ) ) where σ W (k , l ) = (W (k , l )) . Here β denotes a proportionality factor and W (k , l ) is a power spec-
trum of spatially correlated noise in a given orthogonal basis. If σ 2 ( I ) =Const, W (k , l ) is also constant and the threshold values are the same for all blocks. If σ 2 ( I ) ≠Const but this dependence is a priori known or pre-estimated, then one has to a priori know a normalized power spectrum Wnorm (k , l ) (that corresponds to unity variance of noise), to estimate a local mean for each block Iˆ , to calculate expected value of local variance for this block σ 2 ( Iˆ ) and to set thresholds bl
bl
as T (k , l ) = β σ ( Iˆbl ) (Wnorm (k , l )) . Everywhere below we use β=2.6. The considerations in the previous paragraph relate to image homogeneous regions. In image heterogeneous regions (like edges, details, texture present in a block), spectrum properties differ from those ones discussed above. In each block there are few spectral components that relate to information components although these spectral components also have random values due to noise. But absolute values of these components are, on the average, larger than for the corresponding components for the case of image homogeneous blocks. Therefore, these components with higher probability will be remained after thresholding. However, we should note that probability of remaining these components in the case of spectrally varying threshold T (k , l ) depends upon frequency. If σ 2 ( I ) ≠Const , then these probabilities also depend upon a block mean. Assume now that Wnorm (k , l ) and σ 2 ( I ) = Const = σ 02 are exactly known. Let us give an example demonstrating the effectiveness of the modified DCT based filter. Suppose that σ 02 =100, then PSNR of the noisy image is equal to 28.1dB. The output image of the DCT based filter for i.i.d. noise (with the constant threshold equal to T=26=2.6σ0) is presented in Fig. 1,a. As seen, noise is well suppressed and the residual noise is practically not observed (PSNR=34.28 dB). Consider now the case of spatially correlated noise with the same σ 02 =100 (again PSNR=28.1 dB). This noisy image is represented in Fig. 1,b. Note that visually spatially correlated noise is more unpleasant than i.i.d. with the same variance36. The output image for the standard DCT based filter (with the constant T=2.6σ0) is shown in Fig. 1,c. Its PSNR is equal to 30.14 dB, i.e. it has increased by only 2 dB in comparison to the original image PSNR (for i.i.d. noise the improvement due to filtering is more than 6 dB). Finally, the output image for the modified DCT filter that takes into account 2-D DCT spectrum of correlated noise is given in Fig. 1,d. The provided PSNR is 32.17 dB. It is considerably better than for the standard DCT based filter but not so large as for i.i.d. noise case. Residual noise is smaller than for the image in Fig. 1,b, but more intensive than for the image in Fig. 1,a. A question is why spatially correlated noise is not suppressed with the same effectiveness as i.i.d. noise. The reason is that DCT based filters simultaneously with noise removal can introduce distortions into information components of a processed image. Each DCT coefficient D(k , l ) for a given block can be represented as D (k , l ) = Dim (k , l ) + N (k , l ) where Dim (k , l ) and N (k , l ) correspond to noise-free (true) image and noise, respectively. DCT coefficients D( k , l ) that are assigned zero values at the second step of denoising can contain information about true image, i.e. Dim (k , l ) ≠ 0 . Small absolute values of D(k , l ) can be due to rather small absolute values of Dim ( k , l ) T (k , l ) . In the former case introduced distortions are negligible while in the latter case they can be considerable especially if T (k , l ) > βσ .
SPIE-IS&T Vol. 6812 68120W-4
The presented example shows that the modified DCT filter sufficiently outperforms the standard DCT based filter (other examples confirming this will be given later). Then a question is how to estimate W (k , l ) or, more generally, Wnorm (k , l ) for an image at hand.
4,
'A
а)
b)
c)
d)
Fig. 1. I.i.d. and spatially correlated noise filtering: a) the standard DCT based filter output, PSNR=34.28 dB, for the test image Barbara corrupted by i.i.d. noise with σ2=100, noisy image PSNR=28.1 dB, b) noisy image, spatially correlated Gaussian noise, σ2=100, PSNR=28.1 dB, c) the standard DCT filter output for the image in b), PSNR=30.14 dB; d) the modified DCT based filter output, PSNR =32.17 dB.
SPIE-IS&T Vol. 6812 68120W-5
4. BLIND ESTIMATION OF SPATIALLY CORRELATED NOISE POWER SPECTRUM Several methods can be designed for blind evaluation of Wnorm (k , l ) . Consider first the simplest algorithm: 1) Suppose that we have estimated noise variance σˆ 2 for a given image by a technique operating in spatial domain15 (assume that noise is pure additive at the beginning); 2 2 2) Then select some number Num of image fragments of size 8x8 for which σˆ loc
