Combining the discrete wavelet transforms and rank-order based filters for image restoration Samvel M. Atourian Karen O. Egiazarian David Z. Gevorkian Jaakko T. Astola Tampere University of Technology Signal Processing Laboratory P.O. Box 553 FIN-33101, Tampere, Finland E-mail:
[email protected]
Abstract. We propose a filter structure formed as the superposition of an impulse removal filter, based on a modified K -nearest-neighbor operation and discrete wavelet transform based noise reduction. This filter removes impulsive and Gaussian noise and mixed noises well. We also suggest an efficient architecture for implementing the impulse removal filter used in this structure. © 1998 Society of Photo-Optical Instrumentation Engineers. [S0091-3286(98)02301-0]
Subject terms: recognition techniques; image restoration; wavelet denoising; impulse removal filters. Paper ART-160 received June 13, 1997; revised manuscript received Aug. 12, 1997; accepted for publication Aug. 22, 1997.
1
Introduction
Digital image restoration is a field of engineering that studies methods used to recover an original scene from degraded observations.1 This is an essential preprocessing stage of pattern recognition. The main difference of image restoration from image enhancement is that the former requires the existence of a model for the degradation ~i.e., knowledge about how an image was degraded!. In many applications the degradation model is quite complicated and consists of different types of noises: white Gaussian noise, impulsive noise, etc. Linear spatial lowpass filters are commonly used to reduce noise in images. However, there are certain limitations in their use in practice, since they blur the original image and perform poorly in the presense of impulsive noise. In the last few years, there has been considerable interest in the use of wavelet transforms for signal and image restoration.1,2 Very efficient wavelet transform domain methods for removing an additive white Gaussian noise from signals and images have been developed. Having excellent performance in suppressing Gaussian noise, they work pretty well also in several applications where the error is neither white nor Gaussian.2 These applications are noise reduction ~denoising! of synthetic aperture radar ~SAR! signals, medical and geophysical signals, and removing the blocking artifacts in images of Joint Photographic Experts Group ~JPEG! decoded signals.2,3 However, wavelet-based noise reduction methods do not work in the case of images corrupted by even a small percentage of impulses. On the other hand, spatial nonlinear filters efficiently remove impulsive noises, at the same time preserving the image sharpness.4 The most widely used methods of image restoration in the presence of impulsive noises are based on rankings of the pixels in neighborhood according to brightness. Examples of filters based on such methods ~often referred to as rank-order based filters! are the well-known median and, in general, threshold Boolean filters ~TBF!, Opt. Eng. 37(1) 189–201 (January 1998)
0091-3286/98/$10.00
K-nearest neighbor filters, L filters, and various switching filters.5–10 These filters are excellent rejectors of certain kinds of impulsive noise. Among those let us note the K-nearest neighbor ~K-NN! filter introduced by Davis and Rosenfeld.6 This filter performs well for both additive and multiplicative noise.11,12 An obstacle to widespread usage of the K-NN filter was, for a long time, its high computational complexity. However, efficient implementations of different modifications of this filter have been recently suggested in Ref. 13 ~for general-purpose computers! and in Ref. 14 ~for bit-serial architecture!. Rank-order based filters, while excellent against impulsive noise, do not suppress Gaussian noises as efficiently as the wavelet denoising methods do. It is natural to incorporate the best aspects of these two types of methods by using combination filters that involve them both in a unified design for removing mixed noises. The filters to be combined in such a scheme should be adapted to the mixed noise and to each other. For instance, in the case of mixed Gaussian and impulsive noise it is important to suppress impulses first, using an impulse removal filter that is robust against the presence of white Gaussian noise and preserves spectral characteristics of the input image. Then a wavelettransform based filter may be efficiently used. Note that the wavelet-transform based methods used for Gaussian noise removal are global processing procedures applied to the entire image, whereas rank-order based nonlinear filters are local processing procedures applied to the fragments of images within a moving window. Combining these two types of filters will give methods that utilize both local and global statistical analyses. In this paper we propose a new impulse removal filter that selectively processes image pixels using an extended K-nearest neighbor operation. Based on this operation and running min and max operations, the filter efficiently detects both salt-and-pepper and additive impulses in the case of a small percentage as well as in the case of a large percentage ~up to 90%! of impulses presented in the image. © 1998 Society of Photo-Optical Instrumentation Engineers
189
Atourian et al.: Combining the discrete wavelet transforms . . .
Detected impulses are replaced with the result of the same basic ~extended K-NN! operation, which gives quite accurate estimations even in the white Gaussian noisy environment. Due to the selective processing and accurate estimation of pixels corrupted by impulses, the proposed impulse removal filter preserves spectral characteristics of the input image well. For images corrupted with mixed white Gaussian and impulsive noise we propose a novel combination filter based on the superposition of the new impulse removal filter and a modified discrete wavelet-transform based filter with a new thresholding strategy. Extensive experiments confirm the expected high performance of the proposed filters in removing different types of noise. Efficient implementations and computational complexity analysis of the basic operations used in the proposed impulse removal filter are also provided. The rest of the paper is organized as follows. In Sec. 2, the general filter structure is proposed and studied. In Sec. 3, the experimental results and performance analysis of the proposed filter for restoration of images corrupted with different types of noises are presented. In an appendix ~Sec. 5!, an efficient implementation of the running min and max operations is provided. Section 6 regards the implementation of the extended K-nearest neighbor operation, which is the basic operation of the proposed impulse removal filter. 2
In general, the filter structure we study in this paper is formed as the superposition of an impulse removal filter G and a wavelet-transform based filter F: ~1!
where X5 $ x(i, j), i 5 1,2,...,I, j 5 1,2,...,J % , x(i, j) P $ 0,1, . . . ,R21 % , R52 r , is an observation of an image U5 $ u(i, j), i 5 1,2,...,I, j 5 1,2,...,J % corrupted by mixed white Gaussian and impulsive noise of different types, and Y 5 $ y(i, j), i 5 1,2,...,I, j 5 1,2,...,J % is the output of the filter, which tends to estimate the image U. Below we propose the filters F and G to be used in the superposition scheme of Eq. ~1!. First we describe the proposed impulse removal filter, which is based on using a modified K-NN operation for selective processing of image pixels. Then we briefly introduce the wavelet-transform based filter, where we use a new thresholding strategy. 2.1 The Impulse Removal Filter Specific to the filter structure given by Eq. ~1! is that the impulse removal filter G should not destroy frequencies in the image, which is essential for the second stage: the wavelet denoising. ~Otherwise, one could consider the problem of finding G that transfers the image to a form more suitable to wavelet denoising than the given observation after removing impulses. However, this is a very complicated task.! This means that while removing impulses the filter should not change the other image pixels. Such a filter G would not blur edges, which cannot be reconstructed at the second stage. Thus, it is desirable to apply such an impulse removal filter at the first stage that processes image pixels with different filtering actions depending on the probability of impulses appearing in the neigh190
Optical Engineering, Vol. 37 No. 1, January 1998
borhood of the given pixel. This kind of strategy is used in state-conditioned filters, in contrast to the case of conventional rank-order based filters ~TBF, K-NN, L-filter, etc.!, where every pixel is processed uniformly. In state-conditioned techniques the filtering procedure is conditioned on the current state of the algorithm.5,8–15 The output of the filter is defined as K
y ~ i, j ! 5
The Filter Structure
Y 5F ~ G ~ X !! ,
Fig. 1 The general structure of the impulse removal filter.
( a k,s~ i, j !u k~ i, j ! , k51
i51, . . . ,I,
j51, . . . ,J, ~2!
where u k (i, j), k51,...,K, are different estimates of the image pixel u(i, j); s(i, j)P $ 1, . . . ,S % is a state variable that classifies the current pixel into one of the S categories; and a k,s(i, j) are the scalar coefficients corresponding to each category. For example, the ROM filter proposed in Ref. 5 uses u 1 (i, j)5x(i, j) and u 2 (i, j)5ROM(i, j) as the estimates, where ROM(i, j) is the rank-ordered mean ~the average of the fourth and fifth order statistics within the 3 33 window excluding the center pixel!. The minimum-maximum exclusive mean ~MMEM! filter recently introduced in Ref. 8 uses four different estimates: u 1 (i, j)5x(i, j), u 2 (i, j)5mean(U (3) (i, j)), u 3 (i, j) 5 mean(U (5) (i, j)), and u 4 (i, j)5mean(Y (i21, j61),Y (i 21,j),Y (i, j21)), where U (p) (i, j), p53,5, is a set of pixel values that is formed by removing some pixels from the p3 p window V (p) (i, j) centered at x(i, j) according to a certain strategy. A comparative study of this filter with other existing filters8 has demonstrated sufficiently high performance in removing impulsive noises from highly corrupted images. However, its performance becomes worse for images corrupted with a small percentage of impulses, which is often the case in real applications. Below we describe the proposed impulse removal filter, which performs well for images both highly and slightly corrupted with impulsive noises. The general structure of the proposed impulse removal filter is presented in Fig. 1. For every pixel x(i, j), its neighborhood within two windows V (p 1 ) and V (p 2 ) of sizes p 1 3 p 1 and p 2 3 p 2 is analyzed. Pixels that are suspected to be impulses are removed from these windows by a proper discarding strategy. Then an estimate E for the input
Atourian et al.: Combining the discrete wavelet transforms . . .
sample x(i, j) is computed. Depending on the numbers of elements within the discarded windows U (p 1 ) and U (p 2 ) , the estimate is either the result of a filtering action A applied to U ( p 1 ) or the result of a filtering action B applied to U (p 2 ) or the causal prediction obtained from the recent outputs. The final output of the filter is formed as
y ~ i, j ! 5
H
x ~ i, j !
if u x ~ i, j ! 2E u .T,
E
otherwise,
~3!
depending on the threshold T. In our experiments we use different thresholds T for the cases of small and high percentage of impulses in the input image. For slightly corrupted images ~up to about 8% of impulses! we use the threshold T550, and for highly corrupted images ~over 8% of impulses! we use the threshold T530. These values have been obtained through extensive experiments with different images and noise models. The discarding procedure, together with the switching between blocks A, B, and C and with the switching at the last stage @see Eq. ~3!#, determines the state condition of the algorithm ~actually, this gives detection of impulses!. Different techniques can be used for this purpose.5,8,16–18 The actual filtering is done in blocks A and B. With regard to conditioning the MMEM filter works quite well. Its discarding procedure is based on running 333 and 535 min and max operations, which can be implemented efficiently ~see Appendix 1!. However, the estimate E, being the result of just averaging, is quite rough ~especially for slightly corrupted images, because in that case more pixels remain in the window after the discarding procedure!. As a possible alternative we use an extended K-nearest neighbor operation as filtering actions A and B. This operation is described below. Let the filter’s p3p window involve the set V (p) 5V (p) (i, j) 5 $ x(1),x(2),...,x(N) % (N5p 2 ) when located at the pixel x(i, j), i51, . . . ,I, j51, . . . ,J @x(i, j)5x c is regarded as the center pixel#. Consider a window of weights W (p) 5 $ w(1),w(2),...,w(N) % that has the same shape as the filter’s window V (p) . The weight w(n) is associated with the pixel x(n) @we drop the indices i, j in the notation for pixels x(m), m51,2,...,N, for convenience of reading#. For every location of the window V (p) on the image, the following two sets of pixels are considered: ~1! the subset U (p) 5U ( p) (i, j)#V ( p) (i, j) of pixels chosen according to a certain strategy, and ~2! the subset S5S(i, j)#U (p) (i, j) of K pixels ~K5 d d M e ; M 5 u U u , 0, d t,
u z ~ i, j ! u ,t, 0, or soft thresholding: zˆ~i,j !5Ts~z~i,j !,t!
5
H
sgn~z~i,j !!~uz~i,j !u2t!, uz~i,j !u>t, u z ~ i, j ! u ,t,
0,
~7!
1
