1854
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 7, JULY 2007
Local Variance-Controlled Forward-and-Backward Diffusion for Image Enhancement and Noise Reduction Yi Wang, Liangpei Zhang, Member, IEEE, and Pingxiang Li, Member, IEEE
Abstract—In order to improve signal-to-noise ratio (SNR) and contrast-to-noise ratio, this paper introduces a local variance-controlled forward-and-backward (LVCFAB) diffusion algorithm for edge enhancement and noise reduction. In our algorithm, an alternative FAB diffusion algorithm is proposed. The results for the alternative FAB algorithm show better algorithm behavior than other existing diffusion FAB approaches. Furthermore, two distinct discontinuity measures and the alternative FAB diffusion are incorporated into a LVCFAB diffusion algorithm, where the joint use of the two measures leads to a complementary effect for preserving edge features in digital images. This LVC mechanism adaptively modifies the degree of diffusion at any image location and is dependent on both local gradient and inhomogeneity. Qualitative experiments, based on general digital images and magnetic resonance images, show significant improvements when the LVCFAB diffusion algorithm is used versus the existing anisotropic diffusion and the previous FAB diffusion algorithms for enhancing edge features and improving image contrast. Quantitative analyses, based on peak SNR, confirm the superiority of the proposed LVCFAB diffusion algorithm. Index Terms—Anisotropic diffusion, discontinuity measure, forward and backward (FAB), image enhancement, inhomogeneity, local variance, noise reduction.
I. INTRODUCTION IGITAL images often suffer from poor contrast and noise from various sources, such as different illumination conditions, image quantization, compression and transmission, etc. These sources of image degradation normally arise during image acquisition and processing and have a direct bearing on the visual quality of the image [1]. In order to interpret these images correctly, image denoising and enhancement are necessary to reduce/remove the image degradation [2]. Of particular interest to this study is the work related to digital image enhancement and noise reduction that focused on the improvement of signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) [3]–[8].
D
Manuscript received August 18, 2006; revised February 27, 2007. This work was supported in part by the 973 Program of the People’s Republic of China under Grant 2006CB701302; in part by the National Natural Science Foundation of China under Grants 40471088 and 40523005; and in part by the Foundation of State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing under Grant 904151695. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Stanley J. Reeves. The authors are with the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China (e-mail:
[email protected]). Digital Object Identifier 10.1109/TIP.2007.899002
The scale-space concept has first been proposed by Iijima [9]–[11] and became popular later on by the works of Witkin [12] and Koenderink [13]. The theory of linear scale-space supports edge detection and localization, while suppressing noise by tracking features across multiple scales [12]–[17]. In fact, the linear scale-space can be expressed by a linear heat diffusion equation [13], [14]. However, this equation was found to be problematic in that all edge features are smeared and distorted after a few iterations of linear diffusion. In order to remedy the difficulties encountered in the linear scale-space theory, Perona and Malik [18] developed an adaptive smoothing and edge detection scheme in which the linear heat diffusion equation is replaced by a selective diffusion that preserves edges. This development led some researches to focus on the development of various anisotropic diffusion models and diverse numerical schemes to obtain the steady-state solutions [19]–[35]. Among them, two classic anisotropic diffusion algorithms inspire us: the forward-and-backward (FAB) diffusion algorithm [33] and the scale-based diffusive filtering algorithm [34]. In Perona and Malik’s scheme [18], the nonlinear diffusion process should be restricted by the “minimum–maximum” principle. This principle, to avoid creating any new minima or maxima, was obeyed by most nonlinear diffusion processes and guaranteed stability in partial differential equations (PDEs) and, thus, avoided the explosion of the nonlinear diffusion process. Instead of restricting the global extrema for the initial signal, Gilboa et al. [33] pointed out that inverse diffusion with a negative diffusion coefficient should be incorporated into image sharpening and enhancement processes to deblur and enhance the extrema of the initial signal (if the extrema are, indeed, singularities and not generated by noise). However, the linear inverse diffusion is a highly unstable process and results in noise amplification. Thus, nonlinear diffusion methods are further extended and combined with the FAB diffusion process to show that sharpening and denoising can be reconciled in image enhancement. On the other hand, the main motivation for anisotropic diffusion is to reduce noise while minimizing image blurring across boundaries, but this process does have several drawbacks, including fine structures in low SNR or CNR regions often disappear and increased blurring occurs in fuzzy boundaries, since anisotropic diffusion does not account for local structure size. Recently, researchers have used a measure of local scale to control diffusion. Elder and Zucker [36] described local scale as the smallest scale (Gaussian kernel) at which the gradient measure is greater than a statistically reliable threshold value. Saha and
1057-7149/$25.00 © 2007 IEEE
WANG et al.: LOCAL VARIANCE-CONTROLLED FORWARD-AND-BACKWARD DIFFUSION
Udupa [34] proposed a scale-based diffusive filtering scheme that uses “object scale” information to stop smoothing around fine structures and across low-gradient boundaries. Instead of using the spatial variance, Chen [37] presented a novel adaptive smoothing algorithm in which a multiscale measure is used to detect contextual discontinuities for edge preservation and control of the smoothing speed, while the local spatial gradient is used for detecting variability during smoothing. In addition, Gilboa et al. [38] developed an adaptive variational mechanism that controls the level of denoising by local variance constraints to better preserve fine scale features. In this paper, a local variance-controlled FAB (LVCFAB) diffusion algorithm is presented. The major advantages of our algorithm are summarized as follows. First, an alternative FAB diffusion process is proposed based on a better behaved diffusion coefficient, in which the transition length between the maximum and minimum values of diffusion coefficient does not increase with the gradient threshold. As a result, our algorithm is more effective at controlling the behavior of the diffusion function when compared to the existing diffusion approaches. Second, the alternative FAB diffusion scheme is integrated with contextual and local discontinuity measures. In this way, important features are enhanced simultaneously with noise reduction. We believe the proposed LVC diffusion algorithm to be a novel mechanism for digital image enhancement and noise reduction. The remainder of this paper is organized as follows. Section II presents the proposed LVCFAB diffusion algorithm. Section III describes simulations including comparative results between several existing anisotropic diffusion schemes and our proposed algorithm. Section IV discusses related issues, and Section V states our concluding remarks. II. LOCAL VARIANCE-CONTROLLED FAB (LVCFAB) DIFFUSION ALGORITHM In this section, we first demonstrate an alternative FAB diffusion process, and then propose a LVCFAB algorithm. Finally, the proposed algorithm is extended and applied to medical image filtering. A. FAB Diffusion Perona and Malik formulated an anisotropic diffusion filter as a process that encourages intraregional smoothing, while inhibiting interregional denoising. The Perona–Malik (PM) nonlinear diffusion equation is of the form (1) is the divergence operwhere is the gradient operator and is a non-negative monotonically decreasing function ator. is constant, then isotropic diffuof local spatial gradient. If sion is enacted. In this case, all locations in the image, including the edges, are equally smoothed. This is an undesirable effect because the process cannot maintain the natural boundaries of is objects. One common form of (2)
1855
is the gradient threshold, which influences the where anisotropic smoothing process. A large will cause low contrast edge features to be smoothed out, while a small leads to slow diffusion within homogeneous regions. More recently, a better behaved diffusion coefficient was proposed [39]
(3) where controls the steepness of the min–max transition region, and controls the extent of the diffusion region in terms of gradient gray level. and are The diffusion coefficients chosen to be nonincreasing functions of the image gradient. This scheme selectively smoothes regions without large gradients. However, in the FAB diffusion process, the points of extrema are emphasized in signal enhancement, image sharpening and restoration. The emphasized extrema occur if these points are indeed represented by singularities and do not emerge as the result of noise. It was observed by Gilboa et al. [33] that if we want to emphasize large gradients we should like to move “mass” from the lower part of a “slope,” upward. This process can be viewed as “moving back in time” along the scale space, or reversing the diffusion process. Mathematically, this can be accomplished simply by changing the sign of the diffusion coefficient (4) However, we cannot simply use an inverse anisotropic diffusion process for image enhancement, because it is highly unstable. There is a major problem associated with the backward diffusion: noise amplification. To remedy this drawback of the linear inverse diffusion process, Gilboa et al. proposed that two forces of diffusion working simultaneously on the signal are needed. First, a backward force should be used at medium gradients, where singularities are expected, and the second, a forward force, implemented for suppressing oscillations and reducing noise. The FAB forces are combined into one coupled FAB diffusion process with a diffusion coefficient that possesses both positive and negative values. Thus, a diffusion coefficient that controls the Gilboa–Sochen–Zeevi (GSZ) FAB diffusion process was proposed [33]
(5)
where is similar to the role of the parameter in the PM diffusion equation; and define the range of backward diffusion, and are determined by the value of the gradient that is emphasized; controls the ratio between the forward and backward diffusion, in general, ; and the exponent paare chosen as ( , ). The diffusion rameters coefficient (5) is locally adjusted according to image features, such as edges, textures, and moments. The GSZ FAB adaptive diffusion process can, therefore, enhance features while locally
1856
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 7, JULY 2007
Fig. 3. Flux and critical points of the AFAB process. Fig. 1. Diffusion coefficient for different gradient threshold values. (a) Coeffi10, 15, 20, 25, k 80, cient c (k 10, 15, 20, 25). (b) Coefficient c (k w 10).
=
=
=
=
Fig. 2. Diffusion coefficient for different gradient threshold values. (a) Coefficient c (k 20, 25, 30, 35, k 80). (b) Coefficient c (k 20, k 50, 55, 60, 65).
=
=
=
=
denoising smoother segments of images. In [40], Gilboa et al. extended their FAB diffusion process and formulated it as a nonconvex variational problem. They proposed a gradient dependent energy functional based on a triple-well potential and employed the GSZ FAB diffusion process as an energy minimizer flow aimed at reversing blurring effects without noise amplification. The variational method permits incorporation of two additional terms into the functional, to account for the significance of additional image attributes. Apart from this, it facilitates the process of regularization. B. Alternative FAB (AFAB) Diffusion It can be observed from Fig. 1(a) and (b) that the transition length between the maximum and minimum coefficient values varies with the gradient threshold (i.e., in the PM process, the transition length varies with , and in the GSZ FAB process, the transition length varies with ). Furthermore, the transition length increases with (or ), which makes controlling diffusion difficult. Thus, an alternative FAB diffusion coefficient (see Fig. 2), preserving the transition range for all or values, is proposed as follows:
(6) where controls the steepness for the min–max transition region of forward diffusion, and controls the steepness for the min–max transition region of backward diffusion. The “minimum–maximum” principle is not kept in the FAB scheme, since we would like to increase gradients (at least in some ranges) and make edges sharper [33]. This property is adapted to the AFAB diffusion coefficient. However, a backward
diffusion process may cause the explosive instability. Now, we analyze the stability of the AFAB diffusion process. Without loss of generality, assume that the edge is aligned with the axis. Let us define the points of extrema of flux in the AFAB diffusion process as follows (see Fig. 3)
where is the flux and . As the equations are solved numerically, we, therefore, have to analyze the implication of the discrete case. Starting with the original diffusion , we replace the first temporal derivative by the equation forward difference, with a time step of
Assuming , according to [33, theorem 2] it cannot get out of this bound in the next time step; hence, the following condition must be satisfied:
where we regard only the case of positive without loss of generality. Replacing the second spatial derivative by the cen), and using the Euler method, the contral difference ( ditions change to
Assigning , and using the flux bound
, , it is sufficient to prove that
and, finally (7) In order to maintain numerical stability in any such scheme, the known CFL bound [41] must be obeyed, i.e., (in the 1-D case)
WANG et al.: LOCAL VARIANCE-CONTROLLED FORWARD-AND-BACKWARD DIFFUSION
Fig. 4. Diffusion coefficient c and the corresponding flux, plotted as a function of the gradient magnitude.
Fig. 5. Diffusion coefficient c and the corresponding flux, plotted as a function of the gradient magnitude.
1857
Fig. 6. FAB processing of a single step noisy signal (from top to bottom): original signal; Gaussianly blurred signal, ( 2), contaminated by Gaussian noise ( = 10); results of FAB diffusion process with diffusion coefficient c and c after 100 iterations, respectively. [k ; k ; w ] = [1=6; 1; 1=3].
=
the bottom of Fig. 6. The GSZ FAB and the AFAB diffusion schemes are also effective in simultaneous enhancement and noise reduction of images, as illustrated in Fig. 7. A horizontal line from the middle of the image (Fig. 8) highlights the diffusion behaviors of the two FAB processes. D. Adaptive Smoothing by Combining Discontinuity Measures
and, therefore, we must ensure that
In our proposed coefficient, creasing in the range
Substituting
is monotonically de. Thus, it is clear that
, we obtain
and since for any , we may conclude that the theorem holds for the discrete case. Otherwise, if is not monotonically decreasing, (7) provides a bound for the time step. C. A Comparison With GSZ FAB Diffusion In (6), the proposed diffusion coefficient has a similar form to that of the GSZ FAB diffusion coefficient. The similarity is due to the two diffusion coefficients—both combine two opposing forces into the diffusion process: one backward force at medium gradients for sharpening, and the second, a forward force at low gradients for smoothing and noise reduction. Furthermore, the two diffusion coefficients have the same property: negative diffusion coefficients are explicitly employed in a predefined gradient range. Plots of the diffusion coefficients and respective fluxes of and are shown in Figs. 4 and 5, respectively. It can be observed that smoothing is performed when the diffusivity function is positive and sharpening occurs for negative diffusion coefficient values. We use the explicit Euler scheme with a forward difference scheme for the time derivative and the central difference scheme (a 3 3 kernel) for the spatial derivative. An example of a blurred and noisy step signal is shown below (see Fig. 6). The enhancement of the step edge and the denoising of the rest of the signal by the AFAB diffusion scheme are shown at
It is well documented that discontinuities in an image are likely to correspond to important features in the image. However, noise corruption can generate discontinuities as well [42]. Therefore, how to measure discontinuities is very significant. A local measure is used to detect variable local discontinuities and it is sensitive to any local intensity change. Nevertheless, it is difficult for this measure to distinguish significant discontinuities from noise. On the other hand, the contextual information, i.e., the attributes of neighboring pixels, is used to reduce ambiguity even in noisy circumstances. Chen [37] derived a contextual discontinuity measure, i.e., inhomogeneity, from the scale-based affinity theory proposed by Saha et al. [34]. The basic idea underlying the scale-based affinity between two and in a gray-level image, is to pixels, for instance find out a proper neighborhood or local scale at two pixels so that all the pixels in the neighborhood meet an intensity uniformity criterion. By combining the contextual discontinuity measure and the local discontinuity measure (i.e., local spatial gradient) for synergy in noise removal and feature preservation, Chen [37] proposed an adaptive smoothing algorithm. For the sake of clarity, this method is briefly described. in the image, its optimal local scale is For each pixel (8) ) is a parameter that determines the size of this where ( is a tolerance parameter and contextual neighborhood and is the unifixed to be 0.85 as recommended in [34]. formity criterion for testing the similarity between pixel and the ensemble of pixels in the boundary region of its neighborhood of size . Afterward, two neighborhoods are defined and based on their optimal local scales
where and
. Suppose that pixels are an aligned pair of pixels in neighborhood
1858
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 7, JULY 2007
Fig. 7. Comparison between FAB process with diffusion coefficient c and c . (a) Original Lena image. (b) Gaussianly blurred Lena image ( obtained after FAB with c . (d) Image obtained after FAB with c .
Fig. 8. Plot of gray-level values obtained along one line of Fig. 7 (from top to bottom): original signal; blurred signal; result of FAB with diffusion coefficient c and c .
and , respectively. Thus, Chen deas fined the inhomogeneity of pixel
= 2). (c) Image
is a nonnegative monotonically decreasing function. where is used to determine to what exParameter tent potentially important features should be preserved in terms of contextual discontinuities, and parameter determines to what extent local discontinuities should be preserved during smoothing. As suggested by Chen [37], the optimal value of can be chosen as the mean of the lowest 20% inhomogeneity values across a whole image, while the optimal value of can be determined as the mean of local intensity differences in hois upmogeneous regions. While the intensity of pixel and are used jointly to determine dated, both is employed weights for local weighted averaging, and for controlling smoothing. E. LVCFAB Diffusion
(9) indicates the degree of where nonuniformity of the regions, defined by and [43], and is the nearest neighborhood of . Meanwhile, they further introduced a nonlinear pixel transformation to highlight higher inhomogeneity that more likely corresponds to important features. That is (10) and are the maximal and minimal inhomowhere geneity values across the entire image, respectively. Combining the two discontinuity measures—inhomogeneity and spatial gradient—Chen [37] developed an alternative adaptive smoothing scheme as follows:
(11)
where is the intensity of pixel and are weighted functions and defined as
at iteration .
(12a) (12b)
In Section II-B and C, we have established stability for small gradient bands in 1-D case for the gradient-based diffusion coefficient , and verified the feasibility of the AFAB diffusion on a variety of signals and images. To efficiently control the diffusion behavior and facilitate image sharpening and denoising, we use two discontinuity measures, local and contextual, in the AFAB diffusion process. We believe that the joint use of the two discontinuity measures leads to a complementary effect for edge feature preservation. Hence, a forward diffusion with the form of adaptive smoothing in the proposed FAB diffusion process is as follows:
(13)
is the diffusion coefficient of the where and proposed LVCFAB diffusion process, are defined as (14a) (14b) In (13), the local and contextual discontinuity measures are used jointly to control the extent of the forward diffusion. That is, local discontinuities detect the details of local structures and contextual discontinuities specify significant features in a given
WANG et al.: LOCAL VARIANCE-CONTROLLED FORWARD-AND-BACKWARD DIFFUSION
1859
image. In order to make the mathematical style of the equations more clear, two functions are defined as and
Thus, we obtain
(15) Meanwhile, the joint use of two measures controls the extent of backward diffusion for the enhancement of edge features. The diffusion coefficient of backward diffusion is defined as
(16) where is the optimal local scale at pixel deand its neighscribed in Section II-D. Assuming that pixel bors are in a homogenous region, is relatively large and will be very small to minimize the effect of and its neighbors reoscillations. Conversely, when pixel will be relatively side in a boundary of different regions, small and will play a very important role for detecting and enhancing significant features. Similarly, we deand fine two functions: . Afterward
Fig. 9. Diagram of the LVCFAB diffusion algorithm.
is a discretization on a 3 3 lattice. An 8-nearest-neighbor discretization of the Laplacian operator is adopted. That is
(19) (17) Combining (15) with (17), we propose a diffusion coefficient of the LVCFAB diffusion process as follows:
, , , and are the mnemonic Here, , , , , 3.2 subscripts for eight directions (i.e., North, South, East, West, North-East, North-West, South-East, and South-West). The superscript and subscripts on the parenthesis are applied to all the terms enclosed, and the symbol stands for for nearest-neighbor differences. is time step: the numerical scheme to be stable.
(18) denotes the original intensity of Given an image, . We summarize the proposed LVCFAB algorithm pixel as follows. 1) Initialization. 1.1) Input a given image . 1.2) Set parameters , , for forward diffusion; , , for backward diffusion; and for maximal iteration number. , is computed by inho2) For each pixel by object scale estimation (OSE) algomogeneity and rithm. . 3) Iterate until 3.1) For each pixel , is computed by (14b). 3.2) The discretization of the anisotropic diffusion is per. Our FAB diffusion algorithm formed to update
A block diagram of the LVCFAB diffusion algorithm procedure is shown in Fig. 9. F. Medical Image Filtering by the LVCFAB Diffusion In medical images, low SNR and CNR often degrade the information and affect several image processing tasks, such as segmentation, classification and registration. Therefore, it is of considerable interest to improve SNR and CNR to reduce the deterioration of image information. In this section, we extend the LVCFAB diffusion coefficient for medical image processing. In an anisotropic diffusion network, the extent of diffusion at each pixel is determined by diffusion coefficients that are monotonically decreasing functions. Thus, diffusion mainly occurs within homogenous regional boundaries, without diffusing across region boundaries at locations with high gradients. Here, the decision of the gradient threshold ( , , ) that classifies an image into homogenous and edge regions is very important.
1860
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 7, JULY 2007
=
Fig. 10. Enhanced images for the “Parrot” image. (a) Original image with 8-bit gray levels. (b) Gaussianly blurred “Parrot” image, ( 1), contaminated by zero-mean Gaussian noise with variance 50. (c) Image enhanced using anisotropic diffusion with c ( = 0.05). (d) Image enhanced using anisotropic diffusion with c ( = 0.2, = 0.05). (e) Image enhanced using FAB anisotropic diffusion with c (k = 50, k = 200, = 0.125, = 0.5). (f) Image enhanced using proposed FAB anisotropic diffusion with c (k = 50, k = 200, = 0.2, = 0.1, = 0.02, = 0.9).
Our FAB diffusion process consists of two steps. First, an adequate spatial scale is identified for local estimation that depends on the local structure. Second, the resulting scale is used as a threshold to represent the minimum reliable scale [36] for each pixel. In edge detection, the pixel-wise Type 1 error is defined using the critical value as [36] (20) , is the standard deviation of noise, where is the scale for the first derivative Gaussian estimator. and Using a fixed Type 1 error rate, can be obtained as (21) How to sample space is one of the open question of the minimum reliable scale technique that needs more attention. Research on scale space theory demonstrates that for natural images logarithmic scale is sufficient to represent the scale space completely [13]. Using a logarithmic sampled scale space ( ), a critical threshold map ( ) is defined based on the multiscale gradient response of the intensity function [44] (22) is the critical threshold function [36]. In the cases where where no prior information is known, we determine the gradient thresholds in the discrete case by calculating the critical thresh. In our algorithm, we set olds at each pixel . The parameters vary gradually along the signal. Image denoising and sharpening are accomplished by inducing different thresholds of the LVCFAB diffusion in different locations. Therefore, the diffusion coef) as ficient is redefined using the critical threshold map (
(23)
III. EXPERIMENTS In this section, our LVCFAB diffusion algorithm is used to test two kinds of image. As a defining characteristic, iterative operations are inevitably involved in anisotropic diffusion. Therefore, implementation of an iterative algorithm highly depends upon the termination time, which causes what we often refer to as the termination problem. Although there still does not exist a widely accepted analytical method, several heuristic methods have been attempted to determine the stopping time to overcome instability in anisotropic diffusion [27], [45]–[47]. As far as simplicity is concerned, the nonlinear cooling method is most suitable for applications as a general denoising scheme. Gilboa et al. [45] proposed a threshold freezing nonlinear cooling method using the cooling rate we refer to as , and applied it to the anisotropic diffusion scheme. Afterward, Smolka et al. [47] extended the technique and presented a FAB anisotropic diffusion algorithm with an incorporated time dependent cooling process (the cooling rate we refer to as ). In this paper, the two nonlinear cooling methods are adopted as objective stopping criteria when comparing anisotropic diffusion methods. A. General Images We demonstrate many computer simulations in order to validate the proposed FAB diffusion technique. Fig. 10(a) shows a 256 256 “parrot” image and we generate a blurred and noisy version of the image, as shown in Fig. 10(b). The results yielded and are deby the anisotropic diffusion algorithms with picted in Fig. 10(c) and (d), respectively. We can observe that the two anisotropic diffusion algorithms result in the loss of important information from the original image, though the noise is entirely removed. A better combination of smoothing and sharpening is given by the GSZ FAB diffusion. The corresponding result is shown in Fig. 10(e). Finally, the image yielded by our is represented in Fig. 10(f). The LVCFAB diffusion with noise is readily removed and this is due to adaptive forward diffusion. Meanwhile, important features, including most of the fine details, are sharply reproduced with the LVCFAB diffusion algorithm. From the perspective of the image quality, the results given by the GSZ FAB diffusion and LVCFAB diffusion are comparable, because the two processes simultaneously enhance, sharpen and denoise images. However, the degree of enhancement of the LVCFAB diffusion could be fixed more appropriately to account for improving the CNR. In order to appraise
WANG et al.: LOCAL VARIANCE-CONTROLLED FORWARD-AND-BACKWARD DIFFUSION
1861
Fig. 11. Intensity values of the row 100 (a) in the original image, (b) in the blurred and noisy image, and in the results given by (c) anisotropic diffusion with c , . (d) anisotropic diffusion with c , (e) FAB anisotropic diffusion with c , and (f) proposed FAB anisotropic diffusion with c
=
1), contaminated by Fig. 12. Enhanced images for the “Tiffany” image. (a) Original image with 8-bit gray levels. (b) Gaussianly blurred “Parrot” image, ( zero-mean Gaussian noise with variance 25. (c) Image enhanced using anisotropic diffusion with c , = 0.05). (d) Image enhanced using anisotropic diffusion with c ( = 0.2, = 0.05). (e) Image enhanced using FAB anisotropic diffusion with c (k = 50, k = 200, = 0.125, = 0.5). (f) Image enhanced using (k = 50, k = 200, = 0.2, = 0.1, = 0.02, = 0.75). proposed FAB anisotropic diffusion with c
Fig. 13. Intensity values of the row 100 (a) in the original image, (b) in the blurred and noisy image, and in the results given by (c) anisotropic diffusion with c , (d) anisotropic diffusion with c , (e) FAB anisotropic diffusion with c , and (f) proposed FAB anisotropic diffusion with c .
the nonlinear behavior of the different anisotropic diffusion algorithms, the intensity values of a row are graphically depicted in Fig. 11. The original noise-free row number 100 (from top to bottom) is shown in Fig. 11(a). The corresponding row in the blurred and noisy image is represented in Fig. 11(b). According to the previous observation, the anisotropic diffusion algorithms with and blur significant edge features and the results are shown in Fig. 11(c) and (d). The data processed by the GSZ FAB diffusion and by the LVCFAB algorithms are depicted in Fig. 11(e) and (f), respectively. It is quite obvious that the GSZ FAB diffusion algorithm achieves a good compromise between sharpening and denoising, while the proposed LVCFAB diffusion algorithm exhibits the best edge-enhanced diffusion behavior [for a comparison, please refer to the original image in Fig. 11(a)]. In order to objectively evaluate the performance of the different algorithms, we use PSNR, which is defined as the following:
dB
(24)
where is the original image and denotes the recovered image. The list of PSNR values that are reported by the different image enhancement and noise reduction algorithms are found in Table I. The good performance of the proposed LVCFAB diffusion is apparent.
TABLE I PSNR VALUES FOR “PARROT” IMAGE
Fig. 12(a) shows the 256 256 image “Tiffany.” We blur and add Gaussian noise to this image [Fig. 12(b)] to show the effects of the various image enhancement algorithms applied in this study. We apply the four anisotropic diffusion algorithms and illustrate the resultant images in Fig. 12(c)–(f). It is observed that the LVCFAB diffusion produces the best image, judged by subjective image quality, compared to the other three algorithms. Fig. 13 highlights the different behavior of the four enhancement processes in further detail. Notice that the LVCFAB diffusion algorithm produce better results both in denoising and enhancing edge features [for a comparison, please refer to the original image in Fig. 13(a)]. Table II shows
1862
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 7, JULY 2007
Fig. 14. Comparison of the four diffusion methods using a MR image. (a) Brain MR image. (b) Diffusion coefficient c map corresponding to Fig. 14 (first iteration). (c)–(h) Filtered images corresponding to the image in (a) resulting from anisotropic diffusion with c ( 0.05), anisotropic diffusion with c ( 0.2, 0.05), FAB diffusion with c (k 50, k 200, 0.125, 0.5), and the proposed FAB diffusion with c ( 0.1, 0.1, 0.02).
=
=
=
=
=
=
=
=
=
=
map corresponding to Fig. 15 Fig. 15. Comparison among the four diffusion methods using a MR image. (a) Original MR image. (b) Diffusion coefficient c (first iteration). (c)–(h) Filtered images corresponding to the image in (a) resulting from anisotropic diffusion with c ( 0.05), anisotropic diffusion with c ( ( 0:05, = 0.1, 0.2, 0.05), FAB diffusion with c (k 50, k 200, 0.125, 0.5), and the proposed FAB diffusion with c
= 0.02).
=
=
=
=
=
TABLE II PSNR VALUES FOR “TIFFANY” IMAGE
=
=
=
original MR image Fig. 15(a) is 256 256 pixels and the diffusion coefficient ( ) map is depicted in Fig. 15(b). The filtered results from the four diffusion algorithms are shown in Fig. 15(c)–(f). As expected, the four algorithms remove noise present in Fig. 15(a). According to the visual analyses of the image quality, the results given by the FAB diffusion and the LVCFAB diffusion are comparable. However, the LVCFAB diffusion algorithm achieves greater contrast and produces more reliable edges, which is especially useful for segmentation and classification purposes necessary in medical image applications. IV. DISCUSSIONS
the evolution of PSNR corresponding to different algorithms during the diffusion process. It is evident from the comparative results that our scheme outperforms the others in terms of sharpening and restoration of the blurred and noisy image. B. Medical Images In this section, we present two examples from MR patient studies. Fig. 14(a) shows the original MR image and Fig. 14(b) ) map, where black represents is the diffusion coefficient ( the zero diffusion coefficient, and white is a diffusion coefficient of one. The results for the four algorithms are displayed in Fig. 14(c)–(f). As seen from all four filtered images, the smoothness in homogeneous regions, such as white matter, seems to be visually the same in all images, while the boundary contrast appears to be higher using the LVCFAB diffusion algorithm when compared with other algorithms. Also, the LVCFAB diffusion enhances fine details better than other diffusion schemes. In Fig. 15, we present another example of the performance of the LVCFAB diffusion algorithm for MR image filtering. The
The aim of this study is to propose a LVCFAB diffusion algorithm, and to apply it in signal and image enhancement as well as image sharpening. We focus on enhancing and sharpening blurry signals, while still allowing some additive noise and blur to interfere with the process. The experimental results have shown that a limitation of the method is related to the amount of noise corruption and blur. Clearly, the LVCFAB diffusion is intended for applications where the noise variance and the blur variance are not too large (typically, the noise variance and the blur variance ). In the presence of highly corrupted and blurred images, the LVCFAB diffusion may fail to obtain satisfactory results because it is difficult to control the diffusion behavior. In our algorithm, there are six tunable parameters , , , , and . As mentioned above, the parameter is parameter in the PM diffusion similar to the role of the equation; is determined by the value of the gradient that is emphasized. These two parameters can be determined by calculating the mean absolute gradient (MAG) [33]. However, we have adopted the nonlinear cooling method to terminate the and are acceptable iterations; it is found that
WANG et al.: LOCAL VARIANCE-CONTROLLED FORWARD-AND-BACKWARD DIFFUSION
choices for most images that are not over noisy and blurred is the cooling rate and we suggest in our experiments. . As argued in Section II-B and that it is selected as Fig. 2, controls the shape of the diffusion function where controls the shape the diffusion coefficient is positive, and of the diffusion function where the diffusion coefficient is negative. We set the two parameters as fixed values in all and . The parameter the experiments, i.e., determines the ratio between the backward and forward diffusion. The parameter should be carefully selected because too small a value results in an imperceptible enhancement effect and too large a value often leads to noise amplification. Gilboa et al. [33] determined the parameter by using the bound: . However, we set so that the maximum forward flux is larger than the maximum backward flux . In practical applications, the above bounds can usually be increased without experiencing major instability. Furthermore, the automatic scale selection for detecting inhomogeneity took most of the time in our algorithm, due to the complex nature of its locally exhausted search. Therefore, one possible future research area is to use appropriate strategies to speed up the computation of scale. As seen from Section II-E and F, there are two forms of and ) in the proposed the diffusion coefficient ( can be considered as LVCFAB diffusion algorithm. and it provides one way to determine the extension of and in the discrete case. For instance, the parameters . Local adjustment of the parameters can be accomplished by calculating the critical threshold . With this strategy, the parameters map at each pixel vary gradually along the signal and enhancement is accomplished by inducing different thresholds in different locations. This is, indeed, required in cases of medical images (for instance, MR images) because there are different fine structures and noise levels in these images. Thus, can effectively the LVCFAB diffusion algorithm with preserve and enhance the sharpness of object boundaries. V. CONCLUSION Digital image acquisition techniques often suffer from low SNR and CNR, which degrade the information contained in the digital image and, thus, reduce its potential utility for industry. We present a novel LVCFAB diffusion algorithm for image enhancement and noise reduction, so as to improve upon the SNR and CNR that preclude the current utility of digital images for industry. The sharpening and smoothing actions in our algorithm are implemented by jointly adopting an alternative FAB diffusion scheme with two different discontinuity measures, i.e., a local discontinuity measure and a contextual discontinuity measure. There are two primary advantages of the LVCFAB diffusion algorithm. First, the alternative FAB diffusion is better at controlling the diffusion behavior than the GSZ FAB diffusion. Second, the two different discontinuity measures are jointly used for edge enhancement and noise reduction. As a result, an increased amount of detail in image sharpening and noise reduction is achieved. The proposed LVCFAB diffusion algorithm is tested on various digital images, including two general images and two medical images. The results from our
1863
LVCFAB image simulations show an improvement in SNR and CNR over the pre-existing algorithms. The improvement of our LVCFAB diffusion algorithm is especially seen in enhancing important edge features when compared with the existing anisotropic diffusion schemes and the GSZ FAB diffusion algorithm. Nevertheless, the algorithm optimization process, particularly the adaptive parameter selection and reduction of computational complexity, requires further work to improve upon the operational utility of the LVCFAB diffusion approach. Furthermore, future research should also explore the variational interpretation of the LVCFAB diffusion process and seek to apply the algorithm for the enhancement of color images. REFERENCES [1] N. Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans, and A. C. Bovik, “Image quality assessment based on a degradation model,” IEEE Trans. Image Process., vol. 9, no. 4, pp. 636–650, Apr. 2000. [2] D. Marr, Vision: A Computational Investigation into the Human Represent and Processing of Visual Information. San Francisco, CA: W. H. Freeman, 1982. [3] J. S. Lee, “Digital image enhancement and noise filtering by use of local statistics,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2, no. 3, pp. 165–168, Mar. 1980. [4] P. Chan and J. Lim, “One-dimensional processing for adaptive image restoration,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, no. 1, pp. 117–126, Jan. 1985. [5] D. C. C. Wang, A. H. Vagnucci, and C. C. Li, “Gradient inverse weighted smoothing scheme and the evaluation of its performance,” Comput. Graph. Image Process., vol. 15, no. 2, pp. 167–181, 1981. [6] K. Rank and R. Unbehauen, “An adaptive recursive 2-D filter for removal of Gaussian noise in images,” IEEE Trans. Image Process., vol. 1, no. 3, pp. 431–436, Jul. 1992. [7] C. B. Ahn, Y. C. Song, and D. J. Park, “Adaptive template filtering for signal-to-noise ratio enhancement in magnetic resonance imaging,” IEEE Trans. Med. Imag., vol. 18, no. 6, pp. 549–556, Jun. 1999. [8] S. M. Smith and J. M. Brady, “SUSAN—A new approach to low level image processing,” Int. J. Comput. Vis., vol. 23, no. 1, pp. 45–78, 1997. [9] T. Iijima, “Basic theory of pattern observation,” presented at the Papers of Technical Group on Automata and Automatic Control, 1959. [10] T. Iijima, “Basic theory on normalization of a pattern (in case of typical one-dimensional pattern),” Bull. Elect. Lab., vol. 26, pp. 368–388, 1962. [11] J. Weickert, S. Ishikawa, and A. Imiya, “Linear scale-space has first been proposed in Japan,” J. Math. Imag. Vis., vol. 10, no. 3, pp. 237–252, 1999. [12] A. P. Witkin, “Scale-space filtering,” in Proc. Int. Joint Conf. Artificial Intelligence, New York, 1983, pp. 1019–1021. [13] J. J. Koenderink, “The structure of images,” Biol. Cybern., vol. 50, pp. 363–370, 1984. [14] J. J. Koenderink and A. J. V. Doorn, “Generic neighborhood operators,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 6, pp. 597–605, Jun. 1992. [15] T. Lindeberg, “Feature detection with automatic scale selection,” Int. J. Comput. Vis., vol. 30, no. 2, pp. 77–116, 1998. [16] A. L. Yuille and T. Poggio, “Scaling theorems for zero-crossings,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 1, pp. 15–25, Jan. 1986. [17] J. Babaud, A. Witkin, M. Baudin, and R. Duda, “Uniqueness of the Gaussian kernel for scale-space filtering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 1, pp. 26–33, Jan. 1986. [18] P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 7, pp. 629–639, Jul. 1990. [19] F. Catté, P. L. Lions, J. M. Morel, and T. Coll, “Image selective smoothing and edge detection by nonlinear diffusion,” SIAM J. Numer. Anal., vol. 29, no. 1, pp. 182–193, 1992. [20] L. Alvarez, P. L. Lions, and J. M. Morel, “Image selective smoothing and edge detection by nonlinear diffusion,” SIAM J. Numer. Anal., vol. 29, no. 3, pp. 845–866, 1992. [21] L. Alvarez and L. Mazzora, “Signal and image restoration using shock filters and anisotropic diffusion,” SIAM J. Numer. Anal., vol. 31, pp. 590–605, 1994.
1864
[22] G. Gerig, O. Kubler, R. Kikinis, and F. A. Jolesz, “Nonlinear anisotropic filtering of MRI data,” IEEE Tran. Med. Imag., vol. 11, no. 2, pp. 221–232, Jun. 1992. [23] R. T. Whitaker and S. M. Pizer, “A multi-scale approach to nonuniform diffusion,” Comput. Vis., Graph., Image Process.: Image Understand., vol. 57, pp. 99–110, 1993. [24] X. Li and T. Chen, “Nonlinear diffusion with multiple edginess thresholds,” Pattern Recognit., vol. 27, no. 8, pp. 1029–1037, 1994. [25] S. T. Acton, “Multigrid anisotropic diffusion,” IEEE Trans. Image Process., vol. 7, no. 3, pp. 280–291, Mar. 1998. [26] S. T. Acton, “Locally monotonic diffusion,” IEEE Trans. Signal Process., vol. 48, no. 5, pp. 1379–1389, May 2000. [27] J. Weickert, “Theoretical foundations of anisotropic diffusion in image processing,” Computing, vol. 11, pp. 221–236, 1996. [28] J. Weickert, B. M. T. H. Romeny, and M. A. Viergever, “Efficient and reliable schemes for nonlinear diffusion filtering,” IEEE Trans. Image Process., vol. 7, no. 3, pp. 398–410, Mar. 1998. [29] J. Weickert, “Applications of nonlinear diffusion in image processing and computer vision,” Acta Math. Univ. Comen., vol. 70, no. 1, pp. 33–50, 2001. [30] Y. Yu-Li, X. Wenyuan, A. Tannenbaum, and M. Kaveh, “Behavioral analysis of anisotropic diffusion in image processing,” IEEE Trans. Image Process., vol. 5, no. 11, pp. 1539–1553, Nov. 1996. [31] B. Fischl and E. L. Schwartz, “Adaptive nonlocal filtering: A fast alternative to anisotropic diffusion for image enhancement,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 1, pp. 42–48, Jan. 1999. [32] M. J. Black, G. Sapiro, D. H. Marimont, and D. Heeger, “Robust anisotropic diffusion,” IEEE Trans. Image Process., vol. 7, no. 3, pp. 421–432, Mar. 1998. [33] G. Gilboa, N. Sochen, and Y. Y. Zeevi, “Forward-and-backward diffusion processes for adaptive image enhancement and denoising,” IEEE Trans. Image Process., vol. 11, no. 7, pp. 689–703, Jul. 2002. [34] P. K. Saha and J. K. Udupa, “Scale-based diffusive image filtering preserving boundary sharpness and fine structures,” IEEE Trans. Med. Imag., vol. 20, no. 11, pp. 1140–1155, Nov. 2001. [35] F. Voci, S. Eiho, N. Sugimoto, and H. Sekibuchi, “Estimating the gradient in the Perona–Malik equation,” IEEE Signal Process. Mag., vol. 21, no. 3, pp. 39–65, Mar. 2004. [36] J. H. Elder and S. W. Zucker, “Local scale control for edge detection and blur estimation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 7, pp. 699–716, Jul. 1998. [37] K. Chen, “Adaptive smoothing via contextual and local discontinuities,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 10, pp. 1552–1567, Oct. 2005. [38] G. Gilboa, N. Sochen, and Y. Y. Zeevi, “Variational denoising of partly textured images by spatially varying constraints,” IEEE Trans. Image Process., vol. 15, no. 8, pp. 2281–2289, Aug. 2006. [39] J. Monteil and A. Beghdadi, “A new interpretation of the nonlinear anisotropic diffusion for image enhancement,” IEEE Trans. Pattern Anal. Machine Intell., vol. 21, no. 9, pp. 940–946, Sep. 1999. [40] G. Gilboa, N. Sochen, and Y. Y. Zeevi, “Image sharpening by flows based on triple well potentials,” J. Math. Imag. Vis., vol. 20, no. 1, pp. 121–131, 2004. [41] R. Courant, K. Friedrichs, and H. Lewy, “On the partial difference equations of mathematical physics,” IBM J., vol. 11, no. 2, pp. 215–234, 1967. [42] W. K. Pratt, Digital Image Processing, 3rd ed. New York: Wiley, 2001. [43] P. K. Saha and J. K. Udupa, “Optimum image thresholding via class uncertainty and region homogeneity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 7, pp. 689–706, Jul. 2001. [44] E. Bayram, G. Yaorong, and C. L. Wyatt, “Confidence-based anisotropic filtering of magnetic resonance images,” IEEE Eng. Med. Biol. Mag., vol. 21, no. 5, pp. 156–160, May 2002.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 7, JULY 2007
[45] G. Gilboa, Y. Y. Zeevi, and N. Sochen, “Image enhancement segmentation and denoising by time dependent nonlinear diffusion processes,” in Proc. IEEE Int. Conf. Image Processing, Thessaloniki, Greece, 2001, pp. 134–137. [46] P. Mrázek and M. Navara, “Selection of optimal stopping time for nonlinear diffusion filtering,” Int. J. Comput. Vis., vol. 52, no. 2–3, pp. 189–203, 2003. [47] B. Smolka and M. Szczepanski, “Forward and backward anisotropic diffusion filtering for color image enhancement,” in Proc. 14th IEEE Int. Conf. Digital Signal Processing, Santorini, Greece, Jul. 2002, pp. 927–930.
Yi Wang received the B.S. and M.S. degrees from Wuhan University, Wuhan, China, in 2002 and 2004, respectively, where he is currently pursuing the Ph.D. degree. His main research interests are related to PDE-based processes applied to image enhancement and noise reduction. His research interests also include remotely sensed image processing and pattern recognition.
Liangpei Zhang (M’06) received the B.S. degree in physics from Hunan Normal University, ChangSha, China, in 1982, the M.S. degree in optics from the Xi’an Institute of Optics and Precision Mechanics of the Chinese Academy of Sciences, Xi’an, in 1988, and the Ph.D. degree in photogrammetry and remote sensing from Wuhan University, Wuhan, China, in 1998. From 1997 to 2000, he was a Professor with the School of the Land Sciences, Wuhan University. In August 2000, he joined the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, as a Professor and Head of the Remote Sensing Section. He has published more than 120 technical papers. His research interests include hyperspectral remote sensing, high-resolution remote sensing, image processing, and artificial intelligence. Dr. Zhang has served as Co-Chair of the SPIE Series Conferences on Multispectral Image Processing and Pattern Recognition (MIPPR) and the Conference on Asia Remote Sensing in 1999; Editor of the MIPPR01, MIPPR05, Geoinformatics Symposiums; Associate Editor of the Geo-spatial Information Science Journal and the Chinese National Committee for the International Geosphere-Biosphere Programme; and Executive Member for the China Society of Image and Graphics.
Pingxiang Li (M’06) received the B.S., M.S., and Ph.D. degrees in photogrammetry and remote sensing from Wuhan University, Wuhan, China, in 1986, 1994, and 2003, respectively. Since 2002, he has been a Professor with the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University. His research interests include photogrammetry and SAR image processing.
