leads to a concise introduction to field of enhancement, which does not ... in image processing and is discussed in detail in many excellent .... transform (DFT) of the image, and (c) the log-transformed magnitude of ... and, therefore, its pdf by.
Digital Image Enhancement Nikolas P. Galatsanos Illinois Institute of Technology, Chicago, Illinois, U.S.A.
C. Andrew Segall Aggelos K. Katsaggelos Northwestern University, Evanston, Illinois, U.S.A.
INTRODUCTION In this entry, we provide a tutorial survey of digital image enhancement algorithms and applications. These techniques are considered throughout the image-processing literature and depend significantly on the underlying application. Thus, the survey cannot address every possible realization or application of digital image enhancement. To address this problem, we classify the methods based on two properties: whether the processing performed is point or spatial and whether it is linear or nonlinear. This leads to a concise introduction to field of enhancement, which does not require expertise in the area of image processing. When specific applications are considered, simulations are provided for assessing the performance.
OVERVIEW The goal of digital image enhancement is to produce a processed image that is suitable for a given application. For example, we might require an image that is easily inspected by a human observer or an image that can be analyzed and interpreted by a computer. There are two distinct strategies to achieve this goal. First, the image can be displayed appropriately so that the conveyed information is maximized. Hopefully, this will help a human (or computer) extract the desired information. Second, the image can be processed so that the informative part of the data is retained and the rest discarded. This requires a definition of the informative part, and it makes an enhancement technique application specific. Nevertheless, these techniques often utilize a similar framework. The objective of this entry is to present a tutorial overview of digital enhancement problems and solution methods in a concise manner. The desire to improve images in order to facilitate different applications has existed as long as image processing. Therefore, image enhancement is one of the oldest and most mature fields in image processing and is discussed in detail in many excellent references; see, e.g., Refs. [1–3]. 388
Image enhancement algorithms can be classified in terms of two properties. An algorithm utilizes either point or spatial processing, and it incorporates either linear or nonlinear operations. In this vein, the rest of this entry is organized as follows: In ‘‘Point-Processing Image Enhancement Algorithms,’’ both linear and nonlinear pointprocessing techniques are presented. In ‘‘Image Enhancement Based On Linear Space Processing,’’ linear spatial processing algorithms are presented. In ‘‘Image Enhancement Based On Nonlinear Space Processing,’’ nonlinear spatial processing algorithms are presented. Finally, we present our conclusions.
POINT-PROCESSING IMAGE ENHANCEMENT ALGORITHMS Point-processing algorithms enhance each pixel separately. Thus, interactions and dependencies between pixels are ignored, and operations that utilize multiple pixels to determine the value of a given pixel are not allowed. Because the pixel values of neighboring locations are not taken into account, point operations are defined as funcions of the pixel intensity. Point operations can be identified for images of any dimensionality. However, in the rest of this section, we consider the two-dimensional monochromatic image defined by a discrete space coordinate system n = (n1,n2) with n1 = 0,1. . .N 1 and n2 = 0,1. . .M 1. The image data is contained in a N M matrix, and the discrete space image f(n) is obtained by sampling a continuous image f(x,y). (For more details on image sampling, see Chapter 7.1 in Ref. [2] or Chapter 1.4 in Ref. [4].) We also assume that f(n) is quantized to K integer values [0,1. . .K 1]. When 8 bits are used to represent the pixel values, K = 256, we refer to them as gray levels. The fundamental tool of point processing is the histogram. It is defined as the function h(k) = nk, where nk is the number of pixels with gray level k = 0,1. . .K 1, and it describes an image by its distribution of intensity values. While this representation does not uniquely Encyclopedia of Optical Engineering DOI: 10.1081/E-EOE 120009510 Copyright D 2003 by Marcel Dekker, Inc. All rights reserved.
Digital Image Enhancement
identify an image, it is quite meaningful for a variety of applications. This is readily apparent in Fig. 1(a) and (b), where we show an aerial image of an airport and its corresponding histogram. From the histogram, we deduce that the image is relatively dark and with poor contrast. Additionally, we conclude that there are three major types of image regions. This conclusion is derived from the two peaks as well as the shape of the histogram in the brighter
389
intensities. In the image, these regions correspond to the dark background, lighter road and building objects, and the bright airplane features.
Basic Point Transformations Point transformations are represented by the expression gðnÞ ¼ T½ f ðnÞ
ð1Þ
where f(n) is the input image, g(n) is the processed image, and T is an operator that operates only at the pixel location n. For linear point operators, Eq. 1 becomes gðnÞ ¼ af ðnÞ þ b
ð2Þ
Linear point transforms stretch or shrink the histogram of an image. This is desirable when the available range of intensity values is not utilized. For example, assume that A ¼ minn ð fðnÞÞ and B ¼ maxn ð fðnÞÞ. The linear transformation in Eq. 2 can map the gray levels A and B to gray levels 0 and K 1. Using simple algebra, the transformation is given by gðnÞ ¼
Fig. 1 Representing an image with its histogram: (a) original aerial image and (b) the corresponding histogram. While the histogram does not completely describe the image, it does suggest that the image contains three region types.
K1 ð f ðnÞ AÞ BA
ð3Þ
The effect of stretching the histogram of an image is shown in Fig. 2(a), where the histogram of Fig. 1(a) is modified. The resulting histogram appears in Fig. 2(b). As can be seen from the figure, the image in Fig. 2(a) is both more pleasing and more informative that the image in Fig. 1(a). This is especially noticeable on the right half of the image frame. Other transformations are also utilized for image enhancement. For example, the power-law transform describes the response of many display and printing devices. It is given by g(n) = c( f(n))g, where the exponent is called the gamma factor, and it leads to a process called gamma correction.[1] (A television or computer monitor typically has a voltage-to-intensity response that corresponds to 1.5 g 3). As a second example, the log transform compresses the dynamic range of an image and is given as g(n) = clog(f(n) + 1). This transform is often employed to display Fourier spectra. In Fig. 3, e.g., we show (a) an image of the vocal folds obtained by a flexible endoscope, (b) the magnitude of the two-dimensional discrete Fourier transform (DFT) of the image, and (c) the log-transformed magnitude of the Fourier transform. (For this image, c = 1.) Note that the actual magnitude in (b) provides little information about the spectrum when compared to the log-transformed image in (c). This image is revisited in a later section, as it facilitates noise removal.
D
390
Digital Image Enhancement
Fig. 2 Stretching the histogram of an image: (a) aerial image after histogram stretching and (b) the corresponding histogram. The stretching procedure increases the contrast of the image and makes objects easier to discern.
Histogram Processing One of the standard methods for image enhancement is histogram equalization. Histogram equalization is similar to the stretching operation in Eq. 3. However, instead of utilizing the entire dynamic range, the goal of histogram equalization is to obtain a flat histogram. This is motivated by information theory, where it is known that a uniform probability density function (pdf ) contains the largest amount of information.[5]
Fig. 3 Visualizing the Fourier spectrum: (a) image of vocal chords obtained by a flexible endoscope, (b) magnitude of Fourier spectrum, and (c) log-transform of Fourier spectrum magnitude. Note that inspecting the magnitude values in (b) provides little insight into the shape of the spectrum, as compared to the visual representation in (c).
Digital Image Enhancement
391
The method of histogram equalization is well described in Chapter 2 of Ref. [2], where the normalized histogram provides the foundation of the method. This histogram provides the probability of occurrence of gray level k in the image f (n) and is expressed as pf ðkÞ ¼ nk =n
ð4Þ
where n = NM, the total number of pixels in the image. Due to the definition of a histogram in Eq. 4, we know that K 1 X
pf ðkÞ ¼ 1
histogram equalization is more effective in bringing out the salient information of the image than linear histogram stretching. Furthermore, it is important to remember that although the theory of histogram equalization strives for a uniform histogram, this is not always achieved in practice. The reason for this discrepancy is that the algorithm assumes that the histogram is a continuous function, which is not true for digital images.
ð5Þ
k¼0
where the function pf (k) can be viewed as the pdf of f (n). The cumulative density function (cdf) is therefore equal to Pf ðrÞ ¼
r X
pf ðkÞ
ð6Þ
k¼0
where it should be clear that pf ðkÞ ¼ Pf ðkÞ Pf ðk 1Þ,
k ¼ 0,1 . . . K 1
ð7Þ
To explain the process of histogram equalization, we first consider the case of continuous intensity values. In other words, we denote by pf (x) and Pf (x) the continuous pdf and cdf of a continuous random variable, x, respectively. These two functions are related by pf (x) = dPf (x)/dx. Furthermore, Pf 1(x) exists and Pf (x) is nondecreasing. Assume that the sought after transformation is given by g ¼ Pf ð f Þ
ð8Þ
The enhanced image g has a flat histogram because the cdf of g is given by Pg ðxÞ ¼ Prðg xÞ ¼ PrðPf ð f Þ xÞ ¼ Pr f P1 ¼ Pf P1 ¼ x f ðxÞ f ðxÞ
ð9Þ
and, therefore, its pdf by pg ðxÞ ¼ dPg ðxÞ=dx ¼ 1
ð10Þ
To flatten the histogram of a digital image, the following procedure is employed. First, Pf (k) is computed using Eq. 6. Then, Eq. 8 is applied at each pixel. This implies that the function Pf can be applied on a pixel-bypixel basis to the image f(n) according to gðnÞ ¼ Pf ð f ðnÞÞ
ð11Þ
Finally, Eq. 3 is used to stretch the histogram. In Fig. 4(a), we show the result of histogram equalization for the airport image in Fig. 1(a). In Fig. 4(b), we show the histogram of the image in Fig. 4(a). By comparing the images in Figs. 2(a) and 4(a), we observe that
Fig. 4 Equalizing the histogram of an image: (a) aerial image after histogram equalization and (b) the corresponding histogram. Histogram equalization often assists in the analysis of images, as it makes objects distinct. This is evident in the line features in the top left portion of the frame.
D
392
Digital Image Enhancement
While histogram equalization attempts to maximize the information content of an image, some applications may require a histogram with arbitrary shape. For example, it might be desirable to match the histograms of two images prior to comparison. Such an objective has been studied within the context of mapping a random variable with a given distribution to another variable with desired distribution, and it is discussed in Refs. [1,3]. IMAGE ENHANCEMENT BASED ON LINEAR SPACE PROCESSING Linear filtering is the basis for linear spatial enhancement techniques. Linear spatial filtering can also be represented by Eq. 1. However, in this case, the value of g(n) depends not only on the value f(n) of but also on the values of f in the neighborhood of n. The input –output relation in Eq. 1 is therefore written as 1 1 X X gðn1 ; n2 Þ ¼ f ðm1 ,m2 Þ hðn1 ,n2 ;m1 ,m2 Þ m1 ¼ 1 m2 ¼ 1
ð12Þ
where h is the function that describes the effect of the linear system T. An interesting subcategory of linear filtering is space-invariant linear filtering. In such a case, the input – output relation in Eq. 1 is written as 1 1 X X gðn1 ,n2 Þ ¼ f ðm1 ,m2 Þ hðn1 m1 ,n2 m2 Þ ¼ f ðn1 ,n2 Þ* hðn1 ,n2 Þ
ð13Þ
where the function h is called the impulse response of the system and the operation in Eq. 13 is called convolution and represented by *. Although the summation in Eq. 13 is over an infinite range, these limits are finite in practice as images have finite support. A useful property of space-invariant filtering is that the input – output relation remains constant over the entire image. Thus, the value of the impulse response depends only on the distance between the input and output pixels and not on their spatial location. Another useful property of linear space-invariant filtering is that it can be performed both in the spatial and in the Fourier frequency domains. The DFT of the discrete impulse response is given by M 1 X
N 1 X
Hðk1 ,k2 Þ Fðk1 ,k2 Þ ¼ Ffhðn1 ,n2 Þ* f ðn1 ,n2 Þg hðn1 ,n2 Þ f ðn1 ,n2 Þ ¼ F 1 fHðk1 ,k2 Þ * Fðk1 ,k2 Þg
ð15Þ
where the operator F 1 represents the inverse DFT.[4] Thus, linear space-invariant filtering is performed either by convolving the input image with the impulse response of the filter or by multiplying on a point-by-point basis the Fourier transform of the image with the frequency response of the filter. The ability to perform linear filtering in both the spatial and the Fourier domains has a number of advantages. First, it facilitates the design of filters because it helps separate the part of the signal that should be retained from the part that should be attenuated or discarded. Second, it helps improve the speed of computation by utilizing a fast Fourier transform (FFT) algorithm for computing the DFT. We present two examples of linear enhancement. In the first example, we show enhancement by linear filtering in the spatial domain, i.e., by using convolution. In Fig. 5(b), we show the result of convolving the image in Fig. 5(a) by the uniform 3 3 mask 2 3 1=9 1=9 1=9 6 7 h ¼ 4 1=9 1=9 1=9 5 ð16Þ 1=9 1=9 1=9
m1 1 m2 1
Hðk1 ,k2 Þ ¼
discrete frequencies. Utilizing the DFT, input –output relationships of linear and space-invariant systems are expressed according to the convolution theorem in either the spatial or frequency domains
The image in Fig. 5(a) has been corrupted by additive Gaussian noise. In Fig. 5(c), we show the result of the same approach when a 5 5 uniform mask is used. Clearly, this type of filtering removes noise although at the expense of blurring image features. In the second example, image enhancement is performed in the Fourier domain by point-by-point multiplication. The regular noise pattern in the image in Fig. 3(a) manifests itself as a periodic train of impulses in both directions of the Fourier domain.[1] This is observed in the log-transformed spectrum of this image in Fig. 3(c). We selected a filter with frequency response 1 for N1 k1 M1 ; N2 k2 M2 Hðk1 ,k2 Þ ¼ 0 elsewhere ð17Þ
hðn1 ,n2 Þ
n1 ¼ 0 n2 ¼ 0
"
exp j ¼ Ffhðn1 ,n2 Þg
2p 2p n1 k 1 þ n2 k2 M N
!#
ð14Þ
where the function H(k1,k2) is called the frequency response, and k1 = 0,1. . .M 1, k2 = 0,1. . .N 1 are the
where the parameters Mi, Ni for i = 1,2 are selected such that the largest area around the central impulse in Fig. 3(c) that does not include any of the other ‘‘satellite’’ impulses is maintained. The magnitude of G(k1,k2) = F(k1,k2) H(k1,k2) is shown in Fig. 6(a). In Fig. 6(b), we show F 1 fGðk1 ; k2 Þg, the enhanced image. Another very popular method for image enhancement is the Wiener filter. This method is optimal in a mean-
Digital Image Enhancement
393
D
Fig. 5 Linear filtering for noise removal: (a) original image corrupted by additive Gaussian noise, (b) image processed with a 3 3 averaging operations, and (c) image processed with a 5 5 averaging operation. The averaging procedure reduces the high-frequency content of the image and attenuates noise.
squared error sense.[5] When the noise is zero mean and uncorrelated with the image, the frequency response of the Wiener filter is defined as Hðk1 ,k2 Þ ¼
Sf ðk1 ,k2 Þ with Sf ðk1 ,k2 Þ þ Sw ðk1 ,k2 Þ
ð18Þ
k1 ¼ 0,1 . . . M 1, k2 ¼ 0,1 . . . N 1 where Sf (k1,k2) and Sw(k1,k2) are the power spectra of the image and noise, respectively. The filtered image is then given by ^f ðn1 ,n2 Þ ¼ F 1 fHðk1 ,k2 Þ Gðk1 ,k2 Þg
ð19Þ
In general, the power spectra are not known and have to be estimated from the observed data. Finding these quantities is not a trivial problem and is investigated in the literature (e.g., iterative Wiener filter[6]). In Fig. 7(a), we show the result of Wiener filtering the noise-degraded image in Fig. 5(a) when the power spectra are estimated from the original images. In Fig. 7(b), we show the result of Wiener filtering the image in Fig. 5 (a) when the power spectra are estimated from the observed data using the periodogram method. Noise-filtering techniques have been studied extensively in the literature, often as a special case of image restoration techniques, when the degradation system is represented by the identity.[7]
394
Digital Image Enhancement
The number of algorithms described by Eq. 20 is immense.[8–10] In the remainder of this section, however, we consider three specific types of nonlinear filtering methods. These types describe a majority of common enhancement algorithms, and we define them as order-statistic, transform-mapping, and edge-adaptive enhancement techniques.
Order-Statistic Filtering Order-statistic filters attenuate intensity values based on their rank within a local processing window. To construct this type of procedure, the following approach is followed. First, the processing window is defined. This requires a definition for the values of m1 and m2 in Eq. 20 for which h(n1,n2; m1,m2, f,. . .) is always zero. Thus, it is analogous to the kernel of a linear filter in that it determines the neighboring pixels that contribute to the filtered result. Having selected the filter parameters, the next step for an order-statistic filter is to sort all of the pixels within the processing window according to their intensity value. The filter then extracts the intensity value that occupies the desired rank in the list and returns it as the output. The spatial location of the returned value varies across the image frame and is controlled by the value of f(n1,n2). While the order-statistic filter can be described using Eq. 20, it is traditionally expressed as gðn1 ,n2 Þ ¼ Ranki ½ f ðn1 m1 ,n2 m2 Þ,m1 ,m2 2 M ð21Þ
Fig. 6 Linear filtering in the Fourier domain: (a) the central part of the Fourier transform in Fig. 3(c) and (b) the inverse transform of (a).
IMAGE ENHANCEMENT BASED ON NONLINEAR SPACE PROCESSING Nonlinear filtering allows for the preservation of image features and the removal of impulsive noise. Unlike linear enhancement methods, the output of these operators is not defined as a linear sum of the input samples. Instead, input images are filtered with highly configurable and adaptive procedures. Most often, the adaptation is based on information from the unprocessed image frame. In this case, the input – output relationship is expressed as gðn1 ,n2 Þ ¼
1 X
1 X
f ðm1 ,m2 Þ
m1 ¼ 1 m2 ¼ 1
hðn1 ,n2 ;m1 ,m2 , f , . . .Þ where additional parameters are allowable.
ð20Þ
where g(n1,n1) is the filtered result, Ranki is the designated rank, f(n1,n1) is the original image frame, and M is the processing window. For simplicity, we assume that the processing window is square although this needs not be the case. Nevertheless, note that the output of the filter is always equal to an intensity value within the local processing window. This is in stark difference to linear methods, where the output is defined as a weighted sum of the input intensities. Choosing the rank of the filter is an important design parameter, and it depends largely on the application. One common choice is to utilize the median, or middle, intensity value within the sorted processing window. The resulting median filter is well suited for removing impulsive noise that is additive and zero mean, and it can be realized using one of two available methods. In the first approach, the pixels in the processing window are all extracted from the original image frame, as is denoted in Eq. 21. In the second approach, a recursive procedure is utilized and pixels in the window are extracted from the previously filtered result when available. Intensity values at the unprocessed locations are
Digital Image Enhancement
395
D
Fig. 7 Wiener filtering: (a) result of filtering with the actual power spectra and (b) filtering utilizing periodogram estimates. Finding the power spectra from the noisy image is nontrivial.
extracted from the original image, and the procedure is expressed as gðn1 ,n2 Þ ¼ Ranki ½ f ðn1 m1 ,n2 m2 Þ, gðn1 m01 ,n2 m02 Þ, m1 ,m2 2 M1 ,m01 ,m02 2 M2
ð22Þ
where M1 are locations in the processing window that have not been filtered, and M2 are locations that have been previously filtered. The advantage of the second approach is that it provides better noise attenuation given the same processing window. This is illustrated in Fig. 8, where the image in (a) is corrupted by salt-andpepper noise. The nonrecursive and recursive median filters then filter the noisy image, and the results are shown in (b) and (c), respectively. Notice that while the recursive approach provides better noise attenuation, both are adept at removing noise. Another choice for the rank is to utilize the maximum (or minimum) value within the ordered list. This operator removes the spatially small and dark (or bright) objects within the image frame, and it is a fundamental operator in the theory of mathematical morphology.[11–13] This field is concerned with the study of local image structure and is originally cast within the context of Boolean functions, binary images, and set theory. When extended to grayscale images, however, the basic dilation and erosion operators correspond to the maximum and minimum order-statistic filters, respectively. Moreover, concatenating the dilation and erosion produces additional processing methods. For example, dilation followed
by erosion is called a close, while erosion followed by dilation is called an open. (The open and close operators can also be concatenated.) Morphological filters have many interesting properties. For example, the open and close operators are idempotent, which means that an image successively filtered by an open (or close) does not change after the first filter pass. Additionally, the dilation and erosion operators are separable. Combining these properties with the fact that finding the minimum (or maximum) value in a list is computationally efficient, the morphological operators are well suited for a variety of enhancement applications including those subject to computational constraints. The filtering characteristics of the morphological operators are illustrated in Fig. 9. The image in Fig. 8(a) is filtered with an erosion (minimum value) and dilation (maximum value). Results appear in (a) and (b), respectively. From the figures, we see that erosion enlarges the dark image regions and removes small bright features, while dilation enlarges the bright image regions and removes small dark features. (The processing window establishes the definition of ‘‘small.’’). The open and close operators are appropriate when feature size should be preserved. These filters are illustrated in (c) and (d), respectively, and the open operation removes small and dark image regions, while the close operation attenuates small bright image regions. Finally, the open –close and close – open operators are shown in (e) and (f) and remove both bright and dark small-scale features. Note, however, that the results in (e) and (f) are not identical. Modifications to the general order-statistic filter are also useful. For example, when the sample values within
396
Digital Image Enhancement
Fig. 8 Median filtering: (a) image corrupted with salt and pepper noise, (b) image processed with nonrecursive median filter and 3 3 processing window, and (c) image processed with recursive median filter and 3 3 processing window. Note that the nonrecursive method does not remove all of the noise.
the processing windows are known with unequal certainty, a weighted operator can be constructed.[14] In this method, a weight is assigned to each location within the window. The weights are normalized so that the sum is equal to the number of pixels in the processing window, and pixel vales within the widow are sorted according to intensity. Although unlike the traditional order-statistic filter, the assigned rank is not equal to the number of intensity values preceding it in the sorted list. Instead, the assigned rank is equal to the cumulative sum of the weights. The intensity value that is closest to (but greater than) the desired rank is then chosen as the filtered result.
Other modifications to the order-statistic filter combine nonlinear and linear filters. For example, an alpha-trim filter sorts the pixels in the processing window according to intensity value. Instead of choosing a single value from the sorted list, however, a number of pixels are extracted (e.g., the middle five values) and, subsequently, processed by a linear filter. The result is an operation that is less sensitive to impulsive noise than a linear filter but is not constrained to intensity values within the original image frame. As a second example, the linear combination of several morphological operators can be considered. With this filtering procedure, the original image is filtered with several morphological operators (e.g., an open – close and
Digital Image Enhancement
397
D
Fig. 9 Morphological filtering of salt and pepper image: (a) erode, (b) dilate, (c) open, (d) close, (e) open – close, and (f) close – open. The open – close and close – open operators are able to remove much of the noise in Fig. 8(a).
398
Digital Image Enhancement
Fig. 10 Denoising with the wavelet transform: (a) three-level transform, where tL(n) and tH(n) are the low-pass and high-pass filters, respectively, and (b) the resulting transform coefficients. Most of the image content appears in x4(n), which corresponds to the upper left of (b).
close – open) and the results are then combined in a weighted sum. As in the previous method, this exploits the performance of the order-statistic filter while allowing intensity values that do not appear in the original image.
S is the set of semantically meaningful features, and t and t 1 are the forward and inverse transforms respectively. In most applications, the transform operators are linear.
Transform Mapping A second form of the nonlinear enhancement algorithms is the transform-mapping framework. In the approach, an image frame is first processed with a transform operator that separates the original image into two components. One is semantically meaningful, while the other contains noise. A nonlinear mapping then removes the noise, and the enhanced image corresponds to the inverse transform of the modified data. This is expressed as gðn1 ,n2 Þ ¼
1 X
1 X
xðm1 ,m2 Þt1
m1 ¼ 1 m2 ¼ 1
ðn1 ,n2 ;m1 ,m2 , f , . . .Þ
ð23Þ
where xðm1 ,m2 Þ 8 1 1 P > < P f ðn1 ,n2 Þtðm1 ,m2 ;n1 ,n2 , f , . . .Þ, xðm1 ,m2 Þ2 S m1 ¼1 m2 ¼1 ¼ > : 2S 0, xðm1 ,m2 Þ= ð24Þ
Fig. 11 Denoising with the wavelet transform: The noisy image in Fig. 5(a) is transformed with the Haar wavelet, and all transform coefficients smaller than a threshold are set equal to zero.
Digital Image Enhancement
Selecting the transform operator and separating the noise from meaningful features are important problems within the transform-mapping method. One common choice is to couple the discrete wavelet transform with a thresholding operation. The wavelet operation processes the image with a filter bank containing low-pass and highpass filters.[15–17] This is illustrated in Fig. 10, where the filter bank and transform coefficients are shown in (a) and (b), respectively. As can be seen from the figure, the discrete wavelet transform compacts the image features into a sparse set of significant components. These correspond to the low-frequency transform coefficients (at the
399
upper left of the decomposition) as well as the significant coefficients within the high-frequency data. Coefficients with small amplitudes are identified as noise and removed by hard thresholding, or setting all coefficients with magnitudes less than a threshold equal to zero. An example of the wavelet and hard threshold approach appears in Fig. 11. In the figure, the image in Fig. 5(a) is transformed with the discrete wavelet transform. (The Haar wavelet is utilized.) Small transform coefficients are then set equal to zero, and the inverse discrete wavelet transform is calculated. Note that increasing the threshold decreases the amount of noise but
Fig. 12 Edge-adaptive smoothing: (a) estimated locations of edges in Fig. 5(a), (b) processing the image with an adaptive 3 3 averaging operation, and (c) processing the image with an adaptive 5 5 averaging operation. The filters are disabled in the vicinity of edges.
D
400
Digital Image Enhancement
Fig. 13 Edge preservation with an adaptive Gaussian kernel: (a) image processed with linear Gaussian operator and (b) image processed with adaptive procedure defined in Eq. 26. Local variance estimates control the filter and preserve edges.
also smoothes the image data. This smoothing is a result of removing some of the salient image features. Alternatives to hard thresholding may improve the image enhancement procedure. For example, the soft threshold method attempts to preserve image content with the rule 8 > < T½ f ðn1 ,n2 Þ b, T½ f ðn1 ,n2 Þ > b xðm1 ,m2 Þ ¼ T½ f ðn1 ,n2 Þ þ b, T½ f ðn1 ,n2 Þ < b > : 0; otherwise ð25Þ
where T is the forward transform, and b is the threshold. Other examples include the use of spatial correlations within the transform domain, recursive hypothetical test, Bayesian estimation, and generalized cross-validation.[18] Edge-Adaptive Approaches Edge-adaptive methods focus on the preservation of edges in an image frame. These edges correspond to significant differences between pixel intensities, and retaining these features maintains the spatial integrity objects within the scene. The general structure of the approach is to limit smoothing in the vicinity of potential edge features. For example, one can disable smoothing with an edge map. This is illustrated in Fig. 12, where (a) are the location of edges estimated from the Fig. 5(a), (b) is the result of filtering with a 3 3 averaging operation at locations that do not contain an edge, and (c) is the result of adapting a 5 5 averaging operation with the same procedure.
A second approach to edge-adaptive smoothing is realized as hðn1 ,n2 ;m1 ,m2 ,f , . . .Þ ( ) 1 ðn1 m1 Þ2 þ ðn2 m2 Þ2 exp ¼ 1 Z 1 þ g^ s2 ðn1 ,n2 Þ s2
ð26Þ
f
where Z is a normalizing constant, s2 is the variance of the filter, sf2(n1,n2) is an estimate of the local variance at f(n1,n2), and g is a tuning parameter. In this procedure, the variance estimate in Eq. 26 responds to the presence of edges, and it reduces the amount of smoothing to preserve the edge features. This is evident in Fig. 13, where the combination of Eqs. 20 and 26 processes the noisy frame in Fig. 5(a). Results shown in (a) and (b) correspond to tuning parameters of 0 and 0.001, respectively. (In the example, s2 = 4.) This illustrates the impact of adaptivity. When the parameter is zero, the filter does not respond to the edge features and results in linear enhancement. When the parameter is 0.001, the filter preserves edges. A final example of the edge-adaptive approach is the anisotropic diffusion operator. This operator attempts to identify edges and smooth the image simultaneously.[19,20] It is realized numerically with the iteration gtþDt ðn1 ,n2 Þ ¼ gt ðn1 ,n2 Þ X ci,t ðn1 ,n2 Þri gt ðn1 ,n2 Þ þDt i ¼ fN,S,E,Wg ð27Þ
Digital Image Enhancement
401
D
Fig. 14 Smoothing with anisotropic diffusion: (a) image produced after 5 iterations of Eq. 27 and (b) image produced after 25 iterations of Eq. 27. Both experiments utilize the diffusion coefficient in Eq. 28 and illustrate the edge preserving properties of the diffusion operation.
where ci,s(n1,n2) is the diffusion coefficient of the ith direction at location (n1,n2), ri is the derivative of gs(n1,n2) in the ith direction, Dt < 0.25 for stability, and g0(n1,n2) = f(n1,n2). The directional derivatives are defined as the simple difference between the current pixel and its neighbors in the North, South, East, and West directions. Construction of the diffusion coefficient determines the performance of the algorithm. When the coefficient is constant (i.e., spatially invariant), then the diffusion operation is isotropic and is equivalent to filtering the original image with a Gaussian kernel. When the coefficient varies relative to local edge estimates, however, object boundaries are maintained. For example, a diffusion coefficient defined as ( ) ri gs ðn1 ,n2 Þ 2 ci,s ðn1 ,n2 Þ ¼ exp k
ð28Þ
where k is the diffusion coefficient; the amount of smoothing is limited when rigs(n1,n2) becomes much larger than k.[19] Alternatively, the coefficient ( ) ri Gðn1 ,n2 * gs ðn1 ,n2 Þ 2 ci,s ðn1 ,n2 Þ ¼ exp k
ð29Þ
utilizes a filtered representation of the current image to estimate edges, where G(n1,n2) is a Gaussian operator
and * denotes two-dimensional convolution. Other coefficients are discussed in Ref. [21]. An example of diffusion is shown in Fig. 14. In the figure, the image in Fig. 5(a) is processed with 27. The diffusion coefficient in Eq. 28 is utilized, k is defined as the standard deviation of rig0(n1,n2), and Dt = 0.24. Results produced by the diffusion operator after 5 and 25 iterations appear in (a) and (b), respectively. As can be seen from the figures, additional iterations increase the amount of smoothing. Nevertheless, the diffusion method smoothes the noisy image while preserving edges. CONCLUSION In this entry, a tutorial survey of image enhancement methods was presented. This is a very extensive topic; therefore, only certain approaches are presented at a rather high level. The list of provided references, consisting mainly of general purpose books, provides more details and also directs the interested reader to additional approaches not covered here. In this entry, we only address grayscale image. Color images play an important role in most applications. On one hand, most of the approaches presented here can be extended to enhance color images by processing separately each of the planes used for the representation of the color image (e.g., RGB, CMY, or YIQ). On the other hand, the correlation among color planes can be used to develop additional enhancement techniques. Furthermore, mapping a grayscale image
402
to a color (or pseudo-color) image can be utilized as an enhancement technique by itself, e.g., in visualizing image data. The interested reader can find out more about color image processing in a number of books appearing in the references, and more specifically in Refs. [22,23].
REFERENCES 1.
Gonzalez, R.C.; Woods, R.E. Digital Image Processing; Addison-Wesley, 2002; 3. 2. Handbook of Image and Video Processing; Bovik, A.C., Ed.; Academic Press: San Diego, 2000; 1. 3. Pratt, W. Digital Image Processing; John Wiley and Sons, 2001; 3. 4. Dungeon, D.; Mersereau, R. Multidimensional Digital Signal Processing; Prentice Hall, 1984; 1. 5. Jain, A. Fundamentals of Digital Image Processing; Prentice Hall, 1988; 1. 6. Digital Image Restoration; Katsaggelos, A., Ed.; Springer Series in Information Sciences, Springer Verlag, 1991; 1. 7. Banham, M.R.; Katsaggelos, A.K. Digital image restoration. IEEE Signal Process. Mag. 1997, 14 (2), 24 – 41. 8. Kuosmanen, P.; Astola, J.T. Fundamentals of Nonlinear Digital Filtering; CRC Press, 1997; 1. 9. An Introduction to Nonlinear Image Processing; Dougherty, E.R., Astola, J.T., Eds.; Tutorial Texts in Optical Engineering, SPIE Press, 1994; 1. 10. Pitas, I.; Venetsanopoulos, A. Nonlinear Digital Filters: Principles and Applications; Kluwer International Series in Engineering and Computer Science, Kluwer Academic Publishers, 1990; 1.
Digital Image Enhancement
11. Serra, J. Image Analysis and Mathematical Morphology; Academic Press, 1982; 1. 12. Serra, J. Image Analysis and Mathematical Morphology, Vol. 2: Theoretical Advances; Academic Press, 1988; 1. 13. Soille, P. Morphological Image Analysis: Principles and Applications; Springer Verlag, 1999; 1. 14. Arce, G.R.; Paredes, J.L.; Mullen, J. Nonlinear Filtering for Image Analysis and Enhancement. In Handbook of Image and Video Processing; Bovik, A.C., Ed.; Academic: San Diego, 2000; 81 – 100. 15. Kaiser, G. A Friendly Guide to Wavelets; Springer Verlag, 1997; 1. 16. Mallat, S. A Wavelet Tour of Signal Processing; Academic Press, 1999; 2. 17. Prasad, L.; Iyengar, S.S.; Ayengar, S.S. Wavelet Analysis with Applications to Image Processing; CRC Press, 1997; 1. 18. Wei, D.; Bovik, A.C. Wavelet Denoising for Image Enhancement. In Handbook of Image and Video Processing; Bovik, A.C., Ed.; Academic: San Diego, 2000; 117 – 123. 19. Perona, P.; Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12 (7), 629 – 639. 20. Geometry Driven Diffusion in Computer Vision. Computational Imaging and Vision; ter Haar Romeny, B.M., Ed.; Kluwer Academic Publishers, 1994; 1. 21. Acton, S. Diffusion-Based Edge Detectors. In Handbook of Image and Video Processing; Bovik, A.C., Ed.; Academic: San Diego, 2000; 433 – 447. 22. Sharma, G.; Trussell, H.J. Digital color imaging. IEEE Trans. Image Process. 1997, 6 (7), 901 – 932. 23. Plataniotis, K.N.; Lacroix, A.; Venetsanopoulos, A. Color Image Processing and Applications; Springer Verlag, 2000; 1.
