Enhancement Techniques for Local Content Preservation and ...

ENHANCEMENT TECHNIQUES FOR LOCAL CONTENT PRESERVATION AND CONTRAST IMPROVEMENT IN IMAGES Chelsy Sapna Josephus and Remya .S Department of Computer Science, University of Kerala, Thiruvananthapuram, India [email protected] , [email protected]

ABSTRACT There are several images that do not have uniform brightness which pose a challenging problem for image enhancement systems. As histogram equalization has been successfully used to correct for uniform brightness problems, a histogram equalization method that utilizes human visual system based thresholding(human vision thresholding) as well as logarithmic processing techniques were introduced later . But these methods are not good for preserving the local content of the image which is a major factor for various images like medical and aerial images. Therefore new method is proposed here. This method is referred as “Human vision thresholding with enhancement technique for dark blurred images for local content preservation”. It uses human vision thresholding together with an existing enhancement method for dark blurred images. Furthermore a comparative study with another method for local content preservation is done which is further extended to make it suitable for contrast improvement . Experimental results shows that the proposed methods outperforms the former existing methods in preserving the local content for standard images ,medical and aerial images . KEYWORDS Adaptive histogram equalization,human visual system ,human vision thresholding,enhancement ,local content preservation.

1. INTRODUCTION Digital image enhancement is necessary to improve the visual appearance of the image .It is also used to provide a better transform representation for future automated image processing such as image analysis, segmentation and recognition [1-2]. To discern the concealed but important information in the images, it is needed that the usage of various image enhancement methods such as enhancing edges, emphasizing the differences, or reducing the noise is done. Processing techniques for image enhancement can be classified into spatial domain enhancement and transform domain enhancement methods. Adaptive histogram equalization(AHE) [3] belongs to spatially non uniform enhancement technique. Spatial domain enhancement techniques deal with the image’s direct intensity values. While the spatially uniform methods use a transformation applied to all pixels of the image, the later methods use an input output transformation that varies adaptively with the local characteristics of the image. Histogram equalization suffers from the problem of being poorly suited for retaining local detail due to its global treatment of the image. Histogram equalization tends to over-enhance the image contrast if there is a high peaks in the histogram resulting in a undesired loss of visual data, of quality and of intensity scale . Also small scale details that are often associated with the small bins of the histogram are eliminated. AHE applies locally varying gray-scale transformation on each small region of the image. This method does not completely eliminate noise enhancement in smooth regions.

Local histogram equalization (LHE) is one of the popular branches for local image enhancement. Generally, LHE uses a small window to define a contextual region (CR) for the centre pixel of that window .Only the block of pixels that fall in this window is taken into the account for the calculation of cumulative density function(CDF). Therefore, as the window slides, the CDF is modified. The CDF is the main contributor for the LHE transform function. Hence, in LHE, the transform function of a pixel is depending on the statistics of its neighbors in CR. Because the transform function changes as a response to the changes in the contents of CR, LHE is also popularly known as adaptive histogram equalization (AHE).

Multi histogram Equalization is another efficient way of enhancing an image. But Multihistogram equalization has generally been limited to bi-histogram equalization, and it has been previously proved that tri-histogram equalization does not have any consistent advantage. So in the previous methods the introduction of the human visual system based multi-histogram equalization method is done . This algorithm focusses on the advantages of multi-histogram equalization with the benefit of an effective quantitative measure to ensure optimal results while removing useless information to avoid the production of unwanted artifacts.This method overcomes this limitation by separating the image into different regions of illumination instead of thresholding by simple pixel intensity values. In this manner, histogram equalization can be done on each region to correct for non uniform illumination. In order to perform this segmentation, model of the human visual system is utilized . HVS-based image enhancement aims to specify the way in which the HVS discriminates between useful and unwanted data.

.

2. METHODS 2.1 HUMAN VISUAL SYSTEM BASED IMAGE ENHANCEMENT(HVS)[12] Human Visual System (HVS) based image Enhancement aims to emulate the way in which the human visual system diffferentiates between useful and unwanted data and it is based on the background illumination and the gradient of an image . The former is arrived at using the following formula:

1 1

∑

1

∑





B(x,y)=   X(i, j) + X(k,l) + X(x, y) +2........................................(1) 2 4 4 2 Q'    Q  where B(x,y) is the background intensity at each pixel,X(x,y) is the input image, Q is all of the pixels which are directly up, down, left, and right from the pixel, and Q' is all of the pixels diagonally one pixel away. Here a parameter BT is also defined, which is the difference in the maximum and minimum graylevels of an image , arrived at using: BT = max(X(x, y)) - min(X(x, y))……………......................................(2) Further, the gradient information is also needed , which is arrived at in the following formula: G1= X(x,y) -X(x,y+1)…………………………......................................(3) G2 =X(x,y) X(x+1,y)…………………………........................................(4)

X’(x,y)=(|G1|+|G2|)/2…………………………........................................(5) where X'(x,y) is the gradient information and G1, G2 are the directional gradients. Finally, we must also take into consideration some parameters concerning the human eye itself, which is referred as Bxi, i= 1,2,3 and Ki, i= 1,2,3. These are arrived at using the following formulas:

B x1 = α 1 BT ……………..............................(6) B x 2 = α 2 BT ..………….............................(7) B x 3 = α 3 BT …………..............................…(8) K1 =

 X ' ( x, y )  1  ……..........(9) * β * max 100  B ( x, y ) 

K 2 = K1 B x 2 ……..............................…...(10) K3 =

K1 ………..................................…..(11) Bx3

Where α1 , α 2 , α 3 are parameters based upon the three distinct regions of response characteristics displayed by the human eye. As a, is the lower saturation level, it is effective to set this to 0. Here α 2 , α 3 , is determined experimentally. Using this information, the image is first broken up into the different regions of human visual response. These different regions are characterized by the minimum difference between two pixel intensities for the human visual system to register a difference. The next step is to threshold the three regions, removing the pixels which do not constitute a noticeable change for a human observer and placing these in a fourth image. These four images are arrived at using the following formula:

Im1=X(x,y) such that Bx 2 ≥ B ( x, y ) ≥ Bx1 &

Im2=X(x,y) such that Bx3 ≥ B ( x, y ) ≥ Bx 2 &

Im3=X(x,y) such that B ( x, y ) ≥ Bx3 &

Im4=X(x,y)

All Remaining pixels

X ' ( x, y ) B ( x, y )

≥ K 2 …….(12)

X ' ( x, y ) ≥ K1 …….(13) B ( x, y )

X ' ( x, y )

( B ( x , y ) )2

≥ K 3 ……..….(14)

These four images are then enhanced separately and recombined to form the enhanced image.The above mentioned thresholding method is also referred as human vision thresholding . HVS based histogram Equalization is a method that uses the concept of multi-histogram equalization. By separating the image into regions by the quality of illumination, such as overilluminated, well illuminated, and under-illuminated, traditional histogram equalization can be used on each region to correct for non-uniform illumination.For this, the utilization of the human visual system to segment the image, using the measure of image enhancement to select α 2 and α 3 is done . The first three images are then equalized separately and unionized, with the remaining pixels filled in. In summary, the algorithm is executed as follows: 1. Obtain the input image and segment image using Human Vision Thresholding algorithm 2. Equalize the three images separately. 3. Recombine the pixels in the three equalized images 4. Fill in the missing pixels to generate the output image . 2.2 HUMAN VISION BASED SEGMENTATION WITH EDGE PRESERVING CONTRAST ENHANCEMENT(EPCE) (HVSedge)[14] This is another method which uses the human vision based thresholding for segmentation. This method is a variation of HVS method . Here unlike the HVS method , the images Im1, Im2 and Im4 satisfying the corresponding thresholding conditions as in HVS method is considered. Then one of the images obtained is enhanced by edge preserving enhancement method and the remaining images are enhanced using classical histogram equalization method . EPCE(Edge preserving contrast enhancement) is a enhancement algorithm which is designed to preserve edges while improving contrast locally by combining the output of an edge-detection algorithm with the original spatial information of the image. This achieves a more robust enhancement algorithm that is able to perform edge detection or enhancement . This enhancement algorithm can be used with any suitable edge-detection algorithm. It uses preprocessing steps to standardize image brightness and several post processing steps to enhance the edges contained.

Step1 : Initially the following operation is done on each image pixel , using the following formula (eqn 15).This operation is based on the local mean at each image pixel.

I ( x, y ) =

2 1+ e

2τ ( x , y ) / λ ( x , y )

− 1 ……………………(15)

where I(x, y) is the output image, τ (x, y) is the gray scale image, and λ is the local statistic of the image used to adjust the transfer function to the local mean. Finally, λ is

 µ ( x, y )   ………....…...(16)  M 

λ ( x, y ) = C + ( M − C )

where C is a user-selected enhancement parameter, with effective range 0 = C < 256, M is the maximum value of the range, and µ(x, y) is the local mean of the image.

Step 2 :

To enhance the contrast a high-pass filter is applied on the image. This image is called I EN. Similarly an application of edge detection algorithm is applied to the another copy of the image obtained from step1, resulting in a second image which is referred as IED. Step 3: Finally, the following formula gives the output-enhanced image: IFEN = A(I(x, y) + IED(x, y)γ × IEN(x, y)α)…...............................…..(17) Where I FEN is the output image and A, α, and γ are user defined operating parameters. Selection of M and C can be done simply and quickly by a human; however, selecting α and γ is a more time-consuming task.(,M = 260 and C = 50). This method preserves the local content of the input image in the output image better than HVS method . Since the local content preservation is of utmost concern in the case of medical images, new methods for the same need to introduced .In the next section , an extension of this HVS method referred as HVSEDBI is proposed, which preserves the local content better than the former above mentioned methods. The proposed HVSEDBI method is a variation of the classical HVS method . Even though the classical HVS method is a good method for enhancement ,it does not preserve the local content of an image . Further more an another extension of this method called HVSedge ,though preserves the content better than classical HVS method ,does not preserve the contents much in the case of images like medical images where local content preservation is of utmost importance. Therefore HVSEDBI is introduced here. This method uses the enhancement scheme for dark blurred images along with the edge preserving contrast enhancement method. The proposed method is applied to different category of grey scale images like medical images and other standard images .Since HVSEDBI focuses on preserving the content of the image , a comparative study of HVSEDBI with another method referred as ‘Multilayered Contrast Limited adaptive Histogram Equalization for local content preservation’ is done.

3.PROPOSED METHODS 3.1. HUMAN VISION THRESHOLDING WITH ENHANCEMENT FOR DARK BLURRED IMAGES FOR LOCAL CONTENT PRESERVATION (HVSEDBI) Another extension of the HVS method is proposed here. In this method the segmentation of the original image into 3 images is done by means of human visual system based(or human vision ) thresholding. In addition to this after the three images are obtained using the thresholding (Im1 ,Im2 and Im4) , the second image is enhanced by using normal histogram equalization , the third image using the edge preserving contrast enhancement method( mentioned in the previous section) and the third image is enhanced by using the following enhancement scheme . In this enhancement scheme there are mainly 2 steps applied to the first image. The first step is to apply unsharp masking to the original image to get a sharper and more detailed image. Second step is contrast enhancement step that is based on a mapping function. In the following section the description of these steps are given in detail. The steps of this enhancement technique is as follows:

Step 1: Unsharp masking step: This step enhances the small structures and bring out the hidden details in the image by using unsharp masking. It only sharpens the areas, which have edges or large amount of details. Unsharp masking is usually performed by first generating a blurred copy of the original image by using laplacian filter and then subtracting it from the original image. This is given by the following formula . I (i, j) =IO (i, j) –I b (i, j)………………................................................…….(18) where I (i, j) is the unsharp masking image, IO (i, j) is the original image,and I blurred copy.

b

(i, j) is the

Here before passing to the contrast enhancement step the unsharp masked image is multiplied by a value, and added to the original image to get the image that will be contrasted. Here , the large features are not altered much, but the smaller ones are enhanced. The result is a sharperand more detailed image.This operation is given by the following formula . g (i, j) =I (i. j) +k IO (i, j)………................................................(19) where g (i, j) is output image, k is the scaling constant. Values for k is taken between 0.2 and 0.7 Step 2: Contrast enhancement step: In this step ,a sliding map window(3x3 window) is moved from the left side to the right side of original image horizontally starting from the image’s upper right corner.A pixel value in the enhanced widow depends only on its value. This means that if the pixel of interest exceeds a certain threshold value ,its value remain unchanged but if the value of the pixel is under the threshold then it will be remapped to a new value . The process can be described with the mapping function M defined below. O =M (i)…………………......................................................(20) where O and i are the new and old pixel values, respectively. The form of the mapping function M that determines the effect of the operation is M=i*c/1+e-i …….….........................................................(21) According to above mapping function the new value of corresponding pixel will be:

if i > t i  O=  .........................................................................(22) c  i +  i * (1 + e − i  if i < t    where c is a contrast factor which determines the degree of the contrast that is needed . After map window reaches the right side of the image ,it returns to the left side and moves down a step below. The process is repeated until the sliding window reaches the right-bottom corner of the entire image. On careful observation of the results, it is seen that local content preservation by this method is far better than the HVS and HVSedge method . The enhancement of contrast by this method is also comparatively better.

3.2 Multilayered Contrast limited adaptive histogram equalization using Frost filter (MCLAHEFROST) This method is an extension of a local histogram equalization method referred as “Multiple layers block overlapped Histogram Equalization “ (MLBOHE)[17] .As in MLBOHE there are 3 stages here ,which is the enhancement stage ,noise reduction stage and merging stage.In the enhancement stage, MLBOHE uses block overlapped histogram equalization method (BOHE) for the enhancement of images. Unlike MLBOHE ,MCLAHEFROST uses Contrast limited adaptive Histogram Equalization method (CLAHE)[17]. CLAHE is a generalization of LHE method. CLAHE method is proposed in order to overcome the problem associated with BOHE, which is the amplification of the noise level, especially in medical images . This limitation can be tackled if the enhancement rate can be controlled. In CLAHE, the enhancement rate is controlled by restricting the slope of the mapping function. A clipping limit is applied to the contextual region’s histogram to reduce the undesired noise amplification and edge-shadowing effect .This is better suitable for medical images. There is also a change in filter usage in MCLAHEFROST in comparison to MLBOHE. While MLBOHE uses median filter in the second stage , MCLAHEFROST uses Frost filter . The 3 stages are given in detail . The aims of the Enhancement stage are to improve the contrast, to reveal the hidden details, and to redistribute the brightness evenly over the image. In order to achieve this, Stage 1 of MCLAHEFROST generates three versions of CLAHE enhanced image from an input image X. If the image has the dimensions of M XN, the dimensions of the windows used for CLAHE(i.e. M XN) are set to M/2 XN/2 M/4XN/4 ,M/8XN/8, respectively. These three CLAHE enhanced versions are denoted as h1, h2, and h3.It assumes that the brightness distribution of h1 is better than input image X. So, h1 can be used to redistribute the brightness in the image. I alsot assumes that h2 enhances the small objects, hence it can be used to reveal the hidden details. Then, it is assumed that h3 emphasizes the borders of the objects inside the image. So, h3 can be used to sharpen the image. CLAHE tends to enhance the noise level in its output images. Thus, the aim of Noise reduction stage is to reduce the noise level from the CLAHE enhanced images. Apparently the noise generated by CLAHE is speckle noise. .Since speckle noise is a multiplicative noise , median filter will only be able to remove a less amount of such a kind of noise .So instead of using that we use a filter that is specifically used for the reduction of speckle noise .It is referred to as the frost filter The Frost filter replaces the pixel of interest with a weighted sum of the values within the nxn moving kernel. The weighting factors decrease with distance from the pixel of interest. The weighting factors increase for the central pixels as variance within the kernel increases. This filter assumes multiplicative noise and stationary noise statistics and follows the following formula: DN

=

∑

−

kae

α

t

nXn

Where

α =

(4

/ n σ '2

)(σ

2

/ I '2

)

.......................................................................(23) K-normalization constant I’-local mean σ =local variance σ ' -image coefficient of the variation value |t|=|X-X0|+|Y-Y0| n-kernal size

Here we are applying the frost filter to each of the different versions of CLAHE images obtained in the previous stage so that significant amount of speckle noise is removed from each image . Therefore, in this stage, h1, h2, and h3 will be filtered with frost filters. The outputs are denoted as h1S, h2S, and h3S. The aim of the merging stage is to produce an output image without saturation artifacts. In order to reduce this problem the mean intensity value of the enhanced image should not deviate much from the mean intensity of the input image. An approach is proposed to combine the enhanced images from Stage 2 back to the input image X. First, all the CLAHE enhanced versions are merged to become one image, P, as defined by the following equation: P=w1*h1S+w2*h2S+w3*h3S .......................................................................(24) where w1 ,w2 and w3 are weighting factors and in order to make these values image dependent these values are proportional to the entropy values as given below An entropy of the image is given by the following equation L − 1 E = − ∑ p ( x ) log 10 p ( x ) ........................................................................(25) x = 0

w1 w2 and w3 are chosen in such a way that it satisfies the following equation

w1: w2 : w3 :: Eh 1S − Ex  : Eh 2 S − Ex  : Eh 3 S − Ex  ...................................(26) Where E h 1S , Eh 2 S , Eh 3 S refers to the entropy of images ,h2S and h3S and Ex refers to the entropy of input image .This will enable the version with the highest entropy difference value to have the highest weighting value .In the merging process at the point where image P is obtained after using the corresponding weights in combination with the speckle noise reduced images as mentioned in the previous section we apply a median filter[3X3] there so that any impulse noise other than the speckle noise if present is removed so that we might get a better output.Then, in order to maintain the shape of the histogram, the final output image Y is given by: Y =alpha*X + beta*P...................................................................(27) where alpha and beta are positive weighting coefficients (alpha + beta=1) MCLAHEFROST is further extended to colour images . This is done by applying the method on the red ,green and blue components of an image .Since this proposed method is only suitable for local content preservation a modification is made to it so that it can be used as a method for contrast enhancement .For this the proposed MCLAHEFROST method is combined with various concepts of existing methods as given below .

3.2.1 Multilayered Contrast limited adaptive Histogram Equalization with Multi histogram equalization (MCLAHEMHE) Here the proposed MCLAHEFROST method is combined with the following method .To surmount drawbacks of normal Histogram equalization methods , the main idea used here is to decompose the image into several sub-images, such that the image contrast enhancement provided by the HE in each sub image is less intense, leading the output image to have a more natural look.Here the image decomposition process is based on the histogram of the image. The histogram is divided into classes, determined by threshold levels, where each histogram class represents a sub-image. The number of sub images into which the original image should be decomposed on depends on how the image is decomposed.In order to enhance contrast, preserve brightness and produce natural looking images,here a method referred to as Multi-HE (MHE) technique [13] which first decomposes the input image into several sub-images, and then applies the classical HE process to each of them is done here . Here a function is used to

decompose the image, conceiving a method MHE method for image contrast enhancement, i.e., Minimum Within-Class Variance MHE (MWCVMHE). Here the function is used to decompose an image based on threshold levels, whereas an algorithm used to find the optimal threshold levels is specified along with it . A criterion for automatically selecting the number of decomposed sub-images is also taken .A cost function, taking into account both the discrepancy between the input and enhanced images and the number of decomposed subimages, is used to automatically make the decision of in how many sub-images the input image will be decomposed on. (a)Multi-Histogram Decomposition Here, decomposing an image is done in such a way that the enhanced images still have a natural appearance. For that clustering the histogram of the image into classes is done , where each class corresponds to a sub-image. By doing that, minimization of the brightness shift yielded by the HE process into each sub-image occurs. With the minimization of this shift, this method is expected to preserve both the brightness and the natural appearance of the processed image. From the multi-threshold selection literature point of view,the problem stated above can be seen as the minimization of the within-histogram class variance , where the within-class variance is the total squared error of each histogram class with respect to its mean value (i.e., the brightness). That is, the decomposition aim is to find the optimal threshold set Tk= {t1k t2k t3k tk-1k } which minimizes the decomposition error of the histogram of the image into k histogram classes and decomposes the image I[0, L −1] into k sub-images I[ls 1,k, lf 1,k]….. I[ls k,k, lf k ,k] where ls j,k and lf j,k are lower and upper gray-level boundaries of each sub-image j when the image is decomposed into k sub-images. They are defined as ls j,k = tj-1k if j>1 and ls j,k=0 otherwise and lf j,k = tjk+1 if j not equal to k and lf j,k is zero otherwise .The discrepancy function for decomposing the original image into k sub-images following the minimization of within-class variance can be expressed as k

Disc ( k ) =

2

l fj , k

∑ ∑ (l − l (I [l m

j ,k s

,l

j ,k f

])) P [

0 , L −1 ]

l

……………..

(28)

j = 1 l = l sj , k

The method conceived with this discrepancy function will be called Minimum Within-Class Variance MHE (MWCVMHE). Note that the mean gray-level (i.e., the brightness) of each subimage processed by the CHE method is theoretically shifted to the middle gray-level of its range, i.e., lm (O[ls ,l f ]) = lmm (I[ls ,l f ]) = (ls + l f ) / 2 . (b) Algorithm for computing the threshold matrix 1 . for q ← 0; q < L ; q + + doD ( l ) q ← φ (0, q); 2. for p ← L; p ≤ k; p + + do 3. D(p + 1) p ← D ( p ) p −1 + φ ( p − 1, p − 1); 4 .PT ( p + 1, p ) ← p − 1; 5 . forq ← p + 1; q ≤ L − k + p ; q + + do 6 .D ( p + 1) q ← - ∞ ; 7. for l ← p - 1; l ≤ q - 1; l + + do 8. if (D(p + 1) q > D ( p ) l + φ ( l + 1, q )) then 9. D(p + 1, q) ← D(p) l + φ ( l + 1, q ); 10 .PT ( p + 1, q ) ← l ; Here ϕ ( p,q) - discrepancy of sub-image I ( p, q) , D( p)q - disc. function Disc( p) up to level q, PT - optimum thresholds matrix

(c) Finding the Optimal Thresholds The task of finding the optimal k −1 threshold levels which segment an image into k classes can be easily obtained from the threshold matrix by using the following conditions The threshold from threshold matrix PT is obtained as t k j = = PT(j+1, t j +1k* )……………………………………(29) where 1