Synthesis of Flash and No-Flash Image Pairs Using ... - IEEE Xplore

2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN)

Synthesis of Flash and No-Flash Image Pairs Using Guided Image Filtering Anand Sharma

Vikrant Bhateja, Member, IEEE

Abhijeet Kumar Sinha

Dept. of Electronics and Communication Engineering, SRMGPC, Lucknow-227105 (U.P), India [email protected]

Dept. of Electronics and Communication Engineering, SRMGPC, Lucknow-227105 (U.P), India [email protected]

Dept. of Electronics and Communication Engineering, SRMGPC, Lucknow-227105 (U.P), India [email protected]

yield a single image better in quality than the original ones [7]. Many denoising algorithms have been proposed in the past such as Gaussian filter [8], Adaptive Image filter [9], Anisotropic diffusion (AD) filter and their variants [10]-[13], Doubly local Weiner filter [14], Sparse 3-D Transform Domain filter [15], adaptive median filter and their variants [16]-[20] to suppress the noise from Gaussian corrupted images in diverse applications. Conventional averaging filters are generally performance limited, owing to the suppression of high frequency structure of the image, leading to the removal of finer details along with noise [8]. On the other hand, an adaptive image filters smoothes the image selectively according to the decimal object scale [9]. This tends to remove ample amount of details leading to blurred edges and boundaries. Anisotropic fourth order diffusion filter shows that a suitable choice for a set of diffusivity function unevenly controls the strength of the diffusion on the direction of the level set and gradient [11]. Doubly local Weiner filter [14] is used for elliptic directional windows in wavelet domain whereas Sparse 3D filter incorporates image segmentation followed by suppression of noise [15]. A. Buades proposed a non-local means algorithm [21], where the pixel weights are calculated by using similarity of local patches. These filters are remedial solution for the images corrupted with medium and high density Gaussian noise. But for digital photography, the accuracy of these filters are not good, so digital images with low noise density can be effectively processed using guided image filter without fading of details near the edges. It tends to perform better denoising along with preserving fine details for the images. This accentuates the selection of guided image filter in the proposed synthesis approach for the processing of flash/no-flash image pairs.

Abstract—This paper dispenses a consensus approach in digital photography to yield a better visual quality image by merging image pairs. Guided image filter is employed in this work at the acquisition level to transfers the structures and fine details of the guidance image to the filtering output along with restoration of details and fine structures in the image. The proposed synthesis approach carry out sub-band decomposition of denoised (flash and no-flash) images processed via guided filtering using 2DDWT. The wavelet coefficients are then synthesized by using max-max decision rule. Simulations are carried out on flash/noflash image pairs contaminated with different levels of additive Gaussian noise and are evaluated on the basis of a no-reference image quality parameter. Significant improvement in quality of the synthesized image has been perceived in comparison to original ones. Index Terms— Flash image, No-flash image, Guided image filter, Discrete Wavelet Transform (DWT).

I. INTRODUCTION The important traits of photography are brightness, contrast and saturation. Lighting during photography aids to capture a scene and replicate the visual richness of a real environment. With the advent of digital camera, digital photography has made it viable to provide a quick and facile means to capture a pair of images, one with flash and another without flash. Photographs captured in the low light environment (i.e. under no-flash) usually suffer from noise contamination, blur artifacts and distorted edges. On the other hand, flash photography accounts for addition of light to the surrounding in order to overcome the above mentioned constraints of no-flash photography. However, lesser values of camera gain and shorter exposure time’s leads to introduction of noise even in the flash image but sharp edges are obtained in this case [1][2]. This superimposition of noise often degrades the quality of image needs a serious address by incorporating an appropriate denoising as well as edge preserving filter for pre-processing along with restoration of details and fine structures in the image [3]-[6]. Flash images possess higher values of signal to noise ratio providing a better means to capture the details of a scene; whereas no-flash images preserves the ambient illumination. Based on this idea, it can be ascertained that a decisive amalgamation of flash/no-flash image pairs could

978-1-4799-5991-4/15/$31.00 ©2015 IEEE

II. PROPOSED METHOD A. Guided Image Filter Guided image filtering is a filtering technique that includes the usage of a guidance image. Guided filter reckons the filtering output by considering the quality attributes of a guidance image, which can be input image itself or different image. It transfers the structures and fine details of the guidance image to the filtering output. The guided filter possesses an edge preserving smoothing property like bilateral

768

2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN)

B. Synthesis Approach for flash and no-flash image pairs When there are two unenviable images of a same scene, one often anticipates acquiring an adequate image by synthesizing them. For instance, in the context of flash/noflash, no-flash image is smoothened by flash image defining the edges to be preserved. The proposed synthesis approach can be explained with the help of block diagram as shown in fig. 1.

filter but it is more efficient in preserving the edges. It has a fast and non-approximate linear time algorithm, regardless of kernel size and intensity values. It is a linear filter which does not suffers from the gradient reversal artifacts [22]-[23]. To define a guided filter, firstly define the input image p, guidance image I and filtered output image q. It is assumed that the output q is a linear transform of the guidance image I in a window w at a pixel k expressed as:

qi = mk I i + nk , ∀i ∈ wk where:

(1)

START

mk and nk are constant linear coefficients that

minimizes the difference between q and p. Precisely, the cost function in the window can be minimized as:

E (mk , nk ) = ¦ ((mk I i + nk − pi )+ ∈ mk 2

INPUT FLASH IMAGE

(2)

INPUT NO-FLASH IMAGE

GUIDED IMAGE FILTERING

i∈wk

GUIDED IMAGE FILTERING

∈ is a regularization parameter to control the value of mk from being too large. The values of

mk and nk can be

SUB-BAND DECOMPOSITION USING 2D-DWT

expressed as:

mk =

1 w

¦I p −μ i

i

k

pk

i∈wk

σk +∈ 2

μk

and

respectively in wk ,

σ k2

(3) MAX-MAX DECISION RULE

nk = pk − mk μ k where

(4) 2D-IDWT

are the mean and variance of I

w represents the no. of pixels in wk and

DISPLAY STNTHESIZED IMAGE

pk is the mean of input image p. Applying the linear model in the entire image and computing the values of mk and nk for the entire window wk

COMPUTE NSI

in the image, the filtered output can be given as:

qi =

1 w

SUB-BAND DECOMPOSITION USING 2D-DWT

¦ (m I

+nk )

k i

(5)

STOP

k :i∈wk

(6)

qi = mi I i + ni

Fig.1. Flow chart of the proposed synthesis approach.

where,

1 mi = w

k∈wi

1 w

k∈wi

ni =

¦m

(7)

¦n

(8)

k

k

The synthesis approach proffers the blending of flash and no-flash images of a same scene; the preprocessing is being carried out using Guided image filter. Both the image pairs are assumed to be contaminated with some noise at the acquisition stage. Therefore, guided image filter is employed at the preprocessing stage to suppress the noise while preserving the edges. Analysis of the filtered images could be carried out

769

2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN)

process, both the image pairs are contaminated by additive white Gaussian noise (with zero mean). Once the noisy images are obtained, guided filtering algorithm is applied on them. In the algorithm, guidance image is considered to be input image itself. For the denoising process along with the edge preservation, certain preliminary parameters which are required to be set such as: window size and regularization factor. Proper initialization and tuning of filter parameters enables us to give a better visual quality image to a reasonable extent. For the simulations in the present work, the window size is set to 3x3 and regularization factor to 0.0009 for restoration of fine details and structures in the image. 2D-DWT is then applied on the filtered image pairs with ‘dmey1’ wavelet family. The performance evaluation of synthesized image is carried out at different noise variance levels: 1, 5, 8 and 10 and the values of NSI are enlisted under table 1. The noisy flash/no-flash image pairs (noise variance of 10) and the corresponding filtered images using guided filtering algorithm along with the synthesized image are shown in fig.2 for the visual assessment. Fig. 2 illustrates that two images (flash and no-flash images) are taken which are contaminated with a noise variance level of 10. The noisy images are then processed by using an edge preserving guided image filter. Then both the images are synthesized by using max-max decision rule. It can be elucidated from fig.2 that the synthesized image has better visual ascribes than original or filtered images. The analysis has been fortified with lower NSI values in the synthesized image as shown in fig. 3.

using sub-band decomposition approaches employing empirical mode decomposition, wavelets or ridgelets [24]-[30]. Images processed using the guided filtering algorithm in the present work is then subjected to sub-band decomposition using 2D-DWT (Level 1, using Discrete Meyer wavelet family). This leads to the decomposition of the filtered image into low pass and high pass wavelet coefficients at different scales. Wavelet based multi-scale approach is well suited to manage the different image resolutions. The obtained wavelet coefficients of both high and low frequency bands are then synthesized by using max-max decision rule [31]. This rule serves to select the in-focus regions from each of the sub-bands by finding the coefficients with maximum value, resulting in a highly focused synthesis. Further, using the same wavelet family and level of decomposition, Inverse Discrete Wavelet Transform (2D-IDWT) is performed to obtain the synthesis image back into the spatial domain. C. Performance Evaluation Reference based parameters such as PSNR and SSIM are available for the Image Quality Assessment (IQA) [32]-[34]. PSNR computes the quality evaluation between the two images based on the degree of error measurement. It computes the peak value of the signal-noise ratio, in decibels (dB), between two images, one being the reference image and the other being the filtered image. Mathematically, PSNR can be expressed as: 2 § · P SN R = 10 log 1 0 ¨ M A X I ¸ ¨ M SE ¸ © ¹

§ = 2 0 lo g 1 0 ¨ M A X I ¨ M SE ©

(9) · ¸ ¸ ¹

= 2 0 log 1 0 ( M A X I ) − 1 0 lo g 10 ( M S E )

(10) (11)

where: MAXI is the maximum possible pixel value of the image and MSE is the mean squared error. As the obtained synthesized image is better in quality than the original flash/no-flash images; hence, none of these image pairs could be treated as an appropriate reference for IQA of synthesized image. Therefore, a non-reference IQA parameter [35]-[36], Noise Suppression Index (NSI) is used to determine the amount of residual noise contained in the resultant image. Mathematically, NSI is expressed as:

N SI =

σ μ

(12)

where: σ and μ is the standard deviation and mean of the concerned image respectively [33]-[34]. III. SIMULATION RESULTS AND DISCUSSIONS In this section, samples of flash and no-flash image pairs are being used as indigenous images. For the simulations

770

(a)

(b)

(c)

(d)

2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN)

(e)

(f)

Fig.4. Variation of NSI against different values of epsilon for synthesized image

The variation of NSI vs sigma and NSI vs epsilon can easily be perceived by Fig.3. and Fig.4 respectively. In Fig.3 as the value of sigma increases, the value of NSI for synthesized image decreases which divulge that the synthesized image have less residual noise at higher value of noise variance. In Fig.4 as the value epsilon increases, the value of NSI for synthesized image decreases.

(g) Table I. Computation of NSI and PSNR (dB) for filtered flash, filtered noflash and synthesized images at different noise variance levels. Noise Variance Filtered Flash Filtered No-Flash 1 5 8 10

NSI

PSNR (dB)

NSI

PSNR (dB)

0.7498 0.7494 0.7490 0.7589

41.1260 37.3182 35.7193 34.9159

0.6192 0.6198 0.6199 0.6205

39.6633 36.0288 34.4307 33.6483

IV. CONCLUSION In this paper, a synthesis approach to blend the quality attributes of flash and no-flash image pairs has been proposed so as to yield a better visual quality image. This approach produces a new image that is of better visual quality than either of the originals. Guided filtering algorithm has been employed to suppress low density Gaussian noise superimposed at the acquisition stage followed by the synthesis in wavelet domain. The edges of both the images also get unaffected after being processed by guided filter. Simulations were carried out on the flash/no-flash image pairs for different levels of additive white Gaussian noise. Obtained results significantly showed that visual quality of the synthesized image was revamped in comparison to original ones as demonstrated by the values of NSI. This approach will be even more functional as camera start to capture multiple images every time a photographer takes a picture.

The obtained results showed that the synthesized image contains less residual noise and is visually effective when compared to original and filtered images. Moreover, it can also be explicated from table I that the quality of the synthesized image has been improved by 26.41% from the filtered flash images and 4.23% from the filtered no-flash images.

REFERENCES [1] E. Eisemann and F. Durand, “Flash Photography Enhancement via Intrinsic Relighting,” ACM Transactions on Graphics, Vol. 23, No. 3, pp. 673-678, August 2004. [2] V. Bhateja, A. Kalsi, A. Shrivastava and A. Lay-Ekuakille, “A Reduced Reference Distortion Measure for Performance Improvement of Smart Cameras,” IEEE Sensors Journal, vol. xx, Spl. Iss. on- Advancing Standards for Smart Transducer Interfaces, pp. 1-10, October 2014. [3] A. Jain and V. Bhateja, “A Novel Detection and Removal Scheme for Denoising Images Corrupted with Gaussian Outliers,” Proc. of IEEE Students Conference on Engineering and Systems (SCES-2012), Allahabad (U.P.), India, pp. 434438, March, 2012.

Fig.3. Variation of NSI against different values of noise variance for synthesized image

771

2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN)

[4] A. Jain and V. Bhateja, "A Versatile Denoising Method for Images Contaminated with Gaussian Noise", Proc. of (ACM ICPS) CUBE International Information Technology Conference & Exhibition, Pune, India, pp. 65-68, September, 2012. [5] A. Gupta, A. Ganguly and V. Bhateja, “An Edge Detection Approach for Images Contaminated with Gaussian and Impulse Noises," Proc. of (Springer) 4th International Conference on Signal and Image Processing (ICSIP 2012), Coimbatore, India, Vol. 2, pp. 523-533, December 2012. [6] A. Srivastava, V. Bhateja and H. Tiwari, “Restoration Algorithm for Gaussian Corrupted MRI using Non-Local Average Filtering,” Proc. International Conference on INformation Systems Design and Intelligent Applications (INDIA-2015), Kalyani (W.B.), India, Vol. xx, pp. xx, January 2015. [7] G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Cohen, and K. Toyama, “Digital Photography with Flash and No-flash Image Pairs,” ACM Transactions on Graphics, Vol. 23, No. 3, pp. 664-672, August 2004. [8] K. Rank and R. Unbehauen, “An Adaptive Recursive 2-D filter for Removal of Gaussian Noise in Images,” IEEE Transactions on Image Processing, Vol. 1, No. 3, pp. 431-436, July 1992 [9] Xiaoliang Qian and Lei Guo, Bo Yu,”An adaptive image filter based on decimal object scale for noise reduction and edge detection,” IEEE Transactions on Communications, Vol. 1, No. 5, pp. 461–465, May 2010. [10] P. Perona and J. Malik, “Scale-Space and Edge Detection using Anisotropic Diffusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 12, No. 7, pp. 629-639,July 1990. [11] R. Hajiaboli, “An Anisotropic fouth order diffusion filter for image noise removal,” Int. Journal of Comp. Vision, Vol. 92, No. 2, pp. 177-191, April 2011. [12] A. Gupta, A. Tripathi and V. Bhateja, "De-Speckling of SAR Images via An Improved Anisotropic Diffusion Algorithm," Proc. of (Springer) International Conference on Frontiers in Intelligent Computing Theory and Applications (FICTA 2012), Bhubaneswar, India, AISC vol. 199, pp. 747-754 , December 2012. [13] V. Bhateja, A. Tripathi and A. Gupta, “An Improved Local Statistics Filter for Denoising of SAR Images,” Proc. of (Springer) 2nd International Symposium on Intelligent Informatics (ISI’13), Mysore (India), Vol. 235, pp. 23-29, August 2013. [14] Peng-Lang Shui, “Image Denoising Algorithm via Doubly Local Wiener Filtering With Directional Windows in Wavelet Domain,” IEEE Signal Processing Letters, Vol. 12, No. 10, pp. 681-684, November 2005. [15] K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering,” IEEE Transactions on Image Processing, Vol. 16, No. 8, pp. 2080-2095, 2007. [16] T. A. Nodes and N. C. Gallagher Jr., “The Output Distribution of Median Type Filters,” IEEE Transactions on Communications, Vol. 32, No. 5, pp. 532–541, May 1984. [17] A. K. Shukla, V. Bhateja R. L. Verma and Mohd. S. Alam, “An Improved Directional Weighted Median Filter for Restoration of Images Corrupted with High Density Impulse Noise,” Proc. (IEEE) International Conference on Relaibility, Optimnization and Information Technology (ICROIT-2014), Faridabad (Haryana), India, pp. 506-511, Feb 2014.

[18] V. Bhateja, C. Malhotra, K. Rastogi and A. Verma, “Improved Decision Median Filter for Video Sequences Corrupted by Impulse Noise,” Proc. (IEEE) International Conference on Signal Processing and Integrated Netwroks (SPIN-2014), Noida (UP), India, pp. 716-721, Feb 2014. [19] V. Bhateja, K. Rastogi, A. Verma and C. Malhotra, “A NonIterative Adaptive Median Filter for Image Denoising,” Proc. (IEEE) International Conference on Signal Processing and Integrated Netwroks (SPIN-2014), Noida (UP), India, pp. 113118, Feb 2014. [20] V. Bhateja, A. Verma, K. Rastogi, C. Malhotra and S. C. Satapathy, “Performance Improvement of Decision Median Filter for Suppression of Salt and Pepper Noise,” Proc. (Springer)International Symposium on Signal Processing and Intelligent Recognition Systems (SIRS-2014), Trivandrum (Kerala), India, vol. 264, pp. 287-297, March, 2014. [21] A. Buades, B. Coll and J.M. Morel, “A Non-Local Algorithm for Image Denoising,” IEEE Transactions on Computer Vision and Pattern Recognition, Vol. 4, No. 2, pp. 60-65, Febuary 2005. [22] K. He , J. Sun , X. Tang, “Guided Image Filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, No. 1, pp. 257-282, March 2013. [23] T. Qiu , A. Wang, N. Yu and A. Song, “LLSURE: Local SUREBased Edge-Preserving Image Filtering,” IEEE Transactions on Image Processing, Vol. 22, No. 1, pp. 80-90, January 2011. [24] M. Campolo, D. Labate, F. La Foresta, F. C. Morabito, A. LayEkuakille, P. Vergallo, “ECG-derived respiratory signal using Empirical Mode Decomposition,” Proc. (MeMeA), Int. Work. On Medical Measurements and Apps., Vol. , No. 5 , pp. 399-403 , May 2011. [25] D. Labate, F. L. Foresta, G. Occhiuto, F. C. Morabito, A. LayEkuakille, P. Vergallo, “Empirical mode decomposition vs. wavelet decomposition for the extraction of respiratory signal from single-channel ECG: A comparison,” Sensors Journal, IEEE, Vol. 13, No. 7, pp. 2666-2674 , June 2013. [26] P. Vergallo, A. Lay-Ekuakille, N. I. Giannoccaro, A. Trabacca, F.C. Morabito, S. Urooj, V. Bhateja, “Identification of Visual Evoked Potentials in EEG detection by Emprical Mode Decomposition,” Proc. 11th Int. Multi-Conference on Sys., Signals and Devices- Conference on Sensors, Circuits & Instrumentation Systems, Vol. , No. 2 , pp. 1-5 , Febuary 2014. [27] A. Lay-Ekuakille, A. Trotta, “Comparison between adjustable window technique and wavelet method in processing backscattering lidar signal,” In Remote Sensing: International Society for Optics and Photonics, Vol. 5240 , No. 1 , pp. 72-82 , January 2004. [28] A. Krishn, V. Bhateja, Himanshi, A. Sahu, “ Medical Image Fusion using Combination of PCA and Wavelet Analysis,” Proc. 3rd International Conference on Advances in Computing, Communications and Informatics, Vol. , No. 9 , pp. 986-991 , September 2015. [29] A. Krishn, V. Bhateja, Himanshi, A. Sahu, “PCA based Medical Image Fusion in Ridgelet Domain,” Proc. 3rd Int. Conf. on Front. in Intelligent Comp. Theo. and App., Vol. 328, No. xx , pp. 475-482 , 2015. [30] V. Bhateja, R. Verma, R. Mehrotra, S. Urooj, A. Lay-Ekuakille, V. D. Verma, “A Composite Wavelets and Morphology Approach for ECG Noise Filtering,” Proc. 5th Int. Conf. on Pat. Recog. and Mach. Intell., Vol. 8251, No. , pp. 361-366 , 2013.

772

2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN)

[31] D. Sahu, K. Shah, M. P. Parsai, “Different Image Fusion Techniques- A Critical Review,” Int. Journal of Modern Engg. Research, Vol. 2 , No. 5 , pp. 4298-4301 , May 2011. [32] P. Gupta, P. Srivastava, S. Bharadwaj, V. Bhateja, “A HVS based Perceptual Quality Estimation Measure for Color Images,” ACEEE Int. Journal on Sig. & Image. Proc., Vol. 3, No. 1 , pp. 63-68 , January 2012. [33] P. Gupta, P. Srivastava, S. Bharadwaj and V. Bhateja, “A Novel Full-Reference Image Quality Index for Color Images,” Proc. of the (Springer) International Conference on Information Systems Design and Intelligent Applications (INDIA 2012), Vishakhapatnam, India, pp. 245-253, January 2012.

[34] P. Gupta, N. Tripathi, V. Bhateja, “Multiple Distortions Pooling Image Quality Assessment,” Int. Journal on Convergence Computing , Vol. 1, No. 1 , pp. 60-72 , January 2013. [35] M. Trivedi, A. Jaiswal, V. Bhateja, “A No-Reference Image Quality Index for Contrast and Sharpness Measurement,” Proc. 3rd Int. Adv. Comp. Conf., Vol. , No. , pp. 1234-1239 , 2013. [36] S. Singh, A. Jain, V. Bhateja, “A Comparative Evaluation of Various Despeckling Algorithms for Medical Images,” Proc. Int. Info. Tech. Conf. & Ex., Vol. , No. , pp. 32-37 , 2012.

773