Noise Estimation Using Statistics of Natural Images - IEEE Xplore

2011 18th IEEE International Conference on Image Processing

Noise Estimation using Statistics of Natural Images Guangtao Zhai and Xiaolin Wu ECE Department, McMaster University, Ontario, Canada, L8S 4K1 Email: [email protected] Abstract—We develop a framework for estimating noises of natural images using two important properties of natural image statistics: high kurtosis and scale invariance of natural images in certain transform domains. We examine the effects of additive independent noise on the third and fourth moments of the transformed image signal (skewness and kurtosis). By exploring the said priors of high kurtosis and scale invariance of natural image statistics in 2D discrete cosine transform domain and random unitary transform domain, we derive constrained nonlinear optimization algorithms for accurate estimation of noise variance. Simulation and comparative study show that the proposed approach is capable of estimating the variance of Gaussian additive noise with a relative error as low as one percent. Moreover, the new estimation approach is shown to be effective on multiplicative-additive compound noises as well. This work can significantly improve the performance of existing denoising techniques that require the noise variance as a critical parameter.

where y is transformed Y , and n is the noise in the transform domain. Note that N and n will have the same variance, as long as the transform is unitary (see Section II for more details). Since x and n are independent, the skewness of y can be derived straightforwardly

I. N OISE E FFECT ON M OMENT S TATISTICS OF NATURAL I MAGES

From (2) we have µ4 (·) = (K(·) + 3)σ 4 (·), and letting this into (5) leads to σ 4 (n) σ 4 (x) K(x) + 4 K(n). (6) K(y) = 4 σ (y) σ (y) Equations (4) and (6) reveal that the skewness and kurtosis of y are essentially weighted averages of those of x and n and the weights are determined by the ratios between the standard deviation of signals. Again, from the independence between x and n, we have

An important research topic in natural image statistics is the analysis of the distribution of the images in transform domains [1], [2]. The most direct statistical analysis is the study of the moments, and since the commonly used lower moments such as mean and variance do not provide much information regarding the shape of the distribution, in natural image statistics, the most widely studied moment statistics are the skewness and kurtosis [3], which respectively refer to the normalized third and fourth central moments. For a random variable x, its skewness is defined as κ3 (x)

µ3 (x) S(x) = 3/2 = 3 σ (x) κ2 (x) th

µ4 (y)

In this paper we work with excess kurtosis instead of classic fourth standardized moment definition K(x) = σµ44 (x) (x) to facilitate the following analysis. Skewness and kurtosis measure the asymmetry and peakedness of the distribution, respectively. Assuming an additive noise model Y = X + N in the pixel domain, where X is the original image, N is the noise and Y is the noisy observation of X. If X and N are independent, then after transforming X, we have the transformed signal x y = x + n,

978-1-4577-1303-3/11/$26.00 ©2011 IEEE

(3)

=

k4 (x) + k4 (n) + 3σ 4 (y)

=

µ4 (x) − 3σ 4 (x) + µ4 (n) − 3σ 4 (n) + 3σ 4 (y)

=

K(x)σ 4 (x) + K(n)σ 4 (n) + 3σ 4 (y).

 S(y) =

(2)

k4 (y) + 3σ 4 (y)

(5)

(7)

By letting (7) into (4) and (6), we arrive at

th

µ4 (x) κ4 (x) = 4 − 3. κ22 (x) σ (x)

=

σ 2 (y) = σ 2 (x) + σ 2 (n).

(1)

where κk is the k cumulant function [4] and µk is the k central moments. And the kurtosis of x is defined as the fourth cumulant divided by the square of the second cumulant, which equals the normalized fourth central moments minus 3 K(x) =

σ 3 (x) σ 3 (n) µ3 (x) µ3 (n) + 3 = 3 S(x) + 3 S(n). (4) 3 σ (y) σ (y) σ (y) σ (y) For kurtosis, from the relationship between cumulant and the central moments we have k4 (·) = µ4 (·) − 3σ 4 (·). Considering the independence between x and n, we further have S(y) =

σ 2 (y) − σ 2 (n) σ 2 (y)

 32

 S(x) +

σ 2 (n) σ 2 (y)

 23 S(n),

(8)

2  2 2 σ 2 (y) − σ 2 (n) σ (n) K(x) + K(n) (9) σ 2 (y) σ 2 (y) which enable us to quantify the relationships between the third and fourth moments (skewness and kurtosis) of the original and the noisy transformed signals using the second moments (variances) of the noise n and the noisy observation y. We emphasize that the only premise of the analysis in this section is an independent additive noise model as given in (3), and there is no underlying assumption about the distributions of x and n. Note that fifth or higher moments can also be analyzed with the same method outlined in this section. However, higher moments are more sensitive to noises and outliers, and much more samples are needed in the estimation. So only skewness and kurtosis are used in the proposed noise estimation algorithms to be introduced below.

1857



K(y) =

2011 18th IEEE International Conference on Image Processing

Fig. 1.

Frequency index vs. skewness using discrete cosine transform.

Fig. 3.

Frequency index vs. kurtosis using random unitary transform.

frequency indexes for which the skewness and kurtosis can be assumed to be constant, we can estimate the noise variance by solving the following constrained nonlinear programming problems. From (8) and (9) we have

Fig. 2.

b S(x), b S(n), σ b2 (n) =

Frequency index vs. kurtosis using discrete cosine transform.

arg min

X

(10)

S(x),S(n),σ 2 (n) i∈I

‚ ‚ «3 „ 2 «3 „ 2 ‚ ‚ σ b (yi ) − σ 2 (n) 2 σ (n) 2 ‚ ‚b S(x) − S(n)‚ ‚S(yi ) − 2 2 ‚ ‚ σ b (yi ) σ b (yi ) 1 1 X b subject to |S(yi )| ≤ |S(x)| |I| i∈I

II. E STIMATION OF N OISE USING S TATISTICAL P RIORS A. Scale Invariance Under Discrete Cosine Transform Relationships (8) and (9) are derived for transform coefficients rather than pixel values. For the task of noise modeling, to make the powers (variances) of the noises the same in the pixel domain and the transform domain, unitary transforms have to be used. An immediate choice for the unitary transform is the 2D Discrete Cosine Transform (DCT) that has been extensively used in image processing. The distribution of DCT coefficients has been thoroughly studied. Heavily tailed distributions such as Laplacian, generalized Laplacian and generalized Gaussian were proposed to model the DCT coefficients [5], [6]. Huang suggested that DCT coefficients of natural images have large kurtosis and low skewness [7]. And recently, assuming the kurtosis of DCT coefficients is scale invariant, Zoran and Weiss [8] proposed a noise estimation method. However, in this research we found that Zoran and Weiss’s assumption may not hold for many natural images. We have plotted the scale (formed by indexing the 2D DCT frequency from low to high) vs. skewness and kurtosis for two images from the widely used Kodak set in Fig. 1 and Fig. 2, which show many outliers violating the scale invariance assumption. A closer inspection of those outliers over the frequency index reveals that those outliers are mainly caused by the dominant edges in the images in the horizontal and vertical directions. As such, the assumed scale invariance of DCT coefficients is invalid for images with dominant edges in directions aligned with the 2D DCT filters used. In fact, as pointed out by Daugman, those larger outliers of the kurtosis indicate that the orientations of the filters align with major edges in the image [9]. To minimize the effects of those dominant edges, we reject those frequencies corresponding to horizontal and vertical directions, in which most edges of natural images are oriented. By excluding these outliers, the skewness and kurtosis become very close to constant. Applying (8) and (9) on different

and b b K(x), K(n), σ b2 (n) =

arg min K(x),K(n),σ 2 (n)

X

(11)

i∈I

‚ ‚ „ 2 «2 „ 2 «2 ‚ ‚ σ b (yi ) − σ 2 (n) σ (n) ‚b ‚ K(y ) − K(x) − K(n) ‚ ‚ i 2 2 ‚ ‚ σ b (yi ) σ b (yi ) 1 X 1 b subject to K(yi ) ≤ K(x) and K(x), K(n) ≥ −2 |I| i∈I

where I denotes the set of selected frequency indexes. The constraint here reflects our other observation that the additive noise reduces the magnitude of skewness and kurtosis of the DCT coefficients. The second constraint for (11) comes from the fact that kurtosis by definition is equal to or greater than −2. Note that in (10) and (11), the skewness/kurtosis of the noise n and the original signal x can also be estimated, and can be used to further infer the distributions of n and x. We perform `1 minimization in (10) and (11) to further reduce the influence of the outliers. In contrast, the authors of [8] used the `2 -norm in their algorithm. B. Kurtosis Invariance under Random Unitary Transform Huang first noticed that random transform coefficients of natural images have high kurtosis [7]. In addition, we found empirically that for random transform coefficients the kurtosis is not only high but also remains approximately a constant. This is a very useful property in estimating noise variance. For a k × k data matrix A, its 2D random unitary transform (RUT) can be computed as follows: we first compute the QR decomposition of a k × k random matrix C as QR = C. We then extract the diagonal elements of the upper triangular matrix V = diag(R) and normalize it as V = V /|V |. A new upper triangular matrix is formed as R0 = diag(V 0 ) and the random unitary transform matrix is computed as T = QR0 .

1858

2011 18th IEEE International Conference on Image Processing

The unitary of T is obvious since T T T = (QR0 )T QR0 = R0T R0 = Ik where we use the fact that R0 is a diagonal matrix with normalized entries. The 2D RUT for A can then be computed as B = T AT T . We empirically observed that the 2D RUT produces much fewer outliers than 2D DCT for the kurtosis statistics. The kurtosis values are plotted versus 2D RUT coefficient index in Fig. 3. The figure demonstrates the near constancy of the kurtosis values. Based on this constancy the non-linear estimation algorithm of (13) can be used to estimate the noise variance. Here all the 2D RUT coefficients are included in I. III. I MPLEMENTATIONS AND E XPERIMENTAL R ESULTS A. Additive Gaussian Noise Model In image processing, especially for denoising, the noise is often assumed to be Gaussian. Most published denoising algorithms were designed, optimized and tested using pixel domain additive Gaussian noise model. If noise N in image domain is i.i.d. and Gaussian, then the transformed noise n is Gaussian by the central limit theorem, therefore, S(n) = 0 and K(n) = 0. This combined with (10) and (11) yields, ‚ ‚ «3 „ 2 ‚ X‚ σ b (yi ) − σ 2 (n) 2 ‚ ‚b S(x) ‚ ‚S(yi ) − ‚ σ b2 (yi ) S(x),σ 2 (n) i∈I ‚ 1 (12)

b S(x), σ b2 (n) = arg min

where X and Y are the original and noisy images, and s1 and s2 are the additive and multiplicative parameters. While exact statistical modeling of the compound noise in the transform domain is difficult due to the signal-dependent nature of the noise, we can still empirically estimate the kurtosis of the compound noise. Clearly, s1 N corresponds to the additive part of the noise and therefore has zero skewness and kurtosis in the transform domain. For small s2 the term s2 XN can also be regarded as approximately additive. However, since the image signal X is not i.i.d., the transformation of s2 XN will be non-Gaussian. According to the findings of natural image statistics, the transform coefficients of X have very large positive kurtosis, and the multiplication of X with N increases the randomness of the signal, reducing its kurtosis. As a result, the kurtosis of transformed s2 XN will be larger than zero but much smaller than that of clean natural image signals. In fact, according to our experiments with a wide range of noise parameters s1 ∈ (0 20], s2 ∈ (0 0.2], the transform coefficients of s2 XN are leptokurtic, having kurtosis in the range of [0.8 1.2]. As such we empirically set K(n) = 1 in (11) when estimating the variance of the compound noise, and the objective function of the estimation algorithm becomes b K(x), σ b2 (n) = ‚ «2 „ 2 «2 ‚ „ 2 ‚ X‚ σ (n) ‚ σ b (yi ) − σ 2 (n) ‚b K(x) + arg min ‚ ‚K(yi ) − σ b2 (yi ) σ b2 (yi ) ‚ K(x),σ 2 (n) i∈I ‚ 1 (15)

‚ ‚ «2 „ 2 ‚ X‚ σ b (yi ) − σ 2 (n) ‚b ‚ K(x) ‚K(yi ) − ‚ ‚ σ b2 (yi ) K(x),σ 2 (n) i∈I ‚  2 2 1 σ (n) (13) where term reflects the influence of the multiσ b2 (yi )

b K(x), σ b2 (n) = arg min

in which the constraints are dropped for simplicity. The noise estimation methods in (12) and (13) are tested on the Kodak database using additive Gaussian noises N ∼ N (0, σ), σ = [5, 30]. The noise estimation results averaged over the database are listed in Table I, and compared with those of [8]. The proposed algorithms outperform the method of [8] consistently in almost all the test conditions. For low noise level σ ∈ [5, 10], the relative estimation error is around 5%. And as the noise level increases the relative error of the proposed algorithms becomes even smaller. For σn ≥ 25 the new algorithms can estimate σn with a relative error below 1%. The proposed DCT-based algorithms are more accurate for low to moderate noise levels, while the RUT-based algorithm is better for high noise levels. In practice, all three proposed algorithms can be applied and their results can be averaged to generate a more robust estimate. Also, one can run the RUTbased algorithm multiple times and fuse the resulting estimates to improve the accuracy. B. Compound Noise Model Real-world noises are more complex than the mathematically convenient additive Gaussian model. Tian et. al [10] suggested an additive and multiplicative compound noise model for noises of CCD and CMOS sensors Y = X + (s1 + s2 X)N,

N ∼ N (0, 1)

(14)

plicative term of the noise. The estimated noise variance σ b2 (n) can then be fed into existing denoisers to suppress the compound noise. For comparison, we illustrate the denoising results for compound noises using BM3D [11] in Fig. 4 to Fig. 7. Although BM3D was designed for additive Gaussian noise, with properly chosen noise variance, it can also remove multiplicative noises effectively. We can observe that with Gaussian noise assumption K(n) = 0, the algorithm in (13) generally underestimates the noise’s strength, so the noise cannot be completely removed. And with K(n) = 1, the algorithm in (15) substantially improves the accuracy of the estimation and the denoised images are of higher objective and subjective qualities. IV. C ONCLUSION This paper presents a technique of using statistical priors from natural image statistics for noise estimation. The high kurtosis and scale invariant properties of natural images in discrete cosine transform and random unitary transform domains are discussed. Based on these properties, the noise variance can be effectively estimated via constrained nonlinear programming. The estimation algorithm have been tested with additive Gaussian noise as well as additive-multiplicative compound noise and they outperform state of the art noise estimation methods.

1859

2011 18th IEEE International Conference on Image Processing

Fig. 4. Denoising with RUT based kurtosis analysis on kodim01. From left to right: original image; compound noise contaminated s1 = 5, s2 = 0.2, PSNR=19.16 dB; BM3D denoised, σ ˆ = 19.07, K(n) = 0, 23.17 dB; BM3D denoised, σ ˆ = 26.69, K(n) = 1, 25.98 dB

Fig. 5. Desnoising with RUT based kurtosis analysis on kodim06. From left to right: original image; compound noise contaminated s1 = 20, s2 = 0.15, PSNR=15.86 dB; BM3D denoised, σ ˆ = 21.91, K(n) = 0, 19.94 dB; BM3D denoised, σ ˆ = 36.91, K(n) = 1, 25.26 dB

Fig. 6. Denoising with RUT based kurtosis analysis on kodim07. From left to right: original image; compound noise contaminated s1 = 5, s2 = 0.2, PSNR=19.21 dB; BM3D denoised with σ ˆ = 28.57 with K(n) = 0, 19.21 dB; BM3D denoised with σ ˆ = 27.09 with K(n) = 1, 30.07 dB

Fig. 7. Denoising using RUT based kurtosis analysis on kodim08. From left to right: original image; compound noise contaminated s1 = 20, s2 = 0.2, PSNR=14.83 dB; BM3D denoised with σ ˆ = 34.12 with K(n) = 0, 21.31 dB; BM3D denoised with σ ˆ = 41.23 with K(n) = 1, 23.60 dB TABLE I P ERFORMANCE ANALYSIS ON ADDITIVE G AUSSIAN N OISE . KD: DCT- KURTOSIS BASED , SD: DCT- SKEWNESS BASED , KR: RUT- KURTOSIS BASED . σn = 5

σn = 10

σn = 15

σn = 20

σn = 25

σn = 30

¯ KD SD KR [8] KD SD KR [8] KD SD KR [8] KD SD KR [8] KD SD KR [8] KD SD KR K D relative error % 9.44 3.89 13.01 4.38 4.89 2.70 2.51 4.61 4.22 1.39 2.96 2.25 3.81 1.16 2.09 0.85 3.50 0.94 0.35 0.54 5.64 0.80 0.61 0.42

V. ACKNOWLEDGEMENT This work was supported in part by NSFC (61001145), postdoctoral foundation of China 20100480603 and postdoctoral foundation of Shanghai 11R21414200. R EFERENCES [1] E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annual Review of Neuroscience, vol. 24, pp. 1193–1216, 2001. [2] A. Srivastava, A. B. Lee, E. P. Simoncelli, and S.-C. Zhu, “On advances in statistical modeling of natural images,” Journal of Mathematical Imaging and Vision, vol. 18, no. 1, pp. 17–33, 2003. [3] D. B. Mumford, New Directions in Statistical Signal Processing: From Systems to Brain. Cambridge, MA: Massachusetts Institute of Technology Press, 2006, ch. Empirical Statistics and Stochastic Models for Visual Signals, pp. 1–34. [4] M. Kendall and A. Stuart, The Advanced Theory of Statistics, 3rd ed. Griffin, London, 1969, vol. 1.

[5] E. Y. Lam and J. W. Goodman, “A mathematical analysis of the dct coefficient distributions for images,” IEEE Transactions on Image Processing, vol. 9, no. 10, pp. 11 661–1666, 2000. [6] J. Huang and D. Mumford, “Statistics of natural images and models,” in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, 1999. [7] J. Huang, “Statistics of natural images and models,” PhD thesis, Division of Applied Mathematics, Brown University, 2000. [8] D. Zoran and Y. Weiss, “Scale invariance and noise in natural images,” in Proceedings of the IEEE 12th International Conference on Computer Vision, 2009, pp. 2209–2216. [9] J. Daugman, “Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters,” Journal of the Optical Society of America A, vol. 2, no. 7, pp. 23–26, 2007. [10] H. Tian, B. Fowler, and A. Gamal, “Analysis of temporal noise in cmos photodiode active pixel sensor,” IEEE Journal of Solid-State Circuits, vol. 36, no. 1, pp. 92 –101, jan 2001. [11] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007.

1860