Signal dependent raw image denoising using ... - Semantic Scholar

Signal dependent raw image denoising using sensor noise characterization via multiple acquisitions Bosco A.a , Bruna R. A.a , Giacalone D.a , Battiato S.b , Rizzo R.b a STMicroelectronics

b Universit` a

- AST Imaging Lab, Stradale Primosole 50, Catania, Italy; degli Studi di Catania, Dipartimento di Matematica ed Informatica, Viale Doria 6, Catania, Italy - Image Processing Lab - http//iplab.dmi.unict.it ABSTRACT

Accurate noise level estimation is essential to assure good performance of noise reduction filters. Noise contaminating raw images is typically modeled as additive white and Gaussian distributed (AWGN); however raw images are affected by a mixture of noise sources that overlap according to a signal dependent noise model. Hence, the assumption of constant noise level through all the dynamic range represents a simplification that does not allow precise sensor noise characterization and filtering; consequently, local noise standard deviation depends on signal levels measured at each location of the CFA (Color Filter Array) image. This work proposes a method for determining the noise curves that map each CFA signal intensity to its corresponding noise level, without the need of a controlled test environment and specific test patterns. The process consists in analyzing sets of heterogeneous raw CFA images, allowing noise characterization of any image sensor. In addition we show how the estimated noise level curves can be exploited to filter a CFA image, using an adaptive signal dependent Gaussian filter. Keywords: noise estimation, sensor noise, CMOS, raw data, signal dependent noise, curve fitting

1. INTRODUCTION Effective noise filtering of raw image data requires accurate evaluation of some key parameters such as texture content and noise level; the latter is usually expressed in terms of standard deviation of the underlying noise distribution.1 Generally speaking, spatial adaptive filters are tuned such that the smoothing strength is attenuated when the processing mask is centered on a pixel belonging to textured areas, whereas on homogeneous areas the smoothing strength can be increased to reduce the unpleasant signal fluctuations to which the human eye is particularly sensitive. To include such peculiarities into a denoising filter, a texture detector is usually implemented to determine the signal complexity in the processing window, deciding whether and to what extent, the mask is centered on useful image data or random noise. These texture detectors must cope with the inherent difficulty of separating noise from real useful image signal. One useful and valid method widely adopted in the field of noise estimation consists in finding the homogeneous areas of an image where the noise level can be effectively estimated1, 2 ; the rationale behind this assumption is that in flat areas, variations of signal level are mainly caused by random noise and not by image content. By analyzing signal fluctuations in flat areas it is possible to obtain an estimate of the noise level in the image. However, it has been proven that one cannot simply look for homogeneous areas assuming that noise is constant across the whole image3 . As described in Section 2, an accurate noise model must take into account the noise dependency from signal. Sophisticated frameworks4 for estimating noise levels in RGB image exists. Other techniques5 use of wavelets to partition the image in segments of the same intensity in which noise is estimated. We have previously addressed the noise estimation problem by devising solutions for determining a unique noise level for a given image3, 6 . In this paper we propose a framework based on the analysis of many heterogeneous images for characterizing the signal dependent noise behaviour of an imager at all its operating conditions, without the need of tests in a controlled environment and specific test images, allowing characterization of sensor raw noise starting from available raw images.

1

Estimated curves are used to adapt the strength of a Gaussian filter according to the signal intensities. Experimental results show the effectiveness of the proposed signal dependent noise filter, allowing better performances compared to classical Gaussian filters using a unique signal-independent sigma value. The paper is structured as follows. Section 2 contains an overview of employed noise model for raw images. Section 3 illustrates the proposed noise estimation framework and describes in detail each step of our solution. Finally, Section 4 shows a noise reduction algorithm that uses estimated sigma values to filter CFA noisy images, and Section 5 reports experimental results.

2. RAW IMAGES NOISE MODEL Noise in raw images has basically two main sources: 1. NP : caused by the discrete nature of light itself (Poisson distributed) 2. NG : caused by the electric and thermal noise of the imager (Gaussian distributed) The NP term is related to the number of photons impinging on the imager which is not constant over time but follows a Poissonian distribution. Moreover, as the trend of squeezing more pixels into the same space continues, photodiodes become less light-sensitive, hence producing noisier images; this is especially true in consumer devices such as mobile phones. The other noise sources, such as read-out noise, thermal noise, amplifiers noise, quantization noise, etc. are usually modeled by considering that their overlap is Gaussian distributed. All noise sources overlap5 and finally generate a signal-dependent noise whose standard deviation can be modeled using an equation of the form: √ σ(i) = a · i + b (1) where: a, b ∈ 0, finally obtaining the interpolating curve C(l): C(l) = LLS(Φ(l)|Φ(l)>0 ),

l ∈ [0, . . . , 2bpp − 1]

(7)

using the square root approximating function (1) as fitting curve model.

Figure 1. Images can generate noise plots containing missing data for some signal levels. The missing points are obtained via interpolation.

3.4 Global Analysis of the Fitted Data The fitting process is repeated for every image of all the analog gains sets. Given a set of images at a specific analog gain Γ, each image γ ∈ Γ is processed finally generating a set of noise curves C Γγ (l) according to (7). Starting from the set of available noise curves C Γγ (l), we generate the unique noise characterization curve for the analog gain of the images in Γ. A non linear smoothing operator is applied along the y-axis to reject data outliers; a median operator represents a robust outlier rejecter for this application: C Γest (l) = F (median

¡© Γ ª ¢ C γ (l) γ=1,...,|Γ| )

(8)

where F (·) is a low pass smoothing operator applied to reduce small outliers. The final a and b coefficients for the images at analog gain Γ are obtained by LLS interpolation of the points in (8). As the number of images in Γ increases, the estimation errors decrease, especially at high analog gains. 4

4. SIGNAL DEPENDENT NOISE FILTER As noise reduction method that use the signal-dependent noise estimated, can be used a slightly modified version of the Sigma-Filter.6 The filtering process is based on the assumption that the observed pixel value I(x, y) is a good estimate of the local signal mean. The observed pixel value I (x, y) can be expressed as the sum of its representative mean η plus a Gaussian noise term ν: I (x, y) = η + ν (9) We then consider a value δ = 3σ and all the pixels in the range delimited by the central pixel value ±δ. Under the assumption of zero mean AWGN, this range will include ˜99% of the distribution from the same class as the central pixel. Let M be a m1 xm2 filter mask and Pc the value of its central pixel. The final output is a weighted average of the pixels having value close to one of the mask central pixel. Weights decrease as the distance in intensity between the central pixel and the neighborhood augments. Under the assumption of Gaussian noise model, the Sigma filter averages all the pixels whose value fall in the range [Pc − 3σ, Pc + 3σ]. In particular, pixels whose distance falls in the range [Pc − σ, Pc + σ] receive maximum weight wmax . Pixels whose value falls in the range [(P c − σ) − σ, (Pc + σ) + σ] are weighted with medium weight wmid . Finally, pixels whose intensity falls in the range[(P c − 2σ) − σ, (Pc + 2σ) + σ] are weighted with minimum weight wmin . Pixels outside of the range [(P c − 3σ), (Pc + 3σ)] are considered outliers having zero weight in the weighted average.

High Similarity with central pixel intensity Mid Similarity with central pixel intensity Low Similarity with central pixel intensity No Similarity with central pixel intensity

Weight

wmax

2σ

wmid

4σ

wlow

6σ

Pixel Value

0

Figure 2. Sigma Filter.

The Sigma Filter output Pf is computed as the sum of the mask pixels multiplied by their respective weights and divided by the sum of the weights: Pi