A result on a real image will be also pre- sented. ... moval, image enhancement and image restoration in .... The denoising part corresponds to (4) with g( ) 1.
Image coupling, restoration and enhancement via PDE's P. Kornprobst, R. Deriche
G. Aubert
INRIA, 2004 route des Lucioles, BP 93 F-06902 Sophia-Antipolis Cedex, France
Laboratoire J.A Dieudonné URA 168 du CNRS 06108 Nice-Cedex 2 (France)
Abstract
We present a new approach based on Partial Differential Equations (PDE) to restore noisy blurred images. After studying the methods to denoise images, staying as close as possible to the input image and methods to restore discontinuities, we will propose a new scheme which combines all this schemes. Quanti ed numerical test on a synthetic image will demonstate the e ciency of our scheme and the role of varying the parameters for denoising, enhancement and coupling. A result on a real image will be also presented.
1 Methods for denoising images.
A large number of PDE based algorithms have been proposed recently to tackle the problems of noise removal, image enhancement and image restoration in real images. These methods are based on evolving nonlinear partial dierential equations (PDE's) (e.g. Perona & Malik [9], Nordström [7], Shah, Osher & Rudin [10], Proesman et al., Cottet and Germain, Alvarez et al [1, 2], Weickert, Malladi & Sethian [6], Aubert et al. [3]...). This PDE based methodology allows us, for example, to prove existence and uniqueness of solution, propose e cient numerical schemes for the implementation part, nd a discontinuous solution,... In this context, the PDE's based methodology yields superior results compared to traditional techniques such as isotropic Gaussian ltering, Wiener ltering or median ltering and is much faster than functional optimizing regularization techniques [4, 11]. This image processing techniques are usually modeled by:
8 dI = (I; rI; rrI ) >< dt @I = 0 on @
>: @n I ;t = IN (
=0)
(1)
()
where r is the ordinary gradient operator, IN is the initial image (typically the noisy blurred image) and
@ is the boundary of the image noted . If we note = rI=jrI j and the normal vector to , we can
show that most of the classical diusion operators can be rewritten as [5, 4]:
(rI; rrI ) = c I + c I
(2)
where I (resp. I ) is the second directional derivative in the direction (resp. ) and c , c are functions of jrI j. For instance, in the case of the Perona and Malik model [9] and the diusion operator is div((jrI j)rI ) where is a decreasing function, we can show that:
PM (rI; rrI ) =
(j rI j)I + ((j rI j) + jrI j0 (j rI j)) I
(3)
In this case, it is clear that the two coe cients are functionally dependent, since they are both expressed with the same function (). Another interesting denoising model is the anisotropic diusion proposed by Alvarez, Lions and Morel [1]. Depending on the regions, isotropic smoothing or smoothing along direction is performed. The diusion operator can be rewritten in the form:
ALM (rI; rrI ) =
g(j G ? rI j)I + g(j G ? rI j) h(j rI j)I
(4)
where g and h are decreasing functions, the function G is a Gaussian kernel and ? is the convolution operator. g permits to stop smoothing till gradient is high and h permits to de ne where isotropic smoothing is
performed. In this method, we observe that the two degrees of freedom provided by the two functions g and h permit to set independently the coe cients c and c which was not possible in (3). As we observed in [5], this kind of method permits to handle with a large number of noise degradations. Notice that if g() 1, there is always smoothing, even in the neighborhood
of edges which does not cancel the enhancement of edges (see section 2). Coming from the optimization approach for image denoising ([5, 3, 7, 4],: : : ), it is also interesting to add the term (I ? IN ), where IN is the initial noisy image, which assert that we must remain close to the noisy image. Equation (1) becomes: dI dt
= (I ? IN ) + (rI; rrI )
(5)
Because of this coupling, the equations have a priori advantage of having a non trivial steady state, eliminating therefore the problem of choosing a stopping time.
2 Methods for the restoration of discontinuities
The Perona and Malik method (3), permits to denoise images but also 2to enhance edges. Choosing for example (x) = e(? xk ) where k is a constant shows that coe cient c can be negative, yielding locally an inverse heat equation in the direction which enhances edges. However, such a process is numerically unstable. Another solution exclusively devoted to deblurring was proposed by Osher and Rudin [8] who introduced the idea of shock lters based on hyperbolic equations theory. For 1D images (I (x; y) = I (x)) that is to say signals, if (rI; rrI ) = aIx where a is a constant, we know that the solution of (1) is I (x; t) = IN (x + at). The solution is translating with speed ?a. The idea of shock lters is to make the speed a depend on the image structure Ixx (see gure 1). 1D case
This PDE, discretized with suitable schemes, developpes discontinuities in a stable way across the scales. However, this method is very sensitive to noise because of the numerical approximation of the sign of I . It will be necessary, for noisy images, to introduce a Gaussian kernel to have a robust estimation of the speed direction.
3 A method for incorporating denoising, coupling and the restoration of discontinuities
When considering the problem of the restoration of noisy blurred images using the PDE based methodology, the main contribution found in the litterature is the one related to the recent work of Alvarez and Mazorra in [2]. It consist in choosing the following operator:
(I; rI; rrI ) = | r{zI} ? | e sign({zI )jrI}j Denoising
Enhancement
Results are good but several remarks may be formulated. First, the denoising part is a simple diusion operator along direction . This is done even in quasi homogeneous regions. Secondly, we must know when we stop the algorithm to obtain a good result. Finally the enhancement part may be noise sensitive (see section 2). Incorporating all that remarks and the mentionned requirements in previous sections 1 and 2 bounds us to come up with the following scheme modelled by:
(I; rI; rrI ) = f (I ? IN ) | {z } Coupling
+ | r (h (jG ? {z rI j)I + I}) Denoising
It =
? | e (1 ? h (jG ? rI{z j))sign(G ? I )jrI}j ~
?sign(Ixx)Ix
Figure 1: Enhancing discontinuities by means of shock lters Extending the shock lters theory to the 2D case by replacing the speed a by ?sign(I ) leads to the following equation: dI dt
= ?sign(I )jrI j
(6)
Enhancement
(7)
where h (x) = 1 if x < , 0 elsewhere. Data and results are presented in the gure 2 with a complete description of paramaters used on the array bellow. The original synthetic image is compouned of 4 geometric gures. When the original image is available, it is interesting to compute the classical i to noise h 2 (I2)signal ratio: SNR(I1 =I2 ) = 10 log10 2 (I1 ?I2 ) where is
the variance. We reported values of SNR(IO =IR ) and SNR(IO =IN ) where IR is the result. The enhancement part is generated by shock lters theory in which a Gaussian kernel convolution (?sign(G~ ? I )) supplies with an estimation of the speed direction (?sign(I )). Applying this operator to the image (a-1), that is setting ( f ; r ; e ) = (0; 0; 1), yields a particularly good result (a-2). The denoising part corresponds to (4) with g() 1 and setting in (7) ( f ; r ; e ) = (0; 1; 0). As we can observe on gure (b-2), using only the denoising operator is not su cient. Then, if we consider (7) with r and e non-zero, we remark that choosing f = 0 and = 0 (no smoothing along direction ) gives the scheme proposed in [2]. The advantage of taking different from zero appears clearly when comparing images (a-3) and (b-3). Homogeneous regions are better restored because of the isotropic smoothing. Another way to see this is to represent a cross-section of these images (see left image in gure 3). Finally, the in uence of the forcing term is presented in the middle image of gure 3, where we represent the evolution of SNR with CPU time1 for several values of f . It appears that this term has a stabilization action at the cost of a minor quality at convergence. If we compare images (a-4) and (b-4), with f = 0 features disappear with continued iterations, whereas with f = 1 they are maintained. Notice that the function h is also a good edge indicator. A result on a real image is shown in the last row of gure 2. The function h associated to the result is presented in the right image of gure 3.
References
[1] L. Alvarez, P-L. Lions, and J-M. Morel. Image selective smoothing and edge detection by nonlinear diusion (ii). SIAM Journal of numerical analysis, 29:845866, 1992. [2] Luis Alvarez and Luis Mazorra. Signal and image restoration using shock lters and anisotropic diusion. SIAM Journal of numerical analysis, 31(2):590605, April 94. [3] G. Aubert, M. Barlaud, L. Blanc-Feraud, and P. Charbonnier. Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Imag. Process., 5(12), December 1996. 1
Machine caracteristics: SPARC20 (66MHz, 64MB, So-
laris2.5)
[4] Rachid Deriche and Olivier Faugeras. Les EDP en Traitement des Images et Vision par Ordinateur. Traitement du Signal, 13(6), 1996. [5] Pierre Kornprobst, Rachid Deriche, and Gilles Aubert. Image restoration via PDE's. In First Annual Symposium on Enabling Technologies for Law Enforcement and Security - SPIE Conference 2942 : Investigative Image Processing., Boston, Massachusetts, USA., November 1996. [6] R. Malladi and J.A. Sethian. Image processing: Flows under min/max curvature and mean curvature. Graphical Models and Image Processing, 58(2):127141, March 1996. [7] N. Nordström. Biased anisotropic diusion - a uni ed regularization and diusion approach to edge detection. Image and Vision Computing, 8(11):318327, 1990. [8] Stanley Osher and Leonid I. Rudin. Featureoriented image enhancement using shock lters. SIAM Journal of Numerical Analysis, 27(4):919 940, August 90. [9] Pietro Perona and Jitendra Malik. Scale-space and edge detection using anisotropic diusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629639, July 1990. [10] L. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60:259268, 1992. [11] B. M. ter Haar Romeny, editor. Geometry-Driven Diusion in Computer Vision. Kluwer Academic Publishers, Dordrecht, 1994.
(a)
(b)
(c)
(1)
Datas Ref Description (a-1) Original+Gaussian blur ( = 2) (b-1) (a-1)+Gaussian noise ( = 20) (c-1) Processed image (c-2) (c-1)+Gaussian blur ( = 2) (c-3) (c-2)+Gaussian noise ( = 20) a b c
(2) SNR(IO =IR )
9.28 6.97
15.59 8.41
(3) Ref f r e (a-2)a 0 0 1 (b-2) 0 1 0 (a-3) 0 1.5 1 (b-3) 0 1.5 1 (a-4)b 0 2 1 (b-4)b 1 2 1 (c-4)c 0.25 2 1
(4) Results Iterations 0 50 10 20 0 13 14 13 14 50 14 50 4 15
SNR(IO =IN )
18.37 8.58 11.46 10.92 6.1 9.78 15.32
Result obtained on image (a-1) Image h is represented Result shown has been obtained at convergence
Figure 2: Rows (a) and (b) show results for synthetic image (size=100*100). Last row is a result with a real image (size 256*256). All the parameters of the scheme are precised in the table bellow. We took in all experiments = 1 ,~ = 2 and 4t = 0:1.
Figure 3: Left: Horizontal cross-section passing by the center of the square of images (b-3) and (c-3). Middle: SNR evolution with CPU time (seconds) for dierent values of f . Right: Function h corresponding to image (c-4) of gure 2
