J Sci Comput DOI 10.1007/s10915-015-0067-7
Image Deblurring Via Total Variation Based Structured Sparse Model Selection Liyan Ma1,2 · Tieyong Zeng3,4
Received: 12 February 2015 / Revised: 21 May 2015 / Accepted: 26 June 2015 © Springer Science+Business Media New York 2015
Abstract In this paper, we study the image deblurring problem based on sparse representation over learned dictionary which leads to promising performance in image restoration in recent years. However, the commonly used overcomplete dictionary is not well structured. This shortcoming makes the approximation be unstable and demand much computational time. To overcome this, the structured sparse model selection (SSMS) over a family of learned orthogonal bases was proposed recently. In this paper, We further analyze the properties of SSMS and propose a model for deblurring under Gaussian noise. Numerical experimental results show that the proposed algorithm achieves competitive performance. As a generalization, we give a modified model for deblurring under salt-and-pepper noise. The resulting algorithm also has a good performance. Keywords method
Image deblurring · Sparse representation · Dictionary learning · Variational
The research is supported by the National Natural Science Foundation of China (No. 11271049, 61402462), RGC 211911, 12302714 and RFGs of HKBU.
B
Tieyong Zeng
[email protected] Liyan Ma
[email protected]
1
Institute of Microelectronics of Chinese Academy of Sciences, Beijing 100029, China
2
Jiangsu Research and Development Center for Internet of Things, Wuxi 214135, China
3
Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong
4
HKBU Institute of Research and Continuing Education, Shenzhen Virtual University Park, Shenzhen 518057, China
123
J Sci Comput
1 Introduction Image deblurring is an ill-posed problem, and is an unavoidable degradation in many image applications. The degraded model for such problem is g = Au + ξ, where u is the true image, g is the observed image. The known linear operator A represents a convolution with a low-pass filter and ξ denotes the Gaussian noise with zero mean and standard deviation σ . To solve the deblurring problem, one basic approach is to restrict the solution space via regularization term which leads to meaningful solutions. Among them, the TV regularization [37] is widely used since it has good edge-preserving property. However, TV regularization is non-differentiable, lots of works [2,9,22] devote to effectively get the solution of ROF model [37] which is a classical variational model for deblurring under Gaussian noise based on TV regularization. Although TV regularization can preserve edge information, it does not take full advantage of the similarities in images. Efros and Leung [15] pointed out that every small image patch has many similar image patches in the same image. Buades et al. [3] proposed an effective denoising method named non-local means which estimates a pixel value by weighted averaging the pixels with similar structure. Gilboa and Osher [20] defined the non-local TV regularization as an extension of the non-local means to variational methods. It is known that the multi-scale and sparse representation of images is a powerful tool to improve the performance of image restoration algorithms [7]. In the last few decades, deblurring methods based on sparse representation prior are widely studied, such as wavelet based methods [18,43], wavelet tight framelet based methods [7,28]. As a generalization of wavelet, wavelet tight framelet can provide better representation of images and has been applied to image deblurring under various types of noise [19]. The elements of wavelet-like system are fixed. Although there are many types of wavelets, such as Haar, Daubechies, Symlets, they can not adapt to the image of interest. Recently, learning a sparse representation from the image of interest has attracted much attention [1,6,16,17,36]. This approach makes the restoration algorithm achieve much better recovered quality than the TV regularization based and the wavetlet based approaches. The seminal work of sparse representation over learned dictionary is introduced by Olshausen and Field [36]. After that, Elad and Aharon [16] proposed KSVD denoising algorithm which achieves state-of-art performance. KSVD learns an over-complete dictionary to sparsely represent the images, but it does not resort other structure information and needs high computational cost [1]. To obtain more powerful sparse representation, lots of dictionary learning methods are proposed. Mairal et al. [33] proposed an algorithm to learn the multiscale sparse representation. Later, a nonlocal sparse model was introduced [32]. Chatterjee and Milanfar [10] locally learned a dictionary based on clustering. Yu et al. [41] proposed the structure sparse model selection (SSMS) method which only uses a best basis of a family of learned orthogonal bases to represent each image patch. The above methods based on dictionary learning are almost applied to image denoising problem. For deblurring problem, Lou et al. [30] proposed the direct sparse deblurring method via sparsely representing the blurred image in the blurred basis. Hu et al. [24] combined direct sparse deblurring method to solve the blind deconvolution problem. Liu et al. [29] proposed an efficient alternating direction method with dictionary updating. Yu et al. [41] extended SSMS to deblurring under Gaussian noise problem, and obtained better recovered results than the direct sparse deblurring method in most cases. However, since casting the deblurring problem
123
J Sci Comput
into a denoising problem via Wiener filter, the resulting SSMS deblurring algorithm does not perform very well in deblurring problem. In this paper, we investigate the properties of SSMS sparse representation, then propose a new model for deblurring under Gaussian noise. Experimental results show that the proposed algorithm achieves much higher recovered quality than SSMS deblurring algorithm and is competitive with the compared algorithms. The rest of the paper is organized as follows: In Sect. 2, we review the SSMS method, and give some analysis about SSMS. The proposed model is presented in Sect. 3, and the corresponding algorithm is introduced in Sect. 4. In Sect. 5, we give numerical experiments to demonstrate the performance of the proposed algorithm. Then, as a generalization, a modified model is proposed in Sect. 6 for deblurring under salt-and-pepper noise. Section 7 concludes the paper.
2 Structured Sparse Model Selection As mentioned before, it’s time-consuming to calculate the signal estimation when the sparse representation model lacks structure.To alleviate this effect, Yu et al. [41] proposed the structured sparse model selection (SSMS). The basic idea of SSMS is to approximate a signal with a linear approximation in an orthogonal basis which is selected from a family of learned orthogonal bases. With SSMS, calculating the signal estimation is faster than that in an overcomplete dictionary. In this section, we first give the review of SSMS. Then, the analysis of SSMS is presented.
2.1 Structured Sparse Model The dictionary used in SSMS is defined as a composition of K sub-dictionaries D = {Φk }1≤k≤K , and each sub-dictionary Φk = φk,l 1≤l≤L is an orthogonal basis. Given √ √ a discrete image patch of size N × N , converted to a column vector f ∈ R N , then f can be represented as a linear approximation as: f =
L
f, φk0 ,l φk0 ,l + εk0 ,L ,
l=1
where k0 = arg max
L f, φk ,l 2
k=1,...,K l=1
0
(1)
2 is the index of the best orthogonal basis and the approximation error εk0 ,L f 2 , the approximation dimensions L N . The difference of SSMS and other overcomplete dictionary models is obviously. For the denoising problem, the underlying clear image patch x can be estimated from the noisy patch y via f˜ = f, φk0 ,l φk0 ,l , l∈Λ
where
Λ = l : f, φk0 ,l > T ,
and the threshold value T depends on the noise variance.
123
J Sci Comput
For sparse representation problem, the construction of dictionary is very important. In general, learning dictionary from the image of interest leads to better recovered results. In SSMS, the initial dictionary is computed from synthetic black-and-white edge images ◦ ◦ of 18 orientations √ which √ are uniformly sampling from 0 to 180 . Calculating PCA over patches of size N × N collected from synthetic images with the same orientation gives one orthogonal basis. A DCT basis is also included in the dictionary since it can better approximate the complex texture. Thus, the initial dictionary has 19 orthogonal bases. Then, in the final dictionary learning stage, each patch of the image of interest is assigned a model k via model selection (1) with initial dictionary. Recalculating the PCA with each group, the final learned dictionary with 19 orthogonal bases is generated.
2.2 Analysis As we know, edges and gradients in images carry critically important visual information. Almost all of the image processing techniques require edge information or need to preserve the edges to ensure the visual quality of images. In image restoration field, the patch-based approaches lead to better restoration results since an image patch contains more structure information than an image pixel. Many works focus on how to describe the distribution of gradient information in order to enhance image quality [11,27,45]. Actually, in most small patches, the edges only have one orientation. For patches with many small textures, the DCT basis can be an option. This is why the SSMS works well in image modelling. The initial dictionary, which is used to partition the images patches to get the final learned dictionary, is very crucial to SSMS method. The bottom figure shown in Fig. 1 is the learned 19 orthogonal bases from Cameraman image degraded by the Gaussian noise with standard deviation σ = 20. The size of each patch is 8 × 8. It is obviously observed that the learned dictionary captures the different orientations and multiscale information of the edges. The learned dictionary via SSMS suggests the connection with Gabor filter [23,34] which is frequently used to extract multi-orientation and multi-scale information of images. A twodimensional Gabor filter is defined as:
x 2 + γ 2 y 2 G(x, y) = exp − 2σ 2
x exp j 2π + ϕ , λ
where x = x cos θ + y sin θ and y = −x sin θ + y cos θ . σ is the standard deviation of the Gaussian envelope. The spatial aspect ratio γ controls the ellipticity of the support of the Gabor filter. λ represents the wavelength of the sinusoidal factor. ϕ is the phase offset. We form a Gabor filter bank in 8 scales and 18 orientations (the MATLAB codes provided by Haghighat et al. [23]) to extract the variations in different scales and orientations of the images. Then we use this filter bank plus a DCT basis as an initial dictionary of SSMS to learn the final dictionary which adapts to the image of interest. The top figure in Fig. 1 is the learned orthogonal bases from Cameraman image degraded by the Gaussian noise with standard deviation 20. It is obviously observed that this learned dictionary also contains multi-scale information for different orientation of edges. However, in many orientations, the multi-scale information is not distinguish. With the two different initializations, the resulting PSNR [defined in (8)] values of the denoising algorithm on the noisy Cameraman image are 29.52 and 29.67 dB respectively. We can see that the initial dictionary learned from synthetic black-and-white edge images is better for the final recovered quality. This mainly because this initial dictionary captures the crucial information of the edges.
123
J Sci Comput
Fig. 1 The learned dictionary with different initial dictionaries (top Gabor filter; bottom learned from contour images). For each sub-figure, from left to right column: the first 8 atoms of each orthogonal basis learned from Cameraman image degraded by the Gaussian noise with standard deviation 20
In addition, the dictionary learned via SSMS is also related with tight frames. According to the definition of Sect. 2.1, the sub-dictionary satisfies ΦkT Φk = I N , where I N is the identity T matrix of size N × N . The whole dictionary can be rewritten as D = Φ1T , · · · , Φ KT , we have D T D = K I N . As the definition in [6,7], this means the dictionary learned via SSMS is a kind of discrete tight frame. The difference between them is that SSMS only selects a best sub-orthogonal basis rather than the whole dictionary when estimating each image patch.
3 The Proposed Model As mentioned before, SSMS demonstrates well behavior in signal estimation. Yu et al. also extended SSMS to image deblurring under Gaussian noise. Although SSMS deblurring algorithm is superior over the popular ForWaRD algorithm, it suffers the same limitation as ForWaRD since applying Wiener filter to remove the blur effect. Wiener filter doesn’t work well when the noise level is high. In the following, we propose a new model for deblurring problem based on SSMS in order to get better restoration results. Following the definitions introduced in Sect. 2, the natural image u ∈ Rm×n can be sparsely represented with the dictionary D by solving Rs u − Ds αs 22 + min μs αs 1 , αs
s∈P
s∈P
where P denotes the set √of indices √ where small image patches exist. The operator Rs first extracts a patch of size N × N at location s, and then converts it to a column vector. For each patch Rs u, a best orthogonal basis Φk0 of size N × N is selected by SSMS. We use Ds
123
J Sci Comput
to denote the best basis for the patch at location s, and the corresponding coefficient vector is αs . {μs }s∈P are hidden parameters. Since deblurring problem is ill-posed, the recovered image may contain some undesirable artifacts. This’s even worse when utilizing the patch based approach. In order to stabilize the solution, we use the known TV regularization which is pixel-based and can locally smooth the image. Thus, our model for deblurring under Gaussian noise is λ β Au − g22 + Rs u − Dαs 22 + min μs αs 1 + ∇u1 , (2) u,D,αs 2 2 s∈P
s∈P
T where λ and β are positive constants. ∇u1 is the TV regularization and ∇u = ∂x u, ∂ y u . The first term in (2) is the data-fidelity term which is commonly used for removing Gaussian noise.
4 Numerical Algorithm In this section, we present an effective algorithm for the proposed model (2). There are three unknowns: u, D, {αs }. We can employ an iterative scheme to alternatively solve the problem (2). (1) Dictionary learning stage: given u, we learn the dictionary D = diag {Φ1 , . . . , Φ K } via SSMS. (2) Given u and D, the sparse coefficient vector αs of each patch can be obtained by min αs
β Rs u − Ds αs 22 + μs αs 1 . 2
We also use SSMS to select a best orthogonal basis and compute the coefficient vector for each image patch. (3) Given D and {αs }s∈P , the minimization with respect to u is λ β Rs u − Ds αs 22 + ∇u1 . min Au − g22 + (3) u 2 2 s∈P
Denote W :=
s∈P
RsT Rs , M :=
RsT Dαs ,
s∈P
then W and M have same dimensions with u and g. Clearly, the problem (3) is equivalent to λ β Au − g22 + ( W ⊗ u, u − 2 M, u ) + ∇u1 , 2 2 where ·, · is the usual Euclidean inner product and ⊗ is the componentwise multiplication. For any matrix Λ, denote Λi, j as the value at position s = (i, j). Noting that for any (i, j) ∈ P , the weight Wi, j means how many times the pixel at (i, j) is used to construct √ √ image patches of size N × N , thus we have: min u
1 ≤ Wi, j ≤ N . Hence, W ⊗ u, u is strictly convex. Furthermore, the problem (3) is also strictly convex since β > 0.
123
J Sci Comput
Since TV regularization is non-differentiable, it is not easy to solve (3) in its primal formulation. Therefore, we use the primal-dual approach to efficiently solve this problem. Rewriting TV regularization in its dual formulation, we get λ β Rs u − Ds αs 22 − u, div p − δ P ( p), min max Au − g22 + (4) u p 2 2 s∈P
where div is the discrete divergence, δ P is the indicator function of the following set P := p ∈ Rm×n × Rm×n ; p∞ ≤ 1 , and p∞ is the maximum norm of p. Using the first order primal-dual algorithm proposed by Chambolle and Pock [9] to find the solution of (4), we get the following iterations: ⎧ 2 1 ⎪ pi+1 = min − ∇ u¯ i , p + δ P ( p) + 2θ p − pi 2 , ⎪ ⎪ p ⎨ 2 1 Rs u − Ds αs 22 − u, div pi+1 + 2θ u i+1 = min λ2 Au − g22 + β2 u − u i 2 , ⎪ u ⎪ s∈P ⎪ ⎩ i+1 u¯ = 2u i+1 − u i . where i = 0, 1, . . . is the iteration number, θ > 0 is a constant. The first two subproblems both have closed-form solutions. By simple calculation, we have pi + θ · ∇ u¯ i , pi+1 = (5) max 1, pi + θ · ∇ u¯ i and u i+1
T Rs Ds αs + div pi+1 u i + θ λAT g + β s∈P
. = T T 1 + θ λA A + β Rs Rs
(6)
s∈P
Algorithm 1 summarizes the details of the proposed algorithm.
In the following, we
Algorithm 1 Algorithm for image deblurring under Gaussian noise Initialization: u¯ 0 = g, u 0 = g, p 0 = 0, λ, β, θ , the dictionary D and coefficients {αs }s∈P . Output: u i+1 . for iter O = 1 : N O do for i = 0, 1, . . . , N I do Compute pi+1 using (5); Compute u i+1 using (6); u¯ i+1 = 2u i+1 − u i ; end for Update Dictionary D via SSMS with u i+1 ; Compute {αs }s∈P with u i+1 ; end for
discuss the convergence of the inner loop of Algorithm 1. Rewritten the problem (4), we get: min max F(u, p) + ∇u, p , u
where F(u, p) =
λ 2
Au − g22 +
p
β 2
s∈P
(7)
Rs u − Ds αs 22 − δ P ( p).
123
J Sci Comput
Theorem 1 Let B = ∇2 , and let u i , pi be generated by the inner loop of Algorithm 1. If we choose θ such that θ 2 B 2 < 1, then u i , pi converges to a saddle point (u ∗ , p ∗ ) of (7). One can easily prove this theorem following the proof of Theorem 1 in [9]. We remark that for the limit point (u ∗ , p ∗ ), the u ∗ will be the unique solution of (3) that we seek. And since ∇22 ≤ 8 (see [8]), then we have B 2 ≤ 8 for the inner loop of Algorithm 1. Note that in Algorithm 1, we stop the inner loop if a stopping criterion is reached. However, Rs u − Ds αs 22 will limit the recovered quality as the inner the existence of the term s∈P
loop runs. Therefore, we update the coefficients {αs }s∈P with u i before computing u i+1 in the inner loop. This will make the proposed algorithm converge more efficient.
5 Experiments To evaluate the performance, we run the proposed algorithm on eight test images as shown in Fig. 2. Except the Cameraman image size is 256 × 256, other images sizes are all 512 × 512. In our experiments, the test image is first blurred by a kernel, followed by the addition of Gaussian noise with zero mean. Three different types of blur kernels are considered: a 9 × 9 box blur; a 9 × 9 Gaussian blur with standard deviation 2; a motion blur of length 15 pixels and orientation 30◦ . We also consider three different standard deviation of Gaussian noise: 2, 5, 10. The peak signal to noise ratio (PSNR) is chosen as the quantitative measure of image quality which is defined as: PSNR = 20 log10
255 1 mn u˜
− u2
,
(8)
where u˜ is the recovered image and u is the true image.
Fig. 2 Original images. From left to right and top to bottom: Barbara, Boat, Cameraman, Couple, Goldhill, Lena, Man, Stream
123
J Sci Comput
The performance of the proposed algorithm is compared to five related existing image deblurring methods. The first one is TV based method which is solved via Bregmanized operator splitting [44]. The second one is the ForWaRD method [35] which has two stages: first removing blur by Wiener filter, then removing the Gaussian colored noise via wavelet based method. The third one is the SSMS deblurring [41] which also has two stages as ForWaRD. SSMS deblurring removes the Gaussian colored noise via SSMS denoising. The fourth method for comparison is the framelet based image deblurring approach [7] which uses the linear spline wavelet frame system to sparsely approximate the images. The last one is the non-local TV based method [44] which is solved via preconditioned Bregmanized operator splitting. To obtain the restoration results of the compared methods except the SSMS deblurring algorithm, we use the codes provided by the authors respectively. For the TV based method, if the kernel type is the box blur or Gaussian blur, the parameter μ is set to be 4, 15, 25 when the noise standard deviation is 2, 5, 10, respectively. If the kernel type is the motion blur, the parameter μ is set to be 8, 15, 40 when the noise standard deviation is 2, 5, 10, respectively. For ForWaRD algorithm, we use the default parameter setting provided by the authors in their codes. For SSMS deblurring algorithm, we follow the parameter settings recommended in [41], except the thresholds would be determined to obtain the best recovered results for different degraded images. u i+1 −u i
For the framelet based method, we use u i+1 2 < 10−4 as the stopping criterion. The 2 choice of framelet is fixed to be the 4-level piecewise linear framelet. The parameter λ = 0.05. If the kernel type is the box blur or Gaussian blur, the parameter μ is set to be 0.04, 0.12, 0.3 when the noise standard deviation is 2, 5, 10, respectively. If the kernel type is the motion blur, the parameter μ is set to be 0.04, 0.2, 0.45 when the noise standard deviation is 2, 5, 10, respectively. For the NLTV based method, if the kernel type is the box blur or Gaussian blur, the parameter μ is set to be 25, 50, 65 when the noise standard deviation is 2, 5, 10, respectively. If the kernel type is the motion blur, the parameter μ is set to be 40, 50, 70 when the noise standard deviation is 2, 5, 10, respectively. u i+1 −u i
For the proposed Algorithm 1, we use u i+1 2 < 10−3 as the stopping criterion of the 2 inner loop. We set N O = 50, N I = 50, β = 0.5, θ = 0.4, L = 8, T = 5. The patch size is set to be 5 × 5. For the three different noise levels, the parameter λ is set to be 350, 80, 30 for the degraded images of size 256 × 256 and be 300, 60, 20 for the degraded images of size 512 × 512, respectively. Tables 1, 2 and 3 illustrate the PSNR values of the six algorithms on all test images degraded by different blurs and different noise levels respectively. It can be observed that the proposed algorithm achieves higher PSNR values than the compared methods. On a PC equipped with 2.70 GHz CPU, the MATLAB and MEX implementation of the proposed algorithm takes about 131 seconds for an image with size 256 × 256, while the MATLAB and MEX implementation of NLTV [44] takes about 76 seconds. Meanwhile, the MATLAB implementation of other compared methods just takes few seconds. Some visual comparisons are presented in Figs. 3, 4 and 5. There are lots of artifacts in the deblurred images obtained by SSMS deblurring algorithm. This is because SSMS deblurring algorithm casts the deblurring problem to a denoising problem via Wiener filter which is not work well when the noise level is higher. NLTV does not work well sometimes, this is because NLTV is built on the weighted mean of the similar patches. If there do not exist
123
J Sci Comput Table 1 Comparison of the PSNR (dB) of the recovered results by different methods, with respect to the noise level σ = 2 Image Bar.
Boat.
Cam.
Cou.
Gold.
Lena
Man
Str.
Aver.
Kernel
TV
ForWaRD
SSMS
Framelet
NLTV
Ours
Box
23.31
24.14
24.60
24.45
23.76
24.60
Gaussian
23.43
24.01
24.07
24.31
23.57
24.41
Motion
24.05
25.01
26.63
25.93
25.64
27.25
Box
27.97
27.69
27.75
28.90
28.86
29.39
Gaussian
27.91
27.80
27.73
28.62
28.64
28.80
Motion
28.24
28.34
28.03
29.53
29.57
30.33
Box
25.44
25.66
25.69
26.43
26.29
27.05
Gaussian
24.83
25.06
25.04
25.27
24.92
25.62
Motion
26.62
26.89
26.74
28.11
28.38
29.16
Box
27.87
27.56
27.72
28.89
28.67
29.30
Gaussian
27.51
27.46
27.51
28.08
28.10
28.25
Motion
27.80
28.04
27.85
29.09
28.99
29.90
Box
27.78
28.37
28.25
29.77
28.36
30.06
Gaussian
28.02
28.52
28.34
29.50
28.25
29.95
Motion
28.07
28.85
28.40
30.17
28.92
30.64
Box
28.79
29.72
29.82
30.65
29.64
31.18
Gaussian
30.01
30.45
30.68
31.76
30.73
32.28
Motion
29.56
30.60
30.55
31.82
30.97
32.72
Box
26.74
26.42
26.28
27.55
27.26
27.88
Gaussian
27.30
26.73
26.48
27.88
27.58
28.13
Motion
27.53
27.36
26.95
28.73
28.46
29.45
Box
24.61
24.27
24.26
25.16
24.96
25.26
Gaussian
24.77
24.35
24.25
24.96
24.89
25.19
Motion
25.55
24.96
24.76
26.35
26.06
26.79
Box
26.56
26.73
26.80
27.73
27.23
28.09
Gaussian
26.72
26.80
26.76
27.55
27.09
27.83
Motion
27.18
27.51
27.49
28.72
28.37
29.53
Overall
26.82
27.01
27.02
28.00
27.56
28.48
Best performance values are given in bold
many similar patches, it will disturb the effect of NLTV. Since utilizing TV regularization and the structured sparse representation, the proposed algorithm can preserve the fine details and there are few artifacts in the recovered images. In order to illustrate the convergence behavior of the proposed algorithm, in Fig. 6, we present the successive relative error (SRE) of the iteration for the recovered image i+1 u − u i 2 SRE = , u i+1 2
which is common used as stopping criterion in scientific computing and the PSNR values of the intermediate results at each iteration when deblurring the image Cameraman corrupted by motion blur and Gaussian noise with standard deviation σ = 2. The two results show that though our minimization problem is highly non-convex, we can still observe the numerical
123
J Sci Comput Table 2 Comparison of the PSNR (dB) of the recovered results by different methods, with respect to the noise level σ = 5 Image Bar.
Boat.
Cam.
Cou.
Gold.
Lena
Man
Str.
Aver.
Kernel
TV
ForWaRD
SSMS
Framelet
NLTV
Ours
Box
22.73
23.57
23.78
23.79
23.12
23.92
Gaussian
23.11
23.87
23.87
24.02
23.32
24.11
Motion
23.13
23.74
24.53
24.22
23.72
24.48
Box
26.33
26.30
26.45
26.96
27.04
27.28
Gaussian
26.74
27.15
27.14
27.62
27.75
27.76
Motion
25.97
26.26
26.29
26.96
27.21
27.68
Box
24.00
23.83
23.80
24.67
24.73
25.26
Gaussian
24.17
24.28
24.26
24.62
24.64
24.88
Motion
24.28
24.15
24.16
25.15
25.32
26.21
Box
26.30
26.18
26.37
26.90
26.84
27.04
Gaussian
26.50
26.82
26.83
27.17
27.18
27.27
Motion
25.80
26.01
26.05
26.62
26.64
27.14
Box
26.32
27.39
27.43
28.06
26.88
28.32
Gaussian
26.98
28.04
28.11
28.61
27.59
29.03
Motion
26.29
27.29
27.24
28.14
27.04
28.68
Box
27.43
28.52
28.71
28.90
28.23
29.61
Gaussian
28.70
29.93
30.21
30.46
29.64
31.05
Motion
27.48
28.55
28.91
29.43
28.78
30.57
Box
25.41
25.38
25.48
26.01
25.93
26.33
Gaussian
26.37
26.35
26.39
27.00
26.94
27.21
Motion
25.62
25.55
25.47
26.43
26.37
27.05
Box
23.52
23.38
23.33
23.92
23.83
23.89
Gaussian
23.95
23.89
23.88
24.32
24.31
24.44
Motion
23.84
23.50
23.45
24.31
24.30
24.62
Box
25.26
25.57
25.67
26.15
25.83
26.46
Gaussian
25.82
26.29
26.34
26.73
26.42
26.97
Motion
25.30
25.63
25.76
26.41
26.17
27.05
Overall
25.46
25.83
25.92
26.43
26.14
26.83
Best performance values are given in bold
convergence of the proposed algorithm in the sense of SRE and l2 distance to the clean image [see (8)]. Based on SSMS and Gaussian mixture models, Yu et al. [42] recently proposed an algorithm (PLE) which achieved more promising results. Table 4 presents the comparisons. The results of PLE are taken from [42]. To get the results of ForWaRD method and the proposed algorithm, we take the average ISNR value of 10 randomized experiments.
6 Deblurring Under Salt-and-Pepper Noise Following the framework of Algorithm 1, we can solve many other image restoration problems with small modifications based on SSMS. In this section, we just use image deblurring
123
J Sci Comput Table 3 Comparison of the PSNR (dB) of the recovered results by different methods, with respect to the noise level σ = 10 Image
Kernel
TV
ForWaRD
SSMS
Framelet
NLTV
Ours
Bar.
Box
22.40
23.15
23.29
23.31
22.66
23.44
Gaussian
22.81
23.60
23.66
23.67
23.05
23.87
Boat.
Cam.
Cou.
Gold.
Lena
Man
Str.
Aver.
Motion
22.57
23.14
23.51
23.37
22.88
23.75
Box
25.06
25.15
25.26
25.72
25.71
25.90
Gaussian
25.97
26.36
26.30
26.68
26.76
26.92
Motion
24.67
24.94
25.07
25.29
25.50
25.94
Box
23.09
22.35
22.34
23.50
23.38
24.02
Gaussian
23.55
23.43
23.41
24.04
24.11
24.37
Motion
22.89
22.26
22.27
23.43
23.52
24.28
Box
25.06
24.99
25.13
25.62
25.46
25.55
Gaussian
25.76
26.01
25.97
26.31
26.32
26.46
Motion
24.48
24.66
24.79
24.99
25.01
25.48
Box
25.43
26.47
26.66
26.97
25.72
27.23
Gaussian
26.23
27.33
27.31
27.66
26.64
28.17
Motion
25.38
26.33
26.56
26.65
25.78
27.37
Box
26.47
27.46
27.71
27.82
26.92
28.45
Gaussian
27.82
28.96
29.05
29.17
28.43
29.92
Motion
26.18
27.21
27.62
27.53
27.10
28.64
Box
24.55
24.51
24.61
24.98
24.87
25.28
Gaussian
25.61
25.72
25.68
26.12
26.12
26.39
Motion
24.42
24.38
24.45
24.87
24.81
25.54
Box
22.74
22.63
22.66
23.03
22.98
22.99
Gaussian
23.37
23.38
23.21
23.71
23.75
23.79
Motion
22.77
22.61
22.73
23.08
23.09
23.38
Box
24.35
24.59
24.71
25.12
24.71
25.36
Gaussian
25.14
25.60
25.57
25.92
25.65
26.24
Motion
24.17
24.44
24.63
24.90
24.71
25.55
Overall
24.55
24.88
24.97
25.31
25.02
25.72
Best performance values are given in bold
under salt-and-pepper noise problem for an example. This problem is widely studied in recent years [5,13,21,31,38–40]. There are two main types of methods for image deblurring under salt-and-pepper noise. The first one is one-phase approach, and TV-L1 is the favorable model [13,21,38]. However, the one-phase methods do not consider the properties of salt-and-pepper noise. Thus, the one-phase methods change the values of the uncorrupted pixels in a wrong way. To tackle this problem, Cai et al. [4,5] proposed the two-phase methods which yields more promising results than the one-phase methods. In the first phase, the corrupted pixels set is identified via median-type filter. In the second phase, only the information of uncorrupted pixels is used in the data-fidelity term of the recovery model. Later, incorporating the sparse representation over learned dictionary, Ma et al. [31] proposed an two-phase method which achieves the state-of-the-art results.
123
J Sci Comput
(a) Original image
(b) Degraded: 23.31
(c) TV: 27.97
(d) ForWaRD: 27.69
(e) SSMS: 27.75
(f) Framelet: 28.90
(g) NLTV: 28.86
(h) Our: 29.39
Fig. 3 Recovered results [with PSNR (dB)] of different methods on image Boat corrupted by 9 × 9 box blur and Gaussian noise with standard deviation σ = 2
Suppose that the noise level is r (0 ≤ r ≤ 1), the image value corrupted by blur and salt-and-pepper noise at location o is then given by ⎧ ⎨ 0, go = 255, ⎩ (Au)o ,
with probability r/2, with probability r/2, with probability 1 − r.
The modified model for deblurring under salt-and-pepper noise is min λ Λ(Au − g)1 +
u,D,αs
β Rs u − Ds αs 22 + μs αs 1 + ∇u1 , 2 s∈P
(9)
s∈P
123
J Sci Comput
(a) Original image
(b) Degraded: 28.09
(c) TV: 30.01
(d) ForWaRD: 30.45
(e) SSMS: 30.68
(f) Framelet: 31.76
(g) NLTV: 30.73
(h) Our: 32.28
Fig. 4 Recovered results [with PSNR (dB)] of different methods on image Lena corrupted by 9 × 9 Gaussian blur with standard deviation σ = 2 and Gaussian noise with standard deviation σ = 2
where Λo =
0, 1,
if o ∈ N , otherwise,
with the salt-and-pepper noise locations set N which is detected via adaptive median filter (AMF) [26]. The first term in (9) is the data-fidelity term commonly used in two-phase method for deblurring under salt-and-pepper noise. This means only the noise free pixels are used in the data-fidelity term to recover the degraded image. Compared with the model (2), there is only one sub-problem that is different: the minimization with respect to u. To solve this
123
J Sci Comput
(a) Original image
(b) Degraded: 20.31
(c) TV: 26.89
(d) ForWaRD: 26.89
(e) SSMS: 26.74
(f) Framelet: 28.11
(g) NLTV: 26.89
(h) Our: 29.16
Fig. 5 Recovered results [with PSNR (dB)] of different methods on image Cameraman corrupted by motion blur of length 15 pixels and orientation 30◦ and Gaussian noise with standard deviation σ = 2
minimization problem, we introduce two auxiliary variables w and q: γ β η Rs u − Ds αs 22 + q1 + q − ∇u22 . Au − w22 + u,w,q 2 2 2 s∈P (10) Then, using alternating minimization method to solve the problem (10), we obtain min λ Λ(w − g)1 +
2 ⎧ i+1 q = min q1 + η2 q − ∇u i 2 , ⎪ ⎪ q ⎪ ⎨ 2 wi+1 = min λ Λ(w − g)1 + γ2 Au i − w 2 , w ⎪ 2 ⎪ ⎪ ⎩ u i+1 = min γ2 Au − wi+1 22 + β2 Rs u − Ds αs 22 + η2 q i+1 − ∇u 2 . u
s∈P
123
J Sci Comput 0.08
30
0.07
29
0.06
28
0.05
27
0.04
26
0.03
25
0.02
24
0.01
23
0
0
20
40
60
(a)
80 i+1
u
100 i
−u
2
120 i+1
/ u
140
160
22 0
20
40
60
2
80
100
(b)
PSNR
120
140
160
Fig. 6 Convergence behavior of Algorithm 1 on image Cameraman corrupted by motion blur of length 15 pixels and orientation 30◦ and Gaussian noise with standard deviation σ = 2 Table 4 Comparison of the ISNR (dB) of the recovered results by ForWaRD, PLE method [42] and our algorithm
Image
Kernel
Lena
Gaussian (σ = 1) Gaussian (σ = 2) Box (5 × 5)
2.42
House
Boat Best performance values are given in bold Images are blurred by a Gaussian kernel of standard deviation σ = 1, 2 or 5 × 5 box kernel, then corrupted by Gaussian noise with standard deviation 5
Aver.
ForWaRD
PLE [42]
Ours
1.26
3.77
3.89
1.84
3.52
4.00
3.07
4.24 4.22
Gaussian (σ = 1)
2.64
4.38
Gaussian (σ = 2)
3.36
3.90
4.66
Box (5 × 5)
4.10
4.26
5.91 2.95
Gaussian (σ = 1)
1.23
2.72
Gaussian (σ = 2)
1.95
2.48
3.19
Box (5 × 5)
2.69
2.62
4.23 3.69
Gaussian (σ = 1)
1.71
3.62
Gaussian (σ = 2)
2.38
3.31
3.95
Box (5 × 5)
3.07
3.32
4.79
The three sub-problems all have closed-form solutions. For x, τ ∈ R, we define shrink (x, τ ) := max (|x| − τ, 0)
x . |x|
We briefly summarize the details of the algorithm for deblurring under salt-and-pepper noise in Algorithm 2. Figure 7 gives few comparisons of the deblurring results obtained by the algorithm [5] and the Algorithm 2. It is observed that Algorithm 2 can achieve better restoration quality. In those experiments, we set L = 4 and the patch size 4 × 4 for Algorithm 2. That means the size of sub-orthogonal basis used for model selection is 16 × 4.
123
J Sci Comput
Algorithm 2 Algorithm for image deblurring under salt-and-pepper noise Initialization: u 0 = g, w 0 = 0, q 0 = 0, λ, β, η, the dictionary D and coefficients {αs }s∈P . Output: u i+1 . for iter O = 1 : N O do for i = 0, 1, . . . , N I do Compute q i+1 = shrink ∇u i , η1 ; For every entry of wi+1 at location o, computing wo =
(Au i )o , if o = 0. shrink Au i − g , γλ + go , if o = 1; o
Compute
γ AT wi+1 + β u i+1 =
s∈P
γ AT A + β
RsT Dαs + η∇ T q i+1 RsT Rs + η∇ T ∇
;
s∈P
end for Update Dictionary D via SSMS with u i+1 ; Compute {αs }s∈P with u i+1 ; end for2
(a) Degraded: 7.98
(b) Cai2010: 33.54
(c) Our: 34.63
(d) Degraded: 6.60
(e) Cai2010: 29.31
(f) Our: 29.84
Fig. 7 Recovered results [with PSNR (dB)] of different methods on image Cameraman corrupted by 7 × 7 box kernel and salt-and-pepper noise with noise level 50 % (first row) and 70 % (second row)
7 Conclusion This paper studied the image deblurring problem. Sparsely representing the image via structured model selection over a family of learned orthogonal bases, the proposed algorithm iteratively recovers images corrupted by blur and Gaussian noise. The experimental results show that the proposed algorithm is competitive to the compared methods. Also, we modified
123
J Sci Comput
the proposed model to solve the image deblurring under salt-and-pepper noise problem. The resulting algorithm can also obtain good recovered results. Although achieving good restoration quality, the proposed algorithm does not thoroughly explore the non-local properties of images. As the existing studies show that the non-local information of image is very useful [3,12]. In the future, we will consider the similarity among the image patches to get better sparse representation of them. This non-local approach will further improve the performance of the proposed method. The extension to the image restoration problem under non-Gaussian noises such as Poisson noise [19,38] and multiplicative noise [14,25] are also expected.
References 1. Aharon, M., Elad, M., Bruckstein, A.M.: The K-SVD: an algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006) 2. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009) 3. Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4, 490–530 (2005) 4. Cai, J., Chan, R., Nikolova, M.: Two-phase approach for deblurring images corrupted by impulse plus gaussian noise. Inverse Probl. Imaging 2, 187–204 (2008) 5. Cai, J., Chan, R., Nikolova, M.: Fast two-phase image deblurring under impulse noise. J. Math. Imaging Vis. 36, 46–53 (2010) 6. Cai, J., Huang, S., Ji, H., Shen, Z., Ye, G.: Data-driven tight frame construction and image denoising. Appl. Comput. Harmon. Anal. 37, 89–105 (2014) 7. Cai, J., Osher, S., Shen, Z.: Split Bregman mehods and frame based image restoration. Multiscale Model. Simul. 8, 337–369 (2010) 8. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imag. Vis. 20, 89–97 (2004) 9. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2004) 10. Chatterjee, P., Milanfar, P.: Clustering-based denoising with locally learned dictionaries. IEEE Trans. Image Process. 18, 1438–1451 (2009) 11. Cho, T.S., Joshi, N., Zitnick, C.L., Kang, S.B., Szeliski, R., Freeman, W.T.: A content-aware image prior. In: CVPR, pp. 169–176 (2010) 12. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3D transform-domain collaborative filtering. IEEE Trans. Image Process. 16, 2080–2095 (2007) 13. Dong, Y., Hintermüler, M., Neri, M.: An efficient primal-dual method for L1-TV image restoration. SIAM J. Imaging Sci. 2, 1168–1189 (2009) 14. Dong, Y., Zeng, T.: A convex variational model for restoring blurred images with multiplicative noise. SIAM J. Imaging Sci. 6, 1598–1625 (2013) 15. Effros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: Proceedings of the 7th IEEE International Conference on Computer Vision, vol. 2, pp. 1033–1038 (1999) 16. Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15, 3736–3745 (2006) 17. Elad, M., Figueiredo, M., Ma, Y.: On the role of sparse and redundant representations in image processing. Proc. IEEE 98, 972–982 (2010) 18. Figueiredo, M., Bioucas-Dias, J., Nowak, R.: Majorization–minimization algorithms for wavelet-based image restoration. IEEE Trans. Image Process. 15, 2980–2991 (2007) 19. Figueiredo, M., Bioucas-Dias, J.: Restoration of Poissonian images using alternating direction optimization. IEEE Trans. Image Process. 19, 3133–3145 (2010) 20. Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7, 1005–1028 (2008) 21. Guo, X., Li, F., Ng, M.: A fast l 1 -TV algorithm for image restoration. SIAM J. Sci. Comput. 31, 2322–2341 (2009) 22. Goldstein, T., Osher, S.: The split Bregman method for l1 regularized problems. SIAM J. Imaging Sci. 8048, 323–343 (2009)
123
J Sci Comput 23. Haghighat, M., Zonouz, S., Abdel-Mottaleb, M.: Identification using encrypted biometrics. Comput. Anal. Images Patterns 2, 440–448 (2013) 24. Hu, Z., Huang, J., Yang, M.: Single image deblurring with adaptive dictionary learning. In: Proceedings of IEEE ICIP, pp. 1169–1172 (2010) 25. Huang, Y., Ng, M., Wen, Y.: A new total variation method for multiplicative noise removal. SIAM J. Imaging Sci. 2, 20–40 (2009) 26. Hwang, H., Haddad, R.: Adaptive median filters: new algorithms and results. IEEE Trans. Image Process. 4, 499–502 (1995) 27. Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-Laplacian priors. In: NIPS, pp. 1033– 1041 (2009) 28. Li, Y., Shen, L., Dai, D., Suter, B.: Framelet algorithms for de-blurring images corrupted by impulse plus Gaussian noise. IEEE Trans. Image Process. 20, 123–139 (2011) 29. Liu, Q., Liang, D., Song, Y., Luo, J., Zhu, Y., Li, W.: Augmented Lagrangian-based sparse representation method with dictionary updating for image deblurring. SIAM J. Imaging Sci. 6, 1689C–1718 (2013) 30. Lou, Y., Bertozzi, A., Soatto, S.: Direct sparse deblurring. J. Math. Imaging Vis. 39, 1–12 (2011) 31. Ma, L., Yu, J., Zeng, T.: Sparse representation prior and total variation-based image deblurring under impulse noise. SIAM J. Imaging Sci. 6, 2258–2284 (2013) 32. Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: Proceedings of ICCV, pp. 2272–2279 (2009) 33. Mairal, J., Sapiro, G., Elad, M.: Learning multiscale sparse representations for image and video restoration. SIAM Multiscale Model. Simul. 7, 214–241 (2008) 34. Manjunath, B.S., Ma, W.Y.: Texture features for browsing and retrieval of image data. Pattern Anal. Mach. Intell. 18, 837–842 (1996) 35. Neelamani, R., Choi, H., Baraniuk, R.: ForWaRD: Fourier wavelet regularized deconvolution for illcondituined systems. IEEE Trans. Signal Process. 52, 418–433 (2004) 36. Olshausen, B., Field, D.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996) 37. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992) 38. Wu, C., Zhang, J., Tai, X.-C.: Augmented Lagrangian method for total variation restoration with nonquadratic fidelity. Inverse Probl. Imaging 5, 237–261 (2011) 39. Xiao, Y., Zeng, T., Yu, J., Ng, M.K.: Restoration of images corrupted by mixed Gaussian-impulse noise via l 1 − l 0 minimization. Pattern Recognit. 44, 1708–1720 (2011) 40. Yang, J., Zhang, Y., Yin, W.: An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J. Sci. Comput. 31, 2842–2865 (2009) 41. Yu, G., Sapiro, G., Mallat, S.: Image modeling and enhancement via structured sparse model selection. In: ICIP, pp. 1641–1644 (2010) 42. Yu, G., Sapiro, G., Mallat, S.: Solving inverse problems with piecewise linear estimators: from Gaussian mixture models to structured sparsity. IEEE Trans. Image Process. 21, 2481–2499 (2012) 43. Zeng, T., Li, X., Ng, M.: Alternating minimization method for total variation based wavelet shrinkage model. Commun. Comput. Phys. 8, 976–994 (2010) 44. Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3, 253–C276 (2010) 45. Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: ICCV, pp. 479–486 (2011)
123
