Page 1 of 2
The Journal of Engineering
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.
Scale adaptive region selection for deblurring Jinyang Li, Zhijing Liu, Xixi Jia and Xin Huang A scale adaptive region selection method for deblurring based on sparse representation and gradient priors is proposed. Using a pre-trained blurred dictionary, sparse representation can sparsely reconstruct the blur examples in an input blurred image. The initial location of the selected region is a patch which is reconstructed by the minimum number of blurred atoms by solving a minimization problem, making the best example to represent a smooth-like component. With the initial location, the gradient priors based on our observation is used by searching a series of extended discrete scales. The proposed method makes the region selection for deblurring highly effective and efficient by utilizing sparse representation and informative features. Experimental results on synthetic blurred images demonstrate that the proposed method behaves favorably in blur kernel estimation and deblurring quality against state-of-the-art approaches.
Introduction: Image blur sourced from camera motion during exposure is one of the most active fields in computer vision. In most cases, the significant degradation caused by blur is what we want to wipe out. Typically, image blur under a spatially-invariant model is usually formulated as a convolution of a clear latent image with a blur kernel. With the blur kernel known, a blur removal process is called non-blind deconvolution; otherwise, blind deconvolution. It is shown that a non-blind deconvolution blur removal process with the blur kernel well and accurately estimated leads to an excellent results. Hence, how to estimate kernel with only a single blurred image is crucial for the performance of deblurring. Recently, Hu et al. [1] proposed a method based on Conditional Random Field (CRF) that makes use of contextual constraints among image to select good region for kernel estimation. Hu et al. [1] also proposed a new kernel similarity metric that better compares estimated kernels with the ground truth. Based on the information of high variance and low saturation, Fergus et al. [2] designed a method searches and selects a subwindow to automatically locate regions for kernel estimation. In [3], a nonparametric pitch-based strategy using auxiliary variables was introduced for kernel estimation. However, all approaches use the fixed-size subwindows for region selection, which means that the selected regions may contain too many negative impacts. In other words, if good features such as strong edges and salient edges are not fully used, unfavorable results are likely to occur because of the negative impacts. Hence, a scale adaptive gradient-based method for good region selection which contains enough informative features is meaningful for deblurring. In this Letter, we proposed a method based on sparse representation [4] and gradient priors to efficiently and effectively handle the problem. Proposed approach: Typically, sparse representation [5] can be achieved as follows. Given all example patches Y = {yi }ki=1 ∈ Rd×k in a blurred image, each example (patch) yi can be approximatively represented by a sparse linear combination of dictionary atoms as: minkyi − Dαi k22 αi
s.t. kαi k0 ≤ m,
αi
b
Fig. 1: Statistics of horizontal gradients a histogram of horizontal gradients from an input blurred image b histogram of horizontal gradients from a selected region First, our proposed method is based on gradients searching, and thus, the statistics of horizontal gradients can be accomplished by searching a series of extended discrete scales that in following definition: Y(s) = {y(w0 , h0 )|w0 = w + 4w, h0 = h + 4h},
(3)
where Y(s) is the searching space, w and h are the width and height of the searching region in the current searching step, and 4w and 4h are constant step parameters. Secondly, in every searching step, we calculate and collect numerous nonzero horizontal gradients around the initial location yi∗ , the example corresponding to α∗i in Eq. (2). For every intermediate searching region y ∈ Y(s), we calculate horizontal gradients and account all nonzero horizontal gradients using a regularization term: max y(w, h)
s.t.
k5h y(w, h)k0 ≤ ε, k5h Y k0
(4)
where Y is an input blurred image; 5h is horizontal gradient operator; the L0 norm is used for accounting nonzero gradients and ε is a constant. The proposed scale adaptive region selection for deblurring (SARD)-based method is shown in Algorithm 1. Algorithm 1 scale adaptive region selection for deblurring input: An input blurred image Y , a pre-trained dictionary D output: a selected region y(w, h) in blurred image 1: Obtain sparse coefficient vector α∗i 2: Locate patch yi∗ which is corresponding to α∗i
3: y(w, h) ← yi∗ 4: while k5h y(w, h)k0 /k5h Y k0 ≤ ε do 5: y(w, h) ← y(w + 4w, h + 4h) 6: end while
(2)
where D is a blur dictionary off-line trained following Eq. (1) using 80,000 patches randomly chosen from 8,000 blurred images. The
ELECTRONICS LETTERS
a
(1)
where D ∈ Rd×l , k, l ∈ R and k < l is an over-complete dictionary; αi ∈ Rl is the coefficient vector used for reconstructing example yi . Here, we proposed a new method based on [4] and gradient priors to select good region in a single blurred image for deblurring. Inspired by [4], for an input blurred image Y and each (yi , αi ) pair, we search for the sparse coefficient vector by solving a minimization problem: α∗i = arg minkyi − Dαi k22 + kαi k0 ,
first term in the objective function ensures the difference between the recovered example Dαi and the original example yi to be small. The second term in the objective function leads to a sparse representation aiming at producing a small number of nonzero values in the coefficient vector αi . Efficient in suppressing ringing and noise artifacts, gradient priors is commonly introduced during the blur removal process. The proposed gradient-based method is based on the observation that the nonzero horizontal gradients in a region which contains enough strong edges usually account for a considerable proportion of the gradients involved in the whole blurred image. Fig. 1 shows the comparison between horizontal gradients in a whole blurred input image and those in a selected region which is full of informative features.
Experimental results: To test the performance of the proposed method named the SARD for region selection, we evaluated it on nine blurred images generated by convolving different blur kernels with different clear images together with some other region selection
12th December 2011 Vol. 00Only No. 00 IET Review Copy
The Journal of Engineering
Page 2 of 2
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.
methods, including UHP(user-handpicked method), APS [2] and GRD [1]. In the experiment, we set the patch example of size 10 × 10. The pre-trained dictionary D is a 100 × 200 matrix produced following [5] using 80,000 patches randomly chosen from 8,000 blurred images. The parameters 4w and 4h are set to be 10. In order to control the size of the selected subwindow for deblurring, the parameter ε is set to be relative small and relative large for a large input blurred image and a small input blurred image(e.g., ε = 0.11 and ε = 0.42 in our experiments), respectively.
a
b
c
d
truth kernels RMSE
SARD
GRD
APS
UHP
Parking lot
0.7811
0.6636
0.7059
0.7393
Seaside
0.8073
0.9022
1.0096
0.7553
Zebra
0.8240
1.3709
1.0338
0.9097
Arabesquitic
0.8969
0.9581
0.9634
1.3445
Architecture
0.8359
0.9433
0.9532
0.8209
Flyover
0.9095
0.9681
0.8876
0.8711
Old tower
0.5871
0.6480
0.5124
0.6038
Riverside
0.7783
0.9265
0.9089
0.7648
Tiger
1.0114
1.1187
0.9678
1.1320
Average
0.8257
0.9444
0.8825
0.8824
proposed. As is proved that a blurred example can be sparsely reconstructed by dictionary atoms, according to the minimization problem, we take the patch reconstructed by the minimum number of blurred atoms as the starting point. Based on the observation, we extend the searching subwindow to collect enough nonzero horizontal gradients. Experimental results on different region selection methods show that our proposed method achieves better performance in both blur removal quality and blur kernel evaluations.
Fig. 2: Experiment on ‘Seaside’ on different methods for deblurring using [6] a subwindow from UHP with estimated kernel and deblurred result b subwindow from APS with estimated kernel and deblurred result c subwindow from GRD with estimated kernel and deblurred result d subwindow from SARD with estimated kernel and deblurred result Fig.2 shows the visual quality of deblurring following [6] using our inferred regions and compared region selection methods. Besides visual quality of deblurred images, a metric for comparing the ground truth blur kernel with the estimated blur kernel is more convincing. Therefore, a shift-invariant and scale-invariant kernel similarity metric (KS) [1] and a common-used root-mean-squareerror (RMSE) metric are shown in Table 1 and Table 2, respectively.
Table 1: kernel similarity(KS) for comparing estimated kernels with the ground truth kernels KS SARD
Table 2: RMSE for comparing estimated kernels with the ground
GRD
APS
UHP
Parking lot
0.6469
0.6996
0.6509
0.5911
Seaside
0.8118
0.7535
0.4664
0.6017
Zebra
0.7242
0.5031
0.5887
0.7047
Arabesquitic
0.9064
0.9050
0.9153
0.6386
Architecture
0.6741
0.6495
0.6256
0.6473
Flyover
0.7211
0.7522
0.6042
0.6984
Old tower
0.8100
0.6999
0.8550
0.7687
Riverside
0.7368
0.7332
0.5993
0.6425
Tiger
0.7416
0.5981
0.5166
0.5429
Average
0.7526
0.6994
0.6469
0.6485
Acknowledgment: This work has been partially supported by
the National Natural Science Foundation of China(grants 61173091). Jinyang Li, Zhijing Liu and Xin Huang (School of Computer Science and Technology, Xidian University, Xi’an, People’s Republic of China) Xixi Jia (School of mathematics and statistics, Xidian University, Xi’an, People’s Republic of China) E-mail:
[email protected] References 1 Hu, Z., and Yang, M.H.: ‘Good Regions to Deblur’, Proc. ECCV, Firenze, Italy, October 2012, pp. 59-72 2 Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., and Freeman, W.T.: ‘Removing Camera Shake from a Single Photograph’, ACM Trans. Graphics, 2006, 25, (3), pp. 787-794 3 Sun, L., Cho, S., Wang, J., and Hays, J.: ‘Edge-based Blur Kernel Estimation Using Patch Priors’, Proc. ICCP, Cambridge, MA, April 2013, pp. 1-8 4 Shi, J., Xu, L., and Jia, J.: ‘Just Noticeable Defocus Blur Detection and Estimation’, Proc. CVPR, Boston, MA, June 2015, pp. 7-12 5 Aharon, M., Elad, M., and Bruckstein, A.: ‘K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation’, IEEE Trans. Signal Process., 2006, 54, (11), pp. 4311-4322 6 Shan, Q., Jia, J.Y., and Agarwala, A.: ‘High-quality Motion Deblurring from a Single Image’, ACM Trans. Graphics, 2008, 27, (3), pp. 1-10
From Fig. 2a, we can see that the deblurred result is still blurred. Although Fig. 2c and Fig. 2d seem to behave similarly, our proposed SARD has a higher KS and a lower RMSE, which means our estimated kernel is closer to the ground truth blur kernel. With a higher KS and a lower RSME, on average, Table 1 and Table 2 illustrate that our SARD performs better than other region selection methods on these input images. Conclusion: In this Letter, a region selection method for image deblurring based on sparse representation and gradient priors is
IET Review2 Copy Only
