Identification of Parameters and Restoration of Motion Blurred Images R. Lokhande
K. V. Arya
Department of C. S. E. Indian Institute of Technology Kanpur-208 016, India
Department of C. S. E. Indian Institute of Technology Kanpur-208 016, India
[email protected]
[email protected]
[email protected]
ABSTRACT
eters have been proposed in literature [1, 2, 3, 5, 9]. Gennery [6] has discussed the determination of point spread function (PSF) parameters in the spectral domain. Cannon [1] has identified the point spread function (PSF) parameters in the power cepstrum of the image by inspecting the negative peak. The sensitivity of Cannon’s method has been improved by Fabian and Malah [5], where the blurred image is preprocessed to remove the noise using Spectral Subtraction method [10] and transforming the enhanced spectral magnitude function in cepstral [4] domain. In [9], power spectrum is averaged for reducing noise along the lines parallel to the expected minima and then the resulting one-dimensional spectral function is examined to detect the minima. In most of the parameter identification methods it is assumed that the motion is along the horizontal direction, which may not be the case in practice. Li and Yoshida [9] have described the method to detect the blur direction by rotating the coordinate system by an angle φ (varying from 0 to 180◦ ) and then by computing its 1-D spectrum and inspecting the peaks and valleys in it. This paper proposes the algorithms to determine motion blur PSF parameters, i.e. blur direction and blur length, in frequency domain. The blur direction is identified using Hough transform to detect the orientation of line in the log magnitude spectrum of the blurred image. The blur length is found by rotating the binarized spectrum of the blurred image in the estimated direction then by collapsing the 2-D spectrum into 1-D spectrum and finally by taking the inverse Fourier transform and finding the first negative value. These parameters are then used to restore the images. A Weiner filter [7] based technique for restoration of images has been presented in this paper. Section 2 presents the image degradation model. In Section 3 the methods for identification of motion blur parameters have been presented. The restoration method is presented in Section 4. The experimental results are given in Section 5. Conclusion is given in Section 6.
This paper proposes a technique for restoring the motion blurred images. Restoration of blurred images is very important problem in tracking and identification of criminals, where image of a human face or number plate of a running vehicle taken in hit and run situation gets blurred due to relative motion between the imaging system and object/face. For restoration of motion blurred images, knowledge of the point spread function (PSF) is very important. The motion blur PSF is characterized by two parameters, namely blur direction and blur length. This paper also presents a method to identify the parameters of the PSF from the blurred and noisy images using the log spectrum of the blurred images. These parameters are used to restore the images. The experimental results demonstrate that the image can successfully be restored from the image with substantial amount of natural and artificial blur.
Keywords Motion blur, Blur Parameters, Image Restoration, Spectrum, Wiener Filter
1.
P. Gupta
Department of C. S. E. Indian Institute of Technology Kanpur-208 016, India
INTRODUCTION
Images captured in uncontrolled environments invariably represent a degraded version of an original image due to imperfections in the imaging and capturing process. This degradation may be classified into two major categories: blur and noise. Images may be blurred due to atmospheric turbulence, defocusing of the lens, aberration in the optical systems, relative motion between the imaging system and the original scene. It is often required to restore such images, particularly in identification of criminals. The restoration of blurred and noisy images depends on the blurring system model. The motion blur system is characterized by two parameters, namely, blur direction and blur length. Various methods for the identification of blur param-
2.
IMAGE DEGRADATION MODEL
In the model of image degradation [5], Figure 1, the observed image g(x, y) is modeled as the output of a 2-D linear system and can be characterized by its degradation function h(x, y). The noise n(x, y) is assumed to be a Gaussian white noise with zero mean. If the degradation function h(x, y) is linear and spaceinvariant function, then the observed blurred/noisy image in spatial domain is given by,
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’06 April 23-27, 2006, Dijon, France Copyright 2006 ACM 1-59593-108-2/06/0004 ...$5.00.
301
n(x,y)
f(x,y)
h(x,y)
+
This paper deals with images blurred by the uniform linear motion between the image and the sensor during image acquisition. When the scene to be recorded translates relative to the sensor at a constant velocity V with an angle of φ degrees along the horizontal axis during the exposure interval [0, T ], then the motion blur PSF h(x, y) for “blur length” L = V T [8, 12] is given by (3)
g(x,y)
h(x, y) =
Figure 1: Image Degradation Model
1 , L
if 0 ≤ |x| ≤ L cos φ; otherwise
0,
y = L sin φ
(3)
For simplicity, let us assume that motion is along the horizontal direction i.e. φ = 0◦ , then (3) can be expressed as, h(x, y) = Using sin x = (4) one gets (a)
(b)
1 2j
ˆ
1 , L
0,
if 0 ≤ |x| ≤ L; otherwise
y=0
(4)
˜ ejx − e−jx on the Fourier transform of
H(k)
= sinc(πkL).e−jπkL
(5)
The magnitude transfer function of (5) is given by |H(k)| = sinc(πkL)
(6)
Note that H(k) is independent of the vertical frequency coordinate l. Similarly, the expression for motion blur PSF of blur length L whose direction is φ◦ with the horizontal axis is obtained in [1] and given by, (c)
(d) H(k, l) = sinc(πLf ); where f = k cos φ + l sin φ
Figure 2: Spectrum of images: (a) Original Lena Image, (b) Spectrum of original Lena image, (c) Blurred Lena Image with blur direction 45 and blur length 20, and (d) Spectrum of Blurred Lena Image
3.
(7)
IDENTIFICATION OF BLUR PARAMETERS
This section describes the algorithms to determine the motion blur point spread function parameters. g(x, y)
3.1
= h(x, y) ⊗ f (x, y) + n(x, y) =
M −1 N −1 X X i=0 j=0
The identification of blur direction is based on the observation that the spectrum of non-blurred original image is isotropic, whereas that of motion blurred image is anisotropic. As illustrated in Figure 2 for a standard Lena image. It is evident from Figure 2 that spectrum of original image is identical in all direction, whereas that of blurred image is biased in the direction of blur. An image blurred due to motion is usually represented by a linear system of a convolution g(x, y) = f (x, y) ⊗ h(x, y), where h(x, y) is the convolution kernel that causes the motion blur. The motion blur kernel low-passes the image in the direction of the blur. One can see that anisotropy in the spectrum introduced by the motion blur is in the direction, perpendicular to the motion direction. Therefore, to determine the blur direction, the spectrum is treated as an image and the Hough transform is then used to detect the orientation of the line in the spectrum. The Find Line algorithm for finding the line in the spectrum is given in Algorithm 1.
h(x − i, y − j)f (x, y) + n(x, y) (1)
where ⊗ indicates the two-dimensional convolution and g, h, f and n are observed image, degradation function, original image and noise respectively. All of these images are of size M × N , where M and N represent the image size in x and y axis respectively. Since, convolution in the spatial domain is equal to multiplication in the frequency domain then in frequency domain (1) can be written as, G(k, l) = H(k, l)F (k, l) + N (k, l)
Blur Direction Identification
(2)
where (k, l) are the spatial frequency coordinates, and uppercase letters G, H, F and N represent the Fourier transforms of the observed image g, degradation function h, original image f and noise n respectively.
302
Algorithm 1. Find Line Step 1: Let φmin and φmax be the minimum and maximum values of φ. Step 2 : Initialize the accumulator array A(r, φ) to zero. Step 3: For each image point (xi , yi ) For φ = φmin to φmax { r = xi cosφ + yi sinφ A(r, φ) = A(r, φ) + 1 } Step 4: Find the maximum value (say A(rm , φm )) in the accumulator array A and get the corresponding line rm = x cosφm + y sinφm . end
Algorithm 3. Find Blur Length Step 1: Compute the Fourier transform G(k, l) of the blurred image g(x, y). Step 2: Calculate the log spectrum of G(k, l) and covert it to binary. Step 3: Rotate the binary spectrum in the direction opposite to the blur direction. Step 4: Collapse the 2-D data into 1-D data by taking the average along the columns. Step 5: Take the Inverse Fourier Transform of 1-D data obtained in step 4 and locate the first negative value in the real part, which corresponds to the blur length. end In case of noise, the same algorithm is followed with the exception that the blurred image is average filtered before computing its Fourier transform. It is assumed that zero mean white Gaussian noise and average filter effectively remove the Gaussian noise [7, 11, 13].
The Hough transform divides the parameter space into accumulator cells. For each point (x, y) in the image, the corresponding curve given by r = x cosφ + y sinφ is entered in the accumulator by incrementing the count in each cell along the curve. Thus, a given cell in the two-dimensional accumulator eventually records the total number of curves passing through it. After all image points have been treated, the accumulator is inspected to find cells having high counts. If the count in a given cell (r, φ) is N , then N image points lie along the line whose normal parameters are (r, φ). The Hough transform returns the accumulator array in which the maximum value corresponds to blur direction. To reduce the computation cost of the Hough transform, the spectrum is binarized. The steps to identify the blur direction are shown in algorithm 2.
4.
Fˆ (k, l)
Algorithm 2. Find Blur Direction Step 1: Compute the Fourier transform G(k, l) of the blurred image g(x, y). Step 2: Calculate the log spectrum of G(k, l) and covert it to binary. Step 3: Use Hough transform to get the accumulator array. Step 4: Find the maximum value in the accumulator array which corresponds to blur direction. end
3.2
=
"
# H ∗ (k, l)Sf (k, l) G(k, l) (9) Sf (k, l)|H(k, l)|2 + Sn (k, l)
where H(k, l) is the frequency response of PSF h(x, y), H ∗ (k, l) is the complex conjugate of H(k, l), |H(k, l)|2 = H ∗ (k, l)H(k, l) and Sf (k, l) and Sn (k, l) are the power spectrum of the original image and noise respectively. The restored image in the spatial domain is given by the inverse Fourier transform of the frequency domain estimate ˆ l)}. If the noise is zero, then the Fˆ (k, l) i.e fˆ = =−1 {F (k, noise power spectrum vanishes and the Wiener filter reduces to the inverse filter [8]. Equation (9) requires the knowledge of original image’s power spectrum, which is rarely known. Therefore, the equation (9) can be approximated by the following expression » – H ∗ (k, l) Fˆ (k, l) = G(k, l) (10) 2 |H(k, l)| + K
Blur Length Identification
If noise is neglected, the Fourier transform of observed image is equal to the multiplication of Fourier transform of PSF (h(x, y)) and original image. The frequency response of the motion blur PSF, which is given by (3) and whose magnitude is given by (6), is characterized by periodic zeros on the k axis which occur at 1 2 3 k = ± ,± ,± ,... L L L
IMAGE RESTORATION
This paper has used Wiener filter [7] based algorithm to restore the blurred image. It considers images and noise as random process, and aims to find an estimate fˆ of the ideal image f such that the mean square error between them is minimized. Its transfer function is given by [7],
where K is a constant and can be determined through experiments. The experiments reveal that a good estimate for the value of K is, K = 1/w, where w is the width of the image. The algorithm 4 gives the details of motion blurred image restoration.
(8)
5.
That is, zero crossings would occur periodically in H(k, l), or equivalently in G(k, l) along the lines perpendicular to the direction of motion. Therefore, the blur length can be obtained by observing the zero in G(k, l). The procedure to identify the blur length is given in algorithm 3.
EXPERIMENTAL RESULTS
The experiments are performed on the gray scale images with both natural and artificial motion blur. Artificial blurring is done using different values of blur lengths and directions.
303
Algorithm 4. Blur Image Restore Step 1: Identify the blur direction using algorithm Find Blur Direction. Step 2: Identify the blur length using algorithm Find Blur Length. Step 3: Form the PSF by using the blur direction and blur length and calculate its Fourier transform H(k, l). Step 4: Calculate the spectrum of Wiener filter as follows: # " H ∗ (k, l) W (k, l) = |H(k, l)|2 + K Step 5: Multiply the Fourier transform of blurred image G(k, l) by that of Weiner filter W (k, l) to get the Fourier transform Fˆ (k, l) of the original image. Step 6: Obtained the restored image by taking the inverse Fourier transform of Fˆ (k, l) i.e. f (x, y) = =−1 {Fˆ (k, l)}. end The experimental results with natural blurred images are shown in Figure 3 and Figure 4. Figure 3(a) shows the blurred image of a bus number plate and Figure 3(b) shows the corresponding restored image. The detected blur direction and blur length for bus image are 15◦ and 28 respectively. In this case the bus is going away from the camera. Similarly, Figure 4(a) shows natural blurred image of a calendar taken by digital camera. The image is blurred due to camera motion. For calendar image the detected blur direction and blur length are 0◦ and 47 respectively. Figure 4(b) shows the restored image. The exact blur direction and blur length are not known in both of these cases.
(a) Calendar image
Figure 4: Restoration result for the natural blurred image (calendar)
direction 18◦ and length 23 with additive white Gaussian noise of variance 195 and the watch image is blurred with direction 42◦ and length 20 with additive white Gaussian noise of variance 130. Table 1: Table showing detected blur direction and length for artificially blurred images Images
Book Cover
Watch
6. (a) Bus number plate image
(b) Restored image
Blur Length 15 15 23 38 60 15 20 30 35 50
Blur Direction (Degrees) 0 30 18 30 40 45 50 60 15 0
Detected Blur Length 16 15 22 35 53 15 13 27 31 44
Detected Blur Direction (Degrees) 0 29 18 31 39 45 45 58 14 0
CONCLUSION
This paper presented algorithms to determine the parameters of point spread function (PSF). The parameters are blur length and blur direction which are used to restore images from the motion blurred images. A Weiner filter based algorithm is to restore the images from motion blurred and noisy images has also been presented. The experimental results have shown that (i) the parameter of Weiner filter can be approximated to the reciprocal of image width, (ii) in artificially blurred images the identified blur length and blur direction is very close to theoretical value, and (iii) the proposed image restoration technique effectively restores the images from natural motion blurred, artificial motion blurred and noisy images.
(b) Restored image
Figure 3: Restoration result for the natural blurred image (bus number plate) A few artificially blurred images and corresponding restored images are shown in Figure 5. These images are blurred artificially with different blur lengths and directions as shown in Table 1. The 2nd and 3rd columns of Table 1 show the blur length and direction used for blurring the images, whereas 4th and 5th columns show the detected blur length and blur direction respectively. Table 2 shows the detected blur direction and blur length for above mentioned blurred images in the presence of noise. The 2nd , 3rd and 4th columns in Table 2 show the blur length, direction and amount of added noise respectively whereas the last two columns indicate the detected blur length and direction. Figure 6 shows the restoration results of blurred images in the presence of noise. The book cover image blurred with
7.
REFERENCES
[1] M. Cannon. Blind deconvolution of spatially invariant image blurs with phase. IEEE Trans. Acoust. Speech Signal Process., 24(1):56–63, 1976. [2] M. M. Chang, A. M. Tekalp, and A. T. Erdem. Blur identification using the bispectrum. IEEE Trans. Signal Processing, 39(10):2323–2325, 1991. [3] Y. S. Chen and I. S. Choa. An approach to estimating the motion parameters for a linear motion blurred
304
(a) Blurred Book cover image with blur length 38 and direction 30◦
(b) Restored book cover image
(c) Blurred Watch image with blur length 30 and direction 60◦
(d) Restored watch image
Figure 5: Restoration result for artificially blurred images
(a) Noisy blurred book cover image with blur length 23, direction 18◦ and variance 195
(b) Restored book cover image
(c) Noisy blurred watch image with blur length 20, direction 42◦ and variance 130
(d) Restored watch image
Figure 6: Restoration results for artificial noisy blurred images
Table 2: Table showing detected blur direction and length for artificial noisy blurred images Images
Book Cover
Watch
Blur Length 15 15 23 35 38 20 32 35 37 40
Blur Direction (Degree) 20 45 18 0 30 42 47 45 15 10
Noise Variance 65.025 195.075 195.075 130.05 130.05 130.05 65.025 65.025 130.05 130.05
Detected Blur Length 16 9 22 34 29 20 31 34 33 34
Detected Blur Direction 22 45 19 0 31 45 45 45 15 10
image. IEICE Trans. Inf. Syst., E83-D(7):1601–1603, 2000. [4] D. G. Childers. The cepstrum: A guide to processing. Proceedings Of The IEEE, 65(10):1428–1443, 1977. [5] R. Fabian and D. Malah. Robust identification of motion and out-of-focus blur parameters from blurred and noisy images. CVGIP: Graphical, Models and
305
Image Processing, 53:403–412, 1991. [6] D. B. Gennery. Determination of optical transfer function by inspection of frequency domain plot. J. Opt. Soc. Amer., 63(12):1571–1577, 1973. [7] R. C. Gonzalez and R. E. Woods. Digital Image Processing. Pearson Education, 2003. [8] R. L. Lagendijk and J. Biemond. Hand Book of Image and Vedio Processing, chapter Basic Methods for Image Restoration and Identification, pages 125–140. Academic Press, 2000. [9] Q. Li and Y. Yoshida. Parameter estimation and restoration for motion blurred images. IEICE Trans. Fundamentals, E80-A(8), 1997. [10] J. S. Lim. Image restoration by short space spectral subtraction. IEEE Trans. Acoust. Speech Signal Process., 28(2):191–197, 1980. [11] I. Pitas. Digital Image Processing Algorithms And Applications. John Wiley and Sons, Inc., 2000. [12] I. Rekleitis. Visual motion estimation based on motion blur interpretation. Master’s thesis, School of Computer Science, McGill University, 1995. [13] S. E. Umbaugh. Computer Vision and Image Processing: A Practical Approach Using CVIP tools. Prentice Hall, 1998.
