BLIND IMAGE DECONVOLUTION USING ... - CiteSeerX

BLIND IMAGE DECONVOLUTION USING CONSTRAINED VARIANCE MAXIMIZATION Dalong Li∗ , Steven Simske∗∗ , Russell M. Mersereau∗ ∗

Georgia Institute of Technology Atlanta, GA 30332 {dalong,rmm}@ece.gatech.edu

∗∗

Hewlett-Packard Laboratories Fort Collins, CO 80528 [email protected]

ABSTRACT This paper describes an algorithm based on constrained variance maximization for the restoration of a blurred image. Blurring is a smoothing process by definition. Accordingly, the deblurring filter shall be able to perform as a high pass filter, which increases the variance. Therefore, we formulate a variance maximization object function for the deconvolution filter. Using Principal Component Analysis (PCA), we find the filter maximizing the object function. PCA is more than just a high pass filter; by maximizing the variances, it is able to perform the decorrelation, by which the original image is extracted from the mixture (the blurred image). Our approach was experimentally compared with the adaptive Lucy-Richardson maximum likelihood (ML) algorithm. The comparative results on both synthesized and real blurred images are included. 1. INTRODUCTION Due to the imperfections of a physical imaging system and the conditions under which an image is taken, such as atmospheric turbulence, a recorded image may be a noisy blurred version of the ideal image. In many situations, incomplete information about the blurring operator will be available. This problem is called blind image deconvolution. It is desired to recover the ideal image from the observed image before other image processing tasks can be performed, such as, detecting objects in the image. Many blind restoration algorithms have been proposed in the past, and some surveys can be found in [1]. The iterative blind deconvolution algorithm was first proposed by Ayers and Dainty in 1988 [2]. The use of simulated annealing in blind image deconvolution [3] was first proposed by B.C. McCallum. The double iteration algorithm developed by Holmes et al [4], was based on the EM algorithm. The EM algorithm is an efficient image restoration algorithm

and has been widely used in many applications under different names such as “Lucy” and “Richardson” (LR), etc. [5, 6]. The original Lucy-Richardson algorithm is a nonlinear algorithm derived from Bayesian considerations, and based on the knowledge of the point spread function (PSF). This algorithm ensures the positivity of the image and the conservation of the total energy.

In this paper, a very different blind image deconvolution algorithm is proposed. Blurring smoothes the image. Motion blur and atmospheric turbulence are examples. Such blurring operators can be effectively regarded as low pass filters. Thus, it makes sense to construct a deblurring operator in such a way that the variance of the reconstructed image is maximized so that the high frequency contents can be recovered. Principal Component Analysis allows us to find such a filter readily. The function of the PCA can also be interpreted as an operator of the decorrelation. In the paper, we review the deblurring problem and connect it to blind seperation problem for further insight about the algorithm.

The paper is organized as follows. In Section 2 we formulate the variance maximization based on a seemingly naive observation that the deblurring filter shall be a high pass filter, and solve the optimization problem with PCA. We then review the blind deblurring problem as a blind separation problem and interprets PCA’s role of decorrelation. In Section 3, the implementations of our approach and the ML algorithm on both synthetically generated blur image and real blurred images are reported. Conclusions and future work are discussed in Section 4.

2. DECONVOLUTION WITH CONSTRAINED VARIANCE MAXIMIZATION

Cµi = λi µi , i = 1, .., M where M is the length of the filter. Let w = µ1 , then

For simplicity, let us begin with the one-dimensional case. A commonly used degradation model is g =f ∗h+e

J(w)

(1)

where f is the original signal, h is the blurring filter and e is the noise. In our system, partial noise-removal is done as preprocessing. Let us denote the noise-removed signal as g¯. The deblurring model is fˆ = g¯ ∗ w

var(fˆ) 1

Since

(2)

(3)

fˆ = [¯ g (n + 1), g¯(n), g¯(n − 1)][w(1), w(2), w(3)]T = Gw (4) where g¯(n) is a column vector of length N , T denotes the vector transpose , g¯(n + 1) and g¯(n − 1)are shifted versions of g¯(n), and w(1), w(2), w(3) are the coefficients of the filter. Without losing generality, we can assume that the data has had its mean removed, thus = = =

1 (Gw)T Gw n−1 1 wT GT Gw n−1 1 wT Cw n−1

1 µT Cµ1 n−1 1 1 µT λ1 µ1 n−1 1 (7)

the constraint kwk = 1 ensures that the operation preserves the energy. We want to find the w which maximizes J(w). To solve this optimization problem, let us first represent the convolution in matrix-vector form. To simplify the notation, we assume the length of the filter(M ) is 3. Then equation 2 can be written as:

J(w) = var(Gw)

= =

where w is the deblurring filter to be determined. Blurs like averaging, Gaussian are effectively low-pass filters. Therefore the corresponding deblurring filters must boost high frequency so that the high frequency energy can be recovered. Variance can be used as a measurement of high-frequency energy. We want to find w such that the variance of fˆ is maximized. The object function J(w) for this optimization problem is J(w) = kwk =

(6)

(5)

The solution to this optimization problem is the first eigenvector of the covariance matrix C = GT G. To verify this, let us denote the vectors µ and scalars λ as the eigenvectors and eigenvalues, respectively, of the covariance matrix C 2

µT1 µ1 = kµ1 k2 = 1

(8)

1 Then J(w) = n−1 λ1 . Since λ1 is the maximum eigenvalue, we have shown that µ1 maximizes J(w). Then the estimation of the ideal signal is given by

fˆ = Gµ1 + Ψ PM

(9)

1 where Ψ = M ¯i . µ1 , the eigenvector with the maxi=1 g imum eigenvalue, is the deblurring filter. Gµ1 can be interpreted as the projection of G onto µ1 , and fˆ provides the estimate of the ideal signal. It is straightforward to apply the algorithm to an image. Image deblurring can be done in the 1-D fashion (row by row, then column by column) using the 1-D filter or a 2-D filter can be constructed following the same procedure. To derive the image deblurring filter, the image and 2-D deblurring filter are converted to vectors. This is the equivalent of the 1-D case. Since the real blur is spatially varying, the corresponding deblurring filter shall also be spatially adaptive. Therefore each row/column/block is processed independently, and the local variance is maximized this way. The fact that the solution to the optimization problem is the first eigenvector of PCA motivates us to review the deblurring problem. The matrix-vector form of the convolution clearly indicates that the blurring can be viewed as a linear mixing process. The original signal/image and its shifted versions are linearly mixed with each other. As a result, the blurred signal/image is the mixture. Thus the deblurring is to extract the original from the mixture. The deblurring is essentially a separation problem. PCA is able to perform such separation . The assumption then must be that the original image and its shift versions are uncorrelated with each other, that is, the pixels in the image are spatially uncorrelated. Such assumption is not always valid through out in the image. Though the neighborhood pixels are correlated, this is less likely true for the boundary of the objects in the image. Fortunately, blurring always damages the boundary most while affecting the smoother areas in the image less. As an extreme case, blurring can never change a uniform area since it is already smooth and can not be even smoother by any operation. PCA is able to enhance the edge, thus makes the blurred image looks sharper.

The presence of the noise reduces the performance of the de-blurring, De-noising can be done by Wiener filtering. The noise is estimated in the frequency domain rather than in the spatial domain.

The CPU is 500MHz Pentium III, with 256MB of RAM. It takes 1.6 seconds to deblur a 256 by 256 pixel image; while on average the ML algorithm takes 6.8 seconds (a factor of 4.3 longer) using 5 iterations.

3. EXPERIMENTAL RESULTS

4. CONCLUSIONS AND FUTURE WORK We have proposed a variance maximization based algorithm for the restoration of a blurred image. PCA solves the maximization problem. The algorithm obtains an estimate of the ideal image using the projection onto the eigenvector with the maximum eigenvalue. Some of the advantages of the proposed algorithm include the following:

Our algorithm was implemented and tested on a variety of both simulated and real blurred images. The restoration filter support M is the only parameter in our algorithm. The default value of M is set to 3. In the experiments, the LucyRichardson maximum likelihood algorithm (ML) is used to restore the image for comparison purposes. The ML algorithm needs an initial guess of the blur PSF. When the PSF is specified, its size and the values it contains must be estimated. The size of the initial PSF is more important to the ultimate success of the restoration than the values in the PSF. Because the initial guess at the values in the PSF is less important than the size, typically an array of 1’s is specified as the initial PSF. The performance of the ML is very sensitive to the iteration number. An improper iteration number would lead to over-convergence, there would be too many artifacts in the image. In the experiments, we had to manually choose a proper iteration number for ML. Our algorithm is robust to the selections of the parameters, so it is not necessary to adjust the parameters manually for each image. The proposed algorithm was used to restore several degraded images, here are two examples. In the first example, we simulate the atmospheric turbulence, Status of liberty image is divided into 16 × 16 blocks on which moving average filters are applied. The support of the average filters are spatially varying, thus to simulate the effect of the turbulence. The bigger of the support, the stronger the blurring. The blurred image is shown in Fig. 1. The size of the initial PSF for the ML algorithm is set to 3. The values of the initial PSF is the default value. The iteration number is selected as 10 since it gave best result. Fig .2. is the result of the ML. Our algorithm’s result is shown in Fig. 3, which has less artifacts than ML result, especially on the area of head and left hand. In our algorithm, the deblurring filter length is set to 3. In the second example, the image in Fig. 4 is blurred by real atmospheric turbulence. The turbulence is spatialvarying both in the size and in the values of the PSF. Since the true size of the blur is unknown, the size of the initial PSF is set to 3 when the ML algorithm is called. The iteration number is 8 for this example. The result is shown in Fig. 5. The result of the PCA restoration with the same parameters is shown in Fig. 6, which has more details such as the shadow in the holes on the surface. Our algorithm is non-iterative, and thus faster than ML. The programs are run using Matlab on a notebook computer.

1. The filter is computed from the blurred image, therefore, PCA can be applied locally so that different deblurring filters can be obtained for different regions on the image. This property makes the algorithm works well for spatially varying blurs. 2. It is non-iterative, thus the speed is high and parallel implementation is straightforward since rows/columns can be processed independently. The proposed algorithm has been tested experimentally on both simulated blurred images as well as real blurred images. The comparative results are reported. In some situations, more than one blurred image of the same scene/object are available. Future work includes extending our PCA restoration algorithm to multi-channel image restoration. 5. REFERENCES [1] D. Kundur and D. Hatzinakos, “Blind Image Deconvolution Revisited,” IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 61–63, Nov 1996. [2] G. R. Ayers and J. C. Dainty, “Iterative Blind Deconvolution Method and Its Applications,” Optics Letters, vol. 13, no. 7, pp. 547–549, July 1988. [3] B. C. McCallum, “Blind Deconvolution by Simulated Annealing,” Optics Communication, vol. 75, no. 2, pp. 101–105, Feb 1990. [4] T. J. Holmes, “Blind Deconvolution Quantum-Limited Incoherent Imagery: Maximum-Likelihood Approach,” J. Opt. Soc. Am., vol. 9, pp. 1052 – 1061, 1992. [5] W. H. Richardson, “Bayesian-based Iterative Method of Image Restoration,” J. Opt. Soc. Am, vol. 62, pp. 55–59, 1972. [6] A. K. Katsaggelos and K. T. Lay, “Maximum Likelihood Blur Identification and Image Restoration Using the EM Algorithm,” IEEE Transactions on Signal Processing, vol. 39, pp. 729–733, March 1991.

3

Fig. 1. S TATUS OF L IBERTY Image is locally blurred with moving averaging filter

Fig. 4. Real Atmospheric Turbulence Blurred Image

Fig. 5. Restored by ML, InitialP SF size = 3, Iter = 8

Fig. 2. Restored by ML, InitialP SF size = 3, Iter = 10

Fig. 6. Restored by PCA, M = 3

Fig. 3. Restored by PCA, M = 3 4