Adaptive Regularized Restoration Algorihtms ... - Semantic Scholar

The Restoration of HST Images and Spectra II Space Telescope Science Institute, 1994 R. J. Hanisch and R. L. White, eds.

Adaptive Regularized Restoration Algorithms Applied to HST Images Aggelos K. Katsaggelos, Moon Gi Kang, and Mark R. Banham Dept. of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL 60201-3118 Abstract. This paper analyzes the performance of two set theoretic-based iterative image restoration algorithms for Hubble Space Telescope (HST) degraded images. The iterative adaptive constrained least squares and frequency adaptive constrained least squares algorithms are optimized here for HST data, and applied to several simulated and real degraded HST images. Evaluations of both the flux linearity and resolution enhancement of these algorithms are presented and compared to results obtained by the Richardson-Lucy algorithm (Lucy 1974). These results indicate that the iterative algorithms investigated here are quite suitable for HST data, and provide excellent results in terms of all evaluation criteria tested.

1. Introduction There have been a variety of techniques applied to the restoration of HST images over the last several years (White 1991, Weir 1991). Most of these approaches have concentrated on algorithms already developed and well accepted by those in the astronomical community. At the same time, however, the signal and image processing community was developing a number of digital image restoration techniques which were motivated largely by non-astronomical problems (Andrews and Hunt 1977). For a recent review and classification of image restoration algorithms, see Katsaggelos (1991). For the most part, the algorithms introduced by image processing scientists tend to be very general and applicable to a variety of imaging problems. In examining the HST degraded images, it is apparent that many of the image restoration techniques, and in particular the iterative techniques, developed in the realm of signal and image processing would be well suited for application to this special problem. In this paper two iterative image restoration techniques are applied to the HST data restoration problem. These are the generalized iterative adaptive algorithm (Kang and Katsaggelos 1993) and the frequency domain iterative adaptive algorithm (Kang and Katsaggelos 1992). Both are based on a set-theoretic approach to image restoration. The critical factor in choosing these algorithms is that, unlike many regularized restoration techniques, these algorithms need no prior information about the signal and no knowledge of the noise variances present. These are also relatively fast algorithms, and have been developed with convergence speed as a consideration. The generalized iterative adaptive algorithm allows for spatial adaptivity and is applicable to the case of spatially variant degradation. The performance of these algorithms is very good with the HST data that we have tested. The criteria which are most important to the astronomical community have been utilized in compiling the results here. It can be seen from these results that there is much potential for use of these iterative constrained least squares algorithms from the viewpoint of not only quality of results, but also convergence speed. In the next section, we describe the fundamentals of set theoretic image restoration, and describe the iterative techniques that we have developed through this approach.

3

Katsaggelos, Kang, & Banham

4

2. Iterative Image Restoration For the algorithms discussed here, the restoration approaches have been formulated to handle the problem of an image degraded by a spatially invariant or spatially variant blur operator with additive Gaussian noise. This description represents the degradation present in the HST WFPC images. The image degradation process can be modeled according to

y = Dx + n ;

(1 )

where a lexicographical ordering of the observed digital image, y , the original image, x, and the additive noise, n, is used. D represents the degradation operator of the imaging system. The image restoration problem calls for obtaining an estimate of x given y , D, and some characterization of the noise process, n. Iterative approaches may be used to solve this inverse problem very effectively. The primary advantages of iterative techniques are (Schafer et al. 1981, Katsaggelos 1989): (i) there is no need to explicitly implement the inverse of an operator; (ii) knowledge about the solution may be directly incorporated into the restoration process; (iii) the process may be monitored as it progresses; (iv) the effect of noise may be controlled with certain constraints; and (v) parameters determining the solution can be updated as the iteration progresses. The iterative techniques applied here to HST data are developed through a set theoretic approach. In this approach, prior constraints on the solution are imposed by (Katsaggelos et al. 1985, Katsaggelos 1989, Katsaggelos et al. 1991)

and

Qx = fx j kCxk2 E 2g ;

(2)

Qxjy = fx j ky , Dxk2 2 g ;

(3)

where the solution belongs to both ellipsoids described by Eqs. (2) and (3). In Eq. (2), the operator C represents a highpass filter which bounds the high frequency energy of the restored image. If the bounds 2 and E 2 are known, and the intersection of Qx and Qxjy is not empty, the solution may be found by solving (DT D + C T C )x = DT y ; (4 )

where , the regularization parameter, is equal to fidelity to the data, and smoothness of the solution.

=E )2, which controls the trade-off between

(

2.1. Generalized Iteration Adaptive Algorithm The first algorithm we tested for HST data was the generalized iterative adaptive algorithm which simultaneously restores the image and determines a single regularization parameter based on the restored image at each iteration. This algorithm does not depend on the initial conditions. In other words, the smoothing functional to be minimized can be shown to have a unique minimizer. An accurate estimate of the noise variance is also obtained at convergence. The approach we have proposed (Kang and Katsaggelos 1993) extracts the properties of the original image from the partially restored image at each iteration step. In order to solve for an optimal restored image and regularization parameter at each iteration, we use M ((x); x) = ky , Dxk2A(x) + (x)kCxk2B(x) ; (5) as the functional to be minimized. The weighting matrices A(x) and B (x) are included to make the restoration algorithm spatially adaptive. The necessary condition for a minimum is that the gradient of M ((x); x) with respect to x be equal to zero. This results in

DT (A(x) + A(x)T )D

[

+ =

w (x)C T (B (x) + B (x)T )C ]x + kCxk2B(x)rxw (x) DT (A(x) + A(x)T )y :

(6)

Adaptive Regularized Restoration Algorithms

5

When the noise and the high pass filtered image are stationary, the weighting matrices A(x) and B (x) become symmetric, and thus the equation can be rewritten as

DT A(x)D + w (x)C T B (x)C ]x + kCxk2B(x)rx w (x) = DT A(x)y ;

[

(7 )

and it becomes

DT A(x)D + w (x)C T B (x)C ]x = DT A(x)y : (8 ) since rx w (x) = 0 with a proper choice of w (x). When the noise and the high pass filtered image are uncorrelated with themselves, even though they are nonstationary, A(x) and B (x) become [

diagonal, and therefore we obtain the solution for Eq. (7). When the noise and the high pass filtered image are white, then the weighting matrices become constant identity matrices multiplied by the constant variances. The highly nonlinear term kCxk2B (x)rx w (x) can be removed by the proper choice of the regularization functional, based on the global convexity of the smoothing functional. Since Eq. (8) is nonlinear, we can not solve for x in a direct way, but we can use an iterative technique. The regularization parameter is defined here as a function of the original image (but in practice becomes a function of an estimate of the original image). The form of the smoothing functional to be minimized is of great importance since it preserves convexity and exhibits only a global minimizer. After investigating the desirable properties for the regularization functional to satisfy, the following two forms have been shown to provide optimal solutions (Kang and Katsaggelos 1993)

ky , Dxk2 (x) = (1= ) , kCxAk(x2) 1

where 1

=

1=(2ky k2), and

(x) = 2kCxk2B (x) , 1 +

q

(1

B (x)

;

(9 )

, 2 kCxk2B x )2 + 2 2ky , Dxk2A x ; ( )

( )

(10)

where 2 = 3=(4ky k2) controls convergence and convexity. Given an optimal choice for (x), we can solve Eq. (8) by the method of successive approximations with (Schafer et al. 1981, Katsaggelos et al. 1985, Katsaggelos 1989, Katsaggelos et al. 1991): xk+1 = xk + DT A(x)y , (DT A(x)D + (xk )C T B(x)C )xk : (11)

h

i

Using either choice of regularization functional (Eq. (9) or Eq. (10)), the iterative algorithm does not depend on the initial condition despite the nonlinearity of the iteration. This is due to the convexity of the functional and the convergence criteria satisfied by the globally optimal iteration. The algorithm is applicable to any type of degradation D and stabilizing matrix C (both of which may be spatially varying). So, there is no requirement that D and C be block circulant matrices. It is also important to note that no knowledge of the noise variance, or of the bound which determines the ellipsoid that expresses the smoothness of the image, is assumed. 2.2. Frequency Domain Iterative Adaptive Algorithm The second iterative algorithm examined here is a nonlinear frequency domain algorithm in which the regularization parameter is frequency dependent, and updated at each iteration step. In this case, the algorithm considers that D is a block circulant matrix representing a spatially invariant blur. Also, the set-theoretic formulation is constructed in a weighted space, such that

and

Qx = fx j kCxk2R ER2 g ;

(12)

Qxjy = fx j ky , Dxk2P 2P g ;

(13)


6

where P and R are both block circulant weighting matrices. These matrices are chosen to maximize the speed of convergence at every frequency component, and to compensate for the near-singular frequency components of the iteration. The solution which belongs to the intersection of the ellipsoids given by Eqs. (12) and (13) is given by

DT P T PD + C T RT RC )x = DT P T Py : (14) T T We define P P = B , R = PQ and Q Q = A. The block circulant matrix B is the spatial domain representation of the relaxation parameter which will be called (l) in the frequency domain, and A (

is the block circulant spatial domain representation of the regularization parameter which is shown next in the frequency domain as k (l). Again, successive approximation may be used to solve Eq. (14). Since all of the matrices in this equation are block-circulant, the iteration may be written in the discrete frequency domain as

X0(l) Xk+1 (l)

= =

(l)D(l)Y (lh) ; i Xk (l) + (l) D(l)Y (l) , (jD(l)j2 + k (l)jC (l)j2)Xk (l) ;

(15) (16)

where l represents a single 2-D frequency component. In this case,

Pm jY (m) , D(m)Xk(m)j k (l) = P jC (n)X (n)j + (l) k k n

2

2

:

(17)

Here, the k (l) term we use is defined by where 0 <

kused (l) = kconv (l) + (kopt , kconv (l)) ;

< 1, and X jC (m)X (m)j2 ; 2 2 kopt = N 2 max (jXk (l)j jC (l)j ) , k l m

(18)

(19)

and

,2jD(l)j2 Pm jC (m)Xk (m)j2 , Pn jY (n) , D(n)Xk (n)j2jC (l)j2 2jD(l)j2 q 2P jC (l)j jC (l)j ( n jY (n) , D(n)Xk(n)j2)2 + 8jD(l)j2jC (l)Xk (l)j2 Pn jY (n) , D(n)Xk(n)j2 + : 2jD(l)j2 (20) The optimized parameter is given at each frequency component by (Strand 1974) (l) = F (jD(l)j2) 2 4 6 8 = 31:5 , 315jD(l)j + 1443:75jD(l)j , 3465jD(l)j + 4504:5jD(l)j ,3003jD(l)j10 + 804:375jD(l)j12 : (21) According to this iteration, since and k are frequency dependent the convergence of the iteration kconv (l) =

can be accelerated, making this an attractive algorithm where speed is a concern. 2.3. Optimization for HST Data

In the course of testing our algorithms, we have made appropriate consideration of the optimization of these algorithms for HST data. In particular, we have included a positivity constraint at each step of the iteration. This imposes the condition that there should be no negative flux in our image source. The algorithms were developed with additive Gaussian noise as the observation noise in the model. So, these algorithms are well equipped to deal with the read-out noise problem of the WFPC.


Figure 1.

7

Blurred image “sim1.fit” and Richardson-Lucy restoration of “sim1.fit”.

Figure 2. Generalized iterative adaptive restoration of “sim1.fit” and frequency adaptive restoration of “sim1.fit”. 3. Results All of the results presented here use the test data prepared by ST ScI and obtained from the directory software/stsdas/restore at the stsci.edu Internet site. Using the image “sim1.fit”, which represents a star cluster with a globular cluster-like luminosity function, with the point spread function (PSF) “mpsf12.fit”, we generated restored images using the two iterative algorithms presented in this paper. The image “sim1.fit” simulates a monochromatic observation from the Wide Field Camera with a space invariant blur. For comparison purposes, we utilized an implementation of the modified Richardson-Lucy algorithm developed at ST ScI. This algorithm has modifications which address the constant readout noise problem. Fig. 1 shows the original blurred image and the Richardson-Lucy restoration. Fig. 2 shows the restored result using the generalized iterative adaptive algorithm and the frequency adaptive algorithm. For all cases where we have used the generalized iterative adaptive algorithm in this paper, we have taken the weighting matrices in Eq. (11) to be equal to identity, although a spatially adaptive approach may easily be employed as well. For all of the iterative algorithms, we set the maximum number of iterations to be 300. In addition, each algorithm used another termination criterion. For the Richardson-Lucy algorithm, the convergence criteria was that the value of 2 between the data and the blurred model was equal

8


to unity per degree of freedom (Lucy 1974). For the other two algorithms, a convergence criteria based on the L2 norm of the residual was used. In this case, we used a measure of the normalized error at each iteration, defined as kxk , xk,1 k2 =kxk,1k2 < 10,6. The Richardson-Lucy algorithm reached 300 iterations before the parameter 2 was equal to 1, but the value of 2 was changing very slowly at this point, such that further iterations resulted in little change in the restored image. The generalized iterative adaptive method took 292 iterations while the frequency adaptive method required only 39 iterations. The form of the iterations are somewhat different for the RichardsonLucy algorithm and the two algorithms presented here, so a direct comparison in terms of iteration counts is not completely straightforward. However, each iteration of the three algorithms requires approximately the same amount of time, so it can be seen that the frequency adaptive algorithm provides a very fast solution. The mean square error (MSE) was measured for each of these results after normalizing the restored images to the maximum value of the observed image in order to account for any linear scaling present in the different algorithms. The Richardson-Lucy algorithm had an MSE of 1083.69. The generalized iterative adaptive algorithm had an MSE of 1177.41, and the frequency adaptive algorithm had an MSE of 26.27. The MSEs were measured in terms of the 470 stars in the truth list. For images having a spatially varying PSF, it is possible to apply the generalized iterative adaptive algorithm as well. We have applied the spatially varying implementation of this algorithm to the synthetic image “sim3.fit” which represents a star field with a spatially varying blur. For this image the PSF changed between 25 different PSFs at various positions in the image. The varying PSF is easily represented by a full degradation matrix in Eq. (1)., as opposed to a block circulant. For the restored image, the MSE was equal to 3316.3. These simulated star field images provide the best data for testing the flux linearity of the restoration algorithms. We have evaluated the flux linearity of these three results according to the following method. The linearity was measured by taking the residual (measured as: original (truth) image - restored image) for each star in the truth image. The results are displayed by ordering the 470 stars in this image by increasing magnitude. So, the faintest stars are at the left, and the brightest stars are at the right. The flux linearity of the blurred image is shown in Fig. 3. The graph for the Richardson-Lucy implementation is seen in Fig. 4, the spatial iterative adaptive algorithm’s graph is seen in Fig. 5, and the frequency adaptive algorithm’s is seen in Fig. 6. The frequency adaptive algorithm produces the most linear curve for this test, with a very noticeable reduction in the number of outlying stars having a large error. The flux linearity results for spatially varying PSF case (“sim3.fit”) are shown in Figs. 7 and 8. For testing the resolution enhancement properties of our algorithm, we generated a synthetic image containing points of equal intensity separated by progressively increasing distances. These points represent simulated neighboring stars. This image was then blurred with the Gaussian point spread function having a variance of 9, and a support of 50x50 pixels. Using this PSF allows us to test the resolution enhancement for binary pairs of stars which are poorly resolved in the blurred image. The minimum separation between a pair of points in the synthetic image was one pixel, and we measured the resolution of each pair according to the modified Rayleigh criterion (Wu, in Hanisch 1993). Table 1 shows the numerical values of the measured resolution criterion for the blurred image, the Richardson-Lucy algorithm, the generalized iterative adaptive algorithm, and the frequency domain adaptive algorithms. All stars in our simulated image were chosen to be of intensity value (100). The distance separating each pair of stars (in pixels) is given in the first column. Neighboring pairs were all separated by a large distance, so that the PSF never covered more than one pair at a time. The measure expressed here is R1 = (2I2)=(I1 + I3 ) where I1 and I3 are the peak intensities of the stars in the pair, and I2 is the intensity at the middle point between them. These resolution tests show that the frequency adaptive algorithm performed quite well in terms of peak resolution enhancement. It performed better than the Richardson-Lucy algorithm for most pairs. Although the generalized iterative adaptive algorithm and frequency adaptive algorithms did not perform better than the Richardson-Lucy algorithm for all pairs, the amount of resolution enhancement provided by them was significantly better than that of Richardson-Lucy for a large


Figure 3.

Simulated degraded image: “sim1.fit”.

Figure 4.

Richardson-Lucy restoration of “sim1.fit”.

Figure 5.

Generalized iterative adaptive restoration of “sim1.fit”.

9


10

Figure 6.

Frequency adaptive restoration of “sim1.fit”.

Figure 7.

Spatially varying simulated degraded image: “sim3.fit”.

Figure 8.

Generalized spatially varying restoration of “sim3.fit”.


distance 1 3 5 7 9 11 13 15 17 19 21 23 25 27 Table 1.

Blurred 1.0490925312 1.1297003031 1.0652104616 0.8098430634 0.5236188769 0.3096068203 0.1779404581 0.1075951308 0.0745507777 0.0608400740 0.0558008738 0.0541570261 0.0536803119 0.0535573885

R1

Lucy 1.2807079554 1.4553285837 0.7866657376 0.2116829008 0.0355268493 0.0043879338 0.0006407523 0.0002951132 0.0009177358 0.0056707887 0.0136818457 0.0185489822 0.0189928524 0.0157092437

Iter. Adapt 1.0966295004 1.2098157406 0.8871659040 0.2591046989 0.0000000180 0.0000000357 0.0000000000 0.0000000082 0.0000000283 0.0296860356 0.0476930998 0.0536854640 0.0486392528 0.0206987783

11

Freq. Adap. 1.0897836685 1.0970517397 0.3229246140 0.0000000000 0.0000000000 0.0000000000 0.0000000000 0.0239557754 0.0328580774 0.0246167779 0.0694369525 0.0000000000 0.0000000000 0.0005924603

Resolution Tests.

number of the pairs. It should be noted that the Richardson-Lucy algorithm requires more a priori knowledge than the other two iterative algorithms, again making a direct comparison somewhat difficult. The characteristic of the last two iterative algorithms which causes the large number of zero values in columns 4 and 5 stems from the fact that these algorithms exhibit a ringing which is not present in the Richardson-Lucy algorithm. Because of the positivity constraint, a number of the center values between pairs of stars are actually negative values which have been clipped to have a value of zero, resulting in R1 = 0. These results are, however, much sharper than the Richardson-Lucy results. Based on our analysis, we have found that the frequency adaptive iterative algorithm provides an optimal choice for restoring HST data. We have applied this algorithm to some of the real Hubble data as well. One of the real images we have tested with this algorithm is the “j413 crr.fit” image of Jupiter, seen in Fig. 9. The restoration of this image using the frequency adaptive algorithm is shown in Fig. 10. 4. Discussion From the results presented here, it is apparent that the set-theoretic based iterative restoration algorithms provide very good tools for the HST data restoration problem. We have shown that these algorithms are not dependent on any a priori knowledge of the signal and noise variances present. For the generalized iterative adaptive choice of the general regularization parameter, x2.1., there is also no dependency on the initial conditions of the iteration. In addition, the algorithms are relatively fast. This is especially true of the frequency adaptive algorithm which generally converges in a small number iterations. Given these considerations, the algorithms discussed in this paper present a very good alternative to some of the standard approaches being applied to HST data currently. References Andrews, H. C., & Hunt, B. R. 1977, Digital Image Restoration, Prentice-Hall, Englewood Cliffs Hanisch, R. J., ed. 1993, Restoration Newsletter, 1, Space Telescope Science Institute, Baltimore


12

Figure 9.

Figure 10.

Blurred image “j413 crr.fit”.

Frequency adaptive restoration of “j413 crr.fit”.


13

Kang, M. G., & Katsaggelos, A. K. 1992, Proc. SPIE Conf. Visual Comm. and Image Proc., 1414 Kang, M. G., & Katsaggelos, A. K. 1993, Proc. SPIE Conf. Visual Comm. and Image Proc., 1364 Katsaggelos, A. K., Biemond, J., Mersereau, R. M., & Schafer, R. W. 1985, Proc. IEEE ICASSP, 700 Katsaggelos, A. K. 1989, Optical Engr., 28, 735 Katsaggelos, A. K., ed. 1991, Digital Image Restoration, Springer-Verlag, New York Katsaggelos, A. K., Biemond, J., Schafer, R. W., & Mersereau, R. M. 1991, IEEE Trans. Signal Proc., 39, 914 Lucy, L. B. 1974, AJ, 79, 745 Schafer, R. W., Mersereau, R. M., & Richards, M. A. 1981, Proc. IEEE, 69, 432 Strand, O. N. 1974, SIAM J. Numerical Analysis, 11, 798 Weir, N. 1991, in Proc. 3rd ESO/ST-ECF Data Analysis Workshop, P. J. Grosbøl & R. H. Warmels, eds., European Southern Observatory, Garching, 115 White, R. L. 1991, in Proc. 25th Conf. on Information Sciences and Systems, 655