Image recovery from data acquired with a charge-coupled-device ...

1014

J. Opt. Soc. Am. A/Vol. 10, No. 5/May 1993

Snyderet al.

Image recovery from data acquired with a charge-coupled-device camera Donald L. Snyder and Abed M. Hammoud Electronic Systems and Signals Research Laboratory,Department of Electrical Engineering,Washington University, St. Louis, Missouri 63130

Richard L. White Space TelescopeScience Institute, 3700 San Martin Drive, Baltimore, Maryland 21218,and JointInstitute for LaboratoryAstrophysics, University of Colorado,Boulder, Colorado80309 Received July 15, 1992; accepted October 2, 1992; revised manuscript received October 21, 1992

A model for data acquired with the use of a charge-coupled-device camera is given and is then used for developing a new iterative method for restoring intensities of objects observed with such a camera. The model includes the effects of point spread, photoconversion noise, readout noise, nonuniform flat-field response, nonuniform spectral response, and extraneous charge carriers resulting from bias, dark current, and both internal and external background radiation. An iterative algorithm is identified that produces a sequence of estimates converging toward a constrained maximum-likelihood estimate of the intensity distribution of an imaged object. An example is given for restoring images from data acquired with the use of the Hubble Space Telescope.

1.

INTRODUCTION

Charge-coupled-device (CCD) cameras are commonly used for acquiring images of incoherently radiating objects in a diverse range of applications spanning the microscopic in biology

1-3

to the macrocosmic in astronomy. 4 5

Their wide

spectral response makes them useful for acquiring image data throughout the UV, visible, and IR wavelengths. CCD cameras are imperfect. Optical elements that precede the CCD sensor itself, such as field stops and lenses, limit resolution and introduce aberrations. The CCD sensor exhibits photoconversion noise, readout noise, nonuniform flat-field response, nonuniform spectral response, and extraneous charge carriers resulting from bias, dark current, and both internal and external background radiation.6 The significance of these various imperfections in a given application depends on how the CCD camera is implemented and used, such as whether it is cooled and whether stray radiation is present in the field of view containing the object of interest, but all of them are present to some degree in all applications. Various methods are used to compensate for the imperfections. Corrections for nonuniform flat-field response, bias, and background are often made by preprocessing data acquired with the use of a CCD camera before image recovery is attempted. In the usual approach, as described, for example, by Aikens et al.,' two calibration images are acquired by exposing the camera to a dark field and to a uniform or flat field. Then a precorrected image is obtained by forming the difference between the object and dark-field images and then dividing the result, pixel by pixel, by the difference between the flat-field and darkfield images. Any negative values that may occur in this process are set to zero. Restoration is normally performed on the precorrected image to recover the object's 0740-3232/93/051014-10$05.00

intensity distribution by compensating for blurring and aberrations that are caused by the optical elements and, in some applications, also by the medium through which radiation propagates from the object to the camera. There is a large literature on image-recovery methods. The text edited by Stark contains several chapters on the subject and numerous references to work accomplished by others.7 We shall discuss portions of this work as needed to place our current research in context. One area in which our method differs from those of others is that the restoration to compensate for blurring and the corrections for nonuniform flat-field response and background radia-

tion are performed together in a coordinated manner rather than in two successive, noninteracting steps. The method that we develop here addresses the following image-recovery problem. With reference to Fig. 1, there is a field of objects that emit incoherent light with a space-dependent intensity that will be denoted by {A(x), x E W},where T is the object space. This is a nonnegative function having the interpretation that its integral over any region is the average optical energy emanating from that region and received at the entrance pupil of a camera consisting of some optical elements and a spatially distributed CCD photodetector array. We assume initially that the radiation emanating from the field of objects is quasimonochromatic, so the dependence of A( ) on the mean wavelength is suppressed. The extension to include wavelength dependency for spectrally spread radiation is indicated subsequently in our discussion. Light is gathered with the optical elements and focused onto the array, resulting in the production of photoelectrons. The photoconversion intensity will be denoted by { t(y), y E @}, where ¶1 is the detector space. The photoconversion intensity is determined ideally by the object intensity and the point-spread function of the optical elements accord( 1993 Optical Society of America

Vol. 10, No. 5/May 1993/J. Opt. Soc. Am. A

Snyder et al.

- - …-- - - Camera - - - - - -CCD Photodetector:

~~~~~Array

Ss ss| Optical :

I

|

System

ing to (1)

where P = 'q/hi, -qis the quantum efficiency of the detector, h is Planck's constant, i is the mean frequency of the object's radiation, and p(y Ix) is the point-spread function. The point-spread function is nonnegative, space dependent in general, and to a good approximation given by the squared magnitude of the Fourier transform of the opticalaperture function, which accounts for obscurations that are present in the optics and effects of aberrations. It may be assumed without loss of generality that the pointspread function is normalized as a conditional-probability density fyp(y Ix)dy = 1. Modifications of this intensity to account for flat-field variations, background radiation, and spectral dependence are discussed below. We shall discretize the object and detector spaces recognizing that CCD cameras yield data on discrete arrays and anticipating that image recovery will be performed digitally. Let i, for i = 0,1, *,I - 1, and j, for j = 0, 1, ***, J - 1, be index array elements in the object and detector spaces, respectively. Suppressing the dimensions of the array elements, we rewrite Eq. (1) in the discrete form I-1

= P/(3j) 3(p(j I i)A(i),

j = 0,1,

- 1.

(2)

i=O

Under exceptional operating conditions, a CCD camera might be regarded as yielding {I( j),j = 0, 1, -, J - 1}as deterministic data, with no noise or further distortions of the object's intensity distribution. Recovering the image then requires solving the deterministic inverse problem [Eq. (2)], assuming that the constant P and the pointspread function p(- I ) are known, and the function {A(i), i = 0,1, * , I - 1}is to be determined subject to the constraint that all the quantities involved must be nonnegative. We refer the reader to Ref. 8 for a review of the literature that addresses this problem. Whereas this lit-

erature is relevant, it is unfortunately true that CCD cameras do not yield the ideal data

d() because they

exhibit noise and introduce further distortion.

1}. In the early 1970's

spatial distributions of radioactive substances. In the Shepp-Vardi approach the estimate of A(-) is selected to maximize the log likelihood of the Poisson-distributed data, given by

Fig. 1. Object viewed with a CCD camera.

1

-.

Richardson-Lucy iteration in the context of imaging

Space: Y

y E 91,

,J

Richardson9 and Lucyl gave an iterative solution to this problem. In 1982 Shepp and Vardi" rediscovered the

Detector

/.L(y)= Pf p(yIx)A(x)dx,

J - 1 that form a Poisson process with mean parameter {(j), j = 0,1, ---, J - 1} given in Eq. (2), where n(j) is the number of photoconversions measured in the jth pixel of the CCD array. Recovering the image then requires solving the stochastic inverse problem of estimating the nonnegative function {A(i), i = 0,1, ** , I - 1} given A,

p ( ), and {n(j), j = 0,1,

PSF:p(ylx)

Object Sipace:X

1015

The

image-recovery method that we develop is iterative and bears some similarity to those discussed in Ref. 8, but the situation that we consider here takes into account the noise and distortions found under more realistic operating conditions. When photoconversion noise is taken into account, the CCD camera is modeled as yielding data n(j),j = 0,1, ** ,

2(A)=

> (j) +J-1>ln[/(j)]n(j),

J-1 -

j=0

j=0

(3)

where ( ) is defined in Eq. (2). Their numerical solution is derived by using the expectation-maximization algorithm of Dempster et al. 12 and is of the iterative form 1 J 1

A,1(i)

=

p(jI

E=1-1 /3j= Dp(

k(i)-

i)

- n(j).

I')L,(i')

(4)

i'=0

The convergence of the sequence of estimates Ao(), Al(), A2(-),- toward a maximum-likelihood estimate of A(-) has been established.'3 "4 The Shepp-Vardi solution [Eq. (4)], based on maximizing likelihood for Poisson-distributed data, is identical to the Richardson-Lucy iteration. This iteration has the properties that the estimates at each stage satisfy the nonnegativity constraint A(i) 2 0 for L(i) = each i, and the total energy is preserved, / XI'ZA,

zjJ-n(i).

The method that we develop in Sections 2-6 can be viewed as a modification of the Richardson-Lucy iteration

to account for additional effects that are present in CCD data. The modifications are derived by using the expectation-maximization algorithm to maximize likelihood according to the Shepp-Vardi procedure. The novel aspect of the work that we report is a new procedure for recovering images of an object from the data acquired with a CCD camera wherein the procedure accommodates the major effects encountered when such a camera is used in an optical system. In Section 2 we extend the model [Eq. (2)] to account for CCD camera data more accurately, and then we state the stochastic image-recovery problem. In Section 3 a solution is stated. This solution is iterative and produces a sequence of estimates of the object's intensity for which the corresponding sequence of data likelihoods is nondecreasing. The production of stable solutions requires regularization, and this is also discussed in Section 3. Several comments and problems that need to be addressed are reviewed in Section 4. These include evaluation of the performance of the method that we develop, computational issues in producing solutions with our method, and some practical effects in CCD cameras that are not included in our model. An example of the use of our method with data from the Hubble Space Telescope (HST) is given

in Sec-tion 5. Appendix A contains the mathematical development of our method.

1016

J. Opt. Soc. Am. A/Vol. 10, No. 5/May 1993

Snyder et al.

2. STOCHASTIC IMAGE-RECOVERY PROBLEM A model for CCD camera data is given in Subsection 2.A.

Then the image-recovery problem is stated. A.

no(j)= extback(j) + intback(j) +

Charge-Coupled-Device Camera Model

A CCD photodetector array is a matrix of pixels in each of which charge accumulates during an exposure interval. As described by Aikens et al.,' data acquired in the pixels are read out by transferring a row of the matrix array into a serial register, sequentially reading pixel values from that register, and then repeating this process until all rows and their pixel values are read out. Thejth value that is acquired in this process is given by

r(j) = [nbj(j) + nextback( j +

intback( j

+ bias(J) + g(j),

+

dark()

3(J) .p(j | i)A(i). i=o

The function {f,(j),

Then

{no(j),

j = 0,1,

,J - 1}is a Poisson process with

meanfunction{,o(j), j = 0,1, *, J - 1},where 1o0(j) = /extback(j) + Lintback(j) +

= 0,1,

(6)

,J - 1} accounts for

nonuniform quantum efficiency, pattern noise, bad pixels, and charge-transfer inefficiency. /3(j) is the probability that a quantum of optical energy incident on the jth pixel is converted into an electron in that pixel. nextback( j) is the number of photoelectrons that are due to external background radiation in the field of view containing the object of interest. It is a Poisson-distributed random variable with a mean given by Atextback(j), which is in the form of Eq. (6) with A(-)replaced by the intensity of the background radiation. nitback( j) is the number of photoelectrons that are due to internal background radiation from luminescent radiation on the CCD chip itself.6 It is a Poisson-distributed random variable with a mean given by IJint back( i). ndark( j) is the number of thermoelectrons that are generated because of heat. It is a Poisson-distributed random variable with a mean given by /Ldark( J) nbias( j) is the number of electrons that are due to bias or fat zeros. It is a Poisson-distributed random variable with a mean given by bias( J)g( j) is a real-valued random variable accounting for noise that is present in the on-chip amplifier through which all pixel values are read. This is a Gaussiandistributed random variable with a mean of m and a standard deviation of o-. The mean m accounts for a deterministic bias in the amplifier, which is distinct from the random, fat-zero bias modeled by the Poisson-distributed nbias( J)Thus the data r( j) read from the jth pixel are a sum of integer-valued, Poisson-distributed random variables and a real-valued, Gaussian-distributed random variable. We assume that the random variables in this sum are statistically independent of one another and of their values in other pixels. The analog-to-digital quantization of the data r( j) that are present in practice is not included in our

dark(j) + Abias(j), (8)

and the data in Eq. (5) become

r(j) =nobj(j)+ no(j)+ g(j),

I-1

j) =

dark(j)+ nbias(j), j = 0,1,---,J - 1. (7)

j=0,1, **,J-1.

(5)

where the various terms on the right-hand side are defined as follows. nobj(i) is the number of photoelectron conversions that are due to object radiation. It is a Poisson-distributed random variable with a mean given by /Lbj(j), where /obj(

model; see Section 5 for a discussion of this and other effects that are not in the model. To abbreviate the notation, let the counts that are not produced by the object's radiation be denoted by no(-):

(9)

We refer to no(-) simply as background counts. B. Problem Statement

The statistical image-recovery problem is to estimate the

object's intensity function {A(i),i = 0, 1, * , I - 1} in terms of some given data {r( j),j = 0, 1, * , J - 1},assuming that the model parameters are known via separate calibration measurements of the response to dark-field ra-

diation [ideallyyielding/gint back(-)

+

dark( ) +

Lbias( )],

flat-field radiation [ideally yielding /3 0/. + int back(-) + Pdark(') + bias( ) for a field intensity producing 1Z],external background radiation [ideally yielding i(G)gextback( + Aint back(') +

dark(-) +

a short snapshotexposing

bias()]

the camera to no radiation (ideally yielding m and u), and a point radiator [ideally yielding p( )]. The estimate that we seek is one that maximizes the likelihood of the data, as given below, subject to the constraint that the object's intensity is a nonnegative function. Solutions to various versions of this problem are given in Section 3. 3.

SOLUTION

There are numerous references that discuss the maximumlikelihood method for estimating parameters in terms of the noisy data that they influence; see, e.g., Van Trees (Ref. 15, Chap. 2). With this approach, the estimate of the

object's intensity distribution is a function {A(i), i= 0, 1, , I - 1} that maximizes the log-likelihood functional, denoted by 2E(A),of the noisy data {r( j), j = 0, 1, , J - 1},where for the model in Eq. (9) J-1

A=

E1n(I j=O

x

1 n(j)=O (j)+

x exp{-[/Oi,bjj)

[A.b*(j) + 0(J)]n(i) +

LoQj)]}V 1

X exp{-[r(j)- n(j) - m]2/2cr2}),

(10)

where I-1 jAbj(j)

= ,/(j) >p(j i=O

Ii)A(i).

(11)

An explicit nonnegative function A(-)= argmax £(A) that maximizes this loglikelihood is unknown, so some numerical procedure

for producing a solution must be employed.

The expectation-maximization algorithm of Dempster et al.," is well suited to this purpose. We follow Snyder and Miller (Ref. 16, Chap. 3) in applying this iterative method.

ject's mean-value function A(-) in terms of some given Poisson-distributed data of this form was addressed by Politte and Snyder,'7 who obtained the followingmodification of the Richardson-Lucy iteration when the expectation in Eq. (13) was evaluated: Ak+1(i)

Some hypothetical data, termed complete data, are

adopted in applying the expectation-maximization algorithm, and then two operations are performed on the log likelihood of these data at each iteration stage of the algorithm. The complete data that we use are defined in Appendix A. The log-likelihood functional for these complete data is given by I-1

I-1 -eCd(A) =-

+ > ln[A(i)]m(i), >f(i)A(i) i=0 i=0

(12)

Ak+l(i) =

1 -

E[m(i) Ir, AII

(13)

The issue is now to evaluate the conditional expectation on the right-hand

side of this expression, which we do for

three special cases of the above model for data acquired with a CCD camera: an ideal camera with a uniform flat-field response, no background, and no readout noise; a camera with nonuniform flat-field response, background counts, and no readout noise; and a camera with nonuniform flat-field response, background counts, and readout noise. A. Uniform Flat-Field, No Background, No Readout Noise

We first examine the ideal situation when g( ) = 0, so there is no readout noise.

If, in addition, ,B(j) is a con-

stant and no(j) is zero for allj, then the model in Eq. (9) is the same as that used by Richardson,

9

Lucy,'" and Shepp

and Vardi," and, as shown by Vardi et al.,3 is the same as the Richardson-Lucy iteration in Eq. (4) when the conditional expectation is evaluated. B. Nonuniform Flat-Field, Nonzero Background, No Readout Noise

Suppose, again, that g( ) = 0, so that there is no readout noise, but now allow ,B(j) to be spatially varying and no( j) to be nonzero. Then the data in Eq. (9) become =

nobj(j)+ no(j),

= A

I1-1

(i) - E

j = 0,1,**,J - 1. (14)

These data form a Poisson process with mean-value function {Lobj(j) + Ao(j),j = 0,1,---,J - 1}. The problem of determining a maximum-likelihood estimate of the ob-

r( j).

i)

Pj

pP(j i i')Ak(i') + /Lo(j)

/3(i) j=0 P3(j)

i'=O

(15)

It is evident that the nonnegativity constraint is satisfied at each iteration stage provided that it is satisfied initially, but energy normalization is not maintained. In the limit as k -- cc, the following equation holds: I-'

when all terms that do not depend on A(-) are dropped, where /3(i) = )JcJfo(j)p(j I i), {m(i), i = 0,1, ** , I - 1},as defined in Appendix A, is a Poisson process with mean I - 1}. An expectation function {1(i)A(i), i = 0, 1,. (E step) and a maximization (M step) are performed at each stage of the algorithm. The conditional expectation Q(A | AI) = E[-cd(A) r, A] is evaluated for the E step, and a A(-)maximizing this functional is selected for the M step = argmax Q(A|Ak),wherer = {r(j),j = to produceAL+10() 0,1, ***, J - 1}are the data and A (k) is the estimate of A(-) at stage k. Thus from Eq. (12) the E and M steps yield

r(j)

1017

Vol. 10, No. 5/May 1993/J. Opt. Soc. Am. A

Snyder et al.

I-1

j-1

i=O

j=0

3(j) 1p(j =E

Ii')A.(i) r(j)

I-'

/(i) 7p( I L'A(i') +

.L(J)

i'=0 (1(i)

The left-hand side is the average detected energy for photons emanating from the ith pixel in the object. The right-hand side is a weighted sum of the number of electrons read out from the pixels of the CCD array, where the weight given to the count from the jth pixel can be interpreted as the probability that an electron in that pixel is due to the object's radiation. C. Nonuniform Flat-Field, Nonzero Background, Nonzero Readout Noise We now use the full model that we have described for CCD-

camera data, permitting the flat-field response to be nonuniform and background radiation and readout noise to be nonzero. We identify two methods for accommodating the nonzero readout noise. The first method is an approximation that is valid when the readout noise variance o2 iS large. The second entails no approximation, so it holds for all levels of readout noise.

Large readout noise variance. The readout noise g( j) from the jth pixel is a Gaussian-distributed random vari2 able with mean m and variance or'. If m = a' and or is large, then according to Feller (Ref. 18, pp. 190 and 245) g( j) is approximately equal in distribution to a Poissondistributed random variable with mean o-2. If this approximation is made, then the data r( j) in Eq. (9) can be taken as a sum of independent Poisson-distributed random variables, and the expectation-maximization algorithm yields the iteration sequence in Eq. (15) for estimating the object's intensity function, with ,uo(j) being replaced by p'o(j) + cr2. This approximation for accommodating readout noise was suggested by Snyder'

9

for recovery of

images from data acquired with the use of CCD cameras in the HST. Note that the condition m = C iS easily achieved by adding a suitable constant to the data r(-) before the modified iteration [Eq. (15)] is performed. Arbitrary readout noise variance. By slightly modify20 ing a result presented by Llacer and Nufiez so as to

include the effects of nonuniform flat-field response and background radiation, it is possible to accommodate

1018

J. Opt. Soc. Am. A/Vol. 10, No. 5/May 1993

Snyder et al.

the effects of readout noise without making the above large-variance approximation. We do so by using the expectation-maximization algorithm with the complete data being that used in Appendix A for the derivation of Eq. (15) but augmented by the readout-noise process {g( j), j = 0, 1, ** , J - 1}. The conditional expectation required in Eq. (13) can then be evaluated by iterating the expectation as follows:

The modification of Eq. (18) to account for spectral dependency is straightforward. Suppose, first, that there is no prefiltering of the object's radiation, so a single data frame {N(j),j = 0,1, ** , J - 1}is acquired in response to radiation spanning the entire spectral band. We must modify /Lobj(-) to account for spectral dependency. According to Mooney et al.,2 Eq. (6) becomes L-1 1-1

E[m(i) Ir, Ak] = E{E[m(i) r, A, g] Ir, A*},

where g = {g(j), j = 0,1,.. ., J - 1}. Nowthe inner expectation is the same as that required for obtaining Eq. (15) from Eq. (13), with r( j) being replaced by r( j) g(j), so we obtain AL+1(i) =

As(iW)O= (i)E

1-1(l

i3(i)j_0 fj)P

p(j

V= iO

i:v)A(i:v),

~~~~~1 = Ak(i:V)

Ak+(i:V)

y2, a] 7

(18)

J

where we define the function

X

F[r(j), k, a] = E[r(j) - g(j) r,Ak],

(19)

E

/

X3(i)iIi:V

f(jV)p(j|i:v)

L-1

j=°Y, ,=0

/3(Xj:V')p( jI i':v')Ak(i':v') + o(j)

P'=O

X F[r j), A, ],

with

i=O

(23)

where

I-1

Ak(j) = 3() Ep(

|i) k(i) + Oo( j).

(20)

L-1 J-1

,3(i)= 3 2, (j:v)p(jIi:v),

Evaluation of the conditional expectation in the right-

(24)

z'=o j=O

L-1 1-1

hand side of Eq. (19) yields

=3

/k(j)

3(i:)P(iIli VAk(i:V)+ i.o(/) -

V=0 i=O

2 (n/n!)AWnexp[-(r- n F[r, A,oa

m)2/2cr2 ] (21)

=x

3 (1/n!),unexp[-(r

- n - m)'/2o,']

n=O

Special cases of this result are of interest. If /3(j) = 1 and uo(ij) = 0 for allj, then the iteration [Eq. (18)]is the same as that given by Llacer and J. Nufiez.'0 If the variance cr2 of the readout noise tends to zero, then the exponentials in Eq. (21) become very concentrated about the summation index n = r - m, and F[r,A, ] r - m, so the iteration [Eq. (18)] becomes the same as that in Eq. (15) with the data modified to compensate for a fixed bias m in the readout amplifier of the CCD camera. D.

(22)

where we represent the frequency band of interest by L discrete frequencies indexed as v = 0,1,... ,L - 1; 1(j: v) accounts for nonuniformity of flat-field response at the vth frequency; p(j i: v) is the point-spread function at the vth frequency; and A(i:v) is the object's intensity at the vth frequency. Then Eq. (18) becomes

Ii')Ak(i') + o(j)

i'=O

X F[r(j),

2,3(j:v)p

/Lobj(j)= 2

(17)

Alternatively, suppose that L data frames {N( j: v),j = 0,1, .. , J - 1; v = 0,1, ...,L - 1} are acquired by observing the object's radiation through L spectral filters. Then Eq. (6) becomes I-1

=

/Lobj(j:v)

2,(j:v)p(jIi:v)A(i:v), i=O

v = 0,1,

,L - 1, (26)

and, with obvious definitions, Eq. (18) becomes 1 Ak+l(i

V)

=

Ak(i

v)

X 2

Nonuniform Spectral Response

p(i

)

I-1

( j:v)p(jIi:v)

jO2P(i:7)P(

CCD cameras also exhibit variations in spectral response that must be taken into consideration in some applications. Chromatic aberrations in the optical elements pre-

1

Ji':VAkW:V+ Ao(j:P)

iVto

X F[r(i),Ak(i:V),91

ceding the CCD array and variations in the spectral response within and between CCD array elements are sources for spectral nonuniformity in a camera's response. As noted by Mooney et al.,2 these effects are especially pronounced when CCD cameras are used for IR imaging. In astronomical imaging, spectral filters that precede a camera may be used to observe an object's radiation in one or more preselected frequency bands.2 2 There are at least two data models of interest depending on whether the object's radiation is prefiltered into multiple frequency bands.

(25)

(27)

for v 0,1, ** , L - 1. In this instance the object's intensity in each spectral band is estimated by using Eq. (18), and the data are acquired with the prefilter for that band, and this estimate is formed independently of the data acquired in other spectral bands.

4.

REGULARIZATION

The problem of estimating A(-) as stated above is usually ill-posed because of the properties of practical point-spread

Vol. 10, No. 5/May 1993/J. Opt. Soc. Am. A

Snyder et al.

functions. Even without the additional difficulties that arise from statistical effects, determining A(-),for a given W ), as a solution to the deterministic Fredholm integral equation [Eq. (1)] is already well recognized as an ill-posed

23 That problem in the sense of Tikhonov and Arsenin. the data are in the form of noisy measurements of counts with mean A(-)rather than ( ) itself as in Eq. (1) only compounds the problem. Moreover, the discretization

yielding Eq. (2) from Eq. (1) does not resolve the difficulty,

as noted by Tapia and Thompson (Ref. 24, pp. 99-100) in their discussion of dimensional instability. The result is that solutions produced with the iterative solutions that we have stated above exhibit instabilities that become more and more pronounced as iterations proceed. Regularization is needed to overcome this undesirable effect. Snyder 2 6 and Politte and Snyder'7 and Miller,25 Snyder et al., have introduced the method of sieves and resolution kernels for regularization. With this method an estimate is sought not for A( ) itself but rather for a blurred version of this function d(i) = Yi3or(i i')A(i') that we term the desired function. Here r(-) is a resolution kernel that must be selected by the user. The idea is that estimating A(-) to arbitrarily fine resolution is not supported by the finite amount of data available, so one seeks to estimate this intensity up to some finite resolution that is so supported. For example, with the HST, a natural choice for r(-) might be the diffraction-limited point-spread function that would have occurred in the absence of spherical aberration, but performance studies may indicate, as they have for radionuclide imaging,72 6 that a somewhat greater sacrifice in resolution than this intrinsic resolution limit may in fact be necessary. Further regularization is introduced to control the destabilizing effects of quantum noise, which appear as speckle in the estimate. This is accomplished with a sieve Y = {d(-):d( j) = i--Os( j Ii) (i)} to constrain the estimate to be a member of a smooth set of functions, where 6( ) is an arbitrary, summable, nonnegative func25 26 The two kernels, tion and s( ) is a given sieve kernel. r(-) and s( ), can be selected arbitrarily (of course, the per-

formance of the estimator depends on the choice) subject to the requirement that they should be such that a function q( ) exists as a solution to the equation

q(j i')r(i' i),

(28)

k (j i) = ,q(j i')s(i' i)

(29)

p(j i) = and the function

is nonnegative; we assume without loss in generality that these kernels are scaled so that k( ) is normalized as a probability density. The function k(-) replacesp(-) in the iteration

[Eq. (18)], the final iterate of which is blurred by

the sieve kernel to produce the regularized estimate of d(-). In radionuclide imaging, these various kernels are usually selected as Gaussian functions, as discussed in Ref. 26. Another straightforward choice is to select the sieve and resolution kernels to be the same; then k (-) equals the point-spread function p( ), and the iterations proceed in the form of Eq. (18), with a final application of the sieve kernel to produce the estimate. A natural choice for attempting to compensate for aberrations in an optical

1019

system might be to select the resolution and sieve kernels to be equal to the unaberrated diffraction-limited pointspread function; then one would perform the iterations of Eq. (18) with the aberrated point-spread function as the kernel and then apply the ideal point-spread function to the final iterate to obtain an estimate of the object's intensity that would be seen with the unaberrated optical system. The regularized estimation problem becomes as follows: given data r(-) defined in Eq. (9) and given the resolution and sieve kernels, estimate the desired function d(-) subject to the constraint that the estimate be a member of the sieve S. When the sieve and resolution kernels are equal, the result at stage k of the iteration is 1-1

dk(W)= E s(i |i )Ak(i),

(30)

i'=O

where the estimates of A ) are produced by using Eq. (18). Thus, for the choice of equal kernels, one performs the iteration according to Eq. (18) and then blurs the final iterate by the kernel of the sieve. After extensive studies with real and simulated data, this is the choice used for radionuclide imaging," but other choices may be desirable for recovering optical images. Another way in which regularization can be introduced is by adding a penalty function to the log likelihood and then seeking estimates that maximize the combination. A discussion of such penalized likelihood methods is given by Snyder and Miller (Ref. 16, Chap. 3) and references therein. 5.

REMARKS

A. Some potentially important effects present with CCD cameras are not included in the model that we used. For example, we have not included analog-to-digital quantization of the data r(-) because they are read out of the camera in many applications; this effect appears difficult to accommodate mathematically, and its importance on the quality of estimates is unclear. Cosmic rays and other charged particles can cause one or more pixels in a CCD camera to exhibit deviant data values; methods for identifying such pixel values and precorrecting for them before performing image recovery are discussed by Murtagh and Adorf28 and Groth.29 Charge leakage from a saturated pixel into its neighbors is another effect that was not included in our model. B. As we indicate in our notation in Eq. (2), the dimensions I and J of the object and data spaces, respectively, do not need to be the same. However, the estimation problem will be ill-posed when there are more unknowns than data values I > J. In such instances the use of regularization as discussed in Section 4 is required so that the estimates that are produced are not unrealistically rough. Lucy3 0 suggested the use of subpixelization, where I > J and he noted this requirement for regularization. C. Analytical prediction of the quantitative accuracy of estimates that are produced via the iterations in Eqs. (15), (18), (23), and (27) is difficult and remains largely an intractable problem. Extensive experiments in the use of Eq. (15) for radionuclide imaging were performed 7 27 both by computer simulation and by collecting and processing real data from known distributions of radioactivity. Re-

1020

J. Opt. Soc. Am. A/Vol. 10, No. 5/May 1993

suits were evaluated quantitatively for bias and variance, and they were evaluated qualitatively by experienced radiologists and neurologists who regularly view reconstructions of radioactivity distributions. As a result of these studies, Eq. (15), including regularization with equal reso-

lution and sieve kernels, is being used for imaging positron-emitting radionuclides. A similar performance study will be needed, with practical parameter values to state with confidence that Eq. (18) can produce quantitative recovery, with accurate photometry, of images from data acquired with a CCD camera. However, because of the similarity of the mathematical models for the two stochastic inverse problems, our experience suggests that the iteration that we give in this paper will outperform other methods. 6.

EXAMPLE

In this section we present an example of the application of the techniques described above to HST data. As de-

Snyder et al.

scribed by White and Burrows,3 the HST primary mirror suffers from severe spherical aberration because of an error in the figuring and polishing process. The mirror surface is too flat, sagging by -2 Am from center to edge compared with the planned shape. The optical system consequently cannot be properly focused; when a focus setting that is appropriate for the central portion of the mirror is used, the light from the outer part of the mirror is very poorly focused. The resulting point-spread function has a sharp core that is only -0.1 arcsec wide surrounded by a diffuse halo that is -5 arcsec across. The core contains only -15% of the light, but because the light in the halo is spread over such a large area, the surface brightness of the core is considerably higher than that of the halo. The wide-field camera (WFC) of the HST uses 800 800 pixel CCD detectors with readout noise of -15 electrons per pixel and an analog-to-digital conversion factor of 7.5 electrons per digital number. Figure 2 shows a 91 x 91 pixel (9.1 9.1 arcsec) section from an image of

Fig. 2. Hubble Space Telescope image of the star cluster R136 taken with the wide-field camera. The image is 91 displayed with a logarithmic gray scale.

91 pixels and is

Vol. 10, No. 5/May 1993/J. Opt. Soc. Am. A

Snyder et al.

...........

........... . . . .. . .

.. . .. .

1021

....... ......

.................... ............... . . .

.......... . . .. . .. . . . .. . .. . . . .. . .. ....

....

. .....

.. . .. . .. . . . . .

. . . . .. .

.. .. . . . . . . . . . .. . .. . .. ......... . .. .. ....

.... . . .. .. . .. . . .. . ...... .... ... .......... .. . . .. . ..

. .. .

.::-Y: ::X:: .................... .. . . ..

. .. . .. .

. .. .. . .

. . .. . . .. . .. . .. . .. . .. .......................

.. .. . .. . .. . . .. . .. .. .

.. . .. . .. . .. . . .. . ..

. .. . .. . .. . .. . .. . .. .. . ..

. .. . .. . .. . .. . .. . --------.......... -- -

................

.. . .. .

................

.. ... ... . .. .....

Fig. 3. Result of restoring image from Fig. 2 by using 600 iterations of the Richardson-Lucy method

the massive young star cluster R136 in the Large Magellanic Cloud, a satellite to our own Milky Way galaxy, taken

with the WFC in August 1990. The image is displayed with a logarithmic gray scale to make the noise in the sky visible. The effects of the spherical aberration are clearly visible, especially in the center of the cluster, where the overlapping halos of many stars produce a bright diffuse background. The small size of the point-spread function core compared with the WFC pixel size is also apparent.

We have deblurred this image by using both the original Richardson-Lucy iteration and the variant described in Subsection 3.C for the case of large readout noise. In both cases the restored images have pixels half as large as those in the WFC data. The results are displayed in Figs. 3 and 4. The restored images are similar in the center of the cluster where the readout noise is small compared with the Poisson noise. However, at the periphery of the cluster where the exposure levels are lower, the iteration including the effects of readout noise produces much better results. When the readout noise is ignored, the accuracy of the data is overestimated in regions in

which there are only a few counts per pixel; consequently, the restored image overfits the sky noise, producing many small spurious peaks in the sky while doing a poor job of fitting the brighter regions of the data. When the readout noise is properly included, the data are modeled better and the amplification of sky noise is suppressed. For this image no regularization was used: the positiv-

ity constraint alone is sufficient to regularize the solution for a collection of point sources sprinkled

on a

negligible background. For large extended objects such as galaxies and planets, regularization is vital. We performed some experiments on HST images by using the regularization approaches discussed in Section 4 and found the results to be somewhat unsatisfactory: to control the amplification of noise and ringing in the solution, the smoothing kernel must be considerably larger than intuition suggests should be necessary. The resulting images look too smooth compared with the resolution that the eye can discern in the data. We believe that for the restoration of HST images a more sophisticated regularization procedure, as yet unidentified, is required.

1022

J. Opt. Soc. Am. A/Vol. 10, No. 5/May 1993

Snyder et al.

-_-

-

, !::::.: :: ::: -:`:::: "'.*:::::: 777-777..-. _,::.--j:::-.:--,,.--.' - ... :.... . . . . 1. .. ..... ..:::_--_-:.:... ': :.::.:.:i:i:::j:j:::j.,',",.,.,:i::j:j::j:::j::j::::::::::::::::: ....... ' ' ' " ' '" ' -... !:::::::::: :. .................. .. :::;:.:::::::::::::::::::::::::::::::::::l::;::::::,.....X ........-...... :d 77!: I........:::::::::'..:.',:::i::':'::.:,::":::,:::.",: . '. :: ,.: :' , : I::::,I... 1. 11 . I::: .. .:;;-'.' i:-";': ... .::: ... :::::::::....,. .......':::.::, .:,.:,::, - -:1 . ........ ... ............'. :7:_:::::::::':'!":::' :,::!:::::::::::::::::::::::::::::::::::::::::: ::::: ' ::::::: -'-'::: :::::::::::::".::!::::::::::::::::::::::::::::...:. 1. . ... I: : :::i::i:: ::::j: :::::: : :.. :::: .::,.II., . :::!!' .'.......................-... : :: ...':'............................... :::::: ..:,:,:::::::,::i:i:i:::i:i:i:i::::.:::::::,::.: .: .... :.: ,. ... ..... :.:.:.:.:::::::i:::.-..i!::.i..::::::..i....i..,.., .......' :::::::::::.::::::::.::::::::::::::::::::: : : ::':: :::::::1::-'..::::::::::' - .:..:.:: ... !:::::::::::::::::::::::::::::::::::::::::::::::::::..,.........' ., ,...... ::.:::.:::.:.::.::.::.:::.:.:,:::::....?". , ..... .-. : .:::.:.",::%:;k:::.::.ii:ii:iii:'................:.:.::.:.::.:.:...-.......................,.......,.....................................................:.....,. 1 .I . 1. ...I...",I1.',..,.::-:-:-::: . I -'-' .. :.-.. X,: .:.:'::'':.. I . I.. I. "." ". '. ''...::::: .". II I.... : :I.,. ..... ',I :.....:': ... I: .. .. ...:::........::.:,:.:.:.:.:.:.:.:.:.:::.:.:::::::::::.:.::::::,.. '''..... ..... . . . ........-.,::.::::.:.::.:::.::.:., .. . .. .. '...... .:,::::: '.I..... .::::::.: - '. '.:.:::...:.::::: ::.: . :.,.: ':""'':iiiiii]ii - - .:::.::::::::::.::.:::.:.::. ............. I........................ ,,. .1.1-1--..'........... .....,'...'. I-.. :........'....1-... . I. ..'.d-..'...'.'.. ,. .!.:.-",:;i:::::::::.:..::::::ii-,.':]::.:.:.:.:.:.:.:.:.":. ,........ X.. ::::.::; ..' ..,. '" """ . .: .. ....... ..:]ii-'J 1...w: ...... "....... ::::: .. ::""" .-....... ::'::.,..::.:.: I ...... -....:::: : :::::::::::i-' ... ........... II -,-I- .: m:..-. .:::::: :::::ill ..... : : 1:.::::: ::::::::::::::::::::::::::!:::::::::::::...,:.:...:::,.,-II-:-:::-:::-:::-::-:-:::-:,:::,: ' :ii."." ".:.... ---... 1-1 ...I.... :.:........:.:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ... I. :: q: :::..:: . ....- ............ I.. .... 1. .... ... ....... ................ ' . ::::, ::::::::::::.....:::::: ....................... ,.,................,.................. .... - :: 1 :-:-:::::::X: .e:....... .......:: . I-. :: ' .... ... 'i': ... '..........,..........:::,. A--:::::::: . .. :::. :.:: . -, ................ ......... ....... :. ..:.i::,.ii..,....'....iiii' . , .,'.... .............. ............ :.:.::.::::::.::.::.:.::::.:.::.::.:.: . .:, ..... : .::: : ..'. .....:: .................. ..:.:.:.:.:.:.:.:.:......-......-.-..-..-....................... ...::.:;.:.:.::-:-:: ' : -::: ::: x!:-.:: :':I::::: ................. .....................:::-:-:-:-::::-:-:-:I:-:-:... -1::::::::::::.:.::.:.:':. ............ .. ...:,: : ... :......... .'_'.. :!!'...'."!"...'...i"."ii!!."'.....!IllI . .....jj:j;:' ."'............. .............. :.:.::-:::-::-:. '' ." ::::iiii]i:iiii::iiiiiiii]iiiiii:: :::;;i:::::::::::::::: ............ ;'........... .::...:,::.,:::::i:::::::::::::::::::::i::j:j::i .::: ::: ::::::'::: ::]l::::::::i::j:j::i:::::::j:::,:,:.:.:.: ... :.:.'' '' ' ".. .:::::::-:::: .... ,-::-:::-::-:-::-::-:-::-::::::.::::.::i::i:]:.., I. .,....................... :........ :::::: ': ::: ::::: :::. :::X::::::::::: I .. . .. "' ' ''......... .,.,, ,::,.,.:7::::. . ..................... - .:.:.:,. .... ....

:-:::-:::::-:::-::::::-:; ' '::::i::i::::i:i:::i:i:i:i:i:%.,i:i:: .. 1............ '-.. I ... i::iii:ii:]:i;::i;i:ii:i:i ............ ... 1.::::::::::::::: '-....'..1%...-.._..- :::X:::. .:l,:,,:.:.!.7.:.::.,...,..,.....::-::::::::-:-::" .::::.:::,: .'.,"..",""; I.......... ::.:::.:.:.::.:.: .:.:.,.:.::.:.:...,. """""""" :' ... '-... .... .. . .... :::: -....:: ...::' :'::: .... ....::: .. : I1-1....' ... I.. .. ......- 1.,...I.:-:::::.''' : :::i:::::I:ii::i . ': I... ,:::..:::::::::,:::::!::i::j:::::::-.'.7:::::::j:::%,. ..... .. ".,::::::.::.:.:::::: .... ,.:.i:::::j::::i:i::::i:::i:::::;:::::::i:::::i:.: ... .... .... ......,....... .. .. . . - ........ ,. .. ::::"""I"" -. :::::::::::::::::: ... :--::-::- ... :'. .'' ..... , ... ' I,:. , .. ......."..1.1 .... Ii:::,::,::,:,::::::::::::::::::":.:,:,":, 1.I'..:.,: ' .. ...........- "-.. :.:.:.:.:.:.::::::..."......... :''::' ::.:.::::::.:.,: :::::.:::.::.:::.:., ..::-:.... I.:.:.:.::.:.:.:::::::::: ... :. ..:: '... ...... ................... ::::::::: x ''jj:j:j:::::::;::::::::::: :::::::::::::::!:::::: !:.::::.....1..... :I ... :..:,:.:::::::ii.::i::::::::i::Z:i:: :': .... .I... .........'..-.....'.'.-.. ................ I ........, "" -.....'................ , j:::::. ''--...- ..' ::."!.,.,.,..,i:i:.::.::.:'iiii iii .:.:........ . ..... I .. I... I..... . .....1 .: i: :: ::::::::::::. ........ I.:".. .......... .:::7,::.:", ,'.':::::::,:::::::::.: :i:]:i:::: :::: :::i:::j::::::i:]:j:::::i:i*:.:.::.::.:.::.::::::::::.,.::.::!.:.:.: ...... :.: ., :. -:::. ............,.,...",....,.....:::::::i::-::-::::I ........... :::::::::::::::::::::::::::::: ............ :-:-:: . .. ::::::::::.,-"."....... ".....' ........... 1"......... I.......I... . :: .,::.,.:,..:::.::.::..,...-.................. ... I. I........ ....................... ::::::::::::::::::::::::::::::::::::::::::: ,..:....::::: :: :j:::::j::::::j::::' : Militia ....... ..::,.:.:.:.:.::.:...... ................... :. ...I. ::: ::::-::,....;. ,.I.. I I..i...iii::.".iiiiii:i:i...:,.,. .'4,.,.:::: '!:: :::xxx.. :.: "'...............' -::::-X:::: : :' ,.::.::.:.: .:. ".'.'....M .... iiiiiiiiiiii:iiiii:::iiiiii]::::::::::::::::::::::::::::::::::::::::::::::::::::::::::,::,iii:]]ii:iiiiii::::ii:ii . . . .. ............. ......... .:'. :::.:... . ::::: -. ...... . --.. : :::. .:: -::::-::::::::::%:::'::i*i:j:::::::::.::'..... _.....................:............ :1:-. . ..... .. .......... .'...'........'.'......"' ' !i:iiii*: ........ ...... ,-':i::;i:i:i:::i:::::;,i:::i:i:i:i:i::::::::: ' " ' """"-'-'--'-- "". -' -". -' :-::-:-7-':-:::::-:..... ..... ::i:iii]i,.",............ ....... ::;::::. -, ..j:...4.:: .: :.. .......::::.''....:iii:i:iiii::::::: ::: I.:I:': :.: . I... .::::::...::::::::::::::::.::.:::,,:,:: ,... :j:j ]:j::'j ]'j: ' ' ' '' "' .'.. :::::: .. '..-I ........ """ ..:::::::::i§§§:i:::i::i:i$::i.*i::::::; ..... ..... ........ ......... ......'.:.''::::::::::::::::: ' ._..........'.-'::::::::::: :::::::::::::::::::::::::::::::.:::::::::::i:"" :1-.......... .... .:1' ,::::::::Haiti . ,...,".1-1-1.1I. ::: :::::::::::.-..'::: ... ....... I .. :....... ,... - '' .. ...:1: '::-_ ... 1: ....... . _............ ': --:::::I....... ::::::::::.:::::,., -".:.:",.::.:.:..,:.::::i;::i:,i:::j:]::j:]".::::::::::::::::::::....;,.,.,.:::::: .,..-..,::..::::::::::i:i::,:::j:::j:j;;;j:j:::ii::::::::I .. -:::: . ,::::j:j:::j:j:. F:.... .- '..... '::::::.:j.::-. " ."",...,:.,:::::::::::.,." ......... . .::: '. : ::... :::::::::....:....,'......'.. I,:..,::::... -...--.. .. , .... . -I-,'.1, I.: . .. .... "',-... .........................::,::j:::::::::::..::::.:::,:::::, ' :,..: ... .::::::::: .... ......;--.... ,I... -..'..:j:j:j:]:i:i:::::::::j:::.". ::::: _........'.......... ' .::.:: X.,: :: :j::j:::':'j' :'::j:j:::: , .:..,.::::::: .... I.::::::::::::::::::::::::::::-... .:. :::::::::::::i:i:i:5:*i:i::i:::::i,]:j:! ... -:-::::X,...... . ....................... ............................,... .. ..,........................:i:::i:::iii:ii::i:::: ... :::::j::!::::jll: -:W:."': .... ...... ::iii:i:iii::i:iiii ......'...............'-....,i:::::i:::i:i:iiiii:i] ............ ".... .'...........

..*:::: ........ 1.............. :... - -:::-::':: ........................ .......................... .!! 1 ..::I:::::::::::::" X -:-:-:-:: !:: .. ..'.:.:.-. :::::j::: . . 1. .. 11- ...... I.", . :.. ..:::::::::]::iiiiiil::::i::::::'::!,::.:::.::.:.::-:.:.:.::.::......., . .. ....I,:.:.: , .,..",.,"....,.............'......::ij:j:l:i:::::i:i:::::::::, ........... .:::::::''.: ::: : :::::x.,:.:.:::::::xj::::':':::': ' :: ........... ".. .. ........... I. ... .................................................. :-:::..:::: ...:.::.::.,::... ... .-....... ...,.:.:.!: -,i., .....:.:.:.:.:.:::.::..:.:..:........:.::.:..::::::.:... ... :::: ''-::%;:::::!:: ::::::::::::.:.:.:.:.:.:.:.:..:.:., - . ...........,.....i::::::::::::::::: ::::::::":::::::::::,i:i:,ii,:::::i::i:::i:: . I - . .....:'.. ....::...,:::iii]:ii...:: . , -.. .::::: - :. I I . ":' :::::'::.-. .I.i::::::::::.,:::::::::::::::::::::::::::l.,,..,.,..:i::: -:'-:':':: ".:.:::.:::::::::",:; .......................... :::::::::,:.. ,%;i::-....::::iii '.':: i:' i ':i*j:;:..:.:' _...::iii]:i:i:::i:::::i::::i:]:::::j]:j::::i:::]:ii:::::i::i:ii:iii....:..,....:::..-.',.,...-'. .ii.. -':::::_ .. ':7:X:::::::::::::::::.. -.. :w.j:::j:::j':"'j:j: ']':'::: ::,:,:::::::::: ' '' "'- " ' ' ' ' ' ' 'W. ' ::::j:j:j:::j:::::::j

:..:. ..':1..:::::::::-:-'.-... , .... ... :il'. ." -.I'll..- -. 1..... I. ..,..I., -.:i:*-iii::'.. ' ':!iiiii:i ..i,]::::.:,::i:i:i::::::i::::::::::::::::::::i::::::::j:]::j]:i,:]:i:::::i:::i:.,i::::::::::::.::.:..:.:.:.::::.::.:.::::.:::::.:::.:,..,:..:::::::::j:::::::i:i:i:::j: ::i*i:i,. . .,.,.:.!:-.ii: ::::: ': ::::::'...-._-....' ,::::.::.!::.::.:.::.::.:.::.:.::::.::: , ' - : .:.X.:.. .' I ". .1-..... _ '. "' ".... "I ''d:_ :.:,::. :..... - I IF.'..'. 1. . II ::::::-::-::'X:'::.'. ':'.... .. '' .. ...'........... .. ::::::::'::::::::::::'::,:,:::::::::::::::::::: . ..... ' .,.::: ::::: :: . ....... I...j::i::::::::::i:i:i::::i::;::jij:j::i::i,.:I... :::' ::::::::::.:::::':':::::--' :".,::i:i:.'.." -:: '' ...... I:::: ..-I. I7' :!::::'] ._.....' ..- .............: ..-" . .. '..............'... I -... !:::::::::::: . .1-....... 'i::i:::::ii, ..... j:': ......... '::i I..' -1..b.. ... .......I.I... . , ,... --.-...-.. .,.... I.... : :.:: ,... .. ,::.. : i :i%: .,:.:. I.' :: "I" :j::::j:::i::::::j:::j:::::::::::::::::::::::::: " .::.:.:.,.,.,., ' ' ',.......... "' '" .. '' :: : I ..":,:: :: 1,.. -.. :: :':.:.::'!,:.-: ... ::::: _..'..........' . .-6 -.. ''...'.'... I. .1 ... ::::-.-.-.-.: ;::::::::. ....::..iiiii:iii ....... .... '... ....,.....-.................... :.::.::.::.:.::.:::.:.:.:.:...... - ".' :. ..II .: :::::::i, .: .'....... I:j:-:-:-:-:-:-:: .'.....'...... : : :' :: :.:. .. :ii:::* :::]:::::i::::::::::i:i:::i:::: j:::j:' .............W. .... ::::::.::.:,::::.: 1: : ::Ii:iii: :: : .............. ,::: ._' ..... .. ....... ...::::.:::i:;::::!:::....:..::::::::i::i:*:i:i:i:i-i:.:i::::j:::j::i:::::: ' - .. ....................... .. - - -.. - - - .,:.:.:..... ,.,:...::.::l..:,:.i:i.;.:::.: .-...-.-... ''...1.'1,.1::: ... -. X...... - - ......... - -.... '-'-::-:-::-::-:-::-:-::-: :::.:::::.:.: . ,'1"-,'.'..'.. .::::X:::::WK:_: .. i,,:,:::::j:::!:::::::::::::::::::: ... ............ I............................................ .:_ "'.". :.::::::::::: ' .. , -'..:::::::% :::::::: w:::::::: ::::::j' " ........ ' ......':::::::: ........ ' .........'.. ,::::::::::::::::::::::::,..,.,.................................... "..'......... I. ",:i::::::::::iii::i:i::ii.,t "':i. ,: :: :::::,::::::::::::. ,.-.'..i::i:iiiiiiiiii:]:::li]ii'' ' " " " """' .":::::::::::::::::::::::::::::::::::::: .II...... I.. I.1. - .::j:;...I... ..-...-:::::::::::::::' -:.. . ....... .::::::::::::::::::::::::: ......... "' .. :I--.-.--.-"'.' , :j:::j:j:j;;: ' ' I. I ...,. ................... .....--.. ... '..'....'...'.::::i .::*i::*j::-:-:-:.j:j.....:j ........"...............'...,...,........:::::.:.:.:.:.::::::::::::::::::.:::::::::::::: -'. ""' "' "" ... :::: ... .:"-.. : ,*... ':j::: : :j:::j:j:..".:.::::::: ::::::::.::::::...,.... ::: '11:1.2 ::.:j:k-,:.:i%§:i:i:i: i .0 *:::*XSs, M... :.: .:: ",'7:::::::::::i:::ii,::., ":::::::!:::i:j#.,.;ii,:..... . .::*iii* .................. j::::j.:::*:... "' X--...i' ._..':: ::::'..:' ................................... ::::xx :::.,>i::::::::::::::i..';,..,i:...,;,.,...".",.,iii7i *': :iK: ..... ,........".................. ..... .. .::::::.:::j:]::::::::::::::::::::::.,.,..:...:.:.:.:.:, ,' Mi'-:iii:: '--'.....'..'..'.,,'.::.:,::::::::,..ii....'.,.'!i:ii:iiii:i]]i:::::::::::::::::::::::!:::::::::::::::::::::::::::::::: .................... :::::::::::::::::::j::..,:::::::.'.::"..,.,.."..". .::::*:i::::::::;:i:::::::::::,::::::::....................,...............,..--l........I ................ . .::.:.:.:.:.:.::.:.:.:.:.:.:.:.:.:.::.:.:.:.:.:.::.::., :: : i*i:i*i ....I .................................. ::::::::: 1::::j:jj:i::::: 7:::::x x:::: ... .... ':: .............-'' .. ::::::-:::;:j:::j;:::':' :*i:: i i::': :' ]:'.':ii:::j:ijiii: .:.::.:.::::.:::.:.::.::.:.::.::.:.::.:.:.: . - - - -............ . I. . " ... .:. .1 .,:::::::::::::;.-.. ::::. . . -... I...... .. .. :.... .,*::::::.:.::::::.::::::::::...:.:.:.,:,:,....,....,%.I :::;: ::::: " ::::::::' .. ... .. .1:.: ..,...:::::Brazilian.. . .1:.:::::::::::::.:::::.:.::::::::::::::::: ::::..i: : i:i :]i ... .'...'.'..' ........ ::::: i i I Jurisdiction .:::::::::::::: 1. "'.'...".......'.:.::::,:!:::::::::, . .. ::-:::::.:.:.:.:.:.:.' .. __ .,..:::::::::::::::::::::.:,:::::::::::%.,.::,.,..,.::.,..... ..... ...*-:..-: .. : :1-..... *': _.'.'.:."", " .:.:.:.:.:. :-,............... :..".::,::i:i:i;::::::;::::;;;!::::.:...:.:.:.:.:.:.:.: ... ......._:..... -'-'-'-' '-'-'-'....... : - -. ... '.. .. .::: -. ,.:.::.:::.::::::::::.:: .:::.:.:.:. :::::.::.:, . .......': ::,:::::.. ::::::::;.... :.I---. ....- :::i:::::::: .iiisl%.i:.%'i. ".';::i'..". iiii::ii:ii:iiiiiiiiii ::: :::: ;i :::::::::::j::::::::::::::::::::j ' :: :..... :::.. ............................... "'. :... '..;,...*.ii]i:i:::::::::..,..:.:.:.:::.::.::........... ::.::!: ....' :::::::::,. ...... -. '. ::::,.. ..::::::': j;:j:j:j:.:: :::: ::::i:i:i:!:::i:i;ii;::iii!ii! .1 ':::':':: ............. ........... ::.:: :.::::: '-:::::-::':'.'.'...,. ....... 11 ... :::.::: '.':::::!:::::::j::i.::: .. .... '.......'.. ,:::::::::::::::::::::: -::::::i::::::::.,.,.,......,...,:::i::.': ........... . .............. .:.7,:::: ........ ....'... I... ::::jj:::::::::::l::::::::.:... .....- ::i::.::! ................ .,.... ''. -.-.-ll.:.:.:.:,:.:.:.:..:.::.:.::.:.:.:.:::::.::.::.::.::::::::::::: ..... : :... :.:::::.:.,::::::.-::::::::::: .............. '::::::j: I I......,:::: .': :'*::i::: .... ::::::!:!:!:,.:.... .. :.,...'.,.,.::i:::.::::::i::%::.;...::::::::,::::::: .... ::::::::::::::::::::::::::::::::::::::::::::::::::::::... - -.'.-..'::::::::::::::::;:::: ' '-'--'-'-'I:Ij .iiii:::]:ji::::::::i:i:.....,......,. ........ .::::::::::::::::::::::::::...i:i::::::::::::::j:j:]:j::::::i:::.'. ........ .... ..'..........'. ...'- I*W.....'' 1. ......... :..:: ........ .. :::.:::.::.:::.:..:::.::.:.::.:::...-.---.... .. .. ;..,...,..,.-,..,."..""..":: ..:::j,1. ................................... :.:::.::::.::::::::::::::::: 1.:: ::::.::-:..::::.: ..- ,,:,:'':. . ,.................... .......... .................. ....... :.:-:-::-:-:-:-::-:-:-:-: ...... ': ................. ...... I: .. ::,.:.iii:i:i::::::!i:i ... "....,........................''....... fflll,'-''.:..F'.------.-_ :::.::.:::.::.:.::.::.:.::............-...-..............,.........,.,..:.:.::.:.:.:.:.: : ... '.-. '..... .::::::: :.., ' :': , ::**i*i:i:.:.:..:.:':.:: 1...... ....:.. .. ::-:-:::::::::-::-::::-:-:-:::-::-: ....... .... :: :::::: ,i::ii 1::::::::::::::::::::: :::-::-:-::-:-:-:::-:-::-:::-::: ::::::Z::::;::.::,::. ........ ::::.::.::::.::.:.::::.:.::.:.::.::.:.::.:..:.:::::::::::::::::::::::::::::::::::::::::::::::::,." '.........'......... - .. - -'-'- -'.__ -... --'- -'................ - .:.::.. ............... ' . '::': : :.X , : :: 1: .:'.:.:::j:$*': : 1..:: x::X:X.q.."... --. 1 -:-.-:!.:.:::.:* I..II..........,............................. ". ..::.::.:::.:.::.:.:.:.:.:.:::.:.:.::.,::,:.::: -' - - --'- -"-'- ." .' ". .. .......... ...... I... I..................'.... .'.....'..... .-.. ,....11-11 -:i*j :j::*:x' I...............................:..:..:.::.:.:.:.::.:.::.:.:.:::: :::.:: , " " ," " " " ""' , " " "..".-. ,..,,..-.,. " " " ..". .I .1.1 .. i i::*:::i.i:: :.i ..... . -11. ........'... ' ' " ""-i...-..:-'.::-:-::-'--"-' .:: I.1.1... :::::::::i:i::i:ii::i ::..:,.::::::::::::::::::::::::::::::::::::::: .:::." ::::.::::'.:.:.:'j:'. : I.:.:.:.. :' i':.-.. .. ............ ......... ... ::::::::::::::::::.":::::::::::::::::::::::::::::::::: . .4. ,:.:.:..;:.::::::i:::;.::::.:"'',......,..''.,.",,.:-:-;-:-:-X-:S. I.---.. I-_ :. ,::.:::.:-.:::::::::::::::.:.: .: .:::::.:.::::: .,::i 'i :::i-i::::i:::::::::i:i:i:i:i:::i:::i:j ...... I..... :: : .:.:.'_::.':..':.::::: :I... : ::::::.:: -- .. I.. ... ..:.: :j:j:j:j::j:::j:::j.: ,::::::.: X !:.:_-.,*g ..,,.. .,:.:.:.:..:::::,::::::: ':I....:.:::::::7: .... ..... :-:::::::::::-:::-:-::-:-:-:-::-:::-: .1-":::.::: - - ..... :.............:.., .... .... ' ...... -- :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.::::::::::::::::::::::::::::iii'" " ' ""' . .. '', -. : : . I,-", ,.. .'-................ I,,::j::::::::::::::::::::'' .-" I.....I.1.1 . I.. .. I ..-....... :::-:::-::-:::-: :-:::-: -:1-:...... :,:.:.:.:.:::-:::::., .:..... :.:.:,:::-:::-:. .............. .I''::..... ........ !!"...._.'. 1, I.. ':: ......... .................. ..... .1 I '::'dl " 11 iI''.., I1:: : .,.!i..-..-..-'..-....-..-'...-:!, .II. I...I...... , ", .. . . ........... .. ... -. '.....'. ........... 11111.11-.1.1................................. ::::. i :.::...::*;:. ::.. ,:::.: ,:::::::::: : :::,:,: :.% .. .....,'.....-.... ..... ..,...'..", , :: :!::-.::: ........... ,, .W.:;:!::.....,.., .1 :j:j:j::'::.. 1. .:. . . ...... ,.:*:..-:..._ .:' .: -'.!.:.:.:.:.:.:...........,........... ' ,....." ::::::::::::::.:.:::,:.:.::::::::::::::::::::..:: .::::.::::. 11............... ........ .. ...,.......,....,.,::-I ...:,:.::!_::i:i: ..':::: :::::1 .......... I.................. . .. :"" 1.-I.,.... 1: ,.. :' " .:::.:.:::.:.:.:.:. 1. ''. I:::::t:', .. ' ::::::::::::::::::::::::::::::::::::::::::::::...: .. '' .. .- .. .I...I.1 .... .....:::::::::::::::: ... ''. ''::,. %:::::::::::::::::::::::::::::::::::::::::..", ... .... ... - :: .:.::: -""i:::.. 1.I..... .1 . . ,..I "-::-....*i:1 . ::,:: ........ -. 1 .... ::::...,.:::.:.:.:.:.:.:.:,:... I.. -I . . ... ..... 1..::::#:i:iiii '.....""x: ::.::.::.:.::::.:.::.:.:.:.:::..:........ .::.:::::.;:.:.::.::.:.: ".............. " ' " "" ..... . .. ....."......-Il ".-'i , ............. I.. ........ ... ............ ... ,:.::::.:::.::.:.::.:::::::::::.::.:.:.:.:.:.:.:.:.:.:.::.::.:::.:.:.:.::.:.:.: ... ........................... ,:::::::::::::::::::::::::::::::::::::::::::::::: ,.................. ...:: 2"................... ................. .,....'.....'...... -I. .' '.. .. :, :: :i::::.:::::; "" .:-:::-...'--I... :.: . I-...'.. I"::::::::j;::j ... ,...i.. -.... :::::::. - - - ' " ' ":".....:.. -,:::.:: - .. .. :::::.::iiii:i::::ii:.. .. .. ", ................... I. -' .. ", -' - - .". -' -' 1. 1. .... ........ : :I# ... k'ii:::::::::::,,,.,::::: .11'...I.,,..1.1,.... :,:.:: ...... :.:.:::::::::::::.:::: ........ ::.::.:::.::.:::.::.:.::.:.:.:..1-1.1 :. I..I ..': 7'::: 1.I :: ::: I: "...::j:]:::::::::::j: "..'. .:.:.:::.:::::::::::::.::,:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::I ,: -i:i"",,... '....:!!':::: :a....'..'..... ................... ... :: --. .. ,-I ... :-I......,............ v. ::.:!. ........ .... . .... '::j:j:j: ::::::::.::i::::.i:k,.:i::i:ii:::i:::::.:................'. ,.:.:.::.:::.:::::::.::::::.::.::.:.".. ......................... ,...,.,.,. - "..

'

: i:ii-

:I

.,........-.....

".... ...... : :':::-.:.i"' .1.11. X. ii.:I.... ........::1.::i:.-'--'iii::I., ...... Ir ::::::,:: " '...... ' :::::::::........ I.....-':. :::::: . . -..................

-. ..-..... I-!:.::.:.::: .. 1.II11, -:1.1.I-:,..I.. .." :: .....''.:' ...::::..,.:.:.,'*:.:::.:::ii-''*i.:: 11I........I.44.F............. :..:,..,.,..,.k"..',........,...::j; _.:X.--: ,..: .1.1 .. .1 .11I... '.. :X. I: -x"', :: 'I'-iiF-..*::x:.,.IParticipation :.I.:.:. :., !;!;;i*i*i: :::: iii'-...' :X... ., I. ,':::..,.,...I...I.1 :':-: : ... ::::'II :.." I.I I:-':-......:I--...

APPENDIX A:

i:i ........ -:.:.:-:.:.:,:.:.:,:IF,.:.,.:.:.:.:.:.:".. ....

..... :.:.,:,... :.::. z: .:..:::::::::::::-': ::::::-':::: :: :1i. ...,.-..-.................... :':"-':.: ::?:-'-.'-'f"::::i-:j:':ij:i: ::i*

DERIVATION OF EQ. (12)

In this appendix we give the derivation of Eq. (12), from which Eqs. (15) and (18) follow by use of the E and M steps

of the expectation-maximization algorithm, as outlined in the text above.

We follow Ref. 16, Chap. 3, in this

derivation. In the terminology of the EM algorithm, the data r(-) in Eq. (9) are called the incomplete data. We select some hypothetical data, called the complete data, as follows. Let and ',i denote the object and detector spaces, respectively. We imagine that each electron in 01 carries a binary-valued mark a E (0,1), with a = 1 indicating a photoelectron (i.e., resulting from object radiation) and a = 0 indicating a background electron (i.e., resulting from external or internal background, bias, or thermoelectrons). Denote by n(i, j, a = 1) the Poisson-distributed number of photoelectrons in detector pixelj that are produced by radiation from pixel i in the object space, and let

n(j, a = 0) denote the Poisson-distributed number of background electrons in detector pixelj. The mean-value

'....-::: ,.::::! .....

:.-

......

..... ..... .:::::::::::'' :::::::::::::::::::::::::::::::::::::::::::::::::::::: ....... .. %:::::::::::::::::::::::::::::::::::::::::::::::..., "'' .. %:j:j;j::...,::.::.,. i..'.;....I...;;:- ..' ...... .", ..... I...' :.::.:::::::....::::-:-:-:..". :.:::.:::::::::::::::::j:::j::j::j::]:,.....,i:::j::i:i::::::::::,.... ......-, :::::::::-::-:-::-:.:::.:.::::::::::::::::::::::::::::::::::::: j:::::j:::::i::i:i*.-..,:i::::i::::j:::, . ...."" :::.:::: ............ .... ,.::.::.:::.::.:.::::.:.:.:.:.:.:.:.:.:.. ...-.................. ... I I......... ..... I ............... :: : ::::. :...... ... ......................... I.....- , .:::::::::::::::::::::::::::....:.

functions of these Poisson processes are r3( j)p( j I i)A(i) and To( j), respectively, where TiS the exposure time. We select as the complete data the concatenation of these two Poisson processes and the readout noise, {n(i, j, a = 1),

n(j, a = 0),g(j), wherei = 0,1, **,I - 1,j = 0,1,***, J - 1}. It follows that the incomplete data can be expressed in terms of the complete data according to I-1

r(j) =

n(i,j, a = 1) + n(j,a = 0) + g(j), i=O

j=0,1,, J - 1. (Al) The complete data log likelihood is given by I-1 rd(A)

= -TY. i-0

I-1 J-1

(i)A(i) +

E ln[A(i)]n(ij, a = 1),

(A2)

i=Oj0

where all the terms that are not a function of A(-)have been dropped, and f3(i) = 2j,'Olp3(j)p(jIi). This can be

Vol. 10, No. 5/May 1993/J. Opt. Soc. Am. A

Snyder et al.

1023

for positron emission tomography," J. Am. Stat. Assoc. 80,

written in the form of Eq. (12), with

8-35 (1985). 14. C. F. J. Wu, "On the convergence properties

of the EM algo-

rithm," Ann. Stat. 11, 95-103 (1983). 15. H. L. Van Trees, Detection, Estimation, and Modulation

J-1

m(i) = Y_ n(i, j, a = 1).

Theory:

j=o

Part 1 (Wiley, New York, 1968).

16. D. L. Snyder and M. I. Miller, Random Point Processes in

Finally, for Eq. (12) we have normalized A(-) so that the exposure time is unity, r = 1.

Time and Space (Springer-Verlag, New York, 1991). 17. D. G. Politte and D. L. Snyder, "Corrections for accidental

coincidences and attenuation in maximum-likelihood image reconstruction for positron-emission tomography," IEEE Trans. Med. Imaging 10, 82-89 (1991).

ACKNOWLEDGMENTS

18. W Feller, An Introduction to Probability Theory and Its Ap-

This research was supported in part by the National Science Foundation under grant MIP-9101991 and the Division of Research Resources of the National Institutes of Health under grant RR001380. R. L. White appreciates the support of the Visiting Fellows Program at the Joint Institute for Laboratory Astrophysics of the University of Colorado. Figures 2 and 3 are based on observations with the NASA/ESA Hubble Space Telescope, obtained at the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astron-

19. D. Snyder, "Modifications of the Lucy-Richardson iteration for restoring Hubble Space-Telescopeimagery," in The Restoration of HST Images and Spectra, R. L. White and R. J. Allen, eds. (Space Telescope Science Institute, Baltimore,

plications

omy, Inc., under NASA contract NAS5-26555.

Md., 1990), pp. 56-61.

20. J. Llacer and J. Nuiez, "Iterative maximum-likelihood and Bayesian algorithms for image reconstruction in astronomy," in Restoration of Hubble Space TelescopeImages, R. L. White and R. J. Allen, eds. (Space Telescope Science Institute, Baltimore, Md., 1990), pp. 62-69. 21. J. M. Mooney, F. D. Shepherd, W S. Ewing, J. E. Murguia, and

J. Silverman, "Responsivity nonuniformity limited performance of infrared staring cameras," Opt. Eng. 28, 1151-1161 (1989). 22. Ref. 5, p. 8.

23. A. Tikhonov ad V Arsenin, Solutions of Ill-Posed Problems

REFERENCES

(Winston, Washington,

1. R. Aikens, D. A. Agard, and J. W Sedat, "Solid-state imagers for microscopy," Methods Cell Biol. 29, 291-313 (1989).

2. Y Hiraoka, J. W Sedat, and D. A. Agard, "The use of chargecoupled devices for quantitative

(Wiley, New York, 1968).

optical microscopy of biologi-

cal structures," Science 238, 36-41 (1987). 3. J. C. Sutherland, B. M. Sutherland, A. Emrick, D. C. Monteleone, E. A. Ribeiro, J. Trunk, M. Son, P. Serwer, S. K. Poddar,

and J. Maniloff, "Quantitative electronic imaging of gel fluorescence with CCD cameras: applications to molecular

biology," Biotech. 10, 492-497 (1991). 4. J. A. Tyson, "Progress in low-light-level charge-coupled device imaging in astronomy," J. Opt. Soc. Am. A 7, 1231-1236 (1990).

5. R. Griffiths, Hubble Space Telescope: Wide Field and Planetary Camera Instrument Handbook, Version 2.1 (Space Telescope Science Institute, Baltimore, Md., 1990). 6. J. R. Janesick, T. Elliott, S. Collins, M. M. Blouke, and J.

Freeman, "Scientific charge-coupled devices," Opt. Eng. 26, 692-714 (1987).

7. H. Stark, Image Recovery, Theory and Application (Academic, New York, 1987). 8. D. L. Snyder, T. J. Schulz, and J. A. O'Sullivan,

"Deblurring

subject to nonnegativity constraints," IEEE Trans. Signal Process. 40, 1143-1150 (1992). 9. W H. Richardson, "Bayesian-based iterative method of image restoration," J. Opt. Soc. Am. 62, 55-59 (1972). 10. L. Lucy, 'An iterative technique for the rectification of observed distributions," Astron. J. 79, 745-754 (1974). 11. L. A. Shepp and Y. Vardi, "Maximum-likelihood

reconstruc-

tion for emission tomography," IEEE Trans. Med. Imaging MI-1, 113-121 (1982). 12. A. D. Dempster, N. M. Laird, and D. B. Rubin, "Maximum

likelihood from incomplete data via the EM algorithm," J. R.

Stat. Soc. B 39, 1-38 (1977). 13. Y Vardi, L. A. Shepp, and L. Kaufman,

'A statistical

model

D.C., 1977).

24. R. Tapia and J. Thompson, Nonparametric Probability Density Estimation (The Johns Hopkins U. Press, Baltimore, Md., 1978). 25. D. L. Snyder and M. I. Miller, "The use of sieves to stabilize

images produced with the EM algorithm for emission tomography," IEEE Trans. Nucl. Sci. NS-32, 3864-3872 (1985). 26. D. L. Snyder, M. I. Miller, L. J. Thomas, Jr., and D. G. Politte,

"Noise and edge artifacts in maximum-likelihood reconstructions for emission tomography," IEEE Trans. Med. Imaging MI-6, 228-238 (1987). 27. D. G. Politte and D. L. Snyder, "The use of constraints

to

eliminate artifacts in maximum-likelihood image estimation for emission tomography," IEEE Trans. Nucl. Sci. 35, 608610 (1988). 28. F. D. Murtagh and H. M. Adorf, "Detecting cosmic ray hits on

HST WF/PC images using neural networks and other discriminant analysis approaches," in Proceedings of the Fourth International Workshop on Data Analysis in Astronomy, V Di Gesui, L. Scarsi, R. Buccheri, P. Crane, M. C. Maccarone,

and H. V Zimmerman, eds. (Plenum, New York, 1992), pp. 103-111.

29. E. J. Groth, 'An algorithm for removing cosmic rays from two or more cosmic ray split exposures," in Wide Field/Planetary

Camera Final Orbital/ScienceVerification Report, S. M. Faber, ed. (Space Science Telescope Institute, 1992), pp. 14-29.

Baltimore, Md.,

30. L. Lucy, "Restoration with increased sampling-images and spectra," in Restoration of Hubble Space Telescope Images, R. L. White and R. J. Allen, eds. (Space Telescope Science Institute, Baltimore, Md., 1990), pp. 80-87.

31. R. L. White and C. J. Burrows, "The HST spherical aberration and its effects on images," in Restoration of Hubble Space Telescope Images, R. L. White and R. J. Allen, eds. (Space Telescope Science Institute, Baltimore, Md., 1990),

pp. 2-6.