Artifacts in computational optical-sectioning ... - OSA Publishing

1056

McNallyet al.

J. Opt. Soc. Am. A/Vol. 11, No. 3/March 1994

Artifacts in computational optical-sectioning microscopy James G. McNally, Chrysanthe

Preza, Jos6-Angel Conchello, and Lewis J. Thomas, Jr.

BiomedicalComputer Laboratory,Institute for Biomedical Computing, 700 South Euclid Avenue, Washington University, St. Louis, Missouri 63110 Received November 18, 1992; revised manuscript received September 15, 1993; accepted September 29, 1993

We tested the most complete optical model available for computational optical-sectioning microscopy and obtained four main results. First, we observed good agreement between experimental and theoretical pointspread functions (PSF's) under a variety of imaging conditions. Second, using these PSF's, we found that a linear restoration method yielded reconstructed images of a well-defined phantom object (a 10-pum-diameter fluorescent bead) that closely resembled the theoretically determined, best-possible linear reconstruction of the object. Third, this best linear reconstruction suffered from a (to our knowledge) previously undescribed artifactual axial elongation whose principal cause was not increased axial blur but rather the conical shape of the null space intrinsic to nonconfocal three-dimensional (3D) microscopy. Fourth, when 10-pumphantom beads were embedded at different depths in a transparent medium, reconstructed bead images were progressively degraded with depth unless they were reconstructed with use of a PSF determined at the bead's depth. We conclude that (1) the optical model for optical sectioning is reasonably accurate; (2) if PSF shift variance cannot be avoided by adjustment of the optics, then reconstruction methods must be modified to account for this effect; and (3) alternative microscopical or nonlinear algorithmic approaches are required for overcoming artifacts imposed by the missing cone of frequencies that is intrinsic to nonconfocal 3D microscopy.

1.

INTRODUCTION

Computational optical-sectioning microscopy is a powerful tool for three-dimensional (3D) imaging of fluorescently labeled biological specimens.' Various restoration methods have been devised'-" to reduce degradations in optical sectioning, especially out-of-focus light introduced by microscope optics. Most of these restoration methods have been tested by simulation studies and by application to real biological specimens. Although simulation studies are necessary, they are not sufficient tests of a restoration method's efficacy. Virtually all restoration methods have been developed on the basis of a simplified model for imaging in fluorescence microscopy.",,1 3 This optical model assumes that the specimen radiates light incoherently and that image formation is linear and shift invariant, so that the 3D microscopical image is the result of the 3D convolution of the fluorescence-emitter distribution and the point-spread function (PSF) of the optics. A successful reconstruction of simulated data generated by such a model does not en-

sure success with microscopical data, because actual imaging conditions may differ from those modeled. A better test of both the utility of a restoration method and the model on which it is based is to reconstruct a welldefined object imaged by the microscope. Biological specimens are not well suited for this, because typically only qualitative information is available about them. We have used instead 10-pm fluorescent beads to construct a welldefined phantom object (Fig. 1) that mimics one of the biological specimens that we are studying. By comparing reconstructed images of such a phantom with the phantom's known distribution of fluorescence, we have been able to test the efficacy of one restoration method and also to assess the accuracy of the image-formation model. A key component of this model is the PSE We independently tested this component by comparing PSF's 0740-3232/94/031056-12$06.00

measured under a variety of imaging conditions with the theoretical predictions of Gibson and Lanni.' 4 We extended Gibson and Lanni's tests of their own model by examining PSF's for two different objective lenses (one oilimmersion and one dry lens) of lower numerical aperture (N.A.) than the lens tested by Gibson and Lanni. For the dry lens we also compared experimental and theoretical PSF's obtained both for point sources resting on cover slips of different thickness and for point sources located at different depths in an embedding medium. As a test of the accuracy of these experimental and theoretical PSF's, some of the PSF's were used to reconstruct images of the phantom object. 2.

METHODS

Data-Acquisition System Our data-acquisition system for 3D fluorescence microscopy is an emulation of one designed and built by John Sedat and David Agard at the University of California, San Francisco. The system's principal components are an inverted Olympus IMT-2 microscope (Olympus Corporation, Lake Success, NY.), a cooled (-45

0

C) charge-coupled

device (CCD) camera (Photometrics Ltd., Tucson, Ariz.) equipped with a Kodak KAF1400 CCD chip (6.8-pumwell width), and a Titan computer (Kubota Pacific Computer Inc., Santa Clara, Calif.). Point-Spread-FunctionMeasurements We measured PSF's by using red-fluorescent microspheres

(Molecular Probes, Eugene, Ore.) with diameters of 0.26 pLmfor an Olympus D PlanApo 20X/0.7-N.A. dry objective and 0.12 m for an Olympus D PlanApo 40X/ 1.0-N.A. oil-imimersion objective.

The microspheres'

di-

ameters were, for each lens, roughly half the diameter of the diffraction-limited spot. As judged by simulation. studies,'5 these microspheres were small enough to provide © 1994Optical Society of America

0

McNallyet al.

Vol. 11, No. 3/March 1994/J. Opt. Soc. Am. A

OPTICALCEMENT

-

1

r w'

-

0

B~EADS 0

COVERSLIP

z

x

KThe dashed boxes represent

the two 128

128 X

128 data sets examined and also indicate the orientation used for all subsequent xz sections:

the top row of pixels is farthest

from

the objective and deepest in the phantom.

adequate restorations. The refractive index n of the microspheres is 1.59. To measure PSF's under design conditions (which occur when all imaging parameters such as cover-slip thickness, immersion-oil refractive index, and optical tube length match the design specifications for the objective lens' 4 ), we dried microspheres

onto

0.17-mm cover slips and then imaged a spatially isolated microsphere. For the 20x/0.7-N.A. dry objective, cover slips of different thickness

by as much as 3 ttm (see Figs. 4f-4j below).

For all PSF measurements, the 3D data typically occu-

Fig. 1. Schematic of the bead phantom imaged with an inverted microscope.

microspheres at different depths in optical cement of refractive index n = 1.56 (Optical Adhesive 61, Norland Products, Inc., New Brunswick, N.J.). For these measurements, microspheres were mixed with optical cement, and the mixture was dropped onto cover slips and then cured by a 10-s exposure to ultraviolet light from a 100-W mercury-arc lamp at a distance of 5 cm. We estimated microsphere depth by using the microscope's focus-control calibration to determine the distance between the best focus (defined as the plane containing the image's maximal intensity) of the microsphere under study and the best focus of other microspheres that were dried onto the cover slip adjacent to the drop. This procedure is subject to an error of approximately ±1 gim, as judged by repeated depth measurements for the same microsphere. However, we suspect that because of spherical aberration, which increases with bead depth in the medium, the depth of the deepest microspheres studied here could have been underestimated

Y

1057

ranging from 0.0 (i.e., no cover slip) to

0.22 mm were used to test the effect of this parameter on the PSF. We also used this lens to measure PSF's from

expt.

pied a 128 cubic grid composed of voxels whose spacing was either 0.33 X 0.33 X 1.0 gm (Fig. 2) or 1.33 x 1.33 x 1.33 im (Figs. 4 and 5 below) for the 20X/0.7-N.A. objective or 0.17 X 0.17 X 1.0 gm for the 40 X /1.0-N.A. objective.

We measured the transverse (xy) sampling rate by imaging a micrometer grid. We calibrated the axial (z) sampling, which was controlled by a microstepping motor (Compumotor, Petulama, Calif.) attached to the microscope's focus control, by changing focus in a series of ten 1-Am steps. The total displacement was consistently measured to be 10 gm with use of a dial indicator accurate to 1 Am. For all PSF measurements, exposure time was set so that the maximum value in the images was within 10% of the camera's saturation level of 4095 analog-to-digital units. Calculation of Theoretical Point-Spread Functions

We obtained the source code for a program to compute PSF's from Sarah Gibson and Frederick Lanni (Carnegie

theory

20x

40x

Fig. 2. xz medial sections of (a,c) experimental and (bd) theoretical PSF's for (a,b) a 20x/0.7-N.A. dry lens and (c,d) a 40x/1.0-N.A. oil lens measured under design conditions. A logarithmic intensity scale (see Section 2) was used to enhance small values on the PSF tails. Scale bar, 3 gm.

1058

McNallyet al.


Mellon University), who developed the model that we wished to test.' 4 We tested our implementation of theircode by comparing computed PSF's with published PSF's calculated from the model and by comparing transverse (xy) and axial (z) widths of the central peaks of our computed PSF's with theoretical values.'6 To compute theoretical PSF's for different cover-slip thicknesses, we added this parameter to the program. Comparison of Experimental and Theoretical Point-Spread Functions

We first calculated theoretical PSF's by using the Nyquist spatial-sampling rate. Then, to compare to experimental PSF's and mimic the integration over a CCD well that occurs in practice, we averaged an appropriate number of adjacent pixels in theoretical PSF's to match the lower resolution of experimental PSF's. To permit quantitative comparison of PSF's we generated xz sections containing the peak intensity and then scaled each section so that the sections' integrated intensities were equal. We displayed images of experimental and theoretical PSF's by normalizing intensities to the interval [0,1] and then using a nonlinear transformation that maps intensity values to the same interval: I -n log(1 + aI)/[log(a + 1)], where I is intensity and a is a scale factor that was set to 10,000 for Fig. 2, 500 for Fig. 4, and 5000 for Fig. 5. Construction of a Phantom

We dried 9.7 + 0.03-gm-diameter red-fluorescent beads (Molecular Probes, Eugene, Ore.) onto a cover slip. Other beads from the same batch were mixed with the optical cement described above. A small drop (2 gLL)of this mixture was placed on the cover slip containing the dried beads and was cured as described above. Because the optical cement's refractive index (n = 1.56) was close to that of the polystyrene beads (n = 1.59), refraction of excitation or emitted light was minimal at bead-medium interfaces, thus permitting an accurate optical assessment of the bead's fluorescence distribution. By the preceding method we generated a solid, transparent phantom -1 mm in diameter and 200 Am in depth that contained beads at various depths, including some in contact with the cover slip (see Fig. 1). This phantom was designed to approximate a biological specimen that we are studying. We wish to determine, in the cellular slime mold Dictyostelium, the locations and shapes of a few fluorescently labeled cells distributed at random within a hemispherical mound of unlabeled tissue. The individual cells are 10 gm in diameter, and the mound of tissue is -250 gm in diameter and -50 gm deep.

section, but otherwise imaging conditions were identical to those described below for optical sectioning of thick specimens. Optical Sectioning of a Bead Phantom

Images of the phantom were acquired with either an Olympus 40X/1.0-N.A. oil-immersion lens or an Olympus 20x/0.7-N.A. dry lens. With the 40x/1.0-N.A. lens we acquired a 3D image of an isolated bead in contact with the cover slip (see Fig. 1), whereas for the 20x/0.7-N.A. lens we acquired a 3D image from a region of the phantom containing seven beads at depths from 0 to 43 Am (see Fig. 1). The 3D grid sizes were 128 X 128 x 128. Sampling rates were 0.17

0.17 x 1.0 Hm for the 40X1.0-N.A.

lens. At this N.A. and for the wavelength of the redfluorescent light of the phantom (A = 0.6 gim), Nyquist sampling requires

0.145 X 0.145 X 0.88

g.m.

Our under-

sampling was dictated laterally by the width of a well in the CCD and axially by the achievable resolution of the microscope's focus control. For the 20X/0.7-N.A. lens, sampling rates were 1.33 X 1.33 X 1.33 Am. At this N.A. and for A = 0.6 gim, Nyquist sampling requires 0.21 X 0.21 x 1.18 gim. In this case, our undersampling was designed to replicate that used in actual data collection from the biological specimen (Dictyostelium) that the phantom was constructed to mimic. Undersampling of biological data (by pooling counts from adjacent CCD wells) is often necessary both for achieving reasonable signal-to-noise ratios without bleaching the fluorescent dye in use and for achieving high temporal resolution by limiting the amount of collected data. Restoration Procedure

Weused a regularized linear least-squares estimation procedure,'0 summarized here for completeness. Although the regularized linear least-squares estimator 6 k can be obtained rapidly by inverse filtering, it is unstable because of inversion of small eigenvalues gj of the microscope's PSF operator.'0 To avoid this problem we have constrained 6 k to a superposition of the eigenvectors ej, corresponding to the k largest eigenvalues, j: 6 = 1,tj aje, where aj = ji(ej, i)/|pjf 2, (ej, i) denotes the inner product of ej with the optical sections i, and is the complex conjugate of gj. In this procedure, noise in the reconstruction increases as k increases,

whereas edges become smoother as k de-

creases. We determined empirically by selecting the value that yielded a reconstruction with tolerable noise levels and reasonably sharp edges of the bead. We typically used 50% of the total eigenvalues for reconstructions with experimental PSF's and 75% with theoretical PSF's. More eigenvalues could be used for reconstructions with theoretical PSF's because they were noise free (see Sec-

Spatial Distribution of Fluorescence in the Bead

tion 3 below).

To determine the distribution of fluorescence in the beads independently, we imaged them (prepared as described above) with a Zeiss laser-scanning confocal microscope. A Zeiss 40X/1.0-N.A. oil-immersion lens was used. As a separate assessment of bead fluorescence, we obtained physical sections. Beads were embedded in the chemical JB-4 (Polysciences, Warrington, Pa.) according to the pro-

As noted above, the image data that we reconstructed were undersampled. To avoid aliasing in the theoretical PSF's that we used for reconstructions, we computed

tocol for cold embedding

(0o-5 0C) provided by the manu-

facturer. Sections 1 gm thick were cut with a glass knife and examined with an Olympus IMT-2 microscope. A single focal-plane image was acquired for each physical

Nyquist-sampled theoretical PSF's and then Fourier transformed them to yield a theoretical optical-transfer function (OTF). From these OTF's we used only those components at frequencies represented by the sampling rate, i.e., frequencies as high as ±V(2p), where p is the pixel size. We typically performed minimal preprocessing of the data. Flat-fielding corrections 7 were initially performed

McNallyet al.


on some of the phantom PSF data. These had no effect on the principal features observed in the reconstructed images, presumably because neighboring CCD pixels were coalesced during data acquisition, and so the effects of local variations in pixel sensitivity were reduced.

1059

the cone's boundaries theoretically3 8 2 0 and then within those boundaries set all frequency components to zero. 3.

RESULTS

Experimental Tests of the Gibson-Lanni PSF Model Simulations Correctly Accounting for the Missing Cone

In simulating image formation and subsequent reconstructions of phantom objects, we found that the effects of nonconfocal microscopy's missing cone (a cone-shaped region of the spatial-frequency domain where the OTF is exactly 5 9 were easily bypassed inadvertently. zero)3,18'

One reason

for this was that PSF's that are generated computationally or experimentally have finite extent. These PSF's no longer possessed the pivotal property that leads to the missing cone, namely, that the integrated intensity in each plane should be the same. A second reason was that numerically determined PSF's and OTF's are subject to round-off error, and so instead of zero the OTF's missing cone contains small values. When PSF's subject to either or both of the preceding errors were used first to simulate image formation and then to generate the OTF's that were used for reconstruction of the simulated images, we obtained reconstructions that did not exhibit the artifacts described in Figs. 7 and 12 below. Wefound that these OTF's contained nonzero frequencies in what should have been their null space, or missing cone.

Therefore,

to simulate

the effects of the missing cone accurately, we determined

We used a 20x/0.7 N.A. dry lens and a 40x/1.0 N.A. oil lens to measure PSF's under design conditions (see Section 2). Experimental and theoretical PSF's were quite similar qualitatively (see Fig. 2). Wealso found reasonable agreement in quantitative comparisons. The envelopes of the curves describing experimental and theoretical PSF's were quite similar; however, detailed features of the wave forms, such as the number of ripples and their spatial distribution, differed (Fig. 3). Using the 20X/0.7-N.A. dry lens, we also measured a series of nondesign PSF's obtained at different depths in a mounting medium (Fig. 4). The Gibson-Lanni theory accounted well for the increasing negative spherical aberration with increasing depth, manifested in a more pronounced inverted-Y profile in PSF's from deeper point sources. We likewise observed satisfactory quantitative agreement between the envelopes of the experimental and the theoretical curves (data not shown). We also used the 20x/0.7 N.A. lens to measure PSF's obtained by use of cover slips of nondesign thickness (Fig. 5). The Gibson-Lanni theory successfully predicted the positive and negative spherical aberration manifested as up-

20x AXIALPROFILE

40x AXIALPROFILE

-

C0.75 .0

n

a,

0

0.5To 0

C1.25-

-30 -20 10 0 10 20 stage defocus(glm)

stagedefocus(gum)

30

a 20x TRANSVERSEPROFILE

40x TRANSVERSEPROFILE

.4

5-

0.775

ZIn

._ 0) 0S

0.

._ 0C 0M 0)

0.25-

-20 -10 0 10 ZU distance from optical axis (glm)

-10 -5 0 5 10 distancefrom opticalaxis (jim)

d b Fig. 3. The profiles are taken from the images of Fig. 2. Intensities of theoretical and experimental PSF's are indicated by dotted and solid curves, respectively. The axial profiles intersect the PSF point-source location, whereas the transverse profiles lie 6.5 and 3.5 ,um above focus for the 20x and the 40x lenses, respectively. The curves agree quantitatively, except that high-frequency oscillations in the theoretical PSF cannot be detected in the experimental PSF because of both the coarse spatial sampling and the noise present in the experimental data. Spikes in the tails of experimental PSF's indicate the height of the noise floor.

1060

McNallyet al.


DEPTH OF POINT SOURCE Oil.

7".

1 61].

26a

43" .

expt.

theory

Fig. 4. xz medial sections of (a-e) experimental and (f-j) theoretical PSF's for a 20x/0.7-N.A. dry lens focused at the indicated depths in a medium of n = 1.56. For both experiment and theory increasing depth increases the width of the central peak axially and also generates an increasingly distinct inverted Y profile.. For theoretical PSF's the actual location of the point source corresponds to the center of each image, thereby demonstrating the apparent axial shift introduced by increasing spherical aberration (f-j). Scale bar, 8 Aum.

COVER-SLIP THICKNESS O.Omm

0-14mm

0.22mm

expt.

theory

Fig. 5. xz medial sections of (a-c) experimental and d-f) theoretical PSF's for cover slips at the indicated nondesign thicknesses. For both experiment and theory an upright Y profile is transformed to an inverted Y profile as cover slips change from thinner to thicker than the design thickness of 0.17mm (Figs. 4a and 4f show design PSF's for this lens). Scale bar, 8 btm.

McNallyet al. right and inverted Y profiles, respectively, for cover slips thinner or thicker than the design thickness of 0.17 mm. The theory was consistent with experiment even for the extreme case of no cover slip (Figs. 5a and 5d). Again we

found satisfactory quantitative agreement between the envelopes of the experimental and the theoretical curves (data not shown).


1061

examined by confocal microscopy with a 40x/1.0-N.A. oilimmersion objective. Physically sectioned beads were examined by conventional microscopy with a similar lens. An individual bead's fluorescence was consistently distributed in a spherical shell (Fig. 6). As determined by confocal microscopy, the fluorescent shell's outer diameter was 10 gm and its inner diameter 6.5 gm. (We estimated

diameters by determining the distance between halfReconstructions of a 1o-,um Phantom Bead

We first sought to determine independently the distribution of fluorescence in a 10-Am bead. Whole beads were

maximal intensities at either edge of the bead.) The error in these measurements is roughly equal to the pixel size of 0.33 Am.2" In physical sections bead shape and dimen-

Fig. 6. Bead images obtained by (a,b) confocal microscopy and (c) physical sectioning, in all cases with use of a 1.0-N.A.oil-immersion lens.

For the confocal image, (a) xy and (b) xz medial sections are shown.

Scale bar, 3 Atm.

Xy

XZ Diana

Fig. 7. (a-d) xy and (e-h) xz medial sections of bead images obtained by conventional fluorescence microscopy (a,e) and then reconstructed by the regularized linear least-squares method with (b, f) an experimental PSF and (c, g) a theoretical PSE A 40x/1.0-N.A. oilimmersion objective was used. The analytically determined (see text) best-possible linear reconstruction of a hollow fluorescent shell is quite similar (d, h). The reconstructed images presented in this and all subsequent figures contain negative intensities (see Fig. 8 below), because the linear least-squares estimation procedure does not impose a positivity constraint on the estimated intensities. Negative intensities have been retained in the displayed images in order to preserve as much information as possible about the restoration procedure. In practice, negative values can be set to zero for improved contrast in the images.

Scale bar, 3 Atm.

1062

McNallyet al..

J. Opt. Soc. Am. A/Vol. 11, No. 3/March 1994 8000

sphere resting on the cover-slip surface. The restoration method resolved the hollow fluorescence structure of the beads (Figs. 7b and 7f) and also reduced the superimposed X pattern observed in the unprocessed data. However, the bead's axial extent was reduced only to 16 + 1 im. We obtained similar images when reconstructing with a design theoretical PSF for a 40x/1.0-N.A. oil-immersion lens (Figs. 7c and 7g). The artifactual elongation in the reconstructed image of the bead along the microscope's optical axis is predicted

6000 ._ Zn 4000 al ._

2000

theoretically

-2000 _ -24

-12

0

12

24

axial distance(m) from beadcenter

a 14000

._ c)

model.

We demon-

residual frequency components were inverse Fourier

7000

.

0

-7000L -12

by the image-formation

strated this analytically by first defining a bead mathematically as a spherical shell of constant fluorescence intensity at all radii between 3 and 5 Am and of zero fluorescence intensity elsewhere. We determined analytically the Fourier transform of such a shell and then removed all its frequency components within the null space of the OTF of a 40x/1.0-N.A. oil-immersion lens.3 18 20 The shell's

-6

0

6

12

transversedistance(m) from bead center

b Fig. 8. Representative intensity profiles through selected portions of the images in Fig. 7g. a, Axial profiles were generated by plotting intensities from a column of pixels in Fig. 7g that passed through the bead center (solid curve) or at 1.7 gm left of center (dotted curve). b, Transverse profiles were generated by plotting intensities from a row of pixels in Fig. 7g that passed through the bead center (solid curve) or at 4 Am up from center (dotted curve).

The plots show the extent of negative intensities generated by the estimation procedure. The largest negative undershoots occur at the poles and the equator (solid curves). Elsewhere the undershoots are significantly smaller (dotted curves). In all the reconstructed images displayed, negative numbers are included in the linear gray scale.

sions varied considerably, because the polystyrene beads were often warped by the sectioning process (as judged from scanning electron micrographs of the sections). Nevertheless, all sections revealed a ring of fluorescence, whose thickness was always less than that observed by confocal microscopy. The latter observation suggests that the confocal images still suffer blurring that is due to out-of-focus light. Unprocessed 3D images of the beads obtained by nonconfocal microscopy with a 4X/1.0-N.A. oil-immersion objective lens appeared considerably degraded (Figs. 7a and 7e). Although the beads' transverse (xy) extension remained at -10 gim,their apparent axial (z) extension increased to -22 gim. Moreover, the beads' fluorescence distribution did not appear obviously hollow, and a large X-shaped pattern was superimposed upon xz profiles of the image (Fig. 7e).

Wereconstructed these optically sectioned data by using the regularized linear least-squares estimation procedure" (see also Section 2) and a PSF measured from a micro-

transformed numerically to produce the best-possible reconstruction of the shell by a linear method (Figs. 7d and 7h). This reconstructed shell was close to the actual one: in both, the reconstructed object (in a medial xz profile) was diamond shaped with an axial extent of 16 ± 1 gm and was surrounded by a residual X pattern radiating from its perimeter. (Typical intensity profiles through these images are shown in Fig. 8.) We imaged the phantom beads with a 20X/0.7-N.A. dry objective and observed a similar z elongation in the reconstructions. The apparent axial extent was estimated at 14 ± 1.3 Am (Fig. 9e), in good agreement

with 15 gim, the

predicted axial extent of the analytically determined best linear reconstruction of a 10-gm shell for the 20X/0.7-N.A. dry lens (data not shown). With this lens we also examined a region of the phantom that contained beads at various depths and found that deep beads were reconstructed less adequately than shallow beads. As the bead depth into the medium increased (Fig. 9), the reconstructed bead's upper hemisphere became less distinct and the lower hemisphere became more elongated (upper by our convention means farther from the lens; see Fig. 1 and its caption). The reconstructions shown in Fig. 9 were performed with a design PSF (see Fig. 4a) measured from a microsphere in contact with the cover-slip surface, thus corresponding to a depth of 0 Am into the specimen. Because of the PSF's variation with depth in a specimen (as described above), we repeated the reconstructions with PSF's measured at different depths. Reconstructions Accounting for Bead Depth in the Phantom

We reconstructed the phantom beads at various depths with both experimental and theoretical PSF's for the 20X/ 0.7-N.A. dry lens determined at different depths in the specimen (the PSF's are shown in Fig. 4). For both experimental and theoretical PSF's we achieved the best reconstructions when the PSF depth was close2 2 to the 10-gm-bead depth (Figs. 10 and 11). When a significant mismatch occurred, one hemisphere of the reconstructed bead image was increasingly elongated in the direction of the PSF point-source location, and the opposite hemisphere was increasingly obscure. This effect was more pronounced in reconstructions that used experimental PSF's.

McNally et al.


1063

DEPTHS OF PHANTOM BEAD 17u

26>t

42L

xy

xz

Fig. 9. (a-d) xy and (e-h) xz medial sections of reconstructed images of beads located in the phantom at the indicated depths. Images were obtained with a 20x/0.7-N.A. dry lens and reconstructed with an experimental PSF measured under design conditions (see Fig. 4a). Note that the upper half of the bead's xz profile becomes progressively degraded with depth, whereas little change is observed in the medial xy profile. Scale bar, 8 m.

Theoretical PSF's consistently yielded less-noisy and somewhat less-distorted reconstructions.

4.

DISCUSSION

We tested the model for image formation in opticalsectioning microscopy both by performing reconstructions of well-defined phantom objects and by measuring PSF's under a variety of imaging conditions. We obtained two results that suggest that the imageformation model is reasonably accurate. First, we found satisfactory agreement between reconstructed images of hollow fluorescent beads and those predicted theoretically for a spherical shell. This agreement indicates not only that the restoration method that we used is performing adequately but also that the image-formation model on which it is based is fairly accurate. Consistent with this, we also found that experimental and theoretical PSF's were reasonably close qualitatively.

We did observe some

quantitative discrepancies between experimental and theoretical PSF's, but these were restricted to details of the wave form, such as the number and spatial distribution of ripples. These disparities could be due to several

factors that are not incorporated into the theoretical PSF's, including the polychromatic emission spectrum of the bead and the integrated sampling characteristics of the CCD,2` or to measurement errors in the experimental PSF's that were not flat fielded or averaged to reduce noise. Overall, though, we found reasonable quantitative agreement between the envelopes of the curves for experimental and theoretical PSF's. These observations extend Gibson and Lanni's experimental tests of their model to

several new imaging conditions and thus further substantiate the model's validity. We found that, when phantom beads were reconstructed with theoretical PSF's, the reconstructed images were as good as or better than those obtained with experimental PSF's, even when imaging was done under nondesign conditions. One reason for improved reconstructions with theoretical PSF's is the absence of noise. Our experimental PSF's are noisier than they might be, because we have not averaged over a series of PSF measurements as others have done.'7 But because theoretical PSF's yield reconstructions that resemble the best-possible linear reconstruction for the phantom bead and because theoretical PSF's are simpler to obtain for a variety of imaging conditions, these PSF's may in general be preferable for use in reconstruction algorithms. Although our results support the model for the microscope's PSF, they do not prove its validity. Indeed, the model is known to be inaccurate in at least one feature, namely, that it does not incorporate the partially confocal effect observed for high-N.A. lenses.2 4 To identify any other subtle deficiencies will require further tests of the PSF model. Among such tests are repetitions of our experimental approach but with use of different restoration procedures and different phantom objects to determine whether any of our observations is specific to the restoration and phantom that we used. Wealready know that the restoration method that we used is relatively robust to at least some discrepancies in the PSF, in particular to underestimation of the width of the PSE" This property could minimize the effect of any such inaccuracies in the theoretical PSE Future, more stringent tests of the model

1064

McNallyet al.


DEPTH OF EXPERIMENTAL

26u

PSF

43u

D

E

P

5R

T H

0 F

P

H

26 [t

A N

T

0 M

B

E 42[

A

D Fig. 10. xz medial sections of reconstructed images of phantom beads at the indicated depths with experimental PSF's at the indicated depths. The 20x/0.7-N.A. dry lens was used. The best reconstructions for each bead occur along the diagonal (a, e, i), where the bead's depth is closest to the PSF's depth. Abovethis diagonal (b,c, f), where bead images were reconstructed with deeper PSF's, the lower half of the bead image is degraded. The opposite holds below the diagonal (d,g, h). Scale bar, 8 ,m.

should also compare Nyquist-sampled experimental PSF's, which have been flat fielded and averaged to reduce noise, with theoretical PSF's that have been modified to account for the integrated sampling characteristics of the CCD camera.23 Such data would permit a stricter, more quantitative comparison than was possible in this study. Our reconstructions of well-defined phantom objects revealed two artifacts that can arise in computational optical-sectioning microscopy. One artifact appeared as an asymmetrical blurring of either the top or the bottom hemisphere in xz images of the 10-Ambead. This artifact was caused by reconstructing without accounting for shift variance of the PSF. PSF shift variance arises when objects are imaged at different depths in a specimen whose refractive index differs from that of the immersion medium.'4 We introduced shift variance by using a dry lens (n = 1) to image beads at different depths within a specimen of refractive index n = 1.59, and we observed a progressive degradation of reconstructed bead images with increasing bead depth. By demonstrating that this degradation was largely corrected in reconstructions in which

object and PSF depths matched, we showed that it is caused by ignoring the PSF's shift variance. The degradation is significant, since the dimensions of the phantom object were comparable with those of a biological specimen (see Section 2). Thus, for improved reconstructions of images obtained in the presence of PSF shift variance, this effect must be incorporated into restoration methods. In so doing, we would expect even better reconstructions than those that we obtained by matching PSF to depth of the bead center, simply because the PSF changes, even over the 10-am axial extent of the bead, and this is a variation that to our knowledge is not accounted for by any existing restoration method. The need for such improvements is potentially acute when a dry or an oil-immersion lens is used, because the refractive index of biological specimens is significantly different from air or oil. In contrast, for water-immersion lenses the effect should be less severe,' because the refractive indices of the immersion medium and the specimen are closer. We also found evidence for a second previously undescribed (to our knowledge) artifact, namely, an apparent

McNallyet al.


elongation of the bead along the optical axis. The principal cause of this artifact is not the increased axial blur relative to the transverse blur that is often noted in reconstructions of 3D microscopic images.13 6 25 The larger axial spread of the PSF relative to the transverse spread cannot explain how a 10-,um-diameter bead is extended in z to 16 Am, when the first axial null of the operative PSF occurs at ±1.75 m. In fact, the pointed caps observed at the poles of the diamond-shaped xz profiles of the bead images (see Fig. 7) arise not so much from blurred light from the poles of the bead object as from light originating from more-equatorial regions. One can demonstrate this clearly by decomposing a simulated bead object into two components, a torus and poles, and then using the same analytical approach described in Section 3 for the whole bead to simulate image formation and reconstruction of each component (Fig. 12). Remarkably, both the image of the torus and its reconstruction yield a diamond-shaped xz profile (Figs. 12b and 12c), in the absence of any polar caps in the object. The artifactual caps appear in the torus's image and reconstruction because of the PSF's conical shape and the consequent superposition of out-

1065

of-focus light emanating from equatorial regions in the toroidal object. Conversely, when only the caps are present in the object, the image and its reconstruction are of lower intensity (compared with the images of the torus) and moreover yield little information about the location of the actual caps (Figs. 12e and 12f). The latter features are caused by the OTF's missing cone,3" 8"9 which eliminates a number of spatial-frequency components present in the polar-cap object. In bead images these effects conspire to produce a striking artifact: the superposition of out-of-focus light radiating from around the bead equator generates false caps, and, because the intensities from the true poles are significantly attenuated, an illusory elongation is produced. This artifactual elongation arises therefore not from diffraction-limited effects but rather from constraints imposed by geometrical optics, namely, the conical shape of the PSF and its concomitant missing cone in the frequency domain. Earlier nonconfocal 3D microscopy studies had emphasized the blurring induced by loss of high-frequency components axially, but our results demonstrate that the loss of both high- and low-frequency components in the missing

DEPTH OF THEORETICAL PSF

7u

26u

434

D E P

T H 0 F

P H

A

26t

N

T 0 M

B

E

42

p

D Fig. 11. xz medial sections of reconstructed images of beads at the indicated depths with theoretical PSF's at the indicated depths. Imaging conditions as in Fig. 10. The reconstructions are similar to those of Fig. 10, except that with theoretical PSF's the signal-tonoise ratio is improved, and sensitivity to mismatch of object and PSF depth is reduced. Scale bar, 8 gm.

1066

McNallyet al.


I

Fig. 12. xz medial sections of (a) a simulated torus and (d) polar caps, (b, e) their simulated images obtained by convolution with a theoretical PSF for a 40x/1.0-N.A. lens, and (c,f) the best-possible linear reconstructions. The simulated objects (a,c) sum to yield a simulated 10-Am bead, namely, a uniformly intense shell of fluorescence with an outer diameter of 10 ptmand an inner diameter of 8 jim, and zero fluorescence elsewhere. Both the simulated images (b,e) and their reconstructions (c, f) reveal artifacts. Most notably, in the polar-cap images it is difficult to discern the location of the caps (e,f), whereas in the torus images (b,c) caps appear to be present. These results demonstrate that, in the reconstructions of the actual or the simulated bead (see Fig. 7), the apparent axial elongation arises not from blurring at the poles but rather primarily from residual out-of-focus light originally emanating from more-equatorial regions. Scale bar, 4 jIm.

cone can yield gross distortions in the shape of an object. Some form of this elongation artifact arises in many specimens. Reconstructions of our biological data as well as some simulations by Agard (see Figs. 11Gand 11H of Ref. 1) and images made by Erhardt et al.3 and Macias-Garza et al.25 suggest its presence. This elongation artifact is well known in another imaging modality, electronmicroscopic tomography, which also suffers from a missing cone (as a result of a limitation in the range of tilt angles available to the specimen stage). Simulations of electron-tomographic images of an annulus reveal an elongation along the missing cone's axis26 that is strikingly similar to the bead image's elongation. What can be done to limit the effects of this elongation artifact? It will be informative to compare reconstruc-

tions of the bead phantom with nonlinear restoration methods to determine whether any current algorithmic approach can reduce the elongation. In general, nonlinear restoration methods that impose constraints based on

a priori information about an object have the potential to

restore frequencies in the missing cone, since the linear solution may no longer satisfy the constraints imposed.2 6 2 9 An alternative approach to reducing the elongation artifact is suggested by our confocal images of the phantom bead in which xz cross sections reveal virtually no elongation. For exceedingly bright objects such as the phantom bead, confocal microscopy, which does not suffer from a missing cone, is the method of choice for 3D imaging. Unfortunately, many biological specimens are not nearly so bright as beads and so require longer exposure times. When imaged confocally, biological specimens, especially living ones, can suffer from photobleaching or heat damage.30 In such cases partially confocal imaging may permit a lower excitation-light flux while still affording sufficient axial-frequency support. Any residual axial elongation might then be more easily removed even with linear restoration methods, because, in principle, partially confocal microscopes also do not suffer from a missing

McNallyet al.


cone.3 '3 2 Thus it will also be of interest to calibrate achievable axial resolution in 3D microscopy by processing partially confocal images of phantom beads that are obtained either with a variable-aperture confocal microscope or with high-N.A. objectives on a conventional microscope. 2 4

ACKNOWLEDGMENTS

sectioning microscopy," Micron Microsc. Acta 23, 501-513 (1992).

16. M. Born and E. Wolf, Principles of Optics (Pergamon, New York, 1964).

17. Y Hiraoka, J. W Sedat, and D. A. Agard, "The use of a chargecoupled device for quantitative optical microscopy of biological structures," Science 238, 36-41 (1987). 18. B. R. Frieden, "Optical transfer of the three-dimensional object," J. Opt. Soc. Am. 57, 56-66 (1967). 19. N. Streibl, "Three-dimensional imaging by a microscope," J. Opt. Soc. Am. A 2, 121-127 (1985).

We are indebted to John Sedat and David Agard and their colleagues for their extensive assistance in assembling our optical-sectioning microscope. We thank Sarah Gibson and Fred Lanni for providing us with their software for computation of theoretical PSF's. We are also grateful to Fred Lanni for insightful discussions, to G. Michael Veith for the physical sectioning of the bead phantom, and to Steve Senft for collecting the confocal images of the 10-pm bead. This research was supported by National Institutes of Health grant RR01380.

REFERENCES AND NOTES 1. D. A. Agard, "Optical sectioning microscopy," Ann. Rev. Biophys. Bioeng. 13, 191-219 (1984). 2. M. Weinstein and K. R. Castleman, "Reconstructing 3-D

specimens from 2-D section images," in Quantitative Imagery in the Biomedical Sciences I, R. E. Herron, ed., Proc. Soc. Photo-Opt. Instrum. Eng. 26, 131-138 (1971). 3. A. Erhardt,

1067

G. Zinser, D. Komitowski, and J. Bille, "Recon-

structing 3-D light-microscopic images by digital image processing," Appl. Opt. 24, 194-200 (1985). 4. T. J. Holmes, "Maximum-likelihood image restoration for noncoherent optical imaging," J. Opt. Soc. Am. A 5, 666-673 (1988). 5. D. A. Agard, Y Hiraoka, P. Shaw, and J. W Sedat, "Fluorescence microscopy in three dimensions," Methods Cell Biol. 30, 353-377 (1989). 6. F. S. Fay, W Carrington, and K. E. Fogarty, "Three-dimen-

sional molecular distribution in single cells analysed using the digital imaging microscope," J. Microsc. 153, 133-149 (1989). 7. W A. Carrington, "Image-restoration in 3D microscopy with

limited data," in Bioimaging and Two-Dimensional Spectroscopy, L. C. Smith, ed., Proc. Soc. Photo-Opt. Instrum. Eng. 1205, 72-83 (1990). 8. A. Diaspro, M. Sartore, and C. Nicolini, "3D representation

20. J. A. Conchello, "Three-dimensional reconstruction of noisy images from partially confocal scanning microscope," Ph.D. dissertation

(Dartmouth

College, Hanover, N.H., 1990)

21. The error in our measurement of distances by determining half-maximal intensities at an edge is affected predominantly by the pixel size. The Rayleigh diffraction limit would contribute to this error only if the width of the bead's fluorescent shell were comparable with the size of the Airy disk. However, the shell's thickness is approximately ten times the radius of the Airy disk (0.35 Am),and so the bead's fluorescent layer can be considered a step. The incoherent step response preserves the location of the edge as measured by the half33 intensity

point.

22. Exact matches of phantom-bead depth and PSF depth were not always possible for experimental PSF's, because the latter were measured from microspheres embedded at random depths in optical cement. From many such measurements we selected experimental PSF's at depths as close as possible to those of the phantom beads. Subsequently, for comparing reconstructions having these experimental PSF's with those having theoretical PSF's, we computed each theoretical PSF at the same depth as that of the experimental PSF, so as not to give the theoretical PSF's an advantage. 23. F. Lanni and G. J. Baxter, "Sampling theorem for square-pixel image data," in Biomedical Image Processing and ThreeDimensional Microscopy, R. S. Acharya, C. J. Cogswell, and D. B. Goldgof, eds., Proc. Soc. Photo-Opt. Instrum. Eng. 1660, 140-147 (1992). 24. Y. Hiraoka, J. W Sedat, and D. A. Agard, "Determination of

three-dimensional imaging properties of a light microscope system," Biophys. J. 57, 325-333 (1990). 25. F. Macias-Garza, K. R. Diller, A. C. Bovik, S. J. Aggarwal, and J. K. Aggarwal, "Improvement in the resolution of three-

dimensional data sets collected using optical serial sectioning," J. Microsc. 153, 205-221 (1989). 26. K. M. Hanson, "Bayesian and related methods in image re-

construction

from incomplete

ery: Theory and Application, York, 1987), pp. 79-125.

data," in Image Recov-

H. Stark, ed. (Academic, New

of biostructures imaged with an optical microscope," Image

27. D. A. Agard and R. M. Stroud, "Linking regions between

Vis. Computing 8, 130-141 (1990). 9. M. Koshy, D. A. Agard, and J. W Sedat, "Solution of Toeplitz systems for the restoration of 3-D optical sectioning micros-

helices in bacteriorhodopsin revealed," Biophys. J. 37, 589602 (1981). 28. T. J. Holmes and Y. H. Liu, "Richardson-Lucy/maximum-

copy data," in Bioimaging and Two-Dimensional Spectroscopy, L. C. Smith, ed., Proc. Soc. Photo-Opt. Instrum. Eng. 1205, 64-71 (1990). 10. C. Preza, M. I. Miller, L. J. Thomas, Jr., and J. G. McNally,

"Regularized linear method for reconstruction of threedimensional microscopic objects from optical sections," J. Opt. Soc. Am. A 9, 219-228 (1992). 11. S. Joshi and M. I. Miller, "Maximum

a posteriori

estima-

tion with Good's roughness for three-dimensional opticalsectioning (1993).

microscopy," J. Opt. Soc. Am. A 10, 1078-1085

12. K. R. Castleman, Digital Image Processing (Prentice-Hall, Englewood Cliffs, N.J., 1979).

13. I. T. Young, "Image fidelity: transfer

characterizing the imaging

function," Methods Cell Biol. 30, 1-44 (1989).

14. F. S. Gibson and F. Lanni, "Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy," J. Opt. Soc. Am. A 8, 1601-1613 (1991). 15. C. Preza, J. M. Ollinger, J. G. McNally, and L. J. Thomas, Jr.,

"Point-spread sensitivity analysis for computational optical-

likelihood image restoration

for fluorescence

copy: further testing," Appl. Opt. 28, 4930-4938 29. J.-M. Carazo, "The fidelity of 3D reconstructions

micros(1989). from in-

complete data and the use of restoration methods," in Electron Tomography: Three-Dimensional Imaging with the Transmission Electron Microscope (J. Frank, ed. (Plenum, New York, 1992), pp. 117-164. 30. J. J. Lemasters, E. Chacon, G. Zahrebelski, J. M. Reece, and A.-L. Nieminen, "Laser scanning confocal microscopy of liv-

ing cells," in Optical Microscopy: Emerging Methods and Applications, B. Herman and J. J. Lemasters, eds. (Academic, San Diego, Calif., 1993), pp. 339-354. 31. J.-A. Conchello, J. J. Kim, and E. W Hansen, "Enhanced 3-D

reconstructions from confocal scanning microscope images. 2. Depth discrimination versus signal-to-noise ratio in partially confocal images," Appl. Opt. (to be published). 32. J.-A. Conchello and J. W Lichtman, "Theoretical analysis of a

rotating-disk partially confocal scanning microscope," Appl. Opt. 33, 585-596 (1994). 33. J. W Goodman, Introduction New York, 1968), p. 132.

to Fourier Optics (McGraw-Hill,