High-speed spectral imager for imaging transient fluorescence phenomena Curtis E. Volin, Bridget K. Ford, Michael R. Descour, John P. Garcia, Daniel W. Wilson, Paul D. Maker, and Gregory H. Bearman
We describe fluorescence spectral imaging results with the microscope computed-tomography imaging spectrometer ~mCTIS!. This imaging spectrometer is capable of recording spatial and spectral data simultaneously. Consequently, mCTIS can be used to image dynamic phenomena. The results presented consist of proof-of-concept imaging results with static targets composed of 6-mm fluorescing microspheres. Image data were collected with integration times of 16 ms, comparable with video-framerate integration times. Conversion of raw data acquired by the mCTIS to spatial and spectral data requires postprocessing. The emission spectra were sampled at 10-nm intervals between 420 and 710 nm. The smallest spatial sampling interval presented is 1.7 mm. © 1998 Optical Society of America OCIS codes: 000.1430, 170.2520, 300.6500.
1. Introduction
Imaging spectrometry is an application of Newton’s insight regarding the nature of solar radiation as a mixture of primary colors in due proportion.1 Imaging spectrometry is used to measure the due proportions of colors by measurement of intensities associated with narrow and contiguous spectral bands at every position element in an imaging instrument’s field of view ~FOV!. In other words, imaging spectrometry is used to acquire a three-dimensional ~3-D! ~x, y, l! data set, or object cube, which is also referred to as an image cube, a hypercube, or a data cube. In the case of microscopy, the spectral distribution of reflected or transmitted radiance or of emitted radiance is of interest. The instrumental techniques currently available to acquire the object cube are ~a! a filtered camera or multiple cameras, each with its own spectral filter; ~b! a monochromator configured for imaging, that is, equipped with an imaging array in the exit-slit plane; ~c! a Fourier-transform spectrometer equipped with an imaging array2; ~d! a tun-
C. E. Volin
[email protected]!, B. K. Ford, M. R. Descour, and J. P. Garcia are with the Optical Sciences Center, University of Arizona, Tucson, Arizona 85721. D. W. Wilson, P. D. Maker, and G. H. Bearman are with Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California 91109. Received 27 April 1998; revised manuscript received 8 September 1998. 0003-6935y98y348112-08$15.00y0 © 1998 Optical Society of America 8112
APPLIED OPTICS y Vol. 37, No. 34 y 1 December 1998
able illumination source; ~e! a rotating Risley prism chromotomograph3; and ~f ! an acousto-optic tunable filter ~AOTF!4 or a liquid-crystal tunable filter ~LCTF!5 combined with a CCD camera. A feature common to all these approaches is the need for scanning so that the entire ~x, y, l! object cube is acquired. The instrument described in this paper takes advantage of spatial and spectral multiplexing to avoid scanning. The object cube is interpreted as a collection of volume elements ~voxels!. The radiance from each voxel is divided among several detector elements on a large-format imaging array ~Fig. 1! by means of a computer-generated hologram ~CGH! disperser. For example, the instrument described in this paper is equipped with a cooled 1024 3 1024 detector CCD camera. The design and fabrication of the CGH have been described previously.6 Unlike the CGH disperser described in Ref. 6, the disperser used in the microscope computed-tomography imaging spectrometer ~mCTIS! instrument forms a 7 3 7 array of orders with diffraction efficiencies optimized to produce uniform irradiance in all orders for whitelight input. In other words, higher orders are designed with higher diffraction efficiencies to offset their increased dispersion. Applications and background. Fluorescence microscopy has developed into a powerful analytical tool in areas such as biology, cell physiology, and medicine.7 The basis for this power is the development of stable and sensitive analysis devices and fluorescence and luminescence probes, which function as markers for genes, ion concentrations, and assorted metabolic
Fig. 1. Mapping of signal from voxel to imaging array. ~a! Voxel at a center wavelength of 450 nm, and the distribution of that voxel’s signal on the imaging array. ~b! Voxel at a center wavelength of 710 nm, and the distribution of that voxel’s signal on the imaging array.
activities. Although current technology affords the opportunity to view many biological processes, the full advantage of video-rate fluorescence microscopy with high temporal, spectral, and spatial resolution has yet to be realized. The mCTIS can simultaneously acquire spectral data at precise spatial positions; this offers several benefits over existing systems. Scanning rates of currently employed scanners limit the ability to view rapidly occurring events.8 With the mCTIS, this limitation is removed, allowing the observer to visualize dynamic parameters in real time.
The current temporal sampling rate of the mCTIS falls within the required range for most physiological experiments ~0.01–1 s!.9 Increasing the temporal sampling rate requires a faster imaging array. Consequently, the potential temporal resolution is limited only by the signal-to-noise ratio on the array and by the detector read-out rate. High-speed imaging is beneficial in clinical diagnosis and disease staging for which optical biopsy is providing a less invasive alternative to more traditional approaches. Optical biopsy utilizes either the 1 December 1998 y Vol. 37, No. 34 y APPLIED OPTICS
8113
Fig. 2. Layout of the CTIS microscope. Distances are not to scale. See text for details.
autofluorescence of tissue or a fluorescent drug such as Photofrin II to induce an optical contrast between tumors and surrounding normal tissues. In the case of Photofrin II, a higher concentration of the fluorescence localizes in tumors compared with normal tissues.10 As these are often endoscopic procedures, rapid data acquisition is essential in reducing motion artifacts and for patient comfort. Spectrally, the mCTIS provides an advantage over conventional filter-based techniques by providing the user with the complete fluorescence spectrum at every pixel. Thus the two-dimensional ~2-D! image from multiple fluorescence reporter probes can be acquired simultaneously. This capability also provides temporal advantages over spectral imaging devices, which must be scanned to obtain full spatial-imaging capabilities.9 Viewing the entire spectrum for a combination of probes provides additional information about each fluorescence probe as well as its binding characteristics.2 For example, for several specific probes, fluorescence emissions shift between two emission wavelengths, corresponding to the ion bound and the unbound forms of the fluorescence probe. Therefore a ratio of the intensities of the two peak emission wavelengths provides precise information about the relative concentration of the ions within the cells.11 The mCTIS eliminates the need for shifting filters and for splitting the emission beam between multiple detec8114
APPLIED OPTICS y Vol. 37, No. 34 y 1 December 1998
tors because it simultaneously collects the spectral data from both the complexed and the free state of the fluorophores at specific positions within the sample. Information from the entire spectral range is retained. This capability provides a significant advantage in multiple-probe fluorescence in situ hybridization, a technique for identifying genetic defects.2 In addition, the faster acquisition of data decreases the sample’s exposure to excitation light and thereby reduces photobleaching, an irreversible destruction of the excited fluorophore.12 The remainder of the paper is organized as follows: Section 2 provides a description of the instrument components and layout. In addition, an example of a raw image is presented. Section 3 summarizes the modeling of the mCTIS as a linear system and the reconstruction technique used to process the raw image into an ~ x, y, l! object cube. Section 4 presents fluorescence spectral-imaging results from targets composed of 6-mm-diameter microspheres with a fluorescence emission peak at 578 nm. 2. Instrument Description
The mCTIS incorporates two optical subsystems. The CTIS subsystem includes the field stop, the field lens, the collimator lens, the CGH disperser, a reimaging lens, and a CCD detector array. The CTIS components must remain fixed after calibration. The pur-
Fig. 3. Representative raw image collected by the mCTIS microscope. This image results from five fluorescing 6-mm-diameter microspheres.
pose of the field lens is to relay the aperture stop of the microscope objective onto the CGH disperser. The fore-optics subsystem includes the illumination system, an interchangeable microscope objective, and a slide holder. Figure 2 schematically illustrates the two subsystems. The specimen is illuminated by a high-pressure mercury lamp through a light guide. The excitation light reflected from the sample is eliminated from the raw image by means of the dichroic filter ~520 nm cuton!. Figure 3 shows a representative raw image recorded by the mCTIS. This is the image that is formed on the CCD array ~Fig. 2!. The image consists of 49 diffraction orders associated with the CGH disperser. The zeroth diffraction order is located at the center of the image in Fig. 3. The zeroth diffraction order represents a direct view of the spatial radiance distribution in the field stop ~Fig. 2! and features no dispersion. As a result,
this diffraction order can be used to aim and focus the mCTIS. Note that the diffraction efficiency into the zeroth order varies with wavelength; thus the zeroth diffraction order does not represent the original gray-scale image. The remaining diffraction orders exhibit dispersion that increases with order number. The image in Fig. 3 contains five fluorescing microspheres. The broad bandwidth of the fluorescence emission light means that in each diffraction order, we observe a radial blur. Since the spheres are isolated, each sphere in the specimen appears as a linear streak in the nonzero diffraction orders. 3. Instrument Modeling and Object-Cube Reconstruction
The mCTIS instrument is modeled as a linearimaging system. Given the linearity assumption, the instrument can be described in terms of linear 1 December 1998 y Vol. 37, No. 34 y APPLIED OPTICS
8115
Fig. 4. Comparison of calibration images collected at the center and the corner of the field stop. Both images are superimposed to illustrate the spatial shift invariance of the mCTIS. The box indicates the extent of the field stop.
algebra. The 2-D image ~Fig. 3! and the 3-D ~ x, y, l! object cube are reorganized as vectors g and f, respectively. These vectors are related to each other by a system matrix, denoted by H. The experimental acquisition of H has been described previously.13 At higher resolutions, the matrix can grow to appreciable size. For example, the system matrix for the mCTIS occupied 291 MB of memory in a lossless, sparse-matrix compression format.14 Taking advantage of shift invariance and sparsematrix compression, the amount of memory required can be reduced drastically. Comparison of point-spread functions, i.e., calibration images, from the same spectral band proved that the mCTIS is approximately shift invariant. Temporally averaged calibration images were collected at the center and the edge of the field stop ~Fig. 4!. A normalized inner product was employed to compare the experimentally measured edge calibration image and the computationally shifted center calibration image. The typical value of the inner product was greater than 98%. As a result of shift invari-
Fig. 5. Reconstructed spectra from four different locations within the dense-array target shown in Fig. 6. Crosses ~1! denote the comparison spectra measured with a nonimaging, radiometrically calibrated reference spectrometer. 8116
APPLIED OPTICS y Vol. 37, No. 34 y 1 December 1998
Fig. 6. Corresponding locations of the reconstructed spectra in Fig. 5.
ance, only as many calibration images as there are wavelength bands need to be acquired and stored. As matrix images are called on during reconstruction of the object cube, these stored calibration images are shifted to the necessary position. The resulting matrix requires only 350 KB of memory. This process has no effect on the time needed to complete the reconstruction. The reconstruction of the 3-D object cube is performed with the Multiplicative Algebraic Reconstruction Technique iterative algorithm. The progression from the kth estimated object cube, fˆ k, to the ~k 1 1!th occurs according to the equation
fˆ k11 5 fˆ k
HTg HTHfˆ k
(1)
Fig. 7. Raster display of 31 spectral images of the densemicrosphere target. Wavelength increases in raster fashion from left to right, from bottom to top, and in steps of 10 nm. The numbers provided in this figure are in nanometers and are intended to help orient the reader.
~Ref. 15!, where T indicates the matrix transpose and the quotient indicates element-by-element division. Field tests performed in conjunction with a reference spectrometer determined the number of iterations yielding the optimum accuracy and efficiency.13 Typically, seven or eight iterations performed best. Therefore the results presented in the Section 4 were obtained after eight iterations of the reconstruction algorithm. Each iteration required approximately 20 s to complete on a 200MHz Pentium personal computer for 29 3 29 spatial resolution elements with 31 spectral bands. Iteration times as low as 5 s have been recorded for resolutions of 39 3 39 3 33 voxels with more efficient H matrix storage. In contrast, the raw-image data were acquired at an integration time of 16 ms. We expect that given the matrix–vector–product nature of the reconstruction algorithm, the time needed for object-cube reconstruction can be appreciably reduced by parallelization of the reconstruction algorithm. 4. Results
To demonstrate the potential of the mCTIS for highspeed imaging of fluorescence phenomena, we used two targets composed of fluorescing microspheres. The first target consisted of densely packed microspheres, and the second target consisted of an array of isolated microspheres. The microspheres’ fluorescence emission spectra peaked at 578 nm.16 The dense target was designed to simulate cellular structure, whereas the sparse target is intended to demonstrate the best-case reconstruction accuracy of the mCTIS. All raw-image data were collected with a cooled, 16-bit CCD camera. Data cubes were reconstructed with 31 spectral bands ~spectral sampling distance of 10 nm! and 29 3 29 spatial resolution
Fig. 8. Raster display of 31 spectral images of the isolated microspheres target. Wavelength increases in raster fashion from left to right, from bottom to top, and in steps of 10 nm. The numbers provided in this figure are in nanometers and are intended to help orient the reader. 1 December 1998 y Vol. 37, No. 34 y APPLIED OPTICS
8117
elements. The integration time used for imaging the microsphere targets was 16 ms. For reference, spectral signatures of the microspheres’ fluorescence emission were taken directly with a nonimaging spectrometer.17 The reference spectrometer was calibrated with a standard lamp to yield spectral irradiance.18 The accuracy of the mCTIS spectrum at a single pixel is quantified by a relative spectral error ~RSE!. This RSE value at a single pixel is defined by RSE 5
ismCTIS 2 asref i uasref i
,
(2)
where sref is the reference spectrum, smCTIS is the reconstructed mCTIS spectrum at a pixel, ixi denotes the Euclidean norm, and a is a scaling factor that minimizes the RSE. A.
Dense-Array Microspheres Target
The dense-array microspheres target consisted of three clusters of microspheres within the micro-
scope’s 50 mm 3 50 mm FOV. The foreoptics included a 1003, N.A. 5 0.9 Nikon MPlan objective.19 The spatial sampling distance between adjacent pixels is 1.7 mm. The dense-array microspheres target is intended to simulate cellular structure. Figure 5 shows spectra corresponding to the four pixel locations indicated in Fig. 6. The normalized RSE values were 8.5%, 25%, 9.9%, and 7.9% for Figs. 5~a!–5~d!. Figure 7 shows a raster display of the 31 reconstructed spectral images of the dense-array microspheres target. B.
Isolated Microspheres Target
The isolated microspheres target consisted of five microspheres distributed within the microscope’s 50 mm 3 50 mm FOV at the specimen plane. The same 1003 Nikon MPlan microscope objective was used in the fore-optics subsystem. The spatial sampling distance between adjacent pixels is 1.7 mm. The mCTIS can resolve individual microspheres, as shown in the raster display of spectral images ~Fig. 8!. Point-source targets, such as the isolated micro-
Fig. 9. Reconstructed spectra from four different locations within the isolated microspheres target shown in Fig. 10. ~d! Spectrum corresponding to a dark part of the scene. Crosses ~1! denote the comparison spectra measured with a reference spectrometer. 8118
APPLIED OPTICS y Vol. 37, No. 34 y 1 December 1998
grant to M. R. Descour and D. W. Wilson and by NASA contract NAS7-1260. References
Fig. 10. Corresponding locations of the reconstructed spectra in Fig. 9.
spheres target, represent a best-case reconstruction scenario for the mCTIS owing to an absence of overlap between spatial and spectral radiance on the focalplane array. Any error in the reconstructed spectra may be attributed to radiometric calibration errors. Figure 9 shows spectra corresponding to the four pixel locations indicated in Fig. 10. The RSE values were 9.6%, 9.9%, 8.0%, and 83% for Figs. 9~a!–9~d! @see Eq. ~2!#. Note that Fig. 9~d! shows the spectrum corresponding to a dark part of the scene, resulting in a small denominator in Eq. ~2! and thus a large value of the RSE. 5. Conclusions
We have described a spectral-imaging microscope capable of high speed, that is, 30 framesys and faster, acquisition of spatial and spectral data. Image data were collected in integration times of 16 ms. Preliminary imaging results with static targets consisting of fluorescing microspheres have been reported. The fluorescence emission spectra were sampled at a 10-nm interval between 420 and 710 nm. The smallest spatial sampling interval presented in this paper is 1.7 mm. Very good agreement was observed between the reconstructed spectra and the spectra measured with a nonimaging reference spectrometer. Further research will include ~a! reduction in the spatial sampling distance to 0.5 mm so as to form 100 3 100 pixel spectral images and ~b! improvements to the reconstruction algorithm to reduce reconstruction time and reconstruction artifacts. We also plan to conduct dynamic experiments with a video-frame-rate ~30 framesys! digital camera and pH-sensitive probes. With such a camera, raw images ~Fig. 3! will be acquired every 33 ms, thus approaching the goal of 10-ms temporal sampling. The authors thank Eustace L. Dereniak and Dennis Way for their support of and assistance with this research. We are also grateful for Stuart Biggar’s contribution of his time and resources. The research presented in this paper was funded in part by a California Institute of Technology President’s Fund
1. I. Newton, “Prop. V, Theorem IV,” Opticks ~Dover, New York, 1952!, p. 1704. 2. Y. Garini, N. Katzir, D. Cabib, R. A. Buckwald, D. G. Soenksen, and S. Malik, “Spectral bio-imaging,” Fluorescence Imaging Spectrometry and Microscopy, X. F. Wang and B. Herman, eds. ~Wiley, New York, 1996!, pp. 87–124. 3. J. M. Mooney, V. E. Vickers, A. Myoung, and A. K. Brodzik, “High-throughput hyperspectral infrared camera,” J. Opt. Soc. Am. A 14, 2951–2961 ~1997!. 4. E. S. Wachman, W. H. Niu, and D. L. Farkas, “Imaging acousto-optic tunable filter with 0.35-micrometer spatial resolution,” Appl. Opt. 35, 5220 –5226 ~1996!. 5. C. Hoyt, “Liquid crystal tunable filters clear the way for imaging multiprobe fluorescence,” Biophotonics International 4, 49 –51 ~1996!. 6. M. R. Descour, C. E. Volin, T. M. Gleeson, E. L. Dereniak, M. F. Hopkins, D. W. Wilson, and P. D. Maker, “Demonstration of a computed-tomography imaging spectrometer using a computer-generated hologram disperser,” Appl. Opt. 36, 3694 – 3698 ~1997!. 7. X. F. Wang and B. Herman, Fluorescence Imaging Spectrometry and Microscopy ~Wiley, New York, 1996!. 8. R. Y. Tsien and B. J. Bacskai, “Video-rate confocal microscopy,” Handbook of Biological Confocal Microscopy ~Plenum, New York, 1995!, pp. 459 – 478. 9. R. Martı´nez-Zaguila´n, M. W. Gurule´, and R. M. Lynch, “Simultaneous measurement of intracellular pH and Ca21 in insulin-secreting cells by spectral imaging microscopy,” Am. J. Physiol. Cell Physiol. 270 C1438 –C1446 ~1996!. 10. G. A. Wagnie`res, A. P. Studzinski, D. R. Braichotte, P. Monnier, C. Despeursinge, A. Chaˆtelain, and H. E. van den Bergh, “Clinical imaging fluorescence apparatus for the endoscopic photodetection of early cancers by use of Photofrin II,” Appl. Opt. 36, 5608 –5620 ~1997!. 11. R. Y. Tsien and A. Waggoner, “Fluorophores for confocal microscopy,” Handbook of Biological Confocal Microscopy ~Plenum, New York, 1995! pp. 267–279. 12. D. R. Sandison, R. M. Williams, K. S. Wells, J. Strickler, and W. W. Webb, “Quantitative fluorescence confocal laser scanning microscopy ~CLSM!,” Handbook of Biological Confocal Microscopy ~Plenum, New York, 1995!, pp. 39 –53. 13. M. R. Descour, C. E. Volin, E. L. Dereniak, K. J. Thome, A. B. Schumacher, D. W. Wilson, and P. D. Maker, “Demonstration of a high-speed nonscanning imaging spectrometer,” Opt. Lett. 22, 1271–1273 ~1997!. 14. W. H. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in C pp. 78 – 81 ~Cambridge U Press, N.Y., 1992!. 15. A. Lent, “A convergent algorithm for maximum entropy image restoration,” in Image Analysis and Evaluation, SPSE Conference Proceedings, R. Shaw, ed. ~Society of Photographic Scientists and Engineers, n.p.!, pp. 249 –257. 16. Fluoresbrite polychromatic microspheres ~6 mm!, Catalog no. 19111, Polysciences Inc., 400 Valley Road, Warrington, Pa. 18976. 17. S2000 Fiber Spectrometer, Ocean Optics, Inc., 380 Main Street, Dunedin, Fla. 34698. 18. Optronic Laboratories, 4632 36th Street, Orlando, Fla. 32811; 1000-W Spectral Irradiance Standard, courtesy of Stuart Biggar, University of Arizona Optical Sciences Center, Remote Sensing Group ~1998!. 19. Nikon, USA, 1300 Walt Whitman Road, Melville, N.Y. 117473064. 1 December 1998 y Vol. 37, No. 34 y APPLIED OPTICS
8119
