Structured illumination microscopy for super-resolution ... - Springer Link

2 downloads 0 Views 2MB Size Report
Mar 1, 2014 - et al. and the SIM for optical sectioning (SIM-OS) pro- posed by Neil et al. are the most popular ones in the research community of microscopy.
Chin. Sci. Bull. (2014) 59(12):1291–1307 DOI 10.1007/s11434-014-0181-1

csb.scichina.com www.springer.com/scp

Invited Review

Optics

Structured illumination microscopy for super-resolution and optical sectioning Dan Dan • Baoli Yao • Ming Lei

Received: 10 September 2013 / Accepted: 3 December 2013 / Published online: 1 March 2014  Science China Press and Springer-Verlag Berlin Heidelberg 2014

Abstract Optical microscopy plays an essential role in biological studies due to its capability and compatibility of non-contact, minimally invasive observation and measurement of live specimens. However, the conventional optical microscopy only has a spatial resolution about 200 nm due to the Abbe diffraction limit, and also lacks the ability of three-dimensional imaging. Super-resolution farfield optical microscopy based on special illumination schemes has been dramatically developed over the last decade. Among them, only the structured illumination microscopy (SIM) is of wide-field geometry that enables it easily compatible with the conventional optical microscope. In this article, the principle of SIM was introduced in terms of point spread function (PSF) and optical transform function (OTF) of the optical system. The SIM for super-resolution (SIM-SR) proposed by Gustafsson et al. and the SIM for optical sectioning (SIM-OS) proposed by Neil et al. are the most popular ones in the research community of microscopy. They have the same optical configuration, but with different data postprocessing algorithms. We mathematically described the basic theories for both of the SIMs, respectively, and presented some numerical simulations to show the effects of super-resolution and optical sectioning. Various approaches to generation of structured illumination patterns were reviewed. As an example, a SIM system based on DMDmodulation and LED-illumination was demonstrated. A lateral resolution of 90 nm was achieved with gold nanoparticles. The optical sectioning capability of the microscope

D. Dan  B. Yao (&)  M. Lei State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China e-mail: [email protected]

was demonstrated with Golgi-stained mouse brain neurons, and the sectioning strength of 930 nm was obtained. Keywords Microscopy  Structured illumination  Super-resolution  Optical sectioning

1 Introduction 1.1 Illumination for microscopy Illumination of the specimen is the most important variable in achieving high-quality images in microscopy. Ko¨hler illumination was first introduced in 1893 by Ko¨hler [1] as a method of providing the optimum specimen illumination. Since then, Ko¨hler illumination is the predominant technique for sample illumination in modern scientific light microscopy. It acts to generate an extremely uniform illumination of the sample and insures that an image of the illumination source (for example, a halogen lamp filament) is not visible in the resulting image, thus allowing the user to realize the microscope’s full potential. With the improvement and accuracy of lens manufacture, some kind of aberration and blur can be corrected to get an image of better quality. However, even all the optics elements are made perfectly, a resolution limit of about 200 nm still exists in microscopy. This is the so-called diffraction barrier or Abbe diffraction limit [2], its value equals to *0.5 k/NA, determined only by the wavelength of source light and the numerical aperture (NA) of objective lens. Moreover, the wide-field geometry nature of light microscopy leads to a certain depth-of-view on the focal plane. When a recording camera is put on the conjugated position of that focal plane, the in-focus and out-of-focus information are fused into a two-dimensional image. The

123

1292

targeted in-focus information cannot be separated from this fusion two-dimensional image directly. That is the reason why the conventional light microscopy cannot obtain optical sectioning, and the captured image is usually low both in contrast and signal-to-noise ratio (SNR).

1.2 Super-resolution fluorescence microscopy The invention of green fluorescence protein (GFP) opens the era of fluorescence microscopy [3]. Browsing through any cell biological journals, the impact of fluorescence microscopy is obvious, over 80 % of the images of cells in books are usually acquired with fluorescence microscopes [4]. Many new fluorescence proteins, chemicals (such as rhodamine, quantum dots, and so on) [5] and various labeling techniques have been emerged to specify the interested part of specimen [6]. Thanks to these fluorescence label methods, the contrast and SNR of image are greatly improved. Based on fluorescence microscopy, numbers of methods have been proposed to deal with the weaknesses of spatiotemporal resolution and optical sectioning in light microscopy, some aimed to one of them, some to both [7–10]. For example, in stimulated emission depletion (STED) microscopy [11, 12], a normal laser light is focused into a diffraction limited spot to excite the fluorophores to the higher energy state (i.e., the excited state) on the focal plane. At the same time, a dough-nut light called STED light is also focused and superposed with the former spot, in where of the overlapped region, the excited fluorophores are brought down to the lowest energy state (i.e., the ground state) by STED and only leaving the smaller center area to emit fluorescence signals. This reduces the size of the region of molecules that fluoresce, as if the ‘‘focal spot’’ of the microscope is sharpened. Scanning this sharpened spot across a sample then, allows a super-resolution image to be recorded. Thus, this approach elegantly generates an image without the need of any postacquisition processing. Another category of super-resolution method such as photoactivated localization microscopy (PALM) [13, 14] and stochastic optical reconstruction microscopy (STORM) [15–17] utilizes the ‘‘on–off’’ switchable characters of some specific fluorescence molecules. In practice, each time, sparse fluorophores are excited to ‘‘on’’ state to fluoresce. Because the positions of these fluorophores are far away enough for each other, each fluorophores can be precisely located to several nanometers by a localization algorithm. Then, those fluorophores are brought down to ‘‘off’’ state and no longer to be excited. After hundreds and thousands of time repeating, finally, a super-resolution image is reconstructed by using all the localized fluorescent points.

123

Chin. Sci. Bull. (2014) 59(12):1291–1307

1.3 Optical sectioning microscopy On the facet of optical sectioning, confocal microscope has gained popularity in the science and industry communities [18]. By means of point illumination and a spatial pinhole to eliminate out-of-focus light in specimens, optical resolution and contrast can be increased by this technology. After point scanning through the whole specimen both in lateral and axial, confocal microscope enables the reconstruction of three-dimensional structures from the acquired signals. As an alternative to confocal microscope, two or multi-photon excitation fluorescence microscopy [19, 20] allows for imaging of tissues with higher depth due to its deeper penetration and efficient light detection. Being a special variant of the multi-photon fluorescence microscope, two-photon microscope uses red-shifted excitation light (normally IR) to excite fluorescent dyes, during which two photons of the infrared light are absorbed simultaneously to the excited state. Due to the nonlinear threshold effect of the multi-photon absorption, the background signal is strongly suppressed. Using infrared light minimizes scattering in the tissue. Both effects lead to an increased penetration depth for the multi-photon excitation fluorescence microscopy. Selective/single plane illumination microscopy (SPIM) [21] or light-sheet microscopy (LSM) [22] is another fluorescence microscopic technique with an intermediate optical resolution, but good sectioning capabilities. In contrast to epi-fluorescence microscopy, only a thin slice (usually a few hundred nanometers to a few micrometers) of the sample is illuminated perpendicularly to the direction of observation. As only the actually observed section is illuminated, this method reduces the phototoxicity and stress induced on a living sample dramatically, and also the background signal is greatly reduced, resulting in an image with higher contrast, comparable to confocal microscopy. 1.4 Emerging of structured illumination microscopy The techniques described above, whether for super-resolution or for optical sectioning, are time-consuming and complex in configuration. STED, confocal, and two-photon microscopies require a point scanning process to work. For a point detector (such as APD or PMT), exposure time should long enough (usually [10 ls) to maintain an accepted SNR of imaging [23]. Meanwhile, the scanning steps between two adjacent points should be as short (usually less than the half size of resolution) as to follow the Nyquist sampling theorem. These two factors make the scanning process in low speed. On the other hand, an additional configuration is needed to trigger or feedback for synchronization during the whole scanning. It inevitably leads to a complex configuration with low stability. The

Chin. Sci. Bull. (2014) 59(12):1291–1307

pointillism methods of PALM and STORM require hundreds and thousands of raw images to accomplish a superresolution image. Therefore, time cost is also their drawback. Moreover, accuracy and speed must be trade-off due to the localization algorithm of fluorescence molecules. In order to overcome these shortcomings of above techniques, some scientists put insights into the illumination way in microscopy. In 1997, Neil et al. [24] used a structured illumination to obtain optical sectioning on a conventional microscope. In 2000, Gustafsson [25] proposed a structured illumination microscopy (SIM) to break the diffraction limit by a factor of two. These two SIMs are same in the way of illumination (sinusoidal fringe illumination) and phase-shifting method, while with different data processing algorithms. Over 10 years of development, structured illumination is no long limited to sinusoidal fringe pattern. It may be sparse speckles [26–29], chess-like [30, 31] or even random unknown patterns [32], depended on different purposes such as super-resolution, optical sectioning or range measurement. As an alternative to Ko¨hler illumination, SIM is essentially based on wide-field geometry, therefore, requires minimal modification on a conventional microscope with lower cost. It also remains all the advantages of conventional microscopy: (i) less complex optical configuration not requiring point-wise scanning modules; (ii) wide-field full-frame imaging with high recording speed; (iii) low light dose avoiding of harming the live specimens. 1.5 Development of SIM In this section, we concentrate on the SIM for super-resolution (SIM-SR) and optical sectioning. As far as we know, structured illumination has also wide applications in metrology. So, a brief description about this is also mentioned in the end. 1.5.1 SIM-SR In SIM-SR, a sinusoidal fringe patterned light is used to illuminate a sample [25]. When the sinusoidal fringe overlaps with the high spatial frequency component of sample, a new sparse pattern appears due to the wellknown ‘‘Moire´ pattern’’ phenomenon. The spatial frequency of Moire´ fringe is lower than both those of the structured illumination and the high-frequency components of sample. The Moire´ pattern encodes the unresolved structure of sample (i.e., the high-frequency components) into the ‘‘observable region’’ of the limited bandwidth of microscopic objective. Decoding the high-frequency information requires a three-step phase-shifting operation of the illumination pattern. In order to obtain a nearly isotropic frequency spectrum along all directions in lateral,

1293

multiple orientation, structured illumination (at least in two or three different orientations) is necessary. Finally, all the decoded high-frequency components are restored to their proper positions in the frequency spectrum, and then an enhanced resolution image beyond the diffraction limit can be obtained by making an inversed Fourier Transform for the enlarged frequency spectrum. Two similar techniques called laterally modulated excitation microscopy (LMEM) and harmonic excitation light microscopy (HELM) were proposed by Heintzmann and Cremer [33] in 1999 and Frohn et al. [30, 31] in 2000, respectively. It should be noticed that the recently proposed image scanning microscopy (ISM) [34, 35] and the SIM with unknown patterns [32] are based on different super-resolved principles from the SIM-SR described here. In practice, the generation of structured illumination and data post-processing are the two essential issues in SIM. The increased resolution attained by SIM is based on the degree to which high spatial frequencies can be down converted into the passband of the imaging system. To effectively do this, a high contrast high-frequency illumination pattern is required. With a laser source, interference by two or multiple beams is a common way to get the required structured patterns [36, 37]. A grating is often used to diffract incident light into ?1st order and -1st order beams, which will interferer each other to form the sinusoidal fringe pattern [38]. Considering the speed and accuracy in phase-shifting, gratings are replaced by SLM [39–43] (LC-SLM or Ferro-SLM), DOE [44] or hole-arrays [45] with no moving parts in optical setup, to maintain the stability of system and improve the recording speed. In use of incoherent light source (e.g., LED), pattern projection with a structured mask or a modulated DMD chip through objective lens allows a compact system without coherent speckle-noise [46]. In post-processing field, lots of researches focus on optical transfer function (OTF) compensation [47–50], estimation of phase-shifting [51–53], algorithm efficiency [54, 55], and aberration correction [56, 57] to enhance the image quality (SNR, contrast, and resolution). OTF compensation extends the concept of the conventional transfer function by incorporating noise statistics, thus giving a measure of the SNR at each spatial frequency. Since the knowledge of phase-shift in the structured illumination is critical for the data post-processing, any inaccuracy in this knowledge will lead to artifacts in the reconstruction. Estimation of phase-shifting is necessary to insure the accuracy. New algorithms [54, 55] and general programing based on parallel GPU framework are proposed to reduce the data processing time [58]. Adaptive optics models are used to deal with the aberrations leading by the high NA objective lens [56, 57]. Based on the SIM-SR principle, spatial resolution enhancement in all three dimensions (3D) can be carried

123

1294

out by a 3D structured illumination (3D-SIM) [59, 60]. A resolution of 100 nm has been reported with 3D-SIM both in lateral and axial directions [42, 59, 61]. Another choice to get such 3D super-resolution is to incorporate an axial super-resolution scheme (for example, TIRF) with the lateral SIM [42, 62]. Since the spatial frequency of Moire´ fringe is also limited by the passband of the optical system, the maximum resolution extension of SIM is only twofold over the conventional microscopy. Further improvement of resolution can be achieved by exploiting the nonlinear response of fluorescence molecules with saturated excitation, which is called saturated structured illumination microscopy (SSIM) [63–65]. The practical resolving power is determined by the SNR, which in turn is limited by photo-bleaching [66]. A 2D point resolution of 50 nm is possible on sufficiently bright and photo-stable samples [63]. Recently, cellular structure at resolution of 50 nm is revealed with a photo-switchable protein [67, 68]. SIM for super-resolution is compatible with various microscopies, such as LSM [69–71], imaging interferometric microscopy [72, 73], line-scanning microscopy [74], 4Pi microscopy [75], differential interference contrast (DIC) [76], and so forth to extend their functionalities, thus has broad applications in biology [40, 61, 77–82], medicine [83], fluidics [84], nano-morphology [85], and nano-particles [86]. Now, there are numbers of commercial SIM-SR instruments available in market, e.g., DeltaVision from GE Healthcare, N-SIM form Nikon Inc. and ELYRA S.1 from Carl Zeiss Inc. 1.5.2 SIM-OS The basic idea of SIM for optical sectioning (SIM-OS) is to extract the in-focus information and reject the background of out-of-focus portion of image by structured illumination. It is mainly classified into two categories: speckle illumination and grid patterns illumination. Both kinds of microscopies are based on geometry of wide-field and aimed at imaging thick samples with high speed, large field-of-view. Speckle-illuminated fluorescence microscopy proposed by Walker et al. [26, 87–89] is a non-scanning imaging technique that uses a random time-varying laser speckle pattern to illuminate the specimen, and then record a large number of wide-field fluorescence frames and a sequence of corresponding illumination speckle patterns, or reference speckle patterns with a CCD camera. Then, the recorded frames are processed by an averaging formula over the set of recorded frames. Ventalon et al. [90–95] have demonstrated a technique called dynamic speckle illumination microscopy (DSI) that produces fluorescence sectioning with illumination patterns that are neither well defined nor controlled. The idea of this technique is to illuminate a fluorescent sample with random speckle patterns obtained from a laser. Speckle

123

Chin. Sci. Bull. (2014) 59(12):1291–1307

patterns are granular intensity patterns that exhibit inherently high contrast. Fluorescence images obtained with speckle illumination are, therefore, also granular, however, the contrast of the observed granularity provides a measure of how in focus the sample is: high observed contrast indicates that the sample is dominantly in focus, whereas low observed contrast indicates it is dominantly out-of-focus. The observed speckle contrast thus serves as a weighting function indicating the in-focus to out-of-focus ratio in a fluorescence image. While the exact intensity pattern incident on a sample is not known, the statistics of the intensity distribution are well known to obey a negative exponential probability distribution. According to this distribution, the contrast of a speckle pattern scales with average illumination intensity. Thus, weighting a fluorescence image by the observed speckle contrast is equivalent to weighting it by the average illumination intensity (as in standard imaging), however, with the benefit that the weighting preferentially extracts only in-focus signal. Generally, the speckle illumination microscopy needs tens or hundreds of speckle-illuminated raw images varied in space and time to obtain a sectioning image by algorithms in statistics. The drawback is obvious in time–cost. In fact, as early as 1997, Neil et al. [24, 96] proposed a grid pattern illumination microscopy with only three phaseshifted raw images to reduce the recording time on a conventional microscope. This method is based on the fact that under a grid pattern illumination with high spatial frequency, only the in-focus part of image is modulated, while the out-of-focus region is out of modulation. This is because that the high spatial frequency component attenuates very fast with defocus. The out-of-focus signal cannot carry on the high-frequency illumination patterns. The following step is to eliminate the residual patterns in modulated infocus signal, usually adopting a three-step phase-shifting method. Since Neil’s approach has the same structured illumination pattern as SIM-SR, sometimes it also refers to SIM in many journals. In order not to be confused, we refer this approach to Neil-SIM in the following sections. The attractive feature of Neil-SIM is its simplicity in system configuration and data post-processing algorithm. A grating is employed as a plug-in module to a conventional microscope to produce fringe pattern illumination. Meanwhile algorithm is just based on simple elementary operations to three raw images. The structured illumination pattern must be known in prior and the movement (phaseshifting) of the illumination pattern should be accurate. It is a challenge for the grating approach due to the hard movement in precision with stability. Fortunately, the grating can be replaced by various programmable spatial light modulators (SLM) with no moving parts to insure the phase-shifting both in accuracy and speed (as described above in SIM-SR) [97–102]. Moreover, the phase-shifting

Chin. Sci. Bull. (2014) 59(12):1291–1307

process can be avoided by color-coded or polarizationcoded SIM, which allow for providing optical sectioning in a single-shot [103, 104]. In theoretical scope, studies are mostly concentrated on the aspects of image formation [105, 106], quantitative analysis of sectioning and noise [107, 108], optimization and characterization for the microscopy [109, 110]. Combining Neil-SIM with other imaging techniques is of interest in a wide range of research fields, such as fluorescence lifetime imaging microscopy (FLIM) [111– 114], image mapping spectrometer (IMS) [115], optical tomography [116–118], plane illumination microscopy (light-sheet microscopy) [119–121], two-photon excitation microscopy [122], spectral imaging [112], and holography [123]. So far, Neil-SIM has found variety of applications in endomicroscopy [98, 124, 125], multiple-scattering suppression [126], visualization of gas flow [127], cell biology [128], hydrocarbon-bearing fluid inclusions [129], retinal imaging [130], and so on. Nowadays, there are several types of commercial Neil-SIM apparatus available in market, such as ApoTome (Carl Zeiss Inc.), Optigrid M (Olympus Inc.), and OptiGrid module (which can be integrated into specific microscopes from Leica Inc.) 1.5.3 SI-SP Besides utilization in microscopy, structured illumination strategy is also popular in the community of 3D surface measurement for relative large objects [131]. Projecting a narrow band of light onto a three-dimensional shaped surface produces a line of illumination that appears distorted from other perspectives than that of the projector, and can be used for an exact geometric reconstruction of the surface shape.

1295

The structured illumination for surface profilometry (SI-SP) is of great attraction in the domain of range measurement [132], 3D imaging [133], and pattern recognition [134]. It has been widespread used in product inspection, driver assistant system in cars, stereo imaging, automation, and so on.

2 Theoretical basis of SIM 2.1 Linear SIM-SR Microscope, as an imaging system, can be treated and analyzed by the theory of signals and systems which is currently widely used in the field of electronics. In most cases, microscope is a time invariant and linear system. According to the principle of linear signal system, the characteristic of the whole system is determined by the impulse response, that is, once the impulse response function is given, the output of the system can be calculated by the convolution of the input and the impulse response function. In a microscope, the point spread function (PSF) describes the impulse response function to an ideal point source or a point object. It maps the emission distribution of the sample to the intensity distribution on the detector in fluorescence microscopy. Thus, an image of any objects is a convolution of the emission intensity and the PSF. Meanwhile, in the spatial frequency domain, the optical transform function (OTF), which is the Fourier transform of PSF, indicates the allowed passband and the weighted modulation of each frequency of the emission intensity distribution. This basic knowledge on the image formation in a microscope is illustrated with Fig. 1. In mathematics, the image formation is expressed as

Fig. 1 Schematic of image formation in real space and in spatial frequency domain. The image in real space is a process of convolution between the object and the PSF of microscope. In spatial frequency domain, this process corresponds to a multiplication of the spatial spectrum of object and the OTF, which is the Fourier transform of PSF

123

1296

Chin. Sci. Bull. (2014) 59(12):1291–1307

DðxÞ ¼ ½Em ðxÞ  PSFðxÞ;

ð1Þ

where D(x) is the distribution of photons on the detector, Em(x) is the emission distribution of the sample, PSF(x) is the PSF of the microscope, x indicates the space coordinate component, and ˜ denotes a convolution operation. Making a Fourier transform to Eq. (1) and using the convolution theorem, an expression is obtained to describe the spatial frequencies accessible to the microscope 



DðkÞ ¼ ½Em ðkÞ  OTFðkÞ;

ð2Þ

where the tilde denotes the function after Fourier transform, OTF is the Fourier transform of the PSF, and k is the corresponding spatial frequency component. In a conventional fluorescence microscope, the fluorescence emission intensity Em(x) is linearly dependent on the illumination intensity I(x) and the fluorescence yield rate of the dye-labeled sample. Actually, the distribution of fluorescence yield rate across the sample indicates the structure of the sample S(x) Em ðxÞ ¼ IðxÞ  SðxÞ;

ð3Þ













ð5Þ

In the case of uniform illumination, which is the situation of conventional fluorescence microscopy, I(x) = constant, we have 



DðkÞ ¼ S ðkÞ  OTFðkÞ:

ð6Þ

This means all spatial frequencies of the sample are confined by the OTF. If the illumination is not uniform, for example, being one-dimensional sinusoidal structured, I(x) is expressed in IðxÞ ¼ I0 ½1 þ m cosð2p k0 xþuÞ;

ð7Þ

where k0 and u denote the spatial frequency and the initial phase of the sinusoidal fringe illumination pattern, respectively. I0 and m are constants called mean intensity and modulation depth, respectively. By making Fourier transform, Eq. (7) can be expressed in frequency space as 

I ðkÞ ¼ I0 ½dðkÞ þ

m iu m  e dðk  k0 Þ þ  eiu dðk þ k0 Þ: 2 2

ð8Þ

Substituting Eq. (8) into (5), the frequency spectrum of the observed image is presented by

123



 OTFðkÞ: ð4Þ

Substituting Eq. (4) into (2), the frequency spectrum of the observed image is presented in the form DðkÞ ¼ ½ I ðkÞ  S ðkÞ  OTFðkÞ:



DðkÞ ¼ I0 ½ S ðkÞ þ

or expressed in the frequency domain by Em ðkÞ ¼ I ðkÞ  S ðkÞ:

Fig. 2 Principle of spatial frequency spectrum extension in singleorientation for linear SIM-SR. fc is the cut-off frequency of OTF, and k0 is the spatial frequency of the sinusoidal fringe illumination pattern. The arrows in (a) point out the central frequency of the spectrum. The extended frequency spectrum assembled by those three parts in (a) is shown in (b)  m iu  m e S ðk  k0 Þ þ eiu S ðk þ k0 Þ 2 2

ð9Þ

The first term in Eq. (9) represents the normal frequency spectrum observed by the conventional microscope, in which the value scope of k is |k| B fc, where fc = 2 NA/k is the cutoff frequency of OTF, NA is the numerical aperture of objective lens and k is the wavelength. The second and third terms in Eq. (9) contribute additional information. They have the same form as the first term, but the central frequencies are shifted by ?k0 and -k0, respectively, and k satisfy |k - k0| B fc and |k ? k0| B fc, respectively. Thus, the value scope of k is extended to |k| B |fc ? k0|. Because the maximum value of k0 can reach to fc, the maximum spectral bandwidth for linear SIM will be doubled over the conventional microscope. This is the reason why the resolution of the linear SIM-SR is able to be enhanced theoretically by a factor of two. To obtain the three terms of frequency spectra in Eq. (9), three different phases u1, u2, and u3 for the illumination pattern can be set to get the following three independent linear equations, provided that m and OTF(k) are pre-known: 0  1 1 0 10  D1 ðkÞ S ðkÞ 1 m2 eiu1 m2 eiu1 B  C  C B D ðkÞ C ¼ I0 @ 1 m eiu2 m eiu2 AB @ S ðk  k0 Þ A 2 2 @ 2 A   1 m2 eiu3 m2 eiu3 S ðk þ k0 Þ D3 ðkÞ  OTFðkÞ; ð10Þ ~ ~  k0 Þ; and Sðk ~ þ k0 Þ The resultant spectral terms SðkÞ; Sðk solved from Eq. (10) consist of the separated frequency spectra of the sample, as illustrated in Fig. 2a. Taking the overlapped

Chin. Sci. Bull. (2014) 59(12):1291–1307

1297

Fig. 3 The procedure of SIM for super-resolution in horizontal and vertical directions. Two sets of sinusoidal fringe patterns along horizontal and vertical directions are multiplied by the object, respectively. Each set contains three phase-shifted images by step of 2p/3. After convolution with PSF, the raw data of SIM are obtained. By making Fourier transform and using Eq. (10), five frequency spectra are solved and allocated to their proper positions to form a combined and enlarged frequency spectrum in the frequency space, which means an improved resolution in the real space after making an inverse Fourier transform

frequency region and the central frequencies shifted by ?k0 and -k0 into account, we can assemble these three parts together to form an extended frequency spectrum of the sample, as shown in Fig. 2b. To obtain nearly isotropic resolution enhancement, the single-orientation sinusoidal fringe pattern should be rotated in at least two orthogonal orientations. A simulation of SIM-SR is conducted in the following procedure illustrated in Fig. 3. Two sets of sinusoidal fringe illuminated images are used to simulate the structured illumination in horizontal and vertical orientations, respectively. Each set of images contains three images representing the structured illumination in three phases at a gap of 2p/3. Then, each simulated structured illumination image is multiplied by a sample image to achieve the fluorescence emission process under different orientations and phases. Convoluted with PSF of the microscope, the corresponding two sets of simulated recording images can be obtained. The next step is to transform these raw images into frequency space through fast Fourier transform operation (FFT). For two-orientation illumination, according to Eq. (10), five frequency spectra can be solved and allocated to their proper positions in the frequency space. Finally, a combined and enlarged frequency spectrum of the sample is obtained. Through IFFT (Inverse FFT), a super-resolution image can be reconstructed in the real space. 2.2 Nonlinear SIM-SR In above discussion, the precondition is that the emission intensity Em(x) is linearly proportional to the illumination

intensity I(x). If there is a nonlinear fluorescence effect, such as two-photon absorption, stimulated emission, and ground-state depletion, the Eq. (3) will become: Em ðxÞ ¼ F½IðxÞSðxÞ:

ð11Þ

Here, F is a function of the illumination intensity, which depends on the nonlinear type of the sample in response to the illumination light. Normally, it can be expanded as a power series of the intensity with infinite number of terms F½IðxÞ ¼ a0 þ a1 IðxÞ þ a2 I 2 ðxÞ þ a3 I 3 ðxÞ þ   :

ð12Þ

Substitute Eq. (7) into (12) and making a mathematical derivation, we obtain F½IðxÞ ¼

1 X

bn cos½nð2pk0 x þ uÞ;

ð13Þ

n¼0

where bn (n = 0, 1, 2,….) are constants. After transforming Eq. (13) into frequency domain and substituting the result into Eq. (5), we find that the spectrum takes the form of weighted sum of infinite number of components: 

DðkÞ ¼ OTFðkÞ

1 X



bn S ðk  nk0 Þeinu :

ð14Þ

n¼1

Comparing this equation to the linear Eq. (9), we can see that all frequency components from both the conventional OTF and the extended linear structured illumination OTF are involved when taking the terms of n = -1, 0 and 1. Importantly, Eq. (14) also contains infinite number of higher-order harmonics of the fundamental frequency k0. In practice, only a few numbers, N, of the higher-order

123

1298

frequency components are able to be extracted with high enough SNR. Similar to the separation of the linear structured illumination raw data, it is possible to separate the higher-order frequency components if (2N ? 1) images are taken at different phases of the illumination pattern. Again, the single-orientation sinusoidal fringe pattern needs to be rotated in a sufficient number of orientations to fill the entire two-dimensional space isotropically. The simulation process of nonlinear SIM-SR is similar to the linear SIM-SR. What difference is the increased number of shifted phases and the number of separated high-frequency components. Figure 4 illustrates the more extended frequency spectrum from the conventional microscopy to the linear SIM-SR, and then to the nonlinear SIM-SR. 2.3 SIM for optical sectioning On conventional wide-field microscope, the targeted infocus information is always merged with an unwanted outof-focus information, which degrades the imaging quality severely. Optical sectioning is a process of distinguishing the in-focus information out of the merged data. Good optical sectioning, often referred to as minimal depth or z resolution, allows the three-dimensional reconstruction of a sample from images captured at different focal planes. To understand the principle of SIM-OS, we proceed from the discussion on the permitted spatial frequency intensity distribution with defocus on the conventional microscope. The normalized spatial frequency intensity distribution H(z, t) can be expressed in the following formula with the Stokseth’s approximation [24]   J1 ½4utð1  tÞ ; Hðz; tÞ ¼ 2f ðtÞ ð15Þ 4utð1  tÞ 

Chin. Sci. Bull. (2014) 59(12):1291–1307

With

u ¼ 8ðp=kÞsin2 ða=2Þ  z:

ð16Þ

Here J1() represents the first-order Bessel function, f(t) = 1–1.38t ? 0.0304t2 ? 0.344t3. t is the normalized spatial frequency, whose original value falls in the scope of [0, fc], fc = 2NA/k. fc is the cut-off frequency, NA = nsina (n is refraction index, a is the half aperture angle of the objective lens) is the numerical aperture, and k denotes the wavelength. z is the defocus variable, i.e., the axial range apart from the focal plane (z = 0). In the case of t = 0, H (z, t) gets its maximum value of Hmax = 1. In the limit of t = 1, H (z, t) reaches the minimum value of Hmin = 0. Figure 5a, b show the normalized spatial frequency intensity distribution with respect to the defocused range and the normalized spatial frequency according to Eq. (15). Figure 5c indicates the attenuation curve of H (z, t) with respect to t in the focal plane (z = 0). It can be seen that the intensity of the higher frequency component attenuates much rapidly than the lower frequency component with increasing of the defocus. So, if a proper high-frequency sinusoidal fringe light illuminates the sample, it will happens that only the infocus part but not the out-of-focus part will be modulated by the structured illumination. Generally, the higher frequency of illumination will results in thinner depth of optical sectioning. But considering the modulation depth and SNR will degrade with increasing of the illumination frequency, a trade-off should be taken to employ a fringe illumination with proper spatial frequency in practice. In order to specify the optical sectioning capability of Neil-SIM under a fringe illumination with a fixed spatial frequency t, the full width at half maximum (FWHM) of the H (z, t) curve is defined as the optical sectioning strength. According to Eq. (15), the FWHM can be obtained by solving Eq. (17) to get two roots z1, z2, and then FWHM = |z1 - z2|.

Fig. 4 Schematic of frequency spectrum extension from linear SIM-SR to nonlinear SIM-SR. a Frequency spectrum for conventional microscopy; b frequency spectrum for linear SIM-SR in one-orientation extension; c frequency spectrum for linear SIM-SR in four-orientation extension; d frequency spectrum for nonlinear SIM-SR in four-orientation extension

123

Chin. Sci. Bull. (2014) 59(12):1291–1307

1299

Fig. 5 (Color online) The normalized spatial frequency intensity distribution of a microscope with defocus. a, b show the normalized spatial frequency intensity distribution with respect to the defocused range and the normalized spatial frequency in 3D and 2D graphs, respectively. c indicates the attenuation curve of H(z, t) with respect to t in the focal plane (z = 0). d shows the H(z, t) profile with respect to the defocused range at t = 0.5, and the full width at half maximum (FWHM) of the curve is calculated at the given parameters, which measures the optical sectioning strength of the SIM

  J1 ½4utð1  tÞ 1 ¼ ; 2 4utð1  tÞ  2 FWHM ¼

0:04407k : tð1  tÞsin2 ½0:5arcsin(NA=nÞ

ð17Þ ð18Þ

As an example, the intensity profile at t = 0.5 is plotted in Fig. 5d, and the FWHM is calculated to be 0.97 lm under the conditions of k = 0.55 lm, NA = 0.6, which means the optical sectioning strength in this case is 0.97 lm. From formula (18), the dependence of optical sectioning strength FWHM on the normalized spatial frequency and the NA is plotted in Fig. 6. It can be seen that despite what NA is adopted, there exists an optimal normalized spatial frequency t = 0.5, at which the FWHM gets the minimum value. Of course, higher NA will give

narrow FWHM. This means that at a given NA the optimal spatial frequency of the illumination pattern is half of the cut-off frequency to obtain the best optical sectioning effect. Benefiting from the defocus feature of the high spatial frequency illumination near the focal plane, we can simply decompose the merged imaging data into two parts, i.e., the in-focus component Din(x,y) and the out-of-focus component Dout(x,y). Then, the leaving thing is how to eliminate the residuary fringe in the captured images to extract the wanted in-focus information. The distribution of photons on the detector in a conventional wide-filed microscope can be expressed in the form of Dðx; yÞ ¼ Dout ðx; yÞ þ Din ðx; yÞ:

ð19Þ

123

1300

Chin. Sci. Bull. (2014) 59(12):1291–1307

Meanwhile, the conventional wide-field image can also be obtained with 1 Dout ðx; yÞ þ Din ðx; yÞ ¼ ½D0 ðx; yÞ þ D120 ðx; yÞ 3 þ D240 ðx; yÞ:

Fig. 6 The dependence of optical sectioning strength (FWHM) on the normalized spatial frequency and the numerical aperture (NA). High NA gets narrow FWHM. Whatever NA is adopted, there exists an optimal normalized spatial frequency t = 0.5, at which the FWHM gets the minimum value

Under a one-dimensional sinusoidal structured illumination in the form of Eq. (7), only Din(x, y) is modulated while Dout(x, y) is not. For simplicity, assuming I0 = 1 and m = 1, D(x, y) can be rewritten as

ð25Þ

This means that the conventional wide-field image and the optical sectioning SIM image can be obtained simultaneously from the same data. In the following, we simulate the procedure of Neil-SIM to show the optical sectioning effect. The simulated sample is shown in Fig. 7a. Figure 7b gives a defocus model used to simulate the spatial frequency attenuating with defocus in the conventional wide-field microscopy. The image is tilted so that it becomes increasingly out-of-focus toward the bottom. After three-step phase-shifting with SIM (one of the three images is shown in Fig. 7c), the optical sectioning result using the phase-shifting algorithm is shown in Fig. 7d. Due to an increased blurring toward the bottom, much of the defocused components of the image are eliminated and are dim, only the in-focus part is remained and is bright.

3 Generation of structured illumination patterns

To solve the targeted Din(x, y), three-step phase-shifting with interval of 2p/3 is usually used to get the following equations

The generation of high frequency and high contrast of structured illumination patterns is essential to obtain the best quantity of images for both SIM-SR and Neil-SIM. In the following, we will present some approaches to generation of the structured illumination and their virtues and defects.

D0 ðx; yÞ ¼ Dout ðx; yÞ þ Din ðx; yÞ  ½1 þ cosð2pk0 x þ uÞ;

3.1 Interference

Dðx; yÞ ¼ Dout ðx; yÞ þ Din ðx; yÞ  ½1 þ cosð2pk0 x þ uÞ: ð20Þ

ð21Þ D120 ðx; yÞ ¼ Dout ðx; yÞ þ Din ðx; yÞ  ½1 þ cosð2pk0 x þ u 2p þ Þ; 3 ð22Þ D240 ðx; yÞ ¼ Dout ðx; yÞ þ Din ðx; yÞ  ½1 þ cosð2pk0 x þ u 4p þ Þ: 3 ð23Þ Then, Din(x, y) is solved as

It is well known that interference of two or multiple laser beams can form one-, two-, or three-dimensional structured light patterns [36, 37]. In the most usual case, two coherent plane waves, normally formed by beamsplitters, can generate a one-dimensional sinusoidal fringe pattern. The spatial frequency of the fringe is determined by the intersection angle of the two wave vectors, i.e., k0 = 2nsin(a)/k, where a is the half intersection angle, k is the wavelength, and n is the refractive index of surrounding medium. The phase of the fringe can be controlled by adjusting the

pffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 Din ðx; yÞ ¼ ½ðD120 ðx; yÞ  D0 ðx; yÞ2 þ ½ðD240 ðx; yÞ  D120 ðx; yÞ2 þ ½ðD240 ðx; yÞ  D0 ðx; yÞ2 : 3

123

ð24Þ

Chin. Sci. Bull. (2014) 59(12):1291–1307

1301

Fig. 7 Simulation of SIM for optical sectioning effect. a Sample image in a single plane. b The image is tilted from the bottom using a defocus algorithm so that it becomes increasingly out-of-focus toward the bottom. c The tilted sample image under structured illumination; d SIM optical sectioning image where only the in-focus part of the sample is retained

relative optical path difference of the two beams. For this approach, the mechanical stability is a major concern, because the relative length of the two beam paths formed by beamsplitters must remain stable to sub-wavelength precision during the time needed to record an entire set of images. When working with microscopes that possess high aperture angles in the sample plane, it is important to keep the vector nature of light in mind. Beams converging onto the focus plane at an angle of 90 with respect to each other do not generate an intensity interference pattern in p-polarization (i.e., polarization vector is in the plane defined by the two interfering beams), whereas the interference of s-polarization (vector perpendicular to the plane that contains the two interfering beams) results in an optimal fringe contrast [135]. The illuminating light must be a coherent laser source.

(usually possess larger power than other orders) to interfere for producing the sinusoidal fringe pattern through objective lens. The phase-shift of the fringe can be implemented easily by moving the grating in the grating vector direction. To make flexible structured illumination patterns with fast speed of phase-shifting and fringe orientation rotation, the state-of-art programmable SLM is broadly used as a dynamic diffractive optical element instead of the etched grating. There are a few types of SLMs in the market, such as liquid crystal SLM, liquid crystal on silicon (LCOS), digital micromirror device (DMD), etc. The best choice for diffraction grating is to use the pure phase-modulation SLM (e.g., LCOS), which is able to obtain the highest diffraction efficiency, thus increasing the utilizing efficiency of the illumination power.

3.2 Diffraction and interference

Placing an illumination mask in the path of where light comes out from the source is a direct approach to generation of structured illumination. Through lenses (including objective lens), the structured illumination light can be focused into a small size due to the magnification of lenses. The mask can be a real grating or a programmable SLM, either of which works in transmission or reflection modes. The choice of light source is not limited by a coherent or an incoherent one, just with consideration of the fluorescence excitation wavelength. The usage of incoherent illumination light, which can completely suppress the speckle-noise occurred in coherent illumination, is a good merit for choice. Simple arrangement on an available conventional microscope is the best advantage of fringe projection method. ApoTome from Carl Zeiss Inc. and OptiGrid module from Leica Inc. are such devices. As a plug-in module, a grating is employed to generate structured illumination upon various conventional upright microscopes. However, a trade-off in precision and speed is inevitable

As described above, beamsplitters can divide one beam into two or multiple coherent beams. But the drawback is the mechanical stability due to the distributed arrangement of the optical elements. As we know, gratings can diffract incident beams into multiple order beams that have relatively fixed phases and intensity ratios. Because the diffracted order beams pass through the same optical elements, the influence of the environmental disturbance to the optical path difference among them can be reduced to minimum. Thus, the mechanical stability can be improved a lot by using gratings instead of beamsplitters. For example, a one-dimensional grating, which diffracts the incident beam into multiple order beams in the grating vector direction, can be used for generation of onedimensional structured illumination. To achieve the maximum degree of modulation, it is often to block the zeroorder and other higher diffraction orders in the optical path to leave only the ?1st order and the -1st order beams

3.3 Projection

123

1302

due to the movement of grating in phase-shifting. Usually, the phase-shifting must be in the precision of sub-wavelength. The mechanically slow movement speed harms the imaging speed in real-time and also may lead to fluorescence photo-bleaching and phototoxicity. This trade-off problem can be solved by using the programmable SLM with no moving parts, which will be shown in the next section in detail.

4 DMD-based LED-illumination SIM As a concrete example, in this section we present an approach of SIM by using a DMD for fringe projection and a low-coherence LED light for illumination [46]. A lateral resolution of 90 nm is achieved with gold nano-particles samples. The optical sectioning capability of the microscope is demonstrated with Golgi-stained mouse brain neurons, and the sectioning strength of 930 nm is obtained. The maximum acquisition speed for 3D imaging in the optical sectioning mode is 1.6 9 107 pixels/s, which is mainly limited by the sensitivity and speed of the CCD camera. In contrast to other SIM techniques, the DMDbased LED-illumination SIM is cost-effective, ease of multi-wavelength switchable, and speckle-noise-free. The 2D super-resolution and 3D optical sectioning modalities can be easily switched and applied to either fluorescent or non-fluorescent specimens. 4.1 System configuration The DMD-based LED-illumination SIM system is shown in Fig. 8. High brightness LEDs with switchable wavelengths of 365, 405, 450, and 530 nm serve as excitation sources, which allow multi-wavelength excitation. A total internal reflection (TIR) prism is employed to separate the illumination and the projection paths in minimal space. Light modulated by the DMD is reflected by a dichroic beamsplitter into the back aperture of a 1009 objective or a 209 objective. The demagnification of the projection system is 176 for the 1009 objective. The sample is mounted on a manual XY and Z-axis motorized translation stage that can be moved axially in minimum step of 50 nm. A CCD camera (1,024 9 768 pixels with pixel size of (4.65 9 4.65) lm2) is employed to capture the 2D image with a zoom lens, which produces a variable magnification of the image. The CCD camera has a maximum full-frame rate of 27 fps at its minimal exposure time. A long-pass filter is inserted in front of the zoom lens to block the excitation beam for fluorescence imaging. Automated data collection, DMD patterns generation, and motorized stage movement are implemented by custom software.

123

Chin. Sci. Bull. (2014) 59(12):1291–1307

4.2 Fringe projection by DMD A DLP Discovery D4100 kit from Texas Instruments Inc. is used as an amplitude-only SLM for the fringe projection. This DMD chip can switch binary images of two grayscale levels 0 and 255 up to 32 kHz with a resolution of 1,024 9 768 pixels with the pixel size of (10.8 9 10.8) lm2. In common use of DMD fringe projection, the generation of a sinusoidal fringe of 256 grayscale levels requires at least 9 pixels for one period. This will apparently reduce the carrier frequency of the fringe and also limit the refreshing rate to 100 Hz. To solve this problem, a binary grating with a period of only 4 pixels is applied in our scheme, which enables fast transfer of patterns to the DMD up to 32 kHz refreshing rate. A sinusoidal intensity distribution projected in the focal plane of the objective is naturally obtained, because the objective lens automatically blocks the higher order spatial frequencies of the binary grating due to its limited bandwidth. 4.3 Imaging process and data analysis The Fourier-based image reconstruction is processed according to the method of Gustafsson [25]. In the fringe projection by DMD, the commonly used three-step phaseshifting in interval of 2p/3 cannot be performed, because only four pixels per period of binary pattern are used. Therefore, an alternate three-step phase-shifting in interval of p/2 is adopted according to the same principle as described in Sect. 2.3. Here, for each slice, three subimages (denoted by D0, D90, and D180) with a phase-shift of p/2 between two adjacent images are acquired. After the recording process, each image triple is then processed according to Eq. (26) to obtain the sectioned images qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 Din ðx; yÞ ¼ ð2D90  D0  D180 Þ2 þ ðD180  D0 Þ2 : 2 ð26Þ The FWHM of gold nano-particles are measured by fitting their intensity distributions to a Gaussian function with the ImageJ plug-in (PSF estimation tool, computational biophysics lab, ETH Zurich, Switzerland). For optical sectioning, the specimen is scanned axially by the motorized stage. 4.4 Super-resolution imaging To evaluate the spatial resolution of the DMD-based SIM microscope, gold nano-particles (80 nm-diameter) deposited on 170 lm-thick coverslips are used as test samples. The Abbe resolution limit is *151 nm calculated for the 1009 objective of NA = 1.49 at the wavelength of 450 nm. The FWHMs are measured and averaged from 50

Chin. Sci. Bull. (2014) 59(12):1291–1307

1303

Fig. 8 Schematic diagram of the DMD-based LED-illumination SIM microscope. A low-coherence LED light is introduced to the DMD via a TIR-Prism. The binary fringe pattern on DMD is demagnified and projected onto the specimen through a collimating lens and a microscope objective lens. Higher orders of spatial frequencies of the binary fringe are naturally blocked by the optics, leading to a sinusoidal fringe illumination in the sample plane. Fluorescence or scattered light from the specimen is directly imaged onto the CCD camera. Inset (a) shows the principle of the TIR-Prism. The illumination light enters the prism and reflects through total internal reflection (TIR) within the prism and illuminates the DMD. Each micromirror of the DMD has an ‘‘on’’ position of ?12 and an ‘‘off’’ position of -12 from the normal of the DMD. As a result, the illumination light is introduced at 24 from the normal so that the mirrors reflect the light at 0 when working at the ‘‘on’’ state. Inset (b) gives the configuration of the system, including the LED light guide (), the DMD unit (`), the sample stage (´), and the CCD camera (ˆ). Reprinted with permission from [46], Copyright 2013, Nature Publishing Group

Fig. 9 Determination of the lateral SIM resolution using 80 nm-diameter gold nano-particles. As a comparison, image a is captured in the conventional illumination mode, and b is a SIM image. Zooms are shown in c, d. The line-scans of a selected nano-particle in c. d are plotted in e. The experiment was performed with the 1009 objective of NA = 1.49 at a LED wavelength of 450 nm. Reprinted with permission from [46], Copyright 2013, Nature Publishing Group

gold nano-particles. The resultant FWHM for the SIM is (90 ± 10) nm, which is much better than that of the conventional microscopy, i.e., (225 ± 5) nm. Figure 9

presents the obtained images from both conventional illumination mode and the SIM with the 450 nm wavelength of LED.

123

1304

Chin. Sci. Bull. (2014) 59(12):1291–1307

Fig. 10 Optically sectioned images of Golgi-stained mouse brain neuron cells. The signal is from the back-reflection and scattering of the microcrystallization of silver chromate under the illumination of 450 nm LED light. a Shows a stack of optically sectioned images comprised of 186 axial layers obtained with 209, NA = 0.6 objective; b shows the maximum-intensity projection of the 186 planes along Z-axis over a 74 lm thickness; c shows the fine structure of the dendrite enlarged by using 1009, NA = 1.49 objective. Reprinted with permission from [46], Copyright 2013, Nature Publishing Group

To obtain a near isotropic resolution, multi-orientation illuminations are generally required. In the experiments, we applied two perpendicular illumination fringe orientations, i.e., X and Y directions (0 and 90) to achieve higher imaging speed and simplify the pattern design and data analysis. Nevertheless, it is possible to generate three-orientation illumination (0, 60, and 120) [40] or four-orientation illumination (0, 45, 90, and 135) [39] with specifically designed illumination patterns. 4.5 Optical sectioning 3D imaging As a benefit of the fringe projection and low-coherence LED-illumination, DMD-based SIM also has optical sectioning capability in free of speckle-noise. For instance, the apparatus is used for obtaining the 3D structure of Golgistained mouse brain neurons. Figure 10a shows a stack of Golgi-stained neuron cells with the 209, NA = 0.6 objective under the illumination of 450 nm LED light. 186 slicing layers consisting of 558 raw images in 1,024 9 768 pixels are captured with an axial slice interval of 400 nm. The acquisition time is 27 s, including 9 ms exposure time plus 36 ms readout time of the CCD camera for each raw image, 10 ms Z-stage settle time for each slicing layer, and 31 ls loading time for each DMD illumination fringe pattern. Figure 10b presents the maximum-intensity projection of the 186 planes along Z-axis. Figure 10c shows the fine structure of the dendrite enlarged by using 1009, NA = 1.49 objective. So far, we could realize maximal 200 lm

123

penetration depth without obvious degradation of the image quality. In order to describe the sectioning strength of the SIM microscopy, a normalized spatial frequency t = k/(2NAT) is introduced, where k is the wavelength and T is the spatial periodicity of the projected fringe. The case of t = 0 corresponds to the conventional wide-field microscope, while t = 0.5 corresponds to the maximum sectioning strength. In the experimental setup, the period of the illumination fringe is T = 1,225 nm at the 209, NA = 0.6 objective, which corresponds to t = 0.306, from which the FWHM of 930 nm of sectioning strength is obtained. The speed of data acquisition in the optical sectioning mode of the SIM system is mainly limited by the speed and sensitivity of the CCD camera, because both the switching time of DMD and the axial translation stage settle time are negligible. At the present version, a 1,024 9 768 pixels commercial CCD camera with a maximum full-frame rate of 27 fps at its minimal exposure time is used. The maximum acquisition speed of 1.6 9 107 pixels/s is reached at the imaging of the Golgi-stained neuron cells. Higher acquisition speed could be achieved by using faster and sensitive cameras.

References 1. Ko¨hler A (1893) Ein neues Beleuchtungsverfahren fu¨r mikrophotographische Zwecke. Zeitschrift fu¨r wissenschaftliche mikroskopie und fu¨r mikroskopische technik 10:433–440

Chin. Sci. Bull. (2014) 59(12):1291–1307 2. Abbe E (1873) Beitrage zur Theorie des Mikroskops und der mikroskopi schen Wahrnehmung. Arch Mikroskop Anat 9:413–420 3. Patterson GH, Knobel SM, Sharif WD et al (1997) Use of the green fluorescent protein and its mutants in quantitative fluorescence microscopy. Biophys J 73:2782–2790 4. Huang B, Babcock H, Zhuang X (2010) Breaking the diffraction barrier: super-resolution imaging of cells. Cell 143:1047–1058 5. Ferna´ndez-Sua´rez M, Ting AY (2008) Fluorescent probes for super-resolution imaging in living cells. Nat Rev Mol Cell Biol 9:929–943 6. Gonc¸alves MS (2009) Fluorescent labelling of biomolecules with organic probes. Chem Rev 109:190–212 7. Li Z, Zhang J, Yang J et al (2007) Ultrahigh spatiotemporal resolved spectroscopy. Sci China Ser G Phys Mech Astron 50:681–690 8. Zhang X, Han X, Wu F et al (2013) Nano-bio interfaces probed by advanced optical spectroscopy: from model system studies to optical biosensors. Chin Sci Bull 58:2537–2556 9. Lu¨ T, Xiao Q, Li Z et al (2012) Evaluation of ablation differences in air and water for hard tissues using full-field optical coherence microscopy. Chin Sci Bull 57:833–837 10. Ding Y, Xi P, Ren Q (2011) Hacking the optical diffraction limit: review on recent developments of fluorescence nanoscopy. Chin Sci Bull 56:1857–1876 11. Hell SW, Wichmann J (1994) Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. Opt Lett 19:780–782 12. Klar TA, Hell SW (1999) Subdiffraction resolution in far-field fluorescence microscopy. Opt Lett 24:954–956 13. Betzig E, Patterson GH, Sougrat R et al (2006) Imaging intracellular fluorescent proteins at nanometer resolution. Science 313:1642–1645 14. Hess ST, Girirajan TPK, Mason MD (2006) Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys J 91:4258–4272 15. Rust MJ, Bates M, Zhuang X (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat Methods 3:793–796 16. Bates M, Huang B, Dempsey GT et al (2007) Multicolor superresolution imaging with photo-switchable fluorescent probes. Science 21:1749–1753 17. Huang B, Wang W, Bates M et al (2008) Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy. Science 8:810–813 18. Wilson T (1990) Confocal microscopy. Academic Press, London 19. So PTC, Dong CY, Masters BR et al (2000) Two-photon excitation fluorescence microscopy. Annu Rev Biomed Eng 2:399–429 20. So PTC (2004) Multi-photon excitation fluorescence microscopy. Frontiers in Biomedical Engineering. Springer, New York, pp 529–544 21. Huisken J, Swoger J, Bene FD et al (2004) Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 13:1007–1009 22. Keller PJ, Schmidt AD, Wittbrodt J et al (2008) Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy. Science 14:1065–1069 23. Krˇ´ızˇek P, Hagen GM (2012) Current optical sectioning systems in florescence microscopy. Formatex, Spain, pp 826–832 24. Neil MAA, Juskaitis R, Wilson T (1997) Method of obtaining optical sectioning by using structured light in a conventional microscope. Opt Lett 22:1905–1907 25. Gustafsson MGL (2000) Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J Microsc 198:82–87

1305 26. Walker JG (2001) Non-scanning confocal fluorescence microscopy using speckle illumination. Opt Commun 189:221–226 27. Heintzmann R, Benedetti PA (2006) High-resolution image reconstruction in fluorescence microscopy with patterned excitation. Appl Opt 45:5037–5045 28. Ventalon C, Mertz J (2005) Quasi-confocal fluorescence sectioning with dynamic speckle illumination. Opt Lett 30:3350–3352 29. Waterman-Storer CM, Desai A, Bulinski JC et al (1998) Fluorescent speckle microscopy, a method to visualize the dynamics of protein assemblies in living cells. Curr Biol 8:1227–1230 30. Frohn JT, Knapp HF, Stemmer A (2000) True optical resolution beyond the Rayleigh limit achieved by standing wave illumination. PNAS 97:7232–7236 31. Fedosseev R, Belyaev Y, Frohn J et al (2005) Structured light illumination for extended resolution in fluorescence microscopy. Opt Lasers Eng 43:403–414 32. Mudry E, Belkebir K, Girard J et al (2012) Structured illumination microscopy using unknown speckle patterns. Nat Photonics 6:312–315 33. Heintzmann R, Cremer CG (1999) Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating. Proc SPIE 3568:185–196 34. Muller CB, Enderlein J (2010) Image Scanning Microscopy. Phys Rev Lett 104:198101 35. York AG, Parekh SH, Nogare DD et al (2012) Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy. Nat Methods 9:749–754 36. Littletona B, Lai K, Longstaff D et al (2007) Coherent superresolution microscopy via laterally structured illumination. Micron 38:150–157 37. Ryu J, Hong SS, Horn BKP et al (2006) Multibeam interferometric illumination as the primary source of resolution in optical microscopy. Appl Phys Lett 88:171112 38. Wang L, Pitter MC, Somekh MG (2011) Wide-field high-resolution structured illumination solid immersion fluorescence microscopy. Opt Lett 36:2794–2796 39. Chang BJ, Chou LJ, Chang YC et al (2009) Isotropic image in structured illumination microscopy patterned with a spatial light modulator. Opt Express 17:14710–14721 40. Shao L, Kner P, Rego EH et al (2011) Super-resolution 3D microscopy of live whole cells using structured illumination. Nat Methods 8:1044–1046 41. Hirvonen LM, Wicker K, Mandula O et al (2009) Structured illumination microscopy of a living cell. Eur Biophys J 38:807–812 42. Kner P, Chhun BB, Griffis ER et al (2009) Super-resolution video microscopy of live cells by structured illumination. Nat Methods 6:339–342 43. Fiolka R, Shao L, Rego EH et al (2012) Time-lapse two-color 3D imaging of live cells with doubled resolution using structured illumination. PNAS. doi:10.1073/pnas:1119262109 44. Gardeaza´bal Rodrı´guez PF, Sepulveda E, Dubertret B et al (2008) Axial coding in full-field microscopy using threedimensional structured illumination implemented with no moving parts. Opt Lett 33:1617–1619 45. Docter MW, Van den Berg PM, Alkemade PF et al (2007) Structured illumination microscopy using extraordinary transmission through sub-wavelength hole-arrays. J Nanophoton 1:011665 46. Dan D, Lei M, Yao B et al (2013) DMD-based LED-illumination Super-resolution and optical sectioning microscopy. Sci Rep 3:1116 47. Shroff SA, Fienup JR, Williams DR (2008) OTF compensation in structured illumination superresolution images. Proc SPIE 7094:709402 48. Somekh MG, Hsu K, Pitter MC (2011) Effect of processing strategies on the stochastic transfer function in structured illumination microscopy. J Opt Soc Am A 28:1925–1934

123

1306 49. Somekh MG, Hsu K, Pitter MC (2009) Stochastic transfer function for structured illumination microscopy. J Opt Soc Am A 26:1630–1637 50. Somekh MG, Hsu K, Pitter MC (2008) Resolution in structured illumination microscopy: a probabilistic approach. J Opt Soc Am A 25:1319–1329 51. Shroff SA, Fienup JR, Williams DR (2009) Phase-shift estimation in sinusoidally illuminated images for lateral superresolution. J Opt Soc Am A 26:413–424 52. Shroff SA, Fienup JR, Williams DR (2010) Lateral superresolution using a posteriori phase shift estimation for a moving object: experimental results. J Opt Soc Am A 27:1770–1782 53. Wicker K, Mandula O, Best G et al (2013) Phase optimisation for structured illumination microscopy. Opt Express 21:2032–2049 54. Orieux F, Sepulveda E, Loriette V et al (2012) Bayesian estimation for optimized structured illumination microscopy. IEEE Trans Image Process 21:601–614 55. Ryu J, Horn BKP, Mermelstein MS et al (2003) Application of structured illumination in nano-scale vision. IEEE Workshop on Computer Vision for the Nano-Scale, Wisconsin, pp 16–22 56. De´barre D, Botcherby EJ, Booth MJ et al (2008) Adaptive optics for structured illumination microscopy. Opt Express 16: 9290–9305 57. Beversluis MR, Bryant GW, Stranick SJ (2008) Effects of inhomogeneous fields in superresolving structured-illumination microscopy. J Opt Soc Am A 25:1371–1377 58. Lefman J, Scott K, Stranick S (2011) Live, video-rate superresolution microscopy using structured illumination and rapid GPU-based parallel processing. Microsc Microanal 17:191–196 59. Gustafsson MG, Shao L, Carlton PM et al (2008) Threedimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys J 94:4957–4970 60. Frohn JT, Knapp HF, Stemmer A (2001) Three-dimensional resolution enhancement in fluorescence microscopy by harmonic excitation. Opt Lett 26:828–830 61. Schermelleh L, Carlton PM, Haase S et al (2008) Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy. Science 6:1332–1336 62. Fiolka R, Beck M, Stemmer A (2008) Structured illumination in total internal reflection fluorescence microscopy using a spatial light modulator. Opt Lett 33:1629–1631 63. Gustafsson MGL (2005) Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. PNAS 102:13081–13086 64. Heintzmann R (2003) Saturated patterned excitation microscopy with two-dimensional excitation patterns. Micron 34: 283–291 65. Zhang H, Zhao M, Peng L (2011) Nonlinear structured illumination microscopy by surface plasmon enhanced stimulated emission depletion. Opt Express 19:24783–24794 66. Gur A, Zalevsky Z, Mico´ V et al (2011) The limitations of nonlinear fluorescence effect in super resolution saturated structured illumination microscopy system. J Fluoresc 21:1075–1082 67. Regoa EH, Shao L, Macklin JJ et al (2011) Nonlinear structuredillumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution. PNAS. doi:10.1073/pnas. 1107547108 68. Hirvonen L, Mandula O, Wicker K et al (2008) Structured illumination microscopy using photoswitchable fluorescent proteins. Proc SPIE 6861:68610L 69. Keller PJ, Schmidt AD, Santella A et al (2010) Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy. Nat Methods 7:637–642 70. Planchon TA, Gao L, Milkie DE et al (2011) Rapid threedimensional isotropic imaging of living cells using Bessel beam plane illumination. Nat Methods 8:417–423

123

Chin. Sci. Bull. (2014) 59(12):1291–1307 71. Gao L, Shao L, Higgins CD et al (2012) Noninvasive imaging beyond the diffraction limit of 3D dynamics in thickly fluorescent specimens. Cell 151:1370–1385 72. Shao L, Winoto L, Agard DA et al (2012) Interferometer-based structured-illumination microscopy utilizing complementary phase relationship through constructive and destructive image detection by two cameras. J Microsc 246:229–236 73. Neumann A, Kuznetsova Y, Brueck SR (2008) Structured illumination for the extension of imaging interferometric microscopy. Opt Express 16:6785–6793 74. Mandula O, Kielhorn M, Wicker K et al (2012) Line scanstructured illumination microscopy super-resolution imaging in thick fluorescent samples. Opt Express 20:24167–24174 75. Shao L, Isaac B, Uzawa S et al (2008) I5S: wide-field light microscopy with 100-nm-scale resolution in three dimensions. Biophys J 94:4971–4983 76. Chen J, Xu Y, Lv X et al (2013) Super-resolution differential interference contrast microscopy by structured illumination. Opt Express 21:112–121 77. Strauss MP, Liew ATF, Turnbull L et al (2012) 3D-SIM super resolution microscopy reveals a bead-like arrangement for FtsZ and the division machinery: implications for triggering cytokinesis. PLoS Biol 10:e1001389 78. Sonnen KF, Schermelleh L, Leonhardt H et al (2012) 3Dstructured illumination microscopy provides novel insight into architecture of human centrosomes. Open Biol 1:965–976 79. Best G, Amberger R, Baddeley D et al (2011) Structured illumination microscopy of autofluorescent aggregations in human tissue. Micron 42:330–335 80. Fitzgibbon J, Bell K, King E et al (2010) Super-resolution imaging of plasmodesmata using three-dimensional structured illumination microscopy. Plant Physiol 153:1453–1463 81. Markaki Y, Smeets D, Fiedler S et al (2012) The potential of 3DFISH and super-resolution structured illumination microscopy for studies of 3D nuclear architecture. Bio Essays 34:412–426 82. Cogger VC, McNerney GP, Nyunt T et al (2010) Threedimensional structured illumination microscopy of liver sinusoidal endothelial cell fenestrations. J Struc Biol 171:382–388 83. Bullen A (2008) Microscopic imaging techniques for drug discovery. Nat Rev Drug Discov 7:54–67 84. Lu CH, Pe´gard NC, Fleischer JW (2013) Flow-based structured illumination. Appl Phys Lett 102:161115 85. Hao XT, Hirvonen LM, Smith TA (2013) Nanomorphology of polythiophene–fullerene bulk-heterojunction films investigated by structured illumination optical imaging and time-resolved confocal microscopy. Methods Appl Fluoresc 1:015004 86. Chang BJ, Lin SH, Chou LJ et al (2011) Subdiffraction scattered light imaging of gold nanoparticles using structured illumination. Opt Lett 36:4773–4775 87. Jiang S, Walker JG (2004) Experimental confirmation of nonscanning fluorescence confocal microscopy using speckle illumination. Opt Commun 238:1–12 88. Jiang S, Walker JG (2005) Non-scanning fluorescence confocal microscopy using speckle illumination and optical data processing. Opt Commun 256:35–45 89. Jiang S, Walker JG (2009) Speckle-illuminated fluorescence confocal microscopy, using a digital micro-mirror device. Meas Sci Technol 20:065501 90. Ventalon C, Mertz J (2005) Quasi-confocal fluorescence sectioning with dynamic speckle illumination. Opt Lett 30:3350–3352 91. Ventalon C, Mertz J (2006) Dynamic speckle illumination microscopy with translated versus randomized speckle patterns. Opt Lett 14:7198–7209 92. Ventalon C, Heintzmann R, Mertz J (2007) Dynamic speckle illumination microscopy with wavelet prefiltering. Opt Lett 32:1417–1419

Chin. Sci. Bull. (2014) 59(12):1291–1307 93. Lim D, Chu KK, Mertz J (2008) Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy. Opt Lett 33:1819–1821 94. Lim D, Ford TN, Chu KK et al (2011) Optically sectioned in vivo imaging with speckle illumination HiLo microscopy. J Biomed Opt 16:016014 95. Choi Y, Yang TD, Lee KJ et al (2011) Full-field and single-shot quantitative phase microscopy using dynamic speckle illumination. Opt Lett 36:2465–2467 96. Neil MAA, Wilson T, Juskaitis R (1998) A light efficient optically sectioning microscope. J Microsc 189:114–117 97. Fukano T, Miyawaki A (2003) Whole-field fluorescence microscope with digital micromirror device: imaging of biological samples. Appl Opt 42:4119–4124 98. Karadaglic´ D, Jusˇkaitis R, Wilson T (2002) Confocal endoscopy via structured illumination. Scanning 24:301–304 99. Monneret S, Rauzi M, Lenne PF (2006) Highly flexible wholefield sectioning microscope with liquid-crystal light modulator. J Opt A 8:S461 100. Delica S, Blanca CM (2007) Wide-field depth-sectioning fluorescence microscopy using projector-generated patterned illumination. Appl Opt 46:7237–7243 101. Mazhar A, Cuccia DJ, Gioux S et al (2010) Structured illumination enhances resolution and contrast in thick tissue fluorescence imaging. J Biomed Opt 15:010506 102. Krˇ´ızˇek P, Rasˇka I, Hagen GM (2012) Flexible structured illumination microscope with a programmable illumination array. Opt Express 20:24585–24599 103. Krzewina LG, Kim MK (2006) Single-exposure optical sectioning by color structured illumination microscopy. Opt Lett 31:477–479 104. Wicker K, Heintzmann R (2010) Single-shot optical sectioning using polarization-coded structured illumination. J Opt 12:084010 105. Karadaglic´ D (2008) Image formation in conventional brightfield reflection microscopes with optical sectioning property via structured illumination. Micron 39:302–310 106. Karadaglic D, Wilson T (2008) Image formation in structured illumination wide-field fluorescence microscopy. Micron 39:808–818 107. Barlow AL, Guerin CJ (2007) Quantization of widefield fluorescence images using structured illumination and image analysis software. Microsc Res Tech 70:76–84 108. Hagen N, Gao L, Tkaczyk TS (2012) Quantitative sectioning and noise analysis for structured illumination microscopy. Opt Express 20:403–413 109. Schaefer LH, Schuster D, Schaffer J (2004) Structured illumination microscopy: artefact analysis and reduction utilizing a parameter optimization approach. J Microsc 216:165–174 110. Chasles F, Dubertret B, Boccara AC (2007) Optimization and characterization of a structured illumination microscope. Opt Express 15:16130–16140 111. Cole MJ, Siegel J, Webb SED et al (2001) Time-domain wholefield fluorescence lifetime imaging with optical sectioning. J Microsc 203:246–257 112. Siegel J, Elson DS, Webb SED et al (2001) Whole-field fivedimensional fluorescence microscopy combining lifetime and spectral resolution with optical sectioning. Opt Lett 26:1338–1340 113. Cole MJ, Siegel J, Webb SED et al (2000) Whole-field optically sectioned fluorescence lifetime imaging. Opt Lett 25:1361–1363 114. Webb SED, Gu Y, Leveque-Fort S et al (2002) A wide-field time-domain fluorescence lifetime imaging microscope with optical sectioning. Rev Sci Instrum 73:1898–1907

1307 115. Gao L, Bedard N, Hagen N et al (2011) Depth-resolved image mapping spectrometer (IMS) with structured illumination. Opt Express 19:17439–17452 116. Ducros N, Bassi A, Valentini G et al (2013) Fluorescence molecular tomography of an animal model using structured light rotating view acquisition. J Biomed Opt 18:020503 117. Lukic V, Markel VA, Schotland JC (2009) Optical tomography with structured illumination. Opt Lett 34:983–985 118. Be´langer S, Abran M, Intes X et al (2010) Real-time diffuse optical tomography based on structured illumination. J Biomed Opt 15:016006 119. Berrocal E, Kristensson E, Richter M et al (2008) Application of structured illumination for multiple scattering suppression in planar laser imaging of dense sprays. Opt Express 16:17870–17881 120. Kristensson E, Berrocal E, Richter M et al (2008) High-speed structured planar laser illumination for contrast improvement of two-phase flow images. Opt Lett 33:2752–2754 121. Schro¨ter TJ, Johnson SB, John K et al (2012) Scanning thinsheet laser imaging microscopy (sTSLIM) with structured illumination and HiLo background rejection. Biomed Opt Express 3:170 122. Maschio MD, Difato F, Beltramo R et al (2010) Simultaneous two-photon imaging and photo-stimulation with structured light illumination. Opt Express 18:18720–18731 123. Ansari Z, Gu Y, Siegel J et al (2002) Wide-field, real-time depth-resolved imaging using structured illumination with photorefractive holography. Appl Phys Lett 81:2148–2150 124. Karadaglic´ D, Jusˇkaitis R, Wilson T (2002) Confocal endoscopy via structured illumination. Scanning 24:301–304 125. Bozinovic N, Ventalon C, Ford T et al (2008) Fluorescence endomicroscopy with structured illumination. Opt Express 16:8016–8025 126. Berrocal E, Kristensson E, Richter M et al (2008) Application of structured illumination for multiple scattering suppression in planar laser imaging of dense sprays. Opt Express 16:17870–17881 127. Kristensson E, Richter M, Pettersson SG et al (2008) Spatially resolved, single-ended two-dimensional visualization of gas flow phenomena using structured illumination. Appl Opt 47:3927–3931 128. Langhorst MF, Schaffer J, Goetze B (2009) Structure brings clarity: structured illumination microscopy in cell biology. Biotech J 4:858–865 129. Blamey NJF, Ryder AG, Feely M et al (2008) The application of structured light illumination microscopy to hydrocarbon-bearing fluid inclusions. Geofluids 8:102–112 130. Gruppetta S, Chetty S (2011) Theoretical study of multispectral structured illumination for depth resolved imaging of non-stationary objects: focus on retinal imaging. Biomedical Opt Express 2:255 131. Su X, Zhang Q (2010) Dynamic 3-D shape measurement method: a review. Opt Lasers Eng 48:191–204 132. Salvi J, Pages J, Batlle J (2004) Pattern codification strategies in structured light systems. Pattern Recongn 37:827–849 133. Rocchini CMPPC, Cignoni P, Montani C et al (2001) A low cost 3D scanner based on structured light. Proc Eurographics 20:299–308 134. Salvi J, Pages J, Batlle J (2004) Pattern codification strategies in structured light systems. Pattern Recogn 37:827–849 135. Heintzmann R (2006) Structured illumination methods. Handbook of biological confocal microscopy. Springer, New York, pp 265–279

123