Journal of Structural Biology 175 (2011) 277–287
Contents lists available at ScienceDirect
Journal of Structural Biology journal homepage: www.elsevier.com/locate/yjsbi
Three-dimensional reconstruction using an adaptive simultaneous algebraic reconstruction technique in electron tomography Xiaohua Wan a,c, Fa Zhang a,⇑, Qi Chu a,c, Kai Zhang b,c, Fei Sun b, Bo Yuan d, Zhiyong Liu a a
Institute of Computing Technology and Key Lab of Intelligent Information Processing, Chinese Academy of Sciences, Beijing 100190, China Institute of Biophysics, Chinese Academy of Sciences, Beijing, China c Graduate University, Chinese Academy of Sciences, Beijing, China d Department of Computer Science and Engineering, Shanghai Jiaotong University, Shanghai, China b
a r t i c l e
i n f o
Article history: Received 21 October 2010 Received in revised form 3 June 2011 Accepted 6 June 2011 Available online 14 June 2011 Keywords: Electron tomography Three-dimensional reconstruction Iterative methods ASART (adaptive simultaneous algebraic reconstruction technique)
a b s t r a c t Three-dimensional (3D) reconstruction of electron tomography (ET) has emerged as an important technique in analyzing structures of complex biological samples. However most of existing reconstruction methods are not suitable for extremely noisy and incomplete data conditions. We present an adaptive simultaneous algebraic reconstruction technique (ASART) in which a modified multilevel access scheme and an adaptive relaxation parameter adjustment method are developed to improve the quality of the reconstructed 3D structure. The reconstruction process is facilitated by using a column-sum substitution approach. This modified multilevel access scheme is adopted to arrange the order of projections so as to minimize the correlations between consecutive views within a limited angle range. In the adaptive relaxation parameter adjustment method, not only the weight matrix (as in the existing methods) but the gray levels of the pixels are employed to adjust the relaxation parameters so that the quality of the reconstruction is improved while the convergence process of the reconstruction is accelerated. In the column-sum substitution approach, the computation to obtain the reciprocal of the sum for the columns in each view is avoided so that the needed computations for each iteration can be reduced. Experimental results show that the proposed technique ASART is better based on objective quality measures than other methods, especially when data is noisy and limited in tilt angles. At the same time, the reconstruction by ASART outperforms that of simultaneous algebraic reconstruction technique (SART) in speed. Ó 2011 Elsevier Inc. All rights reserved.
1. Introduction Electron tomography (ET) combines electron microscopy (EM) and tomographic imaging to elucidate three-dimensional (3D) descriptions of complex biological structures at molecular resolution (Frank, 2006; Huang et al., 2010; Zhang et al., 2010). This structural information is critical to understanding biological functions (Bilbao-Castro et al., 2006). 3D reconstruction of ET requires a set of projection images acquired at different orientations, via tilting specimens around axes (Marabini et al., 1997). Due to physical limitations of microscopes, the angular tilt range is limited and, as a result, tomographic tilt series have a wedge of missing data corresponding to the uncovered angular range. In order to obtain high resolution reconstructions, specimens are imaged at very low electron doses, which make EM images extremely noisy (with signal-to-noise ratio in the order of 0.1). As a consequence, highresolution electron tomography requires a method of ‘‘3D reconstruction from projections’’ able to deal with limited angle ⇑ Corresponding author. Fax: +86 (010) 62601356. E-mail address:
[email protected] (F. Zhang). 1047-8477/$ - see front matter Ó 2011 Elsevier Inc. All rights reserved. doi:10.1016/j.jsb.2011.06.002
conditions and extremely low signal-to-noise ratios of the projection images. Weighted back-projection (WBP) has been one of the most popular methods in the field of 3D reconstruction of ET, due to its algorithmic simplicity and computational efficiency (Radermacher, 2006). The major disadvantage of WBP, however, is that the results may be strongly affected by limited-angle data and noisy conditions (Fernández et al., 2002). Series expansion methods (i.e. iterative methods) constitute one of the main alternatives to WBP in 3D reconstruction of ET, owing to their good performance in handling incomplete, noisy data. Several traditional iterative methods, such as algebraic reconstruction techniques (ART) (Marabini et al., 1998; Bilbao-Castro et al., 2009), simultaneous iterative reconstruction technique (SIRT) (Sorzano et al., 2001), component averaging methods (CAV) (Fernández et al., 2002), block-iterative CAV (BICAV) (Fernández et al., 2002), and simultaneous algebraic reconstruction technique (SART) (Castano-Diez et al., 2007) have all been utilized to 3D reconstruction of ET. ART enjoys a rapid convergence, however, exhibits a very noisy salt and pepper characteristic of reconstructed images. SIRT and CAV, on the contrary, produce fairly smooth reconstructed images, but still require a
278
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
large number of iterations for convergence. SART is characterized by better robustness than ART under noise and its convergence speed is reported to be faster than SIRT and CAV (Castano-Diez et al., 2007). Overall, the convergence process of SART is still slow and hard to acquire satisfactory results. First, the disadvantage of SART is its slow convergence speed due to highly correlated consecutive projections (Guan and Gordon, 1994). Several methods (e.g. Random Access Scheme (RAS), Multilevel Access Scheme (MAS), Weighted Distance Scheme (WDS)) have been proposed and evaluated to minimize this correlation in the tomography field (Guan and Gordon, 1994; Mueller et al., 2002; Kazantsev et al., 2005). These methods are mostly adopted in the case where projection views are distributed uniformly among 180°. But in electron tomography, the angular tilt range is limited in a certain range typically from 60° (or 70°) to +60° (or +70°), due to physical limitations of microscopes. Thus, the methods mentioned above cannot be directly used in 3D reconstruction of ET. Furthermore, it has been pointed out that SART can be improved by adjusting the relaxation parameters (Herman and Meyer, 1993). Careful selections for the relaxation parameters can lead to the better qualities of reconstructions (Wenkai, 2004; Xiaohua et al., 2009). In SART, however, the relaxation parameters are either constant or at least the same, as a function of the iteration numbers for each given iteration. In each iteration of SART, all the pixels with the same weights receive the same corrections even they have different gray levels. From the reconstruction process of SART (as expressed by Eq. (5) in the next section), it can be seen that pixels with different gray levels have different contributions to the computed projection in each iteration. Thus, the gray levels of the pixels need to be considered for the selection for the relaxation parameters. The central idea of this report lies on the premise that SART can be improved by carefully choosing the order of data access and adjusting the relaxation parameters. We hereby present an adaptive simultaneous algebraic reconstruction technique (ASART) for incomplete data and noisy conditions. Specifically, we develop a modified multilevel access scheme (MMAS) to minimize the correlation between consecutive views in the limited angle range. In addition, we apply an adaptive adjustment of relaxation parameters (AAR) to correct the discrepancy between actual and computed projections for each iteration during the reconstruction process. Furthermore, we develop a column-sum substitution (CSS) technique to reduce the memory requirement and computation time especially when the size of the dataset in ET is large. Experimental results show that ASART makes encouraging improvement in reconstruction quality as well as the speed of the reconstruction process. Overall, ASART yields better solutions under limited-angle and noisy conditions and exhibits faster convergence speed than WBP and SART. The rest of the paper is organized as follows: Section 2 reviews iterative 3D reconstruction methods for ET. Section 3 focuses on the ASART algorithm. Section 4 shows and analyzes the experimental results. Section 5 concludes the paper.
2. Iterative image reconstruction methods In ET, the projection images are acquired from a specimen through the so-called single-axis tilt geometry. The specimen is tilted over a range, typically from 60° (or 70°) to +60° (or +70°) due to physical limitations of microscopes, at small tilt increments (1° or 2°). An image of the same object area is recorded at each tilt angle and then the 3D reconstruction of the specimen is obtained from a set of projection images with iterative methods.
2.1. Iterative methods using blob Iterative methods are based on the series expansion approach (Censor, 1983) in which 3D volume f is represented as a linear combination of a limited set of known and fixed basis functions bj, with appropriate coefficients xj, i.e.
f ðr; /1 ; /2 Þ
XJ
x b ðr; /1 ; /2 Þ; j¼1 j j
ð1Þ
where ðr; /1 ; /2 Þ are spherical coordinates, and J is the total number of the unknown variables xj. During the 1990s, spherically symmetric volume elements (blobs) have been thoroughly investigated and, as a consequence, the conclusion that blobs yield better reconstructions than the traditional voxels has been drawn in 3D reconstruction (Lewitt, 1992). The use of blob basis functions provides iterative methods with better resolution-noise performance than voxel basis functions due to the overlapping nature of their rotational symmetric basis functions. Thus, we consider the blob basis instead of the traditional voxel one. The blob basis here in this report is constructed using generalized Kaiser–Bessel (KB) window functions:
bðrÞ ¼
8 pffiffiffiffiffiffiffiffiffiffiffiffiffim pffiffiffiffiffiffiffiffiffiffiffiffiffi < 1ðr=aÞ2 Im a 1ðr=aÞ2 :
Im ðaÞ
0;
;
06r6a
ð2Þ
otherwise
where Im ðÞ denotes the modified Bessel function of the first kind of order m, a is the radius of the blob, and a is a non-negative real number controlling the shape of the blob. The choice of the parameters m, a, and a will influence the quality of the blob-based reconstructions. The basis functions that developed in (Matej and Lewitt, 1995) are used for the choice of the parameters in our algorithm (i.e. a = 2, m = 2 and a = 3.6). In 3D-ET, the model of the image formation process is expressed by the following linear system:
pi
XJ j¼1
wij xj ;
ð3Þ
where pi denotes the ith measured image of f and wij the value of the ith projection of the jth basis function. Under such a model, the aim is to estimate the unknown value xj from the known projection data pi by means of iterative methods. The spherical symmetry of blobs makes the projection of the blob independent of the direction of the line. Consequently, it is possible to pre-compute the projection of the generic blob called footprint. In this way, the wij terms are computed and stored, and then transformed into simple references to the footprint in reconstruction processes (Matej and Lewitt, 1996). 2.2. Simultaneous algebraic reconstruction technique (SART) SART is a basic iterative method designed to solve the linear system in image reconstruction. Typically, the algorithm begins with an arbitrary X(0) and then begins to iterate until convergence (Andersen and Kak, 1984). It is possible that the arbitrary initial value will greatly deviate from the true value. So the number of iterations can be very large. In order to accelerate the process of convergence, SART further adopts a back-projection technique (BPT), a simple WBP without the weighting functions, to estimate the first approximation X(0) (Herman and Meyer, 1993). BPT is a simple reconstruction method where the gray level of a pixel can be considered as the weighted average of the projections for all of the possible rays passing through the pixel (Herman, 1980; Frank, 2006). Consequently, the initial solution X(0) is defined by ð0Þ
xj
PM wij pi ¼ Pi¼1 ; M i¼1 wij
j ¼ 1; 2 ; . . . ; N:
ð4Þ
279
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
Then, in the iterative process, SART is written by the following expression: ðkþ1Þ
xj
ðkÞ
¼ xj þ
aij ¼ PS s¼1
PS
s¼1 aij ðpi
PN
ðkÞ h¼1 wih xh Þ;
kwij wij
PN h¼1
ð5Þ
wih
where k is the number of iterations, aij is the relaxation parameter, k is the fixed value (in general, 0 < k < 2), S is the number of projections per view, and i = bS + s denotes the ith equation of the system (B is the number of all views and b = (k mod B) is the index of the ðkþ1Þ ðkÞ view). xj is the next iterative value by updating xj . SART adopts a view-by-view strategy, that is, an approximation is updated simultaneously by all the projections of each view. SART has three limitations. First, SART adopts a sequential accessing scheme (SAS) to update views and thus converges slowly owing to the high correlation between consecutive views. Second, SART employs a constant relaxation parameter aij (since k is fixed and wij is invariable during the reconstruction procedure as shown in Eq. (5)). With the constant relaxation parameter, the pixels with large gray level xj have the same back-projection of the discrepancies as the pixels with small xj as seen in Eq. (5). The projection data P is contributed not only by the geometry W but also the gray level X as shown in Eq. (3). Thus, both W and X need to be considered in the computation of the back-projection of the discrepancies. Finally, computing the column sum for each view is time-consuming and storing it requires massive amount of memory for the data. 3. Adaptive simultaneous algebraic reconstruction technique (ASART) To generate high-quality reconstructions with improved computational speed, we have developed and improved a technique named ASART. The key techniques developed in ASART include a modified multilevel access scheme (MMAS) to arrange the order of projection data, an adaptive adjustment of relaxation parameters (AAR) to correct the discrepancy between actual and computed projections, and a column-sum substitution (CSS) to reduce the memory requirement and computation time. 3.1. Modified multilevel access scheme SART adopts SAS to order the views so that there is a high correlation between consecutive views. The convergence can be significantly facilitated if the views are ordered to maximize their orthogonality (Guan and Gordon, 1994). Toward this goal, a multilevel access scheme (MAS) is adopted to substantially decrease the correlated error between the consecutive views. A detailed comparison between MAS and SAS shows that MAS yields the most efficient reconstruction (Guan and Gordon, 1996). Applications in different situations have shown that MAS can obtain promising results (Guan et al., 1998, 2000). MAS is based on the fact that two views of 90° apart are minimally correlated, and the third view is set to the angle halving the former two views to minimize the correlation of three views (Guan and Gordon, 1994). The MAS ordering applies to any number of views though it works best if the number of views is a power of two (then MAS has the same order as one-dimensional fast Fourier transform). Suppose that the views are indexed as 0, 1, . . . , B 1 where B is the number of views. In MAS, views can be organized in a total of L levels where L is expressed by:
L ¼ dlog2 Be
ð6Þ
In level l = 1, view 0 (0°) and view B/2 (90°) with a maximum orthogonality are accessed first. Then in level l = 2, there are two views, i.e. 45° and 135° (or whose indices are B/4 and 3B/4).
Furthermore in level l = 3, the indices of views are respectively B/ 8, 5B/8, 3B/8 and 7B/8. In every level, the views with the odd indices and the following ones with even indices are orthogonal. As described above, MAS is adapted for the complete projection views distributed uniformly from 0° to 180°. However, the projection views of ET are incomplete and limited in a certain range. We cannot always find two projections whose views are 90° apart. If MAS is directly used to arrange the order of the views in ET, some views to which there are no views perpendicular will be left. If these views that remain unprocessed by MAS are then arranged by SAS, there will be a high correlation between the consecutive views. We propose a modified MAS (MMAS) to arrange the order of projections in ASART. Only a series of projections evenly distributed across the whole angular is considered as ET projections. For example, the tilt angle range is from 60° to +60° with a small tilt increment 2°. In MMAS, we adopt the range h (h = 120°) of tilt angles as the selected factor instead of B since the range is 120° rather than 180°. Note that the range for the tilt angles can still be different in different situations. In the first level, we choose view 60° and view 0° between whom the angle is the half of the range (but not 90°). The two views that halve the angles between the first two are in the second level (i.e. view 30° and view 30°). In such a scheme, views in one level will halve (or almost) the views in all previous levels. Fig. 1 shows the situation for the example by labeling the order of access in our MMAS scheme on the corresponding view angles. The proposed scheme clearly covers all angular regions evenly over time as shown in Fig. 1. Table 1 summarizes the results of every level for this example. Note that in any level, the computed value of views may not be included in the projected views or have already been accessed. If so, we search both sides of the computed value until the closest unused value is found and then put it into the sequence. Before the iterative process of ASART is carried out, the order of the overall views is arranged according to MMAS discussed above. 3.2. Adaptive adjustment of relaxation parameters In SART, the gray level X is corrected by the relaxation parameters aij and the discrepancies between the actual projections P and the computed projections P0 in each iteration. As shown in Eq. (5), the relaxation parameters aij are only decided by the weight wij and the fixed value k in the iterative procedure. The convergence process can be faster if the relaxation parameters are adjusted as a function of the number of iterations. The relaxation parameters are decided in a way that they are decreased as
(+60°) 8 (+44°,l=3) 4 (+30°,l=2) 6 (+14°,l=3) 2 (0°,l=1) 7 (-16°,l=3) 3 (-30°,l=2) 5 (-46°,l=3) 1 (-60°,l=1) Fig.1. The access orders of the first three levels for the example. The left bold numbers denote the index of access and l denotes the number of the level.
280
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
Table 1 Access orders for P total projections in the angle range (60°–+60°, h = 120°). 60°, 0° 30°, 30° 46°,+14°, 16°,+44° 54°, 6°, 24°, 36°, 38°, 22°, 8°,+52° 58°,+2°, 34°,+26°, 42°,+18°, 12°,+48°, 50°,+10°, 20°,+40°, 36°, 24°, 4°,+56° 56°,+4°,28°,+32°, 48°,+12°, 18°,+42°, 52°,+8°, 22°,+38°, 40°,+20°, 6°,+54°, 44°,+16°, 26°,+34°, 32°,+28°, 10°, 50°, 2°,+58°,+60°
the number of iterations increases as described in (Herman and Meyer, 1993). In SART, the relaxation parameters are determined only by the weight W while the gray level X is ignored. Thus the pixels with large gray levels will have the same back-projection of the discrepancies as the pixels with small gray levels. In fact, the pixels with different gray levels make different contributions to the discrepancies. In ASART, a data-driven adjustment of relaxation parameters (AAR) is applied during the reconstruction procedure. In AAR, the relaxation parameters are determined according to the gray levels as well as the weights as shown in the following equation: ðkÞ
kwij xj : PN ðkÞ w s¼1 ij h¼1 wih xh
aij ¼ PS
ð7Þ
x1 þ x2 ¼ 2:7 x1 þ 2x2 ¼ 3
:
ð8Þ
The computational procedures for locating the solution from an initial guess (x1 = 1.5, x2 = 1.3) are displayed in Fig. 2. The line in red denotes the procedure with the constant relaxation parameter by Eq. (5), and the line in blue describes the process with the adaptive relaxation parameter by Eq. (7). It is seen that the solution obtained by Eq. (7) is closer to the true solution than that obtained by Eq. (5) after the same number of iterations. Note that in fact we do not need to re-compute the sum of wihxh in Eq. (7) since it is equal to pi which has been computed in the process of reprojection. Extra computation is needed only for the mulðkÞ tiplication of wij and xj , but it is not time-consuming compared with the total computation. 3.3. Column-sum substitution (CSS) technique We define cj as the reciprocal of the sum of the jth column in each view by
c j ¼ PS
1
s¼1 wij
:
ð9Þ
Although the column sums remain unchanged in each iteration, large memory space is needed to store each column sum for a SART reconstruction of large images. In fact, it has been proved that cj has no effect on the final solution to which SART converges (Jiang and Wang, 2003). In another iterative method called BICAV (Fernández et al., 2002), the relaxation parameters are generated by the following equation:
x1+x2=2.7 x1+2x2=3 Eq.(5) Eq.(7)
1.6 Initial guess (x1=1.5,x2=1.3)
1.4
The solutions after 1 iteration
1.2
The solutions after 2 iterations
1 0.8
The solutions after 20 iterations
0.6 0.4 0.2
With this approach, the relaxation parameter for each pixel is adjusted according to its gray level obtained in the previous iteration. Note that wij only represents the geometry contribution of the jth pixel to the ith ray integral. According to Eq. (7), the contribution of the jth pixel to the ith ray integral includes both the geometry contribution wij and the contribution of the gray level xj of the jth pixel. This is different from SART where the gray level xj of the jth pixel is not considered as shown in Eq. (5). To illustrate intuitively how the convergence process can be accelerated by AAR, we consider a case with only two variables x1 and x2 satisfying the following equations:
1.8
x2
l = 1(h/2) l = 2(h/4) l = 3(h/8) l = 4(h/16) l = 5(h/32) l = 6(h/64)
True solution
1
1.5
2
2.5
x1 Fig.2. Illustration of the computational procedures from an initial guess by Eq. (5) (red) and Eq. (7) (blue). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
aij ¼ PN
kwij
2 b h¼1 sh ðwih Þ
ð10Þ
;
where sbh denotes the number of times that components xh of the volume contributes with nonzero value to the equations in the bth block. This is called oblique projection in BICAV. BICAV allows the iteration process based on oblique projections. SART, on the other hand, is based on orthogonal projections. It has been proved that oblique projections allow iterative methods to have a faster convergence speed especially during the early stages of iterations (Fernández et al., 2002). Accordingly, we propose that cj is replaced by a scalar b denoting the maximum number of the nonzero weight wij in each view. In this way, the memory requirement and computation time are reduced, and the high qualities of the results are obtained because of this oblique projections. As shown in Fig. 3, the maximum number of the rays that have effect to every pixel on its reconstructed gray level in each view is four. The relaxation parameter aij with our CSS technique can be expressed by ðkÞ
kwij xj aij ¼ PN : ðkÞ 4 h¼1 wih xh
ð11Þ
This modification can save running time because the algorithm does not need to compute each column sum of W for each view. The modification can also reduce the memory requirement by using the scalar b instead of the vector cj, especially when the size of the image is very large. With the improvements mentioned above, ASART is formulated as follows:
8 PM wij pi > ð0Þ > ; > xj ¼ Pi¼1 M < i¼1
wij
j ¼ 1; 2; . . . ; N
> P wij xðkÞ ðpi > ðkþ1Þ ðkÞ > ¼ xj þ 4k Ss¼1 j PN : xj
PN
ðkÞ w x Þ h¼1 ih h ðkÞ w x h¼1 ih h
:
ð12Þ
Notice that all the views are organized in line with MMAS before the iterative process.
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
Fig.3. In the blob (the radius a = 2), a projected pixel contributes to four neighbor projection rays using only one view.
4. Experimental results The objective of this study is to improve the quality and efficiency of ET 3D reconstruction. To meet this objective, a reconstruction method ASART is developed in this paper. Experiments have been carried out to evaluate the proposed method. The evaluation of ASART is presented from two aspects. One is to compare the reconstruction quality of ASART with those of WBP and SART. The other is to compare the convergence speed of ASART with that of SART. The experiments are all carried out a machine running Ubuntu 9.10 32-bit with an Inter Core 2 Q8200 at 2.33 GHz and 4 GB of DDR2 memory.
4.0.1. 2D data reconstruction In this experiment, a brain phantom shown in Fig. 4 as 2D reconstruction data is used to compare the reconstruction
281
qualities. The phantom simulates the head of an adult with an image of 258 258 pixels. To evaluate the performance of ASART, we have done the 2D reconstructions of the phantom under the two different conditions: with complete and no-noisy projected data, and incomplete and noisy projected data respectively. In the first situation, the views range from 0° to 178° at an equal interval of 2° angle. The number of views in the experiment is 90. Thus the projected view is complete. In each view, there are 369 equally spaced parallel rays. In all the experiments, k is set to be 0.2. In each iteration, all of the projections in all of the views are used to update the reconstruction once. The time of the same iterations in ASART is smaller than that in SART as discussed later in this section. WBP and SART have been used for reconstructions in the experiments for a comparative study. Furthermore, to clearly elucidate the usefulness of the three techniques (i.e. MMAS, AAR and CSS) adopted in ASART, we have done the reconstructions using three methods as well: SART combined with MMAS (named SART + MMAS), SART combined with MMAS and AAR (named SART + MMAS + AAR), and ASART. Fig. 4 shows the results of WBP and the results of other methods after 1, 5, 10, 25 and 50 iterations. It can be seen that after 10 iterations, the result of SART + MMAS has less artifact noise in the area outside the boundary than that of SART (see Fig. 4(c) and (d)). Comparing the results of the column (d) with the column (e), we can see that SART + MMAS requires approximately 25 iterations before a solution of similar quality of SART + MMAS + AAR can be obtained with only one iteration. From the fifth iteration, the results of SART + MMAS + AAR are almost the same as that of WBP. Thus, these results indicate a significant improvement obtained by using the proposed MMAS and AAR techniques. The experimental results indicate that ASART can generate almost the same result as that of WBP, and the result of SART is worse than that of ASART and WBP, in the noise-free and complete projection data situation. For objective and quantitative evaluation of the quality of the reconstructed images, the structural consistent figures of merit (FOMs) (Fernández et al., 2002), namely the square Euclidean distance and correlation coefficient between the phantom and the reconstruction, are used to quantify the reconstruction accuracy:
Fig.4. Simulation results of the reconstructions of ‘‘phantom’’ using five methods: WBP, SART, SART + MMAS, SART + MMAS + AAR, and ASART. Ninety views have been used and no limited angle and no noise data are applied. (a) Original image. (b) Result from WBP. (c) Results of 1, 5, 10, 25, 50 iterations from SART. (d) Results of the same iterations from SART + MMAS. (e) Results of the same iterations from SART + MMAS + AAR. (f) Results of the same iterations from ASART.
282
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
scL ¼ 1:0
Square euclidean distance
1
ð13Þ
Pm Pn corr ¼ h 0.95
0.9
WBP SART SART+MMAS SART+MMAS+AAR ASART (a) scL
0
5
10
15
20
25
30
35
40
45
50
Iteration number 1
0.98
0.96 WBP SART SART+MMAS SART+MMAS+AAR ASART
0.94
0.92
0.9
0.88 (b)corr
0.86
0
5
10
15
20
25
30
35
40
45
50
Iteration number Fig.5. Plots of (a) the square Euclidean distances scL and (b) the correlation coefficients corr in the first experiment (i.e. complete data without noisy condition).
IÞðKði; jÞ KÞ i1=2 ; P P n 2 m 2 j¼1 ðIði; jÞ IÞ i¼1 j¼1 ðKði; jÞ KÞ i¼1
Pm Pn i¼1
0.85
Correlation coefficient
2 1 Xm Xn Iði; jÞ Kði; jÞ i¼1 j¼1 mn 2 j¼1 ½ðIði; jÞ
ð14Þ
represent the pixel values (image averwhere Iði; jÞ (I) and Kði; jÞ (K) age value) in the m n original and reconstructed images, respectively. In general, a higher scL and corr indicate a better reconstructed result. We calculated the FOMs scL and corr of the reconstructed results obtained by SART and ASART. The FOMs were also computed for the WBP results. Fig. 5 presents the evolution with the iterations of the FOM values corresponding to five methods: WBP, SART, SART + MMAS, SART + MMAS + AAR, and ASART. As show in Fig. 5, the measures of WBP remain constant since WBP is independent of the number of iterations. By comparing the data in Fig. 5(a), we can observe that WBP, SART + MMAS + AAR, ASART yield the best reconstructed results whose scL are close to one. The MMAS and AAR techniques can improve the qualities of the reconstructed results and the convergence speed as well. ASART can obtain the scL higher than SART with the same iterations in the first experiment (i.e. with no noisy and complete data condition). As shown in Fig. 5(b), we have obtained the similar curves of the corr FOM. Besides, to compare the convergence speed of different methods discussed above, we compare the running times when the results of these methods have the same qualities according to the corr FOM. In this experiment, the values of the corr FOMs used to evaluate the quality of the reconstructed results are 0.80, 0.85, 0.90, and 0.95, respectively. Fig. 6 shows the results of the running time (in seconds). From Fig. 6, we can see that the running times of WBP, SART + MMAS + AAR and ASART are constant for the four measures of the results since the corr FOMs of WBP, SART + MMAS + AAR and ASART after one iteration are more than 0.95. The speed of SART is faster than that of SART + MMAS, SART + MMAS + AAR and ASART when the corr FOMs of the results get to 0.8 and 0.85. However, SART takes much longer time than that of the three methods when the corr FOMs of the results reach 0.9 and 0.95. Thus, ASART has
Fig.6. The running times of WBP, SART, SART + MMAS, SART + MMAS + AAR and ASART when the corr FOMs of the reconstructed results get to 0.80, 0.85, 0.90 and 0.95, respectively in the first experiment (i.e. complete data without noisy condition).
283
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
Fig.7. Simulation results of the reconstructions of ‘‘phantom’’ using five methods: WBP, SART, SART + MMAS, SART + MMAS + AAR, and ASART. Sixty one views have been used and white noise is added. (a) Original image with the white noise. (b) Result from WBP. (c) Results of 1, 5, 10, 25, 50 iterations from SART. (d) Results of the same iterations from SART + MMAS. (e) Results of the same iterations from SART + MMAS + AAR. (f) Results of the same iterations from ASART.
1
Square euclidean distance
0.98 0.96 0.94 WBP SART SART+MMAS SART+MMAS+AAR ASART
0.92 0.9 0.88 0.86
(a) scL
0
5
10
15
20
25
30
35
40
45
50
Iteration number 0.9
WBP SART SART+MMAS SART+MMAS+AAR ASART
0.85
Correlation coefficient
the faster convergence speed than SART under the condition of the complete and noise-free data. In ET, a specimen is tilted over a range due to the physical limitations of microscopes. In order to preserve as much detailed information as possible, the specimens are imaged at very low electron doses, which make the projection images extremely noisy. To evaluate the performance of ASART in the limited tilt angle and very noisy situation, we repeat the above experiments under another experimental condition where the views range from 30° to 150° at an equal interval of 2°, and the white noise is added to the original image. As shown in Fig. 7, in the incomplete data and noisy situation, the results of WBP, SART and ASART are not promising. Note that ASART still yields much better results than any SART result with the same iterations. The results in Fig. 7(d) and (e) show that MMAS and AAR can greatly improve the qualities of the reconstructions. After 5 iterations, the results of the two methods using SART + MMAS + AAR, and ASART have better resolution than the result of WBP. So ASART can make improvements in the quality and convergence speed of the 2D reconstruction under the limited tilt angle and extremely noisy conditions. In the second experiment (i.e. under noisy and incomplete data condition), the measures of WBP also keep invariable. As shown in Fig. 8(a), the reconstruction quality of SART is worse than that of WBP. After five iterations, the reconstruction quality of ASART has already been better than that of WBP according to the scL FOM. Under the same condition, SART + MMAS + AAR and ASART yield the best results amongst all of the tested techniques. As shown in Fig. 8(b), we can draw the conclusion that after one iteration, the results with ASART are better than those of WBP and SART according to the corr FOM. Thus the experiment results show that ASART can obtain much higher quality than WBP and SART under the condition of noisy and incomplete data. We now compare the running times of the five methods when the corr FOMs of their results get to 0.60, 0.70, 0.80, and 0.90, respectively. As shown in Fig. 9, for the first and second results (the corr FOMs get to 0.60 and 0.70), WBP takes the shorter computing time than other methods. Note for the third and fourth results (the corr FOMs get to 0.80 and 0.90), the running times of WBP are not shown in Fig. 9 because the correlation coefficient
0.8
0.75
0.7 (b) corr
0
5
10
15
20
25
30
35
40
45
50
Iteration number Fig.8. Plots of (a) the square Euclidean distances scL and (b) the correlation coefficients corr in the second experiment (i.e. incomplete data with noisy condition).
284
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
Fig.9. The running times of WBP, SART, SART + MMAS, SART + MMAS + AAR and ASART when the corr FOMs of the reconstructed results get to 0.60, 0.70, 0.80 and 0.90, respectively in the second experiment (i.e. incomplete data with noisy condition).
of the final result of WBP is 0.7898 which is below 0.8. In comparison, the correlation coefficients of the results of SART and SART + MMAS cannot get to 0.9, which are thus not included in the fourth situation as well. From Fig. 9, we can see that only the correlation coefficients of the results of SART + MMAS + AAR and ASART can get to 0.9 and the running time of ASART is less than that of SART + MMAS + AAR. Thus, the convergence of ASART is faster than that of SART and WBP under the noisy and incomplete data condition. Comparing the FOMs of the two experimental results, we can conclude that WBP and ASART can yield similar results which are much better than that of SART under the condition of complete data without noise. However, ASART clearly outperform WBP after a very few iterations under the condition of noisy and incomplete data. Furthermore, the qualities obtained from ASART are higher than those obtained from SART under any condition according to the two experiments discussed above. In order to compare the convergence speed of different iterative methods, we collect the whole processing times with different iterations on the two experiments. The results are listed in Table 2 (in seconds). The running times for WBP under the different conditions are also reported in Table 2. Although the running time of WBP is less than that of ASART for one iteration, ASART for one iteration yields much better reconstruction quality than WBP in the second experiment. From Table 2, note that the time of ASART is Table 2 Comparison of different running times on the two experiments. Iterations
1 5 10 25 50 WBP
First experiment (complete data without noise)
Second experiment (incomplete and noisy data)
SART
ASART
SART
ASART
17.078 30.500 45.141 90.781 164.016 8.341
17.937 27.625 41.844 85.000 147.547
11.406 19.422 29.609 59.969 110.656 6.634
12.031 18.828 27.375 54.390 96.172
longer than that of SART for one iteration under the two conditions because arranging data access with MMAS in the initiation process of ASART requires extra computation. However, the reconstructions with ASART take less time than SART for five and more iterations. SART requires extra time to compute each column sum of W of each view for each iteration. But it is not necessary to compute it in each iterative process owing to CSS adopted in ASART. Note also in Table 2 that the more iteration performed, the less running time is needed for ASART compared with SART. Overall, ASART has faster convergence speed than SART. 4.2. 3D data reconstruction In order to demonstrate the applicability of ASART to real data, we present three real sets of data for analysis. The three experimental data are recorded by China National Key Laboratory of Biomacromolecules. All of the data sets are collected by FEI company’s production-Tecnai 20. The first set of data has gold markers and the other data have no gold markers. Three different experimental datasets are used (denoted by small-sized, medium-sized, largesized) with 56 images of 512 512 pixels, 112 images of 1024 1024 pixels, and 119 images of 2048 2048 pixels, to reconstruct tomograms of 512 512 120, 1024 1024 126 and 2048 2048 430 respectively. To evaluate the performance of ASART, we have performed the 3D reconstructions of the three sets of data using WBP, SART and ASART respectively. We use the popular software IMOD to perform the reconstructions of WBP. We have adopted the relaxation factor k with 0.2 and performed the reconstructions with different number of iterations. The first (small-sized) and the third (large-sized) samples are the caveolae of the porcine aorta endothelial (PAE) cell (Shufeng et al., 2009). Montages showing one z-section of the volume reconstructed with the small datasets are presented in Fig. 10. It is clear that the quality of the image obtained by ASART is superior to the image obtained by WBP. From Fig. 10(c), we can see the legible bilayer outlines of caveolae membrane structure (indicated by the rectangles) which are not clear in Fig. 10(a). The result of ASART
285
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
Fig.10. In the small-sized data (512 512), one of slices along the Z-axis of the reconstructions of the caveolae by (a) WBP, (b) 50 iterations of SART, (c) 50 iterations of ASART. Caveolae membrane structures (indicated by the rectangles) can be clearly seen from (c).
is also better than that of SART as shown in Fig. 10(b) and (c). It is obvious that ASART improves the contrast of the reconstructed results compared with SART. The second test sample is the mitochondria of the mice hepatic cell. As shown in Fig. 11, the reconstruction result of ASART shows a more distinctly increased contrast and a higher resolution compared with the reconstruction results of WBP and SART. The reconstruction result of ASART shows that the mitochondria membrane structures indicated by the rectangles in Fig. 11(c) are much clearer than those using WBP in Fig. 11(a) and SART in Fig. 11(b). Consequently, the results in the three experiments show the advantage of ASART in dealing with extremely noisy and limited-angle conditions, yielding better results than those of WBP and SART at the same size.
Fig.11. In the medium-sized data (1024 1024), one of slices along the Z-axis of the reconstructions of the mitochondria by (a) WBP, (b) 50 iterations of SART, (c) 50 iterations of ASART. Mitochondria membrane structures (indicated by the rectangles) in (c) are much clearer than those in (a) and (b).
In our study, we adopt a projection error e criterion for the comparison of WBP, SART and ASART in a practical situation where scL and corr cannot be measured due to the absence of original 3D images. e measures the mean discrepancy between the ray integral pi and its calculated value p0i , defined as follows:
"
1 XM ðpi pi ’Þ2 e¼ i¼1 M ni
#1=2 ð15Þ
286
a
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287 Table 3 Comparison of running times of SART and ASART.
40 WBP SART ASART
35
Projection error
30
25
20
15
10
5
0
5
10
15
20
25
30
35
40
45
45 WBP SART ASART
40
Projection error
35
30
25
20
15
10
0
5
10
15
20
25
30
35
40
45
50
Iteration number
c
55 WBP SART ASART
50
Datasets
SART
ASART
1 5 10 25 50 1 5 10 25 50 1 5 10 25 50
512 512 512 512 512 512 512 512 512 512 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 2048 2048 2048 2048 2048 2048 2048 2048 2048 2048
335.239 1337.134 2628.233 6433.187 12801.113 1566.401 6604.974 12030.124 29401.317 58211.496 28371.374 103486.574 198433.440 481746.325 954251.284
348.238 1326.283 2575.798 6302.253 12508.239 1584.924 6584.325 11919.362 29153.283 57651.205 28426.244 103424.000 198225.922 481320.963 953344.002
50
Iteration number
b
Iterations
versus the number of iterations are presented in Fig. 12. In the three different experiments, e of the reconstructions with ASART is smaller than that with SART in the same iterations. As shown in Fig. 12, e of the reconstructions with WBP remain constant, with the value of 15.08 in the small-sized data, the value of 20.35 in the mediumsized data, and the value of 23.94 in the large-sized data. In the three different experiments, e of the reconstructions with ASART is smaller than that with SART in the same iterations. In the first experiment, after 25 iterations, the values of e obtained by ASART are smaller than that of WBP. In the second and third experiments, the values of e become smaller than that of WBP from 10 iterations. Consequently, it is shown that ASART has the advantage in dealing with extremely noisy conditions, and yielding better results than those of WBP and SART. In order to compare the computational costs for different datasets and different number of iterations, the processing times are listed in Table 3 (in seconds). We can observe that ASART requires more time for one iteration for the 3D reconstructions of the three datasets than SART. Nevertheless, as shown in Table 3, after five and more iterations, the running times of ASART are less than those of SART owing to CSS. The general rule is that the larger dataset and the more iterations performed, the less time is consumed by ASART than SART. As a result, ASART has a faster convergence speed than SART.
Projection error
45 40
5. Conclusions
35
ET is an important imaging technique for 3D reconstruction of cellular structures. From a set of projection images taken from a single individual specimen, the 3D reconstruction can be obtained with tomographic reconstruction methods. However, owing to the limitation of incomplete information and extremely noisy conditions in ET, high-quality reconstructions cannot be acquired by means of existing reconstruction methods (e.g. WBP and SART). In this study, we have developed an adaptive iterative method named ASART in order to obtain high-quality reconstructions. The contributions of this paper include a scheme MMAS for the organization and sequence of access to projections, a strategy AAR for parameter adjustment to correct the discrepancy between real projections and computed ones in each iteration, and a strategy CSS to process the weights for each view to reduce memory requirement and computation time. Combining the three techniques developed in this paper, ASART can speed up the reconstruction process as well as obtain high quality of 3D reconstruction of ET under the condition of noisy and incomplete data. ASART has been implemented in C, and extensive experiments have been carried out. Experimental results demonstrate
30 25 20 15
0
5
10
15
20
25
30
35
40
45
50
Iteration number Fig.12. Plots of the projection errors e versus the iteration numbers for the reconstructions of the caveolae using WBP, SART and ASART in (a) the small resolution (512 512), (b) the medium resolution (1024 1024) and the large resolution (2048 2048).
where ni ¼ wi wi ; and wi is a J-dimensional vector whose jth component is wij. In the 3D data reconstruction, the qualities of the reconstructions, with SART and ASART respectively, have been evaluated based on the measure e. In general, a smaller projection error indicates a better reconstruction. The curves of the measure e
X. Wan et al. / Journal of Structural Biology 175 (2011) 277–287
that ASART outperforms WBP and SART, especially under the noisy and incomplete data condition. Some topics are worthy of further research, including more accurate reconstruction requirements, new models for reconstruction for higher quality, iterative step adjustments to further speed up convergence. Besides, an extensive use of computational resources and considerable processing time are required for large reconstruction volumes of ET. Acknowledgments This work was supported by grants National Natural Science Foundation for China (60736012, 61070129 and 61003164); Chinese Academy of Sciences knowledge innovation key project (KGCX1-YW-13); National Core-High Tech-basic Program (2011ZX01028-001-002). Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.jsb.2011.06.002. References Andersen, A.H., Kak, A.C., 1984. Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm. Ultrasonic Imag. 6, 81–94. Bilbao-Castro, J.R., Carazo, J.M., Fernandez, J.J., 2006. Parallelization of reconstruction algorithms in three-dimensional electron microscopy. Appl. Math. Mod. 30, 688–701. Bilbao-Castro, J.R., Marabini, R., Sorzano, C.O.S., Garcia, I., Carazo, J., Fernandez, J.J., 2009. Exploiting desktop supercomputing for three-dimensional electron microscopy reconstructions using ART with blobs. J. Struct. Biol. 165, 19–26. Censor, Y., 1983. Finite series-expansion reconstruction methods. Proc. IEEE 71 (3), 409–419. Castano-Diez, D., Mueller, H., Frangakis, A.S., 2007. Implementation and performance evaluation of reconstruction algorithms on graphics processors. J. Struct. Biol. 157, 288–295. Fernández, J.J., Lawrence, A.F., Roca, J., García, I., Ellisman, M.H., Carazo, J.M., 2002. High performance electron tomography of complex biological specimens. J. Struct. Biol. 138, 6–20. Frank, J. (Ed.), 2006. Electron tomography: Methods for Three-Dimensional Visualization of Structures in the Cell, second ed. Springer, New York. Guan, H., Gordon, R., 1994. A projection access order for speedy convergence of ART (algebraic reconstruction technique): a multilevel scheme for computed tomography. Phys. Med. Biol. 39, 2005–2022.
287
Guan, H., Gordon, R., 1996. Computed tomography using algebraic reconstruction techniques (ARTs) with different projection access schemes: a comparison study under practical situations. Phys. Med. Biol. 41, 1727–1743. Guan, H., Gordon, R., Zhu, Y., 1998. Combining various projection access schemes with the algebraic reconstruction technique for low-contrast detection in computed tomography. Phys. Med. Biol. 43, 2413–2421. Guan, H., Yin, F., Zhu, Y., Kim, J.J., 2000. Adaptive portal CT reconstruction: a simulation study. Med. Phys. 27, 2209–2214. Herman, G., 1980. Image Reconstruction from Projections: The Fundamentals of Computerized Tomography. Academic Press, New York. Herman, G., Meyer, L., 1993. Algebraic reconstruction can be made computationally efficient. IEEE Trans. Med. Imag. 12 (3), 600–609. Huang, X., Song, X., Zhu, P., 2010. Cryo electron tomography and its application to biological research. Acta Biophys. Sin. 26 (7), 570–578. Jiang, M., Wang, G., 2003. Convergence studies on iterative algorithms for image reconstruction. IEEE Trans. Med. Imag. 22, 569–575. Kazantsev, I., Matej, S., Lewitt, R.M., 2005. Optimal ordering of projections using permutation matrices and angles between projection subspaces. Electronic Notes Discrete Math. 20, 205–216. Lewitt, R.M., 1992. Alternative to voxels for image representation in iterative reconstruction algorithms. Phys. Med. Biol. 37, 705–716. Marabini, R., Rietzel, E., Schroeder, E., Herman, G., Carazo, J., 1997. Threedimensional reconstruction from reduced sets of very noisy images acquired following a single-axis tilt schema: application of a new three-dimensional reconstruction algorithm and objective comparison with weighted backprojection. J. Struct. Biol. 120, 363–371. Marabini, R., Herman, G.T., Carazo, J.M., 1998. 3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs). Ultramicroscopy 72, 53–65. Matej, S., Lewitt, R.M., 1995. Efficient 3D grids for image-reconstruction using spherically-symmetrical volume elements. IEEE Trans. Nucl. Sci. 42, 1361–1370. Matej, S., Lewitt, R.M., 1996. Practical considerations for 3D image reconstruction using spherically symmetric volume elements. IEEE Trans. Med. Imag. 15, 68– 78. Mueller, K., Yagel, R., Cornhill, J.F., 2002. The weighted-distance scheme: a globally optimizing projection ordering method for ART. IEEE Trans. Med. Imag. 16 (2), 223–230. Radermacher, M., 2006. Weighted back-projection methods. Electron Tomogr. 245, 273. Shufeng, S., Kai, Z., Wei, X., Gang, W., Jianjun, C., Fei, S., 2009. 3D structural investigation of caveolae from porcine aorta endothelial cell by electron tomography. Prog. Biochem. Biophys. 36 (6), 729–735. Sorzano, C., Marabini, R., Boisset, N., Rietzel, E., Schroder, R., Herman, G., Carazo, J., 2001. The effect of overabundant projection directions on 3D reconstruction algorithms. J. Struct. Biol. 133 (2–3), 108–118. Wenkai, L., 2004. Adaptive algebraic reconstruction technique. Med. Phys. 31, 3222–3230. Xiaohua, W., Fa, Z., Zhiyong, L., 2009. Modified Simultaneous Algebraic Reconstruction Technique and its Parallelization in Cryo-electron Tomography. ICPADS’, 2009. Zhang, K., Zhang, Y., Hu, Z., Gang, J., Sun, F., 2010. Development and Frontier of electron microscopy 3D reconstruction. Acta Biophys. Sin. 26 (7), 533–559.
