2009 15th International Conference on Parallel and Distributed Systems
Modified Simultaneous Algebraic Reconstruction Technique and its Parallelization in Cryo-electron Tomography Xiaohua Wan1,2, Fa Zhang1, and Zhiyong Liu1 2 Institute of Computing Technology Graduate University Chinese Academy of Sciences Chinese Academy of Sciences Beijing, China Beijing, China
[email protected],
[email protected],
[email protected]
1
microscopes and (ii) WBP does not implicitly take into account the noise conditions nor the transfer function of electron microscopes [5]. Iterative methods could constitute one of the main alternatives to WBP mainly owing to their flexibility to enrich the model with physical information just by adding side conditions into the mathematical problem [6]. For example, SART (simultaneous algebraic reconstruction technique), a classical iterative method, uses an appropriate weighting factor to reflect the differences between actual and computed projections [7]. In general, these methods can perform even smoother solutions under extremely noisy conditions and show better results under limited-angle conditions than WBP [8][9]. Despite their potential advantages compared to WBP, these iterative methods still have not been used extensively mainly due to their huge computational demands. Recently, most of software packages used in electron tomography, such as IMOD [10], SPIDER [11], BSOFT [12], have implemented parallel reconstruction algorithms with WBP and iterative methods to reduce the processing time. However, the high computational cost of 3D reconstruction still remains a major challenge in electron tomography. Addressing the disadvantages of WBP and iterative methods, we present a modified simultaneous algebraic reconstruction technique (MSART) in this paper. Despite MSART is similar to SART, we apply several key techniques to improve the SART algorithm, which can increase convergence rate and obtain higher-quality results. In MSART, the back projection technique (BPT) is firstly employed to evaluate the initial approximation so as to increase the convergence rate. Furthermore, a data-driven adjustment and amplitude constraints are applied to correct the discrepancy between actual projections and computed projections. This adaptive adjustment can improve significantly the quality of reconstruction. Facing the high computational cost of 3D reconstruction in cryo-ET, a 3D reconstruction parallel scheme for MSART is presented in this study. In parallel processing, the 3D reconstruction problem is decomposed into a set of independent 2D reconstruction problems, which well suited for paralleling. Also, several strategies are introduced to resolve data storage problems and communication problems.
Abstract—Three-dimensional reconstruction of cryo-electron tomography (cryo-ET) has emerged as the leading technique in analyzing structures of complex pleomorphic cellulars. A classical iterative method, simultaneous algebraic reconstruction technique (SART), has been employed to reconstruct volume images in cryo-ET. However, SART starts with an arbitrary approximation and takes into account only a weighted factor when updating density value in every errorcorrection iterative procedure, thus limits the improvement of the reconstruction resolution. Facing these problems, we present a modified simultaneous algebraic reconstruction technique (MSART) which applies several key techniques, a back projection technique (BPT) and an adaptive adjustment of corrections. Experimental results show that MSART can improve significantly the quality of reconstruction. Additionally, in order to address the computational requirements demanded by the reconstruction of large volumes, we have presented and implanted a strategy to parallel the MSART algorithm on DAWNING 4000H cluster system, and obtained a good computational performance. Keywords-cryo-electron tomography; 3D reconstruction; iterative method; modified simultaneous algebraic reconstruction technique (MSART); parallel algorithm
I.
INTRODUCTION
Cryo-electron tomography (cryo-ET) combines electron microscopy and the principles of tomographic imaging to reconstruct the three-dimensional (3D) structures of complex biological samples studied at molecular resolution [1][2]. In cryo-ET, the main strategy is to employ electron microscopes to collect projections of a sample that is tilted in multiple directions at cryogenic temperatures, and then utilize these projections to reconstruct the sample in its entirety [3][4]. The projection data in cryo-ET increases constantly with the aim to attain improved resolution so that 3D reconstruction in cryo-ET requires an extensive use of computational resources and considerable processing time. Therefore, it is significant need to handle the computational challenges in 3D reconstruction of cryo-ET. Weighted backprojection (WBP) [3] has been one of the most popular methods in the field of 3D reconstruction of cryo-ET, due to its algorithmic simplicity and computational efficiency. Nevertheless, the main disadvantages of WBP are that (i) the results may be strongly affected by limited tilt angle data due to the limited tilt capabilities of electron 1521-9097/09 $26.00 © 2009 IEEE DOI 10.1109/ICPADS.2009.21
384
Experimental results on DAWNING 4000H cluster system show that 3D reconstruction parallel scheme for MSART can achieve high and stable speedups. The rest of the paper is organized as follows. Section 2 is reviewed the principle of 3D reconstruction for cryo-ET. The SART algorithm is also briefly described. Section 3 is focused on the MSART algorithm. Section 4 presents the 3D reconstruction parallel scheme for MSART and its parallelization on DAWNING 4000H cluster system. Section 5 shows and analyzes in detail the experimental validation. Finally the conclusions of the work are summarized in the last section. II.
WX
P
where the weighting matrix W, with the element wij denoting the contribution of xj to pi, is the matrix mapping X to P [15][16]. We assume that the width of each ray is equal to the image pixel size. Then the element wij can be calculated (or, at least, estimated) according to the projecting procedure as follows:
1, the ith ray passes the center of the jth pixel ® ¯0, else
wij
THE PRINCIPLE OF 3D RECONSTRUCTION FOR CRYOET
Under this model, the aim is to estimate the unknown value xj of each pixel from the known projection data P by means of iterative methods.
A. Series expansion methods scheme In cryo-ET, projection images are acquired from the specimen by following the so-called single-axis tilt geometry. The specimen is tilted over a range, typically from -60° (or 70°) to +60° (or +70°) due to physical limitations of microscopes, at small tilt increments (1° or 2°). An image of the same object area is then recorded at each tilt angle. The 3D reconstruction problem in cryo-ET is then to obtain the 3D structure of the specimen from a set of projection images. The single tilt axis geometry typically used in cryo-ET allows decomposition of the 3D volume reconstruction into multiple reconstructions of slabs of slices orthogonal to the tilt axis (see sketch in Fig. 1). In this work, a single cube named voxel is a basic element of a 3D volume and the voxel is a basic function. Accordingly, the 3D reconstruction problem can be divided into a set of 2D reconstruction problems, each being the reconstruction in a plane perpendicular to the tilt axis, which can be done by WBP [13] or iterative methods [14]. As is analyzed above, the iterative methods are utilized for 2D reconstruction of one slice in this work. As shown in Fig. 1, a 2D slice can be divided into N square pixels, represented by an N-dimensional column vector X. Each pixel has a gray value xj indexed by j (j=1, …, N). Similarly, 1D projection data can assembled into an Mdimensional column vector P, with elements pi (i=1, …, M). Series expansion methods scheme assumes that the model of the image formation process as the following linear system:
B. Simultaneous algebraic reconstruction technique (SART) Although WBP is the standard reconstruction method in cryo-ET, and appears to be more common for reconstructing 3D volume, it is worth to notice that iterative methods (such as SART) could have the potential to outperform backprojection methods, mainly owing to their flexibility to enrich the model with physical information just by adding side conditions into the mathematical problem [17]. SART is a basic iterative method designed to solve the linear system which appears in image reconstruction. Typically, the algorithm begins with an arbitrary X(0) and repeats the iterative processes [18]. In iterations, the residuals, i.e. the differences between the actual projections P and the computed projections of the current approximation X(k) (k is the iterative number, k=0, 1, …), are computed and then X(k) is updated by the backprojection of these discrepancies. Thus, the algorithm produces a sequence of N-dimensional column vectors X(k). The SART algorithm is typically written by the following expression:
xj
( k 1)
xj
(k )
O
¦
M i 1
S
wij
¦ s 1
wij ( pi ¦ h 1 wih xh ( k ) ) N
¦
N h 1
wih
where wi (for 1iM) denotes the N-dimensional vector whose jth component is wij; Ȝ is the relaxation parameter; S is the number of projections per direction; B is the number of all directions; b=(k mod B) is the index of the block; i=bS+s denotes the ith equation of the system; xj(k+1), which is the next iterative value by updating xj(k), is expected to be closer to the real data than xj(k). The SART algorithm adopts an ordered-subset strategy: an approximation is updated simultaneously by all the projections of each view. The SART algorithm updates each xj only once per view, which means its updating strategy is view-by-view. As analyzed above, SART starts with an arbitrary approximation and then begins to iterate until convergence.
Figure 1. 3D reconstruction from single-tilt axis data with voxel as a basic function
385
It is possible that the arbitrary initial value deviates from the true value far. So the number of iterations may be large until convergence. And the backprojection of the discrepancies only considers the weighted matrix W, which is unreasonable because the projection data P is contributed by not only the geometry W but also the gray value X.
MODIFIED SIMULTANEOUS ALGEBRAIC RECONSTRUCTION TECHNIQUE (MSART)
Facing the shortcomings in SART discussed above, we present the MSART algorithm utilizing several key techniques to improve the SART algorithm, which can generate high-quality reconstructions with improved computational efficiency. These key techniques involve BPT to evaluate the initial solution and a data-driven adjustment of correlations.
M ¦ wij pi i 1
M ¦ wij , j i 1
1, 2, , N
C. The MSART reconstruction algorithm According to several improvements mentioned above, the MSART algorithm is denoted in the form as follows: x ( 0 ) M¦ w p ij i ° j i 1 ° ° ® ° x j ( k 1) x j ( k ) ° °¯
A. BPT to evaluate the initial approximation SART starts with an arbitrary solution X(0) to be a vector with all zero elements in general. Thus the value of the iterative number k may be very large depending upon the initial guess. In our algorithm, we adopt BPT to estimate the first approximation X(0) in order to decrease the value of k. BPT is the standard reconstruction method where the gray value of a pixel can be considered as the weighted average of the projections of all the possible rays passing through the pixel. Consequently, the initial solution X(0) is defined by
(0) xj
N ( k ) ¦ wih xh h 1
With this technique, the correction for each pixel is adjusted according to its value obtained in the last iteration. It is more reasonable to adjust the pixels that have a larger value with a larger adjustment step. This adaptive adjustment of the correction leads to not only speedy convergence but also high-quality reconstruction.
III.
N wi j x j ( k ) ( pi ¦ wih xh( k ) ) h 1
M ¦ w ij , j i 1
O M ¦ w ij i 1
1, 2 , , N
S
¦
s 1
wi j x j ( k ) ( pi
N
¦w
ih
xh(k ) )
h 1
N
¦w
ih
xh(k )
h 1
The MSART correction terms are simultaneously applied for all the rays per view, which means a view-by-view updating policy the same as SART. The pixel update procedure of MSART can be broken down into several steps, illustrated as follows. MSART algorithm 1) Initialize volume Compute the weighted matrix W and the initial X(0) by BPT. Until convergence Select the bth block 2) Image projection: Compute line integrals for all rays in bth block. N p ' ¦ w x (k ) i h 1 ih h
In spite of the artifacts from the reconstruction process involved in BPT, the initial solution X(0) obtained via BPT is much closer to the true value than the arbitrary initial solution with all zero elements. B. Adaptive adjustment of corrections This technique is an extension to the SART algorithm that involves the adaptive adjustment of corrections during each iterative process. Deliberate selection of the correction can lead to better-quality reconstruction, yet usually at the expense of convergence. In SART, the value of X is corrected by the weighting value W and the discrepancy between the actual projections P and the computed projections in iterations. In the projection and reconstruction procedure, nevertheless, the data contributed are the weighting factor W and the density value X. The deviations arise due to the difference between the pixel values of the original image and the reconstructed image. Apparently, correcting the deviations only in terms of W is lopsided. So in MSART, a data-driven adjustment of the correction is given by considering the major factor X added to the correction as follows:
3) Correction images computation: For all pi in the bth block a) Subtract the calculated line integrals from the pi's; b) Normalize by the calculated line integrals. 'pi
pi pi ' pi '
4) Image backprojection: a) Distribute normalized corrections, weighted by wij and xj onto pixels; b) Normalized by accumulated corrections by the sum M ¦ wij ; of weights i 1 c) Scale by ¬.
386
'x j
S
O ¦ wij x j ( k )' pi s 1
M ¦ wij i 1
5) Update image: For all xj refine the current approximation xj by adding xj xj
IV.
( k 1)
x
(k )
j
'x
j
IMPLEMENTATION OF PARALLEL MSART FOR 3D RECONSTRUCTION
Parallel computing has been widely investigated for many years as an effective means for huge computational needs in cryo-ET [2]. Previous works on parallelization in image reconstruction have dealt with either the problem of 3D reconstruction by WBP [17] or the problem of 2D image reconstruction from 1D projections [15]. Yet parallelization is straightforward by using voxels as basic functions. This study has implemented the parallel strategy of MSART on cluster systems.
Figure 2. The 3D volume data decomposition and assignment
1) Initiation: Make a guess for X(0) using by BPT and compute the weighted matrix W; 2) Reprojection: Estimate the computed projection data P’ based on the current approximation X; 3) Comparison: Compare and compute the discrepancy P between the real experimental and calculated projections; 4) Backprojection: Backproject the comparison results over the image space and obtain X; 5) Refinement: Update the current approximation X by incorporating the weighted backprojection X. Fig. 3 shows how to perform the 2D reconstruction subproblem by means of the aforementioned stages. Beginning with the first step of the reconstruction, the initiation makes an estimation on the initial solution X(0) and the weighted matrix W with the index k=0. And then an iterative process involves the next four steps: reprojection, comparison, backprojection and refinement. In the initiation, X(0) is obtained with the BPT technique and the weighted matrix W is defined as a sparse matrix with M rows and N columns. The element wij of W denotes the contribution of xj to pi in terms of the principle that wij is given “1” when the ith ray passes through the center of the pixel xj and is assigned “0” otherwise as described in Section 2. Due to relatively few nonzero value in W, the sparse matrix W employs an array of lists of entries (one list for each row) to decrease its memory storage.
A. 3D reconstruction parallel scheme With the properties of cryo-ET 3D reconstruction, decomposition into independent slices or slabs makes the process well suited for paralleling. Parallelization is straightforward because the 3D reconstruction problem can be decomposed into a set of independent 2D reconstruction problems. The implemented parallel strategies make use of a message passing interface (MPI), a standard in parallel programming, and a single program multiple data (SPMD) approach to perform the coarse-grained parallelization of the reconstruction. And a shared-memory parallel programming (OpenMP) is employed to perform the fine-grained parallel. The hybrid implementation that combines the messagepassing and shared-memory paradigms with MPI and OpenMP has been proven to be better than implementations based solely on MPI [19]. The 3D volume is decomposed into several slabs of onevoxel-thick slices. These slabs of slices orthogonal to the tilt axis of the volume are assigned to an individual node on a cluster. The number of slabs equals the number of nodes. When voxels as basic functions are used, the slabs of slices are independent. So the computing nodes then process the 3D reconstruction of the separated slabs in parallel. And each slab is subdivided into several slices, assigned to all processors to compute in parallel in each node. Consequently, the nodes communicate with each other only to complete the final result of the 3D reconstruction. Fig. 2 illuminates how to decompose and assign the 3D volume data according to the cluster system. B. 2D reconstruction scheme Assuming that the volume to be reconstructed is made up of voxels, the 3D reconstruction can be decomposed into a set of independent 2D reconstruction of slabs of slices orthogonal to the tilt axis. In this work, the generic iterative 2D reconstruction is described as follows:
Figure 3. Flow of the iterative algorithm for 2D reconstruction
387
V.
EXPERIMENTAL RESULTS
The primary purpose of this study is to present the MSART reconstruction algorithm to further improve the reconstruction efficiency of the traditional iterative SART algorithm and to implement the parallelization of the MSART algorithm on the cluster in acceptable computing time. Therefore, the evaluation of the MSART algorithm is presented by comparing with the SART algorithm from two aspects. One evaluation is carried out in terms of the quality of the reconstructions coming from SART and MSART. The other evaluation about the performance of parallel MSART is performed by comparison of running time and speedup on DAWNING 4000H cluster system. Artificial and real data have been utilized for the reconstruction of single axis tiltseries in the experiments presented below. A. 2D artificial data reconstruction The Shepp-Logan phantom shown in Fig. 4 is used as artificial data in following experiments. The size of the phantom is 171×171. In all experiments, the views are arranged from 0eto 178eat an equal angle interval 2°, that is the number of views in the experiment is 90. In each view, there are 245 equally spaced parallel rays. The width of each ray is equal to the image pixel size. For convenience of description, in the following section, an iteration means that all rays are used to update the reconstruction once. For a visual comparison, the 2D images of the Shepp-Logan phantom are reconstructed using 3 methods: SART, SART with BPT, and MSART respectively, as shown in Fig. 4. Comparing the results of the second column with the third column (e.g. see Fig. 4(e) and Fig. 4(f) in 5 iterations), we can realize clearly that BPT can improve the convergence rate. And it is seen in the last column results that MSART adopting adaptive adjustment of the correction has faster convergence speed and higher reconstruction quality than SART. These reconstructed images indicate that MSART applying BPT and adaptive adjustment can generate promising results. Furthermore, the overall reconstruction quality is evaluated by determining a peak signal to noise ratio (PSNR) between the reconstructed image and the original image. The PSNR is most commonly used as a measure of quality of reconstruction of image [19]. And the term for two m×n monochrome images I and K is defined as follows:
PSNR
Figure 4. Simulation results of the reconstructions of “phantom” with different techniques. For the reconstruction, 90 projection images have been used and no missing wedge is applied. (a) Original image, (b-d) results of 1 iteration of SART, SART with BPT and MSART respectively. (e-g), (h-j), (k-m), (n-p) and (q-s) are the results of 5, 10, 25, 50, 100 iterations respectively.
It is clearly shown that the higher PSNR can be achieved as the number of iterations increase in three techniques, which means the quality of the results reconstructed can be higher with more iterations. Comparing the experimental results, we can observe that the techniques of BPT and the adaptive adjustment of the correction can improve the PSNR of the reconstructed image. Results also show that for a limited number of iterations a considerably higher reconstruction PSNR can be obtained by the MSART algorithm. TABLE I.
COMPARISON OF THE PSNR OF THE FOUR TECHNIQUES DEPENDING ON THE NUMBER OF ITERATIONS
m n
Iterations
SART
i 1j 1
1
-18.0127
-10.605
-17.1300
5
-11.2564
-1.6158
11.5470
10
-5.5823
2.6588
11.6780
25
-2.5301
7.4408
12.0830
50
1.8543
9.9810
12.5410
100
4.2380
11.1270
13.4770
10 log10 mn MAX i2 ¦ ¦ [ I (i , j ) K (i , j )]2
Here, MAXi is the maximum possible pixel value of the image. A higher PSNR would normally indicate that a reconstruction is of higher quality. We calculated the PSNR of the reconstruction results obtained by adopting different improvements in MSART compared with SART, as listed in Table ĉ.
388
SART with BPT MSART
B. 3D real data reconstruction In order to demonstrate the applicability of the MSART algorithm on real data, a cryo-ET 3D reconstruction of the sample of the caveolaes from porcine aorta endothelial (PAE) cell has been performed with WBP and the iterative algorithms involved with SART and MSART [20]. The single data set comprises 56 images taken at various rotations around a single tilt axis covering an angular range of 110e and the size of each projection image is 512h512. Furthermore, we have implemented the parallel strategy of MSART of the 3D reconstruction on Linux platform, and tested on DAWNING 4000H cluster system. The experiments are implemented on the different number of nodes to complete the coarse-grained parallelization of 3D reconstruction. In DAWNING 4000H, each computing node includes two Intel Xeon 2.80GHz processors on which to implement the fine-grained parallelization. The processing times, according to the different number of the processors used and the iterations performed, are presented in Table Ċ. The processing time is in seconds. In order to compare the relative computation costs of the different iterations of MSART, the respective speedup obtained on the different number of nodes is computed. Speedup is the metric that is most often used to assess parallel performance [4]. Fig. 5 describes the speedup curves for the different iterations of MSART versus the number of nodes of the cluster. It is clearly observed that the higher acceleration can be achieved as the number of iterations increases. Also, the improvement of speedups is decreased comparatively with the increasing of the number of nodes, as shown in Fig. 5. This is because the more time is consumed for the communication between the different nodes as more nodes are added.
Figure 5. Graphs of speedup for various iterations of MSART versus number of nodes
A montage showing one z-section of the volume reconstructed with the different algorithms, WBP, SART and MSART, is presented in Fig. 6. Since WBP is the standard method in the field, we compare the effect of WBP with the iterative methods, SART and MSART. In Fig. 6, the results of one slice along the Z axis coming from sequential WBP and SART are obtained on one node of DAWNING 4000H. However, one slice of the 3D volume reconstructed with the parallel MSART algorithm is implemented on 36 nodes of DAWNING 4000H.
TABLE II.
COMPARISON OF RUNNING TIME OF DIFFERENT ITERATIONS DEPENDING ON THE NUMBER OF NODES
Iteration
5
10
50
100
processors 1
1671.941 3130.246 17652.647
26990.023
8
427.661
687.125
2982.361
6428.163
16
230.668
405.584
1576.581
3232.368
24
214.704
367.945
1481.113
2323.006
32
123.704
189.443
950.248
1647.314
40
107.540
164.375
840.793
1418.248
48
94.232
152.203
663.631
1211.047
56
87.384
147.191
630.043
1113.466
64
77.291
138.566
547.371
1055.715
72
76.597
119.731
535.909
945.775
(a)
(b)
389
China [90612019 and 60736012]; and CAS knowledge innovation key project [KGCX1-YW-13]. REFERENCES [1]
[2]
[3] [4] [5]
(c) Figure 6. Computational one slice with identical thickness through the reconstruction of the caveolaes by (a) WBP on one node of DAWNING 4000H, (b) 100 iterations of SART on one node of DAWNING 4000H, (c) 100 iterations of MSART on 36 nodes of DAWNING 4000H. The caveolaes of the PAE cell are nicely visible. All reconstructions converge to a similar solution with MSART showing the best contrast.
[6]
[7]
Apparently, the quality of the image with MSART in Fig. 6(c) is superior to the images from WBP in Fig. 6(a) and SART in Fig. 6(b). The outlines of the caveolaes in the sample PAE cell are clearly visible in Fig. 6(c). Consequently, it is shown that the potential of the MSART algorithm deals with extremely noisy conditions, yielding solutions much cleaner than WBP and SART. VI.
[8]
[9] [10]
CONCLUSIONS
[11]
In this study, the adaptive iterative algorithm MSART is presented, which can make an improvement in quality for the cryo-ET 3D reconstruction especially on the limited tilt angle conditions, due to the fact of three improved techniques in MSART, BPT to estimate the first approximation, the adaptive adjustment of corrections to increase convergence rate. Experimental results on artificial and real data show that MSART can significantly increase the convergence rate and improve the quality of reconstruction. Also a 3D reconstruction parallel scheme for MSART is presented, according the nature property of cryo-ET. Experimental results on DAWNING 4000H cluster system show that 3D reconstruction parallel scheme for MSART can achieve high and stable speedups. We will focus our future work on how to improve further the quality of 3D reconstructions and obtain a higher effective computation rate on GPU, as well as cluster system.
[12]
[13] [14]
[15]
[16]
[17]
[18]
ACKNOWLEDGMENT
[19]
We would like to thank Fei Sun and Ka Zhang for providing the experimental datasets. This work was supported by the National Natural Science Foundation for
[20]
390
J.J. Fernandez, “High Performance Computing in Structural Determination by Electron Cryomicroscopy”, J. Struct. Biol, vol. 164, 2008, pp. 1-6. J.J. Fernandez, A.F. Lawrence, J. Roca, I. Garcia, M.H. Ellisman, and J.M. Carazo, “High Performance Electron Tomography of Complex Biological Specimens”, J. Struct. Bio., vol. 138, 2002, pp. 6-20. J. Frank, Three Dimensional Electron Microscopy of Macromolecular Assemblies, Oxford University Press, 2005. J. Frank, Electron Tomography: Methods for Three-Dimensional Visualization of Structures in the Cell, Springer Press, 2006. Diez. D. Castano, H. Mueller, and A. S. Frangakis, “Implementation and Performance Evaluation of Reconstruction Algorithms on Graphics Processors”, J. Struct. Biol., vol. 157, 2007, pp. 288-295. R. M. Lewitt, “Alternatives to Voxels for Image Representation in Iterative Reconstruction Algorithms”, Phys. Med. Biol., vol. 37, 1992, pp. 705-716. K. Mueller and R. Yagel, “Rapid 3D Cone-Beam Reconstruction with the Simultaneous Algebraic Reconstruction Technique (SART) Using 2D Texture Mapping Hardware”, IEEE Trans. Medical Imaging , vol. 19, 2000, 1227-1237. A. H. Andersen, “Algebraic Reconstruction in ET from Limited Views”, IEEE Transactions on Medical Imaging, vol. 8, 1989, pp. 5055. M. Jiang and G. Wang, “Convergence Studies on Iterative Algorithms for Image Reconstruction”, IEEE Trans. on Med. Imag, vol. 22, 2003. J. R. Kremer, D. N. Mastronarde, and J. R. Mclntosh, “Computer Visualization of Three-dimensional Image Data Using IMOD”, J. Struct. Biol., vol. 116, 1996, pp. 71-76. J. Frank, M. Radermacher, P. Penzek, J. Zhu, Y. Li, M. Ladjadj, and A. Leith, “SPIDER and WEB: Processing and Visualization of Images in 3D Electron Microscopy and Related Fields”, J. Struct. Biol., vol. 116, 1996, pp. 190-199. J. B. Heymann, G. Cardone, D. C. Winkler, and A. C. Steven, “Computational Resources for Cryo-electron Tomography in Bsoft”, J. Struct. Biol., vol. 161(3), 2007, pp. 232-242. A. C. Kak, and M. Slaney, Principles of Computerized Tomographic Imaging, IEEE Press, 1988. J. J. Fernandez, J. M. Carazo, and I. Garcia, “Three-dimensional Reconstruction of Cellular Structures by Electron Microscope Tomography and Parallel Computing”, J. Parallel Distrib. Comput., vol. 64, 2004, pp. 285--300. Y. Censor, D. Gordon, and R. Gordon, “Component Averaging: an Efficient Iterative Parallel Algorithm for Large and Sparse Unstructured Problems”, Parallel Comput., vol. 27, 2001, pp. 777-808. X. Li, M. Jiang and G. Wang, “A Numerical Simulator in VC++ on PC for Iterative Image Reconstruction”, J. of X-Ray Science and Technology, vol. 11, 2003, pp. 61-70. J. R. Bilbao-Castro, J. M. Carazo, I. Garcia, and J. J. Fernandez, “Parallelization of Reconstruction Algorithms in Three-dimensional Electron Microscopy”, Applied Mathematical Modeling, vol. 30, 2006, pp. 688--701. A. H. Andersen and A. C. Kak, “Simultaneous Algebraic Reconstruction Technique (SART): a Superior Implementation of the ART Algorithm”, Ultrason. Imaging, vol. 6, 1984, pp. 81-94. B. Wilkinson and M. Allen, Parallel Programming, second ed., Prentice-Hall, 2004. R. V. Stan., “Structure of caveolae”, Biochim. Biophys. Acta., vol. 1746, 2005, pp. 334--348.
