Combining short-axis and long-axis cardiac MR images by applying a super-resolution reconstruction algorithm Sami ur Rahman, Stefan Wesarg Interactive Graphics Systems Group (GRIS), Technische Universit¨at Darmstadt, Fraunhoferstr. 5, 64283 Darmstadt, Germany ABSTRACT In cardiac MR images the slice thickness is normally greater than the pixel size within the slices. In general, better segmentation and analysis results can be expected for isotropic high-resolution (HR) data sets. If two orthogonal data sets, e. g. short-axis (SA) and long-axis (LA) volumes are combined, an increase in resolution can be obtained. In this work we employ a super-resolution reconstruction (SRR) algorithm for computing high-resolution data sets from two orthogonal SA and LA volumes. In contrast to a simple averaging of both data in the overlapping region, we apply a maximum a posteriori approach. There, an observation model is employed for estimating an HR image that best reproduces the two low-resolution input data sets. For testing the SRR approach, we use clinical MRI data with an in-plane resolution of 1.5 mm × 1.5 mm and a slice thickness of 8 mm. We show that the results obtained with our approach are superior to currently used averaging techniques. Due to the fact that the heart deforms over the cardiac cycle, we investigate further, how the replacement of a rigid registration by a deformable registration as preprocessing step improves the quality of the final HR image data. We conclude that image quality is dramatically enhanced by applying an SRR technique especially for cardiac MR images where the resolution in slice-selection direction is about five times lower than within the slices. Keywords: MRI, cardiac imaging, super resolution, image enhancement
1. INTRODUCTION Cardiac magnetic resonance imaging (MRI) is the method of choice for examining the dynamic behavior of the heart due to its superior temporal resolution.1–4 Typically, a series of short-axis (SA) slices covering the whole cardiac cycle is acquired. Based thereon, relevant parameters related to volumetry and cardiac motion are computed.5 This analysis requires a segmentation of the cardiac anatomy, e.g. endocardial and epicardial boundaries have to be extracted from the image data. The better the delineation of these structures can be done, the more accurate the obtained analysis results are. However, cardiac MR image data is highly anisotropic: the slice thickness is several times greater than the in-plane resolution. For achieving good segmentation and analysis results, it is desirable that the image data is provided as isotropic data. This means that the resolution in slice-selection direction is as good as the in-plane resolution, i.e. it is made of cubic voxels. A simple upsampling of the image data may provide cubic voxels, but it does not provide any extra information. A widely used approach for getting isotropic data sets with enhanced information is to use long-axis (LA) slices – being two-chamber (2CH) and four-chamber (4CH) views, respectively – in addition to the commonly acquired SA slices. If both data sets are registered6 and combined to an isotropic data using a simple averaging of the voxels which belong to both image stacks, such high resolution (HR) image data can be computed and used for an improved segmentation.7 But the question arises whether each of the two data sets should contribute equally to the generated HR image at each voxel position. Further author information: (Send correspondence to S.u.R) S.u.R.: E-mail:
[email protected] S.W.: E-mail:
[email protected]
Figure 1. The model of the image acquisition process is used for estimating an HR image f based on the two available LR images g A and g B . Short-axis and long-axis 2CH slices are obtained by transforming, blurring, downsampling of, and adding noise to the HR image to be reconstructed.
This can be answered if the image acquisition process is modeled and this model is used for computing both data sets’ contributions for each voxel of the HR image. Such an observation model describes how the low resolution (LR) images are obtained from a (hypothetic) HR image.8 Thus, we have an inverse problem to deal with: g = ψ ∗ θ ∗ ω ∗ φ(f ) (1) where g is one LR image and f the (hypothetic) HR image. The latter one is modified by a geometric transformation φ, a blurring function ω, a downsampling function θ, and a random sensor noise ψ (Fig. 1). The computation of the contribution of the SA and LA data sets to the HR image can now be considered as an optimization problem where based on the real images g A and g B the HR image f is estimated. For this superresolution reconstruction (SRR) we employ a maximum a posteriori (MAP) approach which has been previously used for the computation of high resolution MR images of the brain.9 Geman and Geman10 have introduced the concept of posterior probability maximization to the image processing community. They formulated the concept of a statistical framework for SRR. For that, they draw an analogy between images and statistical mechanics systems. Pixel gray levels are considered as states of atoms or molecules in a lattice like physical system. The assignment of an energy function in the physical system determines its Gibbs distribution. Because of the Gibbs distribution and Markov random field (MRF) equivalence, this assignment also determines an MRF image model. For a range of degradation mechanisms, including blurring, nonlinear deformations, and multiplicative or additive noise, the posterior distribution is an MRF with a structure similar to the image model. By this analogy, the posterior distribution defines another (imaginary) physical system. Several groups have used this concept in different application areas. Russel et al.11 have extended this framework for the joint MAP registration and HR image estimation. The same group used the concept for the estimation of HR reconstruction for an infrared imaging system.12 Bai et al.9 have employed this method for the SRR of brain MR images. In their work, they used 3D image data represented as axial and coronal slices for computing an HR image. However, the input LR images were already almost isotropic ones (1 mm × 1 mm × 1.5 mm). In this work, we propose the application of an SRR technique for computing a high-resolution cardiac MRI data set from two stacks of SA and LA slices, respectively. We investigate how the image quality is improved compared to a simple averaging technique which has been used in the work of others (see Ref. 7). The SRR
(a) SA original
(b) SA resampled
(c) 2CH original
(d) 2CH resampled
Figure 2. Original volumes with 1.5 mm × 1.5 mm × 8 mm voxel size (left column) and with cubic voxels with size 1.5 mm × 1.5 mm × 1.5 mm after resampling (right column).
algorithm is applied to cardiac MRI images where the resolution is 1.5 mm × 1.5 mm × 8 mm and thereby the in-slice resolution more than five times greater than in axial direction. Thus, the anisotropy of the LR image data is much larger than for the data which has been used in the work of Bai et al.9 We reconstruct an isotropic HR volume grid with a size of 1.5 mm for all directions. Another difference to the SRR of brain MR images is the fact that the brain is not moving during image acquisition, but the heart does. Thus, rigid registration works well for the brain, but for cardiac MR images it is doubtable whether a rigid approach can provide a sufficiently exact registration result. Consequently, we employ a deformable registration scheme and investigate the usability of rigid and deformable registrations for the case of cardiac image data. To the best of our knowledge, no one has used SRR for the combination of SA and LA cardiac images before.
2. IMAGE PREPROCESSING In order to apply the MAP approach for computing the HR image data based on the available SA and LA data sets, an initial estimation of the HR image has to be provided. At this preprocessing step, SA and LA image data sets have to be resampled to the same isotropic voxel size, registered with each other, and finally merged. These three steps are common image processing tasks which are performed employing functionality provided by the Insight Segmentation and Registration Toolkit (ITK)∗ .13 ITK is an open-source, cross-platform software toolkit implemented in C++ which provides algorithms for image analysis, registration, and segmentation of multidimensional image data. See Ref. 14 for more details.
2.1 Resampling of the image data All cardiac test data sets used in this work have a typical 256 × 256 pixel raster per slice and consist of 10 to 20 slices. The pixel resolution for SA and LA slices is 1.5 mm × 1.5 mm with a slice thickness of 8 mm which are standard values used in clinical cardiac image acquisition. An initial resampling of these data sets is done in order to convert them to isotropic image data with cubic voxels with an edge length of 1.5 mm. For that, we used the ResampleImageFilter from ITK setting the interpolation type to (tri-)linear. Fig. 2 shows the result of this step for one of the used test data sets.
2.2 Cardiac imaging characteristics The next preprocessing step is the registration of both resampled data sets. Before discussing that, we want shortly summarize the acquisition of cardiac MR images (see also Ref. 3). For the analysis of cardiac dynamics, so-called cine-MR images are used. Typically, these data sets consist of a series of volumes which cover the whole cardiac cycle. However, the acquisition is not done volume-by-volume until all time steps are recorded, but in a slice-wise manner. This means, a specific slice position is chosen by the clinician. The patient takes a ∗
ITK website: http://www.itk.org
breath, exhales a little bit, and holds the breath. And then for the specified slice position, a series of 2D images is acquired using trigger information from simultaneously recorded electrocardiograpic (ECG) data in order to guarantee an equal temporal distance between the slices. Afterward, the slice position is set to a parallel plane with a certain distance, and another 2D + t data set is acquired. It is obvious that the exhalation position between different 2D + t acquisitions, i.e. different slice positions, is not likely to be exactly the same. Thus, the heart might be shifted between the corresponding slices and cause artifacts when the stack of the 2D + t data is combined to a time series of 3D volumes. In addition, the ECG triggered image acquisition might be incorrect if the heart beats in a non-uniform manner. This means that 2D slices which are considered to belong to the same time step and should therefore belong to the same 3D volume may be acquired at slightly different moments with respect to the cardiac cycle. In that case, they do not represent the same cardiac phase and the reconstructed volume shows artifacts. Consequently, only under rather ideal image acquisition conditions the SA and LA data sets which are considered to be taken at an identical moment within the cardiac cycle will represent the heart in the same pose and shape. But, this is a prerequisite for performing a rigid registration between corresponding SA and LA data sets. If the acquired data shows imaging artifacts which are due to breathing and heart beat, a warping of the image data has to be allowed for achieving acceptable registration results. That is, a deformable registration method has to be used.
2.3 Rigid and deformable registration The SA and LA image data is taken from different perspectives and the two data sets are therefore in different coordinate systems. The SA data is rotated about 90◦ against the LA data set. (For 2CH and 4CH views, this 90◦ rotation is of course with respect to different axes.) The goal of the image registration step is to bring the two images into the same coordinate system, i.e. to compute the transformation that has to be applied to one of the data sets. Since in our case we have two more or less orthogonal data sets, we apply an initial transformation to the LA data set which consists of a translation in order to match the centers of both data sets followed by a 90◦ rotation around the correct axis (depending whether 2CH or 4CH views are used for the SRR). Afterward, a registration algorithm is applied for computing the remaining transformation. For that, we employ again functionality provided by ITK. The ITK transform type used for rigid registration is the VersorRigid3DTransform, the used metric is the MeanSquaresImageToImageMetric. This rather simple metric can be used, since an intra-modality registration is performed.6 In order to reduce the time needed for computing the transformation parameters as well as to achieve good registration results, we clip the volumes to that region of the image data where the left ventricle is localized – one third of the image data’s extent for each dimension. This is possible, because the interesting structures of the heart fit into this clipped volume and this image information is sufficient for yielding satisfying registration results in the region of interest. The computed transformation is afterward applied to the whole image volume. As mentioned above, a rigid registration will not provide correct results at the presence of imaging artifacts due to breathing and heart beat. In this case, a 6 DOF transformation representing translation and rotation can not provide a sufficiently good matching of SA and LA data sets. Consequently, the image data has to be additionally warped, i.e. a deformable registration has to be performed. For that, we use the Demons registration algorithm15 which is implemented in ITK’s DemonsRegistrationFilter.
2.4 Averaging the registered images Once, both image data sets are registered, a fused image data can be computed. We have implemented an ITK-like filter that takes the two registered images as input and applies a voxel-wise mean computation. For regions where both data sets do not overlap, i.e. for one data set there is no information at that voxel position, only the information from the second data set is used without computing the mean. Current methods for combining SA and LA image data simply use this averaged image and perform segmentation and analysis steps thereon.7 Whereas in our case, it is only the initial estimate of the HR image which is afterward iteratively refined employing an MAP approach which we describe in the following two sections.
Figure 3. The blurring step of the observation model. A rect function in z direction corresponds to the slice thickness of the MR scanner. It is upsampled and convoluted with the HR image estimate. The result is a blurred version of the HR image.
3. OBSERVATION MODEL AND MAXIMUM A POSTERIORI APPROACH We follow the approach of MAP reconstruction given in Ref. 9 and apply it to our cardiac images. The basic idea of SRR is as follows: two image volumes – SA and LA images – with orthogonal slice-selection directions are combined into one volume. This way, missing information in one image is obtained from the other one.
3.1 Blurring ω Blurring or point spreading is the concept of smearing of an image source point in the image plane. This means, that a point in the image source is not a point in the image plane, rather the point spreads in the image plane. This is due to the imperfectness of the imaging system. Hence, the point spread function (PSF) defines how a point would appear on the image plane. We follow the argument found in Refs. 8 and 9 and ignore the PSF along the x and y directions, i.e. within the plane of the image slices. Only along the slice-selection direction the PSF is considered. That means, that in our case the PSF reduces from h(x, y, z) to h(z). The PSF in z-direction is a rectangular window function due to the geometry and procedure of the image acquisition.9 Thus, we have h(z) = rect(
z ) ∆z
(2)
where ∆z is the slice thickness given by the MR scanner setup. We use such a rect window function corresponding to the slice thickness of the LR images – 8 mm for our test data. Then, this data is resampled in order to have voxels with a size of 1.5 mm in slice-selection direction too, i.e. the rect function is upsampled. For that, we use a cubic B-spline interpolation due to its superior behavior compared to a tri-linear interpolation. Finally, a convolution between this resampled PSF and the HR image results in a blurred version of the current HR image estimate (see Fig. 3). It has to be noted that we need to obtain two orthogonal observations – the LR images g A and g B . Since the slice selection direction is not the same for the two observations, we perform two different computations φA and ω A as well as φB and ω B (see Fig. 1) corresponding to the two orthogonal slice-selection directions in order to generate the blurred HR image versions.
3.2 Down sampling θ and adding noise ψ The acquired real LR images have a lower sampling rate, i.e. a larger slice thickness, than the estimated HR image. In order to compare the derived LR images with the acquired SA and LA images, a downsampling function θ is applied to the blurred HR image. The slice thickness of this downsampled HR estimation corresponds to that of the acquired LR images. The image acquisition process also adds a random noise ψ to the image data. Thus, we add corresponding to the last step of the observation model independent and identically distributed Gaussian noise with a standard deviation of σ = 3 to the blurred and downsampled image data.
3.3 Posterior probability maximization In MAP based approaches, several estimates of the HR image are considered as possible solutions. From each possible solution f1 , f2 , . . . , fn of an HR image, the LR images g1 , g2 , . . . , gn are computed using the observation model described above (see also Sec. 1). The computed LR images g1 , g2 , . . . , gn are compared with the original LR images. And if the LR image gi which is closest to the original LR image is found – meaning that they have minimum mean square error – then the corresponding fi is considered to be the best estimate of the HR image. Suppose, that we have g1 , g2 , · · · , gk LR observations, where k denotes the number of LR observations, and these observations are independent from each other. Let us further denote the estimate of the HR image f by fˆM AP and the probability by P r. The aim of the MAP approach is to maximize fˆM AP for the k given LR observations. fˆM AP = arg max {P r(f |g1 , g2 , · · · , gk )} (3) f
Using Baye’s rule for posterior probability and a Markov random field9, 10 for the joint probability distribution, the following equation can be obtained. 2 ! Mk n M N N X X X X X 1 1 fˆM AP = arg min wk fn )2 + (g k − (4) αi,j fj f 2σk2 p=1 p n=1 p,n 2λ i=1 j=1 k=1
where σk is the Gaussian noise variance for the observation k, gpk is the pth voxel in the LR image k, fn is the k nth voxel in the HR image estimate, wp,n is the contribution of nth voxel in the HR image to the pth voxel in the LR image k, λ is a tuning parameter, and α represents the Markov random field local potential. For the here presented SRR approach for cardiac images, k has the value 2 – SA image data and LA 2CH image data. If in addition we used a four-chamber (4CH) data set, k would equal to 3.
4. STEPS OF THE SUPER RESOLUTION RECONSTRUCTION In the previous two sections, the theoretical foundation for SRR has been explained. In the following, the several steps of the algorithm (preprocessing, applying the observation model, and gradient-descent optimization) are listed and explained (see also Ref. 9). • Step 1 (Resampling to cubic voxels): Take two orthogonal LR images (SA and 2CH volumes) and resample them into cubic voxels. • Step 2 (Registration): Compute the transformation between the 2CH volume and SA volumes using either a rigid or a deformable registration approach. • Step 3 (Initial estimate of HR image) : Compute an average image for the registered volumes and consider it as the initial estimate of the HR image. • Step 4 (Cost of optimization function) : For finding the cost of the following optimization function (see Eq. 4), 2 ! Mk M N N n X X X X 1 1 X wk fn )2 + (g k − αi,j fj J(f ) = 2σk2 p=1 p n=1 p,n 2λ i=1 j=1 k=1
first compute:
1. LR observations created from the current HR estimate using the observation model. 2. Prior estimation of the current HR estimate by convolving current HR estimate with the Laplacian operator. Find the cost of the optimization function using the actual observations, LR observations created from the current HR estimate and prior estimate of the current HR estimate. • Step 5 (Gradient of optimization function J(f )) : The gradient of the optimization function is given by ! Mk M n N N X X X 1 ∂J(f ) X 1 X wk fn ) + (g k − = αi,j fj . X(f ) = ∂fp σk2 p=1 p n=1 p,n λ i=1 j=1 k=1
Use the computations already done in step 4 by saving those computations in some variables.
• Step 6 (Updating estimate) : Use equation fpc+1 = fpc − ǫc X(fpc ) for updating the SR estimate. For computing the step size, use the following equation: P P PN PpM N N c c α f + λ g − g w f γ m j=1 i,j j i=1 i r=1 m,r r m=1 m ǫc = PpM 2 PN 2 m=1 γm + λ i=1 gi PN PN where γm = r=1 wm,r (gr)r (f n ) and gi = j=1 αi,j gj (f n ).
• Step 7 (Cost of optimization function for updated SR estimate) : Find cost of optimization function using updated SR estimate. Follow the procedure given in step 4. • Step 8 (Stopping criteria) : If optimization function cost of step 7 is less than optimization function cost of step 4, then 1. Current SR estimate = updated SR estimate 2. Go to step 4. Else 1. Final SR estimate = current SR estimate 2. Stop. As a result of this optimization, the HR estimation that provides the best LR images – i.e. minimum mean square error between computed and acquired LR images – is taken as the super-resolution image data. Thus, it represents the final result of the SRR algorithm.
5. RESULTS The evaluation of the SRR approach explained above was threefold. First, we investigated whether an HR image of good quality can be obtained even if the voxels of the acquired LR image data are highly anisotropic – 8 mm in z direction vs. 1 to 1.5 mm in x and y direction. Second, we focused on a comparison of the computed superresolution image with that obtained by a simple averaging method, and there especially the possible improvement of the algorithm outcome when using a deformable registration for the alignment of the two LR images instead of a rigid one. Finally, we compared the results for the segmentation of the left ventricle in standard SA image data and in HR image data obtained by an SRR-based combination of SA and 2CH data sets. Synthetic as well as real cardiac MRI data sets have been used. We want to mention, that we here present initial results obtained by tests with real data sets from only three patients. An extensive evaluation with a larger number of clinical data sets still has to be done.
Figure 4. A synthetic, isotropic data set (256 gray values) containing a regular sinusoidal grid has been generated and downsampled in order to create two different, highly anisotropic LR data sets. The SRR algorithm has been applied for computing an HR image. The difference between the original and the HR image is shown in the rightmost image. Here, a pseudo-color mapping of the values 0 to 31 to a red-to-blue rainbow scale is used for better distinguishing the small but yet existing differences.
5.1 Synthetic data Compared to the work of Bai et al.9 who combined two orthogonal MRI data sets of the brain with nearly isotropic voxels (1 mm × 1 mm × 1.5 mm), we have to deal with a much larger anisotropy of the voxels (1.5 mm × 1.5 mm × 8 mm) in our cardiac MRI data sets. Consequently, first we investigated whether the SRR algorithm does work well also in the case of highly anisotropic voxels. For that, we generated an HR data set containing a regular 3D sinusoidal grid made of 256 × 256 × 256 isotropic voxels with a size of 1 mm × 1 mm × 1 mm and gray values in the range (0 . . . 255) (see Fig. 4). From this data set we created observations representing the virtually acquired LR image data by downsampling it along two orthogonal directions and adding noise. The first downsampled observation had a resolution of 256 × 256 × 32 voxels with a size of 1 mm × 1 mm × 8 mm, the second one a resolution of 256 × 32 × 256 voxels with 1 mm × 8mm × 1mm. We applied the SRR algorithm as described above to the two LR data sets. The registration step could be skipped in this case, since both data sets were perfectly aligned as being downsampled versions of the same initial data set. The results shown in Fig. 4 reveal the high quality of the reconstructed HR image even in the case of strongly anisotropic LR images. Both, original and HR image are visually undistinguishable. Only in the also shown difference image one can see a small disagreement in those regions where the original gray values are in the medium range. In our opinion, this is due to the resampling of the voxels and the interpolation of the gray values made during the downsampling of the original image in order to obtain the LR image data.
5.2 Comparing averaging and SRR For examining the improvement given by the application of an SRR algorithm compared to a simple averaging between SA and LA image data, we used real clinical cardiac MRI data sets acquired with a Siemens Sonata MR scanning device. In subsequent steps, SA as well as 2CH and 4CH data has been acquired. From the available cine-MRI sequences available for three patients, we selected one time step located within the diastole and applied our SRR algorithm to the SA and 2CH data sets. In fact, we could have used in addition the 4CH data, but due to the long computation time (around 20 min on an Intel Quad Core Xeon CPU with 2.6 GHz, single-threaded code) when performing SRR for two orthogonal data sets, we decided to restrict our tests to the combination
Figure 5. Comparison of averaging and SRR technique as well as rigid and deformable registration for three patients. Applying the SRR algorithm clearly improves the image quality: the gray values are distributed more homogeneously, image contrast is enhanced, and small details are better visible (orange arrows). Due to possible artifacts in cardiac image acquisition a deformable registration outperforms a rigid one. In the latter case, image boundaries may become fuzzy in the super-resolution images due to improper image alignment (yellow arrows).
Figure 6. Comparison of segmentation results for the left ventricle of patient 1 in standard SA image data (left) and the generated super-resolution data set (right). The segmentation results based on the HR image data reveal more details of the ventricle, especially the shape and the morphology of the present aneurysm (arrow).
of SA and 2CH image data. The image resolution of the LR images was 256 × 256 voxels in the xy plane, and the number of slices ranged from 10 (SA image data) to 20 (LA image data). The voxel size was for all data sets 1.5 mm × 1.5 mm in the xy plane and 8 mm along the slice selection direction. The volume data sets were approximately orthogonal to each other. In Fig. 5 the SRR results for the three clinical test data sets are shown. There is a noticeable homogenization of the gray values as well as a sharper contrast in the SRR images. Already an averaging of SA and 2CH image data improves the image quality. However, in case where the SRR algorithm has been applied, the image enhancement is even stronger. For patient 1, a contrast enhancement of the small structure representing the papillary muscles can be perceived in both, rigidly and deformable registered super-resolution images. Also for patient 3, we see an improvement for the visibility of small structures – here the mitral valves and the trabecula. The latter one is only clearly separated from the blood pool in case where a deformable registration preceded the application of the SRR algorithm. This is an indication that – as we supposed – a deformable registration is necessary in case where we want to generate super-resolution images of the heart. The necessity of a non-rigid registration is further emphasized, if the sharpness of the right endocardial border in the SRR images shown in Fig. 5 is compared. In the rigid case, this boundary is washed out, and here in terms of sharpness the results of the SRR are even worse than a simple averaging. But, a deformable registration as preprocessing step provides a sufficiently good alignment of both LR data sets in order to lead to superior SRR results.
5.3 Improving image segmentation The enhancement of the image data is not only useful for providing better images for a visual diagnosis made by a cardiologist. It further delivers images where automated segmentation algorithms for the left ventricle which strongly rely on the gray value information contained in the image data16, 17 should perform better. In addition, higher resolution image data allows in general for a more detailed delineation of anatomical structures. As an example, we show the extraction of the left ventricle in the data sets for patient 1 (see Fig. 6). The segmentation in both data sets has been done using an approach which we have developed for the automated extraction of the left ventricle in cine-MRI data sets.16 This patient suffers from an aneurysm in the lower midcavity anterolateral region. When comparing both segmentations of the blood pool, one can observe the much higher detail of the extracted region in case where the super-resolution data set has been used. This has not only the benefit of providing visualizations which are more pleasing to the eye. Also, the planning of a surgical
intervention – in this case ventricular reduction surgery18, 19 – can be done in more detail and furthermore, the derivation of quantitative measures like the left ventricular volume are more exact.
6. DISCUSSION In this paper, we have shown that a combination of SA and LA image data of the heart can be used for an image enhancement. To the best of our knowledge, we are the first who propose to employ an SRR technique for this task. We have shown that even in the case of highly anisotropic voxel data where the resolution in slice-selection direction is more than five times lower compared to the in-slice resolution, high-quality super-resolution image data can be generated using an MAP approach. Thus, we extend the results given by Bai et al.9 who applied a similar algorithm to MRI data of the brain made of nearly isotropic voxels. The second result, which is specific for a proper handling of cardiac image data, is that a deformable registration is mandatory for providing sufficiently well aligned image data. Only in the case where aligned voxels correspond to the same anatomical feature, the SRR approach will generate HR images of improved quality compared to simple averaging approaches. These latter methods are currently used for segmenting cardiac structures simultaneously in SA and LA image data.7 Looking at the results of our comparison of these simple averaging with SRR techniques, a significant enhancement of the image quality can be noticed. Especially, the visibility of small structures like the mitral valves is improved, opening the possibility of a more comprehensive analysis also including these constituents of the heart. Finally, we have shown in a rather qualitative manner how the segmentation of cardiac structures like the left ventricle can profit from the enhanced image data. However, this requires an extended and above all quantitative investigation of the improvements which can be expected from the application of an SRR technique. Future work has also to focus on the expected benefit for the analysis of dynamic cardiac image data.5 For that, it has to be investigated whether the here used method should be extended to a 4D version considering the cine-MRI data as a whole. Currently, we are processing each two SA and LA data sets of a volume independently and apply the SRR algorithm on such 3D data. Doing this for all volumes of a time series requires hours of computation time which limits the usability of the approach in a clinical environment. At least, for the necessary registration done during the preprocessing step, algorithms exist which make use of the possibility to parallelize problems and solve them employing graphics hardware as general purpose processing units.20–22 Another direction for possible improvements is the combination of three orthogonal volume data sets. For the current work, we have used the 2CH image data together with the SA slices. If in addition, the 4CH views are used, the obtained image quality may even be higher. A disadvantage however, would be the increase in computing time. Thus, it is like in many cases: improvements are possible, the quality can be improved, but not for free.
7. CONCLUSION The generation of super-resolution image data is possible for cardiac image data routinely acquired in the clinics. To our knowledge it is the first time that an SRR technique based on a maximum a posteriori approach has been used for the combination of SA and LA cardiac images. Since this method demands for a perfect alignment of the data sets, deformable registration has to be chosen for obtaining the best results. The quantitative gain in segmentation accuracy and the cardiac analysis based thereon has to be determined by comparing the results with manual gold standard segmentations. A possible improvement for a clinical use of the proposed method would be a more intelligent handling of the cardiac 4D image data as a whole.
REFERENCES [1] Castillo, MD, E., Lima, MD, J. A. C., and Bluemke, MD, PhD, D. A., “Regional Myocardial Function: Advances in MR Imaging and Analysis,” Radiographics 23(Special Issue), 127–140 (2003). [2] Pujadas, S., Reddy, G. P., Weber, O., Lee, J. J., and Higgins, C. B., “MR Imaging Assessment of Cardiac Function,” J Magn Reson Imaging 19, 789–799 (June 2004). [3] Finn, J. P., Nael, K., Deshpande, V., Ratib, O., and Laub, G., “Cardiac MR Imaging: State of the Technology,” Radiology (241), 338–354 (2006).
[4] Hennemuth, A., Behrens, S., K¨ uhnel, C., Oeltze, S., Konrad, O., and Peitgen, H.-O., “Novel methods for parameter-based analysis of myocardial tissue in MR images,” in [Medical Imaging 2007], Manduca, A. and Hu, X. P., eds., Proc. of SPIE 6511 (2007). [5] Frangi, A. F., Niessen, W. J., and Viergever, M. A., “Three-Dimensional Modeling for Functional Analysis of Cardiac Images: A Review,” IEEE Trans Med Imaging 20, 2–25 (January 2001). [6] Hill, D. L. G., Batchelor, P. G., Holden, M., and Hawkes, D. J., “Medical image registration,” Physics in Medicine and Biology 46(3), R1–R45 (2001). [7] Koikkalainen, J., Pollari, M., L¨otj¨onen, J., Kivist¨ o, S., and Lauerma, K., “Segmentation of cardiac structures simultaneously from short- and long-axis MR images,” in [MICCAI (1)], Lecture Notes in Computer Science 3216, 427–434, Springer (2004). [8] Greenspan, H., Oz, G., Kiryati, N., and Peled, S., “MRI inter-slice reconstruction using super-resolution,” Magnetic Resonance Imaging 20, 437–446 (2002). [9] Bai, Y., Han, X., and Prince, J., “Super-resolution Reconstruction of MR Brain Images,” in [Proc. of 38th Annual Conference on Information Sciences and Systems (CISS’04) ], (2004). [10] Geman, S. and Geman, D., “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Machine Intell. 6, 721–741 (1984). [11] Hardie, R. C., Barnard, K. J., Bognar, J. G., and Armstrong, E. E., “Joint MAP registration and highresolution image estimation using a sequence of under sampled images,” IEEE Transactions on Image Processing 6, 1621–1633 (1997). [12] Hardie, R. C., Barnard, K. J., Bognar, J. G., and Watson, E. A., “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998). [13] Yoo, T. S., ed., [Insight Into Images], A K Peters, Ltd., 1st ed. (2004). [14] Ibanez, L., Schroeder, W., Ng, L., and Cates, J., eds., [The ITK Software Guide 2.4], Kitware, Inc. (2005). [15] Thirion, J.-P., “Image matching as a diffusion process: an analogy with Maxwell’s demons,” Medical Image Analysis 2(3), 243–260 (1998). [16] Wesarg, S. and Nowak, S., “An Automated 4D Approach for Left Ventricular Assessment in Clinical Cine MR Images,” in [Informatik 2006. Informatik f¨ ur Menschen. Band 1], Hochberger, C. and Liskowsky, R., eds., GI-Lecture Notes in Informatics P-93, 483–490 (2006). [17] Lu, Y., Radau, P., Connelly, K., Dick, A., and Wright, G., “Segmentation of Left Ventricle in Cardiac Cine MRI: An Automatic Image-Driven Method,” in [Functional Imaging and Modeling of the Heart], LNCS 5528, 339–347, Springer (2009). [18] Szab´o, G., So´os, P., Seres, L., Beller, C., and Juh´asz-Nagy, A., “Computer-assisted ventricular reduction surgery,” International Congress Series 1281, 779 – 782 (2005). CARS 2005: Computer Assisted Radiology and Surgery. [19] Ascione, R., Wilde, P., and Angelini, G., “Left ventricular volume reduction,” MMCTS 2005(0628), 760– (2005). ¨ [20] Muyan-Ozcelik, P., Owens, J., Xia, J., and Samant, S., “Fast Deformable Registration on the GPU: A CUDA Implementation of Demons,” in [Computational Sciences and Its Applications, 2008. ICCSA ’08. International Conference on], 223–233 (2008). [21] Grabner, M., Pock, T., Gross, T., and Kainz, B., “Automatic Differentiation for GPU-Accelerated 2D/3D Registration,” in [Advances in Automatic Differentiation ], Lecture Notes in Computational Science and Engineering 64, 259–269, Springer (2008). [22] Jung, F. and Wesarg, S., “3D Registration Based on Normalized Mutual Information: Performance of CPU vs. GPU Implementation,” in [Bildverarbeitung f¨ ur die Medizin], Deserno, T. and Hoffmann, J., eds., Informatik aktuell, 5, Springer (2010).
