The Reduction of Gibbs Artifacts in Magnetic

2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics( CISP-BMEI 2017)

Reduction of Gibbs Artifacts in Magnetic Resonance Imaging Based on Convolutional Neural Network Yida Wang, Yang Song, Haibin Xie, Wenjing Li, Bingwen Hu, Guang Yang* Shanghai Key Laboratory of Magnetic Resonance, School of Physics & Materials Science, East China Normal University Shanghai, China

—In Magnetic Resonance Imaging (MRI), the K-space data is often under-sampled and truncated to shorten the scan time. However, the truncation of K-space also causes Gibbs ringing artifacts in the image, which seriously deteriorates the image quality. Inspired by the recent achievements of deep learning, we propose a novel method to reduce Gibbs artifacts in MRI with Convolutional Neural Network (CNN) in this paper. CNN is trained with a batch of image pairs with and without Gibbs artifacts. Afterwards, images with Gibbs artifacts can be input into the trained network to get the Gibbs-free images. Output of CNN is then transformed into K-space and merged with the sampled K-space data. Finally, inverse Fourier transform is applied to the merged K-space to get the final image. Experiments on both phantoms and real MRI images proved that the proposed method could reduce the Gibbs artifacts to a great degree and keep more image details compared with traditional Tukey filter.

I.

MRI images and Khosro et al.[18] proposed a deep architecture for CNN, which used the appearance (intensity) and anatomical (labels of brain tissues) features as input to nonlinearly map 3T MRI images to 7T MRI-like images. In this paper, we used CNN to reduce Gibbs artifacts in MRI images. The network was trained to learn the nonlinear mapping between MRI images with and without Gibbs artifacts. II.

THEORY

A. Forward-passing training Patch extraction and representation We input the training dataset (ui, Xi) to train the network, where ui is a patch in ground truth images and Xi is the image patch with Gibbs artifacts corresponding to ui. Firstly, we use convolution operation on the Xi to extract a set of feature maps y1:

INTRODUCTION

As one of the most important medical imaging methods, magnetic resonance imaging (MRI) can provide rich structural and functional information. It is a common practice to collect only part of K-space data in order to decrease the acquisition time. The partial K-space data lead to the famous Gibbs ringing artifacts[1], which manifests itself as spurious ringing around sharp edges[2]. The Hamming[3] and Tukey window functions[3] are often used to suppress the Gibbs ringing, but filtering on the K-space data causes the loss of high frequency information and blurs the reconstructed image. Convolutional Neural Network (CNN) is a type of multilayers, fully trainable models that can capture highly nonlinear mappings between inputs and outputs. These models are originally motivated from computer vision problems and thus are intrinsically suitable for medical imaging applications[4-7]. Recently, it has achieved impressive results in areas of image recognition[8-11] and object detection[12-15]. CNN has also achieved excellent performances in biomedical imaging applications. Ciresan et al.[16] used CNN to automatically segment neuronal structures depicted in stacks of electron microscopy (EM) images. Zhang et al.[17] used CNN for segmenting isointense stage brain tissues using multi-modality

978-1-5386-1936-0/17/$31 ©2017 IEEE

where W1 and b1 denotes the filters of size 1 × f1 × f1 × n1 and n1 dimension bias with each element Furthermore, f1 is the filter size and n1 is the number of filters, and * represents the convolution operator. We used the [19] as Nonlinear mapping In the middle layers, we further apply convolution operation to nm-1 dimensional feature maps ym-1, nonlinear mapping them to more abstract nm dimensional feature maps ym. The middle layers of network can be described as follows:

where m is the number of layer, Wm and bm denotes filters of size nm-1 × fm × fm × nm and nm dimension bias respectively. Reconstruction image

were used for testing. Patches of the size of were extracted from each image and there were roughly 12 patches that could be used to train the network. Fig. 1 shows one of the patches in the dataset.

For reconstructing MRI without Gibbs artifacts, we map abstract feature maps yL-1 into image domain. The last layer of network can be described as follows:

where L denotes the last layer number, WL and bL denotes filters of size nL-1 × fL × fL × 1 and 1 dimensional bias respectively. y is the final reconstructed image. B. Backward Propagation Network learns the end-to-end mapping relationship between u and X, acquiring optimizing and updating ={(W1, b1), (W2, b2), …, (WL, bL)}. In this paper, we use Euclidean distance function as the loss function: Figure 1. The MRI images with Gibbs artifacts caused by under-sampling. (a) Fully sampled image. (b) Sampling mask used with sampling rate of Truncated K-space. (d) Image with Gibbs ringing. (e) A patch from the fully sampled image. (f) Corresponding patch from the image with Gibbs ringing.

where n is the number of training data. We use stochastic gradient descent (SGD)[2 to minimize the loss function.

In this paper, we used 3 convolution layers for the proposed network, as Fig. 2 shows.

C. Merge After training, CNN can map images with Gibbs artifacts to images without Gibbs artifacts. To take the advantage of the sampled K-space data, the output image is Fourier transformed into K-space, and then all data that have been sampled are replaced with original sampled data. Finally, inverse Fourier Transformed is applied to the merged K-space to get the final image. III.

EXPERIMENTS AND RESULTS

A. Implementation details For phantom data experiment, Shepp-Logan phantom image of size 512 × 512 was used as ground-truth. with Gibbs artifacts were acquired by acquiring center K-space filling to 512 × 512 matrix size. the images were used for training and the remaining . Each image was split to patches of the size of roughly in the training dataset. For experiments on brain MRI images, the dataset consisted collected in T2 weighted gradient-echo (GRE) sequence (TE = 2.34ms, × 192mm, matrix = 256 × 256 × 192, FA = 7°, slice thickness = 1mm) on SIEMENS 3T Trio scanner. We used phase encoding lines in the K-space data to obtain images with Gibbs artifacts. were used for training and the remaining

Figure 2. The structure of proposed Convolutional Neural Network: Given a MRI image with Gibbs a layer of the CNN extracts a set of feature maps. The second layer nonlinearly maps these feature maps to high level features. The last layer reconstructs t

The parameters were respectively set as f1 = 5, n1 = 64, f2 = 3, n2 = 64, and f3 lter weights of each layer were initialized by random values from a Gaussian distribution with , and all bias values were initialized to work was implemented based on Caffe[21]. The network was trained with SGD method with a mini-batch size of 128 due to the limited capacity of GPU memory. The learning rate s s trained for The experiments were conducted on a

workstation equipped with one took about 12 hours for training the network to converge.

It

B. Results The results of CNN on Shepp-Logan and MRI images are compared with Tukey window (the ratio of taper to constant ) and shown in Fig. 3 and Fig. 4 respectively. For quantitative evaluation, we used peak signal-to-noise ratio (PSNR)[22], structural similarity (SSIM)[23] and highfrequency error norm(HFEN)[24] to compare results of different methods. TABLE 1 and TABEL 2 list the statistical results on Shepp-Logan and MRI images respectively.

TABLE 1. Statistical results on Shepp-Logan images.

Zero-filling

25.59 ±

±

± 31.43

Tukey window

25.38 ±

±

1196.77 ± 7.68

CNN

27.24 ±

±

441.11 ± 19.21

TABLE 2. Statistical results on MRI images

Zero-filling

±

±

±

Tukey window

23.68 ± 2.57

±

448.11 ±

CNN

±

7±

251.41 ± 42.83

From the images and statistical results, we can see that CNN can reduce Gibbs artifacts effectively and achieves better results than Tukey Window, with higher PSNR, SSIM and lower HFEN. Compared with the blurred images resulted from Tukey Window, plenty of image details and structures are captured by the network. In order to see how CNN works, we inspected feature maps in the convolution layers and some of them are shown in Fig. 5. From Fig. 5, we can see that CNN can automatically extract image information including contour of the brain, tissue structures and texture, and other feature information. Some feature maps have successfully captured the traits of Gibbs ringing (see a, e, g). Thus, it is understandable that CNN can be used to reduce Gibbs ringing artifacts from MRI. Figure 3. Reconstructions by different methods on phantom image. Top row denotes the reconstruction by zero-filling (a), Tukey window (b) and CNN (c), respectively. (d), (e), (f) are the absolute difference images (x from (a), (b), and (c) to the fully sampled Shepp-Logan image.

Figure 5. Some feature maps of the first (top row) convolution layer and the second (bottom row) convolution layer.

Figure 4. Results of different methods on MRI images with K-space. Top row shows the results on a sagittal MRi image and bottom row shows the results on a transverse one. From left to right are the fully sampled images ((a), (e)), zero-filling reconstruction ((b), (f)), Tukey window reconstruction ((c), (g)) and CNN results ((d), (h)), respectively.

In Fig. 6, we show some filters of the first and second convolution layer. CNN can learn these filters automatically without manual intervention compared to the traditional machine learning, in which the filters and feature extractors need to be set manually. We can see that filters of the second convolution layer are more complex than the filters of the first

layer which means that a deeper layer could extract more complex structures in the image.

by CNN trained with only images of the same image plane. For universality and convenience, we preferred to use the CNN trained with mixed dataset. TABLE 3. Statistical results on MRI images trained with different datasets.

Sagittal images Transverse images

Figure 6. Some learnt filters of the first (a) and second convolution layer (b).

DISCUSSION In the above-mentioned experiments on MRI images, the CNN was trained from the dataset with mixed sagittal and transverse brain images. In order to study the generalization

datasets to train the network. The three CNN trained from different datasets are used to reconstruct brain images of different image plane and the results are shown in Fig. 7.

PSNR SSIM HFEN PSNR SSIM HFEN

28.84 ± ± 284.84 ± 45.98 29.23 ± 2.83 ± 236.18 ± 47.84

29.21 ± 3.13 ± 246.37 ± 41.98 28.62 ± 3.12 66 ± 298.54 ± 44.74

± 2.91 ± 255.61 ± 28.99 ± ± 245.42 ± 42.35

We extended our approach and used different convolution neural networks to reconstruct the image without Gibbs ringing. To compare with the proposed 3 layers neural network, we trained another two networks containing only 1 convolution layer (the size of filter is 3 × 3, CNN1) and 2 convolution layers (the size of filter is 5 × 5 and 3 × 3, CNN2) with the same mixed dataset, respectively. The reconstructed results of different neural network are shown in Fig.8. For quantitative evaluation, we also used PSNR, SSIM and HFEN to evaluate different neural networks. TABLE 4 shows the statistical results on MRI images. From Fig. 8 and TABLE 4, we can see the quality of image reconstructed by the proposed CNN is much better than CNN1 and CNN2. The proposed CNN has the highest PSNR, SSIM and lowest HFEN. So we assume that the network is deeper, the results is better. Bee Lim et al.[25] developed an enhanced deep super resolution network (EDSR) with performance exceeding those of current state-of-the-art super resolution methods. This paper used deep residual network and get good reconstructed results. Inspired by it, we will study more complex and deeper network to remove Gibbs ringing artifacts in MRI in the future.

Figure 7. Reconstructed results with CNN trained by different datasets. Top row and bottom row denote sagittal and transverse images respectively. From left to right, images with Gibbs artifacts ((a), (e)), CNN results trained with transverse dataset ((b), (f)), CNN results trained with sagittal dataset ((c), (g)) and CNN results trained with mixed dataset ((d), (h)), respectively.

For quantitative evaluation, we used PSNR, SSIM and HFEN to evaluate the CNN results trained with different datasets. TABLE 3 shows statistical evaluation parameters. From Fig. 7 and TABLE 3, we can see that if the training dataset is composed purely of images with a different image plane from the test image, the quality of the reconstructed image is relatively poor. However, CNN trained with images of mixed image plane yields results comparable to those produced

Figure 8. Reconstructed results with different neural networks. Top row and bottom row denote sagittal and transverse images respectively. From left to right, the fully sampled images ((a), (f)), images with Gibbs artifacts ((b), (g)), CNN1 results ((c), (h)), CNN2 results ((d), (i)), and the proposed CNN results ((e), (j)), respectively.

TABLE 4. Statistical results of different neural networks

CNN1

2

±

65 ±

3

299.56 ± 41.91

CNN2

2

±

65 ±

2

289.42 ±

±

±

Proposed CNN

251.41 ± 42.83

[9]

[11]

[12]

CONCLUSION This paper proposed a method to reduce Gibbs artifacts in MRI images. CNN was trained to map images with Gibbs artifacts to images without Gibbs artifacts. Afterwards, the CNN-output image was merged with original sampled data to obtain the final image. The experimental results show that the proposed method can effectively reduce Gibbs artifacts and preserve image details at the same time, and thus improve the image quality obviously. ACKNOWLEDGEMENT This study is supported by National High Technology Research and Development Program of China (Grant .

[13]

[14]

[15]

[16]

[17]

REFERENCES [1]

Huang X and Chen W, ” A fast algorithm to reduce gibbs ringing artifact in MRI.” International Conference of the Engineering in Medicine &

[18]

[2]

Lei T and Udupa J K, “Gibbs ringing artifact and spatial correlation in MRI,” Ph -971. David B. Preston, “Spectral Analysis and Time Series.” Technometrics, 1981, 93(2):179-183. Zhang W, Li R, Deng H, Wang L, Lin W, Ji S and Shen D, “Deep convolutional neural networks for multi-modality isointense infant brain image segmentation,” NeuroImage, 4. Natural image denoising with convolutional networks.” Advances in Neural Information Processing Systems, 769-776. Dai J, He K and Sun J, “Convolutional feature masking for joint object and stuff segmentation,” Proceedings of the IEEE Conference on Computer Zhang, K., Zuo, W., Chen, Y. and Meng, D., “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Transact He K, Zhang X, Ren S and Sun J, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” European Conference -361.

[19]

[3] [4]

[5]

[6]

[7]

[8]

[21]

[22]

[23]

[24]

[25]

He, K., Zhang, X., Ren, S., and Sun, J., “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition, -778. Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., and Oliva, “A. Learning deep features for scene recognition using places database,” Advances in neural information processing systems, -495. Ji, S., Xu, W., Yang, M., and Yu, K, “3D convolutional neural networks for human action recognition,” IEEE transactions on pattern analysis and machin -231. Girshick R, Donahue J, Darrell T and Malik J, “Rich feature hierarchies for accurate object detection and semantic segmentation,” Proceedings of the IEEE conference on computer vision and pattern recognition, -587. Chen H, Yu L and Dou Q, “Automatic detection of cerebral microbleeds via deep learning based 3d feature representation,” Biomedical Imaging 767. Dai, J., Li, Y., He, K., and Sun, J., “R-fcn: Object detection via regionbased fully convolutional networks,” Advances in neural information -387. Ren, S., He, K., Girshick, R., and Sun, J., “Faster R-CNN: Towards realtime object detection with region proposal networks,” Advances in neural information processing systems, -99. Ciresan D, Giusti A, Gambardella L M and Schmidhuber, “Deep neural networks segment neuronal membranes in electron microscopy images,” Advances in neural information processing system -2851. Bahrami K, Shi F, Rekik I and Shen D, “Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features,” Deep Learning and Data Labeling for Medical Applications. Springer Internati Glorot X, Bordes A and Bengio Y, “Deep Sparse Rectifier Neural Networks,” International Conference on Artificial Intelligence and . LeCun Y, Bottou L, Bengio and Haffner P, “Gradient-based learning applied to document recognition.,” Proceedings of the IEEE, 1998, 86(11): 2278-2324. Wang S, Su Z, Ying L, Peng X, Zhu S and Liang F, “Accelerating magnetic resonance imaging via deep learning,” International -517. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J and Girshick R, “ Caffe: Convolutional Architecture for Fast Feature Embedding,” -678. Hore A and Ziou D, “Image quality metrics: PSNR vs. SSIM,” Pattern Recognition ( 2366-2369. Wang Z, Bovik A C and Sheikh H R, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image -612. Ravishankar S and Bresler Y, “MR image reconstruction from highly undersampled K-space data by dictionary learning,” IEEE transactions Lim B, Son S and Kim H, “Enhanced deep residual networks for single image super-resolution,” a