Fully automatic liver segmentation for contrast-enhanced CT images L´ aszl´ o Rusk´ o, Gy¨ orgy Bekes, G´abor N´emeth, and M´arta Fidrich GE Hungary ZRT. Healthcare Division, Akron u. 2., H-2040 Buda¨ ors, Hungary
[email protected]
Abstract. The need for fast and precise segmentation has increased recently due to the spread of systems for computer aided diagnosis and therapy planning. The manual segmentation of the liver is very time consuming, so it is desired to develop a method that can precisely segment the liver without any human interaction. In this paper we propose a fully automatic method for liver segmentation on contrast-enhanced (portal venous) CT images. Our method is essentially an advanced regiongrowing the result of that is improved by various pre- and post-processing steps, like intensity based ROI detection, separation of liver and heart, additional segmentation at the right lung lobe, vessel (IVC) removal, and cavity filling. According to our experiments the method can efficiently segment the liver parenchyma in many cases, however, in some cases the result may exclude very large lesions.
1
Introduction
Computer assisted planning of various liver treatments (minimally invasive therapies, oncology liver sectioning, living donor transplantation) is primarily based on computed tomography (CT), which can be an important aid for operability decisions and visualization of individual patient anatomy in 3D. The planning is based on the liver volume, the anatomical liver segments, the vessel structure, and the relation of lesions to these structures. The detection of the boundaries between the segments is then the first step of the preoperative planning. Radiologists currently use CT images with intravenous contrast infusion, in order to detect lesions and vessels in the liver. The key point of the above-mentioned treatments is the liver volume segmentation. This step is quite time consuming when it is done manually. Our aim is to develop a method that is precise, quick and robust enough to use it in the every day clinical practice. There are several published methods about segmentation of CT images. Most of these methods are some variants of the region-growing, active contour/surface, level-set, or thresholding, classification algorithms (of course, all of them are adapted to the specific situations and they are equipped with several pre- and post-processing operations). In addition, the methods are often based on some statistical, anatomical, or geometric model. Soler [1] proposes a fully automatic method to segment the liver from contrastenhanced CT scans. This method delineates the skin, bones, lungs, kidneys and T. Heimann, M. Styner, B. van Ginneken (Eds.): 3D Segmentation in The Clinic: A Grand Challenge, pp. 143-150, 2007.
spleen, by combining the use of thresholding, mathematical morphology and distance maps then the liver is extracted. A 3D reference model is generated from manually segmented livers and adjusted onto the image with rigid and affine registration. The model is deformed to get the final result. The weakness of this method is that one phase is used, which is acquired according to a special protocol and does not correspond to the general practice. An automatic approach for segmentation of the liver from CT images based on a 3D statistical shape model is presented in the paper of Lamecker [2]. This iterative technique first builds a statistical model from a training set of shapes. Each shape is defined by some anatomically specific points sampled on the surface. The next step is the positioning the mean shape into the image. Then single shape adjustment is applied. Unfortunately, there is no clinical evaluation and the selection of the landmarks is very difficult due to the very variable shape of the liver. The level-set method-family ([3], [4], [5]) has been successfully used for medical image segmentation. The advantages of the level set approach are that it handles topological changes and defines the problem in one higher dimension. The main disadvantages are that these methods are time-consuming and they usually produce over-segmentation. The active contour [6] method is used to segment abdominal organs in the clinical practice. It works well on native images, because the organs are homogeneous. In case of contrast-enhanced images, the contrast agent is cumulated differently in different parts of the liver. For example the vessels and some tumors will have higher intensity than the liver parenchyma. The active contour methods starts from a smaller region and try to blow it up and fit the surface to the contour of the organ. The vessels and tumors set back the regular growing of the surface. The region-growing based approaches [7] can provide good results on contrastenhanced images. Such method starts from a small region (environment of input curve, or point), and each neighboring voxel is added to the actual region, if its intensity is corresponds to a pre-defined range. The region-growing can efficiently close round the vessels and tumors (in contrast with active contour), but it is very sensitive to its input and can easily flow into neighboring organs that have similar intensity. In the next chapter we describe a fully automatic segmentation method for liver segmentation. Our method is based on intensity analysis and regiongrowing, where the issue of under- or over-segmentation is handled in various ways. In Section 3, we present the evaluation of this method using the test exam set provided by the MICCAI Grand Challenge on clinical liver segmentation.
2
The method
Our method consists of the following main steps. First a seed region is determined that involves voxels which are located inside the liver. Then, the liver is separated from the heart to prevent over-segmentation in this region. Starting from the seed
144
region the liver is segmented using an advanced region-growing (RG) method. The segmentation is followed by various post-processing steps, so that the size of under- or over-segmented regions is reduced significantly. 2.1
Determine seed region
When the CT image to be segmented is enhanced using some contrast agent, the abdominal organs can be easier separated due to different contrast intake of the different organs, which can be exploited when the region of the liver is automatically determined. Besides the characteristic contrast intake, the size (largest organ) and the location (mostly on the right side) of the liver make it easier to determine significant portion of its volume without user interaction. In this paper we assume that image to be segmented is a contrast-enhanced CT scan of the liver that is acquired in the portal venous phase (contrast agent is visible in the portal vein). According to several exams, which were acquired using different types of scanners and contrast agents, with different image resolution and quality, the intensity of the liver voxels is in the range of [-50,250] HU. In order to determine the mean intensity of the liver voxels for a particular exam, a histogram can be calculated. If reliable result is needed, only those voxels are incorporated, which are located in the right side of the body and the intensity of which is in the range [-50,250] HU (air, fat, bones are excluded). After smoothing this histogram, it has two significant maxima. One of them belongs to the muscles the other belongs to the liver. Since the intensity of the liver is always higher than that of the muscles, the maximum greater than 80 HU represents the modus of liver intensities in all cases. After this modus is determined, the rough estimation of the minimal and maximal intensity of liver voxels can be easily calculated based on the histogram, which makes it is possible to find a region inside the liver automatically. First, a binary image is created based on the original grayscale image such that the voxels, the intensity of which is in the previously determined range, have value 1, and all other voxels are zero valued. This image usually consists of several regions, the intensity of which is similar to the liver’s intensity. The image of possible liver regions is then eroded so that small regions are deleted. Since the liver has the largest compact volume in the abdomen, a sphere with a relatively large radius can be used for erosion. The value of this radius was determined based on several exams. After eroding the image, the largest connected region is considered as seed region for the segmentation. Although, the size of this region may vary among the different exams, this method provides a reliable set of liver voxels in all cases. The average size of this region for 20 training exams was 15% of the total liver volume. 2.2
Liver heart separation
In the image belonging to the portal venous phase the liver and the heart has nearly the same intensity, so the result of a 3D RG method usually involves the heart. Since large over-segmented regions shall be eliminated, it is important
145
to keep the RG off the heart. The liver-heart separation algorithm takes the advantage of the anatomical feature that the bottom of the lung fits the liver surface. We separate the heart from the liver by means of connecting the bottom of the left and right lung lobes with a surface. The method first determines the bottom of the right and left lung lobes. Then, for each coronal slice a minimallength curve is found, which connects the two lungs and goes along large gradient values. The set of these curves defines a surface that is used to prevent the RG to go into the heart. In case of abdominal CT scans, slices at the top of the image include the bottom of the lung (if not, there is no need for liver heart separation anyway). Starting from the topmost slice both lung lobes can be segmented based on the characteristic intensity of the air. In order to find seed-points for the left and the right lung lobes in the topmost slice, we do the following. First we determine the largest connected region, where the intensity of voxels is higher than -400 HU. This is the body region. Inside this region we determine the largest connected air region for the left and the right side, separately. Using these regions as seed, a 3D RG method can segment the left and right lung lobes. After the lung lobes are segmented, the coronal slices of the CT image are processed. The goal is to determine two curves representing the bottom contour of the right and the left lobes and connect the leftmost point (L) of the right curve and the rightmost point (R) of the left curve. When L and R are available for each coronal slice, we use the following method to connect them. Starting from L we try to reach R such that we encounter voxels with the largest possible gradient value. Going from right to the left, in each step we choose the location of the largest gradient value found in the local environment of the previous point. When we reach the vicinity of R we connect the current point with R with a discrete line. After this curve is determined for each coronal slice the surface separating the liver and heart is calculated by averaging the curves located in the neighboring slices, which provides a smoother solution. Finally, for each coronal slice the voxels, which are located above the surface, are set to an artificial intensity value (3000) so that the RG cannot go into this region. 2.3
Region-growing
The liver parenchyma is nearly homogeneous, so a RG method can efficiently determine most of the liver volume. In order to use a RG method we need some seed-points and the intensity range of the voxels belonging to the liver. The set of initial points is determined according to Section 2.1, while the intensity interval is calculated in the following way. First, we create the histogram of intensities located in the environment (of 5 mm radius) of all initial points. Then, we calculate the intensity interval based on the modus of this histogram, and the left and right standard deviation of intensities with respect to the modus. According to our experiments the intensity range of the liver voxels cannot be perfectly determined. If the RG uses a global intensity range some regions can be under- or over-segmented. Thus, it is important to find different ways to correct
146
the segmentation result in these regions. In order to reduce the probability of over-segmentation we made the following modification to the RG method. During a 3D RG method a voxel is added to the region if all voxels in its neighborhood have acceptable intensity. In the literature usually 6-neighborhood is used. Our experiments showed that using larger neighborhood (e.g. sphere with 5 mm radius) reduces the probability of over-segmentation significantly. When such a large radius is used for RG, the neighborhood consists of a large number of voxels. Due to the noise, we have to use some tolerance, when we check if the voxels’s environment satisfies the intensity condition or not. Even though the noise is reduced (using any filter), under-segmentation may concern several region of the liver, which are less homogeneous. According to our experiments, the region near the lung, some lesions, and vessels, the intensity of which is significantly lower or higher than the intensity of the normal liver parenchyma may be under-segmented. In the following subsection we discuss how these problems can be eliminated. 2.4
Post-processing
In some cases the liver is under-segmented near the right lung lobe, where the ratio of intensities lower than the minimal intensity exceeds the tolerance. This problem can be corrected by additional segmentation that allows lower intensity range in the region located between the surface of the segmented liver and right lung lobe. First, we determine the surface voxels for the right lung and calculate the surface normal vector for each of them. If the normal vector of a surface voxel points toward a liver voxel that is closer than a predefined distance, the surface voxel is marked. In the next step we connect each marked lung surface point with the corresponding liver voxel using a discrete 3D line. Then, the local environment is calculated for each line, which defines a closed connected region between the liver and the right lung lobe. We calculate a new intensity interval based on this region, which is used by an additional RG. This method starts from liver surface points and limited only to this region, so it cannot cause over-segmentation in other part of the liver. In case of portal venous images the intensity of the inferior vena cava (IVC) is very similar to the intensity of the liver parenchyma. Since the diameter of this vessel is significant (20-30 mm), the neighborhood connected RG method leaks out through the IVC in nearly 40% of the cases. In order to remove IVC from the segmentation result we do the following post-processing. Our main idea is to detect those parts of the segmented liver, which are similar to a cylinder with a specified diameter. Since this vessel is vertically oriented, the cross section of the IVC is a circular region in each axial slice. In order to detect circles with a given diameter the circular Hough transform is used. In our case, however the radius of the circle varies, so instead of a circle we use a ring such that the inner radius is smaller and the outer radius is greater than the average radius of IVC. Using this ring the entire 3D contour of the liver is processed, so that a probability map is created, where higher values represent voxels, which are located inside a horizontal tube with radius similar to IVC. After thresholding
147
the probability map, we process it slice by slice. We locate all local maxima in the slice, and for each maximum we check whether a closed contour is found around the location of the maximum inside its local environment. If so, we mark the region around the given maximum in the segmented image as candidate for erasing. In the next step we determine the largest connected region in the segmented image that consists of unmarked liver voxels. All liver regions, which are not connected to the largest one, are also marked as candidate for deleting (such region can be found along the IVC, where the vein has a branching point). Finally, all candidate region is erased, the vertical length of which is greater than a predefined constant (so that we don’t erase the bottom peak of the right and left liver lobes). An intensity-based segmentation will not involve voxels belonging to the vascular structure (portal vein), which have higher intensity. In the clinical practice, however, the vessel is considered as part of the liver as long as it is completely surrounded by liver parenchyma. In order to fill the vessels inside the liver we do the following. We determine the contour of the segmented liver, and calculate the surface normal for each of them. Then, we mark a surface voxel if there is another liver voxel in the direction of surface normal, such that its distance from the corresponding surface point is nearly equal to the average diameter of portal venous branches. Finally, we dilate the liver at each marked surface voxel using a sphere the radius of that is equal the average radius of the portal vein. Here we note, that smaller tumors, which form closed cavity inside the liver volume, can be filled using any simple cavity filling method. According to our experiments, the RG does not need the image to have high resolution. In order to speed up the segmentation we omit slices such that the slice thickness is between 2 and 3 mm. After the segmentation, the result is needed in the resolution of the input image. In order to get a smooth interpolation between the segmented slices, we create the triangular representation of the liver surface, we smooth it, and convert it to voxel image with the resolution of the input image. We note, that this surface representation allows further refinement of the results based on the image gradient, which have not been implemented in this work yet.
3
Results
The organizers of MICCAI Workshop on 3D Segmentation in the Clinic have evaluated our method on 10 test exams, which involves easy, average, and difficult cases. Figure 1 shows our result for each of these cases. Based on the images, we can claim that our method performs well, if the liver does not have very large lesions. In other cases (depicted in the middle and the bottom rows), however, the liver can be very under-segmented, which is the most important problem we have to solve in the future. Here we note, that our method was primarily developed to aid minimally invasive liver therapies (e.g. embolization of tumors), where such large lesions are not considered.
148
Fig. 1. From left to right, a sagittal, coronal and transversal slice from a relatively easy case (1, top), an average case (4, middle), and a relatively difficult case (3, bottom). The outline of the reference standard segmentation is in red, the outline of the segmentation of the method described in this paper is in blue. Slices are displayed with a window of 400 and a level of 70.
Table 1 displays the numerical evaluation of our results with respect to 5 important metrics. When the liver has large lesions (3, 4, 8, 10) our method provides bad result concerning all metrics because the large lesions are undersegmented. Although, our method is based on intensity analysis, it is not sensitive to the majority of lesions, the size of which allows of embolization or ablation. These lesions, which have significantly higher or lower intensity than the liver, are filled by the post-processing steps in most of the cases. Focusing on these cases, (1, 2, 5, 6, 7, 9) our method performs an average 76 of total score, which is a bit better than a non-expert manual segmentation. Our method provides the result in 56 seconds in average (min 31, max 113) for the 20 training exams using an Intel Pentium 4 CPU 3GHz processor. Considering that an average 39% of this time is spent with IVC removal (that was not optimized yet) this method can efficiently segment the liver volume in most of the clinical cases, but it definitely needs further improvements to handle the extreme cases.
149
Dataset Overlap Error Volume Diff. Avg. Dist. RMS Dist. Max. Dist. Total [%] Score [%] Score [mm] Score [mm] Score [mm] Score Score 1 7.0 73 1.1 94 1.1 73 2.0 73 18.1 76 78 2 7.1 72 4.5 76 1.1 73 2.2 70 22.7 70 72 3 20.6 20 -16.0 15 3.9 2 9.0 0 50.0 34 14 4 14.1 45 -5.9 69 2.6 35 6.2 14 45.1 41 41 5 8.4 67 -2.4 87 1.5 63 2.8 61 28.6 62 68 6 7.2 72 1.2 94 1.1 73 2.0 73 20.2 73 77 7 5.8 77 0.7 96 0.9 78 1.7 77 17.4 77 81 8 10.8 58 -9.3 50 1.9 52 3.8 47 26.0 66 55 9 6.9 73 -0.5 98 0.9 78 1.7 77 15.0 80 81 10 19.7 23 -16.6 12 3.3 17 7.0 3 40.1 47 20 Average 10.7 58 -4.3 69 1.8 54 3.8 50 28.3 63 59
Table 1. Results of the comparison metrics and scores for all ten test cases.
Acknowledgment Some parts of the presented method were developed in cooperation of GE Hungary and the University of Szeged. Hereby, we would like to thank the university team members, namely Norbert Bara, Csaba Domokos, Krisztina D´ ora, Tam´ as K´ orodi, L´ aszl´ o Reszegi, ´ ad Tigyi, and Norbert Zs´ Arp´ ot´er for their contribution. We are also very grateful to the medical evaluation team lead by Prof. Andr´ as Palk´ o Ph.D, namely Katalin Gion M.D., Edit Kukla M.D, and Endre Szab´ o M.D. for their very important clinical feedbacks.
References 1. Soler, L., Delingette, H., Malandain, G., Motagnat, J., Ayache, N., Koehl, C., Dourtheb, O., Malassagne, B., Smith, M., Mutter, D., Marescaux, J.: Fully automatic anatomical, pathological, and functional segmentation from ct. Proc. SPIE Medical Imaging 3979 (2000) 246–255 2. Lamecker, H., Lange, T., Seebass, M.: Segmentation of the liver using a 3d statistical shape model. Technical Report ZIB-Report 04-09, Konrad-Zuse-Zentrum fr Informationstechnik Berlin (April 2004) 3. Sethian, J.A.: Level Set Methods and Fast Marching Methods. Cambridge University Press (1999) 4. Shi, Y., Karl, W.C.: A fast implementation of the level set method without solving partial differential equations. Technical report, Boston University, Department of Electrical and Computer Engineering (January, 2005) 5. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. International Journal of Computer Vision 22(1) (1997) 6179 6. Bekes, G., Ny¨ ul, L.G., M´ at´e, E., Kuba, A., Fidrich, M.: 3d segmentation of liver, kidneys and spleen from ct images. Proc. International Journal of Computer Assisted Radiology and Surgery 2(1) (2007) 45–46 7. Pohle, R., Toennies, K.D.: Segmentation of medical images using adaptive region growing. Proc. SPIE Medical Imaging 4322 (2001) 1337–1346
150
