LANDMARK SELECTION FOR SHAPE MODEL CONSTRUCTION VIA EQUALIZATION OF VARIANCE Sylvia Rueda 1∗ , Jayaram K. Udupa 2 , and Li Bai 1 1
CMIAG Group, School of Computer Science, University of Nottingham Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, United Kingdom 2 Medical Image Processing Group, Department of Radiology, University of Pennsylvania Fourth Floor, Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104-6021, USA
[email protected],
[email protected],
[email protected] ABSTRACT Model-based segmentation approaches, such as those employing Active Shape Models (ASMs), have proved to be useful for medical image segmentation and understanding. To build the model, we need an annotated training set representing correspondences among shapes. Manual positioning of landmarks is a tedious, time consuming, and error prone task, and almost impossible in the 3D space. To overcome some of these drawbacks, we devised an automatic method. Our method is guided by the strategy of equalization of the variance contained in a training set for selecting landmarks. The main premise here is that this strategy itself takes care of the correspondence issue and at the same time deploys landmarks very frugally and optimally considering shape variations. The desired landmarks are positioned around each contour in such a manner as to equally distribute the total variance existing in the training set. The method is evaluated on 40 MRI foot data sets. The results show that, for the same number of landmarks, the proposed method is significantly more compact than manual and equally spaced annotations. Index Terms— Automatic Landmark Tagging, Point Distribution Model, Active Shape Model, correspondence problem, variance equalization. 1. INTRODUCTION Segmentation and modeling of organs using model-based approaches such as Active Shape Models (ASMs) [1] requires a priori information usually given by manual annotation of a training set. To overcome the drawbacks of manually creating a training set, it is necessary to perform automatic landmark tagging, which could avoid the errors and the drudgery associated with manual annotation. Automatic annotation should capture the real variability existing among shapes with landmark correspondences expressed among shapes of the train∗ This research is funded by the European Commission Fp6 Marie Curie Action Programme (MEST-CT-2005-021170).
978-1-4244-2003-2/08/$25.00 ©2008 IEEE
1271
ing set. Previous attempts used characteristic shape features, such as curvature [2] or polygonal approximation of contours [3], to establish point correspondences among shapes. In these two cases, the landmarks were obtained in the mean shape and propagated to all the examples of the training set using distance transform methods. Others worked with Minimum Description Length (MDL) methods [4][5][6], treating the landmark correspondence as an optimization problem. In this type of work, the initial landmarks have to be defined on a reference shape (in an equally spaced manner or with a priori knowledge of where they should be) and then, the correspondence is optimized by minimizing a certain objective function. MDL uses stochastic optimization methods or genetic algorithms to find correspondences, while requiring prior distributions of the parameter vectors. All these methods capture the correspondence between landmarks in a global manner, but do not distribute the landmarks considering the local variability of shapes in the training set. The main premise of this paper is that regions with high variance will need more landmarks to be described consistently, while regions with low variance do not need to be described with many landmarks. Simultaneously, the strategy also resolves the landmark correspondence problem. This is a consideration that has been missing in all landmark tagging methods proposed up to now. In this paper, we present a method for automatic landmark tagging that uses a strategy of equalization of the total variance existing in the training set, to select the landmarks and to establish correspondences among shapes. The idea consists of distributing landmarks around each shape in such a manner as to place more landmarks in regions where the variance among shapes is high and fewer landmarks where variance is low. The method is simple to implement, computationally fast, and does not need a reference shape to establish landmark correspondence as opposed to MDL methods. First, we present the method of variance equalization in Section 2. In Section 3, we show the results in a medical application. Our conclusions and future work are stated in Section 4.
ISBI 2008
2. VARIANCE EQUALIZATION Algorithm VE Given a training set of M segmented images, we align them by using affine registration and extract the boundaries {Bj ; j = 1, ..., M } of the segmented structures. After identifying the corresponding starting point in all shapes of the training set, we parameterize all contours so they have the same number of points N equally spaced along each boundary. We define N to be the minimum number of points among all the original contours in the training set, because we want to retain as much information in the original boundaries as possible. This way we obtain a parametric description of the boundaries; that is, {B j (i); i = 1, ..., N ; j = 1, ..., M }. A good model should pick more points where there is more variation so the variation can be captured as best as possible in the Point Distribution Model (PDM). These regions are where the landmarks are less correlated. In other regions, the landmarks will be highly correlated. From the M parameterized boundaries, we can determine the variance in the location of any i th point, i = 1, ..., N , and express it as a function V ar(i). The total variance at each i th point over all M shapes can be estimated by adding the eigenvalues of the covariance matrix of the x and y coordinates at the i th point over all M shapes. Our goal is to place landmarks by making use of this information in V ar(i). We wish to select more landmarks in regions where V ar(i) is higher, and less where it is small. We will achieve this by equalizing the variance V ar(i) between successive landmarks. Therefore, we will use the variance to guide the selection of points so that the variance is more or less equally distributed with point density on the contours. Once we specify the number of landmarks n we want, we distribute them around each contour in a way that the area under the variance curve between each successive pair of landmarks is roughly the same. The selection of points will be done separately for each contour and they will get their point arrangement depending on their shape, the spacing between successive points, and its relationship to the variance graph. Let xi , i = 1, ..., N , denote the i th point generically in any Bj , let V ar(xi ) be the total variance at x i , and let del be the spacing between points in B j . Then, given V ar(x i ), n, and del, we wish to find the points p 1 , ..., pn on Bj such that V ar(p1 ) · |p2 − p1 | + . . . + V ar(pn ) · |p1 − pn | = V ar(x1 )del + V ar(x2 )del + . . . + V ar(xN )del.
Input: Bj (i) : j = 1, ..., M, i = 1, ..., N ; n; del. Output: Landmarks p j1 , ..., pjn , j = 1, ..., M . Step 1: Compute V ar(x i ) for i = 1, ..., N . For j = 1, ..., M , perform Step 2 below. Step 2: Compute the area per point for B j An =
TA , n
(2)
where T A is the total variance (or area under the V ar(x i ) curve) listed before. Step 3: Select the first landmark on each contour as the point on B j at which V ar(xi ) is maximum. Call this point pj1 . Step 4: From p j1 , go forward on B j and skip points until we reach pj2 such that V ar(pj1 ) · |pj2 − pj1 | = An .
(3)
Once pj2 is found, go forward from p j2 and repeat this process until we come back to p j1 . Step 5: Output p j1 , ..., pjn . 3. RESULTS In this section, we illustrate the theory and algorithm explained above on a training set of 40 MR images of the foot focusing on the talus bone of the foot (Fig.1).
(1)
The right side of Equation 1 corresponds to the total variance, denoted T A, in B j over all its points, which is also the area under the V ar(x i ) curve. | · | denotes the distance along the contour B j between successive points. Therefore, |p2 − p1 | = (b + 1) · del, if there are b points in B j between p1 and p2 excluding p 1 and p2 .
Fig. 1. The talus bone of the foot in a MR image.
The algorithm used to implement the variance equalization method is summarized below.
The total variance over all 40 shapes in the training set is shown in Fig.2. Some results on landmark distribution can be
1272
the variation per mode to total variation, given by
150
l
χn,l =
Var(xi)
100
i=1 2n
λi (4)
, λi
i=1
where n represents the number of landmarks selected and l the number of eigenvalues considered. χn,l indicates how well the variation is captured as a function of both the number of eigenvalues (modes) and the number of landmarks selected. It is shown as a surface plot in Fig.4 for the VE method. In Figure 5, a subset of χ n,l is displayed for n = 20 and l = 1, ..., 40 for the three methods.
50
0 0
50
100
150
xi
Fig. 2. Total variance V ar(x i ) for the talus training set.
1.05 1 0.95 0.9
(b) Shape 1, n = 28.
Xn,l
(a) Shape 1, n = 10.
0.85 0.8 30 0.75 0.7 0
20 10 10
20
30
40
50
60
0
n
l
(c) Shape 2, n = 10.
(d) Shape 2, n = 28.
Fig. 4. χn,l for n = 1, ..., 28 and l = 1, ..., 56.
Fig. 3. Landmarks selected after variance equalization for different shapes and different number of landmarks n.
seen in Fig.3. The maximum of variance in this training set corresponds to the bottom part of the shapes where more landmarks are tagged. In the upper part of the structures where variance is much lower, the points tagged are farther apart. We compared the method of automatic landmark tagging based on variance equalization with manual and equally spaced landmark tagging methods for different number of landmarks selected. For the two latter methods, p j1 = Bj (1). In the manual case, other n − 1 points are selected guided by anatomy. This, of course, is not an easy task because of a great deal of ambiguity that exists in selecting consistently the same homologous points. We express the compactness of a model as a fraction of
1273
To represent the comparison independently of the number of modes, we calculate the area under the curve for each of the cases n = 1, ..., 28. If we integrate χ n,l over the l variable, we obtain 2n χn,l χn =
l=1
χ=
i=1
. (5) 2n The χn plot for the three methods is shown in Fig.6. If we integrate χ n over L landmarks, then we can express the overall ability of the method to capture variation by using up to L landmarks: L χi . (6) L The χ values are listed in Table 1 for the three methods. We may note that, as we increase the number of landmarks, differences among the methods’ abilities diminish.
Table 1. Comparison of methods in terms of χ value. Variance Equalization Equally Spaced Manual χ 0.9669 0.9454 0.9240
1
0.9
find landmark correspondences.
X20,l
0.8
0.7
4. CONCLUSIONS
0.6
We have presented a new theory and method for automatic landmark tagging based on the concept of variance equalization between successive landmarks. The method performs significantly better, in terms of compactness, than the manual and the equally spaced landmark tagging methods. We defined three different compactness factors to assess the methods at different levels. We showed that the distribution of landmarks using the variance equalization method is a good way of creating more compact models taking into account the variability existing in the training set. This concept has been missing in all previous attempts at automatic landmark tagging. Our future focus will be on more extensive evaluation, application and testing in segmentation, and extension to the 3D space.
Variance Equalization Equally spaced Manual
0.5
0.4 5
10
15
20
25
30
35
40
l
Fig. 5. χ20,l for n = 20 and l = 1, ..., 40.
1
0.98
0.96
0.94
5. REFERENCES
Xn
0.92
0.9
[1] T. F Cootes, C. J Taylor, D. H Cooper, and J Graham, “Active shape models - their training and application,” Computer Vision Image Understanding, vol. 61, pp. 38– 59, 1995.
0.88
0.86 Manual Equally spaced Variance Equalization
0.84
0.82 0
5
10
15
20
25
[2] S Rueda, J.K Udupa, and L Bai, “A new method of automatic landmark tagging for shape model construction via local curvature scale,” SPIE Medical Imaging, Conference on Visualization, Image-guided Procedures, and Modeling, 2008, to appear.
30
n
Fig. 6. χn for n = 1, ..., 28. This is apparent in the χ factor for L = 28. As we can see in Fig.6, at n = 10 or under, the differences between the VE method and others is substantial, implying that VE can manage with far fewer landmarks. All these results show that the variance equalization method performs better in terms of compactness than the equally spaced and the manual methods. Therefore, distributing the landmarks according to the total variance present on a training set creates a more compact model to build PDM for model-based segmentation methods such as those using ASMs. It does not depend on a starting point as does the equally spaced method, and it avoids the difficulties and drawbacks of manual annotation. Compared to the MDL approach, VE is a simple method, computationally far less expensive than MDL, and does not need a reference shape to
1274
[3] A Souza and J.K Udupa, “Automatic landmark selection for active shape models,” SPIE Medical Imaging, Conference on Image Processing, vol. 5747, pp. 1377–1383, 2005. [4] R.H. Davies, C.J Twining, T.F Cootes, J.C Waterton, and C.J Taylor, “A minimum description length approach to statistical shape modelling,” IEEE Transactions on Medical Imaging, vol. 21(5), pp. 525–537, 2002. [5] R.H. Davies, C.J Twining, P.D Allen, T.F Cootes, and C.J Taylor, “Building optimal 2d statistical shape models,” Image and Vision Computing, vol. 21, pp. 1171–1182, 2003. [6] H. H Thodberg, “Minimum description length shape and appearance models,” Information Processing in Medical Imaging, vol. LNCS 2732, pp. 51–62, 2003.
