Bone morphing with statistical shape models for

Bone Morphing with statistical shape models for enhanced visualization Kumar T. Rajamania , Johannes Hugb , Lutz-Peter Noltea , Martin Stynera a M.E.

M¨ uller Research Center for Orthopaedic Surgery, Institute for Surgical Technology and Biomechanics, University of Bern, P.O.Box 8354, 3001 Bern, Switzerland b sd&m Schweiz AG, World Trade Center, Z¨ urich, Switzerland ABSTRACT

This paper addresses the problem of extrapolating extremely sparse three-dimensional set of digitized landmarks and bone surface points to obtain a complete surface representation. The extrapolation is done using a statistical principal component analysis (PCA) shape model similar to earlier approaches by Fleute et al.1 This extrapolation procedure called Bone-Morphing is highly useful for intra-operative visualization of bone structures in image-free surgeries. We developed a novel morphing scheme operating directly in the PCA shape space incorporating the full set of possible variations including additional information such as patient height, weight and age. Shape information coded by digitized points is iteratively removed from the PCA model. The extrapolated surface is computed as the most probable surface in the shape space given the data. Interactivity is enhanced, as additional bone surface points can be incorporated in real-time. The expected accuracy can be visualized at any stage of the procedure. In a feasibility study, we applied the proposed scheme to the proximal femur structure. 14 CT scans were segmented and a sequence of correspondence establishing methods was employed to compute the optimal PCA model. Three anatomical landmarks, the femoral notch and the upper and the lower trochanter are digitized to register the model to the patient anatomy. Our experiments show that the overall shape information can be captured fairly accurately by a small number of control points. The added advantage is that it is fast, highly interactive and needs only a small number of points to be digitized intra-operatively. Keywords: Statistical Shape Model, Principal Component Analysis, Bone Morphing, Deformable Models, Computer Assisted Visualization

1. INTRODUCTION Computer Assisted imaging techniques such as CT, MRI have gained great acceptance for use in diagnosis and therapy planning. To avoid the high radiation dose and costs associated with such scans, current trends are towards non-ionized or minimal imaging. Image free techniques using model building is also being actively researched to provide surgical guidance. These techniques are applicable in surgeries such as Total Knee Arthroplasty (TKA), Total Hip Arthroplasty and Anterior Cruciate Ligament reconstruction (ACL) where only pre-operative X-Ray is available. Constructing a patient specific 3D surface in non-image based approach is quite challenging. This is usually done by building a deformable model and adapting the model to the patient anatomy. The use of model based a-priori knowledge to simplify and stabilize problems has long been explored in the computer vision community. It began with the introduction of deformable models in its various forms such as snakes,3 deformable templates4 and active appearance models.5 The amount of prior knowledge included in these models varies from simple smoothness assumptions to very detailed knowledge about the surface. In the field of medical imaging, the usage of statistical shape models has found widespread use6, ,7 since the notion of biological shape could be best defined by a statistical description of a large population. In order to provide similar sophisticated visualization similar to image based CAS systems, Fleute et al1 proposed a model based technique to extrapolate the 3D patient specific surface from digitized landmarks and Further author information: (Send correspondence to Kumar T. Rajamani) Kumar Rajamani: E-mail: [email protected], Telephone: +41 31 632 9994

122

Medical Imaging 2004: Visualization, Image-Guided Procedures, and Display, edited by Robert L. Galloway, Jr., Proceedings of SPIE Vol. 5367 (SPIE, Bellingham, WA, 2004) · 1605-7422/04/$15 · doi: 10.1117/12.535000

Figure 1. Four selected proximal femur structure from our collection that consists of 14 samples

bone surface points obtained during surgery. The extrapolation procedure also called bone morphing is done via a statistical principal component analysis (PCA) based shape model. The model surface is fitted to the sparse intra-operative data via jointly optimizing morphing and pose. Chan et al2 use a similar algorithm, but optimize morphing and pose separately using an iterative closest point (ICP) method.8 These approaches to bone morphing based on fitting procedures in Euclidean space have the disadvantage that these are often computationally expensive and only a small set of shape variations can be considered. Also non-spatial information such as patient height, weight, sex information cannot be incorporated in these earlier techniques. In this paper we propose a novel bone morphing scheme that generates patient-specific 3D knee surfaces from sparse intra-operative digitized data. Our main goal is to provide an interactive 3D visualization tool that takes into account the prior knowledge of shapes as far as possible. We also need to get a good estimate and describe the ”unknown” shape of the object with minimal number of digitized points. In our novel bone morphing scheme we progressively remove shape information represented by digitized points from the PCA model.10 The extrapolated surface is then computed as the most probable surface in the shape space given the data. The 3D model is hence obtained by deforming the statistical model to match the digitized data. We can also get additionally a map displaying the expected remaining variability. In this paper we demonstrate proof of principle of our method using a proximal femur model generated from 14 CT datasets and evaluated using leave-one-out experiments. In the following, Section 2 reviews shortly the statistical shape analysis using principal components and explains the mathematical notation. In Section 3, we discuss in detail bone morphing via the progressive subtraction of variation. Section 4 presents out first results of bone morphing using our technique. Finally, Section 5 concludes this report and outlines the next steps.

2. MODEL CONSTRUCTION USING PRINCIPAL COMPONENTS In order to have a efficient statistical shape description at our disposal, we employ a representation that is based on a principal component analysis (PCA) of all object instances in our database. The basic idea of a PCA model11 consists in separating and quantifying the main variations of shape that occur within a training population of objects. PCA defines a linear transformation that decorrelates the parameter signals of the original shape population by projecting the objects into a linear shape space spanned by a complete set of orthogonal basis vectors. If the parameter signals are highly correlated, then the major variations of shape are described by the first few basis vectors. Furthermore, if the joint distribution of the parameters describing the surface is Gaussian, then a reasonably weighted linear combination of the basis vectors results in a shape that is similar to the existing ones. A key step in the construction of statistical model is establishing a dense correspondence between the surface boundaries for the members of the training set. In 2D, correspondence is often established using manually determined landmarks, but this is a time-consuming, error-prone and subjective process. In principle, the method extends to 3D, but in practice, due to very small sets of reliably identifiable landmarks, manual landmarking becomes impractical. Most automated approaches posed the correspondence problem as that of defining a parameterization for each of objects in the training set, assuming correspondence between equivalently parameterized points. Proc. of SPIE Vol. 5367

123

In our earlier paper12 we performed a comparative study of some of the popular correspondence establishing methods for model based applications. We analyzed both the direct correspondence via manually selected landmarks as well as the properties of the model implied by the correspondences, in regard to compactness, generalization and specificity. The studied methods include a manually initialized subdivision surface (MSS) method and three automatic methods that optimize the object parameterization: SPHARM,13 MDL14 and the covariance determinant (DetCov)15 method. In all studies, DetCov and MDL showed very similar results. The model properties of DetCov and MDL were better than SPHARM and MSS. The results suggested that for modeling purposes the best of the studied correspondence method are MDL and DetCov. [1]

[1]

[1]

Our population consisted of 14 proximal femor instances, given as point distribution models pi = [xi , yi , zi ,[M [M ] [M ] ..., xi , yi , zi ]T with M = 4096 points. Five selected examples of population are illustrated in Fig. 1. Correspondence among the population members was initialized using the semi-automatic landmark driven method(MSS) and then optimized based on the MDL criteria. The shapes are first centered by calculating the average model p ¯ and computing the instance specific difference vector ∆pi s . N 1 p ¯= pi , N i=1

∆pi = pi − p ¯,

∆P = [∆p1 · · · ∆pN ]

(1)

missing dimension is due to the linear dependence: NThe difference vectors span a N −1 dimensional space. The 3M ×3M ∆p = 0. The corresponding covariance matrix Σ ∈ I R is hence rank deficient. This circumstance is i i=1 exploited to speed the calculation of the valid eigenvalues and eigenvectors. Instead of calculating the full eigen ˘ with ∆P leads system of the covariance matrix Σ , the multiplication of the eigenvectors of a smaller matrix Σ to the correct principal components:

˘ Σ

=

U

=

1 PCA ˘ ˘ T ∆P T ∆P = U ΛU , N −1 ˘ [u1 · · · uN −1 , uN ] = ψ ∆P U

Λ = diag(λ1 , · · · , λN −1 , 0) ψ (A) = N ormalize columns of A

(2) (3) (4)

The sorted eigenvalues λi and corresponding eigenvectors ui of the covariance matrix are the principal directions spanning a shape space with p ¯ representing its origin. Objects pi in that shape space can be described as ¯ ) into the eigenspace.(5) linear combination with weights bi calculated by projecting the difference vectors (pi − p bi = U T ∗ ∆pi ;

pi = p ¯ + U · bi

(5)

Figure 2 shows the variability captured by the first two modes of variation of our model varied by ±2 standard deviation. The shapes representing the first eigenmode in the first row are calculated by adding the ¯ . The bottom row with the second eigenmodes are calculated weighted first eigenvector u1 to the average model p correspondingly.

3. BONE MORPHING USING PROGRESSIVE ELIMINATION OF VARIATION We developed a novel morphing scheme operating directly in the PCA shape space incorporating a large set of possible variations including parameters additionally to spatial information such as patient height, weight and age. Our method is based on the iterative removal of shape information associated with the digitized points. First we calculate the most probable shape subject to the boundary conditions that are related to the three initially digitized landmarks. The resulting outline is used for registration and as an initial configuration for computing the most probable shape for the next digitized point. Using statistical shape analysis, we examine the remaining shape variability, after the surface information coded by the digitized points is progressively subtracted. This procedure of point selection and variability removal is repeated until a close approximation 124

Proc. of SPIE Vol. 5367

√ -2 λk

√ -1 λk

√ +1 λk

√ + 2 λk

λ1

λ2

Figure √ 2. The first two eigen modes of variation of our model. The shape instances were generated by evaluating x ¯ + ω λk uk with ω ∈ {−2, .., 2}

to the patient anatomy is achieved. The final extrapolated surface represents the most probable surface in the shape space given the digitized landmarks. We detail below the method used to compute the most probable shape given the position of an arbitrary point, and also the method to subtract the variation coded by digitized points.

3.1. Shape-based Basis Vectors for One Point In the context of PCA the mean model is the most probable surface. Hence the ’most probable surface’ that satisfies all the boundary conditions would be the surface that has minimal deviation from the mean. This means that we must choose the model with minimal Mahalanobis-distance Dm . To determine the most probable surface given the position of an arbitrary point j, we must thus find the minimally Mahalanobis-distant surface b that contains the point j at the required position pj . In order to do so for all possible displacements pj , we should seek for three shape vectors ∈ span(U ) that translate the point j in either x- y- or z-direction (in the object space), thereby only inducing a minimal deformation of the overall surface. More precisely, the three shape vectors shall effect a unit displacement of the vertex j in either x- y- or z-direction, and, as there are probably many such vectors, they shall be of minimal Mahalanobis-length Dm . If we have found them, we can satisfy all possible boundary conditions pj with minimal deformation of the surface by just adding the three appropriately weighted “basis” vectors to the mean. This problem gives rise to the following constrained optimization: Let rxj , ryj and rzj denote the three unknown basis vectors causing unit x- y- and z- translation of point j, respectively. The Mahalanobis-length Dm of these three vectors then given by:

Dm (rk ) = (U rk )T Σ−1 U rk = rTk Λ−1 rk =

N −1 e=1

[e]

rk

2

λe

, k ∈ {xj , yj , zj }


(6)

125

Taking into account that xj , yj and zj depend only on three rows of U , we define the sub-matrix Uj according to the following expression:        [2j−2]  U xj xj xj  yj  = y j  + U [2j−1]  b = y j  + Uj b, U [j] = j th row of U (7) [2j] zj zj z U j In order to minimize the Mahalanobis-distance Dm subject to the constraint of a decoupled x- y- or ztranslation by one unit, we establish — as is customary for constrained optimizations — the Lagrange function L: 2       N −1 r [e] 1 0 0 k (8) L(rk , lk ) = − lTk [Uj rk − ek ] , e{xj ,yj ,zj } = {0 , 1 , 0} λe e=1 0 0 1 The vectors lxj , lyj and lzj contain as usual the required Lagrange multipliers. To find the minimum of L(rk , lk ), k ∈ {xj , yj , zj }, we calculate the derivatives with respect to all elements of rxj , ryj , rzj , lxj , lyj and lzj and set them equal to zero: 

 .. .    .. .. δL(rk , lk )   T . =0, . −Uj    δrk   .   .. 2    δL(rk , lk )  λN −1    =0 . . . . . . . . . . . . . . . . . . . . . . . . . δlk   .. . 0 U 2   λ1

j



   .. .. .. .. . . . .        .. ..  .. .. r   0 . 0 . 0  xj . ryj . rzj         .. .. .. ..  =  . .    . .      . . . . . . . . . . . . . . . . . . .  . . . . . . . . . . . . . . . . . . .     .. .. .. .. ex . ey . ez . l . l l xj

yj

zj

j

j

(9)

j

If the basis vectors and the Lagrange multipliers are combined according to Rj = [rxj ryj ] and Lj = [lxj lyj ], Eq. (9) can be rewritten as two linear matrix equations: 2Λ−1 Rj = UjT Lj

(10)

Uj Rj = I

(11)

The three basis vectors rxj , ryj and rzj then result from simple algebraic operations ( resolve (10) for Rj and replace Rj in (11) by the result, use the resulting equation to find Lj = 2[Uj ΛUjT ]−1 . Substitute for Lj in (10) ) and RJ is given by: −1 (12) Rj = rxj ryj rzj = Λ UjT Uj Λ UjT While rxj describes the translation of xj by one unit with constant yj , zj and minimal shape variation, ryj ˇ given the displacement [∆xj , ∆yj , ∆zj ]T and rzj alter yj and zj correspondingly. The most probable surface p of an arbitrary control vertex j is then obviously determined by   ∆xj ˇ = p + U Rj  ∆yj  . p (13) ∆zj

3.2. Point-wise Subtraction of Variation In the previous section we have seen how to estimate the most probable shape given the position of one specific control vertex j. Before we proceed to the next control point, we ensure that subsequent shape modifications will not alter the previously adjusted vertex j. To do so, we remove those components from the statistic that cause a displacement of this point. Therefore, we subtract the variation coded by the point j from each instance

126


√ -2 λk

√ -1 λk

√ +1 λ1

√ + 2 λk

λ1

λ2

Figure 3. The first two eigen modes of variation of our model after the variability associated with one landmark point has been removed from the population.

i, and rebuild the statistic afterwards. For the first part of this operation, we must subtract the basis vectors Rj weighted by the specific displacement [∆xj , ∆yj , ∆zj ]Ti , from each object instance i:   ∆xj ˆ bji = bi − Rj  ∆yj  = bi − Rj Uj bi = (I − Rj Uj ) bi ∆zj i

(14)

By doing so for all instances, we obtain a new shape description that is invariant with respect to point j. On the assumption that the initial shape parameters are however correlated, in particular, we obtain a pointnormalised population whose total variability is smaller than the one of the original collection. In order to verify this assumption and to rebuild the statistic, we apply anew a PCA to the normalised set of shape instances ˆ {bji | i ∈ {1, . . . , N }}. Note that the dimensionality of the resultant eigenspace decreases by three (compared to the original one), as we have just removed three degrees of freedom. The point-normalised principal components, ˆ denoted by U j , confirm the expected behaviour and validate the removal of the variation in respect of point j. The first two dominant one-point invariant eigenmodes are illustrated in Fig.3.

3.3. Computing the Most Probable Surface Having such a progressive shape analysis technique at our disposal, we can now find the most probable surface by repeatedly applying the point selection and elimination procedure, until the overall shape variability is sufficiently small. The morphed surface is hence extrapolated from few landmarks and digitized points. Analogous to traditional parametric curve representations, each digitized point j is associated with the three principal basis functions U Rj that are globally supported. The final most probable surface p ˇ l is given by reversing the point elimination procedure, that is, by combining the surface information that is coded by the various control points. Accordingly, we combine the mean surface sˆ p with all the weighted principal basis functions Rjkk−1 :


127

  ∆xjk sˆ p ˇl = p + U sˆk−1 Rjkk−1  ∆yjk  ∆zjk k=1 l

(15)

sˆ

The weights [∆xjk , ∆yjk , ∆zjk ]T for the basis vectors U sˆk−1 Rjkk−1 depend on the surface defined by the previous principal landmarks sˆk−1 . To emphasise the hierarchical structure of our formalism and to simplify the algorithmic implementation, we use the following recursive definition instead of Eq. (15):

p ˇ0 = p ,

  ∆xjk sˆ p ˇk = p ˇ k−1 + U sˆk−1 Rjkk−1  ∆yjk  ∆zjk

(16)

With this shape representation of p ˇ k , we have a novel bone morphing method that can estimate unknown patient specific models using the simple adjustment of a small number of points, taking into account all the prior knowledge of the surface.

4. RESULTS In this paper we demonstrate proof of principle of our method using proximal femur structures. 14 CT scans of the proximal femur were segmented and a sequence of correspondence establishing methods was employed to compute the optimal PCA model. Then, leave-one-out experiments were carried out to evaluate the new method. Three anatomical landmarks, the femoral notch and the upper and the lower trochanter are used as the first set of digitized points. This simulates the surgical scenario where we need to initially register the model to the patient anatomy. The remaining points are added uniformly across the spherical parameterization so that they occupy different locations on the bone surface. In most surgeries only a portion of the bone is accessible. We simulated this in our leave-one-out analysis with six digitized points, by digitizing only on the femoral head. The average error of the bone morphing across the whole surface was 4.2mm (Figure 5). Our scheme is novel in that it operates directly in the PCA shape space and incorporates the full set of possible variations including other patient parameters such as height, weight and age. It is fully interactive, as additional bone surface points can be incorporated in real-time. The expected accuracy of the current model can be visualized at any stage of the procedure. This could be used to give additional visual (color-coded) feedback to the surgeon to highlight regions of high and poor expected accuracy.

5. CONCLUSIONS In this paper we have demonstrated a novel technique to predict the three dimensional model of a given anatomy using statistical shape models. Earlier bone morphing methods based on fitting procedures in Euclidean space have the disadvantage that these are often computationally expensive and only a small set of shape variations can be considered. Also non-spatial patient information cannot be incorporated in these approaches. Using these methods, the morphed model does not represent the most probable shape given the input data but rather a constrained fit. Our scheme is novel in that it operates directly in the PCA shape space and incorporates the full set of possible variations. We can include parameters additionally to spatial information such as patient height, weight and age. It is fully interactive, as additional bone surface points can be incorporated in real-time. Our experiments on this proximal femur data show that our method can capture the overall shape information fairly accurately by a small number of control points. This method has the advantage in that it is fast, highly interactive and needs only a small number of points to be digitized intra-operatively. A visual (color-coded) feedback to highlight regions of high and poor accuracy assists the surgeon to choose regions to digitize additional points. The expected accuracy of the current model can be visualized at any stage of the procedure.

128


√ -2 λk

√ -1 λk

√ +1 λk

√ + 2 λk

λ1

λ2

Figure 4. The first two eigen modes of variation of our model after the variability associated with three landmark point has been removed from the population.

Figure 5. View of the predicted most probable surface overlaid on top of the actual object in a leave-one-out analysis with six points being digitized only on the femoral head. The average error of the bone morphing in this case was at 4.2 mm


129

The relation between shape and parameters such as sex, height, weight, pathology is an interesting area of research. The formulation and implementation of our bone morphing is such that we can include these other patient parameters into the estimation procedure. Alternatively we can also predict the parameters based on the surface using the morphing method. In our future work we plan to study the importance and the predictive power of these parameters. We also plan to incorporate surface points extracted from ultrasound into the morphing scheme.

ACKNOWLEDGMENTS We thank Guido Gerig, G´ abor Sz´ ekely and Gabor Zsemlye for insightful discussions about modeling and shape prediction. The datasets were provided by Frank Langlotz via the Swiss CO-ME network. The MDL tools were provided by Rhodri H. Davies. The error computations were carried out using MESH available at http://mesh.epfl.ch. This work was funded by AO/ASIF foundation.

REFERENCES 1. M. Fleute and S. Lavallee, “Building a complete surface model from sparse data using statistical shape models,” MICCAI, pp. 879–887, 1998. 2. C. S. Chan, P. J. Edwards, and D. J. Hawkes, “Integration of ultrasound-based registration with statistical shape models for computer-assisted orthopaedic surgery,” SPIE, Medical Imaging, pp. 414–424, 2003. 3. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour models,” Int. Journal of Computer Vision 1(4), pp. 321–331, 1988. 4. A. Yuille, D. Cohen, and P. Hallinan., “Feature extraction from faces using deformable templates.,” Proc. CVPR’89, pp. 104–109, 1989. 5. T. Cootes, C. Taylor, D. Cooper, and J. Graham, “Active shape models - their training and application,” Computer Vision and Image Understanding 61, pp. 38–59, 1995. 6. A. Kelemen, G. Székely, and G. Gerig, “Three-dimensional Model-based Segmentation,” in Proc. of MB3IA’98, 1998. 7. A. Kelemen, G. Székely, and G. Gerig, “Elastic model-based segmentation of 3d neuroradiological data sets,” IEEE Transactions on Medical Imaging 18, pp. 828–839, October 1999. 8. Y. Ge, C. Maurer, and J. Fitzpatrick, “Image registration using the iterative closest point algorithm with a closestpoint transform,” in SPIE, 2710, pp. 358–367, 1996. 9. D. Zorin, P. Schr¨ oder, and W. Sweldens, “Interactive multiresolution mesh editing,” Computer Graphics 31(Annual Conference Series), pp. 259–268, 1997. 10. J. Hug, C. Brechb¨ uhler, and G. Székely, “Model-based initialisation for segmentation,” in Proceedings 6’th European Conference on Computer Vision - ECCV 2000, Part II, D. Vernon, ed., Lecture Notes in Computer Science, pp. 290–306, Springer, June 2000. 11. T. Cootes and C. Taylor, “Active Shape Models - ‘Smart Snakes’,” in British Mach. Vision Conf., pp. 266– 275, Springer-Verlag, 1992. 12. M. A. Styner, K. T. Rajamani, L. P. Nolte, G. Zsemlye, G. Szekely, C. J. Taylor, and R. H. Davies, “Evaluation of 3d correspondence methods for model building,” IPMI, pp. 63–75, 2003. 13. C. Brechb¨ uhler, G. Gerig, and O. K¨ ubler, “Parametrization of closed surfaces for 3-D shape description,” Computer Vision, Graphics, Image Processing: Image Understanding 61, pp. 154–170, 1995. 14. R. Davies, C. Twining, T. Cootes, J. Waterton, and C. Taylor, “A minimum description length approach to statistical shape modeling,” IEEE Transactions on Medical Imaging 21, pp. 525–537, May 2002. 15. A. Kotcheff and C.J.Taylor, “Automatic construction of eigenshape models by direct optimization.,” Medical Image Analysis, pp. 303–314, 1998.

130
