Estimation of three-dimensional knee joint movement

Estimation of three-dimensional knee joint movement using bi-plane X-ray fluoroscopy and 3D-CT Hideaki Haneishi1), Satoshi Fujita2) and Takahiro Kohno2), Masahiko Suzuki3), Jin Miyagi3), Hideshige Moriya3) 1) Chiba University, Research Center for Frontier Medical Engineering 2) Chiba University, Graduate School of Science and Technology 3) Chiba University, School of Medicine ABSTRACT Acquisition of exact information of three-dimensional knee joint movement is desired in plastic surgery. Conventional X-ray fluoroscopy provides dynamic but just two-dimensional projected image. On the other hand, three-dimensional CT provides three-dimensional but just static image. In this paper, a method for acquiring three-dimensional knee joint movement using both bi-plane, dynamic X-ray fluoroscopy and static three-dimensional CT is proposed. Basic idea is use of 2D/3D registration using digitally reconstructed radiograph (DRR) or virtual projection of CT data. Original ideal is not new but the application of bi-plane fluoroscopy to natural bones of knee is reported for the first time. The technique was applied to two volunteers and successful results were obtained. Accuracy evaluation through computer simulation and phantom experiment with a knee joint of a pig were also conducted. Keywords: Bi-plane fluoroscopy, CT, registration, digital reconstruction radiograph, knee joint kinetics

1. INTRODUCTION Acquisition of exact information of three-dimensional knee joint movement is desired in plastic surgery. An X-ray fluoroscopy has been used to observe a moving picture of the knee joint movement [1].Conventional X-ray fluoroscopy just provides dynamic images but each image is only two-dimensional image For this issue, methods using digitally reconstructed radiographs called DRR have been studied [2-6]. Basic common concept is that if the DRR image matches to real fluoroscopy image, the position and direction of the knee is the same as those at the fluoroscopy image acquisition. A few studies of measurement of three dimensional knee kinematics using single-plane fluoroscopy has been reported [4,5]. Matching error in the direction from x-ray source to detector was not good enough in the study [1]. In the paper [5], good registration accuracy was achieved. However, the used object was artificial knee joints. Artificial knee joints have high CT number and easy to extract. On the other hand, for real knee joints, registration error becomes larger. Asano et al proposed to use a bi-plane DRR, however the matching operation was conducted fully manually [2]. Methods using optical markers and optical camera have also been proposed [3]. However, the accuracy was not good enough in that study. In this paper, a method using both bi-plane fluoroscopy and static three-dimensional CT is proposed for acquiring three-dimensional knee joint movement. To the author’s knowledge, it is first published that the registration method using both bi-plane fluoroscopy and CT image is applied to natural knee kinematics. Some details on the image processing are described. Techniques of initial estimate of registration parameters peculiar to the knee joint are included. Accuracy evaluation is also conducted based on a computer simulation and an experiment with a pig knee.

．

2. METHOD 2.1. Outline of the proposed method Outline of the proposed method is illustrated in Figure 1. In image acquisition phase, both bi-plane dynamic fluoroscopy and static three-dimensional CT images are acquired for a patient. From CT image, femur and tibia are first extracted. In computer, the projection geometry same as that of the real bi-plane fluoroscopy is constructed. Using the projection geometry of this virtual projection system, the extracted bones are projected onto virtual detectors. Those obtained image are DRRs as mentioned above. DRRs are then compared with the real fluoroscopy images. If those images

Medical Imaging 2005: Image Processing, edited by J. Michael Fitzpatrick, Joseph M. Reinhardt, Proc. of SPIE Vol. 5747 (SPIE, Bellingham, WA, 2005) 1605-7422/05/$15 · doi: 10.1117/12.594963

1667

exactly match each other, it means that the position and direction of each bone in the computer are the same as those of the real bones at acquisition of fluoroscopy. At initial stage of estimation, those two kinds of images does not match. Thus, an iteration processing is needed to register two kinds of images. Detail of this flow is described below.

Dynamic 2D info. (X ray fluoroscopy)

distortion correction & tone correction

edge enhance.

Bi-plane X-ray fluoroscopy

Static 3D info. (3D CT image)

projection

Bone extract. & segment.

edge enhance.

no

Match? yes

translation and rotation

3D visualization and analysis

Parameter update

Registration cycle Figure 1: Outline of the proposed method.

2.2. Calibration of the fluoroscopy system (1) Distortion model Prior to the above mentioned processing flow, calibration of fluoroscopy system must be conducted. The calibration consists of determination of distortion correction model of the detector and the matrix describing the projection geometry. The distortion correction model of the detector is determined as follows. An acrylic plate on which small metal balls are arranged in square grid is attached onto the detector and a blank image is captured. In the image, shadows of metal balls are clearly observed as a distorted array. From this image, the location of each dot is extracted. Though the extraction was carried out manually this time, automatic extraction would be possible. The distortion is modeled by polynomials of two spatial coordinates and the coefficients of the polynomials are determined by a least square method. u p = a1 + a 2 x p + a 3 y p + a 4 x 2p + a5 x p y p + a6 y 2p p = 1,2 (1)  2 2  v p = b1 + b2 x p + b3 y p + b4 x p + b5 x p y p + b6 y p Here, (u p , v p ) denotes coordinate of distortion corrected image and ( x p , y p ) denotes the coordinate of distorted image. The suffix p denotes the detector number of bi-plane fluoroscopy. (2) Calibration of projection system In bi-plane X-ray fluoroscopy, a cubic object with a metallic ball on each vertex is also captured for calibration as shown in Figure 2. Then the position of each ball in the fluoroscope image is read by manual operation. The relationship between the world coordinate of the object space and the image coordinate of the detector is given by a matrix as

1668

Proc. of SPIE Vol. 5747

u p    sp vp     1 

 c p,  11 =  c p , 21   c p , 31

c p ,12 c p, 22

c p ,13 c p, 23

c p ,32

c p ,33

 x c p ,14    y  c p , 24   .  z c p ,34   1

p = 1,2

(2)

Here the suffix p again denotes the detector number. s p is a value given by the inner product of the third row in the matrix C and the column vector of the world coordinate. In this equation, actual values, (u, v) of vectors in the left and (x, y, z) in the right hand sides are known for real balls on the vertices. The parameters of the matrix are determined from those pairs using a least square method. Image coordinate

v2

v1

u1

u2 z x

y

X ray source #1

X ray source World #2 coordinate (x,y,z) Figure 2: Calibration of fluoroscopy system with a cubic object.

2.3. Preprocessing of fluoroscopy and CT images (1) Fluoroscopy image preprocessing Prior to iteration for determining the registration parameters, there are still some preprocessing required for fluoroscopy and CT images. For fluoroscopy image, the tone curve correction must be done. While the pixel value of DRR image is summation of CT values that is linear with respect to absorption coefficient, the pixel value of the fluoroscopy is linear with respect to intensity of detected X-ray. Those values are reversal each other. Namely, if the fluoroscopy yields ideal output signal, there exists the following relationship between the ideal output signal of fluoroscopy and the projection data from CT, I p = − ln ideal = − ln I ideal + ln I 0 . (3) I0 Here, I ideal and p denotes ideal output signal and the projection data, respectively. I 0 dsenotes an ideal intensity for blank object. In fact, however, it is considered that dependent on the operators setting, the ideal signal is transformed to the real signal with gain A and offset B as (4) I real = AI ideal + B . Therefore, between the real fluoroscopy signal and the projection data from CT there is the following relationship. p = − ln(I real − B ) + log A + log I 0 (5) The unknown parameters, A and B are found as follows. At each frame, two parameters are determined by an optimization algorithm. Fundamentally, objective function is given by the sum of difference in pixel value between real and virtual projection data. (2) Edge enhancement of fluoroscopy image In our method, DRR is constructed from only projection of femur and tibia. On the other hand, real fluoroscopy is projection of not only femur and tibia but also other bones, cartilage and soft tissue such as muscles. So, the profiles of two kinds of projection images do not match inherently. Edge enhanced image tends to delineate the edge of femur and tibia clearly and useful in image matching. Real fluoroscopy images are therefore edge-enhanced as preprocessing. On


1669

the other hand, DRR image is edge-enhanced at each projection operation in the iteration process. Figure 3 shows the edge-enhanced images for real fluoroscopy and DRR image. DRR images (edgeenhanced)

Real X ray images (edgeenhanced)

Figure 3 Edge-enhanced projection images.

(3) CT image preprocessing Bone regions such as femur and tibia are extracted and segmented from 3D CT image of knee as shown Fig. 4. The other parts of CT volume data are eliminated. The rectangular parallelepiped including femur and tibia are defined, respectively, and the coordinate system of each bone whose origin is set at the center of each rectangular parallelepiped is defined. DRR images are synthesized as a sum of projection of two bones and the edge-enhancement is performed for those images. z femur

y

3D CT image x z

y tibia

x

Figure 4: Bone region extraction.

2.4. Iteration (1) Initial value setting The fluoroscopy image is a moving picture and includes many frames. Way to set the initial values depends on if the frame is the first frame or the other ones. For the first frame, initial value setting of six registration parameters for each bone is a harder task than expected. We have developed the following setting technique. In a femur there are a median epicondyle and a lateral epicondyle and those two points are easily identified from CT image. Two points in a bone axis can also be determined roughly with ease. On the other hand, in a fluoroscopy frame image, a bone axis of the femur and a line segment connecting two epicondyles can be given from visual inspection. Then four feature points in the CT image is projected onto a detector plane using the projection matrix and compared with the lines drawn in the fluoroscopy image. Practically the distances of the projected points from the line are calculated and the registration parameters are determined so as to minimize the distances. For the second and later frames, initial parameter setting is easy. Namely, the parameters determined in the optimization in the previous frame are used as those are. Since the movement of knee joint is generally smooth, such setting gives a good initial estimate in general.

1670


(2) Iteration cycle Based on the calibration mentioned above, those segmented regions are numerically projected onto a virtual fluoroscopy detector with the same geometry as the real fluoroscopy. Projection is calculated using shear warp method [7-10] (3) The calculated projection image is compared with the real image obtained by the real fluoroscopy and segmented bones in CT data are adjusted by translation and rotation until correlation between the real and the virtual projected images becomes maximum. Steps (1) to (3) are repeated for each time frame. In the iteration, Powell method was used [11]. The resultant information provides three-dimensional knee joint movement.

3. EXPERIMENT AND RESULTS 3.1. Experiment Image acquisition experiment with X-ray CT and bi-plane fluoroscopy was carried for two volunteers with normal knee movement. Since we have a similar result for two sets of data, we only show the result for one of two sets of data in this paper. Ninety frame fluoroscopy images captured in about three seconds were used for matching and movement of each bone was estimated. Furthermore, for visualization purpose, surface rendering images of bones were generated for each frame and a movie composed of those images was produced. Figure 5 shows a series of surface rendering images of the knee joint. From observation of the movie, the effectiveness of the proposed method has been confirmed.

No.1

No.2

Figure 6: Surface rendering images 40

60

30

30

50

20 10 0 -10

0

20 40 60 80 100 120 140 160 180 200

Translation Z [mm]

40

Translation Y [mm]

20 10 0 -10

0

40 30 20 10

0

Frame number -150

-30

-130

-160

-40

-140

-170

-50

-150 -160 -170 -180 -190

0 20 40 60 80 100 120 140 160 180 200

Frame number

-180 -190 -200 -210 -220

0 20 40 60 80 100 120 140 160 180 200

20 40 60 80 100 120 140 160 180 200

Frame number

-120

Rotation Y [deg]

Rotation X [deg]

Frame number

20 40 60 80 100 120 140 160 180 200

Rotation Z [deg]

Translation X [mm]

Figure 5: Bi-plane fluoroscopy system.

-60 -70 -80 -90 -100

0 20 40 60 80 100 120 140 160 180 200

Frame number Frame number Figure 7: Transition of six registration parameters of femur.


1671

3.2 Evaluation Three kinds of accuracy evaluation have been performed. (1) Distortion correction Goodness of the distortion correction model was first evaluated. Two data sets were obtained in the experiment and each set includes two detectors. Grid pattern of small metal balls detected by the detectors was analyzed. Non-distorted square grid that matches to the captured (distorted) grid in the central region in the image is first determined. The comparison between the non-distorted grid and the captured grid is shown in Fig. 8(a). Then the non-distorted grid is distorted though the distortion model and compared with the captured grid. The result is shown in Fig. 8(b). The graph shows a good agreement of two grids. The distance of the corresponding grid points was calculated as error of the distortion model. The result is summarized in Table 1. Mean error is around 0.4-0.5 pixels and the maximum error is up to around 1.1. The result is almost satisfactory one.

(a) Before correction

(b) after correction

Figure 8: Goodness evaluation of distortion correction model In (a), cross represents distorted pattern imaged by the detector, and square represents non-distorted grid pattern. In (b), cross represents real distorted pattern and square represents the model-distorted pattern.

Table 1 Goodness evaluation of distortion model. The unit is pixel. Data set Detector number Mean error 1 0.408 #1 2 0.400 1 0.433 #2 2 0.462

Maximum error 1.032 0.961 0.987 1.128

Standard deviation 0.437 0.448 0.443 0.448

(2) Computer simulation In order to evaluate the stability and accuracy of parameter optimization, computer simulation was conducted. Eight patterns of registration parameters for femur and tibia were assumed and the corresponding DRR images were generated and taken regard as real fluoroscopy images. For those fluoroscopy images, initial estimate for the parameters were given randomly in the range from -5 mm to +5mm for translation and from -5 degree to +5 degree for rotation, then optimization was performed. Resultant stability and accuracy is summarized in Table 2. It shows that for each bone, less than 1mm for displacement and less than 1 degree for rotation can be achieved. Table 2:Stability and accuracy evaluation of optimization though computer simulation. Displacement [mm] Rotation [degree] bone X Y Z X Y femur 0.084 0.057 0.033 0.110 0.164 tibia 0.294 0.311 0.153 0.511 0.599

Z 0.343 0.599

(3) Experiment with pig knee Accuracy evaluation was carried out using an object that is similar to real human knee as much as possible and makers for evaluation can be attached on. It is considered that knee of a pig is similar to human knee. So, we bought a frozen

1672


knee joint of a pig at a butcher and chip off the soft tissues to obtain only femur and tibia. Small metal balls were fixed on some proper points of the femur and the tibia. Those bones were fixed by an acryl-made fixation tool that can change the angle between the femur and the tibia in seven levels. The result is shown in Table 3. It shows that for each bone, less than 1.3 mm for translation and less than 1 degree for rotation can be achieved. The worst parameter was y translation of the tibia. Two rotation parameters are also bad in tibia. The possible reason is that the tibia has more roundish shape and therefore is not sensitive in terms of rotation in registration. Table 3: Accuracy evaluation of registration with a knee of a pig. Displacement [mm] bone X Y Z femur 0.939 0.627 0.451 tibia 0.489 1.210 0.240

X 0.718 0.898

Rotation [degree] Y 0.474 0.924

Z 0.893 0.582

CONCLUSIONS A method for estimating three-dimensional knee joint movement has been proposed. The successful experimental results have been obtained. Evaluation of the method was performed from three points of view. Accuracy of distortion correction was estimated that mean error is around 0.4-0.5 pixels and the maximum error is up to around 1.1. Stability and accuracy of optimization for registration was good so that for each bone, less than 1mm for displacement and less than 1 degree for rotation can be achieved. Finally, by the experiment with knee of a pig, it is shown that for each bone, less than 1.3 mm for translation and less than 1 degree for rotation can be achieved.

REFERENCES 1.

，

A. Baltzopoulo A videofluoroscopy methods for optical distortion correction and measurement of knee-joint kinematics. Clinical Biomechanics, vol.10, No.2, pp.85-92, 1995 2. T. Asano, M. Akagi, K. Tanaka et al, In vivo three-dimensional knee kinematics using a biplanar image-matching technique, Clinical Ortho and Related Res, 388, pp.157-166, 2001 3. SJ Piazza, PR Cavanagh: Measurement of the screw-home motion of the knee is sensitive to errors in axis alignment, Journal of biomechanics, 33, pp.1029-1034, 2000 4. Rahman H, Fregly BJ, and Banks SA, Accurate measurement of three-dimensional natural knee kinetics using single-plane fluoroscopy, 2003 Summer Bioengineering Conference, Key Biscayne, Florida, 2003 June 5. Yamazaki T, Watanabe T, Nakajima Y, Sugamoto K, Tomita T, Yoshikawa H, and Tamura S, Improvement of depth position in 2-D/3-D registration of knee implants using single-plane fluoroscopy, IEEE TMI, Vol. 23, No. 5, pp. 602-612, 2004 6. G. P. Penney, J.Weese, J. A. Little et al: A comparison of similarity measures for use in 2D-3D medical image registration: IEEE Med.Imag. vol. 17, No.4, pp.586-595, 1998 7. Jurgen Weese, Roland Gocke, Graeme P. Penney et al: Fast voxel-based 2D/3D registration algorithm using a volume rendering method based on the shear-warp factorization: Medical Imaging 1999: Image Processing; Proceedings of SPIE, 3661, 1999, pp.802-810 8. L. Zollei, E, Grimson, A. Norbash at el: 2D-3D Rigid Registration of X-Ray Fluoroscopy and CT images Using Mutual Information and Sparsely Sampled Histogram Estimators: IEEE CVPR, Vol. 2, pp.8-14, 2001 9. Jurgen Weese, Roland Gocke, Graeme P. Penney et al.: Fast voxel-based 2D/3D registration algorithm using a volume rendering method based on the shear-warp factorization: 1996 IEEE Workshop on Mathematical Methods Biomedical Imaging 10. Fast Volume Rendering Using a Shear-Warp Factorization of the Viewing Transformation: Philippe Lacroute, Marc Levoy: Proc. SIGGRAPH '94, Orlando, Florida, July, 1994, pp. 451-458 11. Press WH, Flannery BP, et al: Numerical Recipes in C 2nd Ed., Cambridge Univ. Press, Ch. vol. 10, pp.412-419, 1992


1673