Implicit function-based phantoms for evaluation of ... - Google Sites

Implicit function-based phantoms for evaluation of registration algorithms Girish Gopalakrishnan§ , Timothy Poston‡ , Nithin Nagaraj‡ , Rakesh Mullick§ , and Jerome Knoplioch† § Imaging

Technologies, GE Global Research Center, Bangalore, India Institute of Advanced Studies, Bangalore, India † GE Healthcare, BUC, France.

‡ National

ABSTRACT Medical image fusion is increasingly enhancing diagnostic accuracy by synergizing information from multiple images, obtained by the same modality at different times or from complementary modalities such as structural information from CT and functional from PET. An active, crucial research topic in fusion is validation of the registration (point-to-point correspondence) used. Phantoms and other simulated studies are useful in the absence of, or as a preliminary to, definitive clinical tests. Our virtual-lung-phantom-based scheme can test the accuracy of any registration algorithm and is flexible enough for added levels of complexity (addition of blur/antialias, rotate/warp, and modality-associated noise) to help evaluate the robustness of an image registration/fusion methodology. Such a framework extends easily to different anatomies. It would help the diagnosing clinician make a prudent choice of registration algorithm. Keywords: Image registration, software-based fiducial markers, multi-modality registration, phantom, validation

1. INTRODUCTION The registration problem is to find a transformation that best relates corresponding points in two images, in 2D or 3D: we focus on three dimensional scan images. In analyzing medical image, it is very useful to overlay their different representations of the same object. Co-registered images from the same scanner can aid in detection of temporal change. Those from different sources provide complementary information about the same tissue (for example, PET may detect a tumor while CT maps the bones around it, allowing a surgeon to navigate by fluoroscopy — which does not show the tumor).This is clearly beneficial in medical diagnosis and surgery. Several registration algorithms have been developed, each with a different optimization metric controlling the quality of the output. To select the most suitable technique, it is important to compare their capabilities. Also, a gold standard is easy to obtain and test in the case of rigid registration but not so in the non-rigid case. The lung matching problem is inherently non-rigid. Common tests of appropriateness include radiologist’s expertise, data with fiducial landmarks and phantom-based evaluation.1 The availability problems of the first two complicate both testing and inter-lab comparison. Phantom-based techniques are available ubiquitously and at low cost. An added advantage of using analytical virtual phantoms is that a non-rigid/perspective/affine deformation can be easily modelled on these and the recovery of this deformation can be validated, which is impossible on realdata without the use of invasive fiducial markers (which anyway distort the natural motion/behaviour of the anatomical region). The registration problem is best-posed when each image point corresponds — to equal precision in different modalities — to a point in the anatomy. For simplicity and generality, our initial development has avoided such modality-specific details in our initial development. However, we do not restrict attention to the case where a ‘true’ correspondence exists. Where each voxel’s value reflects the state in some specific small piece Send correspondence to Girish Gopalakrishnan, E-mail: [email protected], Telephone: +91 (80) 2503 3160, Address: Imaging Technologies Lab, General Electric - Global Research, John F. Welch Technology Center, Bangalore-560066, India.

of tissue, registration of two images should ideally overlay voxels that refer to corresponding groups of cells. This is straightforward to create artifically: take one image, deform it artificially, and challenge the registration algorithm to invert the deformation. However, in lung PET, breath motion creates an anisotropic, non-Gaussian blurring that prevents sharp geometric correspondence with breath-hold CT. It is therefore of interest to study registration where smooth topological matches do exist, but there is no uniquely correct one. Our mutable virtual lung-like phantom, defined by polynomial inequalities, is of this kind. We illustrate here its use for some sample registration algorithms and will describe a more comprehensive comparison elsewhere. 2

Figure 1. Changing A in (1) expands or shrinks the separate ‘lungs’ analogously to breath, with the branching little affected.

Figure 2. A small change in C in (2) has results similar to blurring with different choices of threshold, giving a thicker or thinner structure where the lungs meet.

2. ANALYTICAL PHANTOM

Figure 3. The negative sets of (1, 2) in the z = 0 plane.

Most organs within a human body have no sharp edges. Shapes easily built from geometric components like cylinders and ellipsoids usually do have corners and edges, absent major effort to fit interpolators where they meet. Smoothly defined global objects like the example below are simpler to code and modify, and implicitly defined surfaces classify points as In or Out by direct use of an inequality. Define functions f (x, y, z) = x2 y + Ax4 + By 4 + z 2 − C (1) and g(x, y, z) =

³ x ´2 L

+

³ y ´2 ³ z ´2 + −1 M N

(2)

with adjustable parameters A, B, C and L, M , N . The structure of the level sets is dominated by the plane

z = 0, since the z 2 term in f merely makes each point (x, y) with f (x, y, 0) = −c 2 into the center of an f ≤ 0 interval between f (x, y, −c) and f (x, y, c): similarly for g. Fig. X shows f ≤ 0 and g ≤ 0 for typical parameter values. We consider voxel (i, j, k) as ‘thoracic’ if g(x, y, z) < 0, ‘pulmonary’ if f (x, y, z) < 0 and ‘external’ otherwise. This gives a vaguely lung-like virtual phantom for registration tests. Each type of tissue randomly receives intensities to match the appropriate ranges from scanner acquisitions (certain T 1 /T2 values for MR, Hounsfield values for CT, etc.). Fig. 1 shows variations produced by parameter changes, centered on reference values (A, B, C, L, M, N ) = (2, 1/49, 0.00015, 1.08, 1.8, 0.9): only ‘thoracic’ voxels are rendered as opaque. To create a corresponding grid of values in an X×Y ×Z voxel array whose indices start at (0, 0, 0), set µ ¶ 4j 2k 2.4i (x, y, z) = − 1.2, − 3.5, −1 (3) X Y Z We may also transform this phantom by rotation, shear, or non-linear warping to obtain different image arrays. These images are inputs to a to-be-tested registration algorithm. A typical registration deforms one image (usually called the moving image) to fit another (called fixed ). The class of deformations allowed is a major aspect of the algorithm: for example, rigid transformations are appropriate if the scans are geometrically accurate images of an object which may be differently oriented in the scanner but is otherwise unchanged. If there are warping distortions in acquiring either or both images, or if the object may change shape by growth, breathing or pose, curvilinear transformations may be needed. An intermediate class of non-rigid transformations is the affine, which permits shear and unequal scaling but preserves the straightness of lines. Few image-acquisition schemes can be expected to exactly match straightness without rigidity, but affine transformations can often achieve a better-than-rigid correspondence by approximating more closely a ‘true’ curvilinear match specified by many more parameters. It is often faster to find the best affine fit than to search within the constraint of rigidity, which is defined by the non-linear ‘orthogonal matrix’ condition. Moreover, unlike most classes of curvilinear transformations used in registration, the affine ones form a group: the inverse of a one-to-one affine mapping is also affine, unlike the case for splines. The availability of an easily computed inverse is often useful enough to make it worth accepting slightly larger errors in the match. Most registration algorithms optimize some quantity, such as mutual information (MI) between the registered images, or some estimate of error, often in combination with penalty quantities such as for strong bending or folding. (The thin plate spline3 (TPS) algorithm produces by construction an exact ‘thin plate bending energy’ minimizing transformation that aligns the pairs of points — one on the moving image, one on the fixed — that it is told to match, but for unselected points does not guarantee even that the correspondence is one-to-one.) This makes comparison delicate, since for instance one would expect an MI-maximizing algorithm to score high in a scheme that compares MI values. Since our parameter-change shape changes are not created by choosing a deformation from a particular class, the comparison is not biased in favor of registration methods that exploit the same class. For example, matching the deformation produced by a thin-plate spline, is easier if one is optimizing among thin-plate splines and allows the matched points to vary. (Though, since a general TPS has no inverse TPS transformation, this bias is strong only if one matches the source/target direction of deformation.) We have used corresponding virtual fiducial markers (invisible to the algorithms tested) on the moving and fixed images, chosen separately from any fiducials that may be used by the algorithm . If the moving image was created by deforming the fixed one (so that the registration task is to reverse this deformation), fiducials on the fixed image deform correspondingly to points on the moving image. If the two images were created separately, as by different parameter values (A, B, C, L, M, N ), we choose corresponding points by hand. Evidently, the choice of fiducials — even with automatic homologues on the other image — introduces a subjective element. In other research4 we are pursuing the automated optimization of marker placement. Figure 3: Corresponding user-selected fiducials on two images defined by the values (A, B, C, L, M, N ) = and . Changing the parameters (A, B, C) and (L, M, N ) of the functions f and g respectively simulates the expansion and contraction of the lung and thorax during breathing, which changes shapes and volumes in a non-linear way. Changing A shrinks and expands the lung analogously to inspiration and expiration, while B modifies the overall lung size.The parameter C adjusts the thickness of the branching between the two lobes, analogously to

the effect of blurring: a factor such as movement spreads out the region where (choosing the threshold to retain connectedness) the lobes blend into each other. Finally, (L, M, N ) fixes ellipsoid axis lengths for the ‘thorax’.

Figure 4. Volume rendering of the 3D phantom (128x128x128) in the inspiration and expiration phases

3. RATING REGISTRATION ALGORITHMS The flow depicted uses images obtained by setting (A, B, C, L, M, N ) to the values in Figure 3, which form the input to each registration algorithm. We quantify error as the root mean square of the mismatch between transformed virtual fiducials on the moving image and their counterparts on the fixed image, giving a figure comparing the performance of different registration techniques. A transformed fiducial F i whose centroid is at 2 2 2 (xi , yi , zi ) but whose target is centred at (xi , yi , zi ) has squared errorpe2i = (xi − xi ) + (yi − yi ) + (zi − zi ) , and the root mean square (RMS) error for fiducials 0, . . . , i, . . . , n is e20 + . . . + e2i + . . . e2n .

3.1. A sample comparison Figure 5 shows the stages in comparison of four different registration algorithms that optimize different metrics: Algorithm Mattes rigid match5 Affine match∗ Thin-plate Spline Demons Metric optimized Mutual Information Mutual Information ‘bending energy’ Contour matching Algorithm 1 is Mattes’ registration5 that allows rigid translations and rotations, barring reflections and algorithm 2 seeks the best affine registration (the transformation maps parallel lines to parallel lines). Both seek to minimize Mutual Information (MI), using no additional input. Algorithm 3 finds the minimizer of an abstraction of bending energy among maps satisfying user-provided corresponding pairs from the two images (here chosen independently of the software markers). The fourth implements Thirion’s ‘demons’ algorithm, 6, 7 which views each input image as a set of iso-intensity contours. It assumes that voxels that are homologous by representing the same point have equal intensity on both images, after adjustment to match the histograms of the two images. Hence, Demons is good for deformable registration for images acquired with the same modality and the same setting. Where a bright and a dark region (for example, with high and low positron emissions) in one image correspond to regions with a single gray level in the other (for example, by equal X-ray opacities), it is much less effective. Figure 5 Results table: make some actual comments on which tested better! In this case, the original phantom was transformed by ∆x = 6 ∆y = 8 and ∆z = 6 to obtain floating image F Image1. F Image2 was obtained by rotating the fixed image by 5 degrees in all three directions. The rotation matrix used:   0.992 −0.087 0.087 0.000  0.094 0.992 −0.087 0.000   R=  −0.079 0.094 0.992 0.000  0.000 0.000 0.000 0.000

Figure 5. The flow of comparison.

Figure 6. A 3D rendering showing software-based fiducials

F Image3 was obtained after performing a rotation(5◦ in each direction), translation (∆ = 5 voxels along each direction) and scaling (a factor of 1.1) in the same sequence. The matrix used:   1.092 −0.096 0.096 4.964  0.104 1.091 −0.096 4.997   RT S =   −0.087 0.104 1.092 5.039  0.000 0.000 0.000 0.000 The floating image F Image4 was obtained by giving a TPS-based deformation [see Figure 7]. The other four floating images considered F Image5, · · · , F Image8 were obtained by using different values of (A, B, C, L, M, N )

Figure 7. Floating images used in this experiment

RMSE (pixels) Methods MI-based Rigid MI-based Affine MI-based Deformable Demons

Trans 1.46 2.39 0.32 11.72

Linear Trans + Rot Trans + 4.02 3.07 3.18 2.90

Rot + Scaling 9.13 4.52 1.26 15.67

Deformation 6.47 11.38 5.93 7.83

Non-linear A B 15.33 10.62 12.86 13.39 13.98 13.09 17.74 13.66

C 13.67 17.65 11.30 19.93

D 13.10 17.72 11.07 17.22

Table 1. Root Mean Square Error for different Algorithms. The parameters are A = (1.8, 1/49, 0.00015, 1.08, 1.8, 0.9) B = (2.2, 1/49, 0.00015, 1.08, 1.8, 0.9) C = (2, 1/49, 0.00015, 1.08, 1.8, 0.9) D = (2, 1/49, 0.0015, 1.08, 1.8, 0.9)

4. CONCLUSION AND FUTURE DIRECTIONS We have developed a framework for testing image registration algorithms (linear or otherwise) by phantoms with implicit algebraic definitions and modelled breathing by changing the parameters of the functions used. Using these phantoms, we illustrated the use of software-based markers for comparing Rigid, Affine, Landmark-based (thin plate splines) and Deformable (demons) registration algorithms. Such a framework is promising for evaluating linear (rigid and affine) and non-linear (deformable) algorithms for both intra-modality and inter-modality registration. The advantage of placing markers within the body surface (contrary to real images where fiducial markers are usually external) helps the user test an algorithm at complex geometries inside the volume. The ongoing experiments include a broader comparison,2 and a planned simulation of breathing, sinusoidally between inspiration and expiration, with averaged results. This will shed light on the effectiveness of different registration methods in cases where no ‘true’ match between the scanned images can exist.

REFERENCES 1. W. C. Lavely, C. Scarfone, H. Cevikalp, R. Li, D. W. Byrne, A. J. Cmelak, B. Dawant, R. R. Price, D. E. Hallahan, and J. M. Fitzpatrick, “Phantom validation of coregistration of PET and CT for image-guided radiotherapy,” Medical Physics , (Department of Radiology and Radiological Sciences, Vanderilt University, Nashville, USA), May 2004. 2. G. Gopalakrishnan, N. Nagaraj, R. Mullick, and T. Poston, “A software-based comparison of registration methods,” (Imaging Technologies, GE Global Research Center, Bangalore, India), In preperation. 3. F. L. Bookstein, “Principal Warps: Thin Plate Splines and the Decomposition of Deformations,” IEEE Transactions Pattern Analysis and Machine Intelligence Vol.11,Issue 6, pp. 567–585, 1989. 4. G. Gopalakrishnan, N. Nagaraj, R. Mullick, and T. Poston, “Automated optimal placement of virtual fiducials,” (Imaging Technologies, GE Global Research Center, Bangalore, India), In preperation. 5. D. Mattes, D. R. Haynor, H. Vesselle, T. K. Lewellen, and W. Eubank, “Non-rigid multi-modality registration,” Medical Imaging 2001: Image Processing , pp. 1609–1620, 2001. 6. J-P. Thirion, “Fast non-rigid matching of 3D medical image,” Technical report, Research Report RR-2547 , (Epidaure Project, INRIA Sophia), May 1995. 7. J-P. Thirion, “Image matching as a diffusion process; an analogy with Maxwell’s demons,” Medical Image Analysis , pp. 2(3):243–260, 1998.