3D REGISTRATION

IMPROVEMENT OF MANUAL 2D/3D REGISTRATION BY DECOUPLING THE VISUAL INFLUENCE OF THE SIX DEGREES OF FREEDOM Markus Kaiser?†‡

Matthias John‡

Tobias Heimann§

Thomas Neumuth?

Georg Rose†

?

Innovation Center Computer Assisted Surgery (ICCAS), University of Leipzig, Germany † Otto von Guericke University Magdeburg, Germany ‡ Siemens AG, Healthcare Sector, Forchheim, Germany § Siemens AG, Corporate Technology, Imaging and Computer Vision, Erlangen, Germany ABSTRACT

For instrument guidance in X-ray-based interventional procedures, 2D/3D registration is an important method to fuse preop 3D images with live X-ray images. The user modifies the six degrees of freedom (DOF) of the 3D image and thereby aligns a projection of it to the given X-ray images. Because of the projective geometry of X-ray, the classic handling of the six DOF during manual registration has side effects. Changing a specific parameter can visually influence the setting of other parameters and affect already adjusted registrations in other X-ray views. We propose an improved DOF handling such that the visual appearance of the object is consistent with the parameter changes by the user. Our evaluation shows that our method is superior to the classic approach for manually registering an ultrasound probe on biplane X-ray images. Index Terms— Manual 2D/3D-registration, image guided surgery, usability, ground truth estimation 1. INTRODUCTION 2D/3D registration is a crucial part in medical imaging and the enabling technology for image guided minimally invasive

surgery [1]. Here, pre-acquired 3D data of a patient is aligned with interventional 2D X-ray frames and then continuously overlaid and fused with live X-ray sequences. This can help in image understanding and instrument navigation which is shown in a wide range of applications [2]. Another use of 2D/3D registration is the intraoperative alignment of a model of a medical instrument or implants with X-ray images, e.g. to validate its position. In case the instrument is an imaging device like ultrasound or endoscope, such a model-based 2D/3D registration allows an indirect image-to-X-ray fusion [3]. Besides automatic 2D/3D registration, the manual registration is commonly needed. Practically, most 2D/3D registrations for organ-to-organ applications during interventions are done manually. Other examples are manual correction, in case the automatic solution fails, as well as to establish a ground truth transformation for testing data sets. Manual 2D/3D registration is often the best possible way of acquiring a suitable ground truth transformation. In this paper, we focus on the 2D/3D registration of a transesophageal echo (TEE) probe to 2D X-ray frames. But the proposed technique is not limited to this special application of 2D/3D registration. 2D/3D registration can be done on just one X-ray image (“monoplane”). This has the drawback that three so called

Image plane

image plane

+θroll

-θroll z

Optical axis

y

-x +θyaw +θpitch

x

-z

-y -θyaw

n

-θpitch

(b) Rigid parameters of the object

Fig. 1. CT volume of the ultrasound probe and coordinate system of the TEE probe with C-arm projection geometry.

AT T

X-ray source

nT' n

(a) CT volume

AT proj

nT

X-ray source

projection

A

Aproj

T' AT' ψ

AT' proj

Fig. 2. Illustration of the rotation angles observer problem while translating an object under projective geometry.

observer view directions

observer view directions

Plane A

Plane A

cp

image plane c3D

ground truth position

ground truth position

Plane B

Plane B

starting position

(a) Conventional

c'

starting position

central beam

φp

out-of-plane parameters (ty , θpitch , θroll in Fig. 1b) are difficult to determine because they only cause a small change of the projected 3D object which can be hardly visible on the projection image. Therefore, one can do biplane registration where the out-of-plane parameters of one image are partially resolved to in-plane parameters of the other image. The best way is, taking two images of the object from two different directions with an offset of 90◦ . Two of the out-of-plane parameters correspond to in-plane parameters in the other image and are therefore easier to determine with good accuracy. Due to the projective geometry of a C-arm system, several issues occur during monoplane and biplane 2D/3D registrations. If an object is shifted parallel to the image plane (without changing the common 3D rotational parameters), the user can observe a rotation of the projected object in the 2D image. This change depends on the distance from the object to the central beam. In Fig. 2, object A is translated with the transformation T in 3D space parallel to the image plane which results in a new object position AT . Although the model was not rotated, the observer of the projections of the objects Aproj and AT proj has the impression of a changed set of rotational angles. This results in errors ψ of up to 9◦ , depending on the projection geometry of the C-arm. Such effects can have a huge influence on the registration quality and the usability of a 2D/3D registration system, especially during biplane registration. The normal biplane registration process is to (1) try to register the object on image A, (2) transfer the registration matrix to plane B, (3) try to register the object on image B, (4) transfer the registration matrix to plane A and repeat until an adequate result is reached. In a clinical setup, it might not be possible to use two views which are orthogonal to each other. If both views are displaced less or more than 90◦ , changes of translation and rotation in plane B destroy the recent registration of plane A and vice versa. This ends up in an iterative process until the both positions are converging (see Fig. 3a). We introduce a concept which uses “planar parameters” which solves the issue of observed change in rotational parameters. This concept can be used as well to overcome this iterative process and to keep the registration in one plane stable while registering on the other.

m3D

mp

φr

f' e

cmp

φy

(tx,ty,tz)

(b) Planar approach

Fig. 3. Comparison of biplane registration processes.

c

s f

eye

s'

Fig. 4. Planar parameters. 2. MATERIALS AND METHODS We use rotational parameters relative to the projection image plane (see Fig. 1a), i.e. if all angles are set to zero and the object is positioned in the central projection ray, we get an axis-parallel rotational initial position. We define: - The roll angle θr is the rotation around the center line of the object. The center line must be defined, but usually it exists a natural one for each object - The tilting of the object relative to the image plane is given by the pitch θp . - The yaw θy is the rotation parallel to the image plane. The corresponding registration matrix is computed by: R = Ttranslation · Rθy · Rθp · Rθr

(1)

A C-arm can rotate to different angulations, that show an object from different views. A local registration matrix R1 for a C-arm rotation C1 can be converted to a global registration matrix by: R = C1−1 · R1

(2)

R is a representation of the registration relative to the patient table. Therefore, it is independent of the C-arm angulation. The corresponding local transformation matrix R2 for another C-arm rotation C2 can be computed by: R2 = C2 · R

(3)

In the following we are going to introduce the terms planar and spatial parameters. Spatial rotations {θy , θp , θr } rotate around the axes aligned with the image plane and are independent of the translational position of the object. In contrast, we define planar parameters {φy , φp , φr } as the rotational angles observed by the user in the projected image (Fig.

4). By definition, planar parameters remain unchanged if the object is translated or rotated - except for the modified parameter. This is represented by AT 0 and its transformation T 0 in Fig. 2. We propose that the user should only work with planar parameters, because they reflect the observable parameters on the projected object. These parameters must be converted to spatial parameters to calculate the actual 3D transformation matrix of the current object. Therefore, in 2.1 we define functions to bidirectionally convert planar and spatial parameters first for a monoplane systems. In 2.2 we show how to apply the concept to biplane registration .

cpA

(IA)

(IB

)

eye-cA cA PA m'pA

cpB nA

cB mpB

mpA

eye-mB PB

eye-cB nB

LmpA c' m3D m'3D

Lm'pA

f' eyeB

2.1. Planar parameters for monoplane 2.1.1. Converting spatial to planar parameters

eyeA

Fig. 5. Planar parameters for biplane setup.

Giving a set of all six 3D spatial parameters S = {tx , ty , tz , θy , θp , θr } and a projection matrix P , we can compute their planar parameters {φy , φp , φr }. Planar angles are dependent on the vector e, pointing from eye to rotation center m3D . The objects’ base vectors c, s, f , which build the objects rotation matrix, must be adapted to the view direction e. By knowing the center line vector c we establish an orthogonal system which is related to e with:

where Rφp and Rφr are rotation matrices built from their appropriate Euler angles. Rφy is more difficult to compute because it depends on both preceding matrices. To ensure that we use the planar yaw, we determine it based on the 2D image plane which is the direction vector cmp pointing from mp to cp : cp = mp + [sin(φy ), 0, cos(φy )]

s0 = c × e

(4)

f 0 = s0 × c

(5)

c0 = s0 × e

(6)

The planar rotation angles are computed as follows: ( +cos−1 (f 0 ◦ e) : cos−1 (f 0 ◦ c0 ) − π2 < 0 φp = −cos−1 (f 0 ◦ e) : cos−1 (f 0 ◦ c0 ) − π2 ≥ 0 ( +cos−1 (f 0 ◦ f ) : cos−1 (f 0 ◦ s) − π2 < 0 φr = −cos−1 (f 0 ◦ f ) : cos−1 (f 0 ◦ s) − π2 ≥ 0

(7)

(8)

The in-plane yaw angle is computed with the use of the projected center line vector cmp :

(12)

while mp is the projected object center point and cp the projected object point in center line direction with applied planar yaw rotation φy . The matrix Rφy can be composed with the base vectors s, f , c:   sx fx cx 0 sy fy cy 0  (13) Rφy =  sz fz cz 0 0 0 0 1 f = mp − eye

(14)

s = (cp − eye) × f

(15)

c=s×f

(16)

2.2. Planar parameters for biplane cmp = P · c3D − P · m3D ( +cos−1 (cmyp ) : cos−1 (cmxp ) − φy = −cos−1 (cmyp ) : cos−1 (cmxp ) −

(9) π 2 π 2