Haptic computer-assisted patient-specific preoperative ... - CS - Huji

Int J CARS DOI 10.1007/s11548-015-1162-9

ORIGINAL ARTICLE

Haptic computer-assisted patient-specific preoperative planning for orthopedic fractures surgery I. Kovler · L. Joskowicz · Y. A. Weil · A. Khoury · A. Kronman · R. Mosheiff · M. Liebergall · J. Salavarrieta

Received: 12 December 2014 / Accepted: 10 February 2015 © CARS 2015

Abstract Purpose The aim of orthopedic trauma surgery is to restore the anatomy and function of displaced bone fragments to support osteosynthesis. For complex cases, including pelvic bone and multi-fragment femoral neck and distal radius fractures, preoperative planning with a CT scan is indicated. The planning consists of (1) fracture reduction—determining the locations and anatomical sites of origin of the fractured bone fragments and (2) fracture fixation—selecting and placing fixation screws and plates. The current bone fragment manipulation, hardware selection, and positioning processes based on 2D slices and a computer mouse are time-consuming and require a technician. Electronic supplementary material The online version of this article (doi:10.1007/s11548-015-1162-9) contains supplementary material, which is available to authorized users. Shorter and restricted parts of this paper were presented at the 10th IEEE International Symposium on Biomedical Imaging, Apr 7–12, San Francisco, USA, 2013, at the 15th International Conference on Computer Aided Orthopaedic Surgery, June 2014, Milano Italy and at the 10th Annual Asian Conference on Computer Aided Surgery, June 2014, Fukuoka, Japan. I. Kovler · L. Joskowicz (B) · A. Kronman School of Computer Science and Engineering, The Hebrew University of Jerusalem, Givat Ram Campus, 91904 Jerusalem, Israel e-mail: [email protected] L. Joskowicz The Edmond and Lily Safra Center for Brain Research (ELSC), The Hebrew University of Jerusalem, Jerusalem, Israel Y. A. Weil · A. Khoury · R. Mosheiff · M. Liebergall · J. Salavarrieta Department of Orthopaedic Surgery, Hadassah University Hospital, Ein-Karem, Jerusalem, Israel J. Salavarrieta Department of Orthopaedic and Trauma, University Hospital, Fundación Santa Fe de Bogotá, Bogotá, Colombia

Methods We present a novel 3D haptic-based system for patient-specific preoperative planning of orthopedic fracture surgery based on CT scans. The system provides the surgeon with an interactive, intuitive, and comprehensive, planning tool that supports fracture reduction and fixation. Its unique features include: (1) two-hand haptic manipulation of 3D bone fragments and fixation hardware models; (2) 3D stereoscopic visualization and multiple viewing modes; (3) ligaments and pivot motion constraints to facilitate fracture reduction; (4) semiautomatic and automatic fracture reduction modes; and (5) interactive custom fixation plate creation to fit the bone morphology. Results We evaluate our system with two experimental studies: (1) accuracy and repeatability of manual fracture reduction and (2) accuracy of our automatic virtual bone fracture reduction method. The surgeons achieved a mean accuracy of less than 1 mm for the manual reduction and 1.8 mm (std = 1.1 mm) for the automatic reduction. Conclusion 3D haptic-based patient-specific preoperative planning of orthopedic fracture surgery from CT scans is useful and accurate and may have significant advantages for evaluating and planning complex fractures surgery. Keywords Preoperative orthopedic bone fracture surgery · Haptic manipulation · Fracture reduction · Fracture fixation Introduction The treatment for bone fractures is a routine and common procedure in orthopedic trauma surgery. Estimations indicate that there are about 7 million fractures per year in the USA alone [1]. Fractures can be intra-/extra-articular, on load-bearing bones, and range from a simple, non-displaced fissure of an individual bone to complex, multi-fragment, comminuted, dislocated fractures in several bones [2,3].

123

Int J CARS

The aim of orthopedic trauma surgery is to restore the anatomy of the bone fragments and their function to support the bone osteosynthesis process. The two main steps of the surgery are as follows: (1) fracture reduction to bring the bone fragments to their anatomical sites of origin and (2) fracture fixation to maintain the bone fragments in place with fixation hardware including screws, nails, and plates. The current standard of care starts with the acquisition of X-ray images and the evaluation of the fractures. Next, the surgery is planned to determine the surgical approach, the bone fracture reduction, and the type, number, and locations of the fixation hardware. Surgeons can plan the fracture reduction and fixation with commercial software packages, e.g., [4] based on 2D digital overlay templates of the fixation hardware on the X-ray images. In more complex cases, the planning is performed on CT scans 3D bone fragment models [5,6]. During surgery, the surgeon reproduces the preoperative plan based on new fluoroscopic X-ray images and optionally with a navigation system [5,6]. For simple fractures, this process yields adequate results in most cases. However, a higher incidence of complications is reported for complex fractures, including pelvic bone fractures and multi-fragment femoral neck/distal radius fractures [7,8]. Common complications include bone and/or fixation hardware misplacement and inaccurate fracture reduction due to bones fragment misalignment. Complications result in reduced functionality, higher risk of recurrent fractures, fracture reduction failure, and fracture osteosynthesis failure. Revision surgery is required in 10–15 % of all cases [7,8]. The bone fragment locations and the selection and location of fixation hardware have a great impact on the restoration of bone functionality and on the bone fusion process, especially in load-bearing bones, e.g., the pelvis, the femur, and the tibia. Significant variability in the treatment outcome has been reported [2]. In such cases, advanced preoperative planning of the surgery based on a CT scan is indicated. Planning consists of determining the configuration of 3D bone model fragments derived from the CT scan (virtual reduction) and selecting and positioning the fixation screws and plates (virtual fixation). Interactive virtual reduction and fixation with 3D models on a screen with a standard mouse are unintuitive, time-consuming, and error-prone, as there is no direct correspondence between the 2D motions of the mouse and the 3D motions of the models on the screen, especially when coupled rotations are involved. Often times, this planning requires a technician, so it is seldom performed in clinical practice. Previous work Preoperative planning systems based on 3D bone and implant models from CT scans are reported in the literature. They include planning systems for total hip replacement [10], for

123

craniomaxillofacial surgery [11], and for hip fracture surgery [12]. Several haptic-based systems have been recently developed [13–16]. Their aim is to facilitate the manipulation of 3D objects in a virtual environment with realistic tactile sensation. Olsson et al. [11] describe a system for craniomaxillofacial surgery that is similar to ours. It features stereoscopic vision and a single haptic manipulator; it handles multiple bone fragments and is designed for maxillofacial surgery. It does not include fixation hardware creation and placement and was tested by a single surgeon on a single case. Pettersson et al. [12] describe a system for hip fracture surgery simulation that includes visual and haptic feedback, simulated fluoroscopic images, and simulated bone drilling. Harders et al. [13] describe a haptic system for proximal humerus fractures reduction that features stereoscopic rendering. In this system, force rendering is disabled when the bone fragments touch to avoid simulation instability caused by collisions handling. Fürnstahl et al. [14] describe a planning system for proximal humerus fractures. For fractures with displaced fragments, the healthy contralateral humerus is used as a template to guide the reduction process. Fornaro et al. [15,16] present a haptic system for pelvic and acetabular fractures surgery planning. The main differences with our system are as follows: (1) it does not support two-hand manipulation; (2) it does not model ligaments; and (3) its custom plate method creation, based on deformations, is different than ours. Virtual bone fracture reduction is the most time-consuming and challenging aspect of the preoperative planning process. It requires the alignment of bone fragments, possibly with multiple, irregular fracture surfaces that may not fully match due to comminution. A variety of manual, semiautomatic, and automatic virtual fracture reduction methods have been developed. We briefly survey them next. Manual methods provide the surgeon with full control over the placement of bone fragments and fixation hardware. Joskowicz et al. [17] describe FRACAS, the first computerintegrated orthopedic system for closed femoral medullary nailing fixation. Semiautomatic and automatic methods perform virtual fracture reduction by multi-object rigid registration. Semiautomatic methods require the user to identify and pair points on the fracture surfaces to be reduced, e.g., Willis et al. [18]. This can be error-prone and time-consuming, especially when there are no distinguishable anatomical landmarks on the fracture surface. The fragments are then registered by pairwise or simultaneous multi-body rigid alignment optimization with the iterative closest point (ICP) method, e.g., Zhou et al. [19,20]. Thomas et al [21,22] describe a method for virtually reconstructing highly comminuted tibial plafond and articular fractures. The papers describe puzzle-solving algorithms and their applications to clinical cases. The methods rely on bone templates and achieve a high precision. Okada

Int J CARS

et al. [23] present three virtual fracture reduction methods for femoral head fractures. Fracture surfaces are manually annotated and individually registered either to the contralateral healthy bone, to the matching fracture surfaces, or to the fracture surfaces with the contralateral bone surfaces serving as kinematic constraint. The fracture surfaces are then sequentially registered in a predefined order. Papaioannou et al. [24] incorporate curve and surface matching to estimate and minimize the matching error between fragment surfaces. For multi-fragment fractures, all pairwise combinations of the fragment surfaces are matched and aligned. Face boundary curve matching and surface intersection tests are used to reduce the search space. The best reduction corresponds to the set of fragment combination with the smallest cumulative matching error. Their technique requires the fracture surfaces to be nearly planar, which is not applicable to bone fragments with pronounced curvatures, such as femoral and pelvic bones. Automatic virtual bone fracture reduction methods rely on anatomical information for the reduction. Mogaheri et al. [25] use a statistical atlas of the healthy bone to reduce the fracture by registering the bone fragments to it. This requires the construction the atlas, which is not always available and may be inadequate for far-from-average cases. Winkelbach et al. [26] use RANSAC-based optimization to find the bone fragments rigid transformations that maximize surface contact and that minimize the penetration between the bone fragments. This criterion may be inadequate for fractures such as long bones, in which a large contact area is not necessarily the fracture surface area. Winkelbach et al. [27,28] use a Houghlike transform to align cylinder-like shapes. Their method is not applicable to pelvic and vertebras fractures, and to long bones fractures that are too proximal or too distal from the joints. Chowdhury et al. [29] propose a method for virtual craniofacial reconstruction of two-fragment fractures based on graph theory, geometric constraints, and ICP registration. None of these methods has been quantitatively validated on clinical fractures. Materials and methods We have developed a new method for accurate, comprehensive, and interactive preoperative planning of orthopedic fracture surgeries from CT scans. Its main components are as follows: (1) 3D bone fragments selection, segmentation, and model creation from a patient CT scan without manual contours delineation; (2) haptic-based system for 3D virtual fracture reduction and fixation planning; and (3) comparative FE analysis of alternative fixation strategies. In this paper, we describe the second component of our method: virtual fracture reduction and fixation—the first and third components are described elsewhere [9,30]. The unique features of our system and methods include: (1) two-hand

haptic manipulation of 3D bone fragments and fixation hardware models; (2) 3D stereo visualization and multiple viewing modes; (3) ligaments and virtual pivot motion constraints to facilitate fracture reduction; (4) semiautomatic and automatic fracture reduction modes; and (5) interactive custom fixation plate creation to fit the bone morphology. Planning workflow The virtual fracture reduction and fixation with our system proceed as follows. After importing the relevant 3D bone fragment models, the surgeon explores the fracture from various viewpoints, identifies its nature and severity, and determines its type based on established classification schemes [2,3]. Next, the surgeon manipulates the bone fragments to align them to their presumed native locations and to restore their functional roles. The fracture reduction is performed with one or more of three modes: (1) manual, (2) semiautomatic, and (3) automatic. The virtual fracture reduction process allows the surgeon to understand the order and the motions by which the fracture can be reduced and to recognize the difficulties that may be encountered during surgery. The semiautomatic procedure supports accurate anatomical reduction with less effort and user time. In all cases, interactive fine-tuning repositioning of the bone fragments is available to the surgeon at all times to achieve the desired results. Once the fracture has been virtually reduced, the surgeon plans the fixation with screws, nails, and/or plates. The fixation plan consists of the screws, their lengths, diameters, locations, and number and of anatomically shaped custom plates. The surgeon can produce more than one fixation plan for further analysis and comparison with advanced methods [9]. The preoperative plan is then exported for use in the surgery itself. Planning system components Figure 1 shows the block diagram of the system. The inputs are object models created from the CT scan as described in [30] (Fig. 2). The system has six user interaction devices and seven software modules. The devices include a computer screen, a pair of glasses for stereoscopic viewing, a keyboard, a computer mouse, and two PHANTOM Omni (Sensable Technologies Inc., USA) haptic devices (Fig. 3). The haptic device allows the user to touch and manipulate virtual objects by means of a hand-held instrumented 6 degree of freedom stylus mounted on a fixed basis. The stylus applies translational force feedback of 0.8–3.3 Newtons and has a spatial resolution of 0.05 mm. 1. Objects Manager: This module is the database manager. It manages five types of objects: (1) bone and bone fragments: triangles mesh surfaces, their centers

123

Int J CARS

Fig. 1 Block diagram of the haptic planning system

of mass, their locations at all times, and the CT voxels and their intensity values; (2) screws modeled as cylinders defined by their radius, length, axis origin, and axis orientation; (3) fixation plates modeled as triangles surface meshes; (4) ligaments modeled as one-dimensional springs defined by their start/end points; and (5) virtual pivot points modeled as spheres defined by their origin and radius. The module manages both individual objects and groups of objects. 2. Physics Engine: This module performs real-time dynamic rigid and flexible body simulation. It simulates physical behavior and interaction between virtual objects, prevents objects interpenetration, and provides the data for a realistic tactile experience of object grasping and objects contacts during the planning process. It is based on the Bullet Physics Library open source physics engine [31]. It supports hierarchical axis-aligned bounding box collision detection and hard contact physics with the interpenetration penalty function method. 3. Haptic Rendering: This module controls the manipulation of grasped objects and produces the forces that are acted upon by the haptic device for tactile perception. It ensures the stable interaction of the user with the virtual environment and generates the coupling forces and torques for the Physics Engine and for the two haptic devices. interaction between the haptic device and an object is modeled with a proxy [32]. A proxy is a point that follows the position of the haptic device. When moving in free space, the locations of the proxy and of the stylus pointer are identical. When in contact with an object, the

123

Fig. 2 3D model of a pelvic fracture: simultaneous lateral and contralateral views (upper right window). The object orientation trackball is semi-occluded sphere on the upper left corner of the scene

Fig. 3 Two-hands haptic setup and stereoscopic viewing

proxy remains outside the object, but the stylus pointer is slightly inside the object and exerts a reaction force akin to a virtual spring damper coupling. To facilitate grasping, a gravity well is applied to the stylus pointer by simulating an attraction force whose strength is proportional to the stylus proximity to the object surface. The interaction between the screws and the plate is a nonpenetration, hard contact condition. The implementation uses The Haptic Library and Haptic Device Application

Int J CARS

Fig. 4 Fracture fixation: custom fixation plate creation (left) and fracture fixation plate and screws placement in translucid viewing mode (right)

Fig. 5 Viewing of pelvic ring fracture before (top) and after virtual reduction (bottom) and their corresponding simulated X-ray views without (center) and with contours delineation (right)

Program Interfaces [33]. The haptic frame rate is real time for all the cases shown in this paper. 4. Graphics Rendering: This module generates the view of the virtual environment. It includes camera manipulation and four object visualization modes. To facilitate the fracture reduction, it shows the posterior/contralateral view of the scene in the upper right corner of the screen (Fig. 2). The module allows the surgeon to interactively move the camera viewpoint with a virtual trackball. The trackball also provides an intuitive user interface for the interactive rotation of models in space (Fig. 2). The four viewing modes are as follows: monoscopic solid, monoscopic translucid, stereoscopic, and X-ray view. The monoscopic solid mode is best suited for general scene viewing and for fracture exploration (Fig. 2). The monoscopic translucid mode is a see through of the bones that is best suited for screws insertion and positioning inside the bone (Fig. 4). The stereoscopic mode

provides depth perception for accurate positioning during fracture reduction. The X-ray mode shows simulated X-ray images—also called Digitally Reconstructed Radiographs or DRRs—of the bone fragments and their contours as they may appear in the X-ray fluoroscope (Fig. 5). DRRs are computed by standard volume ray casting techniques. The Graphics Rendering module implementation is based on the GLUT and OpenGL libraries [34]. 5. Object Creation: This module supports the grouping/ ungrouping of bone fragments, and the creation of fixation hardware, of ligaments, and of virtual pivot points. Ligaments serve as anatomic motion constraints that restrict the bone fragments motions during reduction and hold them together (Fig. 6). They facilitate the interactive virtual fracture reduction and provide the user with a realistic manipulation experience. Ligaments are modeled as 1D springs with a start and an end point; their number and locations are determined from anatomical atlases.

123

Int J CARS

Fig. 6 3D model showing a pelvic fracture with ligaments (red arrows)

Fixation plates hold the bone fragments together (Fig. 4). To create a custom plate whose shape conforms to the bone surface morphology, the surgeon touches the bone fracture surface with the virtual stylus pointer and starts drawing the plate. A plate surface mesh in contact with bone fragment surface is automatically generated following the pointer trajectory. The plate can span one or more bone fragment surfaces. The surgeon can then adjust the width and thickness of the plate as required. The surgeon creates screws by indicating its start and end points with the virtual tip of the manipulator and then interactively positioning it in their desired location (Fig. 4). To introduce the screws, the outside bone fragment surfaces resistance is automatically disabled. To maintain the screws inside the bone, the surgeon can use the translucid view mode and/or turns on/off the internal resistance of the bone fragment surfaces to prevent the screws from protruding outside the bone. 6. Fracture Reduction: this module computes the transformation between two bone fragments and applies it to one of them to restore the fracture. It also supports the interactive annotation of fracture surface points for the semiautomatic reduction. Manual fracture reduction The surgeon grasps a bone fragment or a bone fragments group with one of the haptic devices and moves it until the fracture is reduced. They key difficulty is to simultaneously achieve the alignment of the contact surfaces. The alignment of two surfaces frequently causes the misalignment of two other surfaces. We provide the surgeon with the option of temporarily reducing the bone fragments freedom by means of virtual pivot points. A virtual pivot point temporarily constraints the translation of the bone fragment (or group) to the center of the pivot, shown as a sphere (Fig. 7). Placing one virtual pivot point

123

Fig. 7 Virtual pivot point (pink) temporarily constraints the relative translation of the bone fragment

at the intersection of two bone fragment surfaces creates a bond so that one object/group can pivot with respect to the other. The virtual pivot point constraints the manipulation of the object/group and facilitates the alignment of the bone fragment surfaces in another location. Placing two virtual pivot points constrains the object/group motions to a single rotational axis, emulating a door hinge. The virtual pivot points can be added and removed at any time as necessary. Semiautomatic fracture reduction When the bone fracture surface interface area is large, the precise alignment of the bone fracture surfaces requires many small and delicate manipulations. We automate this fine-tuning alignment by providing the surgeon with the ability to interactively mark the two bone fragments surfaces that are to be matched. The surgeon starts by interactively selecting the points on the surface of each bone fragment with the virtual tip of the haptic manipulator (Fig. 8). The surgeon then brings two bone fragments into coarse alignment and instructs the computer to perform the fine alignment by ICP rigid registration method with outliers’ points’ pairs removal. 7. Control: This module handles the user commands and determines the actions of the system based on a state machine automaton. Commands and events generated by each of the modules are classified according to the input device that generates them. The mouse controls the camera viewpoint and allows the selection of options from menus in the modules. The keyboard provides direct access to the modules’ options, including the selection of the object types to be created, view mode transitions, motion scaling, and enabling/disabling of haptic rendering. The haptic device enables the user to manipulate virtual objects and to select fracture surface points for semiautomatic reduction.

Int J CARS Fig. 8 Semiautomatic fracture reduction: fracture surface markings (dark points) annotated with virtual tip (blue cone) pointer (left); bone fragments in coarse alignment before the reduction (center); bone fragments after the reduction (right)

Fig. 9 Automatic virtual bone fracture reduction: a inputs: femur fragments models; b femur fracture surfaces intensity profile—red/green colors indicate high/low intensities; c surface maximum curvature—

red/green colors indicate high/low curvature; d contact surface identification; e contact surfaces after ICP registration; f final fracture reduction

Automatic virtual bone fracture reduction

rigid transformations, T1 and T2 , M1 = T1 (F1 ) and M2 = T2 (F2 ). The goal is to find a rigid transformation Treduction = T1 ◦ (T2 )−1 such that bone fragments F2 , F1 are aligned: M3 = Treduction (M2 ) and M3 = T1 (F2 ). The automatic virtual bone fracture reduction algorithm proceeds in three steps: (1) automatic identification of the fracture surface contact points in each bone fragment and their voxels in the CT scan; (2) computation of Treduction to align the bone fracture surfaces by ICP registration; and (3) application of Treduction to bone fragment M2 to align it with M1 .

We have developed a new automatic algorithm for pairwise bone fragment fracture reduction that does not rely on prior shape models or atlases. Our method consists of first automatically identifying the bone fragment contact fracture surfaces and then aligning them by robust ICP registration with outliers’ removal to maximize the contact area between them [19,21–23]. The inputs are the triangle mesh surface models of the two bone fragments and the original CT scan. The resulting transformations are applied to the bone fragments to yield the positions and orientations of the reduced fracture (Fig. 9). The reduction in multi-fragment fractures proceeds by pairwise reduction. We first define the bone fracture reduction problem for two fragments. Let B = {bi } be a healthy bone model defined by mesh points bi and let F1 = { f 1i } and F2 = { f 2i } be two bone fragments models defined by mesh points f 1i and f 2i . The bone fragments are disjoint, F1 ∩ F2 = 0, and their union is the original bone, F1 ∪ F2 = B. A bone fracture is a displacement of the bone fragments defined by two unknown

1. Bone fracture contact surfaces identification: The fracture contact surfaces of bone fragments M1 and M2 are defined as CS12 = { f 1i ∈ F1 | f 1i is a neighbor of F2 } and CS21 = { f 2i ∈ F2 | f 2i is a neighbor of F1 }, where the neighborhood relation defined on the intensity profile of the CT scan and on the curvature of the fragments surface. The bone cortex voxels are identified by intensity thresholding since the cortex density is much higher than that of

123

Int J CARS

the other bone structures. In a healthy bone, the spongy bone and the medullary cavity reside in the interior of the bone; in fractured bones, these tissues are part of the contact surface. Thus, low intensity voxels on the outer surface of the fragments models are considered to be part of the fracture contact surface. Let I = {vi } be the CT scan consisting of voxels vi , each with intensity Ii . The intensity-based contact surface estimations are as follows:

Experimental studies

ICS12 ={vi ∈ M1 |Ii ≤ t1 } and ICS21 ={vi ∈ M2 |Ii ≤ t1 }

(1)

where t1 is a predefined constant. We require the contact surfaces to form a single connected component. Outliers are removed by computing the connected components of ICS12 and ICS21 and retaining the largest one for each. We also use curvature for contact surface identification, as it adds robustness to the estimated contact surface and accounts for CT scans with atypical intensity values, such as low-dose CTs. In most cases, the surface of a healthy bone is smooth, whereas the fracture has sharp edges. Therefore, the contact surfaces are identified by their high maximum principal curvature. Let mesh(M1 ), mesh(M2 ) be the triangle meshes of M1 , M2 . Let ki be the maximum discrete principal curvature of vertex u i in each mesh. The curvature-based estimations are as follows: CCS12 = {u i ∈ mesh (M1 ) |ki ≤ t2 } and CCS21 = {u i ∈ mesh (M2 ) |ki ≤ t2 }

(2)

where t2 is a constant computed from the mesh nodes curvature histogram. Outliers are removed by computing the connected components of CCS12 , CCS21 and retaining the largest one for each. The estimated contact surfaces CS12 , CS21 are the unions of the corresponding intensity and curvature estimations of the contact surfaces. 2. Bone fracture surfaces registration: We register the estimated bone fracture surfaces CS12 and CS21 by coarse principal components analysis (PCA) registration followed by fine ICP registration. Let N1 and N2 be the normalized eigenvectors of the covariance matrices of CS12 and CS21 sorted in descending order. For the coarse registration, we compute a rotation matrix Rcoarse such that N1 = Rcoarse ∗ N2 and apply it to CS21 to obtain a rotated cloud of points. For the fine registration, we apply robust ICP registration with outliers’ removal on the estimated contact surfaces. ICP converges to the local distance minimum, which is the global minimum for initial poses with relatively small rotations—this is the

123

case after the coarse PCA-based transformation has been applied. Once Treduction is computed, we apply it to bone fragment M2 to obtain the reduced bone pose M3 . Note that when the fractured surfaces are not conformal, the results of the semiautomatic and the automatic reduction might be poor. In this case, the surgeon can manually correct the bone fragments positions as required.

We conducted two experimental studies to evaluate our system: (1) manual bone fracture reduction accuracy and (2) automatic virtual bone fracture reduction accuracy. Study 1: accuracy of manual bone fracture reduction This study quantifies the accuracy of the manual anatomical bone fracture reduction using our system. First, we generated the ground-truth reference of the bone fracture reduction by simulating fractures on healthy bone models. We retrospectively selected four CT scans of patients whose pelvic bone was intact and virtually created on the 3D model of the pelvis six realistic perfect-fit two-fragment fractures with no comminution. The CT scans consisted of 333 × 191 × 170 voxels with voxel size 0.97 × 0.97 × 1.5 mm3 . The 3D models had on average 209,000 vertices and 418,000 faces. The position of the resulting bone fragments is the ground-truth final position of the fracture reduction. Note that we created simulated fractures on healthy bones instead of using real fractures so as to have a gold standard for comparison and quantitative accuracy evaluation. For each model, we displace one of the bone fragments and ask five surgeons that were blinded to the virtual fracture creation to manually reduce the fracture with our system. We then compared the bone fragment positions to the ground truth. The anatomical alignment error is the Hausdorff distance between each of the bone fragments with respect to their ground-truth configuration. Five surgeons from the Department of Orthopaedic Surgery, Hadassah Medical Center participated in the study. They were requested to reduce the fractures of six scenarios from the four cases. The user interaction times varied between 10–30 mins depending on the surgeon’s familiarity with the system. The times are approximate, as we assisted the surgeons during the experiment and provided explanations. The surgeons expressed satisfaction with the system in an informal qualitative usability study. Note that manipulating the fragments with a standard 2D mouse and keyboard, which is what is currently done when 3D models are available, is surely is much more tedious and time-consuming than planning with our haptic system.

Int J CARS Table 1 Reduction results tabulated by surgeon (columns) and fractures cases (rows) Surgeons Mean RMS Max

S1

S2

S3

S4

S5

All surgeons

0.28

0.59

0.32

0.38

0.73

0.46

0.32

0.71

0.37

0.45

0.89

0.55

0.74

1.42

0.81

0.93

1.47

0.20

0.35

0.26

0.24

0.26

0.24

0.41

0.31

0.28

0.31

0.53

1.03

0.65

0.60

0.70

0.55

0.61

0.58

0.67

0.75

0.71

1.73

2.30

2.02

0.48

0.67

0.58

0.60

0.81

0.71

1.68

1.93

1.81

Fractures 1

2

3a

3b

4a

4b

All datasets

1.07

0.61

0.63

0.62

0.77

0.79

0.78

2.90

2.80

2.85

0.83

0.12

0.48

0.95

0.15

0.55

1.90

0.35

0.49

0.50

0.29

0.31

0.73

1.13 0.46

0.59

0.60

0.34

0.37

0.89

0.56

1.58

1.64

0.73

0.77

1.47

1.31

Fractures 3 and 4 had two configurations denoted by the letters a and b. Each entry shows the mean, root means square (RMS), and maximum Hausdorff distance between the fracture surfaces. Surgeons S3, S4, and S5 did not complete all cases due to lack of time—thus the empty entries

Table 1 shows the results. All surgeons achieved a mean and RMS surface match error of 1 mm or less. For some cases and for some surgeons, the maximum surface error in specific areas of the fracture surface exceeded 2 mm. The variability between surgeons is also relatively low, although in some cases the maximum error distance is also above 2.9 mm. Overall, the mean, RMS, and mean maximum errors were considered by all surgeons to be very accurate and clinically adequate in all cases. According to Matta [35], a fracture displacement of 1 mm or less is considered an excellent anatomic reduction, 2–3 mm a satisfactory reduction, and 3 mm or more, an unsatisfactory reduction. Study 2: accuracy of automatic virtual bone fracture reduction This study quantifies the accuracy of our virtual bone fracture reduction algorithm. To this end, we create a virtual fracture

on a healthy bone model by “cutting” into two bone fragments, simulate the fracture displacement by offsetting one of fragment with a known rigid transformation, and reducing the resulting fracture scenario with our method. We then compare the configuration of the resulting reduced fracture to the original healthy bone. As in Study 1, we simulated fractures on healthy bones instead of using real ones to obtain an accurate and objective ground truth to quantify the quality of the results. To produce realistic simulated fractures, we developed a new method for fracture simulation (Fig. 10). The method consists of obtaining a realistic fracture surface by segmenting a bone fracture surface in a CT scan of a fractured bone and then using it as a cutting surface template on a healthy bone model, thus replicating a real fracture. The experiments were conducted on four clinical CT scans of femoral bones. Each scan has 156 × 148 × 474 voxels of size 0.7 × 0.7 × 1.0 mm3 . Segmented models of the bones were obtained as described in [28]. For each femur model, three different bone fractures were simulated: a fracture of the femoral neck, a proximal fracture of the femoral shaft, and a distal fracture of the femoral shaft. All fractures were created with actual fractures templates. Following the cutting of the healthy bone, we apply a random rigid transformation to the distal bone fragment with rotations of up to 20◦ in each axis around the fracture contact surface center of mass, and translations of up to 30mm in each axis. The displaced bone fragments were then reduced with our algorithm. We generated 16 different transformations for each of the three types of fractures, that is, 48 scenarios for each bone, for a total of 192 fracture reduction problems for the four healthy bones. In all the experiments, we set the intensity thresholding constant t1 to 100 HU in Eq. (1) and the curvature thresholding constant t2 in Eq. (2) to the top 10% of the mesh curvature histogram. To quantify the results, we computed the final target registration error (TRE f ):  TRE f =

f 2i ∈F2

| f 2i − Treduction (T2 ( f 2i ))|2 N

and compared to the initial target registration error TRE0 :  2 f 2i ∈F2 | f 2i − (T2 ( f 2i ))| TRE0 = N where N is the number of voxels in F2 . This metric measures the root mean square deviation (RMSD) of the fragments points and the transformed fragments points. Table 2 summarizes the results. The mean final TRE is 1.79 mm (std = 1.09 mm). The algorithm running time was 3mins (std = 0.2 mins). The surgeons examined each one of the cases and determined that the reduction was considered clinically acceptable in all cases.

123

Int J CARS

Fig. 10 Six examples of actual pelvic fractures: multisite fractures with comminution Table 2 Results of the automatic virtual bone fracture reduction algorithm (all measures are in millimeters)

Fracture

mm

Surface target registration error

1. Femur neck

Initial TRE range

0–15

15–30

30–45

45–60

0–60

(mean)

(9.0)

(16.9)

(33.8)

(51.8)

(27.9)

Final TRE

1.65

1.59

2.05

2.49

1.94

(std)

(0.67)

(0.39)

(1.17)

(1.2)

(0.97)

Initial TRE range

0–15

15–30

30–45

45–60

0–60

(mean)

(9.2)

(18.9)

(35.3)

(52.0)

(27.0)

Final TRE

1.67

1.54

1.94

2.14

1.82

(std)

(0.8)

(0.79)

(1.1)

(1.3)

(0.98)

Initial TRE range

0–15

15–30

30–45

45–60

0–60

2. Proximal femoral shaft

The left column indicates the fracture location. Even rows show the initial range of the surface TRE (mean) in mm; odd row show the final mean TRE (std) in mm after virtual bone fracture reduction. Columns 3–7 show the initial TRE range (mean) and final mean TRE (std) results in the ranges 0–15, 15–30, 30–45, 45–60, and in the entire range 1–60mm

3. Distal femoral shaft

All fractures

(mean)

(10.7)

(19.0)

(36.0)

(51.7)

(29.35)

Final TRE

1.37

1.04

2.29

1.72

1.61 (1.3)

(std)

(1.24)

(0.68)

(1.77)

(1.2)

Initial TRE range

0–15

15–30

30–45

45–60

0–60

(mean)

(9.58)

(18.2)

(35.0)

(51.7)

(28.1)

Final TRE

1.57

1.4

2.1

2.11

1.79

(std)

(0.92)

(0.64)

(1.33)

(1.24)

(1.09)

The results of our automatic virtual fracture reduction method are comparable to previously reported results in [23], without any of their assumptions. Note that a direct comparison is not possible, as the models and the prototypes are not accessible to us. Our method is applicable to bone fractures consisting of two fragments. It can also be applied to fractures with multiple fragments that can be reduced by pairwise alignment, which are the vast majority of common fractures. Comminuted fractures and fractures in which more than two bone fragments share a common contact surface require extending our method to group-wise registration [21,22], which are currently researching.

123

Conclusion We have presented a novel 3D haptic-based system for patient-specific preoperative planning of orthopedic fracture surgery based on CT scans. The system provides the surgeon with an interactive, intuitive, and comprehensive, planning tool that supports fracture reduction and fixation. Its unique features include: (1) two-hand haptic manipulation of 3D bone fragments and fixation hardware models; (2) 3D stereoscopic visualization and multiple viewing modes; (3) ligaments and pivot motion constraints to facilitate fracture reduction; (4) manual, semiautomatic, and automatic fracture

Int J CARS

reduction modes; and (5) interactive custom fixation plate creation to fit the bone morphology. Our system is multifunctional, incorporates the major aspects of planning orthopedic surgery, and is build of off-the-shelf components of relative low cost. The resulting plan can be further verified with FE methods [9] and can be imported into the operating room. We are currently conducting a comprehensive evaluation of the system that includes more surgeons and various fractures of other bones. We are also exploring the use of the planning system as a training simulator for resident orthopedic surgeons. Although the system is not designed to provide a realistic intraoperative experience, many of its components can serve this purpose. The extensions we are considering include the addition of retractors and the simulation of skin incisions. Acknowledgments Alexander Kravtsov implemented simulated Xray viewing and X-ray contour delineation and created the images in Fig. 5. Conflict of interest None of the authors has any conflict of interest. The authors have no personal financial or institutional interest in any of the materials, software, or devices described in this article. Protection of human and animal rights No animals or humans were involved in this research.

References 1. National Ambulatory Medical Care Survey and American Academy of Orthopaedic Surgeons. http://www.schwebel.com/practice/ broken-bone-injuries/statistics/ 2. Bucholz R, Heckman J (eds) (2009) Rockwood and Green’s Fractures in Adults, 7th edn. Lippincott Williams and Wilkins, Philadelphia 3. Müller AO/OTA Fracture Classification. AO Foundation. https:// aotrauma.aofoundation.org/Structure/education/ 4. TraumaCad Orthopaedic Digital Templating. Brainlab AG Feldkirchen, Germany. https://www.brainlab.com/surgery-products/ orthopedic-surgery-products 5. Liebergall M, Joskowicz L, Mosheiff R (2009) Computer-aided orthopaedic surgery in skeletal trauma. In: Bucholz R, Heckman J (eds) Rockwood and Green’s fractures in adults, 7th edn. Lippincott Williams & Wilkins, Philadelphia, pp 739–770 6. Weil YA, Joskowicz L, Mosheiff R, Liebergall M (2006) Principles of computer-aided surgery in trauma surgery. In: Stiehl J, Konermann A (eds) Navigation and MIS in orthopedic surgery, Springer, Berlin pp 484–494 7. Kurtinaitis J, Porvaneckas N, Kvederas G, But˙enas T, Uvarovas V (2013) Revision rates after surgical treatment for femoral neck fractures: results of 2-year follow-up. Medicina (Kaunas) 49(3):138– 142 8. Palmer SJ, Parker MJ, Hollingworth W (2000) The cost and implications of reoperation after surgery for fracture of the hip. J Bone Joint Surg Br 82–B:864–866 9. Peleg E, Beek M, Joskowicz L, Liebergall M, Mosheiff R, Whyne C (2011) Patient specific quantitative analysis of fracture fixation in the proximal femur implementing principal strain ratios: method and experimental validation. J Biomech 43(14):2684–2688

10. Steinberg EL, Shasha N, Menahem A, Dekel S (2010) Preoperative planning of total hip replacement using the TraumaCad system. Arch Orthop Trauma Surg 130(12):1429–1432 11. Olsson P, Nysjö F, Hirsch JM, Carlbom IB (2013) A haptic-assisted cranio-maxillofacial surgery planning system for restoring skeletal anatomy in complex trauma cases. Int J Comput Assist Radiol Surg 8(6):887–894 12. Pettersson J, Palmerius KL, Knutsson H, Wahlstrom O, Tillander B, Borga M (2008) Simulation of patient specific cervical hip fracture surgery with a volume haptic interface. IEEE Trans Biomed Eng 55(4):1255–1265 13. Harders M, Barlit A, Gerber C, Hodler J, Székely G (2007) An optimized surgical planning environment for complex proximal humerus fractures. In: Proceedings of MICCAI Workshop on Interaction in Medical Image Analysis and Visualization, vol 10, pp 201–206 14. Fürnstahl P, Székely G, Gerber C, Hodler J, Snedeker G, Harders M (2010) Computer assisted reconstruction of complex proximal humerus fractures for preoperative planning. Med Image Anal 16(3):704–720 15. Fornaro J, Harders M, Keel M, Marincek B, Trentz O, Székely G, Frauenfelder T (2008) Interactive visuo-haptic surgical planning tool for pelvic and acetabular fractures. Stud Health Technol Inform 132:123 16. Fornaro J, Keel M, Harders M, Marincek B, Székely G, Frauenfelder T (2010) An interactive surgical planning tool for acetabular fractures: initial results. J Orthop Surg Res 5(1):50–55 17. Joskowicz L, Milgrom C, Simkin A, Tockus L, Yaniv Z (1998) FRACAS: a system for computer-aided image-guided long bone fracture surgery. Comput Aided Surg 3(6):271–288 18. Willis A, Anderson D, Thomas TP, Brown T, Marsh JL (2007) 3D reconstruction of highly fragmented bone fractures. In: Proceedings of SPIE Conference on Medical Imaging, International Society for Optics and Photonics, pp 65121–65126 19. Zhou B, Willis A, Sui Y, Anderson D, Thomas TP, Brown T (2009) Improving inter-fragmentary alignment for virtual 3D reconstruction of highly fragmented bone fractures. In: Proceedings of SPIE Conference Medical Imaging, International Society for Optics and Photonics, pp 725934–725939 20. Zhou B, Willis A, Sui Y, Anderson D, Brown T, Thomas TP (2009) ASB Clinical Biomechanics Award Paper: Virtual 3D bone fracture reconstruction via inter-fragmentary surface alignment. In: Proceedings of 12th IEEE International Conference on Computer Vision, pp 1809–1816 21. Thomas TP, Anderson DD, Willis AR, Liu P, Marsh JL, Brown TD (2011) Virtual pre-operative reconstruction planning for comminuted articular fractures. Clin Biomech 26(2):109–115 22. Thomas TP, Anderson DD, Willis AR, Liu P, Frank MC, Brown TD (2011) A computational/experimental platform for investigating three-dimensional puzzle solving of comminuted articular fractures. Comput Methods Biomech Biomed Engin 14(3):263–270 23. Okada T, Iwasaki Y, Koyama T, Sugano N, Chen YW, Yonenobu K, Sato Y (2009) Computer-assisted preoperative planning for reduction of proximal femoral fracture using 3D CT data. IEEE Trans Biomed Eng 56(3):749–759 24. Papaioannou G, Karabassi EA (2003) On the automatic assemblage of arbitrary broken solid artifacts. Image Vis Comput 21(5):401– 412 25. Moghari MH, Abolmaesumi P (2008) Global registration of multiple bone fragments using statistical atlas models: feasibility experiments. In: Proceedings of 30th IEEE Conference on Engineering in Medicine and Biology, pp 5374–5377 26. Winkelbach S, Wahl F (2008) Pairwise matching of 3D fragments using cluster trees. Int J Comput Vis 78(1):1–13 27. Winkelbach S, Rilk M, Schonfelder C, Wahl F (2004) Fast random sample matching of 3D fragments. In: Rasmussen CE et al (eds)

123

Int J CARS

28.

29.

30.

31.

DAGM 2004, LNCS 3175, Springer, Berlin, Heidelberg, pp 129– 136 Winkelbach S, Westphal R, Goesling T (2003) Pose estimation of cylindrical fragments for semi-automatic bone fracture reduction. In: Michaelis B, Krell G (eds) DAGM 2003, LNCS 2781, Springer, Berlin, Heidelberg, pp 566–573 Chowdhury AS, Bhandarkar SM, Robinson RW, Yu JC (2009) Virtual craniofacial reconstruction using computer vision, graph theory and geometric constraints. Pattern Recognit Lett 30(10):931– 938 Kronman A (2014) METASEG: a new framework for robust medical image segmentation in the clinic. PhD Dissertation, The Hebrew University of Jerusalem Bullet Physics Library. Real-time Physics Simulation. http:// bulletphysics.org/wordpress

123

32. Zilles CB, Salisbury JK (1995) A constraint-based god-object method for haptic display. In: Proceedings of IEEE International Conference on Intelligent Robots and Systems: Human Robot Interaction and Cooperative Robots, vol 3, pp 146–151 33. Haptic Library and Haptic Device Application Program Interfaces, Sensable Inc. http://www.dentsable.com/ openhaptics-toolkit-hdapi.htm 34. GLUT and OpenGL® Utility libraries. https://www.opengl.org/ resources/libraries/ 35. Matta JM (2006) Operative treatment of acetabular fractures through the ilioinguinal approach: a 10-year perspective. J Orthop Trauma 20:S20–S29