Simulation of real-time deformable soft tissues for computer assisted ...

The International Journal of Medical Robotics & Computer Assisted Surgery mrj40607.3d 14/4/04 13:35:07 The Charlesworth Group, Huddersfield +44(0)1484 517077 -

Rev 7.51n/W (Jan 20 2003)

1

ORIGINAL ARTICLE

Simulation of real-time deformable soft tissues for computer assisted surgery A Frisoli, L F Borelli, C Stasi, M Bellini, C Bianchi, E Ruffaldi, G Di Pietro, M. Bergamasco A Frisoli, L F Borelli, C Bianchi, E Ruffaldi, G Di Pietro and M Bergamasco PERCO, Scuola Superiore Sant’Anna, Pisa, Italy C Stasi and M Bellini Department of Internal Medicine, Division of Gastroenterology, University of Pisa, Italy Correspondence to: A Frisoli, E-mail: [email protected]

Abstract The simulation of realistic surgical procedures requires specialized optimized algorithms for the models of organs and tissues, which should comply both with accuracy of results and run-time computation. This paper provides a general survey of methods and approaches used for the simulation of soft tissues in Computer Assisted Surgery, discussing the technological challenges to achieve realistic simulation of deformation. An application example is presented, referring to the simulation of a gastroenterology procedure, the abdominal paracentesis for the treatment of ascites. Keywords: Haptic-based simulation, computer assisted surgery, simulation of soft tissues Paper accepted: 15 January 2004 Published online: 9 April 2004. Available from: www.roboticpublications.com DOI: 10.1581/mrcas.2004.010107

INTRODUCTION

Since the advent of the first surgical simulators, virtual reality technologies have begun to play a more and more determinant role in the medical field. Nowadays the application of simulators cover several branches of medicine, including endoscopic and laparoscopic surgical procedures, arthroscopic interventions, eye surgery or radiological procedures (1–3). There are many reasons for the training of medical procedures in a virtual reality environment. The training of surgeons is typically performed either on live animals or through real surgical interventions under expert supervision. The above two conditions pose several restrictions on the training, due to the high costs and risks associated with them. By employing biometric data derived from CT and MRI scans, most of the relevant features of an interventional procedure can be replicated within a computer simulated environment. The immediate advantages of simulators are the reduction of costs, the possibility of simulating different conditions, including also specific ones rarely occurring in clinical practice.

E2004 Robotic Publications Ltd.

One of the ultimate goals is to achieve surgical planning linked with simulation, with a run-time generation of patient specific models (anatomical variations, pathologies, traumas) (4). Advanced multimodal interfaces, such as haptic interfaces, allow us to add interactivity to the simulation, so that manual procedures can be replicated and also skills involving movement can be taught. Haptic Interfaces are force feedback devices that can simulate the behavior of an operational tool, like a scalpel, by exerting a controlled force on the operator handling it, according to its position and status of the simulation. Haptics specialized surgical instruments allowing multi-hand, multi-instrument, multi-user, collaborative, networked represent one of the challenges in computer assisted surgery. With regards to the use of haptic feedback in medical simulators, there are different strategies that can be combined together for the purpose of training:

N N N

Haptic guidance Haptic restriction to the correct motion Haptic augmentation

Int J Medical Robotics and Computer Assisted Surgery 2004;000(000):1–7

www.roboticpublications.com


2

Rev 7.51n/W (Jan 20 2003)

Frisoli, Borelli, Stasi, et al

Generally two main goals can be achieved during the training of a surgical procedure: teaching a technical skill or teaching a process, such as learning the decision process for complex procedures. According to the users they are directed to and to the skills to be taught, there are different levels of simulators, ranging from simplified simulators up to ultra-realistic ones. For instance for a medical trainee, a highly realistic simulation can complicate the process of the learning of a new procedure, while a simplified one can test their specific abilities and provide a direct measure of their improvement in performance. However the simulation of realistic surgical procedures does not only require knowledge of the biomechanical properties and physical behavior of tissues and organs. In fact in order to set-up a real-time interactive digital environment, models of tissues and organs should be computed and updated in real-time. So the simulation of tissue properties such as deformation, requires specialized optimized algorithms which have to comply both with accuracy of results and computation time. For these reasons research in the field of real-time deformable tissues still remains very open. This paper analyzes some of the more relevant aspects for the simulation of soft tissues. It is organized in two parts. In the first part a general survey of methods and approaches used for the simulation of soft tissues in computer assisted surgery is provided, discussing the challenges and still open issues to achieve realistic simulations. In particular lumped and continuous parameters models are analysed in more detail, highlighting the differences between the two methods and providing some indications for their application. In the second part an application example is presented. The application refers to the simulation of a gastroenterology procedure, the abdominal paracentesis for the treatment of ascites. PHYSICALLY BASED MODELS FOR DEFORMABLE OBJECT SIMULATION Introduction and models characterization Setting up a physically based model for simulation of deformable objects such as human tissues is definitely not a trivial task. It requires a set of mathematical equations capable of adequately describing the mechanical behaviour of tissues, that often present highly non-linear characteristics, such as viscosity, anisotropy and so on, but at the same time not being too complex, in order to make possible their computation in real-time (for haptic

applications this means a refresh rate in the range of 300– 1000 Hz corresponding to 1–3 msec). In general physically based models search a solution to the dynamic equations of the deformable object, that are usually put in the well known form:

where M is the mass matrix of the object, C and K are respectively the associated damping and stiffness matrix, x the vector of the coordinates of the points describing the object and F the vector of external forces applied to it. The expression of terms in (1) depends on the adopted formulation of the mathematical model that can influence both the computational speed and precision of the deformation algorithm. One of first attempts to create a physically based model was made by Terzopolous (5), who devised a method based on a ‘‘deformation energy’’ function depending on the geometrical configuration of the object and representing its elastic behaviour. However the method that he proposed for the implementation of the model (e.g. the central difference method) was not particularly suitable for real-time simulations. Nevertheless in his work it was presented the belief that, to obtain a realistic simulation of deformable objects, it is necessary to use a mathematical model that is subject to physical laws and capable of autonomously responding to the user’s input. The models adopted for the description of deformable objects can be classified in different families according to some basic criteria that reflect the physical assumptions that are made in their mathematical formulation. They can be summarized as follows: 1. Model constitution: Models can be distinguished as lumped and continuous. In lumped models, the simulated object is discretized in particles (also called nodes), where the object mass is considered to be concentrated and linked to each other with connections representing the internal forces arising from the deformation of the object (see Figure 1). In continuous models the set of ruling equations is obtained through the integration carried out over continuous spatial sub-domains (usually called ‘‘elements’’) in which the object is divided under a Finite Element Model (FEM) (see Figure 2). The lumped models produce equations that can be solved faster and simpler than continuous ones that require heavy integral computations. On the other hand, lumped methods are characterized by a lack of precision and





Rev 7.51n/W (Jan 20 2003)

3

Simulation of real-time deformable soft tissues

Figure 1 Example of a superficial model employing a lumped parameters model. The surface is discretized as a grid of points and to each point is associated the elastic action due to its neighborhoods.

stability problems, if they are not correctly implemented (6). 2. Domain representation: Models can be Superficial or Volumetric according to whether they adopt a description of the object surface only or of its entire volume. Superficial methods, obviously, require less calculation than volumetric models, but they tend to give inaccurate results in thin regions, cannot simulate volumetric behaviours of the object (as volume conservation) and cannot simulate operations that change the object topology (3). 3. Characteristic equation: Models can be Static or Dynamic. Static models do not depend on time, and so use only the part Kx = F of the equation ((1)), that describes the elastic behaviour of the object, while dynamic models include time-dependent effects and make use of the whole equation ((1)). Dynamic models allows us to simulate viscous and inertial effects, but require longer calculation times and can present some numerical instability, depending on the method of time integration used to solve the equations. Generally the static simulation is supposed to be acceptable if the object has a high damping factor and if the operator interacts with the model slowly enough (7). 4. Constitutive law: This is the law that expresses the relationship between deformation and stress. It can be linear, such as for metals in the field of little


Figure 2 Example of a volumetric model associated with a FEM subdivision of the volume; each cubic cell represents a volume element of the object exchanging forces with its surroundings.

deformations, or non-linear, such as it is likely to occur in organic tissues. Using a linear relationship, even if it allows the reduction of calculation time, often introduces a non-realistic approximation of biological tissues. 5. Deformation amount: The classical theory of elasticity, which is the basis for most of the simulation methods, stands on the assumption that deformations are small compared with the object dimensions. Unfortunately for human tissues it is not uncommon to notice deformation of 100%, so that this kind of model can be unsuitable. Lumped models

The simplest physically based model is the so called ‘‘Mass-Spring’’ Model. It is a lumped model that can be implemented either as a superficial/volumetric or as a static/dynamic model. The mass of the object is assumed to be concentrated in points connected to each other by springs and/or dampers (see Figure 3). Often, in non-static models, a more complex system of dampers and springs, like the Maxwell model or the Kelvin-Voigt model showed in Figure 4, replaces the simple spring to simulate viscosity or other timedependant characteristics of material (e.g. creep, relaxation effects and so on). Because of its simplicity and its speed of execution, the Mass-Spring-Model has been implemented in many existing medical simulators. On the other hand, the model shows instability when the masses are uncon-




4

Rev 7.51n/W (Jan 20 2003)


Figure 3 A simple representation of a ‘‘Mass-Spring’’ model: nodes Ni, Nj represent the masses in the model, while the branches connecting the nodes represent the elastic Kij and viscous Cij properties of the material.

strained (3) and it is not easy to establish a physical relationship between the mechanical characteristics of springs in the model and the mechanical characteristics of the object to be simulated (6). Another lumped model is the Long Element Method (LEM) (8). In the LEM model, the surface object is divided into smaller areas, and each area is considered to be at the end of a long element, i.e. a kind of long elastic spring with finite volume, filled with an incompressible fluid inside, and connected to adjacent elements by springs. To simulate three-dimensional objects the model is composed of three perpendicular series of LEMs oriented along principal directions. Although initially this method was static, further studies (9) have extended it to include model dynamics. Like the Mass-Spring-Model, the LEM does not reproduce the mechanical characteristics of the objects and needs to be tuned empirically. This model can be more suitable for those objects that can be considered as filled with an incompressible fluid, as this is the case of some human organs like heart or blood vessels.

Figure 4 The Maxwell and Kelvin-Voigt models used for representing relaxation and creep behaviors.

The solution of equations at these nodes, which are of a finite number, can be numerically obtained, and once available, allow us to reconstruct the solution in each sub-domain through an interpolation. For instance in FEM methods the control points coincide with the vertices of polyhedral elements in which the mesh is divided. The computation of stiffness matrix K in (1) is calculated by stating that the work performed by external forces deforming the object is equal to the internally stored elastic energy. This equality can be expressed in an integral form. In a similar way, by using the expression of kinetic energy, it is possible to calculate the inertia matrix M. The damping matrix C has many different formulations, among which the most used is the Rayleigh formulation that can be written as: C = aM+bK

Continuous Models

Continuous mechanics has been used for a long time to solve many different engineering problems. Several different techniques have been proposed for implementing real-time models of deformable objects based on continuous mechanics laws, such as Boundary Elements (10) , Finite Spheres (11) or Finite Element methods. The basic idea common to all these algorithms is that of dividing the object (or simply its boundary in the case of BEM) in to several simpler continuum spatial subdomains, whose description can be completely characterized through a limited set of control points, also called nodes.

(2)

where a and b are coefficients depending on the characteristics of the material. The models based on continuous mechanics are the most accurate and can be used to simulate large deformations, non-linearity anisotropy and so on. As a drawback the real-time execution of their complete formulation can be critical on ordinary computers. Some simplification or reduction techniques are available, that can reduce the complexity of continuous models, in order to render them suitable for haptics simulation. The number of equations to be solved can be drastically reduced by applying ‘‘condensation’’, a





Rev 7.51n/W (Jan 20 2003)

5


Figure 5

The LEM element introduced by

(8)

.

commonly used procedure where a set of nodes, supposed to be ‘‘most significant’’ ones are selected, and the mechanical equations of the remaining nodes are expressed as functions of them (12). AN APPLICATION EXAMPLE: A SIMULATOR OF PARACENTESIS In the following we present an application of the techniques presented in the previous section for the set-up of the simulation of a simple medical interventional procedure. A virtual reality simulation has been developed in order to train the execution of the abdominal paracentesis for the treatment of ascites. The abdominal ascites has been known since ancient times and an imaginative description of a person suffering of ‘‘heavy dropsy’’ is also found in the XXX canto of the Dante Alighieri’s Hell (Circle VIII Bolge X) (13), where Master Adam, one of the falsifiers, is depicted with a prominent abdomen, ‘‘the bloated belly’’, and with the body ‘‘out of proportions’’ with ‘‘the small visage didn’t at all suit’’. Paracentesis is a procedure during which fluid from the abdomen is removed through a needle for a temporary resolution of complications due to a refractory or massive ascites. Paracentesis has been for a long time the preferred treatment for ascites; nevertheless with the usage of diuretic therapy its usage has been reduced, also due to the collateral effects associated with the removal of high quantity of liquid from the abdomen. The onset of ascites in patients affected by refractory cirrhosis shows up in 60% of cases, and on average within 4 years after the clinical diagnosis. The treatment of ascites in cirrhotic patients can be achieved through an appropriate diuretic therapy, aimed at the mobilization of ascitic fluid. In some cases the diuretic therapy cannot be established, either because it is no longer effective or because it would require a removal of large volumes of fluid in a short time, such as in the case of patients


presenting symptoms of tense abdomen, abdominal pain, voluminous hernias or diaframmatic superelevation with respiratory insufficiency. Nowadays the usage of paracentesis is widely applied in all hepatological clinical centers, following this several scientific works have demonstrated its efficacy, associated with the infusion of albumin iv, with respect to the diuretic therapy, and the minor incidence of collateral effects. The puncture is executed through a metal cannula which allows a precise repositioning to allow drainage, while the cannula’s small bore reduces post-procedure leakages from the insertion site. It is usually executed on the left side at the right inferior quadrant, with the patient in a lateral position, in order to avoid injuries to the liver, which is usually of considerable dimensions, within the internal 2/3 of the line joining the navel to the anterior iliac spine. The asepsis should be rigorous to avoid infections, for instance of the ascitic cirrhotic liquid. The ascitic liquid is then collected into several test tubes.

Figure 6 display.

The simulator consisted in a haptic and visual




6

Rev 7.51n/W (Jan 20 2003)


When the cannula is in contact with the body, a force is given back on the stylus and a deformation of the tissue is replicated according to the force exerted by the user through the stylus. When the local deformation of tissue exceeds a given threshold, the penetration of the peritoneal wall begins and is simulated by employing a viscoelastic module to approximate the impedance along the axial direction. The deformation algorithm

Figure 7 program.

An image of the room simulated in the application

The application program

The set-up of the simulator is shown in Figure 6. The application presented here has been devised for education purposes in order to show medical students the execution of the paracentesis operation. The application starts with the visualization of a room (Figure 7), where the body of a patient is presented to the trainee with the abdominal cavity inflated by fluid, lying on their left side, in the same position assumed during the execution of intervention. A haptic interface is used as the input device (14). By manipulating the stylus the trainee can command the motion of the metal cannula, which is simulated in the virtual environment. The contact forces are applied to the endpoint of the stylus, which represents the contact point available for the interaction with the virtual environment.

Figure 8 A polygonal mesh is used for the representation of the body of the patients. The abdominal part is inflated to simulate the presence of the ascitic fluid.

The body of the patient was represented through a polygonal mesh (Figure 8). Due to the nature of the operation, a superficial model was considered suitable for the simulation of the abdominal wall. Moreover the operation was supposed to be executed slowly enough so that the contribution of the inertial and the viscous effects can be negligible (7). According to these assumptions, a static superficial massspring model was adopted. To build a deformable model of the abdomen, all the nodes of the polygonal mesh of the body (Figure 8) were connected to each other with springs. To avoid instability each node was also linked to a ‘‘virtual node’’, put inside the model along the normal direction to the surface, through an additional spring. This provided the model with a sort of ‘‘skeleton’’, capable of supporting it, preventing it from rigid displacements and simulating the effect of the internal pressure, due to the ascitic fluid. By setting the value of the elastic constant of this normal spring, the effect of the fluid pressure can be regulated, while the elastic elements lying on the surface determine the elastic tension of the abdominal wall. The final arrangement of the elastic elements was the same one as shown in Figure 1. During the initialisation phase, the algorithm creates the additional virtual nodes and the elastic connections between the nodes.

Figure 9

An image of the application before deformation.





Rev 7.51n/W (Jan 20 2003)

7


CONCLUSIONS This paper has presented a general survey of the principal methods used for the simulation of deformable tissues in computer aided surgery. An example has shown the application of an algorithm for run-time deformation of surfaces to the simulation of the procedure of abdominal paracentesis. Future work foresees the completion of the application with the visualization of anatomical cues and textual information. REFERENCES Figure 10 An image of the application where the deformation of the abdomen has been exaggerated to make it visible.

After the initialization, the algorithm waits for a collision between the probe handled by the user and the model. When the collision occurs, the surfaces involved in the contact, later on called the ‘‘active surfaces’’, are identified by a proper algorithm, together with the amount of penetration of the probe into the model surface. The deformation algorithm then moves the contact nodes on the model surface according to the amount of penetration of the probe. The imposed displacement of contact nodes generates non-null forces at all the other nodes, because of the perturbation of the elastic equilibrium. Such elastic reaction forces are calculated according to Hooke’s Law:

where xˆij is the versor of the line joining node i and node, kij the stiffness constant of the spring, dij the distance between points, lij0 the rest length of the spring. Then all the nodes are moved according to the direction of resultant nodal forces, and the process is iterated until a new final static equilibrium configuration is reached with null nodal forces. The effect of deformation is shown in Figure 9 and Figure 10. The effect has been exaggerated to make it visible. Forces are generated on the probe according to the reaction forces calculated by the physical model and given back to the user that can adaptively control the amount of deformation according the tactile sensation.


1 Bro-Nielsen M. Finite Element Modeling in Surgery Simulation. Proc. of IEEE, 1998;86:490-503. doi:10.1109/5.662874 2 Di Maio SP, Salcudean SE. Needle Insertion Modeling and Simulation. IEEE Transactions on Robotics & Automation. 2003;19(5) 3 Delingette H. Towards Realistic Soft-Tissue Modeling in Medical Simulation. Proceedings of the 1998 IEEE: Special Issues on Surgery Simulation, 1998;86(3):512–23. doi:10.1109/5.662876 4 Satava RM. Medical Virtual Reality: The Current status of Future. Proc. of Medicine Meets Virtual Reality conference;1996:100–6 5 Terzopoulos D, Fleischer K. Deformable models. The Visual Computer. 1988;4(6):306–31 6 Harders M, Hutter R, Rutz A, Niederer P, Szekely G. Comparing a simplified FEM approach with the Mass-Spring model for surgery simulation. Proc. Of Medicine Meets Virtual Reality. IOS Press;2001:11. 7 Brown J, Montgomery K, Latombe J, Sthephanides M. A Microsurgery Simulation System. Medical Image Computing and Computer-Assisted Intervention (MICCAI 2001). 2001:137–44. 8 Costa I, Balaniuk R. LEM–Aa approach for real time physically based soft tissue simulation. Proceedings of the Fifth PHANToM User Group Workshop; 2000:26–30 9 Balaniuk R, Salisbury K. Dynamic Simulation of Deformable Objects Using the Long Elements Method. Proceedings of 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems; 2002. 10 James D, Pai DK. A Unified Treatment of Elastostatic and Rigid Contact Simulation for Real Time Haptics. Haptics-e, the Electronic Journal of Haptics Research. September 2001;2(1). 11 De S, Hong JW, Bathe KJ. On the method of finite spheres in application: towards the use in ADINA and in a surgical simulator. Computational Mechanics. 2003;31:27–37. doi:10.1007/s00466-002-0390-3 12 Bro-Nielsen M, Cotin S. Real-time volumetric deformable models for surgery simulation using finite elements and condensation. Computer Graphics Forum. 1996;15(3):57–66. doi:10.1111/1467-8659.1530057 13 Zimmerman S. The Inferno of Dante Alighieri. iUniverse; 2000 14 Frisoli A, Checcaci D, Salsedo F, Bergamasco M. Translating inparallel actuated mechanism for haptic feedback. Proc. of ASME (American Society of Mechanical Engineers) IMECE2000, International Mechanical Engineering Congress and Exposition, Orlando (FL-USA); 2000.



Simulation of real-time deformable soft tissues for computer assisted ...

Simulation of real-time deformable soft tissues for computer assisted ...

Suggest Documents

computer assisted si computer assisted simulation of market s ...

Evaluation of the facial soft tissues following surgically assisted

Real-time simulation of biological soft tissues: a PGD approach

Computer Navigation of Soft Tissues in Total Knee ...

Morphogenesis of growing soft tissues

Intuitive Control of Deformable Object Simulation ... - Computer Graphics

Simulation of Deformable Biological Structures

Computer-Assisted Simulation Proofs - Semantic Scholar

Deformable mesh simulation for virtual ... - Semantic Scholar

Virtual Jaw: A 3D Simulation for Computer Assisted Surgery and ...

Deformable mesh simulation for virtual ... - Semantic Scholar

Inversion handling for stable deformable modeling - Computer

Symbolic Signatures for Deformable Shapes - Computer Science

chondrosarcoma of the soft tissues - Semantic Scholar

Computer simulation of the Poisson's ratio of soft polydisperse ... - arXiv

The Importance of Soft Tissues for Structural Support of ... - Biotensegrity

Simulation, Planning, and Execution of Computer-Assisted Surgery

urbax game as a computer assisted simulation of urban ... - CiteSeerX

Simulation, Planning, and Execution of Computer-Assisted Surgery

Real-time elastic deformations of soft tissues for

Deformable Face Fitting with Soft Correspondence Constraints

Real-time elastic deformations of soft tissues for ...

Computer Assisted Pathway Generation for

Requirement Solicitation for Computer Assisted