TR/98/3
Technical Report
Snake Implementation in the UK IUE Project
Julia A. Schnabel and Simon R. Arridge
Email:
[email protected] S.Arridge.cs.ucl.ac.uk URL: http//www.cs.ucl.ac.uk/research/mip/mipue/
Abstract This report describes design issues, specification and implementation details of the concept of active contour models (snakes) in medical image processing as part of the UK IUE Project sponsored by EPSRC. For the DARPA funded Image Understanding Environment (IUE), general package information is provided by Amerinex Applied Imaging at URL http://www.aai.com/AAI/IUE/IUE.html, URL http://s10d.smb.man.ac.uk/IUE/IUE gate.html gives further general project information at the hub site at Manchester, and URL http//www.cs.ucl.ac.uk/research/mip/mipue/ provides local spoke project information, reference guides, and programs. The other two stages of this project are described in [8] and [9].
Keywords Image Understanding Environment (IUE), Medical Image Processing and Understanding Environment (MIPUE), active contour models, snakes, multi-scale medical image analysis.
Department of Computer Science University College London Gower Street London WC1E 6BT, UK
1 Introduction The implementation of active contour models (snakes), first presented by [4] for image segmentation requires the choice of a suitable discrete contour representation, the design of an energy function, and implementation of an optimization method in the IUE. The classic formulation of snakes is given by integrating internal contour characteristics, and external image features and user constraint forces over a parameterized contour v(s), e.g. as Z1
E (v(s)) =
0
Eintern (v(s)) + Eextern (v(s))) ds
(
where Eextern (v(s)) = Eimage (v(s))+ Econstraint(v(s)). Minimizing this energy functional transforms the segmentation problem into an optimization problem, where optimization can be performed locally or globally. A vast diversity of models has been developed by various research groups in recent years, including balloons incorporating an inflation or deflation constraint force [3], geometrically deformed models using subsampling techniques [6], fast greedy snakes with various curvature forces [11], and multi-scale, splinebased active contour models [7]. An extensive overview over current techniques for deformable models in medical imaging is given in [5]. In general, differences in terms of representation, energy computation as well as optimization strategies, impose the need for a very generic snake design in the IUE.
2 Design Issues and Choices 2.1 Snake Representation The IUE offers several possible representation schemes for snakes: 1. Use of an instantiation of the abstract class IUE sequence abs which is based on a Standard Template Library (STL) sequence. This has the advantage that predecessors and successors are defined, and that the elements of the sequence can be of any (homogeneous) type. E.g. a new class IUE snaxel, derived from a IUE point can be defined, which contains an additional energy value of type IUE FLOAT, and which can be of sub-pixel accuracy (if templated to IUE DOUBLE or IUE FLOAT), or discrete (if templated to IUE INT). 2. Use of an instantiation of the abstract class IUE curve. This has the advantage over the previous approach that certain pure virtual curve methods like tangent, normal, binormal, curvature, and torsion are provided. Several suitable subclasses can be used: (a) IUE sampled curve: This is a IUE standard sampled curve 2d is IUE point sample 2d
sequentially ordered set of points. implemented, based on a sequence
A of
(b) IUE spline: Only certain C 1 or C 2 continuous splines are implemented so far as subclasses, while B-splines are not available yet. (c) IUE edge: This represents a curve with a boundary, based on an edge-vertex model. An example is IUE linear edge 2d. It is constructed by two vertices, or by a parametric line segment, or by a sequence of edges (chain). Given that IUE curve provides useful pure virtual methods for the computation of internal curve features, inheriting from this class is the obvious choice, leading to a top-down approach, rather than bottom-up as would be the first option. However, IUE curve provides no direct access to the points lying on the curve, nor do the direct, abstract descendants IUE sampled curve or IUE spline. Only the instantiated classes offer such routines, but use different method names (like knot points() for IUE c1 spline 2d and IUE c2 spline 2d, or samples() for IUE standard sampled curve 2d). Therefore a IUE mixin approach has been implemented which allows to incorporate pure snake specific properties and methods into a socalled mixin class IUE snake mixin which does not inherit from any IUE class, but is a new pure abstract TR/98/3
Page 1
IUE_object IUE_spatial_object IUE_snake_mixin
IUE_curve
IUE_sampled curve
IUE_spline
IUE_sampled_curve_2d IUE_standard_sampled_curve_2d
IUE_snake_standard_sampled_curve_2d
IUE_c1_spline_2d
IUE_c2_spline_2d
IUE_snake_c1_spline_2d
IUE_snake_c2_spline_2d
Figure 1: Overview of IUE curve and snake hierarchy. New snake classes are marked with grey boxes. The instantiable snake classes inherit energy computation and optimization methods from the pure base or mixin class IUE snake mixin, and standard curve methods like constructors, derivatives, curvature and normal computations from their respective parents in the IUE curve hierarchy. base class. This way, a subclass of this class and any instantiable subclass of IUE curve blends pure snake properties and pure curve properties, while avoiding the multiple inheritance of IUE object and associated base constructor and destructor problems. Figure 1 presents an overview over the existing IUE curve hierarchy, and the integration of the new IUE snake hierarchy.
2.2 Snake Energy Function A snake energy function must be based on the chosen snake representation. It can be used to evaluate the snake in a discrete (if based on a sampled representation) or continuous (if based on a spline representation) manner. Both should in principle be possible. It should provide virtual methods for internal, and external image and constraints computations, and return a weighted energy value. The virtual methods can be filled out in a different manner, e.g. by computing the discrete bending, or the analytic curvature. Although it may be desirable to implement an abstract energy function inherited form IUE object, which is instantiated to fulfill different user and application dependent needs, we have identified the common need for internal (elasticity and curvature), external (image and user constraint), and total (sum of internal and external terms) energy computation as a snake property. Therefore we have implemented the internal, external, and total energy terms as pure virtual functions of an abstract snake class, which can be overrun by implementing a new snake descendant. The implemented energy functionalities are the following: 1. Elasticity energy: This first order derivative term is of the form xs (s)2 + ys (s)2 in the analytic case, or kxi ? xi?1 k in the discrete case. Traditionally, the influence of this term is enhanced by taking it to the power of two. 2. Bending/Curvature energy: The second-order bending term is given by xss (s)+ yss (s) in the analytic case, and kxi+1 ? 2xi + xi?1k in the discrete case. In [11] it was pointed out that this is only valid when the curve is arc length parameterized (and uniformly sampled). Otherwise, the curvature needs to be computed by
x (s)yss (s) ? xss (s)yss (s) (s) = s (xs (s)2 + ys (s)2 )3=2
TR/98/3
(1) Page 2
Again, a power of two is usually taken. In [7], an extension to this scheme was made by incorporating an image curvature term to allow for a matching between snake and image curvature. The isophote1 image curvature is given by ? LLvv in a local gauge coordinate system, where w indicates the local w image gradient direction or direction of the normal to the local isophote, and v is perpendicular to w, pointing in the local tangent direction.In Cartesian coordinates:
L2 Lyy ? 2LxLy Lxy + L2y Lxx image = x (L2x + L2y )3=2
(2)
The curvature matching can then be integrated by replacing the standard curvature by its absolute, squared deviation from the underlying image curvature, k(s) image (s)k where the sign depends on the isophote normal direction (depending on the image contrast). Both alternatives (curvature minimization and matching) have been implemented. 3. Image energy: Traditionally, the image energy is given by a combination of local (sometimes thresholded) image intensity (the so-called line energy, given by the image luminance L), an image gradient potential (using the negative squared image gradient magnitude, ?krLk, and a termination term (given by image ) [4]. In our experience, however, the use of a normalized, inverted squared gradient magnitude proved to be the best image potential. Other possible image forces could be given by ridges or creases of the gradient magnitude or the image intensity, but have not been considered here. By inheriting new snake classes, however, the existing image energy can be effectively overridden and redesigned. 4. Constraint energy: The most popular user constraint force is a simple inflation or deflation term, given by (positive or negative) expansion along the snake normal ni [3]. Other possible additions include spring and volcano forces, to steer the snake to or drive it from certain marked points in the image, and a stereo force to intrinsically constrain a snake in a stereo image. Again, such constraints can be computed by inheriting a new snake and overriding the computation of the constraint energy.
2.3 Snake Optimization The optimization process has been designed as a set of virtual methods of the abstract snake class IUE snake mixin. The following local and global methods have been designed, but other techniques can be easily added to the abstract snake class or derived subclasses: 1. optimize classic() : This is the classic approach taken in [4] which formulates the snake within a variational framework using local implicit steps based on pentadiagonal banded matrix inversion for the discrete internal energy computation, and local explicit steps for the external energy terms. 2. optimize euler forward() : Similar to the classic approach, this scheme is based on variational calculus, with the difference that it only takes local explicit steps for all energy terms. 3. optimize greedy() : This techniques performs an efficient local neighbourhood search which is traditionally performed sequentially [11], but can also be done in a quasi-parallel manner. 4. optimize iterated condtional modes() : Originally developed by [2], this ICM technique can be used as a global search. The search space is restricted along the normal at each curve point. The same remarks to sequential/parallel behaviour apply here. 5. optimize dynamic programming() : Not implemented here (but with a method slot provided), this global technique finds the optimal solution within a given search space [1]. 1
An isophote is a contour of constant grey-level
TR/98/3
Page 3
2.4 Design and Implementation 2.4.1 Class Overview The snake hierarchy designed in this project, as stated above, uses a mixin approach by blending in pure snake and pure curve properties. Additionally to the pure base or mixin class IUE snake mixin, the following snake and curve subclasses have been designed: IUE snake standard sampled curve 2d IUE snake mixin
:
public
IUE standard sampled curve 2d,
public
IUE snake c1 spline 2d : public IUE c1 spline 2d, public IUE snake mixin IUE snake c2 spline 2d : public IUE c2 spline 2d, public IUE snake mixin This hierarchy can straightforwardly be extended to B-splines, edge-vertex models or span2 models. The main advantage of such an approach is that default constructors and methods of the instantiated subclasses of IUE curve are automatically provided. 3D snakes, 2D surfaces or 3D volumetric models have not been considered in this concept, but would probably need separate hierarchies.
2.4.2 Class Implementation The abstract class IUE snake mixin contains a set of hard attributes to control the energy computation and optimization of the snake. These attributes are public in order to allow for easy adjustment. The attributes are provided with suitable default values, and new ones can be added to the abstract snake class or - preferably - to derived subclasses. Table 1 lists the implemented attributes for IUE snake mixin. Functionalities common in most snake implementations have been identified, and have been provided as normal methods (which are common for all snake types), virtual methods (which are designed but may also be overrun by subclasses), or pure virtual methods (which need to be implemented by all subclasses, and are dependent on the curve type from which the snake has been derived). Table 2 gives an overview over all methods implemented in IUE snake mixin.
2.5 Application Programming Using the IUE tool iuecreateappl, a task directory for a simple snake application was created. The actual application has then the necessary entry point appl main, and the new shared snake sub-package library libliue-snake.so (which is also included in the local package library libliue-spatial-object.so), has been linked by setting LOCALIUELIBS = -lliue-snake -liue Additionally, small snake utilities for reading, writing, and interpolating snakes have been provided in a separate module in the snake task directory, as no methof in IUE curve is provided for these purposes. The main program reading a snake as IUE snake c2 spline 2d, optimizing using the greedy technique, and writing the resulting snaxels looks like the following: #include #include #include #include
"snake.h"
// // // //
Standard IUE includes Snake includes Fileformat task include Local snake I/O utilities
int appl_main (int argc, char *argv[]) { cout
