3D Active Meshes: fast discrete deformable models for cell tracking in ...

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XXXX XXXX

1

3D Active Meshes: fast discrete deformable models for cell tracking in 3D time-lapse microscopy Alexandre Dufour, Member, IEEE, Roman Thibeaux, Elisabeth Labruy`ere, Nancy Guill´en and Jean-Christophe Olivo-Marin, Senior Member, IEEE

Abstract—Variational deformable models have proven over the past decades a high efficiency for segmentation and tracking in 2D sequences. Yet, their application to 3D time-lapse images has been hampered by discretization issues, heavy computational loads and lack of proper user visualization and interaction, limiting their use for routine analysis of large data-sets. We propose here to address these limitations by reformulating the problem entirely in the discrete domain using 3D active meshes, which express a surface as a discrete triangular mesh, and minimize the energy functional accordingly. By performing computations in the discrete domain, computational costs are drastically reduced, whilst the mesh formalism allows to benefit from real-time 3D rendering and other GPU-based optimizations. Performance evaluations on both simulated and real biological data sets show that this novel framework outperforms current state-of-the-art methods, constituting a light and fast alternative to traditional variational models for segmentation and tracking applications. Index Terms—deformable models, 3D active meshes, cell tracking, biological image sequence

I. I NTRODUCTION ELL segmentation and tracking in multi-dimensional fluorescence microscopy has become a standard need within the biological community, for more and more studies aim at elucidating the links between cell dynamics and phenotype through the quantitative study of cellular motility, deformation and morphological changes [1]–[3]. Together with recent advances in fluorescent probes, the growing popularity of automated imaging systems has lead to an exponential increase in the amount of 3D time-lapse fluorescence microscopy sequences produced per experiment. Dealing with this wealth of data yields three distinct sets of challenges: • Images suffer from a low contrast mechanism and poor depth resolution, yielding delicate cell boundary segmentation. Also, cells that evolve freely in their environment

C

Manuscript received November 19, 2009; revised October 1, 2010; accepted for publication November 20, 2010. Copyright (c) 2010 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to [email protected] A. Dufour and J.-C. Olivo-Marin are with Institut Pasteur, Quantitative Image Analysis Unit, Cell Biology and Infection Department, F-75015 Paris, France (www.bioimageanalysis.org); CNRS, URA2582, F-75015 Paris, France; Institut Carnot “Pasteur Maladies Infectieuses”, F-75015 Paris, France. R. Thibeaux, E. Labruy`ere and N. Guill´en are with Institut Pasteur, Cell Biology of Parasitism Unit, Cell Biology and Infection Department, F-75015 Paris, France; INSERM, U786, F-75015 Paris, France. This project was funded by Institut Pasteur and ANR. Correspondence: {adufour,jcolivo}@pasteur.fr Digital Object Identifier (placeholder)

•

•

are difficult to track, as they may touch and separate over time, or enter and leave the field of view. One must therefore rely on region-based mathematical models to detect cell boundary in a robust manner, and at the same time handle key events such as topological breaks (e.g. cell division) and cell appearance / disappearance [4]. With the increase in imaging data volume, manual or semi-automated segmentation and tracking tools have progressively come to their limit, being that repetitive user intervention is highly time-consuming, lacks reproducibility, and is especially difficult in 3D [5], [6]. Typical experiments can produce several tens of gigabytes of images in multiple colors and dimensions in a just a few hours [7], [8], clearly motivating the development of fully automated algorithms able to process this massive amount of data, while maintaining low computational costs. In addition to quantitative results, qualitative assessment of the observed phenomenon requires adequate 3D visualization tools to produce a realistic rendering of the scene as it evolves. Professional end-user bio-imaging softwares provide various rendering methods that generally benefit from hardware GPU acceleration for fast and fluid rendering. However, such softwares only provide basic image analysis algorithms and are not suited for efficient 3D cell tracking.

Although specific solutions can be found for each of these problems independently, up to now no unified approach has been developed that solves all three problems simultaneously, for they diverge in computational requirements. In this paper, we propose a fast and efficient framework based on deformable models called 3D Active Meshes for multi-cell segmentation and tracking that addresses all of these issues simultaneously. The main contribution of our work is to rewrite the minimization problem of conventional variational approaches directly in the discrete domain, by expressing the deformable surface using discrete mesh models. The discrete formulation yields a number of advantages over standard approaches: a) all computations are performed in the discrete domain, reducing memory and time costs drastically; b) we minimize the same energy functional as traditional approaches, therefore quantitative results are comparable; c) mesh models allow us to exploit numerous optimizations developed in the computer graphics field, such as collision detection, topological flexibility and real-time rendering of the mesh during its deformation. Through simulated and real

pub-id (placeholder)

2

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XXXX XXXX

experiments, we illustrate how this improved framework can impact positively on biological research. The remainder of the paper is organized as follows. We first start in Section II by reviewing recent advances in cell segmentation and tracking methods. Section III then describes the proposed method. Validation on both synthetic and experimental data sets is presented in Section IV. Finally, Section V concludes the paper and discusses several other potential applications of the proposed method in biological imaging. II. R ELATED EFFORTS In this section, we first give a brief overview of recent efforts conducted for cell segmentation and tracking in 3D biological imaging. We then recall the principles of deformable models, and pose the starting point of our work by highlighting the limitations of conventional variational approaches and the benefits of fully-discrete alternatives. A. Segmentation and tracking in biological imaging In previous reviews [2], [3], we browsed through a number of methods developed for segmentation and tracking applied to biological data. Hence here we shall first give an overview of the various families of methods, and then focus more particularly on recent developments in the field of deformable models applied to biological imaging. Depending on how segmentation and tracking methods are paired, they generally follow three different strategies. In the first category of approaches, segmentation and tracking are seen as totally independent processes. Cell segmentation is first performed on each sequence frame, and thereafter a data-association step is used to compute the most probable association between successive detections. Such techniques are mostly employed in particle tracking applications [9], [10]. Although applications to motile cells have been developed [11]–[14], these approaches are less adapted to track the boundary of deforming cells during key events such as cell-cell contact or cell division. Conversely, the second category of approaches consider segmentation and tracking as a unique problem, by merging time and space dimensions and computing the tracks directly from the hyper-dimensional spatio-temporal volume [15]. This strategy has been applied to two-dimensional cell tracking [16]–[18], where curvilinear or tubular structures are extracted directly from the 3D spatio-temporal volume. Recently, a preliminary application of the technique to 3D time-lapse data has been described in [19]. Yet, manipulating 4D hyper-volumes raises both computational and visualization challenges. The last set of approaches lies in between the two previous ones. Here segmentation and tracking are performed in parallel and share information to mutually improve their performance. Typically, segmentation is iteratively performed frame per frame, while temporal information is propagated from previous frames to improve the segmentation of the most recent frame. This strategy alleviates the association problem and allows to detect cell boundaries even during contact events. One may further split this category depending on how prior information

is integrated into the detection step. On the one hand, dataassociation methods can be used to predict various features such as the future position or appearance of a cell [20]. These features are then used to initialize the segmentation algorithm [21]. On the other hand, unified approaches perform propagation and segmentation simultaneously, e.g. using mean-shift approaches [22] or by formulating the problem into an energyminimizing framework. In this latter category, deformable models (also known as active contours) have shown to be particularly efficient, thanks to their flexibility, robustness to noise and quantitative interpretability [23]–[27]. While a full literature review of deformable models falls beyond the scope of this article, we shall briefly recall below the principles of active contours and focus on some of their applications to segmentation and tracking problems in biological imaging. B. Active contours for cells segmentation and tracking The principle of active contours is to draw a contour near an object of interest, and deform the contour iteratively until it reaches the object boundary. The tracking task is then straightforward: once an object has been segmented in a given frame, the resulting contour can be used to initialize the model on the next frame. The deformation is generally obtained by minimizing an energy functional of the form: E = Eimage + Eregul + Eprior . The first term (also referred to as external or data attachment term) links the model to the image data, and its minimization describes how to pull the contour toward the boundaries of interest. The second term is a regularizer of the generally illposed minimization problem, and its minimization tends to smooth the curvature of the deforming contour to reduce the influence of spurious noise artifacts, and thereby the influence of local energy minima. Finally, the third term expresses prior information which may help the contour reach its final position. The energy minimization can then be performed using a number of numerical techniques, e.g. steepest gradient descent. Deformable models are traditionally categorized in two families (explicit or implicit), depending on the mathematical formulation of the problem [?], [28]–[30]. Explicit or parametric models express the contour by means of a parametric curve or surface [28], [31]. The data structure is light, leading to fast computations, however they suffer from important drawbacks such as the dependency on the parametrization, the difficulty to model deformations in the 3D case, and the lack of topological flexibility, i.e. handling of contour splitting and merging. Implicit or level set methods [29] aim at solving these issues by expressing the contour implicitly as the zero-level of a higher-dimensional Lipschitz function. They have become widely popular thanks to their ease of implementation and topological flexibility (implicit contour splitting and merging), however the computational cost raises substantially. An alternative yet related approach to optimize the energy minimization problem resides in the graph-cut theory [?], [30]. By expressing the minimization problem as a graph-based min-cut-max-flow optimization problem [32],

DUFOUR et al.: 3D ACTIVE MESHES: FAST DISCRETE DEFORMABLE MODELS FOR CELL TRACKING IN 3D TIME-LAPSE MICROSCOPY

the optimal contour representing the global energy minimum can be obtained in very few iterations, provided the energy functional can be expressed in terms of graph weights. Graphcut methods are similar to level-set methods in that they are topologically flexible. Yet, recent efforts have been conducted to propose topological constrained graph-cuts [33]. In a cell tracking context, it is important to notice that the implicit topological flexibility can lead to erroneous results. For instance, as soon as two cells come into contact, the zerolevel merges their boundary implicitly, making them impossible to distinguish. Although efforts have been conducted to propose topological constraints on implicit models (either by constraining contours to their initial topology [34], [35], or by coupling multiple contours to prevent them from overlapping each other [1], [24], [36]–[39]), computational cost remains a major issue. Similar efforts have been conducted in the explicit case to implement topological flexibility and parametrization independence, without losing computational efficiency. The idea is to translate the problem into the fully discrete domain, by representing the parametric curve (resp. surface) as a discrete polygon (resp. mesh) [40]. The number of control points is much higher, yielding a better sampling of the image data, while reducing the dependency on the initial parameterization. The deformation is then expressed by applying discrete forces to the control points. Yet, the major interest of discrete models lies in the possibility to handle topological breaks (splitting and merging), either in a semi-automated [41], or fully automated manner [42], [43]. While computational benefits are not significant in the two-dimensional case, 3D discrete models have been widely employed in the medical imaging field, and more particularly for time-critical applications such as haptic feedback simulation for assisted surgery [44]–[46]. For segmentation purposes, mesh models have been applied to iso-level reconstruction [47] as well as edge-based boundary extraction [48]. To our knowledge there has been no report on their application to the minimization of region-based energy functionals, which are of great interest in fluorescence imaging. In [1], we proposed a 3D multi-cell tracking method for fluorescence microscopy, based on the Chan-Vese-MumfordShah model [49]. The model minimizes the well-known Mumford-Shah functional [50] under a piecewise constant assumption. This functional is particularly well suited for fluorescence microscopy, and has been also studied for similar applications by others [39], [51]. Yet, our implementation used multiple coupled level sets to handle cell contacts, inducing substantial computational loads. In the next section, we propose to reformulate the model into the fully discrete domain by introducing 3D active mesh models, which optimize and generalize deformable mesh models by allowing the minimization of both edge-based and region-based terms, while improving segmentation results and reducing computational costs drastically. III. P ROPOSED METHOD In this section we propose a novel implementation of the piece-wise constant Mumford-Shah functional using 3D active

3

meshes1 . We start by recalling the energy functional, and then describe its implementation using the active mesh formalism. A. Multiple coupled active contours with and without edges The original Chan-Vese-Mumford-Shah active contours without edges model [49] is a level set implementation of the two-phase piece-wise constant Mumford-Shah functional [50] to segment regions with fuzzy boundaries. In a tracking context, this model does not handle contact events due to the single level-set implementation. In order to handle contact between cells in a sequence, and to account for the natural variability between cells intensity, we extended this model to the multi-phase case in 2D [24] and 3D [1] by assigning a single level-set function per cell, and by minimizing the overlap region between these level-sets. Additionally, we introduced a volume conservation term in the 3D case to enhance the frontier localization between touching cells, based on the assumption that cells keep a relatively constant volume throughout their displacements. We recall below the general form of the energy functional for an N -dimensional image I defined as a mapping Ω ⊂ R N −→ [0, 1]:
E(R0..n , C1..n ) = λ0 |I − c0 |2 dΩ + R0  
n  gI dC + λ1 |I − ci |2 dΩ + (1) µ Ci

i=1

γ

n  j=i+1


Ri ∩Rj

Ri

dΩ + η

2 


Ri

dΩ − Ai

where gI is the result of an edge detector function applied to the original image, c 0 is the average intensity of the background region R 0 , ci is the average intensity inside the segmented region R i bounded by C i , dC is the N − 1dimensional boundary element, dΩ the elementary image element and λ0 , λ1 , µ, γ, η are empirical weights. The first two terms (weighted by λ {0,1} ) correspond to the data attachment, and is a multi-phase extension of the Chan-Vese-MumforShah functional [49]. The third term (weighted by µ) is the regularizer of the inverse problem and expresses the geodesic length of the contour. The fourth term (weighted by γ) is a prior term that couples multiple contours to prevent them from overlapping during their deformation (allowing to handle objects in contact). The final term (weighted by η) is another prior term stating that the volume of each object remains close to a reference volume A i (computed on the first frame or regularly throughout the sequence) to further improve tracking when this assumption holds (see [1] for further details). Let us now present the general properties of the active mesh framework and how we can minimize (1) using this formalism. B. The 3D active mesh framework In this section we describe the properties of the proposed framework that we call “3D Active Meshes”, where a closed 1 We would like to point out that our model radically differs from the similarly-named active mesh model developed for 2D visual tracking in [52], where a flat two-dimensional mesh with fixed topology is super-imposed on a video sequence, and tracks displacements of the objects corners using the mesh vertices.

4

triangular mesh delineates the 3D surface of a volumetric object. 1) Generalities: An active mesh M is a discrete surface defined by a list of three-dimensional vertices v i = (xi , yi , zi ) defined in a real-coordinate system bounded by the image domain Ω ⊂ R3 . The vertices are linked to form a closed set of counter-clockwise oriented triangles (called faces), such that the mesh boundary represents at all times the contour of a volumetric object, i.e. which is homeomorphic to a sphere. Note that open surfaces may also be represented, however here we restrict our analysis to the segmentation and tracking of cells, and thus consider only closed surfaces. The deformation of the mesh is driven by that of its vertices, under the influence of various forces, computed in our case from the minimization of (1). Since the vertices evolve in a real-space coordinate system, image values are computed using trilinear interpolation of the neighboring image voxels. 2) Surface vs. mesh parametrization: In order to solve spatial derivatives coming from the minimization of (1) in the discrete domain, parametric contours must be regularly sampled using evenly spaced control points. As the contour deforms, the regular sampling condition may no longer be respected, and the contour must be re-parametrized to maintain a correct sampling. For conventional parametric models, this process is generally non-local and time-consuming. Mesh models offer an efficient alternative to this problem by allowing the surface to re-adjust its sampling automatically in a local fashion [40]. The idea is to define a set of local constraints on the mesh, and ensure that all elements of the mesh follow these constraints during the entire deformation process. Typical constraints may relate to the average edge length, triangle area, vertex valence, etc. As the mesh deforms iteratively, any element no longer complying with the constraints is locally re-adjusted using simple and fast topology operators such as vertex insertion, vertex deletion and edge inversion. In the present case we are concerned with regular surface sampling, therefore a constrain on the edge length is imposed, such that each edge length should remain close to a reference distance defining the mesh resolution. Hence, a large reference distance will yield coarse meshes allowing a fast and rough segmentation of the object surface. Conversely, a small reference distance will yield finer meshes, allowing a more precise surface localization, down to sub-pixel precision if the chosen distance is below the image resolution. The choice of resolution is mostly arbitrary, but also depends on the noise level. The local re-sampling operations are further detailed in Appendix A. 3) Topological operators: A well-known drawback of parametric active contours approaches is their inability to handle topology changes, i.e. contour splitting and merging, whereas these changes are implicitly handled with a level-set formulation. Additionally, when contour merging is not wanted, collision detection strategies must be involved to prevent penetration between distinct contours. Mesh models are more flexible in this context, as they allow to detect and handle topological changes explicitly in a controlled manner. Collision between two meshes occurs when one vertex of the source mesh crosses one face of the target mesh. At this

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XXXX XXXX

(a)

(b)

(c)

Fig. 1. Illustration of our mesh merging strategy. Collision is detected when the vertex of the black mesh penetrates a face of the red mesh (a). We then delete the colliding vertex, the faces it belongs to as well as the penetrated face (b). Finally, both meshes are merged by triangulating the gap between each mesh hole (c).

point, the user (or the model) may choose whether the meshes should be merged, or if collision feedback should be applied to prevent penetration. These two cases are described below: a) Mesh merging: The merging operation is illustrated in Fig. 1. First, the source vertex and all corresponding faces in the source mesh are deleted, as well as the crossed face from the target mesh. In a second step, new faces are created between the three vertices of the target mesh and their closest neighbors in the source mesh. After this step, three quadrilateral holes remain, coinciding with each edge of the target face. They are finally filled using pairs of triangles by connecting the closest diagonal vertices. b) Collision detection: Collision detection is a research topic on its own, with many applications in the computer aided design and gaming industries (see [53] and [54] for a comprehensive review). While very efficient algorithms have been developed for rigid objects, algorithms developed for deformable models are not as efficient for two reasons: 1) many of the simplifications made in the rigid case do not apply to the deformable case; 2) computing precise penetration measures at the vertex-scale is an expensive process, even though these measures are necessary in order to produce an adequate and realistic feedback. In [43], a distance threshold was defined in order to detect proximity between two meshes and compute a repulsion force accordingly. Although computationally efficient, this strategy detects mesh proximity rather than actual collision, yielding approximate feedback computations that do not involve the geometry of either mesh. Here we propose a gradual collision detection algorithm that is specifically tailored to the problem at hand, provides exact penetration measures while benefiting from various optimizations in order to minimize the number of intersection tests performed. Consider two meshes M 1 , M2 and their bounding spheres S1 , S2 . We describe below the main steps of the algorithm to determine if M 1 is colliding with M2 (the other case is done by analogy): 1) If S1 and S2 do not intersect, stop. 2) Extract all vertices of M 1 located within S2 . If the list is empty, stop. 3) Intersection test: for each vertex v i of the previous list, construct a vector from v i to the center of M 1 , and test the intersection between this ray and all faces of M2 with same orientation [55]. This technique greatly reduces the number of tests and, hence, computations.

DUFOUR et al.: 3D ACTIVE MESHES: FAST DISCRETE DEFORMABLE MODELS FOR CELL TRACKING IN 3D TIME-LAPSE MICROSCOPY

This algorithm performs progressive collision detection and handling, and performs much faster than brute-force approaches, where each vertex of the source mesh is tested against each face of the target face. Although the algorithm is computationally more demanding than the distance-based approach, the collision detection and feedback computation are more accurate and actually allow mesh boundaries to touch each other. It is worth noting that as collision tests between all mesh pairs are performed before any feedback can actually be applied, collision detection between multiple meshes can be further optimized using parallel computing. Finally, our mesh splitting scheme is similar to [43], although for purpose of completeness we recall the details of the algorithm in Appendix B. 4) Mesh rasterization: The image terms in (1) express the average image intensities of the background and inside each segmented region. These variables must be regularly updated along the minimization process. While this process can be considered cost-free in level set-based approaches (where the sign of the function itself defines the inside and outside of the contour everywhere in the image domain), this is not the case with meshes which only represent the surface boundary. Therefore, to compute these values, one needs to check where each image voxel lies with respect to each mesh (a process known as rasterization). In the brute-force case, for each image voxel one must compute the position of the voxel relative to each face of each mesh, and combine these positions to determine where the voxel lies, which turns out to be a very expensive process. We propose here an optimized strategy based on the curve parity test derived from the well-known Jordan curve theorem. In short, Jordan observed that any ray traced from a point outside (resp. inside) a closed curve crosses this curve an even (resp. odd) number of times. We illustrate this principle in 2D in Fig. 2-a. (note that a pixel is considered inside the curve as soon as its geometrical center is inside). By applying the same strategy for every image voxel, one may thus extract inside and outside regions of a mesh easily, a strategy we call the Point-to-Mesh rasterization scheme (PMR). Nevertheless, although ray-tracing is a standard computer graphics tool which benefits from hardware acceleration, the PMR scheme does not perform in reasonable time due to the huge amount of tests to conduct (one per image voxel), especially for large image stacks. To optimize the rasterization process, we first restrict the rasterized volume to the axis-aligned bounding-box of the mesh, reducing the number of tests from O(N 3 ) (N is the number of image voxels per dimension) to O(M 3 ), M i

Here κ and N express respectively the curvature and outer unit normal of S i , while 1X is the classical indicator function of the set X. Under the mesh formalism, we transpose these equations into the discrete domain to obtain one evolution equation per mesh Mi=1..n , given by:  ∂Mi  = Fint (v) + Fext (v) + Fcpl (v) + Fvol (v), (3) ∂t v where each term expresses respectively the internal (or regularization), external (or data attachment), coupling and volume conservation term. Convergence is detected when the system reaches equilibrium, i.e., ∀i, ∂M i /∂t <  with  1, whereas the case D = 1 boils down to the original non-recursive definition. 2) Internal force: Minimizing the internal energy term requires to minimize the geodesic contour length, as originally described in the 2D case in [57]. In the mesh formalism, the internal force applied to each mesh vertex becomes  Fint (v) = µ gI κv Nv − (∇gI · Nv ) Nv , (5) with the curvature and normal vectors defined as above. 3) External force: The external force driving each vertex towards the object boundary is given by

 Fext (v) = λ0 |c0 − I(v)|2 − λ1 |ci − I(v)|2 Nv , where I(v) is the image intensity value computed by trilinear interpolation from the closest image voxels to v (in the 26 (v) is the normal vector to the mesh at neighborhood), and N vertex v pointing outwards.

4) Coupling force: The coupling force is meant to repel the surface at locations where it overlaps with another surface. While the indicator function 1 Sj ∩Sj only gives a unique value expressing the total overlap volume, we optimize the resulting force by adjusting it to each mesh vertex depending on its penetration depth. To do so we use our efficient collision detection algorithm (cf. III-B3b), which gives us for each vertex the exact distance to the surface of the penetrated mesh. The final coupling force applied to each vertex of a mesh M i thus reads:    n  Fcpl (v) = −γ colldist(v, Mj ) Nv , j=i+1

where colldist(v, Mj ) is the collision distance function calculating the penetration depth of v within the mesh M j . This force has zero-norm if v does not penetrate M j . 5) Volume conservation force: The volume conservation force, as introduced in [1], can be interpreted as an additional feedback force preventing vertices from moving in such a way that the mesh volume changes drastically. This force reads: Fvol (v, Ai ) = η (Ai − vol(Mi )) Nv , where vol(Mi ) expresses the current volume of the mesh M i , and Ai represents its reference volume. If A i > vol(Mi ), then the mesh is too small and should grow to reach its reference volume. v is thus driven in the direction of the mesh normal. Conversely if A i < vol(Mi ), v will move in the opposite direction to shrink the mesh. It is worth noticing that this constraint can also be used to track dividing objects over time. Indeed, the reference volume A i can be computed either after segmentation of the first sequence frame (assuming the object volume is constant over time), or updated after each sequence frame (in case the object undergoes a smooth volume variation). The latter strategy is suitable to track object divisions, provided the weight of the constraint is chosen proportionally to the temporal resolution of the sequence (and inversely proportional to the estimated speed of the division process). D. Initialization It is well known that active contour methods converge faster and more efficiently if they are initialized close to the desired solution. As pointed out in [58], the optimal way to initialize energy-minimizing models is to apply a preprocessing algorithm that conforms best to the heart of the energy functional, i.e. the data attachment term. In our case, the global minimum of the region-based term in (1) boils down to the result of a multi-class K-Means clustering of the pixel intensities (assuming equal region weights). Additionally, prior information such as the average object size can be used to separate clusters during initialization. In [59], we presented such an algorithm for 2D fluorescence images based on intensity quantization, that is able to detect cells with different mean intensities while separating clusters with some success. The idea is to quantize the image into N intensity classes (e.g. using K-Means clustering), and produce a binary image for each threshold. Then, for each threshold in ascending order,

DUFOUR et al.: 3D ACTIVE MESHES: FAST DISCRETE DEFORMABLE MODELS FOR CELL TRACKING IN 3D TIME-LAPSE MICROSCOPY

connected components smaller than a given value (e.g. the maximum known object size) are extracted from the binary image, triangulated into a mesh, and discarded from the remaining binary volume to avoid duplicate extractions. This method is fast and robust (initialization results converge for increasing values of N ), and just requires that the maximum size of the objects is known a priori. Once the binary image volume is obtained, one may construct a triangular mesh from each connected component using 3D reconstruction methods such as Voronoi diagrams, Delaunay triangulation and marching algorithms. Marching algorithms are better suited to our context since the final reconstructed grid should be regular to conform with the homogeneous sampling condition. Therefore we have chosen the marching tetrahedra [60], which removes ambiguities induced by the marching cubes. Once each component is reconstructed, the mesh is directly in the correct form and can be used for deformation right away.

E. Detection of incoming/leaving/dividing objects Another issue in multiple object tracking is the appearance and disappearance of objects that enter or leave the field of view during the acquisition. When an object leaves the field, its contour vanishes and the track is ended. Conversely, new objects entering the field are automatically detected by applying a technique similar to the initialization scheme described above, though restricted to a narrow band around the edge of the image volume, which is both fast and efficient. Finally, in the particular case of cell tracking, cell division is also a key event which may occur during the observation and should also be detected. Here again the active mesh framework is able to handle mesh divisions automatically as the daughter cells progressively move away from each other. Upon division, the mesh initially segmenting the mother cell elongates along the division axis to follow both daughter cells simultaneously, while it shrinks in between the daughter cells, forming a bonelike shape. Ultimately, the mesh shrinks up to a certain limit where a division can be detected, and a new mesh is created for each daughter cell by splitting the mesh segmenting the mother cell (see details and illustration in Appendix B). This division information can then be further exploited to describe vital parameters such as division orientation or cell lineage. IV. E XPERIMENTS AND RESULTS In this section we evaluate the performance of the proposed method on synthetic and experimental data sets in terms of segmentation quality and computational requirements. Throughout the evaluation, we take as reference method the active surfaces model [1], and compare it with our new active meshes implementation. The method has been implemented in Java to integrate our cross-platform image processing software1 . The mesh rendering step was hardware optimized (using the Java OpenGL library). Computation times are given for an Intel 2.8 GHz 64-bit processor, also we stress that these times can be further reduced using other languages such as

7

(a) high SNR (σ = 5) (b) med. SNR (σ = 10)

(c) low SNR (σ = 15)

(d) fluorescence image

Fig. 3. Slice of three synthetic images of size 512 × 512 × 30 with varying SNR (see text for details), and a sample fluorescence image.

C++/C# which provide better integration with GPU libraries like OpenGL and DirectX. A. Validation on synthetic data 1) Data simulation: Ground-truth data is rarely available in biological imaging. This is especially true in 3D, as manual analysis is tedious, prone to user bias and often inaccurate (especially on the top and bottom slices, due to poor depth resolution in fluorescence microscopy). Instead, one usually relies on simulated data, where performance can be evaluated in an objective way, provided the simulation mimics the image formation process sufficiently well. As we evaluate segmentation and tracking independently, we have generated two sets of 100 images each as follows: a) Segmentation data: We have generated synthetic 3D images containing a random-sized ellipse (representing the cell body), plus a random number of smaller ellipses centered close to the main ellipse boundary (representing cell protrusions). Then, these images were edited to simulate a typical fluorescence microscopy system: background auto-fluorescence, poisson distribution of detected photons, gaussian noise induced by the instrument, and anisotropic voxel resolution (lower along the depth axis). As a result, we have generated three sets of 100 image stacks with varying signal-to-noise ratio (SNR) by changing the standard deviation of the final Gaussian noise: 5 voxels for a high SNR, 10 voxels for a medium SNR and 15 voxels for a low SNR. A sample of high, medium and low SNR images is shown in Fig. 3, alongside a real image serving as a visual reference. b) Tracking data: Here we have generated 100 sequences of 3 frames each, where we simulate a contact event between two objects. Each object is generated in a similar manner as above, with the constraint that each cell body keeps a constant size over the sequence. Still, the body shape, as well as the number, shape and size of the simulated protuberances 1 more

information at http://www.bioimageanalysis.org/software.html

8

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XXXX XXXX

TABLE II C OMPARISON OF TRACKING ERRORS ON SIMULATED CONTACT EVENTS . E RRORS REPRESENT THE OVERLAP BETWEEN CELLS , AND THE TOTAL CELL - BACKGROUND AND CELL - CELL CONFUSION ( AS DEFINED IN TEXT ). B OLD VALUES INDICATE BEST RESULTS .

Method (a) Before contact Fig. 4.

(b) During contact

(c) After contact

Slice of three frames of a synthetic contact sequence (low SNR).

TABLE I C OMPARISON OF SEGMENTATION RESULTS ON SIMULATED DATA SETS . R ESULTS ARE AVERAGED OVER 100 IMAGES , AND REPRESENT THE JACCARD DISSIMILARITY INDEX BETWEEN REAL AND SEGMENTED OBJECTS , WITH A DISTANCE TOLERANCE OF 0, 1 AND 2 VOXELS FROM THE GROUND - TRUTH SURFACE . B OLD VALUES HIGHLIGHT BEST RESULTS .

Method

Jaccard dissimilarity (%) Abs. > 1 vox. > 2 vox.

CPU (sec.)

Mem. (MB)

High SNR Classic Active Surfaces Active Meshes

3.8 3.0 2.2

0.02 0.02 0.01

0 0 0

– 52 16

– 60 0.07

Medium SNR Classic Active Surfaces Active Meshes

5.9 5.5 3.2

0.27 0.24 0.2

0.05 0.05 0.04

– 87 19

– 60 0.07

Low SNR Classic Active Surfaces Active Meshes

8.6 8.2 4.5

2.6 1.3 0.8

1.9 0.8 0.1

– 125 27

– 60 0.07

remained random, as it is the case in real life. In order to simulate contact, we imposed that the cells be separated in the first frame, then overlap by a small amount of voxels in the second frame, and finally separate in the third frame. An illustration of a simulated sequence is given in Fig. 4. 2) Results: Segmentation quality is evaluated here as an error ratio derived from the Jaccard similarity index between the segmented and original objects. The Jaccard index expresses a ratio between the intersection and the union of two sample sets, and is equal to zero if the sets do not overlap, and one if both sets are identical. For easier reading, we rather compute the dissimilarity index (i.e., one minus the Jaccard index) and express this index as a percentage. Table I compares the segmentation performance of our active meshes to two reference methods: our active surfaces model [1], and a segmentation method consisting of an anisotropic diffusion [61] followed by an automated thresholding [62] and corrected using mathematical morphology operators (we refer to this method as classic). For each SNR, we compute three error measures: the “absolute” error, plus two “relaxed” errors where misclassified voxels lying within a distance of one and two voxels from the real object boundary are not accounted for. Computation times include initialization and energy minimization, however 3D rendering time is not included in the active surfaces case, whereas it is performed in real-time in the proposed method. Results show that the proposed approach slightly improves

Low SNR Watershed-based Active Surfaces Active Meshes

overlap 0 0.2 0

Error upon contact (%) cell-background cell-cell 7.7 7.5 4.1

1.5 0.6 0.5

the overall segmentation quality compared to the other methods, though significantly more in low SNR conditions. This is due to the fact that the regularizer of the level set approach minimizes the curvature of the level set function and not its zero-level per se, whereas in the proposed approach the curvature of the contour itself is minimized, reducing the influence of bright noise voxels lying in the vicinity of the object. But most importantly, active meshes turn out to be much more efficient in terms of computational load, as both memory and computation times are drastically reduced. Tracking performance is assessed in two steps. First we verified that the objects identity was correctly preserved along all the simulated sequences. Then, we evaluated the accuracy of the coupling term by measuring the overlap between meshes during contact. Here we split the Jaccard index into several measures, as we wanted to separate cell-background (similar as above) and cell-cell confusion. Cell-cell confusion is given here as the ratio of the number of voxels of a cell segmented by the wrong contour over the visible cell volume. We stress that during our simulation process, we mimic cell contact by forcing the objects to overlap by a small amount of voxels, therefore we define the visible cell volume as the total volume of the cell minus this artifical overlap. In this experiment we also compared the performance with the classic segmentation method described above, however touching cells would be considered as a single cluster. Therefore we have added a 3D watershed algorithm [63] to arbitrarily separate the cluster into two cells (this method is referred to as watershed-based). Results are given in table II for the low SNR data set. The watershed-based method necessarily separates the cluster into distinct components, inducing no overlap. Note that in contrast to deformable models, the watershed is only efficient on clusters of pseudo-spherical objects, and would fail to distinguish a concave cell from a cluster. Also, this method does not preserve objects identity, requiring an additional association step for tracking purposes. Active surfaces keep a small overlap upon contact, due the implicitness of the contour while minimizing the coupling term. In contrast active meshes do not overlap thanks to the explicitness of the collision detection algorithm. As expected, cell-background confusion values coincide with table I, though they are slightly lower since the virtual overlap area between the simulated cells is not accounted for. Finally, the cell-cell confusion ratio is much higher with the watershed-based method, since the

DUFOUR et al.: 3D ACTIVE MESHES: FAST DISCRETE DEFORMABLE MODELS FOR CELL TRACKING IN 3D TIME-LAPSE MICROSCOPY

Time (milliseconds)

10
000 K

Point-to-Mesh

1
000 K 100 K 10 K

Line-to-Mesh

1
000 100 10 1 1K

10 K

100 K

Mesh faces

Fig. 5.

Performance of the rasterization schemes for varying mesh sizes.

separation is arbitrarily based on the objects convexity. As expected, results with active meshes are only slightly improved compared to active surfaces, which is not surprising as we apply the same volume conservation constraint in both cases. B. Active meshes: core performance In this section we focus on evaluating the core performance of critical parts of the proposed framework. We first evaluate the performance of the collision detection and mesh rasterization algorithms, then describe the time consumption balance per iteration. 1) Collision detection: We compared the performance of our collision detection algorithm with the original distancebased method described in [43]. For purpose of equity, we have applied in both cases the same bounding sphere-based pruning technique to optimize the number of intersection tests. Not surprisingly, since the original distance-based approach only needs to compute the euclidean distance between pairs of vertices between meshes, the detection and feedback computation time per vertex turns out twice faster than with our approach, where systematic ray-triangle intersection tests are performed at each vertex (this result was independent of the mesh size). As an example, the detection time between two meshes sharing a colliding area of 400 faces required 60 milliseconds using the distance-based approach, compared to 120 milliseconds in our case. We also noted that this ratio is independent of the size of either mesh. Nonetheless, we stress that in the distancebased case, the feedback is approximate since dependent on the empirical distance threshold defining whether a vertex is colliding the opposite mesh or not. As a result, the two meshes never actually touch each other, but remain separated by a minimal distance of the order of its resolution [43]. In our approach, collision feedback is computed from actual penetrations of each vertex in the opposite mesh, therefore the colliding meshes have no empty gap in between. 2) Mesh rasterization: The graph in Fig. 5 plots the performance of the mesh rasterization scheme described in Section III-B4. Each data point represents the time in milliseconds needed to rasterize a mesh into a binary volume, and tests have been conducted on a total of 300 meshes of different sizes. Note that the mesh size is given as the number of faces, which is proportional to the mesh resolution as well as the size of the segmented object. It can be argued that although

9

the number of intersection tests have been reduced by one order of magnitude, the rasterization speed does not follow the same trend. This is due to the fact that in the Point-toMesh scheme, each intersection test is sufficient alone to label the corresponding voxel as “inside” or “outside”, whereas in our Line-to-Mesh scheme the intersection test outputs a list of intersecting faces which much be first sorted in ascending distance to the ray origin before sweeping along the line to label the corresponding voxels. Yet, this difference is still significant as it offers a two-fold improvement in computational cost compared to the previous scheme. Moreover, it is worth noticing that the rasterization process need not be conducted at every iteration, since our intialization scheme (described in Section III-D) already provides a good estimate of both the final surface and the intensity variables. Also, once the variables have been estimated on the first frame, they need not be computed again for the other frames (or at best once per frame, in case the cells intensity decrease due to photobleaching). 3) Time consumption balance per iteration: Finally, we have measured the time consumption balance per iteration on the simulated time-lapse data, so as to highlight the most efficient and most costly operations of the minimization process. The rasterization clearly turns out to be the most expensive process, requiring always over 90 percent of the total time depending on the mesh size (see Fig. 5). The second most costly operation is the collision detection process, which consumes up to 15 percent of the total time, depending on the number of collision tests (i.e. the size of the colliding area). Finally, the remaining operations (force computation, self-parametrization, topology handling, data transfer to the GPU) totalize less than 5 percent of the total time. Potential improvements to the method should therefore focus on the optimization of the two most expensive processes, i.e. rasterization and collision detection. C. Results on real biological data In this section, we illustrate an application of the proposed method to segment and track multiple Entamoeba histolytica parasites crawling in a 3D fibrillar collagen matrix. Parasites are labeled using a cytoplasmic fluorescent dye, and observed using a Nipkov-disk confocal 3D microscope equipped with a 25× (NA 0.8) oil-immersed objective and a 16-bit Andor camera, yielding images of digital size 1024 × 1024 pixels and plane resolution 0.327 × 0.327 µm (i.e. the field of view is 335 × 335 µm wide, allowing us to observe up to 15 parasites simultaneously). Each 3D stack is then obtained by acquiring 32 consecutive planes along the depth axis with an inter-plane distance of 3 µm, resulting in a depth resolution about ten times lower than the axial resolution. Finally, 3D time-lapse sequences are obtained by acquiring 90 of these 3D stacks with an interval of 10 seconds between stacks (i.e. the total observation time per experiment is 15 minutes), yielding a typical data size of 5.625 GigaBytes per 4D sequence. Such a data set illustrates well the computational limit of the multiple coupled level-set approach we proposed in

10

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XXXX XXXX

(a)

(b) (a)

(b)

Fig. 7. Tracking results on two amoeba crawling in a 3D collagen matrix. Last stack of a 4D sequence with 150 stacks of size 400 × 379 × 20 voxels and resolution 0.327 × 0.327 × 3µm. (a) MIP of the stack at t=150. (b) Cell tracks superimposed on the final meshes at t=150 (resolution: 1 µm; size: 5000 to 7000 faces each; time: 3 to 7 seconds per stack).

(c) Fig. 6. Segmentation result on a low contrast cell. Stack size is 80 × 80 × 34 voxels, and voxel resolution is 0.327 × 0.327 × 3 µm. (a) Maximum intensity projection (MIP) of the stack. (b) Pseudo-colored MIP. (c) Final mesh after convergence (resolution: 0.327 µm; size: 16300 faces; time: 11 seconds).

[1]. Since we assign a single level-set to each object, and a level-set is represented as a real-valued function defined over the image domain, the total amount of memory required to segment 15 objects is substantially high (e.g. in our case, almost 4 GigaBytes). Although optimizations techniques such as narrow-band can be implemented [64], such memory load remains impractical for other applications dealing with a greater number of objects. In comparison, the memory load with our active meshes implementation is about 5 MegaBytes for the same data set. High-resolution segmentation on a low contrast image is illustrated in Fig. 6. The mesh resolution has been set to 0.327 µm, which is the image resolution along the plane axis. Despite the low image contrast, the cell boundary is correctly recovered with sub-pixel precision along the depth axis. An illustration of cell-cell contact event is given in figures 7 and 8. In Fig. 7, two amoeba have been successfully segmented and tracked, despite the relatively low fluorescent signal they emit. During this sequence, the two left-most amoeba come into contact, and remain so during several time points until they separate again. This situation is further illustrated in Fig. 8. Upon contact, the mesh surfaces come into contact without overlapping, as expected. In total we have processed 52 time-lapse sequences (i.e. around 125 GigaBytes of data) in a systematic manner, totalizing 274 cells, all correctly tracked by the method. In order to relate quantitative information to previously published data, we calculated an average speed of 11.9 µm/min., suggesting that motility in 3D collagen matrix is lower than on glass or

plastic substrates [65], but comparable to in vivo environments [66]. Additionally, thanks to the collision detection method, we were able to efficiently measure the contact duration between touching cells, which averaged at 75 seconds for a total of 56 contacts. We are now pursuing this study by applying the same method to modified strains, in which proteins involved in the parasites ability to degrade collagen has been inhibited, so as to compare motility statistics between strains. V. C ONCLUSION We have presented a novel deformable model framework based on 3D active meshes for the problem of multi-cell segmentation and tracking in 3D time-lapse fluorescence microscopy. Our method drastically increases the performance of traditional region-based variational frameworks by expressing the surface and the minimization problem directly in the discrete domain, allowing the incorporation of various optimizations taken from the computer graphics field. Quantitative evaluation shows that the proposed framework outperforms other methods both in terms of segmentation quality and computational cost, and allows us to process large experimental data sets that could not be handled using previous approaches. Results on real data show that the proposed approach is able to extract various quantitative information such as motility statistics and contact durations, thus providing a flexible tool that can be applied to other biological problems (e.g. cell-cell interaction studies within the immune system), and more generally to a wider range of 3D/4D image processing applications. A PPENDIX A M ESH SELF - PARAMETRIZATION We start with the assumption that the initial mesh has a constant resolution (i.e. the initial distance between any pair of neighbor vertices). A length interval [d min , dmax ] is then defined to bound the possible edge deformations, and local resampling methods are applied as soon as an edge length falls out of this range. If d < d min , the edge should be merged into a single vertex. Conversely, if d > d max , the edge can either be inverted (with respect to the triangles containing the edge) or split by placing a new vertex inserted in its center (see Fig. 9). We give priority to edge inversion, as it minimizes the mesh

DUFOUR et al.: 3D ACTIVE MESHES: FAST DISCRETE DEFORMABLE MODELS FOR CELL TRACKING IN 3D TIME-LAPSE MICROSCOPY

(a) Before contact (t=4)

(b) After contact (t=5)

(c) Before separation (t=11)

11

(d) After separation (t=12)

Fig. 8. Close-up views of the sequence shown in Fig. 7, illustrating the collision detection system implemented in the framework to handle cell contacts. For display purposes, active meshes are rendered here in wire-frame mode with back-face culling.

complexity. It was shown in [43] that a correct embedding of the surface in R 3 is obtained if dmax > 2.5 dmin . Yet, these values are chosen empirically. High values will lead to a coarse mesh that may not capture the finer details of the cell boundary. Small values induce complex meshes, increasing the computational load as well as the sensitivity to local spurious noise artifacts. (a) Before splitting

Fig. 9. Local mesh re-sampling operations. If an edge gets too long (top left), it is either split to create 2 new faces or inverted if the resulting edge length is within the distance interval. If an edge gets too short (top right), the two corresponding faces are deleted and the vertices are merged

A PPENDIX B M ESH SPLITTING OPERATOR Mesh splitting occurs when a portion of a mesh shrinks into a tubular structure until it eventually self-intersects. Fortunately, this situation can be directly detected during the resampling process described earlier, and is illustrated in Fig. 10. During the re-sampling process, when two vertices (here A and B) are to be merged together, i.e., dist(A, B) < dmin , we browse through both neighborhoods V(A) and V(B) and count the number of common vertices, i.e. n =  card (V(A) V(B)). In a normal situation, it can be easily verified that n = 2 (see, e.g., Fig. 9-top right), which corresponds to the third vertex of the two faces containing A and B. However, if n = 3, both vertices are sharing a third
is not a valid mesh face. We call neighbor (C), while ABC this face the splitting face, and use it to cut the mesh into two new meshes by duplicating vertices A, B and C and creating two new faces to fill the resulting hole in each mesh. ACKNOWLEDGMENT The authors would like to thank the anonymous reviewers for their critical comments and remarks which helped in improving the quality of this manuscript.

(b) After splitting Fig. 10. Description of the mesh splitting scheme (as presented in [43]). As vertices A and B come closer and call for a merge operation, there exists
is not a mesh face. The virtual a third common neighbor C such that ABC
is then used to cut the mesh and fill the holes in each new mesh. face ABC

R EFERENCES [1] A. Dufour, V. Shinin, S. Tajbaksh, N. Guillen, J.-C. Olivo-Marin, and C. Zimmer, “Segmenting and tracking fluorescent cells in dynamic 3d microscopy with coupled active surfaces,” IEEE Transactions on Image Processing, vol. 14, no. 9, pp. 1396–1410, September 2005. [2] C. Zimmer, B. Zhang, A. Dufour, A. Thebaud, S. Berlemont, V. MeasYedid, and J.-C. Olivo-Marin, “On the digital trail of mobile cells,” Signal Processing Magazine, vol. 23, no. 3, pp. 54–62, May 2006. [3] E. Meijering, I. Smal, O. Dzyubachyk, and J.-C. Olivo-Marin, “Timelapse imaging,” in Microscope Image Processing. Elsevier, 2008, pp. 401–440. [4] G. Aubert and P. Kornprobst, Mathematical problems in image processing. Springer, 2001. [5] P. Heid, E. Voss, and D. Soll, “3D-DIASemb: a computer-assisted system for reconstructing and motion analyzing in 4D every cell and nucleus in a developing embryo,” Developmental Biology, vol. 245, no. 2, pp. 329–347, May 2002. [6] D. Baggett, M. Nakaya, M. McAuliffe, T. Yamaguchi, and S. Lockett, “Whole cell segmentation in solid tissue sections,” Cytometry Part A, vol. 67, pp. 137–143, 2005. [7] P. Roux, S. M¨unter, F. Frischknecht, P. Herbomel, and S. L. Shorte, “Focusing light on infection in four dimensions,” Cell Microbiology, vol. 6, no. 4, pp. 333–343, Apr. 2004. [8] C. Vonesch, F. Aguet, J.-L. Vonesch, and M. Unser, “The colored revolution of bioimaging,” IEEE Signal Processing Magazine, vol. 23, no. 3, pp. 20–31, 2006. [9] I. Sbalzarini and P. Koumoutsakos, “Feature point tracking and trajectory

12

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XXXX XXXX

analysis for video imaging in cell biology,” Journal of Structural Biology, vol. 151, pp. 182–195, 2005. N. Chenouard, A. Dufour, and J.-C. Olivo-Marin, “Tracking algorithms chase down pathogens,” Biotechnology Journal, vol. 4, pp. 838–845, 2009. O. Al-Kofahi, R. Radke, S. Goderie, Q. Shen, S. Temple, and B. Roysam, “Automated cell lineage construction: a rapid method to analyze clonal development established with murine neural progenitor cells,” Cell Cycle, vol. 5, no. 3, pp. 327–335, 2006. F. Bunyak, K. Palaniappan, S. Nath, T. Baskin, and G. Dong, “Quantitative cell motility for in vitro wound healing using level-set based active contour tracking,” in Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI 2006), Arlington, VA, USA, Apr. 2006, pp. 1040–1043. A. Mosig, S. J¨ager, C. Wang, S. Nath, I. Ersoy, K. Palaniappan, and S. Chen, “Tracking cells in life cell imaging videos using topological alignments,” Algorithms for Molecular Biology, vol. 4, no. 10, pp. 1–9, 2009. D. Padfield, J. Rittscher, and B. Roysam, “Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis,” Medical Image Analysis, 2010, in press. N. Papadakis and E. M´emin, “A variational technique for time consistent tracking of curves and motion,” Journal of Mathematical Imaging and Vision, vol. 31, no. 1, pp. 81–103, May 2008. W. Tvarusko, M. Bentele, T. Misteli, R. Rudolf, C. Kaether, D. Spector, H. Gerdes, and R. Eils, “Time-resolved analysis and visualization of dynamic processes in living cells,” Cell Biology, vol. 96, pp. 7950 – 7955, Jul. 1999. S. Bonneau, M. Dahan, and L. D. Cohen, “Single quantum dot tracking based on perceptual grouping using minimal paths in a spatiotemporal volume,” IEEE Transactions on Image Processing, vol. 14, no. 9, pp. 1384–1395, Sep. 2005. D. Padfield, J. Rittscher, N. Thomas, and B. Roysam, “Spatio-temporal cell cycle phase analysis using level sets and fast marching methods,” Medical Image Analysis, vol. 13, no. 1, pp. 143–155, Feb. 2009. D. Pastor, M. Luengo-Oroz, B. Lombardot, I. Gonzalvez, L. Duloquin, T. Savy, P. Bourgine, N. Peyrieras, and A. Santos, “Cell tracking in fluorescence images of embryogenesis processes with morphological reconstruction by 4D-tubular structuring elements,” in Proceedings of the 31st International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2009), Minneapolis, MN, USA, Sep. 2009. K. Li, E. D. Miller, M. Chen, T. Kanade, L. E. Weiss, and P. G. Campbell, “Cell population tracking and lineage construction with spatiotemporal context.” Medical Image Analysis, vol. 12, no. 5, pp. 546–566, Oct. 2008. H. Shen, G. Nelson, S. Kennedy, D. Nelson, J. Johnson, D. Spiller, M. White, and D. Kell, “Automatic tracking of biological cells and compartments using particle filters and active contours,” Chemometrics And Intelligent Laboratory Systems, vol. 82, pp. 276–282, 2006. O. Debeir, P. V. Ham, R. Kiss, and C. Decaestecker, “Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes,” IEEE Transactions on Medical Imaging, vol. 24, no. 6, pp. 697–711, June 2005. D. Mukherjee, N. Ray, and S. Acton, “Level set analysis for leucocyte detection and tracking,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 562–572, Apr. 2004. B. Zhang, C. Zimmer, and J.-C. Olivo-Marin, “Tracking fluorescent cells with coupled geometric active contours,” in Proceedings if the International Symposium on Biomedical Imaging (ISBI 2004), Arlington, VA, USA, Apr. 2004, pp. 476–479. C. Zimmer and J.-C. Olivo-Marin, “Coupled parametric active contours,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1838–1842, Nov. 2005. J. Pecreaux, C. Zimmer, and J.-C. Olivo-Marin, “Biophysical active contours for cell tracking i: Tension and bending,” in Proceedings of the IEEE International Conference on Image Processing (ICIP 2006), 2006, pp. 1949–1952. I. Ersoy and K. Palaniappan, “Multi-feature contour evolution for automatic live cell segmentation in time lapse imagery,” in Proceedings of the 30th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2008), Aug. 2008. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes : active contour models,” International Journal of Computer Vision, vol. 1, pp. 321–331, 1988. J. A. Sethian, Level set methods and fast marching methods, 2nd ed. Cambridge University Press, 1999.

[30] N. Xu, R. Bansal, and N. Ajuha, “Object segmentation using graph cuts based active contours,” in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2003), Jun. 2003. [31] D. Terzopoulos and D. Metaxas, “Dynamic 3D models with local and global deformations: deformable superquadrics,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 703–714, Jul. 1991. [32] Y. Boykov and V. Kolmogorov, “An experimental comparison of mincut/max-flow algorithms for energy minimization in vision,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124–1137, 2004. [33] Y. Zeng, D. Samaras, W. Chen, and Q. Peng, “Topology cuts: a novel min-cut/max-flow algorithm for topology preserving segmentation in ND images,” Computer Vision and Image Understanding, vol. 112, pp. 81–90, 2008. [34] X. Han, C. Xu, and J. Prince, “A topology preserving level set method for geometric deformable models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp. 755–768, Jun. 2003. [35] F. Segonne, “Active contours under topology control - genus preserving level sets,” International Journal of Computer Vision, vol. 79, pp. 107– 117, 2008. [36] C. Samson, L. Blanc-Feraud, G. Aubert, and J. Zerubia, “A level set model for image classification,” in Scale-Space Theories in Computer Vision, ser. Lecture Notes in Computer Science, vol. 1682. Springer, Sep. 1999, pp. 306–317. [37] N. Paragios and R. Deriche, “Coupled geodesic active regions for image segmentation: a level set approach,” in Proceedings of the European Conference on Computer Vision (ECCV 2000), vol. 2, 2000, pp. 224– 240. [38] S. Nath, K. Palaniappan, and F. Bunyak, “Cell segmentation using coupled level sets and graph-vertex coloring,” in Proceedings of MICCAI 2006 - Medical Image Computing and Computer-Assisted Intervention, ser. Lecture Notes in Computer Science, vol. 4190. Copenhagen, Danemark: Springer, Oct. 2006, pp. 101–108. [39] O. Dzyubachyk, W. van Cappellen, J. Essers, W. Niessen, and E. Meijering, “Advanced level-set based cell tracking in time-lapse fluorescence microscopy,” IEEE Transactions on Medical Imaging, vol. 29, no. 3, pp. 852–867, Mar. 2010. [40] J. Miller, D. Breen, W. Lorensen, R. O’Bara, and M. Wozny, “Geometrically deformed models: a method for extracting closed geometric models form volume data,” in Proceedings of SIGGRAPH, 1991. [41] H. Delingette, “General object reconstruction based on simplex meshes,” International Journal on Computer Vision, vol. 32, pp. 111–146, 1999. [42] T. McInerney and D. Terzopoulos, “T-snakes: topology adaptive snakes,” Medical Image Analysis, vol. 4, pp. 73–91, 2000. [43] J.-O. Lachaud and A. Montanvert, “Deformable meshes with automated topology changes for coarse-to-fine three-dimensional surface extraction,” Medical Image Analysis, vol. 3, no. 2, pp. 187–207, Jun. 1999. [44] J. Montagnat and H. Delingette, “Volumetric medical images segmentation using shape constrained deformable models,” in (CVRMed-MRCAS) Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine - Medial Robotics and Computer-Assisted Surgery, ser. Lecture Notes in Computer Science, J. Troccaz, W. E. L. Grimson, and R. Moesges, Eds., vol. 1205. Grenoble, France: Springer-Verlag, Mar. 1997, pp. 13–22. [45] S. Cotin, H. Delingette, and N. Ayache, “Real-time elastic deformations of soft tissues for surgery simulation,” IEEE Transactions on Visualization and Computer Graphics, vol. 5, pp. 62–73, 1999. [46] V. Zagrodsky, V. Walimbe, C. Castro-Pareja, J. Qin, J.-M. Song, and R. Shekhar, “Registration-assisted segmentation of real-time 3-D echocardiographic data using deformable models,” IEEE Transactions on Medical Imaging, vol. 24, no. 9, pp. 1089–1099, Sep. 2005. [47] A. Benassarou, J. Freitas-Caires, and E. Bittar, “An integrated framework to analyze and visualize the evolution of multiple topology changing objects in 4D image datasets,” in Proceedings of the 7th workshop on vision, modeling and visualization, G. Greiner, H. Niemann, T. Ertl, B. Girod, and H.-P. Seidel, Eds., Nov. 2002, pp. 147–154. [48] J.-O. Lachaud and B. Taton, “Deformable model with adaptive mesh and automated topology changes,” in Proceedings of the 4th International Conference on 3-D Digital Imaging and Modeling. Banff: IEEE, Oct. 2003, pp. 12–19. [49] T. Chan and L. Vese, “Active contours without edges,” IEEE Transactions on Image Processing, vol. 10, no. 2, pp. 266–277, Feb. 2001. [50] D. Mumford and J. Shah, “Optimal approximations of piecewise smooth functions and associated variational problems,” Communications on Pure and Applied Mathematics, vol. 42, no. 4, pp. 577–685, 1989.

DUFOUR et al.: 3D ACTIVE MESHES: FAST DISCRETE DEFORMABLE MODELS FOR CELL TRACKING IN 3D TIME-LAPSE MICROSCOPY

[51] B. Rizzi and A. Sarti, “Region-based PDEs for cells counting and segmentation in 3D+Time images of vertebrate early embryogenesis,” International Journal of Biomedical Imaging, vol. 2009, p. 9, 2009. [52] D. Molloy and P. Whelan, “Active-meshes,” Pattern Recognition Letters, vol. 21, no. 12, pp. 1071–1080, Nov. 2000. [53] M. Lin and D. Manocha, “Collision detection and proximity queries,” in Handbook of Discrete and Computation Geometry, J. Goodman and J. O’Rourke, Eds. Chapman & Hall/CRC, 2004, 2nd edition. [54] C. Ericson, Real-time collision detection. Morgan Kaufmann, 2005. [55] T. M¨oller and B. Trumbore, “Fast, minimum storage ray-triangle intersection,” Journal of graphics, gpu, and game tools, vol. 2, no. 1, pp. 21–28, 1997. [56] V. Caselles, “Geometric models for active contours,” in Proceedings of the IEEE International Conference on Image Processing (ICIP 1995), vol. 3, 1995, pp. 9–12. [57] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” in Proceedings of the 5th International Conference on Computer Vision (ICCV 1995), Cambridge, USA, Jun. 1995, pp. 694–699. [58] B. Li and S. T. Acton, “Automatic active model initialization via poisson inverse gradient,” IEEE Transcations on Image Processing, vol. 17, no. 8, pp. 1406–1420, Aug. 2008. [59] A. Dufour, V. Meas-Yedid, and J.-C. Olivo-Marin, “Automated quantification of cell endocytosis using active contours and wavelets,” in Proceedings of the 19th International Conference on Pattern Recognition (ICPR 2008), Tampa, FL, USA, Dec. 2008, pp. 1–4. [60] G. Treece, R. Prager, and A. Gee, “Regularised Marching Tetrahedra: improved iso-surface extraction,” Cambridge University Engineering Departement, Tech. Rep., 1998. [61] J. Weickert, B. Romeny, and M. Viergever, “Efficient and reliable schemes for nonlinear diffusion filtering,” IEEE Transactions on Image Processing, vol. 7, no. 3, pp. 398–410, Mar. 1998. [62] N. Otsu, “A threshold selection method from gray level histograms,” IEEE Transactions on Systems, Man and Cybernetics, vol. 9, pp. 62– 66, Mar. 1979. [63] L. Vincent and P. Soille, “Watersheds in digital spaces: An efficient algorithm based on immersion simulations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 6, pp. 583–598, 1991. [64] D. Adalsteinsson and J. A. Sethian, “A fast level set method for propagating interfaces,” Journal of Computational Physics, vol. 118, no. 2, pp. 269–277, 1995. [65] E. Labruy`ere, C. Zimmer, V. Galy, J.-C. Olivo-Marin, and N. Guill´en, “Ehpak, a member of the p21-activated kinase family, is involved in the control of entamoeba histolytica migration and phagocytosis,” Journal of Cell Science, vol. 116, no. Pt 1, pp. 61–71, Jan. 2003. [66] E. Coudrier, F. Amblard, C. Zimmer, P. Roux, J.-C. Olivo-Marin, M.C. Rigothier, and N. Guill´en, “Myosin ii and the gal-galnac lectin play a crucial role in tissue invasion by entamoeba histolytica.” Cell Microbiology, vol. 7, no. 1, pp. 19–27, Jan 2005.

Alexandre Dufour (born in France, 1983) received the B.S. in Computer Science in 2002 and the M.S. in Artificial Intelligence & Pattern Recognition in 2004, both from University Pierre & Marie Curie (Paris, France). He then took a Junior Scientist position at Institut Pasteur Korea (Seoul, South Korea), and obtained his Ph.D. in Image Processing in 2007 from University Ren´e Descartes (Paris, France). Since 2008 he is a Post-Doctoral research fellow at the Quantitative Image Analysis Unit, Institut Pasteur (Paris, France). He is a Member of the IEEE, the IEEE Signal Processing Society, and an Associate Member of the IEEE Bio Imaging and Signal Processing Technical Committee (BISPTC). His main research interests focus on partial differential equations and fast deformable models for segmentation and tracking in 2D/3D biological imaging, and their application to the quantification of morphological and dynamic processes at the cellular and multi-cellular level.

13

Roman Thibeaux (born in New-Caledonia, 1985) graduated with a B.Sc in Biology (2006) and a M.Sc in Genetics (2008) at University Paris Denis Diderot. He is currently achieving his PhD thesis in the Cell Biology of Parasitism Unit at the Institut Pasteur in Paris, where he focuses his work on parasitehost interactions. His work aims at understanding the motility mechanisms employed by Entamoeba histolytica to invade the human colon barrier.

Elisabeth Labruy`ere received the Ph.D. degree in Microbiology and the “Habilitation a` Diriger des Recherches” from the University Pierre et Marie Curie (Paris VI). She is a Permanent Researcher at the Institut Pasteur, Paris, France. Since she has joined the Cell Biology of Parasitism Unit her interest is to understand the cellular and molecular process sustaining Entamoeba histolytica invasion of the human colon. To this purpose, she analyses the behaviour of the parasite during its migration across the different three-dimensional environments forming the human colon barrier, studying both the amoebic factors and human components necessary for invasion.

Nancy Guill´en PhD is Research Director 1 at the National Center for Scientific Research (CNRS), France. She is Head of Cell Biology of Parasitism Unit - INSERM U786, Department of Cell Biology and Infection, Institut Pasteur. Her additional relevant activities in France summarizes as a member of the Scientific Council, Institut Pasteur, member of the Scientific Council, Animal Health Division, INRA and Scientific coordinator of the PasteurWeizmman Research Council. As a microbiologist the research domain of Nancy Guill´en concerns the cellular and molecular biology studies of the pathogenic process in Entamoeba histolytica, the agent of human amoebiasis. The keywords of her reserarch are parasitology, cytoskeleton, myosin, cell motility, transcriptome and physiopathology.

Jean-Christophe Olivo-Marin is the head of the Quantitative Image Analysis Unit, and the Chair of the Cell Biology and Infection department, at the Institut Pasteur. He holds a Ph.D. and an H.D.R. in Optics and Signal Processing from the Institut d’Optique Th´eorique et Appliqu´ee, University of Paris-Orsay, France. He was a co-founder of the Institut Pasteur Korea, Seoul, where he held a joint appointment as Chief Technology Officer from 2004 to 2005. Previous to that, he was a staff scientist from 1990 to 1998 at the European Molecular Biology Laboratory, Heidelberg. His research interests are in image analysis of multidimensional microscopy images, computer vision and motion analysis for cellular dynamics, and in multi-disciplinary approaches to biological imaging. He is a Senior Member of IEEE, a member of SPS and EMBS, Chair of the Bio Imaging and Signal Processing Technical Committee (BISP-TC), member of the Pattern Recognition Society and of the Editorial Board of the journal Medical Image Analysis. He has organized several special sessions dedicated to biological imaging at international biomedical conferences (ELMI’02, ELSO’03, ISBI’04, ICASSP’06, SPIE Wavelets’09, EMBO’11) and was General Chair of the IEEE International Symposium on Biomedical Imaging (ISBI) in 2008.