COMPUTATIONAL MECHANICS New Trends and Applications S.Idelsohn,E. O˜ nate and E. Dvorkin (Eds.) c
CIMNE, Barcelona, Spain 1998
AN ADAPTIVE FINITE ELEMENT PROCEDURE FOR SEGMENTATION PROBLEMS Dan Givoli∗ and Fiana S. Yaacobson† ∗ Department
of Aerospace Engineering Technion — Israel Institute of Technology Haifa 32000, Israel e-mail:
[email protected] †Applied Mathematics Program Technion — Israel Institute of Technology Haifa 32000, Israel
Key words: Segmentation, Finite Elements, Adaptive, Image, Multiphase Continuum Mechanics Abstract. Segmentation-type problems appear in computer vision and also in multiphase solid continuum mechanics. In the former context, the problem consists in finding an “optimal” subdivision of a given image into the objects which appear in it. In other words, one has to find a set of inner boundaries which optimally define the subdomains (objects), and thus to create a sharp “cartoon” associated with the given image. In continuum mechanics, segmentation problems are encountered when a solid microstructure consists of two or more phases, and the boundaries between these phases are to be determined. The optimality criterion used here is given by the Mumford-Shah model, which leads to a strongly nonlinear problem of minimizing a non-differentiable functional. The nonlinear term involves the total length of the inner boundaries, which is related to the surface energy in the multiphase problem. An iterative procedure based on an h-adaptive finite element method is proposed for the solution of the problem. The mesh adaptivity leads to an efficient solution technique, with the use of a basic coarse discretization and a few local regions of high resolution where needed.
1
Dan Givoli and Fiana S. Yaacobson
1 INTRODUCTION The Segmentation problem in computer vision is the problem of finding an “optimal” subdivision of a given image to the objects which appear in it. Stated differently, the problem is that of identifying the edges of the objects in the given image, and generating a new image — a “cartoon” — with these edges drawn sharply and precisely.1,2 Segmentation problems arise in various important applications, such as medical CAT scanning, in which segmentation is used to identify the various organs of the body, and the deciphering of aerial photos. There are various models and approaches for the segmentation problem. Two major types of approaches are the use of pattern recognition techniques3 , based on artificial intelligence tools, and the use of variational models2 . One of the models belonging to the latter category is the model proposed and studied by Mumford and Shah1 and later by Blake and Zisserman4 . This model is described in the next section. In Ref. 1, the mathematical properties of the model are analyzed and some theorems are proved, while in Ref. 4 the emphasis is on a practical solution technique. In Refs. 2, 5 and 6, other variational models and methods are discussed. A related class of variational problems also appears in multiphase micro-mechanics. See, e.g., Refs. 7–9. In this context, segmentation problems are encountered when a solid microstructure consists of two or more phases, and the boundaries between these phases are to be determined. The nonlinear term in the Mumford-Shah model involves the total length of the inner boundaries, which is related to the surface energy in the multiphase problem. In this paper we propose an h-adaptive finite element method for the solution of the image segmentation problem based on the Mumford-Shah model. The mesh adaptivity enables an efficient solution technique, with the use of a basic coarse discretization and a few local regions of high resolution (possibly up to pixel length-scale) only where needed. More details are given in Ref. 10. 2 STATEMENT OF THE PROBLEM Let D be a two-dimensional domain (usually a rectangle). Let g be a given function defined on D. This is the original image: g(x, y) represents the strength of the light signal at point (x, y). The image segmentation problem consists of finding a decomposition of D into N subdomains (objects), D = D1 ∪ D2 ∪ ... ∪ DN ,
(1)
and a new function (image) u(x, y) on D, with the following properties (expressed first very roughly): (a) The image u is “similar” to the original image g. (b) The image u varies “smoothly” within each subdomain Dn . (c) The image u has jump-discontinuities across the boundary between neighboring Dn .
2
Dan Givoli and Fiana S. Yaacobson
(d) The total length of the boundaries between the Dn is not “too large.” A decomposition of the image into subdomains Dn generates a “cartoon” out of the original image g, which sharply identifies the boundaries between the objects that appear in the image. There are various models that achieve this general goal, with different emphases of each of the listed properties2,3 . Here we consider the Mumford-Shah variational model1,4 . Let Γ denote the outer boundary of D. Given a decomposition of D into open subsets Dn , let γ denote the union of all the inner boundaries of the Dn . Thus,
γ = ∪N n=1 ∂Dn − Γ .
(2)
It is assumed that γ is piecewise smooth. Let u be a function which is differential in D − γ and is allowed to be discontinuous across γ. Now, define the “energy functional” Z
E[u, γ] =
D
(u − g)2 dx dy + α1
Z D−γ
|∇u|2 dx dy + α2 `(γ) .
(3)
Here α1 and α2 are given positive constants, and `(γ) is the total length of γ. Then the segmentation problem is: Find the function u(x, y) and the union of subdomain boundaries γ such that E[u, γ] attains a minimum. The interpretation of (3) in view of properties (a)–(d) above is clear. The first term in (3) asks that the new image be an approximation of the original image. The second term asks that u does not vary very much in each Dn . The third term asks that the boundaries γ that define the decomposition be short. The α1 and α2 are given weights, which are determined empirically for a certain class of images (e.g., aerial photos of specific type), and generate the desired balance between properties (a), (b) and (d). It should be remarked that an energy functional similar to (3) appears in certain models of multiphase continuum mechanics; the third term in (3) is analogous to surface energy. 7,9 The problem of minimizing the energy E defined by (3) is strongly nonlinear. What makes it complicated is the dependence of E on the unknown boundary γ. If γ is known, the problem of minimizing E[u] becomes linear. This observation constitutes the basis for the numerical scheme used in this paper. It can be shown 10 that the EulerLagrange equation corresponding to the linear problem is an inhomogeneous modifiedHelmholtz equation. This equation is accompanied by natural boundary conditions on the boundary γ. See Ref. 10 for more details. 3 SOLUTION PROCEDURE Now we discuss the solution procedure proposed for the nonlinear problem of minimizing E[u, γ] defined by (3). The basic idea is to guess γ, solve the resulting linear problem for u by the finite element method, use this solution to update γ and also to adapt the mesh, solve the linear problem again, and so on in an iterative manner.
3
Dan Givoli and Fiana S. Yaacobson
The scheme can be described in more detail as follows. We start by assuming that γ = ∅, i.e. there are no discontinuities. Then the problem reduces to a linear problem. We solve it by the finite element method using a uniform coarse mesh of bilinear rectangular elements, and we find an approximation uh to u. Based on this solution, we then use some criterion, to be described later, for identifying elements which are suspected of including a piece of the boundary γ in them, namely of including jump discontinuities in u. We adopt here the Blake-Zisserman idea4 of assigning a binary “indicator” to each element, to be denoted I e , which indicates whether the element includes a discontinuity (I e = 1) or not (I e = 0). We prepare a list L of all the elements whose indicator is 1. Next, we refine those elements in list L, by dividing each of them into four smaller rectangular elements. This immediately necessitates modifying the mesh locally around these refined elements in order to maintain element conformity (C 0 continuity). Each child element is assigned the indicator 1, like its parent, and the list L is modified accordingly. Then we construct γ using list L, and solve the linear problem again. Then we use the new uh and the discontinuity criterion to identify those elements whose indicator is 1, and we update list L. We keep iterating in this manner until the stopping criterion is met. This criterion consists of two conditions: (a) the list L in the current iteration is the same as that of the previous iteration, and (b) the size of the smallest element in the mesh is below a prescribed value. As mentioned previously, the finite element solution must be in C 0 (D − γ), and must be allowed to possess discontinuities across γ. To achieve this, we use a procedure similar to that used in Ref. 4. The resulting expression for the element stiffness matrix is: 10 Z Z e e kab = α1 (1 − I ) e ∇Na · ∇Nb dΩ + e Na Nb dΩ . (4) Ω
Ω
Here Ωe is the domain of element e, and Na is the shape function associated with element node a. The discontinuity criterion, which determines, based on the solution uh and list L of the last iteration, whether an element is assigned the indicator 0 or 1, is as follows. We consider a small patch of elements around the element e under consideration. We denote this patch by p(e), and the number of elements in this patch by N p(e) . Now, we compute the total energy of this patch based on the current solution uh and current list L, by (cf. (3)) E p(e)[uh , L] =
p(e) Z NX
e˜=1
Ωe˜
(uh − g)2 dx dy + α1 (1 − I e˜)
Z Ωe˜
h 2 e ˜ e ˜ |∇u | dx dy + α2 I h .
(5) Here Ωe˜ is the domain of element e˜, and he˜ is the length of element e˜. If we denote the indicator of element e itself in the last iteration by Iˆe , then (5) is the energy of the patch assuming I e = Iˆe . Now we check the opposite assumption, namely I e = 1 − Iˆe . To this end, we set I e = 1 − Iˆe and solve the miniature linear minimization problem
4
Dan Givoli and Fiana S. Yaacobson
in the patch p(e), and then compute the energy (5) for this case. We compare the two energies, and choose the new indicator I e as the one that gives the lower patch energy. 4 MESH ADAPTATION Mesh adaptivity is performed based on the use of four-node bilinear rectangular elements, and transition elements of the type proposed in Ref. 11. An element which is to be refined is subdivided into four bilinear elements. Then neighboring elements are replaced by transition elements so as to maintain C 0 continuity. An element in the mesh may have a number of nodes ranging from 4 to 7. The shape functions of these elements are simple to derive. 11 More details and other related aspects of h−adaptivity are discussed in Refs. 10–12. 5 NUMERICAL EXAMPLE In the example shown here, we use a square domain D, and the parameters α1 = 20 and α2 = 10 in the energy functional (3). The input is an image obtained by a biomedical imaging system (courtesy Dr. H. Azhari and Prof. J. Rubinstein) and describes a gasfilled closed membrane immersed in fluid. Fig. 1 shows the final mesh obtained after four adaptive steps. In this case, the solution process terminated when the list L of discontinuous elements remained unchanged during two consecutive iterations. For more information and other numerical examples, see Ref. 10. REFERENCES 1. D. Mumford and J. Shah, “Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems,” Comm. Pure Appl. Math., 42, 577–685 (1989). 2. J.-M. Morel and S. Solimini, Variational Methods in Image Segmentation, Birkhauser, Boston, 1995. 3. J.C. Bezdek, L.O. Hall and L.P. Clarke, “Review of MR image segmentation techniques using pattern recognition,” Medical physics, 20, 1033–1043 (1993). 4. A. Blake and A. Zisserman, Visual Reconstruction, MIT Press, Cambridge, MA, 1987. 5. D. Terzopoulos, “Multilevel Computational Processes for Visual Surface Reconstruction,” Computer Vision Graphics & Image Processing, 24, 52–96 (1983). 6. L. Ambrosio and V.M. Tortorelli, “Approximation of Functionals Depending on Jumps by Elliptic Functionals via Gamma-Convergence,” Comm. Pure Appl. Math., 43, 999–1036 (1990). 7. M.E. Gurtin, “On a Theory of Phase Transitions with Interfacial Energy,” Archive Rat. Mech. Anal., 87, 187–212 (1985). 8. J. Goodman, R.V. Kohn and L. Reyna, “Numerical Study of a Relaxed Variational Problem from Optimal Design,” Comp. Meth. Appl. Mech. Engng., 57, 107–127 (1986).
5
Dan Givoli and Fiana S. Yaacobson
Figure 1. The biomedical example: the final 6 mesh obtained after four adaptive steps
Dan Givoli and Fiana S. Yaacobson
9. M. Luskin, “On the Computation of Crystalline Microstructures,” Acta Numerica, 5, 191–257 (1996). 10. F.S. Yaacobson and D. Givoli, “An Adaptive Finite Element Procedure for the Image Segmentation Problem,” Commun. Numerical Meth. Engng., to appear. 11. J.M. McDill, J.A. Goldak, A.S. Oddy and M.J. Bilby, “Isoparametric Quadrilaterals and Hexahedrons for Mesh-grading Algorithms,” Commun. Appl. Num. Meth., 3, 155–166 (1987). 12. D. Givoli, J.E. Flaherty and M.S. Shephard, “Simulation of Czochralski Melt Flows Using Parallel Adaptive Finite Element Procedures,” Modelling Simul. Mater. Sci. Eng., 4, 623–639 (1996).
7
