Image Segmentation based on Active Contours using ... - CiteSeerX

Proc. of Fifth IEEE Int. Workshop on Cellular Neural Networks and their Architecture (CNNA-98), London, UK, Apr. 14-17, 1998

Image Segmentation based on Active Contours using Discrete Time Cellular Neural Networks D.L Vilariño1*, D. Cabello1, M. Balsi2 and V. M. Brea1 1

Dep. of Electronics and Computer Science, University of Santiago de Compostela, Spain E-15706 phone:+ 34-81-563100 ext. 13559, fax:+34-81-549412, e-mail: eldavid, eldiego, elvictor @usc.es 2 Dep. of Electronic Engineering, "La Sapienza" University, via Eudossiana 18, Rome, Italy I-00184 phone: +39-6-44585485, fax: +39-6-4742647, e-mail: [email protected]

ABSTRACT: In this work we present a new proposal for image segmentation using deformable models, as an application of Discrete-Time Cellular Neural Networks (DTCNN) [1]. This approach is based on active contours (also called snakes) which evolve until reaching a final desired location. The contours are guided by both external information from the image under consideration which attracts them towards salient characteristics of the scene, and internal energy from the contour image which tries to maintain the smoothness in the curve shape. The massively parallel processing in DTCNN and the use of local information permit a VLSI implementation, suitable for real time applications.

1. Introduction A deformable model or snake [2], is an approximated elastic curve which, placed on an image, starts to deform from an initial form with the aim of adapting itself to the notable characteristcs of the scene. This deformation is produced by the effect of internal forces which control its stretching and flexion as well as external forces from the image being processed which attract it towards the target characteristics (intensity extremes, edges, ...) depending on each application. Classical strategies approach the problem as a minimisation of a global energy of the snake, given by 1

E snake = ∫ [ Eint (u ( s )) + E ext (u ( s ))]ds

(1)

0

Where Eint and Eext are respectively the internal and external energy of the snake and u(s)=(x(s),y(s)) represents the parametrisation of the curve. In order to calculate the numerical solution corresponding to energy minimum it is necessary to discretise the expression in (1). In classical image processing literature different procedures to approach both the discretisation and minimisation of (1) can be found. However almost all of them evolve a high computational cost, so that they are not appropriate for applications needing short response time. In [3] a new strategy for undertaking the segmentation of image on the basis of deformable models has been proposed. This consists of iterative processes of expansion of the contour and its subsequent thinning, guided by external information towards its desired final location. The design has been realised based on Multilayer DTCNN with cyclic time variable cloning templates [4] in such a way that, unlike classical strategies, in the evolution of the contour all its points contribute, in such a manner that it could be considered a continuous treatment of the contour (pixel level discretisation), allowing a higher precision in the adaptation of the contour. Later, in [5] an enhancement of this structure in [3] has been realised in order to exploit the massively parallel processing in CNN which joined with the use of local information permits a VLSI implementation, suitable for real-time applications [6]. However in those cases in which the external energy is insufficient or is corrupted by noise, anchored pixels may appear in the contour and a correct final location is not reached. A solution might be to integrate a higher neigbourhood level for the external energy. But this would suppose to complicate the subsequent hardware implementation. Also a worse fitted contour due to the smoothing effect on the shape of the energy is obtained. In this work we propose a new structure which manages internal energy, dinamically calculated which permits to avoid that conflictive situations without an increasing in the hardware complexity.

*

Visiting scholar in Department of Electronic Engineering, "La Sapienza" University, Rome, Italy.


2. Proposed structure As we have already commented, the proposal for the dynamics of the evolution of the contour is different from that put forward in classical strategies. It is realised by means of iterative procedures of expansion and guided thinning on the contour [3,5]. This two steps are repeated at each processing for each cardinal directions in the image plane cycle. The information to guide the movement of the snake must be processed from external information previously extracted from the image which contains the objective contour, as well as from internal energy which should be extracted from the contour image itself in a dynamical process. Fig. 1 shows the structure

of the processing element associated to each pixel. Figure 1: Block diagram describing the operation of each pixel of the image being processed The block EXP acts on the contour image (binary pixel image in which black pixels represent the contour location) and EP on the images of the external and internal energy (real valued arrays). Finally, TH block combines both output images from EXP and EP, in order to carry out the correct evolution of the snake. So, unlike the classical proposals in deformable models, the contour reaches a final location after a dynamic process of activation and deactivation of pixels in a contour image. Following, a description of each elementary block is carried out. 2.1 EXP block. Contour expansion stage. This block carries out a contour expansion process, for the direction under consideration. The expansion will be allowed only for one pixel in order to make the subsequent thinning process easier. This block acts on each pixel (i,j) of the binary contour image based on the equation:

y ( i , j ) (k + 1) = sgn(

∑A

(i − k + r , j −l + r ) ( k ,l )∈N r ( i , j )

y ( k ,l ) ( k ) +

∑B

(i − k + r , j −l + r ) ( k ,l )∈N r ( i , j )

u ( k ,l ) + I )

(2)

Where Nr(i,j) fixes the neighborhood topology with r a positive integer number. A template which realizes an one pixel expansion in northerly directions should be [5]:

AEXP = [0]5 x 5

BEXP

0 0  = 0  0 0

0 0 0 0 0 2 0 1 0 −1

0 0 0 0 0

0 0  0  0 0

I EXP = 1

(3)

The contour image is introduced as external input Uexp. To obtain the templates for the other directions, matrix B is suitably rotated. 2.2 TH block. Thinning stage. The following stage is a thinning operation on the expanded contour from the EXP block, guided by information processed in EP block. This operation must be carried out respecting connectivity and taking the evolution of the contour in the correct direction. The output of EP will be a binary image whose white pixels (off state) will indicate the corresponding pixels of the expanded contour image which might be deactivated according to the guide information. However in the output of TH only those authorized pixels that are not needed to maintain a continuos contour must change their state.


The dynamic equation system of this block coincides with (2). The EXP block output image is introduced as input UTH and the EP block output image as initial state YTH(0). Due the complexity of this operation, this block needs four iterations for each direction, with variable templates. Proposed templates for this operation in northerly direction are:

2 0 1 0 0 0     ATH (1) = ATH (2) = 0 6 − 1 BTH (1) = BTH (2) = 0 15 − 1 0 − 4 0  0 0 0  1 0  0 0 0 2    ATH (3) = ATH (4) = − 1 6 0 BTH (3) = BTH ( 4) = − 1 15 0      0 0 0  0 − 4 0

(4)

I TH (n) = −5

n = {1,2,3,4}

Although one pixel thinning is carried, it is realised by a massively parallel process. Therefore, breakpoints might appear when one pixel is deactived due to considere the active state of a neigbour and inversely. So, to guarantee the connectivity, several iterations are needed. The proposed templates in (4) have been designed in order to avoid this kind of ruptures in the contour. Again, the corresponding templates for the other directions are obtained by rotating those of northerly direction. 2.3 EP block. Information processing This block acts only once for each direction. It carries out a thresholded weighted sum of gradients calculated based on external and internal energy. So it will act on two external inputs Uext and Uint. Its operation is lead by the next equation:

out (i , j ) = sgn(

B ∑ (i −k +r , j −l +r ) u (extk ,l ) + λ

( k ,l )∈N r ( i , j )

∑B

( i − k + r , j −l + r ) ( k ,l )∈N r ( i , j )

u (intk ,l ) + I )

(5)

Where λ, weighs the importance of each kind of information. Its value may be constant or it may dynamically decreased from a high initial value to zero in order to allow a better fitting of the final contour. The templates and λ values will depend on the kind of the application and the quality of the information to be processed. Default values will be (for northerly direction):

0 0 0  B EP = 0 1 0   0 − 1 0

B EP

0 0 0  = 0 1 0 0 − 1 0

I EP = 0 λ = 1

(6)

That represents a comparison of energies between a pixel an its neigbour in the direction under processing (simplest gradient aproximation). 2.4 IE block. Internal energy calculus. Sometimes the external information is enough to guide the contour [3,5]. However in those cases in which the external energy is insufficient or is corrupted by noise, anchored pixels may appear in the contour, which can not reach their correct final location. So, the function of internal energy will be to smooth the contour shape in order to avoid rough deformation, so that a best approaching to the final contour is allowed. In classical approaches, the internal energy depends on the tension of the contour and it may be measured in function of distances among adjacent points according to the considered discretisation. This approach can not be directly applied to our case because the contour is not defined as a predetermined number of discretisation points but as a set of black pixels of a binary image, and its evolution is based on activation and deactivation of pixels belonging to this contour image. Also, a pixel-level discretisation with a restricted local neighborhood processing is considered, and so information from the neigbourhood considered becomes insufficient for the calculus of internal energy. However, the desired smoothing effect can be obtained by assigning a higher amount of energy for those pixels in the contour image situated into concavities, with respect to those situated outside. Thus, it is possible to


overcome the influence of external energy, and so to invert the state of the interesting pixels in the output image of the EP block. A way to carry out this process is by means of a block IE with a binary image input (contour image from TH) and continuous output, proportional to the number of active pixels in the neighbourhood considered. In Fig. 2 this idea is illustrated by a simple example.

(a) (b) Figure 2: Relation between internal energy and the number of neigbouring active pixels. For the neighbourhood considered in figure 2 (r=2) the gray pixel in Fig. 2.a presents 5 active neighbouring pixels and the one in Fig. 2.b only 3, therefore the first will have a high internal energy, as it is hoped. Thus, the internal energy appropriately weighted in EP, leads to smooth the contour shape. This influence will be higher in those locations where the contour presents a rough shape and decreases in smooth locations. The smoothness level can be controlled by the parameter λ in EP. The equation system that describes the operation of IE will be:

out (i , j ) =

∑B

(i − k + r , j −l + r ) ( k ,l )∈N r ( i , j )

u ( k ,l )

(7)

And a valid template would be

BIE

0.35 0.45  = K × 0.50  0.45 0.35

0.45 0.70 1.00 0.70 0.45

0.50 1.00 1.00 1.00 0.50

0.45 0.70 1.00 0.70 0.45

0.35 0.45  0.50  0.45 0.35

(8)

With K as a normalisation constant. As we can see the weight of the neigboring pixels in the internal energy is function of the distance (more effective than that uniformed sampled). Unlike the external information, the internal energy would be dynamically calculated after each complete cycle (where a complete cycle represents the processing along the four directions). 2.5 Global structure For a complete processing, the structure seen in Fig. 1 has to be extended in the four directions of the plane (North, South, East and West). In order to do this, one needs only to rotate the templates obtained for the northerly direction. Figure 3 shows the complete cycle processing structure associated of each pixel.

Figure 3: Block diagram for a complete one-cycle processing Each block from the cascade connection represents the one contained in the box in Fig. 1. Thinking about a possible hardware implementation, the proposed structure could be significantly reduced with the used of cyclical templates in the cascade connections. This would enable us to carry out different tasks in the same physical block by using suitable template programming (Fig. 4).


Figure 4: Reduced structure of processing

3. Results In order to show the validity of the proposed structure, we will show results of processing in artificial images as well as in real computational tomography (CT) images. 3.1 Simulation on artificial image. In Fig. 5 we show a simulated processing on an artificially created image. This consists of concentric circles of uniform value of energy (in which the maximum energy is represented by white). For the simulation, the energy image has been corrupted by Gaussian noise and we have started the processing from a rather irregular initial contour.

(a)

(b)

(c) Figure 5:a) processing only with internal energy, b) processing only with external energy and c) processing with both external and internal energy In Fig. 5, we can see the superposition of different samples of the contour evolution on the external energy image. Thus, Fig. 5.a shows the evolution with only internal energy (λ→∞) in order to show the smoothing effect. As the effect of internal energy is to smooth the concavities, its effect on a closed curve will finish only with the collapsing of the contour. However the processing sequence showed is useful to illustrate internal energy influence. Fig. 5.b, shows the evolution without internal energy (λ=0). Since the external information is corrupted with noise, locations exist through which the contour can not go away. So the contour stays anchored in these locations not being able to reach the final desired position. Finally in Fig. 5.c it is showed how the contour can be able to escape of that incorrect locations from a combination of both external and internal energy (λ=1). 3.2 CT images. Lastly, we show the application of the described structure on CT images corresponding to transversal sections of the proximal tibia. The aim of the segmentation process is that of determining the position of the exterior contour of the cortical bone of the tibia in each section, in order to obtain a solid model of the patient's tibia from 3D layering [7]. The external energy for this example has been calculated by means of the expression:

E (i, j ) = γ ∇(G (i, j ) ∗ I (i, j ) + δI (i, j ) + ζe − d ( i , j )

2

(9)


Where d(i,j) denotes the distance between the point (i,j) and the closest border. The first term of (9) corresponds to the gradient of the image convolutioned with a Gaussian in order to reduce noise; the second to the intensity and the third to the distance to the borders obtained from the zero crossing. Finally, the constants γ, δ, ζ weigh the importance of each term. Fig. 6 shows the initial and resulting contours for different simulations, overlapped to the original CT image.

(a) (b) (c) Figure 6: Superposition on the CT image of: a) the initial contour, b) final contour with λ=0, c) final contour with λ=10. In Fig. 6.a we can see that the initial contour (obtained from a previous slice) is too separate for the target location, so the external information can not guide the snake correctly towards the final desired position (Fig. 6.b). Therefore, it is necessary to introduce an internal energy term to reach a valid final position of the contour as is showed in Fig. 6c. The results may be still enhanced by using a more specified external energy in order to force the contour towards the exterior wall of the cortical bone. Finally, by using decreasing internal energy from a high value of λ towards λ=0, a more fitted final contour is possible. Also, the control input becomes constant (only external energy), and so a more stable system is accomplished, which due to the nature of the problem under consideration guarantees the convergence of its outputs.

4. Conclusions In this work we have presented a new proposal for the segmentation of images based on active contours. The processing consists of successive expansion and thinning processes, guided by both external information from the image being processed, and which produces an evolution towards the final contour to be detected, as well as internal energy dynamically obtained from the contour image. By using cellular neural networks, hardware implementation permits massively parallel processing, making the proposal valid for those applications needing high speed of response. ACKNOWLEDGMENTS: This work was supported by the Xunta de Galicia under Grant XUGA 20603B96.

5. References [1] H. Harrer and J.A. Nossek: “Discrete-Time Cellular Neural Networks”, Int. J. Circ. Th. Appl., vol. 20, pp. 453-467,1992. [2] M. Kass, A. Witkin, and D. Terzopoulos “Snakes: Active Contour Models”, Int. J. Comp. Vis., vol 1, pp. 321-331, 1988. [3] D.L. Vilariño, D. Cabello, A. Mosquera and J.M. Pardo: “Application of a Multilayer Discrete-Time CNN to Deformable Models”, in “Biological and Artificial Computation: From Neuroscience to Technology”, (J. Mira, J. Cabestany and R. Moreno-Diaz, Eds), Lect. Notes in Comp. Science, vol 1240, pp. 1193-1202, Springer-Verlag, 1997. [4] H. Harrer, "Multiple Layer Discrete-Time Cellular Neural Networks Using Time-Variant Templates", IEEE Trans.Circuits Syst. Vol. 40, N. 3, pp. 191-199, 1993. [5]D.L. Vilariño, V.M. Brea, D. Cabello and J.M. Pardo: ”Discrete-Time CNN for Image Segmentation by Active Contours”, Pattern Recognition Letters (in pres.). [6]V.M. Brea, D.L. Vilariño and D. Cabello: “Active Contours with Cellular Neural Networks: An Analog CMOS Realisation”, Signal Processing and Communications, (J. Ruiz-Alzola, Eds), pp.419-422, IASTED/ActaPress, 1998.


[7] J.M. Pardo, D. Cabello and J. Heras: “A Snake for Model-Based Segmentation of Biomedical Images”, Pattern Recognition Letters (in pres.).