429
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5 , NO. 3, MARCH 1996
Nonlinear Image Labeling for Multivalued Segmentation U
U
Silvana G. Dellepiane, Member, IEEE, Franco Fontana, and Gianni L. Vemazza, Senior Member, IEEE
Abstract- In this paper, we describe a framework for multivalued segmentation and demonstrate that some of the problems
affecting common region-based algorithms can be overcome by integrating statistical and topological methods in a nonlinear fashion. We address sensitivity to parameter setting, the difficulty with handling global contextual information, and the dependence of results on analysis order and on initial conditions. We develop our method within a theoretical framework and resort to the definition of image segmentation as an estimation problem. We show that, thanks to an adaptive image scanning mechanism, there is no need of iterations to propagate a global context efficiently. The keyword multivalued refers to a result property, which spans over a set of solutions. The advantage is twofold: first, there is no necessity for setting a priori input thresholds; secondly, we are able to cope successfully with the problem of uncertainties in the signal model. To this end, we adopt a modified version of fuzzy connectedness, which proves particularly useful to account for densitometric and topological information simultaneously. The algorithm was tested on several synthetic and real images. Peculiarities of the method are assessed both qualitatively and quantitatively.
I. INTRODUCTION
I
N RECENT YEARS, three classes of techniques for image processing have been particularly improved; we attempt to integrate them to overcome the usual drawbacks of region-based segmentation methods. Statistical, contextual signal characteristics and a priori knowledge are exploited in the Bayesian framework, morphological aspects are taken into account by mathematical morphology (MM) and related methods, and fuzzy techniques handle model uncertainties and represent a useful tool for describing topological features. The proposed adaptive, nonlinear processing method for multivalue region growing overcomes the drawbacks of dependence on analysis order and on initial conditions, of inability to take into account global information, and of sensitivity to parameters. The segmentation problem is reduced to a domainindependent estimation problem, suitable for application to real images of the “bird’s eye view” type. Our noniterative approach is based on a selective growing mechanism and regards an image as a time-series model made up of sequences of random variables, thus avoiding the time and convergence problems typical for iterative methods. Each sequence is analyzed by a first-order nonlinear recursive operator. Manuscript received March 18, 1994; revised May 30, 1995. The associate editor coordinating the review of this paper and approving it for publication was Prof. Patrick A. Kelly. The authors are with the Department of Biophysical and Electronic Engineering, University of Genoa, Genoa, Italy (e-mail:
[email protected]). Publisher Item Identifier S 1057-7149(96)01795-2.
Accordingly, the method is implemented by performing two main processes-the image scanning or growing process and the actual pixel-labeling process. The former decides on the candidate pixels to be analyzed (on the basis of already labeled pixels) and results in the selection of the best paths starting from a fixed seed point. The latter assigns a label to each candidate pixel, in accordance with neighboring points and with the seed point. Second-order statistical features are exploited to account for local and global information. To this end, a modified version of fuzzy connectivity, derived from the classical definition by Rosenfeld [l] and called intensity connectedness, is introduced. This measure allows one to use topology and intensity information simultaneously. As a result, the 2-D image-labeling process is reduced to a sequence of 1-D processes, that is, the image is divided into a set of 1-D signals, each corresponding to the best path from a generic pixel to the seed point. Without setting any parameter and threshold values, we obtain a multivalue segmentation result that compensates for the lack of an objective description of the segmentation. Sets of possible regions and contours of an object, named isoregions and isocontours, are derived from the labeled image. Among these, the most appropriate may be selected either manually (during a semi-interactive session) or automatically, for instance, by exploiting the domain model contained in a knowledge-based system [2]. To sum up, the main innovative aspects of the paper are as follows: multivalue characteristics of segmentation results, obtained independently of parameter and threshold values; reduction of image processing to a set of 1-D processes while preserving topological relations; integration of the statistical approach with fuzzy techniques able to handle also topological information; modified version of fuzzy connectivity to take into account topological and densitometric features at the same time; innovative image-scanning mechanism that, being adaptive to the image content, allows a noniterative approach to account for contextual information; good quality of results and the robustness and the speed of the method, as proved by the results obtained on real images. The paper is organized as follows. After citing works on the topics addressed in the paper, a modification to the classical fuzzy connectedness formula is proposed. The two main tasks
1057-7149/96$05.00 0 1996 IEEE
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
430
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5, NO. 3, MARCH 1996
(i.e., image scanning and image labeling) are then described. Before presenting experimental results, the implementation details and strategies are discussed on the basis of quantitative evaluation tests performed on appropriate synthetic images. Finally, the Appendix briefly explains how the proposed approach could be extended to multiseed applications.
using contextual iterative methods; specific models have been adopted to describe interactions among image pixels in 2-D signal space. In Markov random field (MRF) approaches, this is done by means of specific spatial configurations of cliques [ 101, [ 111. Probabilistic relaxation methods involve the use of compatibility functions that have been ad-hoc designed to embed contextual information, which is used at each iteration step to update current sample probabilities [12]. Although considerable efforts have been expended in the past years to reduce the heuristical components in the aforesaid methods [13], [14], it seems that more promising results have been obtained on edge-based analysis than region segmentation, since, in general, the model of pixel interactions and object classes are easier to describe in an edge context than in a region context. In addition, the estimation of initial probabilities and the problem of convergence to acceptable results still represent common bottlenecks for such approaches. It is interesting to note that, in the present paper, we adopt the framework of Bayes’ theory but develop a noniterative procedure. However, the results obtained by the proposed approach have much in common with those obtained by relaxation and MRF methods. The requirement for including morphological aspects in image processing was met by MM methods [lS]. They regard grey-scale images as topographic reliefs. If topological tools, like watershed transformation, first proposed in 1977 [16], are combined with morphological tools, they constitute the basis for extremely powerful segmentation techniques [ 171. MM uses contextual and topographical information but does not exploit statistical information (unlike the Bayesian approach), nor does it seem appropriate for uncertainty handling. In this respect, a fuzzy framework can cope very successfully with the lack of precise model assumptions by extending and applying methods typical for pattern recognition [ 181, [19] to image processing. It also represents a useful tool for extending some concepts of digital image geometry to fuzzy subsets [20] (e.g., such topological concepts as surroundness, compactness, and connectedness). To prove analogies between MM and fuzzy methods, the equivalence between fuzzy connectivity and connection cost was recently shown [21]. However, it should be noted that these tools are usually adopted to evaluate fuzzy images rather than develop actual processing tools. The very few approaches so far proposed are thresholding or multithresholding techniques, which are inappropriate for complex segmentation problems. In the present paper, the proposed process is essentially a region-growing process. The only assumptions used for the model concern the interactions among the available information sources and a statistical distribution of the parameters used to describe region properties. Contextual information is exploited to describe the interactions in a local pixel neighborhood and the relationship between a pixel and a reference signal sample. Such information is propagated as global information by the selective image-scanning process. The advantage of this mechanism is twofold: first, it does not depend on the order of analysis; second, it is a robust, noniterative procedure that sharply reduces computation time and maintains a low error rate.
11. RELATED WORKS The success of a class of image-segmentation techniques over other classes has been alternate in the last few years. After a boom of region-based methods until the mid-1980’s [3], there followed a period in which edge-based techniques seemed to predominate. The mainly heuristic nature of region-growing approaches has certainly been one of the reasons for their decline in recent years. Most of criticisms concern the dependence of final results on both initial conditions (e.g., the number and the positions of starting points) and the analysis order. Moreover, sensitivity to thresholds and parameters makes inefficient the use of the same setting for different classes of images. Parameter tuning often requires annoying trial-and-error experiments, even though it is basic to the success of an algorithm. At present, statistical and region-based methods for image segmentation are receiving renewed interest, and notable effort is being devoted to designing algorithms that follow more theoretical criteria. As mentioned in the introduction, the segmentation process may benefit from the advantages of three different classes of image processing techniques. The traditional Bayesian framework is very often used to obtain image transformations, corresponding to classification or segmentation results. The image segmentation task is thus reduced to a decision or estimation problem [4]. The methods based on this framework assume prior statistical distributions for the data being estimated and model the image-formation and sensing phenomena as stochastic or noisy processes. Such supervised techniques prove very effective in reducing segmentation errors. The most recent improvements in the Bayesian approach aim to solve the general problem of uncertainty handling. To this end, new methods have been proposed to attain two main goals: solving the problem of incomplete data [SI and assessing the quality of results. In the context of the former goal, unsupervised techniques have been designed to perform model parameter estimation. The simplest techniques [ 6 ] ,[7] estimate model parameters by using image sliding windows, or by an initial segmentation process, and the most elaborate schemes [8], [9] are based on an iterative segmentation-estimation paradigm. The approach proposed by Szeliski in [4] fully exploits the Bayesian approach capabilities to study the properties of prior models and to better characterize the sensors used. It evaluates the quality of results by measuring uncertainties in the computed estimate. However, the inability of first-order statistical methods to take into account also spatial relationships among data has led to severe drawbacks in a variety of applications. Attempts at overcoming these limitations have been made by
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
43 1
DELLEPIANE et ul.: NONLINEAR IMAGE LABELING FOR MULTIVALUED SEGMENTATION
As regards the traditional controversy about probabilistic and fuzzy approaches [22], [23],it is worth clarifying the use of tools typical for such approaches. The Bayesian approach allows us to reduce the segmentation problem to an estimation problem, and fuzzy terminology is exploited, as it seems very suitable for describing some properties of image topology. The uncertain concept of homogeneity can be described by a probability density function as well as by a fuzzy membership function, as will be shown by the definition of the labeling function, discussed in Section VI-A. Therefore, in the present case, fuzzy and probabilistic approaches represent two different formalisms for a single image-segmentation framework. The multivalue result obtained by the proposed labeling process can be indifferently regarded as a probabilistic or a fuzzy-segmentation result. In this paper we use a single seed, so the two interpretations may be equally valid, as they would be in relaxation labeling. In a multiseed extension, one of the two approaches will have to be chosen in order to combine the labels related to different seeds. Some further ideas are outlined in the Appendix. 111. METHODOVERVIEW A. Multivalue Segmentation, Isoregions, and Isocontours Let Z = { As an example, Fig. ](a) shows a simple fuzzy field characterized by the presence of three membership values. It may indifferently be interpreted as a fuzzy field M or as a fuzzy field H derived from a grey-level image. In the latter case, the where P ( q , p ) is an eight-connected sequence of points from image represents a small bright object contained in a larger a pixel q to a pixel p . For each pixel p, its degree of medium-grey object. The seed a is placed inside the smaller internal region. In connectedness e/, to a in the fuzzy field M = { p ( p ) } is this case, the field H and the field X" are exactly the same, obtained as its membership in rz(p). In image segmentation, when we look for bright and con- as the seed is placed at a top at level one. Two paths are shown that link the seed to a point p nected regions, the connected components associated with image tops easily relate to a fuzzy segmentation result. This belonging to the same object (i.e., connected region) of the holds if we apply (3.1) directly to the fuzzy field H == { q ( p ) } seed. Path (1) is the minimum path according to the Euclidean that we can derive from the original image field Z by finding, distance; the fuzzy value of each of its points with respect to as suggested in [l], a rational value T such that, for each a is given in Fig. l(b). Under the "absurd" hypothesis that pixel p , we can write q ( p ) = T . C(p), and ~ ( p is) within path (1) is the optimum path from a to p , the degree of the range [0,1]. Such a transformation may also be seen as connectedness of the point p should be equal to 0.8. In this a normalization operator that allows us to pass from integer example, the values of fuzzy connectedness and of intensity grey levels, ranging from zero to a maximum value G, to real connectedness for each point with respect to the seed are the numbers ranging from zero to one. Obviously, the rational same, as the fuzzy and the X" fields are exactly the same. value I- turns out to be equal to 1/G. The fuzzy (intensity) values of the points belonging to path Similarly, by applying the connectedness formula to the field (2) are given in Fig. l(c). As can be seen, the connectedness H, complement of the field H, the darkest objects can be value of this path is higher than the previous path and is equal
+
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
433
DELLEPIANE et al.: NONLINEAR IMAGE LABELING FOR MULTIVALUED SEGMENTATION
H=X"
PP U
(g)
p
(h)
pixel
;(*
p
U
pixe
(0
(b) Fuzzy and intensity levels along Fig. 1. One-dimensional representation of the concept of intensity connectedness. (a) Fuzzy field H is equal to field Xa. path (1) from a to p . ( c ) Fuzzy and intensity levels along path (2) from a to p . (d) A fuzzy field. (e) Fuzzy levels along path (1). (f) Fuzzy levels along path (2). (8) The field X a derived from the fuzzy field in (d). (h) Intensity values along path (1). (i) Intensity values along path (2).
to one due to the fact that the path lies inside a completely homogeneous region. Fig. l(d) presents a similar situation, but now the smaller region included in the larger one is characterized by a darker grey level. The seed is placed on the same coordinates as before. In this case, the seed is not placed at a top and is characterized by a value 0.8. As a consequence, the field H and the field X u (shown in Fig. l(g)) are not equivalent any more and the fuzzy connectedness and the X-connectedness are not equivalent. The fuzzy values for path (1) are now given in Fig. l(e), and, for path (2), in Fig. 1(f). In this situation, the point p has a degree of fuzzy connectedness equal to 0.8 along both paths. As a consequence, this measure does not allow us to distinguish between the two different situations in which 1) the path is completely contained in the homogeneous region pointed by the seed, and 2) the path passes outside such a region. The behaviors of intensity-connectedness along the two paths are shown in Fig. l(h) and (i). Note that path (1) has
a degree of connectedness equal to 0.8, whereas path (2) has a value of one; this stresses again the fact that, as in the first example, path ( 2 ) is better than path (1) in connectedness terms. As regards topology, the situations of the two paths depicted in Fig. l(a) and (d) are in fact completely analogous, as path (2) is entirely contained in a homogeneous region. As a consequence, we consider it more correct to obtain the same connectivity value as achieved by the X-connectedness formula than to use the classical fuzzy formula. Path (1) in Fig. l(a) is said to be homologous to path (1) in Fig. l(d), and the same holds for path (2). Incidentally, it is worth noting that, after the transformation of the field H into the field X", the two situations are very similar. The two X u fields are the same, except for the background area, which increases up to level 0.2 in the second case, as an effect of its similarity to the seed point value. The change from fuzzy- to intensity-connectedness makes the new measure symmetric, weakly reflexive, and transitive.
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
434
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5 , NO. 3, MARCH 1996
As an important property, it is interesting to note that xconnectedness is invariant to any change in the q axis, as shown in the previous example. It follows that xconnectedness is a measure (in the measure theory sense), as it represents an intrinsic property of the set of points belonging to P ( n , p ) . We refer the reader to [25] for a detailed description of the properties of the proposed X-connectedness.
thanks to the fact that global contextual information becomes available at later processing steps. Obviously, this gives rise to a combinatorial explosion. In order to obtain equivalent results while avoiding the use of iterative methods, a selective image-scanning mechanism is proposed here. Starting from a seed point, all image sites are successively analyzed at recursion steps organized into a sequential order dependent on the image content. This adaptive procedure performs selective growing steps based on the computed intermediate pixel values. The order of site visits leads to the generation of a tree whose root is the seed point and where a node is instantiated for each analyzed image site. The selective image-scanning mechanism is described in the following, independent of the actual labeling criteria used, which will be defined in Section V. The same scanning mechanism can be applied jointly with different labeling criteria, or for tasks other than segmentation.
V. REDUCINGIMAGEPROCESSING TO 1-D PROCESSING
The random variables of interest include the observed data Z (the image) and the state C (not observable), which represents the labeled image, that is, the result we aim to obtain. A method for state estimation is said to be recursive if the state at recursion k , X I , , for given observations till the kth instant, is a function of the previous states, as well as of the current and previous observations. Moreover, a method the A. Label-Driven Scanning Mechanism is iterative if, during the estimation of the new state XI;, previous state estimation C k - 1 may be updated. It follows that An initialization process generates a seed region by assignthe state of an element (in our case, the label of an image site) ing a label value equal to one to the seed point a and to its estimated at a given instant can be changed at the next instant. direct neighbors. Then propagation through the neighborhoods Obviously, iterative approaches perform various visits to occurs in successive recursion steps. At each step, only some a single image site by evaluating and changing the value pixels are selected from among the already processed ones. assigned to that site. Results do not usually depend on the se- These are called generators and their role is to propagate quential order of such visits thanks to the processing repetition the processing by activating their not-yet-analyzed neighbors. and to the casuality of the optimization methods adopted. New sites are therefore analyzed only after they have been On the contrary, when a noniterative algorithm is applied, activated by their already processed neighbors, and this is each image site is processed only once. The sequential order how the propagation mechanism goes on, once the scanning can be organized into recursions. Therefore, at each recursion mechanism has been started. step, a subset of elements to be analyzed is simultaneously More formally, at the instant k , the state XI,represents the considered and processed on the basis of the results provided state of the previously processed pixels, related to all nodes, by the computations at previous recursion steps. The state of from the root to the leaves. each element is then estimated on the basis of the current At each step, the leaves in the tree correspond to sites state, which includes previously computed states and past already assigned but not yet expanded, that is, sites whose observations. neighbors may have not yet been processed. At each step, the In such an approach, the sequential ordering followed in current highest membership value associated with the leaves the organization of recursion steps turns out to be extremely is denoted by. , , ,A delicate, as elements processed earlier represent the past for The J k leaves characterized by the label value, , ,A and the elements processed at a later stage. denoted by { y l ~ , ~ }j , = I, .Jk are selected as generators Contextual methods for image processing (e.g., relaxation and expanded first. The leaves with lower label values are labeling and MRF’s) are applied via iterative algorithms; they temporarily suspended, while the updated, , ,A value prorepeat the assignment of a value to a point according to the gressively decreases so that each lower value will become the current state of its local neighborhood, which continuously current highest value. changes during the iterations. Therefore, a global context can The new observations, s21, (also called “candidate pixels”), influence the processing result through local computations, that are assigned at the recursion step k include the Lh: thus giving rise to severe convergence and computation-time unprocessed neighbors of the generators problems. For the present approach, as we need to take into account both local and global contextual information, we might also adopt an iterative method. The contextual labeling problem is comparable to the problem of finding the shortest path from where N ( p ) denotes the neighbors of the point p . one point to another on a network. It follows that, starting Such a mechanism allows one to consider every pixel, as from the first node, the process looking for the correct solution the stopping condition is the absence of candidate pixels. moves to each neighbor by computing the cost of the new Let us explain this mechanism by way of an example. In path from the starting point, taking into account all previous Fig. 2(a), the colored squares represent the labels assigned computations. The analysis is repeated at each node, and more to the already processed pixels, hence they correspond to correct evaluations can be obtained as processing goes on the current state CI; at the recursion step k. The white
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
DELLEPIANE ef al.: NONLINEAR IMAGE LABELING FOR MULTIVALUED SEGMENTATION
displayed together with only two of the leaves characterized by lower values. The central leaf, denoted by ~ k , 1 ,is the only generator at step k and is also indicated with a cross in Fig. 2(c). It propagates the processing by activating its not-yet, ~ Jwk,3}, , represented by analyzed neighbors, namely, { w k , ~ W three circles in Fig. 2(c). By using a first-order recursive formula, the state of the single observed (candidate) pixel W A , is ~ derived independently of the others at the same recursion step, as a function of its measures and of the past state. More precisely, the state of the new observation is a generic functional F :
k state
(a) k candidature
. -
Yk.1qenerator at steD k N(Yk.i)=(W,l,W.2, Wd) candidate pixels
,
I I
assigned label values
(e)
(4 k+l candidature
k+l assignment
(h)
43s
0)
Fig. 2. Example of application of the selective image-scanning mechanism. are considered At each step, the neighbors of the leaves characterized by ,A,, as candidates.
squares represent not-yet-assigned pixels. The figure shows eight darker sites, which have been assigned a certain label value, and six lighter sites, with a higher label value. The tree representing such a situation includes seven leaves (we exclude the leaves that cannot be expanded) corresponding to the squares in the second row from the top of the displayed matrix. One of these leaves is associated with a higher label value than those of the other leaves. In Fig. 2(b), this leaf is
Such a functional takes into account how the intrinsic measures of the candidate site, as well as its context, influence the estimation of the candidate state. Various criteria for its implementation may be chosen according to the assumed model for image representation and to the processing goal. Section V explains the decisions made for the implementation of the present labeling task. The particular choices described and experimentally evaluated have been made according to a specific criterion for the definition of the functional F ; the criterion is realized thrpugh a so-called labeling function. Similarly, the features used to describe the observation and the state influence the behavior of the labeling function, and so do the relative weights given to the intrinsic measures with respect to the contextual measures. As in the Markov model, the local neighborhood of a pixel is considered to be sufficient to describe the context. However, unlike the MRF approach, which implements a noncausal model, here we deal with a representation of the causal type. Therefore, only already processed pixels may influence the assignment of the current observation label, as they represent its past. A current observation may have various generators and, in addition, other neighbors already processed. All these sites, belonging to the past of the current candidate, may eventually lead to different values of the assigned label. In the present approach, we assume that only one of the past neighbors is sufficient to assign the new label value. In doubtful cases, such a neighbor corresponds to the one that causes the minimum label-value reduction. As a consequence, the state of the new observation turns out to depend on the state U* of the selected neighbor w*,that is,
In the following, we show that, due to the specific choices of the functional F in the present implementation, propagating a label value higher than that propagated by the generators is not always possible for neighbors other than the generators themselves. Going back to the example, the candidate site wk+l,:, in Fig. 2(f) has been activated by the generator y k + l , z . However, it may receive a higher label value, assigned on the basis of one of its past neighbors different from the generator yk+1.2. If this condition holds, the link from y k + l , z to W k + 1 , 5 should be
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
436
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5, NO. 3, MARCH 1996
removed, and a new link should connect the new observation with the actual father, from which the higher label value has been inherited, as shown in Fig. 2(h). After assignment, links are unambiguous, that is, each assigned node has only one link (i.e., the one to the father) despite the presence of several active generators. This points out that propagation is a one-to-one relationship.
labeling process is therefore performed on the basis of such a past neighbor, which fully exploits, in an adaptive way, the contextual information contained in the image at the local and global levels. An intrinsic drawback of the proposed approach is the bias effect due to the seed characteristics. Such an effect turns out to represent an advantage for the present task of labeling pixels belonging to a searched object, especially when contrast is low and objects are not clearly visible. At each recursion step, the observations are processed independently of one another, taking into account only the past labels. Therefore, dependence on the order of analysis does not concern the sequential order in which single observations are examined at a single recursion step, but refers only to the history of each pixel. As the selected neighbor becomes the parent of the current pixel, independent of the pixel or pixels that led to choose the new pixel as a candidate, the actual parent might even turn out not to be characterized by ,a, ,A value. This method does not ensure the detection of the optimum path under any condition; nevertheless, a suboptimal path is always found. The optimum path is detected when the selected neighbor of a pixel is actually the true parent of the pixel itself, that 3 , the neighbor pixel that gives the largest label value, whether it has already been labeled or not. If there is only one neighbor that is not included in the current pixel memory, an error occurs. In this case, the smaller the label than the true value, the larger the error. In cases where the path from a pixel to the seed is very complex, or where points belonging to a searched object are connected by very thin paths, a situation may occur in which the true parent of a pixel turns out not to have been processed yet. In this case, an error is very likely to affect that pixel only, as the small label value assigned to the current pixel causes the growing process to be suspended. The growing process will continue to propagate (together with the error) only when the current , , ,A becomes equal to the wrong label. In conclusion, an error always decreases a true label value. We shall experimentally prove that the number of such propagation errors is very limited, even in very noisy situations. The above considerations point out that the proposed imagescanning mechanism constitutes a good compromise to avoid using iterative methods. Similarities between the present growing approach and traditional relaxation schemes were outlined in [26].
B. Characteristics of the Scanning Mechanism As a side effect of the performed processing, each tree path from the root to a leaf corresponds to a sequence of random variables. Each sequence may share pixels with other sequences. It turns out that the actual data analysis deals with the behaviors of the intensity levels along the 1-D sequences generated. Each sequence maintains all 2-D information about previous observations and previous states provided that it corresponds to the best path from the seed (in terms of intensity connectedness). Errors occur only when this condition is not satisfied. At the point where the selected neighbor is identified, also the sequence d is identified among the active 1-D sequences generated by the scanning mechanism. In such a sequence, the selected neighbor has the index ig. This observation represents the parent of the current observation sample u f i , ~which , is added to the sequence, thus taking on the index 2.9 1. From now on, the observation will then be called W k , l or w::' or wa+', and its state will be accordingly denoted by a k , l or & +1 or 0 2 + ~ .
+
As a label is assigned to the pixel according to the probability functions defined in Section 11-B, in the following we shall refer to f l ( n = as fl(ai+'). As a general advantage, the choice of a single neighbor and the assignment of a new label according to this neighbor instead of exploiting all the neighborhood information allow the detection of even very thin structures. As regards the possibility of using as much contextual information as possible, the present approach differs markedly from the more classical contextual approach. It is correct to say that the MRF approach makes use of more contextual information in the sense that it takes into account all neighbors during the formation of cliques. This represents a considerable advantage for large region detection, as more image sites give greater stability to the processing result. However, this advantage turns out to be a severe drawback, too, as it represents the major source of errors when one deals with border areas or with very thin structures that are very often missed or confused. On the contrary, we believe that in these situations contextual information plays a fundamental role and should be exploited. Spatial-topological relationships must therefore be considered, for instance, to select the subset of neighbors that carry correct information and that must be used for the actual processing. In contrast to the MRF approach, where neighbors are blindly exploited for the formation of cliques, the proposed method selects, as a local context, only those neighbors that are assumed to carry all necessary information. In particular, each image site has a selected neighbor that includes all contextual information relating to the seed in its estimated state. The
at:')
VI. THE LABELING PROCESS A. The Labeling Formula
The label A i = A(w2) is related to the probability for the forthcoming sample w z (located at the ith position in the sequence S) to belong to the object identified by the seed point. Under the hypothesis H I and given the seed point a,A; is expressed by the value of the likelihood function f l ( a z ) where mi denotes the state of the sample under analysis. As the whole processing starts from the knowledge of the seed point, the conditional probability f l ( o i ) must actually be regarded as a probability conditioned by the event a,f l ( a i 1 a).
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
431
DELLEPIANE et al.: NONLINEAR IMAGE LABELING FOR MULTIVALUED SEGMENTATION
where ~ " ( i5)c x a ( i - 1) and reduces to cxa(z - 1) - x " ( i ) in the other case, the last term being always positive. This measure is therefore very easy to extract from an image after an initial transformation has been performed to obtain the field X" from the original image field. Then the function fi (wz I a ) may be estimated through a function of the parameter A+, namely, g ( A c x a ) ,which, in the following, Xz = Sl(O*,w z I a ) = f l ( Q *I a ) . fl(J I c*,a ) will be referred to as the labeling function. = X,-1 . f l ( w z I o*,a ) . The behavior of this function is such that small differences (5.1) in connectedness indicate a good probability for two pixels to We denote the label of selected neighbor by X,-1 = fl(o*I belong to the same object. On the contrary, large differences a ) , for it is located at the i - 1position in the same sequence. It indicate that two pixels do not belong to the same object. can be noted that both local and global contexts are considered However, no threshold is preset, but continuous decreasing in (5.1), as the last term includes the current observation, the variations from one to zero are used to grade properly the selected neighbor, and the seed. various degrees of difference in connectedness. If we rewrite this term as On the basis of these considerations and recalling (Section 11) that the use of some functions other than likelihood can f i ( w Z ( o * , a ) = f i ( w z I ~ * , a ) = f i ( w I ~W ~ - ' , U ) (5.2) realize the same decision criteria, we use (5.1) and derive we change the observation event to a conditioned event with the labeling formula that assigns the label X i to the current respect to the local and global contexts. Therefore, instead of observation i as follows: considering only the measure of an observation, we should A2 = Ai-1 g(Acxa). (5.6) consider a measure that also takes into account conditioning events, and that may then better represent the information vI1. IMPLEMENTATION DETAILSAND APPROACHVALIDATION required. As a conclusion, the label value, through the estimate of A. Discussion of Implementation Strategies the likelihood value, eventually becomes a function of the The label computed by applying the nonlinear recursive conditioned event ( w z I w z - l , a ) . labeling formula is always equal to or smaller than the previous B. The Observation Measure Combining Information Sources ones. Such a nonlinear property is responsible for the different characteristics of the labeling result. In the vicinity of the To describe the conditioned event (wz I w i - I , a ) , we seed point (in densitometric and topological terms), even small propose to exploit the X-connectedness parameter, as defined variations in signal intensity give rise to significant differences in Section 111. This very powerful parameter is able to describe in label values. If we consider the labeling process as a filtering the measure of an observation for a given neighbor and for operation on the original signal, this process turns out to be a a fixed seed point, and conveys at the same time intensity nonlinear filtering process, as more attention is given to details and topological relations. It can be seen that the measure that are closer to the seed, that is, in a more significant image c X a( i ) ,referring to the seed point a,includes an implicit binary area. relationship between the seed and the sample i . It should be noted that different choices of the labeling To include also the relation with respect to the selected function g(Acxa) relate to different assumptions made for neighbor, the following parameter is utilized: the model representing the objects present in the image. If no noise is assumed and if a searched object is completely Ac,a = C y ( 2 - 1) - C,a ( i ) (5.3) homogeneous and connected, the likelihood function as well as the labeling function may be modeled as a d function. which fully characterizes the above event. More realistically, we may assume that noise and uncertainties From the definition of intensity connectedness, we deduce that, for two points belonging to the best path from the seed to in hypotheses definition cause the labeling function to have an observation (i.e., the path that satisfies the max operator), a more spread distribution-more like Gaussian density, for example, with a standard deviation that is dependent on noise the degree of connectedness reduces to intensity and on the degree of uncertainty. Alternatively, g ( A c x a )can be viewed as a fuzzy restriction function, shaped c X a( i ) = min X " ( z ) = min min x"(z)), X " ( i ) ] zEP(a,z) ztP(a,z-l) as a sigmoid curve of the type suggested by Zadeh [27] or as (5.4) an exponential curve, as in [28]. that is, Obviously, changing the shape of the labeling function cXa( i ) = min [exa ( i - I), ~ " ( i ) (5.5) leads to the detection of more or less homogeneous objects. The optimum parameters for each of the labeling functions In other words, the intensity connectedness of a pixel i is (given in Table I) were experimentally estimated by adopting equal to its value in X" or to the X-connectedness of its parent, criteria independent of the application domain and based on if its X-connectedness has not the minimum value along the quantitative analyses of contrast, history, and noise sensitivity. best path from a to i . Then A c X areduces to zero in the case The resulting whole framework is more robust than traditional Analogously, in accordance with the dependence of the state ca on the neighbor state and on the observation itself, as stated in (4.3), the above expression for the likelihood function becomes an estimation based on CJ* and w z , that is, f l (cJ*, w z 1 a). By using Bayes rules, this probability function is further developed until the following equation is obtained:
'
[(
1.
Authorized licensed use limited to: National Central University. Downloaded on October 18, 2009 at 00:59 from IEEE Xplore. Restrictions apply.
438
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5, NO. 3, MARCH 1996
MODELSUSED
TABLE I ESTIMATED PARAMETER
FOR g ( a C x a ) AND
Model
exp(-Acx, I IC)
Exponential
Gaussian
VALUES
Parameter
K=75
0=8.5
The second experiment was carried out by using the ring image shown in Fig. 4(a). It represents concentric circles whose intensity levels decrease by the same contrast value, as the distance from the image center increases. This image is useful in comparing the ability of each function to capture even low contrasts. In Fig. 4(b), edges were extracted from the noiseless original image. The locations and the strengths of the true edges should be compared with those shown in the edge images (Fig. 4(c)-(e)) and extracted from the labeling maps, each figure being associated with a different function shape. Results obtained by the first experiment were confirmed. The exponential model (Fig. 4(c)) seems to ensure a better behavior for the whole image, as it does not loose details far from the seed. On the contrary, it can be seen that the Gaussian function (Fig. 4(e)) provides stronger edges in the proximity of the seed but looses some of the most external contrasts. On the basis of these evaluations, if not otherwise stated, all the experiments described in the following were carried out by using the exponential model. Incidentally, the choice of the exponential labeling function inhibits the possibility of having a father different from the generator.
0.
B. Image-Scanning Mechanism Validation
0.
O.1 0.
'1
5
IO
20
40
60
80 100
120 130 137 140
mexp %gauss asigm iL,
