Uncertainty in Image-Based Change Detection - CiteSeerX

Uncertainty in Image-Based Change Detection Peggy Agouris, Sotirios Gyftakis, Anthony Stefanidis Dept. of Spatial Information Engineering University of Maine 5711 Boardman Hall #348 Orono, ME 04469-5711,USA Ph. +1-207-5812180; Fax. +1-207-5812206 Email: {peggy, sotiris, tony}@spatial.maine.edu URL: http://www.spatial.maine.edu/~peggy/peggy.html

Keywords: change detection, fuzzy sets, uncertainty analysis

Abstract Digital aerial imagery provides the perfect medium for capturing geospatial information. The automation of object extraction from digital imagery has received substantial attention over the last few years in the computer vision and digital image analysis communities, resulting in significant progress in algorithms and theoretical concepts. In this paper we introduce a novel approach to object extraction from digital imagery using accuracy information. We extend the model of deformable contour models (snakes) to function in a differential mode. Prior information with accompanying accuracy estimates is used as input in our methodology. In a departure from standard techniques, the objective of our object extraction is not to extract yet another version of an object from the new image, but instead to update previously collected GIS information. By incorporating accuracy information in our technique we identify local or global changes to this prior information, and update the corresponding GIS information accordingly.

1. Introduction Change detection is a topic of great importance for modern geospatial information systems. Rapidly evolving environments, and the availability of increasing amounts of diverse, multiresolutional datasets bring forward the need for frequent updates in modern GIS. Digital aerial imagery provides the perfect medium to capture geospatial information. The automation of object extraction from digital imagery has received substantial attention over the last few years in the computer vision and digital image analysis communities, resulting in significant progress in algorithms and theoretical concepts. The majority of object extraction methodologies are semi-automatic, whereby a human operator provides manually some approximations (e.g. by selecting points on a monitor display) and an automated algorithm uses these points to extract a complete object outline. Considering roads and similar linear features, these approximations may be in the form of an initial point and approximate direction. This information is used as input by automated algorithms that proceed by profile matching (Vosselman and de Knecht, 1995), edge analysis (Nevatia and Babu, 1980), or even combinations of them (McKeown and Denlinger, 1988). Alternatively, the human operator may provide a set of points that approximate roughly the road, e.g. a polygonic approximation of a long road segment. This information is used by automated methods like dynamic programming and deformable contour models, a.k.a. snakes (Gruen and Li, 1997; Li, 1997). Full automation is pursued by automating the selection of the above mentioned necessary initial information. Examples of fully automatic efforts may be found in (Baumgartner, Steger et al., 1999) and (Barzohar, Cohen et al., 1997). In (Tönjes and Growe, 1998), the authors use

a knowledge-based approach (semantic nets) to combine information from multiple sensors for road extraction. It is common to use pre-existing information (e.g. a road outline from an older map) to approximate the position of this object in a new image. Using complex algorithms, the object is then precisely localized in the image to determine its present position. This information is subsequently used to update the GIS. In this context, information is treated as deterministic in nature, and GIS updates often result in storing multiple representations although an object has actually remained unchanged. In this paper, we introduce a novel approach to object extraction from digital imagery using accuracy information. We extend the model of deformable contour models (snakes) to function in a differential mode. Prior information with accompanying accuracy estimates is used as input in our methodology. In a departure from standard techniques, the objective of our object extraction is not to extract yet another version of an object from the new image, but instead to update previously collected GIS information. By incorporating accuracy information in our technique we identify local or global changes to this prior information, and update the GIS accordingly. In section 2 we present a methodology to obtain accuracy measures for objects extracted from digital imagery using snakes. In section 3, we introduce the model of differential snakes, while section 4 gives an overview of the optimization algorithm used to solve this model. Experimental results are presented in section 5, followed by concluding remarks.

2. Theoretical background In order to describe the proposed methodology for change detection, we will first introduce the general snake model, and will present a methodology to obtain accuracy estimates for the objects extracted by it. 2.1 Snakes A category of computer vision automation tools that can be applied in object extraction is the deformable model (snake). The framework of this tool has been originated in the paper (Kass, Witkin et al., 1987). There is a great advance since then in the literature of snakes both in theoretical and experimental areas. The concept of snake is based on the curve detection by optimizing models of curve contrast and smoothness using elastodynamic models and applied forces. In this paper, we represent the snake as a polygonal (open) line between points. The main problems are how to define the energy at these points and how to solve the energy minimization problem. Energy functions In general, the energy function of a snake contains internal and external forces as well as external constraints. The internal forces allow the contour to stretch or bend at the specific point. The external force attracts the contour to significant features on the image, while the external constraints are user imposed restrictions. The total energy of each point is expressed as follows (Eq. (1)): Esnake = α ⋅ Econt + β ⋅ Ecurv + γ ⋅ Eedge

(1)

where : Econt , Ecurv = first and second order continuity constraints (internal forces), Eedge = edge strength (external force), and α,β,γ = (relative) positive weights of each energy term. Continuity term

If vi = ( xi , y i ) is a point on the contour, the first energy term in Eq.(1) is defined as follows: E cont = d − vi − vi −1 d=

points −1

∑v i =1

i+1

(2)

(points − 1)

− vi

(3)

where d = average distance between points. This term guarantees that the points will be evenly spaced without grouping on some areas and at the same time the distance between them will be minimized. Curvature term E curv = vi −1 − 2vi + vi +1

2

(4)

Since the continuity term produces evenly spaced points, this term (Eq. (2)) gives the curvature multiplied by a constant. This constant becomes insignificant, because the curvature term is normalized in the neighborhood of each point. Edge term

Eedge = −∇I(vi )

(5)

This term (Eq. (5)), in general, forces points to move towards image edges. The gradient of the image at each point should be normalized, to display small differences in values. 2.2 Quality assessment In this section, we will describe how the result of a category of extraction methods can be evaluated. We assume that a deformable model (snake) has been used for the extraction and the coordinates of the extracted road are known. The snake model uses an energy minimization procedure in order to extract the road contour. Low

Medium

High

Et Low

Low

Medium

High

High

U DEt

Figure 1, Quality evaluation diagram For evaluation purposes, we compute the total energy E snake (the one that the snake method has minimized) on several points along the road. These points should be characteristic for each road segment. The next step is to generate values of uncertainty for each point of the

extracted road. This can be accomplished using fuzzy linguistic rules. Fuzzy theory is used, because it can better describe situations where the a priori information we have is vague. For this fuzzy evaluation model, we consider as input parameters the values of the total energy (E t) and the rate of energy change (DEt) at the sampling point. The output will be the uncertainty (U) of the extraction at that point. The following sets of linguistic values are considered : Et={low, medium, high}, DEt={low, high}, U={low, medium, high}. The fuzzy sets for input and output are shown in Figure 1. The membership functions in Figure 1 have simple (triangle) shape. The characteristic points for each triangle are determined from the statistical properties of each input variable. The fuzzy rules that will produce the uncertainty values have the following form and they are displayed in the Table 1: • If the Et is LOW and the DEt is LOW then U is LOW High

Medium

Low

Low

High

Low

Low

High

High

Medium

High

DEt

Et

Table 1, Fuzzy rules for quality evaluation After the defuzzification of the results, the values of uncertainty are saved together with the coordinates of the points. These data will be used in the next step of change detection. In

Figure 2, Quality assesment of road extraction Figure 2 we display the result before and after the quality assessment for a road segment. The different line styles display the different uncertainties of the specific segment.

3. Differential snake model In a differential snake mode, our snake solution is constrained by the pre-existing information. In order to accommodate for this additional constraint, we expand the standard deformable model by introducing an additional energy term Eunc and the (relative) weight _ (Eq. (6)). Esnake = α ⋅ Econt + β ⋅ Ecurv + γ ⋅ Eedge + δ ⋅ Eunc

(6)

This term describes how the uncertainty we have for the original (extracted) point, controls the relocation of this point during the optimization procedure in the new image. The action of this control is similar to the action of an elastic spring, as it is described in Hooke's law : “The force applied to any solid is proportional to the strain it produces within the elastic limit for that solid”. In Figure 3, we can see how the uncertainty energy (and hence force Fin) acts on the moving point. This energy struggles to keep the point close to the original position (v0i), while the rest forces (geometry, edge: Fout) attempt to drive it away, because of the new road

placement. The introduction of the uncertainty energy as a spring action is related to the basic theory of snakes, as the deformable models are represented with elastodynamic equations. Fout vi Fin d v0i

Figure 3, Action of uncertainty on point movement Uncertainty term Eunc = f( Unc(v0i), d )

(7)

The energy derived from the uncertainty is generally a function (Eq.(7)) dependable on the uncertainty of the original point (Unc(v0i)), and the distance of the current point from the original one (d). Different mathematical expressions can describe the above behavior. One simple approach is the following (Eq. (8)): the energy produced by the uncertainty is proportional to the distance from the origin within a threshold (Di) and inverse proportional to the uncertainty of the origin point. After the threshold, the energy remains constant. The diagram of Eunc is displayed in Figure 4. The energy function is given from the equation: Eunc =

1 ⋅d Di ⋅Unc(v0 i )

(8)

where Unc(v0i) = uncertainty of original point, Di = uncertainty threshold, d = distance of current point (vi) from original (v0i). We derive two properties form Eq. (8). The first property is that the uncertainty energy increases as we depart from the original point, and this guarantees that the uncertainty force is Eunc 1/Unc(v0i)

0

Di

d

Figure 4, Uncertainty energy (Eunc) diagram towards the original point. The second property is that the higher the uncertainty we have for the initial point, the smaller the energy is. Hence, the force that holds the point close to the initial position is smaller.

The threshold Di is a parameter that defines the neighborhood over which the spring is actively affecting the current object extraction process. The logic behind its action is fairly simple: when the previously available information was known with high uncertainty, the spring will function over a relatively large area with relatively low strength. When the prior information was known with high accuracy, the equivalent spring action will be strong but over a short interval. Accordingly, the Di parameter is defined at each location along an outline as: Di = D0 ⋅Unc(v0 i )

(9)

where Unc(v0i) = uncertainty of original point, and D0 = global threshold. This makes the energy function of Eq. (8) to be inversely proportional to the square of the uncertainty measure.

4. Optimization algorithm Our proposed method combines the previous information (extracted image with its quality metadata) with the new image. The information is projected (projected metadata) on the new image and an optimization algorithm is employed. During this optimization, the projected metadata is used for change detection. The sampled points are moved close to the new object contour under the geometry, gradient and uncertainty forces. The final distances of the points are compared to the threshold D0, to decide if there is a change or not. In order to compute the minimum of the energy function for the whole snake, there have been developed various techniques. In the original development of snakes (Kass, Witkin et al., 1987) use a variational method (Euler-Lagrange equation) is used to analyze and solve numerically the snake equations. In (Amini, Weymouth et al., 1990) the authors propose a dynamic programming solution to the optimization problem, which is simple and fast. An alternative, faster approach (greedy algorithm) is suggested by Williams and Shah (Williams and Shah, 1992) and this optimization method will be used in this paper.

vi-1

vi

vi+1

Figure 5, Optimization procedure In this greedy algorithm, the energy at each point in a neighborhood (e.g., 3X3) of the current point vi is computed (Figure 5). The previous point vi-1 has already been optimized and moved to its new position. The next point vi+1 has not been moved so far. The locations of the previous and next points are used with the proposed ones of vi in the computations of the energy terms. Special consideration is given for the initial and final points. The minimum energy point location is assigned to vi and this procedure is repeated for every point (one loop). In the end of each loop, the stop criteria are evaluated: number of points moved, and total snake energy changed. If one of them is satisfied, then the loop procedure ends, and the current locations of the snake points are used to create the line segments of the final contour.

5. Experiments 5.1 Synthetic Image

For the first experiment, we use two grey images (Image1, Image2) with resolution 200X300

Figure 6, Result for synthetic image(solid:differential snake, dotted:Image1, dashed: Image2) pixels. From the Image1 a road is extracted using a simple snake model and uncertainty values are generated for quality assessment. Next, change detection is applied on the Image2, where few parts of the first road have changed. The final result (Figure 6) shows: the differential snake in solid line, the road of Image1 in dotted line, and the road of Image2 in dashed line. The points of the differential snake where change has been detected are shown with triangles and the rest with cycles. Also, in Figure 6, we show a zoomed part of the upper left side of the results, where we can see that the first 4 points have not been moved, while the next 2 points have (triangles). The uncertainty values of these 6 points are : 0.50, 0.51, 0.57, 0.48, 0.33, 0.24 respectively. The large values of the first 4 points act as a spring force in a large area, preventing the points to move, while the small values of uncertainty in the rest 2 act in a smaller neighborhood. 5.2 Real Image In this experiment, we have used medium resolution images (250x350 pixels), whose ground resolution is about 5m per pixel. The results are shown in Figure 7. The white dotted line is

Figure 7, Result for real image the road extracted from the first image. The triangles show the points where change has occurred, while the cycles show points with no change. On Figure 7, we can see that the differential snake has moved completely to the new road, except of a segment at the centre of

the image. At that area, the points were hold close to the initial coordinates due to uncertainty forces, while the continuity and edge forces were not strong enough to drive these points away.

6. Conclusion The above presented methodology is a novel approach to integrate object extraction and change detection. By extending the model of deformable contour models, we are able to incorporate previously available information and its accuracy estimates in the new object extraction process. This results in a minimization of information duplication within a GIS, improving its efficiency and accuracy. At the same time, this approach improves the performance of object extraction itself, as prior information improves our ability to overcome image information gaps (e.g. due to occlusions). Acknowledgements This work was supported by the National Science Foundation through CAREER grant number IIS-9702233 and through a National Imagery and mapping Agency NURI award. References - Amini, A., Weymouth, T., Jain, R., 1990, Using Dynamic Programming for Solving Variational Problems in Vision, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12(9), pp. 855-867. - Barzohar, M., Cohen, M., Ziskind, I., 1997, Fast Robust Tracking of Curvy Partially Occluded Roads in Clutter in Aerial Images, in Automatic Extraction of Man-Made Objects from Aerial and Space Images (II), Gruen, A., Baltsavias, E., and Henricsson, O., Eds., Birkhäuser Verlag, pp. 277-286. - Baumgartner, A., Steger, C., Mayer, H., Eckstein, W., 1999, Automatic road extraction in rural areas, Int. Arch. Photogramm. Remote Sensing, XXXII(3-2W5), pp. 107-112. - Gruen, A., Li, H., 1997, Linear Feature Extraction with 3-D LSB-Snakes, in Automatic Extraction of Man-Made Objects from Aerial and Space Images (II), Gruen, A., Baltsavias, E., and Henricsson, O., Eds., Birkhäuser Verlag, pp. 287-298. - Kass, M., Witkin, A., Terzopoulos, D., 1987, Snakes: Active contour models, in Proc. 1st Int. Conf. on Computer Vision, London, pp. 259-268. - Li, H., 1997, Semi-Automatic Road Extraction from Satellite and Aerial Images, Ph.D. Thesis, Institute of Geodesy and Photogrammetry, ETH, Zürich, Switzerland. - McKeown, D., Denlinger, J., 1988, Cooperative Methods for Road Tracking in Aerial Imagery, in Proc. Computer Vision and Pattern Recognition, Ann Arbor, MI, pp. 662-672. - Nevatia, R., Babu, K.R., 1980, Linear Feature Extraction and Description, Computer Graphics and Image Processing, 13, pp. 257-269. - Tönjes, R., Growe, S., 1998, Knowledge Based Road Extraction from Multisensor Imagery, in Proc. ISPRS Commission III Symposium on Object Recognition and Scene Classification from Multispectral and Multisensor Pixels, Columbus, Ohio, pp. 387-393. - Vosselman, G., de Knecht, J., 1995, Road tracing by profile matching and Kalman filtering, in Automatic Extraction of Man-Made Objects from Aerial and Space Images, Gruen, A., Kuebler,V., and Agouris, P., Eds., Birkhäuser Verlag, pp. 265-274. - Williams, D.J., Shah, M., 1992, A Fast Algorithm for Active Contours and Curvature Estimation, CVGIP: Image Understanding, 55(1), pp. 14-26.