Differential Snakes for Change Detection in Road

Differential Snakes for Change Detection in Road Segments Peggy Agouris, Anthony Stefanidis, and Sotirios Gyftakis

Abstract The automation of object extraction from digital imagery has been a key research issue in digital photogrammetry and computer vision. In the spatiotemporal context of modern GIS, with constantly changing environments and periodic database revisions, change detection is becoming increasingly important. In this paper, we present a novel approach for the integration of object extraction and image-based geospatial change detection. We extend the model of deformable contour models (snakes) to function in a differential mode, and introduce a new framework to differentiate change detection from the recording of numerous slightly different versions of objects that may remain unchanged. We assume the existence of prior information for an object (an older record of its shape available in a GIS) with accompanying accuracy estimates. This information becomes input for our “differential snakes” approach. In a departure from standard techniques, the objective of our object extraction is not to extract yet another version of an object from the new image, but instead to update the preexisting GIS information (shape and corresponding accuracy). By incorporating accuracy information in our technique, we identify local or global changes to this prior information, and update the GIS database accordingly. This process is complemented by versioning, where, in the absence of change, the pre-existing information may be improved in terms of accuracy if the new image so permits. Experimental results (using synthetic and real images) are presented to demonstrate the performance of our approach.

Introduction Change detection is a topic of great importance for modern geospatial information systems. Rapidly changing environments, and the availability of consistently increasing amounts of diverse, multiresolutional datasets, bring forward the need for frequent revisions of modern geographic information systems (GIS). Digital aerial imagery provides the perfect medium to capture geospatial information. Accordingly, object extraction from digital imagery is a fundamental photogrammetric operation. Despite substantial work in the photogrammetric and computer vision communities during the last two decades, this is a research topic where a dominant automated methodological or algorithmic trend has not yet clearly emerged. Instead, one can identify numerous approaches and various strategies that have been developed, often targeting specific subtasks with various degrees of success. The evolution of the current state-of-the-art among digital image analysis and computer vision activities on the subject of automated object extraction

P. Agouris and S. Gyftakis are with the Department of Spatial Information Engineering and A. Stefanidis is with the National Center for Geographic Information and Analysis, all at 348 Boardman Hall, University of Maine, Orono, ME 04469-5711, ({peggy; tony; sotiris}@spatial.maine.edu). PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

from digital aerial and satellite imagery may be found in Suetens et al. (1992), Gruen et al. (1995b), Gruen et al. (1997), and Lukes (1998). The majority of current object extraction methodologies are semi-automatic, whereby a human operator provides manually some approximations (e.g., by selecting points on a monitor display) and an automated algorithm uses these points as input to extract a complete object outline. Considering roads and similar linear features, these approximations may be in the form of an initial point and an approximate direction. This information is used as input to automated algorithms that proceed by profile matching (Vosselman and de Knecht, 1995), edge analysis (Nevatia and Babu, 1980), or even combinations of both (McKeown and Denlinger, 1988). Alternatively, the human operator may provide a set of points that roughly approximate the road from start to end, e.g., a polygonic approximation of a long road segment. This information is used by automated methods like dynamic programming and deformable contour models, i.e., snakes (Gruen and Li, 1997; Li, 1997). Full automation is pursued by automating the selection of the above-mentioned necessary initial information (e.g., node locations, road orientation). Examples of substantial efforts towards full automation may be found in Baumgartner et al. (1999) and Barzohar et al. (1997). In To¨njes and Growe (1998), the authors use a knowledge-based approach (semantic nets) to combine information from multiple sensors for road extraction. Research on image-based geospatial change detection is rather limited, at least compared to the body of work on object extraction. Efforts relevant to change detection include work on the geometric accuracy of object boundaries (e.g., the epsilon band method (Dunn et al., 1990)). In Cheng and Molenaar (1997), the authors discuss the role of object definition and thematic vagueness on the geometric uncertainty of extracted objects. The authors use fuzzy logic theory to map the spatial extent of objects and to examine the geometric change detection of them. An approach to model and update building outline information based on gazetteer environments is presented in Agouris et al. (2000), making use of a least-squares template matching method to compare GIS outlines to new imagery. A method for updating a road database was presented by Klang (1998), where preexisting information provides approximations for subsequent object extraction using snakes, followed by a statistical analysis for the detection of new road segments. In Auclair Fortier et al. (2000), the authors extend the above approach by using road intersections. Road intersections improve matching between the road database and the lines on

Photogrammetric Engineering & Remote Sensing Vol. 67, No. 12, December 2001, pp. 1391–1399. 0099-1112/01/6712–1391$3.00/0 䉷 2001 American Society for Photogrammetry and Remote Sensing December 2001

1391

the image, and hypotheses for new road segments are generated from these line junctions. In this paper, we introduce a novel approach for road change detection applied to elongated features (specifically, road segments) using digital imagery and pre-existing information. At the core of our approach is the development of the method of differential snakes, a novel technique that combines object extraction and change detection. In contrast to standard object extraction techniques, the goal of our method is not to obtain yet another version of an object from the new image, but instead to update and/or improve the pre-existing GIS information for that object. We achieve this objective by incorporating prior information (i.e., the last version of the shape of the road centerline and corresponding accuracy estimates) in the object extraction process itself. In a departure from the general definition of snakes, we have expanded the theoretical model of snakes to perform a comparison of image content to prior information (and its uncertainty measures) instead of standard object extraction. This differential extension of the snakes solution allows us to identify local or global changes to the prior information, and to update the GIS accordingly. In addition to change detection, the GIS is also updated by means of versioning if possible. In our framework, versioning is defined as the process of improving the accuracy of the object information already stored in a GIS (if the new imagery so allows), even when no change has actually been detected. Combined, change detection and versioning comprise our GIS update cycle. The paper is organized as follows. The next section presents an overview of our GIS updating process and its function within an integrated geospatial environment. This is followed by a brief overview of the standard deformable contour models. We continue by introducing an approach for uncertainty analysis in these models and present our differential snakes model as an extension of the traditional solution. We also discuss versioning given the introduced uncertainty analysis approach. Experimental results are then presented, followed by conclusions and future work plans.

Image-Based GIS Updating Image-based geospatial change detection operations commonly proceed in two steps. First, they use complex algorithms like the ones mentioned in the previous section to identify objects in a new image. This information is subsequently used to update the GIS in a straightforward manner: a newly extracted object is compared to the older version of the same object in the database and, if it differs, it substitutes that older information to update the GIS. In this context, information is treated as deterministic in nature, and GIS updates often result in storing multiple slightly different representations of an object that has actually remained unchanged. In our approach, we avoid this pitfall by integrating object extraction and change detection in a single process. We assume an integrated geospatial environment, whereby image analysis proceeds by having access to pre-existing information for the processed area (Agouris et al., 1998). Considering road segments (which are the main focus of this work), we assume that older shape information for the road centerline and accuracy estimates for this information are available. This prior information may have resulted from a previous image analysis process (exploiting older imagery), or by any other established method to collect GIS information (e.g., traditional surveying processes). Our approach to this problem of GIS updating is described in Figure 1. We make use of pre-existing object shape information. For small scale aerial or satellite imagery, which is the main focus of this paper, this shape information is the centerline of the road. The reader can easily understand that, for largescale imagery, this shape information may be replaced by each of the two road outlines (left and right edges) separately, both of 1392

December 2001

Figure 1. Change detection and versioning for GIS updates.

them together, or their centerline. The pre-existing object shape information is projected onto the new image using standard orientation parameters and corresponding transformations. There, together with relevant accuracy information, it becomes the input for our differential snakes solution. In our differential snakes approach, we compare the radiometric content of the new image to the geometry and accuracy of the projected outline. As we will demonstrate in the next section, the application of differential snakes allows us to detect changes in this object. The results can then be used to update the GIS information for this object. This updating includes the modification of the object’s shape to reflect the newly detected changes, and the updating of the corresponding accuracy information to reflect the accuracy of the newly extracted shape. For those parts of the road where change was not detected, we proceed with versioning. The difference between change detection and versioning is quite simple. In change detection we recognize that an object has been modified since its last recorded instance (e.g., a road has been widened or diverted). In versioning, we examine whether we can improve the previously recorded information for this object, even if the object has not changed. For example, the new image might offer us the opportunity to improve the accuracy of the recorded information (e.g., it is of higher resolution than the image from which we extracted our previous information). Thus, a key issue in versioning is to recognize the potential to record a new version of an otherwise unchanged object at better accuracy than the already recorded version of it. To do so, we make use of our analysis of the energy of snakes to identify potential improvements. If that is the case, versioning is performed as a highly localized object extraction process. Combined, change detection and versioning form a GIS updating cycle.

Differential Snakes Formulation of Standard Snakes Deformable contour models (i.e., snakes) have been developed in the computer vision community as object extraction tools. They were introduced by Kass et al. (1987), and have since been the subject of substantial research, regarding both theoretical and practical issues. Their theoretical foundation is provided PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

by an approximation of object extraction by elastodynamic models. A deformable line (snake) attaches itself to an edge location in an image similarly to the manner in which a deformable object embedded in a viscous medium deforms its shape until it reaches a state of stability. In its numerical solution, the snake is represented by a polygonal line, defined by nodes and segments connecting these nodes. The geometric and radiometric relations of these nodes are expressed as energy functions, and object extraction becomes an optimization problem. The main issues are how to define the energy function at snake nodes and how to solve the energy minimization problem. Energy Functions In general, the energy function of a snake contains internal and external forces. The internal forces regulate the ability of the contour to stretch or bend at a specific point while preserving some degree of geometric smoothness. The external force attracts the contour to specific image features (commonly edges). Additionally, we may use external constraints to express user-imposed restrictions (e.g., to force the snake to pass by specific points). The total energy of each point is expressed as Esnake ⫽ ␣ ⭈ Econt ⫹ ␤ ⭈ Ecurv ⫹ ␥ ⭈ Eedge

(1)

where Econt and Ecurv are expressions of the first- and secondorder continuity constraints (internal forces), Eedge is an expression of the edge strength (external force), and ␣,␤, and ␥ are relative weights describing the importance of each energy term. A very brief description of these energy functions follows. CONTINUITY TERM

If vi ⫽ (xi ,yi) is a point on the contour, the first energy term in Equation 1 is defined as Econt ⫽ d ⫺ 앚vi ⫺ vi⫺1앚

(2)

where d is the average distance between points: i.e., d⫽

points⫺1

兺

i⫽1

앚vi⫹1 ⫺ vi앚/(points ⫺ 1).

(3)

This term forces the snake nodes to be evenly spaced, avoiding grouping at certain areas, while at the same time minimizing the distance between them. CURVATURE TERM

This term expresses the curvature of the snake contour, and allows us to manipulate its flexibility and appearance: i.e., Ecurv ⫽ 앚vi⫺1 ⫺ 2vi ⫹ vi⫹1앚2.

(4)

EDGE TERM

The above-mentioned two terms describe the geometry of the contour and are referred to as internal forces of the snake. The third term describes the relation of the contour to the radiometric content of the image and is referred to as the external force. In general, it forces points to move towards image edges. An expression of such a force may be defined as Eedge ⫽ ⫺ⵜI(vi).

(5)

This energy model attracts the snake to image points with high gradient values. Because the gradient is a metric for the edges PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

of an image, the snake is attracted to strong edge points. The gradient of the image at each point is normalized to display small differences in values at the neighborhood of that point. As mentioned above, the coefficients ␣,␤, and ␥ are relative weights describing the importance of each energy term in the solution. Increasing the relative values of ␣ and ␤ will result in putting more emphasis on the geometric smoothness of the extracted line. This might be suitable for very noisy images, but might be unsuitable when dealing with sharp angles in the object space. Increasing the relative value of ␥ puts more emphasis on the radiometric content of the image, regardless of the physical appearance of the extracted outline. As is commonly the case in snake solutions, the selection of these parameters, as well as of the additional parameter ␦ that will be introduced in the next section, is performed empirically (see Amini et al. (1990), Lai and Chin (1993), and Williams and Shah (1992)). The automation of the selection of these parameters is a different research topic that is beyond the scope of this paper. Optimization Algorithm Combined, the above three energy terms describe an ideal model of a road segment, i.e., a smooth curve coinciding with a strong edge in the image. The goal of traditional snake-based object extraction is to identify in an image a sequence of points describing a contour that approximates this ideal model. In other words, we are looking for smoothly continuous curves at the vicinity of an edge. This is an iterative process, where we begin with an approximation of this contour and improve this approximation to optimize the total energy function of Equation 1. The snake contour is initialized by selecting a few seed points in an image. Afterwards, the points along the contour move to new locations, to progressively optimize the energy functions. In our case, this is a minimization process: nodes are repositioned to produce a new snake with a lower total energy than its prior stage. These new locations are found through an iterative procedure. Various techniques have been proposed to compute the minimum of the snake energy function. In the original development of the problem, Kass et al. (1987) used a variational method (i.e., the Euler-Lagrange equation) to analyze and solve numerically the snake equations. In Amini et al. (1990), the authors propose a simple and fast dynamic programming solution to the optimization problem. An alternative faster approach (greedy algorithm) is suggested by Williams and Shah (1992). This is the optimization method used in the experiments presented in this paper. Calculation of Uncertainty While there have been substantial research efforts dedicated to the modeling of energy functions and optimization algorithms, the analysis of the accuracy of the results is still rather underexplored. Results are commonly treated as deterministic instead of stochastic quantities, and this often leads to substantial errors in change detection. Here we introduce a method based on fuzzy logic for the evaluation of the extraction results. As mentioned in the previous section, the snake model uses an energy minimization procedure to extract the road contour. Our argument is quite straightforward. By analyzing ● the total energy Et and ● the rate of energy change (DEt)

along the snake solution, we understand how well the extracted contour approximates the ideal road model (expressed by the equations introduced in the previous section). Indeed, the energy function contains all the basic performance information for the object extraction process and, by examining its local December 2001

1393

Figure 2. Quality evaluation diagram.

values and gradients, we estimate the accuracy of the extraction process. Our quality evaluation approach is summarized in Figure 2. The horizontal axes of all three diagrams are values of energy, energy variations, and uncertainty, normalized along a road segment. The vertical axes are membership function values and range from 0 (horizontal axis) to 1 (top of the curves). The generation of uncertainty values is accomplished using fuzzy linguistic rules. The fuzzy output is the uncertainty of extraction (U) at that point. We use fuzzy theory because it allows us to better express the meaning of information that has a certain degree of vagueness associated with it (as is the case with local energy functions evaluated over various spatial intervals). Furthermore, it also allows us to express in a uniform manner numerical values whose range may vary greatly (as is the case with energy functions for different contours in an image). The following sets of linguistic values are considered in our application: ● Et ⫽ {low, medium, high}, ● DEt ⫽ {low, medium, high}, and ● U ⫽ {low, medium, high}.

The fuzzy sets for input and output are shown in Figure 2. The membership functions in Figure 2 have simple Gaussian shape (Tan and Tokinaga, 1999). The characteristic points for each membership function are determined from the statistical properties of each input variable. The fuzzy rules that will produce the uncertainty values have the following form and they are displayed in Table 1. Let us consider an example: ● If Et is LOW and DEt is LOW, then U is LOW.

The rule states that low energy and low energy variations in our snake solution imply low uncertainty in object extraction. Indeed, the first condition reflects a good optimization of the snake’s internal and external forces by minimizing the corresponding energy functions. In other words, the extracted line approximates very well the ideal road model expressed by Equations 1 through 5. The second condition states that the energy function is mostly constant in the vicinity of a snake

point. This addresses the possibility that our algorithm was sidetracked by random, localized, noise-like features in the image. Accordingly, low energy variations reflect a higher possibility that a snake node is part of a longer linear feature. Combined, these two conditions indicate higher precision in object extraction, i.e., that the extracted line is indeed tracing a road segment. Other combinations of energy values and energy gradient are analyzed in a similar manner in Table 1. The table is intended to demonstrate how we proceed with our accuracy characterizations, and should not be perceived as a definitive classification. Instead, the reader can easily see how, if desired, we could identify more intervals to increase the resolution of our accuracy characterizations. The above-mentioned analysis is performed for all points along an extracted outline. By analyzing the local values and variations of energy, we assign to each point a specific local accuracy estimate. After a standard defuzzification of the results, uncertainty coefficients (in the range (0 to 1)) are saved together with the coordinates of the points. The uncertainty coefficient ratios multiplied by global accuracy measures provide us with pixel accuracy measures for the corresponding points along the snake. The global accuracy measures are expressions (in pixel units) of the expected accuracy in extracting a linear object from a specific image. In simple cases, they can be standard accuracy indices, e.g., assuming an accuracy of 0.1 pixel for snake-based road extraction. In more complex cases, we foresee the use of complex formulas where image scale and local content variations are examined to modify these estimates accordingly. The uncertainty information may be entered in a GIS database, and can be used in change detection as it is will be described in the next section. Figure 3 is only a visualization example of the effects of image conditions on object extraction accuracy. The area on the upper left side, where the road outline is covered by trees, is extracted with lower accuracy (dotted line) than the other regions (continuous and dashed lines). So, uncertainty estimates may vary along a feature. Differential Snakes Model Departing from the general definition of snakes, we expand the deformable contour model to perform a comparison of image content to prior information (and its uncertainty measures) instead of standard object extraction. Because this model will be used to identify changes by analyzing imagery at different time instances, we call it the differential snakes model. In a differential snake, our snake solution is constrained not only by the radiometric and geometric terms mentioned earlier, but also by the pre-existing information. To accommodate this additional constraint, we expand the standard deformable model by introducing an additional energy term Eunc and a corresponding relative weight coefficient ␦: i.e., Esnake ⫽ ␣ ⭈ Econt ⫹ ␤ ⭈ Ecurv ⫹ ␥ ⭈ Eedge ⫹ ␦ ⭈ Eunc.

(6)

TABLE 1. FUZZY RULES FOR QUALITY EVALUATION Et DEt

High

Medium

Low

Low Medium High

Medium High High

Low Medium High

Low Medium Medium

1394

December 2001

Figure 3. Example of road extraction quality assessment.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

This additional energy term describes the discrepancy between the current snake solution and the pre-existing information. The action of this term is similar to the action of an elastic spring, as described in Hooke’s law of elasticity (ca. 1660): “The force applied to any solid is proportional to the strain it produces within the elastic limit for that solid.” In other words, the pre-existing information acts as a spring that keeps attracting the snake solution towards it. Change is detected if and only if the new image supports the notion that the object has moved beyond the stochastic range of the older information. If that is the case, the new image will support strongly that the snake moves to a new position away from its older version, and the effect of the spring-like action will be overcome by the radiometric and geometric conditions of the new image. If the image content is only weakly suggesting a very small move well within the stochastic limits of the older information, we do not detect change. In this case, the spring-like force will keep the snake in its earlier location, avoiding the extraction of yet another version that does not differ statistically from its older version. Of course, it is possible that the object has indeed moved by a minimal amount, but the fact remains that, considering the quality of our prior information and current image, we cannot determine whether the object has indeed changed. This is a rather holistic approach to change detection, where prior information, its quality, and the quality of the new image are considered to detect change. In this sense, image-based change detection is performed within an information system, fully exploiting its content. In Figure 4, we can see how the prior information (and corresponding force Fin) acts during a solution on a snake point. The outline on the left indicates the older record of the road projected onto a new image. A small window of that image is shown on the right, and the dot at position vi is the image location of the corresponding snake point during a solution iteration. The fourth energy component of Equation 6 (the one due to prior information) attempts to keep the point close to its original position (v 0i) and acts as a force Fin driving the solution towards v 0i. The total effect of the other three geometric and edge energy components (expressed as a combined force Fout in the figure), is to drive the snake away, towards the radiometric content of the new image (road edge location). If Fout is weak compared to Fin , the solution will gravitate back to the original location v 0i. An example of such an instance may be an occluded segment of the road (e.g., under a tree canopy, flooded). If Fout is stronger than Fin , the solution will gravitate away from the original location v 0i and towards the road edge location in the new image. If this new location is beyond the accuracy range of the prior information, we detect change and update the GIS to reflect it. Prior Uncertainty Energy In general, the energy expressing the attraction of prior information is given by the following function

Figure 4. Effects of prior information and radiometric content on point movement in differential snakes.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

Eunc ⫽ f(Unc(v 0i), D).

(7)

The function states that the energy component at a current snake point (vi) depends on the uncertainty with which we know the corresponding prior information (Unc(v 0i)) and the distance (d ) of the current point from the older outline. Point v 0i is the point in the older outline that is closest to the node vi of the new snake. Various mathematical expressions can be used to describe the above behavior. In our approach we define the energy effect of the prior information within a threshold Di to be ● proportional to the distance between the current solution and the prior outline, and ● inversely proportional to the corresponding uncertainty of the older outline.

The threshold expresses the range over which we allow the prior information to affect the new solution. Beyond the threshold, this energy component is minimized. This can be achieved either by fixing or by nullifying the value of this component. The first option allows a smoother transition in our snake iterations, avoiding adverse perturbations. In mathematical terms, this energy component is given by the set of equations Eunc ⫽

1 ⭈d Di ⭈ Unc(v 0i)

(if d ⬍ Di)

Eunc ⫽

1 Unc(v 0i)

(if d ⬎ Di)

(8)

where Unc(v 0i) is the uncertainty of prior information, Di is the uncertainty threshold, and d is the distance between the current point (vi) and the original outline. The diagram of this function Eunc is displayed in Figure 5 (left). If the new information suggesting a move to a new position is strong enough to attract the snake to the limits of the d range (Di), it will continue to guide the snake to the correct solution. The prior information energy (flat line in Figure 5) will just slow this process slightly. The second option (nullifying the effect of prior energy after the range) offers faster convergence at the risk of unwanted perturbations during the optimization process. Experiments showed that sometimes these perturbations corrupt the solution. The mathematical expression of the energy component for this option is given by the following set of equations Eunc ⫽

1 ⭈d Dt ⭈ Unc(v 0i)

Eunc ⫽ 0

(if d ⬍ Di)

(9)

(if d ⬍ Di)

The diagram of this function Eunc is shown in Figure 5 (right). It is similar to the left one, without the flat line beyond point Di. We derive two properties from Equations 8 and 9. The first property is that the uncertainty energy component increases as we depart from the original point, in accordance with the

Figure 5. Uncertainty energy (Eunc) diagrams.

December 2001

1395

above-mentioned Hooke’s law. The second property is that the higher the uncertainty with which we know the prior information, the smaller its corresponding energy term. When our prior information is known with high uncertainty (low accuracy), its equivalent spring-like force is small but operates over a rather large interval. When it is known with low uncertainty (high accuracy), its equivalent force is strong but operates over a short interval. The threshold Di is a parameter that defines the neighborhood over which the spring is actively affecting the current object extraction process. Its statistical meaning is equivalent to the selection of confidence intervals in statistical analysis (e.g., selecting a global interval D0 for a complete curve to be equal to three times the standard deviation of our solution). Accordingly, the local evaluation Di of this parameter can defined at each location along an outline as Di ⫽ D0 ⭈ Unc(v 0i)

(10)

where Unc(v 0i) is the local uncertainty value, in the range (0,1), and D0 is the global threshold. This makes the energy function of Equation 8 inversely proportional to the square of the uncertainty measure. Point Positioning during the Differential Snake Solution In the beginning of our change detection process, the preexisting information for an object is projected onto the new image using its orientation parameters and common transformations. In each subsequent iteration of the differential snake solution, we use an optimization process to compute the new snake locations. This algorithm attempts to minimize the total energy of each snake point by searching the area around it. Using the previous definitions, the total energy at each point in a neighborhood (e.g., a 3 by 3 or 5 by 5 window) of the current point vi is computed (Figure 6). The size of this neighborhood should be small in comparison to the spacing of snake points, because otherwise it would be impossible to compute the energy terms accurately. Snake solution loops proceed in a chain-like manner, with points moving one after the other along the object outline. Accordingly, when estimating a new position for point vi we have the previous point vi⫺1 already optimized and moved to its new position. The next point vi+1 has not been moved so far in the current loop. The locations of the previous and next points are used with the proposed ones of vi in the computations of the energy terms. Point vi is moved within the search window to a new position which minimizes the energy function, and then we proceed with the next point. After we complete this operation for every snake point, we begin a new iteration. Iterations continue until the snake reaches its final state. An aspect that influences the optimization procedure is the “resolution” of the snake. This resolution characterizes the speed of the optimization. During snake initialization, the user specifies the initial k nodes. Furthermore, the user selects a value d1 (in pixels). A total of n points are added between the

Figure 6. Search windows during the optimization procedure.

1396

December 2001

two existing nodes, if their distance D is greater than d1. Hence, n is the integer part (D/d1) and d1 is the resolution of the snake. Smaller resolution values produce snakes that more closely resemble the actual road segment, but correspond to a slower optimization process. Change detection using differential snakes ends when one of two criteria is met: ● the number of points moved during the last iteration is not significant (expressed as a percentage of the total number of points), or ● the total snake energy has not changed over a threshold (expressed as a percentage of the total energy function).

Versioning The end of a change detection loop identifies certain segments that have changed since their last observation, and records their new position. The GIS is updated with the new information (segment shape, corresponding uncertainty) to reflect the current state of the terrain. This is actually the information that can be provided as an answer to a query of the type “Based on the new image, how has the road changed since its last record?”

For the remaining segments, we can perform versioning. The difference between versioning and change detection in our approach is that versioning does not imply the presence of change. Actually, versioning aims at the improvement of some pre-recorded information for a piece of information for which we have no indications of change. This improvement can be achieved by lowering the uncertainty values for the unchanged segments. This involves an analysis of the energy values for these segments (as described in the Calculation of Uncertainty section of this paper), and the selection of high uncertainty segments as prime candidates for versioning. For these segments we perform a common object extraction using snakes (as described in the Formulation of Standard Snakes section), and compare the product of this process to the pre-existing information in terms of accuracy measures. If an unchanged road segment can be detected with higher accuracy (lower uncertainty) in the new image, we use this version to replace the older, less accurate record in the GIS database. However, this substitution does not indicate the detection of change; it is simply the improvement of the GIS content. In other words, a new version of a road segment corresponds to lower Di values (see the Differential Snakes Model section) than the older version. Of course, to ensure proper comparison, the D values have to be converted from pixels to ground units, considering the resolution of the processed image. Accordingly, during versioning, prior information (extracted from image 1) is replaced by the new information (from image 2) if and only if : D1 ⭈ scale1 ⬎ D2 ⭈ scale2.

Experimental Results The differential snake model introduced in this paper has been implemented in a PC (Pentium II, 300 MHz processor), using the Matlab software environment. Early experimental results with the performance of a common snakes algorithm for object extraction (as described in the Formulation of Standard Snakes section of this paper) in this environment have been reported in (Dal Poz et al., 2000). Briefly, we should mention here that our experiments indicated that the RMS error for the extracted road centerlines (difference between automatically extracted and manually identified pixels) was on the order of 0.6 pixel. Also, the correctness index (percentage of road pixels that are correctly identified as such) was on the order of 80 percent. These experiments were performed using images with a ground pixel resolution of 2 m. The experiments reported here focus on change detection and versioning. In order to examine the performance of our differential snakes model, we employed both synthetic and real imagery. PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

Synthetic Imagery As synthetic imagery, we used two gray-scale image windows (Image1, Image2), both 200 by 300 pixels, corrupted with saltand-pepper noise, with a noise density of 0.05. By adding focused noise along the road (e.g., simulating the effects of occlusions), we can manipulate the accuracy estimates of the extracted object. A road was extracted from the first image (Image1) using a simple snake model as presented in the Formulation of Standard Snakes section and uncertainty values are generated for quality assessment as described in the Calculation of Uncertainty section. The second image (Image2) was generated by modifying (through manual editing) three segments along the road in Image1 to simulate the effects of actual change. One of these segments corresponded to an area of higher uncertainty in the image, the second to an area of low uncertainty, and the third was an area where uncertainty was varying. Change detection (CD) was performed on Image2 using differential snakes, and the result is shown in Figure 7. Our change detection process used as input the road extracted from Image1 and compared it to the content of Image2 to detect changes. Figure 7 shows ● the result of differential snake change detection as a solid line, ● the actual road of Image1 as a dotted line, and ● the actual road of Image2 as a dashed line.

The points where the differential snake detected change are marked by triangles, while the remaining points are marked by circles. We can see that the differential snake considered as change the modifications in the upper right of the figure, and part of the modification in the lower middle, and therefore moved to the new locations in these areas. It did not consider as change the modifications in the upper left part and the remaining lower middle, and therefore remained at its prior location in these areas. The segments where the differential snake did not mark change (even though we had modified the road) are the ones where prior uncertainty values were large. Due to this higher uncertainty, it is statistically acceptable that the new location comes from the same statistical population as the older one; therefore, we cannot state confidently that these areas have indeed changed. For similar reasons, the two segments that the algorithm marked as changed are the ones where prior information was known with very low uncertainty. Accordingly, a modification of similar magnitude in these areas is beyond the statistically acceptable range, and can only be attributed to actual change. It is easily understood that we have exaggerated on purpose the associated uncertainty values in this experiment to demonstrate the performance of the algorithm. If we had used typical accuracy measures, the differential snake process would have detected all three areas as changed segments. The change detection process is complemented by a versioning process, shown in Figure 8. The results of change

Figure 8. Result of versioning using synthetic data.

detection were analyzed to find uncertainty outliers (nodes of high uncertainty). As expected, the nodes where high uncertainty kept the differential snake from detecting change and moving to a new location are the prime candidates for this process. In Figure 8 we have marked with squares the nodes that moved during versioning to the corresponding road locations of Image2. These squares correspond exactly to the modified road segments of Figure 7 (upper left and remaining lower middle) that were not marked as change as explained above. We have marked with circles the nodes of the differential snake that remained unchanged during versioning. To better demonstrate the involved operations, we present in Figure 9 a zoomed part of the upper left side of the extracted road after change detection and versioning. We can see that the top four points did not move during change detection (Figure 9a). In Table 2 we can see their uncertainty values, with the order of the points being top left to bottom right as we see them in Figure 9. Intentionally, we assigned the very high uncertainty values in order to examine their behavior during the experiment. During change detection, the large uncertainty values of these four points act as a spring force over a large area, preventing the differential snake to move them. Therefore, these points stay close to their initial location (road in Image1), and there is no change detected for them. After change detection, the uncertainty values for these points are high (see column After CD of Table 2); hence, they become prime candidates for versioning where indeed they move towards the road of Image2. After versioning, with the prior information disengaged, the new result is of far superior accuracy (see column After VS of Table 2), and is accepted as a new, improved version

(a)

(b)

Figure 9. Zoom-in after change detection (a) and versioning (b).

TABLE 2. UNCERTAINTY VALUES FOR THE TOP FOUR POINTS OF FIGURE 9

Figure 7. Change detection using synthetic data.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

Point

Prior Info

After CD

After VS

1 2 3 4

0.50 0.51 0.57 0.48

0.65 0.60 0.69 0.67

0.15 0.20 0.21 0.31

December 2001

1397

(a)

(b)

Figure 12. Zoom-in after change detection (a) and versioning (b).

Figure 10. The result of change detection.

of the object. Figure 9b shows how these points moved, as a result of versioning, to their new positions (square nodes). Real Imagery For experiments with real imagery, we used medium resolution imagery (250- by 350-pixel windows) with a ground pixel resolution of 5 m. The results of change detection (CD) are shown in Figure 10. The white dotted line is the prior information. It was drawn manually over the image, with intentional discrepancies introduced to examine our change detection process. The result of our differential snakes solution is the line described by triangular and circular nodes. Similar to Figure 7, triangles indicate a detected change while circles indicate nodes that did not move. As we see in Figure 10, the differential snake has moved completely to the image location of the road, except for a small segment in the middle of the image. At that area, the points were held close to the initial coordinates due to uncertainty forces, while the continuity and edge forces were not strong enough to drive the solution to the new location. Figure 11 shows the results of versioning, where even these points moved to their new location. In Figure 12, we present an enlarged window of the above-mentioned middle part of the extracted road, after change detection and versioning.

Figures 13 and 14 provide another visualization of the role of uncertainty using real imagery. A pre-existing road segment (e.g., available from an older map) is projected onto the new image (Figure 13). The circles around the segment nodes express the local uncertainty threshold. It can be easily seen how the new road segment (as captured in the image) lies within the threshold of the three top right nodes, but beyond the threshold of the remaining three nodes. Accordingly, during change detection the three lower nodes move to the new location (and are marked to indicate local change). The result of change detection is shown as a thick dashed line in Figure 14a. This result is shown as a dashed line in Figure 14b and is input for the subsequent versioning process. After versioning, the remaining two nodes move to the corresponding image

Figure 13. Pre-existing information projected onto a new image window.

(a)

Figure 11. The result of versioning.

1398

December 2001

(b)

Figure 14. Change detection (a) and versioning (b).

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

Time (sec.)

Synthetic Image

Real Image

Amini, A., T. Weymouth, and R. Jain, 1990. Using dynamic programming for solving variational problems in vision, IEEE Trans. on Pattern Analysis and Machine Intelligence, 12(9):855–867.

CD VS Total

8.7 5.5 14.2

23.1 9.0 32.1

Auclair Fortier, M.-F., D. Ziou, C. Armenakis, and S. Wang, 2000. Automated updating of road information from aerial images, Proceedings of ASPRS 2000, 22–26 May, CD-Rom, Washington, D.C.

TABLE 3. Computer Time Requirements

location, while the top right node does not move because it was already on the road. The continuous black line in Figure 14b shows the new road segment after versioning. Table 3 shows the computational time estimates for change detection (CD) and versioning (VS), using imagery and hardware as mentioned above. It should be noted that during prototype implementation we were not concerned with computational optimization issues; rather, we focused on the robustness of the implemented technique. Accordingly, some improvements in computational time performance should be anticipated.

Conclusions In this paper we presented a new approach to change detection. This comprises a new variation of snakes to perform automated object extraction making use of prior information, and a new framework to differentiate change detection from the recording of numerous slightly different versions of objects that may remain unchanged. Our change detection methodology is a novel automated approach to integrate object extraction and change detection. By extending the model of deformable contour models, we are able to incorporate previously available information and its accuracy estimates in the new object extraction process. This results in a minimization of information duplication within a GIS, improving efficiency and accuracy. At the same time, this approach improves the performance of object extraction itself, as prior information can also be used to help us overcome information gaps in the analyzed imagery (e.g., due to occlusions). Furthermore, we presented a framework where change detection is complemented by versioning. Versioning allows us to improve information flow between images and GIS, updating the GIS information without falling in the pitfall of recording numerous unnecessary duplications. Combined, we feel that the advancements presented here improve the use of imagery for GIS updating, especially now that image data arrive at constantly increasing rates. Our future plans include the investigation of the effect of coefficient selection in our differential snake solution. Our plan is to consider sample areas within an image as training sites and to transfer the set of coefficients over similar types of terrain and imagery. This would enable even higher levels of automation. Furthermore, we plan to integrate this method, which is aimed towards linear and curvilinear features like roads and rivers, with our least-squares based change detection (highly suitable for buildings and similar regular-type entities) in our spatiotemporal Gazetteer framework (Agouris et al., 2000).

Barzohar, M., M. Cohen, and I. Ziskind, 1997. Fast robust tracking of curvy partially occluded roads in clutter in aerial images, Automatic Extraction of Man-Made Objects from Aerial and Space Images II (A. Gruen, E. Baltsavias, and O. Henricsson, editors), Birkhaeuser Verlag, Basel, Switzerland, pp. 277–286. Baumgartner, A., C. Steger, H. Mayer, and W. Eckstein, 1999. Automatic road extraction based on multi-scale, grouping, and context, Photogrammetric Engineering & Remote Sensing, 65(7):777–785. Dal Poz, A., S. Gyftakis, and P. Agouris, 2000. Semi-automatic road extraction: Comparison of methodologies and experiments, Proceedings of ASPRS 2000, 22–26 May, CD-Rom, Washington, D.C. Gruen, A., O. Kuebler, and P. Agouris (editors.), 1995. Automatic Extraction of Man-Made Objects from Aerial and Space Images, Birkhaeuser Verlag, Basel, Switzerland 320 p. Gruen, A., and H. Li, 1997. Semi-automatic linear feature extraction by dynamic programming and LSB-snakes, Photogrammetric Engineering & Remote Sensing, 63(8):985–995. Gruen, A., E. Baltsavias, and O. Henricsson (editors) 1997. Automatic Extraction of Man-Made Objects from Aerial and Space Images II, Birkhaeuser Verlag, Basel, Switzerland 364 p. Kass, M., A. Witkin, and D. Terzopoulos, 1987. Snakes: Active contour models, Proceedings of 1st Int. Conf. on Computer Vision, 08–11 June, London, United Kingdom, pp. 259–268. Klang, D., 1998. Automatic detection of changes in road databases using satellite imagery, International Archives of Photogrammetry and Remote Sensing, 32 (Part 4):293–298. Lai, K.F., and R.T. Chin, 1993, On regularization, formulation and initialization of the active contour models (Snakes), Proceedings of Asian Conference on Computer Vision ACCV’93, 23–25 November, Osaka, Japan, pp. 542–545. Li, H., 1997. Semi-Automatic Road Extraction from Satellite and Aerial Images, Ph.D. Thesis, ETH, Zu¨rich, Switzerland, 161 p. Lukes, G. (editor.), 1998. Proceedings of DARPA Image Understanding Workshop, 21–23 November, Monterey California, Morgan Kaufmann, New York, N.Y., 1330 p. McKeown, D., and J. Denlinger, 1988. Cooperative methods for road tracking in aerial imagery, Proceedings of Computer Vision and Pattern Recognition, 05–09 June, Ann Arbor, Michigan, pp. 662–672. Nevatia, R., and K.R. Babu, 1980. Linear feature extraction and description, Computer Graphics and Image Processing, 13(4):257–269. Suetens, P., P. Fua, and A.J. Hanson, 1992. Computational strategies for object recognition, ACM Computing Surveys, 24(1):5–61.

Acknowledgments

Tan, K., and S. Tokinaga, 1999. The design of multi-stage fuzzy inference systems with smaller number of rules based upon the optimization of rules by using the GA, IEICE Transactions on Fundamentals of Electronics, Communications, and Computer Science, E82-A (9):1865–1873.

This work was supported by the National Science Foundation through CAREER award number IIS-9702233 and Digital Government award number DGI-9983445, and by BAE Systems.

To¨njes, R., and S. Growe, 1998. Knowledge based road extraction from multisensor imagery, International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 3/2, Columbus, OH, pp. 387–393.

References Agouris, P., S. Gyftakis, and A. Stefanidis, 1998. Using a fuzzy supervisor for object extraction within an integrated geospatial environment, International Archives of Photogrammetry and Remote Sensing, 32 (Part 3/1):191–195. Agouris, P., K. Beard, G. Mountrakis, and A. Stefanidis 2000. Capturing and modeling geographic object change: a spatiotemporal Gazetteer framework, Photogrammetric Engineering & Remote Sensing, 66(10):1241–1250.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

Vosselman, G., and J. de Knecht, 1995. Road tracing by profile matching and Kalman filtering, Automatic Extraction of Man-Made Objects from Aerial and Space Images (A. Gruen, O. Kuebler, and P. Agouris, editors), Birkhaeuser Verlag, Basel, pp. 265–274. Williams, D.J., and M. Shah, 1992. A fast algorithm for active contours and curvature estimation, CVGIP: Image Understanding, 55(1):14–26. (Received 18 September 2000; accepted 28 March 2001; revised 16 May 2001)

December 2001

1399