change detection via morphological comparative filters - ISPRS Annals

ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume III-3, 2016 XXIII ISPRS Congress, 12–19 July 2016, Prague, Czech Republic

CHANGE DETECTION VIA MORPHOLOGICAL COMPARATIVE FILTERS Y. V. Vizilter*. A. Y. Rubis, S. Y. Zheltov, O. V. Vygolov

State Research Institute of Aviation Systems (GosNIIAS), 125319, 7, Viktorenko str., Moscow, Russia – (viz,arubis,zhl,o.vygolov)@gosniias.ru Commission III, WG III/3

KEY WORDS: Change Detection, Mathematical Morphology

ABSTRACT: In this paper we propose the new change detection technique based on morphological comparative filtering. This technique generalizes the morphological image analysis scheme proposed by Pytiev. A new class of comparative filters based on guided contrasting is developed. Comparative filtering based on diffusion morphology is implemented too. The change detection pipeline contains: comparative filtering on image pyramid, calculation of morphological difference map, binarization, extraction of change proposals and testing change proposals using local morphological correlation coefficient. Experimental results demonstrate the applicability of proposed approach.

1. INTRODUCTION This paper addresses some theoretical and practical aspects of the of the change detection problem. It means detecting new or disappeared objects on images registered at different moments of time and possibly in various lighting, weather and season conditions. The most popular and challenging change detection problem appears in analysis of bi-temporal or multi-temporal spaceborne or airborne remote sensing data. A lot of change detection techniques are developed for remote sensing applications (Singh et al., 1989; Lu et al., 2004; Hussain et al., 2013). There are two main categories of change detection techniques: pixel-level and object-level. Pixel-based methods usually provide the attractive computational efficiency, but relatively low detection characteristics. In contrast, the object-based techniques usually provide the high detection quality, but require much more computational efforts. In this paper we present a new change detection technique based on generalized ideas of Morphological Image Analysis (MIA) proposed by Pyt’ev (Pyt’ev, 1993). The overview of modern MIA results is given in (Vizilter et al., 2015). The original morphological change detection approach is based on the analysis of difference between the test image and its projection to the shape of reference image. In our generalized approach the morphological filter-projector is substituted by the comparative morphological filter with weaker properties, which transforms the test image guided by the local shape of reference image. So, such approach implements some important properties of object-level image comparison immediately in the pixel-level image filtering. Due to this, we can talk about the morphological mid-level change detection. It should provide the desired compromise between the computational efficiency of pixel-based methods and detection quality of object-based techniques.

 

new change detection scheme based on morphological comparative filtering; new morphological filters based on guided contrasting.

We need to note that comparative filtering scheme is close in some sense to the guided filtering scheme (He et al., 2010; Zhang et al., 2014), but proposed morphological filters additionally satisfy the special mathematical properties that allows constructing the morphological theory and utilizing the traditional morphological tools for image comparison. The practical contributions of this paper are:  

new algorithmic pipeline for change detection in remote sensing data; implementation of change detection scheme based on both guided contrasting and diffusion morphology previously proposed in (Vizilter et al., 2014).

Qualitative experiments with proposed comparative filters based on guided contrasting are performed on a wide set of real images in different change detection tasks. Quantitative experiments with proposed change detection pipeline are performed on the public benchmark containing simulated aerial images. This paper is structured as follows. In the second section the related works are briefly described. The third section contains the theoretical basics, new ideas, schemes and algorithms. In the fourth section the experimental results are reported. 2. RELATED WORKS A lot of change detection techniques are developed for remote sensing applications (Singh et al., 1989; Lu et al. 2004; Hussain

The theoretical contributions of this paper are: * Corresponding author

This contribution has been peer-reviewed. The double-blind peer-review was conducted on the basis of the full paper. doi:10.5194/isprsannals-III-3-279-2016 279


et al. 2013). In (Hussain et al. 2013) two main categories of methods are pointed: pixel-based change detection (PBCD) and object-based change detection (OBCD). The PBCD category contains the direct, transform-based, classification-based and learning-based comparison of images at the pixel level. The OBCD category contains direct, classified and composite change detection at the object level. We start our brief overview from PBCD techniques and then go to OBCD. The simplest direct comparison techniques are the image difference (Lu et al., 2005) and image rationing (Howarth, Wickware, 1981). Image regression represents second image as a linear function of first (Lunetta, 1999). Change vector analysis (CVA) was developed for change detection in multiple image bands (Bayarjargal, 2006). Change vectors are calculated by subtracting pixel vectors of coregistered different-time dates. Principal component analysis (PCA) is applied for change detection in two main ways: applying PCA to images separately and then compare them using differencing or rationing (Richards, 1984) or merging the compared images into one set and then applying the PCA transform (Deng et al, 2008). Tasseled cap transformation (Kauth and Thomas, 1976) produces stable spectral components for long-term studies of forest and vegetation (Jin, Sader, 2005; Rogan et al., 2002). Some other texture-based transforms are developed in (Tomowski et al., 2011). Classification-based change detection contains the postclassification and composite classification. Post-classification comparison presumes that images are first rectified and classified, and then the classified images are compared to measure changes (Bouziani et al., 2010; Im and Jensen, 2005). The supervised (Ji et al., 2006) or unsupervised classification (Ghosh et al., 2011) can be of use. Unfortunately, the errors from classification are propagated into the final change map (Lillesand et al., 2008). In the composite or direct multidate classification (Lunetta et al., 2006) the rectified multispectral images are stacked together and PCA technique is applied to reduce the number of spectral components. Machine Learning algorithms are extensively utilized in change detection. Artificial Neural Networks (ANN) are usually trained for generating the complex non-linear regression between input pair of images and output change map (Pijanowski et al., 2005). The Support Vector Machine (SVM) approach based on (Vapnik, 2000) considers the finding change and no-change regions as a problem of binary classification in a space of spectral features (Huang et al., 2008; Bovolo et al., 2008). Other machine learning techniques applied for change detection are: decision tree (Im and Jensen, 2005), genetic programming (Makkeasorn et al., 2009), random forest (Smith, 2010) and cellular automata (Yang et al., 2008). Object-based techniques operate with extracted objects. The Direct Object change detection (DOCD) approach is based on the comparison of object geometrical properties (Lefebvre et al., 2008), spectral information (Miller et al., 2005; Hall and Hay, 2003) or texture features (Lefebvre et al., 2008; Tomowski et al., 2011). In Classified Objects change detection (COCD) approach the extracted objects are compared based on the geometry and class labels (Chant, Kelly, 2009). The framework based on post-classification (Blaschke, 2005) presumes extracting objects and independently classifying them (Im and Jensen, 2005; Hansen and Loveland, 2012). Multitemporalobject change detection presumes that the joint segmentation is

performed once for stacked (composite) images (Conchedda et al., 2008). In this paper we present a new change detection technique based on generalized ideas of Morphological Image Analysis (MIA) proposed by Pyt’ev (Pyt’ev, 1993) and further developed in (Evsegneev, Pyt’ev, 2006; Vizilter, Zheltov, 2012; Pyt’ev, 2013; Vizilter et al., 2014). Let’s note that terms “morphology”, “morphological filter” and “morphological analysis” refer to Mathematical Morphology (MM) proposed by Serra (Serra,1982) as well as to MIA. These theories of shape have a common algebraic basis (lattice theory), but different tasks and tools. The overview of MIA and its relation to MM is given in (Vizilter et al., 2015). Morphological change detection approach is based on the analysis of morphological difference map formed as a difference between test image and its morphological projection to the shape of sample image (see section 3.1 for details and formulas). 3. METHODOLOGY This section contains the description of proposed mid-level change detection methodology based on morphological comparative filtering. 3.1 Morphological image analysis and comparative filtering scheme Let’s consider the mosaic image model utilized in MIA

f x, y    f i  Fi x, y , n

(1)

i 1

1,  Fi ( x, y)    0,

if ( x, y)  Fi ; otherwise.

(2)

where n – number of connected regions of tessellation F of the image frame R2, F={F1,…,Fn}; f=(f1,…,fn) – intensity values; Fi(x,y){0,1} – support function of i-th region. The tessellation should be obtained by some image segmentation procedure. The mosaic shape is a set of images with the same frame tessellation:   F   f  x, y    



n

f 

i Fi

x, y , f   f1,, f n , f  R n 

(3)

 

i 1

For any image g(x,y)L2() the projection onto the shape F is defining as averaging pixel intensities of image g over each area of mosaic shape F: g F x, y   PF g x, y    g Fi  Fi x, y , n

g F   F , g   F i

i

i

(4)

i 1

2

, i  1, , n.

The similarity of images f(x,y) and g(x,y) is estimated by the normalized morphological correlation coefficients (MCC) K M ( g, F )  K M ( f , G) 

|| PF g  PO g || , || g  PO g ||

|| PG f  PO f || || f  PO f || ,

(5)



where O(x,y)  const – any constant-valued (flat) image, PO f = fo  mean(f(x,y)) and PO g = go  mean(g(x,y)) – mean values of projected images. Morphological extraction of differences on image g(x,y) relative to shape F is performed via comparison of g(x,y) with its projection to F, and vice versa:

g F  g  PF g , f G f  PG f

3.2 Comparative filters based on guided contrasting Let’s consider the comparative filter based on the local linear correlation. Let w(x,y) is a sliding window at position (x,y). The guided local contrasting filter is defined as:

w  f , g x, y   go w( x, y ) x, y  

(6)



 K f w( x , y ) , g w ( x , y )

Let’s note that this MIA scheme is basically asymmetrical due to different roles of input images: test image is projected to the shape; sample (or reference) image determines the shape. In this paper we propose the weaker scheme of morphological image analysis that excludes the ideas of shape and projection, but preserves the idea of asymmetrical comparative filtering for robust similarity estimation and change detection. Let two images are given: sample (or reference) image f(x,y) and test image g(x,y). The mapping that takes these two images as input data and forms the filtered version gf(x,y) of test image depending on its relation with sample image we refer as а comparative filter:  f , g  : L2 ()  L2 ()  L2 () .

(7)

If sample image f is fixed, then comparative filter takes the usual form with one input and one output image:

 f  g    f , g 

(8)

We call such filter the morphological comparative filter if it satisfies the following conditions: 1)  f , g   g ; 2)   f , f   f ;

(9)

3)  f , o  o ,

K  f , g  

 f , g   g o g  go

w( x , y ) o

x, y 

(12)

 g ( x, y ), if (u, v)  w( x, y ); g w( x, y ) u, v    0  otherwise,

g o w( x, y ) x, y   mean( g w( x , y ) ( x, y )),





K f w( x , y ) , g w( x , y )  

f f

w( x , y )

w( x , y )

 f o w ( x , y ) , g w ( x , y )  g o w( x , y )



 f o w ( x , y ) g w( x , y )  g o w ( x , y )

where gw(x,y)(u,v) is g(x,y) localized in a window w(x,y), gow(x,y) – mean value of g(x,y) in a window w(x,y), K(fw(x,y),g w(x,y)) – local linear correlation coefficient in a sliding window w(x,y). Such filter (12) satisfies conditions (9), preserves the similar details and smooths the non-similar details on a test image g guided by sample image f. So, the difference map (11) based on filter (12) can be applied for change detection. The size of details is determined by the size of window w(x,y). If we need to detect details of different size, one can use the image pyramid. In practice we proceed to guided contrasting filter with local search (in a zone p(x,y)) that provides the robustness relative to weak geometrical discrepancy of images:

 w. p  f , g x, y   g o w( x , y ) x, y    K max



 f , g g x, y   g   max     K  f

K max f , g w( x , y )

where o(x,y)  const – any constant-valued (flat) image. The first and third conditions describe the smoothing properties. Second condition describes the exact matching property: filter should preserve the test image if it is equal to sample one. Due to the properties (9) the similarity of test g(x,y) and sample f(x,y) can be estimated like in MIA by the morphological correlation coefficient (MCC) of the form

 g x, y   g

w( x , y )

w( x , y )

x, y 

w(u ,v )

, g w( x , y )

o

u ,v  p x , y

(13)



The scheme of comparative filtering based on guided contrasting (Fig.1) demonstrates the main idea of this approach. At first we perform the local smoothing of test image. Then we estimate the local similarity of extracted details and recover or not recover them depending on this local similarity. In result, similar details should be recovered, but non-similar details should be extremely smoothed.

(10)

Morphological difference map (MDM) of image g(x,y) relative to image f(x,y) can be calculated in analogous way

  g F  g  ( f , g ) .

(11)

It is easy to see that if sample image f(x,y) satisfies the mosaic model (1), then the mapping (f, g) = PF g satisfies to (7)-(9). Thus morphological projector is a particular case of morphological comparative filter, but morphological comparative filter should not be a projector in general case. Consequently, the described comparative filtering scheme is a correct generalization of MIA including the formulas for MCC (10) and MDM (11) generalizing the MIA formulas (5) and (6) correspondingly.

Figure 1. The scheme of comparative filtering based on guided local contrasting. This simple and general idea can be expressed in the following form of generalized guided contrasting filter



 aw  f , g x, y   g o

 x, y   w( x , y )  g x, y   g o w( x, y ) x, y   a f , g





w( x , y )







(14)



a f , g w( x, y) [0,1]; a g , g w( x, y)  1; a o, g w( x, y)  0,

where a(f,g w(x,y)) is a local similarity coefficient (LSC) of test image fragment g w(x,y) with sample f. Any LSC with such properties generates the filter satisfying (9). Different variants of LSC a(f,g w(x,y)) can be considered for change detection task: - local MCC (5) (Pyt’ev, 1993); - LSC based on mutual information (Maes, 1997); - local mean square MCC (Vizilter, Zheltov, 2012); - geometrical correlation coefficients (Vizilter, Zheltov, 2012), etc. In this paper we implement and explore just the guided contrasting filters (12)-(13) based on local linear correlation. But in the future all of these variants can be implemented and tested both in change detection and in image matching tasks. The main advantage of proposed guided contrasting filters relative to MIA projectors is the elimination of sample image segmentation step that allows obtaining the more precise and computationally efficient solutions. Additionally, this approach provides the robustness relative to weak geometrical discrepancy of images.

Then diffusion filter Pf (16) turns into morphological projector PF (4). So, MIA is a particular case of DM. In general case diffusion model (15) is formed as a heat kernel proposed in (Coifman et al, 2007):  v x, y   v u, v  2  f f   h f ( x, y, u, v)  exp   ,    

(18)

where vf(x,y) is a feature vector describing the sample image f(x,y) in some neighborhood (local window) w(x,y) around the point (x,y); >0 is a tuning parameter controlling the sensibility to feature vectors similarity. But the diffusion filtering with such heat kernel (18) is a time-consuming procedure. So, we prefer the diffusion filters proposed in (Vizilter et al, 2014) and based on point feature descriptor iLBP (intensity + LBP): iLBP x, y   mx, y , LBPx, y ,

(19)

where m(x,y) – mean value of image in a window w(x,y); LBP(x,y) – threshold local binary pattern LBP (Ahonen, 2004) calculated as a binary vector for central pixel (x,y) based on a comparison of its value and values of its neighbors in a window w(x,y). If the value of neighbor pixel is less than the value of central pixel and the difference between them is greater than threshold, then the corresponding bit is set to 1, otherwise – to 0. Correspondingly the heat kernel is

3.3 Comparative filters based on diffusion operators h f ( x, y, u, v)  exp(  m f x, y   m f u, v   2

The theory of diffusion maps was proposed in (Lafon, 2004), (Coifman, Lafon 2006) and (Coifman et al, 2007). It is based on notions of “heat kernels” and “heat dissipation” introduced in (Belkin, Niyogi, 2001) in the context of Laplacian Eigenmap technique for nonlinear dimensionality reduction. In (Vizilter et al, 2014) the generalized MIA formalism called Diffusion Morphology (DM) was proposed based on heat kernels and diffusion maps. It was developed as an image-to-shape matching technique, but it looks reasonable to try this approach in the change detection task too. Moreover, the connection between DM and MIA allows considering morphological diffusion filters as comparative filters in the sense stated above. The diffusion shape model for image f is introduced as a heat kernel hf(x,y,u,v): [0,1], such that h f  x, y, x, y   h f  x, y, u , v ;

(15)

h f  x, y, u , v   h f u , v, x, y ;

  h f x, y, u, v  dx dy du dv  . 2



Corresponding diffusion filter Pf is formed as a linear convolution with normalized diffusion kernel pf(x,y,u,v), such that Pf g x, y    p f x, y, u, v  g u, v  du dv, 

p f x, y, u, v   h f x, y, u, v   h f x, y, a, b  da db,

 p f x, y, u, v  du dv  1.



(16)

If f(x,y) satisfies the mosaic model (1) and heat kernel is formed as a relation “(x,y) and (u,v) are in the same flat region” if   Fi x, y  Fi u, v   1; n

i 1

otherwise

where dham – Hamming distance, β – tuning parameter balancing the importance of intensity and LBP parts in iLBP. Local binary patterns are stored as bit fields, and the computation of Hamming distance is performed via bitwise XOR operation. The exponent is calculated using table values. Mean value in a sliding window is computed by a fast algorithm with sliding sum recalculation. Due to this, the usage of iLBP allows both increasing the computational speed and obtaining heat kernels very similar to (18). As stated above, the projective mapping (f, g) = PF g (4) satisfies to definition of morphological comparative filter and properties (9). From the functional point of view the generalized diffusion mapping (f, g) = Pf g (16) should be a kind of comparative filtering too, because it is a smoothing filter, and it transforms the test image in accordance with diffusion shape of sample image. Unfortunately, it does not formally match the exact matching property (9). But in this case we can talk about the soft matching property: diffusion filters preserve images of sample shape essentially better than images of other shapes. So, we may refer diffusion filters as soft comparative filters and use the corresponding diffusion difference map of form (11) for solution of change detection task. 3.4 Change detection pipeline based on comparative filtering



1,  hF ( x, y, u, v)   0,

(20)

 d ham ( LBPf ( x, y ), LBPf (u, v)),

For the task of change detection in long-range (spaceborne or airborne) remote sensing data we propose the new change detection pipeline based on comparative filtering. It contains the following steps:

(17)



1. Comparative filtering (7) using the image pyramid; 2. Calculation of corresponding morphological difference map (11); 3. Binarization and filtering of morphological difference map; 4. Forming change proposals; 5. Testing change proposals using local morphological correlation coefficient (5); 6. Forming the output binary map of changes. At the first stage of pipeline we apply the morphological comparative filters described above. The pipeline is the same both for guided contrasting (12)-(13) and for diffusion filtering (16)-(20). The use of image pyramid allows detecting details of different size. Morphological difference map obtained at second stage is binarized using graph cut technique (Boykov and Kolmogorov, 2004). Then this binary image is filtered sequentially by morphological closing and opening filters (Serra, 1982) with small disk-shaped structured element. Such filtering allows deleting noisy regions of binarized difference map. The list of change proposals is formed via calculation of minimal bounding rectangles for all connected regions of filtered binarized difference map. Then the each change proposal is checked by the value of local MCC (5) compared with a pre-learned threshold. Local MCC is calculated in a weakly expanded proposal rectangle. The expansion of rectangle is performed in order to add some small neighbourhood of the proposed object. If the value of MCC is greater than threshold, corresponding connected region of filtered binarized difference map is painted on the output binary map of relative changes. Let’s note that this scheme is asymmetrical – it provides the detection of changes in (new) test image g relative to (old) reference image f. If we need to find all differences in both images, it is required to repeat the procedure two times: at first using g as test and f as reference, and then using g as reference and f as a test.

a)

b)

c)

d)

e) f) Figure 2. Stages of change detection pipeline: a) reference image; b) test image; c) morphological difference map (MDM); d) binarized MDM; e) change proposals (regions of filtered binarized MDM); f) accepted change proposals.

We also need to comment the presence of traditional MCC in our pipeline. We have stated above that MCC is not stable and/or fast due to problems with image segmentation. But this problem exists at the global image scale. Coarse or unstable segmentation leads to false object proposals or lost objects. But the mosaic shape of small local areas containing the change proposals is simple enough. The simplest histogram-based segmentation with small number of levels provides local segmentation that looks fine for proposals testing. From the other hand, the properties of morphological projector are essentially different from the properties of comparative filters. So, testing proposals created by comparative filters via MCC means the combination of evidences from independent information sources. Such combination is preferable from the statistical point of view. It provides the more reliable detection results. 4. EXPERIMENTS The results of experimental exploration of both comparative filtering and proposed change detection pipeline are reported in this section. In the first part of section some examples of guided contrasting and corresponding morphological difference map forming are demonstrated applying to real images for different scene types and change detection cases. In the second part the results of change detection experiments on the public benchmark containing simulated aerial images are described. 4.1 Qualitative change detection experiments A lot of qualitative experiments with comparative filters based on guided contrasting are performed on a wide set of real images. Different types of scenes and image acquisition conditions are considered. Fig.3 and Fig.4 demonstrate some examples of morphological difference map forming based on comparative guided contrasting filtering. The first row contains reference images, second row – test images; third row – morphological difference maps. Different columns illustrate scenes and changes of different types. In the first column of Fig.3 the example of outdoor video surveillance change detection case is shown. Such case presumes the short-term changes in illumination conditions combined with presence of moving or appeared/disappeared objects. Second column of Fig.4 demonstrates the building construction case that requires comparison of buildings at different stages of construction based on images captured in different weather and season conditions from the close but not exactly the same viewpoint. In the first column of Fig.4 the example of changes in indoor (inoffice) close-range scene is shown. Some new objects appear on a table in a test image. The second column of Fig.4 demonstrates the most popular case of change detection task in remote sensing imagery (spaceborne or airborne). Such qualitative experiments allow concluding that in all considered cases the proposed approach for difference map forming based on guided contrasting filtering provides reasonable scene change proposals and demonstrates the enough robustness relative to changes in lighting and other image capturing conditions. From the other hand, some true image shape changes are extracted which are not required to be detected as scene changes from the semantical point of view. For example, clouds and flowers in a building construction case really appear in a scene, but they should not be of interest regarding the building construction stage comparison. So, some additional analysis of formed morphological difference map is



needed for final testing of the formed change proposals based on other type of task-specific information. It means that guided contrasting filtering and corresponding morphological difference maps can be useful as parts of different task-oriented change detection pipelines. Such pipeline for change detection in remote sensing data is described above. The results of its testing are reported in next subsection.

1000 corresponding 800x600 ground truth masks. Each pair consists of one reference and one test image. Some of image pairs contain scene changes and illumination differences. The dataset consists of 100 different scenes with moderate surface relief and several objects (trees, buildings etc.). Each scene is rendered with various viewpoints. The cameras are distributed at steps of 10 degrees on a circle of radius 100 meters at approximately 250 meters high, and with a fixed tilt of about 70 degrees. All images are modelled with a ground resolution of about 50cm per pixel. The methodology of our experiments is the following. We select a subset of 100 reference and test image pairs for 50 different scenes with 0 degrees relative camera angle. As proposed in (Bourdis et al., 2011) we compare the detection results with respect to the ground truth at pixel level, but calculate the precision and recall values at the object (region) level. In order to do this, we from the list of ground truth objects and list of detected objects (accepted regions of filtered binarized morphological difference map). Then we perform the object-toobject comparison via computing of object intersection area. If the intersection area is more than 50% then we decide that objects match each other. The numbers of true and false object detections determine the corresponding precision and recall values.

Figure 3. Examples of morphological difference maps based on guided contrasting: first row – reference images; second row – test images; third row – morphological difference maps.

We implement and test our pipeline with following parameters: guided contrasting window size is 77 pixels; number of pyramid levels is 3; the size of disk structuring element in MM opening and closing is 5 pixels, the threshold value for morphological correlation coefficient at the final testing step is 0.5. The obtained results are:  for change detection based on diffusion morphological filtering – Precision=0.61 and Recall=0.6;  for change detection based on guided contrasting – Precision=0.6 and Recall=0.64.

a) b) c) Figure 5. Example of simulated data from benchmark: a) reference image; b) test image; c) ground truth mask.

Figure 4. Examples of morphological difference map based on guided contrasting: first row – reference images; second row – test images; third row – morphological difference maps. 4.2 Quantitative change detection experiments In our experiments with proposed change detection pipeline for long-range remote sensing we use the public Change Detection dataset introduced in (Bourdis et al., 2011). This dataset contains 1000 pairs of 800x600 simulated aerial images and

The reported result of approach (Bourdis et al., 2011) on this database is about Precision=0.51 and Recall=0.52 (by the graph). It is not the totally correct comparison, because this paper utilizes a little bit different testing methodology, but it seems reasonable to state that our results are at least not worse than competing ones. So, we can conclude that the proposed pipeline is useful for change detection in long-range remote sensing data. 5. CONCLUSION This paper presents the new scheme of morphological comparison filtering that generalizes the morphological image analysis (MIA) scheme proposed by Pyt’ev (Pyt’ev, 1993). Our morphological scheme excludes the MIA ideas of shape and projection, but preserves the idea of asymmetrical comparative filtering for robust similarity estimation and change detection. The mapping that takes two images (reference and test) as input



data and forms the filtered version of test image depending on its relation with reference image is called а comparative filter. Such filter is called the morphological filter, if it is a smoothing mapping that preserves any constant (flat) image and preserves the test image equal to the reference image.For implementation of this scheme we propose a new class of morphological filters based on guided contrasting. The idea of guided contrasting is that the local contrast (energy) of filtered test image is controlled by its local similarity with reference image. If we use the similarity estimation with local search, then such filters demonstrate the robustness relative to weak geometrical discrepancy of compared images. Another essential advantage of guided contrasting filters relative to traditional morphological projectors is the elimination of unstable image segmentation step. The new change detection pipeline based on comparative filtering is proposed for analysis of bi-temporal long-range (spaceborne or airborne) remote sensing data. This pipeline contains comparative filtering on image pyramid, calculation of morphological difference map, binarization, extraction of change proposals and testing change proposals using local morphological correlation coefficient. We implement this pipeline based on both guided contrasting filters and morphological diffusion filters previously proposed in (Vizilter et al, 2014) for shape-based matching. Qualitative experiments with guided contrasting filtering in different change detection tasks demonstrate that they provide the reasonable scene change proposals and demonstrate the enough robustness relative to changes in lighting and other image acquisition conditions. Quantitative experiments on the public benchmark containing simulated aerial images demonstrate that the proposed pipeline is useful for change detection in long-range remote sensing data. The future work on proposed morphological schemes will be connected with development and implementation of new comparative filters and new change detection pipelines for different types of tasks. In particular, new guided contrasting filters can be obtained via different combinations of several smoothing procedures and local similarity estimators. The main experimental work will consist in massive testing of implemented comparative filters and corresponding change detection pipelines on the large datasets containing both simulated and real images. ACKNOWLEDGEMENTS This work was performed with the support of Grant of RF President for Leading Science schools of Russia (НШ7247.2016.8) and by Grants 15-08-99580a, 14-07-00914a of Russian Foundation for Basic Research (RFBR), Grant 16-1100082 of Russian Science Foundation (RSF). REFERENCES Ahonen, T., Hadid, A., Pietikainen, M., 2004. Face recognition with local binary patterns. ECCV, pp 469–481. Bayarjargal, Y., Karnieli, A., Bayasgalan, M., Khudulmur, S., Gandush, C., Tucker, C.J., 2006. A comparative study of NOAA–AVHRR derived drought indices using change vector analysis. Remote Sensing of Environment, 105, pp. 9–22. Belkin M., Niyogi P., 2001. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems, 14, pp 585–591.

Blaschke, T., 2005. Towards a framework for change detection based on image objects. Göttinger Geographische Abhandlungen 113, 1–9. Bourdis, N., Marraud, D., Sahbi, H., 2011. Constrained optical flow for aerial image change detection. Geoscience and Remote Sensing Symposium (IGARSS), 2011 IEEE International, pp. 4176 – 4179. Bouziani, M., Goïta, K., He, D.-C., 2010. Automatic change detection of buildings in urban environment from very high spatial resolution images using existing geodatabase and prior knowledge. ISPRS Journal of Photogrammetry and Remote Sensing, 65, pp. 143–153. Bovolo, F., Bruzzone, L., Marconcini, M., 2008. A approach to unsupervised change detection based semisupervised SVM and a similarity measure. Transactions on Geoscience and Remote Sensing 46, 2082.

novel on a IEEE 2070–

Boykov, Y., Kolmogorov, V., 2004. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 26(9), pp. 1124–1137. Chant, T.D., Kelly, M., 2009. Individual object change detection for monitoring the impact of a forest pathogen on a hard wood forest. Photogrammetric Engineering & Remote Sensing, 75, pp. 1005–1013. Lafon, S., 2004. Diffusion maps and geometric harmonics. PhD thesis, Yale University, Dept of Mathematics & Applied Mathematics. Coifman, R., Lafon, S., 2006. Diffusion maps, Appl. Comp. Harm. Anal., Volume 21, Issue 1, pp. 5–30. Coifman, R., Lafon, S., Maggioni, M., Keller, Y., Szlam, A., Warner, F., Zucker, S., 2007. Geometries of sensor outputs, inference and information processing, NIPS. Conchedda, G., Durieux, L., Mayaux, P., 2008. An object-based method for mapping and change analysis in mangrove ecosystems. ISPRS Journal of Photogrammetry and Remote Sensing, 63, pp. 578–589. Deng, J., Wang, K., Deng, Y., Qi, G., 2008. PCA-based landuse change detection and analysis using multitemporal and multisensor satellite data. International Journal of Remote Sensing, 29, pp. 4823–4838 Evsegneev, S., Pyt’ev, Yu, 2006. Analysis and Recognition of Piecewise Constant Texture Images. Pattern Recognition and Image Analysis, 16(3), pp. 398–405. Hall, O., Hay, G.J., 2003. A multiscale object-specific approach to digital change detection. International Journal of Applied Earth Observation and Geoinformation, 4, pp. 311–327. He, K., Sun, J., Tang, X., 2010. Guided image filtering. Computer Vision – ECCV 2010, Vol. 6311, pp. 1–14. Howarth, P., Wickware, G., 1981. Procedures for change detection using Landsat digital data. International Journal of Remote Sensing, 2, pp. 277–291.



Huang, C., Song, K., Kim, S., Townshend, J.R.G., Davis, P., Masek, J.G., Goward, S.N., 2008. Use of a dark object concept and support vector machines to automate forest cover change analysis. Remote Sensing of Environment, 112, pp. 970–985. Hussain, M., Chen, D., Cheng, A., Wei, H., Stanley, D., 2013. Change detection from remotely sensed images: From pixelbased to object-based approaches. ISPRS Journal of Photogrammetry and Remote Sensing, 80, pp. 91–106. Im, J., Jensen, J., 2005. A change detection model based on neighborhood correlation image analysis and decision tree classification. Remote Sensing of Environment, 99, pp. 326– 340. Ji, W., Ma, J., Twibell, R.W., Underhill, K., 2006. Characterizing urban sprawl using multi-stage remote sensing images and landscape metrics. Computers, Environment and Urban Systems, 30, pp. 861–879. Jin, S., Sader, S., 2005. Comparison of time series tasseled cap wetness and the normalized difference moisture index in detecting forest disturbances. Remote Sensing of Environment, 94, pp. 364–372. Kauth, R., Thomas, G., 1976. The Tasselled Cap – A Graphic Description of the Spectral-Temporal Development of Agricultural Crops as Seen by LANDSAT, LARS Symposia, 4B, pp.41–51. Lefebvre, A., Corpetti, T., Hubert-Moy, L., 2008. Objectoriented approach and texture analysis for change detection in very high resolution images. In: Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008. IEEE International, pp. 663–666. Lu, D., Mausel, P., Batistella, M., Moran, E., 2005. Land-cover binary change detection methods for use in the moist tropical region of the Amazon: a comparative study. International Journal of Remote Sensing 26, 101–114. Lu, D., Mauselb. P., Brondízioc, E., Moran, E., 2004. Change detection techniques. Int. J. Remote Sens., 25(12), pp. 23652401. Ludeke, A., Maggio, R., Reid, L, 1990. An analysis of anthropogenic deforestation using logistic regression and GIS. Journal of Environmental Management, 31, pp. 247–259. Lunetta, R.S., 1999. Applications, project formulation, and analytical approach. In: Lunetta, R.S., Elvidge, C.D. (Eds.), Remote Sensing Change Detection: Environmental Monitoring Methods and Applications. Taylor & Francis, London, pp. 1– 19. Lunetta, R.S., Knight, J.F., Ediriwickrema, J., Lyon, J.G., Worthy, L.D., 2006. Land-cover change detection using multitemporal MODIS NDVI data. Remote Sensing of Environment, 105, pp. 142–154.

genetic programming in a semi-arid watershed. Journal of Environmental Management 90, 1069–1080. Miller, O., Pikaz, A., Averbuch, A., 2005. Objects based change detection in a pair of gray-level images. Pattern Recognition, 38, pp. 1976–1992. Pijanowski, B.C., Pithadia, S., Shellito, B.A., Alexandridis, K., 2005. Calibrating a neural network-based urban change model for two metropolitan areas of the Upper Midwest of the United States. International Journal of Geographical Information Science 19, 197–215. Pyt’ev, Yu., 2013. Oblique projectors and relative forms in image morphology. J of Computational Mathematics and Mathematical Physics, 53(1), pp. 1916–1937. Pyt’ev, Yu., 1993. Morphological Image Analysis. Pattern Recognition and Image Analysis. Vol. 3. No 1, pp. 19–28. Richards, J., 1984. Thematic mapping from multitemporal image data using the principal components transformation. Remote Sensing of Environment, 16, pp. 35–46. Rogan, J., Franklin, J., Roberts, D., 2002. A comparison of methods for monitoring multitemporal vegetation change using Thematic Mapper imagery. Rem. Sens. of Env. 80, pp.143–156. Serra, J., 1982 Image Analysis and Mathematical Morphology. Academic Press, Inc. Orlando, USA. Singh, A., 1989. Review article digital change detection techniques using remotely-sensed data. Int. J. Remote Sens., 10(6), pp. 989-1003. Smith, A., 2010. Image segmentation scale parameter optimization and land cover classification using the Random Forest algorithm. Journal of Spatial Science 55, 69–79. Tomowski, D., Ehlers, M., Klonus, S., 2011. Colour and Texture Based Change Detection for Urban Disaster Analysis, Urb. Rem. Sens. Event (JURSE), 2011 Joint, pp. 329–332. Vapnik, V.N., 2000. The Nature of Statistical Learning Theory, second ed. Springer, New York. Vizilter, Yu., Gorbatsevich, V. S., Rubis, A., Zheltov, S, 2014. Shape-Based Image Matching Using Heat Kernels and Diffusion Maps. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-3, pp. 357–364. Vizilter, Yu., Pyt’ev, Yu., Chulichkov, A., Mestetskiy, L., 2015 Morphological Image Analysis for Computer Vision Applications. Computer Vision in Control Systems-1. Mathematical Theory. Intelligent Systems Reference Library 73, Springer International Publishing Switzerland, pp. 9–58. Vizilter, Yu., Zheltov, S., 2012. Geometrical correlation and matching of 2d image shapes. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., I-3, pp. 191–196.

Maes, F., Collignon, A., Vandermeulen, D., Marchal, G. and Suetens, P., 1997. Multimodality Image Registration by Maximization of Mutual Information. IEEE Transactions on Medical Imaging, Vol. 16, No. 2, April, pp.187-198.

Yang, Q., Li, X., Shi, X., 2008. Cellular automata for simulating land use changes based on support vector machines. Computers & Geosciences 34, 592–602.

Makkeasorn, A., Chang, N.-B., Li, J., 2009. Seasonal change detection of riparian zones with remote sensing images and

Zhang, Q., Shen, X., Xu, L., Jia, J., 2014. Rolling Guidance Filter. Computer Vision – ECCV 2014, Vol. 8691, pp. 815– 830.


change detection via morphological comparative filters - ISPRS Annals

change detection via morphological comparative filters - ISPRS Annals

Suggest Documents