Super-Resolution Land Cover Mapping Based on ... - IEEE Xplore

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination. IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

1

Super-Resolution Land Cover Mapping Based on Multiscale Spatial Regularization Jianlong Hu, Yong Ge, Member, IEEE, Yuehong Chen, and Deyu Li

Abstract—Super-resolution mapping (SRM) is a method for allocating land cover classes at a fine scale according to coarse fraction images. Based on a spatial regularization framework, this paper proposes a new regularization method for SRM that integrates multiscale spatial information from the fine scale as a smooth term and from the coarse scale as a penalty term. The smooth term is considered a homogeneity constraint, and the penalty term is used to characterize the heterogeneity constraint. Specifically, the smooth term depends on the local fine scale spatial consistency, and is used to smooth edges and eliminate speckle points. The penalty term depends on the coarse scale local spatial differences, and suppresses the over-smoothing effect from the fine scale information while preserving more details (e.g., connectivity and aggregation of linear land cover patterns). We validated our method using simulated and synthetic images, and compared the results to four representative SRM algorithms. Our numerical experiments demonstrated that the proposed method can produce more accurate maps, reduce differences in the number of patches, visually preserve smoother edges and more details, reject speckle points, and suppress over-smoothing. Index Terms—Fraction images, heterogeneity, homogeneity, multiscale, regularization, remote sensing, spatial dependence, super-resolution mapping (SRM).

I. I NTRODUCTION

S

UPER-RESOLUTION mapping (SRM) [1], [2] is commonly applied as a postprocessing method for spectral unmixing. It predicts the locations of land cover classes in coarse pixels using fraction images derived from spectral unmixing. SRM often requires two assumptions [3]–[13]. The first is a general assumption of the spatial dependence principle, i.e., spatially close observations of a given property are more Manuscript received June 09, 2014; revised October 24, 2014; accepted January 27, 2015. This work was supported in part by the National Natural Science Foundation of China under Grant 41471296, and in part by the Key Technologies Research and Development Program of China (2012BAH33B01). (Corresponding author: Yong Ge.) J. Hu and D. Li are with the School of Computer and Information Technology, Shanxi University, Taiyuan 030006, China (e-mail: weilong@ sxu.edu.cn; [email protected]). Y. Ge is with the State Key Laboratory of Resource and Environmental Information System, Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China, and also with Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing 210023, China (e-mail: [email protected]). Y. Chen is with the State Key Laboratory of Resource and Environmental Information System, Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/JSTARS.2015.2399509

similar to distant observations. The second is that pixels at the fine scale (called subpixels) are all pure pixels and are assigned to specific land cover classes. These two assumptions are the default premises for SRM in this paper. The ill-posed nature of SRM inverse problems has led to a number of spatial regularization methods for obtaining approximately optimal SRM solutions [5], [7], [14]–[19]. These methods add a spatial regularization or smooth term to the objective function. Regularization methods can be split into two categories according to the optimized objective function. In the first, the objective function is the spectral data fidelity that uses spectral vectors as its input. The most representative of these types are the Markov random field (MRF)-based methods [5], [14]–[18]. They use the posterior probability as the data fidelity and the prior probability from the spatial dependence as the regularization term. Kasetkasem et al. [14] first introduced MRF to SRM, followed by Tolpekin and Stein [15], who quantified the effects of land cover class spectral separability based on MRF. An application of MRF-based SRM for identifying urban trees was proposed by Ardila [16]. Ling et al. [5] defined an objective function based on the linear unmixing model with spatial regularization. Li et al. [17] presented an adaptive method for determining the smooth parameter in the MRF model for SRM. Another type of method that uses spectral data as input is based on the observation model in [19]. It uses the error determined using the observation model as the data fidelity, and a prior as the regularization term. MRF-based SRM considers the spectral and spatial constraints simultaneously. However, the uncertainty of the unmixing process in SRM inevitably affects the results. In the second category of methods, the objective function is the fraction data fidelity. It uses fraction images as input, and is not affected by unmixing [7]. In this method, the SRM problem is considered as an objective optimization problem. We minimize the difference between the final map and input fraction images, using the fine scale spatial dependence as a regularization term. Although this method performs very well, the final map is often too smooth, similar to SRM–MRF. This is because it only uses the spatial dependence at a fine scale. Moreover, the additional prior information has been used to enhance the resolution of land cover mappings in methods such as bitemporal different spatial resolution images [20], former fine-resolution maps [21], high–low spatial resolution image databases [22], and high-resolution color images [23]. This study focuses on the regularization framework of a single low-resolution image for SRM, which was proposed by Ling et al. [7]. Because of the limitation introduced by only

1939-1404 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination. 2

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

using the fine-resolution spatial dependence (as discussed in [6] and [8]), this study extends the framework and proposes a multiscale spatial regularization (MSSR) for SRM (SRMMSSR). We achieve this by integrating the spatial dependence at both fine and coarse scales, which we use as the regularization terms. The proposed method maximizes the fine scale spatial dependence to smooth the edges, and uses the coarse scale spatial heterogeneity as a penalty for suppressing oversmoothing. Because of this, SRM-MSSR can handle local details such as the connectivity of linear land cover patterns. We demonstrated the effectiveness of our method by comparing it with four other SRM algorithms: spatial attraction-based SRM (SRM-SA) [9], MRF-based SRM (SRM-MRF) [14], SRM with hybrid intra- and inter-pixel dependence (SRM-HIIPD) [6], and regularization-based SRM (SRM-REG) [7]. Our results show that SRM-MSSR with appropriate parameters was more accurate and more effectively mapped the ground connectivity of land cover patterns, when compared with the other four SRM algorithms. The remainder of this paper is organized as follows. Section II briefly reviews the spatial regularization framework of SRM, as proposed by Ling et al. [7]. Section III describes the proposed SRM-MSSR method, and our experiments are presented in Section IV. We give our concluding remarks in Section V.

II. S PATIAL R EGULARIZATION -BASED SRM We assume that a remotely sensed image with a coarse spatial resolution has m rows and n columns, and that the number of land cover classes is c. Let Y = (y.l )1×c , l = 1, . . . , c, be the fraction images produced by spectral unmixing or down-sampling, where y.l = (yi,l )mn×1 , i = 1, . . . , mn, is a column vector composed of yi,l ∈ [0, 1], the fraction value of pixel i for land cover class l at the coarse resolution. Setting the scale factor to s, the SRM result is the fine-resolution land cover mapping X = (x.l )1×c = (xj. )msns×1 , where x.l = (xj,l )msns×1 , j = 1, . . . , msns, and xj. = (xj,l )1×c , l = 1, . . . , c. Now, xj,l ∈ {0, 1} is defined by (1), which is a logical value for subpixel j that represents whether it is assigned to land cover class l. The subpixel is considered pure at the fine resolution and is assigned to a unique land cover class, c which means that l=1 xj,l = 1 ⎧ ⎨ 1 if subpixel j is assigned to land cover class l; xj,l = (1) ⎩ 0 otherwise. We also assume that the fraction image is generated by downsampling the SRM, and that the general observation model for SRM can be expressed as Y = DX + N

(2)

where D = (di. )mn×1 , i = 1, . . . , mn, is a mn × msns down-sampling matrix and N = (ni,l )mn×c , i = 1, . . . , mn, l = 1, . . . , c, represents a mn × c noise matrix. di. is the 1 × msns down-sampling row vector for pixel i.

For land cover class l, the observation model is y.l = Dx.l + n.l ,

l = 1, . . . , c.

(3)

For pixel i at coarse resolution, the observation model is yi,l = di. · x.l + ni,l , l = 1, . . . , c, i = 1, . . . , m ∗ n

(4)

where di. · x.l is the fraction value of land cover class l for pixel i in the corresponding patch at the fine scale. We use the least squares technique to reconstruct the fineresolution maps, with the minimization cost function E(X, D) = Y − DX22 c = y.l − Dx.l 22 l=1

=

c mn

(yi,l − di. · x.l )2 .

(5)

l=1 i=1

To overcome the ill-posed nature of the SRM problem, the ill-posed inversion problem is converted into an optimization problem coupled with a spatial smooth regularization, as proposed by Ling et al. [7]. That is X ∗ = arg min E(X, D) + λR(X) X

(6)

where R(X) is a regularization term that explicitly considers prior information regarding the super-resolution land cover mapping X, and λ is a weight parameter that balances the contribution of regularization and data fidelity terms. In [7], the regularization model only uses the fine scale spatial dependence. Although, the fine scale regularization term can produce smoother results, it erodes the connectivity of land cover patterns so that they become too smooth. Thus, the fine-resolution mapping should be enhanced by integrating the regularization at both coarse and fine scales, as explained in the following section. III. MSSR FOR SRM Maximizing spatial dependence is equivalent to minimizing spatial inconsistency. Therefore, we used the spatial inconsistency as a fine scale regularization term to produce a consistent objective function. For local details, we assume that there exists little spatial dependence and much heterogeneity between subpixels and neighboring coarse pixels. Considering this, we use the heterogeneity relationship to penalize excessive fine scale smoothing. Thus, we extended SRM-REG [7] to the multiscale regularization model, integrating the spatial dependence at both fine and coarse scales into the SRM. The proposed SRM-MSSR method is defined using the minimization problem X ∗ = arg min E(X, D) + λ1 R1 (X) + λ2 R2 (X) s.t.

c

X

xj,l = 1

(7)

l=1

In this model, we consider three aspects of a configuration X: the fraction error E(X, D), the smooth term R1 (X), and

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination. HU et al.: SUPER-RESOLUTION LAND COVER MAPPING BASED ON MSSR

3

Fig. 1. Neighborhood at coarse and fine scales. (a) Target pixel Pi and its eight neighboring coarse scale pixels. (b) Subpixel pj in Pi and its eight neighboring fine scale subpixels.

penalty term R2 (X). The minimized fraction error E(X, D) guarantees that the resulting super-resolution maps are as consistent as possible with the unmixing results. The smooth term R1 (X) [defined in (8)] uses the inconsistency between a subpixel and its neighboring subpixels. This represents the fine scale spatial homogeneity. If we minimize it, we maximize the local smoothness at the fine scale. R1 (X) favors the class of the majority of the subpixel’s neighbors, so as to form aggregated and homogenized patches. This smoothes the minority classes of the mixed pixels. We introduced a penalty term R2 (X) to deal with this, which is defined in (9). It depends on the class consistency between a subpixel and its neighbors, and represents the coarse scale spatial heterogeneity. By minimizing R2 (X), subpixels tend to be assigned to a different class from the majority of its coarse pixel neighbors. By protecting the minority class in mixed pixels (relative to the neighboring categories), R2 (X) uses the heterogeneity between a subpixel and its coarse neighbors to avoid over-smoothing at the fine scale. This consequently reduces over-smoothing and prevents the homogenization of small local patterns by the majority class of the neighboring subpixels. λ1 and λ2 are both positive real numbers that balance the homogeneity and heterogeneity. If λ1 is too large, the final mapping will be too smooth and lose some details (such as the connectivity of minority classes), and may magnify interrupted thin land cover patterns. Conversely, if λ2 is too large, local details are enhanced and magnified, because of strengthening the edge pixels of minority classes inevitably reduces the accuracy. The neighborhood of subpixels is illustrated in Fig. 1, where the scale factor is set to 3. The white color in Fig. 1(b) represents the target subpixel (pj ) in the parent pixel [Pi , shown in white, in Fig. 1(a)]. The neighboring subpixels are shown in gray, in Fig. 1(b), and the neighboring pixels are shown in gray, in Fig. 1(a). The smooth term [R1 (X)] is defined as R1 (X) =

msns

j=1 k1 ∈Nj1

(1)

φj,k1 δ(j, k1 )

(8)

and the penalty term [R2 (X)] is defined as R2 (X) =

mn

c

i=1 j∈Pi k ∈N (2) l=1 2 i (2)

fine scale subpixel set for subpixel j [e.g., the gray subpixels in Fig. 1(b)]. Subpixels in pixel i have the same coarse neighbor(2) ing pixels in Ni . δ(j, k1 ) in (10) characterizes the dependent relationship according to the local inconsistency between subpixel j and its neighboring subpixel k1 . It is more likely that neighboring land cover classes have the same properties than those that are far from each other. Furthermore, the spatial dependence principle should contribute to the minimum value that is suitable for the framework in (7) ⎧ ⎪ ⎨ 0 if subpixel j and subpixel k1 is assigned to same land cover class; (10) δ(j, k1 ) = ⎪ ⎩ 1 otherwise. (1)

φj,k1 is a weight for the dependence between subpixel j and neighboring subpixel k1 , where (1)

φj,k1 = exp(−d1 (j, k1 )2 /h1 ). (2)

φj,k2 is the weight for the dependence between subpixel j and neighboring pixel k2 , where (2)

φj,k2 = exp(−d2 (j, k2 )2 /h2 ).

(2)

(12)

d1 (j, k1 ) and d2 (j, k2 ) are the Euclidean distances from the center position of subpixel j to the neighboring subpixel k1 and neighboring pixel k2 , respectively. Here, h1 and h2 are the nonlinear parameters of the exponential model [4]. Let Pi represent a specific fine scale patch with s × s subpixels, which corresponds to pixel i in the coarse scale. The smooth term R1 (X) can be reformulated as R1 (X) =

mn

(1)

i=1 j∈Pi k ∈N (1) 1 j

φj,k1 δ(j, k1 )

(13)

and the objective function of subpixel j in Pi for SRM is Lj (X, di. ) =

c

(yi,l − di. · x.l )2

l=1

⎛

⎜ + ⎝λ1

(1)

φj,k1 δ(j, k1 )

(1)

k1 ∈Nj

⎞

c

+ λ2

(2)

k2 ∈Ni

l=1

⎟ (2) xj,l φj,k2 yk2 ,l ⎠

(14)

Thus, the global objective function (7) can be reformulated as L(X, D) =

xj,l φj,k2 yk2 ,l

(11)

mn

Lj (X, di. ).

(15)

i=1 j∈Pi

(9)

where Ni is the neighboring coarse scale pixel set for pixel i (1) [e.g., the gray pixels in Fig. 1(a)], and Nj is the neighboring

We used the simulated annealing algorithm [24] to obtain the global optimum solution for (15). The temperature parameter (T ) for this algorithm is updated using [16], [25] Tite = T0 /log(ite + 1)

(16)



TABLE I N UMBER OF PATCHES U SING F IVE D IFFERENT M ETHODS : SRM-SA, SRM-HIIPD, SRM-MRF, SRM-REG, AND SRM-MSSR

TABLE II ACCURACY OF SRM R ESULTS FOR S IMULATED L INE , U SING F IVE D IFFERENT M ETHODS : SRM-SA, SRM-HIIPD, SRM-MRF, SRM-REG, AND SRM-MSSR W ITH PARAMETERS (λ1 , λ2 )

where ite is the number of iterations. All the subpixel labels are updated at each iteration. As in all other randomized methods [14], [26], the random initial fine-resolution map has a strong influence on the final mapping. In this study, we used a map generated by SRM-SA as an initial map for all the random SRM algorithms (SRMMRF, SRM-HIIPD, SRM-REG, and SRM-MSSR). We change the land cover class label of a subpixel if it reduces the objective function (14). If this change increases the value of the objective function, then it is accepted with some probability that it depends on the current temperature. The algorithm terminates when we have reached a predetermined number of iterations. IV. E XPERIMENTS AND R ESULTS Various processes are necessary before SRM can be implemented. They include registration, endmember selection, and spectral unmixing. These preprocessing steps may produce and propagate errors, which would make it hard to explain our results [27]. As such, we used simulated and synthetic images used in our experiments to avoid the effects of errors from

various preprocessing procedures, and to focus on evaluating the performance of the proposed method. In our experiments, we applied the proposed method on fraction images at a certain scale and compared the results with a hard classification at a fine resolution. We calculated the area proportions of all classes in each low-resolution pixel within a window. The size of the window depends on the predefined scale factor, so that we avoid errors introduced by soft classification. The corresponding fraction images were then considered as the results generated by soft classification and used as inputs for SRM. For these simulated and synthetic images, the spatial dependence was based on eight neighbors. We tested scale factors of 2, 3, and 4. We compared the evaluation indices of the SRM maps with the high-resolution reference maps. We evaluated the proposed method using adjusted kappa index [27] and one landscape index (which depends on the number of patches). We only calculated the adjusted kappa index for the mixed pixels, to provide more information on the SRM algorithm’s prediction abilities [9]. The accuracies of the SRM algorithms are shown in Tables I–IV. The adjusted kappa values in Tables II–IV are the averages of 100 numerical experiments. The maximum


5

TABLE III ACCURACY OF SRM R ESULTS FOR THE IKONOS FARMLAND I MAGE , U SING F IVE D IFFERENT M ETHODS : SRM-SA, SRM-HIIPD, SRM-MRF, SRM-REG, AND SRM-MSSR W ITH PARAMETERS (λ1 , λ2 )

TABLE IV ACCURACY OF SRM R ESULTS FOR THE AVIRIS A IRPORT I MAGE , U SING F IVE D IFFERENT M ETHODS : SRM-SA, SRM-HIIPD, SRM-MRF, SRM-REG, AND SRM-MSSR W ITH PARAMETERS (λ1 , λ2 )

Fig. 2. Simulated line. (a) Reference map. (b) Hard classification of degraded fraction image at scale 2. (c)–(g) Super-resolution maps from SRM-SA, SRM-MRF, SRM-HIIPD, SRM-REG (SRM-MSSR with λ1 = 0.4 and λ2 = 0), and SRM-MSSR with λ1 = 0.4 and λ2 = 0.25.

values of the rows are highlighted in bold. We also implemented a pairwise Z test [14] on the adjusted kappa values, to determine that the differences between the accuracies of two different algorithms were statistically significant at the 95% confidence level. The Z-values are greater than 1.96, so the classification accuracy of the SRM from our algorithm was significantly (95% confidence) higher than the other algorithms.

The results show that SRM-MSSR was almost the most accurate of all the SRM algorithms for the simulated and synthetic images, regardless of scale (and especially at scale 2). Details of our experiments are given in the following text. The number of patches (PN) [28] as calculated by Fragstats [29] is commonly used to quantify fragmentation. Table I contains the number of patches of maps from different SRM



Fig. 3. Adjusted kappa index for the simulated line image at different scales, using SRM-MSSR and different values of λ1 and λ2 . (a)–(c) Adjusted kappa values at scale 2, 3, and 4, respectively.

methods at different scales. The elements of row “REF” are the number of patches for the reference maps. The parameters of SRM-MSSR are given in parentheses in row “SRM-MSSR.”

Fig. 4. IKONOS farmland image. (a) IKONOS panchromatic farmland image. (b) Reference high-resolution land cover map. (c)–(e) Hard classification of degraded fraction images at scale of 2, 3, and 4 using majority rules.

A. Simulated Image In this experiment, we used an example of a simulated line to demonstrate the SRM results generated by various algorithms at scale factor 2, together with the reference image shown in Fig. 2. The image had a 20 × 20 pixels reference map, as shown in Fig. 2(a). The hard classification of the fraction image in Fig. 2(b) had 10 × 10 coarse pixels, which were generated by degrading the simulated line at scale factor 2. The results of different SRM algorithms are shown in Fig. 2(c)–(g). They were generated by restoring the fine resolution map from the input fraction image of Fig. 2(b). The resulting fineresolution land-cover map produced by SRM-SA had many irregular boundaries, as shown in Fig. 2(c). This is because the method does not consider information from the neighboring coarse pixels. SRM-HIIPD produced more fragments than SRM-SA and SRM-MRF [Fig. 2(d)], because of the limitations of the pixel-swap algorithms [30]. SRM-MRF produced a smoother land-cover map, but the subpixels in minority area were homogenized by their neighboring pixels from majority area [Fig. 2(e)]. An over-smooth phenomenon similar to that in SRM-MRF was also present in the SRM-REG results [Fig. 2(f)]. In comparison, the result of the proposed SRMMSSR method with λ1 = 0.4 and λ2 = 0.25 had the obviously higher adjusted kappa (Table II) and a smoother boundary, while maintaining the shape connectivity as much as possible [Fig. 2(g)]. The SRM-MSSR results were also more similar to the reference map from the other algorithms. To determine the appropriate regularization parameters for SRM-MSSR, we tested the proposed method and calculated the adjusted kappa values for 100 × 100 combinations of λ1 and λ2 in the range of [0, 0.99] × [0, 0.99] at 0.01 intervals. We used the simulated line image at scales of 2, 3, and 4, as shown in Fig. 3. In this figure, the adjusted kappa values increase as the color changes from blue to red. The highest adjusted kappa values are clustered in the region by the black rectangle in Fig. 3. We selected the parameters at the center of this region as the optimum parameters for testing synthetic images. Thus, we used parameter pairs for (λ1 , λ2 ) of (0.4, 0.25), (0.18, 0.07), and (0.06, 0.03) for scales 2, 3, and 4, respectively. An increase

in the scale leads to more uncertainty, so λ1 and λ2 contribute less to the SRM at larger scales. The adjusted kappa values were better when λ1 was higher than λ2 , which demonstrates the major aggregated characteristic of land cover class. λ2 contributed more on smaller scales and resulted in better adjusted kappa values, because the details at large scales become much more uncertain. B. IKONOS Farmland Image We used a test area from the IKONOS image, which is a farmland area near Dujiangyan, Sichuan Province, China. The experiment was conducted using an IKONOS panchromatic image (courtesy of [7]), as shown in Fig. 4(a). A reference fine-resolution thematic map of 160 × 160 pixels [1-m spatial resolution; Fig. 4(b)] was generated by manually digitizing the corresponding panchromatic band of the image. We degraded the reference land cover map [Fig. 4(b)] to generate the input fraction images at scale 2, 3, and 4. The hard classifications are shown in Fig. 4(c)–(e). The land cover map included four clearly defined farm land cover classes. The SRM results from the five SRM methods were used to evaluate the performance of the SRM-MSSR method [Fig. 5(a)–(e) at scale 2, Fig. 5(f)–(k) at scale 3, and Fig. 5(l)–(q) at scale 4]. Compared with the reference high-resolution land cover map shown in Fig. 4(b), the results of SRM-SA [Fig. 5(a), (f), and (l)] and SRMHIIPD [Fig. 5(b), (g), and (m)] had many irregular boundaries, especially for the linear Farmland 3 class. The results of SRM-MRF [Fig. 5(c), (h), and (n)] and SRM-REG [Fig. 5(d), (i), and (o)] had over-smooth boundaries and it was obvious that many boundary pixels for the various linear patterns of Farmland 3 was homogenized. In contrast, the proposed SRM-MSSR method with appropriate parameters produced land cover maps [Fig. 5(e), (k), and (q)] that were visually much closer than those produced by the other four SRM algorithms. The boundaries between different land cover classes were smooth and noticeable. When the coarse scale spatial dependence was ignored (i.e., λ2 = 0), the final map showed


7

Fig. 5. Super-resolution maps of the IKONOS farmland image.

fragmented patterns [Fig. 5(d), (i), and (o)]. However, when we added some penalties related to the coarse scale dependence, the patterns were gradually connected for the same land cover patterns [Fig. 5(e), (k), and (q)]. This phenomenon was particularly noticeable for the Farmland 3 class, and is verified in Table I. This shows the contribution of parameter λ2 when determining the landscape fragmentation in the reference map. The adjusted kappa values at different scales (2, 3, and 4) are given in Table III, which shows that the SRM-MSSR method was the most accurate. The adjusted kappa value was 2.79% larger than that from the SRM-REG method at scale 2, 1.2% larger at scale 3, and 0.58% larger at scale 4. Although other methods were more accurate at scale 3 by Z-value of 0.56 < 1.96, the SRM-MSSR result had visually smoother edges [Fig. 5(j)]. Additionally, there was a more connected map when λ2 increased to 0.18 in Fig. 5(k), when compared with 5(g).

Fig. 6. AVIRIS airport image. (a) Google Earth image. (b) Reference land cover map. (c)–(e) Hard classification of degraded fraction images at scale 2, 3, and 4 using majority rules.

C. AVIRIS Airport Image We further tested the performance of the proposed model on an airport area with land cover patches of water, grass, asphalt, and concrete pavement. The airport image is from Moffett Field in the San Francisco Bay area. The reference fine-resolution land cover map of 300 × 280 pixels [Fig. 6(b)] was produced by manual digitization, using an image from Google Earth with a high spatial resolution of 1 m, taken in July 2007 [Fig. 6(a)]. There are four land cover classes in the images: water, grass, dark, and white surfaces. Hard classifications of degraded fraction images at scales of 2, 3, and 4 are shown in Fig. 6(c)–(e). The results of the five SRM algorithm are presented in Fig. 7.

The map generated by the SRM-MSSR method at scale 2 was the smoothest and had connectivity details closest to the reference map, when compared with the other four SRM algorithms. The adjusted kappa values were 1.05% larger than those for SRM-REG at scale 2. Because of the increasing uncertainty, the SRM-MSSR accuracy at scales 3 and 4 did not significantly increase. However, the SRM-MSSR method with appropriate (λ1 , λ2 ) parameter pairs of (0.18, 0.18) at scale 3 and (0.21, 0.24) at scale 4 produced smoother and more connected patterns, as shown in Fig. 7(k) and (q). This is further verified in Table I.



Fig. 7. Super-resolution maps of the AVIRIS airport image.

V. C ONCLUSION We cannot adequately describe spatial relationships by only using the fine scale spatial dependence as a smooth constraint. This makes SRM-REG inaccurate. To overcome this shortcoming, our research introduced a more applicable SRM approach, which minimizes the fraction residual errors using MSSR (SRM-MSSR). The proposed method integrates the fine scale spatial homogeneity as a smooth term and the coarse scale spatial heterogeneity as a penalty term. The spatial heterogeneity calculated at the coarse scale can effectively preserve local details and restrain the over-smoothing produced by the generative model of the fine scale spatial dependence. We used a simulated annealing optimization scheme to solve the SRM problem with MSSR, and evaluated the effectiveness of the algorithm on simulated and synthetic images. Our visual and quantitative comparison with four other representative algorithms (SRM-SA, SRM-MRF, SRM-HIIPD, and SRM-REG) demonstrated the enhanced effectiveness of the proposed SRMMSSR method, especially for small scales. When using the appropriate parameters, the SRM-MSSR technique produced smooth and morphologically similar maps, when compared with the reference maps. In this study, it was important that we determined the appropriate values for the two regularization parameters. We evaluated the adjusted kappa index for different parameter combinations and the simulated line image, so that we could find the optimal parameters for SRM-MSSR at different scales. We applied these experimentally-determined optimal parameter pairs to the synthetic images, which increased the accuracy and resulted in visually better maps. In future work, we will develop

an adaptive method for selecting these regularization parameters. This research should also be considered in the context of spectral data for SRM.

ACKNOWLEDGEMENT The authors would like to thank F. Ling of the Institute of Geodesy and Geophysics, Chinese Academy of Sciences, Wuhan, China, for providing the IKONOS and AVIRIS datasets and insightful discussion. The authors express their sincere gratitude to the five anonymous reviews for their valuable comments and suggestions.

R EFERENCES [1] P. M. Atkinson, M. E. J. Cutler, and H. Lewis, “Mapping sub-pixel proportional land cover with AVHRR imagery,” Int. J. Remote Sens., vol. 18, no. 4, pp. 917–935, 1997. [2] M. Q. Nguyen, P. M. Atkinson, and H. G. Lewis, “Superresolution mapping using a hopfield neural network with LIDAR data,” IEEE Geosci. Remote Sens. Lett., vol. 2, no. 3, pp. 366–370, Jul. 2005. [3] A. J. Tatem, H. G. Lewis, P. M. Atkinson, and M. S. Nixon, “Superresolution target identification from remotely sensed images using a hopfield neural network,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 4, pp. 781–796, Apr. 2001. [4] P. M. Atkinson, “Sub-pixel target mapping from soft-classified, remotely sensed imagery,” Photogramm. Eng. Remote Sens., vol. 71, no. 7, pp. 839–846, 2005. [5] F. Ling, Y. Du, F. Xiao, and X. D. Li, “Subpixel land cover mapping by integrating spectral and spatial information of remotely sensed imagery,” IEEE Geosci. Remote Sens. Lett., vol. 9, no. 3, pp. 408–412, May 2012. [6] F. Ling, X. D. Li, Y. Du, and F. Xiao, “Sub-pixel mapping of remotely sensed imagery with hybrid intra- and inter-pixel dependence,” Int. J. Remote Sens., vol. 34, no. 1, pp. 341–357, 2013.


[7] F. Ling, X. Li, F. Xiao, and Y. Du, “Superresolution land cover mapping using spatial regularization,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 7, pp. 4424–4439, Jul. 2014. [8] F. Ling et al., “Superresolution land cover mapping with multiscale information by fusing local smoothness prior and downscaled coarse fractions,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 9, p. 16, Sep. 2014. [9] K. C. Mertens, B. de Baets, L. P. C. Verbeke, and R. R. de Wulf, “A subpixel mapping algorithm based on subpixel/pixel spatial attraction models,” Int. J. Remote Sens., vol. 27, no. 15, pp. 3293–3310, 2006. [10] Y. Ge, S. P. Li, and V. C. Lakhan, “Development and testing of a subpixel mapping algorithm,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 7, pp. 2155–2164, Jul. 2009. [11] F. Ling, Y. Du, F. Xiao, H. P. Xue, and S. J. Wu, “Super-resolution landcover mapping using multiple sub-pixel shifted remotely sensed images,” Int. J. Remote Sens., vol. 31, no. 19, pp. 5023–5040, 2010. [12] P. M. Atkinson, “Downscaling in remote sensing,” Int. J. Appl. Earth Observ. Geoinf., vol. 22, pp. 106–114, 2013. [13] Y. Ge, “Sub-pixel land-cover mapping with improved fraction images upon multiple-point simulation,” Int. J. Appl. Earth Observ. Geoinf., vol. 22, pp. 115–126, 2013. [14] T. Kasetkasem, M. K. Arora, and P. K. Varshney, “Super-resolution land cover mapping using a Markov random field based approach,” Remote Sens. Environ., vol. 96, no. 3–4, pp. 302–314, 2005. [15] V. A. Tolpekin and A. Stein, “Quantification of the effects of land-coverclass spectral separability on the accuracy of Markov-random-field-based superresolution mapping,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 9, pp. 3283–3297, Sep. 2009. [16] J. P. Ardila, V. A. Tolpekin, W. Bijker, and A. Stein, “Markov-randomfield-based super-resolution mapping for identification of urban trees in VHR images,” ISPRS J. Photogramm. Remote Sens., vol. 66, no. 6, pp. 762–775, 2011. [17] X. D. Li, Y. Du, and F. Ling, “Spatially adaptive smoothing parameter selection for Markov random field based sub-pixel mapping of remotely sensed images,” Int. J. Remote Sens., vol. 33, no. 24, pp. 7886–7901, 2012. [18] L. G. Wang and Q. M. Wang, “Subpixel mapping using Markov random field with multiple spectral constraints from subpixel shifted remote sensing images,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 3, pp. 598–602, May 2013. [19] X. Xu, Y. F. Zhong, L. P. Zhang, and H. Y. Zhang, “Sub-pixel mapping based on a map model with multiple shifted hyperspectral imagery,” IEEE J. Sel. Topics Appl. Earth Observ., vol. 6, no. 2, pp. 580–593, Apr. 2013. [20] X. D. Li, Y. Du, and F. Ling, “Super-resolution mapping of forests with bitemporal different spatial resolution images based on the spatialtemporal Markov random field,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 1, pp. 29–39, Jan. 2014. [21] F. Ling, X. D. Li, Y. Du, and F. Xiao, “Super-resolution land cover mapping with spatial-temporal dependence by integrating a former fine resolution map,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 5, pp. 1816–1825, May 2014. [22] Y. H. Zhang, Y. Du, F. Ling, S. M. Fang, and X. D. Li, “Example-based super-resolution land cover mapping using support vector regression,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 4, pp. 1271–1283, Apr. 2014. [23] Z. Mahmood, M. A. Akhter, G. Thoonen, and P. Scheunders, “Contextual subpixel mapping of hyperspectral images making use of a high resolution color image,” IEEE J. Sel. Topics Appl. Earth Observ Remote Sens., vol. 6, no. 2, pp. 779–791, Apr. 2013. [24] S. Lakshmanan and H. Derin, “Simultaneous parameter estimationand segmentation of Gibbs random fields using simulated annealing,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 8, pp. 799–813, Aug. 1989. [25] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 6, no. 6, pp. 721–41, Nov. 1984. [26] Z. Shen, J. Qi, and K. Wang, “Modification of pixel-swapping algorithm with initialization from a sub-pixel/pixel spatial attraction model,” Photogramm. Eng. Remote Sens., vol. 75, no. 5, pp. 557–567, 2009. [27] K. C. Mertens, L. P. C. Verbeke, E. I. Ducheyne, and R. R. De Wulf, “Using genetic algorithms in sub-pixel mapping,” Int. J. Remote Sens., vol. 24, no. 21, pp. 4241–4247, 2003. [28] L. Fahrig, “Effects of habitat fragmentation on biodiversity,” Annu. Rev. Ecol. Evol. Syst., vol. 34, no. 1, pp. 487–515, 2003.

9

[29] K. S. C. McGarigal and E. Ene, Fragstats v4: Spatial Pattern Analysis Program for Categorical and Continuous Maps. Amherst, MA, USA: University of Massachusetts, 2012 [Online]. Available: http://www.umass.edu/landeco/research/fragstats/fragstats.html [30] Y. Makido and A. Shortridge, “Weighting function alternatives for a subpixel allocation model,” Photogramm. Eng. Remote Sens., vol. 73, no. 11, pp. 1233–1240, 2007.

Jianlong Hu received the B.S. degree in information and computing science and the M.S. degree in computer application technology from Shanxi University, Taiyuan, China, in 2003 and 2006, respectively. During 2014, he was a Visiting Scholar with the State Key Laboratory of Resources and Environmental Information System, Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing, China. His research interests include remote sensing image analysis and machine learning.

Yong Ge (M’14) received the Ph.D. degree in cartography and geographical information system from Chinese Academy of Sciences (CAS), Beijing, China, in 2001. She is a Professor with the State Key Laboratory of Resources and Environmental Information System, Institute of Geographical Sciences and Natural Resources Research, CAS. She has directed research in more than ten national projects. She is the author or co-author of over 80 scientific papers published in refereed journals, one book, and six chapters in books; she is the editor of one book, and she also holds three granted patents in the issue of improving the accuracy of information extraction from remotely sensed imagery. Her research interests include spatial data analysis and data quality assessment. Dr. Ge has been involved in the organization of several international conferences and workshops. She is a Member of the Theory and Methodology Committee of the Cartography and Geographic Information Society, the International Association of Mathematical Geosciences, and the Editorial Board of Spatial Statistics (Elsevier).

Yuehong Chen received the B.S. degree in geographical information system from Hohai University, Nanjing, China, and the M.S. degree in cartography and geographical information system from University of Chinese Academy of Sciences, Beijing, China, in 2010 and 2013, respectively. He is currently pursuing the Ph.D. degree at the State Key Laboratory of Resources and Environmental Information System, Institute of Geographical Sciences and Natural Resources Research, University of Chinese Academy of Sciences. His research interests include remote sensing image processing and super resolution mapping.

Deyu Li received the M.S. degree in basic mathematics from Shanxi University, Taiyuan, China, and the Ph.D. degree in computing mathematics from Xi’an Jiaotong University, Xi’an, China, in 1998 and 2002, respectively. He is a Professor with the Key Laboratory of Computational Intelligence and Chinese Information Processing of the Ministry of Education, School of Computer and Information Technology, Shanxi University. He has published more than 20 articles in international journals. His research interests include rough set theory, granular computing, data mining, and knowledge discovery.