Spatial-Spectral Total Variation Regularized Low ...

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 13, NO. 9, JUNE 2018

1

Spatial-Spectral Total Variation Regularized Low-Rank Tensor Decomposition for Hyperspectral Image Denoising Haiyan Fan, Chang Li, Yulan Guo, Gangyao Kuang, Senior Member, IEEE, Jiayi Ma

For real-word HSIs, there usually exists a combination of different types of noise, e.g. Gaussian noise, random valued impulse noise, salt-and-pepper noise, horizontal and vertical deadlines. Gaussian noise arises when HSIs are affected by atmospheric absorption and instrumental noise. Salt-and-pepper noise appears when the sensors have unpredictable calibration error [9]. For hyperspectral scanners using whiskbroom or pushbroom technology, the resulting HSIs may be corrupted by stripes [10] or deadlines [11], if the detectors are physically damaged. Therefore, HSIs denoising needs to remove a mixture of Gaussian noise and sparse noise. Sparse noise refers to the noise which corrupts only a few pixels in the image with strong level. In this paper, the sparse noise includes random valued impulse noise, salt-and-pepper noise, horizontal and vertical deadlines. HSIs denoising is a well studied problem [12–15]. [16–18] remove the mixture of only Gaussian and salt-and-pepper noise in gray-scale images. However, in a realistic scenario, more general noise should be considered. Recently, many low-rank matrix factorization based methods are proposed to exploit global spectral correlation of HSIs [12, 19]. These methods assume that the clean hyperspectral data should lie in a lowIndex Terms—Hyperspectral image denoising, low-rank tensor rank subspace and can be represented by a linear combination factorization (LRTF), spatial-spectral total variation (SSTV) of finite endmembers. The low-rank matrix recovery (LRMR) method [20] achieves superior performance for mixed noise removal. However, the LRMR method only considers the I. I NTRODUCTION spectral correlation and neglects the spatial structure, which HILE hyperspectral imaging sensors have experienced can lead to spatial distortions. To address this limitation, a significant success [1, 2], HSIs collected in practice low-rank spectral nonlocal approach (LRSNL) was proposed often suffer from various annoying degradations due to several [21]. In this method, low-rank (LR) property is firstly exploited reasons, e.g. fluctuations in power supply, dark current, non- to obtain precleaned patches. Then, a spectral nonlocal (SNL) uniformity of detector response. The degradation of HSIs method is proposed to consider both spectral and spatial caused by various types of noise hinders the effectiveness information. Similarly, a total variation regularized low-rank of subsequent HSI processing tasks, e.g. spectral signature matrix factorization (LRTV) method was proposed in [22]. In unmixing [3–5], segmentation [6, 7] and classification [8]. this method, the low-rank model is used to capture the spectral Therefore, the task of removing the noise in the hyperspectral correlation, and the total variation (TV) regularization is used to imagery is an important research topic. capture the spatial piecewise smooth structure of HSIs. In [23], TV based component analysis was applied for hyperspectral H. Fan is with the Space Engineering University, China (e-mail: feature extraction. The bandwise TV regularization used in [22] hy [email protected]) C. Li is with the Department of Biomedical Engineering, Hefei University is a local model which exploits spatial correlation within a band. of Technology, Hefei 230009, China (e-mail: [email protected]) This TV method may perform poorly on removing the noise Y. Guo and G. Kuang are with the National University of Defense with obvious structure. For example, when HSIs are corrupted Technology, China (e-mail: [email protected], [email protected]). Y. Guo is also with the Institute of Computing Technology, Chinese Academy by strips or deadlines in the same place in most of the noisy of Sciences, China. HSI bands, the bandwise TV method may treat the strips and J. Ma is with the Electronic Information School, Wuhan University, China deadlines as special structures and preserve them, especially (e-mail: [email protected]). Manuscript received June 20, 2017. when the width of deadlines or strips is large (e.g. with a width Abstract—Several bandwise total variation (TV) regularized low-rank based models have been proposed to remove mixed noise in hyperspectral images (HSIs). These methods convert high-dimensional HSI data into 2-D data based on low-rank matrix factorization. This strategy introduces loss of useful multiway structure information. Moreover, these bandwise TV based methods exploit the spatial information in a separate manner. To cope with these problems, we propose a spatial-spectral total variation (SSTV) regularized low-rank tensor factorization (SSTV-LRTF) method to remove mixed noise in HSIs. From one aspect, the hyperspectral data are assumed to lie in a lowrank tensor, which can exploit the inherent tensorial structure of hyperspectral data. The low-rank tensor factorization (LRTF)based method can effectively separate the low-rank clean image from sparse noise. From another aspect, HSIs are assumed to be piecewisely smooth in the spatial domain. The total variation (TV) regularization is effective in preserving the spatial piecewise smoothness and removing Gaussian noise. These facts inspire the integration of the low-rank tensor factorization with TV regularization. To address the limitations of bandwise TV, we use SSTV regularization to simultaneously consider local spatial structure and spectral correlation of neighboring bands. Both simulated and real data experiments demonstrate that the proposed SSTV-LRTF method achieves superior performance for HSI mixed-noise removal, as compared to the state-of-the-art TV regularized and low-rank based methods.

W


2

of 3 pixels or more). To address the limitations of bandwise local spatial structure and spectral correlation of neighboring TV, a spatial-spectral TV model (SSTV) was proposed [24]. bands. Specifically, the inherent structure of HSIs has been exploited Basically, our work is related to the aforementioned works. in both spatial and spectral domains. A similar work was However, there are significant differences between our work proposed in [25]. The work [26] integrated the structure tensor and the others. The TV regularized low-rank based methods TV into the low-rank matrix factorization model, and showed mentioned above, for instance, the LRTV method [22], are that it outperforms band-by-band low-rank matrix factorization based on the popular matrix modeling idea. The matricization model. Some works also considered the removal of Poisson technique should preliminary vectorize all HSI bands at the cost noise, e.g. [27] and [25]. of losing spatial structures of the HSI cube. Whereas, in the proThese low-rank matrix factorization methods mainly convert posed SSTV-LRTF method, direct tensor modeling technique high-dimensional HSI data into 2-D data by vectorizing the could more faithfully preserve the underlying information of data in each band. This strategy will introduce loss of useful the HSI cube without destroying the spatial structures. Besides, multi-way structure information. To address this limitation, a compared to the bandwise TV regularization used in other TV collection of tensor tools can be used to exploit the inherent regularized low-rank based methods, we adopt SSTV to further tensorial structure of hyperspectral data. Two types of tensor exploit the local piecewise smoothness in both spatial and decomposition algorithms are usually used in the literature, spectral domains. A detailed description of our contributions namely, Tucker decomposition and parallel factor analysis can be found in Section I-A. (PARAFAC) decomposition. The Tucker decomposition based denoising methods include the Lower Rank Tensor Approxima- A. Paper contribution tion (LRTA) [28], the genetic kernel Tucker decomposition [29] The contributions of this work are three-fold: and the multidimensional Wiener filtering method [30]. The • The low-rank tensor factorization model is proposed for PARAFAC decomposition based denoising methods include HSIs denoising. From the Bayesian perspective, we also the PARAFAC model [31] and the rank-1 tensor decomposition build up the relationship between the low-rank tensor method [32]. factorization and the rank-constrained Tensor Robust Despite being effective tools for multidimensional data Principle Component Analysis (TRPCA), which can be processing, finding the decompositions of PARAFAC and solved efficiently for HSI denoising. Tucker models requires the solution of a difficult non-convex • SSTV regularization is incorporated into the low-rank optimization problem, which has poor convergence properties. tensor factorization model. The low-rank tensor factorizaBesides, for PARAFAC and Tucker methods, the number of tion model is used to separate the clean spectral signal components needs to be known in priori [33, 34]. Moreover, the from the sparse noise, and the SSTV regularization is aforementioned tensor algebra methods are implicitly developed utilized to remove Gaussian noise and to enhance spatial for additive white Gaussian noise. In [35], an alternative information. approach is proposed, using tensor nuclear norm regulrizers, • Experimental results demonstrate that the proposed which is a convex function of the multi-spectral unknown. method clearly improves the denoising results in terms This new tensor nuclear norm based on tensor singular value of both quantitative evaluation and visual inspection, as decomposition (t-SVD) [36] has been demonstrated to be useful compared to the state-of-the-art TV regularized and lowfor applications such as face recognition and image deblurring rank based methods. [37]. t-SVD is based on a new tensor multiplication scheme and has similar structure to matrix SVD. It allows optimal low-rank representation (in terms of the Frobenius norm) of a B. Paper Organization tensor by the sum of outer product of matrices [36]. The rest of this paper is organized as follows. Section II In this paper, we propose an HSI denoising technique using describes the preliminaries on tensors and gives the notations low-rank tensor factorization. The clean HSI data have its used throughout the paper. In Section III, the proposed underlying low-rank tensor property, even though the real SSTV regularized Low-rank Tensor Factorization (SSTV-LRTF) HSI data may not due to outliers and non-Gaussian noise model for HSI mixed-noise removal is described in details. [28]. Our model is based on t-SVD and its induced tensor In Section IV, experiments are conducted to demonstrate the tubal rank and tensor nuclear norm. To remove the mixed effectiveness of the proposed method. The final section gives noise of hyperspectral data, we combine low-rank tensor concluding remarks. factorization and TV regularization. From one aspect, the hyperspectral data are assumed to lie in a low-rank tensor. II. T ENSOR ALGEBRAIC FRAMEWORK The low-rank tensor factorization-based method can effectively separate the low-rank clean image from the sparse noise. In this section, we give the definitions and notations used From another aspect, HSIs are assumed to be piecewisely throughout the paper. For a more comprehensive discussion, smooth in the spatial domain. TV regularization can effectively we refer the readers to the reviews [33] and [36]. In this paper, preserve edge information and the spatial piecewise smoothness tensors are denoted as capitalized calligraphic letters, e.g. A. [38, 39]. These facts inspire us to integrate the low-rank tensor An m-way tensor is a multi-linear structure in Rn1 ×n2 ×···×nm , factorization with TV regularization. To address the limitations where R is the field of real number. Matrices are denoted of bandwise TV, we use a SSTV regularization to consider both by capitalized boldface letters, e.g. A. Vectors are denoted by


3

boldface letters, e.g. a. Scalars are denoted by lower-case letters, Definition 4 (Tensor nuclear norm[40]): The nuclear norm e.g. a. For an m-way tensor A ∈ Rn1 ×n2 ×···×nm , we denote of tensor A ∈ Rn1 ×n2 ×n3 , denoted as kAkTNN , is defined as ˆ its (i1 , i2 , · · · , im )-element as Ai1 i2 ···im or ai1 ,i2 ,··· ,im . Several the average of the nuclear norm of all the frontal slices of A, norms for vector, matrix P and tensor are used. The l1 -norm is cal- i.e., culated as kAk1 = i1 ,i2 ,··· ,im |ai1 ,i2 ,··· ,im | and the Frobenius n3 X qP 1 X ˆ (i) k∗ = 1 ˆ i, j), 2. kA kAk = S(i, (4) TNN norm is calculated as kAkF = |a | i1 ,i2 ,··· ,im i1 ,i2 ,··· ,im n3 j=1 n3 i,j P The matrix nuclear norm is kAk∗ = i σi (A), where σi (A) is the i-th singular value of A. where k · k∗ is the matrix nuclear norm. In the remaining part of this section, we focus on 3-way tensors. For a three-way tensor A, A(i, :, :), A(:, i, :) and III. SSTV- REGULARIZED L OW- RANK T ENSOR A(:, :, i) are used to represent the i-th horizontal, lateral and FACTORIZATION M ODEL frontal slices. The frontal slice A(:, :, i) can also be denoted A. The Proposed LRTF Model for HSIs Denoising by A(i) . This notation will be used in the rest of the paper. 1) HSI Degradation Model: HSI data contaminated by bcirc(A) is the block diagonal matrix of tensor A with a size M ×N ×p , where M × N is of n1 n3 × n2 n3 . Aˆ is the discrete Fourier transform along the mixed noise are denoted by Y ∈ R n1 n3 ×n2 n3 ¯ ˆ the spatial domain and p is the number of spectral bands. They third dimension of A, i.e., A = fft(A, [], 3). A ∈ R is the block diagonal matrix with each block on diagonal being can be modeled as: ˆ (i) of Aˆ [40]. the frontal slice A Y = F + S + N, (5) In the below, we give the definition of t-product of two tensors. where F ∈ RM ×N ×p is the clean hyperspectral data, S ∈ Definition 1 (t-product of two Tensors[40]): Given two 3- RM ×N ×p and N ∈ RM ×N ×p are sparse noise and Gaussian way tensors A ∈ Rn1 ×n2 ×n3 and B ∈ Rn2 ×n4 ×n3 , the tensor noise, respectively. 2) Low-tubal-rank Tensor Factorization of Clean HSI Data: product of A and B produces a 3-way tensor C of size n1 × Hyperspectral sensors measure the radiance from the observed n4 × n3 : scene in several spectral bands close in wavelengths. Thus, the n2 X C(i, l, :) = A ∗ B = A(i, j, :) ∗ B(j, l, :), (1) signal F has strong correlation in local spectral neighborhoods. Similar to its 2-D counterpart in [22], we express the 3-D j=1 hypersectral data F as a t-linear combination of a spanning where i = 1, 2, · · · , n1 and l = 1, 2, · · · , n4 . This is consistent basis based on the global spectral correlation. with the multiplication between matrices with the t-product ‘∗’ Based on the previous analysis and Definition 2 (Section corresponding to the multiplications. II), we can know that the 2-D data samples of each spectral Definition 2 (Free-Module over the commutative ring[40]): band come from a lower dimensional free submodule of the Let Mnn13 be a set of all 2-D lateral slices of size n1 × 1 × n3 , free module MM , where a free submodule is a subset of MM each element in Mnn13 can be viewed as a vector of tubal-scalars. with a spanningN basis of dimension k ≤ p. The 2-D image N of ~ and coefficient w For each element A ~ ∈ R(Gn3 ), the product each band is reshaped as a lateral slice of size M × 1 × N . ~∗w A ~ is also an element in Mnn13 . Mnn13 is closed under the Then, we arrange these lateral slices to obtain a 3-D tensor tubal-scalar multiplication. F ∗ of size M × p × N . We define an operator: As stated in [40], Mnn13 is also a free module of dimension Map: RM ×N ×p → RM ×p×N . (6) n3 over the commutative ring R(Gn3 ). This means that any n ~ ∈ Mn1 can be uniquely represented as a t-linear element A 3 The operator Map treats each frontal slice of the original tensor ~1 , B ~2 , · · · , B ~k }, i.e., combination of a spanning basis {B as a lateral slice of the new tensor. Similarly, we can define ~= A

k X

the inverse operation as: ~i ∗ w B ~ i.

(2)

i=1

inv-Map:

RM ×p×N → RM ×N ×p .

(7)

Definition 3 (t-SVD): Given p 2-D data samples ~1 , A ~2 , · · · , A ~ n , we arrange them as lateral slices to obtain A 2 a 3-D tensor A of size n1 × n2 × n3 . The t-SVD factorization of tensor A is to compute the spanning basis (principle components) of this free submodule. Specifically, t-SVD is defined as: A∗ = Q ∗ S ∗ V ∗ , (3)

Based on definitions of the two operators, we can have F ∗ = Map(F) and F = inv-Map(F ∗ ). For each lateral slice F ∗ (:, i, :), we wish to find a t-linear ~1 , L ~2 , · · · , L ~ k }, i.e., combination of a spanning basis {L

where Q ∈ Rn1 ×k×n3 and V ∈ Rk×n2 ×n3 are orthogonal. V ∗ is the conjugate tensor of V [41]. S ∈ Rk×k×n3 is a tensor whose frontal slices are diagonal matrices. The tubal scalars S(i, i, :), i = 1, 2, · · · , k are singular tubes, where k is the tubal rank.

~ j ∈ MM and MM is a set of all 2-D lateral where the basis L N N slices of size M × 1 × N , ~rjl ∈ R1×1×N is a N -tuple oriented into the board. Compared to endmember mixture expression in [22], the t-linear mixture expression in Eq. (8) can better preserve the spatial structure, as the image in each band is

F ∗ (:, i, :) =

k X

~ j ∗ ~rjl , L

(8)

j=1


vectorized in [22]. In fact, for F ∗ , we can find two tensors L ∈ RM ×k×N and R ∈ Rk×p×N that: F ∗ (j, l, :) = L ∗ R =

k X

L(i, j, :) ∗ R(j, l, :),

(9)

j=1

4

spatial domain. Considering the fact that the commonly used band-by-band TV regularizer neglects such spectral smoothness, it could be useful to design an SSTV regularizer to explore the piecewise smooth structure in both spatial and spectral domains. 1) SSTV based HSIs Denoising Model: The bandwise HSI TV norm for a hyperspectral image X of p bands can be expressed as [22]:

~ j and R(j, l, :) = ~rjl . where L(:, j, :) = L 3) Low-tubal-rank Tensor Factorization based HSI Denoising: As shown in Eq. (9), we can find two low-rank tensors p X L ∈ RM ×k×N and R ∈ Rk×p×N to satisfy F ∗ = L ∗ R. kX kHTV = kX(j) kTV . (15) Here, k is the tensor tubal rank and k ≤ p. Then, we use the j=1 low-rank tensor factorization to re-model the HSI degradation This HTV norm only uses 2-D differencing operation in the model, i.e., horizontal and vertical dimension. Y = inv-Map(L ∗ R) + S + N , (10) To address this limitation, an additional 1-D finite differencing operation is applied in the spectral dimension in the SSTV where L ∗ R is the low-rank tensor factorization of F ∗ . model [24], i.e., We can obtain the following denoising model: kX kSSTV = kDh X Dk1 + kDv X Dk1 , (16) min 12 kY − inv-Map(L ∗ R) − Sk2F + λkSk1 L,R,S (11) where D is a 1-D finite differencing operator on the spectral 0 0 λ λ + 2l kLk2F + 2r kRk2F . signature of each pixel. 2) The Proposed SSTV-LRTF Model: LRTF is a global ∗ In fact, the tensor nuclear norm kF kTNN is closely related to model which exploits the low-rank property of HSIs. SSTV is a the Frobenius norm of the two factor tensors L and R. The local model which studies the correlations of neighboring pixels relationship is given in Lemma 1. and neighboring spectral bands. We are inspired to combine ∗ M ×p×N Lemma 1: For the third-way tensor F ∈ R , suppose the two complementary models to restore hyperspectral data. its tensor tubal rank is upper bounded by k. Then we have We propose the following SSTV-LRTF model: 1 kF ∗ kTNN = inf { (kLk2F + kRk2F ) : F ∗ = L ∗ R}, (12) min kMap(F)kTNN + τ kFkSSTV + λkSk1 L,R 2 F ,S∈RM ×p×N s.t. kY − F − Sk2F ≤ , rank(Map(F)) ≤ k, where L ∈ RM ×k×N and R ∈ Rk×p×N [40]. 0 0 (17) Assuming that λl = λr = λ2 and the upper bound of the where τ is the parameter used to control the trade-off between tensor tubal rank is k, we can have a more simplified denoising tensor nuclear norm and SSTV. model, i.e., min

F ,S,rank(Map(F )) 1 and βmin , βmax with 0 < βmin < βmax < +∞. Initialize an iteration number k ← 0 and β ∈ (βmin , βmax ). repeat Update F k+1 , X k+1 , S k+1 , Λk+1 , Λk+1 and Λk+1 (see 1 2 3 Appendix), Update the parameter β = min(ηβ, βmax ), if kY − F k+1 − S k+1 kF /kYkF ≤ and kF k+1 − X k+1 k∞ ≤ then Break; else k ← k + 1; end if until the maximum number of iterations is reached.

D. Parameter Determination As described in Algorithm 1, the regularization parameters τ and λ, the desired tensor tubal rank k and the stopping criteria 1 and 2 need to be determined in priori. The parameter τ is used to control the trade-off between the tensor nuclear norm and the SSTV norm. A too small value of τ is insufficient to exploit the potential of SSTV. A too large value of τ may oversmooth the restored image. Although the parameter τ cannot be determined analytically, numerical experiments indicate that a value around 0.01 produces reasonable performance on most datasets. The parameter λ is used to adjust the sparsity of the sparse noise. In experiments, λ can be empirically determined following the approach given in [42]. The desired tubal rank k can be manually adjusted to the optimal. In our experiments, we set as 10−8 .

To demonstrate the effectiveness of our hyperspectral image denoising method, we conducted experiments on both simulated and real data using quantitative evaluation and visual inspection. To thoroughly evaluate the proposed algorithm, four methods are used for comparison, including the LRMR method [20], the LRTV method [22], the SSTV method [24] and the Low-rank Tensor Factorization (LRTF) method. LRMR is a well-established low-rank matrix recovery method for HSI denoising. LRTV combines low-rank matrix factorization and TV regularization to simultaneously exploit the spatial and spectral information. SSTV exploits the inherent structure of hyperpectral data using 2-D total variation along the spatial dimension and 1-D total variation along the spectral dimension. In experiments, we tuned the parameters for the proposed method and the four methods for comparison. The upper bound rank for LRMR was manually adjusted to the optimal. The desired rank r of the LRTV method was set as done in [22]. The parameters λ, µ and v of the SSTV method were empirically set as λ = 0.1, µ = 0.2 and v = 0.2. The parameter needed for the LRTF method, the regularization parameter λ, the desired tensor tubal rank k and the stopping criteria were set in the same way as in the SSTV-LRTF method. A. Experiments on Simulated Data The synthetic data was generated using the Pavia city center dataset. The Pavia city center dataset 1 was collected by the Reflective Optics System Imaging Spectrometer (ROSIS-03). The whole image contains 1400 × 512 pixels and 102 spectral bands. In our experiments, the first 22 bands (containing all the noisy bands) of this data were removed, and a subimage of size 300 × 300 × 80 was used, as shown in Fig. 1. To test different

Figure 1. The Pavia city center dataset used in the simulated experiment. Color image is composed of bands 60, 34, and 9 for the red, green, and blue channels, respectively.

noise removal methods, the Peak SNR (PSNR) and Structural Similarity Index Measurement (SSIM) [43] were used. For HSI, we computed the PSNR and SSIM values between each noise-free band and the denoised band. These PSNR and SSIM values of all bands were averaged to produce the mean PSNR (MPSNR) and mean SSIM (MSSIM) metrics. Before denoising, the gray values of each HSI band were normalized to the range of [0, 1]. After denoising, the gray values of each band were stretched to its original range. In our simulated experiment, four types of noise were added to the Pavia city center image. 1 http://www.opairs.aero/rosis

en.html


6

1) Zero-mean Gaussian noise was added to all bands of 2) PSNR and SSIM values: Finally, the PSNR and SSIM HSI. For different bands, the noise intensity is different. values of each band achieved by these denoising approaches The SNR value of each band varies from 10 to 20 dB, are shown in Fig. 11. It can be observed that our proposed and the mean SNR value of all bands is 15.36 dB. method achieves the best performance on almost all bands. 2) Impulse noise was added to 11 selected bands (from Moreover, for bands corrupted by a mixture of Gaussian noise band 30 to band 40). The percentage of impulse noise and sparse noise, our method achieves comparable results to is 20%. the bands contaminated by Gaussian noise only. This clearly 3) Dead lines were simulated on five bands (from band 50 demonstrates the effectiveness of our method for the removal to band 54). The width of the dead lines ranges from of mixed noise. Table I presents the MPSNR and MSSIM one line to three lines. values achieved by five denoising methods. The best results 4) Stripes were simulated on four bands (from band 61 to for each quality index are labeled in bold. The second-best band 64). The width of the stripes ranges from one line results for each quality index are underlined. It is clear that our to three lines. SSTV-LRTF method outperforms the other four approaches in terms of MPSNR and MSSIM. 1) Visualization of selected bands before and after denoising: Figures 2 and 3 show bands 22 and 57 before and after denoising, which are only contaminated by Gaussian noise with B. Experiments on Real Data SNR values of 10.36 and 19.60 dB, respectively. Comparing the 1) EO-1 Hyperion Data Set: In this section, an Earth denoising results achieved by four methods, it can be observed Observing-1 (EO-1) Hyperion dataset is used. The original that LRMR [20], LRTF, SSTV [24] and the proposed SSTV- image has a size of 1000 × 400 × 242. The water absorption LRTF method can effectively suppress Gaussian noise while bands were first removed, and the final test image was cropped preserving local details of the original image. Although LRTV to the size of 400 × 200 × 166. This Hyperion dataset is [22] can remove Gaussian noise and preserve edge information, mainly corrupted by stripes, dead lines and Gaussian noise. some local details are lost, e.g. the building edges. The bands 1-3, 68, 116, 130-132 and 159-166 are severely Bands 31 and 35 are corrupted by Gaussian noise and polluted by stripes and dead lines. Figure 12 shows the color impulse noise, and the denoising results of these two bands image composed of bands 2, 132, and 166. Figures 13-15 are shown in Figs. 4 and 5, respectively. It is shown that present the denoising results of bands 2, 132 and 166. It can be our proposed SSTV-LRTF denoising method and SSTV can observed that the proposed SSTV-LRTF method obtains the best effectively remove the mixed Gaussian and impulse noise. performance. The proposed method successfully suppresses LRTV can also remove this mixed noise, but the details Gaussian noise and sparse noise, while local details and are oversmoothed and some artifacts are introduced to the structural information of the image are preserved. As shown smooth area. The LRMR method fails to remove impulse in Figs. 14 and 15, both LRMR and LRTF fail to restore noise. The LRTF method performs well on band 31 with low- some stripes. This is mainly because these stripes exist in level Gaussian noise (with SNR of 17.0605 dB). However, the the same place in most of the noisy HSI bands. They are LRTF method fails to remove mixed noise on band 35 with considered as a part of the low-rank clean image by the LRMR high-level Gaussian noise (with SNR of 10.97 dB). and LRTF methods. Some deadlines and strips are remained Figures 6-8 show the noise removal results for the mixture in the image restored by SSTV. SSTV only considers the of Gaussian noise and dead lines on bands 51, 53 and 54, spectral correlation of neighboring bands. Different from lowrespectively. Figures 9 and 10 illustrate the denoising results rank based models, SSTV cannot model the global spectral for the mixture of Gaussian noise and stripes on bands 62 correlation. This limitation leads to inferior performance of and 64, respectively. From Figs. 7(g)-10(g), it can be clearly SSTV. For LRTV, noise is well suppressed, but the image is observed that the proposed SSTV-LRTF method removes over-smoothed and some details (e.g. the ridge texture) are Gaussian noise, dead lines and stripes while local details are lost. preserved. Some deadline and stripes remain in images restored The vertical mean profiles of bands 2, 132 and 166 before by the LRMR method and the SSTV method, as shown in Figs. and after denoising are presented in Figs. 16-18. The rapid 7 and 10. As observed in Fig. 8(d), some dead lines remain fluctuations in Figs. 16(a)-18(a) suggest the existence of stripes in the image restored by the LRTV method. This is mainly and deadlines. From Fig. 16(b), it can be seen that the LRMR because the LRTV method uses bandwise TV constraint to method can successfully remove deadlines in band 2, but cannot preserve the structure of clean image. Such a strategy cannot suppress random noise and strips. The LRTF method can distinguish whether the structure is part of clean image or just remove deadlines and some stripes, but some stripes still remain, part of noise. As shown in Figs. 9(d) and 10(d), LRTV can as shown in Fig. 16(d). The profile in Fig. 16(e) suggests that effectively suppress strips but tends to oversmooth the image. the SSTV method cannot remove deadlines. On the other For example, the building edges are blurred. The LRTF method hand, the reduction of small fluctuations indicates that SSTV can successfully remove stripes and deadlines for images with can effectively remove stripes. From Figs. 16(c) and 16(f), low-level Gaussian noise including band 53 with a SNR of we can see that LRTV and SSTV-LRTF achieve comparable 16.60 dB and band 64 with a SNR of 16.99 dB, as shown noise removal performance. However, LRMR and LRTF fail in Figs. 7(e) and 10(e). However, the LRTF method fails on to remove strips on bands 132 and 166, as the fluctuations still images with high-level Gaussian noise, as shown in Fig. 6(e). exist obviously in Figs. 17 and 18. Some strips remain in the


(a)

(b)

(c)

(d)

7

(e)

(f)

(g)

Figure 2. Denoising results in the simulated experiment. (a) Original band 22, (b) noisy band, (c) LRMR, (d) LRTV, (e) LRTF, (f) SSTV, (g) SSTV-LRTF.

(a)

(b)

(c)

(d)

(e)

(f)

(g)


(a)

(b)

(c)

(d)

(e)

(f)

(g)


(a)

(b)

(c)

(d)

(e)

(f)

(g)


image restored by SSTV, as shown in Figs. 17(e) and 18(e). 104-108, 150-163 and 220. Figure 19 shows the color image Strips can be successfully suppressed by the LRTV method, by combining bands 3, 147 and 219. The Indian Pines dataset but the spectral signatures may be oversmoothed, as presented is mainly corrupted by the atmosphere and water absorption. in Figs. 17(c) and 18(c). The proposed SSTV-LRTF method The first few bands and the last few bands are also seriously can effectively remove strips, as shown in Figs. 17(f) and 18(f). polluted by Gaussian noise and impulse noise. We selected two typical bands, namely band 1 and band 2) AVIRIS Indian Pines Data Set: In this section, we consider the real noisy Indian Pines dataset 2 . This dataset 219, to present the denoising performance of all the compared was acquired by the NASA Airborne Visible/Infrared Imaging methods in Figs. 20 and 21. It is easy to observe that low-rank Spectrometer (AVIRIS) instrument over the Indian Pines test matrix based methods, including LRMR and LRTV, can more site in Northwestern Indiana in 1992. The Indian Pines dataset or less remove some noise, but when the noise is heavy, such has a size of 145 × 145 × 220, covering the wavelength range methods lose their utility and even occurred some degradation of 0.4-2.5 µm. The number of bands is reduced to 200 by of the gray value, as shown in Figs. 20 (c) and 21 (b)-(c).The removing the bands covering the region of water absorption: LRTF method only exploits the global low-rank structure of the data, and fail to restore band 1 with heavy noise, as shown in Fig. 20 (d). And in band 219, unwanted stripes are introduced 2 http://www.ehu.eus/ccwintco/index.php?title=Hyperspectral Remote Sensing Scenes in the image restored by LRTF. The SSTV method can remove


(a)

(b)

(c)

(d)

8

(e)

(f)

(g)


(a)

(b)

(c)

(d)

(e)

(f)

(g)


(a)

(b)

(c)

(d)

(e)

(f)

(g)


(a)

(b)

(c)

(d)

(e)

(f)

(g)


the mixed noise to some extent, and obtain a satisfying result exclusive. The groundtruth of the 16 classes can be found in in band 219, as shown in Fig. 21(e). But for band 1, SSTV the website 3 . The classification process is repeated 10 times loses its utility and the restored image is covered by the noise. (training samples are chosen randomly 10 times). Table II From Figs. 20 (f) and 21 (f), we can see that the proposed shows the classification accuracies achieved by the proposed SSTV-LRTF method can remove lots of noise and preserve SSTV-LRTF method and other four methods. It can be seen that edges and local information. Compared to the other methods, the OA values are improved after denoising the data. Among the SSTV-LRTF method can still get best performance for all classification results, the LRMR method and the proposed removing heavy mixed noise in the Indian Pines dataset. SSTV-LRTF method achieve higher OA values than the other Then, the classification accuracy can be adopted to evaluate methods. 3) HYDICE Urban Data Set: The HYDICE Urban image the denoising performance. Here, a support vector machine (SVM), with cross validation (CV) for selecting the tuning can be downloaded online 4 . The size of the original image is parameters, is utilized to conduct supervised classification for 307 × 307 × 210, and we selected a subimage of size 200 × HSI data [44]. The effectiveness of these methods is evaluated 200 × 210 for our experiment. The urban image presented in in terms of classification accuracy: the overall accuracy (OA). 3 https://purr.purdue.edu/publications/1947/1 The Indian Pines dataset includes 10249 samples and the 4 http://www.erdc.usace.army.mil/Media/Fact-Sheets/ samples are divided into 16 classes which are not mutually Fact-Sheet-Article-View/Article/610433/hypercube/


(a)

(b)

(c)

9

(d)

(e)

(f)

(g)

Figure 10. Denoising results in the simulated experiment. (a) Original band 64, (b) noisy band, (c) LRMR, (d) LRTV, (e) LRTF, (f) SSTV, (g) SSTV-LRTF. 50

1

45

0.9 0.8

40

0.7 0.6

SSIM

PSNR

35 30

SSTV-LRTF SSTV LRTF LRTV LRMR

25 20 15

10

20

30

40

50

60

70

SSTV-LRTF SSTV LRTF LRTV LRMR

0.4 0.3 0.2 0.1

10 0

0.5

80

Band number

0

0

10

20

30

40

50

60

70

80

Band number

(a)

(b)

Figure 11. PSNR and SSIM values of each band achieved by different denoising methods. (a) PSNR value and (b) SSIM value. Table I MPSNR AND MSSIM VALUES OF THE DENOISING RESULTS IN THE SIMULATED EXPERIMENT. B OLDFACE MEANS THE BEST AND UNDERLINE MEANS THE SECOND BEST. Metrics

LRMR

LRTV

LRTF

SSTV

SSTV-LRTF

MPSNR (dB)

37.6638

38.0612

41.4013

43.0953

44.6571

MSSIM

0.9188

0.9574

0.9518

0.9710

0.9849

Figure 12. EO-1 Hyperion dataset used in the real data experiment. Color image is composed of bands 2, 132, and 166 for the red, green, and blue channels, respectively.

Since LRTV uses the spatial TV regularization to exploit the spatial information, it can remove this kind of stripes and preserve the edge information to a certain extent. The SSTV method exploits the local smoothness in the spatial and spectral domains. Thus, it can suppress the stripes to some extent. However, different from the low-rank based method, SSTV cannot model the global spectral correlation. This limitation leads to the inferior performance of SSTV. By combining the spectral TV and spatial TV into a unified SSTV regularization, and using the low-rank tensor factorization, our proposed SSTVLRTF method can better remove the complex mixed noise and preserve the spatial mixture compared with the other methods.

After denoising the HYDICE dataset, the spectral unmixing analysis could be done in order to show the effect of denoising on abundance fraction map extraction.The bands of the Urban Fig. 22 is polluted by stripes, deadlines, the atmosphere, water datset cover the wavelength of 400-2500 nm, which include lowabsorption, and other unknown noise. noise bands, noisy bands, and water-absorption bands. Here we Figs. 23 and 24 show the restorations of band 109 and band use “low-noise” to describe the high-SNR bands, which include 207, respectively. It is not hard to see that the two low-rank bands 5-75, 77-86, 88-100, 112-135, and 154-197. The noisy methods, i.e. LRMR and LRTF, cannot effectively remove the bands include bands 1-4, 76, 87, 101-104, 110-111, 136-138, stripes. This is mainly because of the fact that stripes and 152-153, and 198-207. The rest of the bands are categorized deadlines exist in the same place from band 104 to 110, and in as water-absorption bands, which cannot provide any useful the same place from band 199 to 210. That is, in the low-rank information, only noise. In most researches, only high-SNR and sparse decomposition, the stripes are more likely regarded bands were adopted to unmix the Urban data. However, the lowas the low-rank part, which is assumed to be the clean image. noise bands are also corrupted by sparse noise. The existence


(a)

(b)

(c)

(d)

10

(e)

(f)

Figure 13. Denoising results on the EO-1 Hyperion dataset. (a) noisy band 2, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 14. Denoising results on the EO-1 Hyperion dataset. (a) noisy band 132, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF.

of sparse noise can degrade the unmixing performance. When τ = 0, the SSTV-LRTF model reduces to a LRTF model. In traditional unmixing procedures, only low-noise image When τ is high, the influence of TV regularization becomes is used[45], which consists of 162 low-noise bands of the strong. We calculated MPSNR and MSSIM with different Urban data. In this paper, we use the whole noisy image, which values of parameter τ within the range of [0, 0.1]. The results consists of 162 low-noise bands and 27 noisy bands for spectral are shown in Fig. 25. We can see that MPSNR and MSSIM unmixing. Six types of the signatures named “Asphalt”, “Grass”, are sensitive to the value of τ . When τ is around 0.01, our “Concrete road”, “Roof]1”, “Roof]2” and “Tree” were estimated SSTV-LRTF method achieves the best MSPNR and MSSIM in the image[46]. The reference signatures were collected from results. We can also see that, when τ > 0.02, the values of the spectral library 5 . The SID-based method was used to select MPSNR and MSSIM are lower than those achieved with τ = 0. four pixels as the initial endmember matrix[45]. The SISAL That means, a value smaller than 0.02 should be used for τ to method proposed in [47] was used for spectral unmixing. The achieve good regularization performance. spectral unmixing results were evaluated using the spectral 2) Sensitivity Analysis of Parameter λ: In the proposed angle distance (SAD). The SAD was used to compare the SSTV-LRTF method, λ is the parameter used to restrict the similarity of the endmember signature and its estimate. Table sparsity of the sparse noise. As in the RPCA model presented III gives the SAD values of the spectral unmixing results in [48], the sparsity regularization parameter was set to of the Urban data denoised by different methods. From the λ = 1/√M N (M and N are the two dimensions in the spatial table, it can be clearly seen that the proposed SSTV-LRTF domain), which was enough to guarantee the existence of an opmethod achieves lower mean SAD value than the other methods. timal solution. In Fig. 26, we set λ = q/√M N and changed q This demonstrates the superiority of SSTV-LRTF over other from the set of [0.1, 0.3, 0.5, 0.7, 1, 2, 4, 6, 8, 10, 20, 25]. From denoising methods on abundance fraction map extraction. the figure, it can be observed that the results of the SSTV-LRTF solver are relatively stable, in both MPSNR and MSSIM values, C. Discussion when q was changed from 10 to 25. Hence, in the SSTV-LRTF √ 1) Sensitivity Analysis of Parameter τ : Parameter τ gives a method, the sparsity parameter was set to λ = 20/ M N . trade off between low-rank constraint and TV regularization. 3) Convergence of SSTV-LRTF: For ALM, its convergence 5 http://www.agc.army.mil/ has been well studied when the number of blocks (i.e., unknown


(a)

(b)

(c)

11

(d)

(e)

(f)

Figure 15. Denoising results on the EO-1 Hyperion dataset. (a) noisy band 166, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF. 0.17

0.15

0.16

0.16 0.15

0.14

0.15

Mean DN

Mean DN

0.13 0.12 0.11

Mean DN

0.14

0.14

0.13

0.12

0.13

0.12

0.11

0.1

0.11

0.09 0.1

0.1

0.08 0.07 0

20

40

60

80

100

120

140

160

180

0.09

200

0.09 0

20

40

60

Row number

80

100

120

140

160

180

200

0

20

40

60

Row number

(a)

(b)

0.16

0.16

0.15

0.15

0.14

0.14

80

100

120

140

160

180

200

140

160

180

200

Row number

(c) 0.15

0.13 0.12 0.11

Mean DN

Mean DN

Mean DN

0.14

0.13 0.12 0.11

0.13

0.12

0.11 0.1

0.1

0.09

0.09

0.1

0.08

0.09

0.08 0

20

40

60

80

100

120

Row number

(d)

140

160

180

200

0

20

40

60

80

100

120

Row number

(e)

140

160

180

200

0

20

40

60

80

100

120

Row number

(f)

Figure 16. Vertical mean profiles of band 2 before and after denoising. (a) noisy, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF.

tensor/matrix variables) is up to two [49]. Cai et al. [50] and Li et al. [51] have proved the convergence of ALM with at least one strongly convex function and two convex functions for the three-block case. In Algorithm 1, the three blocks F, X and S are convex but not strongly convex. So, it is difficult to prove the convergence of our algorithm theoretically based on [50] and [51]. However, it is known that the ALM algorithm generally performs well in practice, as illustrated in [52]. In fact, if the parameters are appropriately determined, the proposed algorithm converges before the maximum unmber of iterations is reached. To demonstrate the convergence of SSTV-LRTF, the MPSNR and MSSIM values achieved at each iteration of the simulated experiment are shown in Fig. 27. It can be observed that, the MPSNR and MSSIM values of the proposed method are converged as the iteration progresses. The MPSNR and MSSIM values increase rapidly during the first few iterations, and then

grow slowly in the subsequent iterations. Although the preset maximum number of iterations is 40, the algorithm reaches its stopping criterion at the 36th iteration (as shown in Fig. 27). Therefore, the proposed algorithm can be successfully converged on the test dataset.

4) Computational Time Comparison: Here, we compare the running time of the proposed method with other methods. The experiments were performed on a laptop with a 1.6-2.30 GHz Intel Core CPU and 8 GB memory using Matlab. The time costs on the Pavia city center dataset are reported in Table IV. It can be seen that the time cost of our denoising method ranks in the middle. To make our SSTV-LRTF method more efficient, a Riemannian preconditioning approach can be used for tensor completion, the same as [53].


0.34

0.28

0.32

0.27

0.3

0.26

0.28

0.25

12

0.28

0.27

0.24

Mean DN

Mean DN

Mean DN

0.26

0.26

0.24 0.23

0.22

0.22

0.2

0.21

0.25

0.24

0.23

0.18

0.2

0.16

0.19 0

20

40

60

80

100

120

140

160

180

200

0.22

0.21

0

20

40

60

80

100

120

140

160

180

200

0

20

40

60

Row number

Row number

(a)

80

(b)

0.29

100

120

140

160

180

200

140

160

180

200

Row number

(c)

0.28

0.28

0.27

0.27

0.26

0.26

0.28

Mean DN

Mean DN

0.26 0.25 0.24 0.23 0.22

Mean DN

0.27

0.25

0.24

0.25

0.24

0.23

0.23

0.22

0.22

0.21 0.2 0.19

0.21

0

20

40

60

80

100

120

140

160

180

Row number

(d)

200

0

20

40

60

80

100

120

140

160

180

0.21

200

0

20

40

60

Row number

80

100

120

Row number

(e)

(f)


V. C ONCLUSION In this paper, a method has been proposed to remove mixed noise in HSIs using LRTF and SSTV regularization. LRTF is used to exploit the global low-rank structure of HSI data and to separate the low-rank clean image from sparse noise. The SSTV regularization is employed to consider both edge constraint in 2-D spatial domain and high correlation in the neighbouring bands of an HSI. Several experiments have been conducted on simulated and real-world datasets to demonstrate the superior performance of our method. Although the SSTVLRTF method is originally developed for HSI denoising, it can be applied to other tensorial data processing tasks such as video processing, web data analysis and bioinformatics.

A. Updating of F k+1 For Eq. (20), we can deduce that F k+1

=

l(F, X k , S k , Λk1 , Λk2 )

argmin rank(Map(F ))≤k

argmin kMap(F)kTNN rank(Map(F ))≤k

+ Λk1 , X k − F + Λk2 , Y − F − S k + β2 kX k − F k2F + kY − F − S k k2F =

=

(25)

kMap(F)kTNN + βkF + Λk1 + Λk2 /β k2F

argmin

rank(Map(F ))≤k − 21 Y + X k − S k

For simplicity, we denote the iteration of F k+1 as F k+1

:=

(kMap(F)kTNN + βkF − Qk k2F )

argmin rank(Map(F ))≤k

ACKNOWLEDGMENT

(26) where Qk = 12 Y + X k − S k + Λk1 + Λk2 /β . In fact, Eq. (26) is equivalent to:

This research is supported by the National Natural Science Foundation of China under Grants 61503288, 61601481, F k+1 = argmin (kMap(F)kTNN + βkMap(F) 61602499 and 61471371, the National Postdoctoral Program rank(Map(F ))≤k for Innovative Talents under Grant BX201600172, and China −Map(Qk )k2F ) Postdoctoral Science Foundation. We thank Wei He and (27) Hongyan Zhang from the State Key Laboratory of Informa- we denote that F∗ = Map(F) and Qk∗ = Map(Qk ). According tion Engineering in Wuhan University for their constructive to Definition 4, the optimization problem can be solved by the discussion. following tensor recovery problem in frequency domain:

APPENDIX

In this appendix, we describe the details for the updating of F k+1 , X k+1 and S k+1 in Eqs. (20)-(22).

F k+1 = argmin rank(F∗ ))≤k

(

N X i=1

ˆ (i) ˆ ˆk 2 kF ∗ k∗ + βkF∗ − Q∗ kF )

(28) where Fˆ∗ is the discrete Fourier transform of F∗ along the ˆ k∗ . Eq. (28) can be third dimension. Similarly, we can define Q


13

0.35

0.36

0.4

0.34 0.34

0.33

0.3

0.25

0.32

Mean DN

Mean DN

Mean DN

0.35

0.3

0.32 0.31 0.3

0.28

0.29 0.2

0.26

0.28 0.27

0.24

0.15 0

20

40

60

80

100

120

140

160

180

0

200

20

40

60

80

100

120

140

160

180

0

200

20

40

60

(a)

80

(b)

0.38

100

120

140

160

180

200

140

160

180

200

Row number

Row number

Row number

(c) 0.34

0.35 0.34

0.33 0.33

0.32

0.32

0.32

0.3

0.31

Mean DN

0.34

Mean DN

Mean DN

0.36

0.3 0.29 0.28

0.31

0.3

0.29

0.27

0.28

0.28

0.26 0.25

0.26 0

20

40

60

80

100

120

140

160

180

0

200

20

40

60

Row number

80

100

120

140

160

180

200

0.27 0

20

40

60

Row number

(d)

80

100

120

Row number

(e)

(f)


Using Lemma 2, we can have ˆ ∗(i) F

k(i)

ˆ∗ ) 1 (Q := D 2β

(33)

Therefore, we can obtain the SVD of Fˆ∗ as ˆ S, ˆ V] ˆ = SVD(Fˆ∗ ) [Q,

(34)

and Figure 19. AVIRIS Indian Pines dataset used in real data experiment 2.

ˆ (i) broken up to N independent minimization problems. Let F ∗ ˆ denotes the i-th frontal slice of F∗ : ˆ (i) F ∗ =

(i)

(i)

k(i)

ˆ ∗ k∗ + βkF ˆ∗ − Q ˆ ∗ k) argmin(kF

s.t.

B. Updating of X k+1 (29)

ˆ (i) F ∗

rank(F∗ )) ≤ k

In order to solve Eq. (29), we give the following lemma: Lemma 2: Consider the singular value decomposition (SVD) of a matrix A ∈ Rn1 ×n2 of rank r. A = Q ∗ S ∗ V,

S = diag({σi }1≤i≤r )

ˆ [], 3), S = ifft(S, ˆ [], 3), V = ifft(V, ˆ [], 3) Q = ifft(Q, (35) To take the tensor tubal rank constraint rank(F∗ )) ≤ k into consideration, we set S(i, i, :) = 0, i > k. Then, we have F∗k+1 = Q ∗ S ∗ V and F k+1 = inv-Map(F∗k+1 ).

(30)

where Q ∈ Rn1 ×r and V ∈ Rn2 ×r are orthogonal, and the singular values σi are real and positive. Then, for all τ > 0, we define the soft-thresholding operator D,

For Eq. (21), we can deduce that X k+1

= argmin l(F k+1 , X , S k , Λk1 , Λk2 ) X

= argmin τ kX kSSTV + Λk1 , X − F k+1 X

+ β2 kX − F k+1 k2F

β Λk kX + 1 − F k+1 k2F 2 β X (36) The problem defined in Eq. (36) can be expressed as: = argmin τ kX kSSTV +

β Λk Dτ (A) := Q ∗ Dτ (S) ∗ V, Dτ (S) = diag({(σi − τ )+ }1≤i≤r ) min τ kDh X Dk1 + τ kDv X Dk1 + kX + 1 − F k+1 k2F X 2 β (31) (37) where x+ is an operator x+ = max(0, x). Then, for each τ > 0 This inseparable optimization problem can be rewritten as a and B ∈ Rn1 ×n2 , the singular value shrinkage operator defined constrained separable problem: in Eq. (31) obeys Λk min τ kAk1 + τ kBk1 + β2 kX + β1 − F k+1 k2F 1 2 (38) X Dτ (A) = argmin{ kB − AkF + τ kBk∗ } (32) s.t. A = Dh X D, B = Dv X D 2 B


(a)

(b)

(c)

14

(d)

(e)

(f)

Figure 20. Denoising results on the Indian Pines dataset. (a) noisy band 1, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 21. Denoising results on the Indian Pines dataset. (a) noisy band 219, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF. Table II C LASSIFICATION ACCURACY ACHIEVED ON THE I NDIAN P INES DATASET USING D IFFERENT D ENOISING M ETHODS . ] Samples

Methods

Class

Training/Total

SVM

LRMR

LRTV

LRTF

SSTV

Alfalfa

15/46

0.7391

0.9565

0.8260

0.8695

0.9565

0.9130

Corn-notill

100/1428

0.5329

0.8597

0.6759

0.5572

0.7131

0.7060

Corn-mintill

100/830

0.5287

0.8125

0.7275

0.5812

0.6425

0.8175

Corn

50/237

0.8019

0.8019

0.9758

0.8260

0.9371

0.9468

Grass-pasture

50/483

0.8410

0.9072

0.9249

0.9315

0.9183

0.9470

Grass-tree

100/730

0.8871

0.9971

0.9557

0.9514

0.9100

0.9785

Grass-pasture-mowed

15/28

0.9285

0.9390

0.9200

0.9005

0.9950

0.9296

Hay-windrowed

50/478

0.8861

0.9977

0.8303

0.9553

0.9754

0.9062

Oats

15/20

0.7000

0.9968

0.9958

0.9920

0.5000

0.9992

Soybeans-notill

100/972

0.6167

0.8131

0.6974

0.6613

0.6380

0.7728

Soybeans-mintill

150/2455

0.4853

0.8235

0.5826

0.5059

0.5905

0.7216

Soybeans-cleantill

50/593

0.5133

0.8419

0.5772

0.5257

0.6323

0.8028

Wheat

50/205

0.9675

0.9778

0.9942

0.9885

0.9910

0.9942

Woods

100/1265

94.75

0.7303

0.9724

0.8226

0.8882

0.8858

Buildings-Grass-Trees-Drives

50/386

0.5617

0.8342

0.5983

0.7247

0.7612

0.8707

Stone-Steel-Towers

50/93

0.9523

0.9841

0.9970

0.9047

0.9682

0.9996

Kappa

0.6078

0.8665

0.7013

0.6633

0.7185

0.7966

Overall accuracy

0.6265

0.8754

0.7262

0.6779

0.7434

0.8462

This problem can be solved by the ALM algorithm, which minimizes the following augmented Lagrangian function: min

X ,A,B

τ kAk1 + τ kBk1 + Λk3 , A − Dh X D

+ Λk4 , B − Dv X D + γ2 (kA − Dh X Dk2F +kB − Dv X Dk2F ) + β2 kX +

Λk 1 β

X

A

γ 2 kA

γ Λk kB − Dv X D + 4 k2F 2 γ

− Dh X D +

+ β2 kX +

− F k+1 k2F

γ Λk kA − Dh X D + 3 k2F 2 γ

B

min

Λk 1 β

Λk 3 2 γ kF

+ γ2 kB − Dv X D +

(41)

Λk 4 2 γ kF

− F k+1 k2F

(39)

(42) The sub-problems in Eqs. (40)-(41) are defined in the form:

(40)

1 argmin kY − X k2F + αkX k1 2 X

The above problem can be split into three sub-problems: min τ kAk1 +

min τ kBk1 +

SSTV-LRTF

(43)


15

Table III SAD VALUES OF THE DIFFERENT METHODS WITH THE HYDICE DATASET. B OLDFACE MEANS THE BEST AND UNDERLINE MEANS THE SECOND BEST. Class

Original Data

LRMR

LRTV

LRTF

SSTV

SSTV-LRTF

Asphalt

0.1232

0.1610

0.1131

0.0907

0.1075

0.1107

Grass

0.1460

0.2469

0.2412

0.3120

0.1288

0.2142

Concrete Road

0.4427

0.2297

0.3685

0.3547

0.4012

0.2867

Roof ]1

0.5284

0.2559

0.3866

0.5305

0.1972

0.2497

Roof ]2

0.1716

0.1055

0.1765

0.1537

0.2075

0.1417

Tree

0.1993

0.2055

0.1892

0.1822

0.1776

0.1960

Mean

0.2685

0.2008

0.2459

0.2706

0.2033

0.1998

Table IV T HE RUNNING T IME C OSTED BY D IFFERENT M ETHODS ON THE PAVIA CITY CENTER IN THE SIMULATION EXPERIMENT. B OLDFACE MEANS THE BEST AND UNDERLINE MEANS THE SECOND BEST. Method

LRMR

LRTV

LRTF

SSTV

SSTV-LRTF

Time (s)

79.1866

365.9470

155.3776

600

537.2447

The solution of Eq. (47) is: S k+1

:=

shrink(Y − F k+1 +

Λk λ 2 β , β)

(48)

R EFERENCES

Figure 22. HYDICE urban dataset used in real data experiment 3.

which can be solved by a soft shrinkage operator shrink(Y, α). Here, shrink(·, ·) is an element-wise soft shrinkage operator. Λk For each element a of tensor Y − F k+1 + β2 , we have:    a − α, a > α 0,

shrink(a, α)

 

|a| ≤ α

a + α,

(44)

a < −α

The sub-problem defined in Eq. (42) can be solved by an iterative least square solver, such as LSQR [24] [54]. The Lagrangian multipliers Λk+1 and Λk+1 can be updated 3 4 as follows: Λk+1 = Λk3 + γ(A − Dh X D) 3

(45)

Λk+1 = Λk4 + γ(B − Dv X D) 4

(46)

C. Updating of S k+1 For Eq. (22), we can deduce that S k+1

= argmin l(F k+1 , X k+1 , S, Λk1 , Λk2 ) S

= argmin λkSk1 + Λk2 , Y − F k+1 − S S

+ β2 kY − F k+1 − Sk2F β Λk = argmin λkSk1 + kY − F k+1 − S + 2 k2F 2 β S (47)

[1] A. F. H. Goetz, “Three decades of hyperspectral remote sensing of the earth: A personal view,” Remote Sensing of Environment, vol. 113, no. 9, p. S5S16, 2009. [2] J. M. Bioucas-Dias, A. Plaza, G. Camps-Valls, and P. Scheunders, “Hyperspectral remote sensing data analysis and future challenges,” IEEE Geoscience and Remote Sensing Magazine, vol. 1, no. 2, pp. 6–36, 2013. [3] J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot, “Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches,” IEEE journal of selected topics in applied earth observations and remote sensing, vol. 5, no. 2, pp. 354–379, 2012. [4] H. S. Martin J. Montag, “Hyperspectral unmixing from incomplete and noisy data,” Journal of Imaging, 2016. [5] C. Andreou and V. Karathanassi, “Estimation of the number of endmembers using robust outlier detection method,” IEEE journal of selected topics in Applied earth observations and Remote sensing, vol. 7, no. 1, pp. 247– 256, 2014. [6] Z. Zhang, E. Pasolli, M. M. Crawford, and J. C. Tilton, “An active learning framework for hyperspectral image classification using hierarchical segmentation,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, no. 9, 2016. [7] F. Fan, Y. Ma, C. Li, X. Mei, J. Huang, and J. Ma, “Hyperspectral image denoising with superpixel segmentation and low rank representation,” Information Sciences, vol. 397, pp. 48–68, 2017. [8] R. Archibald and G. Fann, “Feature selection and classification of hyperspectral images with support vector machines,” IEEE Geoscience and Remote Sensing Letters, vol. 4, no. 4, pp. 674–677, 2007.


(a)

16

(b)

(c)

(e)

(f)

(d)

Figure 23. Denoising results on the HYDICE Urban dataset. (a) noisy band 109, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF.

[9] A. Majumdar, N. Ansari, H. Aggarwal, and P. Biyani, “Impulse denoising for hyper-spectral images: A blind compressed sensing approach,” Signal Processing, vol. 119, pp. 136–141, 2016. [10] C. Rogass, C. Mielke, D. Scheffler, N. K. Boesche, A. Lausch, C. Lubitz, M. Brell, D. Spengler, A. Eisele, K. Segl et al., “Reduction of uncorrelated striping noiseapplications for hyperspectral pushbroom acquisitions,” Remote Sensing, vol. 6, no. 11, pp. 11 082–11 106, 2014. [11] H. Shen and L. Zhang, “A map-based algorithm for destriping and inpainting of remotely sensed images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 5, pp. 1492–1502, 2009. [12] Y.-Q. Zhao and J. Yang, “Hyperspectral image denoising via sparse representation and low rank constraint,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 1, pp. 296–308, 2015. [13] C. Jiang, H. Zhang, L. Zhang, H. Shen, and Q. Yuan, “Hyperspectral image denoising with a combined spatial and spectral weighted hyperspectral total variation model,” Canadian Journal of Remote Sensing, pp. 1–20, 2016. [14] M. Wang, J. Yu, and W. Sun, “Group-based hyperspectral image denoising using low rank representation,” in IEEE International Conference on Image Processing (ICIP). IEEE, 2015, pp. 1623–1627. [15] Y. Yuan, X. Zheng, and X. Lu, “Spectral-spatial kernel regularized for hyperspectral image denoising,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 7, pp. 3815–3832, 2015. [16] M. Yan, “Restoration of images corrupted by impulse noise and mixed gaussian impulse noise using blind inpainting,” SIAM Journal on Imaging Sciences, vol. 6,

no. 3, pp. 1227–1245, 2013. [17] Y. Xiao, T. Zeng, J. Yu, and M. K. Ng, “Restoration of images corrupted by mixed Gaussian-impulse noise via l 1–l 0 minimization,” Pattern Recognition, vol. 44, no. 8, pp. 1708–1720, 2011. [18] J. Liu, X.-C. Tai, H. Huang, and Z. Huan, “A weighted dictionary learning model for denoising images corrupted by mixed noise,” IEEE Transactions on Image Processing, vol. 22, no. 3, pp. 1108–1120, 2013. [19] X. Lu, Y. Wang, and Y. Yuan, “Graph-regularized lowrank representation for destriping of hyperspectral images,” IEEE transactions on geoscience and remote sensing, vol. 51, no. 7, pp. 4009–4018, 2013. [20] H. Zhang, W. He, L. Zhang, H. Shen, and Q. Yuan, “Hyperspectral image restoration using low rank matrix recovery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 8, pp. 4729–4743, 2014. [21] R. Zhu, M. Dong, and J. H. Xue, “Spectral nonlocal restoration of hyperspectral images with low-rank property,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, no. 6, pp. 3062– 3067, 2015. [22] W. He, H. Zhang, L. Zhang, and H. Shen, “Total-variationregularized low-rank matrix factorization for hyperspectral image restoration,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 1, pp. 178–188, Jan 2016. [23] B. Rasti, M. O. Ulfarsson, and J. R. Sveinsson, “Hyperspectral feature extraction using total variation component analysis,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 12, pp. 6976–6985, 2016. [24] H. K. Aggarwal and A. Majumdar, “Hyperspectral image denoising using spatio-spectral total variation,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 3,


(a)

17

(b)

(c)

(e)

(f)

(d)

Figure 24. Denoising results on the HYDICE Urban dataset. (a) noisy band 207, (b) LRMR, (c) LRTV, (d) LRTF, (e) SSTV, (f) SSTV-LRTF. 45

1

0.9

40

MSSIM

MPSNR

0.95

0.85

0.8

35 0.75

0.7

30

0.65

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0

0.01

0.02

0.03

0.04

0.05

0.06

= values

= values

(a)

(b)

0.07

0.08

0.09

0.1

Figure 25. Sensitivity analysis for parameter τ . τ varies from 0 to 0.1. (a) The values of MPSNR with different τ , (b) The values of MSSIM with different τ .

pp. 442–446, 2016. [25] M. Ye, Y. Qian, and J. Zhou, “Multitask sparse nonnegative matrix factorization for joint spectral-spatial hyperspectral imagery denoising,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2621– 2639, 2015. [26] Z. Wu, Q. Wang, J. Jin, and Y. Shen, “Structure tensor total variation-regularized weighted nuclear norm minimization for hyperspectral image mixed denoising,” Signal Processing, vol. 131, pp. 202–219, 2017. [27] A. Karami, R. Heylen, and P. Scheunders, “Band-specific shearlet-based hyperspectral image noise reduction,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 9, pp. 5054–5066, 2015. [28] N. Renard, S. Bourennane, and J. Blanc-Talon, “Denoising and dimensionality reduction using multilinear tools for hyperspectral images,” IEEE Geoscience and Remote Sensing Letters, vol. 5, no. 2, pp. 138–142, 2008. [29] A. Karami, M. Yazdi, and A. Z. Asli, “Noise reduction

[30]

[31]

[32]

[33]

of hyperspectral images using kernel non-negative tucker decomposition,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 3, pp. 487–493, 2011. D. Letexier and S. Bourennane, “Noise removal from hyperspectral images by multidimensional filtering,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 7, pp. 2061–2069, 2008. X. Liu, S. Bourennane, and C. Fossati, “Denoising of hyperspectral images using the parafac model and statistical performance analysis,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 10, pp. 3717–3724, 2012. X. G. Guo, X. Huang, L. Zhang, and L. Zhang, “Hyperspectral image noise reduction based on rank-1 tensor decomposition,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 83, pp. 50–63, 2013. T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Review, vol. 51, no. 3, pp. 455–500, 2009.


45

18

1 0.95

MSSIM

MPSNR

0.9 0.85

40

0.8 0.75 0.7

35 0.65 0.6 0.55

30

0.5

0

5

10

15

20

25

0

5

10

q values

15

20

25

q values

(a)

(b)

√ Figure 26. Sensitivity analysis for parameter λ (q from 0.1 to 25 with λ = q/ M N . Here, M and N are the two dimensions in the spatial domain). (a) Change in the MPSNR value. (b) Change in the MSSIM value.

1

45

0.9 0.8

40

MSSIM

MPSNR

0.7

35

30

0.6 0.5 0.4 0.3 0.2

25

0.1 0

20 0

5

10

15

20

25

30

35

40

Iterations (a)

0

5

10

15

20

25

30

35

40

Iterations (b)

Figure 27. The MPSNR and MSSIM results achieved by the proposed SSTV-LRTF method on the simulated data with different number of iterations. (a) MPSNR results. (b) MSSIM results.

[34] R. Bro and H. A. Kiers, “A new efficient method for determining the number of components in parafac models,” Journal of Chemometrics, vol. 17, no. 5, pp. 274–286, 2003. [35] O. Semerci, N. Hao, M. E. Kilmer, and E. L. Miller, “Tensor-based formulation and nuclear norm regularization for multienergy computed tomography.” IEEE Transactions on Image Processing, vol. 23, no. 4, pp. 1678–93, 2014. [36] M. E. Kilmer and C. D. Martin, “Factorization strategies for third-order tensors,” Linear Algebra and Its Applications, vol. 435, no. 3, pp. 641–658, 2011. [37] M. E. Kilmer, K. Braman, N. Hao, and R. C. Hoover, “Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging,” SIAM Journal on Matrix Analysis and Applications, vol. 34, no. 1, pp. 148–172, 2013. [38] J. Ma, C. Chen, C. Li, and J. Huang, “Infrared and visible image fusion via gradient transfer and total variation minimization,” Information Fusion, vol. 31, pp. 100–109, 2016. [39] J. Ma, Y. Ma, and C. Li, “Infrared and visible image fusion methods and applicaions: a survey,” Inf. Fusion, vol. 45, pp. 153–178, 2019. [40] Z. Zhang, D. Liu, S. Aeron, and A. Vetro, “An online tensor robust PCA algorithm for sequential 2-d data,” in International Conference on Acoustics, Speech, and

Signal Processing, 2016. [41] C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, and S. Yan, “Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization,” in Proceedings of CVPR, 2016. [42] Z. Zhang, G. Ely, S. Aeron, N. Hao, and M. Kilmer, “Novel methods for multilinear data completion and denoising based on tensor-svd,” Computer Science, vol. 44, no. 9, pp. 3842–3849, 2014. [43] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. [44] P. K. Gotsis, C. C. Chamis, and L. Minnetyan, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 8, pp. 1778–1790, 2004. [45] W. He, H. Zhang, and L. Zhang, “Sparsity-regularized robust non-negative matrix factorization for hyperspectral unmixing,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 9, no. 9, pp. 4267–4279, 2016. [46] ——, “Total variation regularized reweighted sparse nonnegative matrix factorization for hyperspectral unmixing,” IEEE Transactions on Geoscience and Remote Sensing, 2017. [47] J. M. Bioucas-Dias, “A variable splitting augmented


[48]

[49]

[50]

[51]

[52]

[53]

[54]

lagrangian approach to linear spectral unmixing,” in The Workshop on Hyperspectral Image & Signal Processing: Evolution in Remote Sensing, 2009, pp. 1–4. E. J. Cands, X. Li, Y. Ma, and J. Wright, “Robust principal component analysis?” Journal of the ACM, vol. 58, no. 3, p. 11, 2011. Y. Zhang, “Recent advances in alternating direction methods: Practice and theory,” Tutorial, vol. 3, pp. 1– 3, 2010. X. Cai, D. Han, and X. Yuan, “The direct extension of ADMM for three-block separable convex minimization models is convergent when one function is strongly convex,” Optimization Online, 2014. M. Li, D. Sun, and K. Toh, “A convergent 3-block semi-proximal ADMM for convex minimization problems with one strongly convex block,” Asia Pacific Journal of Operational Research, vol. 32, no. 4, 2014. Q. Qu, N. M. Nasrabadi, and T. D. Tran, “Abundance estimation for bilinear mixture models via joint sparse and low-rank representation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 7, pp. 4404– 4423, 2014. H. Kasai and B. Mishra, “Riemannian preconditioning for tensor completion,” arXiv preprint arXiv:1506.02159, 2015. M. A. Saunders, “Solution of sparse rectangular systems using LSQR and CRAIG,” BIT Numerical Mathematics, vol. 35, no. 4, pp. 588–604, 1995.

Haiyan Fan received the B.S. degree in information engineering from National University of Defense Technology, Changsha, China in 2011. She had studied as a master student majoring in information and communication engineering in National University of Defense Technology from Sept. 2011 to Jan. 2013. She was admitted as a Ph.D. student ahead of graduation time due to her excellent performance in the master study at Feb. 2013. She received the Ph.D. degree from the National University of Defense Technology in 2017. She is now a Lecturer in the Space Engineering University, Beijing, China. Her research interests lie in the areas of tensor analysis, data mining and deep learning.

Chang Li received the B.S. degree of information and computing science from Wuhan Institute of Technology, Wuhan, China, in 2012. He received the Ph.D. degree from the School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China, in 2018. He is now a Lecturer in the Department of Biomedical Engineering, Hefei University of Technology, Hefei, China. His current research interests include in the areas of hyperspectral image analysis, computer vision, pattern recognition and machine learning.

19

Yulan Guo received his B.Eng. and Ph.D. degrees from National University of Defense Technology (NUDT) in 2008 and 2015, respectively. He was a Visiting Ph.D. Student at the University of Western Australia from November 2011 to November 2014. He is currently an Assistant Professor with the College of Electronic Science and Engineering, NUDT. He authored more than 40 peer reviewed articles in journals and conferences, such as IEEE TPAMI and IJCV. He served as a reviewer for more than 30 international journals and conferences. His research interests include computer vision and pattern recognition, particularly on 3D feature learning, 3D modeling, 3D object recognition, and 3D face recognition. He received the NUDT Distinguished Ph.D. Thesis award in 2015 and the CAAI Distinguished Ph. D. Thesis award in 2016.

Gangyao Kuang (M’11) received the B.S. and M.S. degrees from the Central South University of Technology, Changsha, China, in 1998 and 1991, respectively, and the Ph.D. degree from the National University of Defense Technology, Changsha, in 1995. He is currently a Professor and Director of the Remote Sensing Information Processing Laboratory in the School of Electronic Science and Engineering, National University of Defense Technology. His current interests mainly include remote sensing, SAR image processing, change detection, SAR ground moving target indication, and classification with polarimetric SAR images.

Jiayi Ma received the B.S. degree from the Department of Mathematics and the Ph.D. degree from the School of Automation, Huazhong University of Science and Technology, Wuhan, China, in 2008 and 2014, respectively. From 2012 to 2013, he was an Exchange Student with the Department of Statistics, University of California at Los Angeles, Los Angeles, CA, USA. He is currently an Associate Professor with the Electronic Information School, Wuhan University, Wuhan, where he was a Post-Doctoral Researcher from 2014 to 2015. His current research interests include computer vision, machine learning, and pattern recognition. He has been serving as an Associate Editor of IEEE Access. He has authored and co-authored more than 80 scientific articles.

Spatial-Spectral Total Variation Regularized Low ...

Spatial-Spectral Total Variation Regularized Low ...

Suggest Documents

A Novel 3D Anisotropic Total Variation Regularized Low Rank ... - MDPI

Total-Variation Regularized SENSE Reconstruction - CiteSeerX

FTVd is beyond Fast Total Variation regularized

FTVd is beyond Fast Total Variation regularized

DUAL REGULARIZED TOTAL LEAST SQUARES

Regularized Constrained Total Least-Squares

Simultaneous Low-Pass Filtering and Total Variation ... - IEEE Xplore

Laplacian Regularized Low-Rank Representation ... - Semantic Scholar

GRAPH REGULARIZED LOW-RANK MATRIX ... - Google Sites

Optimizing Parametric Total Variation Models

a regularized total least squares algorithm - Semantic Scholar

A FAST ALGORITHM FOR SOLVING REGULARIZED TOTAL LEAST

Regularized estimation of vertical total electron content from Global ...

Adaptively regularized constrained total least-squares image restoration

IMPROVED TOTAL VARIATION-TYPE REGULARIZATION ... - Mathpost

SUBSPACE CORRECTION METHODS FOR TOTAL VARIATION AND ...

Iterative Nonlocal Total Variation Regularization ... - Semantic Scholar

IMAGE RESTORATION: TOTAL VARIATION, WAVELET ... - Maths, NUS

Performance Comparison of Total Variation based ...

Iterative total-variation reconstruction versus ... - Semantic Scholar

Total Variation Image Edge Detection - wseas

Total Variation Minimization with Separable Sensing Operator

Controlled Total Variation regularization for inverse problems

Comparison of Total Variation with a Motion