Compressive hyperspectral imaging recovery by ... - OSA Publishing

Vol. 26, No. 6 | 19 Mar 2018 | OPTICS EXPRESS 7043

Compressive hyperspectral imaging recovery by spatial-spectral non-local means regularization PABLO M EZA , 1,* I VAN O RTIZ , 1 E STEBAN V ERA , 2 M ARTINEZ 1

AND

J AVIER

1 Department 2 School

of Electrical Engineering, University of La Frontera, Temuco, Chile of Electrical Engineering, Pontificia Universidad Católica de Valparaíso, Valparaíso, Chile

* [email protected]

Abstract: Hyperspectral imaging systems can benefit from compressed sensing to reduce data acquisition demands. We present a new reconstruction algorithm to recover the hyperspectral datacube from limited optically compressed measurements, exploiting the inherent spatial and spectral correlations through non-local means regularization. The reconstruction process is solved with the help of split Bregman optimization techniques, including penalty functions defined according to the spatial and spectral properties of the scene and noise sources. For validation purposes, we also implemented a compressive hyperspectral imaging system that relies on a digital micromirror device and a near-infrared spectrometer, where we obtained enhanced and promising reconstruction results when using our proposed technique in contrast with traditional compressive image reconstruction. © 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement OCIS codes: (110.0110) Imaging systems; (110.4234) Multispectral and hyperspectral imaging; (170.1630) Coded aperture imaging.

References and links 1. D. Donoho, “Compressed sensing,” IEEE Transactions on Information Theory 52, 1289–1306 (2006). 2. E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Transactions on Information Theory 52, 489–509 (2006). 3. D. Takhar, J. N. Laska, M. B. Wakin, M. F. Duarte, D. Baron, S. Sarvotham, K. F. Kelly, and R. G. Baraniuk, “A new compressive imaging camera architecture using optical-domain compression,” Proc. SPIE 6065, 606501 (2006). 4. L. Gao, J. Liang, C. Li, and L. V. Wang, “Single-shot compressed ultrafast photography at one hundred billion frames per second,” Nature 516, 74–77 (2014). 5. J. V. Thompson, J. N. Bixler, B. H. Hokr, G. D. Noojin, M. O. Scully, and V. V. Yakovlev, “Single-shot chemical detection and identification with compressed hyperspectral raman imaging,” Opt. Lett. 42, 2169–2172 (2017). 6. M. Borengasser, W. S. Hungate, and R. L. Watkins, Hyperspectral Remote Sensing: Principles and Applications (CRC, 2008). 7. P. Meza, E. Vera, and J. Martinez, “Improved reconstruction for compressive hyperspectral imaging using spatialspectral non-local means regularization,” Proc. IS&T international Symposium on Electronic Imaging 2016, 1–5 (2016). 8. M. E. Gehm, R. John, D. J. Brady, R. M. Willett, and T. J. Schulz, “Single-shot compressive spectral imaging with a dual-disperser architecture,” Opt. Express 15, 14013–14027 (2007). 9. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47, B44–B51 (2008). 10. T. Sun and K. Kelly, “Compressive sensing hyperspectral imager,” Frontiers in Optics 2009/Laser Science XXV/Fall 2009 OSA Optics & Photonics Technical Digest p. CTuA5 (2009). 11. D. Needell and J. Tropp, “Cosamp: Iterative signal recovery from incomplete and inaccurate samples,” Applied and Computational Harmonic Analysis 26, 301–321 (2009). 12. S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM Journal on Scientific Computing 20, 33–61 (1998). 13. E. Candes and T. Tao, “The dantzig selector: Statistical estimation when p is much larger than n,” The Annals of Statistics 35, 2313–2351 (2007). 14. R. Ward, “Compressed sensing with cross validation,” IEEE Transactions on Information Theory 55, 5773–5782 (2009). 15. J. Bioucas-Dias and M. Figueiredo, “A new twist: Two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Transactions on Image Processing 16, 2992–3004 (2007).

#309417 Journal © 2018

https://doi.org/10.1364/OE.26.007043 Received 28 Dec 2017; revised 9 Feb 2018; accepted 10 Feb 2018; published 8 Mar 2018


16. S. Becker, J. Bobin, and E. J. Candés, “Nesta: A fast and accurate first-order method for sparse recovery,” SIAM Journal on Imaging Sciences 4, 1–39 (2011). 17. M. E. Gehm and D. J. Brady, “Compressive sensing in the eo/ir,” Appl. Opt. 54, C14–C22 (2015). 18. L. Gómez-Chova, L. Alonso, L. Guanter, G. Camps-Valls, J. Calpe, and J. Moreno, “Correction of systematic spatial noise in push-broom hyperspectral sensors: application to CHRIS/PROBA images,” Appl. Opt. 47, F46–F60 (2008). 19. A. E. Gamal, B. A. Fowler, H. Min, and X. Liu, “Modeling and estimation of fpn components in cmos image sensors,” Proc. SPIE 3301, 330101 (1998). 20. H. Tian, B. Fowler, and A. Gamal, “Analysis of temporal noise in cmos photodiode active pixel sensor,” IEEE Journal of Solid-State Circuits 36, 92–101 (2001). 21. P. Meza, J. E. Pezoa, and S. N. Torres, “Multidimensional striping noise compensation in hyperspectral imaging: Exploiting hypercubes’s spatial, spectral, and temporal redundancy,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 9, 4428–4441 (2016). 22. C. Li, W. Yin, H. Jiang, and Y. Zhang, “An efficient augmented lagrangian method with applications to total variation minimization,” Computational Optimization and Applications 56, 507–530 (2013). 23. A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” Proc. IEEE Conference on Computer Vision and Pattern Recognition 2, 60–65 (2005). 24. J. Zhang, S. Liu, R. Xiong, S. Ma, and D. Zhao, “Improved total variation based image compressive sensing recovery by nonlocal regularization,” IEEE International Symposium on Circuits and Systems, 2836–2839 (2013). 25. J. Tropp and A. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Transactions on Information Theory 53, 4655–4666 (2007). 26. Y. Li, F. Li, B. Bai, and Q. Shen, “Image fusion via nonlocal sparse k-svd dictionary learning,” Appl. Opt. 55, 1814–1823 (2016). 27. W. Huang, L. Xiao, H. Liu, and Z. Wei, “Hyperspectral imagery super-resolution by compressive sensing inspired dictionary learning and spatial-spectral regularization,” Sensors 15, 2041–2058 (2015). 28. J. Yang, Y. Li, J. C.-W. Chan, and Q. Shen, “Image fusion for spatial enhancement of hyperspectral image via pixel group based non-local sparse representation,” Remote Sensing 9, 53 (2017). 29. T. Goldstein and S. Osher, “The split bregman method for l1-regularized problems,” SIAM Journal on Imaging Sciences 2, 323–343 (2009). 30. M. F. Duarte and Y. C. Eldar, “Structured compressed sensing: From theory to applications,” IEEE Transactions on Signal Processing 59, 4053–4085 (2011). 31. S. Cuomo, P. D. Michele, and F. Piccialli, “3d data denoising via nonlocal means filter by using parallel gpu strategies,” Computational and mathematical methods in medicine 2014, 523862 (2014). 32. D. S. Antony and G. Rathna, “Gpu based fast non local means algorithm,” Journal of Image and Graphics 3, 122–125 (2015).

1.

Introduction

Compressive sensing (CS) theory have provided with the mathematical formalism whereby novel sampling schemes are able to overcome traditional limitations imposed by the Shannon-Nyquist theorem, enabling compressive imaging system architectures that can benefit from reduced size, weight, power and cost requirements. In particular, CS states that a N-pixel image can be reconstructed from a reduced set of M N linear measurements employing a sub-Nyquist sampling rate instead of using an uniformly-spaced isomorphic sampling, which is possible if the signal can be represented by a reduced number of non-zero components in some domain [1, 2]. Since CS requires fewer measurements, it can be applied to reduce the number of detector elements needed for imaging systems. Initially, imaging via CS was experimentally demonstrated by using the single-pixel camera architecture based on a digital micromirror devices (DMDs) that partially multiplexed and coded random parts of the scene for every measurement, which is finally integrated onto a detector [3]. Through the years, CS ideas have also been extended to acquire higher dimensionality imaging data–otherwise not feasible to be performed–such as spatio-temporal datacubes [4] and hyperspectral Raman imaging datacubes [5]. The compressive sampling paradigm have been particularly fruitful for the design of hyperspectral imaging (HSI) systems in order to tackle the data deluge problem given its high dimensionality [6]. In this case, CS would allow that a datacube with N × N-pixels with L spectral bands may be reconstructed from a set of M N 2 L linear measurements. Thus, compressive hyperspectral imaging (CHI) is possible since hyperspectral datacubes are often highly compressible, not only due the fact that natural images–the images formed in every spectral


band–are sparse when projected into an adequate basis, but also due to the high correlation found between adjacent bands [7]. Gehm et al. [8] proposed a snapshot spectral imager, dualdisperser (DD)-Coded aperture snapshot spectral imager (CASSI), based on a 4- f optical system with two prisms as spectral dispersive elements and a binary coded aperture. Later, Wagadarikar et al. [9] presented a different version by omitting one prism, known as single-disperser (SD)-CASSI. Further, CHI can be also implemented by extending the single-pixel camera scheme proposed in [3], replacing the photo-diode with a spectrometer as presented in [10]. From now on we will focus on this CS imaging scheme due to its simplicity and the potential to reuse available spectral equipment. Once the sequence of measurements are taken, several sparse recovery algorithms can be used to estimate the hyperspectral datacube from a highly convex problem with an infinite number of candidate solutions [11]. Despite the fact that this is an ill-posed problem, CS theory provides a set of conditions that, if satisfied, assure an accurate estimation of the signal/image [1]. Reconstruction can be achieved by using basis pursuit with inequality constraints, orthogonal matching pursuit or iterative hard thresholding algorithms [12–14], although the minimization of large-scale problems is more efficiently computed by solvers such as L1-Magic, two-step iterative shrinkage/thresholding (TwIST), or NESTA [15, 16], for instance. Nevertheless, experimental results have shown that CHI is sensitive not only to the selection of sparsity bases but also to noise sources [17]. CHI is affected by inhomogeneous pixel responsivities [18], photodetector’s dark current and readout electronics that introduces fixed and temporal noise per pixel, respectively [19, 20]. Further, CHIs is affected by the limited spectral responsivity of the detectors in the lower and higher spectral bands and low irradiance due to the spectral dispersion [21]. Therefore, the use of prior information beyond pure sparsity have become a mandatory task for reducing the set of feasible solutions. In the imaging case, total variation (TV) edge preserving regularization has been successfully coupled to CS recovery by assuming that the gradients of a natural image are sparse, which has been efficeintly implemented by the TVAL3 state-of-the-art solver [22]. Other priors, such as wavelet-based or non-local means (NLM), are used with the TV approach by assuming spatial redundancy in the scene [23, 24]. Nonetheless, HSI is not only correlated in the spatial domain but also in the spectral domain. In particular, hyperspectral images are highly correlated in the spectral domain, which should also be exploited during the CS recovery process [25]. In this sense, algorithms for fusion and super-resolution in HSI have used the similarity in spatial-spectral patches of data to provide useful information for preserving details. The non-local self-similarity property have been employed in a dictionary learning process for improved image fusion results [26] and as a regularization term by using the similarity of spectral curves for the reconstruction of spatial super-resolution [27,28]. In this work we propose a new recovery algorithm for CHI by posing an optimization problem that includes a modified NLM spatial-spectral regularization term, solved using split Bregman techniques. The presented technique allows reconstruction of datacubes with higher fidelity from fewer compressive measurements. We present simulated results using hyperspectral imagery acquired using a push-broom hyperspectral camera (PBHC) setup and also experimental results from our CHI setup based on [10], where the scene is spatially multiplexed by a coded aperture using a DMD and focused onto a spectrometer. Results demonstrate a competitive performance in contrast with reconstructions obtained via the TVAL3 state-of-the-art reconstruction method [22]. The rest of this article is structured as follows. Section 2 describes the basic CHI model and the mathematical derivations for the proposed algorithm. Further, the metric used to assess the performance of the rendered spectral images are presented. In Section 3, simulated results using PBHC datasets and experimental results with the laboratory CHI setup are described and discussed. Finally, the main conclusions of this work are outlined, as well as future trends for this work, are presented in Section 4.


2.

Proposed algorithm

In the CHI scheme proposed by Sun and Kelly [10], each detector of the linear spectrometer acquires a narrow spectral band of CS spatial measurements. It should be noted that since the scene is spatially multiplexed by the coded aperture before the spectral decomposition takes place, each detector observes the same sensing matrix but applied to its particular band. The acquired sensor response for each detector affected by random noise is modeled by: fk = Φuk + ηk = ΦΨθ k + ηk ,

(1)

where Φ represents the projection matrix and uk corresponds to a 2D spatial image, reshaped in a vector form of N × 1, at the k-th spectral channel. Ψ is the sparse basis matrix of size N × N with the basis vectors ψn as columns and θ k is an N × 1 sparse column vector. The parameter ηk corresponds to the noise column, usually defined as a random zero-mean Gaussian noise. Finally, fk corresponds to the acquired compressive measurements. The reconstruction of the data is tackled as a l 1 -norm optimization problem, subject to some restriction, usually reformulated with inequality constrains [29]. In this work we will start the formulation with the following optimization problem: β min kΨθ k k 1 + k fk − ΦΨθ k k22 . (2) 2 θk ∈R N ×1 To solve it, we have selected a compressive reconstruction scheme based on the gradient domain TV model as the sparsifying domain. The TV semi-norm enforces sparsity of the reconstructed image gradient and can be calculated using the following anisotropic model: kΨθ k k 1 = kuk k 1 = |∇x uk | + |∇y uk |.

(3)

It should be noted that we have selected Φ as a Gaussian projection matrix to ensure the incoherence with the sparse basis matrix [30]. Traditionally, such as is the case of the TVAL3 algorithms [22], the reconstruction is made in a band-per-band basis independently, without considering the information available on the other spectral bands. To exploit the potential spatial-spectral redundancy observed in single-DMD CHI schemes, Eq. (2) is now coupled with a spatial-spectral NLM regularization as follows: L

min kuk kTV +

uk ∈R N ×1

Õ α β Wkl ul k22 + k fk − ΦΨθ k k22, kuk − 2 2 l=1

(4)

where l = 1, . . . , k, . . . , L. The NLM regularization compensates the presence of noise while maintaining the edges and the fine texture image details. It is based on the assumption that it is likely to exist redundant spatial information in the scene. However, the expression has been modified to take into account any spectral redundancy by comparing and averaging spatial-spectral neighborhood patches. Each element of the data matrix Wkl is obtained as follows: wkl (i, j) : j ∈ Oi Wkl = (5) 0 : otherwise, where wkl (i, j) is the weight component calculated as: ! 2 −kuk (Oi ) − ul (O j )k2,g 1 wkl (i, j) = exp , Zk (i) h2 ! 2 ÕÕ −kuk (Oi ) − ul (O j )k2,g Zkl (i, j) = exp . h2 j l

(6)

(7)


The suffixes i and j denote the center pixel of the reference patch and the center pixel of the patches to be compared, respectively. Therefore, ul (Oi ) defines the column vector that contains the pixels of the image patch at the l-th spectral channel, in a square neighborhood of fixed size Oi centered at the i-th position. The term h2 is known as the NLM filtering parameter, usually 2 stands for the Euclidean distance set equal to the standard deviation of the noise. Also, k∗k2,g and g for the standard deviation of the Gaussian kernel that imposes a decaying weight to pixels away from the patch center. In order to minimize Eq. (4), a split Bregman approach have been used, redefining the expression without constraints and replacing the TV with auxiliary variables (dx,k = ∇x uk and dy,k = ∇y uk ). This is done by iteratively solving the unconstrained problem, and then modifying the value of uk , dx,k , dy,k used in the next iteration.

p+1

p+1

p+1

uk , dx,k , dy,k

p+1

ck

p+1

e x,k

p+1

ey,k

L

=

min

uk ,d x, k ,dy, k

|dx,k | + |dy,k | +

Õ β α p kuk − Wkl ul k22 + kΦuk − fk + ck k22 + 2 2 l=1

γ γ p p p p k∇x uk − dx,k + e x,k k22 + k∇y uk − dy,k + ey,k k22 2 2 p

p+1

= ck + Φuk − fk p p+1 p+1 = e x,k + ∇x uk − dx,k p p+1 p+1 = ey,k + ∇y uk − dy,k

(8) (9) (10) (11)

The auxiliary variables used in the split Bregman method, ck , e x,k , and ey,k are the sub-differential of a function in a particular point. A more detailed explanation on how the split Bregman method can be applied to solve L1 problems can be found in [29]. 2.1.

Image quality metrics

Two metrics were selected to evaluate the performance of the CHI reconstruction algorithms: i) the root mean-squared error (RMSE) and ii) the roughness metric. The first metric, RMSE, is used to quantify the distortions between the corresponding ground-truth reference and the estimated image. The mathematical representation of the metric is: "

N 1 Õ ˆ − u(n))2 RMSE(u, ˆ u) = (u(n) N n=1

# 12 .

(12)

a low RMSE value indicates that the estimated image is closer to the reference. The second metric is the roughness (ρ) and measures the amount of image details in the spatial domain. The ρ metric does not use a reference image, unlike the RMSE, and is calculated as follows: ρ(u) ˆ =

kh ∗ uk ˆ + khT ∗ uk ˆ , k uk ˆ

(13)

where h is defined as [1 − 1] , the symbol ∗ correspond to the convolution and k · k to the l 1 -norm. For the ρ metric, a small value (near zero) indicates that the image is smooth, with lower presence of noise or artifacts, since both effects increase the high frequency content of the scene. 3.

Experimental results

In this section we develop a comparative analysis between the state-of-the-art TVAL3 method [22] and the proposed spatial-spectral NLM algorithm. The TVAL3 method is a very efficient and flexible compressed sensing solver to reconstruct images using total variation minimization (TV). Assuming that the gradients of natural images are sparse for CS reconstruction, TVAL3 is able to


deliver prompt reconstructions using augmented Lagrangian and alternating direction algorithms. This method serves as an excellent baseline to show whether adding spatial-spectral information is beneficial for CHI reconstruction, particularly due to the fact that our proposed algorithm only differs in the addition of the NLM spatial-spectral regularization. First, we present the results obtained using the CS reconstruction algorithms on synthetic compressive measurements over a PBHC dataset corrupted by Gaussian noise. Then, we test the algorithms on two sets of CS data captured with a laboratory CHI setup based on the scheme proposed in [10]. 3.1.

Laboratory setups for simulated and real experimental data acquisition

A set of full recorded hyperspectral cubes were obtained through an experimental setup based on a Photonfocus Hurricane 40 V10E PBHC and a calibrated quartz tungsten halogen (QTH) lamp to guarantees an uniform and continuous spectral illumination. The camera has a CMOS array composed of 1024 × 1024 pixels, coupled with a spectrograph that records 574 spectral bands, between 400 to 1000 [nm]. The setup includes also a mobile platform to emulate the along-track scanning procedure, configured to take 800 temporal samples along the spatial trajectory. Also, a two-point calibration procedure was conducted to remove fixed-pattern noise, which allows to add controlled amounts of synthetic noise during the simulations. The calibration device employed here was a Spectralon SRT-99-120, which ensures a diffuse reflectance of 99% in the same spectral range of the PBHC. The experimental setup is presented in Fig. 1(a). In addition, a CHI laboratory prototype was implemented, as seen Fig. 1(b), to experimentally analyze the performance of the proposed CS algorithm. We used a QTH lamp to illuminate the target scene, which is imaged onto the DMD based on a dual lens optical system with a focal length of 12 [mm] (MVL12M23) and 75 [mm] (LA1608). The selected DMD was the DLP4500NIR which consists of an array of electrostatically actuated micro-mirrors with a defined resolution of 800 × 800. The reflected light is optically multiplexed and coupled to the fiber (QP1000-2-VIS-NIR) by a plano-convex lens (LA1131) with a focal length of 50 [mm]. The optical fiber have a core size of 1000 [µm], a numerical aperture of 0.22, and an SMA connector coupled to a USB2000+ spectrometer. This spectrometer has a spectral range that spans from 540-1100 [nm], with a resolution of 16 bits of digital output and 2048 spectral bands. Further, to compensate for the dark noise, two successive measurements (dual-rail) were performed and then subtracted. This procedure consists in dividing each pattern P, projected by the DMD, into a positive and negative pattern, P+ and P− , respectively. Each pattern is calculated as P+ = (J + P) /2 and P− = (J − P) /2, where J is an all 1s matrix and P = P+ − P− .

(a) Fig. 1. Experimental setups based on the a) Photonfocus Hurricane 40 V10E PBHC and the b) CHI laboratory prototype.

(b)


3.2.

Synthetic compressive measurement

Two hyperspectral datacubes, I and II, were acquired as shown in Fig. 2. For each hypercube, two spatially cropped image segments of size 128 × 128 pixels were selected in order to reduce simulation time, respectively blue and red squares. Consequently, the region of interest (ROI) hypercubes have a dimension of 128 × 574 × 128 in the spatial, spectral, and temporal domain, respectively. Two spectral images were selected over the 574 bands to evaluate the performance of the CS reconstruction. Moreover, the matrix Φ was constructed as a random projection matrix considering a 15% of compression ratio. Since the proposed algorithm is based on exploiting the redundant information recorded in neighboring bands, comparisons using 5 adjacent spectral bands during the estimation are considered as well.

(a)

(b)

Fig. 2. Sample images taken at 790 [nm] from the acquired hypercubes a) I and b) II. The two squares, red and blue, depict the cropped hypercubes used to simulate the CS measurements.

As a first experiment, different noise levels of synthetic random Gaussian noise, defined as 0%, 10% and 20% of the dynamic range, were added to evaluate the performance of both reconstruction algorithms. Table 1 shows RMSE values obtained by both algorithms when reconstructing the two cropped hypercubes (H.I and H.II). As expected, both methods render better results when reducing the noise level, however, the proposed algorithm achieve significantly lower RMSE values than TVAL3 in all cases. Henceforth, we will use the worst case scenario, a noise level of 20%, for the rest of the experiments. Table 1. RMSE metrics for the estimated cropped spectral images (blue and red) from hypercubes I and II (H.I and H.II) affected by 0%, 10% and 20% noise, using the TVAL3 and the proposed algorithm.

Metric TVAL-RMSE

Proposed-RMSE

ROI H.I-B H.I-R H.II-B H.II-R H.I-B H.I-R H.II-B H.II-R

λ = 661[nm]-ηk 0% 10% 20% 21.17 24.94 31.76 20.36 23.12 25.16 27.67 39.78 58.97 38.25 50.36 73.52 14.33 18.50 23.18 15.30 16.53 19.93 13.64 20.00 35.30 26.31 41.60 61.25

λ = 790[nm]-ηk 0% 10% 20% 23.08 27.55 30.95 19.58 29.82 36.10 19.13 25.15 30.54 19.24 24.45 36.13 18.42 19.68 24.26 16.24 18.26 19.54 18.79 21.40 24.62 17.60 18.55 19.07


Reconstructed spectral images using TVAL3 and the proposed algorithm, at 661 [nm] and 790 [nm], are presented in Figs. 3 and 4, respectively.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

Fig. 3. Sample images taken at 661 [nm], where (a-d) correspond to the four references images from hypercubes I and II (H.I and H.II). Images from (e-h) and (i-l) correspond to the rendered results using the TVAL3 and the proposed algorithm, respectively.

Figures 3(a)-3(d) and 4(a)-4(d) correspond to both reference cropped images extracted from the hypercubes I and II, as described in Fig. 3. Figures 3(e)-3(h) and 4(e)-4(h) are the resulting images employing the TVAL3 method over the information recorded at a single spectral band. In a similar fashion, Figs. 3(i)-3(l) and 4(i)-4(l) depict the reconstructions obtained with the proposed algorithm exploiting the information in 5 neighboring bands. A naked eye evaluation shows that both algorithms successfully recover the spatial structure of the scene, however, the proposed algorithm using the additional spectral bands produces images with better defined shapes and fewer artifacts. In particular, the original Figs. 3(c)-3(d) show spectral images with a low level of signal power, and also show how the TVAL3 reconstruction is affected in Figs. 3(g)-3(h). On the other hand, Figs. 3(k)-3(l) show how the proposed algorithm is able to surpass this problem. The spectral band selected for the reference images in Figs. 4(a)-4(d) was chosen due to its diversity content in the spatial structure. In a similar manner, Figs. 4(e)-4(h) display some defocusing effect after being reconstructed using TVAL3, which in turn is compensated when using the proposed algorithm, as can be observed in Figs. 4(i)-4(l). For example, when comparing Figs. 4(e) and 4(i), one can assess that both algorithms successfully rendered the spectral images, however, the last image exhibits better defined shapes and fewer artifacts. The following Table 2 shows comparative results for the recovery algorithms over the two cropped hypercubes affected by synthetic random Gaussian noise. Note that all the metrics were calculated over the selected ROI, blue and red, depicted in Fig. 2. Overall, the results show


(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

Fig. 4. Sample images taken at 790 [nm], where (a-d) correspond to the four references images from hypercubes I and II (H.I and H.II). Images from (e-h) and (i-l) correspond to the rendered results using the TVAL3 and the proposed algorithm, respectively.

that the proposed method outperforms TVAL3 by achieving smaller RMSE values. The lowest RMSE is achieved in Fig. 4(h) and 4(l) with a 36.13 versus a 19.07, respectively for the proposed and the TVAL3 algorithm. Similar results for the algorithms were observed in terms of the ρ. More precisely, the ρ metric exhibits that the TVAL algorithm produces a general reduction in high-frequency content, reaching ρ values between 0.17 to 0.27, often smaller than the reference. On the other hand, the higher level of spatial details observed in Figs. 3(i)-3(l) and Figs. 4(i)-4(l) are reflected on the ρ metric, ranging between 0.18 to 0.48. It should be noted that, in this case, a close-to-zero value does not mean a good result because the algorithms are designed to increase the detail content in the estimated images. Given that the reference image is available, achieving a ρ value after reconstruction closer to the reference value means that the algorithm is most likely successfully recovering the original spatial details in the scene. In the particular case for the reconstructed images in Figs. 4(i)-4(l), the ρ values were slightly higher than expected due to fact that the reconstructions were noisier in contrast with pure TV regularization. 3.3.

Raw compressive measurement

For this analysis, raw compressive measurements were acquired using the CHI laboratory prototype described in Fig. 1(b), setting the spectrometer with an integration time of 0.1 [s]. Two DMD resolutions were defined in order to estimate the target scene with two different spatial resolutions, grouping the micro-mirrors in a subset of 8 × 8 and 4 × 4 pixels. Consequently, the estimated spectral datacubes have 2048 spectral bands with a spatial resolution of 100 × 100 and 200 × 200. Further, for each set of spectral data we have constructed a Gaussian random sensing


Table 2. Performance metrics for the estimated cropped spectral images (blue and red) from hypercubes I and II (H.I and H.II) using the TVAL3 and the proposed algorithm.

Metric RMSE

ρ

ROI H.I-B H.I-R H.II-B H.II-R H.I-B H.I-R H.II-B H.II-R

λ = 661[nm] Reference TVAL Proposed 31.76 23.18 25.16 19.93 58.97 35.30 73.52 61.25 0.20 0.19 0.26 0.24 0.26 0.34 0.38 0.26 0.45 0.45 0.27 0.48

λ = 790[nm] Reference TVAL Proposed 30.95 24.26 36.10 19.54 30.54 24.62 36.13 19.07 0.16 0.17 0.19 0.14 0.17 0.18 0.15 0.17 0.22 0.22 0.20 0.24

matrix Φ considering 10%, 15%, and 20% of compression ratio.

Fig. 5. Sample images at 533 [nm] with a spatial resolution of 100 × 100 pixels using the TVAL3 algorithm (a-c) and the proposed algorithm (d-f), considering a 10%, 15%, and 20% compression ratio.

As before, the TVAL3 method is compared with the proposed algorithm exploiting the redundant information recorded in 5 adjacent spectral bands during the estimation. The results for the reconstructed spectral images, with different spatial resolutions, taken at 533 [nm] are presented in Figs. 5 and 6. Figures 5(a)-5(c) and 5(d)-5(f) correspond to the 100 × 100 rendered images using TVAL3 and the proposed algorithm, respectively, considering the aforementioned compression ratios. From the evaluation of Figs. 5(a) and 5(d), it can be appreciated that the spatial structure of the numbers and bars are better reconstructed with the proposed method,


despite the fact that the measurements are heavily affected by electronic noise. In the same manner, Figs. 6(a)-6(c) and 6(d)-6(f) correspond to the 200 × 200 reconstructed images using TVAL3 and the proposed algorithm, respectively. As expected, both algorithms benefit from the increase in spatial resolution, however, the additional spatial information is now exploited by the NLM regularizer. In particular, this can be observed when comparing the rendered Figs. 6(a) and 6(d), where the level of available information is low due to the high compression ratio.

Fig. 6. Sample images at 533 [nm] with a spatial resolution of 200 × 200 pixels using the TVAL3 algorithm (a-c) and the proposed algorithm (d-f), considering a 10%, 15%, and 20% compression ratio.

Table 3 summarizes the results from Figs. 5 and 6 using the roughness metric. The roughness values for the TVAL3 algorithm range from 0.12 to 0.15 and 0.18 to 0.24 for the 100 × 100 and 200 × 200 rendered images, respectively. The increase in ρ value, from both figures, is due to the increase in image resolution and the number of measurements taken, where the best result (0.24) is achieved by Fig. 6(c). For the proposed algorithm, the ρ values range from 0.18 to 0.24 and 0.31 to 0.45, all larger when compared with the respective TVAL3 ρ values. Table 3. Roughness metric for the rendered images taken at 533 [nm], with a spatial resolution of 100 × 100 and 200 × 200, using the TVAL3 and the proposed algorithm.

Figure 5 6

(a) 0.12 0.18

TVAL3 (b) (c) 0.15 0.15 0.21 0.24

Proposed Algorithm (d) (e) (f) 0.17 0.21 0.22 0.31 0.40 0.45

As a final test, the spectral reconstruction properties of the proposed algorithm are analyzed using a constructed scene using 4 band-pass optical filters. The filters FB850-10, FB900-10,


FB1000-10, and FB1100-10 are spectrally tuned at 850, 900, 1000, and 1100 [nm], respectively, with a full-width half-maximum (FWHM) of 10±2 [nm]. Figures 7(a)-7(d) show the reconstructed spectral images at each of the aforementioned wavelengths using the same acquisition parameters than before. The curves presented in Figs. 7(e)-7(f) correspond to the spectral signature measured with the spectrometer and the estimated signature produced by the proposed algorithm, considering 5 neighboring spectral bands during the reconstruction. As can be observed, the spectral reconstruction is similar to the reference measurement, with a notorious increase in signal that can be appreciated specially at 1100 [nm]. It should be remarked that such increase in signal also produces a spectral broadening effect that can be observed at every filter.

(a)

(c)

(b)

(d)

Normalize value

1 0.8 0.6 0.4 0.2 0

600

700

800

900

1000

1100

(e)

900

1000

1100

(f)

Waveleghnt

Normalize value

1 0.8 0.6 0.4 0.2 0

600

700

800

Waveleghnt

Fig. 7. Reconstructed spectral images of 4 band-pass optical filters using the proposed algorithm, considering 5 neighboring spectral band, , tuned at a) 850, b) 900, c) 1000, and d) 1100 [nm]. The curves presented in (e) and (f) correspond to the spectral signature measured with the spectrometer and the estimated signature produced by the proposed algorithm.

As a final discussion, despite the improvements provided by the proposed algorithm in reconstructing spectral features and image details, the introduction of spatial-spectral regularization generates a higher computational complexity. The most time consuming part of the spatial-spectral NLM regularizer is the computation of the weights for every pixel of the hyperspectral datacube. In particular, the algorithm takes a burden of a2 × b2 × c computations per reconstructed pixel, where a2 denotes the size of the spatial similarity window and b2 × c the dimension of the spatial-spectral search cube. Nonetheless, the burden can be alleviated since the original design of the NLM algorithm is prone for parallel computation and several GPU-based implementations have been introduced lately to speed-up the process [31, 32], achieving reported gains of up to


85×. 4.

Conclusions

In this article, we presented a CS reconstruction algorithm that exploits the inherent spatialspectral redundancy that exists in hyperspectral datacubes when using a CHI systems based on the single pixel spectrometer system, originally proposed by Sun and Kelly [10]. The algorithm incorporates regularization terms based on the assumption that it is likely to observe redundant patches of spatial and spectral information contained in neighboring bands. The research conducted here has shown that the spatial-spectral modified NLM regularization can fulfill this task, exploiting the spatial structure of the scene during the reconstruction process, providing better fidelity from fewer measurements. This idea has been tested on spectral data heavily affected by electronic noise on each measured band. The experiments have demonstrated that the proposed algorithm is not only able to compensate for the noise, but also able to reconstruct images with fewer artifacts and blurring in contrast with traditional compressive imaging reconstruction. As future work, we will explore alternatives for automatic parameter estimation while automatically selecting the bands in the search cube, which may not be necessarily neighbor bands. Funding Fondo Nacional de Desarrollo Científico y Tecnológico (FONDECYT) (11140605, 11150476). Acknowledgments The authors are grateful to Prof. Sergio Torres, Universidad de Concepcion, for providing the equipment for acquiring the spectral data used for the simulations.