Comparing predictive abilities of three visible-near

M. Knadel et al., J. Near Infrared Spectrosc. 21, 67–80 (2013) Received: 19 April 2012 ■ Revised: 12 October 2012 ■ Accepted: 22 October 2012 ■ Publication: 21 January 2013

67

JOURNAL OF NEAR INFRARED SPECTROSCOPY

Comparing predictive abilities of three visible-near infrared spectrophotometers for soil organic carbon and clay determination Maria Knadel,a Bo Stenberg,b Fan Deng,a Anton Thomsena and Mogens Humlekrog Grevea a Department of Agroecology and Environment, Faculty of Agricultural Sciences, Aarhus University, Blichers Allé 20, PO Box 50, DK-8830 Tjele, Denmark. E-mail: [email protected] b

Swedish University of Agricultural Sciences, Department of Soil and Environment, SLU, PO Box 234, SE-532 23 Skara, Sweden

Due to advances in optical technology, a wide range of spectrometers is available. Recent interests in soil global libraries and sensor fusion presents a challenge with respect to combining data from different instrumentation. Little research, however, has been done on the comparison of visible-near infrared (vis-NIR) spectrometers for soil characterisation. There is a need for more work on the effects of scanning strategies and use of different soil instrumentation. We compared three vis-NIR spectrometers with varying resolution, signal-to-noise ratios and spectral range. Their performance was evaluated based on spectra collected from 194 Danish top soils and used to determine soil organic carbon (SOC) and clay content. Scanning procedures for the three spectrophotometers where done according to uniform laboratory protocols. Soil organic carbon and clay calibrations were performed using PLS regression. One third of the data set was used as an independent test set. A range of spectral preprocessing methods was applied in search of model improvement. Validation for SOC content using an independent data set derived from all three spectrophotometers provided values of RMSEP between 0.45% and 0.52%, r2 = 0.42–0.59 and RPD = 1.2–1.4. Clay content was predicted with a higher precision resulting in RMSEP values between 2.6% and 2.9%, r2 = 0.71–0.77 and RPD values in the range from 2.2 to 2.5. No substantial differences in the prediction accuracy were found for the three spectrometers, although there was a tendency that, in the tradeoff between noise and resolution, low noise was the more important for SOC and clay predictions. The application of different spectral preprocessing procedures did not generate important improvements of the calibration models either. Additionally, data simulation analysis, including resampling to a coarser resolution and addition of noise, was performed. No, or very little, effect of sampling resolution and additional noise on the performance of the spectrophotometers was reported. The results from this study showed that, as long as strict laboratory scanning protocols were followed, no significant differences in constituent determination were found, despite differences in spectral range, spectral resolution, spectral sampling intervals and sample presentation methods. The differences in predictive abilities between the spectrometers were mostly due to differences in spectral range. Keywords: vis-NIR spectroscopy, predictive ability, soil organic carbon (SOC), clay, spectrophotometer comparison

Introduction During the last ten years, near infrared (NIR) spectroscopy has been increasingly applied to soil analysis.1 As a result of a continuous development of sensing technologies, a wide ISSN: 0967-0335 doi: 10.1255/jnirs.1035

variety of spectrometers is available. The NIR instrumentation can be assembled with optical components employed for UV-visible instruments. Blanco and Villarroya2 and Pasquini3 © IM Publications LLP 2013 All rights reserved

68

presented reviews on NIR instrumentation. The main advantages of the NIR technique are speed of analysis, little or no sample preparation and ability to perform measurements in the field. In addition to laboratory equipment, small portable spectrophotometers suitable for in situ measurements are available. They include hand-held instruments or spectrometers that can be carried in a backpack. Most recently, mobile spectrometer platforms designed for on-the-go field survey have gained interest.4–8 Promising results reported for the regional, national and global libraries9–14 has led to the development of a universal and standardised soil library. The creation of a library has been put forward in order to increase the efficiency of vis-NIR characterisation.15,16 For the construction of global libraries especially, this could mean the amalgamation of sensory and analytical data obtained from a range of instruments and methods. The success of spectroscopic modelling of soil properties is, however, dependent on the accuracy and reliability of the reference methods.17 Additionally, sub-sampling error and the use of fundamentally different reference methods can present sources of error or noise to NIR calibrations.18 Thus, including the results from a variety of analytical methods used by different laboratories for correlation with spectroscopic data may not be successful or at all possible. The calibration may produce high validation errors, but, as long as the analytical methods are well correlated to each other the predictions may be as good as if using one single method. Moreover, the scanning protocols used by the individual laboratories and the employed instrumentation may vary substantially. As reported by Igne et al. 19 differences in technologies, internal instrument features, sample preparation and presentation and the number of replicates have a large impact on the final calibration models. Pimstein et al.20 divided the possible factors affecting reflectance measurement into spectrometer and sampling domains. For spectrometers, spectral configuration, detector performance, optical characteristics, surrounding factors, warm-up time and calibration quality were named, whereas, for sampling factors included, sample homogeneity, geometry of measurement, illumination setup, reference method, measurement conditions and the effects originating from the operator input were listed. Nevertheless, little work has been done on the effects of using different spectrophotometers and scanning methods on spectra quality and the precision of calibration models. Pimstein et al.20 compared spectrometers of the same model and vendor (FieldSpec, ASD Inc., Boulder, CO, USA) using three standard protocols and using three different materials for internal standards (sand, glass, grey reference) on only 12 soils selected from an Israeli soil library. They recommended using a strict scanning protocol and an internal standard for the improvement of spectral measurements and their stability by minimising uncontrolled spectral variations. Mouazen et al.21 compared four commercially available spectrophotometers for different agricultural materials, including soil moisture content. The differences between

SOC and Clay Determination Using Three Vis-NIR Spectrometers

these instruments were not only the wavelength range and measurement resolution, but also the measurement principles. Despite the differences in measurement principals of the spectrophotometers used (a 300–1700 nm diode array, a 350–2500 nm diode array/scanning monochromator and a 400–2500 nm Fourier transform and a scanning monochromator), very small differences for the estimation of soil moisture were found between them. Ge et al. 22 compared SOC calibrations with four spectrophotometers [AgriSpec (35–2500 nm), NIR Systems 6500 (400–2498 nm), LabSpec 5000 (350–2500 nm) and a FieldSpec Pro (350 nm–2500 nm)] on two sets of soil samples. Apart from comparing different instrumentation, different precisions of scanning procedures were tested. Significant differences were found for calibrations when no strict attention was given to the measurement procedures, whereas more consistency in soil spectra and SOC calibration models among these spectrophotometers was obtained by higher scanning control. Igne et al.19 focused on the evaluation of spectral pretreatments for different calibration methods of soil constituents including SOC and clay. Measurements were obtained on air-dry and fieldmoist soils using four different spectrophotometers including: portable Fourier transform IR (4000–400 cm−1), bench-Fourier transform-NIR, bench-Fourier transform-mid-IR and Veris (350–2225 nm). They reported no significant differences existing among pretreatments methods. While calibration transfer between different instruments has been successful22 still more research is needed to achieve comparability between instruments.11 Moreover, due to the number of spectrophotometers available, the selection of the suitable instrument for a specific application can be difficult.21 The choice of instrument is first of all applicationdependent but so also is the issue of cost. The question of multi-functionality and flexibility is also often addressed. Thus, a comparison of predictive abilities of different types of instruments would be useful. The aim of this study was to compare the predictive abilities of three commercially available vis-NIR spectrophotometers commonly used for soil spectroscopy. Two of the spectrophotometers were basically the same, high precision instruments covering the full vis-NIR range (350–2500 nm), but with somewhat different resolution and noise levels. Their sampling intervals were narrow and they were equipped with two different, but commonly used probes. The 3rd instrument had a shorter spectral range (350–2200 nm), intermediate resolution, but broader sampling intervals. Their abilities to predict SOC and clay in representative top soils from Denmark were tested.

Material and methods Soil samples One hundred and ninety-four topsoil samples (Figure 1) were selected from the Danish soil database. Soils from the national

M. Knadel et al., J. Near Infrared Spectrosc. 21, 67–80 (2013)

69

leguminous plants, fallow, cruciferae plants, Beta vulgaris and potatoes. Soil samples were dried and sieved down to 2 mm and were analysed for soil organic carbon (SOC) and clay. Particle size distribution was determined by sieving and hydrometric methods.25 SOC was determined by combustion using a LECO induction furnace (CN-2000 instrument, LECO Corporation, St Joseph, MI, USA). Both SOC and clay are reported in %.

Spectrophotometers

Figure 1. Location of the sampling points across Denmark.

database were collected during a regional soil profile investigation on a 7 km grid between 1987 and 1989.23The samples were chosen to cover a representative range of soil types across the entire country (East Jutland, Middle Jutland, West Jutland, North Jutland, Bornholm, Djursland, Himmerland, North Zealand and Thy) and were classified as Gleysol, Phaeozem, Cambisol, Luvisol, Podzol, Arenosol, Regosol, Fluvisol, Alisol, Histosol, Anthrosol.24 A range of land use systems were also represented: cereal field crops, pasture, coniferous forest,

Three different spectrophotometers were used for scanning the soil samples: ■ A bench top vis-NIR instrument—LabSpec 5100 manufactured by ASD Inc. (Boulder, CO, USA) ■ A portable vis-NIR instrument—FieldSpec Pro FR also manufactured by ASD Inc. (Boulder, CO, USA) ■ A mobile shank-based vis-NIR instrument in a bench top mode (Veris Technologies, KS, USA). Table 1 presents an instrument overview including the internal features of the three spectrometers. Spectral resolution and sampling intervals are listed. Spectral resolution, as defined by ASD Inc., is the full-width-half-maximum (FWHM) of the instrument response to a monochromatic source whereas, spectral sampling interval is the spacing between sample points in the spectrum. Even though both ASD Inc. instruments employ the same detectors they are configured differently. The FieldSpec is designed for use under solar illumination and its integrated fibre optic reduces signal loss because it is continuous from the detector to the tip, whereas, the optic in the LabSpec has two junctions. One is internal at the rear of the scrambler, and the other is an external connection. Approximately 5% of the signal is lost at every cable interface or junction. There is

Table 1. Instrumentation overview.

Spectrometer model Labspec 5100

Wavelength range 350–2500 nm

Spectral resolution

Sampling interval

3 nm @ 700 nm

1.377 nm @ 350–1050 nm

6 nm @ 1400/2100 nm

2 nm @ 1000–2500 nm

Optics

Detectors

Fibre optic

Fixed reflective grating (350– 1000 nm), moving gratings (1001–2500 nm), 512 element Si photodiode array @350–1000 nm, two TE cooled InGaAs photodiodes @1001–1830 nm and 1831–2500 nm

FieldSpec Pro FR

350–2500 nm

3 nm @ 700 nm 10 nm @ 1400/2100 nm

1.377 nm @ 350–1050 nm 2 nm @ 1000–2500 nm

Fibre optic

Fixed reflective grating (350– 1000 nm), moving gratings (1001–2500 nm), 512 element Si photodiode array @ 350–1000 nm, two TE cooled InGaAs photodiodes @1001–1830 nm and 1831–2500 nm

Veris

350–2200 nm

8 nm

6 nm@ 350–1000 nm 5 nm@ 1073–2200 nm

Fibre optic

Fixed reflective grating, 3648 element linear CCD array @350– 1050nm, 256 InGaAs linear image spectrometer @900–2200 nm

70

also a trade-off between resolution and signal strength. Thus, differences in noise levels can be expected.

Sample preparation and spectral measurements All spectral measurements were made under laboratory conditions on air dried, 2 mm sieved and homogenised samples. Controlled scanning environments, including an instrument specific check procedure, sample preparation and scanning procedure adequate for the individual instrument, were assured following the respective laboratory spectrophotometer protocols. To stabilise the instrument temperature, a minimum of half an hour warming was allowed before scanning sessions for all three instruments. As the initial purpose of these measurements was not the comparison of the three instruments, different scanning protocols were followed as described below.


spot size, a built-in DC current stabiliser circuitry and is equipped with a tungsten quartz halogen lamp with the same specification as for LabSpec. The fibre optic has a 25° full conical angle and is 1.5 m long, directly connected with the detectors. The RS3 Windows interface software from ASD Inc. was used for data acquisition. Three replicate spectra of each sample were taken and averaged, rotating the sample container after each measurement. A Labsphere Spectralon white reference was used at the beginning of each scanning session and after every fifth sample. The internal dark current was acquired automatically before each measurement of the white reference. The measurements of the soil samples, white reference and dark current were configured as an average of 30 readings per collected spectrum.

Veris LabSpec5100 The spectrophotometer covers the full visible and near infrared range between 350 nm and 2500 nm. It has the highest resolution of the three instruments in the NIR region (Table 1). The soils were measured using High Intensity Muglight, model-A122106, (ASD Inc.) equipped with a sapphire window using an ASDI sampling tray adapter with a quartz window (approx. 10 g of soil) having a 110 mm2 spot diameter (ASD Inc.). The probe features a built-in light source and acts as a workstation, so that samples can be placed on top of the probe. The source of light is a tungsten quartz halogen lamp (4 W−3.8 V) with built-in DC current stabiliser circuitry. The colour temperature of the halogen bulb is 2901°K ± 10°K. The LabSpec has a removable fibre optic (25° full conical angle) that requires the use of a laboratory style accessory. Two replicates of each sample were scanned. One spectrum from each replicate was collected by IndicoPro 6.0 spectrum acquisition software (ASD Inc., Boulder, CO, USA) and then averaged. A Labsphere Spectralon (www.labsphere.com/ products/reflectance-standards-and-targets/reflectancetargets/spectralon-targets.aspx) white reference was used at the beginning of each scanning session and after every fifth sample. The internal dark current was acquired automatically before each measurement of the white reference. The measurements of the soil samples, white reference and dark current were configured as an average of 50 readings per collected spectrum.

FieldSpec Pro FR FieldSpec Pro includes the same detectors as LabSpec 5100, but the configuration is different (Table 1) and the optic fibre is integrated without junctions. Measurements were made using the High Intensity Contact Probe (ASD Inc., Boulder, CO, USA) equipped with a sapphire window. It has a 100 mm2

The Veris spectrophotometer is part of a mobile sensor platform, designed mainly for on-the-go measurements in the field. In this study it was used in benchtop mode in the laboratory. It comprises two detectors for spectral data collection at a range between 350 nm and 2200 nm with 8 nm spectral resolution, which is intermediate to the two other instruments in the NIR region, but lower in the visible (Table 1). The spectrometers used were an Ocean Optics USB4000vis-NIR, [Ocean Optic Instruments Inc., Dunedin, FL, USA) charge-coupled device (CCD) array (350–1050 nm) and a Hamamatsu C9914GB TG-Cooled NIR II, (Hamamatsu, Shizuoka, Japan) InGaAs linear image spectrometer (900– 2200 nm). The instrument makes measurements through a sapphire window with a 314 mm2 spot size mounted on the bottom of the shank. As with the other two spectrometers, this instrument used a tungsten halogen bulb (5 V, 6.65 W, 2700°K) to illuminate the soil and a fibre optic of 1.5 metres. In order to acquire the dark current and the reference spectra the shutter would first close completely preventing any light coming into the fibre optic and then move a known reference material in front of the optic. The reference measurement is used to compensate for drift in the spectrometer and light source. Soil samples were packed in a Veris sample holder (approx. 1 g of soil) and placed against the face of the shank window for scanning. Samples were scanned only once, where the output spectrum was a result of an average of 100 scans. Data acquisition and processing programs were carried out using the Veris Spectrophotometer Software V1.2 (National Instruments, Austin, TX, USA).

Spectra comparison Spectral data used in the analysis covered the range between 450 nm and 2500 nm for the LabSpec and FieldSpec instruments and the range between 420 nm and 2155 nm for the Veris spectrophotometer. In order to ease the visual comparison of


absorption features, the spectra generated using the LabSpec and FielSpec were reduced by average to the same sampling interval as for the Veris spectrophotometer (every 6 nm at 350–1000 nm and every 5 nm at 1073–2200 nm).

Multivariate data analysis Before further data analysis the very noisy regions near the edges of the spectra were removed. Thus, the analysis of spectral data included ranges of 425–2500 nm, 452–2500 nm and 420–2158 nm for LabSpec, FieldSpec and Veris, respectively.

Principal component analysis (PCA) analysis Princial component analysis was performed on all three data sets separately after applying 1st derivative transform. The PCA was used to explore the data and how samples relate to each other within the spectral data spaces as collected by the three spectrophotometers. Principal component analysis is an analysis appropriate for revealing patterns and internal structure of the data. It can explain the relationships between samples and variables giving a general overview of the main information content in the data set allowing for the interpretation of sample groupings, similarities and differences. A principal component (PC) is a linear representation of variation in the data. Each PC explains a maximum amount of the remaining information contained in the data. Thus, the first PC contains the greatest amount of information from the data set and each subsequent PC contains less information than the previous one. All PC’s will also be orthogonal to each other.

Partial least-squares (PLS) regression Determinations of SOC and clay from spectral data were performed on apparent absorbance (A = log 1/R, where A = absorbance, R = reflectance) using PLS regression with a non-linear iterative partial least squares (NIPALS) algorithm26 in Unscrambler X 10.1 software (Camo ASA, Oslo, Norway). Partial least squares regressions on mean cantred data were developed using a calibration and a test set for an independent validation. In each case, samples were split into calibration (129 samples) and validation (65 samples) sets. Validation data sets for SOC and clay were generated by first sorting their values and second by selecting every third sample starting from number two. The remaining samples were used for calibration. This was done to assure full range coverage of both constituents in both the calibration and validation data sets. Full cross-validation was used to develop

71

the calibration models for both constituents with the three spectrophotometers to find the optimum number of components without over-fitting. Different spectral pre-treatments were tested in order to improve the calibration based on raw data: standard normal variate (SNV) with de-trending,27 a full multiplicative scatter correction (MSC)28 and transformations to the first and second derivative were generated. For the derivatives a Savitzky and Golay (SG)29 smoothing with a 2nd order polynomial over 15 wavelengths was applied; however, different options were tested here to choose the best one (results not shown). The best treatment for cross-validation was considered to be the one resulting in a model with the lowest root mean square error of cross-validation (RMSECV), the highest R2 for calibration (the raw R-square of the model), 2 the highest Rcv for the validation data set (adjusted R-square showing how good a fit can be expected for future predictions) and the highest RPD (ratio of standard error of cross-validation to standard deviation).17

Data simulation As shown in Table 1, the three spectrophotometers compared in this study differ in spectral range and spectral resolution (Table 1). In order to test the effects of sampling resolution and random noise on the performance of the spectrophotometers data simulation, including resampling to a coarser resolution and addition of noise, was performed.

Resampling The sampling resolution of the LabSpec spectrophotometer was adjusted to simulate the coarser resolution of FieldSpec (above 1000 nm) by performing a moving average transformation. In this transformation, data are smoothed according to a chosen size of segments which specify how many adjacent columns should be used to compute the average value. This way, for each point of the curve, a moving average is computed as the average over a segment encompassing the current point. The individual values are replaced by the corresponding moving averages. Resampling of LabSpec data were performed for the spectral range above 1000 nm only at three levels using three, five and seven points in each segment, respectively. In addition, the sampling resolution of the LasbSpec and FieldSpec spectrometers was re-sampled to simulate the coarser resolution of the Veris spectrometer. Again, a moving average transformation at three levels was used with five, seven and nine points in the spectral region between 350 nm and 1000 nm and with three, five and seven points above 1000 nm. The spectral range of the LabSpec and FieldSpec spectrophotometers was also adjusted to the Veris range (420–2158 nm). In the last step, smoothed spectra were reduced by six and five, below and above 1000 nm, respectively, to obtain the same number of variables as the Veris spectrophotometer.

72


(a)

(b)

Figure 2. Box-whisker plots: (a) for soil organic carbon and (b) for clay. The bottom and top of the box represent the 25th and 75th percentile, respectively. The band near the middle of the box is the median. The ends of the whiskers represent the 5th and the 95th percentile. The dot represents outliers.

Noise simulation

Spectra comparison

In order to compare signal-to-noise ratios from LabSpec and FieldSpec spectrophotometers, 30 spectra of a Labsphere Spectralon with a 10 scan average, were collected from both instruments. The level of noise for LabSpec and FieldSpec spectra was in the area of CV (coefficient of variation) equal to 0.5‰ and 0.3‰, respectively. The obtained spectra of the Labsphere Spectralon were converted to absorbance. Signal-to-noise ratio was calculated as standard deviation. The average standard deviation was higher from the LabSpec (0.00025) than from the FieldSpec spectra (0.00015). In order to increase the average value of the standard deviation of FieldSpec spectra 0.00015% of added noise was needed. To simulate the effects of random noise present in the spectral data from LabSpec spectrophotometer different levels of noise were added to the FieldSpec spectra. As a starting point, 0.00015% of noise was added. Furthermore, 0.0015% and 0.015% of noise were added to test when noise addition has a significant effect on spectral calibration. After resampling or noise addition the PLS models for SOC and clay were run again and compared with calibration results based on the original data sets.

Results and discussion Selected descriptive statistics of SOC and clay distributions are presented by box-whisker plots in Figure 2 and in Table 2.

Mean reflectance spectra of the three spectrophotometers were plotted (Figure 3) after applying the SNV transform. By and large, the shapes of the average spectra of the three instruments are similar. In particular, the spectra from the ASDI LabSpec and FieldSpec are similar, as could be expected given their technical similarities. Clearly visible differences are nevertheless present. The differences are visible especially at a higher resolution, with the mean spectra being more similar in the smaller scale features (Figure 3). The main characteristic of the reflectance absorption features occurring in the NIR region for the three spectra are located at 1400 nm and 1900 nm and are associated with water bands. These strong absorption bands are caused by overtones and combinations of the three vibration fundamentals of H2O. It is the overtones of O–H symmetric and aymmetric stretching that absorb near 1400 nm and the combinations of H–O–H bending and OH stretching that absorb near 1900 nm and are involved in clay mineral absorption features.30 In order to show small-scale differences in resolution and noise, spectra of the sample with the highest organic matter content (SOC = 3.9%) were plotted. Figure 4(a) shows a spectrum of the same sample, in the region holding information on soil organic matter (1695 nm–1800 nm), scanned by all instruments. In accordance with what could be expected from the lower resolution and the unbroken fibre-optic connection of the FieldSpec, this spectrum is the smoothest of the three. The spectrum obtained by the LabSpec model, which has the

Table 2. General statistics of soil organic carbon and clay (%) in the calibration and test sets.

Constituent SOC clay SD is standard deviation; SOC is soil organic carbon.

Data set

Minimum

Maximum

Mean

Median

SD

0.64

3.89

1.7

1.51

0.68

Test

0.75

3.82

1.72

1.51

0.70

Calibration

2.5

51.7

9.53

8.2

6.39

Test

2.5

24.9

9.43

8.3

5.49

Calibration


Figure 3. Standard normal variate (SNV) transformed mean reflectance spectrum obtained with three instruments (LabSpec, FieldSpec and the Veris).

highest resolution and a fibre with two junctions, shows the highest degree of small-scale noise. A comparison of LabSpec and FieldSpec spectra between 2020–2500 nm are shown in Figure 4(b). This region carries information on mineral and organic matter content. The same large-scale absorption features can be recognised for both spectra. The spectrum obtained using the LabSpec instrument was again evidently more noisy than the spectra from the FieldSpec instrument.

Principal component analysis In the PCA analyses, similar patterns representing all three spectrophotometers were apparent for the first three PCs (Figure 5). The first three components explained 95%, 91%

(a)

73

and 95% of the total variation for LabSpec, FieldSpec and Veris spectrophotometer, respectively. In order to ease the general overview of the main information content in the data sets, PC scores were grouped into three classes according to clay content. Despite some differences among the mean spectra obtained from the three spectrophotometers, as discussed above, the patterns within the score plots were nearly the same, indicating that basically the same information was obtained using all three spectrophotometers. When comparing PC scores from LabSpec and FieldSpec data in particular, the main patterns within the samples were almost identical. Due to the nature of PCA, with each component as a linear representation of the maximum un-explained variation, the first three components will focus on larger scale variation.26 This means that more subtle, but important differences may still occur. The clay groupings for the PC1 vs PC2 plots indicate that PC1 contained substantial information related to clay content. Similar clear patterns for SOC could not be found (not shown). Few potential outliers were revealed. These were, however, samples with the most extreme clay and SOC content. As far as general spectral data structures are concerned, no important difference between the spectrophotometers were observed.

Partial least squares regression Cross-validation results for SOC and clay are shown in Tables 3 and 4, respectively. The independent validation results from the best models, based on cross-validation results, are shown in Figure 6. As prediction results were not substantially worse when compared with the ASDI instruments, the potential draw back of only one replicate with the Veris was apparently compensated for by the larger viewed area used with this instrument and the coarser spectral sampling interval was not detrimental either.

(b)

Figure 4. (a) Spectrum of a sample with the highest soil organic carbon content (3.9%) in the range between 1695 nm and1800 nm generated by Veris, LabSpec and FieldSpec spectrophotometers. (b) Spectrum of a sample with the highest soil organic carbon content (3.9%) in the range between 2200 nm and 2500 nm generated from LabSpec and FieldSpec spectrophotometer.

74


Figure 5. Score plots for the first three principal components in a principal component analysis (PCA) of spectral data from three instruments (for LabSpec, FieldSpec and Veris).

The predictive ability for SOC was low from all three spectrophotometers. Lower predictive abilities were recorded for the independent validation in comparison with crossvalidation results for both ASDI instruments. In relation to the standard deviation of the data set, the RMSEP was about 50% higher than expected from a compilation of previously published data. 31 The final statistics for the independent validation show the highest predictive ability for SOC from the FieldSpec data using 1st derivative spectra. Very similar results were generated from the Veris spectrometer. Models derived from the LabSpec spectra however, performed with slightly lower precision. Clay content was determined with a higher precision than SOC and with somewhat lower RMSEP than expected from previously published results.31 The independent validation showed lower predictive abilities for LabSpec and Veris than the results from cross-validation. Higher predictive ability was once again recorded for the validation results based on data obtained with the FieldSpec instrument. It was the MSC

transformed spectra that produced the best model for clay determination from this instrument. The results of the independent validation for the LabSpec and Veris were almost identical. The best models were generated with MSC and SNV de-trending transformed spectra for LabSpec and Veris, respectively. When using the prediction statistics of the two soil constituents to evaluate and compare the spectrometers’ performance, it was the FieldSpec instrument that delivered the best prediction results for SOC and clay. The differences were very small, so it is difficult to conclude if they are due to chance or to slightly more robust calibrations due to less noise in the FieldSpec spectra. In any case, our data do not support the hypothesis that a higher spectral resolution, as with the LabSpec instrument, brings additional spectral information about SOC and clay content. Despite the lower spectral range and broader sampling intervals of the Veris measurements, prediction results for SOC content were similar to the ASD Inc. instruments.


Table 3. Cross-validation (on 129 samples) results for soil organic carbon for the three spectrometers.

LabSpec

FieldSpec

75

Table 4. Cross-validation results for clay for the three spectrometers.

Veris

LabSpec

Cross-validation log 1/R

1st SG

2nd SG

MSC

SNV

RMSECV

0.42

FieldSpec

Veris

Cross-validation 0.46

0.49

log 1/R

RMSECV

2 RCV

0.62

0.56

0.49

R

RPD

1.6

1.5

1.4

RPD

Factors

10

6

7

RMSECV

0.44

0.43

0.50

1st SG

2 CV

7

9 2.5

0.60

0.47

R

1.4

RPD

Factors

8

9

4

RMSECV

0.47

0.47

0.45

2 CV

3

4

7

2.7

2.5

0.53

0.53

0.53

R

1.4

1.4

RPD

7 MSC

0.85 2.5

2.5

1.4

0.51

0.83 2.4

RMSECV

RPD

3

0.85 2.5

Factors

2 RCV

0.48

0.84 2.5

2.7

1.6

3

0.81 2.3

7

0.59

0.44

0.87 2.8 2.5

1.5

RMSECV

2.5

RMSECV

RPD

Factors

2.8

Factors

2 RCV

2nd SG

2.3

2 CV

0.85

0.82

0.85

2.5

2.4

2.5

Factors

1

1

5

RMSECV

2.3

2.3

2.5

2 RCV

0.58

0.50

0.44

R

0.87

0.87

0.85

RPD

1.5

1.4

1.3

RPD

2.8

2.8

2.5

Factors

8

5

5

Factors

4

5

4

RMSECV

0.50

0.50

0.50

RMSECV

2.4

2.8

2.5

2 RCV

0.47

0.46

0.46

R

0.85

0.81

0.84

RPD

1.3

1.4

1.4

RPD

2.7

2.3

2.5

Factors

4

4

6

Factors

3

4

3

log 1/R is absorbance where R is reflectance; 1st SG is the 1st Savitzky–Golay derivative; 2nd SG is the 2nd Savitzky–Golay derivative MSC is multiplicative scatter correction; SNV is standard normal variate; RMSECV is a root mean square error of cross-validation RPD is ratio of standard error of cross-validation to standard deviation; Results highlighted in bold are the results from the best calibrations.

That is probably explained by the fact that organic matter is spectrally active through the entire vis-NIR spectrum. The overtones and combination bands here are the result of stretching and bending of NH, CH and CO groups. The most important bands for soil organic matter are located around 1700–1800 nm, 2050 nm and 2200–2400 nm, but bands near 1100–1200 nm and 1600 nm can also be of significance.31,32 The regression coefficients for SOC models (on SNV transformed spectra) are shown in Figure 7(a). The general features were similar for all three spectrophotometers. More noise, however, was introduced in the SOC model derived from Veris measuremtns as a result of the higher number of factors used in this calibration (six factors instead of four with the other two instruments). The important bands for SOC calibration from the three models are located at wavebands around 600 nm, 800 nm, 1400 nm, 1700–1800 nm, 2070 nm, 1900 nm and additionally around 2200-2300 nm for ASDI instruments [Figure 7(a)].

SNV

2 CV

2 CV

For abbreviations refer to Table 3.

The major mineral diagnostic regions for clay minerals are located between; 1300–1400 m, 1800–1900 nm and 2200–2500 nm and are holding information of the major clay minerals in Denmark such as smectite, kaolin or illite. The spectral features of clay minerals are due to overtones and combination vibtrations modes of OH functional groups as lattice water or as part of absorbed water.30 However, the lack of a 2200– 2500 nm region in the analysis did not cause a decrease in the performance of the Veris clay calibrations. It seems as if the interactions between water and hydroxyls and clay minerals near 1400 nm and 1900 nm33 were able to capture the variation in clay content sufficiently on their own. Regression coefficients from the three clay models present distinct wavebands at both 1400 nm and 1900 nm [Figure 7(b)]. Additional important wavebands obvious for ASDI instruments are located between 2200 nm and 2300 nm. The results of others reported in the literature on the effects of using different spectrophotometers and scanning methods on spectral quality and the precision of calibration models are similar to ours. In comparison, no major differences for the estimation of soil moisture content among four spectrophotometers were found by Mouazen et al.19 Not only had the wavelength range and measurement resolution of the used instruments differed, as in case of our study, but also the measurement principles.

76


Figure 6. Independent validation results of soil organic carbon and clay for the LabSpec, FieldSpec and Veris spectrometers.


77

(a)

(b)

Figure 7. Regression coefficients from PLS regression (for Standard Normal Variate transformed data) for soil organic carbon (a) and clay (b). The number of factors used in PLS for soil organic carbon determination: LabSpec—4, FieldSpec—4 and Veris—6. Factors used in PLS for clay determination: LabSpec—4, FieldSpec—4 and Veris—3.

Yet the final results seemed not be affected significantly. In the study by Ge et al.,22 SOC calibrations using two and three different vis-NIR spectrophotometers, with and without scanning control, respectively, were presented. The approach of using two spectrometers with a higher scanning control is more relevant to the methodology presented in our study, and showed statistically very similar SOC calibration models regardless of spectrometer type. Similar to our findings, no major differences among various spectral pretreatments were found in the literature. In the

work by Igne et al.19 no significantly different results for SOC and clay calibrations were obtained, among as many as 18 spectral pretreatments, for data from four vis-NIR and MIR spectrophotometers. Vasques et al.34 also reported that only a small gain in predictive accuracy was achieved from using different spectral pretreatments for SOC modelling using a PLS regression method for 554 soils in the Santa Fe River Watershed in Florida. They found that preprocessing transformations were more effective for other parametric multivariate techniques.

Table 5. The best cross-validation results for soil organic carbon from the LabSpec and FieldSpec original spectral data covering the spectral range from 425 nm to 2500 nm and after range adjustments from 420 nm to 2158 nm to match the Veris spectral range.

LabSpec (425–2500 nm)

FieldSpec (452–2500 nm)

Veris (420–2158 nm)

Labspec (420–2158 nm)


RMSECV

0.42

0.43

0.44

0.47

0.47

2 RCV

0.62

0.60

0.58

0.53

0.53

RPD

1.6

1.6

1.4

1.4

1.4

Factors

10

9

7

5

7


Table 6. The best cross-validation results for clay from the LabSpec and FieldSpec original spectral data covering the spectral range from 425 nm to 2500 nm and after range adjustments from 420 nm to 2158 nm to match the Veris spectral range.

LabSpec (425–2500 nm)


Veris (420–2158 nm)



RMSECV

2.3

2.3

2.5

2.3

2.6

2 RCV

0.87

0.87

0.84

0.87

0.84

RPD

2.8

2.8

2.5

2.8

2.5

Factors

4

5

3

5

5


78


Table 7. Results from the independent validation from from the LabSpec and FieldSpec original spectral data covering the spectral range from 425 nm to 2500 nm and after range adjustments from 420 nm to 2158 nm to match the Veris spectral range.

LabSpec (425–2500 nm) SOC

Clay


Veris (420–2158 nm)



RMSEP

0.52

0.45

0.55

0.58

0.58

r2

0.49

0.59

0.42

0.35

0.36

RPD

1.3

1.4

1.2

1.2

1.2

RMSEP

2.9

2.6

2.9

3.1

2.9

0.71

0.77

0.70

0.68

0.71

2.2

2.5

2.2

2.1

2.2

r

2

RPD

RMSEP is a room mean square error of prediction; RPD is ratio of standard error of prediction to standard deviation; SOC is soil organic carbon.

Data simulation Resampling No effect of resampling LabSpec spectra to simulate FieldSpec resolution was found regardless of the size of a smoothing segment in moving average transformation. Calibration and validation models showed exactly the same precision as models based on original data. Similarly, no effect was recorded when adjusting LabSpec and FieldSpec resolution to Veris internal settings. As resampling LabSpec spectra to FieldSpec resolution did not degrade calibration models, additional data simulation analysis, including resampling to the point when it does make a difference, was tested. Still, using even as many as 100 segments in moving average transformation did not affect the calibration and validation of SOC and clay. A degraded effect on calibration models was first visible after narrowing the spectral range down to 420–2158 nm. Tables 5 and 6 contain cross-validation results from the best models for SOC and clay calibrations, respectively, generated on the original data and after narrowing spectral range to the Veris range. All calibration models from LabSpec and Fieldspec data performed worse after adjustments of the range. The results from SOC models after narrowing the spectral range were nearly identical for both LabSpec and FieldSpec, being of a slightly lower predictive ability than from Veris data (Table 5). The most noticeable difference was that the optimum number of components was reduced to the level of Veris when narrowing the spectral range. Slightly better results from clay calibration were obtained on LabSpec than FieldSpec data (Table 6). The best cross-validation result for clay generated from the FieldSpec with a shorter range performed nearly identically to the Veris. Results from the independent validation for both SOC and clay (Table 7) confirmed degraded effects of shorter spectral range on the predictive ability of the ASD instruments and with the shorter range differences between all three instruments were very small and random.

Noise simulation The addition of noise to FieldSpec spectral data at different levels (0.00015%, 0.0015% and 0.015%) to simulate LabSpec signal-to-noise ratio, did not affect the PLS calibrations of SOC and clay. No significant difference in the calibration/validation statistics (recorded differences were at the level of the fourth digit), nor differences in the number of factors were found after noise addition. Therefore, these results are not shown. A higher level of noise present in LabSpec spectral data were removed with the application of spectral pretreatments and thus, had no effect on the calibration results.

Conclusions Laboratory measurements on soil samples under controlled laboratory conditions using three commercially available spectrophotometers were compared. No substantial differences between predicted SOC and clay content were reported. Those minor differences in calibration results are difficult to explain but might be related to different scanning methods or sample presentation. Calibration models using the Veris instrument, which offers a reduced spectral range, were surprisingly similar to the models obtained by the two ASDI spectrophotometers. Instruments with a higher resolution or sample interval did not perform substantially better for SOC and clay predictions than instruments with a lower resolution. The results from data simulation indicate that differences among the predictive abilities of the spectrophotometers were random or might be dependent on the sample presentation and not affected by noise level or sampling resolution. The only factor that had some effect was spectral range. We conclude that, for the measurements performed in controlled environments, carefully following the laboratory protocols, neither the spectrophotometer type nor the scanning procedure significantly affected the predictive abilities of the final models.


In future work calibration transfer between the instruments will be investigated.

Acknowledgement Financial support for this work came from the HOBE—research centre for hydrology and the SINKS project.

References 1. V. Bellon-Maurel and A. McBratney, “Near-infrared

(NIR) and mid-infrared (MIR) spectroscopic techniques for assessing the amount of carbon stock in soils—critical review and research perspectives”, Soil Biol. Biochem. 43, 1398 (2011). doi: 10.1016/j.soilbio.2011.02.019 2. M. Blanco and I. Villarroya, “NIR spectroscopy: A rapidresponse analytical tool”, Trends Anal. Chem. 21, 240 (2002). doi: 10.1016/S0165-9936(02)00404-1 3. C. Pasquini, “Near infrared spectroscopy: fundamentals, practical aspects and analytical applications”, J. Braz. Chem. Soc. 14, 198 (2003). doi: 10.1590/S010350532003000200006 4. M. Knadel, A. Thomsen and M.H. Greve, “Multisensor on-the-go mapping of soil organic carbon content”, Soils Sci. Soc. Am. J. 75, 1799 (2011). doi: 10.2136/ sssaj2010.0452 5. X. Huang, W. Senthilkurnar, S. Kravchenko, A. Thelen and J.G. Qi, “Total carbon mapping in glacial till soils using near-infrared spectroscopy, landsat imagery and topographical information”, Geoderma 141, 34 (2007). doi: 10.1016/j.geoderma.2007.04.023 6. C.D. Christy, “Real-time measurement of soil attributes using on-the-go near infrared reflectance spectroscopy”, Comput. Electron. Agric. 61, 10 (2008). doi: 10.1016/j.compag.2007.02.010 7. A.M. Mouazen, M.R. Maleki, J. De Baerdemaeker and H. Ramon, “On-line measurement of some selected soil properties using a VIS-NIR sensor”, Soil Tillage Res. 93, 13 (2007). doi: 10.1016/j.still.2006.03.009 8. R.S. Bricklemyer and D.J. Brown, “On-the-go VisNIR: potential and limitations for mapping soil clay and organic carbon”, Comput. Electron. Agric. 70, 209 (2010). doi: 10.1016/j.compag.2009.10.006 9. K.D. Shepherd and M.G. Walsh, “Development of reflectance spectral libraries for characterization of soil properties”, Soils Sci. Soc. Am. J. 66, 988 (2002). doi: 10.2136/sssaj2002.0988 10. D.J. Brown, K.D. Shepherd, M.G. Walsh, M.D. Mays and T.G. Reinsch, “Global soil characterization with VNIR diffuse reflectance spectroscopy”, Geoderma 132, 273 (2006). doi: 10.1016/j.geoderma.2005.04.025

79

11. L. Cécillon, B.G. Barthes, C. Gomez, D. Ertlen,

V. Genot, M. Hedde, A. Stevens and J.J. Brun, “Assessment and monitoring of soil quality using near-infrared refl ectance spectroscopy (NIRS)”, Eur. J. Soil Sci. 60, 770 (2009). doi: 10.1111/j.13652389.2009.01178.x 12. H. Bellinaso, J.A.M. Dematte and S.A. Romeiro, “Soil Spectral Library and its use in soil classification”, Rev. Bras. Cienc. Solo 34, 861 (2010). doi: 10.1590/S010006832010000300027 13. V. Genot, G. Colinet, L. Bock, D. Vanvyve, Y. Reusen and P. Dardenne, “Near infrared reflectance spectroscopy for estimating soil characteristics valuable in the diagnosis of soil fertility”, J. Near Infrared Spectrosc. 19, 117 (2011). doi: 10.1255/jnirs.923 14 M. Knadel, F. Deng, A. Thomse and M. Greve, “Development of a Danish national Vis-NIR soil spectral library for soil organic carbon determination”, in Digital Soil Assessments and Beyond. CRC Press, pp. 403–408 (2012). doi: 10.1201/b12728-79 15. R.A.A. Viscarra Rossel, “Working group on proximal soil sensing (WG-PSS)”, Pedometron. 25, 27 (2008). 16. J.B. Sankey, D.J. Brown, M.L. Bernard and R.L. Lawrence, “Comparing local vs. global visible and nearinfrared (VisNIR) diffuse reflectance spectroscopy (DRS) calibrations for the prediction of soil clay, organic C and inorganic C”, Geoderma 148, 149 (2008). doi: 10.1016/j. geoderma.2008.09.019 17. P.C. Williams and K.H. Norris, “Implementation of near-infrared technology”, in Near-Infrared Technology in the Agricultural and Food Industry, 2nd Edn, Ed by P.C. Williams and K. Norris. American Association of Cereal Chemists, St Paul, MN, USA, p. 145 (2001). 18. D.F. Malley, P.D. Martin and E. Ben-Dor, “Application in soil analysis”, in Near Infrared Spectroscopy in Agriculture. Ed by A. Roberts, J. Workman, Jr and J.B. Reeves, III. Agronomy 44, Am. Soc. of .Agron Inc., Madison, Wisconsin, USA, pp. 729–784 (2004). 19. B. Igne, J.B. Reeves, G. McCarty, W.D. Hively, E. Lund and C.R. Hurburgh, “Evaluation of spectral pretreatments, partial least squares, least squares support vector machines and locally weighted regression for quantitative spectroscopic analysis of soils”, J. Near Infrared Spectrosc. 18, 167 (2010). doi: 10.1255/jnirs.883 20. A. Pimstein, G. Notesco and E. Ben-Dor, “Performance of three identical spectrometers in retrieving soil reflectance under laboratory conditions”, Soils Sci. Soc. Am. J. 75, 746 (2011). doi: 10.2136/sssaj2010.0174 21. A.M. Mouazen, W. Saeys, J. Xing, J. De Baerdemaeker and H. Ramon, “Near infrared spectroscopy for agricultural materials: an instrument comparison”, J. Near Infrared Spectrosc. 13, 87 (2005). doi: 10.1255/ jnirs.461 22. Y. Ge, C.L.S. Morgan, S. Grunwald, D.J. Brown and D.V. Sarkhot, “Comparison of soil reflectance spectra and calibration models obtained using multiple spectrom-

80

eters”, Geoderma 161, 202 (2011). doi: 10.1016/j.geoderma.2010.12.020 23. H.B. Madsen, A.H. Nørr and K.A. Holst, “The Danish soil classification”, in Atlas of Denmark I, 3. Reitzel, Copenhagen, Denmark (1992). 24. L. Krogh and M.H. Greve, “Evaluation of World reference base for soil resources and FAO soil map of the World using nationwide grid soil data from Denmark”, Soil Use Manag. 15, 157 (1999). 25. G.W. Gee and J.W. Bauder, “Physical and chemical methods”, in Methods of Soil Analysis, Part 1, Ed by Klute, Soil Sci. Soc. Am. and Am. Soc. Agron., Madison, WI, USA, pp. 383–412 (1986). 26. H. Martens and T. Næs, Multivariate Calibration. John Wiley & Sons Ltd, Chichester, UK (1989). 27. R.J. Barnes, M.S. Dhanoa and S.J. Lister, “Standard normal variate transformation and de-trending of nearinfrared diffuse reflectance spectra”, Appl. Spectrosc. 43, 772 (1989). doi: 10.1366/0003702894202201 28. H. Martens, S.A. Jensens and P. Geladi, “Multivariate linearity transformations for near infrared reflectance spectroscopy”, in Proc. Nordic Symp. Applied Statistics, Ed by Stokkland, Forlag, Stavanger, Norway, p. 205 (1983). 29. A. Savitzky and M. Golay, “Smoothing and differentiation of data by simplified least squares procedures”, Anal. Chem. 36, 1627 (1964). doi: 10.1021/ac60214a047


30. R.N. Clark, “Spectroscopy of rocks and minerals and

principles of spectroscopy”, in Remote Sensing for the Earth Sciences. Manual of Remote Sensing. Ed by A.N. Rencz. John Wiley & Sons Ltd, Chichester, UK, pp. 3–58 (1999). 31. B. Stenberg, R.A. Viscarra Rossel, A.M. Mouazen and J. Wetterlind, “Visible and near infrared spectroscopy in soil science”, Adv. Agron. 107, 163 (2010). doi: 10.1016/ S0065-2113(10)07005-7 32. B. Stenberg, “Effects of soil sample pretreatments and standardised rewetting as interacted with sand classes on vis-NIR predictions of clay and soil organic carbon”, Geoderma 158, 15 (2010). doi: 10.1016/j.geoderma.2010.04.008 33. R.N. Clark, T.V.V. King, M. Klejwa and A. Swayze, “High spectral resolution reflectance spectrosocpy of minerals”, J. Geophys. Res. 95, 12653 (1990). doi: 10.1029/ JB095iB08p12653 34. G.M. Vasques, S. Grunwald and J.O. Sickman, “Comparison of multivariate methods for inferential modeling of soil carbon using visible/near-infrared spectra”, Geoderma 146, 14 (2008). doi: 10.1016/j.geoderma.2008.04.007