Evaluation of the Transferability of a Knowledge-Based ... - Springer Link

Chapter 14

Evaluation of the Transferability of a Knowledge-Based Soil-Landscape Model J. McKay, S. Grunwald, X. Shi, and R.F. Long

Abstract Knowledge-based digital soil mapping has been used extensively to predict soil taxonomic and physico-chemical soil characteristics. Fuzzy logic knowledge-based models allow explicit integration of knowledge and expertise from soil mappers familiar with a region. Questions remain about the transferability of soil-landscape models developed in one region to other regions. Objectives of this study were to develop and evaluate a knowledge-based model to predict soil series and fuzzy drainage classes and assess its transferability potential between similar soil landscapes in Essex County, Vermont. Two study areas, study area W1, 3.5 km2 in size and study area W2, 1.9 km2 in size, were sampled at 128 and 42 sites, respectively. Both study areas are located in Essex County, Vermont. Rule-based fuzzy inference was used based on fuzzy membership functions characterizing soilenvironment relationships to create a model derived from expert knowledge. The model was implemented using the Soil Inference Engine (SIE), which provides tools and a user-friendly interface for soil scientists to prepare environmental data, define soil-environment models, run soil inference, and compile final map products. Defuzzified raster predictions were compared to field mapped soil series and fuzzy drainage class properties to assess their accuracy. In W1 the model was 73.7 and 88.8% accurate, respectively, in predicting soil series and fuzzy drainage classes using an independent validation set. In W2, similar results were achieved, with 71.4 and 89.9% accuracies, respectively. It was shown that the prediction model was transferable to a landscape with similar soil characteristics. For future soil prediction applications it is critical to identify constraints and thresholds that limit transferability of prediction models such as SIE to other soil-landscapes. Keywords Digital soil mapping · Transferability · Soil inference engine · Vermont · Knowledge-based J. McKay (B) USDA Natural Resources Conservation Service, 481 Summer Street, Suite 202, St. Johnsbury, VT 05819, USA e-mail: [email protected]

J.L. Boettinger et al. (eds.), Digital Soil Mapping, Progress in Soil Science 2, C Springer Science+Business Media B.V. 2010 DOI 10.1007/978-90-481-8863-5_14,

165

166

J. McKay et al.

14.1 Introduction For years, soil scientists have been working to build quantitative predictive models to a large extent based on the five factors of soil formation as described by Jenny (1941): S = f (Cl, O, R, P, T)

(14.1)

where soil is a function of climate, organisms, relief, parent material, and time. McBratney et al. (2003) pointed out that soils can be predicted from their properties in combination with CLORPT factors, where soil properties can be derived from remote or proximal sensing or from expert knowledge. Knowledge-based models are composed of three main elements: environmental data, a knowledge base, and an inference engine which combines the data and the knowledge base to infer logically valid conclusions about the soil (Skidmore et al., 1996). See Section 26.2 for a discussion on different types of expert knowledge. The Soil Inference Engine (SIE) is an expert knowledge-based inference engine designed for creating soil maps under fuzzy logic. There are two main types of knowledge that SIE uses: rules, which are defined in parametrical space; and cases, which are defined in geographical space. Both rule-based reasoning (RBR) and casebased reasoning (CBR) can be used to perform inference. SIE also provides tools for result validation, terrain analysis, pre- and post-processing of raster data, and data format conversion (Shi et al., 2009). Predictive models are often based on the catena concept (Milne, 1935), which indicates that soil profiles occurring on topographically associated landscapes will be repeated on similar landscapes. One major question that remains in the field of soil landscape modeling is that of model transferability. In particular, knowledge-based empirical models are constrained by soil geographic space boundaries, whereas mechanistic pedogenic simulation models do not face this constraint. Boundary conditions that describe the attribute space used to develop specific soil prediction models are often not well defined in digital soil mapping studies (Grunwald, 2009). Local or site-specific predictions that are implemented within limited geographic extent are documented extensively in the literature (Baxter and Oliver, 2005; McKenzie and Ryan, 1999; Mitra et al., 1998) but their transferability to other regions is unclear. Predictive capabilities are limited, especially over large areas, because the relationships between soil properties and landscapes are either nonlinear or unknown (Lagacherie and Voltz, 2000). Prediction of soil properties becomes even more difficult when factors other than topography begin to play more of a role, such as different parent materials or changes in climate (Thompson et al., 2006). However, Lagacherie et al. (2001), in studying the applicability of detailed soil surveys to larger areas, found some factors to be especially important, including the expert delineation of reference areas and the necessity of those reference areas of being highly representative of the prediction zone. Keeping these factors in mind, it is

14 Evaluation of the Transferability of a Knowledge-Based Soil-Landscape Model

167

reasonable to expect that expert soil scientists can achieve high model transferability. Section 24.6 contains a good discussion on the inherent bias in expert driven methods. This study aims to take a soil prediction model developed for a relatively small study area in a complex landscape and test how well it transfers to another, similar study area a few kilometers away. The specific objectives are to populate a knowledge-based soil model to predict soil series (classified according to U.S. Soil Taxonomy (Soil Survey Staff, 2006)) and fuzzy drainage classes in one soil region in Vermont and validate model predictions in a nearby soil region.

14.2 Materials and Methods 14.2.1 Study Area Two study areas were used to empirically evaluate the transferability of soil scientists’ knowledge; specifically, the extent to which a soil-landscape model built by soil scientists for one area is applicable to a second, “similar” area. Both study areas W1 and W2 are in Essex County, Vermont and a comparison of their biophysical characteristics is given in Table 14.1. Study areas W1 and W2 are assumed to be dominated by one catena of soils, and the model reflects this assumption. Three soil series that dominate these areas are Cabot, Colonel, and Dixfield (Table 14.2). Dixfield soils are found

Table 14.1 Study area comparison Study areas U.S. geological survey (USGS) Quad Size (km2 ) Elevation (m) Geology Vegetation Topography Slope (%) Mean annual temperature Mean annual precipitation Land use Soils (general knowledge)

W1

W2

Averill lake

Bloomfield

3.5 Min: 468 Max: 833 Mean: 664 Std. Dev.: 51.9 Phyllite and schist (Gile mountain formation) Mixed northern-hardwood and spruce-fir forests Hills and narrow valleys Min: 0.02 Max: 86.08 Mean: 15.42 Std. Dev.: 12.02 6◦ C 97 cm Long term timber management Deep, loamy basal till; some very poorly drained organic materials in depressions

1.9 Min: 375 Max: 618 Mean: 475 Std. Dev.: 49.67 Phyllite and schist (Gile mountain formation) Mixed northern-hardwood and spruce-fir forests Hills and narrow valleys Min: 0.10 Max: 54.82 Mean: 12.93 Std. Dev.: 7.38 6◦ C 97 cm Long term timber management Deep, loamy basal till; some very poorly drained organic materials in depressions

168

J. McKay et al.

Series name

Table 14.2 Soil series modeled in W1 and W2 Drainage class Taxonomic class according to U.S. soil taxonomy

Cabot

Poorly

Colonel Dixfield

Somewhat poorly Moderately well

Loamy, mixed, active, nonacid, frigid, shallow Typic Humaquepts Loamy, isotic, frigid, shallow Aquic Haplorthods Coarse-loamy, isotic, frigid Aquic Haplorthods

highest on the landscape, on the steepest, most convex slopes, and Cabot soils are found lowest on the landscape, on the flattest, most concave slopes. Colonel soils occur between Cabot and Dixfield both in terms of hillslope position and slope shape. Soils that occur to a lesser extent on the landscape were designated based on which of the three dominant series they most closely resembled morphologically.

14.2.2 Digital and Field Data Two factors that were proven to provide a good basis for rules based on expert knowledge were slope and compound topographic wetness index. Other layers, such as vegetation, landform, and relative position were investigated and ultimately not used in this study. Investigation of potential data layers involved a visual assessment and comparison to known soil locations by expert soil scientists. Landform and relative position were too general, because the soils being investigated occur on multiple landforms and positions on the landscape. Vegetation showed limited correlation with soil series, and land cover was relatively homogenous, thus was not considered a promising predictor. On the other hand, wetness index is a powerful data layer that gives information not only on flow accumulation, but also indirectly on curvatures and, in addition, serves as a large scale relative position map. Both of the layers used were derived from a 5 m digital elevation model (DEM), aggregated from 1 m Light Detection and Ranging (LiDAR) data. The terrain attributes (slope and wetness index) were created using SIE. The neighborhood size for slope was 30 m, with a square neighborhood shape. The lag in calculating slope for wetness index (known as multi-path wetness in SIE) was 1 pixel. Field sampling in W1 consisted of 128 points randomly chosen from a previously laid out grid design. Seventy percent (90 points) were used to aid model development and 30% (38 points) were used for model validation within W1. In order to validate the model in W2, a sampling design similar to random catena sampling (McBratney et al., 2006) was used. Six randomly placed sampling sites along seven catenas were selected, for a total of 42 sampling points. At each of the selected sampling locations in both W1 and W2, a soil pit was exposed and taxonomic soil descriptions were derived to the soil series level. Drainage classes were identified at each site, as represented by depth to redoximorphic features.


169

14.2.3 Inference Models For model development, the soil scientists provided global knowledge, which covers the entire mapping area (Shi et al., 2009), prepared data layers and rules, and ran SIE. The initial round of output maps was verified by the soil scientist and the rules were adjusted to fit known sample points. This verification and rule adjustment process was repeated multiple times. After the last round of output maps was verified, post-processing tools from SIE and other geographic information system (GIS) tools (specifically ArcGIS spatial analyst (ESRI, 2010)) were used to integrate the results and generate hardened (defuzzified) maps. The hardened maps are created by aggregating all three of the fuzzy membership maps for each study area using SIE to assign, at each pixel, the soil series with the highest fuzzy membership. Once the model was fully developed for W1, it was run on the W2 study area and evaluated using an independent validation dataset consisting of 42 sample points. The rules (Table 14.3) developed for the three soil series are straightforward and represent the understanding of the soils as they occur on the landscape in relation to one another. Figure 14.1 illustrates an example of the inference interface which shows the membership function.

Table 14.3 Rules for Cabot, Colonel, and Dixfield soils Full membership at Wetness index

8

6.3

Colonel

15

Dixfield

15

Series Cabot

Slope %

0.5 membership at

Curve shape

P function

Slope %

Wetness index Slope

Wetness

Slope

Wetness

20

4.8

Z-shaped

S-shaped

3.9

35

2.4, 5.4

Z-shaped

3.4

8

4.9

S-shaped

Bellshaped Z-shaped

Limiting factor Limiting factor Limiting factor

Limiting factor Limiting factor Limiting factor

14.2.4 Evaluation The results were evaluated in two ways. First a one-to-one comparison of the hardened map and the soil series name at the calibration and validation sample points was done and represented in confusion matrices. Second, the soil series delineations were further modified using fuzzy drainage class classifications as outlined in Table 14.4. A set of criteria (Table 14.4) was developed which allows illustration of typical and atypical conditions within each soil series based on drainage characteristics. For example, the Dixfield series falls into the “moderately well drained” drainage class, and has a range of characteristics defined that allows all soils that have redoximorphic features between 41 and 101 cm to be grouped in the same category. Some soils that are classified as Dixfield may

170

J. McKay et al. a

b

Fig. 14.1 Inference interface for Colonel (ArcSIE). (a) Bell-shaped curve for wetness index, (b) Z-shaped curve for slope

be considered more typical of Dixfield while some are still Dixfield but are on the dry fringe and others are on the wet fringe. Every validation point was then assigned a fuzzy value (Table 14.5) based on a comparison of the SIE results and the evaluation of whether the field results were typical for the series’ drainage class. Average accuracy numbers were determined based on these fuzzy membership designations. Drainage class was the key characteristic, or interpretation, used to decide if the highest fuzzy membership was correct when compared to the mapped soil series at the sample points. In this particular soil survey area, parent materials were previously identified, so the difference in these catena members is essentially soil


171

Table 14.4 Evaluation criteria for fuzzy drainage class Drainage class (soil Wetter fringe Drier fringe series) Typical characteristics characteristics characteristics Poorly drained (Cabot)

O horizon 0–15 cm, chroma 2 in profile

O horizon 15–20 cm

Somewhat poorly Redox between 23 and drained (colonel) 36 cm Moderately well Redox between 56 and drained (Dixfield) 86 cm a

Redox between 0a and 23 cm Redox between 41 and 56 cm

Chroma 3 within 76 cm of top of mineral soil; must be chroma 2 somewhere Redox between 36 and 41 cm Redox between 86 and 102 cm

Includes morphologically similar soils.

Table 14.5 Matrix of fuzzy membership designations comparing SIE results and fuzzy drainage classes Field results SIE Output

Cabot (poorly drained)

Colonel (somewhat poorly drained)

SIE output

Wet fringe

Typical

Dry fringe

Cabot Colonel Dixfield

1 0.25 0

1 0.5 0

1 0.75 0

Wet fringe 0.75 1 0.25

Typical 0.5 1 0.5

Dixfield (moderately well drained) Dry fringe 0.25 1 0.75

Wet fringe

Typical

Dry fringe

0 0.75 1

0 0.5 1

0 0.25 1

wetness, which is represented by drainage class. There are no other significant interpretive differences that affect use and management of these soil series.

14.3 Results 14.3.1 Results from Model The initial output maps from SIE show fuzzy results for each soil series (Figs. 14.2 and 14.3). Darker colors represent higher fuzzy memberships for that soil. The final prediction maps (Figs. 14.4 and 14.5) for each study area are hardened maps of the SIE results, and also serve as a proxy for drainage class maps, because each soil type has a drainage class associated with it.

14.3.2 Accuracy Assessment of Model The one-to-one comparison of the hardened map to the soil series found in the field yielded 42.6% accuracy for the calibration sites in W1.

172

J. McKay et al.

Cabot

Colonel

Dixfield

High : 100

High : 100

High : 100

Low : 0

Low : 0

Low : 0

Fig. 14.2 Fuzzy prediction maps for study area W1 (McKay, 2008)

Fig. 14.3 Fuzzy prediction maps for study area W2 (McKay, 2008)


173

Fig. 14.4 Final prediction map of soil series for study area W1 (McKay, 2008)

The one-to-one comparison of the hardened map to the soil series found in the field yielded 73.7% accuracy overall in W1 (validation sites) and 71.4% accuracy overall in W2. The percent accuracy results by series name are represented in confusion tables (Tables 14.6, 14.7, and 14.8). Nine iterations of statistics were done using different arrangements of points to represent calibration versus validation points within W1 (Tables 14.9 and 14.10) to accommodate for bias in splitting the dataset. The fuzzy drainage class results show an overall average between classes of 88.8% accuracy in W1 and 89.9% accuracy in W2 (validation sets). The calibration points were 62.6% accurate overall when comparing fuzzy drainage class prediction results. While the calibration points still had lower accuracy numbers than the validation points, the drainage class results show higher accuracy (Tables 14.11, 14.12, and 14.13) than the one-to-one soil series comparison.

174

J. McKay et al.

Fig. 14.5 Final prediction map of soil series for study area W2 (McKay, 2008)

Table 14.6 Calibration prediction results based on SIE compared to observed soil series using 90 model development sites in W1 Observations Calibration sites (n:90) %

Cabot

Colonel

Dixfield

Predictions

42 21 9

25 47 52

33 33 39


Table 14.7 Validation prediction results based on SIE compared to observed soil series using 38 independent evaluation sites in W1 Observations Validation sites (n:38) %

Cabot

Colonel

Dixfield

Predictions

73 15 0

27 77 30

0 8 70



175

Table 14.8 Validation prediction results based on SIE compared to observed soil series using 42 validation independent evaluation sites in W2 Observations Validation sites (n:42) %

Cabot

Colonel

Dixfield

Predictions

69 11 0

31 63 10

0 26 90


Table 14.9 Calibration prediction results based on SIE compared to observed soil series using 90 model development sites in W1 using 9 calibration runs Observations Calibration sites (n:90) %

Cabot

Colonel

Dixfield

Predictions

42–56 (mean: 51) 18–27 (mean: 22) 0–10 (mean: 7)

18–32 (mean: 26) 40–58 (mean: 50) 36–55 (mean: 47)

18–33 (mean: 23) 21–40 (mean: 28) 35–55 (mean: 47)


Table 14.10 Validation prediction results based on SIE compared to observed soil series using 38 independent evaluation sites in W1 using 9 validation runs Observations Validation sites (n:38) %

Cabot

Colonel

Dixfield

Predictions

50–73 (mean: 60) 6–31 (mean: 19) 0–18 (mean: 5)

9–45 (mean: 24) 38–77 (mean: 52) 30–64 (mean: 43)

0–27 (mean: 16) 8–40 (mean: 29) 36–70 (mean: 52)


Table 14.11 Calibration prediction results based on SIE compared to observed drainage classes using 90 model development sites in W1 Observations Calibration sites (n:90) %

Poorly drained

Somewhat poorly drained

Moderately well drained

Predictions

68 17

32 54

0 29

0

33

66

Poorly drained Somewhat poorly drained Moderately well drained

176

J. McKay et al.

Table 14.12 Validation prediction results based on SIE compared to observed drainage classes using 38 independent evaluation sites in W1 Observations Validation sites (n:38) %

Poorly drained

Somewhat poorly drained


Predictions

78 7

21 87

0 6

0

15

85


Table 14.13 Calibration prediction results based on SIE compared to observed drainage classes using 38 independent evaluation sites in W2 Observations Validation sites (n:42) %

Poorly drained

Somewhat Poorly drained


Predictions

76 6

24 73

0 21

0

5

95


14.4 Discussion The results from both the direct comparison between the hardened map and field results and the fuzzy drainage class comparison show that the model is highly transferable between the two study areas. The model should therefore transfer well to other areas that are similar to these study areas. As one or more environmental factors change, the transferability of the model can be expected to decline. The hierarchy of these environmental factors may depend on the scale of mapping (see also Chapter 12). The calibration points showed lower accuracy than the validation points (single run), which could be a result of the fact that the calibration set is larger than the validation set in W1; thus capturing more variability in the landscape. This effect was less pronounced when multiple iterations (runs) were used. Assigning fuzzy drainage class memberships not only improves accuracy numbers; but it also points to the concept of a continuous field model, with soils changing gradually across the landscape rather than having discrete boundaries between one another. Fuzzy results for each series can be used to depict the uncertainty associated with the hardened map. There are constraints to this model. Of the five CLORPT factors (climate, organisms, relief, parent material, or time), relief most likely plays the biggest role in vari-


177

ability in this region, corresponding to the environmental factors that were used to create the model: slope and compound topographic wetness index. This is supported by the conceptual catena model that attributes variation in drainage and wetness to topographic changes.

14.5 Conclusion In summary, a knowledge-based model such as SIE can be formed and transferred between similar areas effectively, as long as environmental factor constraints are recognized.

References Baxter, S.J., and Oliver, M.A., 2005. The spatial prediction of soil mineral N and potentially available N using elevation. Geoderma 128:325–339. ESRI, 2010. ArcGIS Spatial Analyst, Environmental Systems Research Institute – ESRI, Inc., Redlands, CA (http://www.esri.com/software/arcgis/extensions/spatialanalyst/index.html; Last accessed 22 April 2010). Grunwald, S., 2009. Multi-criteria characterization of recent digital soil mapping and modeling approaches. Geoderma 152(3–4):195–207. Jenny, H., 1941. Factors of Soil Formation, A System of Quantitative Pedology. McGraw-Hill, New York, NY. Lagacherie, P., Robbez-Masson, J.M., Nguen-The, N., and Barthès, J.P., 2001. Mapping of reference area representativity using a mathematical soilscape distance. Geoderma 101:105–118. Lagacherie, P., and Voltz, M., 2000. Predicting soil properties over a region using sample information from a mapped reference area and digital elevation data: a conditional probability approach. Geoderma 97:187–208. McBratney, A.B., Mendonca Santos, M.L., and Minasny, B., 2003. On digital soil mapping. Geoderma 117:3–52. McBratney, A., Odgers, N., and Minasney, B., 2006. Random catena sampling for establishing soillandscape rules for digital soil mapping. Proceedings of 18th World Congress of Soil Science (July 9–15, 2006). IUSS, Philadelphia (USA). McKay, J., 2008. Using a knowledge-based system to test the transferability of a soil-landscape model in northeastern Vermont. M.S. thesis, University of Florida, Gainesville, FL. McKenzie, N.J., and Ryan, J.P., 1999. Spatial prediction of soil properties using environmental correlation. Geoderma 89:67–94. Milne, G., 1935. Some suggested units of classification and mapping particularly for East African soils. Soil Research 4:183–198. Mitra, B., Scott, H.D., Dixon, J.C., and McKimmey, J.M., 1998. Applications of fuzzy logic to the prediction of soil erosion in a large watershed. Geoderma 86:183–209. Shi, X., Long, R., DeKett, R., and Philippe, J., 2009. Integrating different types of knowledge for digital soil mapping. Soil Science Society of America Journal 73:1682–1692. Skidmore, A.K., Watford, F., Luckananurug, P., and Ryan, P.J., 1996. An operational GIS expert system for mapping forest soil. Photogrammetric Engineering and Remote Sensing 62:501–511. Soil Survey Staff, 2006. Keys to Soil Taxonomy, 10th ed. USDA-Natural Resources Conservation Service, Washington, DC. Thompson, J.A., Pena-Yewtukhiq, E.M., and Grove, J.H., 2006. Soil-landscape modeling across a physiographic region: topographic patterns and model transportability. Geoderma 133:57–70.