Sampling Theory

Trim Size: 170mm x 244mm

Sampling Theory PETER N. PEREGRINE Lawrence University and the Santa Fe Institute, USA

k

There is a single objective underlying sampling theory: to create a subset of a population in which all cases in the subset have an equal probability of being selected from the population. A sample produced in this way will accurately reflect the range of variation in the population. All probabilistic methods of sampling are designed to achieve this basic objective. Four sampling methods based in probability theory are commonly used in archaeology: simple random sampling, systematic random sampling, stratified sampling, and cluster sampling. These methods all produce samples that match the statistical parameters of the population, and thus meet the requirements for statistical analysis. Several non-probabilistic methods are also commonly used in archaeology, but are not recommended to produce samples intended for analysis beyond basic descriptive statistics (and even these may be misleading). Probabilistic sampling is based on the assumption that all natural populations have a “normal” distribution of variation from which the proportion of individuals with a given characteristic can be estimated. This assumption is visually demonstrated through the “normal curve” (see, e.g., Thomas 1986, Figure 7.12). A “normal curve” is a form of a histogram, where the range of variation for a given characteristic is shown on the x-axis and the number of individuals with that characteristic is shown on the y-axis. In a “normal curve” the arithmetic mean (the sum of all individual scores on the given characteristic divided by the number of individuals in the population) is the high point of the curve. The population variance is calculated as the sum of the squared differences between the value of each individual on the given characteristic and the mean, divided by the number of individuals in the population—in essence it is a measure of the average difference between an individual score and the population mean. In practice the square

k

saseas

saseas0516.tex

V1 - 03/16/2018

2:07 P.M. Page 1

root of the variance, called the standard deviation, is used to describe the population variance, as it is an arithmetically more useful measure. Knowing the mean and variance (and standard deviation) of a population with a “normal curve” allows us to estimate the number of individuals in the population with a particular value on a given characteristic. For example, if the population has a mean of 100, a standard deviation of 10, and contains 1000 individuals, we can estimate that there should be about 20 individuals with a value of 110. An accurate sample of 100 individuals from this population should therefore contain 2 individuals with a score of 110. A good sample should contain cases that accurately match the proportion of cases with the same score in the population. Achieving that is the foundation of all probabilistic sampling methods. Simple random sampling is the purest form of probabilistic sampling. To produce a simple random sample, every case in the population of interest is given a number. Cases for the sample are then selected using a random number generator, so that every case is selected randomly and thus each has an equal probability of selection. A sample produced by simple random sampling will have the same mean and variance as the parent population. Typically, only 10% of cases in the parent population need to be selected in order for a sample to accurately match its mean and variation. Another method of random sampling is called systematic random sampling, and is often easier to perform in practice than simple random sampling. In systematic random sampling every case in the population of interest is assigned a number, just as in simple random sampling. But rather than selecting cases for the sample using a random number generator, in systematic random sampling only two random numbers are used—a random start and a random interval. The case with the random start is the first case selected for the sample, and then every case falling at the random interval following the start case is selected. For example, given a population of 1000, a researcher might simply use a dice to select a start case, say case number 2, and then a dice

The SAS Encyclopedia of Archaeological Sciences. Edited by Sandra L. López Varela. © 2019 John Wiley & Sons, Inc. Published 2019 by John Wiley & Sons, Inc.

k

k

Trim Size: 170mm x 244mm

2

k

k

saseas

saseas0516.tex

V1 - 03/16/2018

2:07 P.M. Page 2

SA MP L I NG T HE ORY

for the random interval, say 4. The researcher would select cases 2, 6, 10, 14, 18, 22, and so on for the sample. A sample produced in this manner will, like one produced by simple random sampling, match the mean and variation of the parent population. However, there is a caveat: systematic random sampling should not be used if there is a known or suspected periodicity in the population of interest, as the random start and interval might, by chance, match that periodicity, and thus all cases of a certain kind might be either systematically selected or systematically missed (Orton 2000, 22). There is a significant problem in doing random sampling in practice, whether simple or systematic—the full range of the parent population is usually not known. Taking a simple example, we might want to do a study of the ceramics from an archaeological site. To create a simple random sample of that population we would first have to identify all the ceramics, which is pragmatically impossible. We know we could not have collected every ceramic unless we excavated the entire site, and even in that case we would have difficulty defining the boundaries of the site itself. And because ceramics are usually fragmentary, we would also have to decide how much of a given ceramic object we need to consider it part of the ceramic population. Because of problems like these, it is impossible to identify a “true” parent population in all but the most unique situations, and we must define them using specific criteria and focusing on a specific point in time and space. Thus, any population used in research is only an approximation of the “true” parent population, and any sample is an approximation of that approximation. This is a problem that no sampling method can overcome and we should remain mindful of it in making generalizations about any population we are studying (Thomas 1986, 439–47). A second problem with random sampling is that we may be most interested in particular segments of a given parent population, and if these segments are relatively rare in the parent population, they will be equally rare in the sample. To acquire sufficient rare cases to perform analyses might require a very large, and perhaps quite costly and time-consuming, sample to be taken from the parent population. In addition, if our parent population is spatially dispersed,

collecting data on the cases selected through random sampling might be extremely costly and time-consuming, as the sampled cases might be separated by great distances. In both cases a more focused sample, either including more of the cases of interest or identifying cases that are spatially close, might be desired. These can be achieved using stratified or clustered random sampling. In stratified random sampling we select separate random samples from several different groups within the parent population; for example, a random sample of those cases in which we are most interested and a random sample from the rest of the population. Knowing the mean and variance of the individual samples allows us to estimate the mean and variation of the parent population as well. And if our research requires us to compare the cases of interest with the rest of the population, this method of sampling provides a way to identify an adequate number of the rare cases to make those comparisons. In cluster sampling we select simple random samples from defined spatial areas within the parent population. We might, for example, randomly select individual areas within a site and collect data from all cases in those areas. This might make the data collection much less costly and time-consuming than having to go all over the site to collect data from cases selected through simple or systematic random sampling. As with stratified random sampling, we can estimate the mean and variance of the parent population through those of the samples. We can also readily compare various areas within a site or region, if that addresses a question of interest. Probabilistic samples are technically the only ones that should be used in statistical analyses, but pragmatically this is extremely difficult to achieve in archaeology, primarily because we typically know very little about our parent populations. With this in mind there are also several non-probabilistic sampling methods that are reasonable to use in situations where the parent population is completely unknown or where probabilistic sampling is impossible. These methods do not typically provide samples that match the mean and variance of the parent population, and thus they cannot be used to make generalizations about the parent population (Kalton 1983, 90).

k

k

Trim Size: 170mm x 244mm

k

saseas

saseas0516.tex

V1 - 03/16/2018

2:07 P.M. Page 3

SA MP L I NG T HE ORY

k

Opportunistic sampling selects all available individuals with a given characteristic for inclusion in the sample. For example, an archaeologist might analyze all the ceramics available in a local museum collection. Grab bag sampling is similar, but the archaeologist would select a group of those ceramics by non-systematic choice; for example, the first 20 ceramic objects s/he found in the museum collection. Grab bag sampling, although it may appear to use a random method, does not, because the probability of selecting a given case changes as items are chosen for the sample. Although often easier than probability sampling, non-probability sampling should only be used when probability sampling is impossible. Regardless of the sampling technique used, it is important to determine an appropriate sample size. Sample size should balance the cost of collecting data and the accuracy required to evaluate a given hypothesis. By convention, a 10% random sample is considered adequate for most research problems. However, questions where very small differences (or effect sizes) between individuals or populations need to be evaluated require a larger sample, and where there are large expected differences, a large sample may actually provide too much accuracy, always finding differences even when they are not actually there (a Type I error). Determining sample size is often done through power analysis. There are a number of power calculators on the Internet that provide a

3

quick way to determine the appropriate sample size for a given hypothesis. It is important to note, however, that power analysis is only useful for random samples—another reason why random sampling should always be used whenever possible.

SEE ALSO: Descriptive Statistics; Hypothesis Testing; Human Populations; Inferential Statistics

saseas saseas saseas saseas

REFERENCES Kalton, Graham. 1983. Introduction to Survey Sampling (Quantitative Applications in the Social Sciences 35). Newbury Park, CA: Sage. Orton, Clive. 2000. Sampling in Archaeology. Cambridge: Cambridge University Press. Thomas, David H. 1986. Refiguring Anthropology. Prospect Heights, IL: Waveland Press.

FURTHER READING Baxter, Michael. 2010. Statistics in Archaeology. Chichester: Wiley. Drennan, Richard. 2009. Statistics for Archaeologists: A Common Sense Approach. New York: Springer. VanPool, Todd and Robert D. Leonard. 2011. Quantitative Analysis in Archaeology. Chichester: Wiley-Blackwell.

k

k

Trim Size: 170mm x 244mm

k

saseas

saseas0516.tex

V1 - 03/16/2018

2:07 P.M. Page 4

Please note that the abstract and keywords will not be included in the printed book, but are required for the online presentation of this book which will be published on Wiley’s own online publishing platform. If the abstract and keywords are not present below, please take this opportunity to add them now. The abstract should be a short paragraph upto 200 words in length and keywords between 5 to 10 words.

ABSTRACT There is a single objective underlying sampling theory: to create a subset of a population in which all cases in the subset have an equal probability of being selected from the population. A sample produced in this way will accurately reflect the range of variation in the population. All probabilistic methods of sampling are designed to achieve this basic objective. A variety of probabilistic sampling methods have been developed for archaeology so that appropriate methods can be applied to a wide range of sampling contexts. There are also non-probabilistic sampling methods used in archaeology, although they do not produce samples that accurately reflect the parent population. k

k

KEYWORDS Sampling; probability; normal distribution

k