Advances in Dental Research

Advanceshttp://adr.sagepub.com/ in Dental Research

ICS-II USA Research Design and Methodology H. Rana, R.M. Andersen, T.T. Nakazono and P.L. Davidson ADR 1997 11: 217 DOI: 10.1177/08959374970110020401 The online version of this article can be found at: http://adr.sagepub.com/content/11/2/217

Published by: http://www.sagepublications.com

On behalf of: International and American Associations for Dental Research

Additional services and information for Advances in Dental Research can be found at: Email Alerts: http://adr.sagepub.com/cgi/alerts Subscriptions: http://adr.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav

Downloaded from adr.sagepub.com by guest on July 21, 2011 For personal use only. No other uses without permission.

ICS-II USA RESEARCH DESIGN AND METHODOLOGY

T

H. RANA 1 R.M. ANDERSEN2 T.T. NAKAZONO1 P.L. DAVIDSON3

Research Associate Wasserman Professor 3 Study Director ICS-II USA Ethnicity & Aging Project Department of Health Services School of Public Health University of California, Los Angeles Los Angeles, CA 90095 2

Adv Dent Res ll(2):217-222, May, 1997

Abstract—The purpose of the WHO-sponsored International Collaborative Study of Oral Health Outcomes (ICS-II) was to provide policy-makers and researchers with detailed, reliable, and valid data on the oral health situation in their countries or regions, together with comparative data from other dental care delivery systems. ICS-II used a cross-sectional design with no explicit control groups or experimental interventions. A standardized methodology was developed and tested for collecting and analyzing epidemiological, sociocultural, economic, and delivery system data. Respondent information was obtained by household interviews, and clinical examinations were conducted by calibrated oral epidemiologists. Discussed are the sampling design characteristics for the USA research locations, response rates, sample size for interview and oral examination data, weighting procedures, and statistical methods. SUDAAN was used to adjust variance calculations, since complex sampling designs were used.

Key words: Research design, comparative study, oral health, sampling design.

he purpose of the ICS-II was to provide policymakers and researchers with detailed, reliable, and valid data on the oral health situation in their countries or regions, together with comparative data from other oral health care systems. Ultimately, results may be used to improve oral health policies, dental care services, and health promotion programs in industrialized and middleincome developing countries (Chen et aL, 1997). ICS-II used a cross-sectional design. There were no explicit control groups or experimental interventions. Rather, the study involved the collection of basic descriptive information on oral health status and factors influencing it. Multivariate associations and qualitative understanding of dental care delivery systems are expected to provide important insights about potential strategies for improving oral health. A standardized methodology was developed and tested for collecting and analyzing epidemiological, sociocultural, economic, and delivery system data needed for the evaluation of oral health care systems across a set of countries and regions. Respondent information was obtained by household interviews where a standardized ICS-II questionnaire instrument was used. The ICS-II social survey measured sociodemographic characteristics, oral health beliefs, oral health behaviors, perceived oral health status, quality of life, and consumer satisfaction with dental care. Clinical examinations were conducted by calibrated oral epidemiologists using a World Health Organization (WHO) Oral Examination Form. The WHO oral examination measured periodontal status, dentition status, and recommended treatment needs. Different Principal Investigators and research teams organized and implemented data collection at each of the ICS-II research locations. The UCLA research team was responsible for cross-site comparative analyses of the adult samples for the three USA research sites. Summarized below are the sampling design characteristics, response rates, and sample sizes for interviews and oral examinations for ICS-II USA research locations. Also discussed are the adjustments made to the data, weighting procedures, and statistical methods. [For a more complete discussion on the contents of the survey and clinical examination and the field procedures, refer to Chen et ai (1997).]

SAMPLING DESIGN This paper results from a four-year project on "Ethnicity, Aging, and Oral Health Outcomes: Findings from the ICS-II USA Research Locations", funded by the Agency for Health Care Policy & Research, with additional support from the National Institute of Dental Research. The ICS-II was conducted by the World Health Organization and the Center for Health Administrative Studies, University of Chicago.

Table 1 presents sampling design characteristics for ICS-II USA research locations. Data were collected from ageeligible adults 35-44 and 65-74 years of age by means of single-stage sampling (Indian Health Service, IHS) and more complex sampling methods involving stratification and cluster design (Baltimore and San Antonio). Sampling procedures used in each of the ICS-II USA research locations


217

218

RANA ET AL.

ADV DENT RES MAY 1997

TABLE 1 ICS-II USA RESEARCH LOCATIONS SAMPLING DESIGN CHARACTERISTICS Research Location

Area Sampled

Household Sampling Design

Stratification

Baltimore, Maryland

Baltimore SMS A

Multistage design incorporating area probability techniques

Age, ethnicity, type of area

IHS-Navajo

Navajo Reservation

Random sample of ageeligible individuals registered at IHS hospital

Age, reservation service unit boundaries

IHS-Lakota

Aberdeen Area IHS

Random sample of ageeligible individuals registered at IHS hospital

Age, reservation service unit boundaries

San Antonio, Texas

San Antonio SMSA

Multistage design incorporating area probability techniques

Age, ethnicity

are summarized below. A more detailed description of these procedures can be found in the WHO ICS-II report on the second international study (Chen et ai, 1997). Baltimore, Maryland Data were collected from June, 1990, through March, 1991, in Baltimore, Maryland, where a multistage design incorporating area probability techniques was used to sample eligible adults (35-44 and 65-74 years old). At the first stage, a random sample of 200 primary sampling units (blocks or block clusters of households) was selected from 7588 sampling units in the Baltimore Standard Metropolitan Statistical Area (Research Triangle Institute, 1994). The residences in all but two of the primary sampling units, which were on military installations, were counted and listed. At the second stage, a sample of one segment of approximately 40 housing units was selected from each primary sampling unit. At the third stage, most of these units (94.4%) were screened for the presence of age-eligible individuals to serve as study participants. Indian Health Service (IHS) Data were collected from Fall, 1989, through Spring, 1991, from two reservations selected by IHS to participate in the ICS-II: the Navajo of New Mexico and Arizona, and the Lakota (formally Sioux) of South Dakota. IHS selected only reservation-residing Native Americans living in IHS service areas to participate in ICS-II. Non-reservation-residing Native Americans are more likely to be younger and employed, with a higher income and better access to dental care services than those residing on reservations. The Navajo sample was drawn from the Native American adult population (35-44 and 65-74 years old) registered on medical lists of IHS hospitals in New Mexico and Arizona and residing on the respective reservations (Lyttle, 1993a).

The estimated coverage of reservation-residing Native American adults from the hospital lists was greater than 90%, including all individuals who utilized ambulatory or inpatient services or received prescription drugs in the past ten years. Within each reservation, patient names and addresses from each age group were extracted from IHS hospital patient lists. Samples were drawn by means of a fixed-interval, randomstart methodology. A similar methodology was used to sample the Lakota reservation. The universe for the Lakota sample was the noninstitutionalized Native American adult population (35-44 and 65-74 years old) registered on medical lists of two South Dakota IHS hospitals, with mailing addresses on one of two reservations selected for the study (Lyttle, 1993b). San Antonio, Texas Data for San Antonio, Texas, were collected from October, 1991, through March, 1992 (Brown, 1994). A multistage, clustered sampling design was used to select age-eligible adults (35-44 and 65-74 years old). The first stage consisted of a simple random sampling of 14 of 140 census tracts in the San Antonio SMSA. Census tracts with rural areas, military bases, and with unknown levels of fluoride in the drinking water supply were excluded. The second stage involved a random selection (without replacement) of blocks within selected tracts, and households were listed for these selected blocks. At least two attempts were made to reach each household by telephone. If there was no response, households were visited in person to ascertain whether age-eligible adults resided in the household. The household screening success rate for age-eligible adults was 59%.

RESPONSE RATES Household interviews (n = 4412) and oral examinations (n =


YOL.Ild)

RESEARCH DESIGN &

219

METHODOLOGY

TABLE 2 COMPLETED SAMPLE SIZE FOR ICS-II USA RESEARCH LOCATIONS1 White

Baltimore Afr.-Amer.

IHS Navajo

Lakota

San Antonio Hispanic White

Total interview 35-44 yrs 65-74 yrs

1382 733 649

387 215 172

1020 490 530

780 447 333

276 111 165

567 297 270

Total exam 35-44 yrs 65-74 yrs

439 253 186

95 43 52

904 413 491

748 413 335

155 50 105

314 136 178

Both interview and exam2 35-44 yrs 65-74 yrs

439 253 186

95 43 52

885 404 481

723 394 329

155 50 105

314 136 178

Other ethnic groups or cases with no reported ethnicity (Baltimore, n = 44; San Antonio, n = 94) were removed from the data. All those completing exam also completed interview, except for IHS (n = 44).

2655) were conducted with middle-aged and older adults from multiple ethnic groups sampled in the three ICS-II USA research locations. Table 2 reports completed sample size for interviews and oral examinations by ethnic group and age cohort. Almost three-fourths of the Baltimore adult sample completed the household interview (35-44 years, 74%; 65-74 years, 72%), while response rates were much lower for the oral examination (23% and 21%, respectively). IHS response rates were highest among USA research locations for both interview (35-44 years, 76%; 65-74 years, 85%) and oral examination (67% and 81%, respectively). San Antonio ICSII reported both lower interview (35-44 years, 40%; 65-74 years, 39%) and lower oral examination response rates (35-44 years, 18%; 65-74 years, 25%). We were unable to provide response rates by ethnic group for Baltimore and San Antonio because the ethnicity of non-respondents was unestimated. Low response rates for the Baltimore oral examination and the San Antonio interview and oral examination potentially produce bias estimates of the study variables. The deviation of a sample estimate from the true population value, due to small sample size, is referred to as sampling error. Additionally, we are concerned about the sample bias which potentially occurs due to incomplete sampling frames and incomplete data collection. Sampling bias, which is nonrandom and difficult to detect, is often more damaging to sample accuracy. We conducted a bias analysis to examine whether systematic differences existed between respondents and non-respondents.

BIAS ANALYSIS Due to low response rates in Baltimore and San Antonio research locations, bias analyses were conducted to determine whether systematic differences existed between respondents

and non-respondents. Bias analysis consisted of two stages: (1) comparison of respondents with non-respondents (San Antonio), and (2) comparison of respondents who completed both interview and oral examination with respondents completing interview only (Baltimore and San Antonio). Respondents vs. non-respondents in San Antonio Response rates of middle-aged (35-44 years) and older (6574 years) adults in the San Antonio sample were correlated with selected variables from the 1990 San Antonio SMS A census data. Demographic variables selected from census data included ethnicity, percent of the population in the 35-44 and 65-74 age cohorts, respectively, gender, percent reporting greater than high school education (> 12 years), and median family income. Table 3 shows the correlation between response rates of sample data and selected census variables. None of the census variables was significantly correlated to the middleage or older cohort response rates. No significant differences were found with regard to ethnicity, age cohort, gender, educational attainment, and median family income. Respondents completing only interview Due to the considerable number of interview respondents failing to complete the oral examination in Baltimore and San Antonio, a comparison was made to assess statistically significant differences in key study variables by means of Chi-square and Student's t tests. In Baltimore, no differences were found with regard to ethnicity, age cohort, gender, occupation, marital status, dentition status, usual source of care, and dental benefits. However, respondents completing both interview and oral examination reported significantly higher household income, stronger belief in the seriousness of oral diseases, more oral pain and other oral symptoms, and


ADIYUENTKESMAYIW/

RANA ET AL.

220

TABLE 3 CORRELATION BETWEEN CENSUS TRACT RESPONSE RATES AND CENSUS TRACT CHARACTERISTICS BY AGE GROUPS (35-44 years, 65-74 years) Mean Response Rate by Census Tract for Adults Adults 35-44 yrs Adults 65-74 yrs

% Female

% Hispanic

% 35-44 Age Group

% 65-74 Age Group

% with College Education

Median Family Income

-0.1798 (0.5385)1

0.3169 (0.2696)

-0.1724 (0.5556)

-0.4709 (0.0892)

-0.3685 (0.1948)

0.0459 (0.8761)

-0.0107 (0.9710)

0.0676 (0.8182)

-0.0838 (0.7756)

-0.1927 (0.5091)

-0.1927 (0.9710)

0.0734 (0.8030)

Numbers in parentheses are the significance levels. reported dental visits more often. In San Antonio, no differences were found with regard to ethnicity, occupation, marital status, dentition status, usual source of care, dental benefits, oral pain, and reported dental visits. However, older persons, females, and those reporting more oral symptoms and problems with dentures were significantly more likely to complete both interview and oral examinations. Baltimore and San Antonio ICS-II investigators reported reasons for low oral examination response rates, such as fear of dentists, fear of HIV infection, inability to contact subjects at home (i.e., persons work more than one job or both spouses work), population mobility resulting from increased unemployment, street violence, and home theft, resulting in less trust (Brown, 1994; RTI, 1994).

ADJUSTMENTS TO THE DATA Adjustments to the data were required before further analyses, including: (1) weighting procedures, (2) correction for sampling design effects, and (3) imputation of critical variables. Weighting procedures ICS-II was designed to produce unbiased estimates for the entire population of eligible 35-44 and 65-74 age cohorts by ethnic group within each ICS-II USA research location. Thus, sample data were weighted to reflect more accurately the population from which the samples were drawn. The inflation factor, or weight, for each individual was the product of several adjustments, requiring one or more adjustments for each stage of the sampling procedure. Five types of adjustments were required, including probability of selection, response rates, unadjusted weight, post-stratification, and standardization (Lyttle 1993a,b). The probability of selection is equal to the size of the sample (a) divided by the size of the list (P). Selection probabilities were calculated as a/(3. The response rates were calculated as 7/8 where 7 is the number of individuals responding to the given instrument for

the age group, and 8 is the total number of eligible individuals on the sampling list for the age group. Rates were calculated for responses to the questionnaire, oral examination, and both instruments. The unadjusted weights for selection and response rates were constructed by multiplying the inverse of the selection rate by the inverse of the response rate, by census tract. This adjustment assumes that the characteristics of the nonrespondents in census tracts are the same as those for respondents. While this assumption will not be totally accurate, we believe that the characteristics of the population as a whole will be better reflected by a sample adjusted for differential response at the census tract level than a sample with weights not adjusted for this differential. The unadjusted weights were calculated as (a/p) * (7/8). A post-stratification adjustment was computed for each gender by ethnicity based on populations derived from 1990 census data. Post-stratification achieves improvement in precision similar to that attained had the sample been drawn from a population stratified by gender and ethnicity. Each of the weights was standardized to a mean of one for each of the age groups. For certain analyses, ethnicity (Baltimore White and African-American; IHS Navajo and Lakota; San Antonio White and Hispanic) and age cohort (35-44 years and 65-74 years) were included as independent variables in the regression model. The IHS data collected from two research locations had to be weighted further to pool the data for these analyses. A new "ethnic group weight" was created to reflect, using census proportions, the true proportions for Navajo and Lakota age cohorts in their respective populations. The final ethnic group weight ensured that the total contribution of the Navajo sample was equal to the total contribution of the Lakota sample. Adjusting for sampling design effects using SUDAAN Since complex sampling designs were used to collect data, variance estimates and tests of significance will be biased if simple random sampling is assumed. Multistage (clustered) sampling, used in Baltimore and San Antonio (Table 1), is


VOL11(2)

RESEARCH DESIGN

&

221

METHODOLOGY

statistically less efficient than simple random sampling. It is the equivalent of using simple random sampling with smaller samples of cases. Consequently, multistage sampling produces a large standard error, resulting in a more conservative test statistic than that obtained when simple random sampling is assumed. Therefore, SUDAAN was used to adjust variance calculations based on stratification and clustering (Shah et ai, 1991).

0.05 significance level. The t test statistic for testing the equality of means Xj and x2 from two independent samples with equal variances and nl and n2 observations is:

Imputation ICS-II contained a number of important core variables called "critical items". The imputation procedure is essential for preserving the maximum sample size for the critical items in the study. If a critical item had missing or non-meaningful values for 25 cases or more, imputation was performed by the modified bootstrap imputation procedure. Household income was imputed for Baltimore and San Antonio research locations. Variables measuring employment, type of work, household income, reason for no dental visit, and type of dental insurance were imputed for IHS. The modified bootstrap imputation procedure is explained briefly below. "Bootstrap" imputation, more properly called prior probability random number generation, is a process particularly appropriate for use when values for categorical variables are imputed. The basic concept of the bootstrap method of imputation is to assign values to missing cases based on the distribution of values found in the data set. The bootstrap procedure has the virtue of preserving the observed variance of the variable being imputed. Given the generation of random numbers, imputed values should distribute across the variable categories in approximately the same proportions as the observed values.

where s2 is the pooled variance,

STATISTICAL METHODS Bivariate and multivariate statistical methods were used to analyze data. Other statistical and mathematical methods (e.g., principal component analysis) were used for the construction of oral health indicators. Bivariate tests Differences in sample characteristics and oral health outcomes were compared by ethnic group and age cohort. Significance tests were conducted (1) between two age cohorts (35-44 years and 65-74 years old) within each ethnic group by means of pairwise t tests, and (2) across all six ethnic groups (Baltimore White and African-American; IHS Navajo and Lakota; San Antonio White and Hispanic), with Bonferroni's multiple means test to control the overall Type 1 error rate (i.e., the rejection of the null hypothesis when it is in fact true). [To calculate the significance tests from the data reported in the Tables, first obtain the variance, s2, which can be calculated from the standard error of the mean and sample sizes: s2 = SE2 x n, where s2 is the variance, SE is the standard error, and n is the sample size.] Pairwise t test. Pairwise Student's t test was used to hypothesize mean differences between age groups at the p