C 2005) Journal of Behavioral Medicine, Vol. 28, No. 2, April 2005 ( DOI: 10.1007/s10865-005-3666-1
Reliability and Construct Validity of the Pain Distress Inventory Augustine Osman,1,4 Francisco X. Barrios,1 Peter M. Gutierrez,2 Braden Schwarting,1 Beverly A. Kopper,1 and Mei-ChuanWang3 Accepted for publication: July 5, 2004
We conducted three studies to evaluate further the reliability and construct validity of a new self-report instrument, the Pain Distress Inventory (PDI; Osman et al., 2003, The Pain Distress Inventory: Development and initial psychometric properties, J. Clin. Psychol. 59: 767– 785). In Study I, exploratory and confirmatory factor analytic results confirmed the replicability of the four-factor oblique solution of the PDI in a mixed sample of students and nonstudents. We also found strong evidence for criterion-related validity of scores on this instrument. In Study II, multisample analyses results found further evidence for equivalence of structure of the PDI across African American and Caucasian young adults. Ethnic and gender group differences were obtained on two of the PDI scale scores. Internal consistency reliability estimates on the PDI total and scale scores were good in both Studies I and II. In Study III, additional analyses of internal consistency and known-groups validity established strong support for construct validity of the PDI. KEY WORDS: pain distress; self-report; assessment; PDI; psychometrics.
INTRODUCTION
including factorial validity, internal consistency reliability and concurrent validity estimates in samples of college undergraduates. The current studies were carried out to examine further the reliability and construct validity of this new instrument. In developing the PDI, Osman and colleagues reasoned that some of the self-report instruments that are used most frequently to assess affective distress in the pain literature may lack content validity. For example, several items on the Symptom Check List-90R (SCL-90R; Derogatis, 1994; Item 34, your feelings being easily hurt), the Beck Depression Inventory-II (BDI-II; Beck et al., 1996; Item 5, I feel guilty all of the time), and the State-Trait Anxiety Inventory (STAI-T; Spielberger et al., 1983; Item 13, I feel secure) are specifically designed to assess psychological distress, not distress related to pain. The authors designed the PDI in response to the need for a reliable and valid self-report instrument that taps specific pain-related distress responses. In addition to the strong rationale given for developing this new instrument, the authors used a number of contemporary test development
Recently, Osman et al. (2003) developed and validated a new self-report measure to assess painspecific distress responses. This 26-item instrument, the Pain Distress Inventory (PDI), is composed of four distress factors that have been linked with a range of pain conditions: Depression (seven items), Anger (six items), Pain Sensitivity (six items), and Somatic Anxiety (seven items). The PDI items are rated on a 5-point scale ranging from 0 (not at all like me) to 4 (very much like me). The item ratings are summed and then averaged to derive a composite score for the PDI. The authors reported strong preliminary psychometric properties for the instrument
1 Department
of Psychology, The University of Northern Iowa, Cedar Falls, Iowa. 2 Northern Illinois University, DeKalb, Illinois. 3 University of Memphis, Memphis, Tennessee. 4 To whom correspondence should be addressed at Department of Psychology, The University of Northern Iowa, 334 Baker Hall, Cedar Falls, Iowa 50614-0505; e-mail:
[email protected].
169 C 2005 Springer Science+Business Media, Inc. 0160-7715/05/0400-0169/0
170 approaches to generate content valid items and to examine initial psychometric properties of reliability and validity of the PDI in samples of college undergraduates. For example, to maximize content validity, the PDI items were drawn from multiple sources as recommended in the psychometric literature (see DeVellis, 2003; Haynes et al., 1995). The items were rated by experts in terms of content relevancy and specificity. The factor scales were derived and validated when the authors submitted the initial 30 items to an exploratory principal-axis factor analysis. After dropping four items, the four factors that were retained in Study I accounted for 65.10% of the variance. All the factor scales showed good evidence of internal consistency (i.e., alpha estimates >0.70). In subsequent studies reported in the same paper, the authors also reported strong evidence of concurrent validity for the PDI total and scale scores. For example, scores on the PDI scales correlated moderately and significantly with scores on other measures relevant to pain-related construct including the Pain Catastrophizing Scale (PCS; Sullivan et al., 1995; rs ranged from 0.48 to 0.63) and the Survey of Pain Attitudes Scale-35 (SOPA-35; Jensen et al., 2000). However, unlike most existing self-report instruments designed to assess pain-related behaviors, such as the PCS (Sullivan et al., 1995) and the Fear of Pain Questionnaire-III (FPQ-III; McNeil and Rainwater, 1998), the PDI psychometric data are limited to the instrument-development samples. Osman et al. (2003) did not extend the validation investigations to a variety of nonclinical samples; their analyses were limited extensively to psychology undergraduate samples. In addition, other psychometric properties of the PDI such as comparison of the four-factor structure of the PDI across Caucasian and African American samples (multisample) have not been examined. In the present investigation (Study I), it was hypothesized that the PDI total score would make significant relative contribution to the prediction of scores on pain-related indexes including pain interference and pain distress severity. A major methodological limitation of most selfreport measures of pain-related behaviors including the PDI is that the development and validation processes of these instruments do not allow for comparisons of responses across ethnic groups using mean scale scores. For example, Osman et al.‘s (2003) samples were predominantly Caucasian (i.e., across the instrument development studies, 78.4–95.0% of the
Osman et al. study participants were Caucasian). Indeed, the importance and value of validating pain assessment instruments across different populations have been noted in the pain literature (see Turk and Melzack, 1992). In the present study, we evaluated the psychometric properties of the PDI with African American and Caucasian nonclinical samples. To date, the paucity of well-validated self-report instruments for use with African Americans is striking given that African Americans also present with a variety of pain-related conditions (see Barbarin and Christian, 1999; Greenwald, 1991; O’Maria and Arenella, 2001; Riley-III et al., 2002). We conducted three studies to examine the reliability and construct validity of the PDI. Although Osman et al. (2003) reported several forms of validity estimates for this instrument, those findings are considered to be at the preliminary stages of instrument development; they need replication in other investigations. In addition, participants in the instrumentdevelopment studies were all drawn from the same institution. Therefore, there is a need to attempt to replicate findings including factorial validity and internal consistency reliability estimates of the PDI in independent samples. The first study (Study I) investigated the psychometric properties of factor structure, criterion-related validity, and internal consistency of the PDI. In Study II, we examined the responses of African American and Caucasian young adults on this instrument. We used a multigroup procedure to evaluate invariance of the four-factor oblique solution of the PDI. In Study III, we evaluated the ability of scores on the PDI to differentiate between a self-reported pain and an appropriate control group to replicate evidence of known-groups validity. Because pain symptoms are continuously distributed in the general population, the inclusion of samples with differing degrees of painrelated distress is considered clinically relevant for normative comparisons (see Barlow, 1981; Osman et al., 1997).
STUDY I Method Study I was designed to extend construct validation of the PDI to a heterogeneous sample of nonclinical student and nonstudent adult samples. First, we assessed replicability of factor structure using exploratory and confirmatory factor analyses.
Validity of the PDI Second, given that reliability estimates are sample specific, we examined the internal consistency of the PDI total and scale scores using both coefficient alpha estimates and mean interitem correlations for the study samples. Third, we evaluated evidence for convergent-discriminant validity, and the relative contributions of scores on the PDI to the prediction of pain-related indexes.
Participants and Procedure The participants were a mixed sample of undergraduate and graduate students (n = 74 men and 125 women) and nonstudent adults (n = 27 men and 30 women). The student participants were recruited from a Midwestern university and a community college. Specifically, students were drawn from psychology and other (Social and Behavioral Sciences) graduate and undergraduate courses. The nonstudent adults were recruited by students from advanced psychology courses in exchange for course extra credit. Data from all the participants were combined to maximize the heterogeneity of the present nonclinical sample (N = 256). All participants were given the questionnaires to complete at home and return within 2 days. Consistent with the approved Institutional Review Board procedures, each participant completed a written consent form to volunteer study participation. Data were collected over two consecutive semesters to maximize sample size. The mean age of the sample was 23.58 years (SD = 5.41; range = 18–53 years). Men (M = 23.06 years, SD = 4.11; range = 18–49 years) and women (M = 23.92 years, SD = 6.10; range = 18–53 years) did not differ significantly in age, t(254) = 1.25, p = 0.21. The sample was 91.4% Caucasian, 3.9% African American, 2.7% Asian American, and 2.0% of other ethnicities. Approximately 77.3% were single, never married; 11.3% were married; 5.9% were divorced/separated/widowed, and 5.5% were cohabiting.
Measures In addition to the PDI, participants completed a background information questionnaire containing demographic information and specific pain indexes, and five other validation self-report instruments. One item was included to assess worst pain experienced
171 in the past 4 weeks (0 = no pain to 10 = very intense pain); one item was designed to evaluate current level of “pain intensity/discomfort” (0 = no pain or discomfort to 10 = very intense pain discomfort); and one item was designed to obtain information on the frequency of visits to a medical care facility for at least one physical pain-related condition. Physical Pain Symptom Index (PSI; Osman et al., 1997). The background information questionnaire also contained nine items for assessing the extent to which physical pain symptoms (e.g., headache pain; pains in the neck, throat, or shoulders) have been experienced in the past week, including today. Each PSI item is rated on a 1 (no pain at all) to 5 (extremely painful) Likert-type scale. Exploratory principal-axis factoring (PAF) identified a single factor defined by all 9 items. The internal consistency for the PSI was good (coefficient alpha = 0.74, 95% CI = 0.69–0.78; mean interitem r = 0.24). Scores on the PSI were summed to obtain a physical pain distress index. Pain Interference Index (PII; Osman et al., 1997, 2003). Three items were included in the background questionnaire to assess the extent to which painrelated symptoms interfere with functioning in three areas: social/recreational activities, ability to work, and ability to carry out most day-to-day activities. Each item is rated on a 10-point scale ranging from 1 (no interference) to 10 (extreme interference). We summed scores on these items to obtain an index of pain interference (coefficient alpha = 0.91, 95% CI = 0.89– 0.93; mean interitem r = 0.77). Pain Anxiety Symptoms Scale (PASS; McCracken et al., 1992). The PASS is a 40-item selfreport measure of fear and anxiety symptoms that are related to pain (fear, cognitive, escape/avoidance, and physiological). Each item is rated on a 6-point scale ranging from 0 (never) to 5 (always); five items are reverse scored. McCracken et al. (1992) reported good psychometric information regarding reliability and validity of the PASS in pain clinic samples. In other studies, scores on the PASS have also been shown to correlate moderately and significantly with cognitive (r = 0.57) and somatic (r = 0.58) distress symptom measures in community adult samples (e.g., Osman et al., 1994). The total PASS score was used in the present study as a measure of pain-related (distress) fear and anxiety symptoms. Pain Catastrophizing Scale (PCS; Sullivan et al., 1995). The PCS is a 13-item self-report inventory designed to evaluate the degree of catastrophizing in response to pain (rumination, magnification, and
172 hopelessness). Items are rated on a scale ranging from 0 (not at all) to 4 (all the time). The instrument developers reported good internal consistency, convergent and discriminant validity estimates for this instrument. In nonclinical and clinical populations, the PCS scores have shown excellent factor structure, reliability and convergent validity (see Osman et al., 1997; 2000; Sullivan et al., 1995). Scores on the PCS have been shown to predict scores on measures of pain severity (standardized β = 0.23, p < 0.001) and pain interference (standardized β = 0.37, p < 0.001) in nonclinical samples (see Osman et al., 2000). The PCS total score was used as a painrelated measure of catastrophizing in the present study. Inventory of Negative Thoughts in Response to Pain (INTRP; Gil et al., 1990). The INTRP is a selfreport measure of the frequency of negative statements during pain episodes. The items are rated on a 5-point scale with anchors of 1 (never) to 5 (always). The scale has excellent internal consistency estimates, convergent, and discriminant validity in clinical and nonclinical samples (Gil et al., 1990; Osman et al., 1993). Recently, Osman et al. (2002) found that scores on the INTRP were useful in predicting scores on a pain severity index (standardized β = 0.25, p < 0.001) in a nonclinical sample. The INTRP total score was used in the present study as a pain-related (cognitive) measure of the frequency of negative thoughts. Psychological Distress Measures. We included three self-report distress instruments with well established factor solutions and excellent psychometric properties in the psychological assessment literature. Scores on these instruments were used, in part, to establish evidence of discriminant validity for the total PDI score. The specific psychological distress inventories were (1) the State-Trait Anxiety Inventory— Trait (STAIT-T; Spielberger et al., 1983), a 20-item self-report measure of trait anxiety; (2) the State Trait-Depression Adjective Check List (ST-DACL Form F; Lubin, 1994), a list of 34 adjectives designed to tap trait depressed mood; and (3) the Mood and Anxiety Symptom Questionnaire—90 (MASQ-90; Watson and Clark, 1991), a 90-item self-report measure of psychological distress (anxious arousal, anhedonic-depression, general distress—anxious symptoms, general distress—depressed symptoms, and mixed anxiety-depressive symptoms). Only the mixed anxiety-depressive (i.e., MASQ-90-General Disturbance) symptom scale was used in this study.
Osman et al. Results and Discussion Factor Structure of the Pain Distress Inventory We evaluated the factor structure of the PDI at two levels: (a) item-level analyses, using exploratory factor analytic procedures, and (b) parcel-level analyses (i.e., means of two or three item scores serving as items), using confirmatory factor analytic (CFA) procedures.5 Item Level Exploratory Factor Analysis (EFA). Preliminary analyses indicated that scores on the PDI items showed moderate departure from normality (Mardia’s normalized estimate = 38.57). Thus, items were subjected to the maximum likelihood parameter estimation (with robust standard errors and a mean adjusted chi-square test statistic; MLM) procedure with promax rotation in the Mplus 2.12 program ´ and Muthen, ´ 2003). Preliminary analy(see Muthen ses of the scree plot, eigenvalues ≥1.0, and parallel analyses (PA; 95th percentile eigenvalue) suggested that four to five factors could be extracted (see Zwick and Velicer, 1986). We took advantage of the extracting functions in Mplus to successively extract 1–5 solutions. Only solutions with (a) eigenvalues ≥1 and (b) loadings ≥0.40 for five or more items on a factor were initially retained. The final decision to retain a factor solution was determined by parsimony and interpretability of the solution. Results of the EFA are presented in Table I. The four-factor solution was retained because it met all the preestablished criteria. Factor I (eigenvalue = 4.74; 18.22% of the variance) consisted of six of the seven Somatic Anxiety items. Item 24 had a loading of less than 0.40 on this factor. Factor 2 (eigenvalue = 4.20; 16.14% of the variance) was composed of all seven Depression items; Factor 3 (eigenvalue = 3.28; 12.60% of the variance) was composed of all six Pain Sensitivity items, and Factor 4 (eigenvalue = 3.21; 12.34% of the variance) was composed of all six original Anger items. The four factors explained 59.30% of the total variance. We repeated the analyses using CFA. Results of the Robust-CFA were as follows: Robust chi square = 510.17, df = 293 (scaling correction = 1.23); 5 We
conducted preliminary analyses to examine mean PDI scale score differences between the student and nonstudent adult samples. The overall multivariate analysis of variance (MANOVA) across the four scale scores was not statistically significant, (F [4, 251] = 1.27, p = 0.28), suggesting the lack of differences between groups on the PDI.
Validity of the PDI
173
Table I. Exploratory Factor Analysis for the Pain Distress Inventory: Item Level Analyses Factor Factor 1:
Factor 2:
Factor 3:
Factor 4:
Items (loadings) PDI-somatic anxiety 1(0.87), 2(0.64), 5(0.50), 7(0.43), 9(0.64), 24 (0.28), 25(0.40) Eigenvalue = 4.74; variance (%) = 18.22 PDI-depression 4(0.51), 13 (0.69), 17(0.67), 18(0.90), 19(0.76), 22(0.70), 23(0.61) Eigenvalue = 4.20; variance (%) = 16.14 PDI-pain sensitivity 3(0.78), 11(0.81), 15(0.77), 20(0.72), 21(0.70), 26(0.83) Eigenvalue = 3.28; variance (%) = 12.60 PDI-anger 6(0.70), 8(0.63), 10(0.68), 12(0.77), 14(0.53), 16(0.69) Eigenvalue = 3.21; variance (%) = 12.34
comparative fit index (CFI) = 0.916; Tucker–Lewis Index (TLI) = 0.907; root-mean-square error of the approximation (RMSEA) = 0.058. The R-squared values ranged from 0.298 (Item 25) to 0.682 (Item 19). Overall, the CFI suggests that the current four-factor model is a better fitting model compared to the null model. Parcel Level Confirmatory Factor Analysis (CFA). Floyd and Widaman (1995) have noted that it is difficult to confirm the factor structure of an instrument when five or more individual items are allowed to load freely on a factor. They recommended the use of parcels in establishing the fit of stronger factor solutions. Thus, we computed the mean score for two or three items within each scale to serve as an item (parcel) in the CFA. We note that because of the high internal consistency estimates obtained for the PDI scales, items were randomly assigned to each parcel. For example, for the Pain Sensitivity scale, Items 3
Coefficient alpha (95% CI)
Mean interitem (r)
0.82 (0.79–0.85)
0.40
0.90 (0.88–0.92)
0.56
0.90 (0.88–0.92)
0.60
0.82 (0.78–0.85)
0.44
and 11 were assigned to the first parcel, Items 15 and 21 to the second parcel, and Items 20 and 26 to the third parcel (see Table II). We evaluated the fit of three models to the present sample data: (a) a one-factor model, to evaluate intercorrelations among the PDI items when constrained to load on a single factor, (b) a fourfactor oblique model, to assess the fit of the EFA solution, and (c) a second-order solution of the firstorder oblique model, to assess the intercorrelations among the factor solutions. We note that confirmation of the second-order solution would argue for deriving a composite PDI score. We used the Mplus 2.12 program in the analyses. Because the chi-square estimate is sensitive to large sample sizes, we identified five relative fit estimates to evaluate the fit of each model: (a) scaling correction (i.e., values of 2 or less), (b) comparative fit index (CFI; values of 0.95 or higher), (c) Tucker–Lewis index (i.e., TLI values
Table II. Confirmatory Factor Analyses for the Four-Factor Oblique Model: Parcel-Level Analyses Parcel PDI-somatic anxiety (PDI-SOM) 1. PDI-SOM1 2. PDI-SOM2 3. PDI-SOM3 PDI-depression (PDI-DEP) 4. PDI-DEP1 5. PDI-DEP2 6. PDI-DEP3 PDI-pain sensitivity (PDI-PSI) 7. PDI-PSI1 8. PDI-PSI2 9. PDI-PSI3 PDI-anger (PDI-ANG) 10. PDI-ANG1 11. PDI-ANG2 12. PDI-ANG3 Note. PDI = Pain Distress Inventory.
Cronbach’s alpha (95% CI)
Mean interitem (r)
Standardized loading (R2 )
1, 5, 7 2, 9 24, 25
0.70 (0.63–0.76) 0.73 (0.65–0.79) 0.62 (0.52–0.70)
0.44 0.58 0.45
0.754 (0.569) 0.767 (0.589) 0.662 (0.439)
4, 13, 18 22, 23 17, 19
0.82 (0.77–0.85) 0.73 (0.65–0.79) 0.82 (0.77–0.86)
0.60 0.58 0.70
0.830 (.688) 0.791 (.626) 0.869 (.756)
3, 11 15, 21 20, 26
0.79 (0.74–0.84) 0.78 (0.71–0.83) 0.84 (0.80–0.98)
0.66 0.64 0.72
0.808 (0.652) 0.809 (0.654) 0.883 (0.779)
6, 12 8, 10 14, 16
0.70 (0.61–0.76) 0.56 (0.43–0.65) 0.58 (0.46–0.67)
0.55 0.39 0.42
0.761 (0.579) 0.847 (0.718) 0.715 (0.512)
Parcel items
174
Osman et al. Table III. Goodness-of-Fit Estimates for the Pain Distress Inventory Models Model 1. Baseline 2. One-factor 3. Four-factor Oblique 4. Second-order
χ2 -R
df
χ2 −R/df a
CFI
TLI
RMSEA
SRMR
1,283.69 433.34 68.12 74.18
66 54 48 50
1.30 1.23 1.23
0.688 0.983 0.980
0.619 0.977 0.974
0.166 0.040 0.043
0.109 0.036 0.040
Note. χ2 -R: Robust, chi-square; CFI: comparative fit index; TLI: Tucker–Lewis Index; RMSEA: rootmean-square error of approximation; SRMR: standardized root-mean-square residual. a Scaling correction factor.
of 0.95 or higher), (d) root mean square error of approximation (RMSEA; values of 0.06 or less), and (e) standardized root mean square residual (SRMR; values of 0.08 or less) (see Hu and Bentler, 1999). The variances of the factors were fixed to 1.0 in the firstorder oblique model; the first loading on each firstorder factor was fixed to 1.0 in the analyses involving the second-order model. The MLM estimation method was used to perform each analysis. Results of the fit estimates for the analyses are presented in Table III. The one-factor model provided poor fit to the sample data. Both the oblique and the second-order models provided excellent fit estimates to the sample data. It should be noted that results of the second-order model provided strong support for deriving a total score for the PDI; result of the oblique model provided support for the four conceptual dimensions of this instrument.
Clark and Watson, 1995). Overall, the internal consistency reliability estimates for all the study measures were good. Coefficient alpha estimates for the PDI total and scale scores were comparable to those reported in Osman et al. (2003). The present study contributed to the assessment of internal consistency of the PDI by providing mean interitem correlations and the 95% CIs for the alpha estimates.
Convergent-Discriminant Validity We included a set of validational measures to assess evidence of convergent and discriminant validity. Scores on the pain measures (the PASS, PCS, and INTRP) were used, in part, as the convergent validity measures; scores on the psychological measures (the STAIT-T ST-DACL, and MASQ-90-GD) were used to examine evidence for discriminant validity. Convergent Validity. Pearson correlation analyses revealed moderate to high correlations between the PDI total score and scores on the pain-related measures (range, rs = 0.45 to 0.72). As expected, the correlation between the PDI and PASS scores were high and significant; both instruments are designed to assess distress related to pain, although the PASS includes several cognitive-related items (e.g., Item 10, I feel disoriented and confused when I hurt).
Descriptive Data and Internal Consistency Estimates for the Study Measures Table IV lists the means and standard deviations for the PDI and validation scales. The table also presents the internal consistency estimates. For research purposes, mean interitem correlation values of 0.15 or higher are considered adequate (see
Table IV. Intercorrelations of the Study Measures, Descriptive Statistics, and Reliability Estimates Measurea 1. PDI 2. PASS 3. PCS 4. INTRP 5. STAIT-T 6. ST-DACL 7. MASQ-GD
1
2
1.00 0.72 0.59 0.45 0.54 0.36 0.53
1.00 0.72 0.55 0.45 0.28 0.43
3
1.00 0.57 0.44 0.31 0.42
4
1.00 0.54 0.50 0.50
5
1.00 0.68 0.69
6
1.00 0.49
7
Mean (SD)
Coefficient alpha (95% CI)
Mean interitem r
1.00
0.97 (0.60) 54.27 (26.72) 12.11 (8.99) 1.53 (0.54) 40.64 (8.85) 6.06 (5.49) 30.05 (9.49)
0.93 (0.91–0.94) 0.94 (0.93–0.95) 0.93 (0.91–0.94) 0.93 (0.92–0.95) 0.88 (0.85–0.90) 0.87 (0.85–0.89) 0.87 (0.86–0.91)
.32 0.28 0.50 0.41 0.26 0.24 0.35
Note. PDI: Pain Distress Inventory; PASS: Pain Anxiety Symptoms Scale; PCS: Pain Catastrophizing Scale; INTRP: Inventory of Negative Thoughts in Response to Pain; STAIT-T: State-Trait Anxiety Inventory-Trait; ST-DACL: State Trait-Depression Adjective Check List; MASQ-GD: Mood and Anxiety Symptom Questionnaire-General Disturbance. a All p values are significant at the 0.01 level.
Validity of the PDI Discriminant Validity. We attempted to establish evidence of discriminant validity in two steps. First, we hypothesized that the correlations between the PDI and the PASS scores (pain-distress measures) would be higher than the correlations between the PDI and any of the cognitive pain-related measures (i.e., the PCS and INTRP). Results of the dependent correlation analyses showed that the correlation between the PDI and the PASS scores (r = 0.72) was significantly higher than the correlation between the PDI and (a) the PCS scores (r = 0.59), z = 3.89, p < 0.001; and (b) the INTRP scores (r = 0.45), z = 6.06, p < 0.001. Next, we hypothesized that the relation between the PDI and the PASS would be higher than the correlation between the PDI and any of the psychological distress measures. As expected, the correlation between the PDI and the PASS scores was higher than the correlation between the PDI and (a) the STAIT-T score (r = 0.54), z = 3.93, p < 0.001; (b) the ST-DACL score (r = 0.36), z = 6.36, p < 0.001; and (c) the MASQ-90-GD score (r = 53), z = 4.07, p < 0.001. Criterion-Related Validity We conducted several simultaneous multiple regression analyses to examine the contributions of the pain-related measures to the prediction of four pain indices: worst pain, current pain intensity/discomfort, frequency of physical pain distress (PSI), and pain interference (PII). Specifically, scores on the PDI, PASS, PCS, and INTRP were included simultaneously as predictors of the pain indexes. Results of the regression analyses are presented in Table V. In each model, the PDI total score was the only significant predictor of the target criterion index. These results provide support for scores on the PDI as having strong relationships with pain-related indexes such as frequency of pain interference and pain intensity in a nonclinical sample. STUDY II Method Study II was designed to examine ethnic group differences in the PDI total and scale scores. We included two subsamples, Caucasian and African American college-age students. As noted previously, the validation of most measures of pain-
175 Table V. Summary of Regression Analyses Predicting Pain Indexes sr2
Predictors
Standardized β
paina
Predicting worst PDI 0.277 0.403 PASS −0.038 −0.066 PCS −0.022 −0.034 INTRP 0.080 0.100 Predicting current painb PDI 0.122 0.178 PASS 0.006 0.010 PCS −0.061 −0.094 INTRP 0.094 0.118 Predicting physical pain distress indexc PDI 0.272 0.396 PASS 0.042 0.072 PCS −0.088 −0.134 INTRP 0.074 0.093 Predicting pain interference indexd PDI 0.333 0.485 PASS −0.031 −0.053 PCS −0.059 −0.091 INTRP 0.086 0.108
t 4.762∗∗ −0.659 −0.379 1.376 1.982∗ 0.095 −0.995 1.522 4.751∗∗ 0.738 −1.540 1.294 5.919∗∗ −0.546 −1.056 1.528
Note. PDI: Pain Distress Index; PASS: Pain Anxiety Symptoms Scale; PCS: Pain Catastrophizing Scale; INTRP: Inventory of Negative Thoughts in Response to Pain. a R2 = 0.153 Adjusted R2 = 0.139 F (4, 251) = 11.33 p < 0.001. bR2 = 0.044 Adjusted R2 = 0.029 F (4, 251) = 2.87 p < 0.024. c R2 = 0.179 Adjusted R2 = 0.166 F (4, 251) = 13.71 p < 0.001. d R2 = 0.206 Adjusted R2 = 0.194 F (4, 251) = 16.316 p < 0.001. ∗ p < 0.05. ∗∗ p < 0.01.
related behaviors have involved relatively small samples of other ethnic groups (e.g., Osman et al., 2003). Given the prevalence of pain-related symptoms in other ethnic groups, there is indeed a need for validating existing self-report instruments for use with other ethnic groups (see Turk and Melzack, 1992). Participants, Procedure, and Measures Participants were 325 undergraduate and graduate students recruited from a Midwestern university and two community colleges. They included 170 Caucasian (men = 52, women = 118) and 155 African American (men = 45, women = 110) students. The mean age of the Caucasian students was 22.29 years (SD = 4.75; range = 18–52 years) and the mean age of the African American students was 23.59 years (SD = 7.80; range = 18–52 years). The groups did not differ significantly in age, t(323) = 1.83, p = 0.07; gender composition, χ2 (1, N = 325) = 0.094, p = 0.76, or marital status, χ2 (5, N = 325) = 7.59, p = 0.18.
176 The background information questionnaire and the PDI that we used were the same as those described in Study I. The study was also approved by the university’s research review board. Each participant provided written consent before completing the study instruments. Results and Discussion Invariance of the Pain Distress Inventory Structure Across Ethnic Groups We conducted multigroup CFAs to evaluate invariance of factorial structure of the PDI across the Caucasian and the African American groups. To minimize error and report on strong solutions, we used parcels as items (see Study I) in each model. Byrne (1994) recommended several steps in evaluating measurement invariance. Briefly, we first evaluated the fit of baseline models in the separate Caucasian and African American samples. Next, we evaluated factor intercorrelation invariance, factor loadings invariance, and factor intercorrelation and factor loadings invariance, respectively. We used the Robust estimation procedure, and the following goodness-of fit estimates to evaluate the fit of each model: the scaling χ2 −R/df index of 2 or less, the CFI and TLI values of 0.95 or higher, and the RMSEA value of 0.060 or less. We found that the goodness-of-fit estimates for the oblique four-factor baseline models were adequate for the Caucasian [scaling index = 1.23, p = 0.13; CFI = 0.989; TLI = 0.984; RMSEA = 0.037 (90% CI = 0.000, 0.065)], and the African American [scaling index = 1.71, p < 0.01; CFI = 0.961; TLI = 0.946; RMSEA = 0.068 (90% CI = 0.041, 0.092)] samples. In evaluating factor intercorrelation invariance, the goodness-of-fit estimates met all the expected criteria, scaling index = 1.42, p < 0.01; CFI = 0.976; TLI = 0.969; RMSEA = 0.036 (90% CI = 0.021, 0.49). Similarly, the goodness-of-fit estimates for invariance of the factor loadings invariance test was adequate, scaling index = 1.41; CFI = 0.977; TLI = 0.971; RMSEA = 0.036 (90% CI = 0.021, 0.048). The final step involving factor intercorrelation and factor loadings invariance test met all the fit estimates, scaling index = 1.37; CFI = 0.978; TLI = 0.973; RMSEA = 0.034 (90% CI = 0.019, 0.46). Overall, we found that the PDI fourfactor solution was robust across the Caucasian and the African American study samples.
Osman et al. Ethnic Group Differences on the PDI Total Score To evaluate PDI score differences in gender and ethnicity, we conducted a univariate analysis of variance (ANOVA). The results showed a significant main effect for ethnicity only, [(F (1, 321) = 6.01, p < 0.015; partial η2 = 0.018]. The main effect for gender, as well as the gender by ethnicity interaction effect were not significant (all ps > 0.05). The Bonferroni post hoc test showed that the Caucasian students (M = 1.34, SD = 0.67) reported significantly higher PDI total scores than the African American students (M = 1.13, SD = 0.72), F (1, 323)7.71, p < 0.006, Cohen’s d = 0.30 (small effect). Ethnic Group Differences on the Four PDI Scale Scores To examine ethnic group differences in the PDI scale scores, we conducted a multivariate analysis of variance (MANOVA), using the four scale scores as dependent variables, and ethnicity and gender as independent variables. The Box’s Test was significant (Box’s M = 56.56, p < 0.004); thus, we used the Pillai’s Trace to interpret the MANOVA results. The main effect for ethnicity was highly significant, Pillai’s Trace = 0.43; F (4, 318) = 3.55, p < 0.008; partial η2 = 0.043. The Bonferroni’s post hoc comparison showed that the Caucasian students obtained significantly higher scores than the African American students on the PDI-Anger (Cohen’s d = .43) and the PDI-Somatic Anxiety (Cohen’s d = 0.29) scales (see Table VI for descriptive data). In addition, the main effect for gender was statistically significant, Pillai’s Trace = 0.034, F (4, 318) = 2.79, p < 0.026; partial η2 = 0.034. The follow-up Bonferroni’s test showed that women had higher scores than men on the PDI-Depression (Men, M = 1.70, SD = 0.93; Women, M = 1.91, SD = 0.91, Cohen’s d = 0.23) and the PDI-Somatic Anxiety (Men, M = 0.97, SD = 0.70; Women, M = 1.17, SD = 0.79, Cohen’s d = 0.26) scales. The ethnicity by gender interaction was not significant. Examination of the effect size estimates showed that the obtained group differences on the PDI scores were small. Internal Consistency Reliability Table VI shows the descriptive statistics (means and standard deviations) and internal consistency reliability estimates (i.e., coefficient alphas and related
Validity of the PDI
177
Table VI. Responses of African Americans and Caucasians on the PDI Total and Scale Scores African American (N = 155) PDI PDI-somatic anxiety PDI-depression PDI-pain sensitivity PDI-anger PDI-total scale
Caucasian (N = 170)
M (SD)
α (95% CI)
ra
1.00 (0.76) 1.76 (0.97) 0.88 (0.99) 0.79 (0.75) 1.13 (0.72)
0.83 (0.79–0.87) 0.90 (0.87–0.92) 0.92 (0.90–0.94) 0.87 (0.83–0.90) 0.94 (0.93–0.96)
0.42 0.55 0.66 0.53 0.39
M (SD)
α (95% CI)
ra
1.22 (0.76) 1.93 (0.87) 1.00 (0.84) 1.13 (0.82) 1.34 (0.67)
0.85 (0.81–0.88) 0.91 (0.89–0.93) 0.91 (0.89–0.93) 0.87 (0.84–0.90) 0.94 (0.93–0.95)
0.45 0.59 0.63 0.54 0.38
Note. PDI: Pain Distress Inventory. interitem correlations.
a Mean
95% Confidence Intervals, and the mean interitem correlations) of the PDI total and scale scores for the African American and Caucasian study samples. The alpha coefficients of the scales, for each group, exceeded the traditional 0.70 acceptable criterion. Additionally, the mean interitem correlations for the scales were greater than 0.30 for each group. Overall, all the internal consistency reliability estimates exceeded the traditional cutoff scores. STUDY III This study was conducted at a Midwestern State university to evaluate the ability of the PDI total score to differentiate between known-groups of self-reported pain-related illness conditions and an appropriate control group. We also attempted to replicate previous findings of reliability and criterionrelated validity (see Osman et al., 2003). As noted in the pain assessment literature, the reports of nonclinical participants with differing levels of pain-related responses are useful in validating the responses of individuals with more severe or chronic pain. Method Participants, Procedures, and Measures Two subgroups of participants were included in the present study: Self-reported pain and control groups. Self-Reported Pain Sample. Potential participants completed a 3-item screening instrument to provide information on (a) the number of health care visits in “the past 3 months” for severe pain-related medical illness, sickness, or injury condition(s); (b) level of suffering related to the specified medical illness condition(s), using a 6-point rating scale (1 = no suffering to 6 = extreme suffering); and (c)
the extent of pain interference with daily activities in “the past 3 months,” using a 6-point interference scale (1 = no interference to 6 = extreme interference). Of the 314 potential participants, 45 who reported two or more health care visits for severe painrelated health problems, and also obtained ratings of 2 (moderate) or higher on the suffering and interference screening items were interviewed further in our laboratory. All interviews were conducted by trained advanced undergraduate (n = 3) and graduate (n = 2) research assistants. The individual interviews focused on the re-administration of the initial three screening questionnaire items to establish consistency in self-reported status, as well as to obtain information on the times and dates of the self-reported hospital visits. All 45 participants were included in the selfreported pain group, based on consistency in the preand postresponses to the screening items. This sample was composed of 12 men (mean age = 19.58 years, SD = 1.31) and 33 women (mean age = 19.97 years, SD = 1.47). We note that because we relied only on each participant’s ratings on the screening items, we named this group “Self-Reported.” Symptom-Free Control Sample. The control sample, matched by gender, also included 12 men (mean age = 19.58 years, SD = 2.23) and 33 women (mean age = 19.94 years, SD = 1.17) drawn from the same pool of volunteers. All participants were recruited from undergraduate and graduate courses. Specifically, students who (a) had not sought health care services for severe medical conditions in the “past 3 months;” (b) had ratings less than 2 on the suffering and interference screening items; and (c) were currently not using prescribed pain medication were included in the control group. The control group and the self-reported pain group did not differ in age, t(88) = 0.07, ns. Additional chi-square analyses on the ethnicity and marital status variables
178 also did not show differences between the groups (all ps > 0.05). The self-reported and control participating students were administered the background information questionnaire used in Study I, the PDI, and the PCS (Sullivan et al., 1995) instruments in small groups in our laboratory within 2 weeks of the screening. All participants received research participation credit, consistent with the research protocol approved by the university’s review board. Results and Discussion Known-Groups Validity To evaluate the ability of the PDI total score to differentiate significantly the self-reported pain from the control groups recruited from the same population, we conducted a one-way analysis of variance (ANOVA) using the mean PDI total score as the dependent variable. The ANOVA result showed that the self-reported pain group (M = 1.77, SD = 0.80) reported higher PDI total scale scores than the control (M = 1.14, SD = 0.63) group, F (1, 88) = 17.47, p < 0.001; η2 = 0.16 (large effect). To examine further group differences on the PDI scale scores, we conducted a one-way multivariate analysis of variance (MANOVA), using the PDI subscales as dependent variables and group as the independent variable. Results showed that the overall MANOVA was significant, Pillai’s Trace = 0.23, p < 0.001. Using the Bonferroni’s adjustment procedure (0.05/4), we found that the self-reported pain group reported higher scores on the PDI-Somatic Anxiety (M = 1.59, SD = 0.97 vs. M = 0.97, SD = 0.63; effect size d = 0.75), the PDI-Depression (M = 2.49, SD = 0.88 vs. M = 1.67, SD = 0.86; effect size d = 0.94), and the PDI-Pain Sensitivity (M = 1.39, SD = 1.10 vs. M = 0.68, SD = 0.84; effect size d = 0.72) scales than the control group. The groups did not differ significantly in their responses on the PDIAnger scale. Overall, the obtained effect size estimates ranged from medium to large, suggesting meaningful differences between the groups recruited to be marginally different on the PDI. Reliability Analyses Coefficient alpha estimates for the PDI total scale were 0.94 (95% CI = 0.91 − 0.96; mean interitem r = 0.37) for the self-reported pain group,
Osman et al. and 0.92 (95% CI = 0.88 − 0.95; mean interitem r = 0.33) for the control group. Analyses at the scale level also showed that each of the four PDI scales had adequate internal consistency reliability estimates for the self-reported pain group (coefficient alpha, range = 0.81–0.92) and the control group (coefficient alpha, range = 0.77 to 0.96). Criterion-Related Validity We conducted a forward logistic regression analysis to determine whether scores on the PDI and PCS would be useful in differentiating between the self-reported pain group (coded as 1) and the control group (coded as 0). The PCS and PDI scores were entered simultaneously into the equation. However, only scores on the PDI total scale score were useful in differentiating between the groups (standardized β = 1.10, SE = 0.43; odds ratio = 2.99, 95% CI = 1.30, 6.90). GENERAL DISCUSSION The major objectives of the present investigations were to examine evidence of internal consistency and construct validity for the PDI (Osman et al., 2003). Unlike current self-report measures of psychological distress, this new instrument is designed specifically to assess distress related to pain. These four dimensions are as follows: paindepression, pain-sensitivity, pain-somatic anxiety, and pain-anger. We decided to limit the general discussion to the major findings and methods of the present and prior research with the PDI for two reasons. First, by using a pain-specific (i.e., PDI) rather than a general psychological distress measure (e.g., BDI), we found it difficult to make comparisons of our findings with findings of investigations that used psychological distress measures. Second, our samples were nonclinical, a major limitation with studies with the PDI. In the present studies, we were interested in evaluating the responses of a heterogeneous sample of nonclinical adults with differing levels of selfreported pain on the PDI. The results of our reliability analyses across the studies showed that scores on the PDI have excellent estimates of internal consistency. A contribution of the present studies is that we used contemporary estimation procedures to evaluate evidence of internal consistency reliability for this new instrument. We found that the coefficient alpha estimates, in these
Validity of the PDI studies, exceeded the traditional 0.70 cutoff score. In addition, the contemporary mean interitem correlations exceeded the expected cut off score of 0.15, suggesting that the PDI is adequate for use in research settings. Overall, these results provide strong support of the findings of excellent internal consistency of the PDI scores reported by Osman and colleagues from the instrument development samples. The analyses that were conducted to evaluate evidence of factorial validity for the PDI provided strong evidence for the replicability of the fourfactor structure of this instrument. At the item-level analysis, only one item (Item 24) loaded less than the prespecified 0.40 item retention criterion. As in Osman et al. (2003), evidence for the second-order structure of the PDI also was strong, using parcels as items in Study I. Thus, the rationale for deriving a total PDI score was supported by the adequacy of fit for the second-order solution. Furthermore, the present investigations contributed to the PDI literature by evaluating invariance of the fourfactor solution across African American and Caucasian nonclinical samples (Study II). The presence of ethnic group differences on two of the four PDI scale scores suggests the need for evaluating ethnic group responses on our measures of pain-related responses. Future investigations are indicated to explore further (a) ethnic group differences on the PDI items, and (b) invariance of the PDI structure across gender. Results of the convergent-discriminant validity and criterion-related validity analyses also contributed support for the construct validity of the PDI. The findings reported in Studies I and III are similar to findings in Osman et al. (2003), who found that scores on the PDI are useful in predicting scores on measures of pain-related responses. In Study I, the PDI scores performed well in correlating more with other measures of pain-related responses than with scores on psychological distress measures. In addition, when compared with other self-report measures of pain, only scores on the PDI made significant contributions to the prediction of scores on pain-related responses. Future research should evaluate the relative predictive validity of the PDI in clinical and other nonclinical populations. In the analyses involving the self-reported and control groups, scores on the PDI were also useful in differentiating between participants from similar populations with differing degrees of pain-related distress. Additional studies should be conducted to replicate these findings within clinic samples. For ex-
179 ample, in validating scores on the Fear of Pain Questionnaire (FPQ-III), McNeil and Rainwater (1998) included adults with chronic pain and a clinic control sample from a medical setting. Descriptive data (i.e., means and standard deviations) reported in Study III would be useful in validating scores obtained from other clinic and nonclinical settings. A limitation of the present studies (specifically, Studies I and II) is that validation of the PDI relied solely on self-reported questionnaire data. We recommend that future investigations with the PDI include other methods for maximizing internalizing responses in nonclinical samples. Despite this and other limitations such as the use of nonclinical samples, results of these studies show that scores on the PDI were very reliable in these samples. In addition, evidence for construct validity including factorial validity, convergent validity, and criterion-related validity were all very strong. We invite replications of these findings. ACKNOWLEDGMENT We thank Elizabeth M. O’Neill and Kimberly J. King for their assistance with data collection for Studies I and II. REFERENCES Barbarin, O. A., and Christian, M. (1999). The social and cultural context of coping with sickle cell disease: I. A review of biomedical and psychosocial issues. J. Black Psychol. 25: 277–293. Barlow, D. H. (1981). On the relation of clinical research to clinical practice: Current issues, new direction. J. Consult. Clin. Psychol. 49: 147–155. Beck, A. T., Steer, R. A., and Brown, G. K. (1996). Manual for the Depression Inventory, 2nd ed., Psychological Corporation, Texas. Byrne, B. M. (1994). Structural Equation Modeling With EQS and EQS/Windows: Basic Concepts, Applications, and Programming. Sage, Thousand Oaks, CA. Clark, L. A., and Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychol. Assess. 7: 309– 319. Derogatis, L. R. (1994). Symptom Checklist 90-R: Administration, Scoring, and Procedures Manual. National Computing Services, Minnesota. DeVellis, R. F. (2003). Scale Development: Theory and applications, 2nd ed., Sage, Thousand Oaks, CA. Floyd, F. J., and Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychol. Assess. 7: 286–299. Gil, K. M., Williams, D. A., Keefe, F. J., and Beckham, J. C. (1990). The relationship of negative thoughts to pain and psychological distress. Behav. Ther. 21: 349–362. Greenwald, H. P. (1991). Interethnic differences in pain perception. Pain 44: 157–163.
180 Haynes, S. N., Richard, D. C. S., and Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychol. Assess. 7: 238–247. Hu, L., and Bentler, P. M. (1999). Cutoff criteria for fit in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Eq. Model. 6: 1–55. Jensen, M. P., Turner, J. A., and Romano, J. M. (2000). Pain belief assessment: A comparison of the short and long versions of the Survey of Pain Attitudes. J. Pain. 1: 138–150. Lubin, B. (1994). State Trait—Depression Adjective Check Lists: ST-DACL (Form F). Psychological Assessment Resources, Odessa, FL. McCracken, L. M., Zayfert, C., and Gross, R. T. (1992). The Pain Anxiety Symptoms Scale (PASS): Development and validation of a scale to mea sure fear of pain. Pain 50: 67–73. McNeil, D. W., and Rainwater, A. J., III (1998). Development of the Fear of Pain Questionnaire—III. J. Behav. Med. 21: 389– 410. ´ L. M., and Muthen, ´ B. O. (2003). Mplus User’s Guide, Muthen, ´ & Muthen, ´ Los Angeles. Muthen O’Maria, A. M., and Arenella, C. (2001). Minority representation, prevalence of symptoms, and utilization of services in a large metropolitan hospice. J. Pain Sympt. Manage. 21: 290–297. Osman, A., Barrios, F. X., Gutierrez, P. M., Kopper, B. A., Butler, A., and Bagge, C. L. (2003). The Pain Distress Inventory: Development and initial psychometric properties. J. Clin. Psychol. 59: 767–785. Osman, A., Barrios, F. X., Gutierrez, P. M., Kopper, B. A., Merrifield, T., and Grittmann, L. (2000). The Pain Catastrophizing Scale: Further psychometric evaluation with adult samples. J. Behav. Med. 23: 351–365. Osman, A., Barrios, F. X., Kopper, B. A., Hauptmann, W., Jones, J., and O’Neill, E. (1997). Factor structure, reliability, and
Osman et al. validity of the Pain Catastrophizing Scale. J. Behav. Med. 10: 263–276. Osman, A., Barrios, F. X., Osman, J. R., Schneekloth, R., and Troutman, J. A. (1994). The Pain Anxiety Symptoms Scale: Psychometric properties in a community sample. J. Behav. Med. 17: 511–522. Osman, A., Breirenstein, J. L., Barrios, F. X., Gutierrez, P. M., and Kopper, B. A. (2002). The Fear of Pain Questionnaire— III: Further reliability and validity with non-clinical samples. J. Behav. Med. 25:155–173. Osman, A., Bunger, S., Osman, J. R., and Fisher, L. (1993). The Inventory of Negative Thoughts in Response to Pain: Factor structure and psychometric properties in a college sample. J. Behav. Med. 16: 219–224. Riley, J. L., III, Wade, J. B., Myers, C. D., Sheffield, D., Papas, R. K., and Price, D. D. (2002). Racial/ethnic differences in the experience of chronic pain. Pain 100: 291–298. Spielberger, C. D., Gorsuch, R. C., and Lushene, R. E. (1983). Manual for the State-Trait Anxiety Inventory (Form Y), Consulting Psychologists, Palo Alto, CA. Sullivan, M. J. L., Bishop, S. C., and Pivik, J. (1995). The Pain Catastrophizing Scale: Development and validation. Psychol. Assess. 7: 524–532. Turk, D. C., and Melzack, R. (1992). The measurement of pain and the assessment of people experiencing pain. In Turk, D. C., and Melzack, R. (Eds.), Handbook of pain assessment New York: Guilford Press, pp. 3–12, 233–238. Watson, D., and Clark, L. A. (1991). The Mood and Anxiety Symptom Questionnaire-90 (MASQ-90). Unpublished manuscript, University of Iowa, Iowa City. Zwick, W. R., and Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychol. Bull. 99: 432–442.
