(TRIN) Scales

ASSESSMENT

10.1177/1073191105284453 Handel et al. / MMPI-2 AND MMPI-A TRIN SCALES

An Evaluation of the MMPI-2 and MMPI-A True Response Inconsistency (TRIN) Scales Richard W. Handel Eastern Virginia Medical School

Randolph C. Arnau University of Southern Mississippi

Robert P. Archer Eastern Virginia Medical School

Kristina L. Dandy Texas Christian University The Minnesota Multiphasic Personality Inventory–Adolescent (MMPI-A) and Minnesota Multiphasic Personality Inventory–2 (MMPI-2) True Response Inconsistency (TRIN) scales are measures of acquiescence and nonacquiescence included among the standard validity scales on these instruments. The goals of this study were to evaluate the effectiveness of these scales in detecting varying degrees of acquiescence and nonacquiescence and to evaluate cutoff scores for clinical use. After the removal of invalid protocols from the MMPI-2 and MMPI-A normative samples, each normative sample was randomly divided in half. For each measure, one half of the normative sample served as a comparison group and the other half was modified with increasing degrees (10%, 20%, 30%, 40%, and 50%) of randomly inserted true or false responses. The results for a 9.1% base rate of acquiescence or nonacquiescence provide support for TRIN cutoff scores at or near those presented in the MMPI-A and MMPI-2 manuals. Keywords: personality assessment; MMPI-2; MMPI-A; validity scales; TRIN The Minnesota Multiphasic Personality Inventory–2 (MMPI-2; Butcher et al., 2001) and Minnesota Multiphasic Personality Inventory–Adolescent (MMPI-A; Butcher et al., 1992) include an array of validity scales designed to evaluate the test-taker’s response style. The MMPI (Hathaway & McKinley, 1943) originally included three validity scales (i.e., L, F, and Cannot Say) and the K (correction) scale was added later (McKinley, Hathaway, & Meehl, 1948). These validity scales were augmented with several new indices with the advent of the MMPI-2. Two measures of inconsistent responding were developed

for the MMPI-2 based on the work of Auke Tellegen (1982, 1988). The Variable Response Inconsistency (VRIN) scale consists of pairs of items that are either similar or opposite in content, and the total raw score for this scale is calculated by summing the total number of pairs answered inconsistently (Butcher et al., 2001). Therefore, the VRIN scale is an index of intentional or unintentional random responding. In contrast, the True Response Inconsistency (TRIN) scale consists entirely of pairs of items that are opposite in content. For this scale, excessive inconsistent responding (i.e., answering multiple pairs of

We thank the University of Minnesota Press for permission to use the MMPI-2 and MMPI-A normative samples in this study. Correspondence concerning this article should be addressed to Richard W. Handel, Eastern Virginia Medical School, Department of Psychiatry and Behavioral Sciences, 825 Fairfax Avenue, Norfolk, VA 23507; e-mail: [email protected]. Assessment, Volume 13, No. 1, March 2006 98-106 DOI: 10.1177/1073191105284453 © 2006 Sage Publications

Downloaded from asm.sagepub.com at Eastern Virginia Medical School Library on March 3, 2015

Handel et al. / MMPI-2 AND MMPI-A TRIN SCALES 99

items as both true or both false) indicates either an acquiescent (“yay-saying”) or a nonacquiescent (“nay-saying”) response set. The two forms of inconsistency measured by the VRIN and TRIN scales are virtually uncorrelated but are not independent. An extreme score on TRIN precludes a high VRIN score and vice versa. Although empirical studies have investigated the MMPI-2 VRIN scale (e.g., Berry et al., 1991) and the MMPI-A VRIN scale (e.g., Archer, Handel, Lynch, & Elkins, 2002; Baer, Kroll, Rinaldo, & Ballenger, 1999), little research has been conducted on the TRIN scale for the MMPI-2, and no studies have investigated the MMPI-A’s version of the scale. Jackson and Messick conducted a series of investigations during the late 1950s and 1960s on the potentially pervasive influence of response styles such as acquiescence on MMPI response patterns (e.g., Jackson & Messick, 1961). As previously indicated, the work of Tellegen was highly influential in the development of inconsistency scales for the MMPI-2 capable of assessing acquiescence and nay-saying tendencies. Tellegen’s (1982) Multidimensional Personality Questionnaire (MPQ) includes a TRIN scale that consists of 27 item pairs such that “the members of each pair are similar in content if scored in opposite directions (true-false or false-true)” (Tellegen, 1988, p. 631). Tellegen (1988) noted that the MPQ’s TRIN raw score is calculated by simply totaling all of the items answered true. High scores are indicative of indiscriminate true responding, whereas low scores suggest indiscriminate false responding. There are some significant differences between the MPQ’s TRIN scale and its sister scales on the MMPI-2 and MMPI-A. The MMPI-2 TRIN scale consists of 20 item pairs; but of the 40 possible true-true and false-false response pairs, only 14 true-true and 9 false-false response pairs are scored as inconsistencies. The MMPI-2 TRIN raw score is the total number of inconsistent true responses minus the total number of inconsistent false responses plus 9. The value of 9 is added to the total raw score to avoid negative values. Raw scores less than 9 are indicative of some degree of inconsistent false responding, whereas scores greater than 9 suggest inconsistent true responding. The MMPI-2 TRIN raw scores are first converted to linear T-scores, and the T-scores below 50 are then “reflected” (e.g., an initial T-score of 29 would be converted to a Tscore of 71). As a result, the minimum possible T-score is 50 for men and women. The direction of inconsistent responding on TRIN is designated by a “T” or an “F” following the T-score value (e.g., T-score = 71F in the earlier example). According to the MMPI-2 manual (Butcher et al., 2001), a T-score ³ 80 suggests either an acquiescent or nonacquiescent response set depending on the direction (i.e., T or F) of the score. The MMPI-A TRIN scale is very similar to its MMPI-2 counterpart, with a few minor dif-

ferences. The MMPI-A TRIN scale includes 21 rather than 20 item pairs and a total of 24 rather than 23 item response pairs that are counted as inconsistencies. For the MMPI-A TRIN scale, the minimum possible T-score is 51F for boys and 53F for girls. Finally, the recommended cutoff score for detecting an acquiescent or nonacquiescent response set is T ³ 75 according to the MMPI-A manual (Butcher et al., 1992). TRIN and its counterpart VRIN are particularly useful when one is attempting to determine the possible reasons for elevations on other MMPI-2 or MMPI-A validity scales. For example, on the MMPI-2, F (Infrequency), Fb (Back F), Fp (Infrequency-Psychopathology), L (Lie), K (Correction), and S (Superlative) can all be extremely elevated due to various forms of inconsistent responding. By incorporating TRIN scores into a comprehensive profile validity analysis, the probability of inaccurate validity interpretations can potentially be reduced. We are aware of only one unpublished study on the MMPI-2 TRIN scale and of no studies on the MMPI-A. Wetter and Tharpe (1995) randomly inserted varying degrees of true or false responses into 50 existing MMPI-2 protocols produced by normals, compared these protocols to those produced by 50 normal adult participants, and concluded that TRIN makes a significant contribution to the detection of acquiescent and nonacquiescent response styles on the MMPI-2. The purpose of the present study was to examine the usefulness of the MMPI-2 and MMPI-A TRIN scales in detecting varying degrees of inconsistent true and false responding using the normative samples for these instruments. Furthermore, practical suggestions regarding possible cutoff scores for these inconsistency scales are presented.

METHOD Participants Data analyses for this study were based on the MMPI-2 and MMPI-A normative samples. Butcher et al. (2001) and Graham (2006) provided detailed descriptions of the MMPI-2 normative sample. Briefly, this is a sample of 2,600 individuals (1,138 men and 1,462 women) with a mean age of 41.04 years (SD = 15.29) and 14.72 years of education (SD = 2.60). The sample is 81% Caucasian, 12% African American, 3% Hispanic, 3% Native American, and 1% Asian American (Graham, 2006). Cases were removed from the MMPI-2 normative sample using the recommended validity scale cutoff scores for nonclinical settings listed in the MMPI-2 manual (Butcher et al., 2001). Because TRIN was the scale under investigation in the present study, no cases were removed based on TRIN


100

ASSESSMENT

scores. The application of validity criteria Cannot Say (CNS) < 30, VRIN T-score < 80, F T-score < 80, FB T-score < 90, FP T-score < 100, L T-score < 80, S T-score < 75, and K T-score < 75 resulted in a final sample of 2,495 (1,093 men and 1,402 women). The MMPI-A normative sample is described in detail in Butcher et al. (1992) and Archer (2005). The sample consists of 1,620 adolescents (805 boys and 815 girls) between the ages of 14 and 18. The mean age for boys is 15.54 years (SD = 1.17), and the mean age for girls is 15.60 years (SD = 1.18). The grade level for boys and girls in the normative sample ranges from 7th through 12th (boys M = 10.05, SD = 1.14; girls M = 10.11, SD = 1.19). The MMPIA normative sample is 76% Caucasian, 12% African American, 3% Asian American, 3% Native American, 2% Hispanic, and 4% other or not reported. For the MMPI-A, we selected validity criteria that mirrored the nonclinical setting recommendations for the MMPI-2. Therefore, validity criteria included CNS < 30, L < 80, F < 80, K < 75, and VRIN < 80. The final MMPI-A sample after the removal of invalids consisted of 1,556 (779 boys and 777 girls) protocols. Procedure The MMPI-A sample of 1,556 was randomly divided in half, and the first subsample served as a comparison group for all MMPI-A analyses. In the second subsample, increasing levels of true responses were randomly inserted at the individual protocol level. First, 10% of the items for each individual protocol were randomly selected and modified to all true responses. Next, 20% of the items were randomly selected and forced to all true responses. An identical procedure was followed for the 30%, 40%, and 50% true response insertion conditions. The modification of protocols at the individual level makes it highly unlikely that any two protocols would have an identical set of modified items. The unmodified protocols that served as the comparison group were then copied nine times to create a 10:1 ratio of unmodified to modified protocols. The purpose of this last step was to simulate a base rate of “yaysaying” or “nay-saying” that was more realistic than 50%. Hit rate, positive predictive power (PPP), negative predictive power (NPP), sensitivity, and specificity for various TRIN cutoff scores were calculated for both the 1:1 and 10:1 conditions of unmodified to modified protocols. For the second phase of the procedure, new sets of items were randomly selected at the individual protocol level in the same manner (i.e., 10%, 20%, 30%, 40%, and 50%) but replaced with “false” rather than “true” responses. Finally, the entire true and false response insertion procedures outlined above for the MMPI-A were repeated for the MMPI-2 data.

TABLE 1 Minnesota Multiphasic Personality Inventory– Adolescent Means and Standard Deviations for True Response Inconsistency (TRIN) Raw Scores, T-Test Results, and Cohen’s d for Varying Degrees of True Response Insertion TRIN

Comparison group 10% True response insertion 20% True response insertion 30% True response insertion 40% True response insertion 50% True response insertion

M

SD

9.40 10.61 11.76 13.08 14.41 16.00

1.62 1.84 1.91 1.99 2.04 2.04

t Cohen’s (df =1,554) d

13.80* 26.29* 40.02* 53.79* 71.00*

.70 1.33 2.03 2.73 3.60

*p < .001.

RESULTS MMPI-A True response insertion. Table 1 provides mean MMPIA raw scores, standard deviations, t-test results, and effect sizes for the varying degrees of true response insertion. Analyses were conducted on TRIN raw scores (rather than T-scores) because, as indicated earlier, TRIN T-scores are “reflected.” Prior to implementation of the true response insertion procedure, the mean TRIN raw score for the second MMPI-A subsample was 9.25 (SD = 1.63), and as shown in Table 1, the mean TRIN raw score for the comparison group was 9.40 (SD = 1.62). Table 1 outlines the effects of inserting increasing degrees of true responses. T tests were conducted and effect sizes were calculated using the comparison group as a benchmark for all conditions. As shown in Table 1, TRIN raw scores increased consistently with increasing degrees of true response insertion. With the exception of the 10% true response insertion condition, all effect sizes are in excess of d = .80, Cohen’s (1988) criterion for a large effect size. As a reference point, mean TRIN raw scores exceeded the recommended T-score cutoff of 75 in the MMPI-A manual (Butcher et al., 1992) in the 40% condition (for boys a raw score of 14 = 77T and for girls a raw score of 14 = 79T). False response insertion. MMPI-A results for the false response insertion procedure are shown in Table 2. As expected, mean raw scores decreased with increasing degrees of false response insertion. Here, too, effect sizes are quite large according to Cohen’s (1988) criteria, with the lone exception being a moderate effect size for the 10% false response insertion condition. According to the MMPI-A manual, a raw score of 4 corresponds to a T-score of 79F for boys, and a raw score of 5 results in a T-score of



TABLE 2 Minnesota Multiphasic Personality Inventory– Adolescent Means and Standard Deviations for True Response Inconsistency (TRIN) Raw Scores, T-Test Results, and Cohen’s d for Varying Degrees of False Response Insertion

TABLE 3 Minnesota Multiphasic Personality Inventory–2 Means and Standard Deviations for True Response Inconsistency (TRIN) Raw Scores, T-Test Results, and Cohen’s d for Varying Degrees of True Response Insertion

TRIN

Comparison group 10% False response insertion 20% False response insertion 30% False response insertion 40% False response insertion 50% False response insertion

M

SD

9.40 8.36 7.51 6.62 5.67 4.77

1.62 1.67 1.68 1.73 1.67 1.71

TRIN t Cohen’s (df = 1,554) d

12.47* 22.63* 32.74* 44.71* 54.83*

.63 1.15 1.66 2.27 2.78

*p < .001.

Comparison group 10% True response insertion 20% True response insertion 30% True response insertion 40% True response insertion 50% True response insertion

M

SD

9.00 10.18 11.41 12.63 14.03 15.39

1.32 1.57 1.70 1.88 1.96 2.01

t Cohen’s (df = 2,493) d

20.19* 39.52* 55.63* 75.03* 93.87*

.81 1.58 2.23 3.01 3.76

*p < .001.

78F for girls. For the false response insertion procedure, the mean TRIN raw score reached a value less than 5 in the 50% false response insertion condition. MMPI-2 True response insertion. Table 3 outlines the corresponding results for the MMPI-2. Prior to modification, the mean TRIN raw score in the second subsample (i.e., the sample that was modified) was 8.93 (SD = 1.27). TRIN raw scores increased with increasing degrees of true response insertion, and the effect sizes in each condition were large. Similar to the findings for the MMPI-A, the mean raw score exceeded the recommended cutoff value in the MMPI-2 manual (Butcher et al., 2001) in the 40% true response insertion condition. A raw score of 14 corresponds to a T-score of 86T for men, and a raw score of 13 equals a T-score of 80T for women. False response insertion. MMPI-2 results for the false response insertion procedure are presented in Table 4. Similar to the results for the MMPI-A, effect sizes were large according to Cohen’s definitions, with the exception of a moderate effect size in the 10% false response insertion condition. For the MMPI-2, a T-score value of 80 is reached or exceeded at a raw score of 4 for men (T-score = 85F) and 5 for women (T-score = 80F). Classification Accuracy Results MMPI-A. Table 5 outlines hit rate, PPP, NPP, sensitivity, and specificity for the MMPI-A true response insertion procedure at various cutting scores. For all MMPI-A and MMPI-2 classification analyses, any case in the comparison group with a T-score elevation above the designated cutting score (ignoring whether the elevation suggested an acquiescent vs. a nonacquiescent response style) was con-

TABLE 4 Minnesota Multiphasic Personality Inventory–2 Means and Standard Deviations for True Response Inconsistency (TRIN) Raw Scores, T-Test Results, and Cohen’s d for Varying Degrees of False Response Insertion TRIN

Comparison group 10% False response insertion 20% False response insertion 30% False response insertion 40% False response insertion 50% False response insertion

M

SD

9.00 8.13 7.25 6.43 5.52 4.67

1.32 1.39 1.48 1.54 1.59 1.59

t Cohen’s (df = 2,493) d

16.06* 31.12* 44.63* 59.49* 74.13*

.64 1.25 1.79 2.38 2.97

*p < .001.

sidered a false positive. For the 1:1 ratio condition, PPP values were generally high. Even in the 10% true response insertion condition, the minimum PPP value was .67 at a cutoff score of T ³ 65. Once true response insertion reached a level equal to or greater than 30%, all PPP values were in excess of .90 at all cutting scores greater than or equal to a T-score of 70. NPP values also increased with increasing degrees of true response insertion across conditions and decreased with increasing cutting scores within condition. For the 10:1 ratio condition, PPP values were generally modest in the 10% and 20% true response insertion conditions but consistently began to exceed chance classification levels at T-scores ³ 75 in the 30% true response insertion condition. In the 40% and 50% true response insertion conditions, PPP values reached or exceeded .75 at cutting scores of T ³ 75. Sensitivity values were fairly modest at all but the lowest cutting scores until reaching the 40% true insertion condition. Specificity values were equal to or greater than .85 at all cutting scores.


102

ASSESSMENT

TABLE 5 Hit Rate, Positive Predictive Power (PPP), Negative Predictive Power (NPP), Sensitivity (SEN), and Specificity (SPE) for the Minnesota Multiphasic Personality Inventory–Adolescent True Response Inconsistency (TRIN) Scale at Five Cutting Scores With Varying Degrees of True Response Insertion Hit Rate True Response Insertion Percentage

Cut Score (T ³)

PPP

NPP

10:1

1:1

10:1

1:1

10:1

1:1

SEN

SPE

10

65 70 75 80 90

.80 .87 .89 .90 .91

.57 .54 .52 .51 .50

.17 .20 .22 .25 .41

.67 .72 .74 .77 .88

.92 .92 .91 .91 .91

.55 .52 .51 .51 .50

.29 .14 .06 .03 .01

.85 .94 .98 .99 1.0

20

65 70 75 80 90

.82 .89 .91 .91 .91

.70 .64 .58 .53 .51

.27 .37 .44 .45 .63

.79 .85 .89 .89 .94

.95 .93 .92 .92 .91

.65 .59 .54 .52 .51

.54 .33 .17 .08 .02

.85 .94 .98 .99 1.0

30

65 70 75 80 90

.85 .91 .93 .92 .92

.82 .78 .69 .61 .54

.35 .52 .65 .72 .87

.84 .92 .95 .96 .99

.98 .96 .94 .93 .92

.80 .71 .62 .56 .52

.78 .61 .41 .23 .08

.85 .94 .98 .99 1.0

40

65 70 75 80 90

.86 .93 .95 .94 .93

.89 .89 .82 .73 .61

.39 .59 .75 .84 .95

.86 .94 .97 .98 .99

.99 .98 .97 .95 .93

.93 .85 .75 .65 .56

.93 .83 .67 .47 .23

.85 .94 .98 .99 1.0

50

65 70 75 80 90

.87 .94 .97 .97 .95

.92 .95 .94 .88 .76

.40 .63 .81 .89 .98

.87 .94 .98 .99 1.0

1.0 1.0 .99 .98 .95

.99 .96 .91 .81 .67

.99 .96 .90 .76 .52

.85 .94 .98 .99 1.0

Because the same comparison group was used for all conditions, specificity values did not change between conditions. Finally, hit rate (which like PPP and NPP is heavily influenced by base rate) is relatively high in all conditions and at all cutting scores in the 10:1 ratio condition due to the large number of protocols in the comparison group and relatively modest at cutting scores in the 1:1 ratio condition until reaching the 40% true insertion condition. Table 6 provides similar results for the MMPI-A using the false response insertion procedure. Here, PPP values were again generally high in the 1:1 ratio condition. For PPP findings in the 10:1 ratio condition, a higher degree of false response insertion (i.e., 40%) was required to begin to improve on chance classification at most cutting scores. NPP values were generally slightly more modest than their true response insertion counterparts in the 1:1 ratio condition. The findings for sensitivity are similar to those observed for the true response insertion procedure except these too are more modest than their true response insertion counterparts. Specificity values are identical to those

presented in Table 5 because we used the same comparison group for these analyses. MMPI-2. True response insertion findings for the MMPI-2 are presented in Table 7. PPP values were generally quite high in the 1:1 ratio condition and were comparable to the results for the MMPI-A true response insertion procedure. Similar to findings for the MMPI-A, NPP values declined as cutting scores were increased within conditions but increased for identical cutting scores as greater degrees of true responses were added to the protocols. For the 10:1 condition, PPP values climbed substantially above .50 for TRIN cutting scores greater than or equal to 75 even with a modest (i.e., 20%) degree of true response insertion. Similar to findings for the MMPI-A, NPP values were consistently high in all conditions (greater than .90). Sensitivity values were generally slightly greater than corresponding values for the MMPI-A, whereas specificity values were comparable to those found for the MMPI-A. Finally, Table 8 shows the results of the false response insertion procedure for the MMPI-2. In general, PPP re-



TABLE 6 Hit Rate, Positive Predictive Power (PPP), Negative Predictive Power (NPP), Sensitivity (SEN), and Specificity (SPE) for the Minnesota Multiphasic Personality Inventory–Adolescent True Response Inconsistency Scale at Five Cutting Scores With Varying Degrees of False Response Insertion Hit Rate False Response Insertion Percentage

Cut Score (T ³)

PPP

NPP

10:1

1:1

10:1

1:1

10:1

1:1

SEN

SPE

10

65 70 75 80 90

.80 .87 .89 .90 .91

.54 .51 .51 .50 .50

.13 .13 .13 .07 .09

.60 .60 .60 .42 .50

.92 .91 .91 .91 .91

.52 .51 .50 .50 .50

.22 .09 .03 .01 .00

.85 .94 .98 .99 1.0

20

65 70 75 80 90

.81 .87 .90 .90 .91

.62 .56 .53 .51 .50

.21 .25 .25 .21 .23

.72 .77 .77 .73 .75

.93 .92 .91 .91 .91

.58 .54 .51 .50 .50

.38 .19 .07 .02 .00

.85 .94 .98 .99 1.0

30

65 70 75 80 90

.83 .89 .90 .91 .91

.72 .64 .58 .53 .51

.29 .37 .44 .42 .60

.80 .86 .89 .88 .94

.96 .93 .92 .91 .91

.68 .59 .54 .51 .51

.60 .33 .17 .06 .02

.85 .94 .98 .99 1.0

40

65 70 75 80 90

.85 .91 .92 .91 .91

.82 .76 .67 .57 .53

.35 .51 .62 .62 .81

.84 .91 .94 .94 .98

.98 .96 .94 .92 .91

.80 .69 .60 .54 .51

.78 .58 .36 .15 .06

.85 .94 .98 .99 1.0

50

65 70 75 80 90

.86 .93 .94 .93 .92

.88 .85 .76 .66 .58

.38 .57 .71 .79 .92

.86 .93 .96 .97 .99

.99 .98 .96 .94 .92

.91 .79 .68 .60 .54

.91 .75 .55 .33 .15

.85 .94 .98 .99 1.0

sults for the 1:1 condition were roughly comparable to findings from the MMPI-A false response insertion procedure. However, PPP values were slightly greater for the MMPI-2, particularly for small degrees (i.e., 10% and 20%) of false response insertion. In the 10:1 ratio condition, PPP values begin to substantially improve on simply classifying an elevated case randomly at T-scores ³ 75 in the 30% condition. In the 40% and 50% conditions, PPP values exceeded .80 for cutting scores of T ³ 75. NPP findings for both the 10:1 and 1:1 ratio conditions were highly comparable to findings from the MMPI-A false response insertion procedure. Similar to the MMPI-A findings, NPP values (1:1 condition) were generally more modest for the false response insertion procedure in comparison to true response insertion. For sensitivity, a pattern similar to the one found for the MMPI-A emerged for the MMPI-2. That is, sensitivity values for the false response insertion procedure were generally more modest than the findings for true response insertion shown in Table 7. Furthermore, sensitivity values for the MMPI-2 were somewhat greater than corresponding values for the MMPI-A (shown in

Table 6). Specificity values were comparable to those found for the MMPI-A.

DISCUSSION The goals of this study were to examine the usefulness of the MMPI-2 and MMPI-A TRIN scales in detecting varying degrees of acquiescence and nonacquiescence and to evaluate cutoff scores in the current samples. To date, little research has been conducted on these scales to empirically evaluate the effectiveness of the cutoff scores recommended in the MMPI-2 (Butcher et al., 2001) and MMPI-A (Butcher et al., 1992) manuals. The MMPI-A and MMPI-2 TRIN scales are clearly quite sensitive to the random insertion of true or false responses. Mean TRIN raw scores increased with increasing degrees of true response insertion for both the MMPI-A and MMPI-2. Similarly, raw scores decreased when false responses were randomly inserted into existing protocols. Effect sizes were large even with fairly modest (e.g., 20%)


104

ASSESSMENT

TABLE 7 Hit Rate, Positive Predictive Power (PPP), Negative Predictive Power (NPP), Sensitivity (SEN), and Specificity (SPE) for the Minnesota Multiphasic Personality Inventory–2 True Response Inconsistency Scale at Five Cutting Scores With Varying Degrees of True Response Insertion Hit Rate True Response Insertion Percentage

Cut Score (T ³)

PPP

NPP

10:1

1:1

10:1

1:1

10:1

1:1

SEN

SPE

10

65 70 75 80 90

.78 .87 .91 .91 .91

.62 .56 .53 .52 .50

.18 .23 .40 .36 .44

.69 .75 .87 .85 .89

.93 .92 .91 .91 .91

.58 .54 .52 .51 .50

.42 .19 .07 .05 .01

.81 .94 .99 .99 1.0

20

65 70 75 80 90

.80 .90 .92 .92 .91

.76 .71 .62 .59 .52

.27 .43 .70 .68 .84

.79 .88 .96 .96 .98

.97 .95 .93 .92 .91

.74 .64 .57 .55 .51

.71 .47 .25 .19 .04

.81 .94 .99 .99 1.0

30

65 70 75 80 90

.82 .92 .95 .94 .92

.84 .83 .76 .71 .58

.32 .54 .83 .83 .95

.82 .92 .98 .98 1.0

.99 .97 .95 .95 .92

.87 .77 .68 .64 .54

.87 .72 .52 .43 .16

.81 .94 .99 .99 1.0

40

65 70 75 80 90

.83 .94 .97 .96 .94

.89 .93 .89 .84 .69

.34 .60 .88 .89 .98

.84 .94 .99 .99 1.0

1.0 .99 .98 .97 .94

.97 .92 .82 .77 .62

.97 .91 .78 .70 .39

.81 .94 .99 .99 1.0

50

65 70 75 80 90

.83 .94 .98 .98 .97

.90 .96 .96 .94 .84

.35 .61 .90 .91 .99

.84 .94 .99 .99 1.0

1.0 1.0 .99 .99 .97

.99 .97 .93 .90 .75

.99 .97 .93 .88 .67

.81 .94 .99 .99 1.0

degrees of true or false response insertion for both the MMPI-2 and MMPI-A. Classification Accuracy A number of points must be considered regarding the classification accuracy results from this study. Validity decisions are not strictly categorical in nature but rather are usually stated in terms of degrees of certainty regarding validity. This point is perhaps particularly important when discussing TRIN because we believe base rate estimates of acquiescence and nonacquiescence in clinical and nonclinical populations are unknown. The base rate of a given phenomenon in various settings has a substantial impact on the appropriateness of any proposed cutoff score (e.g., Meehl & Rosen, 1955). Furthermore, during an actual MMPI-2 or MMPI-A assessment, it is important to integrate findings from TRIN with the entire family of validity scales in these instruments. Finally, the goals of any specific assessment must be considered when making validity decisions. In some contexts, such as screening for

psychopathology in a setting such as a school or university, it may be more important to minimize false positives (i.e., maximize positive predictive power). In other cases, such as treatment planning for an individual on a psychiatric inpatient unit, one may strive to minimize false negatives (i.e., maximize negative predictive power). The following discussion is based on the results for the 10:1 ratio conditions because this base rate would likely appear more realistic than 50% to most individuals who use the MMPI-2 or MMPI-A extensively. MMPI-A For the MMPI-A true response insertion procedure, positive predictive power was generally modest until reaching 30% true response insertion at the cutoff score of T ³ 75 recommended in the MMPI-A manual (Butcher et al., 1992). However, it is also unlikely that small degrees of acquiescence (e.g., 10% or 20%) would have a substantial impact on the T-scores for MMPI-A clinical, content, or supplementary scales. As seen in Table 5, positive pre-



TABLE 8 Hit Rate, Positive Predictive Power (PPP), Negative Predictive Power (NPP), Sensitivity (SEN), and Specificity (SPE) for the Minnesota Multiphasic Personality Inventory–2 True Response Inconsistency Scale at Five Cutting Scores With Varying Degrees of False Response Insertion Hit Rate False Response Insertion Percentage

Cut Score (T ³)

10:1

1:1

PPP 10:1

NPP 1:1

10:1

1:1

SEN

SPE

10

65 70 75 80 90

.76 .86 .90 .90 .91

.54 .53 .51 .51 .50

.12 .17 .24 .19 0.00

.58 .68 .76 .70 0.00

.92 .92 .91 .91 .91

.52 .52 .51 .50 .50

.26 .13 .03 .02 0.00

.81 .94 .99 .99 1.0

20

65 70 75 80 90

.78 .88 .91 .91 .91

.63 .61 .55 .53 .50

.19 .32 .52 .47 .47

.70 .82 .92 .90 .90

.94 .93 .92 .92 .91

.59 .57 .53 .52 .50

.44 .29 .11 .08 .01

.81 .94 .99 .99 1.0

30

65 70 75 80 90

.80 .90 .92 .92 .91

.73 .73 .63 .60 .51

.25 .45 .73 .69 .78

.77 .89 .96 .96 .97

.96 .95 .93 .93 .91

.69 .66 .58 .55 .51

.64 .51 .28 .20 .03

.81 .94 .99 .99 1.0

40

65 70 75 80 90

.81 .92 .94 .94 .92

.82 .83 .74 .69 .55

.30 .54 .83 .82 .93

.81 .92 .98 .98 .99

.98 .97 .95 .94 .92

.82 .78 .66 .62 .53

.82 .73 .49 .39 .10

.81 .94 .99 .99 1.0

50

65 70 75 80 90

.82 .93 .96 .96 .93

.87 .91 .84 .80 .61

.33 .59 .87 .87 .97

.83 .93 .99 .99 1.0

.99 .99 .97 .96 .93

.92 .88 .76 .71 .56

.93 .88 .69 .60 .22

.81 .94 .99 .99 1.0

dictive power values were all reasonably high at a T-True response insertion percentage score cutoff of 75 for 30%, 40%, and 50% true response insertion. T-score cutoffs above 75 resulted in fairly low sensitivity (i.e., fewer than 25% of protocols were elevated) for 30% true response insertion. Therefore, the findings suggest that the TRIN cutoff score recommended in the MMPI-A manual appears reasonable for protocols with moderate degrees (and greater) of acquiescence. NPP values were quite high in all conditions. For the false response insertion procedure, a cutoff of T ³ 75 also appears reasonably effective. At this T-score cutoff, PPP (and sensitivity) values were less than those found for true response insertion. It appears that applying cutoff scores greater than 75 comes at the cost of very low sensitivity. For example, even when inserting a substantial degree (i.e., 40%) of false responses, only 15% of protocols reached T-scores of 80 or greater. The somewhat more modest findings for the false response insertion procedure are likely due to the fact that TRIN has fewer false response pairs. Hence, TRIN may be less reliable for detecting nonacquiescence as opposed to acquiescence.

MMPI-2 For the MMPI-2 true response insertion procedure, positive predictive power was reasonably high even at a very modest 20% level of true response insertion. For the MMPI-2, a T-score ³ 75 appears to be a reasonable cutoff in terms of PPP for the current sample. The cutoff score of T ³ 80 recommended in the MMPI-2 manual (Butcher et al., 2001) did not result in an appreciable gain in PPP over a T-score cutoff of 75, and of course, the use of the higher cutoff score comes at some cost in terms of decreased sensitivity. NPP values were all greater than .90 in the 10:1 ratio condition. For the false response insertion procedure, a T-score ³ 75 also appears to be the best MMPI-2 TRIN cutoff score in terms of PPP. The modest PPP findings in the 10% and 20% conditions are probably of limited concern, because these small degrees of nonacquiescence would probably not dramatically influence T-scores on other MMPI-2 scales. Again, raising the cutoff score to T ³ 80 did not result in a substantial increment in PPP. Similar to the MMPI-A, the present results suggest that the MMPI-2


106

ASSESSMENT

TRIN scale may perform better in detecting an acquiescent versus a nonacquiescent response style. Here, too, NPP values were all consistently high. This study has a number of methodological limitations. First, the various degrees of true and false response insertion reported in this study are overestimates. In actuality, the number of modified items in each condition was undoubtedly less than the percentage stated because some items selected for modification already exhibited the target response. For example, in the 20% true response insertion condition, 20% of items were selected for modification, but a percentage of these items likely already consisted of true responses and therefore would not be modified. A second limitation relates to the TRIN scores for the comparison groups. In the present study, all cases with T-score elevations on TRIN above the designated cutoff score (ignoring the T or F direction of the elevation) served as false positives. Therefore, this method provided a very conservative test of TRIN because undoubtedly many of the cases in the comparison groups designated as false positives were, in fact, true positives. Finally, unmodified protocols were duplicated to simulate base rates of acquiescence and nonacquiescence less than 50%. Ideally, the use of a very large sample of unmodified protocols would be preferred to the method used in this study. However, the subsamples used for the comparison groups in this study were reasonably large prior to the duplication procedure, and therefore, the use of this method probably did not have a significant negative influence on the overall findings. In sum, the MMPI-A and MMPI-2 TRIN scales are sensitive in detecting acquiescence or nay-saying response styles created by varying degrees of true or false response insertion. Findings suggest that TRIN may perform somewhat better when the goal is to detect an acquiescent (vs. a nonacquiescent) response style. As expected, the MMPI-A and MMPI-2 TRIN scales were more effective as the degrees of acquiescence or nonacquiescence were increased, particularly for levels of response insertion above 20%. Finally, with an emphasis on PPP and using a relatively low base rate of fixed responding, results largely supported cutoff scores at or near those currently recommended in the MMPI-A and MMPI-2 manuals.

Baer, R. A., Kroll, L. S., Rinaldo, J., & Ballenger, J. (1999). Detecting and discriminating between random responding and overreporting on the MMPI-A. Journal of Personality Assessment, 72, 308-320. Berry, D. T. R., Wetter, M. W., Baer, R. A., Widiger, T. A., Sumpter, J. C., Reynolds, S. K., et al. (1991). Detection of random responding on the MMPI-2: Utility of F, Back F, and VRIN scales. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3, 418-423. Butcher, J. N., Graham, J. R., Ben-Porath, Y.S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). MMPI-2 (Minnesota Multiphasic Personality Inventory-2): Manual for administration and scoring. Minneapolis: University of Minnesota Press. Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A., Ben-Porath, Y. S., & Kaemmer, B. (1992). Minnesota Multiphasic Personality Inventory–Adolescent (MMPI-A): Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Graham, J. R. (2006). MMPI-2: Assessing personality and psychopathology (4th ed.). New York: Oxford Press. Hathaway, S. R., & McKinley, J. C. (1942). The Minnesota Multiphasic Personality Inventory (Rev. ed.). Minneapolis: University of Minnesota Press. Jackson, D. N., & Messick, S. (1961). Acquiescence and desirability as response determinants on the MMPI. Educational and Psychological Measurement, 21, 771-790. Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194-216. McKinley, J.C., Hathaway, S.R., & Meehl, P.E. (1948). The MMPI: VI.The K scale. Journal of Consulting Psychology, 12, 20-31. Tellegen, A. (1982). Brief manual for the Differential Personality Questionnaire. Unpublished manuscript, University of Minnesota, Minneapolis. [Since renamed Multidimensional Personality Questionnaire]. Tellegen, A. (1988). The analysis of consistency in personality assessment. Journal of Personality, 56, 621-663. Wetter, M. W., & Tharpe, B. (1995, March). Sensitivity of the TRIN scale on the MMPI-2. Paper presented at the 30th Annual Symposium on Recent Developments in the Use of the MMPI-2 and MMPI-A, St. Petersburg Beach, FL.

REFERENCES

Robert P. Archer, Ph.D., is the Frank Harrell Redwood Distinguished professor of psychiatry and behavioral sciences and the director of the Psychology Division at Eastern Virginia Medical School.

Archer, R. P. (2005). MMPI-A: Assessing adolescent psychopathology (3rd ed.). Mahwah, NJ: Lawrence Erlbaum. Archer, R. P., Handel, R. W., Lynch, K. D., & Elkins, D. E. (2002). MMPI-A validity scale uses and limitations in detecting varying levels of random responding. Journal of Personality Assessment, 78, 417-431.

Richard W. Handel, Ph.D., is an associate professor in the Department of Psychiatry and Behavioral Sciences at Eastern Virginia Medical School. His research is focused on the assessment of personality and psychopathology using the MMPI-2 and MMPI-A instruments. He received his Ph.D. in clinical psychology from Kent State University. Randolph C. Arnau, Ph.D., is an assistant professor of psychology at the University of Southern Mississippi. He conducts research in personality, psychometrics, and the beneficial effects of hope and optimism. He obtained his Ph.D. in clinical psychology from Texas A&M University.

Kristina L. Dandy is an experimental psychology doctoral student at Texas Christian University. Her research interests include using animal models to explore the neurobiological and behavioral changes associated with drug use and abuse.
