courses at Seneca Community College, Toronto, rang- ing in age from 19 to 26 years, with a mode of 21 years. They volunteered for participation. A list was.
Journal of Consulting and Clinical Psychology 1970, Vol. 35, No. 3, 381-387
EFFECTS OF ROLE DEMANDS AND TEST-CUE PROPERTIES ON PERSONALITY TEST PERFORMANCE: REPLICATION AND EXTENSION * ROLF 0. KROGER 2 AKD WILLIAM TURNBULL * University of Toronto In a replication of earlier findings by Kroger, the hypothesis was tested that the situation affects test performance by generating a set of role demands. Two randomly constituted groups of community-college students were asked to describe • themselves on the Strong Vocational Interest Blank, Taylor Manifest Anxiety scale, and Welsh Figure Preference Test after being exposed to implicit social cues intended to induce differential role taking. Highly reliable, rolespecific response differences were obtained on the three tests which increased when controls for accuracy of role perception and for test-cue properties were introduced. The previous results were successfully replicated over different Ss, different Es, and an additional test under conditions of improved control over extraneous variables. The present findings were interpreted as supporting the hypothesis and as extending its generality.
ployed, (c) the setting was held constant over conditions, and (d) an additional measure, the Taylor Manifest Anxiety (MA) scale (Taylor, 1953), was included. Role demands were manipulated by varying the test titles, the ostensible purpose of testing, and the social position of E. The 5s were randomly assigned to two conditions: (a) a "military" one where the ostensible purpose was the study of officer effectiveness, the E a military officer, and the tests labeled as devices for the study of officer effectiveness; and (b) an "artistic" one where the ostensible purpose was the study of artistic creativity, the E a psychologist, and the tests labeled as devices for the study of artistic creativity. In the first condition, 5s were expected to take the role of "experimental 5 in a study of officer effectiveness" (military officer) in response to the role demands and to enact the role by endorsing test items in role-relevant ways. In the second condition, 5s were expected to take and to enact the role of "experimental 5 in a study of artistic creativity" (creative artist)—Hypothesis 1. Since there study was assisted under Grant 165 of the are individual differences in the accuracy with Ontario Mental Health Foundation. The authors wish to thank B. J. Findlay for help in recruiting which 5s identify role demands, it was exthe8 Ss. pected that 5s whose identifications are inRequests for reprints should be sent to Rolf O. accurate would not enact the induced roles as Kroger, Department of Psychology, University of well as 5 whose identifications are more acToronto, Toronto S, Canada. 8 Now at the University of North Carolina, Chapel curate—Hypothesis 2. Finally, since test cues Hill. vary in their degree of role relevancy, it was 381 Kroger (1967) has shown that Ss adapt their personality test responses to situational requirements by enacting a situationally specific role in terms of test items. He concluded that the "test score represents a trait-methodrole unit [p. 304]" which reflects in varying proportions the 5's personal characteristics, the method of measurement (Campbell & Fiske, 1959), and the conditions of measurement. The present experiment is a replication and extension of that study. One drawback of the earlier study was the employment of differential methods of recruiting 5s to the conditions of the experiment. This arrangement may have affected the randomness of assignment of 5s to conditions. In the present experiment, the same method of recruitment was employed for all 5s, and randomness of assignment to conditions was strictly enforced. The present study differs from the previous one in the following respects as well: (a) a different population of 5s was used, (b) different Es were em-
382
ROLF 0. KROGER AND WILLIAM TURNBULL
expected that role enactment would be more effective when S was confronted with items containing highly salient cues than when he was confronted with items containing less salient or irrelevant cues—Hypothesis 3. Note that Ss were not instructed to take the two roles; they were merely exposed to the role demands and told to describe themselves. METHOD Subjects find Experimenters All Ss were male students enrolled in social science courses at Seneca Community College, Toronto, ranging in age from 19 to 26 years, with a mode of 21 years. They volunteered for participation. A list was made of all Ss who were then assigned at random to either the military or artistic conditions. The Ss responded anonymously and the conditions were run simultaneously. There were 22 Ss in each condition. The £s were two graduate students who appeared older than their age (28 years). They were recruited from other graduate departments (communications, sociology) and did not know the hypotheses under test. This was done to minimize the risk of .E-expectancy effects (Rosenthal, 1966). The "military" E identified himself by title (lieutenant) and last name, stating that he was from the "Canadian Defense Research Board." He was dressed in the appropriate naval uniform. The "artistic" E identified himself by title (doctor) and last name, stating that he was from the Department of Psychology, University of Toronto.
Experimental Situations and Procedure In the military condition, Ss were told that the purpose of the study was to determine "what makes a good military officer." The Strong Vocational Interest Blank (SVIB) (Strong, 1951), the MA scale (Taylor, 19S3), and the Welsh Figure Preference Test (Welsh, 1959) were reproduced on stencils and given the titles "Military Interest Questionnaire," "Military Opinion Questionnaire," and "Military Aptitude Test IV: Spatial Organization," respectively. To enhance the credibility of the experimental manipulation, Ss first completed the "Military Knowledge Test," which contained five questions regarding S's knowledge of some military matters. In the artistic condition, Ss were told that the purpose of the study was to determine "what makes people artistically creative." The three tests were presented in the same way, except that the term "artistic" was substituted for the term "military" in the titles. To increase the credibility of this condition, color reproductions of four paintings (Van Gogh, "Sunflower"; de Holesch, "Courtship"; Degas, "Two Dancers with Fan"; Chagall, "The Rabbi of Vitebsk") were mounted on the blackboard with instructions to name the painters and to rate the paintings in order of preference prior to taking the tests.
The Ss were clearly instructed to describe themselves; they were not asked to respond as if they were officers or artists. In short, the standard test instructions were given, including the admonition to work as fast as possible. The latter instruction was designed to reduce the possibility of faking. As Campbell (1966) points out, "Since it takes time to figure out 'proper' answers, this emphasis on speed may lead to more honest answers [p. 3]." Most Ss completed the SVIB within the average time of 35 minutes. In each condition, the tests were followed by a postexperimental inquiry and by an explanation of the experiment. The procedures devised by Kroger (196?, pp. 305306) were used (a) to assess the effectiveness of role enactment in terms of test items, and (6) to identify the cue properties of the SVIB and Welsh Figure Preference Test items. Briefly, a scale was defined as possessing primary cue properties if its content reflected clearly the role expectations for officer and artist, and as having secondary cue properties if its content merely correlated with those expectations. The remaining scales were defined as having neutral cue properties. No special procedures were employed for the MA scale. Effective role enactment on the MA scale was defined as a low score in the military condition and as a higher score in the artistic condition, on the basis that the role expectations for officers proscribe anxiousness but that those for artists allow it.
Postexperimental Inquiry The effects of the situation on test-taking behavior, specifically the effects of S's perception of the role demands, may be ascertained not only through inferences from S's test-taking behavior, but also from S's report of his perceptions. Such data must be interpreted cautiously, however, since postexperimental inquiries may be contaminated by S's intervening experimental responses. The Ss were asked to describe in written freeresponse form (a) the role of E, (6) the ostensible purpose of testing, (c) the actual purpose of testing, and (d) the test titles. Responses to these four questions were rated independently by two judges to determine the extent to which S had perceived "accurately" the intended role demands or, conversely, discovered the true purpose of the experiment. Interrater reliability was .967 for 43 protocols (one S failed to complete the inquiry). Seven military Ss and five artistic Ss were identified as "inaccurate" perceivers of the experimental role demands.
RESULTS Effects of Role Demands Hypothesis 1, stating that test-taking behavior varies directly with the social position of S in the testing situation, was tested by comparing the responses of Ss in the military and artistic conditions on the three dependent
PERSONALITY TEST PERFORMANCE
383
10 for scales with primary, secondary, and neutral cue-properties, respectively. For the comparisons involving accurate 5s only, the corresponding percentages are 100, 52, and 31. In addition, the consistency of the results is underscored by the absence of any reversals in the predicted direction of performance. It is apparent from these results that 5s responded to the experimental role demands as predicted and that the effect of such demands interacted with the saliency of the cues contained in the test items. This is reflected in the clearly declining percentages of reliable results found on the primary, secondary, and neutral scales.
measures. Since the direction of the mean differences was predicted, one-tailed t tests were used, except in the analyses involving scales with neutral cue properties, where no differences were predicted (cf. Kroger, 1967). (Note that the direction of the predicted mean differences varies with the content of the scales.) The results (Tables 1,2, and 3) are given separately for the three levels of the cueproperty variable. Each table presents two analyses: (a) comparison of experimental groups containing all 5s who participated, and (b) replication of the comparison after eliminating those 5s who were identified by the postexperimental inquiry as inaccurate role perceivers. Since some of the SVIB scales are intercorrelated, and since the overall trend in the results rather than the responses to individual scales is important for the present analysis, the results in Tables 1,2, and 3 are best summarized by noting the percentage of statistically reliable results at each level of the cueproperty variable. For this purpose, a result at the p < .10 level is regarded as reflecting a reliable trend. For the comparisons involving all 5s, these percentages are 91, 33, and
Effects oj Accuracy of Role Perception Hypothesis 2, stating that test-taking behavior varies directly with the accuracy of 5's perception of the role demands, was tested by contrasting the performance of the two experimental groups before and after elimination of inaccurate role perceivers. If test-taking behavior so varies, the differences between the groups should increase after inaccurate perceivers are eliminated from the analysis. Since a direct statistical comparison of the differences between experimental groups before and after eliminating inaccurate 5s is not
TABLE 1 COMPARISON OF MILITARY AND ARTISTIC GROUPS ON ROLE-ENACTMENT MEASURES WITH PRIMARY CUE PROPERTIES All subjects
Measure
Military (AT =22)
PD«
58.18 12.95 25.09 22.14 35.36 31.77 30.09 34.77 25.95 29.23 20.64 45.86
Military t*
rm
=
X
X Welsh Figure Preference Test Taylor MA scale SVIB Artist Architect Musician Advertising Author Aviator Policeman Army officer Forest service Masculinity-femininity
Artistic (AT =22)
Accurate subjects only
X
>
.35 >.30
60.87 13.47
>
33,32 27.86 41.55 38.09 35.14 28.14 20.77 20,77 16.68 41.41
2,90**** 1,68* 2.03** 1.61 1.59 2.00* 1.66 2.19** 1.15 1.35
>.45 >.25 >,30 >,20 >.20 >,30 >,25 __ >.30
22.00 20.27 32.00 28,13 26.80 37.53 26.87 32.00 23.20 48,87
• Predicted direction of difference between groups. >> One-tailed test. * p < .05. ** p < .025. *** f < .01. **** * < .005. ***** p < .0005.
PD»
>,20
Artistic (AT = 17) ft
fm
X >
.20
>
34.12 29.24 43.65 40.35 36.82 27.06 20.19 20.12 16.94 38.41
3.93***** 2.27** 3.55**** 2.76**** 2.77**** 2.60*** 1.66 2.51*** 1.56 2.79****
>,55 >,35 >.50 >.45 >.4S >.40 >.25 >.40 >.2S >.45
384
ROLF O. KROGER AND WILLIAM TURNBULL TABLE 2 COMPARISON on MILITARY AND ARTISTIC GROUPS ON ROLE-ENACTMENT MEASURES WITH SECONDARY CUE PROPERTIES All subjects
Measure
Military (N =22)
Artistic (tf-22) PD»
25.10 24.32 23.45 22.91 23.50 30.00 30.64 30.50 30.00 22.45 31.14 29.05 27.91 25.77 32.00 36.50 31.77 35.00 24.77 30.36 29.09
Military (b
rm
25.10 25.80 23.82 26.86 27.09 32.64 30.91 23.41 30.68 21.41 33.59 26.86 19.91 20.23 28.27 31.50 29.55 32.27 24.59 33.23 31.55
(W = 1 "0 \i\ — LOJ
Artistic (2V = 17)
PD«
X
X
X
SVIB Psychologist Physician Psychiatrist Osteopath Dentist Music Teacher Lawyer Production Manager Farmer Carpenter Printer Math-Science Teacher Senior CPA Accountant Office Worker Credit Manager Purchasing Agent Business Education Teacher Banker Mortician Pharmacist
Accurate subjects only
.00
1.36
>.20
1.50 1.32
>.20
.11 .79
.09 2.39** -.23
.25 -.81 .63 2.17** 1.85* 1.17 1.35 .65 .89 .06 -1.04 -.68
>.20 >.35
>.30 >.25 >.20
25.67 22.40 23.00 23.00 22.40 27.93 28.40 32.67 30.80 24.47 32.06 31.00 31.00 27.33 33.67 38.33 34.00 37.07 25.80 30.07 30.93
ft"
Tm
X
27.00 28.65 25,29 26.24 26.53 35.29 31.53 20,88 28.35 19.88 32,11 26,35 19,82 18,88 27.76 31.35 27.41 31.76 22.12 32.41 29.82
.35
1.82*
>.30
1.06 1.29 1.84* .78 3.46****
>.20 >.30
.51
.68 .91
-.02 1.15 2.58*** 2.58*** 1.64 1.60 1.61 1.57 1.10 -.67
>.50 >.20 >.40 >.40 >.25 >.25 >.25 >.25
.24
a
Predicted direction of difference between groups, b One-tailed test. * P < .05. ** p < .025. ***p ,20
>.30 = ,50
>.20
-.39
(b
Tm
X _
25.47 13.20 12.73 23.13 20.33 30.07 37.07 34.07 34.87 31.00 38,40 19.67 31.13 26.07 25.40 31.40 37.47 28.67
=
=
=
24.40
27.82 16.29 13.71 18.00 18.59 22.53 24.29 31.29 36.88 34.12 32.59 16.47 32.00 33.47 26.29 31.82 39.29 34.71 25.00
-.66 -.84 -.24 1.16 .37 2.31* 4.38*** .79 -.47 -.81 1.20 .83 -.18 -1.65 -.35 -.12 -.58 -1.65
>.20 >.35 >.60
>.20
>.25 >.25
-.19
• Predicted direction of difference between groups. >> Two-tailed test. * p < .05. **p ^ -~--^r^-^So^*:W^^ ^-F^Si
PERSONALITY TEST PERFORMANCE
387
REFERENCES sion of the previous findings to an additional test adds to the generality of the role-taking CAMPBELL, D. P. Manual for the Strong Vocational Interest Blank. Stanford, Calif.: Stanford Univereffect. sity Press, 1966. Thus, the previously found role-taking efD. T., & FISKE, D. W. Convergent and fect was successfully replicated over different CAMPBELL, discriminant validation by the multitrait-multiSs, different Es, and an additional instrument, method matrix. Psychological Bulletin, 1959, 56, and under conditions of strict randomness, 81-105. reduced manipulational impact, and in the EATON, A. M. Response stability as related to implicit role demands and subject item distance. absence of special motivation for compliance. Unpublished doctoral dissertation, University of This conclusion is supported by evidence reChicago, 1969. cently reported by Eaton (1969) who success- FRIEDMAN, H. Magnitude of experimental effect and a table for its rapid estimation. Psychological fully created implicit role demands for male Bulletin, 1968, 70, 245-251. and female high school students to take a R. 0. Effects of role demands and test-cue scientific or artistic role in answering items KEOGEK, properties upon personality test performance. from standard interest inventories. Journal of Consulting Psychology, 1967, 31, 304Finally, note that instructions designed to 312. reduce faking by emphasizing speed in an- KROGER, R. O., & ROBERTS, T. The test label as a minimal cue in test performance, Unpublished swering did not reduce the tendency of the manuscript, University of Toronto, 1966. 5s to respond in situationally appropriate ROSENTHAL, R. Experimenter effects in behavioral terms. This does not mean that individuals research. New York: Appleton-Century-Crofts, are "dishonest" in answering personality 1966. tests; rather, it appears to mean that they STRONG, E. K. Manual for the Strong Vocational Interest Blank. Stanford: Stanford University represent themselves via test items in ways Press, 1951. which are appropriate to a particular situaTAYLOR, J. A. A personality scale of manifest anxition. This is what individuals generally do ety. Journal of Abnormal and Social Psychology, in social situations, including testing situa1953, 48, 285-290. tions. And it seems that they do this in terms WELSH, G. S. Manual for the Welsh Figure Preference Test. Palo Alto, Calif.: Consulting Psyof their conception of their social role of the chologists Press, 1959. moment and not in terms of the psychologist's conception of personality traits. (Received July 25, 1969)
List of Manuscripts Accepted for Publication in the Journal of Consulting and Clinical Psychology Telemetered Heart Rate and Skin Potential of a Chronic Schizophrenic Patient Especially during Periods of Hallucinations and Periods of Talking: Frederic M. Levine* and Lester Grinspoon: Department of Psychology, State University of New York at Stony Brook,-Stony Brook, New York 11790. Trained Clients as Reinforcers of Counselor Behavior; Richard Dustin*: School of Education, University of Missouri, St. Louis, 8001 Natural Bridge Road, St. Louis, Missouri 63121. Intellectual and Cognitive Functions in Parkinson's Disease: Ralph M. Reitan* and Thomas J. Boll: Neuropsychology Laboratory, Indiana University Medical Center, 1100 West Michigan Street, Indianapolis, Indiana 46202. Microconsultation and Behavior Analysis: A Method of Training Psychologists as Behavioral Consultants: Dwight L. Goodwin,* William P. Garvey, and James R. Barclay: Department of Psychology, San Jose State College, 125 South Seventh Street, San Jose, California 95114. Effect of Seating Distance on Interpersonal Attraction in an Interview Situation: Michael L. Boucher*: 2736 Sheridan Drive, Tonawanda, New York 14150. The "Current Functioning" Myth: An Overlooked Fallacy in Psychological Assessment: Donald N. Bersofi*: Psychology and Education, Ohio State University, 65 South Oval Drive, No. 1 Page Hall, Columbus, Ohio 43210. * Asterisk indicates for whom address is supplied.
(Continued on page 405)
