one of three instruction types (honest, non-targeted faking in which participants behaved as if ...... (2002) study did not include an impression management scale.
EFFECTS OF ITEM RANDOMIZATION AND APPLICANT INSTRUCTIONS ON DISTORTION ON PERSONALITY MEASURES
Katherine Anne Wolford
A Thesis Submitted to the Graduate College of Bowling Green State University in partial fulfillment of The requirements for the degree of
MASTER OF ARTS August 2009 Committee: Michael Zickar, Ph.D., Advisor Scott Highhouse, Ph.D. William O’Brien, Ph.D.
© 2009 Katherine Wolford All Rights Reserved
iii ABSTRACT Michael Zickar, Ph.D., Advisor The present study evaluated the impact of test format and instructions on success at faking a Big Five personality assessment. Participants (N=320) completed personality and impression management measures under one of two testing format (blocked or randomized) and one of three instruction types (honest, non-targeted faking in which participants behaved as if applying for any job of their choice, and targeted faking in which they were asked to behave as if applying for a single specified job). No practically-significant differences in scores were observed based upon format. Scores on all scales were significantly higher in the targeted faking condition than in the other conditions although differences between honest and non-targeted were significant for only a few scales. Significant interactions between format and instructions were found for Conscientiousness, Openness, Emotional Stability, and Impression Management. For these scales, scores increased linearly across conditions when surveys were given in a randomized format but a marked decrease was seen between honest and non-targeted faking, followed by a sharp increase in scores from non-targeted to targeted faking, when the tests were given in blocked format. Limitations, future directions, and practical implications are discussed.
iv
Dedicated to my late grandmother, in memory of her love and constant support.
v ACKNOWLEDGMENTS First, I would like to acknowledge and thank my parents and family members for helping me become the person I am today. I consider myself incredibly lucky to have such support in my life. I would also like to acknowledge my advisor, Dr. Michael Zickar, and my committee members Dr. Scott Highhouse and Dr. Bill O’Brien. Without their help and suggestions, the present thesis would be neither possible nor passable. Finally, I would like to acknowledge my best friend and fiancé, Mike. His assistance in creating the present work as well as maintaining my sanity during the process was invaluable.
vi TABLE OF CONTENTS Page CHAPTER I. INTRODUCTION .........................................................................................
1
Personality Testing in the Workplace ........................................................................
1
Faking Behavior .........................................................................................................
2
Reducing Susceptibility to Distortion .......................................................................
5
Impression Management ...............................................................................
5
Item Randomization ......................................................................................
7
Applicant Instructions ...................................................................................
12
CHAPTER II. METHOD.....................................................................................................
14
Participants
...........................................................................................................
14
Measures
...........................................................................................................
14
Personality Measures.....................................................................................
14
Impression Management Measures ...............................................................
15
Procedure
...........................................................................................................
15
CHAPTER III. RESULTS ...................................................................................................
18
Omnibus Testing .......................................................................................................
18
Tests of Hypotheses ..................................................................................................
18
Hypotheses 1 & 2 ..........................................................................................
18
Hypothesis 3..................................................................................................
19
Hypothesis 4..................................................................................................
20
Hypothesis 5..................................................................................................
20
CHAPTER IV. DISCUSSION.............................................................................................
22
vii Test Format
...........................................................................................................
22
Instructions
...........................................................................................................
23
Interactions
...........................................................................................................
25
Limitations
...........................................................................................................
27
Future Directions.......................................................................................................
29
REFERENCES.......... ...........................................................................................................
30
APPENDIX A. SAMPLE PERSONALITY AND IMPRESSION MANAGEMENT ITEMS .........................................................................................
35
APPENDIX B. SAMPLE JOB DESCRIPTION .................................................................
36
APPENDIX C. APPLICANT INSTRUCTIONS BY CONDITION...................................
37
TABLE 1. Main Effects of Format on Scale Scores .............................................................
38
TABLE 2. Main Effects of Instructions on Scale Scores......................................................
39
TABLE 3. Format by Instruction Interactions ......................................................................
40
FIGURE 1. Format by Instruction Interaction for Conscientiousness .................................
41
FIGURE 2. Format by Instruction Interaction for Extraversion ...........................................
42
FIGURE 3. Format by Instruction Interaction for Openness ................................................
43
FIGURE 4. Format by Instruction Interaction for Agreeableness ........................................
44
FIGURE 5. Format by Instruction Interaction for Emotional Stability ................................
45
FIGURE 6. Format by Instruction Interaction for Impression Management .......................
46
viii LIST OF TABLES Table
Page
1
Main Effects of Format on Scale Scores ...................................................................
38
2
Main Effects of Instructions on Scale Scores............................................................
39
3
Format by Instruction Interactions ............................................................................
40
ix LIST OF FIGURES Figure
Page
1
Format by Instruction Interaction for Conscientiousness .........................................
41
2
Format by Instruction Interaction for Extraversion...................................................
42
3
Format by Instruction Interaction for Openness .......................................................
43
4
Format by Instruction Interaction for Agreeableness................................................
44
5
Format by Instruction Interaction for Emotional Stability........................................
45
6
Format by Instruction Interaction for Impression Management ...............................
46
1 CHAPTER 1. INTRODUCTION The use of personality testing as a tool for employee selection is currently a popular business trend (Hoffman, 2001). Companies who utilize the results of such tests to help in the decision-making process assume that the results accurately reflect the characteristics of the individuals being tested (Allen, 1965). Unfortunately, response distortion by applicants, in an effort to appear more desirable as a candidate, is quite common (Alliger, Lilienfield, & Mitchell, 1996. Rosse, Stecher, Miller, & Levin, 1998; Christiansen, Burns, & Montgomery, 2005). The presence of faking calls into question the predictive validity of such measures and has led to the development of several techniques for reducing or detecting distortion. Of these methods, randomization of test items, manipulation of applicant instructions, and the inclusion of impression management items are some of the most common (Furnham, 1990; Mael, 1991; Ryan & Sackett, 1987). Despite their common purpose, use of these methods in conjunction with one another has not been analyzed. The purpose of the present study was to synthesize the distinct streams of research on response distortion on personality measures, use of embedded impression management scales, item placement, and test format, to determine whether a particular combination of blocking or randomization and applicant instructions has a significant impact upon a prospective employee’s ability to fake successfully on a personality test. Personality Testing in the Workplace Personality testing offers many benefits that other selection tools do not, such as being simple and consistent in its administration, relatively inexpensive, and free from subjective judgments. Additionally, such tests typically produce less adverse impact than do other preemployment testing or assessment methods (Bobko, Roth, & Potosky, 1999; Day & Silverman,
2 1989). Personality assessments can be used to predict several job-related behaviors and outcomes, including managerial aptitude (Barrick & Mount, 1991), general level of job performance (Tett, Jackson, & Rothstein, 1991), and proactive and counterproductive work behaviors (Roccas, Sagiv, Schwartz, & Knafo, 2002; Salgado, 2002). Although personality assessments are better predictors of success in certain jobs as compared to others, a large body of research suggests that personality testing for applicant selection can be a useful tool. Faking Behavior Past research has indicated that a significant proportion of job candidates who take personality tests as part of an application process answer in a less-than-forthright manner. In experiments utilizing students and other non-applicant participants, instructing participants to behave as job applicants leads to considerable rates of faking behavior, even if the instructions given did not indicate that responses should be altered in any specific way (Alliger, Lilienfield, & Mitchell, 1996; Alliger & Dwight, 2000; Ellingson, Sackett, & Hough, 1999; Hurtz & Alliger, 2002; Zickar & Robie, 1999). Evidence of faking has also been observed in samples of real applicants. Comparisons of applicant and incumbent personality assessment scores have indicated that applicants’ scores are generally inflated (Rosse, et. al., 1998). Furthermore, when confidential post-test surveys are given to applicants, a significant percentage admit to having altered their answers in an effort to portray themselves as better qualified for desired positions (Christiansen, Burns, & Montgomery, 2005; Donovan, Dwight, & Hurtz, 2003). The assumption that results reflect the true nature of respondents’ beliefs and behaviors is especially tenuous when one considers just how much motivation there is for respondents to fake; that is, there is motivation for them to answer in the manner that they think they should, rather
3 than in a manner that is truly indicative of who they are and how they behave. Simply being put in the position of being a potential employee can be enough to compel people to alter their responses in an effort to appear more desirable (Villanova & Bernardin, 1991). There is likely to be a greater tendency for people to manipulate their answers when responding to personality assessment questions in high-pressure situations (Bass, 1957; Leary & Kowalski, 1990; McCrae & Costa, 1983); a job application scenario in which a person feels he or she must perform strongly in order to obtain a desired position is certainly pressure-filled. An applicant may feel that the procurement of a position hinges upon the results of his or her personality test. Whether this is true or not, it may lead applicants to believe that they must respond dishonestly in order to match what they believe to be the job requirements and the traits of the ideal employee (Leary & Kowalski, 1990; Rosse, et. al., 1998). Further strengthening this tendency for test-takers to fake is the fact that personality measures are generally unverifiable. Applicants may be more likely to exaggerate their positive attributes or minimize their faults when they believe that their statements will not, or can not, be verified by those who are administering the test (Fiske & Taylor, 1991; Rosse, et. al., 1998). There is rarely a way for employers to determine whether a prospective employee is answering in a manner that is honest and truly indicative of how he or she thinks and behaves. An applicant’s endorsement of an item may indicate that he or she is very outgoing, for example, but there is no practical way for a prospective employer to determine whether such a statement is truly accurate. The generally unverifiable nature of personality self-assessments makes it more probable that test-takers will feel comfortable altering their responses in an effort to appear more desirable.
4 Applicant manipulation of responses should be taken as a serious matter because previous research, using student samples as well as real-life applicants and incumbents, shows that testtakers are effectively able to raise their scores. In fact, a meta-analysis conducted by Viswesvaran and Ones (1999) revealed that merely instructing a group of test-takers to “fake good,” or answer in what they believe to be the most desirable manner, lead to final scores that over half a standard deviation higher than those produced by honest test-takers. Test-takers were able to achieve this inflation without the use of any special coaching or training. Additionally, the included studies involved participants applying for (or being told to behave as if applying for) several different types of jobs. Results were consistent across job-types, indicating that real-life and pseudo-applicants are able to determine which trait levels are most universally desirable and target their faking to these specific points (Furnham, 1990; Paulhus & Bruce, 1984; Viswesvaran & Ones, 1999). In instances where response distortion occurs, it is more likely for applicants engaging in distortion to score higher in the ranking distribution compared to honest test-takers. The inflated scores produced through faking lead to an increased likelihood of surpassing any cutoff point and, because ranking is a common practice in hiring, “fakers” may have a better chance of being selected for a position. One prime example of this effect was seen in a study which revealed that, in situations where at least half of an applicant sample engages in response distortion, those who do so comprise up to 83% of the top portion of the distribution (Zickar, 2000). These top-ranked applicants would appear to employers to be the candidates best suited to the positions in question, yet they actually possess lower levels of the desired attributes than would be indicated by their test scores. The resulting imbalance between true and perceived levels of job-related attributes leads to an increased chance of employers making costly false-positive decisions when hiring
5 (Donovan, Dwight, & Hurtz, 2003). Response distortion among applicants, then, may upset the hiring process and decrease the validity of pre-employment personality assessment processes (Alliger, Lilienfield, & Mitchell, 1996; Douglas, McDaniel, & Snell, 1996; Ellingson, Sackett, & Hough, 1999; Rosse et al., 1998). Reducing Susceptibility to Distortion Impression Management As there is no singularly effective way to completely eliminate response distortion, several methods have been developed to reduce or detect the behavior. The majority of methods to control response distortion have focused on the intentional aspect of response distortion, known as impression management, rather than the unintentional, supposedly unconscious process of self-deceptive enhancement (Paulhus, 1998). The latter occurs when a test-taker actually believes that he or she possesses extraordinarily high levels of desirable traits. The applicant does not knowingly alter his or her responses, so this behavior is not identified as faking. Therefore, the inclusion of impression management items, alternately referred to as social desirability or unlikely virtues measures, is the preferred method for detecting response distortion as they purportedly indicate whether applicants are intentionally altering their responses in an effort to appear in a more favorable light. An applicant’s level of endorsement of impression management items is used as a measure of intentional response distortion (Alliger, Lilienfield, & Mitchell, 1996; Ryan & Sackett, 1987) with higher scores indicating greater levels of impression management. Impression management items most commonly ask test-takers to rate their endorsement of the extremes of the personality spectrum. The items generally preface virtue or morality-based statements with words such as ‘never’ or ‘always’ to create assertions that appear socially
6 preferable but are actually highly improbable (Hoffman, 2001; Miller, 2001). For instance, given the item “I never gossip,” applicants may be instinctively inclined to agree when trying to appear in the best light possible. However, practically every individual has gossiped at some point in life. Thus, endorsement of this item would be seen as an indication that an applicant was providing answers not truly representative of him or herself in an effort to appear more desirable. Including several such items is seen as a way for administrators to identify applicants engaging in conscious response manipulation (Alliger, Lilienfield, & Mitchell, 1996; Ryan & Sackett, 1987). The popularity of the practice of embedding impression management items within personality measures has led many researchers to investigate whether impression management represents a true facet of personality. Although no definite conclusion can be drawn at this point, the results of studies suggest that impression management items do, indeed, measure a viable personality construct. A meta-analysis by Ones, Viswesvaran, and Reiss (1996), for instance, showed that when the results of personality assessments were corrected for situational factors influencing faking behavior, test-takers’ social desirability scores showed low to moderate positive correlations with all of the Big Five measures except neuroticisim. McCrae and Costa (1983) also argued that social desirability was directly related to personality, although not directly related to any performance criteria. In this study, test-takers’ social desirability scores were also found to relate to others’ ratings of the test-takers, something that should not have happened were the social desirability scores purely the result of inflation on the test-takers’ part. Clearly, the research results regarding the effectiveness of social desirability items and their validity as measures of a true personality construct have been mixed. Although more research on the subject is certainly needed, it would appear that the tendency to answer questions in a socially desirable manner does represent a legitimate personality trait. Some people, whether
7 in applicant situations or not, will simply feel a greater desire to make their answers fit their perception of an ideal profile than will others. Whether or not currently available social desirability measures are the more accurate method through which to assess this trait is debatable. However, the fact remains that social desirability/impression management items are popular in practice and will not likely fall out of favor in the near future. Item Randomization One popular way to inhibit faking is to present personality assessments to applicants in randomized formats (Anastasi, 1976). In such tests, items are randomly dispersed throughout the test rather than being “blocked,” or separated out into discrete sections, each containing only items addressing a single underlying construct. The Big Five inventories, the International Personality Item Pool, the Hogan Personality Inventory, and the NEO Five Factor Inventory are all examples of widely-used personality inventories that use a randomized format (Costa & McCrae, 1985; Goldberg, 1999; Hogan & Hogan, 1995). In fact, it is difficult to find a commonly used, commercially available personality measurement instrument that is not randomized. This practice presumably developed based on the assumption that blocking makes it easier for test-takers to determine which personality traits are being measured (Mumford & Stokes, 1992) and which answers would be considered the best. Although the topic of blocked versus randomized test formatting affecting response patterns has been discussed in the personality testing literature, little consensus has been reached due to conflicting results. Outside of the personality-specific literature there appears to be support for the idea that blocked personality test formats would be more easily analyzed by applicants and would, in turn, be more susceptible to faking efforts. Using biodata items, Mael (1991) showed that blocked
8 formats can be more transparent and make it easier for test-takers to identify what they are being tested on. Although not directly tested, this transparency may increase the probability of applicants successfully faking on such tests because item transparency increases a test-taker’s ability to retrieve relevant information (Schmit, Ryan, Stierwalt, & Powell, 1995). Bornstein, Rossner, Hill, and Stephanian (1994) also found that test-takers become increasingly more adept at indicating what a desirable response would be as their knowledge of testing purpose increases. Although custom supports the use of randomized tests, the literature on format’s effects on the psychometric properties of tests has actually produced mixed results. Several studies have even shown that a blocked format may be preferable in certain situations. A LISREL factor analysis by Schriesheim, Solomon, and Kopelman (1989) indicated that discriminant and convergent validity were enhanced for two sets of self-report measures, job satisfaction and job design, when grouped formats were used for both. The authors indicated that using a grouped format somewhat improved the psychometric qualities of already weak measures. Schriesheim and Denisi’s (1980) previously mentioned study also showed that practical benefits to test-takers could be had through using blocked formats. Grouping items together into their respective categories lessened fatigue effects for test-takers and made the purpose of the items clearer to them. This could be beneficial because as Harrison, McLaughlin, and Coalter’s (1996) work with organizational justice items reveals, people have limited cognitive capacity when answering self-report items. If this capacity is exceeded, reactions to one item may unduly influence reactions to later ones. Additionally, Tourangeau and Rasinski’s (1988) article indicates that increasing test-takers’ understanding of the true purpose behind items may be a good idea. The results of this study revealed that participants were able to provide more accurately reflective answers when they understood the meaning behind items.
9 In general, it is difficult to discern any clear indication as to what is the ideal format for self-report tests. In some instances, aspects of one format that are touted as benefits in one article, such as increased transparency, may be seen as detrimental in another. With empirical evidence supporting the claims of proponents of both options, it seems that it would be wise to further investigate this topic. Given the overwhelming preference for randomized test formats in the high-stakes world of applicant personality testing, there is certainly a need to determine which, if either, of the options are superior in this specific context The research on the relationships among test format, faking, and impression management variables has been scant at best, and only one previous study (McFarland, Ryan, and Ellis, 2002) has directly examined the impacts of randomization on test susceptibility to distortion. McFarland, Ryan, and Ellis studied the effects of item placement on faking behavior by having student volunteers complete either randomized or blocked forms of the NEO-FFI in both an honest (control) condition as well as one of two faking conditions. Half of the volunteers were told to pick the best answer during the portion in which they faked whereas the rest were told to answer as if they were applying for a job. The results of McFarland, Ryan, and Ellis’ study were generally mixed and marginal. Test administration (honest vs. fake) did have a significant main effect on personality scores, such that those in the faking condition achieved higher scores on all scales than did those in the honest condition. However, a main effect for item placement (grouped vs. random) was only found for the conscientiousness scale, with those completing the grouped format test achieving slightly greater increases between honest and faking scores (d=1.85) and between honest and applicant scores (d=2.17) than those completing the randomized test (d=1.53 and d=1.54, respectively). An interaction effect was also found between test administration and item
10 placement for conscientiousness and neuroticism, such that grouped-format scores were lower than randomized format scores in the honest condition but higher than randomized scores in the applicant and faking conditions. Beyond this, it was difficult to draw any solid conclusions from this article’s results regarding the effects of item randomization on personality test fakability. One possible explanation for these mixed and often marginal results is that the study itself suffered from a handful of weaknesses for which the present study was intended to correct. McFarland et al. (2002) examined the effects of test format and instruction on faking behavior using a within-subjects model in which all participants were exposed to both an honest and a faking condition. Furthermore, participants took the test under both conditions on the same day, with one administration immediately following the other. Despite counter-balancing by the researchers, there is no way to rule out effects of previous exposure on the results. Because past research has shown that previous exposure to a test will influence results during re-testing, a between-subjects model would be the logical choice. The present study utilized such a model in an effort to reduce confounds introduced when a single participant is exposed to more than one condition. Between-subjects designs suffer from higher susceptibility to individual differences, so an a priori power analysis was used to ensure that an adequate sample size was obtained in order to correct for decreased power and higher error variance. Another drawback to McFarland et al’s. (2002) study is that participants in the applicant faking condition were asked to behave as applicants but were not given a specific job to consider, a format known as non-targeted faking. Due to the vast differences in the personality-based requirements of jobs (e.g., accountant vs. comedian), it is not surprising that the results for participants within the applicant faking condition were not highly consistent. Research has shown that when participants are given a specific job to consider when behaving as applicants,
11 known as targeted faking, they answer in a much more uniform and consistent manner (Donovan, Dwight, & Hurtz, 2003). The use of targeted faking mimics real-life applicant situations much more closely and the effects of such targeted coaching will be investigated in the present study. Finally, the McFarland, et al. (2002) study did not include an impression management scale. The present study will include an impression management scale in addition to the Big Five personality scales. Taken together, the results of the aforementioned studies on test format support the longheld belief that blocked test formats are more transparent and therefore, more easily faked than randomized test formats. The present study tested the following hypothesis regarding item placement’s effects on ability of applicants to successfully distort their responses: H1: Participants receiving a blocked format test will achieve higher personality scale scores than those receiving the randomized format. Although no literature exists to indicate how blocking will affect scores on impression management items, it was believed that applicants receiving the blocked format would have slightly lower impression management scores due to transparency. It is likely that an applicant would be able to more easily discern the underlying aim of the questions: to catch test-takers who are trying to portray themselves in a more favorable light. When randomly interspersed throughout a test, the strong wording and moral implications of the individual statements would likely be diffused across the test. If all of these items were grouped together, though, the repeated use of polarizing words such as “always” and “never” would likely alert most test-takers to their unique properties and make them more likely to select safer, more neutral answers. The following hypothesis was proposed regarding test format and impression management:
12 H2: Participants receiving a blocked format test will achieve lower impression management scores than those receiving the randomized format. Applicant Instructions Applicant instructions have also been manipulated in an effort to reduce response distortion. Instructing applicants that they should answer in an honest manner has been only slightly effective in curbing distortion (Bornstein et al., 1994). Coupling instructions to be honest with an indication that answers will be verified, whether or not it is really possible, has led to more noticeable decreases in response distortion (Furnham, 1990; Rosse et al., 1998). Researchers have also found instructions to “fake good” to be effective at increasing faking (Viswesvaran & Ones, 1999). When these instructions are made specific to one individual job, an instruction format known as targeted faking, an even greater increase in distortion is seen (Donovan, Dwight, & Hurtz, 2003), further validating the conclusion that pre-test instructions play a significant role in determining applicant distortion behavior. The present study sought to replicate the positive and negative findings of these and other studies by comparing the effects of both targeted and non-targeted faking instructions on level and effectiveness of respondents’ response distortion. The following hypothesis was tested: H3: Participants receiving targeted faking instructions will achieve greater increases in personality scale scores over the honest test-takers than will those in the non-targeted condition. An important overall aim of the present study was to synthesize the streams of research on personality item placement and applicant instructions. Of specific interest was the impact that the combination of these two factors would have upon the susceptibility of a personality assessment to response distortion. It is likely that the
13 increased transparency of the blocked testing format, coupled with the tendency of targeted faking instructions to produce higher, more consistent scores among applicants, will lead to the highest increase in scores across instruction and format conditions. An interaction between item placement and applicant instructions was tested for to determine whether their combined effects significantly impacted response distortion to a greater extent. The following hypothesis was tested: H4: An interaction will be found between item placement and applicant instructions such that the effects of the blocked formatting will be stronger for participants in the targeted instruction condition leading to even greater increases in scores for this group. To better understand the effects of test format, the present study included an exploratory analysis examining the transparency of Impression Management items. To the best of the author’s knowledge, this is the first such manipulation to be conducted in a personality faking study. Participants completed an Impression Management identification activity following completion of the full personality assessment. In short, this activity required participants to identify items that they believed to be Impression Management items. This activity allowed for a direct assessment of the comparative transparency of this specific scale across formatting conditions. Due to past research indicating that blocked tests tend to be more transparent, the following hypothesis was proposed with regards to transparency of Impression Management items: H5: Participants receiving a blocked format test will be able to correctly identify more impression management items than those receiving a randomized format.
14 CHAPTER 2. METHOD Participants A total of 320 undergraduate student participants were included in the study. Participants were self-selected for participation and received credit in their respective courses in return for participation. Participants ranged in age from 18 to 35 years with an average age of 19.13 (SD=1.82) years. A majority of participants were female (64.4%) and a majority were white (81.3%) with the remaining participants identifying themselves as African American (13.7%), Hispanic (2.5%), Multi-racial (1.9%), or Asian (0.6%). Of the participants, a slight majority (53.8%) were not currently employed with the remainder holding either part-time (43.0%) or full-time (3.2%) jobs. When asked, 36.9% indicated that they had taken a personality test as part of a job application process at least one time in their lives. Measures Personality Measures Personality measures were drawn from Goldberg’s (1999) International Personality Item Pool (IPIP) inventories and included all five of the standard Big Five personality factors: Extraversion, Conscientiousness, Openness, Emotional Stability (sometimes known as Neuroticism), and Agreeableness. Sample items for each scale are included in Appendix A. Each factor was represented by twenty items for a total of 100 Big Five personality scale items. Responses were recorded using a five-point Likert scale ranging from very inaccurate to very accurate. For the purpose of analyses, higher scores were considered more preferable responses for job selection. Internal consistency (Cronbach’s) alphas for the individual sub-scales have been found to be approximately .86 for Extraversion, .86 for Emotional Stability, .81 for
15 Conscientiousness, .77 for Agreeableness, and .82 for Openness (International Personality Item Pool, 2001). Impression Management Measures Impression management measures were drawn from the Balanced Inventory of Socially Desirable Responding (BIDR; Paulhus, 1998). A sample item is provided in Appendix A. Only items from the Impression Management dimension, rather than the self-deception dimension, were included as the present study is concerned with the intentional response distortion measured by the former; internal consistency alpha for this portion of the BIDR has been found to be approximately .80 (BIDR; Paulhus, 1998). Responses were recorded on the same five-point Likert scale used for the personality items. Twenty impression management questions were drawn from the BIDR and included in the personality assessment administered to participants. Procedure Participants were randomly assigned to complete the full 120-item personality assessment in one of two formats: Blocked or Randomized. In the blocked arrangement, the questions covering each of the personality factors and the impression management scale were separated into discrete blocks, visually delineated by a heavy border. Items in the randomized test format were identical to those in the blocked test but were randomly distributed throughout the length of the assessment. In addition to the test format, the instructions participants were given prior to completing the test were varied. Participants were assigned to take the test under one of three instruction sets: Honest, Non-Targeted Faking, or Targeted Faking. Those in the honest group were instructed to complete the test as honestly as possible as their responses would remain completely confidential. Those in the non-targeted faking group were told to answer each question as if they were
16 applying for a job of their choice (no specific job was suggested). The targeted faking group was given a sample job description for a customer service position and was asked to answer the questions in the manner most likely secure the specific job outlined in the description. The position of customer service representative was chosen because all participants were expected to have at least some basic knowledge of its requirements to begin with (either as customer service employees or as retail shoppers). This was desirable as it would better allow participants to envision the “ideal” job applicant. The sample job description included information drawn from the U.S. Department of Labor’s Occupational Outlook Handbook (http://www.bls.gov/oco/). The job description included customer service tasks, education requirements, and necessary skills and abilities. See Appendix B for job description and Appendix C for targeted, non-targeted, and honest condition instructions. In all conditions, participants were asked to complete Impression Management item identification task following the completion of the personality test but prior to completion of the demographic survey. Participants were informed that certain questions within the assessment that they had just completed were designed to detect when a person answered in a socially desirable manner. They were asked to examine the last page of their completed personality assessment and determine which of the items were these “lie detection” questions. In both the Randomized and the Blocked formats of the test, the same number of Impression Management items was shown on the final page (10). This allowed for the raw number of correctly identified Impression Management items to be compared across participants, regardless of test format. Motivation was induced in both the Targeted and Non-Targeted faking conditions through the use of a possible monetary reward. Participants in these faking conditions were told that the reward would be given only to those receiving the scores that best matched the job for
17 which they were applying. For those in the Non-Targeted Faking condition who were allowed to choose their own position, face validity of this motivational offer was preserved by asking that each person indicate the job for which he/she was applying. Participants in this condition were told that their personality profile would be compared against the job’s candidate profile. Motivational rewards were $50 gift cards to the sponsoring university’s bookstore. The possibility of a reward was not mentioned to those in the honest condition until after the testing period had finished. Participants were debriefed individually as they completed their surveys. Those in the two faking conditions were informed that receipt of the gift cards would be by lottery and would not be tied to actual results. Those in the honest condition were informed of the possibility of the reward during debriefing. All participants were informed of the number of gift cards to be given away (5) and were also informed that each person who participated would have an equal chance of receiving one.
18 CHAPTER 3. RESULTS Omnibus Testing As an omnibus test, a MANOVA was computed with Instructions (Honest, Non-Targeted, Faking, Targeted Faking) and Format (Blocked, Randomized) as between-subjects variables and the five personality measures (Extraversion, Conscientiousness, Emotional Stability, Openness, and Agreeableness) as multiple dependent variables. This test was conducted to determine whether relationships existed between the experimental manipulations and test-takers’ scores on the Big Five measures. Significant main effects were found at the omnibus level for format, F(5,273)=2.32, p
