Conscientiousness, Its Facets, and the Prediction of ... - SSRN papers

Volume 21 Number 1 March 2013

International Journal of Selection and Assessment

Conscientiousness, Its Facets, and the Prediction of Job Performance Ratings: Evidence against the narrow measures Jesús F. Salgado*, Silvia Moscoso** and Alfredo Berges*** *Facultad de Relaciones Laborales, University of Santiago de Compostela, Campus Vida, 15782 Santiago de Compostela, Spain. [email protected] **University of Santiago de Compostela, Santiago de Compostela, Spain ***University of Zaragoza, Zaragoza, Spain

This study empirically tested the predictions of the three basic perspectives on the bandwidth debate about the relationship between personality and job performance, regarding the validity of conscientiousness and its facets. The sample consisted of 226 police officers. Conscientiousness and three facets (order, industriousness, and self-control) were correlated with three performance criteria (overall job performance, task performance, and orderliness). A Schmid–Leiman transformation made it possible to residualize the variance of the facets and to isolate their unique contribution to the prediction of performance measures. The results showed that conscientiousness predicted the three criteria (true validities of .25, .28 and .37, respectively) and that the facets neither predicted job performance nor showed incremental validity over conscientiousness. Finally, the implications of the findings for theory and practice are commented on, and future research is suggested.

1. Introduction

T

bs_bs_banner

here is a controversy in applied personality assessment regarding whether personality dimensions, such as the Big Five, are better predictors of job performance than their facets (Hogan & Roberts, 1996; Ones & Viswesvaran, 1996; Paunonen, Rothstein, & Jackson, 1999; Schneider, Hough, & Dunnette, 1996). This controversy is known as the bandwidth debate. In this respect, there are at least three main perspectives. On the one hand, Ones and Viswesvaran (1996) suggested that the Big Five, and in general, broader measures of personality (e.g., integrity tests and other occupational personality scales) are better predictors than narrower personality measures of both narrow and broad performance measures. Furthermore, according to Ones and Viswesvaran (1996), broad personality measures are better for advancing in the theoretical account of organizational behavior. On the other hand, Ashton, Paunonen, Tett, and their associates (Ashton, 1998; Ashton, Jackson, Paunonen, Helmes, & Rothstein, 1995; Paunonen et al., 1999;

Paunonen & Nicol, 2002; Tett, Steel, & Beauregard, 2003) suggested that narrow personality measures are better predictors of narrow performance criteria and that they show incremental validity for broad criterion measures. They consider that too much information is lost when data are aggregated to the level of the Big Five personality dimensions. The third perspective was proposed by Hough and her associates (Hough, 1992; Hough & Schneider, 1995; Schneider & Hough, 1995; Schneider et al., 1996), who sustained that validity depends on the alignment of predictors and criteria. This view is shared by Hogan and Roberts (1996), who suggested that the nature of the criterion bandwidth dictates the choice of the predictors. In fact, Hogan and Roberts (1996) considered that overall job performance may be too broad to be used as a criterion with the majority of personality measures. Consequently, broader predictors would predict broader criteria better, and narrower predictors would predict narrower criteria better. In connection with the Five-Factor Model of personality, Ones and Viswesvaran’s approach suggests that the Big Five are better than their

© 2013 Blackwell Publishing Ltd., 9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main St., Malden, MA, 02148, USA

Conscientiousness, Its Facets, and Job Performance facets for predicting both narrow and broad performance criteria. Ashton, Paunonnen, Tett and their associates’ approach suggests that facets are better than factors for predicting narrow performance criteria, and that they add validity over factor validity for broad performance criteria. Finally, Hough et al.’s approach suggests that the Big Five are better for predicting broad performance criteria and the facets are better for predicting narrow performance criteria. A number of studies, both primary and metaanalytic, have been carried out in the last 15 years to examine the merits of these three approaches (perspectives) regarding the prediction of job performance by personality measures. However, in our opinion, no empirical study has unequivocally demonstrated until now the superiority of one approach over the other two. For example, some meta-analyses have examined the validity of the facets and also the validity of the factors, comparing their respective sizes (e.g., Dudley, Orvis, Liebiecki, & Cortina, 2006; Hough, 1992; Mount & Barrick, 1995; Ones & Viswesvaran, 1996; Vinchur, Schippmann, Switzer, & Roth, 1998). Hough (1992) published the first meta-analysis in which the validity of both factors and facets for predicting job performance was examined. She found that achievement (classified as a facet of surgency) was the best predictor across criteria, but she also found that the pattern of the correlations varied across the criteria. Unfortunately, Hough (1992) did not report the validity of conscientiousness as a whole. The second best predictor was dependability. The meta-analysis by Mount and Barrick (1995) concluded that conscientiousness showed larger validity than its facets for predicting overall job performance, and a similar validity to achievement and dependability (two of its facets) for predicting narrower performance criteria. Ones and Viswesvaran (1996), re-analyzing the meta-analytic data reported by Barrick and Mount (1991) and Hough (1992), found that the Big Five showed larger validity than their facets and that a composite measure of two or four of the Big Five dimensions had a larger validity than any single dimension included in the composite. Vinchur et al. (1998) found that conscientiousness and extroversion predicted both objective criteria and performance ratings for salespeople, but that potency (a facet of extraversion) and achievement (a facet of conscientiousness) showed a larger validity than their respective factor. Dudley et al. (2006) examined the validity of conscientiousness and four facets of conscientiousness, including achievement, order, prudence, and dependability, for predicting broad and narrow criteria. They found that the facets of conscientiousness predicted above and beyond conscientiousness and also showed incremental validity, although some results are difficult to interpret from a theoretical point of view. For example, they found that conscientiousness is negative

© 2013 Blackwell Publishing Ltd.

75 for job dedication and task performance, and that cautiousness is negative for overall performance, job dedication, and interpersonal facilitation but positive for counterproductive work behaviors (see Dudley et al.’s table 5). Taken together, these last findings suggest a large amount of multicollinearity among the variables, which is the typical sign of the presence of a common factor within the correlations. A common problem with the five meta-analytic studies commented on above is that meta-analysis does not separate the common variance (factor variance) from the specific variance (facet variance) and, consequently, one cannot conclude that the facet validity estimated in these meta-analyses is not due to the common variance within the respective factor rather than the specific variance of the facet. Primary studies were also conducted by Ashton (1998), Stewart (1999), and Tett et al. (2003). Ashton (1998), in a study using a sample of students (part-time employees) and a scale of counterproductive behaviors at work as a criterion, found that two scales of the Jackson Personality Inventory (JPI, Jackson, 1994), responsibility and risk taking, had higher validities than the Big Five for predicting workplace delinquency (-.40 and -.30, respectively). Ashton assumed that responsibility and risk taking were facets. However, a factor analysis showed that the responsibility scale loaded .56 in agreeableness and .44 in conscientiousness, and that the risk taking scale loaded -.20, -.26, .41, -.44, and .29, in agreeableness, emotional stability, openness, conscientiousness, and extroversion, respectively (see also Paunonen & Jackson, 1996). Therefore, rather than facets (specific factors), the scales of responsibility and risk taking appear to be composite measures. As Asthon did not separate the specific variance from the common variance (factor), one cannot discard the possibility that the validity of the scales was due to the effects of the global personality dimensions rather than the specific facet assessed by the scales. Stewart (1999), using two small samples of salespeople, examined the validity of conscientiousness and two facets, order and achievement, for predicting sales in two stages of employee tenure. He found that conscientiousness showed a consistent relationship with sales in both the transition stage and the maintenance stage, but order was strongly correlated with sales in the transition stage and achievement in the maintenance stage. Both facets showed incremental validity over conscientiousness in the specific stages. Thus, the validity of conscientiousness would be stable and the validity of the facets would be moderated by employee tenure. Nevertheless, Stewart (1999) did not isolate the specific facet variance and, therefore, we can not clearly conclude on the superior validity of the facets. When he pooled the two samples, the facets did not show larger validity than conscientiousness.

International Journal of Selection and Assessment Volume 21 Number 1 March 2013

76 A limitation of previous primary studies is that they used broad measures of job performance and, therefore, the effects of an alignment in breadth between personality and performance measures were not tested. Tett et al. (2003) conducted a study that can be considered an advance with respect to the previous studies, as they assessed job performance on three levels of broadness. Tett et al. (2003) found that the Big Five predicted the broader performance criteria better than they predicted the narrower (specific) performance criteria, and that some facets of the Big Five showed larger validity than the factors for predicting the narrowest level of job performance. These results agreed with their hypotheses. However, despite of its specific merits, this study suffers the same limitation as the previous studies, that is, although narrow facets may show higher correlation with performance than the factor, the way in which they were measured can contain both common and specific variance. In order to test whether facets or factors are better predictors of job performance, it is necessary to use a method that permit the separation of common and specific variance of the personality measures. There are three methods for separating (residualizing) facet variance from factor variance. The first is unrotated principal component analysis (PCA); the second method is unrotated principal factor analysis (PFA); and the third method is the Schmid–Leiman transformation (Schmid & Leiman, 1957). Using PCA, the first unrotated principal component serves as a measure of the personality dimension (e.g., conscientiousness, emotional stability, and so on) and the remaining unrotated principal components as measures of the facets. This method was successfully used by Ree and his associates (Olea & Ree, 1994; Ree & Earles, 1991a; Ree, Earles, & Teachut, 1994) in their studies about the relative superiority of g over specific abilities for predicting job performance and training proficiency. Using PFA, the first unrotated principal factor also serves as a measure of the personality dimension and the remaining unrotated principal factors as measures of the facets. This method is very similar to the PCA with the difference that instead of using the total variance of variables, the common variance is used. Schmid–Leiman’s transformation is a kind of hierarchical factor analysis, which produces a general (higher-order) factor (the personality dimension) and a number of residualized factors (the facets) in which the variance of the higher factor was removed. Each method has advantages and disadvantages. For example, PCA and PFA have the advantage of their mathematical simplicity, but the disadvantage that even though the principal components (and principal factors) have mathematical meaning, they cannot always have psychological meaning. The Schmid–Leiman method is more mathematically complicated and none of the typical statistical packages (e.g., Statistical Package for the Social Sciences [SPSS;


Jesús F. Salgado, Silvia Moscoso and Alfredo Berges SPSS Inc., Chicago, IL]) calculate the residualized factor scores, but the resulting factor and facets have both mathematical and psychological meaning. As a whole, the three methods produce convergent solutions (Ree & Earles, 1991b). None of the previous studies about the bandwidth debate, primary and meta-analytic, used any of these three methods. Together with the methodological limitations pointed out, there are also three additional concerns regarding previous research. A first concern is that some studies used criteria of narrowness that are logically flawed. For example, the studies by Ashton (1998), Ashton et al. (1995), Paunonen (1998; Paunonen et al., 1999; Paunonen & Nicol, 2002), Tett et al. (2003) based the narrowness criterion on the number of items that a personality measure has. A simple proof that this criterion is flawed is that some measures of conscientiousness (e.g., the mini-markers or the NEO Five-Factor Inventory [NEO-FFI]) would be narrower measures than measures typically used for assessing facets in their studies, such as the scales of the JPI (Jackson, 1994) and the Personality Research Form (Jackson, 1997), which contain 20 and 16 items per scale, respectively. As Hunter and Schmidt (2004, pp. 430–439, on conceptual replication) point out, the number of items has essentially two effects, one on the measurement reliability (i.e., the more parallel items, the higher the reliability), and another on the construct validity (i.e., the more parallel items, the higher the construct validity, if the items are measures of the same underlying trait). Consequently, a small number of parallel items would produce smaller reliability, smaller construct validity, smaller observed criterion validity, and larger variance among the validity coefficients found across studies. A second concern is about the definition of facets, factors, and composite variables. According to Ashton, Paunonen, and their associates (see, for instance, Ashton et al., 1995, p.433; Paunonen & Nicol, 2002, p. 164), the Big Five would be composites consisting of a number of facets (specific factors) rather than latent variables. This distinction between a composite (compound traits) and a latent variable is crucial for the bandwidth debate in the case of the Big Five. Schneider et al. (1996, see also Hough & Schneider, 1995; and Schneider & Hough, 1995) used the terms multifaceted traits for referring to factors and facets, and compound traits for referring to composites and linear combinations. A factor is a latent (unobserved) variable that explains the common variance among a number of elements (e.g., traits, facets, and so on). The Big Five are higher-order latent variables which explain the common variance among the facets (lower-order latent variables) or the common variance among traits (basic level of analysis). For its part, a composite is a number of separate entities (e.g., several factors, elements, and so on) that are mathematically combined (i.e., they are added


Conscientiousness, Its Facets, and Job Performance together), but do not have a latent variable that explains the common variance. A composite indicator is formed when individual indicators are compiled in a single index on the basis of an underlying model (e.g., socioeconomic status). In this sense, for example, integrity (as typically assessed by integrity tests) is a weighted linear combination of three or four personality dimensions, but it is not a factor or latent variable. A composite, therefore, is typically a multidimensional measure, while the Big Five are unidimensional (multifaceted) measures (in the same sense g factor is a unidimensional construct). A third limitation of the previous research is that some studies (e.g., Dudley et al., 2006; Paunonen, 1998; Stewart, 1999) did not use the appropriate indices of cross-validated multiple correlation. As Ree and his associates noticed (Olea & Ree, 1994; Ree & Earles, 1991a; Ree et al., 1994) both multiple R and Wherry’s adjusted multiple R grossly overpredict the validity of the set of predictors in a new sample, and, therefore, they are not appropriate estimates for comparing the validity of factors and facets. Several formulas were developed to statistically estimate the cross-validated multiple correlation (e.g., Browne, 1975; Claudy, 1978; Darlington, 1968; Lord, 1950; Nicholson, 1960; Rozenboom, 1978), and its efficiency was empirically demonstrated (e.g., Cotter & Raju, 1982; Kennedy, 1998; Kromrey & Hines, 1996; Schmitt, 1982; Yin, 2001). The problem with the use of the inadequate formula is that the validity of the facets is overestimated and, therefore, the facets are placed at an advantage over the factor. In summary, to our knowledge, no previous study has properly tested the effects of breadth of personality measures on the validity for predicting job performance. Currently, it is not strongly possible to conclude as to the superior validity of the factor or the facets or which of the three main perspectives in the debate is better supported by data.

2. Aims of this study The main objective of this study was to empirically compare the predictions of the three basic perspectives that exist in the bandwidth debate about the relationship between personality and job performance. Ones and Viswesvaran’s (1996) hypothesis would be correct if conscientiousness predicts broad and narrow criteria equally well, and better than its facets. The hypothesis by Schneider et al. (1996) and Hogan and Roberts (1996) would be correct if conscientiousness predicts broad criteria better and facets predict narrow criteria better. Finally, the hypothesis by Paunonen and associates (1998; Ashton, 1998; Paunonen et al., 1999; Paunonen & Nicol, 2002), and Tett et al. (2003) would


77 be correct if facets predict narrow criteria better and show incremental validity over conscientiousness for predicting broad criteria.

3. Method 3.1. Sample Participants were 226 police officers from a large city (N > 750,000) in central Spain. They were 92% male and 8% females. The mean age was 34 at the time of testing. All the subjects possessed a high school education, and over 40% had obtained some college education. All had at least 5 years of experience as a police officer. They applied for a position in a newly developed unit with the Police. The new position has a larger compensation package than the position of police officer. The participants were the total number of applicants for the new position and all of them provided useful data. Therefore the response rate was 100%.

3.2. Personality Conscientiousness was assessed with the Spanish version of the IP/5F (Salgado, 1996, 1998a). This 200-item questionnaire was developed using rational methods (e.g., content analysis; items related to job performance), as well as a factor analysis, in order to measure the Big Five personality dimensions. The items were grouped into 29-Homogenoeous Item Clusters (HIC) based on the item content. The items have three answer alternatives: agreement (A), indecisive (I), and disagreement (D). Internal consistency coefficients for emotional stability (ES), extraversion (E), openness (O), agreeableness (A), and conscientiousness (C) were 0.90, 0.86, 0.80, 0.74, and 0.87, respectively (N = 760). Test– retest reliabilities 1 year later were 0.91, 0.90, 0.79, 0.65, and 0.72 for ES, E, O, A, and C, respectively (N = 95). Convergent and discriminant validity evidence was found using the Spanish versions of the Hogan Personality Inventory (HPI, Hogan & Hogan, 1995), the NEO-FFI (Costa & McCrae, 1992), the Revised NEO Personality Inventory (NEO-PI-R; Costa & McCrae, 1992), the Eysenck Personality Inventory (EPI; Eysenck & Eysenck, 1975), Cuestionario de Estilos Personales (CEP; Moscoso & Salgado, 2004), and the 16PF (Cattell, Eber, & Tatsuoka, 1970). For example, the correlations between IP/5F and the NEO-PI-R factors measuring corresponding constructs were 0.84, 0.80, 0.64, 0.72, and 0.78 for ES, E, O, A, and C, respectively (N = 410), and the correlations with the NEO-FFI were .70, .88, .55, .55, and .58 for ES, E, O, A, and C, respectively (N = 47). The correlation between the EPI and the IP/5F were .46 and .54 for emotional stability and extraversion, respectively (N = 173). In another study (N = 200), the correlations between the HPI and the


78 IP/5F (Salgado, Moscoso, & Lado, 2003) were .75 for emotional stability, .69 and .74 for extroversion (HPI divides extroversion into two subfactors), .85 for openness, .51 for agreeableness, and .67 for conscientiousness. Therefore, it showed excellent convergence with other personality inventories assessing the same personality dimensions. Exploratory and confirmatory factor analyses confirmed the five-factor structure of the IP/5F (Salgado, 1996). The 40 items of Conscientiousness items were grouped in the following six HICs: work effort, order and organization, care and thoroughness, prudence, high performance, high behavior standards. In the present study, Cronbach’s alpha was .90 for conscientiousness. Comparing the standard deviation of conscientiousness in this sample with the standard deviation of the normative sample for police (N = 1,400), the current data have some range restriction (u = .89).

3.3. Criteria In the present study, the immediate supervisor assessed the performance of each police officer on ten behaviorally anchored rating scales (BARS), which were established through job analysis using the critical incident technique (Flanagan, 1954). The BARS included problem-solving, decision making, work organization, job knowledge, productivity, stress resistance, interpersonal competence, initiative, extra-role behavior, and thoroughness. Each BARS was defined by a brief explanation and coupled with a 5-point rating scale ranging from 1, low performance to 5, excellent performance. Anchors were developed for 1, 3, and 5 points. These BARS served to create three performance composites of different broadness. The broader criterion was overall job performance. The intermediate broad criterion was task performance, and the narrow criterion was orderliness. The overall performance composite was formed by unit weighting and adding the 10 BARS. The task performance composite was formed by unit weighting and adding seven BARS: problem-solving, decision making, work organization, job knowledge, productivity, stress resistance, and thoroughness. The orderliness composite was formed by unit weighting and adding two BARS, work organization, and thoroughness. Cronbach’s alpha was .93, .91, and .73 for overall performance, task performance, and orderliness, respectively. Because each participant was rated by the immediate supervisor only, we were not able to estimate an inter-rater reliability coefficient for the present sample. However, in an independent sample of police officers, using the same appraisal instrument, the average inter-rater reliability was .68 (N = 109) (Sáez, 2007, 2011).Therefore, we assume that the inter-rater reliability in the present sample is similar. The immediate supervisors were trained for the purpose of the ratings, and they were assured that the ratings would be used for research purposes only. The


Jesús F. Salgado, Silvia Moscoso and Alfredo Berges supervisors were 22 and they rated 11 police officers on average.

3.4. Procedure All participants were assessed simultaneously in a large room. Together with the personality inventory described earlier, the participants took a cognitive test, a clinical personality test, and they filled out a questionnaire on personal data. After the session of tests, the participants were individually interviewed by a team of psychologists working in the human resource department of the city hall.

3.5. Analyses We conducted a sequential set of statistical analyses. The first step consisted of separating common variance from specific variance in personality variables. In order to do this, we conducted a PCA with an oblique rotation, followed by a Schmid–Leiman transformation. This analysis showed that there were three specific facets and one general factor of conscientiousness. In the first facet, the main loadings (>.9) corresponded to HICs called order and organization, and care and thoroughness. The HICs of high performance and work effort showed main loadings (>.9) in the second facet. In the third facet, the main loadings (>.9) were prudence and behavior norms. Consequently, the first facet was named order, the second one was named industriousness, and the third facet was called self-control. These three labels were also found by Roberts, Chernyshenko, Stark, and Goldberg (2005) in their analyses of seven popular personality inventories. After the Schmid– Leiman transformation, conscientiousness and the residualized facets are fully independent and, therefore, it is possible to calculate the correlation of the facets with the criteria in two ways. First, using the observed (raw) scores and, later, using the residualized scores. The first score type contains factor variance plus facet variance, while the second one is a pure (specific) facet score. In order to calculate the correlations, we used the factor score and the facet score before and after the Schmid–Leiman transformation. The third type of analyses was multiple regression. We conducted two multiple regression analyses, one using the set of facets and another using the factor and the facets. Therefore, we had three validity estimates for each criterion measure. The first one is the factor validity. The second one is the validity of the facets. The third estimate is the validity of the factor supplemented by the facets. Our last type of analysis was to calculate the statistical estimates of the cross-validated multiple correlation and the statistical estimates of the cross-validated square multiple correlation. In order to do this, we used the


Conscientiousness, Its Facets, and Job Performance

79

Browne (1975) formula, following the procedures described by Lautenschlager (1990) and Yin (2001) regarding the computation of Wherry’s estimates included in Browne’s formula. Cattin (1980a, 1980b), Cotter and Raju (1982), Kromrey and Hines (1996), Lautenschlager (1990), and Yin (2001), among others, found that Browne’s formula outperformed other alternative formulas for statistically estimating the cross-validated multiple correlation. These statistical estimates served to compare the respective validity of the factor, facets, and factor plus facets, and they are the essential indicator of the superiority of one type of personality measure over the other. Therefore, they are the key estimator for testing the three perspectives that exist on the relationship between job performance and factor and facets. It must be noted, that we conducted the multiple regression analyses with both the observed correlations and with the correlations corrected by criterion unreliability and range restriction in the predictors. In this last case, as was demonstrated by Schmidt, Hunter, and Larsson (1988; demonstration reproduced by Ree et al., 1994), the sample size that must be used in Wherry’s formula is the effective sample size, because the standard error of the observed correlations is not the same as that of the corrected correlations. The effective sample size is smaller than the sample size of the observed correlations. Ree et al. (1994) reproduced Schmidt et al.’s formula for estimating the effective N.

4. Results Table 1 presents the observed correlations among the three performance criteria with conscientiousness and the three facets, when the raw data are used. As can be seen, conscientiousness predicted significantly the three criteria, with the larger correlation for the narrow criterion. This finding agrees partially with Ones and Viswesvaran’s perspective, but self-control was also predictor of the three criteria, order was predictor of task performance and orderliness, and, as did conscientiousness, they showed a larger correlation with the narrow

criterion. Furthermore, self-control facet showed a larger validity than conscientiousness for the three criteria. These last findings agree with the perspective of Hough and associates (Hough & Schneider, 1995; Schneider et al., 1996), and also with the perspective of Ashton, Paunonen, Tett, and associates (Ashton, 1998; Ashton et al., 1995; Paunonen et al., 1999; Tett et al., 2003). As a whole, the results of this first analysis showed a pattern of correlations that is very similar to that reported in previous primary and meta-analytic studies. Consequently, as in previous studies, these analyses resulted inconclusive. However, as mentioned earlier, in order to have a clear picture of the correlation between the facets and the criteria, it is necessary to residualize the facets. The Schmid–Leiman transformation produces separate and orthogonal scores for conscientiousness, and its three facets, that is, order, industriousness, and self-control. The correlations of the three residualized facets and the three criteria appear in Table 2. In this analysis, the measure of each facet contains specific facet variance only. Therefore, these correlations are really the observed correlations between the facets and the criteria. As can be seen, none of the facets significantly correlated with the criteria, the size of the correlation is negligible, and the sign was negative in some cases. Therefore, it is possible to conclude from this analysis that the facets do not predict job performance, irrespective of whether this is measured with a broad or a narrow measure. This analysis also illustrates the problem of the lack of residualization of the variance of facets in previous studies, because there are important differences in the size of the correlations when raw scores and residualized scores are used. Globally, this second analysis of correlations supported Ones and Viswesvaran’s approach and did not support the other two perspectives. Table 3 presents the multiple correlation of all facets for predicting the three criteria, and it also presents the multiple correlation of conscientiousness supplemented by the three facets for predicting the three criteria. The table also includes the adjusted multiple correlation

Table 1. Observed correlation among conscientiousness, their facets, and job performance criteria Variable

Mean

SD

OJP

TKP

ORDS

CONC

ORD

IND

S-CTL

OJP TKP ORDS CONC ORD IND S-CTL

.00 .00 .00 .00 .00 .00 .00

1.00 1.00 1.00 1.00 1.00 1.00 1.00

(.93) .96** .85** .14* .11 .05 .16**

(.91) .81** .16* .13* .06 .18**

(.93) .20** .17** .08 .20**

(.90) .86** .71** .78**

(.86) .44** .54**

(.69) .29**

(.78)

Note: OJP = overall job performance; IND = industriousness; S-CTL = self-control. *p < .05; **p < .01.


TKP = task

performance;

ORDS = orderliness;

CON = conscientiousness;

ORD = order;


80

Jesús F. Salgado, Silvia Moscoso and Alfredo Berges

Table 2. Observed correlations of job performance criteria with the residualized facets Variable

Mean

SD

OJP

TKP

ORDS

CONC

ORD

IND

S-CTL

OJP TKP ORDS CONC ORD IND S-CTL

.00 .00 .00 .00 .00 .00 .00

1.00 1.00 1.00 1.00 1.00 1.00 1.00

(.93) .96** .85** .14* -.02 -.07 .08

(.91) .81** .16* -.01 -.08 .09

(.93) .20** .00 -.08 .08

(.90) .00 .00 .00

(.86) -.47** -.42**

(.69) -.61**

(.78)

Note: OJP = overall job performance; IND = industriousness; S-CTL = self-control. *p < .05; **p < .01.

TKP = task

performance;

Table 3. Multiple regression of conscientiousness and its facets for predicting job performance criteria Variable OJP C F C+F TKP C F C+F ORD C F C+F

R

R2

R2adj

.142 .087 .167

.020 .008 .028

– -.001 .015

.159 .094 .185

.025 .009 .034

–

.195 .088 .214

.038 .008 .046

– -.001 .033

.000 .021

Note: R2adj = adjusted square multiple correlation using Wherry’s formula 1. OJP = overall job performance; TKP = task performance; ORD = orderliness.

calculated with Wherry’s formula. As can be seen, the multiple correlation of the composite of facets is smaller that the bivariate correlation of conscientiousness for the three criteria. However, a composite consisting of conscientiousness supplemented by the three facets resulted in a larger multiple correlation than the observed correlation of conscientiousness alone for the three criteria. Based on these analyses, someone could erroneously conclude that the facets showed incremental validity over conscientiousness. Nevertheless, in order to reach a robust conclusion, it is necessary to statistically estimate the cross-validated multiple regression using Browne’s (1975) formula. Table 4 shows, in columns 6 and 8, the statistical estimate of the cross-validated multiple correlation of the facets and the statistical estimate of the crossvalidated multiple correlation of conscientiousness plus the facets for predicting the three criteria. In the case of the facets, the statistical estimate of the cross-validated multiple correlation is zero for the three criteria, which means that the facets are useless for predicting job performance in the present study. The statistical estimate of the cross-validated multiple correlation is positive


ORDS = orderliness;

CON = conscientiousness;

ORD = order;

and relatively large for predicting orderliness in the case of conscientiousness plus facets. However, as the observed validity of conscientiousness alone is larger than the statistical estimate of the cross-validated multiple correlation of conscientiousness plus facets, the conclusion is that the facets did not show incremental validity. These analyses were repeated using the correlation corrected by measurement error in predictor and criteria as well as direct range restriction in the predictor. The results are similar to the previous one, with the only remarkable difference of the validity magnitude. Both observed and corrected statistical estimate of the crossvalidated multiple correlations, once again, clearly supported Ones and Viswesvaran’s perspective and rejected the perspectives by Hough and associates and Ashton, Paunonen, Tett and associates. It is interesting to note that conscientiousness showed true validity coefficients of .25, .28, and .37 for predicting overall job performance, task performance, and orderliness. These values have also relevant implications. First, the validity for predicting overall job performance was similar to the validity found by Barrick and Mount (1991) and Salgado (1997). Second, according to these results, conscientiousness predicted narrow criteria better than broad ones, if the narrow criterion is content-aligned with conscientiousness. Finally, as a note about the convergence between the different analytical methods, the analyses of correlations and multiple correlations were also conducted using unrotated PCA and produced the same results. Therefore, the method of estimating factor and facet variance does not alter the findings.

5. Discussion The main objective of this research was to empirically test the predictions of the three existing perspectives on the relationship between personality measures and job performance, in connection with the breadth of the measures. In this respect, this study has made several unique contributions to the bandwidth debate. The most important contribution is to show that only con-


Conscientiousness, Its Facets, and Job Performance

81

Table 4. Results of validity and incremental analyses for the three performance criteria Criterion Observed correlation Overall performance Task performance Orderliness Average True correlations Overall performance Task performance Orderliness Average

(1) r

(2) r2

(3) Rf

(4) Rf + d

(5) R2cvf

(6) Rcvf

(7) R2cvf + d

(8) Rcvf + d

(9) D

.142 .159 .195 .165

.020 .025 .038 .028

.087 .094 .088 .090

.167 .185 .214 .189

-.014 -.013 -.014 -.014

.000 .000 .000 .000

-.013 .006 .017 .003

.000 .070 .131 .067

-.142 -.089 -.064 -.098

.248 .284 .366 .299

.062 .081 .134 .092

.141 .154 .143 .146

.302 .343 .412 .352

-.036 -.033 -.036 -.035

.000 .000 .000 .000

.002 .023 .083 .036

.042 .150 .288 .160

-.206 -.134 -.078 -.139

Note: r = conscientiousness validity; r2 = squared conscientiousness validity; Rf = multiple correlation of facets; Rf + d = multiple correlation of dimension plus facets; R2cvf = statistical estimate of the cross-validated square multiple correlation of facets; Rcvf = statistical estimate of the cross-validated multiple correlation of facets; R2cvf + d = statistical estimate of the squared cross-validated multiple correlation of dimension plus facets; Rcvf + d = statistical estimate of the cross-validated multiple correlation of dimension plus facets; D = incremental validity of facets over dimensions (column 8 minus column 1).

scientiousness predicts job performance. The facets of conscientiousness, when the common (factor) variance is excluded and, therefore, the facet measure consists of only specific variance, do not predict job performance. The findings of some previous studies mentioned in the introduction to this article, which suggest that facets can be better predictors of job performance than conscientiousness, appear to be an artifactual result produced by methodological limitations of their statistical analyses. As the analytical methods used in those studies do not allow the separation of common and specific variance of the scales of facets included in many popular personality inventories, the resulting correlations included variance of conscientiousness, which subsequently produced significant correlations between the facets and job performance. The second unique contribution is to show that a composite of facets does not predict job performance either. In other words, an aggregate of several measures of independent facets (i.e., order, industriousness, and self-control), when the common (conscientiousness) variance is excluded, does not result in a valid composite, which can replace a measure of conscientiousness. Only when the measures of facets contain conscientiousness variance does it make sense to aggregate them in a composite. In this last case, the composite would be equally valid as a measure of conscientiousness. The third unique contribution is to show that the facets do not show incremental validity over conscientiousness. This is another important contribution as it illustrates that it does not make sense to include conscientiousness measures and measures of facets together as predictors of job performance. This is also relevant from a theoretical point of view, as it means that a theory of job performance does not require the


inclusion of conceptual elements linked to facets and does not require hypotheses about the relationships between performance and personality facets. The fourth unique contribution is to show that conscientiousness predict both broad and narrow job performance criteria. The multidimensional nature of job performance does not appear to affect the validity of conscientiousness. We speculate that this is due to the fact that conscientiousness is a quality required by all occupations. It is difficult to think of a job that does not require order, perseverance, prudence, self-control, effort, or self-motivation. Related to this contribution is the fact that conscientiousness seems better to predict criteria that are conceptually aligned with the personality dimension content (e.g., orderliness). In this sense, if job performance measures contain dimensions conceptually related with conscientiousness, the validity of this personality factor could be larger, but this issue is independent of whether the measure of performance is broad or narrow. Our findings have important implications for the three main perspectives that have been suggested about the superior validity of factor over facets and vice versa. Consequently, the findings have important implications for the theory. Our results clearly support Ones and Viswesvaran’s (1996) perspective and do not support either Ashton, Paunonen, and Tett’s perspective (e.g., Ashton, 1998; Paunonen, 1998; Paunonen et al., 1999; Tett et al. 2003), nor the approach by Hough and associates (e.g., Hough & Schneider, 1995; Schneider & Hough, 1995) and the approach by Hogan and Roberts (1996). Nevertheless, these last two perspectives claim that the validity coefficients would be larger if the criterion and personality measures were aligned, not only in terms of breadth but also in terms of content. With regard to this last point, our results


82 suggest that aligning criteria and personality measures in terms of their respective content can produce a large validity coefficient. Warr (1999) found these kind of results in a previous study. The present research is also relevant for showing that the Schmid–Leiman transformation is a very useful method for testing the different perspectives in the bandwidth debate. This transformation allows the residualization of the specific variance of the facets and, therefore, to estimate their specific relationships with the criteria. Due to the fact that the Schmid–Leiman transformation produces a hierarchical factor solution, and that it produces lower factors with psychological meaning (unlike unrotated PCA and PFA), this method should be the preferred one in future studies on this debate. This study also has some limitations, which should be taken into account. A first limitation is that we have used one specific job and, therefore, other studies with different jobs should be carried out in order to confirm our findings. Second, our criteria were supervisor ratings of job performance and our conclusions are valid only for this kind of criteria. Studies should also be conducted with other organizational criteria, such as objective measures, output, counterproductive behaviors, training proficiency, and career progress, for instance. A third limitation is that the sample, although large enough (see Salgado, 1998b) for conducting a validity study, pertain to a specific organization, and the generalization of validity can not be taken for granted. It is possible that idiosyncratic characteristics within this organization could have some effects on the results and, therefore, other studies with other law enforcement organizations would be desirable. We also cannot conclude that these findings can be generalizable to other personality dimensions (e.g. emotional stability, extroversion) as it is possible that a different pattern of relationships could be found between these factors and the facets and between the facets and the criteria. Future studies should examine this issue. The findings of this study also have implications for the practice of personnel selection. If one of the purposes of personnel selection procedures is to test the applicants’ fit to the job at the cheapest possible price then, based on our results, we recommend using broad and large measures of conscientiousness because they predict broad and narrow performance measures and avoiding using short scales of facets. The higher reliability of the large and broad measures of conscientiousness will produce better hiring decisions because of their superior validity. In summary, this research has demonstrated that the bandwidth debate can be solved using the proper methodological analyses. This initial study strongly supported the perspective of Ones and Viswesvaran (1996) that conscientiousness predicts broad and narrow criteria


Jesús F. Salgado, Silvia Moscoso and Alfredo Berges and that the facets do not add incremental validity over conscientiousness.

Acknowledgements We would like to thank the invaluable help of Joan P. Ferrando regarding the Schmid–Leiman analysis included in this article. We also thank Frank Schmidt and Malcolm J. Ree for their comments to previous versions of this article. The research reported in this article was partially supported by Grant SEJ-2008-3070 from the Ministry of Science and Innovation (Spain) to Jesús F. Salgado.

References Ashton, M. C. (1998). Personality and job performance: The importance of narrow traits. Journal of Organizational Behavior, 19, 289–303. Ashton, M. C., Jackson, D. N., Paunonen, S. V., Helmes, E., & Rothstein, M. G. (1995). The criterion validity of broad factor scales versus specific factor scales. Journal of Research in Personality, 29, 432–442. Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1–26. Browne, M. W. (1975). Predictive validity of a linear regression equation. British Journal of Mathematical and Statistical Psychology, 28, 79–87. Cattell, R. B., Eber, H. W., & Tatsuoka, M. M. (1970). Handbook for the Sixteen Personality Factor Questionnaire (16 PF). Champaign, IL: Institute for Personality and Ability Testing. Cattin, P. (1980a). Estimation of the predictive power of a regression model. Journal of Applied Psychology, 65, 407–414. Cattin, P. (1980b). Note on the estimation of the squared cross-validated multiple correlation of the regression model. Psychological Bulletin, 87, 63–65. Claudy, J. G. (1978). Multiple regression and validity estimation in one sample. Applied Psychological Measurement, 2, 595– 607. Costa, P. T., & McCrae, R. R. (1992). Revised NEO personality inventory (NEO-PI-R) and NEO five factor inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources. Cotter, K. L., & Raju, N. S. (1982). An evaluation of formulabased population squared cross-validity estimates and factor score estimates in prediction. Educational and Psychological Measurement, 42, 493–519. Darlington, R. B. (1968). Multiple regression in psychological research and practice. Psychological Bulletin, 69, 161–182. Dudley, N. M., Orvis, K. A., Liebiecki, J. E., & Cortina, J. (2006). A meta-analytic investigation of conscientiousness in the prediction of job performance: Examining the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology, 91, 40–57. Eysenck, H. J., & Eysenck, S. B. G. (1975). Manual for the Eysenck personality questionnaire. London: Hodder & Stoughton.


Conscientiousness, Its Facets, and Job Performance Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327–358. Hogan, J., & Roberts, B. W. (1996). Issues and non-issues in the fidelity–bandwidth trade-off. Journal of Organizational Behavior, 17, 627–637. Hogan, R. T., & Hogan, J. (1995). Manual for the hogan personality inventory. Tulsa, OK: Hogan Assessment Systems. Hough, L. M. (1992). The ‘Big Five’ personality variables: Confusion construct-description versus prediction. Human Performance, 5, 139–155. Hough, L. M., & Schneider, R. J. (1995). Personality traits, taxonomies, and applications in organizations. In K. R. Murphy (Ed.), Individual differences and behavior in organizations (pp. 31–88). San Francisco, CA: Jossey-Bass. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis. Correcting error and bias in research findings (2nd ed.) Newbury Park, CA: Sage. Jackson, D. N. (1994). Jackson personality inventory – revised manual. Port Huron, MI: Research Psychologists Press. Jackson, D. N. (1997). Personality research form manual. Port Huron, MI: Research Psychologists Press. Kennedy, E. (1998). Estimation of the squared cross-validity coefficient in the context of best subset regression. Applied Psychological Measurement, 12, 231–237. Kromrey, J. D., & Hines, C. V. (1996). Estimating the coefficient of cross-validity in multiple regression: A comparison of analytical and empirical methods. The Journal of Experimental Education, 64, 240–266. Lautenschlager, G. J. (1990). Sources of imprecision in formula of cross-validates multiple correlation. Journal of Applied Psychology, 75, 460–462. Lord, F. M. (1950). Efficiency of prediction when a regression equation from one sample is used in anew sample. (Research Bulletin No. 50-40). Princeton, NJ: Educational Testing Service. Moscoso, S., & Salgado, J. F. (2004). ‘Dark side’ personality styles as predictors of task, contextual, and job performance. International Journal of Selection and Assessment, 12, 356–362. Mount, M. K., & Barrick, M. R. (1995). The Big Five personality dimensions: Implications for research and practice in human resources management. In K. M. Rowland & G. Ferris (Eds.), Research in personnel and human resource management. (Vol. 13, pp. 153–200). Greenwich, CT: JAI Press. Nicholson, G. E. (1960). Prediction in future samples. In L. Olkin, S. G. Ghurye, W. Hoefding, W. G. Madow, & H. B. Mann (Eds.), Contribution to probability and statistics (pp. 322–330). Stanford, CA: Stanford University Press. Olea, M. M., & Ree, M. J. (1994). Predicting pilot and navigator criteria: Not much more than g. Journal of Applied Psychology, 79, 845–851. Ones, D. S., & Viswesvaran, C. (1996). Bandwidth–fidelity dilemma in personality measurement for personnel selection. Journal of Organizational Behavior, 17, 609–626. Paunonen, S. V. (1998). Hierarchical organization of personality and prediction of behavior. Journal of Personality and Social Psychology, 74, 538–556. Paunonen, S. V., & Jackson, D. N. (1996). The Jackson personality Inventory and the Five-factor Model of Personality. Journal of Research in Personality, 30, 42–59.


83 Paunonen, S. V., & Nicol, A. A. A. M. (2002). The personality hierarchy and the prediction of work behaviors. In B. W. Roberts & R. Hogan (Eds.), Personality psychology in the workplace (pp. 161–191). Washington, DC: American Psychological association. Paunonen, S. V., Rothstein, M. G., & Jackson, D. N. (1999). Narrow reasoning about the use of broad personality measures for personnel selection. Journal of Organizational Behavior, 20, 389–405. Ree, M. J., & Earles, J. A. (1991a). Predicting training success: Not much more than g. Personnel Psychology, 44, 321– 332. Ree, M. J., & Earles, J. A. (1991b). The stability of g across different methods of estimation. Intelligence, 15, 271–278. Ree, M. J., Earles, J. A., & Teachut, M. S. (1994). Predicting job performance. Not much more than g. Journal of Applied Psychology, 79, 518–524. Roberts, B. W., Chernyshenko, O. S., Stark, S., & Goldberg, L. R. (2005). The structure of conscientiousness: An empirical investigation based on seven major personality questionnaires. Personnel Psychology, 58, 103–139. Rozenboom, W. W. (1978). Estimation of cross-validated multiple correlation: A clarification. Psychological Bulletin, 85, 201–211. Sáez, J. (2007). Diseño y validación de una entrevista conductual estructurada para la selección de agentes de policía local (Design and validation of a behavioral structured interview for selecting local police officers). Revista de Psicología del Trabajo y de las Organizaciones, 23, 57–74. Sáez, J. (2011). Determinantes del desempeño individual en agentes de policia local. [Determinants of individual performance in local police officers]. Unpublished Doctoral Dissertation. Santiago de Compostela: University of Santiago de Compostela. Salgado, J. F. (1996). Análisis exploratorio y confirmatorio del IP/5F [Exploratory and confirmatory analysis of the IP/5F]. Psicológica, 17, 353–366. Salgado, J. F. (1997). The five-factor model of personality and job performance in the European Community. Journal of Applied Psychology, 82, 30–43. Salgado, J. F. (1998a). Manual Técnico del Inventario de Personalidad de Cinco Factores IP/5F [Technical manual for the Five-factor Personality Inventory IP/5F]. Santiago de Compostela: Tórculo. Salgado, J. F. (1998b). Sample size in validity studies of personnel selection. Journal of Occupational and Organizational Psychology, 71, 161–164. Salgado, J. F., Moscoso, S., & Lado, M. (2003). Evidence of cross-cultural invariance of the Big Five personality dimensions in work settings. European Journal of Personality, 17, 67– 76. Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53–61. Schmidt, F. L., Hunter, J. E., & Larsson, M. (1988). General cognitive ability vs. general and specific aptitudes in the prediction of training performance: Some preliminary findings. Paper presented at the 96th Annual Convention of the American Psychological Association, Atlanta, GA. Schmitt, N. (1982). Formula estimation of cross-validated multiple correlation. Paper presented at the annual meeting of the American Psychological Association, Washington,


84 DC. (ERIC Document Reproduction Service No. ED 227 137). Schneider, R. J., & Hough, L. M. (1995). Personality and industrial/organizational psychology. In C. L. Cooper & I. T. Robertson (Eds.), International review of industrial and organizational psychology (Vol. 10, pp. 75–129). Chichester: Wiley. Schneider, R. J., Hough, L. M., & Dunnette, M. D. (1996). Broadsided by broad traits: How to sink science in five dimensions or less. Journal of Organizational Behavior, 17, 639– 655. Stewart, G. L. (1999). Trait bandwidth and stages of job performance: Assessing differential effects for conscientiousness and its subtraits. Journal of Applied Psychology, 84, 959– 968.


Jesús F. Salgado, Silvia Moscoso and Alfredo Berges Tett, R. P., Steel, J. R., & Beauregard, R. S. (2003). Broad and narrow measures on both sides of the personality–job performance relationship. Journal of Organizational Behavior, 24, 335–356. Vinchur, A. J., Schippmann, J. S., & Switzer, F. S. III, & Roth, P. L. (1998). A meta-analytic review of predictors of job performance for salespeople. Journal of Applied Psychology, 83, 586–597. Warr, P. (1999). Logical and judgmental moderators of the criterion-related validity of personality scales. Journal of Occupational and Organizational Psychology, 72, 187–204. Yin, P. (2001). Estimating R2 Shrinkage in multiple regression: A comparison of different analytical methods. Journal of Experimental Education, 69, 203–224.
