Following the Ants

Corrected proof – not final Original Article

Following the Ants Development of Short Scales for Proactive Personality and Supervisor Support by Ant Colony Optimization Anne B. Janssen,1 Martin Schultze,2 and Adrian Grötsch3 1

Jacobs University Bremen, Germany, 2Freie Universität Berlin, Germany, 34A-SIDE GmbH Braunschweig, Germany

Abstract. Employees’ innovative work is a facet of proactive work behavior that is of increasing interest to industrial and organizational psychologists. As proactive personality and supervisor support are key predictors of innovative work behavior, reliable, and valid employee ratings of these two constructs are crucial for organizations’ planning of personnel development measures. However, the time for assessments is often limited. The present study therefore aimed at constructing reliable short scales of two measures of proactive personality and supervisor support. For this purpose, we compared an innovative approach of item selection, namely Ant Colony Optimization (ACO; Leite, Huang, & Marcoulides, 2008) and classical item selection procedures. For proactive personality, the two item selection approaches provided similar results. Both five-item short forms showed a satisfactory reliability and a small, however negligible loss of criterion validity. For a twodimensional supervisor support scale, ACO found a reliable and valid short form. Psychometric properties of the short version were in accordance with those of the parent form. A manual supervisor support short form revealed a rather poor model fit and a serious loss of validity. We discuss benefits and shortcomings of ACO compared to classical item selection approaches and recommendations for the application of ACO. Keywords: Ant Colony Optimization algorithm, item selection, innovative work behavior, proactive personality, supervisor support

The present paper addresses a frequent challenge in organizational and industrial psychology – collecting substantial amounts of reliable data in a short period of time. Usually, data on work-related constructs are gathered at the workplace and during work hours. Companies hence often restrict the time and costs the data collection should take. In order to receive the relevant information from employees and supervisors, investigators tend to employ short though valid and reliable measures. Scale development and scale shortening of reliable measures have therefore been of interest in the literature for quite some time (e.g., MacDonald & Paunonen, 2002; Stocking & Swanson, 1993). Most researchers use traditional approaches rooted in classical test theory or item response theory (Rammstedt & Beierlein, 2014). In contrast to previous studies (e.g., Rogers, Creed, Searle, & Hartung, 2010; Wester, Vogel, O’Neil, & Danforth, 2012), we employed Ant Colony Optimization (ACO) – an item selection approach originally introduced by Leite, Huang, and Marcoulides (2008). ACO controls for other approaches’ limitation that shortening original scales often comes with a loss of validity (e.g., Rammstedt & Beierlein, 2014). Using ACO, we generated two short scales of proactive personality and supervisor support. These scales allow practitioners to measure individual differences that are frequently included in employee surveys Ó 2015 Hogrefe Publishing

but which so far relied on time-consuming original scales (e.g., Crant, 1995; Eisenberger, Huntington, Hutchison, & Sowa, 1986). Shorter scales reduce questionnaire length, which can improve data quality by decreasing the cognitive load from long questionnaires (Galesic, 2006; Rolstad, Adler, & Rydén, 2011; see Bradburn, 1978). Also, lower nonresponse rates and less uniform answers have been found for shorter questionnaires (Crawford, Couper, & Lamias, 2001; Galesic & Bosnjak, 2009). Based on those employee ratings, it is possible to predict a wide range of individual and organizational outcomes. An extended inventory of thoroughly developed short scales that cover organization-focused individual and contextual factors will further both the understanding of work-related psychological processes and their practical implementation.

Innovative Work Behavior As the domain to apply the ACO approach to, we chose innovative work behavior, defined as the extent to which employees generate new and novel ideas, disseminate those ideas and the ideas of others, implement innovations themselves, or help others to do so (Axtell et al., 2000; Ng, Feldman, & Lam, 2010). As innovation has become European Journal of Psychological Assessment 2015 DOI: 10.1027/1015-5759/a000299

2

Corrected proof – not final A. B. Janssen et al.: Scale Short Form Development by ACO

more critical to a firm’s survival in the long run (Choi & Chang, 2009; Sampson, 2007), innovative work behavior has come to be seen as a more critical component of an employee’s performance contribution to a firm (Welbourne, Johnson, & Erez, 1998; Yuan & Woodman, 2010). Hence, innovative work behavior gives substantial information on employees’ overall job performance. Crant (2000) suggests that individual innovation is predicted by proactive personality and supervisor support (see also Bindl & Parker, 2010). Proactive personality refers to ‘‘one who is relatively unconstrained by situational forces and who effects environmental change’’ (Bateman & Crant, 1993, p. 105). Seibert, Kraimer, and Crant (2001) reported a positive association between proactive personality and individual innovation (see also Parker & Collins, 2010). Similar results were found by Parker, Williams, and Turner (2006) who also tested the relation between proactive personality and innovation mediated by flexible role orientation and role breadth self-efficacy. Proactive personality was positively related to individual innovation with flexible role orientation and role breadth self-efficacy mediating this relationship. In addition to individual differences, Crant (2000) assumed that the organizational environment including supervisor support affects innovative work behavior. In terms of individual innovation, the influence of supervisor support is discussed controversially in the literature. Ohly, Sonnentag, and Pluntke (2006) reported a negative effect of supervisor support on idea suggestion, whereas Axtell and colleagues (2000) did not find any significant relationship between the two variables. On the other hand, Axtell et al. (2000) found empirical support for their assumption that supervisor support and idea implementation are positively associated.

Short Scale Development Short scale development strategies can be grouped into three major categories: statistics-driven, judgmental, and ad hoc (e.g., Kruyen, Emons, & Sijtsma, 2013). Within the statistics-driven category, the most often used approaches are rooted in classical test theory (CTT) and use reliability estimates, factor loadings, item-total correlations, or the results of factor analyses as indicators of item quality, opting to retain those items that show the greatest extent of unidimensionality. Additionally, item difficulties and the distribution of items in a sample are often used as selection criteria pertaining to the internal structure, while correlations of items with external criteria can also be used to determine which items are best retained in short forms. However, many of these statistics suffer from the sample dependence of the obtained estimates thus making cross validation in a new sample necessary. This can be avoided by using techniques based on item response theory (IRT), which implement stricter assumptions about item characteristics but result in sample-independent item statistics. European Journal of Psychological Assessment 2015

In case of reducing an already validated scale, classical approaches potentially have a substantial downside: poor items can be detected through item reliability or item-total correlation (i.e., item discrimination). However, depending on the excluded single item, the statistics for the remaining items and the overall test will change. Hence, a stepwise item selection for the development of a short form will result in different sets of items depending on the order of eliminated items. Stepwise selection is therefore unlikely to provide the best solution for a scale’s short version. This problem can be avoided when using IRT items since these are independent and omitting an item does not influence the item statistics of the remaining items. The resulting statistical quality measures, whether they stem from analyses based on CTT or IRT, should be combined with aspects derived from judgmental and, possibly, ad hoc strategies to provide a short form with the best reliability, internal structure, and validity possible (e.g., Kruyen et al., 2013; Stanton, Sinar, Balzer, & Smith, 2002; Ziegler, 2014). Ziegler, Kemper, and Kruyen (2014) provide a summary of the five common mistakes within this approach to short-form construction and solutions to them, while Maloney, Grawitch, and Barber (2011) suggest an item selection strategy within this framework and Stanton and colleagues (2002) provide an overview of best practices as well as possible compromises, when these cannot be met. In this article, we use an alternative approach based on ACO (Leite et al., 2008). This approach addresses some major challenges occurring during item selection for short-form development by establishing a measurement model in the framework of confirmatory factor analysis (CFA) and repeatedly estimating this measurement model with different sets of selected items. The following section gives a brief introduction to this approach. It should be noted, that this approach will select items based on the statistical properties of the short version and the items included in it. Theoretical considerations, such as the relative importance of an item, must be investigated separately to ensure that the shortened scale also meets validity criteria that are not easily included in the ACO selection process.

Ant Colony Optimization ACO was first proposed by Colorni, Dorigo, and Maniezzo (1991) as a meta-heuristic for solving a wide array of combinatorial problems and is applicable in many cases in which a problem has many possible solutions with varying degrees of quality. This approach does not require an optimal solution to exist, but instead focuses on finding a solution within the set of possible solutions that best meets certain criteria. One type of problem with many possible solutions – some more adequate than others – is the construction of shortened scales. In this context any combination of selected items is a possible solution and these possible solutions vary in their degree of adequacy. This adequacy can be comprised of many different facets. As pointed out above, traditional approaches determine the adequacy of shortened scales via properties of the items. Ó 2015 Hogrefe Publishing


As a significant advantage, this approach enables adaptive item selection based on multiple criteria simultaneously. These criteria include classic criteria such as scale reliability or item discrimination, but ACO also allows considering the short form’s correlation with an external criterion or the correlation between subscales of the parent form. In their original article, Leite et al. (2008) included the regression on an external variable to ensure that the shortened scale optimally depicts a known relationship to an external criterion. The ACO framework has its name from an analogy of the employed problem-solving technique to the behavior of colonies of ants. In their search of the shortest way between the formicary and a food source and back, different ants randomly choose a path, leaving a pheromone trace on the ground. On the shortest path, an ant needs the least time, so that the pheromone trace accumulates faster than on any other track and becomes more intensive (Deneubourg & Goss, 1989; Deneubourg, Pasteels, & Verhaeghe, 1983). As Watkins (1964) reported, the likelihood for an ant following a path increases with the intensity of the pheromone trace. Therefore, the shortest track between the formicary and the food source is more likely to be taken than any other path. Finally, most ants will take the shortest path, whereas other tracks are rarely used. The idea of rewarding the most promising alternative is now applied to the item selection process. To apply this technique to constructing shortened scales, a pheromone function must be defined. This function determines the degree to which the selections made in one short scale are rewarded and therefore become more likely selections in the next iteration. This function can contain any number of criteria, such as the overall model fit of the measurement model, reliability of the items, and correlations or regressions with other variables. At the start of the algorithm, a set of items for a short version is randomly selected, with each item having exactly the same selection likelihood. In the following, this likelihood is referred to as the pheromone level and in the first iteration this pheromone level is set equal for all items. The examined short version attains a score based on its quality of fitting the defined criteria. Due to the origin of the approach as an emulation of the behavior of real ants, this score is called pheromone (denoted u in this paper). The pheromone is then assigned to all items that were selected in this short version. In order to reduce the risk of nonoptimal solutions, several sets of items are compared simultaneously. The set fitting the required criteria best is called best-so-far solution. Following Leite et al. (2008), the pheromone level of all other items not belonging to the best-so-far solution is reduced by 5%. Thus, items that were part of a short version that fulfilled the set criteria well are more likely to be selected in the next iteration than items that were part of a short version that did not fulfill the criteria well. To stay within the analogy to ants’ behavior: the pheromones of the not-selected items are evaporating. Since selection likelihoods for poor items are only reduced without omitting them completely, this procedure also avoids the abovementioned problem with changing item-total correlations of the remaining items when excluding single items. In this Ó 2015 Hogrefe Publishing

3

manner (random selection of items, evaluation of the shortened version, update of the probability) a short version of a scale is constructed iteratively. This procedure is repeated for at least 100 iterations. However, each time the bestso-far solution is updated, the count starts again with 1. If a best-so-far solution is not improved after 100 iterations, this solution is referred to as the overall best solution. In the present study, we aimed at constructing useable, reliable short scales via the ACO approach. While Leite and colleagues (2008) emphasized the value of ACO for the reduction of large and complex scales, we were also interested in constructing a functional short form of an already relatively short original scale. Thus, we conducted ACO short-form construction for a one-dimensional and relatively short original scale and a more complex twodimensional scale with numerous items. Due to their high practical relevance in the IO research, we employed scales for proactive personality and supervisor support predicting innovative work behavior. In addition to the presentation of two ACO short scales that will enable organizations to assess these constructs in a fraction of the previous assessment time, we discuss the added value of this approach compared to classical items’ selection procedures and the shortcomings that come along with an algorithm-based scale reduction.

Materials and Methods We conducted two empirical studies. In the first study, we aimed at developing short forms of two reliable and valid scales commonly used in the organizational research – one to assess proactive personality and a second one for supervisor support. In order to determine the added value of the ACO procedure compared to classical item selection, we developed both an ACO short form and a manual short form of the two scales. The second study used a new sample to validate all developed short scales.

Participants In a first step, we collected online data of white-collar workers (mean age = 42.69 years, SD = 10.98, 61.2% females). In total, N = 279 observations were included in the analyses. In order to validate the short scales, we used a second independent sample of white-collar workers. Participants were N = 155 employees with a mean age of 42.27 years, SD = 10.38, and 51.3% females.

Materials Aimed at demonstrating the benefits of ACO for scale reduction, we chose a short and one-dimensional scale and a rather complex two-dimensional scale with numerous items. In the first study, we used the supervisor support scale developed by Kidd and Smewing (2001). Three of European Journal of Psychological Assessment 2015

4


the scale’s five dimensions focused on the supervisor’s general social skills (trust and respect, interpersonal skills) or technical skills (expertise). However, since the main interest of the present study was in the prediction of individual innovation and employees’ development, we only employed the supervisor support dimensions career promotion (12 items; Cronbach’s a = 0.95, glb = 0.95) and feedback and goal setting (7 items; Cronbach’s a = 0.89, glb = 0.88). Kidd and Smewing (2001) reported a correlation of r = 0.79 between the two subscales; in our sample, we found a correlation of r = 0.84. Proactive personality was assessed by the one-dimensional 10-item proactive personality scale by Seibert, Crant, and Kraimer (1999; Cronbach’s a = 0.84, glb = 0.86). In order to test the predictive validity of the two shortened scales, we employed Janssen’s (2000) innovative work behavior scale as the external criterion. The three-dimensional scale included the subscales idea generation (Cronbach’s a = 0.89, glb = 0.88), idea promotion (Cronbach’s a = 0.90, glb = 0.89), and idea realization (Cronbach’s a = 0.85, glb = 0.86) with three items each. Reliability for the overall scale was Cronbach’s a = 0.95 and glb = 0.95. In the second sample, we applied the manually and ACOshortened six-item supervisor support scales with the subscales career promotion (three items; Cronbach’s a = 0.88, glb = 0.88 for ACO short scale; Cronbach’s a = 0.79, glb = 0.78 for manual short scale) and feedback and goal setting (three items; Cronbach’s a = 0.77, glb = 0.74 for ACO short scale; Cronbach’s a = 0.78, glb = 0.77 for manual short scale) and the five-item proactive personality scales (Cronbach’s a = 0.79, glb = 0.78 for ACO short scale; Cronbach’s a = 0.81, glb = 0.82 for manual short scale). All items of the ACO and classically reduced forms and their factor loadings are presented in the Appendix. In addition, we assessed innovative work behavior again. All collected data were self-ratings. Supervisor ratings on the employee’s innovative work behavior were not assessed because especially idea generation as one major aspect of innovative behavior is a rather cognitive process that the supervisor does not necessarily become aware of. Furthermore, the employee is rather able to rank certain innovative activities since he or she has comprehensive information about the intention, history, context, and further background of the own innovative work behavior (Janssen, 2001).

ACO Item Selection We chose the Ant Colony Optimization item selection approach to shorten the original scales. A confirmatory factor analysis (CFA) framework was chosen to evaluate the quality of the shortened scales. In the case of the supervisor support scale, items were modeled to measure the theoretical subscales career promotion and feedback and goal setting without any cross-loadings and in the case of the proactive personality scale items were modeled to measure a single latent variable unidimensionally. We set the number of items for the reduced scales so as to maximize the time saved by employing the short forms while adequately identifying the measurement models. European Journal of Psychological Assessment 2015

We used Leite et al.’s (2008) algorithm and adapted it onto our requirements. Even though the user is free to define any selection criteria, we recommend considering model fit indices commonly used to rate a scale’s or a measurement model’s quality. Leite et al. (2008) included the CFI, TLI, and the RMSEA. Both CFI and TLI compare the target model chi-square to the null model chi-square and give hence similar information. In order to keep the algorithm parsimonious, we chose the CFI due to its relative independence of sample size (Fan, Thompson, & Wang, 1999) and the RMSEA to contribute to the overall pheromone of a random short scale. The pheromone contribution of the RMSEA was defined as uRMSEA ¼ 1

1 1 þ e5100RMSEA

ð1Þ

and the pheromone contribution of the CFI was defined as uCFI ¼

1 : 1 þ e4550CFI

ð2Þ

In addition to model fit and different from Leite et al. (2008), reliability of the single items was chosen as a criterion to ensure short versions that measure constructs as reliably as possible. The pheromone contribution of the average reliability of the items was defined as uRel ¼

1 : 1 þ e410Rel

ð3Þ

For the construction of the short form of a twodimensional scale assessing supervisor support, we added the correlation between the two latent constructs as a criterion. Preliminary analysis revealed the error adjusted correlation between these two subscales to be .889 in a CFA including all items of the original version of the scale. To ensure that the constructed short version of this scale matches the interrelations of the subscales found in the long version as closely as possible, the pheromone contribution of the correlation between the two latent variables was defined as 2

uCorr ¼ e40ðCorr0:889Þ :

ð4Þ

In the case of the proactive personality scale, the pheromone contributions shown in Equations 1 through 3 were combined to uProact ¼

0:5ðuRMSEA þ uCFI Þ þ uRel 2

ð5Þ

to ensure a function that is limited to the range between 0 and 1 and considers reliability and model fit in an equally important manner. Because we added the criterion of the correlation between the two latent variables for supervisor support the pheromone function was defined as uSupS ¼

0:5ðuRMSEA þ uCFI Þ þ uRel þ uCorr 3

ð6Þ

Ó 2015 Hogrefe Publishing

Corrected proof – not final


16 15 14 13 12 11 10 9 8 7 6 5 4

1.22 (.95) 1.36 .92*** (.89) 1.28 .93*** .77*** (.90) 1.29 .94*** .79*** .84*** (.85) 0.83 .45*** .43*** .41*** .43*** (.84) 0.88 .14* .05 .21*** .11 .11 (.96) 0.93 .14* .06 .22*** .11 .13* .98*** (.95) 0.90 .12 .05 .19** .09 .08 .93*** .84*** (.89) 0.97 .49*** .46*** .46*** .44*** .92*** .08 .09 .05 (.79) 0.95 .11 .04 .18** .08 .11 .96*** .94*** .91*** .07 (.89) 1.07 .10 .02 .17** .08 .12 .91*** .94*** .77*** .07 .94*** (.88) 0.96 .09 .04 .16** .06 .10 .88*** .79*** .93*** .06 .92*** .74*** (.77) 0.88 .51*** .49*** .45*** .46*** .91*** .08 .09 .04 .91*** .06 .07 .05 (.81) 0.87 .13* .04 .21** .10 .09 .94*** .90*** .93*** .07 .91*** .83*** .86*** .05 (.85) 0.98 .14* .06 .22*** .11 .12* .89*** .91*** .75*** .10 .84*** .84*** .71*** .08 .91*** (.79) 0.95 .09 .02 .15* .08 .04 .82*** .71*** .93*** .02 .80*** .65*** .85*** .01 .90*** .63*** (.78) Notes. *p < .05. **p < .01. ***p < .001.

Means, standard deviations, zero-order correlations, and reliabilities for the original, the ACO, and the manual short scales used in sample 1 are shown in Table 1. Reliabilities of all reduced scales decreased only moderately compared to the parent forms and Cronbach’s a were a .77 for all overall scales and all subscales and hence satisfactory. Following the aforementioned item selection procedures, we received a set of five items for the ACO proactive personality scale (referred to as Proact-ACO) and a six-item ACO supervisor support scale (SupS-ACO) including the subscales career promotion (CP-ACO) and feedback and goal setting (FGS-ACO) with three items each. These short scales were, in a first step, employed

5

2.78 3.04 2.54 2.77 4.89 2.89 2.78 3.07 4.77 2.98 2.73 3.22 5.01 2.99 2.70 3.29

Study 1

External 1. Innovative work behavior criterion 2. Idea generation 3. Idea promotion 4. Idea realization Original 5. Proactive personality scales 6. Supervisor support 7. Career promotion 8. Feedback & goal setting ACO short 9. Proact-ACO scales 10. SupS-ACO 11. CP-ACO 12. FGS-ACO Manual 13. Proact-man short 14. SupS-man scales 15. CP-man 16. FGS-man

Results

3

After completing the item selection process which was run in R version 2.15.2 and Mplus 5.1 (Muthén & Muthén, 1998–2007), we used linear regression analyses to quantify the predictive values of the parent and the short scales for the innovative work behavior.

2

Statistical Analysis

1

In order to evaluate the benefits of ACO, we developed short forms of the proactive personality scale and the supervisor support scale employing classical item selection approaches. We followed recommended selection criteria and deleted items with correlation r .80 indicating redundancy, factor loadings .40 and high modification indices indicating high cross-loadings. Further, items were selected so that the correlation between the two shortened latent subscales of supervisor support was similar to the one in the parent form. Though suggestions for more elegant selection procedures have been made (e.g., Stanton et al., 2002), we chose this approach to allow for a comparison to the strategies as they are often seen in applications (Kruyen et al., 2013).

SD

Manual Item Selection

M

to accommodate the equal importance of model fit, reliability, and the latent correlation. All equations were adapted from those reported by Leite et al. (2008). However, we used thresholds of > 0.95 for CFI and < 0.05 for RMSEA as suggested by Hu and Bentler (1999). Furthermore, all equations represent logistic functions with the highest discrimination around the inflection points. Hence, highly reliable items or those contributing to a good model fit (high CFI value and low RMSEA value) result in higher values in the pheromone function and are rewarded by higher pheromone levels.

Table 1. Means, standard deviations, and zero-order correlations of original and reduced overall scales and subscales (Cronbach’s alphas on diagonal; N = 279) in sample 1

A. B. Janssen et al.: Scale Short Form Development by ACO

European Journal of Psychological Assessment 2015

6


Table 2. Model fit statistics of ACO and manual short forms for the proactive personality scale and the supervisor support scale in samples 1 and 2 Sample 1 v

2

Proactive personality scale ACO short form 5.34 Manual short form 6.24 Supervisor support scale ACO short form 7.88 Manual short form 18.06

Sample 2 2

df

p

CFI

TLI

RMSEA

SRMR

v

df

p

CFI

TLI

RMSEA

SRMR

5 5

.38 .28

1.00 0.99

0.99 0.99

.02 .03

.02 .02

5.56 7.99

5 5

.35 .16

0.99 0.98

0.99 0.97

.03 .06

.03 .03

8 8

.45 .02

1.00 0.98

1.00 0.97

.00 .07

.02 .03

6.08 15.84

8 8

.64 .04

1.00 0.98

1.00 0.97

.00 .08

.02 .03

on sample 1. CFA testing a one-factor model for the ACO proactive personality scale revealed a very good model fit with v2(df ) = 5.34(5), p = .38, CFI = 0.99, TLI = 0.99, RMSEA = 0.02, SRMR = 0.02. Similarly, for the ACO supervisor support scale, a model testing for two factors also fitted the data of the first sample perfectly (v2(df ) = 7.88(8), p = .45, CFI = 1.00, TLI = 1.00, RMSEA = 0.00, SRMR = 0.02). All model fit indices are reported in Table 2. In addition to good reliability and model fit, we required the ACO-shortened subscales of supervisor support to correlate similarly high as the subscales in the parent form. Whereas the correlation in the original form was r = .84, the two subscales in the reduced form were correlated at r = .74. It needs to be considered that the correlation value of r = 0.889 that was included in the algorithm based on the relationship between the latent supervisor support factors. The latent correlation in the short form was r = 0.883. Thus, in the latent model, the deviation was very small. Furthermore, the correlation was integrated in the ACO algorithm with a weight of one-third. We hence had to expect a deviation from the primary correlation. Same CFA analyses were conducted for the manually reduced scales. For the proactive personality scale (referred to as Proact-man), we tested a one-factor model which showed a very good model fit (v2(df ) = 6.24(5), p = .28, CFI = 0.99, TLI = 0.99, RMSEA = 0.03, SRMR = 0.02). We further received a six-item scale for supervisor support scale (SupS-man) with three items for each subscale (CP-man and FGS-man). The model fit for this short form was only acceptable with v2(df ) = 18.06(8), p = .02, CFI = 0.98, TLI = 0.97, RMSEA = 0.07, SRMR = 0.03. The correlation between the two subscales was r = 0.63 for the manifest subscales and r = 0.81 for the latent factors. In order to test for the criterion validity of the ACO and the manually reduced scales, we used both the parent and the short forms to predict innovative work behavior and its subscales. As shown in Table 1, proactive personality was positively related to all innovative work behavior dimensions in both the original and the short version. Supporting this finding, the 95% confidence intervals of the unstandardized beta coefficients of the parent form covered all respective beta coefficients derived from the Proact-ACO and the Proact-man (see Figure 1). Even though the supervisor support parent and ACO and manual European Journal of Psychological Assessment 2015

short forms produced deviant p-values for the prediction of innovative work behavior (see Table 3), we can still confirm the SupS-ACO and SupS-man to be as valid as the original scale. This is due to the fact that all unstandardized regression coefficients of both the SupS-ACO and the SupS-man lay within the confidence intervals of the respective unstandardized beta coefficients of the original scale. Thus, the ACO reduced supervisor support scale predicts innovative work behavior and its subscales in a similar fashion to the original scale. Finally, the original proactive personality scale correlated with r = 0.92 to the Proact-ACO scale and with r = 0.91 to the Proact-man scale. Similarly high correlations were found between the overall supervisor support scale and its short versions (ACO: r = 0.96; manual: r = 0.94). The supervisor support subscales and their reduced forms were associated with r = 0.94 for CP-ACO and r = 0.91 for CP-man, and r = 0.93 for both FGSACO and FGS-man (see Table 1).

Study 2 In the second study, we used a separate sample to test the factor structure and criterion validity of the ACO and manual short forms for proactive personality and supervisor support. Means, standard deviations, intercorrelations, and reliabilities of sample 2 employed reduced scales and innovative work behavior scale are shown in Table 4. Results attest an acceptable to good reliability of the ACO short scales with Cronbach’s a ranging between .75 and .89. The classically developed short forms included a supervisor support subscale with a poor reliability of a = .70. To test the factor structure of the Proact-ACO, Proact-man, SupS-ACO, and SupS-man, we used CFA with a one-factor model for proactive personality and a two-factor model for supervisor support. The overall model fit indices are shown in Table 2 and indicate a very good model fit for the ACO proactive personality scale with v2(df ) = 5.56(5), p = .35, CFI = 0.99, TLI = 0.99, RMSEA = 0.03, and SRMR = 0.03. Compared to the ACO short form, the manually reduced proactive personality scale (Proactman) revealed a slightly weaker, however still good model fit (v2(df ) = 7.99(5), p = .16, CFI = 0.98, TLI = 0.97, RMSEA = 0.06, SRMR = 0.03). The SupS-ACO Ó 2015 Hogrefe Publishing


7

Figure 1. Confidence intervals of unstandardized regression coefficients for regressing innovative work behavior on proactive personality, supervisor support, career promotion, and feedback and goal setting. Comparison of original scales with ACO and manual short scales in sample 1 and 2.

two-factor model fitted the data perfectly with v2(df ) = 6.08(8), p = .64, CFI = 1.00, TLI = 1.00, RMSEA = 0.00, and SRMR = 0.02. Hence, the suggested factor structure was fully supported. The correlation between CP-ACO and FGS-ACO was r = 0.78. Finally, the model fit of the manually shortened supervisor support scale was barely acceptable (v2(df ) = 15.84(8), p = .04, CFI = 0.98, TLI = 0.97, RMSEA = 0.08, and SRMR = 0.03). The supervisor support subscales CP-man and FGS-man correlated at r = 0.76. In sample 2, we also calculated the regression of innovative work behavior on proactive personality and supervisor support. All regression coefficients are shown in Table 3. Starting with the SupSACO, all unstandardized regression coefficients were found to be covered by the confidence intervals of the original form’s beta coefficients as displayed in Figure 1. We can hence register that the external validity of the SupS-ACO could also be supported on a new sample. Results indicating a loss of validity were found for the SupS-man when predicting overall innovative work behavior and its facet Ó 2015 Hogrefe Publishing

idea realization. Here, two regression weights of the subscale feedback and goal setting were not included in the 95% confidence intervals of the corresponding original form’s coefficients. Thus, the classically reduced feedback and goal setting subscale revealed both poor reliability and validity. For proactive personality, both the ACO and the manual short form showed highly significant positive associations with innovative work behavior and its three subdomains. However, whereas the beta coefficient confidence intervals of the parent form covered the unstandardized regression coefficient of the ACO and manual short form for overall innovative work behavior, idea generation, and idea promotion, we found deviations for idea realization as shown in Figure 1. Irrespective of deviating predictive values for the different short forms, we can state that proactive personality was confirmed to be a strong predictor of both overall innovative work behavior and its three subdomains. The original scale and all short scales of proactive personality showed a strong positive relation to the proactive behavior outcome European Journal of Psychological Assessment 2015

Corrected proof – not final 8

Sample 1 (N = 279) Original scales External criterion Innovative work behavior

Idea generation

Idea promotion

Idea realization

Sample 2 (N = 155)

ACO short scales

Manual short scales

ACO short scales

Manual short scales

Predictor

B

CI

SE

T

B

SE

T

B

SE

T

B

SE

T

B

SE

T

Proactive personality Supervisor support Career promotion Feedback & goal setting Proactive personality Supervisor support Career promotion Feedback & goal setting Proactive personality Supervisor support Career promotion Feedback & goal setting Proactive personality Supervisor support Career promotion Feedback & goal setting

.66 .19 .18 .16 .69 .08 .08 .07 .63 .31 .30 .27 .66 .16 .15 .13

0.51–0.81 0.02–0.35 0.03–0.34 0.00–0.31 0.52–0.86 0.10–0.26 0.09–0.25 0.11–0.25 0.46–0.79 0.14–0.48 0.14–0.46 0.10–0.43 0.50–0.83 0.02–0.33 0.01–0.32 0.04–0.29

.08 .08 .08 .08 .09 .09 .09 .09 .08 .09 .08 .08 .08 .09 .08 .09

8.58*** 2.27* 2.32* 1.93 7.92*** 0.90 0.93 0.77 7.52*** 3.64*** 3.67*** 3.19** 8.03*** 1.78 1.86 1.46

.62 .13 .11 .12 .64 .04 .03 .05 .61 .24 .21 .21 .59 .10 .09 .08

.07 .08 .07 .08 .07 .09 .08 .08 .07 .08 .07 .08 .07 .08 .07 .08

9.44*** 1.68 1.66 1.52 8.73*** 0.49 0.36 0.62 8.69*** 2.96** 2.93** 2.62** 8.33*** 1.22 1.28 1.05

.70 .18 .18 .11 .76 .07 .09 .02 .66 .30 .29 .21 .68 .15 .15 .10

.07 .08 .07 .08 .08 .09 .08 .09 .08 .09 .08 .08 .08 .09 .08 .08

9.88*** 2.13* 2.39* 1.47 9.52*** 0.73 1.07 0.26 8.54*** 3.51** 3.78*** 2.57* 8.80*** 1.72 1.85 1.25

.51 .22 .20 .19 .60 .14 .12 .12 .50 .31 .28 .27 .41 .21 .20 .17

.09 .09 .08 .09 .09 .10 .10 .10 .09 .10 .09 .09 .10 .10 .09 .10

5.87*** 2.41* 2.37* 2.13* 6.42*** 1.34 1.30 1.18 5.47*** 3.23** 3.15** 2.90** 4.15*** 2.06* 2.10* 1.76

.54 .30 .23 .32 .65 .22 .17 .24 .52 .37 .31 .36 .46 .30 .20 .34

.09 .09 .09 .09 .10 .10 .10 .10 .10 .10 .09 .09 .11 .10 .10 .09

5.91*** 3.29** 2.68** 3.70*** 6.51*** 2.13* 1.73 2.49* 5.29*** 3.92*** 3.50** 4.00*** 4.32*** 2.97** 2.13* 3.63***

Notes. *p < .05. **p < .01. ***p < .001.



Table 3. Relationship between innovative work behavior and proactive personality and supervisor support. Comparison of original and short scales for proactive personality and supervisor support derived from ACO and classical item selection procedures


Corrected proof – not final (.70) (.83) .76***

Discussion

(.75) .05 .88*** .81*** .84*** (.85) .78*** .08 .88*** .89*** .75*** (.89) .95*** .94*** .07 .93*** .90*** .84*** (.77) .04 .04 .03 .93*** .04 .08 .00 (.82) .32*** .16* .17* .14 .33*** .23** .17* .28*** (.88) .76*** .40*** .25** .25** .23** .39*** .30*** .27** .31*** (.84) .77*** .78*** .46*** .11 .10 .10 .46*** .17* .14 .20* Notes. *p < .05. **p < .01. ***p < .001.

(.93) .92*** .91*** .92*** .43*** .19* .19* .17* .43*** .26** .21* .29*** 1.06 1.17 1.11 1.16 0.89 0.92 1.00 0.95 0.83 0.91 0.97 0.96 2.50 2.73 2.27 2.51 4.78 2.55 2.38 2.73 4.97 2.62 2.33 2.92 1. Innovative work behavior 2. Idea generation 3. Idea promotion 4. Idea realization 5. Proact-ACO 6. SupS-ACO 7. CP-ACO 8. FGS-ACO 9. Proact-man 10. SupS-man 11. CP-man 12. FGS-man


9

(t-values between 4.15 and 9.88). Supervisor support and its subscales consistently predicted employees’ idea promotion, whereas idea generation and idea realization seemed rather unaffected by perceived supervisor support.

(.77) .07 .11 .03

(.87) .94*** .93***

12 11 10 9 8 7 6 5 4 3 2 1 SD M

Table 4. Means, standard deviations, and zero-order correlations of ACO-shortened overall scales and subscales (Cronbach’s alphas on diagonal; N = 155) in validation sample 2


The present study aimed to develop two reduced scales and to show some of the benefits of implementing ACO-based scale reduction for developing reliable and valid short forms of a proactive personality scale and a supervisor support scale. To achieve this, scales were shortened based on criteria concerning reliability and internal structure of the short forms and compared to variants obtained by manual scale shortening in both the initial sample and a second validation sample. Overall, the scales shortened via the ACO approach achieved acceptable reliabilities and maintained the factorial structure as well as the correlations between the factors of the original scales. Additionally the short forms were able to reproduce the predictive validity of the original forms to a large extent. The ACO-shortened scales outperformed manually shortened scales with regard to reproducing the factorial structure of the supervisor support scale. According to Smith, McCarthy, and Anderson (2000), developers of short forms should avoid nine frequent mistakes or sins to ensure the usefulness of the shortened scale. The ACO context chosen here has the inherent aim of avoiding the shortcomings of short forms that Smith and colleagues (2000) classify as sin 3 (p. 104) – dramatic reductions in the reliability of the overall scales, as well as its subscales, and sin 5 (p. 106) – a failure to maintain the original factor structure. Because the original factor structure is provided as the measurement model in the CFA and items are selected to maximize the model fit of this factor structure (with its factor interrelations) and the reliability of the chosen scales (Equations 5 and 6) both of these problems are directly minimized. This resulted in short scales with acceptable reliabilities in both samples, with Cronbach’s a ranging from .75 to .89 and a very close approximation of the relationship between the subscales of the supervisor support. While acceptable reliabilities were also found in the manually shortened scales, this approach was not as successful in reproducing the internal structure of the supervisor support scale. Validation of the shortened scales in a second sample (sin 7; Smith et al., 2000, p. 107) then revealed that the factor structure, reliability, and factor correlations are reproducible and not results of specific characteristics of the initial sample. To determine the criterion validity of the shortened versions, innovative work behavior was regressed on both proactive personality and supervisor support. For sample 1 these analyses showed that both ACO and manual short forms were able to reproduce the relationship between the two predictors, namely proactive personality and supervisor support and the criterion innovative work behavior. All unstandardized regression coefficients of the short scales were covered by the confidence European Journal of Psychological Assessment 2015

10


intervals of the original scales’ regression weights what confirms high criterion validity of these scales. Such coverage of beta weights by confidence intervals was also found for the ACO-based short form of supervisor support in sample 2. Deviations from the confidence intervals occurred for the other three short scales in sample 2. The Proact-ACO and Proact-man did not validly predict idea realization. Here, the predictive value of proactive personality was underestimated compared to the parent form. This indicates a loss of validity. However, there are two reasons to assume that such loss of validity does not necessarily have negative practical implications. First, both the original scale and the short scales predicted idea realization in a highly significant way. Second, the underestimation of the relationship between proactive personality and idea realization implies that Proact-ACO provides a rather conservative estimation. Practitioners hence avoid the risk of focusing solely on proactive personality in order to explain individual differences in innovative behavior which is, as we know, an outcome of a host of antecedents including individual and contextual factors (e.g., Axtell et al., 2000; Crant, 2000; Krause, 2004). Thus, the present lack of criterion validity of the Proact-ACO and the Proact-man is unlikely to have practically relevant downsides. Things are different regarding SupS-man. The subscale FGS-man did not validly predict overall innovative work behavior and its subdomain idea realization. In contrast to the reduced proactive personality scales, the FGS-man produced significant relationships to these two particular outcomes, whereas the regression coefficients were not significant in the original scales. Hence, the prediction of our criterion by SupS-man is biased. We can conclude that Proact-ACO does not show significant advantages compared to Proact-man in terms of model fit or the prediction of the external criterion. However, SupS-ACO seems to outperform SupS-man. The ACO short form for supervisor support revealed both a better model fit and higher criterion validity than the manual short scale.

Benefits and Shortcomings of ACO In contrast to most CTT-based approaches, the ACO approach allows for simultaneous evaluation of multiple statistical criteria concerning the short-form as a whole, and not only those of the items themselves. This is achieved by predefining a measurement model within the CFA context and testing the appropriateness of this model – a step often not undertaken in traditional manual short form development approaches (e.g., Carr, Moss, & Harris, 2005; Kruyen et al., 2013). Because these models are based in CTT they can be made to be much less restrictive than those most commonly found in IRT approaches. In contrast to IRT approaches, it is not necessary to have an item set that consists of theoretically interchangeable items in item selection via the ACO approach. This is possible because the evaluation of the short-form is based on CFA models


and their comparisons, which does not imply assumptions about the items not included in the model. This comes at the expense of necessitating a cross-validation, because the models and their comparisons are sample-dependent. Additionally, the ACO approach allows for including external criteria (as shown by Leite et al., 2008) and, as in the present study, for maintaining the dimensional structure of the long scale. Finally, though this was not done in this study, ACO allows for including theoretically important anchor items that need to be included in the short form of a scale. This latitude in defining the parameters of item selection comes at the price of lacking guidelines for selection criteria. Not only are there no proven guidelines as to which criteria to specify concerning specific components (e.g., model fit, reliability, and latent correlations) and to what degree to increase or reduce pheromone levels of a certain set of items but there is also no information concerning the interaction and combination of these components to one specific criterion. The present results support Leite et al.’s (2008) suggestion that the ACO approach can be gainfully used especially for reducing multidimensional scales with many items; in that case manual item selection would imply comparing myriads of models with different sets of items. We can hence assume to have found the best solution for a set of items meeting the previously defined item selection criteria. After Leite et al.’s (2008) simulation studies, the present study is – to our knowledge – the first confirming this advantage of ACO on real life data. Even though the present ACO results for the supervisor support scale confirmed our assumption that this approach has major benefits compared to classical scale reduction, we encourage researchers and practitioners to further evaluate the ACO item selection approach by applying this technique on scales with a larger number of items and a more complex factor structure. According to Leite and colleagues (2008), ACO-based item selection is less recommended for unidimensional scales with few items due to the manageable number of model comparisons that need to be conducted to find the best solution. We applied ACO to reduce the original proactive personality scale by one half and received a set of five items with an acceptable reliability for the short form. However, analyses indicated a slight loss of criterion validity. Thus, regarding idea realization, items of the ProactACO assessed slightly different information on proactive personality than the original scale. Still, both the parent and the Proact-ACO predicted all dimensions of innovative work behavior positively and highly significantly which allows the conclusion that the Proact-ACO also works as a good measure to predict innovative work behavior and that the loss of criterion validity can be neglected. Manual item selection for the proactive personality scale generated a similar set of items with a very good model fit as well. The Proact-man revealed the same small validity problems. Hence, we did not identify any significant benefits of ACO when shortening a unidimensional scale with a relative small number of items. These empirical findings support Leite et al.’s (2008) suggestions.



Conclusion To sum up, the present study provides a reliable and valid short version of a two-dimensional supervisor support scale assessing supervisors’ career promotion and feedback and goal setting behavior with a time saving of approximately two-thirds. Especially in IO psychology, the support received from the supervisor is of major interest for numerous practical and research questions. Hence, this great time saving while maintaining scale reliability and validity will be beneficial for any investigator collecting data in organizational settings. Even though applying the Proact-ACO and Proact-man would halve assessment time, this short version cannot fully replace the 10-item parent form of the proactive personality scale because of its small lack of criterion validity. However, owing to the high correlation between the parent and the short form and the replicated direction and intensity of the relation to the external criterion, the short forms of proactive personality can still be recommended as a reliable instrument when only the proactive personality construct is of interest or for short screenings that may only include very few items. As intended in the study, we tested the benefits of the ACO approach for different lengths and complexities of original scales compared to classical item selection. According to the present results and the great range of selection possibilities, we recommend ACO for item selection of multidimensional scales with a large number of items, whereas ACO does not carry significant benefits compared to classical approaches when reducing unidimensional and rather short original forms.

References Axtell, C. M., Holman, D. J., Unsworth, K. L., Wall, T. D., Waterson, P. E., & Harrington, E. (2000). Shopfloor innovation: Facilitating the suggestion and implementation of ideas. Journal of Occupational and Organizational Psychology, 73, 265–285. doi: 10.1348/096317900167029/abstract Bateman, T. S., & Crant, J. M. (1993). The proactive component of organizational behavior: A measure and correlates. Journal of Organizational Behavior, 14, 103–118. doi: 10.1002/job.4030140202/abstract Bindl, U. K., & Parker, S. K. (2010). Proactive work behavior: Forward-thinking and change-oriented action in organizations. In S. Zedeck (Ed.), APA handbook of industrial and organizational psychology: Vol. 2. Selecting and developing members for the organization (pp. 567–598). Washington, DC: American Psychological Association. Bradburn, N. M. (1978). Respondent Burden. Proceedings of the Survey Research Methods, DHEW Publication No. (PHS) 7. Alexandria, VA: American Statistical Association, pp. 35–40. Carr, T., Moss, T., & Harris, D. (2005). The DAS24: A short form of the Derriford Appearance Scale DAS59 to measure individual responses to living with problems of appearance. British Journal of Health Psychology, 10, 285–298. doi: 10.1348/135910705X27613 Choi, J. N., & Chang, J. Y. (2009). Innovation implementation in the public sector: An integration of institutional and collective dynamics. Journal of Applied Psychology, 94, 245–253. doi: 10.1037/a0012994 Ó 2015 Hogrefe Publishing

11

Colorni, A., Dorigo, M., & Maniezzo, V. (1991). Distributed optimization by ant colonies. In F. Varela & P. Bourgine (Eds.), Proceedings of ECAL91–European Conference on Artificial Life (pp. 134–142). Paris, France: Elsevier Publishing. Crant, J. M. (1995). The Proactive Personality Scale and objective job performance among real estate agents. Journal of Applied Psychology, 80, 532–537. doi: 10.1037/00219010.80.4.532 Crant, J. M. (2000). Proactive behavior in organizations. Journal of Management, 26, 435–462. doi: 10.1177/ 014920630002600304 Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001). Web surveys: Perceptions of burden. Social Science Computer Review, 19, 146–162. doi: 10.1177/089443930101900202 Deneubourg, J. L., & Goss, S. (1989). Collective patterns and decision-making. Ethology Ecology & Evolution, 1, 295–311. doi: 10.1080/08927014.1989.9525500 Deneubourg, J. L., Pasteels, J. M., & Verhaeghe, J. C. (1983). Probabilistic behaviour in ants: A strategy of errors? Journal of Theoretical Biology, 105, 259–271. doi: 10.1016/S00225193(83)80007-1 Eisenberger, R., Huntington, R., Hutchison, S., & Sowa, D. (1986). Perceived organizational support. Journal of Applied Psychology, 71, 500–507. doi: 10.1037/0021-9010. 71.3.500 Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 6, 56–83. doi: 10.1080/10705519909540119 Galesic, M. (2006). Dropouts on the Web: Effects of interest and burden experienced during an online survey. Journal of Official Statistics, 22, 313–328. Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73, 349–360. doi: 10.1093/poq/nfp031 Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. doi: 10.1080/10705519909540118 Janssen, O. (2000). Job demands, perceptions of effort-reward fairness and innovative work behaviour. Journal of Occupational and Organizational Psychology, 73, 287–302. doi: 10.1348/096317900167038 Janssen, O. (2001). Fairness perceptions as a moderator in the curvilinear relationships between job demands, and job performance and job satisfaction. The Academy of Management Journal, 44, 1039–1050. doi: 10.2307/ 3069447 Kidd, J. M., & Smewing, C. (2001). The role of the supervisor in career and organizational commitment. European Journal of Work and Organizational Psychology, 10, 25–40. doi: 10.1080/13594320042000016 Krause, D. E. (2004). Influence-based leadership as a determinant of the inclination to innovate and of innovation-related behaviors: An empirical investigation. The Leadership Quarterly, 15, 79–102. doi: 10.1016/j.leaqua. 2003.12.006 Kruyen, P. M., Emons, W. H. M., & Sijtsam, K. (2013). On the shortcomings of shortened tests: A literature review. International Journal of Testing, 13, 223–248. doi: 10.1080/ 15305058.2012.703734 Leite, W. L., Huang, I.-C., & Marcoulides, G. A. (2008). Item selection for the development of short forms of scales using an Ant Colony Optimization Algorithm. Multivariate Behavioral Research, 43, 411–431. doi: 10.1080/ 00273170802285743 European Journal of Psychological Assessment 2015

12


MacDonald, P., & Paunonen, S. V. (2002). A Monte Carlo comparison of item and person statistics based on item response theory versus classical test theory. Educational and Psychological Measurement, 62, 921–943. doi: 10.1177/ 0013164402238082 Maloney, P., Grawitch, M. J., & Barber, L. K. (2011). Strategic item selection to reduce survey length: Reduction in validity? Consulting Psychology Journal: Practice and Research, 63, 162–175. doi: 10.1037/a0025604 Muthén, L. K., & Muthén, B. O. (1998–2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Muthén & Muthén. Ng, T. W., Feldman, D. C., & Lam, S. S. (2010). Psychological contract breaches, organizational commitment, and innovation-related behaviors: A latent growth modeling approach. Journal of Applied Psychology, 95, 744–751. doi: 10.1037/ a0018804 Ohly, S., Sonnentag, S., & Pluntke, F. (2006). Routinization, work characteristics and their relationships with creative and proactive behaviors. Journal of Organizational Behavior, 27, 257–279. doi: 10.1002/job.376 Parker, S. K., & Collins, C. G. (2010). Taking stock: Integrating and differentiating multiple proactive behaviors. Journal of Management, 36, 633–662. doi: 10.1177/0149206308321554 Parker, S. K., Williams, H. M., & Turner, N. (2006). Modeling the antecedents of proactive behavior at work. Journal of Applied Psychology, 91, 636–652. doi: 10.1037/0021-9010. 91.3.636 Rammstedt, B., & Beierlein, C. (2014). Can’t we make it any shorter? The limits of personality assessment and ways to overcome them. Journal of Individual Differences, 35, 212–220. doi: 10.1027/1614-0001/a000141 Rogers, M. E., Creed, P. A., Searle, J., & Hartung, P. J. (2010). The physician values in practice scale-short form: Development and initial validation. Journal of Career Development, 38, 111–127. doi: 10.1177/0894845310363593 Rolstad, S., Adler, J., & Rydén, A. (2011). Response burden and questionnaire length: Is shorter better? A review and metaanalysis. Value in Health, 14, 1101–1108. doi: 10.1016/ j.jval.2011.06.003 Sampson, R. C. (2007). R&D alliances and firm performance: The impact of technological diversity and alliance organization on innovation. Academy of Management Journal, 50, 364–386. doi: 10.5465/AMJ.2007.24634443 Seibert, S. E., Crant, J. M., & Kraimer, M. L. (1999). Proactive personality and career success. Journal of Applied Psychology, 84, 416–427. doi: 10.1037/0021-9010.84.3.416 Seibert, S. E., Kraimer, M. L., & Crant, J. M. (2001). What do proactive people do? A longitudinal model linking proactive personality and career success. Personnel Psychology, 54, 845–874. doi: 10.1111/j.1744-6570.2001.tb00234.x Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological Assessment, 12, 102–111. doi: 10.1037/1040-3590.12.1.102


Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55, 167–194. doi: 10.1111/ j.1744-6570.2002.tb00108.x Stocking, M., & Swanson, L. (1993). A method for severely constrained item selection in adaptive testing. Applied Psychological Measurement, 17, 277–292. doi: 10.1177/ 014662169301700308 Watkins, J. (1964). Laboratory experiments on the trail following of army ants of the genus Neivamyrmex (Formicidae: Dorylinae). Journal of the Kansas Entomological Society, 37, 22–28. doi: 25083355 Welbourne, T. M., Johnson, D. E., & Erez, A. (1998). The rolebased performance scale: Validity analysis of a theory-based measure. Academy of Management Journal, 41, 540–555. doi: 10.2307/256941 Wester, S. R., Vogel, D. L., O’Neil, J. M., & Danforth, L. (2012). Development and evaluation of the Gender Role Conflict Scale Short Form (GRCS-SF). Psychology of Men & Masculinity, 13, 199–210. doi: 10.1037/a0025550 Yuan, F., & Woodman, R. W. (2010). Innovative behavior in the workplace: The role of performance and image outcome expectations. Academy of Management Journal, 53, 323–342. doi: 10.5465/AMJ.2010.49388995 Ziegler, M. (2014). Comments on Item Selection Procedures. European Journal of Psychological Assessment, 30, 1–2. doi: 10.1027/1015-5759/a000196 Ziegler, M., Kemper, C. J., & Kruyen, P. (2014). Short scales – Five misunderstandings and ways to overcome them. Journal of Individual Differences, 35, 185–189. doi: 10.1027/16140001/a000148

Date of acceptance: June 15, 2015 Published online: October X, 2015

Anne B. Janssen Jacobs University Bremen Psychology & Methods Campus Ring 1 28759 Bremen Germany Tel. +49 421 200-4751 E-mail [email protected]



13

Appendix Table A1. Items and CFA factor loadings of ACO and manual short forms for the proactive personality and the supervisor support scale. Scale Proact-ACO

Proact-man

SupS-ACO CP-ACO

FGS-ACO

SupS-man CP-man

FGS-man

Items Wherever I have been, I have been a powerful force for constructive change. If I see something I don’t like, I fix it. I love being a champion for my ideas, even against others opposition. I excel at identifying opportunities. I can spot a good opportunity long before others can. Wherever I have been, I have been a powerful force for constructive change. If I see something I don’t like, I fix it. I excel at identifying opportunities. I am always looking for better ways to do things. I can spot a good opportunity long before others can. My supervisor . . . Suggests strategies to advance my career. Helps me identify important skills, interests, and values regarding my career. Is realistic in discussing my career progression.

Sample 1 (N = 279)

Sample 2 (N = 155)

Factor 1

Factor 1

0.77

0.66

0.53 0.54

0.53 0.67

0.78 0.74

0.68 0.68

0.74

0.62

0.55 0.79 0.57 0.74

0.57 0.65 0.65 0.71

0.84 0.88

0.83 0.90

0.80

0.70

Makes clear what the goals and objectives of the organization are. Gives specific guidance as to how I can improve. Keeps me informed of how well I am doing. My supervisor . . . Gives me tasks requiring people who can influence my career. Helps me identify important skills, interests, and values regarding my career. Helps me participate in high visibility activity either inside or outside the organization. Agrees goals and objectives to measure my current performance. Makes clear what the goals and objectives of the organization are. Identifies critical job elements.


Factor 2

0.63 0.83 0.71

0.52 0.81 0.78

0.72 0.85

0.87 0.88

0.66

0.76 0.74 0.71 0.74

Factor 2

0.73 0.54 0.73
