Validation of ERI Test and JCQ Questionnaire
Ansoleaga, Montaño and Vézina
Validation of two complementary instruments for measuring work stress in Chilean workers. Elisa Ansoleaga1*, Rosa Montaño2 3, Michel Vézina4 Department of Psychology, Universidad Diego Portales, Santiago, Chile. Department of Mathematics and Computer Science, Faculty of Science, Universidad de Santiago, Santiago, Chile. 3 Institute of Public Health, Universidad de Chile, Santiago, Chile. 4 Social and Preventive Medicine Department, Laval University, Quebec, Canada. 1
2
* Elisa Ansoleaga Department of Psychology, Universidad Diego Portales, Grajales 1898, Santiago Centro, Santiago, Chile – email:
[email protected]
Received: 08.02.2013 Accepted: 19.11.2013 Published: 08.12.2013
Abstract Instruments based on Karasek’s demand/control/support and Siegrist’s effort-reward imbalance models have been extensively used to evaluate exposure to psychosocial workplace risks linked to physical and mental health, but have not been validated in Chile. The study aimed to analyze the factorial structure and concurrent and criteria validity in two complementary scales to measure psychosocial risk in Chilean workplaces. This transversal study was conducted at the national level (Chile), with a random sample that included 3,010 workers (51% male and 49% female) in a household survey. The analysis included correlations, structural equations (SEM) and logistic regression. The results showed that the internal consistency of Karasek’s global scale was α=0.74, while in Siegrist’s global scale it was α=0.72. The models evidenced good structural adjustment (Karasek: RMSEA=0.051 and CFI=0.97; Siegrist: RMSEA=0.054 and CFI=0.98), and dose-response association between incremental exposure to the psychosocial dimensions of work and distress. In conclusion, instruments based on the Karasek and Siegrist models may be a useful tool for assessing psychosocial risk among Chilean workers. However, we suggest eliminating two social support items and adding items to the psychological demands scale in order to improve the consistency of Karasek’s instrument. Keywords: effort-reward imbalance; demand-control-support model; work stress; psychometric properties.
5
Scandinavian Journal of Organizational Psychology, December 2013
Volume 5, Issue 2
The analysis of the relationship between exposure to psychosocial risk at work (henceforth PSRW) and mental health has received increased attention (Bonde, 2008; Godin, Kittel, Coppieters and Siegrist, 2005; Stansfeld and Candy, 2006; Vézia, 2002). The evidence is based mainly on the analysis of two models for measuring PSRW: Siegrist’s (Siegrist, 1996a) effort-reward imbalance model (ERI) and Karasek and Theorell’s demand-control-support (D-C-S) model (Karasek and Theorell, 1990).
therefore can be applied to large population samples or to specific organizational contexts in Chile, the present work sought to analyze the factorial structure, concurrent validity, and criteria validity in two brief instruments that measure PSRW and to propose an instrument for measuring psychosocial risk in the Chilean working population.
Instruments based on the Karasek and Siegrist models have been translated and adapted into more than 10 languages and validated in different national settings with satisfactory results (Karasek et al., 1998; Siegrist 1996a). In a trans-cultural and multinational study of the Job Content Questionnaire (JCQ) that included over 16,000 cases, Karasek et al. (1998) proved that the psychometric properties of the instrument are quite stable. Moreover, rather than revealing differences among countries due to cultural aspects, what was observed were greater differences between occupations (Karasek et al., 1998). A similar observation was made in the psychometric analysis of the ERI test in comparisons between European countries (Siegrist et al., 2004) and in the validation of the brief scale in Spain (Fernandez-Lopez, Martin-Payo, Fernandez-Fidalgo and Rodel, 2006; Macías et al., 2003). These instruments have also been validated in samples of Scandinavian workers. Sweden, Denmark and Finland have validations for the ERI test and the JCQ (Leineweber et al., 2010; (Leineweber et al., 2010; Sanne, Torp, Mykletun and Dahl, 2005; Santavirta, 2003; Weyers, Peter, Boggild, Jeppesen and Siegrist, 2006)
Sample and Design
Material and Method Within the framework of the “Research, Policy and Practice with Regard to Work– related Mental Health Problems in Chile: A Gender Perspective” financed by the GHRI Initiative of the IDRC (Canada) and carried out in Chile by Centro de Estudios de la Mujer and University of Ottawa (20072012), we analyzed data collected by the Work Conditions and Health survey. The design was transversal and the survey was applied in households from 20 April to 6 July 2010. The proportion of response was 57%. The sample design was probabilistic and stratified in four levels: i) municipalities in Chilean regions; ii) city blocks in municipalities; iii) households in city blocks, and iv), people in households. The national and representative sample included 3,010 wage-earning male and female workers (20 to 65 years of age) belonging to all socioeconomic levels and residing in urban areas of Chile. The data were weighted according to the sampling frame and information from the Chilean National Institute of Statistics. Females were over-represented in the sample in order to obtain an equivalent ratio for both genders.
These validations have yielded satisfactory results that are consistent with those reported in the literature across countries (Karasek et al., 1998; Siegrist et al., 2004). Furthermore, the instruments have been used in the Scandinavian countries to study associations between PSRW, symptoms of depression and poor mental health (Kivimäki, Vahtera, Elovainio, Virtanen and Siegrist, 2007; Rugulies, Bültmann, Aust and Burr, 2006; Ylipaavalniemi et al., 2005).
Measures Two scales were used to evaluate exposure to PSRW: Karasek’s Job Content Questionnaire (Karasek et al., 1998) in its Canadian version (IRSST, INSPQ, and ISQ, 2010) and the brief version in Spanish of Siegrist’s ERI test (Siegrist, Wege, Puhlhofer and Wahrendorf, 2009). Also, we evaluated negative health effects using the Kessler scale (K6).
Given the absence of brief instruments that allow for rapid application and which 6
Validation of ERI Test and JCQ Questionnaire
Ansoleaga, Montaño and Vézina
Procedures and Pilot Application
known as social support. This dimension considers the perception of support received in relation to instrumental support, emotional support, and hostility shown by peers and superiors towards the worker. Additionally, this model considers two second-order factors: job strain originating from the combination of high psychological demands and low decisional latitude, and iso-strain, which considers the combination of work situations involving job strain combined with low social support.
The Canadian version (in French) of Karasek’s instrument was translated and counter-translated by qualified translators and then reviewed by professionals in both languages. The Spanish version of Siegrist’s instrument was used (Fernandez-Lopez, Martin-Payo, Fernandez-Fidalgo and Rodel, 2006). Prior to the application and training of survey-takers, a pilot application was conducted. A process of semantic adaptation was undertaken in relation to two of the Siegrist items (ERI 6: I am being subjected to–or expecting–a worsening of my work conditions was modified as follows: It worries me/distresses me that my work conditions may become worse; and ERI 7: My job is at risk was changed to My job stability is low) following the pilot. This work was approved by the Ethics Committee of the University of Ottawa, Canada, and by Universidad Diego Portales, Chile. All participants signed an informed consent form.
Effort-reward imbalance (ERI) test. The brief scale of the ERI (Siegrist et al., 2009; Fernandez-Lopez et al., 2006; Macías et al., 2003) is made up of 16 items, and considers five first-order factors and one second-order factor. The extrinsic effort (3 items) is a firstorder factor that encompasses the number of tasks, the pace at which they are performed and interruptions during the work. Rewards is the second first-order factor and is disaggregated into three components (firstorder): esteem (2 items), related to social rewards; security (2 items), referring to job stability and work conditions; and promotion (3 items) at work, referring to organizational reward (opportunities for promotion) and salary rewards. Finally, this model implies a second-order factor called efforts-reward imbalance (10 items), which results from the extrinsic effort and rewards dimensions.
Description of the Instruments Both instruments use a Likert measuring scale with four alternatives (1=Strongly Disagree to 4=Strongly Agree) in which the higher number represents a more marked presence of the attribute/dimension. Karasek’s Job Content Questionnaire. The Canadian version of the JCQ (18 items) was used (IRSST et al., 2010; Karasek et al., 1998). The Karasek model considers three first-order latent factors and two secondorder factors, as follows: a) psychological demand: refers to requirements in term of speed, amount of work, the time given to perform it, receiving contradictory demands, and highly demanding mental work (5 items); b) decisional latitude: considers two components, which are control (3 items), referring to the possibility of deciding how to do the work, the worker’s influence on how things happen at work, and monotony at work, and skills (2 items) referring to the possibility of learning new things at work and the level of qualifications/skills required for the work: and c) social support: social support by peers (4 items) and social support by superiors (4 items), both of which contribute to the third first-order factor
Negative health effects. Negative health effects were evaluated using Kessler’s K6, an instrument for screening of nonspecific psychological distress for monthly prevalence (Kessler et al., 2002). This scale has been developed and tested in developed countries and is used in population surveys in the United States, Australia and Canada (Kessler et al., 2002). The internal consistency of the scale in the sample of Chilean workers was α=0.88. Covariates: gender, age and socio-economic level (lowest vs. middle and highest).
Analysis of the Data Concurrent validity was tested by means of a polychoric correlation matrix used to analyze the item-dimension correlations for each of the scales. The internal consistency
7
Scandinavian Journal of Organizational Psychology, December 2013
of the scales was evaluated with Cronbach’s Alpha index. The factorial structures of the Karasek and Siegrist models were evaluated by structural equations models. The degree of adjustment of the models was analyzed with three competing models. The first model assumes the existence of a large factor which groups all the dimensions; the second models shows the presence of first-order factors and the third includes second-order factors. The degree of adjustment of the models and the relationships between them were evaluated by means of three indices. The first is the Comparative Fit Index (CFI) which measures incremental adjustment and tells us how much better the proposed model is compared to a null model, where values greater than 0.90 are considered satisfactory (Byrne, 2001). Similarly, we employed RMSEA (root mean square error of approximation), which evaluates the discrepancy in the covariance matrices between the predicted values and the observed values by the degrees of freedom. Finally, we analyzed the CAIC index which tells us about the parsimony of the model, based on the degree of adjustment, the number of estimated parameters and the size of the sample; smaller values indicate a more adequate specification of the model (Hu and Bentler, 1995). Additionally, criterion validity was evaluated by logistic regression analysis using the presence of high distress as the outcome and controlling for age, gender and socioeconomic level. The analyses were performed using the Stata 11.0 (Stata Corp., 2010) and Lisrel 8.8 (SSI, 2011) statistical packages.
Volume 5, Issue 2
moderate to strong for the demands factor (r=0.54 to 0.79), for decisional latitude (r=0.68 to 0.80) except for the item on monotonous work (r=-0.45), and for the social support factor (r=0.65 to 0.78) except for the item “my coworkers show interest or solidarity with me” (r=0.56). However, the two items related to the hostility component (my superior or my coworkers have a hostile or confrontational attitude towards me) in the social support scale correlated very weakly with the sub-scales of support by superiors (r=0.004) and by peers (r=0.054). Reliability of the scales. The alpha of Siegrist’s global scale was 0.72 and the alpha of Karasek’s global scale was 0.74. In two of the three scales of Karasek’s model there were adequate reliability values, but they were weak for the psychological demands scale (α = 0.59). Similarly, the decisional latitude scale showed an alpha of 0.62 and was higher in the sample of males than females (0.65 and 0.58 respectively). The social support scale showed an alpha of 0.81. In relation to Siegrist’s instrument, the efforts scale showed an alpha of 0.63 and the rewards scale had an alpha of 0.74. A scale is considered reasonably consistent if its alpha is between the values of 0.65 and 0.90, as values over 0.90 often indicate item redundancy (Streiner, 2003).
Factorial Structure of the Instruments The study analyzed the factorial structure of both instruments by means of three models and compared the degree of adjustment of the models with the data. The first model (model 0) considered that all the items correlate in one large factor and that errors are not correlated. As shown in Table 1, this model had a weak adjustment both for Karasek’s instrument and for Siegrist’s instrument. When we included the firstorder factors in the model (model 1), the adjustment improved substantially. Finally, in the third model (model 2) the first-order factors were disaggregated into all their components. In Karasek’s model the social support factor was disaggregated into its components of peers and superiors, and the decisional latitude factor into its control and skills components. For Siegrist’s model, the rewards factor was disaggregated into its esteem, security and promotion components.
Results The average age of the sample is 39 years (SD 11) with an equivalent ratio for both genders; 12% of the sample belongs to the low socio-economic level and 18% evidences high or very high psychological distress.
Concurrent Validity Correlations. The results of the polychoric correlation matrix analysis indicated that Siegrist’s model revealed marked itemdimension correlations for the efforts factor (r=0.77 to 0.85) and for the rewards factor (r=0.54 to 0.74). The Karasek model showed item-dimension correlations ranging from 8
Validation of ERI Test and JCQ Questionnaire
Ansoleaga, Montaño and Vézina
Table 1: Summary of indices of adequacy of adjustment of three competitive structural models for Karasek’s and Siegrist’s instruments Adjustment indices
Model 0
Model 1
Model 2
Demand-Control-Support Model RMSEA (90% IC)
0.079 (0.077-0.082)
0.069 (0.066-0.071)
0.051 (0.049-0.054)
CFI (Comparative fit index)
0.70
0.78
0.97
CAIC MODEL
3014.00
2349.82
1525.70
Effort-Rewards Imbalance Model RMSEA (90% IC)
0.11 (0.10-0.11)
0.070 (0.065-0.075)
0.054 (0.048-0.059)
CFI (Comparative fit index)
0.90
0.96
0.98
CAIC MODEL
1392.95
725.17
518.20
When comparing it with the other models, this third model showed good adjustment of the data with the structure proposed in both models and it complied with the criteria for a well-adjusted model according to Hu and Bentler (Hu and Bentler, 1995), according to which they should show a CFI>=0.95 and a RMSEA
