The Work Environment Impact Scale

447

Work 42 (2012) 447–457 DOI 10.3233/WOR-2012-1418 IOS Press

The Work Environment Impact Scale – Self-Rating (WEIS-SR) evaluated in primary health care in Sweden Birgitta A. Wästberga,∗, Lena Haglundb and Mona Eklunda a

b

Department of Health Sciences, Division of Occupational Therapy and Gerontology, Lund University, Sweden Department of Social and Welfare Studies, Faculty of Health Sciences, Linköping University, Linköping, Sweden

Received 13 November 2009 Accepted 24 November 2010

Abstract. Objective: To develop a self-report alternative to the Work Environment Impact Scale (WEIS). Participants: First the novel instrument was used and evaluated by ten occupational therapists and 45 clients in primary health care. Then the instrument was used by 26 clients who participated in a rehabilitation programme in another primary health care district. Methods: The instrument was investigated in two steps. First content validity and utility were investigated through a questionnaire addressed to occupational therapists and their clients respectively. The response distribution was calculated by frequencies. Internal consistency was investigated. In the second step, a revised version of the instrument was investigated for test-retest reliability and internal consistency. The test-retest reliability was calculated by weighted kappa. The internal consistency of the WEIS-SR was calculated by means of Cronbach’s alpha. Results: In step one the content validity was good to moderately good, the utility was good, and the internal consistency was satisfactory (0.72). In step two the internal consistency was good (0.88/0.89) and the test-retest reliability was mostly good to moderate (0.35–0.78, median 0.61). Conclusions: The instrument will be further investigated in other populations and take into consideration additional psychometric properties such as sensitivity to change, predictive validity, and concurrent validity. Keywords: Return to work, work environment, assessment, psychometric properties, MOHO

1. Introduction 1.1. Return to work Returning to work after sick leave is an important issue in Sweden as well as in other western countries. Different factors have been shown to be important in returning to work, such as motivation and trust in one’s own ability [1–4], and belief in the relevance of returning to work [5]. The client’s ability to work, as ∗ Address for correspondence: Birgitta A. W¨ astberg, Department of Health Sciences, Division of Occupational Therapy and Gerontology, Lund University, Faculty of Medicine, P.O. Box 157, SE 221 00 Lund, Sweden. Tel.: +46 462221951; Fax: +46 462221959; E-mail: [email protected].

well as to perform other daily functions, depends on the demands of the tasks to be performed, the person’s working skills and the properties of the environment where the work is to be performed [6–10]. This study will focus assessment of the work environment. 1.2. The environment Scientists [11,12], as well as the International Classification of Functioning, Disability and Health (ICF) [13], agree that the environment is important for people’s health. The ICF also states that the environment can offer both facilitators and barriers for performing duties, such as one’s job, when an individual is suffering a disability [13].

1051-9815/12/$27.50  2012 – IOS Press and the authors. All rights reserved

448

B.A. Wästberg et al. / WEIS-SR evaluated in primary health care in Sweden

The concept of an occupation in this paper was used to denote not only work, but also other everyday tasks, such as dressing oneself, eating, taking care of one’s children, domestic tasks, leisure activities, etc. According to the Model of Human Occupation (MOHO), the individual’s occupational choices and behaviours emerge out of an interdependent relationship between the environment and the individual’s values, interests, personal causation, roles, habits, and performance capacities [7]. The environment furnishes opportunities for occupational performance as it represents specific potential for action. On the other hand, the environment presses for certain types of occupational behaviours in forms of expectations and demands. When performing occupations, the individual receives feedback that affects volition, habituation, and the performance capacity in a never-ending process [7,14]. Kielhofner, as well as other scientists, has stated that cultural values influence, and are implicit in, any aspect of the environment and the environment might be the critical dimension that either supports or interferes with the client’s occupation [7,8,10,13,15]. In the MOHO the environment comprises physical space, objects, and social groups [7,14]. The social environment, defined as the people with whom the individual is interacting [14] has gained increasing interest in recent years, not least in work science [15–18]. Support from employers and superiors has been found to be important for keeping healthy [18], avoiding absences due to illness and returning to work after sick leave [16,19,20]. The opposite, poor support and relations to superiors and colleagues, leads to increased stress [21]. Also, in work rehabilitation, support from the staff has been found to be important in facilitating a return to work [17]. By assessing the characteristics of the work environment and the impact these have on the individual’s work performance, satisfaction and health, the rehabilitation staff might identify needs for accommodation of the work environment, in order to enhance the individual’s work capacity [22,23]. In such assessments, reliable instruments are essential. The observable qualities of occupational performance can be measured objectively, while its subjective attributes may be more appropriately measured by a self-report [8]. Sandqvist and Henriksson stated that a work assessment should consider both an objective and a subjective perspective to catch the therapist’s assessment as well as the client’s perception in order to identify any conflicting opinions that might jeopardise the work rehabilitation [9]. The importance of subjective assessment is supported by

Theorell and Hasselhorn [24] who concluded that selfreport assessments have an important role in stress research. Also in clinical practice, when evaluating the result of different interventions, it is warranted to use subjective measurements [25]. Different scientists have also emphasised the importance of collaboration with the client in work rehabilitation. Taking the client’s opinion into account and letting the client participate and take responsibility have shown to give a better understanding of the client’s return to work behaviour [5] and facilitate success in the rehabilitation [1,2,19]. Still, since there might be a bias in self-reports in the area of vocational rehabilitation, the perspective of others, such as an independent researcher or a professional involved in the rehabilitation process, is also important. 1.3. Measurement of the work environment An instrument that may be used by occupational therapists in work rehabilitation is the Work Environment Impact Scale (WEIS), which is a semi-structured interview instrument designed to gather information about the client’s experiences of his/her work environment [26]. The WEIS interview is based on seventeen items which reflect different aspects of the physical and social environment at the client’s working place and the impact of those aspects on his/her ability to return to work or to maintain employment after illness or injury. The instrument has been translated into Swedish [26], and after some evaluation and minor revisions [27] a second version of the Swedish WEIS was distributed in 2000 [28]. The construct validity has been found to be good [26] and the internal consistency to be adequate [22]. In a recent study the WEIS was found to be useful for providing information about how the work environment supports or interferes with the clients’ work performance [23]. After conducting the WEIS interview, which takes about 25 minutes, the occupational therapist rates each of the items on a four-point scale to indicate how strongly the factor it represents supports or interferes with the return to work [28]. This rating takes another 15 minutes to complete [23]. A related instrument, the Worker Role Interview (WRI), focuses on the worker role. Both instruments are based on the Model of Human Occupation [7]. Bearing in mind the need for instruments that highlight the client’s perspective, a self-report alternative to the WRI, Worker Role Self-assessment (WRS) was developed and tested in a pilot study [29]. As there has been a request from clinical occupational therapists for a self-rating perspective


449

also regarding the work environment the authors developed a self-report alternative to the WEIS, called the Work Environment Impact Scale – Self Rating (WEISSR), where the client him/herself reads the items and do the ratings. When developing new outcome measures or changing old ones, it is necessary to investigate the psychometric properties of the new instrument [30,31]. Clinicians need to use sound and credible instruments. Baseline information, the quality of intervention planning and progress evaluation all stem from the reliability and the validity of the measures used [15,30]. An investigation of work-related assessments and the extent to which they showed evidence of validity and reliability, demonstrated that most of the assessments had limited evidence in these respects [32,33].

“agree”, “agree slightly”, “disagree slightly”, and “disagree”. The WEIS-SR items are about the physical and the social environments and the work tasks, for example “I have necessary and well-functioning work tools” (physical environment), “there is good communication with my superiors” (social environment), and “my work tasks are stimulating” (work tasks). The psychometric properties of the WEIS-SR were investigated in two steps. The first version of the instrument was used and tested in Sample 1, presented below. According to this test’s results, a second version of the instrument was formed where the items were the same but the response alternatives were changed to a five-point rating scale: “Strongly agree”, “Moderately agree”, “Partly agree”, “Moderately disagree”, and “Strongly disagree”. The revised version was tested in the second step, with Sample 2, also described below.

2. Aim

3.1. Subjects

Thus, instruments with good psychometric properties need to be developed, not least self-report measures that reflect the client’s own perspective. Therefore, the aim of this study was to investigate the content validity, utility, internal consistency, and test-retest reliability of the novel self-report instrument WEIS-SR. Internal consistency is the degree to which the measurement’s items target the same phenomenon, while content validity is the extent to which the instrument covers the domains of interest [31–34]. Utility has been defined as an instrument’s usefulness [35] and addresses issues such as accuracy, comprehensiveness, credibility, flexibility, practicality and relevance. Ceiling and floor effects (when most respondents use the highest or lowest response options), might influence the utility, because there is no room for change. Test-retest reliability means the stability of the result of a measurement between two test occasions [31].

Step 1: The first part of the investigation took place in two primary health care districts in southern Sweden. Occupational therapists working with vocational rehabilitation in these districts were asked to participate by using the WEIS-SR with their clients. The occupational therapists and the clients were to assess utility and content validity from their different points of view regarding the novel instrument (as therapists and clients, respectively). All occupational therapists working within vocational rehabilitation in the selected districts were asked to participate and in turn ask the clients who matched the inclusion criteria for using the WEIS-SR. These criteria were the clients should understand written and spoken Swedish, and have a working place to which they could relate the items. Clients both working and on sick leave were included. The clients obtained written and oral information about the investigation, including that participation was voluntary. In all, 10 occupational therapists and 45 clients who took part in occupational therapy interventions participated, forming Sample 1. The occupational therapists contributed with 1–13 clients (median 3 clients). None of the clients asked to participate declined. Step 2: In the next step, the second version of the instrument (WEIS-SR-2) was used with women on sick leave for stress-related disorders who participated in a work rehabilitation programme in primary health care in two towns in southwest Sweden. The clients filled out the WEIS-SR-2, as well as other instruments not used for the present study, before they entered the rehabilitation programme, after the programme, and at

3. Material and method The WEIS-SR was developed on the basis of the items of the WEIS, which were reformulated and adapted to fit into a self-assessment scale. In order to reduce the number of items, three items of the WEIS, physical comfort, physical design and sensory characteristics, were abridged to form one item in the WEIS-SR, namely the physical environment. This first version of the instrument contained fifteen items and used a four-point rating scale. The response alternatives were

450

B.A. Wästberg et al. / WEIS-SR evaluated in primary health care in Sweden Table 1 Characteristics of the clients in Sample 1 and Sample 2

Gender Age in years Country of origin On sick leave Months on sick leave Prior rehabilitation Health problems/sick leave diagnosis1

Male Female Range Median Sweden Nordic Other Yes No Data missing Range Median Yes No Data missing Mental diagnostics/symptoms Pain-related symptoms Physical symptoms2 Cognitive problems Other problems3 No problems Data missing

Sample 1 n = 45 10 35 26–61 43 36 2 7 24 17 4 0.5–120 11 12 33 0 26 22 6 5 5 2 6

Sample 2, n = 26 0 26 24–62 45 23 0 3 26 0 0 2–18 5.5 3 22 1 24 3 2 0 0 0 0

1 In

Sample 1, the clients were asked to answer which health problems prevented them from working, while in Sample 2 the clients were asked what caused them to be on sick leave. Some of the clients had problems from more than one group, which means that the sum of health problems exceeds the number of clients. 2 Physical symptoms were dizziness, cardiac arrhythmia, digestive problems. 3 Other problems were heavy workload, working alone, too much new technology, difficulty in carrying and lifting things.

two follow-ups, 6 and 12 months after completion of the rehabilitation. The clients’ conditions were regarded to be fairly stable at the six-month follow-up, and therefore this measurement point was used as the first measurement in the test-retest investigation. In total 35 clients participated in the six-month follow-up, but five of them had no working place to relate to, either because they had left their jobs or they had been away from their place of work for too long. Therefore they could not complete the WEIS-SR. One client declined to participate in the test-retest-study, which made twenty-nine clients who filled out the WEIS-SR2 at the six-month follow-up. Twenty-six clients of them, forming Sample 2, completed the retest of the WEIS-SR-2, which was administered 3–7 days later. 3.2. Characteristics of the participants The occupational therapists (Sample 1) were all women. Their ages varied between 29–60 years of age (median = 45 years old), and they had been working as occupational therapists for 2–31 years (median = 19.5 years). The characteristics of the clients (age, gender, country of origin, on sick leave or not, how

long they had been on sick leave and why, prior rehabilitation or not) in Samples 1 and 2 are described in Table 1. In Sample 1, two clients reported that they had no medical problems and were not on sick leave, while six clients did not answer the question about health problems. Two of those were not on sick leave, one had not answered the question about sick leave either, while the other three responded that they had been on sick leave for 4, 10 and 96 months respectively. 3.3. Procedures In the first stage, a validity and utility questionnaire was used to obtain the opinions of the occupational therapists and the rehabilitation clients concerning the WEIS-SR (Table 2). The questionnaire contained questions about content validity composed in accordance with Streiner and Norman [31]. The occupational therapists were to answer all questions, while the clients were to answer all except the question about irrelevant items. Four questions about utility, according to Innes and Straker’s [35] definition, were also included. One of those questions, about how easy it was to respond


451

Table 2 The questionnaire used to determine the validity and utility of the WEIS-SR instrument Questionnaire items (response options)

In your opinion, how easy was it to understand the content and the meaning of the items in the WEIS-SR? (Scale 1–5: 1 = very easy, 5 = very difficult) In your opinion, how easy was it to respond to the items? (Scale 1–5: 1 = very easy, 5 = very difficult) How well do you think the WEIS-SR covers the work environment? (Scale 1–5: 1 = very well, 5 = very poorly) Do you think anything important about the work environment is missing on the WEISSR? (Yes/No) If something is missing, please indicate what. Do you think the WEIS-SR contains items that are irrelevant to the work environment? (Yes/No) Please specify any irrelevant items How many minutes did it take you/the client to fill in the WEIS-SR? In your opinion, how acceptable was the amount of time spent on responding? (Scale 1–5: 1 = very reasonable, 5 = very unreasonable)

Aspect assessed Content Utility validity X

Respondents Occupational Clients therapists X X

X

X

X

X

X

X

X

X

X X

X X

X

X

X X

X X X

X X

Note. A cross in the first two columns indicates whether the item corresponds to validity or utility, while a cross in the last columns shows which respondents the item addresses.

to the items, was only addressed to the clients. The clients’ demographic characteristics, such as age, gender, country of origin, health problems, sick leave and prior rehabilitation, were collected by another questionnaire answered by the clients in order to define the investigated sample. The clients filled out the instruments when they met the occupational therapists for the first time. After having completed the WEIS-SR, as well as the questions about demographic characteristics, the clients answered the questionnaire about content validity and utility. The occupational therapists answered the validity and utility questionnaire only once, regardless of how many clients they met. All clients were guaranteed anonymity, and the questionnaires were given a code number by the occupational therapists before sending them to the researchers. In step 2, the revised version of the instrument, WEIS-SR-2, was used. The clients’ WEIS-SR-2s at the six-month follow-ups were used as baselines and the instrument was re-administered by ordinary mail 3–7 days later, together with a stamped return envelop for return within a week. The clients’ characteristics were collected when they entered the rehabilitation programme. 3.4. Ethical considerations The study complied with stipulations in the Swedish act designated for the Ethical Review of Research Involving Humans [36], and the principle of informed consent was applied. In addition, approval for Sam-

ple 2 was given by the Regional Ethical Review Board in Lund, since an intervention was linked to the project (case no 922/2004). 3.5. Data analysis 3.5.1. Content validity The responses concerning content validity were analysed by descriptive statistics [31],and the analyses were made separately for the occupational therapists and the clients. Content validity was categorised in accordance with Innes and Straker [33]: “good” is when most respondents agree that the test measures what it intends to and that all relevant components are included, “moderate” is when most experts consider there is some relationship between the test and the intentions, but some relevant items are missing, while “poor” means little relationship between the test and the intentions. Content validity was investigated only in Sample 1. 3.5.2. Utility The questionnaire responses about utility were also analysed by descriptive statistics, separately for occupational therapists and clients in Sample 1. To investigate the distribution of the responses to the WEIS-SR and the WEIS-SR-2 and check for any ceiling or floor effects, frequencies were calculated, and both samples were used for this purpose. 3.5.3. Internal consistency Cronbach’s alpha was used to investigate the internal consistency. If a scale contains more than four-

452


teen items the Cronbach’s alpha value should be 0.70 or higher, although a value above 0.8 is more desirable [31]. Some scientists think the alpha value should be 0.6 or 0.7, while others recommend 0.8 or 0.9 [37]. Internal consistency was calculated for Sample 1 but also for Sample 2, as the response alternatives had been changed.

kind of problem related to their working place and led to revision of the response options to a five-point rating scale. In the second step, with the revised version (WEIS-SR-2) and another sample, there was no ceiling effect.

3.5.4. Test-retest Weighted kappa (κw ), was calculated to examine the test-retest agreement. Weighted kappa is recommended with scales that, as WEIS-SR-2, use more than two response alternatives [38]. The maximum result with this statistic is 1 (perfect agreement), and 0 indicates no agreement better than chance [38]. According to Altman [38], five categories of agreement can be discerned: poor (0–0.19), fair (0.2–0.39), moderate (0.4– 0.59), good (0.6–0.79), and very good (0.8–1.0). Testretest agreement was investigated only in Sample 2. The software used was the SPSS version 11.5 in all analyses, except the weighted kappa, which was calculated by means of a formula accessible on the Internet.

The Cronbach’s alpha value was higher in Sample 2 (using the revised version WEIS-SR-2) compared to Sample 1 (using the first version WEIS-SR), as shown in Table 4 together with the corrected item-total correlations (CITC). The results based on Sample 1 showed that deleting item 11 would have yielded an alpha value of 0.77 for the whole scale, while deleting items 6 and 15 would not have made a notable difference in the alpha value. According to the retest in Sample 2, deleting item 4 would have yielded a marginally higher alpha-value (0.90).

4. Result 4.1. Content validity and utility Most of the occupational therapists and the clients perceived the content validity and the utility were good, but some of them thought items were missing. They missed some aspects of the work environment, such as physical load and characteristics, support from and trust of colleagues and managers/supervisors, possibilities to influence the work situation and the balance between one’s own demands and demands from the environment. Additionally, some clients thought that the WEIS-SR lacked questions for those self-employed or who work alone. One of the occupational therapists believed there were irrelevant items and meant that two of the items were too similar, and that another question was unclear. Table 3 shows the distribution of answers from occupational therapists and clients separately, on each question in the content validity and utility questionnaire. Two of the items (nos. 5 and 10) showed a ceiling effect in the first step of the investigation, using the first version of the WEIS-SR with Sample 1 (more than 80% of the clients had given the highest ratings). This was seen as cumbersome since all clients had some

4.2. Internal consistency

4.3. Test-retest reliability A majority of the items, 8 out of 15, showed good test-retest agreement, as shown in Table 5. Especially items nos. 7 and 10, which exhibited kappa values above 0.70, proved to have good test-retest reliability. Only item 4 had a kappa value below 0.40, and it was still in the realm of “fair”. The remaining six items showed moderate agreement.

5. Discussion 5.1. Psychometric properties 5.1.1. Content validity Most of the clients and the occupational therapists thought the WEIS-SR covered the work environment well, but some of them missed items pertaining to certain aspects of the environment. However, some of the items that were viewed as missing by the clients and the occupational therapists are in fact included in the items of the WEIS-SR, such as the physical work environment (including visual and auditory distractions), meaningful work tasks, work demands and support from superiors. Still, these comments demonstrate a need for additional item clarification, preferably by developing a manual that gives instructions and explains the meaning of and the difference between the

Items about possibility for breaks Feedback on item number 11 Items about meaningful work tasks Enough support from superiors More about physical environment such as monotonous, physical work load

1 One

5 = very reasonable/7 4/2 3/1 2/0 1 = very unreasonable/0 5 = very reasonable/21 4/20 3/1

5 = Very easy/9 4/25 3/5 1–60 minutes (median 6 minutes)

– 1–60 minutes (median 6 minutes)

2/1 1 = very unreasonable/1

2/5 1 = very difficult/0

Clients, n = 451 Response category/number of responses 5 = Very easy/15 4/18 3/8 2/3 1 = very difficult/0

Occupational therapists, n = 10 Response category/number of responses 5 = Very easy/6 4/3 3/1 2/0 1 = very difficult/0

client did not answer the questionnaire about content validity and utility.

Clinical utility Question – How easy was it to understand the content and the meaning of the items? – How easy was it to respond to the items? – How many minutes did it take the client/you to fill in the WEIS-SR? – How acceptable was the amount of time spent on responding?

–

Items about breaks Items for those self-employed Items for those who work alone Space, noise, outdoor work Trust in/support from the management Balance own demands and environmental demands Mental work load The items should be more explicit – –

No/33

Yes/11

Yes/4

No/6

Clients, n = 451 Response category/number of responses 5 = very well/12 4/14 3/14 2/2 1 = very poorly/0

Occupational therapists, n = 10 Response category/number of responses 5 = very well/4 4/5 3/1 2/0 1 = very poorly/0

– Does the WEIS-SR contain items that are irrel- Yes/1 No/9 evant to the work environment? – Irrelevant items? Items number 5 and 6 are too similar. Item number 4 could be misunderstood to deal with different activities at work

Content validity Question – How well does the WEIS-SR cover the work environment? – Is anything important about the work environment missing? – What is missing?

Table 3 Answers to the questionnaire about content validity and clinical utility B.A. Wästberg et al. / WEIS-SR evaluated in primary health care in Sweden 453

454


Table 4 Internal consistency of the WEIS-SR (Sample 1), and the revised version WEIS-SR-2 (Sample 2)

WEIS-SR Items 1–5, 7–10, 12–15 Item 6 Item 11 Item 15 WEIS-SR-2 first test WEIS-SR-2 retest Items 1–3, 5–15 Item 4

Cronbach’s alpha value 0.72

0.88 0.89

Corrected item-total correlations (CITC) 0.23–0.62 0.05 −0.07 0.03 0.35–0.77 0.37–0.77 0.16

items, thereby also meeting the occupational therapists’ thoughts about irrelevant items. Items numbers 5 and 6 presuppose that the client has colleagues and is part of a group at work. Those items are not applicable to those who work alone, and item number 7 (interaction with superiors) is not applicable to those who do not have a superior at work. The instruction says that “if any of the questions is nonapplicable, please note that beside the question”. However, no one utilised this opportunity, indicating that also this option needs to be further explained in a manual. Besides, a column for “not applicable” should be added on the questionnaire. Regarding the WEIS, Kielhofner and co-workers [26] found that the item “Interaction with others” was ill-fitting because clients who did not co-operate with others nevertheless answered the question. They suggested that there should be a response alternative such as “not applicable” [26]. Having that response alternative to each item is an alternative for our response scale as well. Most of the clients and occupational therapists were satisfied with the WEIS-SR. The content validity must be verbalised to be good to moderately good, according to the criteria set by Innes and Straker [33]. Streiner and Norman [31] stated that if there are important aspects missing in a scale, it is not possible to draw valid inferences. However, although some of the occupational therapists gave comments regarding missing items, according to the generally positive content, validity results in the WEIS-SR should allow for valid inferences. 5.1.2. Utility Most of the clients and the occupational therapists thought it was “very easy” or “rather easy” to understand the meaning and the content of the items in the WEIS-SR. The clients were also supposed to answer about how easy it was to respond to the items, and the

Table 5 The WEIS-SR-2 test-retest agreement calculated by weighted kappa

Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10 Item 11 Item 12 Item 13 Item 14 Item 15

Linear weighting 0.35 0.59 0.50 0.40 0.65 0.62 0.71 0.45 0.61 0.78 0.62 0.68 0.58 0.67 0.42

Agreement according to Altman (38) Fair Moderate Moderate Moderate Good Good Good Moderate Good Good Good Good Moderate Good Moderate

majority meant that it was “very easy” or “rather easy”. Only a few thought it was “in between” or “rather hard”, respectively. The time for completing the WEIS-SR was on average 6 minutes, and it was seen as “very reasonable” or “rather reasonable” by the most of the clients and occupational therapists. According to this result, the WEIS-SR has good utility. 5.1.3. Internal consistency The alpha value of the first version of the WEIS-SR was 0.72 and according to Streiner and Norman [31] and Clark and Watson [37] it must be regarded as satisfactory. The revised version, WEIS-SR-2, showed alpha values of 0.88 and 0.89, which must be regarded as very good. The corrected item-total correlations were low for three WEIS-SR items in the first step of the investigation, but in the second step only one item-total correlation was below 0.20. These improved values might be due to the fact that in Sample 1 the WEIS-SR was handled by ten different occupational therapists, who had got oral instructions about the instrument and the investigation, while in Sample 2 the instrument was handled by a few data collectors who were more used to the instrument. If this explanation is valid it further emphasises the importance of developing a manual to be used in further studies. The increases in alpha values indicated by removal of some of the items were only marginal and did not imply that item deletion would be warranted. Besides, any removal should be based on more thorough investigation of consistency, such as factor analysis or Rasch analysis, which also requires larger samples.


5.1.4. Test-retest reliability. All items but one showed good or moderate testretest reliability. This means that the WEIS-SR may be used in outcome studies based on the targeted population. However, additional studies in other populations are needed, especially since Sample 2 is comprised only of women and it is important to ensure that instruments function for both genders. 5.2. Need for further development of the WEIS-SR The need for instruments assessing characteristics of the job environment has been addressed in previous work. Ekberg and Wildhagen [39] highlighted the importance of including the clients’ work conditions in rehabilitation, since these determine which interventions may be appropriate and also influence the possibilities for returning to work. Furthermore, Ekberg [19] pointed out the importance of incorporating the work environment in programs of prevention, involving physical adjustments as well as organisational and psychosocial interventions. The WEIS-SR, which contains both physical and psychosocial items, could be a valuable tool in preventive work for early detection of work environmental problems. However, the validity of subjective outcome measures has been discussed because the targeted phenomenon can be perceived differently by the clients [40]. On the other hand, the clients’ subjective perceptions of their possibilities and obstacles are significant to the outcome of rehabilitation. According to the MOHO, the clients’ own opinions and experiences are the most important ones because the way in which they perceive their environment affects their occupational choices and performance. The results of the present study indicate a need for continued development and investigation of the WEISSR. In order to distinguish the two versions tested in the present study the revised form has been denote the WEIS-SR-2, but we subsequently refer to this revised version as WEIS-SR. The items need to be clarified in a manual and a “not applicable” response alternative should be added. Hopefully these measures will increase the reliability of the instrument and reduce the measurement error. The WEIS-SR should also be tested regarding other psychometric properties, such as sensitivity to change, predictive validity and concurrent validity. It should also be tested with other populations, including people who are not on sick leave, for psychometric properties and discriminating ability. We recommend using both the WEIS-SR and the original (interview) WEIS in future studies, since the former gives

455

the client perspective and the latter the occupational therapist’s view. Besides, comparing results from the two instruments would yield an estimate of criterion validity, in terms of concurrent validity. 5.3. Methodological considerations The methods used for evaluating content validity, utility, internal consistency, and test-retest reliability in this study were in accordance with the literature. By testing the WEIS-SR in two stages it was possible to develop the instrument, in this case revising the response alternatives. The two samples also offered different possibilities: By allowing the clients in Sample 1, as well as the occupational therapists, evaluate the content validity and the utility we obtained opinions from people with various experiences. The clients in the second sample participated in a rehabilitation programme with follow-ups which provided us the opportunity to make the test-retest reliability evaluation when their situation was expected to be rather stable. Thus, this was done in connection with the follow-up evaluation after six months. Future studies should investigate also other psychometric properties, such as concurrent validity, predictive validity, and sensitivity to change. It would also be valuable to include factor analysis or Rasch analysis to investigate the homogeneity of the instrument, based on a larger sample, in further studies of the psychometric properties. Both samples were rather small, but there was a distribution according to age and sick-leave duration. Sample 1 was also heterogeneous according to gender, health problems, being on sick leave or not, and whether or not having received previous rehabilitation. The fact that the WEIS-SR was tested and evaluated at primary health care clinics in Sweden might however limit the ability to generalise to similar units. Still, this was the first investigation of the psychometric properties of the WEIS-SR and the results indicate that it is worth continuing the development and evaluation of the instrument. Further studies will focus on other samples and other psychometric properties, as mentioned above. 5.4. Clinical implications There is an obvious need for measurements of the work environment, and the clients’ perceptions in this respect are very important for successful rehabilitation (19,24). The WEIS-SR was found to be easy to understand and to use. The time needed to fill out the instru-

456


ment was perceived as rather short and reasonable. Another factor that may add to the usability is the fact that the WEIS-SR is based on theory, the MOHO, which helps interpret the results and plan interventions. Thus, the WEIS-SR can be regarded a valuable tool for clinicians in their evaluations of clients’ work environments and when planning interventions together with clients. Polit and Hungler [41] regard some properties as essential when introducing new routines: clinical relevance, scientific merit, and implementation potential. According to the results, the WEIS-SR seems to have clinical relevance and the fact an instrument is based on theories in occupational therapy should strengthen its implementation potential. The scientific merits are still modest, but are likely to increase with additional revision and investigations of the WEIS-SR. The results of our investigation show that the WEIS-SR may fit well for use in primary health care, but it also might be a valuable tool in other health care clinics. Several rehabilitation actors, such as the Social Insurances Offices, base their discussions and considerations regarding return to work on the ICF [13]. As the WEIS-SR is largely compatible with the ICF regarding its emphasis on the importance of the work environment, it might also fit in with and supplement the routine assessments of these actors. Self-report instruments, such as the WEIS-SR, require that the client is able to read and understand the items and the response alternatives. Therefore, the WEIS-SR might not be suitable for those who have problems understanding the language used in the instrument, and for those who have difficulties reading for other reasons, such as cognitive problems. An alternative is that the therapist supports the client by reading the questions aloud. Moreover, the target group of the WEIS-SR is limited. It addresses only those who have employment or a specific job to evaluate. Besides, the WEIS-SR rating shows only what the client thinks about his/her work environment, and nothing about why he/she thinks that way. The WEIS-SR could be seen as an alternative to the WEIS and other measurements of work environment conditions. It might also be combined with objective assessments of the work environment and the professionals’ evaluations of the clients’ working skills. Additionally, the WEISSR may well be used together with the WRS (Worker Role Self-assessment) [29] to give a comprehensive picture of the client’s own perceptions of his/her work situation.

6. Conclusion The content validity of the WEIS-SR was seen as good to moderately good, the utility as good and the internal consistency was satisfactory to very good. The test-retest reliability was found to be good to moderate. The results indicate that it may be used in the revised form, but also that a manual with explanations and instructions should be developed, and that further development of the instrument is warranted. Thereafter we expect the WEIS-SR to be a tool for aiding clinicians when evaluating work environment and planning rehabilitation interventions. Acknowledgements We want to thank the occupational therapists within the primary health care for helping us collect the data. We are also grateful for statistical help from PhD Per Nyberg. The study was financed by the Swedish Council for Working Life and Social Research and the Primary health care in Region Skåne. References [1] [2] [3] [4] [5]

[6]

[7] [8]

[9] [10]

Berglind H, Gerner U (2002) Motivation and return to work among the long-term sick-listed: An action theory perspective. Disability and Rehabilitation 24(14), 719–726. Grahn B (1999) Quality of life, motivation and costs in multidisciplinary occupational rehabilitation, Doctoral thesis. The University of Lund. Hansen A, Edlund C, Henningsson M (2006) Factors relevant to a return to work: a multivariate approach. Work 26, 179– 190. Mårtensson L (2005) Rehabilitation of persons with chronic pain. An approach built on understanding, Doctoral thesis. The University of Göteborg. Shaw L, Segal R, Polatajko H, Harburn K (2002) Understanding return to work behaviours: promoting the importance of individual perceptions in the study of return to work. Disability and Rehabilitation 24(4), 185–195. Dunn W, Brown C, McGuigan A (1994) The ecology of human performance: A framework for considering the effect of context. American Journal of Occupational Therapy 48(7), 595–607. Kielhofner G (2008) Model of Human Occupation. Theory and application (4th ed.), Williams & Wilkins. Law M, Cooper B, Strong S, Stewart D, Rigby P, Letts L (1996) The Person-Environment-Occupation Model: a transactive approach to occupational performance. Canadian Journal of Occupational Therapy 63(1), 9–23. Sandqvist JL, Henriksson CM (2004) Work functioning: a conceptual framework. Work 23, 147–157. Townsend EA, Polatajko HJ (2007) Enabling occupation II: advancing an occupational therapy vision for health, wellbeing & justice through occupation, CAOT Publications.

B.A. Wästberg et al. / WEIS-SR evaluated in primary health care in Sweden [11]

[12]

[13]

[14] [15] [16]

[17] [18]

[19] [20] [21]

[22] [23] [24]

[25]

Stewart D, Law M (2003) The environment: Paradigms and practice in health, occupational therapy, and inquiry. In Using Environments to Enable Occupational Performance, Letts L, Rigby P, Stewart D, eds. Slack, pp. 3–15. Theorell T, Harms-Ringdahl K, Ahlberg-Hultén G, Westin B (1991) Psychosocial job factors and symptoms from the locomotor system – a multicausal analysis. Scandinavian Journal of Rehabilitation Medicine 23, 173–191. World Health Organisation (2001) International Classification of Functioning, Disability and Health (ICF) http://www. who.int/classifications/icf/en/index.html, Accessed on February 7, 2008. Kielhofner G (1995) A Model of Human Occupation. Theory and application (2nd ed.), Williams & Wilkins. Law M, Baum C, Dunn W (2005) Measuring Occupational Performance. Supporting best practice in occupational therapy (2nd ed.), Slack. Holmgren K, Dahlin Ivanoff S (2004) Women on sickness absence – views of possibilities and obstacles for returning to work. A focus group study. Disability and Rehabilitation 26(4), 213–222. Klanghed U, Svensson T, Alexanderson K (2004) Positive encounters with rehabilitation professionals reported by persons with experience of sickness absence. Work 22, 247–254. Oxenstierna G, Ferrie J, Hyde M, Westerlund H, Theorell T (2005) Dual source support and control at work in relation to poor health. Scandinavian Journal of Public Health 33, 455– 463. Ekberg K (1995) Workplace changes in successful rehabilitation. Journal of Occupational Rehabilitation 5(4), 253–269. Hansen Falkdal A, Edlund C, Dahlgren L (2006) Experiences within the process of sick leave. Scandinavian Journal of Occupational Therapy 13, 170–182. MacDermid JC, Geldart S, Williams RM, Westmorland M, Lin C-YA, Shannon H (2008) Work organization and health: a qualitative study of the perceptions of workers. Work 30, 241–254. Corner RA, Kielhofner G, Lin FL (1997) Construct validity of a work environment impact scale. Work 9, 21–34. Ekbladh E, Thorell LH, Haglund L (2010) Perceptions of the work environment among people with experience of long term sick leave. Work 25, 125-136. Theorell T, Hasselhorn HM (2005) On cross-sectional questionnaire studies of relationships between psychosocial conditions at work and health – are they reliable? International archives of occupational and environmental health 78(7), 517–522. Roos E (1999) Knee injury and knee osteoarthritis: development, evaluation and clinical application of patient-relevant questionnaires, Doctoral thesis. The University of Lund.

[26] [27]

[28] [29]

[30]

[31] [32] [33] [34] [35] [36]

[37] [38] [39]

[40] [41]

457

Kielhofner G, Shei Lai J, Olson L, Haglund L, Ekbladh E, Hedlund M (1999) Psychometric properties of the work environment impact scale: a cross-cultural study. Work 12, 71–77. Ekbladh E (1999) Prövning av svenska versionerna av Worker Role Interview och Work Environment Impact Scale. (Investigation of the Swedish versions of the Worker Role Interview and the Work Environment Impact Scale) (In Swedish), Master’s thesis. The University of Linköping. Ekbladh E, Haglund L (2000) WEIS-S version 2. The Work Environment Impact Scale (In Swedish). The University of Linköping. Wästberg BA, Haglund L, Eklund M (2009) Psychometric properties of the Worker Role Self-assessment instrument used to evaluate unemployed people in Sweden. Scandinavian Journal of Occupational Therapy 16(4), 238–246. Jerosch-Herold C (2005) An evidence-based approach to choosing outcome measures: a checklist for the critical appraisal of validity, reliability and responsiveness studies. British Journal of Occupational Therapy 68(8), 347–53. Streiner DL, Norman GR (2003) Health Measurement Scales. A practical guide to their development and use (3rd ed.), Oxford University Press. Innes E, Straker L (1999) Reliability of work-related assessments. Work 13, 107–124. Innes E, Straker L (1999) Validity of work-related assessments. Work 13, 125–152. Kazdin AE (2002) Research Design in Clinical Psychology (4th ed.), Allyn and Bacon. Innes E, Straker L (2003) Attributes of excellence in workrelated assessments. Work 20, 63–76. SFS (2003:460). Lag om etikprövning av forskning som avser människor. [The Act Concerning the Ethical Review of Research Involving Humans.] (In Swedish). Riksdagen, Stockholm. Clark LA, Watson D (1995) Constructing validity: Basic issues on objective scale development. Psychological Assessments 7(3), 309–319. Altman DG (1991) Practical statistics for medical research, Chapman & Hall, London. Ekberg K, Wildhagen I (1996) Long-term sickness absence due to musculoskeletal disorders: the necessary intervention of work conditions. Scandinavian Journal of Rehabilitation Medicine 28, 39–47. Smith, DM, Brown SL, Ubel PA (2008) Mispredictions and misrecollections: challenges for subjective outcome measurement. Disability and Rehabilitation 30(6), 418–424. Polit DF, Hungler BP (1999) Nursing research. Principles and methods (6th ed.), Lippincott, Philadelphia.