Validation of the Behavioural Observation Scale ... - Wiley Online Library

ORIGINAL ARTICLE

Validation of the Behavioural Observation Scale 3 for the evaluation of pain in adults M. Frasca1, B. Burucoa1, S. Domecq2, N. Robinson2, V. Dousset3, M. Cadenne3, F. Sztark4, M. Floccia5 1 2 3 4 5

Department of Palliative Care, University Hospital of Bordeaux, France CCECQA Coordination and Clinical Evaluation of Quality in Aquitaine, Bordeaux, France Centre of Evaluation and Treatment of the Pain, University Hospital of Bordeaux, France Department of Anaesthesia and Resuscitation, University Hospital of Bordeaux, France Department of Geriatrics, University Hospital of Bordeaux, France

Correspondence Matthieu Frasca E-mail: matthieu.frasca@chu-bordeaux.fr Funding sources None. Conflicts of interest None declared.

Accepted for publication 8 March 2017 doi:10.1002/ejp.1049

© 2017 European Pain Federation - EFICâ

Abstract Background: Many behavioural scales are available to assess pain but none are suitable for a quick evaluation of non-sedated and nongeriatric adults. The Behavioural Observation Scale 3 (BOS-3) is short, composed of five items. This study examined its feasibility and diagnostic performances. Methods: Adult patients were included from medical and surgical departments of the University Hospital of Bordeaux. In a cross-sectional study, BOS-3 was compared to Numerical Rate Scale (NRS) with communicating patients (CP) and Behavioural Scale for the Elderly Person (ECPA2) with non-communicating patients (NCP). Each time, BOS-3 and reference scale were performed by an internal caregiver and an external expert. Results: We included 447 patients: 395 communicating and 52 noncommunicating. All patients were assessed by the BOS-3 and the reference test. All BOS-3 were carried out in less than one minute with only four missing data. Its reproducibility (ICC = 0.77 [95% CI 0.73– 0.81] with CP and 0.93 [95% CI 0.89–0.97] with NCP) and its internal consistency (Cronbach a = 0.67 with CP and 0.70 with NCP) were good. In non-communicating patients, ROC analysis set a threshold at 3 on 10. Sensitivity was 0.87 [95% CI 0.77–0.96], specificity 0.97 [95% CI 0.93–1.00], positive predictive value 0.93 [95% CI 0.86–0.99] and negative predictive value 0.95 [95% CI 0.89–1.00]. In communicating patients, sensitivity decreased to 0.34 [95% CI 0.28–0.38] but specificity reached 0.96 [95% CI 0.94–0.98] and positive predictive value 0.75 [95% CI 0.70–0.79]. Conclusions: BOS-3 had good metrological properties in non-communicating adults. With communicating patients, a positive BOS-3 could be an additional tool to confirm pain, when underestimated on the NRS. Significance: This study describes the diagnostic performances of a behavioral pain assessment scale designed for non-geriatric and non-sedated adults. The results show its validity in noncommunicating patients and suggest its usefulness as an ancillary tool in communicating patients in whom simple numerical scales are often insufficient.

Eur J Pain

(2017)

–

1

Validation study

1. Introduction In France, 20% of adult patients hospitalized in medicine and surgery departments are unable to evaluate their pain, or at least, verbally express its intensity (Patrice, 2005). This population have been scrutinized for pain evaluation (Brahimi et al., 2000; De Tovar et al., 2002; McNair et al., 2004; Chatelle et al., 2008). According to guidelines, self-reported scales assessing pain intensity (e.g. the numeric rating scale) must be used in first-intention when patients can express their pain. However, these scales can sometimes fail to detect pain experience, especially in case of repeated measures (Herr et al., 2004). In that regard, the observation of pain behaviour is considered as an interesting complementary screening tool (McCahon et al., 2005). The American Geriatric Association (AGS) defined six categories of behaviours related to pain namely: facial expressions, verbalizations or vocalizations, body movements, changes in interpersonal interactions, activity patterns or routine or mental status (AGS, 2002). The rationale is to observe and determine if the patient behaviour is modified by a physical pain (ANAES, 2000). Behaviour long scales as Doloplus scale, Behavioural Scale for the Elderly Person (ECPA2) or Simplified Behavioural Scale (ECS) require a training period, a large number of items to be completed, a perfect knowledge of usual behaviours of the patient (Le Quintrec et al., 1995; Pautex et al., 2005; Herr et al., 2006a,b; Zwakhalen et al., 2006; Morello et al., 2007). Caregiver teams usually find them too constraining. Several simplified behaviour scales have been proposed. But for adults, i.e. people aged over 15 years old, most of these scales are designed for geriatric or sedated and intubated patients as Algoplus scale and Behavioural Pain Scale (BPS), respectively (Aissaoui et al., 2005; Zwakhalen et al., 2006; Aubin et al., 2007; Liu et al., 2010; Rat et al., 2011). Behavioural evaluation of pain in non-geriatric and non-sedated adults is hampered by the lack of adequate tool. One consequence of this situation is that behavioural scales are available in only a quarter of French departments and only 11% of health professionals declare using them (Patrice, 2005). French General Meetings for Pain thus called for distribution of behaviour tools for adults. New and short tools designed for a wider adult population would facilitate use of behavioural scales by several adult departments’ teams not specifically specialized in geriatrics or intensive care. Behavioural Observation

2 Eur J Pain (2017) –

M. Frasca et al.

Scale 3 (BOS-3) was designed by Francßois Boureau and complemented by our group with his agreement (Boureau et al., 1993). It is a simplified and short version of existing scales including most of the dimensions recommended by the AGS. The aim of this research was to validate the BOS-3 with non-sedated and non-geriatric adult patients hospitalized in medicine and surgery departments. This addressed the concerns of the 2006–2010 plan for optimizing pain support and the French General Meetings for Pain which called for the distribution of behaviour tools for adults. This cross-sectional study evaluated the feasibility and the acceptability of the BOS-3, its criterion and content validity, its reproducibility and its diagnostic performances.

2. Methods 2.1 Evaluation tools The BOS-3 was constructed from the ECFB (Behavioural Scale of Francßois Boureau) designed by Francßois Boureau. In its original form, the ECFB contained four items, each referring to an observable behaviour: audible (complaints) and visible (facial expressions, analgesic postures, cautious gestures). Each item was evaluated on a three-level scale (0: absent; 1: weak; 2: noticeable) (Boureau et al., 1993). After a literature review and a consultation with the multi-disciplinary expert group appointed for the BOS-3 project, a fifth item was introduced: aggressiveness/agitation or mutism/prostration. This item is usually found in behaviour scales and belongs to the six categories of painrelated behaviours identified by the AGS (2002). In addition, this kind of behaviour is easy to observe. Overall, the BOS-3 included five items, which enabled a global score to be computed ranging from 0 to 10. Definitions of each item were written out (Table 1). To compare to the BOS-3, the Numeric Rate Scale (NRS) was selected as the gold standard for the communicating patients. For non-communicating patients, the first difficulty was to determine an efficient scale for comparison with the BOS-3. No gold standard had been validated for the screening and the evaluation of non-communicating adult patients unknown to caregivers (Wary et al., 2001; Gomas et al., 2004). A series of possible scales were shortlisted from the literature according to practical constraints (patients and caregivers’ acceptability, time to complete). Four scales were initially selected: the Doloplus, the ECS (Sainte-P erine Simplified © 2017 European Pain Federation - EFICâ

M. Frasca et al.

Validation study

Table 1 Behavioural Observation Scale – 3.

2.2 Design

For each item, rate the situation best describing the patient Moans, complaints Cries, moans, screams with or without tears Wrinkled forehead, tense face Face expression, gaze expression and painful grimace Analgesic postures used to protect a body part when resting, ‘sitting down or lying’ Active search for an unusual posture, or spontaneous and continuous use of a protective posture for a presumably painful body part Cautious movements When solicited, coordinated or uncoordinated defence reaction to protect a presumably painful body part, or avoidance to use a presumably painful body part Aggressiveness/agitation or mutism/prostration More intense communication as revealed by a high agitation, or absence of/refusal to communicate as revealed by an absence of motion or a withdrawal Total Score The patient is in pain if the total score is superior or equal to 3

Score

Absent Weak Noticeable Absent Weak Noticeable Absent Weak Noticeable

0 1 2 0 1 2 0 1 2

Absent Weak Noticeable

0 1 2

Absent Weak Noticeable

0 1 2

/10

Behavioural Scale), the San Salvadour and the ECPA2 scales (Morello et al., 2007). Doloplus scale would have required too much time to perform. The ECS appeared not enough precise for a psychometric study and the San Salvadour scale too specifically designed for disabled patients. Algoplus scale, recently developed, was not published at this time. Even if it would have been, this scale is not validated for patients under 65 years old (Rat et al., 2011). The ECPA2, a geriatric scale, designed for careinduced acute pain is widely used in France and highly recommended by the French High Authority for Health (HAS) (ANAES, 2000). The main reason of choosing ECPA-2 as gold standard to validate BOS-3 is that both of these scales were designed for adult patients whatever age and kind of pain. The ECPA2 scale does not provide a set threshold to determine the existence of pain. However, the ECPA2 validation study showed that pain was nonexistent or very weak for a score of 8.5 2.9 (Morello et al., 2007). Based on these findings, a score strictly superior to 7 on the ECPA2 was defined as a positive score for the aim of this study. © 2017 European Pain Federation - EFICâ

An observational, comparative, cross-sectional study was run in order to examine the psychometric criteria of the BOS-3. The Pain Unit (USD), the Committee against Pain (CLUD) and the Committee for the Coordination and Clinical Evaluation of Quality in Aquitaine (CCECQA) initiated this research at the University Hospital of Bordeaux. The sample comprised communicating and noncommunicating patients aged over 15, hospitalized in short-stay units, who volunteered to a call for participation from the department in charge of quality. The following departments were excluded from the study: paediatrics and neonatology since the BOS-3 is primarily designed for adults; post-surgery surveillance units (recovery room) because of the high frequency of patients under sedation. A patient was excluded from the study if at least one of the following criteria was met: under the age of 15 years old, patient’s refusal or hypo-vigilant patient (score above 3 at Rudkin scale). Patients were categorized as communicating if they were able to evaluate their pain with the NRS. Otherwise, the reason for failing to respond to the NRS was researched (disorganized thoughts – dementia, delirium, encephalopathy, severe psychiatric disorders, phasic disorders, lack of understanding of the French language). Each patient was categorized as communicating or not by both the external expert and the internal caregiver, at the moment of the evaluation.

2.3 Data collection The study was carried out in 30 units at the University Hospital of Bordeaux between November 2007 and February 2008. Oral information was given to the patient or his family and oral consent was collected before any data collection. If no consent was obtainable or if patient or his family refused the study, patient was not included. In accordance with French regulations, publication of the data collected anonymously that do not deviate from the routine management of patients does not need to be declared or submitted to a research ethics board. Sample size estimate was based on the number of individuals required to achieve a 95% CI around the sensitivity and specificity estimates for the BOS-3 of up to 20%. We supposed performances of BOS-3 would approach those of Algoplus scale in the noncommunicating group (Rat et al., 2011). Settling an hypothesis of BOS-3 sensitivity and specificity at Eur J Pain

(2017)

–

3

Validation study

80%, 60 non-communicating patients were needed. According to literature, 20% of non-communicating patients were expected in our entire sample thus estimated at 300 patients. For both communicating and non-communicating patients, pain was evaluated following two steps. (1) For each included patient, an internal caregiver of the unit first measured pain with the BOS-3. Then, an external expert from the Pain Unit, blinded to the first result, independently scored the BOS-3. This external expert did not know the patient. The aim was to examine the reproducibility of the BOS-3 when conducted by two independent observers. (2) Then, less than one minute later, the evaluation was repeated by internal caregivers either with the NRS for communicating patients or with the ECPA2 for non-communicating patients. The aim was to examine how the BOS-3 could help discriminate amongst patients in pain and those not in pain, as compared to the gold standards. The caregivers involved in the study were trained to use a behaviour scale approach and were regularly using the ECPA. We purposefully decided not to train them to use the BOS-3 in order to better estimate its reproducibility. However, they could use the item definitions provided in the BOS-3 testing sheet. In addition, age, gender, admission date and available information related to pain (aetiology, localization, analgesic treatment) were collected for each patient.

2.4 Data analysis Data from communicating and non-communicating patients were analysed separately. Feasibility and acceptability of the scale were evaluated through the completion time of the BOS-3, the item comprehension and the level of completion of each item. Spearman correlations between the BOS-3 scores and the NRS or ECPA2 scores were estimated. A correlation above 0.70 was considered as high. Diagnostic performances were evaluated calculating the sensitivity, specificity, predictive values and likelihood ratios of the BOS-3 performed by internal caregivers and compared to the gold standards. PostBOS-3 probability of pain was estimate using the calculated positive likelihood ratio and a literature based estimation of pain prevalence at 30%. Gold standards’ thresholds were established according to literature and were set as a score ≥3 for the NRS or score ≥8 for the ECPA2 (Wary et al., 4 Eur J Pain (2017) –

M. Frasca et al.

2001). A ROC analysis was performed to determine the BOS-3 cut-off threshold that would maximize both its sensitivity and specificity. The content validity was examined through a principal component analysis (PCA) with the five BOS-3 items, with items considered as relevant if their factorial weight was above 0.40. The internal consistency was estimated with Cronbach alpha coefficients, removing items one at a time (a value above 0.70 corresponded to a good internal consistency, that is, the informative content of the item was deemed to be satisfying). Finally, the inter-observer reproducibility was estimated through an intra-class coefficient (ICC) (a value above 0.70 was taken for an excellent reproducibility). We calculated 95% CIs for proportions using the normal approximation formula if effectives were sufficient otherwise the exact binomial formula.

3. Results Nine surgery units were involved, together with 17 medicine units, three intensive care units and one emergency unit. Overall, 533 patients took part in the study. Eighty-six patients(16%) were excluded (50 were not in their room at the time of the evaluation; 18 refused to take part; 11 were in a coma; six were aged under 15 and one was deceased). Finally, 447 patients were included: 52 (12%) were categorized as non-communicating and 395 patients (88%) as communicating (Table 2). Characteristics of the population are described in Table 3. Out of the 52 non-communicating patients, 54.9% were men, 50.0% were 35–74 years-old and 71.2% had been admitted less than 1 month. Delirium and aphasia were responsible for communication’s disabilities in 42 of the non-communicating patients (81.1%). Out of the 395 communicating

Table 2 Patients’ distribution as a function of their ability to communicate and of the type of departments participating.

Surgery Medicine Intensive care Emergency Total

Communicating patients

Non-communicating patients

N

%

N

%

Total

132 229 32 2 395

93.0 87.4 91.4 25.0 88.4

10 33 3 6 52

7.0 12.6 8.6 75.0 11.6

142 262 35 8 447


M. Frasca et al.

Validation study

Table 3 Description of the communicating and non-communicating groups’ characteristics for the BOS-3 diagnosis performances validation study.

Characteristics Socio-demographic data Men Age (years) 15–34 35–54 55–74 75 and above Duration of stay Less than 48 h 48 h to 1 week 1 week to 1 month More than a month Analgesics No analgesic prescription Non-opioids only At least one weak opioid At least one strong opioid Pain aetiology Post-surgery Acute evolutionary disease Cancer Central (VCA, medullary injury. . .) Post-trauma Peripheral (diabetes, avulsion. . .) Headache and orofacial pain Muscular and articulatory diseases Arteritis and other vascular ailments Inflammatory polyarthropathy Skin-related pain (slough) None Pain localization Leg, arm Thorax Abdomen, pelvis, sacred area Face, head, neck Back Unstable Global None Communication disabilities’ causes Delirium Aphasia Delirium + aphasia Non French-speaking people




N = 395

%

N = 52

%

215

54.3

29

54.9

24 84 152 135

6.0 21.2 38.5 34.1

3 13 13 23

5.8 25.0 25.0 44.2

116 116 118 45

29.4 29.4 29.8 11.1

8 11 18 15

15.4 21.2 34.6 28.8

164 94 76 61

41.5 23.8 19.2 15.5

19 17 7 9

36.5 32.6 13.5 17.4

121 56 42 22

30.6 14.2 10.6 5.6

6 5 4 9

11.5 9.7 7.7 17.3

19 16

4.8 4.1

5 3

9.7 5.7

15 14

3.8 3.5

0 1

0.0 1.9

5

1.3

1

1.9

5

1.3

0

0.0

4 76

1.0 19.2

1 17

1.9 32.7

103 46 46 56 41 1 15 87

26.1 11.6 11.6 14.3 10.4 0.2 3.8 22.0

12 1 4 9 6 0 4 16

23.1 2.0 7.7 17.3 11.5 0.0 7.7 30.7

27 10 5 5

51.9 19.2 9.6 9.2

Table 3 (Continued)

Characteristics



N = 395

N = 52

%

Severe psychological troubles Delirium + severe psychological troubles

%

4 1

7.7 2.0

patients, 54.3% were men, 59.7% were 35– 74 years-old and 88.6% had been admitted less than 1 month. Concerning analgesic treatment, two-thirds of non-communicating patients were given an analgesic prescription included only non-opioids drugs in 32.6% of cases, at least one weak opioid in 13.5% and at least a strong opioid in 17.4%. For communicating patients, proportions were similar: 58.5% were given an analgesic prescription included only non-opioids drugs in 32.6% of cases, at least one weak opioid in 13.5% and at least a strong opioid in 17.4%. Pain localization and its potential aetiology were studied. In the non-communicating patients group, central nervous system disorders were the most frequent cause of pain. This was followed in equal proportions by surgery, acute diseases, cancer, traumas and peripheral nervous system disorders. In communicating patients group, more than half of pains were due to surgery and other acute diseases. About one quarter corresponded to cancer and nervous system disorders pains. During data collection, all BOS-3 and reference tests were completed and all BOS-3 evaluations were carried out in less than one minute. All of the five items appeared to be clear and comprehensible with the simple use of the definitions provided within the scale. The additional item ‘Aggressiveness or Mutism’ was clearly understood and accepted by all professionals. Items were completed at a level ranging from 99.6% to 99.9%. The ‘Cautious Movements’ item was the only one with four missing data on 52. For the 52 non-communicating patients, mean scores provided by internal caregivers averaged at 1.7/10 (SD: 2.1; median: 1; interquartile intervals [0;3]) and at 1.8/10 when provided by external experiments. Score distribution as a function of experimenter evaluation (internal of external) is provided in Table 4.

Eur J Pain

(2017)

–

5

Validation study

M. Frasca et al.

Table 4 Distribution of BOS-3 items’ scores for internal caregivers and external experts in each group of patients. For communicating patients (n = 395)

From internal caregivers Moans Face Analgesic postures Cautious movements Agressiveness/Prostration From external experts Moans Face Analgesic postures Cautious movements Agressiveness/Prostration

For non-communicating patients (n = 52)

MD

Absent N (%)

Weak N (%)

Noticeable N (%)

MD

Absent N (%)

Weak N (%)

Noticeable N (%)

0 0 0 2 0

376 306 321 304 385

(95.2) (77.5) (81.3) (77.4) (97.5)

11 66 58 67 7

(2.8) (16.7) (14.7) (17.0) (1.8)

8 23 16 22 3

(2.0) (5.8) (4.1) (5.6) (0.8)

0 0 1 1 0

43 30 41 39 39

(82.7) (57.7) (80.4) (76.5) (75.0)

5 15 6 7 10

(9.6) (28.8) (11.8) (13.7) (19.2)

4 7 4 5 3

(7.7) (13.5) (7.8) (9.8) (5.8)

1 1 1 1 1

376 289 342 308 382

(95.4) (73.4) (86.8) (78.2) (97.0)

10 86 43 62 10

(2.5) (21.8) (10.9) (15.7) (2.5)

8 19 9 24 2

(2.0) (4.8) (2.3) (6.1) (0.5)

0 0 0 0 0

44 35 42 34 39

(84.6) (67.3) (80.8) (65.4) (75.0)

4 11 7 10 6

(7.7) (21.2) (13.5) (19.2) (11.5)

4 6 3 8 7

(7.7) (11.5) (5.8) (15.4) (13.5)

MD, missing data.

As assessed with the ECPA, 15 patients (28.8%) were categorized as being in pain at the therapeutic threshold of 8; scores varied between 0 and 19, with a mean of 5. Correlation between the BOS-3 and the ECPA was at 0.82. The Fig. 1 shows the ROC curve of BOS-3 compared with ECPA2 scale for the noncommunicating patients. Diagnosis performances reached a maximum for a BOS-3 pain threshold of 3 with a sensitivity at 0.87 [95% CI 0.77–0.96] and a specificity at 0.97 [95% CI 0.93–1.00]. At this threshold, 14 patients on 52 were diagnosed as suffering from pain (26.9%). The positive predictive value was at 0.93 [95% CI 0.86–0.99] and the negative predictive value at 0.95 [95% CI 0.89–1.00]. The positive likelihood ratio was at 29 and the

1

negative likelihood ratio at 0.13. The post-test probability of pain reached 0.93. Factorial weights for the five items were above 0.40 on the first PCA factor (Table 5). Cronbach alpha was at 0.70, with an equal contribution from all items (Table 6). Finally, the ICC comparing scores from internal caregivers and external experimenters was at 0.93 [95% CI 0.89–0.97]. This latter result suggests an excellent reproducibility of the use of the BOS-3 with non-communicating patients. For the 395 communicating patients, BOS-3 scores provided by the internal caregivers from the different units were averaging at 0.9/10 (SD: 1.5; median: 0; interquartile intervals [0;1]); external experimenters’ evaluations were on average at 0.8/10 as shown in Table 4. Scores’ distribution as a function of experimenters’ evaluations (internal of external) is provided in Table 4. The mean score for the NRS was 1.6/10 (n = 393), with 106 patients (27%) being categorized as in pain at the therapeutic threshold of 3.

Sensibility

0.8

Table 5 Principal component analysis on BOS-3 scores in each group of patients.

0.6

0.4

0.2

0

0

0.2

0.4

0.6

0.8

1

1 - specificity Figure 1 ROC curve of BOS compared with ECPA2 scale for the 52 non-communicating patients.

6 Eur J Pain (2017) –

Moans Face Analgesic postures Cautious movements Aggressiveness/prostration Eigen value Cumulated variance (%)



F1a

F2a

F1a

F2a

0.56 0.74 0.72 0.76 0.25 2.0 40.3

0.11 0.25 0.29 0.18 0.92 1.0 61.1

0.77 0.81 0.53 0.81 0.55 2.5 49.6

0.34 0.31 0.65 0.22 0.62 1.1 70.8

a

F1 and F2: main components.


M. Frasca et al.

Validation study

Table 6 Cronbach alpha coefficients of BOS-3 before and after removal of each item in each group of patients.

Globala Without ‘Moans’ Without ‘Face’ Without ‘Analgesic Postures’ Without ‘Cautious Movements’ Without ‘Agressiveness/ Prostration’


Non communicating patients

Nb of items

Cronbach alpha

Nb of items

Cronbach alpha

5 4 4 4

0.67 0.65 0.56 0.60

5 4 4 4

0.70 0.62 0.58 0.71

4

0.56

4

0.62

4

0.70

4

0.69

4. Discussion

a

Before removal.

The Fig. 2 shows the ROC curve of BOS-3 compared with NRS for the communicating patients. Sensitivity and specificity reached a maximum at ROC analysis for a BOS-3 pain threshold of 1 (sensitivity: 0.71 [95% CI 0.66–0.75]; specificity: 0.73 [95% CI 0.68–0.77]) (Fig. 2). At the threshold of 3 used for non-communicating patients, the sensitivity was non-informative at 0.33 [95% CI 0.28–0.38] but the specificity reached 0.96 [95% CI 0.94–0.98]. At this threshold, the positive predictive value was found at 0.75 [95% CI 0.70–0.79] and the negative predictive value at 0.80 [95% CI 0.76–0.83]. The positive likelihood ratio was at 8.25, and the negative likelihood ratio at 0.80. The post-test probability of pain reached 0.78. 1

Sensibility

0.8

0.6

0.4

0.2

0

0

0.2

0.4

0.6

0.8

1

1 - specificity Figure 2 ROC curve of BOS compared with Numeric Rating Scale with the 395 non-communicating patients.


According to the PCA results (Table 5), loadings were higher than 0.40 for four items in the first factor. The only item to reach a low factorial weight of 0.25 was ‘Aggressiveness or Mutism’ with the communicating patients’ group. Cronbach alpha was at 0.67, and reached 0.70 without the ‘Aggressiveness or Mutism’ item (Table 6). The correlation between scores from the NRS and the BOS-3 was at 0.50. The ICC between the caregivers and the external experimenters was 0.77 [95% CI 0.73–0.81].

The high level of items completion showed an excellent ease of use of the BOS-3. Each item was well understood, easy to remember and ecologically valid. Completion time of the whole scale was fast, less than one minute. Using the BOS-3 didn’t require any prior training or knowledge of the patient. The scores collected were no significantly different between the trained external evaluators and the untrained internal caregivers. BOS-3 displayed an excellent inter-observer reliability. The analysis of pain features revealed that this tool is usable regardless of the type of pain the patient is in. In the non-communicative patients group, comparative analysis of the evaluations with the BOS3 and the ECPA2 revealed a good correlation between two tools, allowing to determine a pain threshold for the BOS-3 at 3 for a maximum score of 10. The BOS-3 was characterized by good metrological properties with high sensitivity and specificity, good predictive values and an interesting post-test probability of pain. These good performances with noncommunicative patients were probably due to the fact that 5 BOS-3 items are very well correlated to the six categories of pain-related signs and symptoms described by the AGS. Over the past 10 years many different items of behavioural scales were studied (de Knegt et al., 2013). Reliability of items related to the six AGS items has been estimated with patients having cognitive impairments (Lukas et al., 2013). In 2012, Lints-Martindale et al. reported that, even when non-communicating patients were not confused, measurements related to the six AGS items were still useful in distinguishing between patients in pain and not in pain (Lints-Martindale et al., 2012).This corroborates diagnostic performances of BOS-3 items. Eur J Pain

(2017)

–

7

Validation study

BOS-3 sensitivity was comparable to that of Algoplus scale used for elderly people but specificity was higher (Rat et al., 2011). With equal prevalence of pain, positive predictive value would be better using BOS-3 with non-communicating adult patients than using Algoplus with non-communicating elderly patients. This difference may be due to facial expressions evaluation. This item is the most sensitive and specific item for pain evaluation out of the AGS dimensions (Sheu et al., 2011). But, it might be responsible for leading to more false positive and lower specificity with elderly people because of a potential risk of overrating painless patients presenting for example anxiety or depression (Schuler et al., 2004). The analysis of BOS-3 internal structure with noncommunicating patients showed a satisfying internal consistency (Cronbach alpha at 0.70) as well as a good content validity. Indeed all items were correlated with one another and were informative for the evaluation of pain in non-communicating adults, including the one added by our research group: aggressiveness/prostration. In the communicating patients group, comparative analysis of BOS-3 and NRS revealed a good correlation between two tools and a good reproducibility of the BOS-3. But, two points differed from non-communicating group. First, the threshold at 1/10 with the best sensitivity-specificity couple at ROC analysis appeared not relevant for clinical practice. However, at the threshold of 3/10, specificity and positive predictive value became interesting. This result suggests that BOS-3 could help to confirm pain with patients who would have difficulties to express it with an auto-evaluation scale like the NRS. This is particularly true for patients with chronic pain. Failure rate of selfreported scales is thus more important when measures are repeated with up to 4% with NRS and 14.3% with Visual Analogic Scale after seven uses even with young adult patients (Herr et al., 2004). Complaint is often under-estimated, due to fatigue, lack of concentration, fear of underlying disease or potential treatments implied. Standardized observational behavioural procedures are then recognized as a good complement of self-report pain intensity in these situations (McCahon et al., 2005; Dansie and Turk, 2013). Performing a BOS-3 evaluation as an additional scale could help caregivers dealing with communicating adults in chronic pain whatever their age. A BOS-3 at 3/10 or more would confirm the presence of a physical pain, inciting to discuss more carefully with the patient about the interest of 8 Eur J Pain (2017) –

M. Frasca et al.

refining therapeutic options. However, at this threshold, the decrease of sensitivity prevents from interpreting a negative result suggesting that BOS-3 cannot be utilized to exclude pain in communicative patients. Second, the additional ‘Aggressiveness or Mutism’ item was poorly correlated (factorial weight under 0.40) to the four others and not highly informative (leading to a reduced Cronbach alpha). The nonrelevance of this item might be explained by the fact that this behaviour is less usual when patients are able to communicate and explain their suffering. However, if this item did not contribute to enhancing the BOS-3’s pain detection power, nor did it contribute to worsen it: sensitivity and specificity remained the same using only the first four items of the scale. Thanks to the large number of patients included, our study kept a good discriminative power. Only 13% of patients were non-communicating, which is less than the 20% expected from the French General Meetings for Pain (Patrice, 2005; Boerlage et al., 2013). This discordance might be mainly due to the sedated patients exclusion in eligibility criteria. A misinterpretation by experimenters of the patient ability to communicate is also possible but unlikely due to the double evaluation by internal caregiver and external expert. However, this smaller than expected sample did not prevent us from obtaining interpretable confidence intervals in the two groups. Our sample was representative of hospitalized adult patients’ population as far as age, gender, unit and duration of hospitalization are concerned. The little proportion of missing data and the random nature of exclusion causes enhance its representativeness. Pain’s prevalence in both groups approached 30% (28.8% of non-communicative patients and 27% of communicative patients) which is in line with data of French hospitalized population (Bouhassira et al., 2008). The study design allowed us to evaluate in a same study the feasibility, reproducibility and validity of BOS-3. Regarding reproducibility, internal caregivers involved in this study were representative of experimenters population likely to use BOS-3 in hospital. Furthermore, blinded evaluations between internal caregivers and external expert insured independent results. Concerning diagnostic performances, all patients were evaluated with index and reference tests, avoiding verification bias. Considering its excellent feasibility and reproducibility, without any training required before evaluation, BOS-3 appears to be a promising tool to © 2017 European Pain Federation - EFICâ

M. Frasca et al.

Validation study

Table 7 Comparison of behavioural scales used for pain assessment in adult patients. BOS

ALGOPLUS

DOLOPLUS

ECPA

BPS

Pain characteristics

Acute, chronic, induced

Acute, induced

Chronic

Acute, induced

Population used for validation

Adult over 15 years old with and without communication disabilities 1 min No French Medical and surgical departments for adults

Elderly with communication disabilities

Elderly with communication disabilities 5 min Yes French Geriatric departments

Acute, chronic, induced Elderly with communication disabilities 5 min Yes French Geriatric departments

Time to perform Form training Language Designed for

1 min No Frencha Geriatric departments

Sedated, intubated patients 1 min No French Intensive care departments

a

Validation in English, German, Italian, Spanish and Portuguese in progress.

optimize the screening, follow-up and tracking of pain. Compared to other behavioural scales used for adults, BOS-3 is more feasible than long scales as the ECPA2 and validated in non-geriatric and nonsedated adult population contrary to Algoplus and BPS, respectively (Table 7). Thus, BOS-3 should be useful for several adult departments’ teams not specifically specialized in geriatrics or intensive care. Our study had some limitations. First, in the noncommunicating group, the use of a reference test that itself had an uncertain threshold could possibly alter the estimation of BOS-3 diagnostic performances in this group. However, the choice of a pain threshold at 8 for the ECPA had been made according to its metrological properties (Morello et al., 2007) and with its creators advices. Furthermore, similar pain prevalence between our sample and the one observed in literature for this population reinforces this choice. Additionally, evaluation of BOS-3 before reference scales and by the same experimenters could have enhanced correlation between the two tests, especially with non-communicating patients. However, the absence of a predefined BOS-3 threshold prevented experimenters from interpreting it as positive or negative, thus probably limiting these potential biases. Some categories of pain aetiology were poorly represented in our patient sample, as pain induced by cancer or by pressure sore. This study also failed to report whether the pain under evaluation was acute, induced or chronic. Our results are applicable to the population of our study, i.e. adults hospitalized in medical or surgical departments of a French university hospital. The high prevalence of pain in this population may explain the good positive predictive value of BOS-3 in this study. Our findings should be confirmed in different samples before broadening the use of this scale in other populations. © 2017 European Pain Federation - EFICâ

The BOS-3 remains an one-dimensional scale as it doesn’t offer the possibility to estimate the level of pain. Further studies are needed to determine a possible correlation between BOS-3 scores and pain intensity.

5. Conclusions In summary, our findings shows that BOS-3 allows a quick evaluation of pain with a positivity threshold set at 3/10. This preliminary study confirms the diagnostic performances of this scale in a population of hospitalized non-sedated adults. In non-communicating patients, BOS-3 could be used as a first-intention scale, as it has good metrological properties. In communicating patients, its good specificity places BOS-3 as a second intention scale and suggests that caregivers could perform it to confirm the presence of pain in case of NRS-underrating suspicion. A shorter BOS-3 with only the first four items could be used in this indication. BOS-3 is an easy-to-use tool without any training required before evaluation and allows a good detection of pain in a wide hospitalized adult population. It will be thus particularly useful for all non-geriatric adult departments. Further diagnostic studies should be conducted to evaluate diagnostic performances of BOS-3 in other populations, particularly with nonFrench-speaking people. Randomized trials could be also performed to assess how BOS-3 can modify patient’s cares. Acknowledgements Tribute to Dr. Francßois Boureau for the conception of the scale and thanks to his team. Thanks to UPSA institute of the pain for support, Dr. Caroline Floccia, Louise Freeman and Lauran Brown for translation, to National Center of Resources for Palliative care for the documentation and to

Eur J Pain

(2017)

–

9

Validation study

all Committee members of study of pain – University Hospital of Bordeaux.

Author contributions All authors had substantial contributions to the analysis and the interpretation of data. All of them revised the article critically for important intellectual content and approved the version to be published. B.BURUCOA, V.DOUSSET, S.DOMECQ and N.ROBINSON particularly contributed to the conception and the acquisition of data and M.FRASCA to draft the article.

References AGS (2002). The management of persistent pain in older persons. J Am Geriatr Soc 50, S205–S224. Aissaoui, Y., Zeggwagh, A.A., Zekraoui, A., Abidi, K., Abouqal, R. (2005). Validation of a behavioral pain scale in critically ill, sedated, and mechanically ventilated patients. Anesth Analg 101, 1470–1476. ANAES (2000). Evaluation et prise en charge de la douleur chez les personnes ^ ag ees ayant des troubles de la communication verbale. Aubin, M., Giguere, A., Hadjistavropoulos, T., Verreault, R. (2007). The systematic evaluation of instruments designed to assess pain in persons with limited ability to communicate. Pain Res Manag 12, 195–203. Boerlage, A.A., Valkenburg, A.J., Scherder, E.J., Steenhof, G., Effing, P., Tibboel, D., van Dijk, M. (2013). Prevalence of pain in institutionalized adults with intellectual disabilities: A cross-sectional approach. Res Dev Disabil 34, 2399–2406. Bouhassira, D., Lanteri-Minet, M., Attal, N., Laurent, B., Touboul, C. (2008). Prevalence of chronic pain with neuropathic characteristics in the general population. Pain 136, 380–387. Boureau, F., Luu, M., Hericord, P. (1993). Evaluation de l’intensite de la douleur au service d’accueil et des urgences. Interêt des echelles d’ evaluation. [Pain intensity assessment at emergency departments. Pain scales interest.]. Rean Urg 2, 331–335. Brahimi, L., Wood, C., Benkerrou, M., Vieyra, M., Bourillon, A., Vilmer, E. (2000). Etude prospective de l’utilisation de deux outils d’ evaluation de la douleur l’echelle visuelle analogique (EVA) et l’ echelle des visages (FPS) chez l’enfant drepanocytaire d’âge scolaire. Extrait des actes du colloques UNESCO 2000 “La douleur de l’enfant, quelles r eponses ?” Paris. Chatelle, C., Vanhaudenhuyse, A., Mergam, A.N., De Val, M., Majerus, S. et al. (2008). Pain assessment in non-communicative patients. Rev Med Liege 63, 429–437. Dansie, E.J., Turk, D.C. (2013). Assessment of patients with chronic pain. Br J Anaesth 111, 19–25. De Tovar, C., Alibeu, J.P., Arvieux, C.C. (2002). Auto-evaluation de la douleur post-operatoire chez l’enfant. Echelle Visuelle Analogique (EVA) versus Faces Pain Scale Revised (FPS-R). SFETD congress. Gomas, J.M., Petrognani, A., Cudennec, T., Jousselin, C., Leroux, F., Delporte, J., Knoreck, F., Le Dastumer, B. (2004). Specificites des traitements de la douleur chez le sujet âge. Centre National de Ressources de lutte contre la douleur.

10 Eur J Pain (2017) –

M. Frasca et al.

Herr, K.A., Spratt, K., Mobily, P.R., Richardson, G. (2004). Pain intensity assessment in older adults: Use of experimental pain to compare psychometric properties and usability of selected pain scales with younger adults. Clin J Pain 20, 207–219. Herr, K., Bjoro, K., Decker, S. (2006a). Tools for assessment of pain in nonverbal older adults with dementia: A state-of-the-science review. J Pain Symptom Manage 31, 170–192. Herr, K., Coyne, P.J., Key, T., Manworren, R., McCaffery, M., Merkel, S., Pelosi-Kelly, J., Wild, L., American Society for Pain Management, N. (2006b). Pain assessment in the nonverbal patient: Position statement with clinical practice recommendations. Pain Manag Nurs 7, 44–52. de Knegt, N.C., Pieper, M.J., Lobbezoo, F., Schuengel, C., Evenhuis, H.M., Passchier, J., Scherder, E.J. (2013). Behavioral pain indicators in people with intellectual disabilities: A systematic review. J Pain 14, 885–896. Le Quintrec, J.L., Maga, M., Baulon, A. (1995). L’echelle comportementale simplifiee (ECS), un nouvel outil d’evaluation de la douleur en long sejour geriatrique. Revue Geriatr 20, 363–368. Lints-Martindale, A.C., Hadjistavropoulos, T., Lix, L.M., Thorpe, L. (2012). A comparative investigation of observational pain assessment tools for older adults with dementia. Clin J Pain 28, 226–237. Liu, J.Y., Briggs, M., Closs, S.J. (2010). The psychometric qualities of four observational pain tools (OPTs) for the assessment of pain in elderly people with osteoarthritic pain. J Pain Symptom Manage 40, 582–598. Lukas, A., Barber, J.B., Johnson, P., Gibson, S.J. (2013). Observer-rated pain assessment instruments improve both the detection of pain and the evaluation of pain intensity in people with dementia. Eur J Pain 17, 1558–1568. McCahon, S., Strong, J., Sharry, R., Cramond, T. (2005). Self-report and pain behavior among patients with chronic pain. Clin J Pain 21, 223–231. McNair, C., Ballantyne, M., Dionne, K., Stephens, D., Stevens, B. (2004). Postoperative pain assessment in the neonatal intensive care unit. Arch Dis Child Fetal Neonatal Ed 89, F537–F541. Morello, R., Jean, A., Alix, M., Sellin-Peres, D., Fermanian, J. (2007). A scale to measure pain in non-verbally communicating older patients: The EPCA-2 Study of its psychometric properties. Pain 133, 87–98. Patrice, Q. (2005). Le Livre Blanc de la Douleur. Les Etats Generaux de la Douleur, 141. Pautex, S., Herrmann, F., Le Lous, P., Fabjan, M., Michel, J.P., Gold, G. (2005). Feasibility and reliability of four pain self-assessment scales and correlation with an observational rating scale in hospitalized elderly demented patients. J Gerontol A Biol Sci Med Sci 60, 524–529. Rat, P., Jouve, E., Pickering, G., Donnarel, L., Nguyen, L. et al. (2011). Validation of an acute pain-behavior scale for older persons with inability to communicate verbally: Algoplus. Eur J Pain 15, 198 e191– 198 e110. Schuler, M., Njoo, N., Hestermann, M., Oster, P., Hauer, K. (2004). Acute and chronic pain in geriatrics: Clinical characteristics of pain and the influence of cognition. Pain Med 5, 253–262. Sheu, E., Versloot, J., Nader, R., Kerr, D., Craig, K.D. (2011). Pain in the elderly: Validity of facial expression components of observational measures. Clin J Pain 27, 593–601. Wary, B., Serbouti, S., Doloplus, C. (2001). Validation d’une echelle d’evaluation comportementale de la douleur chez la personne âgee. Douleurs 2, 35–38. Zwakhalen, S.M., Hamers, J.P., Abu-Saad, H.H., Berger, M.P. (2006). Pain in elderly people with severe dementia: A systematic review of behavioural pain assessment tools. BMC Geriatr 6, 3.
