ORIGINAL ARTICLE
Cross-validation of the Children’s and Infants’ Postoperative Pain Scale in Brazilian Children Marta M. O. Alves, MD; Paulo R. A. Carvalho, PhD, MD; Mario B. Wagner, PhD; Alan Castoldi, MD; Michele M. Becker, MD; Cláudia C. Silva, RN Departamento de Pediatria, Universidade Federal do Rio Grande do Sul, Hospital de Clínicas de Porto Alegre, Porto Alegre, RS, Brazil.
䊏 Abstract Objective: To validate CHIPPS (Children’s and Infants’ Postoperative Pain Scale) in Brazilian children. Background: Cross-validation is needed in order to apply this scale in a different language and culture. Methods: We applied a Portuguese version of CHIPPS to 100 children aged 0 to 5 years. The scale was translated and tested for inter-rater reliability, internal consistency, and construct, content, and concurrent validity. The children’s behavior was videotaped before, during and after a procedure in a primary care unit. Three observers then rated pain behavior from videotapes. Results: The scale showed excellent inter-rater reliability (intraclass correlation coefficient: 0.89) and a very good internal consistency, with Cronbach’s alpha of 0.86. The positive correlation between CHIPPS and Modified Behavior Pain Scale supports concurrent (criterion) validity (Spearman coefficient 0.70 before and 0.81 after vaccinations). Construct validity was determined by comparing the scores of each child before vaccination (without pain) and during the procedure (experiencing pain), and the difference in pain scores was statistically significant (Wilcoxon signed rank test; P < 0.001). Content validity (by expert review) was very good. Address correspondence and reprint request to: Marta M O Alves, MD, Rua Santana 1357 # 501, CEP 90040–373, Porto Alegre, RS, Brazil. E-mail:
[email protected]. Submitted: September 20, 2007; Revision accepted: January 8, 2008
© 2008 World Institute of Pain, 1530-7085/08/$15.00 Pain Practice, Volume 8, Issue 3, 2008 171–176
Conclusions: Based on the results obtained, we can infer that CHIPPS is a valid and reliable tool for Brazilian children aged 0 to 5 years old. 䊏 Key Words: pain measurement, observational assessment, infants, toddlers, validation studies
pain
INTRODUCTION Pain response is affected by several psychological factors, including cultural differences, observational learning, cognitive appraisal, and coping style. Quantification is central to the investigation of pain and analgesia, and pain measurement is essential in clinical practice.1 Routine pain assessment has been shown to improve pain management in children.2 But such assessment is a challenge for health professionals who work with children, especially those in preverbal or nonverbal stages of development. Behavioral and physiological indicators have been used to measure pain in this population. Physiological parameters are convenient and show rapid response to brief nociceptive stimuli, but their utility is diminished by homeostatic mechanisms.1 Furthermore, they are only loosely correlated with painful events and may occur in response to many other states such as exertion or fever.3 Behavioral parameters such as crying, facial expression, body movements, sleep patterns, and consolability have also been used for pain assessment. To
172 • alves et al.
date, many scales with behavioral parameters have been developed, but an acceptable tool must be valid, reliable, have clinical utility, and be feasible to use.4 Furthermore, it must be short, easy to memorize, and generalizable to different settings. After a literature review, we chose the Children’s and Infants’ Postoperative Pain Scale (CHIPPS) because of its very good and well-recognized psychometric properties. Children’s and Infants’ Postoperative Pain Scale is a scale developed by Büttner and Finke.5 They studied 582 newborns and infants (4,238 observations) in a sequential prospective process of studies. Starting with 26 items suggested by literature, those authors obtained a 5-item scale through a 3-stage analysis. First, a factor analysis resulted in a two-factorial solution with the behavioral items loading on one factor and the physiological parameters on the other. Next, they studied the variability of those items under postoperative conditions, identified the dimensions of behavioral items and physiological parameters, evaluated their selectivity, internal consistency, inter-rater reliability and construct, and concurrent validity. Finally, they calculated sensitivity and specificity of the positively identified item-sets. The resulting scale (CHIPPS) was comprised by the following reliable, sensitive, and specific items: crying, facial expression, posture of the trunk, posture of the legs, motor restlessness. Each of such items had a value from 0 to 2 points; the range of the total score spanned from 0 (no pain) to 10 (maximal pain). Cronbach’s alpha for internal consistency was 0.92 for toddlers and 0.96 for infants. The inter-rater reliability coefficient was 0.93. The scale was validated constructively through intravenous administration of analgesics and equally identified pain-free situation and analgesic demand in 87.4% of instances. The ranges observed for sensitivity and specificity were calculated to be 92% to 96% and 74% to 95%, respectively. The narrow range of sensitivity values indicated that the system could be relied upon in the case of analgesic demand. The wider specificity range suggested that a score of 3 points or greater might have been induced either by pain or by other influences on the child’s comfort. However, it was guaranteed that a child who did not suffer pain would have a CHIPPS score lower than 4 points. Moreover, the tool proved to be practical as it could be completed within 15 seconds. As CHIPPS was originally published in English, and considering that cross-validation is needed in order to apply such instruments in different cultures,6 it was necessary to validate it for Brazilian children. According
to Streiner and Norman,7 translating a scale requires the translation itself, back translation, and then re-establishing the reliability and validity within the new context; in essence, exactly the same steps that are required for developing a new scale. That can be viewed as a cross-validation procedure. The purpose of this study was to validate CHIPPS in Brazilian children, addressing both reliability and validity issues.
METHODS We studied 100 children, aged 0 to 5 years, at the moment they received immunization injections at a primary care unit in Porto Alegre, Brazil. Exclusion criteria were any development delay, chronic disease or having received any analgesic or anesthetic drug shortly before the injection. After the study protocol had been approved by the ethics committees of Hospital de Clínicas de Porto Alegre and the Municipality Health Department, at least one parent of each subject signed an informed consent document. The children were observed immediately before, during, and after the vaccination procedure. Vaccination procedures were carried out according to routine by the primary care staff, without the interference of the investigators. No pain-relieving methods were used. Behaviors were recorded by a digital video camera (Sony DCR-DVD92) from 15 seconds before through 30 seconds after vaccination, under constant conditions. Each of three observers––a pain expert anesthetist, a pediatric nurse, and a child neurologist—received a copy of the videotapes and rated them independently, after a brief training. Their scores were recorded on individual data sheets by ticking the corresponding behavior for each of the 5 items of a Portuguese version of CHIPPS. Children’s and Infants’ Postoperative Pain Scale was translated from English to Portuguese by a Brazilian bilingual translator and then translated back to English by a different Portuguese-speaking bilingual translator. Finally, the back-translated scale was rechecked with the original scale by a third bilingual translator whose native language was English, to ensure that the scale items retained their original meaning. Besides the translation process, cross-cultural validation was obtained through reliability and validity testing. Reliability, ie, the ability of the instrument to produce similar results when used by different individuals or at different times by the same individual, was assessed through inter-rater and internal consistency measures. For inter-rater reliability, the agreement between the three referred raters’ scores was assessed.
Cross-validation of the CHIPPS • 173
For internal consistency, the observers were asked to indicate each item of the scale that corresponded to the behavior observed. Validity, ie, the ability to establish the extent to which an instrument actually measures what it intends to measure, was evaluated according to its content, construct, and concurrent aspects. For content validity, the scale was tested, on the basis of content relevance, coverage, and scaling by the pediatric committee of the Brazilian Association for the Study of Pain (SBED), affiliated to the International Association for the Study of Pain. For the construct validity, we compared the CHIPPS scores before and after vaccination. For concurrent validity, an anesthetist (pain expert) scored CHIPPS and the Modified Behavioral Pain Scale (MBPS) for all 100 children. MBPS is the Behavioral Pain Scale (BPS) modified for use in infants by Anna Taddio and colleagues.8 The scale has 12 items divided in three categories of behavior: verbal expression, cry, and body movements. Zero to three points are attributed to each item with a total score of 0 to 10. Its psychometric properties were also tested in children receiving routine immunizations, and post procedural pain was assessed by videotape.
Table 2. The excellent ICC of the 3 raters’ assessments supported the inter-rater reliability, showing high agreement between the scores. Cronbach’s alpha of 0.86 for the 5 scale items indicated the very good internal consistency of CHIPPS. Table 3 summarizes the results of the validation tests. The expert committee considered the content of the scale adequate, confirming content validity. Construct validity was demonstrated by the increase of CHIPPS scores with the administration of vaccines (baseline CHIPPS scores were significantly lower than postvaccination scores). The correlation between CHIPPS and MBPS scores before vaccination is shown in Figure 1. Concurrent validity (criterion validity) was demonstrated by the positive correlation between postvaccination CHIPPS scores and postvaccination MBPS scores, while baseline
Table 1. Portuguese version of Children’s and Infants’ Postoperative Pain Scale Item Choro
Expressão facial
Statistical Analysis We estimated the sample size to be 100 subjects, considering a significance level of 5%, and allowing for a maximum 10% error margin for the correlation coefficients involved in the validation process. We used anova two-way random effects model with average consistency to find the intraclass correlation coefficient (ICC) for the analysis of inter-rater reliability. An ICC = 0.75 is considered to be excellent, between 0.75 and 0.40 as moderate and below 0.40 as weak.9 The internal consistency was calculated through Cronbach’s alpha (average of two raters, 5 items); Cronbach’s alpha between 0.70 and 0.90 was indicative of good reliability.7 To examine construct validity, scores before vaccination were compared with those after the procedure using Wilcoxon signed rank test. Spearman’s coefficient was used to determine the strength of association between scores obtained using CHIPPS and MBPS by the anesthetist, supporting the concurrent validity. All analyses were performed with SPSS for Windows version 14.0.
RESULTS The Portuguese version of CHIPPS is provided in Table 1. Demographic characteristics are detailed in
Postura do tronco
Postura das pernas
Inquietação motora
Estrutura
Pontos
nenhum gemido grito relaxada/sorrindo boca retorcida careta (olhos e boca) neutra variável arqueado para trás neutra, solta chutando pernas tensionadas nenhuma moderada inquieta
0 1 2 0 1 2 0 1 2 0 1 2 0 1 2
Table 2. Children’s Characteristics and Type of Vaccine (n = 100) Characteristic Age in months, media (range) Sex Girls Boys Gestational age Term Preterm Type of vaccine DTP-Hib* Hepatitis B MMR† BCG‡ Rabies * Diphteria, tetanus and pertussis + Haemophilus influenzae type b. † Measles, mumps, rubella. ‡ Bacillus of Calmette Guérin.
Value 8.5 (0–53) 60% 40% 92% 8% 54% 14% 13% 2% 1%
174 • alves et al.
Table 3. Psychometric Properties Observed for Children’s and Infants’ Postoperative Pain Scale During Validation Process, Porto Alegre, RS, Brazil (n = 100) Psychometric Property
Statistics
Value
95% CI
P
Inter-rater reliability Internal consistency Construct validity
Intraclass correlation coefficient Cronbach’s alpha Wilcoxon test mean 1 SD
0.84 to 0.92 0.70 to 0.90 -
