Interobserver reliability in Pirani clubfoot severity scoring between a ...

366 Original article

Interobserver reliability in Pirani clubfoot severity scoring between a paediatric orthopaedic surgeon and a physiotherapy assistant Samir Shaheena, Hiba Jaiballab and Shafique Piranic The Ponseti method, now regarded as the standard of care for congenital clubfoot, is equally effective whether provided by orthopaedic surgeons or orthopaedic paramedics. Therefore, it is particularly suitable for underresourced nations with lack of surgeons and physicians. At the Sudan Clubfoot Clinic, physiotherapy assistants (3-year diploma nurses with additional physiotherapy experience) are part of the Ponseti clubfoot treatment team, with the role of assessing the degree of deformity by the Pirani score to assist the team in providing treatment. However, the reliability of Pirani scores measured by physiotherapy assistants in this context is unknown. After obtaining informed consent, we measured the interobserver reliability between a physiotherapy assistant and an orthopaedic surgeon in measuring Pirani scores in 91 virgin clubfeet in 54 infants (41 males and 13 females) at the Sudan Clubfoot Clinic. Scores were measured independently before the onset of treatment and analysed by the j statistic for interobserver reliability. The j statistic was 0.61 for posterior crease, 0.72 for empty heel, 0.51 for rigid equinus, 0.54 for the hid-foot score, 0.57 for medial crease, 0.54 for curved lateral border, 0.56 for lateral head of talus, 0.50 for the midfoot score and 0.50 for the

Introduction Congenital talipes equino varus clubfoot affects males more than females and is a common problem in paediatric orthopaedic clinics [1,2]. Dr Ponseti developed his method of treatment of clubfoot in the 1940s. This is now accepted as a standard method for managing cases with clubfoot [1,3]. Scoring of the deformity can be carried out by orthopaedic surgeons or properly trained paramedics. The Ponseti method is highly suitable for treating children born with clubfeet in underdeveloped regions of the world [4,5]. Good results are achieved when the method is applied to correct the deformity, with long-term good to excellent functional outcomes and a low frequency of complications. As part of the Sudan Clubfoot Project at Khartoum Cheshire Home and Soba University Hospital, we trained nurses working as physiotherapy assistants for 1 week in Pirani clubfoot severity scoring and plaster casting. The results of the first 85 patients were published in Research in Action in 2006 [6]. In the Sudan clubfoot clinic, Pirani scoring is used both for initial assessment and for follow-up of treatment [7,8]. To date, over 800 children with clubfeet have been treated by the Sudan Clubfoot Project. Because of shortage of suitably trained doctors and physiotherapist, c 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins 1060-152X

total score. The mean percentage of agreement of both observers for all Pirani components was 83%. We found moderate to substantial interobserver reliability for the Pirani clubfoot severity score and all its subcomponents. Properly trained physiotherapy assistants are efficient in assessing the degree of severity of clubfoot. This is particularly useful in developing countries, where orthopaedic surgeons are few. Clubfoot treatment can be made more affordable by using paramedical healthcare workers such as physiotherapy assistants. J Pediatr c 2012 Wolters Kluwer Health | Orthop B 21:366–368 Lippincott Williams & Wilkins. Journal of Pediatric Orthopaedics B 2012, 21:366–368 Keywords: interobserver reliability, orthopaedic surgeons, physiotherapy assistant, Pirani clubfoot severity score a Department of Orthopaedics and Traumatology, Faculty of Medicine, University of Khartoum, bKhartoum Cheshire Home, Khartoum, Sudan and cDepartment of Orthopaedics, University of British Columbia, New Westminster, BC, Canada

Correspondence to Dr Samir Shaheen, MD, Associate Professor of Orthopaedics, Department of Orthopaedics and Traumatology, Faculty of Medicine, University of Khartoum, P.O. Box 102, Khartoum, Sudan Tel: + 249 912 376929; fax: + 249 183 771 211; e-mail: [email protected]

many patients are assessed by trained physiotherapy assistants in our clinics. A physiotherapy assistant in Sudan is a holder of a 3-year diploma in nursing, followed by an on the job training to perform basic physiotherapy activities for disabled children at our paediatric orthopaedic unit and clubfoot clinic. It is not known if physiotherapy assistants can be as accurate as doctors and trained physiotherapists in assessing the degree of deformity in the clubfeet of the children seen in the clinic. The purpose of this study was to determine whether, after a short training in the Pirani method, a physiotherapy assistant can be as accurate as a doctor or a physiotherapist in assessing the degree of deformity in clubfoot.

Methods A physiotherapy assistant who had undergone a 1-week training in Pirani clubfoot severity scoring and the Ponseti method of casting at the Sudan Clubfoot Clinic was a second observer and an orthopaedic surgeon (the author) was the first observer. From 25 January 2007 to 29 December 2007, 54 patients less than 6 months of age who presented to the Sudan Clubfoot Clinic with virgin DOI: 10.1097/BPB.0b013e3283514183

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Interobserver reliability in Pirani clubfoot Shaheen et al. 367

clubfeet were enrolled in this study. All clubfeet were independently assessed by both observers using the Pirani score.

Table 1 Means of scores for each parameter for both observers 1 and 2 in 54 patients (91 feet)

Patients who had received prior treatment were excluded from the study. All clubfeet were managed by the Ponseti method. The data were analysed for interobserver reliability using the k statistic and point-by-point interobserver agreement. The k statistic interobserver reliability (strength of agreement) was judged as poor (< 0.00), slight (0.00–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80) or almost perfect (0.81–1.00) [9].

PC EH RE HFS MC CLB LHT MFS TS

Results There were 54 patients (41 males and 13 females) aged between 3 and 252 days, with 91 feet (13 right, four left and 37 bilateral). Patients (53 of 54 patients; 98.1%) were less than 8 months of age. For each of the 91 feet, all Pirani clubfoot severity scores were recorded twice, once by each observer, for a total of 1092 observations. Differences between the means of scores for each severity component of the deformity including the sum of midfoot scores (MFS), hindfoot scores (HFS) and total foot scores were less than 0.05 (Table 1). The k values for the observed scores were 0.61 for posterior crease, 0.72 for empty heel (EH), 0.51 for rigid equinus, 0.57 for medial crease (MC), 0.54 for curved lateral border and 0.56 for lateral head of talus. The k values for the calculated scores were 0.54 for HFS, 0.50 for MFS and 0.50 for the total score (TS); the k statistical interobserver reliability of all the severity components was rated moderate to substantial (Table 2). Percentages of point-by-point agreement between both observers ranged between 93.4% (85 of 91 feet) in the EH component (being the highest) to 74.7% (71 of the 91 feet) for the MC component (being the lowest; Table 3).

Discussion The terms classification and evaluation are often used interchangeably with respect to clubfeet [10], and can lead to confusion. For the purposes of this article, classification is considered as a characteristic of clubfoot that does not change with time or aetiology such as postural, congenital idiopathic, neuromuscular or syndromic forms. It is useful as it can be of prognostic significance. Evaluation, in contrast, describes an assessment of the degree of deformity present at the time of evaluation. Scoring of the degree of deformity (clubfoot severity score) may help guide treatment. Many classification systems are used in orthopaedic practice, some of which have reliable consistent interobserver scores, but many others do not [11]. Both Cummings et al. [10] and Catterall [12] have commented on the problematic nature of clubfoot severity scoring. Recently, Pirani et al. [14] developed a valid and reliable method of clinically evaluating the degree of

Parameters

Average observer 1

Average observer 2

0.75 0.8 0.8 2.35 0.55 0.75 0.65 2.00 4.35

0.8 0.8 0.75 2.35 0.6 0.75 0.7 2.05 4.40

The second group indicates features of midfoot contracture. CLB, curvature of the lateral border; EH, empty heel; HFS, hindfoot score; LHT, reducibility of the lateral part of the head of the talus; MC, medial crease; MFS, midfoot score; PC, posterior crease; RE, rigid equines; TS, total score.

Table 2 Pirani score parameters and the j statistic interobserver strength of agreement Parameters PC EH RE HFS MC CLB LHT MFS TS

k

Rate

0.61 0.72 0.51 0.54 0.57 0.54 0.56 0.50 0.50

Substantial Substantial Moderate Moderate Moderate Moderate Moderate Moderate Moderate

The second group indicates features of midfoot contracture. CLB, curvature of the lateral border; EH, empty heel; HFS, hindfoot score; LHT, reducibility of the lateral part of the head of the talus; MC, medial crease; MFS, midfoot score; PC, posterior crease; RE, rigid equines; TS, total score.

Table 3 Percentage of point-to-point agreement for each Pirani score parameter between both observers in 54 patients (91 feet) Pirani scoring parameters PC EH RE MC LHT CLB Mean

Percentage of agreement 82 85 71 68 73 79 76.33

(90.1) (93.4) (78) (74.7) (80.2) (86.8) (83.5)

CLB, curvature of the lateral border; EH, empty heel; LHT, reducibility of the lateral part of the head of the talus; MC, medial crease; PC, posterior crease; RE, rigid equines.

deformity in a clubfoot. The Pirani clubfoot severity scores describe the degree of deformity in the entire foot. These are TS, the HFS and the MFS. Six signs of clubfoot are scored either zero (no abnormality), half (moderate abnormality) or one (severe abnormality). Three are scored in the hindfoot: posterior crease, EH and rigid equinus. Their sum forms the HFS, which can total from zero to three. Three are in the midfoot: MC, curved lateral border and lateral head talus. The TS is a sum of HFS and MFS and can be between 0 and 6. Flynn et al. [13] found this scoring system to be reliable, quick and easy to use after a short initial training phase.


368 Journal of Pediatric Orthopaedics B 2012, Vol 21 No 4

Our study has shown moderate to substantial interobserver reliability between a paediatric orthopaedic surgeon and a physiotherapy assistant in assessing the various elements of the Pirani clubfoot severity score. Using the k score in a study similar to ours, Pirani et al. [14] found the interobserver strength of agreement in clubfoot scoring to be substantial or almost perfect. However, in their study, the second observer was an orthopaedic resident, not a paramedic. In the present study, the mean interobserver point-by-point agreement for all deformity components was found to be 83%. Flynn et al. [13] reported 89% agreement between observers but in their study both the examiners were physicians. Interobserver variations can be attributed to differences in the training and background of both observers [15]. In other reported studies, the evaluation was between doctors or a doctor and a trained physiotherapist [5,13]. In the present study, the comparison was between a physiotherapy assistant and an orthopaedic surgeon. The Pirani clubfoot severity score is useful in managing clubfeet. It is used to monitor correction of deformity, detect relapse, predict when tenotomy is indicated and to predict the number of casts needed [7,8]. Reliability refers to the relationship between measurement error and the expected distribution of measurements over time and across observers and situations [16]. Different types of reliability studies are available. If interobserver reliability proves to be acceptable, then no additional reliability test is needed [16]. In studying reliability, certain key questions need to be addressed regarding the appropriateness of the research question, whether raters will apply the instruments in practice, whether study patients represented the population that will be rated in practice and whether data were analysed using appropriate statistics [16]. In this study, all these were taken into consideration. This method of scoring and clubfoot management proved to be suitable for developing countries [17]. Our finding of moderate to substantial interobserver reliability for the Pirani clubfoot severity score between a paediatric orthopaedic surgeon and a nonmusculoskeletal paramedic trained in the score allows the treating doctor to have confidence in the ability of a trained paramedic to assess the degree of deformity in a clubfoot. Therefore, the trained paramedic can assist in performing this aspect of care in the clubfoot clinic, relieving the doctor of the time burden associated with the assessment.

Furthermore, by performing assessments that are usually performed by physicians, they can relieve the burden on doctors in clubfoot clinics and clubfoot services for communities where no trained medical personnel or physiotherapists are available. It is less expensive to use paramedics than physicians in assessing club foot. We recommend using trained physiotherapy assistants in assessing clubfoot deformity in areas where there is a scarcity of physicians.

Acknowledgements Conflicts of interest

The authors state that there is no conflict of interest and no benefits in any form have been received or will be received from any party related directly or indirectly to the subject in this article.

References 1

2 3

4

5

6

7

8 9 10 11

12 13 14

15

Conclusion

16

Our study showed moderate to substantial interobserver reliability between a paediatric orthopaedic surgeon and a physiotherapy assistant in scoring clubfoot deformity.

17

Lehman WB, Mohaideen A, Madan S, Scher DM, Van Bosse HJ, Iannacone M, et al. A method for the early evaluation of the Ponseti (Iowa) technique for the treatment of idiopathic clubfoot. J Pediatr Orthop B 2003; 12:133–140. Dobbs MB, Morcuende JA, Gurnett CA, Ponseti IV. Treatment of idiopathic clubfoot: an historical review. Iowa Orthop J 2000; 20:59–64. Sud A, Tiwari A, Sharma D, Kapoor S. Ponseti vs. Kite’s method in the treatment of clubfoot –a prospective randomised study. Int Orthop (SICOT) 2008; 32:409–413. Shack N, Eastwood DM. Early results of a physiotherapist-delivered Ponseti service for the management of idiopathic congenital talipes equinovarus foot deformity. J Bone Joint Surg Br 2006; 88:1085–1089. Janicki JA, Narayanan UG, Harvey BJ, Roy A, Weir S, Wright JG. Comparison of surgeon and physiotherapist-directed Ponseti treatment of idiopathic clubfoot. J Bone Joint Surg Am 2009; 91:1101–1108. doi: 10.2106/ JBJS.H.00178. Shaheen S, Edward P, Mannion S. Experience with management of clubfoot in Sudan. Bethune Round Table on International Surgery 2006, Research in Action, University of Toronto, Page 15, 416-480-5260. Available at: http:// www.utoronto.ca/ois [Accessed 14 October 2011]. Dyer PJ, Davis N. The role of the Pirani scoring system in the management of clubfoot by the Ponseti method. J Bone Joint Surg Br 2006; 88: 1082–1084. Fixsen J. Associate Editor, Data Supplement. J Bone Joint Surg Br 2006; 88:1082–1084, . Wainwright AM, Auld T, Benson MK, Theologis TN. The classification of congenital talipes equinovarus. J Bone Joint Surg Br 2002; 84:1020–1024. Cummings RJ, Davidson RS, Armstrong PF, Lehman WB. Congenital clubfoot. J Bone Joint Surg Am 2002; 84:290–308. Brumback RJ, Jones AL. Inter-observer agreement in the classification of open fractures of the tibia: the results of a survey of two hundred and fortyfive orthopaedic surgeons. J Bone Joint Surg Am 1994; 76:1162–1166. Catterall J. A method of assessment of the clubfoot deformity. Clin Orthop 1991; 264:48–53. Flynn JM, Donohoe M, Mackenzie WG. An independent assessment of two clubfoot classification systems. J Paediatric Orthop 1998; 18:323 –327. Pirani S, Hodges D, Sekeramyi F. A reliable and valid method of assessing the amount of deformity in the congenital clubfoot deformity. J Bone Joint Surg Br 2008; 90 (Suppl):53. Andriesse H, Ha¨gglund G, Jarnlo GB. The Clubfoot Assessment Protocol (CAP); description and reliability of a structured multi-level instrument for follow-up. BMC Musculoskelet Disord 2005; 6:40. Karanicolas PJ, Bhandari M, Kreder H, Moroni A, Richardson M, Walter SD, et al. Evaluating agreement: conducting a reliability study. J Bone Joint Surg Am 2009; 91:99–106. Cooper DM, Dietz FR. Treatment of idiopathic clubfoot: a thirty-year followup note. J Bone Joint Surg Am 1995; 77:1477–1489.
