A new clinical outcome measure of glenohumeral joint instability: The ...

A new clinical outcome measure of glenohumeral joint instability: The MISS questionnaire Lyn Watson, BAppSci (Physio) GradDipManip,a Ian Story, BBSc(Hons), PhD,b Rodney Dalziel, FRACS, FAOrthA,c Greg Hoy, FRACS, FAOrthA, FACSP, FASMF,c Andrew Shimmin, FRACS, FAOrthA,c and David Woods, MBBS,c Melbourne, Australia

Many standard shoulder outcome measures do not adequately cover the range of problems and issues specifically encountered in glenohumeral joint instability and have been shown not to be sensitive enough to detect clinical change with intervention adequately. The purpose of this report is to present a prospective evaluation of a new self-administered patient questionnaire specifically designed to assess glenohumeral joint instability. The evaluation involved test-retest reliability and comparison with the Shoulder Rating Questionnaire (SRQ). Sixtyfour patients with confirmed glenohumeral joint instability were assessed with both the Melbourne Instability Shoulder Scale (MISS) and SRQ12 preoperatively and at 6 months after shoulder reconstructive surgery. Twenty-two patients were recruited into a reliability study of the MISS questionnaire. The test-retest reliability of the MISS was found to be 0.98 (interclass correlation coefficient, mixed-model analysis of variance, absolute agreement). Assessment of agreement between the MISS and SRQ questionnaires indicated very poor pretest agreement (0.33) and moderate agreement at 6 months (0.66). The differences between the MISS and SRQ were statistically significant both before surgery (paired t ⫽ 13.2, degrees of freedom [df] ⫽ 63, P ⬍ .001) and at 6 months’ follow-up (paired t ⫽ 7.9, df ⫽ 63, P ⫽ .001). Change in the questionnaire scores measured from surgery to 6 months’ follow-up was significantly greater in the MISS (mean, 30; SD, 19.1; median, 30.8) than in the SRQ (mean, 16.6; SD, 12.8; median, 14.3) (Wilcoxon test: z ⫽ ⫺5.8, P ⫽ .0001). The results of this study show that the MISS questionnaire is a reliable outcome questionnaire and has a greater range to detect changes in shoulder instability than more global outcome questionFrom LifeCare Prahran Sports Medicine, Prahran,a School of Physiotherapy,b The University of Melbourne, Parkville, and Melbourne Orthopaedic Group, Windsor.c Reprint requests: Dr L. Watson, LifeCare Prahran Sports Medicine, 1st Floor, 316 Malvern Rd, Prahran 3181, Australia. Copyright © 2005 by Journal of Shoulder and Elbow Surgery Board of Trustees. 1058-2746/2005/$30.00 doi:10.1016/j.jse.2004.05.002

22

naires such as the SRQ. The higher scores encountered on the SRQ may mean that it underestimates the severity of a patient’s instability problem. (J Shoulder Elbow Surg 2005;14:22-30.)

T

he increasing expectation that clinicians provide outcome data to support their management strategies has highlighted the need for outcome measures that have been proven to be reliable, valid, and sensitive to the benefits of treatment. However, few assessments of the treatment methods that are available use independent outcome measures. Many use scoring systems that are dependent on either patient interview or physical examination,14,16 which both may be open to observer bias.13,17 The reliability of these methods has also been found to be limited as a result of poor interobserver or intraobserver reliability.10,12,13,17,18 Standard assessment scores (such as the Short Form 3618; ConstantMurley4; Shoulder Rating Questionnaire [SRQ]12; Disabilities of the Arm, Shoulder and Hand [DASH]7; or American Shoulder and Elbow Surgeons Assessment [ASESA]1) have limited application in the assessment of glenohumeral joint instability because most of these concentrate on measurement of, or any combination of, pain, range of motion, functional loss, or strength.3,5,13 However, the primary, and sometimes only, complaint in instability is apprehension or avoidance of activity because of fear of an unstable event occurring. Hence, there is a need for outcome measures that are specifically designed to measure clinical change in symptoms or glenohumeral joint instability. Self-administered patient questionnaires have been shown to be a valid and reliable method by which to determine the effects of treatment on the health and quality of life of the patient.5,6,8,12,15 The purpose of this report is to present an initial evaluation of a new instability-specific, self-administered questionnaire—the Melbourne Instability Shoulder Score (MISS) (Appendix 1)—in comparison to a standard shoulder assessment outcome score (SRQ). The SRQ was selected as the comparative measure for the study, as this is one of the few self-administered shoulder scores that has been shown to be a reliable and valid measure for the assessment of glenohumeral instability.6

Watson et al

J Shoulder Elbow Surg Volume 14, Number 1

23

MATERIALS AND METHODS Development of questionnaire The MISS was developed through a process of item generation, item reduction, pretesting, and test-retest reliability analysis. Items were generated from a thorough review of the literature, modifications of similar disease-specific quality-oflife outcome measures (ASESA1 and Rowe et al16), discussions with clinicians experienced in the area of both outcome measures and glenohumeral joint instability, and formal interviews with 50 patients with confirmed glenohumeral joint instability. An initial draft questionnaire underwent item reduction via a formal interview with 20 preoperative patients. A second draft questionnaire was tested in a pilot trial in another group of 20 preoperative and 20 postoperative patients. A relative percentage of the patient’s perception of importance of each section of the questionnaire was also compiled and used in combination with input from 5 shoulder orthopaedic specialists in developing the weighting system for scoring the MISS (Appendix 2).

Design of study Eighty patients who presented to a single group practice of shoulder surgeons were recruited consecutively over a period of 18 months into a prospective study. A patient was included in the study if he or she had a diagnosis of glenohumeral dislocation or subluxation that he or she and the surgeon had mutually agreed required surgical stabilization. All subjects gave informed consent, and the study was approved by the Melbourne Orthopaedic Group Research Committee (Melbourne, Australia). Sixty-four patients were still available for follow-up at 6 months after surgical intervention. Sixteen patients dropped out of the study. Ten either had decided not to go ahead with surgery or had delayed surgery, four relocated without notification of further contact details, and two had further surgical intervention because of failure of the original surgical procedure before the 6-month date and, therefore, were excluded from the study. The first 64 patients who were still available for follow-up at 6 months after surgical intervention became the study’s patient population. There were 21 women and 43 men with a mean age of 26.45 years (SD, 7.46 years; median, 24 years). The dominant shoulder was involved in 41 patients and the nondominant shoulder in 23. Fifty-one had unilateral shoulder involvement, and thirteen had bilateral involvement. The median duration of symptoms was 2 years (range, 1 month [minimum] to 15 years [maximum]). The patient population comprised a mix of athletic and nonathletic groups. All patients had a diagnosis of glenohumeral joint instability based on a combination of history, clinical examination, medical notes, radiography, and/or computed tomography findings. Forty-two patients were assessed as having unidirectional instability, while twenty-two had multidirectional instability. At initial evaluation, each patient completed the MISS questionnaire and the SRQ, which was selected as the comparative measure for this study. Patients were also asked to give an overall percentage rating of their shoulder compared with normal (subjective rating score). Twenty-two patients whose symptoms had not changed repeated the MISS questionnaire (Appendix 1) and the subjective percentage rating 1 week later as part of the reliability assessment of the study. Subjects were timed during perfor-

Figure 1 Mean MISS and SRQ scores for all patients before and after surgery (mean ⫾ SEM).

mance of this task and took a mean of 4 minutes 45 seconds to complete it. All patients underwent a surgical stabilization procedure for the glenohumeral joint. They were followed up prospectively and reassessed with both forms at 6 months after surgery.

RESULTS Reliability

The test-retest reliability of the MISS over the two occasions was found to be 0.98 (interclass correlation coefficient [ICC]; mixed-model analysis of variance; absolute agreement, which demonstrates a high level of agreement). The ICC is a widely used measure of interrater reliability for the case of quantitative ratings.9 The test-retest reliability of the subjective ratings by patients was also found to be high (ICC ⫽ 0.94, mixed-model analysis of variance, absolute agreement). Assessment of agreement between the MISS and SRQ by use of an ICC value (mixed-model analysis of variance, absolute agreement) indicated very poor agreement before testing (0.33) and moderate agreement at 6 months (0.66). This is due to the fact that the MISS has a much lower preoperative score (mean, 45.3; SEM, 2.0) than the SRQ (mean, 67.1; SEM, 1.9). Although all scores increased postoperatively, this relationship is still present with the MISS score reflecting a lower score than the SRQ (Figure 1). The differences between the MISS and SRQ were statistically significant both before surgery (paired t ⫽ 13.2, degrees of freedom [df] ⫽ 63, P ⬍ .001) and at 6 months’ follow-up (paired t ⫽ 7.9, df ⫽ 63, P ⫽ .001). Change measured from before surgery to 6 months’ follow-up was significantly greater for the MISS (mean, 30; SD, 19.1; median, 30.8) than the SRQ (mean, 16.6; SD, 12.8; median, 14.3) (Wilcoxon test: z ⫽ ⫺5.8, P ⫽ .0001). The comparison of each patient’s MISS scores to the subjective rating score showed good linear correlation for the MISS questionnaire both preoperatively and postoperatively (Figures 2 and 3). The same comparison between SRQ scores and subjective rating score indicated that no relationship existed preoperatively

24

Watson et al

J Shoulder Elbow Surg January/February 2005

Figure 2 Relationship between MISS scores and patients’ subjective assessment (%) of shoulder before surgery (R2 ⫽ 0.605).

Figure 3 Relationship between MISS scores and patients’ subjective assessment (%) of shoulder 6 months after surgery (R2 ⫽ 0.733).

(Figure 4) and that the postoperative relationship was displaced in favor of the SRQ (Figure 5). The minimal detectable change in MISS scoring was calculated as 4.5 points. The minimal clinically important difference was determined to be 5 points.

DISCUSSION The results of this study clearly show that the MISS questionnaire is a reliable outcome measure demonstrating excellent test-retest reliability. The subjective rat-


Watson et al

25

Figure 4 Relationship between SRQ scores and patients’ subjective assessment (%) of shoulder before surgery (R2 ⫽ 0.0001).

Figure 5 Relationship between SRQ scores and patients’ subjective assessment (%) of shoulder 6 months after surgery (R2 ⫽ 0.557).

ing score was also found to have high test-retest reliability. The fact that the subjective rating score test-retest reliability was lower than the MISS may reflect that the MISS is, in fact, a more reliable and uniformly stable

measure of patient outcome than a subjective rating score. Overall, the MISS scores were generally lower than the SRQ scores. In particular, the MISS preoperative

26

Watson et al

scores were much lower (mean, 45; SD, 16) than the preoperative SRQ scores (mean, 67; SD, 15). Although both questionnaires were able to detect a significant change (P ⬍ .001) in the preoperative and postoperative scores, the lower preoperative scores of the MISS indicate that the MISS has a greater range to show change than the SRQ. The very poor agreement between the two questionnaires at both the pretest and postoperative conditions indicates that the MISS questionnaire is clearly measuring outcome differently than the SRQ. This suggests that a more specifically tailored questionnaire may have a greater capacity to detect change in a selected pathology than a more generalized outcome questionnaire. The fact that it is harder to get a high score on the MISS than on the SRQ is one of the strong points of this questionnaire. The authors wanted to develop a questionnaire that truly reflected the full functional status of their patients and, therefore, believed that a greater spread of questions related to instability avoidance and functional tasks was required than is examined in other instability questionnaires such as the Western Ontario Shoulder Instability Index (WOSI),11 DASH,7 or ASESA,1 which may underestimate the patient’s clinical impairment. The score achieved on the MISS is really a percentage of the maximum or perfect shoulder function. Although some patients were able to achieve these high results, it is clear from examining the patients’ subjective percentage that very few felt that their shoulders were 100% at 6 months after surgery. Hence an accurate and sensitive outcome measure should not be producing all high scores. Comparison of the MISS and SRQ to the overall percentage rating score of the patient confirms that the MISS is a much more accurate measure of clinical outcome. The MISS demonstrated good correlation to the patients’ percentage rating score, whereas the SRQ demonstrated very poor agreement. This demonstrates that the MISS is clinically relevant and accurately reflects the patients’ perception of their problem, which has previously been one of the major limitations with many shoulder scoring systems.5 The MISS questionnaire was shown to be valid, reproducible, and sensitive to clinical change in both the nonathletic and athletic population. A minimum difference of 5 points is required to detect any clinically significant change. It fulfills the essential criteria of a health status index for use in outcome research.2,12 Although it is applicable to the population with instability, it is not applicable to the general shoulder population, and this may limit its use. For other shoulder pathologies, such as glenohumeral joint arthritis or rotator


cuff tears, a broader questionnaire, such as the SRQ,12 might be more appropriate. We would like to acknowledge Ian Story, BBSC(Hons), PhD, from the University of Melbourne and Associate Professor Caroline Finch from Monash University, Melbourne, Australia, for their assistance with statistical analysis, as well as Mr Simon Balster and Ms Tegan Colliver for editorial assistance. REFERENCES

1. American Shoulder and Elbow Surgeons Research Committee, Richards RR, An K, et al. A standardized method for the assessment of shoulder function. J Shoulder Elbow Surg 1994;3:347-52. 2. Beaton DE, Richards RR. Measuring function of the shoulder. A cross-sectional comparison of five questionnaires. J Bone Joint Surg Am 1996;78:882-90. 3. Conboy VB, Morris RW, Kiss J, Carr AJ. An evaluation of the Constant-Murley shoulder assessment. J Bone Joint Surg Br 1996; 78:229-32. 4. Constant CR, Murley AHG. A clinical method of functional assessment of the shoulder. Clin Orthop 1987;214:160-4. 5. Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. J Bone Joint Surg Br 1996; 78:593-600. 6. Dawson J, Fitzpatrick R, Carr A. The assessment of shoulder instability. The development and validation of a questionnaire. J Bone Joint Surg Br 1999;81:420-6. 7. Disabilities of the arm, shoulder and hand. Outcomes data collection instrument, version 2.0. Toronto: Dash Publications; 1997. 8. Fitzpatrick R, Fletcher A, Gore S, et al. Quality of life measures in health care. I: Applications and issues in assessment. BMJ 1992;305:1074-7. 9. Fleiss JL. Statistical methods for rates and proportions. 2nd ed. Wiley series in probability and mathematical statistics. New York: John Wiley & Sons; 1981. p. 218, 225-7. 10. Gartland JJ. Orthopaedic clinical research. Deficiencies in experimental design and determinations of outcome. J Bone Joint Surg Am 1988;70:1357-64. 11. Kirkley A, Griffin S, McLintock H, Ng L. The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability. The Western Ontario Shoulder Instability Index (WOSI). Am J Sports Med 1998;26:764-72. 12. L’Insalata JC, Warren MD, Cohen SB, Altechek DW, Peterson MGE. A self administered questionnaire for assessment of symptoms and function of the shoulder. J Bone Joint Surg Am 1997;79:738-48. 13. Matsen FA, Ziegler DW, DeBarolo SE. Patient self assessment of health status and function in glenohumeral degenerative joint disease. J Shoulder Elbow Surg 1995;4:345-51. 14. Meenan RF, Anderson JJ, Kazis LE, et al. Outcome assessment in clinical trials. Evidence for the sensitivity of a health status measure. Arthritis Rheumatol 1984;27:1344-52. 15. Romeo AA, Bach BR Jr, O’Halloran KL. Scoring systems for shoulder conditions. Am J Sports Med 1996;24:472-6. 16. Rowe CR, Zarins B, Giullo JV. Recurrent anterior dislocation of the shoulder after surgical repair. Apparent causes of failure and treatment. J Bone Joint Surg Am 1984;66:159-68. 17. Sapega AA. Current concepts review. Muscle performance evaluation in orthopaedic practice. J Bone Joint Surg Am 1990;72: 1562-74. 18. Wade JE. SF-36 health survey manual and interpretation guide. Boston: Nimrod Press; 1993.

Watson et al


APPENDIX

Appendix Figure 1 MISS.

27

28

Watson et al


Appendix Figure 1 (Continued)

Watson et al


Appendix Figure 1 (Continued)

29

30

Watson et al


Appendix Figure 2 Scoring system for MISS.