Brief Report: Development and Validation of a Semiautomated Method ...

ARTHRITIS & RHEUMATOLOGY Vol. 68, No. 2, February 2016, pp 332–336 DOI 10.1002/art.39459 C 2016, American College of Rheumatology V

BRIEF REPORT

Development and Validation of a Semiautomated Method to Measure Erosion Volume in Inflammatory Arthritis by Computed Tomography Scanning Jeffrey Duryea,1 Ruby Russell,1 Ellen M. Gravallese,2 Jonathan Kay,2 Roger Han,1 Bing Lu,1 and Daniel H. Solomon1 Objective. Valid measurement of erosion volume in rheumatoid arthritis (RA) will facilitate the testing of treatments that may help to heal erosion. This study was undertaken to develop and validate a software method to measure erosion volume on computed tomography (CT) scans of the hand and wrist. Methods. Duplicate CT acquisitions of both hands of 5 patients with RA were evaluated using a semiautomated software tool to measure erosion volume in the entire hand and wrist and in each of 6 subregions. Reproducibility was quantified using the intraclass correlation coefficient (ICC), root mean square standard deviation (RMSSD), and coefficient of variation (CV), and the analysis was performed at the level of the hand (n 5 10) and the subject (n 5 5). Results. The ICCs between 2 repositioned acquisitions were excellent, ranging from 0.97 to 1.00. At the hand level, the RMSSD was 15.6 mm3 with a CV of 7.3%, and the CVs at the 6 regions ranged from 7.6% to 21.0%. At the subject level, the RMSSD was 31.2 mm3 with a CV of 3.7%, and the CVs at the 6 regions ranged from 0.5% to 15.8%. Conclusion. We have developed a novel semiautomated software method to measure erosion volume on hand and wrist CT scans. The method is reproducible and can be used to detect changes in erosion volume. This will facilitate the testing of treatments intended to reduce erosion volume.

Articular bone erosions in rheumatoid arthritis (RA) are strongly associated with long-term disability and remain an important treatment target (1). With more aggressive treatment, progression of joint erosions is observed less commonly. Several biologic and small-molecule disease-modifying antirheumatic drugs (DMARDs) have been shown to arrest or slow erosion progression, but none has been demonstrated to reduce erosion volume impactfully (2–4). Drugs developed to alter bone metabolism have been tested for their effects on joint erosions, with mixed results: administration of highly potent aminobisphosphonates has reduced erosion number in some studies (5), and denosumab treatment has produced a slight improvement in the Sharp score (6). It is possible that therapies that are anabolic for bone, in contrast to those that are antiresorptive, may reduce erosion volume. However, a more sensitive measure of erosion volume may be needed to demonstrate such change. Imaging modalities, including conventional radiography, computed tomography (CT), and magnetic resonance imaging (MRI), are used extensively to assess structural consequences of RA, both in clinical and in research settings. For research purposes, 2 methods have been used to evaluate images: semiquantitative scoring and quantitative measurement. The Sharp (7) and van der Heijde (8) scoring systems both provide semiquantitative assessments of joint space width (JSW) and erosions seen on hand and wrist radiographs. Several reports have described automated software used for quantitative measurement of JSW (9–12) and erosion size (13) on digital radiographs. More recently, these methods have been applied to 3-dimensional (3-D) imaging modalities, such as MRI and CT. The RAMRIS (RA MRI Scoring) system was developed to provide a semiautomated assessment of hand and wrist MRI in patients with RA (14). Several studies have measured structural changes in RA using hand or wrist CT (4,15–17); however, to our knowledge, none has used automated software to quantify true erosion volume.

Supported by Eli Lilly and company. 1 Jeffrey Duryea, PhD, Ruby Russell, BA, Roger Han, MD, Bing Lu, PhD, Daniel H. Solomon, MD, MPH: Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts; 2Ellen M. Gravallese, MD, Jonathan Kay, MD: University of Massachusetts Medical School, Worcester. Address correspondence to Jeffrey Duryea, PhD, Radiology Department, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis Street, Boston, MA 02115. E-mail: jduryea@bwh. harvard.edu. Submitted for publication May 29, 2015; accepted in revised form September 29, 2015. 332

SEMIAUTOMATED SOFTWARE METHOD TO MEASURE EROSION VOLUME ON CT SCANS IN RA

333

The purpose of this study was to develop and validate a method using semiautomated software to measure erosion volume on CT scans, which could be applied in longitudinal studies of RA. This method uses specially designed software that facilitates segmentation (outlining) of erosions in 3-D and then estimates erosion volume by calculating the number of voxels in the outlined region. We demonstrate the repositioning reproducibility of the method, using bilateral hand and wrist CT scans from 5 subjects with RA, and suggest its potential use in future trials of agents that might reduce erosion volume. PATIENTS AND METHODS Subjects and study design. We recruited 5 patients at the Brigham and Women’s Hospital Arthritis Center who fulfilled the American College of Rheumatology/European League Against Rheumatism 2010 classification criteria for RA (18). Four of the patients were known to have had erosions and the other had minimal or no erosions seen on plain radiography. No other selection criteria were applied. Both hands of each patient were scanned twice, 10–15 minutes apart, using a Siemens Somatom Definition AS 64slice CT scanner. Each patient was placed prone on the scanner table, in a “Superman” posture, with 1 arm held above the head and keeping the wrist in as straight a position as possible. The study used the Siemens “small field of view” protocol since this provided improved in-plane spatial resolution. Between duplicate scans, subjects were removed from the CT scanner and repositioned. The CT protocol required unilateral hand and wrist scans to be performed at 140 kVp and 120 mA. Images were reformatted to achieve a slice spacing of 0.3 mm; the in-plane resolution was between 0.132 mm and 0.264 mm. Image analysis. A semiautomated software tool was used to segment the erosion margins in 3-D. A board-certified radiologist (RH) initially identified the location of individual erosions in the radius, ulna, and carpal bones and in the carpometacarpal (CMC), metacarpophalangeal (MCP), and proximal interphalangeal (PIP) joints and marked each erosion on the images using the computer mouse. Subchondral cysts were generally distinguished from RA erosions by finding an opening to the cortical bone margin. The distal interphalangeal joints were excluded from this assessment since RA is much less likely to involve these joints. A research assistant (RR) then used the software tool to draw an outline around the erosion margins on all CT axial slices in which erosions were present (Figure 1A). For the vast majority of erosions, the software was able to determine the bone–erosion margin automatically. The margin representing the missing bone edge at the site of an erosion was indicated using a manual drawing component of the software tool. When erosion margins were determined by the software, the radiologist subsequently performed a quality assurance evaluation, reviewing each case and indicating where corrections were necessary. A 3-D rendering (Figure 1B) provided the image readers with an accurate visualization of each erosion and could be used as a final step to

Figure 1. A, Example of erosion segmentation in the lunate bone on a single computed tomography slice. B, Example of a 3-dimensional rendering of the same lunate bone as in A.

check for segmentation errors that might be recognized less easily on the 2-D slices. Statistical analysis. The total erosion volume was calculated for all 5 digits on each hand, for the wrist, and for each of the 6 subregions (radius, ulna, proximal carpals [scaphoid, lunate, triquetrum, and pisiform], distal carpals [capitate, hamate, trapezium, trapezoid, and the CMC joints], MCP joints, and PIP joints). The reproducibility analysis was performed both at the level of the subject (n 5 5) and at the level of the hand (n 5 10). The hand-level analysis assumed that each of the 10 hands was an independent subject. Repositioning reproducibility was measured by comparing pairs of duplicate scans and calculating the intraclass correlation coefficient (ICC), root mean square standard deviation (RMSSD), and coefficient of variation (CV) yielded by the following equations, where V1i and V2i are the duplicate volume measurements, N is the number of comparisons (either 5 or 10), and V is the average volume over all 2N measurements:

334

DURYEA ET AL

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N uX u ðV1i 2V2i Þ2 u t i51 RMSSD 5 2N

Table 2.

Sample size projections based on study results Sample size needed per group* No. of hands

Difference, %

and

10 15 20 25

RMSSD CoV 5 V

No. of patients

ICC 0.20

ICC 0.50

ICC 0.20

ICC 0.50

120 54 30 20

150 68 38 24

60 27 15 10

75 34 19 12

* ICC 5 within-patient intraclass correlation.

RESULTS The patients ranged in age from 42 to 71 years; 4 of the 5 were female. Disease duration ranged from 5 to 20 years. Three of the patients were positive for rheumatoid factor and/or anti-citrullinated protein antibody, and all 5 had taken DMARDs in the past or were taking them currently. One of the 5 had undergone surgery in a wrist, which did not affect the image quality. Reproducibility results for the 10 hands and for the 5 patients are presented in Table 1. The average total erosion volume in a single hand and wrist was 214.0 mm3. Average erosion volume was smallest in the ulna (3.2 mm3) and largest in the distal carpals (72.4 mm3). The ICC values were excellent, ranging from 0.97 to 1.00. At the hand level, the RMSSD was 15.6 mm3 with a CV of 7.3%. The CVs for the 6 measured hand regions ranged from 7.6% to 21.0%. At the subject level, the RMSSD was 31.2 mm3 with a CV of 3.7%. The CVs for the 6 measured hand regions ranged from 0.5% to 15.8%. Individual regions with increased erosion volume tended to have correspondingly larger RMSSD values, Table 1.

Reproducibility results at the hand and patient levels*

Region Hand level (n 5 10) Full hand Radius Ulna Proximal carpals Distal carpals MCP joints PIP joints Patient level (n 5 5) Both hands Radius Ulna Proximal carpals Distal carpals MCP joints PIP joints

ICC

Average volume, mm3

RMSSD, mm3

CV, %

1.00 1.00 0.98 0.99 1.00 0.97 0.99

214.0 32.3 3.2 57.1 72.4 39.5 9.6

15.6 3.6 0.7 8.3 5.5 7.5 1.4

7.3 11.1 21.0 14.6 7.6 19.0 14.2

1.00 1.00 0.97 1.00 1.00 0.98 0.99

428.1 64.6 6.4 114.2 144.8 78.9 19.1

31.2 7.1 1.4 16.6 11.0 15.0 2.7

3.7 0.5 14.9 5.7 4.9 15.8 11.7

* ICC 5 intraclass correlation coefficient; RMSSD 5 root mean square standard deviation; CV 5 coefficient of variation; MCP 5 metacarpophalangeal; PIP 5 proximal interphalangeal.

while the dependence of the CV on the total volume was less pronounced. DISCUSSION Articular bone erosions in RA correlate strongly with disability. Although treatment of RA disease activity has become more successful over the last 2 decades, no currently available treatment has been proven to reproducibly heal erosions. To determine whether a drug therapy might heal erosions, erosion volume must be assessed accurately. We have developed a semiautomated software method to measure erosion volume seen on CT scans of the hand and wrist. The very high ICC values indicate excellent correlation between duplicate readings. The RMSSD and/or CV provide a direct assessment of the estimated differences in erosion volume measurement between the 2 readings. These values reflect the amount of change in erosion volume that could be detected in a longitudinal study, such as a prospective randomized controlled trial. Our data indicate that measurement error increases as a function of the total erosion volume in a particular anatomic region. This suggests that our semiautomated software tool may be used to detect change, if the change in erosion volume is proportional to the total volume of the erosion. In contrast, if the amount of erosion volume change is independent of the erosion volume, then the actual change in erosion volume may be lost in the error associated with the measurement method. Future longitudinal studies of therapies targeting erosion volume should help to address this issue. We have used the estimates derived from this replication exercise to explore sample size calculations for a longitudinal study using the 3-D CT methods described herein. We considered analyses both at the level of the subject and at the level of the individual hand. Assessing individual hands provides twice as many observations (2 hands for each patient), but these obser-

SEMIAUTOMATED SOFTWARE METHOD TO MEASURE EROSION VOLUME ON CT SCANS IN RA

vations are not independent of one another. In our pilot analysis that evaluated 5 patients (10 hands), log transformation was applied to total volumes to achieve an approximately normal distribution. The average value of log-transformed total volume was 5.32, with a standard deviation of 1.34. Assuming that a reasonable correlation of the measures between the 2 hands of the same patient (intraclass correlation) ranges from 0.20 to 0.50, we estimated the sample size needed to achieve 80% power, given a range of differences with a significance level of 0.050 using a 2-sided test (19) (Table 2). To detect a moderate difference of 15–25% between groups, the sample size needed to achieve adequate statistical power would range from 10 to 34 per study group. Several prior studies have evaluated erosion size using CT. Duryea et al described an automated software method to measure erosion volume based on CT data, but the study was limited to simulated erosions on anthropomorphic hand phantoms (16). Dohn and colleagues used several different imaging modalities (CT, MRI, and radiography) to assess reliability (15). In that study, erosion volume on the CT scans was determined using manual, and not semiautomated, segmentation. Similarly, Salaffi et al reported a manual segmentation technique to calculate CT erosion volume in a study that also compared measurements across modalities (17). Finzel and colleagues described an indirect method to assess volume based on measurements of the width and depth of the erosions using micro-CT (3,4). Unlike those used in each of these earlier studies, our method used automated software to determine the entire margin of the erosions on each patient CT scan in 3-D. Our study has several limitations. The number of subjects evaluated was small, although this was mitigated by use of both hands and identification of erosions in multiple joints of the hand. For this study, the reader was able to view both of the paired CT scans simultaneously; this is consistent with the reading method that is generally used for longitudinal studies. However, erosion volumes were not calculated until after the final reading, and quality assurance measures had been taken so that the readers were blinded with regard to these results. It is possible that the differentiation between erosions and subchondral cysts was not perfect since the opening to the cortical shell may not always have been visible. Patients with advanced RA may also have marked osteopenia. However, as long as the Hounsfield value (gray-scale level) of the osteopenic bone significantly exceeds the soft tissue value, the proper erosion margins can be detected with our method. We did not calculate the time that it took a reader to perform the erosion measurement, but we observed that this duration was highly dependent on the number of erosions that must be

335

evaluated: scans from patients with a large number of erosions took the reader substantially more time to evaluate than did those of patients with fewer erosions. In summary, we have developed and validated a novel, semiautomated, and fully quantitative software method to measure erosion volume on hand and wrist CT scans. To our knowledge, this technical approach to measurement of structural changes in RA is unique and will be useful for future longitudinal studies of treatments intended to reduce erosion volume in patients with inflammatory arthritis. We anticipate validating this method to assess erosion volume change with greater power in a future longitudinal clinical trial that will enroll substantially more patients than were included in this initial validation study. AUTHOR CONTRIBUTIONS All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Duryea had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study conception and design. Duryea, Russell, Gravallese, Kay, Han, Solomon. Acquisition of data. Duryea, Russell, Gravallese, Kay, Han, Solomon. Analysis and interpretation of data. Duryea, Gravallese, Kay, Han, Lu, Solomon.

ROLE OF THE STUDY SPONSOR Eli Lilly and Company facilitated the study design and reviewed and approved the manuscript prior to submission. The authors independently collected the data, interpreted the results, and had the final decision to submit the manuscript for publication. Publication of this article was not contingent upon approval by Eli Lilly and Company.

REFERENCES 1. Scott DL, Pugner K, Kaarela K, Doyle DV, Woolf A, Holmes J, et al. The links between joint damage and disability in rheumatoid arthritis. Rheumatology (Oxford) 2000;39:122–32. 2. Genovese MC, Bathon JM, Martin RW, Fleischmann RM, Tesser JR, Schiff MH, et al. Etanercept versus methotrexate in patients with early rheumatoid arthritis: two-year radiographic and clinical outcomes. Arthritis Rheum 2002;46:1443–50. 3. Finzel S, Rech J, Schmidt S, Engelke K, Englbrecht M, Stach C, et al. Repair of bone erosions in rheumatoid arthritis treated with tumour necrosis factor inhibitors is based on bone apposition at the base of the erosion. Ann Rheum Dis 2011;70:1587–93. 4. Finzel S, Rech J, Schmidt S, Engelke K, Englbrecht M, Schett G. Interleukin-6 receptor blockade induces limited repair of bone erosions in rheumatoid arthritis: a micro CT study. Ann Rheum Dis 2013;72:396–400. 5. Jarrett SJ, Conaghan PG, Sloan VS, Papanastasiou P, Ortmann CE, O’Connor PJ, et al. Preliminary evidence for a structural benefit of the new bisphosphonate zoledronic acid in early rheumatoid arthritis. Arthritis Rheum 2006;54:1410–4. 6. Cohen SB, Dore RK, Lane NE, Ory PA, Peterfy CG, Sharp JT, et al, on behalf of the Denosumab Rheumatoid Arthritis Study Group. Denosumab treatment effects on structural damage, bone mineral density, and bone turnover in rheumatoid arthritis: a

336

7.

8.

9. 10.

11.

12.

13.

DURYEA ET AL

twelve-month, multicenter, randomized, double-blind, placebo-controlled, phase II clinical trial. Arthritis Rheum 2008;58:1299–309. Sharp JT, Young DY, Bluhm GB, Brook A, Brower AC, Corbett M, et al. How many joints in the hands and wrists should be included in a score of radiologic abnormalities used to assess rheumatoid arthritis? Arthritis Rheum 1985;28:1326–35. Van der Heijde DM, van Leeuwen MA, van Riel PL, Koster AM, van ’t Hof MA, van Rijswijk MH, et al. Biannual radiographic assessments of hands and feet in a three-year prospective followup of patients with early rheumatoid arthritis. Arthritis Rheum 1992;35:26–34. James MF, Heald G, Shorter JH, Turner RA. Joint space measurement in hand radiographs using computerized image analysis. Arthritis Rheum 1995;38:891–901. Angwin J, Lloyd A, Heald G, Nepom G, Binks M, James MF. Radiographic hand joint space width assessed by computer is a sensitive measure of change in early rheumatoid arthritis. J Rheumatol 2004;31:1050–61. Finckh A, de Pablo P, Katz JN, Neumann G, Lu Y, Wolfe F, et al. Performance of an automated computer-based scoring method to assess joint space narrowing in rheumatoid arthritis: a longitudinal study. Arthritis Rheum 2006;54:1444–50. Sharp JT, Angwin J, Boers M, Duryea J, Finckh A, Hall JR, et al. Multiple computer-based methods of measuring joint space width can discriminate between treatment arms in the COBRA trial: update of an ongoing OMERACT project. J Rheumatol 2009;36:1825–8. Sharp JT, Gardner JC, Bennett EM. Computer-based methods for measuring joint space and estimating erosion volume in the

14.

15.

16.

17.

18.

19.

finger and wrist joints of patients with rheumatoid arthritis. Arthritis Rheum 2000;43:1378–86. Conaghan P, Bird P, Ejbjerg B, O’Connor P, Peterfy C, McQueen F, et al. The EULAR-OMERACT rheumatoid arthritis MRI reference image atlas: the metacarpophalangeal joints. Ann Rheum Dis 2005;64 Suppl 1:i11–21. Dohn UM, Ejbjerg BJ, Hasselquist M, Narvestad E, CourtPayen M, Szkudlarek M, et al. Rheumatoid arthritis bone erosion volumes on CT and MRI: reliability and correlations with erosion scores on CT, MRI and radiography. Ann Rheum Dis 2007;66:1388–92. Duryea J, Magalnick M, Alli S, Yao L, Wilson M, GoldbachMansky R. Semiautomated three-dimensional segmentation software to quantify carpal bone volume changes on wrist CT scans for arthritis assessment. Med Phys 2008;35:2321–30. Salaffi F, Carotti M, Ciapetti A, Ariani A, Gasparini S, Grassi W. Validity of a computer-assisted manual segmentation software to quantify wrist erosion volume using computed tomography scans in rheumatoid arthritis. BMC Musculoskelet Disord 2013;14:265. Aletaha D, Neogi T, Silman AJ, Funovits J, Felson DT, Bingham CO III, et al. 2010 rheumatoid arthritis classification criteria: an American College of Rheumatology/European League Against Rheumatism collaborative initiative. Arthritis Rheum 2010;62: 2569–81. Brown H, Prescott R. Applied mixed models in medicine. Chichester: John Wiley & Sons; 2006.