GIRADS reporting system for ultrasound ... - Wiley Online Library

Ultrasound Obstet Gynecol 2011; 38: 450–455 Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/uog.9012

GI-RADS reporting system for ultrasound evaluation of adnexal masses in clinical practice: a prospective multicenter study ´ ´ F. AMOR*, J. L. ALCAZAR†, H. VACCARO*, M. LEON‡ and A. ITURRA§ *Centro Ecografico Ultrasonic Panoramico, Santiago, Chile; †Department of Obstetrics and Gynecology, Clinica Universidad de Navarra, University of Navarra, Pamplona, Spain; ‡Department of Obstetrics and Gynecology, Cl´ınica Las Lilas, Santiago, Chile; §Department of Obstetrics and Gynecology, Cl´ınica Indisa, Santiago, Chile

K E Y W O R D S: adnexal masses; ovarian cancer; reporting; ultrasound

ABSTRACT Objective To assess the clinical usefulness of a structured reporting system based on ultrasound findings for management of adnexal masses. Methods This was a prospective multicenter study comprising 432 adnexal masses in 372 women (mean age, 44.0 (range, 13–78) years) over a 36-month period. Ninetythree (25%) women were postmenopausal and 279 (75%) women were premenopausal. Patients were evaluated with transvaginal ultrasound by one of three examiners expert in gynecological ultrasound. Reporting was provided to referring clinicians according to the Gynecologic Imaging Report and Data System (GI-RADS) classification. A predetermined management protocol was offered to referral clinicians. It was suggested that patients classified as GI-RADS 2 be managed with follow-up scan, patients classified as GI-RADS 3 undergo laparoscopic surgery and patients classified as GI-RADS 4 or 5 be referred to a gynecologic oncologist. Definitive histologic diagnosis was available in 370 cases and 62 additional cases were considered as benign because of spontaneous resolution during follow-up. These outcomes were used as the gold standard for calculating the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+) and negative likelihood ratio (LR−) of GI-RADS classification for identifying adnexal masses at high risk of malignancy, considering GI-RADS 4 and 5 as being malignant. Results Of the 432 tumors, 112 were malignant and 320 benign. The GI-RADS classification rate was as follows: GI-RADS 2, 92 (21%) cases; GI-RADS 3, 184 (43%) cases; GI-RADS 4, 40 (9%) cases; GI-RADS 5, (27%) 116 cases. Sensitivity for this system was 99.1% (95% CI, 95.1–99.8%), specificity was 85.9% (95% CI,

81.7–89.3%), LR+ was 7.05 (95% CI, 5.37–9.45) and LR− was 0.01 (95% CI, 0.001–0.07). PPV and NPV were 71.1% and 99.6%, respectively. Conclusions The GI-RADS reporting system performed well in identifying adnexal masses at high risk of malignancy and seems to be useful for clinical decisionmaking. Copyright  2011 ISUOG. Published by John Wiley & Sons, Ltd.

INTRODUCTION Ultrasonography is currently considered as the primary imaging modality for identifying and characterizing adnexal masses1 . Several approaches have been proposed for their characterization using this technique, including examiner’s subjective impression2 , simple descriptive scoring systems3 , mathematically developed scoring systems4 , logistic regression models5 and neural networks6 . Subjective impression of an experienced examiner is currently believed to be the best approach and no other method has been proven its superior7,8 . However, the examiner’s impression is entirely subjective and recent evidence has shown that this fact affects not only the performance of the method itself9 , but also the examiner’s confidence in providing a diagnosis10 . Furthermore, a recent randomized study demonstrated that examiner experience affects performance and decision-making in clinical practice11 . Due to the subjective nature of the examiner’s impression there is a need for a standardized nomenclature and definition for all tumor features evaluated by ultrasound. This was provided by the International Ovarian Tumor Analysis (IOTA) consensus12 . Undoubtedly, this

´ Correspondence to: Dr J. L. Alcazar, Department of Obstetrics and Gynecology, Clinica Universidad de Navarra, Avenida Pio XII, 36, 31008 Pamplona, Spain (e-mail: [email protected]) Accepted: 17 March 2011

Copyright  2011 ISUOG. Published by John Wiley & Sons, Ltd.

ORIGINAL PAPER

GI-RADS reporting of adnexal masses

451

consensus has allowed a better, homogeneous description of adnexal masses. However, there is still significant variation in the reporting of ultrasound examination results for adnexal masses13 . In fact, a recent consensus conference of the Society of Radiologists in Ultrasound concluded that ‘investigation into structured reporting of adnexal cysts to allow for improved communication of results and recommendations for follow-up’ is needed14 . In 2009 we proposed a reporting system similar to that used for breast ultrasound (BI-RADS): the Gynecology Imaging Reporting and Data System (GI-RADS), developed to facilitate communication between sonologists/sonographers and referring clinicians15 . This GIRADS classification is based on ultrasound findings, representing a summarized standardized report of those findings and also providing an estimated risk of malignancy for a given adnexal mass. The aim of this study was to assess prospectively the use of this reporting system for decision-making in clinical practice.

SUBJECTS AND METHODS This was a prospective study comprising all women diagnosed as having an adnexal mass and evaluated at two different centers, one in Spain (Clinica Universidad de Navarra, Pamplona) and one in Chile (Centro Ecografico Ultrasonic Panoramico, Santiago), from January 2008 to December 2010. Institutional review board approval was obtained and all women gave informed consent to participate. All patients were evaluated by transvaginal or transrectal (in cases of virgo-intacta women) ultrasound using a Voluson 730 Expert or Pro machine (GE Medical Systems, Zipf, Austria) according to a predetermined scanning protocol15 . Three expert examiners (F.A., H.V. and J.L.A.), each with more than 15 years’ experience in gynecological ultrasound, performed all examinations and between one and five representative images were stored on the machine’s database, to be used in the report (Figure S1 online). Reporting was performed according to GI-RADS classification15 . This system is based on pattern recognition analysis and provides an a priori risk estimation of

probability of malignancy, based on data from previous studies15 – 17 . The reporting system includes five categories (Table 1) and the report includes a description of the mass as well as a final GI-RADS classification (Figure S1). During the examination, tumor volume was also estimated according to the prolate ellipsoid formula (length × width × height × 0.5233, expressed in mL), but this feature was not taken into consideration for assigning a GI-RADS classification. The meaning and goal of GI-RADS classification was explained to referring clinicians in several clinical sessions before the study started. A management protocol was offered to referral clinicians with the aim of determining whether this reporting system could be useful for deciding patient management and in avoiding confusion for clinicians. However, while we followed up patients to determine how they were managed ultimately, we were not involved in clinical decision-making. The suggested management protocol was based on risk of malignancy as estimated by GI-RADS classification. Those patients classified as GI-RADS 1 (e.g. normal ovaries at ultrasound) were excluded from the study and from further analysis. GI-RADS 2 patients were considered for expectant management by follow-up sonography on the basis that these lesions were assumed to be functional. GI-RADS 3 patients underwent surgery by general gynecologists on the basis that these lesions were considered to be probably benign and expected to persist over time. Laparoscopy was preferable, although the surgeon managing the patient made the final decision regarding surgical approach (laparoscopy or laparotomy). Patients classified as GI-RADS 4 and 5 were referred to gynecological oncologists for appropriate additional imaging techniques (computed tomography or magnetic resonance imaging) and surgical management, on the basis that these lesions were considered to be probably or very probably malignant. When surgical removal of the tumor was performed, a definitive histologic diagnosis was obtained. Tumors were classified according to World Health Organization criteria18 and malignant tumors were staged according to FIGO criteria19 . Borderline tumors were considered as

Table 1 Gynecologic Imaging Report and Data System (GI-RADS) classification system for adnexal masses GI-RADS grade

Diagnosis

Est. prob. malignancy

1 2

Definitive benign Very probably benign

0% < 1%

3

Probably benign

1–4%

4

Probably malignant

5–20%

5

Very probably malignant

> 20%

Detail Normal ovaries identified and no adnexal mass seen Adnexal lesions thought to be of functional origin, e.g. follicles, corpora lutea, hemorrhagic cysts Neoplastic adnexal lesions thought to be benign, such as endometrioma, teratoma, simple cyst, hydrosalpinx, paraovarian cyst, peritoneal pseudocyst, pedunculated myoma, or findings suggestive of pelvic inflammatory disease Any adnexal lesion not included in GI-RADS 1–3 and with one or two findings suggestive of malignancy* Adnexal masses with three or more findings suggestive of malignancy*

*Thick papillary projections, thick septations, solid areas and/or ascites, defined according to IOTA criteria12 , and vascularization within solid areas, papillary projections or central area of a solid tumor on color or power Doppler assessment5 . Est. prob., estimated probability.


Ultrasound Obstet Gynecol 2011; 38: 450–455.

Amor et al.

452

malignant for analytic purposes. STARD guidelines were followed for designing and conducting the study20 .

Statistical analysis Categorical variables were compared using the chisquare test and tumor volumes were compared using the Mann–Whitney U-test. We calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+) and negative likelihood ratio (LR−) of the GI-RADS system for identifying adnexal masses at high risk of malignancy, considering GI-RADS 2 and 3 as low risk and GI-RADS 4 and 5 as high risk. The gold standard was histologic diagnosis (benign or malignant) or spontaneous resolution of the cyst during follow-up (benign). To determine how useful they found the GI-RADS reporting system for understanding ultrasound findings and for making decisions regarding patient management, referral clinicians involved in patient clinical decisionmaking were asked to complete a simple survey. This survey consisted of a single question: ‘How useful do you think GI-RADS reporting system is for understanding ultrasound findings and giving confidence in clinical decisions regarding your patient?’ and there were five possible answers: (A) totally useful; (B) quite useful; (C) neither useful nor useless; (D) useless; (E) completely useless. To assess interobserver reproducibility of GI-RADS classification, two examiners (J.L.A. and A.I.) performed a separate analysis in 60 consecutive women who were already included in the study. Both examiners performed a transvaginal scan, blinded to each other’s results, and each one provided a GI-RADS report. To determine the concordance between examiners we used a weighted Kappa index.

RESULTS A total of 372 women with adnexal masses were included in this study (279 from the Cl´ınica Universidad de Navarra ´ ´ ´ and 93 from Centro Ecografico Ultrasonic Panoramico). Their mean age was 44 (range, 13–78) years. Ninetythree (25%) women were postmenopausal and 279 (75%) were premenopausal. Sixty (16%) patients had bilateral tumors, giving a total number of 432 adnexal masses assessed. The prevalence of malignant tumors was 26% (112 malignant tumors in 87 patients). Malignant tumors were more frequent in postmenopausal women (43.2%) than in premenopausal women (13.2%) (P < 0.001). Of the 432 masses assessed, 92 (21%) were classified as GI-RADS 2, 184 (43%) as GI-RADS 3, 40 (9%) as GI-RADS 4 and 116 (27%) as GI-RADS 5. Tumor volume was significantly smaller in GI-RADS 2 and 3 cases compared with GI-RADS 4 and 5 cases, while there was no difference in tumor volume between GI-RADS 2 and 3 cases or between GI-RADS 4 and 5 cases (Table 2). Most referring clinicians managed their patients according to GI-RADS classification. Figure 1 summarizes the classifications, management and final outcomes of the study population, and final histological diagnoses are given in Table 3. There was no malignant tumor classified as GI-RADS 2. There was one such case classified as GI-RADS 3; this false-negative case was a 73-year-old woman with a 580 mL cyst diagnosed as benign serous cyst, but histology showed it to be a serous ovarian carcinoma, Stage Ia. The sensitivity for the GI-RADS reporting system in predicting malignancy was 99.1% (95% CI, 95.1–99.8%), specificity was 85.9% (95% CI, 81.7–89.3%), LR+ was 7.05 (95% CI, 5.37–9.45) and LR− was 0.01 (95% CI, 432 masses 372 women

GI-RADS 2 92 masses 88 women

Follow-up 86 masses

Spontaneous resolution 71 masses

Persistence 15 masses

Surgery for pain symptoms 6 masses

Histology benign 6 masses


Follow-up 2 masses

Surgery: general gynecologist 182 masses

Spontaneous Histology Histology resolution malignant benign 2 masses 1 mass 181 masses


Surgery: general gynecologist 3 masses

Histology benign 3 masses

Referral: gynecologic oncologist 37 masses


Referral: gynecologic oncologist 116 masses

Histology Histology Histology Histology malignant malignant benign benign 29 masses 8 masses 13 masses 103 masses

Surgery 15 masses Histology benign 15 masses

Figure 1 Flow chart showing classification by Gynecologic Imaging Report and Data System (GI-RADS), management and final outcome of the study group of 372 women with 432 adnexal masses.



GI-RADS reporting of adnexal masses

453

Table 2 Tumor volume according to Gynecologic Imaging Report and Data System (GI-RADS) in 372 women with 432 adnexal masses

Table 4 Diagnostic performance of Gynecologic Imaging Report and Data System (GI-RADS) reporting system in 372 women with 432 adnexal masses

Tumor volume (mL)

2a

GI-RADS GI-RADS 3b GI-RADS 4c GI-RADS 5d

Number of tumors classified as:

Median

Interquartile range

Range

32.0 52.6 96.3 106.0

16.7–72.0 25.1–117.1 38.9–302.9 41.9–326.4

3.9–170.0 1.6–2677.7 6.9–2237.1 2.1–3728.9

Final diagnosis

a vs b: P = 0.130; a vs c: P = 0.01; a vs d: P = 0.001; b vs c: P = 0.028; b vs d: P = 0.0001; c vs d: P = 0.684.

0.001–0.07) (Table 4). The PPV and NPV were 71.1% and 99.6%, respectively. All fifteen (six in Spain and nine in Chile) referring clinicians considered this reporting system to be ‘quite useful’ or ‘useful’ for clinical decision-making in adnexal masses. The interobserver agreement for GI-RADS classification of adnexal masses was very good (weighted kappa index = 0.846) (Table 5).

DISCUSSION Reporting in ultrasound evaluation of adnexal masses is an important issue. A recent study from Canada has shown that current reporting practices for ultrasound assessments in women with ovarian masses vary considerably and concluded that the use of a synoptic reporting system would be useful13 . Inappropriate reporting may lead to unwarranted concern by the patient and referring

GI-RADS 2–3

GI-RADS 4–5

Total

1 275 276

111 45 156

112 320 432

Malignant Benign Total

clinician and could lead to unnecessary additional tests and surgery21 . In fact, investigation into structured reporting of adnexal masses to allow for improved communication of results and recommendations for management has been advised recently14 . For this reason we recently developed a simple reporting system based on the concept developed for breast imaging (the BI-RADS classification), which was originally developed for mammographic findings but has been applied successfully to breast ultrasound. As for BI-RADS, the lexicon of our new system is intended to provide a unified language for ultrasound reporting and to avoid confusion in the communication between the sonographer/sonologist and the clinician. We called this reporting system GI-RADS15 . In the present study we assessed prospectively the use of our GI-RADS reporting system for ultrasound evaluation of adnexal masses and clinical decision-making. A strength of the study is that the ultrasound examiners were not involved in the decision-making process. The GI-RADS reporting system is based on the use of pattern recognition analysis of the tumor2 and the

Table 3 Gynecologic Imaging Report and Data System (GI-RADS) classification according to specific histologic diagnosis in 372 women with 432 adnexal masses Number of tumors classified as: Histologic diagnosis Functional cyst* Serous cystadenoma Mucinous cystadenoma Endometrioma Teratoma Paraovarian cyst Hemorrhagic cyst Cystadenofibroma Peritoneal cyst Fibroma Hydrosalpinx Tubo-ovarian abscess Leiomyoma Brenner tumor Low malignant potential tumor Primary ovarian cancer Metastatic cancer Total

GI-RADS 2

GI-RADS 3

GI-RADS 4

GI-RADS 5

Total

71 5 0 6 0 0 9† 1† 0 0 0 0 0 0 0 0 0 92

2 36 10 78 28 3 2 5 3 3 9 2 0 2 0 1 0 184

0 7 4 2 4 0 0 3‡ 2 6 2‡ 0 1 1 2 5 1 40

0 0 3 0 0 0 0 2 0 3 1 1 1 2 12 75 16 116

73 48 17 86 32 3 11 11 5 12 12 3 2 5 14 81 17 432

*Spontaneous resolution at follow-up. †Five hemorrhagic cysts and the cystadenofibroma comprised the six GI-RADS 2 cases which underwent surgery following diagnosis due to pain symptoms. ‡Two hydrosalpinges and one serous cystadenofibroma comprised the GI-RADS 4 cases which underwent laparoscopic surgery by a general gynecologist.



Amor et al.

454

Table 5 Agreement analysis between two observers for assigning Gynecologic Imaging Report and Data System (GI-RADS) classification in 60 women with unilateral adnexal masses Examiner A Examiner B

GI-RADS 2

GI-RADS 3

GI-RADS 4

GI-RADS 5

Total

GI-RADS 2 GI-RADS 3 GI-RADS 4 GI-RADS 5 Total

10 1 0 0 11

1 21 2 0 24

0 3 9 2 14

0 0 0 11 11

11 25 11 13 60

Data are given as number of tumors.

a-priori risk of malignancy of different tumor features15 – 17 . Although one could argue that pattern recognition is a subjective assessment, there is evidence that this is the best method for characterizing adnexal masses7,8 and that pattern recognition is reproducible among expert examiners22 – 24 . In terms of diagnostic performance, this reporting system performed well, with a very high sensitivity and acceptable specificity. This is not surprising bearing in mind that it is based on IOTA criteria, which have been tested extensively in several multicenter studies and shown to be good criteria for discriminating between benign and malignant adnexal masses25 – 27 . However, one possible selection bias in our study is the relatively high prevalence of malignant tumors, which could affect estimation of sensitivity and specificity. Notwithstanding, both PPV and NPV were high and these figures are not affected by disease prevalence. Our data have shown that the GI-RADS classification system is useful for clinical decision-making and referral. Furthermore, all referring clinicians involved in patient management considered it to be useful. We therefore propose a standardized nomenclature for reporting ultrasound findings of adnexal masses, applying the same rationale as that of BI-RADS classification for breast ultrasound. While it is true that adequate referral may be achieved using logistic models such as the risk malignancy index28,29 , scoring systems30 or just pattern recognition analysis as does IOTA31 , a standardized reporting nomenclature is lacking. To the best of our knowledge, this is the first such standardized reporting/classification system applicable to adnexal masses. It is likely that this reporting system would not be needed in those institutions where ultrasound examiners and clinicians participating in clinical decision-making have good and direct communication and decisions about patient management are collegiate, or even in those practices where expert sonologists themselves decide about their own patients’ management. However, this system could be useful in those settings in which clinicians managing patients do not perform ultrasound examinations, instead reading the report of the morphological description of the tumor. It could also be useful for small hospitals and for private practitioner-gynecologists who must refer patients with suspicious masses to tertiary care hospitals with gynecologic oncology facilities.


There were some limitations to the study. A possible bias is that expert examiners performed all ultrasound examinations; this is known to potentially affect diagnostic performance when using pattern recognition analysis9,32 . Therefore, further research into how this reporting system performs when used by non-expert examiners is needed. Another bias of this study is that a management protocol according to GI-RADS classification was offered to referral clinicians before starting the study. This could have biased their decision as to how to manage the patients. An interesting issue regarding the suggested management protocol is the use of surgery in cases of GI-RADS 3. In fact, expectant management could also be offered safely to these patients33 . A further weakness of this study is the fact that most GI-RADS 4 lesions were benign, although they were classified as being probably malignant. However, there was still a 20% risk of malignancy (8/40). One option for improving the predictive value of this group would be further classification into subgroups depending on degree of likelihood of malignancy according to the examiner’s impression. In conclusion, this prospective study has shown that GI-RADS classification performs well as a reporting system in adnexal masses and it seems to be useful for clinical decision-making.

REFERENCES 1. ACOG Practice Bulletin. Management of adnexal masses. American College of Obstetricians and Gynecologists. Obstet Gynecol 2007; 110: 201–214. 2. Valentin L. Pattern recognition of pelvic masses by gray-scale ultrasound imaging: the contribution of Doppler ultrasound. Ultrasound Obstet Gynecol 1999; 14: 338–347. 3. Granberg S, Wikland M, Jansson I. Macroscopic characterization of ovarian tumors and the relation to the histological diagnosis: criteria to be used for ultrasound evaluation. Gynecol Oncol 1989; 35: 139–144. ´ ´ 4. Alcazar JL, Mercé LT, Laparte C, Jurado M, Lopez-Garc ıá G. A new scoring system to differentiate benign from malignant adnexal masses. Am J Obstet Gynecol 2003; 188: 685–692. ´ ´ 5. Alcazar JL, Errasti T, Laparte C, Jurado M, Lopez-Garc ´ıa G. Assessment of a new logistic model in the preoperative evaluation of adnexal masses. J Ultrasound Med 2001; 20: 841–848. 6. Timmerman D, Verrelst H, Bourne TH, De Moor B, Collins WP, Vergote I, Vandewalle J. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol 1999; 13: 17–25.


GI-RADS reporting of adnexal masses 7. Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of ‘pattern recognition’ and logistic regression models for discrimination between benign and malignant pelvic masses: a prospective cross validation. Ultrasound Obstet Gynecol 2001; 18: 357–365. 8. Timmerman D. The use of mathematical models to evaluate pelvic masses; can they beat an expert operator? Best Pract Res Clin Obstet Gynaecol 2004; 18: 91–104. 9. Van Holsbeke C, Daemen A, Yazbek J, Holland TK, Bourne T, Mesens T, Lannoo L, Boes AS, Joos A, Van De Vijver A, Roggen N, de Moor B, de Jonge E, Testa AC, Valentin L, Jurkovic D, Timmerman D. Ultrasound experience substantially impacts on diagnostic performance and confidence when adnexal masses are classified using pattern recognition. Gynecol Obstet Invest 2010; 69: 160–168. 10. Yazbek J, Ameye L, Testa AC, Valentin L, Timmerman D, Holland TK, Van Holsbeke C, Jurkovic D. Confidence of expert ultrasound operators in making a diagnosis of adnexal tumor: effect on diagnostic accuracy and interobserver agreement. Ultrasound Obstet Gynecol 2010; 35: 89–93. 11. Yazbek J, Raju SK, Ben-Nagi J, Holland TK, Hillaby K, Jurkovic D. Effect of quality of gynaecological ultrasonography on management of patients with suspected ovarian cancer: a randomised controlled trial. Lancet Oncol 2008; 9: 124–131. 12. Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I; International Ovarian Tumor Analysis (IOTA) Group. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the International Ovarian Tumor Analysis (IOTA) Group. Ultrasound Obstet Gynecol 2000; 16: 500–505. 13. Le T, Fayadh RA, Menard C, Hicks-Boucher W, Faught W, Hopkins L, Fung-Kee-Fung M. Variations in ultrasound reporting on patients referred for investigation of ovarian masses. J Obstet Gynaecol Can 2008; 30: 902–906. 14. Levine D, Brown DL, Andreotti RF, Benacerraf B, Benson CB, Brewster WR, Coleman B, Depriest P, Doubilet PM, Goldstein SR, Hamper UM, Hecht JL, Horrow M, Hur HC, Marnach M, Patel MD, Platt LD, Puscheck E, Smith-Bindman R. Management of asymptomatic ovarian and other adnexal cysts imaged at US: Society of Radiologists in Ultrasound Consensus Conference Statement. Radiology 2010; 256: 943–954. ´ ´ M, Craig JM, Martinez J. 15. Amor F, Vaccaro H, Alcazar JL, Leon Gynecologic imaging reporting and data system: a new proposal for classifying adnexal masses on the basis of sonographic findings. J Ultrasound Med 2009; 28: 285–291. 16. Guerriero S, Ajossa S, Garau N, Piras B, Paoletti AM, Melis GB. Ultrasonography and color Doppler-based triage for adnexal masses to provide the most appropriate surgical approach. Am J Obstet Gynecol 2005; 192: 401–406. ´ 17. Alcazar JL, Royo P, Jurado M, M´ınguez JA, Garc´ıa-Manero M, ´ R, Lopez-Garc ´ Laparte C, Galvan ´ıa G. Triage for surgical management of ovarian tumors in asymptomatic women: assessment of an ultrasound-based scoring system. Ultrasound Obstet Gynecol 2008; 32: 220–225. 18. Scully RE, Sobin LH. WHO histological classification of ovarian tumors. World Health Organization: Geneva, 1999. 19. Heintz AP, Odicino F, Maisonneuve P, Quinn MA, Benedet JL, Creasman WT, Ngan HY, Pecorelli S, Beller U. Carcinoma of the ovary. FIGO 6th Annual Report on the Results of Treatment in Gynecological Cancer. Int J Gynaecol Obstet 2006; 95: S161–S192. 20. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP,

455

21. 22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC. Standards for Reporting of Diagnostic Accuracy. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Clin Radiol 2003; 58: 575–580. Brown DL, Dudiak KM, Laing FC. Adnexal masses: US characterization and reporting. Radiology. 2010; 254: 342–354. ¨ Timmerman D, Schwarzler P, Collins WP, Claerhout F, Coenen M, Amant F, Vergote I, Bourne TH. Subjective assessment of adnexal masses with the use of ultrasonography: an analysis of interobserver variability and experience. Ultrasound Obstet Gynecol 1999; 13: 11–16. Guerriero S, Alcazar JL, Pascual MA, Ajossa S, Gerada M, Bargellini R, Virgilio B, Melis GB. Intraobserver and interobserver agreement of grayscale typical ultrasonographic patterns for the diagnosis of ovarian cancer. Ultrasound Med Biol 2008; 34: 1711–1716. Guerriero S, Alcazar JL, Pascual MA, Ajossa S, Gerada M, Bargellini R, Virgilio B, Melis GB. Diagnosis of the most frequent benign ovarian cysts: is ultrasonography accurate and reproducible? J Womens Health 2009; 18: 519–527. Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of ‘pattern recognition’ and logistic regression models for discrimination between benign and malignant pelvic masses: a prospective cross validation. Ultrasound Obstet Gynecol 2001; 18: 357–365. Valentin L, Ameye L, Jurkovic D, Metzger U, Lécuru F, Van Huffel S, Timmerman D. Which extrauterine pelvic masses are difficult to correctly classify as benign or malignant on the basis of ultrasound findings and is there a way of making a correct diagnosis? Ultrasound Obstet Gynecol 2006; 27: 438–444. Sokalska A, Timmerman D, Testa AC, Van Holsbeke C, Lissoni AA, Leone FP, Jurkovic D, Valentin L. Diagnostic accuracy of transvaginal ultrasound examination for assigning a specific diagnosis to adnexal masses. Ultrasound Obstet Gynecol 2009; 34: 462–470. van den Akker PA, Aalders AL, Snijders MP, Kluivers KB, Samlal RA, Vollebergh JH, Massuger LF. Evaluation of the Risk of Malignancy Index in daily clinical management of adnexal masses. Gynecol Oncol 2010; 116: 384–388. Raza A, Mould T, Wilson M, Burnell M, Bernhardt L. Increasing the effectiveness of referral of ovarian masses from cancer unit to cancer center by using a higher referral value of the risk of malignancy index. Int J Gynecol Cancer 2010; 20: 552–554. ´ Alcazar JL, Royo P, Jurado M, M´ınguez JA, Garc´ıa-Manero M, ´ R, Lopez-Garc ´ Laparte C, Galvan ıá G. Triage for surgical management of ovarian tumors in asymptomatic women: assessment of an ultrasound-based scoring system. Ultrasound Obstet Gynecol 2008; 32: 220–225. Yazbek J, Helmy S, Ben-Nagi J, Holland T, Sawyer E, Jurkovic D. Value of preoperative ultrasound examination in the selection of women with adnexal masses for laparoscopic surgery. Ultrasound Obstet Gynecol 2007; 30: 883–888. Van Holsbeke C, Daemen A, Yazbek J, Holland TK, Bourne T, Mesens T, Lannoo L, De Moor B, De Jonge E, Testa AC, Valentin L, Jurkovic D, Timmerman D. Ultrasound methods to distinguish between malignant and benign adnexal masses in the hands of examiners with different levels of experience. Ultrasound Obstet Gynecol 2009; 34: 454–461. ´ Alcazar JL, Castillo G, Jurado M, Garc´ıa GL. Is expectant management of sonographically benign adnexal cysts an option in selected asymptomatic premenopausal women? Hum Reprod 2005; 20: 3231–3234.

SUPPORTING INFORMATION ON THE INTERNET The following supporting information may be found in the online version of this article: Figure S1 Sample report for the Gynecologic Imaging Report and Data System (GI-RADS) classification.

