Indian J Med Res 131, Januray 2010, pp 88-91
Interobserver variation in the interpretation of Nugent scoring method for diagnosis of bacterial vaginosis Srujana Mohanty, Seema Sood, Arti Kapil & Suneeta Mittal*
Departments of Microbiology & *Obstetrics & Gynaecology, All India Institute of Medical Sciences New Delhi, India
Received November 18, 2008 Background & objectives: Bacterial vaginosis (BV) is the most common form of vaginal infection and an important cause of morbidity in women of reproductive age. This study was carried out to examine the interobserver variation on interpretation of Nugent scoring method in the diagnosis of BV. Methods: This prospective study was conducted in a rural primary health care center of north India from May 2003 to April 2004 and included 601 married, sexually active women between 18-49 yr of age presented with self-reported symptoms of vaginal discharge and/or genital itching and/or genital burning. Specimens collected from the lateral wall of vagina were subjected to Gram staining and the microscope slide smears were examined by 3 independent observers. Each of the three observers scored and interpreted the slides for diagnosis of bacterial vaginosis using the Nugent method. Results: Complete agreement amongst the three observers was found in 76.2 per cent of cases. In 22.13 per cent cases, two observers were in agreement while interpretation of the slides were in complete disagreement only in 1.66 per cent of cases. The interrater reproducibility was found to be excellent between observers 1 and 3, while between observers 1 and 2, and 2 and 3 it was good to fair. Interpretation & conclusions: Nugent scoring system appears to be a reliable and convenient method for laboratory evaluation of cases of bacterial vaginosis. At the same time, one must be aware of the factors that might lead to discrepant results.
Key words Bacterial vaginosis - gram stain - interobserver variation - nugent scoring method
Bacterial vaginosis (BV), a clinical entity characterized by a change in vaginal ecology where the normal flora of lactobacillus morphotypes is replaced by a mixed microbial flora consisting of anaerobes and Gardneralla vaginalis, is considered to be the most common form of vaginal infection among women of reproductive age1. Clinically, malodorous vaginal discharge, especially a fishy
odour, is the most common symptom. Various studies have found the prevalence of BV to range from 15 to 30 per cent in non pregnant women and upto 50 per cent in pregnant women2. Clinical studies have demonstrated an association of BV with adverse pregnancy outcomes, upper genital tract infections such as pelvic inflammatory disease, endometritis, post-gynaecologic surgery infections, cervicitis, 88
MOHANTY et al: Nugent scoring method for bacterial vaginosis
urinary tract infections, cervical intraepithelial neoplasia, and increased risk of sexual acquisition of human immunodeficiency virus infection3,4. Given its high prevalence and association with obstetrical and gynaecological complications, detection of this entity in women is of paramount importance. Diagnosis of BV is confirmed clinically using the composite criteria described by Amsel et al5. These include a thin homogenous discharge, elevated vaginal pH above 4.5, release of amines on addition of 10 per cent potassium hydroxide to vaginal fluid and the presence of clue cells, of which three need to be present for the diagnosis of bacterial vaginosis. However, Amsel’s criteria may not be adequate to diagnose patients, as approximately 50 per cent of BV patients may be asymptomatic4. Further, there exists a continuum from normal lactobacillusdominated flora through “to severe BV” with an intermediate catergory which the Amsel’s criteria fail to recognize. Thus, grading the microbial flora seen in Gram-stained vaginal smears, especially the scoring criteria proposed by Nugent et al6, as an alternative method has become useful as a diagnostic tool. In this system, large Gram-positive rods (Lactobacillus morphotypes), small Gram-negative to Gram-variable rods (Gardneralla vaginalis and Bacteroides morphotypes) and curved Gramnegative rods (Mobiluncus species) are quantitated and a summation score is obtained. This method has been shown to be sensitive for diagnosing BV, however, observers have often felt it to be complex and time consuming for routine practice and doubts have been raised regarding its interpretation by different observers7. In this study, we aim to report our experience on the interobserver variation on interpretation of Nugent scoring method in the diagnosis of BV. Material & Methods This prospective study was conducted from May 2003 to April 2004 at a primary health care center in Ballabhgarh, a rural field practice area in north India, under the All India Institute of Medical Sciences (AIIMS), New Delhi. It was done as part of a larger study of evaluation of the use of syndromic management of reproductive tract infections. The ethical review committees of AIIMS, New Delhi; Thomas Jefferson University, Global Network for Perinatal and Reproductive Health (GNPRH), USA; and Population Council approved the project.
89
Patient population: The inclusion criteria for the study were married and sexually active women between 18-49 yr of age who presented to the health care center with self reported symptoms of vaginal discharge and/or genital itching and/or genital burning. Pregnant women, women with severe medical disorders requiring immediate referral to higher level of healthcare, women who were currently menstruating, never been sexually active, who had a hysterectomy, had taken a course of antibiotics within last 3 wk and who had been previously enrolled in this study were excluded. Written informed consent was obtained from each participant woman. The specific aetiology of the genitourinary symptoms including the detection of bacterial vaginosis was determined. A sterile, cotton tipped swab was used to collect specimen from the lateral wall of the vagina and a smear was prepared on a dry glass slide. After fixation, smears were Gram stained8, and examined under oilimmersion at ×1000 magnification. The microscope slide smears were examined by 3 independent observers, all being qualified microbiologists. At the beginning of the study, all the three observers had been trained with twenty known slides. Each of the three observers scored and interpreted the slides independently using the Nugent method6. Briefly, in this system, the three morphotypes (Lactobacillus, Bacteroides/Gardnerella & Mobiluncus spp.) were quantitated on a scale of 1 to 4 as follows: 1+ (< 1 cell per field), 2+ (1 to 5 cells per field), 3+ (6 to 30 cells per field) and 4+ (> 30 cells per field). Lactobacillus and Bacteroides/Gardnerella morphotypes were then assigned scores of 0-4 based on the quantitation whereas Mobiluncus morphotypes were assigned scores of 0-2. The total scores were then computed by adding the weighted quantitation (0 to 4+) of the three morphotypes. Nugent results were graded as 0-3 (normal vaginal state), 4-6 (intermediate grade bacterial colonization), and 7-10 (bacterial vaginosis) independently by each observer and the results for each slide smear were then compared. If all the three observers had the same interpretation, it was deemed to be the final diagnosis. In case of discrepancy, the slides were reviewed and a final diagnosis made after thorough discussion amongst the three observers. Statistical analysis: The weighted kappa statistic (STATA 9.0) was used to measure agreement about the Gram-stain interpretation between the observers. Calculated kappa values of < 0.4 are considered to reflect poor reproducibility or agreement, those of 0.4 to 0.75 reflect good to fair reproducibility or agreement, while > 0.75 reflect excellent agreement9.
90
INDIAN J MED RES, january 2010
Results & Discussion Of the total 710 women screened, only 611 patients were eligible for enrollment. Vaginal smears could be collected and analyzed from 601 women as the remaining declined internal examination. Thus, a total of 601 vaginal smears from the same number of women were included in the study and subjected to Nugent scoring using Gram stain examination. Complete agreement amongst the three observers was found in 76.2 per cent of cases i.e., in the 458 slides with agreement on diagnosis, all investigators scored the same group normal, intermediary or BV. In 22.13 per cent cases, two observers were in agreement while interpretation of the slides were in complete disagreement only in 1.66 per cent of cases (Table I). A final diagnosis of bacterial vaginosis was made in 121 (20.13%) women. According to the criteria for kappa (Table II), the combined inter-rater reliability amongst the three observers was good to fair, (almost approaching excellent) in our setting. In fact, the interrater reproducibility was found to be excellent between observers 1 and 3, while between observers 1 and 2, and 2 and 3 it was good to fair. Table I. Interobserver variation among three observers when interpreting vaginal slide smears by Nugent scoring method Parameter 1
2 3
Complete agreement ; i.e.; interpreted by all three as (i) Negative or normal vaginal state (ii) Intermediate grade (iii) Positive or BV Complete disagreement; i.e., interpreted by all three as different Partial agreement, i.e., interpreted similarly by two and differently by third (i) negative by 2, intermediate by 1 (ii) negative by 2, positive by 1 (iii) intermediate by 2, negative by 1 (iv) intermediate by 2, positive by 1 (v) positive by 2, negative by 1 (vi) positive by 2, intermediate by 1
No. of slides (%) (n=601) 458 (76.2) 341 (56.7) 30 (4.9) 86 (14.3) 10 (1.66) 133 (22.13) 41 (6.8) 6 (1 ) 39 (6.5) 17 (2.8) 4 (0.6) 26 (4.3)
Table II. Per cent agreement and kappa statistics for Gram stained smears (n= 601) Observer
% agreement
Kappa*
1 versus 2
82.75
0.70
2 versus 3
82.0
0.72
1 versus 3
85.0
0.77
1, 2 & 3 combined
76.2
0.73
*
For all values, P< 0.001
Originally, the standard method for diagnosis of BV was culture of vaginal specimens for G. vaginalis10. However, with the recognition that BV is associated with a diverse group of organisms, many of which are difficult, cumbersome and costly to culture in the laboratory, it became increasingly more common to process these specimens by Gram stain alone, without culture11. Other tools for diagnosis of BV in recent times include the use of rapid diagnostic kits such as the FemExam12 or the BVBlue test13. However, these suffer from the limitations of being costly and non specific diagnostic indicators as compared to the Gram stain. Grading the microbial flora seen in Gramstained vaginal smears for diagnosing BV was first described by Spiegel et al14, but was found to be only moderately reliable because of the wide variability in recognition of bacterial morphotypes by Gram smear evaluators. Nugent et al6 by modifying this method to a semi-quantitative scoring system provided a more standardized method of Gram stain interpretation for BV. In India, the prevalence of BV in ever-married women of reproductive age group has been reported to vary from 20.5 per cent15 to 48.5 per cent16 using the Gram stain method. However, the reproducibility with which Gram-stained slides are interpreted by different observers has not been well documented in our setting. Two previous studies17,18 from outside India on reproducibility of Gram-stained vaginal smears for diagnosis of BV had shown excellent to near-perfect agreement. Similarly, in another international study19 on interpretation of vaginal smears for diagnosis of BV, inter-observer reproducibility of Nugent scores for the diagnosis of BV was shown to be high. The authors19 discussed some issues that needed to be looked at carefully. First, they found major discrepancies in scoring when the lactobacilli morphotypes were few in number which may be influenced by the method of fixation, different sampling devices, methods to actually collect the specimen, variance in site of the vagina from where the sample is collected, and variance in the homogeneity and thickness of the sample while spreading on glass. Secondly, there may be disagreement regarding which morphotypes should be considered to be Gram-positive rods and thus scored as the Lactobacillus morphotype. A frequently occurring staining phenomenon is the tendency of the old lactobacilli to lose their Grampositive appearance. Small bacteria morphotypes (Gardneralla and Prevotella) may vary in size and form from round to more elongated forms and it may be difficult to differentiate them from lactobacilli
MOHANTY et al: Nugent scoring method for bacterial vaginosis
morphotypes. According to Dunkelberg20, majority of disagreements regarding diagnosis of BV are due to falsely denoting diphtheroids as Haemophilus vaginalis (now Gardneralla vaginalis). Some or all of these factors may also have contributed to discrepancies amongst the observers in the present study thus giving a combined kappa statistic of good to fair though excellent agreement was also observed between two observers. Thus Nugent scoring system appears to be a reliable, convenient and cost-effective method for laboratory evaluation of cases of bacterial vaginosis. At the same time, one must be aware of the various factors that might lead to discrepant results. References 1.
Sobel JD. Bacterial vaginosis. Annu Rev Med 2000; 51: 34956.
2.
Nelson DB, Macones G. Bacterial vaginosis in pregnancy: current findings and future directions. Epidemiol Rev 2002; 24 : 102-8.
3.
Cauci S, Driussi S, De Santo D, Penacchioni P, Iannicelli T, Lanzafame P, et al. Prevalence of bacterial vaginosis and vaginal flora changes in peri- and postmenopausal women. J Clin Microbiol 2002; 40 : 2147-52.
4.
McGregor JA, French JI. Bacterial vaginosis in pregnancy. Obstet Gynecol Surv 2000; 55 (5 Suppl) : S1-19.
5.
Amsel R, Totten PA, Spiegel CA, Chen KC, Eschenbach D, Holmes KK. Nonspecific vaginitis. Diagnostic criteria and microbial and epidemiologic associations. Am J Med 1983; 74 : 14-22
6.
Nugent RP, Krohn MA, Hillier SL. Reliability of diagnosing bacterial vaginosis is improved by a standardized method of gram stain interpretation. J Clin Microbiol 1991; 29 : 297301.
7.
Taylor-Robinson D, Morgan DJ, Sheehan M, Rosenstein IJ, Lamont RF. Relation between Gram-stain and clinical criteria for diagnosing bacterial vaginosis with special reference to Gram grade II evaluation. Int J STD AIDS 2003; 14 : 6-10.
8.
Duguid JP. Staining methods. In: Mackie & McCartney practical medical microbiology. Collee JG, Duguid JP, Fraser AG, Marmion BP, editors, 13th ed. Edinburgh: Churchill Livingstone; 1989. p. 38-63.
9.
91
The measurement of interrater agreement. In: Fleiss JL, editor. Statistical methods for rates and proportions. New York: John Wiley; 1981. p. 212-36.
10. Gardner HL, Dukes CD. Haemophilus vaginalis vaginitis: a newly defined specific infection previously classified nonspecific vaginitis. Am J Obstet Gynecol 1955; 69 : 962-76. 11. Milatovic D, Machka K, Brosch RV, Wallner HJ, Braveny I. Comparison of microscopic and cultural findings in the diagnosis of Gardnerella vaginalis infection. Eur J Clin Microbiol 1982; 1 : 294-7. 12. West B, Morison L, Schim van der Loeff M, Gooding E, Awasana AA, Demba E, et al. Evaluation of a new rapid diagnostic kit (FemExam) for bacterial vaginosis in patients with vaginal discharge syndrome in The Gambia. Sex Transm Dis 2003; 30 : 483-9. 13. Myziuk L, Romanowski B, Johnson SC. BVBlue test for diagnosis of bacterial vaginosis. J Clin Microbiol 2003; 41: 1925-8. 14. Spiegel CA, Amsel R, Holmes KK. Diagnosis of bacterial vaginosis by direct Gram stain of vaginal fluid. J Clin Microbiol 1983; 18 : 170-7. 15. Rao PS, Devi S, Shriyan A, Rajaram M, Jagdishchandra K. Diagnosis of bacterial vaginosis in the rural setup: comparison of clinical algorithm, smear scoring and culture by semiquantitative technique. Indian J Med Microbiol 2004; 22 : 47-50. 16. Aggarwal AK, Kumar R, Gupta V, Sharma M. Community based study of reproductive tract infections among ever married women of reproductive age in a rural area of Haryana, India. J Commun Dis 1999; 31 : 223-8. 17. Mazzulli T, Simor AE, Low DE. Reproducibility of interpretation of Gram-stained vaginal smears for the diagnosis of bacterial vaginosis. J Clin Microbiol 1990; 28 : 1506-8. 18. Joesoef MR, Hillier SL, Josodiwondo S, Linnan M. Reproducibility of a scoring system for gram stain diagnosis of bacterial vaginosis. J Clin Microbiol 1991; 29 : 1730-1. 19. Forsum U, Jakobsson T, Larsson PG, Schmidt H, Beverly A, Bjørnerem A, et al. An international study of the interobserver variation between interpretations of vaginal smear criteria of bacterial vaginosis. Acta Pathol Microbiol Immunol Scand 2002; 110 : 811-8. 20. Dunkelberg WE Jr. Diagnosis of Hemophilus vaginalis vaginitis by Gram-stained smears. Am J Obstet Gynecol 1965; 91 : 998-1000.
Reprint requests: Dr Arti Kapil, Additional Professor, Department of Microbiology, All India Institute of Medical Sciences Ansari Nagar, New Delhi 110 029, India e-mail:
[email protected]
