JOURNAL OF CLINICAL MICROBIOLOGY, Nov. 2003, p. 5159–5166 0095-1137/03/$08.00⫹0 DOI: 10.1128/JCM.41.11.5159–5166.2003 Copyright © 2003, American Society for Microbiology. All Rights Reserved.
Vol. 41, No. 11
Comparison of DNA Amplification, mRNA Amplification, and DNA Hybridization Techniques for Detection of Cytomegalovirus in Bone Marrow Transplant Recipients Francisco Diaz-Mitoma,1* Chantal Leger,2 Helen Miller,1 Antonio Giulivi,2,3 Rita Frost,1 Laura Shaw,1 and Lothar Huebsch2 Division of Virology, Children’s Hospital of Eastern Ontario,1 Divisions of Hematology and Oncology and the Canadian Blood and Bone Marrow Transplant Program, University of Ottawa,2 and Health Canada,3 Ottawa, Ontario, Canada Received 1 October 2002/Returned for modification 5 May 2003/Accepted 12 July 2003
A total of 676 specimens from 63 recipients of bone marrow allografts were tested for cytomegalovirus (CMV) by the following assays: CMV pp67 NucliSens (NS), AMPLICOR CMV MONITOR (RA), and the Digene CMV DNA test (DG). In a consensus analysis, the sensitivities and specificities were 60 and 99% (NS), 96 and 98% (RA), and 90 and 76% (DG), respectively; for detection of symptomatic CMV infection, they were 60 and 97% (NS), 65 and 97% (RA), and 95 and 77% (DG), respectively. In multivariate analysis, the major risk factor for symptomatic CMV infection was an increase in the viral load in the DG assay; in univariate analyses, maximum viral loads in both DG and RA assays and a rising viral load in the RA assay were also significant. The earliest detection of CMV replication was provided by the RA assay (mean, 39 days posttransplantation), followed by the DG assay (mean, 48 days) and the NS assay (mean, 58 days). are not standardized across laboratories. The commercial molecular assays, which are now widely available, are standardized and provide a basis for comparing differing methods. In the search for reliable prognostic laboratory markers of CMV disease, some groups have found that detection and/or quantitation of DNA in peripheral blood leukocytes (PBL) provides better clinical correlation than detection and/or quantitation of DNA in plasma (14, 20, 22), and qualitative detection of mRNA has also been shown to correlate well with the onset of CMV disease (5, 10, 25, 27). Several recent studies of quantitative tests for CMV nucleic acids have focused on identifying cutoff points of prognostic value in particular patient populations (22, 24, 26). While individual studies have shown promising results, no consensus has emerged as to either the most appropriate and useful laboratory techniques or appropriate cutoff levels for prognostic testing of different groups of immunosuppressed patients. We were interested in comparing the value of detection of CMV DNA in PBL and in plasma with that of detection of CMV mRNA for identifying BMT recipients at risk for CMV disease. Convinced of the importance of using well-standardized, readily available assays as a basis for clinical decisions, we designed a longitudinal study to test three commercial CMV assays in parallel, using sequential blood specimens from recent BMT recipients.
Cytomegalovirus (CMV) disease is an ever-present risk for immunosuppressed patients, in particular for the recipients of bone marrow transplants (BMT), in whom the morbidity and mortality associated with CMV are particularly high. The clinical diagnosis of CMV disease, difficult in all populations, is extremely complex in BMT patients, who frequently have multiorgan involvement from a variety of other factors, including regimen-related toxicity and other ongoing infections. The laboratory diagnosis of CMV disease is also, unfortunately, problematic, and there are as yet no universally standardized assays which serve as reliable prognostic markers for CMV disease (21, 25, 28). While some BMT units treat all patients prophylactically with ganciclovir to prevent CMV-related complications, there is a real risk of complications due to the toxicity of the drug. Algorithms for the identification and management of CMV-infected patients at high risk for disease vary greatly among laboratories, physicians, treatment centers, and patient groups: a recent survey (1) of BMT programs in the United States found that the majority of centers use a wide range of laboratory tests, with widely differing sensitivities, in their protocols for determining which patients are treated preemptively with ganciclovir. Our center uses an algorithm based on clinical judgement and laboratory analysis, which is described in Fig. 1. The increasing use of non-culture based (molecular) laboratory techniques has increased both the attainable sensitivity and the rapidity of CMV detection, but improvements in specificity and predictive value for disease have in general been less marked (13, 23, 25). Evaluation and comparison of the results of different molecular methods has been difficult until recently, because many procedures have been developed in house and
MATERIALS AND METHODS Study population. Between July 1997 and May 2000, 65 patients underwent allogeneic stem cell transplantation at the Ottawa Hospital for acute leukemia, chronic myelogenous leukemia, myelodysplasia, non-Hodgkin’s lymphoma, aplastic anemia, or an autoimmune disorder. Two patients were lost to follow-up; all but 1 of the remaining 63 patients were first-time stem cell transplant recipients. CMV serostatus testing was performed on all donors prior to the donation of peripheral stem cells or bone marrow and on all patients prior to transplantation; all patients received CMV-negative blood products. All patients were given prophylactic acyclovir (for herpes simplex virus) but not specific CMV prophylaxis (i.e., ganciclovir). Nine of the 63 patients had received transplants
* Corresponding author. Mailing address: Division of Virology, Children’s Hospital of Eastern Ontario, 401 Smyth Rd., Ottawa, ON K1H 8L1, Canada. Phone: (613) 737-7600, ext. 2736. Fax: (613) 7384825. E-mail:
[email protected]. 5159
5160
DIAZ-MITOMA ET AL.
J. CLIN. MICROBIOL.
FIG. 1. Clinical decision algorithm representing the standard of clinical care at the Ottawa Hospital for BMT patients being evaluated for CMV infection. This simple algorithm was not based on a systematic approach or decision analysis but rather on clinical judgment and interpretation of pp67 NASBA and DG assay results. Patients may or may not have laboratory results when CMV infection is suspected. (A) If test results are available, they may be categorized as negative, strongly positive (⬎5,000 genomes per ml of blood for the DG assay and/or a positive NASBA result), or weakly positive (2,000 to 5,000 genomes per ml of blood for the DG assay and a positive or negative NASBA result). (B) Clinical suspicion of a CMV infection, even in the absence of a confirmatory test, would trigger antiviral therapy.
between 2 and 58 weeks prior to the beginning of the study (median, 9 weeks); the remaining 54 were followed up from the date of transplantation. Appropriate biweekly CMV surveillance was begun following engraftment of neutrophils and platelets. Between August 1998 and August 2000, a total of 701 blood specimens were collected from the 63 patients and were tested immediately using the CMV pp67 NucliSens NASBA Diagnostics assay (NS) and the CMV DNA Test (version 2.0), Digene Hybrid Capture System (DG). Batch assays using the Roche AMPLICOR CMV MONITOR test (RA) were performed at the conclusion of the study period. Definitions. When CMV mRNA (NS) or DNA (DG) was detected in patients with clinical symptoms attributable to CMV and when other laboratory evidence (e.g., virus isolation from bronchoalveolar lavage fluid or detection of CMV in histological specimens) confirmed the presence of CMV (15), patients were identified as having CMV disease. In the absence of confirmation of CMV replication by other laboratory means, such patients were considered to have symptomatic CMV infection. Patients in whom CMV mRNA or DNA was detected but who had no unexplained clinical signs attributable to CMV were considered to have asymptomatic CMV infection. Figure 1 describes the clinical decision algorithm used for BMT patients with suspected CMV infections. This algorithm describes the standard of care in this population at the Ottawa Hospital. Specimen collection and assays. Blood was collected into three EDTA-anticoagulated tubes, one tube for each of the following three test procedures. NS (Organon Teknika, Durham, N.C.) is a qualitative assay which detects CMV pp67 mRNA in whole blood. RA (Roche Diagnostics, Branchburg, N.J.) is an automated quantitative assay which detects CMV DNA in plasma. DG (Digene Corporation, Beltsville, Md.) detects human CMV (HCMV) DNA in PBL; we used it as a quantitative test. At the time of our study, NS and DG were licensed for diagnostic use while RA was licensed for research use. All specimens were processed according to the manufacturers’ instructions, when possible within 24 h of collection. Specimens for the NS and DG assays were held at 4°C for immediate testing; those for the RA assay were stored at ⫺80°C until the tests were run. The information, procedural notes, and instructions provided by the manufacturers of the three assays were followed rigorously. NS is a PCR-based assay employing an electrochemiluminescence detection
system. In brief, after disruption of cells by the addition of a lysis buffer and addition of a system control, the nucleic acids were bound to silica particles and the unbound material was removed by repeated washing. After thorough drying, the bound nucleic acids were eluted in an elution buffer, and two amplification primers were added along with reverse transcriptase, RNase H, and T7 RNA polymerase. After incubation, each amplification reaction mixture was diluted and duplicate aliquots were placed in two series of tubes containing rutheniumlabeled probes specific for HCMV pp67 mRNA and the system control RNA, respectively. Following incubation, tripropylamine substrate was added and the tubes were placed in the carousel of the NucliSens reader, which performed the analysis automatically and interpreted the results as negative, positive, or invalid. RA is an automated PCR assay employing an enzyme immunoassay detection system. Plasma specimens and the manufacturer’s controls were each added to a lysis buffer containing a quantitation standard. DNA was precipitated with alcohol, and specimen diluent and the PCR master mix were added. Specimens and controls were then placed in a COBAS AMPLICOR analyzer, which performed the amplification, detection, and calculation and validation of results, with results expressed as the number of CMV DNA copies per milliliter. The linear range of the assay is 400 to 100,000 copies per ml. In the DG assay, DNA hybridization is followed by signal amplification and chemiluminescence detection. Cell pellets were first diluted and, along with the controls, denatured in an alkaline solution at 70°C. In our quantitative procedure we used Digene’s CMV DNA test panel as controls. After a mixture of four CMV RNA probes was added to each tube and hybridized at 70°C, the contents of each tube were transferred to a capture tube coated with alkaline phosphatase-conjugated polyclonal antibodies to RNA-DNA hybrids; the capture tubes were incubated, washed repeatedly, and dried thoroughly. After addition of a chemiluminescent substrate and incubation at 20 to 22°C, luminescence was read in a DCR-1 luminometer. Early in the study, the range of the control panel was 2.1 to 830 pg per ml (1,446 to 571,514 DNA copies per ml); this was altered to 4.2 to 830 pg per ml (2,892 to 571,514 DNA copies per ml) after part of the study had been completed. Assay results were read from the control curve and calculated in terms of both the volume of blood and the PBL count. Based on observations published previously, a clinically significant positive result in the
VOL. 41, 2003
CMV IN BONE MARROW TRANSPLANTS
DG assay was defined as ⬎1,000 genome copies per 106 PBL or ⱖ5,000 CMV genome equivalents per ml of blood (3). Results of the NS assay were reported to the clinical unit as positive, negative, or invalid. Results falling within the range of the standard control panel for the DG assay were reported both as the number of DNA copies per milliliter and as the number of DNA copies per 106 PBLs. Results falling within the linear range of the RA assay were recorded as the number of DNA copies per milliliter but were not reported to the clinical unit because of the licensing of the assay and the delay in testing. Because there is no “gold standard” for the diagnosis of CMV infection, a consensus positive result with which to compare each of the CMV detection methods was designed. A consensus standard was defined as a sample that had two positive tests out of the three CMV detection methods. Statistical analysis. Demographic, clinical, and laboratory data were analyzed using the Enterprise Miner and Insight functions of the SAS system software. The Enterprise Miner is able to find hidden patterns in large amounts of data. These patterns may result in the production of predictive models to find defined targets, such as risk factors or death. Several analytical tools are used to develop the predictive models, such as regression models or decision tree analysis. Decision trees split information into branches according to relevant variables that lead to a target value. This method is used to produce models of pertinent courses of action and their consequences based on clinical and laboratory results, including CMV load. The Insight software includes simpler statistical tests such as correlation statistics and linear and multiple regression. Risk factors for symptomatic CMV infection were assessed using both multivariate and univariate regression. Viral load measurements in the two quantitative assays were compared using linear regression and correlation procedures. Response operator curves (ROC) were calculated for CMV load according to published procedures (17). Briefly, ROC were used to display the relationship between the sensitivity and specificity of CMV detection methods. Plotting sensitivity against the falsepositive rate (1 ⫺ specificity) created ROC graphs. Mean and maximum viral loads were compared using a standard t test.
RESULTS Specimens. The number of specimens obtained from individual patients ranged between 1 and 40. One to 10 specimens each were received from 34 patients, 11 to 20 specimens from 18 patients, 21 to 30 specimens from 10 patients, and 40 specimens from 1 patient. Seventeen of the 701 specimens were insufficient in quantity for testing in all three procedures, and an additional eight specimens gave invalid results in one test (NS). Valid results in all three tests were obtained for the remaining 676 specimens. For purposes of analysis, specimens that were reactive in two of the three assays were considered truly reactive, while those that were discrepantly reactive in one assay were considered truly negative. Patients. Table 1 summarizes the demographic and other pre- and posttransplantation characteristics of the 63 consecutive allogeneic stem cell transplant recipients who underwent CMV surveillance during the study period. The great majority of patients underwent transplantation for malignant disease, and 57% had a matched related donor. Sixty-eight percent of the recipients received bone marrow as the sole stem cell source, while 24% received peripheral blood and the remainder received a combination of both. The pretransplantation regimen included total-body irradiation and anti-thymocyte globulins for 48 and 39% of transplant recipients, respectively. Forty percent of the recipients were CMV seronegative and 60% were CMV seropositive prior to transplantation; 46% (18) of the seropositive patients had seronegative donors. Posttransplantation ganciclovir therapy was initiated for 26 patients and foscarnet therapy was initiated for 1 patient in whom CMV infection was suspected on clinical grounds, or because of a positive laboratory result in the NS or DG assay,
5161
or for both reasons. Three additional patients were treated with both ganciclovir and foscarnet. Based on posttransplantation clinical and laboratory evidence, the 63 patients fell into four groups. Group 1 consisted of 14 patients who were identified clinically as CMV infected and who were reactive in at least two of the three CMV assays. CMV disease was confirmed for 3 patients, and 10 patients received antiviral therapy for CMV (primarily ganciclovir; three also received foscarnet). Group 2 consisted of six patients who were identified clinically as having symptoms of CMV infection but who were nonreactive in two of the three assays. Three of the patients were treated briefly with antiviral agents. Group 3 consisted of 12 patients who were reactive in at least two of the three assays but who remained CMV asymptomatic. Seven of the 12 were treated preemptively with ganciclovir based on laboratory reports of the presence of CMV mRNA and/or DNA. Group 4 consisted of 31 patients who were nonreactive in at least two of the three assays and who remained CMV asymptomatic. Five of the patients were nonreactive in all three assays. Eleven patients received ganciclovir. The Enterprise Miner function of SAS uses a set of input data to generate a hierarchy or decision tree of data categories, which together tend to identify a selected data element (the target) to the exclusion of other elements. The input for each patient included all available demographic and posttransplantation clinical data, reactivity in the three assays, maximum viral loads in the RA and DG assays, and the presence or absence of increasing viral loads in sequential specimens. The targets were membership in each of the four patient groups above and in the set of patients identified as CMV symptomatic (group 1 plus group 2). The laboratory data figured prominently in most of the resulting decision trees. The trees for group 1 patients and for all CMV symptomatic patients were essentially the same, in that the primary risk factor in each was a rising viral load in the DG assay (defined as at least a twofold increase over a 2-week period) and the secondary factor in patients exhibiting such an increase was a return to an absolute neutrophil count of ⬍0.5 within 22 days following transplantation. When laboratory data were eliminated from the input for CMV symptomatic patients, the primary category became graft-versus-host disease (GVHD) prophylaxis: 37% of patients who received methotrexate plus cyclosporine, but only 14% of those who were treated with T-cell depletion plus cyclosporine, were CMV symptomatic. Group 2 was not susceptible to separate analysis because of its small size (six patients). The decision tree for group 3, those patients who were clinically asymptomatic but reactive in the assays, was more complex than the tree for group 1. The primary risk factor for asymptomatic infection was reactivity in the RA assay, followed in the nine reactive patients by a rising DG viral load and then by a posttransplantation time lag of ⬎22 days to restoration of a neutrophil count of ⬎0.5. For patients who were nonreactive in the RA assay, the second-level risk factor for asymptomatic infection was reactivity in the NS assay. By contrast, the primary factor associated with lack of CMV infection (group 4) was lack of reactivity in the RA assay: 80% of all RA-nonreactive patients belonged to this group, while
5162
DIAZ-MITOMA ET AL.
J. CLIN. MICROBIOL.
TABLE 1. Pre- and posttransplantation characteristics Characteristica
Valueb
No. of patients ...........................................................................65 Median age (range) in yr .........................................................43 (13–59) Sex Male ........................................................................................34 (54) Female ....................................................................................31 (46) Underlying disease Acute leukemia......................................................................27 (42) Chronic myelogenous leukemia...........................................17 (26) Myelodysplasia....................................................................... 5 (8) Multiple myeloma ................................................................. 3 (5) Non-Hodgkin’s lymphoma ...................................................11 (17) Aplastic anemia ..................................................................... 1 (2) Autoimmune disorder........................................................... 1 (2) Pretransplantation CMV serology (recipient/donor) Negative/negative ..................................................................17 (26) Negative/positive.................................................................... 9 (14) Positive/negative ....................................................................18 (28) Positive/positive .....................................................................21 (32) Type of transplant Matched related ....................................................................37 (57) Mismatched related............................................................... 1 (2) Matched unrelated ................................................................21 (32) Mismatched unrelated .......................................................... 6 (9) Stem cell source Selected bone marrow ..........................................................41 (63) Unselected bone marrow ..................................................... 3 (5) PBPC.......................................................................................16 (25) Bone marrow ⫹ PBPC ......................................................... 5 (7) Conditioning regimen Regimens with TBI ...............................................................32 (49) Regimens without TBI .........................................................33 (51) Regimens with ATG .............................................................26 (40) Regimens without ATG .......................................................39 (60) GVHD prophylaxis Methotrexate ⫹ cyclosporine ..............................................50 (77) T-cell depletion ⫹ cyclosporine ..........................................15 (23) Engraftment Median days (range) to neutrophil ....................................18 (2–65) count of ⬎0.5 ⫻ 109/liter Median days (range) to platelet count ..............................21 (7–90) of ⬎20 ⫻ 109liter Acute GVHD grade (n ⫽ 63)c None........................................................................................42 (65) Grade 1...................................................................................10 (15) Grade 2...................................................................................10 (15) Grade 3................................................................................... 1 (2) Grade 4................................................................................... 2 (3) Symptomatic CMV infection (n ⫽ 63)c .................................20 (32) a PBPC, peripheral blood progenitor cells; TBI, total-body irradiation; ATG, anti-thymocyte globulin. b Each value is the number (percentage) of patients with the indicated characteristic, except where otherwise indicated in the “Characteristic” column. c Two patients were lost to follow-up.
the single patient misidentified by the RA assay represented only 6% of the RA-positive patients. At the second level, 90% of all RA-reactive, non-NS-reactive patients were members of group 4. In multivariate analysis using a stepwise forward regression, the most significant risk factor for symptomatic CMV infection was the presence of a rise in viral load in the DG assay (r2 ⫽ 0.31; t ⫽ 4.98). In univariate analyses the maximum viral loads in both assays and an increasing viral load in the RA assay were also associated with clinical illness, but less significantly so (t, 2.47 to 3.24).
Assays. The three assays were compared both by a consensus analysis and with respect to their ability to detect symptomatic CMV infection. Results of less than 400 genome copies per ml in the RA assay and results below the lowest defined concentration of CMV DNA in the DG assay were interpreted as nonreactive. In order to assess the stability of samples stored for RA testing and to ensure that loss of sensitivity did not occur for this test, paired samples were tested 18 to 24 months after storage at ⫺80 C. There was no statistical difference in the virus load in six pairs of specimens tested. Because the clinically significant threshold for the DG assay was considered to be 1,000 genome copies per 106 PBL, the laboratory data were analyzed twice. The results of the consensus analysis of the three assays are displayed in Table 2. The RA test was both more sensitive (95.9%) than NS (60.3%) and DG (90.4%) and more specific than DG (97.5 versus 75.5%, respectively). The NS and RA assays were concordant for 92% of the specimens, NS and DG were concordant for 72%, and RA and DG were concordant for 74%; when the threshold of 1,000 genomes per 106 PBL was used in the calculations, the concordance between DG and NS improved to 86%, and that between DG and RA improved to 87%. With the exception of the negative predictive values of the RA and DG assays, all of the interassay differences were statistically significant (⬎2 standard deviations). The measurements of viral load per milliliter and per 106 PBL in the DG assay were highly correlated (Pearson correlation coefficient, 0.936), but the correlations between DG and the other two assays were higher when all viral loads were expressed in terms of volume. The correlation coefficient for the RA and DG assays was 0.652, that for the NS and RA assays was 0.433, and that for the NS and DG assays was 0.356 (P ⬍ 0.0001 in all cases). We calculated ROC for the two quantitative tests, RA and DG, by using RA values up to 2,000 genomes per ml and DG levels up to 30,000 genomes per ml and 10,000 genomes per 106 PBL (Fig. 2). The sensitivity of the RA assay was greater than that of the DG assay at all specificity values, and the manufacturer’s cutoff of 400 genomes/ml for the RA assay appears to maximize the attainable sensitivity and specificity of the test. For the DG assay, on the other hand, a cutoff between 6,000 and 10,000 genomes/ml would appear to be more appropriate on the basis of this curve. The ROC generated by the two sets of figures for the DG assay are similar; 6,000 genomes/ml corresponds to the clinical cutoff used in this study, 1,000 genomes/106 PBL. We also calculated the sensitivities, specificities, and predictive values of the three assays based on (i) the clinical diagnosis of CMV symptoms posttransplantation (patient groups 1 and 2) and (ii) the absence of clinical or laboratory evidence of CMV infection (patient group 4) (Table 3). The sensitivity and specificity of the NS assay, and those of the DG assay when measurements were expressed per milliliter of blood, were similar to those obtained by consensus analysis. When the DG results were calculated in terms of PBL, the clinical sensitivity and specificity of the assay were considerably lower than when the kit cutoffs were used (75 and 65%, respectively, compared with 95 and 77%). Sensitivity and specificity also declined markedly in the clinical analysis when higher cutoffs (6,000 and 10,000 genomes/ml and 2,000 genomes/106 PBL) were used.
VOL. 41, 2003
CMV IN BONE MARROW TRANSPLANTS
TABLE 2. Detection of consensus positive specimens by the NS, RA, and DG assays Assay
With DG results expressed per ml of bloodc NS RA DG With DG results expressed per 106 PBLd NS RA DG
Sensitivity (%)
Specificity (%)
PPVa (%)
NPVb (%)
60 96 90
99 98 76
92 82 31
95 99 99
64 100 83
99 97 91
85 75 50
96 100 98
a
PPV, positive predictive value. NPV, negative predictive value. c Threshold, 5,000 CMV genome equivalents per ml of blood. d Threshold, 1,000 genome copies per 106 PBL. b
For the RA assay, the sensitivity in the clinically based calculation was markedly lower than in the consensus analysis (65% compared with 96%); the specificity remained at 97%. With the exception of the DG results expressed in terms of PBL, positive predictive values were the same (NS) or higher when
5163
the clinical standard was used, and negative predictive values were lower. Early detection of DNAemia. For patients who were reactive by at least two of the three assays, the earliest detector of virus replication was the RA assay (mean, 39 days posttransplantation; median, 37 days). For the DG assay, the mean was 48 days (median, 38), and for the NS assay, the mean was 58 days (median, 48). The periods during which replication was detected ranged from 2 to 16 weeks (median, 6 weeks). No assay was reactive at all time points, and in no case was the mRNAbased NS assay the sole, earliest detector of infection. Viral load. The quantitative results of the RA and DG assays were comparable, though measures of viral load were generally higher in the DG assay: The median viral loads in the 64 concordantly positive specimens were 11,855 genomes/ml for DG (range, 2,086 to 571,000) and 3,230 genomes/ml for RA (range, 431 to 100,000) (Fig. 3). The results of the two assays are correlated, as indicated by the Pearson correlation coefficient of 0.652 (P ⬍ 0.0001), and logarithmic regression showed a linear relationship (r2 ⫽ 0.23; t ⫽ 4.46; P ⬍ 0.0001), which improved slightly when DG results calculated in terms of PBL count were used (r2 ⫽ 0.31; t ⫽ 5.45; P ⬍ 0.0001). In both the RA and DG assays, mean viral loads were higher
FIG. 2. ROC constructed from the DG and RA CMV load results for 63 BMT patients. The sensitivity, or true-positive results, of each of the assays was plotted against the false-positive results, or 1.0 ⫺ specificity. Red line, RA assay; green line, DG assay with results expressed as virus load per 106 cells; blue line, DG assay with results expressed as genome equivalents per milliliter of blood. The results show that the sensitivity of the RA assay was greater than that of the DG assay at all specificity values. For the RA assay, a cutoff of 400 genomes/ml appears to maximize the attainable sensitivity and specificity, while for the DG assay, a cutoff between 6,000 and 10,000 genomes/ml would appear to be more appropriate on the basis of this curve.
5164
DIAZ-MITOMA ET AL.
J. CLIN. MICROBIOL.
TABLE 3. Detection of symptomatic CMV infection by the NS, RA, and DG assays Assay
NS RA DG Results expressed per ml of bloodc Results expressed per 106 PBLd
Sensitivity (%)
Specificity (%)
PPVa (%)
NPVb (%)
60 65
97 97
92 93
78 81
95
77
44
87
75
65
43
69
a
PPV, positive predictive value. NPV, negative predictive value. Threshold, 5,000 CMV genome equivalents per ml of blood. d Threshold, 1,000 genome copies per 106 PBL. b c
for patients whose pretransplantation CMV serology indicated a positive-recipient–negative-donor combination than in other patients. Mean and maximum viral loads were also higher for CMV symptomatic patients than for asymptomatic patients. However, none of these differences reached statistical significance (t test). Of the 14 patients whose specimens were reactive in all three assays, 5 (36%) survived the posttransplantation period. By contrast, survival rates were 50% (6 of 12) among patients whose specimens were reactive in two of the three assays and 68% (25 of 37) among those whose specimens were reactive in only one or none of the assays. For the 26 patients (groups 1 and 3) whose specimens were reactive in at least two of the three assays, higher viral loads tended to be associated with failure to survive 6 months posttransplantation. Mean viral loads in specimens from the 11 surviving patients were 3,668 copies/ml by the RA assay and 17,218 copies per ml by the DG assay; for the 15 who did not survive, mean viral loads were 16,429 and 45,639 copies per ml, respectively, by the two assays. Decision tree analysis showed that for 81% of patients, a viral load of ⱖ8,485 copies per ml in the RA assay was associated with failure to survive; when the NS assay was also positive, this figure rose to 89%. Specimens from patients who did not survive were more likely to be reactive in the DG assay than specimens from patients who did (43 versus 35%, respectively). DISCUSSION Although mRNA has been found to be an early indicator of CMV replication, appearing near the onset of clinical symptoms (27), the mRNA-based NS assay was reactive with the earliest positive specimen in only 2 of the 17 incidents of CMV viremia observed here, and for both of those specimens the RA assay was also positive. Neither of our statistical analyses showed a relationship between results in the NS assay and clinical illness, and at a sensitivity of 60%, NS ranked the lowest of the three assays. These data are in accord with results obtained recently (21) for a patient population similar to ours, where twice as many positive results were detected by the RA assay as by the pp67 mRNA assay (NS) during reactivation of CMV infections. Other recent studies with differing patient populations (4, 12, 20) have also noted that this assay is not optimal for detection of disease. The relative instability of
mRNA in transit and/or the short period over which it is present during infection or reactivation may in part explain this; alternatively, it is possible that a different target may be required. A clear relationship between detection of CMV immediate-early antigen mRNA and clinical findings has been reported (5, 9, 10, 12, 29), and the latter study indicates that a new commercial assay which targets an immediate-early transcript may be both more sensitive and a better early indicator of infection than the pp67-mRNA assay. Although it has been found previously (6, 7, 19) that detection of CMV replication tends to occur earlier in PBL-based assays than in plasma-based assays, 2 of 17 incidents of posttransplantation CMV DNAemia reported here were detected earlier by the RA assay than by the DG assay, and 9 of the incidents were detected simultaneously by the two assays. While the RA assay performed considerably better in the consensus analysis than in the clinical analysis, in the latter it remained more sensitive than the NS assay and more specific than the DG assay. At 65 and 97%, respectively, its sensitivity and specificity were considerably better for these BMT patients than were the comparable values (55 and 74%) reported recently (20) for this assay in a renal transplant population. The decision tree for group 4 also indicates that RA negative results are highly reliable. Contrary to the suggestion of other workers (12), who compared the RA assay to an in-house qualitative PCR test, our ROC indicates no reason to lower the detection limit of this assay, despite the fact that the correlation between the RA assay and symptomatic CMV infection was not as good as that of the DG assay. The DG assay performed equally well in the consensus and clinical analyses, but its specificity, at 76 to 77%, could be improved only at considerable sacrifice of sensitivity. With the sensitivity and specificity of the assay both optimized at a cutoff of 6,000 genomes per ml, the sensitivity was near 77% in both the consensus and clinical analyses but the specificity, though 91% in the consensus analysis, was only 68% in detecting patients with symptomatic CMV infection. The clinical threshold of 1,000 genome copies/106 PBL in the DG assay (equivalent to 6,000 genomes per ml) improved the specificity of the test considerably when the standard was the concordance of two of the three assays, eliminating 94 of 148 discordant DG results. It had the opposite effect, however, when the standard was the detection of symptomatic CMV infection, as four of the six patients in group 2 had no specimens which were positive at that level. Studies of the DG assay with differing populations of immunosuppressed patients (16, 26) have identified different potential cutoff levels and attainable levels of sensitivity and specificity: the available data would indicate that this assay requires rigorous and extensive validation in each patient population for which it is used. The obvious goal in comparing the performance of different assays is to identify which one provides the best detection of symptomatic CMV infection in the patient population concerned, and in this sense a clinical standard of comparison is preferable to the laboratory-based consensus standard. The consensus standard has also been used extensively, however, both because the relevant clinical data are often not available and because no single, widely accepted laboratory gold standard for CMV detection exists. In this study, the reliability of the indicators of assay performance according to the clinical
VOL. 41, 2003
CMV IN BONE MARROW TRANSPLANTS
5165
FIG. 3. Comparison of CMV loads measured by the DG and RA assays. Column set 1, median virus loads; column set 2, average virus loads. Error bars, standard deviations. The values obtained by the DG assay, which represent the total content of viral DNA in both the cellular and plasma compartments, were higher than the RA values. However, the results of the two assays were correlated, as demonstrated by a Pearson correlation coefficient of 0.652.
standard is subject to variation due to the relatively small size of the patient population; this may in part explain the differences in some of the sensitivities and specificities of the two assays as measured against the two standards. In this study, increasing viral load in one of the two quantitative assays (DG) emerged from multivariate regression analysis as the most significant risk factor for symptomatic CMV infection. The stepwise forward regression used may have failed to identify other measures of viral load because of the correlation between the results of the DG and RA assays: in univariate analysis, increasing viral load in the RA assay and peak viral loads in both assays were also significantly related to symptomatic CMV infection, though at lower levels. In two recent studies, one involving a population of 110 BMT patients (11) and the other involving a population of 359 solid-organ and marrow transplant recipients (8), recipient-donor serostatus, acute GVHD, and immunosuppressive regimen were also identified as risk factors, but only factors related to viral load remained significant in multivariate analysis in those studies. High viral loads do not invariably correlate with more-severe infections, nor low viral loads with freedom from complications due to CMV (2, 20, 29). It has been recommended that assays with low thresholds of detectability and low cutoff levels for significant results should be used, in particular for marrow transplant recipients, in whom CMV disease can be rapidly
fatal (6). This report highlights the fact that even for BMT recipients, a low threshold may fail to discriminate between simple reactivation and potentially serious disease; reliance on such thresholds may result in antiviral treatment of many patients who are not necessarily at elevated risk for CMV disease. In the case of the antigenemia assay, it was found recently (18) that, while rising antigenemia levels did not correlate with CMV disease in allogeneic stem cell recipients, all patients who developed CMV disease had rising levels of antigenemia. Comparably, in our data, 77% (10 of 13) of the increases in viral load in the DG assay and 86% (6 of 7) of the increases in viral load in the RA assay occurred in the 20 patients who were or later became CMV symptomatic. It appears that in practice, a rising viral load in either assay could serve as an indicator of risk when used in combination with clinical indicators. With the exception of the three patients whose CMV disease was confirmed by other laboratory techniques, the associations made in this report between reactivity in the assays or high viral loads and failure to survive do not imply direct causal relationships: other factors such as severe GVHD were implicated in some of the fatal cases. It does appear, however, that CMV contributes to morbidity in these patients. From a technical standpoint, the three assays are all straightforward and easy to perform. The suppliers provide all specialized equipment, and the kit inserts provide extensive back-
5166
DIAZ-MITOMA ET AL.
ground information and procedural notes. On a cost-per-test basis, we found the DG assay to be the least costly ($Can 70), the NS assay to be intermediate in cost ($Can 90), and the RA assay to be the most expensive ($Can 130); these figures include labor and expendables. The DG assay requires a large specimen volume, 3.5 ml of whole blood, but is technically simple and can be completed in one 8-h day. The NS assay requires 100 l of whole blood and takes about twice as much hands-on time as the other two assays; for runs of more than 8 to 10 samples, we found it more efficient to complete the testing on the following day. The RA assay, which requires 200 l of plasma, is fully automated apart from the extraction procedure. With a run turnaround time of about 5 h, it is easily integrated into the existing workload of a laboratory. Unfortunately, RA tests must be batched in order for the assay to be cost-effective. In summary, our results indicate that the NS pp67 assay is neither a sensitive nor an early indicator of CMV infection in marrow transplant recipients. More recently, the NASBA pp67 has been used in parallel with the detection of immediate early-1 mRNA, which has an increased sensitivity for CMV detection and may be a useful marker for early initiation of antiviral therapy (5). With its consistently high specificity, the RA assay is reliable as a negative indicator of CMV replication and is also an effective early indicator of CMV DNAemia. The specificity of the DG assay, which has the advantages of technical simplicity and low cost, can be improved at some cost in specificity, and the assay will require extensive validation in any population in which it is used. In this study, however, it was the DG assay that provided the most significant marker for symptomatic CMV infection.
J. CLIN. MICROBIOL.
9. 10. 11.
12.
13.
14. 15. 16.
17. 18.
19.
ACKNOWLEDGMENT This project was funded in part by a grant from Health Canada.
20.
REFERENCES 1. Avery, R. K., K. A. Adal, D. L. Longworth, and B. J. Bolwell. 2000. A survey of allogeneic bone marrow transplant programs in the United States regarding cytomegalovirus prophylaxis and pre-emptive therapy. Bone Marrow Transplant. 26:763–767. 2. Barrett-Muir, W., J. Breuer, C. Millar, J. Thomas, D. Jeffries, M. Yaqoob, and C. Aitken. 2000. CMV viral load measurements in whole blood and plasma—which is best following renal transplantation? Transplantation 70: 116–119. 3. Bhorade, S. M., C. Sandesara, E. R. Garrity, W. T. Vigneswaran, L. Norwick, S. Alkan, A. N. Husain, M. A. McCabe, and V. Yeldandi. 2001. Quantification of cytomegalovirus (CMV) viral load by the hybrid capture assay allows for early detection of CMV disease in lung transplant recipients. J. Heart Lung Transplant. 20:928–934. 4. Blank, B. S. N., P. L. Meenhorst, J. W. Mulder, G. J. Weverling, H. Putter, W. Pauw, W. C. van Dijk, P. Smits, S. Lie-A-Ling, P. Reiss, and J. M. Lange. 2000. Value of different assays for detection of human cytomegalovirus (HCMV) in predicting the development of HCMV disease in human immunodeficiency virus-infected patients. J. Clin. Microbiol. 38:563–569. 5. Blok, M. J., I. Lautenschlager, V. J. Goossens, J. M. Middeldorp, C. Vink, K. Hockerstedt, and C. A. Bruggeman. 2000. Diagnostic implications of human cytomegalovirus immediate early-1 and pp67 mRNA detection in wholeblood samples from liver transplant patients using nucleic acid sequencebased amplification. J. Clin. Microbiol. 38:4485–4491. 6. Boivin, G., R. Be´langer, R. Delage, C. Beliveau, C. Demers, N. Goyette, and J. Roy. 2000. Quantitative analysis of cytomegalovirus (CMV) viremia using the pp65 antigenemia assay and the COBAS AMPLICOR CMV MONITOR PCR test after blood and marrow allogeneic transplantation. J. Clin. Microbiol. 38:4356–4360. 7. Boivin, G., J. Handfield, E. Toma, G. Murray, R. Lalonde, V. J. Tevere, R. Sun, and M. G. Bergeron. 1998. Evaluation of the AMPLICOR cytomegalovirus test with specimens from human immunodeficiency virus-infected subjects. J. Clin. Microbiol. 36:2509–2513. 8. Emery, V. C., C. A. Sabin, A. V. Cope, D. Gor, A. F. Hassan Walker, and P. D.
21.
22.
23.
24. 25.
26.
27.
28. 29.
Griffiths. 2000. Application of viral-load kinetics to identify patients who develop cytomegalovirus disease after transplantation. Lancet 355:2032– 2036. Gerna, G., F. Baldanti, J. Middeldorp, and D. Lilleri. 2000. Use of CMV transcripts for monitoring CMV infections in transplant patients. Int. J. Antimicrob. Agents 16:455–460. Goossens, V. J., C. Vink, W. Mullers, J. M. Middeldorp, and C. A. Bruggeman. 2001. Different profiles of cytomegalovirus RNA transcripts and anti-cytomegalovirus IgM antibodies in renal transplant recipients. J. Clin. Virol. 23:87–95. Gor, D., C. Sabin, H. G. Prentice, N. Vyas, S. Man, P. D. Griffiths, and V. C. Emery. 1998. Longitudinal fluctuations in cytomegalovirus load in bone marrow transplant patients: relationship between peak virus load, donor/ recipient serostatus, acute GVHD and CMV disease. Bone Marrow Transplant. 21:597–605. Halwachs-Baumann, G., M. Wilders-Truschnig, G. Enzinger, M. Eibl, W. Linkesch, H. J. Dornbusch, B. I. Santner, E. Marth, and H. H. Kessler. 2001. Cytomegalovirus diagnosis in renal and bone marrow transplant recipients: the impact of molecular assays. J. Clin. Virol. 20:49–57. Hebart, H., A. Schro ¨der, J. Lo ¨ffler, T. Klingebiel, H. Martin, B. Wassmann, F. Gerneth, H. Rabenau, G. Jahn, L. Kanz, C. A. Muller, and H. Einsele. 1997. Cytomegalovirus monitoring by polymerase chain reaction of whole blood samples from patients undergoing autologous bone marrow or peripheral blood progenitor cell transplantation. J. Infect. Dis. 175:1490–1493. Jabs, D. A., M. Forman, C. Enger, and J. B. Jackson. 1999. Comparison of cytomegalovirus loads in plasma and leukocytes of patients with cytomegalovirus retinitis. J. Clin. Microbiol. 37:1431–1435. Ljungman, P., and S. A. Plotkin. 1995. Workshop on CMV disease; definitions, clinical severity scores, and new syndromes. Scand. J. Infect. Dis. Suppl. 99:87–89. Mazzulli, T., L. W. Drew, B. Yen-Lieberman, D. Jekic-McMullen, D. J. Kohn, C. Isada, G. Moussa, R. Chua, and S. Walmsley. 1999. Multicenter comparison of the Digene hybrid capture CMV DNA assay (version 2.0), the pp65 antigenemia assay, and cell culture for detection of cytomegalovirus viremia. J. Clin. Microbiol. 37:958–963. Metz, C. E. 1998. Basic principles of ROC analysis. Semin. Nucl. Med. 8:283–298. Nichols, W. G., L. Corey, T. Gooley, W. L. Drew, R. Miner, M. Huang, C. Davis, and M. Boeckh. 2001. Rising pp65 antigenemia during preemptive anticytomegalovirus therapy after allogeneic hematopoietic stem cell transplantation: risk factors, correlation with DNA load, and outcomes. Blood 97:867–874. Pellegrin, I., I. Garrigue, C. Binquet, G. Chene, D. Neau, P. Bonot, F. Bonnet, H. Fleury, and J. L. Pellegrin. 1999. Evaluation of new quantitative assays for diagnosis and monitoring of cytomegalovirus disease in human immunodeficiency virus-positive patients. J. Clin. Microbiol. 37:3124–3132. Pellegrin, I., I. Garrigue, D. Ekouevi, L. Couzi, P. Merville, P. Merel, G. Chene, M. H. Schrive, P. Trimoulet, M. E. Lafon, and H. Fleury. 2000. New molecular assays to predict occurrence of cytomegalovirus disease in renal transplant recipients. J. Infect. Dis. 182:36–42. Preiser, W., S. Bra ¨uninger, R. Schwerdtfeger, U. Ayliffe, J. A. Garson, N. S. Brink, S. Franck, H. W. Doerr, and H. F. Rabenau. 2001. Evaluation of diagnostic methods for the detection of cytomegalovirus in recipients of allogeneic stem cell transplants. J. Clin. Virol. 20:59–70. Schafer, P., W. Tenschert, L. Cremaschi, M. Schroter, B. Zollner, and R. Laufs. 2001. Area under the viraemia curve versus absolute viral load: utility for predicting symptomatic cytomegalovirus infections in kidney transplant patients. J. Med. Virol. 65:85–89. Sia, I. G., J. A. Wilson, M. J. Espy, C. V. Paya, and T. F. Smith. 2000. Evaluation of the COBAS AMPLICOR CMV MONITOR test for detection of viral DNA in specimens taken from patients after liver transplantation. J. Clin. Microbiol. 38:600–606. Tong, C. Y. W., L. E. Cuevas, H. Williams, and A. Bakran. 2000. Comparison of two commercial methods for measurement of cytomegalovirus load in blood samples after renal transplantation. J. Clin. Microbiol. 38:1209–1213. Velzing, J., P. H. Rothbarth, A. C. M. Kroes, and W. G. V. Quint. 1994. Detection of cytomegalovirus mRNA and DNA encoding the immediate early gene in peripheral blood leukocytes from immunocompromised patients. J. Med. Virol. 42:164–169. Wattanamano, P., J. L. Clayton, J. J. Kopicko, P. Kissinger, S. Elliot, C. Jarrott, S. Rangan, and M. A. Beilke. 2000. Comparison of three assays for cytomegalovirus detection in AIDS patients at risk for retinitis. J. Clin. Microbiol. 38:727–732. Wolff, D., M. Skourtopoulos, D. Ho¨rnschemeyer, C. Wolff, M. Korner, and R. Korfer. 1996. Longitudinal monitoring of latent and active human cytomegalovirus infections in peripheral blood of heart transplant recipients by single-tube nested RT-PCR. Microbiol. Res. 151:343–349. Yen-Leiberman, B. 2000. Diagnosis of human cytomegalovirus disease. Clin. Microbiol. Newsl. 22(14):1–5. Zipeto, D., S. Morris, C. Hong, A. Dowling, R. Wolitz, T. C. Merigan, and L. Rasmussen. 1995. Human cytomegalovirus (CMV) DNA in plasma reflects quantity of CMV DNA present in leukocytes. J. Clin. Microbiol. 33:2607– 2611.
