Laboratory QA Diagnostic Accuracy Among 4 Point-of-Care Tests for the Screening of HIV Infection Karina Flores, MT,2 Segundo R. Leon, MT (ASCPi), MT&ID1,3*
ABSTRACT The aim of this study was to validate and compare 4 point-ofcare (POC) devices for the detection of antibodies to human immunodeficiency virus 1 (HIV-1) and/or HIV-2 in cryopreserved serum specimens. We analyzed 99 specimens that had tested positive for antibodies to HIV-1/HIV-2 by Western blot and 101 samples with negative test results. All the samples were analyzed by each of the 4 rapid tests. The POC devices assessed in this study use lateral flow immunochemistry technology and provided results within 15 to 20
More than 30 million people worldwide are infected with human immunodeficiency virus (HIV), including 2.3 million children younger than 15 years. According to the Joint United Nations Programme on HIV/AIDS (UNAIDS), more than 2 million people are newly infected every year.1 HIV infection is detected using antibodies against viral proteins, or by using antigens as a targets of specific antiHIV antibodies using a wide range of available enzyme immunoassays (EIAs).2 The scaling up of diagnosis and treatment in low-resource settings and in the developed world is still limited because many infected individuals are unaware of their infection.3 Rapid detection of HIV infection
minutes. We obtained an average sensitivity of 97.57%, specificity of 98.77%, positive predictive value of 98.52%, and negative predictive value of 97.82%. The among-test correlation was 97.57%, and the kappa index was 0.96. Assessment of the performance of new rapid HIV tests must be regularly practiced where regular enzyme immunoassay (EIA) tests are not widely available. All 4 rapid HIV tests assessed in this study performed with acceptable accuracy. Keywords: HIV rapid test, POC, sensitivity and specificity, Peru
is crucial for diagnosis, appropriate treatment, and prevention of transmission.4 Laboratory diagnosis of HIV using point-of-care (POC) tests has been widely developed and implemented, particularly in low-resource settings, where access to accurate diagnosis is limited.5 Given the difficulties in providing accurate laboratory diagnoses and having a wide range of POC tests available, the performance of POC tests for HIV infection need to be assessed.6 The goal of this study was to assess the accuracy of 4 POC tests for the rapid screening for HIV infection using frozen serum specimens.
DOI: 10.1309/LMUY1PHAC0S0DMJR
Materials and Methods
Abbreviations
Laboratory Procedures
HIV, human immunodeficiency virus; UNAIDS, Joint United Nations Programme on HIV/AIDS; EIAs, enzyme immunoassays; POC, point-ofcare; ID, identification; WB, Western blot; NPVs, negative predictive values; PPVs, positive predictive values; WHO, World Health Organization; USD, United States dollars; IgG, immunoglobulin G; IgM, immunoglobulin M; IgA, immunoglobulin A; RT, room temperature; NS, not specified Unit ofHealth, Sexuality and Human Development; and Laboratory of Sexual Health, Cayetano Heredia University, Lima, Peru, 2School of Medical Technology, Faculty of Medicine, San Marcos National Major University, Lima, Peru, 3Laboratory of Molecular Epidemiology and Genetics, Tropical Medicine Institute, UNMSM, Lima, Peru 1
*To whom correspondence should be addressed. E-mail:
[email protected]
172
Lab Medicine Spring 2013 | Volume 44, Number 2
Two hundred samples were randomly selected from the serum bank at the Laboratory of Sexual Health at Cayetano Heredia University in Lima, Peru. The samples were aliquoted using numerically-identified (ID, Identifying Data) cryovials; the specimens were thawed and stored at 2º to 8ºC until tested. Of these 200 samples, 101 originally had positive results on 2 types of HIV EIA test (Vironostika HIV1/2 plus O Microelisa System, BioMérieux, Marcy l’Etoile, France; and EIA HIV Genetic System Bio-Rad Laboratories, Inc, Hercules, CA); positive results had been confirmed by Western blot (WB) testing (WB HIV, BioMérieux).
www.labmedicine.com
Laboratory QA
Table 1. Characteristics of 4 Point-of-Care Tests for Rapid HIV-1and HIV-2 Diagnosis Based on Data Provided By the Manufacturer Company and Country Trade Name of Origin Technology Samples Sensitivity Specificity
Time for Shelf Cost Per Results, min Life, mo Unit, USDe
Hexagon HIVa Human, Immunochromato- Serum, plasma, 100% 99.5% 20 Diagnostics, graphic rapid test for whole blood Germany antibodies against HIV-1 and HIV-2 (3rd generation)
18, at 2°-30°C
2.40
Determine Inverness Lateral flow Serum, plasma, HIV-1 positive, 99.75% 15 Medical, whole bloodf HIV-2 positive, HIV 1/2b Japan HIV-1 subtype A-G, HIV-1 group O Europe, North America, and Asia, 100%; Africa, 99.91% SD Bioline Standard Immunochromato- Serum, plasma, WHO, South Africa, WHO, 99.3%; 20 HIV-1/2 Diagnostics, graphic rapid test whole blood Zimbabwe, Malawi, South Africa and Korea for the qualitative China, Honduras, Zimbabwe, 100%; 3.0c detection of all Uruguay, and South Malawi, 99.7%; antibodies of all Korea, 100%, China, Honduras, isotypes (IgG, IgM, and Uruguay, and IgA) specific 100%; South to HIV-1, including Korea, 99.8% subtype O and HIV-2
14, at 2°-30°C
2.70
18, at 1°-30°C
1.10
One Step One Step HIV Test, HIV Testd,g Taiwang
In vitro, qualitative, Serum, whole NS Untried 15 rapid immunoblot blood assay for the detection of the antibodies to HIV-1 and/or HIV-2
18, at RT 1.00
HIV, human immunodeficiency virus; USD, United States dollars; IgG, immunoglobulin G; IgM, immunoglobulin M; IgA, immunoglobulin A; WHO, World Health Organization; RT, room temperature; NS, not specified. a HUMAN GmbH, Wiesbaden, Germany. b Alere Inc, Waltham, MA. c Standard Diagnostics Corporation, Kyonggi-Do, Korea. d OneStepHIVTest, Taichung, Taiwan. e Regular price paid by the Laboratory of Sexual Health in Peru from 2010 through 2011. f Needs Chase buffer. g Also promoted as a self-performed test.
The remaining 99 samples had negative results on the same 2 EIAs. The samples were collected as part of HIV-prevalence studies in high- and low-risk populations in Peru and were aliquoted and stored at -70°C. Samples were processed individually and tested by the following HIV POC devices: the Hexagon (HUMAN GmbH, Wiesbaden, Germany), Determine (Alere Inc, Waltham, MA), SD Bioline HIV-1/2 3.0 (Standard Diagnostics Corporation, Kyonggi-Do, Korea), and One Step (OneStepHIVTest, Taichung, Taiwan) (Table 1).
www.labmedicine.com
Tests were performed in duplicate according to the manufacturer’s instructions.
Statistical Analysis Data collected were analyzed using the Stata 10 Intercooler software, version 10.1 (StataCorp LLC, College Station, TX). The statistical tests we applied to data were calculation of sensitivity, specificity, and concordance among test results via kappa testing. Western blot results on each sample served as the reference values from which we interpreted the original status of the samples. Negative predictive values
Spring 2013 | Volume 44, Number 2 Lab Medicine
173
Laboratory QA
Table 2. Laboratory Performance of 4 Rapid Tests for HIV-1 and HIV-2 Diagnosis Using Results from Stored Serum Samples With Known HIV Status as Reference Values Rapid test No.
HIV +/ HIV -/ no.+a no.-b
False False Sensitivity Specificity PPV Negative Positive (%) (%) (%)
200 98/99 100/101 1 Hexagon HIVc Determine HIV 1/2d 200 98/99 99/101 1 SD Bioline HIV-1/2 3.0e 200 97/99 101/101 2 One Step HIV Testf 198 93/99 97/101 6
1 2 0 2
99.0 99.0 98.0 94.3
99.0 98.1 100.0 98.0
98.0 98.0 100.0 98.0
NPV (%)
Concordance (%) Kappa
99.0 99.0 98.5 98.5 99.0 99.0 95.0 95.0
0.98 0.97 0.98 0.91
n+ stands for all samples with known HIV-positive status, using the Western blot as a reference test. HIV+/no.+ represents the results of the tests performed. n- stands for all HIV known negative samples, using the Western Blot as a reference test. HIV-/no.- represents the results of the tests performed. HUMAN GmbH, Wiesbaden, Germany. d Alere Inc, Waltham, MA. e Standard Diagnostics Corporation, Kyonggi-Do, Korea f OneStepHIVTest, Taichung, Taiwan. a
b c
(NPVs) and positive predictive values (PPVs) were assessed by Bayesean analysis, considering a possible high prevalence of HIV infection of 30% and a low prevalence of infection of 0.5%.
false-positive results. We noted that samples with original weak EIA HIV test results yielded a low-quality spot or a less-highly-colored line via the assessed rapid tests.
Results
Discussion
In the 200 samples we randomly selected for further testing using 4 rapid EIA tests for laboratory diagnosis of HIV, we obtained 97.6% sensitivity and 98.8% specificity, on average. We calculated NPVs and PPVs; the kappa index between tests was 0.96 (Table 2). We also calculated sensitivity and specificity for each test, using WB results as the reference values. Finally, we measured agreement between the results of each test compared with the reference value of the sample (Table 2).
In this study, 2 rapid tests, the Hexagon HIV (HUMAN GmbH) and the Determine HIV 1/2 (Alere Inc), yielded 99% sensitivity; all 4 tests yielded at least 98% specificity. Based on Pan American Health Office recommendations,7 all 4 assessed rapid tests have adequate accuracy and comply with current guidelines.
Two tests, the Hexagon HIV (HUMAN GmbH) and the Determine HIV 1/2 (Alere Inc), yielded the most positive results; these results correlated well with the HIV reference result of the samples. All the assessed rapid-test results were negative in 1 sample that had originally yielded weak positive results for both EIAs and had fewer bands but still a positive result via WB testing. The One Step test (OneStepHIVTest) result was invalidated in 2 samples because the test performance was inadequate per the characteristics specified in the manufacturer’s instructions. One HIVnegative sample tested positive via the Hexagon test and the other 2 tests, the Determine and the One Step, also yielded a positive result; these samples had been processed in duplicate and were considered to have
174
Lab Medicine Spring 2013 | Volume 44, Number 2
Samples in this study with false-negative results are the first reported, to our knowledge, to have had a low optical density value via the results of previously performed regular HIV EIA testing (data not shown) but to still have positive results on WB testing. These results can be explained by different factors, including concentration and type of antigen on the membrane and concentration and quality of antibodies in the patient sample.8 Two samples also yielded positive results for HIV-2 when tested using the Hexagon HIV (HUMAN GmbH) and the SD Bioline HIV-1/2 3.0 (Anti-HIV 1/2; Standard Diagnostics Corporation), tests that differentiate HIV-1 and HIV-2 by providing 2 different colored bands. Although the bands showed a weak pattern, they still yielded positive results on the HIV-1 EIA and WB tests. The use of tests that detect antibodies to HIV-1 and HIV-2 is a common, nonregulated practice in Peru; however, the prevalence of HIV-2 is unknown because no current test is
www.labmedicine.com
Laboratory QA
available or in use for specific HIV-2 diagnosis in that country, to our knowledge. Further approaches to investigating the possibility of cross-reactions and, potentially, HIV-2 infection are being considered.9,10 In our study, 2 serum specimens were found to have false-positive results via SD Bioline HIV-1/2 3.0 testing (Standard Diagnostics Corporation), of which 1 sample had a false-positive result when tested with Determine (Alere Inc) and the other had a falsepositive result via Hexagon HIV (HUMAN GmbH; Table 2). According to UNAIDS and the World Health Organization (WHO), some tests with high specificity could yield false-positive results; these tests revealed greater than 99% specificity. Explanations for this observation include, but are not limited to, autoimmune diseases, hypergammaglobulinemia, vaccination, and concomitant active infections.11,12
To read this article online, scan the QR code, http://labmed. ascpjournals.org/content/44/2/172.full. pdf+html
QR code generated on http://qrcode.littleidiot.be
It is widely agreed in the medical community that efforts should be made to develop accurate, rapid tests for HIV diagnosis, especially for the detection of the p24 antigen and to differentiate between HIV-1 and HIV-2 infection. The window in which testing can be performed also needs to be shortened.5,13 Health care professionals need to implement rapid HIV testing in places where lack of resources, lack of facilities, and lack of trained personnel have become standard. Assessment of the performance of available tests for HIV diagnosis must become a routine practice in countries where regular EIA testing is not widely available. The performance of each of the 4 rapid HIV tests assessed in this study was acceptable for use in the rapid screening for HIV infection in low-resource settings. LM
Acknowledgments We thank the Laboratory of Sexual Health for providing access to its facilities and sample bank.
Conflict of Interest The conclusions and opinions expressed in this article are those of the authors and do not necessarily reflect those of the participating institutions. The authors declare no conflict of interest. All the materials and test kits were purchased by the Laboratory of Sexual Health.
www.labmedicine.com
References 1. Joint United Nations Programme on HIV/AIDS (UNAIDS). UNAIDS report on the global AIDS epidemic. 2011. http://www.unaids.org/ en/media/unaids/contentassets/documents/epidemiology/2012/ gr2012/20121120_UNAIDS_Global_Report_2012_en.pdf. Accessed on February 24, 2013. 2. McMichael AJ, Borrow P, Tomaras GD, Goonetilleke N, Haynes BF. The immune response during acute HIV-1 infection: clues for vaccine development. Nat Rev Immunol. 2010;10(1):11-23. 3. Parekh BS, Kalou MB, Alemnji G, Ou C-Y, Gershy-Damet G-M, Nkengasong JN. Scaling up HIV rapid testing in developing countries: comprehensive approach for implementing quality assurance. Am J Clin Pathol. 2010;134(4):573-584. 4. Yerly S, Hirschel B. Diagnosing acute HIV infection. Expert Rev Anti Infect Ther. 2012;10(1):31-41. 5. Peeling RW, Mabey D. Point-of-care tests for diagnosing infections in the developing world. Clin Microbiol Infect. 2010;16(8):1062-1069. 6. TDR Diagnostics Evaluation Expert Panel: Banoo S, Bell D, Bossuyt P, et al. Evaluation of diagnostic tests for infectious diseases: general principles. Nat Rev Microbiol. 2006;4(9 Suppl):S21-S31. 7. Guidelines for the Implementation of Reliable and Efficient Diagnostic HIV Testing. Washington, DC: Pan American Health Organization; 2008. 8. Ly TD, Martin L, Daghfal D, et al. Seven human immunodeficiency virus (HIV) antigen-antibody combination assays: evaluation of HIV seroconversion sensitivity and subtype detection. J Clin Microbiol. 2001;39(9):3122-3128. 9. Jafa K, Patel P, Duncan A, et al, for the OraQuick Study Group. Investigation of false positive results with an oral fluid rapid HIV-1/2 antibody test. PLoS ONE. 2007;2(1):e185. 10. O’Connell RJ, Agan BK, Anderson SA, Malia JA, Michael NL. Sensitivity of the Multispot HIV-1/HIV-2 Rapid Test using samples from human immunodeficiency virus type 1-positive individuals with various levels of exposure to highly active antiretroviral therapy. J Clin Microbiol. 2006;44(5):1831-1833. 11. UNAIDS/WHO Working Group on Global HIV/AIDS/STI Surveillance. Guidelines for Using HIV Testing Technologies in Surveillance: Selection, Evaluation, and Implementation; 2001. 12. UNAIDS/WHO Department of Essential Health Technologies. HIV Assays: Operational Characteristics–Report 15. 2004. 13. Pandori MW, Branson BM. 2010 HIV Diagnostics Conference. Expert Rev Anti Infect Ther. 2010;8(6):631-633.
Spring 2013 | Volume 44, Number 2 Lab Medicine
175
