First international collaborative study to evaluate ...

Author's Personal Copy Journal of Virological Methods 238 (2016) 77–85

Contents lists available at ScienceDirect

Journal of Virological Methods journal homepage: www.elsevier.com/locate/jviromet

First international collaborative study to evaluate rabies antibody detection method for use in monitoring the effectiveness of oral vaccination programmes in fox and raccoon dog in Europe M. Wasniewski a,∗ , I. Almeida b , A. Baur c , T. Bedekovic d , D. Boncea e , L.B. Chaves f , D. David g , P. De Benedictis h , M. Dobrostana i , P. Giraud j , P. Hostnik k , I. Jaceviciene l , S. Kenklies m , M. König n , K. Mähar o , M. Mojzis p , S. Moore q , S. Mrenoski r , T. Müller s , E. Ngoepe t , M. Nishimura u , T. Nokireki v , N. Pejovic w , M. Smreczak x , B. Strandbygaard y , E. Wodak z , F. Cliquet a ANSES − Nancy Laboratory for Rabies and Wildlife, Technopôle Agricole et Vétérinaire, CS 40009, 54220 Malzéville, France Laboratório Nacional de Investigação Veterinária (LNIV), Estrada de Benfica No 701, 1549-011 Lisboa, Portugal c Vet Med Labor GmbH, Division of IDEXX Laboratories, Mörikestr. 28/3, 71636 Ludwigsburg, Germany d Croatian Veterinary Institute Laboratory for Rabies/Virology, Savska cesta 143, Zagreb 10000, Croatia e Institute for Diagnosis and Animal Health, NRL For Rabies, no 63, Dr. Staicovici Street, sector 5 050557 Bucharest, Romania f Laboratório de Diagnóstico da Raiva, Instituto Pasteur − Secretaria de Estado da Saúde de São Paulo, Avenida Paulista, 393 − Cerqueira César, São Paulo/SP 01311-000, Brazil g Kimron Veterinary Institute Rabies Laboratory, Derech Hamacabim street, Bet Dagan 50250, Israel h Istituto Zooprofilattico Sperimentale delle Venezie, Viale dell’Università 10, 35020 Legnaro, Padova, Italy i Institute of Food Safety, Animal Health and Environment “BIOR” Animal Diseases Diagnostic Laboratory, Lejupes iela 3, LV-1076 Riga, Latvia j Laboratoire Départemental d’Analyses du Pas-de-Calais, Parc de Hautes technologies des Bonnettes 2, rue du genévrier, 62022 Arras cedex 2, France k National Veterinary Institute, Laboratory for Virology, Gerbiceva 60, 1 000 Ljubljana, Slovenia l National Food and Veterinary Risk Assessment Institute, Virology Unit, Kairiukscio Str. 10, LT-08409 Vilnius, Lithuania m Landesamt für Verbraucherschutz Sachsen-Anhalt, Fachbereich Veterinärmedizin, Haferbreiter Weg 132-135, 39576 Stendal, Germany n Institute of Virology, Faculty of Veterinary Medicine, JLU-Giessen, Schubertstr. 81, 35392 Giessen, Germany o Estonian Veterinary and Food Laboratory, Virology and Serology Department, Kreutzwaldi 30, 51 006 Tartu, Estonia p State Veterinary Institute Zvolen, Pod drahami 918, 960 86 Zvolen, Slovakia q Kansas State University Rabies Laboratory, 2005 Research Park Circle, Manhattan, KS 66502, USA r University Ss Cyril and Methodius in Skopje, Faculty of Veterinary Medicine in Skopje, Department for Microbiology and Immunology, Lazar Pop Trajkov 5-7, 1000 Skopje, Republic of Macedonia s Institute of Molecular Biology, Friedrich-Loeffler-Institute, Federal Research Institute for Animal Health, Südufer 10, 17493 Greifswald − Insel Riems, Germany t Agricultural Research Council-Onderstepoort Veterinary Institute (ARC-OVI), 100 old Soutpan road, Onderstepoort 0110 Pretoria, South Africa u Research Institute for Animal Science In Biochemistry and Toxicology, 3-7-11, Hashimotodai, Midori-ku, Sagamihara-Kanagawa 252-0132, Japan v Finnish Food Safety Authority, Evira Department Veterinary Virology, Mustialankatu, 3 00790 Helsinki, Finland w Diagnostic Veterinary Laboratory − Podgorica Bul. Dzordza Vasingtona, bb p.fah 69, 81000 Podgorica, Montenegro x National Veterinary Research Institute, Department of Virology, Partyzanow av. 57, 24-100 Pulawy, Poland y DTU, National Veterinary Institute Division of Virology, Lindholm Kalvehave Havnevej 51 DK- 4771 Kalvehave, Denmark z AGES, Institute for Veterinary Disease Control Mödling, Department for Virology Robert Koch Gasse 17 A-2340 Mödling, Austria a

b

a b s t r a c t Article history: Received 5 July 2016 Received in revised form 6 October 2016 Accepted 13 October 2016 Available online 14 October 2016 Keywords: ELISA Rabies Oral vaccination campaigns Collaborative study Rabies antibodies

The most effective and sustainable method to control and eliminate rabies in wildlife is the oral rabies vaccination (ORV) of target species, namely foxes and raccoon dogs in Europe. According to WHO and OIE, the effectiveness of oral vaccination campaigns should be regularly assessed via disease surveillance and ORV antibody monitoring. Rabies antibodies are generally screened for in field animal cadavers, whose body fluids are often of poor quality. Therefore, the use of alternative methods such as the enzyme-linked immunosorbent assay (ELISA) has been proposed to improve reliability of serological results obtained on wildlife samples. We undertook an international collaborative study to determine if the commercial BioPro ELISA Rabies Ab kit is a reliable and reproducible tool for rabies serological testing. Our results reveal that the overall specificity evaluated on naive samples reached 96.7%, and the coefficients of concordance obtained for fox and raccoon dog samples were 97.2% and 97.5%, respectively. The overall agreement values obtained for the four marketed oral vaccines used in Europe were all equal to or greater than 95%.

∗ Corresponding author. E-mail address: [email protected] (M. Wasniewski). http://dx.doi.org/10.1016/j.jviromet.2016.10.006 0166-0934/© 2016 Elsevier B.V. All rights reserved.

Author's Personal Copy 78

M. Wasniewski et al. / Journal of Virological Methods 238 (2016) 77–85

The coefficients of concordance obtained by laboratories ranged from 87.2% to 100%. The results of this collaborative study show good robustness and reproducibility of the BioPro ELISA Rabies Ab kit. © 2016 Elsevier B.V. All rights reserved.

1. Introduction Thanks to large scale oral rabies vaccination (ORV) of wildlife, in Europe, currently rabies only occurs in Southern and Eastern Europe, with the main reservoir and vector species being the red fox (Vulpes vulpes) (Cliquet et al., 2014; Muller et al., 2015). Several species of bats are also reservoirs for Lyssaviruses in Europe, involving distinct epidemiological cycles (EFSA AHAW Panel, 2015). In eastern Europe, raccoon dogs (Nyctereutes procyonoides) are also considered as another major reservoir host (Singer et al., 2008). The most effective and sustainable method to eliminate rabies in wildlife is the oral rabies vaccination (ORV) of target species, namely foxes and raccoon dogs (Cliquet and Aubert, 2004). ORV has eliminated rabies in 12 European countries (Cliquet and Aubert, 2004; Cliquet et al., 2014; Freuling et al., 2013) and is still currently being used in the great majority of rabies affected European countries (Borg, 2013). The effectiveness of ORV should be regularly assessed via disease surveillance of suspect animals and monitoring (Cliquet et al., 2010; EFSA AHAW Panel, 2015; OIE, 2013; WHO, 2013) of hunted target animals from vaccinated areas (Cliquet et al., 2010). The latter comprises (i) screening for a post mortem biomarker (tetracycline), incorporated in the vaccine bait, to verify bait uptake, and (ii) quantifying rabies specific antibodies to demonstrate adequate response to immunisation. The two current OIE-prescribed serological reference methods the fluorescent antibody virus neutralisation (FAVN) test and the rapid fluorescent focus inhibition test (RFFIT) (OIE, 2013) are based on cell culture and are therefore sensitive to any cytotoxic products and contaminating agents present in field samples (Cliquet et al., 2000). ELISAs are less time-consuming, easier techniques and preferred for assessing the serological response in countries carrying out mass vaccination of dogs and oral vaccination of wildlife (Barton and Campbell, 1988; Cliquet et al., 2012, 2000; Knoop et al., 2010; Singh et al., 2011; Suzuki et al., 2008; Yakobson et al., 2014; Zienius et al., 2014). At a European level, both seroneutralisation tests and ELISA tests are used interchangeably and on different types of samples collected for vaccination follow-up (De Benedictis et al., 2012). In 2013, an analysis related to rabies diagnosis and follow-up of ORV including 17 National Reference Laboratories (NRLs) showed that at least four different tests are currently used for serological testing in Europe (Robardet and Cliquet, 2014). The diversity of serological tests used and their differences in sensitivity and specificity values hinder the comparison of serological levels among countries (De Benedictis et al., 2012; Robardet and Cliquet, 2014). These differences are an obstacle to the appropriate evaluation of the serological tests used to monitor oral vaccination programmes in Europe by the European Commission. Moreover, to date, no interlaboratory test dedicated to serological tests used with regard to ORV has been implemented in Europe to assess laboratory performance (Mojzis et al., 2008; Wasniewski et al., 2013). The BioPro ELISA kit was developed as an alternative serological assay with the aim to check the effectiveness of ORV (Mojzis et al., 2008). The overall concordance between this kit and the FAVN test has been demonstrated in the red fox and the raccoon dog (Wasniewski et al., 2013). We organised a collaborative study involving numerous international laboratories to carry out an evaluation of the sensitivity and specificity of this method across

laboratories and to determine if this kit could be reliably used for serological testing for monitoring of ORV campaigns in Europe. Only non-haemolysed samples were tested to avoid any effect of sample quality on results and thus obtain a general and reliable evaluation of the performance of this kit. This study also assessed the large-scale reproducibility of this kit in 26 international laboratories testing the same panel of sera with the same technique.

2. Materials and methods Twenty-six laboratories received a panel of 41 coded samples for testing in three independent runs using the BioPro ELISA kit. The panel included samples from foxes and from raccoon dogs with different rabies antibody titres.

2.1. Material provided to participants 2.1.1. Ethical aspects Blood samples were collected at the ANSES-Nancy experimental station (Atton, Meurthe-et-Moselle département, France) which was approved by the French Veterinary Services on 19 April, 2011 (approval C-54-431-1). Silver foxes and raccoon dogs used originated from approved Finnish fur farms and were housed under similar conditions. The environment was enriched with branches, plastic balls and bones that were given alternatively. Experiments and husbandry were conducted in compliance with European Directive 2010/63/EU (Buzek and Chastel, 2010) and French regulations on ethics in animal experimentation.

2.1.2. Preparation of the samples Blood from each animal’s jugular vein was collected and centrifuged to obtain serum. Naive samples consisted of pooled sera from non-vaccinated animals to obtain sufficient volume for the collaborative study. Twelve raccoon dogs and six foxes were sampled and their sera were pooled to constitute one naive raccoon dog sample and one naive fox sample, respectively. Each serum was previously tested with the FAVN test to check for the absence of rabies antibodies before pooling. Moreover, a control sample containing only PBS buffer was included in the panel and was considered as a “naive sample” for the analysis of results. Sera from animals vaccinated against rabies were included as test sera. One fox and one raccoon dog were vaccinated with one of the four marketed vaccines currently used in Europe for ORV: Fuchsoral (IDT), Lysvulpen (Bioveta), Rabigen SAG2 (Virbac) and Raboral V-RG (Merial). The first three vaccines are modified live virus vaccines based on the Street Alabama Dufferin (SAD) strain (Cliquet and Aubert, 2004). Another vaccine, Raboral V-RG (Merial), a genetically engineered recombinant vaccine, (Cliquet and Aubert, 2004) was also used. The four vaccines comply with national regulations for vaccines. For each animal species and each vaccine, the following sera was included: one strong positive serum, one weak positive serum and two negative sera. To obtain the different categories of sera, the sampled serum, collected from vaccinated animal, was diluted in phosphate buffered saline (PBS) solution.

Author's Personal Copy M. Wasniewski et al. / Journal of Virological Methods 238 (2016) 77–85

79

Table 1 Description of the 41 samples included in the panel to evaluate the specificity (naive samples) and the coefficient of concordance (negative samples and positive samples). Performance evaluation

Species

Oral vaccine

Sample status

Number of samples included in the panel

Total number of samples tested

Specificity

Fox Raccoon dog PBS

/ / /

Naive Naive Naive

2 2 1

146 146 73

Coefficient of concordance

Fox

Fuchsoral Lysvulpen Rabigen SAG2 Raboral V-RG Fuchsoral Lysvulpen Rabigen SAG2 Raboral V-RG Fuchsoral

Negative Negative Negative

2 2 2

146 146 146

Negative Negative Negative Negative

2 2 2 2

137 146 146 146

Negative Weak positive Strong positive Weak positive Strong positive Weak positive

2 1 1 1 1 1

146 73 73 73 73 73

Strong positive Weak positive Strong positive Weak positive Strong positive Weak positive Strong positive Weak positive

1 1 1 1 1 1 1 1

73 73 73 73 73 73 73 73

Strong positive Weak positive Strong positive

1 1 1

73 73 73

Raccoon dog

Fox

Lysvulpen Rabigen SAG2 Raboral V-RG Raccoon dog

Fuchsoral Lysvulpen Rabigen SAG2 Raboral V-RG

2.1.3. Preparation of the panel The panel of the collaborative study was composed of 41 samples (see Table 1) and was previously tested in the ANSES-Nancy laboratory using the FAVN test and the BioPro ELISA test to assess their consistency before aliquoting. The homogeneity and stability of the panel were checked using the BioPro ELISA test and results showed that all items were found to be sufficiently homogeneous and stable (data not shown). Two naive fox samples (duplicate), two naive raccoon dog samples (duplicate), eight negative samples (duplicate) and one PBS sample were included to evaluate the specificity of the ELISA kit (Table 1). A total of 16 positive samples was included in the panel to assess the sensitivity of the ELISA kit (Table 1) with strong positive titres (FAVN test results ranging from 4.56 IU/mL to 23.93 IU/mL), and weak positive titres (FAVN test titres ranging from 1.08 IU/mL to 3.25 IU/mL).

2.1.4. Identification of the samples For each panel, samples were coded randomly. The sample code denoted the study, the laboratory (each laboratory was assigned a number) and the individual sample reference.

2.2. Methods 2.2.1. FAVN test Rabies-neutralising antibodies were determined with the FAVN test essentially as described (Cliquet et al., 1998); (Wasniewski et al., 2013) The neutralising titres were expressed in international units per millilitre (IU/mL) by comparing results obtained with the test serum with those of the positive standard. According to previous studies using the same method, the threshold of positivity was 0.24 IU/mL (Cliquet et al., 2000; Wasniewski et al., 2013).

2.2.2. ELISA test The participating laboratories were asked to titrate the samples with the BioPro ELISA kit according to the manufacturer’s recommendations. This test is a blocking ELISA that detects rabies virus antibodies as previously described (Wasniewski and Cliquet, 2012). The BioPro ELISA Rabies Ab kits (BioPro ELISA) with reagents were sent directly by BioPro (Prague, Czech Republic) to the participating laboratories. The conditions of validation described by the manufacturer had to be met to validate the results obtained for the different samples. The panel of positive control sera (CS1, CS2 and CS3) ensured that the test was working in optimal conditions. The percentage of blocking (%PB) was calculated for each sample according to the manufacturer’s recommendations (i.e. %PB = [(ODNC-ODsample)/(ODNC-ODPC)] × 100) where ODNC is the optical density of the negative control, ODPC the optical density of the positive control and OD sample is the optical density of the sample. The positivity threshold recommended by the manufacturer for assessing the effectiveness of ORV is 40%.

2.3. Study design and participants 2.3.1. Participants Twenty-six laboratories from 22 countries (17 from Europe, 2 from America, 2 from Asia and 1 from Africa, see Materials and methods, Section 2.3.1) participated in this study (see the names of the laboratories in the list of authors’ affiliations). Twenty-five of the participating laboratories were approved for rabies serological testing with regard to international trade (http://ec.europa.eu/ food/animal/liveanimals/pets/approval en.htm). Each laboratory was randomly assigned a number which does not necessarily represent the order as listed above in the author list.



2.3.2. Instructions to participants A detailed protocol of the study was sent with the panel. In addition, a questionnaire was sent out to every participant to solicit their views regarding the advantages/disadvantages of this kit. The laboratories were instructed to store the samples constituting the panel at −20 ◦ C until testing. The participating laboratories were invited to test the panel in three independent runs using the BioPro Rabies ELISA Ab kit. Each sample was to be tested in one well only. Between two independent tests, the samples could be stored at +4 ◦ C for up to one week but at −20 ◦ C for a longer time. The equipment used in laboratories, such as plate washers and spectrophotometers, were standardised among laboratories because most participating laboratories perform ELISA techniques for other diseases. Each laboratory had to return its results within two months after receiving the panel by filling out the result form sent with the panel. 2.3.3. Duration of the study The majority of the panels were sent from Nancy (France) to participating laboratories between 23 October, 2012 and 29 October, 2012. Three panels were sent later on 15 November, 2012, 05 December, 2012 and 01 April, 2013 due to lengthy customs procedures. The last results of this collaborative study were received in ANSES-Nancy on 03 June, 2013. 2.4. Statistical analysis 2.4.1. Type of data Each participating laboratory had to report results on a sheet provided by the ANSES-Nancy and send it back to the ANSES-Nancy. For each serum tested, the ELISA results had to be expressed in OD values and in terms of positive or negative result according to the percentage of blocking obtained. To check for calculation errors between the OD value and the final result (positive or negative), all OD values from each participating laboratory were recorded in a spreadsheet to convert them to %PB to obtain a final “positive” or “negative” result for each sample. This also allowed the ANSES-Nancy laboratory to compare and analyse the results in an objective and uniform way. Prior to its use, this spreadsheet was validated against the calculation file developed by BioPro for its ELISA test and available on request on its website (http://www.biopro.cz/en/Diagnostics/BioPro-RabiesELISA-Ab-kit/). 2.4.2. Statistical methods Specificity was assessed by checking that laboratories tested the five naive samples as negative. It was expressed as the percentage of true negative results (%PB below 40% in ELISA) obtained for these five naive samples. Specificity was evaluated for naive samples, per species and per laboratory. Diagnostic specificity was expressed as the percentage of true negative results (%PB value below 40% in ELISA) on sera of vaccinated animals. Diagnostic sensitivity was expressed as the percentage of true positive results obtained for each laboratory compared to the expected values previously determined with the FAVN test and the ELISA BioPro kit at ANSES-Nancy. The coefficient of concordance was expressed as the percentage of true positive results (%PB value equal or above 40% in ELISA) plus the true negative results (%PB value below 40% in ELISA) obtained by ELISA compared with the expected values previously determined with the FAVN test and the ELISA BioPro kit at ANSES-Nancy. The coefficient of concordance was evaluated for all samples, per species, per oral vaccine and per laboratory. For all variables, the 95% confidence intervals (CI) were determined.

The Bernouilli test (␣ = 0.05) was used to compare proportions obtained between fox and raccoon dog groups. The Chi2 test (p = 0.05) was used to compare proportions obtained between the different groups of samples resulting from vaccination with the four oral vaccines. A Kruskal-Wallis test was performed to compare means of the three categories of samples: naive samples, negative samples and positive samples. 3. Results 3.1. Analysis of the performances of the BioPro ELISA kit 3.1.1. Analysis of the results obtained for the positive and negative controls Of 78 runs (26 × 3) performed by participating laboratories, 4 runs were not validated according to the values obtained for the positive and negative controls (runs 1, 2 and 3 for laboratory L20 and run 1 for L25). As these values did not meet the conditions of validation required by the manufacturer, these runs were not included in the statistical analysis. Laboratory L12 performed only two runs instead of three. A total of 73 runs were analysed. 3.1.2. Overall results 3.1.2.1. Specificity. On the five samples (see Table 1) containing no rabies antibody (PBS and sera from naive foxes and raccoon dogs), tested in 73 runs using the BioPro ELISA kit in participating laboratories, a positive titre was obtained 12 times (L01, L03, L07 and L16). Specificity was therefore 96.7%. 3.1.2.2. Coefficient of concordance. Of the 37 samples tested 73 times each in 25 participating laboratories, (16 negative samples + 5 naive samples + 16 positive samples), a true negative titre was obtained 1458 times (350 times for naive samples and 1108 times for negative samples) and a true positive titre was obtained 1161 times. Therefore the coefficient of concordance was 97.3%. Fig. 1 shows the frequency distribution (in percentage in each group) for the three categories of samples tested by ELISA by participating laboratories according to the FAVN test titres determined by ANSES-Nancy: the naive samples (n = 365), the negative samples (n = 1159) and the positive samples (weak and strong) (n = 1168). The results obtained for the four groups were significantly different (Kruskal-Wallis test). This test (␣ = 0.05) significantly discriminated naive (n = 365) and negative samples (n = 1159) from weak (n = 584) and strong positive samples (n = 584). The naive and negative groups were not significantly different whereas the weak and strong positive groups differed significantly. These three categories, i.e. naive and negative groups, weak positive group and strong positive group could be well discriminated. 3.1.3. Fox samples (Table 2) Table 2 indicates the percentage of specificity obtained per species as well as the coefficient of concordance. 3.1.3.1. Specificity. Of the two samples containing no rabies antibody, 4 out of 73 runs gave a positive result with%PB values of 44.6%, 60.4%, 73.3% and 73.3% (L03, L07 and L16), giving a specificity of 97.3%. 3.1.3.2. Coefficient of concordance. Of 1305 tested samples, 692 (142 naive samples + 550 negative samples) were true negatives while 577 were true positives giving a coefficient of concordance of 97.2%. Of the eight samples with a positive rabies antibody titre, a negative titre was obtained in seven of 73 runs (L05, L12, L13, L19 and L22). For the eight samples with a negative rabies antibody titre


81

Fig. 1. Frequency distribution of percentage of blocking values obtained for the three sample categories: naïve samples, negative samples and positive samples. Each vertical bar represents the frequency (percentage) within the group.

Table 2 Results for specificity and coefficient of concordance according to animal species.

Fox

Raccoon dog

Sample category

Criterion

Number of samples

Number of concordant samples

Coefficient of concordance (%)

95%CI

Laboratories showing discordant results

Naive samples Negative samples

Specificity

146

142

97.3

94.67–99.93%

L03, L07, L16

Diagnostic specificity

575

550

95.6

93.92–97.28%

Positive samples Total

Diagnostic sensitivity Coefficient of concordance

584

577

98.8

97.92–99.68%

1305

1269

97.2

96.30–98.10%

L01, L03, L06, L07, L13, L14, L16, L17, L19, L22 L05, L12, L13, L19, L22 L01, L03, L05, L06, L07,L12, L13, L14, L16, L17, L19, L22

Naive samples Negative samples

Specificity

146

139

95.2

91.73–98.67%

L01, L03, L07, L16

Diagnostic specificity

584

558

95.6

93.94–97.26%

Diagnostic sensitivity Coefficient of concordance

584

584

100

100%

L01, L03, L05, L13, L14, L15, L16, L17, L22, L25 /

1314

1281

97.5

96.66–98.34%

Positive samples Total

tested in 73 runs, a positive titre was obtained in 25 runs (L01, L03, L06, L07, L13, L14, L16, L17, L19 and L22). 3.1.4. Raccoon dog samples (Table 2) 3.1.4.1. Specificity. Two of the samples without rabies antibody were tested in 73 runs, a positive titre was obtained for seven samples with %PB values ranging from 41.6% to 80.2% (L01, L03, L07 and L16). Specificity was thus 95.2%. 3.1.4.2. Coefficient of concordance. Among the 1314 samples tested, 697 (139 naive samples + 558 negative samples) were true neg-

L01, L03, L05, L13, L14, L15, L16, L17, L22, L25

atives and 584 were true positives; therefore the coefficient of concordance was 97.5%. Among the eight samples with a negative rabies antibody titre tested in 73 runs, a positive titre was obtained 26 times (L01, L03, L05, L13, L14, L15, L16, L17, L22 and L25). Among the eight samples with a positive rabies antibody titre tested in 73 runs, no negative titres were observed. A Chi2 test (p = 0.05) comparing the results obtained for the three categories of naive samples (fox, raccoon dog and PBS samples) revealed that there was no significant difference between these three groups.



3.2.1. Specificity Twenty-one laboratories (84%) obtained 100% specificity. The four remaining laboratories (L01, L03, L07 and L16) showed specificities ranging from 66.7% to 93.3%. 3.2.2. Diagnostic specificity and sensitivity Twelve laboratories (48%) obtained 100% diagnostic specificity. The remaining 13 laboratories (L01, L03, L05, L06, L07, L13, L14, L15, L16, L17, L19, L22 and L25) showed diagnostic specificities ranging from 81.3% to 97.9%. Twenty laboratories (80%) obtained 100% for the sensitivity criterion regardless of the status of the positive samples (weak or strong positive). The remaining five laboratories (L05, L12, L13, L19 and L22) showed sensitivities ranging from 95.8% to 97.9%. 3.2.3. Coefficient of concordance Eleven laboratories (44%) obtained a coefficient of concordance of 100%. The remaining 14 laboratories (L01, L03, L05, L06, L07, L12, L13, L14, L15, L16, L17, L19, L22 and L25) had coefficients of concordance between 87.2% and 98.6%. As shown in Fig. 2, although several laboratories did not reach 100% for all four criteria, they obtained values greater than or equal to 80% for specificity, diagnostic specificity and sensitivity, except only one laboratory that had a specificity value lower than 80%. 4. Discussion Fig. 2. Performances of the BioPro ELISA obtained by the participating laboratories on the same panel of samples.

Bernouilli tests (␣ = 0.05) were performed to compare the proportions obtained for fox and raccoon dog samples. There was no significant difference between these two groups for negative samples and coefficients of concordance. 3.1.5. Vaccines (Table 3) Table 3 gives the specificity and the coefficient of concordance according to the vaccine used for immunisation. 3.1.5.1. Coefficient of concordance. Of the 283 samples with a negative rabies antibody titre, a positive titre was observed in 23 samples (L01, L03, L06, L13, L15, L16, L17, L19 and L22) for the Raboral V-RG vaccine. Among the 292 samples with a positive rabies antibody titre, no negative titres were observed for the Fuchsoral and Lysvulpen vaccines. Among these 292 samples, a negative titre was observed in one sample (weak positive sample) (L12) for the Rabigen SAG2 vaccine and in six strong positive samples (L05, L13, L19 and L22) for the Raboral V-RG vaccine. For the four oral vaccines, the coefficients of concordance were 99.3% (Fuchsoral vaccine), 98.1% (Lysvulpen vaccine), 97.6% (Rabigen SAG2 vaccine) and 95% (Raboral V-RG vaccine), respectively. Comparisons of the proportions obtained for the four oral vaccines revealed that the coefficient of concordance was significantly lower for results on Raboral V-RG samples from vaccinated animals compared with those from Fuchsoral, Lysvulpen or Rabigen SAG2 vaccinations (Chi2 (p = 0.05)). There was no significant difference between samples from animals vaccinated with Fuchsoral, Lysvulpen or Rabigen SAG2 for this criterion. 3.2. Analysis of the performance of the BioPro ELISA kit per participating laboratory The results obtained for the three runs were analysed as a whole for each participating laboratory in terms of specificity, diagnostic specificity and sensitivity and coefficient of concordance (Fig. 2).

Rabies serology tests are used to evaluate the effectiveness of oral vaccination campaigns. To date, only two seroneutralisation methods are recommended by the WHO and OIE for monitoring the level of rabies antibodies in domestic carnivores and wildlife: the FAVN test (Cliquet et al., 1998) and the RFFIT (Smith et al., 1973). These tests are considered as gold standards for rabies serology but their complexity makes them more difficult to perform than simpler antibody assays such as ELISA. The latter has been previously assessed as a reliable and consistent method, particularly for monitoring ORV campaigns of wildlife. Moreover, the FAVN and RFFIT neutralisation tests are not adapted for large-scale screening of body fluids taken from animal cadavers that can be cytotoxic and available only in small quantities (Bedekovic et al., 2013; De Benedictis et al., 2012). In contrast to serological testing of individual animals in the frame of animal trade or the pet travel scheme, where individual virus neutralising antibody titres are correlated with protective immunity of the respective animal, for evaluation of the efficacy of ORV campaigns determination of herd immunity is crucial. For these reasons, several ELISA tests have been developed as an alternative to virus neutralisation tests for animals (Barton and Campbell, 1988; Cliquet et al., 2000; De Benedictis et al., 2012; Servat et al., 2007; Singh et al., 2011; Vengust et al., 2011; Wasniewski et al., 2014). These alternative assays, however, need to be carefully evaluated against the current reference techniques prior to use. A recent review of serological tests used within Europe highlighted the diversity, variations in sensitivity and specificity of ELISA kits, which may hinder the comparison of serological levels among countries involved in ORV (De Benedictis et al., 2012 Robardet and Cliquet, 2014). To date, there have been no interlaboratory tests dedicated to the serological tests used with regard to ORV in European countries to assess laboratory performance. This raises the issue of harmonisation of serological testing used to monitor ORV programmes in Europe. The BioPro ELISA kit was initially developed to assess the effectiveness of the ORV campaign in achieving rabies antibodies in foxes (Mojzis et al., 2008). The overall concordance between the BioPro ELISA kit and the reference FAVN test has been previously assessed both in the red fox and the raccoon dog (Wasniewski et al., 2013). We therefore organised an international collaborative study


83

Table 3 Results for the coefficient of concordance according to vaccine used for animal immunisation. Sample category

Fuchsoral vaccine

Lysvulpen vaccine

Negative samples Positive samples Total Negative samples Positive samples Total

Rabigen SAG2 vaccine

Negative samples Positive samples Total

Raboral V-RG vaccine

Negative samples Positive samples Total

Criterion





Number of samples

Number of samples found concordant

Coefficient of concordance (%)

95%CI

292 292 584

288 292 580

98.6 100 99.3

97.25–99.95% 100% 98.62–99.98%

292 292 584

281 292 573

96.2 100 98.1

94.01–98.39% 100% 96.99–99.21%

292 292 584

279 291 570

95.6 99.7 97.6

93.25–97.95% 99.07–100.33% 96.36–98.84%

283 292 575

260 286 546

91.9 98 95

88.72–95.08% 96.39–99.61% 93.22–96.78%

to complete the validation, in terms of robustness and reproducibility, of this ELISA kit as a potential alternative to seroneutralisation assays for ORV monitoring. This type of approach is frequently organised for other disease surveillance programmes to collect information on the performance accuracy of serological diagnostic tests that laboratories use (Sanchini et al., 2013), to determine disease prevalence (Peyraud et al., 2014), to test a new method or new material and demonstrate its applicability, suitability and validity (Bedrina et al., 2013; Cliquet et al., 2003; Gross et al., 2010; Mott and Dixon, 1982; Servat and Cliquet, 2006; Wasniewski et al., 2014), to improve the reliability and reproducibility of assays (Kapasi et al., 2012; Servat et al., 2008; Xing et al., 2011), to compare different ELISA tests (Lynn et al., 1996), to validate a common Standard Operating Procedure to be used in reference laboratories (Eklund et al., 2012) or to characterize reference material (Lyng, 1994; Lyng et al., 1992; Xing et al., 2009). The data collected in this international study corroborate those obtained previously (Mojzis et al., 2008; Wasniewski et al., 2013) and therefore demonstrate that this ELISA kit has a highly satisfactory performance. The overall specificity evaluated on naive raccoon dog and fox samples as well as on PBS control reached 96.7%. In particular, the specificity was 98.6% for the PBS control and 97.3% and 95.2% for the fox and raccoon dog samples, respectively. The results obtained for these three categories (PBS, naive fox and raccoon dog samples) were not significantly different. The 12 false positive results were observed in only four laboratories (L01, L03, L07 and L16) and six values were very close to the threshold of positivity of 40% (%PB value ranging from 41.6% to 45%). These results are slightly different from those obtained for domestic and wild carnivores in which specificity reached 100% for both populations (Wasniewski and Cliquet, 2012; Wasniewski et al., 2013). In 2000, our group developed and validated an in-house ELISA method with a specificity equal to 95.4%, which was considered as satisfactory for monitoring herd immunity of wild carnivores vaccinated against rabies (Cliquet et al., 2000). The intra-laboratory repeatability was also assessed during this collaborative study and results obtained were satisfactory proving the good reliability of this ELISA kit. Indeed, coefficients of variation were satisfactory (not greater than 8%) for 70% of runs (data not shown). The coefficients of concordance obtained for fox and raccoon dog samples were 97.2% and 97.5% respectively. These values are satisfactory but slightly higher than those previously obtained close to 95% (Wasniewski et al., 2013). This high concordance can

be attributed to the good quality of samples used here. In this study, only non-haemolysed samples were analysed, whereas in other studies, most samples were collected from the field after ORV (in France and Estonia), and these samples showed various degrees of haemolysis. These concordance values are also higher than those found using an in-house ELISA where the agreement for distinguishing positive and negative titres in a fox population was 93%, considered as sufficiently high to monitor the efficacy of ORV (Cliquet et al., 2000). Even if the FAVN and RFFIT seroneutralisation methods are considered as gold standards, they have their own inherent variability that may explain the difference in sensitivity compared with the ELISA tests. Moreover, ELISA tests detect binding antibodies, not neutralising antibodies, which may also explain the discrepancies obtained between the two methods (De Benedictis et al., 2012). During other collaborative studies, the results obtained were less satisfactory as expected according to the performance validation data. A similar situation was noted during a collaborative study carried out to evaluate the Serelisa kit (Synbiotics, France) on domestic carnivores (Servat and Cliquet, 2006). For example, significantly different results were obtained for diagnostic specificity and sensitivity criteria between a collaborative study including 16 laboratories and performance validation data previously obtained in two laboratories involved (Cliquet et al., 2004). Similarly, the evaluation of the Platelia Rabies II kit (Bio-Rad, France) also gave significantly different results on diagnostic specificity and sensitivity during a large-scale study on pet domestic carnivore samples, involving 23 international laboratories (Wasniewski et al., 2014) compared with the data obtained during the validation of the kit (Servat et al., 2007, 2008), resulting in the unsatisfactory performance of this kit. However, discrepancies during large-scale evaluations of the performances of the Serelisa and Platelia rabies II kits were in much higher proportion than those observed here for the BioPro ELISA kit. The overall agreement obtained for the four oral vaccines marketed in Europe were equal or greater than 95% (99.3% for Fuchsoral, 98.1% for Lysvulpen, 97.6% for Rabigen SAG2 and 95% for Raboral V-RG), a very satisfactory result. The coefficient of concordance was significantly lower for samples issued from animals vaccinated with Raboral V-RG compared with those obtained with Fuchsoral, Lysvulpen or Rabigen SAG2 vaccinations. This may be attributed to the fact that Raboral V-RG is a recombinant vaccinia virus, expressing the rabies glycoprotein of the ERA rabies virus (Kieny et al., 1984) and not an attenuated live virus strain.



Regarding the results obtained in each laboratory, 84% of participating laboratories obtained specificities of 100% on naive samples. For diagnostic specificity, carried out on negative samples, 48% of participating laboratories obtained 100% and the remaining 52% observed specificities between 81.3% and 97.9%. This result is satisfactory compared to those obtained for the collaborative study for the Serelisa kit, in which only 25% of participating laboratories obtained 100% for diagnostic specificity and 50% obtained a percentage between 85% and 96.7% (Servat and Cliquet, 2006). However, for the collaborative study on the Platelia Rabies II kit, diagnostic specificity was higher, with 65.2% of participating laboratories reaching 100% and 30.43% observing values between 86.2% and 96.7% (Wasniewski et al., 2014). For diagnostic sensitivity, 80% of participating laboratories obtained 100%, whereas the remaining five laboratories observed sensitivities between 95.8% and 97.9%. The results are better than those for the collaborative study on the Platelia Rabies II kit, in which only 43.5% of laboratories obtained a diagnostic sensitivity above or equal to 80% (Wasniewski et al., 2014). Laboratories L01, L03, L06 and L07 showed discrepancies in specificity and diagnostic specificity. Laboratories L05, L13, L19 and L22 showed discrepancies in diagnostic specificity and diagnostic sensitivity. Based on the questionnaires filled out by participants (data not shown), most laboratories favourably viewed the use of this ELISA kit, indicating that it is fast and very easy to handle, although a few laboratories disliked the overnight incubation and the requirement for plate shakers at different temperatures. 5. Conclusion Although the results obtained for this study were satisfactory, additional studies are necessary to evaluate the impact of the biological status of the field samples on the performance of the BioPro ELISA Rabies Ab kit. It should be important to consider other target species such as raccoon, skunks and dogs, as well. The results obtained in the 25 participating laboratories during this collaborative study showed that BioPro ELISA Rabies Ab kit provides satisfactory robustness and reproducibility. This kit seems to be therefore a good candidate for assessing the rabies antibody level in wildlife samples when monitoring the effectiveness of ORV campaigns in Europe. Conflict of interest The authors declare that they have no competing interests. Acknowledgments We would like to thank all laboratories that kindly took part in this study, the four oral vaccine production companies (Bioveta, IDT, Merial and Virbac), as well as the staff at the ANSES-Nancy serology-virology and experimental station in Atton. References Barton, L.D., Campbell, J.B., 1988. Measurement of rabies-specific antibodies in carnivores by an enzyme-linked immunosorbent assay. J. Wildl. Dis. 24, 246–258. Bedekovic, T., Lemo, N., Lojkic, I., Mihaljevic, Z., Jungic, A., Cvetnic, Z., Cac, Z., Hostnik, P., 2013. Modification of the fluorescent antibody virus neutralisation test-elimination of the cytotoxic effect for the detection of rabies virus neutralising antibodies. J. Virol. Methods 189, 204–208. Bedrina, B., Macian, S., Solis, I., Fernandez-Lafuente, R., Baldrich, E., Rodriguez, G., 2013. Fast immunosensing technique to detect Legionella pneumophila in different natural and anthropogenic environments: comparative and collaborative trials. BMC Microbiol. 13, 88. Borg, T., 2013. Commission implementing decision of 29 November 2013 approving annual and mutiannual programmes and the financial contribution from the Union for the eradication, control and monitoring of certain animal

diseases and zoonoses presented by the member states for 2014 and the following years. Off. J. Eur. Union, 101–110. Buzek, J., Chastel, O., 2010. Directive 2010/63/EU of the European parliament and of the council of 22 September 2010 on the protection of animals used for scientific purposes. Off. J. Eur. Union, 33–79. Cliquet, F., Aubert, M., 2004. Elimination of terrestrial rabies in Western European countries. In: Schudel, A., Lombard, M. (Eds.), Control of Infectious Animal Diseases by Vaccination. Karger, Basel, pp. 185–204. Cliquet, F., Aubert, M., Sagne, L., 1998. Development of a fluorescent antibody virus neutralisation test (FAVN test) for the quantitation of rabies-neutralising antibody. J. Immunol. Methods 212, 79–87. Cliquet, F., Sagne, L., Schereffer, J.L., Aubert, M.F.A., 2000. ELISA test for rabies antibody titration in orally vaccinated foxes sampled in the fields. Vaccine 18, 3272–3279. Cliquet, F., Muller, T., Mutinelli, F., Geronutti, S., Brochier, B., Selhorst, T., Schereffer, J.L., Krafft, N., Burow, J., Schameitat, A., Schluter, H., Aubert, M., 2003. Standardisation and establishment of a rabies ELISA test in European laboratories for assessing the efficacy of oral fox vaccination campaigns. Vaccine 21, 2986–2993. Cliquet, F., McElhinney, L.M., Servat, A., Boucher, J.M., Lowings, J.P., Goddard, T., Mansfield, K.L., Fooks, A.R., 2004. Development of a qualitative indirect ELISA for the measurement of rabies virus-specific antibodies from vaccinated dogs and cats. J. Virol. Methods 117, 1–8. Cliquet, F., Freuling, C., Smreczak, M., van der Poel, W.H.M., Horton, D., Fooks, A.R., Robardet, E., Picard-Meyer, E., Muller, T., 2010. Development of harmonised schemes for monitoring and reporting of rabies in animals in the European Union. EFSA Sci. Rep. (60 p). Cliquet, F., Robardet, E., Must, K., Laine, M., Peik, K., Picard-Meyer, E., Guiot, A.L., Niin, E., 2012. Eliminating rabies in Estonia. PLoS Negl. Trop. Dis. 6, 1–17. Cliquet, F., Picard-Meyer, E., Robardet, E., 2014. Rabies in Europe: what are the risks? Expert Rev. Anti Infect. Ther. 12, 905–908. De Benedictis, P., Mancin, M., Cattoli, G., Capua, I., Terrregino, C., 2012. Serological methods used for rabies post vaccination surveys: an analysis. Vaccine 30, 5611–5615. EFSA AHAW Panel, 2015. Update on oral vaccination of foxes and raccoon dogs against rabies. Scientific Panel on Animal Health and Welfare (AHAW). EFSA J. 13 (70 p). Eklund, C., Unger, E.R., Nardelli-Haefliger, D., Zhou, T., Dillner, J., 2012. International collaborative proficiency study of human papillomavirus type 16 serology. Vaccine 30, 294–299. Freuling, C.M., Hampson, K., Selhorst, T., Schroder, R., Meslin, F.X., Mettenleiter, T.C., Muller, T., 2013. The elimination of fox rabies from Europe: determinants of success and lessons for the future. Philos. Trans. R Soc. Lon. Ser. B Biol. Sci. 368, 20120142. Gross, S., Janssen, S.W., de Vries, B., Terao, E., Daas, A., Buchheit, K.H., 2010. Collaborative study for the validation of alternative in vitro potency assays for human tetanus immunoglobulins. Biologicals 38, 501–510. Kapasi, A., Meade, B.D., Plikaytis, B., Pawloski, L., Martin, M.D., Yoder, S., Rock, M.T., Coddens, S., Haezebroeck, V., Fievet-Groyne, F., Bixler, G., Jones, C., Hildreth, S., Edwards, K.M., Messonnier, N.E., Tondella, M.L., 2012. Comparative study of different sources of pertussis toxin (PT) as coating antigens in IgG anti-PT enzyme-linked immunosorbent assays. Clin. Vaccine Immunol. 19, 64–72. Kieny, M.P., Lathe, R., Drillien, R., Spehner, D., Skory, S., Schmitt, D., Wiktor, T., Koprowski, H., Lecocq, J.P., 1984. Expression of rabies virus glycoprotein from a recombinant vaccinia virus. Nature 312, 163–166. Knoop, E.V., Freuling, C.M., Kliemt, J., Selhorst, T., Conraths, F.J., Muller, T., 2010. Evaluation of a commercial rabies ELISA as a replacement for serum neutralization assays as part of the pet travel scheme and oral vaccination campaigns of foxes Berl. Münch. Tierärztl. Wochenschr. 123, 278–285. Lyng, J., Bentzon, M.W., Ferguson, M., Fitzgerald, E.A., 1992. Rabies vaccine standardization: international collaborative study for the characterization of the fifth international standard for rabies vaccine. Biologicals 20, 301–313. Lyng, J., 1994. Calibration of a replacement preparation for the International Standard for Rabies Immunoglobulin. Biologicals 22, 249–255. Lynn, F., Reed, G.F., Meade, B.D., 1996. Collaborative study for the evaluation of enzyme-linked immunosorbent assays used to measure human antibodies to Bordetella pertussis antigens. Clin. Diagn. Lab. Immunol. 3, 689–700. Mojzis, M., Korytar, P., Jerg, S., 2008. Development and validation of ELISA test for detection of rabies anti-glycoprotein antibodies. In: International Conference on Rabies in the Americas (RITA XIX), Atlanta, pp. 48–49. Mott, K.E., Dixon, H., 1982. Collaborative study on antigens for immunodiagnosis of schistosomiasis. Bull. World Health Organ. 60, 729–753. Muller, T.F., Schroder, R., Wysocki, P., Mettenleiter, T.C., Freuling, C.M., 2015. Spatio-temporal use of oral rabies vaccines in fox rabies elimination programmes in Europe. PLoS Negl. Trop. Dis. 9, e0003953. OIE, 2013. Rabies. In: Manual of Diagnostic Tests and Vaccines for Terrestrial Animals (mammals, Birds and Bees), Online Version. OIE, Paris. Peyraud, A., Poumarat, F., Tardy, F., Manso-Silvan, L., Hamroev, K., Tilloev, T., Amirbekov, M., Tounkara, K., Bodjo, C., Wesonga, H., Nkando, I.G., Jenberie, S., Yami, M., Cardinale, E., Meenowa, D., Jaumally, M.R., Yaqub, T., Shabbir, M.Z., Mukhtar, N., Halimi, M., Ziay, G.M., Schauwers, W., Noori, H., Rajabi, A.M., Ostrowski, S., Thiaucourt, F., 2014. An international collaborative study to determine the prevalence of contagious caprine pleuropneumonia by monoclonal antibody-based cELISA. BMC Vet. Res. 10, 48.

Author's Personal Copy M. Wasniewski et al. / Journal of Virological Methods 238 (2016) 77–85 Robardet, E., Cliquet, F., 2014. Review of the analysis related to rabies diagnosis and follow-up of oral vaccination performed in NRLS in 2013. In: Report, Anses Nancy EURL for Rabies. Malzeville, pp. 1–14. Sanchini, A., Donoso-Mantke, O., Papa, A., Sambri, V., Teichmann, A., Niedrig, M., 2013. Second international diagnostic accuracy study for the serological detection of West Nile virus infection. PLoS Negl. Trop. Dis. 7, e2184. Servat, A., Cliquet, F., 2006. Collaborative study to evaluate a new ELISA test to monitor the effectiveness of rabies vaccination in domestic carnivores. Virus Res. 120, 17–27. Servat, A., Feyssaguet, M., Blanchard, I., Morize, J.L., Schereffer, J.L., Boue, F., Cliquet, F., 2007. A quantitative indirect ELISA to monitor the effectiveness of rabies vaccination in domestic and wild carnivores. J. Immunol. Methods 318, 1–10. Servat, A., Labadie, A., Hamen, A., Boue, F., Cliquet, F., 2008. Inter-laboratory trial to evaluate the reproducibility of a new ELISA to detect rabies antibodies in vaccinated domestic and wild carnivores. Biologicals 36, 19–26. Singer, A., Kauhala, K., Holmala, K., Smith, G.C., 2008. Rabies risk in raccoon dogs and foxes. Dev. Biol. (Basel) 131, 213–222. Singh, M.P., Goyal, K., Majumdar, M., Ratho, R.K., 2011. Prevalence of rabies antibodies in street and household dogs in Chandigarh India. Trop. Anim. Health Prod. 43, 111–114. Smith, J.S., Yager, P.A., Baer, G.M., 1973. A rapid reproducible test for determining rabies neutralizing antibody. Bull. World Health Organ. 48, 535–541. Suzuki, K., Gonzalez, E.T., Ascarrunz, G., Loza, A., Perez, M., Ruiz, G., Rojas, L., Mancilla, K., Pereira, J.A., Guzman, J.A., Pecoraro, M.R., 2008. Antibody response to an anti-rabies vaccine in a dog population under field conditions in Bolivia. Zoon. Public Health 55, 414–420. Vengust, G., Hostnik, P., Cerovsek, M., Cilensek, P., Malovrh, T., 2011. Presence of antibodies against rabies in wild boars. Acta Vet. Hung. 59, 149–154.

85

WHO, 2013. WHO Expert Consultation on Rabies. Second Report. World Health Organization, Geneva (139 p). Wasniewski, M., Cliquet, F., 2012. Evaluation of ELISA for detection of rabies antibodies in domestic carnivores. J. Virol. Methods 179, 166–175. Wasniewski, M., Guiot, A.L., Schereffer, J.L., Tribout, L., Mähar, K., Cliquet, F., 2013. Evaluation of an ELISA to detect rabies antibodies in orally vaccinated foxes and raccoon dogs sampled in the field. J. Virol. Methods 187, 264–270. Wasniewski, M., Labbe, A., Tribout, L., Rieder, J., Labadie, A., Schereffer, J.L., Cliquet, F., 2014. Evaluation of a rabies ELISA as an alternative method to seroneutralisation tests in the context of international trade of domestic carnivores. J. Virol. Methods 195, 211–220. Xing, D., Wirsing von Konig, C.H., Newland, P., Riffelmann, M., Meade, B.D., Corbel, M., Gaines-Das, R., 2009. Characterization of reference materials for human antiserum to pertussis antigens by an international collaborative study. Clin. Vaccine Immunol. 16, 303–311. Xing, D., Markey, K., Newland, P., Rigsby, P., Hockley, J., He, Q., 2011. EUVAC.NET collaborative study: evaluation and standardisation of serology for diagnosis of pertussis. J. Immunol. Methods 372, 137–145. Yakobson, B., Goga, I., Freuling, C.M., Fooks, A.R., Gjinovci, V., Hulaj, B., Horton, D., Johnson, N., Muhaxhiri, J., Recica, I., David, D., O’Flaherty, R., Taylor, N., Wilsmore, T., Muller, T., 2014. Implementation and monitoring of oral rabies vaccination of foxes in Kosovo between 2010 and 2013-an international and intersectorial effort. Int. J. Med. l Microbiol.: IJMM 304, 902–910. Zienius, D., Pridotkas, G., Jaceviciene, I., Ruzauskas, M., 2014. The field efficiency of oral rabies vaccination in the Lithuanian red fox population from 2006 to 2013. Vet. Med. 59, 299–306.