Veterinary Parasitology 194 (2013) 106–109
Contents lists available at SciVerse ScienceDirect
Veterinary Parasitology journal homepage: www.elsevier.com/locate/vetpar
Trichinella detection: Identification and statistical evaluation of sources of error in the magnetic stirrer method for pooled sample digestion Katharina Riehn a,∗ , Dirk Hasenclever b , David Petroff c , Karsten Nöckler d , Anne Mayer-Scholl d , Gregor Makrutzki a , Ernst Lücker a a b c d
Institute of Food Hygiene, Faculty of Veterinary Medicine, University of Leipzig, Germany Institute for Medical Informatics, Statistics and Epidemiology, Medical Faculty, University of Leipzig, Germany Coordination Centre for Clinical Trials, Medical Faculty, University of Leipzig, Germany Federal Institute for Risk Assessment, Berlin, Germany
a r t i c l e
i n f o
Keywords: Trichinella Detection Magnetic stirrer Standardized protocols
a b s t r a c t Proficiency testing (PT) is the use of inter-laboratory comparisons to determine the performance of individual laboratories for specific tests or measurements, and to monitor a laboratory’s performance. Participation in proficiency testing provides laboratories with an objective means of assessing and demonstrating the reliability of the data they are producing. To ensure the reliability of Trichinella detection and meat hygiene within the European Union and afford optimal protection to the consumer, PT is conducted under the direction of the European National Reference Laboratories for Trichinella. Evaluation of data from the national PT showed that lab-internal shortcomings are frequent. These shortcomings are specifically related to: (1) improper sample collection and preparation; (2) incorrect transposition and application of the protocol as laid down in Annex I, Chapter I, Nr. 3 (a–g) of the Commission Regulation (EC) No. 2075/2005; (3) insufficient sedimentation times; and (4) improper equipment.(e.g. Prost and Nowakowski, 1990; Rossi and Pozio, 2008; Forbes and Gajadhar, 1999; Rossi and Pozio, 2008). To test the hypothesis that both method based errors as well as internal lab errors can influence the accuracy and precision of the magnetic stirrer method for pooled sample digestion (MSM), we initiated a study to evaluate the analytical uncertainty of the MSM. Results presented here are based on: (i) data from PT in Germany (2008, 2009, and 2010); (ii) within-lab performance conducting high volumes of MSM; (iii) larval recovery experiments; and (iv) statistical evaluation of data resulting from these procedures. Quantitative data from the PT show that on average only 60% of Trichinella larvae were detected. Even laboratories that showed relatively good performance (>80% larva recovery, no false negative or false positive results), frequently reported samples with an unexpectedly low larval count (loss of >2 larvae). In our own laboratory, high numbers of repeated analyses of standards and re-analyses of residual fluids indicated that these outliers could be described by a binomial distribution based on a laboratory-specific Trichinella-detection probability. Results of recovery experiments indicate that only a part of the total larval losses can be attributed to lab-internal shortcomings inasmuch as a significant number of L1 could be isolated from the residual and washing fluids. © 2013 Elsevier B.V. All rights reserved.
1. Introduction ∗ Corresponding author. E-mail address:
[email protected] (K. Riehn). 0304-4017/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.vetpar.2013.01.031
In order to monitor and efficiently control trichinellosis in humans and animals, the European Union (EU)
K. Riehn et al. / Veterinary Parasitology 194 (2013) 106–109
established a legal framework applicable to each Member State: Directive 2003/99/EC defines the monitoring of zoonoses and zoonotic agents, and Regulation (EC) No. 2075/2005 defines specific rules to control for Trichinella in meat. According to these legislations, mandatory examination of all slaughtered pigs and other susceptible animal species is still the main method for combating and controlling trichinellosis in Europe. In this context, the application of suitable and sensitive detection methods is a key factor for ensuring a high level of consumer protection. Following introduction of the artificial digestion of muscle tissue in the 1970s, a new and better method for the detection of Trichinella was established: the magnetic stirrer method for pooled sample digestion (MSM). This procedure was validated in 1979 by Köhler and later introduced into German and EU legislation for Trichinella meat inspection. Over the years, numerous research groups (e.g. Forbes and Gajadhar, 1999; Gamble, 1999) evaluated the MSM and other digestion techniques. Because of the benefits of the MSM specifically related to the detection of non-encapsulated Trichinella spp., better larval recovery rates, reduced examination times, and lower costs, this method was defined as a reference method when Commission Regulation (EC) No. 2075 was introduced in 2006. However, it was noted that due to non-uniform larval distribution within tissues, and some technical limitations, the sensitivity of this method is limited to 3 larvae per g when examining the prescribed 1 g of meat (Forbes and Gajadhar, 1999). Apart from these methodological limits, lab-internal shortcomings in the implementation of the MSM protocol can also influence the sensitivity of the method. Table 1 gives a detailed overview of many error sources in the different steps and sub-steps of MSM. To ensure the quality of the MSM and to evaluate the competence of laboratories in Trichinella detection, proficiency testing (PT) must be conducted in each Member State of the EU in accordance with Regulation (EC) No. 882/2004 and under the direction of the National Reference Laboratories for Trichinella. In Germany, PT has been in place since 2004. The statutory accreditation of all official Trichinella laboratories within the scope of Regulation (EC) No. 2075/2005 requires among other things, the regular participation in inter-laboratory testing. As a result, the number of participants has been steadily growing from 33 in 2004 (Nöckler and Reckinger, 2005) to108 in 2010 (Mayer-Scholl et al., 2011). Beginning in 2008 and in order to meet the legal requirements, every participating laboratory is mandated to analyze samples using both qualitative and quantitative methods. Three ranges of tolerance based on the z-score were defined for each sample size; a within tolerance range, an intermediate range (yellow), and an outside tolerance range (red). Outside the tolerance range means effectively that fewer than 50% of L1 were found in small samples (2), 12 labs (80%) had 1 outlier, and 2 labs (13%) had 2 outliers. This means that low L1 counts occur regularly even in the hands of experienced laboratories. Data analyses show further, that this error is not predictable and is not uniformly distributed.
108
K. Riehn et al. / Veterinary Parasitology 194 (2013) 106–109
Table 1 Error sources in the magnetic stirrer method for pooled sample digestion. Step
Error sources
Reference
Sample collection
Improper target tissue Too small amount of sample Fat and fascia residues in/on the sample Larvae disruption by too intensive blending Use of undiluted HCl Expired or unsuitable chemicals Inappropriate temperature Unsuitable mesh size Too short sedimentation time Trapping of larvae on the edge of the drain opening Remain of larvae in the separatory funnel if not enough volume is drained in the beaker Turbulences caused by too fast evacuation of supernatant Insufficient microscope Unskilled examiners Worn out or dirty working equipment Use of non-calibrated instruments
Prost and Nowakowski (1990) Rossi and Pozio (2008)
Sample preparation Digestion fluid
Sieve Sedimentation Stopping cock Draining Evacuation Visualisation Equipment
3.2. Evaluation of in-lab performance, recovery experiments, and error analysis Of the 213 samples evaluated for within-lab performance (Table 2), the two examiners agreed on 188 samples, differed by one L1 in 24 samples and by two L1 in a single sample. Using a binomial model and choosing to minimize the sum of square differences between the model and the observed counts, we estimate that the chances of failing to see a larva under the microscope are roughly 1.3%. In total, 1789 of 2130 L1 (84%) were detected using MSM and full L1 counts (10 L1) were determined in 49 cases (23%) of all digests. We generally searched for missing L1 in cases where more than 2 L1 were lost. In the 53 samples for which we went beyond the standard MSM protocol and looked for missing L1, 358 of 530 larvae (68%) were found initially during the first microscopic examination and 55 of the missing 172 L1 (32%) were subsequently detected in the residual and washing liquids. On the basis of these results, quantification of L1 losses in different stages of the MSM was performed (Table 2). In total, 899 (98%) of 920 L1 were detected in the separation funnel by the examiners during the first sedimentation step and three of the 21 (14%) missing L1 were detected in residual and washing liquids. Although the mean L1 loss in this stage was low (2%), a maximum of two out of 20 (10%) L1 in an individual sample could not be retrieved in some cases. During the second sedimentation in the glass tube, 25 of 400 (6%) L1 were lost 5 (20%) which were secondarily recovered from the sucked off liquids (40 ml). The maximum L1 loss per individual sample was 3 of 20 (15%). When investigating the potential influence
Forbes and Gajadhar (1999) Gamble (1999) Gamble (1999) Forbes and Gajadhar (1999) Forbes and Gajadhar (1999) Forbes and Gajadhar (1999) Gamble et al. (2000) Rossi and Pozio (2008)
of different visualization methods and procedures, no significant difference was observed between the approaches described in Annex I, Chapter I, 3 (I) (o) of Regulation (EC) No. 2075/2005. Of the 600 L1 to be found, 586 (98%) were found using the trichinoscope and 585 using the stereo microscope. These data clearly indicate that there was no relevant difference between the two methods. These 585 L1 found using a counting basin were compared with the 589 larvae found using a Petri dish (p = 0.55 using Pearson’s chi-squared test) again demonstrating no significant difference. However, at the stage of visualization, losses are not negligible: 40 out of 1800 (2%) L1 were not detected by the three examiners where the maximum L1 loss per individual sample was 3 of 20 (15%). 4. Discussion Proficiency testing is a useful tool to verify that a laboratory is able to properly perform a given protocol, and to compare results between laboratories using the same methods i.e. benchmarking (Wiegers, 2004). Analysis of the data from the German national PT from 2008 to 2010 shows on one hand improvement in qualitative results, but little progress in obtaining quantitative results. Overall, the findings indicate that others sources of errors may influence quantification in addition to already known lab-internal errors. This hypothesis is also supported by results obtained from our own experiments. Performing a large number of digests under highly standardized conditions that excluded specific error sources (Table 1) still resulted in only 84% of all L1 being detected. However, it was possible to recover a significant number of
Table 2 Experimental conditions and details of in-lab performance and recovery experiments. Experiment
Number of L1/sample
Runs
Number of examiners
Evaluation of in-lab performance Recovery/error analysis Sedimentation step I (separation funnel) Sedimentation step II (glass tube) Visualization (Stereo microscope vs. trichinoscope) Visualization II (Petri dish vs. larval counting basin)
10
213
2
20 20 20 20
46 20 15 15
2 2 3 3
K. Riehn et al. / Veterinary Parasitology 194 (2013) 106–109
previously unrecovered L1 by careful examination of wash and residual liquids. This suggests that L1 losses are not only attributable to previously known individual failures, but also to other deficits such as PT samples containing an incorrect number of L1, incomplete sedimentation i.e. dead L1 with uncoiled shapes, or possibly the MSM itself despite consideration being given to known sources of error. Taking into account that the sources of error are additive in a first order approximation, this may explain the unexpectedly high L1 losses in MSM. The low detection rate of 84% even under optimal conditions can therefore be explained by the aforementioned stepwise error sources assuming that in each step, 2–3% of the larvae are lost. An additional factor in the assessment of laboratory performance during Trichinella proficiency testing is the appropriateness of the methods employed. The current definition of the tolerance range is inappropriate since it labels a substantial number of labs as unacceptable for examining large sample sizes. Additional work will focus on developing a simple laboratory model based on the inherent characteristics of the lab which will not suffer from a dependence on sample size. 5. Conclusions Our results indicate that inadequate quantification results in PT may not only be attributed to insufficient laboratory performance, but also to deficits of MSM used for Trichinella detection. If the first step of the current protocol is changed such that the sedimentation times is raised, we estimate that the detection probability would increase from 84% to 89%. This alone would decrease the chances of a false negative result from a sample with 5 L1 by a factor of 7. Moreover, our current work suggests that optimization of other steps in the protocol could lead to further improvement. The main targets for further optimization are; (i)
109
identification of other possible error sources, (ii) quantification of single error sources and determination of the total error, and (iii) definition of the total analytical uncertainty. Conflict of interest statement None of the authors of this manuscript has declared any conflict of interest within the last three years which may arise from being named as an author on the manuscript. References Forbes, L.B., Gajadhar, A.A., 1999. A validated Trichinella digestion assay and an associated sampling and quality assurance system for use in testing pork and horse meat. J. Food Prot. 62, 1308–1313. Gamble, H.R., 1999. Factors affecting the efficiency of pooled sample digestion for the recovery of Trichinella spiralis from muscle tissue. Int. J. Food Microb. 48, 73–78. Gamble, H.R., Bessonov, A.S., Cuperlovic, K., Gajadhar, A.A., van Knapen, F., Noeckler, K., Schenone, H., Zhu, X., 2000. ICT recommendations on methods for the control of Trichinella in domestic and wild animals intended for human consumption. Vet. Parasitol. 93, 393–408. Köhler, G., 1979. Untersuchungen mit der Stomachermethode im Vergleich zu anderen direkten Verfahren b Nachweis d. Trichinellose des Schweines. Fleischwirtschaft 9, 1258–1263. Mayer-Scholl, A., Reckinger, S., Nöckler, K., 2011. Ringversuch zum Nachweis von Trichinellen in Fleisch (2010). Fleischwirtschaft 91, 127–130. Mayer-Scholl, A., Reckinger, S., Nöckler, K., 2010. Ringversuch zum Nachweis von Trichinellen in Fleisch (2009). Fleischwirtschaft 90, 174–178. Mayer-Scholl, A., Reckinger, S., Nöckler, K., 2009. Ringversuch zum Nachweis von Trichinellen in Fleisch (2008). Fleischwirtschaft 89, 110–114. Nöckler, K., Reckinger, S., 2005. Ringversuch zum Nachweis von in Schweinefleisch (2004). FleisTrichinella-Muskellarven chwirtschaft 85, 99–104. Prost, E.K., Nowakowski, Z., 1990. Detectability of Trichinella spiralis in muscles by pooled-sample-digestion-method. Fleischwirtschaft 70, 595–596. Rossi, P., Pozio, E., 2008. Guidelines for the detection of Trichinella larvae at the slaughterhouse in quality assurance system. Ann. Ist. Super. Sanita 44, 195–199. Wiegers, A.L., 2004. The quality assurance of proficiency testing programs for animal disease diagnostic laboratories. J. Vet. Diagn. Invest. 16, 255–263.
