Food Anal. Methods (2008) 1:61–72 DOI 10.1007/s12161-008-9016-5
Method Validation and Quality Management in the Flexible Scope of Accreditation: An Example of Laboratories Testing for Genetically Modified Organisms Jana Žel & Marco Mazzara & Cristian Savini & Stephane Cordeil & Marjana Camloh & Dejan Štebih & Katarina Cankar & Kristina Gruden & Dany Morisset & Guy Van den Eede
Received: 28 November 2007 / Accepted: 14 January 2008 / Published online: 5 April 2008 # Springer Science + Business Media, LLC 2008
Abstract Quality assurance is a prerequisite for accurate and reliable results in food and feed testing, ISO/IEC 17025 being recognized worldwide as the base standard. A flexible scope of accreditation enables testing laboratories to react quickly to customer demand and to cope with the large number of new methods, which have to be introduced in the laboratory. Precisely defined procedures for the validation of methods, together with performance and acceptance criteria, are the key points for flexible scope of accreditation. Testing for genetically modified organisms (GMO) is a challenging exercise, especially with more GMOs entering the world market. We describe here the organization and performance of validating quantitative detection methods for GMO testing in the context of the European Union legislation. Operational procedures for method validation organized by the Community Reference Laboratory for Genetically Modified Food and Feed, assisted by the European Network of GMO Laboratories, are described. A protocol for validating methods within an individual laboratory is proposed and discussed in terms of the requirements of flexible scope of accreditation. The J. Žel (*) : M. Camloh : D. Štebih : K. Cankar : K. Gruden : D. Morisset National Institute of Biology, Večna pot 111, 1000 Ljubljana( Slovenia e-mail:
[email protected] M. Mazzara : C. Savini : S. Cordeil : G. Van den Eede European Commission, Directorate General Joint Research Centre, Institute for Health and Consumer, via E. Fermi 1-TP 331, I-21020 Ispra (VA), Italy
system setup can be an example for other similar fields of analytical work. Keywords Validation . Genetically Modified Organisms . Flexible Scope . Accreditation . Community Reference Laboratory . Testing
Introduction Quality assurance is necessary for providing reliable and accurate results of food and feed analysis. Laboratory accreditation according to ISO/IEC 17025 (International Organization for Standardization 2005a) is a requirement in many areas of food testing that require reliable results for official control purposes. The validity of ISO/IEC 17025 is recognized worldwide as the key standard for establishing the technical competence of testing laboratories in conducting specific tests. In the European Union (EU), in the area of food and feed testing, accreditation according to ISO/ IEC 17025 is a prerequisite for the designation of laboratories for the official control required for verifying compliance with feed and food law, animal health and animal welfare rules (European Commission 2004b). The accreditation and assessment of testing laboratories may be limited to individual tests or groups of tests (European Commission 2004b). Analytical quality assurance comprises the complete set of measures a laboratory must take to ensure that it can always generate high-quality data (Taverniers et al. 2004). In recent years, a clear need for flexibilization of the scope of accreditation has been expressed by both industry and testing laboratories (Holmgren 2006; Leclercq 2002). It is
62
difficult for testing laboratories to react quickly to a customer’s demand to handle a new testing task and to obtain, in time, formal acceptance of the necessary modifications from the accreditation body (Steffen 2002). Flexibilization of the scope would allow a laboratory that has shown appropriate technical competence in the past to introduce new methods or to modify methods within their scope of accreditation, without having to undergo a new audit (assessment) (Steffen 2002; EA-EUROLAB-EURACHEM Permanent Liaison Group 2001). However and in contrast to fixed scope accreditation, flexible scope accreditation includes additional requirements, focused particularly on precisely defined systems of validation of the methods. One of the most complex types of analysis in food and feed control is that required for the detection of genetically modified organisms (GMOs). During the past 10 years, more than 40 countries have adopted labelling regulations regarding GMO presence, but the characteristics and degree of implementation vary greatly (Gruere and Rao 2007). Traceability and labelling of products containing GMOs is demanded in many countries, and labelling thresholds set for unintended presence range from 0.9% in the EU (European Commission 2003b; European Commission 2003a) to 5% in some others countries such as Japan (Gruere and Rao 2007). Additionally, the Cartagena Protocol on Biosafety to the Convention on Biological Diversity establishes an agreement procedure for ensuring that countries are provided with the information necessary for taking informed decisions concerning the import of living modified organisms into their territory (Secretariat of the Convention on Biological Diversity 2000). More and more GMOs are entering the world market, and their detection and identification is every day becoming more complex. Many new GMOs are in process of authorization or have only recently been authorized. In addition, some outbreaks of unapproved GMOs have occurred—Bt 10 maize (European Commission 2005a), LLRICE601 (European Commission 2006a) and Chinese rice Shanyou 63 (Akiyama et al. 2007). New analytical methods for GMO testing have to be introduced and validated quickly to react to such situations and to avoid serious disruption of trade. The need for flexibility of the GMO enforcement laboratories is thus essential. According to EU legislation, the key elements in granting authorization to place on the common market a GMO intended for food and feed use include a safety assessment carried out by the European Food Safety Authority (EFSA), and the availability of validated eventspecific detection methods (European Commission 2003a; European Commission 2002). Identification of the transformation event by means of detection methods specific for each GMO is crucial (1) to ensure full traceability of the GMO products at all stages of
Food Anal. Methods (2008) 1:61–72
their being placed on the market through the production and distribution chain; (2) to allow consumers to make informed choices on dietary preferences based on the labelling of GMO products and (3) to conduct post-marketing surveillance programs (European Commission 2003b; European Commission 2003a; European Commission 2004a). The availability of validated detection methods is thus essential to guarantee high quality and reliable GMO testing data across control laboratories of the EU Member States. The Community Reference Laboratory for GM Food and Feed (CRL-GMFF, http://gmo-crl.jrc.it), established by Regulation [European Commission (EC)] 1829/2003 on GM food and feed (European Commission 2003a), is the European Commission’s Joint Research Centre. It is responsible for testing and validating the method submitted by applicants for authorization to detect and identify the transformation event. The CRL-GMFF is assisted by National Reference Laboratories (NRLs), members of a consortium of laboratories referred to as the ‘European Network of GMO laboratories’ (ENGL, http://engl.jrc.it), established under the auspices of the EU in 2002 (European Commission 2003a). Laboratories assisting the CRL-GMFF in testing and validating methods for detection must be accredited or be in the process of being accredited according to ISO/IEC 17025 or an equivalent international standard (European Commission 2006b). At the end of a validation exercise, the methods for identifying individual GMOs and the validation report are published on the CRLGMFF website (http://gmo-crl.jrc.it/) and made available for further use in control laboratories testing for GMOs. Accreditation according to ISO/IEC 17025 or certification according to an appropriate scheme is accepted also as a criterion for quality assurance in member states’ laboratories carrying out GMO analyses (European Commission 2004c). Outside the EU, although not laid down by law, such a requirement is also de facto considered essential, especially in cases of disputes or emergency measures (European Commission 2006a). Key elements for the accreditation of molecular biology methods for GMO detection according to ISO/IEC 17025 have been described (Žel et al. 2006). GMO testing is the leading area for development of molecular methods, especially quantification with quantitative polymerase chain reaction (PCR), and for standardization of these methods (International Organization for Standardization 2005b). Testing is done on a very wide range of matrixes, different food, feed and seeds, as well as the target analytes (Holst-Jensen et al. 2006). For this reason, the analytical procedures used in GMO testing constitute a suitable model for method validation within a flexible scope of accreditation. In this paper, we aim at clarifying what would be the appropriate scope of accreditation for GMO detection considering the growing numbers of GMOs on the world
Food Anal. Methods (2008) 1:61–72
market. Further on, we clarify the method-performance characteristics in validation of GMO testing methods that have to be met by an applicant who seeks authorization for placing a new GMO on the EU market. Additionally, criteria for validating the performance of a method within an individual laboratory are proposed and discussed in terms of the requirements of a flexible scope of accreditation.
Scopes of Accreditation: Fixed or Flexible? Primarily, accreditation has been based mainly on a fixedscope accreditation. This means that a laboratory is accredited for the materials and/or products tested, the methods used (standard or non-standard) and the tests performed. Thus, the detailed scope of accreditation in many cases consisted of a long list of materials or products the laboratory was competent to test, in addition to test parameters and methods. In the fixed scope, no modification of the list of accredited methods is allowed before next audit (Leclercq 2002; EA-EUROLAB-EURACHEM Permanent Liaison Group 2001). The scope of accreditation reflects the situation when the audit was conducted (Leclercq 2002). Modification of the list, addition of new test methods or withdrawal of a method is possible only after a new audit performed by the accreditation body. Fixedscope accreditation is convenient for laboratories working on a routine basis and using standard methods. On the other hand, flexible-scope accreditation has many advantages when compared to fixed scope. It allows the optimization of given test methods and the introduction/ development of additional test methods within the accredited types of tests (Steffen 2002). However, it is important to note that a laboratory applying for flexible scope must already have experience in modification of methods and, more importantly, in being able to validate the modified method in keeping with the original accreditation standards. It has to be pointed out that there are differences between national interpretations of the term ‘flexible scope’. Some accreditation bodies handle flexible scope with a degree of flexibility, while others impose rather strict limitations (Holmgren 2006). In the European co-operation for Accreditation (EA) document (EA-EUROLAB-EURACHEM Permanent Liaison Group 2001), different types of flexibility are allowed, like optimization of given test methods (adaptation to client needs, new edition of test standards) and development of additional test methods within the type of test already accredited. In the annex to the accreditation certificate, the scope can also allow combinations of flexible and fixed parts of test methods for the laboratory. Accreditation of a flexible scope places more responsibility on the laboratory itself for demonstrating that valid, fit-for-purpose tests are undertaken competently and con-
63
sistently (United Kingdom Accreditation Services 2004). The laboratory must demonstrate that its management system can control the proposed flexibility and that it is in compliance with the requirements of ISO/IEC 17025. Laboratories applying for a flexible scope must, among others, be able to demonstrate their technical capability to validate new or modified methods; authorize appropriate personnel as competent to take responsibility for key tasks in the process of method validation; keep an updated list of accredited test methods including any newly introduced, modified or developed (United Kingdom Accreditation Services 2004). The great need for flexibilization has been already demonstrated in a different field of testing, for example in food and water microbiology (Leclercq 2002). In GMO testing, where more and more GMOs are coming onto the world market and detection and identification of individual GMOs is becoming every day more complex, the need for flexibilization is also evident.
Methods Used in GMO Detection The methods available for GMO testing have been surveyed comprehensively (Van den Eede et al. 2002; Holst-Jensen et al. 2003; Holst-Jensen 2007; Hernandez et al. 2005; Ahmed 2004; Rodriguez-Lazaro et al. 2007). The most reliable methods, that also enable accurate quantification of GMOs present in foodstuffs, are molecular methods based on the detection and quantitation of the DNA. Typically, PCR-based methods are composed of two analytical steps, also referred as modules: extraction of DNA and PCR. The real-time PCR application allows the amounts of a target sequence specific to the GMO (GMspecific target sequence) and of a species-specific target sequence to be determined in a sample, thus leading to the determination of the percentage GMO content. Appropriate reference materials have to be used to ensure accurate results (Trapmann and Emons 2005). Methods published as international, regional or national standards should preferably be used by the laboratories (International Organization for Standardization 2005a). A comprehensive set of standards for GMO detection in food products was published, describing instructions for DNA and protein-based analyses, with detection methods annexed (International Organization for Standardization 2005b, c, d, 2004, 2006; European Committee for standardization 2006). Since a significant number of new GMOs and products thereof are entering the market, some of the detection methods for their analysis are not yet included in these standards. In the EU, a method specific for the GMO (event-specific method) that is the object of an application
64
for authorization on the market must be proposed by applicants and validated by the CRL-GMFF, typically through an international collaborative study (International Organization for Standardization 1994a; Horwitz 1995). Other methods, like construct-specific methods (recognizing GMOs with similar genetic constructs) and screening methods (detecting many GMOs within the same analysis but not identifying them) are still subject to development and introduction by testing laboratories. Analytical procedures for detecting GMOs are composed of successive steps, termed modules (Holst-Jensen and Berdal 2004). Modularity is also recognized in rules for implementing Regulation (EC) No 1829/2003 of the European Parliament and of the Council, and methods for detection, sampling and event-specific identification of the transformation event are described ((EC) No 641/2004) (European Commission 2004a). According to this principle, the applicant is allowed to refer to existing methods for a certain module(s), if available and if appropriate. This could be, for instance, a DNA-extraction method from a certain matrix. In such a case, the applicant has to provide experimental data from an in-house validation in which the method module has been successfully applied in the context of the application for authorization (EC) No 641/2004.
Method Validation Definition and Importance of Method Validation Validation is the confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use are fulfilled (International Organization for Standardization 2005a). Validating a method is investigating whether the analytical purpose of the method is achieved, that is the acquisition of analytical results with an acceptable uncertainty level (Thompson et al. 2002). Validation demonstrates whether the method is fit for a particular analytical purpose. Fitness for purpose is the extent to which the performance of a method matches the criteria agreed between the analyst and the end-user of the data (Thompson et al. 2002). The range and accuracy of the values obtainable from validated methods for the intended use have to be relevant to the customers’ needs (International Organization for Standardization 2005a). The history of a detection method progresses through different stages: method development, pre-validations in a single or small number of laboratories, full validation in an inter-laboratory trial and possible further adoption by an internationally recognized standardization body. Finally, before a method is introduced into an individual laboratory,
Food Anal. Methods (2008) 1:61–72
an additional procedure confirming that the laboratory is achieving the performance characteristics of the method has to be applied. An example of responsibilities of method developer, CRL-GMFF and control laboratory in validation of GMO testing method against the method-acceptance criteria is shown in Table 1. The validation has to be as extensive as is necessary to meet the requirements of the given application or field of application (International Organization for Standardization 2005a). In ISO/IEC 17025, requirements for validation are listed, but which parameters have to be tested by the individual laboratory at the relevant stage of validation of the method is not sufficiently well defined. In practice, method validation is done by evaluating a series of method-performance characteristics, such as precision, trueness, selectivity/specificity, linearity, operating range, recovery, limit of detection, limit of quantification, sensitivity, ruggedness/robustness and applicability (Horwitz 1995; Thompson et al. 2002; International Organization for Standardization 2005e; Gonzalez and Herrador 2007). An overview of validation in analytical chemistry and food laboratories and of method-performance characteristics was published by Taverniers et al. (2004). Validation always entails striking a balance between costs, risks and technical possibilities (International Organization for Standardization 2005a). A modular approach, involving a combination of validations of individual analytical steps, is recognized as leading to reasonably accurate uncertainty Table 1 Definition of responsibilities of the method developer (applicant), of the CRL-GMFF and of the control laboratory in testing the method against the method-acceptance criteria established by the European Network of GMO Laboratories Method-acceptance criteria
Applicability Practicability Specificity Dynamic range Accuracy Amplification efficiency Linearity (R2) Repeatability standard deviation (RSDr) Limit of quantification (LOQ) Limit of detection (LOD) Robustness a b
Bodies responsible Applicant
CRL-GMFF-
Control Laboratory
Yes Yes Yes Yes Yes Yes
Yes Yes Yesa Yes Yes Yes
No No No Yes Yes Yes
Yes Yes
Yes Yes
Yes Yes
Yes
Yes
Yes
Yes
No
Nob
Yes
Yesa
Yes
Whenever and to the extent that is deemed necessary Necessary when the method is used for qualitative purposes
Food Anal. Methods (2008) 1:61–72
estimates and reduced costs, as compared with a nonmodular approach (Holst-Jensen and Berdal 2004). Validation of Quantitative Detection Methods for GMO Testing in the Context of EU Legislation GMO testing is mostly used for official control where precise, accurate and internationally comparable results are required, and method-acceptance criteria and method-performance requirements have been defined to this end (European Commission 2005b; see later in the following section). Validation of the detection method is one of the most important factors for assuring precise and accurate analyses. Reg. (EC) No 641/2004 lists the type of technical information about detection methods that must be provided by the applicant and that is needed to satisfy the preconditions for the fitness of the method for the purpose of enforcing the EU legislation in the field of testing and traceability. As well as the method itself, information has to be included about the data generated when testing control samples and samples of food and feed with the detection method (European Commission 2004a). The method-acceptance criteria and method-performance requirements have been compiled by the ENGL in a document entitled ‘Definition of minimum performance requirements for analytical methods of GMO testing’ (European Commission 2005b), available on the CRLGMFF website (http://gmo-crl.jrc.it/) and European Commission (2004a). ‘Method-acceptance criteria’ are criteria that have to be fulfilled prior to the initiation of any method validation by the CRL-GMFF. The ‘methodperformance requirements’ define the minimum performance characteristics of the method that have to be demonstrated upon completion of a validation study carried out according to internationally accepted technical provisions. This latter
Fig. 1 CRL-GMFF validation process
65
requirement is needed in order to certify that the method validated is fit for the purpose of enforcement of Regulation (EC) No 1829/2003 (European Commission 2004a). Other guidance documents are available at CRL-GMFF website (http://gmo-crl.jrc.it/guidancedocs.htm; European Commission 2004d, 2006c, 2007a, b c). Method Validation Performed by the CRL-GMFF The Community Reference Laboratory (CRL-GMFF, http:// gmo-crl.jrc.it) is responsible for testing and validating methods for detecting and identifying the transformation event. In compliance with its duties and roles as defined by the Annex to Reg. (EC) No 1829/2003 (European Commission 2003a) and subsequently amended by Reg. (EC) No 1981/2006 (European Commission 2006b), the CRL-GMFF operates under a quality system applied to the entire validation process in accordance with the requirements of ISO 9001:2000 as certified by “The Swiss Association for Quality and Management Systems (SQS)” and according to the flexible scope of accreditation EN ISO/IEC 17025:2005 granted by the “Deutsche Akkreditierungsstelle Chemie GmbH (DACH)” for testing in the field of “Biology (DNA extraction and PCR method validation for the detection and identification of GMOs in food and feed materials)”. The validation process involves the following steps, also summarized in Fig. 1: Step 1. Reception of samples and methods provided by the applicant Step 2. Scientific assessment of the documentation and data Step 3. Experimental testing of the samples and methods Step 4. Inter-laboratory validation Step 5. Reporting to the European Food and Safety Authority
66
Food Anal. Methods (2008) 1:61–72
Step 1 and 2: Reception of Samples and Method, and Scientific Assessment The first step entails the checking and registering of samples and relevant documentation received from the applicant. The items received from EFSA and from the applicant are checked for completeness and visual integrity, prior to any scientific assessment. The next step involves the scientific assessment of the documentation related to the method and to the data supplied with the samples. The CRL-GMFF verifies that the methods and samples provided fulfill the requirements in the Annex I to Reg. (EC) No 641/2004 (European Commission 2004a) which endorses the method-acceptance criteria set by the European Network for GMO Laboratories. If the verification is positive, the CRL-GMFF initiates the validation process, using the control samples and the samples of food and feed provided by the applicant. In the case where the submitted method has already been validated through a collaborative study, the method may not have to undergo a full validation process. Step 3: CRL-GMFF Experimental Testing The third step in the process is the experimental testing of the method(s), using samples provided. During the experimental testing, the CRL-GMFF carries out the following: – – –
– –
Design of the collaborative study if the method shall undergo a full validation process Checks on the quantity and quality of the control samples received from the applicant according to the requirements of the method validation Preparation of samples and reagents for full validation or, in the case of single laboratory evaluation of a method already validated through a collaborative study, method verification Testing the detection method(s) provided by the applicant Testing the DNA extraction method(s) provided by the applicant if the method has not been previously validated
The results of the experimental testing are used to assess whether the method(s) fulfils the method-acceptance criteria and whether the control samples are suitable for use in the full validation process through a collaborative study. Should the need for clarification on the detection method arise, the CRL-GMFF may refer to the applicant in step 2 and/or 3 of the validation process. Step 4: Collaborative Study The inter-laboratory validation is organized by the CRL-GMFF according to requirements defined in the International Union of Pure and Applied Chemistry protocol for the design, conduct and interpretation of method-performance studies (Horwitz 1995), and in
the international standard (ISO) 5725 on accuracy, trueness and precision of measurements, methods and results (International Organization for Standardization 1994a). The experimental work is carried out by 12 or more European laboratories. Regulation (EC) No 1981/2006, lists in Annex II the laboratories appointed as National Reference Laboratories assisting the CRL-GMFF for testing and validating methods of detection and identification and, in Annex I, the requirements with which they have to comply (European Commission 2006b). Step 5: Reporting The results of the collaborative study are reported to EFSA and published, together with the validated protocols, on the CRL-GMFF website in the form of validation reports containing the results of the validation study and validated protocols containing the detailed description of the validated method(s). An example of validation data resulting from the ring-trial is given in Table 2 for the method of detecting DAS 59122 and TC1507 genetically modified maize; these include the expected concentration of the GMO in the samples submitted to the laboratories (GM levels), the number of participating laboratories, the number of outliers for each GM level (based on the applications of Cochrane and Grubbs’ tests according to ISO 5725-2), the mean value measured for each GM level and its associated variability in terms of repeatability and reproducibility.
Single-Laboratory Validation “The laboratory will validate non-standard methods, laboratory-designed/developed methods, standard methods used outside their intended scope, and amplifications and modifications of standard methods to confirm that the methods are fit for the intended use” (International Organization for Standardization 2005a). Reading this ISO/IEC 17025 requirement, it is possible to understand, mistakenly, that if the method is already validated, the laboratory can directly use it in the laboratory. However, once the method has been studied in a collaborative trial, the laboratory then has to verify that it is capable of achieving the performance characteristics of the method published in the collaborative validation study (Thompson et al. 2002). The validation parameters chosen are dependent on the degree of preliminary data obtained on the method to be introduced into quality system. (Thompson et al. 2002). The extent of validation also depends on the method-performance parameters, which were not tested in the previous validations of the published method. If the laboratory develops its own method, it has to test all method-performance parameters. The need for detection of GMOs predated the establishment of CRL-GMFF, and methods for GMO-testing were
Food Anal. Methods (2008) 1:61–72
67
Table 2 Summary of the validation data for maize DAS 59122 and TC 1507 (source: http://gmo-crl.jrc.it/statusofdoss.htm) Expected value (GMO %) DAS 59122
Laboratories having returned results Samples per laboratory Number of outliers Reason for exclusion Mean value Repeatability relative standard deviation (%) Repeatability standard deviation Reproducibility relative standard deviation (%) Reproducibility standard deviation Bias (absolute value) Bias (%)
TC 1507
0.10
0.40
0.90
2.00
4.50
0.00
0.10
0.50
0.90
2.00
5.00
14
14
14
14
14
14
14
14
14
14
14
4
4
4
4
4
2
2
2
2
2
2
0 – 0.13 18.16
0 – 0.46 13.89
0 – 0.98 15.84
0 – 2.13 13.59
1 1, C. test 4.43 8.45
0 – 0.000 0.00
0 – 0.106 18.11
1 G. test 0.480 11.70
2 G. test; C. test 0.933 7.68
1 C. test 1.966 8.48
0 – 5.420 14.41
0.02
0.06
0.16
0.29
0.37
0.00
0.02
0.06
0.07
0.17
0.78
24.59
21.80
21.77
14.94
13.15
0.00
19.91
14.78
10.24
21.19
21.65
0.03
0.10
0.21
0.32
0.58
0.00
0.02
0.07
0.10
0.42
1.17
0.03 29
0.06 15
0.08 9
0.13 7
0.07 −1
0.00 0.00
0.006 6.00
−0.02 −4.00
0.033 3.70
−0.034 −1.70
0.42 8.40
Acceptance criteria: dynamic range, 1/10—at least five times the target concentration, which is 0.9%; reproducibility relative standard deviation (%),
