2. Agenda. ▫ Method validation. – Complex global web. – Comparison of various
schemes ... AOAC International (AOAC protocol – Harmonized with ISO 16140) ...
The Importance of Validated Rapid Methods for International Trade
Robert Koeritzer Sr. Technical Manager 3M Food Safety © 3M 2009. All Rights Reserved.
Past-President, AOAC International
Agenda Method validation – Complex global web – Comparison of various schemes – Moving towards harmonization
Trends in method validation.
2
Analytical Science in Trade/Commerce
Codex Technical Committees
Import Testing
Emerging Chemical Contaminates 3
Quality Systems
Emerging Pathogens
European standards EN ISO 17025
Manufacturer QC/QA 4
Horizontal Methods
Preparation of test samples
Alternative methods
Vertical
EN ISO 8261 / IDF 122
diluents
EN ISO 6887 1-2-3-4
Accreditation
Reference methods International European national Routine Routine ISO NF ISO National National or EN NF EN,… NF,DIN DIN,… ,… or EN ISO NF EN ISO NF, EN ISO 6887-1
EN ENISO ISO16140 16140 Validation Validation
ISO 11133 : Guidelines on QA and performance testing of culture media ISO 7218 : General rules for microbial examination
European Recognition Committees (Alternative Method) Country
France
Germany
UK
Nordic
Europe
Standards Bodies
AFNOR
DIN
BSI
NMKL
CEN
Recognition Systems
AFNOR
None, Technical report
EMMAS NordVal
Since 2004, Standardisation of the alternative method performance assessment with EN ISO 16140 is possible
MicroVal
European requirements for alternative methods Requirements: “The use of alternative analytical methods is acceptable, when the methods are validated against the reference method set in Annex and certified by a third party in accordance with the protocol set in EN/ISO standard 16140 or other internationally accepted similar protocols. If the food business operator wishes to use analytical methods other than those validated and certified as described above, the methods shall at least be well documented and scientifically validated.”
Consequences:
6
AFNOR validated methods will be accepted everywhere in Europe AOAC validated methods will be accepted if they have been validated against an EN ISO method
Three major global validation AFNOR (ISO 16140 protocol) MicroVal (ISO 16140 protocol) AOAC International (AOAC protocol – Harmonized with ISO 16140)
7
Validation Methods Comparison Overall AOAC R-I, PTM
AOAC Intl OMA
ISO 16140:2003(E)
Proprietary
Proprietary & Noncommercial
Proprietary & Non-commercial
Reference Methods
AOACI, FDA, USDA, ISO
AOACI, FDA, USDA
ISO, CEN
Time to Validation
6-12 months
12-18 months (minimum)
12 months (minimum)
Methods Reviewed
Required yearly
N/A
Required every 4 yrs
Type of Method
8
Part 1: Methods Comparison [Pre-collaborative]
AOAC R-I, PTM
AOAC Intl OMA
ISO 16140:2003(E)
Food Categories
Claim Dependent
6 (for all or most foods)
5 (for all or most foods)
Food Types
Claim Dependent
20 representative food types
3 per category
3 (H, M, L) + uninoculated
3 (H, M, L) + uninoculated
5 levels
if necessary
if necessary
5 per level
5 per level
2 per level (preferably 5-10)
Required in 1 food type
Required in 1 food type
NA
Required
Suggested
NA
Level of Analyte Quantitative
# of Samples
Competitive micro flora Ruggedness
9
Part 2: Methods Comparison Inter-laboratory study (Collaborative) AOAC R-I, PTM
AOAC Intl OMA
ISO 16140:2003(E)
Participating Labs
NA
Minimum of 8
Minimum of 8
Food Categories
NA
6 (for all or most foods) 1 Food Proposed 4/2010
1
Food Types
NA
1 per category
1
Level of Analyte
NA
3 contamination levels, plus uninoculated
3 contamination levels, plus uninoculated
if necessary
if necessary
2 per level
2 per level
# of Samples
10
NA
AFNOR The principles of the certification procedure
The method validation study is carried out in 2 phases: [technical protocol based on EN ISO 16140]
Phase 1: (preliminary study) Characterize the method and assess it’s effectiveness by comparing it to the effectiveness of the standardized reference method. Comparison carried out at the expert laboratory. Diversity of samples and experimental conditions (sufficient number of naturally contaminated samples) reflect real conditions in the field.
11
AFNOR The principles of the certification procedure
Phase 2: (inter-laboratory study) Compare the efficacy of the method to be validated and the reference method, in several laboratories (8-10 minimum). Under defined conditions of reproducibility and repeatability. AFNOR Difference – Emphasis on sample diversity (natural contamination) – Confirmation of positive results – Evaluation of the quality assurance procedure at the production site
12
MicroVal Offers:
13
An open platform for all European Certification Bodies.
Aiming at a harmonized interpretation of the EN ISO 16140.
Working towards a harmonized international market for all test kits.
Is headed by one European platform with all European stakeholders involved.
A neutral MicroVal secretariat not part of a certification organization.
14
15
AOAC International Validation/Standardization
16
Volunteer Consensus Standards
Process of developing standards is open and transparent, with written procedures covering each step of the process, such as you find in the Official Methods Program. System is known as voluntary because participation is on a voluntary basis and because compliance with the standards is voluntary (unless adopted or referred to by a government authority, like Official Methods)
17
Initial Method Validation “Pre-collaborative
Performance parameters – – – – –
Precision Accuracy Ruggedness Linearity Limit of detection
Comparison to other methods Acceptable method performance (Expert Lab study) Meets Claims in package insert
18
Method Collaborative Study
8-10 laboratories minimum – Minimum of 8 labs with acceptable data (chemistry and quantitative microbiology) – Minimum of 10 labs (qualitative microbiology)
5 samples or test materials (chemistry) – Materials must cover the range of assay applicability
Blind duplicates Standard RM or Control
19
AOAC “Communities” model Issue/Need Training & Conferences Collaborative Study Validation
Engagement Industry/Regulatory/Academia
Single Lab Validation
Community Formed Expert Review Panel Formed
20
AOAC Communities Marine & Freshwater Toxins – Gathering Stakeholders (Industry and regulatory) – International Workshops – Prioritize methods needed – Develop methods using “best science” – Validate methods – Set the industry standards
21
AOAC Communities Dietary Supplements – Gathering Stakeholders (Industry and regulatory)
– Prioritize methods needed – Develop methods using “best science” – Validate methods – Set the industry standards
22
Trends in Method Validation: Business Perspective Cost
Reference Lab Government/ Regulatory Diagnostic Manufacturer
Food Processor
Time to Results Routine Diagnostic Use
Accuracy Inclusivity Exclusivity
Validation Costs: People/Resources, Time, Euros 23
Diagnostic Manufacturer Validation Costs and Time Validation Costs per Method
$70,000 $60,000 $50,000 $40,000 $30,000 $20,000 $10,000 $-
Study/Lab Expenses Application Fees
AOAC-PTM AOACOMA
AFNOR
Time to Complete Validation (per Method)
Acronyms: AOAC-PTM: Performance Tested Method AOAC-OMA: Official Method of Analysis
40 35 30 25 Weeks 20 15 10 5 0
Protocol, Analysis, Submit Results Complete Study AOAC PTM Initial Approval AOAC-PTM AOAC-OMA
24
AFNOR
Food Processors: Wait and Re-Validate? Post Method Approval Validation – Food Matrix and Diluent Validation – Local Flora Evaluation – Local Reference Method Validation For EACH country!
Protection from Litigation OR: Internal Validation (AOAC or AFNOR protocol) – – – –
25
Third-Party approvals are taking too long Internal Study: cost can be US$ 200,000 External (reference lab) Study – usually very limited in scope Participation in Proficiency Testing Programs
Governments & Regulatory Agencies Validations Regulatory Balancing Act – International approvals acceptable for international commerce – Are validated proprietary methods acceptable for testing local foods? • If “yes”, what validations are acceptable?
Better test Accuracy
Government Challenges – Scarce Resources – Who pays for the validation study? – Proprietary Method Validation Processes – Many proprietary methods are seeking governmental validation 26
Consumer Safety
Time and Resources
Governments & Regulatory Agencies Validations (con’t) Collaborative Model: AOAC Official Method of Analysis Food Processor
Government Agency
AOAC Collaborative
Diagnostic Manufacturer
Regulatory Agency Method Adoption
– Are Third Party validations diminishing in importance? – Fewer Food Processors are volunteering for collaborative studies
AOAC Performance Tested Method – 3rd Party Validation – Lower Cost; Shorter Time – AOAC PTM approvals are more compatible with rapidly changing technology 27
Validation Trends Cost and Time Requirements Increasing – Increasing number of products requiring validation – Increasing numbers of regulatory agencies – Products marketed to more and more countries (import regulations)
Convergence – Harmonization of methods • AOAC and ISO 16140 closely aligned – “Fit for Purpose” Validations • AOAC Performance Tested Method (AOAC RI) • Screening versus Actionable 28
29
30
Back Up Slides
31
BPMM Methodology Review scientific literature. Review current International Standards. Consult other (topic) experts. Review ISO, AFNOR, and AOAC validation protocols. Discuss/Debate via teleconference and email. Make consensus recommendations.
32
Key Issues To Address
Does the intended purpose of a method necessitate different levels of sampling and testing. Consider factors influencing confidence (uncertainty) of data. Method Extension to new matrices and/or target organisms.
33
Key Issues To Address Validation in absence of a reference method, confirmation procedure, or superior performance of new method Scientific and statistical bases for: – Determining the limit of detection – Developing performance standards for methods – How to validate performance to meet a performance standard?
What are effective means for determining and articulating the uncertainty associated with microbiological methods?
34
BPMM Recommendations
Statistics Work Group Use International Standards • ISO 16140 (validation of proprietary methods) • ISO 5725 (precision & trueness) • ISO 11843 (capability of detection) • ISO 19036 (measurement uncertainty)
Use robust statistics Define Measurement Uncertainty • Use LOD50 for Qualitative methods • Use ISO 19036 for Quantitative methods
35
BPMM Recommendations
Matrix Extension Work Group Standardize recommended species/strains (include in ISO 16140 revision)
Use categorization of food groups based key attributes – Charts define when additional validation is needed (include in ISO 16140 revision) – Focused on testing for inhibitory or other unique technical considerations that might prevent application of the method
36
BPMM Recommendations
Sampling Work Group Follow ISO standards. (statistical process control) Identify and discuss sources of variation as part of method validation.
37
BPMM Recommendations
Detection Limits Work Group Use a detailed protocol for preparation of inoculum. (include in ISO 16140 revision)
Confirm method results where alternative method is more sensitive then reference method. (include in ISO 16140 revision)
38
