ISO 17025:2017. Tightens definitions, explicitly: Validation: Verification, where the specified requirements are fit for an intended use. Verification: Provision of ...
ViVA - Verification in VAlidation. Results from an observational study Angus Marshall & Richard Paige, Dept. of Computer Science
Hypothesis ISO/IEC 27041 suggests that evidence of a tool's verification against a declared set of requirements can be used as means to reduce the amount of validation required for processes in which the tool participates. i.e. it suggests that those process requirements which are wholly satisfied by the tool, and for which evidence of verification exists, need not be subjected to further testing
ViVA scenarios
1. Specialist single-purpose tool provides subset of method requirements
ISO 17025:2005 Depends on ISO 9000 definition of validation as “Confirmation, through the provision of objective evidence, that requirements for a specific intended use or application have been fulfilled.”
Why is this useful? Reduction in number of repeat tests: If evidence of verification can be published, tools need only be tested once unless being used in a unusual way
Reduction in re-validation: If an updated tool can be shown to be verified to the same level as a previous version, it can be adopted without further testing Does not eliminate validation completely, but potentially reduces it to a more manageable level
Assumptions A. Method implementers have defined requirements B. Tool producers have defined requirements C. Tool producers have evidence of testing against requirements If all 3 are present, then A and B can be mapped against each other and C used as evidence in validation
What we did ● Review of volunteered SOPs and validation records from accredited organisations ● Visits to tool producers ● Conversations with tool producers ● Questionnaire for tool producers
What we found ● Methods are defined in terms of “end-user requirements”. ○ Non-technical, CJS considerations ● Tool producers are unwilling or unable to disclose anything about ○ development methods ○ requirements capture ○ requirements validation ○ evidence of testing
Possible Explanations 1. SOP standard format does not contain explicit requirements section 2. End-users do not understand/express technical issues 3. Other bodies using ISO 17025 have published industry/national/international standards against which they test. Some have published methods (cf “wet sciences”). 4. Language used in ILAC and FSR guidance puts undue emphasis on end-user (investigator, CJS) and ignores technical correctness
Interpretation - effect Accreditation is not achieving one of its goals i.e. Few organisations, if any, can show that methods are technically correct because they have not defined “correct”in terms of verifiable requirements.
ISO 17025:2017 Tightens definitions, explicitly: Validation: Verification, where the specified requirements are fit for an intended use Verification: Provision of objective evidence that a given item fulfils specified requirements
Much closer to the ISO/IEC27041 position
Proposal Community needs to create a corpus of standard requirements which can be adopted to define methods, inform tool development, and assist verification and validation across the sectors.
