Benchmark for Evaluating the Quality of DNA

Clinical Chemistry 52:4 728 –736 (2006)

Laboratory Management

Benchmark for Evaluating the Quality of DNA Sequencing: Proposal from an International External Quality Assessment Scheme Simon J. Patton,* Andrew J. Wallace, and Rob Elles Background: In the past 15 years, clinical laboratory science has been transformed by the use of technologies that cross the traditional boundaries between laboratory disciplines. However, during this period, issues of quality have not always been given adequate attention. The European Molecular Genetics Quality Network (EMQN) has developed a novel external quality assessment scheme for evaluation of DNA sequencing. We report the results of an international survey of the quality of DNA sequencing among 64 laboratories from 21 countries. Methods: Current practice for DNA sequence analysis was established by use of an online questionnaire. Participating laboratories were provided with 4 DNA samples of validated genotype. Evaluation of the results included assessing the quality of sequence data, variant genotypes, and mutation nomenclature. To accommodate variations in mutation nomenclature, variants indicated by participants were scored for compliance with 3 acceptable marking schemes. Results: A total of 346 genotypes were analyzed. Of these, 19 (5%) genotyping errors were made. Of these, 10 (53%) were false-negative and 9 (47%) were falsepositive results. A further 27 (8%) errors were made in naming mutations. Results were analyzed for 3 indicators of data quality: PHRED quality scores, Quality Read Length, and Quality Read Overlap. Most laboratories produced results of acceptable diagnostic quality as judged by these indicators. The results were used to calculate a consensus benchmark for DNA sequencing

against which individual laboratories could rank their performance. Conclusions: We propose that the consensus benchmark can be used as a baseline against which the aggregate and individual laboratory standard of DNA sequencing may be tracked from year to year. © 2006 American Association for Clinical Chemistry

Clinical laboratory science has been transformed by generic molecular genetic technologies that cross traditional boundaries between diverse laboratory disciplines, including genetics, hematology, clinical chemistry, and microbiology. However, in bringing these new technologies into diagnostic practice, issues of quality have not been given adequate attention. External quality assessment (EQA)1 schemes have been established to help laboratories measure their performance against other centers and against fixed standards. Examples include schemes offered by the United Kingdom National External Quality Assessment schemes (UKNEQAS) (1 ), the College of American Pathologists (CAP) (2 ), Quality Control in Molecular Diagnostics (QCMD) (3 ), Multi-National External Quality Assay (EQA) Programmes in Clinical Molecular Diagnostics (EQUAL project) (4 ), the European Cystic Fibrosis Thematic Network (5 ), and the European Molecular Genetics Quality Network (EMQN) (6 ). The EMQN was formed in 1997 in response to concerns about the lack of provision of EQA schemes for diagnostic molecular genetics laboratories, to improve and harmonize EQA for molecular genetic testing in Europe. From 1999 to 2002, EMQN received funding from the European Commission under the Framework IV program for Standards, Measurement and Testing (contract no. SMT4CT98-7515). Since then, it has been supported through subscription of the participants, and has been coordinated

European Molecular Genetics Quality Network, National Genetics Reference Laboratory, St. Mary’s Hospital, Manchester, United Kingdom. * Address correspondence to this author at: European Molecular Genetics Quality Network (EMQN), c/o National Genetics Reference Laboratory, St. Mary’s Hospital, Hathersage Road, Manchester M13 0JH, United Kingdom. Fax 44-161-276-6606; e-mail [email protected]. Received October 11, 2005; accepted December 22, 2005. Previously published online at DOI: 10.1373/clinchem.2005.061887

1 Nonstandard abbreviations: EQA, external quality assessment; EMQN, European Molecular Genetics Quality Network; QRL, Quality Read Length; QRO, Quality Read Overlap; and HGVS, Human Genome Variation Society.

728

Clinical Chemistry 52, No. 4, 2006

from the National Genetics Reference Laboratory at Manchester, United Kingdom (7 ). Recently, the European Commission’s Joint Research Centre highlighted the need for improved quality assurance and harmonization of genetic testing services in the European Union (8 ). To date, most EQA schemes for molecular genetics have focused on gene-specific targets relevant to clinical applications such as inherited breast cancer (9 ), cystic fibrosis (10, 11 ), familial thrombophilia (12 ), and hepatitis B testing (13 ). Laboratory performance has been assessed on the basis of the ability to accurately detect genotypes. Some EQA schemes related to clinical testing have also sought to assess clinical interpretation by use of expert opinion and consensus guidelines such as those provided by EMQN (14 ) and the European Academy for Andrology (15 ). Although there is a strong demand to add additional gene-specific schemes, the number of gene targets relevant to clinical testing, ⬎1000 reported to date (8 ), place a practical limit on the coverage of EQA in this format. In response, EQA scheme providers have begun to develop generic technical schemes that are widely applicable to clinical, core technology, and other testing laboratories (16 –20 ). DNA sequencing is an example of a widely used analytical technology in many biological laboratories. To address this need, the EMQN has developed an EQA scheme for the evaluation of DNA sequence analysis. The first pilot scheme ran in 2002 and involved 34 laboratories from 15 countries (21 ). Here we report the results of the second international survey of the quality of DNA sequencing among 64 laboratories from 21 countries. The EQA scheme assessed laboratory performance in DNA sequencing through the ability to detect variants and also assessed raw DNA sequence data, using standard statistical measures.

Materials and Methods design of the eqa scheme Laboratories registered for the scheme on-line, using the EMQN website (6 ), and were allocated a unique identification number to allow anonymization of their participation to all but the EQA scheme organizer. The EQA scheme was split into 2 parts. Part 1 consisted of an online questionnaire to assess current practice in DNA sequencing. This included the laboratory setting, the sequencing technology used, and use of reagents such as fluorescently tagged primers. This information allowed the organizers to adjust the practical scheme to meet the requirements of all participants.

729

primer pair 1 (see below). To accommodate laboratories with sequencing strategies using universal primers, samples tagged with M13 tails (primer pair 2 or 3) were also provided. The primer sequences were as follows: Primer pair 1: primer 1f, 5⬘-GCA GAG TAC CTG AAA CAG GA-3⬘; primer 1r, 5⬘-CAT TCA CAG TAG CTT ACC CA-3⬘ Primer pair 2: primer 2f, 5⬘-CAC GAC GTT GTA AAA CGA CGC AGA GTA CCT GAA ACA GGA-3⬘; primer 2r, 5⬘-GGA TAA CAA TTT CAC ACA GGC ATT CAC AGT AGC TTA CCC A-3⬘ Primer pair 3: primer 3f, 5⬘-TGT AAA ACG ACG GCC AGT GCA GAG TAC CTG AAA CAG GA-3⬘; primer 3r, 5⬘-CAG GAA ACA GCT ATG ACC CAT TCA CAG TAG CTT ACC CA-3⬘ The amplification conditions were as follows: initial denaturation for 5 min at 95 °C, followed by 35 cycles of 94 °C for 1 min, 55 °C for 1 min, and 72 °C for 1 min, with a final 5-min extension at 72 °C. The reaction conditions were 20 ng of DNA, 1⫻ CM102 Reddyload PCR mixture (Abgene), and 2 ␮M each of the forward and reverse primers in a total volume of 40 ␮L. Each sample was purified free of excess primers by use of a QuickStepTM Kit (Edge Biosystems).

validation Before distribution, sample genotypes were validated by the scheme organizer and by an independent laboratory. All variant genotypes were concordant between the 2 centers.

distribution EQA samples for distribution were aliquoted by use of a liquid-handling robot (Tecan Miniprep 75) into prelabeled 0.6-mL screw-cap tubes. Each sample tube contained 20 ␮L of DNA at a concentration of 35 ng/␮L. The primer tubes contained 20 ␮L of either the forward or reverse primer at a concentration of 5 ␮M. Each participating laboratory received a package containing the scheme materials and instructions sent by next-day courier.

instructions Participants were provided with a datasheet including a reference sequence, the protein translation, and location of primer sequences. Laboratories were asked to return a preprinted proforma with their genotyping results, along with color copies of the sequencing electropherograms and electronic copies of the sequence data files.

samples The scheme organizer provided 450-bp PCR-amplified fragments of exon 10 of the cystic fibrosis transmembrane conductance regulator (CFTR) gene (GenBank accession no. M55115). The 4 test samples included all of the main types of sequence changes laboratories were likely to encounter. In addition, a stated wild-type control was provided. Samples were prepared by amplification using

evaluation Evaluation of the results included assessment of the genotypes and the quality of sequence data. To accommodate variations in the mutation nomenclature used, 3 marking schemes were devised (for an example, see Table 1). Correct genotypes were assigned a maximum of 2.00 marks. Incorrect data led to deductions from this

730

Patton et al.: International EQA for DNA Sequencing

Table 1. Marking scheme 1 (fully HGVS compliant; nucleotide numbering from the translation initiation site). Sample

1

2

3

4

a

Change

Mark

Allele 1

Systematic name Nucleotide change Mutation effect

c.1519_1521 del ATC ATC del p.Ile507del

0.50 0.50 0.50

Allele 2


Normal NAa NA

0.50 0.50 0.50

Allele 1


c.1408 A⬎G ATG⬎GTG p.Met470Val

0.50 0.50 0.50

Allele 2


Normal NA NA

0.50 0.50 0.50

Allele 1


c.1408 A⬎G/c.1477 C⬎T ATG⬎GTG/CAG⬎TAG p.Met470Val/p.Gln493X

0.25 each 0.25 each 0.25 each

Allele 2


Normal NA NA

0.50 0.50 0.50

Allele 1


c.1408 A⬎G/c.1466 C⬎A ATG⬎GTG/TCA⬎TAA p.Met470Val/p.Ser489X

0.25 each 0.25 each 0.25 each

Allele 2


c.1408 A⬎G/c.1466 C⬎A ATG⬎GTG/TCA⬎TAA p.Met470Val/p.Ser489X Maximum total score

0.25 each 0.25 each 0.25 each 12.00

NA, not applicable.

score, with a completely wrong genotype scoring 0.00 marks. A score was awarded for 3 criteria (correct assignment of the nucleotide change, the systematic name, and the mutation effect) for each allele. The total score for each correct genotype was 3.00 marks. The maximum possible score for the scheme was 12.00 marks. The quality of the sequence data provided by participants was also assessed, and laboratory performance was ranked. The following measures were used.

of each read were quality-clipped wherever a window of 30 bases dropped below a mean PHRED score of 20. The proportion of unclipped data was expressed as a percentage of the total possible read for the fragment, i.e., the number of bases from the end of the sequencing primer to the end of the fragment. The QRL is a measure of the proportion of each sequence read in a single orientation comprising good analytical quality data (minimum basecalling confidence of 99%).

PHRED quality scores. Each sequence data file was analyzed by use of a PHRED basecaller, which we used in conjunction with the InterPhace program (available from CodonCode Corporation) (22 ). PHRED assigns a quality score for each called base on a logarithmic scale. The PHRED quality score estimates the confidence with which each base has been called. A PHRED score of 20 equates to 99% confidence (1% probability of error). We measured the quality of the data by calculating the number of bases that equaled or exceeded 3 thresholds: PHRED 20, 30, and 40 (99%, 99.9%, and 99.99% confidence, respectively).

Quality Read Overlap (QRO). The QRL scores for each sample (from both forward and reverse sequence reads) were combined together to yield a region of data, the QRO, where good diagnostic quality data were obtained in both orientations. The QRO is expressed as a percentage of the maximum possible QRO, which is the number of nucleotides between the ends of the forward and reverse sequencing primers on the target DNA. The QRO represents the data from each pair of reads that can be considered as “reportable quality”.

Quality Read Length (QRL). Sequence data files were assembled into contigs by use of Pregap4, a contig assembly program available as part of the Staden analysis package (23 ). Sequencing data at the beginning and end

A total of 64 laboratories from 21 countries (see Table 1 in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/ vol52/issue4) registered for the scheme, including 47

Results

731


negative results and 9 (47%) were false-positive results. In one case, a laboratory recorded a false-positive mutation that was not reported by any other laboratory. A further 27 errors were made in the naming of the mutations (Table 2). When a laboratory had miscalled or missed an allele, we examined their electronic sequence data to establish whether the change was identifiable. In all cases, we could see the expected change in the electronic data provided by the laboratory. A genotype score was calculated as the sum of the total score divided by the maximum possible score (Fig. 2; also see Fig. 1 in the online Data Supplement). Seventeen (28%) laboratories had a score that fell below the mean (0.94).

quality of sequence data Fig. 1. Sequencing platforms used by participating laboratories. a

Includes one each of the following instruments: ABI373, Alf-Express II, Beckman CEQ 8000, LICOR IR DNA 4200, MegaBASE 1000, MegaBASE 500, MJ BaseStation, and Pyrosequencer. b ABI (Applied Biosystems Incorporated).

public- and 7 private-sector diagnostic laboratories, 4 public-sector academic laboratories, 2 private-sector research laboratories, and 4 commercial core sequencing facilities. The services offered by participating laboratories focus predominantly on mutation detection with a broad range in the scale of activity. There was significant bias toward sequencing platforms provided by a single manufacturer (Fig. 1). All 64 laboratories completed the online questionnaire and were provided with samples for the practical part of the scheme. Sixty-one (95%) returned their results and raw data by the closing date and were included in the scheme evaluation.

genotyping All laboratories obtained results from the materials provided. Thirty-six (59%) of the participants scored the maximum 12.00 marks for genotyping when measured against the marking schemes (Table 1). Of the 28 (41%) laboratories that failed to obtain maximum marks, 19 (5%) genotyping errors were made from 346 genotypes analyzed; 10 of the genotyping errors (53%) were false-

The output from 57 (93%) of the participating laboratories was analyzed. Data from 4 laboratories were excluded because the format used was incompatible with the analysis programs. The majority of sequence data exceeded a PHRED 20 quality score (Fig. 3A; also see Fig. 2A in the online Data Supplement). The proportion of base-calls meeting this standard averaged over all centers was 87.49% (range, 67.45%–95.89%). This was mirrored by the data for the PHRED 30 (Fig. 3B; also see Fig. 2B in the online Data Supplement) and PHRED 40 (Fig. 3C; also see Fig. 2C in the online Data Supplement) analyses, although a lower proportion of data produced by the centers exceeded these thresholds [mean scores of 79.77% (range, 46.28%– 92.38%) and 63.78% (range, 15.07%– 83.72%), respectively]. The distribution of QRL and QRO scores expressed as a percentage (Figs. 4 and 5; also see Figs. 3 and 4 in the online Data Supplement) broadly mirrored the distributions of the PHRED scores (mean score, 86.09%; range, 57.48%–99.78%). However, there was a more pronounced drop off at the lower end of the distribution. Data from this subset of laboratories were generally of lesser quality in both sequence orientations. Laboratories with PHRED 40 scores above the mean value tended to have good QRL and QRO scores (for example, data from laboratories 99

Table 2. Breakdown of the genotyping results. Errors

Sample

Mutation(s)

1 2 3

c.1519_1521 del ATC; p.Ile507del (heterozygous) c.1540 A⬎G; p.Met470Val (heterozygous) c.1477 C⬎T; p.Gln493X (heterozygous) c.1540 A⬎G; p.Met470Val (heterozygous) c.1598 C⬎A; p.Ser489X (homozygous) c.1540 A⬎G; p.Met470Val (homozygous) Total

4

a

False negativea

1

False positiveb

Naming mutation(s) incorrectly

7

6 6 6c

6

2

9c

10

9

3

Failed to find the mutation. b Reported a mutation that was not present in the sample. c Many of these laboratories made the same nomenclature/interpreting error as for sample 2 when naming the c0.1540 A⬎G; p.Met470Val change.

27

732


Fig. 2. Laboratory genotyping performance rankings based on genotyping scores (calculated as the sum of the total score divided by the maximum possible score).

and 98). However, there were some exceptions to this correlation as some laboratories with poorer quality data as measured by the PHRED score thresholds had good QRL and QRO scores. For example, as can be seen in Fig. 3 (also see Fig. 2 in the online Data Supplement for Laboratory ID codes), the quality of the sequence data

from laboratory 34 as measured by PHRED was below average. However, as can be seen in Fig. 5 (see also Fig. 4 in the online Data Supplement), the QRO from this same laboratory was above average, indicating that the data must be of even quality in both orientations. To resolve this apparent anomaly, we contacted the laboratory di-


733

Figs. 3. PHRED quality score distributions. (A), PHRED 20 quality score distribution; (B), PHRED 30 quality score distribution; (C), PHRED 40 quality score distribution.

rectly. The combination of the chemistry, polymer, and capillary length they use is optimized to allow them to rapidly run several different types of assays at the same time. Although their protocol is not optimized to give “perfect” sequence data, it does provide consistent quality data with long read lengths in both orientations. Overall, the trend was for laboratories that produce low PHRED quality scores to also have low scores for QRL and QRO.

To allow laboratories to gauge their performance, we established a consensus standard for the quality of the sequencing data by ranking the performance of each participant against the %QRO (Fig. 6). To ensure that the data will be comparable in successive EQA schemes and to allow laboratories to determine their performance relative to their peers, we normalized the ranking on a centile scale of 0 –100 and established decile performance

734


Fig. 4. QRL distribution.

indicators (D1 to D10). Including an identical fragment in successive EQA schemes will allow tracking of the aggregate standard of DNA sequencing among the participants. In Fig. 6, laboratories that made genotyping errors are highlighted by white (false-negative), black (false-positive), and black-and-white-shaded (false-positive and -negative) arrows. The spread of these laboratories across the ranked data set suggests that it is not the quality of data that leads to genotype errors. The most likely explanation is human error, failure of mutation scanning software, or a combination of both because many laboratories use a manual check after an initial pass through a software system. In all cases involving false negatives, the assessors were able to identify the change from the raw data.

Discussion The reported EQA scheme establishes an objective method for judging the quality of DNA sequencing performance. One key problem identified was that of mutation naming. It was apparent that several different mutation nomenclature systems were used. To accommodate this, we allowed 3 alternative nomenclature systems. However, we recommended in the final report to the participants (24 ) that laboratories should follow the mutation nomenclature scheme produced by the Human Genome Variation Society (HGVS) (25 ). Although we appreciated that this could cause inconsistencies with

Fig. 5. QRO distribution.

preexisting locus-specific naming schemes, the HGVS recommendations are clear on this point, stating: When new reports describe variants according to current recommendations, instead of using traditional descriptions, experts in the field experience problems “recognizing” these variants. However, nomenclature rules should be universal and thus can not be made to apply for specific situations.

We noticed a common error in mutation naming whereby laboratories incorrectly prefixed the mutation with a “c.” instead of a “g.” when classifying according to a genomic sequence. We did not deduct marks for this but recommended that laboratories take greater care when naming any changes they find in a DNA sequence. Twenty-seven (59%) of the errors identified were errors in nomenclature, indicating that this is an evolving and poorly standardized area. The remaining 19 (41%) errors were false-positive or -negative observations. There appears to be no correlation between the quality of sequence data and the likelihood of a laboratory making a genotyping mistake (Fig. 6). Identifying mutations is made difficult by the high information content of sequence data. The feature that sets DNA sequencing apart from other mutation scanning systems is the large number of data points to be analyzed. Electronic DNA sequencing data can be analyzed for mutations visually or by use of software. Visual inspection is generally accepted as being the “gold standard”, but the mental effort needed means that only small amounts of data can be reliably

735


Fig. 6. Performance ranking of laboratories based on %QRO. Decile performance indicators are shown as D1 through D10. Laboratories highlighted by an open arrow made genotyping errors leading to a false-negative result; filled arrows indicate false-positive genotyping errors; and shaded arrows indicate false-positive and -negative genotyping errors. x axis ranking for each data point is calculated as laboratory %QRO ranking/number of laboratories ranked ⫻ 100.

analyzed. Many software packages have been designed for research applications, where false negatives are not as critical. The performance of automated systems, for example, SeqScape (26 ) and Mutation Surveyor (27 ), falls short of the needs of analytical laboratories. Laboratories should ensure that any software that they use to analyze sequence data is adequately validated for the intended end use of the data, especially in a clinical setting. This EQA scheme measures quality and provides laboratories and core sequencing facilities with an indicator of performance. The lowest mean PHRED 20 score in this series (67.45%) indicated that for the 450-bp fragment sequenced, on average, 303 bp (67.45% ⫻ 450 bp) of the sequence had a base-calling error rate ⱕ1%. Therefore, by definition, 147 bp (32.55%) of that sequence had an error rate ⱖ1%. Given that more than 100 bp of data had been called at this level of error, it is highly likely that these data will contain base-calling errors. Assuming an error rate of 1% for the 147-bp poor-quality portion of the read, the likelihood of making at least one mistake in basecalling is 77.2% [probability calculated as 1 ⫺ (0.99147) ⫻ 100]. Sequence quality underlies base-calling accuracy, although base-calling errors need not necessarily lead to variant detection error. Normalizing the QRO ranking on

a centile scale and using decile performance indicators allows laboratories to measure the improvement or deterioration of their performance relative to their peers in successive EQA schemes. The inclusion of one material common to successive schemes will allow aggregate performance to be followed (even if different sets of laboratories participate from scheme to scheme). These measures are suggested as a benchmark for laboratory performance in the analysis of DNA sequence.

This work was supported by the UK Department of Health.

References 1. The United Kingdom National External Quality Assessment Scheme (UKNEQAS) for Molecular Genetics. http://www.ukneqasmolgen.org.uk (accessed October 2005). 2. College of American Pathologists (CAP). http://www.cap.org (accessed October 2005). 3. Quality Control in Molecular Diagnostics (QCMD). http://www. qcmd.org (accessed October 2005). 4. EC4-EQUAL project. http://www.ec-4.org/equal/home.htm (accessed October 2005).

736


5. European Cystic Fibrosis Thematic Network. http://www.cfnetwork.be/ (accessed October 2005). 6. The European Molecular Genetics Quality Network (EMQN). www. emqn.org (accessed October 2005). 7. National Genetics Reference Laboratory (Manchester). http:// www.nrgl.org.uk/manchester.html (accessed October 2005). 8. Ibaretta D, Elles R, Cassiman JJ, Rodriguez-Cerezo E, Dequeker E. Towards quality assurance and harmonisation of genetic testing services in the European Union. Nat Biotechnol 2004;22:1230 –5. 9. Mueller CR, Kristoffersson U, Stoppa-Lyonnet D. External quality assessment for mutation detection in the BRCA1 and BRCA2 genes: EMQN⬘s experience of 3 years. Ann Oncol 2004;15(Suppl 1):114 –7. 10. Dequeker E, Cassiman JJ. Genetic testing and quality control in diagnostic laboratories. Nat Genet 2000;25:259 – 60. 11. Dequeker E, Ramsden S, Grody WW, Stenzel TT, Barton DE. Quality control in molecular genetic testing. Nat Rev Genet 2001;2:717–23. 12. Preston FE, Kitchen S, Jennings I, Woods TA. A UK National External Quality Assessment scheme (UK NEQAS) for molecular genetic testing for the diagnosis of familial thrombophilia. Thromb Haemost 1999;82:1556 –7. 13. Valentine-Thon E, van Loon AM, Schirm J, Reid J, Klapper PE, Cleator GM. European proficiency testing program for molecular detection and quantification of hepatitis B virus DNA. J Clin Microbiol 2001;39:4407–12. 14. Voelckel M-A, Ramsden S, Matthijs G. Draft best practice guidelines for the molecular diagnosis of Prader-Willi and Angelman syndromes (2002). Available on the EMQN website. http://www. emqn.org (accessed October 2005). 15. Simoni M, Bakker E, Krausz C. EAA/EMQN best practice guidelines for molecular diagnosis of Y-chromosomal microdeletions: state of the art 2004. Int J Androl 2004;27:240 –5. 16. Casini Raggi C, Pinzani P, Paradiso A, Pazzagli M, Orlando C. External quality assurance program for PCR amplification of genomic DNA: an Italian experience. Clin Chem 2003;49:782–91.

17. Orlando C, Casini Raggi C, Pazzagli M. Molecular diagnosis and quality control; a program for DNA analysis with PCR amplification [Abstract]. Clin Chem Lab Med 2000;7:A10. 18. Braun A, Deufel T, Geilenkeuser WJ, Neumaier M, Rohle G, Roscher A, et al. External quality assessment of molecular biology-based methods used in laboratories of clinical chemistry and human genetics. Clin Chem Lab Med 1998;36:231– 4. 19. Neumaier M, Braun A, Gessner R, Funke H. Experiences with external quality assessment (EQA) in molecular diagnostics in clinical laboratories in Germany. Working Group of the German Societies of Clinical Chemistry (DGKC) and Laboratory Medicine (DGLM). Clin Chem Lab Med 2000;38:161–3. 20. Birch L, English CA, Burns M, Keer JT. Generic scheme for independent performance assessment in the molecular biology laboratory. Clin Chem 2004;50:1553–9. 21. Patton S, Taylor G. Final report of a pilot EQA scheme for DNA sequencing (2002). Available on the EMQN website. http://www. emqn.org (accessed October 2005). 22. CodonCode Corporation. http://www.codoncode.com (accessed October 2005). 23. SourceForge.net http://staden.sourceforge.net (accessed October 2005). 24. Patton S, Wallace A. Final report: EMQN pilot external quality assessment scheme for DNA sequencing (2003). Available on the EMQN website http://www.emqn.org (accessed October 2005). 25. Human Genome Variation Society (HGVS). http://www.hgvs.org (accessed October 2005). 26. Patel Y, Wallace A. Mutation scanning—Applied Biosystems VariantSEQr and SeqScape v2.1—an assessment using a model system [Technology Assessment Report]. September 2005. Available on the NGRL website. http://www.ngrl.org.uk/Manchester (accessed October 2005). 27. Patel Y, Wallace A. Automated mutation detection using SoftGenetics Mutation SurveyorTM v2.51 [Technology Assessment Report]. September 2005. Available on the NGRL website. http:// www.ngrl.org.uk/Manchester (accessed October 2005).