Editorial
Benchmarking Cytometry Attila Tarnok1,2*
ALTHOUGH Cytometry defines itself as the science and technology of a quantitative single cell analysis and is a key technology in the field of immunology, cancer biology and more, benchmarking of frequently done and important measurements is still an ongoing issue and a major source of uncertainty. Referring to internationally accepted standards such as the “International Prototype Meter” maintained by the International Bureau of Weights and Measures in France, the organization that includes member institutions from 17 nations and meets regularly to upgrade these standards since 1875, is still a view into the future for cytometrists. Several reasons are accounting for this lack. These reasons lay on the one hand in the diverse technical solutions to perform cytometric analysis by flow and image cytometry with a substantial variance in optical paths, light sources, optical filters and detectors. On the other hand a plethora of applications exists from cell phenotyping over functional analysis to the quantitation of microbes and sub-micron particles that would profit from internationally accepted standards. In the era of large scale cytomics cohort studies like the Euroflow (1), the GEIL (2), the ONE (3), the LIFE (4), and the HOM SWEET HOMe (Heterogeneity of Monocytes in Subjects Who Undergo Elective Coronary Angiography) (5), the need for benchmarking becomes evident. Benchmarking is essential to transport knowledge gained from these types of investigation to other institutions and laboratories in order to gain and extract reliable and reproducible information from cellular data. Such high level of standardization and quality control is relevant for example to ensure that the differences between studies on gender, age, life-style or geographic region are not merely due to technical differences. This unification is essential for the development of future strategies for diagnosis,
predictive medicine and individualized therapy. Beyond that, standardization and benchmarking are also highly desirable from basic cellular research to quantitation of biocoenosis of microorganisms in their natural habitat for environmental protection and ecology. Fortunately, several research groups including those from large governmental organizations took over the effort to benchmark specific critical analytical areas in cytometry. These organizations include metrological intuitions like the NIST (National Institute of Standards and Technology, USA) and the PTB (Physikalisch-Technische Bundesanstalt, Germany). The topics addressed are for example the quantitative cell viability measurement by flow and image cytometry (6) and the benchmarking of fluorescence intensity measurements per cell (MFI, mean fluorescence intensity). The latter is of high relevance, as it reflects cell activation values that can be of diagnostic relevance as shown in numerous studies but their cross-platform validation is challenging. In a NIST/ISAC study, Hoffman and colleagues (7) presented approaches to use commercial beads for MFI calibration but they also pinpointed the problems arising from their improper use. Solly and colleagues (2) from the GEIL study investigated ways for cross-platform comparison of MFI values between two different instruments and five centers. In this issue, Halter and colleagues from NIST (this issue, page 978) investigated ways to standardize fluorescence microscopy intensity measurements for image cytometry. They report that using commercially available fluorescent glasses that are highly photostable in combination with an automated protocol is appropriate for intensity calibration of one instrument type over time. However, they also elaborate that due to various technical differences between fluorescence microscopes, a
1
Department of Pediatric Cardiology, Heart Centre Leipzig, Leipzig, Germany
This work was made possible by funding from the German Federal Ministry of Education and Research (BMBF, AT: PtJ-Bio, 1315883).
2
Translational Centre for Regenerative Medicine (TRM), University of Leipzig, Leipzig, Germany
Published online in Wiley Online Library (wileyonlinelibrary.com)
Received 13 September 2014; Accepted 30 September 2014 *Correspondence to: Prof. Attila T arnok, Dept. Pediatric Cardiology, Heart Centre Leipzig, University of Leipzig, Str€ umpellstr. 39, 04289 Leipzig, Germany. E-mail:
[email protected]
Cytometry Part A 85A: 909 910, 2014
DOI: 10.1002/cyto.a.22576 C 2014 International Society for Advancement of Cytometry V
Editorial cross-platform standardization cannot be achieved if the optical settings are different. Another critical point next to benchmarking of the measurement process is the reproducible data analysis. In image cytometry the first line of data analysis is identification and segmentation of cells from digital images (8). This procedure then yields complex single cell data comparable to what is obtained by flow cytometry. In the case of high-dimensional polychromatic flow cytometry data sequential manual gating clearly produces a biased approach to reading the data based on individual preferences of each reader (3,4). This bias needs to be reduced if not eliminated by automation. An automated and unsupervised data analysis that was thoroughly tested and reviewed by Aghaeepour et al. (9) would be an ideal solution but it is still under development. Here, Frederiksen and colleagues (this issue, page 969) report a web-based system for analyzing complex flow cytometry data, NetFCM. They used leukocytes from HIV infected individuals that were challenged with HIV or CMV peptides. NetFCM was able to distinguish virus specific responses from the study samples as well as in a test set obtained from the FlowCAP II study (9). Mixed microbial populations are indeed challenging with regard to data analysis, sample preparation and storage. Koch et al. addressed this issue by developing software for unsupervised analysis of complex microbial communities and recently comparing it with other data analysis tools (10,11). With regard to preservation of natural samples Marie and colleagues (this issue, page 962) designed an improved protocol for their storage and measurement. A combination of numerous successful standardization efforts are needed to yield a reliable global survey of environmental changes. And this brings me to possibly the largest challenge in sample analysis, namely, the impact of pre-analytics including the sample collection, labelling, storage and transport.
910
The improper sample handling causes an exceedingly larger error compared to that resulting from the data acquisition and analysis as determined in many large-scale studies (see for example references 3, 4). The error for immunophenotyping experiments can be estimated at roughly 50%. Such differences render substantial errors in multicentric and follow-up studies and require a lot of effort to develop “universal” standards such as reliable and generally accepted model cells for benchmarking cell staining and preparation.
LITERATURE CITED 1. Stuchly J, Kalina T. Analyses of large flow tometry datasets. Cytometry A 2014; 85A(3):203–205. 2. Solly F, Rigollet L, Baseggio L, Guy J, Borgeot J, Guerin E, Debliquis A, Drenou B, Campos L, Lacombe F, Bene MC. Comparable flow cytometry data can be obtained with two types of instruments, Canto II, and Navios. A GEIL study. Cytometry A 2013;83A(12):1066–1072. 3. Streitz M, Miloud T, Kapinsky M, Reed MR, Magari R, Geissler EK, Hutchinson JA, Vogt K, Schlickeiser S, Kverneland AH, Meisel C, Volk HD, Sawitzki B. Standardization of whole blood immune phenotype monitoring for clinical trials: panels and methods from the ONE study. Transplant Res 2013;2(1):17. 4. Bocsi J, Melzer S, D€ahnert I, Tarnok A. OMIP-023: 10-Color, 13 antibody panel for in-depth phenotyping of human peripheral blood leukocytes. Cytometry A 2014; 85A(9):781–784. 5. Rogacev KS, Cremers B, Zawada AM, Seiler S, Binder N, Ege P, Große-Dunker G, Heisel I, Hornof F, Jeken J, Rebling NM, Ulrich C, Scheller B, B€ ohm M, Fliser D, Heine GH. CD1411CD161 monocytes independently predict cardiovascular events: a cohort study of 951 patients referred for elective coronary angiography. J Am Coll Cardiol 2012;60(16):1512–1520. 6. Kummrow A, Frankowski M, Bock N, Werner C, Dziekan T, Neukammer J. Quantitative assessment of cell viability based on flow cytometry and microscopy. Cytometry A 2013;83A(2):197–204. 7. Hoffman RA, Wang L, Bigos M, Nolan JP. NIST/ISAC standardization study: variability in assignment of intensity values to fluorescence standard beads and in cross calibration of standard beads to hard dyed beads. Cytometry A 2012;81A(9):785–796. 8. Dima AA, Elliott JT, Filliben JJ, Halter M, Peskin A, Bernal J, Kociolek M, Brady MC, Tang HC, Plant AL. Comparison of segmentation algorithms for fluorescence microscopy images of cells. Cytometry A 2011;79A(7):545–559. 9. Aghaeepour N, Finak G, Hoos H, Mosmann TR, Brinkman R, Gottardo R, Scheuermann RH. Critical assessment of automated flow cytometry data analysis techniques. Nat Methods 2013;10:228–238. 10. Koch C, Fetzer I, Harms H, M€ uller S. CHIC-an automated approach for the detection of dynamic variations in complex microbial communities. Cytometry A 2013; 83A(6):561–567. 11. Koch C, Harnisch F, Schr€ oder U, M€ uller S. Cytometric fingerprints: evaluation of new tools for analyzing microbial community dynamics. Front Microbiol 2014;5: 273. doi: 10.3389/fmicb.2014.00273. eCollection 2014.
Benchmarking Cytometry
