Breast Cancer (2016) 23:745–751 DOI 10.1007/s12282-015-0634-7
ORIGINAL ARTICLE
Comparison between Ki67 labeling index determined using image analysis software with virtual slide system and that determined visually in breast cancer Ichiro Maeda1 • Kayoko Abe2 • Hirotaka Koizumi1 • Chika Nakajima3 • Shinya Tajima1 • Hiromi Aoki1 • Junichi Tsuchiya1 • Seiko Tsuchiya4 • Kyoko Tsuchiya4 • Arata Shimo4 • Koichiro Tsugawa4 • Takahiko Ueno5 Shinobu Tatsunami5 • Masayuki Takagi1
•
Received: 9 July 2014 / Accepted: 2 August 2015 / Published online: 14 August 2015 Ó The Japanese Breast Cancer Society 2015
Abstract Background In recent papers, Ki67 labeling index (LI) has been used to classify breast cancer patients into the low and high Ki67LI groups for comparison studies, which showed significant differences in many prognostic factors. It has not been clarified whether image analysis software can be used for calculating LI in breast cancer. In our study, we examined whether Ki67LI in breast cancer calculated using image analysis software correlates with that measured on the basis of visual. Methods Fifty patients were randomly selected among breast cancer patients who underwent surgical operation from March, 2010 to May, 2010 in our hospital without preoperative chemotherapy. In this study, for the virtual slide system (VSS: VS120-L100, Olympus, Tokyo, Japan), the high-resolution VSs of all the 50 patients were prepared as samples. The image analysis software use for calculating LI was Tissuemorph Digital Pathology (Tissuemorph DP: Visiopharm, Hoersholm, Denmark). The calculated LI was extracted from 3 to 5 views containing hot spots. The LI
& Ichiro Maeda
[email protected] 1
Department of Pathology, St. Marianna University School of Medicine, 2-16-1 Sugao, Miyamae-ku, Kawasaki 216-8511, Japan
2
Department of Clinical Pathology, St. Marianna University Hospital, Kawasaki, Japan
3
Olympus Corporation, Tokyo, Japan
4
Department of Breast and Endocrine Surgery, St. Marianna University School of Medicine, Kawasaki, Japan
5
Unit of Medical Statistics, Faculty of Medical Education and Culture, St. Marianna University School of Medicine, Kawasaki, Japan
calculated using Tissuemorph DP was designed as LI/image/T. The digital image of 3 to 5 LI/image/T views was printed out, and on the digital photograph, we counted visually the number of Ki67-immunopositive cells in exactly the same area, and the percentage of Ki67-immunopositive cells was designed as LI/direct. Moreover, a pathologist’s assistant (PA) determined the tumor area in the same specimen using VSS and calculated LI using Tissuemorph DP, which was designed as LI/image/PA. The chief pathologist (CP) similarly calculated LI which was designed as LI/image/CP. We evaluated the degree of agreement between different data sets ‘‘LI/image/T and LI/ direct’’ and ‘‘LI/image/T, LI/image/CP, and LI/image/PA’’ by using interclass correlation coefficient (ICC). Results The average counts of cells were as follows: LI/direct, 3209.7 ± 1970.4 (SD); LI/image/T, 2601.6 ± 1697.1; LI/image/PA, 2886.5 ± 2027.5; LI/image/CP, 18805.5 ± 22293.4. The values of LI/direct and LI/image/ T showed almost perfect agreement as showed by an ICC of 0.885 (95 % CI, 0.806–0.933; p \ 0.001). The agreement among three investigators was almost perfect. The obtained ICC was 0.825 (95 % CI, 0.739–0.890; p \ 0.001) among the data of LI/image/T, LI/image/CP and LI/image/PA. There were five cases that immunopositivity for Ki67 showed a more than 10 % disagreement between LI/direct and LI/image/T. Conclusion The merits of calculating Ki67 LI using Tissuemorph DP are as follows. First, the staining intensity of the cells to be counted can be adjusted. Second, the portion of a tumor including ‘‘hot spots’’ for counting can be chosen. Third, many cancer cells can be counted more rapidly using Tissuemorph DP than by visual observation. However, it is important that pathologist should check and carry out the final decision of the data, when Ki67 LI using Tissuemorph DP is calculated.
123
746
Keywords Breast cancer Ki67 labeling index MIB1 Virtual slide Virtual microscopy
Introduction In recent papers, Ki67 labeling index (LI) has been used to classify breast cancer patients into the low and high Ki67 LI groups for comparison studies, which showed significant differences in many prognosis factors. Such factors showing significant differences are ‘‘overall survivals’’ [1, 2], ‘‘disease-free survival’’ [3], ‘‘recurrence-free survival’’ [1], ‘‘distant metastasis [2, 4]’’, ‘‘Her2-overexpression’’ [1, 3], ‘‘tumor size’’ [3], ‘‘tumor grade’’ [3], and ‘‘peritumoral vascular invasion’’ [3]. However, the cutoff for Ki67 LI for dividing between low- and high-Ki67 LI patients has not been clearly defined. Various researchers reported various cutoff Ki67 LIs: 20 % [4] , 13.25 % [5], 11 % [3], and 10 % [1, 2, 6]. However the portion of the tumor to be measured or the methods of measuring Ki67 LI staining and Ki67 LI have been unclarified. It is also unclear whether Ki67 LI should be measured in the so-called ‘‘hot spots’’ or should the average Ki67 LI of the whole tumor be taken. Moreover, using image analysis software Ki67 [7], ER [8], phosphohistone H3 [9, 10] have also been examined. However there have been a few reports on the correlation between Ki67 LI calculated on the basis of visual observation and that they are calculated using image analysis software. In this study, the Ki67 LI of breast cancer was measured using Tissuemorph Digital Pathology (Tissuemorph DP), which is one of the image analysis software programs. Then, its correlation with Ki67 LI of breast cancer measured on the basis of visual observation was examined.
Materials and methods Patient selection and tissue handling Fifty patients were selected randomly among those with invasive breast cancer without neoadjuvant chemotherapy but who underwent surgical operation in St. Marianna University School of Medicine Hospital from March 2010 to May 2010. Tumors were fixed in 15 % formalin immediately after resection. After fixation for 24-72 h of, a pathologist cuts specimens to make paraffin-embedded blocks. Immunohistochemical staining We prepared 4-um-thick slices of representative blocks, which were then placed on silane-coated glass slides. Tissue sections (4 um thick) were dewaxed and rehydrated.
123
Breast Cancer (2016) 23:745–751
Antigen retrieval was performed by incubating slides in 0.01 M citric acid buffer (pH 7.0) at 100 °C for 10 min. To remove excess reagents from the section, was added dropwise 2 % peroxidase blocking reagent on the slide for 10 min at room temperature reaction, they were incubated with an anti-Ki67 antibody as the primary antibody (Clon: MIB1, DAKO, Glostrup, Denmark) at room temperature for 60 min. The slides were inculated with Histofine simple Stain MAX-PO(MULTI) reagent (NICHIREI BIOSCIENCES INC, Tokyo, Japan) at room temperature for 30 min. Peroxidase activity was visualized by staining with a diaminobenzidine tetrahydrochloride solution (NICHIREI BIOSCIENCES INC, Tokyo, Japan). The reaction of the retrieved antigen to DAB was visualized by staining with using an automated immunostainer (HISTOSTAINER 36A, Nichirei Biosciences Inc., Tokyo, Japan). Virtual slide system (VSS) For all the samples obtained from the 50 patients, Ki67 slides were scanned in a VSS (VS120-L100, Olympus, Tokyo, Japan) using the 940 objective lens. Image analysis software (Tissuemorph Digital Pathology: Tissuemorph DP) Ki67 immunopositivity was analyzed in the nuclei using Tissuemorph DP (TM: Visiopharm, Hoersholm, Denmark). The Tissuemorph DP was set as follows: (1)
(2) (3)
(4) (5) (6)
The setting for ‘‘Detection of Nuclei’’ was ‘‘F(2): HDAB-DAB- [ Mean unsharp’’: Sensitivity, 40.0 %; Size, 4.0um. The setting for ‘‘Classification of Nuclei’’ was ‘‘RGB-R’’: Threshold, 50.0 %. The setting for ‘‘Separate Nucleus Type’’ was ‘‘F(1): H&E-Haematoxylin- [ Mean unsharp’’: sensitivity, 34.0 %; Size, 4.0 um. The setting for ‘‘Detection of Cytoplasm: ‘‘Inactive’’. The setting for ‘‘Detection of Membranes: ‘‘Inactive’’. The setting for ‘‘Detection of Gene Probes: ‘‘Inactive’’.
Tissuemorph DP shows Ki67-immunonegative cells in blue, weakly Ki67-immunopositive cells in green, and strongly immunopositive cells in red (Fig. 1). LI/image/T, LI/image/PA, and LI/image/CP Technologist (T) who handled Tissuemorph DP was trained that the cells to be Ki67-immunopositive were counted. A ‘‘hot spot’’ with many Ki67-positive cells was identified under a low-power field (59) of on VSS. Under a high-power field (5009), three to five independent areas in
Breast Cancer (2016) 23:745–751
747 Table 1 Judgement of ICC
Fig. 1 a Immunohistochemistry of Ki67: the nests of tumor cells and a few fibroblasts (up arrow). Ki67-immunopositive tumor cells and negative tumor cells were found. b Analysis using Tissuemorph DP: Ki67-immunonegative tumor cells, blue; weakly positive cells, green; strongly positive cells, red. The cells shown in blue include not only many tumor cells but also some fibroblasts
the tumor including the so-called ‘‘hot spot’’ were selected [11]. As far as possible, the nests of carcinoma cells excluding lymphocysts, fibroblasts, and myoepithelial cells were identified in manually, when the number of Ki67immunopositive nuclei was counted using Tissuemorph DP in three to five areas of each specimen by the T. For each specimen, images of these areas were saved and printed out. In the same manner, LI/image/PA and LI/image/CP were LI that a pathologists’ assistant (PA) and a chief pathologist (CP) selected and calculated. Three researchers whose were T, PA, and CP calculated Ki67-immunoreactive cells using Tissuemorph DP with same setting. They calculated without being guided by someone and the time to calculate was different separately. Ki 67 labeling index, LI/direct From the saved and printed out images of these 3-5 areas for calculating Li/image/T, the percentage of Ki67-immunopositive cells with respect to the total number of tumor cells in exactly the same area was determined. LI/ direct was defined as percentage of Ki67-immunopositive cells in three to five areas. In this study, ki67-immunopositive cells included weakly and strongly Ki67immunopositive cells. Statistical analyses We evaluated the degree of agreement among different data sets ‘‘LI/image/T and LI/direct’’, ‘‘LI/direct and LI/ image/PA’’, ‘‘LI/direct and LI/image/CP’’, and ‘‘LI/image/
ICC
Judgment
0.0–0.20
Slight
0.21–0.40
Fair
0.41–0.60
Moderate
0.61–0.80
Substantial
0.81–1.00
Almost perfect
PA and LI/image/CP’’ by using interclass correlation coefficient (ICC). ICC, its 95 % CI, and the probability P were computed using SPP version 19 (IBM Japan Ltd., Tokyo, Japan). We used the wordings proposed by Landis and Koch [12] for the interpretation of ICC. The judgment of ICC was considered ‘‘almost perfect’’ when the magnitude of the ICC becomes larger the 0.81 according to the classification by Landis and Koch [12] (Table 1).
Results All the 50 patients had primary breast cancer: invasive carcinoma of no special type (ICNST), 39 patients (papillotubular type, 11 patients; solid tubular type, 8 patients; scirrhous type, 20 patients); carcinoma of mixed type (ICNST and mucinous carcinoma), a patient; invasive lobular carcinoma, 3 patients; mucinous carcinoma, 3 patients; small cell carcinoma, a patient. Average counts of tumor cells were as follows: LI/direct, 3230.9 ± 1934.6(SD); LI/image/T, 2601.6 ± 1697.1; LI/image/PA, 2886.5 ± 2023.5; and LI/image/CP, 18805.5 ± 22293.4 (Table 2). The values of LI/direct were 4.71–55.5 % (average value ± SD, 22.9 ± 12.8 %; median value, 19.98 %). The values of LI/image/T were 3.01–54.61 % (average value ± SD,20.2 ± 13.8;median value, 15.59 %). The values of LI/image/PA were 2.11–51.85 % (average value ± SD, 20.4 ± 12.7; median value, 18.24 %). The values of LI/image/CP were 1.62–49.85 % (average value ± SD, 18.85 ± 10.9 %; median value, 18.06 %) (Table 2). The values of LI/direct and LI/image/T showed almost perfect agreement as showed by an ICC of 0.885 (95 % CI, 0.806–0.933; p \ 0.001) (Table 3). The agreement among three investigators was almost perfect. The obtained ICC was 0.825 (95 % CI, 0.739–0.890; p \ 0.001) among the data of LI/image/T, LI/ image/CP and LI/image/PA (Table 3). There were five cases that immunopositivity for Ki67 showed a more than 10 % disagreement between LI/direct and LI/image/T (Table 4; Fig. 6).
123
748
Breast Cancer (2016) 23:745–751
Table 2 Count of tumor cells and positive cell rate of Ki67 Count of tumor cells Minimum
Positive cell rate of Ki67
Maximum
Average value ± SD 3230.9 ±193.6
Mean value
Minimum
Maximum
Average value ± SD
Mean value
LI/direct
233
7325
3230
4.71
55.5
22.9 ± 12.8
19.98
LI/image/T
822
8153
2601.6 ± 1697.1
2946
3.01
54.61
20.2 ± 13.8
15.59
LI/image/PA
888
10,884
2886.5 ± 2023.5
2316
2.11
51.85
20.4 ± 12.7
18.24
LI/image/CP
2498
148,470
18805.5 ± 22293.4
12,599
1.62
49.85
18.9 ± 10.9
18.06
Table 3 Value of ICC
ICC LI/direct vs. LI/image/T
0.885 (95 % CI = 0.806-0.933)
LI/image/T vs. LI/image/Dr vs. LI/image/PA
0.825 (95 % CI = 0.739-0.890)
Table 4 Immunopositivity for Ki67 showed a more than 10 % disagreement between LI/direct and LI/image/T LI/direct The number of positive cells
LI/image/T The number of total count cells
Positive rate (%)
The number of positive cells
The number of total count cells
Positive rate (%) 48.58
Case 8
499
1408
35.44
696
1436
Case 22
483
4905
9.85
895
4089
25.44
Case 23 Case 41
533 2443
1231 5640
43.30 43.32
387 3312
1561 6112
23.49 55.50
Case 43
1082
1806
59.91
1170
2401
35.24
Discussion When the same image of a breast cancer specimen was evaluated by several observers by measuring the Ki67 LI of breast cancer, it has been reported that the Ki67 LIs determined by several observers showed a very high correlation [11]. In our study, Ki67 LI (LI) was determined using the same photograph and by visual observation method made by CP, and for the values of LI and Ki67LI (LI/image/T), which the technologist measured using image analysis software (Tissuemorph DP), the ICC was very high at 0.885. Moreover, the ICC of LI/image/T examined by the technologist, LI/image/PA examined by a pathologists’ assistant, and LI/image and LI/image/CP examined by the chief pathologist examined was 0.825 which indicated and almost perfect agreement. Moreover, it was indicated that Ki67 LI could be measured using Tissuemorph DP. There were five cases that immunopositivity for Ki67 showed a more than 10 % disagreement between LI/direct and LI/image/T (Table 4). There are some merits in using Tissuemorph DP for calculating Ki67 LI by counting Ki67-immunopositive cells. When Tissuemorph DP is used for calculating Ki67 LI, the person conducting the measurement can adjust the
123
number of tumor cells or tumor sites and count freely. Thus, many cells can be counted more quickly than in the case of measuring visually by adjusting the area for counting. It is also possible to count only the limited cell. The method of measuring Ki67 LI has not yet been established. However in the case of using Tissuemorph DP, the domain including hot spots is measured or the average is obtained; Tissuemorph DP can be used to both of the methods [13, 14]. Tissuemorph DP can also be used for measuring staining intensity. Moreover, Tissuemorph DP can be used even when the staining results are ambiguous; for example, very weakly Ki67-immunopositive cells can be defined Ki67-immunopositive or -immunonegative. However, there are also some demerits in using Tissuemorph DP for calculating Ki67 LI by counting Ki67immunopositive cells. Ki67 LI measurement by image analysis of a tumor with a large amount of lymphocytic infiltrates is difficult, because imaging analysis is applicable in counting the lymphocytes with a high Ki67 LI immunopositivity. On the other hand, when using Tissuemorph DP for calculating Ki67 LI and analyzing a tumor with a large amount of lymphocytic infiltrates, it is possible to count only the tumor cells excluding many lymphocytes by making only the tumor as measurement area. However,
Breast Cancer (2016) 23:745–751
Fig. 2 a Immunohistochemistry of Ki67: some Ki67-immunopositive and -negative tumor cells. b Analysis using Tissuemorph DP: the cytoplasm of immunopositive tumor cells was occasionally counted as a Ki67-immunonegative cell (filled triangle) and one cell is wrongly counted as more than one Ki67-immunonegative cell (up arrow)
the demerit in using Tissuemorph DP for calculating Ki67 LI is the accidental counting of interstitial cells among tumor cells. Although this accidental counting may be avoided, by demarcating and choosing only the tumor cells by manual operation of in certain types of Tissuemorph DP. However, it is difficult to completely exclude interstitial cells in certain types of the tumor (Fig. 1), for example, invasive carcinoma of no special type (scirrhous type) or invasive lobular carcinoma. The cytoplasm of tumor cells immunopositive for Ki67 is occasionally counted as an immunonegative cell or one cell may be mistakenly counted more than once as being both Ki67-immunonegative and -immunopositive (Figs. 2, 3). Myoepithelial cells and lymphocytes may be difficult to not distinguish from epithelial cells (Fig. 4). In addition, there were five cases that immunopositivity for Ki67 showed a more than 10 % disagreement between LI/direct and LI/image/T. In the three cases of those five cases, the total tumor cell number was falsely low, because Tissuemorph DP did not recognize the unclearly stained nucleus of tumor cells immunonegative for Ki67 (Fig. 5). In the two cases, the cytoplasm of immunopositive tumor cells for Ki67 occasionally was counted a immunonegative cell and one cell was falsely recognized with more than one cells of immunonegative or -immunopositive (Figs. 2, 3). Measurement of Ki67LI of such cells using Tissuemorph DP may be difficult, and visual measurement may be required. Pathologist needs to differentiate tumor cells from inflammatory or interstitial cells. Although, it is important that pathologist should check and carry out the final decision of the data, when Ki67 LI using Tissuemorph DP is calculated (Fig. 6).
749
Fig. 3 a Immunohistochemistry of Ki67: two strongly Ki67-immunopositive tumor cells. b Analysis using Tissuemorph DP: one cell is wrongly counted as three immunopositive cells
Fig. 4 a Immunohistochemistry of Ki67: invasive carcinoma and two in situ lesions. b Analysis using Tissuemorph DP: the cells shown in blue include not only many immunonegative tumor cells but also some lymphocytes (filled triangle) and myoepithelial cells (up arrow)
Fig. 5 a Immunohistochemistry of Ki67: immunopositive and negative tumor cells. b Analysis using Tissuemorph DP: some tumor cells aren’t recognized in blue, green, and red
123
750
Breast Cancer (2016) 23:745–751
Fig. 6 Ki67 labeling index of LI/direct and LI/image/T showed in all 50 cases
70.00
60.00
50.00
Percentage
40.00 LI/image/T LI/direct
30.00
20.00
10.00
0.00 0
10
20
30
40
50
Case number
It has been reported that the ICC of Ki67 LI between observers is 0.66 [11], which indicates an sufficient concordance rate among observers.On the other hand, in our study using Tissuemorph DP, the ICC among LI/image/T, LI/image/PA, and LI/image/CP was high 0.825. That is, Ki67 LIs of a specimen of virtual slide (VS) which calculated using Tissuemorph DP under the conditions that include hot spots showed little difference between observers. The method using Tissuemorph DP counted many cancer cells more quickly than by visual observation. This thing may make that the difference between the observers in tumor heterogeneity of Ki67-immunopositive cells is decreased. Calculating Ki67 LI using Tissuemorph DP also showed with high reproducibility, because the ICC of Ki67 LI among LI/image/T, LI/image/PA, and LI/image/CP was almost perfect, and the ICC between LI/direct and LI/image/T was also almost perfect. Ki67 LI, LI/image/PA, and LI/image/CP were measured in different parts in VS. However, the ICC among LI/image/T, LI/image/PA, and LI/image/CP is higher than those previously reported [11]. This indicates that Tissuemorph DP provides a high coincidence rate between observers and may thus have clinical applications.
Conclusion The merit of calculating Ki67 LI using Tissuemorph DP are as follows. First, the staining intensity of the cells to be counted can be adjusted. Second, the portion of a tumor including hot spots for counting can be chosen. Third, many cancer cells can be counted more quickly than by visual observation. However, it is important that
123
pathologist should check and carry out the final decision of the data, when Ki67 LI using Tissuemorph DP is calculated. Compliance with ethical standard Conflict of interest of interest.
The authors declare that they have no conflict
References 1. Bago-Horvath Z, Rudas M, Dubsky P, Jakesz R, Singer CF, Kemmerling R, Greil R, Jelen A, Bohm G, Jasarevic Z, et al. Adjuvant sequencing of tamoxifen and anastrozole is superior to tamoxifen alone in postmenopausal women with low proliferating breast cancer. Clin Cancer Res. 2011;17(24):7828–34. 2. Millar EK, Graham PH, McNeil CM, Browne L, O’Toole SA, Boulghourjian A, Kearsley JH, Papadatos G, Delaney G, Fox C, et al. Prediction of outcome of early ER? breast cancer is improved using a biomarker panel, which includes Ki-67 and p53. Br J Cancer. 2011;105(2):272–80. 3. Viale G, Giobbie-Hurder A, Regan MM, Coates AS, Mastropasqua MG, Dell’Orto P, Maiorano E, MacGrogan G, Braye SG, Ohlschlegel C, et al. Prognostic and predictive value of centrally reviewed Ki-67 labeling index in postmenopausal women with endocrine-responsive breast cancer: results from Breast International Group Trial 1-98 comparing adjuvant tamoxifen with letrozole. J Clin Oncol. 2008;26(34):5569–75. 4. Reyal F, Hajage D, Savignoni A, Feron JG, Bollet MA, Kirova Y, Fourquet A, Pierga JY, Cottu P, Dieras V, et al. Long-term prognostic performance of Ki67 rate in early stage, pT1-pT2, pN0, invasive breast carcinoma. PLoS ONE. 2013;8(3):e55901. 5. Cheang MC, Chia SK, Voduc D, Gao D, Leung S, Snider J, Watson M, Davies S, Bernard PS, Parker JS, et al. Ki67 index, HER2 status, and prognosis of patients with luminal B breast cancer. J Natl Cancer Inst. 2009;101(10):736–50. 6. Kobayashi T, Iwaya K, Moriya T, Yamasaki T, Tsuda H, Yamamoto J, Matsubara O. A simple immunohistochemical panel comprising 2 conventional markers, Ki67 and p53, is a
Breast Cancer (2016) 23:745–751
7.
8.
9.
10.
powerful tool for predicting patient outcome in luminal-type breast cancer. BMC Clin Pathol. 2013;13:5. Gudlaugsson E, Skaland I, Janssen EA, Smaaland R, Shao Z, Malpica A, Voorhorst F, Baak JP. Comparison of the effect of different techniques for measurement of Ki67 proliferation on reproducibility and prognosis prediction accuracy in breast cancer. Histopathology. 2012;61(6):1134–44. Haglund F, Ma R, Huss M, Sulaiman L, Lu M, Nilsson IL, Hoog A, Juhlin CC, Hartman J, Larsson C. Evidence of a functional estrogen receptor in parathyroid adenomas. J Clin Endocrinol Metab. 2012;97(12):4631–9. Skaland I, Janssen EA, Gudlaugsson E, Klos J, Kjellevold KH, Soiland H, Baak JP. Phosphohistone H3 expression has much stronger prognostic value than classical prognosticators in invasive lymph node-negative breast cancer patients less than 55 years of age. Mod Pathol. 2007;20(12):1307–15. Janssen EA, Slewa A, Gudlaugsson E, Jonsdottir K, Skaland I, Soiland H, Baak JP. Biologic profiling of lymph node negative
751
11.
12. 13.
14.
breast cancers by means of microRNA expression. Mod Pathol. 2010;23(12):1567–76. Mikami Y, Ueno T, Yoshimura K, Tsuda H, Kurosumi M, Masuda S, Horii R, Toi M, Sasano H. Interobserver concordance of Ki67 labeling index in breast cancer: Japan Breast Cancer Research Group Ki67 Ring Study. Cancer Sci. 2013;104: 1539–43. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. Niikura N, Sakatani T, Arima N, Ohi Y, Honma N, Kanomata N, Yoshida K, Kadoya T, Tamaki K, Kumaki N et al. Assessment of the Ki67 labeling index: a Japanese validation ring study. Breast Cancer (Tokyo, Japan) 2014. Knutsvik G, Stefansson IM, Aziz S, Arnes J, Eide J, Collett K, Akslen LA. Evaluation of Ki67 expression across distinct categories of breast cancer specimens: a population-based study of matched surgical specimens, core needle biopsies and tissue microarrays. PLoS ONE. 2014;9(11):e112121.
123
