International Journal for Quality in Health Care 2012; Volume 24, Number 4: pp. 411 –418 Advance Access Publication: 16 May 2012
10.1093/intqhc/mzs020
Feasibility of evaluating quality cancer care using registry data and electronic health records: a population-based study ADELE CALDARELLA1, GIANNI AMUNNI2, CATIA ANGIOLINI3, EMANUELE CROCETTI1, FRANCESCO DI COSTANZO4, ANGELO DI LEO5, FRANCESCO GIUSTI1, ANDREA LOPES PEGNA6, PAOLA MANTELLINI1, LUCIO LUZZATTO2 AND EUGENIO PACI1 1
Epidemiologia Clinica e Descrittiva, ISPO, Firenze, Italia, 2Istituto Toscano Tumori, Firenze, Italia, 3Oncologia Medica, Azienda Sanitaria di Firenze, Firenze, Italia, 4Oncologia Medica, Azienda Ospedaliero Universitaria Careggi, Firenze, Italia, 5Oncologia Medica, Azienda Sanitaria di Prato, Prato, and 6U.O.Pneumologia I, Azienda Ospedaliero Universitaria Careggi, Firenze, Italia Address reprint requests to: Adele Caldarella, Tel: þ39-055-7972507; Fax: þ39-055-7972535; E-mail:
[email protected] Accepted for publication 11 April 2012
Abstract Objective. To evaluate the quality of patients care, a set of indicators of the standards of cancer care were defined. Design, Setting, Participants. We developed a set of indicators to assess the implementation in daily practice of recommendation produced by a regional network (Istituto Toscano Tumori). This set was tested in a retrospective study in the resident population of the Tuscany Region; the regional health system is organized on 12 local health authorities which refer to three macro areas (Area Vasta). The study included incident colorectal, lung and breast cancer cases listed in 2004 for the Tuscan Cancer Registry, a population-based registry which collected tumor cases diagnosed in all residents in Tuscany. Electronic data from registry database were used to determine the compliance with each indicator for patients in 2004. To validate the results, an ad hoc clinical survey including the same geographical area for the year 2006 was performed. Intervention. None. Main Outcome Measures. The proportion of patients who fulfilled each of the indicators. Results. Our study showed the feasibility of the evaluation of the quality of cancer care using cancer registry populationbased data and major computerized information systems. The estimation of the selected indicators confirmed a good homogeneity among areas, and globally revealed a good intraregional performance. Conclusions. Further work is needed to develop specific quality measures, particularly about structural data and to continually revise indicators of quality of care. Data from a cancer registry, however, can be useful to evaluate quality of cancer care Keywords: indicators, quality of care, cancer, lung cancer, breast cancer, colorectal cancer
Introduction Evidence-based guidelines are increasingly regarded as essential in cancer care. Indicators are measurement tools that can be used for monitoring all aspects of the health care process or outcome and to implement evidence-based medicine [1, 2]. Numerous organizations have developed quality measures for surgical and oncology care [3]: recently, it has been suggested that indicators may be useful to assess adherence of hospital with individual aspects of care and to improve evidence-based actions in local networks [3 – 5]. However, the use of indicators for population-based assessment of the quality in large network is still rare.
In Tuscany a population-based regional network to coordinate and to improve the quality of cancer care for the entire region, the Istituto Toscano Tumori (ITT, www.ittumori.it), has been set up since 2003. The first aim of the ITT was to produce shared cancer clinical recommendations for the management of the most frequent types of solid tumors as well as indicators of the standards of cancer care. Information was obtained on the entire population of Tuscany, involving tracing all cancer patients resident in the Region, whether cared for by public or private providers, based on the regional cancer registry where all the information of cancer cases in residents of the region was collected for the year 2004.
International Journal for Quality in Health Care vol. 24 no. 4 # The Author 2012. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved
411
Caldarella et al.
The main objective was to show the feasibility of evaluating quality of care at population level using cancer registry data. Another objective was to assess the level of adherence to the ITT cancer care recommendations, through quality indicators for colorectal cancer, lung cancer and breast cancer. To validate the 2004 results an ad hoc population-based survey was used for year 2006.
Methods Data sources The study included all the resident population of the Tuscany Region (3585000 inhabitants in 2004). The regional health system is organized on 12 local health authorities that refer to three macro areas (Area Vasta). A university hospital is the cancer reference center within each Area Vasta. In this study participants were considered eligible if they had a new diagnosis of female breast or colorectal or lung cancer and were registered as incident cases in the Tuscan Cancer Registry. The cancer registry collects all cancers incident during the years of registry activity. In our study, exclusion criteria included prevalent tumors, multiple tumors and cases cared for in other regions. Results were analyzed at three sub-regional areas. The Tuscany Cancer Registry has been active in about one-third of the Tuscany region since 1985. The first year of extension to all the regional population was 2004. Incident cases were identified through hospital discharge notes, pathology reports, death certificates, oncological archives: once the case is defined, each source of information contributes with specific data, including, for example, tumor histology, stage at diagnosis, type of treatment and follow-up [6]. Not all sources can be available for all cases, thus, quality of data in a cancer registry is evaluated by the percentage of microscopically verified cases, by the percentage of cases known from death certificate only, and by the mortality/incidence ratio. The Tuscany Cancer Registry showed a good quality of data, with a high percentage of microscopically verified cases and very low percentage of cases known from the death certificate only (80 and 1%, respectively) [7]. In total 2591 breast, 2381 lung and 2419 colorectal cancer incident cases were included in this study (tab 1). For each patient all the electronic available health information from the Tuscany region information system was collected: pathology reports, hospital discharge notes including day hospital and computerized records of outpatient. Data were checked for quality and completeness, if needed, in cooperation with the local hospital departments using local electronic or clinical health records, particularly for therapeutic information. Validation set For 2006, a sample of eligible patients who received surgery was selected from the hospital discharge notes of all public and private hospitals using the same criteria used during 2004. Prevalent cancer cases were excluded by merging with the cancer registry. Clinical charts and pathology reports
412
from all the regional providers were collected. In order to avoid over-representing patients from larger clinical centers, the sampling fraction for larger hospitals .150 cancer cases was the 20%, whereas in smaller centers all cases were investigated. In total 357 breast, 116 lung and 264 colorectal cancer cases were sampled in 2006. Surgery specimens were available for 357 breast, 112 lung and 258 colorectal cancer patients. Regarding the pathological stage, pT was available in all breast, 103 lung and 248 colorectal cancer patients; pN was available in 345 breast, 103 lung and 248 colorectal patients. A chart abstraction tool was created in Microsoft Access Softwarew to record all relevant data. Chart review was performed by independent investigators who were not involved with the care of the patients. Data obtained included detailed information about the tumor characteristics, diagnostic methods, staging, type, date and duration of therapies. The list of indicators The ITT had published recommendations on diagnosis and treatment of solid tumors [8] in 2005 based on an expert panel of health professionals in a consensus process based on their experience and on scientific literature. The preliminary set of indicators was compared with the list of indicators available from scientific literature, other organizations [9, 10], guidelines [11 – 14], extensive systematic literature reviews, and review of articles published in the Medline database from 2000 to 2006. Web sites of professional societies and organizations that monitor quality of medical care were accessed to identify currently available clinical practice guidelines and quality measures. The indicators encompassed the diagnostic and therapeutic phases of breast, colorectal and lung cancer care, and were reviewed by a multidisciplinary panel of members belonging to the regional network previously involved in developing the ITT recommendations. Indicators were identified regarding events on diagnosis, surgical therapy, adjuvant and neoadjuvant therapy. Structural indicators were not included. A core set of 32 quality indicators regarding diagnosis, treatment and follow-up care was selected: 10 were specific for lung, 7 for colon, 11 for rectum and 14 for breast cancer. The most important literature references for each indicator are presented in Tables 2– 4. Analysis To verify the possibility to measure agreement between clinical practice and recommendations, the selected indicators were retrospectively evaluated on data from the Cancer Registry in 2004. Data were analyzed for the whole region and for macro areas for patients younger and older than 75 years separately, as older patients are less likely to receive standard treatment than younger ones [15]. Only results regarding patients younger than 75 years and for the whole region were presented. Furthermore, in this paper data are presented by
Use of registry data in quality care †
Table 1 Reporting of information for breast, lung and colorectal cancer patients in 2004 Cancer site
Patients included
Patients who received surgical intervention
Available parameters
....................
pT
pN
....................................................................................
Breast Colon Rectum Lung
2591 1683 736 2381
2369 1683 736 491
2256 1443 602 393
1992 1390 580 385
Quality Measurement and Health Information Technology
residence of patients and not by providers of diagnosis or treatment. The proportion of patients who fulfilled each of the indicators was determined. The average percent compliance was calculated with 95% confidence intervals for each of the indicators. For each item, the numerator was the number of patients who were compliant with each of the indicators and the denominator was the total number of eligible patients.
Results The most relevant characteristics of the patients are listed in Table 1. Mean age was 63.3 years for breast, 71.2 years for
Table 2 Quality indicators for breast carcinoma: data from cancer registry 2004 and regional average 2006 Indicator
Eligible Patients Sample patients meeting the patients 2004 (n8) criterion 2006 (n8) 2004 (n8)
Regional average 2004 % (CI 95%)
Regional average 2006 % (CI 95%)
.............................................................................................................................................................................
1 2 3 4 5 6
7 8
9
10 11
12
13
14
Proportion of cancer screening detected Proportion of patients with microscopic preoperatory diagnoses [30, 32, 33, 34] Proportion of patients with tumor diameter ,2 cm who undergo breast conservative surgery [30, 34] Proportion of patients with carcinoma in situ who undergo axillary dissection [30, 31, 33] Proportion of patients with invasive carcinoma with sentinel lymph node evaluation [32, 33] Proportion of patients with invasive carcinoma with histological analysis of 10 lymph nodes or more [30, 32, 34] Proportion of patients with reconstructive surgery among patients who underwent mastectomy [30, 31] Proportion of patients with invasive carcinoma and metastatic lymph nodes who receive adjuvant chemotherapy (age 50 – 74) [31, 32, 33] Proportion of patients with invasive carcinoma and no metastatic lymph nodes who receive adjuvant chemotherapy (age 50 – 74) [31, 32] Proportion of patients with adjuvant chemo-therapy performed within one month after surgery [30, 32] Proportion of patients with radiotherapy among patients who underwent breast conservative surgery [30, 31, 32, 33] Proportion of patients who receive hormone-therapy among patients with metastatic lymph nodes and age 50 – 74 years [31, 32, 33] Proportion of patients who receive hormone-therapy among patients with no metastatic lymph nodes and age 50 – 74 years [31, 32, 33] Proportion of patients with invasive carcinoma in whom c-erb analysis was performed
1420 1784
502 1598
n.a. 263
35 (33 – 37) 90 (89 – 91)
n.a. 90 (86– 94)
1292
1117
139
87 (85 – 89)
97 (94– 100)
277
242
54
87 (83 – 91)
83 (73– 93)
1492
528
244
35 (33 – 37)
60 (54– 66)
886
781
99
88 (86 – 90)
94 (89– 99)
370
101
45
27 (22 – 32)
42 (28– 56)
459
353
75
77 (73 – 81)
89 (82– 96)
641
164
129
26 (23 – 29)
78 (71– 85)
795
267
95
34 (31 – 37)
19 (11– 27)
1266
972
75
77 (75 – 79)
84 (76– 92)
n.a.
217
n.a.
67 (61– 73)
n.a.
129
n.a.
71 (63– 79)
263
64 (62 – 66)
95 (92– 98)
1948
1252
n.a.: Not available
413
Caldarella et al.
Table 3 Quality indicators for colorectal carcinoma: data from cancer registry 2004 and regional average 2006 Indicator
Eligible Patients Sample Regional Regional patients meeting the patients average 2004 average 2006 2004 (n8) criterion 2006 (n8) % (CI 95%) % (CI 95%) 2004 (n8)
.............................................................................................................................................................................
COLON CANCER Proportion of cancer screening detecteda [29] Proportion of patients with microscopic preoperatory diagnoses 3 Proportion of patients with surgery performed within one month after diagnostic endoscopy [35] 4 Proportion of patients with invasive carcinoma with histological analysis of 12 lymph nodes or more [28, 29] 5 Proprotion of patients with pathological stage II who receive adjuvant chemotherapy [36] 6 Proportion of patients with pathological stage III who receive adjuvant chemotherapy [36] 7 Proportion of patients who receive adjuvant chemotherapy within two months after surgery among patients who receive adjuvant chemotherapy [35] RECTUM 1 Cancer screening detecteda 2 Microscopic preoperatory diagnoses 3 Proportion of patients with surgery performed within one month after diagnostic endoscopy [35] 4 Proportion of patients with invasive carcinoma with histological analysis of 12 lymph nodes or more [28, 29] 5 Proportion of patients with pathological stage II who receive adjuvant chemotherapy [36] 6 Proportion of patients with pathological stage III who receive adjuvant chemotherapy [36] 7 Proportion of patients who receive adjuvant chemotherapy within two months after surgery among patients who receive adjuvant chemotherapy [35] 8 Proportion of patients who undergo abdominoperineal resection among patients who undergo surgery [29] 9 Proportion of patients who receive neoadjuvant radiotherapy/radiochemotherapy in patients with pathological stage II – III 10 Proportion of patients who receive adjuvant radiotherapy/radiochemotherapy in patients with pathological stage II – III 11 Proportion of patients who receive rehabilitative treatment after anterior rectal resection and colostomy 1 2
1242a 890
132a 756
n.a. 101
11 (9 – 13) 85 (83 – 87)
n.a. 96 (92 – 100)
606
504
80
83 (80 – 86)
74 (64 – 84)
832
618
111
74 (71 – 77)
76 (68 – 84)
254
157
37
62 (56 – 68)
54 (38 – 70)
252
214
42
85 (81 – 89)
79 (67 – 91)
556
465
52
84 (81 – 87)
89 (80 – 98)
1242a 430 350
132a 380 217
n.a. 41 50
16 (13 – 19) 88 (85 – 91) 62 (57 – 67)
n.a. 98 (94 – 100) 73 (61 – 85)
400
250
80
63 (58 – 68)
76 (67 – 85)
116
71
11
61 (52 – 70)
82 (59 – 100)
121
108
15
89 (83 – 95)
80 (60 – 100)
262
193
30
74 (69 – 79)
87 (75 – 99)
467
79
41
17 (14 – 20)
17 (6 – 28)
221
44
26
20 (15 – 25)
8 (0 – 18)
221
178
26
81 (76 – 86)
81 (66 – 96)
230
107
n.a.
47 (41 – 53)
n.a.
n.a.: Not available. a Colon and rectum.
colon, 69.0 years for rectum and 70.6 years for lung cancer patients. There were 3150 male and 4241 female patients; by cancer site, 51% of colon, 60% of rectum and 78% of lung cancers were male patients.
414
Breast cancer Adherence with the indicators of breast cancer care ranged from 27 to 90% among the 2591 patients diagnosed in 2004.
Use of registry data in quality care †
High adherence to recommendations was generally showed for indicators related both to diagnosis and treatment. However, some critical aspects were found: proportion of breast cancer patients with adjuvant chemotherapy performed within 1 month after surgery was ,40% and proportion of patients who underwent reconstructive surgery after mastectomy was 27% (Table 2). The most important and interesting finding was the use of intraoperative diagnosis which was clearly preferred in the surgical departments of an area of the ITT network. Furthermore, a non-homogeneous use of new approaches was found, like sentinel lymph node and c-erb-2 testing for breast cancer. University hospitals were the only ones performing these techniques in 2004. Moreover, regarding sentinel lymph node evaluation, a difference among geographical areas was also shown, with a larger use of this technique only in one among the three areas. The use of chemotherapy in women with breast cancer was higher in patients with lymph node involvement (Table 2) than in women without lymph node metastasis, according to literature [16]. Colorectal cancer Adherence with the indicators ranged from 62 to 85% among the 1683 colon cancer patients and from 20 to 89% among the 736 rectal cancer patients diagnosed in 2004. In
Quality Measurement and Health Information Technology
rectal cancer care, an high proportion of patients with pathological cancer stage III received adjuvant chemotherapy (89%) while for patients with pathological cancer stage II the proportion was lower (61%) (Table 3). An adequate number of lymph nodes examined (12 or more lymph nodes) in at least 70% of colon cancer patients (Table 3) was found, according to recommendation. Lung cancer Adherence to each indicator of lung cancer care ranged from 2 to 95% among the 2381 lung cancer patients diagnosed in 2004. The indicators related to therapeutic phase showed that the proportion of patients who underwent surgery, including all stages disease, was 21%, according to data from the Surveillance, Epidemiology and End Results, where the percentage of lung cancer patients who had resectable disease in 2002 was 23% [17]. In the diagnostic phase, the use of mediastinoscopy was very low in all areas of the network (2%). This indicator, however, was analyzed for all lung cancer patients, whereas it could be useful to stratify data by clinical stage. The proportion of patients who underwent neoadjuvant chemotherapy in 2004 was lower (15%) than the proportion of patients who underwent adjuvant radiotherapy (38%) (Table 4). Among patients with small cell lung cancer, a high proportion of patients underwent chemo/radiochemotherapy (88%). The indicator on use of
Table 4 quality indicators for lung cancer: data from cancer registry 2004 and regional average 2006 Indicator
Eligible Patients Sample Regional average patients meeting the patients 2004 % 2004 (n8) criterion 2006 (n8) (CI 95%) 2004 (n8)
Regional average 2006 % (CI 95%)
.............................................................................................................................................................................
1
Proportion of patients who receive 1472 mediastinoscopy [30] 2 Proportion of patients who receive PET [30] 1472 3 Proportion of patients who undergo surgical 1285 resections [37] 4 Proportion of patients who receive surgery 131 other than atypical resection in patients with stage I NSCLC [30, 36, 37] 5 Proportion of patients with SCLC who not 153 underwent surgical resection [30] 6 Proportion of patients who died within 30 days 351 after surgery [37] 7 Proportion of patients who receive neoadjuvant 89 chemotherapy [30, 37] 8 Proportion of patients with N2 pathological 69 stage who receive adjuvant radiotherapy [30, 37] 9 Proportion of patients with SCLC who receive 153 chemo/radiochemotherapy [30] 10 Proportion of patients who receive 866 chemotherapy within one month prior death
25
169
1.7 (1 – 2.4)
1.2 (0 – 2.8)
453 337
169 n.a.
30.8 (28.4 – 33.2) 23.1 (16.7 – 29.5) 26.2 (23.8 – 28.6) n.a.
124
35
94.8 (91 – 98.6)
139
n.a.
90.8 (86.2 – 95.4) n.a.
10
90
2.8 (1.1 –4.5)
1.1 (0 – 3.3)
13
25
14.6 (7.3 – 21.9)
20 (4.3 – 35.7)
26
22
37.7 (26.3 – 49.1) 36.4 (16.3 – 56.5)
135
n.a.
88.3 (83.2 – 93.4) n.a.
210
n.a.
24.2 (21.3 – 27.1) n.a.
94.3 (86.6 – 100)
n.a., not available.
415
Caldarella et al.
therapy within 1 month prior to death showed that in 24% of patients chemotherapy was performed. The low perioperative mortality (2.8%) suggested a good quality of surgery. Data from 2006 Results from data set in 2006 resembled data from the Cancer Registry in 2004. Regarding breast cancer patients, results increased in 2006 for two indicators: proportion of patients with sentinel lymph node evaluation (regional average 60% in 2006, 35% in 2004) and proportion of patients with c-erb analysis (95% in 2006, 64% in 2004).
Discussion The main result of this study is that a population-based evaluation of the quality of cancer care using cancer registry data and major computerized information systems is feasible, also in large areas. The use of the 2004 Cancer Registry data was satisfactory for the estimate of the whole target population. Population-based data from a registry are useful to evaluate adherence of indicators to guideline, also analyzing the geographical differences among regional areas. Priority was given to citizen perspective. The local health authority must offer a high-quality level for each indicator independent on the kind of provider. There are many providers available to the citizen resident in a local authority area (local versus the university hospital, for example) but the main responsibility of the local health authority is to provide a good standard of quality in any setting of care the patient may choose. Indicators were selected evaluating the feasibility of measurement through the informative regional data set, the possibility to perform analysis in a short period of time and the volume of providers of interest. We reported regional average, without referent threshold; in the literature, however, quality standards were reported [14, 18 – 21], particularly in the evaluation of screen-detected breast cancers [13]. In our experience, the use of cancer registry has been fundamental for monitoring the care of cancer patients in a community setting [22]. Population-based cancer registries provide a large number of patients and include all persons in the community. They provide the date of incidence and the living status of the patient, essential information for the evaluation of the clinical pathway of each cancer case [23 – 25]. A limitation in the use of data collected by the cancer registry is that some information, particularly regarding treatment, may not be completely recorded or may be missing. However, for 2004 all incident cases collected from cancer registry were considered and the completeness of the cancer registry in Tuscany is high [26]. Data quality of registry was confirmed by quality indicators which are similar to the average of the Italian registry network. Analysis on data set from 2006, based on data extracted from the same population used for registry, generally
416
confirmed the results obtained with cancer registry data in 2004 or showed an increased adherence to recommendation. The data from 2006 were from hospital discharge notes integrated with the pathology report and clinical charts, with very good data completeness. Our results can be compared with the literature that describes the quality of cancer care. In 2004, the result in the Tuscany regional system confirmed a good homogeneity among areas and globally the estimate of the selected indicators showed a good performance. Patients with breast, lung and rectum cancer receive .60% of recommended care, and patients with colon cancer receive .75% of recommended care in 2004; adherence to several quality measures reached 90%. Globally, our data showed that cancer care in our region is as good as that reported in the literature. The results of the National Initiative for Cancer Care quality showed that patients with breast cancer receive 86% of recommended care and patients with colorectal cancer receive 78% of recommended care [27], whereas other studies found that patients with colorectal cancer receive 86% of recommended care [28]. In each indicator, for some of these selected measures rates were similar to data from the literature, as, for example, for the proportion of patients with stage III rectal cancer who received adjuvant therapy, whereas in stage II our data found a lower proportion of patients than in other recent studies [28]. The proportion of patients whose colorectal cancer was detected by screening was 16.5% in a recent analysis, according to our data [29]. A low adherence was observed for some quality measures, particularly regarding use of adjuvant therapy. For some of these measures, as sentinel lymph node evaluation in breast cancer, there was variability across the three regional areas and the adoption of these techniques was higher in university hospitals. Our analysis found that data from cancer registry linked with electronic health records can be useful to evaluate quality of cancer care. Further work, however, is needed to develop specific quality measures about structural data, like hospital volume or access to care. Furthermore, indicators will need to be continually updated and revised with the improvement of scientific knowledge and of the information system.
Acknowledgements The authors thank N. Susini (logistic coordinator), C. Buzzoni, F. Martinelli, C. Sacchettini, G. Manneschi, V. Bucciarelli, F. Filippi, V. Meoni, M. Romei, E. Sereno, G. Diadoro, E. Innocenti, M. Romanu, D. Tanzini, D. Casella and T. Intrieri. They also thank the nurses, doctors and managers who cooperated in the data collection and revision.
Funding This research has been funded by the Tuscany Region and by the Italian Ministry of Health Research Grant ( project:La
Use of registry data in quality care †
funzione di sorveglianza e monitoraggio della rete oncologica regionale: gli strumenti di rilevazione e il sistema di indicatori per la valutazione degli enti e della innovazione tecnologica”).
References 1. Mainz J. Defining and classifying clinical indicators for improvement. Int J Qual Health Care 2003;15:523 –30. 2. Mainz J, Hansen AM, Palshof T et al. National quality measurement using clinical indicators: the Danish National Indicator Project. J Surg Oncol 2009;99:500 –4. 3. Bilimoria KY, Bentrem DJ, Lillemoe KD et al.; on behalf of the American College of Surgeons’ Pancreatic Cancer Quality Indicator Development Expert Panel. Assessment of pancreatic cancer care in the United States based on formally developed quality indicators. J Natl Cancer Inst 2009;101:848 –59.
Quality Measurement and Health Information Technology
of patients with non small cell lung cancer: a first step toward implementing a multidisciplinary, evidence based guideline. Lung Cancer 2006;54:117–24. 15. Malin JL, Kahn KL, Kwan L et al. Validity of cancer registry data for measuring the quality of breast cancer care. J Natl Cancer Inst 2002;94:835– 44. 16. Harlan LC, Clegg LX, Abrams J et al. Community-based use of chemotherapy and hormonal therapy for early stage breast cancer: 1987– 2000. J Clin Oncol 2006;24:872 –77. 17. Surveillance, Epidemiology and End Results (SEER): www.seer. cancer.gov/seerstat 18. Hearnshaw HM, Harker RM, Cheater FM et al. Expert consensus on the desiderable characteristics of review criteria for improvement of health care quality. Qual Health Care 2001;10:173– 8. 19. McGory ML, Shekelle PG, Ko CY. Development of quality indicators for patients undergoing colorectal cancer surgery. J Natl Cancer Inst 2006;98:1623 –33.
4. McColl A, Roderick P, Wilkinson E et al. Clinical governance in primary care groups: the feasibility of deriving evidence-based performance indicators. Quality Health Care 2000;9:90– 7.
20. Palmer RH, Reilly MC. Individual and institutional variables which may serve as indicators of quality of medical care. Med Care 1979;17:693–717.
5. Gelormino E, Pagano E, Appiano S et al. Adapting clinical practice guidelines to a regional oncology network: the Piedmont experience. Epidemiol Prev 2007;31:25 –33.
21. Gagliardi AR, Simunovic M, Langer B et al. Development of quality indicators for colorectal cancer surgery, using a 3-step modified Delphi approach. Can J Surg 2005;48:441–51.
6. Paci E, Crocetti E, Benvenuti A et al. Cancer incidence in Florence and Prato (1998 –2002). In: Curado MP, Edwards B, Shin HR, Storm H, Ferlay J, Heanue M, Boyle P (eds). Cancer Incidence in Five Continents. Vol. IX. IARC Scientific Publications No. 160. Lyon: IARC, 2007.
22. Malin JL, Asch SM, Kerr EA et al. Evaluating the quality of cancer care. Cancer 2000;88:71–707.
7. AIRT Working Group. The ‘AIRT database in’ ‘Italian Cancer Figures-Report 2006: Incidence, mortality and estimates’. Epidemiol Prev 2006;30 (Suppl 2):12 –6. 8. Istituto Toscano Tumori. Raccomandazioni cliniche per i principali tumori solidi, 2005: www.ittumori.it 9. Mc Evoy SP, Ingram DM, Byrne MJ et al. Breast cancer in Western Australia: clinical practice and clinical guidelines. MJA 2004;181:305–9.
23. Edwards BK, Brown ML, Wingo PA et al. Annual Report to the Nation on the Status of Cancer, 1975 –2002, featuring population-based trends in cancer treatment. J Natl Cancer Inst 2005;97:1407 – 27. 24. Rubin HR, Proovost P, Diette GB. From a process of care to a measure: the development and testing of a quality indicator. Int J Quality Health Care 2001;13:489 –96. 25. Brien SE, Dixon E, Ghali WA. Measuring and reporting on quality in health care: a framework and road map for improving care. J Surg Oncol 2009;99:462 –6.
10. Perry N, Broeders M, de Wolf C et al. European guidelines for quality assurance in breast cancer screening and diagnosis. Fourth edition-summary document. Ann Oncol 2008;19:614 –22.
26. Crocetti E, Miccinesi G, Paci E et al. An application of the twosource capture-recapture method to estimate the completeness of the Tuscany Cancer Registry, Italy. Eur J Cancer Prev 2001;10:417– 23.
11. Barni S, Venturini M, Beretta GD et al. Agreement between oncology guidelines and clinical practice in Italy: the ‘right “program. A project of the Italian Association of Medical Oncology (AIOM). Ann Oncol 2007;18 (suppl 6):vi179 –84.
27. Malin JL, Schneider EC, Epstein AM et al. Results of the national initiative for cancer care quality: how can we improve the quality of cancer care in the United States? J Clin Oncol 2006;24:626 –34.
12. Prosnitz R, Patwardhan MB, Samsa GP et al. Quality measures for the use of adjuvant chemotherapy and radiation therapy in patients with colorectal cancer. A systematic review. Cancer 2006;107:2352–60.
28. Chung KP, Chang YJ, Lai MS et al. Is quality of colorectal cancer care good enough? Core measures development and its application for comparing hospitals in Taiwan. BMC Health Serv Res 2010;10:27.
13. Mantellini P, Casella D, Bianchi S. Distante V and Gruppo Regionale per la raccolta degli indicatori di trattamento delle lesioni mammarie screen-detected: “I controlli di qualituˆ del trattamento chirurgico per il carcinoma mammario: l’esperienza di alcune aziende sanitarie della Toscana” in “I programmi di screening della Regione Toscana: ottavo rapporto annuale”. Eds Firenze Scientific Press, 2007. 14. Hermens RP, Ouwens MM, Vonk-Okhuijsen SY et al. Development of quality indicators for diagnosis and treatment
29. Vergara-Fernandez O, Swallow CJ, Victor JC et al. Assessing outcomes following surgery for colorectal cancer using quality of care indicators. Can J Surg 2010;53:232 –40. 30. Centro di riferimento per l’epidemiologia e la prevenzione oncologica in Piemonte (CPO). Commissione oncologica regionale: Linee guida clinico organizzative per la Regione Piemonte: Tumore della mammella, 2002.
417
Caldarella et al.
31. SIGN: Scottish intercollegiate guidelines network, 1998: www. sign.ac.uk 32. CMA: Canadian medical association, 2001: www.cma.ca 33. GLICO: Gruppo di lavoro interaziendale oncologico, provincia di Modena, Italy, 2000: www.ceveas.it 34. Rosselli Del Turco M, Ponti A, Bick U et al. Quality indicators in breast cancer care. Eur J Cancer 2010;46:2344 –56.
418
35. Jackson GL, Melton LD, Abbott DH et al. Quality of non metastatic colorectal cancer care in the department of veterans affairs. JCO 2010;28:3176 –81. 36. Nielsen SS, He Y, Ayanian JZ et al. Quality of cancer care among foreign-born and US born patents with lung or colorectal cancer. Cancer 2010;116:5497 – 506. 37. Tanvetyanon T. Quality of care indicators for non small cell lung cancer. Cancer Control 2009;16:335 –41.
