Quality Indicators for Non-Small Cell Lung Cancer Operations With ...

Quality Indicators for Non-Small Cell Lung Cancer Operations With Use of a Modified Delphi Consensus Process

Division of Thoracic Surgery, Department of Surgery, University Health Network, Toronto; Division of Thoracic Surgery, Department of Surgery, London Health Sciences Centre, Western University, London; Department of Surgery, Lakeridge Health Corporation, Oshawa; Cancer Care Ontario; Department of Surgery and Samuel Lunenfeld Research Institute, Mount Sinai Hospital; and Department of Surgery and Department of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada

Background. The aim of this project was to develop a set of quality indicators to assess surgical decision making in the care of patients with non-small cell lung cancer (NSCLC). Methods. A multidisciplinary Expert Panel of 16 physicians used a modified Delphi process to identify quality indicators that evaluated the processes of care in patients with NSCLC. A systematic review identified potential indicators, which were rated on actionability, validity, usefulness, discriminability, and feasibility in two rounds of questionnaires. The first questionnaire was completed by the Expert Panel and by the larger thoracic surgical community of practice; the second questionnaire was sent to only the Expert Panel. Expert Panel members attended an in-person meeting to review the results of the two questionnaires and to compile the final list of indicators by consensus.

Results. From the literature review, 41 potential indicators were identified. An additional 16 indicators were suggested by the Expert Panel: 13 indicators in the two rounds of questionnaires and three after the discussion at the in-person meeting. One further indicator was identified after the in-person meeting. In the end, 17 indicators were chosen from seven domains: preoperative assessment, staging, surgical procedures, pathology, adjuvant therapy, surgical outcomes, and miscellaneous. Conclusions. By use of a modified Delphi process, 17 indicators to assess the quality of processes of surgical care for patients with NSCLC were developed.

E

procedures were performed in these centers. In an effort to assess the quality of surgical care, an Expert Panel was convened to develop a set of quality indicators that would assess the surgeon decision making component of care of patients with non-small cell lung cancer (NSCLC). This article outlines the process and outcome of those deliberations.

nsuring access to high-quality cancer surgical procedures is one of the mandates of Cancer Care Ontario (CCO), which is the principal adviser to the Ontario government on the delivery of cancer services. An Expert Panel was convened to develop thoracic surgical oncology standards because data from the province of Ontario and from other jurisdictions reported improved outcomes for pneumonectomy and esophagectomy for high-volume centers (unpublished data). The standards document stipulated volume levels, surgeon criteria, and hospital resources for level I and level II centers. Before 2004, approximately 46 hospitals in Ontario were performing thoracic surgical procedures. After the implementation of the standards, as of December 31, 2010, there were 13 level I and two level II designated thoracic surgery centers, and approximately 90% of all

Accepted for publication March 4, 2014. Address correspondence to Dr Darling, Toronto General Hospital, Division of Thoracic Surgery, 200 Elizabeth St, 9N955, Toronto, Ontario M5G 2C4, Canada; e-mail: [email protected].

Ó 2014 by The Society of Thoracic Surgeons Published by Elsevier Inc

(Ann Thorac Surg 2014;98:183–90) Ó 2014 by The Society of Thoracic Surgeons

Material and Methods A modified Delphi method was used to develop the Lung Cancer Surgery Quality Indicators (LCSQI) (Fig 1). This method combines the nominal group process, which uses structured meetings to gather information from experts, and the Delphi process, which uses questionnaires to elicit responses in a systematic manner over several rounds [1]. In the modified Delphi process, participants attend at least one in-person meeting. During development of the LCSQI, a systematic review of the literature was performed to extract potential indicators, which were reviewed and rated by clinical experts in two rounds of questionnaires and at an in-person meeting. 0003-4975/$36.00 http://dx.doi.org/10.1016/j.athoracsur.2014.03.001

GENERAL THORACIC

Gail Darling, MD, Richard Malthaner, MD, John Dickie, MD, Leigh McKnight, BMSc, Cindy Nhan, BSc, Amber Hunter, MBA, and Robin S. McLeod, MD, on behalf of the Lung Cancer Surgery Expert Panel

184

DARLING ET AL LUNG CANCER OPERATION QUALITY INDICATORS

Ann Thorac Surg 2014;98:183–90

GENERAL THORACIC

research evidence including clinical practice guidelines, systematic reviews, metaanalyses, population-based studies, and consensus statements that supported interventions and practices leading to improved patient outcomes. The MEDLINE database was searched by use of the following text/MeSH subject headings: “Surgical procedures,” “Operative/Surgery/Surgery Department,” “Hospital/Thoracic Surgery,” “Thoracic Surgical Procedures,” and “Thoracic Neoplasms.” The search was limited to human studies in the English language published between 1995 and the second week of January 2009. Two reviewers (CN and LM) independently reviewed the abstracts, and disagreements were resolved by consensus. Articles were selected for inclusion if they were fully published English-language reports. The grey literature was searched with the Google search engine. Also searched for were nonpublished organizational reports, guidelines, and statements. Reviewers hand-searched the reference lists of the included articles and all review articles to identify any additional articles missed in the literature search. Each retrieved article was examined by two independent reviewers (CN and LM) to extract possible indicators for consideration. For each potential quality indicator, the phase of care, type of article, and citation were collected, and a list of unique indicators with the level of supporting evidence was created. The indicators were organized into seven categories that covered the continuum of care (preoperative assessment, staging, surgical procedures, pathology, adjuvant therapy, surgical outcomes, and miscellaneous).

Round 1 Fig 1. Modified Delphi process used to select and prioritize quality indicators for surgical procedures for patients with non-small cell lung cancer. (CoP ¼ community of practice.)

Panel Selection Three thoracic surgeons representing both academic and community centers led the panel (GD, JD, RM). Other members were nominated by these physician leaders, ensuring that the majority of regions of the Province of Ontario were represented. In addition to thoracic surgeons, representatives from radiation oncology, medical oncology, respirology, diagnostic radiology, nuclear medicine, and CCO administrators were included on the panel. In total, the Expert Panel was composed of 16 members, including the three physician leaders.

Literature Search A literature search was performed by two of the authors (CN and LM) to identify potential evidence-based quality indicators that were relevant to surgical decision making (Fig 2). The target population was adult patients with stage I, II, or III NSCLC who underwent surgical resection with curative intent. The goal was to identify synthesized

The list of unique quality indicators was formatted by category in a questionnaire and distributed by e-mail to all members of the Expert Panel and to all other thoracic surgeons practicing at designated centers in Ontario, with the exception of the three physician leaders. Respondents were asked to rate the indicators by three criteria: (1) actionability, which refers to whether the care that the indicator assesses is under the control of the stakeholder (ie, the surgeon); (2) validity, which assesses the indicator as a measure of quality; and (3) usefulness, which determines whether the stakeholders would find data on the indicator informative. Each criterion was ranked on a seven-point Likert scale (1 ¼ disagree, 7 ¼ agree). Respondents were also given the opportunity to provide written comments and were asked to suggest additional indicators. E-mail reminders were sent 2 weeks after the initial distribution, with follow-up phone calls to Expert Panel members a week later. Questionnaire responses were analyzed and results organized according to whether there was consensus for inclusion (>51% of respondents ranked all three criteria 5, 6, or 7 on the Likert scale), consensus for exclusion (>51% of respondents ranked all three criteria 1, 2, 3, or 4 on the Likert scale), or unclear consensus (indicators that did not meet the criteria for inclusion or exclusion).



185

GENERAL THORACIC

Fig 2. Literature search to identify quality indicators for surgical procedures for patients with lung cancer.

Round 2 In round 2, the indicators were organized according to those for which there was consensus for inclusion, consensus for exclusion, or unclear consensus based on the results of round 1. Additional indicators suggested in the first round were included for rating. Information on the frequency of responses and comments from round 1 were also included. Panel members were asked to rate the original indicators on two additional criteria: (1) discriminability, which refers to whether the indicator shows variability in practice and (2) feasibility, which refers to whether the indicator is feasible to measure. Respondents could answer either “yes”, “no,” or “unsure” to these criteria. The additional indicators suggested in round 1 were rated by all five criteria (actionability, validity, usefulness, discriminability, feasibility). The questionnaire was distributed by e-mail exclusively to the 13 Expert Panel members (excluding the three physician leaders), followed by a reminder e-mail 2 weeks later. In a manner similar to round 1, responses were analyzed, and frequencies were calculated and categorized as clear consensus for inclusion, clear consensus for exclusion, or unclear consensus. On the basis of these results, the indicators were separated into the three groups for discussion at the Expert Panel meeting.

Round 3 Panel members were invited to an in-person Expert Panel meeting, facilitated by two of the physician leaders (GD, JD). It was anticipated that after two rounds, there would be fewer potential indicators to discuss. However, there was actually an increase from 41 original indicators to 54 for review because of additional indicator suggestions and only one indicator having clear consensus for exclusion after rounds 1 and 2. The indicators were divided into seven categories of care for discussion at the in-person meeting. The goal was

to have approximately 10 to 15 indicators, with one to three indicators from each of the seven categories. The Expert Panel members discussed each potential indicator and by consensus eliminated some indicators, leaving between one and five potential indicators in each category. Modifications, such as wording changes and combining of indicators, were also made. Discussion resulted in the addition of three more indicators to the list. At the end of the discussion period, the selected indicators were included in the final (third) questionnaire, which was completed by all Expert Panel members who attended the in-person meeting. They were asked to rank the top three indicators in each category that were thought to be the best measures of quality of lung cancer surgical procedures. The top indicator in each category was assigned three points, the second was assigned two points, and the third was assigned one point. If there were fewer than three indicators in a category, all indicators were ranked. The indicator in each category with the most points was automatically selected for inclusion. The rest of the indicators were selected by consensus of the physician leaders. After the in-person meeting, the physician leaders made several modifications to the chosen indicators, which included the addition of one indicator, the omission of one indicator, and wording modifications to several indicators to more accurately reflect the evidence. The evidence for each indicator was also reviewed again. The final list was e-mailed to all Expert Panel members for their review and approval.

Results The development process began in October 2008 and concluded in December 2011. The Expert Panel included nine surgeons, two radiation oncologists, a medical oncologist, a respirologist, a radiologist, a nuclear

186


medicine physician, and a regional vice president of CCO, representing 11 of 13 regions in Ontario with a designated thoracic surgery center. As shown in Figure 2, 592 articles were identified in the literature search, of which 263 full-text articles were reviewed and 25 articles were used to identify 41 unique potential LCSQI.

Round 1

GENERAL THORACIC

The response rate for the first-round questionnaire was 100% for the Expert Panel and 33% (11/33) for thoracic surgeons who were not on the panel. One questionnaire was excluded because of missing fields. There was clear consensus among respondents that all indicators in questionnaire No. 1 were important, and an additional 13 indicators were suggested.

Round 2 The second questionnaire contained 54 quality indicators (41 original plus 13 additional indicators suggested from round 1). The response rate for the second questionnaire was 85%. After round 2, one indicator was excluded, 42 indicators had clear consensus for inclusion, and 11 indicators had unclear consensus.

Round 3 Eleven Expert Panel members (including two of the three leaders) and the surgical oncology program staff attended the meeting. All 54 potential indicators and the accompanying results of rounds 1 and 2 were presented. After the discussions and by consensus from the Expert Panel members, 34 indicators were eliminated. The remaining 20 LCSQI across the continuum of care were deemed good measures of quality and were included in questionnaire No. 3 (Preoperative assessment, 3; Staging, 4; Surgical Procedures, 5; Pathology, 2; Adjuvant Therapy, 1; Surgical Outcomes, 4; Miscellaneous, 1). Seventeen of these indicators had been ranked in the rounds 1 and 2 questionnaires, and the other three were suggested during discussion at the in-person meeting. The results of questionnaire No. 3 were tallied according to the previously described ranking system, and a preliminary list of quality indicators was chosen, which was then modified by the physician leaders. In addition to minor wording changes to several indicators, the physician leaders eliminated one indicator from the preliminary list (Proportion of resected stage I and stage II NSCLC patients older than 75 years who receive an anatomic resection). They also added an indicator that had not been previously discussed (Proportion of patients scheduled to have an elective lung cancer resection that is canceled). After these modifications were made, 17 LCSQI remained. This final list was distributed to the members of the Expert Panel for their review, and there was unanimous agreement, resulting in the final list of 17 LCSQI (Table 1).

Comment Performance measurement is important for health care organizations interested in assessing and reporting on the


quality of services. Valid, clinically relevant, and sensitive indicators are essential to this process. Quality indicators are measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of care [23]. Structural, process, or outcome quality indicators may be chosen, depending on the intended stakeholders. Thus, in the development of indicators, the purpose of the indicators should be explicitly stated, and the indicators should have face validity to the stakeholders. Indicators should be supported by evidence. Depending on the strength of the evidence, expert consensus may be required. Measurement of the indicators must be feasible and actionable under the control of the stakeholders. We chose to use a modified Delphi process rather than a consensus conference because it avoids domination of the group by thought leaders. It ensures anonymity among respondents, allowing them more freedom in their responses and opinions [24]. Last, experts can be surveyed several times regardless of their geographical location or time constraints [24]. CCO has used this method previously to develop quality indicators for surgical procedures for colorectal cancer [25] and ovarian cancer [26]. Although the intention was to follow a modified Delphi process, our indicator development process varied slightly because we had an increase in the number of potential indicators after two rounds of questionnaires (only one potential indicator was excluded, and 13 more were added), whereas usually indicators would be eliminated in the rounds 1 and 2 questionnaires. Thus, in an attempt to decrease the total number of indicators, it was arbitrarily decided that only one to five indicators would be included in each category in the final list. At the inperson meeting, the potential indicators in each category were discussed, and by informal agreement, 34 indicators were omitted, so 20 indicators remained. The process also differed in that 16 additional potential indicators were added to the list of indicators identified through a review of the literature: 13 were suggested in round 1 and three at the in-person meeting. A formal systematic review to identify supporting evidence for these indicators was not conducted. In addition, after completion of the third questionnaire, the Expert Panel leaders made several modifications to the indicator list, which included omitting one indicator deemed to be redundant, adding one indicator that had not initially been identified, and making wording modifications to more accurately reflect current evidence. The list of LCSQI with the modifications was distributed to the Expert Panel, and the final list was approved. The final list of 17 LCSQI assesses aspects of care that are at the discretion of surgeons. Fourteen are process measures, and three are outcome measures. Process measures tend to be more useful because outcome indicators may not be immediately available (eg, 5-year survival) and may be affected by factors other than the health care system (eg, patient status, preferences). As shown in Table 1, 12 of the indicators were identified in the systematic review; four were suggested by the Expert

Evidence

Indicator No.

Indicator Definition

Preoperative assessment 1 Proportion of NSCLC patients who underwent operations who received a CT scan of the chest and a pulmonary function assessment preoperatively 2 Time from date of consultation with the surgeon to date of decision to treat Staging 3 4

5

7 8

9 Pathology 10

Metaanalysis

Six [2, 3, 4, 5, 6a, 7]

Systematic Review

PopulationBased Study

One [8]

Expert Consensus

Additional Comments

One [9]

Not identified in evidence; consensus by Expert Panel

Proportion of resectable clinical stage I-IIIA NSCLC Five [4, 16, 5, 7, 6a] patients who receive preoperative PET-CT scanning Proportion of NSCLC patients with no extrathoracic Five [7, 5, 12, 6a, 2] metastases who receive invasive mediastinal staging (excluding those who have a normal CT results and negative PET-CT results in the mediastinum and a peripheral clinical stage 1A tumor) Proportion of NSCLC patients who receive biopsy One [6a] of at least one ipsalateral, one contralateral, and one subcarinal node to include suggestive nodes on preoperative imaging during invasive mediastinal staging

One [11] One [13]

Two [9, 11]

Expert Panel added “suspicious nodes” to indicator

Proportion of patients with resected stage I and stage II NSCLC who receive an anatomic resection

Five [14, 2, 19, 7, 5]

One [15]

One [16]

Proportion of all patients with stage I and stage II NSCLC who receive a resection Proportion of all patients with stage I and stage II NSCLC older than 75 years who receive a resection

Five [14, 2, 10, 7, 5]

One [15]

One [16]

Proportion of patients found to be unresectable at the time of operation

One [2]

Proportion of patients with resected NSCLC who receive adequate lymph node dissection or sampling (>10 lymph nodes removed and at least 3 mediastinal lymph node stations sampled) at the time of resection

Five [14, 19, 10, 5, 7, 4]

Two [2, 3]

Expert Panel separated out indicator for anatomic resection from all resections

One [8]

Two [17, 18]

Expert Panel changed age from 71 years to 75 years

One [11]


Operation 6

Clinical Practice Guideline


Table 1. Final List of NSCLC Operation Quality Indicators and Supporting Evidence

(Continued)

187

GENERAL THORACIC

GENERAL THORACIC

188

Table 1. Continued Evidence

Indicator Definition

11

Proportion of patients with resected NSCLC with an R0 resection Adjuvant therapy 12 Proportion of patients with completely resected stage II NSCLC who received a consultation with a medical oncologist for consideration of adjuvant chemotherapy

Metaanalysis

One [19]

Three [20, 15, 21]

a

Expert Consensus

Additional Comments

One [22]

One [11]

Expert Panel changed from referral to a medical oncologist to consultation with a medical oncologist Not identified in evidence; consensus by Expert Panel Not identified in evidence; consensus by Expert Panel Not identified in evidence; consensus by Expert Panel

30-day reoperation rate after surgical resection

Miscellaneous 16 Proportion of patients with clinical or pathologic stage IIIA and stage IIIB NSCLC who receive multidisciplinary evaluation either by direct consultation or discussion at a multidisciplinary cancer conference 17 Proportion of patients scheduled to have an elective lung cancer resection that is canceled

PopulationBased Study

One [10]

Surgical outcomes 13 Proportion of patients undergoing resection who stay in an acute care hospital >14 days after anatomic resection 14 5-year overall survival rate for resected patients, by stage 15

Systematic Review


Indicator No.

Clinical Practice Guideline

Three [10, 19, 3]

Not identified in evidence; added by leaders of Expert Panel

Guideline based on expert consensus.

CT ¼ computed tomography;

NSCLC ¼ non-small cell cancer;

PET ¼ positron emission tomography.


Panel, either in one of the questionnaires or at the inperson meeting; and one indicator was suggested by the leaders of the Expert Panel after the in-person meeting. All 12 indicators identified in the systematic review were supported by clinical practice guideline recommendations, metaanalysis, or systematic review, with three of those indicators also supported by population-based studies. The remaining five indicators identified were based on Expert Panel consensus, and supporting evidence was not assessed. The indicators chosen also have face validity because nine members of the panel were thoracic surgeons. Although morbidity and mortality are two commonly measured outcome indicators, CCO already collects mortality data, so those two terms were not included in our list. Morbidity was not included in our indicator list primarily because many factors other than the surgeon’s judgment or skill influence morbidity. These indicators were developed to assess the quality of lung cancer surgical procedures in the province of Ontario, specifically to determine the effect of implementation of the Thoracic Surgical Oncology Standards, which regionalized thoracic surgical procedures to 15 centers in the province. Currently, data for eight indicators can be obtained from the Canadian Insititute for Health Information database (includes all hospital discharges in Canada with the exception of Quebec), the CCO ePath database (which collects pathology and stage data by means of electronic synoptic reporting), and through Ontario Health Insurance Plan billing claims. These initial data will be analyzed and presented to the thoracic surgeons in Ontario in aggregate and anonymized form. Individual surgeon data is not collected. Our immediate goal is to determine current practice and acceptable variations and then to determine potential quality gaps. Previous indicators have been developed to assess the spectrum of care from diagnosis to treatment rather than just treatment related to the operations. Hermens and colleagues [27] developed a set of 15 indicators to assess compliance with a guideline previously developed by their group. Four indicators are similar to those developed by our group. Reifel [28] reported on a set of indicators developed by Research and Devlopment (RAND) Corporation in 2000. Indicators were again developed to assess the quality of care across the spectrum and for both NSCLC and small cell lung cancer. Some indicators are similar to ours, including the proportion of patients who have pulmonary function assessment and an electrocardiogram, and the proportion of stage I and II NSCLC patients who undergo surgical procedures. Quality indicators are important for evaluating the quality of health care delivery. In thoracic operations, historically the prime indicators used are in-hospital mortality, length of stay, and 5-year survival. Although these are important, those responsible for health care delivery require more detailed and timely measures to understand the quality of care provided. By use of a modified Delphi process, 17 quality indicators related to processes of care and outcome measures were identified for evaluating the quality of thoracic surgical care in Ontario. This process ensured that the LCSQI have face validity, and it also promotes the quality agenda in a


189

collegial collaboration among surgeons, other health care providers, and administrators. The authors thank the members of the Lung Cancer Surgery Expert Panel, Drs Paul Chiasson, Bill Evans, Marisa Finlay, Ken Gehman, Karen Gulenchyn, Matthew Kilmurry, Donna Maziak, Ken Reid, Shafeequr Salahudeen, Michael Sanatani, Julius Toth, Yee Ung, and John Vlasschaert, for their help in developing and determining the final list of Lung Cancer Surgery Quality Indicators.

References 1. Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health 1984;74:979–83. 2. British Thoracic Society and Society of Cardiothoracic Surgeons of Great Britain and Ireland Working Party. Guidelines on the selection of patients with lung cancer for surgery. Thorax 2001;56:89–108. 3. Collice GL, Shafazand S, Griffin JP, et al. Physiologic evaluation of the patient with lung cancer being considered for resectional surgery. Chest 2007;132:161S–77S. 4. Silvestri G, Gould MK, Margolis ML, et al. Noninvasive staging of non-small cell lung cancer. Chest 2007;132: 178S–201S. 5. National Comprehensive Cancer Network. NCCN clinical practice guidelines in oncology: non-small cell lung cancer. 2009. 6. De Leyn P, Lardinois D, Van Schil P, et al. European trends in preoperative and intraoperative nodal staging: ESTS guidelines. J Thorac Oncol 2007;2:357–61. 7. Scottish Intercollegiate Guidelines Network. Management of patients with lung cancer: A national clinical guideline. 2005. 8. Jaklitsch M, Mery C, Audisio R. The use of surgery to treat lung cancer in elderly patients. Lancet Oncol 2003;4:463–71. 9. American Thoracic Society/European Respiratory Society. Pretreatment evaluation of non-small cell lung cancer. Am J Respir Crit Care Med 1997;156:320–32. 10. National Institute for Clinical Excellence. Lung cancer: the diagnosis and treatment of lung cancer. Clinical Guideline 24. 2005. 11. D’Addario G, Felip E. Non-small-cell lung cancer: ESMO clinical recommendations for diagnosis, treatment and follow-up. Ann Oncol 2008;19(Suppl 2):ii39–40. 12. Detterbeck FC, Jantz MA, Wallace M, Vansteenkiste J, Silvestri GA. Invasive mediastinal staging of lung cancer. Chest 2007;132:202S–20S. 13. Kramer H, Groen H. Current concepts in the mediastinal lymph node staging of non-small cell lung cancer. Ann Surg 2003;238:180–8. 14. Scott WJ, Howington J, Feigenberg S, Movsas B, Pisters KI. Treatment of non-small cell lung cancer stage I and stage II. Chest 2007;132:234S–42S. 15. Meert AP, Sculier JP. What has the meta-analysis contributed to today’s standard of care in the treatment of thoracic malignancies? Lung Cancer 2008;61:141–51. 16. Mery C, Pappas A, Bueno R, et al. Similar long-term survival of elderly patients with non-small cell lung cancer treated with lobectomy or wedge resection within the surveillance, epidemiology, and end-results database. Chest 2005;128: 237–45. 17. Wright G, Manser RL, Brynes G, Hart D, Campbell DA. Surgery for non-small cell lung cancer: systematic review and meta-analysis of randomized controlled trials. Thorax 2006;61:597–603. 18. Alam N, Flores RM. Video-assisted thoracic surgery (VATS) lobectomy: the evidence base. JSLS 2007;11:368–74.

GENERAL THORACIC


190


GENERAL THORACIC

19. Robinson L, Ruckdeschel J, Wagner H Jr, Stevens C. Treatment of non-small cell lung cancer – stage IIIA. Chest 2007;132:243S–65S. 20. Berghmans T, Paesmans M, Meert AP, et al. Survival improvement in resectable non-small cell lung cancer with (neo)adjuvant chemotherapy: results of a meta-analysis of the literature. Lung Cancer 2005;49:13–23. 21. Nakamura H, Kawasaki N, Taguchi M, Kabasawa K. Survival following lobectomy vs limited resection for stage I lung cancer: a meta analysis. Br J Cancer 2005;92: 1033–7. 22. Sedrakyan A, van Der Meulen J, O’Byrne K, Prendiville J, Hill J, Treasure T. Postoperative chemotherapy for non-small cell lung cancer: a systematic review and meta-analysis. J Thorac Cardiovasc Surg 2004;128:4149. 23. Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743–8.


24. Campbell SM, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816–9. 25. Gagliardi A, Simunovic M, Langer B, Stern H, Brown A. Development of quality indicators for colorectal cancer surgery, using a 3-step modified Delphi approach. Can J Surg 2005;48:441–52. 26. Gagliardi A, Fung MF, Langer B, Stern H, Brown A. Development of ovarian cancer surgery quality indicators using a modified Delphi approach. Gynecol Oncol 2005;97:446–56. 27. Hermens RP, Ouwens MM, Vonk-Okhuijsen SY, et al. Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: a first step toward implementing a multidisciplinary, evidence-based guideline. Lung Cancer 2006;54:117–24. 28. Reifel J. Quality of care for oncologic conditions and HIV: lung cancer. RAND Monograph Report. Chapter 7:133–71.

INVITED COMMENTARY Evaluation of quality of care in thoracic surgery is of paramount importance for patients, providers, and administrators. Quality is an abstract and complex construct, which cannot be captured or explained by a single indicator. Several components may influence what is commonly defined as quality of care. Selection of quality indicators therefore becomes critical to the definition and grading of quality of care. Traditional outcome endpoints (mortality and morbidity) are insufficient to fully capture the complexity of lung cancer patient management and are influenced by many factors often not under the direct control of the surgeon or their team. Process indicators appear more appropriate for quality improvement initiatives as they are actionable by the providers. Darling and colleagues [1] reported the work of a multidisciplinary expert panel mandated by the Cancer Care Ontario to identify a set of quality indicators aimed at assessing the surgeon decision-making components of care for non-small cell lung cancer patients. Seventeen indicators (14 processes and 3 outcomes) were finally agreed using a modified Delphi method. There are some caveats that in my opinion would need to be addressed in future investigations. (1) Process indicators need corresponding legitimate exclusion criteria. Allowing for selected patients to legitimately fall out of the predetermined process scheme would prevent to penalize those units dealing with very high-risk patients. (2) The outcome indicators need to be risk adjusted to be fairly applied. This would prevent risk-aversion attitudes. Future analyses should focus at developing risk-adjusting models for the selected outcomes. (3) The inclusion of both outcomes and processes warrants exploratory analyses to identify methods for

Ó 2014 by The Society of Thoracic Surgeons Published by Elsevier Inc

their concomitant application. Composite performance scores have been previously proposed to this purpose [2, 3]. Composite scores are appealing because they express with a single number the performance of a provider. Nevertheless, a composite score must be actionable and interpretable by the provider and should retain the possibility to be decomposed into its individual components. This would allow providers to analyze their performance in specific areas and to formulate improvement strategies. The authors should be commended for this important effort, which represents in my opinion a milestone contribution on this subject and will constitute a solid reference for future lung cancer management quality initiatives. Alessandro Brunelli, MD Department of Thoracic Surgery St. James’s University Hospital, Bexley Wing Beckett St Leeds, LS9 7TF United Kingdom e-mail: [email protected]

References 1. Darling G, Malthaner R, Dickie J, et al. Quality indicators for non-small cell lung cancer operations with use of a modified Delphi consensus process. Ann Thorac Surg 2014;98:183–90. 2. Shahian DM, Edwards FH, Ferraris VA, et al. Quality measurement in adult cardiac surgery: part 1—Conceptual framework and measure selection. Ann Thorac Surg 2007;83: S3–12. 3. Brunelli A, Rocco G, Van Raemdonck D, Varela G, Dahan M. Lessons learned from the European thoracic surgery database: the Composite Performance Score. Eur J Surg Oncol 2010;36(Suppl 1):S93–9.

0003-4975/$36.00 http://dx.doi.org/10.1016/j.athoracsur.2014.04.011