Developing and Testing TEAM (Team Evaluation and Assessment ...

Ann Surg Oncol (2012) 19:4019–4027 DOI 10.1245/s10434-012-2493-1

ORIGINAL ARTICLE – HEALTHCARE POLICY AND OUTCOMES

Developing and Testing TEAM (Team Evaluation and Assessment Measure), a Self-assessment Tool to Improve Cancer Multidisciplinary Teamwork C. Taylor, MA1, K. Brown, PhD2, B. Lamb, MRCS2,3, J. Harris, MSc1, N. Sevdalis, PhD2, and J. S. A. Green, FRCS3,4 Florence Nightingale School of Nursing and Midwifery, King’s College London, London, England; 2Department of Surgery and Cancer, Imperial College London, London, England; 3Department of Urology, Whipps Cross University Hospital, London, England; 4Faculty of Health and Social Care, London South Bank University, London, England

1

ABSTRACT Background. Cancer multidisciplinary teams (MDTs) are well established worldwide and are an expensive resource yet no standardised tools exist to measure performance. We aimed to develop and test an MDT self-assessment tool underpinned by literature review and consensus from over 2000 UK MDT members about the ‘‘characteristics of an effective MDT.’’ Methods. Questionnaire items relating to all characteristics of MDTs (particularly Leadership and Chairing; Teamworking and Culture; Patient-centred care; Clinical decision-making process; and Organisation and administration during meetings) were developed by an expert panel. Acceptability, feasibility and psychometric properties were tested by online completion of the questionnaire by 23 MDTs from 4 UK NHS Trusts followed by interviews with 74 team members including members from all teams and nonresponders. 10 of the MDTs also completed questionnaires that directly translated each characteristic to an item (for the five domains above) to test content validity. Results. A total of 47 items were created, each rated for agreement on a 5-point scale. A total of 329 (52 %) of 637 team members completed the questionnaire, including representation from medical, nursing and clerical MDT members. Responses correlated well with domain-specific questionnaires (r [ 0.67, p = 0.01), most domain-scales had acceptable internal consistency (Cronbach alpha

Ó Society of Surgical Oncology 2012 First Received: 27 February 2012; Published Online: 21 July 2012 C. Taylor, MA e-mail: [email protected]

[ 0.60), and good item discrimination (majority of items r \ 0.20). Team members were positive about its value. Conclusions. Self-assessment of team performance using this tool may support MDT development.

The increasing complexity of treatment and management decisions for cancer patients has led to the need for the relevant nursing, surgical, medical and diagnostic experts to work closely together in order to optimise patient care. As a consequence, multidisciplinary teams (MDTs) are firmly established at the core of cancer care in the UK and in many other countries worldwide. In the UK it is mandatory for all new patients to be discussed in weekly MDT meetings (akin to multidisciplinary case conferences or tumour boards) where all relevant clinical and patientbased information should be shared and discussed and treatment recommendations agreed. The concept of the MDT varies among countries though many countries have a forum for ensuring that relevant expertise is included when making treatment recommendations.1 In England MDTs are organised within 28 cancer networks. Care is provided in local hospitals by a local MDT (members of which may be based in one or more hospital) who are able to manage and treat the more common cancers and less complex presentations of cancer. For rarer cancers and more complex cases, there exist specialist teams that are based (or hosted) in cancer centres. Generally there is at least one cancer centre per cancer network. The membership and structure of MDTs is tumour-specific and detailed in Improving Outcomes Guidance (published by NICE). MDTs are a very expensive resource. There are approximately 1500 cancer MDTs in England, and based simply on data about the time taken for radiologists and

4020

pathologists to prepare for MDT meetings, it has been estimated they cost £100 million a year for attendance and preparation time.2,3 This investment in resources has todate been unmatched by robust evidence of their effectiveness. Indeed we know very little about how well MDTs are functioning either in or outside the weekly meetings. In England, MDTs are subject to a mandatory self-assessment and independent peer review process whereby every MDT has to provide evidence to demonstrate compliance with tumour-specific best practice, and may also be reviewed by an independent panel of health professionals and patient representatives (http://www.cquins.nhs.uk).4 Data from this process provide evidence of wide variability in performance in relation to the standards assessed—including standards for the structure of MDTs (e.g., membership and attendance at meetings) and having relevant protocols in place for referral and treatment.5 In addition, there is evidence that MDTs also vary in relation to their team processes—such as leadership, teamworking and team decision making. Such processes are not captured via the UK peer review system, but have shown to be instrumental for optimising decision making.6,7 MDT meeting discussions influence the quality of diagnostic and treatment decisions.8–10 This is likely to require inclusive discussion across a range of occupational groups.11 Poor teamworking in MDMs may lead to noninclusive discussions resulting in information bias, with discussions centred on the tumour rather than the person.12,13 Failure of the team to consider all relevant information may lead to poor decisions, or to decisions which are not implemented as they are unacceptable to patients or clinically inappropriate.14–17 Nonimplementation of MDT meeting recommendations can have both clinical and financial consequences if further discussion is required and treatment is delayed. In order to optimise team performance and team decision making in cancer MDTs it is necessary first to develop valid performance measurement tools to enable teams to identify where they are performing well and where they could improve.17 Within health care, an expanding body of evidence shows that team performance relates to patient safety as well as other parameters of quality in patient care.18,19 Team assessment tools and relevant training modules have been developed and systematic reviews demonstrating their efficacy are now available, though little of this work has involved cancer MDTs.20–23 Assessment tools have the added advantage that they can be both diagnostic and interventional if used to provide structured feedback. This technique has yet to be evaluated systematically for cancer MDTs though has been shown to improve performance in other health care teams.24 A valid self-report instrument that enables an MDT to comprehensively self-assess the core functions of their team

C. Taylor et al.

and team meetings is desired by team members.25 In 2010, the National Cancer Action Team (NCAT) published ‘‘The Characteristics of an Effective MDT,’’ comprising recommendations for effective MDT-working based upon consensus from over 2000 MDT members.25,26 We aimed to develop and test the acceptability, feasibility and psychometric properties of a team assessment questionnaire, underpinned by the ‘‘Characteristics of an effective MDT’’ and intended as a stimulus to team self-assessment and improvement.

METHODS Development of TEAM On the basis of preliminary work undertaken with over 60 MDTs (over 300 team members) it was determined that MDT assessment should include a 360-degree questionnaire completed by all individual team members to act as a discussion stimulus (unpublished data). Intended to be developmental rather than judgemental, it was agreed that the resulting data would be shared with team members only (in the first instance) in an anonymised format. Individual questionnaire items were created by an expert panel that included members with clinical expertise (J.G., B.L.) and academic expertise (C.T., K.B., N.S.) in both MDT working and questionnaire development. The questionnaire content was informed by the research evidence regarding influences on effectiveness in MDT; the recommendations within the ‘‘Characteristics of an Effective MDT’’ report; and results from the national survey completed by over 2000 MDT members in the UK.3,6,25–27 The Characteristics report organises recommendations for optimal MDT-working under 5 domains: (i) the team; (ii) infrastructure for meetings; (iii) meeting organisation and administration; (iv) patient-centred clinical decisionmaking; and (v) clinical governance.26 These 5 domains are further separated into 17 subdomains (Table 1). We used the 5 domains/17 subdomains structure for our survey. Forty-two questionnaire items were created. Items covered all 17 subdomains of teamworking with particular emphasis on Leadership and Chairing; Teamworking and Culture; Patient-centred care; Clinical decision-making process; and Organisation and administration during meetings. This was due to (i) evidence of their importance to optimal patient care; (ii) not being assessed through the National Peer Review Programme; and (iii) confirmation in preliminary work involving over 60 MDTs that team assessment should prioritise these domains (unpublished data). In addition, the expert panel developed five items aimed at measuring team members’ general perceptions of the effectiveness of their team including one ‘‘global’’ item. All 47 items were designed to be rated on 5-point

TEAM to Self-assess MDTs

4021

TABLE 1 Sample TEAM items according to domains/subdomains of teamworking Domain of teamworkinga The team

Subdomain (no. of TEAM items)a

Sample TEAM items

Membership (1) Attendance (2)

Core team members or their deputies are always present at our MDT Meetings

Leadership & Chairing (5)

Leadership could be improved in our Team

Teamworking & Culture (9)

I feel part of a true team—we are more than a group of individuals who come together for the weekly MDT meeting If training needs are identified for Multidisciplinary Team members, action is taken to address those needs

Personal development & Training (2) Infrastructure for meetings

Physical environment of meeting venue (1) Technology & equipment (1)

Organisation and administration for meetings

Patient-centred clinical decision making

Team governance

Scheduling of MDT meetings (2) Preparation prior to MDT meetings (2)

My Trust provides standard of equipment for interhospital compatibility, real-time viewing of imaging and pathology, and documentation of recommendations I am able to attend our MDT Meeting regularly, and this is reflected in my job plan

Organisation/administration during MDT meetings (4)

Locally agreed minimum data sets of information are prepared and cases are prioritised in a logical order at our MDT Meetings

Post-MDT meeting coordination of service (1)

Clinical information is presented to a high standard at our MDT Meetings

Who to discuss? (1) Patient-centred care (4)

Someone the patient has met always attends the MDT Meeting to discuss their case

Clinical decision-making process (4)

I would like my Team to look after me if I were a cancer patient

Organisational support (1)

Our Team collects information and reviews it in order to continually improve on equality issues and clinical outcomes, and benchmark against best practice

Data collection, analysis & audit of outcomes (1) Clinical governance (1)

We compare and audit our Team’s recommendations against the actual treatment received and any serious complications that may occur

From ‘‘The characteristics of an effective MDT’’ (NCAT, 2010)26

agreement scales (anchored at 1 = completely disagree, 5 = completely agree). A ‘‘don’t know’’ option was also included. Finally, two free-text questions were also included asking respondents to state at least three areas where they felt their team worked well and three areas they could improve upon. The publication rights to TEAM are owned by the NCAT. Sample items for each domain are provided in Table 1.

participate (one Trust provided seven MDTs, and another provided six).

Testing Acceptability, Feasibility, and Psychometric Properties

Procedure The survey was administered online via SurveyMonkey (http://www.surveymonkey.com/). All team members were sent a link to an online questionnaire via e-mail using the automated invitation system provided by SurveyMonkey and given approximately 3 weeks to respond. Non-responders received a maximum of two weekly e-mail reminders. The second reminder also included a Microsoft Word version of the survey in case non-response was due to technological problems.

Sample 23 MDTs from 4 NHS Trusts were recruited to test the questionnaire. A Trust is the provider of health care services and in most cases equates to a hospital site. Teams were recruited via their Trust cancer services manager, who in turn had approached NCAT or the research team to express an interest to be involved. Each Trust cancer services manager was asked to recruit five MDTs willing to

Content of Online Survey All team members completed background questions (professional group and whether they were the MDT-lead or chair for the team) and the TEAM questionnaire. In order to test the content validity of the questionnaire, team members from 10 of the MDTs (from two NHS Trusts) also completed a second questionnaire (further details follow).

4022

C. Taylor et al.

Psychometric Properties

Ethical Review

Content Validity Questionnaires were developed for each of the five subdomains prioritised for inclusion: leadership and chairing (29 items); teamworking and culture (11); patient-centred care (14); clinical decision-making process (24); organisation and administration for meetings (26).28,29 These were created by translating each recommendation into a statement rated for agreement using the same 5-point scale as TEAM. In order to minimise response burden team members (from the 10 MDTs participating in this validation exercise) received a questionnaire containing a maximum of three out of the five subdomains. Average ratings for each subdomain were calculated by summing the ratings given to individual items within the scale and dividing by the number of items. The correlation of average ratings between TEAM responses and the domain-specific questionnaires was then measured by calculating Pearson’s correlation coefficients.

The protocol for this work was reviewed by UK National Research Ethics Service (NRES) and confirmed to be classified as service development and not eligible for review.

Internal Consistency Internal consistency was examined for the subdomains within the 47-item questionnaire and for each of the domain-specific questionnaires by calculating Cronbach’s alphas. Typically acceptable alphas ought to reach 0.70 or higher; lower alpha values (i.e., between 0.60 and 0.70) can be accepted for short scales and for preliminary research purposes.30 Item Discrimination In order to assess for potential (statistical) overlap between items, the correlation between each individual item within the 47-item questionnaire was measured by calculating Pearson’s correlation coefficient. Acceptability and Feasibility The acceptability and feasibility of TEAM was assessed by conducting telephone interviews with a purposive sample of approximately four team members from each team after they had completed TEAM and discussed their team responses. Individual team members were sampled to represent different professional groups; those who completed/did not complete the online survey; and included MDT leads/chairs. Interviews were semi-structured following a topic-guide aimed at determining the acceptability of the length and content of the questionnaire and feasibility of online completion. They were digitally recorded, and interview data were coded directly from recordings by one of four researchers (all trained in qualitative analytic methods). Codes were applied as appropriate and illustrative quotations taken from each interview in relation to each relevant code. A subset of recordings (n = 8) were double-coded by a researcher with substantial experience of the project who confirmed the reliability of coding (88 % agreement).

RESULTS Participants In total the survey invitation was e-mailed to 637 team members across the 23 teams, including core and extended team members. A total of 329 team members (52 %) responded to the survey (Table 2). Forty-eight percent of team members did not even open the survey link. Of those who opened the survey link, 77 % answered the whole questionnaire and 90 % answered at least 40 items. In all teams there was good representation of the main professional groups amongst those who responded (medical, nursing and administrative members). The response rates were similar across the 4 Trusts (range 48–59 %). The range for individual teams was 26–83 %, and both the highest and the lowest response rates came from the same Trust (Trust D). There was a wide variation in response rates within and between tumour types (colorectal 35–63 %; gynae 60–83 %; head and neck 26–58 %; urology 45–52 %; lung 53–63 %). By professional group, the highest response rate was from clinical nurse specialists (59 %). Other types of nurses, managers and allied health professionals (AHPs) were most likely to not even open the survey link, and Managers and AHPs were the most likely to stop completing the survey after the background items had been completed.

Acceptability, feasibility and psychometric properties of the questionnaire Content Validity Responses to the 47-item questionnaire were strongly and significantly correlated with responses to the domain-specific questionnaires for all 5 domains that were tested (r [ 0.67, p \ 0.01) (Table 3). Internal Consistency The scales for two domains (Leadership and Teamwork and Culture) had acceptable alpha ratings (Cronbach alpha [ 0.70) with a further five domains achieving alpha ratings C0.60 (Table 4). Attendance, Scheduling of MDT Meetings, and Preparation for MDT meetings, each having only two items, had alpha ratings between 0.52 and 0.59. The domain-specific questionnaires all had very high internal consistency ([0.89).

TEAM to Self-assess MDTs TABLE 2 Response rate to online survey according to Trust and team

4023

Trust/Team

No. members contacted

Trust A

Did not open survey link

118

62 (53 %)

8 (7 %)

14

8 (57 %)

2 (14 %)

4 (29 %)

Colorectal

17

6 (35 %)

1 (6 %)

10 (59 %)

Gynae

20

12 (60 %)

2 (10 %)

6 (30 %)

Head and neck

24

14 (58 %)

2 (8 %)

8 (33 %)

Urology

15

7 (47 %)

1 (7 %)

7 (47 %)

Lung

16

10 (63 %)

0 (0 %)

6 (38 %)

Lymp/Haem

12

5 (42 %)

0 (0 %)

7 (58 %)

48 (40 %)

56 (48 %)

117

59 (50 %)

2 (2 %)

Colorectal

28

15 (54 %)

0 (0 %)

13 (46 %)

Head and neck Lung

31 15

11 (36 %) 8 (53 %)

1 (3 %) 0 (0 %)

19 (61 %) 7 (47 %)

Skin

10

8 (80 %)

0 (0 %)

2 (20 %)

Urology Trust C

33

17 (52 %)

1 (3 %)

15 (46 %)

179

86 (48 %)

4 (2 %)

89 (50 %)

Head and neck

52

24 (46 %)

0 (0 %)

28 (54 %)

HPB

36

19 (53 %)

1 (3 %)

16 (44 %)

Lung

29

16 (55 %)

0 (0 %)

13 (45 %)

Sarcoma

33

14 (42 %)

1 (3 %)

18 (55 %)

Urology

29

13 (45 %)

2 (7 %)

14 (48 %)

223

99 (44 %)

9 (4 %)

115 (52 %)

Gynae

29

24 (83 %)

0 (0 %)

5 (17 %)

Head and neck

47

12 (26 %)

2 (4 %)

33 (70 %)

Colorectal (a)

36

13 (36 %)

1 (3 %)

22 (61 %)

Trust D

A total of 253 (83 %) of 306 participants answered all 47 items; 296 (97 %) of 306 participants answered 40? items

Answered demographic items only

Breast

Trust B

a

Answered at least some of TEAMa

Upper GI

35

12 (34 %)

3 (9 %)

20 (57 %)

Urology Colorectal (b)

57 19

26 (46 %) 12 (63 %)

3 (5 %) 0 (0 %)

28 (49 %) 7 (37 %)

637

306 (48 %)

23 (4 %)

308 (48 %)

Total

Item Discrimination There was little correlation between responses to individual items in the 47-item questionnaire. Only 4 items had a moderate correlation of r = 0.60–0.65: ‘‘I feel part of a first class team,’’ ‘‘There is mutual respect, trust and healthy debate between our Multidisciplinary Team members,’’ ‘‘I would like my Team to look after me if I were a cancer patient,’’ and ‘‘I feel part of a true team— we are more than a group of individuals who come together for the weekly MDT meeting.’’ The majority of items had weak correlations (r \ 0.20), indicating that items are each measuring discrete aspects of teamworking (analysis available on request). Acceptability and Feasibility of the Questionnaire Completion of TEAM (based on SurveyMonkey time logs) took on average 17 minutes (standard deviation 11 minutes). In total, 74 telephone interviews were conducted with a purposively selected range of team members. All professional groups were well represented. This included

interviews with 15 team members that had not responded to the online survey. The reasons given by these team members for not completing the survey were explained by at least one of three themes: (1) being on annual leave (the testing phase coincided with two consecutive UK bank holiday weekends); (2) not having time to dedicate to it: ‘‘Just [not enough] time and workload, had been on holiday previously—it was not that I didn’t want to fill it in’’; and (3) having technical problems with the online system: ‘‘I couldn’t get into it—clicked on the link and it didn’t work and then forgot to go back to it.’’ One team member stated that the bank holidays had led to a backlog of work as part of the reason for him not participating and then also added that he ‘‘knew a lot of other people completed it and felt as though not everyone needed to complete it.’’ The majority of team members were positive about the questionnaire (51 of 74, 69 %, of team members interviewed). This included positive comments about the

.55**,a .62**,b

.49** .61**,b

Clinical decision-making

Teamworking and culture

Organisation and admin during mtgs .61**,b

.35* .19 .33* .38** .40** .34**

Leadership only

Chairing only

Patient centred care

Clinical decision-making Teamworking and culture

Organisation and admin during mtgs

.67**,b

.64**,b

.68**,b

.67**

.42** .23 .34**

.68**,b

.34*

.40**

.35*

.41**

,b

.40** .36*

Clinical governance

.46** .60**,b

.46**

.38**

.38**

.39**

Attendance

.63**,b

.63**,b

,b

.45**

.40** .68**

.46**

.34* .40**

.26

.18

.30*

.22

Environment

.61**,b

.79**,c

.57**,a

.40**

.60**,a

,b

.64**,b

.59**,a

.05 .41**

.07

-.11

-.03

-.10

Technology

.77**,c

.70**,c

.44**

.24

.54**,a

.47**

.58**,a

.38**

.22 .36**

.33*

.26

.13

.21

Post-mtg actions

.58**

.34*

.43** .35*

.25

.46**

.35*

.49**

Who to discuss

.68**,b

.57**,a

.34*

.08

.50**,a

,a

.59**,a

.05

-.02 .16

-.17

.03

.08

.08

Organisational support

.06

.04

.35*

.22

.14

.29

.23

Correlation [ 0.60

Correlation [ 0.70

b

c

** p \ 0.01

* p \ 0.05 (2-tailed)

Correlation [ 0.50

a

n = 39–65. Analyses include members who completed all items within the scales correlated. Each domain-specific questionnaire was sent to members from five MDTs

.24

Leadership & chairing

Scheduling

Team

.59**,a

.32*

Patient centred care

Domain-specific questionnaire

.67** .79**,c

.36*

.51**,a

.53**

.69** .81**,c

.49**

Chairing only

,a

,b

Leadership only

.52**,a

.80**,c

.54**,a

.38**

.40* .29*

.46**

.50**,a

.54**,a

.49**

Data collection & audit

.58**,a

.54**,a

.46**

.31*

.53**,a

.40**

.53**

General Leadership & Clinical Patient Teamworking & Organisation & Personal development Membership Preparation feelings chairing decision-making centred care culture admin during mtgs and training prior to mtgs

Team

Leadership & chairing

Domain-specific questionnaire

TABLE 3 Content validity: correlation between the ratings for team performance between TEAM and domain-specific questionnaires

4024 C. Taylor et al.

TEAM to Self-assess MDTs

4025

TABLE 4 Internal consistency of scales within TEAM and the domain-specific questionnaires Subdomain of teamworking (scale)

TEAM items

Domain-specific items

No. of items

Item mean (SD)

Alpha

Attendance

2

4.04 (0.32)

0.59

Leadership & chairing

5

3.77 (1.08)

0.76a

Teamworking & culture

9

3.99 (1.01)

0.81a b

No. of items

Item mean (SD)

Alpha

NA

NA

NA

29

3.80 (0.81)

0.97a

11

3.87 (1.00)

0.89a NA

Personal development & training

2

3.45 (0.17)

0.60

NA

NA

Scheduling of MDT meetings

2

3.92 (0.45)

0.59

NA

NA

NA

Preparation prior to MDT meetings

2

3.75 (0.19)

0.52

NA

NA

NA

Organisation admin during MDT mtgs Patient-centred care

4 4

3.69 (0.67) 3.77 (1.13)

0.62b 0.62b

26 14

3.44 (1.88) 3.98 (0.62)

0.94a 0.95a

Clinical decision-making process

4

4.17 (0.33)

0.65b

24

3.95 (1.15)

0.95a

3.74 (0.60)

b

NA

NA

NA

General feelings

4

0.65

Based on data from 22 teams (n = 225–291 for TEAM items, n = 16–57 for domain-specific items). Single item domains removed (Membership; Physical environment of meeting venue; Technology & equipment; Postmeeting coordination of services; Who to discuss; Organisational support; Data collection, analysis & audit; Clinical governance) a

Alpha C 0.70

b

Alpha C 0.60

TABLE 5 Acceptability and feasibility of TEAM Theme (no. of team members supporting theme)

Illustrative quotations

Content: good content and coverage ‘‘It’s thorough and an easy survey to complete. You feel motivated to fill in, as you want to know what (35) other people in the team think’’ Too long, did not like aspects of content (6)

‘‘The questions were clear and it wasn’t too onerous’’ ‘‘It didn’t take me that long at all, it was fine’’ ‘‘The survey provided an opportunity to reflect on the running of our MDT and the actual functionality of MDT process, how it works, and those things not covered by peer review’’ ‘‘I thought some of the questions were a bit unclear’’ ‘‘It took longer to fill in than I thought it would’’

Method: 360-degree process is useful/reassuring (23)

‘‘The 360 degree feedback, overall it was reassuring that there was a lot of consensus that things are going well, and very reassuring as the lead that it’s not just my opinion’’ ‘‘[360 degree] gave overall picture about what people think of it’’

Importance of team engagement (12)

‘‘It gives people an opportunity to judge how they feel the MDT process works in an anonymous way, and it allows evaluation of the strengths and weaknesses of the team’’ ‘‘Ownership in the process, feeling it has relevance, and a belief that the process will make a difference is important’’ ‘‘It’s important to engage people to participate, there must be someone to drive it, if there is a lack of realisation within the team of importance of process then people may treat as a tick box exercise. The lead needs to be able to push to ensure its completed by the team’’

Importance of good IT (19)

‘‘You are never going to get a 100 % response rate . . . but you need good IT and need it to work on the iPad’’ ‘‘The survey was slow changing screens, the internet speed varies in the Trusts so it’s probably down to that, but it was very slow’’

content, length and the 360-degree method enabling everyone to contribute (Table 5). A common theme was the importance of team engagement to ensure participation in the process, and also ensuring that the IT works properly (this was particularly mentioned by some team members from one Trust who had technical difficulties due to poor Trust IT systems).

DISCUSSION We have developed a team performance assessment questionnaire (TEAM) based upon input from over 300 team members from over 60 MDTs. The questionnaire is underpinned by recommendations contained within ‘‘The Characteristics of an Effective MDT,’’ which in turn is

4026

based on clinical consensus from over 2000 MDT members nationwide.26 TEAM has been tested with 23 MDTs from four hospital Trusts and has been found to be acceptable in content and length, and to have reasonable psychometric properties including content validity, internal consistency and item discrimination. It has been designed and tested as part of a broader team assessment process within which all teams were able to identify areas for improvement of teamworking and some changes were implemented immediately.31 To-date TEAM has mostly been tested by local cancer teams treating common tumours. Adaptation of TEAM may be necessary to ensure coverage of the characteristics of effectiveness in teams with different team structures or processes (e.g., for specialist teams working across multiple sites and using videoconferencing for their weekly meetings). It may also be necessary to review the items for scales that did not reach acceptable levels of internal consistency. Furthermore, whilst we have designed TEAM on the basis of clinical consensus from over 2000 MDT members regarding the features of an effective MDT, these characteristics of effectiveness have yet to be validated against outcomes—including clinical processes, patient outcomes and patient experience. Given the complexity of cancer as a disease but also its care pathway, some of these measures may be more or less sensitive to correlations with TEAM. Examining the relationship between TEAM and a range of clinical processes and outcomes, including patient experience may help validate the clinical consensus regarding key ingredients of effectiveness.18 In addition, although we report generally positive views regarding the content and length of TEAM, on average only just over half of team members completed it. Team members were only given a short time (maximum of 3 weeks) to respond and this coincided with UK national holidays. The majority of those who opened the link completed the survey (83 % answered whole survey and 97 % answered at least 40 of 47 items). Compared to postal surveys, online surveys may be more easily ignored or deleted, less attention-grabbing; and feel less anonymous.32 Interviews with a number of non-responders established that their non-participation was mostly the result of lack of time and/or workload pressures at that time rather than any of these reasons, and they confirmed the representativeness of their team feedback reports indicating that a full response rate may not have changed the eventual team outcomes. The ease of completion of e-surveys may therefore outweigh any potential downside. Although not cited as a particular reason for non-response, there were some technological issues that may have impeded participation. Development of a format that is compatible with NHS IT resources and/or enables team members to participate using different forms of technology, such as

C. Taylor et al.

smartphones, would be necessary for TEAM to have wider utility and acceptability. Team members emphasised the importance of both team and organisational engagement for the process to lead to improved teamworking. Integration of TEAM into other MDT assessment processes may enhance its potential to facilitate improvements by providing leverage and incentives for participation and improvement. In the UK it is mandatory for all cancer teams to hold an annual meeting to discuss their teamworking; TEAM could provide focus and structure to these meetings, and could also be used as supplementary evidence for the mandatory annual Peer Review assessments, and also for team or individual job appraisals. The NCAT, responsible for supporting implementation of cancer policy in England, funded the development of TEAM through their MDT Development Programme. Upon completion of the validation and testing of TEAM its widespread use will be promoted by NCAT. This is likely to include dissemination through the National Development Programme (a biannual meeting attended by senior cancer clinicians and managers across England) and also through other streams of national work aimed at improving patient care such as the National Patient Experience programme and CONNECTED (national communication skills training programme). Following other health care specialties, like surgery and anaesthesia, cancer care is increasingly focussed on ensuring optimal teamwork and decision-making.7,17 To ensure validity in such team assessment exercises, the tools used to assess how a cancer team functions and how it can improve ought to be scientifically robust and the process clinically meaningful. Our preliminary work suggests that TEAM meets these criteria and can make an important contribution to improving teamwork in multidisciplinary cancer teams, and by doing so will ultimately improve patient care. ACKNOWLEDGMENT We thank the team members who participated in the development and testing of TEAM (as part of testing MDT-FIT) and the Trust personnel who facilitated their involvement; other affiliate members of Green Cross Medical Ltd who have supported this work; and the NCAT MDT Development steering group and subcommittee members for their input and comments. Supported in part by the NCAT. Sevdalis, Brown, and Lamb are also affiliated with the Imperial Centre for Patient Safety and Service Quality, which is funded by the National Institute for Health Research (NIHR).

REFERENCES 1. Saini KS, Taylor C, Ramirez AJ, et al. Role of the multidisciplinary team in breast cancer management: results from a large international survey involving 39 countries. Ann Oncol. 2011; 23:853–9. 2. Kane B, Luz S, O’Briain DS, McDermott R. Multidisciplinary team meetings and their impact on workflow in radiology and pathology departments. BMC Med. 2007;5:15.

TEAM to Self-assess MDTs 3. Taylor C, Munro AJ, Glynne-Jones R, et al. 2010 Multidisciplinary team working in cancer: where are we now? BMJ. 2007;340 c951. 4. Department of Health. Manual for cancer services. London: Department of Health; 2004. 5. National Cancer Action Team. National Cancer Peer Review Programme. Report, 2009/2010: an overview of the findings from the 2009/2010 National Cancer Peer Review of cancer services in England. London: National Cancer Action Team; 2010. 6. Lamb B, Brown K, Nagpal K, Vincent C, Green JSA, Sevdalis N. Quality of care management decisions by multidisciplinary cancer teams: a systematic review Ann Surg Oncol. 2011;18:2116–25. 7. Lamb BW, Green JSA, Sevdalis N. Decision-making in surgical oncology. Surg Oncol. 2011;20:163–8. 8. Davies AR, Deans DA, Penman I, et al. The multidisciplinary team meeting improves staging accuracy and treatment selection for gastro-esophageal cancer. Dis Esophagus. 2006;19:496–503. 9. Coory M, Gkolia P, Yang I, Bowman R, Fong K. Systematic review of multidisciplinary teams in the management of lung cancer. Lung Cancer. 2008;60:14–21. 10. Burton S, Brown G, Daniels IR, Norman AR, Mason B, Cunningham D. MRI directed multidisciplinary team preoperative treatment strategy: the way to eliminate positive circumferential margins? Br J Cancer. 2006;94:351–7. 11. Haward R, Amir Z, Borrill C, et al. Breast cancer teams: the impact of constitution, new cancer workload, and methods of operation on their effectiveness. Br J Cancer. 2003;89:15–22. 12. Lanceley AS, Savage J, Menon U, Jacobs I. Influences on multidisciplinary team decision-making. Int J Gynecol Cancer. 2008; 18:215–22. 13. Kidger J, Murdoch J, Donovan JL, Blazeby JM. Clinical decision-making in a multidisciplinary gynaecological cancer team: a qualitative study. BJOG. 2009;116:511–7. 14. Leo F, Venissac N, Poudenx M, Otto J, Mourox J; Groupe d’Oncologie Thoracique Azureen (GOThA). Multidisciplinary management of lung cancer: how to test its efficacy? J Thorac Oncol. 2007;2:69–72. 15. Stalfors J, Lundberg C, Westin T. Quality assessment of a multidisciplinary tumour meeting for patients with head and neck cancer. Acta Otolaryngol (Stockh). 2007;127:82–7. 16. Blazeby JM, Wilson L, Metcalfe C, Nicklin J, English R, Donovan JL. Analysis of clinical decision-making in multidisciplinary cancer teams. Ann Oncol. 2006;17:457–60. 17. Lamb BW, Sevdalis N, Arora S, Pinto A, Vincent C, Green JS. Teamwork and team decision-making at multidisciplinary cancer conferences: barriers, facilitators, and opportunities for improvement. World J Surg. 2011;35:1970–6. 18. Vincent C, Moorthy K, Sarker SK, Chang A, Darzi AW. Systems approaches to surgical quality and safety: from concept to measurement. Ann Surg. 2004;239:475–82.

4027 19. Hull L, Arora S, Aggarwal R, Darzi A, Vincent C, Sevdalis N. The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg. 2012;214:214–30. 20. Hull L, Arora S, Kassab E, Kneebone RL, Sevdalis N. Observational Teamwork Assessment for Surgery (OTAS): content validation and tool refinement. J Am Coll Surg. 2011;212:234– 43. 21. Undre S, Koutantji M, Sevdalis N, et al. Multi-disciplinary crisis simulations: the way forward for training surgical teams. World J Surg. 2007;31:1843–53. 22. McCulloch P, Rathbone J, Catchpole K. Interventions to improve teamwork and communications among healthcare staff. Br J Surg. 2011;98:469–79. 23. Lamb BW, Sevdalis N, Mostafid H, Vincent C, Green JSA. Quality improvement in multidisciplinary cancer teams: an investigation of teamwork and clinical decision-making and cross-validation of assessments. Ann Surg Oncol. 2011;18:3535– 43. 24. Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2003;(3):CD000259. 25. Taylor C, Ramirez AJ. Multidisciplinary team members’ views about MDT working: results from a survey commissioned by the National Cancer Action Team. London: National Cancer Action Team; 2009. http://www.ncin.org.uk/mdt. 26. National Cancer Action Team. The characteristics of an effective multidisciplinary team. London: National Cancer Action Team; 2010. http://www.ncin.org.uk/mdt. 27. Fleissig A, Jenkins V, Catt S, Fallowfield L. Multidiscpilinary teams in cancer care: are they effective in the UK? Lancet Oncol. 2006;7:935–43. 28. Abell N, Springer DW, Kamata A. Developing and validating rapid assessment instruments. Oxford: Oxford University Press; 2009. 29. Spector PE. Summated rating scale construction. London: Sage Publications; 1992. 30. Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78:98–104. 31. Taylor C, Brown KB, Sevdalis N, Green JSA. Developing and testing a novel, evidence-based and user-tested toolkit for assessing and improving teamworking in multidisciplinary cancer teams (abstract). Paper presented at: European Cancer Congress Stockholm, September 25, 2011. http://new.ecco-org.eu/ecco_ content/StockholmAbstractbook/index.html. 32. Yun GW, Trumbo CW. Comparative response to a survey executed by post, e-mail, and Web form. J Comp Mediat Commun. 2000;6(1).