The Evaluation of the Dissemination, Implementation ...

The Evaluation of the Dissemination, Implementation and Impact of NICE Guidance

Final Report

PROFESSOR NICKY CULLUM DIANE DAWSON DR ANNETTE LANKSHEAR KARIN LOWSON JAMES MAHON DR PAULINE RAYNOR PROFESSOR TREVOR SHELDON PROFESSOR IAN WATT DR PETER WEST DIANNE WRIGHT DR JOHN WRIGHT

APRIL 2004

1.

Introduction

This study, to evaluate the dissemination, implementation and impact of NICE guidance was commissioned by the National Co-ordinating Centre for Research Methodology (NCCRM) in July 2001on behalf of NICE. The research was undertaken by a multi-disciplinary team from the Department of Health Sciences, the York Health Economics Consortium and the Centre for Health Economics at the University of York.

1.1

Policy Background

In April 1999 the National Institute for Clinical Excellence (NICE) was established with the aim of identifying best practice and advising health professionals on which treatments work best for patients and are cost effective. It issues authoritative national guidance for the NHS with the aim of: • • •

securing consistent access to and quality of service promoting economic use of resources (both in making its decisions and in funding its resource consequences) improving the quality of decision making

NICE issues guidance on 30 - 50 technologies a year. The policy to establish the National Institute for Clinical Excellence was the product of three main forces: i) ii)

iii)

the pressure on costs posed by new and potentially expensive (in price and volume terms) health technologies, particularly pharmaceuticals (Stevens et al, 1999); inconsistent rationing decisions in the past. There was considerable disquiet about decisions as to whether and how new drugs would be funded. Where this had been delegated to health authorities (e.g. beta interferon), it had resulted in considerable geographical variation in access - ‘postcode rationing’ - conflicting with the strong principle of equity within the NHS and resulting in embarrassing legal cases (Dyer, 1997). Where decisions had been made centrally (e.g. Viagra) there was considerable concern about the rather ad hoc and opaque fashion in which decisions were made. the need for the NHS to focus more consistently on improving the quality of care. The government has placed the quality of care delivered by the NHS at the heart of its agenda for the service. This policy has been reinforced and accelerated by a series of high profile cases of bad medical practice, most notably the paediatric heart surgery scandal in Bristol (Smith, 1998). The quality agenda was laid out in A First Class Service: Quality in the new NHS and extended by the NHS Plan (NHS Executive, 1998).

2

In addition, there was some pressure to make better use of the evidence base that was being generated nationally through the NHS R&D programme and internationally in health technology assessment. Until the establishment of the quality strategy, guidance on the effectiveness and occasionally, on the cost-effectiveness of treatments tended to come in three forms: • clinical guidelines, often produced for the NHS under the auspices of the Medical Royal Colleges; • Department of Health circulars, often reflecting the advice of bodies such as SMAC or CSAG and sometimes incorporating the analysis of DEC reports; • the Effective Health Care bulletins, produced by the University of York and other ‘evidence-based health care’ initiatives. While the aim of NHS guidance has been to reduce variation in clinical practice by focusing attention on good practice, significant variations remained. One longstanding problem has been that Department of Health guidance setting out relevant standards of care is not binding on clinicians. Within the existing legal framework, local adoption of uniform national standards depends on the effectiveness of the new arrangements in changing behaviour. Thus one key to the success of current policy is whether the new arrangements (such as those of clinical governance) will increase the likelihood that national guidance will be implemented. This evaluation of the dissemination and implementation of NICE guidance will contribute to this discussion. A multitude of factors is likely to contribute to observed variation in conformity with national guidance. These factors can be grouped under five broad headings: 1) Ignorance or inertia on the part of some key stakeholders (clinicians or administrators within Health Authorities/PCGs/Trusts), structural obstacles to coordinated action between individuals, organisations or sectors and the lack of clinical skills in using the health technology. 2) Disagreement by clinicians and others with the results of and/or methodology used by NICE in arriving at its recommendations. 3) Lack of implementation skills and experience in introducing behavioural and organisational change. 4) Differences in local opportunity costs of implementing the guidance. 5) Patient preferences and exercise of choice for options that do not conform to the guidance. Effective methods of dissemination and implementation along with clinical governance and clinical audit might be expected to secure compliance with guidance where variation is primarily related to the factors under (1), but may not be effective if failure to comply is related to the other factors. In fact it may be neither efficient nor equitable to seek to eliminate variations from national standards generated by (4) if this displaces activity of higher local priority which is more cost-effective in general or particularly given local needs or conditions, (Cookson et al, 2001).

3

1.2

Aims and Plan of Investigation

The aim of the project was to assess the degree to which NICE guidance had been implemented by health care organisations and to explore explanations for variations in implementation within and across organisations. It should be stressed that NICE is not responsible for the implementation of its guidance; several other agencies and parts of the NHS as well as health care professionals have important roles to play. Thus this evaluation is not an evaluation of NICE itself but of the implementation of its guidance. Nor did we consider the methods used by NICE to derive its guidance. The project consisted of the three phases outlined below. More details of the methods are provided at relevant sections of the report. Phase I An analysis of routine national or regional data and surveys of local data (high level data) to assess the degree to which there is evidence of implementation and the variation in implementation of examples of NICE guidance (tracer guidance). These are summarised in Table 1. •

A questionnaire was sent to all hospital pharmacies to gather collect data on the prescription of drugs by hospitals covered by tracer NICE guidance This will be used to assess if there are trends in use compatible with the NICE guidance.

§

PACT data was used to assess trends in prescribing in primary care for drugs covered by tracer guidance where the drug is used only for a narrow range of conditions covered by the NICE guidance e.g. zanamivir and orlistat.

§

Hospital Episode Statistics (HES) data were used to assess the trends in use of and variations in use of invasive procedures.

§

A range of other sources of national, regional and local data were accessed or directly collected: § § § §

Dental Practice Board and the Scottish Practitioner Services data for information on extractions of wisdom teeth in the community Trent Arthroplasty Audit Group and Welsh Arthroplasty Audit Group Database (TWAAG) for information on prostheses used in hip replacement British Pacing and Electro-physiology Group ICD database for information on Implantable Cardioverter Defibrillators for arrhythmias. A new survey of hearing aid provision sent to 228 audiology departments in England and Wales.

Where there were long enough continuous series of data on use of technologies before and after the issuing of guidance (e.g. 6 months before and after), interrupted time series analysis was used to assess if there has been a change or interruption in the pattern of practice after the issuing of NICE guidance (McDowall et al, 1980) and so infer whether the intervention had an impact.

4

Phase II An analysis of clinical practice based on an audit of patient records. The analysis of most routine data sources from phase I will be limited by the fact that most NICE guidance provides criteria for appropriate use rather than just simply recommending or not recommending the technology. Thus trends showing increased or decreased use after the release of NICE guidance will usually not permit reliable inferences about the correct implementation of the guidance. To assess the appropriateness of use in the context of NICE guidance requires the scrutiny of patient records. Thus in this part of the evaluation we selected a sample of hospitals and PCTs where we ‘drilled down’ further in order to assess the appropriateness of the implementation of the guidance, using samples of patient records. Local audit staff were recruited to extract data from patient records using a data proforma. This allowed us to assess the use of the relevant health technologies and also whether the indications for use described in the NICE guidelines were present. Phase II used methods developed for appropriateness studies (Park et al, 1989; Shekelle et al, 1998; Filardo et al, 2001). The study aimed to examine the implementation of a subset of guidance across a random sample of about 20 hospitals and 20 large GP practices. In each selected organisation we aimed to review 50 case notes (for each of the 6 tracer guidance see Table 1) of patients with a clinical condition or who had a procedure relevant to the guidance selected (e.g. cases of ovarian cancer, or those having had a total hip replacement). The case note review was piloted in a large acute trust and a GP practice for each of the chosen guidance technologies. Reliability was enhanced by quality checks on the samples of audits. The proportion of cases conforming to the NICE indications was calculated for each unit and guidance and then the overall average estimated as well as indications of the degree of variation. Phase III: Quantitative data collection was followed up with qualitative research in the nonprimary care trusts which participated in the audit. A survey of and interviews with a sample of chief executives, medical directors, lead clinicians and clinical governance leads was carried out in order to assess the process of dissemination of NICE guidance, and the rationale for adopting or modifying sets of guidance. Specifically: •

A brief questionnaire survey of the trusts which participated in the phase II audit of patient notes. This was directed at chief executives, clinical governance leads, as well the relevant clinical speciality leads to assess whether guidance has been seen, what was done with the guidance documents, whether there are mechanisms in place to deal with NICE guidance when it is issued and whether and how local implementation of NICE guidance is audited. Some of the questions were generic and some related to a small sample of NICE guidance relevant to that sector.

•

Semi-structured interviews with key decision makers (clinical and non-clinical) in 5 acute hospital trusts selected mainly for the different patterns of adherence to guidance found in phase II. These interviews were tape recorded, transcribed, and subjected to thematic content analysis (Morse, 1991).

5

Table 1.

Data sources by guidance

Guidance Removal of wisdom teeth

Prostheses for hip replacement

High level data • Hospital episode statistics (HES) • Dental practice board • HES • Joint replacement register

Taxanes for treatment of breast cancer

•

Hospital pharmacy

Taxanes for treatment of ovarian cancer

•

Hospital pharmacy

Hearing aids

•

Guidance specific survey

ICDs for arrhythmias Zanamivir for influenza

• •

Laparoscopic surgery for the treatment of colorectal cancer Laparoscopic surgery for the treatment of inguinal hernias

•

Guidance specific database Prescribing (PACT) data from Prescription Pricing Authority Annual flu levels from PHLS bulletins HES

•

HES

Donepezil, rivastigmine and galantamine for Alzheimer’s disease Orlistat for obesity

• •

PACT data Hospital pharmacy

• • •

PACT data Hospital pharmacy Hospital pharmacy

•

Taxanes for non-small cell lung cancer

1.3

Trust and GP level data • Patient audits • Survey • Interviews • Patient audits • Survey • Interviews • Patient audits • Survey • Interviews • Patient audits • Survey • Interviews • Patient audits • Survey • Interviews • None collected • None collected

•

None collected

• • • • •

Patient audits Survey Interviews Patient audits Survey

•

Patient audits

•

None collected

Structure of this Report

The team has already submitted to the NCCRM three interim reports, which contain detailed findings associated with each of the first two phases of the study. This is the final report which summarises the whole project and includes the results from Phase III. This draft final report has three main sections. The first section describes the methods and the second presents the results for each set of tracer guidance. The third section draws together the results and examines ways in which Trusts manage and respond to NICE guidance. We have included, as Appendices, copies of all data collection tools including audit and survey forms and letters sent to Trusts at various stages of the study.

6

2.

METHODS

2.1

Choice Of Guidance

The focus of the study was the assessment of the response of the NHS to 12 pieces of guidance (tracer guidance) issued by NICE. We selected specific pieces of tracer guidance that would reflect a range of drugs, devices and procedures; different care settings and cost consequences. Guidance was only selected for study if sufficient time had elapsed for some implementation to be measurable. Our first step was to summarise and analyse the guidance to better understand the recommendations of the guidance and its implications for practice. This was needed in order to be really clear what would represent compliant practice (not always clear from the guidance). Table 1.1 gives an overview of the 12 pieces of tracer guidance selected, along with how each was conceptualised as a research (or audit) question.

7

T ABLE 1.1:

OVERVIEW OF THE

12 SETS OF GUIDANCE SEL ECTED AS “TRACER ” GUIDANCE FOR THE PURPOSE OF THIS PROJECT

Guidance 1

Removal of wisdom teeth

Health technology Procedure

Healthcare sector

Summary of NICE guidance

Research questions

Hospitals and general dental practitioners

•

The routine practice of prophylactic removal of pathology-free impacted third molars should be discontinued in the NHS The surgical removal of impacted third molars should be limited to patients with evidence of pathology Surgeons should use prostheses for total hip replacement which either have a demonstrable replacement rate of 10% or less at ten years, or a minimum of three years provided that their performance is consistent with the ten year benchmark Paclitaxel and docetaxel should be used for advanced cancer when previous chemotherapy has failed

•

Has there been a reduction in the number of wisdom teeth removed in situations where there are no apparent complications

•

Are approved prostheses used in replacement operations Are the numbers of approved prostheses rising and of nonapproved falling

•

Has the usage of paclitaxel and docetaxel increased to NICE recommended levels Has the usage of paclitaxel increased to NICE recommended levels Has there been a change in the availability of analogue hearing aids in NHS audiology centres Has the NHS Purchasing and Supply Agency reviewed the NHS range of analogue hearing aids Has there been an increase in the use of ICDs to NICE recommended levels

Published in March 2000 • 2

Prostheses for hip replacement

Hospitals and NHS Purchasing and Supply Agency

•

Drugs

Hospitals

•

Procedure

Published in April 2000

3


•

6

Published in May 2000 Taxanes for ovarian cancer

Drugs

Hospitals

•

Paclitaxel should be used after surgery

•

8

Published in June 2000 Hearing aids

Devices

Audiology centres and NHS Purchasing and Supply Agency

•

The full range of analogue hearing aids in the current NHS range should be available at all NHS audiology centres, including binaural fitting, and reflecting patient choice The NHS Purchasing and Supply Agency should review the existing NHS range of analogue aids

•

The use of ICDs for patients with specific pathologies should be routinely considered

•

Published in July 2000

• 11

Implantable cardioverter defibrillators (ICDs) for arrhythmias

Devices

Hospitals

•

Published in March 2000

8

•

Guidance 15

Zanamivir for influenza

Health technology Drugs

Healthcare sector

Summary of NICE guidance

Research questions

Primary care

•

Zanamivir should only be prescribed to at-risk adults, who have presented within 36 hours of the onset of influenza like illness (ILI), when influenza is circulating in the community

•

Procedure

Hospitals

•

Open rather than laparoscopic resection should be the preferred procedure for the treatment of colorectal cancer Laparoscopic surgery should only be undertaken for colorectal cancer as part of s randomised controlled clinical trial Open (mes h) surgery should be the preferred method of repair for primary inguinal hernia Laparoscopic surgery should be considered for repair of recurrent and bilateral inguinal hernia

•

Published in November 2000

17

Laparoscopic surgery for the treatment of colorectal cancer

•

Published in December 2000

18

Laparoscopic surgery for the treatment of inguinal hernia

Procedure

Hospitals

•

Published in January 2000

19

22

Donepezil, rivastigmine and galantamine for Alzheimer’s disease Published in January 2001 Orlistat for obesity

•

Chemotherapy for non small cell lung cancer Published in June 2001

Has there been an increase in the number of laparoscopic repairs for patients with recurrent and bilateral inguinal hernia, and a reduction for those with primary inguinal hernia Has the usage of the three drugs increased

Drugs

Hospitals and primary care

•

The three drugs should be made available to people with mild and moderate Alzheimer’s diseases, with MMSE scores above 12 points as assessed in specialised clinics

•

Drugs

Hospitals and primary care

•

•

Has the usage of orlistat increased

Drugs

Hospitals

•

Orlistat should only be made available to people with sustained weight loss prior to prescription and who have a BMI of 30kg/m2 or more with no co-morbidities or 28kg/m2 or more with comorbidities Gemcitabine, paclitaxel, and vinorelbine should each be considered as part of initial (first-line) chemotherapy. Docetaxel should be used for locally advanced cancer but only where previous chemotherapy has failed

•

Has the usage of the four drugs increased

Published in March 2001

26

•

Has there been in an increase in the prescribing levels of zanamivir concomitant with increases of the presence of ILI in the community Is laparoscopic surgery being undertaken on patients with colorectal cancer outside clinical trials

9

2.2

Phase 1

Collection and analysis of national & local data

2.2.1 Overview Table 2.2 summarises the high level data that we used to explore trends in implementation of each set of tracer guidance. The detailed description of data sources and the problems with their use and interpretation is given in Appendix 1. Copies of the survey forms used to collect data from pharmacies and audiology departments (for drugs and devices for which national ‘high level’ data were not available) are also in the Appendix.

Table 2.2

Summary of ‘high level’ data used by guidance

GUIDANCE Removal of wisdom teeth Prostheses for hip replacement

Description of data Number of procedures for removal of wisdom teeth undertaken in hospital departments and in general dental practices Number of hip replacements undertaken Types and benchmarking data applied to those prostheses used


Usage of taxanes for treatment of breast cancer in hospitals

Taxanes for treatment of ovarian cancer

Usage of taxanes for treatment of ovarian cancer in hospitals

Hearing aids

Availability of hearing aids in hospital audiology services Number of ICDs implanted by hospital

ICDs for arrhythmias

Zanamivir for influenza Laparoscopic surgery for colorectal cancer Laparoscopic repair of inguinal hernia Donepezil, rivastigmine and galantamine for Alzheimer’s disease

Level of prescribing of zanamivir in general practice compared to levels of flu in community Number of procedures for treatment of colorectal cancer using laparoscopic surgery Number of procedures for treatment of inguinal hernia using laparoscopic surgery Level of prescribing of donepezil in general practice and hospitals

Orlistat for obesity

Level of prescribing of orlistat in gen eral practice and hospitals

Taxanes for nonsmall cell lung cancer

Usage of taxanes for treatment of non-small cell lung cancer in hospitals

10

Source of data National hospital episode statistics (HES) and General Dental Practice Board for period 19922001 (three quarters) analysed per annum HES and Trent Arthroplasty Audit Group and Welsh Arthroplasty Audit Group (TWAAG) data base HES data for period 1992 -2001, TWAAG for 1998 –2002 (five months), both analysed monthly Data from a sample of hospital pharmacy information systems for 1997 – February 2002, analysed monthly Data from a sample of hospital pharmacy information systems for 1997 – February 2002, analysed monthly Questionnaire sent to all hospital audiology departments in November 2002, British Pacing and Electro-physiology Group ICD data base covering period of 1995/96 to 2001/02 analysed quarterly Prescribing data from Prescription Pricing Authority, annual for 2001 and 2002 and, annual flu levels from PHLS bulletins to May 2002 HES data for period 1992 -2001 (three quarters) analysed monthly HES data for period 1992 -2001 (three quarters) analysed monthly Prescribing data from Prescription Pricing Authority, for period February 1999 to February 2002 and hospital prescribing data from a sample of hospital pharmacy information systems for 1997 – February 2002, both data sets analysed monthly Prescribing data from Prescription Pricing Authority for period February 1999 to February 2002 and hospital prescribing data from a sample of hospital pharmacy information systems for 1997 – February 2002, both data sets analysed monthly Data from a sample of hospital pharmacy information systems for 1997 – February 2002, analysed monthly

2.2.2 Statistical analysis Where sufficient data were available, time series modelling was used to detect changes in the use of a health technology associated with the publication of NICE Guidance. The modelling was designed to detect whether in the time following the publication of Guidance the factors generating a data series in some way changed. Such a change is known statistically as a “structural break” (Box & Jenkins, 1984). Structural breaks can be detected statistically using several different techniques. An inherent problem in their identification is the ambiguity surrounding the point when it is known something could have resulted in a structural break and the actual point that that would become apparent in the data series. In the case of NICE Guidance, medical practitioners could have been aware of the NICE recommendations some months before publication and so modified their behaviour some time prior to publication. Alternatively, it may take practitioners some time to modify their behaviour due to clinical inertia in adopting new ideas. As such, if NICE Guidance did result in a structural break this could be evidenced in the data series at some indeterminable point before or after the publication date. Statistical theory offers the choice of either identifying the specific points in time of potential structural breaks in a data series or of testing for a structural break at a pre-specified time. Whilst testing for a structural break at a specific time can prove difficult due to the reasons outlined above, the results produced are more readily interpretable then trying to identify the exact time of potentially multiple structural breaks. For example, if a structural break is identified some four months after the production of guidance, the conclusion that this break is attributable to the guidance will be subjective, this defeats the purpose of using statistical analysis to provide objective evidence. In light of the above, the modelling process chosen was ARIMA (autoregressive integrated moving average) modelling using dummy variables to examine the impact of publication on the growth rate of use of a technology and the average rate of use of a technology once any growth rate had been removed (Box & Jenkins, 1976). A detailed explanation of the modelling technique used is provided in appendix 1

2.3

Phase 2

Collection and analysis of patient audit data

2.3.1 Background We assessed the degree of implementation of the tracer NICE sets of guidance, by examining samples of case notes in trusts and general practices selected by random sample. This enabled us to look at the appropriateness of the use of these technologies rather merely changes in use, as measured by the high-level data study. Although twelve sets of guidance had been used for the high level analysis, only a sub-set of these was appropriate for, or amenable to, audit at the patient level.

11

We designed the sampling frame to reflect the guidance and the date issued. We were not only interested in whether guidance appeared to be followed by clinicians and managers post publication, but also whether compliance with guidance improved over time therefore unless otherwise stated, we sampled notes from two time periods, viz. the first 6 months and the second 6 months after the publication of the guidance. 20 acute, 18 mental health and 21 primary care Trusts participated in the audit; each trust was asked to select 50 sets of patient case notes according to a given sampling instruction. The audit forms for the acute trusts were distributed in November 2002, and those for PCTs and mental health organisations in January 2003. Not all Trusts were able to provide a full set of notes. Copies of all guidance specific audit forms are given in Appendix 2 Table 2.3 describes the sets of guidance audited, and in which types of trusts.

Table 2.3

NICE guidance audited

NICE guidance no.

Subject

Acute trusts

1


Y

2

Selection of prostheses for primary total hip replacement

Y

3

Use of taxanes for ovarian cancer

Y

6

Use of taxanes for breast cancer

Y

8

Hearing aid technology

Y

18

Use of laparoscopic surgery for inguinal hernia

Y

19

Use of donepezil, rivastigmine and galantamine for Alzheimer’s Disease

22

Use of orlistat for the treatment of obesity

Mental health trusts

PCTs

Y

Y Y

The decision of the Multi-Centre Research Ethics Committee chair approached was that this phase of the study was an audit, rather than research and as such did not require full MREC approval. A copy of the letter is included in Appendix 3.1.

2.3.2 Selection of Trusts Acute and mental health trusts A random sample of 40 acute NHS Trusts was selected using computer-generated random numbers applied to the Department of Health 2001-2002 list of acute Trusts, ordered by Trust code. Letters were sent to the Chief Executive and Clinical Governance Leads of 37 Trusts (three were removed due to changed status), inviting them to participate in this audit of the implementation of NICE guidance and a concurrent audit of implementation of the NPSA alert relating to concentrated potassium chloride solution. Twenty-one trusts agreed to participate, of which the first twenty were selected and one became the reserve. Letters were sent to the 20 mental health trusts associated with the acute trusts in the sample above, of which 17 agreed to participate, although one subsequently did return any

12

audit forms. The resulting trusts reflected a good cross-selection of geographical spread and size, including trusts from Wales.

Primary Care Trusts A random sample of 49 PCTs was selected using two sets of 25 computer generated random numbers (one PCT was duplicated). Letters were sent to the Chief Executive and Clinical Governance Leads of these PCTs inviting them to participate; 21 agreed to participate. Each PCT provided a list of practices together with the size of population each covered. We use a stratified sample for selection of practices to ensure that we reflected a range of practice sizes. Using random number tables (Altman, 1999), we chose five practices from each PCT such that we selected one practice from each of the first to third quartiles, and two practices from the fourth. We also selected one reserve practice from the first and second quartiles, and one from the third and fourth. The sample of 50 patient notes for a PCT was to be drawn from the practices accordingly: • • • •

Three patients from the practice in the first quartile; Seven patients from the practice in the second quartile; Ten patients from the practice in the third quartile; Fifteen patients from each of the two practices in the fourth quartile.

In fact many PCTs were unable to select this number of sets of notes for audit because insufficient patients in a practice satisfied the selection criteria.

2.3.3 Response rates Access visits were made to all acute trusts and PCTs, in order to discuss the audit requirements and to ensure arrangements were in place, since we recognised that there was likely to be great variation in how the audits would be undertaken and by whom. We also ran a hotline facility to identify problems early in the audit process and to give consistent advice to audit staff. In addition, we ensured that a quality check was in place at all trusts. Where no process was in place, the York team undertook the quality check. Few problems were identified. Each acute trust was sent 300 audit forms, comprising 50 for each of the six sets of guidance. Each mental health trust was sent 50 forms, and each PCT was sent 100 forms, comprising 50 for each of two sets of guidance. There were considerable delays in receipt of forms from some trusts. Some acute trusts did not return all sets and 3 PCTs, which had originally agreed to participate, failed to return any completed forms. Table 2.5 shows the response rate by guidance. The number of forms returned does not necessarily equate with the number used for analysis as forms were sometimes discarded because of important missing data or non-compliance with the sampling criteria. Reasons for the exclusion of forms are given in each section.

13

Table 2.5:

Total number of packs of fifty forms returned by Guidance with reasons for failure to complete Total number used

18 20 17

Total number of individual completed audit forms returned. 892 990 708

Ovarian cancer

16

521

520


18

875

875

Inguinal hernia

19

950

938

Drugs for Alzheimer’s Disease (MH) Drugs for Alzheimer’s Disease (PCT) Orlistat in primary care

17

703

583

18

215

180

18

689

689

Guidance

No of trusts participating

Wisdom teeth Hip replacement Breast cancer

836 980 707

Reasons for non-return

Service provided by PCT Not applicable Clinicians in 2 Trusts declined to participate. 1 Trust did not complete as not a cancer centre Clinicians in 2 Trusts declined to participate. 1 Trust did not complete as not a cancer centre One service provided by PCTs; 1 Trust declined to participate as pilot site for digital aids 1 Trust did not return – no reason given 1 participating trust did not return – no reason given 3 Trusts did not return forms – no reason given 3 Trusts did not return – no reason given

2.3.4 Data extraction and verification All the audit forms were subjected to the same process for data extraction and verification. All the data were entered into a database. 10% of the total number of forms for each trust, were then re-entered on to a second database, and the two compared. The error percentage calculation was: The number of errors divided by the number of cells (number of cases entered multiplied by the number of variables) x100.

Data entry had a non-text error rate of less than 2% and was therefore satisfactory. The total error rate for breast cancer and hearing aids was greater than 2%, although most of these errors were text errors, in other words mismatch in the input of free text, usually in questions which required a phrase to elucidate ‘other’.

2.4

Phase 3 Collection and analysis of trust information through surveys and interviews

This phase of the study comprised the collection of information from the 20 acute and 16 mental trusts, which participated in the patient audit phase, through the use of questionnaires and

14

structured interviews. We excluded PCTs from this phase because PCTs as organisations did not exist at the time of the release of the sets of guidance under review. The aim was to assess the extent and effectiveness of the mechanisms in place to deal with NICE guidance when it is issued and to determine how local implementation is audited. The interviews provided us to explore in more detail possible reasons for variations in guidance implementation. Trusts were sent: • • •

A survey to be completed by the Chief Executive Guidance specific questionnaires, to be completed by the lead clinician in each trust or by the Chief Executive A consent form requesting permission to interview lead clinicians

Copies of all letters sent to trusts, survey forms and interview schedules are included in Appendices 3.2 – 3.4. The response rates by trust are shown in tables 2.6 and 2.7. Table 2.6

Response rates for acute trusts

ACUTE TRUSTS

CEO survey returned

Guidance surveys Wisdom Total teeth hip replacement

Taxanes for Breast Cancer

Taxanes for Ovarian Cancer


Trust 1 Trust 2 Trust 3 Trust 4 Trust 5 Trust 6 Trust 7 Trust 8 Trust 9 Trust 10 Trust 11 Trust 12 Trust 13 Trust 14 Trust 15 Trust 16 Trust 17 Trust 18 Trust 19 Trust 20 Total Response rate

Y N Y Y N Y Y Y Y N Y Y N Y N N Y Y Y N 13 65%

Y N N Y Y Y Y Y N N Y Y Y Y N N Y Y Y N 13 65%

Y N Y Y Y N Y Y N N Y N N N N N Y N Y Y 10 50%

Y N Y Y Y N Y Y N N Y N Y Y N N Y N Y Y 12 60%

Y N Y Y N N N Y N N Y Y N Y N N Y Y Y Y 11 55%

Y N Y N Y Y N Y N N Y Y N N N N Y Y N Y 10 50%

15

Laparoscopic Surgery for hernias Y N Y N Y N N Y N N Y Y Y Y N N Y Y Y Y 12 60%

Total returns

6 0 5 4 5 2 3 6 0 0 6 4 3 4 0 0 6 4 5 5 68 56.7%

TABLE 2.7

RESPONSE R ATES FOR MENTAL HEALTH TRUSTS

MENTAL HEALTH TRUSTS

CEO Survey

Alzheimers Guidance

Trust 1 N N Trust 2 N N Trust 3 Y Y Trust 4 N N Trust 5 N N Trust 6 Y N Trust 7 Y Y Trust 8 Y Y Trust 9 Y Y Trust 10 Y Y Trust 11 Y Y Trust 12 Y Y Trust 13 N N Trust 14 N N Trust 15 Y* N Trust 16 Y Y Total 9 8 Response Rate 56% 50% * This trust returned one consent form and one CEO survey form as it provides both acute and mental health services

We selected five trusts, from those that returned positive consent forms, to participate in the interview phase. These five were selected because they represented a range of levels of implementation of sets of guidance (see Table 2.8). Trusts were classified by their average implementation “score” across all sets of relevant guidance and trusts were selected on the basis of high average score (good compliance across guidance topics), low average (poorer compliance across topics), or variable compliance (good compliance for some topics, poor for others).

16

Table 2.8

Trust Trust 1 Trust 2 Trust 3 Trust 4 Trust 5 Trust 6 Trust 7 Trust 8 Trust 9 Trust 10 Trust 11 Trust 12 Trust 13 Trust 14 Trust 15 Trust 16 Trust 17 Trust 18 Trust 19 Trust 20

Levels of implementation by Trust by guidance (5 high, 0 low)

Wisdom teeth

Total Taxanes hip Taxanes for Lap Hearing inguinal replace for breast ovarian aids -ments cancer cancer hernia

Trust average

Consent

Comments

5

5

5

3

0

5

3

3

0

5

5

5

5

5

2

4

2

5

4

0

5

5

4

5

4

5

5

4

5 4

3

2

4

Yes

interview

5

0

2.7

No

replace

5

3.6

Yes

4

2

3.8

Yes

3

5

0

3.2

Yes

0

5

5

3.2

Yes

interview

2

0

2.75

Yes

interview

0

3

5

3.7

Yes

Option

5

1

2

5

3.7

No

5

5

5

4

0

4.0

No

3

5

1

1

2.8

Yes

2

5

3

4

5

3.8

Yes

5

5

5

5

3

0

3.8

Yes

5

0

5

2

5

5

3.7

No

4

0

2.3

No

3

5

4.2

Yes

4.3

Yes

3 5

4

5

3

4

4

5

2

5

3

3

4

3.7

Yes

3

1

5

3

2

5

3.2

Yes

3

5

1

2

5

3.2

No

5

Option

interview

Option

interview

Within each trust, the Chief Executive, Medical Director and lead clinicians for the guidance topic were approached. In addition to the general interview schedule appended, trust and guidance-specific interview schedules were drawn up to reflect findings from the audit. Table 2.9 summarises the interviews undertaken by trust.

17

Table 2.9 Trust Trust Trust Trust Trust Trust

Interviews by trust and guidance CE or MD D Y Y Y(2) Y

Wisdom teeth Y Y Y Y

Hearing aids Y Y Y Y

THR

Taxanes

Y Y

Y R R Y Y

Y Y

Laparoscopic Hernia R Y Y Y Y

NB: D=Delegated to Clinical Audit lead, R=Refused

All interviewees were offered face-to-face interviews but where necessary telephone interviews were held (3 out of 28 interviews were carried out by telephone). All interviews were semistructured in format, were conducted in the interviewees’ own office or in a place designated by them and, with permission and where possible, were recorded and transcribed. Where recording proved impossible (by reason of busy and shared offices) notes were returned to the interviewee for checking. Once all interviews were complete, thematic analysis was undertaken.

18

3.

The Removal of Wisdom Teeth – Results

3.1

Introduction

The guidance was published in March 2000, and advised that: •

The routine practice of prophylactic removal of pathology-free impacted third molars should be discontinued in the NHS.

•

The standard routine programme of dental care by dental practitioners and/or paraprofessional staff need be no different, in general, for pathology free impacted third molars (those requiring no additional investigations or procedures).

•

Surgical removal of impacted third molars should be limited to patients with evidence of pathology. Such pathology includes unrestorable caries, non-treatable pulpal and/or periapical pathology, cellulitis, abscess and osteomyelitis, internal/external resorption of the tooth or adjacent teeth, fracture of tooth, disease of follicle including cyst/tumour, tooth/teeth impeding surgery or reconstructive jaw surgery, and when a tooth is involved in or within the field of tumour resection.

•

Specific attention is drawn to plaque formation and pericoronitis. Plaque formation is a risk factor but is not in itself an indication for surgery. The degree to which the severity or recurrence rate of pericoronitis should influence the decision for surgical removal of a third molar remains unclear. The evidence suggests that a first episode of pericoronitis, unless particularly severe, should not be considered an indication for surgery. Second or subsequent episodes should be considered the appropriate indication for surgery.

Our research question was whether there had been a reduction in prophylactic removal of pathology free impacted third molars and, therefore, a reduction in the number of surgical removals of wisdom teeth related in time to the publication of the NICE guidance. Our analysis of activity levels took account of the fact that the Faculty of Dental Surgery of England had issued guidance in September 1997 and the NHS Centre for Reviews and Dissemination had published an article in the British Dental Journal 1997, Song et al, 1997 and published an Effectiveness Matters bulletin in 1998.

3.2

Evidence from National Data

3.2.1

Data sources

We used four data sources: two covering England and Wales, and two covering Scotland as we were interested in whether the different sets of guidance issued in Scotland had a similar impact to those issued in England. The data sources were: • • •

•

Hospital Episode Statistics (HES) Data for England and Wales covering the years 1992 through to the most recent available 2001/02 (quarters 1 to 3 only). Dental Practice Board Data for England and Wales covering the years 1992 to the most recent 2001. Scottish Morbidity Records (the Scottish equivalent of HES) covering the years 1992 to most recent available. Scottish Practitioner Services data (the Scottish equivalent of the Dental Practice Board data) from the GP17 Dental Remuneration System, from 1992 to the most recent available.

19

3.2.2 Findings of national data Figure 3.1

Wisdom teeth extraction activity

Wisdom Teeth Extraction Activity 1992 to 2001 140000

120000

NICE Guidance March 2000

Number of extractions

100000

80000

DPB Wisdom only E&W Scottish DPB data HES Data Operation 1 E&W Scottish HES Data Operation 1 all

60000

40000

National Clinical Guidelines 1997

Effectiveness Matters Vol 3 Issue 2 Oct 1998

20000

0 1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

Year

Figure 3.1 indicates a sharp decline in the number of extractions taking place between 1998 and 2001, falling by over 50% from the peak of just over 200,000 in 1995 to below 100,000 in 2001. NICE Guidance was issued in March 2000. Although the number of extractions fell between 2000 and 2001, this appears to be the continuation of the downward trend that began in 1995. We found no evidence following publication of guidance there was any additional change in the trend of extraction numbers beyond what would have occurred had guidance not been published.

3.3 Evidence from the audit 3.3.1

Sample

The sample for the audit of the extraction of wisdom teeth was defined as follows: “First twenty-five patients who underwent Wisdom Teeth Extraction (OPCS4 F091, F093) on o r after 1 September 2000 and first twenty-five on or after 1 March 2001.”

Seventeen Trusts returned audit forms, although a few were spoilt or incomplete. In all 836 forms were returned, 412 relating to extractions during the first period and 424 to the second.

20

3.3.2

Questions

Three questions were designed to identify those extractions that clearly complied with the NICE guidance. Question 3 Question 4 Question 5

Was the extraction part of other surgery, e.g. for a tumour (reconstructive jaw surgery), or internal fixation of fracture following trauma? Was the extraction carried out prior to orthodontic treatment or orthognathic surgery or because of other medical or surgical conditions? Was any of the following pathology reported? Unrestorable Caries Non -treatable Pulpal Pathology Non -treatable Periapical Pathology Cellulitis Abscess with Osteomyelitis Internal/External Absorption of the Tooth or Adjacent Teeth Fracture of Tooth Cystic Disease of Follicle Pericoronitis (first episode)* Pericoronitis (Infection) Recurrent Plaque Formation Periodontal Disease Other (please list below) Don’t know

* Not compliant – see text

3.3.3

Findings from the audit

Those audit forms which clearly indicated a reason for extraction that accorded with the guidance were identified. Of the remainder, 182 offered some other information pertaining to the extraction. This information was sent to an independent expert, an Emeritus Professor of Dentistry, who was asked to grade the rationale offered as: • • • • •

Unequivocally compliant with NICE guidance; Probably compliant with NICE guidance; Don’t know; Probably not compliant with NICE guidance; Unequivocally non-compliant with NICE guidance.

These judgements formed an important part of the compliance ratings for some Trusts: 46% and 55% forms respectively were referred for judgement for two trusts, whilst, no forms were referred from a third trust. The forms judged unequivocally or probably compliant were added to those already deemed compliant. In total, 90% of wisdom tooth extractions were regarded as compliant with NICE guidance. Table 3.1 presents compliance rates from September 2000 (6 months after the guidance was issued) and after March 2001 (12 months after the guidance was issued).

21

Table 3.1:

Compliance rates before and after March 2001

Number of forms used 48 49 50 47 48 50 50 50 47 50 50 DNP 49 DNP 50 48 50 50 50 DNP

TRUST ID Trust 1 Trust 2 Trust 3 Trust 4 Trust 5 Trust 6 Trust 7 Trust 8 Trust 9 Trust 10 Trust 11 Trust 12 Trust 13 Trust 14 Trust 15 Trust 16 Trust 17 Trust 18 Trust 19 Trust 20 Total and mean % 836 overall DNP = Did not participate

Before March 2001 Total Number of Total Number of extractions extractions compliant compliant as % 24 100.0% 19 79.2% 17 68.0% 20 90.9% 19 79.2% 22 91.7% 24 96.0% 23 92.0% 24 100.0% 25 100.0% 20 80.0% DNP DNP 25 100.0% DNP DNP 23 95.8% 21 87.5% 22 95.7% 20 80.0% 21 87.5% DNP DNP

After March 2001 Total Number of extractions compliant 24 20 19 25 22 22 23 25 23 23 20 DNP 24 DNP 25 22 23 19 26 DNP

Total Number of extractions compliant as % 100.0% 80.0% 76.0% 100.0% 91.7% 84.6% 92.0% 100.0% 100.0% 92.0% 80.0% DNP 100.0% DNP 96.2% 91.7% 85.2% 76.0% 100.0% DNP

369

385

91%

90%

The Table shows that the compliance rate varied little from the first period to the second, suggesting that there was no evidence from patient level data of a significant trend towards greater compliance. The most common reasons given for extraction that appeared to be in conflict with the guidance (i.e. not valid reasons) included impaction (n=82); first episode of pericoronitis (n=56) and pain (n=51) including recurrent pain (n=15). The complete list is in Appendix 1.4.

3.4

Evidence from the Phase 3 survey

In response to the guidance specific questionnaires that we sent to the lead clinicians in the 20 acute trusts which participated in the audit phase, we received 13 completed forms, a response rate of 65%. All respondents stated that they had fully implemented the guidance. The factors believed by the clinical leads to contribute to the successful implementation of this guidance are shown in Table 3.2.

22

Table 3.2

Factors reported as contributing to the successful implementation of NICE Wisdom Teeth Guidance within a trust

Factors

No of times cited* 11 8 6 5 4 2 0 36

Single specialty service Low cost High level of evidence or recommendation Other Clinical lead or local champion High local priority Agreed in local development plan Total cita tions NB Multiple factors could be cited

% of citations 31% 22% 17% 14% 11% 6% 0%

The comments under ‘other’ stated that this practice was already in place. Therefore the most influential factors given were the low cost and the ease with which it could be implemented in a single specialty service, especially where the guidance reinforced existing practice. Additionally, ten responses (76.9%) stated that there were no significant cost consequences of implementing this guidance. Only two trusts stated that this guidance was high on the list of its priorities. No trust stated that that the implementation of the guidance had displaced plans for other service developments. Finally, five trusts provided additional comments about the role of NICE or of the methodologies developed. Two of the comments referred to the fact that the practice had been implemented prior to the issue of the guidance, one believed that the guidance was not clear, and the final substantive comment stated that the guidelines work best when applied to the referrals being made by general dental practitioners.

3.5

Evidence from the interviews

3.5.1 Response to the guidance Four oral surgeons were interviewed. All reported that the prophylactic removal of third molars had ceased some years before the publication of the NICE guidance, especially where the consultants had produced local guidelines to manage referrals from General Dental Practitioners(GDPs). The drivers for this earlier change in practice were perceived to be: • • • • •

Increasing referrals from GDPs (for whom the cost of the procedure was greater than the fee) Need to undertake greater numbers of more complex procedures Publications indicating the percentage of symptom-free molars being removed “Cost conscious ethos of the Thatcher years” Growing awareness of risk of nerve damage and consequent litigation

All four interviewees agreed with the guidance, although one indicated that local oral surgeons were divided on the advisability of removing an asymptomatic wisdom tooth at the same time as other teeth in cases, for example, of food packing in a patient with a history of poor oral hygiene. Such a strategy of prophylactic removal was seen as avoiding an inevitable second referral and anaesthetic.

23

3.5.2 Comments on the audit Two of the four oral surgeons highlight reasons why the audit might have under-estimated the extent of compliance in Trusts. The evidence that a wisdom tooth extraction was “NICEguidance compliant” may often be contained in the referral letter from the GDP, or on X-rays, rather than re-stated by the oral surgeon in the notes of the operation. Two respondents claimed to have experienced such difficulties when undertaking their own audits and we accept this as a possibility but nevertheless, compliance with this piece of NICE guidance was extremely high.

3.5.3 Comments on the guidance There was some surprise that the guidance had been necessary in view of the level of awareness of the issues in the profession, but all had found the publication useful in their dealings with GDPs and some with patients.

3.6 • • • • •

Summary of findings The downward trend in the number of extractions which began in 1995, continued following publication of the NICE guidance. There was no evidence of a change in the rate of decline. The audit showed that 90% of extractions over the year from 1 September 2000 met the NICE guidance criteria for extraction; Clinicians suggested that the figure for audited compliance might be an under-estimate as a result of poor recording by surgeons; The compliance rate for the whole sample is 90% with a range from 72% in one trust to 100% in three trusts; The most common centre of non compliance (56) was due to extractions carried out for first episode of pericoronitis.

.

24

4.

Prostheses for Hip Replacements- Results

4.1

Introduction

The guidance was published in April 2000, and advised that: • •

Surgeons should use prostheses for total hip replacement which either have a demonstrable replacement of 10% or less, at ten years, or a minimum of three years provided that the performance of the prostheses is consistent with the ten year benchmark; The NHS Purchasing and Supply Agency should issue information on the performance of prostheses to the NHS

The guidance also states that: •

•

Prostheses (cemented, uncemented and hybrid) that have not been shown to achieve either of these benchmarks should be the subject of comparative clinical evaluation before they can be recommended for routine use in the NHS; There is currently more evidence of the long-term viability of cemented prostheses, which in many cases occupy the lower end of the range of prostheses cost, than there is for uncemented and hybrid prostheses.

The research question was whether hip prostheses which meet the benchmark standard are being used in hip replacement operations and whether the number and proportion of benchmarked hip prostheses had increased since the publication of the NICE guidance. One of the important issues in this case than is which prostheses were deemed to meet these benchmark standards?

4.2

Role of NHS Purchasing and Supply Agency (NHS PASA)

NICE also recommended that •

The NHS Purchasing and Supply Agency has agreed with NICE to work with hip prostheses suppliers to collate and disseminate information about the performance of individual products against the benchmark(s) in order to avoid trusts having to collect this information individually

The NHS PASA collected data from hip prosthesis manufacturers including whether the prostheses were cemented or cementless and modular or monobolic, the revision rates and declaration of benchmark information for 3 and 10-years, as well as design features. This database was made available to the NHS on the NHS Net, through the NHS PASA website in September 2001, over a year after the issuing of the guidance. We have anecdotal evidence to suggest that the NHS PASA database was not widely accessed, since few people knew about its existence. The NHS PASA database only included assessments from manufacturers, on whether the individual hip prostheses met the benchmarking criteria. However, there had been guidance in the three earlier health technology assessments (Sheldon et al, 1996; Faulkner et al, 1998; Fitzpatrick et al, 1999) as to which prostheses were associated with better outcomes.

25

The NHS PASA, in consultation with a number of professional bodies and the industry (both through the ABHI and the individual suppliers), established an expert panel, (the Orthopaedic Data Evaluation Panel) (ODEP) in Autumn 2002. Initially ODEP reviewed data submitted by manufacturers, for prostheses meeting the 10-year benchmark. It was intended that the results of this process would be made available to trusts by the end of August 2003. However, problems with collecting and verifying data from manufacturers led to a delay. The results are now likely to be issued in early 2004, and work on collecting data on the 3-year benchmark will be collected thereafter.

4.3

EVIDENCE FROM NATIONAL AND REGIONAL DATA

4.3.1 Data sources We used three data sources: • •

•

Hospital Episode Statistics (HES) Data for England and Wales covering the years 1992 through to the most recent available, quarters 1-3 of 2001/02 The Trent Arthroplasty Audit Group and the Welsh Arthroplasty Audit Group Database (TWAAG). This is a register of knee and hip replacements carried out in the area covered by the former NHS Trent Region and the North Wales region. The database is managed by a team based at the University of Leicester and was established in 1990 NHS Purchasing and Supply Agency Database , which provides benchmarking data to the NHS, collected from manufacturers.

4.3.2 Comparison of HES and TWAAG Data Although we could identify levels of hip replacement activity using the HES these data do not identify the prostheses used in the operations. At the time of undertaking the research, the TWAAG database was the only significant and routine reliable source of UK data on prostheses used. The HES data indicate total levels of activity for hip replacements, which have increased over the period of investigation. The TWAAG database contained the records of 7898 patients who had received hip replacements since January 1998 in 22 hospitals. The data accounted for 40% of the hip replacements undertaken in these hospitals.

4.3.3 Use of prostheses in the TWAAG database The TWAAG database indicated that 51 differently named products were in use. 68% of patients received a single prosthesis (defined as cup and stem from same manufacturer); one manufacturer accounted for 45% (The Charnley) of the replacements, and the three most popular brands together accounted for 60% of the replacements. The dominant brand also accounted for 26% of the cups in mixed prostheses (defined as having different manufacturers for cup and stem), with five prostheses accounting for 77% of all cups. The dominant brand (The Exeter which was not the dominant brand for cups) accounted for 74% of the stems in mixed prostheses.

26

4.3.4 Analysis of the performance of prostheses in the TWAAG database The NHS PASA provided the team with data on the performance of prostheses against the 3 and 10-year benchmarks. We applied this NHS PASA benchmarking information to the data on prostheses in the TWAAG database. We categorised data according to: • • •

Declared by the NHS PASA as meeting the 3-year or 10-year benchmark; Not declared, i.e., the prostheses are included on the NHS PASA database but the manufacturers have either not provided benchmarking information to the NHS PASA or do not have the evidence that they meet the benchmarks; Unknown, i.e., the prostheses are not on the NHS PASA database therefore the benchmark status is unknown.

Appendix 4.2 gives the results of that classification process. Table 4.1 shows that the percentages of prostheses meeting the benchmarks in all categories declined slowly across the period from 1998-2002. The number of hip replacem ents is rising, however the use of benchmarked prostheses is not rising commensurately.

27

Category of prosthesis Single Prostheses

1998

1999

2000

10 yr Benchmark declared 3 yr Benchmark declared

77.7% 82.6%

72.8% 84.0%

63.9% 80.7%

Cups in mixed prostheses 10 yr Benchmark declared

63.3%

58.3%

53.8%

3 yr Benchmark declared

86.7%

82.6%

78.5%

Stems in mixed prostheses 10 yr Benchmark declared

86.7%

88.3%

81.9%

GUIDANCE ISSUED APRIL 2000

Table 4.1 The percentage of implanted prostheses meeting the 3-year and 10-year benchmarks (TWAAG database) 2001

2002

Total

66.3% 78.5%

66.6% 76.1%

68.5% 80.7%

42.2%

25.4%

46.6%

78.2%

75.6%

79.2%

87.2%

84.7%

85.5%

3 yr Benchmark declared 96.0% 97.7% 97.6% 97.4% 92.8% NB Prostheses within the 1 0-year benchmarking will also include those in the 3-year benchmarking.

96.7%

We also examined use of the prostheses identified by the three previous reviews as having good long-term outcomes; the evidence in all three reports was similar in respect of which prostheses had the best outcomes. One prosthesis, the Charnley cemented prosthesis, was identified as having the greatest body of evidence and as having performed well as measured by revision rates and degree of thigh pain by all three reviews. This prosthesis was also the one most commonly used in the NHS, in comprising 45% of all prostheses from a single manufacturer and model, and 26% of the cups in mixed prostheses. However, one review expressed concern that the existing evidence may not be entirely relevant to the Charnley because the design has changed. The Exeter prosthesis was identified in both NHS HTA reports as performing well. The Exeter comprised only 5% of single manufacturer prostheses, but 75% of the stems in mixed prostheses. The reports also identified five more prostheses whose performance was deemed of good or intermediate standard (including the Stanmore, Howse, Muller, McKee-Farrar and Lubinus) of which only the Stanmore and the Muller are used in the NHS and only the Muller is used to any great extent. More than 50 different prostheses are being used, whilst the technology assessments indicated favourable or intermediate findings for seven of which only four appear to be used in the NHS.

4.3.5 Discussion of the data from TWAAG database It is noteworthy that a large minority of hip replacements recorded in the TWAAG Register use prostheses for which benchmarking evidence is not available. The proportion of implanted prostheses which do not meet the benchmark standard have not in general fallen following the issue of the NICE guidance. However, it does appear as if the use of benchmarked prostheses has stabilised over this period. Possible reasons may include consultants:

28

• • •

•

Not accessing or unable to access the database provided by NHS PASA, either because they were not aware of its existence or because the database was not available for periods of time Having access to alternative data sources, such as data in publications, or data that they have collected over time Having personal experience of using certain types of prostheses and preferring to continue using these despite the absence of declared benchmarking data Trying newer prostheses, possibly in clinical trials

The evidence suggests that the downward trend in the use of 10-year benchmarked single prostheses levelled out after publication of the NICE guidance. We also found that the relative variation between the Trusts in the use of 3 year benchmarked prostheses did not materially change before and after the publication of the guidance.

4.4

Evidence from the trust level audits

4.4.1 Sample The sample for the audit of the use of hip prostheses was defined as follows: “ First twenty-five patients to undergo total hip replacement (OPCS 4: W370, W371, W372, W373, W378, W379, W380, W381, W382, W383, W388, W389, W390, W391, W392, W393, W394, W398, W399) on or after 1 October 2000 and first twenty-five to undergo total hip replacement after 1 April 2001.”

Twenty trusts returned a total of 990 forms. Ten of these forms were discarded from the analysis, as dates had not been recorded on the audit forms.

4.4.2 Questions Three questions on the audit form sought to establish whether the prostheses used complied with the benchmark specified in the NICE guidance: Question 5 Question 6

Question 7

Description of procedure. Details of components and prostheses used, broken down by: • Manufacturer, model and material; • Cup, head and stem. Is any reason recorded in the notes for the choice of prosthesis (e.g. patient age/activity pattern)?

4.4.3 Analysis Audit clerks in the Trusts recorded the make of prosthesis (head, cup and stem) on the audit forms. Two manufacturers were dominant, accounting for 69% of heads, 66% of stems and 61% of cups, with one make accounting for almost 50% of the stems. Six manufacturers together accounted for 90% of the heads, 81% of stems and 82% of cups. The total number of different brands of prostheses with positive benchmarking evidence that were in use, as

29

recorded during the audit of patient notes was 15 for heads, 19 for stems and 21 for cups. This accords with data from the TWAAG database. We applied the NHS benchmarking information that we had previously collected and as shown in table 4.2, this analysis indicated that there was no increase in the use of benchm arked prostheses after issue of the NICE guidance. Table 4.2: Implanted prostheses identified from patient level audit achieving at least 3 year benchmark

Total

Head Before 1st April 2001 No %

On or after 1st April 2001 No %

Cup Before 1st April On or after 1st 2001 April 2001 No % No %

Stem Before 1st April On or after 1st 2001 April 2001 No % No %

325

321

358

373

67%

65%

74%

369

74%

77%

361

73%

Table 4.3 indicates the percentage of implanted prostheses meeting the benchmark across trusts. Table 4.3

Percentage of prostheses meeting the benchmark across trusts

TRUST ID

Head

Trust 1 Trust 2

No 50 10

% 100.0% 22.2%

No 50 31

% 100.0% 68.9%

No 50 13

% 100.0% 28.9%

Trust 3 Trust 4

7 48

14.6% 98.0%

22 48

45.8% 98.0%

19 48

39.6% 98.0%

Trust 5 Trust 6

34 0

81.0% 0.0%

35 23

83.3% 46.9%

34 16

81.0% 32.7%

Trust 7 Trust 8 Trust 9

34 34 35

70.8% 68.0% 70.0%

43 43 41

89.6% 86.0% 82.0%

43 43 48

89.6% 86.0% 96.0%

Trust 10 49 Trust 11 43

98.0% 86.0%

50 38

100.0% 76.0%

50 25

100.0% 50.0%


52.0% 98.0%

39 47

78.0% 94.0%

37 48

74.0% 96.0%


34.0% 96.0%

47 30

94.0% 60.0%

46 49

92.0% 98.0%

Trust 16 47 Trust 17 30 Trust 18 29

95.9% 60.0% 58.0%

26 40 22

53.1% 80.0% 44.0%

47 32 25

95.9% 64.0% 50.0%


36.0% 76.0%

12 40

24.0% 80.0%

22 39

44.0% 78.0%

Total

65.9%

727

74.2%

734

74.9%

646

Cup

Stem

30

There are two reasons for categorising prostheses as not achieving the 3- or 10-year benchmark: either the data are not available in the NHS PASA database, or the data are available but show that the prostheses do not meet the benchmark. We also analysed the prostheses for which there was no positive benchmarking evidence. This showed that for more than 67% of these prostheses, the data were not available, rather than the prostheses failed to meet the benchmark. The reason for choosing a particular prosthesis was given on only 38 (3.8%) of the 990 audit forms reviewed. None of the notes from nine trusts had reasons recorded, and in a further nine trusts, fewer than five audit forms had a reason given. One trust had reasons given in 27% of the notes. The most common reasons recorded (recognising that these reasons may not reflect the situation across the majority of trusts or patients) were: • • •

Age of patient, often in conjunction with rationale for using cementless (there is a perception that cementless prostheses are easier to revise and therefore preferable in younger patients who are more likely to require a revision) Activity level of patient; Clinical reason, such as presence of avascular necrosis or acetabulum deficiency.

These reasons accorded with those given in the interviews (see below).

4.5

Evidence from the surveys

In response to the guidance specific questionnaires that we sent to the lead clinicians in the 20 acute trusts which participated in the audit phase, we received 10 completed forms (50%). Six (60%) of the trusts stated that they had fully implemented the guidance; two trusts stated that the guidance had only been partly implemented and two trusts stated that the guidance had not been implemented. The factors perceived by respondents as contributing to the successful implementation of this guidance are shown in Table 4.4:. Table 4.4 Factors perceived as contributing to the successful implementation of the NICE Guidance within a trust Factors Other High local priority Single specialty service High level of evidence or recommendation Clinical lead or local champion Low cost Agreed in local development plan Total citations *Multiple factors could be cited

No of times cited* 3 2 2 2 1 1 0 11

% of citations 27% 18% 18% 18% 9% 9% 0%

Two of factors under ‘other’ stated that the guidance reflected standard practice, the third stated that all consultants had been requested to review their practice in line with the guidance. Of those who gave reasons for no or limited compliance, one trust said the guidance had not been regarded as a priority, two trusts cited the “low level of the evidence”, whilst two were critical of the guidance: the comments are quoted below:

31

• •

“[Guidance] doesn’t pay enough attention to existing recent technology. Clinical view is that potentially restrictive decisions should be made on the basis of many years experience of individual clinical situations…” “The document was a patient accountancy orientated fudge..”

No trusts stated that there were any cost implications of implementing the guidance. Only two trusts stated that the guidance was high on the list of local priorities, and none stated that the guidance displaced plans for other service developments. Six trusts provided comments about the role of NICE or of the methodologies developed. Two trusts stated that the practices in the guidance were already well established, whereas a further three criticised the guidance: • •

•

4.6

“The NICE guidelines were too vague to be useful…I do not feel the guidelines added anything useful” “Unlike the reasoned discussion in the hip resurfacing document of June 2002, the THR document shows a clear bias towards Charnley cemented hip without appreciating the complexity of hip surgery demands a wider approach to implant selection to manage the young” “Feels a bit like ‘big daddy’ dictating. Very similar to the political party in power at the moment”


4.6.1 Introduction Four orthopaedic surgeons were interviewed. Two worked in Trusts which appeared to use a high proportion of benchmarked, cemented hips (as suggested by the guidance) and two worked in Trusts which used significant numbers of hydroxyapatite (HA) -coated hips. In the second group one surgeon was using a stem that s/he had personally designed and which was manufactured to that specification by a company who also funded the evaluative research. This respondent was unaware of how many other surgeons were using this stem.

4.6.2 Receipt of the guidance None of the interviewees had a clear memory of how the guidance had come to them – one suggested that it came from the British Orthopaedic Association. The respondents using established cemented hips were surprised that the guidance was necessary as the conclusion was obvious. Those using HA coated hips were relieved that the guidance did not specifically preclude their use.

4.6.3 Action taken on receipt of the guidance Interviewees felt that no action had been required. Each surgeon said that they personally had more than ten-year data on the hips they were using, and that this was satisfactory. The rationale for the use of HA coated hips was their cementless nature. It is inevitable, it was argued, that a cemented hip will fail, because the cement will become increasingly friable with the result that debris from wear and tear gets into the joint. An HA coated hip, where the

32

prosthesis is described as held in place by outgrowth of bone into the biological coating, is thought to have the potential to outlast a cemented prostheses; of particular importance in younger patients. One respondent claimed that a trial comparing HA coated and cemented hips was in its fifteenth year and had shown that the HA coated variety was at least as good. It was also claimed that, should a revision for aseptic loosening be required, these are easier as there is less bone destruction at the time of the original operation. HA coated prostheses are significantly more expensive, but one of the Trusts where their use was high through servicing a large debt recovery plan had not challenged the use of HA coated hips. The surgeons using HA coated hips clearly believed that they were fully compliant with the guidance on the basis of ten year data, regardless of the fact these data were not in the public domain or the NHS PASA data or that they were using, almost exclusively the more expensive, uncemented hips.

4.6.4 Feedback on NICE guidance Two of the interviewees were of the opinion that the guidance did not adequately reflect real clinical challenges, and that this was, they perceived, due to a lack of orthopaedic surgeon involvement in the drawing up of the guidance.

4.7 •

•

• • • • • •

Summary of findings Formally there was no evidence of an increased in the use of 3 or 10-year benchmarked hip prostheses after publication of the NICE guidance. However, for mixed cup prostheses there is evidence of a statistically significant increase in use of 10-year benchmarked prostheses. The nature of the data collected and the small numbers suggest that this may be due to a single surgeon or small number of surgeons modifying their behaviour after the guidance. Audit data suggested that although cemented prostheses were used in 81% of total hip replacements (in accordance with NICE guidance), there is great variation across trusts. Nine trusts used at least 90% cemented prostheses whereas 98% of prostheses were cementless HA coated in one trust 9% of hip prostheses are HA coated, and are used by one-half of the trusts: 3% of hip prostheses are described as “press fit”, and 4% of hip prostheses are described as hybrid. Although the numbers are small in total, five trusts use these in more than 10% of procedures; The greatest variation is seen in the use of modular hip joints, although only half the trusts used them, three trusts used them in more than 50% of cases Surgeons using HA coated hips believed that cement will inevitably fail and that HA coated prostheses would be proven to out-perform traditional, cemented hips. Surgeons using uncemented hips for which they had their own follow up data, often of more than ten-years’ duration, believed themselves to be fully compliant with the NICE guidance. It is unclear whether benchmark data become invalid when prostheses from different manufacturers are “mixed” within a patient e.g., a stem from one manufacturer and a cup from another. It is conceivable that whilst both may individually have good revision rates, these rates may be different when the two are used in combination.

33

5.

The use of docetaxel, paclitaxel, vinorelbine and gemcitabine for non-small cell lung and breast and ovarian cancer - Results

5.1

Introduction

Although the NICE guidance relates to the use of taxanes for breast and ovarian cancers and docetaxel, paclitaxel, vinorelbine and gemcitabine for non-small cell lung (NSCL) cancer, these drugs are also licensed for use in other cancers. National and local level data only show the total use of each drug and not the cancer for which it was prescribed. The effect of an individual guidance cannot, therefore, be analysed in isolation, nor can it be disassociated from use not covered by NICE. Data1 on incidence (for 1998) show that breast cancer is the most common, with 39,550 cases, and the number of cases of ovarian cancer is small at 6,880. Whilst there are large numbers of lung cancer (38,780 cases) only 15-20% of lung cancers are NSCL.

5.2

Guidance For Breast Cancer

The guidance tested by audit was that issued in May 2000 (Guidance no. 6). The nature of the guidance offered was as follows:

•

Both docetaxel (Taxotere) and paclitaxel (Taxol) should be available for the treatment of advanced breast cancer where initial cytotoxic chemotherapy (including anthracycline) has failed or is inappropriate .

The guidance did not make entirely clear whether the initial cytotoxic therapy could have been given in the context of the early form of the disease or whether it referred solely to its use in advanced cancer. The second guidance (no. 30), released fifteen months after the first, in September 2001, stated unequivocally that the taxanes were not recommended for use as first line treatments in advanced breast cancer .

5.3

Guidance for ovarian cancer

The guidance was issued in June 2000 and advised that: • •

•

1

Paclitaxel in combination with a platinum therapy (cisplatin or carboplatin) should be the standard initial therapy for patients with ovarian cancer following surgery; The use of paclitaxel/platinum combination therapy in the treatment of recurrent or resistant ovarian cancer is recommended if the patient has not previously received this drug combination; The use of paclitaxel/platinum combination therapy is not recommended outside the context of a clinical trial, if the patient has previously received this drug combination.

Data taken from Cancer Research UK, ONS and Nice Guidance.

34

5.4

Guidance for non small cell lung cancer

The guidance was issued in June 2001 and advised that: • •

5.5

Gemcitabine, paclitaxel and vinorelbine should each be considered as part of initial (first-line) chemotherapy Docetaxel should be used for locally advanced cancer but only where previous chemotherapy has failed

Evidence from National and Regional Data

5.5.1 Data Sources 5.5.2 Introduction Data from hospital pharmacies were analysed by the mass of the drug used, in grams and milligrams. It is not possible to estimate the number of patients for whom the drugs have been prescribed since the dosage for each patient is different and we have not been able to obtain patient-based data from hospital pharmacy systems. Furthermore it was not possible to discern which cancer the drugs were prescribed for.

5.5.3 Descriptive Analysis of Data The figures below show the prescribing of taxanes across the 24 hospitals (including 9 cancer centres) from which we collected usable data.

Figure 5.1:

Use of docetaxel Total Docetaxel

30,000

Breast Cancer Guidance updated Sept. 2001 25,000

15,000

NSLC Guidance brought in June 2001

10,000

Breast Cancer Guidance brought in May 2000 5,000

0

Ap r-9 7 Jun -97 Au g-9 7 Oc t-9 7 De c-9 7 Fe b-9 8 Ap r-9 8 Jun -98 Au g-9 8 Oc t-9 8 De c-9 8 Fe b-9 9 Ap r-9 9 Ju n-9 9 Au g-9 9 Oc t-9 9 De c-9 9 Feb -00 Ap r-0 0 Jun -00 Au g-0 0 Oc t-0 0 De c-0 0 Fe b-0 1 Ap r-0 1 Jun -01 Au g-0 1 Oc t-0 1 De c-0 1 Fe b-0 2

mg

20,000

Month

35

Ap r-97 Jun -97 Au g-97 Oc t-9 De 7 c-97 Feb -9 Ap 8 r-98 Jun -9 Au 8 g-9 8 Oc t-98 De c-9 8 Feb -99 Ap r-99 Jun -99 Au g-9 9 Oc t-99 De c-99 Feb -0 Ap 0 r-00 Jun -00 Au g-00 Oc t-0 De 0 c-00 Feb -01 Ap r-01 Jun -01 Au g-0 1 Oc t-01 De c-01 Feb -02

Grammes

Figure 5.2: Use of gemcitabine

TotalGemcitabine

800

700

600

500

400

300

200 Guidance brought in June 2001for NSLC

100

0

Month

36

FIGURE 5.3:

USE OF PACLITAXEL Total Paclitaxel

100 90 NSLC Guidance brought in june 2001

80 70

Grammes

60 Ovarian Cancer Guidance brought in june 2000

50 40 30 20

Breast Cancer Guidance brought in may 2000

10

Ap r-9 7 Jun -97 Au g-9 7 Oc t-9 7 De c-9 7 Fe b-9 8 Ap r-9 8 Jun -98 Au g-9 8 Oc t-9 8 De c-9 8 Fe b-9 9 Ap r-9 9 Jun -99 Au g-9 9 Oc t-9 9 De c-9 9 Fe b-0 0 Ap r-0 0 Jun -00 Au g-0 0 Oc t-0 0 De c-0 0 Fe b-0 1 Ap r-0 1 Jun -01 Au g-0 1 Oc t-0 1 De c-0 1 Fe b-0 2

0

Month

Figure 5.4:

Use of vinorelbine

Total Vinorelbine 18,000

16,000

14,000 NSLC Guidance brought in June 2001

12,000

mg

10,000

8,000

6,000

4,000

2,000

Ap r-9 7 Jun -97 Au g-9 7 Oc t-9 7 De c-9 7 Fe b-9 8 Ap r-9 8 Ju n-9 8 Au g-9 8 Oc t-9 8 De c-9 8 Fe b-9 9 Ap r-9 9 Jun -99 Au g-9 9 Oc t-9 9 De c-9 9 Fe b00 Ap r-0 0 Jun -00 Au g-0 0 Oc t-0 0 De c-0 0 Fe b-0 1 Ap r-0 1 Jun -01 Au g-0 1 Oc t-0 1 De c-0 1 Fe b02

0

Month

There was a statistically significant increase in the use of docetaxel and paclitaxel of 1112 patient months and 3.7 patient months respectively, following publication at NICE Guidance.

37

This corresponds to an increase in the constant parameter of the time series analysis of 1.9 (95% CI:530, 2222) and 0.67 (95% CI: 1.1, 7.8) respectively. In the case of paclitaxel, guidance was issued for its use for breast cancer, ovarian cancer and NSCLC; the guidance on breast and ovarian cancer were published simultaneously. As breast cancer dominates the number of cancer cases in the UK, identification of a structural break focused only around the time of the publication of the breast cancer guidance. We found no evidence of a change in the prescribing of gemcitabine and vinorelbine as a result of the guidance.

5.6

EVIDENCE FROM THE AUDIT ON THE USE OF TAXANES IN BREAST CANCER

5.6.1 Sample The absence of a clear, universally accepted, definition of advanced breast cancer, combined with the ambiguity of the guidance and the statement that taxanes ‘should be available’ ruled out an audit of appropriateness in patients with advanced/ metastatic disease. We, therefore, took the decision to concentrate on the use of the drugs in early breast cancer (i.e. inappropriate use) and defined the sample by the date of first diagnosis, thus:

“First twenty-five women first diagnosed with Breast Cancer (ICD10: C50.0 - C50.9) on or after 1 December 2000 and first twenty-five women first diagnosed after 1 May 2001.”

5.6.2 Questions We asked four questions about the use of taxanes in the treatment of breast cancer. Question 3a Question 3b Question 3c

Question 3d

Was the treatment described as: Neoadjuvant Adjuvant Was treatment part of a randomised controlled trial? Did patient receive: Docetaxel (Taxotere) Paclitaxel (Taxol) Epirubicin (Pharmorubicin) Mitoxantrone (Mitozantrone, Novantrone, Onkotrone) Vinorelbine (Navelbine) Other (please list below) If patient did not receive any of the drugs listed in 3c above, was treatment: Refused by patient Contra-indicated (Please give reasons for refusal or nature of contra-indication)

38

5.6.3 Analysis of breast cancer results We asked for information on three courses of treatment and queried whether the treatment in each case was adjuvant or neo-adjuvant. A negative response to these questions was regarded as an indication of the disease being ‘advanced.’ A total of 707 forms were returned from 17 Trusts. Three trusts did not participate and a further four trusts interpreted the sample as women who had been diagnosed with breast cancer and who had gone on to receive chemotherapy. A number of the Trusts were cancer centres, others referring some or all of their patients elsewhere. In 679 cases out of the 707 included (96%), no taxanes had been prescribed in any of the three courses of chemotherapy following diagnosis. A total of 19 patients had received docetaxel in the first course, of whom 18 were receiving it as an adjuvant treatment. Fifteen of the 18 patients were recorded as having been recruited into a randomised controlled trial, research acknowledged and promoted by the guidance. The trial status of the other three (all from Trust 11) was ‘not known’ (See Table 5.1). Seventeen of those receiving docetaxel in the first course were also receiving an anthracycline (adriamycin or epirubicin) or anthracycline-containing combination. Two others were receiving CMF therapy (cyclophosphamide, methotrexate and fluorouracil) together with the docetaxel. The one remaining individual of the 19 taxane recipients was said to be neither receiving it as an adjuvant nor neo adjuvant treatment and was, therefore, deemed to have an advanced cancer. This was in accordance with at least one interpretation of the guidance, as discussed earlier. Seven patients received docetaxel in the second course, all of whom had received an anthracycline in the first course. A further two patients, not said to be in an RCT, received paclitaxel in the second course. Seven sets of notes reported that no information on drug regimens was available from the cancer centre and 25 recorded “nothing in the notes.” These findings suggest a lack of communication between referring trusts and cancer centres. There was some anecdotal evidence to support this. Table 5.1:

Use of Taxanes by Trust

Trust Id

Number Consultant of audit Present in 2 forms Trust returned

Trust 1

36

Trust 2

9

8

Trust 3 Trust 4

50

14

Trust 5

40

Ist Course of Chemotherapy Patients Number of Number of in an Patients patients RCT receiving receiving docetaxel docetaxel in an RCT 1

2

Number of Patients receiving paclitaxel*

1 1

1

37

1

1

1 40

2nd Course of Chemotherapy Number of Number of Patients patients receiving receiving docetaxel** docetaxel in an RCT

10

3

5

1

3

2

Consultant responsible for chemotherapy treatment is employed by the trust in which audit is being undertaken

39

Trust 6

50

Trust 7

DNP

Trust 8 Trust 9

37 35

35

1 2

1

1

Trust 10

41

41

18

9

8

Trust 11

50

17

6

1

D/K

Trust 12

50

1

Trust 13 Trust 14

50 DNP

1

Trust 15

50

1

1

Trust 16

DNP

Trust 17

22

Trust 18 Trust 19

19

15

45

23

5

50 50

14

3 1

Trust 20

50

44

TOT AL

707

281

50

1 1

1

7

1

2

* No patients in RCT * All patients different to those in 1st course

5.6.4 • • • • • •

Conclusions In the 13 Trusts which returned audit forms and interpreted the audit inclusion criteria correctly, 57% of women received chemotherapy; In the 17 participating Trusts, 19 patients received docetaxel in the first course, of whom 15 were known to be in an RCT and 1 was believed to have an advanced form of the disease; 3 patients in one Trust, whose trial status was not known, received docetaxel in the first course. If not in a trial, these treatment regimes are inconsistent with the guidance; All 9 patients who received taxanes (7 docetaxel and 2 paclitaxel) in the second course had received an anthracycline in the first course (this is consistent with the guidance) Significant problems were experienced by coding staff in the identification of the sample There were also some problems, although fewer than we were led to expect, with communication between cancer centres and the “home Trust” of patients.

We can reasonably deduce that there is very little evidence that taxanes are being used inappropriately in the treatment of early breast cancer. Given the finding of the high-level data that taxanes are being increasingly prescribed, it app ears that these are being used for more advanced forms of the disease, or in the context of randomised controlled trials.

5.7

Evidence from the audit on taxanes for ovarian cancer

5.7.1 Sample The sample for the audit of the use of taxanes for the treatment of ovarian cancer was defined as:

40

“First twenty- five patients first diagnosed with ovarian cancer (ICD10: C56) on or after 1 November 2000 and first twenty-five on or after 1 April 2001.”

Sixteen trusts submitted completed forms: clinicians in two trusts declined to participate, and another two trusts believed that they were not eligible to participate, as they did not treat patients with ovarian cancer. Because of small numbers, many trusts were unable to identify 50 patients split between two samples of 25. We, therefore, focused our analysis on total patient numbers.

5.7.2 Questions We asked five questions which reflected the patient pathway, and the requirements of the guidance: Question 2 Question 3a Question 3b Questions 5 & 6

Has the patient received an oophorectomy? Has the patient received radiotherapy and/or chemotherapy? Was treatment part of a randomised controlled trial? Description of drugs received in first and subsequent courses of chemotherapy.

5.7.3 Analysis of ovarian cancer results A summary of the analysis of the results that we received in response to the above questions is shown in decision tree format in Figure 5.5.

41

Figure 5.5:

Patient pathway for treatment of ovarian cancer In RCT (2) Received Paclitaxel as 2nd course (11) Received Paclitaxel in combination with either Cisplatin or Carboplatin as 1st course (126)

Not in RCT (8) Don't Know (1)

Did not receive Paclitaxel as 2nd course (115) In RCT (1) Received Paclitaxel in combination with either Cisplatin or Carboplatin as 2nd course (2) Don't Know (1)

Had Oopherectomy (379) Didnot receive Paclitaxel as 1st course (251)

Did not receive Paclitaxel as 2nd course (241) Received Paclitaxel alone as 2nd course (8)

Not in RCT (8)

Did not receive Paclitaxel as 2nd course (1) Received Paclitaxel alone as 1st course (2)

Audit Forms returned (520)

Received Paclitaxel alone as 2nd course (1)

In RCT (1)

Received Paclitaxel as 2nd course (1)

Not in RCT (1)

Received Paclitaxel in combination with either Cisplatin or Carboplatin as 1st course (19) Did not receive Paclitaxel as 2nd course (18) Did not have Oopherectomy (129)

Don't Know (2) Received Paclitaxel in combination with Cisplatin or Carboplatin as 2nd course (2) Did not receive Paclitaxel as 1st course (110)

Did not receive Paclitaxel as 2nd course (104) Received Paclitaxel alone as 2nd course (4)

Don't Know (12)

43

Not in RCT (4)

Of the 520 patients for whom we have data, 379 (73%) were said to have had an oophorectomy; 129 (25%) did not have an oophorectomy and we do not have data regarding oophorectomy status for 12 (2%). The guidance states that paclitaxel should be offered to women following surgery; and only to other women with ovarian cancer within an RCT. Of the 379 women who had surgery, 126 (33%, or 24% of all women) received paclitaxel of whom 11 had a second course of paclitaxel. Of the 11 women who had a 2nd course of paclitaxel, only 2 received it as part of an RCT. Of the 253 women who did not receive paclitaxel, 107 received alternative chemotherapy regimens, and 136 did not appear to have received any chemotherapy. Of the 129 women who did not have surgery, 25 (19%, or 5% of all women) were offered paclitaxel of whom 3 had a second course of paclitaxel in conjunction with cisplatin or carboplatin, and 4 received paclitaxel alone in a second course. None of the 25 women received their paclitaxel within an RCT. Of those not offered paclitaxel, 37 were offered alternative chemotherapy regimens. We analysed data about women who had or had not undergone oophorectomy as 2 separate groups, since the guidance advises on the use of paclitaxel in combination with a platinum agent following surgery. Treatments given varied considerably across trusts. Table 5.2 presents the treatment regimens given, by individual Trust. The percentage of women who received paclitaxel in combination with a platinum agent following surgery, in accordance with the NICE guidance, also varied across trusts. Analysing the receipt of paclitaxel in combination a platinum agent as the first course of treatment, Table 5.2 shows that the percentage of women receiving the treatment appeared to be less than 20% in three trusts and only just over 50% in a further three trusts. The average across all the trusts that completed the audit, was 33%. The NICE guidance also recommends that women should not receive a second course of treatment with paclitaxel in combination with a platinum agent except within an RCT. Eleven women in two trusts received paclitaxel in combination with a platinum agent in the first and second courses of treatment, and 2 of these cases were in an RCT. We examined the treatment regimens offered to women following surgery across all trusts. This analysis shows that 29% of women received carboplatin or cisplatin, and that for 36% of women there was no record of chemotherapy having been given at all. The remainder received standard chemotherapy. The analysis further shows that there is great variability in the numbers and types of treatment regimen offered, particularly in the number of women offered paclitaxel in combination with carboplatin or cisplatin, or no chemotherapy following surgery. One trust offered carboplatin or cisplatin to 70% of the women following surgery, and a second trust offered one of these drugs to 50% of women. Two trusts appeared to offer no chemotherapy to over 70% of women following surgery, and in a further two trusts, to around 50% of the women. We explored the reasons for the variability in the follow-up interviews.

44

Table 5.2: Trust ID

Number (%) who had oophorectomy

No

Trust 1 Trust 2 Trust 3 Trust 4 Trust 5 Trust 6 Trust 7 Trust 8 Trust 9 Trust 10 Trust 11 Trust 12 Trust 13 Trust 14 Trust 15 Trust 16 Trust 17 Trust 18 Trust 19 Trust 20 Total

5.8

Analysis of patient treatment

DNP 32 49 35 21 6 DNP 17 29 22 19 16 28 DNP 10 DNP 29 23 26 17 379

Number (%) receiving paclitaxel in combination as 2 nd course of chemotherapy post surgery No % of those with surgery

Number (%) receiving paclitaxel in combination as 1 st and 2 nd courses of chemotherapy post surgery No % of those with surgery

Number (%) receiving paclitaxel in combination as 1 st & 2 nd courses of chemotherapy post surgery in RCT

% all women by trust

Number (%) receiving paclitaxel in combination as 1 st course of chemotherapy post surgery No % of those with surgery

No

% of those with surgery

65% 98% 81% 70% 20%

11 26 10 8 0

34% 53% 29% 38% 0%

0 8 0 0 0

0% 16% 0% 0% 0%

0 8 0 0 0

0% 16% 0% 0% 0%

0 2 0 0 0

0% 4% 0% 0% 0%

94% 97% 67% 63% 76% 56%

2 0 16 2 6 15

12% 0% 73% 11% 38% 54%

0 0 0 0 0 0

0% 0% 0% 0% 0% 0%

0 0 0 0 0 0

0% 0% 0% 0% 0% 0%

0 0 0 0 0 0

0% 0% 0% 0% 0% 0%

83%

3

30%

0

0%

0

0%

0

0%

94% 58% 90% 71% 73%

10 9 6 2 126

35% 39% 23% 12% 33%

1 3 1 0 13

3% 13% 4% 0% 3%

0 3 0 0 11

0% 13% 0% 0% 3%

0 0 0 0 2

0% 0% 0% 0% 4%

Evidence from the audit of and survey for taxanes for non small cell lung cancer

We did not undertake an audit of patients who had been treated for non-small cell lung cancer because the guidance was insufficiently precise for us to draw up a clear audit questionnaire. Nor did we include taxanes for non-small cell lung cancer in the survey.

5.9

Evidence from the survey

We sent guidance specific questionnaires on the use of taxanes in breast and ovarian cancer to the lead clinicians that in the 20 acute trusts which participated in the audit phase. We received 10 completed forms for breast cancer, a response rate of 50%, and 12 completed forms in respect of ovarian cancer, a response rate of 60%. For breast cancer, nine (90%) of respondents stated that their trust had fully implemented the guidance, and one respondent stated that the guidance had not been implemented. For ovarian cancer, eleven (92%) of the respondents stated that their trust had fully implemented the guidance, and one respondent stated that the guidance had not been implemented.

45

The factors perceived to contribute to the successful implementation of this guidance are shown in Table 5.3. Table 5.3 Factors perceived by respondent to contribute to the successful implementation of NICE Guidance within a trust Factors

Clinical lead or local champion High local priority High level of evidence or recommendation Single specialty service Agreed in local development plan Other Low cost Total citations Multiple factors could be cited

Taxanes for Breast cancer No of times % of citations cited* 8 31% 6 23% 6 23%

Taxanes for Ovarian cancer No of times % of cited* citations 7 29% 5 21% 4 17%

3 2 1 0 26

2 3 2 1 24

12% 8% 4% 0%

8% 13% 8% 4%

The ‘other’ factor for breast cancer was that the Trust’s Medicines Management Group supported a business case for the additional funding required to implement the guidance. One of the ‘other’ factors for ovarian cancer stated that the procedures were already considered part of normal policy. For both breast and ovarian cancers the principle factors cited as contributing to the successful implementation were having a clinical lead or champion, being a local priority and having a strong evidence base. The one trust that did not implement either of the sets of guidance stated that all chemotherapy is managed within another trust; therefore it had no role in implementing this guidance. Five of the ten respondents stated that there were cost consequences for the trust in implementing the guidance on breast cancer. Two respondents said that equipment was a factor, one trust said that recruitment was a factor, whilst six respondents gave drug costs as a reason. Nine respondents said that the guidance was high on the list of local priorities and one respondent stated that this guidance displaced plans for other service developments, although no specific example was given. Four respondents out of 8 stated that there were significant cost consequences for the trust, implementing taxanes for ovarian cancer and two stated that they didn’t know. Of the cost consequences, two stated that facilities were a factor, one stated equipment, one stated staff training and two stated ‘other’, comprising drug and staffing costs. Seven responded that the ovarian cancer guidance was high on the list of local priorities, one respondent said it was not on the list of local priorities because all patients go to a specialist hospital. Two respondents stated that the guidance had displaced plans for other service developments, but gave no specific examples . Four respondents provided additional comments about the role of NICE in the context of guidance on breast cancer, two of which are given below. The remaining two comments merely indicated that progress in implementation was slow. •

•

“Excellent links with purchasers, but difficulty with new drugs that haven’t been “NICE’d” is a problem” Taxanes were introduced prior to NICE guidance through clinical trials and directorate management of the drug budget”

46

Six trusts commented on the role and methodology of NICE in the context of guidance on ovarian cancer. One supported the NICE process, although another felt that it should be quicker.

5.10 Evidence from the interviews on taxanes for breast and ovarian cancer 5.10.1 Introduction A total of four respondents (3 oncologists and 1 surgeon) agreed to be interviewed. The surgeon worked in a Trust that was not a cancer centre. In all cases the questions related to guidance superseded in January 2003 with advice that “either paclitaxel in combination with platinum based therapy, or platinum based therapy alone, is offered as first line treatment for women with ovarian cancer.” Another oncologist was not in post in the trust at the time of publication of the guidance and so his evidence was hearsay.

5.10.2 Reaction to the guidance The three oncologists reported some reservations about the guidance on ovarian cancer, believing that it overstated the evidential case for the effectiveness of taxanes, whilst recognising that it freed up funding in some parts of the country, giving clinicians choice. For one, specialising in ovarian cancer treatment, the guidance was just one in an array of conflicting recommendations at the time including the outcome of the International Collaborative Ovarian Neoplasm (ICON3), the recommendations of the Gynaecologic Oncology Group (GOG) and the American Society of Clinical Oncology (ASCO). In fact this interviewee’s treatment plan included taxanes as first line treatment in advanced disease, although for some of the interview s/he clearly believed s/he was in breach of the guidelines . One reported that colleagues had had concerns about the restriction of the use of taxanes in breast cancer for the adjuvant treatment of high-risk women.

5.10.3 Implementation In one case, the funding for taxanes was given to the Cancer Centre. It was stated that this precluded the use of the taxanes in clinics held in the Trust that participated in the audit, so that patients were given the choice of travelling thirty miles (a one hour journey by car largely on B roads) to receive the drug, or receiving other cytotoxic drugs in their “home” trust.

5.10.4 Feedback on the audit Two respondents took issue with the findings of our audit. One referred to a database recording the treatment of patients in a later six-month period showed that all but one of sixteen patients had been given paclitaxel and carboplatin. In the other case, an oncologist in a cancer centre expanded concern that the audit figures showed far too few patients having taxanes for both breast and ovarian cancer and confirmed that a new set of patient notes would be kept in the centre. Neither of these respondents had known about the conduct of the audit or had been involved in any way. The third oncologist thought that the figures were probably correct and the surgeon was unable to comment.

47

5.11 Summary of findings • • •

•

•

There was a statistically significant increase in use of docetaxel and paclitaxel following publication of the NICE Guidance The audit revealed little evidence of taxanes being inappropriately used in early breast cancer There were significant variations in prescribing; the audit suggested that the percentage of women receiving paclitaxel for ovarian cancer was lower than 20% in three trusts and was only just over 50% in a further three trusts. The average proportion of women with ovarian cancer who were prescribed paclitaxel across all trusts audited was 33%. Two out of the three oncologists interviewed disagreed with the findings from the audit, since they had not been involved in the audit. Notes in Trusts that are not cancer centres may not be good sources of evidence on cancer treatment.

48

6.

Availability of Hearing Aids

6.1

Introduction

The guidance was issued in July 2000 and advised that: • • •

•

A full range of analogue hearing aids in the current NHS range should be available at all NHS audiology centres; Patient choice should be a factor in offering suitable hearing devices and binaural fitting should be available; The NHS PASA should review the range of aids; There is insufficient evidence to support the nationwide introduction of digital hearing aids outside the pilots then being evaluated.

Our aim was to investigate whether and how NHS audiology services had changed since the NICE guidance was published. Our primary research question was whether the range of analogue hearing aids prescribed by NHS audiology had increased since July 2000. We also aimed to determine whether rates of binaural aid fitting had increased. The period for our review of the impact of the guidance was from the release in July 2000 and up to 2003, prior to its withdrawal in May 2003. The NICE guidance on hearing aids was introduced against a backdrop of problems for NHS audiology services. These can be summarised as: •

•

•

Long waiting lists and waiting times for patients waiting for assessment and fitting. Audiology waiting lists are not included in the overall NHS outpatient waiting lists and, therefore, have not benefited from the extra funding associated with this initiative; Most audiology departments only offer a limited range of aids, and around one-third of hearing aids prescribed are not used because they do not meet the needs of the patient. Patient follow-up after the fitting of an aid is often not satisfactory; The introduction of digital technology has been proceeding slowly with a phased rollout following piloting. Financing of digital aids has been identified as an issue along with a shortage of trained staff;

For these and other reasons, many patients choose not to use NHS audiology services, but to purchase their hearing aids privately from specialist suppliers, who may advertise widely and assertively, and whose hearing aids may be high cost.

6.2

Evidence from national and local data sources

6.2.1 Data availability There are no routinely available sources of information on the availability of different types of hearing devices. Therefore, we sent a questionnaire to 228 audiology departments in England and Wales. The questionnaire was reviewed by researchers based at the MRC Institute of Hearing Research at the University of Nottingham and was piloted and assessed by a small group of audiologists in the NHS in advance of the main study. A copy of the questionnaire is attached in Appendix 1.4 114 completed questionnaires from a total of 228 were returned, giving a response rate of 50%. We explored:

49

• • •

The range of hearing aids across departments at the points in time; Whether a greater number of departments were offering a greater range of hearing aids after the issue of the NICE guidance; What factors were perceived to have contributed to any changes in the availability of hearing aids.

6.2.2 Findings There was an increase in the number of departments offering a choice of fewer than 5 analogue hearing aids from the NHS range (from 11 to 18) and a reduction in the number offering more than 21 aids (from 46 to 40), indicating that the choice of analogue hearing aids available had reduced rather than increased. The number offering other ranges either remained unchanged or fell slightly. 32% of the total respondents reported being a pilot site for the supply of digital aids. Of the 34 aids recommended by the NHS PASA only 11 saw an increase in availability, whereas 21 saw a reduction in their availability. There are indications that departments may have been expanding their range of digital aids, whilst reducing their range of analogue aids from the NHS PASA approved list. Finally, when asked if the range of aids supplied to patients changed as a result of the NICE guidance, only 23% stated that it had. Audiology departments responded favourably to the recommendation from NICE with regard to offering patients with hearing loss in both ears two hearing aids. Prior to NICE guidance only 39% routinely offered two aids to relevant patients, by December 2000 (six months post NICE guidance) this percentage had risen by 8% to 47% and rose again by a further 29% to 67% of departments routinely offering two aids to patients who required them at the time of our data collection. A question designed to determine the factors influencing the range of hearing aids offered revealed that the main driver is financial, with ‘professional standards and guidance’ following closely. 77% of respondents reported that that they had made a request for funding, but only 26% of those actually received any money. NICE was, on average, deemed to be the least important determining factor.

6.3

Evidence from Audit

6.3.1 Sample The sample for the audit of the availability of hearing aids was drawn from two periods respectively six and twelve months after the release of the guidance as follows: st

“ first twenty-five patients prescribed a hearing aids on or after 1 January 2001 and first st twenty-five prescribed a hearing aid after 1 July 2001.”

17 Trusts returned 50 forms and the 18th returned only 25.

50

6.3.2 Questions Six questions particularly addressed the range of hearing aids, patient choice and availability of binaural fitting. Question 4 Question 7

Question 8 Questions 10 Questions 11 Questions 12

Does the patient have impairment in one ear or both? Left/Right/Both/Don’t know Has the patient been prescribed hearing aids for: Right ear only (monoaural) Left ear only (monoaural) Both ears (binaural) Don’t know Make and/or model of hearing aid prescribed if specified Are any reasons recorded for choice of type of hearing aid (e.g. patient choice/ease of use)? (If aid was offered but refused by patient, please also record this here) Was the patient provided with information and/or advice Yes/No/Don’t know Was the patient given a follow-up audiology appointment Yes/No/Don’t know

6.3.3 Analysis of Results Trusts were asked to record the type of hearing aid prescribed for each patient. Of the 17 trusts that provided completed audit forms for the dates in question, 10 appeared to have increased the number of hearing aids available, 6 had reduced the number, whilst for one the number has remained the same. Because of the relatively small sample size, we would not expect to sample the full range of hearing aids available. The average number of aids available by trust, according to the patient notes audit, did not materially change (7 and 6.5 per trust). It was evident that two brands of hearing aid accounted for 51% of the hearing aids prescribed; 5 brands accounted for a total of 71% of the aids prescribed and a further 38 manufacturers account for the remaining 29% of aids. Twenty-one brands were prescribed on only a single occasion. Two questions focussed on the provision of binaural hearing aids. The expectation was that the hearing aids prescribed would depend upon the outcome of the patient assessment, as described in Question 4. The majority of trusts assessed more than 90% patients as having impairments in both ears, although one trust identified only 6% of patients as having bilateral impairment, and one trust in only 32% of cases. Four trusts diagnosed bilateral problems for between 80-89% of patients. Overall only 13% of people assessed as having a hearing deficit in both ears were prescribed binaural hearing aids. There was, however, significant variation within this figure. In one trust, no patient received two hearing aids, whilst in another, 56% of patients received binaural aids (9 patients). In seven trusts, the percentage of patients prescribed binaural aids was less than 10%, and 10% - 19% in a further five It was not possible, from the patient level audit, to determine the reasons for hearing aid choices in many trusts, and unfortunately this was the case for many of the trusts

51

with the lowest rates of binaural provision. Analysing the information at a summary level, the most common reasons provided for choice of hearing aid include: • • • • •

•

Mild loss in a single ear; Patient only wanting a hearing aid in one ear; Patient confused or unable to cope with two aids; Use of National Acoustic Laboratories NAL-NL1 prescription formula (the reason given for every patient in one Trust); Patient already has a prescribed hearing aid in the other ear; Clinical suitability or clinical judgement (the reason given primarily in another trust).

The NICE guidance also indicates that there is currently a significant under provision of patient education and follow-up. All trusts recorded that they gave information and advice, except for three trusts for which a large number of don’t knows was recorded; rising from one trust with 22% to 98% in another trust. These differences may reflect audit practice, rather than audiology practice. The audit showed that most trusts offered follow-up appointments to patients, however, two did not; one trust offered follow-ups to only 24%, and two trusts offered follow-ups to 60% and 64% of patients respectively.

6.4


We sent guidance specific questionnaires to the lead clinicians in the 20 acute trusts which participated in the audit phase. We received 11 completed forms in respect of hearing aid technology, a response rate of 55%. Four (36%) of the respondents stated that their trust had fully implemented the guidance, and seven respondents (64%) stated that the guidance had been partially implemented. The factors perceived to contribute to the successful implementation of this guidance are shown in Table 6.1: Table 6.1 Factors perceived as contributing to the successful implementation of the Hearing Aid NICE Guidance within Trusts Factors Agreed in local development plan Other Clinical lead or local champion Single specialty service High local priority Low cost High level of evidence or recommendation Total citations NB Multiple factors could be cited


% of citations 33% 33% 17% 17% 0.0% 0.0% 0.0%

Both respondents giving ‘other’ as the reason qualified this by stating that the guidance had been accepted practice in their trusts prior to the Guidance. The main reasons for the lack of implementation of this piece of Guidance within a trust are given in Table 6.2.

52

Table 6.2

Main factors perceived as contributing to the lack of implementation of NICE Hearing Aid Guidance within Trusts

Factors

No of times cited*

% of citations

Extra investment and/or resources required Other Not agreed in HIMP, local development plan or through the SAFF round Implementation intended but has not yet been carried out Not regarded as a priority Too complex or involves other organisations Low level of evidence Lack of clinical leadership or local champion Total citations NB Multiple factors could be cited

7 4 3

39% 22% 17%

2 1 1 0 0 18

11% 6% 6% 0.0% 0.0%

Three of the four respondents who cited ’other’ described problems with funding at their trust as the main reason. The fourth respondent explained that their trust had been accepted into the NHS digital hearing aid programme. Eight (72.7%) of the respondents stated that there had been a significant cost consequence associated with this guidance, the most frequently cited cost being equipment, as shown in Table 6.3: Table 6.3 Cost consequences associated with the implementation of the NICE Hearing Aid Guidance in a Trust Factors

No of times cited* 6 3 4 2 2 17

Equipment Facilities Other Staff training Recruitment Total citations NB Multiple factors could be cited

% of citations 35% 18% 23% 12% 12%

Three of the respondents, which gave ‘other’ as a reason, qualified this by stating that the problem was the cost of hearing aids. Five of the respondents stated that the guidance was high on the list of local priorities, whilst 4 stated that it was not. Other developments taking priority included trust waiting times, provision of digital hearing aids and improved critical care and A&E facilities. No trusts stated that the guidance had displaced other service priorities.

6.5


6.5.1 Introduction Interviews were undertaken with lead clinicians in four trusts. In all cases the guidance had been positively received, with respondents reporting relief that the known service shortfalls had been recognised. In each case the audiology service had undertaken an audit of its work against the standards set out in the guidance and

53

all apart from one found a shortfall, the extent of which varied. One very large Trust claimed that it did not meet any of the standards. Audiology departments differed in their perception of the adequacy of the funding available to provide a wide choice of hearing aids. However, all respondents, regardless of their baseline funding position, argued that whilst they might occasionally have to fit an unusual aid for a patient with a particular hearing configuration, they tended to use a small number of aids based on their experience of patient acceptability, reliability and ease of use. The baseline picture regarding the prescribing of binaural aids also varied. All acknowledged that age-related deafness tends to be bilateral, and in terms of NICE guidance would be expected to require two aids. One Trust always offered two, two Trusts fitted one but invited patients to return for a second and the fourth asked patients who wished a second aid to return to their GPs and be re-referred. Two pointed out that patients tended not to be comfortable with binaural aids for the following reasons: • • •

Denial of the extent of hearing problem Stigma associated with the wearing of an aid. Feeling of alienation engendered by binaural aids

In two Trusts it was said that people requiring good directional sound in their daily lives, such as teachers, or mothers of young children would always be encouraged to have binaural aids.

6.5.2 Action taken On receipt of the guidance, Trusts in which a service shortfall was identified instigated meetings with trust managers and produced a business case for investment to comply with the guidance, the cost varying from £50,000 to £800,000. Meetings were instigated with Health Authorities, and later, PCTs, but the response to requests for further funding in each case was negative. One PCT in the area with the worst service agreed to fund the developments over 4 years, costing more than £250,000 per year. This funding ceased when the NICE guidance was withdrawn in May 2003. In all cases but this last, the situation had been redeemed by the hypothecated funding associated with the Modernising Hearing Aid Services (MHAS) initiative.

6.5.3 Internal audit of NICE guidance The NICE guidance recommended that audiology departments undertake an audit. All Departments had complied with this aspect of the guidance. One Trust reported auditing their service on a regular basis, considering waiting times, the types of hearing aids fitted and the proportion of binaural to monaural aids. None reported a formal audit by the Trust audit department, other than that undertaken as part of the present study.

54

6.5.4 Comments on the guidance Two respondents reported that the guidance had been insufficiently prescriptive, and one that it failed to consider the patient experience, but all said that they would have been positive about the guidance had funding been available.

6.5.5 Comments on the audit It was suggested that in some Trusts the fitting appointment was viewed as a follow up, and so this question may have been subject to variable interpretation.

6.6 • •

• •

• • •

Overall conclusions The NICE Guidance on Hearing Aids appears to have been enthusiastically received by audiology departments; all of those surveyed had undertaken an immediate audit of their service against the NICE Guidance requirements; Funding appeared to be a major impediment to implementation in those Trusts that were not already compliant. It appeared that the publication of NICE Guidance did not, in this instance, raise the profile of audiology services to the extent that this was became high priorities with commissioners; The range of analogue hearing aids offered by trust audiology departments does not appear to have been extended as a result of the NICE guidance. The number of aids issued tended to be fewer than the number available. Two of the models of hearing aid accounted for 51% of the hearing aids prescribed, and five accounted for 71%. The audit data revealed that two aids featured in 51% of fittings and the interviews suggested that patient experience, reliability and ease of use were key features. Binaural fitting is not widely available, even when patients are assessed as having impairment in both ears. Whilst funding was perceived as an issue in some Trusts, others reported low take up by patients, of binaural aids. There is variation in the provision of routine follow-up appointments. No Trust with a perceived funding shortfall reported that it had received the funding to meet the requirements of the NICE guidance in full.

55

7.

Implantable Cardioverter Defibrillators (ICDs) for Arrhythmias – Results

7.1

Introduction

The guidance was issued in March 2000, and advised that: •

Implantable cardioverter defibrillators (ICDs) should be routinely considered for patients with given pathologies and/or previous histories of myocardial infarction, with the objective of either primary or secondary prevention

ICDs were first implanted in 1980 and during the 1990s; seven trials were published with largely favourable results.3

7.2

Data Sources

7.2.1 Introduction We were unable to use data from HES because the OPCS codes for implantation of cardiac pacemakers cover a range of pacemaker implantations and do not allow the differentiation of ICD implantation. •

•

K601 Cardiac pacemaker system introduced through vein – implantation of intravenous cardiac pacemaker system K611 Other cardiac pacemaker system – implantation of cardiac pacemaker system nec

We were fortunate therefore, to be able to use data provided by the British Pacing and Electrophysiology Group (BPEG) Register. Data are submitted to the register by hospital clinicians. The data covered the period from the first quarter of 1995 to the last quarter of 2001. The number of centres on the Register was 33 in 1995, rising to 58 in 2001. We have no reason to believe that completeness of the database has changed over time, and BPEG, who have sent direct queries to manufacturers and to hospitals to ascertain completeness, believe that the register represents over 95% of ICD activity. Comparisons in two postal questionnaires to hospitals also showed close agreement.

7.2.2 Descriptive analysis of data Figure 7.1 shows the growth in the total number of ICDs implanted since 1995. Overall, ICD implantation rates were growing steadily from 1995 and there was an increased rate of growth in 1998. Growth slowed thereafter, although the number of centres increased by a further 18 by 2001, after the release of the Guidance.

3

See HTA Appraisal Documents, NICE 2000 at http://www.nice.org.uk/pdf/HTSICDs.pdf

56

FIGURE 7.1 BY BPEG)

TOTAL NUMBER OF ICDS IMPLANTED (DATA SUPPLIED

Total Number of ICDs Implanted by Quarter 350 300 250 NICE Guidance September 2000

200

Quarter Total 150 100 50

Figure 7.2

20 01 .3

20 01 .1

20 00 .3

20 00 .1

19 99 .3

19 99 .1

19 98 .3

19 98 .1

19 97 .3

19 97 .1

19 96 .3

19 96 .1

19 95 .3

19 95 .1

0

Average Number of ICDs Implanted by Years of Experience

Annual Average Implants by Years of Experience of Centres

50 45 40 35

7 Years 6 Years

30

5 Years 25

4 Years 3 Years

20

2 Years 1 Year

15 10 5 0 1995

1996

1997

1998

.

57

1999

2000

2001

7.2.3 Statistical Analysis There is no evidence of a statistically significant change in either the time trend or constant parameters post publication of NICE guidance. However, given that we only have data for six quarters post guidance, there is low power to detect a change as statistically significant. Nonetheless visual inspection does not indicate any structural break. The publication of the Guidance may also have taken time to alter behaviour.

7.2.4 Discussion Although there has been a growth in the number of ICDs implanted, the numbers implanted were still (at 1,234 in 2001) someway below the national figure anticipated by NICE of 1,800. The lower figure may reflect: • The high costs of ICDs, at around £20,000 per device • Patient selection, which may be conservative if budgets are tight • Scarcity of skilled clinicians to undertake the implantation, although it has been suggested by clinicians that the extra skills required are not great, given the similarities between implanting conventional pacemakers and ICDs • The original NICE appraisal may have over-estimated the likely number of eligible candidates It is noteworthy that when examining implantation rates by centre (Figure 7.2) one can see that those centres that were early adopters and began implanting ICDs in 1995 have been the main implant centres for the whole of the intervening 7 years and remain so today: • These 33 centres undertook 90% of implants in 2001 whilst only comprising 57% of implant centres • These centres were implanting an average of 8 ICDs per annum in 1995, rising to an average of only 33 per centre in 2001 • The 25 non-market leaders only increased activity levels to an average of 7 per annum in 2001, with the exception of one centre, which carried out no implants in 2000, but 46 in 2001. It is unclear whether the increase in activity was as a direct result of the issuing of the guidance • There may also be differences in compliance rates with the BPEG audit but this was not testable as data were released anonymously.

7.3

EVIDENCE FROM AUDIT

The guidance on ICDs was not included in the patient notes audit because the small number of centres would have compromised anonymity.

7.4

Conclusions

NICE anticipated that around 1,800 ICDs would be implanted per annum in the NHS in England and Wales, equating to 10 per centre per annum; a 50% increase on current rates. It seems unlikely that physical capacity or skill levels are restricting implantation rates, and in our assessment, it is more likely to be funding constraints with some impact due to other developments draining support from ICDs. Finally, we do not know the completeness of the BPEG Register of ICDs and so we may have missed some growth in use. This would only pose major problems if completeness changed over time.

58

8.

Zanamivir - Results

8.1

Introduction

This NICE guidance was issued in November 2000, and advised that: Zanamivir should only be prescribed for at-risk adults, who present within 36 hours of the onset of influenza like illness (ILI), when influenza is circulating in the community. At-risk adults are defined as individuals falling into one of more of the following categories: • Age 65 years and over • Chronic respiratory disease (including chronic obstructive pulmonary disease and asthma) requiring regular medication • Significant cardiovascular disease (excluding individuals with hypertension) • Immunocompromised • Diabetes mellitus

8.2

Data Sources

Data have been obtained from: • •

Prescription Pricing Authority (PPA) reports up to and including March 2003. Because the levels of prescribing were very low, we did not undertake a more detailed analysis using PACT data; National Surveillance of Influenza Activity from the Health Protection Agency, which reports on the levels of flu in the community.

8.3

Results

Table 8.1, taken from the PPA Reports of January and March 2003, shows that prescribing levels have not only been very low, but that the number of items prescribed has fallen since 2001. Table 8.1:

Prescribing levels and costs of zanamivir

Year to date to September 2001 Items Cost £12960 499

8.4

Year to date to March 2002 Items

Cost

Year to date to March 2003 Items Cost

190

£5208

124

£3719

Discussion and Conclusions

There are several possible explanations for the low rates of zanamavir prescribing : a)

No major influenza outbreaks since the guidance was issued:

The HPA national surveillance activity levels show that the 2000/01, 2001/02 and 2002/03 influenza seasons were characterised by low levels of activity associated with influenza A and B throughout the whole UK. Figure 8.1 shows activity levels over the last 15 years.

59

Figure 8.1

Weekly consultation rates for influenza and influenza like illness: RCGP Weekly Returns Service, 1988-2003

From HPA National Surveillance of Influenza Activity4 Using the RCGP sentinel surveillance scheme, weekly GP consultations for influenza and ‘influenza-like illness’ remained below the baseline (defined as less than 50 new episodes per 100,000 population) until week five in 2001 and peaked at 81/100,000 in week six. This rate remained at the lower end of the range for ‘normal seasonal activity’ of 50-200/100,000 population until it declined rapidly and returned to baseline levels in week ten. A similar picture is described for consultation rates for those with acute bronchitis and GP consultation rates for total respiratory disease remained lower than the levels seen during 1999/00 with a peak rate of 939/100,000 compared with 1118/100,000 in 99/00. For the 2001/02 and 2002/03 seasons, consultation rates with GPs for ILI remained similar to the rate of baseline activity, defined as 50/100,000 population throughout the two whole seasons for the first time. GP consultation rates for acute bronchitis peaked in February 2002 with a rate of 236/100,000, which was similar to the peak seen during the 2000/01 season. b)

Constraints on Prescribing:

The experience from the US indicates that zanamivir has had very low take up due to its approval only for treatment, and not prophylaxis. Also, being in inhaler form is potentially more difficult to administer for those patients unused to drugs in this form. GPs were also not too supportive of the guidance, partly because of implications for workload; patients with flu symptoms are generally not encouraged to seek medical attention.

8.5

Comment

In the absence of significant influenza outbreaks, we cannot comment on the impact of the guidance. We note that amended guidance was issued by NICE in February 2003, which included two additional drugs. However, audit of this guidance was beyond the scope of this project.

4

Taken for website: www:hpa.org.uk/infections/topics_az/influenza/graphmenu0203.htm

60

9.

Use of laparoscopic surgery for the treatment of colorectal cancer

9.1

Introduction

The guidance was published in December 2000 and advised that: • •

For colorectal cancer, open rather than laparoscopic resection should be the preferred surgical procedure Laparoscopic surgery should only be undertaken for colorectal cancer as part of a randomised controlled trial

9.2


9.2.1 Data sources HES data were used to identify patients diagnosed with cancer and were analysed for the years January 1992 to December 2001, for finished consultant episodes with ICD codes of: • • • •

C18 Malignant neoplasm of colon C19 Malignant neoplasm of rectosigmoid junction C20 Malignant neoplasm of rectum C21 Malignant neoplasm of anus and anal canal

With OPCS code which identified the laparoscopic approach of • Y508 Approach through abdominal cavity – other specified

9.2.2 Descriptive analysis of data Table 9.1 shows that the number of cases treated with laparoscopic surgery rose slightly from a total of 163 in 1998 to 189 in 2001, and that the percentage of cases of colorectal cancer having laparoscopic surgery remained stable at around 0.10%. Table 9.1

England

Wales

Total

Cases of colorectal cancer treated with laparoscopic surgery

No of cases No with Lap surgery % with lap surgery No of cases No with Lap surgery % with lap surgery % with lap surgery

1998

1999

2000

2001

Total

121,159 137 0.11% 12,066 26 0.2% 0.12%

171,355 155 0.09% 14,300 21 0.1% 0.09%

183,500 149 0.08% 15,751 17 0.1% 0.08%

181,149 165 0.09% 14,954 24 0.2% 0.10%

657,163 606 0.09% 57,071 88 0.2% 0.10%

Figure 9.1 shows the number of laparoscopic operations as a percentage of all patients diagnosed with colorectal cancer, on a monthly basis.

61

Figure 9.1 Percentage of colorectal cancer patients having laparoscopic surgery England and Wales Colorectal Cancer operations with Y508 code as a percentage of all patients dignosed with colorectal cancer 0.18% 0.16% 0.14% 0.12% Nice Guidance December 2000 %

0.10% E&W Y508 as % E&W Tot 0.08% 0.06% 0.04% 0.02%

Ap r-9 8 Ju n-9 8 Au g-9 8 Oc t-9 8 De c-9 8 Fe b-9 9 Ap r-9 9 Ju n-9 9 Au g-9 9 Oc t-9 9 De c-9 9 Fe b-0 0 Ap r-0 0 Ju n-0 0 Au g-0 0 Oc t-00 De c-0 0 Fe b-0 1 Ap r-0 1 Ju n-0 1 Au g-0 1 Oc t-0 1 De c-0 1

0.00%

Month and Year

9.2.3 Statistical analysis We found no evidence of changes in the numbers of laparoscopic colorectal cancer operations, which continued to be low.

9.2.4 Caveats on the data analysis It is unclear whether coding of the laparoscopic approach is accurate in HES data. This suspicion is compounded by the apparent lack of patients coded in HES who are understood to have been treated at centres participating in the MRC funded CLASICC trial. However, the total number of patients over 2000-2002, with coding for laparoscopic surgery is in line with the total number of patients recruited into experimental arm of the trial.

9.3

Evidence from the audit

The guidance was unequivocal in its recommendations with no indications for appropriate use, therefore, we did not undertake an audit of patient notes. Also the use of the technology is rare and would have required a much larger sample of notes to estimate compliance rates. Although we did not seek to interview clinicians on this topic (because it was not included in our audit schedule), one respondent volunteered the view that the advice on colorectal cancer was contrary to common surgical opinion and also that it had very significant implications for training. This respondent claimed that Trust figures demonstrated superior results for a right hemi-colectomy conducted laparoscopically in respect of both the length of stay and complication rates.

62

10.

Use of Laparoscopic Surgery for the Treatment of Inguinal Hernia - Results

10.1

Introduction

The Guidance was issued in January 2000 and stated that: •

Open (mesh) surgery should be the preferred method of repair for primary inguinal hernia; Laparoscopic repair should be considered for repair of recurrent and bilateral inguinal hernia; When laparoscopic surgery is undertaken for inguinal hernia, the totally extraperitoneal (TEP) procedure should be preferred; Laparoscopic surgery for inguinal hernia should only be undertaken by operating teams who regularly undertake these procedures.

• • •

10.2

Data Sources

10.2.1 Introduction We used the single data source of Hospital Episode Statistics (HES) Data for England and Wales covering the years 1992 through to the most recent available 2001/02 (quarters 1 to 3 only, inflated to give a whole year equivalent figure).

10.2.2 HES data analyses (for England and Wales) HES data for finished consultant episodes (FCEs) only were examined using the following Office of Population Censuses and Surveys 4th Edition (OPCS 4) operation codes: • •

•

T19 T20 T21

Simple incision of inguinal hernia sac; Primary repair of inguinal hernia; Repair of recurrent inguinal hernia.

We analysed data for the above codes when the following surgical code appeared as a sub code, which identified the laparoscopic approach to the hernia. Y508

Approach through abdominal cavity – Other specified

The focus of the work was to identify activity and location of activity. As the guidance recommends that the laparoscopic approach should not be used for the primary repair of inguinal hernias, we expected to find a decrease in activity where Y508 appeared with T19 or T20. For the repair of recurrent inguinal hernias the guidance recommends that laparoscopic surgery should be used, therefore, in analysing the data, we were looking for an increase in the activity of Y508 with T21.

10.2.3 Findings from HES data It was found that in England and Wales, 3.9% (England 4.1% and Wales 1.8%) of all inguinal hernia repairs were carried out using laparoscopic surgery. The data indicated that the number and percentage of repairs of recurrent hernias (T21) undertaken laparoscopically is increasing, although the number of people admitted with recurrent hernias is declining. The percentage of primary inguinal hernias

63

undertaken using laparoscopic surgery is declining, although the levels are still very small. There is also considerable variation across hospitals in the amount of laparoscopic surgery undertaken, both in absolute values and as a proportion of their hernia activity. Looking at the HES activity for primary repair of inguinal hernia (T20) for 2001 calendar year, which is not endorsed by the guidance: • • • •

2544 procedures were undertaken across England and Wales, representing 3.8% of the primary repairs 50% of the total activity is carried out by 49 (25%) trusts. The remaining 50% is carried out by 147 trusts; The number of laparoscopic procedures undertaken in a trust varies from a maximum of 221 to a minimum of none; The percentage of primary repair of inguinal hernia carried out, laparoscopically varies from a maximum of 50.8% (in a trust only carrying out 217 primary repairs of which 110 are coded as laparoscopic) to a minimum of 1.4% (in a trust undertaking 1074 primary repairs but only carrying out 15 using a laparoscopic procedure).

Looking at the HES activity for recurrent repair of inguinal hernia (T21) with laparoscopic surgery for 2001 calendar year, which is endorsed by the guidance: • • •

There are still small numbers of laparoscopic repairs, 285, representing 6.0% of recurrent repairs 18 trusts undertake 54% of the laparoscopic surgery. For these trusts, the maximum number undertaken is 25 and the minimum is 5. Most trusts undertook between 1 and 4; When looking at such small numbers, coding deficiencies may account for the variance rather than clinical practice.

10.2.4 Statistical Analysis We found no evidence of a change in the number of primary or recurrent inguinal hernia repairs performed using the laparoscopic technique around the publication of the guidance.

10.3 Findings from the audit 10.3.1 The Audit sample The sample specified in the audit exercise was defined as follows: “First fifty patients who had Laparoscopic Surgery for Repair of Primary or Recurrent Inguinal Hernia (OPCS4: Y508 with T19, T20 and T21) on or after 1 July 2001. If insufficient, make up to fifty using patients who had open (non-laparoscopic) Inguinal Hernia Repair (OPCS4: T19, T20 and T21 without Y508).”

The rationale for including patients who had undergone non-laparoscopic procedures was twofold – first to check whether the dearth of laparoscopic procedures was accurate, or attributable to coding practices, and secondly, to identify those cases, found in the analysis of the HES data, of recurrent or bilateral procedures where the guidance suggested a laparoscopic technique might be appropriate.

64

Nineteen Trusts returned completed forms, although five returned fewer than 50 forms. A number were rejected as their date of operation did not fall into the periods stated and thus the total number of patient notes included in the analysis was 938. Of these, however, 50 were returned from one Trust, which did not specify whether the operation was for primary or recurrent hernia. We do know, however, that the procedures performed were open, not laparoscopic and we also have data as to whether they were unilateral or bilateral so these figures have been included.

10.3.2 Questions Two questions on the form sought to establish whether the procedure had been undertaken laparoscopically and if so, which approach was utilised. Question 2 Question 5

Was the procedure undertaken via Laparoscope (i.e. described as Laparoscopic)? Surgical approach specified Trans -abdominal Pre-peritoneal (TAPP) Totally Extra-peritoneal (TEP) Not Specified Other (please describe)

10.3.3 Analysis of Results It was noted that, in 103 cases, a negative or non-response (no or don’t know) to the question: “Was the procedure undertaken via Laparoscope (i.e. described as Laparoscopic)?” Was coupled with a positive response to the question indicating whether a transabdominal pre-peritoneal (TAPP) or totally extra-peritoneal (TEP) approach had been adopted. In order to carry out the analysis, we interpreted a positive response to either as being a procedure carried out by the laparoscopic route. We expressed the results as a compliance rate as indicated in Table 10.1, where compliance is defined as the use of an open procedure for unilateral inguinal hernias . The data show that 6 of the Trusts (trusts 6, 8, 12, 16, 18 and 19) carried out no laparoscopic procedures for unilateral primary right or left inguinal hernia – i.e. were fully compliant with the guidance and that another 4 (Trusts 3, 9, 15 and 20) were more than 95% compliant. Eight of these 10 Trusts (Trust 3, 6, 8, 12, 16, 18, 19 and 20) had not carried out any laparoscopic procedures for bilateral or recurrent hernia either, suggesting that the guidance may not the sole factor influencing behaviour. We analysed the HES data for the twenty trusts that participated in the audit. The data for primary hernias are shown in Table 10.1 (column E). This shows that there are some important differences between the audit data and the % compliance calculated from HES data. This may reflect the small sample size of our audits and also the poor coding in many NHS hospitals where in some Trusts a significant proportion of laparoscopic repairs appear to have been coded as open repairs.

65

Table 10.1 Number and percentage of primary unilateral inguinal hernias carried out using laparoscopic surgery.

TRUST ID

A B C Primary unilateral Primary unilateral inguinal hernias inguinal hernias Total primary using using unilateral Laparoscopy open procedure hernias

Trust 1 Trust 2 Trust 3 Trust 4 Trust 5 Trust 6 Trust 7 Trust 8 Trust 9 Trust 10 Trust 11 Trust 12 Trust 13 Trust 14 Trust 15 Trust 16 Trust 17 Trust 18 Trust 19 Trust 20

D E % primary repairs % primary repairs carried out via carried out via open procedure open procedure (i.e. compliant (i.e. compliant with NICE with NICE guidance) guidance) From audit data HES data 66% 98% 0% 97% 97% 100% 66% 99% 0% 22% 100% 100% 0% 50% 100% 97% 98% 100% 0% 96% DNP 100% 100% 100% 13% 91% 3% 100% 97% 99% ND 100%

11 22 33 26 0 26 1 34 35 13 25 38 20 0 20 0 2 2 29 0 29 0 44 44 1 39 40 25 0 25 DNP DNP DNP 0 8 8 28 4 32 35 1 36 1 28 29 ND ND ND 2 14 16 88% 0 43 43 100% 0 44 44 100% 1 44 45 98% Total 193 352 545 65% DNP = Did not participate ND = Trust did not specify whether the operation was for primary or recurrent hernia Column D shows compliance with guidance

99% 88% 100% 100% 96%

10.4 Evidence from the Survey We sent guidance specific questionnaires to the lead clinicians in the 20 acute trusts which participated in the audit phase. We received 12 completed forms, a response rate of 60%. Five respondents stated that they had fully implemented the guidance, four respondents stated that the guidance had been partially implemented, and three respondents stated that the guidance had not been implemented at all. Factors perceived to contribute to the successful implementation of this guidance are shown in Table 10.2. Table 10.2

Factors perceived to contribute to the successful implementation the NICE Guidance on laparoscopic surgery for hernia repair within Trusts

Factors High level of evidence or recommendation Other

No of times cited* 4 2

66

Clinical lead or local champion Low cost Single specialty service High local priority Agreed in local development plan Total citations Multiple factors could be cited

1 1 1 0 0 9

Both respondents giving ‘other’ as the reason qualified this by stating that the guidance had already been accepted practice in their trusts. The main reasons perceived to be contributing to the lack of implementation of this piece of NICE Guidance within Trusts are given in Table 10.3:

Table 10.3

Main reasons for the lack of implementation of NICE Guidance on laparoscopic hernia repair

Factors

No of times cited*

Other Extra investment and/or resources required Low level of evidence Not regarded as a priority Lack of clinical leadership or local champion Implementation intended but has not yet been carried out Not agreed in HIMP, local development plan or through the SAFF round Too complex or involves other organisations Total citations Multiple factors could be cited

4 2 2 1 1 0 0 0 10

A wide variety of reasons were cited under ‘other’ reasons, including: • Existing practice in line with guidelines • Patient choice and preference • Procedure deemed too expensive with no great evidence of benefits • Not yet fully assessed inside the organisation • Technically difficult requiring the training of other surgeons Several of these may actually explain compliance, rather than lack of compliance with the first part of the guidance. Only four of the respondents stated that they believed there to have been a significant cost consequence. The summary of these cost consequences is given in Table 10.4: Table 10.4 Cost consequences for a Trust associated with the implementation of NICE Guidance on Laparoscopic hernia Repair Factors Staff training Other Equipment Facilities Recruitment Total citations Multiple factors could be cited


67

The two Trusts, which gave ‘other’ as a reason, qualified this by stating that the problem was the cost of the procedures. Ten respondents stated that this guidance was not high on the list of local priorities. Examples of other developments cited instead are: • Improving the efficiency of the existing open surgery • Other surgical advances • The purchase of laparoscopic equipment No respondents stated that the guidance had displaced plans for other service developments, and seven of the respondents explicitly stated that it had not.

10.5


10.5.1 Receipt of the guidance All three Trusts participating in the interviews had surgeons trained in the use of laparoscopes for this procedure, although one reported that the Trust had largely ceased undertaking laparoscopic repairs in advance of the guidance on the basis of the available evidence.

10.5.2 Action taken on receipt of the guidance Local data held in one centre of excellence, in which there were four surgeons who had carried out in excess of 10,000 such operations, had persuaded Trust managers and commissioners that the laparoscopic repair of all hernias should continue. Moreover, this group continued to use the trans-abdominal pre-peritoneal (TAPP) rather than the recommended totally extra-peritoneal approach (TEP), the latter being, in the view of the interviewee, a much more difficult procedure to learn. In the other Trust, despite reservations from the clinicians involved, the procedure was summarily discontinued in accordance with the guidance. Reservations about the guidance and the data on which it was based, were expressed as follows: •

• • • •

Operation times, recurrence and complication rates included data from inexperienced surgeons (Two respondents stated that the learning curve was of the order of 50 procedures after which the length of operation and complication rate showed an improvement on the open procedure) Costs considered by NICE included the use of disposable equipment. Macroeconomic advantages arising from patients’ faster return to work were not costed in; Given choice, patients choose laparoscopic procedures; Savings accruing from the simultaneous repair of occult hernias (encountered in up to 30% of cases) was not factored into the NICE costings.

Discussions in other Trusts with high rates of laparoscopic hernia repair suggested that senior management would not be keen to terminate laparoscopic surgery if this would result in the loss of a good local surgeon in a potentially hard to recruit area.

68

10.6 CONCLUSIONS • • • • •

There is no evidence of a change in the practice of hernia repair by laparoscopic methods as a result of the NICE Guidance, however, it is possible that the Guidance prevented an increase in its use for primary repair. During the period January 1998-December 2001, 244,350 primary repairs for inguinal hernia (T20) were carried out in England, of which 9904 (4%) were undertaken laparoscopically ; 50% of laparoscopic hernia repairs are carried out in 49 (25%) trusts. The remaining 50% is carried out by 147 trusts; 8 of the 10 Trusts that had not carried out any primary laparoscopic repair had not carried out laparoscopic procedures for bilateral or recurrent hernia either; Some surgeons challenged the basis of the NICE guidance with respect to both the complications and costs when used by experienced practitioners.

69

11. Donepezil, Rivastigimine and Galantamine for Alzheimers Disease - Results 11.1 Introduction The Guidance was issued in January 2001, and advised that: The three drugs donepezil, rivastigmine and galantamine should be made available in the NHS as one component of the management of those people with mild and moderate Alzheimer's disease (AD) whose mini mental state examination (MMSE) score is above 12 points 5 (see paragraph 2.6 and 4.3) under the following conditions: 1.1.1 Diagnosis that the form of dementia is AD must be made in a specialist clinic according to standard diagnostic criteria. 1.1.2 Assessment in a specialist clinic, including tests of cognitive, global and behavioural functioning and of activities of daily living, should be made before the drug is prescribed. 1.1.3 Clinicians should also exercise judgement about the likelihood of compliance; in general, a carer or care-worker who is in sufficient contact with the patient to ensure compliance should be a minimum requirement. 1.1.4 Only specialists (including Old Age Psychiatrists, Neurologists, and Care of the Elderly Physicians) should initiate treatment. Carers' views of the patient's condition at baseline and follow-up should be sought. If General Practitioners are to take over prescribing, it is recommended that they should do so under an agreed shared-care protocol with clear treatment end points. 1.1.5 A further assessment should be made, usually two to four months after reaching maintenance dose of the drug. Following this assessment the drug should be continued only where there has been an improvement or no deterioration in MMSE score, together with evidence of global improvement on the basis of behavioural and/or functional assessment. 1.1.6 Patients who continue on the drug should be reviewed by MMSE score and global, functional and behavioural assessment every 6 months. The drug should normally only be continued while their MMSE score remains above 12 points, and their global, functional and behavioural condition remains at a level where the drug is considered to be having a worthwhile effect. When the MMSE score falls below 12 points, patients should not normally be prescribed any of these three drugs. Any review involving MMSE assessment should be undertaken by an appropriate specialist team, unless there are locally agreed protocols for shared care.

11.2


11.2.1

Data sources

We used from two sources: • •

Pharmacy data from 37 hospitals PACT data on GP prescribing, obtained from the Prescribing Support Unit for the periods 19992001. The data was analysed at Health Authority level, and was not mapped to PCT level

5

The Guidance states that an MMSE of 12 is the threshold below which the drugs should not be prescribed. We have therefore taken 12 as the minimum score to meet the guidance criteria

70

11.2.2

Descriptive analysis

Analysis of hospital data Figure 11.1 shows the use of these drugs in hospital.

Figure 11.1 Hospital use of the three Alzheimer’s drugs combined Total Alzheimers drugs - Hospital Usage 3000

2500

Patient Months

2000

1500

Guidance brought in January 2001

1000

500

Ap r-9 7 Jun -97 Au g-9 7 Oc t-9 7 De c-9 7 Feb -98 Ap r-9 8 Ju n-9 8 Au g-9 8 Oc t-9 8 De c-9 8 Fe b-9 9 Ap r-9 9 Jun -99 Au g-9 9 Oc t-9 9 De c-9 9 Fe b-0 0 Ap r-0 0 Jun -00 Au g-0 0 Oc t-0 0 De c-0 0 Feb -01 Ap r-0 1 Jun -01 Au g-0 1 Oc t-01 De c-0 1 Fe b-0 2

0

Month

Use of the Alzheimer drugs was almost zero until September 1999, after which use increased, particularly from February 2000, indicating that the drugs were being prescribed in most hospitals for which we have data for more than 12 months prior to the issue of the guidance in January 2001. However, additional hospitals appear to have commenced prescribing following the issue of the guidance whilst prescribing levels in those already prescribing the drug also continued to rise.

Analysis of primary care data Figure 11.2 shows the trend for the prescribing of the drugs in the community. Total use, as measured by patient months, has increased steadily and dramatically from January 2001, when the guidance was introduced, to the current levels of around 13,000 per month. For the previous 18 months from February 1999 to November 2000, there has been a slow and steady increase from 2,000 to 4,000 per month, indicating that some GPs were prescribing the drug prior to the issue of the NICE guidance. No evidence could be found to support the hypothesis that the publication of NICE Guidance was responsible for the increase in the growth rate of Alzheimer drugs prescribing in January 2001. The statistical analysis indicated that Alzheimer drug prescribing had been growing logarithmically since February 1999. Once this had been taken into account, the apparent increase in the growth rate at the time of publication of guidance was not statistically detectable. However, the data are also compatible with a view that there was a significant increase in trend shortly before the guidance was formally issued.

71

Figure 11.2 Primary care use of the three drugs combined Total Alzheimers Drugs - Community Usage 16000

14000

12000

Patient Months

10000 Guidance brought in January 2001 8000

6000

4000

2000

Ap r-0 1 M ay -01 Jun -01 Jul -01 Au g-0 1 Se p01 Oc t-01 No v-0 1 De c-0 1 Jan -02 Fe b-0 2

Fe b-9 9 M ar99 Ap r-9 9 M ay -99 Ju n99 Ju l-9 9 Au g-9 9 Sep -99 Oc t-99 No v-9 9 De c-9 9 Jan -00 Fe b-0 0 M ar00 Ap r-0 0 M ay00 Jun -00 Jul -00 Au g-0 0 Sep -00 Oc t-00 No v-0 0 De c-0 0 Jan -01 Fe b-0 1 M ar-0 1

0

Month

We expected that there would be an increase in the availability of these drugs following publication of the NICE guidance. The hospital data was censored before the point at which the last month of no prescribing occurred, and the data was transformed logarithmically for both sets. We found no statistical evidence that the publication of NICE Guidance impacted the prescribing behaviour for Drugs for Alzheimer’s Disease We analysed use relative to the elderly populations in Health Authorities. Comparison of drug use, expressed as rates per 1,000 population 6 estimated to suffer from dementia, shows significant variation, as shown in tables 11.1 and 11.2 below. Table 11.1

Demographically standardised GP prescribing of drugs for Alzheimer’s Disease (in patient months) by Health Authorities in 2000 (pre Guidance) and 2001 (post Guidance)

Mean (standard deviation) Coefficient of variation Median (I-Q range) Top prescriber Bottom prescriber

2000 67.2 (74.1) 1.10 38.8 (21.9-70.6) 354.5 5.8

6

2001 191.5 (186.2) 0.97 126.9 (75.0-212.7) 1034.9 27.6

Taken from the MRC Cognitive Function and Ageing Study: MRC CFAS (1998) Cognitive function and dementia in six areas in England and Wales: the distribution of MMSE and prevalence of GMS organicity level in the MRC CFA Study. Psychological Medicine, 28, pp 319-335

72

Table 11.2

Changes in demographically standardised GP prescribing of drugs for Alzheimer’s Disease (in patient months) by Health Authorities between 2000 (pre Guidance) and 2001 (post Guidance)

Average total change (standard deviation) Median total change (I-Q range) Largest absolute change Smallest absolute change Average % change (standard deviation) Median % change (I-Q range) Largest % Change Smallest % Change

124.3 (121.7) 86.5 (49.3-159.3) 680.5 6.7 270% (260%) 194 % (119%-299%) 1741% 6%

All Health Authorities increased their use of Alzheimer’s Disease drugs (as determined from the standardised data) in the year following publication of NICE guidance, compared to the year preceding publication. Although the standard deviation of use was higher in 2001 than in 2000, the coefficient of variation was lower in 2001 than in 2000. This indicates that the relative variation in drug prescribing across the former health authorities fell in the period following the publication of NICE guidance.

11.3

Evidence from the audit

11.3.1 Sample The audit was carried out in two settings, the first in secondary care, in both mental health trusts and PCTs where these managed secondary mental health care. Secondly, an audit was carried out in primary care. No PCT was approached for both samples. The samples are given below: The sample requested from mental health care organisations was defined as: “First twenty-five patients prescribed one or more of donepezil (Aricept), rivastigmine (Exelon) or galantamine (Reminyl) on or after 1 July 2001 and first twenty-five patients prescribed one or more of these drugs after 1 December 2001”; The sample requested from general practice was defined as: “First three, seven, ten or fifteen patients in a practice (the exact sample size will be given in advance and will depend on practice size) who have been prescribed one or more of donepezil (Aricept), rivastigmine (Exelon) or galantamine (Reminyl) on or after 1 July 2001”

Fourteen mental health care organisations (MHOs) participated in the audit and 703 forms were returned of which 583 (83%) were used for analysis. 17 PCTs participated and returned 215 forms, of which 180 (84%) were used for analysis. The reason for the low response rate in primary care may include: • GPs have few patients prescribed drugs for Alzheimer’s Disease • GPs do not manage these patients and/or are not aware of the drug regimens of these patients

73

11.3.2

Questions

Twelve questions were asked about the treatment regime of each patient: Question 2

Patient year of birth

Question 3

Date of first prescription of the drugs (for each of DA, RE and GR)

Question 4

By whom was drug first prescribed: General Practitioner Elderly Care Physician Physician (Other) Elderly Care Psychiatrist Adult Psychiatrist Other (please specify)

Question 5

Were any of the following tests administered to assess cognitive function at first prescription or in preceding three months: Mini mental state examination (MMSE) Clinician’s interview-based impressions of change (CIBIC) Alzheimers Disease assessment scale (ADAS) Progressive deterioration scale (PDS) Other (please specify)

Question 6

What score was achieved in MMSE in above assessment

Question 7

Was the drug first prescribed in the context of a dedicated memory clinic

Question 8

Was a relative or carer consulted as part of first assessment of patient’s mental state

Question 9

Has patient been reviewed for Alzheimers Disease since first prescription

Question 10

Please provide information of follow-up reviews (maximum of six follow-ups): Date/MMSE/Presence of carer/relative during assessment/By whom review carried

out Question 11

Was the drug prescribed at first assessment continued at follow-up reviews

Question 12

Please give dates of any change in or cessation of drug regimes

Question 13

Is patient still being prescribed any of the above drugs

11.3.3

Analysis of results

Introduction The analysis is reported in three parts: the first focuses on the management of the first prescription of the drugs, the second on the management of the patient followups and the third on the management of the continuation of the drug regimes

Management of first prescription We summarised the overall level of compliance, across the five requirements that: • • • •

The patients are assessed by a specialist doctor; The patients are assessed using MMSE; The patients score 12 or greater on the MMSE score; The patients are assessed in a specialist memory clinic;

74

•

A carer or relative was consulted as part of the first assessment.

The results for mental health organisations showed that the prescribing of drugs complied with: • Recommended assessment levels and tests undertaken by specialist doctors for 85% of patients; • First prescribing in specialist memory clinics for 58% of patients; • The involvement of a relative or carer for 80% of patients. Analysing the results for primary care, shows a lower level of compliance across NICE guidance, the prescribing of drugs at first prescription complied with: • • •

Assessment levels and tests undertaken by specialist doctors for 45% of patients; First prescribing taking place in specialist memory clinics for 21% of patients; The involvement of a relative or carer for 38% of patients.

Management of patient follow-up We assessed performance against a series of criteria for the management of patient follow-ups. • How many follow-up visits (a maximum of six) and whether these took place within six monthly intervals, or at longer timescales; • The MMSE score, which should not be less than 12 • Whether a carer or relative was present; • Whether the review was carried out by a specialist or in the context of shared care between primary and secondary care. The compliance rates for the management of patients against all criteria were low, due mostly to missing data. For patients managed in MHOs: • Only 5% of patients had no follow-ups, rising to 19% who had four or five follow-ups • The management of 25% of patients with one follow-up complied with all the criteria. This declined to 11% of patients with three follow-up and zero percent of patients who had five follow-up visits • The percentage of patients whose management met all the criteria at each follow-up rose from 26% at first follow-up to 47% at fourth follow-up and declined to 34% at sixth follow-up For patients managed in primary care • 30% had no follow-up visits, declining to 3% who had six follow-up visits • The management of 16% of patients with one follow-up complied with all the criteria. This declined to 7% of patients with three follow-ups and zero percent of patients who had five follow-ups, of which none were fully compliant • The percentage of patients whose management met all the criteria at each follow-up rose from 17% at first follow-up to 25 % at third follow-up and declined to zero% at fifth follow-up. However, these were very small numbers for the third to sixth follow-up visits.

75

The management of drug regimes The NICE guidance states that treatment should not usually be continued beyond 12 months and never beyond 24 months. We collected data on the date of the last prescription and whether the drugs were still being prescribed for the patient at the time of the audit. Our analysis showed that at the time of the audit: • 82% of patients managed in MHOs and 73% of patients in primary care were on the same drugs as prescribed at first assessment • 75% of patients managed in MHOs and 86% of patients managed in primary care were still being prescribed drugs for Alzheimers, This shows that a significant proportion of the patients, whose notes were audited were still receiving one of the acetylcholinesterase inhibitors for Alzheimer’s Disease.

11.3.4


We sent guidance specific questionnaires to the lead clinicians in the 16 mental health care organisations, comprising mental health trusts, or PCTs which managed mental health services, which participated in the audit phase. We received 8 completed forms, a response rate of 50%. Five of the respondents stated that they had fully implemented the guidance, and three respondents stated that the guidance had been partially implemented. The factors perceived to be contributing to the successful implementation of this guidance are shown in Table 11.3. Table 11.3

Factors perceived to contribute to the successful implementation of NICE Guidance on Alzheimer’s Drugs within Trusts

Factors Clinical lead or local champion High level of evidence or recommendation High local priority Single specialty service Other Agreed in local development plan Low cost Total citations Multiple factors could be cited


Comments under “other” included one respondent who stated, “we just did it” and a second who said it was easy to implement apart from issues with shared care with GPs. The main reasons perceived to contribute to the lack of implementation of the NICE Guidance for drugs for Alzheimer’s disease within a trust are given in table 11.4.

76

Table 11.4

Main reasons for the lack of implementation of NICE Guidance on Alzheimer’s drugs within Trusts

Factors

No of times cited*

Extra investment and/or resources required Lack of clinical leadership or local champion Too complex or involves other organisations Not regarded as a priority Low level of evidence Not agreed in HIMP, local development plan or through the SAFF round Other Implementation intended but has not yet been carried out Total citations Multiple factors could be cited

3 2 2 1 1 1 1 0 11

The respondent who gave ‘other’ stated that individual clinical practice still prevailed. Five (62.5%) respondents stated that there were significant cost consequences for their trusts, two (25.0%) stated that there were not, and one did not know. The summary of these cost consequences is given in Table 11.5. Table 11.5 Perceived cost consequences for Trusts associated with the implementation of NICE Guidance on Alzheimer’s drugs Factors Recruitment Other Facilities Equipment Staff training Total citations Multiple factors could be cited


Three trusts gave ‘other’ as a reason, which included: • Cost of drugs • Cost of development of memory clinics • Cost of staff time in developing service Five trusts stated that the guidance was high on their list of priorities; two stated that it was not, and one trust did not know. Those, which stated it was not, cited the development of community mental health teams and the implementation of the NSF guidelines as examples of other developments taking priority. No trusts responded that the guidance had displaced plans for other service developments. Four trusts provided general comments, two of which stated that the guidance was welcome because it reinforced existing practice and shared care protocols.

11.4 Conclusions Total use of drugs for Alzheimer’s Disease prescribed by hospitals and primary care has increased steadily from January 2001, when the NICE Guidance was introduced, however, there had been a steady increase in hospitals from September 1999 and in primary care from February 1999, showing that the drug was being prescribed prior to the issue of the NICE guidance.

77

Of concern is that we have been unable to demonstrate that the treatment regimens are being applied consistently in accordance with those laid out in the guidance. The main problem is that the data on MMSE scores, who is undertaking follow-ups, or whether relatives have been present at consultations for example, are often not recorded in the notes. The problem appears to be greater in primary care than in mental health care organisations. Because of the complexity of the NICE guidance itself, and the requirements for regular monitoring to which are attached several conditions, missing data has impacted significantly on our analyses, particularly for patients managed in primary care, where information appears not to be collected systematically at patient follow-up or is not easily available.

78

12. THE USE OF ORLISTAT IN GENERAL PRACTICE 12.1

INTRODUCTION

Guidance on the use of orlistat for the treatment of obesity in adults in General Practice was issued in March 2001 and advised that: •

Orlistat should only be prescribed for people who have lost at least 2.5kg in weight by dietary control and increased physical activity alone in the month prior to the first prescription and meet one of the following criteria: o A body mass index (BMI) of 28kg/m or more in the presence of significant co-morbidities which persist despite standard treatment (e.g. type 2 diabetes, high blood pressure and/or high cholesterol levels); o A BMI of 30kg/m or more with no associated co-morbidities.

•

Orlistat should only be prescribed for people between the ages of 18 and 75 years;

•

When treatment with orlistat is offered, arrangements should be made for appropriate health professionals to offer specific concomitant advice, support and counselling on diet, physical activity and behavioural strategies;

•

Continuation of this therapy beyond three months should be supported by evidence of a cumulative weight loss of at least a further 5% of body weight from the start of drug treatment;

•

Continuation of this therapy beyond six months should be supported by evidence of a cumulative weight loss of at least 10% of body weight from the start of drug treatment;

•

Treatment should not usually be continued beyond 12 months and never beyond 24 months.

12.2

Data sources

We used two data sources: • •

Hospital pharmacy data from 29 hospitals PACT data on GP prescribing, obtained from the Prescribing Support Unit for the periods 1999-2001. The data was analysed at Health Authority level, and has not been mapped to PCT level

12.2.1 Descriptive Analysis Analysis of hospital data Figure 12.1 indicates that orlistat was being prescribed from early 1999, but prescribing levels were low. Most of the hospitals prescribing it from 1999 appeared to continue prescribing at the same levels, although two increased the prescribing levels, one dramatically, when the guidance was issued. Only small numbers of hospitals appear to have commenced prescribing following guidance issue.

79

Figure 12.1 Total hospital use for orlistat Total Orlistat Drugs - Hospital usage 250

Patient Months

200

150

100

Guidance brought in March 2001

50

Ap r-9 7 Ju n-9 7 Au g-9 7 Oc t-9 7 De c-9 7 Fe b-9 8 Ap r-9 8 Jun -98 Au g-9 8 Oc t-9 8 De c-9 8 Feb -99 Ap r-9 9 Jun -99 Au g-9 9 Oc t-9 9 De c-9 9 Fe b-0 0 Ap r-0 0 Jun -00 Au g-0 0 Oc t-0 0 De c-0 0 Feb -01 Ap r-01 Jun -01 Au g-0 1 Oc t-0 1 De c-0 1 Fe b-0 2

0

Month

Analysis of primary care data Figure 16.2 below shows that between March 1999 to February 2001 there was a slow increase in prescribing from 5,000 to 10,000 patient-equivalents per month, indicating that GPs were prescribing orlistat prior to the issue of the guidance. From March 2001, use increased and has reached a high plateau of between 35,000 to 40,000 patient equivalents per month.

12.2.2 Statistical Analysis The data was transformed logarithmically before the analysis was undertaken. The hospital data was censored before the point where the last month of no use occurred to avoid further bias. Analysis pointed to the hospital data being adequately modelled with a time trend and constant, whereas the PACT data had an autoregressive component. For the hospital data there is no statistical evidence that use of orlistat changed after the publication of the NICE guidance. From the PACT data, there was a statistically significant increase in the prescribing of orlistat following publication of the guidance; the model pointing to an increase in the constant parameter of 0.425 (95% CI, 0.3 0.5)

80

Figure 12.2 Total primary care use for Orlistat Total Orlistat Drugs - Community Usage 45000

40000

35000

Patient Months

30000 Guidance brought in March 2001

25000

20000

15000

10000

5000

Feb-02

Jan-02

Dec-01

Nov-01

Oct-01

Sep-01

Aug-01

Jul-01

Jun-01

May-01

Apr-01

Mar-01

Feb-01

Jan-01

Dec-00

Nov-00

Oct-00

Sep-00

Aug-00

Jul-00

Jun-00

May-00

Apr-00

Mar-00

Feb-00

Jan-00

Dec-99

Nov-99

Oct-99

Sep-99

Aug-99

Jul-99

Jun-99

May-99

Apr-99

Mar-99

0

Month

We also analysed variation across health authorities, expressed as a standardised rate per relevant (those aged between 18-75 years) 1,000 population. The results are shown in tables 12.1 and 12.2 below. Table 12.1:

Standardised GP prescribing of orlistat (in patient months) by Health Authorities in the 12 months prior and following Guidance publication in March 2001

Mean (standard deviation) Coefficient of variation Median (I-Q range) Top prescriber Bottom prescriber

Table 12.2

March 2000-Feb 2001

March 2001-Feb 2002

4.8 (1.9) 0.4 3.9 (2.8-5.2) 10.3 0.4

12.7 (3.8) 0.3 12.3 (9.8-14.8) 23.6 3.8

Changes in standardised GP prescribing of orlistat (in patient months) by Health Authorities between in the 12 months prior and following Guidance publication in March 2001

Average total change (standard deviation) Median total change (I-Q range) Largest absolute change Smallest absolute change Average % change (standard deviation) Median % change (I-Q range) Largest % Change Smallest % Change

8.6 (2.6) 8.4 (6.8-10.2) 15.4 2.4 263% (165%) 216% (157%-298%) 1114% 31%

81

All Health Authorities increased their use of orlistat (as determined from the standardised data) in the year following publication of NICE guidance, compared to the year preceding publication. Although the standard deviation of use was higher in 2000/01 than in 2001/02, the coefficient of variation was slightly lower in 2001/02 than in 2000/01. This indicates that the relative variation in drug prescribing across the former health authorities fell slightly in the period following the publication of NICE guidance.

12.3

Findings from the audit

12.3.1 Sample The sample for the audit of the use of orlistat in general practice was defined as: “First three, seven, ten or fifteen patients in a practice (the exact sample size will be given in advance and will depend on practice size) who have been prescribed Orlistat by a GP on or after 1st September 2001.”

Of the eighteen PCTs (out of 21, a response rate of 86%) that returned completed forms, three returned 50 completed questionnaires, nine between 30 and 49 and six returned fewer than 30. Three did not participate.

12.3.2 Questions Ten questions were designed to collect information about whether the clinical condition of the patient and their treatment management complied with the requirements of the guidance as summarised above. Question 2 Question 3 Question 4 Question 5 Question 6 Question 7 Question 8

Question 9 Question 10 Question 11

Patient year of birth Was prescribing of Orlistat initiated by GP practice? Yes/No/Don’t know If no, where was prescribing initiated? Was there a documented weight loss in the 3 months prior to date of first prescription Yes/No/Don’t know If yes, please record weight and body mass index at beginning of the three-month period At first prescription of Orlistat, what was patients: Weight/BMI/date of first prescription Weight record at patient visits following prescription of Orlistat: Date of visit/weight at visit (or not recorded)/ BMI at visit (or not recorded)/ whether patient continued on drug For visits at or around: 6 weeks/3 months/6 months/9 months/one year/18 months/2 years Is Orlistat still being prescribed? Yes/No/Don’t know What was date of last prescription? Did the patient have any of the following co-existing illness at the time of first prescription? Diabetes Mellitus/Hypertension/Cholesterol>5

Only 22% of the forms were fully completed, however, although one PCT returned 100% fully completed forms. These figures indicate that much of the data, which should be held in patient notes in order to manage the treatment of a patient in accordance with the NICE guidelines, appears not to be recorded. We have, therefore, analysed the questions separately, and have calculated the percentage of patients whose treatment is not in line with the guidance. This has allowed us to assess the likely range of compliance with the criteria for prescription of orlistat.

82

There was also a question requesting either the size of the practice from which the patient was selected. The audit clerks completing the forms were asked to complete as much of the audit form as possible even if data were missing.

12.3.3 Analysis of results The prescription of orlistat was initiated in primary care for 635 patients (92%). For the 29 patients (4.2%) for whom prescribing was initiated elsewhere, the most common location given was the diabetic clinic at the local hospital. We had no data on the rest. Analysis of the age of the patient showed that 19 patients were above the recommended age 75. Analysis of the BMI of patients at the time of prescription, in conjunction with the presence of co-existing illness showed that 3 months prior to prescription, 27% had a BMI greater than 30kg/m 2 and data were not available and/or provided for 77% of patients. Furthermore at the time of prescription, 45% of patients had a BMI of greater than 30kg/m 2 data were not available and/or provided for 53%. Only 12% of patients at 3 months prior to prescription and 21% at the point of prescription had documented co-morbidities. Analysis of the documented weight loss of the patient in the three months prior to the date of the first prescription shows that 308 (48%) of the patients had a weight loss recorded and of those with a recorded weight, 123 (40%) were not within the weight loss recommended in the guidance. The NICE guidance states that continuation of the therapy after 3 months should be accompanied by weight losses of 5%, and after 6 months, weight loss of 10%. We would, therefore, expect patients who are in receipt of orlistat to be reviewed on a regular basis. We therefore, requested information about patient follow-up visits, including their frequency, and the patient weight and BMI at that visit. The information that was recorded in response to this question was of variable quality. As a first step, we analysed whether there was evidence of ongoing monitoring. The NICE guidance does not specify the frequency of the follow-up visits, and so we made no judgement as to what the frequency should be, and patients in receipt of orlistat for a period of over 3 months had variable visiting patterns. This analysis shows that: •

• •

112 patients (16%) appeared to be followed up for a period of up to 6 weeks and 125 patients (18%) for up to 3 months. It is likely that some of these groups of patients discontinued the treatment after 6 weeks and 3 months respectively; 57 (8%) patients continued making visits to the surgery for up to1 year, and 53 patients (7.7%) for more than a year, 23 of whom have visits recorded for up to 2 years: For 122 patients (18%) no data were recorded.

The guidance implies that weight should be monitored as patients are expected to lose between 5-10% of their body weight. Therefore, the second step in analysing the data was the assessment of the recording of weight and BMI data.

83

The analysis shows that: •

• •

Of those 567 patients for whom we have a record of visits made, there appears to be no record of weight for 66 (12%) of them. This applies equally to patients who have been taking orlistat for only 6 weeks, as for those who have been taking orlistat for more than one year; The weight of more than 50% of patients who have been taking orlistat for up to and including one year, has been monitored on a frequent basis; Monitoring the weight of patients who have been on orlistat for more than one year appears to be less frequent, with 43% of patients having made fewer than 3 visits. This finding should be of concern since the NICE Guidance states that orlistat should not usually be taken for more than one year.

Our analysis shows that the weight was not necessarily recorded at each of the patient visits. We do, however, know that 45% of the total number of patients for whom we have data had their weight recorded at 3 months and 29% at 6 months. We have calculated for those patients whom we know are were taking orlistat at the time weight was recorded, the number who have had a 5% loss in weight after 3 months, and 10% loss in weight after 6 months. Table 12.3 shows that, in patients for whom we have been able to calculate weight loss, 127 (41%) of those still being prescribed orlistat at 3 months had reduced their weight by 5% since the date of the first prescription, as advised in the NICE guidance. The percentage is lower for those patients being prescribed orlistat after 6 months, with only 40 (25%) having reduced their weight by 10%, as advised in the NICE guidance. If these figures are calculated as a percentage of the 689 patients for whom forms were returned (including those for whom no information was given on weight at 3 and/or 6 months) the percentages drop to 18% and 6% respectively. Table 12.3:

Percentage weight loss at 3 and 6 months

PCT 1 PCT 2 PCT 3 PCT 4 PCT 5 PCT 6 PCT 7 PCT 8 PCT 10 PCT 11 PCT 12

Patients losing 5% at 3 months as % of patients for whom we have data 42% 45% 58% 27% 25% 41% 33% 57% 63% 20% 65%

Patients losing 10% at 6 months as % of patients for whom we have data 0% 11% 0% 29% 0% 25% 0% 42% 15% 14% 36%

PCT 13 PCT 14 PCT 15 PCT 16 PCT 17 PCT 19 PCT 21 Total

8% 60% 46% 28% 36% 47% 22% 41%

11% 50% 38% 15% 16% 14% 17% 20%

Compliant at 3 Compliant at 6 months as % of total months as % of total no for whom forms no for whom forms completed completed 17% 0% 23% 2% 14% 0% 11% 7% 12% 0% 28% 13% 5% 0% 26% 10% 26% 4% 8% 4% 61% 28% 2% 19% 29% 13% 23% 19% 7% 18%

84

2% 19% 11% 5% 7% 5% 4% 6%

In Table 12.4 we have brought together the results of the analysis of age, BMI and weight loss by PCT, for those patients for whom we have a complete set of data. This analysis shows that across all patients we have data demonstrating that the treatment regimen for 84 (12%) is fully compliant with the NICE guidance. This varies across PCT from a minimum compliance rate of 0% in two PCTs to a maximum of 72% in one PCT. For the patients we have a full set of data not achieving the requisite weight loss is the primary reason for non-compliance.

12.4 Evidence from the Survey and Interviews Because PCTs were not in existence when the Guidance was issued, we did not send survey forms to these organisations nor did we undertake interviews in PCTs in respect of prescribing of orlistat. We did not feel there would be value from interviewing individuals in PCTs, who would have had no influence on the early prescribing patterns of GPs.

12.5 Summary of findings We found a statistically significant increase in the prescribing levels of orlistat in primary care as shown by the PACT data, following the publication of the guidance. However, there is evidence that a significant proportion of GPs are not applying the prescribing criteria laid out in the NICE guidance. Most importantly, relevant information on weight and weight loss are not routinely recorded, and when they are, a significant proportion does not meet the criteria.

85

Table 12.4: Trust ID

PCT 1 PCT 2 PCT 3 PCT 4 PCT 5 PCT 6 PCT 7 PCT 8 PCT 9 PCT 10 PCT 11 PCT 12 PCT 13 PCT 14 PCT 15 PCT 16 PCT 17 PCT 18 PCT 19 PCT 20 PCT 21 Total

Analysis of compliance levels by trust

No of returns

29 44 50 28 49 47 20 50 DNP 46 50 18 44 16 45 40 44 DNP 42 DNP 27 689

All requirements of NICE guidance recorded and met

Missing data in one or more fields

Patients for whom a full data set is available Age non compliant

BMI non compliant

Weight loss non compliant

Weight loss and age non compliant

Weight loss and BMI non compliant

No

No

0 0 0 0 0 0 0 0

% returns 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

No

4 2 4 3 4 8 1 2

% returns 13.8% 4.5% 8.0% 10.7% 8.2% 17.0% 5.0% 4.0%

No

0 0 0 0 0 0 0 0

% returns 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

No

0 0 0 0 0 0 0 1

% returns 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 2.0%

No

21 39 41 17 34 36 18 44

% returns 72.4% 88.6% 82.0% 60.7% 69.4% 76.6% 90.0% 88.0%

No

3 3 5 7 9 3 1 3

% returns 10.3% 6.8% 10.0% 25.0% 18.4% 6.4% 5.0% 6.0%

1 0 0 1 2 0 0 0

% returns 3.4% 0.0% 0.0% 3.6% 4.1% 0.0% 0.0% 0.0%

4 9 13 1 0 6 8 5

8.7% 18.0% 72.2% 2.3% 0.0% 13.3% 20.0% 11.4%

39 34 0 41 16 38 25 33

84.8% 68.0% 0.0% 93.2% 100.0% 84.4% 62.5% 75.0%

0 0 0 0 0 0 1 0

0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 2.5% 0.0%

0 2 0 0 0 0 0 1

0.0% 4.0% 0.0% 0.0% 0.0% 0.0% 0.0% 2.3%

3 5 5 2 0 1 5 4

6.5% 10.0% 27.8% 4.5% 0.0% 2.2% 12.5% 9.1%

0 0 0 0 0 0 1 1

0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 2.5% 2.3%

0 0 0 0 0 0 0 0

0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

4

9.5%

37

88.1%

0

0.0%

0

0.0%

1

2.4%

0

0.0%

0

0.0%

0 84

0.0% 12.2%

24 537

88.9% 77.9%

0 2

0.0% 0.3%

0 3

0.0% 0.4%

3 57

11.1% 8.3%

0 2

0.0% 0.3%

0 4

0.0% 0.6%

86

13. NHS Trust response to NICE guidance 13.1 Introduction This section is based on an analysis of data extracted from the survey of acute and mental health Trusts and interviews with Chief Executives, Medical Directors and clinicians from acute NHS Trusts,

13.2 Evidence from the survey 13.2.1 Methodology We surveyed Trust Chief Executives about general issues regarding the implementation of NICE guidance and asked clinical leads about guidance-specific issues. Forms were sent to the 20 acute and 16 mental health organisations, which had participated in the patient audit phase. We received twenty two completed Chief executive surveys, a response rate of 61%, of which 13 (a response rate of 65%) were from acute trusts and eight (a response rate of 50%) were from mental health trusts. We received 76 completed guidance survey forms in total across all the trusts and all sets of guidance, an average response rate of 57% from acute trusts and 50% from mental health trusts We followed up non-responding trusts, several of which told us they were not able to participate in this phase of the study. A variety of reasons were given such as pressure of work and staff turnover. Several trusts, despite our best efforts, never responded nor returned forms. We used the questionnaires designed and used by SIGN (Scottish Intercollegiate Guidelines Network) as the basis for the questionnaires and CEO survey Those questions that focused on management issues were incorporated into the CEO survey. Those questions that focused on clinical issues were incorporated into the clinical questionnaires, for which we prepared one for each set of guidance. We also included three questions, which specifically focused on the opportunity costs of implementing a set of guidance.

13.2.2 Findings from Chief Executive Survey The analysis is based on twenty-two responses, 13 from acute trusts and 9 from mental health organisations, which have been combined since the trusts were asked the same questions. In 63% of trusts, the medical director was the clinical governance lead, and in 11% it was the chief executive. In 19% of trusts, which gave ‘other’, the lead included associate medical director and the Director of Nursing. 19 trusts said they had a strategy for disseminating guidelines to appropriate people. We asked trusts to describe which of a number of steps best described their strategy (they could tick all that applied). All CEOs replied to this question. 73% of respondents indicated that registering receipt and establishing responsibilities and follow-up, plus having a targeted approach to those affected are the most popular

87

methods for dissemination. 50% of respondents also gave follow-up reminders as a method. Other approaches were also described: • • • • • •

Assessment of impact of guidance undertaken by directorate or division and reviewed internally by a NICE Implementation Group. Funding also agreed. Quarterly Report to Clinical Governance Board Audit of guidelines implementation undertaken by Trust Audit and Effectiveness Team Distributed by Trust Audit and Effectiveness Group Incorporated in to the Services’ agendas through Service Meetings Regular presentations on progress to Clinical Audit and Effectiveness Committees A Guidance will be made Trust policy with implementation plan

16 respondents (73%) also stated that their trust had a strategy for implementing guidance after dissemination. Table 13.3 shows the responses to questions on whether trusts have a monitoring framework in place.

Table 13.3

Existence of monitoring framework

Monitoring Framework for: Introduction of NICE Guidance Implementation of NICE Guidance Evaluation of NICE Guidance Recommendations in NICE Guidance

Answered Yes Number % 20 91% 19 86% 15 68% 16 73%

Answered No Number 1 2 5 4

No response Number 1 1 2 2

Trusts were asked to describe their local frameworks: the responses have been analysed across a number of common themes, as shown in table 13.4. Table 13.4 Implementation and monitoring frameworks – analysis of qualitative data Theme

No of Trusts reporting

Sub theme

Impact assessment Assessment of fit with local clinical opinion Assessment of cost implications • Before guidance publis hed • After guidance published • Formal response required from individual/locality Implementation Formal appointment of clinical lead Committee to manage all NICE implementation • Established committee • Steering group/task force set up for each guidance Full implementation strategy exists

1 6 1 6 2 5

Devolved to directorate/locality Responsibility of named individual • Clinical governance manager • Medical director or assistant

2

Audit/monitoring

Audit department named Audit responsibility with pharmacy for drugs Monitoring by reports

88

9 2 2

2 2 10 1 2

We asked whether any of our tracer sets of guidance had been difficult to introduce. Table 13.5 shows that more than 50% of respondents answered no for seven of the sets of guidance. Only in the case of the guidance for drugs for Alzheimer’s were difficulties in introducing the guidance reported, although the number of trusts here is small (only 56% of trusts participating in the audits). Table 13.5

Difficulties in the introduction of guidance

Guidance

Acute trusts (n=13) Removal of wisdom teeth Prostheses for THR Use of taxanes in breast cancer Use of taxanes in ovarian cancer Hearing aid technology Use of ICDs Laparoscopic surgery for inguinal hernia Laparoscopic surgery for colorectal cancer Use of drugs for treatment of non small cell lung cancer Mental health trusts and PCTs (n=9) Use of drugs for Alzheimer’s disease

Answered Yes

Answered No

Number

%

Number

%

No response/ not applicable Number %

0 1 0 0 3 2 3

0% 8% 0% 0% 23% 15% 23%

13 9 11 11 7 6 7

100% 69% 85% 85% 54% 46% 54%

0 3 2 2 3 5 3

0.0% 23% 15% 15% 23% 38% 23%

2

15%

6

46%

5

38%

2

15%

7

54%

4

31%

5

56%

4

44%

0

0%

Finally, in 90.9% of organisations, NICE guidance forms part of the clinical audit programme.

13.2.3

Survey of implementation of sets of guidance

We received 76 completed guidance survey forms in total across all the trusts and all sets of guidance. The results presented here present the summary analysis. The analysis of the individual sets of guidance has been discussed in the preceding sections. Respondents reported that 70% of the sets of guidance were implemented fully, 23.7% were partly implemented and 7% were not implemented. The results are shown, by guidance, in Table 13.6

Table 13.6 Trusts

The extent to which the guidance been implemented in

Guidance

Fully Number 13 6 9 11 4 5 5 53

Removal of wisdom teeth Prostheses for THR Use of taxanes in breast cancer Use of taxanes in ovarian cancer Hearing aid technology Laparoscopic surgery for inguinal hernia Use of drugs for Alzheimer’s Disease Totals

% 100% 60% 90% 92% 36% 42% 63% 70%

Partly Number 0 2 1 1 7 4 3 18

Not at all Number 0 2 0 0 0 3 0 5

We compared these results with those from the Chief Executive survey on the perceived difficulties of implementation. Both sets of survey results reported that there were no difficulties in implementing the guidance on wisdom teeth, nor on taxanes for breast and ovarian cancer. Table 13.7 shows the implementation of sets of guidance by trust, according to the results of the survey.

89

Table 13.7

Implementation rates for acute trusts as self-reported from the survey of trust guidance clinical leads

ACUTE TRUSTS Trust

Guidance surveys Wisdom Total hip teeth replacement

Trust 1 Trust 2 Trust 3 Trust 4 Trust 5 Trust 6 Trust 7 Trust 8 Trust 9 Trust 10 Trust 11 Trust 12 Trust 13 Trust 14 Trust 15 Trust 16 Trust 17 Trust 18 Trust 19 Trust 20 Total Responses

F NR NR F F F F F NR NR F F F F NR NR F F F NR 13

NB

F NR N NR F P NR F NR NR P F NR NR NR NR F F NR F 10

Taxanes for Breast Cancer N NR F F F NR F F NR NR F NR NR NR NR NR F NR F F 10

Taxanes for Ovarian Cancer N NR F F F NR F F NR NR F NR F F NR NR F NR F F 12

NR indicates that the trust did not return the forms N indicates that the trust reports did not implement the guidance P indicates that the trust reports only partially implemented the guidance F indicates that the trust reports fully implemented the guidance

90

Hearing aid technology P NR F P NR NR NR P NR NR P F NR P NR NR F F P P 11

Laparoscopic Surgery for hernias P NR F NR P NR NR F NR NR P N N P NR NR F F F N 12

% fully implemented 33% 80% 75% 80% 50% 75% 80% 50% 75% 66.6% 50% 100% 100% 80% 60% 71%

Total returns 6 0 5 4 5 2 3 6 0 0 6 4 3 5 0 0 6 4 5 5 68

For those sets of guidance reported as having been fully implemented, the factors perceived to have contributed to successful implementation across all sets of guidance are shown in table 13.8. Table 13.8 Reported factors contributing to the successful implementation of a NICE Guidance Factors

No of times cited* 26 25 22 17 18 11 8 127

Clinical lead or local champion High level of evidence or recommendation Single specialty service High local priority Other Low cost Agreed in local development plan Total citations Multiple factors could be cited

% of citations 21% 20% 17% 13% 14% 9% 6% 100. 0%

Having a clinical lead or implementing a guidance that is based on strong evidence, therefore appeared to be the most influential factors to aid implementation In 12 cases, respondents said that practice was already in accordance with the guidance, these were for wisdom teeth; hip replacement (one respondent) and ovarian cancer (one respondent). The wisdom teeth guidance was said to be “simple” to implement. For those sets of guidance reported to have only been partially implemented or not implemented at all, the main reasons perceived as contributing to lack of implementation are shown in table 13.9 Table 13.9

Reported reasons for the lack of implementation of Guidance

Factors

No of times cited*

% of citations

Other Extra investment and/or resources required Low level of evidence Too complex or involves other organisations Not regarded as a priority Not agreed in HIMP, local development plan or through the SAFF round Lack of clinical leadership or local champion Implementation intended but has not yet been carried out Total citations

16 12 5 5 4 4

31% 24% 10% 10% 8% 8%

3 2 51

6% 4% 100.0%

Multiple factors could be cited

The requirement for extra investment, and ‘other’ reasons are by far the most common reasons given. ‘Other’ reasons included cancer treatment being provided by another Trust, over which the participating trust had no control. Two responses (in respect of laparoscopic procedures for inguinal hernia) suggested that the respondent believed that the guidance was that the procedure should be used. As rationale for NOT implementing the guidance, two respondents stated:

“Procedure deemed too expensive no great evidence of benefits.” “Training of other surgeons required. Technically difficult to do.” Respondents were asked whether there was a significant cost consequence to the Trust in implementing a set of guidance. 35% of all responses answered yes. The responses, by guidance are shown in table 13.10.

91

Table 13.10 Whether there were significant costs consequences for a Trust associated with the implementation of guidance Guidance Removal of wisdom teeth Prostheses for THR Use of taxanes in breast cancer Use of taxanes in ovarian cancer Hearing aid technology Laparoscopic surgery for inguinal hernia Use of drugs for Alzheimer’s disease Total

Answered Yes Number % 0 0% 0 0% 5 50% 4 33%

Answered No Number % 10 77% 7 70% 2 20% 4 33%

Don’t know Number % 3 23% 3 30% 3 30% 4 33%

8 4

73% 33%

2 7

18% 58%

1 1

9% 8%

5

63%

2

25%

1

13%

26

34%

34

45%

16

21%

The main cost consequences cited are shown in table 13.11. Table 13.11 Cost consequences for a trust associated with the implementation of NICE Guidance Factors


Other (Mainly drugs, staff time and hearing aids cited) Equipment Facilities Staff training Recruitment Total citations

% of citations 32% 20% 17% 17% 15% 100%

Multiple factors could be cited

Trusts were asked whether implementing a set of guidance was high on their list of local priorities for new expenditure, to which 39% answered yes. The results, by guidance, are shown in Table 13.12 below.

Table 13.12 Whether implementation of NICE guidance was high priority within Trusts Guidance Removal of wisdom teeth Prostheses for THR Use of taxanes in breast cancer Use of taxanes in ovarian cancer Hearing aid technology Laparoscopic surgery for inguinal hernia Use of drugs for Alzheimer’s disease

Answered Yes Number % 2 15% 2 20% 9 90% 7 58% 5 46% 0 0%

Answered No Number % 4 31% 5 50% 0 0% 1 8% 4 36% 10 83%

Don’t know Number % 7 54% 3 30% 1 10% 4 33% 2 18% 2 17%

5

2

1

63%

25%

13%

Total examples of other developments with higher priority included: • Tackling waiting times • Community mental health teams • Digital hearing aids • Improving critical care • New equipment Trusts were asked whether, implementation of a set of guidance had or was likely to displace other service developments. Only three responded yes. Although no explicit

92

examples were given, one of the responses to this question captured the essence of the comments made: “Not sure, but given limited NHS budget other developments must have suffered as a consequence” Another commented that it was drugs that had not been subject to examination by NICE now suffered in funding discussions with PCTs.

13.2.4

General Comments

Finally, trusts were asked if they wished to make any comments about the role of NICE or of the methodologies developed for the preparation or dissemination of guidance. Thirty responses were recorded, as summarised below: Theme Guidance welcomed Funding Perceived quality of guidance

Criticism of process

Sub-theme Already accepted as good practice Seen as useful weapon Should accompany guidance Cost consequences for other drugs Old news Insufficient specialist involvement Need to be reviewed more frequently Insufficiently clear Slow process Withdrawal inappropriate

Frequency 4 3 3 2 3 2 1 1 2 1

Choice of topics

1

13.3 Evidence from the interviews 13.3.1 Introduction Interviews with senior managers proved difficult to arrange. We interviewed three Chief Executives, two Medical Directors and one Director of Research and Development nominated by a Chief Executive. In one Trust we also followed up invitations to speak to finance and business managers and one manager responsible for NICE implementation. Finally, in this section on the overall management of guidance within Trusts, we drew on findings from the interviews with clinicians.

13.3.2 Trust-level action The Trusts varied widely in the sophistication of their systems for managing NICE guidance, and inevitably systems had evolved over time. One Trust, (Trust 15) shown by our audit to have a good record of acting in accordance with NICE guidance, reported taking a pro-active approach to NICE guidance and working to an explicit, clinician-devised implementation model that had been agreed with all stakeholders in the locality. A member of the clinical governance department reported that as soon as NICE reported that an appraisal on any given topic was to be undertaken, any available systematic reviews were placed on the website, a lead clinician was appointed, a local view established and the cost consequences identified and presented to the PCT.

93

The major sub-themes discernible within this category were the degree of locality involvement in decisions to implement guidance, the means of dissemination within the organisation, the perceived need to prioritise NICE guidance and funding.

Locality involvement Three Trusts (Trust 7, Trust 11 and Trust 15) reported a locality-wide system for receiving NICE guidance, the decision to implement going hand in hand with a realistic appraisal of costs. The locality model was particularly beneficial for Trusts that had to relate to several primary care trusts, as it ensured a consistent, agreed approach which, as is reported below, only broke down in the case of referrals from PCTs outside the locality.

Dissemination within Trusts All senior managers participating in the interviews stated that NICE guidance was circulated in hard copy to the appropriate clinicians. In two Trusts it was also said to be placed on the Trust intranet. Where clinicians could recall the source of the guidance, they tended to state that they had received it directly from NICE, or from their professional association. Only in one Trust (Trust 15) did the majority of clinicians interviewed cite the Trust website as the source of the guidance.

Prioritisation of NICE All senior managers interviewed reported funding problems (see next section) and in two it was reported that implementation was a function of affordability. One stated that the implementation of NICE guidance was viewed as mandatory, but that that was subject to funding and the bottom line was, “no funding, no implementation.” Within the audit, this Trust showed apparently low levels of compliance in relation to the use of taxanes for breast and ovarian cancer, hearing aids and wisdom teeth. Two respondents, from a Trust with a challenging financial recovery plan, implied that the implementation of NICE guidance came below the priorities set out in the performance management criteria, stating that the perceived priorities of the DH were adjudged to be the reduction of waiting lists and the achievement of financial solvency and that NICE was seen as potentially disruptive of these goals. In the latter Trust, NICE guidance was routinely disseminated and drugs were discussed in the drugs and therapeutics committee, but responses to specific pieces of guidance were left to individual clinicians, subject to affordability One Chief Executive (Trust 11) reported that drugs were not included in the Trust’s formulary until funding had been agreed, and prescribing outside the formulary was “a disciplinary offence.” The converse did not hold. One clinician in this Trust reported that s/he looked carefully at guidance and would adopt it only if it accorded with his/her interpretation of the available evidence. The hernia guidance provides an example of variation in response to NICE which appeared to be cultural. Although surgeons in two trusts were united in their disagreement with the guidance, in one trust the procedure was halted, whereas in the other it continued. In Trust 13, clinicians did not feel under any pressure from the Trust to comply with guidance, and with one exception (of a clinician who recalled giving an assurance of compliance) were unaware of any processes designed to disseminate or manage the implementation.

94

Although several respondents reported pressure from patients for specific treatments, only one reported that these views were, in some cases, driven by NICE guidance. Newspapers with their reports of ‘wonder drugs’ (herceptin was named), and the experience of other people were more common stimulants of patient demands.

13.3.2 Funding In all Trusts the cost and resource implications for local implementation were assessed on publication of the guidance, either by the lead clinician, pharmacist, or by a sub-group of the locality group. Identification of costs did not, however, inexorably lead to the provision of funding. All senior managers interviewed reported a history of “sterile arguments” with commissioners regarding the funding of newly published guidance; the response of funders being that funding for the implementation of NICE guidance was already including in “the baseline”. Relationships between commissioners and Trusts were said to have begun to mature where joint NICE implementation groups, or broader policy groups had been set up. The costs and resource consequences of implementing specific pieces of NICE guidance were clearly an issue of concern for all respondents. Within the present study it was clear from respondents that NICE Guidance relating to implantable cardioverter defibrillators (ICDs), the use of taxanes for breast and ovarian cancer and hearing aids had had major financial consequences for Trusts. In some cases, such as the taxanes, for example, the cost consequences of the guidance were mitigated by the fact that NICE recommendations concurred with the prior conclusions and practice of clinicians, so that there were few additional costs. In two areas, it emerged that small pots of money had been set aside by commissioners to fund NICE guidance published during the financial year in which the interviews took place (2003-4), but it was argued that the amount was unlikely to be sufficient to eradicate arguments over funding. Even where funding had been agreed locally, one tertiary centre reported that in cases where outlying PCTs would not fund certain treatments, patients of those PCTs did not receive the treatment. Overall, the funding of drugs was said to be less of a problem than the resourcing of infrastructure development. The implementation of the taxanes, ICD and hearing aid guidance all required staffing increases for review and monitoring, and even in areas where funding for drugs was generally forthcoming, these costs proved difficult to meet. One Trust (Trust 15) added in the costs of audit to those of implementation, arguing that the commissioners should know whether their money had been appropriately spent within the Trust. NICE guidance was cited as being a strong weapon in achieving funding for specific treatments and services, although four respondents suggested that it distorted local priorities, to the detriment of other essential services (a CAT scanner and new wheelchairs were cited as high priorities that lost out to NICE-supported technologies.) The hearing aid guidance was one example of a low-priority service that was not raised to a sufficiently high priority to achieve all, or in some cases, any, funding, with the result that no changes were possible in some Trusts. Cancer drugs on the other hand were already a high priority and in this speciality, the stated problem was that drugs that had been “NICEd” took precedence over those that had not been reviewed.

95

One oncologist in a cancer centre reported that they had been given funding for the implementation of NICE taxane guidance but that this, together with the perceived need to monitor these new drugs from the cancer centre itself, precluded the treatment being offered to patients who could or would not travel to the centre and who were therefore treated in outlying Trusts where funding was not explicitly available. These patients received the alternative, carboplatin only, treatment. This statement was echoed by a Business Manager in another cancer centre. Taking these statements together with those made by a Chief Executive, that if sufficient money was not forthcoming the guidance would only be partially implemented, it is suggested that competition for funding at a local level is perceived as a major constraint to the national implementation of NICE guidance. A recent communication from the NHS Director of Finance to Chief Executives of StHAs reveals that the expectation for 2004-5 is that NICE guidance will result in a net saving. All senior management respondents stated that only if hypothecated funding was available for the implementation of NICE guidance would access to treatments be equitable. Failing that, health communities would continue to make decisions on the basis of local priorities.

13.3.3 Monitoring of compliance All of the trusts had, by participating in our study, undertaken audits in the areas about which we were requesting. Audit departments varied from four to thirty people and processes could be described on a continuum from centralisation to devolution. In some Trusts there were audit clerks who extracted the data, and in others audit facilitators were attached to directorates in which clinicians undertook audits. One interviewee (Trust 7) reported that the scarce resources of the audit department were focussed on areas where it was known that the Trust did not comply. Another (Trust 15) had a total of 272 audits ongoing at the end of 2003. Trust 15 had a sophisticated tracking system for NICE guidance and regular reports on monitoring and audit were received by the Trust Board. Trust 13 relied on the cancer network to audit compliance with all cancer guidelines. As will be reported in the analyses relating to individual guidance some regular auditing was being carried out at departmental level. In relation to audit, several clinicians reported keeping records of treatment and follow up for present or future research purposes. One orthopaedic surgeon had commercial funding to analyse collected data, whilst another described the systematic longitudinal collection of outcome data including infection, dislocation, loosening, leg length problems, limping and the use of the Oxford hip score at regular intervals but with no resources for analysis. Strategic Health Authorities had not sought assurances of compliance with NICE guidance. Like that of the Trust management their attention was said to be was focussed on consultant numbers, financial health and waiting lists.

13.4 Feedback on NICE We invited respondents to comment on the role of NICE, or on the methods adopted for the preparation, dissemination and implementation of NICE guidance. The themes discernible in the interview data are as shown in table 13.2 below:

96

Table 13.2

Feedback on NICE: Themes

Theme

No.

Issues raised

Underpinning Philosophy

2

Perceived benefits of NICE guidance Presentation and dissemination Enforceability

2

Relevance

4

Evidence base

5

Consultation process Composition of panel

2

Perceived as organisation set up to cut costs Concern re stifling innovation and developments Perceived as open to political/business pressure Powerful aid to achieving funding Has potential to highlight low priority services Raises awareness of funding issues Website difficult Document too wordy Continue to distribute in hard copy Different approaches in different Trusts Has to be weighed up against other demands No pressure from StHAs to comply so perceived as less important than waiting lists Postcode prescribing persists In some cases guidance promotes standard practice It would be helpful if NICE published guidance on classes of drugs rather than individual drugs Large regional differences in practice Sometimes superficial assumptions (Charnley hip is not same prosthesis as used originally) Local evidence may differ from national data Evidence is statistical, not clinical Does not always give sufficient emphasis to the interests and wishes of patients Perception of lack of consultation with relevant experts

Future or extended role

1

2

4

2

Panels are too focussed on London Need for practising clinicians on panel Sometimes perceived as unbalanced in terms of viewpoint Clarify link between NICE and research commissioning process

It is emphasised that the guidance we were using as tracers was some of the earliest produced when NICE was seeking to pronounce on evidence that had been available for some time. Clearly, some of the issues in the table have already been addressed by NICE, including the publication of a short-form document.

13.5 Conclusions from survey and interviews • • • • •

There is a perception in the Trusts visited that whilst NICE-recommended treatments are more likely to be funded than others the publication of guidance does not guarantee funding and hence adoption. Lack of funding was universally perceived as the main barrier to the implementation of NICE guidance Trusts vary in their attitudes to NICE guidance and the enthusiasm with which they address its implementation The establishment of locality-wide NICE groups was believed to have aided the maturation of the commissioning process The implementation of NICE guidance is a lower priority for many Trusts than meeting financial and waiting list targets.

97

14. Conclusions This section triangulates the findings of the analysis of national and regional data, audit of patient notes, survey and interviews. The first part is structured by guidance, and includes an assessment of the success of the guidance, for which we have used two indicators: Impact assessment, defined as the degree to which the guidance appeared to cause a change in practice as measured by high-level trend data and interviews. Compliance assessment defined as the extent to which current practice complies with the recommendations contained within the guidance. It goes on to consider the Trusts, with a particular focus on the management features of one Trust that showed the most consistent degree of compliance across all the guidance audited. It is important to emphasise that this piece of work does not seek to evaluate the impact of NICE guidance per se; only whether NICE guidance is adhered to. It is impossible to assess the impact of any piece of guidance since we do not know what would have happened in the absence of guidance. For example, rates of laparoscopic surgery were low before and after publication of NICE guidance however it is impossible to tell whether this demonstrates that the guidance had low impact or if the guidance prevented an increase in use of laparoscopic surgery. Similarly it is impossible to know whether increases in the use of drugs such as taxanes and orlistat would have happened without NICE guidance.

14.1 Conclusions arranged by guidance 14.1.1 NICE guidance no 1


Issued April 2000 Impact assessment Compliance assessment

Low High

Good national data were available from HES and the Dental Practice Boards, which showed a downward trend in extractions that began in 1995. According to interviewees this downward trend was attributable to the cost-effectiveness ethos of the early nineties, the growing need to control and reduce referrals from GDPs in order to create space to undertake more complex procedures and to the growing body of evidence concerning the risk of nerve damage and consequent litigation. The survey suggested that the guidance had been simple and straightforward to implement. However, there is no evidence that rate of fall of extraction changed following publication of the guidance. Both survey and audit data indicated high levels of compliance and interviewees suggested that actual compliance was even higher than that suggested by the audit, due to poor recording in patient notes. It was also claimed that, when published, the guidance reflected current practice and required no change in attitudes or behaviour. One can conclude that NICE backed an “early winner” in the case of wisdom teeth. Professional practice was already changing in line with previous technology

98

assessment reports and it was probably beneficial that an early piece of guidance encapsulated what was already viewed by the professionals as good practice. The guidance may also have had an effect in sustaining the change that had already happened. The interview data suggest that support for guidance on this topic from the Royal College of Surgeons Faculty of Dental Surgery predated NICE. The Faculty was already developing a guideline at the time when the early reviews of wisdom teeth were published.

14.1.2 NICE guidance No 2 Selection of Prostheses for Primary Total Hip Replacement Issued, April, 2000 Impact assessment Low Compliance assessment High At the time of publication of the guidance the NHS Purchasing and Supply Agency (NHS PASA) was tasked to supply a definitive list of prostheses that met the benchmarks. A preliminary list, based on the manufacturers’ data was published on the NHS PASA website in September 2001, eighteen months after publication of the guidance, but was withdrawn from the website in the following year. It is this list that we have used as the basis for our benchmarking exercise. The definitive list is imminent at the time of writing, in February 2004. In reviewing use of prostheses, no national data were available and our analysis is based on one regional database. Of the prostheses in use that are deemed not in accordance with the guidance, 67-77% represent prostheses for which there is no benchmarking data at the NHS PASA, rather than prosthesis for which there is data suggestive of failure. The database showed that the use of benchmarked prostheses did not rise following the publication of the guidance and this was supported by the consistency of the audit data 6 and 12 months after the publication of the guidance. Evidence from the survey suggested that the guidance was not difficult to introduce, 8 out of 10 claiming to have fully implemented it. Interviewees using traditional cemented prostheses suggested the guidance demanded no change in their practice nor that of their colleagues. However, those using hydroxyapatite (HA) coated, cementless hips also stated that, as these met the ten-year criterion, they too were fully compliant. Although cemented prostheses were, according to the audit, used in 81% of total hip replacements, in line with the NICE guidance, there is great variation across trusts. Overall 10% of hip joints are HA coated, and are used by one-half of the trusts. Two clinicians interviewed claimed that the cementless, HA coated hip was at least as good as traditional cemented hips and over time would prove superior to a cemented prosthesis. The audit revealed that the main rationale for the use of a cementless prosthesis was the age and activity level of a patient. The regional database showed that 68% of patients had a single prosthesis with the remainder having some combination of cups, heads and stems. Whilst evidence may be available for individual components, there is little in respect of their use in combination. Interviewees rightly pointed out that although there are many years of

99

evidence available for some of the older prostheses, these have undergone so many changes that it is questionable whether the early findings are relevant, in other words whether any prostheses are really supported by 10 year performance data at all. Prosthetic devices is an area where the technology is constantly evolving and it is fair to say that neither the earlier HTAs nor the NICE appraisal gave sufficient recognition to this. Lilford (2000) has suggested that “tracker studies” are one possible R&D solution to challenge of evaluating evolving technologies but it is hard to find an example and certainly not in the area of hip prosthetics. Selection of hip prostheses is an area where there had been much previous activity in terms of systematic reviews, health technology assessments etc however little of this activity appeared to have engaged the profession (orthopaedic surgeons). In contrast with other early appraisals by NICE e.g., wisdom teeth, the guidance was not “pushing at an open door”. At the time the guidance was published there was no clear leadership from professional bodies (such as the Royal College of Surgeons) on this topic. Moreover the topic is extremely complex but the guidance failed to recognise this complexity and lacked clarity regarding what it required surgeons to do. We in turn struggled with the ambiguity and lack of clarity in the guidance when designing the audit. The guidance states that “the best prostheses (using long term viability as the determinant) demonstrate a revision rate (the rate at which they need to be replaced) of 10% or less at 10 years” and that “The evidence used in support of any prosthesis… should have been published or be available for peer review”. We were unclear what counts as “available for peer review” and some of our participants believed that if they had their own audit data demonstrating low revision rates at 10 years they were acting in accordance with the guidance. Locally collected audit data would be unlikely to have the statistical power to give valid insights into the performance of hip prostheses over the long term (and may also be susceptible to bias) and yet if surgeons were happy to make it available it would appear they were behaving in a manner compliant with the guidance. A further complexity is introduced as many surgeons use combinations of prosthetic components which were not designed or manufactured for use in combination (e.g., cups and stems from different manufacturers) and it is unclear whether benchmark data is still relevant when prostheses are mixed in this way. The NICE guidance did not address this issue. The guidance was difficult for surgeons to apply since it did not “stand alone;” surgeons could not pick it up and use it without recourse to other sources of information and much interpretation. To comply with the guidance a surgeon needed to find out which prostheses met 3 and 10 year revision rate benchmarks and yet for a considerable time there was no obvious place to find this out. It became clear from the interviews that surgeons held strong views regarding the relationship between patient age and appropriate hip replacement. Whilst the NICE guidance addressed the issue of ‘ease of revision’ it had not done so in a way that seemed to convince surgeons. Although the guidance states that: “No reliable evidence was available to support the proposition that the potential ease of revision of a hip prostheses [sic] would outweigh its poorer revision rate, notwithstanding any evidence that cementless and hybrid prostheses may lead to less bone loss and are therefore potentially easier to revise than cemented prostheses” Clearly some surgeons felt that their professional experience provided them with the kind of convincing evidence that could not so easily be discounted. Furthermore

100

orthopaedic surgery is a “craft” in which the complex interrelationship between surgeon preference, experience, skill and choice of hip in not understood. An individual surgeon may obtain better than average clinical outcomes from their preferred choice of prosthesis. This complexity means that it is harder to produce evidence that is relevant for practice and consequently harder to change practice. It could be argued that the guidance did not fully respect this complexity. Ideally it should be possible to monitor use of hip prostheses from routinely collected data. Hip registries continue to fulfil an essential role since HES data does not detail which prostheses are inserted. This study has further demonstrated that coding of this complex activity presents a problem. Whilst most of the emphasis has been placed on the need for clinicians to implement the evidence it could be argued that in the case of hip prostheses the NHS did not get its act together and establish a reliable source of benchmarked data – essential to enable surgeons to comply with the guidance. It could be argued that whilst this was potentially an early win for NICE (cost was not a factor in implementation) it was done too quickly. The guidance on hip prostheses is a good example where a better understanding of clinical decision making around the use of the technology might have led to guidance more likely to result in a behaviour change. If the guidance had been informed by a better understanding of the context of implementation it may have been better implemented. Finally, since this study identified particular centres which had poor rates of compliance with the NICE guidance as a result of, for example, strongly held beliefs as to the benefits of HA coated hips, NICE might consider the possibility of allowing such centres to continue their current practice within an observational study of outcomes along the lines of the “risk sharing” scheme for ß interferon.

14.1.3 NICE guidance no 3

Guidance on the use of taxanes for ovarian cancer

Issued June 2000 Impact assessment Compliance assessment

High Variable

A survey of pharmacies yielded data from which it was not possible to differentiate between taxane use for different cancers, although prescribing for breast cancer will contribute the largest proportion. The data for docetaxel and paclitaxel showed an increase in prescribing following release of the NICE guidance – the increase for docetaxel occurred immediately after guidance publication, whereas the increase for paclitaxel started slightly in advance of publication. In the survey, 92% of the 12 Trusts that responded said that they had fully implemented the guidance, although one third stated that there were significant cost consequences for the trust (drug, facilities, equipment, staff training and staffing were cited). The key finding for taxanes as an ovarian cancer treatment was the high degree of variability between Trusts in the extent to which the guidance had been implemented.

101

The percentage of women who received paclitaxel in combination as the first course of treatment was lower than 20% in three trusts and only just over 50% in a further three trusts; the average amongst audited trusts being 33%. Two of the three oncologists interviewed were based in Trusts with low recorded use of taxanes, one of which was a cancer centre and one not. Neither had been involved in the audit and both stated that the audit did not reflect their practice, which was, in both cases highly compliant. In this case, compliance m ay be higher than indicated as a result of non-involvement of clinicians and poor communication between cancer centres and referring trusts. The interviews revealed the potential impact of patients living at a distance from a cancer centre. In one case patients had the option of travelling to a cancer centre up to 35 miles from their “home” Trust (and where the audit was carried out). Although the NICE guidance had been implemented for patients treated in the cancer centre, patients unwilling or unable to travel there who were treated in their home Trust had no access to taxane therapy. The variability in implementation of the taxanes for ovarian cancer probably reflects several issues including the high cost of the drugs and uncertainty around the evidence base. The evidence base was changing quite rapidly; there were conflicting recommendations around from international professional societies and clinicians were aware of impending trial results. In a sense the NICE guidance gave the clinicians more choice as to what they could use although the guidance itself was very prescriptive. Since initial publication of the guidance (May 2000) evidence from the ICON3 trial and updated results from the GOG111 and OV10 trials became available. Consequently the guidance changed when it was subjected to routine update in January 2003 and became: “women…should be offered…either paclitaxel in combination with a platinumbased compound or platinum-based therapy alone (cisplatin or carboplatin) for first-line chemotherapy - usually following surgery”. This represented a change from the previous guidance which recommended that paclitaxel in combination with platinum therapy was the standard therapy for women with ovarian cancer following surgery. One could legitimately conclude that the variations in practice captured by the audit had therefore been shown to be appropriate.

102

14.1.4 NICE guidance no 6 Guidance on the use of taxanes in breast cancer Issued June 2000 Impact assessment High Compliance assessment High

Data obtained from hospital pharmacies demonstrated an increase in the use of docetaxel associated with publication of the guidance however patient level audit was required to establish whether these taxanes were being used appropriately i.e. in accordance with the guidance. In fact compliance with the guidance was extremely high. Only 19 of the 707 patients identified from the total sample of 17 Trusts received docetaxel in their first course of chemotherapy, of whom 15 were known to be in an RCT and 1 was believed to have an advanced form of the disease. The trial status of the remaining 3 women was not known, but it is known from the interviews that the Trust were involved in clinical trials at the time. All 9 patients who received taxanes (7 docetaxel and 2 paclitaxel) in the second course had received an anthracycline in the first course. Despite the fact that five (50%) of the trusts responding to the survey stated that there were cost consequences for the trust in implementing the guidance, 10 (90%) claimed to have fully implemented it and oncologists interviewed supported this statement. Both survey and interviews indicated that cancer drugs in general were high on the list of the Trusts’ priorities, there being some indications that they displaced plans for other service developments. The apparently successful implementation of this piece of guidance can probably be partly explained by the fact that there was a receptive, expectant audience eager for the guideline to be published. Medical oncologists had been eager to use taxanes for some time. Furthermore cancer treatment had moved to a high position in the policy agenda; publication of the National Cancer Plan and National Cancer Guidance and appointment of the Cancer Czar provides evidence of the high place of cancer treatment on lists of national health priorities. There was also a strong evidence base with little disagreement. Demand for taxanes had hitherto been constrained by managers due to its high cost but the publication of the guidance legitimised their use and the drugs were taken up quickly.

14.1.5 NICE guidance no 8

Hearing disability - new advances in hearing aid technology

Issued December 2000; withdrawn May 2003 Impact assessment Compliance assessment

Low Low

Hospital audiology has long been a Cinderella service in stark contrast with cancer services which have been receiving more attention at both local and national levels. In the absence of national data, we sent a questionnaire to all audiology departments. This demonstrated that the choice of analogue aids was either unchanged or decreased slightly after the issuing of the guidance. The audit too showed no increase in the range of analogue aids in use and in fact 2 brands of hearing aids accounted for 51% of prescriptions; 5 brands for 71%. Although data

103

from the interviews suggested that the guidance was warmly welcomed within audiology departments, the range of aids was less of an issue than other recommendations within the guidance. Even where a wide range of aids was carried, audiologists stated that they generally used a small number that were the most reliable, easily used and best tolerated by patients. With regard to the requirement that binaural aids should be made available, audiology departments reported an increase of 40% to 68% in binaural aids offered. The audit showed significant variation across trusts, but on average only 13% of those assessed as having bilateral impairment received a second hearing aid. Although it was clearly stated that financial constraints operated in some Trusts, even those in which a second aid was routinely offered if appropriate, confirmed that the take-up tended to be low, for a number of reasons. The majority of Trusts offered information although this might be in the form of a leaflet. All but two trusts recorded offering follow up, but it was revealed in interview that in some trusts the fitting was regarded as a follow up appointment, whereas in other trusts follow up was interpreted as post-fitting appointment. This may indicate that compliance with the requirement for follow-up may be lower than is recorded in the audit. The lack of implementation of the hearing aid guidance seems in large part to be explained by a lack of local investment; there was no sense of professional resistance and the guidance had been enthusiastically received in the Trusts we surveyed, who had all undertaken their own audit of services on receipt of the guidance. Audiology had rarely been regarded as a high priority service at the time around the publication of the guidance. Audiology services were briefly mentioned in the NSF for Older People (March 2001) in the context of there being a need for research to help modernise hearing aid services. Action On ENT was launched by the NHS Modernisation Agency in 2000 to look at ways of reducing waiting times for ENT patients. As part of this programme of work audiology services were chosen for more detailed attention and in September 2000 the Modernising Hearing Aid Services (MHAS) programme was launched; funded by the Department of Health and managed by the Royal National Institute for the Deaf. The aim of the MHAS is to modernise hearing services through improving patient services and increasing access to the latest hearing aid technology (viz. digital hearing aids) for NHS patients. NICE Guidance Number 8 (which concluded that “There is insufficient robust scientific evidence to support the nationwide introduction of digital hearing aids at present”) was withdrawn on 2nd May 2003. The withdrawal of the guidance had major funding consequences for one Trust in our study in which staged funding ceased when the guidance was withdrawn.

14.1.6 NICE Guidance No. 11

Guidance on the Use of Implantable Cardioverter Defibrillators for Arrhythmias

ISSUED SEPTEMBER 2000 Impact assessment Compliance assessment

Low Unknown

104

Data for our study of ICDs were provided from the British Pacing and Electrophysiology Group (BPEG) Register. We found that overall, ICD implantation rates in England had grown steadily from 1995. There was substantial growth in 1998, when the number of centres carrying out implantation also rose substantially from 33 to 58. Growth slowed somewhat after 1998, potentially due to a “wait and see” attitude by centres considering adopting ICDs. In the three quarters before the final guidance was issued, growth was above the long-run trend, suggesting an effect from the issue of the interim guidance or other evidence emerging. However, the number of centres grew only slowly in 1999 and 2000, with a much larger rise (18 more centres) in 2001. The analysis showed no evidence of an increase in use after publication of NICE Guidance. However, the post publication data are limited to just six quarters. It may also be the case that the publication of the Guidance may have taken time to alter behaviour or initiated a standard S-shaped innovation curve (Rogers, 2003) but it is impossible to say at this time why the guidance has not been implemented. In a recent study in one Trust Plummer and McComb (2003) found that there was a considerable shortfall between expected and observed ICD implantation rates. They concluded that this lack of uptake was due to a combination of failure to consider appropriate patients for ICDs; eligible patients undergoing alternative procedures such as surgical or catheter based ablation and failure to screen patients for eligibility post myocardial infarction. We did not undertake any interviews with cardiologists about ICDs as we had not undertaken a case not audit however this is clearly a topic where further research is required to understand the associated clinical decision making and better understand the lack of uptake.

14.1.7 NICE Guidance No 15 Zanamivir for the Treatment of Influenza in Adults Issued December 2000; Replaced February 2003 Impact assessment Compliance assessment

Low High

Using data obtained from the Prescription Pricing Authority report of January 2003 and the Public Health Laboratory assessments on the levels of flu in the community, we found low and falling levels of zanamivir prescribing, commensurate with a low incidence of influenza. It is impossible to tell whether zanamivir would have been used appropriately had influenza outbreaks occurred after the guidance was published however there seemed to be a fair amount of opposition to this guidance from GPs and others (see letters in BMJ 2001;322:489).

14.1.8

NICE guidance no 17 Guidance on the Use of Laparoscopic Surgery for Colorectal Cancer

Issued December 2000 Impact assessment Compliance assessment

none discernible Unknown

105

Though there are some doubts as to the accuracy of HES coding of the laparoscopic approach, on average over the time period considered only around 15 laparoscopic operations to treat colorectal cancer were performed per month. There is no evidence to indicate that in the period post publication of NICE Guidance the average number of operations changed. This might indicate that the guidance kept the numbers very low.

14.1.9 NICE guidance no. 18 Guidance on the use of laparoscopic surgery for inguinal hernias Issued January 2001 Impact assessment Compliance assessment

Low High

Analysis of HES data revealed that only 4% (2544) of primary inguinal hernia repairs in England and Wales were undertaken laparoscopically i.e. in contravention with NICE guidance. However there were large variations between Trusts in concordance with the guidance; the number of laparoscopic procedures undertaken in a single Trust varied between 10 and 221. Nine (19%) Trusts carried out 50% of the laparoscopic primary repairs, six carried out no laparoscopic procedures. There appear to be small groups of surgeons continuing to undertake laparoscopic primary repair. The results of our analysis of laparoscopic surgery for inguinal hernia repairs are broadly similar to those reported by Bloor et al (2003). HES data for laparoscopic repair of recurrent inguinal hernia (T21) (a practice endorsed by the guidance) for the calendar year 2001, it appears that 6.0% of 285 procedures were carried out laparoscopically. Of the 19 Trusts that participated in this audit, 6 had carried out no laparoscopic hernia repairs at all for either primary or bilateral/recurrent repairs and another 4 were more than 95% compliant with the guidance in respect of primary hernias. From the survey we learned that 5 trusts (42%) reported that they had fully implemented the guidance. Of the 4 that had not implemented the guidance reasons such as patient choice and preference, cost, lack of local assessment and training deficits were cited. Some trusts that undertook no laparoscopic procedures claimed not to have implemented the guidance so there may be some confusion at senior management level as to the content and meaning of NICE recommendations on this topic. Two surgeons who were interviewed argued that factors identified in the guidance (such as extended theatre times and increased costs associated with laparoscopic surgery) are restricted to the period when surgeons are developing laparoscopic skills; they estimated this to take approximately 50 procedures. Once fully skilled, they claimed, both complication rates and operation time fall and the ease of detection and simultaneous repair of occult hernias make it a highly cost-effective procedure. One Trust, employing four highly experienced surgeons who carried out the largest number of laparoscopic hernia procedures, was supported in this activity by its commissioners.

106

It is impossible to say whether NICE guidance has slowed the diffusion of this approach to hernia repair or had no effect. Some surgeons reported that patients themselves requested laparoscopic repair. There is also the argument that skills will not be able to develop given the “embargo” on laparoscopic repair. One could question whether it is appropriate to extrapolate from average costs and outcomes to the small number of highly skilled surgeons. Individual surgeons may be correct in concluding that their individual cost effectiveness is as good if not better with laparoscopic surgery than open. NHS managers in Trusts with highly skilled laparoscopic protagonists may be less likely to demand implementation for fear of losing the surgeon. It may be useful to make a distinction at this point between skills-based technologies such as surgery, where the clinician is part of the technology, and pharmaceuticals where an interaction between doctor and outcomes is less likely and extrapolation from “average” outcomes in a trial is more straightforward. One strategy for NICE in situations such as this (where “deviant” behaviour is concentrated in a few organisations/practioners) may be to undertake an observational study of the outcomes and costs associated with the few. The fact that NICE guidance may have resulted in this technology remaining in the hands of a few highly skilled surgeons could be an optimal result.

14.1.10 NICE guidance no 19 Donepezil, rivastigmine and galantamine for the treatment of Alzheimer's disease Issued January 2001 Impact assessment Compliance assessment

High Medium - Low

Prescribing of drugs for Alzheimer’s disease in hospitals and primary care has increased steadily since January 2001 when the NICE Guidance was introduced. Time series analysis has not provided clear evidence however that the guidance had an impact on prescribing patterns, though this is increasing. We have detected a slight reduction in variability though this is difficult to interpret. “001 also saw the publication of the NSF for Older People which demanded better treatment for people with Alzheimer’s disease. The respondents to the survey indicated that the guidance was either fully or partially implemented in mental health organisations, although three respondents stated that the extra resources and investment in drugs and development of memory clinics had contributed to the limited implementation. However, 63% of respondents said implementation was high on the trust list of priorities The audit of patient notes was undertaken both in primary care and in mental health trusts and was a complex undertaking as a result of the number of criteria in the guidance. Compliance with the five recommendations in the guidance at first prescription varies between 52-85% for mental health organisations and only 21-46% in primary care. Compliance with the recommendations at follow up is very low, mainly due to missing data.

107

We have been unable to demonstrate that treatment regimens are being applied consistently in accordance with the guidance mainly because information regarding MMSE scores, follow-ups, or the presence of relatives at assessment does not appear to be routinely recorded on patients’ notes. The absence of notes related to these issues appears to be greater in primary care than in mental health organisations. Therefore, use of these drugs has increased, in many cases it is not possible to say that they are being prescribed in accordance with guidance.

14.1.11 NICE guidance no 22 Orlistat for the treatment of obesity in adults Issued March 2001 Impact assessment Compliance assessment

High Low

Although orlistat was being prescribed in hospitals from early 1999, prescribing levels were low. Only small numbers of hospitals appear to have commenced prescribing following the issue of the guidance. There is no statistical evidence that the Guidance had an impact on prescribing levels in hospitals. There was a slow increase in primary care prescribing between March 1999 and February 2001. Prescribing increased following the publication of guidance and appeared to plateau from February 2002. This is statistically significant, indicating that the issue of the Guidance had an impact in primary care. This guidance was audited in primary care only. As evidenced by the returns, data recording in patients in primary care was either poor, or the auditors were unable to identify the relevant information. We had missing data in one or more fields in 80% of returns, particularly in respect of weight and weight loss. Where data were recorded, weight loss was not compliant in 8% of cases and so the rise in prescribing of Orlistat does not necessarily imply a rise in appropriate prescribing. Drug usage the evidence (or, more accurately, lack of evidence) suggests that the drug was not being prescribed in accordance with guidance.

14.1.12 NICE guidance no 26 Guidance on the use of docetaxel, paclitaxel, gemcitabine and vinorelbine for the treatment of non small cell lung cancer Issued June 2001 Impact assessment Compliance assessment

variable Unknown

There was no evidence of any change in prescribing of gemcitabine and vinorelbine associated with publication of this guidance, although usage of both drugs was rising before the publication of the guidance and continued to rise for a period afterwards.

108

It is not possible to ascertain, from pharmacy data, the indications for which these drugs are being used. We did not undertake case note audit for this guidance. Two Trusts (9%) reported in the survey that they were having difficulties implementing the guidance, the remainder stating that they did not, or that it was not applicable.

14.2 Conclusions arranged by trust 14.2.1

Trust A

On receipt of guidance in Trust A, a very large teaching hospital, a review is undertaken within the Clinical Effectiveness department before the guidance is passed to the individual departments. Once devolved, Clinical Managers/Service Managers within the areas affected by the guidance develop a local perspective including current compliance, service and financial commitments, action required and timescales for compliance. At the same time, the guidance is reviewed within an area-wide local strategic policy group, a body originally set up by commissioners and of which a Health Economist is a member. At the conclusion of the process a jointly owned document, pertaining to the NICE guidance, is produced. Apart from the cancer audits, Trust A participated in all parts of the study. NICE guidance was perceived as a high priority, subject to the agreement of local clinicians. Audit resources were focussed on areas where the trust was known to be non-compliant. The following table shows self assessment (from the survey), implementation assessment (from the audit) and impact and comment (from the interviews) Summary of Implementation Data – Trust A Guidance

Comment

Self Impact assessment assessment

Implementation assessment

Wisdom teeth

NK

Low

High

Total Hip Replacement Ovarian cancer Breast cancer

Already compliant

Fully implemented No response

Low

High

NK

NK

NK

NK

Hearing Aids

Funding sought but not achieved No appointments but open access clinic Stock a wide range although tend to prescribe a few All who need them, though limited take-up 49% primary and 68% recurrent laparoscopic hernias

Fully implemented Fully implemented Not implemented

Low

High

No

Low

Low - Trust continues to carry out primary laparoscopic repairs High - guidance allows highly experienced teams to continue

Follow up Range of aids

Offering binaural aids Laparoscopic hernias

NK NK

NR indicates that the trust did not return the forms; NK indicates not known (no data)

109

14.2.2 Trust B Trust B, a very large teaching hospital, reported the existence of a ten-stage NICE implementation model. The Trust has an appointee within the clinical governance department with specific responsibility for the implementation of technology guidance. Two clinical respondents within this trust reported that the Trust took a pro-active approach to NICE guidance, anticipating developments as far as possible. A member of the clinical governance department reported that as soon as NICE reported that an appraisal on any given topic was to be undertaken, any available systematic reviews were placed on the website, a lead clinician was appointed, a local view established and the cost consequences identified. The cost of audit of implementation is explicit within the model and the intention is that funding for audit is established at the same time as funding for implementation. At any given time it is possible to identify on the website, the status of the guidance and the lead clinician. Each guidance is also awarded a grading of A-D as follows: A = audited compliance B = statement of compliance but no audit C = initial opposition statement provided by lead clinician D = no information available. In the case of oncology, communication with commissioners had begun immediately NICE was established, a business case having been put forward for the use of the first 10 NICE drugs, agreed by the purchasers. Two to three consultant posts resulted, although those were also tied to outreach expansion The philosophy of the Trust with regard to NICE guidance was that compliance is mandatory, and clinicians reported complying even where they personally or as a group found themselves at odds with the guidance. This trust had a large audit department and reported a total of 272 audit programmes being undertaken. The following table shows self assessment (from the survey), implementation assessment (from the audit) and impact and comment (from the interviews) Summary of Implementation Data – Trust B Guidance

Comment

Wisdom teeth Total Hip Repl Ovarian cancer

Fully compliant in advance Fully compliant in advance Fully funded although introduced before guidance. Fully funded although introduced before guidance Well funded department One review after fitting Wide range although prescribed few Selected patients urged to accept 2; all requiring 2 told they could return for second Regularly undertaken – completely ceased on receipt of NICE guidance

Breast cancer Hearing Aids Follow up Range of aids Offering binaural aids Laparoscopic hernias

Self assessment NR NR NR

Impact assessment Low Low Medium

Compliance Implementation High High High

NR

Medium

High

NR

Low

High

NR

High

High

NR indicates that trust did not return forms; NK indicates not known (no data); NA indicates not applicable

110

14.2.3 Trust C Trust C is a medium sized Trust, which, after a long period of financial instability is currently following a 4-year recovery plan. The chief executive reported that NICE guidance was routinely disseminated, although it was normally the case that clinicians had received the information by other routes. The mechanism thereafter was that it should be discussed in the drugs and therapeutics committee, but action on the publication of technology appraisals was left to individual clinicians, and compliance was assumed to be a function of affordability. The view expressed in Trust C, was that the main concern of the DH is adjudged to be the reduction of waiting lists and the achievement of financial solvency and NICE was seen as potentially disruptive of the latter goals. The following table shows self assessment (from the survey), implementation asses sment (from the audit) and impact and comment (from the interviews) Summary of Implementation Data – Trust C Guidance

Comment



Wisdom teeth

Fully compliant in advance 98% HA coated hips Not a cancer centre Not a cancer centre Poorly funded department None Restricted range

Fully implemented NR

NK

High

Low

Low

NR

NA

High

Fully implemented NR

NA

High

Low

Low

Not implemented

Low

High

Total Hip Replacement Ovarian cancer Breast cancer Hearing Aids Follow up Range of aids Offering binaural aids Laparoscopic hernias

First fit basis only. May return for a second if appropriate Low numbers undertaken – completely ceased before NICE guidance

NR indicates that the trust did not return the forms NK indicates not known (no data) NA indicates not applicable

111

14.2.3 Trust D In Trust D, a very large acute Trust, NICE guidance is managed within the whole health economy through a NICE implementation group, chaired by a PCT public health director. New guidelines are received by the implementation group and a subgroup does work on the implications and costings, including those of infrastructure, particularly around the consultant numbers demanded by some of the recent cancer guidelines. Financially, the trust has taken what it describes as a hard line on drugs, stating “no money no implementation”. Until NICE drugs are sanctioned by the whole group, they do not go into the trust formulary and prescribing out with the formulary was said to be a disciplinary offence. It was stated that although the local health economy was signed up to NICE, the politics were of priorities and choice. Nevertheless, by and large NICE outweighs other issues and is seen as mandatory subject to funding. As a tertiary centre, for outlying PCTs, treatment is decided on a case-by-case basis – approval is sought for funding and if the funding is forthcoming the patient receives the treatment. It is acknowledged that whilst NICE does assist the Trust to receive funding for new treatments, it also skews priorities and removes local freedoms. The following table shows self assessment (from the survey), implementation assessment (from the audit) and impact and comment (from the interviews) Summary of Implementation Data – Trust D Guidance

Comment



Wisdom teeth

Fully compliant in advance NK

Fully implemented Partly implemented Fully implemented

Low

High

NK

High

Medium

Low (disputed)

Fully implemented Partly implemented

NK

High

Low

Low

Partly implemented

NK

NK

Total Hip Replacement Ovarian cancer

Breast cancer Hearing Aids Follow up Range of aids Offering binaural aids Laparoscopic hernias

Fully funded before guidance. Audit showed 10% of women received paclitaxel although clinician stated fully compliant NK Poorly funded department None Narrow range

None

NK

NR indicates that the trust did not return the forms NK indicates not known (no data) NA indicates not applicable

112

14.2.4 Trust E We have very little background information from Trust E, a large acute Trust that merged in 2002, as neither the Chief Executive nor Medical Director were available for interview, and only two survey responses were received. However a brief conversation with a delegated respondent informed us that on publication of NICE guidance, a form is sent to a lead clinician, requesting information on the current position and on the cost of implementation. The following table shows self assessment (from the survey), implementation assessment (from the audit) and impact and comment (from the interviews) Summary of Implementation Data – Trust E Guidance

Comment

Self assessment

Impact assessment


Wisdom teeth

Already in place before publication

Low

High

Total Hip Replacement

Clinician involved in the development and effectiveness of a prosthesis using 20 years of own data. Cancer centre situated in another Trust 35 miles distant. Offered taxanes only to patients who could attend centre. Disputes audit findings said to underestimate compliance. NK NK

Fully implemente d Partially implemente d

Low

Low

NR

Low

Low (disputed)

NR NR

NK NK

High Medium

NR

NK

High

Ovarian cancer

Breast cancer Hearing Aids

Follow up Range of aids Offerin g binaur al aids Laparoscopic hernias

NK

NR indicates that the trust did not return the forms NK indicates not known (no data)

113

15. Limitations of the study 15.1 Introduction The research resulted in a large amount of data collection on the degree to which clinical practice was in accordance with the recommendations of NICE guidance. The analysis of national and regional data allowed us to explore overall trends in relation to the date of publication. The audits allowed us to drill deeper and assess whether the details of practice corresponded to the indications of appropriateness in the guidance and thus to get a more accurate picture of likely implementation compared to more subjective surveys (e.g. Mace and Taylor, 2002). Our survey offered a view of the management context into which the guidance was received and the interviews were useful in highlighting the managerial, financial and clinical perspectives which may explain the patterns we describe. The high level of co-operation encountered in carrying out this study was indicative of the significance of the National Institute for Clinical Excellence in the eyes of senior managers and clinicians in the NHS. Most of the Trusts with which we worked had clear systems in place for the management of NICE guidance and in some cases these were both sophisticated and inclusive of the entire health community. Despite the goodwill that existed, the research was hampered at all stages by a lack of availability and inaccuracy of key data at both national and Trust level.

15.2 Regional and national data Our trends analysis was predicated on the available information. Whilst we were satisfied with the completeness of data taken from HES and PACT sources (whilst unable to comment on the accuracy), those obtained from hospital pharmacies were less complete. Of the information sent to us by pharmacists we were only able to use a fraction for the following reasons: • Trusts had difficulty in extracting the management information from their pharmacy systems: • Trusts were unable to produce monthly data; • Trusts gave total costs not actual usage. • Trusts gave total monthly figures across all drugs added together; • Trusts gave data for packs of tablets or individual tablets. Overall, for docetaxel, gemcitabine, paclitaxel and vinorelbine, we were able to use data from 24 Trusts (37% of the cancer centres in the sample); for donepezil, rivastigmine, and galantamine, data from 37 Trusts, and for orlistat, 29. We had received data from 199 trusts, but had to disregard most of the information, as it was not in the format, or provided us with the necessary information that we needed. We recognise that the usable response rate is low and may not be representative of the NHS. We were also unable to obtain information about the use to which the drugs were put, with the consequence that, particularly for chemotherapy agents, we were unable to link them to cancers at specific sites and therefore to specific NICE recommendations. Overall, the problems that we had with obtaining and analysing basic data from hospital pharmacy systems gave rise to concern about the ability of trusts to undertake local monitoring of adherence to NICE guidance using routine data from this source.

114

In relation to wisdom teeth, we were aware that the downward trend visible from 1995 took no account of private dental work, and the dip from that point may have been attributable, at least in part, to a wholesale transfer of this work from NHS general dental practitioners to the private sector at a point at which the NHS rate became uneconomic. On the other hand, the interviews revealed other reasons for reduction third molar extractions over this period. In relation to hip prostheses (in which the range of use of benchmarked devices in our audit varied from 0% in some Trusts to 100% in others), at the time the guidance was issued there was not a clear list of prostheses that NICE would have regarded as meeting the benchmark. Eventually, the NHS Purchasing and Supply Agency produced a list based on manufacturers’ data, more inclusive than that supported by the evidence presented some years earlier in an Effective Health Care Bulletin and two HTA reports. However this list was subsequently withdrawn from the NHS PASA’s website. In addition there was no definition of “ten year data”; this could be taken to refer to any ten-year collection of data, however unreliable. The response rate to our questionnaire on the range of hearing aids was 50%, representing 114 Trusts.

15.3

The audit of patient records

Designing an accurate audit form It was sometimes difficult to design audits of the uptake of pieces of guidance where the original wording of the NICE guidance had been complex, ambiguous or both. The fact that these were difficult pieces of guidance to interpret for audit implies that they must have been equally difficult to interpret for implementation. The taxanes for breast cancer guidance for example, contained a number of points of ambiguity. The statement that the drug should be “available” does not require that it be used so this raises a difficulty for audit. It was also not clear whether the “initial cytotoxic therapy” should have been prescribed in the advanced stage of the disease (rendering taxanes second line treatment for advanced cancer) or whether the anthracyclines could have been given at an earlier stage of progression. Whilst this was clarified in the update of this guidance (no 30) (which advised the former interpretation), this was not formally available to those making decisions at the time to which the audit sample refers. Nor is the definition of “inappropriate” clear, although the phrase makes the use of taxanes possible as a first line treatment in advanced breast cancer. Finally, there is no clear definition of “advanced” disease; the term encompassing locally advanced tumours, recurrent or metastatic disease. We decided to take as the baseline the date of diagnosis, assuming that first treatment would be for early disease but recognise that women actually present for diagnosis at a range of stages. In the case of drugs used in the treatment of Alzheimer’s disease, it was not clear whether a fall in MMSE score from 20 to 18, for example, should indicate that use of the drug should cease. Whilst the guidance on wisdom teeth was unequivocal, the list of pathology required to justify removal was detailed and predictably beyond the expertise of even experienced personnel in Trust audit departments. More clinicians were engaged in this audit than any other that we undertook.

115

We found that in the treatment of breast cancer, the practice in the Trust in which the documents were piloted differed from that in the rest of the country. For Trusts that completed the audit, this did not seem to present a barrier, as they simply wrote in the names of the drugs in use, but this may have contributed to the relatively low number of Trusts completing this audit. There was some lack of clarity about one question (5) on the ovarian cancer form, which referred to a “course of treatment,” and then asked for details of drugs given in the first “cycle”. The question was intended to gather information about the first course of treatments with one drug, or combination of drugs. Some respondents completed information on the drugs given for three successive months, and in that case, we used only the data for the first month. Whilst a maximum of 1000 audit forms were completed for each guidance, at the level of feedback to individual trusts, the numbers were small ( 5

No

Don’t know

1

2

3

4

5

6

7

8

9

When all fifty forms for Orlistat are complete, please return to: Dianne Wright NICE Audit Coordinator YHEC FREEPOST YO405 University of York Market Square Vanbrugh Way Heslington YORK YO10 5ZZ

NB THIS IS A FREEPOST ADDRESS, YOU DO NOT NEED A STAMP

Audits should be completed and returned by 31 March 2003.

Q:\wpwin60\kvl\questionnaires\Z9727b Audit Forms\Orlistat-jan03.doc

HSH/09.01.03

3

APPENDIX 3

3.1 3.2 3.3 3.4

Correspondence with MREC Copies of All Types of Letters Sent to Trusts Copies of Survey Forms Sent to Trusts Including Guidance Specific Copy of General Interview Schedule

3.1

CORRESPONDENCE WITH MREC

Appendix 3

i

3.2

COPIES OF ALL TYPES OF LETTERS SENT TO TRUSTS

Appendix 3

ii

Department of Health Sciences 17 July 2003 «Chief_Title» «Chief_Initial» «Chief_Surname» Chief Executive «TRUST_NAME» «ADDRESS_1» «ADDRESS_2» «ADDRESS_3» «TOWN» «POSTCODE»

1st Floor Seebohm Rowntree Building Heslington York YO10 5DD Professor Trevor A Sheldon Head of Department Telephone E-mail

(01904) 321300 [email protected]

Dear «Chief_Title» «Chief_Surname» RE. NATIONAL AUDITS OF THE IMPACT OF NICE GUIDANCE As you are aware this University has been commissioned by the National Coordinating Centre for Research Methodology on behalf of the National Institute for Clinical Excellence to undertake an audit of the implementation of NICE guidance and we are writing to ask for your further assistance. The study, of twelve sets of NICE guidance in total, is in three phases, consisting of: • • •

A statistical analysis of routine data on prescribing and the use of procedures and devices; An audit of patient notes; Survey of chief executives followed by interviews with a sample of senior managers and clinicians.

Thank you for your enthusiastic participation in the audit phase which is coming to a successful completion. We are approaching you with a view to participation in the final phase. In order to help us understand some of the competing priorities with which you have to deal, we enclose a questionnaire for completion by 7 August 2003. We appreciate that you may not be aware of the detail of implementation of specific guidance and others may have to be involved in the completion of these sections. You may be approached for an interview following the return of the survey. We must stress that we are an independent organisation and that we appreciate the economic and clinical realities and the potential for a wide range of legitimate views to be held regarding NICE guidance. All interviews will be confidential and no trust or individual will be identified with any particular position in our final report, although it would be our intention to acknowledge the help of the 60 acute, mental health and primary care trusts that have participated in the study. / Continued …….

«Chief_Title» «Chief_Initial» «Chief_Surname» Chief Executive «TRUST_NAME»

17 July 2003

We are acutely aware of, and sympathetic to, your many competing priorities, but we suspect that the subject of NICE implementation is already near the top of your agenda. This phase of the study has had ethics approval from the Multi-Centre Research Ethics Committee. The enclosed agreement form indicates your own consent to be interviewed and asks for contact details of lead clinicians appropriate to the guidance who may be approached for consent in the event of our wishing to interview them. Please return this form immediately so that we may make preparations for the next phase. We have enclosed an A5 envelope for the agreement form and an A4 envelope for the questionnaire, when completed. Both of these are FREEPOST, so no stamps are required. However, your and / or your lead clinicians may not wish to participate in this part of the study, and we would appreciate you notifying us if this is the case. We look forward to your response. Yours sincerely

PROFESSOR TREVOR SHELDON Project Manager Encs.

2

Trust Code:

Agreement Form

«TRUST_ID»

National Audit of the Impact of NICE Guidance Consent to Participation «TRUST_NAME» «ADDRESS_1» «ADDRESS_2» «ADDRESS_3» «TOWN» «POSTCODE» The «TRUST_NAME» is prepared to participate in the final phase of this national audit. I give consent to be approached for interview and offer the contact details of key people. I understand that all interviews will be confidential and that no Trust will be identified with any result.

Signed:

.......................................................................................................

Printed Name:

.......................................................................................................

Position in Organisation:

.......................................................................................................

Name and contact details of medical director: ....................................................................................................................................................... .......................................................................................................................................................

Contact Number/Extension:

...................................................................................................

Name and contact details of lead clinicians for (specific NICE guidance): Breast Cancer Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... Ovarian Cancer Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... 3

Hearing Aid Technology Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... Total Hip Replacement Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... Repair of Inguinal Hernia Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... Extraction of Wisdom Teeth Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... Please return in the A5 freepost envelope to: Dr Annette Lankshear FREEPOST YO405 Market Square University of York Vanbrugh Way Heslington YORK YO10 5ZZ

Q:\wpwin60\kvl\letters 2003\Z9727\Letter to chief execs-july03.doc

4

HSH/16.07.03

Department of Health Sciences 28 July 2003 «mhafc_title» «mhafc_initial» «mhafc_surname» «mhafc_job_title» «trustname» «mhafc_address1» «mhafc_address2» «mhafc_address3» «mhafc_town» «mhafc_county» «mhafc_postcode»

1st Floor Seebohm Rowntree Building Heslington York YO10 5DD Professor Trevor A Sheldon Head of Department Telephone E-mail


Dear «mhafc_title» «mhafc_surname» RE. NATIONAL AUDITS OF THE IMPACT OF NICE GUIDANCE As you are aware this University has been commissioned by the National Coordinating Centre for Research Methodology on behalf of the National Institute for Clinical Excellence to undertake an audit of the implementation of NICE guidance and we are writing to ask for your further assistance. The study, of twelve sets of NICE guidance in total, is in three phases, consisting of: • • •

A statistical analysis of routine data on prescribing and the use of procedures and devices; An audit of patient notes; Survey of chief executives followed by interviews with a sample of senior managers and clinicians.

Thank you for your enthusiastic participation in the audit phase which is coming to a successful completion. We are approaching you with a view to participation in the final phase. In order to help us understand some of the competing priorities with which you have to deal, we enclose a questionnaire for completion by 11 August 2003. We appreciate that you may not be aware of the detail of implementation of specific guidance and others may have to be involved in the completion of these sections. You may be approached for an interview following the return of the survey. We must stress that we are an independent organisation and that we appreciate the economic and clinical realities and the potential for a wide range of legitimate views to be held regarding NICE guidance. All interviews will be confidential and no trust or individual will be identified with any particular position in our final report, although it would be our intention to acknowledge the help of the 60 acute, mental health and primary care trusts that have participated in the study. / Continued …….

«mhafc_title» «mhafc_initial» «mhafc_surname» «mhafc_job_title» «trustname»

28 July 2003

We are acutely aware of, and sympathetic to, your many competing priorities, but we suspect that the subject of NICE implementation is already near the top of your agenda. This phase of the study has had ethics approval from the Multi-Centre Research Ethics Committee. The enclosed agreement form indicates your own consent to be interviewed and asks for contact details of lead clinicians appropriate to the guidance who may be approached for consent in the event of our wishing to interview them. Please return this form immediately so that we may make preparations for the next phase. We have enclosed an A5 envelope for the agreement form and an A4 envelope for the questionnaire, when completed. Both of these are FREEPOST, so no stamps are required. However, your and / or your lead clinicians may not wish to participate in this part of the study, and we would appreciate you notifying us if this is the case. We look forward to your response. Yours sincerely

PROFESSOR TREVOR SHELDON Project Manager Encs.

2

National Audit of the Impact of NICE Guidance Consent to Participation «trustname» «mhafc_address1» «mhafc_address2» «mhafc_address3» «mhafc_town» «mhafc_county» «mhafc_postcode» The «trustname» is prepared to participate in the final phase of this national audit. I give consent to be approached for interview and offer the contact details of key people. I understand that all interviews will be confidential and that no Trust will be identified with any result. Signed:

.......................................................................................................

Printed Name:

.......................................................................................................

Position in Organisation:

.......................................................................................................

Name and contact details of medical director: ....................................................................................................................................................... ....................................................................................................................................................... Contact Number/Extension:

...................................................................................................

Name and contact details of lead clinicians for (specific NICE guidance): Alzheimers Disease Name: ........................................................................................................................................... Address: ....................................................................................................................................... Phone Number: ............................................................................................................................. Email Address: ............................................................................................................................. Relevant Guidance: ..................................................................................................................... Please return in the A5 freepost envelope to: Dr Annette Lankshear FREEPOST YO405 Market Square University of York Vanbrugh Way Heslington YORK YO10 5ZZ Q:\wpwin60\kvl\letters 2003\Z9727b Mental Health\Letter to trusts-july03.doc

3

HSH/28.07.03

Letter to clinicians Department of Health Sciences 1st Floor Seebohm Rowntree Building Heslington York YO10 5DD Professor Trevor A Sheldon Head of Department Telephone E-mail


Dear RE. NATIONAL AUDITS OF THE IMPACT OF NICE GUIDANCE As you are aware this University has been commissioned by the National Co-ordinating Centre for Research Methodology on behalf of the National Institute for Clinical Excellence to undertake an audit of the implementation of NICE guidance and we are writing to ask for your further assistance. The study, of twelve pieces of NICE guidance in total, is in three parts: consisting of a statistical analysis of HES and prescribing data; an audit of patient notes and interviews with clinicians. We are grateful for your trust’s participation in the audit of patient notes. This phase has now reached completion and we are approaching you with a view to participation in the final part of the study - that of interviews with clinicians. We seek your consent to interview you for no more than forty-five minutes to explore the factors influencing the implementation of the following guidance. (subject of NICE guidance) We must stress that we are an independent organisation with a comprehensive understanding of economic and clinical realities and of the potential for a wide range of legitimate views to be held regarding the various guidelines produced by NICE. No trust or individual will be identified in our final report. We are acutely aware of, and sympathetic to, your many competing priorities, but we believe that the views of clinicians are of vital importance to the future development of clinical guidance. The enclosed form indicates your agreement to be interviewed. However, you may not wish to participate in this part of the study, and we would appreciate you notifying us if this is the case. We look forward to your response. Yours sincerely

PROFESSOR TREVOR A SHELDON Project Manager

Encs. Q:\wpwin60\kvl\letters\Z9727\Letter to clinicians-jun03.doc

HSH/05.06.03

3.3

COPIES OF SURVEY FORMS SENT TO TRUSTS INCLUDING GUIDANCE SPECIFIC

Appendix 3

iii

Trust Code:

«TRUST_ID»

Page 1

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE Survey of Chief Executives Trust name

1.

Who has the lead for clinical governance within the organisation? Chief Executive Medical Director Risk Manager Clinical Governance Lead Other (Please specify)

1 2 3 4 5

..................................................................................................................................................... General Questions 2.

Is there a strategy for disseminating NICE guidelines to appropriate people? Yes No Don’t know

1 2 3

If yes, move to question 3, If no, move to question 4. 3.

Which of the steps below best describe your strategy? (Please tick all that apply) Registering receipt of NICE guidelines and establishing follow-up actions and responsibilities A targeted approach to those who will use the guideline

1

2

Follow-up reminders 3

Producing a written or electronic version that is easily available to consult 4

Producing or distributing patient-specific versions of the guidance 5

Incorporation into locally-produced protocol 6

Any other steps (please describe) 7

..................................................................................................................................................... ..................................................................................................................................................... .....................................................................................................................................................

Trust Code:

4.

«TRUST_ID»

Page 2

Is there a lead clinician for each piece of guidance? Yes No Don’t know

5.

6.

1 2 3

Do you have a strategy for implementing guidance after dissemination? Yes No Don’t know

1 2 3

Does your organisation have a framework to monitor the: Yes

No

Introduction Implementation Evaluation Recommendations in NICE guidance

1

0

1

0

1

0

1

0

Could you describe this please: ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 7.

Have any of the following NICE guidances been particularly difficult to introduce? Removal of wisdom teeth Selection of prostheses for primary total hip replacement Use of taxanes for ovarian cancer Use of taxanes for breast cancer Hearing aid technology Use of implantable cardiac defibrillators for arrhythmias Use of laparoscopic surgery for inguinal hernia Use of laparoscopic surgery for colorectal cancer Use of docetaxel, paclitaxel, gemcitabine and vinorelbine for the treatment of non-small-cell lung Cancer

8.

Yes

No

D/K

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

Does NICE guidance form part of the clinical audit programme within your organisation? Yes No Don’t know

Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\Survey-july03.doc

1 2 3

HSH/16.07.03

Trust Code:

«TRUST_ID»

Page 3

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE Extraction of Wisdom Teeth 1.

To what extent has this guidance been implemented? Fully Partly Not at all

2.

1 2 3

If this guidance has been fully implemented, could you indicate the factors which contributed to its successful implementation? Clinical lead or local champion High local priority Low cost Agreed in Local Development Plan Single speciality or service High level of evidence or recommendation Other (please describe)

1 2 3 4 5 6 7

..................................................................................................................................................... 3.

If this guidance has not been fully implemented (ie only partially or not at all), could you indicate the main reasons for lack of implementation? Not regarded as a priority Extra investment / resources required Implementation intended but has not yet been carried out Low level of evidence Lack of clinical leadership or local champion Not agreed in HIMP / local development plan or through the SAFF round Too complex or involves other organisations Other (please describe)

1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.

Were there significant cost consequences for the Trust of implementing this guidance? Yes No Don’t know

1 2 3

Trust Code: 5.

«TRUST_ID»

Page 4

If yes, what were these? Facilities Equipment Staff training Recruitment Other (please specify)

1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.

At the time the guidance was issued, was implementing this guidance high on your list of local priorities for new expenditure? Yes No Don’t know

7.

1 2 3

If no, please give examples of other developments with higher local priority:

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.

If the guidance has been implemented, or steps are being taken to implement it, has it displaced plans for any other service developments? Yes No Don’t know

1 2 3

If yes, please give specific examples: ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 9.

Are there any additional comments you would like to make about the role of NICE or of the methodologies developed for the preparation or dissemination of guidance?

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name

Please provide the name and contact details of the lead clinician for this guidance: Email address

Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\WisdomTeethSurvey-july03.doc

Tel No.

HSH/16.07.03

Trust Code:

«TRUST_ID»

Page 5

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE Total Hip Replacement 1.


2.

1 2 3


1 2 3 4 5 6 7

..................................................................................................................................................... 3.


1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.


1 2 3

Trust Code: 5.

«TRUST_ID»

Page 6


1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.


7.

1 2 3


..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.


1 2 3



..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name


Tel No.

Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\HipReplacementSurvey-july03.doc

HSH/16.07.03

Trust Code:

«TRUST_ID»

Page 7

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE The Use of Taxanes for the Treatment of Breast Cancer 1.


2.

1 2 3


1 2 3 4 5 6 7

..................................................................................................................................................... 3.


1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.


1 2 3

Trust Code: 5.

«TRUST_ID»

Page 8


1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.


7.

1 2 3


..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.


1 2 3



..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name


Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\BreastCancerSurvey-july03.doc

Tel No.

HSH/16.07.03

Trust Code:

«TRUST_ID»

Page 9

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE The Use of Taxanes for the Treatment of Ovarian Cancer 1.


2.

1 2 3


1 2 3 4 5 6 7

..................................................................................................................................................... 3.


1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.


1 2 3

Trust Code: 5.

«TRUST_ID»

Page 10


1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.


7.

1 2 3


..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.


1 2 3



..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name


Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\OvarianCancerSurvey-july03.doc

Tel No.

HSH/16.07.03

Trust Code:

«TRUST_ID»

Page 11

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE Hearing Aid Technology 1.


2.

1 2 3


1 2 3 4 5 6 7

..................................................................................................................................................... 3.


1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.


1 2 3

Trust Code: 5.

«TRUST_ID»

Page 12


1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.


7.

1 2 3


..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.


1 2 3



..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name


Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\HearingAidSurvey-july03.doc

Tel No.

HSH/16.07.03

Trust Code:

«TRUST_ID»

Page 13

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE Laparoscopic Surgery for Primary or Recurrent Inguinal Hernia 1.


2.

1 2 3


1 2 3 4 5 6 7

..................................................................................................................................................... 3.


1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.


1 2 3

Trust Code: 5.

«TRUST_ID»

Page 14


1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.


7.

1 2 3


..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.


1 2 3



..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name


Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\InguinalHerniaSurvey-july03.doc

Tel No.

HSH/16.07.03

Trust Code:

«trustid»

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE The Use of Donepezil (Aricept), Rivastigmine (Exelon) and Galantamine (Reminyl) for the Treatment of Alzheimers Disease 1.


2.

1 2 3


1 2 3 4 5 6 7

..................................................................................................................................................... 3.


1 2 3 4 5 6 7 8

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 4.


1 2 3

Trust Code: 5.

«trustid»


1 2 3 4 5

..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 6.


7.

1 2 3


..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 8.


1 2 3



..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... 10. Name


Tel No.

Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\AlzheimersMentalHealthSurvey-july03.doc

2

HSH/28.07.03

Trust Code:

«trustid»

NATIONAL AUDIT OF THE IMPLEMENTATION OF NICE GUIDANCE Survey Trust name

1.

Who has the lead for clinical governance within the organisation? Chief Executive Medical Director Risk Manager Clinical Governance Lead Other (Please specify)

1 2 3 4 5

..................................................................................................................................................... General Questions 2.

Is there a strategy for disseminating NICE guidelines to appropriate people? Yes No Don’t know

1 2 3

If yes, move to question 3, If no, move to question 4. 3.

Which of the steps below best describe your strategy? (Please tick all that apply) Registering receipt of NICE guidelines and establishing follow-up actions and responsibilities A targeted approach to those who will use the guideline

1

2

Follow-up reminders 3

Producing a written or electronic version that is easily available to consult 4

Producing or distributing patient-specific versions of the guidance 5

Incorporation into locally-produced protocol 6

Any other steps (please describe) 7

..................................................................................................................................................... ..................................................................................................................................................... .....................................................................................................................................................

Trust Code:

4.

«trustid»

Is there a lead clinician for each piece of guidance? Yes No Don’t know

5.

6.

1 2 3

Do you have a strategy for implementing guidance after dissemination? Yes No Don’t know

1 2 3

Does your organisation have a framework to monitor the: Yes

No

Introduction Implementation Evaluation Recommendations in NICE guidance

1

0

1

0

1

0

1

0

Could you describe this please: ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... ..................................................................................................................................................... .....................................................................................................................................................

7.

Has the following NICE guidance been particularly difficult to introduce? The Use of Donepezil (Aricept), Rivastigmine (Exelon) and Galantamine (Reminyl) for the Treatment of Alzheimers Disease

8.

Yes

No

1

D/K

2

Does NICE guidance form part of the clinical audit programme within your organisation? Yes No Don’t know

Q:\wpwin60\kvl\questionnaires\Z9727b Chief Execs\Survey-MH-july03.doc

2

3

1 2 3

HSH/28.07.03

3.4

COPY OF GENERAL INTERVIEW SCHEDULE

Appendix 3

iv

APPENDIX 4

Additional Data to Support Analysis About Sets of Guidance 4.1 4.2

Extraction of Wisdom Teeth: Most Common Other Reasons for Extraction NHSPSA Benchmarking Database

4.1

EXTRACTION OF WISDOM TEETH: MOST COMMON OTHER REASONS FOR EXTRACTION

Appendix 4

i

Appendix 4.1:

Extraction of Wisdom Teeth: Most common “other” reasons for extraction

Reasons for extraction

Total incidence

1

2 10 7 11

82 Impacted 56 1 First episode of pericoronitis* 36 3 Pain 21 Partially erupted 19 8 Non functional 19 3 Caries in 7 15 Pain (recurrent) 14 1 Caries 14 2 2 Food trap / packing / lodgement 12 1 Swelling 11 Over-erupted 8 1 Draining of cyst 8 1 1 Persistent cheek biting 7 1 1 Abscess(es) 6 Traumatic / erupted 6 1 Associated with pathology in 7 6 Surgeons decision 5 1 Erupted 4 1 Fractured mandible / old fracture set 3 Discharge of pus 3 Soreness 3 1 Pre-Osteotomy 3 GDP request 3 Troublesome 2 Asymptomatic * Not referred to dentist for judgement - identified in question 5

3 30 4 2 2 3 4

4 1 0

5 1 1

6 1 9

1 1 1 1

1

1

2

1

1

7 10 5 3 6 4 3 3 1 1 5 2 2

8 1 0

9 0

Trust 10 11 2 8 9 4

4 1

2 1 1 1 1 2

2 3 1

1

1 1 2 1 1

1

12 X X X X X X X X X X X X X X X X X X X X X X X X X

13 11 6 1 9

1

1 1 1 1


15 3 3 1

1 2 3

16 10 0 7 2 1 1 5 3 6 1 7 1

2 1 1 1 6

17 4 3 4 2 3 3 2

2 2 1 2

1 1

2 1 3 1 1

2

18 1 0

19 0

3


4.2

NHSPSA BENCHMARKING DATABASE

Q:\wpwin60\kvl\reports\Z9727\FinalAppendices-mar04.doc

Appendix 4

HSH/05.03.04

ii

Appendix 4.2:

NHSPSA benchmarking database

Prostheses

10 yr UK launch benchmark Date delared

10 yr revision rate

3 yr benchmark 3 yr revision rate declared

1 Charnley

1962

yes