Melbourne Institute of Technology by the Department of Human Services and Health. Congenial ... The Royal College of Surgeons: Patient Satisfaction with.
THE ROLE OF PATIENT SATISFACTION SURVEYS IN A NATIONAL APPROACH TO HOSPITAL QUALITY MANAGEMENT
Mary Draper and Sophie Hill
Faculty of Social Sciences and Communications Royal Melbourne Institute of Technology
for Department of Human Services and Health October, 1995
ACKNOWLEDGMENTS This report was funded through the Centre for Applied Social Science Research at the Royal Melbourne Institute of Technology by the Department of Human Services and Health. Congenial space and administrative support was provided by URCOT. Particular thanks to Bruce Wilson, Jenny Crozier and Karen Davies. Research assistance for material in chapter 7 was provided by Merinda Northrup. Many other individuals were generous with their time and reflections.
TABLE OF CONTENTS EXECUTIVE SUMMARY CHAPTER 1: INTRODUCTION Structure of the Report Quality in Health Care Consumers and the Right to Comment Satisfied Patients – Better Outcomes The Market Relationship Assumptions About Consumer Feedback
1 2 3 5 6 7 7
PART ONE: RESEARCH DATA ON CONSUMERS’ EXPERIENCES AND ASSESSMENTS OF HOSPITAL CARE Introduction to Chapters 2-8
11
CHAPTER 2: CONSUMER REPORTS OF HOSPITAL CARE Introduction Consumer Views Issues Emerging From the Research
13 13 16 24
CHAPTER 3: RESEARCH BY GOVERNMENTS INTO CONSUMERS’ SATISFACTION WITH, AND VIEWS ABOUT HOSPITAL CARE FOR CHILDBIRTH Consumer Views of Maternity Services in Western Australia Consumer Views of Birthing Services in Victoria Issues Arising from the Research
29 29 32 34
CHAPTER 4: RESEARCH BY HOSPITALS The Mercy Hospital, Victoria – Access and Equity Issues The Adelaide Medical Centre for Women and Children Issues Emerging from the Research
39 39 41 42
CHAPTER 5: COMPLAINTS AND ADVERSE INCIDENTS Main Areas of Complaint Issues Arising from the Data Research into Adverse Incidents Issues Emerging from the Research
45 45 47 47 49
CHAPTER 6: OTHER REPORTS OF HOSPITAL EXPERIENCES Links Between Hospitals and Services in the Community Older People in Dubbo – Their Use and Experiences of Hospital Services Patient Satisfaction Surveys of Specific Services
51 51 52 53
TABLE OF CONTENTS CHAPTER 7: STATE GOVERNMENT PATIENT SATISFACTION SURVEYS New South Wales Victoria Queensland Western Australia South Australia Northern Territory Tasmania Summary Comments on State Patient Satisfaction Surveys
55 55 64 68 71 73 74 74 74
CHAPTER 8: SUMMARY OF PART ONE Consumer Research State Patient Satisfaction Surveys Government Reviews/Birthing Surveys Hospital Projects Aimed at Particular Populations Academic Research Issues Affecting Consumer Feedback Mechanisms Promoting Changed Models of Care
79 79 80 83 83 84 84 85
PART TWO: METHODOLOGICAL AND ORGANISATIONAL ISSUES CHAPTER 9: REVIEW OF THE ISSUES IN SURVEYING PATIENT SATISFACTION The History and Context of Measuring Satisfaction How is Patient Satisfaction Defined? How is the Data Used? The Scope of Satisfaction Surveys Issues Related to the Method Chosen Policy Issues Summary CHAPTER 10: PATIENT SATISFACTION INITIATIVES IN THE UNITED KINGDOM Centre for Health Economics, York 1989 National Consumer Council 1990 King’s Fund Centre for Health Services Development: Consumer Feedback Resource 1991 Greater London Association of Community Health Councils 1992 The Royal College of Surgeons: Patient Satisfaction with Surgical Services (1991-1993) Audit Commission 1993 College of Health: Consumer Audit Summary Remarks
89 90 92 94 95 96 100 103 105 105 108 109 113 114 118 120 122
TABLE OF CONTENTS CHAPTER 11: PATIENT SATISFACTION IN THE UNITED STATES Office of Technology Assessment Patient Judgements of Hospital Quality Medical Outcomes Trust The Picker-Commonwealth Survey Public Citizen’s Health Research Group Summary
125 125 129 131 131 134 135
CHAPTER 12: WHAT IS BENCHMARKING? Benchmarking is About Process Benchmarking is About Finding Best Practice Who Decides What are the Criteria that Establish ‘Best Practice’? Summary
137 137 138 139 140
PART THREE: OPTIONS AND RECOMMENDATIONS CHAPTER 13: PATIENT SATISFACTION AND HOSPITALS: OPTIONS AND RECOMMENDATIONS State Surveys: Summary of Findings Consumer Research: Summary of Findings Methodology: Summary of Findings Options for a National Approach to Consumer Feedback Concurrent Recommendations to Improve the Quality of Patient Satisfaction Work in Australia
145 145 146 147 148 153
APPENDIX A: SUMMARIES OF CONSUMER REPORTS OF HOSPITAL EXPERIENCES
159
REFERENCES
173
EXECUTIVE SUMMARY This report was commissioned as part of the National Hospital Quality Management Program. Its purpose is to report on the feasibility of national benchmark questions for patient satisfaction surveys. The substantive intent here is to reflect consumer perspectives within the exercise of setting national quality benchmarks. We have undertaken a thorough appraisal of the issues relevant to patient satisfaction surveys. This report is very critical of much of the activity undertaken under the rubric of patient satisfaction and subjects the concept itself, as well as its operation, to critical analysis. The methodology of patient satisfaction surveys ⇒ There has been a shift in thinking about the role of the consumer as patient. It is more likely now that researchers want to know what consumers think; that they accept that what the patient tells them is an accurate reflection of what happened; and that this information can be used to improve the quality of care. ⇒ Coinciding with this shift, and related to it, is a serious questioning of the conception of satisfaction as a unitary concept whose causal variables can be measured. ⇒ Researchers now are more in favour of using several research methods, drawn from qualitative and quantitative research, to inform their survey instruments, and gather data. ⇒ The methodological complexities of patient satisfaction research are considerable and should not be downplayed. ⇒ Certain groups of people, whose social position or state of health may make them vulnerable to poorer quality care, are extremely difficult to reach via the conventional questionnaire. The use of patient satisfaction surveys ⇒ There is a need to develop greater expertise, greater support to those doing patient satisfaction work, more coordination at hospital level, greater commitment to acting on the results, and involvement of consumers at all stages of patient satisfaction work, including acting on the results. ⇒ Much patient satisfaction work treats consumers passively, that is, as providers of information, which administrators and providers may or may not do something with. ⇒ There is a need to develop appropriate infrastructure for undertaking patient satisfaction work, to develop the level of expertise, and to consider the role which independent organisations based on a consumer perspective might play in undertaking consumer appraisal activities and research in joint activities with hospitals and purchasers.
i
⇒ There are questions about whose views are sought in patient satisfaction surveys and there is a challenge to incorporate the view of those consumers who are most in need of good quality services, who are very ill, whose views are regarded as being difficult to obtain and those who are often at risk in the quality of their care. Benchmarking ⇒ The purpose for undertaking consumer feedback activity needs to be articulated and owned by hospitals as part of their overall organisational strategy. ⇒ The processes that produce patient satisfaction are more important to understand than producing a league table of patient satisfaction scores. ⇒ The component elements that contribute to patient satisfaction or dissatisfaction are the key elements to concentrate on, rather than overall measures of satisfaction. ⇒ Benchmarking is not a simple process of comparing performance scores on a state/territory or national basis, but a process of identifying best practice and seeking to learn from it. The common themes which emerged from the consumer research ⇒ There are several related issues which cluster around the general mantle of: • communication, • being treated with respect, and • being involved in decision-making. ⇒ Sub-issues related to these are: • informed consent, • privacy, and • access to interpreters. ⇒ People want more information on all aspects of the hospital stay, and in particular on pharmaceuticals. They want information in verbal and written forms, and in other languages. ⇒ Discharge planning is frequently inadequate for all types of patient. There is little involvement of family or carers either. ⇒ The role of the professional team is critical. People are more positive about: • multidisciplinary teams, • good teamwork, and • communication between professionals. This includes communication between professionals in and outside the hospital. ⇒ A related sub-issue for public patients is seeing the same principal doctor, within a team, ie continuity of care. ii
⇒ For some people, access to hospital care is important. This includes:• the cost of, or the lack of public transport, • they are not informed about waiting, • they can’t get in for some reason. The philosophy adopted by a specialty area may have a significant impact on the way care is structured and the approach taken towards the consumer. Some specialties treat the person as an equal, promote self management and teamwork, involve other professions, and/or develop a model of good practice. These treatment models are appreciated by consumers who wish them promoted more widely. Consumer issues affecting patient satisfaction surveys ⇒ Anonymity is very important, particularly for regular users of hospital services, carers, and people in city fringe, rural or remote areas, where there may be no choice among hospitals, or staff are known. ⇒ Some people fear that criticism of a service will result in its loss. ⇒ The severity of the condition may have a significant bearing on how the consumer copes with the hospital stay or use of services. ⇒ Not much is known of the separate or complementary needs of patients and their families or carers. Consumer research Consumer organisations could benefit by the following: ⇒ Research projects need to be better resourced. ⇒ Research staff would continue to benefit from training on methods, or would benefit by access to a clearing house and specialist advice on consumer feedback research. ⇒ There should be more joint projects, for example, between a hospital or private insurance company and consumer group, or between a specialist college and a consumer group. Options for a national approach to consumer feedback The report does not recommend national reporting on an overall patient satisfaction score. While this is canvassed in the discussion of options, alternative options for an overall national approach are outlined and discussed. The options are not mutually exclusive and a staged development of some aspects of each option or approach would be possible. Implementation of any approach is only effective if consumer feedback is part of hospital services development, whether this is at individual hospital level, or within networks of
iii
hospitals. Patient satisfaction surveys need to be undertaken in a context that goes beyond comparison of results to a context where hospitals use benchmarking, or other approaches, to establish what the processes are that lead to good practice. Option 1. Report on comparative national data on patient satisfaction This is the type of option which was envisaged when this report was commissioned. If this option is favoured, there are two ways it could be done A. A common set of questions are asked in current statewide surveys The Report suggests that this is not straightforward, and that comparisons between state patient satisfaction surveys as currently constructed are not valid, due to differences in design, administration and sampling. At hospital level, these difficulties are multiplied. B. Development of a special purpose-built national survey, undertaken under the auspice of the Department of Human Services and Health, based on a population sample (of hospital users) Within Option 1, this is our preference. The Department of Human Services and Health could implement this by initiating a rolling program, focussing on different areas, eg day surgery, outpatients, casemix related information, patient information, communication, hospital discharge, availability of interpreters (as reported by consumers). The data would then be used strategically to highlight issues of concern, rather than expecting that hospitals can use the data directly to improve services. It is open to the Commonwealth to evaluate further the comparative contributions of patient reporting (HCA type) or patient rating (Picker-Commonwealth type) approaches to the information that would fit its purposes. Option 2. Identify quality issues highlighted by existing consumer and patient satisfaction research and report on the more measurable aspects This option would not see patient satisfaction reported in its own right. However, patient satisfaction reports and other forms of consumer feedback would be used to identify quality issues of concern to consumers and there would be national reporting on these. Hence, it would be performance on the quality issues themselves, not patient satisfaction, which is reported. Areas such as the timely provision of information, access, hospital discharge, and continuity of care would be amenable to such an approach. Broader issues of communication may be more effectively dealt with at more local or specialised levels, eg by clinical colleges, hospital networks, health areas or states. The British Royal College of Surgeons’ patient satisfaction survey is a good model of such a strategy. iv
If this overall approach were adopted, we recommend that it includes a focus on issues important to particular population groups, eg access to interpreters, rural access. There would need to be consultation with indigenous health workers to establish those national hospital indicators most appropriate to Aborigines and Torres Straight Islanders. Option 3. Require hospitals to undertake consumer feedback but do not require national reporting of results The Commonwealth could legislate to require hospitals to obtain and make use of consumer feedback. This could be done in the context of consumer charter development. Alternatively, this could be negotiated between the Commonwealth and the states as part of the Medicare Agreement. This could be done with or without formal national reporting on the processes or results. This option requires further development of principles, components and indicators for consumer feedback. Concurrent recommendations to improve the quality of patient satisfaction work in Australia These recommendations are about improving the infrastructure for consumer feedback and patient satisfaction activity. Without some improvement in the quality of patient satisfaction survey work, none of the options above will be effective at hospital level. If these recommendations alone were adopted, this would represent a considerable improvement on current practices and arrangements. There is a need to develop a range of centres of expertise that are in the public realm. Publication disseminates expertise and ideas and contributes to the development of knowledge. It also has the important effect of making information and ideas available to consumers, and consumer organisations. There is value for hospitals in being able to call on a range of professional expertise to assist them in designing, administering, interpreting and using patient feedback, and in some circumstances, being able to take advantage of broader sampling and analysis expertise. In addition, we consider that the model of ‘consumer audit’ is a worthwhile addition to the range of ways that could be available to obtain consumer feedback. This would require the development of consumer based expertise to conduct consumer audit. Finally, there is an opportunity for the professional Colleges to consider patient satisfaction instruments as part of their own quality programs. Mary Draper Sophie Hill October, 1995
v
CHAPTER 1: INTRODUCTION This report was commissioned as part of the National Hospital Quality Management Program. Its purpose is to report on the feasibility of national benchmark questions for patient satisfaction surveys. The substantive intent here is to reflect consumer perspectives within the exercise of setting national quality benchmarks. Consumer perspective is an extremely important element within the National Hospital Quality Management Program and responds to arguments by the health consumer movement that quality in health has tended to be defined from clinicians’ perspectives, and that there needed to be a focus also on quality of health care as seen through patients’ eyes. In this context, we have undertaken a thorough appraisal of the issues relevant to patient satisfaction surveys. The questions we ask throughout include the following: • is overall patient satisfaction something which lends itself to a benchmarking approach? • if not, are there component elements of patient satisfaction which might lend themselves to a benchmarking approach? • what are the key issues about the purposes, design, and use of patient satisfaction surveys? • what have people said so far about their experiences of hospitals and how has this been obtained? In this report, we aim to canvass the literature which will facilitate consideration of these questions. This report is very critical of much of the activity undertaken under the rubric of patient satisfaction and subjects the concept itself, as well as its operation, to critical analysis. Australian hospitals have made use of patient satisfaction surveys over the last few years, partly in response to requirements in Australian Council on Healthcare Standards (ACHS) accreditation standards for a mechanism to incorporate patients’ feedback into quality assurance. In this report, we have not systematically surveyed hospitals, although we have spoken to a number of individuals in hospital about their experience of undertaking patient satisfaction activity, and have collected material from them. Their experiences have been important in shaping our review of the literature and our recommendations. Our overall impression of the state of patient satisfaction activity in Australian hospitals is that: • there is a great deal of good will and enthusiasm towards patient satisfaction surveys in many hospitals; • there is a lack of expertise in the design and execution of patient survey work; • there is an expressed need for advice and information about undertaking and improving the quality of patient satisfaction activity;
1
• there are large numbers of very poorly designed patient satisfaction surveys used in hospitals; • nurses are most likely to be the ones who are interested in getting patient satisfaction work undertaken; • there are often considerable difficulties in getting hospitals to act on the information thrown up by their own patient satisfaction surveys; and • there is very little strategic use of patient satisfaction surveys by hospitals. Westbrook (1993) suggest also that hospitals do not have access to the expertise to develop valid and reliable tools to measure patient satisfaction, that existing surveys focus on hotel services, and that there is confusion about sampling and response rates. She also proposes that health care providers and managers lack understanding of the role patient satisfaction plays in health care. All this makes it a very good time to produce a report on patient satisfaction surveys, and to consider their role in hospital quality management, because in addition: • • • •
there is a new focus on health outcomes; there is an important new focus on hospital-wide quality; there is a lot of health debate about methods; and there are some good recent examples of getting quality feedback from consumers.
A more recent entry to the field has been that of state health departments, many of whom are now undertaking state wide patient satisfaction surveys. This activity is discussed in chapter 7. In our work for this report, we have considered carefully how any national program of quality management both fits with this prior activity and builds on what is already occurring. The report also looks at the roles of other stakeholders, such as consumer organisations. This report will be very critical of much of the practice around patient satisfaction surveys, and will argue that the methods have been problematic. We do not suggest, however, ‘throwing the baby out with the bath water’. This report is strongly supportive of the principle of obtaining structured consumer feedback and consumer appraisal of hospital services, and of the role patient satisfaction-related research could play within this. Structure of the report The report is written in three sections. The first section (chapters 2-8) examines what Australian health care consumers have said and written about their experiences with hospitals. The first of these chapters sets out to identify those issues that consumers themselves have identified as key issues. This is then followed by a series of chapters which report on research that has been undertaken in Australia about consumers’ views of their hospital care, or which contribute to such a picture. This includes state based patient satisfaction surveys of different kinds, published patient satisfaction surveys, complaints data
2
and adverse events, and some qualitative work undertaken by hospitals into the views and experiences of particular hospital populations. Section 2 (chapters 9-13) canvasses a set of methodological, organisational and policy issues in relation to consumer feedback. This includes a discussion of the methodological debates around the concept of patient satisfaction; this is not a straight forward or simple concept. This section also canvasses literature about patient satisfaction from the United States and Britain. The US literature is pre-occupied with patient satisfaction methodology, although we also discuss some material about strategies of making information available to consumers. The British literature also discusses a number of methodological issues, but in particular looks at organisational issues about consumer feedback and the policy context of consumer feedback. Chapters 13 examines the concept and use of benchmarking. The final section makes a number of proposals and recommendations about patient satisfaction surveys in the context of national approaches to hospital quality management. This introductory chapter briefly outlines some assumptions underlying our approach. Quality in health care The current broad pre-occupation with quality management is one whose literature is essentially derived from manufacturing. Zeithmal and her colleagues (1990) argue that the most surprising aspect of the evolution towards a service economy is the comparative poverty within the quality management literature of attention to service quality as opposed to product quality. Essentially, the quality literature was developed in manufacturing, often with a focus on improving competitiveness. The conceptual basis of such quality literature is a primary focus on systems, with attention to how all the processes of manufacturing affect the final product. Services, on the other hand, Zeithmal and her colleagues (1990:15-16) argue are basically intangible. They argue that quality cannot be preset, as it can be in manufacturing where the quality of the product is already assured by the time it goes on the shelf. In services, good quality is only present in the performance and the experience of the service. Consequently, quality criteria are difficult to set and performance evaluation is difficult to capture. They argue that services, especially those with a high labour content, are heterogeneous, unlike products, and vary from person to person, so that standardisation is inherently problematic. Finally, they argue that production and consumption are inseparable, that is, quality is delivered at the point of service delivery, and in the interaction between the customer and the provider. A survey of the literature that does focus specifically on service quality suggests that: • service quality is more difficult to evaluate than goods quality. Consumer assessment of service quality in health care is more difficult and complex than consumer assessment of the quality of their motor car; • consumers assess services on the process of service delivery as well as the outcome, and
3
• consumers have a crucial role in setting the criteria for evaluating service quality (Zeithmal et al 1990:16). It is clear that in the services industry, the quality of customer delivery is inseparable from the staff who deliver the services, so that staff issues are a crucial key to good service delivery. However, while it is true that good quality is present at the point of service delivery, there are a range of organisational processes that need to work to ensure good quality at the point of delivery and which make a difference to how staff are able to deliver good quality services. Staff who have to continually apologise for systems that don’t work cannot guarantee the best service. An area like health, however, raises a further set of issues for quality, where service delivery is often in the context of relationships and where the nature of those relationships have a particular, if contested, character. Clinicians often object to the idea of a consumerist relationship, because they see the doctor-patient relationship as special and implying a sense of commitment to the patient. While the consumer movement has contested aspects of that relationship, in particular those of power and authority, it nevertheless also sees this relationship as a special one, and which they often describe as a partnership. We start from the point that the idea of co-production is a good one to describe this relationship; that is, the clinician and patient/consumer together make decisions about health and illness and work together to produce health or deal with illness. Health is an area that is central to people’s experiences of themselves as human, and moments of birth, illness and death are defining moments of being human. People also enter the relationship with providers as citizens with the rights and responsibilities of citizens. The quality focus in health traditionally was resistant to the argument of pioneers such as Codman in the early 20th century who argued for a focus on ‘end results’ (outcomes), preferring instead to focus on the accuracy of clinical decision-making, pursued through strategies of peer review and clinician education and certification. It tended to be in-house. There was little room here for consumer perspectives or information. In recent years, the quality debate in health has broadened out, drawing on Donabedian’s triad of structure, process and outcome, but also drawing more broadly on sources of information, including consumer information and opinion. Here again, Donabedian’s account (1992) of the roles of consumers as standard setters, informants and evaluators reflects the shift towards different relationships between health care providers and consumers. The quality management approaches now also focus on the whole hospital, and the contribution of all elements to quality of care. There are aspects of the quality literature about products that can make a contribution to quality management in hospitals, such as the analysis of system effects. There are aspects of the quality literature about services that are helpful. The traditional peer review approaches also remain a critical bedrock of quality in health. But the nature of the relationships in health adds a complexity that requires additional conceptualisation. Consequently, patient satisfaction feedback on, and appraisal of, their hospital care has components that cannot be addressed in the same way as asking consumers about their satisfaction with a banking product, or the servicing of their telephone. For example, it may be a source of irritation to have to queue for a bank teller, but this is very different from being faced with the information that a person has a potentially terminal illness and dealing with the
4
communication with the clinician about this. Chapter 9 (on the issues in surveying patient satisfaction) discusses this in some detail. Patient satisfaction surveys as part of the broader set of ideas about consumer feedback have emerged for a range of different reasons. In the following discussion, we suggest that there have been three major contributing pressures, the emergence of an explicit consumer voice in health, the desire to get good outcomes through compliance in treatment, and the recent influence of market ideas in health. These do not share the same assumptions about the legitimacy and role of consumer involvement. We pursue this further by reflecting on different conceptual approaches to consumer improvement. This discussion is intended to draw out and articulate some of the values that underlie these different approaches, to demonstrate some of the conflicting assumptions at work, as a background to the more detailed discussion of the literature undertaken in the report. Consumers and the right to comment The consumer movement in Australia has in different ways been seeking consumers' views for many years, often in the context of a policy initiative, ie. the organisation is seeking to influence government or providers about the need and ways to change health services. A good example in Australia is the Australian Pensioners' and Superannuants' Federation which has been conducting focus groups and undertaking surveys and questionnaires since the 1980s, and which has produced a substantial body of work. This provides a detailed picture of people's interactions with health services. Overseas, a number of academics have been researching people's views on health services since the 1970s, such as Ann Cartwright and Winifred Raphael in Britain, but there has been little comparable activity in Australia. The relative paucity of this type of research in Australia is confirmed by the recent literature reviews for this report, a recent report by the authors on general practice (Hill and Draper 1995), and research on women’s experiences of treatment for breast cancer (NH&MRC Working Party on the Treatment of Diagnosed Breast Cancer 1995). Underlying the research effort of consumer groups has been the clear premise that people who use health services (users, families or relatives in contact with services, carers, and the public as taxpayers at large) all have a social right as members of a community, and as individuals, to be heard. Their views are considered essential in terms of deciding the type of services and the level of need, and in evaluating the service in relation to their needs. In Australia, consumer organisations have not been particularly concerned with the concept of satisfaction, and only in recent years have they developed policies in relation to the promotion of good quality care. However, there is a large body of work which should be drawn on, in which people articulate their good and bad experiences and their expectations. It is fair to say that the consumer movement is often sceptical of formal patient satisfaction surveys. This reflects their experience of often poorly designed questionnaires, which do not allow them to reflect their concerns, often appear to consumers to be overly pre-occupied with amenity (there is nothing in the literature to support the preoccupation that many surveys have with amenity), and in the past often have not been allowed to include questions about clinical care provided by doctors.
5
There is also a substantial body of comment from consumers which has fed into government policy reviews, such as the Birthing Review in Victoria, and from complaints authorities. Consumer organisations, such as the Health Issues Centre, have consistently argued that complaints data should be collected systematically, and be regarded as essential information for quality assurance and quality management. Satisfied patients – better outcomes There is some research into the relationships between satisfied patients and improved health outcomes which has shown that satisfied patients may be more prepared to seek help, comply with treatment and maintain an ongoing relationship with the doctor (Carr-Hill 1992:236). More recently, there has been an interest in satisfaction as an outcome in itself, which tends to be linked to economic appraisal of the costs and benefits (broadly defined) of particular treatments. Both the patient satisfaction literature and the newly emergent health outcomes literature, as typified in the Medical Outcomes Short Form 36 (SF-36) contain similar debates over the type of information obtained from quantitative methods compared to qualitative methods. In other words, does the method of gaining the user's experiences and views lead to an accurate reflection of what they think, and do structured questionnaires capture the meaning of people's experiences as well as less structured interview methods (Hill and Draper 1995). This report describes research in patient satisfaction where the method chosen influences the type of data collected. In both areas, the quantitative method reduces the complexity of the experience. The actual language and idea of ‘compliance’ itself is one which is contested by consumers. Chapter 9 suggests that ‘compliance’ approaches were the initial stimulus for patient satisfaction work. Australian consumer organisations are very wary of the notion of the ‘compliant' patient. A study by the Australian Consumers' Association and the Combined Pensioners' Association of New South Wales (1988), found that many older people are very compliant, use the same doctor all the time, rarely ask questions, and view the doctor as the main source of information. However, at the same time, many had been prescribed tranquillisers for substantial periods with little initial or ongoing discussion about side effects, addictive properties. The report argues that compliance is not the issue: that whether the care is necessary is the issue, and the consumer is unlikely to raise that with the doctor. Consumers who ask more questions are more likely to bring the significance of symptoms to the doctor, are more likely to draw attention to medication they already take, and more likely to alert doctors and nurses to issues in treatment and medication. Nor is it always clear that patient satisfaction is always the best goal. Unsatisfied patients may be precisely the ones who draw attention to gaps in medical knowledge and treatment, who become advocates for improved health care, and whose fighting spirit contributes to their own survival or increased length of life. The research for this report indicates that across a range of literature, the direction of change is that consumers consider that it is important to be involved in decision making about their health and that the language of cooperation in health decisions and treatment is more
6
appropriate than the traditional language of ‘compliance’, given the undertones in this language. The market relationship Advocates for a market view of health are not new, but have been given a boost by recent reforms under the Conservative Thatcher and Major governments in Britain, and in New Zealand. This represents in part dissemination of ideas from the US, which has a stronger reliance on these approaches than many other countries. We note in passing that it is important to understand these different policy contexts, because the ‘solutions’ that emanate from these environments are ones that arise in particular health care contexts, and often appear in international application to be ‘solutions’ in search of problems to solve. Chapters 10 and 11 (on patient satisfaction approaches in Britain and the United States) attempt to place some of the consumer feedback and patient satisfaction literature in context by locating it in policy approaches and debates within those health systems. In Britain there has been a great deal of debate about the reform process which began with the Griffiths Report of 1983. McIver (1992b) argued that Griffiths confused individual needs with community needs, which are not always the same. McIver also argues that the voices of the most disadvantaged in the community will not be heard or realised in practice if they are not consulted in particular and able to influence the provision of services (1992:19). There is nevertheless a glimpse within this approach of the major shift it would take to put consumers at the centre of planning and decision making within health care, and hospital care in particular. Approaches in Australia which draw from this approach have tended to concentrate on information aimed at consumers, customer focus, and competition between providers, the latter already a feature of the private sector in Australia. In reality, these approaches have shifted the focus very little. The most frequent method of getting consumer feedback has been by self-completion patient satisfaction questionnaires, despite doubts over the value of the method in obtaining genuine consumer opinion, and concern about the passive role this method often assigns to consumers (McIver 1992a:17). Patient satisfaction surveys appear to value consumer opinion, but are often used to justify services, given that satisfaction levels are usually high in these surveys (85 per cent satisfaction, or thereabouts, is standard). The challenge for patient satisfaction surveys is to get at information about dissatisfaction and to explore the reasons for it. Patient satisfaction surveys frequently treat consumers passively, simply as impersonal sources of information. Market research is rarely participative in its execution. Patient satisfaction surveys remain often a form of consumer feedback as patronising in its application as the traditional health relationships that consumers have criticised.
7
Assumptions about consumer feedback Table 1 illustrates some of the different assumptions at work in ways of thinking about more patient focussed health care. It is derived from some of the discussion in the Health Issues Centre 1992 publication Casemix, Quality and Consumers. Table 1: Approaches to Consumer Empowerment Scientific approaches
Market solutions
Legal Approaches
Democratic participation
Rely on objective measurement and statistics
Rely on either market information as a way of influencing health services, or more responsive services which attract consumers
Rely on legally, or administratively defined rights, and access to judicial and semi-judicial institutions
Rely on ways for consumers to participate individually and collectively in health decisions
Consumers are subjects of research
Consumers are informed choosers
Consumers are citizens with rights
Consumers are equal partners and citizens
Strategies standards, outcomes, guidelines, variations, classification, patient surveys
Strategies information on providers, marketing, statement of expectations, consumer surveys
Strategies health charters, right to complain, legal redress, legislation, transparent decision making, advocacy
Strategies consultation, involvement in decisions, participation in planning decisions, hand held records, representation on committees, accountability to consumers
We argue here that the various ways of obtaining consumer feedback do not necessarily share the same assumptions about consumer involvement. Some approaches only allow for a minor role for consumers. Some do not give credit to consumer views and further, they reduce the complexity of people’s experiences. This may minimise the degree of dissatisfaction and obscure the reasons for it. While ideas about the value of consumers’ ideas and experiences derive from democratic participative approaches, patient satisfaction surveys come out of both scientific and market approaches. It is not our argument that this makes them inherently problematic, but to illustrate their limitations as a form of consumer feedback. In other words, they do not
8
involve consumers in planning and evaluation, nor do they usually involve consumers in discussions and decisions about solutions. This report will argue for more science in patient satisfaction surveys. We are sympathetic to the view that patient satisfaction surveys that are poorly designed and executed are a form of censorship of consumers’ views (Whitfield and Baker 1992:152). At the same time, we will argue that a proper use of science in consumer feedback supports the idea that there is not a single method for obtaining consumer feedback. The most popular form of patient satisfaction feedback has been that of closed choice questions administered as selfcompletion questionnaires, undertaken by post or on the ward. However, recent good practice would recommend the use of different methods, methods which are appropriate to the particular groups of consumers and to the area of inquiry. In particular, qualitative and quantitative research would both be seen to have a contribution to make. Quantitative methods will often tell you where to look, but will not tell you any detail about what you will find, or what to do about it. Better science in patient satisfaction surveys also entails experimentation and accumulation of a body of knowledge. At a couple of points in the report, it is clear also that the ‘science’ of the construction of patient satisfaction surveys is not exempt from the usual politics of the health care sector. Indeed, it often appears that if hospitals and clinicians regarded consumer information as valid information, they could avail themselves of simpler, less expensive, and often more accurate accounts of consumer experiences. In this report, we treat consumer feedback as the primary concept rather than patient satisfaction, and see patient satisfaction surveys as one means of obtaining consumer feedback. We also argue that deciding on the purpose of consumer feedback is the critical first step in undertaking research into patient satisfaction and patient experiences. This report will argue that patient satisfaction surveys are not an end in themselves. It will suggest that: • the purpose is quality improvement, • obtained by involving consumers, • through using appropriate methods to get feedback. Well designed patient satisfaction surveys have a role to play in obtaining consumer feedback, along with other approaches. There is a challenge, however, to define what ‘well designed’ entails, which this report discusses.
9
10
PART ONE
RESEARCH DATA ON CONSUMERS’ EXPERIENCES AND ASSESSMENTS OF HOSPITAL CARE
INTRODUCTION TO CHAPTERS 2 - 8 In the following six chapters, results from research into aspects of consumers' hospital experiences, and their assessments of care are presented. The chapters report both on the nature of people's experiences, and on methodological issues raised by the researchers. Chapter 8 summarises the key issues arising from the research. The organisation of material by chapter is largely according to the body conducting the research: consumer group, government department, hospital or academic researcher. It is not assumed that there is a hierarchy of ‘truth’ in the veracity of material gained by different researchers: rather that different bodies adopt different methods and have varying purposes. For example, the research by consumer organisations is more explicitly consumerist, while the research by doctors focuses on their concerns more directly. In both, however, matters which affect the quality of people's stay in hospital are addressed. The chapters are organised in the following way: •
Chapter 2: reports by consumer and community organisations of hospital users' experiences;
•
Chapter 3: reports by government personnel of consumers' satisfaction with, and experiences of hospital care, conducted as part of a policy review process;
•
Chapter 4: research reports by hospitals on the consumer orientation or inadequacies of their services, in terms of users' needs. This chapter does not contain data from hospital patient satisfaction surveys.;
•
Chapter 5: summary of the issues arising from annual reports of independent health complaints authorities;
•
Chapter 6: research by health care providers or academic bodies, chosen for both its comment on the consumers' experience and because of the methodological issues raised.
•
Chapter 7: patient satisfaction surveys sponsored by state and territory governments:
•
Chapter 8: summary of the key issues.
Not all the research has the same level of thoroughness, sophistication or rigour. However, it has been chosen to illustrate broadly the range of experiences, to highlight the varying experiences 11
of particular groups in the population, and to note some concerns regarding the methods used to collect data.
12
CHAPTER 2: CONSUMER REPORTS OF HOSPITAL EXPERIENCES Introduction Chapter 2 presents a summary of the findings of selected research by consumer organisations, and by community or academic bodies who have explicitly sought consumers’ views for the express purpose of presenting them. The various themes or issues from this body of research are extracted to highlight recurring matters of concern. It is also used to identify gaps in knowledge or in the types of research undertaken. Research by consumer organisations has traditionally been neglected as a source of information on quality and people’s experiences and assessments of care, either in the hospital arena or in the area of primary health care. This chapter, therefore, provides one of the few summaries of this material. A secondary purpose of the chapter is also to provide material for comparison with patient feedback from other sources, for example complaints data, satisfaction surveys, interviews with staff and patients. Details of each research project are included in Appendix A. What research was chosen? Fourteen pieces of research were selected to provide a general survey of consumer views across Australia. The period of publication of the reports spans 1989 to 1995. Research by consumer groups was selected, as were some reports by community or academic organisations, where the research explicitly sought consumers’ views on their experiences. It was not possible to find relevant research from each state and territory. However, we have gathered a wide range of reports from several states, particularly focussing on smaller states, as they are often neglected. Various pieces were chosen because the consultation undertaken for the research spanned different regions of Australia, or because there had been an attempt to get information from a population group. In brief, most of the research has been undertaken by qualitative methods, and is small-scale. One of the principal strengths of consumer research, apart from its ability to tap directly into people’s experiences, is its focus on population sub-groups, for example older people, people with a chronic illness. Table 2 over page presents a short summary of the subject matter of the reports selected, the type of research done, and whether the research had an explicit focus on hospital services.
13
Table 2: Summary of consumer research Report author
No of consumers
Groups targeted
Type of research
Focus on hospital?
WA Health Consumers Council 1994
Not applicable
Environment for users of casualty
Observational
Consumers’ Forum 1994
Unknown. Held 30 consultations and public meetings
High users, socially disadvantaged. Aust. wide. Included 3 mtgs with people from NESB Older people City & rural Targeted people from NESB Older people who had been in hospital, NSW Mothers, living on fringe of Melbourne and Vic’n provincial cities Women from Australia, and from N America, Europe. Mainly white, m/class
Qualitative Public meetings focus groups
and
Yes. Accident & emergency dept’s in 4 WA hospitals Yes. All aspects of hospital care
Qualitative Focus groups interviews
and
Health
Council on the Ageing Vic 1994
40-50 Consumers, as well as health workers
Delegation of Seniors’ Organisations 1994 Gilley for Brotherhood of St Laurence 1994
208
NHMRC W/party on treatment for diagnosed breast cancer 1994
46 publications
WA Health Consumers’ Council 1994
932
Aust. Council on Social Service 1993
200 (approx.)
NW Suburbs H’th & Social Welfare Council & Migrant H’th Serv., SA 1993 Marion Brighton Glenelg H’th & Social Welfare Council, SA 1992 Wositzky 1992
293 consumers 148 health workers
Aust’n Pensioners’ & Superannuants’ Federation 1991
142
Marion Brighton Glenelg H’th & Social Welfare Council, SA 1991 Marion Brighton Glenelg H’th & Social Welfare Council 1989
61
146
People reading daily paper in Perth; no demographic analysis People disadvantaged in terms of health status. Australia-wide Those using interpreters in S. Australia
Yes, discharge planning
Questionnaire
Yes, discharge planning
Interviews Demographic survey
Yes
Personal accounts, letters Systematic review of women’s views Input to gov’t inquiries Short questionnaire in daily paper
No. Covered all aspects of treatment, including hospital care and radiotherapy .
Focus groups
No. Questions on care after hospital stay; and feedback mechanisms No. Covered hospital and general use
Phone-in and survey
No, but major focus of responses
Several hundred
Consumers and carers using mental health services
Public meeting Interviews Workshops
No. Hospital care covered by look at system
79
High users of pharmaceuticals. Subset of above included people with special communication needs Older people with diabetes and/or arthritis. Sydney or Melbourne. 2 groups from NESB. 12 group from hostels Mothers in region
Focus groups
No. Hospital care covered by look at systems
Focus groups
No. Hospital care covered by look at system
Survey Group consultations
Yes; tests conducted in pregnancy
Public meeting
Yes; care during pregnancy and birth
60 (mainly consumers)
Mothers and some health workers in region
14
Presentation of issues The following sections present the views of consumers by theme. Their views concern what they regard positively and negatively in the hospital system, the common nature of their experiences, and their assessments of aspects of care. This material amounts to an assessment of some of the elements of quality care and for this reason is considered important. Consumer groups rarely separate out the notion of ‘satisfaction’. Rather they build up a picture of what constitutes quality from the assessments of consumers. The broad thematic areas are: • • • • • • • • •
overall comment role of consumer/ hospital culture access discharge from hospital information communication discrimination environmental continuity of care.
These headings emerged from reading and summarising the reports. Comments under ‘overall comment’ were included only if the report used this category, or if people were asked for an overall assessment. Issues of respect (being treated as a person) are included in 'role of consumer', as are issues of privacy. The use of interpreters is placed under 'communication'. Not all comments can be summarised under these headings. For instance, in a review of women’s views on breast cancer treatment (NHMRC 1994) it was noted that the trauma of treatment was exacerbated by long waits for treatment or waiting a long time to see a provider; the lack of privacy, particularly when receiving radiotherapy; poor amenity of services; the lack of team approach; or the structuring of treatment which did not take into account women's other responsibilities. The issue of collecting case specific information is discussed further at the end of the chapter. Some reports concerned a specific aspect of care or related to a disease/illness or type of care category. Summaries of two - regarding childbirth services - are presented separately after the thematic issues, to highlight the benefits of looking at the whole process of care. The chapter concludes with a listing of the issues which emerge from the research surveyed here, in terms of understanding the experiences of consumers, issues which affect feedback mechanisms and issues for research methodology.
15
Consumer views Overall comment In 1994 the Consumers' Health Forum (CHF) conducted consultations with people around Australia as part of a project informing the Commonwealth Department of Human Services and Health about concerns with the introduction of casemix, and, in particular, its possible effect on quality of care. The purpose of the consultations was to survey generally the types of problems encountered by consumers in hospitals. Groups with special needs or people who were more vulnerable were targeted. Given the documented experiences of these people in the health system generally, it is not surprising that the final report has a strongly critical flavour. It should be noted that a population based survey may find higher levels of satisfaction with elements of care. Some hospitals were noted by participants as excellent. Good practice should be acknowledged and models promoted. Overall, though, there was much negative comment. People from a non-English speaking background wished their appreciation of the hospital system noted, but experienced considerable communication problems. There were suggestions that the quality of care had deteriorated in recent years, and the staff time available for patients had decreased, possibly due to budget cuts in some states. There were suggestions that the greater use of agency nurses had led to uncoordinated care and possibly lesser quality care. Certain specialty units had developed a high quality approach and this was appreciated. The Council on the Ageing of Victoria (COTA (Vic) 1994) undertook a series of focus groups with older people and service providers for a project designed to establish principles for better discharge planning. The results illustrated the pivotal role discharge planning played in ensuring the episode of care was well handled (detailed comments in ‘discharge’ section below). This finding was echoed by a study by the Australian Pensioners’ and Superannuants’ Federation (APSF 1991), which reported that older people who had a chronic illness had noted incidences of premature discharge and inadequate planning for care at home − situations that led to further complications and possibly readmission. Their overall conclusion was that for older people with a chronic illness, those who fare well have access to multi-disciplinary clinics (which emphasise self management, combined with prompt treatment), well coordinated care between hospitals and local services, and access to allied services (in hospitals or in the community). People who do not fare well do not have access to this system approach, but, in addition, experience discrimination due to age, and cannot access needed allied health services due to cost, and the discouraging attitudes of some doctors, ie some doctors were dismissive of people’s desire to see certain allied health care providers.
16
Looking broadly at the interaction of people’s social and economic circumstances and their use of the health system, the Australian Council of Social Service (1993) reported that the lack of social supports for people in disadvantaged circumstances was a crucial factor and may increase their health risks or adversely affect the quality of their interactions with the health system. Role of consumer/ hospital culture In the study by the CHF (1994) the comments indicated a subordinate role for consumers. Hospitals were seen as places where people were disempowered, ie where people surrendered their autonomy and control, and the right to be informed about what was happening. Not everyone minded but many did. The fragmentary nature of hospital care was noted as worsening the sense that the whole person was being treated. Consumers' perceptions of their low status were compounded by the frequently stated fear of complaining, and in particular, the fear of reprisals. Certain marginalised groups expressed the fear of being labelled a troublemaker, as did residents of rural areas (where they were known by service staff). Frequent hospital users also discussed similar fears. Older people may be less likely to complain, or take an active role in seeking information, according to a Victorian older person’s organisation (COTA (Vic) 1994). Another study into older people who had diabetes and/or arthritis (APSF 1991) found that the experiences of people with diabetes were qualitatively different from those with arthritis. The treatment model of the former attributed an equal role to the consumer, emphasising education, self management and access to multi-disciplinary clinics. Therefore, people with diabetes were more satisfied with the system than those with arthritis. In a review of consumer literature on the perspectives of women about the treatment for diagnosed breast cancer (NHMRC 1994), 46 publications (including two major Australian reports) were surveyed. Material which in this report has been placed under the heading of the role of consumer was discussed under the heading of ‘choice and control’. The report noted that most women wish to be involved in treatment decisions, and wish to exert some control over whatever they can in the treatment process. The shock of breast cancer meant that additional care was needed to ensure this occurred at a time when women were stressed and vulnerable. Barriers to enhancing choice and control included the lack of continuity of care in public hospitals, lack of time between diagnosis and treatment, lack of good information, lack of consultation time, and settings which did not promote privacy or dignity. People who had a mental illness or who were a carer, involved in a South Australian study (Marion Brighton Glenelg Health & Social Welfare Council 1993), stressed the need to be listened to, the desire to be involved in treatment, and to be assisted or encouraged to assume responsibilities.
17
A West Australian newspaper survey (WA Health Consumer Council 1994), with more than 900 people responding, reported that 96.5% believed they should be informed of what to expect from health services, how to access them, and what to do if not satisfied. Access According to the CHF (1994) report, some people had difficulty getting in to hospital, for example, there were long waiting lists, or people with a chronic illness had difficulty getting through casualty. People experienced anxiety while waiting, due to the absence of information or communication about when admission may occur. Socially disadvantaged groups with ongoing concerns about access to hospitals included Aboriginal people, people from a nonEnglish speaking background, and people with a disability or chronic medical condition. A finding of the ACOSS (1993) study of people disadvantaged in health status terms was that Medicare was very important in ensuring access to care for these people. In addition, access to public dental hospitals was very poor and many had experienced extremely long waits. In South Australia people who had used mental health services were concerned with the reduction in inpatient beds and some had reported difficulty getting admitted despite being unwell. People wanted hospital and community based services to both be available. Some expressed concern with the lack of 24 hour access to appropriate care. People with certain conditions such as agoraphobia also experienced difficulties accessing services. In a study of older people with a chronic illness (APSF 1991), the length of wait to get in to hospital could be an issue. Some people with diabetes would have preferred a medical admission for stabilisation instead of the sometimes daily visits to the clinic. This burden could be exacerbated if the person could not drive and there was no hospital transport. A third of the pensioners interviewed had private cover to ensure access to hospitals. Another finding was that for people requiring allied health services after a hospital stay, for example, access to a physiotherapist or podiatrist, but who could not afford private care, there could be long waiting times for services in a hospital outpatient department. The review of women’s views on treatment for breast cancer highlighted the lack of access to radiation therapy in rural areas, which may reduce choice of treatments (NHMRC 1994). Access to aids and appliances via a hospital may be dependent on the time of application, ie when the hospital had funds (APSF 1991). For many disadvantaged people, there is some suggestion that the costs of pharmaceuticals and medical aids and equipment can be prohibitive (ACOSS 1993). The cost and time of travel to and from hospitals for rural and remote area consumers was not well understood by all hospitals, and there were difficulties with accommodation for family members. Reimbursement schemes could be slow. With the development of major specialist services in city areas, several trips were involved in one episode (CHF 1994).
18
Discharge from hospital The process of discharge (or lack of a process) was the topic of much criticism in the meetings around Australia conducted by the CHF (1994). However, certain specialist units had good records. Some respondents to South Australian research on mental health service issues (Marion Brighton Glenelg Health & Social Welfare Council 1993) said that they had been discharged without adequate follow up and support, or access to rehabilitation services. Discharge is often the most stressful time for older consumers, reported the Council on the Ageing (COTA) (Victoria) from its 1994 focus groups. Hospitals have no systematic way of assessing people's needs for care post discharge. The period immediately after discharge now appeared to be largely unplanned for despite shortening lengths of stay. People already linked into community care programs fared better. What planning which occurred focused on those most in need: those not obviously in need, but possibly with potential home difficulties, slipped through the net. Interviews with service providers for the project acknowledged that hospital patients, their families and carers should be involved in discharge planning but that this didn't occur enough. Hospitals had also made incorrect assumptions about the ability of a carer to take responsibility of the person after discharge. A similar picture – highlighting the common absence of discharge planning for older people – is seen in New South Wales, where, on behalf of the Delegation of Seniors' Organisations, the COTA of NSW undertook a short survey of the experiences of discharge planning for people who rated themselves as Seniors, and who had been in hospital any time since January 1993. The report (Delegation of Seniors' Organisations 1994) notes the following: •
of the 208 valid responses, 44.5% reported they were not seen by a discharge planner;
•
of those not seen by a discharge planner, 50.3% reported that they required additional care;
•
from the total group, and of those requiring additional care, 66.25% reported that no community or other support services were arranged by the hospital.
An earlier study by the APSF (1991) of the experiences of older people with diabetes and/or arthritis also suggested that some people suffered adversely because of possible early discharge, compounded by no backup. Some people had been discharged to a home situation where they cared for a disabled person, but there had been no communication with hospital staff about whether they could cope. The West Australian Consumers’ Council (1994), from a short survey placed in the daily newspaper in Perth, reported that:
19
•
44% of the 932 respondents said they would not be able to arrange help at home and someone to care for them, after receiving day surgery,
•
77% said they would expect a nurse to visit them at home after surgery; 52% expected a doctor to visit them.
Information The desire and need for more information – verbal, written and in various languages – was a common issue emerging from most research projects. For example, in the CHF (1994) Hospital Casemix project, the lack of timely and appropriate information on all aspects of the hospital episode was noted repeatedly. The lack of information on medications was raised at all consultations. This latter aspect was raised strongly in focus groups examining medication issues for people who were high users of the health system (Wositzky 1992). The report noted that some people with intellectual or psychiatric disabilities had experienced considerable problems getting sufficient information on medications, or in getting trained to cope with a complex drug regime. Furthermore, most people wanted more information than they had been given on possible sideeffects of medications. This finding was echoed in the South Australian research of people with mental health problems (Marion Brighton Glenelg Health & Social Welfare Council 1992). The verbal and written information people receive prior to discharge is crucial, but is often inadequate (COTA (Vic) 1994). In the review of women’s views about breast cancer treatment (NHMRC 1994), women reported wanting more information than they tended to receive, and information on a broader range of topics than just the disease and treatment options. Some women did not but many did, and their need for information on what would happen and was happening grew after surgery, rather than diminished. Written and audio-visual information was important but did not replace the need for interaction with health care providers. Two studies examined information issues in particular circumstances – for women undergoing tests during pregnancy, and for people waiting in accident and emergency departments. In the former, the Marion Brighton Glenelg Health and Social Welfare Council (1991) found that many women were not well informed of the reason for some tests, and the implications. A third of the women thought the tests were compulsory, raising issues of informed consent. Information was preferred in written and verbal forms but the former was provided in only about half the cases (61 women surveyed). Very little information was available in languages other than English. This targeted study had arisen because in an earlier consultation day on women’s views of birthing services undertaken by the Council (1989), the lack of information during pregnancy and the concern with the sense of passivity felt by many women, had emerged as important matters. In the other study, the West Australian Health Consumers' Council (1995) undertook a snapshot survey of accident and emergency departments in four major public hospitals and looked at the information made available to users. This was a rare example of research where the issue of the literacy of people of English speaking background was noted. 20
The survey raised the following issues: •
the hospitals were relying on people's capacity to read and understand English;
•
important information was not always prominently displayed;
•
information on the adjustments of waiting times was crucial as many people could not see that other more urgent cases had been given priority.
Communication The importance of good communication to the quality of the hospital stay was mentioned repeatedly. This encompassed the dialogue between consumer and health professionals, most importantly the main doctor; communication between the treating team and family members or carers; and communication between professionals working in different settings, about the one patient. Crucially, access to trained interpreters when required made a significant difference to the experiences of people not conversant in English. The CHF (1994) report provided a somewhat sobering overview of the experiences of some high users of hospitals services. Specialists were seen as uncommunicative, although an improvement in recent years was noted. Hospitals did not communicate with general practitioners (GPs). Consumers were not involved in decision making. Some older people had been told to sign consent forms, without explanation. The Victorian research on discharge planning (COTA (Vic) 1994) noted that communication between the hospital and local services can be poor. Specific areas mentioned included pharmaceuticals, the cost of services, and the roles and responsibilities of different services in providing care. A similar point was made by Wositzky (1992), who reported that people with a chronic illness or disability experienced communication problems between institutional and local doctors. The situation of carers was highlighted by those caring for someone with a mental illness. They reported that they could be ignored, not informed or involved in decision making, yet were expected to provide full support for the person upon discharge (Marion Brighton Glenelg Health & Social Welfare Council 1992). Routine use of trained interpreters was not occurring (CHF 1994; APSF 1991), hampering communication (ACOSS 1993). The APSF (1991) reports adds that older people of non-English speaking background experienced similar communication problems to those from English speaking backgrounds, but these were compounded by the lack of interpreters, as well as cultural differences and (for some) ignorance of how the health system worked. The review of women’s views of treatment for breast cancer (NHMRC 1994) noted certain barriers to good communication. These were the use of jargon, lack of time, the negative 21
portrayal of information on their condition, the lack of attention to the concerns expressed by women, and not treating women as 'whole' people or as equals. One research project (SA North West Suburbs Health & Social Welfare Council & the Migrant Health Service 1993) examined the use of interpreters by both consumers and health workers, and the results highlight some of the existing and potential problem areas: •
hospital services were the most common service for which interpreters were used;
•
for those who had been refused an interpreter service in hospital, common reasons were that they were told to bring a friend, they should have booked in advance, and no funding available;
•
there were reasonably high levels of satisfaction with the quality of the services, although concerns were expressed about serious breaches in confidentiality. This led some people to refuse further interpreters. There were some concerns with the fullness and adequacy of the interpretation;
•
people were not willing to criticise interpreter services because they believed it would lead to the service being removed;
•
people expressed the need for more specialised interpreters, for example, in mental health, older persons care;
•
respondents also expressed the need for more bilingual workers, increased service levels after hours, and greater flexibility in choice of interpreter.
Discrimination Various people identified receiving particular and unwelcome approaches from hospital staff, which they attributed to their own social status or identity. The points they made have therefore been separated out from the general issues, because while, for example, the perceived discriminatory treatment may be around access, it compounds the unsatisfactory nature of the experience for the consumer. According to people consulted by the CHF (1994), discrimination was possible if you had a mental illness, especially when seeking general health care; if you were homeless; an Aboriginal person; or an older person. In the ACOSS (1993) consultations with people socially disadvantaged in some way, similar points were made. People commented that service providers had a stereotypical view of them as ‘single mother’ or ‘mentally ill’ and this coloured their interactions. In both the CHF and ACOSS consultations, some people commented that they believed they would receive more respectful treatment if they had private health insurance. Older people who had a chronic illness reported to the APSF (1991) that they felt that they had poorer access to treatment because they were old, had a chronic illness (acute had higher priority), and they had no private insurance. Environmental 22
This issue was rarely raised. The CHF (1994) consultations noted that some people had said the food and the physical environment was not good in some hospitals. The poor amenity of psychiatric hospitals was noted. In the NHMRC (1994) review of women’s views on breast cancer treatment it was noted that the treatment process was made worse by poor amenity of facilities. Continuity of care Continuity of care was interpreted as having the same main doctor in charge of treatment. The importance of this was raised in three research projects. It was a source of satisfaction for women having a baby (Gilley 1994; Marion Brighton Glenelg Health and Welfare Council 1989), and was seen as particularly important for people with mental health problems (Marion Brighton Glenelg Health and Social Welfare Council 1992). Childbirth services The results of two very different projects, examining childbirth services from women’s point of view, are presented below. They illustrate how various aspects of care contribute to an assessment of satisfaction. Similar issues emerge as in the detailed surveys of childbirth, which were conducted for state reviews of services (see chapter 3). Gilley (1994), for the Brotherhood of St Laurence, interviewed 146 mothers living in 'service poor' areas of Victoria, including outer urban areas, semi rural areas, and areas within and outside of provincial cities. They were asked their views on access to, and the quality of services, for pregnancy, childbirth and for postnatal care. Of the findings relevant to use of hospital services, the following comments were made: •
antenatal care was generally viewed favourably. However, there were criticisms by women using the public system of long waits, lack of privacy, and discontinuity of care;
•
overall, most were satisfied with the care while in hospital. Sources of satisfaction were friendly and supportive staff; continuity of care, particularly in labour; information during labour and while in hospital; privacy; good advice on baby care and breastfeeding; and high quality food. Sources of dissatisfaction were the obverse of these;
•
some mothers in the Cranbourne area felt they had been discharged too quickly after the birth, but this was not an issue in the other areas;
•
some mothers did not receive sufficient follow up care after discharge and some were charged for services but had not been informed they had to pay prior to receiving the service;
•
hospital use after birth was common - one quarter of mothers required them for their children. All the critical comments related to one hospital;
23
•
the lack of public transport for some mothers was a critical issue in gaining access to a range of services.
The results of this study were also compared with the results of a study of mothers and children who lived in inner urban areas. Whilst these people had better access to services, they had less social supports, had greater language barriers, used birth classes and specialists less, and had less knowledge of available services. The report comments that in the study of mothers in outer urban and regional areas, their experiences and sources of dissatisfaction echoed the results of the Consumer Survey of the Victorian Birthing Review. The second study considered here was conducted by the South Australian Health and Social Welfare Council of the region of Marion, Brighton and Glenelg. Approximately 60 women (principally consumers) attended a seminar and consultation on birthing. At the meeting women discussed those aspects of care important to them, their expectations of services, and what they would have liked to change about the care they did receive. The report (Marion Brighton Glenelg Health and Social Welfare Council 1989) notes common issues which emerged, as well as detailed comments. The main areas of concern were: • • • • • •
continuity of care; information and education; the need to feel comfortable and secure; more midwife involvement desired; more informal sources of support were needed; and women wanted more information and better classes.
Issues emerging from the research Assessing the experiences of consumers There are common themes which emerge from the consumer research. 1.
First, there are several related issues which cluster around the general mantle of:
• • • •
communication; being treated with respect; not being the subject of discriminatory treatment; and being involved in decision-making.
Sub-issues related to these are: • • •
informed consent; privacy; and access to interpreters.
24
What binds these issues are assumptions about the position the consumer occupies in their interactions with the health system. A consumer who is not considered an equal partner in the process, deserving of respect, courtesy and involvement in their care, is one who may experience a less than satisfactory time in hospital. 2.
For some people, timely access to hospital care is an important issue. Several factors are mentioned:
•
the cost of, or the lack of public transport. This is an issue for city and country dwellers but the specifics may vary;
•
some people are not adequately informed about the length of wait; and
•
some people cannot get into hospital for some reason.
3.
People want more information on all aspects of the hospital stay, and in particular on the pharmaceuticals they are prescribed. They want information in verbal and written forms, and in languages other than English, where required.
4.
The role of the professional team is critical. People are more positive about:
•
multidisciplinary teams;
•
good teamwork; and
•
communication between professionals. This includes communication between professionals in and outside the hospital.
A related and important sub-issue for public patients is seeing the same principal doctor, within a team, ie continuity of care. 5.
Discharge planning is frequently inadequate for all types of patient. There is little involvement of family or carers either.
Issues affecting consumer feedback mechanisms The research throws up other issues to consider if consumer feedback mechanisms are to be developed. They are: •
anonymity is very important, particularly for regular users of hospital services, carers, and people in city fringe, rural or remote areas, where there may be no choice among hospitals, or staff are known;
•
some people fear that criticism of a service will result in its loss. This may be more of an issue for a service that targets one population group, ie a specialist service, or in remote areas where it is difficult to attract and retain specialised staff;
25
•
some sources of dissatisfaction may relate to structural factors related to resource levels, and may be difficult for staff to respond to, ie increased use of agency staff, inadequate numbers of interpreters;
•
the severity of the condition may have a significant bearing on how the consumer copes with the hospital stay or use of services;
•
the philosophy adopted by a specialty area may have a significant impact on the way care is structured and the approach taken towards the consumer. Some specialties treat the person as an equal, promote self management and teamwork, involve other professions, and/or develop a model of good practice. These treatment models are clearly recognised and appreciated by consumers who wish them promoted more widely;
•
consumers who are disadvantaged in some way are keenly aware of the less than satisfactory treatment they believe they have received. Some people, for example, older people, may not want to complain, but when interviewed at length, may point to incidents of care that have inconvenienced, discomfited or adversely affected them;
•
not much is known of the separate or complementary needs of patients and their families or carers;
•
nothing is known directly of the problems of children and young people or people not literate in English, either of a non-English speaking background or of English speaking background (see Turrell and Najman [1995] for some discussion about low income and literacy in relation to research design).
Methodological issues arising from the research Consumer organisations have tended to direct their research to their members and consequently it predominantly focuses on population groups, for example, older people, people from a nonEnglish speaking background. Many of these groups have ‘members’ who are disadvantaged socially or financially in some way. This focus has enabled some organisations to build up a body of work, for example, the work on older people. Social groupings whose experiences have not been examined include children or adolescents, people not literate, and low or occasional users. Men, as a group, have been reached in the work with older people, but are less represented in other work. There are several pieces of research which are directed towards the experiences that people with a specific condition or illness have. This enables models of care to be illuminated. Consumer organisations have also looked at the issues of the process of care, ie getting in and out, or the linkages between services. They are less likely to see hospital care as an isolated event and that particular aspects of the episode do not exist in isolation from other aspects. Most of the research has been qualitative, with focus groups the predominant type of method. When sufficiently resourced, the organisation has conducted interviews, and sometimes small 26
demographic surveys to complement the interview or focus group data. Better resourced projects have sampled views around the country, in city and country areas, but most projects have targeted the group’s local members. A few organisations have tried different methods, for example, observational research. Much of the work of consumer organisations is directed to the policy process of state or Commonwealth governments, but less commonly to specific hospitals. There is little published material on whether the report had an impact on hospital practice or government policy development. Anecdotal reports suggest that consumer groups face considerable barriers in convincing hospitals of the value of their work, although this is not a uniform problem. Some hospital staff may be sympathetic to the issues, but changing hospital practice requires that the research is built into a joint policy development and research process. Research by consumer organisations provides a valuable and often cost effective way of hearing the direct experience of consumers. The types of research data presented here illustrate the breadth of problems people experience in their dealings with hospitals, and to some extent, the depth. Many of the issues raised provide the basis of what issues of the hospital experience should be the subject of feedback mechanisms, as well as the topics of further research, for amplification and verification. However, consumer organisations could benefit by the following: •
research projects need to be better resourced;
•
research staff require training on methods, or would benefit by access to a clearing house and specialist advice on consumer feedback research;
•
there should be more joint projects, for example, between a hospital or private insurance company and consumer group, or between a specialist college and a consumer group;
•
the results of research should be disseminated more widely; and
•
there needs to be more written on how the research did or did not affect practice, and why.
27
28
CHAPTER 3: RESEARCH BY GOVERNMENTS INTO CONSUMERS' SATISFACTION WITH, AND VIEWS ABOUT HOSPITAL CARE FOR CHILDBIRTH This chapter summarises the results of research projects sponsored by governments into childbirth services. They have been chosen because they represent a thorough attempt to obtain consumers’ views about services for a specific condition. We are not aware of any comparable consumer surveys into other condition-specific services. First, the results of the two major reviews into birthing services, in West Australia and Victoria, are summarised. The West Australian review is noteworthy because it included a group survey of Aboriginal women. Several academic papers have flowed from the Victorian survey and these have been drawn on extensively to highlight the methodological issues of population sampling and for a discussion of the lessons from this exercise. Consumer views of maternity services in Western Australia As part of the Ministerial Review of Obstetric, Neonatal and Gynaecological Services two projects were undertaken to seek the views of West Australian women. General birthing services The first, reported on by Bartlett and Pennebaker (1990), was a survey which was mailed to the first 2000 women who had a birth in May 1989. The purpose of the survey was to examine consumers' views of birthing services, and how these varied by certain factors, and to obtain data not already available. Two thousand women received the survey three months after the birth. There were two mailouts, both including the questionnaire and a prepaid return envelope. The overall sample was reduced to 1945 due to the number of undelivered letters. The final number of respondents was 1315 (67.6%). The survey comprised a self completion questionnaire with a combination of pre-coded, restricted option, and open questions. For pre-coded questions asking for ratings, a five point scale was used. There were 47 questions which included 83 parts. There was a page for additional comments. The questionnaire was piloted in three venues and completed with the researchers in attendance. Comment was also invited from self help groups, nurses and the ministerial review taskforce. The limitations of the research are that the researchers have no information on those who didn't respond. The study is cross-sectional but not representative. Groups under-represented are women from a non-English speaking background, and low income women.
29
The results are as follows: Antenatal care Overall there was a high level of satisfaction. A fifth were unhappy with the waiting time for seeing the doctor. Comparing responses from public and private patients, a higher proportion of private patients found waiting time to be reasonable. In the metropolitan area, women in the public system were more likely than private patients to wait longer. A third made suggestions for improvement: the majority suggesting the need for more caring, personalised and respectful staff, but more time and advice from doctors, and more feedback on antenatal progress were also important. (See 'specific groups' below.) Birth of baby The summary of the report notes that women were largely satisfied with explanations given them about procedures. Most reported that doctors and nurses were helpful during labour. Regarding improvements, 40% wanted more information on what to expect, 26% wanted more information on the progress of labour, and 27% wanted more on procedures. Public patients were more likely to report that the labour and delivery were unpleasant. When commenting on the worst experiences, women noted the birth itself, including the pain, long labour, lack of privacy and emotional support; also the clinical procedures and the way they were handled; the poor manners of staff; and the lack of control over what happened. Postnatal care Most were satisfied with their length of stay (ave 7.5 days). There was overall agreement that care was better than expected, but more private than public patients thought this. Negative comments concerned the lack of recognition to mothers' needs, understaffing and lack of assistance, and conflicting advice from nurses. Comments and ratings of the amenities noted that public patients had less privacy; and that hospital noise prevented sleep (a major complaint). Most were happy with the information on discharge, although 15% wanted more information. Specific groups More women from a non-English speaking background, on low incomes or Aboriginal women reported poorer experiences of childbirth care, particularly in terms of how they were treated. They also received their first consultation later, had longer to travel to care, used more public transport, and had longer waits for the doctor. Although a very small group (3.3% of the sample), non Christians (Muslims, Hindus, Buddhists, Jews and others) reported significantly poorer experiences.
30
Services for Aboriginal women The second project was a consultation with Aboriginal women around the state (Henderson, Lee & Harte 1990). A questionnaire was sent to Aboriginal Health Workers to complete in meetings with women. The questionnaire sought views on obstetric and gynaecological services, and demographic information. The questionnaire was developed after feedback on a draft from people in Aboriginal Medical Services and in Health Department Regional Offices. The questionnaire involved a combination of open and limited option questions. With the latter, the worker marked whether all, some or none of the women in the meeting agreed with the answer to the question. 150 questionnaires were sent out and 58 (38.7%) were returned and used. The number of women attending these meetings was 478. There were 30 questions with 67 separate parts. In terms of limitations, it was noted that the comments represented group opinions, although individuals were able to complete questionnaires if they wished. Criticism received during the process included comments that the questionnaire was too long, and there were problems with the wording. For example, not all understood the terms used, eg midwife. Some women had only recently been exposed to services such as antenatal care. Some women were embarrassed to discuss certain issues, for example, sexual health. A final point was that Aboriginal women in remote communities shared the same problems as other nonAboriginal women: they were not necessarily different. The results are as follows: Access Most were able to receive care when needed. For childbirth, more women wanted an increased use of local facilities. One in five women did not have local antenatal classes. Access to a doctor during pregnancy was difficult for 30% of women. Most women had no transport to their appointments and had difficulties with childcare. One in four women preferred a woman doctor for the range of their health needs. Cultural and communication issues Women experience isolation and fear attending hospital. Some need interpreters. There was strong support for the regular involvement of Aboriginal Health Workers and local Child Health Nurses. Information Women wanted more information on birthing options, and on various health and medical issues. There should be educational programs emphasising antenatal and postnatal care.
31
Satisfaction Women using childbirth services were less happy with the service received than women using services for general needs. For the latter, 70% felt that medical care was adequate or above average. Consumer views of birthing services in Victoria As part of the Ministerial Review of Birthing Services a Consumer Survey of Recent Mothers was undertaken in 1989 to seek the views of Victorian women. The results and accompanying recommendations have been recorded in the review's final report (HDV 1990), and in more detail in Small, Brown and Lumley (1992), Brown and Lumley (1993, 1994), and in Small et al. (1994). An extended discussion of the survey is presented in The Experience of Motherhood, (1994) by Brown and her colleagues. The consumer survey was mailed to a representative sample of women who had given birth in February 1989, or in one hospital's case, in September 1989. The purpose of the survey was to examine consumers' views of birthing services, as part of an extensive process of community consultation including submissions and consultations. A primary reason for the survey was to determine how representative were the views of women who had submitted comments to the Review, or met with the Review team (Brown & Lumley 1994:4). The survey would gather factual information, and record women's satisfaction with care received, their preferences for future births, and their emotional well-being after birth. A total of 1193 women received the questionnaire eight to nine months after the birth. There were three mail-outs at two-weekly intervals, each including the questionnaire, a covering letter, and a prepaid return envelope. The overall sample was reduced to 1107 due to various reasons: there were 790 useable responses (71.4%). Key issues regarding the sample size were the need to satisfy service providers that at least 1% of total births were included, and that subgroups of interest were large enough to detect significant differences in satisfaction. The required size of the subgroups was calculated from the demographic data in the Victorian Perinatal Data Collection. The survey comprised a self completion questionnaire with a combination of pre-coded, restricted option, and open questions. For pre-coded questions asking for ratings, a five point scale was used. There were 66 questions. There was a page for additional comments. The questionnaire was developed using two models: a British survey of maternity services, used by the Institute of Social Studies in Medical Care; and the West Australian survey of maternity services (see above). It also incorporated questions addressing issues raised repeatedly by women in submissions or consultations, or which were pertinent to the Review (Brown & Lumley 1993:96). The questionnaire was piloted in two venues and completed with the researchers in attendance and some minor changes made.
32
Limitations noted were that the response rate of women born overseas of non-English speaking background, single women, and younger women (age under 20 or 20-24) was less than the level required for it to be representative of these groups. Attempts to enhance the response rate of Vietnamese and Arabic women were not successful. The Review followed up by resourcing community groups to consult with women seen to have additional needs and it is reported by Brown and Lumley (1993:101) that while not statistically significant, the results of these meetings may be more useful. The results are as follows: Antenatal care The following summary is taken from the detailed presentation of results in Brown and Lumley (1993). Most women rated their antenatal care as good or very good (85.9%). Those more likely to be dissatisfied were younger women, those without a partner, or those born overseas and of nonEnglish speaking background. There were significant differences in satisfaction between those attending a private obstetrician, a general practitioner (GP) or a public hospital clinic, in that descending order. Women of non-English speaking background were significantly more likely to be dissatisfied with antenatal care by a GP. Uninsured women who attended a private obstetrician were more likely to be unhappy about aspects of their care. The study results disposed of a common assumption based on anecdotal reports, that middle class women have higher expectations and complain more. In fact, women disadvantaged in some way reported higher levels of dissatisfaction; Care in labor and birth The article by Brown and Lumley (1994) summarises in detail the findings of the Consumer Survey for the experience of labor and birth. The overall rating of satisfaction with care in labor was derived from those responses which were clearly positive. If there were some aspects not liked, this was taken to mean there were elements of dissatisfaction. Arising from the public consultation process was a particular concern of women for an active say in the birthing process, and a question was included on this for women who did not have an elective caesarean section. Results: Two-thirds of women were satisfied with the management of labor and birth; 27% said their care was mixed; and 6% said it was not as they liked.
33
When looking at the association between social characteristics and dissatisfaction, women who were more dissatisfied were those having their first baby; those who had not completed secondary school; and rural women. Four separate factors were strongly associated with dissatisfaction, ie dissatisfaction with any one may lead to a negative rating of care. They were access to information during labor, the nature of the relationships with caregivers, the extent of involvement in decision-making, and exposure to intervention. Health after childbirth Small et al. (1994) have reported on a follow up study of the emotional well-being of a controlled and randomly selected group of women, chosen from the population-based postal Consumer Survey of birthing experiences. In all, ninety women were interviewed in their homes, using a semi-structured interview method. The women also completed five standard questionnaires about their health status, children, and social settings. The findings demonstrate the importance of the social context of motherhood after the birth to the emotional well-being of women. The research also demonstrates one of the ways researchers can follow up larger postal surveys, having received participants' consent to be involved in further research. Issues arising from the research Assessing the experiences of consumers The WA survey showed the link between public patient status and lesser quality experiences as reported by women. They experienced longer waits, unpleasant labour and delivery, and less privacy. Women who were from non English speaking backgrounds, Aboriginal women and women on low incomes were more likely to report poorer experiences, as were non Christian women (although small in number). Important issues were information, communication and support, involvement in decision making, and caring and respectful staff. For Aboriginal women, there were specific problems such as access to antenatal care, the desire for a women doctor, feelings of isolation and fear, and education on the benefits of certain services. The Victorian survey indicated significant dissatisfaction with antenatal care amongst certain groups in the population, for instance, younger women. Women attending public hospitals for antenatal care were least satisfied.
34
There was a considerable degree of dissatisfaction with care during labour and birth. Four critical factors were associated with dissatisfaction during labour and birth. Issues affecting consumer feedback mechanisms In their articles on the results of the Victorian survey, Brown and Lumley (1993, 1994) make the following points: •
women were keen to complete the questionnaire, write lengthy comments in addition to the questions, and give a phone number for follow up research;
•
it has been shown that respondents are less prepared to be critical if they know caregivers will see the questionnaires. The independent auspice of the project was considered important;
•
the authors report on various studies which show that women remember accurately the events during pregnancy, labor and birth;
•
the authors make the point in both papers that the largely representative sample obtained successfully counters the stereotypes of certain consumers − stereotypes which are used to discredit critical feedback;
•
women may be more prepared to express a negative view of caregivers if they use private services but are not insured;
•
the greatest increase in odds of dissatisfaction were associated with not having an active say in labor. This result brings in to sharp contrast possible tensions between specialists and consumers. When the survey was being developed, some obstetricians wanted the question regarding an active say removed: some said women would not be able to answer it; others claimed the answers would be biased. Whatever "active say" meant to women, it was interpreted by most as crucial. Furthermore, not having an active say was associated with less emotional well-being after the birth;
•
the association between dissatisfaction and primary caregiver and insurance status found in this study raises a key issue of how to address causes of dissatisfaction. Women with restricted choices of which type of antenatal care to use (young, without a partner, on a low income, or of non-English speaking background) were more dissatisfied, and the authors note that it is these women who may be in need of more sensitive care.
35
The Victorian and West Australian postal surveys, with follow up reminders, had high response rates (67.6% and 71.5%). Despite this, both had difficulties getting adequate or representative numbers of women considered to be disadvantaged in some way. One could assume that these women are more likely to be public patients, and furthermore, people of lower socio-economic status are higher users of general hospital care. Care for public patients was more subject to criticism than that for private patients. If unrepresentative samples are obtained, then the amount and nature of dissatisfaction will be unreported. The importance of continuity of care by one doctor for public patients emerges from these birthing studies, and builds on the findings of several projects reported in chapter 2. This raises the issue of how to attribute responsibility for ‘satisfaction’ with care, if a person attending hospital publicly has several doctors. Dissatisfaction with discontinuous physician care suggests that responses by the hospital will require some re-organisation of care. The birthing surveys were timed to maximise interest and opportunity to respond, with a period of good recall of events. The high level of interest, and the desire of many to be involved in further research, suggests a high level of interest in the community to be active participants in the process of researching and improving health care. This has implications for the type of research, as traditionally, consumers have only been allowed to play a passive reporting role. This may not be acceptable to, or preferred by many. Methodological issues arising from the research As with the matters above, Brown and Lumley (1993,1994) include some important reflections on methodological issues arising from their research, and these have been summarised below: •
postal surveys are a relatively inexpensive method of reaching a representative sample;
•
the response rate for this survey falls in the upper range for reported results of postal surveys, and was enhanced by the additional mailings containing new copies of all materials;
•
the time chosen for the survey was aimed to reduce the effects of specific times on the nature of responses, ie women are less critical immediately after birth, having survived a major ordeal, and are less critical after a longer period;
•
they accept that pre-coded scales may underestimate the extent of dissatisfaction, and also note that asking people to rate satisfaction of such a major event where there may have not been alternatives, is "highly contradictory" (1994:10). Despite this caveat, they note with concern the very high degree of dissatisfaction with labor and birth;
•
the authors report that overall measures of satisfaction mask differences in satisfaction with particular components, confirming an earlier supposition of Locker and Dunt (1978);
•
hospitals were not rated and compared in the final report, but specific reports were made available for possible use in quality assurance programs;
36
•
the findings − that women were more critical of care during labor and birth, rather than antenatally − led the Review to tailor its recommendations to achieve more change in this area;
•
a follow up survey is planned for three years from this one (completed 1995).
The Victorian survey reported that overall measures of satisfaction mask dissatisfaction with specific aspects of care. The West Australian material also is indicative of the complexity and difficulties inherent in reporting satisfaction data. For example, with the issue of helpful care from doctors and midwives, the report summary notes that "a large majority of mothers" found care during labour helpful (Bartlett & Pennebaker 1990:ii). In Table 11 on page 15 the breakdown of ratings for care by doctors is as follows:
no.of women Very helpful Fairly helpful Some help Very little help No help at all
557 178 67 53 38
% of women 62.3 20.0 7.5 6.0 4.2
Some may interpret 'fairly helpful' as not a very strong response, and indicative of times when care was not all that helpful. Even if one assumes that this category is a positive response, nearly one fifth didn't receive much help from doctors, and one in ten received little to no help. The finding that the frequent comments made to the consultation phase of the Victorian Birthing Review were substantiated strongly by the survey data, despite the misgivings of doctors, indicates that methods other than a representative survey still produce valid data. While not giving the researchers the confidence of its representative nature, the information that women wanted an active say in birth influenced the subject matter of the survey.
37
38
CHAPTER 4: RESEARCH BY HOSPITALS This chapter outlines the results of research projects undertaken by two hospitals - the Mercy Hospital for Women in Melbourne, and the Adelaide Medical Centre for Women and Children. It does not include the results of more conventional patient satisfaction surveys undertaken by hospitals or hospital departments. This chapter deals with research which has utilised various methods to explore complex service issues, not just patient satisfaction. However, both projects gathered feedback from users or staff on the quality of care and patients’ experiences. The Mercy Hospital, Victoria – access and equity issues The Mercy Hospital for Women, in Melbourne, undertook a major research project to examine issues of access to culturally sensitive services at the hospital. The report of the hospital research (Immigrant Women's Health Project. Hospital-based Research 1995) also summarises the findings of an associated project by the Key Centre for Women's Health in Society, at the University of Melbourne, which examined the health experiences of women from selected ethnic groups in the community. The findings of the two studies were similar and point to the difficulties experienced by people from a non-English speaking background when using public hospitals. It is important to note that the hospital-based study did not seek the views of hospital users - only the hospital staff. In combination with other research techniques, the researchers were able to present a comprehensive picture of the hospital environment. Methods The project utilised several methods: •
review of hospital policies and protocols;
•
examination of the hospital's board, management and committee structures;
•
focus group discussions, interviews, and a survey of staff, regarding their roles and responsibilities; and
•
focus groups, meetings and interviews with health and welfare workers in community organisations.
The project had also hoped to undertake a study by participant observation of the neo-natal intensive care area, but it had to be abandoned.
39
Findings of the hospital-based study From the findings, the following points are summarised: Role of consumer/hospital culture •
only the social work department and the multicultural services unit had policies relating to non-English speaking background patients;
•
nursing staff had been discomforted by the practice of not informing or asking permission for medical students to examine non-English speaking background patients;
•
staff and patients were unaware of the avenues for making a complaint to the hospital or to the Health Services Commissioner;
•
on entering the hospital, women from non-English speaking background and English speaking backgrounds were not advised of their rights and responsibilities, or their right to ask for an interpreter, if required;
•
there was an absence of representation of people from non-English speaking background on the hospital board and key hospital committees;
•
promotional material for the hospital did not reflect the ethnic mix of its patients.
Access •
women from a non-English speaking background were under-represented as users of preventive services;
•
there was much less use of interpreters on the weekends and after hours (see communication);
•
some women from non-English speaking background who had a strong preference for women doctors had changed hospitals to ensure they saw a woman doctor.
Discharge Early discharge was a concern when people from non-English speaking background had no family or community support networks. Information Materials produced for patients were rarely available in community languages. Information videos were only in English.
40
See role of consumer (above) regarding the lack of information on rights, means of making a complaint. Communication There were several major findings, summarised below, which reflected communication problems for people from a non-English speaking background: •
the inadequate use of interpreters at certain times, and the difficulties with non-hospital interpreters were barriers to communication identified by staff;
•
the process of getting informed consent from people not fluent in English was deficient in many cases;
•
there was an absence of bilingual staff in key access points in the hospital;
•
communication difficulties were not just due to the language barrier. The multicultural services unit had assisted in understanding some specific cultural issues.
Discrimination Staff had identified widespread ethnocentrism which translated into negative behaviour towards women from a non-English speaking background. Environment The food services needed modifying, for example, pork free menus for Muslim patients. There was a lack of international signs in key areas of the hospital. The Adelaide Medical Centre for Women and Children The Adelaide Medical Centre for Women and Children undertook a ‘Consumer Consultancy’ to examine ways to increase consumer participation in the planning, management and service delivery. The project used several methods to achieve its aims, including action-oriented research, as well as more conventional review and planning techniques. The following points have been extracted from the two volumes of the report (Adelaide Medical Centre for Women & Children 1992) and concern only the issues relating to adolescents. This is because this is a group of the population neglected in other research so far discussed. The material below comes from two sources. During the course of the project, planning seminars for people of community organisations and some users were held, in order to discuss strategic planning and service issues. The development of services for adolescents was the subject a workshop. In addition, the Consumer Consultant facilitated discussions between parents of children with cystic fibrosis and hospital staff.
41
The issues raised through these meetings are: •
children and adolescents may be lumped together as a group but they have separate needs;
•
access for young people may be problematic. For example, they may seek help in crisis, but being public patients may be subject to long waits. They may need fast-tracking;
•
there are significant communication barriers. Confidentiality may be over-ridden, as doctors deal with parents. Issues of responsibility, if the young person does not do what is expected, maybe unresolved;
•
adolescents have a high need for privacy which may not be recognised;
•
there is a lack of appropriate information for adolescents;
•
ward rules for centres catering for adolescents should be formulated by staff, parents and young people. These should be regularly reviewed by a group representing these parties.
Issues emerging from the research Assessing the consumers’ experiences Good or bad communication between parties is not purely a matter concerning individuals. It is facilitated by structures and policies where the goal of enhancing communication is assured. For example, individual staff members may benefit from hospital programs where cross cultural issues are explored, as not every person has the opportunity to learn of these matters. The problems of people who do speak English are echoed by the experiences of people from a nonEnglish speaking background who do not. However, problems for the latter may be exacerbated by the absence of interpreters, or the use of interpreters not familiar with the specifics of that hospital’s environment. Issues affecting consumer feedback mechanisms Staff may be an important source of information on consumers’ problems. However, they should not be seen as ‘substitutable’ for consumer views. In telephone discussions with one of the authors of the report at the Mercy Hospital, it was stressed that many women from a nonEnglish speaking background will not speak out in public meetings, even in a sympathetic environment, because of fear of retribution, or unwillingness to harm their relationship with the service, or for other culturally related reasons. Given this, there may be issues of importance of which staff are unaware. Not all problems are clear cut, and two groups of users, ie parents and their children, may have differing views of what should be done to improve aspects of the care. These may conflict. Even in the exercises at the Adelaide Medical Centre, young people were not major participants in the meetings.
42
There may be important subgroups of users in the broader population groups such as adolescents who have special needs but are hard to reach for getting feedback. For example, young people in crisis, or homeless, or in precarious circumstances, may be hard to retain in hospital situations. They may not attend follow up, may not wait long in casualty, may not have an address or phone, or may not be literate or interested enough to fill in a survey.
43
44
CHAPTER 5: COMPLAINTS AND ADVERSE INCIDENTS Records of health service complaints, and research into consumers’ and doctors’ experiences of adverse incidents, are important data sources which complement the information on quality care and sources of satisfaction, discussed in chapters 2-4. Some of the British research into patient satisfaction that led to the development of audit instruments, for instance, used complaints data as one of the sources to ascertain the ‘full experience’ of consumers. The information on general matters about hospitals collected by consumer organisations (and discussed in chapter 2), rarely throws up issues of concern with treatment, concerns with technical care, or legal proceedings. It is not clear why this is so but may be due to the framing of the research question. A selection of key issues from complaints data and research into medical mishaps has therefore been included in this chapter to illustrate these issues are important for consumers, and to highlight that existing methods of obtaining information on the ‘full experience’ of consumers are deficient. Only four governments have established independent complaints authorities: New South Wales, Queensland, Victoria and the Australian Capital Territory. The last has only just been established and data is not yet available. For this chapter, an overview of the data from annual reports from the other three States has been taken to illustrate the broad areas where consumers make complaints. For both States/Territories with a complaints authority, and in those without, consumers are encouraged or have to make their complaint direct to the hospital or provider concerned. A number of Victorian hospitals, for example, have databases on complaints. This information has not been sought for this project due to time considerations, but would provide a valuable data source for people developing a questionnaire. Finally, the chapter briefly considers some information from research into adverse incidents, highlighting both the importance of consumers’ experiences, and methodological issues of researching complex events. Main areas of complaint Victoria and Queensland use a similar system of categorising health complaints. From their Annual Reports of 1993-1994, the following breakdown of complaints is presented. For each, the total number of complaints is noted, with the percentages by area of main complaint.
45
Victoria, 1929 complaints, Jan 93 - July 94 Treatment Communication Rights Access Costs Administration
48% 15% 13% 11% 10% 3%
Queensland, 2257 complaints, July 93 - June 94 Treatment Access Communication Rights Costs Administration
40.5% 17.5% 15.3% 13.4% 7.1% 6.2%
The Annual Report 1993-94 for the complaints unit of NSW (now the Health Care Complaints Commission) presents their complaints data as follows: New South Wales, 1475 complaints, July 93 - June 94 Treatment Professional Conduct Communication Diagnosis Services Hygiene Pharmaceutical Records Assault Consent Other issues (less than 1% of complaints each issue)
42.7% 13.2% 11.2% 8.4% 3.1% 3.1% 3.1% 2.9% 1.5% 10.8%
The main service/provider groups against which complaints are made are as follows: Victoria Hospitals Medical Practitioners Dental Government Nurses/Allied Health
36% 41% 9% 6% 4%
46
Other Organisations Health Insurers Nursing Homes
2% 1% 1%
In New South Wales the data is broken down by profession and service type. For 1993-94, public hospitals attracted the majority of complaints among service types (58.3%). Nursing homes attracted 8.5%, private hospitals received 6.2% and psychiatric hospitals 5.7% of complaints to the Complaints unit. In New South Wales, doctors were the subject of 80.8% of complaints, followed by nurses at 6.3%. In Queensland, there is a further breakdown of data. For example, the category of doctors is broken down by speciality. In 1993-94, the figures are as follows: General Practice Other Specialists Other Surgery Obstetrics and gynaecology Orthopaedic Surgery
52.6% 23.2% 9.4% 7.4% 7.4%
Issues arising from the data Only a small sample of the data in annual reports of complaints authorities has been presented here. Further, each report discusses the main areas of complaint with examples. These illustrate the complexity of certain situations, as well as the data’s richness. It would be of assistance for the various bodies to adopt a common system of reporting on complaints, and this is presently being discussed by the Commissioners. It would also be of interest to know how the issues of complaint match the provider/service category. It is not possible to know from these reports whether particular types of complaints are made about one type of provider/service more frequently than another. The dominance of complaints about treatment in all three states’ data sets illustrates that consumers do have concerns about the technical quality of care, and where possible, make those concerns known. This information must therefore be seen as complementing the information presented in chapter 2. Research into adverse incidents The following two short sections concern research into consumers’ and doctors’ experiences of adverse incidents. They have been included to highlight the primacy of qualitative indepth research techniques for investigating complex events. Medical/health care injury and the compensation system As part of the Review of Professional Indemnity Arrangements for Health Care Professionals, research was conducted into the experiences of people who had received a medical injury, and
47
who had sought compensation (Commonwealth Government. Review of Professional Indemnity Arrangements for Health Care Professionals 1993). Twenty-four case studies were undertaken, using semi-structured in-depth interview techniques. A few salient comments from the discussion of findings have been included here (1993:77-81). Many had taken action because of a sense of the importance of preventing future incidents. However, study participants believed the health system and personnel were not geared to this aim. Rather, it was felt that professionals were unaccountable for their actions. The process of seeking compensation was very stressful. Those who fared worst were those more vulnerable to start with. The authors suggest that there may be many people in this situation who have not mounted a claim due to the many barriers. The report concurs with other research suggesting that poor communication after the injury may accelerate legal action. The message from the material - that many people felt that the prevention of incidents was not an aim of the system and that health professionals were largely unaccountable - rings a warning bell for those implementing patient feedback mechanisms. The legitimacy of those efforts will only be assured if there is clear commitment to acting on the results. The research into adverse incidents illustrates the value of indepth case studies in an area where little is known. Adverse incidents – research sponsored by the Professional Indemnity Review, Commonwealth Department of Human Services and Health As part of the Review of Professional Indemnity Arrangements for Health Care Professionals, various research projects have been funded to investigate the incidence of adverse patient outcomes resulting from medical care, and the nature of preventive strategies to reduce their occurrence. Seven projects were undertaken, under the auspice of the Clinical Colleges of general practice, anaesthetics, intensive care medicine, emergency medicine, psychiatry, obstetrics and gynaecology, and endoscopic medicine. Summarised below are methodological issues or research results pertinent to the consumer's stay in hospital and their care post hospital. General practice Analysis of 402 critical incident reports by general practitioners (GP) showed that along with other contributing factors to adverse incidents occurring, the poor communication between hospitals and GPs was an area where major improvements should be made (RACGP 1994:73). There were consistent reports of the lack of feedback to GPs by hospitals on tests and investigations performed, drugs prescribed, and treatment and further care required.
48
The report noted (1994:73) that "considerable reliance is placed on the patient to co-ordinate his or her own care, report events or re-present for follow-up." Furthermore, those most vulnerable to problems flowing from this lack of communication were "aged patients, with multiple problems and medications, those from non-English speaking background backgrounds and those with psychological conditions". Research into critical incidents by various specialties Most of the research by specialties involved using a combination of quantitative and qualitative methods. The quantitative methods were used to gain factual information while the qualitative reports asked people to answer what happened, why, was it preventable type questions (Beckmann 1994; Ryan et al. 1994; Vinen et al. 1994; Wright & Parker 1994). Narrative sections or open ended questions directed to these matters were (in the words of one set of researchers) designed "to elicit the meanings and processes of each incident" (Ryan et al. 1994:56). The qualitative material allowed themes to emerge, which in turn pointed to issues of context or the absence of adequate infrastructure (pp. 59-60). The Quality in Australian Health Care Study (Australian Hospital Care Study) This major study is based on the Harvard Medical Practice Study. However, while the Harvard study focused on negligence, the Australian study focused on preventability. This latter focus results in more focus on minor, temporary disability than in the Harvard study, which not only is important in terms of hospital expenditure, but also consumers’ experiences of quality care. The results of the first part of this study are due to be published in the Medical Journal of Australia in early November. The study has found that 16.6% of admissions involve an adverse event. This can involve more than one admission for the individuals concerned. About half of these involve temporary disability of less than one month, but in 20% of adverse events, the result was permanent disability or death. It should be noted that this is often in the context of intervention where serious illness is already present. Not all adverse events were judged to be preventable. The risk to a patient of an adverse event increases with age. As the results are still in press, the focus here is on aspects of the methodology. The first stage of the study involved a quantitative study to discover ‘how many’. There is little in these results which provides information about why these events occurred, or what could be done to prevent them. This information relies on the second stage of the study, a qualitative study of the texts on the adverse events. This is a point that will also be made about patient satisfaction surveys in chapter 9. Issues arising from the research Analysis of complaints data and adverse events are important sources of information that need to be combined with other quality data, including patient feedback, to provide a composite picture of hospital quality. In all these areas, the ways in which quality needs to be addressed involves considering how hospitals work as organisations which need to ensure safe, effective, responsive care as well as meeting efficiency and equity requirements.
49
50
CHAPTER 6: OTHER REPORTS OF HOSPITAL EXPERIENCES In this chapter are listed the findings of a selection of reports which have dealt in some way with people's experiences in hospital. They have been included to highlight aspects of the consumers' experience, as well as barriers to better care experienced by doctors, and to illustrate some different kinds of research currently underway in Australia. The chapter includes brief summaries of a sample of patient satisfaction surveys published in the academic literature which have dealt with various aspects of care. It is stressed that this is a selection of material only, and there is a considerable number of research reports to draw on. However, the examples chosen illustrate an important point. Research into how the system works and how people fare tend to favour multiple methods, ie. suiting the method to the question asked. Research into ‘patient satisfaction’ or ‘patient experiences’ are less likely to use a combination of methods to gain data. Links between hospitals and services in the community South Australian Community Health Research Unit study: Links between hospitals and services in the community Research by the South Australian Community Health Research Unit (SACHRU) into the links between hospitals and community-based services, including general practitioners (GPs) (SACHRU 1994a), is the first of three studies into the coordination of people's care across a range of services. The study involved interviews with one or two people from all health organisations – and in larger organisations – all departments, in the southern metropolitan region of Adelaide. There were no refusals for interview. Among the findings was the recognition that coordination at the service level was happening, and was supported. However, the inadequate level of resources to some community-based services, for example, mental health services, meant that a well planned discharge was not always possible. A finding specific to hospitals was that of the hospital departments which provided services in the community, 13 had conducted a formal evaluation of patient care, by patient satisfaction survey for instance (1994a:25). Eight had informally reviewed the services they provided outside the hospital. South Australian Community Health Research Unit study: Problems experienced by GPs The second project (SACHRU 1994b) examined the links between GPs and others. The research with GPs was undertaken by survey, which was developed from focus groups and from the earlier survey of hospitals and services in the community. The survey included pre-coded and open questions, and 63 out of 100 GPs (63%) responded. The results relevant to this report are:
51
•
GPs were least satisfied with the referral process to specialists in public hospitals (compared to referring to private specialists), because of waiting times, and the difficulty in nominating a specialist or retaining the same one for the patient;
•
there was considerable dissatisfaction with the continuity of care between the GP and the public hospital. Particular problems were poor or non-existent communication and feedback, and late discharge summaries;
•
discharge summaries were not only late but also of poor quality. Phone contact was noted as important in many cases.
South Australian Community Health Research Unit study: Problems of uncoordinated care experienced by consumers The SACHRU is presently undertaking a third study in this series of the links between services, and is examining consumers' experiences. A number of consultations have been organised involving the five South Australian Health and Social Welfare Councils. A report will not be available until late 1995. A literature review on obtaining the views of consumers in primary and community-based services will also be published in 1995. Older people in Dubbo – their use and experiences of hospital services The Office on Ageing in NSW has sponsored a longitudinal study of the use of hospital, residential and community services by older people living in Dubbo. The report by McCallum and his colleagues (Office on Ageing, NSW 1994) documents the findings of research with 2805 people over a period of 50 months. The discussion here concerns only the findings of a telephone survey and focus groups with a sub-sample of people, about their experiences of hospitals and care after discharge. Care after discharge All of the people discharged in the last six months of the investigation period (n = 152) were telephoned and asked open-ended questions about Home and Community Care services (1994:22-23). Of these, 19% reported instances of unsatisfactory care or difficulties in accessing services. Very few people wanted extra time in hospital: most were happy to be discharged when they did, while some would have liked to come out earlier. Hospital experiences Focus group meetings were held with 15 of those telephoned (eight men and seven women), and who had been part of the study and discharged from a hospital during a nominated period (1994:23-8). The groups were split along gender lines to facilitate and improve discussion.
52
People were very positive about their local services. Some of the men commented that they did not want to go to Sydney for care. The women noted the comfortable and caring approach of local hospitals. There was an appreciation of improvements in hospital meals. Most were happy with the timing of discharge: some wanted it earlier, although there were anecdotes of problems experienced by others. There was some discussion of poor communication between specialists and the older people. One communication breakdown resulted in an adverse outcome. Men were less reflective of difficulties but there were illustrations that it could be a problem for them as well. Waiting lists and times were not an issue for the men, although they were for women, who worried about access for their grandchildren and themselves. An important methodological issue was raised in the report. In the telephone interviews, the two people experiencing adverse events had reported positively on the quality of services, whereas in the focus groups they had been much more concerned to discuss these incidents. The authors note that their research could suggest that the number of adverse incidents could be higher than expected and there is the presumption that this data would not be elicited by telephone interviews. Patient satisfaction surveys of specific services A search of the two major journals for reporting health service research which focuses on quality of care issues - 'The Australian Clinical Review', now 'The Journal of Quality in Clinical Practice' and 'The Australian Health Review' - reveals the relatively sparse reports on patient satisfaction. The area of paediatric care has received attention, due to its specialised nature. There are two surveys of day surgery care, and a selection of survey reports of various aspects of care. The following summary highlights the main findings of some of the key pieces. Paediatric care There are three articles dealing with aspects of care: parents' responses to the reporting of cases of non-accidental injuries (Lawrence & Harrison 1994); parent satisfaction with paediatric physiotherapy services (Skinner 1991); and a survey of why parents stay overnight on wards when their child is in hospital (Thomas et al. 1989). McIver, of the Consumer Feedback Centre at the King’s Fund Institute in England, suggests that children over ten years could be interviewed in appropriate circumstances. In Australia, research into attitudes towards smoking has surveyed children as young as ten years. However, these surveys have not done so, and parents are the respondents in all cases.
53
Day surgery units Two articles deal with surveys of day surgery units: O'Connor, Gibberd & West (1991), and King (1989). Both surveys were of day surgery units in New South Wales. The results indicate the importance of tailored surveys, as factors such as a follow up phone call after day surgery was a critical issue for many people. The article by King (1989) is one of the few sighted which discusses some of the changes made in response to the pilot and then subsequent survey, as well as raising the issue of the importance of sharing the survey results with all staff. Detailed survey of psychiatric inpatient care Taylor and Clarke (1993) present the results of a long survey of people who were or had been inpatients of a psychiatric hospital. Of interest in their results is the critical role communication between staff played in leading to better outcomes for patients. The article makes the additional points: •
that the survey demonstrates that psychiatric inpatients are able to provide reliable data with good response rates;
•
notes that the survey questions were developed in consultation with ex-patients, members of mental health groups, patient advocates, and carers;
•
notes that some elements of care may be rated very poorly, but in the patients' view, are not critical factors in improving outcomes.
Surveys of various departments The journals also contain articles about various hospital departments, including a review of patient complaints and of those who walk out from accident and emergency departments in the Camperdown Hospital, New South Wales (Hanson, Clifton-Smith & Fasher 1994); food services in New South Wales (Williams & Brand 1988); patient education in Wollongong Hospital (Stuber, Sheehan & Sisko 1986); and staff-patient communication in a Brisbane hospital (Sinclair & Ward 1985). It is noted that these reports rarely build on other work (with the exception of the accident and emergency report), and there is no published follow up of the implementation of any changes flowing from the surveys.
54
CHAPTER 7: STATE GOVERNMENT PATIENT SATISFACTION SURVEYS The purpose of this chapter is to survey what the various states are doing in relation to patient satisfaction surveys at a state wide level. It includes some background on the general policy context in which the patient satisfaction was undertaken, information about the survey in use, the details of administration, and reports of the results where these are available. There is also a comment on what was done with the results. The chapter ends with summary comments on state wide patient satisfaction surveys. NEW SOUTH WALES Policy context NSW developed a “Customer Focus” policy under which the patient satisfaction survey was developed. This reflected the broader agenda of the Greiner and Fahey governments. The term ‘customer’ was used to include staff, the government, medical practitioners and so forth as well as patients and the community (external and internal customers). Customer focus had a strong emphasis on a quality improvement framework. It was implemented through a range of projects in individual hospitals. These projects were called “Pathfinder Projects” and focused on issues such as quality improvement and identification of the cost of poor quality. Customer focus forums were held and a newsletter produced to get providers to think about services as perceived by customers. Involvement of consumers in health services was to be facilitated by the introduction of Customer Councils to be established in 1995. This policy appeared to be essentially a cultural change approach and facilitated a number of innovative projects aimed at reorienting services. The private sector market approach it relied on did not win over potential allies. The ‘customer’ approach used a language that sat at odds with the traditional language of hospital care. Neither providers nor consumer organisations were at home with it. There seem to be few examples of consumer partnership projects. The policy has been controversial and was disbanded by the incoming Labor government. Another relevant program is the Health Outcomes Program. The emphasis is on results and the focus is on disease conditions. This Program relates to best clinical practice, prevention, quality review and consumer and provider information. The emphasis of the policy is on informed decision making. This has included some work on patient satisfaction with care and identification of priority outcomes in a number of settings. Outcomes Councils have also been developed at the local level. Consumers have been involved in the range of structures, most effectively at the level of specific health problems. Diabetes was identified as the prototype condition for the planning and implementation of services to improve outcomes, with consumers involved in all aspects.
55
History The NSW patient satisfaction survey originated in 1992 as part of the Customer Focus Policy. The aim of the survey work was not only to quantify patient satisfaction levels, but also to examine expectations, reasons for satisfaction and dissatisfaction and particular aspects of care. The development and management of the NSW patient satisfaction survey is contracted to TQA (consultant Tony Quint) on behalf of the Department of Health. Method Questionnaires were handed to patients at time of discharge, completed at home, and returned to TQA. There were four questionnaires • inpatient and general admissions • day surgery • accident and emergency • outpatients The inpatient questionnaire took about 20 mins to complete. Follow up on concerns about accident and emergency which emerged in the first pilot were researched using focus groups. Questionnaire design The questionnaire is based on the US Hospital Corporation of America (HCA) questionnaire, distributed by the Medical Outcomes Trust. This questionnaire is discussed in chapter 11. Focus groups with consumers were used to check out the areas of concern and how to phrase questions. 1. 1992/93 Pilot Sample The sample was drawn from 11 hospitals, - urban, rural base, and rural. Response rate Response rates for individual hospitals were from 33% to 71% The overall response rate was 45% - 3,116 questionnaires Results The results are published. Individual hospital results are available to hospitals. All results which are reported here for the 1992/93 pilot and subsequent surveys are taken directly from
56
TQA reports. Tables are also reported direct from TQA reports as indicated. This provides a flavour of the information available from the survey. 95% of respondents were very or fairly satisfied, with a Patient Satisfaction Index of 87. 74% were definitely prepared to recommend the hospital to others. (Note - this suggests that the difference between very and fairly satisfied does matter and contains important information. The categories very satisfied and fairly satisfied are combined to give the overall satisfaction percentage.) Overall comments • • • • •
perceived quality of care and treatment is the dominant driver of patient satisfaction nurses are perceived to be the best asset of public hospitals doctors rate high overall, but there are issues in the communications area there are problems with A&E - waiting times and waiting environment meals are generally good, but the heat of meals a problem in some large hospitals
Key sources of dissatisfaction These are revealed in questions about why the hospital is not recommended. The consultants report suggests that this is a key question area and is able to discriminate between hospitals. Patients are forgiving of hospital shortfalls in rating their own experience. The report suggests a hierarchy of concerns and that patients are also more concerned about communication and personal attention than about environmental factors. Key sources of dissatisfaction were: • waiting times, • lack of personal and individual attention, • perceived uncaring attitude of some staff, • lack of explanation and communication, • environment in need of renovation, • lack of cleanliness. Key sources of satisfaction Quality of care and treatment and personal/individual attention are key drivers of satisfaction • patients in pain are less satisfied • patients needing a lot of assistance are more satisfied • where the condition is not helped, patients are less satisfied • younger patients are less satisfied about aspects of care • people from non-English speaking backgrounds are less satisfied, although results are good overall
57
Table 7.1. The Key Statistics - 1992/93 NSW pilot - TQA Report MEASUREMENT
INDAY A&E PATIENTS SURGERY 89 90 80
OUTALL PATIENTS AREAS 88 97 (sic)*
97
96
88
96
95
% 'DEFINITELY RECOMMEND' 75 HOSPITAL TO OTHERS
81
64
74
74
PATIENT SATISFACTION INDEX (0-100 SCALE) % OF PATIENTS SATISFIED
KEY PERFORMANCE INDEXES (0-100 SCALE) z Quality of Care & Treatment
91
92
94
84
91
z Information Given
79
81
74
82
79
z Knowing You as an Individual
79
81
74
82
76
Person
89
91
80
85
87
z Doctors Overall
83
N.A.
N.A.
80
82
z Doctors' Communications
92
94
86
89
91
z Nurses Overall
86
N.A.
N.A.
84
85
z Nurses' Communications
68
78
N.A.
N.A.
71
z Restful Atmosphere
76
N.A.
N.A.
N.A.
76
z Buildings
N.A.
N.A.
67
N.A.
67
z A&E Waiting Environment
74
N.A.
N.A.
N.A.
74
z Meals
52
53
N.A.
N.A.
53
z Parking * This figure is as reported in the table in the TQA summary, but based on the figure reported elsewhere in the TQA Report, the correct figure appears to be 87.
58
Areas of care and treatment that could be improved • • • • •
knowing patient as an individual person, introductions to staff, recognition of patient’s opinion informing family and friends, information given.
Additional useful information is provided in the report, for example, 29% of inpatient and day surgery patients were not asked if they had enough support when they arrived home. 2. 1993 A&E focus group follow up research This was indepth qualitative feedback on patient perceptions of service delivery, with details of likes and dislikes and suggestions for improvement. Waiting times attracted the most comment. It was not so much having to wait that mattered, but the lack of communication, acknowledgment and information updates which mattered. Other issues were also raised which pointed to organisational issues, communication and information. 3. 1993/94 Consumer Satisfaction Survey Sample 34 Health Services • General Hospitals • Mental Health • Community Health Five of the 34 hospitals had been in the 1992/93 pilot. Response Rate 20, 310 questionnaires were distributed The overall response rate was 38% - 7, 722* questionnaires General Hospitals Mental Health Community Health
6,121* 295* 806*
* These figures are quoted from the TQA report. The quoted figures add up to 7,222.
59
Table 7.2. Key Results - 1993/94 survey - TQA Report KEY INDICATORS − ALL SERVICE AREAS INDICATOR
GENERAL MENTAL COMMUNITY ALL HOSPITALS HEALTH HEALTH SERVICE HOSPITALS/ SERVICES AREAS CLINICS
CUSTOMER SATISFACTION INDEX (1-100 SCALE)
84
82
90
85
% Customer/Clients satisfied
94
91
98
94
% Customer/Clients very satisfied 61
56
72
62
% 'Definitely recommend' to others % Saying 'worse than expected'
72
68
82
73
5
7
2
5
Performance Ratings (0-100 Scale) Care, Treatment & Communication
INDICATOR
COMMUNITY ALL GENERAL MENTAL HEALTH SERVICE HOSPITALS HEALTH AREAS HOSPITALS/ SERVICES CLINICS
Quality of Care and Treatment
89
84
93
90
Compassionate, Reassuring Attitude Knowing you as an Individual Person Information and Instructions
82
N/A
88
82
72
N/A
N/A
72
77
80
85
79
Introductions
69
78
82
72
60
Key Results - 1993/94 survey - TQA Report (cont.) Staff
INDICATOR
COMMUNITY ALL GENERAL MENTAL HEALTH SERVICE HOSPITALS HEALTH AREAS HOSPITALS/ SERVICES CLINICS
Doctors − overall
84
81
87
84
Doctors − information and communication Nurses − overall
79
79
N/A.
79
90
87
89
90
Nurses − information and communication Main Person who helped you
82
82
N/A
82
N/A
N/A
90
90
83
N/A
83
N/A
96
96
Case Manager − how well he/she N/A helped N/A Home Nurse − concern, caring attitude and personalised attention
Comfort/Meals
INDICATOR
COMMUNITY ALL GENERAL MENTAL HEALTH SERVICE HOSPITALS HEALTH AREAS HOSPITALS/ SERVICES CLINICS
Condition/Look of Room (Inpat. Only) Cleanliness of Ward Toilets/ Showers Restful Atmosphere
75
80
N/A
76
79
80
N/A.
79
68
68
N/A
68
Comfort of Bedding
69
76
N/A
70
Meals
75
72
N/A
75
61
Key Results - 1993/94 survey - TQA Report (cont.) Waiting Aspects
INDICATOR
COMMUNITY ALL GENERAL MENTAL HEALTH SERVICE HOSPITALS HEALTH AREAS HOSPITALS/ SERVICES CLINICS
Emergency Department -− % saying treatment wait UNACCEPTABLE
17
N/A
N/A
17
− % left in cubicle with no communication for 20+ minutes
15
N/A
N/A
15
OUTPATIENTS/CHS WAIT − % saying waiting time unacceptable
18
10
5
16
SINGLE DAY ADMISSIONS − % wait over 10 minutes to be attended
11
N/A
N/A
11
Again the question about whether patients were prepared to recommend the hospital was a key indicator and discriminated between hospitals from 96% for highest performing hospital to 62% for lowest performing hospital. What was done with the results? Reporting of results of the 1993/94 survey was made available publicly, including a pamphlet. The pilot had identified problems with A&E. This was then followed up with focus groups and with Pathfinder projects. Individual results were made available to hospitals. Comparisons in five hospitals common to both samples showed some improvements in waiting times. Other comments The Report comments that patients are not inclined to voice their complaints or express their concerns while in hospital, and that it is important to involve parties from outside the health delivery organisations.
62
TABLE 7.3. Drivers of overall satisfaction General Hospital In Patients (NSW 1993/4 survey) Interpretation: The closer to the top of the chart, the more important is the factor. The size of each box shows the proportional contribution made by each factor to customer satisfaction. The index allows comparison of importance to the most important aspect, CARE AND TREATMENT (100). Within each box are the most important variables 'driving' the overall factor, in order. INDEX 100
1.
Care and Treatment − Compassionate, Reassuring Attitude of Staff − Information and Instructions − Knowing you as an Individual Person − Recognition of Your Opinion − Introductions − Informing Family and Friends
INDEX 89
2.
Physical Facilities − Condition, Look and Appearance of Your Room − Cleanliness of Room − Cleanliness of Ward Toilets, Showers, Bathrooms − Restful Atmosphere − Privacy − Comfort of Bedding
INDEX 86
3.
Nurses − − − −
Concern, Respect, Caring and Personalised Attention Attention of Nurses to your Condition Information and Communication of Nurses Skill of Nurses
INDEX 61
4.
Doctors − Information and Communication − Concern, Respect and Personalised Attention − Attention of Doctors to your Condition − Skill of Doctors
INDEX 54
5.
Meals − − − −
Quality of Food Range & Appeal of Menus Food Temperature Receiving the Meals you Ordered
63
See cautionary footnote
INDEX 54
6.
Discharge Aspects − Discharge Instructions − Discharge Efficiency − Adequate Notice of Discharge
INDEX 54
7.
Other General Aspects − Meal Delivery Staff − Housekeeping Staff − Hospital Kiosk − Others
INDEX 33
8.
Admission Aspects − Efficiency of Admission Procedures − Manner and Courtesy of Admission Staff − Preparation for Admission
Footnote:
Physical facilities tend to play a role when they are very good or very bad, but not so much between these extremities.
The consultants commented in discussion that two aspects can affect the sample: having health staff hand out the questionnaire runs the risk of biasing the sample by omitting patients who for one reason or another staff judge to be unsuitable, and the procedures demanded by ethics committees can introduce bias in the sample through cumbersome processes to protect privacy and gain consent. VICTORIA Policy context Responsibility for quality is located in two different sections of the Department of Community Services and Health. Public Health is particularly concerned with health outcomes, information, guidelines, the Cochrane collaboration on clinical trials on patient information, and data collection. The Services Development section in the Acute Hospital Section is responsible for the patient satisfaction survey and is currently releasing a discussion paper on quality. The emphasis of the current Victorian policy in relation to hospital services focuses very much on the structure of services, as indicated in Victoria’s Health to 2050, and more recently in Towards a New Framework for Quality in Victorian Hospitals on the structures for monitoring quality
64
Under the heading of “Patient Focussed Hospitals”, Victoria’s Health to 2050 refers to patient involvement in outcome assessment and patient satisfaction. Otherwise, ‘patient focus’ seems to address issues of “seamless” service delivery, managed competition, marketing and location of services. Under the heading of “A Focus on Quality”, the emphasis is on access; clinical practice guidelines; mandatory standards; industry wide indicators; and disease specific measures monitored by a third party. The emphasis in terms of consumer input is on provision of information to consumers. Information is the key to the current Victorian approach and is consistent with the government’s broader competitive market approach. However, there seems to be no broader role for consumers beyond the role of “controlling” clinical decisions through informed decision making. Information about the results of the Victorian Patient Satisfaction Survey, and the kind and detail of the results thrown up by this approach will only be available when stage 2 of the survey is completed, although hospitals have not yet agreed to any publication of results. For the purposes of this review, it is not possible to make any comments about how well this questionnaire performs until there are some results available for analysis in the public realm. Results published by Cleary and his colleagues (1991) and Gerteis and her colleagues (1993) on the use of this questionnaire in the US setting provided an indication about what the results could be expected to look like. This would suggest that the questionnaire has some power to highlight areas for improvement. One issue worth exploring further from this questionnaire is the consumer responses to the telephone interview. Ethics committees are often concerned about privacy issues in this type of research. However, anecdotal responses to this and other hospital initiated patient research suggests that many patients are often very happy to talk about their experiences and interpret such interviews positively as indicating concern about their experience and recovery. While there have been many problems with the development and use of this survey, the Victorian questionnaire and the NSW questionnaire have placed the quality of patient satisfaction surveys firmly on the agenda and taken Australian patient satisfaction surveys onto a new level of professionalism. However while the use of consultants with good expertise has strengthened the validity, reliability and analysis of questionnaires, there still remains an issue of what is done with the results and how such information is used in the management of hospitals. Discussions with staff of CASPE in the UK reveal a similar concern there. We have alluded elsewhere to the poverty of literature on this aspect of implementation. The Victorian Patient Satisfaction Survey has been very slow in development and no information about the actual results are in the public realm, despite a development process that commenced in 1992. This is problematic in a policy context which relies on information to ‘empower consumers’. It is fair comment, at the same time, that the initial pilot sample of two hospitals made those hospitals very identifiable while the willingness to participate suggests good will by those hospitals.
65
The second stage has been held up first within the department and then by hospitals which have sought assurances that individual hospital results will not be available under Freedom of Information (FOI) legislation. The department has agreed to seek exemptions from FOI. During the first pilot phase, the department released individual hospital mortality rates as a response to an FOI application without prior consultation with hospitals. This appears to have adversely affected the progress of the patient satisfaction survey. There are some suggestions that the relations between hospitals and the department in a turbulent policy environment continued to make this sensitive. These issues do not appear to have dogged the NSW survey, despite the controversial nature of the Customer Focus Policy of which it was part. Doubt still remains about what will be reported in the public realm. It is also worth considering that the initial decision to use the survey as a means of monitoring quality in individual hospitals as part of the casemix policy, rather than as a policy of improving quality, could also be a key to its bumpy gestation, as it has diminished the ownership of the project by hospitals. History Victoria decided to embark on a patient satisfaction survey in 1992 and convened a reference group. This was part of the development leading to the introduction of casemix funding in Victoria. The patient satisfaction survey was announced as one of the quality measures to accompany the introduction of casemix funding in June, 1993. In September 1993, a pilot was undertaken in two hospitals, the Austin and Bendigo hospitals. This first pilot was conducted by Ramis Corporation for the Department of Health and Community Services. A report was made to the department in February, 1994. In 1995, a second stage of implementation is currently being undertaken involving 25 hospitals. This is being conducted by TQA (consultant Tony Quint) for the department. Method This patient satisfaction survey is a telephone questionnaire conducted by a consultancy organisation. Interviews were/are conducted post discharge within three weeks of discharge. The telephone interview takes about 27 minutes on average. Telephone interpreters were used for some language groups. Questionnaire design The questionnaire is based on the Picker-Commonwealth questionnaire, developed in Boston US and used also in research at McMaster University in Canada. This questionnaire is discussed in chapter 11. This survey has been used by Cleary and his colleagues (1991) to conduct a nationwide survey in the US and by McMaster to conduct a Canadian survey. The survey has also been adapted for a UK study of NHS hospitals (see chapters 9 and 11). Some of the questions were adapted for Victoria and the method of scoring the overall index of care was altered. There are further adaptations in the second pilot, which appear to have been fought out by a committee. This questionnaire is built on questions about what happened to patients rather than judgments about satisfaction.
66
1. The 1993 Pilot Sample The sample comprised every patient discharged in the two hospitals for a two week period which was extended to three weeks to get the sample. There were some exclusions, including those transferred to nursing homes, those with psychiatric conditions, hearing disabilities or intellectual disabilities, and those who did not speak English, Italian or Greek. Response rate The total sample was 591. The total number contacted was 470. Interviewing ceased when 310 patients had been interviewed. This is a response rate for those contacted in the midsixties (67.1% and 63.7%). Consent Hospital ethics committees required an active consent process. This involved getting the patient to sign a consent form prior to discharge. Patient information sheets and consent forms were translated, and interpreters used to explain the project. These were not available in one third of instances. Signed consent forms were retained by the hospital, but not given to the patients, who took home the patient information sheet. Letters were also sent to the patients by the department about the project. In Stage 2, the consent process is being handled by non clinical staff. Results These are not available as the first pilot was aimed at testing the tool and for reasons outlined later. The Report on the Pilot canvassed issues about the methodology and administration of the questionnaire. This survey, following Cleary et al(1991), uses an overall index of care. People from non-English speaking background There was a lower participation by people from non-English speaking backgrounds. The Report notes that: • • • •
telephone interviews may not be appropriate for this group there may be a reluctance to identify care as unsatisfactory and that this is affected by cultural factors many of the non-English speaking background patients were older patients a trial of use of personal interviews was recommended for Stage 2.
For Victorian hospitals, and for some in particular, people from non-English speaking backgrounds constitute significant proportions of users of hospitals, so this issue is important. This suggests that some caution needs to be exercised in generalising from patient satisfaction information for people from non-English speaking backgrounds. 67
Other issues Some questions were raised about the length of the questionnaire, the relevance of some of the questions for the purposes of Victorian hospitals and extent to which hospitals could act on the results. It was recommended the sampling arrangement was unsuitable for hospitals with less than 100 patients, which are excluded for this reason from the large US surveys undertaken using this questionnaire (Cleary et al 1991). In Stage 2, smaller hospitals are to be aggregated. What was done with the results? Mostly the pilot results have been used to inform Stage 2. Little appears to have been done with the substantive results despite attempts by hospital staff to provide reports on these results. The pilot nature of the questionnaire was part of its lack of legitimacy and the questionnaire tended to have a lukewarm press around Victorian hospitals with routine comments about its length. Stage 2 will need to test whether hospitals do anything with the results. There is little hospital ownership of the patient satisfaction survey to date. 2. Stage 2 This is currently under way. Interviews took place in August/September, aiming at 5,200 interviews. Further work is being undertaken through personal interviews with people from non-English speaking background to explore how to get reliable feedback from this group. We understand this remains a problem area. The consultant has advised a parallel process of focus groups to get more information about drivers of satisfaction and dissatisfaction. Reports to individual hospitals will compare hospitals against the mean for the group of hospitals. QUEENSLAND Queensland has a regional approach to health services delivery in the context of a large geographical area, multiple regional cities and considerable population variations. Regional decision making can be fiercely defended. Budgets are allocated to regions on the basis of a resource allocation formula based on a population based formula. Casemix is used to set prices for hospital activity, These prices are set on state wide cost weights. Queensland has undertaken some patient survey work in the context of specific service evaluations. Under the Finance and Audit Act, programs are to be evaluated at least every 5 years. All programs are required to develop a set of client standards by 1 July, 1996 which are
68
then to be monitored and reported on. A patient satisfaction survey was done in 22 major casualty departments, as part of the lead up to emergency medicine becoming a program on 1 July 1995. The patient satisfaction survey was funded by the evaluation section as a way of getting a client view into evaluation. and developing an information base. Individual hospitals and sections of hospitals, as in all states, have been doing some patient survey work. It was described as ‘data without theory’, and like the early generation of patient satisfaction surveys in Australia often producing results that are too generalised to be useful in providing information for improving services or not capable of identifying the key changes that affect patient satisfaction. The patient satisfaction survey of A&E was contracted to TQA (consultant Tony Quint) as a way of utilising pre-existing experience and with the intention of some interstate benchmarking. The latter has not happened at this stage. The HCA survey is the basis of the Queensland survey. Sample The study was done of 21 A&E departments selected by the Health Department, and chosen for different levels of ‘traffic’. Site coordinators distributed the questionnaires at each hospital. A further sample of patients admitted through A&E was conducted at two sites for purposes of comparison. Due to logistical and staffing problems such as lack of cooperation at some hospitals, it is suggested that the sample is biased towards quieter periods. Method On specific days, questionnaires were handed to all patients or their relatives at the time of departure from A&E. Questionnaires were posted direct to TQA. Questionnaire design This is a modified version of the Health Corporation of America (HCA) questionnaire the inpatient version of which was also the basis of the NSW survey. The Health Department identified a range of issues in which they were interested. Response rate The response rate was 29% with 1,898 usable questionnaires. In terms of some significant subgroups, there were 59 completed questionnaires from people from non-English speaking background and 43 from Aboriginal and Torres Straight Islanders. Aboriginal Liason Officers were available for assistance but were not utilised. Results Overall satisfaction was 87%, varying from 57% at Thursday Island through to 93%. Older patients expressed higher levels of satisfaction. The issues identified with satisfaction were:
69
• • • • •
waiting time to receive medical attention lack of understanding and communication if wait prolonged dissatisfaction with overall efficiency of hospital lack of introductions by staff and explanations of their roles difficulty with car parking.
The average range of waiting times varied from 13 minutes to 80 minutes, with 8% of patients waiting two hours, and 5% three hours. The tolerance range of time for waiting was between 40-60 minutes. Once again the question about whether consumers were prepared to return to this A&E department and to recommend it to others was used to identify some key drivers. Some key issues identified were: • • • •
long wait for attention, lack of staff, uncaring/unhelpful staff, inexperienced staff/doctors.
The environmental factors were rated as less important than the staff issues. While this question about recommending the hospital to others in both NSW and Queensland surveys produces some useful information, realistically, some consumers have no real alternative, particularly in places like Thursday Island. Consumers identified prompt attention and caring, personal attitudes of staff as key expectations. There is some important information reported in the survey about the reasons for using A&E, for example, the groups which had a more than average use of A&E included: • • • •
patients with disabilities patients from non English speaking backgrounds older patients Healthcare Cardholders.
They often had quite logical reasons for thinking that this was the best option. What was done with the results? Hospitals are reported by the Health department to have found the results useful and used the information as an aid to problem solving. A state wide report of the results is available. Comparing patient and provider views about consumer expectations An interesting feature of the Queensland study, prompted by the work of Zeithaml and her colleagues (1990) on quality in service industries, was a study of patient views and provider
70
views about consumer expectations which found a persistent disjunction between these assessments. WESTERN AUSTRALIA Western Australia has adopted a policy based on the purchaser provider split. The Health Department has sought to decentralise most aspects of health services. Western Australia has a majority of small hospitals. It has a very significant indigenous population. Patient satisfaction surveys have been conducted at state wide and hospital level. One would anticipate that purchaser authorities would want to develop some consumer feedback mechanism to inform purchasing decisions. In 1990 a state wide customer satisfaction survey was conducted following a pilot in 1989 and yearly there-after. While WA has the longest history of a state wide survey, the survey currently in use has some limitations. It relies on 10 very general questions about satisfaction, in a survey which was designed to work across a variety of services. Hospital specific surveys have undergone a lot of development since this survey was designed (see chapter 11). One unique and useful feature of the survey, however, is the focus on redress if things go wrong. Method The survey is conducted on one day across the state and given to all patients who are within 48 hours of discharge. The questionnaires are filled in hospital and can be placed in sealed envelopes (optional) for collection. The questionnaires are returned to the Health Department for analysis. It was initially conducted for general surgery, general medicine, and midwifery. In 1995, the questionnaire was given to all patients at discharge. Questionnaire design There have been three different questionnaires. The initial questionnaire was developed as a purpose built questionnaire, using focus groups conducted by a consultant to work out satisfiers and dissatisfiers. The questionnaire had about 20 question with some supplementary questions to give some information about the patient. The questionnaire sought information about the following areas: 1. core areas of satisfaction - overall satisfaction 2. satisfaction with higher order needs • staff concern • staff thoughtfulness • information and communication • staff involvement 3. a qualitative assessment - open comments which are then classified
71
After 1992, a shortened version was run. These have now been replaced by a new patient satisfaction survey, administered by the Epidemiology Branch of the State Health Purchasing Authority. It is based on a 1983 US instrument with minor adaptations (Nguyen et al 1983). This is a ten question survey which asks patients to rate satisfaction with various aspects of care. Areas of care which have a particular question are helpfulness of staff, information, redress, and waiting times. There is no space for patients to add their own comments. The information it collects is at a very general level. Response rate The response rate for 1990 was 76%. This is the most clearly reported response rate and information was sought on the characteristics of nonrespondents. The response rate for the 1995 survey was 49% (the range was 39% to 90%, lowest in the metropolitan area). This is low for surveys administered in this way. Results Reports have been sighted for 1990, 1991 and 1995. A 1993 report on a survey in mental health services is also available. The two areas where concerns were identified in 1991 and 1992 were ‘staff thoughtfulness’ and ‘staff involvement. These related to questions about respect for privacy, being treated as an individual, staff empathy with the client’s condition and participation in decision making. Particularly low scores were found amongst the maternity patients, confirming once again the findings of the survey conducted as part of the WA Birthing Review. Results were analysed according to a range of patient characteristics, for example: •
women had lower scores than men on the thoughtfulness construct;
•
20-29 year olds had lower scores on core areas of satisfaction, concern, thoughtfulness and involvement;
•
patients with unplanned admissions scored information lower, and
•
there were regional variations.
The 1995 report, based on a quite different questionnaire, reported at a more general level on satisfaction, reflecting the nature of the questionnaire. Results included: • older patients were generally more satisfied, and conversely, younger patients were the most critical; • women rated their overall satisfaction higher than men;
72
• patients from non-teaching hospitals rated satisfaction higher than those in teaching hospitals, and • there were regional variations. The questions that contributed to lower satisfaction were waiting times, understanding information received, and the way that things that went wrong were put right. Aboriginal quality feedback Fremantle Hospital also developed a specific instrument for use with Aboriginal consumers. This has involved working with Aboriginal people to identify issues and to develop an approach. This instrument is administered in face to face interviews with Aboriginal consumers, with Aboriginal health workers as the interviewers. Written questionnaires and the use of non-Aboriginal interviewers were found to be unacceptable. Within the Aboriginal users of the hospital services, there are also quite marked differences between different aboriginal groups, for example between urban and tribal Aborigines on their areas of concern and cultural practices about speaking. What has been done with the results? Reports on state wide results are published. The results have been accepted as an effectiveness indicator for the portfolio for purposes of reporting to the Auditor-General. Feedback has been sought from hospitals on the usefulness of the survey. However, approximately 80% of hospitals use an in-house satisfaction survey and this has led to a set of issues around the acceptability of a state wide patient satisfaction survey. SOUTH AUSTRALIA South Australia has decided to introduce a state-wide patient satisfaction survey. It is a modified version of the HCA questionnaire. It will be done as a 10% sample of in-patient admissions for the year, administered by post after discharge. The details of the implementation of this decision are still being worked out. It appears at this stage that the analysis will be of state wide data. Policy context Like Victoria, South Australia has a multitude of new policy initiatives aimed at restructuring the health system. South Australia introduced casemix funding in a similar model to Victoria in July, 1994. The current South Australian government has a policy of privatising the management of public hospitals and has taken steps to implement this with the privatisation of Modbury hospital.
73
NORTHERN TERRITORY Northern Territory does not currently have a state wide survey, but is discussing the possibility. There are five public hospitals in the Northern Territory. The Royal Darwin hospital is the only one at present to have a hospital wide survey. This is done on a random sample of in-patients every three months as part of its quality program. It is administered while people are in hospital. The survey is based on the hospital charter of patient expectations and utilises some of the accreditation standards. Its results are used to identify areas of hospital organisation that need to be improved, eg admission procedures, involving patients in the management of their care. It is not clear how this would relate to any state wide survey. There is also a project about getting feedback from Aboriginal hospital users. TASMANIA There is no state wide patient satisfaction survey. Quality is handled at a regional level. There is some use at a local level of patient surveys, for example the North West region uses surveys in particular discipline areas in order to focus on particular areas of care. These include telephone interviews and face to face interviews. These are usually related to the work of Quality Improvement Teams. For consumer feedback at a broader level, Tasmania has Health Care Forums at a sub regional level. These are filled by local advertisement and are of varying effectiveness. SUMMARY COMMENTS ON STATE PATIENT SATISFACTION SURVEYS Most states, but not all, have or are developing some form of statewide patient satisfaction survey. There are considerable differences between them in terms of the following: • • • • • • • •
the questionnaires themselves; their length and scope; the type of issues which they focus on; the basis of questionnaire design; how they are administered; whether they are administered in hospital or after discharge; the types and scales of ratings used, and the sampling. There is no consistency of sampling.
These all affect the nature of the information they make available. Valid comparisons are not possible between them as they stand. In particular, some are administered in hospital (WA and NT) and some post discharge (VIC, NSW, SA). Satisfaction has been shown to change over time and there are a set of issues about how comfortable consumers feel in commenting adversely on their care while in hospital. There is also literature to suggest that using clinical staff to hand out the surveys is a potential source of bias in the sample. There are differences in sampling and response rates.
74
The recent state wide surveys, particularly those in NSW, Victoria and Queensland, have brought a greater professionalism to patient satisfaction surveys. Reports on the HCA surveys in NSW and Queensland are available. The current pilot in Victoria should provide better information about the value the of the Picker-Commonwealth survey than is currently available (see chapter 11 also). Interestingly, Royal North Shore has combined elements of both surveys for a research project on the link between satisfaction and the technical quality of care. There are also differences in the purposes and objectives of the patient satisfaction survey, which affects the choice of instruments. If hospitals and Health Departments are to get the kind of information from patient satisfaction surveys which enables quality to be improved, then a different detail of information is required than if some overall sense of ‘satisfaction’ is required for monitoring, comparison, or accountability purposes. Again, if the purpose is to build up a research base on the aspects of health care organisation that matter to consumers, then the rigour of all aspects of patient satisfaction surveys, including sampling and design are paramount, and more than overall measures of satisfaction are needed. There is the potential for state wide surveys to conflict with the surveys which hospitals wish to conduct themselves, particularly in the smaller states and for confusion to arise about the various surveys. Some of the UK experience suggest ‘patient satisfaction fatigue’ can develop, as evidenced by falling response rates. There are issues about ownership of the surveys by hospitals which affects their willingness to work with the results. There are similar issues affecting the confidence consumer organisations have in the state wide surveys. Only Victoria has involved consumer organisations in the process around the implementation of the patient satisfaction surveys. There are important questions about the organisation which undertakes a statewide survey. There is a great deal of literature which suggests that a neutral auspice is an advantage in assuring a sense of confidentiality and confidence in the analysis. While hospitals have an obvious stake, governments too may have a stake in defending their policies. At the same time, the use of private consultancy organisations, while they bring both a potentially independent auspice as well as welcome expertise, has some disadvantages as far as disseminating knowledge about the design and use of surveys given the commercial value of the knowledge, skills and expertise. This is particularly so as there are currently no other real centres of developed expertise and knowledge accumulation other than the expertise developed by the Centre for the Study of Mothers’ and Childrens’ Health at La Trobe University. We argue elsewhere in the report that publication of discussions about the results and methodology of patient satisfaction surveys is important to building Australian expertise. In the research on this chapter and others, it was clear that there was a need for greater exchange and dissemination of knowledge and experience about patient satisfaction surveys. There is a deal of reinventing the wheel, but more importantly people not feeling that they had a body of knowledge to draw on and the choice of instrument may depend more on the article one has access to, or the generous public domain distribution of some surveys by the Medical Outcomes Trust in the US, than an appraisal of the options available. There is no home grown Australian hospital satisfaction survey of other than a hospital specific kind.
75
No state-wide survey has attempted to work with the condition of the patient, although some other characteristic have been used, eg level of pain. There is some evidence from UK and here that condition matters, eg patients with diabetes were more happy with their care in the APSF research (see chapter 2) than patients with other conditions, and the British Royal College of Surgeons work (see chapters 9 and 10) showed that the patients condition, such as a potentially fatal cancer, mattered in the way they judged communication between surgeons and patients. Following the analysis of Gerteis and her colleagues (1993) using the PickerCommonwealth survey (see chapter 11), it is also apparent that the casemix of hospitals may affect their scores, as indicated in the different profiles of teaching and non-teaching hospitals. This may also be an interpretation of the 1995 WA results which show a similar pattern. A patient satisfaction score of about 85% seems to be a fairly usual level of satisfaction and this is pretty standard across the patient satisfaction literature. The important issues from a quality perspective lie in the dissatisfaction results, so that it is important that patient satisfaction surveys provide some guide to where the important issues lie and for whom. There is also debate in the literature and among practitioners about the value of such aggregate scores, given the real information is in the detail and that quite small differences at the higher end contain very important information (Ware and Davies 1983:295). There is also the challenge of getting beyond the gratitude response. It is clear in some of the analysis that expressions of satisfaction conceal reservations that people have about particular aspects of their care.. Response rates vary. The NSW and Queensland surveys have response rates of 38% and 27% (TQA argues that 25% is adequate for reporting on aggregate scores). Victoria, albeit a telephone survey, had a response rate in the mid sixties. By comparison, the Victorian Birthing Review, a postal survey, had a response rate of 71.4%. We argue that it is important to have some information about nonrespondents and to have information about particular categories of people, so that results can be analysed in terms of different groups of patients. Ware and his colleagues suggest on the basis of research that both telephone and postal surveys have a persistent nonresponse bias in terms of education, income and employment and that telephone surveys were less vulnerable than postal surveys to nonresponse bias on the basis of age (McHorney, Kosinski and Ware 1994). These are important issues given the poorer health experienced and greater use of health services by these groups. Some of the states’ response rates mean the results on some groups, eg people from non-English speaking background and Aboriginal and Torres Straight Islanders are not reliable. Nor would the samples sustain an analysis by DRG or some other way of getting at condition. Response rates warrant further discussion. Ware and Berwick (1990:S40) suggest 60-70% plus is possible for the HCA questionnaire with proper follow up. Carr-Hill (1992:245) argues that 50% and lower is not acceptable and that better response rates should be attempted. Response rates of good surveys discussed throughout this report tend to be in the 70-80% range and higher. While there are issues about the nature of sampling that affect the capacity to generalise from the sample, epidemiologists work on the basis of 60-70%. Interestingly. across our literature survey, it is the longer and more complex questionnaires that appear to have the better response rates - the ones often feared to be burdensome to
76
patients. At the same time, response rates are affected by many factors, including auspice, and how well the surveys are presented to people in the first place. There are other issues about particular groups. The Victorian work suggests that there are unresolved issues about consumers from non English speaking background. It is not clear that either self-completed or telephone surveys are satisfactory, even with translating of surveys and interpreting of interviews. This is being further explored in their current pilot. The Western Australian work suggests that written surveys are inappropriate for Aboriginal and Torres Straight Islander people and that personal interviews by aboriginal health workers are preferred. Only Western Australia has reported analysis in terms of gender. The surveys consistently tend to show older consumers indicate more satisfaction with hospital care, yet it is this group which the Quality in Australian Health Care Study has found were more likely to suffer from an adverse event in hospital and specific focus group research with groups of older people suggest there are some particular problems. The work of Ware and others suggests that this group needs attention in terms of response rates. On the other hand. in these surveys, as in the consumer research surveyed in the previous chapters, younger people are less satisfied with hospital care, suggesting that there will be a more critical body of consumer opinion in the future. This is also a group that now regards universal access through Medicare as something they take for granted, whereas many older consumers have experienced a change in access and costs. None of the state-wide surveys have any information about what rights patients have and what standards patients ought to be able to expect. The only Australian example we found of any such inclusion was the Darwin hospital survey which has a statement of consumer expectations. There are resource considerations in conducting patient satisfaction surveys at all levels, some of which are absorbed in house. The more complex the questionnaire, the more resources are necessary for administration and analysis. Good surveys cost money. Telephone surveys clearly carry more costs. More work needs to be done on the cost benefit trade-off. There are issues about ethics committees which warrant further discussion. State wide surveys are likely to continue to require Ethics Committee approval at individual hospital level. The line between quality assurance and research is becoming less clear as better research methods, such as well designed patient satisfaction surveys based on a population sample, are used in quality assurance. These are issues here for what ethics committeess traditionally consider to be their role. There are some concerns about the impact on sampling of some of the ‘opt-in’ processes required by some ethics committees. Finally, perusal of the results of these surveys shows very clearly that what these surveys tell you is essentially linked to the assumptions built into the survey design. Patient satisfaction surveys tell you essentially what you ask. As the British National Consumer Council also argues, what these surveys tell you is what consumers think of what they are getting, not what they want (see chapter 10). All the state surveys fold ‘very satisfied’ and ‘satisfied’ or ‘very satisfied’ and ‘fairly satisfied’ into overall satisfaction ratings. The Victorian Birthing review, on the other hand, regarded anything other than the top rating as expressing some degree of reservation. In other words, there are decisions that are made about how the categories of
77
satisfaction are treated that affect the reporting of satisfaction. None of the state surveys discussed here report any analysis of the open ended questions. Postscript Since this report was completed, Victoria has reported aggregate results of their patient satisfaction survey. Overall patient satisfaction is reported as 97%, with 73% of respondents very satisfied. As in the NSW survey, the overall satisfaction score combines ‘very satisfied’ and ‘fairly satisfied’. Overall, the results profile is similar to NSW, with a higher ‘very satisfied’ score in Victoria, which should be interpreted with caution for all the reasons stated earlier. The different questionnaires tend to highlight some different areas and this is reflected in the reporting. There is no reporting at this point of what the range of scores were for different hospitals, although there is reporting by hospital category with higher scores for smaller regional and rural hospitals and lower scores for teaching hospitals (see chapter 11 for some discussion of these issues). NSW has decided not to continue with a statewide patient satisfaction survey.
78
CHAPTER 8: SUMMARY OF PART ONE This chapter summarises and restates conclusions from part one of the report, focusing on the themes which emerge about consumer experiences and stating issues relevant to methodology which emerge from the research. Consumer research Consumer organisations have tended to direct their research to their members and consequently it predominantly focuses on population groups, for example, older people, people from a non English speaking background. Many of these groups have ‘members’ who are disadvantaged socially or financially in some way. This focus has enabled some organisations to build up a body of work, for example, on older people. Consumer organisations have also looked at the issues of the process of care, ie getting in and out, or the linkages between services. They are less likely to see hospital care as an isolated event and that particular aspects of the episode do not exist in isolation from other aspects. Research by consumer organisations provides a valuable and often cost effective way of hearing the direct experience of consumers. The types of research data presented here illustrate the breadth of problems people experience in their dealings with hospitals, and to some extent, the depth. Many of the issues raised provide the basis of what issues of the hospital experience should be the subject of feedback mechanisms, as well as the topics of further research, for amplification and verification. There are common themes which emerge from the consumer research. First, there are several related issues which cluster around the general mantle of: • • • •
communication; being treated with respect; not being the subject of discriminatory treatment; and being involved in decision-making.
Sub-issues related to these are: • • •
informed consent; privacy; and access to interpreters.
What binds these issues are assumptions about the position the consumer occupies in their interactions with the health system. A consumer who is not considered an equal partner in the process, deserving of respect, courtesy and involvement in their care, is one who may experience a less than satisfactory time in hospital.
79
People want more information on all aspects of the hospital stay, and in particular on the pharmaceuticals they are prescribed. They want information in verbal and written forms, and in languages other than English, where required. Discharge planning is frequently inadequate for all types of patient. There is little involvement of family or carers either. The role of the professional team is critical. People are more positive about: •
multidisciplinary teams;
•
good teamwork; and
•
communication between professionals. This includes communication between professionals in and outside the hospital.
A related and important sub-issue for public patients is seeing the same principal doctor, within a team, ie continuity of care. For some people, timely access to hospital care is an important issue. Several factors are mentioned: •
the cost of, or the lack of public transport. This is an issue for city and country dwellers but the specifics may vary;
•
some people are not adequately informed about the length of wait; and
•
some people cannot get into hospital for some reason.
State patient satisfaction surveys Table 8.1 summarises results where these are available using the categories derived from the consumer research. The results of the patient satisfaction surveys confirm the themes emerging from the consumer research, although the different forms of research throw up different detail, and there are some differences which emerge from the hospital focus of the patient satisfaction surveys in contrast with the more system oriented approach which emerges in consumer research. The themes of being treated with dignity and respect as a person and communication and information stand out across all the research. What this means for particular populations is explored in the subsequent sections. Being treated with dignity and good or bad communication between parties is not purely a matter concerning individual health providers. It is facilitated by structures and policies where the goal of enhancing a focus on personal care and communication is assured.
80
Overall satisfaction rates are high, in the usual range for patient satisfaction surveys. This research found that people did have a hierarchy of concerns. Being treated with dignity as a person and communication, both interpersonal aspects of the quality of care and treatment, were more important than the physical environment.
A key issue raised was being treated as an individual and a person. The technical skills of doctors were taken for granted. Quality of care and treatment called upon further qualities of attitude and communication skills in hospital staff.
The capacity to access inpatient health services is not tested by this approach. However, aspects of access which are highlighted are waiting times in Accident and Emergency which emerged as an important issue.
Being asked about care and support at home was an area for improvement, as was notice of discharge in mental health services.
Communication was identified as a key issue,
Overall comment
Role of consumer/ hospital culture
Access
Discharge
Communication
NSW
81
This survey identifies communication issues as
Not relevant in this survey.
This is the key area addressed by this survey. Waiting times are a major concern in this survey. The survey also highlights the important place A & E has for some groups in their access to health care
This is not explicitly addressed in either
This is not addressed in these questionnaires and the manner of IP administration does not lend itself to this, but some comments did emerge in the open question section of the older survey.
Not addressed in either survey
The new questionnaire does not address these issues. This is an area the earlier questionnaire particularly targeted, with privacy and dignity, being treated as an individual emerging as key issues where patients rated at lower scores.
Levels of satisfaction are generally high, across both surveys, and this measure in the earlier survey includes views on technical competence.
Overall levels of satisfaction are high, but it is clear from the report that this varies from hospital to hospital.
This survey again points to the importance of issues such as being treated as a person and this being reflected in the way services are organised.
WA
QLD A&E only
Table 8.1 State patient satisfaction information according to consumer categories
The 1993/94 survey finds no significant difference in satisfaction amongst people from people from non-English speaking background, people with disabilities, and Aboriginal /Torres Strait Islanders, however the sample for the latter was not sufficient to draw conclusions for this group and some caution needs to be exercised on all of these subgroups.
Environment tended to count when it was very good or very bad but not in between. Patients were more concerned about communication and individual care than they were about physical environment. Aspects of environment identified included cleanliness, a restful environment, the heat of meals in large hospitals, and parking in urban hospitals.
Discrimination
Environment
particularly for doctors, with a range of scores across different hospitals.
82
This is not a feature of the new questionnaire. It was addressed in part 3 of the earlier questionnaire. Hospital food got a mixed press in the older questionnaire. Comments are also made, in the older questionnaire about funding cuts and their impact on staff and their availability to patients.
Both questionnaires looked at different responses from men and women. Specific approaches have been developed for aboriginal patients.
It is possible to infer from this survey that some groups use A & E despite its frustrations, because it caters for their needs better than other parts of the health system. Because of response rates, this survey is not reliable for people from non-English speaking background and A&TSI consumers. One notes however, the poor ratings for Thursday Island. Outside parking problems, environment issues did not receive much attention and were less important than the way A&E is organisaed and consequential waiting times.
questionnaire, However both address information which emerged as an area that needs to be improved for some groups in particular, eg people with unplanned admissions and the 1995 survey suugests some general problems with understanding information.
important
There are some omissions across these surveys and across the consumer research as well, in particular focus on outcomes of hospital care. Several questions in this area were omitted from the 1995 Victorian pilot. There is a rather artificial line being drawn between the focus on consumer satisfaction and the focus on outcomes. Only the WA patient survey asks a question about redress, what was done when something goes wrong. Government reviews/ birthing surveys There was a considerable degree of dissatisfaction reported in both states with care during labour and birth. Important general issues were access to information, the nature of the relationships with care givers, and involvement in decision making. Both surveys showed the link between public patient status and lesser quality experiences as reported by women. They experienced longer waits, unpleasant labour and delivery, and less privacy. These surveys underline the importance of getting information about important subgroups and having an adequate sample to do this. The surveys show that younger women, women who were from non English speaking backgrounds, Aboriginal women and women on low incomes were more likely to report poorer experiences. For Aboriginal women, there were specific problems such as access to antenatal care, the desire for a women doctor, feelings of isolation and fear, and education on the benefits of certain services. The importance of continuity of care by one doctor for public patients emerges from these birthing studies, and builds on the findings of several projects reported in chapter 2. This raises the issue of how to attribute responsibility for ‘satisfaction’ with care, if a person attending hospital publicly has several doctors. Dissatisfaction with discontinuous physician care suggests that responses by the hospital will require some re-organisation of care. The finding that the frequent comments made to the consultation phase of the Victorian Birthing Review were substantiated strongly by the survey data, despite the misgivings of doctors, indicates that methods other than a representative survey still produce valid data. While not giving the researchers the confidence of its representative nature, the information that women wanted an active say in birth influenced the subject matter of the survey. Hospital projects aimed at particular populations Hospitals have undertaken projects aimed at getting views from particular populations, using a range of methods and a range of sources of information, including staff. The problems of people who do speak English are echoed by the experiences of people from a non English speaking background who do not. However, problems for the latter may be exacerbated by the absence of interpreters, or the use of interpreters not familiar with the specifics of that hospital’s environment. This is an aspect of being treated with dignity as a person. There are challenges is accessing the view of people from non English speaking backgrounds. People from non English speaking backgrounds will not speak out in public meetings, even in a sympathetic environment, because of fear of retribution, or unwillingness to harm their relationship with the service, or for other culturally related reasons. Given this, there may be
83
issues of importance of which staff are unaware. The state wide patient satisfaction surveys are also unsatisfactory for this group. There may be important subgroups of users in the broader population groups such as adolescents who have special needs but are hard to reach for getting feedback and who are usually omitted from state patient satisfaction surveys. For example, young people in crisis, or homeless, or in precarious circumstances, may be hard to retain in hospital situations. They may not attend follow up, may not wait long in casualty, may not have an address or phone, or may not be literate or interested enough to fill in a survey. Particular hospitals, for example, Fremantle and Darwin, have undertaken projects aimed at getting the views of aboriginal patients. The importance of aboriginal health workers is underscored in the Fremantle project. Academic research The major sustained academic research effort is into hospital experiences of consumers is that undertaken about women’s birthing experiences by the Centre for the Study of Mothers’ and Children’s’ Health, reported under birthing surveys and initiated as part of a government review. As noted elsewhere, this is the only Australian survey to generate an extensive publication record on both the results and the methodology. The survey has earned its own credibility. Some hospitals are using the 1995 survey for their own management review. With this exception, academic research in Australia consists usually of one-off research exercises and provide important but limited information. Issues affecting consumer feedback mechanisms The research throws up other issues to consider if consumer feedback mechanisms are to be developed. They are: •
Consumers can be very happy to comment on their experiences of hospital. For example, in the Birthing Review, women were keen to complete the questionnaire, write lengthy comments in addition to the questions, and give a phone number for follow up research.
•
It has been shown that respondents are less prepared to be critical if they know caregivers will see the questionnaires. The independent auspice of surveys is important and anonymity is very important, particularly for regular users of hospital services, carers, and people in city fringe, rural or remote areas, where there may be no choice among hospitals, or staff are known.
•
Some people fear that criticism of a service will result in its loss. This may be more of an issue for a service that targets one population group, ie a specialist service, or in remote areas where it is difficult to attract and retain specialised staff.
84
•
Some sources of dissatisfaction may relate to structural factors related to resource levels, may be difficult for staff to respond to, and require management decisions, ie increased use of agency staff, inadequate numbers of interpreters.
•
Consumers who are disadvantaged in some way are keenly aware of the less than satisfactory treatment they believe they have received. Some people, for example, older people, may not want to complain, but when interviewed at length, may point to incidents of care that have inconvenienced, discomfited or adversely affected them.
•
The association between dissatisfaction and primary caregiver and insurance status found in the Victorian Birthing survey study raises a key issue of how to address causes of dissatisfaction. Women with restricted choices of which type of antenatal care to use (young, without a partner, on a low income, or of non English speaking background) were more dissatisfied, and the authors note that it is these women who may be in need of more sensitive care.
•
Representative samples, while they may reveal very similar results to other forms of obtaining consumers views, can gave a firmer basis for policy and counter stereotypes of consumers which are used to discredit critical feedback, eg only middle class consumers are dissatisfied. Well designed surveys can also provide more specific information. Findings that women were more critical of care during labor and birth, rather than antenatally led the Birthing Review to tailor its recommendations to achieve more change in this area.
•
The greatest increase in odds of dissatisfaction in the Birthing Review were associated with not having an active say in labor. This result brings in to sharp contrast possible tensions between specialists and consumers. When the survey was being developed, some obstetricians wanted the question regarding an active say removed: some said women would not be able to answer it; others claimed the answers would be biased. Whatever "active say" meant to women, it was interpreted by most as crucial. Furthermore, not having an active say was associated with less emotional well-being after the birth. Likewise, the Victorian patient satisfaction survey went through a committee process where politics were as much at play as science.
Promoting new models of care The philosophy adopted by a specialty area may have a significant impact on the way care is structured and the approach taken towards the consumer. Some specialties treat the person as an equal, promote self management and teamwork, involve other professions, and/or develop a model of good practice. These treatment models are clearly recognised and appreciated by consumers who wish them promoted more widely. Much of the work of consumer organisations is directed to the policy process of state or Commonwealth governments, but less commonly to specific hospitals. There is little published material on whether the report had an impact on hospital practice or government policy development. Anecdotal reports suggest that consumer groups face considerable barriers in convincing hospitals of the value of their work, although this is not a uniform problem. Some
85
hospital staff may be sympathetic to the issues, but changing hospital practice requires that the research is built into a joint policy development and research process. Similar problems can be found with the statewide surveys, where work needs to be done to establish what changes as a result of patient satisfaction surveys and other forms of consumer feedback. From the perspective of quality improvement, the findings of such surveys need to be linked back to the organisational characteristics of hospitals that provide care that treats people as individuals with dignity and enhances communication not only between consumers and providers but also between health care providers. The high level of interest in the Victorian Birthing survey, and the desire of many to be involved in further research, suggests a high level of interest in the community to be active participants in the process of researching and improving health care. This has implications for the type of research, as traditionally, consumers have only been allowed to play a passive reporting role. There are other forms of data that need to be part of information available to governments and hospitals that form a composite picture of care. These include complaints data and adverse events. the Quality of Australian Health Care Study (formerly the Australian Hospital Care Study) convincingly demonstrates that the safety of hospital care needs to be improved and will indicate some of the ways to do this. The methodological approach in this study supports the broader principle which the next chapter will address - that quantitative research can tell you ‘how many’, but needs to be complemented by qualitative research to tell you ‘why’ and what needs to be changed.
86
PART TWO
METHODOLOGICAL AND ORGANISATIONAL ISSUES
87
88
CHAPTER 9: REVIEW OF THE ISSUES IN SURVEYING PATIENT SATISFACTION This chapter is a review of the main issues involved in undertaking research to obtain consumer feedback about hospital services. To date, most researchers have sought feedback from adult patients, or adults speaking on behalf of children and young people. Data has been obtained by using what are usually termed satisfaction questionnaires. Commonly, these are precoded questionnaires allowing patients to respond to questions in a fixed format. Variations on this format allow respondents to elaborate more, or report what they choose. The evaluative feedback being sought is usually couched in terms of the phenomenon of satisfaction, although more recent questionnaires avoid this word. Satisfaction questionnaires are themselves a phenomenon of the last 25 years, and their rise in popularity parallels the social and economic changes and preoccupations of these years, for example, the emerging recognition of the role patients play in the health care process, and the increasing focus on evaluation of health care programs. Throughout the period, but more particularly in recent years, various researchers have seriously questioned the theoretical underpinnings of the measurement of satisfaction. The main criticisms have been that research has assumed it is a phenomenon independent of the social context of people’s experiences. Further, the measurement of satisfaction has ignored the structural limits of the resource context, and the interpersonal dynamics and fluidity of the health care process. However, the belief that satisfaction can be measured has its roots in positivist sociological practice, and possibly for this reason, has remained a major preoccupation of researchers in the United States for much of the period. In the present decade, though, researchers in Britain and the United States, where much of the literature emerges, have moved away from the measurement of satisfaction, to an interest in recording the experiences of patients (and sometimes other users). This chapter presents some of the main debates and issues in the field from the academic literature. It should be read in conjunction with the discussion of recent research in Britain and the United States, in chapters 10 and 11. The material was sought in the following ways. A hand search was conducted for articles from leading academic journals which covered the social sciences, quality care and evaluation, published in Britain and the United States. A survey of Australian patient satisfaction research is contained elsewhere in chapter 6. However, a small number of Australian articles have been drawn on for their discussion of methodological issues. The material derived from the search was then complemented by a number of important articles by key practitioners in the field. We do not pretend, therefore, to have surveyed all the literature on patient satisfaction for the last 25 years, as this is a mammoth task. There may be significant omissions from this review. However, this chapter does not aim to present a checklist of all the ideological and methodological aspects of patient satisfaction surveys, but to highlight some of the major debates and questions in the field today. There is not a consensus of the best way to proceed with patient satisfaction research, and the precise role patients and their families or carers play in the quality care process is still unclear.
89
This chapter is organised in the following sections: • • • • • •
the history and context of measuring satisfaction; how is patient satisfaction defined?; how is the data used?; the scope of satisfaction surveys; issues related to the method chosen; and policy issues.
The history and context of measuring satisfaction Patient satisfaction surveys have developed from several streams of thought and practice, hence their supporters may have different assumptions about the value and purpose of the information collected. Early work revolved around the link between satisfaction and compliance (Williams 1994:510). This helped legitimise the patient’s perspective and led to the (optional) inclusion of their views into quality review processes. In the 1960s and 1970s, the consumer movement argued that the patient as consumer of services had a social right as members of a community and as individuals to be heard, not just in order to improve compliance. Furthermore, some were critical of the assumption that all treatments were appropriate and the consumer must automatically obey the doctor and comply with the treatment regime. In the 1980s the consumerist stream was joined by a substantial increase in evaluative activities, as governments searched for more efficient and effective ways to provide services within a fixed level of resources (Williams 1994:510). Research on health outcomes was promoted within a policy context of the economic appraisal of the costs and benefits of particular treatments. Some have sought to determine the relationships between satisfaction and better health outcomes. Others have seen satisfaction as an outcome in itself. Recent literature (see Donabedian 1992:250), introduces the concept of consumers and providers as co-producers of care, illustrating the assumption of equally sharing the power (or control) of care, and indicating that both could have views essential to ensuring quality care is maintained. This has parallels in the consumer literature in models of partnership, with their assumption of the co-responsibility for treatment. The promotion of the concept of the health sector working more like a market, and regularly seeking the views of users to improve services, has received a substantial boost from the British reforms under the Conservative Thatcher and Major governments, since the early 1980s. Consumer organisations in Britain, responding to reforms promoting the notion of a more sovereign consumer in the health market, undertake consumer audit or consumer appraisal exercises on a regular basis (Steele 1992; Rigge 1994). Health services have taken the next step, and the inclusion of satisfaction data into other outcome data is now being used in Britain for decisions about resource allocation (Meredith, Emberton & Devlin 1993:72). Carr-Hill (1992:236-7) writes that, in the context of British reforms to promote consumer sovereignty, satisfaction would be an outcome of the health care process, if patients’ demands and preferences shape the provision of care. To be useful, however, the measurement of 90
satisfaction must be sensitive to changes in quality. This, he argues, rules out the majority of inpatient satisfaction questionnaires, which tend to report that the majority are satisfied. Of more use for hospital managers are feedback mechanisms which explore why the minority are dissatisfied. Government sponsorship or involvement in surveys raises a political rationale for conducting surveys, for example, to demonstrate the population’s satisfaction with care. In Britain, where the role of the public sector provision of health care shapes the debate about satisfaction, writers have acknowledged the multiple objectives of government and the complexity this engenders. McIver (1992:17) argues that a focus on the aggregated views of individuals’ satisfaction with care masks the balancing role of public services, which have to meet the needs of a community. This is not the same as satisfying the wishes of individuals. Furthermore, the promotion of marketing tools to gain feedback ignores the reality that for many using public services, choices are limited. They cannot move to the private sector if care does not improve, or necessarily to other public services. Hopkins, Gabbay and Neuberger (1994:204) have noted that groups in the population are rarely involved in making judgements about the trade-offs inherent in questions about, for instance, satisfaction with access, such as comparing the amenity of local, but less specialised services, with access to specialised but centrally located services. Patient satisfaction research from the United States from the 1970s and early 1980s is characterised by its positivist epistemological roots. As Baum (1995:461) notes, this is based on the idea that “phenomena can be reduced to their constituent parts, measured and then causal relationships deduced”. In the past ten years, qualitative methods of investigating complex phenomena have become more accepted, with this work drawing on interactionist or constructivist epistemologies (Baum 1995:460-1). These approaches stress the importance of context and the need for a holistic understanding of events. Aharony and Strasser (1993:50,63), in a review of patient satisfaction research, make the following points regarding American work in this area: •
most work has been empirical rather than theory testing or theory building;
•
the body of literature is large, diffuse and unfocussed;
•
methodological dilemmas continue, perhaps due to the absence of an accepted theoretical model of satisfaction - although the authors do not link these points;
•
no standardised approaches to surveying, and little clarity and consistency in understanding the determinants of satisfaction;
•
few comparative studies of outpatient and inpatient care; and
•
no consensus within the medical profession about the role surveys should play in quality assurance.
In addition, most work has been about mechanical relations, rather than examining the interactional or contextual circumstances (Aharony & Strasser 1993:66-7). For instance, the
91
models test how certain provider variables cause levels of satisfaction. There is no thought given to the role the patient may play in the process of care. Extraordinarily, Aharony & Strasser report (1993:67) that there are no studies of a systematic analysis of the comments written at the end of surveys, despite the fact that open-ended questions produce more negative ratings and comments than closed questions (1993:57). Aharony and Strasser make no mention of any American government policy framework for consumer input to quality but they write that little is known of the effect of satisfaction studies on the organisation of care (1993:51). Chapter 11 discusses patient satisfaction in the United States and moves to improve the quality of information given to people on quality care, and the development of recent surveying instruments. How is patient satisfaction defined? Williams (1994) is highly critical of patient satisfaction research which is built on the assumption that satisfaction is an “independent phenomenon”. He writes that surveys force people to “actively construct it by forcing (them) to express themselves in alien terms”. The consequences of this are inferences made from results which may misrepresent users’ beliefs (1994:514-5). He argues that people using services “have a complex set of important and relevant beliefs which cannot be embodied in simple expressions of satisfaction” (p.515). It is not known how people evaluate services, whether they think in terms of satisfaction, or whether they are passive in the face of organised care. The “reductionism necessitated by quantitative methodology” may have removed the collected data of any meaning and collapsed people’s complex responses to a rating of satisfaction. This is suggested, he writes, by the number of qualitative studies showing the levels of disquiet about their hospital experiences not picked up by conventional surveys. In a related branch of research, looking at self-reported health status, some have made similar arguments about the reductionism implicit in ‘tick-the-box’ measures. Jylha (1994), in an examination of self rated health status, argues that the nature of ‘health’ is relative and contextual, and can include contradictory elements. This opposes the view that health is an abstract variable that is presented in interview or questionnaire. Many studies, however, had seen it as a “unidimensional, decontextualised variable” (1994:989). Interviews exploring how people rated their health demonstrated the difficulty of subsuming people’s views of their health under simple categories of ‘excellent’, ‘good’, ‘fair’ or ‘poor’. Earlier work in building a theoretical base to patient satisfaction research attempted to model the correlation between expectations and values and people’s satisfaction with care. Williams (1994), in a review of this work, noted that while there was some correlation found, “very little satisfaction has been explained in terms of expectations and values” (1994:512). They may be related but not in a simple sense. He speculates that expectations may be modified during the process of care, which would influence subsequent evaluations. Researchers for the Royal College of Surgeons of England concur. Instead of seeing satisfaction as a fixed artefact, waiting to be measured, Meredith and Wood (1995a: 68) have assumed it is emergent and fluid. For example, a patient’s assessment of a surgeon’s communicative abilities may change if, in the course of treatment, the diagnosis changes dramatically. Given this
92
assumption, in their surveys of surgery patients for the College, they administer their survey twice over a period of eight weeks. In research into patients’s evaluations of neurological outpatient services, Fitzpatrick and Hopkins (1983) noted that people made forceful evaluative comments regarding the outcome of the outpatient visit, yet were only tentatively able to express what their expectations had been and often these formed in the interview with the researcher. They conclude (1983: 308) that expectations “seemed an unlikely basis from which varying levels of satisfaction might emerge”. Furthermore, the focus on measuring expectations confuses expectations drawn from cultural and social circumstances with what people would like to happen in the context of a specific encounter. For example, Meredith and Wood (1995c:4) found that whilst younger people are more demanding of detailed information on the hospital experience than older people, a high percentage of older people wish to be fully involved. In the United States, Greene and colleagues (1994) warn of adopting stereotypes of population groups, adding that in not so many years, the category of ‘older’ people will include people who now are more demanding. Australian material is salutary here, as it illustrates that the literature on expectations may also confuse the social origins of the beliefs. In other words, why is it that researchers assume that older people as a matter of course expect less. Brown and Lumley (1993:102), reporting on the results of the Consumer Survey for the Victorian Birthing Review, remark that anecdotal reports from doctors were that middle class women had higher expectations of the antenatal and birthing process. They were therefore more likely to be critical because these could not be met. In fact, in the Consumer Survey, women disadvantaged in some way reported higher levels of dissatisfaction about antenatal care. Carr-Hill (1992) adopts a similarly critical stance of the conceptualisation of satisfaction, particularly of the notion of satisfaction being a unitary concept. He makes the following points: •
it is a “complex concept that is related to a number of factors including life style, past experiences, future expectations and the values of both individuals and society” (p.237);
•
it is predominantly a derived concept. People not needing care may have difficulty discussing a rating;
•
because it is derived, the search for sources of dissatisfaction is the most important aspect. This in turn may reflect on the differences in power between doctors and patients. Put another way, people may be dissatisfied with doctors who communicate poorly but who are nevertheless technically proficient;
•
satisfaction is “likely to be defined differently by different people, and by the same person at different times” (p.237) because the sources of dissatisfaction vary so widely;
•
much of the conceptualisation of satisfaction places consumers as passive recipients of care;
93
•
because, in his view, there is no underlying unity to ‘satisfaction’ it is not correct to combine satisfaction scores with various dimensions of care into one index of satisfaction. Furthermore, high levels of dissatisfaction with one element of care may mask satisfaction with other elements.
The subsuming of particular sources of dissatisfaction was initially pointed out in a seminar article by Locker and Dunt (1978) and has been confirmed in Australia by the results of the Consumer Survey of birthing services in Victoria (Brown & Lumley 1993). The researchers conclude (1993:101): Our study confirms this view. While only a minority of women rated their antenatal care as ‘mixed’, or ‘poor’, half did not feel that caregivers had been considerate of their needs or that the information given to them before the birth had been a good aspect of their care.
How is the data used? There may be three related but distinct purposes of patient satisfaction surveys: as non-technical evaluations of quality care, as outcomes variables, and as indicators of what should be changed to improve patient response (Meredith & Wood 1995c, following Locker & Dunt 1978). If the third objective is important, Meredith and Wood argue, then the survey needs to be designed in a way that ascertains the parts played by all − the main care provider, the service setting and the patient. So far, the role of the patient had been neglected in research, as the focus had been on measuring degrees of satisfaction and linking these to variables, such as provider behaviour. Scott and Smith (1994) see patient survey data used in two main ways: for quality assurance mechanisms for the identification of problem areas; and as an outcome measure when allocating resources. They note that little has been written about the way the data from surveys is interpreted and used, although the absence of a clear conceptual base is considered a barrier to this process (1993:354). This assumes two things. First, it assumes that surveys which have primarily focussed on measuring satisfaction, and linking this to variable causal agents of dissatisfaction, give service providers and administrators information they can act on to improve services. This assumption is problematic. Second, it is not clear from the literature that the primary focus of surveys has been to improve care in a way that involves patients’ viewpoints. To date, much attention has been on the uncovering of sources of satisfaction. In the pursuit of quality, patient satisfaction should be maximised and the purpose of surveys is to uncover the way to do this (Scott & Smith 1993:354-5). Whilst the maximisation of satisfaction is not necessarily the same as isolating the reasons for dissatisfaction to address problem areas, the authors make the salient point that whilst one aspect of hospital care may rate the greatest amount of dissatisfaction, it is not necessarily the aspect consumers would wish to change first, assuming they had an opportunity to prioritise. In addition, the resource costs of improving that aspect to the degree that satisfaction is maximised may be too great for the benefit. Taylor and Clarke (1993), writing about their survey of inpatients of a Victorian psychiatric hospitals, make a similar and related point − that while elements of care may be rated very poorly, patients did not consider these aspects to be critical in improving outcomes.
94
These points therefore cast a reflection on the role of consumers in the survey process. To what extent are they the passive recipients of a questionnaire which has been couched in terms of the researcher’s preoccupations? How can they play an active role in terms of deciding what services should be reviewed, having input to the survey design, and then reviewing the priorities for change which may emerge from the survey data? The scope of satisfaction surveys Aharony and Strasser (1993:58), in their review of research questions which remain to be answered, report that patient satisfaction (presumably in the USA) is related to “sociodemographic characteristics; physical and psychological status; attitudes and expectations about medical care; and the structure, process, and outcomes of care”. However, these is no consensus about the strength of these factors. Context and the type of care may be important, as are technical matters, ie what questions are asked and how, the survey timing and so on. Hall and Dornan (1988a,b;1990) have undertaken a meta-analysis of 221 quantitative patient satisfaction studies, mainly conducted in the United States. They conclude (1990) that sociodemographic characteristics are a predictor of satisfaction, but that the relations are small, and for some variables, go in opposite directions. However, there is no discussion in their article of the context of the studies, ie whether patients were public or private, or whether the samples were representative of the population sub-samples using the health service. Furthermore, this analysis assumes there is an underlying unitary concept of satisfaction (Carr-Hill 1992:243). The research by Brown and Lumley (1993), demonstrating the dissatisfaction of women who were disadvantaged, suggests that dissatisfaction should not be seen in isolation of the restricted choices these women had. Hall and Dornan’s (1988b) meta-analysis also highlights the lack of experimentation in surveys. For example, children or adolescents were rarely respondents despite paediatric care being the focus of 11% of surveys. Nearly half the surveys referred to adult ambulatory care, compared to 12% referring to inpatient care. Respondents were spread across socio-economic groups, but in only less than half the surveys did the authors report on which was the largest ethnic group. Nearly a quarter surveyed were black people but only 3% were Hispanic people. There is no mention of other ethnic groups. The majority of surveys were administered immediately after the episode of care. Finally, they note that most of the survey instruments used were developed for the survey and were not used again. By contrast with the majority of the surveys noted by Hall and Dornan (1988a), where people had to rate their satisfaction with elements of care by some scale, Cleary et al. (1991) report on a survey of 6455 adult hospital patients, where respondents were asked to report on their experiences. This became the Picker-Commonwealth survey and is discussed in detail in chapter 11. This survey illustrates an important shift in thinking in two regards: they have moved from a preoccupation with causal variables of satisfaction, to an interest in the use of patient reporting data to improve quality; and they are using the data to compare the performance of hospitals. The authors suggest the possibility of comparing both clinical units and hospitals, using the survey model, but there is no discussion of whether condition related data may have a bearing on the results (1991:265). Carr-Hill (1992:244) argues, following Locker & Dunt (1978), that “data on consumer satisfaction are comparable across environments
95
only to the extent that service consumption coincides”. He believes that interpreting results from even a sample from generic services is difficult. The use of consumer reports on the technical quality of care has been less accepted than their comments on other aspects such as interpersonal relations or the amenity of the hospital (Westbrook 1993:81). However, Davies and Ware (1988:44), in a review of the material, argue that consumer ratings of technical quality are a cost-effective source of information for quality review activities. Westbrook (1993) confirms this, and notes, in addition, that poor quality care which reduces satisfaction, may in turn adversely affect health outcomes. Carr-Hill (1992:244) notes that most studies assume that patients cannot or should not evaluate clinical practice, although there is work with people who have chronic conditions, demonstrating they become ‘experts’ in their field, and subsequently more critical and less satisfied. Meredith and Wood (1995b:7) argue that questionnaires should be sensitive enough “to record the full experience of ... patients”. Furthermore, if the principal goal of the survey is to improve services, it should take account of the role played by all parties, including the patient, the rationale being that the patient should be equipped to make the most of the encounter (1995b:13). In order to gain an understanding of the entire experience, from outpatients visit to recovery, their questionnaire for the Royal College of Surgeons of England was administered in two stages (see chapter 10). Issues related to the method chosen In his critique of Hall and Dornan’s meta-analysis, Carr-Hill (1992:243) argues that even assuming patient satisfaction studies can be aggregated and compared, the methodological variations undermine the attempt at comparison. More fruitful, he writes, are attempts to understand how results from surveys are sensitive to their design, as there are considerable opportunities for manipulating the design and hence the findings (1992:237,243). Whitfield and Baker (1992:152), who have researched patient satisfaction with general practice, go further, noting that “poor questionnaires act as a form of censorship imposed on patients”. The following discussion highlights some of the ways the results can vary as a result of the method chosen. Quantitative vs qualitative In the main, most patient satisfaction surveys have been used to calculate the percentage of people satisfied with elements of care, by adding up their ratings to pre-coded questions. This is regarded as quantitative method. In part because of cost, but also because quantitative methods were the dominant type for many years, qualitative research in evaluating patients’ care is much less common. In this type of research, commonly the patient (or carer or family member) is interviewed, in a semi or unstructured way, and asked to recount their experiences or their preferences. A small number of research projects have compared the results obtained from qualitative and quantitative research methods. Daly and McDonald (1993) have shown that, in their study of the social impact of echocardiography, interviews revealed much higher rates of doubt and anxiety
96
than questionnaires (1993:140). Sometimes these concerns did not emerge until late in the interview. Meredith and Wood (1995c), in their surveys conducted for the Royal College of Surgeons of England, used qualitative research (in-depth interviews with patients) as one of the key inputs to the development of the pilot survey. This ensures that the right questions are being asked, they argue. The findings of the two methods can then be compared. They found, for example, that dissatisfaction with care would be under-reported in “typical service-oriented, closed response questionnaire(s)” (1995c:12). This was in part because people blamed themselves rather than the provider. Closed option, ‘satisfaction-with’ type questions focussed attention on the provider rather than on patients’ views of the roles both had played. Research in the field of evaluation, and the area of the ‘medical encounter’ has argued for the “complementarity (sic) of qualitative and quantitative research methods (Roter & Frankel 1992:1097), and there are some examples of projects where two methods have been employed (see for example, Perreault et al. 1993; Greene et al. 1994). However, it cannot be assumed that qualitative research translates into a method where people’s views are sought actively, rather than passively. For instance, two of the research teams which are referenced above use, and cite the growing popularity of, audio or video taping medical encounters, and then analysing the tapes. Brown and Lumley (1994) report on the critical inclusion into the Consumer Survey for the Victorian Birthing review of a question regarding the importance of having an active say in birth. This issue had arisen frequently in the public submissions to the Review. Some obstetricians had wanted the question removed and there were claims that the question would bias results or women could not answer it (see chapter 3 for a discussion of this survey). In the survey results, having an active say emerged as a critical influence on satisfaction amongst women. In the consumer literature the importance of being involved actively in treatment also emerges (see chapter 2). As has been mentioned in this report, the desire of many people to be active participants in their treatment suggests that they will not favour quality review mechanisms which assign them a passive role. As an aside, the importance of the ‘active say’ question and its emergence from submission reports suggests two critical points: submission data is valid information on consumers’ preferences and experiences; and second, it demonstrates the necessity of using information from consumers to develop a survey. The different responses which are elicited from consumers, depending on the method of research, have been noted in the related branch of health outcomes research. For example, Hill and Harries (1993) found that the UK Short Form-36 was not as sensitive as interviews in picking up the complex improvements in the health of people who had undergone cataract surgery. Condition or casemix In the British research with surgical patients, Meredith and Wood (1995c:12) found that people with potentially serious conditions involving cancer may be more anxious and more negative in their comments than people undergoing more routine surgery. Condition “was a significant variable in the differences in responses specifically in the areas of communication self-rating in the outpatient meeting; the need for follow-up counselling to this meeting; the amount of
97
contact with the surgeon and the need for information literature” (p.14). It is noted that no patient satisfaction research has explored this variable. Furthering this work requires attention to sampling from the general population of patients being surveyed (p.15). Sampling Brown and Lumley (1993:96-7) write that one of the important criteria for establishing an adequate sample size in the Victorian Birthing Review survey was that subgroups of interest were large enough to detect significant differences in satisfaction. The key sub-groups were women in rural areas, women having a caesarean delivery, and women of non-English speaking background. They used the Victorian Perinatal Data Collection to assess the representativeness of respondents. This allowed them to determine and report that certain population sub-groups did not respond to the survey in sufficient representative numbers. They were women born overseas of non-English speaking backgrounds, and single women. More older than younger women responded. This method enabled them to discuss the strengths and weaknesses of the data from their final sample. The population sampling in the British modified Picker-Commonwealth survey (Bruster et al. 1994), excluded patients in the specialties of paediatrics, maternity, psychiatric, and geriatric care. It is claimed that the results “give an unbiased, representative sample of patients”, yet it is acknowledged that those who were not interviewed were more likely to be “seriously ill, senile, (or) ethnic minorities” (1994:1545). The bias caused by nonresponses Responses to surveys may vary greatly. Few in the United States have investigated whether nonrespondents would rate care less favourably. Contrary positions have been obtained in the research that has been done (Aharony and Strasser 1993:55). Follow up cards or calls do influence response rates (p.55), and are recommended to improve response rates. Carr-Hill (1992:244-5) notes from the British research that the characteristics of the intended and achieved samples are rarely compared. Good practice, he writes, would dictate that a small sample of non-respondents are always interviewed. On the issue of response rates, Carr-Hill (1992:245) suggests that around 50% and less is lower than acceptable, and that much higher response rates can be achieved. The response format Aharony and Strasser (1993:56-7) report on various studies that indicate that the way responses are sought may influence the degree of criticism. For example, they have anecdotal evidence that open-ended questions produce more negative comment than closed questions, a finding supported by Perreault et al. (1993), in a study of quantitative and qualitative measures of satisfaction with outpatient psychiatric services. Different types of questions (reporting on occurrences or rating satisfaction) produce different kinds of data (Locker & Dunt 1978:286; Carr-Hill 1992:244). They argue that both types should be included “to avoid under-reporting and to assess patients’ priorities” (Carr-Hill 1992:244). No study was sighted where patients were asked if ratings such as ‘very satisfied’ or ‘not satisfied’ were terms applicable to their
98
experiences. However, Fitzpatrick and Hopkins (1983:309) found, for example, that when discussing various aspects of a consultation with a neurologist, people rarely expressed themselves in terms of satisfaction or dissatisfaction, nor that their comments could be reduced to attitudinal reactions. More important were judgements of outcome, and it is suggested by the authors that these could be a firmer basis for an evaluation of patient-centred care. As discussed above, there are difficulties in using global evaluations of satisfaction as an indicator of quality or for comparative purposes, because of the possibility that dissatisfaction with specific elements is masked, and because the measure is too crude for meaningful comparison (Locker and Dunt 1978:287). A similar point is made by Black and Sanderson (1993) in their British study of day surgery. Ware and Davies (1983:295) found that “relatively small differences in satisfaction rating scales... have noteworthy consequences for patient behaviour”. Phone versus mail In part because of the cost of face-to-face interviews, surveys conducted by phone or via mail have become more popular. McHorney, Kosinski and Ware (1994) have compared various attributes and responses to both. They report the following; •
the total attributable costs per respondent case was higher for phone than mail survey. Both methods included follow ups in the same and alternate mode;
•
the response rate for mail surveys was significantly higher;
•
for both modes, nonrespondents had less education, less income, and were less likely to be employed. There was a higher representation of younger people in mail nonresponses compared to phone;
•
there was more missing data from mail surveys;
•
respondents to the mail survey reported more chronic conditions and less favourably on their health, an effect of the anonymity of the mail survey.
The authors (1994:564-5) note the trade offs which must be made between mail and phone surveys, and the critical issue is the characteristics of the target population in relation to the method used. Perreault et al. (1993) suggest that for people using psychiatric services, the oral administration of questionnaires is “better adapted” to the clientele than self-administered surveys, although there is no discussion of the impact of the two methods of administration. In Australia, Taylor and Clarke (1993) have reported that inpatients of psychiatric hospitals provide reliable data with good response rates. Timing of survey
99
Ratings of care in surveys change over time. Aharony and Strasser (1993:56) report on a study by Ley, Kinsey and Atherton, who found a U-shaped relationship between the length of time post-discharge and a rating of satisfaction with communication. Aharony and Strasser note that no one in the United States has studied whether people’s satisfaction changes when they get the bill for care. The Victorian review of birthing services scheduled their survey to avoid the less critical attitude immediately after birth (surviving a major event), and before sufficient months had elapsed, diluting criticism (Brown & Lumley 1993). Patient or carer In the United States literature, there are some suggestions that family members or carers may rate care more negatively than patients. There is no discussion of whether the two may have separate needs (Aharony & Strasser 1993:60). The process of care Most patient satisfaction research starts with the episode of care. This may miss out on, or downplay the experiences people have while waiting to get into hospital. For instance, a consumer audit of people’s experiences before and after orthopaedic surgery (Rigge 1994) showed that the quality and coordination of care before surgery was not as good as after the operation. This meant that some people entered the hospital in worse condition than may have been possible. Meredith and Wood (1995c:15) concur with Carr-Hill about the need to assess the relative importance different elements of care have to the patient. For example, they found that women with breast cancer placed greater importance on the outpatient visit compared to other aspects of the hospital episode. Auspice and anonymity Several researchers have indicated the importance of having a neutral party to conduct the survey, to remove positive biases such as the tendency to want to express gratitude, or the fear of voicing criticism (Brown & Lumley 1993; Meredith & Wood 1995a). Way of administering questionnaires Some researchers have favoured ward-based distribution of questionnaires to improve response rates. This is argued against by Meredith and Wood (1995b:6-7), who found that ward delivery is disruptive to staff. Distribution may be adversely affected by staff selectivity, and may remove the possibility of a representative sample. They achieved a 78% response to their first postal survey and 93.5% response to their follow up survey (p.5). The arbitrary selection process of staff distribution was confirmed by Ehnfors and Smedby (1993), in Sweden, who concluded that those excluded were those patients whose needs ought to have been particularly noted, ie people who were seriously ill, and those who didn’t speak the country’s principal language. Policy issues
100
If the conduct of patient satisfaction surveys is being promoted in the context of government policies to improve or sustain the quality of care, the following issues should be considered. Charters and standards McIver (1991) argues that people should be asked to assess their care and give feedback in response to a clear statement of rights or standards. Those pertaining to the service should be listed in brief at the front of any patient satisfaction questionnaire. This places people’s feedback in the context of the service goals and standards of an organisation, rather than skewing the research to people’s individual expectations. However, Carr-Hill and Ng (1992) sound a warning, following the plethora of local charters which have developed in Britain, after the publication of the national Patient’s Charter. Some charters confused rights and standards; in some geographical areas, multiple charters were available; and monitoring the standards via satisfaction surveys may ignore organisational or resource constraints. A survey of 36 hospitals in Britain (using a modified Picker-Commonwealth survey), compared results to rights in the Patient’s Charter (Bruster et al. 1994). This identified broad areas where patients’ rights were not met, for example, choice over participation in student training. A consumer framework Carr-Hill (1992:238-40) has argued that a framework of consumer principles should provide the skeleton from which to flesh out any satisfaction instrument. He uses the principles developed by the British National Consumer Council, which in theory aim to redress the imbalance in power between service providers and consumers. The principles cover seven aspects: • • • • • • •
access; choice; information; redress; safety; value for money; and equity.
Although Carr-Hill is conscious of the difficulties of applying these in the health market, and of the inapplicability of assessing equity issues via surveys, he notes that the aspects of access, information and overall quality are the only issues to be addressed consistently in most British surveys to date. He concludes that the development of any standardised instrument should either give a systematic consideration to these principles, or provide a “coherent, theoretically based account of why it is appropriate to treat satisfaction as a unitary concept” (1992:240). Relationship to outcome data Meredith and Wood (1995a:68) argue that objective clinical outcome data are only meaningful in their correspondence with information on how patients perceive outcomes. On the continuum
101
between effective care (defined clinically) and quality care (using both sources), patient satisfaction data “lies somewhere between these concepts”. Aharony and Strasser (1993:52) conclude that while there is a relationship between satisfaction and better outcomes, it is not simple and direct. The links between satisfied patients, who are compliant and, by assumption, more likely to have better outcomes is made. This is problematical, in our view, in the light of practice variations research, the lack of knowledge of what contributes to better outcomes, and the absence of criteria defined by consumers on what constitutes good outcomes. Furthermore, the literature on compliance, which questions the automatic assumption that the uncompliant patient is forgetful, stupid or difficult to manage, suggests that patients make more complex trade-offs between treatment, health state and social well-being than the ‘compliance’ literature would indicate (see Donovan & Blake 1992). Suiting the method to the question Baum (1995) has argued of the need to suit the method to the question. Qualitative methods are best suited to explore how people interact, how they make sense of the experiences of health and disease, and how people in positions of differing power status interact and participate in whatever processes are being studied. Quantitative methods allow questions such as ‘how much?’ and ‘how many?’ to be answered. She notes that public health practitioners, and by presumption, health policy makers, will need to ask both types of questions. Shmerling, Schattner and Piterman (1993), writing of research into Australian general practice, give several examples of how the two methods may complement each other, giving different information “of equal importance” (1993:622). The research by qualitative method into adverse incidents, by various Australian speciality colleges (discussed in chapter 5) illustrates the growing acceptance that different methods suit different purposes, and that meaningful data from one method cannot be used to answer all questions. The policy context The context for much of the patient satisfaction research in the United States has been academic interest. Hall and Dornan (1988b), after analysing 221 studies, ask the question why so many studies have been conducted? They conclude it is either academic interest or the desire of services to retain patients, ie market share (1988b:643). They conclude that little if anything is known of the use to which the data is put, although they comment that in-house surveys which are used in quality assurance are unlikely to be published. In more recent times in the United States, the policy debates have been dominated by a preoccupation with quality. In a review article on the relationships between quality and market issues, O’Connor and Bowers (1990:197) write that “As the issue of quality becomes more important, providers will begin to segment their markets and to differentiate their products based on quality”. The link here with patient satisfaction surveys is that ensuring patients are satisfied “will result in increased referrals, larger volume, and a greater number of individuals who return to the institution” (p.197).What is not discussed is the degree to which patients in the United States’ private system have control over the hospital choice or whether this decision is made via choice of doctor or results from the membership of a particular insurance fund or health maintenance organisation.
102
Concern with the interpersonal shortcomings of health care has become a focus of British researchers, argues Meredith (1993), in the context of the Conservative Government’s policy documents – Working for Patients (1989), and The Patient’s Charter (1991), with their emphasis on promoting individual choice, and the ‘patient consumer’. The increasing use of patient satisfaction surveys, the growth of consumer audit and the public debate over the Charter by consumer, provider and academic organisations all signal the more visible role consumers have played in the British policy process. Summary There are six major concluding points which emerge from this review of key issues in the literature: •
there has been a shift in thinking about the role of the consumer as patient. It is more likely now that researchers want to know what consumers think; that they accept that what the patient tells them is an accurate reflection of what happened; and that this information can be used to improve the quality of care;
•
coinciding with this shift, and related to it, is a serious questioning of the conception of satisfaction as a unitary concept whose causal variables can be measured;
•
researchers now are more in favour of using several research methods to inform their survey instruments, and gather data;
•
the methodological complexities of patient satisfaction research are considerable and should not be downplayed;
•
stereotypes of consumers have been discredited. Future researchers should be wary of assuming that sub-groups of people think as they suppose; and
•
certain groups of people, whose social position or state of health may make them vulnerable to poorer quality care, are extremely difficult to reach via the conventional questionnaire. These people remain the challenge of researchers and policy makers.
103
CHAPTER 10: PATIENT SATISFACTION INITIATIVES IN THE UNITED KINGDOM The main impetus for work on consumer feedback in the UK was the 1983 Griffiths Report into management of the National Health Service (NHS) which recommended that the NHS pay more attention to the experience and perceptions of those who used it. Some work on patient feedback had been done before this, largely by the King’s Fund (Raphael) and the Institute for Social Studies in Health (Cartwright). Within a broader approach to consumer feedback, a great deal of interesting work has been done on patient satisfaction surveys. There are a range of key organisations associated with this work. This chapter reports on a range of key documents and activities by highlighting the range of organisations which have produced the reports. The chapter is structured by auspicing organisation and by time frame. The emergence of a complex structure based on the purchaser - provider split has seen a variety of parties develop an interest in patient experience and satisfaction, including purchaser groups, provider groups and consumer organisations as well as individual hospitals. Within the literature consequently, there is often a broad approach to hospital based surveys. There are a range of bodies with a stake in the information that can be provided by well constructed patient satisfaction surveys. In this respect, the British work on patient satisfaction/feedback reflects a policy approach to the use of patient feedback information for varieties of purpose which include quality assurance and planning of health services. The contribution of this chapter is to reflect on the use of patient satisfaction surveys. Centre for Health Economics, York 1989 This project was commissioned by the NHS in early 1988 to gain an overview of how health authorities were approaching the issue of quality assurance and customer relations. It had three components: • a survey of customer relations structures in the NHS; • an introduction to survey methods used in customer feedback surveys; and • a review of current practices. The survey was conducted from March to October 1988. The first part involved a survey of all regional health authorities, district health authorities, special health authorities and community health councils in England and Wales, seeking details of studies which had been undertaken on consumers’ views over a five year period, and any information about the use of such surveys in service planning. The second part was based on structured interviews with personnel with Quality Assurance or Customer Relations responsibilities across all health regions. The Executive Summary (Carr-Hill, McIver and Dixon 1989) emphasises three key points about patient surveys: • the importance of being clear about the aims of the investigation; • being sure that the survey method is the appropriate form of investigation and that it can provide relevant and new information; and 104
• clarifying the research design and ensuring that everyone has a clear understanding of the enterprise. The report argues that the reliance on formal methods such as patient satisfaction has a tendency to devalue less formal and less technical research methods as ways of getting consumer feedback (Dixon and Carr-Hill 1989:3). There are many forms of informal data collection that are often part of the daily routine. Figure 1 illustrates their broader argument that services tend to reach for patient satisfaction surveys with all their expense, unforeseen pitfalls and methodological complexities, when other means of obtaining information would be more appropriate. Dixon and Carr-Hill (1989:1-2) argue in the report that there are a range of pitfalls to patient satisfaction surveys: • the research has not been agreed with the relevant staff; • staff are likely to oppose the eventual proposals; • the group doing them has no access to decision making, has low status, or is marginalised; • there are not likely to be the resources to implement any proposals arising; • there are insufficient technical and staff resources to conduct the research effectively, especially the analysis and reporting; • the time-scale is too tight; • the data is likely to be inconclusive, and • the data is available from other sources. In their review of the then current British patient satisfaction surveys, Dixon and Carr-Hill (1989) suggest that: • the lack of attention to design and to the capacity to obtain useful data means that much money, time and effort is wasted because the research can not achieve its aims; • most consumer surveys tell you very little that is new, but may provide a firmer basis for policy; • hastily planned, poorly executed, casually distributed questionnaires are counterproductive;
105
Figure 1 When Not To Do A Survey
Strategies
Policy Options
Do we know enough to decide
What do we need to know?
No
Can the information be obtained from existing sources?
Yes
Yes
No
How can the information be obtained?
Consultation
DO NOT DO A SURVEY
Analysis of existing data
Observations of clinics, etc.
Asking people
Perhaps do a survey
Yes
Source: Dixon, P. and Carr-Hill, R. (1989), The NHS and its Customers: Customer Feedback Surveys - an introduction to Survey Methods, Centre for Health Economics, University of York.
106
• absolute percentages of satisfaction have little meaning and that even when problems are well-known, patients may record high satisfaction; • locally designed questionnaires run the risk of offering no opportunity for user comparability, but at the same time not being specific enough to obtain the potential benefit of local design. While the intention of the project had been to develop a guide to ‘good practice’, this proved to be premature in terms of the relative infancy of many of the components of customer feedback policy and practice. The research found that, overall, there was little integration between customer relations and quality assurance (QA) and that the consumer perspective was under-emphasised in quality assurance (McIver and Carr-Hill 1989:13). National Consumer Council 1990 The work of the National Consumer Council on health services follows earlier work on local government services. The context for the earlier work was concern by the National Consumer Council that pressure on local government to provide value for money too often focussed on ‘economy’ and ‘efficiency’ without an equal pre-occupation with ‘effectiveness’. Measuring Up: Consumer assessment of local authority services: a guideline study (1986) aimed to encourage local authorities to set explicit targets for their ‘consumer performance’ and to evaluate and report on their achievements, and to provide local government consumers with information which would enable them to raise questions about the performance of their authorities (National Consumer Council 1986:ii). They argued for twin strands of greater public/consumer accountability and more consumer oriented decision making. Their aim was to encourage collection of information about consumers, as well as information which could be released to consumers. Basic to their approach was the idea that in assessing the effects of a service on consumers, consumers themselves must be asked which aspects of the service matter to them. Consumers, they argued, were particularly interested in measures which would demonstrate: • • • • •
the way services meet consumer needs and preferences; the quantity of services provided; their quality; their cost to the user and the community; their effectiveness, including the extent to which they satisfy users and the community.
An earlier paper Consumer Principles (1983) noted that the development of performances indicators was not straight forward, particularly on service quality and consumer satisfaction. They cautioned that it would be a mistake to focus only on improving performance that was more easily quantified, and ignoring the more qualitative elements, thus allowing the process of measurement to dictate policy. The 1989 White Paper Working for Patients introduced further thinking about “quality” and endorsed the principle of medical audit. However, the National Consumer Council argued that 107
it was weak on ideas about involving consumers in a more systematic evaluation of the effectiveness of services and the Council proposed a more active version of consumer involvement (National Consumer Council 1990:5). Reviewing work done on patient satisfaction surveys, their 1990 study Consulting Consumers in the NHS commented that the popular CASPE survey (see next section), while it aimed to provide information to improve services, effectively used patients’ views to reinforce current practices. What patients wanted from a service was not addressed, they argued, but rather what they thought of what they were already getting (National Consumer Council 1990:7). There was no cycle set up by a questionnaire that was used for continuous monitoring that would see standards reset on the basis of the information in previous surveys. Their study, Consulting Consumers in the NHS, tested the view that consumers’ concerns should be established before service standards are set, so that performance can be evaluated in order to improve quality. The study selected services for elderly persons at home suffering from dementia as a case study to test their approach. Figure 2 charts the consumer criteria developed by the National Consumer Council to evaluate services. This study used qualitative research methods based on in-depth interviews with individuals, some pairs of carer and cared for, and small groups of carers. Elderly people with dementia were interviewed, as well as carers. The study proposed checklists, questionnaires and performance indicators based on their findings. There are three issues worth noting about this study of consumer feedback: • this group is particularly disadvantaged in access to services; • this group presents challenges for consumer feedback; • qualitative research was the preferred approach. These reports provide good frameworks and suggestions about criteria for service evaluation. They discuss issues that go to the heart of the purpose for obtaining consumer feedback. The National Consumer Council report provides a good example of a consumer feedback project aimed at a group that produces difficulties for obtaining feedback, and demonstrates that patient satisfaction surveys are not always the most appropriate method. King’s Fund Centre for Health Services Development: Consumer Feedback Resource 1991 The Consumer Feedback Resource was set up within the Quality Improvement Program at King’s Fund, where Shirley McIver produced a series of publications on Obtaining the Views of Users of Health Services. These provide a practical guide to obtaining consumer feedback, as well as a discussion of a range of issues pertinent to obtaining consumer feedback and the design of patient satisfaction surveys. Patient satisfaction surveys are considered within the broader context of approaches to consumer feedback. This material appears to build on the earlier work done by the Centre for Health Economics at York.
108
McIver suggests that prior to 1980s, social research was undertaken either by academics for their own purposes or Community Health Councils, whose agenda was closer to that of health care providers and whose research was under funded, inexperienced and lacking specificity (McIver 1991:1). Exceptions were Winifred Raphael at King’s Fund and Ann Cartwright at the Institute for Social Studies in Medical Care. What has emerged since the debates of the 1980s are the development of the health surveys that health care providers can use. The King’s Fund publications on Obtaining the Views canvass a range of approaches to consumer feedback, including approaches to different groups of consumers, the range of purposes for obtaining such information, quantitative and qualitative methods, and ways to disseminate results. McIver distinguishes two broad approaches to consumer feedback: quantitative methods, which ask ‘how many?’ questions, and qualitative methods based on narrative information which asked the ‘why?’ and ‘how?’ questions. In reality, hospitals, purchasers and funders often need a mix of both, and these are not an either/or (1992a:3). McIver (1992a) argues that in addition to the methods chosen, other important issues are: • the provision of good quality information to consumers so that they know what to expect and how to judge the service they receive, and • the development of a patient-oriented culture so that all staff listen to patients and are prepared to change attitudes and practices. McIver highlights four key limitations of questionnaires: • the agenda. Like a number of other authors, McIver argues that it is important for questionnaires to be developed on issues of concern to patients, not just those of providers. This is where qualitative research is important. Qualitative research can provide information about those aspects of the hospital visit of most concern to consumers as well as the language and criteria consumers use to describe their likes and dislikes (McIver 1992a:71) • sensitivity. This is about questions of the amount of detail, the length and superficiality or otherwise of the questions. Many Australian consumers are sceptical and scornful of patient satisfaction surveys on exactly these grounds. They regard questions as superficial, focused predominantly on hotel factors, as well as not reflecting the more major concerns of consumers • selectivity. Self-completed satisfaction surveys are not a good instrument for some people those who cannot read, use their hands, have impaired eye-sight, those with poor English literacy, those who feel ill. • lack of consumer participation. Patient satisfaction surveys are consistent with quite passive approaches to consumer participation. Consumers can be treated simply as providers of information which providers act on, rather than partners in decision-making 109
about the planning of health services. McIver (1991) argues for results to be fed back to consumers. A more robust consumer approach would also involve consumers in helping to determine service responses to survey results. McIver (1992a) reviews a number of important surveys in use in Britain on which there are now central databases of results. 1) The Raphael questionnaire Developed in the 1970s, this is still in use, but its age shows and it is skewed in the direction of hotel/environmental factors. Only three out of 24 questions are about care. In an important step, Raphael asked hospitals to report on action taken as a result of the survey, along with some attempt to categorise where the service changes were made. 2) The “What the Patient thinks” questionnaire This was developed on the basis of doctoral thesis work by Dr. Andrew Thompson, and distributed through the Health Policy and Advisory Unit (HPAU), a national charity which conducts independent research on health care. Those who use the survey purchase the questionnaire, sampling frame, standard letters, pre-paid reply envelopes and staff training. It is a postal survey. Hospitals distribute the questionnaire, and HPAU receives the completed questionnaires and analyses the results. The survey has 90 main questions based on arriving at hospital, ‘inside the ward’, the people in hospital, information and ‘learning hospital’. While this is a long questionnaire, the average response rate is reported to be over 70 per cent. The statistical report provided to hospitals is large, but HPAU also provides a management action plan. 3) PATSAT This has been developed by Clinical Accountability, Service and Planning Evaluation (CASPE), a consultancy group auspiced by the King’s Fund at Bayswater. There is some other discussion of issues raised by this survey in the methodology chapter. The PATSAT system offers continuous monitoring. PATSAT tries to deal with issues about the length of patient satisfaction surveys by designing a short general questionnaire, accompanied by subsidiary questionnaires to be used where dissatisfaction is shown to occur. PATSAT works through the hospital administration computer system, coding patient details onto the questionnaire. Questionnaires are optically read and analysed using PATSAT software. Once costs are met up front, it is cheap to run over time and is used by some hospitals to run continuous monitoring. However, the limitation of this approach is that the generality of the questions in the original short questionnaires is such that it does not encourage a discriminating patient response and is
110
likely to understate patient dissatisfaction, and thus not trigger the use of the secondary questionnaire. Response rates are low and there are consequently concerns about representativeness. Response rates are also falling over time. An evaluation of CASPE (Rafferty & Zarb 1990) followed up non-respondents of PATSAT on an individual basis. Of those who subsequently completed the questionnaire, nearly a quarter were found to require assistance to fill in the questionnaire. This evaluation, based on the follow-up interviews, also proposed more attention needed to be given in PATSAT to interpersonal aspects of medical care as well as questions to discriminate between different types of nurses. There are two final useful issues noted by McIver: • the proliferation of questionnaires is already raising problems of coordination and potential patient irritation. This suggests that there is a need for greater liaison on patient satisfaction surveys; • qualitative research by the British Market Research bureau found that the most important issues for patients were `their health and whether they felt better for their stay in hospital’ (McIver 1992a:67). This aspect of the hospital stay is rarely found in patient satisfaction surveys and suggests that, from a consumer perspective, outcomes, as consumers define them are not a completely separate issue from patient satisfaction. Overall, McIver favours purpose built questionnaires based on consumer priorities, unlike the Centre for Health Economics who had favoured use of the standard questionnaires, although Carr Hill appears to have changed this view (see chapter 9). McIver notes, however, the relative impersonality of patient satisfaction surveys. Although surveys collect information from large numbers of people, they are not McIver argues, necessarily ‘a very good example of a person-centred attitude towards service provision’ (McIver 1992a:97). Greater London Association of Community Health Councils 1992 Community Health Councils (CHCs) were first set up in 1973 and are now the statutory body for representing user’ interests in the NHS. They particularly represent the interests of some of the groups most disadvantaged in access to health. Their history has tended to be one of marginalisation, although the more effective of the CHCs seem to be carving out a more defined role in the contract environment and this has been emphasised recently by the NHS Management Executive. In 1992, Greater London Association of Community Health Councils reported on a study of the involvement of CHCs and users in medical audit. They found that there was very little beyond patients providing some information to the monitoring phase. This is where patient satisfaction surveys tend to be used. Seeking patient views seemed to be something that doctors did to patients rather than with them. Health services users had little access to the selection of topics for audit, little access to setting criteria and standards, and concerns about confidentiality seemed to limit dissemination of findings. Where patients views were sought, they were often on an agenda of issues set by doctors rather than patients themselves. There 111
was also a tendency to use clinicians such as GPs as proxies for consumer views. The Greater London Association of Community Health Councils argued that were patient satisfaction surveys were used, they should reflect patient concerns and not just the ‘charm-school-andbetter-wallpaper’ approach to consumer input (Pollitt 1988). In 1992, the Greater London Association of Community Health Councils, in conjunction with the King’s Fund, published a report on London’s acute health services, A User Perspective: Views on London’s Acute Health Services. It opts to concentrate on ‘London users’ and the impact of the crisis in London acute health services on ‘London users’, particularly in light of the deficiencies in primary health care services in London which have been well documented. In doing so, it argues that the term ’user’ needs to include not only present users, but those who wish to use services or would do so if services more adequately met their needs. They argue that preoccupation with user satisfaction studies and quality initiatives tend to work only with those who have managed to make their way into services. The wisdom of understanding who the users and potential users are has subsequently been underlined by the deepening crisis that followed the reforms which resulted from the King’s Fund Commission on the Future of London’s Acute Health services. The Royal College of Surgeons: Patient Satisfaction with Surgical Services (1991-1993) The Royal College of Surgeons in 1990 initiated a major project to develop and implement an Audit instrument to assess patient satisfaction with surgical services with the employment of a sociologist, Philip Meredith, as Senior Research Fellow. The College now offers this consumer audit as a service to surgeons, with the College acting as an independent auspice. This is extensively reported in publications from the College and British and Australian journals. It was developed and piloted over a three year period as an initiative of the Royal College of Surgeons. This was undertaken by the College under the auspice of its Audit and Quality Assurance Committee as a response to requirements to obtain consumer feedback contained in The Patient’s Charter and as part of clinical audit, and is now being implemented in a further 8 hospitals. The steering group for this project included surgeons, CHC and other patient representation. The aim of the project was to develop an investigative procedure which could be used routinely to assess patient’s attitudes to the surgical component of their treatment which could be incorporated into its permanent clinical audit program and into its educational programs. This was also seen as having the capacity to contribute to ‘best practice’ protocols and standards. Both patient evaluations, and established measure of outcome, provide information about quality. This project was also designed to research the impact of changes in service delivery on patient evaluations. The survey also includes sections which outline the relevant service standards and national benchmarks and entitlements against which consumers can evaluate their own experience. The project explored several areas that were thought to be problematic:
112
• the process by which the procedures are explained to the patient and relatives by the surgeon; • the patient’s level of comprehension; • the adequacy of information throughout the treatment period, with particular reference to the details of technique and prospective outcomes; and • patient perception of surgical care. The College of Surgeons project provides an important variation on patient satisfaction surveys, being aimed at patients surgical experiences rather than the more general hospital experience. Methodology The methodology of the College project illustrates some key aspects in assessing patient satisfaction. The patient satisfaction instrument was developed following a data gathering exercise involving multiple sources based on qualitative work with patients, nurses and surgeons over 10 weeks in six hospital sites. Existing literature, complaints data, interviews with consultant surgeons and surgical trainees and open interviews with patients were used to develop an instrument that could be piloted. The results were tested back against the responses in the open interviews. Over 800 patients were involved in the developmental stage. This process enabled the project to look at some of the more interactive elements of patient satisfaction, such as communication, in an innovative way. The qualitative research found that when patients evaluated communication they criticised themselves as well as their consultant. Patients would comment that the problem was that they did not know what questions to ask, or felt inadequate to respond on the spot to findings.. They felt that time to discuss issues was lacking, and the atmosphere was not conducive to such discussions. The literature on women with diagnosed breast cancer makes similar comments about women being expected to take in, in a single session, all that is involved in being told of a potentially life threatening diagnosis and being expected to make decisions about treatment. This approach to communication suggests that there is more to the issue than better communication skills for surgeons, although this is important. For public patients and for clinicians, there are structural issues about the time available and not feeling pressured by the line of other waiting patients. This raises resource issues as well as organisational issues. It also suggests that it is important to give consumers information and time to frame their own questions. The qualitative research also suggested that previous hospital experience could influence their experience of the outpatient visit through being better prepared for the consultation. There was also a suggestion that patient discontent with waiting can lay the ground for an unsatisfactory consultation which can be then attributed to failure of communication. This again illustrates the point that both patients and clinicians bring something to the consultation and that it is an active process on both sides. As such, the solutions to communication problems need to be addressed on both sides. 113
The College of Surgeons patient satisfaction audit instrument appears to be the only patient satisfaction instruments to look at casemix. This arose also from the qualitative phase, which established that patient judgments were different depending on the person’s condition, particularly if malignancy was involved. More women than men were negative about the outpatient visit, as were those with more anxiety-producing conditions. This revolved around the adequacy of the visit for understanding the diagnosis and proposed treatment, and for being involved in decision-making. The NH&MRC Guidelines on Treatment of Diagnosed Breast Cancer (1995) proposes that women should not be expected to decide on treatment at the first interview and proposes a range of ways to provide accessible information to women (and the few men who are so diagnosed). In the ward, patients were concerned about a set of issues about communication and information, often not understanding test results, and lacking a single doctor whom they could trust for information and interest in them. This project attempts to come to grips with the idea that patient satisfaction is a derived concept, as discussed in chapter 9. That is, patient satisfaction is not something that resides in the patient, waiting only to be measured by patient satisfaction surveys. The qualitative phase found, as other research has also hinted at, that some experiences may be rated as ‘satisfactory’ or ‘good’, but remain a cause of discontent. There are methodological issues about how patients form judgments and how these are best incorporated in patient satisfaction surveys. There are also, this research suggests, problems about evaluation given that patients can feel ill equipped to get the information they need, and the research design needs to reflect this. A patient satisfaction instrument is of value only if it provides information that is useful in improving quality. To do this, it has to get to honest and frank assessments of what patients think. It needs to get past the gratitude response. It needs to provide ways that disaggregate questions about providers and experiences. Patients have interactions with particular providers, not ‘doctors’ and ‘nurses’, and their experience in hospital is based on these particular relationships. As opposed to most other questionnaires which ask about patient’s ratings of ‘doctors’ or ‘nurses’, this instrument seeks to obtain information about individual consultants as well as the team. Another feature of the instrument in that consumers are surveyed twice, shortly after discharge and eight weeks later, thus capturing emergent satisfaction judgments over time. The first part of the pilot contained 74 questions, and the second form 32. This was refined down to 55 and 15 questions. The two part administration enables the survey to capture something of the dynamic nature of patient evaluations, as well being able to see at which stage communication issues emerge. It also enables it to address discharge issues and health outcomes as patients experience them. One section of the survey is based on the outcome of surgery. College administration The College of Surgeons patient satisfaction survey is offered to hospitals by the College. The College auspice is based on a principle similar to that of other surveys, that is, that consumers are often cautious about expressing dissatisfaction about service providers and generally
114
grateful overall, and a neutral party provide confidentiality and anonymity for consumers to express their opinions. This same confidentiality can enable individual surgeons, should they wish to do so, to get reports on their individual performance and ratings in comparison to those of fellow surgeons. Overall reporting can be in a variety of forms, eg by clinical directorate, by ward, by hospital. Over time, the aim is to generate indicators of improvement in service delivery. The College will also advise the hospital about what follow up qualitative research is necessary to identify the precise nature of weaknesses in service delivery. The questionnaire instrument consists of two postal questionnaires, distributed by the hospital within two weeks and at eight weeks after discharge with forms being returned to the College. The College uses software to read and code the questionnaires and data is analysed using SPSS software. Sampling Patients are sampled until a target sample size per surgeon (about 400) is reached, in order to ensure representativeness, a spread of conditions and bed duration. This can take six months. This sample size is needed for analysis that accounts for casemix. Response rates The response rates for the pilot, based on two reminders, varied per hospital, from 84.1% to 93.6%. This is exceptionally good. For full implementation, the College is operating on the assumption of 80% response rates. Comment This is a particularly good patient satisfaction instrument and one of the better instrument in the field. It deals well with the difficult methodological issues in patient satisfaction surveys. It combines the advantage of being purpose built and having rigorous design, with a format which provides comparative data, but also the capacity to customise the questionnaire. It can generate casemix based information. It is also targeted at the hard to measure aspect of patient satisfaction - communication and information. The College based nature of this exercise is an interesting model, one we see as being additional to, but not replacing other auspices. It tackles patient evaluations as an aspect of clinical audit with a group of clinicians traditionally somewhat resistant to the legitimacy of such exercises. It could provide a useful contribution to the processes for developing and implementing practice guidelines as well as College educational activities, particularly in respect of interpersonal aspects of clinical care. For example, the recent Australian development of practice guidelines for the treatment of diagnosed breast cancer used a literature review of women’s experiences of the treatment of breast cancer to highlight aspects of the interpersonal aspects of clinical care which could be built into the guidelines. The College type of patient survey would assist monitoring and evaluation, add to the specificity and localisation of
115
information already available, and provide information in areas where less research has been done. Finally, it is an instrument whose design makes it an appropriate type of questionnaire for public patients in Australia. Private sector assumptions about hospital choice are key design assumptions in US instruments. The focus on the outpatient visit, the inpatient experience, hospital discharge and post hospital care captures the whole episode of care as opposed to a sole focus on the inpatient experience, and in organisational settings that are appropriate to the experience of public patients. Audit Commission 1993 The Audit Commission is responsible for auditing the accounts of NHS bodies in England and Wales and examining the economy, efficiency and effectiveness of resource use. An interesting feature of this Audit Commission is its focus on quality and effectiveness review. This has involved the Commission in developing greater understanding of the needs of service users and the impact of services on them. The Audit Commission’s 1993 Report What seems to be the matter: Communication between Hospitals and Patients about communication with patients in acute general hospitals is the first national report to assess a service solely from the point of view of the user, in this case, patients, family, friends and carers. Contributing to the difficulty patients experienced with this area, the overall common findings were (Audit Commission 1993:1-2): • staff did not work well together, including: • professionals not talking to each other and therefore failing to coordinate patient care; • managers and clinicians not cooperating to improve processes and systems; • poor communications between departments so that good ideas don’t spread. • shortage of skills and standards in a number of areas including written information for the general public, and conducting surveys; and • lack of clarity about resource availability. We report here only on the Audit Commission’s findings about ways hospitals have to elicit patients views and not on the rest of the material on communication with patients. The Audit Commission found that patients have problems with the following areas (Audit Commission 1993:45-47) Coverage • the distribution of questionnaires was uneven and the vast majority of patients had never been asked their views.
116
Poor research methods • hospitals reached for questionnaire based surveys, when these might not be the best research tool. Telephone and face to face interviews may sometimes be more appropriate; • the questions were poorly designed, eg ambiguous, and questionnaires were poorly structured; • there was poor design, print size and layout of questionnaires; and • they were not accessible to people who did not read English. Irrelevant questions • the wrong questions were often asked, eg only about hotel services, and not about clinical care and communication; • patients were often asked if they were ‘satisfied’ with communication with entire groups of staff. Feedback • feedback was rarely given to patients about results and what happened. The Audit Commission argued that the underlying cause of this were as follows: Poor standards • There was a lack of direction and poor communication. Often one part of a hospital undertook a survey and other parts knew nothing about it; • there was much reinventing of the wheel. Previous published work was not sought out. Varieties of methodologies were not explored; • there was no benchmarking developed or used; • patient satisfaction projects were embarked on without aims and purposes being sorted out and there was no thinking at this stage about what would be done with the results; and • results were often not reported to staff, let alone to consumers who took part (Audit Commission 1993:47). Lack of direction and poor coordination. • in most hospitals, no one knew about what work was being done on patients views. Most of it was done without central coordination and often with little support. Surveys could be done in the same part of the hospital designed and administered by different people (Audit Commission 1993:47). • clinical Audit was often carried out entirely separately from patient feedback (Audit Commission 1993:47). There were often no links or integration between the
117
different sources of information, such as patient feedback and complaints, and quality monitoring and improvement. Audit Commission’s Recommendations The Audit Commission argued that patient opinion research needed to be put on a more expert basis. The aims need to be clarified, research methods chosen and research instruments developed. Support and guidance should be available to those undertaking the projects. A literature review should be undertaken of work in the same area as a preliminary step. The purpose of work should be clear, as well as how results were to be handled, and how feedback was to be given. Patients or consumers should be involved in drafting and testing research instruments, and the study methods should be piloted. This is a very useful report and is important because much of what it has found is just as true of Australian hospitals and reflects the feedback from Australian hospitals, along with clear opinions that we need to do it better, with better support provided to such activities. Additionally, the report provides an excellent account of the range of issues about communication with patients, some of which is discussed in reference to particular clinical areas. College of Health: Consumer Audit The College of Health was set up in 1983. It is a national charity, funded by member subscriptions. Its aims are: • to help you protect your health and to safeguard our valuable health services; • to complement the Royal Colleges of Surgeons, Physicians and General Practitioners; and • to help to keep healthy: to look after themselves better and to make the most effective use of available health care The services it provides includes courses, a self-help clearing house, an information service about health services as well as a journal. The College of Health is discussed here briefly because of its role in Consumer Audit. Consumer Audit, as developed by the College of Health, is a process for looking at health services from the patient’s point of view, rather than that of the clinician or manager. Consumer Audit is a qualitative approach to obtaining feedback from patients about health services. It emphasises emotional as well as physical well-being and also the quality and clarity of information and communication with patients. This process uses a range of research techniques and is intended to complement other forms of assessment, such as audit and organisational audit (College of Health 1991:6). It was developed because it was rare for patients to be asked to participate in studies of quality of care. Audits tend to treat patients as cases, as casemix classifications, or the subject of medical records (Ragg 1994) Consumer audits are done on the basis of contracts between purchasers, such as Regional Health Authorities, Family Health Service Authorities and fund holding general practices, or 118
providers such as hospital trusts or community trusts. The audits are undertaken on the basis of joint work between the College of Health and the commissioning organisation, so, that there is some skills transfer. The College has undertaken a wide range of consumer audits across general practice, A&E units, day surgery, services for particular groups of consumers such as older people or women with breast cancer, experiences of patients waiting for orthopaedic surgery, as well a range of issues for different ethnic groups. In addition, some projects have focused in particular on the provision of information services. The methods used in consumer audit include • in-depth semi-structured interviews with patients and carers in their own homes, as well as in outpatient departments, on wards and at GPs’ surgeries, based on samples drawn from patient records; • focus group discussions with patients, carers and with members of voluntary and self-help groups; • observation, with the aid of check lists, in outpatient departments, on wards and in GPs’ surgeries; • interviews and group discussions with management, medical, nursing, clerical and other staff; • review of written information for patients, and • reviews of relevant literature. The reports of consumer audits available from the College of Health contain a great deal of interesting information. One example from a consumer audit is taken from a study of day surgery. The hospital in a bid for greater efficiency had changed the transport of patients to surgery to make sure surgeons were not held up and that fewer porters and nurses were needed, but the patients interviewed commented on the length of time - over an hour, they were lying on trolleys waiting ‘looking at the ceiling and counting the dead flies in the light fitting’ or waiting, cold, with glasses and teeth removed, so that even talking was difficult (College of Health 1992:33). The College of Health lists the benefits of consumer audit as follows: • • • • • •
like clinical audit, it is an educational process for professionals; staff development; listening to the carers and patients who take part; identifying improvements which are often easy and cheap to make; developing an audit cycle; setting clinical standards which incorporate the patients’ perspective, knowledge and feedback, and • identifying good practice (College of Health 1992:14-16)
119
The College of Health suggest that there are a range of ways of involving users of services in Consumer audit: • • • • •
deciding what should be audited; deciding what methods to use and what groups should be involved; carrying out the audit and providing feedback about needs and services; joint discussion of findings; agreeing on changes and areas for implementation. (College of Health 1992:132)
While the College of Health is not particularly associated with patient satisfaction surveys, the discussion is included here to illustrate the principle that there are means other than patient satisfaction surveys to get consumer appraisal in a properly structured and disciplined way which can be used to improve the quality of health services. Secondly, it is an interesting example of a principle that there is a role for consumer based organisations to undertake consumer audits. Summary remarks There is much of interest in the British literature, only some of which is highlighted here. Four features are of particular interest. Context of purchasing environment. The development of consumer feedback mechanisms and patient satisfaction surveys has accelerated in the context of the ‘internal market’, within the context of greater specification about health services contracts and a more explicit purchasing approach. In some respects, this has emerged out of deliberate attempts by consumer organisations and others, such as the King’s Fund and the Audit Commission, to balance the focus on ‘efficiency’ and ‘economy’ with an equal focus on ‘effectiveness’, as well as broader attempts within the NHS to use contracts to focus on health promotion. Further, as contract environments mean that essentially purchasers make choices and decisions on behalf on consumers, the question is raised as to whether they purchase for consumers or with consumers. Focus on purpose The literature in the US has been preoccupied with the development of survey methodology and instruments as well as attempts to understand the behavioural aetiology of patient satisfaction and is mostly written within an academic context. While there is now some good work on the construction of patient satisfaction surveys, the British material puts questions about the purposes and uses of patient satisfaction work at the centre. This work is also done in the context of publicly funded and managed health services. The organisational context of patient satisfaction surveys and how hospital use patient feedback and appraisal are key preoccupations. The British work on patient satisfaction surveys is discussed in a policy context. The development in patient satisfaction and consumer feedback and appraisal approaches has developed a great deal since the Centre for Health Economics at York described the scene as underdeveloped, although the Audit Commission found that much of this has not yet 120
translated into ordinary hospital practice. The Audit Commission argues for a more strategic approach to patient satisfaction work at the hospital level. The push for patient feedback activities has occurred in the context of more appropriate purchasing, better planning of health services, improved hospital management, and incorporation of a consumer perspective into medical audit. Consequently, the British literature provides example of the use of patient satisfaction surveys by hospitals for quality improvement, by purchasing authorities for service development and planning, and by the College of Surgeons for medical audit. Principal methods The British literature locates patient satisfaction surveys within a broader context of a range of consumer feedback mechanisms and raises a set of questions about when patient satisfaction surveys are the right tool. It also canvasses the need to combine methods to assess complex problems. The British material illustrates the point that consumer feedback in general will rely on a mix of qualitative and quantitative approaches. For many patients, patient satisfaction surveys are not very effective in seeking their views and experiences and qualitative work is more appropriate. Even where quantitative approaches such as patient satisfaction surveys are used, qualitative research both precedes and follows on from the quantitative work in order to convert the data into useable information about quality. To use patient satisfaction feedback for quality improvement, the ‘how many’ needs to be complemented by the ‘why’ and ‘how’. The role of consumers Much patient satisfaction work treats consumers passively, that is, as providers of information, which administrators and providers may or may not do something with. Patient satisfaction surveys can be quite impersonal, as Shirley McIver notes. Undertaken this way, they are not a good example of an approach which empowers consumers. They do not express the principle of consumer partnership in health, which the modern consumer movement has argued for, and which is the direction of change in health care delivery. The British literature discusses the role of consumers in a way that takes the debate beyond that of consumers as providers of information or which simply relies on market choices by consumers. In some of the literature, consumers are seen as having a role to play in deciding what should be evaluated - what the criteria are, in being informed about the results, and what is done as a result of the information obtained. Many of the reports and projects reviewed in this report have some consumer representation on their steering committees. Lessons for Australia In general, as the Audit Commission points out, there is a long way to go before all this really works on the ground. The Audit Commission report on the state of play in Britain is very similar to that in Australia; the need to develop greater expertise, greater support to those doing patient satisfaction work, more coordination at hospital level, greater commitment to acting on the results, and involvement of consumers at all stages of patient satisfaction work, including acting on the results. The same challenges confront us, in part reflecting a shift
121
towards incorporation and legitimation of a central consumer oriented perspective, and the need to develop the tools and strategies to effect this. A number of organisation have emerged in Britain which have now developed different kinds of expertise around consumer feedback and appraisal. Some of this activity, such as that by the King’s Fund, is more by way of initiating and disseminating ideas about what health services and hospitals can do. Some, such as the College of Health and the College of Surgeons, have developed an infrastructure for undertaking consumer appraisals of health services. This highlights the need to develop appropriate infrastructure for undertaking patient satisfaction work, to develop the level of expertise in Australia, and to consider the role which independent organisations based on a consumer perspective might play in undertaking consumer appraisal activities and research in joint activities with hospitals and purchasers (eg health departments, area health authorities, purchasing authorities, regions, private health insurance organisations). As the College of Surgeons patient satisfaction survey illustrates, patient satisfaction surveys need to consider the episode of care from a consumer perspective and not only the inpatient hospital episode. As casemix funding starts to come to grips with funding whole episodes, or at least broader episodes of care, and as COAG puts issues of continuity of care and chronic illness on the funding agenda, then the quality approaches need to broaden accordingly to deal with whole episodes of care. And as the same survey also demonstrates, the outcomes for consumers of hospital care are part of patient satisfaction with care. Finally, the British literature at several points raises vexed questions about whose views are sought in patient satisfaction surveys and the challenge of incorporating the view of those consumers who are most in need of good quality services, who are very ill, whose views are regarded as being difficult to obtain and those who are often at risk in the quality of their care.
122
CHAPTER 11: PATIENT SATISFACTION IN THE UNITED STATES The material collected from the United States mainly concerned the design of patient satisfaction surveys, as well some policy literature about the role of quality information in influencing consumer choice of health services. This reflects the positivist and pragmatic preoccupations of the social and behavioural sciences in the US. The chapter gives some information about two of the patient satisfaction surveys used by state health departments in Australia. Office of Technology Assessment In 1988, the US Congress requested the Office of Technology Assessment (OTA) to assess whether valid information could be developed and disseminated to the public to assess their choices of physicians and hospitals. This took place in the context of a market approach to the delivery of health care and against a background of policies designed to inject greater price competition into health care. The Director of the Office of Technology Assessment notes that ‘according to competition theory, consumers who are sensitive to both price and quality will bring these considerations to bear as they select health insurance and medical providers....Purchasers of medical care (individual consumers, employers, health insurers) need to know about differences in quality so that they can weigh quality as well as cost in making decisions (OTA 1988:iii).’ There was also a concern to ensure that financial constraints did not affect quality of care. The project was overseen by a panel, which included major stakeholders, key individuals who have contributed to the quality literature, and consumer organisations, such as the American Association of Retired Persons and the Public Citizen Health Research Group. The Report by the Office of Technology Assessment, Quality of Health Care: Information for Consumers canvasses a number of quality indicators which could be reported. They are assessed in relation to the following categories: • reliability, whether a measure of the same case will produce the same results on successive trials; • validity, whether a measure measures what it purports to measure, and • feasibility, whether it is practical to use an indicator to convey information to the public there is also some canvassing of the potential for perverse incentives to be created. The Report is based on a systematic review of literature, based on Medline and Healthline data bases, as well as some unpublished material. The objective was to evaluate each of the indicators in respect of the above criteria as well as some analysis of the results. The report uses Donabedian’s triad of structure, process and outcome to set up a number of indicators. The Report argues that all are necessary to the assessment of quality. The structure of care describes the potential for good quality care, but cannot guarantee it and it is necessary to demonstrate the link to outcomes. Some of the processes of care have validity only to the extent that research can establish that they improve the outcomes of care. 123
Outcomes are only a valid measure of the quality of care, if they can be reliably attributed to the prior medical care. The Report argues that process and outcome measures are not alternatives as quality indicators, but complementary (OTA 1988:9). Patients’ assessments, in their view, relate to the structures, processes and outcomes of care. The Report notes that patients differ quite markedly in the outcomes they desire, making different assessments of the trade offs between pain, suffering, longevity, illness and mobility and their assessments of the personal value of intervention. The rationale for seeking input from consumers, they argue, is based on three arguments: • it ensures that evaluations represent the values of individual consumers of medical services; • patients are the only source of confirmation about aspects of the patient-provider relationship (especially some of the interpersonal aspects) and may also provide information that supplements other sources, such as the medical record, and • it is no more costly, and often cheaper, than data from other sources (OTA 1988:231). The key findings of the report in assessing the use of consumer assessment as an indicator were that there were a number of strengths in the use of patient assessments: • patients ratings are a valid indicator of the quality of interpersonal aspects of care and of patients’ satisfaction with physicians’ and hospitals’ inpatient care; • patients’ assessments relate to good and poor care and to access, and • patients’ ratings and reports of technical aspects of care are promising as quality indicators, especially for physicians’ ambulatory care, but they had not yet been validated (OTA 1988:15). However, the weaknesses were: • adequate data collection methods and instruments have not been developed and standardised; • potential biases in assessments may result from patients’ preferences or other characteristics, and • special surveys are required to collect data (OTA 1988:15). The Report is critical of the ways that were available at the time for obtaining patient assessments, particularly for inpatient care. Patient assessments of quality often represent considerable aggregation of various aspects of quality, such as global measures of satisfaction, and no coherent taxonomy of quality of inpatient care from a patient perspective existed (OTA 1988:231). Further, global measures of satisfaction do not provide any clues to
124
which aspects of quality are responsible for the differences in rating (OTA 1988:247). Their literature review incidentally found no support for the proposition that patient assessments of quality are determined by the amenities (OTA 1988:247) - contrary to the assumptions that seem to underlie a whole generation of hospital patient satisfaction surveys and which have brought patient satisfaction surveys into disrepute with consumers. The Report distinguishes two approaches to patient assessment, patient ratings and patient reports. Two of the US patient satisfaction instruments developed since this report was done and currently in use in Australia, the PJHQ survey (or HCA survey), and the PickerCommonwealth survey (see chapter 7) are each based in these two different principles, with the PJHQ survey being designed on patient ratings and the Picker-Commonwealth based on patient reports of what happened. The Report discusses some of the available evidence about biases in patient ratings. As reported elsewhere, older patients tend to rate their care more favourably than younger patients (OTA 1988:237). It is not clear from the literature what the significance of this is, but the finding is persistent across national boundaries. It is also suggested that people consider it socially desirable to have a good doctor and this inflates their assessments. The Royal College of Surgeons’ study alludes to this contributing to the ‘gratitude response’ that leads to high patient satisfaction ratings. The wording of questions can also bias results, if the attitude questions are skewed towards favourable descriptions as this is shown to bias results, particularly for patients from lower socio-economic groups. These issues are a challenge for the design of questionnaires. There is further discussion of these issues in chapter 9. At the time of publication, the OTA Report argues, the standard of valid and reliable published survey instruments was very poor and the Report argued that it was important for more developmental work to be done. At the time, the Hospital Corporation of America, Harvard Community Health Plan and the Rand Corporation were developing an instrument (the Patient Judgments of Hospital Care) that attempted to deal with the dimensions of hospital care that mattered to quality and reflected patient values (see next section). The OTA Report suggests that given people often liked to talk about their medical care, good response rates were possible - in the vicinity of 70% and higher. Lower samples, it argued raised questions of sample bias and were the result of poor follow up. Given the importance of measurement and patient sampling to the results, the Report recommends (OTA 1988:247) that if comparisons are to be made between physicians and hospitals, they must be based on: • standardised survey instruments; • standardised data collection procedures (eg personal, telephone, or postal questionnaire); • standardised survey methods (eg timing of administration), and • representative samples. The role of consumers 125
As indicated earlier, the Office of Technology Assessment Report was undertaken in a particular policy context, and the assumptions that underlie it are based on a market based approach to the role of consumers. This assumes that consumers will affect quality of care through rational market-type choices, based on the model of an informed consumer. There is evidence to suggest that consumers do make choices about the choice of clinician, usually based on information from family and friends. In the area of childbirth, these choices are increasingly researched by consumers. The literature reviewed by the OTA suggests that technical competence and interpersonal aspects are elements in decisions about choice of clinician. The Health Issues Centre (19920 has argued that there are inherent limitations to such market-based approaches. They include the following: • people bring unequal resources, financial and otherwise, to markets; • the choice of hospital is illusory, as it depends on where the clinician, or speciality, is available; • with public hospitals, it is often the local hospital that people are sent to particularly in regionally based hospital systems based on specialised functions; • in the case of emergency, it depends which hospital has a bed and is not on bypass; • ‘voice’ rather than ‘exit’ (Hirschman, 1970) may be the more appropriate option in many circumstances, and • imperfect information remains a feature of the medical relationship. Nevertheless, Health Issues Centre (1992) has argued that the focus on patient information is a great strength of this approach, that it is essential to those choices which consumers are able to make and that it is a key aspect of public accountability and consumer voice. The ability to exercise ‘voice’ (having an active say) as well as ‘exit’ (being able to change services) is assisted by publicly available information. The Office of Technology Assessment also suggests that patient assessments are critical elements in assessments of quality, particularly of interpersonal aspects of care, and perhaps with a role to play in technical aspects of care. In this respect, the other important features of the Office of Technology Assessment Report are its considerations about making information available to consumers. It suggests that the following elements are necessary: • • • • •
stimulate consumer interest in the quality of care; provide information that is easy to comprehend; use many media and formats to present the information; use respected sources to interpret the information; make the information readily accessible, and
126
• provide consumers with the skills to use and physicians the skills to provide the information (OTA 1988:10). The report comments on the complexities of this area, the potential sensitivity and public impact of such information, and the lack of evidence that consumers do indeed make choices based on published quality information. Important elements in its discussion are that it is not enough simply to publish technical information. How the information is interpreted and the credibility of those who interpret the information matters. Consumers need to consider more than one quality indicator, and the publication of trend lines is important. The Report suggest that it may well be that it is the responses of providers, organisational purchasers, and consumer advocacy groups to quality information that is the most productive in encouraging providers to review their performance (OTA 1988:12). Notwithstanding the limitations and precautions, it is clear that many consumers will continue to require more information about their health care and that there are questions about fairness, informed decision making and public accountability that make these issues important ones to sort out. The point made by the OTA is that it needs a deliberative strategy. Patient Judgments of Hospital Quality This is the patient satisfaction survey used in NSW, in Queensland accident and emergency, and proposed in SA. It was developed over a 6 month period and based on a study funded by the Commonwealth Fund. It was developed by the Hospital Corporation of America, Harvard Community Health Plan and the Rand Corporation. It is sometimes known in Australia as the Hospital Corporation of America (HCA) questionnaire. The study is reported in a range of articles in Medical Care (Supplement, September 1990). The contributors are Paul Batalden, Donald Berwick, Ron Hayes and John Ware. The aims of the study were to describe what a hospitalised patient experiences and how hospital performance can be evaluated in ways that reflect patient concerns. Three methods were used: • a literature review of published studies of patient assessments of their inpatient experiences and the means used to determine this. Interestingly, despite the volume of US literature, the researchers found it dated and that little of it focused on inpatient care, and raised questions about its relevance. The work of Raphael and Cartwright in the UK was the main literature to explore patient concerns, but was 10 years old; • a content analysis and taxonomy of patient concerns. Focus groups were conducted and verbatim comments collected from questionnaires, and • a pilot study in 10 hospitals. The instrument that was developed is a self-administered mail questionnaire that aimed to be comprehensive enough to reflect consumer concerns, but brief enough to elicit a good response rate: where the subscales independently and closely related to patients’ overall ratings of quality of care, and which discriminated between hospitals. The questionnaire also attempts to gauge the patient’s behavioural intent towards the hospital.
127
The sections of the questionnaire are: 1. 2. 3. 4. 5. 6. 7. 8.
Admission Daily care in hospital Nursing care Medical care Other hospital staff Living arrangement and the hospital environment Discharge Billing.
The pilot found support for the hypothesis that patients do assess the different components of hospital care. The crude response rate for the pilot was 65% and the overall adjusted response rate (allowing for exclusions) was 71%. Both telephone and mail methods were trialed and the response rates were much the same. There was, however, substantial variation in response rates across hospitals for both forms of administration, with uncertainty about the reasons for this (Nelson et al.. 1990). Ware and Berwick (1990:S40) suggest that a good response rate is possible (60-70%), if follow up methods are used to boost response rates, and that the sample size will depend on the analysis you want to make. The questionnaire covers eight components: • • • • • • • •
descriptive information on the patient’s admission and stay ratings of multiple aspects of hospital care summary of the quality of care assessment of health changes during the patient’s stay ratings of satisfaction with care intention to return to the hospital and recommend the hospital to family and friends open-ended questions demographic details of the patient.
While the questionnaire showed up differences between hospitals, it is not able to say why these exist or what the significance of these differences are (Ware and Berwick 1990:S41). McIver suggests that the questionnaire performs better at allowing hospitals to be compared than finding out patient view’s so that quality in individual hospitals can be improved from the patient’s perspective (McIver 1992:65). The PJHQ was developed by the authors to be available in the public domain and was part of the effort referred to in the Office of Technology Assessment Report to develop standardised instruments for patient assessment. It is available from the Medical Outcomes Trust.
128
Medical Outcomes Trust The Medical Outcomes Trust was set up in 1992 as a non-profit public service organisation to provide state-of-the-art technology for health and health care assessment. It has the following objectives: • assure the development of high quality health outcomes measuring instruments; • assure the widespread availability of a comprehensive library of health outcomes measuring instruments; • achieve standardisation of health outcomes measuring instruments; • advance applications technology for ease of use and interpretation, and for greater practical value in everyday medical practice; • integrate outcomes measurement with appropriateness, practice guidelines, and utility information in continuous quality improvement systems. The Trust makes a number of instruments, whose scientific standards they guarantee, freely available in the public realm, although a handling charge has now been introduced. The package includes a copy of the questionnaire, protocols for use, scoring instructions, reprints of published articles, a bibliography and other relevant information. The Medical Outcomes Trust distributes the Short Form 36 (SF-36) and the SF-10 for measuring health outcomes and asks that any use of it be registered on their data base. It also distributes some patient satisfaction instruments, primarily the HCA Patient Judgments of Hospital Quality (PJHQ), the HCA Patient Hospital Report Card, the Group Health Association of America (GHAA) Patient Satisfaction Survey, the Patient Satisfaction Questionnaire (PSQ) - a tool for studies of the general population, and the HCA Medical Staff Judgments of Hospital Quality Form. The Picker-Commonwealth survey This is the patient satisfaction survey used in Victoria. It was developed by Cleary and his colleagues at Harvard for the Picker-Commonwealth Program for Patient Centred Care. The program uses the term ‘patient centred care’ to mean an approach that consciously adopts the patient’s perspective (Gerteis et al. 1993:5). The purpose for conducting the survey was: to assess how patient centred hospital care was in the US; to assess whether some patients were more at risk, in particular, those who were poorer, sicker, older or without medical insurance; and how much patient- centred care varied from hospital to hospital (Gerteis et al. 1993:12; Cleary et al. 1991:254). The survey was developed using focus group research, which talked with discharged patients about their hospital experiences. This was followed up with a brief questionnaire based on the focus group discussions and used as the basis for telephone interviews across a broader sample. Family and friends of patients were also interviewed. Separate focus groups were
129
conducted with physicians and other hospital staff and a literature review was conducted of pertinent literature (Gerteis et al. 1993:5). Seven primary dimensions of patient care were elaborated (Gerteis et al. 1993:5-11): 1. Respect for patients values, preferences and expressed needs: • quality of life, • involvement in decision making, • dignity, • needs and autonomy. 2. Coordination and integration of care: • coordination and integration of clinical care, • coordination and integration of ancillary and support services, • coordination and integration of ‘front line’ patient care. 3. Information, communication, and education: • information on clinical status, progress and prognosis, • information on processes of care, • information and education to facilitate autonomy, self-care and health promotion. 4. Physical comfort: • pain management, • help with activities of daily living, • surroundings and hospital environment. 5. Support and alleviation of fear and anxiety: • anxiety over clinical status, treatment and prognosis, • anxiety over the impact of illness on self and family, • anxiety over the financial impact of the illness. 6. Involvement of family and friends: • accommodation of family and friends, • involving family in decision making, • supporting the family as care giver, • recognising the needs of the family. 7. Transition and continuity: • information, • coordination and planning, • support. The Picker-Commonwealth survey, which was constructed from these dimensions, attempts to elicit reports from patients about the aspects of care that they perceive to be important (Gerteis et al. 1993:11). It does not use patient satisfaction rating scales, in an attempt to minimise the influence of confounding variables such as patient expectations, personal relationships, gratitude, age, gender or ethnicity. Whereas the HCA survey asks a patient to rate ‘the courtesy and helpfulness of your doctor’, the Picker /Commonwealth asks, ‘Were you told about the purpose of your medication in a way you could understand?’ (Cleary et al. 1991: 255). It relies on patient reports rather than patient ratings.
130
After pilot testing, the survey was conducted across the US using a stratified random sample, from which were selected a random sample of 100 patients per hospital. Hospitals with less than 100 patients, for-profit and long stay hospitals were excluded from the sampling. Telephone interviews were conduced three months after discharge. Of 141 hospitals selected, 62 agreed to participate. Participating and non-participating hospitals were compared for characteristics. The patient sample was compared with the overall profile of patients discharged from short stay hospitals. A sample of results is as follows (Cleary et al. 1991: 259-60): • 39% reported that they did not have a relationship of trust with any other hospital staff other than the physician in charge of their care; • 45% of patients said that they were not told about daily routine in the hospital; • 28.4% said that nurses were overworked and too busy to take care of patients; • on discharge, 36.5% were not told what the patient should or should not eat, 30.2% not told of important side effects of medication, and 26.5% not told what danger signals to watch out for. Perceived poor health status led to average problem scores nearly twice as high as patients who reported excellent health. Poor patients and patients of colour also reported higher problem scores. There was some effect of gender, but income and health status were more pronounced. Regression analysis found that health status was the strongest predictor of problems occurring, and being poor and the interaction of being poor and in poor health were strong, significant predictors of problems occurring. However, only 9% of variance was explained by patient characteristics (Cleary et al. 1991: 261-3). Analysis by hospitals found some distinct patterns of differences, with high and low performing hospitals consistent across score areas. This led onto one other noteworthy outcome of the survey, which was to identify the characteristics of the more patient-centred hospitals. Interdisciplinary teams and consultants visited those hospitals with a view to establishing the management practices and organisational characteristics affecting patient centred care (Gerteis et al. 1993:12). Those characteristics are reported in Through the Patient’s Eyes (Gerteis et al. 1993). This book sets out to provide hospitals with information gleaned from the survey and the literature and contains practical suggestions about what steps and activities hospitals can undertake to provide more patient-centred care. Almost all the top performing hospitals were non-teaching hospitals, many of the lowest performing hospitals were urban academic centres, and the worst performing hospitals were often public academic health centres. There is potentially some very interesting information here. Gerteis et al. (1993:232-3) suggest that some hospitals present greater challenges for patient centred care, in part because of the nature of demand on them, from the nature of illnesses presenting and from the more complex class and ethnicity mix of the patient population as well as from the complex mix of hospital and academic authority lines. That is, it reflects the external environment, as well as the hospital culture itself. This suggest that
131
comparisons of hospitals needs to be undertaken with some caution and gives some further support to the idea that casemix counts. This Picker-Commonwealth survey has been used by McMaster University in Canada to survey Canadian patients, and in the UK, with some modification for the NHS, by the Department of General Practice at St. Mary’s Hospital Medical School and Social and Community Planning Research (Bruster et al. 1994). The UK study also used a random sampling of hospitals and patients. Of patients approached, 86% agreed to participate. The main non participation was by hospitals. It is interesting to note the reasons: too many surveys being conducted locally; ward closures and building work; ethics committee refusal; disagreement with the methodology (the questionnaire was considered by one hospital to be too taxing for the patients). Like Cleary and his colleagues (1991), Bruster and his colleagues. (1994) consider that their sample is likely to be representative, but if anything, likely to underestimate problems. It needs to be noted that some categories of patients, eg psychiatric and geriatric patients, were excluded from the sample. The UK study identified problems around communication, pain management and discharge. The results were compared with the standards of the Patient’s Charter and found that the standards were rarely met, eg 46% of patients did not know that they had the right to look at their health record, and 51% of those worried about leaving hospital thought they had not got help from staff. (Bruster et al. 1994:1543-4). The UK study did not find significant differences by income as the US study did. Compared with US and Canadian results, however, more problems were reported (Bruster et al. 1994:1545). Public Citizen’s Health Research Group The Public Citizen’s Health Research Group is a non-profit consumer advocacy organisation devoted to research and education on health care issues. It does not appear to carry out the type of research relevant to this report. However, it has an active interest in the mechanisms used in the United States to monitor quality of health care and the information that is available to consumers and in that respect is pertinent here. A study published in 1994 examined the question of who oversees the care that hospitals provide, making sure that patients are well treated and safe and how well this was met by deferring decisions to the Joint Commission On Accreditation Of Health Care Organisations (JCAHO). Their survey found reservations by state officials about relying on JCAHO to license hospitals. The Public Citizen’s Health Research Group concluded that: • JCAHO was subject to inherent conflicts of interest due to its governance, funding and multiple roles. • JCAHO was not bound by - and did not voluntarily observe - the same standards of public accountability required of government agencies.
132
• While government reliance on private accreditation appeared to save public funds, the public still payed the bulk of hospital survey costs and overall health expenditure remained the same. Summary This chapter has already noted the different policy context in which these developments have occurred, in particular the reliance on market mechanisms. This shapes health policy and consumer organisations in particular ways, leading to a focus on information and regulations. This context is undergoing change with the shift to managed care, with an expansion of the purchasing role of non-government organisations, both profit making and non profit making. However, there is an interesting focus of the provision of consumer information and the concept of the public citizen, with some consideration about how public information for health consumers is used effectively. This draws from legal and market approaches to consumer empowerment (see chapter 1). The major contribution and the major pre-occupation is methodology. The two questionnaires discussed in this chapter are used by state governments in Australia and both have good credentials. Both were designed at Harvard and were funded by the Picker-Commonwealth program. The two questionnaires, however, have different design principles. The structural concepts are as follows. PJHQ (HCA) Questionnaire
Picker-Commonwealth questionnaire
1. Admission
1. Respect for patients values, preferences and expressed needs
2. Daily care in hospital 2. Coordination and integration of care
3. Nursing care 4. Medical care
3. Information, communication, and education . 4. Physical comfort
5. Other hospital staff 6. Living arrangements hospital environment
and
the 5. Support and alleviation of fear and anxiety
7. Discharge 6. Involvement of family and friends 8. Billing. 7. Transition and continuity The PJHQ (HCA) questionnaire is a rating instrument and attempts to measure satisfaction by rating satisfaction with particular aspects of care. The Picker-Commonwealth is a reporting 133
instrument. It attempts to avoid some of the methodological problems with the concept of satisfaction (see chapter 9) by asking patients about what did and did not happen. In a project in the UK, this was extended to include questions about the standards in the Patient’s Charter. Both were designed using focus groups with patients. The PJHQ (HCA) questionnaire was developed with a view to either telephone or postal administration. The PickerCommonwealth was designed as a telephone survey. The HCA is the simpler instrument, but McIver (1992:65) suggests that the PJHQ (HCA) questionnaire works better at allowing hospitals to be compared than providing information to assist individual hospitals to improve care from a consumer perspective. The PickerCommonwealth appears to provide more detail. In the spirit of benchmarking, further constructive use of the Picker-Commonwealth survey was the analysis by Gerteis and her colleagues (1993) of the results to construct profiles of the best and worst performing hospitals, followed by team visits to explore the organisational features that led to best performance. As in the UK (chapter 10), the Picker-Commonwealth program and the Medical Outcomes Trust, points to the importance of some infrastructure support for patient satisfaction work.
134
CHAPTER 12: WHAT IS BENCHMARKING? Benchmarking is described by Spendolini (1992:9) as a method of organisational improvement that involves ‘continuous, systematic evaluation of the products, services and processes of organisations that are recognised as representing best practices for purposes of organisational improvement’. It is part of the array of new organisational approaches which cluster around quality improvement. Three types of benchmarking are distinguished in the literature: 1. internal benchmarking which involves benchmarking between operations in a business; 2. industry, or competitive benchmarking which involves benchmarking between enterprises in the same product market, and 3. generic or process benchmarking which involves benchmarking between businesses in different industries that share common processes (Macneil et al. 1994:14). Benchmarking is outward looking. It looks where-ever, within Australia or overseas, to find organisations that do particular things well. Benchmarking is a tool aimed at driving change and undertaken in order to produce change. It is an organisational strategy that needs to be embraced throughout the organisation. In benchmarking, targets are set that are continually moving and changing as the organisation looks for change directed at improvement. Benchmarking is about process A number of overseas and Australian authors on benchmarking emphasise the point that benchmarking is a continuous process, concerned more with an understanding of best practice processes than the numbers that prove what best practice is at any time (Macneil et al. 1993:2). It is an analytical tool to assist organisations to improve the way they do things. Robert Camp, who introduced benchmarking at Xerox, has commented that 90% of Xerox’s benchmarking is process benchmarking (Macneil et al. 1994:29). Camp argues that in the benchmarking approach, the danger is that: ‘measures are overemphasised and processes are overlooked’ and also argues that ‘what without how is an empty statement’(Camp 1994:17). Camp then goes on to comment that ‘concentration on the measure is really an empty statement until it is traced back to the practice - the best practice - in the process that achieved the performance’ (Camp 1994:17). In the same vein, Spendolini (1992:4) argues that: with benchmarking, the focus is extended beyond the scope of the finished product or services to concentrate extensively on process issues. The emphasis (is) not only on what another organisation produced but also on how the product or service was designed, manufactured, marketed and serviced.
It is this approach that distinguishes benchmarking from comparative performance measurement.
135
In other words, benchmarking is useful when it identifies the processes that lead to best practice. As Macneil and her colleagues (1994:190) argue in Benchmarking Australia, ‘good benchmarking analysis is understanding what factors, process methods, practices, skills, systems, technologies, resource distribution, and so on enable best practice.’ It is this type of information that is able to inform changes in organisations. Benchmarking is about finding best practice The crux of benchmarking is ‘choosing what you want to benchmark, and then finding the best practice against which to compare it’ (Macneil et al. 1994:206). Camp (1994:18) argues that it is only by finding better practices through benchmarking and incorporating them in the work processes that the desired result will be achieved. Benchmarking is not just about being compared against the industry average. One type of approach to benchmarking takes the form of publication of industry data, with standardised performance information so that organisations can be compared against the rest of the sample. This begs the question of what best practice is. While organisations might get some information about other organisations’ outcomes, this approach says nothing about how organisations got there (Macneil et al. 1994:159). This is comparison of performance data; it is not really benchmarking. The whole point of benchmarking is the move beyond knowing how other organisations perform compared to yours, to understanding why their performance is different. The Commonwealth Best Practice Program’s Benchmarking Self Help Manual (1993:3) emphasises a similar point about what they call ‘data base or number fixation’: Databases can be useful, and performance indicators and targets are essential aspects of any well managed benchmarking approach. However, they will only be used effectively if combined with a change program based on shared and detailed understanding of the practices and structures which produce exemplary performance levels. Furthermore, because of problems arising in making genuine ‘like with like’ comparisons, numbers tend not to be particularly effective drivers of change.
This raises questions for patient satisfaction. What is best practice in consumer focused, or patient centred care? Put another way, which good practice in hospital care reflects consumer priorities and perspectives? It is not enough to publish league tables of hospitals scores in patient satisfaction, or even to compare individual hospitals with the national or state average. This is not benchmarking. It may have the effect of breaking ‘paradigms of comfort’ by highlighting shortfalls in performance (Mcneil et al. 1994:113-114), but it is not benchmarking until it leads to an understanding of what the best performing hospitals, clinics or wards do, which improves consumers’ experiences of particular aspects of their hospital care. Through the Patient’s Eyes (Gerteis et al. 1993), discussed in chapter 12, provides an example of an approach that would lend itself to benchmarking. Here the patient satisfaction survey was undertaken in the context of the Picker-Commonwealth Program for Patient Centred Care, that is, it was undertaken as part of a process of improving models of care. Analysis of the results was undertaken to try to understand what characterised the most
136
patient centred hospitals and what were the organisational and management practices that contributed to this. Profiles of hospitals were constructed from the results. The analysis of the results and the resultant hospital profiles were the first step only. This was followed up with visits by interdisciplinary teams to those high performing hospitals to try to identify those management practices and organisational characteristics which led to patient centred care. This approach is consistent with the idea that the benefits of best practice lie in the opportunity for organisations to learn from concrete examples of good practice (Macneil et al. 1994:81). Who decides what are the criteria that establish ‘best practice’? Deciding what to benchmark on is another critical step in benchmarking. Macneil and her colleagues (1994:90) argue that stakeholder analysis is a key step in the process, and that it is essential to understand stakeholder needs. They suggest that this is particularly vital in the public sector where the stakeholders include both consumers (or clients) and citizens more broadly as well as governments. They see genuine consultation as a key issue. This is where the literature on benchmarking needs to be extended. As with the literature on quality management (see chapter 1), the literature on benchmarking is mostly derived from manufacturing. There is not a lot of literature on the service sector, or the public sector, or on the health sector. What there is focuses on benchmarking of human resources. There are examples of best practice suggested in the literature reviewed for this report. One example of the way consumers might set up the criteria for ‘best practice’ can be seen in the following extracts from the review of consumer literature undertaken for the NH&MRC Working Party on the Treatment of Diagnosed Breast Cancer (see chapter 2 and appendix). The literature review concluded that the model of care of breast cancer most consistent with the needs and preferences articulated by women is one which: • provides continuity of care; • has a team approach enabling a management plan to be determined and implemented in a coordinated fashion, taking into account a range of clinical and psycho-social factors; • offers psycho-social support; • ensures women's access to relevant health professionals from one location; and • emphasises liaison with community supports such as general practitioners and community nurses. The literature review also concluded that women prefer a treatment context in which: • their work and family responsibilities are acknowledged and accommodated as far as possible; • there is an holistic approach to care; 137
• waiting times at appointments, for treatments and receipt of results are minimised; • their dignity, privacy and confidentiality are respected; and • attention has been given to amenity. In the area of communication, the review concluded that women appear to prefer communication styles and patterns which: • acknowledge the emotional aspects of breast cancer and its treatment; • convey messages in a positive but accurate fashion; • convey friendly interest in the woman as a person; and • cultivate a positive attitude. Other examples from the literature reviewed here include the APSF research which was positive about the models of care offered in the treatment of diabetes. The literature from consumer organisations suggests versions of good practice in health care. Much could be gained from understanding them and having them promoted. These consumer reports point to the multi-faceted nature of good practice care and highlight the potential difficulties of subsuming these issues under numerical values of satisfaction. This discussion of benchmarking highlights also a point made in earlier chapters. The better patient satisfaction surveys say something about where patients are not satisfied. They do not, however, usually convey information about what can be done to do it differently, or what the features of hospitals are which perform better on these aspects of care. This is not always just about the clinical relationship, but, according to the consumer literature, may be about better communication between health professionals. One of the features of the benchmarking literature is the focus on identification of critical points for improvement. Put another way, patient satisfaction research would benefit from taking a strategic approach. The next chapter provides a summary of the findings of the review of consumer research undertaken for this report, and it indicates themes that are consistent with the breast cancer treatment example given here. These relate to the relationship between clinicians and consumers. as well as to ways that hospitals and other health services are organised. The consumer literature goes beyond listing dissatisfaction to suggest some features of hospitals that would lead to better care from the consumer perspective. Summary There are several points here in this discussion of benchmarking that are consistent with points made elsewhere in this report:
138
• the purpose for undertaking consumer feedback activity needs to be articulated and owned by hospitals as part of their overall organisational strategy; • the processes that produce patient satisfaction are more important to understand than producing a league table of patient satisfaction scores; • the measurement of patient satisfaction is not an end in itself; • the component elements that contribute to patient satisfaction or dissatisfaction are the key elements to concentrate on, rather than overall measures of satisfaction; • if an organisation wants to use consumer feedback to drive a process of change, then it is important to understand dissatisfaction, or things that don’t go so well, and to use methods that will give accurate and reliable feedback on this, as well as ideas about what can be done to change these elements of care. Simplistic patient satisfaction surveys will not accomplish this; • stakeholders, which includes consumers, need to be part of the processes for identifying what needs to be benchmarked; • benchmarking is not a simple process of comparing performance scores on a state/territory or national basis, but a process of identifying best practice and seeking to learn from it; • neither the Commonwealth or the states/territories can set targets for consumer feedback, but it useful for them to undertake the activities that will prompt and facilitate hospitals to seek to identify and benchmark against good practice.
139
140
PART THREE
OPTIONS AND RECOMMENDATIONS
141
142
CHAPTER 13: PATIENT SATISFACTION AND HOSPITALS: OPTIONS AND RECOMMENDATIONS The following two sections briefly restate some of the findings of the chapters on statewide patient satisfaction surveys and consumer research on consumer experiences of hospital services. The recommendations and options need to be considered against this summary discussion and the preceding chapter on benchmarking. STATE SURVEYS: SUMMARY OF FINDINGS State government patient satisfaction surveys vary in the ways they are undertaken. There are differences in methodology, administration, and sampling. Direct comparisons between their results are not valid. Little is known on what is done with the results. The main documented change in hospital is some reductions in waiting times (NSW 1994 survey). In Victoria, hospitals were reported to be reluctant to act on the results from the initial pilot. The public dissemination of results varies, with NSW releasing overall results publicly, but not hospital based results. In Victoria, the release of results at any level is still very uncertain. The questions in surveys are rarely linked formally to, or analysed with respect of, standards or entitlements as per the Medicare state charters. There is usually little attention to outcomes as consumers see them. In the smaller states, there is a confused overlap between the statewide surveys and those done by major public hospitals in those states, which raises an issue of the role and purpose of the statewide surveys. The level of professionalism in bigger states has increased with the advent of state-wide surveys, which can be further built on. There is much rediscovering of the wheel and little sharing of information and expertise. The major patient satisfaction surveys have been done by private firms, with one firm currently dominating the field. There is little research being done in this context beyond the pragmatic refining of instruments. There is no reporting by casemix, or sampling which would allow such reporting. We have covered arguments which suggest that this affects particular aspects of satisfaction. The 1995 WA survey shows up some differences between teaching and non-teaching hospitals, as does the Picker-Commonwealth survey in the US. These are probably, at least in part, related to overall casemix and external demands. There is no knowledge of non-respondents. Current consent arrangements in some states make any follow up research on non respondents impossible. In some state surveys, the response rates are unacceptably low, follow up is not done to improve response rates, and we have suggested that this is problematic. Those who do not respond may be those for whom considerations of equity and poor health status apply. The data on people from non English speaking background and Aboriginal and Torres Straight Islanders is not adequate. Only the
143
Birthing Survey in Victoria attempted to reach representative numbers of important subgroups, eg young women and this was the only survey clear about its population sample and its failure to obtain representative numbers in certain subgroups. There are significant omissions, eg children, adolescents and those with psychiatric illnesses. The surveys are unlikely to get the views of those who are very ill. Family members and carers are not surveyed either. The experiences of people who die, particularly in relation to the humaneness and technical competence of their care, are an important feature of hospital life, and a group for whom patient satisfaction surveys are quite inappropriate, but whose experiences, and those of their intimate family and friends, matter to the quality of hospital care. CONSUMER RESEARCH: SUMMARY OF FINDINGS This report has noted that overall in Australia, research on consumer experiences and perspectives is not extensive, particularly that undertaken by professional researchers. However, consumer groups have an important history of undertaking research on people’s experiences of health care. However, research on the hospital as an institution and care within its walls has not been a major focus until recently. Consumer research has tended to focus on policy issues of the day, reflecting availability of funding, and has been sometimes limited in its methodology by resource constraints and the small size of such organisations. Nevertheless, there is now some expertise built up, and research by consumer organisations is beginning to address this important gap. There is much to learn from analysing the accumulated results. Consumer research is mainly based on qualitative research, often focus group research. The research focuses more on people disadvantaged in some way, and on population groups. The recent, and better resourced work, has sampled views around the country, but research is usually based around the constituency of the organisation. Similar issues emerge as in the more expensive patient satisfaction surveys. There is more attention to ‘casemix’ type issues than in the satisfaction surveys, that is, care may be considered to be better for people with a specific condition than for others with different conditions. This is because consumer organisations are often formed around people with conditions in common, or life cycle stage, eg older people. The common themes, indicating the problem areas, which emerged from the consumer research findings are: 1. There are several related issues which cluster around the general mantle of: • communication, • being treated with respect, and • being involved in decision-making.
144
Sub-issues related to these are: • informed consent, • privacy, and • access to interpreters. 2. People want more information on all aspects of the hospital stay, and in particular on pharmaceuticals. They want information in verbal and written forms, and in other languages. 3. Discharge planning is frequently inadequate for all types of patient. There is little involvement of family or carers either. 4. The role of the professional team is critical. People are more positive about: • multidisciplinary teams, • good teamwork, and • communication between professionals. This includes communication between professionals in and outside the hospital. A related sub-issue for public patients is seeing the same principal doctor, within a team, ie continuity of care. 5. For some people, access to hospital care is important. This includes:• the cost of, or the lack of public transport, • they are not informed about waiting, • they can’t get in for some reason. Note that there is little currently in this literature about safety (adverse events) and outcomes, although these issues are often voiced in consumer circles, particularly by individuals. This suggests that it is the research or consultation frameworks which tend not to highlight these issues and not the absence of consumer concern. Many people are drawn to consumer organisations because of concerns about the outcomes of treatment. It is worth noting that the various Health Commissioners’ Reports indicate that treatment issues remain the most significant main subject of complaint. This category includes wrong or inadequate diagnosis, and inadequate or wrong treatment. METHODOLOGY: SUMMARY OF FINDINGS There are six major concluding points which emerge from this review of key issues in the literature: •
There has been a shift in thinking about the role of the consumer as patient. It is more likely now that researchers want to know what consumers think; that they accept that what the patient tells them is an accurate reflection of what happened; and that this information can be used to improve the quality of care.
•
Coinciding with this shift, and related to it, is a serious questioning of the conception of satisfaction as a unitary concept whose causal variables can be measured.
145
•
Researchers now are more in favour of using several research methods to inform their survey instruments, and gather data.
•
The methodological complexities of patient satisfaction research are considerable and should not be downplayed.
•
Stereotypes of consumers have been discredited. Future researchers should be wary of assuming that sub-groups of people think as they suppose, and
•
Certain groups of people, whose social position or state of health may make them vulnerable to poorer quality care, are extremely difficult to reach via the conventional questionnaire. These people remain the challenge of researchers and policy makers.
OPTIONS FOR A NATIONAL APPROACH TO CONSUMER FEEDBACK Three broad options are presented here for consideration, followed by a number of concurrent recommendations aimed at improving the expertise available and the quality of the patient satisfaction work which is undertaken. The options we discuss could each stand alone, but they are not mutually exclusive. It would be possible to pick up some elements of all three options, and to implement them in a staged sequence. One clear principle underlies our approach. Implementation of any approach is only effective if consumer feedback is part of hospital services development, whether this is at individual hospital level, or within networks of hospitals. In this respect, such research and use of consumer feedback will only be effective if there is local hospital/hospital network ownership of the activity. We have drawn all our options and recommendations with this in mind as the test of effectiveness. Once again, we reiterate that patient satisfaction surveys need to be undertaken in a context that goes beyond comparison of results to a context where hospitals use benchmarking, or other approaches, to establish what the processes are that lead to good practice. We have also been cognisant of the activity at state government level, covering quality and outcomes measurement. We have sought to avoid recommending duplication of effort and have attempted to draw out a national role for the Commonwealth, against the background of both state and hospital activity. Our concept of an appropriate national role is that of promoting the concept of consumer feedback, initiating developmental work into survey instruments and tools, resourcing measures to improve the quality of work on patient satisfaction, pulling information together at national level where this is relevant, and identifyng gaps in the information about particular groups of consumers as this report has identified.
146
Option 1. Report on comparative national data on patient satisfaction Issues The purposes of funders, purchasers, providers and consumers are different in their emphases and need to be disaggragated. The most useful information from the point of view of providers may need to have local relevance. The most useful information about dissatisfaction is that which also provides some clues about how to respond to it. The purposes of the Commonwealth are those of a funder who provide funds to the states, but is responsible for some overall national quality, equity and efficiency standards. The states act as purchasers of health services and one would expect that they specify quality standards in more detail, and have an interest in the processes that produce good quality care. Consumers have two sets of interest. As service users, they want to be able to make good decisions about their health. As citizens, and the ultimate funders of health services, consumers may want to be sure that health services perform in ways that accord with public values. Advantages 1. This would be seen to take consumer feedback issues seriously, as consumer organisations have advocated. 2. There is a lot of interest in the area of patient satisfaction surveys and some support for ideas of finding ways to benchmark patient satisfaction. Disadvantages 1. It is not clear what ‘satisfaction’ tells you. Satisfaction is a construct, not a given, and it is not clear what surveys measure. 2. There are inherent limitations in the methodology, and patient satisfaction surveys are not a simple or easy way to get at patient experiences and judgments. 3. If there were to be national reporting, methodology issues would need to be negotiated with the states/territories, who need in turn to negotiate these with hospitals. 4. There are issues about the purpose, the ownership of the exercise, and linking patient satisfaction to information that enables services to improve quality. While on the one hand, no one questionnaire can accommodate all the purposes discussed above, on the other hand, a confusion of multiple questionnaires can push the tolerance of the hospitals who administer questionnaires, as well as those who fill them in. 5. There are concerns about what is done with the information made available, and the purposes to which it is to be put needs clarification, eg to compare states, to compare hospitals, to compare hospitals within categories, to identify issues in health care delivery which need attention, or to give consumers information. It is the view of the consultants, based on the work undertaken for this report, that national reporting on an overall patient satisfaction score is not meaningful or reliable. Any reporting
147
is only of value if it is concerned with the various components of patient satisfaction or experience, assuming that such data is collected in a uniform way. If this option is favoured, there are two ways it could be done A. Common set of questions are asked in current statewide surveys Issues • The chapter on state wide surveys (chapter 7) raises questions about the feasibility of this approach, given the variation in methodology, administration, sampling and coverage of population groups and casemix. We do not consider that it is possible to insert a set of standard questions into the current questionnaires. There are a range of issues about whether ‘like’ is being compared with ‘like’. • This would require negotiation with the states to get relative conformity on a standardised survey instrument, standardised data collection procedures, standardised survey methods and representative samples. • There is no research yet done in Australia to evaluate the comparative power of instruments for Australian circumstances. Additionally, it is premature to foreclose the development of patient satisfaction approaches, given the under developed state of the art in Australia. • It would be necessary to negotiate what material is made public in order to get hospital cooperation, given variation across the states. • The states have different charters, and therefore the questionnaires may relate to different public standards. • On the positive side, there are the makings of an infrastructure and some experience, although it is tending to concentrate in one private firm. While this may have some value in the potential for standardisation, it has some negatives in that it does not encourage further Australian research into patient satisfaction survey methodology and uses. B. Development of a special purpose-built national survey, undertaken under the auspice of the Department of Human Services and Health, based on a population sample (of hospital users) Issues • This is our preferred way of implementing option 1, given the issues noted above, but it needs to be supported by infrastructure development. • The cost of collection needs to be considered against the value.
148
• It would still require negotiation with the states, and there are issues, such as Ethics Committee approvals to be considered. • An issue to be considered would be the potential for confusion with state based and hospital based surveys. • The Commonwealth Department of Human Services and Health could implement this by initiating a rolling program, focussing on different areas, eg day surgery, outpatients, casemix related information, patient information, communication, hospital discharge, availability of interpreters (as reported by consumers). • It is open to the Commonwealth to evaluate further the comparative contributions of patient reporting (HCA type) or patient rating (Picker-Commonwealth type) approaches to the information that would fit its purposes. • The data would then be used strategically to highlight issues of concern, rather than expecting that hospitals can use the data directly to improve services. • This approach would have the disadvantage that results would not be compared over time, as the program would move on, but on the other hand, it could provide a basis for hospitals, the Australian Council on Healthcare Standards or the clinical colleges, to initiate their own benchmarking exercises in the areas where shortcomings are identified. The Commonwealth could then report at an appropriate time span on what was done as a result of their surveys. Option 2. Identify quality issues highlighted by existing consumer and patient satisfaction research and report on the more measurable aspects This option would not see patient satisfaction reported in its own right. However, patient satisfaction reports and other forms of consumer feedback would be used to identify quality issues of concern to consumers and there would be national reporting on these. This could build on the summary of consumer issues earlier in this chapter. Hence, it would be performance on the quality issues themselves, not patient satisfaction, which is reported. For example, the consumer and patient satisfaction research identifies waiting times as an issue, which is already a feature of the National Hospital Quality Management Program. Others could be identified. Performance could be measured either through the identification of key structural arrangements (eg to ensure continuity of care) or by patient reporting (eg whether patients received timely information), depending on the issue. It could include issues which are easier to quantify, but also those better reached through qualitative approaches. If either of the other options is adopted, it would be possible to implement this option concurrently. Examples of what we mean are given below. Communication/ respect/ being informed
149
Indicators or benchmarks could be developed around the sub-theme areas, or where patients can report on things that did/didn’t happen: • being asked about treatment options • giving informed consent • access to an interpreter. Note that we have categorised access to interpreters as a component of a system of care that respects patients as persons and treats them humanely, and not simply as a matter of access. Information given • verbal or written or both • in other languages • on particular aspects, eg what would happen with treatment, on medication, on what to expect when discharged, how long to expect to wait etc Access • various waiting times, this could be disaggregated by age and condition • access by particular groups, eg people from non English speaking background, Aborigines and Torres Straight Islanders • rural access. Areas such as the timely provision of information, access, hospital discharge, and continuity of care would be amenable to such an approach. While we have indicated that there are some aspects of communication that that could be dealt with by such an approach, broader issues of communication may be more effectively dealt with at more local or specialised levels, eg by clinical colleges, hospital networks, health areas or states. The British Royal College of Surgeons’ patient satisfaction survey is a good model of such a strategy. Some of these issues would also be amenable to the partnership approaches with consumer organisations we discuss elsewhere. For very moderate outlays of resources, this could yield a lot of useful detailed information. If this overall approach were adopted, we recommend that it includes a focus on issues important to particular population groups, eg access to interpreters, rural access. There would need to be consultation with indigenous health workers to establish those national hospital indicators most appropriate to Aborigines and Torres Straight Islanders. Advantages 1. It would relate the National Hospital Quality Management Program to issues which consumers identify as important, building the program and its reporting mechanisms and indicators on issues which consumers consider to matter. 2. Many of the matters that are raised by the consumer research, eg professional teamwork and intercommunication, need to be addressed by other than patient satisfaction feedback. 3. It does not rely on the concept of patient satisfaction, which is problematic as a category.
150
Disadvantages 1. There is some inherent value in obtaining consumer views and this approach may be seen to not value such feedback in its own right. 2. This could still require some surveying of the patient reporting type and so retains some of the administrative complexity of patient satisfaction surveys However, it avoids the particular problem about the reliability of the concept of patient satisfaction by relying on reporting rather than rating. Option 3. Require hospitals to undertake consumer feedback but do not require national reporting of results The Commonwealth could legislate to require hospitals to obtain consumer feedback, with or without formal reporting on the processes or results. This could be done in the context of consumer charter development. Alternatively, such requirements could be negotiated with the states as part of the Medicare Agreement. In this approach, the Commonwealth as funder would have as a quality requirement that hospitals obtained and made use of consumer feedback as part of their quality management. States as purchasers would then be responsible for greater specification of those aspects of consumer satisfaction where hospitals needed to demonstrate improved quality or otherwise develop an appropriate policy framework for consumer participation in and appraisal of hospital care. If reporting were still desired, rather than reporting nationally on results as part of the National Hospital Quality Management Program, there could be reporting on process indicators, such as credible structures and processes for obtaining consumer feedback or for doing patient satisfaction surveys. Implementation of this option would require some further work to be done to specify the principles, components and indicators of adequate systems of consumer feedback. This could be based on the material covered in this report and a companion report in progress on consumer feedback into hospital quality management which will address the more qualiliative approaches. Additional work should include development of guidelines about good quality patient satisfaction surveys, appropriate ways of undertaking and using surveys, and ways of demonstrating the uses of information gained from consumer feedback. It should be noted that this approach would be inclusive of a range of measures for obtaining consumer feedback. It should not be assumed that this implies the necessary use of patient satisfaction surveys, although it could be expected that these might be one of the tools. This report has favoured a range of methods for obtaining consumer feedback. Implementation of the concurrent recommendations below would be essential if this option were to be effective. Advantages
151
1. This option does something to ensure that hospitals use consumer feedback as part of their quality management program without buying into the methodological and practical problems of obtaining comparative data at a national level. 2. Benchmarking is more appropriately undertaken between groups and networks of benchmarking partners, so this approach could be seen as leaving such activity to be undertaken at an appropriate level. 3. Best benchmarking works on benchmarking processes. Disadvantages 1. This could be seen as a soft option as it is very difficult to ensure that processes actually result in what they claim to provide. CONCURRENT RECOMMENDATIONS TO IMPROVE THE QUALITY OF PATIENT SATISFACTION WORK IN AUSTRALIA We were not requested to make further recommendations in this brief. However, we have indicated a number of areas where these should be considered. This arises out of the poor infrastructure available to support consumer feedback activity, including patient satisfaction surveys. As such, this series of recommendations is designed to accompany all of the above options and in our view are required to implement any of them. They are all recommendations about improving the infrastructure for consumer feedback and patient satisfaction activity. Without some improvement in the quality of patient satisfaction survey work, none of the options above will be effective at hospital level. If these recommendations alone were adopted, this would represent a considerable improvement on current practices and arrangements. There is a need to develop a range of centres of expertise that are in the public realm, To date, the only Australian hospital patient satisfaction survey work to have resulted in significant publication of material on both the results and the methodology is the Birthing Review in Victoria. The best overseas example in this respect is the British Royal College of Surgeons’ patient satisfaction survey. There is also research published on the HCA survey (the methodology) and the Picker/Commonwealth (its use in research and hospital service development). Publication is vitally important for two reasons. It disseminates expertise and ideas. Secondly, it contributes to the development of knowledge, both about patient feedback mechanisms, but also about organisational and service delivery that works better. It also has the important effect of making information and ideas available to consumers, and consumer organisations. There is value for hospitals in being able to call on a range of professional expertise to assist them in designing, administering, interpreting and using patient feedback, and in some circumstances, being able to take advantage of broader sampling and analysis expertise.
152
In addition, we consider that the model of ‘consumer audit’ offered by the British College of Health is a worthwhile addition to the range of ways that could be available to obtain consumer feedback. This would require the development of consumer based expertise to conduct consumer audit. There are precedents for this in Australia that could be built on, eg joint activities between the Royal Melbourne Hospital and the Health Issues Centre on outpatient clinics, and between Council on the Ageing (Victoria) and some hospitals and community services using consumer experiences to develop hospital discharge protocols. This formal expertise might be developed in selected organisations, but these organisations could develop partnerships with self help or illness-based advocacy organisations, such as a breast cancer advocacy group. Consumer audits could be undertaken in partnerships with hospitals and/or area/regional authorities. There are several roles consumer organisation already play, eg self help, advocacy, consultation, representation of consumer views in policy development, research into consumer perspectives on health. This proposal does extend their activity in a new direction. We recommend that there be consideration of the following: 1. The Department of Human Services and Health considers a three year program to fund research on consumer feedback on hospital quality management, including a subprogram on good quality patient satisfaction research. The objective would be to build both expertise about methods, but also a body of substantial research on consumer experiences. This would be accomplished by a range of measures: • to establish a range of quality surveys and approaches which would be available for use and adaptation by hospitals and health regions, and • to strengthen Australian based research and theory development. If this is acceptable, the program should • encompass a range and mix of methods and approaches; • encourage research that examines feedback mechanisms that are effective for important sub groups and those groups routinely excluded from state based surveys; • require material to be published in the public realm; • fund examples of partnerships between consumer organisations and hospitals; • include some focus on skill development, eg through a Technical Advisory Group (TAG). 1. Within the National Hospital Quality Management Program, that funding be made available to enable one or more consumer organisations to develop and trial models for consumer audit undertaken in partnership with hospitals. This is likely to be most usefully undertaken in state based consumer organisations, with a view to longer term contracting of such activity by hospitals, networks, or area health boards. 2. Linked to the proposal for funding of developmental research in Recommendation One, the Department of Human Services and Health explores ways to establish a clearing house function on patient satisfaction surveys within pre-existing organisations eg ACHS, AIH&W or a University department. This could be let by tender, with requirements about some association with consumer organisations. The purpose of this would be to disseminate information and advice, and the chance to exchange experiences with doing 153
patient satisfaction work. It could collect a data base on patient satisfaction work that has been done already, provide hospital with access to pre-existing research so that everyone is not continually starting from scratch, and provide literature about the full mix of methods. It should collect and disseminate research about what is done by hospital and related health services as a result of patient satisfaction surveys. 3. As part of the National Hospital Quality Management program, the Department of Human Services and Health ensures the development of integrated hospital quality management approaches which link patient satisfaction data and other data from consumer feedback to the other forms of quality information and data, so that consumer data is part of hospitals’ strategic planning and quality management. Likewise, newly developed patient survey instruments should be linked to and draw upon information from other types of quality data. 4. The Department of Human Services and Health initiates discussions involving the ACHS and consumer organisations to strengthen and establish process and performance criteria for assessing the adequacy of consumer feedback in the accreditation standards administered by the ACHS. While there is a standard in the ACHS accreditation about obtaining patient feedback, it is unspecified. There are no ‘best practice’ indicators, and the current standard risks the proliferation of a myriad of poorly designed and administered patient satisfaction surveys. 5. The Department of Human Services and Health needs to link patient satisfaction development with development of a consumer’s health rights charter, which builds further on the Medicare charter and which establishes a set of public standards against which performance can be measured 6. The Department of Human Services and Health takes steps to ensure that its own activities continue to recognise the importance of linking consumer feedback and patient satisfaction data with other policies relevant to quality management, eg adverse events research, health complaints, guideline development on clinical practice, and consumer and professional research on health outcomes. Consumer perspectives and research need to be elaborated within all of these areas. 7. In the short term, in order to facilitate a broader discussion of the issues about patient satisfaction surveys raised by this report, the Department of Human Services and Health initiates a national workshop on patient satisfaction surveys to canvass issues and exchange information about current uses of patient satisfaction surveys, how hospitals use surveys and the data they produce, quality issues in the design and use of patient satisfaction surveys, and infrastructure support for patient satisfaction activity and other consumer feedback. 8.Complaints data is an important source of consumer information about quality, but is not presented in ways that maximise its value. Currently, health complaints data does not exist in forms that allow national comparisons and aggregation of data, nor is the data available in a manner that will allow more detailed analysis of complaints to provide useful information for quality improvement and the prevention of such events. We understand that Health Commissioners have work under way on this. We consider this is essential for
154
better use of complaints data. We recommend that this project build links to the broader patient satisfaction and patient reporting data, such as we have drawn together in this report. Finally, while our recommendations have been directed at the Commonwealth government, and have considered some issues at state government and hospital level, there is another level of potential activity raised in this report, namely, the use which the Colleges could make of patient surveys as part of their own strategies to improve aspects of quality of care. The Royal College of Surgeons’ survey in the UK, discussed in this report, provides an example of the use of a patient satisfaction survey as part of College quality activities and particularly targeted at elements of care such as communication and information. In this respect, we suggest referral of this report to the Colleges.
155
156
APPENDIX A
SUMMARIES OF CONSUMER REPORTS OF HOSPITAL EXPERIENCES
157
158
APPENDIX A: SUMMARIES OF CONSUMER REPORTS OF HOSPITAL EXPERIENCES This appendix presents summaries of people’s experiences of hospital care. It has been gathered from reports by consumer or community organisations, or research organisations which specialise in researching policy issues through focussing on people’s experiences. Table A1 below provides a list of the reports by date of publication, the organisation responsible for the research, and the general topic of research for the subsequent material presented in detail in the Appendix. Table A1: Summary of research Date of study
Organisation/ researcher
General topic
No 1. 1995 No 2. 1994 No 3. 1994 No 4. 1994
WA Health Consumers’ Council Consumers’ Health Forum Council on the Ageing (VIC) Delegation of Seniors Org’s (Council on the Ageing (NSW)) Gilley, for the Brotherhood of St Laurence
Accident and emergency Hospital care Discharge planning Discharge planning
No 5. 1994 No 6. 1994 No 7. 1994 No 8. 1993 No 9. 1993 No 10. 1992 No 11. 1992 No 12. 1991 No 13. 1991 No 14. 1989
NHMRC W/party on treatment for diagnosed breast cancer WA Health Consumers’ Council Aust. Council of Social Service SA North West Suburbs Health & Social Welfare Council & Migrant Health Service Marion Brighton Glenelg Health & Social Welfare Council Wositzky, for Risk Study in General Practice, Monash University. Aust. Pensioners’ & Superannuants’ Federation Marion Brighton Glenelg Health & Social Welfare Council Marion Brighton Glenelg Health & Social Welfare Council
Mothers in fringe areas - experiences of hospital and other health services Experiences of consumers Care after hospital stay Poverty and health Use and experiences of interpreter services People with mental illness experiences of the health system Medication issues for high service users Experiences of people with chronic illness (diabetes or arthritis) Information given on pregnancy tests Pregnancy and birth
1. Accident and emergency services The West Australian Health Consumers' Council (1995) undertook a rough and ready snapshot survey of accident and emergency departments in four major public hospitals and looked at the information made available to users, using observational techniques.
159
The survey raised the following issues: •
the hospitals were relying on people's capacity to read and understand English;
•
important information was not always prominently displayed;
•
information on the adjustments of waiting times is crucial as many people cannot see that other more urgent cases have been given priority.
2. The Consumers' Health Forum's Casemix Project In 1994 the Consumers' Health Forum (CHF) conducted 30 consultations with people around Australia as part of a project informing the Commonwealth Department of Human Services and Health about concerns with the introduction of casemix, and, in particular, its possible effect on quality of care. The purpose of the consultations was to survey generally the types of problems encountered by consumers in hospitals. Groups with special needs or people who were more vulnerable were targeted. Given the documented experiences of these people in the health system generally, it is not surprising that the final report has a strongly critical flavour. It should be noted that a population based survey may find higher levels of satisfaction with elements of care. How the data was obtained Of the 30 consultations, eight were public meetings (CHF 1994:19), and the remainder targeted groups whose members had a high use of hospital services, or who may have been socially disadvantaged in some way. These groups included Aboriginal people, people from non-English speaking backgrounds, people with a chronic medical condition, or a disability, or with a mental illness, older people, homeless people, and people in rural and remote areas. Three meetings were held with the assistance of interpreters (language groups not specified). A representative sample was not sought. The age range of participants was from teenage years to people in their eighties. The majority were women although men were the majority at five meetings (van der Heide 1994:2) The meetings and focus groups used an Information Kit as background to the discussion, and meetings were loosely structured around the themes of pre, during, and post hospital experiences. Notes were taken at the meetings and they were analysed using a qualitative computer package by an external academic consultant (van der Heide 1994:3). Summary of main issues The following is a summary of the main findings of the consultations (CHF 1994).
160
Overall Some hospitals were noted by participants as excellent. Good practice should be acknowledged and models promoted. Overall, though, there was much negative comment. People from a non-English speaking background wished their appreciation of the hospital system noted, but experienced considerable communication problems. There were suggestions that the quality of care had deteriorated in recent years, and the staff time available for patients had decreased, possibly due to budget cuts in some states. There were suggestions that the greater use of agency nurses had led to uncoordinated care and possibly lesser quality care. Certain specialty units had developed a high quality approach. Role of consumer/hospital culture Hospitals were seen as places where people were disempowered, ie where people surrendered their autonomy and control, and the right to be informed about what was happening. Not everyone minded but many did. The fragmentary nature of hospital care was noted as worsening the sense that the whole person was being treated. Consumers' perceptions of their low status were compounded by the frequently stated fear of complaining, and in particular, the fear of reprisals. Certain marginalised groups expressed the fear of being labelled a troublemaker, as did residents of rural areas (where they were known by service staff). Frequent hospital users also discussed similar fears. Access Some people had difficulty getting in (long waiting lists, people with a chronic illness had difficulty getting through casualty). People experienced anxiety while waiting, due to the absence of information or communication about when admission may occur. Socially disadvantaged groups with ongoing concerns about access to hospitals included Aboriginal people, people from a non-English speaking background, and people with a disability or chronic medical condition. The cost and time of travel to and from hospitals for rural and remote area consumers was not well understood by all hospitals, and there were difficulties with accommodation for family members. Reimbursement schemes could be slow. With the development of major specialist services in city areas, several trips were involved in one episode. Discharge The process of discharge (or lack of a process) was the topic of much criticism. Certain specialist units had good records.
161
Information The lack of timely and appropriate information on all aspects of the hospital episode was noted repeatedly. The lack of information on medications was raised at all consultations. Communication Although improvements were noted, specialists were seen as uncommunicative. Hospitals did not communicate with general practitioners (GPs). Consumers were not involved in decision making. Some older people had been told to sign consent forms, without explanation. Routine use of trained interpreters was not occurring. Discrimination Discrimination was possible if you had a mental illness, especially when seeking general health care; if you were homeless; an Aboriginal person; or an older person. Some people felt that private health insurance would guarantee better treatment. Environmental Food, and the physical environment was not good in some hospitals. The poor amenity of psychiatric hospitals was noted. 3. Older people and discharge planning in Victoria The Council on the Ageing of Victoria (COTA (Vic) 1994) undertook a series of focus groups with older people and service providers for a project designed to establish principles for better discharge planning. How the data was obtained Focus groups were conducted with members of the COTA in the western and inner city areas of Melbourne. One meeting was held with members of a support group whose members were predominantly from non-English speaking background. A general meeting of COTA members was held to discuss discharge planning. Interviews were conducted with older people attending a hospital day care centre in a northern Victorian town. The total number of consumers involved was 40 - 50. A schedule of questions guided the focus groups. The interview structure is not reported. The focus groups were taped. Summary of main issues Overall At the point of discharge people experience the best and worst of hospital care.
162
Role of consumer/ hospital culture Older people were less likely to complain or take a more active role in seeking information about discharge. A number of Victorian hospitals have statements of rights and responsibilities which are given to people on arrival. Discharge Discharge is often the most stressful time for older consumers. Hospitals have no systematic way of assessing people's needs for care post discharge. People already linked into community care programs fared better. What planning which occurred focused on those most in need: those not obviously in need, but possibly with potential home difficulties, slipped through the net. Service providers acknowledged that hospital patients, their families and carers should be involved in discharge planning but that this didn't occur enough. The period immediately after discharge now appeared to be largely unplanned for despite shortening lengths of stay. Hospitals make many assumptions about the ability of a carer to take responsibility of the person after discharge. Information The verbal and written information people receive prior to discharge is crucial, but is often inadequate. Communication Communication between the hospital and local services can be poor. Specific areas mentioned include pharmaceuticals, the cost of services, and the role and responsibilities of different services in providing care. 4. Older people and discharge planning in NSW On behalf of the Delegation of Seniors' Organisations in New South Wales, the Council on the Ageing (COTA) of NSW undertook a short survey of the experiences of discharge planning for people who rated themselves as Seniors, and who had been in hospital any time since January 1993. The short report (Delegation of Seniors' Organisations 1994) notes the following: •
208 valid responses were received from a total of 344 people surveyed (60%; note they were self-selected);
163
•
44.5% reported they were not seen by a discharge planner;
•
of those not seen by a discharge planner, 50.3% reported that they required additional care;
•
from the total group, and of those requiring additional care, 66.25% reported that no community or other support services were arranged by the hospital.
5. Access to and quality of hospital services for mothers living on the fringes of the Melbourne area, or the fringe of regional cities How the data was obtained For the Brotherhood of St Laurence, Gilley (1994) interviewed 146 mothers living in 'service poor' areas of Victoria, including outer urban areas, semi rural areas, and areas within and outside of provincial cities. They were asked their views on access to and the quality of services for pregnancy, childbirth and for postnatal care. The women were interviewed in a series of 23 small group discussions. They also could complete a small questionnaire on social and demographic matters, and on their use of health and community services. Summary of results Of the findings relevant to use of hospital services, the following comments are made: •
antenatal care was generally viewed favourable. However, there were criticisms by women using the public system of long waits, lack of privacy, and discontinuity of care;
•
overall, most were satisfied with the care while in hospital. Sources of satisfaction were friendly and supportive staff; continuity of care, particularly in labour; information during labour and while in hospital; privacy; good advice on baby care and breastfeeding; and high quality food. Sources of dissatisfaction were the obverse of these;
•
some mothers in the Cranbourne area felt they had been discharged too quickly after the birth, but this was not an issue in the other areas;
•
some mothers did not receive sufficient follow up care after discharge and some were charged for services but had not been informed they had to pay prior to receiving the service;
•
hospital use after birth was common - one quarter of mothers required them for their children. All the critical comments related to one hospital;
•
the lack of public transport for some mothers was a critical issue in gaining access to a range of services.
164
The results of this study were also compared with the results of a study of mothers and children who lived in inner urban areas. Whilst these people had better access to services, they had less social supports, had greater language barriers, used birth classes and specialists less, and had less knowledge of available services. The report comments that in the study of mothers in outer urban and regional areas, their experiences and sources of dissatisfaction echoed the results of the Consumer Survey of the Victorian Birthing Review. 6. Women's views on treatment for breast cancer In a review of consumer literature on the perspectives of women about the treatment for diagnosed breast cancer (NHMRC Working Party on the Treatment of Diagnosed Breast Cancer 1994), 46 publications (including two major Australian reports) were surveyed for comments under the following themes: Choice and control (role of consumer) Most women wish to be involved in treatment decisions, and wish to exert some control over whatever they can in the treatment process. The shock of breast cancer meant that additional care was needed to ensure this occurred at a time when women were stressed and vulnerable. Barriers to enhancing choice and control included the lack of continuity of care in public hospitals, lack of time between diagnosis and treatment, lack of good information, lack of consultation time, and settings which did not promote privacy or dignity. Access The lack of access to radiation therapy in rural areas may reduce women's choice of treatments. Information Women wanted more information than they tended to receive, and information on a broader range of topics than just the disease and treatment options. Some women did not but many did, and their need for information on what would happen and was happening grew after surgery rather than diminished. Written and audio-visual information was important but did not replace the need for interaction with health care providers. The review also examined where women sought information, who should give women information, when it should be given, and information from former patients or self help groups. Communication
165
Barriers noted were the use of jargon, lack of time, the negative portrayal of information on their condition, the lack of attention to the concerns expressed by women, and not treating women as 'whole' people or as equals. The treatment process The trauma of treatment was exacerbated by long waits for treatment or waiting a long time to see a provider, the lack of privacy, particularly when receiving radiotherapy, poor amenity of services, the lack of team approach, or the structuring of treatment which did not take into account women's other responsibilities. 7. West Australian Health Consumers' Council: Survey by newspaper An example of a rarely used method of getting feedback is a short questionnaire by the West Australian Health Consumers' Council (1994), placed in the major daily newspaper, seeking people's views about health goals and need for some types of care. Although there are some problems with the framing of the questions (they could be seen as somewhat leading), the example is given to illustrate a relatively inexpensive method which reached a different audience. There were six questions, with yes/no and don't know options. They covered priorities for the health system, resources for educating health consumers, support following day surgery, being informed on health policy and having an avenue to complain, care for people with mental illness, and coordination of care by GPs. The Council received 932 replies which were analysed by the statistics section of the West Australian Health Department. Results Relevant responses to this report are: •
44% of respondents said they would not be able to arrange help at home and someone to care for them, following day surgery;
•
77% said they would expect a nurse to visit them at home after surgery; 52% expected a doctor to visit them;
•
96.5% believed they should be informed of what to expect from health services, how to access them, and what to do if not satisfied.
8. Poverty and health Following on from the National Health Strategy (NHS) Research report, Enough to make you sick : How income and environment affect health, the Australian Council of Social Service (ACOSS) and the CHF conducted focus groups to explore some of the reasons behind the documented inequalities in health (ACOSS 1993). The purpose of the research was also to
166
illustrate potential inter-sectoral strategies. As with the project above, the summary comments below only concern hospital related experiences. How the data was obtained There were 12 focus groups involving about 200 people. The targeted groups were those identified in the NHS report as the most disadvantaged in terms of their health and use of services. These were sole parents, unemployed, homeless or Aboriginal people, and people with a mental illness, or from a non-English speaking background. Meetings were held in West Australia, New South Wales, Queensland, South Australia, Tasmania, and Victoria. All the meetings were in capital cities, except two, attended by Aboriginal people. Two were with people from Laos, and one was with non-English speaking background women who were mostly service providers. Meetings usually lasted four hours. The method of recording or transcription is unknown. Summary of main issues Overall The lack of social supports for people in disadvantaged circumstances was a crucial factor and may increase their health risks or adversely affect the quality of their interactions with the health system. Access Medicare was very important in ensuring access to care. Access to public dental hospitals was very poor and many experience extremely long waits. There is some suggestion that the costs of pharmaceuticals and medical aids and equipment can be prohibitive. Communication Communication was hampered by the lack of use of trained interpreters for people of nonEnglish speaking background. Discrimination Some people felt that service providers had a stereotypical view of them, as 'single mother' or 'mentally ill' and this coloured their interactions. Some felt that if they had private insurance they would receive more respectful care. 9. The use of interpreter services
167
A report by the South Australian North West Suburbs Health and Social Welfare Council and the Migrant Health Service (1993) documents the results of a phone-in and survey of consumers and health workers regarding the use of interpreter services in health organisations. A total of 474 people responded to the survey: 293 were consumers; 148 were workers in the health system; and 31 were workers outside the health system. Key results are: •
hospital services were the most common service for which interpreters were used;
•
for those who had been refused an interpreter service in hospital, common reasons were that they were told to bring a friend, they should have booked in advance, and no funding available;
•
there were reasonably high levels of satisfaction with the quality of the services, although concerns were expressed about serious breaches in confidentiality. This led some people to refuse further interpreters. There were some concerns with the fullness and adequacy of the interpretation;
•
people were not willing to criticise interpreter services because they believed it would lead to the service being removed;
•
people expressed the need for more specialised interpreters, for example, in mental health, older persons care;
•
respondents also expressed the need for more bilingual workers, increased service levels after hours, and greater flexibility in choice of interpreter.
10. Hospital use for people with mental illness In 1992 the Marion Brighton Glenelg Health and Social Welfare Council in South Australia conducted a major consultative project gathering input from people who used mental health services about their views on the changes needed. How the data was obtained The report (Marion Brighton Glenelg Health and Social Welfare Council 1993) documents the methods of consultation: five public meetings (124 attended); 150 people interviewed individually or in groups; and three workshops with consumers and carers to develop a plan for change. The emphasis was on meeting people who had experienced a mental illness or carers of people who had. The report concerns people's views on changes to the system of mental health care in South Australia: the summary below documents comments about hospital care. Summary of main issues Role of consumer/ hospital culture 168
People stressed the need to be listened to, the desire to be involved in treatment, and to be assisted or encouraged to assume responsibilities. Access There was concern with the reduction in inpatient beds. People wanted hospital and community based services to both be available. Some expressed concern with the lack of 24 hour access to appropriate care. Some people had experienced difficulty getting admitted despite being unwell. People with certain conditions such as agoraphobia also experienced difficulties accessing services. Continuity of care Continuity of care was seen as particularly important, especially having one relationship with one doctor. Discharge Some people had been discharged without adequate follow up and support, or access to rehabilitation services. Information Information on the side effects of medication was very important, particularly given its central role in treatment. Communication Carers reported being ignored, not informed or involved in decision making, yet being expected to provide full support for the person upon discharge. 11. Medication issues for frequent health service users In a study of frequent users of general practitioner (GP) services, many of whom were socially disadvantaged, Wositzky (1992:10-15) reported from focus groups that: •
people with a chronic illness or disability experienced communication problems between institutional and local doctors;
•
some people with intellectual or psychiatric disabilities had experienced considerable problems getting sufficient information on medications, or in getting trained to cope with a complex drug regime;
•
most people wanted more information than they had been given on possible side-effects of medications. This compared to a related study where GPs had thought that information on side-effects would cause anxiety. 169
12. The experience of older people with chronic illness The Australian Pensioners' and Superannuants' Federation (APSF) undertook a study of the health system experiences of older people with a chronic illness (diabetes or arthritis), which contributed to the National Health Strategy (APSF 1991). The study was conducted in a planning context where significant factors were the increasing proportion of older people, the rising prevalence of chronic illness, and the high use of health services by older people with a chronic illness. The purpose of this research was to explore the nature of people's experiences with all parts of the system, and therefore not all of these are relevant here. The summary comments (listed below the outline of the method) reflect only those which involved the use of hospitals. How the data was obtained Six focus groups with 142 older people were conducted in Sydney or Melbourne in June 1991 (APSF 1991:4-5). Participants were sought via workers in the aged care sector, through community health centres, hospital clinics, pensioner and self help groups. The number of participants and their age breakdown was 105 (60-75 years), 31 (75-85 years), 4 (85 plus), and 2 were under 60. There were two focus groups conducted for people from non-English speaking background: one for people who spoke Greek, and one for Vietnamese speakers. One focus group in Sydney was conducted for hostel residents. Meetings usually lasted four hours. The method of recording or transcription is unknown. Summary of main issues Overall Experiences while in hospital were generally positive. The difficulties arose in getting in, in staying in long enough, and in the quality of the discharge planning. For older people with a chronic illness, those who fare well have access to multi-disciplinary clinics (which emphasise self management, combined with prompt treatment) well coordinated care between hospitals and local services, and access to allied services (in hospitals or in the community). People who do not fare well do not have access to this system approach, but, in addition, experience discrimination due to age, and cannot access needed allied services due to cost and the discouraging attitudes of some doctors. Role of consumer/service culture
170
The experiences of people with diabetes were qualitatively different from those with arthritis. The treatment model of the former emphasised education, self management and access to multidisciplinary clinics. Therefore, people with diabetes were more satisfied with the system than those with arthritis. Access Length of wait could be an issue. Some people with diabetes would have preferred a medical admission for stabilisation instead of the sometimes daily visits to the clinic. This burden could be exacerbated if the person could not drive and there was no hospital transport. A third of the pensioners interviewed had private cover to ensure access to hospitals. For people requiring allied health services after a hospital stay, for example, access to a physiotherapist or podiatrist, but who could not afford private care, there could be long waiting times for services in a hospital outpatient department. Access to aids and appliances via a hospital may have been dependent on the time of application, ie when the hospital had funds. Discharge Some people suffer adversely from early discharge without adequate backup. The report notes (1991:14) that "Many had experienced serious post-operative complications after discharge". Some people were discharged to a home situation which they could not cope with physically, for example, no one in the hospital knew they also cared for another disabled person. Communication The experience of people from non-English speaking background was similar to English speaking, but difficulties were compounded by the infrequent use of trained interpreters, as well as cultural differences and ignorance of how the health system worked. Discrimination Some people felt that they had poorer access to treatment because they were old, had a chronic illness (acute had higher priority), and they had no private insurance.
171
13. Information on pregnancy tests for South Australian women Arising from research with women about their experiences of childbirth (see below), the Marion Brighton Glenelg Health and Social Welfare Council (1991) in South Australia surveyed and consulted with 61 women in the region about the information they received on tests for birth defects. The lack of information during pregnancy and the sense of passivity felt by many women had emerged as a key issue in the earlier research. Women were reached via playgroups, self help groups, parent support groups and kindergartens. The purpose of the report was to be a source of information for health professionals. Results •
many women were not well informed of the reason for some tests, and the implications. A third of the women thought the tests were compulsory, raising issues of informed consent;
•
information was preferred in written and verbal forms but the former was provided in only about half the cases;
•
very little information was available in languages other than English;
•
some positive test results were not handled sensitively.
14. Pregnancy and birth - a consultation with South Australian women Approximately 60 women (principally consumers) attended a seminar and consultation on birthing, run by the Marion Brighton Glenelg Health and Social Welfare Council in South Australia. At the meeting women discussed those aspects of care important to them, their expectations of services, and what they would have liked to change about the care they did receive. Results The report (Marion Brighton Glenelg Health and Social Welfare Council 1989) notes common issues which emerged, as well as detailed comments. The main areas of concern were: • • • • • •
continuity of care information and education the need to feel comfortable and secure more midwife involvement desired women wanted more information and better classes more informal sources of support were needed.
REFERENCES
172
Adelaide Medical Centre for Women and Children. 1992. Consumer consultancy. Final Report. Volumes I and II. Prepared by K. Alexander. Adelaide, Adelaide Medical Centre for Women and Children. Aharony, L. and Strasser, S. 1993. ‘Patient satisfaction: what we know about and what we still need to explore’. Medical Care Review 50(1):49-79. (ACOSS) Australian Council of Social Service. 1993. Warning. Poverty is a health hazard. ACOSS Research Paper No. 7. Sydney, ACOSS. (APSF) Australian Pensions' and Superannuants' Federation. 1991. A window on the health system. Sydney, APSF. Audit Commission 1993. What Seems to be the Matter: Communication between Hospitals and Patients. National Health Services Report No. 12. UK. Bartlett, H. and Pennebaker, D. 1990. ‘Consumer views of maternity services in Western Australia’. In Report of the Ministerial Task Force to Review Obstetric, Neonatal and Gynaecological Services in Western Australia. Volume III, Survey Reports. Health Department of Western Australia, January, pp. 1-66.. Baum, F. 1995. ‘Researching public health: behind the qualitative-quantitative methodological debate’. Soc Sci Med 40(4):459-68. Beckmann, U. 1994. 'Australian incident monitoring study - intensive care'. Incident monitoring and risk management in the health care sector. Conference Proceedings. The Review of Professional Indemnity Arrangements for Health Care Professionals. Canberra, Commonwealth Department of Human Services and Health, pp.66-8. Best Practice Program and National Industry Extension Service 1993. Benchmarking Self Help Manual: Your organisations guide to acheiving best practice. AGPS, Canberra. Birmingham (FHSA) Family Health Services Authority 1994. Putting Patient Feedback Into Practice. An action pack produced by the College of Health for Birmingham FHSA. Black, N. and Sanderson, C. 1993. ‘Day surgery: development of a questionnaire for eliciting patients’ experiences’. Quality in Health Care 2(3):157-61. Brown, S. and Lumley, J. 1993. 'Antenatal care: a case of the inverse care law?' Australian Journal of Public Health 17(2): 95-103. ___ 1994. 'Satisfaction with care in labor and birth: a survey of 790 Australian women' Birth 21(1): 4-13. Brown, S., Lumley, J., Small, R., and Astbury, J. 1994. Missing Voices: The Experience of Motherhood. Melbourne, Oxford University Press.
173
Bruster, S., Jarman, B., Bosanquet, N., Weston, D., Erens, R. and Delbanco, T. 1994. ‘National survey of hospital patients’. British Medical Journal 309 (10 December):1542-46. Camp, R.C. 1994. Planning to Deliver High Quality Goods and Services and Improve the Business. AQC 7th National TQM Conference. Carers Association of Australia 1994. Listen to the Carers! The Many Voices of Care. Report of the National Carers Consultation to the International Year of the Family Council prepared for Carers Association of Australia. Canberra. September. Carr-Hill, R. ‘The measurement of patient satisfaction’. Journal of Public Health Medicine 14(3):236-49. Carr-Hill, R. and Ng, F. 1992. ‘A confusion of charters’. Health Service Journal 5 November, pp.25-6. Cleary, P., Edgeman-Levitan, S., Roberts, M., Maloney, T., McMullan, W., Walker, J. and Delbanco, T. 1991. ‘Patients evaluate their hospital care: a national survey’. Health Affairs 10(4):254-67. Cockburn, J., Hill, D. De Luise, T. and Flint-Richter, D. 1993. 'Satisfaction of attenders during the establishment of an Australian mammography screening program'. Australian Journal of Public Health 17(2): 103-8. College of Health 1992. Consumer Audit − Pilot Project Report. A joint project by South Bedfordshire Health Authority, South Bedfordshire Community Health Council and The College of Health. College of Health. March. ___ 1992. Consumer Audit − Patient Feedback Project for Practice Managers in the Merton, Sutton and Wandsworth FHSA. College of Health. August. Commonwealth Government. Review of Professional Indemnity Arrangements for Health Care Professionals. 1993. The health/medical care injury case study project. Prepared by the National Centre for Socio-legal Studies, La Trobe University (authors G. Coventry, J. Daly, M. Evans, C. Lowy, M. McMahon, and G. Roberts). Canberra, Australian Government Publishing Service. (CHF) Consumers' Health Forum. 1994. Consumers' Health Forum casemix project. Final report. Canberra, CHF. (COTA (VIC)) Council on the Ageing (Victoria). 1994. Removing the boundaries: hospital discharge practices and older people returning to the community. Prepared by J. Kearney. Melbourne, COTA (VIC). Daly, J. and McDonald, I. 1993. The social impact of echocardiography. Australian Institute of Health and Welfare, Health Care Technology Series No 9. Canberra, Australian Government Publishing Service.
174
Davies, A. and Ware, J. 1988. ‘Involving consumers in quality of care assessment’. Health Affairs Spring:33-48. Delegation of Seniors' Organisations. 1994. 'Discharge planning survey'. By L. Kaplan. Sydney, Council on the Ageing (New South Wales), November. Department of Industrial Relations and Australian Manufacturing Council 1992. International Best Practice: Report on the Overseas Study Mission. AGPS. Canberra. Donabedian, A. 1992. ‘Quality assurance in health care: consumers’ role’. Quality in Health Care 1(4):247-51). Donovan, J. and Blake, D. 1992. ‘Patient non-compliance: deviance or reasoned decisionmaking’. Soc Sci Med 34(5):507-13. Draper, M. and Hill, S. 1995. Consumers and general practice: understanding and assessing consumers’ experiences by using patient satisfaction surveys. A report prepared for the Consumers’ Health Forum General Committee, May. (Not published.) Ehnfors, M. and Smedby, B. 1993. ‘Patient satisfaction surveys subsequent to hospital care: problems of sampling, non-response and other losses’. Quality Assurance in Health Care 5(1):19-32. Fitzpatrick, R. and Hopkins, A. 1983. ‘Problems in the conceptual framework of patient satisfaction research: an empirical exploration’. Sociology of Health and Illness 5(3):297-311. Gerteis, M., Edgman-Levitan, S., Daley, J. and Edlbanco, T.L. (eds) 1993. Through the Patient’s Eyes − Understanding and promoting patient-centred care. The PickerCommonwealth Program for Patient-Centred Care, Beth Israel Hospital. Jossey-Bass Inc. Gilley, T. 1994. Beyond the city: access to services for mothers and babies. Melbourne, Brotherhood of St Laurence. Greene, M., Adelman, R., Friedman, E. and Charon, R. ‘Older patient satisfaction with communication during an initial medical encounter’. Soc Sci Med 38(9):1279-88. Hall, J. and Dornan, M. 1988a. ‘Meta-analysis of satisfaction with medical care: description of research domain and analysis of overall satisfaction levels’. Soc Sci Med 27(6):637-44. ___. 1988b. ‘What patients like about their medical care and how often they are asked: a metaanalysis of the satisfaction literature’ Soc Sci Med 27(9):935-9. ___. 1990. ‘Patient sociodemographic characteristics as predictors of satisfaction with medical care: a meta-analysis’. Soc Sci Med 30(7):811-18. Hanson, R., Clifton-Smith, B. and Fasher, B. 1994. 'Patient dissatisfaction in a paediatric accident and emergency department'. J. Qual. Clin. Practice 14(3): 137-43.
175
(H&CS) Health and Community Services 1994. Patient Satisfaction with Victorian Hospitals. Prepared by Ramis Corporation, Melbourne. ___ 1994. Patient Satisfaction Survey − Stage 2 - Implementation Tender Brief. December. ___ 1995. Towards a New Framework for Quality in Victoria’s Hospitals − Issues and Options : A Discussion Paper. June. ___. Putting Patients First − Public Hospitals: What Do They Offer You?Melbourne. Health Consumers' Council. 1994. Consumer identified health goals. Report of a questionnaire inserted into a newspaper. Perth, Health Consumers' Council, October. ___. 1995. Hospital Emergency Departments - Snapshot Survey. Perth, Health Consumers' Council, January. Health Issues Centre. 1992. Casemix. Quality and Consumers. Health Issues Centre, Melbourne. (HDV) Health Department of Victoria. 1990. Having a baby in Victoria. Final report of the Ministerial review of birthing services in Victoria. Chairperson J. Lumley. Melbourne, HDV. (HDWA) Health Department of Western Australia 1990. Performance Indicator − Quality Assurance 1989/90. Organisation Development Branch. August. ___ 1990. Summary Report of 1990 Patient Satisfaction Survey. Organisation Development Branch. December. ___ 1991. 1991 Patient Satisfaction Survey. Combined Operations Branch. August. ___ 1993. Patient Satisfaction with Mental Health Services A Pilot Study. Quality Management Section, Clinical Liaison Unit. March. Henderson, H., Lee, N. and Harte, C. 1990. ‘Consultations with Aboriginal women’. In Report of the Ministerial Task Force to Review Obstetric, Neonatal and Gynaecological Services in Western Australia. Volume III, Survey Reports. Health Department of Western Australia, January, pp. 67-156. Hill, S. and Harries, U. 1993. ‘The outcomes process: some reflections from research with people in their 60s and 70s’. Critical Public Health 4(4):21-8. Hirschman, A. O. 1970 Exit, Voice, and Loyalty: Response to decline in firms, organisations and states Cambridge, Harvard University Press. Hopkins, A., Gabbay, J. and Neuberger, J. 1994. ‘Role of users in health care in achieving a quality service’. Quality in Health Care 3(4):203-9.
176
Immigrant Women's Health Project. Hospital-based Research. 1995. Report. By R. Lopez and L. Fazzolari. East Melbourne, Mercy Hospital for Women. Joule, N. 1992. User Involvement in Medical Audit − A spoke in the wheel or a link in the chain?. The Greater London Association of Community Health Councils. September. Jylha, M. 1994. ‘Self-rated health revisited: exploring survey interview episodes with elderly respondents’. Soc. Sci. Med. 39(7):983-90. Kempson, E. 1987. Informing Health Consumers: A Review of Consumer Health Information Needs and Services. College of Health. Kershaw, M. 1992. South Bedfordshire Consumer Audit − Day Surgery at the The Luton and Dunstable Hospital NHS Trust. A joint project between The College of Health, The Luton and Dunstable Hospital NHS Trust, South Bedfordshire Community Health Care Trust, Bedfordshire Health and South Bedfordshire Community Health Council. College of Health. College of Health. November. ___ 1993. South Bedfordshire Consumer Audit − Accident and Emergency Services at the Luton and Dunstable Hospital HNS Trust. A joint project between The College of Health, The Luton and Dunstable Hospital NHS Trust, South Bedfordshire Community Health Care Trust, Bedfordshire Health and South Bedfordshire Community Health Council. College of Health. May. King, B. 1989. 'Patient satisfaction survey: day surgery unit'. Aust. Clin. Rev. 9: 127-29. Lawrence, A. and Harrison, C. 1994. 'Reporting of cases of non-accidental injury: a survey of parents' responses'. J. Qual. Clin. Practice 14(4): 207-15. Locker, D. and Dunt, D. 1978. ‘Theoretical and methodological issues in sociological studies of consumer satisfaction with medical care’. Soc Sci Med 12:283-92. Macneil, J.,Rimmer, M., Testi, J. 1993. Raising the Standards: Benchmarking and Best Practice in Australia: Progress in the top 500 Enterprises. National Key Centre in Industrial Relations. Monash University. Melbourne. Macneil, J., Testi, J., Cupples,J., Rimmer, M., 1994. Benchmarking Australia: Linking Enterprises to World Best Practice. Longman, Melbourne. McCallum, J. and Gelfand, D. 1990. Ethnic women in the middle: a focus group study of daughters caring for older migrants in Australia. Canberra, National Centre forEpidemiology and Population Health. McHorney, C., Kosinski, M. and Ware, J. 1994. ‘Comparisons of the costs and quality of norms for the SF-36 health survey collected by mail versus telephone interview: results from a national survey’. Medical Care 32(6):551-67.
177
McIver, S. 1991. An Introduction to Obtaining the Views of Users of Health Services. King’s Fund Centre for Health Services Development. ___ 1992a. Obtaining the Views of Inpatients and Users of Casualty Departments. King’s Fund Centre. McIver, S. 1992b. ‘Counting customers or making customers count’. Critical Public Health 3(1): 16-21. ___ 1993. Obtaining the Views of Users of Primary and Community Health Care Services. King’s Fund Centre. McIver, S. and Carr-Hill, R. 1989. The NHS and its Customers − I. A Survey of the Current Practice of Customer Relations. Centre for Health Economics, University of York. January. Marion Brighton Glenelg Health and Social Welfare Council. 1989. Pregnancy and birth. Adelaide, Marion Brighton Glenelg Health and Social Welfare Council. ___ 1991. Tests conducted during pregnancy. By J. Turner. Adelaide, Marion Brighton Glenelg Health and Social Welfare Council. ___ 1993. Mental Illness Project. By H. Palmer. Adelaide, Marion Brighton Glenelg Health and Social Welfare Council. Martin, P., Wiles, R., Pratten, B., Gorton, S. and Green, S. 1992. A User Perspective − Views on London’s acute health services. Kings Fund and GLACHC (The Greater London Association of Community Health Councils). Meredith, P. 1993. ‘Patient participation in decision-making and consent to treatment: the case of general surgery’. Sociology of Health & Illness 15(3):315-36. Meredith, P., Emberton, M. and Devlin, HB. 1993. ‘What value is the patient’s experience of surgery to surgeons?: the merits and demerits of patient satisfaction surveys’. Ann R Coll Surg Eng, Bulletin. Vol. 76:72-3. Meredith, P. and Wood, C. 1994. Patient Satisfaction with Surgical Services − Report of the development of an audit instrument (1991-1993). Surgical Audit Unit, The Royal College of Surgeons of England. October. ___ 1995a. ‘The development of the Royal College of Surgeons of England’s patient satisfaction audit service’. Journal of Quality in Clinical Practice 15(June):67-74. ___. 1995b. ‘Patient satisfaction with surgery 1: challenges of questionnaire design, distribution and response’. (Submitted to Social Science and Medicine). ___. 1995c. ‘Patient satisfaction with surgery 2: a comparison of interview and survey data’. (Submitted to Social Science and Medicine).
178
Metropolitan Hospital Planning Board 1995. Victoria’s Health to 2050 Melbourne Metropolitan Hospital Planning Board. National Consumer Council 1986. Measuring Up − Consumer assessment of local authority services: a guideline study. Spring. ___ 1990. Consulting Consumers in the NHS − Services for elderly people with dementia living at home. London. June. (NHMRC) National Health and Medical Research Council Working Party on the Treatment of Diagnosed Breast Cancer. 1995. 'What do women say about treatment for breast cancer?' A review of the literature conducted for the Women's Perspectives Sub-group of the NHMRC Working Party on the Treatment of Diagnosed Breast Cancer. By K. Webster. November. National Health Strategy 1991. Hospital Services in Australia − Access and Financing. Issues Paper No. 2. Nelson, E. R., Rubin, H. R., Hays, R. D., and Meterko, M. 1990 Responses to Questionnaire Medical Care 28(9) Supplement: S18-21. New South Wales Complaints Unit 1994. Annual Report: in the public interest. Sydney, Health Care Complaints Commission. North West Suburbs Health and Welfare Council and the Migrant Health Service. 1993. Speak out on interpreter services. South Australian health system. Adelaide, North West Suburbs Health and Welfare Council and the Migrant Health Service. (NTIS) National Technical Information Service 1988. Quality of Medical Care: Information for Consumers. Office of Technology Assessment, US Department of Commerce. June. Nguyen, T. D., Attkisson, C. C. and Stegner, B. L. 1983 ‘Assessment of Patient Satisfaction: Development and Refinement of a Service Questionnaire’ Evaluation and Program Planning 6: 299-314. O’Connor, S. and Bowers, M. 1990. ‘An integrative overview of the quality dimension: marketing implications for the consumer-oriented health care organization’. Medical Care Review 47(2):193-219. O'Connor, S., Gibberd, R. and West, P. 1991. 'Patient satisfaction with day surgery'. Aust Clin. Rev. 11: 143-9. Office on Ageing, Social Policy Directorate, New South Wales. 1994. Hospital and home: a longitudinal study of hospital, residential and community service use by older people living in Dubbo, NSW. Written by J. McCallum, L. Simons, J. Simons, and J. Wilson. Sydney, Office on Ageing, NSW. Perreault, M., Leichner, P., Sabourin, S. and Gendreau, P. 1993. ‘Patient satisfaction with outpatient psychiatric services’. Evaluation and Program Planning 16:109-18. 179
Queensland. Health Rights Commission. 1994. Annual Report. Brisbane, Health Rights Commission. Rigge, M.1994. ‘Quality of life of long wait orthopaedic patients before and after admission: a consumer audit’. Quality in Health care 3(3):159-63. Roter, D. and Frankel, R. 1992. ‘Quantitative and qualitative approaches to the evaluation of the medical dialogue’. Soc Sci Med 34(10):1097-1103. (RACGP) Royal Australian College of General Practitioners. 1994. Incident monitoring in general practice: a pilot study. Report by the Family Medicine Research Unit, University of Sydney and the Research and Health Promotion Unit of the RACGP under the auspice of the RACGP, to the Professional Indemnity Review, Commonwealth Department of Human Services and Health. Ryan, M. 1994. Evaluating assisted reproductive technology programmes: an Australian pilot study using willingness to pay. Centre for Health Economics Research and Evaluation Discussion Paper No. 26. Sydney, Centre for Health Economics Research and Evaluation. Ryan, M., Campbell, J., Reti, L. and Rasmussen, M. 1994. 'Report on the pilot project on incident monitoring in obstetrics and gynaecology'. Incident monitoring and risk management in the health care sector. Conference Proceedings. The Review of Professional Indemnity Arrangements for Health Care Professionals. Canberra, Commonwealth Department of Human Services and Health, pp.55-64. Scott, A. and Smith, R. 1994. ‘Keeping the customer satisfied: issues in the interpretation and use of patient satisfaction surveys’.Soc Sci Med 6(4):353-9. Shmerling, A., Schattner, P. and Piterman, L. 1993. ‘Qualitative research in medical practice’. Medical Journal of Australia 158(1):619-22. Skinner, K. 1991. 'Parent satisfaction survey: paediatric physiotherapy services'. Aust. Clin. Rev. 11(1/2): 3-10. Small, R., Brown, S. and Lumley, J. 1992. ‘To Stay or Not to Stay: are fears about shorter postnatal stays justified?’. Midwifery 8:170-7. Small, R., Astbury, J., Brown, S. and Lumley, J. 1994. 'Depression after childbirth. Does social context matter?' The Medical Journal of Australia 161(8): 473-7. (SACHRU) South Australian Community Health Research Unit 1991. ‘Surveys and Questionnaire Design’. Paper No. 3, Research and Evaluation in Community Health Series. Flinders Press. ___ 1993. Neoporendi. Some features of social health and social action amongst Aboriginal people of Southern Adelaide. By G. Brice. Bedford Park, SACHRU.
180
___. (1994a). Links between hospitals and community-based health services. Bedford Park, SACHRU. ___. (1994b). Links between general practitioners, hospitals and community-based health services. Bedford Park, SACHRU. Sinclair, P. and Ward, M. 1985. 'Staff - patient communications in a medical ward'. Aust. Clin. Rev. 5(19): 172-7. Spendolini, M. 1992. The Benchmarking Book. American Management Association. State Health Purchasing Authority 1995. 1995 Patient Satisfaction Survey Summary Report. Epidemiology Branch. Western Australia. July. Steele, K. 1992. ‘Patients as experts: consumer appraisal of health services’. Public Money and Management 12(4):31-7. Stieber, J. and Wolfe, S.M. 1994. Who’s Watching Our Hospitals? Public Citizen’s Health Research Group. August. Stuber, D., Sheehan, T. and Sisko, A. 1986. 'Patient education survey'. Australian Health Review 9(2). Taylor, B. and Clarke, R. 1993. 'Client feedback - development of an instrument for psychiatric inpatients'. Australian Health Review 16(3): 231-44. Thomas, S., Henry, P., McCoy, A. and Smith, J. 1989. 'Why do parents stay overnight with children in hospitals? A survey of parents at Royal Children's Hospital'. Australian Health Review 12(2): 39-48. TQA Research Pty Ltd 1993. Patient Satisfaction in NSW Public Hospitals Summary. Prepared for NSW Health Department, Sydney. ___ 1994. NSW Health External Customer Satisfaction Survey 1993/1994 Summary Report. Prepared for NSW Health Department, Sydney. ___ 1994. Market Research Report: Queensland Health Patient Satisfaction Survey − Accident & Emergency Departments. Prepared for Queensland Health. September. Turrell, G. and Najman, J. ‘Collecting food related data from low socioeconomic groups: how adequate are our current research designs?’ Australian Journal of Public Health 19(4):410-6. van der Heide, G. 1994. 'What consumers want from hospitals". Paper presented at the TASA '94 Sociology Conference, Deakin University, 7-10 December 1994. Victoria Health Services Commissioner 1994. Annual Report. Melbourne, Health Services Commission of Victoria.
181
Vinen, J., Gaudry, P., Ashby, R., Epstein, J. and Blizard, P. 1994. 'Critical incident in emergency departments: a synoptic review'. Incident monitoring and risk management in the health care sector. Conference Proceedings. The Review of Professional Indemnity Ware, J. and Davies, A. 1983. ‘Behavioural consequences of consumer dissatisfaction with medical care’ Evaluation and Program Planning 6:291-7. Ware, J.E. and Berwick,D.M. 1990. ‘Conclusions and Recommendations’ in Medical Care 28(9) Supplement: S39-44. Westbrook, J. 1993. ‘Patient satisfaction: methodological issues and research findings’. Australian Health Review 16(1):75-88. Whitfield, M. and Baker, R. 1992. ‘Measuring patient satisfaction for audit in general practice’. Editorial. Quality in Health Care1(3):151-2. Williams, B. 1994. ‘Patient satisfaction: a valid concept?’. Soc Sci Med 38(4):509-16. Williams, P. and Brand, J. 1988. 'Food service departments in New South Wales hospitals - a survey'. Australian Health Review 11(1). Wositzky, K. 1992. Patient drug information. What do consumers want from general practitioners and pharmacists? Report of a consumer consultation in Footscray. Melbourne, The Risk Study in General Practice, Department of Community Medicine, Monash University and the National Heart Foundation. Wright, M. and Parker, G. 1994. 'RANZCP critical incident pilot study'. Incident monitoring and risk management in the health care sector. Conference Proceedings. The Review of Professional Indemnity Arrangements for Health Care Professionals. Canberra, Commonwealth Department of Human Services and Health, pp.37-40. Zeitaml, V. A., Parasuraman, A. and Barry, L. L. 1990 Delivering Quality Service: Balancing Customer Perceptions and Expectations New York, The Free Press.
182
