Using and understanding consumer satisfaction ... - Wiley Online Library

Journal of Psychiatric and Mental Health Nursing, 2009, 16, 272–278

Using and understanding consumer satisfaction to effect an improvement in mental health service delivery S. BRUNERO1 rn dipappsc bhsc mnurs(nurs prac), S. LAMONT2 rn rmn & G . FA I R B R O T H E R 3 r n ba m p h 1

Clinical Nurse Consultant, Department of Liaison Mental Health Nursing, 2Clinical Nurse Consultant, Mental Health Service, and 3Nurse Manager, Research, Prince of Wales Hospital, Sydney, NSW, Australia

Correspondence: S. Brunero Prince of Wales Hospital EBB, NERU, rm 7 High Street Randwick NSW 2031 Australia E-mail: scott.brunero@ sesiahs.health.nsw.gov.au

BRUNERO S., LAMONT S. & FAIRBROTHER G. (2009) Journal of Psychiatric and Mental Health Nursing 16, 272–278 Using and understanding consumer satisfaction to effect an improvement in mental health service delivery Consumer satisfaction is today, widely accepted as a measure of the level and quality of service received by consumers. The aim of this survey-based study is to explore consumer satisfaction with quality of care, staff, environment and discharge in a south eastern Sydney adult acute inpatient mental health unit. A cross-sectional analysis is pursued in order to identify aspects of the patient stay, which form an associative relationship with an overall rating of consumer satisfaction on a 10-point scale. During the survey period, there were 182 discharges. Seventy questionnaires (38.5%) were returned from this group. The survey results highlight a number of areas of identified need, enabling the service to prioritize organizational systems around meeting these needs. Multiple regression analysis identified three items in the survey, which were independently significant associates of overall consumer satisfaction. They included being happy with the service provided by the consumer support worker, having support for services on discharge and feeling safe and secure on the ward. The model containing these three items accounted for 50% of the variation in overall satisfaction. Two primary interventions have been developed because survey administration which, it is hoped, will address issues raised in the survey. The interventions were the development of an admission and discharge pathway and a ward-based psychosocial intervention programme, which includes the involvement of consumer support workers. Keywords: consumer involvement, inpatient units, mental health, patient satisfaction Accepted for publication: 31 October 2008

Introduction Consumer satisfaction is a widely used measure of the quality of service received by consumers (Cleary et al. 2003). Satisfaction with health services has been suggested to be an integral part of measuring the outcomes and performance of clinical service delivery. Previous 272

studies have shown that the perspective a patient has, regarding effective service delivery, often differs from that of clinicians, managers, legislators and family members. A patient’s perspective and experiences of care are increasingly understood to be central to the outcomes that are achieved (Howard et al. 2003). © 2009 Blackwell Publishing

Consumer satisfaction

Defining consumer satisfaction Consumer expectations play a significant part in determining satisfaction. Consumers’ prior beliefs about what they are entitled to and their expectations based on previous hospitalization or other healthcare event can be determinants of satisfaction (Linder-Pelz 1982). The World Health Organization identifies eight dimensions of responsiveness in assessing hospital performance. Responsiveness is conceptualized as the way in which individuals are treated within the care environment (Murray & Frenk 1999). These are autonomy, choice, communication, confidentiality, dignity, quality of basic amenities, prompt attention, access to family and community support. Linder-Pelz (1982) defines consumer satisfaction as ‘the individual’s positive attitude toward i.e. positive evaluation of the health care(s) he experienced’. The definition provided by Lebow (1982), in some of the earliest works around mental health consumer satisfaction, is used in this paper as an operational definition: ‘the extent to which treatment gratifies the wants, wishes and desires of the client for services’ (p. 72).

Studies of consumer satisfaction in mental health Edlund et al. (2003) telephoned 9585 European consumers on the relationship between satisfaction and technical quality of care with mental health services and found that ratings of technical quality of care were significantly associated with ratings of overall satisfaction. Gigantesco et al. (2002) in Italy studied 855 psychiatric consumers and 265 relatives of these consumers. Consumers reported higher levels of satisfaction than relatives. A perceived lack of transparency by staff was associated with lower overall satisfaction ratings. Cleary et al. (2003) in Australia studied mental health consumers’ discharge planning needs to ascertain consumers’ perceptions of helpful practice. The authors report that while the vast majority of consumers were satisfied with their overall hospital stay and discharge needs, consumers requested more contact with ‘consumer consultants’, and more information on medications, mental health problems and relapse prevention. Howard et al. (2003) in a study of two public psychiatric hospitals in North America (n = 204) found consumers were dissatisfied with their lack of involvement in treatment planning, lack of family involvement and lack of medication education. Howard et al. (2001) reported on the development of the Kentucky consumer satisfaction instrument and its use in 189 consumers with mental health disorders. Their analysis showed higher education levels and a bipolar disorder diagnosis as predictors of high © 2009 Blackwell Publishing

satisfaction. Consumers were most dissatisfied with the lack of involvement with their carers in the discharge planning process. Rosenheck et al. (1997), in a North American study of 4968 hospital discharges from mental health units, found the strongest and most consistent predictors of consumer satisfaction were older age and better self-reported health. Longer length of stay was also associated with greater satisfaction on a majority of their subscales. Gharabawi et al. (2006), in a study about medication-related satisfaction among people suffering with schizophrenia, report that satisfaction was positively associated with psychotic symptom improvement, suggesting satisfaction may be related to adherence. In consumers with mood disorders, individual and illness characteristics were found to be important predictors of consumer expectations that lead to consumer satisfaction reports (Wilhelm et al. 2005).

Study aim The aim of this study was to determine the level of consumer satisfaction with an adult acute inpatients mental health service and to cross-sectionally identify key associates of overall satisfaction from within the survey content domain.

Questionnaire development A questionnaire was developed in partnership with consumers and health staff by using a series of discussion groups, which were dedicated to the task of questionnaire design. The Australian National Mental Health Standards (1996) were used as a content guide during the group work-based question building exercises. The authors reviewed the national mental health standards and relevant literature on patient satisfaction. A draft questionnaire that reflected the national standards and contemporary literature was taken to a group of senior mental health professionals (nurse consultant, psychiatrist, psychologist and social worker), for judgement on relevance and applicability for use in an inpatient unit. Once the senior group achieved consensus on the four key domain areas (quality of care, staff, environment and services and discharge), these domains were then taken to a series of discussion groups (n = 3) with consumers, who were remunerated for their involvement. The final questionnaire content was decided upon within these discussion groups in an iterative manner. An inclusive facilitative style of group discussion was employed throughout the questionnaire development process. The questionnaire was designed to allow for domainbased subscales: quality of care, staff, environment/services and discharge. Questions were rated on a 5-point Likert273

S. Brunero et al.

Consumer satisfaction survey subscales

National Mental Health Standards Rights

Quality of Care

Staff

Environment and Services

Discharge

Consumer and Carer Participation Privacy and Confidentiality Prevention and Mental Health Promotion Cultural Awareness Safety Inpatient Care

Assessment and Review Treatment and Support Safety Consumer and Carer Participation Privacy and Confidentiality Delivery of Care

Delivery of Care Privacy and Confidentiality Safety

Planning for Exit Consumer and Carer Participation Community Living Exit and Re-entry Prevention and Mental Health Promotion Delivery of Care

type scale, from 1 strongly disagree, 2 disagree, 3 neither agree nor disagree, 4 agree to 5 strongly agree. An overall satisfaction rating of the hospitalization experience was included, and 10-point Likert-type scale was offered. Figure 1 illustrates the survey content domains.

Method All consumers on their day of discharge were asked to complete the survey. The study period was from 1 September to 30 November 2005. The study site was a 50-bed acute unit made up of two wards, the Observation Ward and the General Ward. The Observation Ward is a 16-bed secure admission ward that provides care for consumers 274

Figure 1 Questionnaire domains versus national mental health standards

who are in the acute phase of their illness. Most consumers admitted to the Observation Ward are being treated under the Mental Health Act. Consumers on the 30-bed General Ward and four-bed annex are both voluntary and involuntary. Ethics approval was obtained from the South Eastern Sydney Illawarra Area Health Service Research Ethics Committee (Northern Sector). Data were analysed by using spss™ version 11.5.

Results During the study period there were 182 discharges. Seventy questionnaires from this group were returned (38.5%). Table 1 outlines a description of the sample. The © 2009 Blackwell Publishing

Consumer satisfaction

Table 1 Subject characteristics

Age Length of stay

Men n = 27 mean (SD)

Women n = 43 mean (SD)

Total n = 70 mean (SD)

35.9 (10.8) 12 (9.3)

40.3 (11.8) 16.7 (13.8)

38.6 (11.6) 15.2 (12.3)

English speaking background Non-English speaking background Own home Rental home Hostel Friends’ home Family members’ home No fixed abode First psychiatric admission More than one psychiatric admission Had a primary carer Did not have a primary carer

n (%)

n (%)

n (%)

24 (88.9) 3 (11.1)

36 (83) 7 (16.3)

60 (86) 10 (14)

6 (22.2) 12 (44.4) 3 (11.1) 1 (3.7) 4 (14.8) 1 (3.7) 10 (37) 17 (63)

16 (37.2) 16 (57.1) 1 (2.3) 3 (7) 6 (14) 1 (2.3) 17 (39.5) 26 (60.5)

20 (28.6) 28 (40) 4 (6) 4 (6) 10 (14) 2 (3) 27 (38.6) 43 (61)

21 (77.8) 6 (22.2)

36 (83.1) 7 (16.3)

57 (81) 13 (19)

was rated well at 80% in agreement. Consumers who identified their carer as an immediate family member were more likely to rate ‘I was happy with the service provided by the welfare officers’ in the positive (x2 = 10.5, P = 0.03). Environment and services The environment and services subscale (Table 2) contained three items canvassing consumer feedback about environmental cleanliness (Item 17), privacy (Item 18) and food quality (Item 19). All three items attracted >75% satisfaction. Of the consumers whom it was their first admission, there was a negative response to the question ‘I found the ward clean and comfortable’ (x2 = 10.05, P = 0.04).

male : female ratio was 27:43, and mean age was 38.6 years. A majority of respondents had had a prior psychiatric admission, and the average length of inpatient stay was 15 days.

Discharge The items on the discharge processes contained the two highest satisfaction ratings yielded by the survey. Item 20 asked consumers whether they were involved and informed in the planning of their discharge. Eighty-seven per cent indicated agreement. Item 21 asked whether consumers felt confident about having enough support from mental health services upon discharge. Eighty-six per cent indicated agreement.

Satisfaction ratings

Overall satisfaction

Table 2 outlines all the items used in the questionnaire and displays median and mean rating scores and percentage in agreement with each statement. Percentage in agreement was calculated by adding the response to 4 (agree) or 5 (strongly disagree) on the Likert scale.

Consumers were asked to rate their overall satisfaction by marking a number from 1 to 10 (where 1 = very bad and 10 = excellent). A mean score of 6.8 resulted. Respondent characteristics and individual question items were assessed against the overall satisfaction rating by using univariate statistical procedures. Among the respondent characteristics, only the presence of a community case manager was significantly associated with higher overall patient satisfaction ratings (f = 4.507, P = 0.037). Among the individual questionnaire items, all excepting items 3 and 19 were significantly associated with the overall rating. All the significant associates of overall consumer satisfaction were entered into a linear regression model, against the 10-point scale of overall consumer satisfaction, in order to identify the strongest independent associates of the overall rating. Three items emerged as independently predictive of overall satisfaction. These were item 15 ‘I was happy with the service provided by the consumer support workers’ (b = 0.96, P < 0.001)), item 21 ‘I feel confident that I have enough support from the mental health service now that I have been discharged from hospital’ (b = 0.64, P = 0.007) and item 8 ‘I felt safe and secure on the ward (for instance, free from harassment)’ (b = 0.53, P = 0.022). The model containing these three items accounted for 50% of the variation in the overall satisfaction scale (adjusted r2 = 0.51).

Quality of care The quality of care subscale yielded some results that were suggestive of the need for action. Item 3 ‘My rights and responsibilities under the Mental Health Act were explained to me’ attracted the lowest agreement ratings (53% in agreement). Item 2 ‘The reason for my hospital admission was explained to me’ was rated at a moderately low 64% in agreement. Increasing age was associated with a positive response to item 7 ‘I was able to practise my cultural and religious beliefs in hospital’ (r = 0.2, P = 0.043). Staff Consumers were asked to rate their satisfaction with the service that the various disciplines provided (Items 9–15). Ratings were generally high. Nurses were rated at 80% satisfied, doctors 80%, welfare workers 73%, social workers 72%, psychologist 70%, occupational therapy 64% and consumer support workers 60%. Item 16 ‘The treating team dealt promptly with concerns and requests’ © 2009 Blackwell Publishing

275

S. Brunero et al.

Table 2 Consumer satisfaction questionnaire1 Median

Mean

Percentage in agreement

4

3.7

73

4 4

3.8 3.2

64 53

Quality of care 1. On admission or transfer I was shown around the ward, and the ward routine was explained to me. 2. The reason for my hospital admission was explained to me. 3. My rights and responsibilities under the Mental Health Act were given to me and explained (answer only if you were admitted against your wishes). n = 59. 4. I was involved in the decisions regarding my treatment throughout my admission. 5. I was given the choice to involve my family or other relevant people when deciding my care and treatment. 6. My medications were explained clearly to me and the possible side effects were discussed. 7. I was able to practise my cultural and religious beliefs in hospital. 8. I felt safe and secure on the ward (for instance, free from harassment).

4 4

3.7 3.6

71 70

4 4 4

3.7 3.7 3.7

69 67 71

Staff 9. 10. 11. 12. 13. 14. 15. 16.

4 4 4 4 4 4 4 4

3.9 3.9 3.8 3.8 3.7 3.8 3.7 3.8

80 80 73 72 64 70 60 80

4 4 4

3.8 3.9 3.7

76 82 76

I was happy with the service provided by my doctors. I was happy with the service provided by my nurses. I was happy with the service provided by the welfare workers. I was happy with the service provided by the social workers. I was happy with the service provided by the occupational therapists. I was happy with the service provided by the psychologist. I was happy with the service provided by the consumer support workers. My treating team dealt with my concerns and requests appropriately and promptly.

Environment and services 17. I found the ward clean and comfortable. 18. I was able to have privacy and space from others while on the ward. 19. I was able to have a choice of foods. Discharge 20. I was involved and informed in the planning of my discharge from hospital. 21. I feel confident that I have enough support from the mental health service now that I have been discharged from hospital. 22. I was made aware of services in the community that I can use. 23. My family/carers and other health professionals were involved in my discharge planning.

4 4

3.9 3.9

87 86

4 4

3.82 3.81

77 74

Overall rating2 How would you rate your overall experience in hospital?

7

6.82

Not applicable

1

Likert scale range from 1 (strongly disagree) to 5 (strongly agree). Likert scale rating from 1 (very bad) to 10 (excellent).

2

Discussion The survey has yielded a positive picture, with some areas of concern notable. The poor result in relation to consumers having their rights and responsibilities explained to them may be accounted for by the absence of written information being readily available, as well as consumer’s inability to retain information because of their mental status. In the follow-up to the survey, it was ensured that a comprehensive booklet regarding consumer’s rights and responsibilities was made readily available to all mental health consumers. The moderately low result with regards to the questionnaire item canvassing whether the reason for admission was explained is worrying. It reflects a mismatch between consumer expectation and service delivery and has been followed up in a workshop and discussion with unit staff. The feedback about unit staff was encouraging. Nurses and doctors are higher in number compared with other staff categories, and consumers are exposed to inter276

actions with these staff members more often. This may have led to the higher ratings these staff received compared with the less commonly encountered staff. Our study did not identify a strong link between socio-demographic characteristics and overall satisfaction. Adherence, general health perception and symptom improvement were not canvassed in our study. Encouragingly, freedom to practise religious and cultural beliefs was well rated, with some variation in mean rating by age and language group. Being able to practise one’s religious beliefs has been associated with better depression outcomes, lower suicide rates and better general overall wellbeing (Koenig 1998). Identifying the appropriate clinical areas and times in a ward programme for consumers to practise their religious beliefs should be routinely considered in mental health services. First admission was associated with a negative perception about cleanliness and comfort. This association probably demonstrates how consumer expectations play a part in satisfaction levels. The © 2009 Blackwell Publishing

Consumer satisfaction

notion of expectations being directly related to satisfaction reports was shown by Jackson et al. (2001), in their study of 500 consumers at a general medicine walk in clinic. The presence of unmet expectations in this study led to decreases in satisfaction reports. The multivariate regression of individual items and characteristics against the 10-point overall satisfaction scale highlighted three key associates of consumer satisfaction: being happy with the service provided by the consumer support workers, confidence regarding availability of support services upon discharge and feeling safe and secure on the unit. This result suggests that the admission– discharge nexus is a prime area where consumer satisfaction is either ‘won or lost’ by the mental health system. Two primary interventions have been developed which, it is hoped, will address some of the issues raised by the survey. First, an admission and discharge pathway has been developed. It was implemented in January 2006. The pathway identifies the three stages of the admission and discharge process: early admission, crisis stabilization and transition. It incorporates some key components of the survey: orientation to ward, reason for admission being explained, mental health act explanation, involvement in decisions during treatment and involvement of family members. The pathway checklist is available from the authors, by request. The pathway allows for an audit trail, with measurement against externally accepted national standards (Australian National Mental Health Standards 1996). A second intervention developed as a result of the survey is a ward-based group psychosocial intervention programme, involving nursing, allied health staff and consumer support workers. Clinical staff hope that this intervention will further improve consumers’ experience of service delivery, while promoting recovery and improved outcomes. Being happy with the service provided by the consumer support worker is highlighted as being significant in the regression analysis as a key associate of overall consumer satisfaction. This gave rise to discussion on how to better involve consumer support workers within the service. They are now actively involved in a weekly community meeting, which aims to provide a channel of communication between staff and consumers, to ascertain views on current and future service provision and to encourage consumers to take an active role in decision making processes related to the general functioning of the ward. The item of ‘feeling safe and secure on the ward’ is addressed in the ward community meeting. Community group forums allow for consumers to ventilate concerns over safety and security, while promoting communication and decision making regarding day-to-day operations and housekeeping issues (Lange & Bradley 2001). © 2009 Blackwell Publishing

Study limitation The authors were unaware of reasons for which consumers refused to complete the survey. Potential reasons may have been out of hours’ discharges, mental status or people may have been dissatisfied and therefore unwilling to complete the survey. Literacy levels, cultural and language barriers may also have contributed to the survey not being completed. The measurement of consumer satisfaction is often used to meet service providers’ needs and to rate the success of providers at meeting consumers needs. The notion of consumers judging their satisfaction of care by comparing their expectations of care with their prior expectations of health care has been questioned (Avis et al. 1997). Satisfaction surveys almost uniformly find high levels of satisfaction, with some doubts about their accuracy and sensitivity (Avis et al. 1997, Williams et al. 1998). Little is known about the confounding variables that may bias and or influence consumers expectations and reports of satisfaction. Variables such as health beliefs, prior experiences, knowledge levels and cultural attitudes may all influence this, but are hard to control for in survey type studies. Consumer satisfaction surveys can force consumers into a set structure of questions and reasoning which may bias their answers. Systematic biases have been noted in survey results – for example, older consumers are generally more satisfied with their hospital experience than younger consumers; consumers with lower socio-economic circumstances are generally more satisfied than wealthier consumers. Future studies should consider whether high ratings of consumer satisfaction are related to superior clinical outcomes.

Conclusions From the survey results, interventions have been developed, which are hoped to further improve the service and enhance consumer outcomes. This study has shown that consumers with mental health problems can influence service redesign and quality improvement. This study has demonstrated how mental health consumers can work collaboratively with clinicians for better outcomes.

References Australian National Mental Health Standards (1996) National Mental Health Standards. Australian Government Department of Health and Family Services, Canberra. Avis M., Bond M. & Arthur A. (1997) Questioning consumer satisfaction: an empirical investigation in two out consumer clinics. Social Science and Medicine 44, 85–92. Cleary M., Horsfall J. & Hunt G. (2003) Consumer feedback on nursing care and discharge planning. Journal of Advanced Nursing 42, 269–277.

277

S. Brunero et al.

Edlund M.J., Young A.S., Kung F.Y., et al. (2003) Does satisfaction reflect the technical quality of mental health care? Health Services Research 38, 631–645. Gharabawi G., Greenspan A. & Rupnow M. (2006) Reduction in psychotic symptoms as a predictor of consumer satisfaction with antipsychotic medication in schizophrenia: data from a randomised double-blind trial. BMC Psychiatry 6, 1– 7. Gigantesco A., Picardi A., Chiaia E., et al. (2002) Consumers and relatives; satisfaction with psychiatric services in a large catchment area in Rome. European Psychiatry 17, 139–147. Howard P.B., Clark J.J., Rayens M.K., et al. (2001) Consumer satisfaction with services in a regional psychiatric hospital: a collaborative research project in Kentucky. Archives of Psychiatric Nursing 15, 10–23. Howard P.B., El-Mallakh P., Kay Rayens M., et al. (2003) Consumer perspectives on quality of in consumer mental health services. Archives of Psychiatric Nursing 17, 205–217. Jackson J., Chamberlain J. & Kroenke K. (2001) Predictors of consumer satisfaction. Social Science and Medicine 52, 609– 620. Koenig H. (1998) Religious attitudes and practices of hospitalised medically ill older adults. International Journal of Geriatric Psychiatry 13, 213–224.

278

Lange C. & Bradley J. (2001) Community meetings on a military in consumer psychiatric unit: a question of balance. Military Medicine 166, 48–52. Lebow J. (1982) Consumer satisfaction with mental health treatment. Psychological Bulletin 91, 244–259. Linder-Pelz S. (1982) Social psychological determinates of consumer satisfaction: a test of five hypotheses. Social Science & Medicine 16, 583–589. Murray C. & Frenk J. (1999). A WHO Framework for Health System Performance Assessment. In GPE Discussion Paper Series: No.6. World Health Organization, Geneva. Available at: http://www.who.int/health-systems-performance/docs/articles/ paper20.pdf (accessed 5 January 2007). Rosenheck R., Wilson N.J. & Meterko M. (1997) Influence of consumer and hospital factors on consumer satisfaction with in consumer mental health treatment. Psychiatric Services 48, 1553–1561. Wilhelm K., Wedgwood L., Malhi G., et al. (2005) Great expectations: factors influencing consumer expectations and doctors recommendations at a Mood Disorders Unit. Journal of Affective Disorders 88, 187–192. Williams B., Coyle J. & Healy D. (1998) The meaning of consumer satisfaction: an explanation of high reported levels. Social Science & Medicine 47, 1351–1359.

© 2009 Blackwell Publishing