A Quality Framework to Guide Measurement ...

A Quality Framework to Guide Measurement, Improvement and Research

Dr. Nadiya Sunderji, Dr. Abbas Ghavam-Rassoul, Allyson Ion, Dr. Elizabeth Lin

A Quality Framework to Guide Measurement, Improvement and Research

Suggested citation: Sunderji, N., Ghavam-Rassoul, A., Ion, A., and Lin, E. “Driving improvements in the implementation of collaborative mental health care: A quality framework to guide measurement, improvement and research.” 2016. Toronto, Canada. Contact: [email protected]

CONTENTS EXECUTIVE SUMMARY

1

BACKGROUND 3 The Role for Collaborative Mental Health Care

3

Problems with Collaborative Care Implementation

3

A Quality Framework to Drive Improvement in Collaborative Care

4

METHODOLOGICAL APPROACH

6

SYSTEMATIC LITERATURE REVIEW

7

Method 7 Systematic Review Results QUALITATIVE STUDY

9 11

Method 11 Qualitative Study Results

12

RESEARCH TEAM SYNTHESIS

17

ADVISORY GROUP

17

Method For Modified Delphi Consensus Process

18

Advisory Group Contributions and Modified Delphi Results

19

QUALITY FRAMEWORK

22

Domains of Collaborative Care Quality

22

Specific Dimensions Of Collaborative Care Quality

24

DISCUSSION 30 Strengths and Limitations of the Methodology

30

Strengths and Limitations of the Framework

30

Implications 31 Future Directions

34

RESEARCH TEAM

36

ACKNOWLEDGMENTS AND DISCLOSURES

37

Advisory Group Members

37

REFERENCES 39 APPENDIX A Example Search String (Medline Search)

43

APPENDIX B Collaborative Care Quality Framework Domains’ Alignment with IoM Domains

44

EXECUTIVE SUMMARY Access to specialty mental health care is poor for many Canadians experiencing mental illness and/or addictions. Collaborative mental health care models can improve access to quality mental health care in primary care settings, and have demonstrated effectiveness, cost effectiveness, and population impact. However, implementation of Collaborative Care is highly variable and may not conform to evidence-based practice: the most rigorously tested models have been inconsistently implemented and other untested models have been implemented, leading to uncertainty regarding the key ingredients of Collaborative Care. We developed a quality framework that can guide Canada’s primary care organizations towards measuring and improving key domains and dimensions of Collaborative Care. This new quality framework for collaborative care uniquely draws upon widely accepted generic quality frameworks from Donabedian and from the Institute of Medicine to ensure comprehensiveness, coherence, relevance and transferability. Furthermore, it represents a synthesis of multiple forms of knowledge production to address issues of efficacy (e.g. based on empirical studies), as well as implementation and effectiveness in real-world settings (e.g. based on qualitative research and expert consultation), to advance an understanding of core components of Collaborative Care and how to transfer this complex intervention into different contexts. We developed the framework in three steps. First, we conducted a systematic review of published and unpublished literature to identify, critically appraise, and thematically analyze existing measures of Collaborative Care implementation and outcomes. Second, we interviewed health care providers and clientsaa regarding their experiences with Collaborative Care. Provider interviews were guided by the Theoretical Domains Framework, which addresses evidence-to-practice gaps, barriers and enablers of changing organizational behaviour, potential interventions to address modifiable barriers and enablers, and meaningful outcome measures. Client interviews explored experiences of Collaborative Care, perceived strengths and drawbacks, and recommendations for change. Third, we convened an advisory group of diverse experts, and through an iterative process, and a combination of inductive and deductive approaches, we developed a consensus on the essential domains and dimensions of Collaborative Care quality for inclusion in the quality framework. We propose a quality framework with 11 broad domains and 52 specific dimensions of quality that includes: structural measures; care processes pertaining to collaboration, integration, and quality of care; and outcomes at the individual, population and health care system levels. The framework can drive improvement by: a) guiding comprehensive and balanced program evaluation, b) providing a menu from which organizations can select a specific focus for quality improvement, and c) informing the selection of measures for future research evaluating Collaborative Care interventions. In the next stage of the project we are developing and testing quality measures in several primary care teams across Ontario, exploring ways of co-developing measures with clients, using the measures to support local quality improvement projects, and exploring the transferability of the framework to other jurisdictions. Given its foundation in the widely adopted Donabedian and Institute of Medicine quality frameworks, and its comprehensive synthesis of diverse stakeholder perspectives represented in empirical studies, qualitative methods, and group deliberative processes, this new quality framework for Collaborative Care will complement recent compendia of measures, and will be relevant and useful in any milieu where there is an interest in improving the quality of primary mental health care.

a Our research team and advisory group considered various terms that have been proposed to describe individuals receiving mental health care, including patients, clients, consumers, service users, and people with lived experience of mental illness. We decided to use the term “client” as an inclusive term that includes caregivers and families to represent individuals who have received collaborative mental health care. We discuss and incorporate the widely accepted terms “patient-centredness,” “patient safety”, “patient reported outcome measures”, “patient reported experience measures” and “patient centred medical home” in this report.

DRIV ING IM PROV EM ENT S IN THE IM PLEM ENTATION OF COLL A BOR ATIV E M ENTA L HE A LTH C A RE

1

2


BACKGROUND THE ROLE FOR COLLABORATIVE MENTAL HEALTH CARE Mental health and substance use disorders are the leading cause of years lived with disability worldwide (22.9% of all non-fatal disease burden).1 However, in Canada, as in many other parts of the world, detection and treatment of mental illness is low or delayed, and is hampered by shortages of trained psychiatric and other mental health providers, inefficient use of existing health human resources and financial resources, and stigma leading to avoidance of mental health care utilization.2–7 Primary care is the first and continuing contact for most Canadians, and many mental health problems are managed solely or primarily in primary care.8,9 In the past 15 years, Canada has undergone extensive primary care reform, introducing patient centred medical homes (PCMHs) and enabling: a) comprehensive multidisciplinary care, including for mental health; b) expanded access (e.g. after-hours care, potentially open-access scheduling, and diversified methods of communication with clients); c) routine quality measurement and improvement; and d) updated health information technology (e.g. electronic health records).10–12 Derived from Wagner’s chronic care model, Collaborative Care involves trained mental health specialists supporting care managers and primary care providers to deliver evidence-based mental health care guided by patient reported outcome measures (PROMs) to a defined population (e.g. by monitoring population outcomes in a clinical registry, delivering stepped care, and addressing social determinants of health).9,13–16 Numerous systematic reviews and meta-analyses have demonstrated that Collaborative Care improves access to mental health care, clinical outcomes and cost effectiveness of care. 4,15,17–19 Collaborative Care is a promising approach to reducing the population burden of mental illness, and is a key component of provincial mental health and addictions strategies and Canada’s vision for primary care.2,4,8,20–27

PROBLEMS WITH COLLABORATIVE CARE IMPLEMENTATION However, Collaborative Care implementation in Canada’s organized primary care settings is highly variable, does not conform to evidence-based practice, and has not been evaluated.9,12,28 Key components of the most rigorously studied models of Collaborative Care have been infrequently and inconsistently implemented.12,29–32 This could be due to a historically limited articulation of the active ingredients of Collaborative Care (by researchers) or disregard of such knowledge when developing local initiatives (by potential knowledge users), and/or the difficulty transferring complex interventions into contexts other than the one where they were developed. Instead, less well studied models of “shared care” or “integrated care” have been implemented, whereby specialist integration varies along a spectrum from parallel specialist and primary care practice (i.e. essentially no integration) to more collaboration between providers (e.g. with regular case conferencing and joint appointments), yet even the latter models have not used population registries, care management, or other components of the chronic care modelb.9,12,28 This has contributed to confusion about what are the essential components or functions of Collaborative Care that are applicable to any clinical context, as well as a continuing knowledge gap regarding the comparative effectiveness of the different practices that teams and organizations may use to achieve these functions.

b Furthermore, inequitable access to Ontario’s new primary care models, may have resulted in the exclusion of people with mental illness and addictions from preventive care and chronic disease management initiatives, and services delivered by interprofessional healthcare providers.10,30,31


3

Evidence-to-practice gaps have also been noted in other primary care settings, where organizational, financial and attitudinal barriers hinder the delivery of Collaborative Care.23,33–36 In turn, poor or incomplete implementation of Collaborative Care contributes to poor integration of physical and mental health care, inappropriate variation in clinical care, non-guideline-based pharmacotherapy, delayed follow up after treatment initiation, treatment drop-outs and less improvement in symptoms.33–36 In order to meet Canada’s population mental health needs, it is vital that we develop a shared understanding of the critical components of Collaborative Care that require consistency, and identify and close gaps in implementation of Collaborative Care in primary care practice.

A QUALITY FRAMEWORK TO DRIVE IMPROVEMENT IN COLLABORATIVE CARE Quality frameworks and measuresc provide clarity regarding standards of care and heighten focus on opportunities to improve care in ways that are meaningful to clients and families.38–40 Meaningful and valid quality measurement has been identified as key to improving Collaborative Care implementation, yet there is scant literature on quality frameworks and measures by which to evaluate Collaborative Care in primary care settings.21,23,41,42 Some researchers have attempted to define dimensions of quality of primary mental health care, however, with limited consideration of Collaborative Care practice and evidence. 43,44 Other efforts have focused on integrated care, however they have been limited by problems with the measures (e.g. a small number of measures focused on care processes for single diseases, or generic measures “applicable to health care in general”), and by the lack of a coherent framework to guide the selection of measures. 23,45,46 We therefore sought to develop a quality framework, measures, and a measurement strategy for collaborative mental health care in PCMHs across Canada. These products will serve as a critical first step towards improving Collaborative Care in primary care by identifying the key domains and dimensions of Collaborative Care that, if well implemented, are likely to positively impact outcomes at the individual, organizational, population, and health care systems levels. In this first project, we have developed a novel quality framework to guide the subsequent development of specific measures. Importantly, this Collaborative Care framework is grounded in widely accepted frameworks for health care quality that can ensure relevance, comprehensiveness, transferability, and ultimately accountability. Donabedian’s quality framework states that the organization and structure of the health care delivery system shapes health care processes, which in turn affect outcomes (e.g., clinical outcomes and health service utilization).38,40 The Institute of Medicine’s (IoM) seminal report on quality of care identified the following six aims for health care: safety, effectiveness, patient-centredness, timeliness, efficiency, and equitability. 47 More recently Health Quality Ontario (HQO) adopted these same six domains. 48 Health system characteristics affect organizations’ abilities to deliver care that meets individual and community health needs. Thus, our quality framework includes structural measures that assess resources used (e.g. through organizational reporting rather than measuring) to aid in determining whether the conditions under which care is provided support or detract from the delivery of good care. 49 Structural components may also be more feasible to measure than some of the complex care processes that they are intended to enable.50 For example, intricate processes of communication and coordination that require role changes, personal investment and different types of working relationships are critical to Collaborative Care c Quality indicators (or quality measures) have been defined as “population-based measure[s] that enables users to quantify the quality of a specific aspect of care by comparing it to evidence-based criteria. Indicators require defining both those patients whose care meets the indicator criteria (the numerator) and those who are eligible for the indicator, or the population of focus (the denominator).”37

4


functioning, yet may be difficult to measure. However, structural measures must be used with some caution due to the limited evidence to link specific structures to outcomes of interest, as well as the potential gap between organizations’ self-assessment and their actual performance. This quality framework defines the ‘key ingredients’ of high quality Collaborative Care and has the potential to guide the development of Collaborative Care across Canada for at least the next decade. Thus, to ensure a sound guide, we specifically chose to include domains and dimensions of Collaborative Care quality that may not be feasible to measure at present, which will highlight important areas for further development. A coherent and comprehensive strategy for assessing quality of Collaborative Care will pave the way for quality improvement and translational research, and ultimately help eliminate the “quality chasm” between evidence and practice. 23,47


5

METHODOLOGICAL APPROACH We aimed to ensure the quality framework, measures and measurement strategy are informed by the best evidence. Because no single type of evidence would identify all the important elements required for widespread implementation of high quality Collaborative Care, we deliberately included diverse methods of knowledge production. Empirical studies identified in our literature search addressed Collaborative Care efficacy, and our qualitative study and expert consultation addressed issues of implementation and effectiveness in real-world settings. Synthesizing these disparate sources of knowledge for the first time will advance an understanding of core components of Collaborative Care and how to transfer this complex intervention into different contexts. Accordingly, we asked the following research questions of our knowledge sources:

1

PEER-REVIEWED AND GREY LITERATURE. What are the key components of efficacious Collaborative Care? How have Collaborative Care practices and outcomes been evaluated to date?

2

QUALITATIVE INTERVIEWS. How has Collaborative Care been implemented in real-world primary care settings? What are the barriers and enablers of transferring empirically tested models into our local context? What do primary care providers, mental health care providers, and clients prioritize as areas for measurement and improvement?

3

RESEARCH TEAM SYNTHESIS. How can we organize the many found quality measures into an initial framework that builds upon existing generic frameworks of health care quality, clearly communicates core principles and critical functions of Collaborative Care, and reflects diverse perspectives that are integral to the successful implementation of these complex interventions?

4

KNOWLEDGE TO ACTION ADVISORY GROUP OF EXPERTS. How should we design our study and interpret our findings to elicit and hear essential perspectives? Can ongoing dialogue and engagement with diverse stakeholders representing multiple sectors and levels lead to a consensus on the necessary quality domains and dimensions for a final framework that is comprehensive, balanced, and reflects wellconsidered priorities?

This study was approved by the St. Michael’s Hospital Research Ethics Board (protocol 14-318).

6


SYSTEMATIC LITERATURE REVIEW METHOD The systematic review inventoried all existing and recommended measures evaluating collaborative mental health care delivered in primary care settings. The review also thematically analyzed the found measures to explore the key functions of Collaborative Care that other authors have emphasized in evaluating program performance. There are almost no quality measures outlining evidence-based standards of Collaborative Care that all programs should adhere to and outlining how attainment of the standards should be measured. Thus, we conducted a broad search for ways of measuring Collaborative Care implementation and outcomes. We developed a study protocol a priori.51 We included peer-reviewed, published literature and grey literature that met all of the following criteria: a) described mental health care provided to any age group and for any condition, in a primary care setting; b) described a collaborative mental health care model, e.g. consistent with the typology / parameters described by the Agency for Healthcare Research and Quality (AHRQ)42,52 , for example, consultationliaison, integrated care, or chronic care models; c) described any measures implemented or proposed to assess the quality of Collaborative Care. We identified: English language literature indexed in the electronic databases Medline, EMBASE, PsychINFO, CINAHL and PubMed prior to July 3 2014 (see Appendix A for an example search string); grey literature retrieved through Google searches and the websites of pertinent organizations and conferences, and; relevant references cited by included studies (see Figure 1 below for Preferred Reporting Items for Systematic Reviews and Meta-analyses, PRISMA diagram).53,54 Two research team members independently conducted abstract screening, followed by full text screening


7

FIGURE 1 –PRISMA FLOW DIAGRAM

Total Sources Found (n=3997): • Search Strategy (to 03Jul2014) – n=3656 U Medline = 2392, Embase = 980, CINAHL = 154, PsychINFO = 130 • Grey Literature – n=274 • Reference Check of Included Studies – n=48 • Reference Check of Grey Literature Sources – n=19

Duplicates (n=217) • Search Strategy – n=204 • Grey Literature – n=13

Total Sources Screened (n=3780): • Search Strategy – n=3452 • Grey Literature – n=261 • Reference Check of Included Studies – n=48 • Reference Check of Grey Literature Sources – n=19

Excluded = 3356 (Did not meet screening criteria, duplicates)

Sources Forwarded to Full Text Screening - n=424 • Search Strategy – n=303 • Grey Literature – n=67 • Reference Check of Included Studies – n=35 • Reference Check of Grey Literature Sources – n=19

Excluded = 227

Sources Included for Data Extraction = 197 • Search Strategy – n=76 • Grey Literature – n=67 • Reference Check of Included Studies – n=35 • Reference Check of Grey Literature Sources – n=19

Data not extracted from n=25 • Systematic reviews – n=24 • Data not available – n=1

Independent Sources Extracted From = 172 • Search Strategy – n=72 • Grey Literature – n=50 • Reference Check of Included Studies – n=31 • Reference Check of Grey Literature Sources – n=19

8


Using a structured data collection form we extracted: a) study or report characteristics; b) population and setting; c) measures used or proposed to evaluate Collaborative Care, and d) details of measures if available (e.g. data source, measurement method, and implementation). Given our goal of constructing a quality framework and quality measures, we critically appraised each found measure in the literature with respect to characteristics of good measures, using a scale developed by Stelfox and Strauss based on instruments from the AHRQ and RAND.55,56 We hand searched article reference lists and conducted targeted searches of primary literature where necessary to enable critical appraisal, for example to ascertain the reliability or validity of a previously published measure. Following data extraction, we conducted a qualitative analysis and synthesis of the Collaborative Care implementation and outcome measures in two stages.57,58 First, we used content analysis to code and group all extracted measures into a final list of unique measures, with careful consideration of the aggregation versus uniqueness of each measure. This stage was primarily led by one author (AI), in regular consultation with the lead author of this report (NS).d We used descriptive statistics to summarize the frequency of different types of measures in the literature (i.e. by Donabedian and IoM domain) and the overall quality of the measures per the critical appraisal. Second, through a thematic analysis and using the constant comparative method we inductively grouped the unique measures into broad domains and specific dimensions of collaborative care program functioning that authors have identified as important to measure program performancee.59 The lead author (NS) primarily conducted the thematic analysis stage in regular consultation with the research team. Thus, the content analysis captured the frequency with which particular measures appeared in the literature, whereas the thematic analysis explored the ability of different themes to describe the core characteristics of Collaborative Care programs and their functioning.58

SYSTEMATIC REVIEW RESULTS We identified 3,997 literature sources, of which 197 met screening criteria of implementing or proposing measures of Collaborative Care in primary care settings. For sources that were literature reviews (n=24) we extracted measures from the primary studies only, and for one source there was insufficient data. From the remaining 172 sources, we extracted 1,255 measures, which following content analysis, were condensed into 148 unique measures. Existing literature is heavily weighted toward evaluation of: a) individual clinical outcomes, such as depression symptom severity, health status and level of function; b) cost effectiveness, such as incremental cost of reducing depression symptoms or increasing quality-adjusted life years (QALYs), and; c) evidencebased care processes, such as appropriateness and adequacy of pharmacotherapy for a specific illness condition. Thus, there is a vast emphasis on effectiveness and efficiency (IoM domains) and a roughly equal emphasis on care processes and outcomes (Donabedian framework) (see Table 1). Throughout the entire literature search we did not locate any measures of patient safety, and few measures of equitability, accessibility or timeliness of care (e.g. addressing disadvantaged populations, stigma, and wait times).

d For example, we grouped the following found measures: a) clients are offered choices of treatment modalities, b) clients receive copies of their records, and c) care considers health literacy into a unique measure of “client engagement in their own care, e.g. active participation in care and treatment plan”. e For example, the unique measure “client engagement in their own care, e.g. active participation in care/treatment plan” informed the development of several themes including client centredness and chronic care model (subtheme informed, activated client).


9

Several measures were identified that, from our perspective, did not align with any existing IoM domains of quality but rather focused on the provider experience (e.g. buy-in, trust, confidence, turnover); these were categorized in a novel domain that we termed “culture of health care”. Client perspectives on Collaborative Care implementation and outcomes were grossly absent from the literature. Client-centredness was measured through unidimensional scales of satisfaction with care (also known as patient reported experience measures, PREMs), and through economic impact on clients (e.g. out of pocket costs of care, financial or housing status, employability). However, satisfaction was not ‘unpacked’ to understand the client experience, nor was it clear how authors had selected these economic outcomes to represent those important to clients. TABLE 1 – NUMBER OF FOUND MEASURES IN THE LITERATURE, BY IOM AND DONABEDIAN DOMAINS DOMAIN

STRUCTURE

PROCESS

OUTCOME

Effective

16

46

33

Efficient

2

1

18

Patient Centred

0

5

13

Timely

0

4

0

Safe

0

0

0

Equitable

0

1

2

Culture of health care f

3

2

2

Found measures varied greatly in their quality, with the strongest being validated patient-reported outcome measures (PROMs) of individual treatment effectiveness, such as mental symptoms, physical symptoms (e.g. pain), level of function and quality of life. Other measures were limited by imprecise definition, lack of evidence for reliability or validity, lack of risk adjustment (for outcome measures), and/or high measurement burden (e.g. requiring chart review).

f Culture of health care refers to measures that did not align with any existing IoM domains of quality but rather focused on the provider experience including buy-in, trust, confidence, turnover, etc.

10


QUALITATIVE STUDY As noted earlier, front line perspectives are needed to understand gaps in the adoption of evidencebased practices, as well as to understand outcomes that are important to different stakeholders. Local contextual factors have significant influence on the implementation, impact, and scalability of complex interventions, yet are often under-recognized and under-reported in the literature.60–63 Transferring successful interventions to different contexts requires engaging new perspectives, teams and settings, as well as balancing standardization and adaptation of care models and processes.64 Therefore, we conducted in-depth interviews with health care providers who had participated in delivering Collaborative Care. As it became evident that client perspectives were under-represented in the literature our advisory group aided us in revising our study design to incorporate interviews with clients who had received Collaborative Care.

METHOD Our approach was informed by the Theoretical Domains Framework, which addresses evidence-to-practice gaps, barriers and enablers of changing organizational behaviour, perceived feasibility and acceptability of interventions, and meaningful outcome measures.65 We sought health care providers’ experiences and opinions of: how Collaborative Care has been implemented in their primary care setting; what constitutes good quality of Collaborative Care, and; what would be required in order for organizations to implement effective Collaborative Care programs. We sought clients’ experiences, perceptions and opinions of: receiving Collaborative Care; strengths and drawbacks (including by comparison or contrast with other experiences of mental health care); recommendations for evaluation, and; recommendations for change. Participating health care providers were family physicians, psychiatrists, interprofessional health care providers or administrators in primary care teams throughout Toronto. We used purposive sampling to identify and engage information-rich cases that shed light on the questions under study. Eligible clients had received collaborative mental health care (e.g. consultation with a co-located psychiatrist, providerto-provider consultation and support, or intensive team-based care) and were recruited by their family physicians at two primary care teams in Toronto. Data collection and analysis were iterative, and we continued data collection until reaching informational saturation (i.e. no new emerging themes). The interviews followed a semi-structured interview guide, ranged from 29-57 minutes in length, and were conducted from December 2014 to October 2015 by a Research Coordinator (AA). All interviews were audio-recorded, professionally transcribed, and retained until the end of the study. We initially conducted qualitative content analysis to identify health care providers’ and clients’ experiences of Collaborative Care as implemented in primary care settings, factors influencing feasibility, uptake, and acceptability of these complex interventions, and recommended areas for quality measurement and improvement (categorized by Donabedian and IoM domains of quality). 40,47,65,66 Three transcripts were selected by the Research Coordinator (AA) from the initial 6 interviews conducted based on their richness, uniqueness and likelihood to generate a variety of codes. Three research team members (NS, GJ, AA) independently read and generated codes from these three transcripts then met and compared codes to develop an initial codebook, which was then used by at least two team members to code each remaining transcript. We met regularly and added, revised, merged or deleted codes as needed. Transcripts and codes were organized using NVivo10 software. We then conducted a thematic analysis to uncover patterns of meaning across our dataset and explore convergent and divergent themes across health care provider and client participants, including by examining frequency of codes for each participant group.67,68


11

QUALITATIVE STUDY RESULTS We conducted 23 in-depth interviews: 14 with providers of Collaborative Care and 9 with clients (see Tables 2 & 3 for participant characteristics). TABLE 2 – DEMOGRAPHIC CHARACTERISTICS FOR PROVIDER INTERVIEWS (N=14) CHARACTERISTIC

MEAN (RANGE)

Number of years working in Collaborative Care

10.1 (0-25)*

Number of years working at organization

9.1 (2-25)

PROFESSION

PERCENT (NUMBER)

Social Worker

21% (3)

Family Physician

29% (4)

Psychiatrist

29% (4)

Nursing

14% (2)

Executive Director

7% (1)

* One key informant was an expert on health system performance measurement who did not work in Collaborative Care.

TABLE 3 – DEMOGRAPHIC CHARACTERISTICS FOR CLIENT INTERVIEWS (N=9) CHARACTERISTIC

PERCENT (NUMBER)

AGE GROUP 25-34

11% (1)

35-44

22% (2)

45-54

22% (2)

55-64

44% (4)

GENDER Female

67% (6)

Male

33% (3)

RACE/ETHNICITY

12

White or Caucasian

89% (8)

Black or African Descent

11% (1)


As we anticipated, through the interviews health care providers and clients often discussed concepts related to quality of Collaborative Care that were not represented in the systematic review of the literature, including structural facilitators and barriers to real-world implementation of Collaborative Care models, collaborative interpersonal and interprofessional processes of care, and numerous dimensions of the client experience (see Table 4 for counts of quality measures suggested by interview participants and mapped to Donabedian and IoM domains). For example, providers frequently discussed physical space, time, staff roles, skills, training, and leadership as structures that facilitated or posed barriers to Collaborative Care implementation and effectiveness. Clients offered new ways of understanding wait times for Collaborative Care services, with heightened attention to: a) services offered while waiting for specialist care, and b) waiting for recommended care (e.g. psychotherapy) after specialist assessment. Clients also indicated that client centredness includes honouring the client’s desire to give back, and meaningfully engaging clients in the design, delivery, and improvement of Collaborative Care. Thus, the qualitative study yielded important new information about the elements required for successful transfer and scale up of evidence-based models of Collaborative Care (including to ensure acceptability to health care providers and clients), and crucial insights regarding experiences and outcomes of care that matter to clients. TABLE 4 – NUMBER OF SUGGESTED MEASURES IN INTERVIEWS, BY IOM AND DONABEDIAN DOMAINS DOMAIN

STRUCTURE

PROCESS

OUTCOME

Effective

6

8

7

Efficient

1

1

3

Patient Centred

2

7

17

Timely

3

3

1

Safe

2

4

1

Equitable

3

1

1

Culture of health care

3

2

5

Providers and clients’ perspectives converged on the importance of five themes representing the ‘what’, ‘how’ and ‘why’ of Collaborative Care implementation, yet they had diverse – and at times divergent – ideas about how each theme manifests in Collaborative Care. The five themes that emerged from our interviews were: a) Co-Location of Care; b) Continuity of Care; c) Team Composition and Functioning; d) Client Centredness; and e) Comprehensive Care for Individuals and Populations (see Table 5).


13

HEALTH CARE PROVIDER AND CLIENT PERSPECTIVES ON COLLABORATIVE MENTAL HEALTH CARE

CO-LOCATION OF CARE ɚɚ Facilitates timely and flexible communication ɚɚ Enables coordination of care ɚɚ Reduces stigma ɚɚ Increases acceptance of mental health support ɚɚ Provides reassurance about illness and care “We now have a psychiatrist…who sees patients onsite…[I] have found to be helpful because we know where she is physically so I can quickly touch base with her, get her opinion, she comes to our team meetings so we can discuss things in a more informal way.” (Provider)

“The fact that it’s in one group I find very reassuring. I can go to the same office; I can go to the same location. I don’t have to experience a new subway stop. I don’t have to know that my information has been transferred to the person I’m about to see. The security and the safety and the familiarity of being in the same environment is very reassuring and calming for me. It’s not scary out there. It’s somewhere that I’m safe.” (Client)

CONTINUITY OF CARE ɚɚ Informational continuity ɚɚ Relational continuity ɚɚ Management continuity ɚɚ Systems-level continuity ɚɚ Facilitated by shared electronic health records (EHRs)

“I know that it works because I feel looked after and I feel cared for. I know that I will not fall through the cracks and that’s the biggest complaint with health care out there - something was missed, somebody didn’t follow up, there was a bad result but nobody told me… I don’t believe that would, could ever happen here and that is unbelievably reassuring.” (Client)

“I think there would have to be a mix of formal and informal opportunities to communicate and share information about clients, right. There has to be something that is responsive to in-the-moment needs, as well as things that can be discussed and mulled over and responded to over time.” (Provider)

TEAM COMPOSITION AND FUNCTIONING ɚɚ Offers different lenses on the same problem and choices for clients ɚɚ Enacts mutual trust and respect ɚɚ Includes broad membership, e.g. clients as members of their own care team, receptionists ɚɚ Fosters knowledge exchange and shared decision-making “If you’re going to see a social worker they’re going to have a certain idea… and a psychiatrist is going to have a certain idea… but if they’re able to communicate and share their knowledge instead of, you know, deciding one way is better than the other you get…overall better care because you’re having different influences… available to you and none of them are being shot down…it’s available for you and you can sort of choose what to, to leave and what to take.” (Client)

14

“To use the variety of expertise around… to surround the patient with the services that they need, to have everybody operating within their scope of practice, and, you know, being used to the best of what their role is to, [and] to figure out a way of having those things flow.” (Provider)


CLIENT CENTREDNESS ɚɚ Meaningfully engages clients in their own care (e.g. shared decision making, defining important outcomes) “She’s always patient with me, she’s always listening and I feel like we have a dialogue where we’re deciding … how we’re going to take care of my health together which is nice so it’s better that way… I feel like she’s interested in my input and we can… make the decision together.” (Client)

“Another goal of Collaborative Care is to have the patient have more of a voice, that the patient can set the agenda, can have copies of their records, can inform a coordinated care plan, that you know everything that happens starts from the basic premise of ‘what’s important to me in my health care’ so collaboration should not disempower the patient or have them removed from the experience. I think they ought to be central to the whole notion of collaboration.” (Provider)

COMPREHENSIVE CARE FOR INDIVIDUALS AND POPULATIONS ɚɚ Whole person care to individual clients – Integrated physical and mental health care – Care that addresses the social determinants of health ɚɚ Care that is equitable ɚɚ Population-based care

“I like the team because they’re not only helping… your own specific problems but they also try to find out if you have any other problems that contributes to your main problems and try to find solution for every direction of your problem; especially in the mental health you need people like that you know…It’s not only your medication that can cure you… you need somebody to understand the issue or the problem you have.” (Client)

“I wonder if there is a barrier that we’re not even aware of in terms of are we missing any clients who could use this, and… are people falling through the cracks, because the way it’s structured it’s based on individual assessments and client initiated requests… [versus systematic screening of the population].” (Provider)

“After all my head stuff started to get straight I started to notice other things in my body that were going wrong that I hadn’t noticed before. They’re really good at just keeping stuff up to date and asking, like for example [about whether I completed a scheduled test].” (Client)

“First you need to do a needs assessment in terms of the patient population you serve. Do we need addiction counsellors? Do we need income stabilizers? What other disciplines do we need other than nursing and physicians and that’s assuming we need both. How many do we need? So having a staffing complement that meets the needs of the community and the patient population you’re serving.” (Provider)


15

These five themes are interconnected and multi-determined. For example, for providers, co-location of mental health and primary care providers facilitates timely communication and effective care coordination, thus contributing to improved informational and management continuity of care. For clients, co-location of mental health and primary care providers enabled their continuous and trusting relationship with their primary care provider to foster a vicarious trust in other co-located team members and alleviated apprehension about seeking mental health care. Similarly, when team composition and functioning includes the client as a vital member of their own care team and incorporates knowledge exchange and shared decision making as core processes informing care planning, this: a) increases the coherence of care plans and complementarity of services (i.e. management continuity and whole person care), and b) meaningfully engages clients in their own care and in defining priorities for outcomes (i.e. aspects of client centredness).

16


RESEARCH TEAM SYNTHESIS We took several steps to organize the quality measures found in the systematic review and qualitative study into an initial draft framework. Because we sought to build upon existing generic frameworks of health care quality while still being open to new concepts emerging (e.g. the “culture of health care” as one example) we did not explicitly use the Donabedian x IoM grid at this stage of framework construction. Rather, we sought to expand and innovate beyond existing frameworks by combining deductive and inductive approaches, and engaging in iterative prototyping and continuous dialoguing with experts and stakeholders. First, we amalgamated all the found measures from the systematic review and qualitative study into a unified and complete list of unique quality measures of Collaborative Care. Second, we created an initial scaffold for this material based on: a) other authors’ proposed key ingredients for effective implementation of evidence based Collaborative Care models,69,70 and b) content clusters created by our advisory group during a sorting exercise at an in-person meeting. Third, we invited our advisory group and other individuals with diverse expertise including Collaborative Care models, public health, health policy, quality and patient safety, outcomes measurement, and health information technology (IT), to nominate additional measures or categories of measures for inclusion to address any perceived gaps. At this stage, we aimed to err on the side of over-inclusion, entrusting that the subsequent advisory group process would aid in parsing and prioritizing measures. Through this process we developed a draft framework with 14 broad domains and 184 specific dimensions of quality of Collaborative Care, which formed the basis for the first survey in the advisory group’s modified Delphi consensus process.

ADVISORY GROUP Informed by the Canadian Institutes for Health Research Knowledge to Action (KTA) framework, our integrated knowledge translation strategy for this project included an advisory group of knowledge users.71,72 We invited content experts and stakeholders in the areas of primary care, mental health, quality improvement, research, health policy, and lived experience of mental illness; 26 individuals, each representing one or more of the perspectives listed above, agreed to participate (see Table 5). The advisory group met in person three times during the two-year study period to guide all phases of the project from inception to interpretation and uptake of findings.


17

TABLE 5 – ADVISORY GROUP MEMBER CHARACTERISTICS (N=26) DISCIPLINE/PROFESSION

PERCENT (NUMBER)

Management (Non-clinical)

8% (2)

Health Policy

19% (5)

Mental Health Clients

11% (3)

Psychiatry

11% (3)

Family Medicine

19% (5)

Social Work

8% (2)

Nursing

4% (1)

Psychology

4% (1)

Quality Improvement/Performance Measurement

16% (4)

METHOD FOR MODIFIED DELPHI CONSENSUS PROCESS In the second year of this project the advisory group contributed to the selection of the final broad domains and specific dimensions for inclusion in the quality framework. In October 2015, they completed an anonymous survey (via surveymonkey.com) in which they rated and rank ordered proposed quality domains and dimensions from the draft framework. They were asked to rate 14 potential domains for inclusion in the quality framework using a 5 point Likert-type scale ranging from “definitely must include” to “definitely don’t include”, and they were asked to rank order the domains from most to least essential. Then, for their top five ranked domains they were asked to rate the specific dimensions (range 3 to 17 dimensions per domain) for inclusion using the same 5-point scale. Twenty-two advisory group members participated in this initial survey (response rate 85%). The advisory group then met for a daylong meeting in November 2015, which included large group discussions and small group work. Sixteen advisory group members (62% of those invited) and all six members of the research team attended. This group deliberated on the overall organization of the framework, addressed potential gaps and redundancies, indicated perceived priorities for inclusion, and proposed modifications including merging, deleting, rewording or otherwise refining a number of domains and dimensions. The interrelatedness of the various domains and dimensions was acknowledged and was used as an opportunity to ensure core principles were woven throughout the framework, thus balancing comprehensiveness with parsimony. The research team incorporated feedback from the in-person advisory group meeting and drafted a revised quality framework of 11 domains and 52 dimensions (range 2 to 9 dimensions per domain). We sought final feedback on the draft framework from the advisory group in June 2016. In this second anonymous survey, advisory group members were asked to rate how essential each of the domains and dimensions is to quality of collaborative mental health care using a 5 point Likert-type scale ranging from “strongly disagree” to “strongly agree.” Finally, they were invited to provide comments regarding the quality framework and feedback on a preliminary draft of this report. Sixteen advisory group members participated in the final survey (response rate 62%).

18


ADVISORY GROUP CONTRIBUTIONS AND MODIFIED DELPHI RESULTS The advisory group played a critical role in: a) clarifying goals of the study (e.g. developing quality measures to guide quality improvement rather than for performance measurement/reporting, and including important dimensions of Collaborative Care quality regardless of their current feasibility to measure), b) shaping the research methods (e.g. including clients in the qualitative interviews), c) interpreting the findings of all research stages, and d) refining and validating the quality framework. The group envisioned new components of Collaborative Care delivery that don’t currently exist in our context (e.g. sophisticated disease registries with real-time application to support illness detection and treatment optimization), while also identifying opportunities to optimize the use of existing resources. Key points raised during the November 2015 meeting included those below. 1. In the Collaborative Care framework client centredness may be better described by the concept of “Client Inclusion and Participation”, connoting absence of stigma, inclusion in one’s own care (e.g. shared decision-making, self-management support, client-defined outcomes), and participation in shaping care delivery (e.g. peer support, quality improvement). 2. Inclusion of multiple perspectives is paramount yet also poses a challenge to defining certain domains of quality, most notably the concept of Value and Efficiency raised the question of “value from whose perspective?” 3. It is important to avoid enshrining processes that have not yet been demonstrated to improve quality of care, for example, team members providing joint appointments as a dimension of the (subsequently deleted) domain “Collaborative Processes of Care”. Three domains were discussed and discarded as non-essential or redundant based on forced choice rankings by the small and large groups at the in-person meeting: 1. COLLABORATIVE PROCESS OF CARE: specific ways in which health care providers work together to promote collaboration (e.g. use of case conferences and shared care plans for clients whose care is complex). 2. BUILDING CAPACITY OVER TIME: providers learn “on the job” and can better care for clients over time (e.g. illness-specific education is provided; the team can look after more complex clients in primary care over time). 3. STAKEHOLDER ENGAGEMENT: Collaborative Care program and team engage with internal and external stakeholders (e.g. achieves family physician buy-in for team-based care, there is open and proactive communication with community based organizations). After integrating this feedback into a revised quality framework, we re-surveyed the advisory group (Table 6 outlines the self-identified roles or perspectives of those respondents who completed the follow-up survey.


19

TABLE 6 – FOLLOW-UP SURVEY RESPONDENTS (N=16) ROLES AND PERSPECTIVES OF RESPONDENTS

PERCENT (NUMBER)*

Hospital / acute care

19% (3)

Primary care

56% (9)

Mental health and/or addictions

31% (5)

Community agency

13% (2)

Physician

37% (6)

Interprofessional health care provider

19% (3)

Client, caregiver or family member

13% (2)

Administrator

6% (1)

Health policy expert

13% (2)

Quality improvement expert

37% (6)

Researcher

37% (6)

Other

13% (2)

* Sum is greater than 16 as many respondents indicated more than one role.

Respondents validated the revised quality framework, with an overwhelming consensus to include all 11 domains and 52 dimensions of quality, and with no identified gaps or redundancies according to their comments. Based on average ratings on the 5-point Likert-type scale, respondents’ top five priority domains remained consistent with the in-person meeting (November 2015) including: Client Care Outcomes (mean rating 4.82), Client Inclusion and Participation (4.82), Access and Timeliness of Care (4.82), PopulationBased Care (4.71), and Evidence-Based Practices (4.65). They agreed or strongly agreed with all 52 dimensions of quality (mean ratings varied from 3.75 to 4.94)g. Comments also supported the conclusion that the final product is a comprehensive, parsimonious and balanced framework that clarifies the key components of effective Collaborative Care and that incorporates overarching principles, evidence, and front-line perspectives on implementation.

g We have excluded the ratings for dimensions under the Value and Efficiency domain because an error in survey design may have compromised this data.

20


ADVISORY GROUP MEMBERS AT THE NOVEMBER 2015 MEETING


21

QUALITY FRAMEWORK This quality framework for Collaborative Care is intended to guide quality measurement and quality improvement for mental health care in primary care teams across Canada. Thus, the framework is deliberately broad to ensure relevance to the diversity of people experiencing mental illness and addictions in Canada, the conditions they face, and the settings where they receive care, and to offer ideas for improvement regardless of the current Collaborative Care model currently place. The target population (and denominator) for each quality measure developed from this framework will vary depending on the specific primary setting and purpose, and will require specification when developing a particular application of this framework.h The framework describes broad domains of quality, and within each domain specific dimensions. The broad domains represent major quality constructs describing the aims and objectives of Collaborative Care, the key care processes required to achieve those aims, and the necessary infrastructure and supports for successful implementation. The specific dimensions capture organizational-, team- and individual-level phenomena that are evidence-informed, and that could be measured to understand how Collaborative Care is functioning. All Donabedian and IoM quality domains are integrated, however, we recommend avoiding overemphasis on structural measures, as they influence outcomes indirectly and may have weaker evidence to support their effects.73

DOMAINS OF COLLABORATIVE CARE QUALITY The broad domains and their definitions are listed below with the first five representing the highest priorities according to our advisory group. CLIENT CARE OUTCOMES Care achieves good results for clients (e.g. improves symptoms of mental illness, improves quality of life). POPULATION-BASED CARE Appropriate care is delivered to the whole population of clients who are, or who should be, served by the primary care team (e.g. services are allocated equitably to those in need). EVIDENCE-BASED PRACTICES Programs and treatments are designed and implemented with consideration of the best available research and the local context. CLIENT INCLUSION AND PARTICIPATION Care is geared toward providing the best possible experience for clients, and achieving outcomes that are important to clients (e.g. promotes self-efficacy and recovery).

h For example, some measures may apply to clients known to have a specific mental illness (e.g. timely access to evidence-based psychotherapies for depression) or known to receive a specific treatment (e.g. metabolic monitoring for clients taking antipsychotic medications). Other measures may apply to a population at risk (e.g. depression screening for clients with multiple co-morbid medical conditions). An extreme example would be a measure of equitable access to PCMHs for people experiencing mental illness and/or addictions, for which the population of focus (and denominator) may extend beyond the primary care team’s existing patient population.

22


ACCESS AND TIMELINESS OF CARE Clients can easily receive care within a reasonable timeframe considering their illness severity, level of risk, and level of function (e.g. timely identification of mental illness, wait time for psychotherapy after recommendation is made). INFRASTRUCTURE, LEADERSHIP AND MANAGEMENT Care is provided under appropriate conditions (e.g. appropriate physical space, having skilled health care providers from different disciplines). LEVEL OF INTEGRATION BETWEEN MENTAL HEALTH AND PRIMARY CARE SERVICES Services are well coordinated within the collaborative mental health program in primary care, and also between the primary care team and outside mental health specialists (e.g. hospital-based psychiatric care). TEAM FUNCTIONING The clinical team of primary care and mental health providers work well together. COLLABORATION FOR PATIENT SAFETY Collaborative Care program is organized to provide the safest possible care (e.g. promotes safe medication prescribing practices, engages all team members in improving patient safety). QUALITY IMPROVEMENT Collaborative Care team and program are continuously working to improve quality (e.g. program is routinely evaluated from multiple perspectives and the results inform program development and provider training). VALUE AND EFFICIENCY From a system perspective care delivers good value considering the costs. Multiple perspectives and systems are considered when measuring cost effectiveness (e.g. health care, social support, justice, child protection, client incurred costs). As each domain represents an overarching construct it may include structural, process, and outcomefocused dimensions; however, most domains may be understood as predominantly representing either structures, processes, or outcomes as described in Figure 2 on the following page. Similarly, each domain aligns well with one or more IoM domains as outlined in Appendix B.


23

FIGURE 2. INTER-RELATIONSHIP OF COLLABORATIVE CARE DOMAINS OF QUALITY

COLLABORATION IN PRACTICE Client Inclusion and Participation Team Functioning

QUALITY OF CARE INFRASTRUCTURE Infrastructure, Leadership and Management

Evidence-Based Practices

OUTCOMES

Quality Improvement

Client Care Outcomes

Collaboration for Patient Safety

Population-Based Care

Population-Based Care (processes)

Value and Efficiency

Access and Timeliness

SYSTEMS OF CARE Level of Integration Between Mental Health and Primary Care Services

SPECIFIC DIMENSIONS OF COLLABORATIVE CARE QUALITY For each domain of Collaborative Care quality, the quality framework enumerates specific dimensions that are evidence-informed and for which measures could be developed. We denote the evidence sources using the following abbreviations: SR systematic literature review; PI provider interviews; CI client interviews; KTA advisory group.

CLIENT OUTCOMES

Care achieves good results for clients (e.g. improves symptoms of mental illness, improves quality of life). D I M EN S I O N S

1. Care reduces mental illness symptom severity and increases remission rates (illness specific) (SR) 2. Care improves physical health status (SR) 3. Care improves quality of life (SR, CI, KTA) 4. Care improves social and role functioning (SR, CI, KTA) 5. Clients achieve the outcome they hoped for (KTA)

24


POPULATION-BASED CARE

Appropriate care is delivered to the whole population of clients who are, or who should be, served by the primary care team (e.g. services are allocated equitably to those in need). D I M EN S I O N S

1. The Collaborative Care team optimizes mental and physical health care delivery with respect to appropriateness of care (e.g. under or over-utilization), and equity/disparities in care (e.g. using a clinical registry) (SR, KTA) 2. Collaborative Care team (providers and the program as a whole) conducts “opportunistic case finding” cued by multiple data sources (e.g. health care system utilization and other sources of information/ protocols) in order to be responsive to individual and population health needs (ideally in real time) (KTA) 3. Collaborative Care team assesses and responds to the social determinants of health at an individual and population level (KTA, CI)

EVIDENCE-BASED PRACTICES

Programs and treatments are designed and implemented with consideration of the best available research and the local context. D I M EN S I O N S

1. Care team implements the chronic care model: a) client self-management support, b) clinical information systems (e.g. registries, reminders, timely clinical measures), c) delivery system redesign (e.g. staffing, roles, panel or population review), d) provider decision support (e.g. expert consultation or algorithm/ simplified clinical practice guideline), e) links to community resources, and f) organization-level support (e.g. leadership and tangible resources) (SR, PI, KTA) 2. Team has a shared understanding and uptake of a specific evidence informed strategy of care for a specific condition (e.g. depression) (KTA) 3. Providers present clients with evidence-informed choices to inform shared decision making in all aspects of care (CI, KTA)

CLIENT INCLUSION AND PARTICIPATION

Care is geared toward providing the best possible experience for clients, and achieving outcomes that are important to clients (e.g. promotes self-efficacy and recovery). D I M EN S I O N S

1. Clients are a central member of their care team, and are supported and encouraged to be as involved as they wish to be in their care (e.g. involved in determining goals, understanding available options, treatments and rationales, care planning, offering feedback, and/or having their records) (SR, PI, CI, KTA) 2. Care is comprehensive and multi-faceted: providers assess and endeavour to respond to each person’s biopsychosociospiritual needs (PI, CI)


25

3. Clients experience continuity of care when receiving care from multiple providers concurrently or sequentially, and maintain primary care contact (CI, KTA) 4. Clients and families are meaningfully engaged in program development & evaluation, care delivery and quality improvement (CI, KTA) 5. Care promotes self-efficacy and recovery (e.g. hope, purpose, empowerment) (CI, KTA) 6. There are opportunities to give and/or receive peer support (within the primary care team or through partnerships or collaborations) (CI, KTA) 7. Care is appropriate to and responsive to individual clients’ culture, literacy level, and socioeconomic status (SR, KTA) 8. Team members collaborate with each other to reduce stigma about mental illness and facilitate client engagement (SR, CI) 9. Clients are satisfied with their care (i.e. have positive perception of care) (SR, PI, CI)

ACCESS AND TIMELINESS OF CARE

Clients can easily receive care within a reasonable timeframe considering their illness severity, level of risk, and level of function (e.g. timely identification of mental illness, wait time for psychotherapy after recommendation is made). D I M EN S I O N S

1. Mental health services are available in a range of intensities according to client needs (e.g. severity of illness) and provider needs (e.g. for assistance making a specific diagnosis) (SR, KTA) 2. Wait times from referral to mental health assessment, and from assessment to service (e.g. psychotherapy) are minimized and clients are offered relevant supports while awaiting specialized services (SR, CI, KTA) 3. Written and oral communications between team members are timely and facilitate client care (PI, CI, KTA) 4. Team monitors attendance and seeks to understand and minimize no show rates (KTA)

INFRASTRUCTURE, LEADERSHIP AND MANAGEMENT

Care is provided under appropriate conditions (e.g. appropriate physical space, having skilled health care providers from different disciplines). D I M EN S I O N S

1. The Collaborative Care program has adequate funding and uses it efficiently (PI, KTA) 2. The team optimizes allocation and use of physical space and telemental health infrastructure for collaborative practices (e.g. for provider interactivity) (PI) 3. The team allocates and optimizes use of time for collaborative practices (e.g. including client-provider and provider-provider interaction) (SR, PI) 4. There are sufficient and skilled human resources appropriate to the needs of the population served (SR, PI, CI) 5. The organization supports the team to build capacity and skills for Collaborative Care and for mental health and addictions care over time (SR, PI)

26


6. Organizational leadership supports and enables collaborative practices (SR, PI) 7. IT infrastructure exists to support population-based care (e.g. clinical registries support population interventions including risk identification, monitoring, applied health research and quality improvement) (SR, KTA) 8. IT infrastructure exists to support individual clinical care (e.g. shared electronic health record (EHR) supports communication, collaboration, and decision support) (SR, PI, CI) 9. The Collaborative Care program has operational reliability whereby day-to-day service delivery is consistent and care processes occur as planned/intended (KTA)

LEVEL OF INTEGRATION BETWEEN MENTAL HEALTH AND PRIMARY CARE SERVICES

Services are well coordinated within the collaborative mental health program in primary care, and also between the primary care team and outside mental health specialists (e.g. hospital-based psychiatric care). D I M EN S I O N S

1. Mental health and primary care services share a common mission and goal (SR, PI) 2. Primary care and mental health providers (and departments where relevant) jointly decide which services will be offered, where and to whom (KTA) 3. Bidirectional care pathways facilitate transitions between mental health and primary care (e.g. system navigation, informational continuity) (KTA) 4. Community mental health and addictions agencies are partnered in service provision (PI, KTA)

TEAM FUNCTIONING

The clinical team of primary care and mental health providers work well together. D I M EN S I O N S

1. Providers have clarity regarding their own and each other’s roles and scopes, and these are reassessed as needed (PI, CI) 2. The team dynamic and group process support ongoing Collaborative Care skill development and provision; all team members’ perspectives are valued and represented in clinical care and knowledge exchange (PI, CI) 3. Clients experience the well-functioning team by being provided with multiple perspectives of their clinical problems and a choice of treatment/care options (CI) 4. Clinical leadership is effective in supporting teamwork and collaboration (PI) 5. Staff turnover doesn’t erode team or program function (SR, PI, KTA) 6. Providers are satisfied with care, i.e. they have a positive experience of delivering Collaborative Care (e.g. providers feel engaged, care delivery is rewarding, providers feel supported) (SR, PI, KTA) 7. Team members share common principles to guide care (KTA)


27

COLLABORATION FOR PATIENT SAFETY

Collaborative Care program is organized to provide the safest possible care (e.g. promotes safe medication prescribing practices, engages all team members in improving patient safety). D I M EN S I O N S

1. Organization has a strong “safety culture” (i.e. individual, group and management values, perceptions, competencies and patterns of behaviour demonstrate a commitment to safety. Risk is acknowledged, there is a blame-free environment, collaboration occurs across ranks to find solutions to reduce vulnerabilities, and sufficient resources are made available to address safety concerns) (KTA) 2. Team conducts population level safety interventions (e.g. collaborates to search EHR for clients receiving unsafe medications/doses and provides interventions such as consultation and/or education to providers to improve safe prescribing) (KTA) 3. Medication prescribing is safe (e.g. medication reconciliation occurs at key points of vulnerability; there are low rates of potentially hazardous prescribing practices (SR, KTA) 4. The organization and team manage near misses, errors, and negative outcomes effectively (e.g. systematically identification, disclosure, review, learning, provider support) (KTA)

QUALITY IMPROVEMENT

Collaborative Care team and program are continuously working to improve quality (e.g. program is routinely evaluated from multiple perspectives and the results inform program development and provider training). D I M EN S I O N S

1. Quality of care is evaluated from multiple perspectives (e.g. client/family, provider, organization, system) and results inform quality improvement activities (SR, KTA) 2. Primary care team’s quality improvement program includes item(s) addressing care of clients with mental illness (KTA)

VALUE AND EFFICIENCY

From a system perspective care delivers good value considering the costs. Multiple perspectives and systems are considered when measuring cost effectiveness (e.g. health care, social support, justice, child protection, client incurred costs). D I M EN S I O N S

1. Services are prioritized and delivered with attention to cost effectiveness and/or incremental net benefit with respect to client outcomes (SR, KTA) 2. Collaborative Care reduces overutilization and underutilization, such that the appropriate level of care is delivered (e.g. by escalating level of care within the team or facilitating targeted referrals or transfers of care) (SR, KTA)

28



29

DISCUSSION STRENGTHS AND LIMITATIONS OF THE METHODOLOGY The Collaborative Care quality framework is an essential first step towards measuring and improving Collaborative Care implementation in Canada’s primary care settings. A strength of our proposed framework is the rigorous methodology behind its development. We comprehensively reviewed existing measures; explored the experiences of both health care providers and clients ‘in the field’, and; incorporated a multitude of stakeholder perspectives dialoguing with each other in an iterative and sustained manner throughout our project. We integrated the learnings from all phases of the study into the framework. Clients (‘experts by experience’), health care providers, researchers, policymakers, and other knowledge users were involved at all stages of framework development, including study design, interpretation of the results, a modified Delphi consensus process, and knowledge translation of the quality framework. Our work exemplifies the role for client inclusion in developing quality frameworks and measurement strategies, which has the capacity to promote client empowerment and activation in their care and in the health care system. However, clients interviewed in the qualitative phase of the study were overwhelmingly positive regarding their experience of Collaborative Care and their relationship with their family physicians and other members of the primary care team. Given that clients were recruited by a provider within their circle of care (as stipulated by the Research Ethics Board) it is possible that a selection bias may have resulted in underrepresentation of divergent perspectives and limited identification of negative quality measures of Collaborative Care. Furthermore, most of the health care providers who participated in interviews and in the advisory group were based in Toronto (although several had experience in their clinical practices, research and/or policymaking at the provincial or national level). Therefore, further work is needed to confirm the applicability of the qualitative results and the framework beyond our urban and comparatively well-resourced context. Finally, our approach did not generate evidence of causality. We did not identify which structures are key to enable which processes of care, nor which processes impact outcomes or how they may do so. This is a known and continuing knowledge gap in the field of Collaborative Care that will need to be addressed through different methods designed to elucidate the active ingredients of such interventions and ‘what works for whom’.15 Realist reviews, comparative effectiveness research and innovative clinical trials are possible methodological approaches that could be employed to facilitate these investigations. Nonetheless the Collaborative Care quality framework is a leap forward toward articulating hypotheses about mechanisms of action and implementation requirements in real-world contexts.

STRENGTHS AND LIMITATIONS OF THE FRAMEWORK This novel quality framework for Collaborative Care is the first of its kind, and provides a comprehensive overview of what Collaborative Care can and should be according to evidence, experts, health care providers and clients. Prior research has tested the efficacy of Collaborative Care interventions in randomized controlled trials (and other experimental designs) that aim to eliminate the ‘noise’ introduced by contextual factors, to arrive at a theoretical model of Collaborative Care. Our framework crucially advances upon the existing literature by reintroducing context and implementation factors that are vital to Collaborative Care sustainability and spread. At the same time, the quality framework organizes those factors to provide clarity, focus and rigor. Thus, the framework charts a direction for program design, implementation, improvement and evaluation.

30


The framework carefully balances comprehensiveness and parsimony, and achieves coherence through the interrelationships between the various domains and dimensions. Finally, another strength of the framework is its flexible application, as discussed in subsequent sections of this report; the quality framework is highly adaptable and thus highly adoptable. Several limitations to this framework require consideration. The various components of the framework may not have equal importance and have not been weighted; for example, the number of dimensions per domain varies from two to nine but should not be taken as a signal of the importance of the domain. We chose not to differentiate essential versus aspirational domains or dimensions, however, this may pose a barrier to uptake. The advisory group in-person discussion and on-line survey both pointed to the following five most essential and important domains: Client Outcomes, Population-Based Care, EvidenceBased Practices, Client Inclusion and Participation, and Access and Timeliness of Care. Many people experiencing mental illness and/or addictions receive care in multiple settings concurrently or sequentially; the framework has not dealt with the complexities of measuring – or sharing accountability for - the quality of their care.

IMPLICATIONS IMPROVING COLLABORATIVE CARE IMPLEMENTATION

The quality framework can increase awareness of evidence-informed standards of care and chart a course for Collaborative Care program development, implementation and evaluation across Canada for years to come, with the potential to impact outcomes for millions of Canadians who will experience mental illness and/or addictions in their lifetimes. Quality measures arising from this framework can be used in several ways, including: a) comprehensive program evaluation and identification of priority areas for improvement, and/or b) selection of one or two measures for regular and repeated use during quality improvement initiatives until sustained improvement is achieved. Most primary care teams will need to identify one or two priorities, ideally aligned with other initiatives within the organization or its milieu, to secure resources and ensure follow through. Once an area has achieved consistently improved performance, measurement efforts can be shifted to a new priority area.

FROM FRAMEWORK TO MEASURES: DEVELOPING AND TESTING INDICATORS

As was originally envisioned, quality measures may exist or be readily developed for some but not all of the aforementioned dimensions of Collaborative Care quality for two reasons: 1) some dimensions represent standards or structural measures that may lend themselves to PCMH reporting rather than measurement, and 2) infrastructure and capacity for quality measurement are underdeveloped, particularly at the interface of mental and physical health care.23,41,47 Dimensions that are not readily measured at present should nonetheless be considered important aspects of care for further research and measure development. Sound measures are important to improve, evidence-informed, clearly defined (e.g. with a numerator, denominator, data source, measurement method, and analytic method), feasible to implement routinely (i.e. without excessive measurement burden), acceptable to users, reliable, valid, easily interpreted, and actionable to drive improvements in care.39,55,56,74 The research team has catalogued validated tools and scales throughout our research process and critically appraised them, and these can be incorporated into


31

measures, along with other key specifications such as population, data sources, measurement method, analytic method, and performance targets where available. Examples of measures currently under development include:

RATES OF BENZODIAZEPINE PRESCRIBING FOR ELDERLY CLIENTS. This measure would address dimension #3 within the domain of Collaboration for Patient Safety. It is strongly aligned with the Choosing Wisely campaigns in the United States and Canada, endorsed by multiple professional groups including geriatrics and psychiatry.75,76 The Choosing Wisely campaigns engage clients in informed and shared decision-making with their health care providers, and thus are also consistent with other Collaborative Care domains such as Client Inclusion and Participation.77 There is strong evidence regarding the hazards of this prescribing practice and regarding relatively low-intensity interventions that are effective in reducing this practice.78,79 Measurement can be done through EHRs or administrative databases for the population of elderly clients (both rostered and non-rostered) in a primary care team. As part of this process, data quality, measurement reliability and validity will need to be established.

LEVEL OF FUNCTIONING AND QUALITY OF LIFE. These measures would address dimensions #3 and 4 within the domain of Client Care Outcomes. Validated scales for client-reported functioning and quality of life (e.g. the SF-1280 and the Sheehan Disability Scale81) are consistent with the evidence for Collaborative Care outcomes15,82,83 , are client centred, and could be readily implemented into routine clinical practice to guide individualized care. Population level data could also be drawn to assess overall program effectiveness. These measures would likely have high acceptability with clients and health care providers alike.

CHRONIC CARE MODEL IMPLEMENTATION. This measure would address dimension # 1 within the domain of Evidence-Based Practices. Models of care based upon Wagner’s chronic care model are the most robustly evidenced Collaborative Care interventions and have demonstrated impact on clinical outcomes, cost effectiveness, and client-reported outcomes, yet these models of care have scarcely been implemented in Canada, leaving much room for improvement.15,17,19,84 Several existing tools can engage and guide primary care teams in assessing their practices for the degree of implementation of principles and tasks associated with the chronic care model. 45,85–87 However, efforts to transfer and scale up implementation of the chronic care Collaborative Care model for depression have not consistently demonstrated the desired clinical outcomes, leaving some uncertainty about how to effectively target this area for improvement, a challenge that the quality framework may help to address.50

32


GUIDING RESEARCH AND POLICYMAKING IN COLLABORATIVE CARE AND QUALITY MEASUREMENT

The framework will guide future research in real-world effectiveness of Collaborative Care, as well as research and policy in quality measure development (e.g. assisting those developing measures to orient, organize and evaluate their work, and informing researchers and policymakers regarding areas where further work is needed). The Collaborative Care quality framework supports many of the same future directions that were recently highlighted by Pincus and colleagues, including: • Expansion of client-reported outcomes beyond disease-specific measures and in line with the recovery movement; • Creation of structural measures of chronic care model implementation; • Emphasis on health equity and preventive and physical health care for people experiencing mental illness; and • Attention to client access to evidence-based psychosocial interventions and client choices. 41 In order to successfully develop and implement quality measures that effectively support improvements in care, numerous obstacles will need to be addressed, including: a) poor EHR functionality and data quality, b) lack of linked data sources capturing care provided across multiple settings and supporting shared accountability, c) limited evidence regarding the critical micro-processes and contextual factors that may significantly impact Collaborative Care effectiveness, d) inadequate provider and program awareness of what constitutes high quality Collaborative Care, e) lack of provider and program competency for rigorous improvement efforts based on quality improvement science, f) insufficient engagement of clients in health care co-design, evaluation and improvement, and g) overall measurement burden. As other authors have recently highlighted, these obstacles may necessitate difficult choices among alternative ways of measuring a particular quality dimension, with implications for measurement feasibility, granularity of data to drive quality improvement, acceptability to clients and health care providers, prioritization of areas for improvement, and ultimately, accountability.88 Furthermore, measurement alone will be insufficient to effect change unless met with supportive health systems structures, including appropriate funding models to support delivery models, IT infrastructure, and availability of community, primary care, and specialist resources. Canada’s primary care teams are mandated to measure and improve quality, and and mental health must compete with many other health conditions and aspects of care to be on the quality improvement agenda. Thus, the framework can and should inform the development of health policies and funding models that ensure appropriate structural conditions to support Collaborative Care. HQO has newly developed quality standards for care of depression, schizophrenia and dementia that will apply to acute care, primary care, community agency, and long term care facility settings.89–91 It is vital that new quality strategies focus on processes and outcomes that are meaningful to clients, and that support teams in improving the implementation of Collaborative Care, which has demonstrated ability to deliver many of the highlighted care needs including client education, timely adjustments to treatment, access to psychological treatments, preventive and physical health care, and good transitions between care levels/settings. Like HQO’s new standards, our framework is geared towards engaging teams in voluntary measurement for formative feedback and quality improvement. When used for other purposes such as external accountability and to determine funding, performance indicators may have unintended negative consequences by corroding the culture of quality improvement, creating perverse incentives, promoting a narrow and concrete focus on the indicator itself, and missing out on insights into how processes are functioning and why (i.e. the contextualizing factors).92 DRIV ING IM PROV EM ENT S IN THE IM PLEM ENTATION OF COLL A BOR ATIV E M ENTA L HE A LTH C A RE

33

TRANSFERABILITY TO OTHER CONTEXTS

We believe a new quality framework for Collaborative Care will have broad relevance and utility in any setting where there is an interest in improving the quality of primary mental health care. The framework benefited from an early and comprehensive review of published and unpublished measures of Collaborative Care implementation and outcomes, thus grounding the project in international perspectives from the outset. Furthermore, many countries are increasingly emphasizing the foundational roles of primary care in their health care systems, with similar drivers including but not limited to population health needs, health care costs, and the robust evidence to support the effectiveness of integrating mental health specialists into primary care.

FUTURE DIRECTIONS This research points to several important future directions for quality measurement and quality improvement, client inclusion and participation, and effectiveness research in Collaborative Care. In the next phase of our research beginning Summer 2016, we are developing, implementing and evaluating specific quality measures of Collaborative Care in two primary care teams in Toronto. With our collaborators at the St. Michael’s Academic Family Health Team and Taddle Creek Family Health Team we will: 1. Delineate specific and detailed measures for implementation (e.g. rationale, definition and specifications including target population, numerator, denominator, data sources, data collection, and scoring) 2. Evaluate each piloted measure, including: a) importance, relevance, validity, feasibility, acceptability and actionability, and b) resource requirements for, and quality of, data collection (e.g. accuracy, completeness, reliability where applicable, interpretability) 3. Appraise the extent to which the overall set of quality measures is balanced (i.e. representative of the whole system, comprehensive, and parsimonious). Although not planned as part of this phase, we are already finding the framework is stimulating quality improvement initiatives, for example in the areas of de-prescribing, and joint planning of services between the hospital’s primary care and mental health programs. Our eventual strategy will be the development of a community of practice in Collaborative Care quality improvement across Ontario’s primary care settings. We believe there is good potential for a learning collaborative to improve Collaborative Care implementation for depression care, where there is strong evidence, significant implementation gaps and opportunity for improvement, and good alignment with HQO’s new quality standards for depression.93.94 Client inclusion and participation in health care design, delivery, evaluation and improvement is an important new frontier for Collaborative Care and for health care generally. As research shifts from efficacy to real-world effectiveness a key set of outcomes relates to client perceptions of their health and of their care. Client perspectives are vital to how client-reported outcome measures and client-reported experience measures (i.e. PROMs and PREMs) are defined, collected, analyzed, and interpreted. However, there are multiple perspectives on client roles; in our qualitative interviews, some health care providers identified clients as central members of their care team in co-producing client outcomes and no providers identified clients as contributors to service design, delivery, or quality improvement. As we move forward with

34


meaningfully including clients in these areas we will also need to acknowledge tensions and barriers to providing measurement-based care with measures that are important and relevant to clients, and explore what would be required to shift perspectives and adopt client-oriented research and evaluation. Finally, our quality framework builds upon prior theoretical and experimental models of Collaborative Care efficacy to introduce factors thought to be important in effective real-world implementation, and further research will be needed to test these hypothesized components. As a starting point, we and other researchers will need to consider what are the appropriate questions to ask and study designs to use. Should comparative effectiveness research methods be used to conduct head-to-head comparisons of different models in their entirety (Model 1 vs. Model 2)? Alternatively, how can we formulate and measure degree of integration and test its relationship to outcomes? Yet another approach – should we break down and analyze each of the various components and functions of Collaborative Care models for its essentiality? Furthermore, how can research designs reflect differing individual needs (i.e. ‘what works for whom’)? The answers to these questions can guide the development of innovative clinical trial designs aimed at discovering how to implement Collaborative Care models to scale across Canada’s organized primary care settings to achieve outcomes that are meaningful to clients and families. Ultimately this research can accelerate uptake of Collaborative Care models that will impact individual and population health outcomes, client and provider experience, and health system sustainability.


35

RESEARCH TEAM The following research team members contributed to this project:

AUTHOR TEAM DR. NADIYA SUNDERJI (co-PI), MD, MPH, FRCPC Assistant Professor Mental Health and Addictions Service, St. Michael’s Hospital Department of Psychiatry, University of Toronto DR. ABBAS GHAVAM-RASSOUL (co-PI), MD, MHSc, CCFP, FCFP Assistant Professor Department of Family and Community Medicine, St. Michael’s Hospital Department of Family and Community Medicine & Dalla Lana School of Public Health, University of Toronto ALLYSON ION, MSc Research Coordinator, St. Michael’s Hospital PhD Candidate, School of Social Work, McMaster University DR. ELIZABETH LIN, PhD Associate Professor Independent Scientist, Centre for Addiction and Mental Health Department of Psychiatry, University of Toronto

OTHER MEMBERS OF THE RESEARCH TEAM DR. GWEN JANSZ, MD, PhD Assistant Professor Department of Family and Community Medicine, St. Michael’s Hospital Department of Family and Community Medicine, University of Toronto ANJANA AERY, MPH Research Coordinator, St. Michael’s Hospital DR. AMANDA ABATE, MD Psychiatry Resident Department of Psychiatry, University of Toronto

36


ACKNOWLEDGMENTS AND DISCLOSURES This research was funded by the Ministry of Health and Long Term Care (MoHLTC) through the Alternate Funding Program Innovation Fund. The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding source. The funders had no role in the design and conduct of the study; collection, management, analysis and interpretation of the data; and preparation, review or approval of the report. None of the authors or research team members have any financial support or other conflicts of interest to disclose, including no equity ownership, profit-sharing agreements, royalties, patents, and research or other grants from private industry or closely affiliated nonprofit funds. We are grateful to Dr. Sydney Dy, Associate Professor, Department of Health Policy and Management, Bloomberg School of Public Health, Johns Hopkins University and Dr. Anna Ratzliff, Associate Professor, Department of Psychiatry and Behavioral Sciences, University of Washington, for feedback throughout this project and on this report. We acknowledge the significant role of the Librarians Robyn Butcher, Department of Family and Community Medicine, University of Toronto and Carolyn Ziegler, Li Ka Shing Knowledge Institute, St. Michael’s Hospital who were instrumental in the development of rigorous search strategies and access to the literature. Our thanks to volunteer student Abinaya Sathiyanesan for her help with retrieving literature. We appreciate all the clients and health care providers who participated in the qualitative phase of the study. We also thank members of the St. Michael’s Hospital Academic Family Health Team and the Taddle Creek Family Health Team, which have agreed to implement and provide feedback on new quality measures.

ADVISORY GROUP MEMBERS We are grateful to all advisory group members who contributed to this project including: DR. MONICA AGGARWAL, PhD, Innovative Health Care Management Solutions Inc., Managing Director, University of Toronto, Assistant Professor JENNIFER CHAMBERS, Empowerment Council DR. AMY CHEUNG, MD MSc, Associate Professor, Department of Psychiatry, University of Toronto, Bell Canada Chair in Adolescent Mood and Anxiety Disorders DR. NAUSHABA DEGANI, PhD, Manager, Health System Performance, Health Quality Ontario DR. SHERRY ESPIN, PhD, RN, Associate Professor, Daphne Cockwell School of Nursing, Ryerson University ROBIN GRILLER, (former) Director, Mid East Toronto Health Link; (current) Executive Director, St. Michael’s Homes DR. CHRIS HAYES, MD MSc MEd, (former) Medical Director, Quality and Performance, St. Michael’s Hospital; (current) Chief Medical Information Officer, St. Joseph’s Healthcare Hamilton, Assistant Professor, Department of Medicine and Institute for Health Policy, Management and Evaluation, University of Toronto DR. NOAH IVERS, MD PhD, Family Physician, Women’s College Hospital and Assistant Professor, Department of Family and Community Medicine, University of Toronto DR. NICK KATES, MD, Chair, Department of Psychiatry and Behavioural Neurosciences, McMaster University SHERRY KENNEDY, Executive Director, Taddle Creek Family Health Team


37

SHELDON LAPORTE, Empowerment Council DR. ELIZABETH LIN, PhD, Associate Professor, Department of Psychiatry, University of Toronto MAUREEN MCGILLIVRAY, MSW, RSW, Mount Sinai Hospital MARVELOUS MUCHENJE DR. GILLIAN MULVALE, PhD, Assistant Professor, Health Policy and Management, DeGroote School of Business, McMaster University DR. MICHELLE NAIMER, MD, MHSc, Family Physician, Mount Sinai Academic Family Health Team, Associate Professor, Department of Family and Community Medicine, University of Toronto RAELENE PRIETO, Mental Health Therapist, Women’s Health in Women’s Hands Community Health Centre, Registered Psychotherapist, College of Psychotherapists of Ontario ZARSANGA POPAL, Policy Analyst, Canadian Mental Health Association

Report design by Pulp & Pixel www.pulpandpixel.ca

38


REFERENCES 1. Whiteford, H. A. et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 382, 1575–1586 (2013). 2. Saxena, S., Thornicroft, G., Knapp, M. & Whiteford, H. Resources for mental health: scarcity, inequity, and inefficiency. Lancet 370, 878–889 (2007). 3. World Health Organization. Human Resources and Training in Mental Health. (2005). 4. World Health Organization, World & World Organization of Family Doctors (Wonca). Integrating Mental Health Into Primary Care: A Global Perspective. (2008). 5. Canadian Institute for Health Information. Health Care in Canada, 2012: A Focus on Wait Times (2012).

17. Archer, J. et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst. Rev. 10, CD006525 (2012). 18. Butler, M. et al. Integration of Mental Health/Substance Abuse and Primary Care. No. 173 (Prepared by the Minnesota Evidence-based Practice Center under Contract No. 290-020009.). (Agency for Healthcare Research and Quality, 2008). 19. Gilbody, S., Bower, P., Fletcher, J., Richards, D. & Sutton, A. J. Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch. Intern. Med. 166, 2314–2321 (2006). 20. Mulvale, G., Abelson, J. & Goering, P. Mental health service delivery in Ontario, Canada: how do policy legacies shape prospects for reform? Health Econ. Policy Law 2, 363–389 (2007).

6. Mental Health Commission of Canada. Changing Directions, Changing Lives: The Mental Health Strategy for Canada. (2012).

21. Durbin, A., Durbin, J., Hensel, J. M. & Deber, R. Barriers and enablers to integrating mental health into primary care: A policy analysis. J. Behav. Health Serv. Res. 43, 127-139 (2016).

7. Office of the Auditor General of Ontario. Chapter 4 Section 4.06. Ministry of Health and Long-Term Care. Community Mental Health: Follow-up on VFM Section 3.06, 2008 Annual Report. (2010).

22. Chisholm, D., Sanderson, K., Ayuso-Mateos, J. L. & Saxena, S. Reducing the global burden of depression: populationlevel analysis of intervention cost-effectiveness in 14 world regions. Br. J. Psychiatry. 184, 393–403 (2004).

8. College of Family Physicians of Canada. A Vision for Canada: Family Practice - The Patient’s Medical Home. Position Paper. (2011).

23. Goldman, M. L., Spaeth-Rublee, B. & Pincus, H. A. Quality indicators for physical and behavioral health care integration. JAMA 314, 769–770 (2015).

9. Kates, N. et al. The evolution of collaborative mental health care in Canada: A shared vision for the future. Can. J. Psychiatry Rev. Can. Psychiatr. 56, Insert 1-10 (2011).

24. Ontario Ministry of Health and Long Term Care. Open Minds, Healthy Minds: Ontario’s Comprehensive Mental Health and Addictions Strategy. (2011).

10. Glazier, R. H. & Redelmeier, D. A. Building the patient-centred medical home in Ontario. JAMA 303, 2186–2187 (2010).

25. British Columbia Ministry of Health Services & Ministry of Children and Family Development. Healthy Minds, Healthy People: A 10-Year Plan to Address Mental Health and Substance Use in British Columbia. (2010).

11. Price, D., Baker, E., Golden, B. & Hannam, R. Patient Care Groups: A New Model of Population Based Primary Health Care for Ontario. (Ontario Ministry of Health and Long Term Care, 2015). 12. Mulvale, G., Danner, U. & Pasic, D. Advancing communitybased collaborative mental health care through interdisciplinary Family Health Teams in Ontario. Can. J. Commun. Ment. Health 27, (2008). 13. Peek, C. J. & National Integration Academy Council. Lexicon for Behavioral Health and Primary Care Integration: Concepts and Definitions Developed by Expert Consensus. (Agency for Healthcare Research and Quality, 2013).

26. Province of New Brunswick. The Action Plan for Mental Health in New Brunswick 2011-18. (2011). 27. Government of Alberta. Creating Connections: Alberta’s Addiction and Mental Health Action Plan 2011-2016. (2011). 28. Addiction and Mental Health Collaborative Project Steering Committee. Collaboration for Addiction and Mental Health Care: Best Advice. (Canadian Centre on Substance Abuse, 2014).

14. Wagner, E. H. et al. Improving chronic illness care: translating evidence into action. Health Aff. 20, 64–78 (2001).

29. Jeffries, V. et al. Collaborative Care for Mental Health and Substance Use Issues in Primary Health Care: Overview of Reviews and Narrative Summaries. (Mental Health Commission of Canada, 2013).

15. Woltmann, E. et al. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am. J. Psychiatry 169, 790–804 (2012).

30. Petch, J. & Tepper, J. The next challenges for primary care in Ontario. healthydebate.ca Available at: http://healthydebate .ca/2012/10/topic/community-long-term-care/the-next -challenges-for-primary-care-in-ontario. (Accessed: 9th November 2013)

16. Vanderlip, E. R. et al. Dissemination of Integrated Care Within Adult Primary Care Settings: The Collaborative Care Model. (American Psychiatric Association and Academy of Psychosomatic Medicine, 2016).


39

31. Steele, L. S., Durbin, A., Sibley, L. M. & Glazier, R. Inclusion of persons with mental illness in patient-centred medical homes: cross-sectional findings from Ontario, Canada. Open Med. 7, e9-20 (2013). 32. Glazier, R. H., Zagorski, B. M. & Rayner, J. Comparison of Primary Care Models in Ontario by Demographics, Case Mix and Emergency Department Use, 2008/09 to 2009/10. (Institute for Clinical Evaluative Sciences, 2012). 33. Bauer, A. M. et al. Implementation of collaborative depression management at community-based primary care clinics: an evaluation. Psychiatr. Serv. Wash. DC 62, 1047–1053 (2011). 34. Kilbourne, A. M. et al. Translating evidence-based depression management services to community-based primary care practices. Milbank Q. 82, 631–659 (2004). 35. Chaney, E. F. et al. Implementing collaborative care for depression treatment in primary care: a cluster randomized evaluation of a quality improvement practice redesign. Implement. Sci. IS 6, 121 (2011). 36. Knowles, S. E. et al. Better together? a naturalistic qualitative study of inter-professional working in collaborative care for co-morbid depression and physical health problems. Implement. Sci. IS 8, 110 (2013). 37. National Quality Measures Clearinghouse. Varieties of Measures in NQMC. Available at: http://www.qualitymeasures .ahrq.gov/tutorial/varieties.aspx. (Accessed: 3rd April 2016) 38. McDonald, K. et al. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies (Technical Reviews, No. 9.7). Available at: http://www.ncbi.nlm.nih.gov/books /NBK44015/. (Agency for Healthcare Research and Quality, 2007). (Accessed: 13th Sept 2013)

45. Agency for Healthcare Research and Quality. The Academy Atlas of Integrated Behavioral Health Care Quality Measures. Available at: https://integrationacademy.ahrq.gov/atlas. (Accessed: 28th March 2016) 46. McCusker, J. et al. Developing an evaluation framework for consumer-centred collaborative care of depression using input from stakeholders. Can. J. Psychiatry Rev. Can. Psychiatr. 58, 160–168 (2013). 47. Institute of Medicine (US) Committee on Crossing the Quality Chasm: Adaptation to Mental Health and Addictive Disorders. Improving the Quality of Health Care for Mental and Substance-Use Conditions. (National Academies Press (US), 2006). 48. Health Quality Ontario. Quality Matters: Realizing Excellent Care for All. (2015). 49. Mainz, J. Defining and classifying clinical indicators for quality improvement. Int. J. Qual. Health Care J. Int. Soc. Qual. Health Care ISQua 15, 523–530 (2003). 50. Solberg, L. I. et al. A stepped-wedge evaluation of an initiative to spread the collaborative care model for depression in primary care. Ann. Fam. Med. 13, 412–420 (2015).

39. NHS Institute for Innovation and Improvement. The Good Indicators Guide: Understanding How to Use and Choose Indicators. Available at: http://www.apho.org.uk/resource /item.aspx?RID=44584 (2008). (Accessed 26th Jan 2013).

51. Sunderji, N., Ghavam-Rassoul, A., Ion, A., Aery, A. & Abate, A. Evaluating the implementation of collaborative mental health care: a systematic review of quality measures. University of York Centre for Reviews and Dissemination (PROSPERO International prospective register of systematic reviews, 2016). Available at: http://www.crd.york.ac.uk/PROSPERO /display_record.asp?ID=CRD42016038387 (Accessed 4th May 2016)

40. Donabedian, A. Selecting approaches to assessing performance. In: An Introduction to Quality Assurance in Health Care, Chapter 4, pp. 45-57. (Oxford University Press, 2003).

52. Butler, M. et al. Integration of mental health/substance abuse and primary care. No. 173 (Prepared by the Minnesota Evidence-based Practice Center under Contract No. 290-020009.). (Agency for Healthcare Research and Quality, 2008).

41. Pincus, H. A., Scholle, S. H., Spaeth-Rublee, B., Hepner, K. A. & Brown, J. Quality measures for mental health and substance use: Gaps, opportunities, and challenges. Health Aff. 35, 1000–1008 (2016).

53. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. & PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6, e1000097 (2009).

42. Miller, B. F., Kessler, R., Peek, C. J. & Kallenberg, G. A. A Framework for Collaborative Care Metrics: A National Agenda for Research In Collaborative Care. (Agency for Healthcare Research and Quality, 2011).

54. PRISMA: Transparent Reporting of Systematic Reviews and Meta-Analyses. Available at: http://www.prisma-statement. org/Default.aspx. (Accessed: 29th June 2016)

43. Continuous Enhancement of Quality Measurement in Primary Mental Health Care: Closing the Implementation Loop Project. Available at: http://www.ceqm-acmq.ca/ceqm/ (Centre for Applied Research in Mental Health and Addiction, Simon Fraser University, 2006). (Accessed 27th Oct 2013)

40

44. Centers for Medicare & Medicaid Services. Consensus core set: ACO and PCMH/primary care measures, version 1.0. Available at: https://www.cms.gov/Medicare/Quality-Initiatives -Patient-Assessment-Instruments/QualityMeasures/Downloads /ACO-and-PCMH-Primary-Care-Measures.pdf (2016). (Accessed 23rd July 2016)

55. Stelfox, H. T. & Straus, S. E. Measuring quality of care: considering measurement frameworks and needs assessment to guide quality indicator development. J. Clin. Epidemiol. 66, 1320–1327 (2013). 56. Stelfox, H. T. & Straus, S. E. Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation. J. Clin. Epidemiol. 66, 1328–1337 (2013).


57. Noblit, G. W. Meta-Ethnography: Synthesizing Qualitative Studies. (Sage Publications, 1988). 58. Dixon-Woods, M., Agarwal, S., Jones, D., Young, B. & Sutton, A. Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10, 45–53 (2005). 59. Strauss, A. L. & Corbin, J. M. Basics of Qualitative Research : Grounded Theory Procedures and Techniques. (Sage Publications, 1990).

72. Canadian Institutes of Health Research. Guide to Knowledge Translation Planning at CIHR: Integrated and End-of-Grant Approaches. Available at: http://www.cihr-irsc.gc.ca/e/45321 .html. (Accessed: 2nd January 2013) 73. Seow, H. et al. A framework for assessing quality indicators for cancer care at the end of life. J. Pain Symptom Manage. 38, 903–912 (2009). 74. McGlynn, E. A. & Asch, S. M. Developing a clinical performance measure. Am. J. Prev. Med. 14, 14–21 (1998).

60. Wells, M., Williams, B., Treweek, S., Coyle, J. & Taylor, J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials 13, 95 (2012).

75. Canadian Geriatrics Society. Geriatrics: Five Things Physicians and Patients Should Question. Choosing Wisely Canada (2014). Available at: http://www.choosingwiselycanada.org /recommendations/geriatrics/. (Accessed: 21st December 2014)

61. Ling, T., Brereton, L., Conklin, A., Newbould, J. & Roland, M. Barriers and facilitators to integrating care: experiences from the English Integrated Care Pilots. Int. J. Integr. Care 12, e129 (2012).

76. Canadian Psychiatric Association, Canadian Academy of Geriatric Psychiatry & Canadian Academy of Child and Adolescent Psychiatry. Psychiatry: Thirteen Things Physicians and Patients Should Question. Choosing Wisely Canada (2015). Available at: http://www .choosingwiselycanada.org /recommendations /psychiatry/. (Accessed: 14th April 2016)

62. Tomoaia-Cotisel, A. et al. Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Ann. Fam. Med. 11 Suppl 1, S115-123 (2013). 63. Bayliss, E. A. et al. Understanding the context of health for persons with multiple chronic conditions: moving from what is the matter to what matters. Ann. Fam. Med. 12, 260–269 (2014). 64. Sunderji, N., Battacharyya, O. & Stergiopoulos, V. Grand Challenges in the Care of Medically Complex Patients: Ontario’s Investment and Impact. (2015). 65. French, S. D. et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement. Sci. IS 7, 38 (2012). 66. Forman, J. & Damschroder, L. J. in Empirical Methods for Bioethics: A Primer (eds. Jacoby, L. & Siminoff, L.) 39–62 (Elsevier JAI Press, 2008). 67. Braun, V. & Clarke, V. in APA Handbook of Research Methods in Psychology: Research Designs (ed. Cooper, H.) 2, 57–71 (APA Books, 2012). 68. Braun, V. & Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101 (2006).

77. Choosing Wisely Canada. What Is CWC? Available at: http://www.choosingwiselycanada.org/about/what-is-cwc/. (Accessed: 28th March 2016) 78. Tannenbaum, C., Martin, P., Tamblyn, R., Benedetti, A. & Ahmed, S. Reduction of inappropriate benzodiazepine prescriptions among older adults through direct patient education: the EMPOWER cluster randomized trial. JAMA Intern. Med. 174, 890–898 (2014). 79. Canadian Agency for Drugs and Technologies in Health. Discontinuation Strategies for Patients with Long-term Benzodiazepine Use: A Review of Clinical Evidence and Guidelines. (2015). 80. Ware, J., Jr, Kosinski, M. & Keller, S. D. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med. Care 34, 220–233 (1996). 81. Sheehan, D. V., Harnett-Sheehan, K. & Raj, B. A. The measurement of disability. Int. Clin. Psychopharmacol. 11 Suppl 3, 89–95 (1996). 82. Lin, E. H. B. et al. Effect of improving depression care on pain and functional outcomes among older adults with arthritis: a randomized controlled trial. JAMA 290, 2428–2429 (2003).

69. Kwan, B. M. & Nease, D. E. in Integrated Behavioral Health in Primary Care: Evaluating the Evidence, Identifying the Essentials (eds. Talen, M. R. & Burke Valeras, A.) 65–98 (Springer, 2013).

83. Ell, K. et al. Collaborative care management of major depression among low-income, predominantly Hispanic subjects with diabetes: a randomized controlled trial. Diabetes Care 33, 706–713 (2010).

70. Mullin, D. J. & Funderburk, J. S. in Integrated Behavioral Health in Primary Care: Evaluating the Evidence, Identifying the Essentials (eds. Talen, M. R. & Burke Valeras, A.) 273–298 (Springer, 2013).

84. Gillies, D., Buykx, P., Parker, A. G. & Hetrick, S. E. Consultation liaison in primary care for people with mental disorders. Cochrane Database Syst. Rev. 9, CD007193 (2015).

71. Graham, I. D. et al. Lost in knowledge translation: time for a map? J. Contin. Educ. Health Prof. 26, 13–24 (2006).

85. Solberg, L. I. et al. Care quality and implementation of the chronic care model: a quantitative study. Ann. Fam. Med. 4, 310–316 (2006).


41

86. AIMS Center: Advancing Integrated Mental Health Solutions. Patient-Centred Integrated Behavioral Health Care Principles & Tasks. Available at: http://uwaims.org/files/AIMS _Principles_Checklist_final.pdf (Accessed 27th March 2016) 87. Kessler, R. S. et al. Development and Validation of a Measure of Primary Care Behavioral Health Integration. Fam. Syst. Health J. Collab. Fam. Healthc. (2016). e-pub doi:10.1037/ fsh0000227 88. Iyer, S. P., Spaeth-Rublee, B. & Pincus, H. A. Challenges in the operationalization of mental health quality measures: An assessment of alternatives. Psychiatr. Serv. (2016). 89. Health Quality Ontario. Major Depression: Care for Adults and Adolescents. (2016). Available at: http://www.hqontario .ca/Evidence-to-Improve-Care/Quality-Standards/Major -Depression (Accessed 17th November 2016) 90. Health Quality Ontario. Schizophrenia: Care for Adults in Hospitals. (2016). Available at: http://www.hqontario.ca /Evidence-to-Improve-Care/Quality-Standards/Schizophrenia (Accessed 17th November 2016) 91. Health Quality Ontario. Behavioural Symptoms of Dementia: Care for Patients in Hospitals and Residents in Long-Term Care Homes. (2016). Available at: http://www.hqontario.ca /Evidence-to-Improve-Care/Quality-Standards/Behavioural -Symptoms-of-Dementia (Accessed 17th November 2016) 92. Freeman, T. Using performance indicators to improve health care quality in the public sector: a review of the literature. Health Services Management Research 15, 126-137 (2002). 93. Wilson, T., Berwick, D. M. & Cleary, P. D. What do collaborative improvement projects do? Experience from seven countries. Jt. Comm. J. Qual. Saf. 29, 85–93 (2003). 94. ØVretveit, J. et al. Quality collaboratives: lessons from research. Qual. Saf. Health Care 11, 345–351 (2002).

42


APPENDIX A EXAMPLE SEARCH STRING (MEDLINE SEARCH) 1. 2. 3. 4. 5.

exp Interprofessional Relations/ exp Patient Care Team/ exp “Delivery of Health Care”/ consultation liaison.mp. (share adj3 care).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 6. (integrat* adj3 care).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 7. (collaborat* adj3 care).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 8. exp Primary Health Care/ 9. exp Community Health Services/ 10. exp General Practice/ 11. exp Community Medicine/ 12. exp Psychiatry/ 13. exp Mental Health/ 14. exp Mental Health Services/ 15. exp Mental Disorders/ 16. (mental adj2 ill*).mp. [mp=title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier] 17. exp Quality Control/ 18. exp Quality Improvement/ 19. exp Quality Indicators, Health Care/ 20. exp Quality Assurance, Health Care/ 21. quality chasm.mp. 22. quality framework.mp. 23. structure.mp. 24. process.mp. 25. outcomes.mp. 26. 23 and 24 and 25 27. 1 or 2 or 3 or 4 or 5 or 6 or 7 28. 8 or 9 or 10 or 11 29. 12 or 13 or 14 or 15 or 16 30. 17 or 18 or 19 or 20 or 21 or 22 or 26 31. 27 and 28 and 29 and 30 32. limit 31 to (english language) 33. remove duplicates from 32 2205 results


43

APPENDIX B COLLABORATIVE CARE QUALITY FRAMEWORK DOMAINS’ ALIGNMENT WITH IoM DOMAINS COLLABORATIVE CARE QUALITY DOMAIN

IoM DOMAIN

OUTCOMES Client Care Outcomes

Effective

Population-Based Care

Equitable

Access and Timeliness of Care

Timely

Value and Efficiency

Efficient

COLLABORATION IN PRACTICE Client Inclusion and Participation

Patient Centred

Team Functioning

N/A *

QUALITY OF CARE Evidence-Based Practices

Effective

Quality Improvement

All IoM domains

Collaboration for Patient Safety

Safe

Population-Based Care (processes)

Equitable

SYSTEMS OF CARE Level of Integration Between Mental Health and Primary Care Services

Effective Timely

INFRASTRUCTURE Infrastructure, Leadership and Management

Effective

* Dimensions and indicators of quality emerged during our study phases that did not align with any of the IoM domains of quality, many of which were included under our domain team functioning. As such, we have proposed a 7th domain termed “Culture of Health Care” to reflect this gap in the IoM Framework (see page 10 in report for more information).

44


A Quality Framework to Guide Measurement ...

A Quality Framework to Guide Measurement ...

Suggest Documents

A Guide to NSW Quality Teaching Framework

The Framework of Quality Measurement

a quality measurement framework for turkish firms ...

Software Quality Measurement: A Framework for Counting Problems ...

A Quality Measurement Framework for IT Services

A Runtime Quality Measurement Framework for Cloud ... - InterACT - KIT

A Homogeneous Framework to Measure Data Quality

On Using a Semiotic Quality Framework to

Proposed Quality Evaluation Framework to Incorporate Quality

Erratum to: A Framework for Measurement Feedback to ... - Springer Link

TOWARDS A FRAMEWORK TO GUIDE THE ...

Good Practice Guide No. 118 A Beginner's Guide to Measurement

A Guide to Geospatial Data Quality

Measurement Framework - Access to Justice BC

A conceptual modeling quality framework

Open Data Quality Measurement Framework: Definition ... - Nexa Center

Open Data Quality Measurement Framework: Definition ... - Nexa Center

Extending the IT Service Quality Measurement Framework ... - CiteSeerX

Full Reference Printed Image Quality: Measurement Framework and ...

A framework to support performance measurement at ...

Using a Performance Measurement Framework to Overcome the Odds ...

A Measurement Framework to Assess SME ... - Semantic Scholar

Refocusing quality measurement to best support ...

AMEC's Social Media Measurement Framework User Guide [PDF]