evaluating primary health care in canada

EVALUATING PRIMARY HEALTH CARE IN CANADA: THE RIGHT QUESTIONS TO ASK!


Prepared by: Jeannie Haggerty, PhD

Carmel Martin, MD, PhD

Département de Médecine familiale

Northern Ontario School of

Université de Montréal

Medicine

Québec.

Ontario

For Health Canada

March 2005

Table of Contents Executive Summary ....................................................................................................................... i I. Introduction and Mandate ....................................................................................................... 1 A. Background ................................................................................................................... 1 B. The PHC Transition Fund Objectives ........................................................................... 2 II. Overview of Method ................................................................................................................ 4 III. Results .................................................................................................................................... 5 A. Policy Document Review............................................................................................... 5 Box 1: Comprehensive Primary Health Care: ........................................................ 8 B. Interpretation of Policy Language .................................................................................. 8 Box 2. Primary health care organization.............................................................. 11 C. Primary Health Care Objectives ................................................................................... 12 Box 3: Structures and Inputs that Support PHC .................................................. 13 D. Evaluation Questions for Canada ................................................................................. 14 IV Conclusion .............................................................................................................................. 14 Bibliography Figure 1: Flow Chart of Method Used to Identify PHC Evaluation Questions Figure 2: Results-Based Primary Health Care Logic Model Table 1: Common Objectives of the Primary Health Care Transition Fund and Modifications Recommended by the PHC Expert Consultation Group Table 2: Recommended Evaluation Questions for the PHC National Evaluation Strategy on PHC Objectives. Table 3: Recommended Evaluation Questions for the PHC National Evaluation Strategy on PHC Inputs and Supports. Appendix 1: Detailed Method Used to Identify the PHC Evaluation Questions Appendix 2: Working Set of 100 Evaluation Questions from which the Final Set Was Selected. Appendix 3: Objectives for Primary Health Care in Canada: Operational Definitions of Key Attributes to be Evaluated

Acknowledgements : The authors would like to thank Louise Rosborough and Georgia Livadiotakis from the Primary and Continuing Health Care Division of Health Canada for their careful reading of the evaluation questions at their various stages. We also thank Greg Webster, Indira Pulcens, and Lisa Mitmaker at the Canadian Institute for Health Information for providing feedback on the questions to ensure that they were moving toward indicator development. The following persons also provided valuable feedback on the set of questions that were provided to the group of Canadian Primary Care Experts: Diane Watson (University of British Columbia), Brian Hutchison (University of McMaster), David Gass (Nova Scotia Department of Primary Health Care)


Executive Summary Mandate In this first phase of Health Canada’s National Evaluation Strategy for primary health care (PHC), the mandate is to identify the evaluation questions in national and international policy documents on initiatives to transform the delivery of PHC, then to recommend the most important questions over the next years for Canada. The final questions provide an overview of the performance of the whole primary health care system and not just the renewal initiatives or demonstration projects. In the second phase the Canadian Institute for Health Information will identify indicators for the key questions; for those questions where no measures are available, an evaluation tool kit will be developed by Howard Associates.

Overview of Method The evaluation questions were identified through scan of national and international of policy documents on primary health care renewal, conducted between 15 October 2004 and 28 January 2005. The international scan encompassed the United Kingdom, Australia and New Zealand. The common objectives of the Primary Health Care Transition Fund (PHCTF) were used as the organizing framework to classify the questions. Over 800 explicit and implicit evaluation questions were abstracted from the documents and were mapped to the PHC Transition Fund objectives. With the input of a small group of primary health care experts in Canada, UK and Australia, the initial set was further synthesized into 100 questions that addressed all the major inputs, activities, outputs and outcomes of in PHC Logic Model. Wherever possible, questions were formulated in a way that would suggest indicators. The insights that emerged from the scan of the policy documents and the synthesized questions were presented to a pan-Canadian group of PHC researchers and evaluators at a two-day workshop (23 & 24 February 2005, Ottawa). They accomplished two major tasks: 1) clarification of the PHC objectives to be evaluated, and 2) identification of the key evaluation questions. After additional synthesis to ensure coherence and consistency of language, the results were circulated to all workshop participants for final input.

i

Results PHC Objectives Evaluation issues around some current achievements in PHC were ignored in Canadian policy documents of PHC renewal (acute care, provider-patient relationships, person-centred care), where as several other evaluation concerns that went beyond the PHC Transition Fund objectives were consistently raised (quality of care, better management of mental heath conditions, continuity of care, responsiveness, human resources, payment methods, equity, efficiency). Furthermore, some of the language in the objectives was ambiguous and interpreted in different ways in the policy documents. Consequently, the expert group reformulated the objectives to align them with the general goals and principles of PHC and the broader health system objectives embodied in the Health Accords. The modified PHC objectives are as follows: 1. To increase the proportion of the population that receives ongoing care from a primary health care provider who assumes principal responsibility for their care and who knows their personal and health characteristics. 2. To increase the number of primary health care organizations who are responsible for providing planned services to a defined population 3. To enhance the provision of whole-person, comprehensive primary health services, including acute, episodic and ongoing care and increased emphasis on health promotion, disease and injury prevention, management of common mental health conditions and chronic diseases. 4. To enhance 24/7 access for patient-initiated urgent care which is effectively linked with the patients’ usual primary health care provider. 5. To deliver high quality and safe primary health services and to promote a culture of quality improvement. 6. To ensure that primary health care is acceptable to patients and meets their reasonable expectations of how they should be treated (responsiveness). 7. To facilitate integration and coordination between and among healthcare institutions and healthcare providers to achieve continuity of patient care.

Finally, the policy documents referred consistently to structures or processes that were considered to be important for PHC. In the PHC Logic Model, these correspond to inputs that are largely beyond the direct control of PHC organizations but that should nonetheless be evaluated. These are: health human resources for PHC, establishment of multidisciplinary teams, information technology, funding of PHC, provider payment methods, and sustained policy support for PHC. Key Evaluation Questions At the consultation workshop, primary health care experts worked in small groups around a specific PHC objective. Each group was instructed to integrate the broader system objectives of Equity, Cost-effectiveness, and Sustainability of public funding into the questions. They endorsed or reformulated the questions from the policy document review or they posed new questions. Thirteen evaluation areas corresponding to the seven PHC objectives and the six structures and inputs for effective and efficient PHC delivery were classified. Within these there are 19 key evaluation questions (some with secondary questions) for the seven PHC objectives (Table 2) and 20 evaluation questions for the PHC supports (Table 3). The tables also show the area of the PHC Results-based Logic Model that is addressed by the question, the evaluation level of the unit of analysis implied and the source of the question.

Conclusion The analysis of the policy documents revealed areas where further clarity is needed in terms of the roles and responsibilities of the primary health care. The modified PHC objectives provide some clarity on these issues. The final set of key evaluation question is a comprehensive set which address both PHC renewal and broad principles of PHC; both PHC delivery and key structures or inputs. Not all questions will lead to indicators; some can only be answered by indepth research either because of knowledge gaps or the need to shed light on the meaning of the indicators.

I. Introduction and Mandate As one of the initial steps in developing a national evaluation strategy for primary health care, Health Canada has commissioned an environmental scan of the questions that are being asked to guide the evaluation of national and international initiatives to transform the delivery of primary health care (PHC). These evaluation questions are sufficiently comprehensive to provide an overview of the whole primary health care system and not just the renewal initiatives or demonstration projects. The principal interest is in high-level questions of system performance. The questions most pertinent to the Canadian context are selected and will provide a framework for the identification and development of indicators of PHC performance by the Canadian Institute for Health Information.

A. Background The general goals of PHC are to provide high quality care in a community setting that is accessible, affordable, person-centred and continuous over different health episodes; they address most health care needs (comprehensive) and are coordinated with care from other parts of the health care system. The PHC National Evaluation Strategy aims to evaluate the performance of the PHC system relative to these goals as well as the broad principles of the Canadian health system renewal embodied in the Health Accords (FMM2000) which include: 

Principles of universality, accessibility, comprehensiveness, portability and public administration for insured hospital and medical services as per the Canada Health Act;



Accountability: a commitment to report regularly to Canadians on health status, health outcomes, and the performance of publicly funded health services, and the actions taken to improve these services;



Population Responsiveness and Acceptability: the adaptation of services to key priorities and emerging needs of Canadians, and to meet their expectations for prompt, respectful and confidential services;



Equity: ensuring access to health care and to quality services on the basis of health needs, not individual or social characteristics; this includes a specific commitment to 1

collaborate with Aboriginal people, their organizations and governments to improve their health and well being. 

Effectiveness: the extent to which the outputs or products of the health system make a positive contribution to the health and wellness of Canadians;



Efficiency: the extent to which an organization, policy, program or initiative is producing its planned outputs in relation to expenditure on resources; and,



Sustainability: policy direction and framework that is enduring over time. In the Canadian context, the capacity to maintain a publicly funded program.

The September 2000 Health Accord between the First Ministers agreed that improvements in the primary health care system were crucial to providing Canadians with a sustainable health system that provides timely access to quality health services (ref: FMM2000). Following on the Accord, the federal government established an $800 million Primary Health Care Transition Fund (PHCTF) to accelerate primary health care renewal initiatives. The Fund supports the transitional costs of implementing sustainable, large-scale projects that act as catalysts of change in the primary health care system. These initiatives are expected to lead to fundamental changes in the organization, funding and delivery of primary health care services such that care results in improved access, better health outcomes, more satisfied providers, and the relief of pressures elsewhere in the health system such as emergency rooms and hospital services.

B. The PHC Transition Fund Objectives The common objectives, agreed to by federal, provincial, and territorial governments, for the Primary Health Care Transition Fund are: 1. To increase the proportion of the population having access to primary health care organizations that are accountable for the planned provision of a defined set of comprehensive services to a defined population; 2. To increase emphasis on health promotion, disease and injury prevention, and management of chronic diseases; 3. To expand 24/7 access to essential services; 2

4. To establish multidisciplinary primary health care teams of providers, so that the most appropriate care is provided by the most appropriate provider; 5. To facilitate coordination and integration with other health services, i.e. in institutions and in communities. Provinces and territories have proposed renewal initiatives that are appropriate to the local context; nonetheless there are many strategies in common. The most common strategy across the provinces and territories is an expansion of the primary health care team or core team beyond family physicians. The core team is defined variously but usually includes physicians and nurses. Other multidisciplinary team include social workers, pharmacists and midwives, though this is less widespread. Shared care between primary and secondary or tertiary levels is another variant on the extended team, especially for specific conditions such as chronic diseases and mental health. All provinces and territories envisioned an expansion of health promotion, prevention and chronic disease management. Over and above the PHC Transition Fund objectives, provinces and territories have expanded the service emphasis to mental health and to rural and remote health needs. The common strategies of telehealth services and help lines support wider options for access to care. Information technology and communications in the form of electronic health records are increasing the integration between different health care institutions. Almost all provinces and territories have experimented with alternative payment models for PHC providers and training of providers for new roles in multidisciplinary work. The PHCTF objectives are the initial organizing framework for identifying the right evaluation questions to ask for Canada

3

II. Overview of Method The process of identifying the evaluation questions is illustrated in Figure 1 and is presented in detail in Appendix 1. Here we provide an overview. The evaluation questions were identified through review of policy documents conducted between 15 October 2004 and 28 January 2005. The national and provincial health care commissions were reviewed as well as the provincial and territorial policy documents pertaining to PHC policy renewal. International documents on primary health care renewal were also identified and reviewed for the United Kingdom, Australia, and New Zealand. Explicit and implicit evaluation questions were abstracted from the documents and were mapped to the PHC Transition Fund objectives or broad system goals. The approximately 800 questions, grouped by objective, were analysed qualitatively to see what themes emerged. These themes provided insight into how the objectives or terms in the objectives could be interpreted. The initial questions were included in a draft report submitted to Health Canada and the Canadian Institute for Health Information to provide further suggestions for synthesis. The questions were then synthesized to make a reduced list of 100 questions. Wherever possible, questions were formulated in a way that would suggest indicators. The PHC Results-Based Logic Model (Figure 2, Watson et al, 2004) was used as a framework to ensure that the synthesized questions addressed all relevant areas of PHC structures, processes and outcomes to be evaluated. The questions were submitted for feedback to a selected key informant group of 10 primary health care experts in Canada, the UK, Australia, and New Zealand. From the 100 questions a subset of 30 recommended questions were identified. The 100 questions that were presented to the PHC experts are provided in Appendix 2 where they are mapped to the PHC Transition Fund objectives. Insights from the policy document review and the resulting questions were presented to a panCanadian group of 42 primary health care researchers and evaluators at a two-day workshop in Ottawa (23 & 24 February 2005). Since some of the language in the PHCTF objectives was ambiguous and the objectives did not address all the policy concerns of interest in primary health care performance evaluation, the expert group modified the objectives by consensus to include more specific terms and to align with broad PHC principles and not just renewal initiatives. 4

They then worked in small groups around specific objectives to select, reformulate or formulate evaluation questions were most pertinent for Canada, using the 100 questions as a base. Following the workshop, the questions were again reviewed for comprehensiveness against the PHC Logic Model and some were added. The ensuing questions were sent to all workshop participants for final input.

III. Results A. Policy Document Review Canadian Policy Documents The PHC Transition Fund objectives essentially reflect the vision of the Health Accords and the national and provincial commissions on health care that identify areas where improvement is needed. Policy documents rarely mention the existing strengths or achievements in PHC. For instance, evaluation issues were rarely raised around effective management of acute and episodic conditions; the service emphasis was almost exclusively on health promotion, disease prevention and management of chronic conditions. Policy documents also rarely mention the strong patientprovider relationships which have been the foundation of family medicine and community care. Canadians consistently value this relationship and the academic literature clearly supports its importance for maintaining health, enhancing access, and ensuring comprehensive and coordinated care. Likewise, person-centred care or the biopsychosocial approach--at the heart of family medicine and community nursing--was rarely explicitly mentioned. If the evaluation questions or the National Evaluation Strategy were limited only to issues that were raised around PHC renewal, there would be a risk of losing of elements of primary care that we take for granted. There is a need to monitor existing strengths and any unintended consequences of the transition process and of the subsequent national evaluation strategy. Additionally, new issues emerged in provincial and territorial policy documents that are not specifically addressed in the PHCTF objectives. For instance, while all policy and evaluation documents strongly support the management of chronic diseases, there is also a national call for better detection and management of mental health problems in PHC. Likewise, questions around funding of comprehensive services and health professionals in primary care and how to most 5

effectively remunerate providers figure as enormous concerns across Canada. The objective of establishing multidisciplinary teams for PHC further raises questions about health human resources, how to train providers for multidisciplinary care and how to attract health professionals to and retain them in PHC. The broader system goals of an efficient, effective and equitable system were not explicit in the PHC Transition Fund objectives; they were presumed to be implicit. Nonetheless, policy documents and evaluators consistently raised issues of efficiency, quality of health care and responsiveness or acceptability for the population or for patients. These are attributes of care that are intermediate to achieving system efficiency, effectiveness and equity, and they largely under the direct control of PHC providers and should be included as objectives to be evaluated.

International Document Review Common issues exist across Canadian, New Zealand, UK and Australian primary health care strategies and evaluation. These countries have all placed PHC renewal as central to national health system policy. Canada and Australia are of similar size, have decentralized health systems where individual states or provinces/territories have principal responsibility for health care, and they face similar challenges of service delivery to a geographically dispersed population. Like Canada, the Australian PHC strategies focus on multidisciplinary teams, expansion of prevention, health promotion and chronic disease management and coordination and linkage between organizations. Similar tensions exist between the roles of PHC and public health in delivering preventive and promotive services. In Australia, there are similar concerns about how to integrate general practice (physician) services with more comprehensive PHC services and financial concerns around funding of PHC and provider incentives are recurring concerns. In evaluation terms, there is a wide range of evaluation activity with different States being at different stages of development, similar to Canadian provinces... The New Zealand PHC strategies and approaches have more in common with the UK approaches, than with Australia and Canada. Like the UK, New Zealand is introducing more complex primary health care organizations and is using a fund-holding approach. An interesting feature of the New Zealand primary health care strategies and evaluations is the strong 6

commitment to Maori health and community participation. This mainstreaming of Aboriginal health within primary health care delivery was also a feature of Australia. In Canada, there were almost no policy questions about Aboriginal people’s needs within mainstream PHC, despite the Accords’ key principles of equity of care and outcomes of care for Aboriginal Canadians. Clinical outcomes and explicit health outcomes are a major feature of Australian and New Zealand PHC evaluation documents, but not of the policy documents. Similar to the Accords, the Federal and Provincial Commissions’ reports did not emphasize quality of clinical care indicators, but focused on organizational and service change. An important difference between the United Kingdom and the Canadian policy documents is the UK emphasis on reinforcing and enhancing the role of the primary health care physician. In the Canadian documents, the role of the physician is often ignored. The United Kingdom has defined a very clear evaluation framework and indicators. Furthermore, PHC financing is tied to achievement of quality standards, ensuring that data for monitoring is available. However, our consultation with colleagues in the UK also tells a cautionary tale in terms of evaluation. Linking payment or financing to performance based on indicators, no matter how sound they appear, creates some perverse effects and can lead to distortion of the data that actually masks the true picture. Evaluation with indicators can never paint the complete picture. Experts in the three countries recommended starting the evaluation strategy from goals and outcomes and work backwards, rather than chasing elusive definitions of PHC. Both Australian and New Zealand consultants identified the question in Box 1, below, to be a critical starting point for evaluation:

7

Box 1: Comprehensive Primary Health Care: Do PHC organizations have defined policies to ensure that their practice populations receive:  rapid management of acute, urgent health problems?  timely provision of non-urgent routine care (including well care and chronic illness management)?  recommended preventive services?  referral to hospitals and specialist?  follow-up care after hospitalization?  primary mental health care?  full maternity and child care?  coordinated care of the frail elderly?

 end-of-life care?

B. Interpretation of Policy Language The variance in the use of terms in the policy documents presented a challenge for mapping questions to objectives, but also provided an opportunity for insight how the objectives were interpreted. At the expert consultation workshop we presented our interpretation of the language in the PHC Transition Fund objectives in light of what appeared to be inferred in evaluation concerns raised in the policy documents. The experts largely endorsed our interpretations in the following key questions: 

What are the minimal sets of services that define comprehensive PHC?



What are the essential services that require 24/7 access?



What is the minimal constitution of the multidisciplinary PHC team?



What is understood by “appropriate provider” at a system level and within the PHC team?

8



What is the responsibility and role of PHC organizations vs. public health agencies or regional health authorities with respect to planning of services and the scope of health promotion activities?

Comprehensive care The need for “comprehensive” care was invoked repeatedly without any definitional clarity. Comprehensiveness that goes beyond the Canada Health Act is inferred but not stated explicitly. The documents suggest that changes in funding formulae for PHC physicians and the inclusion of other professionals on a multidisciplinary PHC team will expand the range and type of services provided under public funding. Although there was no statement about the minimal set of services that define comprehensive care, the Canadian PHC policy documents taken as a whole suggest the elements outlined in the evaluation question in Box 1. These are the types of services that policy makers expect from primary health care; the evaluation strategy will monitor the extent to which providers and organizations ensure that they are offered. Access to essential services One PHC Transition Fund objective is to increase 24/7 access to “essential services” without specifying what is meant by “essential”. Only the PHC documents from the Northwest Territories were explicit about interpreting this to mean emergency services for major, acute health problems. The strategies for enhancing access involve increasing the options for accessing needed care, including access to telephone and online advice and information and expanding the availability of services offered in community offices. Despite policy reference to “24/7 access to comprehensive care”, it seems clear that the intent is to provide 24/ 7 access to urgent care and not the range of comprehensive services mentioned in Box 1. At the expert consultation, there was consensus to respect urgency as defined by the patient rather than by health professionals; hence the objective to respond to patient-initiated care. Multidisciplinary teams The establishment of a multidisciplinary team is one of the PHC Transition Fund objectives and is the PHC strategy that seems to invoke the highest expectations for renewal of the PHC system. This objective stands out from the other Transition Fund objectives in that the focus is on the structure rather than on the type of care to be achieved. Yet its function as a means to achieve 9

the other objectives seem clear from the Canadian PHC documents. Policy-makers fully expect collaborative multidisciplinary teams to: 1) provide new or broader care options (comprehensiveness) for patients through broadening the skill mix of the core PHC team; 2) increase the emphasis on promotion, prevention and chronic disease management; 3) expand the options for first-contact access, and 4) enhance linkages between different health and social institutions. However, the constitution of the team is not clearly nor consistently defined in the policy documents, with some referring to a core team of physicians and nurses or nurse practitioners and others referring to an expanded team including social workers, pharmacists, midwives, dieticians and psychologists. This is an area where research is still needed to determine the optimal constitution of the multidisciplinary team for different local contexts and practice populations, and on how best to deploy the full skill set of all professionals to achieve the expected objectives. Appropriate provider Care from the “appropriate provider” was repeatedly used without specifying criteria for appropriateness. For instance, many documents refer to access to an appropriate provider as part of 24/7 access, and the content analysis implies the realization of cost-savings from having services provided by the least costly provider who can competently manage the problem. The policy documents of the United Kingdom go as far as recognizing and enhancing the role of practice secretaries in orienting and advising patients. In other documents, “appropriate” refers to including health professionals on the PHC team who have the best training and skills to manage the delivery of promotive, preventive, chronic disease and mental health care in order to realize quality of care gains. At a system level, appropriateness also refers to shifting the management of common problems to the primary level rather than to more expensive resources such as hospital emergency rooms and specialists. It is totally legitimate to make judicious use of health care providers to realise quality of care and efficiency gains within primary health care and in the system as a whole. However, from an evaluation viewpoint, it would be important to specify the objective of interest in defining appropriateness. Because of the multiple usages of this term, we have tried to avoid the unqualified use of the term “appropriate provider” in the objectives and the questions. 10

PHC vs. public health Finally, there were areas where there was an enormous variance in the use of terms or in the implied responsibility of PHC. These areas of lack of definitional clarity about the role and responsibility of PHC may present problems for evaluation or the development of indicators. There was a recurring tension about whether PHC providers and organizations were responsible to a geographically defined catchment population (a public health or population approach) or to patients or clients (a clinical approach). This tension was evident particularly around questions of health promotion and service planning. Are PHC providers and organisations responsible for outreach activities to vulnerable populations that do not consult as patients? What is the “defined population” for which they must plan services? The participants at the workshop endorsed the need for a more public health orientation in PHC, but the evaluation questions reflect the need to continue to track this tension in the evaluation efforts. Primary health care organization A final issue of language was raised at the workshop around the term “primary health care organization” which is used in the objectives and in many of the questions. The definition provided in Box 2 is based on a general definition of an organization as a structure which mobilizes material, fiscal and human resources through defined policies and procedures to accomplish a collective objective.

Box 2. Primary health care organization A PHC organization is any structure where material, fiscal and human resources are mobilized systematically to accomplish the collective objective of providing high quality primary health care. The more accessible, comprehensive, continuous, and coordinated the care objective, the more varied the resources required and the more complex the organizational procedures deployed to mobilize resources to accomplish the objective.

11

This definition is intentionally broad to encompass the wide variety of PHC organizational models that are current and emerging in Canada. The definition can be applied to more complex demonstration PHC models and community health centers but also to the traditional models of group or family medicine practices that predominate in Canada. The Health Accord makes a commitment to significant annual progress so that citizens routinely receive needed care from multidisciplinary primary health care organizations or teams, with a target of the ensuring access to 50% of the population as soon as possible (FMM2003). By specifying the multidisciplinary nature of the organization, the Health Accord seems to be clearly referring to the assurance that Canadians routinely have access to PHC that is more accessible, comprehensive, continuous and coordinated.

C. Primary Health Care Objectives In view of the insights that emerged from the review of the documents and the understanding that the evaluation strategy addressed PHC in general and not just renewal initiatives, the expert group decided that the objectives needed to be modified. The objectives for the National Evaluation Strategy should reflect the overall objectives for PHC in the Canadian context in the long term. Also, the language of the objectives should be clear and unambiguous. For instance, in the Transition Fund objectives “access” was used in two different objectives; one usage appeared to refer to availability of providers and the other, to ability to obtain timely care. The PHC objectives proposed by the group are shown in Table 1, contrasted with the PHC Transition Fund objectives that were the point of departure. The proposed objectives endeavour to be more explicit in their language and include two new objectives about quality of care and responsiveness / acceptability. These reflect the goals and principles of primary care rather than the means to achieve them. So, for instance, the objective on integration is now explicit about the goal being continuity of patient care and not integration in itself. Likewise, the PHC Transition Fund objective on multidisciplinary teams has been removed as an objective, per se, but is nonetheless tagged for evaluation as an important PHC structure or input. When PHC evaluators in the UK, Australia, and New Zealand were asked to comment on the PHC Transition Fund objectives and evaluation questions, several remarked that the objective on the establishment of the multidisciplinary PHC team stood out as being a structural input rather 12

than an objective of care, and while its implementation merited evaluation it should not be seen as an objective of PHC in itself. The participants at the expert consultation concurred with this analysis, and this is reflected in the new configuration of the objectives and structures. Appendix 3 presents operational definitions for the many of the key attributes that are invoked in the PHC objectives formulated here. The operational definitions come from a different set of experts but are presented here to provide some definitional clarity for subsequent measurement. The group recognized that there are structures and inputs that are critical for achieving the objectives of PHC but that are not PHC objectives, per se, such as multidisciplinary teams. These are mostly policy inputs that are beyond the immediate control of PHC providers or organizations but that need to be the object of the PHC National Evaluation Strategy. These structures and inputs are outlined below in Box 3:

Box 3: Structures and Inputs that Support PHC  Adequate supply of health human resources to meet primary health care needs.  Established multidisciplinary primary health care teams.  Information technology that meets primary health care needs and links primary health care organizations with the rest of the health care system.  Needs-based resource allocation for primary health care.  Provider payment methods that align with PHC and health system goals.  Sustained policy support for primary health care

13

D. Evaluation Questions for Canada Using the new primary health care objectives as a framework as well as the key PHC structures and inputs, the experts endorsed some of the questions that emerged from the policy documents as being essential. They also proposed some additions or clarification around other questions, and in some cases, formulated entirely new questions. After the workshop, all the questions were reviewed as a whole. Some questions were reintroduced to ensure that the totality of questions was comprehensive in scope (covering all relevant areas of the PHC Logic Model), high level, and clear. Final modifications and deletions were made based on the written feedback from workshop participants who responded to the mailing. Table 2 has the 19 key evaluation questions for the seven PHC objectives; some of these are secondary questions, and yet other secondary questions may be implied. Table 3 presents the 20 evaluation questions for the PHC inputs supports. The tables also show the area of the PHC Results-based Logic Model that is addressed by the question, the evaluation level (unit of analysis implied) and the source of the question. Sources that indicate a province or country mean that the general question was raised in one of their policy documents and endorsed by expert group; “Ex” means that the question was formulated by the expert group at the workshop.

IV Conclusion This analysis of PHC policy documents in Canada, Australia, New Zealand and the United Kingdom has shown areas of considerable consensus on the evaluation questions to be addressed in a national evaluation strategy. It has also revealed areas where there is a need for better defining the roles and responsibilities of primary health care providers and organizations and for clarifying what criteria are to be used to define success. Although the initial interest was in evaluation questions on PHC renewal initiatives in relation to the objectives of the PHC Transition Fund, it quickly became evident that the questions needed to be more comprehensive and to address the attainment of PHC in general in line with well-established PHC goals. Although the mandate of this report was to identify a set of questions that would lead to the development of indicators to track PHC renewal efforts, not all questions in the final set of 14

questions lend them selves to the development of indicators. Many are questions that can only be addressed by in-depth research. For some key strategies, such as multidisciplinary teams, considerable knowledge gaps remain about implementation and impact in the Canadian context. Even in areas where clear indicators are suggested, it is critical for evaluation efforts by institutions such as the Canadian Institute for Health Information to be accompanied and supported by research efforts in the academic community to provide information on the accuracy of indicators over time, feedback to understand the causal pathways around indicators, and to monitor unintended effects of the evaluation strategy itself. Canada has been commended by several of the international consultants on the initiative to define a national evaluation strategy based on a broad view of PHC principles and strategies rather than the narrow perspective of current public policy and concerns.

15

Bibliography

A. National (Canadian) Policy Documents Canadian Institute for Health Information. Health System Performance - Health Indicators. Catalogue no. 82-221XIE . Catalogue no. 82-221-XIE. 2000. CIHI. Canadian Institute for Health Information. Canadian Roadmap Initiative. 2005. www.cihi.ca First Ministers Accord 2000. Health Canada. . http://www.hc-sc.gc.ca. First Ministers Health Accord on Health Care Renewal, 2003. Health Canada http://www.hc-sc.gc.ca. Kirby, M. J. & LeBreton, M. The Health of Canadians - The Federal Role: Recommendations for Reform. Volume 6, Final Report: Recommendations for Reform. 6, -392. 2002. Ottawa, Canada, Government of Canada. The Standing Senate Committee on Social Affairs, Science and Technology. Romanow Roy J. (2002). Building on Values - The Future of Health Care in Canada - Final Report. (Commission on the Future of Health Care in Canada ed.).

B. Provincial/Territorial PHC Policy Documents British Columbia British Columbia. Primary Health Care Practice Models. Government of British Columbia. 2004. http://www.healthservices.gov.bc.ca/phc/pdf/renewphc.pdf (accessed on August 19, 2004) British Columbia. Renewing Primary Health Care for Patients. Government of British Columbia. 2004. http://www.healthservices.gov.bc.ca/phc/pdf/renewphc.pdf (accessed on August 19, 2004) Farrally V. PHCTF Evaluation Activity British Columbia (working document) . Canadian Health Services Research Foundation. Evaluating Primary Healthcare in Canada (An invitational workshop) . [7 April 2004]. 2004. Toronto, Ontario.

Alberta Alberta Health and Wellness. “What we heard. A Report on Alberta Health and Wellness Consultations with Providers and Other Stakeholders on Primary Health Care Reform in Alberta”. Department of Health and Wellness: Government of Alberta. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. Summer 2002, p. 20-49. Alberta Health and Wellness. “Evaluation of Health Link”. Department of Health and Wellness, Government of Alberta. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. 2003. Alberta Health and Wellness. “Evaluation of Primary Health Care Capacity-Building Fund Initiatives”. Department of Health and Wellness: Government of Alberta. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. 2004.

Bibliography

1

Alberta. Umbrella Alberta Primary Health Care Project. Government of Alberta. 2004. http://www.health.gov.ab.ca/about/phc/resource/index.htm (accessed on December 1, 2004). Government of Alberta (2001). A Framework for Reform: Report on the Premier's Advisory Council on Health , Rep. No. ISBN 0-7785-1547-8 Report; ISBN 0-7785-1548-6 Appendices. http://www.premiersadvisory.com/reform.html: Government of Alberta.

Saskatchewan Government of Saskatchewan (2001). Caring for Medicare: Sustaining a Quality System (Rep. No. ISBN # 09687942-1-1). Regina, Saskatchewan: Policy and Planning Branch, Saskatchewan Health. Government of Saskatchewan. Programs and Services Primary Health Care 2003. Government of Saskatchewan. 2003. http://www.health.gov.sk.ca/ps_phs_services_over.html (accessed on August 18, 2004). Krakowski V. (2004). Primary Health Care Evaluation Initiatives in Saskatchewan. Primary Health Services Branch. The Saskatchewan Action Plan for Primary Health Care. Primary Health Services Branch. 2002.

Manitoba Manitoba. Primary Health Care. In 2002, Manitoba Health developed a Primary Health Care Policy Framework to guide the ongoing renewal and reform of Primary Health Care in Manitoba. Manitoba Health Primary Health Care. 2002. http://www.gov.mb.ca/health/primaryhealth.html (accessed on August 19, 2004). Moffatt M. (2004). Primary Health Care Evaluation in Manitoba.

Ontario Coyte, P., Wickremaarac, & , S. “Blueprint for Comprehensive Primary Health Care Reform in Ontario” . 3-262004. Peter C. Coyte, Department of Health Policy, Management and Evaluation, University of Toronto , [email protected]. Martin C. (2004). Comparisons of models of Primary Health Care in Ontario. Government of Ontario (2000). Looking Back,Looking Forward : The Ontario Health Services Restructuring Commission (1996-2000) A Legacy Report Toronto, Ontario: Government of Ontario. Government of Ontario. Building On Success: A Blueprint for Effective Primary Health Care. Association of Ontario Health Centres. 1995. http://www.aohc.org/article_display.asp?ArticleID=51 Government of Ontario. Primary Health Care Strategy. Health Services Restructuring Commission. Government of Ontario. 1999. http://www.health.gov.on.ca/hsrc/phase2/rr_phc_final.doc Wilson R. (2004). Evaluation of Primary Care Reform Pilots in Ontario.

Quebec Clair, M. (2000). Commission d'étude sur les services de santé et les services sociaux: Les Solutions Émergentes, Rapport et recommandations (Rep. No. ISBN 2-550-36958-0). Gouvernement du Québec.

Bibliography

2

Government of Québec. Network. Ministry of Health and Social Services Quebec. 2004. http://www.msss.gouv.qc.ca/en/reseau/lsn.html Trahan L. (2004). Primary Healthcare Assessment Initiatives Quebec.

Labrador and Newfoundland Barrett J. (2004). Primary Health Care Evaluation, Newfoundland and Labrador. Newfoundland and Labrador. Primary Health Care. Government of Newfoundland and Labrador. 2002. http://www.gov.nf.ca/health/matterofhealth/primaryhealthcare. Newfoundland & Labrador Health and Community Services. Evaluation plan for the Primary Health Care Renewal Initiative in the Twillingate/New World Island Area. Harry Cummings and Associates Inc. and Med-Emerg International Inc.: Government of Newfoundland & Labrador. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. 2004.

New Brunswick Health and Wellness New Brunswick. Annual Progress Report - 2003 / 2004. Department of Health and Wellness: Government of New Brunswick. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. Annual progress report - 2003 / 2004, 1-45. Morrison D. (2004). Primary Health Care Evaluation New Brunswick Initiatives. New Brunswick. Health Renewal - A Discussion Paper. New Brunswick. 2000. http://www.gnb.ca/0089/cpqs/pdfs/health.pdf

Nova Scotia Nova Scotia (1999). Strengthening Primary Care in Nova Scotia, an Evaluation Initiative. Nova Scotia. Primary Health Care Renewal - Action for Healthier Nova Scotians. Nova Scotia. 2002. http://www.gov.ns.ca/health/phcrenewal/Highlights.pdf Nova Scotia. Primary Health Care Evaluation: Literature Review and Environmental Scan. Nova Scotia Department of Health & Health Canada. 2004, Unpublished Work Nova Scotia. Primary Health Care Transition Fund Evaluation Workshop: Workshop Proceedings. Department of Primary Health Care Section: Government of Nova Scotia and Pyra Management Consulting Services. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. 2004 Nova Scotia Department of Health. Primary Health Care Evaluation Scope Definition and Implementation Steps. Department Primary Health Care Section of Government Nova Scotia, Pyra Management Consulting Services Inc., and Research Power Incorporated. Submission to the Primary Health Care Transition Fund Provincial/Territorial Envelope, Health Canada. 2004. Ungurain M. (2004). Primary Health Care Evaluation Nova Scotia.

Prince Edward Island Prince Edward Island. Primary Health Care. Government of Prince Edward Island. 2004. Unpublished Work

Bibliography

3

Northwest Territories Northwest Territories. NWT Primary Community Care Framework. Government of the Northwest Territories. 2002. http://www.hlthss.vov.nt.ca/content/Publicatins/Reports/PrimaryCare/PCCFrameworkAug2002.pdf

C. International PHC Policy Documents United Kingdom Department of Health (2004). Indicators and targets web address for NHS United Kingdom. Department of Health (2005). Performance Indicators for Primary Care Trusts in the NHS United Kingdom. Department of Health (2005). The NHS Plan United Kingdom. NHS (1997). The new NHS modern. dependable (The White Paper), Rep. No. Cm 3807. The Stationery Office.

New Zealand Australia and New Zealand register. Joint Accreditation Services Australia and New Zealand. 2004, Internet Communication New Zealand Ministry of Health. The Primary Health Care Strategy. 2001. New Zealand, New Zealand Government. New Zealand Ministry of Health. (2004). The New Zealand Health Strategy and The Primary Health Care Strategy. New Zealand Ministry of Health. PHO Clinical Performance Indicators Status Report. 2005 New Zealand Ministry of Health. RSM Performance Indicators and Payments, Policy Paper No. 3. 2005

Australia Joint Accreditation Services Australia and New Zealand register. Joint Accreditation Services Australia and New Zealand. 2004. Internet Communication Australian Institute of Health and Welfare. Rural, regional and remote health: information framework and indicators. [Version 1]. 2003. Canberra, Australian Institute of Health and Welfare. Australian Primary Health Care Research Institute. Advice on a national quality and performance system for divisions consultancy report. 2004. Harris, M. Integration between GPs, hospitals and community health services. Commonwealth of Australia. General Practice in Australia: 2000. 2000. Canberra, Commonwealth of Australia. National Health Performance Committee. National Health Performance Framework Report. Queensland Health. 2001. Brisbane, Queensland Health. New South Wales Health Department. Strategic Directions for Health 2000 - 2005. 2000. Sydney, NSW Health Department Sibthorpe, B. A Proposed Conceptual Framework for Performance Assessment in Primary Health Care. 2004.

Bibliography

4

D. Conceptual References Campbell, S. M., Roland, M. O., & Buetow, S. A. (2000). Defining quality of care. Social Science & Medicine, 51, 1611-1625. Evans, D. B., Edejer, T. T., Lauer, J., Frenk, J., & Murray, C. J. (2001). Measuring quality: from the system to the provider. Int J Qual Health Care, 13, 439-446. Ferlie, E. B. & Shortell, S. M. (2001). Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q., 79, 281-315. Watson DR, Broemeling A.-M., Reid RJ, & Black C (2004). A Results-Based Logic Model for Primary Health Care. College of Health Disciplines - The University of British Columbia. Wilson R. & Dorland J. (2004). Implementing Primary Care Reform. Barriers and Facilitators. (Mc-Gill Queen's Press ed.). World Health Organization (1978). Primary Health Care- report of the International Conference on Primary Health Care (Rep. No. 1). Geneva. World Health Organization. Primary Health Care: A Framework for Future Strategic Directions. 2003. Geneva, World Health Organization.

Bibliography

5

Figure 1: Flow Chart of Method Used to Identify PHC Evaluation Questions First Minister’s Accords

Provincial Policy Documents

National and Provincial Commissions

International documents

Document Review

Initial 800 Questions

Health

Objectives - Thematic Analysis, Clarification and

Canada

Several Iterations

Refinement of PHCTF and other Objectives -

Development of PHC Objectives

Canadian Institute for Health Information

Questions - Mapping to PHCTF Objectives, Logic Model & system level

- Clarification and Refinement of 100 Questions for consultation - Mapping 100 Questions to PHC Transition Fund Objectives

PHC Expert Group Consultation Meeting and Follow-up Survey

International and Canadian Experts

1 Final PHC Objectives and Questions

Figure 2: Results-Based Logic Model for Primary Health Care: Evaluation Framework Contexts

Social, Cultural, Political, Policy, Legislative/ Regulatory, Economic, and Physical Contexts & Characteristics of the Population (Predisposing, Enabling, Need)

Fiscal

RESOUR PHC Efficiency

Health Human Resources

Material RESOURCES

Policy- and Governance-level Activities

Inputs

CES Management- level Activities & Decisions

Clinical Activities & Decisions Activities (Enable PHC

Access to services (availability meets demand)

delivery)

PHC Effectiveness

PHC products and services: Volume, Type (e.g., prevention, curative, palliative), Qualities (i.e., Comprehensive, Continuous, Coordinated, Responsive, Interpersonal and Technical Effectiveness)

Outputs

Maintain or improve work-life of PHC workforce

Increased knowledge about health and health care among the population

Reduced risk, duration and effects of acute and episodic conditions

Reduced risk and effects of continuing health conditions

(Products & Services) CONTEXTS Immediate

Appropriateness of Place and Provider

Health care system efficiency

Acceptability

(Direct) Outcomes Final outcomes

Health care system equity

AND

EXTERNAL FACTORS

Sustainable, Affordable, Accountable Health Care System

Improve and/or Maintain Functioning, Resilience & Health for Individuals

2 Improved Population-level Health and Wellness

Table 1 : Common objectives of the Primary Health Care Transition Fund and modifications recommended by the PHC Expert Consultation Group to reflect general goal principles of PHC and the Health Accords. PHCTF Objectives

Modified PHC Objectives

To increase the proportion of the population having access to primary health care organizations that are accountable for the planned provision of a defined set of comprehensive services to a defined population;

To increase the proportion of the population that receives ongoing care from a primary health care provider who assumes principal responsibility for their care and who knows their personal and health characteristics. To increase the number of primary health care organizations who are responsible for providing planned services to a defined population

To increase emphasis on health promotion, disease and injury prevention, and management of chronic diseases;

To enhance the provision of whole-person, comprehensive primary health services, including acute, episodic and ongoing care and increased emphasis on health promotion, disease and injury prevention, management of common mental health conditions and chronic diseases.

To expand 24/7 access to essential services;

To enhance 24/7 access for patient-initiated urgent care which is effectively linked with the patients’ usual primary health care provider.

To establish multidisciplinary primary health care teams of providers, so that the most appropriate care is provided by the most appropriate provider;

(Moved to structures and process that support PHC)

To facilitate coordination and integration with other health services, i.e. in institutions and in communities.

To facilitate integration and coordination between and among healthcare institutions and healthcare providers to achieve continuity of patient care.

None

To deliver high quality and safe primary health services and promote a culture of quality improvement.

None

To ensure that primary health care is acceptable to patients and meets their reasonable expectations of how they should be treated (responsiveness).

1

Table 2: Recommended evaluation questions for the PHC National Evaluation Strategy around objectives of PHC. Secondary questions are indicated as subsets (x.1, x.2). Each question shows the corresponding area in the PHC logic model and the analytic unit that needs to be evaluated (micro=individual persons; meso=providers or practices, macro=policy or system level). The source column shows the policy documents from which the question was abstracted and which was endorsed by the expert group; question with source is EX were raised only by the expert group and HM only by Haggerty and Martin. No. PHC Logic Model Evaluation Source * Evaluation Question, PHC Objectives (Table 2) Level Objective 1. To increase the proportion of the population that receives ongoing are from a primary health care provider who assumes principal responsibility for their care and who knows their personal and health characteristics. 1.

1.1

What proportion of the population can identify a primary care provider Outputs, Access who assumes principal responsibility for their care and knows their health needs and personal values systematically?

Does that proportion differ by geographic region? By socioeconomic group? By health status? By cultural or ethnic group?

Indirect outcomes, equity

micro

Romanow, NZ

macro

HM

Objective 2. To increase the number of primary health care organizations who are responsible for providing planned services to a defined population. 2

2.1

Do PHC organizations know the composition of their catchment and practice populations in terms of age structure, morbidity profile, cultural diversity, socio-economic status, social and physical environment?

Do PHC organizations have a registry of patients with chronic conditions (diabetes, asthma, heart disease, stroke, depression) for whom they develop specific programs?

Activities, Managementlevel, community participation

meso

NB, NZ

Activities, Managementlevel, Quality of care

meso

BC, UK, Aus(Vic)

Table 2- Questions PHC Objectives: 1

No.

3

4.

Evaluation Question, PHC Objectives (Table 2) What processes for planning services for their defined population do PHC organizations have?

Do regional authorities support PHC organisations with information and processes that allow them to target services and provide referrals to hard-to-reach individuals and communities (e.g. ethnic minorities, intravenous drug users, shut-ins, adolescent parents, those in remote areas)?

PHC Logic Model

Evaluation Level

Source *

Activities, Managementlevel, Community participation

meso

NS, NB, NFLD, Romanow, NZ, UK, Aus

Activities, ManagementLevel, Community participation

meso

Qc, UK, NZ, Aus

Objective 3. To enhance the provision of whole-person, comprehensive primary health services, including acute episodic and ongoing care with increased emphasis on health promotion, disease and injury prevention and management of common mental health conditions and chronic diseases. 5.

Do PHC organizations have defined policies to ensure that their practice populations receive:  rapid management of acute, urgent health problems?  timely provision of non-urgent routine care (including well care and chronic illness management)?  recommended preventive services?  referral to hospitals and specialist?  follow-up care after hospitalization?  primary mental health care?  full maternity and child care?  coordinated care of the frail elderly?  end-of-life care?

Activities, Managementlevel, Comprehensiveness

meso

Alta, BC, NB, NFLD, QC


No.

Evaluation Question, PHC Objectives (Table 2)

6.

Has there been a reduction in health risk (lower BMI, lower smoking rates, higher activity, lower rates of sexually transmitted disease, lower adolescent pregnancy rates, less substance misuse)? Do people attribute reduced health risks to orientation and advice that they received in primary health care?

7.

Do PHC organizations enable patients with chronic health conditions (e.g. diabetes, asthma, coronary heart disease, depression, hypertension) develop competencies and self-efficacy for better managing their health?

7.1

Do self-management strategies for patients with chronic conditions significantly improve quality of life, reduce the number of visits to specialists, reduce hospital admissions (number and length of stay), and achieve better health outcomes?

PHC Logic Model

Evaluation Level micro

Indirect outcomes, Effectiveness,

Source * Alta, UK, Aus (Vic)

Activities, Clinical-level

micro

Alta, UK, Aus (Vic)

Activities/Intermediate outcomes, Self-management, effectiveness

micro

BC, UK, Aus (NSW), (Vic)

Objective 4. To enhance 24/7 access for patient-initiated urgent care which is effectively linked with the patients’ usual primary health care provider. Activities/Output, micro NB, Alta, NS, BC 8. What proportion of the population has a usual primary health care Management-level provider that has organizational arrangements for 24/7 access that are /Accessibility effectively linked to the usual provider?

9.

What are the costs and consequences of providing 24/7 access alternatives for patient-initiated urgent care (other than physician contact services) in terms of health outcomes, patient and provider satisfaction, and utilization of health care?

Output/Intermediate, Access/ meso Appropriateness, Appropriate provider

BC, CIHI, Kirby, Romanow, UK,


No.


PHC Logic Model

Evaluation Level micro

CIHI, Acc, Kirby

Intermediate outcome, Acceptability

micro

EX

Intermediate outcome, Equity

micro

HM

Outputs, Access 10.

Source *

What is the wait time for acute and episodic care? For routine nonurgent care (including well care and chronic illness management)? For referred care?

10.1

What is the level of patient satisfaction with wait times?

10.2

Do wait times differ systematically by urban/rural/remote region? By socio-economic group? By ethnic group?

Objective 5. To deliver high quality and safe primary health services and to promote a culture of quality improvement in primary health care organizations. 11.

12.

12.1

What percent of recommended preventive care guidelines by the Canadian Task for on Preventive Health Services are implemented by PHC providers?

Does the care for specific key conditions (diabetes, COPD/asthma, congestive heart failure, depression, hypertension, smoking) conform to current evidence and commonly-accepted standards?

Does the emphasis on management of common chronic diseases (diabetes, COPD/asthma, heart disease, depression) compromise the quality of care received by people with other chronic diseases or with multiple co-morbidities?

Outputs, Technical effectiveness,

meso

BC, Romanow, CIHI

Outputs, Technical effectiveness

meso

BC, UK,

Immediate outcomes, Technical effectiveness

meso

HM, EX


No.

13.

14.

Evaluation Question, PHC Objectives (Table 2) Do PHC organizations have defined, non-prejudicial, confidential processes for staff to report potential errors in delivery, treatment or management?

Do PHC organizations measure their performance against recognized standards and modify their practices in response (including issues of patient safety)?

14.1

Are there structures and processes in place to ensure optimal and safe medication management?

4.2

Do PHC professionals participate in continuing professional development that reflects the needs of the PHC organization and the local health needs of the community?

PHC Logic Model

Evaluation Level meso

UK


meso

EX


meso

NZ

Activities, Clinical-level, Quality of care

meso

BC, NFLD, UK, CBF


Source *

Objective 6. To ensure that primary health care is acceptable to patients and that it meets their reasonable expectations of how they should be treated (responsiveness).

15.

Are patients satisfied that the PHC organization and providers respect their right to privacy, confidentiality and dignity?

16.

Are patients confident that PHC organizations and providers are responsive to their culture and language needs?

Intermediate, Acceptability

micro

QC

Intermediate, Acceptability

micro

HM


No.


PHC Logic Model

Evaluation Level

Source *

Objective 7. To facilitate integration and coordination between healthcare institutions and healthcare providers to achieve informational and management continuity of patient care. Activities, Management-level meso EX 17. What types of structures and activities have been developed to link primary health care organizations with other health care organizations? Activities, Clinical-Level 17.1

meso

Outputs, Continuity of care 18.

18.1

19.

EX

Do these structures and activities lead to active collaboration and facilitated referral and feedback between primary health care organizations and other health care organizations? EX

Do patients experience management continuity of care?

Do patients undergo repeated investigations when they see different providers?

Outputs/Intermediate, Continuity/Efficiency

micro

2003 Accord

meso

EX

Outputs, Continuity/Coordination

meso

CIHI

Output, continuity/Coordination

meso

HM

Do providers experience informational continuity of care?

19.1

Do providers have complete information at the point of care about individual patients' health and previous care received from other providers?

19.2

Are providers confident that their care plan and actions will be recognized and considered by other providers?

* Sources : Kirby= The Health of Canadians: Federal Role, Romanow = Future of Health Care in Canada, BC=British Colombia, Alta = Alberta, Sask=Saskatchewan, MN=Manitoba, ON=Ontario, QC=Quebec, NFLD=Newfoundland and Labrador, NB=New Brunswick, NS=Nova Scotia, PEI=Prince Edward Island, NWT=Northwest Territories, YK=Yukon, UK=United Kingdom, Aus=Australia, NZ=New Zealand, CIHI = Canadian Institute for Health Information, EX=expert consultation, HM=Haggerty-Martin.


Table 3: Recommended evaluation questions for the PHC National Evaluation Strategy around PHC inputs and supports that are beyond the direct control of PHC organizations. Secondary questions are indicated as subsets (x.1, x.2). Each question shows the corresponding area in the PHC logic model and the analytic unit that needs to be evaluated (micro=individual persons; meso=providers or practices, macro=policy or system level). The source column shows the policy documents from which the question was abstracted and which was endorsed by the expert group; question with source is EX were raised only by the expert group and HM only by Haggerty and Martin. No.

Evaluation Question, PHC Supports (Table 3)

PHC Logic Model

Evaluation Level

Source

Adequate supply of health human resources to meet primary health care needs. 1.

Is there sufficient number of PHC health professionals, in particular primary care nurse practitioners and family physicians, to meet the demand for PHC?

Input, HHR

macro

HM/EX

2.

What incentives attract and retain health professionals in PHC organizations (financial, work flexibility, continuing professional development)?

Immediate outcomes, Worklife

meso

BC, NS

3.

Are PHC professionals working to their full scope of practice (as per training and regulation)?

Activities, Clinical-level, HHR

meso

Kirby, Romanow, UK, NS

4.

Is the quality of work-life acceptable to staff and health care providers?

Immediate outcomes, Worklife

meso

EX

5.

Does the regional authority have an assessment of health human resources to meet the community's needs?

Input, HHR

macro

EX

Table 3- Questions PHC Supports: 1

No. 6.

Evaluation Question, PHC Supports (Table 3) Do provincial authorities have plans to recruit and train health human resource requirements to meet the needs of the jurisdiction?

PHC Logic Model

Evaluation Level

Source

Input, HHR

micro

EX

Input, HHR

meso

CIHI, NB, NS

Input, Governance level

macro

EX

Multidisciplinary primary health care teams. 7.

7.1

What is the extent and nature of multidisciplinary teams?

How should the mix and number of providers on a multidisciplinary team reflect the needs of the community or practice population?

8.

How do changes in the mix and number of providers on the PHC team impact on the responsiveness, quality and the costeffectiveness of care?

Activities/Intermediate, Efficiency, effectiveness

meso

CIHI

9.

What factors facilitate health care providers working together to provide comprehensive PHC (scope of practice regulation, primary health care funding, training, continuing professional development)?

Activities, Management-level

meso/macro

Alta, NB, NFLD, Kirby, NS

Information technology that is adapted to primary health care and links primary health care organisations with the rest of the health care system. 10.

Do PHC organizations have computerized information systems to support clinical activities? (decision support, electronic health records, electronic prescribing, electronic test requisitions and reporting, electronic consultation reporting)? Which systems are being used?

Input, material

meso

CIHI


No.

Evaluation Question, PHC Supports (Table 3)

11.

Do PHC organisations, in different geographic settings, have communication linkages with teletriage and advice services? with telehealth services? with emergency services? with hospitals? with laboratories? with long-term care facilities?

PHC Logic Model

Evaluation Level

Source

Input, material

meso

Alta, CIHI, BC, NFLD

Needs-based resource allocations for primary health care.

12.

Do regional funding allocations for PHC reflect population age and morbidity structure and vulnerable groups?

Input, Fiscal

macro

NS, Kirby

13.

Has the range of publicly funded services provided (directly or indirectly) by PHC organizations increased over time? Are there gaps in whole-person, comprehensive care because of resource limitations?

Outputs, comprehensiveness

micro

NS, Kirby

14.

What is the per capita operational cost of providing primary health care services at a practice level? At a regional health authority level (accounting for geographic location)?

meso

EX

15.

Have capital investments increased for new technology and equipment for PHC? For physical facilities? For information technology?

Input, Material

macro

2000 Accord

Input, Governance

macro/meso

EX

Provider payment methods that align with PHC goals

16.

How are PHC providers paid?


No. 17.

17.1

Evaluation Question, PHC Supports (Table 3) How does provider remuneration method affect the volume, type and quality of services that are provided?

Do non-FFS payment systems for physicians increase the proportion of clinical time dedicated to prevention and chronic disease management activities? To planning and quality improvement activities?

PHC Logic Model

Evaluation Level

Source

Intermediate outcomes, Efficiency, effectiveness

meso

NS, Aus

Output, Type

micro

Romanow, Aus

Ongoing support from policy-makers for primary health care.

18.

What kind of policies are in place to influence or contribute to ongoing renewal and sustainability of PHC? (e.g., FPT agreements, provincial plans, tripartite agreements, legislation)

Input, Policy

macro

HC and HM

19.

Have the responsibilities of PHC organizations been clearly identified in the health system, especially related to a central role in coordination of patient care?

Input, Policy, Integration

macro

NB, Romanow, BC

20.

What amounts of financial and human resources are dedicated to PHC?

Input, Fiscal

macro

Romanow, NS, NB, Kirby

* Sources : Kirby= The Health of Canadians: Federal Role, Romanow = Future of Health Care in Canada, BC=British Colombia, Alta = Alberta, Sask=Saskatchewan, MN=Manitoba, ON=Ontario, QC=Quebec, NFLD=Newfoundland and Labrador, NB=New Brunswick, NS=Nova Scotia, PEI=Prince Edward Island, NWT=Northwest Territories, YK=Yukon, UK=United Kingdom, Aus=Australia, NZ=New Zealand, CIHI = Canadian Institute for Health Information, EX=expert consultation, HM=Haggerty-Martin.
