A Conceptual Framework for a Systems Thinking Approach to US ...

Systems Research and Behavioral Science Syst. Res (2016) Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2420

■ Research Paper

A Conceptual Framework for a Systems Thinking Approach to US Population Health Julie M. Kapp1*, Eduardo J. Simoes1, Anne DeBiasi2 and Steven J. Kravet3 1

Department of Health Management and Informatics, University of Missouri School of Medicine, Columbia, MO USA 2 Trust for America’s Health, Washington, DC USA 3 Johns Hopkins Community Physicians, Division of General Internal Medicine, Johns Hopkins School of Medicine, Baltimore, MD USA

US health outcomes remain poorer than those of high-income peer countries despite collective efforts directed at improving health and healthcare. Regarding population health, such collective impact efforts increasingly come from community-based organizations, as federal budgets have been cut over the years. The Institute of Medicine recently identified core metrics to facilitate collective efforts and specified the Secretary of Health and Human Services as the position to lead the nation’s efforts to improve the health of the population. However, integration across such a complex system requires a clear, deliberate systems approach. We adapt The Malcolm Baldrige Framework for Performance Excellence as a conceptual model with which to apply systems thinking to population health improvement. We offer specific recommendations necessary to build a national systems thinking approach towards improving the health of communities and populations if we hope to ameliorate the US health disadvantage. Copyright © 2016 The Authors Systems Research and Behavioral Science published by International Federation for Systems Research and John Wiley & Sons Ltd. Keywords population health; public health systems; system dynamics; health policy; evaluation ‘A system generally goes on being itself, changing only slowly if at all, even with complete substitutions of its elements—as long as its interconnections and purposes remain intact’.‘Changing relationships usually changes system behavior’.Donella H. Meadows * Correspondence to: Julie M. Kapp, Department of Health Management and Informatics, University of Missouri School of Medicine, CE717 CS&E Bldg., One Hospital Drive, Columbia, MO 65212, USA. E-mail: [email protected]

INTRODUCTION The ‘Why’ A US Population Health Disadvantage The US has been labelled with a ‘health disadvantage’ (National Research Council and Institute of Medicine, 2013). The US lags behind on a number of critical health outcomes compared with its high-income peer

Copyright © 2016 The Authors Systems Research and Behavioral Science published by International Federation for Systems Research and John Wiley & Sons Ltd.

Received 9 July 2015 Accepted 24 July 2016

This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is noncommercial and no modifications or adaptations are made.

RESEARCH PAPER countries, including a shorter life expectancy. This difference in life expectancy has been growing for the past three decades (National Research Council and Institute of Medicine, 2013), underscoring the need for reform in how the US manages population health. What complicates our progress is our inability to define, monitor, and improve on modifiable premature mortality, even after 100 years (Kapp, 2013).

The ‘What’ The Affordable Care Act’s Focus on Population Health The 2010 Patient Protection and Affordable Care Act (ACA) includes a number of provisions that attempt to address US population health needs. The ACA created a national strategy to improve healthcare quality (i.e. the National Quality Strategy) which includes a focus on population health; this focus is reinforced in related frameworks such as the Triple Aim (Berwick et al., 2008). Section 4001 of the ACA likewise addresses population health by establishing the National Prevention, Health Promotion, and Public Health Council, chaired by the Surgeon General, and a provision that the National Prevention, Health Promotion, and Public Health Council establishes the nation’s first National Prevention Strategy (National Prevention Council, 2011; hereafter referred to as the ‘Strategy’). This is intended to be the key strategy and vision for managing US population health. Population health management is a nebulous term, the spirit of which intends to identify the population health needs of a defined service area and align those needs with targeted strategies to improve health outcomes. The Strategy’s core values align with the idea of reducing the US health disadvantage—that Americans live longer and healthier through an increased emphasis on, and investment in, prevention. Yet, even with a strategy for ‘what’, there remains many uncertainties about ‘how’ to improve population health.

Syst. Res ‘How?’ Collective Inter-agency Efforts Are Not Enough The Robert Wood Johnson Foundation and Trust for America’s Health emphasized both the need for federal agencies to increase attention on prevention and health promotion, along with the need to work across agencies external to the healthcare system in order to have a major impact on improving the health of all Americans (Levi et al., 2014). Hester et al. (2015) suggested ‘collaborating entities need organizational structures in order to guide, grow, and sustain their joint efforts over time’. Hardcastle et al. (2011) argued for a fully integrated health system, requiring ‘all government policies reflect the ultimate goal of improving the health of the population’. ‘Collective impact’ (Kania and Kramer, 2011) is a buzzword among community-based organizations to reflect the inter-agency commitment of cross-sector organizations towards a common goal and is comprised of five conditions for success: (i) a common agenda; (ii) a backbone support organization (a highly structured process managed by an umbrella organization); (iii) shared measurement systems (ways success will be measured and reported); (iv) mutually reinforcing activities; and (v) continuous communication (Kania and Kramer, 2011). The Strategy’s focus on engaging cross-sector partners in prevention efforts (National Prevention Council, 2011), along with its goals and strategies for reducing the US health burden, constitutes a collective impact ‘common agenda’ to which related stakeholders must align. The idea of collective impact has become increasingly popular, given the rise in the number of nonprofit organizations and the subsequent increase in public sector (Appleton-Dyer et al., 2012) and academic partnerships, with the public sector generally including a range from healthcare, social services, education, law enforcement, housing, and other local government services. The idea of the federal government driving a collective impact model across agencies is not new. For example, in 2009, the American Recovery and Reinvestment Act appropriated $650 million for health promotion (Parekh et al., 2014); the


Syst. Res (2016) DOI: 10.1002/sres.2420

Julie M. Kapp et al.

Syst. Res Centers for Disease Control and Prevention then funded 10 non-profit organizations to collaborate with Communities Putting Prevention to Work (Parekh et al., 2014). The increased popularity in cross-sector partnerships and collective impact is challenged by the increased number of competing stakeholders across sectors. Between fiscal years of 2005 and 2013, the Centers for Disease Control and Prevention funding had decreased by about 15% ($1.1 billion; Levi et al., 2014). From 1999 to 2009, 501 (c)(3) public charities grew from 631 902 organizations to 1 006 670, an increase of 59.3% (National Center for Charitable Statistics at the Urban Institute, 2010a). During that same timeframe, 501(c)(3) private foundations grew from 77 978 organizations to 120 617, an increase of 54.7% (National Center for Charitable Statistics at the Urban Institute, 2010a). The total revenue for public charities in the USA in 2010 totalled $1.38 billion (National Center for Charitable Statistics at the Urban Institute, 2010c), and for private foundations, totalled $43 billion (National Center for Charitable Statistics at the Urban Institute, 2010b). Despite collective impact efforts, US health and healthcare activities are often uncoordinated, fragmented, misaligned, and lack measurement to track change that matters most (Institute of Medicine, 2009; Institute of Medicine, 2015).The Institute of Medicine (IOM) recently reported on a set of ‘core metrics’ to align and integrate within and across health and healthcare, recognizing that measurement is a tool for performance improvement. Yet, even a shared measurement system for the status of health and healthcare at the national, state, local, and institutional levels is not enough to enhance the effectiveness and efficiency of population health performance (Institute of Medicine, 2015).

The Need for a Systems Thinking Framework ‘Systems thinking’ is an approach to a problem that considers how components within the larger structure operate and interact over the lifecycle of the system and how to optimize the design, implementation, and evaluation of that system. Systems thinking ‘can best be described as the

RESEARCH PAPER application of system concepts to frame our understanding of the world, and it is also about possible future action—what ought to be or could be’ (Rajagopalan and Midgley, 2015). What is a system? A system includes at least two elements that interact; the elements are all interdependent (Ackoff, 1999) and are integrated in ways that continually affect each other in feedback loops over time and which operate as a whole towards a common purpose (Kim, 1999; Meadows, 2008). Not all systems thinkers agree on what systems thinking is (Cabrera, 2006). Here, we refer to the ideas of a few scholars as a starting, not an ending, point (Cabrera, 2006). The practice of systems thinking does not just focus narrowly on a single technical solution but rather assesses a problem from a comprehensive holistic view of the overall challenge (Ackoff, 1981). Organizations are purposeful, with a set of goals, objectives, and frequently ideals; organizations themselves are part of larger purposeful systems with their own goals, objectives, and frequently ideals (Ackoff, 1974, 1981). Systems go beyond organizations. Eoyang and Berkas (1998) identify human organizations as complex adaptive systems, which have characteristics over all times and at all scales of being: dynamic, massively entangled, scale independent, transformative, and emergent. Through its provisions and strategies, the ACA has generated attention for programs that drive a systems engineering approach to healthcare services (Cassel and Saunders, 2014; Executive Office of the President: President’s Council of Advisors on Science and Technology’s, 2014). Indeed, a call for re-engineering the US healthcare system by using complex adaptive systems theories is not new (Institute of Medicine and Committee on Quality of Health Care, 2001). The IOM introduced these concepts in 2001; however, this was applied to the US healthcare system (Best, 2011; Fraser and Greenhalgh, 2001; Plsek and Greenhalgh, 2001; Plsek and Wilson, 2001; Zimmerman et al., 1998); not population health, and did not provide a practical approach to this end. Given that many of the strongest predictors of health and well-being fall outside of the traditional healthcare setting (National Prevention


Systems Approach to Population Health

Syst. Res (2016) DOI: 10.1002/sres.2420

RESEARCH PAPER

Syst. Res

Council, 2011), applying a systems thinking approach to population health would be a highimpact strategy towards reducing the health disadvantage. While some articles begin to introduce these concepts (Atun, 2012; Carey et al., 2015; Diez Roux, 2011; Leischow and Milstein, 2006; Leischow et al., 2008; Peters, 2014; Peters, 2014), they do not introduce methodological innovations as is needed (Cabrera, 2006) and are less applied in their approach. What is needed is a model or a framework of systems thinking, not [another] theory of systems (Cabrera, 2006). We argue beyond examining programs that drive a systems engineering approach to re-defining the system itself. No studies have yet applied a systems thinking framework to US population health. We first propose a significant contribution not only to the literature but also to the traditional management of US population health in adapting The Malcolm Baldrige Framework for Performance Excellence (www.nist.gov/baldrige/) for use as a unifying conceptual systems thinking framework. This framework uses a systems thinking approach already established by the US government to drive global competitiveness, making it appropriate for addressing the US health disadvantage. We then make recommendations about ‘how’ to implement this systems framework.

organizations (National Institute of Standards and Technology, 2014). The Baldrige Framework (Baldrige Performance Excellence Program, 2013) is described in a booklet of self-study questions, which are only briefly explained here. The Framework centres around two triads and seven criteria of managing and performing as a system: the Leadership Triad (leadership, strategic planning, and customer focus); measurement, analysis, and knowledge management; and the Results Triad (workforce focus, operations focus, and results). The framework is powerful as it creates the expectation that a system’s leadership considers all of the criteria comprehensively. We will hereafter refer to Baldrige terms in italics. The Leadership Triad establishes a knowledge and information system interwoven with process improvement. This ultimately drives results explicitly reflective of the defined strategic goals. The seven criteria exist within the larger framework of what Baldrige calls the organizational profile, which considers the greater context of environment, relationships, and the larger strategic situation. Through this alignment, the Baldrige Framework for Performance Excellence could provide the nation the necessary systems structure to better assure desired results, where stakeholders’ goals are specifically aligned towards the broader collective scope.

METHODS

Baldrige as a Systems Approach

The Malcolm Baldrige Framework

The Baldrige Framework is founded on being an integrated and aligned systems model (Borawski and Brennan, 2008). The Baldrige Framework can be used not only within organizations but also within systems (Communities of Excellence, 2015; Kruse and Norling, 2014). The Baldrige Framework meets the system criteria definition: (i) its seven criteria are the elements; (ii) the criteria are interdependent and interconnected, given their focus on alignment and integration; and (iii) the organizational profile, strategic planning, and the results criteria drive the purpose of the system. We use systems thinking to apply the Baldrige Framework to US population health. We consider US population health as a supra-system,

In 1987, Public Law 100–107 created the Malcolm Baldrige National Quality Improvement Act and the Malcolm Baldrige National Quality Award to support US companies in becoming more globally competitive through focusing on quality (The National Institute of Standards and Technology, 2010). The Baldrige Framework is an integrated and aligned systems model to hard wire performance by setting direction, by using a fact-based decision process, aligning incentives, and driving value (Borawski and Brennan, 2008). In 1999, the scope of the Baldrige Framework was expanded to education and healthcare, and in 2005, to non-profit and governmental


Syst. Res (2016) DOI: 10.1002/sres.2420


Syst. Res

RESEARCH PAPER

containing many subsystems at federal, state, and local levels. We recognize that the application of the Baldrige Framework to US population health is an idealized design. As an idealized design, it is believed to be unattainable but towards which progress is believed to be possible, and is ideal-seeking in its purpose while also being (i) technologically feasible; (ii) operationally viable; and (iii) capable of rapid learning and adaptation (Ackoff, 1977, 1981, 1999).

RESULTS Applying the Baldrige Framework to Population Health Figure 1 outlines our US Population Health Systems Framework, adapted from the Baldrige model. If US population health is our organization, its organizational profile would include understanding key population health characteristics, concepts relating to health, and key health challenges, including competitors to population health (Ackoff, 1981). The Strategy already contains elements of an organizational profile, including the nation’s purpose, vision, values, and mission towards population health improvement. Leadership would be supported by the Executive Office of the President and Congress, but as described in the recent IOM report, the Secretary of Health and Human Services (HHS) is the most appropriate entity to provide leadership towards

progress in health and healthcare, as well as align functions across federal agencies (Institute of Medicine, 2015): It is the HHS secretary who directs the agencies most involved in the collection and use of health data; who signs off on reporting requirements and responsibilities; who is centrally positioned to convene and work with the key stakeholders; and who, as the leader most responsible for the nation’s effectiveness and efficiency in delivering better health at lower cost, has the greatest potential to unlock the capabilities of the core measure set. Strategic planning includes policy and legislation created by leadership with the intention of improving and protecting the health of the US population. This includes the ACA and the Strategy. A customer focus, geared towards the US population, generates questions about identifying individuals and communities at high risk for premature mortality and reduced quality of life. It also implies the need for evidence-based ways to engage these high-risk individuals and communities in tailored and targeted long-term health promotion behaviours to improve their health and reduce premature mortality. The US measurement efforts are uncoordinated, vertically and horizontally, at the national, state, local, and institutional levels (Institute of Medicine, 2015). In practice, organizations conceptualize their own definitions of the problems,

Figure 1 A conceptual framework for a systems thinking approach to US population health Copyright © 2016 The Authors Systems Research and Behavioral Science published by International Federation for Systems Research and John Wiley & Sons Ltd.


Syst. Res (2016) DOI: 10.1002/sres.2420

RESEARCH PAPER outcomes, and the ultimate impact goal. Even definitions of ‘population health’ vary (Stoto, 2013). Similarly, implementation scientists lack consensus on measures (Rabin et al., 2012). There is a call for a set of standardized measures for assessing the intrinsic health of communities in and of themselves; a set of standardized health outcome indicators for national, state, and local use; and a summary measure of population health for estimating and tracking healthadjusted life expectancy nationally (IOM, 2011). After considering a broad range of sources, the IOM recently identified a recommended set of metrics for widespread implementation for health and healthcare in the US (Institute of Medicine, 2015). This report also identifies the HHS secretary as the ‘appropriate person to steward the eventual process of amending the core measure set’. Measurement, analysis, and knowledge management is the system foundation and is critical not only to managing effectively but also to aligning stakeholders and improving performance and competitiveness by creating a fact-based, knowledge-driven system (Baldrige Performance Excellence Program, 2013). The system foundation links the Leadership Triad to the Results Triad, assuring alignment and interconnections. This means that the HHS secretary, Congress, and special health committees that implement or create legislation could monitor the progress of their policies on advancing results for the customers by using an evidence-based systems thinking dashboard, as described below. The workforce focus includes those who carry out the Strategy, particularly under the leadership of the HHS secretary. We define workforce as national (level 1 workforce), state (level 2 workforce), and local (level 3 workforce) to include federal, state, and local government agencies, community-based and non-profit organizations, universities and institutions, and Indian tribes, consistent with ACA legislation. The workforce needs to align on incorporating the Strategy into their operations and how to assess workforce capability, needs, and change management. The operations focus becomes a question of how to design, align, and integrate processes to incentivize progress towards results for the Strategy, in

Syst. Res practice. The Baldrige Results Triad (workforce focus, operations focus, and results) is a composite of product and process and the performance results they yield (Baldrige Performance Excellence Program, 2013).

OPERATIONALIZING THE FRAMEWORK How to Implement a Systems Thinking Approach to Population Health Logic models are widely used in public health organizations as a useful roadmap to align stakeholders and make the infrastructure visible in order to understand the intended pathway to move from activities to outcomes. Logic models are intended to represent a theory of change through a series of ‘if–then’ statements across constructs of activities, outputs, outcomes, and impacts (or some variation thereof), as operationalized in Figure 2. For example, we generally expect improvement in an individual’s knowledge and understanding (at least of how to operationalize behaviour change, if not a better understanding of the circumstances) to precede a change in behaviour; sustainable changes in behaviour are then expected to lead to a change in related outcomes. ‘Even less easy to identify are those metrics which capture possible actions by …systems that might reduce morbidity and mortality at the community level. A stronger body of evidence is needed for innovative clinical and total community metrics’ (Hester et al., 2015). When organizations develop their tracking metrics to measure progress towards a program’s intended outcomes, the metrics rarely represent a theory of change perspective with leading and lagging indicators to monitor progress towards success. ‘Currently, the evidence base to support the linkage between specific standards for public health…and improved public health outcomes is very limited’ (Exploring Accreditation Planning Committee, 2007), arguably one of the greatest weaknesses of the current Strategy for improving population health. Even the IOM’s core measure set is described as a set of flat metrics, rather than in a theory of change format.


Syst. Res (2016) DOI: 10.1002/sres.2420


Syst. Res

RESEARCH PAPER

Figure 2 Systems thinking logic model to monitor ‘measurement, analysis, and knowledge management’ of population health efforts

If the core metrics support a theory of change, and local, regional, and state-based organizations can pursue evidence-based actions that align with progress on the metrics, we can better monitor and manage US population health improvement. This is an integrated and aligned purposeful, if not idealized, systems thinking approach with interconnected elements.

Recommendation 1: Drive a Strategic Outcome-oriented Rather than Action-oriented Approach by Creating an Evidence-based National Reporting Dashboard First, we organized the core metrics into a proposed theory of change sequence based on the IOM’s definitions of the best current measures (Institute of Medicine, 2015; Figure 2). This core metrics’ sequence order was based on the standard logic model structure that activities drive change in knowledge which drives change in behaviour which drives change in outcomes which, over a broadenough scope, drives impact (rounded boxes

with definitions). The core metrics sequence used here is for illustrative purposes; it would need testing and confirmation. The final core metrics sequence becomes the beginning of the dashboard for US population health, forming the centralized, transparent, national ‘shared measurement system’ (Kania and Kramer, 2011) with which to monitor progress on the nation’s health. Second, health and healthcare communitybased organizations would need to be incentivized to align and integrate their within-organizational metrics to the shared metrics. This will help align and integrate the population health system towards one focused on activities to one of outcomes (the systems align with the supra-system). The proposed federal and national responsibilities (level 1 workforce) are represented in boxes below the appropriate theory of change definition, with bold and underlined text (Figure 2). The proposed state and local responsibilities are represented in boxes with italics text (levels 2 and 3 workforce). Funds are a powerful tool to influence and incentivize systems thinking and



Syst. Res (2016) DOI: 10.1002/sres.2420

RESEARCH PAPER community–partner engagement beyond simply collaborating or performing activities related to an overarching theme. Therefore, the operations focus would require the level 1 workforce to set clear expectations for federal funds through requests for proposals (RFP) that require organizational (levels 2 and 3 workforce) alignment with the metrics. This is the ‘how’ of designing, managing, and improving key products and work processes and ensuring effective management of these operations. The operations focus of the RFPs would be nonprescriptive and adaptable for community-based agencies, as the RFPs would focus on common needs (at risk/high need), rather than on common procedures (Baldrige Performance Excellence Program, 2013). The levels 2 and 3 workforce can respond to RFPs, Strategy goals, and targets with ‘mutually reinforcing activities’ (Kania and Kramer, 2011) that are ‘creative, adaptive, and flexible approaches, fostering incremental and major (breakthrough) improvement through innovation’ (Baldrige Performance Excellence Program, 2013). These RFP directives still leave organizations the freedom to innovate their activities, but those activities must nonetheless fit into the overarching plan (the Strategy) if combined efforts are to succeed (Kania and Kramer, 2011). Levels 2 and 3 activities then become evidence-based drivers for progress on the core metrics. Third, as the dashboard evolves, documentation would need to be available through the dashboard that ensures transparency and clarity in reporting the evidence-base for the theory of change and cross-process linkages. In other words, the dashboard must have indicators for the level of evidence (perhaps a grading mechanism like the US Preventive Services Task Force uses; a five-star rating scale; or a red, yellow, and green level of evidence) connecting the theory of change constructs for each fundee’s attribution to progress on the outcomes. The dashboard would be a ‘living’ dashboard (the HHS secretary may modify the metrics as appropriate, although there should be a deliberate process and strategy behind the frequency and type of changes), interoperable, likely housed on a government or foundation website, and

Syst. Res interactive. This dashboard then also serves as the ‘reinforcing feedback loop’ to foster efficiencies and growth in progress over time. In the era of big data and informatics, this might be displayed in a fractal design, meaning, clicking on the linkages (e.g. arrows) across components would allow one to ‘drill down’ through the back-end dataset to see the level of evidence for the linkage, the evidence-based model used, what the target population was, the geographic region, the magnitude of effect, the period of sustainability, and any number of other required standardized fields of reporting. Levels 2 and 3 workforce end-of-project reports and quantitative results could be catalogued through hyperlinks. In this way, the dashboard also supports ‘expansionism’, in that understanding proceeds from the whole to its parts, and knowledge proceeds from the parts to the whole (Ackoff, 1981). Finally, over the long term, the dashboard would help leadership and level 1 workforce monitor cross-organizational progress, improve efficiencies, reduce redundancies, and identify gaps (feedback loops). If a metric does not improve after funding multiple organizations over a specified period of time, dashboard data (list of funded activities and associated organizations and dollars granted towards those organizations) would support decision-making towards an RFP continuous quality improvement process. The dashboard would also allow stakeholders to see where certain indicators may have been underfunded relative to well-funded indicators that did not improve. This would facilitate moving away from an optimistic interpretation of Strategy successes based largely on ecological fallacies (National Prevention Council, 2012, 2013, 2014) towards an evidence-based, datadriven, causal-path model.

Recommendation 2: Improve Operational Effectiveness of the Workforce With the growing number of stakeholders and community-based organizations competing for federal funds, it is critical to push more withinorganizational effectiveness from communitybased partners towards an evidence-based yield of performance.


Syst. Res (2016) DOI: 10.1002/sres.2420


Syst. Res First, the RFPs would require grantees to complete their own organizational profile, providing a clearer picture of their operating environment (key relationships, customers, suppliers, partners, and stakeholders) as well as the competitive environment, key strategic issues, and the system for performance improvement. Following this assessment, further proposal questions would ask not only how governance and leadership guide and sustain their organization but also how they create an environment for engagement, innovation, ethical behaviour, and societal responsibility. The assessments would look at how the organization (i) develops and implements its strategy; (ii) engages and obtains information from customers; (iii) manages information and data to drive performance; (iv) builds an effective workforce environment where team members know how they fit and why they matter; and (v) manages to improve work processes and operations. Most importantly, such an assessment would ask how the articulated desired results of the organization, including those tied to process effectiveness, customer-focused performance, leadership and governance, and financial and marketplace performance, measurably align with the goals of the Strategy and the dashboard (i.e. core metrics). Second, to be eligible to apply for federal funds, levels 2 and 3 workforce would be required to complete a government-approved certification in evaluation, logic model planning, organizational capacity, quantitative metrics, and assessment (balancing feedback loop). The positive movement towards voluntary national accreditation of public health departments is consistent with this recommendation for improved organizational and operational effectiveness. However, the current standards and measures for public health department accreditation are activities— rather than outcomes—focused. It is possible for an agency to successfully execute the activities without driving the necessary systems thinking outcomes. Third, RFPs would require levels 2 and 3 fundees to demonstrate core competencies in performance enhancement or continuous quality improvement processes through effective use of evaluations. While current federal and foundation grants and contracts often require

RESEARCH PAPER evaluations, community partners often struggle in performing useful evaluations and using them for continuous quality improvement, and there is a significant amount of heterogeneity among evaluators (Kane et al., 2013). They lack an understanding to tie together organizational effectiveness, programmatic improvement, and impact (Liket et al., 2014), likely exacerbated in smaller organizations. For example, frontline workers of non-profit organizations often engage in the soft-skill relational work with clients, adjusting what they do to meet the specific needs of those they serve (Benjamin, 2012), resulting in mission creep. Consequently, community-based organizations often deviate from their defined programmatic model, which may negate their evidence of success in working towards a clearly defined set of outcomes. One example of how requiring evaluations is necessary but not sufficient is with non-profit hospitals. Section 9007 of the ACA requires that all non-profit hospitals conduct a community health needs assessment of the community served by the hospital and adopt strategies to meet those needs. Few community benefit programs, as related to the ACA taxexempt hospital mandates, are rigorously evaluated or provide evidence of program impact (Burke et al., 2014).

DISCUSSION Requiring an organizational profile and improved operational effectiveness would help address the need for levels 2 and 3 workforce to better design, manage, and improve their products and work processes. Implementing the Baldrige Framework across all criteria, and all levels of the workforce is key to driving collaboration activities, a systems thinking approach, and accelerating the greater amount of ‘continuous communication’ (Kania and Kramer, 2011). Creating a common cross-sector vocabulary takes time (Kania and Kramer, 2011), and implementing the Baldrige Framework is a long-term commitment of leadership. While the workforce is actively engaged in many of the right efforts needed for improving population health, what is missing is an overall



Syst. Res (2016) DOI: 10.1002/sres.2420

RESEARCH PAPER management framework integrating all of the components into a systems perspective. We propose a systems thinking approach to population health by adapting the Baldrige Framework and making two main recommendations. This systems thinking framework is flexible enough to be adapted as new evidence is produced, while also providing a working infrastructure for long-term initiatives. Aligning societal goals across a wide spectrum of organizations would be accelerated by a comprehensive framework to assure that resources are being applied and allocated effectively, maximizing taxpayers’ and funders’ investments. Implementing result-based management initiatives alone are not enough for a systems thinking approach. The United Nations indicated that their results-based management initiative was part of a broader agenda of reforms, and failed, in part, for a number of reasons (Bester, 2012). Included in these reasons, the United Nations states that ‘agencies are good at defining and measuring results at the output level as outputs are generally amenable to measurement. Meaningful definition and measurement of outcomes is not easy, and many agencies experience difficulty in developing realistic, technically sound indicators for these levels of results. Some suggest that vague outcomes are also a way to avoid being held accountable’. We wholeheartedly agree. In our recommendation 1, we discuss having a set of defined core metrics; allowing the workforce to create additional tailored metrics addresses this point. To further support the need for core metrics, ‘many organizations persist in their attempts to measure as many things as possible. This propensity towards complexity is partly driven by multiple reporting demands put on organizations in the public sector. It is also driven by lack of clarity about which results are the most important’. This is also true among community-based non-profits. Our core metrics and logic model approach will reduce crosssector reporting burdens and inefficiencies; our evidence-based, rating system approach in a dashboard format will reveal which activities are the most important in driving outcomes forward. ‘There must be incentives in place for managers and staff to use results-based management’.

Syst. Res We address this through the RFP funding mechanism. ‘Incentives should be tailored to the context and culture of the organization’. We address this through the continuous quality improvement cycle of leadership to ensure that the RFPs are always addressing needed areas and funding the most effective workforce. ‘It is equally important to remove formal and informal disincentives to results-based management’. In a true system, the levels 2 and 3 workforce will ensure that their mission, vision, values, and goals align, integrate, and support the Strategy, which means that the Strategy becomes integrated into their operations. Our conceptual framework is not result-based, but process- and integration-based.

Anticipated Challenges and Future Directions We acknowledge that these ideas are conceptual and are intended to provide a theoretical basis for the ideas of implementing a systems thinking approach to US population health by using the Baldrige Framework, which is entirely innovative. We anticipate several major challenges to the implementation of these ideas. First, buy-in by the office of the Secretary of HHS is clearly a critical step, as that office is central to defining access to many of the ACA population health dollars, the largest incentive for state and local stakeholders to adopt this approach. Second, some resistance to alignment on the Strategy and core metrics will come from levels 2 and 3 workforce, while others will welcome the clarity and sense of direction. Our conceptual framework lays the groundwork for a long and challenging journey to transform the ‘how’ of US population health management. The next steps for research involve identifying the right core metrics for the logic model and testing their if–then sequence in community-based pilot projects. Identifying mandatory reporting metrics by levels 2 and 3 workforce to be included in the metrics would then facilitate the creation of a prototype (even theoretical) working dashboard. Implementing a systems approach will take at minimum decades. But if the workforce is already investing time and billions of dollars in activities, then making calculated, deliberate, incremental


Syst. Res (2016) DOI: 10.1002/sres.2420


Syst. Res changes that are aligned and integrated within a deliberate systems approach offers our best chance for those activities to move outcomes.

CONCLUSIONS Not until we begin to align and integrate and can visually display health and healthcare organizations’ shared metrics, allocated dollars on shared metrics, programs and activities on shared metrics, progress reports on shared metrics, and evidence-based and effective practices on shared metrics, will we begin to see system level change towards reducing the US health disadvantage. Most importantly, given our health outcomes’ standing among peer countries and our failure to make progress on improving the proportion who succumb to modifiable premature mortality after 100 years of awareness (Kapp, 2013), we cannot afford not to consider an aligned and integrated systems thinking perspective for improving US population health.

ACKNOWLEDGEMENTS JMK and SJK were Baldrige Executive Fellows in the 2014 cohort. JMK is formerly the executive director of the Partnership for Evaluation, Assessment, and Research at the University of Missouri —St Louis. We are grateful to the leadership of the Baldrige Executive Fellows Program for their expertise, especially Harry Hertz, Bob Fangmeyer, Bob Barnett, and Patrician Hilton. We thank Keith Mandel, MD, formerly of the University of Missouri Health System, for his expert review and feedback on an early version of this manuscript. We thank Steffani Webb and her insightful team at the University of Kansas Medical Center for their feedback. We thank Editor Dale Smith for his thoughtful editorial feedback.

REFERENCE Ackoff RL. 1974. The social responsibility of operational research. Operational Research Quarterly 25: 361–371.

RESEARCH PAPER Ackoff RL. 1977. National development planning revisited. Operations Research 25: 207–218. Ackoff RL. 1981. Creating the Corporate Future. Plan or Be Planned For. John Wiley & Sons: New York. Ackoff RL. 1999. Ackoff’s Best. His Classic Writings on Management. John Wiley & Sons, Inc.: New York. Appleton-Dyer S, Clinton J, Carswell P, McNeill R. 2012. Understanding evaluation influence within public sector partnerships: a conceptual model. American Journal of Evaluation 33: 532–545. Atun R. 2012. Health systems, systems thinking and innovation. Health Policy and Planning 2012/10/04: iv4–iv8. BPEP. 2013. 2013–2014 Criteria for Performance Excellence. U.S. Department of Commerce, National Institute of Standards and Technology: Gaithersburg, MD. Benjamin LM. 2012. Nonprofit organizations and outcome measurement: from tracking program activities to focusing on frontline work. American Journal of Evaluation 33: 431–447. Berwick DM, Nolan TW, Whittington J. 2008. The triple aim: care, health, and cost. Health Affairs (Millwood.) 27: 759–769. Best A. 2011. Systems thinking and health promotion. American Journal Of Health Promotion: Ajhp 2011/03/ 03: eix–ex. Bester A. 2012. Results-based management in the United Nations Development System: progress and challenges. A report prepared for the United Nations Department of Economic and Social Affairs, for the Quadrennial Comprehensive Policy Review. Borawski P, Brennan M. 2008. The Baldrige Model: An Integrated and Aligned Systems Approach to Performance Excellence. Journal of Association Leadership Summer. Burke JG, Truong S, Albert S et al. 2014. What can be learned from the types of community benefit programs that hospitals already have in place? Journal of Health Care Poor Underserved 25: 165–193. Cabrera DA. 2006. Systems Thinking. Cornell University: Ann Arbor. Carey G, Malbon E, Carey N, Joyce A, Crammond B, Carey A. 2015. Systems science and systems thinking for public health: a systematic review of the field. BMJ open 2016/01/01 e009002. Cassel CK, Saunders RS. 2014. Engineering a better health care system: a report from the President’s Council of Advisors on Science and Technology. The Journal of the American Medical Association 312: 787–788. Communities of Excellence. 2015. Communities of Excellence 2026. Available at: http://www.communities ofexcellence2026.org/framework/, Accessed 15 September 2015. Diez Roux AV. 2011. Complex systems thinking and current impasses in health disparities research. American Journal Of Public Health 2011/07/23: 1627–1634.



Syst. Res (2016) DOI: 10.1002/sres.2420

RESEARCH PAPER Eoyang GH, Berkas TH. 1998. Evaluation in a complex adaptive system. https://c.ymcdn.com/sites/plexu sinstitute.site-ym.com/resource/resmgr/files/evali ncas.pdf, Accessed 15 September 2015. Executive Office of the President: President’s Council of Advisors on Science and Technology. 2014. Report to the president. Better Health Care and Lower Costs: Accelerating Improvement Through Systems Engineering. Exploring Accreditation Planning Committee. 2007. Exploring accreditation. Final Recommendations for a Voluntary National Accreditation Program for State & Local Public Health Departments. Fraser SW, Greenhalgh T. 2001. Coping with complexity: educating for capability. BMJ (Clinical Research ed.) 2001/10/06: 799–803. Hardcastle LE, Record KL, Jacobson PD, Gostin LO. 2011. Improving the population’s health: the Affordable Care Act and the importance of integration. The Journal Law Medicine Ethics 39: 317–327. Hester J, Auerbach J, Choucair B, Heishman H, Kuehnert P, Monroe J. 2015. New directions in public health services and systems research. Academy Health. Accessed from: http://www.academyhealth.org/ files/phsr/PHSSR%20Paper%202015_FINAL.pdf Institute of Medicine. 2009. HHS in the 21st Century: Charting a New Course for a Healthier America. The National Academies Press: Washington DC. Institute of Medicine. 2015. Vital Signs: Core Metrics for Health and Health Care Progress. The National Academies Press: Washington, DC. Institute of Medicine & Committee on Quality of Health Care. 2001. Crossing the quality chasm: a new health system for the 21st century. IOM (Institute of Medicine). 2011. For the Public’s Health: The Role of Measurement in Action and Accountability. The National Academies Press: Washington, DC. Kane C, Alexander A, Hogle JA, Parsons HM, Phelps L. 2013. Heterogeneity at work: implications of the 2012 Clinical Translational Science Award Evaluators Survey. Evaluation and the Health Professions 36: 447–463. Kania J, Kramer M. 2011. Collective impact. Stanford Social Innovation Review: 35–41. Kapp JM. 2013. Evaluation of the national prevention strategy in reference to historical population health. American Journal of Health Promotion 27: 281–283. Kim DH. 1999. Introduction to Systems Thinking. Pegasus Communications: Waltham, MA. Kruse L, Norling R. 2014. Communities of Excellence 2026. National Civic Review Spring: 53–54. Leischow SJ, Best A, Trochim WM et al. 2008. Systems thinking to improve the public’s health. American Journal of Preventive Medicine 35: S196–S203. Leischow SJ, Milstein B. 2006. Systems thinking and modeling for public health practice. American Journal of Public Health 96: 403–405.

Syst. Res Levi J, Segal LM, St. Laurent R. 2014. Investing in America’s health 2014. A State-by-State Look at Public Health Funding and Key Health Facts Robert Wood Johnson Foundation. Liket KC, Rey-Garcia M, Maas KEH. 2014. Why aren’t evaluations working and what to do about it: a framework for negotiating meaningful evaluation in nonprofits. American Journal of Evaluation 35: 171–188. Meadows DH. 2008. Thinking in Systems: A Primer. Chelsea Green: White River Junction, VT. National Center for Charitable Statistics at the Urban Institute. 2010a. Number of nonprofit organizations in the United States, 1999–2009. Retrieved at: http://nccsdataweb.urban.org/PubApps/profile1. php?state=US, Accessed 8 May 2014a. National Center for Charitable Statistics at the Urban Institute. 2010b. Number of private foundations in the United States, 2010. Retrieved at: http://nccsda taweb.urban.org/PubApps/profileDrillDown.php? state=US&rpt=PF, Accessed 8 May 2014b. National Center for Charitable Statistics at the Urban Institute. 2010c. Number of public charities in the United States, 2010. Retrieved at: http://nccsda taweb.urban.org/PubApps/profileDrillDown.php? state=US&rpt=PC, Accessed 8 May 2014c National Institute of Standards and Technology. 2014. Baldrige by sector: nonprofit. Retrieved at: http://www.nist.gov/baldrige/enter/nonprofit. cfm, Accessed 8 May 2014. National Prevention Council. 2011. National prevention strategy: America’s plan for better health and wellness. [Electronic version]. Available at: http:// www.surgeongeneral.gov/priorities/prevention/ strategy/report.pdf, Accessed 15 September 2015. National Prevention Council. 2012. Annual Status Report. U.S. Department of Health and Human Services, Office of the Surgeon General: Washington, D.C.. National Prevention Council. 2013. Annual Status Report. U.S. Department of Health and Human Services, Office of the Surgeon General: Washington, D.C. National Prevention Council. 2014. Annual Status Report. U.S. Department of Health and Human Services, Office of the Surgeon General: Washington, D.C. National Research Council & Institute of Medicine. 2013. U.S. Health in International Perspective: Shorter Lives, Poorer Health. National Academies Press: Washington, DC. Parekh AK, Scott AR, McMahon C, Teel C. 2014. Role of public–private partnerships in tackling the tobacco and obesity epidemics. Preventing Chronic Disease 11: E99. Peters DH. 2014. The application of systems thinking in health: why use systems thinking? Health Research Policy and Systems / Biomed Central 12: 51. Plsek PE, Greenhalgh T. 2001. Complexity science: the challenge of complexity in health care. BMJ (Clinical Research ed.) 323: 625–628.


Syst. Res (2016) DOI: 10.1002/sres.2420


Syst. Res Plsek PE, Wilson T. 2001. Complexity, leadership, and management in healthcare organisations. BMJ (Clinical Research ed.) 323: 746–749. Rabin BA, Purcell P, Naveed S et al. 2012. Advancing the application, quality and harmonization of implementation science measures. Implementation Science 7: 119. Rajagopalan R, Midgley G. 2015. Knowing differently in systemic intervention. Systems Research and Behavioral Science 32: 546–561.

RESEARCH PAPER Stoto M. 2013. Population health in the Affordable Care Act era. The National Institute of Standards and Technology. 2010. Baldrige Performance Excellence Program: history. Retrieved at: http://www.nist.gov/baldrige/ about/history.cfm, Accessed May 8 2014. Zimmerman B, Lindberg C, Plsek PE. 1998. Edgeware: Insights from Complexity Science for Health Care Leaders. VHA Inc: Irving, Tex.



Syst. Res (2016) DOI: 10.1002/sres.2420