What are the strategic responses, conflicts, tensions and behaviours that arise? ...... council), healthcare (a strategic health authority and an ambulance trust), ...
The Roles of performance measurement in the English public sector Pietro Micheli, Andy Neely and Mike Kennerley Centre for Business Performance, Cranfield School of Management, Cranfield, MK43 0AL, UK
Abstract Despite the significant volume of research that has been conducted to date in the field of public sector performance measurement, there is a relative paucity of empirical research that describes in detail the systems and processes being used at different levels of government to measure and manage performance. Furthermore, academics in the performance measurement field tend to adopt either a pragmatic perspective or a more theoretical stance. Rarely do authors appear to provide a firm theoretical grounding to shed light on their more practical findings. Therefore, the researchers decided to carry out a wide project on the roles of performance measurement in the English public sector, adopting three organization theories – agency, institutional and resource dependence theory. The research consists of two main phases. Initially, two exploratory cases were carried out in a police force and primary care trust. Following the findings of the first phase, the authors performed several pilot interviews and set out to carry out multiple holistic case studies to complete their research. This paper presents and discusses in depth the first phase of the research, and the methodology the authors decided to adopt in the second.
1 – Introduction and Theoretical bases 1.1 Performance Measurement in the Public Sector: History and Context Over the last thirty years the field of Performance Measurement has been the focus of considerable attention from academics and practitioners alike (Neely, 1999). Many of the initial studies in the field concentrated on performance measurement in the private sector (Kaplan, 1983; Johnson and Kaplan, 1987), but in recent years a substantial number of studies have explored performance measurement in the public sector (Hood, et al., 1998 and Smith, 1995). Governments are demonstrating growing interest in this issue, as are the press and media. Governments globally – most notably Australia, Canada, the UK and the US - are using performance targets and league tables in their attempts to push through modernization programs and demonstrate that value for taxpayers’ money is being delivered. Observers of these changes coined the phrases “New Public Sector” or “New Public Management” (NPM) to describe them (Hood, 1991; 1995). In many advanced economies, such as those of the Anglo-Saxon countries and Scandinavia, public services have come under increasing pressure to improve their efficiency and effectiveness, reduce their demands on taxpayers, but maintain the volume and quality of services supplied to the public. In the drive to achieve this, public service organizations have been subjected to the introduction of various ‘private sector’ management techniques, frequently complemented by the adoption of some form of neo-market system in which the purchasers and providers of public services are split and required to contract with each other (Brignall and Modell, 2000). Effectively the reforms that have been introduced place great emphasis on agency performance, customer focus, stakeholder's interests and other methods of assessment (Kouzmin et al., 1999). 1
In the literature that deals specifically with performance measurement there are five broad topics that authors interested in New Public Management have focused on. These include: 1- The characteristics and uses of the organization's performance measurement system (PMS) (Berman and Wang, 2000; Halachmi and Bouckaert, 1994); 2- The role of stakeholders, especially in the phase of PMS design (McKevitt and Lawton, 1996; Smith, 1995; McKevitt, Millar and Keogan, 2000); 3- The existence of benchmarking within or outside a certain organizational field (Bowerman, 1995; Poister and Van Slyke, 2002); 4- The balance between internal and external (mandatory) measures (Streib and Poister, 1999; Hyndman and Eden, 2002); 5- The satisfaction related to the PMS and the trade-off between costs and benefits (Cavalluzzo and Ittner, 2004; Boyne et al., 2002). The first area of interest includes the examination of: a) The relationship between PM, organizational mission and strategy (Berman, 2002, Behn, 2003; McAdam and Bailie, 2002); b) The adequacy of the Information System in place (Chen and Perry, 2003; Fuller and Roffey, 1993); c) The competencies and training of people involved in the measurement of performance (Birkett, 1992); d) The level of commitment of managers in the development of a PMS (Bourgault and Tremblay, 1994; Hennessey, 1998). The second consists of issues such as: a) The involvement of the organization or the people responsible for PM in setting the objectives (De Lancer Julnes and Holzer, 2001); b) The effects of public sector reforms on PMS and/or modification of the indicators to be measured (McKevitt and Lawton, 1996); c) The impact of the collected data on the political debate (Smith, 1995; Stewart and Walsh, 1994); d) The number of stakeholders and regulators that influence the organization (Hood, et al., 1998); e) The role of the citizen/customer’s preferences in developing the PMS (McKevitt, Millar and Keogan, 2000); f) The difficulties encountered in designing measures (Di Francesco, 1999; Dobmeyer, Woodward and Olson, 2002; Propper and Wilson, 2003); g) The actual quantification of the service delivered (Heinrich, 2002). The third comprises: a) The communication between the organization and other organizations/companies about PM and how much this communication is promoted internally (Ammons, 1995, 1999; Bowerman, 1995); b) The spread of best or better practices among PSO within or outside an organizational field (sub-sector) and the possibility of cross-national comparisons and spread of 2
practices (Christensen and Yoshimi, 2001; Eshima, Katayama and Ohno, 2001; Rubienska and Bovaird, 1999); c) The possibility to apply lessons learned in the private sector (Poister and Van Slyke, 2002); d) The possibility to have benchmarking within and outside an organizational field (Kouzmin et al., 1999). The fourth area of interest includes the examination of: a) Measures or Key Performance Indicators (KPIs) designed internally in comparison to mandated ones (Sanderson, 2001; Usher et al., 2001; Wiggins and Tymms, 2002); b) The internal support and the level of agreement on the development of the PMS (Streib and Poister, 1999); c) The involvement of the organization as a whole in the process of regulations and KPIs setting and the change of measures and targets over time (i.e. change of the whole measure, change of target, change of people involved etc.) (Van Peursem, Pratt and Lawrence, 1995); d) The involvement and motivation of employees during the development of the PMS, and the frameworks and guidelines the organization has recently had to adopt/comply with (Best Value, Investors in People, GPRA etc.) (Hoggett, 1996; Hyndman and Eden, 2002; Johnsen, 1999; Keenan, 2000; Martin and Davis, 2001). Finally, both academics and practitioners have looked at issues like: a) The impact of the PMS on the organization in terms of efficiency, effectiveness, transparency and accountability (Boyne et al., 2002); b) The quantification of costs associated with PM and of benefits in relation to costs (Cavalluzzo and Ittner, 2004; Grant, 1990; Halachmi, 2002; Hood et al., 1998); c) The perceived benefits and the satisfaction with the PMS (Kelly and Swindell, 2002); d) The use of the collected data (Jackson, 1993; Propper and Wilson, 2003); e) The perceived usefulness of the PMS for the organizations and the employees (Hirschmann, 2002). 1.2 The Need for Theoretical Lenses From an academic perspective, the challenge with much of the work undertaken to date on performance measurement in the public sector is that it either adopts a pragmatic perspective, dealing with the issues outlined above, or a more theoretical stance. Rarely do authors appear to provide a firm theoretical grounding to shed light on their more practical findings. To address this criticism three organization theories will be adopted in this paper – agency theory, institutional theory and resource dependence theory. Agency theory is one of the most frequently used to explain performance measurement systems. In essence, agency theory is based on the premise that agents, although working for principals [the owners of the organization’s assets], have their own agenda to pursue and that this agenda will not necessarily align with the principals’ interests. Hence the need for control mechanisms – performance measurement and incentive systems – designed to ensure alignment between the priorities of the principals and the activities of the agents (Holmstrom and Milgrom, 1991; Jensen and Meckling, 1976). 3
Institutional theorists adopt the position that “no organization is just a technical system and that many organizations are not primarily technical systems. All social systems, hence all organizations, exist in an institutional environment that defines and delimits social reality” (Scott, 1987: 507). The merit of the institutional approach is that it focuses on the objectified and taken-for-granted nature of organizations and organizational environments, and it emphasises the role of institutionalised values in shaping and legitimising the actions and interactions of both organizations and the individuals within them (Aldrich, 1999). Institutional constraints provide the framework within which human interaction takes place (North, 1990). In order to better explain the relationships and interactions between organizations and institutions, institutional theorists have utilised concepts such as isomorphism (DiMaggio and Powell, 1983), legitimacy (Suchman, 1995) and institutional pressure (Oliver, 1991). In institutional environments, such as government organizations, environmental agents have the authority to impose organizational practices on subordinate units or to specify conditions for remaining eligible for continuation of funding (Scott, 1987 and Geiger and Ittner, 1996). Although institutions play an important role in every sector, public sector organizations are more dependent on them both in terms of legitimacy and resource (Meyer and Rowan, 1977; DiMaggio and Powell, 1983). “Private and other market organizations may be particularly persuaded by logics that appeal to economic efficiency, and public and other non-market organizations may be particularly influenced by a ‘logic of appropriateness’” (March and Olsen, 1989 in Casile and Davis-Blake, 2002: 192). It is possible to relate these reflections to other papers found in the more “practitioner” body of literature. Public sector organizations have been depicted as having “no profit maximising focus, little potential for income generation and, generally speaking, no bottom line against which performance can ultimately be measured” (Boland and Fowler, 2000: 417). Moreover, PM systems have been described as always subject to political and social choices, especially in the public sector (Van Peursem, Prat and Lawrence, 1995), and to have important limitations as “drivers” of change and improvement, particularly to the extent that these systems are externally imposed (Sanderson 2001). Resource Dependence Theory has been seen as complementary to Institutional Theory, since it “states that organizational stability is achieved through the exercise of power, control, or the negotiation of interdependencies for purposes achieving a predictable or stable inflow of vital resources and reducing environmental uncertainty” (Oliver, 1991: 149). Other academics have emphasised the complementary nature of these two theories and the possibility of using them together, but mostly at a theoretical level (Etherington and Richardson, 1994; Abernethy and Chua, 1996; Carpenter and Feroz, 2001; Modell, 2001). From a Resource Dependence point of view performance measurement systems can be considered as closely linked with power, self-interest and political advocacy. Regarding interdependence and the acquisition of critical resources, the public sector certainly has specific characteristics that make it different from the private sector. Political decision makers, for example, most often do not directly experience the consequences of their actions (Pfeffer and Salancik, 1978). Moreover, through the use of concepts drawn from resource dependence theory it is possible to better explain and understand the interplay between proactive behaviours and more passive acquiescence towards pressures exerted by institutions on public sector organizations, and organizations’ attempts to acquire legitimacy.
4
1.3 Theoretical framework and conceptual research questions The issues identified in the literature enabled the researchers to formulate the theoretical framework and research questions that drive the research. The characteristics and uses of an organization's performance measurement system, the role of the different stakeholders, and the balance between internal and external (mandatory) measures are among the most important issues identified in the New Public Management literature and require further investigation. Looking at the use of organization theories in the field of PM, various authors have called for an empirical study of the responses deployed by organizations to institutional pressures, and the interconnections between legitimacy, efficiency and effectiveness. Therefore, the conceptual issue this research addresses regards the understanding of the interactions between organizations and institutions in the development of performance measurement targets and indicators. The two conceptual research questions are the following: -
What are the relations between public sector organizations and institutions in the development of performance targets and indicators?
-
What are the strategic responses, conflicts, tensions and behaviours that arise?
1.4 Empirical Research Questions Despite the significant volume of research that has been conducted to date in the field of Public Sector performance measurement, there is a relative paucity of empirical research that describes in detail the systems and processes being used at different levels of government to measure and manage performance. In this context, the phrase “golden thread” is used to track the targets set by central government down to delivery at the local level, i.e. to have an overall coherence of the “chain” of government set targets. Therefore, “golden threads” are present if objectives, targets and performance indicators are consistent throughout the different levels from Central Government to single organizations. In recent years, the English Government has used the metaphor of the “golden thread” to cascade and align objectives, targets and performance indicators throughout the hierarchical levels of the public sector from Central Government to local organizations with the aim of enhancing performance, transparency and accountability, and of influencing behaviour. This research aims to provide deeper insights by studying the previously described theoretical issues in a specific context: the English public sector. Following the insights gained through the literature reviews it is possible to formulate the following empirical research questions: 1- How appropriate is the use of the “golden thread” metaphor in the English public sector? (I.e. How consistent are the practice and theory of the “golden thread” in the English public sector?) 2- What is the influence of institutions on the design of performance targets and indicators in local English public sector organizations and what are the strategic responses and behaviours that arise? The research questions express the need to examine the interactions between institutions and public sector organizations in the development of performance measurement systems. More specifically, the research focuses on the examination of strategic responses to institutional
5
pressures and the nature of managerial choices deployed by public sector organizations in the development of PMS. 1.5 Research Design and Methodology This research consists of two main phases. In the first, more exploratory one, the authors set out to map the performance measurement framework that is effectively in operation in part of the English Public Sector. To do so, they illustrate on one side the linkages between the performance measurement systems used by the Treasury, the Home Office and one of the country’s police forces, and on the other the relations between the PMS developed by the Treasury, the Department of Health, the Healthcare Commission for Primary Care Trusts (PCT) and the indicators used by one PCT. Initially, the authors familiarized themselves with the performance measurement framework in use, as perceived by the Treasury. The data collection methods used included participant observation, document analysis and interviews with key informants. Subsequently, the authors used document analysis to investigate how the framework designed by the Treasury has been applied in the Home Office and deployed through to a Police Force. Similarly, the researchers examined how the framework developed by the Department of Health relates to the one designed by the Healthcare Commission and how these two frameworks are deployed in a Primary Care Trust. The aim of this phase was to establish whether the performance measurement framework in use could be mapped using publicly available materials that Departments and Delivery Agencies use for reporting their performance both to the Treasury and to the Electorate. In the second phase the researchers have decided to broaden their study of the roles of performance measurement in the English public sector. Following the findings of the exploratory phase and subsequent pilot interviews, the researchers decided to use case studies as research strategy, and to perform the collection of data through structured interviews and document analysis in English public sector organizations. This paper presents and discusses in depth the first phase of the research, and the methodology the authors decided to adopt in the second. The collection and analysis of data will be completed in early 2006.
2 – Exploratory cases 2.1 The English Performance Measurement Framework: A Treasury Perspective At the heart of the English performance measurement framework is the Public Service Agreement (PSA) regime. PSAs were first introduced in 1998 and form a central plank in the contract between the Treasury and government departments. In England Spending Reviews are conducted every two years. During these the Treasury and government departments negotiate over departmental priorities, deliverables and budget. The agreed priorities and deliverables are encapsulated in the PSAs, which are published upon completion of the Spending Review process. To assess delivery each PSA consists of a number of objectives, targets and an accompanying Technical Note, which explains how progress towards each target will be measured. The objectives within departments’ PSAs are designed to encapsulate the full scope of the departments’ responsibilities. However, PSA targets focus on departments’ key priorities - e.g. areas where a national aspiration is considered appropriate in order to drive forward a step-change in performance, or where it is appropriate to set a national ‘standard’. In the 2002 Spending Review this process resulted in around 130 6
PSA targets being proposed for 20 different departments. This compares favorably with the 250 performance targets set in 1998 and the 160 targets set as a result of the 2000 Spending Review. In the 2004 spending review this figure has reduced even further, with 110 PSA targets being agreed. For the purposes of this paper, however, the 2002 targets are used, as the 2004 targets had not been agreed at the time the original data on the police and health service was collected. Interestingly, it is apparent from discussions that take place during the Spending Review, that while departments recognize the need to rationalize the number of PSA objectives, targets and measures, there is also a political pressure in each department to increase the number of PSA objectives and targets they have as this increases the department’s perceived influence at governmental level. This observation is in line with notions of organizations wishing to increase their legitimacy as discussed by institutional theorists1. From a Treasury perspective the PSA objectives, targets and accompanying Technical Notes are the major elements of the UK Public Sector Performance Measurement Framework. Once agreed the Treasury recognises that Departments will seek to translate the PSA objectives, targets and accompanying Technical Notes into more detailed statements, but the Treasury plays no formal part in this cascade process. The first phase of this research focuses on two specific departments – the Home Office and the Department of Health – and how these Departments seek to translate the subset of their PSA objectives that relate respectively to the Police Force and Primary Care Trusts. 2.2.1 Police Force - A Departmental Perspective Under the 2002 Spending Review the Home Office agreed an overarching aim, 7 PSA objectives, including one Value for Money target with the Treasury, with 10 associated targets and 37 resultant indicators. Once the PSA objectives, targets and accompanying Technical Notes have been agreed with the Treasury, the individual government departments determine how they will break down these high-level objectives into specific initiatives and actions. There is no mandated process for doing this and hence each Department has discretion over how they translate the PSA objectives and associated targets into language that is meaningful and appropriate. In this section the paper focuses on the approach adopted by one major department - the Home Office – and explored how the Home Office translates its PSA targets into an appropriate performance measurement framework for the English Police Force. Examining the 37 indicators the Home Office agrees with the Treasury, it is possible to see how the first 9 and the 37th apply specifically to the Police Force. Therefore, these are indicators to be considered in this research. In 2002 the Home Office released the first National Policing Plan, covering the time period 2003-2006. In 2003 this plan was updated to cover 2004-2007. The introduction to the first plan explains the performance measurement framework designed by the Home Office as follows: “The Government and the public need to be able to judge whether forces are delivering the quality of service that everyone expects. This requires a robust and transparent performance management system for assessing the effectiveness of forces and individual Basic Command Units (BCUs) in tackling crime and the 1
One of the authors of this paper, Professor Andy Neely, is a member of the Treasury’s Performance Information Panel and observes such discussions on a regular basis.
7
fear of crime. The planning framework at the end of this document sets out how local plans will be measured in terms of raising police performance. Central to this framework are the Home Office Public Service Agreements, which are linked to the national priorities set out above. The key PSAs for the police service (PSAs 1 and 2) require a reduction in crime and the fear of crime and an improvement in the performance of all forces. (…) These national targets are supported by Best Value Performance Indicators (BVPIs). Each police authority sets local targets against the BVPIs and will report its performance annually” (Anon, 2002, p. 4). In essence, the Home Office’s view in 2002 was that the PSAs would translate into BVPIs and these could be used to track performance at the local level. In addition the 2002 plan makes reference to the Policing Performance Assessment Framework (PPAF) which will “develop a set of national standards on performance to ensure a consistent, first class service” (Anon, 2002, p. 32). The PPAF consists of six domains – citizen focus; helping the public; reducing crime; investigating crime; promoting public safety; and resource usage. At the time of data collection, the English Police Forces were assessed on the basis of an interim framework, consisting of 49 separate indicators, documented in National Policing Plan 20032006. These 49 indicators cluster around three broad categories. Six of them are identified as relating to PSA 1: •
PSA 1: Reduce crime and the fear of crime; improve performance overall, including by reducing the gap between the highest crime CDRP areas and the best comparable areas; and reduce: Vehicle crime by 30% from 1998-99 to 2004. Domestic burglary by 25 % from 1998-99 to 2005. Robbery in the ten Street Crime Initiative areas by 14% from 1999-2000 to 2005. And maintain that level.
Other 12 indicators are classified as relating to PSA 2: •
PSA 2: Improve the performance of all police forces, and significantly reduce the performance gap between the best and worst performing forces; and significantly increase the proportion of time spent on frontline duties.
The remaining 31 indicators are derived from the Best Value Regime and mapped onto the six dimension of the PPAF framework. 2.2.2 Police Force - A Local Perspective For the Police, the local level is the county or regional level. The researchers chose to focus on one of these regions – Cambridgeshire. In Cambridgeshire two further plans are developed – the Police Authority Strategic Plan and the Local Delivery Plan. The first of these plans covers the time period 2003-2005, while the second covers twelve months. It should be noted that in the introduction to the latter document the Chief Constable states: “The Police Authority has to maintain an effective and efficient police service for the county under the requirements of the Police Act 1996. Under this same Act we also have to produce a Local Policing Plan. For 2003/04, this continues to be part of an overall Local Performance Plan which includes the key delivery requirements from the National Policing Plan, a Local Priority, the Efficiency 8
Plan, the Best Value Performance Plan, Community Safety Strategies and our Finance and Resourcing Plan” (authors’ emphasis). In the section devoted to performance measures the plan states: “Cambridgeshire Police Authority has a responsibility to monitor and report on the Constabulary’s performance in service delivery in a number of different areas. This responsibility comes from a number of sources, including: •
The National Policing Plan, outlined in the Local Policing Plan;
•
The National Crime Reduction Strategy;
•
The suite of Best Value performance indicators, which support the Best Value Performance Plan; and
•
The Constabulary’s Service Charter.”
Therefore, while in theory the measures should cascade neatly from PSAs to PPAF to Local Delivery Plans, in practice the Local Delivery Plans have also to take account of (i) National Policing Plans, (ii) Local Policing Plans, (iii) Best Value Key Performance Indicators, (iv) Constabulary Service Charters, (v) Efficiency Plans, (vi) Community Safety Strategies and (vii) Financing and Resourcing Plans. This multitude of influences results in the 78 performance indicators used to track performance at the local level. Importantly, while at Treasury and Home Office levels the PSA and associated indicators are most widely discussed, at the local level the Best Value Regime appears to become extremely influential. This is probably because, while central government see the Best Value Regime as outdated, the regime was enshrined in the statute books through the Local Government Act 1999 and hence is still influential at the local level. 2.2.3 Police Force - Analysis of the Different Perspectives Analysing the previously mentioned documents, some interesting and important issues emerge. The first point to note is the rate of growth of indicators through the system. Through the PSA process the Treasury and the Home Office agree 7 PSAs and 37 separate indicators, of which 3 PSAs and 10 indicators are relevant to the Police. In the Home Office’s interim framework these 3 PSAs and 10 indicators are converted into 49 separate indicators. At the local level the 49 indicators identified in the Home Office’s interim framework are converted into 78 separate indicators. Effectively then the cascade process manages to convert the 3 PSAs into 78 separate indicators, giving an average of 26 indicators per PSA. If every department followed the same process, then the 130 PSA targets defined in 2002 would translate into approximately 3380 indicators at the local level. Therefore it is important to question why this happens and whether the resources required to track this number of indicators are justified. In the case of the police it appears that the answer to this question is that the local level performance indicators are derived from the Best Value Review framework introduced by the Labour Government when it first came to power. Indeed it could be argued that there is a disconnect between the Treasury’s PSA Regime and the Best Value Regime. The second point to make is that it has become best practice in the private sector to develop strategy maps (or “success maps”) when designing measurement systems (Eccles and Pyburn, 1992; Neely et al, 2002; Kaplan and Norton, 2004). Yet nowhere in any of the documentation reviewed has there been any mention of them. Strategy maps are designed to help people identify which are the critical indicators and to understand – or at least endeavour 9
to articulate – the relationships between the different measures. Potentially these approaches might prove valuable to those in the Public Sector designing measurement systems as they seek to rationalise the number of measures they use. The third point involves a more detailed analysis of the relationships between the measures. For each of the 78 indicators the research team explored whether it could be mapped directly back onto: (i) the original PSAs; (ii) the 10 PSA indicators; and (iii) the 49 interim indicators identified in the Home Office’s interim framework. This analysis demonstrated that 66.7% of the local level indicators can be linked back directly to the 10 PSA indicators via the 49 interim indicators. However, if the two PSA indicators – 7 and 8 – are removed, then this figure drops to 33.3% of the local indicators. PSA indicators 7 and 8 refer to “performance in the six domains of the Policing Performance Assessment Framework”, which in itself is a very broad ranging framework. The breadth of these two PSAs is illustrated by the fact that 74% of the local level indicators can be mapped onto them. This in itself calls into question their validity. It is interesting to carry this analysis further, for of the 10 PSA indicators, there are 160 mappings – i.e. 160 separate occasions that local level indicators map onto specific PSA indicators. Of these 160 mappings, 4 PSA indicators – 1, 3, 7 and 8 – account for 91.25% of them. This suggests that the measures used at the local level are heavily skewed towards two broad categories of PSA indicators – those that relate to the overall level of crime and those that relate to the PPAF framework. It also causes one to question why the remaining six PSA indicators have been included at the same level in the framework, or at least to ask whether much progress will be made against them. The same analysis can be applied to the interim indicators. In this case there are 138 mappings, split across 42 indicators. For 7 interim indicators there is no corresponding local level indicator, whereas 10 interim indicators account for 52.18% of all mappings, suggesting a bias of attention towards certain specific areas (i.e. violent crime, public satisfaction and fear of crime). 2.3.1 Healthcare - A Departmental Perspective In the UK’s healthcare system there are various bodies involved in the design and development of performance measurement systems. Firstly, the Treasury, through the Department of Health, details the aims, objectives and targets of the whole sector, according to the previously formulated Spending Review. Under the 2002 Spending Review, the Department of Health agreed an overarching aim, 3 PSA objectives, including one Value for Money target, 12 associated targets and 32 resultant indicators. Second, the Healthcare Commission sets specific targets and indicators for the different trusts (Acute, Ambulance, Mental Health, and Primary Care). Third, before each Primary Care Trust applies or adapts these measures to its own particular situation, several other bodies intervene and influence the way performance targets and indicators are structured. The analysis of published materials enabled us to identify four different organizations or frameworks that played a major role in this sense: the Priorities and Planning Framework (PPF); the Personal Social Services Performance Assessment Framework (PSS PAF); Best Value and the Strategic Health Authorities (SHA). Finally, the last level of this hierarchical structure consists of individual Primary Care Trusts, which formulate their objectives, targets and indicators. In the sections that follow the linkages between the performance measurement systems developed by the Department of Health and the Healthcare Commission and how they relate to a Primary Care Trust are examined. The first set of documents to be analysed provided an 10
overview of the performance measurement regime in the UK’s healthcare sector. It was clear from this analysis that a multitude of bodies were involved in defining this regime. Indeed, while the published information is rich, the connections between these different bodies are not always clear, particularly when it comes to the Healthcare Commission and Primary care Trusts. A deeper understanding of these relations requires a more direct contact (e.g. interviews) with people working in this field. 2.3.2 Healthcare - Document Analysis To understand the characteristics and roles of the Department of Health and the Healthcare Commission various documents and Internet sites were consulted. This allowed gathering most of the information about objectives, targets and indicators. The documents related to the Department of Health show the organizational aims, objectives, targets and indicators. The web pages are organized in a very clear and hierarchical structure. The documents related to the targets and indicators set by the Healthcare Commission for the PCTs are much less clear, in that no explicit connections are shown between “key targets” and “Balanced Scorecard” indicators. The documents were analyzed in a four-step process. First the internal consistency of the Department of Health performance measurement system was examined. Second, a similar procedure was followed for the Healthcare Commission’s performance measures for Primary Care Trusts. Third, the consistency between these two systems was examined. Finally the two frameworks were compared with the indicators developed by a Primary Care Trust – the West Suffolk PCT. In the case of healthcare, the authors also focused on the language used in the documents and the identification of possible differences in emphasis placed on specific measures. Using the NVivo software, the elements (objectives, targets and indicators) were mapped and coded. 2.3.3 Document Analysis - Department of Health Each indicator developed by the Department of Health is related to a specific target and each target to an objective. The high level of consistency amongst the different elements enhances the clarity of the system - i.e. targets are consistent with objectives and the indicators are good metrics for the targets to which they refer. For example, the first PSA objective is to “improve service standards”; the targets related to this objective cover five main areas: (1) inpatient and outpatient waiting lists; (2) accident and emergency (A&E) wait; (3) access to professionals; (4) hospital appointments; and (5) accountability to public. It is possible to see how the targets are connected to the overall objective (they all relate to the provision of service) and they express the willingness of the Department of Health to improve the service that has been delivered so far. Examining the value for money target, it is interesting to notice how three keywords of the New Public Management literature, such as value for money, efficiency and effectiveness are all used. A good degree of consistency also exists between targets and indicators. The first target for the first PSA objective, for example, is “reduce the maximum wait for an outpatient appointment to 3 months and the maximum wait for inpatient treatment to 6 months by the end of 2005, and achieve progressive further cuts with the aim of reducing the maximum inpatient and day case waiting time to 3 months by 2008”.
11
The two associated indicators are: (1) the number of people waiting longer than 13 weeks for a first outpatient appointment; (2) the number of people waiting longer than 6 months for inpatient treatment. The first indicator translates the first part of the target in an operational metric (waiting for an outpatient appointment), while the second one strongly relates to the second part of the target (waiting for inpatient treatment). The last part is not translated into an indicator, probably because it refers to a longer term. 2.3.4 Document Analysis - Healthcare Commission The examination of the documents pertaining to the Healthcare Commission (PCTs section) provides a very different picture. In this case the “key targets” are by no means related to the “Balanced Scorecard (BSC) indicators”. Furthermore, it is not clear what it is meant by “Balanced Scorecard”, since the BSC originally developed by Kaplan and Norton (1992; 1996) refers to a totally different model to the one proposed by the Healthcare Commission. It is curious that the description of the “key targets” also includes the description of the indicators that make them operational and that the “Balanced Scorecard” indicators are structured in the same way as the “key targets”. Confusion is therefore added to what is considered a target and what an indicator. Furthermore, two questions emerge from the analysis: 1- What is the relation between PCT key targets and BSC indicators? 2- What is the relation between PCT key targets and BSC indicators, and the Department of Health targets and indicators? To answer the first question, the authors checked whether key targets and BSC indicators dealt with similar issues. The first coding showed little connection between the two, namely no code could be applied to both a key target and a BSC indicator. To confirm or refute these initial findings the researchers undertook a discourse analysis, which enabled them to draw some interesting conclusions: 1- The relation between “key targets” and “BSC indicators” is almost non-existent: just few key terms are used in both parts and there is no evidence of a relationship between them. This applies even to terms that are usually widely utilised in documents regarding performance measurement in healthcare, e.g. child, community, equity, teenager, A&E etc.; 2- A few terms are used in both “key targets” and “BSC indicators” sections, but they often refer to different issues (e.g. access, GP, smoking etc.); 3- In the two areas where there are similarities (inpatient/outpatient waiting and staff surveys), there are differences between the sources of data (information comes from different databases) and the timescales of the indicators; 4- The main difference between “key targets” and “BSC indicators” appears to be in terms of use of technical terms. In the latter some of the targets and indicators relate to specific expressions (CAMHS, CHD, Cervical screening etc.), whereas the former do not use such terms. Therefore, it is possible to conclude that the “key targets” and the “Balanced Scorecard indicators” developed by the Healthcare Commission do not show any evident relation among each other. This contradicts the general assumption in the performance measurement 12
literature that targets and indicators should be strongly connected. Furthermore, the names “key targets” and “BSC indicators” are also inappropriate, since they both refer to targets and indicators at the same time, and, as previously stated, the phrase “Balanced Scorecard” is not properly used either. It appears that the Healthcare Commission could rename their “key targets” as “general targets and indicators”, and the “Balanced Scorecard indicators” as “specific targets and indicators” without significant loss of form or function. Furthermore, in future revisions of the Healthcare Commission’s performance measurement system it would seem appropriate to show how these different sets of targets and indicators are connected, in terms of content, sources of data, type of indicators etc. Although it was not possible to use the same codes in both parts of the Healthcare Commission document, targets and indicators could still be grouped according to the issues that they refer to. In so doing, the main priorities of the Healthcare Commission could be identified and compared with the Department of Health documentation. Eight groups are identified: there are some common areas of interest (mainly access and treatment), but also issues, which are included either in the “key targets” or in the “BSC indicators” (e.g. financial, internal and patient surveys). Furthermore, even in areas of interest like “access”, where two targets and two BSC indicators mapped in each category, it is possible to see how targets and indicators differ. The key targets, in this case, refer to “the percentage of patients able to be offered a routine appointment to see a GP (or a PCP) within one working day”. The first BSC indicator, on the contrary, is “constructed by aggregating responses from the survey questions which relate to access and waiting”. The second concerns “single telephone access through NHS Direct to GP out-of-hours care”, which still refers to access, but by measuring a very specific aspect, not included in the key targets. 2.3.5 Document Analysis: Department of Health and Healthcare Commission Having examined the internal structure of the performance measurement systems developed by the Department of Health (DoH) and the Healthcare Commission (HC) separately, the consistency between the two is now evaluated. The DoH, in fact, sets the overarching aims, objectives, targets and indicators for all public healthcare organizations. The HC translates these into targets and indicators specifically for the acute, ambulance, mental and primary care trusts. This examination focuses on the identification of common aspects and differences between the targets and indicators developed by the DOH and the ones designed by the HC for the PCTs. Furthermore, an analysis of the terms used is performed to explore if there are differences in emphasis on particular issues. The grouping and comparison of concepts (i.e. the nodes of the coding structure) show that several common issues exists, particularly: access to professionals; patient satisfaction; inpatient, outpatient and A&E waiting times; and some aspects regarding certain categories of patients (e.g. children and older people) and specific treatments or technical targets (e.g. drug users, cancer and heart disease). However, to better assess the degree of similarity between the targets and indicators described in the two documents, several terms and phrases (48 in total) were interrogated.
13
These searches confirmed that there were various issues that were dealt with in both documents, but they also emphasised several differences: 1- The DoH targets and indicators and the HC “key targets” show some similarities, while the correspondence with the BSC indicators are limited. This is partly due to the level of detail that the documents divulge - i.e. the BSC indicators often involve technical terms, while the DoH document and the HC key targets do not. 2- Although there are targets that belong to similar areas, the way performance measures are designed is different in most of the cases, e.g. outpatient and inpatient waiting, suicide prevention etc. 3- The HC document shows a strong focus both on service delivery (patient satisfaction) and staff satisfaction, while the DOH one just considers service delivery (the word “staff” is never used throughout the whole document, while in the HC document it is utilised 37 times). Furthermore, while appraisals and surveys are core concepts in the HC document, they are not even mentioned by the DOH. 4- The formulation of the DOH document has been strongly influenced by the New Public Management lexicon, contrarily to the HC one. NPM keywords, like accountability, effectiveness, efficiency and value for money, can be found in the DOH text, but not in the HC text. 5- Some words or phrases, which could be expected to appear in such documents, are totally absent in the DOH document, e.g. equity, human resource, ethnic, financial, information, safety, staff, smoking, etc. 6- Comparing the two documents, it seems that while the objectives, targets and indicators developed by the DOH have the aim of informing citizens, the HC mixed similar elements with metrics that will be used to assess the way various Trusts operate. This means that external indicators concerning service delivery, for instance, are sometimes mixed with internal measures regarding imposed mechanisms – e.g. audits correctly carried out, etc. This generates confusion about the main goal of the performance measurement system. The comparison of the Department of Health and the Healthcare Commission documents emphasises a substantial degree of incongruence and, certainly, an insufficient explicit linkage between the two. The DOH never mentions the Healthcare Commission, whereas the HC refers just seven times to the Department of Health and only twice to Public Service Agreements. In this respect, a more explicit connection is strongly advocated as well as the explanation of the existence of certain targets and indicators. Finally, looking at the main elements of the two documents is it possible to notice how the DOH targets correspond just partially to the HC ones. More specifically, just 4 of the PSA targets are totally covered (i.e. 1, 3, 5, 6), 6 are partially covered (2, 7, 9, 10, 11, 12) and 2 are not covered at all (4, 8). 2.3.6 Document Analysis: DoH, HC and a Primary Care Trust Following the procedure used for the Police Force, the researchers compared the documents produced by the Department of Health and the Healthcare Commission with those produced by the West Suffolk Primary Care Trust. This analysis investigated the differences between the targets and indicators used by the West Suffolk PCT and the ones set by the DoH and the HC. The results is that the degree of similarity is very low. The comparison, in fact, shows 14
that just 21% of the indicators used by the PCT are present also in the two other documents. While a further 15% are used in one of them, but not the other and in 43% of cases it was not possible to find any correspondence.
3. Discussion and Conclusions The analysis reported in the previous chapters suggests that in both the police and health service cases the level of consistency in terms of measures, and therefore targets and priorities, is relatively low across the hierarchy that constitutes the public sector. Therefore, it is important to understand the reasons why there is no coherent set of priorities for the English public services, which align with a golden thread running from central government to local delivery agencies. The three organization theories introduced at the beginning of the paper help to shed light on the findings obtained in this first phase of the study. As stated before, agency theory is concerned with aligning the interests of principals and agents. In the public sector it is not immediately clear who the principals are. From a departmental perspective it could be argued that the Treasury is the principal. Yet members of the Treasury itself, and particularly elected officials, might argue that the electorate is the principal. However, the Treasury’s objective is “to raise the rate of sustainable growth and achieve rising prosperity and a better quality of life, with economic and employment opportunities for all”, while one of its ten subobjectives is to “improve the quality and the cost effectiveness of public services”. Hence the Treasury implicitly sees itself either as the principal or the principal by proxy. Its interest is in ensuring that the agents - government departments and agencies - pursue courses of action that are in the principal’s best interests. At the departmental level, however, the concepts of legimacy and efficiency take precedence, as the aim of departments is to survive and increase their influence. This is transparent to those closely involved in the Public Spending Review process, where it is clear that departments want to maximise their PSA targets as a way of demonstrating their importance and legitimacy in delivering the government’s agenda. Concepts drawn from institutional and resource dependence theory seem particularly powerful in explaining what happens also further down the hierarchy. At the local level, the constant battle is for resources, which can be acquired either pro-actively or by accommodating external institutional pressures. Organizations like the Cambridgeshire Police Force and the West Suffolk Primary Care Trust need access to resource if they are to deliver service. They are forced to adopt measurement targets and indicators that can be used in the political negotiation process. And yet this is where the system breaks down. For it is not always clear at the local level who holds the power, and hence access to finance, at the central level. At the local level organizations are forced to respond to the requests of the PSA framework imposed by the Treasury, but also to frameworks imposed by others – e.g. the Best Value regime of the Office of the Deputy Prime Minister. It is these competing demands that cause much of the confusion at the front line about the true priorities. 3.1 Implications for the Policy and Research Communities One of the most significant implications of the findings of this first phase is that indicators cascading through the public sector appear to be multiplying at a significant rate. At the level of the Treasury relatively few targets and indicators are agreed. At the level of the central government department – the Home Office and the Department of Health - significantly more 15
indicators are introduced, while at the local level the number of indicators mushrooms once again. In some ways this is unsurprising. In any organization there is always more that can be measured at the front line than in the equivalent of the Head Office. What is concerning, however, is the lack of any explicit sense of priority amongst the multitude of indicators at the local level or indeed amongst those at the central government departmental level. From a theoretical perspective it seems clear that part of the problem is the interaction between the conflicting priorities expressed by the organizations involved through their performance measurement systems. The Treasury is concerned by agency problems, departments by legitimacy concerns, while resource dependence is the dominant issue at the local level. Recognising the different stances and hence responses of different actors at different levels in the system is essential if we are to make some progress in reconciling and rationalising the performance measurement regimes that exist. Finally, as a result of this first phase, a number of questions emerged. The first concerns the lack of learning in the public sector from the private sector. In the private sector there has been much progress in addressing some of the issues identified and many of the lessons learned could be applied to the public sector. Second, while investigating measurement in the public sector, the identification of the role of the measurement system becomes a primary concern. It could be argued that the Public Service Agreements process (and indeed the Best Value regime) is designed to control local delivery, or to improve accountability, or to influence behaviour and action in the public services. Undoubtedly this last role is the one that most private sector organizations appeared most interested in today. The findings of this phase, however, suggest that it is the one that the examined PMS least enable.
4. Future research 4.1 The second phase of the project Building on the findings of the first phase of the research, the authors decided to broaden their research on the roles of performance measurement in the English public sector. In order to define the research and data analysis strategy to be used in the second phase, several pilot interviews were carried out in different organizations. The organizations chosen belong to several public sector domains - i.e. local government (two county councils and a borough council), healthcare (a strategic health authority and an ambulance trust), an environmental organization, a police force, a fire authority and a cultural institution. In so doing, it was possible to obtain a variety of issues wider than the one achievable by looking at just one area. Furthermore, this provided the researchers with a better understanding of the feasibility of a study across different sectors. The pilot interviews were “informal conversation interviews”, which relied “entirely on the spontaneous generation of questions in the natural flow of an interaction" (Patton, 1990: p. 280). The authors just prompted the discussion on performance management issues and asked questions following the interviewees’ answers. In so doing, it was possible to explore different issues across the interviews and to identify some contingent or common factors, some of which could be related to the previously reviewed literature. In particular, the following themes emerged: -
Although there are many documents available on the web, it is extremely hard to gather all the relevant information on objectives, targets and indicators of a public sector organization. Very rarely are there integrated PMS at the local level and this is reflected in an incredible dispersion of data, even where performance managers and
16
performance teams are present. Many measures are owned by different individuals and reported to other people and organizations, often without being checked by a central department within the agency. Furthermore, the indicators to be published are very often the statutory ones, whereas this whole project would like to concentrate also on what is done differently by the organizations and why; -
Most of the interviewees showed great interest – and, at the same time, concern regarding the development of a performance culture within the organization. The imposition of performance targets and indicators from other institutions (often identified with the expression “performance monitoring”), and a culture of blame seem to be the main barriers to the achievement of a culture of performance within organizations;
-
There is a high variability in the targets and indicators used by public organizations. The highly political nature of designing them and the frequent changes in general policy (also, but not only, as a result of changes in national or local political leadership) make it very hard to establish cause-and-effect relationships. Furthermore, there are very often differences in timescales: measures adopted at a certain point in time may result in changes in different moments also depending on contingent factors;
-
It is very difficult to quantify inputs, outputs, outcomes, productivity and quality in the public sector. Many authors have criticized the publications of league tables and public sector reports, as statistically flawed and drivers of wrong behaviours (Micheli, Mason and Kennerley, 2005). Consequently, the assessment of costs and benefits related to PM in local organizations, although very interesting, seems not feasible or, at least, very much dependent on a number of assumptions and caveats;
-
In several organizations, the number of performance indicators against which data are currently collected are surely neither manageable, nor are they sufficiently related to strategy or organized according to priorities;
-
The complexity of an organization’s PMS, if present, greatly varies across local organizations. While in certain agencies all the indicators belong to just one level, in other ones their number may be extremely high. In County Councils, for example, there is often a corporate scorecard plus 32 frameworks or lists of indicators for each of the separate services, which are then sub-divided in various other levels. In a similar case, it would be extremely hard to carry out a comprehensive analysis of such a stratified and complicated system.
4.2 Research design The conclusions of the first phase and the insights gained through the pilot interviews led the researchers to choose multiple and holistic case studies (Yin, 1994) and a combined qualitative and quantitative approach as research strategies and ways to analyze the data in this second phase of the project. Case study research has been depicted as an indispensable building block for theorising in management accounting (Kaplan, 1983; Carpenter and Feroz, 1992). Case studies are mostly used “when the investigator has little control over events” (Yin, 1994; p. 1) and when the research deals with a contemporary phenomenon within some real-life context and the context is deliberately part of the design. Case study is also a preferred research strategy when different types of evidence are utilised, the so-called methodological triangulation. In 17
this research, methodological triangulation will be achieved mainly by analysing written texts and performing structured interviews. While carrying out the cases, the researchers will continuously compare theory and data, iterating toward a theory that strongly fits the data (Eisenhardt, 1989). Triangulation through the use of different methods of data collection is expected to strengthen the validity of the research (Voss, Tsikriktsis and Frohlich, 2002), and to enrich the researchers’ understanding of the phenomena under investigation by allowing for new or deeper dimensions to emerge. Finally, the use of qualitative methods will play a prominent role by eliciting data and suggesting conclusions which other methods could not enable to draw, and “potentially generate what anthropologists call “holistic work” or “thick description”” (Jick, 1979: p. 609). The cases will be primarily exploratory and descriptive and only partly explanatory. Organizations will be selected on the basis of theoretical sampling - cases are chosen for theoretical, not statistical, reasons (Eisenhardt, 1989) - either through literal replication (trying to predict similar results) or theoretical replication (to produce contrasting results but for predictable reasons). The diversity of the organizations considered will improve the significance of cases and the possibility to reach theoretical saturation (Yin, 1994). Considering organizations that belong to different public sector domains implies that the researchers would have to get familiar with various types of public agencies with dissimilar structures and ways in which performance is measured and managed. Carrying out multiple cases in a single domain would be less onerous, but it would limit the breadth of the conclusions the authors will be able to draw. Furthermore, most academics are specialists in particular areas of government – e.g. local government, healthcare and education - so comparisons across and between sectors would be valuable. 4.3 Data Collection and Analysis – A Realist approach Following the findings of the pilot interviews, the researchers will start the data collection and analysis phase. The structured interviews will consist of questions derived from the literature and the insights gained through the pilot phase. A matrix has also been designed to capture and structure the information related to the development of the indicators, and to identify possible discrepancies between how an indicator is mandated and the way it is defined and used by the local organization. The matrix will be filled in by one person per organization (the performance manager or the CFO) (Table 1). The interviews will be carried out (structured with a limited number of probes) in line with a realist approach to data collection and analysis (King, 2004). The researchers’ concern will be to obtain accurate information from the interviewees, untained by relationship factors. While trying to be self-reflexive, acknowledging their personal bias and previous knowledge, the authors will try to minimize the impact of inter-personal processes on the course of the interviews. Through this approach it is assumed that “the accounts participants produce in the interviews bear a direct relationship to their 'real' experiences in the world beyond the interview situation” (King, 2004: p. 12). Finally, a sample representing important distinctions within the organizational population will be included; this will improve the validity of the analysis, since it increases the number of different viewpoints collected via interviews.
18
Local level 1
Measure
2
Metric/formula
3
Target level
4
Frequency of measurement and reporting
5
Source of data
6
Reasons for introducing the measure – expected benefits
7
Use of the information
8
Overlap with other measures
Higher level
Match
Table 1: Map of the indicators – Matrix
The information gathered through documents and interviews will be analysed consistently (the NVivo software will be utilised for the coding) and themes and findings will be compared, following a methodological triangulation. Analysis will be quantitative (content analysis and counting) and qualitative (the richness will be rendered mainly through exemplifications and quotes to illustrate the arguments). At the end of the analysis the data obtained will be compared to the theory and theoretical propositions will be formulated using concepts, constructs and categories coming from the literature and emerging from the data. The researchers are currently carrying out the case studies and the project is expected to be completed in early 2006.
5. References Abernethy, M. A. and Chua, W. F. (1996), ‘A field study of control system "redesign": The impact of institutional processes on strategic choice’, Contemporary Accounting Research, Vol. 13, No. 2, pp. 569-606. Aldrich, H. E. (1999), “Organizations Evolving”. Sage Publications, London. Ammons, D.N. (1995), “Overcoming the Inadequacies of Performance-Measurement in LocalGovernment - the Case of Libraries and Leisure Services”, Public Administration Review, Vol. 55, No. 1, Jan-Feb, pp. 37-47. Anon (2002), “The National Policing Plan: 2003-2006”, Home Office, November. Behn, R.D. (2003), “Why Measure Performance? Different Purposes Require Different Measures”, Public Administration Review, Vol. 63, No. 5, Sep-Oct, pp. 586-606. Berman, E. (2002), “How Useful Is Performance Measurement”, Public Performance and Management Review, Vol. 25, No. 4, Jun, pp. 348. Berman, E. and Wang, X. (2000), “Performance Measurement in U.S. Counties: Capacity for
19
Reform”, Public Administration Review, Vol. 60, No. 5, Sep/Oct, pp. 409-420. Birkett, W.P. (1992), “A Question of Competence”, Australian Accountant, Vol. 62, No. 8, September, pp. 35. Boland, T. and Fowler, A. (2000), ‘A systems perspective of performance management in public sector organizations’, International Journal of Public Sector Management, Vol. 13, No. 5, pp. 417–446. Bourgault, J. and Tremblay, Y. (1994), “Performance Management at the Crossroads - Between Rigor and Laxity”, Canadian Public Administration-Administration Publique Du Canada, Vol. 37, No. 4, Win, pp. 547-572. Bowerman, M. (1995), “Auditing Performance Indicators: The Role of the Audit Commission in the Citizen’s Charter Initiative”, Financial Accountability and Management, Vol. 11, No. 2, May, pp. 171-183. Boyne, G., Gould-Williams, J., Law, J. and Walker, R. (2002), “Plans, Performance Information and Accountability: the Case of Best Value”, Public Administration, Vol. 80, No. 4, pp. 691-710. Brignall, S. and Modell, S. (2000), “An Institutional Perspective on Performance Measurement and Management in the “New Public Sector”“, Management Accounting Research, Vol. 11, September , pp. 281-306. Carpenter, V. and Feroz, E. (1992), ‘GAAP as a symbol of legitimacy: New York State's decision to adopt generally accepted accounting principles’, Accounting, Organizations and Society, Vol. 17, No. 7, p. 613-643. Carpenter, V.L. and Feroz E H (2001), 'Institutional theory and accounting rule choice: An analysis of four US State governments' decisions to adopt generally accepted accounting principles', Accounting, Organizations and Society, Vol. 26, No. 7-8, pp. 565-596. Casile, M. and Davis-Blake, A. (2002), ‘When accreditation standards change: Factors affecting differential responsiveness of public and private organizations’, Academy of Management Journal, Vol. 45, No. 1, pp. 180-195. Cavalluzzo, K. S. and Ittner, C. D. (2004), “Implementing Performance Measurement Innovations: Evidence from Government”, Accounting, Organizations and Society, Vol. 29, No. 3-4, Apr/May, pp. 243-267. Chen, Y-C. and Perry, J. (2003), “Outsourcing for E-Government: Managing for Success”, Public Performance and Management Review, Vol. 26, No. 4, Jun , pp. 404. Christensen, M. and Yoshimi, H. (2001), “A Two-Country Comparison of Public Sector Performance Reporting: the Tortoise and the Hare?”, Financial Accountability and Management, Vol. 17, No. 3, August, pp. 271-289. De Lancer Julnes, P. and Holzer, M. (2001), “Promoting the Utilization of Performance Measures in Public Organizations: an Empirical Study of Factors Affecting Adoption and Implementation”, Public Administration Review, Vol. 61, No. 6, Nov/Dec, pp. 693-708. Di Francesco, M. (1999), “Measuring Performance in Policy Advice Output: Australian Developments”, The International Journal of Public Sector Management, Vol. 12, No. 5, pp. 420. DiMaggio, P.J. and Powell, W.W. (1983), 'The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields', American Sociological Review, Vol. 48, pp. 147160. Dobmeyer, T.W., Woodward, B. and Olson, L. (2002), “Factors Supporting the Development and Utilization of an Outcome-Based Performance Measurement System in a Chemical Health Case Management Program”, Administration in Social Work, Vol. 26, No. 4, pp. 25-44. Eccles, R.G. and Pyburn, P.J. (1992) “Creating a Comprehensive System to Measure Performance”, Management Accounting [US], October, pp. 41-44.
20
Eisenhardt, K. M. (1989), ‘Building theories from case study research’, Academy of Management Review, Vol. 14, No. 4, pp. 532-550. Eshima, Y., Katayama, T. and Ohno, T. (2001), “Public Management Innovation in Japan: Its Characteristics and Challenges”, International Review of Administrative Sciences, Vol. 67, No. 4, Dec, pp. 699-714. Etherington, L. and Richardson, A. (1994), ‘Institutional pressures on university accounting education in Canada’, Contemporary Accounting Research, Special Education research issue, pp. 141-162. Fuller, D. and Roffey, B. (1993), “Improving Public-Sector Accountability and Strategic DecisionMaking”, Australian Journal of Public Administration, Vol. 52, No. 2, Jun, pp. 149-163. Geiger, D. R. and Ittner, C. D. (1996), ‘The influence of funding source and legislative requirements on government cost accounting practices’, Accounting, Organizations and Society, Vol. 21, No. 6, pp. 549-567. Grant, B. (1990), “Measuring Performance in the Public Sector”, Management Accounting, Vol. 68, No. 8, September, pp. 58. Halachmi, A. (2002), “Performance Measurement, Accountability, and Improved Performance”, Public Performance and Management Review, Vol. 25, No. 4, Jun, pp. 370. Halachmi, A. and Bouckaert, G. (1994), “Performance Measurement, Organizational Technology and Organizational Design”, Work Study, Vol. 43, No. 3, May/Jun, pp. 19-25. Heinrich, C.J. (2002), “Outcomes-Based Performance Management in the Public Sector: Implications for Government Accountability and Effectiveness”, Public Administration Review, Vol. 62, No. 6, Nov-Dec, pp. 712-725. Hennessey, J.T. (1998), “"Reinventing" Government: Does Leadership Make the Difference?”, Public Administration Review, Vol. 58, No. 6, Nov-Dec, pp. 522-532. Hirschmann, D. (2002), “Thermometer or Sauna?: Performance Measurement and Democratic Assistance in the United States Agency for International Development (Usaid)”, Public Administration, Vol. 80, No. 2, pp. 235-255. Hoggett, P. (1996), “New Modes of Control in the Public Service”, Public Administration, Vol. 74, No. 1, Spr, pp. 9-32. Holmstrom, B. and Milgrom, P. (1991) “ Multitask Principal Agent Analyses: Incentive Contracts, Asset Ownership and Job Design” Journal of Law, Economics and Organisation, 7, pp. 24-52 Hood, C. (1991), ‘A public management for all seasons?’, Public Administration, Vol. 69, No. 1, pp. 3–19. Hood, C. (1995), ‘The “New Public Management” in the 1980s: Variations on a theme’, Accounting, Organizations and Society, Vol. 20, No. 2/3, pp. 99-109. Hood, C., James, O., Jones, G., Scott, C. and Travers, T. (1998), “Regulation Inside Government: Where New Public Management Meets the Audit Explosion”, Public Money and Management, Vol. 18, No. 2, Apr-Jun, pp. 61. Hyndman, N.S. and Eden, R. (2002), “Executive Agencies, Performance Targets and External Reporting”, Public Money and Management, Vol. 22, No. 3, Jul-Sep, pp. 17-24. Jackson, P.M. (1993), “Public Service Performance Evaluation: A Strategic Perspective”, Public Money and Management, Vol. 13, No. 4, Oct-Dec, pp. 9-14. Jensen, M.C. and Meckling, W.H. (1976) “Theory of the Firm: Managerial behaviour, Agency Costs and Ownership Structure”, Journal of Financial Economics, 3, pp. 305-360. Jick, T. D. (1979), ‘Mixing qualitative and quantitative methods: Triangulation in action’, Administrative Science Quarterly, Vol. 24, No. 4, pp. 602-611.
21
Johnsen, A. (1999), “Implementation Mode and Local Government Performance Measurement: A Norwegian Experience”, Financial Accountability and Management, Vol. 15, No. 1, Feb, pp. 4166. Johnson, H.T. and Kaplan, R.S. (1987), Relevance Lost – the Rise and Fall of Management Accounting, Harvard Business School Press, Boston, MA. Kaplan, R.S. (1983), “Measuring Manufacturing Performance: A New Challenge for Managerial Accounting Research”, The Accounting Review, Vol. 58 No. 4, pp. 686-705. Kaplan, R.S. and Norton, D.P., (1992), ‘The Balanced Scorecard - Measures that drive performance’, Harvard Business Review, Vol. 70, No. 1, pp.71-79. Kaplan R. S. and Norton D.P. (1996), ‘Using the Balanced Scorecard as a strategic management system’, Harvard Business Review, Vol. 74, No. 1, pp. 75-85. Kaplan, R.S. and Norton, D.P. (2004), “Strategy Maps: Converting Intangible Assets into Tangible Outcomes”, Harvard Business School Press, Boston, MA. Keenan, J. (2000), “Just How New Is Best Value?”, Public Money and Management, Vol. 20, No. 3, Jul-Sep, pp. 45-49. Kelly, J.M. and Swindell, D. (2002), “A Multiple-Indicator Approach to Municipal Service Evaluation: Correlating Performance Measurement and Citizen Satisfaction Across Jurisdictions”, Public Administration Review, Vol. 62, No. 5, Sep-Oct, pp. 610-621. Kennerley, M. and Neely, A.D. (2003) “Measuring Performance in a Changing Business Environment”, International Journal of Operations and Production Management, 23, 2, 213-229. King, N. (2004), ‘Using interviews in qualitative research’, in Cassell, C. and Symon, G. (eds.), Essential guide to qualitative methods in organizational research. Sage Publications. pp. 11-22. Kouzmin, A., Loffler, E., Klages, H. and Korac-Kakabadse, N. (1999) “Benchmarking and Performance Measurement in Public Sectors: Towards Learning for Agency Effectiveness”, The International Journal of Public Sector Management, 12(2):121. March, J. G. and Olsen, J. P. (1989), Rediscovering institutions: The organizational basis of politics, Free Press: New York. Martin, S. and Davis, H. (2001), “What Works and for Whom? The Competing Rationalities of Best Value”, Policy and Politics, Vol. 29, No. 4, Oct, pp. 465-475. McAdam, R. and Bailie, B. (2002), “Business Performance Measures and Alignment Impact on Strategy: The Role of Business Improvement Models”, International Journal of Operations and Production Management, Vol. 22, No. 9/10, pp. 972. McKevitt, D. and Lawton, A. (1996), “The Manager, the Citizen, the Politician and Performance Measures”, Public Money and Management, Vol. 16, No. 3, Jul-Sep, pp. 49-54. McKevitt, D., Millar, M. and Keogan, J.F. (2000), “The Role of the Citizen-Client in Performance Measurement: the Case of the Street Level Public Organization (Slpo)”, International Review of Administrative Sciences, Vol. 66, No. 4, Dec, pp. 619-636. Meyer, J.W. and Rowan, B. (1977), 'Institutionalized organizations: Formal structure as myth and ceremony', American Journal of Sociology, Vol. 83, No. 3, pp. 340-363. Micheli, P., Mason, S., Kennerley, M. and Wilcox, M. (2005). “Public sector performance: Efficiency or quality?”, Measuring Business Excellence, Vol. 9, No. 2. Modell, S. (2001), 'Performance measurement and institutional processes: A study of managerial responses to public sector reform', Management Accounting Research, Vol. 12, No. 4, pp. 437464. Neely, A.D. (1998) “Measuring Business Performance: Why, What, How”, Economist Books, London.
22
Neely, A.D. (1999) “The Performance Measurement Revolution: Why Now and Where Next”, International Journal of Operations and Production Management, Vol. 19, No. 2, pp. 205-228. Neely, A.D.; Adams, C. and Kennerley, M. (2002) “The Performance Prism: The Scorecard for Measuring and Managing Business Success”, Financial Times/Prentice Hall, London. North, D. C., (1990), “Institutions, Institutional Change and Economic Performance”. Cambridge University Press, Cambridge. Oliver, C. (1991), 'Strategic responses to institutional processes', Academy of Management Review, Vol. 16, No. 1, pp. 145-179. Patton, M. Q. (1990), Qualitative evaluation and research methods, 2nd Edition, Sage Publications. Pfeffer, J. and Salancik, G. R. (1978), The external control of organizations: A resource dependence perspective. New York: Harper and Row. Poister, T.H. and Van Slyke, D.M. (2002), “Strategic Management Innovations in State Transportation Departments”, Public Performance and Management Review, Vol. 26, No. 1, Sep, pp. 58. Propper, C. and Wilson, D. (2003), “The Use and Usefulness of Performance Measures in the Public Sector”, Oxford Review of Economic Policy, Vol. 19, No. 2, Summer, pp. 250. Rubienska, A. and Bovaird, T. (1999), “Performance Management and Organizational Learning: Matching Processes to Cultures in the Uk and Chinese Services”, International Review of Administrative Sciences , Vol. 65, No. 2, Jun, pp. 251-268. Sanderson, I. (2001), “Performance Management, Evaluation and Learning in “Modern” Local Government”, Public Administration, Vol. 79, No. 2, Summer, pp. 297-313. Scott, W. R. (1987), ‘The adolescence of Institutional Theory’, Administrative Science Quarterly, Vol. 32, No. 4, pp. 493-511. Smith, P. (1995), “Performance Indicators and Outcome in the Public-Sector”, Public Money and Management, Vol. 15, No. 4, Oct-Dec, pp. 13-16. Stewart, J. and Walsh, K. (1992), “Change in the Management of Public-Services”, Public Administration, Vol. 70, No. 4, Winter, pp. 499-518. Streib, G.D. and Poister, T.H. (1999), “Assessing the Validity, Legitimacy and Functionality of Performance Measurement Systems in Municipal Governments”, American Review of Public Administration, Vol. 29, No. 2, Jun, pp. 107-123. Suchman, M.C. (1995), 'Managing legitimacy: Strategic and institutional approaches', Academy of Management Review, Vol. 20, No. 3, pp. 571-610. Usher, C.L., Locklin, E., Wildfire, J.B. and Harris, C.C. (2001), “Child Welfare Performance Ratings: One State”s Approach”, Administration in Social Work, Vol. 25, No. 1, pp. 35-51. Van Peursem, K.A., Pratt, M.J. and Lawrence, S.R. (1995), “Health Management Performance: a Review of Measures and Indicators”, Accounting, Auditing and Accountability Journal, Vol. 8, No. 5, pp. 34. Voss, C., Tsikriktsis, N., Frohlich, M. (2002), ‘Case research in operations management’, International Journal of Operations & Production Management, Vol. 22, No. 2; pp. 195-220. Wiggins, A. and Tymms, P. (2002), “Dysfunctional Effects of League Tables: a Comparison Between English and Scottish Primary Schools”, Public Money and Management, Vol. 22, No. 1, Jan-Mar, pp. 43-48. Yin, R. K. (1994), Case study research. Sage Publications, Beverly Hills, CA.
23
