32nd EGPA Annual Conference Toulouse, France 8 to 10 September 2010
Performance Measurement in Spanish Local Governments. An Empirical Study about the State of the Art. Lourdes Torres, Vicente Pina and Caridad Martí Autobiographical note Dr. Lourdes Torres is Full Professor of Public Sector Accounting at the University of Zaragoza (Spain). http://gespublica.unizar.es. E-mail:
[email protected] Dr. Vicente Pina is Full Professor of Accounting and Finance at the University of Zaragoza (Spain). http://gespublica.unizar.es. E-mail:
[email protected] Dr. Caridad Martí is Lecturer of Accounting and Finance at the University of Zaragoza (Spain). http://gespublica.unizar.es. E-mail:
[email protected]. Contact Author and address: Vicente Pina, Department of Accounting and Finance, University of Zaragoza, Gran Vía, 2. 50.005 Zaragoza. ACKNOWLEDGEMENT: This study has been carried out with the financial support of the Spanish National R&D Plan through research projects SEJ2007-62215ECON/FEDER and ECO2010-17463 -ECON/FEDER.
1
Performance Measurement in Spanish Local Governments. An Empirical Study about the State of the Art. ABSTRACT It is not possible the assessment of public action efficiency without performance measures. Through the analysis of the implementation and the state of the art of performance measures in the Spanish Local Governments, this paper shows the difficulty of comparing developments of performance measurement systems when adoption is non-mandatory. The results identify different approaches for the adoption of performance measures: decision-making, accountability and just collected for their intrinsic value. Performance indicators have the intrinsic value of giving taxpayers an image of managerial rationality, but the mere collection of performance indicators does not guarantee improvements in decision-making and in accountability.
Key words: performance measurement, local governments, accountability, decisionmaking.
2
Introduction The development of tools to measure performance for improving efficiency, accountability and management by results is on the agenda of many local governments as a public expression of their concern about the good use of taxes. The goal of improving efficiency and effectiveness is not only a matter of managerial rationality but especially a political issue on the agenda of OECD countries. More than two-thirds of OECD countries1 include non-financial performance data in the budget documentation available to managers and policy makers (internal use) and provide reports on their performance to the public (external use). The disclosure of non-financial information and the introduction of performance indicators are not usually required by law in OECD countries. When there are requirements, they do not refer to specific indicators and permit governmental discretion for their definition and introduction. As the introduction of performance indicators is usually an internal decision, heterogeneous practices have arisen. From the perspective of internal use, performance measurement systems create incentives that help to align individual goals with the objectives of the organization, provide valuable feedback information on the progress towards these objectives, and form the basis for internal and external accountability (Cavalluzzo and Ittner, 2004; Heinrich, 2007; Kravchuk and Schack, 1996). From the perspective of external use, transparency has become a widespread symbol of ‘good governance’ in many different contexts today (O’Neill, 2008) and a key piece for holding governments more accountable to citizens. The image of professionalism and rational decision-making derived from the collection of information through performance measurement symbolizes government sensitiveness towards the wasting and the adequate allocation of taxpayer resources.
3
This paper analyses the context and the extent to which non-mandatory performance measures are developed in Spanish local governments in order to determine if they are used for internal purposes (decision-making), external (accountability) purposes or they are not used, and just collected for their intrinsic value. It seeks reasons for the decision of local governments to adopt performance indicators and the implications of these reasons. After the contextual and methodological sections, the alternative uses of performance measures are discussed in the light of the analysis of the survey results with the purpose of identifying the factors that drive the decision to adopt these measures voluntarily. The study of what is taking place in Spain with regard to performance indicator developments will be useful to academics and practitioners to better understand the problems of introducing and, especially, of using non-financial performance measures in local governments managed under administrative law.
Performance measures in public sector reforms Various authors have shown the difficulties of establishing theoretical frameworks for explaining public management changes. According to Lee, Perara and Rahman (2006), where studies involve phenomena which are multi-faceted and about which little is known, it is difficult to find theories that explain these changes satisfactorily. Because of this, it will sometimes be better to analyse the rationality of public management changes based first on empirical evidence, rather than developing a priori hypotheses and then testing them (‘grounded theory’). This approach is appropriate in order to learn from participants and to capture their experience of the phenomenon under investigation (for example, see Gibbins, Richardson and Waterhouse, 1990; Holland, 2005). A range
4
of theories has been used for explaining the introduction of performance measures into governments. Some papers focus on a rational and economic argumentation according to which the role of performance measures is analysed according to their contribution to enhancing efficiency, effectiveness and internal decision-making processes (Anthony and Young, 1984; Aidemark, 2001; Baraldi, 2005). Other papers consider the concept of institutional pressure as a determinant of performance measure developments (Lapsley, 2001; Nyland and Pettersen, 2004). The use of performance measures to decision-making purposes (internal use) seeks to move the focus of budgeting, management and accountability away from inputs towards results in order to produce better decisions. This interest has led to a substantial increase in the quantity of performance information available to decision-makers in the last three decades. Nevertheless, as Broadbent and Laughlin (2003) and Letza, Smallman and Sun (2005) pointed out, in the public sector, accountability relationships between principals and agents are complex and open-ended or not explicitly defined and, thus, not easily monitored. Government officials do not volunteer more information than is required or in their interest (Zimmerman, 1977)2. It is, therefore, not surprising that, financial and performance disclosures are often made only in response to demand. The use of performance measures to accountability (external use) has become a prevailing principle in public management which all public sector organizations seem to be obliged to observe. However, as the introduction of rational decision-making processes into governmental entities is complex, compliance and enforcement of that principle may be limited to the visible components of rational decision-making, especially to the collection and display of information (Dalehite, 2008). So, it may not come as a surprise that managerial innovation could be undertaken simply for image
5
and legitimacy since, after two decades of NPM reforms, no government can say that performance measuring is not good. Many studies about the implementation of public sector management reforms have used the institutional theory (DiMaggio and Powell, 1983) to explain the features of these implementations and the gap between rhetoric and actual results. According to this theory, the adoption of performance measures can be viewed as a process of formal compliance with the wishes and expectations of the external environment and stakeholders. The intentional or unintentional separation between external image and actual structures and procedures has been referred to as 'decoupling'. Public sector entities are required to demonstrate accountability, and managerial systems are seen as one way in which public sector entities can legitimize their operations. Other authors such as Wilson (1992), Matland (1995) and Barrett (2004) highlight the concepts of ‘conflict’ and 'ambiguity’, often involved in public management reforms, to explain the gap found by implementation studies between rhetoric and actual results in public sector management reforms. These authors explore how the conflict level of reforms and ambiguity in the implementation policy of reforms affect the way in which these reforms are implemented. For Matland (1995), rational models assume that there is social consensus about how to maximize individual or social welfare functions (subject to a set of constraints), whereas bureaucratic models (Allison 1971; Halperin 1974) state that these utility functions cannot be identified because there is no consensus about the goals between actors. The rational model defines reform processes as if conflicts do not exist and the bureaucratic model highlights conflicts as a central issue in reform processes. On this basis, Matland (1995) proposes four implementation perspectives grounded on the concepts of ambiguity and conflict: administrative implementation, where there are low levels of conflict and ambiguity and in which the
6
process of implementation is primarily managerial; experimental implementation, where the level of conflict is low but the level of ambiguity is high and key actors experiment with implementation; political implementation, where the level of conflict is high but the level of ambiguity is low and in which actors have clearly defined goals; and symbolic implementation, where there are high levels of conflict and ambiguity and in which disagreements tend to be resolved by coercion, bringing about ‘decoupling’ between reforms and present organizational structures, which results in little real organizational change. Other factor in the decision to adopt performance measures is the role of education through public administration schools, specialized masters in universities and training by governmental, professional organizations and consultancy firms which have been transmitting the performance measure approach to graduates in public management.
The Spanish Case Spain shows characteristics common to other Southern European countries, which belong to a public administration culture grounded in administrative law and influenced by the French legal model. During the 80s and 90s, Spain implemented a number of NPM reforms. In the early 80s, when public sector downsizing and performance measures were gaining popularity in the USA, Canada and the UK, social pressure for improving the quantity and quality of Spanish public services led to a growth in public expenditure from 24.4 per cent of the GDP in 1975 to 45.5 in 1995. The majority of Spanish public sector employees are civil servants and, as a consequence, they are not ruled by private sector labor legislation. Their special status
7
increases the difficulties for managerial reforms focused on introducing competition, performance and outcomes evaluation, and performance-related systems of payment. Notwithstanding, attempts to disclose information on the services provided and results achieved are expanding in Spain and other Southern European Union countries. The Act that implements the Spanish Public Sector Chart of Accounts at local level, effective from 2006, states that local governments with more than 50,000 inhabitants should include information about the financial performance, cost and effectiveness of the services delivered in their financial report. Notwithstanding, the Act only provides examples of non-financial performance indicators that local governments can follow on a voluntary basis, so most local governments do not disclose this information. The lack of guidelines and the lack of political will to monitor compliance with the regulation means that the actual use of performance measures in Spain remains a voluntary decision. Performance indicators will probably remain optional because of the potential “high level of conflict” involved in any public management reform due to the civil servant status of local government staff and the power of public sector trade unions and because of the long Spanish tradition of municipal self-governance in which directly-elected politicians make the decisions on the organization of the delivery of local services, taxes, fees and performance management models. Methodology The sample is made up of the 144 Spanish local governments with more than 50,000 inhabitants as of January 1, 2008, because they are the ones legally required by the Spanish Local Government Chart of Accounts of 2004 to disclose performance indicators in their annual report together with the financial statements. Below this
8
threshold, performance measurement is irrelevant in practice. We called all these 144 cities by telephone to find out whether they had implemented performance measures, before sending a questionnaire adapted from Poister and Streib (1999)3. Finally, the questionnaire was answered by 30 local governments (see Annex A) with some kind of experience in the development of performance measures. Taking into account that 42 municipalities claimed not to use in fact performance indicators, our response rate is 50%, which is considered a mean survey response rate for these studies (see Nazari, Kline and Herremans, 2006). The respondent municipalities include all the big Spanish cities and, in total, the local government respondents represent 22.9% of the population and 25.86% of Spanish local government expenditure.
The questionnaire was addressed to the expert manager or technical staff in charge of the development of performance measures in each local government, as identified in the exploratory phone contact. The survey included questions about the background of the municipality, the development, operation, outcomes and applications of performance measurement. Eight questions were to be answered with yes/no, four blocks of questions included a Likert scale from 1 to 4, and in two sections the municipalities had to indicate what services have performance information and the kind of performance indicator. Univariate analysis and the Cronbach coefficient alpha have been applied to analyze the data collected. The Cronbach coefficient alpha, which is the internal reliability measurement4, is based on the average inter-correlation among the items and tests the reliability and/or consistency of multi-item variables or blocks of questions. It ranges from 0 to 1. Values of 0.60 and 0.70 are deemed the lowest level of acceptability. The
9
Cronbach coefficient alpha is the primary method used to assess survey instrument reliability and validity in many studies (Nazari, Kline and Herremans, 2006).
Analysis of results Use and Context of Performance Measurement Table 1 shows the number and percentage of respondent municipalities which agree with the statements of the background section of the survey. The first question refers to the motivations to implement performance measures in local authorities. The best ranked are ‘the desire to make better management decisions’ (83.3%) and ‘the pressure from elected city officials’ (43.3%). Notwithstanding, the frequency with which the information is provided and the answers to subsequent questions about the non-use of performance measurement information for decision-making purposes show that performance measures are elaborated only for the collection of information and not for managerial purposes. Thus, the first is a rhetorical answer, because in other questions, they almost always say that the information is provided annually or quarterly and these frequencies mean that local governments do not use this information for decisionmaking purposes. The answers to question one reflect the concern of local governments to make the use of rational decision-making techniques, which are not effectively used, visible through the implementation of data-gathering processes. Another answer, ‘Citizen demands for greater accountability’, is stated by 23.3% of those interviewed. This answer is related to the institutional image value of collecting and displaying information, since ‘greater accountability’ is not usually required directly by citizens but expresses a formal and democratic principle.
10
The audiences of performance measurement reporting scoring over 60% are the ‘heads of departments, program managers and other line managers’, ‘the Mayor’ and ‘the City council members’. Most of them report to various offices, so that the results highlight the most important internal destinations of performance measures information. The frequency of the distribution of performance information to the users is annual, in 76% of cases. So, most local governments distribute annual performance measurement information among internal staff as a visible component of rational decision-making, although in fact, it is not used for this purpose. The information is mostly prepared by operating departments and the City council staff office. Only in seven out of thirty cases is the information prepared by the Budget Office. The Spanish local governments studied show concern for the use of performance measures, even though the development of true performance measurement systems is still at an early stage. The main reason for their implementation is for the collection of information and the fulfilment of legality and accountability purposes, rather than for rational decision-making. Another question was whether the municipalities conduct a comprehensive citizen survey every year. Ten out of thirty municipalities answered positively and another two municipalities reported that they conduct a citizen survey every two years. The information obtained from these surveys is taken into account in further developments of performance measures. A performance measurement system may lead to decisions to privatize or to externalize some of the operations of the municipality. Six local governments have used this information for privatization, especially in services without social added value, such as garbage collection. In any case, garbage collection, urban transport, water supply and street cleaning are often outsourced by Spanish local governments, although the lack of
11
performance measurement disclosures in almost all Spanish local governments makes benchmarking difficult. INSERT HERE TABLE 1 Blocks B, C, D and E were designed for local governments with centralized, city-wide performance measurement systems and they were filled in by the eighteen local governments which reported having implemented centralized systems. Block B deals with background issues related to the importance of performance indicators in the city’s management processes, such as ‘management by objectives’, ‘strategic planning’, ‘benchmarking’ or ‘incentive systems’. Table 2 shows the mean and standard deviation of each dimension of this block. According to our respondents, the management processes for which performance indicators are considered most important are: ‘performance-oriented budgeting processes’ together with ‘strategic management processes’. Local governments prefer not to use of performance measures for ‘pay-forperformance programs’ or ‘personnel oriented processes’. These answers are not surprising since budgeting by performance and strategic management systems are related and performance measures are very useful for these purposes. But, to some extent, the answers are rhetorical, since, in Table 1, budget departments are not within the main destinations of performance measures. This highlights that, questions of image and accountability are primary purposes of performance measures. This block of questions scores a Cronbach coefficient alpha of 0.927, which shows a high level of internal reliability. Block C analyses the development of performance measures. As can be seen in Table 2, we have distinguished between the positive factors and some difficulties in the implementation of performance measurement. The answers of the cities show that managers play an important role in the development of performance measures and that
12
most of them seem to have trouble in involving lower-level employees. Lower-level employees are not usually involved in developing performance measures and, might even, create resistance, which has been avoided or limited by not linking payments with performance. Citizens are not directly involved in the development of performance measures, although they are indirectly involved through opinion surveys. The Cronbach’s alpha obtained in section C2 is also quite high (0.764). INSERT HERE TABLE 2 Table 3 collects the most common services provided by law by Spanish cities with more than 50,000 inhabitants. The survey included a section about the services for which performance indicators are provided and the kind of indicators defined: output, unit cost or efficiency, outcomes or effectiveness, service quality and citizen satisfaction. All the cities studied have developed performance indicators for key local government services, although with different structures. Output indicators, which represent the workload of the services, have been indicated more than other kinds of performance measures. For example, in police and waste collection and disposal, the percentage of municipalities which elaborate output measures surpasses 70%. The local governments studied have also set up effectiveness indicators in more programs than they have efficiency indicators. By contrast, efficiency indicators are only elaborated for some services. A pragmatic reason is that it is easier to elaborate output information because, considered alone, it is not sensitive to political or managerial criticism in possible benchmarking and/or evaluation processes. These results are similar to those found by Poister and Streib (1999)’s on USA local governments. INSERT HERE TABLE 3
13
Problems and Impacts of Performance Measures Block D deals with the operation of performance measurement systems and a variety of potential problems with such systems. The answers show ambiguity and vagueness in the use of performance measurement by Spanish local authorities. Regarding the question referring to the capacity of the municipality staff to analyse the performance measurement data, the mean is 2.78, which is between sometimes and rarely. So, there is some heterogeneity within local authorities in relation to the qualifications of the staff for analysing performance indicators. Municipalities also state that there is sometimes trouble measuring the quality of the programs and services, as well as keeping performance measures up-to-date. Block E of the survey studies the outcomes of performance measurement (Table 4). Local governments studied give a positive global opinion about the implementation experience and about improvements in the quality of decisions, quality of services and accountability of individual managers. Their opinion is not so positive about the benefits of performance measure implementation in ‘changing program priorities’, in ‘improving the level of employee focus on organizational goals’ and ‘reduction of costs’. The most important outcomes are in the items with high ambiguity such as: ‘improvements in service quality’, ‘improvements in the accountability of individual managers’ and ‘improvements in the quality of decisions or decision-making capacity’. The lowest scores in the outcomes of performance measurement are again those related to the measurement of personnel performance such as ‘improvements in the objectivity of personnel performance appraisals’. The results of this table confirm the lack of use of performance measurement systems in issues related to human resources management, such as ‘personnel performance appraisals’ or in other issues such as ‘the focus or
14
emphasis of programs’ and ‘the reductions in the cost of city operations’. The Cronbach coefficient alpha obtained in this block of questions is 0.877. INSERT HERE TABLE 4 Discussion The local governments studied show that the motivations for the introduction of performance measurement seem to fall into two categories: intra-organizational determinants of performance measurement, basically using an economic and rational decision-making argumentation –all local governments agree that performance measures help produce better decisions– and extra-organizational determinants of performance measurement, deriving from the idea that organizations adapt their structures and processes to external demands and expectations. Our results show that the Spanish local governments studied give priority to the intrinsic value of collecting information with limited consideration to its use and dissemination. These results show, as Zimmerman (1977) stated, that government officials do not volunteer more information than is required or in their interest and that performance disclosures are often made only in response to demand. The Spanish local governments are a case of the discretionary introduction of performance measures into local governments since there is no top-down pressure from central government and the great variance among the cities studied is probably due to this fact. Notwithstanding, the mismatch between the dual perception of external vs internal use remains. Anglo-Saxon countries have imposed, through legal disposition, audit office or professional bodies, the disclosure of a pool of comparable performance indicators which are not used for decision-making because they are not aligned with the managerial objectives of the entities. In the Spanish case, where local governments have
15
no guidelines but freedom to design their own performance measurement systems, the problem of the duality of external vs internal use is reproduced. In designing an implementation policy, ‘conflict’ and ‘ambiguity’ are often negatively correlated. Ambiguity is the Spanish way to limit conflicts. The Spanish central government, instead of using guidelines and clear procedures for the design of performance measures and coercion to be sure about their actual implementation, has adopted a policy that minimizes potential conflicts, possibly because of the political autonomy of Mayors and local governments, the powerful public sector trade unions and the special status of civil servants. These issues were revealed by the answers related to the non-use of performance measures in introducing pay-for-performance. The policy of avoiding conflicts makes the Spanish case a mix of ‘experimental’ and ‘symbolic’ models. The Spanish approach could be considered as ‘experimental’ because the implementation policy of performance measures has been to avoid conflicts by giving local governments freedom whether to design and disclose non-financial performance measures. It could be considered as ‘symbolic’ because the weak integration in the decision-making structures means that local governments maintain their work routines and procedures after the implementation, which leads to a decoupling between performance measures and job responsibilities. The professional training of local government staff could play an especially important role in the highly ambiguous policies implemented in Spain. Traditionally, in Spanish public administration, the top level staff has law degrees and severe shortages in management techniques. It is difficult for them to obstruct reforms focused on improvements in accountability and transparency because they are very well considered by the public and, at least nominally, by elected officials. In these cases, managers look for ways of minimizing the impact of public management reforms on their work by
16
treating performance measures as a formal requirement that has little practical importance for their real work and routines and this contributes to creating the above mentioned decoupling effect. Several lessons could be learnt by practitioners and academics alike. Performance indicators have the intrinsic value of giving taxpayers an image of managerial rationality, but the mere collection of performance indicators does not guarantee improvements in decision-making and in accountability. How to combine and make the dual perspective of external and internal use of performance measures compatible is a pending matter worldwide. Both the Anglo-Saxon approaches, based on the introduction of top-down performance measures and guidelines, and the Spanish and other European continental countries’ initiatives, based on non-mandatory bottom-up approaches, fail to overcome the decoupling effect. In Anglo-American and EU continental approaches, the mismatch between external and internal uses remains. So, a worldwide challenge for academics and, especially, for practitioners will be to strengthen the link between the performance measures disclosed and the performance information use for decision making. Conclusions The results show that the symbolic or institutional image value of making decisions under rational decision-making models is the primary driver of implementation processes in the Spanish case. Even though, in most cases, managers plan to use performance measurement information for decision-making purposes in the future, at present this information is rarely used for these purposes. This paper shows the difficulty of comparing developments of performance measurement systems when adoption is non-mandatory, because of the lack of actual
17
results against which to assess the impact of each implementation process. This study shows that performance measurement systems, in practice, are designed to fit into the traditional organization structure of each local government. As each local government has different approaches, goals and objectives, the choice of suitable performance indicators is at the discretion of the management. This makes benchmarking between cities difficult since performance indicators, when they are disclosed, are not comparable. The answers of questionnaires show that central governments or oversight bodies have no influence over performance indicator design and disclosure by local governments. Although public management literature shows the impact of the institutional image value on the use of performance measurements, in the Spanish cases it is not clear how the design of these measurements can serve the ad hoc interests of the actors who are involved in the processes. As citizens remain unaware of the implementation processes and performance measures have no real impact on organizational structures, an explanation seems to be that of creating a reputation for the city of being ‘well managed’, which is consistent with the interests of politicians and senior managers. Notes
1
OECD (2005) ‘… ignoring municipal accounting reforms is a rational act…’ (Zimmerman, 1977:111), ‘or, … municipal accounting reforms will continue to be ignored until the reformers are able to specify changes in the institutional framework which provide different incentives for the public officials (or voters). The most obvious institutional change would be federal legislation… The impact of such legislation is, ultimately, an empirical issue’. (Zimmerman, 1977, pp. 134 – 135) 3 During the phone calls, 42 municipalities stated that they were not familiar with performance measures and no further contact was made, 7 answered by email stating they do not have a performance indicators system, and others preferred to read the survey in order to see if they were able to answer it. 4 Examples of the application of the Cronbach’s alpha in the accounting literature are Cavalluzzo and Ittner (2004) and Wang and Berman (2001). 2
18
References Aidemark, Lars-Göran (2001). The Meaning of Balanced Scorecards in the Health Care Organisation. Financial Accountability and Management. 17:1 pp 23- 40. Allison, G. T. (1971). Essence of Decision. Little, Brown and Co. Anthony, R.N. and Young, D. (1984). Management Control in Non-profit Organizations. Irwin Chicago. Baraldi, S. (2005). Il Balanced Scorecard nelle aziende sanitarie. Milano: McGraw-Hill. Barrett, S.M. (2004). Implementation Studies: Time for a Revival? Personal Reflections on 20 Years of Implementation Studies. Public Administration, 82:2 pp 249- 262. Broadbent, J. and Laughlin, R. (2003). Control and legitimation in government accountability processes: the private finance initiative in the UK. Critical Perspectives on Accounting, 14:1/2 pp 1-22. Cavalluzzo, K.S. and Ittner, C.D. (2004). Implementing performance measurement innovations: evidence from government. Accounting, Organizations and Society, 29:2-3 pp 243-267. Dalehite, EG. (2008). Determinants of Performance Measurement: An Investigation into the Decision to Conduct Citizen Surveys. Public Administration Review, 68:5 pp 891-907. DiMaggio, P. J. and Powell, W. (1983). The iron cage revisited: institutional isomorphism and connective rationality in organisational fields. American Sociological Review, 48 pp147-160. Gibbins, M., Richardson, A. and Waterhouse, J. (1990). The Management of Corporate Financial Disclosure: Opportunism, Ritualism, Policies and Processes. Journal of Accounting Research, 28 (1) pp 121-143. Halperin, M. H. (1974). Bureaucratic Politics and Foreign Policy. Washington: Brookings Institution. Heinrich, C. (2007). Dynamics in performance measurement system design and implementation. EGPA Conference, Madrid, September. Holland, J. (2005). A grounded theory of corporate disclosure. Accounting and Business Research, 35:3, pp 249-267. Kravchuk, R.S. and Schack, R.W. (1996). Designing Effective Performance Measurement Systems under the Government Performance and Results Act of 1993. Public Administration Review, 56:4 pp 348–58.
19
Lapsley, I. (2001). Accounting, Modernity and Health Care Policy. Financial Accountability and Management, 17:4 pp 331-350. Lee, J., Perara, H. and Rahman, A. (2006). The Grounded theory: a theory discovery method for accounting research. In Methodological issues in accounting research. Theories, methods and issues, ed. Zahirul Hoque. Spiramus Press Great Britain, pp 129- 160. Letza, S.R., Smallman, C. and Sun, X. (2004). Reframing privatisation: Deconstructing the myth of efficiency. Policy Sciences 37:2 pp 159-183. Matland, R. E. (1995). Synthesizing the Implementation Literature: The Ambiguity-Conflict Model of Policy Implementation. Journal of Public Administration Research and Theory, 5: 2 pp 145-174. Nazari, J., Kline, T. and Herremans, I. (2006). Conducting Survey Research in Management Accounting, included in Hoque, (ed.) Methodological Issues in Accounting Research. Theories, Methods and Issues, Spiramus. Nyland, K. and Pettersen, I.J. (2004). The Control Gap: The Role of Budgets, Accounting Information and (Non)-Decisions in Hospital Settings. Financial Accountability and Management, 20:1 pp 77-102. OECD. (2005). Modernising Government: The way forward. Paris: OECD. O’Neill, O. 2008. Transparency and the Ethics of Communication. In Transparency: The key for better governance?, eds. Christopher Hood and David Heald, Oxford University Press for the British Academy, 75-91 Poister, T. and Streib, G. (1999). Performance Measurement in Municipal Government: Assessing the State of the Practice. Public Administration Review, 59:4 pp 325-335. Wang, X. and Berman, E. (2001). Hypotheses about Performance Measurement in Counties: Findings from a survey. Journal of Public Administration Research and Theory, 11:3 pp 403- 428. Wilson, D. (1992). A Strategy of Change: Concepts and Controversies in the Management of Change. London: Routledge. Zimmerman, J.L. (1977). The municipal accounting maze: an analysis of political incentives. Journal of Accounting Research, 15 (Supp) pp 107–144.
20
Table 1. BACKGROUND INFORMATION Motivations to implement performance measures - The desire to make better management decisions - Citizen demands for greater accountability - Pressure from elected city officials - Pressure from the business community - State or regional legislative mandates - Recommendations or Best practices Primary audience for performance measurement reports - City manager, chief administrative officer or other executive staff - Mayor or professional staff in the mayor’s office - City council members - Department heads, program managers, other line managers - Budget officials, personnel officials, other professional staff - State and federal funding agencies - Citizen advisory boards or groups - Other: (Citizenry etc.) Frequency of the distribution of performance information to the users - Monthly - Quarterly - Six monthly - Annually Primary responsibility for managing and maintaining the performance measurement system is from: - The City Manager’s Office - Mayor’s Office - Operating Departments - Planning Department - City Council Staff Office - The Budget Office Do you conduct a comprehensive citizen survey every year?
Have data from your performance measurement system led to decisions to privatize or to externalize some of your operations?
Cities
%
25 7 13 2 7 6
83.3% 23.3% 43.3% 6.7% 23.3% 20%
13 21 18 23 13 5 6 5
43.3% 70% 60% 76.7% 43.3% 16.7% 20% 16.7%
8 12 5 23
26.7% 40% 16.7% 76.7%
2 1 12 1 14 7 10 2
6.7% 3.3% 40% 3.3% 46.7% 23.3% 33% every year; 6,7% every two years 20%
6
21
Table 2. BACKGROUND INFORMATION AND THE DEVELOPMENT OF PERFORMANCE MEASURES (For users with centralized, city-wide systems) B. How important are performance measures to the following management processes in your city? ‘1’ Very important; ‘2’ Important; ‘3’ Somewhat important; ‘4’ Not at all important.
Mean
Standard Deviation 1.031
a) Personnel-oriented processes, such as management-by-objectives or 2.44 appraisal by objectives. b) Performance-oriented budgeting processes, such as program 1.65 0.862 budgeting, target-based budgeting, or results-oriented budgeting systems. c) Strategic planning activities, such as identifying organizational 1.81 0.834 strengths and weaknesses or tracking program needs, etc d) Strategic management processes, such as tracking the progress of 1.73 0.961 strategic initiatives, monitoring the achievement of strategic goals and objectives, or measuring performance in key results areas. e) “Benchmarking” of city departments and services against other, 2.20 1.014 similar cities. f) Collecting baseline data for monitoring the effectiveness of total 2.13 0.957 quality management or continuous process improvement efforts. g) Incentive systems such as pay-for-performance, shared savings, or 2.76 0.970 gain-sharing programs. h) Targeting particular programs for more intensive evaluations. 2.71 0.726 i) Managing the impact of the local government activities on the 1.94 0.998 environment (21 Agenda) Cronbach’s alpha for Section B: 0.927 THE DEVELOPMENT OF PERFORMANCE MEASURES C.1.To what extent do the following statements describe how performance measures are developed in your city Mean Standard Deviation We involve managers in the development of performance 1.56 0.705 measures We involve citizens and/or citizen’s groups in the development of 3.19 0.750 performance measures. Data from citizen surveys help to determine criteria to include in 2.21 1.051 our measurement system. We involve lower-level employees in the development of 2.28 0.958 performance measures Cronbach’s alpha for Section C.1: 0.599 C.2. To what extent do the following statements describe how performance measures are developed in your city ‘1’ Usually; ‘2’ Sometimes; ‘3’ Seldom; ‘4’ Never
Mean We have trouble getting the city council to support our performance measurement system. We have trouble getting lower-level employees to support our performance measurement system We have trouble getting citizens to support our performance measurement system. Cronbach’s alpha for Section C.2: 0.764
2.71
Standard Deviation 0.89
2.33
1.113
3.00
1.044
22
Table 3. Performance Measure Applications
Police Fire Prevention/ Suppression Animal Control Community Planning and Zoning Code Enforcement/ Inspection Housing Water Supply/ Sewerage Solid Waste Collection/ Disposal Street Maintenance/ Construction Traffic Public Transport Libraries Parks & Recreation City Clerk Purchasing Data Processing Budget & Finance Personnel/ Human Resources
Output 75.00% 35.71% 57.14% 53.57% 46.43% 46.43% 60.71% 78.57% 57.14% 50.00% 57.14% 50.00% 60.71% 25.00% 35.71% 50.00% 57.14% 46.43%
Efficiency Effectiveness Service Quality 39.29% 60.71% 50.00% 14.29% 25.00% 25.00% 32.14% 32.14% 21.43% 17.86% 28.57% 25.00% 10.71% 21.43% 25,00% 14.29% 25,00% 14.29% 21.43% 32.14% 28.57% 39.29% 39.29% 25.00% 28.57% 39.29% 25.00% 21.43% 32.14% 21.43% 17.86% 28.57% 28.57% 17.86% 28.57% 21.43% 25.00% 28.57% 17.86% 10.71% 10.71% 0.00% 10.71% 10.71% 10.71% 14.29% 21.43% 10.71% 32.14% 35.71% 10.71% 17.86% 21.43% 7.14%
Satisfaction 46.43% 21.43% 14.29% 17.86% 14.29% 10.71% 25.00% 39.29% 32.14% 28.57% 35.71% 25.00% 28.57% 0.00% 0.00% 7.14% 7.14% 3.57%
23
Table 4. OPERATION AND OUTCOMES OF PERFORMANCE MEASUREMENT THE OPERATION OF PERFORMANCE MEASURES D.1. To what extent do the following statements describe how performance measures operate in your city? ‘1’ Commonly; ‘2’ Sometimes; ‘3’ Rarely; ‘4’ Not at all. Mean Standard Deviation We use similar measures to compare performance among 2.33 1.237 operating units. We use measures to track performance over time. 1.78 1.003 When developing performance measures, we focus on what is 1.89 0.900 important, not on the data available. Our measures are derived from the mission, goals, objectives and 1.78 0.878 service standards. Data from our performance measurement are used in pay rises and 2.11 0.758 promotions. Cronbach’s alpha for Section D1: 0.674 D.2. To what extent do the following statements describe how performance measures operate in your city? ‘1’ Commonly; ‘2’ Sometimes; ‘3’ Rarely; ‘4’ Not at all Mean Standard Deviation We have trouble measuring the quality of our programs and 2.11 0.758 services. We have trouble keeping our performance measures up-to-date. 2.31 0.793 Our staff lacks the analytical skills needed to analyse the 2.78 0.943 performance measurement data. Cronbach’s alpha for Section D2: 0.727 PERFORMANCE MEASUREMENT OUTCOMES E. How would you characterize the impact of your performance measurement system on the following areas? ‘1' Substantial; ‘2’ Moderate; ‘3’ Slight; ‘4’ None Mean Standard Deviation Changes in the focus or emphasis of programs. 2.35 0.862 Improvements in the quality of decisions or in decision-making capacity. 1.82 0.883 Improvements in the objectivity of personnel performance appraisals. 2.94 0.998 Improvements in service quality. 1.75 0.577 Improvements in the level of employee focus on organizational goals. 2.19 0.834 Reductions in the cost of city operations. 2.38 0.957 Improvements in the relationship between administrators and elected 2.06 1.063 officials. Improvements in the accountability of individual managers. 1.81 0.834 Cronbach’s alpha for Section E: 0.877
24
Appendix A: List of Spanish municipalities which answered the questionnaire
Municipality
Population
Autonomous Community
A Coruña
245,164 Galicia
Alcalá de Henares
203,645 Madrid
Badalona
215,329 Catalonia
Barcelona
1,615.908 Catalonia
Bilbao Castellón de la Plana Cornellá de Llobregat Getafe Getxo
353,34 Basque Country 177,924 Valencian Community 85,18 Catalonia 164,043 Madrid 81,26 Basque Country
Gijón
262,47 Asturias
Girona
92,186 Catalonia
Madrid
3,213.271 Madrid
Málaga
566,447 Andalusia
Manresa
75,053 Catalonia
Mataró
119,78 Catalonia
Mijas
70,437 Andalusia
Mollet del Vallés
51,912 Catalonia
Palma de Mallorca
396,57 Balearic Irelands
Pamplona
197,275 Navarra
Pozuelo de Alarcón
81,365 Madrid
Roquetas de Mar
77,423 Andalusia
Sabadell
203,969 Catalonia
San Cugat
76,274 Catalonia
Sant Boi de Llobregat
81,335 Catalonia
Santa Coloma de Gramanet
117,336 Catalonia
Santander
182,302 Cantabria
Sevilla
699,759 Andalusia
Tarragona
137,536 Catalonia
Viladecans
61,718 Catalonia
Zaragoza
666,129 Aragón
25
