CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS, 2018 https://doi.org/10.1080/10286608.2018.1425403
Measuring water utility performance using nonparametric linear programming Gamze Güngör-Demircia, Juneseok Leea and Jonathan Keckb a
Department of Civil and Environmental Engineering, San José State University, San José, CA, USA; bCalifornia Water Service Company, San José, CA, USA ABSTRACT
ARTICLE HISTORY
In this study, a cutting-edge methodology for measuring the performance of water utilities based on two-stage Data Envelopment Analysis (DEA) was applied to individual districts of a California-based water utility. A bootstrap technique involving the construction of confidence intervals was implemented to overcome the deterministic nature of conventional DEA, and a number of exogenous variables were incorporated into the model to help identify the factors affecting technical efficiency. Results indicated high overall performance achieved by the utility on average (92%). The number of connections and precipitation were found to be statistically significant exogenous variables, and both were determined to have a negative impact on efficiencies. The findings of this study are expected to be useful for guiding subsequent managerial improvement initiatives.
Received 2 April 2017 Accepted 3 January 2018 KEYWORDS
Data envelopment analysis; double bootstrap; exogenous variables; performance assessment; water utility
Introduction As in many parts of the world, California has been suffering severe droughts, a particularly acute problem since it is one of the most populous states in the US. Due to the large gap between the state’s available water supply and continually rising demand, the implementation of more efficient management practices by water utilities has become vital. The technical, environmental, financial, legal and social considerations associated with these managerial efforts have led many water utilities in California (and elsewhere) to incorporate the Effective Utility Management (EUM). The EUM framework emphasises ten core elements, which have a central effect on a water utility’s operations as well as structure. These ten elements are: (i) water resource sustainability, (ii) product quality, (iii) customer satisfaction, (iv) employee and leadership development, (v) operational optimisation, (vi) financial viability, (vii) infrastructure strategy and performance, (viii) enterprise resiliency, (ix) community sustainability, and (x) stakeholder understanding and support (EUM Utility Leadership Group 2017). The California Water Service Company (Cal Water) serves in four states: California, Washington, New Mexico, and Hawaii (as both a regulated and non-regulated). Cal Water’s service elements include production, testing, storage, treatment, purchase, distribution as well as the sale of water. Cal Water supplies to water to approximately 478,000 CONTACT Juneseok Lee
[email protected]
© 2018 Informa UK Limited, trading as Taylor & Francis Group
2
G. GÜNGÖR-DEMIRCI ET AL.
customers (1.7 million people) within California and has 24 different districts. The primary infrastructure includes about 5700 miles of water main, 970 booster stations, 650 wells, 420 water storage facilities, and 450 supervisory control and data acquisition (SCADA) transmission units in CA (Keck and Lee 2015). With the size and complexity of Cal Water’s operations, not all its component districts are expected to function at the same level of efficient operation. The authors, therefore, sought to develop a useful performance measurement tool that can be applied to the individual districts of Cal Water as part of the utility’s operational benchmarking and optimisation effort. Performance assessment of the individual service areas will help identify the deficiencies in their management practices as well as economic performance, while also providing valuable insight into the barriers that must be overcome to operate a fully efficient water utility. Finally, by providing these collective insights, the results of this study are expected to help Cal Water more deeply incorporate EUM elements/ philosophies into its culture. Performance assessment in the water industry has been paid attention recently (Norton and Weber 2009; Renzetti and Dupont 2009; Berg and Marques 2011; Zschille and Walter 2012; Ananda 2014; Marques, Berg, and Yane 2014; Guerrini et al. 2015; Wibowo and Alfen 2015). Data Envelopment Analysis (DEA) is an optimisation based tool for measuring the efficiencies of the organisational units (Charnes, Cooper, and Rhodes 1978). Building on the conventional DEA methodology, two additional analytical stages were included to examine the exogenous variables’ impacts on the individual districts’ performance, namely, double-bootstrap truncated regression (Simar and Wilson 2007) and Tobit regression (Henningsen 2016). To the best of authors’ knowledge, this is the first study that applies two-stage DEA based on double-bootstrap truncated and Tobit regression to a water utility in the US, and also the first attempt to assess the California water utility’s performance using DEA. It has been observed that expert judgments have been preferred over methodological procedures in selecting water sector’s input and outputs (Ananda 2014; Guerrini et al. 2015; Molinos-Senante, Maziotis, and Sala-Garrido 2015). This article is also contributing to the literature by applying a stepwise input and output selection procedure (Wagner and Shimshak 2007). The next section provides details of the methodology used, generalised DEA theory, the selection and use of representative variables (in terms of inputs and outputs), the bootstrapping procedure, and the considerations of exogenous variables. The results of the research will then be presented, followed by a discussion of their practical implications. The paper concludes with a series of observations, remarks, and suggestions for future research.
Methods Efficiency calculation by DEA DEA is a nonparametric linear programming (LP) based technique to measure the efficiencies of the considered units, which are designated as Decision Making Units (DMUs) (Charnes, Cooper, and Rhodes 1978). Charnes, Cooper, and Rhodes (1978) model was based on the assumptions of constant returns to scale (CRS) and input orientation. In CRS, an increase in inputs will lead to a proportional output increase, while an input-
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
3
oriented model minimises inputs while holding outputs constant. In practice, it is reasonable to adopt input-orientation for the water sector since water utilities are required to provide service to all customers within their service areas (Norton and Weber 2009; Renzetti and Dupont 2009; Zschille and Walter 2012; Ananda 2014; Marques, Berg, and Yane 2014; Guerrini et al. 2015). A mathematical model based on the assumptions of input orientation and CRS is described as follows (Coelli et al. 2005). For a sample of N DMUs having K inputs and M outputs, the dataset consists of an input matrix of X = K × N and an output matrix of Y = M × N. For the nth DMU, these inputs and outputs can be represented by the column vectors xn and yn, respectively. Using the duality in linear programming, the problem can be solved by the following equivalent envelopment form: Minimizeu,l u
(1)
subject to: −yn + Y l ≥ 0, uxn − X l ≥ 0, l ≥ 0 where θ = the scalar measure of technical efficiency, λ = the N × 1 constants (weights) with non-negativity, yn = the M × 1 vector of outputs produced by the nth DMU, xn = the K × 1 vector of inputs used by the nth DMU, Y = the M × N matrix of outputs of N DMUs in the sample, and X = the K × N matrix of inputs of the N DMUs (Coelli et al. 2005). Although the above model based on the CRS assumption (Eq 1) is valid under the condition of the all DMUs’ operating at optimality, in the real world, this is not always possible. The above model was therefore advanced by Banker, Charnes, and Cooper (1984) to create a model that allows working under variable returns to scale (VRS) that presumes that a given increase in inputs will result in a disproportionate outputs. The linear programming equation of VRS model is written as: Minimizeu,l u
(2)
subject to: −yn + Y l ≥ 0, uxn − X l ≥ 0, N1′ l = 1, l ≥ 0 where N1 = the N × 1 vector of ones (Coelli et al. 2005). The efficiency value calculated through VRS model (Eq 2) gives the pure technical efficiency, without taking scale efficiency into account. Most water utility performance measurement studies have utilised the VRS model since the assumption of optimality presented by the CRS model may not work for water utilities (Ananda 2014; Marques, Berg, and Yane 2014; Guerrini et al. 2015; Molinos-Senante, Maziotis, and Sala-Garrido 2015). Therefore, in this study, the DEA with VRS model was performed with an R package ‘rDEA: Robust DEA for R’ Version 1.2-4 (Simm and Besstremyannaya 2015).
Input and output variables First, a comprehensive literature review was performed to identify the most commonly used input and output parameters for drinking water industry applications. It was found out that operating expenses, capital expenses, network length, number of employees, energy expenses, staff expenses, and material expenses were among the most widely used input variables. Operating revenue, the number of connections, the volume of water distributed, measures of water quality, population served were the mostly preferred output variables. Table 1 summarises the critical DEA studies.
4
G. GÜNGÖR-DEMIRCI ET AL.
Table 1. Critical DEA studies. Reference
Dataset
Inputs
101 Danish utilities in 2010 269 Indonesian Municipal Water Utilities
Production cost; Distribution cost; Customer handling cost Number of staff per 1000 connections, operation and management cost to 1000 connections, ratio of overhead cost to revenue
Ananda (2014)
53 Australian utilities from 2005 to 2011
Operating expenditure; Length of mains
Guerrini (2013)
64 Italian utilities
Storto (2013)
53 Italian utilities
Sum of amortisation, depreciation and interest paid; Staff costs; Other operating costs; Length of mains Network length; Total production cost
Thanassoulis (2000)
32 utilities in England and Wales in 1992/93 108 Japanese utilities in 1993
Guerrini et al. (2015) Wibowo and Alfen (2015)
Aida et al. (1998)
Operating expenditure
Number of employees; Operating expenses; Net plant and equipment value; Population; Length of pipes
Outputs Volume of water sold Accounted for water, coverage area, production efficiency, operating hours, billing effectiveness, return on equity, cash ratio, solvability, ratio of revenue to operating cost Total urban water supplied Number of water quality complaints
Population served; Total revenues
Total revenue
Number of connections; Volume of water delivered; Length of mains Operating revenues; Volume of water billed
Key findings Efficiency was not affected by size, scope of operation and customer density. The first and second stage DEA scores were found to be 55 and 32 percent, respectively. The size of the population served was the most important exogenous environmental variables. There was a significant relation between efficiency and the proportion of groundwater source, customer density and residential consumption. The results confirmed the existence of economies of scale, scope and density. Involvement of private operators may improve efficiency. Scale and agglomeration economies as well as the operator typology and its geographical location affected efficiency. The potential operating expenditure savings was found to be 26.7% amounting £144 M. Small scale water suppliers were more efficient than others.
Then, the availability of Cal Water data was determined through an extensive inspection. Of the most commonly used input-output variables, a complete set of data for eight variables, namely, operating expenses, network length, number of employees, energy expenses, operating revenue, number of connections and volume of water sold were found for the five years from January 2010 to December 2014 for 22 of the 24 Cal Water districts in California. The DEA results largely depend on the selected inputs and outputs, i.e. the discrimination power of the DEA model decreases with increasing numbers of inputs and outputs (Jenkins and Anderson 2003; Avkiran 2006). Therefore, selection of inputs and outputs is a critical in DEA. Studies have found out the relationship between the number of inputs-outputs and the minimum number of DMUs, which is to obtain a DEA model with a good discriminative power regarding the number of technically efficient
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
5
and inefficient units (Sarkis 2006; Khezrimotlagh 2015). According to Golany and Roll (1989), the minimum number of DMUs (N ) should be twice the total number of inputs (K ) and outputs (M ), while Bowlin (1998) argued for a higher value of three times the total number of inputs (K ) and outputs (M ). Dyson et al. (2001) took a slightly different approach, recommending instead that the number of DMUs (N ) should be at least twice the product of the numbers of inputs (K) and outputs (M ). Since the number of DMUs in current study is 22, which is relatively low compared to previous DEA studies conducted (Ananda 2014; Marques, Berg, and Yane 2014; Guerrini et al. 2015; MolinosSenante, Maziotis, and Sala-Garrido 2015), neither Bowlin’s (1998) nor Dyson et al.’s (2001) guidelines, both of which are more stringent than those suggested by Golany and Roll (1989), can be satisfied if all eight of the inputs and outputs mentioned above are used. It was therefore deemed necessary to eliminate the input and output variables and specify the efficiency frontier using a smaller number of variables. To select the most representative inputs and outputs, a few approaches have been suggested in the literature. These include applying expert knowledge and judgmental screening to identify the most relevant variables (Golany and Roll 1989), conducting correlational analyses to find highly correlated variables and eliminate those deemed unnecessary (Zschille and Walter 2012), or considering the effects of the exclusion or inclusion of individual variables on the estimation of efficiency scores (Wagner and Shimshak 2007). In this study, a stepwise procedure using a backward approach was used as the primary selection method (Wagner and Shimshak 2007). This first-order result was then verified using a correlational analysis (Zschille and Walter 2012). Before the application of DEA, all input and output variables were mean-normalized since any imbalance in the data magnitudes can lead to a variety of processing and output/reporting problems (i.e. overall software execution, algorithmic/numerical convergence, and round-off errors; Sarkis 2006; Ananda 2014).
Bias correction for efficiency scores by bootstrapping The efficiency scores obtained using conventional DEA do not allow for statistical inferences. However, they are subject to uncertainty due to the existing outliers (if any) as well as involved serial correlations (Ananda 2014; De Witte and Marques 2010). The deterministic nature of DEA leads to limitations in its potential application, but this drawback can be overcome by applying bootstrap techniques in which empirical distributions of efficiencies are derived. The bootstrapping procedure is based on resampling with replacement from a given sample and then calculating the statistic from a large number of repeated samples (Tziogkidis 2012; Mirzaei et al. 2015). First introduced by Efron (1979), it is a widely used and well-established technique for determining the accuracy of statistics under which the confidence intervals cannot be obtained analytically, or when an approximation based on the limit distribution is not satisfactory. It is also an appropriate way to control and check the stability of the results in a relatively simple manner (Efron and Tibshirani 1993; Davison and Hinkley 1997; Gocht and Balcombe 2006; Güngör-Demirci et al. 2016). In this study, the double bootstrap method was used for bias correction due to its consistency and robustness (Kneip, Simar, and Wilson 2003; Simar and Wilson 2007; Ananda 2014). Bias-corrected efficiency scores are regressed against exogenous variables, the
6
G. GÜNGÖR-DEMIRCI ET AL.
definitions, and details of which will be explained in the following section. Further details of the double bootstrap algorithm can be found in Simar and Wilson (2007) and other studies (Latruffe, Davidova, and Balcombe 2008; Ananda 2014). The total number of replications was 2000 for bootstrap operations.
Exogenous variables The exogenous variables are those variables that affect the technical efficiencies of water utilities but are not under managerial control while being location and utility specific (Renzetti and Dupont 2009; De Witte and Marques 2010; Marques, Berg, and Yane 2014). Although many different variables have been treated as exogenous variables in literature, based on the data availability and discussions with managers and engineers at Cal Water, five exogenous variables were chosen for this second stage of analysis: 1) number of connections, 2) customer density, 3) ratio of groundwater volume to total water production, 4) total number of leaks in the given year, and 5) total annual precipitation. Definitions, units and descriptive statistics for these variables (for 2014) are listed in Table 2. The number of connections (NOCON) was selected as an exogenous variable to reflect the effect of size. The effect of the number of connections is controversial in the literature, where it has been reported to have both negative and positive impact (Byrnes et al. 2010; Ananda 2014; Storto 2013). CUSDEN was chosen since previous studies have suggested that it has a positive impact on efficiency, at least up to a point, because utilities in densely populated areas can achieve lower operating and capital costs for the delivery of each volume unit (e.g. cubic metre) of water (Coelli and Walding 2006; Guerrini, Romano, and Campedelli 2013; Storto 2013; Ananda 2014; Marques, Berg, and Yane 2014). For a majority of Cal Water’s districts, groundwater is used as a water source, either alone, or in combination with the surface or purchased water. To further investigate the role of groundwater ratio, a variable GROUND was therefore included in the analysis. The effect of groundwater ratio is known to be positive as long as the energy prices remain unchanged (Byrnes et al. 2010; Zschille and Walter 2012; Ananda 2014). Total leak number (LEAK) was added into the analysis to see the effect of water losses. Although this variable seems under managerial control, repairing leaking networks is often done gradually and may take up to several years. Therefore, from a short-term perspective, preventing all leaks and water loss is not realistic. Although network losses are undesirable, its effect can be positive (Byrnes et al. 2010; Ananda 2014) while some others found it negative (Bhattacharyya et al. 1995) or not effective (Marques, Berg, and Yane 2014). Finally, to Table 2. Summary statistics of input, output and exogenous variables for 2014a. OPEX
ENERGY
OPREV
NOCON
CUSDEN
GROUND
LEAK
PRECIP
Mean 19,398 1036 22,563 18,559 46.1 0.56 28.3 408.4 SD 17,729 1567 21,037 17,533 10.2 0.39 42.8 274.7 Min. 2050 118 2339 1361 29.8 0.00 1.0 114.3 Max. 61,471 7080 75,167 69,083 77.3 1.00 187.0 1268.2 a OPEX = operating expenses excluding energy but including chemicals, payroll, cost of water purchased, taxes, depreciation and other operational, maintenance, administration and general expenses (×$1000), ENERGY = energy expenses (×$1000), OPREV = operating revenue (×$1000), NOCON = number of connections (including single and multiple family residential, commercial, industrial, government and other. Flat rate customers are also included where applicable), CUSDEN = Customer density (number of connections per km of network), GROUND = ratio of groundwater volume to total water production, LEAK = Total number of leaks in the given year, PRECIP = total annual precipitation (mm).
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
7
test the effect of climate, total annual precipitation (PRECIP) was added to the analysis. Renzetti and Dupont (2009) and Byrnes et al. (2010) reported on the negative effect of precipitation.
Regression model As the second stage of analysis, bias-corrected efficiency scores were regressed against the exogenous variables listed above using a double-bootstrap truncated regression (Simar and Wilson 2007), as well as a Tobit regression (Henningsen 2016). The regression equation utilised here is as follows: 1/u∗i = a + b1 NOCON + b2 CUSDEN + b3 GROUND + b4 LEAK + b5 PRECIP + 1i
(3)
where u∗i = a bias corrected efficiency estimate of DMUi, a = a constant term, βi = a vector of coefficients, and εi = the statistical noise (Ananda 2014; Wijesiri, Viganò, and Meoli 2015). Wijesiri, Viganò, and Meoli (2015) pointed out the limitations of the double bootstrap method: relying on several parametric assumptions like a linear model and truncated normal error terms. Therefore, in this study, both double bootstrap truncated and Tobit regression methods are used to examine the robustness of the results. The second stage analysis was conducted only for the year 2014 as it was the most recent year for which data was available. The exogenous variable values were mean-normalized before the analysis to receive the same methodological treatment as the input and output variables.
Results and discussion Stepwise selection of inputs and outputs The procedure began by including all possible (eight) inputs and outputs available in the Cal Water database. The DEA model was constructed for the 2014 data, i.e. the most recent year for which all data was available. Then, the variable that provided the smallest average difference in efficiency scores was dropped from the model at each step. Although this sequential procedure could theoretically continue until only one input and one output remained in the model, incorporating a stopping rule into the model was deemed more reasonable. According to Avkiran (2006), the number of efficient DMUs should not exceed one-third of the total number of DMUs. So, once the number of efficient DMUs in the DEA model, considered to be those providing the minimum average efficiency difference compared to the previous step, fell to 7 or less, the stepwise procedure was stopped. With this approach, the procedure stopped at the fifth step when the number of efficient DMUs decreased to 7. The final model at the fifth step included OPEX and ENERGY as inputs, and OPREV as the output. The resulting change in average efficiency between the initial model and the model at step 5 was 4%, which is comparable to that reported by Castro and Guccio (2014), who also used the stepwise procedure. According to Wagner and Shimshak (2007), moderate changes in efficiency scores are overridden by the gains of a more parsimonious model. Therefore, based on this selection methodology, OPEX and ENERGY were the inputs and OPREV was the output in this study. This selection
8
G. GÜNGÖR-DEMIRCI ET AL.
of variables was also supported by a subsequent correlation analysis, which found that correlation between OPEX and ENERGY was not significant, and there was a positive correlation between inputs (OPEX and ENERGY) and output (OPREV).
Stage I – initial DEA The DEA model described in Eq 2 was applied for 22 Cal Water districts for each of the five years from 2010 to 2014 for the first stage. For the most recent year, 2014, the initial DEA with input orientation and VRS assumption revealed seven (7) technically fully efficient districts, while the remaining (15) districts were considered technically inefficient. These seven (7) technically efficient districts were deemed to constitute the best practice frontier. The average efficiency of all 22 districts is relatively high, at 0.958 with a standard deviation of 0.045 (Table 3). This means that the districts can decrease its inputs by 4.2% (i.e. (1.00– 0.958) × 100) while holding its output constant (thus still enabling it to work as efficiently as its efficient peers). Among all the districts, D14 was the most inefficient district, with a score of 0.858 (Table 3). Figure 1 shows the chronological variations in efficiency scores for the 2010–2014 timeframe. Direct comparison among the years was not performed because efficiency scores were relative to the best practicing districts in each year (Byrnes et al. 2010). The average efficiencies were 93.2%, 93.6%, 95.7% and 96.7% for 2010, 2011, 2012 and 2013, respectively. Therefore, the high level of average performance (above 93%) had been consistent over the five years. Districts D1, D3, and D6 were the fully efficient (i.e. benchmark) districts for all five years (Figure 1). Eight of 22 districts (i.e. D10, D11, D12, D14, D15, D17, D20, and D22) had never been a benchmark district during this five-year period (Figure 1). Table 3. Ranking of districts depending on biased (θ) and bias-corrected (θ*) efficiency scores for 2014. 95% Confidence interval District
θ
Ranking
u∗i
Ranking
Lower bound
Upper bound
D10 D1 D5 D21 D6 D22 D7 D12 D9 D17 D18 D2 D11 D3 D8 D13 D16 D4 D20 D15 D19 D14 Mean
0.999 1.000 1.000 1.000 1.000 0.970 0.982 0.972 0.964 0.960 0.964 1.000 0.957 1.000 0.939 0.923 1.000 0.920 0.879 0.894 0.895 0.858 0.958
8 1 4 7 5 11 9 10 12 14 13 2 15 3 16 17 6 18 21 20 19 22
0.981 0.973 0.961 0.954 0.949 0.947 0.946 0.946 0.944 0.938 0.938 0.936 0.932 0.914 0.912 0.899 0.891 0.874 0.859 0.859 0.858 0.831 0.920
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
0.965 0.947 0.926 0.914 0.906 0.927 0.915 0.923 0.925 0.918 0.915 0.901 0.913 0.853 0.888 0.877 0.871 0.842 0.841 0.828 0.833 0.810
1.004 1.019 1.013 1.007 1.021 0.980 0.988 0.978 0.977 0.972 0.972 0.972 0.959 0.984 0.949 0.936 0.915 0.906 0.889 0.899 0.894 0.858
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
Figure 1. Technical efficiency scores for all districts between 2010 and 2014.
9
10
G. GÜNGÖR-DEMIRCI ET AL.
To deal with the uncertainty of the efficiency scores, bias-corrected efficiencies, as well as 95% confidence intervals, were calculated using the double bootstrap procedure for 2014 (Table 3). The point estimates of the efficiency scores calculated previously are given in the second column as biased efficiency scores (θ). The third column gives the ranking of each district based on θ. The fourth and the fifth columns present the bias-corrected efficiency scores (θ*) and the corresponding ranking of the district based on this efficiency score, respectively. The last two columns provide the lower and upper bounds of the θ* at the 95% confidence level. The efficiency scores decreased after bias-correction, which is consistent with the trends reported in previous studies (Gocht and Balcombe 2006; Ananda 2014; Wijesiri, Viganò, and Meoli 2015). The average efficiency decreased to 0.920, which can be translated as an 8% (0.080 = 1.000 − 0.920) input reduction requirement while holding output constant. The rankings of the districts (in terms of efficiency scores) also changed after bias correction, but generally, the rankings remained quite close when comparing the before and after bias-correction results except three districts (D2, D3, and D16). Beforehand, these three districts were ranked first due to their unitary biased efficiency scores, but dropped to 12th, 14th, and 17th, respectively, after bias correction procedure were applied. This is likely due to some degree of measurement noise in the initial DEA, as later evidenced by the bootstrap (Gocht and Balcombe 2006). As Ananda (2014) indicated, however, it is important to bear in mind that the relative comparison among districts is a sensitive matter and vigilant attention should be given to the interpretation of the rankings awarded due to the overlap between confidence intervals. For instance, D3, which was fully efficient in the initial DEA, became 91.4% efficient after bias correction and ranked 14th. This suggests that this utility could reduce its inputs 8.6% and still obtain the same output. However, the 95% confidence interval for this district shows that it could reduce its inputs by between 1.6% (given its upper bound of 0.984) and 14.7% (given its lower bound of 0.853). In overall, the input reduction required by the districts can change between zero and 19% (i.e. (1.00–0.810) × 100) based on the upper and lower bound values of the confidence interval of efficiency scores. Although there are some changes in both the efficiency scores and rankings of the districts, the difference between any two of the scores is relatively low, with an average of 0.038 and a maximum of 0.109. This indicates an average 3.8% decrease in the biased efficiency scores. As the data presented in Figure 2 shows, the biased efficiency scores are either inside the 95% confidence interval or very close to the upper bound (except for D16).
Stage II – exogenous variables As described earlier, exogenous variables are selected to fully account for the variations in the efficiency scores, with the goal being to explain factors that affect the district’s efficiency that are beyond the managers’ control. Five different exogenous variables were selected for this study: NOCON (number of connections), CUSDEN (customer density), GROUND (ratio of groundwater volume to total water production), LEAK (Total number of leaks in the given year) and PRECIP (the total annual precipitation in mm). The results of the double-bootstrap truncated regression, as well as the Tobit regression analysis, are summarised in Table 4. Both regression results produced very similar
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
Figure 2. Biased and bias-corrected efficiency scores and 95% confidence intervals for 2014.
11
12
G. GÜNGÖR-DEMIRCI ET AL.
Table 4. Double bootstrap truncated and Tobit regression results for 2014. Double bootstrap truncated regression
Tobit regression
95% Confidence Interval Variable Constant (a) NOCON CUSDEN GROUND LEAK PRECIP a
Coefficients (β)
Lower bound
Upper bound
Coefficients (β)
1.0014 0.0298b 0.0136 −0.0084 0.0006 0.0461a
0.8577 −0.0034 −0.0949 −0.0388 −0.0180 0.0128
1.1234 0.0631 0.1193 0.0225 0.0189 0.0816
1.0266 0.0241b 0.0052 −0.0062 0.0012 0.0383a
Significant at 1% level; bSignificant at 10% level.
outcomes. Note that since the dependent variable used in the regression computations is greater than or equal to 1 (see Eq 3), a positive sign for the estimated coefficient indicates a negative correlation, as an increase in a variable results in a decrease in the efficiency estimate (θ*) while a decrease in a variable leads to an increase in θ*. In contrast, a negative sign for the estimated coefficient is an indication of a positive correlation, because an increase or decrease in a variable again causes a commensurate increase or decrease in θ*. These results showed that CUSDEN, GROUND and LEAK had a statistically insignificant effect on efficiencies. The number of connections (NOCON) had a negative and statistically significant impact on efficiency, showing that an increase in the number of connections results in a decrease in efficiency. Related findings reported in the literature are mixed: although Ananda (2014) found that larger utilities serving more than 100,000 customers performed efficiently than their smaller counterparts, and Worthington (2011) stated that scale economies are optimised around 90,000 connections. Byrnes et al. (2010) found this effect to be negative, albeit up to a limit of 10,000 connections. The number of connections for each of the individual districts of Cal Water is all much lower than 90–100,000, so this may indicate the existence of a threshold value above which size may have a positive impact. Similar to NOCON, precipitation (PRECIP) had a negative and statistically significant impact. The negative correlation between efficiency and precipitation was expected since a reduction of water used (sales) often accompanies patterns of increased rainfall (wet periods).
Conclusions Drought conditions and population increase along with strengthening environmental standards makes water service provisions difficult. Furthermore, rising energy and other operating costs such as chemical, engineering and labour costs which have risen more than consumer prices makes the operations even harder. In the states, such as California, where severe drought conditions have been observed for many years, the increasing cost of drought management initiatives such as mandatory water restrictions and voluntary conservation campaigns brings extra pressure on utilities. Thus, more careful examination of the operational performances of water utilities becomes crucial and this study was designed to investigate the relative efficiencies of the individual service areas of Cal Water. This research contributes to the existing literature on performance measurement for water utilities by applying a cutting-edge two-stage DEA to a large (multi-district) California-based water utility. In this case, a double bootstrap method was applied to overcome the uncertainty in efficiency scores and also to reveal the effects of exogenous variables. Assessing the efficiency of all service areas can help identify the deficiencies in their
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
13
management practices, while also generating quantitative results to guide further operational and managerial improvements. The findings highlight the high overall performance achieved by the districts of Cal Water on average (92%). Some districts require certain levels of reduction in their operating and energy cost to be as efficient as their peers. In this study, the input reduction necessary across all districts can range between zero and nineteen percent (0%–19%). The number of connections and total annual precipitation were found to be statistically significant exogenous variables, and both were determined to have a negative impact on utility efficiencies. The findings of this study are expected to be useful in guiding subsequent managerial improvement initiatives and actions in order to provide better service to the customers. Especially, districts having the lowest efficiency scores should examine their current practices and reshape their policies to achieve simultaneous improvements. In this regard, the business plans and management strategies of the fully efficient districts can serve as effective models for the not fully efficient districts. Regarding future research, a more comprehensive evaluation of other water utilities in California as well as other states will be beneficial toward operational optimisation efforts. This will help identify the specific impacts/relationships of exogenous variables on the efficiency scores. Also, chronological changes of relative efficiency scores will be useful for in-depth understanding of water utilities’ performance (Charnes et al. 1984). Finally, but not the least, it is noted to interested researchers that DEA is a very data-intensive method, which require not only data analytical skills but also good communication with water utilities personnel to have the quality data that are of use to the analysis.
Disclosure statement No potential conflict of interest was reported by the authors.
Funding This paper is a significantly extended version of the conference paper originally presented at the American Society of Civil Engineer’s World Environmental and Water Resources Congress (West Palm Beach, Florida, May 22–26, 2016).
References Aida, K., W. Cooper, J. Pastor, and T. Sueyoshi. 1998. “Evaluating Water Supply Services in Japan with RAM: A Range-Adjusted Measure of Inefficiency.” Omega 26 (2): 207–232. Ananda, J. 2014. “Evaluating the Performance of Urban Water Utilities: Robust Nonparametric Approach.” Journal of Water Resources Planning and Management 140: 04014021. Avkiran, N. 2006. Productivity Analysis in the Service Sector with Data Envelopment Analysis. 3rd ed. Accessed November 28, 2016. http://www.users.on.net/~necmi/financesite/DEA%20Book% 203rd%20Edition%202006_AVKIRAN.pdf. Banker, R. D., A. Charnes, and W. W. Cooper. 1984. “Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis.” Management Science 30 (9): 1078–1092. Berg, S., and R. Marques. 2011. “Quantitative Studies of Water and Sanitation Utilities: A Benchmarking Literature Survey.” Water Policy 13 (5): 591. Bhattacharyya, A., R. Harris, R. Narayanan, and K. Raffie. 1995. “Technical Efficiency of Rural Water Utilities.” Journal of Agricultural and Resource Economics 20 (2): 373.
14
G. GÜNGÖR-DEMIRCI ET AL.
Bowlin, W. F. 1998. “Measuring Performance: An Introduction to Data Envelopment Analysis (DEA).” The Journal of Cost Analysis 15: 3–27. Byrnes, J., L. Crase, B. Dollery, and R. Villano. 2010. “The Relative Economic Efficiency of Urban Water Utilities in Regional New South Wales and Victoria.” Resource and Energy Economics 32 (3): 439–455. Castro, M. F., and C. Guccio. 2014. “Searching For the Source of Technical Inefficiency in Italian Judicial Districts: An Empirical Investigation.” European Journal of Law and Economics 38 (3): 369–391. Charnes, A., C. T. Clark, W. W. Cooper, and B. Golany. 1984. “A Developmental Study of Data Envelopment Analysis in Measuring the Efficiency of Maintenance Units in the U.S. air Forces.” Annals of Operations Research 2 (1): 95–112. Charnes, A., W. Cooper, and E. Rhodes. 1978. “Measuring the Efficiency of Decision Making Units.” European Journal of Operational Research 2 (6): 429–444. Coelli, T. J., D. S. P. Rao, C. J. O’Donnell, and G. E. Battese. 2005. An Introduction to Efficiency and Productivity Analysis. Springer-Verlag US: Springer Science & Business Media. Coelli, T., and S. Walding. 2006. “Performance Measurement in the Australian Water Supply Industry.” In Performance Measurement and Regulation of Network Utilities, edited by T. Coelli and D. A. Lawrence. Northhampton Massachusettes USA: Edward Elgar Publishing. Davison, A. C., and D. V. Hinkley. 1997. Bootstrap Methods and Their Application. NY: Cambridge University Press. De Witte, K., and R. Marques. 2010. “Incorporating Heterogeneity in Non-parametric Models: A Methodological Comparison.” International Journal of Operational Research 9 (2): 188. Dyson, R. G., R. Allen, A. S. Camanho, V. V. Podinovski, C. S. Sarrico, and E. A. Shale. 2001. “Pitfalls and Protocols in DEA.” European Journal of Operational Research 132: 245–259. Efron, B. 1979. “Bootstrap Methods: Another Look at the Jackknife.” The Annals of Statistics 7: 1–26. Efron, B., and R. Tibshirani. 1993. An Introduction to the Bootstrap. New York: Chapman & Hal. EUM Utility Leadership Group. 2017. Effective Utility Management A Primer for Water and Wastewater Utilities. January 2017. Gocht, A., and K. Balcombe. 2006. “Ranking Efficiency Units in DEA Using Bootstrapping an Applied Analysis for Slovenian Farm Data.” Agricultural Economics 35: 223–229. Golany, B., and Y. Roll. 1989. “An Application Procedure for DEA.” Omega 17: 237–250. Guerrini, A., G. Romano, and B. Campedelli. 2013. “Economies of Scale, Scope, and Density in the Italian Water Sector: A Two-Stage Data Envelopment Analysis Approach.” Water Resources Management 27 (13): 4559–4578. Guerrini, A., G. Romano, C. Leardini, and M. Martini. 2015. “The Effects of Operational and Environmental Variables on Efficiency of Danish Water and Wastewater Utilities.” Water 7 (7): 3263–3282. Güngör-Demirci, G., J. Lee, M. Mirzaei, and T. Younos. 2016. “How Do People Make a Decision on Bottled or Tap Water? Preference Elicitation with Nonparametric Bootstrap Simulations.” Water and Environment Journal 30 (3-4): 243–252. Henningsen, A. 2016. Package ‘censReg’, Version 0.5-22. Accessed December 1, 2016. https://cran.rproject.org/web/packages/censReg/index.html. Jenkins, L., and L. Anderson. 2003. “A Multivariate Statistical Approach to Reducing the Number of Variables in Data Envelopment Analysis.” European Journal of Operational Research 147: 51–61. Keck, J. C., and J. Lee. 2015. “A New Model for Industry - University Partnerships.” Journal - American Water Works Association 107 (11): 84–90. Khezrimotlagh, D. 2015. How to Deal with Numbers of Decision Making Units and Variables in Data Envelopment Analysis. arXiv preprint arXiv:1503.02306. Kneip, A., L. Simar, and P. W. Wilson. 2003. Asymptotics for DEA Estimators in Non-parametric Frontier Models. Technical Rep. No. 0323, IAP Statistics Network. Interuniversity Attraction Pole, Belgium. Latruffe, L., S. Davidova, and K. Balcombe. 2008. “Application of a Double Bootstrap to Investigation of Determinants of Technical Efficiency of Farms in Central Europe.” Journal of Productivity Analysis 29: 183–191.
CIVIL ENGINEERING AND ENVIRONMENTAL SYSTEMS
15
Marques, R. C., S. Berg, and S. Yane. 2014. “Nonparametric Benchmarking of Japanese Water Utilities: Institutional and Environmental Factors Affecting Efficiency.” Journal of Water Resources Planning and Management 140 (5): 562–571. Mirzaei, M., Y. F. Huang, A. El Shafie, T. Chimeh, J. Lee, N. Vaizadeh, and J. Adamowski. 2015. “Uncertainty Analysis for Extreme Flood Events in a Semi-Arid Region.” Natural Hazards 78: 1947–1960. Molinos-Senante, M., A. Maziotis, and R. Sala-Garrido. 2015. “Assessing the Relative Efficiency of Water Companies in the English and Welsh Water Industry: A Metafrontier Approach.” Environmental Science and Pollution Research 22 (21): 16987–16996. Norton, J., and W. Weber. 2009. “Water Utility Efficiency Assessment Using a Data Envelopment Analysis Procedure.” Journal of Infrastructure Systems 15 (2): 80. Renzetti, S., and D. Dupont. 2009. “Measuring the Technical Efficiency of Municipal Water Suppliers: The Role of Environmental Factors.” Land Economics 85 (4): 627–636. Sarkis. 2006. “Preparing Your Data for DEA – Information Technology.” In Productivity analysis in the service sector with data envelopment analysis, edited by N. Avkiran. Simar, L., and P. W. Wilson. 2007. “Estimation and Inference in Two-stage, Semi-parametric Models of Production Processes.” Journal of Econometrics 136: 31–64. Simm, J., and G. Besstremyannaya. 2015. Package ‘rDEA’, Version 1.2-4. Accessed June 24, 2016. https://cran.r-project.org/web/packages/rDEA/index.html. Storto, C. L. 2013. “Are Public-private Partnerships a Source of Greater Efficiency in Water Supply? Results of a Non-parametric Performance Analysis Relating to the Italian Industry.” Water 5: 2058–2079. Thanassoulis, E. 2000. “The Use of Data Envelopment Anlaysis in the Regulation of UK Water Utilities: Water Distribution.” European Journal of Operationsl Research 126 (2): 436–453. Tziogkidis, P. 2012. Bootstrap DEA and Hypothesis Testing. Cardiff Economics Working Papers E2012/ 18, Cardiff, UK. Wagner, J. M., and D. G. Shimshak. 2007. “Stepwise Selection of Variables in Data Envelopment Analysis: Procedures and Managerial Perspectives.” European Journal of Operational Research 180 (1): 57–67. Wibowo, A., and H. W. Alfen. 2015. “Benchmarking the Efficiencies of Indonesia’s Municipal Water Utilities Using Stackelberg Data Envelopment Analysis.” Benchmarking: An International Journal 22 (4): 588–609. Wijesiri, M., L. Viganò, and M. Meoli. 2015. “Efficiency of Microfinance Institutions in Sri Lanka: A Twostage Double Bootstrap DEA Approach.” Economic Modelling 47: 74–83. Worthington, A. 2011. Productivity, Efficiency and Technological Change in Australia’s Urban Water Utilities. Canberra: National Water Commission. Zschille, M., and M. Walter. 2012. “The Performance of German Water Utilities: A (Semi)-parametric Analysis.” Applied Economics 44 (29): 3749–3764.
