ICPR-AEM 2016 & QIEM 2016 - Proceedings

2016 International Conference on Production Research – Africa, Europe and the Middle East 4th International Conference on Quality and Innovation in Engineering and Management

EFFICIENCY ANALYSIS WITH DEA METHOD OF THE SUPPORT SERVICES IN UNIVERSITIES: A CASE STUDY G.V. Olariu, S. Brad Research Centre for Engineering and Management of Innovation, Technical University of Cluj-Napoca 400641 B-dul Muncii 103-105, Cluj-Napoca, Romania, [email protected], [email protected]

ABSTRACT: This paper introduces an empirical research to evaluate the relative efficiency of faculty secretariats from a higher education institution. The relative efficiency scores have been determined with data envelopment analysis (DEA) method. The model used is variable returns to scale (VRS), oriented towards maximizing the outputs. The case study was designed considering that customer-orientation (satisfaction of teachers and students) is the most important factor for efficiency analysis. Data were obtained through on-line questionnaires addressed to faculty secretariats, students and teachers. The average number of employees in each secretariat for the period 2012-2015 was taken as input. Results from the particular case study show that DEA application to assess relative efficiency between different units of a university is a reliable approach by providing quantitative indicators for comparative analysis. Keywords: DEA, efficiency, support services, analysis, teachers, students

efficiency in identifying the hotels with the best management practices geared toward innovation from ninety-five hotels located in the counties of Cluj and Brașov (Romania) is reported in paper [14]. Data have been collected by means of questionnaires. In this research, a set of five output variables (level of employment, financial results - incomes and expenses, fluctuation of personnel, employee satisfaction and customer satisfaction) and four inputs (human resource management practices, management practices related to information and communication technology, management practices concerning to the orientation towards quality, managerial practices and market competition) have been taken into account [14]. Evaluation of efficiency by combining DEA and Malmquist Productivity Index method in the framework of six companies from the engineering industry throughout a six-year period 2005-2010 is presented in [17]. International literature is richer in terms of DEA application for solving practical problems. For the field of higher education, some representative works of DEA consideration for efficiency assessment are further reported. By using DEA with sensitivity analysis, relative performance of academic departments, evaluating technical, pure and scale efficiencies of 19 departments at Indian Institute of Technology Roorkee, considering different combinations of inputs (academic staff, nonacademic staff and departmental operating cost) and output variables (total enrolled students, progress – combine two types of progress – number of students placed for different jobs and number of PhD degree awarded and research index, number of publications in journals and conference proceedings, number of research projects) is reported in paper [15]. Another example is the use of DEA to examine efficiency of 109 universities in England using six inputs (total number of undergraduate students, number of postgraduate students, number of full academic staff, total depreciation and interest payable in £, total expenditure on central libraries and information services, expenditure on central administration and central services excluding academic staff costs and depreciation in £) and three outputs (total number of graduates, total number of higher degrees awarded, value of the recurrent grant for research) [13]. Also, evaluation of research in higher education with DEA in 16 universities from China is reported in [3]. In this case, analysis used three inputs (index of labour-power sum of the teaching staff, financial strength index and physical index-teaching building, lab equipment, books, etc.) and two outputs (index number of graduates and scientific research, index of scientific and

1 INTRODUCTION The efficiency of support services in universities is important for getting high overall performance at institutional level. There are several possibilities to assess efficiency of a process, by calculating the normalized ratio between outputs and inputs. However, the same process can have different types of efficiency, depending the sets of inputs and outputs considered in the calculation; or in other formulation, depending on the goals towards efficiency is assessed. This is also the case of support services in universities. A very pragmatic approach to assess efficiency of a complex system is Data Envelopment Analysis Method (DEA) [4]. It has been successfully applied in various fields such as education, health and social assistance, hotels, public administration, etc. [1, 3, 10, 14, 16]. However, poor references exist in its application for support services, and even less within institutions of higher education. This paper illustrates the use of DEA method to evaluate the relative efficiency of the faculty secretariats in a higher education institution. The efficiency scores identify which secretariats are more efficient compared to their peers. In the second section of this article, basic references of DEA application in higher education is introduced. The third section highlights basic elements regarding DEA application. The fourth section sets out the work done for the analysis of nine faculty secretariats within the Technical University of Cluj-Napoca (TUCN), with inputs on data processing and calculation of secretariats’ efficiency by means of DEAP 2.1 software tool. Results are introduced in the fifth section of the paper, whereas the sixth section comprehends conclusions and possible future researches related to the increase of efficiency for such support services. 2 LITERATURE SURVEY In this section, basic references of DEA use in various application fields is introduced. A major focus is on the input and output variables that have been considered for assessing efficiency of various systems. Very few researches that used DEA are reported by Romanian researchers. Some representative results are further highlighted. In this respect, efficiency analysis with DEA method of the research, development and innovation (RDI) activities for twenty-eight European countries is reported by Roman and Suciu [16]. This work used a variable output (invention patents) and three inputs (costs of R&D, R&D personnel, hiring in technology and in the sectors that require extensive knowledge) [16]. Evaluation of 215

research development) [3]. Potential of DEA was also testing for evaluating performances of 19 Saudi Arabia universities using four inputs (budget for each university, total number of academic teaching staff, total number of non-academic teaching staff and number of colleges) and three outputs (total number of enrolled students, total number of new students, total number of graduates last year) [9]. Determining teaching and research efficiencies for 54 universities in the special case of sharing resources between two domains (physics and chemistry) is illustrated in [11]. Investigation considered three inputs (general expense, equipment expenditure, research income) and eight outputs (number of undergraduates, number of taught postgraduates, number of research postgraduates, research income-as a proxy for research output in terms of publication and/or citations if a department is rated outstanding at research, if a department is above average, below average and average at research) [11]. Measuring the research performance of Chinese higher education institutions using DEA for the case of research production of 109 universities in years 2003 and 2004 is presented in [12]. Here, six inputs (staff time, quality of the staff inputthe percentage of the faculty with associate professor position or higher, all students who are postgraduates, research funding, books-an index of library books and an index of the area of the buildings) and three outputs (an index of the prestige of the higher education institutions, an index of the total number of research publications to capture the total volume of research activity and an index of research publications per member of academic staff) were considered [12]. No specific application of DEA for the case of support services of the university was yet reported in the scientific literature. It is therefore a good opportunity to experiment DEA’s potential for this type of systems, too. In the next section, a brief description of DEA is considered. It will facilitate the understanding of the case study from the second half of this paper. 3

X = input matrix of size K×N for all DMU units decision Y = output matrix of size M×N for all DMU units decision. For each DMU we obtain a measure of the ratio of all outputs over all inputs, such as u' yi / v' xi , where u is an M×1 vector of output weights and v is a K×1 vector of input weights. max u, v (u' yi / v' xi) , (u' yj / v' xj) ≤ 1, j = 1, 2, ...., N,

st

(1)

u, v ≥ 0. The program is solved by getting those values for u and v so that we can maximize the efficiency unit i under the restriction that all efficiency measures must be less than or equal to 1. The program could has an infinite number of solutions. If (u*, v*) is a solution, then (αu*, αv*) is another solution, etc. To avoid this situation, one can impose the constraint v'xi = 1, and a new program is obtained: maxu, v (µ' yi) υ ' xi = 1

st

µ 'yj - υ 'xj ≤0, j= 0, 1, 2, ...., N,

(2)

µ, υ ≥ 0, where the notation changed from u and v to µ and υ reflects the transformation that took place in the new programme known as the multiplier form of the linear programming problems [5]. Using the duality in linear programming, it can derive a form of this problem, namely: minθλ θ st

-yi + Yλ ≥ 0 θxi - Xλ ≥ 0

(3)

λ ≥ 0, where θ is a scalar and λ represents a vector of constants by size N×1. The value of θ obtained will be the efficiency score for the ith DMU. It will satisfy θ ≤ 1, where a value of 1 represents a point on the frontier and indicates a technically efficient DMU, according to the Farell (1957) definition. The linear programming problem must be solved N times, for each DMU. A value of θ is then obtained for DMU [5].

DATA ENVELOPMENT ANALYSIS METHODOLOGY

DEA was developed by Charnes, Cooper and Rhodes (1978) and further improved by Banker, Charnes and Cooper (1984). The purpose of this section is to analyse the efficiency of some decision making units (DMU) through the use of inputs that produce one or more outputs [2, 4]. The models used for the measurement of efficiency can be with constant returns to scale (CRS) or with variable returns to scale (VRS), oriented towards minimizing the inputs or maximizing the outputs [4]. The efficiency scales are associated with each type of surface envelope caused by each DMU and this surface refers to the efficiency frontier [4]. Those elements who belong to this surface (or determine it) are considered efficient and those that do not belong to are considered inefficient. To make inefficient decision-making units becoming efficient, we can choose from the following variants: reducing inputs while outputs remain constant (input-oriented analysis); output increasing while inputs remain constant (output-oriented analysis); output increasing simultaneously with reducing input (the dual version). Input-oriented measurement equals output-oriented measurement of technical efficiency (TE) only in the case of constant returns to scales (CRS) [10]. 3.1 The Constant Returns to Scale Model (CRS)

3.2 The Variable Returns to Scale Model (VRS) Branker, Charnes and Cooper (1984) suggested an extension of the constant return to scale (DEA CRS) to account for variable return to scale (DEA VRS). The problem of the linear programming to explain CRS to VRS, adding the condition of convexity, N1´λ = 1 to (3): minθλ θ st

-yi+ Yλ ≥ 0 θxi -Xλ ≥ 0

(4)

N1´λ = 1 λ ≥ 0, where N1 is an N×1 vector of ones. If there are differences between technical efficiency determined with CRS (TECRS) and technical efficiency determined with VRS (TEVRS), then DMU has a lack of scale obtained by difference between TEVRS and TECRS. It results that CRS technical efficiency measure is discomposed into "pure" efficient technique VRS and scale efficiency, that means: TECRS = TEVRS × SE. The problem in measuring of scale efficiency is that it is not indicated whether the DMU is operating at increasing or the

We shall begin with some notation, namely: N = number of units of decision DMU K = inputs for each DMU units decision N M = outputs for each DMU units decision N xi = all inputs related to DMU i yi = all outputs related to DMU i 216


decreasing returns to scale. Therefore, it was added to DEA programs known problem that we have nonincreasing returns to scale (NIRS), which was replaced restriction on linear programming, namely N1´λ = 1 with N1´λ ≤ 1: minθλ θ st

The application of DEA model for the TUCN secretariats has involved the following steps. 4.1 Choice of orientation in efficiency analysis In DEA, the homogeneity of DMUs must satisfy three rules. First, the DMUs must have similar activities and the same objectives. Second, they should utilize similar inputs to produce the same outputs and, thirdly, they should operate within similar environments [8]. All secretariats of the university that we have chosen are homogeneous decision units because they use similar inputs and produce the same outputs.

-yi + Yλ ≥ 0 θxi - Xλ ≥ 0

(5)

N1´λ ≤ 1 λ ≥ 0.

In this study we have decided to use VRS model oriented to maximizing outputs and the analysis to be done with the program DEAP Version 2.1. In the DEAP software we chose the multi-stage DEA method, which is more computationally demanding that the other two methods (one stage DEA and two-stage DEA) [6].

Inefficiency for a particular DMU can be determined by comparing the value of the technical efficiency by NIRS with TEVRS. If they are not equal then the DMU is operating under IRS and they are equal then the DMU is operating under DRS. The output-oriented models are similar to the inputoriented models, discussed in sections 3.1 and 3.2. For example, a model oriented to VRS output with variable returns to scale (VRS DEA) is obtained by the program:

4.2 The choice of input and output variables Since we did not identify in the literature a case study on the analysis of the efficiency for university support services, we have designed a new one, considering that customer-orientation (teachers and students) is the most important factor for the efficiency analysis of the university secretariats. We have worked out some questionnaires on the management practices regarding the human resource and the technology of information and communication at the level of TUCN’s secretariats, as well as two models of questionnaires designed for the main target groups whose satisfaction shows the secretariats’ degree of efficiency. Therefore, the inputs taken into consideration refer to client-oriented managerial practices and the average number of employees in the respective secretariats for the period 2012-2015, namely:

maxϕ λ ϕ st

-ϕ yi +Yλ ≥ 0 xi - Xλ ≥ 0

(6)

N1´λ = 1 λ ≥ 0, where 1≤ ϕ ≤ ∞ and ϕ-1 represents proportional increase in outputs with input quantities held constant for each ith DMU [5]. Cook, Tone and Zhu (2014) submit certain aspects related to the use of DEA. These issues include the model orientation, input and output selection/deﬁnition, the use of mixed and raw data, and the number of inputs and outputs to use versus the number of decision making units (DMUs) [7]. DEA is more clearly defined if analysing a production process where the inputs and outputs are obviously defined by the resources used and needed for inputs and the results obtained for the outputs. In the case of a general comparative analysis of the issues, the inputs are usually the "less-the-better" type of performance measures and the outputs are usually the "more-thebetter" type of performance measures [7]. 4

1.

Input 1: management practices regarding the human resources and the technology of information and communication at the secretariat level, which consists of five variables that relate to human resources, and five variables that relate to the technology of information and communication.

2.

Input 2: the average number of employees within each secretariat. We took into account the average of employees in these services for a period of four years according to the data obtained from Rector’s reports in 2013, 2014 and 2015 on the web site of TUCN3.

In regards to outputs, we have taken into account the main factors which lead to the satisfaction of the two main categories of target groups of a university: teachers and students. Questionnaires designed for them included questions related to the assessment of these services from their point of view, namely: 1. Output 1: a questionnaire on students' satisfaction which includes four variables related to the appreciation of the secretariat staff (daily office hours and schedule; frequency of information delivery, promptitude in solving requests, kindness in relation with students). 2. Output 2: a questionnaire on teachers' satisfaction which includes two variables related to the assessment of the secretariat staff (daily office hours and schedule, frequency of information delivery).

RESEARCH DESIGN

The case study introduced in this paper is an empirical research with DEA for faculty secretariats of the Technical University of Cluj-Napoca (TUCN). TUCN is part of "the first value category in Romania, universities of advanced research and education"1. The structure of TUCN is made up of several support services, as it is presented in the institution's organizational structure 2 . The personnel of these services is divided into three main categories: auxiliary staff within the faculty (which provides practice with students, maintenance of equipment in laboratories, etc.); personnel supporting the educational process (secretariats, publishing house, library, workshops, various offices of counselling, recruiting candidates for admissions, etc.); auxiliary personnel (financial accounting, internal audit, human resources, legal advisors, maintenance workshops and social services).

The chosen outputs represent the result of managerial practices relating to secretarial services within a university. The number of inputs and outputs is less than

1http://www.utcluj.ro/universitatea/despre/mesaj-

conducere/rector/ 2 http://www.utcluj.ro/universitatea/structura/

3

publice/ 217

http://www.utcluj.ro/universitatea/despre/informatii-

the number of secretariats, which is a necessary condition for a proper DEA process. If the number of variables taken into account approaches the number of decision units, it is possible to obtain a too high number of efficient decision units, not reflecting reality, as specified in literature [1, 7]. 4.3

Note: CRSTE = technical efficiency from CRS DEA VRSTE = technical efficiency from VRS DEA SCALE= scale efficiency = CRSTE/VRSTE DRS = decreasing returns to scale IRS = increasing returns to scale NIRS = non-increasing returns to sale.

Collection of data and sample

The sample used in this study covers nine secretariats of the university. All the necessary data were obtained through questionnaires for the faculty secretariats, students and teachers. The average number of employees within each faculty secretariats for the period 2012-2015 was considered. The questionnaires were designed and distributed to the staff of the secretariats, as well as to the students and teachers from each of the faculties. Getting the data for the present study was achieved through Google forms. From secretariats we collected 31 responses, from students 154 responses and from teachers 178 responses. The mean values of responses have been finally considered. Results are illustrated in Table 1, on a scale from 1 to 10.

The secretariats that achieved coefficient scores equal to 1 were regarded as efficient ones (see column VRSTE in Table 2). The results showed that out of nine secretariats, four secretariats (6, 7, 8, 9: see the corresponding names to the numbers in Table 1) were identified as more efficient than the other five (1, 2, 3, 4, 5). The four secretariats in the top of the list reveal they are more efficient in utilizing their resources to produce the two investigated outputs: student’s satisfaction and teacher’s satisfaction. By far, the lowest relative efficiency has the secretariat of the Faculty of Electronics and Telecommunications (No. 4 in Table 1 and Table 2). The DMU 4 also has a relatively low CRSTE (0.652 in Table 2). The average efficiency score for CRS DEA and VRS DEA is 0.904 and 0.948. This means there are no large discrepancies between the efficiencies of the nine investigated DMUs. Three DMU were operating at IRS, four DMU at DRS and two at NIRS (see Table 2). For those DMUs operating under IRS, the hint is they can improve their efficiency if some things will be done more at a time than doing them one at a time. The DMUs operating under DRS highlight they are in a situation where making more things at a time is not possible because of various resource constrains. In those cases, supplementation of resources is necessary.

INPUT2

OUTPUT1

OUTPUT2

DMU 1. Secretariat, Automation and Computer Science 2. Secretariat, Civil Engineering 3. Secretariat, Machine Buildings 4. Secretariat, Electronics and Telecommunications 5. Secretariat, Electrical Engineering 6. Secretariat, Mechanical Engineering 7. Secretariat, Materials and Environmental Engineering 8. Secretariat, Building Services 9. Secretariat, Architecture and Urban Planning

INPUT1

Table 1 : Data collection

7.95

6.25

5.35

7.26

8.02

5.75

8.67

7.25

8.37

5.25

8.98

7.11

9.76

5.25

6.46

6.84

8.05

4.25

5.18

7.63

9.62

3.50

9.50

8.19

6.90

2.25

6.81

7.50

8.40

2.00

9.44

8.60

7.70

1.75

8.08

7.00

6 CONCLUSIONS The purpose of this paper was to investigate the application of DEA method for analysing the relative efficiency of faculty secretariats. In this investigation, DEA framework was designed towards maximizing outputs. Two inputs and two outputs have been defined in the model. The focus was on students and teachers satisfaction in relation to secretariats services. Data have been collected via on-line surveys in several iterations and university official reports. Interpretation of results shows that in terms of target group satisfaction only four secretariats are efficient and five are inefficient. The analysis of results also show that DMU 1, 4 and 5 could become more efficient by maximizing output 1 (students’ satisfaction) and DMU 2 and 3 by maximizing output 2 (teachers’ satisfaction). Experiences could be shared between secretariats to bring more efficiency where this is necessary. Analysis of efficiency with DEA in this case study was conditioned by the number of inputs and outputs considered. Their sum cannot overpass the size of DMUs. Normally, the best case is when the number of DMUs is twice higher than the sum of inputs and outputs in the DEA framework. This was respected for the current case study, but a deeper investigation will require reconsideration of other value analysis tools. A possible approach is cost/impact analysis, as well as the ratio costs of results/costs of constrains (capacity index). Beyond this aspect, the challenge occurs in the design and implementation of improvement projects. For this purpose, specific approaches are necessary. A possible line of research and investigation in this respect would be the lean six sigma methodology.

5 RESULTS AND DISCUSSIONS Data have been processed with DEAP v.2.1. Results are presented in Table 2. Table 2 below shows the technical efficiency from CRS DEA, the pure technical efficiency from VRS DEA and scale efficiency for the nine secretariats of the TUCN faculties. Analysis results are obtained by maximizing outputs. Table 2: Results from data analysis with DEAP v.2.1 DMU 1 2 3 4 5 6 7 8 9 mean

CRSTE 0.840 0.962 0.955 0.652 0.872 0.879 1.000 1.000 0.978 0.904

VRSTE 0.878 0.988 0.957 0.795 0.915 1.000 1.000 1.000 1.000 0.948

SCALE 0.957 0.973 0.998 0.820 0.954 0.879 1.000 1.000 0.978 0.951

DRS IRS IRS DRS DRS DRS NIRS NIRS IRS

7 ACKNOWLEDGMENTS We acknowledge all students, teachers and secretariat staff that kindly contributed to the provision of data in this empirical research. This work is part of the PhD study program of Mrs. Gabriela-Vica Olariu. 218


8

REFERENCES

1.

Avkiran, N.K., Investigating technical and scale efficiencies of Australian universities through data envelopment analysis, Socio-Economic Planning Sciences, Vol. 35, No. 1, pp. 57-80, (2001). Banker, R.D., Charnes, A., Cooper, W.W., Some models for estimating technical and scale inefficiencies in data envelopment analysis, Management Science, Vol. 30, No. 9, pp. 1078-1092, (1984). Chao, Z., Mingzhe, W., The evaluation research on higher education efficiency with data envelopment analysis (DEA), Management and Service Science, MASS '09, pp. 1-7, (2009). Charnes, A., Cooper, W., Rhodes, E., Measuring the efficiency of decision making units, European Journal of Operations Research, Vol. 2, pp. 429-444, (1978).

2.

3.

4.

5.

Coelli, T.J., A guide to DEAP V2.1. A data envelopment analysis (computer) program, CEPA Working Paper 96/08, Department of Econometrics, University of New England, Armidale, Australia, (1996). Retrieved from CEPA - Centre for Efficiency and Productivity Analysis: http://www.uq.edu.au /economics/cepa/deap.php

6.

Coelli, T.J., A multi-stage methodology for the solution of orientated DEA models, Journal Operations Research Letters, Vol. 23, No. 3-5, pp. 143-149, (1998). Cook, W.D., Tone, K., Zhu, J., Data envelopment analysis: prior to choosing model, Omega, Vol. 44, pp. 1-4, (2014).

7.

8.

Dyson, R.G., Pitfalls and protocols in DEA, European Journal of Operational Research, Vol. 132, No. 2, pp. 245-259, (2001).

9.

El-Razik, E.A., Data envelopment analysis a technique for measuring efficiency, British Journal of Mathematics & Computer Science, Vol. 5, No. 6, pp. 763-779, (2015).

Cibernetică Economică, Vol. 46, No. 1-2, pp. 5-18, (2012). 17. Tanase I., Morar L., Productive performance analysis in machinery industry using Malmquist index, Applied Mechanics and Materials (Appl Mech Mater), Vol. 245, pp. 220-226, (2013).

10. Fare, R., Lovell, C.A.K. Measuring the technical efficiency of production, Journal of Economic Theory, Vol. 19, pp. 150-162, (1978). 11. Beasley, J.E., Determining teaching and research efficiencies, Journal of the Operational Research Society, Vol. 46, pp. 441-452, (1995). 12. Jill, J., Li, Y., Measuring the research performance of Chinese higher education institutions using data envelopment analysis, China Economic Review, Vol. 19, pp. 679-696, (2008). 13. Johnes, J., Data envelopment analysis and its application to the measurement of efficiency in higher education, Economics of Education Review, Vol. 25, pp. 273-288, (2006). 14. Marin, A., Managementul diversificării serviciilor în unitățile de turism, Cluj, Summary PhD Thesis (in Romanian), Universitatea Babeș-Bolyai Cluj Napoca, (2012). 15. Tyagi, P., Shiv P.Y., Singh, S.P., Relative performance of academic departments using DEA with sensitivity analysis, Evaluation and Program Planning, Vol. 32, pp. 168-177, (2009). 16. Roman, M.M., Suciu, C.M., Analiza eficienței activității de cercetare dezvoltare inovare prin metoda DEA (in Romanian), Studii și Cercetări de Calcul Economic și 219