Government Online Services: Preliminary Results - Springer Link

Towards an Evaluation Model for the Quality of Local Government Online Services: Preliminary Results Filipe Sá1,4, Joaquim Gonçalves2, Álvaro Rocha3, Manuel Pérez Cota4 2

1 Câmara Municipal de Penacova, Penacova, Portugal Instituto Politécnico do Cávado e Ave, Escola Superior de Tecnologia, Barcelos, Portugal 3 Universidade de Coimbra, Departamento de Engenharia Informática, Coimbra, Portugal 4 Universidad de Vigo, Escuela Superior de Ingeniería Informática, Vigo, Spain [email protected], [email protected], [email protected], [email protected]

Abstract. This paper sets forth the results obtained after applying a first round of the Delphi Method process towards the development of an evaluation model for the quality of local government online services. In this first round, we submitted a list of thirty quality dimensions divided amongst four domains to a group of fifty local e-government experts. The results obtained led us to exclude one dimension and assign a set of twenty two dimensions to their respective domains. Additionally, the results reveal a certain degree of hesitation as to the domain assigned to seven dimensions. Three new dimensions emerged. Keywords: E-Government, E-Services, Service Quality, Public Services, Local Government

1

Introduction

With the emergence and the massive dissemination of the Internet, the provision of egovernment services became the norm all over the Globe. The Web constitutes a great source of information for its interactive character, ease of use and low cost [1]. The first step involves the definition of an e-government. According to Rocha, Silva, Lamas, Castro and Silva [2] an e-government, following an encompassing perspective, involves the adequate and advantageous use of information and communication technologies by government agencies - central, regional or local both in their internal and external relations, and particularly in their relationship with citizens. Santos, Melo and Dias [3] refer that the function of an Electronic Government is to restructure the organization of public services, adopting mechanisms that promote communication amongst different entities, thus simplifying processes. An e-government is more than a mere instrument providing cheaper and quicker services, it constitutes a mode of operation that ought to be implemented on central, regional and local levels in every country [4]. Official government websites

Ó Springer International Publishing Switzerland 2016 Á. Rocha et al. (eds.), New Advances in Information Systems and Technologies, Advances in Intelligent Systems and Computing 444, DOI 10.1007/978-3-319-31232-3_53

567

568

F. Sá et al.

offering information and services first appeared on the internet in the mid-nineties, followed by the emergence of the term "eGovernment" in society [5]. Although we could conceptually perceive the degree of citizen satisfaction obtained from interactions with the Central and the Local Government as being similar, from our perspective, we are talking about two different realities, as the proximity between the citizen and the local administration sphere, as well as the jurisdiction deriving from local legislation, cause the citizen to assume a different posture in the relationship he establishes with the governmental service provider. That is, the level of demand towards a local government is higher owing to its specificity and proximity. Although local e-government services may be perceived as an extension of central government services, the former have a number of specificities that need to be stressed. In this sense, in the local public services context, the Government provides electronic services that are heavily influenced by their powers, typology, different territorial and local distribution, disparate levels of literacy and age amongst populations, different organizational structures, a strong proximity with citizens and local companies, different IT maturity levels, disparate channels for the provision of services and, last but not least, their strong influencing role over local and national legislation policies. The definition of e-Government, consisting of "using information technologies to support government operations, involve citizens and provide public services", remains valid for the Local Government context [6]. Local e-governments form a relevant topic and a solution for the implementation of democracy on a local level, providing e-services and rendering access to available information easier [7]. As they enjoy a certain pseudo autonomy, it is not uncommon to observe significant differences in the provision of the same services by neighbouring municipalities or between municipalities of similar sizes. Local governments must be closer to people and companies, providing high quality services, and displaying foresight and leadership to local communities. Local Public Services are thus facing the challenge of administrative modernization, trying to bring the residents closer to their services and, simultaneously, dematerializing their processes [8, 9]. Accordingly, concepts, models, frameworks and methodologies need to be evaluated in order to measure, in the specific context of local authorities, the quality of Electronic Government services, with a view to improve the level of satisfaction attached to these services. Within the scope of the present study, the concept of local e-government concerns the provision of e-services through the internet, that is, online.

2

Methodology

This paper is part of an investigation process for the creation of an instrument to evaluate the quality of local government online services, and results from a first round of a Delphi Method process. In a previous stage we carried out a literature review, analysing a carefully selected sample of books, dissertations, theses and papers in this area, which allowed us to identify quality evaluation approaches for traditional, electronic and e-Government services, as well as an empirical study based on a series

Towards an Evaluation Model for the Quality of Local Government …

569

of interviews to Local Government experts [10-12]. Based on the collected and analysed elements, we prepared a preliminary list comprised by thirty quality dimensions, divided into four domains, which formed the basis for the first round of the Delphi Method process. The Delphi Method, originally developed by RAND Corporation, consists of a structured and interactive communication technique that adopts a series of questionnaires or rounds to collect and provide information, until a consensus is reached for a given theme [13-15,17]. In short, this method tries to identify and obtain expert views through a series of questionnaires, called rounds, until a maximum consensus is reached. Following the same line of thought, Santos [15] believes that this method is especially suited to reach consensus involving opinions, judgements or choices, namely to determine, predict and explore group attitudes, needs and priorities. This method stands out due to three fundamental characteristics: anonymity, interaction with a controlled feedback and the provision of statistics for group answers. For this first round of the Delphi Method process we invited, by email, fifty experts and obtained forty four answers. The members of the panel were chosen from a group of experts including IT technicians from local governments, researchers/academy members, local government software developers and experienced users. The questionnaire, including a list of domains and respective quality dimensions, was made available through the web, using a PHP and MySQL tool developed by the researchers, and asked the experts to score the relevance of each dimension in a five point Likert scale. The experts were also asked to attach a domain to each dimension. Additionally, the respondents could suggest new dimensions and domains. The criterion adopted to reject a dimension was based on the percentage agreement, coefficient of variation of the mean [1] and the percentile of the mean. This questionnaire was available from September 1 to September 30, 2015.

3

Previous Dimensions

In order to render the Delphi Method process easier, in this first round, and despite the fact that the experts were allowed to suggest new dimensions and domains, we adopted a predefined list of thirty dimension and four domains defined in the previous stages of this investigation [10-12]. This very list reflecting the perception of the authors, was developed in two stages. The first stage, as was previously mentioned, was based on a thorough literature review [11], in the course of which fourteen approaches were identified within the scope of Service Quality. From this group, nine are specifically connected to generic electronic services: SITEQUAL by Yoo and Donthu [18]; WebQual by Loiacono, Watson and Goodhue [19]; WebQual by Barnes and Vidgen [20]; eTailQ by Wolfinbarger and Gilly [21]; E-S-Qual by Parasuraman, Zeithaml and Malhotra [22]; e-TRANSQUAL by Bauer, Falk and Hammerschmidt [23]; the Review of the SERVQUAL scale by Li and Suomi [24]; the Framework of Sam and Tahir [25]; and e-SELFQUAL by Ding, Hu and Sheng [26]. The remaining five approaches concern the e-government context: E-GovQual by Papadomichelaki and Mentzas [27]; e-

570

F. Sá et al.

GOSQ Framework by Agrawal et al. [16]; the Proposal by Alanezi, Kamil and Basri [28]; e-GSQA Framework by Zaidi and Qteishat [29]; and the Framework of Hien [30]. The second stage [10], which served as a complement to the literature review, involved the elaboration of twenty-five interviews, carried out in the form of a questionnaire administered to experts and/or users of online e-services involving the local public sphere, with a view to prospect and identify new dimensions and domains.

4

Application of the Delphi Method

The first step to set the Delphi Method process in motion involved the development of a tool to administer the questionnaires. To favour the design, and bearing in mind usability, we developed a questionnaire from scratch (Figure 1), programming in HTML, PHP, Javascript and MySQL. A panel of experts was selected in order to initiate the process. Bearing in mind the intended profile and resorting to a convenience sample, we invited fifty prospective participants, including IT department responsible figures from local governments, software suppliers, university professors and regular e-service users. Despite the adoption of a convenience sample, we invited experts from different geographical areas. The invitation was sent by email, which included the password and the link to access the questionnaire. First and foremost, the respondents were given access to the statements explaining the purpose of the study and the instructions to fill the questionnaire. In this first round, we presented a list of thirty quality dimensions asking the participants to select those that should be included in an evaluation model for the quality of local government online services. For each chosen dimension the experts were also asked to choose a respective domain. The experts were also asked to score, in a five point Likert scale, the level of relevance attached to each dimension within a domain. We obtained forty four answers, from twenty three IT responsible figures/technicians from local governments, nine service users, seven university professors, three politicians responsible for the modernization of local public services and two collaborators from software development companies. The retention of dimensions for the next round was based on the compliance of at least two of the following requirements: • Agreement above 50% • Mean above the percentile 5 • Coefficient of variation below 33%


571

Fig. 1. Online questionnaire (http://www.cm-penacova.pt/egovernment)

5

Results from the First Round of the Delphi Method

The size of the sample (N=44>30) allowed us to treat the selected ordinal qualitative scale of scores assigned to each dimension as a quantitative scale, without losing accuracy and in order to estimate the adopted statistical indicators. 5.1 Relevance of the dimensions We estimated the mean, the standard deviation and the coefficient of variation of the scores assigned to each dimension, based on the scores assigned by the experts who recognized their relevance. We also determined, for each dimension, the percentage of experts that recognized its relevance (Table 1).

572

F. Sá et al.

Table 1. Mean and standard deviation of the assigned scores for each dimension Mean

Standard deviation

Safety

4,7

0,599

Coefficient of variation 12,7%

Transparency

4,56

0,666

14,6%

97,7%

Privacy

4,55

0,593

13,0%

95,5%

Deadline Compliance

4,55

0,663

14,6%

100,0%

Information Quality

4,45

0,663

14,9%

100,0%

Usability

4,43

0,63

14,2%

95,5%

Reliability

4,38

0,774

17,7%

90,9%

Service Availability

4,34

0,745

17,2%

100,0%

Complaints

4,3

0,773

18,0%

97,7%

Customer Support

4,3

0,674

15,7%

97,7%

Task Information

4,28

0,724

16,9%

88,6%

Online Services Advantage

4,26

0,724

17,0%

86,4%

Accessibility

4,25

0,781

18,4%

100,0%

Process Management

4,25

0,806

19,0%

81,8%

Agreement 97,7%

Compatibility

4,1

0,8

19,5%

93,2%

Interoperability

4,08

0,818

20,0%

86,4%

Empowerment

4,03

0,811

20,1%

88,6%

E-Participation

4

0,751

18,8%

90,9%

Technical Website Quality

3,95

0,749

19,0%

90,9%

Website Access Speed

3,91

0,895

22,9%

97,7%

Variety of Services Offered

3,89

0,831

21,4%

86,4%

Processing Speed

3,88

0,9

23,2%

93,2%

Online Integrity

3,81

1,023

26,9%

84,1%

Customization

3,8

1,08

28,4%

56,8%

Alternative Channels

3,78

1,031

27,3%

84,1%

E-Democracy

3,76

0,796

21,2%

84,1%

Politicians Role

3,68

0,945

25,7%

63,6%

Website Innovation

3,66

1,004

27,4%

72,7%

Website Design

3,34

1,056

31,6%

79,5%

Emotional Appeal

3,18

1,131

35,6%

38,6%

Only 38,6% of the experts recognized relevance in the “Emotional Appeal” dimension. Moreover, the obtained mean is below the percentile 5 (3,268) when


573

compared to all the other means, while revealing a coefficient of variation above 33%. Accordingly, this dimension does not comply with any of the requirements set forth for its retention, being thus removed from the second round of the process. The remaining dimensions comply with the predefined requirements, and they will be used in the next round. 5.2 Assignment of domains to the dimensions We analysed the frequency of domain assignment to each dimension by the experts (Table 2). Table 2. Frequency of domain assignment to each dimensions Management

Technical

Service

Information

Emotional Appeal

58,8%

11,8%

23,5%

5,9%

Customer Support

39,5%

9,3%

34,9%

16,3%

E-Democracy

73,0%

0,0%

18,9%

8,1%

E-Participation

65,0%

2,5%

25,0%

7,5%

Politicians Role

78,6%

0,0%

7,1%

14,3%

Complaints

39,5%

11,6%

39,5%

9,3%

Transparency

53,5%

0,0%

9,3%

37,2%


48,6%

16,2%

18,9%

16,2%

Empowerment

46,2%

25,6%

23,1%

5,1%

Process Management

75,0%

2,8%

16,7%

5,6%

Online Services Advantage

44,7%

13,2%

18,4%

23,7%

Information Quality

4,5%

0,0%

6,8%

88,6%

Task Information

5,1%

5,1%

0,0%

89,7%

Deadline Compliance

6,8%

2,3%

90,9%

0,0%


0,0%

11,4%

88,6%

0,0%

Online Integrity

2,7%

8,1%

83,8%

5,4%

Customization

4,0%

4,0%

92,0%

0,0%

Privacy

7,1%

7,1%

73,8%

11,9%

Safety

2,3%

20,9%

67,4%

9,3%

Processing Speed

2,4%

14,6%

80,5%

2,4%

Interoperability

7,9%

13,2%

78,9%

0,0%


5,3%

0,0%

86,8%

7,9%

Reliability

2,5%

5,0%

90,0%

2,5%

Website Innovation

0,0%

93,8%

0,0%

6,3%


0,0%

92,5%

0,0%

7,5%

Usability

0,0%

95,2%

2,4%

2,4%

574

F. Sá et al.

Management

Technical

Service

Information


2,3%

90,7%

4,7%

2,3%

Website Design

0,0%

100,0%

0,0%

0,0%

Accessibility

0,0%

86,4%

11,4%

2,3%

Compatibility

0,0%

97,6%

2,4%

0,0%

The shading indicates the highest assignment frequency value for each dimension. The results reveal that the “Management” domain displays lower agreement values. The “Complaints” dimension divides the experts between the “Management” and the “Service” domains. In order to determine the consistency of each assignment we estimated the percentile 75 for the frequency of each domain (Table 3). Table 3. Consistency of each assignment of each domain

Percentile 75

Management

Technical

Services

Information

46,8%

41,0%

79,3%

12,5%

The shadings in Table 2, namely in Customer Support, Complaints, Empowerment, Online Services Advantage, Privacy and Safety, indicate those whose frequency of assignment values fall out of the percentile 75, revealing lower consistency. It is important to note that four of the eleven dimensions of the Management domain and four out of eleven in the Services domain fall below percentile 75 (the dimension Complaints appears in both domains below percentile 75), while all the dimensions highlighted in the “Technical” and “Information” domains are above percentile 75. However, it must be noted that percentile 75 displays a significantly higher frequency value (79,3%) in the Services domain, indicating that the dimensions in this domain have greater consistency. The scores obtained for each dimension considered the quartile of the average score for each dimension. Dimensions in the 4th quartile were assigned four points, while the ones in the 3rd quartile were assigned three points. The 2nd quartile received two points and, lastly, the 1st quartile was assigned only one point. As to the “Information” domain, only the median was considered, as only two domains existed. In this case, the most scored dimension was assigned two points and the other one point. From the application of this method, the dimension “Emotional Appeal” was excluded, and a set of twenty two dimensions was obtained. Additionally, the results reveal a certain degree of hesitation as to the domain assigned to seven dimensions. For each dimension we also defined a weight within the assigned domain. Table 4 shows these results. The scores displayed for the dimensions without a defined domain reflect their weight when placed in either domain. It is worthy of note that three experts allowed us to identify three new dimensions: • “Utility” in the existing Service domain; • “Settlement” in the existing Service domain;


575

• “ Decision Democratization” in a new “Democratization” domain. Table 4. Weight by dimension Dimension

Domain

Weight

Transparency

Management

4

Process Management

Management

4

E-Participation

Management

3

E-Democracy

Management

2

Politicians Role

Management

1

Deadline Compliance

Service

4

Reliability

Service

4


Service

3

Interoperability

Service

3


Service

2

Processing Speed

Service

2

Online Integrity

Service

1

Customization

Service

1

Information Quality

Information

2

Task Information

Information

1

Usability

Technical

4

Accessibility

Technical

4

Compatibility

Technical

3


Technical

3


Technical

2

Website Innovation

Technical

1

Website Design

Technical

1

Service|Information

4|4

Privacy

Service|Information

4|4

Complaints

Service|Management

3|4

Customer Support

Service|Management

3|4


Service|Management

1|2

Safety

Online Services Advantage Capacitation

Management|Information

4|1

Management|Technical

3|2

In the second round of the method we will ascertain the domain to which these dimensions should be assigned and adjust all the weights.

576

6

F. Sá et al.

Conclusion

This paper sets forth the results from the first round of the Delphi Method process applied to the evaluation of local government e-services approach under development. The Delphi Method process has, to the present moment, proved to be effective and allowed us in this first round, and with a significant level of consensus, to present a list of dimensions assigned to their respective domains, based on relevance ranking. This round has also shown that the preliminary list, which served as a basis for this investigation process, is aligned with our initial purpose, to the extent that from all the identified dimensions only one, Emotional Appeal, was discarded. Doubts were raised as to the domain and the relevance assigned to Safety, Privacy, Complaints, Customer Support, Alternative Channels, Online Services Advantages and Empowerment. The remaining twenty three dimensions already reveal a high level of consensus, which, notwithstanding, must be validated in the next round of the Delphi Method process. It is worthy of note that three of the experts allowed us to identify three new dimensions. As a result, we need to resume the Delphi Method process until a consensus is reached as to the relevance of the dimensions and their assignment to their respective domains, creating metrics and weights. Subsequently, we will define a method to evaluate the quality of local government online services.

Acknowledgments. We acknowledge the financial support of AISTI (Iberian Association for Information Systems and Technologies), which permitted the registration in the WorldCIST'16 (4th World Conference on Information Systems and Technologies), held at Recife, Brazil, 22 - 24 March 2016, and consequently this publication.

References 1.

2.

3.

4. 5.

Leite, P., Gonçalves, J., Teixeira, P., & Rocha, Á. (2014). Towards a model for the measurement of data quality in websites. New Review of Hypermedia and Multimedia, 20(4), 301-316. Rocha, A., Silva, C., Lamas, M., Castro, R., & Silva, S. (2005). Governo Electrónico nas Juntas de Freguesia: Situação na Região do Minho. In Actas da 6ª Conferência da Associação Portuguesa de Sistemas de Informação. Bragança, Portugal. Santos, P., Melo, A. I., & Paiva Dias, G. (2013, June). Administrative modernisation and e-government: The case of Águeda. In Information Systems and Technologies (CISTI), 2013 8th Iberian Conference on (pp. 1-6). IEEE. Jukić, T., & Vintar, M. (2006). E-government: The state in Slovenian local selfgovernment. Organizacija, 39(3). Jinmei, H. (2011, August). Quality evaluation of e-government public service. In Management and Service Science (MASS), 2011 International Conference on (pp. 1-4). IEEE.


6.

7. 8. 9.

10. 11.

12.

13.

14. 15.

16. 17.

18.

19. 20. 21. 22.

23.

24.

577

Paiva Dias, G., & Gomes, H. (2014, June). Evolution of local e-government maturity in Portugal. In Information Systems and Technologies (CISTI), 2014 9th Iberian Conference on (pp. 1-5). IEEE. Kunstelj, M., & Decman, M. (2005). Current state of e-government in Slovenian municipalities. Electronic Journal of e-Government, 3(3), 117-128. Rocha, Á., & Sá, F. (2013). Planning the information architecture in a local public administration organization. Information Development, 0266666913489841. Sá, F., & Rocha, Á. (2012). Definição da arquitetura de informação em organismo da administração pública local. RISTI-Revista Ibérica de Sistemas e Tecnologias de Informação, (10), 51-64. Sá, F., Rocha, Á., & Cota, M. P. (2016). Potential dimensions for a local eGovernment services quality model. Telematics and Informatics, 33(2), 270-276. Sá, F., Rocha, Á., & Cota, M. P. (2015). From the quality of traditional services to the quality of local e-Government online services: A literature review.Government Information Quarterly. Sa, F., Rocha, A., & Pérez Cota, M. (2015, June). Preliminary dimensions for a quality model of Electronic Local Government services. In Information Systems and Technologies (CISTI), 2015 10th Iberian Conference on (pp. 1-10). IEEE. Keeney, S., Hasson, F., & McKenna, H. P. (2001). A critical review of the Delphi technique as a research methodology for nursing. International journal of nursing studies, 38(2), 195-200. Linstone, H. A., & Turoff, M. (Eds.). (1975). The Delphi method: Techniques and applications (Vol. 29). Reading, MA: Addison-Wesley. Santos, L.D.d., Factores Determinantes do Sucesso de Serviços de Informação Online em Sistemas de Gestão de Ciência e Tecnologia, in Escola de Engenharia. 2004, Universidade do Minho. p. 259. Agrawal, A., Shah, P., & Wadhwa, V. (2009). EGOSQ-Users’ Assessment of egovernance Online-Services. CSI India. Day, J., & Bobeva, M. (2005). A generic toolkit for the successful management of Delphi studies. The Electronic Journal of Business Research Methodology,3(2), 103116. Yoo, B., & Donthu, N. (2001). Developing a scale to measure the perceived quality of an Internet shopping site (SITEQUAL). Quarterly Journal of Electronic Commerce, 2(1), 31-47. Loiacono, E. T., Watson, R. T., & Goodhue, D. L. (2002). WebQual: A measure of website quality. Marketing theory and applications, 13(3), 432-438. Barnes, S. J., & Vidgen, R. T. (2002). An Integrative Approach to the Assessment of E-Commerce Quality. J. Electron. Commerce Res., 3(3), 114-127. Wolfinbarger, M., & Gilly, M. C. (2003). eTailQ: dimensionalizing, measuring and predicting etail quality. Journal of retailing, 79(3), 183-198. Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). ES-QUAL a multiple-item scale for assessing electronic service quality. Journal of service research,7(3), 213233. Bauer, H. H., Falk, T., & Hammerschmidt, M. (2006). eTransQual: A transaction process-based approach for capturing service quality in online shopping. Journal of Business Research, 59(7), 866-875. Li, H., & Suomi, R. (2009). A proposed scale for measuring e-service quality.International Journal of u-and e-Service, Science and Technology, 2(1), 1-10.

578

F. Sá et al.

25. Sam, M., Fazli, M., & Tahir, M. N. H. (2009). Website quality and consumer online purchase intention of air ticket. International Journal of Basic & Applied Sciences, 9(10). 26. Ding, D. X., Hu, P. J. H., & Sheng, O. R. L. (2011). e-SELFQUAL: A scale for measuring online self-service quality. Journal of Business Research, 64(5), 508-515. 27. Papadomichelaki, X., & Mentzas, G. (2009). A multiple-item scale for assessing egovernment service quality. In Electronic Government (pp. 163-175). Springer Berlin Heidelberg. 28. Alanezi, M. A., Kamil, A., & Basri, S. (2010). A proposed instrument dimensions for measuring e-government service quality. International Journal of u-and eService, 3(4), 1-18. 29. Zaidi, S. F. H., & Qteishat, M. K. (2012). Assessing e-Government Service Delivery (Government to Citizen). International Journal of eBussiness and eGovernment Studies, 4(1), 45-54. 30. Hien, N. M. (2014). A Study on Evaluation of E-Government Service Quality. International Journal of Social, Management, Economics and Business Engineering. 8(1).