A QoS-based Service Acquisition Model for IS Services Xian Chen
Paul Sorenson
Department of Computing Science University of Alberta, Edmonton AB, Canada, T6G 2E8
Department of Computing Science University of Alberta, Edmonton AB, Canada, T6G 2E8
[email protected]
[email protected]
ABSTRACT
1. What is the most efficient way to produce an effective product and/or service?
The emergence of new information system (IS) service technologies yields opportunities and challenges in identifying and monitoring quality of services that providers are delivering. Unfortunately, very few approaches have applied accurate and unbiased views on service quality that takes into account the true value of the service provided. This paper poses the definition of quality of service for IS services (QoSIS) and examines how to discover and select high-quality IS services from the viewpoint of the service customers. This will be accomplished through the development of a QoSIS-based service acquisition model by introducing a quality assurance party (QAP) and two quality notions that are respectively referred for traditional customer/provider service and SaaS (software as a service).
2. What type of product and/or service attracts customers?
Categories and Subject Descriptors D.2.9 [Software Engineering]: Management—Software quality assurance (SQA); K.6.4 [Management of Computing and Information Systems]: System Management—Quality assurance
General Terms Management, Measurement
Keywords Service Acquisition Model, QoSIS
1.
INTRODUCTION
Service-oriented architecture (SOA) has now become one of the hotspots in information system (IS) development. However, the SOA solutions might not necessarily deliver high quality IS services. As a result, interest in the service quality management has increased accordingly. Service quality management originates from two basic questions on how to deliver services better [13]:
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WoSQ’08, May 10, 2008, Leipzig, Germany. Copyright 2008 ACM 978-1-60558-023-4/08/05 ...$5.00.
41
The goal of service quality management is to provide lower cost, better products and services, and higher customer satisfaction. If the service providers understand what customers want from a service, define detailed specifications based on the customer needs, manage the variables in the service delivery process that can lead to deviation from specifications, and deliver the service accurately, they are properly managing service quality. On the other hand, if the service customers are equipped with a service quality model that captures quantified and qualified attributes with respect to customer satisfaction and benefit, they can select and consume a service of high-level quality. The growth of Web service technologies raises various SOA solutions, such as UDDI registry and QoS/SLA specification languages. There are also a variety of information system (IS) service management frameworks, such as IT Infrastructure Library (ITIL)[11], Control Objectives for Information and related Technology (COBIT)[9], IT Service Capability Maturity Model (IT Service CMM)[10], Application Services Library (ASL)[14], and so on. These yield opportunities to explore the hierarchical approaches to measure and improve quality in IS services. Most of current quality management approaches have emphasized on quality of service (QoS) and service level agreement (SLA) specification. We believe that this view is very important, yet limited due to ignoring the voice of service customers. In the meantime, the relative maturity of the software business makes it possible for software vendors to deliver effective software applications as a service over Internet using the new delivery model called Software as a Service (SaaS). SaaS applications are generally priced on a per-user basis, and sometimes with additional fees for extra bandwidth and storage. Under SaaS, the service customer simply receives the benefits of the software, with clearly understandable and efficient costs, at a contractually defined service level [18]. While many successful commercial applications like Salesforce.com, Google Apps, have grown at great pace, quality management remains one of the biggest challenges for SaaS. Both service provider (the vendor) and customer require a new approach to manage service quality. The overall goal of our work is to propose and verify the utility of a systematic approach that allows service customers to assess more easily the services offered by a service provider. As the IS industry moves towards SaaS solu-
tions, the need for systematic assessment models will grow. This paper presents some of our initial work in meeting this goal. Specifically, we propose the initial elements of a reference model to evaluate QoS for IS services (QoSIS). The main contribution of this integrated QoSIS-based reference model is that it extends traditional conformance-based quality techniques by integrating three quality approaches to better serve the requirements of customers. The remainder of the paper is outlined as follows: Section 2 poses and analyzes the notion of QoSIS from four different perspectives. This will be used as basis of our service acquisition model in Section 3. We discuss the applications of the model in two cases: traditional customer/provider service and SaaS. Section 4 discusses related work. Finally, we conclude with future work of integrating quality evaluation approaches.
2.
(a) establishing a means of defining quality requirements; (b) correctly translating the requirements (or guarantees) into specifications/standards; and (c) monitoring, reporting and analyzing the performance against the specifications by measurement. At least two existing approaches can be applied to manage conformance quality: the first is to define a QoS specification language in which the quality requirements, quality capabilities and quality agreements in service level agreement (SLA) can be expressed. The language then supports QoS-based automated service discovery, measurement, monitoring, as well as negotiation. The second approach is to apply service level standards. Because service quality management is one of the main goals of service level management (SLM), the elements of QoS standards can be found in many IT management frameworks. In the SLM section of ITIL [11], an SLA skeleton is provided where it is convenient to establish and monitor metrics during SLM processes. Other examples can be found in CMMI [2], SPICE [8], IT Service CMM [10]. The measurement against specifications of service levels makes it possible to detect opportunities for quality improvement, but it does not provide guideline on how to perform quality improvement. On the other hand, the specifications may be easy to define for the IT product, while it may be more difficult or not be appropriate for the service support, especially when a high degree of human interaction is involved. Conformance quality facilitates measurement and assurance, but is not necessarily supportive of service improvement.
QUALITY DEFINITION FOR IS SERVICES
The definition of “quality” has been addressed and debated for a long time in a number of academic and industrial publications. David Garvin identified five major perspectives to the definition of quality [6], but it is difficult to separate product from service for modern-day information systems so the product-based view is not explicitly considered in our work. For QoSIS, we only consider the four perspectives of quality indicated in [7] [16]: • Conformance Quality is defined as conformance to specifications. Typically the focus is internal and on determining that performance matches original design specifications often expressed in SLAs.
2.2
• Gap Quality is defined as whether customer expectations are met or exceeded. This is the most pervasive definition of quality particularly as applied to business management. It is typically complex to define but can be related to quality improvement. • Value Quality is defined as direct benefit to the customer. It is a universal measure for widely different types of objects, and can be an appropriate guideline for continuous quality improvement. • Excellence Quality is defined as recognition of excellence. It stresses the features and characteristics of quality, but may change dramatically and rapidly. It is usually externally defined and hard to be related to quality improvement. To construct a better understanding of QoSIS, we analyze each perspective of quality, as defined above, with respect to quality measurement and improvement of IS services. It is noted that the general IS service may stand for both IS product and its service support. We then summarize the characteristics of the four types of quality.
2.1
Gap Quality
Gap quality is perceived and evaluated as the extent to which a service meets and/or exceeds the expectations of customers. This can be both one of the most complex and most accurate ways to define quality. Using a gap quality approach, it is possible to capture the most important parts of services for customers. Therefore, most marketing researchers have concentrated on this perspective. The Gaps Model of Service Quality [19] defines a framework to identify service quality in the form of five gaps that exceed or fail to meet customers’ expectations. The Gaps Model approach is supported by the SERVQUAL instrument [12] to measure the difference between perceived quality and expected quality, which is widely used in service management. One shortcoming of SERVQUAL instrument is that it requires customers to remember the expectation before their use of services. This is typically difficult to do given that, in many instances, customers might not even know their expectation. Some other models that do not use or emphasize expectations appear as good substitute instruments, such as SERVPERF (which only measures by perceived quality) [3] and American Customer Satisfaction Indices (ACSI) model (which provides an evaluation model to measure impact from perceived quality on customer satisfaction) [5]. Gap quality is evaluated from the customer’s perspective, and is responsive to marketing changes. In IS services, it can be measured on both product and service support. However, customers may make choices on a number of features and characteristics that it may not be difficult to improve.
Conformance Quality
Conformance quality is quantified as an objective measurement on performance from established standards. It is possible to measure the difference between conformance qualities of two or more services, as well as the difference between qualities of services for two or more periods. The tasks to manage conformance quality in IS services include:
2.3
Value Quality
Value quality is defined on customer benefits from the
42
service, with the required costs. This observation comes directly from the marketplace, meaning that the consumption of services is based on both price and perceived usefulness. In IS services, value quality allows one to compare explicitly disparate product and service support. The approaches including return on investment (ROI) analysis and risk analysis can help determine the aspects of quality improvement For traditional IS services, it is not easy to calculate the total cost of ownership (TCO) because many costs like product installation and configuration, training, potential maintenance fee are hidden and to service customers. In contrast, with SaaS the total costs are knowable in advance and can be explicitly defined in contract. The service customer can capture value quality with explicitly defined costs, such as annual fee. Both product and service support can be measured with value quality. It can be appropriately used in quality improvement, though the differentiation between constructs of quality and value, and unknown weight of individual components of value judgment need to be complemented by other approaches.
2.4
before and after the use of services, and reveals the opportunities to create positive qualities (perceptions exceeding expectations) or eliminate negative qualities (perceptions short of expectations). Value quality identifies the customer’s full benefits from the services leading to strategic partnership development. Realistically, service quality management requires the flexible application of QoSIS involving the support of all three concepts of quality. Based on QoSIS, we propose a service acquisition model to integrate approaches on the three quality notions for quality evaluation. The model supports the following functionalities: • Service registry enables the service providers to publish their products and services. • Quality assurance party (QAP), usually a third-party agent, identifies service quality and helps service customer with QoSIS-based service selection. • Unified interface enables service customers to define QoSIS requirements and service providers to define service offers.
Excellence Quality
From the perspective of excellence, quality means investment of the best skill and effort possible to produce the best possible results. In IS services, the excellence quality is marked by uncompromising standards and high performance, and can be used directly as promise or advertisement. Therefore, it focuses mostly on marketplace. Excellence quality offers little guidance to IT managers, because the definition of excellence is difficult to articulate and agree upon universally. Researchers also find it difficult to measure and compare the impact of excellence quality on the performance of product and service support. The concept of excellence in the view of marketing can be volatile and thus hard to measure for both product and service support, which makes systematic quality improvement impossible.
2.5
• QoSIS monitor with an integrated evaluation model is introduced to evaluate service quality and check the compliance of service offers against customer’s QoSIS requirements.
3.1
• Service Provider is a party that provides services to service customer. Service provider defines service offers with specifications of QoSIS level and related price. In the model, service provider also interacts with service registry and quality assurance party (QAP).
Quality Summary
• Service Customer is a party that purchases and consumes services from service provider. Service customer defines the QoSIS requirements and aims to receive higher satisfaction with the acquired services. In the model, service customer also interacts with QAP.
Both strengths and weaknesses of each notion of quality in IS services are summarized in Table 1. We also list their measurability, as well as the possible approaches that can be applied for measurement and improvement. By comparison, we conjecture that the first three notions have some direct relevance to quality in IS service delivery, and that gap quality and value quality can be used as part of a continuous improvement strategy for IS service delivery. We leave out excellence quality because it is immensely focused on marketing and no suitable approaches can be found to measure and apply it. The quality notions are used as bases for quality model in different types of IS services. The details are presented in the next section.
3.
Four Parties in the Model
Evolved from the relevant concepts in web service delivery, the following four parties (roles) are defined for an IS services selection model:
• Service Registry is a party that centralizes access and control of service metadata. It is the key infrastructural component for service deployments. The service registry for IS services is equivalent to UDDI registry for web services and interacts with service providers and QAP. • Quality Assurance Party (QAP) is a party that is requested to verify and validate the QoSIS against customer requirements. The QAP performs the key role in the model. It gathers and processes all necessary data for service acquisition, including customer requirements, service offers and QoSIS-related information. In practice, there might be two types of QAP. It can be either an individual department (e.g. service quality assurance (SQA) department) or an independent third-party agent that does the similar work.
A SERVICE ACQUISITION MODEL
Table 1 implies a possibility to manage quality measurement using a mixed strategy. From the view of customers, the three types of quality (except excellence quality) would represent quality requirements over a spectrum from essential to full value assessment in a strategic partnership arrangement. Conformance quality only guarantees what customers may require before the purchase and use of services. Gap quality indicates the difference between qualities
43
Table 1: Four Perspectives of QoS for IS services Definition Conformance Quality
Strengths - facilitates precise measurement and assurance
Weaknesses - does not provide guideline on how to perform quality improvement - not appropriate for service support - hard to improve
Measurability Measurable for product, but not for service support
Gap Quality
- accurate in marketplace, possible to capture the most important parts of services for customers - evaluated from the customers, and responsive to marketing changes - allows one to compare explicitly disparate product and service support - ROI analysis and risk analysis can really help determine the aspects of quality improvement. - can be used directly as promise and advertisement.
- the most complex way to define - customers may make choices by a number of features and characteristics that may not be improvable
Measurable for both product and service support
- the constructs of quality and value are different - the weight of individual components of value judgment is unknown
Measurable for both product and service support
ROI analysis, Risk analysis, ITIL (Service Level Management, Financial Management)
- hard to define ahead, measure and compare
Hard to measure both product and service support
No IS metrics-based approaches
Value Quality
Excellence Quality
In a simplified version of the model, service registry and QAP can be combined and represented by a third-party agent. In this section we only discuss the complete version of four parties. The interactions between the parties are as follows:
Approaches QoS/SLA specification languages (WSLA, WSAgreement, WSPolicy) Service level standards (SLM, CMMI, SPICE, IT Service CMM) SERVQUAL, SERVPERF, ACSI
7. Service selection: Service customer makes decision by comparison and selects one service from the list. 8. Service delivery: For traditional customer/provider services, an SLA is established between service provider and service customer. For SaaS, service customer purchases the selected service. Service provider then delivers the service to the customer.
1. Service publishing: Service providers advertise their services to the service registries. The registered information of each service includes unique identifiers (name, address, etc.) and commercial/functional categorizing parameters.
Figure 1 depicts the parties in the model, and sequence of the activities during service acquisition.
2. Service request: Service customers send requests to the QAP with certain commercial/functional parameters and the QoSIS requirements. 3. Service inquiry: The QAP inquires services according to categorizing parameters from the service registries. The service registries return a list of services with the identifiers that matches the categorizing parameters. In some cases, service offers can be pre-analyzed for verification. Therefore, the QAP is regularly holding information of service offers that customers could be interested in. If the information is up-to-date, this process will be optional. 4. Service verification: The QAP requests the service providers for service description and service offers. The service providers return their service offers. The QAP verifies service offers against customer requirements. When the QAP holds the up-to-date information on offers, it will verify the offers locally.
Figure 1: Activities during Service Acquisition. The following two subsections show how the model works with two typical services: traditional customer/provider service, and SaaS.
3.2
5. Service validation: The QAP checks and validates the offers against QoSIS in the requirements by applying measurement with the integrated quality approaches.
Conformance Quality Based Model
For traditional customer/provider services, service customers usually stress on conformance to the advertised servicelevel quality. For example, the SLA in outsourcing contract is mainly used to measure QoSIS. In this sense, both customer requirements and service offers are highly related to
6. Notification: The QAP notifies the service customer of a list of most appropriate services.
44
• SLA Negotiation
conformance quality used as basis for service selection and acquisition. The limitation of conformance quality with respect to ongoing assessment of user’s perspective of the service delivery can be compensated by introduction of gap quality. A survey approach [1] is designed to capture the customer view of gap quality. This allows QoSIS to be monitored over time and fosters activities of QoSIS improvement. The characteristics shown in Section 3 imply that it is difficult to conduct comprehensive value quality assessment for traditional customer/provider service agreements.
3.3
His paper emphasizes the selection of QoS specification language, but it only lists conventional quality-related characteristics and fails to explore the nature of QoS. Ran’s web services discovery model [15] extends the UDDI registry model and introduces a role called web service QoS certifier that certifies and verifies QoS information. In this model, a registry search would return web services that meet both required functionality as well as desired set of quality service indicators. An extended data structure is provided to support the model showing how quality information would link into the defined UDDI data structures. The model gives web service consumers confidence about QoS of the discovered web services, yet it requires extension of the UDDI standard and therefore its application is constrained. The WS-QoS framework developed in [17] introduces a specification on the QoS properties itself. Based on a WSQoS XML schema, the approach enables an efficient and dynamic QoS-aware selection and monitoring of web services. QoS issues related to web services are investigated and a prototype is then implemented with following functionalities:
Value Quality Based Model
Although the conformance quality based model may work well with traditional customer/provider service, it has some obvious shortcomings. The model focuses on conformance quality usually expressed in SLAs and ignores value quality due to lack of appropriate quality measurement instrumentation. In addition, the managed information of QoSIS that is collected is often incomplete and sometimes biased. For SaaS, service customers pay more attention to measurable and trackable value quality than in traditional service level agreements. We believe it is important to apply a strategic model that takes into account value quality of the service by using cost analysis approaches like ROI assessment and cost avoidance approaches like risk analysis. With SaaS the basis for service selection and acquisition is primarily value quality integrated, as appropriate, with conformance quality and gap quality monitoring and assessment of service delivery. All the activities for service acquisition in SaaS are webbased. The QAP is responsible for assessing and evaluating actual implemented versions of the service. Since costs and revenues of SaaS are usually periodically recurring, it is possible to turn the activities in the value quality based model into a process cycle which can be used for quality improvement. The applications of the service acquisition model with integrated quality approaches are shown in Figure 2.
• Application developers can state QoS properties associated with Web services by applying an API or using a graphic user interface (GUI) • Requirement manager for retrieving and updating client application QoS requirements • Web service broker for dynamic and efficient service selection • Monitor for checking the compliance of service offers • QoS proxies that map the QoS requirements from the higher layer onto the actual QoS-enabling transport technology at runtime. Unfortunately the paper does not provide much detail on mechanisms for compliance checking other than human observation.
5.
Figure 2: Applications of service acquisition model.
4.
• Adapt an SLA model and specification language approach to be incorporated in the model for capturing conformance quality concerns.
RELATED WORK
Dobson’s review [4] presents that the model on managing QoS should be able to support the following activities: • • • •
CONCLUSIONS AND FUTURE WORK
In this paper, we have discussed the QoSIS notions from four perspectives and developed a service acquisition model based on integrated service quality approaches. The model introduces the QAP to verify and validate QoSIS. Conformance quality and value quality are respectively used as model bases for traditional customer/provider service and SaaS. Our ongoing work is to build the service quality model for IS services by adapting quality evaluation approaches. Specific steps include:
• Develop a survey approach to gap quality measurement to analyze and determine how well an IS service is meeting customer perceptions and expectations. The preliminary design is described in [1] and further tool on the approach requires repeated pilot survey activities.
Definition of QoS/SLA Specification QoS-based Service Discovery QoS Measurement QoS Monitoring
45
[8] ISO/IEC-15504. Information Technology - Process Assessment. Technical report, International Organization for Standardization (Ed.), 2003. [9] IT Governance Institute. Cobit 4.1. ISACA, 2007. [10] F. Niessink, V. Clerca, T. Tijdinka, and H. van Vlietb. IT Service CMM Version 1.0 Release candidate 1, 2005. [11] Office of Government Commerce. Service Delivery, IT Infrastructure Library. The Stationery Office, 2001. [12] A. Parasuraman, V. A. Zeithaml, and L. L. Berry. [13] V. J. Peters. Total service quality management. Managing Service Quality, 29(1):6–12, 1999. [14] R. van der Pols. Application Services Library (ASL), a framework for Application Management. Van Haren Publishing, 2005. [15] S. Ran. A Model for Web Services Discovery with QoS. ACM SIGCOM Exchanges, 4(1):1–10, 2003. [16] C. A. Reeves and D. A. Bednar. Defining quality: alternatives and implications. Academy of Management Review, 19(3):419–445, 1994. [17] M. Tian, A. Gramm, H. Ritter, and J. Schiller. Efficient Selection and Monitoring of QoS-aware Web services with the WS-QoS Framework. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence (WI’04), Beijing, China, September 2004. [18] B. Waters. Software as a service: A look at the customer benefits. Journal of Digital Asset Management, 1(1):32–39, 2005. [19] V. A. Zeithaml, L. L. Berry, and A. Parasurama. Delivering quality service: Balancing customer perceptions and expectations. New York: The Free Press, 1990.
• Use ROI analysis and risk analysis as approaches for capturing value quality for investment made, especially for SaaS. The QAP provides tools helping determine best value. • Refine the design of the model by integrating the three quality approaches. A prototype tool will be built to support the model and implement part of approaches. The viability and impacts are analyzed and determined for sample IS services.
6.
REFERENCES
[1] X. Chen and P. Sorenson. Towards TQM in IT Services. In Workshop on Automating Service Quality (WRSAQ), pages 42–47, Atlanta, GA., November 2007. [2] CMMI. CMMI-SE/SW, V1.1 Capability Maturity R - Integrated for System Engineering/Software Model° Engineering, Version 1.1 Continuous Representation. Technical Report CMU/SEI-2002-TR-028, Software Engineering Institute, Pittsburgh, PA., 2002. [3] J. J. Cronin and S. A. Taylor. Measuring service quality: a re-examination and extension. Journal of Marketing, 56(3):55–68, 1992. [4] G. Dobson. Quality of service in Service-Oriented Architectures, 2004. [5] C. Fornell and M. D. Johnson. The American Customer Satisfaction Index: nature, purpose, and findings. Journal of Marketing, 60(4):7–18, 1996. [6] D. A. Garvin. What Does “Product Quality” Really Mean? Sloan Management Review, Fall:25–43, 1984. [7] P. Hernon and D. A. Nitecki. Service Quality: A Concept Not Fully Explored. Library Trends, 49(4):687–708, 2001.
46
