provides new opportunities like instant service provisioning and a pay-as-you-go .... The Platform Cloud prototype features an AppStore that offers managed .... Table I: Negotiation Domain, Constraints and Preference Profiles. Negotiation ...
Optimal Negotiation of Service Level Agreements for Cloud-based Services through Autonomous Agents Edwin Yaqub∗ , Ramin Yahyapour∗ , Philipp Wieder∗ , Constantinos Kotsokalis, Kuan Lu∗ and Ali Imran Jehangiri∗ ∗ Gesellschaft f¨ ur wissenschaftliche Datenverarbeitung mbH G¨ottingen (GWDG) G¨ottingen, Germany Email: {edwin.yaqub, ramin.yahyapour, philipp.wieder, kuan.lu, ali.jehangiri}@gwdg.de Email: {constantinos.kotsokalis}@tu-dortmund.de
Abstract—Cloud-based services have become a cornerstone of today’s IT. The self-service feature inherent in Cloud systems allows customers to play a greater role in service procurement. However, this restricts the value propositions and Service Level Agreements (SLAs) that Cloud providers offer because Quality of Service (QoS) and Non Functional Property (NFP) requirements vary from customer to customer. In featurerich SLA templates, the contract space gets large, objectives are confidential and preferences over QoS and NFP often conflict between providers and customers. Hence, an SLA-gap exists between the two and contemporary providers bind their offerings to the inflexible take-it-or-leave-it SLAs. In this work, we address this problem by presenting a robust and computationally inexpensive negotiation strategy, using which agents can efficiently create near-optimal SLAs under time constraints. Experimental evaluations validate that our strategy performs at par with state of the art learning and non-learning strategies against a variety of metrics including utility, social welfare, social utility and the Pareto-optimal bids. This enables a dynamic SLA negotiation mechanism on top of our OpenShift (PaaS) based Cloud system designed using Service Oriented Cloud Computing Infrastructure (SOCCI) architecture. Negotiated procurement of services is shown to improve satisfaction of participants and reducing the SLA-gap. Keywords-Service Level Agreement (SLA); SLA Negotiations; Agents; Cloud Computing; SaaS; PaaS; IaaS
I. I NTRODUCTION The motivation of this work lies in the fact that customer needs in terms of service QoS and NFP issues are very diverse and ever growing. The emergence of Cloud Computing provides new opportunities like instant service provisioning and a pay-as-you-go model, but present-day SLAs do not meet customer demands for better guarantees. The European Cloud Computing strategy[1] criticizes take-it-or-leave-it SLAs as undesirable for users and requires terms like data integrity, confidentiality, ownership and service continuity to be part of SLA. Hence, a gap between customer needs and provider capabilities has been identified. Reducing this “SLA-gap” is a major problem faced by all service-centric economies where the Cloud seems to be at forefront. Nevertheless, Cloud-based feature-rich SLAs are hard to negotiate by humans especially because multi-issue contract space is large, objectives are not shared and preferences often conflict. Autonomous agents provide an ideal
solution to intelligently resolve conflicts between providers and customers within short times. Upon success, an SLA is created which governs the provisioning of resources. Treating negotiations as a minimal addition to on-demand service procurement, we propose a disruptive but symbiotic paradigm shift from how the Cloud market operates today. This win-win paradigm raises certain mechanical and decision making challenges. For the former, we developed a generic (rule-based) approach to develop and execute multiple protocols as part of the Generic SLA Management (GSLAM) framework[2] of the SLA@SOI[26] project. This allowed automated agents to seamlessly interact in futureproof manner. We address the latter challenges in this work. We use a standard negotiation scenario by deriving requirements for offering SLA-based services from the PaaS service model. Gartner estimates that current PaaS market will reach $2.9 billion[29] by 2016. IDC forecasted the PaaS market to grow over $14 billion by 2017[30]. However, [4] shows that PaaS and SaaS stakeholders not only require an extensible ecosystem for application development on the Cloud where multi-tenant deployments are securely isolated, but also quality guarantees for production services. Hence, we use the OpenShift[27] framework, which addresses these challenges and can be leveraged to deliver measured service levels for QoS and NFP guarantees, catalyzing competition among differentiating providers. As PaaS enables accelerated management of mission critical business applications in the Cloud, the need for feature-rich SLAs is underlined and indeed deserves a deeper investigation. Therefore, in this work we reduce the identified SLA-gap by contributing a robust negotiation strategy using which agents can efficiently produce feature-rich, near-optimal and fair SLAs. Experimental evaluations against a variety of metrics reveal important insights into markets composed of state of the art strategies. We believe that such dynamically negotiated SLAs empower customers, which will improve adoption and migration of services to Cloud. The direction of this work also prepares providers to systematically strategize their value propositions in order to improve payoffs in future markets where SLAs shall be dynamically negotiated. The remainder of this paper is organized as follows. In Section II, we discuss related work. In Section III, we
present the architecture of our Platform Cloud. Section IV, shows how SLA negotiations are modeled and simulated using the GENIUS platform. We present our evaluation results in Section V, which prove the efficacy of our strategy. Finally, we conclude the paper in Section VI. II. R ELATED W ORK The popularity of Service Oriented Infrastructures has led to active interest in negotiating service SLAs. Several negotiation support systems (NSS) have been developed to actuate negotiations. [5] presents an agent-based architecture for SLA negotiations to solve the webservice composition problem. In[6], a marketplace framework is introduced which negotiates SLAs on behalf of stakeholders. [7] delivers a broker framework where business parameters are mapped to decision functions to negotiate SLAs for webservices. These approaches require stakeholders to trust NSS with their objectives. In the SLA@SOI[26] project we developed a generic negotiation platform[8] that allows confidential bilateral negotiations and loose coupling of strategies from the negotiation protocol[9] deployed in various use cases. GENIUS[11] is an excellent simulation platform to benchmark strategies used for ANAC[12][13] competitions. While NSS plays a necessary mechanical role, negotiation strategies provide the inevitable decision making capability to the users of NSS. In his seminal work[14], Raiffa has systematically studied various aspects of negotiation strategies including utility functions, deciding under uncertainty and the role of time. In[15], parameterized polynomial and exponential functions are given to model conceding behavior of agents as Boulware, Conceder or Linear Conceder. [16] has developed a strategic negotiation model and applied it to data and resource allocation problems. In[17], a Q-learning based negotiation strategy is developed for Grid scheduling. Opponent modeling through learning techniques has received special attention in adaptive strategy design. In[18], kernel density estimation is evaluated for eliciting issue preferences of opponent. Similarly, in[19], a scalable Bayesian algorithm is used. Many recent strategies implemented as agents have been disseminated as part of the GENIUS platform. Agent FSEGA[20] uses Bayesian learning to negotiate SLAs for computational grids. AgentK[21] analyzes the mean and variance of opponent’s bids to target a time varying utility value, accepting a bid if improvement is not expected. HardHeaded[22] models an opponent’s utility function by analyzing the frequency of bids’ values. CUHK[24] uses sparse pseudo-input Gaussian processes to model the behavior of opponent. MetaAgent[23] is an interesting (meta)strategy, which uses neural network to predict performance of existing strategies against a given domain. It uses the strategy predicted to outperform others. Most of these works evaluate strategies using utility as a single metric while we evaluate the performance of our strategy against a variety of metrics including utility, social
welfare, social utility and Pareto-optimal bids. Hence, we take fairness and mutual gain into account in addition to competitiveness. Secondly, as results from ANAC[13] show that learning alone does not necessarily guarantee a wider success margin, we study a variety of markets composed separately of non-learning, learning and mixed strategies in order to gain insight how well agents fare considering the integrative but high-conflict nature of our domain. Our strategy also extends the state of art on tit-for-tat strategies without employing a complex learning method, hence staying computationally inexpensive in comparison to competitors. We note that SLA negotiations have been mostly studied in relation to allocation and composition problems in Grids, web services and eCommerce domains, but getting much attention in recent Cloud-based projects[3]. In line with our ongoing work on the PaaSage[28] project, we are interested in QoS-aware services that use the Cloud as a delivery platform and focus on PaaS-based value propositions. III. A RCHITECTURE As the service provider for Max-Planck Society of Germany and the University of G¨ottingen, GWDG owns high performing compute and storage systems consisting of 42 physical servers with a total of 2496 CPU cores and 9.75 Terabytes of RAM. There are 38 Dell PowerEdge C6145 servers with AMD Interlagos Opteron processors with 18.55 Terabytes raw disk capacity, and 3 Petabytes of distributed data storage. On this, a Compute (IaaS) Cloud has been developed using the KVM and OpenStack1 framework. Using our self-service web-based interface, customers can instantly provision Virtual Machines (VMs). However, this leaves a lot of operational overhead for SaaS users, e.g., to install a software, configure it with source repositories, load balancers and databases, which usually lie on different VMs. A. GWDG Platform Cloud The Platform-as-a-Service (PaaS) model solves these issues by automating these tasks; allowing SaaS users to focus on applications. For this purpose, GWDG has prototyped a Platform Cloud using OpenShift Origin[27], an open-source PaaS framework. For scaling flexibility, we based it on top of our OpenStack Compute Cloud. OpenShift provides an ecosystem that supports multiple languages, databases and middlewares in the form of pluggable and extensible execution environments called Cartridges. It also provides a template-based approach called QuickStart2 that allows hooks to control life cycle of an application. Quickstarts and Cartridges Cloud-enable an application. API commands reuse a QuickStart to provision an application on resource-constrained containers called Gears, which can be auto-scaled. A Gear can have small, medium or large sizes based on the capacity of CPU, RAM, Bandwidth and 1 http://www.openstack.org 2 https://github.com/openshift-quickstart
Customers
2
SaaS Provider/ Developer
Service Access
1 Public IaaS Providers EC2 ...
HP Cloud
SLA Negotiation or Renegotiation
Capacity Manager
Billing Manager
Monitoring Manager
Provisioning Manager
Throughput
use
Service Level to Gear Mappings
QuickStart Reportoire
10, 20, 30
Operational Stack
Tomcat
Redmine
Sugar
Magento
...
Moodle
SLA Store
Large
Metering Management
Configuration Management
Virtualization Management
Location Management
Gear 1 [ Small ] (Code)
Gear 2 [ Small ] (Load Balancer)
Gear 3 [Medium] (DB)
Gear n
Compute
Storage
Network
Facilities
5 gears , 6 gears , 7 gears [1300,2000] , [600,1299] , [100,599]
...
Availability
contained in IaaS (OpenStack)
1 gear , 2 gears , 3 gears [1300,2000] , [600,1299] , [100,599] 3 gears , 4 gears , 5 gears [1300,2000] , [600,1299] , [100,599]
comprise of
store
Medium
Gear Quantity & Response Times
PaaS (OpenShift) use
70, 80, 90
provision
SaaS (value-added propositions) WordPress
40, 50, 60
Gear Quality Small
AppStore Web Interface
SLA Negotiation or Renegotiation
fetch
acquire Negotiation Manager
API / Command Line Interface
1
SaaS User
Service Level to Gear Mappings
SOCCI Management Building Blocks Business Stack
OpenShift Node VM 1
OpenShift Node VM 2
performed by
OpenShift Broker VM
monitor ...
OpenShift Node VM n
External IaaS Provider 1: 51% - 79% External IaaS Provider 2: 79% - 99% Internal IaaS Provider:
51% - 99%
Backup controls federation
Never, Daily, Weekly, … , Monthly
Figure 1: Architecture Diagram of GWDG Platform Cloud Prototype
Disk resources assigned to it using technologies like kernel namespaces and control groups (cgroups). Gears are thinprovisioned on the Linux kernel, while SELinux ensures secure isolation of multi-tenant Gears over single VMs. cgroups and SELinux provide a technological breakthrough to benchmark what service levels for QoS and NFP(s) can be guaranteed with different quality and quantities of Gears. These results are stored as Service Level to Gear mappings. The Platform Cloud prototype features an AppStore that offers managed SaaS instances to customers e.g., Redmine CMS, Tomcat web-container, Sugar CRM, WordPress blogging tool, etc. Under the hood, it leverages the design principles of Service Oriented Cloud Computing Infrastructure (SOCCI) architecture by OMG3 . SOCCI synergizes SOA with Cloud Computing by standardizing service orientation components called Management Building Blocks (MBB) in order to manage Cloud infrastructure as a service. Fig.1 shows the MBBs for GWDG Platform Cloud highlighted in blue and divided among business and operational stack. We leverage the negotiation platform of the SLA@SOI project by a Negotiation Manager component to allow dynamic multi-round SLA (re)negotiation with customers and internal/external providers. Hence, negotiations displace direct procurement of services as the first interaction and service is accessed in a second interaction. The Provisioning Manager uses a QuickStart from its repertoire and the Service Level to Gear mappings to provision an application instance against the negotiated SLA. The Billing Manager uses metering management of OpenShift to bill the customer on pay per use basis and service levels. The Monitoring Manager uses our OpenTSDB based mechanism to collect resource usage for accounting and display purpose[10]. The Capacity Manager supervises virtualiza3 http://www.opengroup.org/soa/source-book/socci/index.htm
tion management of OpenShift, and decides when to acquire more VMs from the IaaS. In envisioned Cloud markets[26], this acquisition is performed by the Negotiation Manager to negotiate SLA-based resources like VMs, possibly bursting out to public IaaS like EC2, etc. if private IaaS lacks capacity. Since OpenShift is IaaS-agnostic, cross provider infrastructure resources can be added to its federation on the fly by running configuration scripts that connect them to its Broker. IV. N EGOTIATION D OMAIN AND U TILITY S PACE Service benchmarking leads to formulating an SLA template (or negotiation domain) representing a large space of contracting possibilities. In this work, we treat that part as a blackbox in order to focus on negotiation strategies. GENIUS allows to model this abstraction by creating an XML-based negotiation domain model. Our model called the “PaaS domain” comprises of negotiable issues vital to any business-critical service hosted in the Cloud. These are Availability, Performance and Backup, defined as: Availability: the probability that a service is up and running. Formally, it is defined as (T −d)/T where T is the monitored time during which total downtime was d. Performance: relation < a, x > where x is the maximum time to return the response of a request r, received in time period t where the maximum request arrival rate (throughput) a is not exceeded. Here, throughput and response time are considered together, respecting preferential independence among issues - a necessary condition to use linear additive utility function. Container technologies position a PaaS provider to treat Performance as a superfluous issue[14] to create conceding illusions during negotiation. Backup: the ability to archive code and data of an application at specific times. OpenShift can automatically orchestrate this using Cron jobs on times agreed in the SLA.
Using GENIUS, we conduct tournaments where agents written by various researchers autonomously negotiate over these issues by exchanging bids using Rubinstein’s alternating offers protocol (bilateral variant). The worth of a bid b = {x1 , ..., xN }, where x is the chosen value of N issues is given by the linear additive utility function: u(b) =
N X
wi Vi (xi )
(1)
i=1
where u(b) is the utility of the bid b, wi is the weight N P for issue xi such that wi = 1 and Vi (xi ) = i=1
eval(xi )/max(eval(xi )) ∈ [0, 1] is the normalized evaluation (real) value for the chosen value xi (shown as best and worst column in table I) of the ith issue. eval is an evaluation function which assigns utility to each issue value. Providers do this in accordance with resource usage determined by Service Level to Gear mappings, while customers according to business demand. A customer wanting the highest throughput and lowest response time associates best evaluation value of 290 to Performance issue values < 90, 100 >, while the worst evaluation of 0 to < 10, 2000 >, and so on. Evaluation values alongside issue priorities (weights) constitute a preference profile for customer and provider. In this manner, domain experts specify business objectives through utility function parameters, while the agents aim at maximizing individual utility with no information about the opponent’s values. The PaaS domain, its customer and provider preference profiles are shown in table I. Even with only three issues, the PaaS domain is reasonably large consisting of (9x20)x13x7=16,380 possible agreements. This is challenging for humans as each bid can be mesmerizing, whereas an agent can deal with this complexity far more efficiently. This burden on a human is quantified in Fig. 4(a) where the author of PaaS domain and preference profiles negotiated with the CUHK agent[24] for 2 minutes. Despite complete information and subsequent cocessioning by human, only 83 rounds were exchanged due to human think-time and negotiations broke off, whereas in agent to agent negotiations, rounds easily scale to thousands which increases convergence rate. In this work, we chose preferences for the two sides based on interests representative of Cloud customers and providers. These are often at high-conflict with each other, as the evaluation values indicate. Customers value a high performing and highly available service while a provider would require significant investment to offer the same, hence it may associate a lower evaluation to such demanding guarantees even with uniform priorities. The opposition between utility functions[13] of two sides is given by: opposition(Ω) = min distω∈Ω (ω, ω ¯)
(2)
where ω ∈ Ω and Ω is the utility space representing all outcomes or bids. Customer and provider’s utility functions
are used to map each ω to a real value in range [0,1]. This value is plotted as a single red dot on the negotiation chart as shown in Fig. 2 where our agent, ReactiveExploiter, acts as a buyer against the CUHK agent. Each point is a possible SLA while the red curve is the Pareto-frontier of this negotiation domain. dist is the Euclidean distance between all points ω and ω ¯ where ω ¯ is the highest (but impossible) payoff point (1,1) for all ω. The relatively extended frontier and the large gap with ω ¯ indicates a strong opposition between the opponents. The blue and green symbols show the bids exchanged by each agent during the negotiation dance[14]. The red square shows the actual agreement.
Figure 2: ReactiveExploiter as buyer against CUHK An agreement struck on the Pareto-frontier is one measure of optimality, however as shown in Fig. 2, it may be highly biased in favor of one agent, as opposed to Fig. 3, where it is rather close to the mid-section. The Nash point, shown as black square represents an agreement where the product of two utilities is maximum, while the Kalai-Smorodinsky point shown in blue square represents an agreement with maximum fairly proportioned utilities for both opponents. Price is not a negotiable issue here as we are interested in quickly resolving conflicts on QoS and NFP issues as part of a service procurement command. High utility deals would translate to large quantity or high quality Gears, having high price but provisioning is confined to user’s quota with the Platform Cloud. In integrative negotiation scenarios such as the PaaS or similar non zero-sum domains, the importance of mutual benefit cannot be neglected. This highlights fairness measures like social welfare, which is the averaged sum of utilities of two agents in a negotiation tournament. Another fairness measure is social utility, which is the sum of utility and social welfare achieved in a tournament. A. Negotiation Strategy A negotiation strategy is modeled abstractly as a 3-tuple: S = (B, Mo , A) where B is the bidding function that generates a counter offer in response to opponent model and time, Mo is the opponent model based on received bids
Table I: Negotiation Domain, Constraints and Preference Profiles Negotiation Domain
Customer Evaluation
Provider Evaluation
Negotiable Issues
Unit
Min
Max
Selection Constraint
Worst
Best
Weight
Worst
Best
Weight
Availability
percentage
51
99
increment by 4
50
99
0.45
51
99
0.33
Performance
0
290
0.28
14
100
0.34
Backup
time
2
10
3
9
0.27
2
10
0.33
choose from {2,4,5,7,8,9,10}
Algorithm 1 Reactive Exploitation (RE) Strategy. Let tmax be the maximum time, tc is current time and Bsort is list of all bids sorted in descending order on utility. Bl is list of last l opponent bids. u gives the utility of a bid, ur is the reserved utility value below which a bid is not accepted. br is the received bid and bs is the bid to be sent. bbest is best bid received. h is the measure of opponent’s hardheadedness and α is acceptable threehold for h. ρ is probability of concession and β is acceptable threshold for ρ.
Figure 3: ReactiveExploiter as seller against CUHK
and/or opponent moves, and A is the acceptance function which decides to accept a bid and conclude negotiation. Prior art shows a depth of variety and complexity induced by these individual functions (Section II). Consequently, this adds to the computational load on negotiation platform. One of our goals is to investigate if computationally inexpensive yet reasonably adaptive functions can also produce effective results. This led to the design of our negotiation strategy named “Reactive Exploitation” detailed in Algorithm 1. Reactive Exploitation (RE) is a quasi Tit-for-Tat strategy which produces concessions if the opponent is perceived to be doing the same and vice versa. The quasi part stems from how lenient or strict its behavioral parameters (α and β) are configured, which may lead to wide spectrum of behaviors. For our experiments, we set α = 2, β = 0.6 and ur = 0.6. These settings afford it a demanding Boulware attitude. Negotiations take place within a standardized time t ∈ [0, 1]. To avoid a possible impasse at the very beginning, it starts by giving a minimal concession, hoping to lure the opponent into reciprocity. Then on, it maintains a mean value of opponent’s last l-many bids which it uses to model the hardheadedness h ∈ {0, l − 1} of its opponent. h is directly proportional to an increase or decrease in mean value. Lines 8-17 show its bidding function which generates a counter proposal bs considering α and β thresholds, provided that the proposed bid has better than its reserved utility value ur . Given utility u of opponent’s bid br , the concession rate ρ
Require: tmax , ur , l, Bsort , α, β 1: bs ← getStartingBid(); 2: while tc ≤ tmax do 3: br ← receiveBid; 4: bbest ← updateBestBid; 5: h ← updateHardheadedness; 6: tc ← remainingT ime; 7: ρ ← getConcessionP robability(br , tc ); 8: if h < α and ρ > β then 9: temp ← nextConcessionedBid(Bsort ); 10: if u(temp) > u(bbest ) and u(temp) ≥ ur then 11: bs ← temp; 12: else if u(bbest ) ≥ ur then 13: bs ← bbest ; 14: end if 15: else 16: bs ← choose(bbest , bs , ur ); 17: end if 18: if isAcceptable(br , bs , tc ) then 19: return accept(br ); 20: else 21: return bs ; 22: end if 23: end while
is adapted to current time tc and opponent as: p u − 2utc + 2(tc − 1 + (tc − 1)2 + u(2tc − 1)) ρ= (3) 2tc − 1 ρ has been suggested as acceptance function in[31], but we employ it in our bidding function instead. The value of ρ increases for low utility bids br as time tc increases. RE also maintains the best bid bbest received from opponent in a negotiation session and resorts to it, if it cannot find a better bid. Otherwise, it chooses the next bid from bbest or (last) bs , whichever has higher utility as shown in line 16, until ρ and h change in subsequent negotiation round(s).
Lines 18-22 show its acceptance function A, which takes into account br , bs and tc . Practical experience showed that A increases chances of convergence when acceptable thresholds are fixed against time, rather than relying on dynamic methods. Hence, we define A to accept a bid when either of the following conditions are met:
linearly till it approaches its reserved utility, while Boulware and TheNegotiator follow exponential rates. The latter gets highly active but only after 80% of time has elapsed. The behavior of RE is similar when acting as a customer but not shown due to space shortage. The final results are shown in Fig. 4(f). RE emerges as the winning strategy from tournament 1. TheNegotiator ranks second.
u(br ) > 0.98
(4)
u(br ) > u(bs ) ∧ u(br ) > 0.85 ∧ tc > 0.95
(5)
B. Tournament 2
u(br ) ≥ ur ∧ tc > 0.998
(6)
In this tournament, the participating agents are top performers of ANAC competitions and hence considered state of the art. These include AgentK (winner 2010), HardHeaded (winner 2011), CUHK (winner 2012) and the runner up of 2013 - the MetaAgent4 . These agents employ complex opponent learning and/or prediction models as well as mixed conceding tactics, making negotiations very competitive. The final results are shown in Fig. 4(g). CUHK takes the first position, while HardHeaded ranks second.
This also induces a late agreement behavior i.e., RE utilizes maximum negotiation time in most cases as major concessions are usually derived in last moments. V. E XPERIMENTAL E VALUATIONS Employing dynamic negotiations of SLAs for real world services can have drastic impact whose full extent must be well understood beforehand. Therefore, negotiation simulations are vital to gain necessary insights. Considering this, we devised systematic experiments to evaluate the robustness of RE against state of the art competitors in separate non-learning, learning and mixed tournament compositions where agents negotiate using customer and provider profiles. The first tournament includes non-learning agents, who mostly rely on time to generate concessions. RE also participates in this tournament. The second tournament includes learning-based agents who are top performers in past ANAC competitions. We compose a third tournament by choosing the top two contestants from both tournaments, ranked per one optimality metric (utility) and one fairness metric (social utility). For a holistic validation, we also present scores against social welfare and the percentage of Pareto-optimal bids or simply referred to as the Pareto-optimal bids. Finally, we bring all eight agents together in the fourth tournament to analyze how results vary in a market constituted of widely heterogeneous strategies. Since we are interested in using negotiations as a service procurement method, we use a reduced time of 2 minutes instead of 3 (used in ANAC). This also caps the slight edge learning-based agents possess over their non-learning counterparts. A. Tournament 1 This tournament includes Boulware, Linear Conceder and TheNegotiator agents, all of whom do not employ learning to model opponent. TheNegotiator[25] - a rather recent strategy acts hardheaded till a predetermined time, after which it turns more concessive. Fig. 4(b) and (c) show a two-sided XRay view of concessions produced and received respectively when RE negotiates with opponents as a provider. Fig. 4(b) exhibits a tough Boulware-like behavior, where RE concedes only after more than half the time has elapsed. A cautious tit-for-tat response is observed where RE does not give in too quickly. Fig. 4(c) confirms that Linear Conceder concedes
C. Tournament 3 This tournament represents an interesting mix of best nonlearning and learning strategies for our domain. Fig. 4(d) and (e) reveal the concessions produced and received respectively, as RE negotiates with opponents as a customer. A notable difference from Fig. 4(b) and (c) is that now the agents negotiate hard with notable concessions appearing after 85% of time has elapsed. The amount of concessions produced is much higher seen by an almost overlap of colors. TheNegotiator extracts concession from RE earlier than HardHeaded or CUHK, but only manages to minimally lower RE’s utility. Negotiation with HardHeaded provide an interesting insight. Note the red line indicating concessions extracted by HardHeaded takes a steep dive but just as the time is nearly finished, it starts an upward surge as the opponent’s bids start improving RE’s utility. This is indeed the result of bids found on the iso-curve[14]. Acting rationally, RE picks up this increase in utility, deflecting from its usual tit-for-tat tactic in order to maximize its payoff. Finally, we see that CUHK is comparatively the most demanding strategy which starts to give in when the time is almost ending seen by the blue line. However, the acceptance function of RE and CUHK help avoid a break off. The behavior of RE is similar when acting as a provider. The final results are shown in Fig. 4(h). Here, CUHK takes the first place and RE ranks second. D. Tournament 4 Tournament 4 includes all agents, culminating the weakest and strongest strategies. The final results are shown in Fig. 4(i). The CUHK scored best in terms of utility, followed by HardHeaded and RE. For social utility, RE scored best followed by CUHK and TheNegotiator. 4 We chose MetaAgent since negotiations broke off with the winner TheFawkes. The difference in their performance is statistically negligible.
E. Discussion The utility gap between the best and worst is largest in tournament 4 at 41%, comparing to 35% in tournament 1, 15% in tournament 2 and 24% in tournament 3. This confirms that learning indeed increases market competitiveness. However, the mean utility of markets stand at 0.688, 0.685 and 0.681 in tournaments 1, 2 and 3 respectively, with a minimal decrease in tournament 4 at 0.67. This shows that overall the markets remain stable even with the presence of weaker strategies. Although weaker agents were worse off, no agent quitted or crashed in any tournament. Tournament 4 also scored low for mean social welfare value at 1.19, while that of tournament 1, 2 and 3 stood at 1.377, 1.371, 1.363. This indicates that the joint gains achieved by markets with wider spectrum of strategies will be lower than homogenous ones. This is reinforced by mean social utility values, which are 2.066, 2.057, 2.044 for tournaments 1, 2 and 3 but a mere 1.869 for tournament 4. RE arguably uses a simplistic single parameter opponent model h representing the perceived hardheadedness, alongwith a Boulware-oriented utility-decreasing bidding function that prevents freefall concessioning. Void of complex learning method, RE exploits its opponent’s learning skill to retain high utility bidding. This is evident from the high percentage of generated bids that lied on the Paretofrontier, in which terms it ranks best in all participated tournaments. The results also show that RE secures a wider utility margin over weaker opponents, while not breaking off any negotiation against stronger opponents. Opponents against whom it lost individually in terms of utility, only won with a maximum margin of 21%, while those against whom RE won, lost with a maximum margin of 57%. RE won some matches against CUHK, while loosing all against HardHeaded and Boulware although by a maximum margin of only 16% and 18% respectively. RE won all matches against TheNegotiator, Linear Conceder, AgentK and MetaAgent. This way, it accumulated enough overall advantage to dominate various metrics. RE’s strong performance in social utility shows that agreements had a fair distribution of utility, entailing participant satisfaction. The results are encouraging and prove RE’s efficacy in integrative but high-conflict domains where markets are composed of learning, non-learning or mixed strategies. VI. C ONCLUSION AND F UTURE W ORK In this work, we presented a robust negotiation strategy which can be employed in service-based markets to dynamically negotiate near optimal and fair SLAs. Our experiments showed that the proposed strategy performs at par with state of the art opponents. We grounded our work on Cloud-based services focusing on the PaaS domain which offers promising potential. Future prospects include expanding evaluations to analyze the generality of our strategy,
exploring low-conflict domains suitable for fault tolerant services and including metrics like selfishness and altruism. ACKNOWLEDGMENT Research leading to these results is supported by the EU (FP7/2007-2013) project PaaSage under grant no. 317715 and Gesellschaft f¨ur wissenschaftliche Datenverarbeitung mbH, G¨ottingen (GWDG). R EFERENCES [1] E. Commission, “Unleashing the Potential of Cloud Computing in Europe,” 2012. [2] M. A. R. Gonzalez, et al., “GSLAM - The Anatomy of the Generic SLA Manager,” in Service Level Agreements for Cloud Computing, pp. 167–186, Springer-Verlag, 2011. [3] E. Commission, “Cloud Computing Service Level Agreements - Exploitation of Research Results,” 2013. [4] S. Garcia-Gomez, et al., “Challenges for the comprehensive management of Cloud services in a PaaS framework.”, in Scalable Computing: Practice and Experience, 13, no.3, 2012. [5] M. Chhetri, J. Lin, S. Goh, J. Yan, J. Zhang, and R. Kowalczyk, “A coordinated architecture for the agent-based service level agreement negotiation of Web service composition,” in Australian Software Engineering Conference, pp. 10 pp., 2006. [6] E. D. Nitto, M. D. Penta, A. Gambi, G. Ripa, and M. Villani, “Negotiation of service level agreements: An architecture and a search-based approach,” in ICSOC 2007, pp. 295–306, 2010. [7] F. H. Zulkernine and P. Martin, “An Adaptive and Intelligent SLA Negotiation System for Web Services,” IEEE Transactions on Services Computing, vol. 4, pp. 31–43, Jan. 2011. [8] E. Yaqub, P. Wieder, C. Kotsokalis, V. Mazza, J. L. Rueda, and S. Garcia, “A Generic Platform for Conducting SLA Negotiations,” in Service Level Agreements for Cloud Computing, pp. 187–206, Springer-Verlag, 2011. [9] E. Yaqub, R. Yahyapour, P. Wieder, and K. Lu, “A Protocol Development Framework for SLA Negotiations in Cloud and Service Computing,” in GECON, pp. 1–15, Springer, 2012. [10] A. Jehangiri, E. Yaqub and R. Yahyapour, “Practical Aspects for Effective Monitoring of SLAs in Cloud Computing and Virtual Platforms,” in CLOSER, pp. 447–454, 2013. [11] Lin Raz, S. Kraus, T. Baarslag, D. Tykhonov, K. Hindriks, and C. M. Jonker, “GENIUS : An Integrated Environment for Supporting the Design of Generic Automated Negotiators,” Computational Intelligence, 2008. [12] T. Baarslag, K. Hindriks, C. Jonker, S. Kraus, and R. Lin, “The first automated negotiating agents competition (ANAC 2010),” in New Trends in Agent-Based Complex Automated Negotiations, pp. 113–135, Springer, 2010. [13] T. Baarslag, K. Fujita, E. H. Gerding, K. Hindriks, T. Ito, N. R. Jennings, C. Jonker, S. Kraus, R. Lin, V. Robu, and C. R. Williams, “Evaluating practical negotiating agents: Results and analysis of the 2011 international competition,” Artificial Intelligence, vol. 198, pp. 73–103, May 2013. [14] H. Raiffa, The Art and Science of Negotiation. Harvard University Press, 1982. [15] P. Faratin, C. Sierra, and N. R. Jennings, “Negotiation decision functions for autonomous agents,” Robotics and Autonomous Systems, vol. 24, no. 3-4, pp. 159–182, 1998. [16] S. Kraus, Strategic Negotiation in Multiagent Environments. The MIT Press, 1 ed., 2001. [17] J. Li and R. Yahyapour, “Learning-based negotiation strategies for grid scheduling,” in CCGRID’06, pp. 8-pp, 2006.
1.00 0.90
Utility
0.80
Linear Conceder Boulware TheNegotiator
0.70
Utility
0.0 0.2 0.4 0.6 0.8 1.0
Human CUHK
0
20
40
60
80
0.5
0.6
Negotiation Rounds
0.7
0.8
0.9
1.0
Time
(b) Concessions given by RE
(c) Concessions received by RE
1.5
Score
Boulware ReactiveExploiter TheNegotiator LinearConceder
0.0
0.5
1.0
0.90 0.80
CUHK HardHeaded TheNegotiator
0.70
Utility
2.0
1.00
2.5
(a) Utility for human
0.70
0.80
0.90
1.00
Utility
Social.Welfare Social.Utility
2.5 Pareto.Bids
Utility
2.0
Social.Welfare Social.Utility
Metric
(g) Tournament 2 evaluation
1.5
Score
0.5 0.0
0.5 0.0 Social.Welfare Social.Utility
TheNegotiator ReactiveExploiter MetaAgent HardHeaded CUHK LinearConceder Boulware AgentK
1.0
2.0 1.5
Score
CUHK ReactiveExploiter HardHeaded TheNegotiator
1.0
1.5 1.0 0.0
0.5
Score
AgentK HardHeaded CUHK MetaAgent
(f) Tournament 1 evaluation 2.5
(e) Concessions received by RE
2.0
(d) Concessions given by RE
Utility
Pareto.Bids
Metric
2.5
Time
Pareto.Bids
Metric
(h) Tournament 3 evaluation
Utility
Social.Welfare
Social.Utility
Pareto.Bids
Metric
(i) Tournament 4 evaluation
Figure 4: Experimental Results
[18] R. M. Coehoorn and N. R. Jennings, “Learning on opponent’s preferences to make effective multi-issue negotiation tradeoffs,” in ICEC ’04, p. 59, ACM Press, 2004. [19] K. Hindriks and D. Tykhonov, “Opponent modelling in automated multi-issue negotiation using bayesian learning,” in AAMAS 2008, pp. 331–338, 2008. [20] G. C. Silaghi, L. D. Serban, and C. M. Litan, “A timeconstrained SLA negotiation strategy in competitive computational grids,” in FGCS, vol. 28, pp. 1303–1315, 2012. [21] S. Kawaguchi, K. Fujita, and T. Ito, “AgentK: Compromising Strategy Based on Estimated Maximum Utility for Automated Negotiating Agents,” in New Trends in Agent-Based Complex Automated Negotiations, pp. 137-144, Springer-Verlag, 2012. [22] T. van Krimpen, D. Looije, and S. Hajizadeh, “HardHeaded,” in Complex Automated Negotiations: Theories, Models, and Software Competitions, pp. 223–227, Springer-Verlag, 2013. [23] L. Ilany, and Y. Gal. “Algorithm Selection in Bilateral Nego-
tiation,” in Workshops at 27th AAAI Conference on AI, 2013. [24] S. Chen, H. Ammar, K. Tuyls, and G. Weiss, “Optimizing complex automated negotiation using sparse pseudo-input gaussian processes,” in AAMAS 2013, pp. 6–10, 2013. [25] A. Dirkzwager, M. Hendrikx, and J. D. Ruiter, “TheNegotiator: A Dynamic Strategy for Bilateral Negotiations with Time-Based Discounts,” in Complex Automated Negotiations: Theories, Models & Software Competitions, pp. 217–22, 2013. [26] SLA@SOI. [Online]. Available: http://sla-at-soi.eu. [27] OpenShift. [Online]. Available: https://www.openshift.com. [28] PaaSage. [Online]. Available: http://www.paasage.eu. [29] Gartner Press Release, 2012. [Online]. Available: http://www. gartner.com/newsroom/id/2242415. [30] IDC Press Release, 2013. [Online]. Available: http://www.idc. com/getdoc.jsp?containerId=prUS24435913. [31] GENIUS. [Online]. Available: http://ii.tudelft.nl/genius.
