Towards a Fuzzy-based Model for Human-like ... - Semantic Scholar

2007 IEEE/WIC/ACM International Conference on Intelligent Agent Technology

Towards a Fuzzy-based Model for Human-like Multi-Agent Negotiation Zaynab Raeesy Jakub Brzostwoski Ryszard Kowalczyk Faculty of Information and Communication Technologies Swinburne University of Technology Hawthorn, Victoria 3122, Australia {zraeesy, jbrzostowski, rkowalczyk}@ict.swin.edu.au Abstract

Each linguistic value consists of two parts: a syntactic label, and a semantic meaning. The syntactic part is the label of a linguistic value and is chosen from a set of terms in a natural or artificial language. The semantic value is a fuzzy set, showing the distribution of belongingness of values in the interval of linguistic value. For example, for the price of a DVD, the set {very cheap, cheap, medium, expensive, very expensive} is the syntactic label set, and the fuzzy values for each of the members of the set represents the semantic part. Agents should be able to interact with humans, and to act in a relatively similar manner. That is, agents also need to prioritize their preferences and, to deal with uncertain values in a way similar to what humans do during negotiation. Agent can be soft about the offers they propose to the opponent and thus they should be able to propose an imprecise offer to the opponent(s). Faratin et al. [5] investigate agents’ different behaviours in decision making during negotiation. In their model, basic decision making strategies are utilized in different scenarios within various negotiation environments. Their work majorly focuses on manipulating service-oriented negotiation while exchanging quantitative values. They also investigate negotiation with qualitative values, by considering them as predefined and pre-weighted fuzzy values (where agents can choose a precise value in a fuzzy interval to make offers). However, they do not consider generating and manipulating fuzzy and non-precise offers in their approach. Carbo et al. [4] propose a fuzzy approach to generate and present counter-offers. These counter offers are meant to describe a rejection in reply to the opponent’s offer. The authors apply a numeric and a linguistic fuzzy set for creating the counter offers. Although their fuzzy set approach wastes fewer computational resources, it yields to less overall benefit. However, their method utilizes the fuzzy values sent only from buyer side to clarify the unsatisfactory parts of sellers’ crisp offer. Therefore, their fuzzy label approach improves the satisfaction level of only the buyer from the agreement, while making it last longer.

Intelligent agents are applied to automate the process of negotiation. Typical multi-agent negotiation strategies only allow the exchange of precise quantitative values as attributes. In the real-world applications, many of the negotiation issues such as qualitative concepts, cannot be expressed by exact numeric values. In this paper, we propose a new fuzzy-based model for negotiation. Our model leads to a human-like negotiation, and enables negotiation parties to act flexibly.

1. Introduction Intelligent agents are capable of performing autonomous actions to move towards their predefined goals [10]. In practice, agents usually work in a collaborative environment. Agents may have opposite preferences in their goals, which maybe conflicting. The process by which several agents try to come to an agreement by resolving their conflicts, is called negotiation [6, 9]. Negotiation can be performed on single, or multiple issues, and can simultaneously occur between two or multiple agents. For quantitative concepts, negotiation issues can be simply expressed by crisp values (e.g. price = $10 for a DVD). However, when negotiation issues are qualitative, it is either impossible or very complicated to apply precise numeric values for expressing them (for instance, “cheap” price of a DVD). Linguistic variables [12] and fuzzy sets [11] have been suggested as effective tools for addressing these issues. According to Zadeh [12], a linguistic variable receives a term or sentence as its value. Fuzzy sets and linguistic values are applied in many areas related to negotiation and decisionmaking; for example, linguistic decision analysis [1, 7], multi-criteria decision making [2], and software development [8].

0-7695-3027-3/07 $25.00 © 2007 IEEE DOI 10.1109/IAT.2007.86

523 521 515

Authorized licensed use limited to: SWINBURNE UNIV OF TECHNOLOGY. Downloaded on March 15,2010 at 04:57:32 EDT from IEEE Xplore. Restrictions apply.

fuzzy number (I) fuzzy number (J)

In another work, Carbo et al. [3] apply learning algorithms to predict partners’ offer. They describe their approach via a system containing a buyer, and multiple merchant agents. The buyer proposes fuzzy counter-offers to the merchants, which then translate the offers to crisp values. However, their main contribution is the utilization of machine learning to discover the preferences based on fuzzy counter-offers. In general, previous work in this area, although includes qualitative values, it is mostly focused on crisp offer exchange. In this paper, we propose a model that utilizes fuzzy values to represent offers on both sides of negotiation.

1.0 Membership

0.8 0.6 0.4 0.2 0.0

a

c z b d

Figure 1. Triangular fuzzy values I and J when they are almost certain about not losing in the negotiation. Our model considers four different criteria before n decision-making: 1) the deadline is met, 2) xtB→A is fuzzy and there is no overlap in the domains of fuzzy values, 3) n xtB→A is fuzzy and the stretches of the two last fuzzy offers n exchanged overlap and 4) xtB→A is not fuzzy. The decision functions for the mentioned criteria are respectively:

2. Negotiation We propose a fuzzy-based model for negotiation. We utilize the crisp negotiation model of Faratin et al. [5] as our baseline. Although our model has similarities with the crisp negotiation in terms of score values and concession making tactics, it uses different criteria for termination conditions, offer generation and decision-making methods. Thus, we apply both of the models for negotiation in this research. For simplicity, we only investigate negotiating with fuzzy values. Designing a more sophisticated linguistic model based on the fuzzy negotiation and predefined linguistic values is left for future work.

n 1. decisionA (tn+1 , xtB→A ) = stop; if tn+1 ≥ DA n 2. ( decisionA (tn+1 , xtB→A )= tn+1 tn n accept(xB→A ) if VA (xtB→A ) ≥ VA (xA→B ) tn+1 otherwise propose(xA→B ) n 3. ( decisionA (tn+1 , xtB→A )= tn+1 tn+1 propose(zA→B ) if VA (zA→B ) ≥ VA (threshtAn ) tn+1 otherwise propose(xA→B )

Negotiation Model. In this study, the negotiation model is a single-issue bilateral negotiation. The model consists of two agents pursuing opposite goals, in a way that the more one agent gains out of a negotiation, the more the opponent loses (distributive negotiation). Each agent A has a deadline DA to fulfill the negotiation by that time, and has a scoring function–defined as VA (x) for an offer x–that evaluates each offer. Each agent A has a minimum (minA ), and a maximum (maxA ) value, that together specify negotiation domain of the agent. The reservation value for an agent is the value that yields the lowest benefit for that agent.

The intersection point z is determined to be the point that maximizes the membership value of the fuzzy set in the intersection area (colored triangle in Fig. 1). n 4. decisionA (tn+1 , xtB→A )=  tn tn accept(zB→A ) if VA (zB→A ) ≥ VA (threshtAn ) tn+1 propose(xA→B ) otherwise

where the threshold for each agent in every round is calculated based on the time left to the end of negotiation: threshtAn =

Negotiation Values and Parameters. In our proposed negotiation protocol, agents are capable of sending both a fuzzy and crisp values as their offers. The crisp value, is a simple numeric value, while the fuzzy value is defined as a special type of triangular fuzzy number (TFN)–examples are provided in Fig. 1. As the negotiation proceeds, in each step, sequentially agents send their offers to the opponent. When an agent receives an offer, it has three options for the next step: accepting the offer, rejecting the offer and sending a counter offer, or terminating the negotiation. We propose a double-value protocol in which agents negotiate with fuzzy values, but can switch to crisp protocol

tn−1 DA −tn (peak(xA→B )) DA

+

tn−1 tn (end(xA→B )) DA

The peak is equal to the value with the highest belongingness to the fuzzy offer (membership value equal to one), and the end is the last value that belongs to the fuzzy offer (with membership value equal to zero).

Negotiation Tactics. The mechanism of decision making and generating counter-offers for an agent, varies based on the tactic that the agent is designed to apply. Among the influencing factors on negotiation strategies, are the time passed so far from the agent’s deadline (time dependent method), the resources that have already been used (resource dependent method), and the opponent’s strategy for making concession (behaviour dependent method) [5]. In

516 522 524

Authorized licensed use limited to: SWINBURNE UNIV OF TECHNOLOGY. Downloaded on March 15,2010 at 04:57:32 EDT from IEEE Xplore. Restrictions apply.

1.0

Cost Adjusted Utility

Buyer’s B = 0.02

Buyer’s B = 0.1

Buyer’s B = 0.5

Buyer’s B = 1.0

Buyer’s B = 2.0

Buyer’s B = 10

Buyer’s B = 50

0.5

0.0

a

-0.5

Fuzzy Buyer Crisp Buyer

-1.0 1.0

Cost Adjusted Utility

Seller’s B = 0.02

Seller’s B = 0.1

Seller’s B = 0.5

Seller’s B = 1.0

Seller’s B = 2.0

Seller’s B = 10

Seller’s B = 50

0.5

0.0

b

-0.5

Fuzzy Seller Crisp Seller

B = 10

B = 2.0

B = 50

B = 1.0

B = 0.5

B = 0.1

B = 50

B = 10

B = 0.02

B = 2.0

B = 0.5

B = 0.1

B = 1.0

B = 50

B = 0.02

B = 10

B = 2.0

B = 1.0

B = 0.5

B = 0.1

B = 50

B = 10

B = 0.02

B = 2.0

B = 1.0

B = 0.5

B = 0.1

B = 50

B = 0.02

B = 10

B = 2.0

B = 1.0

B = 0.5

B = 0.1

B = 50

B = 0.02

B = 10

B = 2.0

B = 1.0

B = 0.5

B = 0.1

B = 50

B = 10

B = 0.02

B = 2.0

B = 1.0

B = 0.5

B = 0.1

B = 0.02

-1.0

Figure 2. Cost adjusted utility values for long deadlines for (a) first-player of negotiation with a smaller stretch in fuzzy-based model (b) second-player of negotiation with a smaller stretch in fuzzybased model.

stretch reduction depends on the assigned β stretch , that can be Boulware, Conceder or Linear. As can be seen in Fig. 1, in our model, the fuzzy value consists of a peak value and a stretch (support). For generating the fuzzy offers, the agent is initiated with a Ps value (scaled in percentage measure) that shows the starting stretch of the negotiation, and a Pf value indicating the ending stretch of negotiation. The formulas below expresses this more precisely:

this work we only focus on time dependent method for making concessions. We leave investigation on other factors for future research. The time dependent tactic is based on how far an agent is from its deadline for ending the negotiation [5]. Equations 1 and 2 show the formulas for offer generation when the score value is respectively decreasing and increasing. n xtA→B = minA + αA (n) · (maxA − minA )

(1)

n = minA + (1 − αA (n)) · (maxA − minA ) xtA→B

(2)

t

stretch An+1 = Ps · (maxA − minA )+

where, αA (n) = kA + (1 − kA ) · (

min(tn , DA ) β1 ) DA

[(Ps − Pf ) · (maxA − minA ) · (1 − αstretch (n))] (4) A (3) where,

Here, kA is a constant that, multiplied by the agent’s interval determines the agent’s first offer, and β determines the concavity of agent’s concession curve. The time dependent tactic is categorized into three types: In Boulware methods (0 < β < 1), the agent makes small concessions when there is plenty of time to the deadline. When it is close to the deadline, the agent makes big concessions towards the reservation value. In Conceder methods (β > 1), agent is very cooperative and concedes so quickly to the reservation value, then makes small concessions till negotiation is terminated. When (β = 1), agent makes concessions Linearly. Note that in fuzzy negotiation, the concession should take place on two factors: the peak value, and the stretch (or support) of the offer. In this work, we use time dependent tactics for both of them. We apply Eq. 1 and Eq. 2 to create the offers peaks. For stretch, we utilize Eq. 4 that reduces the support of the offer in each round. The rate of

αstretch (n) = e A

(1−

min(tn ,DA ) β stretch ) A lnkA DA

(5)

Utility and Gain. Agents’ intrinsic benefit from each offer is calcualted by agent’s scoring function. There are some special costs that the agent confronts while negotiating, such as the time the agent spends for negotiation, or the cost of communication and protocols. Thus, when calculating the overall gain of the agent for the agreed upon value, the cost parameter has to be considered too. We choose the cost to be the same as Faratin’s [5]. Cost(tn ) = tanh(|xA↔B | · τ )

(6)

Here, |xA↔B | is the length of negotiation thread between the two agents up to time tn . The function tanh() maps real numbers to [0, 1]. We compute the cost according to the number of rounds at the end of negotiation. The constant τ has been assigned to adjust the change rate of tanh().

517 523 525

Authorized licensed use limited to: SWINBURNE UNIV OF TECHNOLOGY. Downloaded on March 15,2010 at 04:57:32 EDT from IEEE Xplore. Restrictions apply.

Considering the cost of negotiation, the Utility is computed by Eq. 7: U (x ) = V (x) − Cost(tn )

The results show that when a player has a smaller stretch in comparison with the opponent, and the agents are not playing extremely Boulware or Conceder (i.e. are playing moderately), in most of the cases the player negotiating with fuzzy model gains better cost adjusted utility compared to its corresponding player in the crisp negotiation.

(7)

For fuzzy numbers, there is not a single value to calculate the score. Thus, we use an aggregation on V (x) for all of the values in the interval of the fuzzy offer. For a fuzzy offer i defined in the interval [a, b], for all xj ∈ [a, b] we have: V (offeri ) = max (min(1, V (xj ))) xj ∈[a,b]

4. Discussion Our experimental results suggest that, when an agent negotiates with a smaller stretch than the opponent, and it is the starter of the negotiation, it often comes to a significantly better achievement in the cost adjusted utility than its corresponding agent in crisp. The negotiation plot for a scenario when agents are playing with long deadlines and moderate manner is presented in Fig. 3(a) and 3(b). In these graphs, each of agents’ offers is represented by two values, the peak and the end. As these graphs suggest, the negotiation in fuzzy-based model 3(a) terminates in less overall time–with a better outcome for the starter–compared to the crisp negotiation in 3(b). Comparisons in the overall number of rounds confirms that for long deadlines the length of negotiation is always and for medium deadlines in %93 of the scenarios less than or equal to the corresponding crisp negotiation. In our preliminary experiments we noticed that for very short deadlines, the fuzzy-based model may not produce good results. This is due to the fact that, when negotiating with very tight deadlines, agents do not have time to improve their offers from non-precise values to more precise ones, while making proper concessions. Instead, they have to rapidly fulfill the negotiation and come to an agreement. We evaluated the negotiation model for both medium and long deadlines and noticed that for a more extreme concave concession curve, the outcome of fuzzy negotiation does not show much superiority over crisp. However, it does not show significantly worse results either. In Fig. 3(e), a fuzzy negotiation plot for extreme strategy is represented. Figure 3(f) shows the same plot (with identical environmental parameters) for crisp negotiation. The figures suggest that the negotiation outcomes are very similar for these two cases. When behaving extremely Boulware or Conceder, the agent’s offers are approaching the opponent’s offer either very slowly, or very quickly. Therefore, when agents’ offers overlap, the peaks are very close to each other. This makes the experimental results not showing much difference between models. Based on these results, we do not claim that our fuzzybased model always outperforms the standard crisp negotiation. However, we argue that the proposed fuzzy-based approach leads to a more flexible negotiation. It also addresses the problem of negotiating by offering qualitative values, that cannot be done by current methods.

(8)

Since, we use right angle triangular fuzzy numbers, the aggregation shown in Eq. 8 is in fact the possibility distribution of the fuzzy value and the score function.

3. Experiments and Evaluation To evaluate the negotiation methods, we run our experiments in various environments–with different parameters mentioned in section 3. We assign the parameters in our experiments to be minA = minB = 10, maxA = maxB = 40, and kA = kB = 0.1. We use DA = DB = 10 for short, DA = DB = 20 for medium, and DA = DB = 40 for long deadlines. The β values of time dependent behaviour are in the range {0.02,0.1,0.5,1,2,10,50} that includes all Boulware, Conceder and Linear scenarios. If a deal is not made, the accepted value and the intrinsic utility are considered to be zero. The τ in our experiments is set to 0.05.1 Experiments with the fuzzy model include a few additional parameters: the parameter β stretch for the stretch shrinking method, and the starting and finishing percentages of stretch (Ps ,Pf ). Note that in these experiments agents are assigned to have identical deadlines, and a linear decreasing method for the stretches. In order to investigate the impact of differences between stretches on negotiation, we deploy two stretch percentage pairs; (30%,15%) and (15%,10%). We use a monotonic score function to evaluate each offer as in Faratin et al. [5]. For brevity, we only provide the results for two scenarios both of long deadlines (we found the results for medium and long deadlines to be very similar). For evaluation, we compare the first and second players’ final gain out of negotiation and the number of negotiation rounds between fuzzy and crisp negotiations. In Fig. 2(a) and 2(b) the cost adjusted utilities of first and second players with deadlines of 40 are presented. The labels in the middle of the graphs represent the player’s β value, while the labels on the x axis show the opponent’s β value. 1 We adjust the cost factor τ in a manner that starting with the first round the cost is equal to zero, and at the end of negotiation the cost is one.

518 524 526

Authorized licensed use limited to: SWINBURNE UNIV OF TECHNOLOGY. Downloaded on March 15,2010 at 04:57:32 EDT from IEEE Xplore. Restrictions apply.

Negotiation Values Negotiation Values

40

a A

c

30 Buyer’s Peak Buyer’s End Seller’s Peak Seller’s End

20

e

Buyer’s Peak Buyer’s End Seller’s Peak Seller’s End

Buyer’s Peak Buyer’s End Seller’s Peak Seller’s End

10 40

b B

d B

30

f

Buyer’s Offer Seller’s Offer

Buyer’s Offer Seller’s Offer

Buyer’s Offer Seller’s Offer

20

10 0

10

20

30

40

0

10

Time

20

Time

30

40

0

10

20

30

40

Time

Figure 3. Negotiations between the buyer and the seller agents with deadlines of 40 (a)–(b) respectively, fuzzy and crisp negotiations,behaving moderately. Buyer plays with smaller stretch than seller in fuzzy (c)–(d) respectively, fuzzy and crisp negotiations where Seller plays with smaller stretch than buyer in fuzzy (e)–(f) respectively, fuzzy and crisp negotiations playing extremely Conceder.

5. Conclusions

[2] J. J. Buckley. The multiple judge, multiple-criteria ranking problem : A fuzzy set approach. Fuzzy Sets and Systems, pages 23–37, 1984. [3] J. Carbo and A. Ledezma. A machine learning based evaluation of a negotiation between agents involving fuzzy counteroffers. In Proc. of AWIC’03, pages 268–277, 2003. [4] J. Carbo, J. M. M. L´opez, and J. D. Muro. Reaching agreements through fuzzy counter-offers. In Proc. Int. Conf. on Web Engineering, pages 90–93, 2003. [5] P. Faratin, C. Sierra, and N. R. Jennings. Negotiation decision functions for autonomous agents. Int. J. Robotics and Autonomous Systems, 24(3-4):159–182, 1998. [6] S. S. Fatima, M. Wooldridge, and N. R. Jennings. Multi-issue negotiation under time constraints. In Proc. AAMAS’02, pages 143–150, 2002. [7] F. Herrera and E.Herrera-Viedma. Linguistic decision analysis: Steps for solving decision problems under linguistic information. Fuzzy Sets and Systems, 2000. [8] H. M. Lee. Group decision making using fuzzy sets theory for evaluating the rate of aggregative risk in software developement. Fuzzy Sets and Systems, pages 261–271, 1996. [9] J. S. Rosenschein and G. Zlotkin. Rules of Encounter. The MIT Press, 1994. [10] G. Weiss. Multiagent Systems: A modern approach to distributed artificial intelligence. MIT Press, Cambridge, Massachusetts, London, England, 2000. [11] L. Zadeh. Fuzzy sets. Information Control, 90:338–353, 1965. [12] L. Zadeh. From computing with numbers to computing with words – from manipulation of measurements to manipulation of perceptions. IEEE Trans. on Circuits and Systems, pages 105–119, 1999.

We have proposed a fuzzy-based model for negotiation with qualitative values that is a basis for linguistic negotiation. For fuzzy-based negotiation, we used a double protocol, that allows both fuzzy and crisp values to be proposed to the opponent. We have evaluated the outcomes of negotiations, with agents applying different negotiation tactics for generating their offers and counter offers. Our experimental results suggest that the fuzzy-based model often takes fewer number of rounds to finish the negotiation. Exchanging fuzzy values as offers leads to a more flexible negotiation. Intuitively, when agents play more flexibly, the risk of coming to a failure should be less. We have recently investigated the risk of conflict in negotiation for both fuzzy and crisp models. According to our preliminary experiments, using fuzzy-based negotiation model reduces the rate of conflict, and opens a promising direction for future research. In this work, we have used the simplest format to keep our model easy and to focus more on the concept of negotiating with qualitative values. Many open questions are left for future work. For instance, the transition rate of stretches of offers, the shape of fuzzy offers, the impact of one party’s stretch on the decision making of the other, and finally negotiating over predefined linguistic values, are some of the issues to be investigated.

References [1] G. Bordogna and G.Pasi. A fuzzy linguistic approach generalizing boolean information retrieval: A model and its evaluation. JASIS, pages 288–307, 1984.

519 525 527

Authorized licensed use limited to: SWINBURNE UNIV OF TECHNOLOGY. Downloaded on March 15,2010 at 04:57:32 EDT from IEEE Xplore. Restrictions apply.