A Multi-Agent based Hyper-Heuristic Algorithm for the ... - ScienceDirect

Available online at www.sciencedirect.com Available online at www.sciencedirect.com

Available online at www.sciencedirect.com Available online at www.sciencedirect.com

ScienceDirect

Procedia Computer Science 00 (2017) 000–000 Procedia Computer Science (2017) 000–000 Procedia Computer Science 11200 (2017) 117–126 Procedia Computer Science 00 (2017) 000–000

www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia

International Conference on Knowledge Based and Intelligent Information and Engineering Systems, on KES2017, 6-8 September 2017, Marseille, Franceand Engineering International Conference Knowledge Based and Intelligent Information International Conference Knowledge Based and Intelligent Information Systems, on KES2017, 6-8 September 2017, Marseille, Franceand Engineering A Multi-Agent based Hyper-Heuristic Algorithm for the Winner Systems, KES2017, 6-8 September 2017, Marseille, France

A Multi-Agent basedDetermination Hyper-Heuristic Algorithm for the Winner Problem A Multi-Agent based Hyper-Heuristic Algorithm for the Winner Determination Problem a,∗ a Ines Sghir , Ines Ben JaafarProblem , Khaled Ghédiraa Determination a,∗ a a Ines Sghira,∗ENSI, , Ines Bene de Jaafar , Khaled Ghédira Universit´ la Manouba, 2010, Manouba, Tunisie a a Ines Sghir ENSI, , Ines Ben Jaafar , Khaled Gh´ e dira Université de la Manouba, 2010, Manouba, Tunisie

a SSOIE/COSMOS, a SSOIE/COSMOS, a SSOIE/COSMOS,

ENSI, Université de la Manouba, 2010, Manouba, Tunisie

Abstract Abstract In this paper, we propose a Multi-Agent based Hyper-Heuristic algorithm for the Winner Determination Problem named MAH 2 2 Abstract WDP. This algorithm explores a set of cooperating agents to select the appropriate operation using learning techniques. In this paper, we propose a Multi-Agent based Hyper-Heuristic algorithm for the Winner Determination Problem named MAH MAH 2 -WDP is specialized for local search methods and evolutionary methods where the following agents are seeking to improve the WDP. This algorithm explores a set of cooperating agents to select the appropriate operation using learning techniques. In this paper, we propose a Multi-Agent based Hyper-Heuristic algorithm for the Winner Determination Problem named MAH 2 search within reinforcement the mediator agent, local search agents, perturbation andtechniques. two to recombination 2 WDP specialized for locallearning: search methods and evolutionary methods where thethe following agentsagent are seeking improve WDP. is This algorithm explores a set of cooperating agents two to select the appropriate operation using learning MAHthe agents. Our computational study shows that the proposed algorithm performs well on the tested benchmark instances in terms of search reinforcement the mediator agent, two local search agents, perturbation and two to recombination WDP iswithin specialized for locallearning: search methods and evolutionary methods where thethe following agentsagent are seeking improve the solution quality. agents. Our computational showsthe that the proposed performs well on tested benchmark in terms of search within reinforcementstudy learning: mediator agent, algorithm two local search agents, the the perturbation agent andinstances two recombination Keywords: Multi-agent; study Winner Determination Problem; hyper-heuristic; diversification; solution quality. agents. Our computational shows that the proposed algorithm performsintensification; well on the tested benchmarkmetaheuristics. instances in terms of c 2017 The Authors. Published by Elsevier B.V. Keywords: solution quality.Multi-agent; Winner Determination Problem; hyper-heuristic; intensification; diversification; metaheuristics. Peer-review responsibility of KES International. c Keywords: 2017 The under Authors. Published by Elsevier B.V. Problem; hyper-heuristic; intensification; diversification; metaheuristics. Multi-agent; Winner Determination © Published B.V. c 2017 Peer-review under responsibility of Elsevier KES International. 2017 The TheAuthors. Authors. Publishedby by Elsevier B.V. Peer-review under responsibility of KES International Peer-review under responsibility of KES International.

1. Introduction 1. Introduction auction is an auctioneer who wishes to maximize his/her selling revenue and a set of bidders who wishes to 1. The Introduction minimize their cost. the wishes most widely known auctions are therevenue Englishand auction, Holland’s auction, The auction is an Examples auctioneerofwho to maximize his/her selling a set the of bidders who wishesthe to 1 Sealed and theofVickrey auction . known These auctions treat only a single item forthe sell. who Combinatorial The envelope auction isauction an Examples auctioneer who to maximize his/her selling revenue and a set ofeach bidders wishesthe to minimize their cost. the wishes most widely auctions are the English auction, Holland’s auction, 2 1 known . auction, auctionsenvelope are multi-item auctions, which allow bids on a combination items minimize their cost. Examples the most widely auctionstreat areofthe English Holland’s auction, the Sealed auction and theofVickrey auction . These auctions only a single item forthe each sell. Combinatorial 1 2single item In Combinatorial auctions, a set of items arebids exposed to buyerstreat who offer bids. bid sell. defined by a subset Sealed envelope auction and the Vickrey auction . These auctions forAeach Combinatorial . auctions are multi-item auctions, which allow on a combination ofonly itemsadifferent 2 of items and its price.auctions, Ifauctions, two bids share at least one item, they declared conflicting bids. winner determination . auctions are multi-item which allow on a combination of as items In Combinatorial a set of items arebids exposed to buyers who offer different bids. The A bid defined by a subset problem (WDP) is theauctions, problem conflict-free of items which maximizes auctioneer’s In Combinatorial aofsetfinding of items areone exposed to buyers who offer different bids. The Athe bid defined by revenue. a subset of items and its price. If two bids share ataleast item,allocation they declared as conflicting bids. winner determination This revenue is price. the sum of the of the winning bids. The WDP is known to beThe athe NP-hard combinatorial of items and its If two bids share ataleast one item,allocation they declared as conflicting bids. winner determination problem (WDP) is the problem ofvaluations finding conflict-free of items which maximizes auctioneer’s revenue. optimization problem with number of practical likeThe e-commerce, games theory and resources allocation problem (WDP) is the problem ofvaluations finding a conflict-free allocation of items maximizes auctioneer’s revenue. This revenue is the sum ofathe of theapplications winning bids. WDPwhich is known to be athe NP-hard combinatorial 3,2 in multi-agent .ofathe This revenue problem issystems the sum valuations of theapplications winning bids. WDP is known be aand NP-hard combinatorial optimization with number of practical likeThe e-commerce, games to theory resources allocation 3,2of amnumber Formally, problem given a set items M = {1, 2, ..., m} to sell and set of n bidsgames N = {1, 2, ...n}. bid j allocation is a tuple optimization with of practical applications likeae-commerce, theory andEach resources in multi-agent systems . 3,2 < S , P > where S is a subset of items covered by bid j, and P is the price of bid j. Let B be a m × n binary j j in multi-agent systems . m items M = {1, 2, ..., m} to sell and ja set of n bids N = {1, 2, ...n}. Each bid j is amatrix Formally, given aj set of tuple = 1 if object im∈ items Sof BiM 0{1, otherwise. Furthermore, weofprice define variable for bid j such that B i j given j , items j ==covered seta of 2, ...,bym}bid to sell and n bids N decision =j.{1, jeach is amatrix tuple < SFormally, ,j P j > where S aj is subset j, and P jaisset the ofthe bid Let2,B...n}. be aEach m ×x njbid binary < S j ,that P j >Bwhere is a subset by bidFurthermore, j, and P j is the biddecision j. Let Bvariable be a m ×x nj for binary if jobject i ∈ Sof Bi j = covered 0 otherwise. we price defineofthe eachmatrix bid j such ij = 1 S j , items = 1 if object i ∈ S , B = 0 otherwise. Furthermore, we define the decision variable x for each bid j such that B i j j i j j ∗

Corresponding author. [email protected] ∗ c 2017 The Authors. Published by Elsevier B.V. 1877-0509 ∗ Corresponding author. [email protected] Peer-review under responsibility of KES International. ∗ c 2017author. 1877-0509 The Authors. Published by Elsevier B.V. Corresponding [email protected] Peer-review KES International. ∗ cunder 1877-0509 2017responsibility The Authors.of Published by Elsevier B.V. 1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of KES International. Peer-review under responsibility of KES International

10.1016/j.procs.2017.08.183

Ines Sghir et al. / Procedia Computer Science 112 (2017) 117–126 I.Sghir et al. / Procedia Computer Science 00 (2017) 000–000

118 2

such that x j = 1, if bid j is a winning bid, 0 otherwise. The WDP can be formulated using the following binary integer optimization problem. Maximize f (x) =

Pjxj

(1)

j∈N

subject to j∈N

Bi j x j ≤ 1, i ∈ M

(2)

The objective function (1) is to maximize the auctioneer’s gain calculated by the sum of prices of the winning bids. The constraints (2) ensure that an item appears at most in one winning bid. Several algorithms were proposed to solve the winner determination problem. These algorithms can be divided into two categories: the exact algorithms and the stochastic algorithms. For the exact algorithms, we can quote: Branch-on-Bids (BoB) 4 , Combinatorial Auction Structural Search (CASS) 5 , Combinatorial Auctions Multi-unites Search (CAMUS) 6 . Nisan 7 proposed a linear programming algorithm for the WDP. Some stochastic algorithms were proposed for the WDP: Casanova 8 was the first local search algorithm proposed by Hoos and Boutilier. In 9 , Guo et al. proposed the SAGII algorithm which is a simulated annealing combined with the Branch-and-Bound algorithm. The local search (SLS) proposed by Boughaci et al. 10 starts with a possible initial allocation and tries to improve it, by searching for a better solution in the current neighborhood. The tabu search (TS) for the WDP was elaborated by Boughaci et al. 10 . Boughaci et al. 11 proposed a memetic algorithm (MA) for the WDP. In 12 , Sghir et al. presented a dedicated tabu search algorithm (TSX WDP). In 13 , Wu and Hao developed an algorithm for the WDP by recasting the WDP into the maximum weight clique problem (MWCP). An effective discrete dynamic convexized method (DCM) was proposed by Lin et al. 14 . This method uses an adaptive penalty function to convert the winner determination problem into an equivalent unconstrained integer programming problem. These approaches aim at moving the optimization process from one local optimum to another. However, they do not have an efficient way to know when activating the suitable techniques according to the search process. Therefore, we can assume that existing methods do not integrate an intelligent mechanism to control the resolution process. Recently, the computational cost having been reduced almost dramatically, researchers all over the world are coming up with new opportunities to achieve better solutions for combinatorial optimization problems. Recent advancements in optimization area introduced new methods to overcome drawbacks and to improve the existing optimization techniques, especially in integrating of techniques that lead to artificial intelligence such as multi-agent systems. In this paper, a hyper heuristic algorithm called a Multi-Agent based Hyper-Heuristic algorithm, MAH 2 -WDP, is proposed for solving the winner determination problem. A hyper-heuristic is an approach that, given a particular problem instance and a number of heuristics, can select and apply an appropriate heuristic at each decision point. Otherwise, Hyper-heuristics are (meta-) heuristics that can operate on (meta-) heuristics. A multi-agent system (MAS) is a system composed of agents who interact, most of the time, according to cooperation and competition modes. It can be used as a liaison between different metaheuristics for solving optimization problems. The advantages of using MAS are to refine and combine the solutions built by the metaheuristics. In MAH 2 -WDP, the cooperation of agents is based on the reinforcement learning in order to know if the search needs exploration or exploitation. In the next section, we will present the proposed algorithm. Section 3 will contain the experimentations of MAH 2 -WDP using the WDP benchmarks. Finally, section 4 concludes this work. 2. A Multi-Agent based Hyper-Heuristic algorithm for the Winner Determination Problem ( MAH2 -WDP) MAH 2 -WDP is based on multi-agent system and combines some features of several other well-established metaheuristics including tabu search, variable neighborhood search and evolutionary methods. The used agents are the mediator agent, two local search agents, the perturbation agent and two recombination agents. Fig. 1 describes the structure of the search. Below, we present the behaviors of each agent.


119 3

3a-Fitness of each Local search agent is the fitness of the local best allocation generated. 2 Local Search Agents

1-Fitness of the Mediator agent is the fitness of the best global solution it discovered.

Local Search Agent 1

Perturbation Agent

one Allocation

One_Bid_neighbor1 2 Allocations

one Allocation Local Search Agent 2

All_Bids_neighbor2

2 Allocations

Reduced_Perturbation Strong_Perturbation 4a-Activate the perturbation agent based on the reinforcement learning.

Mediator Agent

The common archive

................................................................................... 2 Recombination Agents Recombination Agent 1

2-Select between the Local search agents and the Recombination agents based on the reinforcement learning

one Allocation

Crossover_Operator1 Recombination Agent 2

3b-Fitness of each Recombination agent is the fitness of the local offspring obtained.

Crossover_Operator2

Fig. 1. The structure of the search using cooperative agents in MAH 2 -WDP

2.1. Mediator agent The mediator agent generates a simple non conflicting allocation by selecting random items. The mediator agent uses learning technique (see section 2.5) which helps him to decide which agents to activate between recombination agents and local search agents. It maintains all high-quality solutions, received from other agents, in a shared memory.

2.2. Local search agents Two local search agents are responsible for the intensification search of MAH 2 -WDP. During their search, these agents can exchange with another alive local search agent or with a perturbation agent based on learning technique (see section 2.5). Local search agents send the best allocation found to the mediator agent. A candidate solution is an allocation A (a dynamic vector) which contains the winning bid. Each bid is composed of the list of items and the associated prices. The first local search agent applies the neighborhood strategy proposed by Boughaci et al. 10 . In each iteration of the considered local search, the best neighbor is selected. To produce neighbor solutions, two moves are built in the following way: • The search space is composed of the items which are not covered by the bids in the current allocation. The best bid covering such items is chosen. All incompatible bids in the current allocation are removed; • After generating all neighbor configurations, the best configuration, which maximizes the revenue of the current allocation, is selected to be candidate allocation. To escape the visited allocations, a list maintains the bids recently selected. The second local search agent uses the neighborhood strategy presented in other work 12 by performing the following neighborhood strategy:


120 4

• Sort the initial unsatisfied bids according to their prices; • For each unsatisfied bid Bx , verify if it can increase the revenue of the current allocation when it is inserted based on a binary gain function (expression 3); Gain(Bx ) = f (A) − f (Q) + f (Bx )

(3)

• The gain function returns true if f (Q) < f (Bx ) and returns false otherwise, such as Q is the set of winning bids that are in conflict with the current unsatisfied bid Bx , f (Q) is the revenue of the set of winning bids Q, and f (Bx ) is the price of the unsatisfied bid Bx . • An unsatisfied bid Bx can enter in the current allocation only if its price f (Bx ) is higher than the revenue of other winning bids which are conflicting with Bx in the current allocation (i.e., the gain function is true); • When a bid Bx is inserted in the current allocation A, the bids of Q which are conflicting with Bx are removed from A; • The steps mentioned previously are iterated until all the initial unsatisfied bids are visited and possibly added in the current allocation A. A tabu list is explored to forbid recently visited solutions from being revisited. When a bid is inserted in the current allocation A, it is forbidden to be removed for the next tt iterations . This tabu tenure is fixed dynamically by the function proposed in 15 : tt = L + + f (A) where L is randomly chosen from the interval [0, 9] and is empirically fixed to 0.6. 2.3. Perturbation agent The perturbation agent is activated by a local search agent. This agent creates a new perturbed solution that manages the search towards other regions. It performs two parallel behaviors which are reduced perturbation behavior and strong perturbation behavior. The resulting solution is sent to the local search agent. The reduced perturbation technique is activated when the local search agent observes a slight search stagnation. The perturbation agent selects randomly one candidate unsatisfied bid from the available ones. Then, the selected bid is added in the allocation received from the local search agent. All the contradictory bids are removed from this allocation. The strong perturbation technique is applied, when the local search agent observes a strong search stagnation. Based on the archive of elite solutions, the perturbation agent extracts the number of occurrence of each bid appeared in high-quality allocations. Then, the bids, which have the smallest occurrence number, are inserted in the current non conflicting allocation. In order to create a new solution in each call of the perturbation agent, data structures are employed. 2.4. Recombination agents Two recombination agents are activated, when the mediator agent observes a local optimum reached. These two agents apply crossover operations to produce new offspring allocations. The first recombination agent uses the crossover operator which was proposed by Boughaci et al. 11 . This agent selects C individuals from the current population P to participate in the reproduction phase. C contains the best individuals C1 , which have the highest fitness, and the diverse individuals C2 , which are the individuals the most diverse in the population P. The diversity is measured using a similarity function which calculates the number of the common bids between two individuals. Two parents are selected randomly from C. They are combined to generate a new individual. From the first parent to the end of the second parent, this agent decides which parent will give all its bids to the offspring by discarding all conflicting bids. The second recombination agent explores the recombination operator which is described in 12 . This operator aims to transform the good properties of the parents towards the offspring. These criteria have to assure that the offspring inherits the properties of the parents. Given two parents allocations from the common archive. These parents share the highest number of bids. The second crossover agent constructs the offspring in k steps until all the bids of the two parents are visited. This operator is inspired by the idea of backbone used in 16 . In the first step, the set of bids, that are shared by the parents, are identified and directly transmitted to the offspring. Then the following steps are performed:


• • • •

121 5

Pick up the bid having the lowest price from each parent. From the two selected bids, insert in the offspring the best bids which have the highest total revenue. Remove the selected bids from their parents, even if they are not inserted in the offspring. Repeat the previous steps until all the bids of the parents are examined and removed.

The two allocations, generated by the recombination agents, are sent to the mediator agent. They will be the new current allocation for the search process.

2.5. Agents communication In MAH 2 -WDP, the selection of agents to trigger is made dynamically based on decision matrices whose values are adjusted using the reinforcement learning. The values of these decision matrices are generated by a decision process to select an operator according to the optimization states. The states cover significant situations that may occur in the search process. The set of operators are the agents to trigger. We use the same mechanism presented in our previous work 17 to decide which operators to active. In Fig. 2, we present the sequence diagram of the communication between the mediator agent, the recombination agents and the local search agents.

Mediator Agent S=Search_Initial_Solution

loop

(Stopping condition not reached) Operator_Type=Select_Oprerator _Type using reinforcement learning

alt

[Operator_Type=Local search Agents]

Local search Agents

Activate Local search Agents & send S

S_best_LS=Search_Best_Allocation(S) (Section 2.2)

send S_best_LS

alt

[Operator_Type=Recombination Agents] Activate Recombination Agents

Recombination Agents S_cr=Search_crossover (Section 2.4)

Send S_cr

alt

[S_cr!=null Or S_best_LS!=null] S_best=best_solutions between S AND (S_cr OR S_best_LS) S=best_solutions between solutions S_cr OR S_best_LS

Up_Date_decision_matrix based on the improvement of S_best

Up_Date_Archive(S_cr Or S_best_LS)

Fig. 2. Sequence diagram of the communication between mediator agent, local search agents and recombination agents


122 6

For the mediator agent, we consider four states and two operators. When the search process needs to be intensified, two states are considered. The first state is that the algorithm does not reach m0 generations. The second state is that the local or global best solution is improved in recent m1 generations and this improvement is a large gain improvement. In these cases, local search agents operator is rewarded by increasing the related weights in the decision matrix. When the search process requires diversification search, two other states are considered which are: the improvement of the local or global best solution is small improvement in recent m1 generations, or there is no improvement in recent m2 generations. Notice that m0 , m1 and m2 are parameters set by the user according to the total generation number. In these cases, the weights of activating recombination agents operator are rewarded. The mediator agent waits the solutions from the activating agents and finishes the search if the best solution is found. The solutions received are maintained in an archive. Fig. 3 presents the sequence diagram of the communication between the local search agents and the perturbation agent.

Local Search Agent 2

Local Search Agent 1 loop

(iteration q4 ), the weights related to the perturbation agent with the strong perturbation behavior, are increased. q3 , q4 and q5 are parameters that represent the total run time fixed by experimentations. When the perturbation agent is triggered by local search agents, the perturbation agent creates a disturbed solution. This solution is then sent to the requesting local search agent for further improvement. At the end of each local search agent run, the best solution found by each agent is sent to the mediator agent. 3. Experimentations This section describes experimental results of MAH 2 -WDP on the set of well-known WDP benchmarks. MAH 2 WDP was implemented in Java using the Jade platform. The program was run on a computer with a Core I5 2.5GHz, 8GB of RAM. Tests were applied on various benchmarks of diverse sizes defined in 18 . These benchmarks take into account several factors like the prices, bidders preferences and object distribution on bids. They can be divided into five groups where each group contains 100 instances: • • • • •

REL 500-1000: From in101 to in200: m = 500, n = 1000 REL 1000-1000: From in201 to in300: m = 1000, n = 1000 REL 1000-500: From in401 to in 500: m = 1000, n = 500 REL 1000-1500: From in501 to in 600: m = 1000, n = 1500 REL 1500-1500: From in601 to in 700: m = 1500, n = 1500

The parameters of MAH 2 -WDP are adjusted by an experimental study. The number of iterations for each local search agent (iter max) was fixed to 500. The parameters max opt (for mediator agent) and max opt T S (for local search agents), which evaluate the improvement of solutions between generations, were fixed to 20 and 25 respectively. As interval, we considered the same value 1000 for the same agents. The rate µ used in updating the weight matrices was fixed to 0.9. Tables 1 and 2 present the computational results of MAH 2 -WDP on the set of the five groups of benchmarks. Given we have 500 instances, we show only some results of each group. Columns provide the following computational statistics of each tested instance: the maximum revenue obtained by the MAH 2 -WDP algorithm over the 10 independent trials Rbest and the average CPU time in seconds AvgT ime. For comparison purposes, these tables also include the solution quality (Rbest) and CPU time (AvgT ime) obtained by MA 11 , TSX WDP 12 , MN/TS 13 and DCM 14 . The data of these algorithms are taken from their references directly. Tables 1 and 2 show that the results obtained by our MAH 2 -WDP are competitive with respect to many State of the art algorithms in terms of solution quality. It can reach previous best known results for many instances for which very few algorithms are able to attain the best known results. Table 3 illustrates the general comparative results for each group. We show the comparative study of MAH 2 -WDP with other algorithms from the literature: Casanova 8 , SAGII 9 , SLS 10 , TS 10 , MA 11 , TSX WDP 12 , MN/TS 13 and DCM 14 . µ is the average of best objective value of the 100 instances in each group. T ime is the average time to reach the best solution. δ(%) is the deviation of the MAH 2 -WDP algorithm with respect to each reference algorithm. The deviations are calculated respectively by the following equation: δ = (µ MAH 2 −WDP − µalgo X )/µ MAH 2 −WDP

(4)

where algo X is one of the eight reference algorithms. Since the compared algorithms are implemented in different languages and run in different computer, the comparison is focused on solution quality that can be reached by each algorithm. The computing time is provided only for indicative purposes. The results of the reference algorithms


124 8

are extracted from the corresponding papers expect the results of Casanova which are given by 9 . Table 3 shows that MAH 2 -WDP gives an improvement between 32% and 48% in solution quality compared to Casanova in shorter time. MAH 2 -WDP outperforms all reference algorithms expect MN/TS: TS (the improvement rate is between 4% and 11%), SLS (the improvement rate is between 4% and 10%), MA (the improvement rate is between 2% and 9% ), SAGII (the deviation is between 2% and 8 %), TSX WDP (the deviation is between 0.3% and 3%) and DCM (the improvement rate is between 1% and 4%). The results of MAH 2 -WDP are close to the results of MN/TS. The deviation is between -2% and 0%. Table 1. Some comparative results obtained by MAH 2 -WDP with other algorithms on REL 500-1000 and REL 1000-1000 instances for benchmarks Instance in101 in102 in103 in104 in105 in106 in201 in202 in206 in207 in208 in209

MAH 2 -WDP Rbest 69585.298 72518.22 72129.50 72709.64 75646.127 71258.61 81557.74 90537.285 87544.451 93129.24 91774.549 86441.696

AvgTime 88 65 74 76 79 86 66 61 64 68 57 59

MA Rbest 71728.600 67797.61 66350.99 64618.41 66376.83 65481.64 77499.82 90464.19 86881.42 91033.51 83667.76 81966.65

AvgTime 40.457 132.18 133.34 135.14 153.96 140.96 98.26 106.68 93.99 100.90 101.29 96.42

TSX WDP Rbest 69585.298 72518.222 69730.618 71327.641 73351.044 66361.943 81557.742 89289.573 87544.451 93115.569 91774.549 86441.696

AvgTime 88 76 71 74 93 73 56 52 56 57 56 54

MN/TS Rbest 72724.61 72518.22 72129.50 72709.64 75646.127 71258.61 81557.74 90708.127 91518.96 93129.24 94904.67 87268.96

AvgTime 5.46 19.91 1.94 7.33 1.70 9.05 9.45 2.47 0.75 11.23 4.24 11.25

DCM Rbest 71728.600 72391.300 69730.618 72709.64 75646.127 67203.885 81324.258 90708.127 87544.451 87544.451 88249.675 86755.797

AvgTime 40.457 43.670 45.381 44.597 41.339 42.818 70.342 79.424 72.403 72.403 75.510 73.263

Table 2. Some comparative results obtained by MAH 2 -WDP with other algorithms on REL 1000-500, REL 1000-1500 and REL 1500-1500 instances for benchmarks Instance in401 in402 in403 in408 in409 in410 in501 in502 in503 in504 in601 in602 in603 in604

MAH 2 -WDP Rbest 77417.482 76273.33 74843.958 77018.833 73188.62 73791.658 88656.95 86236.911 83718.749 85600.002 107823.098 99718.150 105121.022 103568.86

AvgTime 10 11 10 10 15 18 112 95 93 84 120 85 82 85

MA Rbest 72948.07 71454.78 74843.958 72580.04 68724.53 71791.57 79132.03 80340.76 83277.71 81903.02 99044.32 98164.23 94126.96 103568.86

AvgTime 37.07 37.20 38.81 39.07 15 41.9 107.82 108.71 114.15 116.11 110.62 114.18 110.71 110.60

TSX WDP Rbest 77417.482 76273.336 74843.958 77018.833 70035.52 73628.485 83738.040 83297.340 83718.749 83944.901 107246.248 99668.269 98577.454 101713.602

AvgTime 12 10 10 10 9 10 107 82 81 76 117 87 75 78

MN/TS Rbest 77417.482 76273.33 74843.958 77018.833 73188.62 73791.658 88656.95 86236.911 87812.377 85600.002 108800.44 105611.476 105121.022 107733.80

AvgTime 0.16 0.38 3.02 0.16 0.38 0.51 1.47 1.76 19.63 4.62 9.12 1.72 1.21 16.62

DCM Rbest 77417.482 76273.33 74843.958 77018.833 73188.62 73791.658 88656.95 86236.911 87812.377 85600.002 105561.129 105611.476 105121.022 105950.193

AvgTime 26.59 24.514 25.696 27.551 25.617 25.486 149.284 128.050 137.571 139.613 154.690 144.276 137.196 138.490

4. conclusion In this paper, we proposed a Hyper-Heuristic algorithm for the Winner Determination Problem ( MAH 2 -WDP). The local search agents are responsible for the intensification search. The perturbation agent trigged by local search agents aims at escaping the current local optimum. The recombination agents employ crossover operators as another tool for the diversification of the search space. The new descendant solution can change the direction of the search since it is the new starting point for other iterations of the tabu search. The proposed MAH 2 -WDP algorithm is evaluated on a set of 500 benchmark instances. The comparative study with reference algorithms shows that it is able to reach solution of very high quality. As perspectives, we can improve the technique used for the reinforcement learning. References 1. Klemperer, P. (2004) Auctions: Theory and Practice Princeton, N.J.: Princeton University Press. 2. Jawad, A., Teodor, GC., Michel, G., & Monia, R. Combinatorial auctions. Annals of Operational Research; 2007. 153(1), 131–164.

Table 3. Comparative results between MAH 2 -WDP, Casanova, MA, SLS, TS, SAGII, TSX WDP, MN/TS and DCM on WDP benchmarks: rows µ correspond to the average of the best objective value of the 100 instances in each group. Columns time represent the average time to reach the best solution. Test Set 100 instances REL-500-1000 REL-1000-500 REL-1000-1000 REL-1000-1500 REL-1500-1500 MAH 2 -WDP Time 70 10 53 85 101 µ 70215.711 75540.68 87.292.848 87041.037 106093.955 Casanova Time 119.46 57.74 111.42 168.24 165.92 µ 37053.78 51248.79 51990.91 56406.74 65661.03 δ MAH 2 −WDP/Casanova (%) 47.22 32.15 40.44 35.19 38.11 TS Time 91,07 25.84 104,30 223,37 175.68 µ 65286.94 71985.34 81633.63 77931.41 97824.64 δ MAH 2 −WDP/T S (%) 7.01 4.7 6.48 10.46 7.79 SLS Time 22.35 5.91 14.19 14.97 16.47 µ 64216.14 72206.07 82120.31 79065.08 98877.07 δ MAH 2 −WDP/S LS (%) 8.54 4.61 5.92 9.16 6.8 MA Time 56.64 14.98 33.05 24.51 28.22 µ 65740.25 73604.62 83304.20 79644.64 99957.96 δ MAH 2 −WDP/MA (%) 6.37 2.56 4.56 8.49 5.78 SAGII Time 38.06 24.46 45.37 68.82 91.78 µ 64922.02 73922.10 83728.34 82651.49 101739.64 δ MAH 2 −WDP/S AGII (%) 7.53 2.14 4.08 5.04 4.1 TSX WDP Time 74.19 9.45 48.98 75.92 90.61 µ 69647.975 75274.184 86786.159 85577.806 103178.732 δ MAH 2 −WDP/T S X WDP (%) 0.8 0.3 0.58 1.68 2.74 MN/TS Time 12.28 0.38 3.12 6.39 2.64 µ 71470.93 75540.68 89158.98 89552.18 108627.17 δ MAH 2 −WDP/MN/T S (%) -1.75 0 -2.09 -2.8 -2.33 DCM Time 43.403 29.275 70.148 135.176 147.991 µ 67768.547 74097.638 85528.073 85361.810 103821.182 δ MAH 2 −WDP/DCM (%) 3.48 1.91 2.02 1.92 2.14

Ines Sghir et al. / Procedia Computer Science 112 (2017) 117–126 I.Sghir et al. / Procedia Computer Science 00 (2017) 000–000 125 9

126 10


3. Vries de S,Vohra, R. Combinatorial auctions a survey. INFORMS Journal on Computing; 2003. 15, 284–309. 4. Sandholm, T., & Suri, S. Improved optimal algorithm for combinatorial auctions and generalizations. In: Proceedings of the 17th National Conference on Artificial Intelligence; 2000. 90–97. 5. Fujishima, Y., Leyton-Brown, K., & Shoham, Y. Taming the computational complexity of combinatorial auctions: optimal and approximate approaches. In: 16th International Joint Conference on Artificial Intelligence. Sweden; 1999. 48–53. 6. Leyton-Brown, K., Shoham, Y., & Tennenholtz, M. An algorithm for multi-unit combinatorial auctions. In: Proceedings of the 17th National Conference on Artificial Intelligence, Atlanta; 2000. 7. Nisan, N. Bidding and allocation in combinatorial auctions. In: Proceedings of ACM Conference on Electronic Commerce (EC’00). ACM Press. Minneapolis; 2000. 1–12. 8. Hoos, HH., & Boutilier, C. Solving combinatorial using stochastic local search. In: Proceedings of the 17th National Conference on Artificial Intelligence; 2000. 22–29. 9. Guo, Y., Lim, A., Rodrigues, B., & Zhu, Y. Heuristics for a bidding problem. Computers and Operations Research; 2006. 33(8), 2179–2188. 10. Boughaci, D., Belaid, B., & Habiba, D. A memetic algorithm for the optimal winner determination problem. Soft Computing; 2009.13(8-9), 905-917. 11. Boughaci, D., Belaid, B., & Habiba, D. Local Search Methods for the Optimal Winner Determination Problem in Combinatorial Auctions. Journal of Mathematical Modelling and Algorithms in Operations Research; 2010. 9(2), 165–180. 12. Sghir, I., Hao, J.K., ben Jaafar, I., & Ghédira, K. A Recombination-Based Tabu Search Algorithm for the Winner Determination Problem. a` P.Legrand et al. (Eds), AE 2013, LNCS. Bordeaux; 2013. 8752, 157–168. 13. Wu, Q., & Hao, J.K. Solving the winner determination problem via a weighted maximum clique heuristic. Expert Systems with Applications; 2015.42(1), 355–365. 14. Lin, G., Zhu, W., & Ali, M. M. An effective discrete dynamic convexized method for solving the winner determination problem. Journal of Combinatorial Optimization; 2016. 32(2), 563–593. 15. Galinier, P., & Hao, J.K. Hybrid evolutionary algorithms for graph coloring. J. Comb Optim.; 1999. 3(4), 379–397. 16. Benlic, U., & Hao, J.K. A multilevel memetic approach for improving graph k-partitions. IEEE Transactions on Evolutionary Computation; 2011. 15(5), 624–472. 17. Sghir, I., Hao, J.K., ben Jaafar, I.,& Ghédira, K. A Multi-Agent Based Optimization Method applied for the Quadratic Assignment Problem. Expert Systems With Applications Journal; 2015. 18. Lau, H.C., & Goh, Y.G. An intelligent brokering system to support multi-agent web-based for thparty logistics. In: Proceedings of the 14th International Conference on Tools with Artificial Intelligence. Washington, DC; 2002. 54-61.