A New Strategy for Solving Multiple-Choice

A New Strategy for Solving Multiple-Choice Multiple-Dimension Knapsack Problem in PRAM Model Md. Waselul Haque Sadid1, Md. Rabiul Islam1, S.M. Kamrul Hasan2 1,2

Department of Computer Science & Engineering, Rajshahi University of Engineering & Technology, Bangladesh 1{whsadid, Rabiul_cse}@yahoo.com, [email protected]

Abstract. This paper presents a new heuristic algorithm for the MultipleChoice Multi-Dimension Knapsack Problem (MMKP) in PRAM model. MMKP is a variant of the classical 0-1 knapsack problem, has a knapsack with multidimensional capacity constraints, groups of items, each item having a utility value and multidimensional resource constraints. The problem is to maximize the total value of the items in the knapsack with the constraint of not exceeding the knapsack capacity. Since MMKP is an NP-Hard problem, its exact solution is not suitable for real time problems, so heuristic based approximation algorithms are developed. We present a parallel heuristic algorithm here that runs in O (log nl (log n + log m + log log l )) time in CRCW PRAM machine; taking O ( n log n (log n + ml )) operations. Experimental results show that it achieves 95% optimal solution on average. This also means that we have a sequential heuristic running in O ( n log n (log n + ml )) time which seems to be remarkable since M-HEU, a celebrated sequential heuristic although achieves

2 2

96% of optimal value, takes O ( mn l ) running time.

1 Introduction Multiple-Choice Multi-Dimension Knapsack Problem (MMKP) [5] is a variant of the classical 0-1 KP, where there are different groups of items and exactly one item can be picked up from each group; and the constraints are multi-dimensional. Let there be n groups of items, Group i containing li items, Item j of Group i has a utility value vij G and an m dimensional resource cost rij = ( rij1 , rij 2 , " , rijm ) . The resource constraint G R = ( R1 , R 2 , " , Rm ) is also m dimensional. The MMKP problem is to maximize the V = ∑in=1 ∑lji =1 xij v ij subject to the resource constraint utility value ∑in=1 ∑lji =1 xij rijk ≤ R k where 1 ≤ k ≤ m , xij ∈ {0, 1} and ∑lji =1 xij = 1 . Fig. 1 illustrates an MMKP with 3 groups and 2 resource requirements. Values and resource

requirements of each item are shown inside the boxes representing items. Objective is to pick up exactly one item from each group to maximize the total value of picked items maintaining ∑ r1 of picked items ≤ 17 and ∑ r2 of picked items ≤ 15. Item

V=10 r1=5, r2=7

V=15 r1=6, r2=3

V=12 r1=7, r2=7

V=14 r1=4, r2=7

V=11 r1=8, r2=2

V=7 r1=5, r2=3

V=9 r1=5, r2=5

V=13 r1=6, r2=4

V=17 r1=10, r2=8

Group 1

Group 2

Group 3

Maximum allowable Resource Type r1: 17 Type r2: 15

Knapsack

Fig. 1. Example of an MMKP

The motivations of our research are basically twofold and are described below. Firstly, the QoS management problem or equivalently the admission control problem in an adaptive Multimedia Server Systems can be easily mapped to an MMKP [5]. However, since the MMKP is an NP-Hard problem [7] its exact solution may not be feasible in any real time problem like the admission control problem in an Adaptive Multimedia Server system. Therefore, heuristic based solutions are used. This is why tremendous amount of research has been performed on various heuristic based solutions in this regard. Secondly, since the processor speed has reached almost the saturation state, multi-processor machines are becoming more and more popular. Single processor computing servers requiring huge CPU resources can be replaced by multiprocessor machines in order to provide the huge computing resources required. Admission control algorithms can achieve good performance using the above idea. We here present a parallel heuristic that runs in CRCW PRAM machine taking O (log nl (log n + log m + log log l )) parallel time and O ( n log n(log n + ml )) operations achieving 95% of optimal value.

2 Related Works There are different algorithms for solving variants of the classical 0-1 KP. The MultiDimension Knapsack Problem (MDKP) is one kind of KP where the resources are multidimensional i.e. there are multiple resource constraints for the knapsack. The MCKP is another KP where there are one or more groups of items and exactly one item will be picked from each group. MMKP is the combination of MDKP and MCKP. A greedy approach has been proposed [5] to find near optimal solutions of the knapsack problems. Applying the greedy method to the MDKP, Toyoda [9] proposed a new measurement called aggregate resource consumption. Khan [5] has applied the

concept of aggregate resource consumption to pick a new item in a group to solve the MMKP. His heuristic HEU selects the lowest-valued items by utility or revenue of each group as initial solution. It then upgrades the solution by choosing a new item from a group, which has the highest positive change in aggregate consumed resource (the item which gives the best revenue with the least aggregate resource). If no such item is found then an item with the maximum value gain per unit aggregate resource expended is chosen. A pick where the infeasibility factor of each resource is less than or equal to 1 is called a feasible solution of the knapsack problem. The sum of the values of the items picked in a feasible solution is called the value of the solution. Thus the solution of the MMKP is the feasible solution, which maximizes the sum of the value of the selected items [1]. Moser’s [8] heuristic algorithm uses the concept of graceful degradation from the most valuable items based on Lagrange’s Multipliers to solve the MMKP. This algorithm is also suitable for real time application of MMKP problems. Later, M-HEU, a heuristic developed by Akbar et al.[1], modifies HEU by adding a pre-processing step to find a feasible solution if the initial solution is not feasible because there may be a solution using higher-valued items that requires fewer resources. It also uses a post-processing step to improve the total value of the solution found by HEU with one upgrade followed by one or more downgrade. This is because there might be some higher-valued items in the MMKP, selection of which make the solution infeasible, but if some of the other groups are downgraded a feasible solution may be obtained. Knapsack problems also have been subject to much work on the development of efficient parallel algorithms [2]. For exact methods Branch-and-Bound and Dynamic programming (DP) are the most useful approaches. A DP algorithm for 0-1 KP may run in O (nc log( p ) / p + n log p ) for p processors, where c is the capacity of the knapsack [2]. A pipeline-architecture containing a linear array of p processors has a time complexity O(nc / p + m ) [2]. A divide and conquer approximation algorithm with a 2 time complexity O log (n ) log(c ) on nc c processors is presented in. Another approximation algorithm realized on the hyperbolic architecture with a time complex2 2 ity O log (n ) log(c ) on O nc processors [2].

(

(

) ( )

)

( )

3 Parallel Heuristic Algorithm for MMKP In this section we present the PRAM heuristic algorithm for solving MMKP. The underlying idea to find a near-optimal solution is essentially the same as in the sequential heuristics HEU or M-HEU: we start with an initial feasible solution and try to reach a near-optimal (if not optimal) solution by upgrading the solution iteratively. Without the loss of generality we assume that each group of the MMKP has one (dummy) item with zero value and zero constraint in all dimensions. The initial solution is constructed using these dummy items. Items of each group in the MMKP are sorted in non-decreasing order according to the value associated with them. First the relative change in aggregate consumed resource,

Δarij of each item is calculated in parallel for each group. Δarij = {v,b }is a vector, where r Δaij =

( (

)

⎧⎪ Δvij Δaij ,0 if Δaij ≤ 0 ⎨ if Δaij > 0 ⎪⎩ Δaij ,1

)

And Δarij>Δarkl , if b(Δarij)>b(Δarkl) or (b(Δarij)=b(Δarkl) and v(Δarij)>v(Δarkl)) Where, the change of aggregate resource consumption, Δaij = ∑ riρ [i ]k − rijk × C k k and the change of revenue, Δvij = viρ [i ] − vij .

(

)

C k = ∑ riρ [i ]k , ρ[i] is the currently selected item in group i. Then we find the items with the highest Δarij from each group in parallel. The items found in the previous step, are sorted in non-decreasing order with respect to the value of their relative change in aggregate consumed resource using pipelined merge sort algorithm on CREW PRAM model. We want to upgrade our current solution by replacing nl/2h items in Iteration h with larger valued items without violating the constraints. We call the number of items we want to replace as ‘scheduled upgrade’. Essentially scheduled upgrade = nl/2h. If scheduled upgrade is greater than n (i.e. the number of groups), we cannot take them all and hence we have to consider only n items in such cases. We call the total number of items we tentatively calculate for possible upgrade as ‘tentative upgrade’ and the difference between tentative upgrade and scheduled upgrade is defined pending upgrade. To increase the performance, we add the pending upgrade with the scheduled upgrade and it is the minimum of this summation and n, which is ultimately considered for possible upgrade. So, in effect, in every iteration we have tentative upgrade = min (pending upgrade + scheduled upgrade, n). Then the prefix sum is computed of the resource consumptions of the tentative upgrade items using logarithmic time prefix computation algorithm. The selected tentative upgrade items may not give the feasible solution due to the resource constraints. If these items do not give the feasible solution, then we find out the maximum number of items for which the solution will be feasible and these items will be upgraded. The whole procedure runs on Concurrent Read Concurrent Write Parallel Random Access Machine (CRCW PRAM). 3.1 Algorithm

Step 1: In each group add one dummy item with value 0 and constraints 0. Construct the initial solution with the lowest valued item (i.e. dummy item) in each group. Step 2: In Iteration h (0 ≤ h ≤ log nl), execute the following sub-steps. Step 2.1: For all group, relative aggregate resource consumption, Δarij of each item having a higher utility value than the item selected from the same group are computed in parallel. Step 2.2: Find the item with highest Δarij in each group using parallel maximum finding algorithm.

Step 2.3: Sort the items found in Step 2.2 in descending order of Δarij using parallel sorting algorithm. Step 2.4: Compute prefix sums of consumed resources for the first min (pending upgrade + nl/2h, n) items found in Step 2.3 Step 2.5: Using parallel maximum finding algorithm, find the maximum number of items that can be upgraded at a time in this iteration without violating the constraint and then perform the upgrade. Step 3: Deliver the solution if there is no dummy item in the solution. If there is a dummy item in the final solution, it implies that ‘no solution is found’. Also the introduction of a dummy item can be used as a special case of MMKP where the restriction of picking exactly one item from each group is relaxed and it indicates that no item will be taken from the corresponding group. 3.2 Complexity Analysis

For the convenience of the analysis we assume all the groups have the same number of items i.e. l1 = l 2 = l 3 = """ = l n = l . Step 1 and Step 3 can be performed easily in O(1) parallel time, O(n) operations on EREW PRAM. However, Step 2 with a number of nontrivial sub-steps needs some analysis. In Step 2.1, m additions, m multiplications and m subtractions are needed. The additions can be done in parallel in log m time using O(m) operations. For the subtraction, m processors are directly employed and it can be done in O(1) time using O(m) operations. m multiplications can also be done in O(1) time using O(m) processors, i.e. total number of operations is O(n). Then the results of m subtractions and m multiplications should be added and it can be added in parallel. Similarly, to calculate the value of Ck, we need n additions, which can be done in parallel in log n time spending O(n) operations. We have to calculate the relative change of aggregate resource consumption, Δarij for each item (l items) from each group (n groups). So in total Step 2.1 runs in O(log m + log n) time using O(lnm + nm) = O(lnm) operations on CREW PRAM. Step 2.2 directly employs the parallel maximum finding algorithm on l items in each group. Since there are n groups this can be done in O(loglog l) time with O(nl) operations on CRCW PRAM. In Step 2.3 we apply the parallel sorting algorithm on n elements which can be done in O(log n) time, O(nlog n) operations on CREW PRAM. In Step 2.4 we compute prefix sums of consumed resources for tentative_upgrade elements. Computing prefix sum on k elements in parallel takes O(log k) time using O(k) operations in EREW PRAM. Since the resources are m dimensional we can do the prefix sum separately on different dimensions in O(lg(tentative_upgrade)) time using O(m × tentative_upgrade) operations. However, to construct the Boolean array indicating whether resource constraints are met or not we have to check all the dimensions in parallel. We can do the job in O(1) parallel time using O(tentative_upgrade × m) operations as follows. We employ one processor each for each of the resource

dimensions for each of the tentative_upgrade elements. Each processor checks the assigned dimension against the corresponding available resource dimension. A group of m processors, corresponding to the m resource dimension of a particular element writes to particular entry of the array. The array is initialized with “Yes”. If any check of any processor turns out to be negative, it just writes “No” in the corresponding entry (please see the example shown in Fig 2 for a demonstration of building the array). So the PRAM model needed is CRCW. Since maximum value of tentative_upgrade is n, Step 2.4 can be performed in O(log n) time using O(mn) operations in CRCW PRAM. Boolean Array 1 Yes

Array of Indices 1 1

2

Yes

2

2

3

No

3

-1

4

Yes

4

4

5

No

5

-1

6

Yes

6

6

7

Yes

7

7

8

No

8

-1

Maximum of the elements in Array of indices will indicate actual_upgrade

Fig. 2. Finding actual number of upgrades

In Step 2.5, we find the maximum number of items, i.e., the value of actual_upgrade for which the solution is feasible. In this step we can employ the parallel maximum finding algorithm directly and it will take O(loglog n) time with O(n) operations on CRCW PRAM, since maximum value of tentative_upgrade can be in the worst case n. Finally performing the upgrade can be done in O(1) parallel time using O(n) operations. Since the Step 2 iterates for lognl time, the overall running time of Step 2 should be O (log nl (log n + log m + log log l )) parallel time. The number of operations needed is O(nlognl(logn+lm)). It is easy to see that the running time and the total operations of the algorithm remain those of Step 2 and the PRAM model required is CRCW. 3.3 Experimental Results

PRAM machines are not available in reality. So we have simulated the above algorithm as if it runs on a CRCW PRAM. For each set of parameters n, l and m, we used 10 randomly generated MMKP instances, which are not correlated with resource consumptions and 10 MMKP instances, which are correlated with resource consumption (see [11] for benchmark data sets on the MMKP). For the same data we have also executed the M-HEU algorithm, which is sequential algorithm and achieves 96% of optimal value and the upper bound, a bound which is equal to or higher than the optimal value of the objective function of the MMKP. Our solution is then benchmarked

Achieved Optimality(%) i

with the result of upper bound as well as M-HEU and we find that it is on an average 98.5% of the value of M-HEU, and about 95% of the upper bound. 1.02 1 0.98 0.96 0.94 0.92 0.9 0.88

Correlated M-HEU Uncorrelated M-HEU Correlated Parallel HEU 0

200

400

600

800

1000

Number of Groups

Uncorrelated Parallel HEU

Achieved Optimality(%) i

Fig. 3. Performance of Parallel HEU and M-HEU normalized with the upper bound for the MMKP data sets with l=10 and m=10 1.02 1

Correlated M-HEU

0.98 0.96 0.94

Uncorrelated M-HEU Correlated Parallel HEU

0.92 0.9 0

10

20

30

40

50

60

Number of Resource Constraints

Uncorrelated Parallel HEU

Achieved Optimality(%) i

Fig. 4. Performance of Parallel HEU and M-HEU normalized with the upper bound for the MMKP data sets with n=500 and l=10 1.02 1 0.98 0.96 0.94 0.92 0.9 0.88

Correlated M-HEU Uncorrelated M-HEU Correlated Parallel HEU 0

20

40

Number of Items

60

Uncorrelated Parallel HEU

Fig. 5. Performance of Parallel HEU and M-HEU normalized with the upper bound for the MMKP data sets with n=500 and m=10

4 Conclusion Heuristic HEU, developed by Khan, achieves 93% of the optimal value for an MMKP. M-HEU, a modification of HEU by Akbar achieves 96% of the optimal 2 2 value. Both of them are sequential algorithms and run in O ( mn l ) time. We have presented a parallel heuristic that runs on CRCW PRAM in O (log nl (log n + log m + log log l )) time using O ( n log n(log n + ml )) operations. It is worth-mentioning that we, in effect, have presented a sequential heuristic for an MMKP that runs in O ( n log n(log n + ml )) time, far better than M-HEU or HEU, and achieves 95% of the optimal value, which is slightly worse than M-HEU. In our parallel algorithm, we have used the same candidate item evaluation criteria as used in HEU by Khan or M-HEU by Akbar. However, we have considered only upgrading of the solution as in hill-climbing approach and HEU. As a result, there might be some cases where we may get stuck in a local maximum. To recover, we have to downgrade our solution and then upgrade later on as in simulated annealing technique. For future improvement we may think of adopting this. There is no parallel algorithm for exact solution of MMKP. We may work on that further.

References 1.

M. M. Akbar, Eric G. Manning, Gholamali C. Shoja, S. Khan, Heuristic Solutions for the Multiple-Choice Multi-Dimension Knapsack Problem, International Conference on Computational Science, San Fracsisco, California, (2001) 659-668. 2. V. N. Alexandrov, G. M. Megson, Parallel Algorithms for Knapsack type problems, World Scientific Publishing Co. Pte. Ltd., (1999). 3. A. H. Newton, W. H. Sadid, M. M. Akbar, A Parallel Heuristic Algorithm for MultipleChoice Multi-Dimensional Knapsack Problem, 6th International Conference on Computer and Information Technology, Bangladesh, (2003). 4. B. Ahmed. A Muqim, W. H. Sadid, A Heuristic Solution of The Multiple-Choice Multiple-Dimension Knapsack Problem by Constructing Convex Hull in Parallel Method, ICCSA-2004, San Diego, California, ( 2004)105-109. 5. S. Khan, K. F. Li, E. G. Manning, M. M. Akbar, Solving the Knapsack Problem for Adaptive Multimedia Systems, Studia Informatica Universalis, Vol. 2(1), (2002)161-181. 6. M. Magazine and O. Oguz, A Heuristic Algorithm for Multidimensional Zero-One Knapsack Problem, European Journal of Operational Research, Vol. 16(3), (1984)319-326. 7. S. Martello and P. Toth, Algorithms for Knapsack Problems, Annals of Discrete Mathematics, Vol. 31, (1987)70-79. 8. M. Moser, D. P. Jokanovic and N. Shiratori, An Algorithm for the Multidimensional Multiple-Choice Knapsack Problem, IEICE Transactions on Fundamentals of Electronics, Vol. 80(3), (1997)582-589. 9. Y. Toyoda, A Simplified Algorithm for Obtaining Approximate Solution to Zero-one Programming Problems. Management Science, Vol. 21, (1975)1417-1427. 10. Joseph Jaja, AnIntroduction to Parallel Algorithms, Addision-Wesley Publishing Company Inc., (1992). 11. ftp://panoramix.univ-paris.fr/pub/CERMSEM/hifi/MMKP.