Institute of Business Informatics, University of Muenster,. Germany .... After a predefined time interval, all local solutions are sent to the master. The master then ...
Improving Sequencing Algorithms Based on Hopfield Nets by More Efficient Problem Representation Prof. Dr. Karl Kurbel, Dipl.-Inform. Bernd Schneider Institute of Business Informatics, University of Muenster, Germany
Abstract This paper describes an efficient algorithm adaptable to several types of sequencing problems. In the first section, three such problems are outlined briefly: the travelling-salesman problem, VLSI cell placement, and job scheduling. An intuitive and straightforward solution using Hopfield nets is presented. When the underlying structure of sequencing problems and the way they are attacked in Hopfield nets is examined in detail, a simpler problem representation and a more efficient algorithm can be formulated. Although the new algorithm does not lend itself to parallelization in a straightforward way, we give an outlook how parallel computing facilities can be exploited to obtain good solutions faster.
Three Types of Sequencing Problems There are basically three kinds of sequencing problems that have been discussed in the literature: travelling-salesman problems [1], VLSI cell placement [2], and job scheduling [3]. The travelling-salesman problem (TSP) is characterized by a number of places, e.g. cities, which all have to be visited. Each place is connected with each other place. The distances between places are known. The problem is to find a route that satisfies the following constraints: Each place is visited exactly once, and the route connecting the places is as short as possible. The VLSI cell-placement problem is slightly different. When a set of VLSI cells and the number of connections between cells is given, the problem is to assign the cells to raster blocks on the chip. The goal is here to find those placements which minimize the overall length of connections between cells. In job scheduling, the term job stands for production order. In the general case, production orders require several operations to be performed in a specific sequence (such as drilling, grinding, painting, etc.). The scheduling problem is to assign a number of operations belonging to different jobs to a limited number of manufacturing resources (such as machines). The result of the assignment task is called a schedule. An objective function is used to measure schedule quality with respect to some goal, e.g. minimize total elapsed time or maximize capacity utilization.
Intuitive Solution Using Hopfield Nets In this chapter, we outline briefly how each of the three problems described above can be mapped to a Hopfield net [4]. When a travelling-salesman problem with n places to be visited is to be mapped, a net with n x n neurons is required. This leads to an n x n-matrix structure as shown in figure 1. Each cell of the matrix stands for one neuron.
1 2 3 4 5 6 7 8 a b c d e f g h Fig. 1: Mapping a sequencing problem to a Hopfield net Rows represent the places to visit, while columns stand for the order in which places will be visited during the tour. For example, a "1" - represented by a symbol - in row one and column two means that place "a" will be visited as the second one of the tour. Besides calculating the weights representing distances and the sums representing the optimization goal, the algorithm basically makes exchanges regarding the order in which the places are visited on the route. However, the simulated-annealing algorithm does not guarantee that the solution found is consistent. Therefore, additional consistency checks are necessary. Mapping VLSI cell-placement to a Hopfield net is very similar. n cells to be placed on n raster blocks also lead to n2 neurons arranged in an n x n matrix. Rows now represent VLSI cells and columns represents the order in which VLSI cells are put into the raster blocks. In figure 1, for example, a "1" in row two and column three means that cell "b" will be mapped onto the third raster block of the chip. Optimizing assignments of VLSI cells to raster blocks means exchanging ones in the matrix in a way that the energy function comes to a minimum. As in the travellingsalesman problem, consistency is not guaranteed; thus additional consistency checks have to be done. For the job-scheduling problem, the rows of figure 1 can be interpreted as operations to be performed on a particular machine, whereas columns stand for sequence numbers. A "1" in row two and column five would indicate that operation "e" will be scheduled as the third one. The production process (sequence of operations) corresponding to figure 1 is depicted in figure 2.
c a e g h b d f order Fig. 2: Sequence of operations in the production process The figure assumes that machines are treated separately (i.e. operations are assigned to particular machines by a production planning system or by employing some heuristic rule), and that there are no alternative machines for these operations (i.e. no machine groups). If these premises are removed, the Hopfield net becomes more complex. It will be represented by a three-dimensional matrix where a "1" at position (i, j, k) stands for an operation i to be scheduled as the k-th one and to be manufactured on machine j. As we have shown, the three sequencing problems can be represented in the same way. Furthermore, they all have in common that additional checks are necessary to ensure consistency (in TSP) or even feasibility (in cell placement and scheduling). Empirical studies show that it takes more CPU time to do those checks for each solution than it takes to generate a new solution.
A More Efficient Representation and Algorithm The three problems have in common that the solution process can be viewed as finding an order of the respective problem-dependent objects. Another - non-binary - representation of an order is a list. Transforming the matrix of figure 1 into a list gives a sequence as shown in figure 3.
1 2 3 4 5 6 7 8 a b c d e f g h
HopfieldNetwork
c a e g h b d f
sequence (list)
Fig. 3: Mapping a binary matrix to a sequence For the problems outlined above, it is sufficient to initialize the sequence with a trivial solution and, in analogy to flips in a Hopfield net, to move elements of the list to other positions of the list. Since the move operation only turns one consistent solution into another consistent solution, the final one will also be consistent. Consistency is maintained because items of the list can neither be doubled nor removed. Therefore, a consistent solution needs to be generated only once. If the process is initiated with a trivial but consistent solution, no more time-consuming consistency checks are necessary. Likewise, all the penalty terms can be avoided as well, because the matrix representing the Hopfield net will have exactly one "1" in each row and column. The energy function can easily be adapted to the new algorithm. An extension has to be made for real-world job-scheduling problems. When many machines have to be considered at the same time because production orders require several operations to be performed on different machines in a particular sequence, a more intelligent move operator is needed to create consistent solutions only. The permissible range for a particular move is no longer the whole list but only a subset. Its lower bound is the index of the preceding operation of the same job, or the beginning of the list, respectively. Its upper bound is the index of the successor operation of the same job, or the end of the list, respectively. In figure 4, for example, operation 2 may be arbitrarily shifted within the range limited by operations 1 and 3. permissible range for moving operation 2 permissible range for moving operation 1
permissible range for moving operation 3
... op. 1
op. 2
op. 3
op. 4
start of list
end of list
operations belonging to the same job operations belonging to other jobs Fig. 4: Ranges for the move operator Generating an initial consistent solution can be done in many ways. For example, start with all operations of the first job (production order) in the prescribed order, then append all operations of the second job, etc. An alternative
initialization procedure would take all first operations of all jobs, then append all second operations of all jobs, and so on. Our experience with the revised algorithm based on a list has shown that run-time efficiency is by orders of magnitude better than for the initial implementation based on a Hopfield net.
Outlook: Parallelization Inherent parallelism is often considered a general advantage of neural nets. Our algorithm based on a list does not lend itself to parallelization in a natural way. Nevertheless, performance may be improved if it is run on a parallel computer. Instead of executing one algorithm that is parallelized, a set of identical algorithms can be run simultaneously. Communication and cooperation among the corresponding processes would be as follows: On each processor, one optimizer as described above is started. Each one begins with different external parameters (such as the temperature and the annealing schedule). For example, starting with a high temperature in combination with slow cooling gives good results, but it takes a long time. On the other hand, quick cooling leads to poorer results, but it is much faster. After a predefined time interval, all local solutions are sent to the master. The master then decides which solution is the most promising one to start optimization from in the next phase. Local solutions may also be interpreted as individuals of a genetic algorithm. The master performs the genetic crossover operation and generates new individuals by combining two solutions. The resulting population will then be distributed to the local optimizers for the next round. Different initialization, communication, and migration strategies can be employed. Development and evaluation of such strategies will be subject to further research by the authors.
References [1] Aarts, E.H.L., Korst, H.M.: Boltzmann Machines and their Applications; in: Parallel Architectures and Languages Europe, Eindhoven NL, 1987, pp. 34-50. [2] Singh, K., George, M.S., Rambabu, P.: Recurrent Networks for Standard Cell Placement; in: Balagurusamy, E., Sushila, B. (Eds.): International Computer Conference, Hyderabad, India, 1993, pp. 141-148. [3] Kurbel, K.E.: Production Scheduling in a Leitstand System Using a Neural-net Approach; in: Balagurusamy, E., Sushila, B. (Eds.): International Computer Conference, Hyderabad, India, 1993, pp. 297-305. [4] Gee, A.H., Aiyer, S.V.B., Prager, R.W.: Neural Networks and Combinatorial Optimization Problems - The Key to a Successful Mapping; Technical Report 77, Cambridge University Engineering Department, Cambridge 1991.
