Proposal of Multi-Agent based Model for Dynamic Scheduling in ...

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp193-198)

Proposal of Multi-Agent based Model for Dynamic Scheduling in Manufacturing ANA MADUREIRA

JOAQUIM SANTOS

Computer Science Department Institute of Engineering - Polytechnic of Porto GECAD – Knowledge Engineering and Decision Support Research Group Porto, Portugal http://www.dei.isep.ipp.pt/~anamadur/

Abstract: - Some recent trends in manufacturing in particular and business in general, lead to new approaches regarding the organisation and software architecture, mainly adopting distributed solutions. Such organisations imply organisational and technological evolution through agility, distribution, decentralisation, reactivity and flexibility. New organisational and technological paradigms are needed in order to reply to the modern manufacturing systems challenges. The Multi-Agent paradigm represents one of the most promising approaches to build complex, flexible, and cost-effective scheduling systems because of its distributed and dynamic nature. Modelling the Scheduling of Manufacturing Systems by means of two technologies like Meta-Heuristics and Multi-Agent Systems seems to be an interesting way to see Industrial Systems in the future. A multi-agent based model for support dynamic scheduling in manufacturing environments is proposed.

Key-Words: - Multi-Agent Systems, Dynamic and Distributed Scheduling, Meta-Heuristics, Manufacturing

1 Introduction During the last years it was felt a need to deal with new challenges on Manufacturing Systems. Market globalization and E-business trends are making it critical for manufacturers to integrate sales-order entry and production scheduling. Current global and highly competitive market, enterprises must be aware of momentary market opportunities, and quickly and properly react to customers' demands. These directions include the reduction of order's dimension; increasing product's variety and complexity; client's participation in the development process, providing individual solutions oriented to customer’s specific needs, the reduction of product's life cycles; and concurrent execution of manufacturing process activities. Real world scheduling requirements are related with complex systems operated in dynamic environments. This means that they are frequently subject to several kinds of random occurrences and perturbations, such as new job arrivals, machine breakdowns, employee’s sickness, jobs cancellation and due date and time processing changes, causing prepared schedules becoming easily outdated and unsuitable. Scheduling under this environment is known as dynamic.

Traditional scheduling methods, encounter great difficulties when they are applied to some real-world situations. Several attempts have been made to modify algorithms, to tune them for optimization in a changing environment. A major challenge in the area of global market economy is to develop new techniques for solving real world scheduling problems. Indeed, any industrial organization can only be economically visible by maximizing customer services, maintaining efficient, low cost operations and minimizing total investment. The dynamic scheduling problem is a distributed problem, from logic and structural point of view. In a structural view of the manufacturing systems, the system involves several resources (machines, CNC’s, AGV’s , conveyors, robots) and several tasks can be carried out at the same time. From a logic point of view this kind of system can be compared to a distributed problem, due to needs of tasks simultaneous processing. Since the manufacturing process is intrinsically dynamic, it is impossible to know the exact structure or topology of the system in advance. The number of products and orders, as well as different alternative production routes, account for the highly complex nature of manufacturing systems. All of the above

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp193-198)

make the scheduling of manufacturing systems an excellent candidate for the application of agent-based technology. Thus, the requirements for today's and future computer-supported manufacturing systems suggest autonomy, distribution, decentralization and flexibility, while stressing the need for coordination among production units. It is expected that rigid, static and centralized manufacturing systems will give way to systems that are more adaptable to rapid change. The Multi-Agent paradigm represents one of the most promising approaches to building complex, flexible, and cost-effective scheduling systems because of its distributed and dynamic nature [6][9]. This paper will describe and discuss the application of Multi-Agent Systems (MAS) to the resolution of dynamic and distributed scheduling problems. The remaining sections are organized as follows: Section 2 summarizes some related work and the research on the use of multi-agent technology for dynamic scheduling resolution. In section 3 the scheduling problem under consideration ( Extended Job-Shop Scheduling Problem) is described. Section 4, presents some terms and definitions related with MAS context, and presents the Multi-Agent Model for Dynamic Manufacturing Scheduling. Finally, the paper presents some conclusions and puts forward some ideas for future work.

2 Related Work During the last decades, the organization of mass production has been evolving towards flexible manufacturing and customized products. From a technological point of view, it has been observed that current manufacturing systems have several drawbacks, in particular excessive rigidity and centralization. Furthermore, future manufacturing systems are expected to be characterized by globally distributed production units, small quantities of a large variety of products, the provision of individual solutions tailored to each customer’s specific needs, and concurrent execution of all the activities in the manufacturing process [14]. In the case of scheduling systems we need to operate through the cooperation of many interacting subsystems, each with its own objective, and modes of operation. Multi-Agent approach appears to be well suited to complex scheduling problems, especially those with a good deal of interaction between components, and for which static methods cannot provide efficient results. One important issue in the manufacturing research area is that the schedule itself is only valid until the first disturbance

(new job arrivals, machine breakdowns, employees sickness, jobs cancellation and due date and time processing change, etc.). Since manufacturing control and execution is a real time application, the need to find a feasible solution, incorporating the recent perturbations, is much greater than to find a optimal solution. The system as a whole must reach a stable and feasible schedule without too much interruption of the shop floor. The problem of dynamic scheduling is one that is receiving increasing attention amongst both researchers and practitioners. In spite of all the previous trials the scheduling problem still known to be NP-complete. This fact incites researchers to explore new directions. Multi-Agent technology has been considered as an important approach for developing industrial distributed systems. Considering the complexity inherent to the manufacturing systems, the dynamic scheduling is considered an excellent candidate for the application of agent-based technology. Ramos [3] proposed a structure in which task agents and resource agents are coordinated by a task manager and a resource manager, respectively. Shaw et al. [9] were one of the first to propose the use of agents in manufacturing for scheduling and factory control. Yams has presented is another of the earliest agent based manufacturing system which assigns an agent to each node in a control hierarchy [4]. Rabelo et al. [12], have proposed a multiagent autonomous architecture for dynamic scheduling in FMS where resources are represented by agents. The resource agents are responsible for scheduling the resources and they have no control over each other. Henseler [5] proposed the distribution of the schedule among several independently running agents: order agents and machine agents. Sousa [11] proposed a holonic architecture for scheduling in manufacturing systems in which tasks and resources are represented by holons and used the CNP for scheduling/rescheduling of tasks. Many multi-agent based approaches have been developed. For further works developed on MAS for dynamic scheduling, see for example, [4][5][6][7][10].

3 Extended Job-Shop Problem Definition

Scheduling

Most real-world multi-operation scheduling problems can be described as dynamic and extended versions of the classic or basic Job-Shop scheduling combinatorial optimization problem. The general

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp193-198)

Job-Shop Scheduling Problem (JSSP) of size n x m can generally be described as a decision-making process concerning about the allocation of a limited set of m={0, 1, …,m} resources over time to perform a set of n={0, 1, …, n} tasks or jobs. Most real-world multi-operation scheduling problems can be described as dynamic and extended versions of the classic or basic Job-Shop scheduling combinatorial optimization problem. In this work we consider several extensions and additional constraints to the classic JSSP, namely: the existence of different job release dates; the existence of different job due dates; the possibility of job priorities; a machine could process more than one operation in the same job (recirculation); the existence of alternative machines; precedence constraints among operations of the different jobs are common because, most of the times, mainly in discrete manufacturing, products are made of several components that can be seen as different jobs whose manufacturing must be coordinated; the existence of operations on the same job, on different parts and components, processed simultaneously on different machines, followed by components assembly operations, which characterizes Extended Job-Shop Scheduling Problem (EJSSP)[1][2]. Therefore, in this work, we define a job as a manufacturing order for a final item that could be Simple or Complex. It may be simple, like a part, requiring a set of operations to be processed. We call it a Simple Product or Simple Final Item. Complex Final Items, requiring processing of several operations on a number of parts followed by assembly operations at several stages, are also dealt with. Moreover, in practice, scheduling environment tend to be dynamic, i.e. new jobs arrive at unpredictable intervals, machines breakdown, jobs are cancelled and due dates and processing times change frequently.

4 Multi-Agent based Scheduling System Considering that centralized scheduling is therefore large, complex, and difficult to maintain and reconfigure. On the other hand, the inherent nature of much industrial and service process is distributed we will try solving the complex dynamic scheduling problems in a distributed way using the Multi-Agent paradigm. Modeling the Scheduling of Manufacturing Systems by means of two technologies like MetaHeuristics and Multi-Agent Systems seems to be an interesting way to see Industrial Systems in the future.

In Multi-Agent System for Dynamic Manufacturing Scheduling using Meta-Heuristics (MADynScheMH) System we will model a Manufacturing Systems by means of a Multi-Agent Systems, where each agent may represent a processing entity (e.g. a machine). The system has as objective to deal with the complex problem of Dynamic Scheduling in Manufacturing Systems. We want to prove that a good global solution for a scheduling problem may emerge from a community of machine agents solving locally their schedules and cooperating with other machine agents that share some relations between the operations/jobs (e.g. a precedence relation). Meta-Heuristics (Tabu Search or Genetic Algorithms) can be adapted to deal with dynamic problems, reusing and changing solutions/populations in accordance with the dynamism of the Manufacturing System. The self-parameterization of the Meta-Heuristics allows a better adaptation to the situation being considered. The idea is that each agent adopt and provides the self-parameterization in accordance with the problem being solved (the method and parameters can change in run-time). Meta-Heuristics (e.g. Tabu Search and Genetic Algorithms) are adequate for static problems, however real scheduling problems are quite dynamic (new orders arriving, being cancelled, machine delays or faults, etc.).

4.1 MAS Terms and Definitions The definition of the term Agent has not common consent. In the last few years most authors agreed that this definition depends on the domain where agents are used. In Ferber [6] is proposed a definition: “An agent is a virtual or physical autonomous entity which performs a given task using information gleaned from its environment to act in a suitable manner so as to complete the task successfully. The agent should be able to adapt itself based on changes occurring in its environment, so that a change in circumstances will still yield the intended result." An agent is generally characterized by the following properties [6] [8][10]: ‰ Autonomy - agents can work without a constant external intervention. They are able to take decisions by their own and to maintain control about their actions and internal state. ‰ Social ability - agents interact or communicate with other agents, in order to obtain better solutions for their problems. ‰ Reactivity - agents have the capability of acquire information and knowledge from their environment. In presence of this kind of

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp193-198)

information they can respond to it in order to improve is own performance. ‰ Pro-activity - agents can have their own objectives. In order to accomplish them they are able to adapt their own behavior to take the initiative. Agents are capable of handle heavy and complex tasks. All the important decisions about how and where to perform their own tasks must be made by the agent ‰ Temporal continuity: agents are continuously running processes. ‰ Mobility: an agent has the ability to transport itself from one computer to another, retaining its current state. A Multi-Agent System (MAS) can be defined as “a system composed by population of autonomous agents, which cooperate with each other to reach common objectives, while simultaneously each agent pursues individual objectives" [6]. The main advantages of a Multi-Agent system are the abilities of coordination and cooperation in order to accomplish a common objective. We can see MAS like a society of agents that cooperates to work in the best way possible. With this we gain the ability of solve complex problems like dynamic and distributed scheduling. Considering the complexity inherent to the manufacturing systems, the dynamic scheduling is considered an excellent candidate for the application of agent-based technology. In many implementations of multi-agent systems for manufacturing scheduling, the agents model the resources of the system and the tasks scheduling is done in a distributed way by means of cooperation and coordination amongst agents [8]. There are also approaches that use a single agent for scheduling (centralized scheduling algorithm) that defines the schedules that the resource agents will execute [4][8][11]. When responding to disturbances, the distributed nature of multi-agent systems can also be a benefit to the rescheduling algorithm by involving only the agents directly affected, without disturbance to the rest of the community that can continue with their work.

4.2 Multi-Agent based Scheduling System The agent technology can be considered an important approach when we are trying to solve dynamic and distributed scheduling systems. Multi-Agent systems can easily respond to scheduling disruptions, adapting the entire plan in order to turn them feasible and with the capacity of respond to the organization needs. This kind of approach is able of generate good global solutions, because they operate like an organized society,

where each one knows what must do. Their capabilities of change the behavior in order to take the initiative of perform something without any human intervention; allow the creation of intelligent autonomous systems that are able of conducting their own activities to fulfill their directives and objectives. As we can see in the Figure 1, the use of MultiAgent systems to solve dynamic and distributed scheduling, permits to obtain a cooperative society of agents which purpose is searching for the best schedule.

Figure 1 – Proposed Multi-Agent Model

As objectives of the Multi-Agent System for Dynamic Manufacturing Scheduling using MetaHeuristics (MADynScheMH) we identify the following assertions: ‰ Manufacturing Systems are well modeled by means of Multi-Agent Systems, where each agent may represent a processing entity in the Manufacturing System (e.g. a machine) ‰ A global solution for a scheduling problem may emerge from a community of machine agents solving locally their schedules and cooperating with other machine agents that shares some relations between the operations/jobs (e.g. a precedence relation) ‰ Meta-Heuristics can be adapted to deal with dynamic problems, reusing and adapting individuals/populations in accordance with the dynamism of the Manufacturing System ‰ The self parameterization of the Meta-Heuristics will allow a better adaptation to the situation being considered. The main purpose is to create a Multi-Agent system where each agent represents a resource (machine) in a Manufacturing System (Machine Agents). Each machine agent is able to obtain

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp193-198)

schedules for manufacturing orders based on MetaHeuristics (e.g. Genetic Algorithms or Tabu Search). Each machine agent is able to change the parameters of the basic algorithms according to the current situation or even to commutate from one algorithm to the other if the last start to be preferable. MetaHeuristics algorithms used in the Machine Agents will be prepared to handle dynamism (new jobs arriving, cancelled jobs, changing jobs attributes) by adapting the solutions or population elements. A coordination mechanism will be established between machine agents involved in the execution of operations (jobs) with precedence relations in order to deal with the feasibility of the generated schedules. In a real manufacturing system a product is produced, step by step, passing in several machines. In each machine it will be performed one operation (job) of the process plan.

Figure 2 – Multi-Agent Architecture

In this model we will use a Repair Approach, where the system wait for the solutions obtained by the machine agents and then apply a repair mechanism to shift some operations in the generated schedules till a feasible solution is obtained because we want to validate the system evolution. These method allow us to repair the integrated solution, in order to turn it feasible. It implicates the evaluation of all sequences in order to coordinate them. These coordination mechanisms are prepared to accept agents subjected to dynamism (new jobs arriving, cancelled jobs, changing jobs attributes, etc.). Coordination and Repair Mechanisms are used to guarantee the feasibility of schedules. Notice that solving locally problems and joining them will not guarantee the feasibility of schedules (e.g. precedence relations could not be guaranteed). The coordination mechanism will be established between machine agents involved in the execution of operations (jobs) with precedence relations in order to deal with the feasibility of the generated schedules in run-time. A

repair mechanism will allow solving problems found in the feasibility of the schedules, when these problems are not solved by the coordination mechanism. The proposed architecture involves three types of agents: the Task Manager, The Resource Agent and the Coordinator Agent. ‰ The Task Manager Agent (TMA) is responsible for the coordination of all tasks in the system. ‰ The resource agent (RA) has the capability of generate scheduling plans for him. The generation of the plan can be made with the application of some Meta-Heuristics that can help this process. This plan is generated in accordance with the knowledge that he has about is own processing capabilities and the needs of the system. ‰ The coordinator agent (CA) is responsible for the coordination of a group of agent’s resources. This corresponds to a distributed architecture where the generation of the global plan is made in small pieces, distributed for all the agents in the system. A distributed approach like this has some important problems to solve, like can all the pieces be joined without the application of o validation method, that can say if the plan is feasible or not. Most of the times the scheduling plans generated by each resource agent are not compatible because when it was generated the information of all the other resources in the system was not considered. To solve this problem the coordinator agent is able of change and repair the plans of is own group of agent in order to obtain a feasible plan. If in any time one agent obtains new information that can possibly turn the schedule obsolete, it will generate a new scheduling plan and it will communicate it with is own coordinator that will take all the needed measure to create a new global feasible scheduling plan. This task can be made in two different ways: the resource agent creates a totally new plan able of responds to new challenges made by the system. This can be called rescheduling. This way can be problematic if the system is exposed to many disruptions in short periods of time, because the agent will be obliged to generate several plans. The other possible way is the application of repair methods to the existing plan. This oblige that the agent must evaluate the existing plan to decide what changes must be made to respond to the real time information acquired. This can be characterized like dynamic approach that is always ready to adapt and create plan to accomplish real time information that is arrived at the system [1][2].

Proceedings of the 6th WSEAS Int. Conf. on EVOLUTIONARY COMPUTING, Lisbon, Portugal, June 16-18, 2005 (pp193-198)

4.3 Communication System All agents present in the system can communicate using a communications network, but they are not able to take decisions about any internal problem of other agents [1]. Each agent is completely autonomous in their own decisions, but if a decision changes something that can be important for the other agents in the system, the agent must give that information to the others. The communication method can be considered similar to the method proposed by Hino [13]. Event or perturbation Event notification to next operation

Event notification from previous operation

Agent MH1

Reaction to the perturbation in the previous operation

Reaction to the perturbation in the next operation

Figure 3 – Information Processing in an Agent [1]

5 Concluding Remarks and Future Work Manufacturing systems are changing its structure and organization. Supply chains are evolving to more coupled organizations like virtual enterprises, though maintaining the single entities autonomy, adaptability and dynamism properties. The Multi-Agent paradigm represents one of the most promising approaches for building complex, flexible, and cost-effective scheduling systems because of its distributed and dynamic nature. This paper have described and discussed the application of Multi-Agent systems to the resolution of dynamic and distributed scheduling problems. In this paper a multi-agent model for support dynamic scheduling in manufacturing environments has been proposed. In MADynScheMH system we model a Manufacturing Systems by means of a MultiAgent Systems, where each agent may represent a processing entity (resource). We consider that a good global solution for a scheduling problem may emerge from a community of machine agents solving locally their schedules and cooperating with other machine agents that shares some relations between the operations/jobs). The self parameterization of the Meta-Heuristics will allow a better adaptation to the situation being considered (related with the problem dimension and neighbourhood/population size).

References: [1] Ana M. Madureira, Meta-Heuristics Application to Scheduling in Dynamic Environments of Discrete Manufacturing, PhD Thesis, University of Minho, 2003 (in portuguese). [2] Ana Madureira, Carlos Ramos, Silvio C. Silva, Toward Dynamic Scheduling Through Evolutionary Computing, WSEAS Transactions on Systems, Issue 4, Volume 3, ISSN 1109-2777, pp. 1596-1604, 2004. [3] Carlos Ramos, An architecture and a negation protocol for the dynamic scheduling of Manufacturing Systems, IEEE Int. Conference on robotics and automation, Vol. 4, 1994. [4] H. Van Parunak, Manufacturing experience with contract net, Distributed Artificial Intelligence Huhns, M,N., ed., Pitman, pp 285-310, 1987. [5] H. Henseler , From reactive to active scheduling by using multi-agents, University Oldenburg, Germany, 1994. [6] J. Ferber, Les Sístemes multi-agents: versune intelligence collective. Interedition, 1995. [7] J. Liu and K.P. Sycara, Distributed problem solving through coordination in a society of agents, In proceedings of the 13th International workshop on DAI, 1994. [8] M. Wooldridge, An Introduction to Multiagent Systems, John Wiley and Sons, 2002. [9] M.J. Shaw and A.B. Whinston, Distributed planning in cellular flexible manufacturing systems, Tech Report, Management Information Research Center, Purdue University, 1983. [10] P. Cowling, D, Ouelhadj, S. Petrovic, MultiAgent Systems for Dynamic Scheduling, 2000. [11] P. Sousa, C. Ramos, and J. Neves, The Fabricare scheduling prototype suite: Agent interaction and knowledge base, Journal of Intelligent Manufacturing, 14(5):441–455, 2003. [12] R. Rabelo, L. M. Camarinha-Matos and H. Afsarmanesh, Multi agent perspectives to agile scheduling, Intelligent Systems for Manufacturing, Multi-agent Systems and Virtual Organizations, edited by Luis M. Camararinha-Matos Hamideh, Afsarmanesh Vladimir Marik, 1998. [13] Rei Hino, Kouichi Izuhara, Toshimichi Moriwaki, Message Exchange Method for Decentralized Scheduling, 4th IEEE International Symposium on Assembly and Task Planning, Japan, 244-249, 2001. [14] Visionary Manufacturing Challenges for 2020, Committee on Visionary Manufacturing Challenges, National Research Council, National Academic Press, 1999.