An agent-based optimization of service fulfillment in next-generation ...

An agent-based optimization of service fulfillment in next-generation telecommunication systems Goran Dumic*, Vedran Podobnik*, Gordan Jezic*, Krunoslav Trzec** and Ana Petric* *

University of Zagreb, Faculty of Electrical Engineering and Computing, Department of Telecommunications, Unska 3, HR-10000 Zagreb, Croatia ** Ericsson Nikola Tesla, R&D Center, Krapinska 45, HR-10000 Zagreb, Croatia [email protected], [email protected], [email protected], [email protected], [email protected]

Abstract — In order to increase revenues, reduce operating costs and satisfy new user demands, network providers have to develop new approaches for their service provisioning operations. One of the possible solutions for this problem is utilization of intelligent software agents. In such a scenario, software agents act on behalf of the network provider while performing certain actions in competitive environment of the telecom market. This paper presents an agent-based approach for optimization of service provisioning in a dynamic environment of the next-generation networks. Proposed solution is based on a genetic algorithm and assures a minimum operational costs regarding provisioning of new services to users.

I. INTRODUCTION Tremendous developments in wireless technologies and mobile telecommunication systems, as well as rapid proliferation of various types of portable devices, have significantly amended computing lifestyle, thus advancing Weiser’s vision of ubiquitous computing [1] toward technical and economic viability [2]. Weiser’s ideas are becoming reality in the new generation of communication systems. Upcoming next-generation network (NGN) is characterized with an evolution towards an all-IP network, as well as with convergence of mobile and wireline networks into a single unified infrastructure. Fig. 1 presents the key processes and the main actors which can be identified during the service provisioning in the NGN environment. Processes characteristic for the service provisioning lifecycle are discovery, fulfillment and charging. The main actors are consumers with their mobile/wireless terminals, network operators and content providers together with software suppliers. Usually, the network operator offers services to consumers, establishing a Business-to-Consumer (B2C) relationship with them. On the other hand, since the network operator very often does not posses the content or software needed for service provisioning, it also has to establish Businessto-Business (B2B) relationship with the content providers and software suppliers. In order to increase revenue, reduce operating costs and satisfy new consumer demands, network providers have to develop new approaches for their service provisioning operations. They have to optimize their provisioning operations according to context information such as types of target consumers, service tariffs, location of service execution and type of

service content. The A-STORM (Agent-based Service Telecom Operations Management) project aims at designing and prototyping novel solutions for all the phases of service provisioning in NGN. Content providers

Business manager agent

Software suppliers

Provisioning manager agent

ACL

ACL Charging manager agent

ACL ACL

A-STORM project

ACL Database Manager agent

Databases

Consumers Network operator

Figure 1. Service provisioning in next-generation networks

The most of our former work was in the field of service discovery [3, 4]. These papers have envisioned NGNs as semantic-aware environments populated with a vast number of consumers, which posses diverse types of network-aware terminals. Content providers, on the other hand, continuously offer a remarkable selection of digital commodities (information/multimedia) through a set of services. Services are provisioned by network operators, which own indispensable infrastructure. In such an environment the consumer should access all available services anytime, from any place, and by using any of its terminals. Consequently, efficient service discovery mechanism was identified as an essential step in the creation of the successful service provisioning model in

Dumic, Goran; Podobnik, Vedran; Jezic, Gordan; Trzec, Krunoslav; Petric, Ana. An Agent-Based Optimization of Service Fulfillment in Next-Generation Telecommunication Systems. Proceedings of the 9th International Conference on Telecommunications ConTEL 2007 / Car, Zeljka; Kusek, Mario, editor(s). Zagreb, Croatia: IEEE, 2007. 57-63.

the NGN. After successfully implementing semanticaware proof-of-concept prototype for matching consumers’ demands (requested services) to network operators’ supplies (available services), the focus of the research moved on in the area of business-aware service fulfillment [5]. This paper presents an agent-based approach for optimization of the service fulfillment process in a dynamic environment of NGN. The organization of the paper is as follows. Section 2 presents key technologies used in the creation of a proofof-concept prototype for service provisioning in NGN. In Section 3, the service fulfillment problems in NGNs are presented and briefly described. The proof-of-concept prototype for service provisioning in NGN is presented in Section 4. Section 5 proposes directions for future work and concludes the paper. II.

TECHNOLOGICAL FOUNDATIONS

This section presents the two main technologies used in creation of proof-of-concept prototype for service provisioning in NGN: intelligent software agents and the genetic algorithm. In the designed multi-agent system, intelligent software agents, supported by genetic algorithm, are used to enable automated coordination of network operator’s operations and automated interaction between all entities in an NGN environment [6].

(i.e. non main) containers and must be told where to find (host and port) their main container. If another main container is started somewhere in the network it constitutes a different platform to which new normal containers can possibly register. Fig. 3 illustrates the above concepts through a sample scenario showing two JADE platforms, one composed of two containers (Platform 1) and the other composed of one container (Platform 2). ADAPTIVITY

Knowledge base

The proposed multi-agent system used is implemented using the Java Agent DEvelopment (JADE) framework. JADE (http://jade.tilab.com) is a software framework used for developing agent-based applications in compliance with the Foundation for Intelligent Physical Agents (FIPA, http://www.fipa.org) specifications for interoperable multi-agent systems. The goal of this framework is to simplify the development while ensuring standard compliance through a comprehensive set of system services and agents. Consequently, JADE can be considered both as an agent middleware that implements an agent platform and as an agent development framework. It deals with all those aspects that are not peculiar of the agent internals and that are independent of the applications, such as message transport, encoding and parsing, or agent life-cycle. Each running instance of the JADE runtime environment is called a Container as it can contain several agents. The set of active containers is called a Platform. A single special Main Container must always be active in a platform and all other containers register with it as soon as they start. Consequently, the first container to start in a platform must be a main container, while all other containers must be “normal”

Knowledge about environment

Reasoning mechanisms

Enabler of

INTELLIGENT SOFTWARE AGENT INTELLIGENCE AUTONOMY REACTIVITY

A. Intelligent Software Agents The dynamic and distributed nature of both data and applications in the NGN requires computer programs to not only respond to requests for resources but to intelligently anticipate and adapt to their environment while actively seeking ways to support their principals. Therefore, an intelligent software agent is an autonomous program which acts on behalf of its principal while conducting complex information and communication actions over the Internet. Intelligent software agents enable automated process execution and coordination, thus creating added value for its principal. Fig. 2 presents the relations between the main features of the intelligent software agents [7, 8, 9, 10, 11].

Enabler of

Owner's profile

Learning capabilities

PROACTIVITY COOPERATIVITY

TEMPORAL CONTINUITY

MOBILITY

Figure 2. A model of intelligent software agent

JADE agents are identified by a unique name (Agent IDentifier - AID). Assuming that the agents know each others names, they can communicate transparently regardless of their actual location in the: same container (e.g. Agent 2 and Agent 3), different containers in the same platform (e.g. Agent 1 and Agent 2) or different platforms (e.g. Agent 2 and Agent 4). The architecture is based on two agents that are started every time the platform is started - the AMS (Agent Management Service) and the DF (Directory Facilitator). Third component of the platform is the ACC (Agent Communication Channel). The AMS is the agent who exerts supervisory control over access to and use of the agent platform. Only one AMS exists in a single platform. The AMS provides a naming service (i.e. ensures that each agent in the platform has a unique name) by maintaining a directory of AIDs and agents’ states. Each agent must register with the AMS in order to get a valid AID. It also represents the authority in the platform (for instance it is possible to create/kill agents on remote containers by requesting that from the AMS). Therefore, the AMS provides life-cycle service and White Page service. The DF provides a Yellow Pages service by means of which an agent can find other agents providing the service it requires in order to achieve his goals.

The ACC is a Message Transport System that controls the exchange of all messages within the platform, including messages to/from remote platforms.

Main Container DF

Agent 1

AMS

Platform 1 Container 1 Agent 2

Agent 3

defined over the genetic representation and measures the quality of the represented solution. It is always problem dependent. The evolution starts from a population of completely random individuals and happens in generations. In each generation, the fitness of the whole population is evaluated, multiple individuals are stochastically selected from the current population (based on their fitness) and modified (mutated or recombined) to form a new population. The new population is then used in the next iteration of the algorithm. genome or chromosome allele gene

Main Container

1

0

1

1

population

DF

Agent 4

Figure 4. The principle of genetic algorithm

AMS

Platform 2 Figure 3. JADE example

The most important feature that JADE agents provide is the ability to communicate. The communication is performed by using ACL (Agent Communication Language) messages which agents exchange between themselves. The communication process is based on asynchronous message passing. Each agent has a sort of mailbox (message queue) where the JADE runtime posts messages sent by other agents. Whenever a message is posted in the message queue the receiving agent is notified. If and when the agent actually picks up the message from the message queue to process it, is completely up to the programmer. ACL Messages are specified by the ACL language defined by the FIPA international standard for agent interoperability. B. Genetic Algorithm . The next important technology used for optimization of multiple service requests is a genetic algorithm (GA) [12]. It is a search technique used in computer science to find approximate solutions for optimization and search problems. It is categorized as global search heuristic which classifies it as an incomplete search. The GA is a particular class of evolutionary algorithms that use techniques inspired by evolutionary biology such as inheritance, mutation, selection and crossover (also called recombination). In order to use a GA, two things need to be defined: a genetic representation of the solution domain and a fitness function. A fitness function is

The principle of GA is shown in Fig. 4. A standard representation of the solution is as an array of genes (i.e. a problem is represented as a genome or chromosome.). Every gene represents one part of final solution, where each gene is coded with bits - alleles. Traditionally, solutions are represented in binary as strings of 0s and 1s, but different encodings are also possible. The main property that makes these genetic representations convenient is that their parts are easily aligned due to their fixed size, which facilitates simple crossover operation. Variable length representations were also used, but crossover implementation is more complex in this case. The evolution starts from a population of completely random individuals and happens in generations. The initial population is created from the first set of chromosomes. In each generation, the fitness of the whole population is evaluated, multiple individuals are stochastically selected from the current population (based on their fitness) and modified (mutated or recombined) to form a new population, which will be used in the next iteration of the algorithm. The fitness function is defined over one chromosome and measures the quality of the represented solution. III.

SERVICE FULFILLMENT IN THE NEXT-GENERATION NETWORKS

The introduction of new software components that enable advanced mobile multimedia services has rapidly increased service provisioning complexity in the fulfillment phase. From the technology viewpoint, NGN architecture consists of different types of networks, nodes

and terminals. This heterogeneity causes problems in the service fulfillment area. The problems especially arise because NGN concept demands that services should be seamless for consumers (anytime and anywhere) and automatically configured regarding to user terminal, profile, preferences and network capabilities. When a NGN provider wants to provide its subscribers with a new multimedia service, there are several steps that must be made. For start, technical division must obtain the software needed for provisioning of this multimedia service and the same software has to be installed on different nodes inside the network. The next step is to purchase the new content, taking into consideration its different parameters (length, codec, content type, etc.). At last, additional software has to be installed on the terminals of those consumers that are subscribed for this type of content. This process has to be repeated every time a new service occurs within the NGN provider’s coverage area. In the A-STORM project, the event based system is used to automate earlier mentioned operations using agents, Web Services and rule engines. In this system, network operator defines business rules in Java Expert System Shell (JESS) scripts. These scripts define how agents respond to the events. Events are defined by software suppliers and the electronic market (e-market). When several events match currently defined business rules, agents will trigger operations that will cause the system to deploy new services. The triggered operations are completely automated. The initial service deployments are stored in the Jess rule engine that is handled by the intelligent agent for business support. Consequently, the intelligent agent decisions are guided by service deployments that reflect the changes in the competitive environment (telecom market) in which the network provider is embedded. Every service deployment has different operative expenses of realization. These expenses reflect differences between deployments regarding their parameters defined in business rules, as well as time required for its realization in a mobile network. During the service provisioning process, behaviors, location and even balance of the user’s account should be taken into consideration. Therefore we are proposing an automated business-driven solution, which will help the operator to efficiently introduce the new service to the consumers. The Fig. 1 (already shown in Section 1) presents the key entities introduced into network operator’s architecture. These entities allow business decisions to be implemented more easily, with special emphasis put on elements important for service deployment. After the content providers and software suppliers release a new version of the software/content component for a particular type of a mobile terminal, they will send a notification to the Business Manager Agent (BMA). When all of the prerequisites of the business rules (which are defined by the network operator) have been fulfilled, the rule engine will inform the BMA to start the process of automated service deployment. The BMA sends the service deployment order and the service parameters to the Charging Manager Agent (CMA) and the Provisioning Manager Agent (PMA). CMA checks the balance of consumers’ accounts while the PMA finds the targeted consumers for the new service. Both, the CMA and PMA contact the DataBase Manager Agent

(DBMA) to find the actual mobile users (their International Mobile Subscriber Identity (IMSI) numbers). The DBMA serves as an interface to the network operator’s database that contains user information. After the CMA and the PMA generate the list of potential consumers, the PMA coordinates the deployment of software/content. IV. IMPLEMENTATION OF USED TECHNOLOGIES The optimization problem is solved using Java Genetic Algorithm Package (JGAP, http://jgap.sourceforge.net). JGAP is a genetic algorithms component written in the form of a Java framework. It provides basic genetic mechanisms that can be easily used to apply evolutionary principles to problem solutions. A. Optimization Criteria An example describing the minimization of operational costs when introducing new service, along with optimization criteria is shown in Fig. 5. Provisioning of each new service requires initial service deployment, during which the delivery of new software and/or content on specific network nodes or directly to the consumer terminal is committed. Additionally, due to different complexity, every service deployment operation is characterized with its cost, which depends on various parameters (e.g. the actual time needed to install and set up all required software and/or content). The main idea of our optimization model is that the deployment operations are not preformed every time a new service is introduced, but rather after a bundle of new services is gathered for the initial deployment. New services are determined both with software and content. If we have in mind a vast number of consumers characteristic for mobile telecommunication environments, we can conclude that new services are introduced every second. Therefore, waiting to gather a bundle of initial service deployment operations is not time critical from the consumer point of view. It also enables twofold optimization from the network operator point of view. Not only it prevents the situation of multiple deployment of the same software/content if one software/content is contained in more initial service deployment operations from the same bundle, but also enables the discovery of cost-optimal deployment sequence. Imagine we have only two service deployment operations. Each initial service deployment operation has a certain cost and contains a number of software/content components needed for implementation. Relative service deployment cost is defined as total service cost divided by quantity of software/content components. If all the products of relative prices with a number of yet unused software/component components (for each service) are summed together, then the total deployment cost of new services bundle is calculated. We can also take a look at the numbers. The conformation 1 – 2 has a total deployment cost of 2, which is better then the other conformation's cost of 7/3. Obviously, in the example presented in Fig. 5 the minimal costs will be achieved by using the service deployment conformation 1 – 2.

New service #1

New service #2

Cost: 1

Cost: 2

Software/content: 100, 101, 102

Software/content: 101, 102, 200, 300

R=

Cost Software/content quantity

Relative cost

Total price = R1 × S1 + R2 × S2 + ... Quantity of yet undeployed software/content

Conformation: Total deployment cost = 1 × 3 + 2 × 2 = 2 3 4 100, 101, 102

200, 300

Conformation: Total deployment cost = 2 × 4 + 1 × 1 = 7 4 3 3 101, 102, 200, 300

100

Figure 5. Optimization criteria

B. Setting Up At the heart of the genetic algorithm is a chromosome. A chromosome represents a potential solution and is divided into multiple genes. In JGAP genes represent distinct aspects of the solution as a whole, just as human genes represent distinct aspects of individual people, such as their sex or eye color. During the JGAP evolution process, chromosomes are exposed to multiple genetic operators and then are chosen for the next generation during a natural selection phase based upon their fitness, which is a measure of how optimal that solution is relative to other potential solutions. The goal of the genetic algorithm is to simulate the natural evolution process in order to produce superior solutions. First step is to define a chromosome. In this solution, genes represent new services, while chromosomes define the order of initial service deployments. This is shown in Fig. 6.

Figure 6. Chromosome and gene structure

Next step is definition of a fitness function. Since JGAP has no knowledge of the specific problem we are actually trying to solve, and hence has no intrinsic way of deciding if one potential solution is any better than another potential solution for a specific problem, we have to implement a single method that accepts a potential problem solution and returns an integer value that indicates how good (or "fit") that solution is relative to other possible solutions. The higher the integer value is, the better is the solution. The lower the integer value is (with value of 1 being the lowest legal fitness value), the poorer is the solution. JGAP will use these fitness measurements to evolve a population of solutions toward a more optimal set of solutions. When implementing a fitness function, service deployment execution time and software/content components must be taken into consideration (see Fig. 5). Finally, a population needs to be created from a set of chromosomes. As the optimization process goes on, the population continues to evolve. Consequently, the genetic algorithm provides better and better solutions. C. Optimization Issues During evolvement, the genetic algorithm performs a number of tasks: one of them is applying genetic operators to chromosomes. Genetic operator crossover is likely to cause distortion to the chromosome, in other words - to pick up the same genes from other chromosomes. Fig. 7 shows the result of applying crossover on two following chromosomes: 1 2 3 4 5 6 7 and 7 6 5 4 3 2 1 (the numbers represent service deployment order). The result of crossover in this example is an invalid execution order since all genes must differ one from another.

Figure 7. Repeating elements problem

The resulting chromosome became 1 2 3 4 5 2 1 which is corrupted. Repeating elements should be excluded. In order to resolve this problem chromosomes have to be modified. One of the possible solutions is to encode the chromosome. Encoding is done in the following way. The initial chromosome will be fixed at value: 1 2 3 4 5 6 7. Every other chromosome will then correspond to index of its gene in the initial chromosome (see Fig. 8). Gene number 2 has corresponding index = 2 in the initial chromosome. When performing chromosome encoding process, the index number is remembered and the gene itself is removed from the initial chromosome.

service ID = 1 deployment cost = 2 software/content list = 100, 102, 103

New service

Business Manager Agent

Jess rule engine Figure 8. Encoding the chromosome’s first element

Figure 11. Receiving service deployment tasks from Jess rule engine

Now all genes move one index to the left, so for example gene number 4 has an index = 3 in the initial chromosome. Encoding the next element is shown in Fig. 9. The procedure is analogue.

When a triggering number of service deployment operations is reached, optimization using JGAP is initiated, to achieve minimum operational costs. At the end, the results are sent to the PMA using the FIPA request interaction protocol (see Fig. 12). 1.

initial deployment 3

2.

initial deployment 1

3.

initial deployment 4

4.

initial deployment 2

3 1 4 2

Optimal deployment order Figure 9. Encoding the chromosome’s second element

The process is repeated for all elements in example chromosome. The final result of encoding is shown in Fig. 10. Now, the repeating elements are no longer a problem, since the repeating elements are not real gene values anymore, they are simply coded in such way. And since decoding is made the same way – relative regarding to the initial chromosome, it will bring us back to the original sample chromosome: 2 4 3 1 7 6 5 .

Figure 10. Encoded chromosome

D. Agent Task Solution and Results The BMA receives new service deployment tasks from Jess Rule Engine (see Fig. 11). Each service deployment contains a number of parameters, of which three are relevant for optimization. They are: serviceID by which service deployments are distinguished, operationCost and a list of required software/content components.

Business Manager Agent

Provisioning Manager Agent

Figure 12. Sending optimal execution order to Provisioning Manager agent

E. Analysis of the proposed solution The efficiency of presented solution is going to be demonstrated through comparison with solution based on classic permutations. In a real world NGN environment bundles of new services for the initial deployment are going to be generated very often, because introduction of new services is a result of every newly available content. For example, services like multimedia (video) news generate fresh content almost every second. Assume the network operator has to optimize the initial deployment order for a bundle of 12 new services. The number of possible solutions is 12! = 479 001 600. Each of the solution is potentially the best one. The permutation process would include the following actions: take the first conformation, evaluate fitness value, permutate and evaluate again. The next solution can be either better or worse. This process must be repeated for all 479 001 600 combinations before it can return the optimal solution. But, unlike permutations, the genetic algorithm is smarter. It evolves a solution with each new iteration, so it does not just

provide the next solution: each time we get closer towards a better solution. Additionally, the genetic algorithm also adds diversity to solutions, because it is not just a sequential process. It also generates new solutions by mutation and crossover, not just simply by taking next conformation from a pool of solutions. Tables I and II demonstrate, through comparison of presented solution with the one based on classic permutations, why it is important to choose the right optimization algorithm. Additionally, Table I shows how genetic algorithm parameter tuning has a tremendous effect on execution time. The first two columns represent values of basic genetic algorithm parameters required for evaluation. Maximal allowed evolutions represent a number of algorithm iterations, and population size defines how many chromosomes will be involved in evaluation process and make a population in every iteration of the algorithm. The total time of evaluation after the defined number of iterations is listed in the third column, while the value of the best solution found is presented in the last column. TABLE I.

COMPARATION OF GENETIC ALGORITHM PARAMETERS' EXECUTION TIME

applications. The proposed solution is based on a genetic algorithm and assures the minimum operational costs regarding the provisioning of new services to consumers. For related work by other authors, we recommend "Web Search Using a Genetic Algorithm" [13], which developed a system called Webnaut. It is an intelligent multi-agent system that uses a genetic algorithm to collect and recommend Web pages. Future work will focus the research in the area of service charging in the NGN. After designing service charging model we plan to integrate our solutions for all the phases of service provisioning in NGN into integral prototype. ACKNOWLEDGMENT This paper resulted as part of FER/ETK Summer Camp 2006 workshop, supported by Ericsson Nikola Tesla, Croatia. REFERENCES [1] [2] [3]

[4]

[5]

[6]

While the genetic algorithm’s execution time is measured in seconds, even when a lot of evolutions and vast populations are allowed, the permutation’s execution time is much worse. In tested and presented scenario, the permutation based solution required 7.481 hours, and the genetic algorithm found the optimal solution in 0.274 seconds, which is almost 100 000 times better. The permutation execution time is shown in Table II. TABLE II.

PERMUTATION EXECUTION TIME

[7] [8] [9] [10]

[11]

[12]

V. CONCLUSION AND FUTURE WORK This paper presented successful bounding of several technologies: Jess rule engine, genetic algorithm (JGAP) and intelligent software agents. It has also shown that these technologies can be successfully applied for automation of provisioning operations in next-generation network environments. Proposed service fulfillment model is not only interesting from a scientific point of view, but is also very amenable to real world

[13]

M. Weiser, “The computer for the 21st Century,” Scientific American, vol. 265(3), pp. 94-104, 1991. D. Saha, and A. Mukherjee, “Pervasive computing: a paradigm for the 21st Century,” IEEE Computer, vol. 36(3), pp. 25-31, 2003. V. Podobnik, G. Jezic, and K. Trzec, “An agent-mediated electronic market of semantic Web services,” in Proceedings of the AAMAS Workshop on Business Agents and the Semantic Web (BASeWEB ’06), Hakodate (Japan), pp. 1-10, 2006. V. Podobnik, K. Trzec, and G. Jezic, “An auction-based semantic service discovery model for e-commerce applications,” Lecture Notes in Computer Science, vol. 4277, pp. 97-106, 2006. G. Dumic, “Optimization of execution order of service fulfillment business strategies,” in Proceedings of the Summer Camp 2006 (Ericsson Nikola Tesla & Faculty of Electrical Engineering and Computing, Zagreb, Croatia, 2006), pp. 226-255. K. Trzec, I. Lovrek, and B. Mikac, “Agent behaviour in double auction electronic market for communication resources,” Lecture Notes in Artificial Intelligence, Subseries of Lecture Notes in Computer Science, vol. 4251, pp. 318-325, 2006. J. M. Bradshaw, Software agents. Cambridge (USA): MIT Press, 1997. D. N. Chorafas, Agent technology handbook. New York: McGraw-Hill, 1998. W. T. Cockayne, and M. Zyda, Mobile agents. Greenwich (USA): Manning Publications, 1998. G. Jezic, M. Kusek, and V. Sinkovic, “Teamwork coordination in large-scale mobile agent network,” Lecture Notes in Artificial Intelligence, Subseries of Lecture Notes in Computer Science, vol. 4251, pp. 236-243, 2006. M. Kusek, I. Lovrek, and V. Sinkovic, “Agent team coordination in the mobile agent network,” Lecture Notes in Artificial Intelligence, Subseries of Lecture Notes in Computer Science, vol. 3053, pp. 240-246, 2005. G. Dozier, A. Homaifar, E. Tunstel, and D. Battle, “An introduction to evolutionary computation” (Chapter 17), Intelligent control systems using soft computing methodologies, A. Zilouchian, and M. Jamshidi (Eds.), Boca Raton (USA): CRC Press, 2001, pp. 365-380. Z. Z. Nick and P. Themis, “Web Search Using a Genetic Algorithm,“ IEEE Internet Computing, vol. 5(2), pp. 18-26, 2001.