The Artificial Life and Adaptive Robotics Laboratory. ALAR Technical Report Series. Land Combat Scenario Planning: A Multiobjective Approach. Ang Yang ...
The Artificial Life and Adaptive Robotics Laboratory ALAR Technical Report Series
Land Combat Scenario Planning: A Multiobjective Approach Ang Yang, Hussein A. Abbass, Ruhul Sarker TR-ALAR-200605010
The Artificial Life and Adaptive Robotics Laboratory School of Information Technology and Electrical Engineering University of New South Wales Northcott Drive, Campbell, Canberra, ACT 2600 Australia Tel: +62 2 6268 8158 Fax:+61 2 6268 8581
Land Combat Scenario Planning: A Multiobjective Approach Ang Yang, Hussein A. Abbass, Ruhul Sarker Defence and Security Applications Research Centre (DSA) Univ. College, Univ. of New South Wales ADFA, Canberra ACT 2600, Australia Email: {Ang.Yang, h.abbass, r.sarker}@adfa.edu.au Abstract The simulation of land combat operations is a complex task. The space of possibilities is exponential and the performance criteria are usually in conflict; thus finding a sweet spot in this complex search space is a hard task. This paper focuses on the effect of population size and mutation rate on the performance of NSGA–II, as the evolutionary multiobjective optimization technique, to decide on the composition of forces using a complex land combat multi-agent scenario planning tool.
I. I NTRODUCTION Land combat is a complex task. Identifying a suitable composition of forces - in terms of size of each group in a mission, type of weapons and communication used by each group, and ammunition load - is normally planned by a team of experts, human–based simulation, or computer simulation. Military has used computer simulation for a long time as a cheap way for testing complex military concepts. Recently, the use of agent–based simulation (ABS) is replacing traditional simulations. In an ABS, agents can be autonomous, grouped in teams, have different capabilities and personalities, and use different strategies. ABS has opened many opportunities for testing military concepts. However, the search space of these black–box simulations is very complex. Moreover, the success of a military operation is almost always determined through a set of conflicting objectives. Thus, it is natural to look at the use of evolutionary multiobjective optimization (EMO) techniques to search these complex landscapes. However, the performance of EMO, like other evolutionary computation techniques, can be sensitive to parameters and the optimal set of parameters is normally problem dependent [1]. In this paper, we investigate the effect of two parameters, in particular, for NSGA–II [2]. In the following section, we present the experimental setup followed by the results and analysis. Finally conclusion and future work are discussed. II. E XPERIMENTAL SETUP A. The land combat simulation system We use the warfare intelligent system for dynamic optimization of missions (WISDOM–II), which is based on the Network centric multi-agent architecture (NCMAA) [3], as the multi–agent combat simulation engine. WISDOM has been used in many experiments for capability planning. It has five distinct components: the command, control, and communication (C3) component, the sensor component, the engagement component, the visualization component and the reasoning component. Four agent-types are supported in WISDOM–II: combatant, group leader, team leader and swarm commander. Agents are defined by their characteristics and personalities. Each agent has nine types of characteristics: health, skill, probability to respect - thus follow - the command, visibility, vision, communication, movement and engagement. The swarm commander can build plans and give orders to combat groups. The personality in WISDOM–II is a defined by two values: a magnitude and a direction vector representing the attraction–repulsion direction and weight for each agent. The movement of each agent is determined by its situation awareness and personality vector. In each time step, the agent can only move to its neighbor cells based on the overall influence of all perceived agents. A strategic decision is made by the swarm leader of each force based on the common operating picture (COP), which is the
TABLE I T HE CAPABILITY OF THE RED FORCE
# of Agents Vision Range Communication
Weapon
Range Lost Probability Latency Minimal Distance Range Strength Radius
Group R1 5 4 5 0 0 3 4 3 1
Group R2 20 5 5 0 0 0 4 3 0
Group R3 20 5 5 0 0 0 4 3 0
Group R4 1 10 10 0 0 -
global view of the battle field for that force. WISDOM–II collects information for each entity as well as for the interaction between entities. In this way, a large number of statistics are collected, which are fed to the reasoning engine, where natural language interpretation is provided to the user. WISDOM–II also provides capabilities such as interactive simulation. For more details of WISDOM-II, please refer to Yang et al.[3]. B. The scenario setup The scenario used in this paper includes two forces - blue and red force - playing against each others (Figure 1). The simulation environment is 30x30 cells and the destination flag (each force has a goal to occupy an area) is located at the middle of the environment. Both blue and red forces are composed of four groups, each of which consists of a set of homogenous agents. The capability of the red force is fixed during the simulation and defined in the table I. The group R4 is for surveillance. BHQ B2
B1
B4
B3
Red target Blue target
R2
R4
R1
R3
RHQ
Fig. 1. The combat scenario. BHQ and RHQ is the headquarter for the blue and red force respectively. The B1, B2, B3 and B4 are groups of the blue force while the R1, R2, R3 and R4 are the groups in the red force.
In the blue force, the group B4 has a single surveillance agent while the other groups have 10 agents each. Therefore, the total number of agents in the blue force is 31. The capability of the blue force is determined by the evolutionary process and vary as shown in Table II. C. The evolutionary setup A chromosome is divided into 4 blocks, each block corresponds to one group of the blue force. Each of the first three blocks consists of eight variables corresponding to vision, communication and weapon parameters. The last block has four variables only because the surveillance agent does not have weapons. The total number of variables in the chromosome is 28.
TABLE II T HE RANGE OF THE BLUE CAPABILITY Vision Range Communication
Weapon
Range Lost Probability Latency Minimal Distance Range Strength Radius
1 − 10 1 − 10 0 − 0.5 0−2 1−8 1−8 2−8 0−2
Yang et al. [4], [3] argued that taking a simple linear combination of the objectives may hide some information which is crucial in understanding the dynamics within a warfare simulation. Therefore, we explicitly represent the two objectives in the problem and use the non-dominated Sorting Genetic Algorithm – II (NSGA–II) [2], a multiobjective optimization algorithm. Two objectives are defined: minimizing the cost of the blue force and minimizing the casualty of the blue force. We run each experiment for 1000 generations, four different population sizes - 20, 40, 80 and 100, and three different mutation probabilities - 0.01, 0.036 and 0.1, respectively. The reason for choosing 0.036 is to follow some 1 literature which is suggesting the mutation rate to be the reciprocal of the number of variables ( 28 ≈ 0.036) [6], [7], [8], [2], [9], [10], [11] Therefore we have 12 different configurations. We fix the probability of crossover to 0.9, distribution index for crossover to 15, and the distribution index for mutation to 20 [2], [9], [10], [11]. Each individual is simulated 30 times with different seeds, each for 150 time steps. The fitness of each individual will be the average of 30 simulations. Each configuration is repeated 10 times with different seeds. III. R ESULTS AND ANALYSIS To compare the performance of each configuration, we adopt a statistical comparison method proposed by Knowles and Corne [14] in 2000, which extended the method introduced by Fonesca and Fleming in 1996 [13]. First, we try to understand the convergence during the search process by comparing the performance of the pareto-optimal sets of generation n with that of generation n − 50 to see when the improvement stops (see Figure 2). The figure shows that for different mutation probabilities, stagnation does not occur over the 1000 generations for population size 20 while it occurs at around generation 320, 220 and 280 for population sizes 40, 80 and 100 respectively. One may also notice that the level of mutation probability does not influence the convergence for population sizes except for population size 20, where there are some insignificant differences. Stagnation can be seen as an indication for the convergence of the algorithm. However, it is not enough to know when it occurs. It is also important to know the quality of each non–dominated set obtained by each setup as presented in Figure 3. The sub–figures on left hand side represent a comparison among all four setups with different population sizes. Each sub-figure corresponds to a different mutation level. Each comparison is done after 400 objective evaluations (this is a common factor for the four population size). The non–dominated sets achieved by each setup after each 400 objective evaluations are collected and compared. All figures show consistently that population size 80 has the best overall performance. The sub–figures on the right hand side make a more precise comparison between the two best performing population sizes: 80 and 100. One can see that with mutation rate of 0.036, population size 80 is consistently better than 100. We now turn our attention to comparing the different mutation rates. It seems from Figure 4 that the performance of mutation probability 0.1 is better than any of the other. This may suggest that a higher mutation probability is beneficial for this problem. We thus test a mutation probability of 0.2 and 0.5 for population size 80. Figure 5 presents the performance with population size 80 at five different mutation probabilities. We can now have more confident to suggest that a population size of 80 with mutation probability of 0.1 gives the best performance for this scenario, and the convergence occurs at generation 220.
100
100 Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
90
80
80
70
70
Percentage(%)
Percentage(%)
90
60 50 40
60 50 40
30
30
20
20
10
10
0
0 0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
Generation
500
600
700
800
900
1000
Generation
(a) Population size: 20
(b) Population size: 40
100
100 Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
90
Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
90
80
80
70
70
Percentage(%)
Percentage(%)
400
60 50 40
60 50 40
30
30
20
20
10
10
0
0 0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
Generation
400
500
600
700
800
900
1000
Generation
(c) Population size: 80
(d) Population size: 100
Fig. 2. The percentage of the objective space where the pareto-optimal set of the generation x outperforms that of the generation x − 50 within a single configuration.
IV. C ONCLUSIONS AND FUTURE WORK In this paper, we tested the effect of population size and mutation probability on the evolution of force compositions in a multi–objective setting. It was found that population size 80 with mutation probability 0.1 gave us the best performance. A key question in this work is how to generalize this result to other land combat problems. Fortunately, the military usually has a small number of generic scenarios to test different concepts (normally less than a dozen). Thus repeating this process for each scenario is not a difficult task. However, once we can identify the parameter setting for each scenario, we can undertake more experiments with different scenario setups. These setups may change the fitness landscape in terms of signals, but from our previous experiments, they do not change the characteristics of the landscape. In other words, the characteristics of the landscape vary from one scenario to another, but do not vary much when a scenario is run with different settings. Thus, we expect that the parameter setting for one scenario will continue to be useful when running this scenario with different scenario parameters. Our current work is focusing on studying approximation methods to reduce the computationally very expensive task of evaluating on the actual simulator which can be very time consuming. ACKNOWLEDGEMENT This work is supported by the University of New South Wales grant PS04411 and the Australian Research Council (ARC) Centre on Complex Systems grant number CEO0348249. The authors also wish to thank Mr. Lam Bui for useful discussions.
100
100
Pop size: 20 Pop size: 40 Pop size: 80 Pop size: 100
Pop size: 100 Pop size: 80
90
80
80
70
70
Percentage(%)
Percentage(%)
90
60 50 40
60 50 40
30
30
20
20
10
10
0
0
0
5
10
15
20
25
30
35
40
45
50
0
20
40
Number of Evaluation (x400)
60
80
100
120
140
160
180
200
Number of Evaluation (x400)
30
100
Pop size: 20 Pop size: 40 Pop size: 80 Pop size: 100
25
Pop size: 100 Pop size: 80
90 80 70
Percentage(%)
Percentage(%)
20
15
10
60 50 40 30
5 20
0
10 0
−5
0
5
10
15
20
25
30
35
40
45
50
0
20
40
Number of Evaluation (x400)
60
80
100
120
140
160
180
200
Number of Evaluation (x400)
30
100
Pop size: 20 Pop size: 40 Pop size: 80 Pop size: 100
25
Pop size: 100 Pop size: 80
90 80 70
Percentage(%)
Percentage(%)
20
15
10
60 50 40 30
5 20
0
10 0
−5
0
5
10
15
20
25
30
35
40
45
50
0
Number of Evaluation (x400)
20
40
60
80
100
120
140
160
180
200
Number of Evaluation (x400)
Fig. 3. The percentage of the objective space where the non–dominated set of one population size outperforms that of other population sizes at the same number of objective evaluations. Form top to bottom, the mutation rates are: 0.010, 0.036, 0.10.
R EFERENCES [1] Ursem, R.K.: Models for Evolutionary Algorithms and Their Applications in System Identification and Control Optimization. Ph.d. thesis, University of Aarhus, Denmark (2003) [2] Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6 (2002) 182–197 [3] Yang, A., Abbass, H.A., Sarker, R.: WISDOM-II: A network centric model for warfare. In: Ninth International Conference on Knowledge-Based Intelligent Information & Engineering Systems (KES 2005), LNCS 3683, Melbourne, Australia (2005) [4] Yang, A., Abbass, H.A., Sarker, R.: Landscape dynamics in multi-agent simulation combat systems. In: Proceedings of 17th Joint Australian Conference on Artificial Intelligence, LNAI 3339, Cairns, Australia, Springer-Verlag (2004)
100
100 Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
90
80
80
70
70
Percentage(%)
Percentage(%)
90
60 50 40
60 50 40
30
30
20
20
10
10
0
0 0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
Generation
(a) Population size: 20
700
800
900
1000
100 Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
90
Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100
90
80
80
70
70
Percentage(%)
Percentage(%)
600
(b) Population size: 40
100
60 50 40
60 50 40
30
30
20
20
10
10
0
0 0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
Generation
500
600
700
800
900
1000
Generation
(c) Population size: 80 Fig. 4.
500
Generation
(d) Population size: 100
The percentage of the objective space where the pareto-optimal set from one mutation probability outperforms that of any others. 5
110
Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.200 Mutation Rate: 0.500 Mutation Rate: 0.100
4
100 90 80
Percentage(%)
Percentage(%)
3
2
1
70 60 50 40 30 20
0
Mutation Rate: 0.010 Mutation Rate: 0.036 Mutation Rate: 0.100 Mutation Rate: 0.200 Mutation Rate: 0.500
10
−1
0
100
200
300
400
500
600
Generation
700
800
900
1000
0
0
100
200
300
400
500
600
700
800
900
1000
Generation
Fig. 5. Left: The percentage of the objective space where the pareto-optimal set from one mutation probability outperforms that from all other mutation probability with the population size of 80. Right: The percentage of the objective space where the pareto-optimal set from one mutation probability is not outperformed by that from any other mutation probabilities with the population size of 80.
[5] Yang, A., Abbass, H.A., Sarker, R., Curtis, N.J.: Evolving capability requirements in WISDOM-II. In Abbass, H.A., Bossamier, T., Wiles, J., eds.: Advances in Artificial Life, Proceeding of The Second Australian Conference on Artificial Life (ACAL05), Sydney, Australia, World Scientific Publisher (2005) 335–348
[6] Deb, K., Agrawal, R.: Simulated binary crossover for continuous search space. Complex Systems 9 (1995) 115–148 [7] Deb, K., Goyal, M.: A combined genetic adaptive search (GeneAS) for engineering design. Computer Science and Informatics 26 (1996) 30–45 [8] Deb, K., Agrawal, S.: Understanding interactions among genetic algorithm parameters. In Banzhaf, W., Reeves, C., eds.: Foundations of Genetic Algorithms 5. Morgan Kaufmann, San Francisco, CA (1999) 265–286 [9] Ballester, P.J., Carter, J.N.: Real-parameter genetic algorithms for finding multiple optimal solutions in multi-modal optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference, Lecture Notes in Computer Science 2723, Chicago, USA, Springer-Verlag (2003) 706–717 [10] Khare, V., Yao, X., Deb, K.: Performance scaling of multi-objective evolutionary algorithms. In Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L., eds.: Proceedings of the 2nd International Conference on Evolutionary Multi-Criterion Optimization (EMO’03), LNCS 2632, Faro, Portugal, Springer-Verlag (2003) 376–390 [11] Iorio, A., Li, X.: A cooperative coevolutionary multiobjective algorithm using non-dominated sorting. In: Proceeding of Genetic and Evolutionary Computation Conference 2004 (GECCO’04), Lecture Notes in Computer Science (LNCS 3102), Seattle, USA, SpringerVerlag (2004) 537–548 [12] Zitzler, E., Thiele, L., Deb, K.: Comparision of multiobjective evolutionary algorithms: Emprical results. Evolutionary Computation 8 (2000) 173195 [13] Fonseca, C.M., Fleming, P.J.: On the performance assessment and comparison of stochastic multiobjective optimizers. In Ebeling, V.W., Rechenberg, I., Schwefel, H.P., eds.: Proceedings of the 4th International Conference on Parallel Problem Solving from Nature, LNCS 1141, Springer-Verlag (1996) 584593 [14] Knowles, J.D., Corne, D.W.: Approximating the nondominated front using the pareto archived evolution strategy. Evolutionary Computation 8 (2000) 149–172
