Constraint Handling and Multi-Objective Methods for ...

JOURNAL OF GUIDANCE, CONTROL, AND DYNAMICS Vol. 38, No. 4, April 2015

Engineering Notes Constraint Handling and Multi-Objective Methods for the Evolution of Interplanetary Trajectories

competition time frame, the best known trajectory is a result obtained through evolutionary techniques by an ESA team [13]. The approach was awarded with the Humies gold medal [14] (for humancompetitive results produced by a form of genetic and evolutionary computation), proving how advanced evolutionary optimization setups are competitive with traditional mission analysis methods and, in some cases, superior. The work of this Note is built upon the methods developed during GTOC6 [13], and few advanced evolutionary techniques able to evolve complex Jupiter capture options are proposed. The evolutionary process is set up to be able to deal with constraints, as well as with multiple objectives and show the use of advanced evolutionary techniques to effectively search the solution space in both cases. In particular, introduced for the first time is the use of parameter control [15], constraint handling techniques such as coevolution [16], death penalty [17], and immune systems [18], in the context of trajectory problems, and evolutionary problem decomposition [19] for multi-objective cases is explored. It will be shown how these techniques allow the evolution of solutions of use to trajectory designers for the complex trajectory case here considered, where more naive approaches fail to find acceptable solutions in a reasonable time. In fact, experiments on the problem considered in this work show how popular evolutionary approaches, such as particle swarm optimization [20] (with constriction factor and ring topology), genetic algorithms [21] (with roulette selection, Gaussian mutation, and binomial crossover), simulated annealing [22] (with adaptive neighborhood selection), differential evolution [23] (in its original implementation) and nondominated sorting genetic algorithm [24] (NSGA II in its original implementation), all fail to provide interesting solutions in a reasonable time.

Dario Izzo,∗ Daniel Hennes,† and Annalisa Riccardi† ESA, 2201 AZ Noordwijk, The Netherlands

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

DOI: 10.2514/1.G000619

I.

Introduction

I

NTERPLANETARY spacecraft trajectory design is a nontrivial and widely studied problem. Planetary motion never allows for a repeating constellation and the nonlinear nature of gravity prevents simple and intuitive steering strategies to be pursued. The pioneering work made by the young Minovitch [1,2] paved the way to missions such as Voyager and Pioneer by introducing the planetary gravityassist technique. Later, the manufacturing of electric propulsion engines, remarkably similar to those imagined by Tsiolkowsky and Goddard at the beginning of the past century, enriched considerably the set of possible trajectories to explore the solar system. None of these pioneers, though, could have guessed nor imagined that interplanetary trajectories could be effectively designed by letting an artificial evolutionary process select, generation after generation, the best designs in terms of mission objectives and requirements. Early publications, dating back to the 1990s, used genetic algorithms to design low-thrust trajectories [3,4] and high-energy transfers [5]. In the following years, work focused, on one hand, on finding efficient encodings to represent interplanetary trajectories as an artificial chromosome and, on the other hand, in setting up the artificial evolutionary process in an effective way (see [6–8] as a few examples treating the subject). Skepticism on this approach by mission analysis experts has always been grounded in the noncompetitiveness of the results achieved by these methods. In a widely quoted 1998 review paper [9] on trajectory optimization, these methods were not even mentioned. In a more recent book dedicated to spacecraft trajectory optimization [10], 3 out of 10 chapters cover evolutionary techniques. A fair comparison between traditional optimization methods and evolutionary techniques, in the context of spacecraft trajectory optimization, is very difficult because performance is greatly influenced by the particular problem at hand. The global trajectory optimization competition (GTOC) event [11], organized biyearly by the international scientific community, is a good forum where teams using diverse approaches compete to solve a well-defined difficult interplanetary trajectory problem. Although evolutionary techniques have been used by many of its participants, it was only in its 2012 edition [12] (GTOC6) that evolutionary techniques obtained competitive results. Although outside of the

II.

Test Case: A Jupiter Capture Trajectory

The selected test case is inspired from the problem released by the Jet Propulsion Laboratory (JPL) in occasion of the sixth edition of the global .trajectory optimization competition [12]. The problem was that of mapping the surface of all of the Galilean moons by taking pictures from a spacecraft performing multiple flybys. For the purpose of this Note, the focus is on the capture part of that trajectory and the moon encounter sequence is fixed. The simple ephemerides provided by JPL in the GTOC6 problem description [12] are also used. A spacecraft arrives at Jupiter with a relative velocity jv0 j  3.5 Km∕s at R0  1000 Jupiter radii (JR). The spacecraft needs to make a first encounter with the moon Callisto and then three successive ones with Ganymede. The requirements for the trajectory are that the total flight time TOF < T Max  365.25 days and that the minimum distance from Jupiter d > Dmin  2 JR. The spacecraft is assumed to be equipped with a propulsion system able to deliver impulsive velocity changes: The objective is to minimize their sum ΔV tot . The problem is quite complex because multiple global and local optima exist and the fitness landscape is extremely rugged. The relatively short planetary periods of the Jupiter moons considered (7.15 days for Ganymede and 16.7 for Callisto) allow for an extremely large number of possible moon relative phasings, which, in turn, adds much complexity to the fitness landscape. As a comparison, one can easily work out that, during the interplanetary part of, say, the Cassini trajectory, Venus made 9–10 revolutions, whereas during the mission here considered, Ganymede is allowed to make more than 50 revolutions and Callisto 22. Besides, the moon resonance is not simplifying the possible geometries because the planetary motion is effectively three-dimensional and the orbital period matching is not perfect. The mission analysis for similar capture trajectories was recently studied in great detail using standard techniques [25].

Received 12 March 2014; revision received 5 September 2014; accepted for publication 8 September 2014; published online 16 December 2014. Copyright © 2014 by Dario Izzo. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. Copies of this paper may be made for personal or internal use, on condition that the copier pay the $10.00 per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923; include the code 1533-3884/14 and $10.00 in correspondence with the CCC. *Advanced Concepts Team Scientific Coordinator, European Space Research and Technology Centre, Keplerlaan 1; [email protected]. † Advanced Concepts Team Research Fellow, European Space Research and Technology Centre, Keplerlaan 1. 792

J. GUIDANCE, VOL. 38, NO. 4:

In a real case, mission planners are not only interested in the single trajectory satisfying all the constraints, rather they like to have a portfolio of trajectories that offer tradeoffs between different requirements. To address these needs, first the trajectory design as a single-objective constrained optimization problem is considered and few evolutionary techniques, most suited to search for good trajectories, are compared. At a second stage, the constraints (focusing on the time of flight) are considered as objectives and the problem Pareto front (i.e., the set on nondominated solution) is sought. A. Trajectory Encoding

One may look at evolutionary techniques as global optimization algorithms trying to find the best solutions x to a problem P in the form

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

P:

minimize : subject to :

fx ceq x  0 cineq x ≤ 0 lb ≤ x ≤ ub

(1)

where x ∈ Rn is the decision vector (or chromosome), f: Rn → Rnf is the objective function (or fitness), ceq : Rn → Rne are the equality constraints, cineq : Rn → Rni are the inequality constraints, and lb ∈ Rn and ub ∈ Rn are the box bounds. The process of defining the interplanetary trajectory in a chromosome x is called encoding and is crucial to the success of evolutionary techniques (see [26] for a discussion on the evolvability of a system). The encoding used in this Note is an updated version of the MGA-1DSM encoding [27] and is briefly described here. The trajectory is divided into four legs. The chromosome encoding the full trajectory is the concatenation of the chromosomes encoding each single leg. Except for the first leg, all legs are encoded in the same way. The full chromosome is given by x  t0 ; u; v; T 0  

3 X

r~pi ; βxi ; βyi ; ηi ; T i 

i1

P where  and denote concatenation operations. The first trajectory leg starts at t0 (encoded in x) in the point ^ r0  Rs cos θ cos ϕi^  sin θ cos ϕj^  sin ϕk ^ k^ are inertially fixed axes. The starting where Rs  1000 JR and i,^ j, angles ϕ, θ are found by the following equations: θ  2πu and ϕ  cos−1 2v − 1 − π∕2, where u; v ∈ 0; 1 are directly encoded in x. The next planetary encounter (in the test case, Ganymede) happens after T 0 (encoded in x) in r1 , which is Ganymede’s position at the epoch t0  T 0 . Solving Lambert’s problem [28] between r0 and r1 , it is possible to compute the spacecraft velocities v0 , v−1 at the beginning and the end of the first trajectory leg. All remaining i  1; : : : ; 3 legs are encoded in the following way. Starting from v−i , the spacecraft will perform a flyby of the moon that will change its velocity to v i according to the following equations: v~ 

v−i

− vmoon

e  1  rp ∕μmoon v~2 δ  2 sin−1 1∕e ~ vj ~ i  v∕j j  i × vmoon ∕ji × vmoon j ki×j ~ v  cosβ sinδj  sinβ sinδk i  vmoon  jvjcosδi where rp is the periplanet radius along the moon centric hyperbola. This value is encoded in x as rpi  r~pi Rpi , where r~pi is directly encoded in x. The moon gravity parameter is μmoon and its velocity

793

ENGINEERING NOTES

vmoon is computed from the ephemerides at epoch T 0  T 1  : : :  T i . The variable β determines the flyby plane orientation and is defined as follows: q q x  βx 1 − 1∕2β2y y  βy 1 − 1∕2β2x and, β  atan 2x; y where βx , βx ∈ −1; 1 are directly encoded in x. Starting from the computed v i velocity, it is possible to integrate forward in time along a Keplerian orbit for ηi T i to find the position rDSMi of the deep space maneuver (DSM) and solve one more Lambert problem to connect rDSMi to ri1 and thus compute the deep space maneuver magnitude ΔV i , the arrival velocity v−i1, and the closest distance to Jupiter di along the leg (this is computed separately for the ballistic arcs before and after the DSM and then only the minimum value is considered). The values encoded in x thus allow one to reconstruct the entire interplanetary trajectory, including multiple revolutions. The upper and lower bounds of all variables, as considered in the particular problem instances here studied, are shown in Table 1. Note the use of very short times of flights between the first and the second moon encounter. This technique, also named double flyby [25], can result in very fast and efficient ballistic captures, though the spacecraft must be navigated skillfully to actuate such a maneuver. B. Constrained Problem

First, the single-objective constrained problem P c is considered: P minimize : ΔV tot x  3i0 ΔV i x JR; i  0; : : : ; 3; subject to : di ≥ 2P Pc : T 0  3i1 T i ≤ 365.25; lb ≤ x ≤ ub This optimization problem represents the Jupiter capture described earlier. The objective is to minimize the cumulative velocity increments ΔV i subject to the constraints on the total time of flight and on the closest approach to Jupiter. C. Multi-Objective Problem

Designers are not only interested in the best possible trajectory that fulfills their requirements, but more important, on tradeoffs between the mission objectives and requirements. In other words, mission designers want to know how much to gain in terms of the primary objective if, for example, a requirement is loosened. This type of analysis can also be performed making use of evolutionary techniques. To this purpose, let us consider the following problem: minimize : fx  ΔV tot x; TOF subject to : lb ≤ x ≤ ub Table 1

Bounds for the variables encoded in x

t0 (MJD2000) u v βxi βyi ηi T 0 , days T 1 , days T 2 , days T 3 , days

lb 10,460 0 0 −1 −1 0 180 0.1 10 10

ub 10,480 1 1 1 1 1 200 5 150 40

rp1

Rp1 50 km Rp1

Rp1 1000 km Rp1

rp2

Rp2 50 km Rp2

Rp2 1000 km Rp2

rp3

Rp3 50 km Rp3

Rp3 1000 km Rp3

794

J. GUIDANCE, VOL. 38, NO. 4:

In this multi-objective problem, the two objectives are to minimize the cumulative velocity increment and the total time of flight TOF  T 0 

3 X

Ti

i1

Such a problem has a multitude of solutions and the Pareto front spans from the minimum ΔV ideal point to very high ΔV solutions that have a minimum time of flight. In fact, from ΔV tot → ∞ it follows that TOF → 0. Typically, designers are only interested in a small fraction of such a Pareto front (i.e., the one defined by the inequality TOF > T  ). To focus the search in that area, the second objective is considered to be, rather than directly the time of flight, the violation of the inequality TOF < T  (note the change in the inequality sign). In this case, the trajectories with short time of flight will be dominated by the one trajectory having minimum ΔV and TOF < T  . The formal description of the multi-objective problem considered is thus

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

P mo :

minimize : subject to :

fx  ΔV tot x; maxTOF − T  ; 0 lb ≤ x ≤ ub

The two ideal points of such a problem are of particular interest and play an important role in the seeding technique proposed later in this Note. The ideal points, in a multi-objective minimization problem, are the points in the Pareto front where at least one of the objectives is minimized. In the analyzed case, these would be the minimum ΔV trajectory, which will turn out to be purely ballistic and with a TOF  305 days, and the minimum ΔV trajectory having TOF < T  . In the analyzed test case, it is set T   280 days and a ΔV  230 m∕s is found. Figure 1 shows the visualization of the two ideal points of the Pareto front. The minimum ΔV solution is on the left. The minimum ΔV trajectory with TOF < T   280 days is on the right.

III.

Evolution, Parameter Control, and Decomposition

Evolutionary techniques maintain a population of chromosomes (or individuals) xi and apply different operators to generate offspring that are then inserted in the population, improving its overall performance (as measured in terms of the objective functions and constraints violation). Their use in the context of interplanetary trajectory design has been explored in the last decade in many publications, but mainly in the context of single-objective optimization. Paradigms such as differential evolution [23], particle swarm optimization (PSO) [20], genetic algorithms [21], or simulated annealing [22] have been applied with different degrees of success to trajectory design problems. Most of the work done, though, does not extend to different contexts or problems than the ones it originated in, the main problem being that of parameter tuning and its coupling with the problem instances considered. For a successful application of any of the aforementioned techniques, a designer is asked to first tune parameters (manually or with an automated procedure) to tweak the evolutionary process toward most convenient search space areas. The parameters make sure that the genetic operators, responsible to produce new solutions to be evaluated against previously found designs, are efficient and lead to an

ENGINEERING NOTES

evolvable system. To alleviate this problem, the use of parameter control is proposed and tested (i.e., a technique that makes algorithmic parameters change during one run trying to adapt the genetic operators to the fitness landscape). In [15], parameter control is categorized into three classes: 1) deterministic parameter control: whenever the parameters are changed according to some deterministic rule; 2) adaptive parameter control: whenever the parameter change is defined by some law that accounts for the fitness landscape as sampled by the algorithm; and 3) self-adaptive parameter control: whenever the parameters are, themselves, inserted into the chromosome and made subject to evolutionary operators. Clearly, the use of parameter control also implies the possibility of tuning the genetic operators to each phase of the evolutionary process. This technique is particularly interesting in trajectory design problems where, especially in the early design phases, a great number of different problem instances are studied, each one with its own different and peculiar landscape that need an adapted evolutionary process. In the experiments, the self-adaptive differential evolution (jDE) [29] and the adaptive parameter control covariance matrix adaptation evolutionary strategy (CMA-ES) [30] are adopted: two advanced approaches that make use of parameter control and that have been identified to be suitable to tackle single-objective trajectory problems. In the context of multi-objective interplanetary trajectory problems, the use of nondominated sorting or of the Pareto strength indicator has been proposed. A more recent evolutionary approach to these problem is the multi-objective evolutionary algorithm by decomposition (MOEA/D) framework, which is argued to be a better choice in many problems with simple and complex Pareto fronts [19], but that was never tested in connection to interplanetary trajectory problems. In this Note, the use of MOEA/D is also proposed, showing its performances relatively to NSGA II [24], a well-known state-of-the art algorithm using crowding distance and nondominated sorting. A short description of some of the less-known (to the aerospace engineering community) algorithms used in this work is given next. The results are also compared against the use of nonevolutionary techniques. In particular, a monotonic basin hopping algorithm (MBH) is used, a widely studied algorithm that is found to perform very well in the context of interplanetary trajectory optimization [31,32] and which is also briefly described. Note that most evolutionary algorithms that do not make use of parameter control are likely to produce similar results, or worse, with respect to MBH on the problem class here considered. In a recent work [33], a comparison between many techniques confirm this is the case. For this reason, MBH is used as a benchmark for the proposed algorithms. A. Self-Adaptive Differential Evolution

In the differential evolution algorithm (variant rand/1/exp), a mutant vector vi is created for each population individual from three randomly selected chromosomes: vi  xr1 i  Fxr2 i − xr3 i  where r1 , r2 , and r3 denote the indexes of the three randomly selected chromosomes. Then, a trial vector is created by performing an exponential crossover between xi and vi . This is done setting ui  xi ,

Fig. 1 Visualization of the two ideal points of the Pareto front.

J. GUIDANCE, VOL. 38, NO. 4:

selecting a random starting index L and applying ui;L  vi;L with probability CR. L is then increased and the substitution applied again until the probability check fails. The new trial vector ui is then compared with xi and, if better, substituted to it. In the self-adaptive version of differential evolution, as proposed in [29], the parameters F and CR are also part of the chromosome, but they do not evolve subject to the same operators, rather their values for the next generation are set as ( Fnew  ( CRnew 

F

if τ > rand1

0.1  0.9 rand2

otherwise

F

if τ > rand1

rand3

otherwise

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

where τ  0.1. The jDE algorithm was used in [13] to evolve a solution to a very complex interplanetary trajectory problem involving more than 100 flybys. B. Covariance Matrix Adaptation Evolutionary Strategy

The CMA-ES algorithm creates, at each generation k, a new population sampling from a multivariate normal distribution N mk ; σ k Ck . The mean m, the step σ, and the covariance matrix C are then adapted at each generation. As an example, the average m is set to be the weighted average of the best λ individuals. The adaptation of σ and C is more complex and its details can be found in [30]. Although CMA-ES is not well known in the aerospace community, it has been consistently winning competitions for derivative-free black-box optimization in the past decade.

795

ENGINEERING NOTES

each MBH step (following the methodology outlined by Deb et al. [24]). Because we are comparing derivative-free methods in this work, numerical differentiation is used to obtain the necessary gradients for the SQP steps and the necessary objective function/ constraints evaluations are accounted for.

IV.

Constraints Handling Techniques

Evolutionary algorithms are by nature unconstrained optimization procedures. Hence, there has been an effort to find a way to incorporate constraints into the evolutionary process [36]. A straightforward approach is to include penalty factors in the fitness formulation. However, this approach has some limitations, mainly due to the need for tuning of the (problem-dependent) penalty parameters. Therefore, more advanced constraints handling schema have been devised. The trajectory optimization problem discussed in this work is not a highly constrained problem because it is not difficult to reach feasibility with an unlimited budget. Thus, a wide feasible region with multiple local optima is expected. The difficulty of the problem lies in solving the multimodal problem to optimality within the feasible region. In the following, four techniques are discussed: two belonging to the class of penalty methods and two biologically inspired techniques (coevolution and immune system). These are generic methodologies that can be applied to any population-based heuristic method because they do not redefine the specific algorithm operators. Indeed, they just design alternative formulations of the fitness function or collaborative optimization schema among partitions of the given population. A. Death Penalty and Static Penalty

C. Multi-Objective Evolutionary Algorithm Based on Decomposition

The rejection of infeasible individuals is the easiest way to handle constraints and its named death penalty. When a solution violates the constraints, its fitness is updated as follows:

MOEA/D [19] associates to each population chromosome a decomposed problem (single objective):

f x  fx  KΔcvio

Pi:

minimize : fxjλi ; z  subject to : lb ≤ x ≤ ub

where z ∈ Rnf is the reference point, whose components are defined as zi  max fi x in the search space given by the box constraints. The MOEA/D-DE variant that uses the Tchebycheff decomposition method is considered, so that fxjλi ; z   max1≤i≤nf λi jfi − zi j The algorithm loops over the population, creating new trial chromosomes by applying a best/1/bin differential evolution step, where the two needed chromosomes are selected from the neighborhood of the current individual i defined in terms of the Euclidean distance between the weight vectors. Each trial chromosome is then compared, using the decomposed fitness, to the chromosomes in the neighborhood and, if better, is inserted. A maximum number of two reinsertions are allowed per trial chromosome to maintain diversity in the population. D. Monotonic Basin Hopping

MBH [34] is a nonevolutionary optimization technique that starts a local optimization algorithm A multiple times from points obtained perturbing a current best solution vector xbest. This point is updated whenever A converges to a better solution. The algorithm is stopped whenever m1 successive runs of A do not produce any improvement. The algorithm performance is heavily dependent from the choice of the local technique A and the method to perturb xbest. In the tests, the sequential quadratic programming (SQP)-based local optimization algorithm SNOPT [35] is used, limiting its major iterations to p  100. Each component of xbest is perturbed randomly in an interval of width m2  5% the bounds width and m1  100 is used. These parameters were chosen manually, tuning the algorithm performance over several runs. Constraints, when present, are passed to the SQP method and accounted for when selecting the new best at

where K is a large constant value. In this way, the infeasible solutions are discarded during the evolution process regardless of their degree of infeasibility and how close they are to the feasible regions. If no feasible solution is available in the current population, the evolution will not proceed any further because all solutions have been assigned the same fitness value. This approach is therefore applicable to constrained problems where the feasible region is a large portion of the whole search space and a feasible solution is present in the initial population. The preceding technique can be easily extended defining one penalty parameter for each constraint, in which case we speak about static penalty. The corresponding fitness function is given by f x  fx 

ni X

Ki max0; ci x

i0

where K i is a set of values defined at the beginning of the optimization process and which will remain constant during the entire evolution. The evolution process is very sensitive to the choice of such coefficients: A heavy penalization will push the solutions very quickly inside the feasible region without giving the possibility to explore its boundary regions or eventually disjointed feasible areas. On the other hand, too weak of a penalization can affect the efficiency of the whole evolution because time will be spent exploring infeasible regions of the search space. Therefore, the estimation of the penalty coefficients requires an a priori knowledge about the problem and its degree of infeasibility, representing the main drawback and limitation of such approach. B. Coevolution

More advanced adaptive penalty approaches have been developed to overcome the limitations of the previously presented techniques. In particular, the coevolution method proposed in [16] makes use of two populations, P1 and P2 ; the first encodes the penalty coefficients, and the second encodes the optimization variables. In such a schema, a population Pj2 is associated to each individual j of the first population.

796

J. GUIDANCE, VOL. 38, NO. 4:

Let yj  yj0 ; : : : ; yjni  be the encoding of the jth individual of P1 , then the individuals of Pj2 evolve according to the following rule: f x  fx 

ni X

yji max0; ci x;

x ∈ Pj2

i0

The adopted formulation diverges from the original formulation of Coello Coello [16], where the aggregate violation and the number of violated constraints were used as the two penalized terms. The proposed approach instead encodes in the penalty population a penalty coefficient for each constraint. The updating rule for the fitness of each individual of population P1 is a measure of the infeasibility of the population Pj2 , namely, f yj  

X fx  nf nf j

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

x∈F P2 

where F Pj2  is the set of feasible individuals in Pj2 and nf  jF Pj2 j. The two populations are evolved cooperatively toward feasibility and optimality. The main drawback of such approach is its efficiency, both in space requirement and number of function evaluations. C. Immune Systems

Hybrid techniques are evolutionary techniques coupled with other numerical methods (usually local searches or other numerical optimizations methods) that are able to handle constrained optimization problems. The immune system is a hybrid technique emulating the biological behavior of an immune system where the antigene molecules are recognized by the system and antibodies are generated to combat their growth [18]. In evolutionary optimization, this is reproduced by assigning to each individual a fitness value that is a measure of how good they are in recognizing antigenes. Hence, the initial population is divided in two pools, one containing the feasible individuals (antigene) and one containing the infeasible ones. The infeasible solutions are then subject to immune system simulations where they try to bind to the antigenes, the feasible solutions. The binding and the generation of antibodies is based on a bit-by-bit or Euclidean distance matching process. Therefore, the population of antibodies generated after the immune system simulation will be able to recognize antigenes in the population and evolve toward immunity, hence feasibility.

V.

Experiments

In the first place, the constrained problem P c is investigated. In the first experiment, the death penalty constraint handling technique is used in combination with both algorithms CMA-ES and jDE using parameter control. In preliminary experiments, other popular algorithms that do not have parameter control, such as PSO (with constriction factor and ring topology), genetic algorithms (with roulette selection, Gaussian mutation, and binomial crossover), simulated annealing, and differential evolution, have also been tested, but these algorithms did not reach satisfactory performances, thus they have been excluded from the set of final experiments. The implementation of all algorithms is made available as part of the open source scientific library PaGMO‡ and its python front end PyGMO.§ For the first experiment, an increasing number of individuals NP  f20; 50; 100g and death penalty as the constraint handling technique are used. Both algorithms are run up to convergence defined by tolerances on the chromosomic and fitness variability xtol  10−6 , ftol  10−6 of the offspring. The algorithms are restarted up to when the total function evaluation budget fbudget  107 is consumed. Restarts happen consistently for CMA-ES because ‡ Data available online at https://github.com/esa/pagmo/wiki [retrieved 1 November 2013]. § Data available online at http://esa.github.io/pygmo/ [retrieved 1 November 2013].

ENGINEERING NOTES

the algorithm is able to quickly converge to one point in the solution space, whereas jDE is less likely to converge with few function evaluations, especially for larger population sizes. The concatenation of algorithmic runs (or restarts) up to fbudget is called one algorithmic trial. One hundred trials are performed, recording the percentage that achieved to converge to the target value within a given tolerance. A second set of the same experiment is run fixing the population size to NP  50 for jDE and NP  100 for CMA-ES and using different constraint handling techniques: static penalty, death penalty, coevolution, and immune systems. To investigate the performance of the different constraint handling techniques, the empirical cumulative distribution functions of the number of function evaluations needed to find the best trajectory in a successful trial is computed, similar to what is requested in black-box optimization benchmarking [37]. The function target value is defined as ft  min f  Δf, where min f  0 m∕s is the minimum ΔV solution (ballistic, in the analyzed case) and Δf  5 m∕s a convergence threshold. The empirical cumulative distribution function (ECDF) is then given by n 1X f1 if fevali < feval ∧ fi < ft else 0g F^ n feval   n i1

where fi is the minimum function value reached in trial i with fevali function evaluations. The ECDF gives the probability of convergence within the target value for a number of function evaluations less than or equal to feval . The expected run time [ERT, also referred to as SP2] is also computed with respect to the number of function evaluations. In particular, the formulation in [38] has been followed, where ERT is defined as ERT 

1 − ps fbudget  ERTs  ps

where ERTs  is the expected number of function evaluations for a successful trial and ps is the probability of convergence to the target value ft . A second experiment is then performed on the problem P mo . In preliminary experiments, the direct application of multi-objective algorithms (NSGA II and MOEA/D-DE) on this problem failed to get any good approximation of the Pareto front. The problem complexity is very high and single-objective optimization techniques are already struggling to find the minimum ΔV solution. Thus, it is expected that a multi-objective approach, dealing concurrently with ΔV and the constraint violation,¶ cannot find the two ideal points and the Pareto front. A seeding technique is considered where the initial population already contains the ideal points. In the context of preliminary mission analysis, these ideal points are likely to be known when the tradeoff analysis is requested. The task of the multi-objective approach is then to reconstruct the front in between the ideal points. The hypervolume indicator is used to measure the quality of a nondominated front. The hypervolume represents, for a two-objectives problem, the area in the fitness space dominated by the front. Larger hypervolumes are thus representing better fronts. Its efficient computation is the subject of a very active research field, and we here use the exact methods implemented in the PaGMO library. To compute the hypervolume, nondimensional units are introduced in both objectives. This way, the hypervolume will be in the 0; 1 interval. In particular, 100 independent trials of NSGA II and MOEA-D/DE have been run to a maximum function evaluation budget of fbudget  5 · 107 . To compute the hypervolume of the resulting Pareto fronts, the objective space is rescaled by mapping the points (0, 280), (250, 280), (0, 310), and (250, 310) to the unit square. The hypervolume is computed in the rescaled space with respect to the reference point R  1; 1. As mentioned earlier, the seeding ¶ Essentially, only the time of flight violation is considered because the constraint on the minimum Jupiter distance turned out to be much easier to satisfy.

J. GUIDANCE, VOL. 38, NO. 4:

technique is fundamental to the success of the multi-objective evolution. One may argue that the added complexity of finding the ideal points upfront beats the purpose of later performing a multiobjective optimization. In reality, the nondominated front shows a lot more information than the two ideal points, for example, discontinuities that are typically related to changes in the trajectory strategy or the convexity of the front that determines the interesting areas where objectives can be traded off satisfactorily among each other.

VI.

Table 2 Expected run time for 100 trials of CMA-ES and jDE with different population sizes Algorithm CMA-ES NP  100 jDE NP  50 jDE NP  20 jDE NP  100 CMA-ES NP  50 MBH-SNOPT CMA-ES NP  20

ps 1.00 0.92 0.75 0.74 0.63 0.26 0.08

Figure 2 shows the ECDF for CMA-ES and jDE for population sizes NP  f20; 50; 100g. In all cases, death penalty is used as the constraint handling technique. A ranking of the algorithms can be extracted computing the ERT as reported in Table 2. It can be easily seen how CMA-ES is very effective when coupled to a large population size NP  100, ranking as the best approach for this selected test case, and particularly bad with small population sizes, ranking as the worst algorithm. The jDE, on the other hand, performs relatively well with a small population size, ranking as second (NP  50) and third (NP  20) in terms of ERT. The main reason for this overall ranking can be found on the number of restarts each algorithm is able to make within fbudget as detailed in Table 3. Such a ranking is expected to be rather sensitive to the stopping criteria

Table 3 Average number of restarts per trial NP

Restarts Algorithm jDE

20 50 100

5.77 2.48 1.21 Algorithm CMA-ES

20 50 100

Empirical cumulative distribution function for CMA-ES and

2.54 24.32 47.71

Table 4 Expected run time for CMA-ES with NP  100 for different constraint handling techniques Algorithm CMA-ES static penalty CMA-ES death penalty CMA-ES immune system CMA-ES coevolution

Fig. 2 jDE.

RTs ERT 976,990 976,990 3,701,622 4,571,187 3,058,893 6,392,226 7,429,054 10,942,567 3,633,948 9,506,964 3,661,997 32,123,535 3,003,612 118,003,612

Results

A. Constrained Evolution

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

797

ENGINEERING NOTES

RTs ERT ps 1.00 738,650 738,650 1.00 976,990 976,990 1.00 1,504,739 1,504,739 0.88 2,428,595 3,857,166

defined for the single runs. In the case of jDE, the criteria on ftol and xtol may not be appropriate because they force the algorithm to use function evaluations even when the search has lost its effectiveness. The ECDF for different constraint handling techniques and CMA-ES with NP  100 is then shown in Fig. 3. A ranking of the different techniques can be extracted computing the expected run time as reported in Table 4. Death penalty and static penalty show equal performance. The immune system technique requires slightly more function evaluations but eventually matches the performance of static penalty for feval > 2 · 106. Coevolution suffers from added complexity and requires significantly more function evaluations to reach the same convergence percentage. For jDE with NP  50, the same ranking of static penalty, immune system, death penalty, and coevolution is obtained. The ECDF is shown in Fig. 4. Tables 2, 4, and 5 give the ERT computed over 100 trials with fbudget  107 , which confirm the preceding considerations. From the preceding results, it is possible to conclude that, although the static penalty method does perform very well, the death penalty and immune system approach (not requiring domain knowledge to choose an additional parameter) are equally suitable and should thus be preferred. B. Multi-Objective Evolution

Fig. 3 Empirical cumulative distribution function of CMA-ES (NP  100) using different constraint handling techniques.

Figure 5 shows a box plot of the normalized hypervolume as computed during 100 single runs of MOEA/D-DE and NSGA II and sampled every 5 · 106 function evaluations. In this case, it was necessary to change the comparison methodology because the ERT and the ECDF could not be computed because there is no global optimal hypervolume value known (i.e., the Pareto front is completely unknown). Thus, the algorithms have been run 100 times each and the hypervolume trend recorded. Results show that MOEA/ D-DE clearly outperforms NSGA II. The best hypervolume found by

798

J. GUIDANCE, VOL. 38, NO. 4:

ENGINEERING NOTES 310

Time of Flight [days]

305

300

295

290

285

280 0

100

150

200

250

Fig. 6 Best seeded Pareto front NSGA II. Normalized hyper volume  0.636.

310

305

Time of Flight [days]

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

Fig. 4 Empirical cumulative distribution function of jDE (NP  50) using different constraint handling techniques.

50

300

295

290

285

280

Fig. 5

Hypervolume obtained by NSGA II and MOEA/D.

NSGA II is 0.636, whereas MOEA/D-DE achieves 0.672. The best nondominated fronts found are shown in Figs. 6 and 7 for NSGA II and MOEA/D-DE, respectively. Although NSGA II shows plateaus in the Pareto front, MOEA/D-DE is capable of reconstructing slopes that allow the tradeoff between ΔV and time of flight. The nondominated fronts are discontinuous. The four distinct areas in the nondominated set correspond to different resonance strategies at the moon Ganymede. The third trajectory leg, a Ganymede–Ganymede transfer, is in fact always close to a 1∶ν resonant transfer. This means that the spacecraft makes one orbit while Ganymede revolves ν times. The discontinuity points along the nondominated front correspond to ν ∈ 11, 10.76, 10, 9, 8, 7. Although both algorithms are able to locate these important switches, only MOEA/D-DE can, within one resonance, find small adjustment in the other encoded variables, resulting in a slope and thus in a much higher hypervolume. Note that the ν  11 resonance is, at a certain point, dominated by a nonTable 5 Expected run time for jDE with NP  50 for different constraint handling techniques Algorithm jDE immune system jDE static penalty jDE death penalty jDE coevolution

ps 0.94 0.93 0.92 0.72

RTs 3,218,928 3,425,462 3,701,622 6,666,666

ERT 3,857,226 4,178,150 4,571,187 10,555,555

0

50

100

150

200

250

Fig. 7 Best seeded Pareto front MOEA/D-DE. Normalized hyper volume  0.672.

resonant transfer having ν  10.76, accomplished using some ΔV immediately after the Ganymede encounter. This opportunity does not seem to exist for the other resonances. To conclude, in the multiobjective case, the reconstructed nondominated front is not necessarily the Pareto front (which is unknown). The plan is to release the code to the community with the challenge to find better hypervolumes.

VII.

Conclusions

Evolutionary techniques that emerged in the last decade are able to evolve increasingly complex trajectory designs while dealing with multiple mission objectives and constraints. In the case of the Jupiter capture mission here considered, well-known methods, such as differential evolution, particle swarm optimization, monotonic basin hopping, or nondominated sorting genetic algorithm, are all incapable of delivering satisfactory results and one must resort to different, more recent approaches. In particular, parameter control, implemented in algorithms such as CMA-ES and jDE, allows to offload from the designer the parameter tuning task, while efficiently searching the solution space and should thus always be used in singleobjective evolutionary optimization. Death penalty methods or immune system techniques can deal with constraints having performances similar the more common static penalty method, but without requiring any problem domain knowledge and should thus be preferred. Problem decomposition is found to be a much better technique than nondominating sorting in the context of the design of missions with multiple objectives. In all cases, seeding using the problem ideal

J. GUIDANCE, VOL. 38, NO. 4:

points, that is, the two optimal trajectories with respect to the single objectives, is extremely beneficial.

Downloaded by ESA-ESTEC on July 21, 2015 | http://arc.aiaa.org | DOI: 10.2514/1.G000619

References [1] Minovitch, M., “Alternative Method for Determination of Elliptic and Hyperbolic Trajectories,” NASA TM-312-118, Feb. 1961. [2] Minovitch, M., “Method for Determining Interplanetary Free-Fall Reconnaissance Trajectories,” NASA TM-312-130, Feb. 1961. [3] Rauwolf, G. A., and Coverstone-Carroll, V. L., “Near-Optimal LowThrust Orbit Transfers Generated by a Genetic Algorithm,” Journal of Spacecraft and Rockets, Vol. 33, No. 6, 1996, pp. 859–862. doi:10.2514/3.26850 [4] Coverstone-Carroll, V., “Near-Optimal Low-Thrust Trajectories via Micro-Genetic Algorithms,” Journal of Guidance, Control, and Dynamics, Vol. 20, No. 1, 1997, pp. 196–198. doi:10.2514/2.4020 [5] Biesbroek, R. G., and Ancarola, B. P., “Optimization of Launcher Performance and Interplanetary Trajectories for Pre-Assessment Studies,” IAF Abstracts, 34th COSPAR Scientific Assembly, Vol. 1, 2002, p. 40, http://adsabs.harvard.edu/abs/2002iaf..confE..40B. [6] Lee, S., Russell, R. P., Fink, W., Terrile, R. J., Petropoulos, A. E., and Von Allmen, P., “Low-Thrust Mission Trade Studies with Parallel, Evolutionary Computing,” IEEE Aerospace Conference, IEEE Publ., Piscataway, NJ, 2006, p. 12. doi:10.1109/AERO.2006.1656038 [7] Lee, S., Fink, W., Russell, R. P., Von Allmen, P., Petropoulos, A. E., and Terrile, R. J., Evolutionary Computing For Low-Thrust Navigation, Jet Propulsion Lab., NASA, Pasadena, CA, 2005, http://ntrs.nasa.gov/ search.jsp?R=20060050337. [8] Sentinella, M. R., and Casalino, L., “Cooperative Evolutionary Algorithm for Space Trajectory Optimization,” Celestial Mechanics and Dynamical Astronomy, Vol. 105, Nos. 1–3, 2009, pp. 211–227. doi:10.1007/s10569-009-9223-4 [9] Betts, J. T., “Survey of Numerical Methods for Trajectory Optimization,” Journal of Guidance, Control, and Dynamics, Vol. 21, No. 2, 1998, pp. 193–207. doi:10.2514/2.4231 [10] Conway, B. A., Spacecraft Trajectory Optimization, Vol. 32, Cambridge Univ. Press, Cambridge, England, U.K., 2010. [11] “GTOC Portal,” ESA, 2014, http://sophia.estec.esa.int/gtoc_portal/ [accessed 1 May 2014]. [12] Petropoulos, A. E., “Problem Description for the 6th Global Trajectory Optimisation Competition,” http://sophia.estec.esa.int/gtoc_portal/wpcontent/uploads/2012/11/gtoc6_problem_stmt-2.pdf. [13] Izzo, D., Simões, L. F., Märtens, M., de Croon, G. C., Heritier, A., and Yam, C. H., “Search for a Grand Tour of the Jupiter Galilean Moons,” GECCO 2013: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, ACM, New York, 2013, pp. 1301–1308, http://dl.acm.org/citation.cfm?id=2463524. [14] “Annual HUMIES Awards,” 2014, http://www.genetic-programming .org/combined.php [accessed 1 May 2014]. [15] Eiben, A. E., Hinterding, R., and Michalewicz, Z., “Parameter Control in Evolutionary Algorithms,” IEEE Transactions on Evolutionary Computation, Vol. 3, No. 2, 1999, pp. 124–141. doi:10.1109/4235.771166 [16] Coello Coello, C. A., “Use of a Self-Adaptive Penalty Approach for Engineering Optimization Problems,” Computers in Industry, Vol. 41, No. 2, 2000, pp. 113–127. doi:10.1016/S0166-3615(99)00046-9 [17] Hoffmeister, F., and Sprave, J., “Problem-Independent Handling of Constraints by Use of Metric Penalty Functions,” EP’96: Proceedings of the Fifth Annual Conference on Evolutionary Programming, MIT Press, Cambridge, MA, Feb. 1996, pp. 289–294. [18] Hajela, P., and Lee, J., “Constrained Genetic Search via Schema Adaptation: An Immune Network Solution,” Structural Optimization, Vol. 12, No. 1, 1996, pp. 11–15. doi:10.1007/BF01270439 [19] Li, H., and Zhang, Q., “Multiobjective Optimization Problems with Complicated Pareto Sets, MOEA/D and NSGA-II,” IEEE Transactions on Evolutionary Computation, Vol. 13, No. 2, 2009, pp. 284–302. doi:10.1109/TEVC.2008.925798 [20] Eberhart, R., and Kennedy, J., “New Optimizer Using Particle Swarm Theory,” Proceedings of the Sixth International Symposium on Micro Machine and Human Science (MHS’95), IEEE Publ., Piscataway, NJ,

ENGINEERING NOTES

[21] [22]

[23]

[24]

[25]

[26]

[27]

[28] [29]

[30]

[31]

[32]

[33]

[34] [35]

[36]

[37]

[38]

799

1995, pp. 39–43. doi:10.1109/MHS.1995.494215 Goldberg, D. E., and Holland, J. H., “Genetic Algorithms and Machine Learning,” Machine Learning, Vol. 3, No. 2, 1988, pp. 95–99. doi:10.1023/A:1022602019183 Corana, A., Marchesi, M., Martini, C., and Ridella, S., “Minimizing Multimodal Functions of Continuous Variables with the Simulated Annealing Algorithm,” ACM Transactions on Mathematical Software (TOMS), Vol. 13, No. 3, 1987, pp. 262–280. doi:10.1145/29380.29864 Storn, R., and Price, K., “Differential Evolution–A Simple and Efficient Heuristic for Global Optimization Over Continuous Spaces,” Journal of Global Optimization, Vol. 11, No. 4, 1997, pp. 341–359. doi:10.1023/A:1008202821328 Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T., “Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II,” IEEE Transactions on Evolutionary Computation, Vol. 6, No. 2, 2002, pp. 182–197. doi:10.1109/4235.996017 Lynam, A. E., Kloster, K. W., and Longuski, J. M., “Multiple-SatelliteAided Capture Trajectories at Jupiter Using the Laplace Resonance,” Celestial Mechanics and Dynamical Astronomy, Vol. 109, No. 1, 2011, pp. 59–84. doi:10.1007/s10569-010-9307-1 Wagner, G. P., and Altenberg, L., “Perspective: Complex Adaptations and the Evolution of Evolvability,” Evolution, Vol. 50, No. 3, 1996, pp. 967–976. doi:10.2307/2410639 Izzo, D., “Global Optimization and Space Pruning for Spacecraft Trajectory Design,” Spacecraft Trajectory Optimization, 2010, pp. 178– 201, Chap. 7. doi:10.1017/CBO9780511778025 Battin, R. H., Introduction to the Mathematics and Methods of Astrodynamics, AIAA, Reston, VA, 1999, pp. 295–342. Brest, J., Greiner, S., Boskovic, B., Mernik, M., and Zumer, V., “SelfAdapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems,” IEEE Transactions on Evolutionary Computation, Vol. 10, No. 6, 2006, pp. 646–657. doi:10.1109/TEVC.2006.872133 Hansen, N., and Kern, S., “Evaluating the CMA Evolution Strategy on Multimodal Test Functions,” Parallel Problem Solving from NaturePPSN VIII, Springer–Verlag, Berlin, 2004, pp. 282–291. doi:10.1007/978-3-540-30217-929 Addis, B., Cassioli, A., Locatelli, M., and Schoen, F., “Global Optimization Method for the Design of Space Trajectories,” Computational Optimization and Applications, Vol. 48, No. 3, 2011, pp. 635–652. doi:10.1007/s10589-009-9261-6 Yam, C. H., Di Lorenzo, D., and Izzo, D., “Constrained Global Optimization of Low-Thrust Interplanetary Trajectories,” IEEE Congress on Evolutionary Computation (CEC), IEEE Publ., Piscataway, NJ, 2010, pp. 1–7. doi:10.1109/CEC.2010.5586019 Vasile, M., Minisci, E., and Locatelli, M., “On Testing Global Optimization Algorithms for Space Trajectory Design,” AIAA/AAS Astrodynamics Specialist Conference and Exhibit, AIAA Paper 20086277, 2008, http://arc.aiaa.org/doi/abs/10.2514/6.2008-6277. Leary, R. H., “Global Optimization on Funneling Landscapes,” Journal of Global Optimization, Vol. 18, No. 4, 2000, pp. 367–383. doi:10.1023/A:1026500301312 Gill, P. E., Murray, W., and Saunders, M. A., “SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization,” SIAM Review, Vol. 47, No. 1, 2005, pp. 99–131. doi:10.1137/S0036144504446096 Coello Coello, C. A., “Theoretical and Numerical Constraint-Handling Techniques Used with Evolutionary Algorithms: A Survey of the State of the Art,” Computer Methods in Applied Mechanics and Engineering, Vol. 191, Nos. 11–12, 2002, pp. 1245–1287. doi:10.1016/S0045-7825(01)00323-1 Hansen, N., Auger, A., Finck, S., and Ros, R., “Real-Parameter BlackBox Optimization Benchmarking: Experimental Setup,” Institut national de recherche en informatique et en automatique (INRIA) TR 00362649v2, 2013. Auger, A., and Hansen, N., “Performance Evaluation of an Advanced Local Search Evolutionary Algorithm,” Congress on Evolutionary Computation (CEC 2005), Vol. 2, IEEE Publ., Piscataway, NJ, 2005, pp. 1777–1784. doi:10.1109/CEC.2005.1554903