Optimization. â Deterministic. -Grid search. -Branch and bound. â Stochastic. -Simulated Annealing. -Tabu Search. -Genetic Algorithm. -Statistical Algorithm ...
Global Optimization Software
Doron Pearl Jonathan Li Olesya Peshko Xie Feng
What is global optimization? Global optimization is aimed at finding the best solution of constrained optimization problem which (may) also have various local optima.
General global optimization problem (GOP) z
Given a bounded, robust set D in the real n-space Rn and a continuous function f: DÆ R ,find global min f(x), subject to the constraint x Є D
Note: robust set : the closure of its nonempty interior.
First, we have to tell you..
No single optimization package can solve all global optimization problems efficiently.
Two General Classes In Global Optimization z
Deterministic -Grid search -Branch and bound
z
Stochastic -Simulated Annealing -Tabu Search -Genetic Algorithm -Statistical Algorithm
Deterministic class and software Actually, We can further classify deterministic class into two different subclass : Explicit Function Required Such as ..Baron z
Explicit Function isn’t required Such as ..LGO( Lipschitz Global Optimization ). z
Remark: 1. In present deterministic solvers, the number of solvers in first class is more then in second class. 2. Even though LGO is regard as using deterministic way to solve the problem, the solution isn’t always guaranteed to be “deterministic” global optimal. 3. There are some more solvers in first class won’t be discussed in detail, but in later slides, they will be included in comparison.
LGO
Lipschitz Global Optimization
min f ( x) x ∈ D = {x ∈ D0 : f j ( x) ≤ 0, j = 1,..., J } D0 ∈ Rn represents a ‘simple’ explicit constraint set:
frequently, it’s a finite n-dimensional interval or simplex, or Rn. Furthermore, the objective function and constraint functions are Lipschitz-continuous on D0.That is,they satisfy the relation
f j (x1) − f j (x2 ) ≤ Lj x1 − x2
LGO
Lipschitz Global Optimization
Three Key Components in the approach: z
Lipschitz Continuous Function
z
Adaptive Partition Strategy
z
Branch and Bound
Lipschitz Continuous Function With Lipschitz Continuous Property :
f j (x1 ) − f j ( x2 ) ≤ Lj x1 − x2 We can conclude the following observation of the function: z
The ’slope’ is bounded with respect to the system input variables x.
z
If a function is Lipschitz Continuous on certain compact domain, it’s guaranteed that the bound of the function exists.
z
On the other hand, without the property, on the sole basis of sample points and corresponding function values, one cannot provide a lower bound after any finite number of function evaluations of D.
Remark: z It’s not necessary to compute L in global optimization, but the existence of it is a necessary condition to have lower bound.
Lipschitz Continuous Function
In Lipschitz continuous function , the more sample points we have , the more accurate approximation of the lower bound we can obtain.
Adaptive partition strategy z
Usually implement on the relaxed feasible set, such as: -Interval set:
a
