Solving Tough Optimal Control Problems by Network Enabled Optimization Server (NEOS) Jinsong Liang Dept. of Mechanical & Aerospace Engineering Utah State University Logan, Utah 84322-4130, USA
YangQuan Chen CSOIS, Dept. of Electrical & Computer Engineering Utah State University Logan, Utah 84322-4160, USA
[email protected]
[email protected]
Max Q.-H. Meng Department of Electrical Engineering Chinese University of Hong Kong Shatin, N.T., Hong Kong, China
Rees Fullmer Dept. of Mechanical & Aerospace Engineering Utah State University Logan, Utah 84322-4130, USA
[email protected]
[email protected]
Abstract— The Network Enabled Optimization Server (NEOS) for solving tough optimal control problems (OCPs) is introduced. Though NEOS is generally used to solve static optimization problems, in solving dynamic optimizations (OCPs) in a general form, we show that the NEOS Server has some advantages over the existing dedicated optimal control software packages. After a step by step introduction of how to solve a textbook bang-bang control problem by using NEOS, a tough fed-batch fermentation process optimal control problem is solved as a demonstration using NEOS with significantly better performance. Index Terms— Network Enabled Optimization Server (NEOS); optimal control; static optimization; dynamic optimization; fed-batch fermentation.
I. I NTRODUCTION Optimal control is an important branch in control theory. Due to its complexity, most optimal control problems have to be solved numerically. There are several existing software packages for numerically solving control problems, including SOCS [1], RIOTS [2], DIRCOL [3], or MISER3 [4]. Although these optimal control software packages are powerful and relatively easy to use, there are also limitations and disadvantages in using them. First, most of these software packages are commercial, or even costly. Second, these software packages are only capable of solving limited range of optimal control problems. To our best knowledge, no software package for optimal control can solve the globally optimal control problems. On the other hand, if appropriately discretized, all optimal control problems (OCPs) can be converted to static optimization problems, also called mathematical programming problems. Then, optimal control problems can be solved using the static optimization methods. Since there are far more software packages and individual routines available for static optimization problems, this conversion is appealing in many situations. The Network Enabled Optimization Server (NEOS) is such an excellent environment generally used for solving static optimization problems. In this paper, we introduce the main features of the NEOS Server and demonstrate through two examples how to use the NEOS Server to solve optimal control problems. The Corresponding author: Dr YangQuan Chen. E-mail:
[email protected]; Tel. 01-435-7970148; Fax: 01-435-7973054. URL:http://www.csois.usu.edu/people/yqchen.
power of the NEOS Server is especially demonstrated in the second example, which is an optimal control competition problem. To the best of our knowledge, the controller we designed using the NEOS Server has achieved the best performance among all the presented solutions. This problem is challenging to any dedicated software packages for optimal control problems. The paper is organized as follows. In Sec. II, we introduce the NEOS server. In Sec. III, the general steps using NEOS to solve optimal control problems are introduced. Section IV presents the first example solution of the textbook bangbang OCP with detailed steps applying NEOS with codes listed in the appendices. Section V presents the second example - a tough fed-batch fermentation process optimal control problem with detailed explanation of the solution process using NEOS and the achieved significantly better performance. Finally, Sec. VI concludes the paper. II. I NTRODUCTION
TO THE
NEOS S ERVER
The NEOS Server is an environment for solving optimization problems remotely over the Internet, originally launched in 1994 by the Optimization Technology Center with the support from the Department of Energy and Northwestern University [5] [6] [7]. Compared with the conventional application-based optimization methods, NEOS server has the following advantages: • It is simple. The user only needs to define the optimization problems in some popular formats, be it a programming language (FORTRAN, C, Matlab) or a modelling language (AMPL [8], GAMS [9]). The details of the solution process are handled by the NEOS Server and are transparent to the users. • It is powerful. With more than fifty solvers, the NEOS Serve covers almost every branch of optimization techniques. The solvers are based on the algorithms developed and tested by academic and commercial researchers for years and have been proven to be very robust and efficient. • It is fast. The machines running the NEOS Server usually have powerful processors and large amount of memory. Some algorithms make use of parallel computing techniques. Thus, a low-end PC is enough for the user to solve large optimization problems.
It is cheap. Users do not need to purchase any optimization software, which is usually very expensive. The NEOS Server is free. • It is flexible. The parameters of the solvers are adjustable by the users, although using the default parameters is sufficient for most optimization problems. The users can communicate with the NEOS Server via email, the web, or the submission tools running on their own computers. Intermediate results can be viewed for time-consuming problems. Jobs submitted to the NEOS Server can be killed at anytime. Currently, there are more than fifty solvers available, including linear/nonlinear, unconstrained/constrained optimization, mixed integer programming, linear network programming, non-differentiable optimization, stochastic linear programming, complementary problems, semi-infinite optimization, and global optimization. •
III. G ENERAL
STEPS USING NEOS SERVER TO SOLVE OPTIMAL CONTROL PROBLEMS
Because the NEOS Server is different from the traditional software packages running on the users machine and most solvers are for static optimization problems, using the NEOS Server to solve optimal control problems has its unique features. Generally, it takes the following steps to solve an optimal control problem. 1) Choose the appropriate solver. When choosing from more than fifty solvers, bear in mind that the optimal control problem will be converted to the static optimization problem, i.e., the control inputs, which are functions of time, will be discretized into parameters to be optimized in a static optimization process. Following issues should be considered. • The properties of the optimal control problem. If a global optimal control solution is required, a global optimization solver has to be chosen. If the control inputs are constrained by bounds, a bound constrained optimization solver is a good choice. If the optimal control problem involves trajectory/end-point constraints, a nonlinearly constrained optimization solver may be the best solver. • The input format. For a specific solver, not all input formats are acceptable. For example, a solver may require AMPL as its only input format. If for some reason AMPL can not be chosen, then another solver has to be found. 2) Convert the optimal control problem to the static optimization problem. This step is done through the discretization of the control inputs, the state differential equations, the constraints, and the objective function. The simplest discretization scheme for control inputs is by piecewise constants. Piecewise linear discretization has some advantages over the piecewise constant discretization, but it is more complicated. Other more complicated discretization methods, such as the splines technique, is generally avoided unless needed for some special applications. The state differential equations are discretized through finite difference method. If a nonlinearly constrained optimization solver is chosen,
the trajectory constraints can be directly discretized into the simultaneous equations. However, if a bound constrained optimization solver is chosen, which does not accept the general constraints on the variables, the constraints on the trajectory/control have to be penalized in the objective function. 3) Convert the mathematical description of the static optimization problem to the required input format. If several input formats are accepted, then choose the format best suited for describing the optimization problem. 4) Submit the problem via the interface provided by the NEOS Server. The NEOS Server offers four interfaces for submitting optimization problems. They are email, web browser, the NEOS Submission Tool (NST) [10], and Kestrel [11]. Email is the authors’ favorite. Users need only edit one submission file following the template and submit it via email. The NEOS Server then sends back the solution information via email. The web browser is the most popular interface for problem submission. It is easy to understand and learn. However, multiple files need to be edited and submitted via the web browser. NST is capable of monitoring the compilation and execution processes online. However, it requires the user to download and install a piece of software, which is a little trickier than the previous two methods. Kestrel is the most recent interface. It is a modeling language environment capable of choosing a solver, receiving the results, interpreting and manipulating the results directly. The disadvantage is that problems must be submitted in AMPL or GAMS format. The use of email interface is straightforward. Users first open their favorite email client (outlook express, Eudora, etc.), fill in the email address with
[email protected], paste the content of the submission file to the body, and submit the email. 5) Interpret and post-process the results sent back from the NEOS Server. If necessary, change the parameters of the optimization problem and re-submit. In the next two sections, two examples will be presented to demonstrate how to use NEOS to solve tough optimal control problems. For tutorial purposes, the first example is chosen to be the textbook bang-bang optimal control problem. IV. E XAMPLE -1: A
TEXTBOOK BANG - BANG OPTIMAL CONTROL PROBLEM
This is a textbook OCP problem [12] with control bounds and free final time. This OCP is defined as follows: min J(u, T ) = T u,T
(1)
subject to x˙ 1 = x2 ; x˙ 2 = u;
x1 (0) = 0, x1 (T ) = 300
(2)
x2 (0) = 0, x2 (T ) = 0
(3)
and −2 ≤ u(t) ≤ 1,
t ∈ [0, T ]
(4)
x˙ 1 (t) = x˙ 2 (t) = x˙ 3 (t) =
x2 (t)x3 (t) u(t)x3 (t) 0
1.5
1
0.5
0
u
where u(t) is the control input and T is the free final time. This problem has an analytical solution which is given by T ∗ = 30. When t ∈ [0, 20), u∗ (t) = 1, x∗1 (t) = t2 /2, x∗2 (t) = t while when t ∈ [20, 30], u∗ (t) = −2, x∗1 (t) = −t2 + 60t − 600, x∗2 (t) = 60 − 2t. This is a minimum-time problem with two states and one input. To convert this optimal control problem with free final time to static optimization problem, first it needs to be transcribed to a fixed final time problem by using an extra state variable. The idea is to specify a nominal time interval, [0, Tf ], and to use the extra state as a scale factor to scale the duration of the real time. The transformed state differential equations become
−0.5
−1
−1.5
−2
(5)
−2.5
where the extra state x3 (t) is actually a constant to be optimized, t ∈ [0, Tf ] is the nominal time and Tf is an arbitrarily chosen positive constant. Now the real final time becomes x3 Tf . Next we discretize the state differential equations (5) as follows. x1 [n+1]−x1 [n] = x2 [n]x3 ∆t (6) x2 [n+1]−x2 [n] = u[n]x3 ∆t
Fig. 1.
where we used the piecewise constant discretization. Clearly, (6) can be re-arranged into the following difference equation x1 [n + 1] = x2 [n + 1] =
x1 [n] + x2 [n]x3 ∆t x2 [n] + u[n]x3 ∆t
(7)
The objective function (1) becomes J = x3 Tf
(8)
Now the variables to be optimized become u[n] and x3 . Next we need to choose the proper solver in NEOS. There are several choices. We can choose the nonlinearly constrained optimization solvers and treat the fixed final state as the constraints; or we can choose the bound constrained optimization solvers and penalize the final difference between the actual states and the desired states in the objective function. We will try the first method. Specifically, here we choose the SNOPT [13] solver with FORTRAN input. If for some reasons SNOPT fails, we can try other solvers. Following the requirement of SNOPT, we need to write the subroutines evaluating the objective function and constraint functions. We also need to write the subroutines defining the starting points, the bounds on the variables, and the bounds on the constraints. All these subroutines are listed in the appendices. Finally, we choose the number of variables to be 51, i.e., 50 discretized control inputs plus the scale factor. The number of constraints is 2, i.e., two constraints on the final states. About ten seconds after submitting the above subroutines and parameters via Email, the optimized variables and the optimal value of the objective function are sent back from the NEOS Server via Email. The optimized variables is plotted in Fig. 1, which is clearly bang-bang, as we expected. The optimal value of the objective function is 30.0006, very close to the analytical optimal value T ? = 30.
0
5
10
15 time (sec)
20
25
30
Bang-bang OCP solution
V. E XAMPLE -2: A
FED - BATCH FERMENTATION PROCESS OPTIMAL CONTROL PROBLEM
This example is a competition problem raised by Biotechnological Control Forum [14]. The goal was to optimize the performance of a fed-batch fermentation process, using any technique to model the the batch process and to design the controller. Making use of the NEOS Server, the control solution we obtained has achieved the best performance among all presented solutions. For more detailed information, refer to [15]1 . Several features make this optimization problem difficult. First, the model of the plant is given as a black box model (in scrambled C source code) that simulates a fedbatch fermentation process, i.e., no mathematical model is available. The users can only apply input signal to the black box and get the corresponding output signal from the black box. Second, both the initial states of the process and the parameters of the model vary randomly from batch to batch. For the same input, the output for each batch would not be the same. The presence of randomness makes the normal gradient-based optimization methods very hard and inefficient to implement, if not impossible. Third, this is an MIMO (multiple input multiple output), nonlinear (according to its input-output properties) system. The fed-batch fermentation process produces a metabolite as the product. Two substrates (S1 and S2) are needed as the inputs. The operators have known from the experience that: • Nominal feed profiles (as given) will give a reasonable production. Feeds of substrates at too high or too low levels seem to reduce the production. • Feed rate of S2 seem to be complementary with feed rate of S1. Only feeding of S1 does not yield any production while only feeding of S2 yields small product. The goal was to investigate the optimal feed pattern in order to improve the process productivity. To study the effect of these two feeds, other process environment variables, such 1 The complete codes for this OCP is online available from http://mechatronics.ece.usu.edu /yqchen/code/miniprojectfiles/
as temperature and pH, are assumed to be kept constant at their optimum. To make the exercise comparable to working on a practical plant, the model representing the plant will not be given to the investigators explicitly. The investigators will be given the following information: • A black box plant simulator model in scrambled source code or compiled code. • A set of typical input/output data. • Some explicitly specified constraints. • Some control objectives. The process to be modeled and controlled is supplied as a black box with a set of specified inputs. Apart from the basic differential equations describing the process, there are following features inside the black box: • The initial state which varies randomly within a subspace for each batch. • The parameters of the model which vary within specified bounds. • A set of unmeasurable disturbances. • A set of constraints. The black box model is described as shown in Fig. 2. This black box will be supplied to the investigators in a scrambled C source code. The users can apply input signals to the black box and get the output signals from the black box. The users may use this information for modeling purposes as well as for controller design. Note that as the initial state vector of the process varies randomly within a subspace for each batch, and the parameters of the model vary from batch to batch as well as there are unmeasurable disturbances, the users are reminded that for the same input signal time series, they would not obtain the same output time series for each batch.
To analyze/model the black box process. To design a controller to control the process for a certain period of time (one batch) and for certain number of batches. The suggested performance criterion to evaluate the controller is • •
J=
(product concentration)*(volume) time length of a batch
As the batch process starts with different initial values for each batch and due to the process parameter variations from batch to batch, investigators are requested to produce simulation results for a number of batches. Only in this way then can they adequately verify the performance of their approaches. We will convert this optimal control problem to a nonlinearly constrained static optimization problem. The plant to be controlled is given as scrambled C code. So, only solvers accepting C/C++ code as the input format can be selected. There are two candidate solvers: BLMVM (Limited Memory Variable Metric Method) [16] and APPS (Asynchronous Parallel Pattern Search) [17]. The algorithm of BLMVM needs gradient information of the objective function. We have to rely on the automatic gradient computation via the automatic differentiation (AD) technique [18], one of the most impressive features of the NEOS Server. However, the randomness in the fedbatch fermentation process makes the gradient computation highly unreliable. The authors tried to calculate the objective function 20000 times to get the averaged value, aiming to remove the uncertainty, the BLMVM still failed. Finally, we chose the APPS solver, which does not rely on the gradient information. Being an improved algorithm to the traditional parallel pattern search algorithm, APPS is capable of solving nondifferentiable optimization problems, since it does not rely on the gradient information. Though it converges slower than gradient-based algorithm, APPS takes advantage of parallel computing, which reduces substantially the overall computational time cost. First we need to define the objective function. We add an additional tuning knob c to the original performance function (9) as the objective function: J 0 = J − c · Std(batch output), c ≥ 0
Fig. 2.
Block diagram of the black box model
The process has two inputs, U1 and U2, in terms of substrate feed rate. There are five measurements which are: X – biomass, S1 – concentration of substrate 1, S2 – concentration of substrate 2, P – product concentration and V – volume. There are two inputs in terms of substrate feed rate. There are limits on the maximum feed rate. The volume of the fermenter is also limited. Investigators are required to carry out the following tasks:
(9)
(10)
where the constant c is used to penalize the standard deviation of the batch output such that the variance of the batch output can be kept from being too big. Affected by noise, the objective function varies even for the same input profile in different batches. To overcome this problem, we simply calculate the averaged value of the objective function based on 5000 calculations of objective function for the same input. Next we choose the design variables. The given nominal model uses the sampling time of one second, which corresponds to 240 discrete control inputs. Since we need to compute the objective function 5000 times to get the average, we found that it is impractical to use the sampling time of one second, given the limit on the number of iterations on NEOS
Server. We choose the sampling time to be five seconds and the obtained results are still satisfactory. It is required that when the volume of fermenter reaches 4000, the inputs should be set to zero. This requirement could not be specified in a format accepted by APPS server, which accepts only hard constraints on the input. To work around this problem, during the computation of objective function, if the volume of fermenter reaches 4000, we choose the maximum value of the objective function obtained so far as the final objective function value and exit the calculation. By observation, after the volume of fermenter reaches 4000, it is very unlikely that the objective function will improve further. So, our proposed work-around is reasonable. Comparisons between the nominal and the optimal profile are given below. If we are only concerned about the maximum averaged J (rate of output) and do not care how big the variance of J is (c = 0), comparisons are shown in Fig. 3 and Fig. 4 for twenty simulation runs. We can see that not only the maximum average response is improved by 105% (from 8.3 × 104 to 1.7 × 105 ), but also the optimal batch period, which is the time corresponding to the maximal rate of output, is reduced from 115 to 60. We know that the reduction of batch period is highly desirable. So, the overall productivity improvement is about 292%.
nominal input U1 optimal input U1
50 40 30 20 10 0 0
20
40
60 time
80
100
120
nominal input U2 optimal input U2
50 40 30 20 10 0 0
20
40
60 time
80
100
120
Fig. 4. Nominal input vs. optimal input without variance optimization (T = 120) 5
x 10
2.5
nominal output optimal output
2
1.5 J
1 5
2.5
x 10
0.5 nominal output optimal output 2 0
0
20
40
60 time
80
100
120
1.5
Fig. 5. Nominal output vs. optimal output with variance optimization (T = 120)
J
1
nominal input U1 optimal input U1
50 40
0.5
30 20 0
0
20
40
60 time
80
100
10
120
0 0
20
40
Fig. 3. Nominal output vs. optimal output without variance optimization (T = 120)
60 time
80
100
120
nominal input U2 optimal input U2
50 40
The control of output variance is also very important in fed-batch fermentation process. Smaller output variance means better quality consistence. This is why we add an additional tuning knob c to the original performance function. If we optimize the maximum averaged response while penalizing the variance of batch response at the same time (c = 2), the results are shown in Fig. 5 and Fig. 6. We can see that although the maximum averaged response is smaller (1.5 × 105 ), the standard deviation is greatly reduced from 2.0 × 104 to 7.2 × 103 . The comparisons are summarized in Table I.
30 20 10 0 0
Fig. 6. 120)
20
60 time
80
100
120
Nominal input vs. optimal input with variance optimization (T =
TABLE I C OMPARISON BETWEEN NOMINAL CONTROL AND OPTIMAL CONTROL input
VI. C ONCLUDING R EMARKS We have introduced the NEOS Server and how to use it to solve optimal control problems. Compared with the dedicated software packages for optimal control, the NEOS Server is more flexible and is able to solve wider range
40
nominal optimal (c = 2) optimal (c = 0)
maximum of J 8.3 × 104 1.5 × 105 1.7 × 105
optimal batch time 115 80 60
standard deviation 4.6 × 103 7.2 × 103 2.0 × 104
of optimal control problems. The disadvantage of using the NEOS Server to solve optimal control problems is that significant user involvement in the solution process is required. This sometimes requires the user to understand the theory of optimal control, optimization and/or numerical approximation methods. The NEOS Server provides users with powerfulness, ease of use, and free availability. We believe that NEOS Server could be the first choice for tough optimization problems, especially in the Internet age. VII. REFERENCES [1] J. T. Betts, “SOCS: the sparse optimal control software family,” http://www.boeing.com/assocproducts/socs/index.html, Tech. Rep., 1996. [2] A. Schwartz, E. Polak, and Y. Q. Chen, RIOTS: a MATLAB toolbox for solving optimal control problems, 1997. [Online]. Available: http://www.accesscom.com/∼adam/RIOTS [3] O. v. Stryk, “DIRCOL: a direct collocation method for the numerical solution of optimal control problems,” 1999. [Online]. Available: http://www.sim.informatik.tu-darmstadt.de/sw/dircol/dircol pub.html [4] L. S. Jennings, M. E. Fisher, K. L. Teo, and C. J. Goh, “MISER3 optimal control software: theory and user manual,” 2002. [Online]. Available: http://www.cado.uwa.edu.au/miser/manual.html [5] M. C. Ferris, M. P. Mesnier, and J. J. Mor´e, “Neos and condor: solving optimization problems over the internet,” ACM Transactions on Mathematical Software, vol. 26, no. 1, pp. 1–18, 2000. [6] E. D. Dolan, R. Fourer, J. J. Mor´e, and T. S. Munson, “The neos server for optimization: version 4 and beyond,” Argonne National Library, Tech. Rep. Preprint ANL/MCS-P947-0202, 2002. [7] E. D. Dolan, “Neos server 4.0 administrative guide,” Argonne National Library, Tech. Rep. Technical Memorandum ANL/MCS-TM250, 2002. [8] R. Fourer, D. M. Gay, and B. W. Kernighan, AMPL: a modeling language for mathematical programming. Duxbury Press, 2002. [9] A. Brooke, D. Kendrick, A. Meeraus, and R. Raman, GAMS: A users guide, 2002. [10] J. Czyzyk, M. Mesnier, and J. J. Mor´e, “The neos server,” IEEE Journal on Computational Science and Engineering, vol. 5, no. 3, pp. 68–75, 1998. [11] E. D. Dolan and T. S. Munson, “The kestrel interface to the neos server,” Argonne National Laboratory, Tech. Rep. ANL/MCS-TM-248, 2001. [12] A. E. Bryson and Y. C. Ho, Applied Optimal Control. Hemisphere Publishing Corp., 1975. [13] P. E. Gill, W. Murray, and M. A. Saunders, “User’s guide for Snopt version 6, a Fortran package for large-scale nonlinear programming,” 2002. [Online]. Available: http://www.sbsi-soloptimize.com/products snopt desc.htm [14] R. Leigh, “Fed-batch fermentation process modeling and control competition – editorial,” Transactions of the Institute of Measurement and Control, vol. 20, no. 1, pp. 1–3, 1998. [15] J. Liang and Y. Chen, “Optimization of a fed-batch fermentation process control competition problem using neos,” Proc. of Inst. of Mechanical Engineers, JOURNAL OF SYSTEMS AND CONTROL ENGINEERING Part-I., vol. 217, no. (in press), pp. 1–6, 2003. [16] S. J. Benson and J. J. Mor´e, “A limited memory variable metric method for bound constrained minimization,” Argonne National Library, Tech. Rep. Preprint ANL/MCS-P909-0901, 2001. [17] P. D. Hough, T. G. Kolda, and V. J. Torczon, “Asynchronous parallel pattern search for nonlinear optimization,” SIAM Journal on Scientific Computing, vol. 23, no. 1, pp. 134–156, 2001. [18] A. Griewank, D. Juedes, and J. Utke, “Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/ C++,” ACM Transactions on Mathematical Software, vol. 22, no. 2, pp. 131–167, 1996.
A PPENDIX I THAT EVALUATES THE OBJECTIVE FUNCTION FOR SNOPT subroutine fcn ( n, x, f ) integer n double precision x(n), f double precision T T = 10.0 f = x(n)*T return end
S UBROUTINE
A PPENDIX II THAT EVALUATES THE CONSTRAINT FUNCTIONS FOR SNOPT subroutine cfcn ( n, x, m, c ) integer n, m double precision x(n), c(m) integer i double precision x1, x2, dt, T T = 10.0 dt = T/(n - 1) * ---------------------------------* Initial value of x1 and x2 * ---------------------------------x1 = 0.0 x2 = 0.0 do 100, i = 1, (n-1) x1 = x1 + x2*dt*x(n) x2 = x2 + x(i)*dt*x(n) 100 continue c(1) = x1 c(2) = x2 return end S UBROUTINE
S UBROUTINE
A PPENDIX III THAT DEFINES THE STARTING POINTS FOR SNOPT
subroutine initpt( n, x ) integer n double precision x(n) integer i do 200, i = 0, n-1 x (i) = 0.0 200 continue x(n) = 1 return end
A PPENDIX IV THAT DEFINES THE BOUNDS ON THE VARIABLES FOR SNOPT subroutine xbound( n, xl, xu ) integer n double precision xl(n), xu(n) integer i do 200, i = 1, n-1 xl(i) = -2 xu(i) = 1 200 continue S UBROUTINE
xl(n) = 1 xu(n) = 10 return end
A PPENDIX V THAT DEFINES THE BOUNDS ON THE CONSTRAINTS FOR SNOPT subroutine cbound( m, cl, cu ) integer m double precision cl(m), cu(m) integer i cl(1) = 300.0 cu(1) = 300.0 cl(2) = 0.0 cu(2) = 0.0 return end
S UBROUTINE
