Preprints, 5th IFAC Conference on Nonlinear Model Predictive Preprints, Control 5th Preprints, 5th IFAC IFAC Conference Conference on on Nonlinear Nonlinear Model Model Predictive Predictive September 17-20, 2015. Seville, SpainAvailable online at www.sciencedirect.com Control September September 17-20, 17-20, 2015. 2015. Seville, Seville, Spain Spain
ScienceDirect
IFAC-PapersOnLine 48-23 (2015) 016–025
Nonlinear Model Predictive Control of Nonlinear Model Predictive Control of Systems with Probabilistic Time-invariant Systems with Probabilistic Time-invariant Uncertainties Uncertainties Joel A. Paulson ∗∗ Eranda Harinath ∗∗ Lucas C. Foguth ∗∗ ∗ Joel A. Paulson Eranda Harinath D. Braatz∗∗ ∗∗Lucas C. Foguth ∗∗ Paulson ∗ Richard ∗ Richard Richard D. D. Braatz Braatz ∗ ∗ Massachusetts Institute of Technology, Cambridge, MA 02139 USA ∗ ∗ ∗ Massachusetts Instituteeranda, of Technology, MA 02139 USA (e-mail: jpaulson, lcfoguth, Cambridge, and
[email protected]). (e-mail: jpaulson, eranda, lcfoguth, and
[email protected]). Abstract: Many model predictive control algorithms have been formulated with the objective of Abstract:chance Many constraints model predictive control algorithms have been formulated with the objective of including and the shaping of probability density functions (PDFs) of system including constraints andmany the shaping of probability density functions of system states andchance outputs. Although algorithms consider time-varying (TV)(PDFs) uncertainty, the states Although many algorithms time-varying uncertainty, the consideration of time-invariant more challenging since(TV) the Markov property states and and outputs. outputs. Although (TI) manyuncertainty algorithmsisconsider consider time-varying (TV) uncertainty, the consideration of time-invariant (TI) uncertainty is more challenging since the Markov property cannot be applied directly to the state-space system. Additional challenges associated with cannot beTIapplied directly to the system. Additional challenges associated with handling uncertainty include the state-space incorporation of feedback, uncertainty propagation through handling TI include the feedback, through the uncertain system, the handling of hard input of and chance uncertainty constraints, propagation computational cost, handling TI uncertainty uncertainty include the incorporation incorporation of feedback, uncertainty propagation through the system, the of constraints, computational cost, and guaranteeing stability of the closed-loop systemand andchance feasibility of the control policy. Efforts the uncertain uncertain system, the handling handling of hard hard input input and chance constraints, computational cost, and guaranteeing stability of the closed-loop system and feasibility of the control policy. Efforts to employ polynomial chaos theory (PCT) to mitigate some of these challenges are reviewed. to employ polynomial theory (PCT) toanalysis mitigateusing somePCT of these challenges are reviewed. Much of this literaturechaos performs stochastic online in a receding-horizon Much performs analysis PCT receding-horizon framework. discuss differences between the PDFsusing calculated andaa the PDFs of the Much of of this thisWeliterature literature performs stochastic stochastic analysis using PCT online online in in receding-horizon framework. We differences between the calculated online and the the actual closed-loop system, and provide an example that proves that chance framework. We discuss discuss differences between the PDFs PDFs calculated online and constraints, the PDFs PDFs of ofwhen the actual closed-loop system, and provide an example that proves that chance constraints, when implemented in the receding-horizon manner, are not necessarily satisfied by the closed-loop implemented in the receding-horizon manner, are not necessarily satisfied by the closed-loop system. Opportunities for future research are suggested, which include the use of PCT for the system. Opportunities for research the of for offline of closed-loop systems andare thesuggested, modeling which of TI include uncertainty as TV uncertainty system.analysis Opportunities for future future research are suggested, which include the use use of PCT PCT for the the offline analysis of closed-loop systems and the modeling of TI uncertainty as TV uncertainty coupled with a low-pass filter. coupled with a low-pass filter. © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Model Predictive Control (MPC), Stochastic MPC, Nonlinear MPC, Polynomial Keywords: Model Predictive Control (MPC), MPC, MPC, Polynomial Chaos Theory, Chance Constraints, Uncertainty Propagation, Optimal Control Keywords: Model Predictive ControlTime-invariant (MPC), Stochastic Stochastic MPC, Nonlinear Nonlinear MPC, Polynomial Chaos Theory, Chance Constraints, Time-invariant Uncertainty Propagation, Optimal Chaos Theory, Chance Constraints, Time-invariant Uncertainty Propagation, Optimal Control Control 1. INTRODUCTION stochastic disturbances (e.g., Hashimoto, 2013; Kouvar1. stochastic disturbances (e.g., 2013; Kouvar1. INTRODUCTION INTRODUCTION itakis et al., 2010; Oldewurtel et al., 2008; Chatterjee stochastic disturbances (e.g., Hashimoto, Hashimoto, 2013; Kouvaritakis et al., 2010; Oldewurtel et al., 2008; Chatterjee Model uncertainties are inevitable, which has motivated et al., 2011). SMPC algorithms have been developed Model uncertainties are which has 2011). measurement SMPC algorithms have output been developed thatal.,consider noise with feedback the formulation of numerous model predictive control et Model uncertainties are inevitable, inevitable, which has motivated motivated the predictive control consider measurement noise with (Hokayem output feedback (Hokayem et al., 2012), affine systems et al., (MPC) algorithmsof guaranteemodel the satisfaction hard that the formulation formulation of tonumerous numerous model predictive of control al., systems (MPC) algorithms to satisfaction of 2009), and et multiplicative TV uncertainty (Cannon et et al., (Hokayem et al., 2012), 2012), affine affine systems (Hokayem (Hokayem et al., input state constraints for allthe possible realizations of (Hokayem (MPC)and algorithms to guarantee guarantee the satisfaction of hard hard input and state constraints for all possible realizations of 2009), and multiplicative TV uncertainty (Cannon et al., the Such robust (RMPC) algorithms inputuncertainty. and state constraints forMPC all possible realizations of 2009). The stability and performance obtained by stochasthe Such robust MPC algorithms The systems stabilityfor andgeneral performance obtained by with stochastic control nonlinear systems TV can be conservative to neglecting the probability the uncertainty. uncertainty. Such due robust MPC (RMPC) (RMPC) algorithms 2009). can due neglecting control systems for general nonlinear systems with and TV uncertainty have also been investigated (Chatterjee distribution of the uncertain (Cannon et al., tic can be be conservative conservative due to to parameters neglecting the the probability probability have distribution the uncertain parameters et Lygeros, 2014). uncertainty have also also been been investigated investigated (Chatterjee (Chatterjee and and 2012). RMPCof typically determined(Cannon by minimizing distribution oflaws the are uncertain parameters (Cannon et al., al., uncertainty 2012). RMPC laws determined by minimizing a2012). worst-case function, which can a low Lygeros, 2014). RMPC objective laws are are typically typically determined byhave minimizing These types of systems with TV uncertainty have several aprobability worst-case objective function, which can have a occurrence. a worst-caseof objective function, which can have a low low These types of with TV have several convenient example, consider the open-loop These typesproperties. of systems systemsFor with TV uncertainty uncertainty have several probability probability of of occurrence. occurrence. convenient properties. For example, consider the open-loop In stochastic MPC (SMPC) techniques, the probability discrete-time nonlinear system In MPC techniques, the probability distribution (PDFs) of system states outputs discrete-time nonlinear In stochastic stochasticfunctions MPC (SMPC) (SMPC) techniques, the and probability Xk+1system = h(Xk , Wk ) (1) distribution functions system and outputs are shaped in optimal(PDFs) controlof X h(X (1) distribution functions (PDFs) offormulations system states statesthat andconsider outputs k+1 = k ,, W k) X = h(X W ) (1) k+1 k k k+1 k k where W is a random TV uncertainty and all of the are control that consider information the uncertainty distributions. Hard con- where Wkk is a random TV uncertainty and all of the are shaped shaped in inonoptimal optimal control formulations formulations that consider W are independent. Since X depends only on X k k+1 k k information on distributions. Hard straints are typically relaxed in SMPC algorithms re- Wkk are independent. information on the the uncertainty uncertainty distributions. HardbyconconXk+1 depends Xkkk k+1 and Wk , then Xk+1 isSince a random variable only whoseonPDF k k k+1 straints are typically relaxed in SMPC algorithms by replacement chancerelaxed constraints, intended to guarantee straints arewith typically in SMPC algorithms by re- and Wk , then Xk+1 is a random variable whose PDF depends the state Xk and variable the PDFwhose of WkPDF . In and Wk , only thenon Xk+1 is a random placement with guarantee that the constraints are constraints, satisfied withintended a certainto placement with chance chance constraints, intended toprobability guarantee dependsk only on k+1 the state Xkkkpresent and the PDF of system, Wkkk . In other words, conditional on the state of the that the constraints are satisfied with a certain probability of occurrence. In this SMPC provide a other words, conditional on the present state of the system, that the constraints are way, satisfied withalgorithms a certain probability its past (X , Xk−2 , . . . ) and its future state Xk+1 are of In this algorithms provide tradeoff between performance and constraint of occurrence. occurrence. In closed-loop this way, way, SMPC SMPC algorithms provide aa its past (Xk−1 , Xk−2 ) and its future state Xk+1 k−1 k−2 k+1 statistically independent, k−1 k−2 , . . . i.e., k+1 are tradeoff satisfaction. tradeoff between between closed-loop closed-loop performance performance and and constraint constraint statistically k−1 independent, i.e., statistically independent, i.e., satisfaction. fXk+1 |Xk ,Xk−1 ,... (xk+1 |xk , xk−1 , ...) = fXk+1 |Xk (xk+1 |xk ). satisfaction. Most of the literature on SMPC has focused on linear fX fX k k k+1 |X k ,X k−1 ,... k+1 |X k (xk+1 k+1 k k−1 k+1 k X |X ,... X k k+1 k , xk−1 k−1 k+1 |xand k ). Xk+1 |Xk ,Xk−1 ,... (xk+1 Xk+1 |X k+1 k ,X k−1 k+1 |X k This useful property is|x known as, ...) the=Markov property, Most on has focused discrete-time systems with additive (TV) This Most of of the the literature literature on SMPC SMPC hastime-varying focused on on linear linear useful property is known as the Markov property, and forproperty which this property holds are referred as This useful is known as the Markov property,toand discrete-time discrete-time systems systems with with additive additive time-varying time-varying (TV) (TV) systems systems for this holds are referred to as Financial support is acknowledged from the NSF Graduate ReMarkov processes. processes have been studied systems for which which Markov this property property holds are referred to for as Financial support is acknowledged from the NSF Graduate Re Markov processes. Markov processes have been studied for over a hundred years, and a plethora of useful research and search Fellowship andisNovartis Pharmafrom AG.the NSF Graduate ReFinancial support acknowledged over a hundred years, and a plethora of useful research and search Fellowship and Novartis Pharma AG. Copyright 2015 IFAC 16 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2015, IFAC (International Federation of Automatic Control) Peer review©under of International Federation of Automatic Copyright 2015 responsibility IFAC 16 Control. 10.1016/j.ifacol.2015.11.257
2015 IFAC NMPC September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
17
tic nonlinear systems with TI uncertain parameters and presents a closed-loop stochastic optimal control problem for this class of systems. Section 3 discusses the challenges of this optimal control problem. An uncertainty propagation tool used to tackle these challenges, termed polynomial chaos theory (PCT), is described in Section 4. In Section 5, the PCT-based stochastic control literature is reviewed in line with the aforementioned challenges (in Section 3). Finally, an outlook for future research is discussed in Section 6.
results can be applied to design effective control strategies for these types of processes. The consideration of time-invariant (TI) uncertainty, on the other hand, results in a process that does not satisfy the Markov property. As an example, consider the simple system with a state dimension of one, Xk+1 = ΘXk , (2) and assume that Xk and Xk−1 are not equal to zero. Then the PDF of Xk+1 given Xk is 1 xk+1 fXk+1 |Xk (xk+1 |xk ) = fΘ (3) |xk | xk where fΘ indicates the prior of Θ, e.g., a normal distribution. However, the PDF of Xk+1 given Xk and Xk−1 is x2k (4) fXk+1 |Xk ,Xk−1 (xk+1 |xk , xk−1 ) = δ xk+1 − xk−1 where δ(·) denotes the Dirac delta function. This latter equation holds because knowledge of both Xk−1 and Xk allows exact knowledge of the parameter from Θ = Xk /Xk−1 , so then the exact value of Xk+1 can be determined with certainty.
Notation. Hereafter, N = {1, 2, . . .} is the set of natural numbers; N0 := N ∪ {0}; R≥0 and R>0 are the set of nonnegative and positive real numbers, respectively; Z[a,b] := {a, a + 1, . . . , b} is the set of integers from a to b; E[·] is the expected value; Var[·] is the covariance matrix; Pr[·] denotes probability; f(·) is the PDF of any random variable; the symbol ∼ means “distributed as”; N (µ, Σ) is the Gaussian distribution with mean µ and covariance Σ; D(a) is the Dirac delta distribution at a; and · p and · are the standard p-norm and Euclidean norm, respectively. Capital letters are used to denote stochastic random variables and lower case letters to denote their realizations unless otherwise clear from the context.
Clearly the above PDFs are not equivalent except in very special cases (e.g., the distribution of Θ is a Dirac delta function), indicating that uncertain systems with TI uncertainty are not necessarily Markov processes.
2. PROBLEM STATEMENT Consider a discrete-time nonlinear dynamical system described by (5) Xk+1 = h(Xk , uk , Θ), X0 ∼ D(x0 ) where k ∈ N0 is the discrete-time index, X denotes the random vector associated with the system states x ∈ Rnx , u ∈ Rnu are the system inputs, Θ is a Rnθ -valued random vector of TI system parameters with known PDF fΘ and support, S ⊆ Rnθ h : Rnx × Rnu × Rnθ → Rnx is a known nonlinear algebraic function, and x0 ∈ Rnx is initial states which are assumed to be known. To focus on the technical issues that arise when considering TI probablistic uncertainties without complicating distractions, the states are assumed to be observed exactly at any time. Some comments on removing that assumption are presented in the future outlook section. Remark 1. Although the explicit control problem formulations are presented for discrete-time systems (5) in this section, the formulations straightforwardly extend to continuous-time systems. Examples of continuous-time formulations are provided in Section 5 where the literature on SMPC for systems with TI uncertainties is reviewed.
This example can be used to illustrate some additional points concerning systems with TI uncertainty. Once X1 and X2 are measured, the parameter Θ can be determined with certainty by taking their ratio, and then all future states are known with certainty. Even with exactly known state x0 , all future states are probablistic at time k = 0, before a Θ is realized, as no future states are known with certainty at that time instance. Once a value of Θ has been realized, however, the system is completely deterministic with no randomness, as Θ does not change for all future time instances. In other words, after a particular value of Θ has been realized, the “true” PDF of Θ is a Dirac delta function—it is just the position of this Dirac delta function that is unknown at k = 0. This “mismatch” between the deterministic plant and the stochastic process model arises due to the TI nature of the uncertainty, and does not arise for TV uncertainty. For TV uncertainty, the system remains uncertain for all future time instances even if the states were measured exactly at all time instances, in which case the Markov property can be used to greatly simplify the stochastic analysis.
Although Xi (x0 , u0 , . . . , ui−1 , Θ) is a function of the initial state x0 , input sequence u0 , . . . , ui−1 , and random parameter Θ, these explicit functional dependencies are dropped for notational convenience. Note that Θ ∼ fΘ is the only source of uncertainty appearing in Xi .
An immediate consequence of this discussion is that, for systems with TV uncertainty, the PDFs of the future states actually have physical meaning, as “true” PDFs describing a frequency of occurrence given current information. In contrast, for systems with TI uncertainty, after a value of Θ has been realized, non-degenerate PDFs of future states only serve as an expression of subjective belief. These differences in the philosophical interpretation of probability theory are important prerequisites for a solid understanding of probabilistic uncertain processes and their analysis/control when the uncertainties are TI.
Hard bounds on the input are considered (6) uk ∈ U, ∀k ∈ N0 nu where U ⊆ R is the set of input constraints. We also consider joint chance constraints on the state (7) Pr[Xk ∈ X] ≥ β, ∀k ∈ N where X ⊆ Rnx is any set of state constraints and β ∈ (0, 1) is the lower bound on the probability that this collection of constraints must be satisfied. For example, for the
This paper is organized as follows. Section 2 discusses a general problem formulation for discrete-time stochas17
2015 IFAC NMPC 18 September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
polytopic constraint set X := {x ∈ Rnx : Hx ≤ g} for H ∈ Rr×nx and g ∈ Rr for r > 1, all r inequalities need to be satisfied jointly with a probability of at least β.
(1) Feedback. The first challenge associated with solving Problem 1 is the incorporation of feedback into the optimization. A well-known fact in the RMPC community is that feedback can be used to drastically decrease the conservatism of RMPC algorithms (Scokaert and Mayne, 1998; Lee, 2014), and the same is certainly true of SMPC algorithms.
Optimal control methods are a natural way to stabilize the system (5) to the origin. This article discusses methods for solving these type of model-based control problems stochastically by incorporating the statistical descriptions of the parametric uncertainty. These approaches enable shaping the PDF of the states, which is critical for seeking tradeoffs between closed-loop performance and robustness to system uncertainties. Below is a very general optimal control formulation.
As stated in Problem 1, µ are arbitrary functions within a class of state feedback policies, which cannot be directly determined in a standard optimization solver. Theoretically, µ could be constructed by solving the dynamic programming (aka Bellman) equations, derived from the principle of optimality. However, these equations can be exactly solved only for very simplified problems. Approximate dynamic programming methods have been developed and related to model predictive control strategies (e.g., see discussion by (Lee, 2014)) but are very computationally expensive, so feedback is more commonly incorporated via parameterization of the control law.
Problem 1 (Closed-loop stochastic optimal control): Let N ∈ N denote the prediction horizon of the optimal control problem. A full state feedback control law µ is defined, over horizon N , by (8) µ := {µ0 , µ1 (·), . . . , µN −1 (·)} where µ0 ∈ U is a control action as x0 is known and i nx µi : → Rnu is some feedback control law k=0 R that maps all previous states to the input. Given the known initial state x0 and parameter PDF fΘ , the general stochastic optimal control problem is cast as (9) min J(x0 , µ) µ s.t. X0 ∼ D(x0 ), Xk+1 = h(Xk , µk (X0 , . . . , Xk ), Θ), ∀k ∈ Z[0,N −1] Pr[µk (X0 , . . . , Xk ) ∈ U] = 1, Pr[Xk+1 ∈ X] ≥ β, Θ ∼ fΘ
A common parameterization is linear state feedback −1 µi (xi ) = Ki xi + ci for all i = 0, . . . , N − 1 where {Ki }N i=0 N −1 and {ci }i=0 become the decision variables in the optimization (9). This explicit parameterization makes the optimization problem more tractable, but is suboptimal and very restrictive. It is also difficult to choose an explicit parameterization a priori that will yield good performance for general nonlinear systems. Another common suboptimal control law parameterization is the receding-horizon control policy whereby the control inputs are computed based on optimizing over model predictions from the most recently measured state. This MPC method, in the context of (5), is cast as follows.
where J(x0 , µ) is the objective function of interest, which will typically be of the form N −1 J(x0 , µ) = E c(Xk , µk (X0 , . . . , Xk )) + cF (XN )
Problem 2 (SMPC): Let u := {u0 , u1 , . . . , uNp −1 } be the input values (to be optimized) over the prediction horizon Np ∈ N. Given a current state x ∈ Rnx and the initial parameter PDF fΘ , the SMPC problem is stated as min J(x, u) (10)
k=0
where c : Rnx × U → R≥0 is a stage cost function and cF : Rnx → R≥0 is a final cost function.
The solution to (9) generates an optimal sequence of feedback control laws µ := {µ0 , µ1 (·), . . . , µN −1 (·)}. The specific input applied to the system (5) at a given time depends on the realization of the state, i.e., µk (x0 , . . . , xk ), which is assumed to be perfectly measured so as to focus on the specific technical issues associated with TI probablistic uncertainty descriptions. Note that {xi }i≥0 denote the realized states for a given realization of the uncertainty parameter Θ, and are implicitly a function of Θ.
u
¯ k , uk , Θ), ∀k ∈ Z[0,N −1] ¯ k+1 = h(X s.t. X p uk ∈ U, ∀k ∈ Z[0,Np −1] ¯ k ∈ X] ≥ β, Pr[X ∀k ∈ Z[1,Np ] ¯ X0 ∼ D(x), Θ ∼ fΘ
¯ are stochastic model predictions of the state. This where X problem defines an implicit receding-horizon control law that is the first part of the optimal input trajectory u (x) from Problem 2 such that the closed-loop system, for a given parameter realization Θ = θ in (5), becomes xk+1 = h(xk , u0 (xk ), θ), ∀k ∈ N0 (11) from the known initial condition x0 . Remark 2. Problem 2 is formulated with the input u as a sequence of values that can be directly optimized over using a standard solver. This formulation can straightforwardly be extended to control laws that are explicitly parametrized as functions of the predicted state, with the main difference being that the input constraints must hold for all Θ ∈ S as the predicted state is a function of Θ.
Challenges that arise when constructing and solving Problem 1 can be broadly categorized as (i) incorporation of state feedback, (ii) uncertainty propagation, (iii) implementation of state chance constraints, (iv) algorithms for fast implementation, and (v) guaranteed stability and recursive feasibility. In the literature, these challenges have been addressed in different ways for various problem classes. The below sections thoroughly describe these challenges and how they have been addressed in the literature for stochastic control with probabilistic TI uncertainty. 3. CHALLENGES IN STOCHASTIC CONTROL WITH PROBABILISTIC TIME-INVARIANT UNCERTAINTY
As shown in Section 6, Problem 2 may not satisfy state chance constraints in the closed-loop system, which can be intuitively understood by noting the predicted states
Below are challenges associated with solving Problem 1. 18
2015 IFAC NMPC September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
¯ i }N with X ¯ 0 ∼ D(xk ) from measurement xk for {X i=0 any k ≥ 1 does not match the actual state distribution {Xi }k+N i=k from (5) regardless of control law.
which holds for random variables X with mean µ and variance σ 2 for any a ∈ R\{0}. The union bound (aka Boole’s inequality) can also been used to separate joint chance constraints into a series of individual chance constraints, but can be very conservative in certain applications.
(2) Uncertainty propagation. The second challenge is the accurate propagation of probabilistic uncertainty through the uncertain system. As a simple example, consider propagating uncertainty through a discrete-time system a single step forward in time, (12) X1 = A(Θ)x0 + B(Θ)u0 , where Θ is a continuous random variable with a known PDF. Given x0 and u0 , the distribution of X1 will be some function of the uncertain parameter, X1 (x0 , u0 , Θ). Determining the distribution of X1 is then equivalent to calculating a derived distribution.
(4) Fast implementation. Another challenge associated with Problems 1 and 2 is the computational cost of the algorithm. In order to be practical, algorithms need to be fast enough for real-time implementation. This concern is particularly apparent in sampling-based algorithms, which typically require a large number of simulations in order to obtain information about distributions. Ensuring convexity of the optimization is useful when implementing controllers in receding-horizon fashion as efficient algorithms have been developed for convex problems that are guaranteed to find the globally optimal solution.
When Θ is a single random variable and X1 (x0 , u0 , Θ) has certain nice properties (e.g., linear or strictly monotonic) then expressions for the derived distribution can be derived analytically. However, in general, the procedure for determining a derived distribution requires calculating the cumulative density function (CDF) of X1 (x0 , u0 , Θ), fΘ (θ)dθ, (13) FX1 (x1 ) =
(5) Guaranteed stability and feasibility. The fifth challenge associated with Problems 1 and 2 involves ensuring stability of the closed-loop system and recursive feasibility of the online optimization. In RMPC, robust stability is usually guaranteed by guaranteeing stability for all possible realizations of the parameters. This guarantee can be provided by using, for example, robustly positively invariant sets, tubes, or in some cases by guaranteeing stability for each vertex of the uncertainty set (Langson et al., 2004; Campo and Morari, 1987; Kothare et al., 1996). In contrast, in Problems 1 and 2, parameters can lie in a distribution with an unbounded support. Guaranteeing stability for all possible realizations of the parameters is challenging and results in a large amount of conservatism. In the worst case, some of the possible parameter values may result in a realization of an unstabilizable system, making the control design problem unsolvable.
{θ|A(θ)x0 +B(θ)u0 ≤x1 }
and differentiating to obtain the desired PDF. This procedure can be very difficult in general, and the calculation becomes even more difficult for higher dimensional Θ, where convolutions and other more complicated operations become necessary. This procedure is further complicated by the fact that the distributions need to be known as a function of the initial state and the input sequence when solving an SMPC problem such as Problem 2.
Since propagating entire distributions exactly through uncertain systems is intractable for all but the most simple systems, two main classes of methods have been used to tackle this challenge. This first set of methods involves sampling-based techniques, such as Monte Carlo or scenario approaches. The second set of methods, rather than attempting to propagate entire distributions through the uncertain system, propagates statistics of the distributions, for example, the means or the covariances.
These considerations are further complicated by hard bounds on the input that must be satisfied regardless of the system uncertainty. Based on this discussion, it may be beneficial to require stability in a probabilistic sense. A nonconservatism analysis, however, is at least as challenging as implementing a chance constraint nonconservatively.
(3) Chance constraints. The third challenge is the implementation of chance constraints. RMPC requires that state constraints x ∈ X hold for all possible realizations of the uncertainty. In order to combat the conservatism resulting from this robust implementation, an alternative approach is to implement chance constraints of the form (7), which guarantee satisfaction of constraints with a given probability β. Even if a PDF for the state is available, these chance constraints are typically difficult to implement since they are, in general, nonconvex. The nonconvexity of these constraints has inspired the use of probabilistic bounds for implementing these constraints. Particularly useful bounds include variations on the Markov inequality (Rice, 2006) E[X] Pr[X ≥ a] ≤ , (14) a which holds for random variables X that take only nonnegative values, and the Chebyshev inequality σ2 Pr |X − µ| ≥ a ≤ 2 , (15) a
19
4. UNCERTAINTY PROPAGATION USING POLYNOMIAL CHAOS THEORY 4.1 Generalized Polynomial Chaos Theory Let (Ω, F, P ) be a probability space where Ω is the sample space, F is the σ-algebra of the subsets of Ω, and P is the probability measure. Let ∆(ω) : (Ω, F) → (Rn , B n ) be an Rn -valued continuous random variable where n ∈ N and B n is the σ-algebra of the Borel subsets of Rn . PCT can be used to represent a general second-order stochastic process ∞ ak Φk (∆(ω)), (16) ψ(ω) = k=0
viewed as a function of the random event ω (Xiu and Karniadakis, 2002), where ak denotes the expansion coefficients and Φk (∆(ω)) denotes the multivariate polynomial basis functions in terms of the random variables ∆(ω). The functions {Φk (∆(ω))} are an orthogonal basis in L2 (Ω, F, P ), i.e.,
19
2015 IFAC NMPC 20 September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
E[Φi Φj ] = E[Φ2i ]δij (17) where δij is the Kronecker delta and E[·] is the expected value with respect to probability measure dP (ω) = f∆ (∆(ω))dω and PDF f∆ (∆(ω)). Henceforth, ∆(ω) is denoted by ∆ for notational convenience. According to the Cameron-Martin theorem (Cameron and Martin, 1947), the series expansion (16) converges to any L2 functional in the L2 sense. When the polynomials are selected from the Wiener-Askey scheme, the expansion exhibits the fastest convergence rate (Xiu and Karniadakis, 2002). Table 1 shows the Wiener-Askey polynomials corresponding to particular distributions in ∆. The inner product induced by the PDF of ∆ is equivalent to the expectation Φi Φj = Φi (∆)Φj (∆)f∆ (∆)d∆ = E[Φi Φj ] (18) Ω
such that orthogonality on this inner product holds by (17), i.e., Φi Φj = Φ2i δij . For practical implementation, the polynomial chaos (PC) expansion (16) is approximated using a finite number of expansion terms, L ˆ ψ(∆) ≈ ψ(∆) := ak Φk (∆). (19) k=0
The total number of terms L + 1 is (n + m)! , (20) L+1= n!m! where m ∈ N is the highest order of the polynomial basis functions in the truncated expansion (19).
Fig. 1. General description of the PCT method, to systematically select the polynomial order to achieve a desired level of accuracy.
The PC coefficients {ak } can be determined either by using sample-based techniques, such as probabilistic collocation methods, or by evaluating integrals analytically. Galerkin projection can be used for linear and polynomial stochastic systems. A thorough discussion of methods for computing PC coefficients can be found in (Kim et al., 2013).
the state PDFs can be approximated with a collection of truncated PC expansions (19) in terms of Θ. The PC coefficients will be functions of time (e.g., in the context of optimal control formulations, see Fisher and Bhattacharya, 2009; Mesbah et al., 2014; Paulson et al., 2014), and PCT can be utilized to analyze and design control techniques for stochastic systems such as (5).
One of the main advantages of PCT is the ability to approximate statistical moments of the stochastic variable ψ(∆) efficiently (Fisher and Bhattacharya, 2009). For example, the orthogonality property of {Φk (∆)} in (17) can be used to show that ˆ (21) E[ψ(∆)] ≈ E[ψ(∆)] = a0
Example. As an example, consider the simple TI uncertain system (23) Xk+1 = Θ1 Xk + Θ22 + 2 where X0 ∼ D(1), Θ1 ∼ N (0.5, 0.01), and Θ2 ∼ N (0, 0.25). This model can be written in terms of standard normal random variables ξ1 and ξ2 as
ˆ Var[ψ(∆)] ≈ Var[ψ(∆)] =
L
a2k E[Φ2k ].
Xk+1 = (0.01ξ1 + 0.5)Xk + 0.25ξ22 + 2.
(22)
Because the uncertainties are Gaussian, the polynomial chaos expansion (PCE) basis functions should be written in terms of Hermite polynomials. The PCE for X1 is exact when a second order (or higher) PCE is employed:
k=1
Fig. 1 summarizes the basic steps of PCT, and provides a systematic way to select the number of coefficients L + 1.
X1 = ψ(ξ1 , ξ2 ) = 2.75 + 0.1ξ1 + 0.25(ξ22 − 1) (25) = 2.75 + 0.1Φ1 (ξ1 ) + 0.25Φ2 (ξ2 ).
4.2 PCT for Dynamic Systems As the system (5) is a function of TI random variables (with a stationary distribution), the dynamic evolution of
For systems in which the PCE cannot be derived exactly (or when it is inconvenient to do so), any of the methods described above can be used to calculate the coefficients of the expansion.
Table 1. Choice of polynomial basis {Φk (∆)} based on the distribution of uncertainty ∆. PDF Beta Chebyshev Gamma Gaussian Uniform
Support (−1, 1) (−1, 1) (0, ∞) (−∞, ∞) [−1, 1]
(24)
When a particular value of the parameters Θ are realized, the resulting value of ψ can be approximated by substituting in the corresponding value of ξ in the PCE ˆ expansion ψ(ξ). For example, consider the case where Θ1 and Θ2 realize values of 0.6 and 0.25, respectively, which correspond to ξ1 = 1 and ξ2 = 0.5. In this case, the value
Wiener-Askey polynomial Jacobi Chebyshev Laguerre Hermite Legendre
20
2015 IFAC NMPC September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
21
The distributional results were compared to results obtained using direct Monte Carlo and Monte Carlo applied to a power series expansion. The PCE provided better distributional information than a power series expansion at a much lower computational cost than direct Monte Carlo. The times required to compute the distribution of process outputs over an entire batch run for these different methods are listed in Table 2. The reduction in computational cost using PCT was about three orders of magnitude (compare first and last rows in the table). Although Nagy and Braatz (2007) mention that this propagation technique can be used in online control to shape the final distribution of the outputs, such a control algorithm was not presented.
realized by x1 will be x1 = ψ(1, 0.5) = 2.75 + 0.1(1) + 0.25(0.52 − 1) = 2.6625.
These types of substitutions can be extremely useful. For example, they allow distributions to be estimated by performing Monte Carlo on the PCE, which can be much more computationally efficient than performing Monte Carlo on the full original model.
Figure 2 shows the evolution of the PDF of Xk as a function of time k. This evolution was calculated using collocation to calculate the PCE coefficients and then using Monte Carlo sampling on the PCE to determine the distribution. Notice that PCT is not limited by skewed distributions or distributions with long tails.
Table 2. Cost of different approaches for computing the distribution of process outputs during a batch run. Reproduced with modifications from Nagy and Braatz (2007). Method Monte Carlo simulation of full dynamic model (80,000 points) First-order power series expansion (50 confidence regions) Monte Carlo applied to second-order power series expansion (80,000 points) Monte Carlo applied to polynomial chaos expansion
Computational Time 8 hr 1s 4 min 20 s
5.2 Stochastic NMPC with Chance Constraints Mesbah et al. (2014) extended Nagy and Braatz (2007) to stochastic NMPC (SNMPC) with chance constraints for discrete-time nonlinear systems (Problem 2). Individual state chance constraints were considered, which are of the form (7) with X = {x ∈ Rnx : c x ≤ d}, i.e.,
Fig. 2. Illustration of PCE’s ability to capture dynamic evolution of uncertain systems. The main advantage of the PCT framework is that it is a non-sampling based approach that can approximate (with arbitrary accuracy) the evolution of dynamic state uncertainty introduced by TI uncertain parameters. Using PCT, stochastic dynamics can be transformed into deterministic dynamics of a higher dimensional state space. This increase in dimensionality is often significantly lower than that of sampling-based methods due to the optimality of the basis functions (Table 1). This advantage becomes even more prevalent as nθ increases.
where c ∈ R
Pr[c Xk ≤ d] ≥ β, ∀k ∈ N
nx ×r
(26)
and d ∈ R . r
As discussed in detail in Section 3, these probabilistic chance constraints are nonconvex and difficult to evaluate due to the integral form of the probability expression. Mesbah et al. (2014) mitigated this problem by replacing the individual chance constraints with a conservative deterministic approximation that is distributionally robust (i.e., holds for all distributions with a particular mean and variance). This deterministic expression was derived in Theorem 3.1 of Calafiore and El Ghaoui (2006) and is based on the first two moments of the distribution such that (26) can be guaranteed to hold when c E[Xk ] + κ c Var[Xk ]c ≤ d (27) is satisfied for all k ∈ N where κ = β/(1 − β).
The next section reviews the literature on the analysis and control of linear and nonlinear stochastic systems (5) using PCT with primary emphasis on model predictive control. 5. ANALYSIS AND CONTROL OF TIME-INVARIANT STOCHASTIC SYSTEMS USING PCT 5.1 Analysis of Stochastic Control Methods One of the first applications of PCT to control problems analyzes the distribution of closed-loop finite-time processes with uncertain parameters, which was demonstrated for a simulated batch crystallization process (Nagy and Braatz, 2007). An optimal input profile of the process was determined offline. PCT was then applied to the closedloop system using a collocation method to calculate the coefficients of the expansion. The Monte Carlo method was then used on the PCE model to approximate distributions of several process outputs.
PCT was used to approximate statistics of the uncertain state as a function of the input, which greatly improves the tractability of SNMPC problems of the form of Problem 2 as the moments, used in the objective function and (27), can be efficiently computed from the PCE coefficients. For a nonlinear system, the coefficients are most easily computed using a non-intrusive method (Monte Carlo sampling or a specified set of collocation points). Nonintrusive methods avoid the integrals required in the 21
2015 IFAC NMPC 22 September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
intrusive methods (e.g., Galerkin projection), which can be quite difficult to compute for a variety of nonlinearities. The algorithm of Mesbah et al. (2014) assumed perfect state measurements. At every sampling time, the model was initialized with the observed state. The system was then simulated over the time horizon N ∈ N for s ∈ N uncertainty samples. The PCE coefficients were estimated over N from this simulation data using least-squares. As statistics of the uncertain state can be approximated using these PCE coefficients, the SNMPC optimization can be solved for the optimal input values over N . However, the state statistics are a function of the inputs so that every optimization iteration, which directly computes the objective and constraints for a candidate input profile, requires repeated simulation of the system and estimation of the PCE coefficients. This approach can be computationally expensive depending on the system size, type of nonlinearity, and number of samples s. This algorithm is even more expensive for a receding-horizon implementation where only the first input in the sequence is applied to the system and entire procedure is repeated at the next time point.
Fig. 3. Histograms of the control objective function for (a) NMPC using a nominal model (b) SNMPC using algorithm from Mesbah et al. (2014) based on 100 closed-loop simulations. In each simulation, the plant has fixed parameter values that are obtained at the beginning of a run by drawing a single random sample from the parameter PDFs. Reproduced with modifications from Mesbah et al. (2014).
Mesbah et al. (2014) applied the above algorithm to the stochastic control of polymorphic transformations in batch crystallization. The batch crystallization dynamics for each crystal type (i.e., α-form and β-form crystals of L-glutamic acid) are described by a partial differential equation known as the population balance model that can be converted to a set of nonlinear ODEs using the method of moments, and coupled with a solute mass balance that is directly written as an ODE. Five of the kinetic parameters were treated as Gaussian random variables whose statistics were determined using parameter estimation.
evaluate the objective function and state constraints in the resulting optimization problem, were computed using a Monte Carlo sampling technique. Hard state constraints were replaced with chance constraints of the form (7). The chance constraints were first written in terms of the state PCE coefficients and then evaluated using samples. Furthermore, an offline method was presented for tightening the chance constraints to achieve “guaranteed feasibility probability.” For example, consider the chance constraint Pr[g(X) ≤ 0] ≥ β, (28) where X is the random state vector. Then, the proposed offline method can be used to find an upper bound for β, βcorr > β so that when a new chance constraint Pr[g(X) ≤ 0] ≥ βcorr , (29) is satisfied, the original chance constraint (28) will be satisfied with a given confidence level. The offline constraint tightening method is completely decoupled from the optimization problem, and it is a simple statistical correction for finite sampling.
The control objective was to minimize the mass of β-form crystals nucleated at the end of the batch, while preventing the nucleation and growth of α-form crystals and avoiding the dissolution of β-form crystals during polymorphic transformation. This latter objective was incorporated into the control optimization problem via a chance constraint on the solute concentration. Specifically, the solute concentration was required to be below the solubility curve of α-form crystals with at least 95% probability. Results for this example showed that the proposed SNMPC approach resulted in a tighter objective function distribution (Figure 3) and lower constraint violation than that of the NMPC approach when using the nominal system model. Although the example resulted in 99% of closed-loop simulations meeting the solubility chance constraint, so the chance constraint was satisfied, we show in Section 6 that such satisfaction is not generally guaranteed due to the recedinghorizon implementation of the controller.
Streif et al. (2014) state that the offline method can be used to systematically find a sample size, but the procedure was not clearly presented. The sampling-based optimization problem was not formulated in the article, and it is not obvious how to implement the proposed samplingbased SNMPC algorithm. Moreover, other implementation issues such as state feedback and initialization are not discussed. The proposed SNMPC algorithm was compared with nominal NMPC in a simulation case study. For this case study, it was observed that constraints were not violated when using the SNMPC algorithm while constraints were violated 4.8% of the time using nominal NMPC.
5.3 Fast implementation An SNMPC algorithm for continuous-time nonlinear systems is presented by Streif et al. (2014). The objective function was formulated in terms of moments of the state distribution. Galerkin projection was used to construct an analytic set of ODEs that describe the evolution of the PCE coefficients over time for a faster implementation compared to Mesbah et al. (2014), which can be done when the nonlinear dynamics can be transformed to a polynomial form. The statistics of the states, needed to 22
2015 IFAC NMPC September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
23
provide any guarantee that the chance constraints are actually fulfilled by the closed-loop system, that is, taking the effect of the recursive computations into account. A proof of this assertion is shown by counterexample below.
An extremely fast SMPC algorithm is presented by Paulson et al. (2014) for continuous-time linear systems described by differential algebraic equations (DAEs) of high state dimension. The control objective function was a setpoint tracking problem formulated in terms of the mean and variance of the system outputs. The Galerkin projection was first used to determine a deterministic surrogate model for the PCE coefficients of the state (that is a sparse system of linear DAEs). This deterministic model was used to find a low-dimensional finite step response (FSR) model between the inputs and the PCE coefficients of the outputs. An extension of the quadratic dynamic matrix control (QDMC) algorithm was developed using the FSR model for the output PCE coefficients. As the proposed SMPC algorithm has an online cost that is indepenent of the process state dimension, it is inherently much faster than its counterparts (based on state-space models) for systems with large state dimension.
Theorem 1. Let u (x) be the optimal input trajectory for Problem 2 for any x ∈ Rnx and let κN : Rnx → Rnu be the receding-horizon control law κN (x) = u 0 (x) defined by the first part of this optimal trajectory from Problem 2. Then, the stochastic closed-loop system, derived by applying κN to (5), given by (30) Xk+1 = h(Xk , κN (Xk ), Θ), ∀k ∈ N0 , may not satisfy the state chance constraints (7), even though they are included in the open-loop optimization. PROOF. This result is proven by finding any dynamic system of the form (5), parameter PDF fΘ , state chance constraint of the form (7), input constraint (6), and prediction horizon Np ∈ N such that its receding-horizon implementation allows a feasible input sequence that produces states that do not satisfy the chance constraints in the closed-loop. Consider the scalar system Xk+1 = ΘXk + uk , X0 ∼ D(1), ∀k ∈ N0 ,
The input-output SMPC algorithm was applied for control of an end-to-end continuous pharmaceutical manufacturing plant with nearly 8000 states. For 200 closed-loop simulations, the fast SMPC algorithm yielded much less variability in the active pharmaceutical ingredient (API) and production rate than nominal QDMC. As the fast SMPC optimization was solved in less than one second at every iteration, it could easily be implemented in realtime to obtain improved performance in the presence of probabilistic uncertainty (Paulson et al., 2014).
fΘ (θ) = 0.5δ(θ − 1) + 0.5δ(θ − 2),
Pr[Xk ≤ 2] ≥ 0.5, uk ∈ R, N = 2. From any initial condition x ∈ R, the PDFs of the ¯ 2 , as functions of inputs u0 and ¯ 1 and X predicted states X u1 , can be derived to be x1 ) = 0.5δ(¯ x1 − (x + u0 )) fX¯ 1 (¯ + 0.5δ(¯ x1 − (2x + u0 )),
6. OUTLOOK FOR FUTURE RESEARCH Any particular linear or nonlinear model predictive control algorithm can be categorized in terms of practical value and theoretical rigor. QDMC, for example, is one of the most widely applied advanced controllers for real industrial processes (Garcia and Morshedi, 1986). This extremely practical algorithm is easy to implement and often results in very good performance when applied to real industrial processes. From a theoretical perspective, however, the algorithm is less advanced than other more complicated (and perhaps less practically useful) algorithms that have elegant proofs of stability or recursive feasibility. Research on both practically useful and theoretically rigorous algorithms are necessary to advance the field of control, and the two areas can inform and complement each other.
fX¯ 2 (¯ x2 ) = 0.5δ(¯ x2 − (x + u0 + u1 )) + 0.5δ(¯ x2 − (4x + 2u0 + u1 )). For the first iteration of Problem 2, we know that x = 1 as X0 ∼ D(1) always takes on the value 1. We can select u = [−1.1, 0.3] as a feasible input sequence that satisfies the chance constraint over the horizon. However, only the first input u0 = −1.1 will be supplied to the system in the receding-horizon implementation. There are two cases to explore in the second iteration as Θ will realize a particular value, either 1 or 2. First, consider the case where Θ realizes a value of 1 in the true system, which yields x1 = −0.1. Initializing the predicted state PDFs from x = −0.1 above, we can select u = [2.2, 0] as a feasible input sequence. When implementing u1 = 2.2 on the true system, x2 = 2.1 so the desired constraint x2 ≤ 2 is violated when the parameter is TI with a value 1.
The majority of research on SMPC for systems with TI uncertainty has been focused on generating practical algorithms, such as reducing the online computational cost to be feasible, and demonstrating improved robustness to uncertainties compared to nominal MPC algorithms. Mesbah et al. (2014); Streif et al. (2014); Fisher (2008) have produced algorithms which, in simulation studies, consistently and in some cases quite drastically outperformed nominal algorithms. The theoretical development of these types of algorithms, however, could be significantly improved. In particular, the MPC algorithms lack stability or recursive feasibility proofs or consideration of the truncation error associated with PCE approximations. Although chance constraints have been incorporated into each step in receding horizon control systems, such as by Mesbah et al. (2014), these implementations do not
Now, consider the case where Θ realizes a value 2 in the true system, which yields x1 = 0.9. Now, initializing the predicted state PDFs from x = 0.9 above, u = [1.1, 0] is feasible as it satisfies constraints over the horizon. After implementing the first input u1 = 1.1 on the true system, x2 = 2.9 so the the state violates the desired constraint x2 ≤ 2 when the parameter is TI with a value 2. In this example, each step of the receding-horizon control policy, indicates that the chance constraint is feasible, that is, that the probability of the state at time instance k = 1 and k = 2 is greater than or equal to 50%. Under this “feasible” receding-horizon policy, the true system reaches x2 = 2.1 and x2 = 2.9 at time instance k = 2, each
23
2015 IFAC NMPC 24 September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
with 50% probability, indicating that Pr[X2 ≤ 2] = 0 such that the desired state chance constraint is violated. Each possible input sequence u results in a state x2 that violates the state constraint, so the probability that the true system actually achieving that constraint is zero. As this “feasible” control policy may be selected with some choice of objective function J (e.g., a trivial case is that J = 0), the proof is complete.
which fΘ is interpreted as a prior distribution (representing our initial belief about Θ) that can be obtained from either prior knowledge or from parameter estimation techniques such as the Markov Chain Monte Carlo method (Gelman et al., 2014). In this case, chance constraints can be used to ensure that we have a particular confidence that the state constraints will be satisfied for a particular system or batch. However, since the distribution of the predicted states cannot be interpreted as frequency of occurrence, guaranteed satisfaction of these chance constraints does not provide a direct guarantee on the deterministic system (31) for a given control law as it either meets the state constraints or does not. Problem 2 does not actually take this approach to chance constraints either, since its correct implementation would involve updating the prior fΘ based on measurements. This update could be done by calculating, after each measurement, a posterior distribution of Θ conditioned on all of the previous measurements using Bayes’ rule fXk |Θ,Πk−1 (xk |θ, Πk−1 )fΘ|Πk−1 (θ|Πk−1 ) fΘ|Πk (θ|Πk ) = fXk |Πk−1 (xk |Πk−1 )
This proof shows that chance constraints, when implemented in a receding horizon manner based on an openloop optimal control policy calculated at each time instance, are not theoretically guaranteed to be satisfied by the closed-loop system. This disconnect can be understood intuitively from the fact that the distribution of the predicted states, which are reinitialized from measured states along a trajectory corresponding to a particular Θ realization, do not match the true state distribution assumed in (5). The example does not show that feasiblity of the chance constraints will be lost for all systems and choices of objective functions. From a practical perspective, Mesbah et al. (2014); Streif et al. (2014) demonstrated that their chance constraints were met by the closed-loop system in simulation examples. Their reported good closed-loop agreement in simulation examples could be due to, for example, the conservatism of the Cantelli inequality or the biasing of the feasible input choice by the objective function. However, a more theoretically rigorous treatment could improve the algorithms and provide rigorous guarantees that the closed-loop system will satisfy chance constraints. This more rigorous treatment of chance constraints could be approached in at least two different ways corresponding to two different interpretations of probability and chance constraints.
where Πk = {x0 , x1 , . . . , xk } is the set of all state measurements up until time instant k and the recursion is initialized with the prior fΘ|Π0 = fΘ . Then, fΘ|Πk (θ|Πk ) should be used in place of fΘ (θ) in Problem 2.
Another approach for incorporating methods for the analysis of the effects of probabilistic TI uncertainties into receding-horizon control design problems is by performing the stochastic analyses on the overall closed-loop system rather than only in the optimization at each sampling instance. Such closed-loop approaches are expected to involve more complicated theoretical analyses that, to our knowledge, have not been explored in the literature. PCT offers an efficient methodology for performing these types of analyses. An open question is how to bound the approximation error over time in the PCEs as a function of the number of terms kept in the truncated expansion (19). This quantification is required for theoretical advances in guaranteed constraint satisfaction as well as controller stability and feasibility.
The first potential direction involves the interpretation of probability as a physical frequency of occurrence. This interpretation considers a collection of batches or systems, each corresponding to a different possible value of Θ and weighted according to distribution fΘ . Examples of such an application would be in the crystallization of pharmaceuticals in which the model parameters can be quite different from one batch to another due to varying upstream batches of chemicals produced from a chemical reactor, but are essentially TI within each batch. The amount of data collected during a single batch at the industrial scale is too limited in such a system to be able to identify the values of the model parameters during each batch, so a single nonadaptive model predictive control system would be desired that it is robust to the parametric uncertainties so that same control system can be used for all batches. In such cases, by implementing chance constraints, the objective is to guarantee that a certain percentage of these systems (or batches) satisfy the chance constraints. As shown in the above proof, Problem 2 does not guarantee chance constraints in this sense.
A way to avoid some of the above technical issues is to allow the probabilistically uncertain parameters to be TV. This approach is employed in the SMPC literature that considers i.i.d. process noise with no uncertain parameters (Oldewurtel et al., 2008; Hokayem et al., 2009; Chatterjee et al., 2011). While having arbitrarily fast time variation in noise is not necessarily conservative, having arbitrarily fast time variation in model parameters would be conservative for describing real physical systems. An approach to reducing conservatism would be to place low-pass filters in series with the TV parameters so that their time variation is bounded. This approach may allow a much simpler theoretical analysis and control algorithm design for dynamical systems with stochastic measurement noise and disturbances, as well as slowly time-varying probablistic model parameters.
Another potential direction involves the interpretation of probability as a quantification of subjective belief. During a given run, the system dynamics are at fixed parameter values θ , i.e., ˜ k , uk ), ∀k ∈ N0 . (31) xk+1 = h(xk , uk , θ ) = h(x
All of the theoretical issues associated with optimal control of systems with probabilistically uncertain parameters and state and input constraints become much more complicated for distributed parameter and mixed continuousdiscrete systems (so-called hybrid systems), which are
This description of the process during a given run should be distinguished from the TI uncertainty model (5) for
24
2015 IFAC NMPC September 17-20, 2015. Seville, Spain
Joel A. Paulson et al. / IFAC-PapersOnLine 48-23 (2015) 016–025
commonplace in many industries. These topics are also fairly unexplored in the literature.
25
the American Control Conference, 2413–2419. Portland, Oregon. Morari, M. and Lee, J. H. (1999). Model predictive control: past, present and future. Computers & Chemical Engineering, 23(4-5), 667–682. Nagy, Z. K. and Braatz, R. D. (2007). Distributional uncertainty analysis using power series and polynomial chaos expansions. Journal of Process Control, 17(3), 229–240. Oldewurtel, F., Jones, C. N., and Morari, M. (2008). A tractable approximation of chance constrained stochastic MPC based on affine disturbance feedback. In Proceedings of the IEEE Conference on Decision and Control, 4731–4736. Cancun, Mexico. Paulson, J. A., Mesbah, A., Streif, S., Findeisen, R., and Braatz, R. D. (2014). Fast stochastic model predictive control of high-dimensional systems. In Proceedings of the IEEE Conference on Decision and Control, 2802– 2809. Los Angeles, California. Qin, S. J. and Badgwell, T. A. (2003). A survey of industrial model predictive control technology. Control Engineering Practice, 11(7), 733–764. Scokaert, P. O. M. and Mayne, D. Q. (1998). Min-max feedback model predictive control for constrained linear systems. IEEE Transactions on Automatic Control, 43(8), 1136–1142. Streif, S., Karl, M., and Mesbah, A. (2014). Stochastic nonlinear model predictive control with efficient sample approximation of chance constraints. ArXiv:1410.4535. Xiu, D. and Karniadakis, G. E. (2002). The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal on Scientific Computing, 24(2), 619–644. Campo, P. J. and Morari, M. (1987). Robust model predictive control. In Proceedings of the American Control Conference, 1987, 1021–1026. Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2014). Bayesian Data Analysis, volume 2. Taylor & Francis. Langson, W., Chryssochoos, I., Rakovic, S. V., and Mayne, D. (2004). Robust model predictive control using tubes. Automatica, 40(1), 125–133. Rice, J. A. (2006). Mathematical Statistics and Data Analysis. Cengage Learning. Kothare, M. V., Balakrishnan, V., and Morari, M. (1996). Robust constrained model predictive control using linear matrix inequalities. Automatica, 32(10), 1361–1379.
REFERENCES Calafiore, G. C. and El Ghaoui, L. (2006). On distributionally robust chance-constrained linear programs. Journal of Optimization Theory and Applications, 130(1), 1–22. Cameron, R. H. and Martin, W. T. (1947). The orthogonal development of non-linear functionals in series of Fourier-Hermite functionals. Annals of Mathematics, 48(2), 385–392. Cannon, M., Cheng, Q., Kouvaritakis, B., and Rakovi´c, S. V. (2012). Stochastic tube MPC with state estimation. Automatica, 48(3), 536–541. Cannon, M., Kouvaritakis, B., and Wu, X. (2009). Model predictive control for systems with stochastic multiplicative uncertainty and probabilistic constraints. Automatica, 45(1), 167–172. Chatterjee, D., Hokayem, P., and Lygeros, J. (2011). Stochastic receding horizon control with bounded control inputs: A vector space approach. IEEE Transactions on Automatic Control, 56(11), 2704–2710. Chatterjee, D. and Lygeros, J. (2014). On stability and performance of stochastic predictive control. IEEE Transactions on Automatic Control, 60(2), 509–514. Fisher, J. R. and Bhattacharya, R. (2009). Linear quadratic regulation of systems with stochastic parameter uncertainties. Automatica, 45(12), 2831–2841. Fisher, J. R. (2008). Stability Analysis and Control of Stochastic Dynamic Systems Using Polynomial Chaos. Ph.D. thesis, Texas A&M University. Garcia, C. E. and Morshedi, A. M. (1986). Quadratic programming solution of dynamic matrix control (QDMC). Chemical Engineering Communications, 46(1-3), 73–87. Hashimoto, T. (2013). Probabilistic constrained model predictive control for linear discrete-time systems with additive stochastic disturbances. In Proceedings of the IEEE Conference on Decision and Control, 6434–6439. Florence, Italy. Hokayem, P., Chatterjee, D., and Lygeros, J. (2009). On stochastic model predictive control with bounded control inputs. ArXiv:0902.3944. Hokayem, P., Cinquemani, E., Chatterjee, D., Ramponi, F., and Lygeros, J. (2012). Stochastic receding horizon control with output feedback and bounded controls. Automatica, 48(1), 77–88. Kim, K.-K. K., Shen, D. E., Nagy, Z. K., and Braatz, R. D. (2013). Wiener’s polynomial chaos for the analysis and control of nonlinear dynamical systems with probabilistic uncertainties. IEEE Control Systems, 33(5), 58–67. Kouvaritakis, B., Cannon, M., Rakovi´c, S. V., and Cheng, Q. (2010). Explicit use of probabilistic distributions in linear predictive control. Automatica, 46(10), 1719– 1724. Lee, J. H. (2014). From robust model predictive control to stochastic optimal control and approximate dynamic programming: A perspective gained from a personal journey. Computers & Chemical Engineering, 70, 114– 121. Mesbah, A., Streif, S., Findeisen, R., and Braatz, R. D. (2014). Stochastic nonlinear model predictive control with probabilistic constraints. In Proceedings of 25
