Optimal Trajectory Generation With Probabilistic ...

Optimal Trajectory Generation With Probabilistic System Uncertainty Using Polynomial Chaos James Fisher

∗

Raktim Bhattacharya†

Aerospace Engineering Department, Texas A&M University, College Station, TX 77843-3141.

Abstract In this paper we develop a framework for solving optimal trajectory generation problems with probabilistic uncertainty in system parameters. The framework is based on the generalized polynomial chaos theory. We consider both linear and nonlinear dynamics in this paper and demonstrate transformation of stochastic dynamics to equivalent deterministic dynamics in higher dimensional state space. Minimum expectation and variance cost function are shown to be equivalent to standard quadratic cost functions of the expanded state vector. Results are shown on a stochastic Van der Pol oscillator.

1

Introduction

Trajectory generation for constrained dynamical systems has been an active area of research for many years now. The problem of trajectory generation or motion planning [1] is hard due to the presence of dynamics, obstacles and uncertainty in the vehicle dynamics and environment. Motion planning under uncertainty, or decision-theoretic motion planning, refers to planning in an uncertain environment and when the mobile agent model might be uncertain or imperfectly known. Uncertainty is modeled in a deterministic worst case, nondeterministic or in a probabilistic fashion [1]. However, nondeterministic uncertainty models tend to be unduly conservative and can lead to very sluggish system performance. Thus, probabilistic models are the most widely studied, and used, models for uncertainty. Various methods from the dynamic programming literature [2] are used to solve motion planning problems under such probabilistic uncertainty. Such problems are usually referred to as Markov decision problems (MDP) or partially observed Markov decision processes (POMDP), depending on the sensing architecture and the associated observability properties of the states [3, 1]. The solution of such problems is usually computationally intractable for high dimensional state spaces because of the curse of dimensionality [3]. In this paper we focus on trajectory generation problems with probabilistic uncertainty in system parameters, in the framework of stochastic optimal control theory. The stochastic optimal control problem is transformed into a deterministic optimal control problem using polynomial chaos expansions. The resulting deterministic optimal control problem can then be solved using existing methods. Polynomial chaos (PC) was first introduced by Wiener [4] where Hermite polynomials were used to model stochastic processes with Gaussian random variables. According to Cameron and Martin [5] such an expansion converges in the L2 sense for any arbitrary stochastic process with finite second moment. This applies to most physical systems. Xiu et al. [6] generalized the result of Cameron-Martin to various continuous and discrete distributions using orthogonal polynomials from the so called Askey-scheme [7] and demonstrated L2 convergence in the corresponding Hilbert functional space. This is popularly known as the generalized ∗ Graduate † Assistant

Student, [email protected] Professor, [email protected]

1

polynomial chaos (gPC) framework. The gPC framework has been applied to applications including stochastic fluid dynamics [8, 9], stochastic finite elements [10], and solid mechanics [11, 12]. Application of gPC to control related problems has been surprisingly limited. Hover et al. [13] addresses stability & control of a bilinear dynamical system, with probabilistic uncertainty in the system parameters. In the context of stochastic optimization, Hover [14] also demonstrates use of Legendre polynomial chaos and gradient based dynamic optimization to parameterize optimal system trajectories based on probabilistic uncertainty. Hover et al. generates a family of optimal control trajectories, parameterized by the random system parameters. Thus the control trajectories are also stochastic. These control trajectories minimize the expectation of a quadratic cost function of state and control. Our work is related to Hover’s work, but here we are interested in determining a single control trajectory that minimizes a cost function based on the statistics of the system trajectories. The main contribution of this paper is the transformation of stochastic optimal control problems to equivalent deterministic optimal control problems in higher dimensional state space, which can be solved using standard numerical methods available for deterministic optimal control problems. We consider both linear and nonlinear system dynamics. For linear systems, we provide a generalized expression for the equivalent higher order deterministic system. We also provide generalized expressions for expectation and covariance of quadratic costs, in terms of the gPC coefficients. Uncertainty is assumed to be probabilistic in the system parameters. We will assume that for each type of uncertainty, the probability distribution function is known. The paper is organized as follows. We first describe the gPC framework in brief. We then discuss transformation of stochastic dynamical systems to equivalent deterministic systems, with special emphasis on linear and polynomial-nonlinear dynamical systems. This is followed by derivation of typical cost functions in stochastic optimal control problems, in the gPC framework. The paper includes numerical examples based on the stochastic Van der Pol oscillator that highlight implementation issues.

2

Wiener-Askey Polynomial Chaos

Let (Ω, F, P ) be a probability space, where Ω is the sample space, F is the σ-algebra of the subsets of Ω, and P is the probability measure. Let ∆(ω) = (∆1 (ω), · · · , ∆d (ω)) : (Ω, F) → (Rd , B d ) be an Rd -valued continuous random variable, where d ∈ N, and B d is the σ-algebra of Borel subsets of Rd . A general second order process X(ω) ∈ L2 (Ω, F, P ) can be expressed by polynomial chaos as X(ω) =

∞ X

xi φi (∆(ω)),

(1)

i=0

where ω is the random event and φi (∆(ω)) denotes the gPC basis of degree r in terms of the random variables ∆(ω). In practice, the series in eqn.(1) is truncated after p + 1 terms, which is determined by the dimension d of ∆ and the order r of the orthogonal polynomials {φk }, satisfying p + 1 = (d+r)! d!r! . The functions {φi } are a family of orthogonal basis in L2 (Ω, F, P ) satisfying the relation E[φi φj ] = E[φ2i ]δij ,

(2)

where δij is the Kronecker delta and E[·] denotes the expectation with respect to the probability measure dP (ω) = f (∆(ω))dω and probability density function f (∆(ω)). Henceforth, we will use ∆ to represent ∆(ω). For random variables ∆ with certain distributions, the family of orthogonal basis functions {φi } can be chosen in such a way that its weight functions has the same form as the probability density function f (∆). These

2

orthogonal polynomials are members of the Askey-scheme of polynomials [7], which forms a complete basis in the Hilbert space determined by their corresponding support. Table 1 summarizes the correspondence between the choice of polynomials for a given distribution of ∆ [6]. Random Variable ∆ Gaussian Uniform Gamma Beta

φi (∆) of the Wiener-Askey Scheme Hermite Legendre Laguerre Jacobi

Table 1: Correspondence between choice of polynomials and given distribution of ∆(ω) [6].

3 3.1

Stochastic Dynamics and Polynomial Chaos Linear Dynamics

Define a linear continuous time stochastic system in the following manner x(t, ˙ ∆) = A(∆)x(t, ∆) + B(∆)u(t, ∆),

(3)

where x ∈ Rn , u ∈ Rm . The system has probabilistic uncertainty in the system parameters, characterized by A(∆), B(∆), which are matrix functions of random variable ∆ ≡ ∆(ω) ∈ Rd with certain stationary distributions. Due to the stochastic nature of (A, B), the system trajectory x(t, ∆) will also be stochastic. By applying the Wiener-Askey gPC expansion of finite order to x(k, ∆), A(∆) and B(∆), we get the following approximations, x ˆ(t, ∆) u ˆ(t, ∆) ˆ A(∆) ˆ B(∆)

= = = =

p X i=0 p X i=0 p X i=0 p X

xi (t)φi (∆), xi (t) ∈ Rn ;

(4)

ui (t)φi (∆), ui (t) ∈ Rm ;

(5)

Ai φi (∆), Ai =

hA(∆), φi (∆)i ∈ Rn×n ; hφi (∆)2 i

(6)

Bi φi (∆), Bi =

hB(∆), φi (∆)i ∈ Rn×m . hφi (∆)2 i

(7)

i=0

The inner product or ensemble average h·, ·i, used in the above equations and in the rest of the paper, utilizes the weighting function associated with the assumed probability distribution, as listed in table 1. It is defined as Z hφi (∆), φj (∆)i := φi (∆)φj (∆)f (∆)d∆. D∆

The number of terms p is determined by the dimension d of ∆ and the order r of the orthogonal polynomials {φk }, satisfying p + 1 = (d+r)! d!r! . It is important to note that when u(t, ∆) is deterministic u(t) = u0 (t) with all other coefficients as zero. The n(p + 1) time varying coefficients, {xi (t)}; k = 0, · · · , p, are obtained by substituting the approximated solution in the governing equation (eqn.(3)) and conducting Galerkin projection on the basis functions {φk }pk=0 , to yield n(p + 1) deterministic linear system of equations, which given by ˙ X(t) = AX(t) + BU(t), (8)

3

where X(t) U(t)

= =

[x0 (t)T x1 (t)T · · · xp (t)T ]T , T

T

[u0 (t) u1 (t)

T T

· · · up (t) ] .

(9) (10) (11)

Matrices A ∈ Rn(p+1)×n(p+1) and B ∈ Rn(p+1)×m  HA (E0 ⊗ In ) .. −1  A = (W ⊗ In )  .

are defined as 



 HB (E0 ⊗ Im )   .. −1   , B = (W ⊗ In )  , . HA (Ep ⊗ In ) HB (Ep ⊗ Im )

(12)

where HA = [A0 · · · Ap ], HB = [B0 · · · Bp ], W = diag(hφ20 i, · · · , hφ2p i), and 

hφi , φ0 , φ0 i  .. Ei =  . hφi , φp , φ0 i

··· ···

 hφi , φ0 , φp i  .. , . hφi , φp , φp i

with In and Im as the identity matrix of dimension n × n and m × m respectively. It can be easily shown that E[x(t)] = x0 (t), or E[x(t)] = [In 0n×np ] X(t). Therefore, transformation of a stochastic linear system with x ∈ Rn , u ∈ Rm , with pth order gPC expansion, results in a deterministic linear system with increased dimensionality equal to n(p + 1).

3.2

Nonlinear Dynamics

Here we consider certain types of nonlinearities that may be present in the system model. The nonlinearities considered here are rational polynomials, transcendental functions and exponentials. If x, y are random variables with gPC P expansions gPC expansion of the expression xy can be Pp similar to eqn.(4) then theP p p written as z = xy = i=0 j=0 xi yj φi φj . Representing z = k=0 zk φk (∆), we can determine zk as zk =

1 XX xi yj hφi φj φk i. hφ2k ) i j

Using this approach, the gPC expansions for multi-variate polynomials can be determined in general. When nonlinearities involve non polynomial functions, such as transcendental functions and exponentials, difficulties occur during computation of the projection on the gPC subspace. The corresponding integrals may not have closed form solutions. In such cases, the integrals either have to be numerically evaluated or these nonlinearities are first approximated as polynomials using Taylor series expansions and then the projections are computed using methods described above. While Taylor series approximation is straightforward and generally computationally cost effective, it can become severely inaccurate when higher order gPC expansions are required to represent the physical variability. A more robust algorithm is presented by Debusschere et al. [15] for any non polynomial function u(x) for which du dx can be expressed as a rational function of x, u(x).

4

Optimal Control with Probabilistic System Parameters

In this research work we generate trajectories for dynamical systems in the optimal control theoretic framework. Optimality with probabilistic uncertainty in system parameters results in stochastic optimal control problems. In this section, we derive two standard cost functions that are encountered in stochastic optimal control problems, in terms of the polynomial chaos expansions. Here we consider minimum expectation and minimum variance cost functions. In the following analysis, we assume x(t) is stochastic and u(t) is deterministic. 4

4.1

Minimum Expectation Trajectories

Minimum expectation optimal trajectories are obtained by minimizing the following cost function, analogous to the Bolza form, Z  tf

(xT Qx + uT Ru)dt + xTf Sxf ,

min E u

(13)

0

where x ≡ x(t) ∈ Rn , u ≡ u(t) ∈ Rm and xf = x(tf ), Q = QT > 0, R = RT > 0, S = S T > 0. For scalar x, the quantity E[x2 ] in terms of its gPC expansions is given by 2

E[x ] =

p X p X

Z xi xj

φi φj f d∆ = xT W x,

(14)

D∆

i=0 j=0

where D∆ is the domain of ∆ , xi are the gPC expansions of x, f ≡ f (∆) is the probability distribution of ∆; W = diag[hφ2i (∆)i], and x = (x0 · · · xp )T . Note that the integral in eqn.(14) is essentially the weighted inner product of φi and φj . Orthogonality with respect to weight f results in the diagonal matrix W . The expression E[x2 ] can be generalized for x ∈ Rn where E[xT x] is given by E[xT x] = XT (W ⊗ In )X,

(15)

In ∈ Rn×n is the identity matrix and ⊗ is the Kronecker product, and X is given by eqn.(??). The cost function in eqn.(13) can now be written in terms of the gPC expansions as Z tf  T  min X Qx¯ X + uT Ru dt + XTf Sx¯ Xf , (16) u

0

where Qx¯ = W ⊗ Q, Sx¯ = W ⊗ S and Xf = X(tf ).

4.2

Minimum Covariance Trajectories

For x ∈ R, the variance σ 2 (x) in terms of the gPC expansions is given by σ 2 (x) = E[x − E[x]]2 = E[x2 ] − E2 [x] = xT W x − E2 [x], where E[x] = E

" p X i=0

or

# x i φi =

p X

xi E[φi ] =

i=0

p X i=0

Z xi

φi f d∆, D∆

  1 φ f d∆ D∆ 0  0    .. E[x] = xT F, where F =   =  .. .  . R φ f d∆ D∆ p 0  R

   . 

Therefore, σ 2 for scalar x can be written in a compact form as σ 2 = xT (W − F F T )x, which may enter the cost function in integral form or as final cost. The covariance of a vector process x(t) : R 7→ Rn is given by Cxx (t)

= E[(x(t) − x ¯(t)) (x(t) − x ¯(t))T ] = E[(x(t)x(t)T ] − x ¯(t)¯ x(t)T , 5

(17)

where

 xT1   x ¯(t) = E[x(t)] =  ...  F, xTn 

and xi is defined by eqn.(??). Therefore  xT1  ..  T  . FF xTn  T x1 F F T x1  ..  .  x ¯(t)¯ x(t)T

=

=

T xT1  ..   .  xTn 

xTn F F T x1

··· ···

 xT1 F F T xn  .. . . T T xn F F xn

Similarly, xT1 W x1  .. E[xxT ] =  . xTn W x1 

··· ···

 xT1 W xn  .. . . T xn W xn

Therefore, in terms of gPC coefficients, Cxx can be written as   T x1 (W − F F T )x1 · · · xT1 (W − F F T )xn   .. .. Cxx =  . . . T T T T xn (W − F F )x1 · · · xn (W − F F )xn

(18)

An important metric for covariance analysis is Tr[Cxx ], which can be written in a compact form as, Tr[Cxx ] = XT Qσ2 X,

(19)

where Qσ2 = (W − F F T ) ⊗ In .

5

Example - Van der Pol Oscillator

In this section we apply the polynomial chaos approach to solve an example stochastic optimal control problem based on the Van der Pol oscillator, which highlights numerical solution of stochastic optimal control problems. Consider the well known forced Van der Pol oscillator model  x˙ 1 = x2 (20) x˙ 2 = −x1 + µ(∆)(1 − x21 )x2 + u where µ is a random variable with uniform distribution in the range µ(∆) ∈ [0, 1]. For uniform distribution the basis functions are Legendre polynomials and D∆ = [−1, 1]. Since the dimension of ∆ in this case is one, p is equal to the order of the Legendre polynomial. For this example we chose p = 4. Representing the gPC expansions of x1 , x2 , µ similar to eqn.(4), the dynamics of the Van der Pol oscillator in terms of the gPC expansions can be written as x˙ 1,m

= x2,m ,

x˙ 2,m

  p X p p X p X p X p X X 1  = −x1,m + µi x2,j eijm − µi x1,j x1,k x2,l eijklm hu, φm i , hφm , φm i i=0 j=0 i=0 j=0 k=0 l=0

6

for m = {0, · · · , p}; where eijm = hφi φj φm i and eijklm = hφi φj φk φl φm i. Note that the projection hu, φm i = 2u for m = 0, and hu, φm i = 0 for m = {1, · · · , p}. Therefore the deterministic control only enters the equation for x2,0 . The stochastic optimal control problem for this example is posed as  Z 5 (x21 + x22 + u2 )dt min E u(t)

(21)

0

subject to stochastic dynamics given by eqn.(20) and constraints x1 (0) = 3, x2 (0) = 0, E[x1 (5)] = 0, E[x2 (5)] = 0. In the posed problem, we have assumed that there is no uncertainty in the initial condition of the states and the terminal equality constraint is imposed on the expected value of the states at final time. In terms of the gPC expansions, the constraints are  x1,0 (0) = 3,    x1,m (0) = 0, m = {1, · · · , p},   x2,m (0) = 0, m = {0, · · · , p}, (22)   E[x1 (5)] = x1,0 (5) = 0,    E[x2 (5)] = x2,0 (5) = 0. The optimal control problem was solved using OPTRAGEN [16], a MATLAB toolbox that transcribes optimal control problems to nonlinear programming problems using B-Spline approximations. The resulting nonlinear programming problem was solved using SNOPT [17]. Figure (1(a)) shows the trajectories of the optimal solution in the subspace spanned by B-Splines. The solid (red) state trajectories are the mean trajectories of x1 (t), x2 (t) and they satisfy the constraints defined by eqn.(22). The dashed (blue) state trajectories are the remaining gPC expansions of x1 (t), x2 (t). The suboptimal cost for these trajectories is 15.28 and took 10.203 seconds to solve for in MATLAB environment. The trajectories were approximated using B-Splines consisting of ten 5th order polynomial pieces with 4th order smoothness at the knot points. To verify the optimal control law for the stochastic system, we applied u∗ (t) to the Van der Pol oscillator for various values of µ, uniformly distributed over the interval [0, 1], and computed the expected value of the state trajectories from the Monte-Carlo simulations. Figure (1(b)) shows the comparison of the expected trajectories E[x∗1 (t)], E[x∗2 (t)], obtained from gPC and Monte-Carlo methods. The trajectories are identical. Thus, the generalized polynomial chaos framework provides a powerful set of tools for solving stochastic optimal control problems. Figure (2) shows the evolution of the probability density function and the mean trajecctory due to uncertainty in µ. From this figure, it can be inferred that the trajectories satisfy the terminal constraint in an average sense. However, none of the individual trajectories arrive at the origin. This is not surprising, as the constraints were only imposed on the expected value of the trajectories. For nonlinear systems the uncertainty evolves in a non-Gaussian manner. Therefore, analysis based on expectation can lead to erroneous interpretations and it is important to include higher order moments in the analysis. For this problem, localization of the state about the origin at final time can be achieved by including a terminal cost or constraint related to the covariance at final time. Similar approach is presented in the work by Darlington et al. [18]. Figure (3) shows the probability density function at final time when a terminal constraint Tr[Cxx ] < 0.2 was imposed in the optimal control problem. Figure (3) also shows the probability density function, at 7

final time, obtained without the terminal constraint. It is clear from the figure that inclusion of terminal covariance based constraint has localized the covariance about the origin. Although none of the trajectories for µ ∈ [0, 1], even for the constrained case, arrive at the origin. This terminal constraint however incurred a higher cost of 128.79. The state and control trajectories are shown in fig. (4(a)). We observe that introduction of terminal constraint Tr[Cxx ] < 0.2 results in higher control magnitudes. The optimal control obtained in this case also agrees with the Monte-Carlo simulations over µ ∈ [0, 1], and is shown in fig. (4(b)). The gPC framework is well suited for evaluating short term statistics of dynamical systems. However, their performance degrades upon long term integration. As shown by fig.(5), the mean trajectories of the stochastic Van der Pol oscillator from eqn.(20), obtained from gPC calculations with 5th order expansions, deviate from those obtained from Monte-Carlo simulations. This deviation arises due to finite dimensional approximation of the probability space (Ω, F, P ). Several methods have been proposed to reduce this divergence, including adaptive [19] and multi-element approximation techniques [20]. Another limitation arises from the factorial increase in the state-space dimension for increase in number of random parameters. Although, gPC theory provides a more computational efficient framework to analyze effect of parametric uncertainty, it is limited to low dimensional parametric uncertainty.

6

Summary

In this paper a framework for numerically solving stochastic optimal control problems, with application to trajectory generation for constrained mechanical systems, was developed in the framework of polynomial chaos expansions. Expansion of stochastic dynamics in polynomial chaos framework leads to equivalent deterministic dynamics in higher dimensional state space. Cost functions such as minimum expectation and variance translate to standard quadratic forms. Thus a stochastic optimal control problem is transformed to a deterministic optimal control problem and standard practices in solving deterministic optimal control problems are applicable. In this paper we have used the so called direct method to transcribe the optimal control problem to a nonlinear programming problem. The results show very good agreement with MonteCarlo simulations and lead us to believe that polynomial chaos based approach is a strong candidate for solving stochastic optimal control problems.

7

Acknowledgements

This research work was conducted in part under NASA funding from NRA NNH06ZEA001N-HYP: Fundamental Aeronautics: Hypersonics Project, Topic 5.4 Advanced Control Methods, Subtopic A.5.4.1 Advanced Adaptive Control with Dan Moerder from NASA Langley as the technical monitor.

References [1] S. M. LaValle. Planning Algorithms. Cambridge University Press, 2006. [2] D. P Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, 1997. [3] D. P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 2000. [4] N. Wiener. The Homogeneous Chaos. American Journal of Mathematics, 60(4):897–936, 1938. [5] R. H. Cameron and W. T. Martin. The Orthogonal Development of Non-Linear Functionals in Series of Fourier-Hermite Functionals. The Annals of Mathematics, 48(2):385–392, 1947.

8

[6] Dongbin Xiu and George Em Karniadakis. The Wiener–Askey Polynomial Chaos for Stochastic Differential Equations. SIAM J. Sci. Comput., 24(2):619–644, 2002. [7] R. Askey and J. Wilson. Some Basic Hypergeometric Polynomials that Generalize Jacobi Polynomials. Memoirs Amer. Math. Soc., 319, 1985. [8] Thomas Y. Hou, Wuan Luo, Boris Rozovskii, and Hao-Min Zhou. Wiener Chaos Expansions and Numerical Solutions of Randomly Forced Equations of Fluid Mechanics. J. Comput. Phys., 216(2):687– 706, 2006. [9] Dongbin Xiu and George Em Karniadakis. Modeling Uncertainty in Flow Simulations via Generalized Polynomial Chaos. J. Comput. Phys., 187(1):137–167, 2003. [10] Roger G. Ghanem and Pol D. Spanos. Stochastic Finite Elements: A Spectral Approach. Springer-Verlag New York, Inc., New York, NY, USA, 1991. [11] Roger Ghanem and John Red-Horse. Propagation of Probabilistic Uncertainty in Complex Physical Systems Using a Stochastic Finite Element Approach. Phys. D, 133(1-4):137–144, 1999. [12] R.G. Ghanem. Ingredients for a General Purpose Stochastic Finite Elements Implementation. Comput. Methods Appl. Mech. Eng., 168(1-4):19–34, 1999. [13] Franz S. Hover and Michael S. Triantafyllou. Application of Polynomial Chaos in Stability and Control. Automatica, 42(5):789–795, 2006. [14] FS Hover. Gradient dynamic optimization with Legendre chaos. Automatica, 44(1):135–140, 2008. [15] Bert J. Debusschere, Habib N. Najm, Philippe P. P´ebay, Omar M. Knio, Roger G. Ghanem, and Olivier P. Le Maˆıtre. Numerical Challenges in the Use of Polynomial Chaos Representations for Stochastic Processes. SIAM J. Sci. Comput., 26(2):698–719, 2005. [16] R. Bhattacharya. OPTRAGEN: A MATLAB Toolbox for Optimal Trajectory Generation. 45th IEEE Conference on Decision and Control, pages 6832–6836, 2006. [17] P. E. Gill, W. Murray, and M. A. Saunders. SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization. SIAM Journal on Optimization, (12):979–1006, 2002. [18] J. Darlington, CC Pantelides, B. Rustem, and BA Tanyi. Decreasing the sensitivity of open-loop optimal solutions in decision making under uncertainty. European Journal of Operational Research, 121(2):343–362, 2000. [19] R. Li and R. Ghanem. Adaptive Polynomial Chaos Expansions Applied to Statistics of Extremes in Nonlinear Random Vibration. Probabilistic Engineering Mechanics, 13:125–136, April 1998. [20] Xiaoliang Wan and George Em Karniadakis. An Adaptive Multi-Element Generalized Polynomial Chaos Method for Stochastic Differential Equations. J. Comput. Phys., 209(2):617–642, 2005.

9

(a) Optimal gPC trajectories for the Van der Pol oscillator. Solid (red) is the mean trajectory. Dashed (blue) are the other gPC coefficients.

(b) Verification of stochastic optimal control law using Monte-Carlo simulations.

Figure 1: Comparison of trajectories obtained from gPC expansions and Monte-Carlo simulations.

10

Figure 2: Evolution of the probability density function of the state trajectories, and the mean trajectory due to µ(∆). The solid (red) line denotes the expected trajectory of (x1 , x2 ). The circles (blue) denote time instances, on the mean trajectory, for which the snapshots of pdf are shown.

11

Figure 3: PDF at final time, due to the terminal constraint based on covariance.

12

(a) Optimal gPC trajectories for the Van der Pol oscillator with terminal covariance constraint. Solid (red) is the mean trajectory. Dashed (blue) are the other gPC coefficients.

(b) Verification of stochastic optimal control law using Monte-Carlo simulations.

Figure 4: Comparison of trajectories obtained from gPC expansions and Monte-Carlo simulations.

13

Figure 5: Comparison of mean trajectories obtained from Monte-Carlo simulations and gPC theory for Van der Pol System

14