2014 American Control Conference (ACC) June 4-6, 2014. Portland, Oregon, USA
A Tensor Decomposition Approach to High Dimensional Stationary Fokker-Planck Equations Yifei Sun and Mrinal Kumar Abstract— This paper addresses the curse of dimensionality in the numerical solution of stationary Fokker-Planck equations. Combined with Chebyshev spectral differentiation, the tensor approach significantly reduces the degrees of freedom of the approximation essentially in exchange for nonlinearity, such that the resulting discretized nonlinear system is solved by alternating least squares. Enforcement of the normality condition via a penalty method avoids the need for exploration of the the null space of the discretized Fokker-Planck operator. The proposed method enables a drastic reduction of degrees of freedom required to maintain accuracy as dimensionality increases. Numerical results are presented to illustrate the effectiveness of the proposed method.
I. I NTRODUCTION The exact description of uncertainty propagation through nonlinear dynamical systems, subject to stochastic excitation and uncertain initial conditions, is given by the FokkerPlanck equation (FPE) [1]. However, the well-known curse of dimensionality [2] fundamentally limits the use of FPE for uncertainty quantification in high dimensional systems [3], [4]. Recently, the meshless partition of unity finite element method (PUFEM) was employed to counter this problem with moderate success [4], [5], [6]. While the unstructured node-based approximation paradigm of PUFEM simplifies discretization of high-dimensional spaces, the number of basis functions needed still increases rapidly with dimensionality. This can be attributed to the factorial growth of a complete set of polynomial basis functions. In related literature, the representation of data in a tensor product structure has been recognized as a key for managing the dimensionality issue [7]. Methods based on tensor decompositions have become well-established in a wide range of applications [8], [9], [10]. The essence of their application to numerical multidimensional partial differential equations (PDEs) [11], [12], [13], [14], [15] lies in the separation of dimensions, following which expensive high dimensional operations are decomposed into a series of simple onedimensional operations. As an outcome, the complexity scales linearly rather than exponentially with dimensionality. The price to pay is that a resulting highly nonlinear problem needs to be solved. This work was supported by the National Science Foundation under Grant No. ECCS-1254244 and the NASA Marshall Space Flight Center Grant No. NNM13AA07G Y. Sun is a graduate student in the Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL 32608, USA
[email protected] M. Kumar is with the Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL 32608, USA
[email protected] 978-1-4799-3274-0/$31.00 ©2014 AACC
Beylkin et al. [11] solved the Schrodinger equation (as well as certain other linear PDEs) with solution approximated in the CANDECOMP/PARAFAC decomposition (CPD) form by the alternating least squares algorithm. In Refs.[14], [16], the so-called proper generalized decomposition (PGD) method was developed, in which the approximation was constructed in CPD form through a sequence of enrichment steps using the finite element method. It is important to note that at each enrichment step, only a single component rank-one tensor was obtained. In Ref.[14] and subsequent papers by the same authors, PGD was used to solve high dimensional FPEs for multi-bead-spring models of polymer chains encountered in the kinetic theory of complex fluids. Compared with methods such as finite element/finite difference for this problem, an improvement of several orders of magnitude was shown in terms of the degrees of freedom (DOF) of the approximation. However, the application of PGD to problems outside the domain of kinetic theory of complex fluids is limited and in the experience of the current authors not very successful. The current paper is concerned with FPE commonly encountered in the field of nonlinear vibrations. In particular, the stationary FPE is solved by combining the CPD structure with the Chebyshev spectral differentiation framework. This approach benefits from the superior performance of the Chebyshev spectral approach over other differentiation methods for smooth functions on domains of regular geometry [17], [18]. The normality constraint is enforced via a penalty method, following which all component rank-one tensors are obtained simultaneously at each enrichment step using the alternating least squares method of [11]. It is shown through numerical examples that the total DOF scales favorably, which is a significant result for the FPE. The remainder of this paper is organized as follows: section II introduces the tensor decomposition approaches. The Chebyshev spectral method are discussed in section III. Section IV describes the proposed method and numerical examples are provided in section V. Finally, a summary and future research directions are provided in section VI. II. T ENSOR D ECOMPOSITION The idea of separating the dimensions, such as tensor decomposition approaches, is now being recognized as a key for breaking the curse of dimensionality [7] for solving PDEs. Viewed as a generalization of a matrix in two dimensions, a tensor of order P and size n1 × n2 × · · · × nP is a P dimensional array denoted by F ∈ Rn1 ×n2 ×···×nP . In numerous applications, multivariate functions are discretized
4500
using tensor grids, the storage alone QP of which is difficult owing to the exponential growth ( d=1 nd ) of its elements with respect to P . One remedy is to decompose the tensor into a summation of several rank-one tensors. A tensor F ∈ Rn1 ×n2 ×···×nP is of rank one if F = f1 ⊗ f2 ⊗ · · · ⊗ fP ,
necessarily requires a tensor product representation of the discretized FP operator. In general, the operator L(·) of a linear PDE can be approximated in tensor product form through discretization in the following manner: L≈
(1)
RA O P X
Ald ,
(4)
l=1 d=1
where fd are nd × 1 vectors for d = 1, 2, . . . , P and “⊗” is the standard tensor product. The following important tensor decomposition of F was introduced by [19] and [20], named CANDECOMP/PARAFAC decomposition (CPD): F≈
R O P X
fdr .
(2)
r=1 d=1
Note that the above represents an approximation. The minimum R for which the “≈” can be replaced with “=” is called the rank of tensor F. We define factor matrices of the above decomposition as Fd = fd1 fd2 · · · fdR , d = 1, 2, . . . , P . Once the CPD is found, a tensor of order P can be reconstructed using its P factor matrices. For a prescribed accuracy, the PPtotal number of elements in the above approximation (R d=1 nd ) grows linearly with P , provided that R does not exhibit explosive growth. Fortunately it is observed that in many applications, a tensor of high order can be very well approximated by CPD of relatively low rank, although no theoretical results are available [21]. This makes the CPD structure a good candidate for numerically dealing with high dimensional PDEs. Alternating least squares (ALS) is a commonly used method for computing the factor matrices of a CP decomposition. For a tensor G ∈ Rn1 ×n2 ×···×nP , the problem of finding its CPD with prescribed R can be formulated as the following optimization problem:
2 R O P
X
r min R, R = G − f (3)
, d
{fdr } r=1 d=1
F
where Frobenius norm is used, and the unknowns are the vectors fdr , d = 1, 2, . . . , P and r = 1, 2, . . . , R or equivalently the factor matrices Fd . Although Eq.3 appears to be a standard nonlinear least squares problem whose P objective P function is a 2P th degree polynomial with R d=1 nd variables, there exist no well established algorithms for its solution [21]. Among existing techniques, alternating least squares (ALS) is arguably the most popular, underlying which is the basic idea of holding all factor matrices except one constant, thus translating the original nonlinear least squares problem into a series of sequential linear least squares problems. For a comprehensive introduction to tensor decompositions, the reader is referred to the review paper [8]. III. C HEBYSHEV S PECTRAL D IFFERENTIATION There are two main ingredients of the proposed approach: (i) discretization of the Fokker-Planck operator via spectral differentiation and, (ii) CPD representation of the unknown probability density function (pdf), obtained using the ALS algorithm. The CPD representation of the unknown pdf
where Ald are square matrices, which in order to be compatible with Eq.2 must be of size nd × nd . By virtue of the above separated form, high dimensional operations involving L(·) can now be decoupled into simple onedimensional operations on Ald . Numerical discretization techniques (most commonly the finite difference method) are used to determine the above form. It is most desirable if the linear operator can be approximated accurately by a small sized tensor, which is crucial for reducing the computational burden as dimensionality increases. In the current application of solving the stationary FPE admitting smooth solution on a hpercuboid, a much better choice for spatial differentiation is the Chebyshev spectral method [18]. It is known that when a function is sufficiently smooth, its derivatives at given point can be approximated arbitrarily well by a linear combination of its function values at the nodes used to discritize the domain, where the distribution of the nodes are optimally determined by extrema of the Chebyshev polynomials. Most importantly, this approximation error decreases geometrically as more nodes are used, hence the name Chebyshev spectral method. In other words, a corresponding Chebyshev spectral differentiation matrix of significantly smaller size will be sufficient to approximate the same differential operator with equal accuracy as compared with the finite difference method. This is crucial for amelioration of the curse of dimensionality in the FPE as will be illustrated via examples in section V. For more details about Chebyshev spectral method, see [17], [18], [22]. We note that as opposed to the attractive band-limited differentiation matrix of the finite difference method, the Chebyshev spectral differentiation matrix is dense. However, since only one-dimensional differentiation is required for the tensor decomposition based method used here, the dense structure fails to have an adverse impact. IV. A C HEBYSHEV S PECTRAL T ENSOR D ECOMPOSITION S OLUTION OF THE S TATIONARY FPE In this section, we outline the use of Chebyshev spectral differentiation to discretize the FP operator. The CPD form is used for tensorization and the ALS method [11] is used to solve the resulting discrete system while incorporating the normality constraint for a valid probability density function. It is shown through examples that the DOF of the approximation grows benignly with dimensionality, which is in sharp contrast with the exponential growth observed in traditional finite difference/element methods. A. Fokker-Planck Equation Consider a nonlinear dynamical system perturbed by white noise excitation with initial condition uncertainty, modeled
4501
identically zero. The vector U is sought in a CPD form:
by the following Itˆo stochastic differential equation: dx = f (t, x)dt + g(t, x)dB(t),
x ∈ RP ,
(5)
where B(t) denotes the Brownian motion process with correlation function Qδ(t1 − t2 ), and f (t, x) : [0, ∞) × RP → RP and g(t, x) : [0, ∞) × RP → RP ×M are deterministic functions. The uncertainty in the state x(t) is quantified by its time varying probability density function (pdf) W(t, x). Initial condition uncertainty is prescribed as W(t0 , x) = W0 (x). The FPE [1] corresponding to Eq.5 is a PDE that captures the time evolution of the state pdf: ∂ W(t, x) = LF P [W(t, x)], ∂t is the Fokker-Planck operator given by
where LF P
(6)
LF P = −
P P X P X X ∂2 ∂ (1) (2) Di (·) + Dij (·) , (7) ∂x ∂x ∂x i i j i=1 j=1 i=1
D(1) (t, x) = f (t, x),
D(2) (t, x) =
1 g(t, x)QgT (t, x). 2
(8)
B. Representation in Tensor Product Form To begin tensorization of the FP operator, we assume that f (x) and g(x) are separable functions so that the drift vector as well as the diffusion matrix can be expressed exactly in the following separable form: (1)
R1i Y P X j
(2)
Dij (t, x) =
fdj (xd ),
Due to the special structure of the above decomposition, Eq.11 is nonlinear in terms of the component functions uld (xd ) despite the fact that it is clearly linear in terms of the unknown tensor U. Solution schemes for Eq.11 first appeared in [11], where one method is to cast it as an equation error minimization problem and use ALS to solve it. This paper uses a modified version of this approach to suit the current application. The developments below describe the solver for the general Eq.11, which inPaddition to Eq.12, has A = Nof PRAcase RG N P P iA iG iA and G = A d=1 gd ; where Ad iG =1 d=1 d iA =1 iG iU are nd × nd matrices while ud and gd are nd × 1 vectors. Transforming Eq.11 into an optimization problem, we get (13)
and define R =k AU − G k2F . The necessary condition for minimization of R is ∂hAU, AUi ∂hAU, Gi −2 = 0, r ∂uk ∂urk
=M
Once the FP-operator are tensorized, the stationary FPE is reduced to the following linear algebraic form: (11)
where U is the approximation of the unknown stationary pdf W(x). Due to the homogeneous nature of the FPE, G is
=N
where Mi,j and Ni are submatrices of the block matrices M and N respectively, given by (see [22] for more details) RA X RA X
Mi,j =
d=1
C. CPD Approximation of the Discretized FPE
(14)
for k = 1, 2, . . . , P and r = 1, 2, . . . , RU . Collecting terms in Eq.14 for all r’s and fixed dimension k, we have 1 uk N1 M1,1 · · · M1,RU .. .. .. .. .. . = . , (15) . . . U NRU MRU ,1 · · · MRU ,RU uR k | {z } {z } |
(9)
where R1i and R2ij are finite. If the functions f (x) and g(x) are not separable, they can be approximated by CP decomposition discussed in section II. In such case, the second “=” of Eq.9 and Eq.10 must be replaced with “≈”. For more details of writing FP-operator in tensor product form, see [22].
(12)
l=1 d=1
(AjkA )T AikA
iA =1 jA =1
R2ij P XY 1 pkd (xd ). = g(t, x)QgT (t, x) 2 ij k d=1 (10)
AU = G,
uld (xd ).
min k AU − G k2F ,
The above equation, called the stationary FPE, is the concern of the current paper.
Di (t, x) = fi (x) =
RU Y P X
{urk }
In Eq.7, D(1) and D(2) are called drift coefficient vector and diffusion coefficient matrix respectively. Under certain conditions (see [1]), there exists a unique global asymptotically stable steady state pdf Ws (x) that solves the following simplified form of Eq.6: LF P [W(x)] = 0.
W(x) ≈ U(x) =
Ni =
RA X RG X
Y
hAidA ujd , AjdA uid i,
(16)
d6=k
(AikA )T gkiG
iA =1 iG =1
Y
hAidA uid , gdiG i.
(17)
d6=k
Eqs.16 and 17 first appeared (in a different form than above) in Ref. [11]. In the ALS framework, Eq.15 is reduced to a linear system and solved sequentially for each dimension in an iterative manner. Therefore, the number of unknowns in a single iteration is nk RU , which is independent of the dimensionality P once the user prescribes RU . Moreover, a significant amount of computation is saved by noting that only a small portion of the terms involved need to be recalculated for different k and RU . In implementation, we begin with RU = 1 and random initial values for uld , and then increase RU gradually until stopping criteria are met. In order for the ALS scheme to return a nontrivial answer (note that for the stationary FPE, G ≡ 0), we must incorporate the normality constraint as described below.
4502
S1 (see below) in met: – Initialize urd for d = 1, 2, . . . , P and r = 1, 2, . . . , RU . – Repeat for d = 1, 2, . . . , P until stopping criterion S2 (see below) is met: ∗ Solve Eq.21 to obtain urd for r = 1, 2, . . . , RU while holding urd ’s for other dimensions constant. r • Return ud for d = 1, 2, . . . , P and r = 1, 2, . . . , RU . Stopping Criteria: The above stopping criteria are: q QPR < 1 or RU = Rmax , where Rmax is • S1 : d=1 nd the maximum allowable number of component rank-one tensors. . • S2 : δ = |Rcurr − Rprev | < 2 or maximum number of iterations reached.
D. Constraints A probability density function that solves the stationary FPE must satisfy the following constraints: Z lim W(x) = 0, W(x)dx = 1, (18) x→∞
Ω
which are vanishing boundary condition and normality condition respectively. Any spatial-discretization based method requires a compact domain for implementation, due to which the vanishing boundary condition is imposed on a conservatively chosen “large-enough” domain. Here, the boundary condition is enforced by simply removing the boundary rows and columns of AidA ’s and uidU ’s in Eq.15. Normality Constraint: Adding the normality constraint to Eq.8 precludes the trivial solution. By virtue of the CP structure of U, the multi-dimensional integral of Eq.18 can be decoupled into a product of one-dimensional integrals and therefore scales linearly with system dimensionality. Corresponding to the Chebyshev extrema points used for spatial discretization, the Clenshaw-Curtis quadrature are used for integration [17], resulting in the following tensorized structure of the normality condition: B=
P O
bd ,
hB, Ui = 1,
(19)
d=1
where bd is a nd × 1 vector, containing the Clenshaw-Curtis weights for dimension d. To impose the normality condition, a penalty term is added to the minimization problem of Eq.13. With G = 0, we have α 2 (20) min k AU k2F + (hB, Ui − 1) , 2 {urk }
V. N UMERICAL E XAMPLES In this section, we present numerical examples involving relatively high dimensional systems. To facilitate illustration, RU is referred to as the “number of enrichment steps” while uidU for iU = 1, 2, . . . , RU are called basis functions for dimension d. All shown basis functions are scaled to have unit norm via the use of a scaling vector. Due to the random initialization of basis functions, the comparisons and conclusions are made in an average sense. It is shown that the DOF in the proposed method scales favorably with the system dimensionality and presents a strong case for curbing the curse of dimensionality associated with FPE.
where α is a positive penalty parameter. Similarly, the equations below can be obtained following the first order conditions for minimization (see [22] for more details): 1 uk C1 B1,1 · · · B1,RU . .. . .. .. . = α .. M + α .. . . BRU ,1
···
BRU ,RU
U uR k
where M is the same as in Eq.15 and Y Y Bi,j = bk (bk )T (bd )T uj (bd )T uid , d
d6=k
Ci = bk
CRU (21)
(22)
(bd )T uid .
•
Use Chebyshev extrema points for spatial discretization of each dimension and approximate the FP-operator by Chebyshev differentiation matrices. Rewrite the FP operator in tensor product structure and incorporate the normality constraint to obtain Eq.21. Cycle for RU = 1, 2, . . . , Rmax until stopping criterion
Numerical results for the 2-state and 4-state systems
A. Example 1: 2-State System Consider the following two-state nonlinear oscillator: x ¨ + bx˙ + x + a(x2 + x˙ 2 )x˙ = gξ(t),
(23)
The ALS scheme discussed in section IV-C can now be used directly to solve Eq.21. A summary of the entire approach is provided below:
•
Fig. 1.
d6=k
Y d6=k
•
(a) Approximated stationary solu- (b) Approximated x1 −x2 marginal tion for the 2-state system pdf for the 4-state system
(24)
where ξ(t) is white noise with intensity Q and a = 0.125, b = −0.5, g = 1, Q = 0.4 are constants. FPE of the above system admits the following stationary solution: i 1 ha 2 (x + y 2 )2 + b(x2 + y 2 ) , Ws (x, y) = C exp − 2 g Q 2 (25) . where y = x˙ and C is the normalization constant. Note that the true solution is not a separable function. The current method was implemented using 29 Chebyshev extrema points per dimension on the finite domain [−4, 4] ⊗ [−4, 4].
4503
The approximation after five enrichment steps (RU = 5) is shown in Fig.1(a), where the total DOF of this approximation is 290. The error between the approximated and in this case, known true solution in mean square sense (MSE) evaluated on the grid is 4.403 × 10−9 . The basis functions {uidU }5iU =1 for d = 1 (dimension x) and d = 2 (dimension y) are shown in Figs.2(a) and 2(b) respectively with scaling vector [0.6883, 0.4570, 0.1192, 0.0263, 0.0046]. In this and all subsequent examples, the basis functions are ordered from most to least significant in terms of their contribution to the final approximation, as reflected by the relative magnitudes of corresponding components of the scaling vector. In comparison, if second order finite difference spatial discretization in tensor-product form is used, 83 nodes per dimension provide an MSE of 1.242 × 10−8 after 8 enrichment steps, where the total DOF is now 1328, with a worse MSE. The advantage of Chebyshev spectral differentiation should be further evident for higher dimensional cases.
(a) Basis functions: x1
(b) Basis functions: x2
(c) Basis functions: x3
(d) Basis functions: x4
Fig. 4.
Basis functions for the 4-sate system
2D2 (∼ ξ2 ) = 8. Parameters have been selected to ensure that D1aM = Db2 I = 2T , because under this condition the analytical solution can be written as
(a) Basis functions: x dimension Fig. 2.
(b) Basis functions: y dimension
Basis functions for the 2-state system
(a) 2-State System Fig. 3.
Ws (x1 ,x2 , x3 , x4 ) = a 2 b 2 1 x − x , C exp − V (x1 , x2 ) − 2T 2D1 3 2D2 4
(b) 4-State System
Evolution of equation error: Examples 1 and 2
B. Example 2: 4-State System Next consider the 4-state nonlinear isolating suspension model given in [23], x˙1 = x3 ,
x˙2 = x4 , 1 ∂V + ξ1 , x˙3 = −ax3 − M ∂x1 1 ∂V + ξ2 . x˙4 = −bx4 − I ∂x2 The coupled potential function V is chosen as V (x1 , x2 ) = k1 x21 +k2 x22 +(λ1 x41 +λ2 x42 +µx21 x22 ). System parameters are a = 0.5, b = 1, k1 = 0.5, k2 = −0.5, = 0.5, λ1 = 0.25, λ2 = 0.125, µ = 0.375. The noise process is two-dimensional with intensities 2D1 (∼ ξ1 ) = 4 and
which is once again not separable. The described method was implemented using 33 nodes per dimension on the finite domain ⊗4 [−6, 6]. The x1 − x2 marginal pdf of the approximated solution is presented in Fig.1(b). After three enrichment steps, the MSE between the approximated and true marginal is 5.118 × 10−8 . The time of execution of the entire algorithm (on a PC with 2.5 GHz CPU and 4 GB RAM) was less than 20 seconds. The basis functions for each dimension, i.e. {uidU }3iU =1 are shown in Fig.4 labeled in decreasing order of significance with scaling vector [0.0930, 0.0154, 3.077 × 10−4 ], and the total DOF is only 396. Once again, the second order finite difference method in its tensor-product form provides an MSE of 7.297 × 10−8 following six enrichment steps, requiring a total of 1608 DOFs (67 nodes per dimension). Fig.3 illustrates the convergence characteristics of ALS for examples 1 and 2. Clearly, the two-state system exhibits relatively “gradual” convergence whereas the four-state system goes through extended periods of sluggish performance in between abrupt improvements in the approximation. Similar slow convergence of ALS has been noted widely in the literature (e.g. see [21], [24]). The four-state system has linear dependency between the bases {ui3U } and {ui4U } (Figs.4(c) and 4(d)), leading to a bottleneck scenario and the concomitant sluggish convergence (Fig.3(b)). In addition, we note that there is another factor in play, namely the dimensionality of the underlying system which can potentially cause sluggish convergence.
4504
C. Extension to Higher Dimensional Systems In addition, the tensor approach is applied to nonlinear vibration isolation systems with state space in up to ten dimensions. The growth of DOF versus dimensionality is shown in Fig.5, in comparison with some recently developed methods for the same problem. Note that FEM ([25], [26]) becomes quickly infeasible as dimensionality increases. Partition of unity based meshless methods (such as standard/particle-PUFEM) were able to achieve much more favorable scaling and polynomial-like DOF growth ([4], [3]). In these methods, although the number of nodes needed does not grow exponentially, the number of basis functions needed per node suffers from factorial growth, resulting in excessive computational burden. In comparison, the present approach scales very well and is able to solve high dimensional systems in a reasonable amount of time. Data for FEM, sPUFEM and pPUFEM are not available for dimensions greater than 4, 4 and 5 respectively because of infeasiblity on the computational platforms. It must be noted that Fig.5 must be interpreted with caution because of variability in the type of problems solved in various dimensions and different methods. This figure is only meant to provide a gross orderof-magnitude comparison.
Fig. 5.
A comparison of FEM, s/p-PUFEM and the current approach
VI. C ONCLUSIONS AND F URTHER R ESEARCH In this paper, the stationary FPE was solved by incorporating Chebyshev spectral differentiation into the CPD framework with ALS algorithm. The FP operator was expressed in a tensor product structure and the normality constraint was enforced via penalty method. It was shown through numerical results that the DOF of the proposed approach scales favorably, making it an attractive approach for nontrivial uncertainty quantification problems. Also the computational time involved is reasonable, potentially allowing the present approach for use in nonlinear state estimation. Extending the current method for transient FPE is under investigation. R EFERENCES [1] A. Fuller, “Analysis of nonlinear stochastic systems by means of the fokker–planck equation,” International Journal of Control, vol. 9, no. 6, pp. 603–655, 1969. [2] R. E. Bellman, Dynamic Programming. Princeton University Press, Princeton, New Jersey, 1957. [3] M. Kumar, S. Chakravorty, and J. L. Junkins, “A semianalytic meshless approach to the transient fokker–planck equation,” Probabilistic Engineering Mechanics, vol. 25, no. 3, pp. 323–331, 2010.
[4] M. Kumar, S. Chakravorty, P. Singla, and J. L. Junkins, “The partition of unity finite element approach with hp-refinement for the stationary fokker–planck equation,” Journal of Sound and Vibration, vol. 327, no. 1, pp. 144–162, 2009. [5] M. Kumar, S. Chakravorty, and J. L. Junkins, “On the curse of dimensionality in the fokker-planck equation,” in AAS Astrodynamics Specialist Conference, Pittsburgh, PA, USA, Aug 9-13, 2009. [6] Y. Sun and M. Kumar, “A meshless p-pufem fokker-planck equation solver with automatic boundary condition enforcement,” in American Control Conference (ACC), 2012. IEEE, 2012, pp. 74–79. [7] B. N. Khoromskij, “Tensors-structured numerical methods in scientific computing: Survey on recent advances,” Chemometrics and Intelligent Laboratory Systems, vol. 110, no. 1, pp. 1–19, 2012. [8] T. Kolda and B. Bader, “Tensor decompositions and applications,” SIAM Review, vol. 51, no. 3, pp. 455–500, 2009. [9] L. Grasedyck, D. Kressner, and C. Tobler, “A literature survey of low-rank tensor approximation techniques,” arXiv preprint arXiv:1302.7121, 2013. [10] W. Hackbusch, B. N. Khoromskij, and E. E. Tyrtyshnikov, “Hierarchical kronecker tensor-product approximations,” Journal of Numerical Mathematics, vol. 13, no. 2, pp. 119–156, 2005. [11] G. Beylkin and M. Mohlenkamp, “Algorithms for numerical analysis in high dimensions,” SIAM Journal on Scientific Computing, vol. 26, no. 6, pp. 2133–2159, 2005. [12] J. Ballani and L. Grasedyck, “A projection method to solve linear systems in tensor format,” Numerical Linear Algebra with Applications, vol. 20, no. 1, pp. 27–43, 2013. [13] D. Kressner and C. Tobler, “Krylov subspace methods for linear systems with tensor product structure,” SIAM journal on matrix analysis and applications, vol. 31, no. 4, pp. 1688–1714, 2010. [14] A. Ammar, B. Mokdad, F. Chinesta, and R. Keunings, “A new family of solvers for some classes of multidimensional partial differential equations encountered in kinetic theory modeling of complex fluids,” Journal of Non-Newtonian Fluid Mechanics, vol. 139, no. 3, pp. 153– 176, 2006. [15] V. Kazeev, M. Khammash, M. Nip, and C. Schwab, “Direct solution of the chemical master equation using quantized tensor trains,” 2013. [16] A. Ammar, F. Chinesta, E. Cueto, and M. Doblar´e, “Proper generalized decomposition of time-multiscale models,” International Journal for Numerical Methods in Engineering, vol. 90, no. 5, pp. 569–596, 2012. [17] L. Trefethen, Spectral Methods in MATLAB, ser. EngineeringPro collection. SIAM, 2000. [18] J. P. Boyd, Chebyshev and Fourier spectral methods. Courier Dover Publications, 2001. [19] J. D. Carroll and J.-J. Chang, “Analysis of individual differences in multidimensional scaling via an n-way generalization of eckart-young decomposition,” Psychometrika, vol. 35, no. 3, pp. 283–319, 1970. [20] R. A. Harshman, “Foundations of the parafac procedure: models and conditions for an” explanatory” multimodal factor analysis,” UCLA Working Papers in Phonetics, 1970. [21] P. Comon, X. Luciani, and A. L. F. de Almeida, “Tensor decompositions, alternating least squares and other tales,” Journal of Chemometrics, vol. 23, no. 7-8, pp. 393–405, 2009. [22] Y. Sun and M. Kumar, “Numerical solution of high dimensional stationary fokker-planck equations via tensor decomposition and chebyshev spectral differentiation,” Computers and Mathematics with Applications (submitted). [23] S. T. Ariaratnam, “Random vibrations of non-linear suspensions,” Journal of Mechanical Engineering Science, vol. 2, no. 3, pp. 195–201, 1960. [24] M. Rajih, P. Comon, and R. A. Harshman, “Enhanced line search: A novel method to accelerate parafac,” SIAM Journal on Matrix Analysis and Applications, vol. 30, no. 3, pp. 1128–1147, 2008. [25] S. F. Wojtkiewicz, L. Bergman, and B. F. SpencerJr., “Numerical solution of some three state random vibration problems,” in Vibration of Nonlinear, Random and Time-Varying Systems, L. A. Bergman and B. F. SpencerJr., Eds., Boston, MA, USA, 1995. [26] S. F. Wojtkiewicz and L. A. Bergman, “Numerical solution of high dimensional fokker planck equations,” in 8th ASCE Specialty Conference on Probablistic Mechanics and Structural Reliability, Notre Dame, IN, USA, 2000.
4505
