Proceedings of the 2006 American Control Conference Minneapolis, Minnesota, USA, June 14-16, 2006
WeB13.3
Optimal Reduced-Order Modeling for Nonlinear Distributed Parameter Systems Jeff Borggaard Abstract— A method to develop reduced-order models for nonlinear distributed parameter systems is studied. The method is based on Galerkin projection, but the reduced-basis vectors are optimal for the dynamic model, found by minimizing the error between given full-order simulation data and the reducedorder model. This is achieved by formulating the basis selection problem as an optimal control problem with the reducedorder model as a constraint. This methodology allows a natural extension of reduced-order modeling ideas to nonlinear systems. A numerical experiment comparing the optimal reduced-order model to the popular proper orthogonal decomposition method is provided.
I. INTRODUCTION
(or x˙ = F (x, u))
(1)
with output measurements y = C x,
(or y = C (x))
(2)
and data, x(0) and yd (t) (with known control input), to construct a low dimensional system that approximates the data (and nearby trajectories) well. We assume that the state variable x(t) ∈ H1 , the output variable y(t) ∈ H2 , with H1 and H2 Hilbert spaces and the appropriate definition of the operators A , B , C and F . The common approaches to reduced-order modeling are based on projection (cf. [2] for an overview). Thus, one seeks a low dimensional basis {φ1 , . . . , φr } to either represent the state x, with φi ∈ H1 , or to represent the output y, with φi ∈ H2 for i = 1, . . . , r. We consider a reducedbasis for the state in this study (C = I ). By representing an This work is supported in part by the Air Force Office of Scientific Research (under contract AFOSR FA9550-05-1-0449) and the National Science Foundation (under contract NSF DMS-0513542) Jeff Borggaard is with the Department of Mathematics and the Interdisciplinary Center for Applied Mathematics, Virginia Tech, Blacksburg, VA 24061-0531, USA
[email protected]
1-4244-0210-7/06/$20.00 ©2006 IEEE
r
xr (t) = ∑ ai (t)φi ≡ Φa(t),
(3)
i=1
a set of differential equations for the coefficient functions, a, can be obtained by projecting the state equation (1) Φa(t) ˙ − ΦT AΦa(t) − ΦT Bu(t), ΦH1 = 0
(4)
for the linear system, or Φa(t) ˙ − F (Φa(t), u(t)), ΦH1 = 0
(5)
for the nonlinear system, with initial conditions
Reduced-order models have a number of practical applications. These include fast predictions for forecasting, optimization [6], or uncertainty analysis [1] for complex PDE systems such as CFD [25], [26], [27] and weather forecasting [22]. Other areas, such as exposing fundamental mechanisms in complex systems (eg. turbulent flows [5], [23], [28]) and in developing controls for large-scale or infinite dimensional systems [4], [8], [11], [15], [18], [24] have also been beneficiaries of reduced-order modeling. The principle is that given a complex linear (or nonlinear) model, e.g. x˙ = A x + B u,
approximate state as
Φa(0) − x(0), ΦH1 = 0. The reduced-order models such as (4) or (5) usually lead to significantly faster predictions. In the case −1of Tlinear Φ A Φ, feedback control, the system with Ar = ΦT Φ T −1 T Br = Φ Φ Φ B and Cr = C Φ can be used to design the control u. In the latter case, balanced truncation [24] is the method of choice for large scale finite dimensional [24] or infinite dimensional problems [9], [10]. In the remainder of this section, we present the KarhunenLo`eve expansion (KLE) [16], [21] also commonly known as the proper orthogonal decomposition (POD) [23] and the proposed methodology for producing dynamically optimal basis vectors. In Section II, we discuss approaches for solving the optimality system that arises from applying the necessary conditions to the optimization problem. Natural extensions to this method are discussed briefly in Section III. We demonstrate the effectiveness of this approach for linear and nonlinear problems using a numerical example in Section IV. We close by presenting conclusions and some directions for future work. A. KLE/POD Basis Vectors The most common approach for computing basis vectors, especially for nonlinear problems [2] is the Karhunen-Lo`eve expansion [16], [21]. This approach has roots in a variety of fields and goes by many names including principle component analysis [13], [24] and empirical eigenfunctions [22]. Given a set of simulation data xd , we choose the kth KLE/POD vector recursively as the one that minimizes 2 Z k 1 T J (φk ) = (6) xd (t) − ∑ a˜ j (t)φ j dt T 0 H1 j=1 subject to the conditions a˜ j (t) = xd (t), φ j H ,
1150
1
j = 1, . . . , k,
(7)
and φi , φ j H1 = δi j , for i, j = 1, . . . , k. The inner-product space in (6) and (7) is usually taken as L2 (if H1 ⊂ L2 ) or 2 (if H1 ⊂ 2 ). This is usually stated in the equivalent form of maximizing a Rayleigh quotient, but we use the statement above to contrast KLE/POD with the methodology below. In typical implementations, the integral in (6) is replaced by quadrature since the data usually comes from discrete time experiments or numerical simulations. Additionally, the KLE/POD basis may be obtained by utilizing the singular value decomposition of the data xd (t) =
∞
∞
j=1
j=1
∑ σ j φ j ψ j (t) = ∑ φ j a˜ j (t)
(φi , φ j = δi j and (1/T )ψ j 2 = 1) and with a finite dimensional Hilbert space or a finite number of time snapshots, xd (t) =
N
∑ φ j a˜ j (t).
j=1
Thus, with KLE/POD, if the amplitude coefficients are computed using the projection (7), there is precise estimate of the truncation error in the reduced-order model (3) 1 T
Z T 0
xd (t) − xr (t)2H1 dt = Z ∞ 1 T ∑ σ j ψ j (t)φ j T 0 j=r+1
2 dt = H1
familiar to the optimal control community (cf. [20]). Thus, we seek the set of basis vectors, Φ, that minimize
J (Φ) =
1 T
Z T 0
xd (t) − Φa(t)2H1 dt
(8)
subject to either (4) or (5) and conditions on Φ. Some choices for the latter are • φi , φ j = δi j , i, j = 1, . . . , r (as in KLE/POD) • φi , φi = 1, i = 1, . . . , r (as in the Rayleigh principle) • no constraints In the latter case, issues with scaling, i.e. Φa = cΦ(a/c), may cause problems in the optimization algorithm, so we don’t consider it here. In formulating the Lagrangian using the first two cases, either r(r + 1)/2 or r multipliers are needed. In this paper (as in [29]) we choose the second, i.e. the norm constrained case. Thus, we have the Lagrangian (in the linear case) Z 1 T L (a, Φ, µ, λ) = xd (t) − Φa(t)2H1 T 0 − µ(t)T Φa(t) ˙ − AΦa(t) − Bu(t), ΦH1 r 2 − ∑ λ j φ j H1 − 1 dt. j=1
∞
∑
σ2j .
The first-order optimality system for the finite dimensional case leads to the following equations:
j=r+1
•
The relative error is usually expressed as
The state equation ΦT Φa(t) ˙ = ΦT AΦa(t) + ΦT Bu(t)
∑rj=1 σ2j Er = 1 − ∞ ∑ j=1 σ2j •
where the last quotient is interpreted as the percentage of “energy captured” in the reduced-order model. Note that this estimate does not account for the fact that the eventual reduced-order model comes from a dynamical system (4), (5) or some possible extension. In addition, for the control problem, the reduction is not optimal in the sense that it does not account for the fact that certain states are better expressed in the output than others. This latter case can be better treated by techniques such as balanced truncation [10], [24] if the system is of the form (1)–(2), but not many strategies exist for the nonlinear case (see [19] for one example). Although KLE/POD is remarkably effective, there are a number of problems where it is not, leading to alternative approaches to KLE/POD. A small collection of this work can be found in References [3], [7], [12], [14], [30].
with ΦT Φa(0) = ΦT x(0), the adjoint equation −ΦT Φ˙µ(t) = ΦT AT Φµ(t) + 2ΦT [xd (t) − Φa(t)] (10)
•
with µ(T ) = 0, the r constraint equations φi , φi H1 = 1,
•
i = 1, . . . , r,
(11)
and the r gradient equations −2
1 T
Z T 0
ai (t)xd (t)dt + 2Φ −Φ
+ AΦ
Z 1 T
T
0
Z 1 T
T
0
−B
1 T
Z T 0
1 T
Z T 0
ai (t)a(t)
µi (t)a(t) ˙ + a˙i (t)µ(t)dt
µi (t)a(t)dt + AT Φ
B. Dynamically Constrained Best Approximation A limitation of KLE/POD is that it does not directly use the eventual dynamical system. This is relevant in that a.) there is no guaranteed preservation of stability [30], and b.) the vectors do not account for nonlinearities in the system. A natural approach (also recently studied in [29] for the linear heat equation) is to revisit the KLE/POD formulation in the context of a dynamically constrained optimization problem
(9)
1 T
Z T 0
ai (t)µ(t)dt
µi (t)u(t) − 2λiφi = 0.
(12)
for i = 1, . . . , r. Note that the adjoint equation is forced by the discrepancy between the data and the reduced-order model. This is typical in PDE-constrained optimization. The nonlinear equations above need to be solved for Φ. We outline one such procedure in the next section.
1151
Given an initial (orthonormal) estimate Φ0 (for example via KLE/POD), set the vector λ0 = 0. (1) Using Φn , solve the state equation (9) for an . (2) Using Φn and an , solve the adjoint equation (10) for µn . R (3) Precompute integrals such as 0T aT (t)µ(t)dt (4) Using a quasi-Newton method, solve the coupled equations (11)–(12) for λn+1 and Φn+1 (5) Test for convergence. If necessary, increment n and return to step (1).
1
0.8
0.6
0.4
0.2 0 0.2
0 0
0.4 0.2 0.6
0.4 0.6
0.8 0.8 1
1
t
x
Fig. 1.
Algorithm for solving optimality system (9)–(12) Fig. 2.
Burgers Equation Simulation Data
II. SOLVING THE OPTIMALITY SYSTEM There are numerous approaches for solving the above set of coupled nonlinear equations. We focus on two that decouple the state and adjoint equations from the remaining equations. The first algorithm appears in Figure 1. The actual algorithm we use is a variant in which only one step of the quasi-Newton method in step (4) is taken. This led to faster convergence of the algorithm since less work was spent solving the nonlinear equations with suboptimal values of a and µ. The speedup comes from the fact that steps (1)–(3) are much faster than (4). Note that multiple representations of the same optimal answer exist. For example, from one solution Φ, a simple reordering of the vectors would also be a dynamically optimal solution. The same would be true of any set of linearly independent vectors that span the same space as span(Φ). However, for our examples, we found a tendency to find nearly orthonormal vectors presumably because of the conditions placed on the initial guess. III. POSSIBLE EXTENSIONS This problem could become prohibitively expensive if the dimension of H1 is large and r is substantial (eg. 50-100). An alternative is to restate the problem in a suboptimal manner. For example, determine the size of a computationally tractable set, say r1 , and find sets of vectors in groups of r1 . Thus, we use the above algorithm to find φ1 , . . . , φr1 , fix these vectors and seek the next set of r1 vectors, φr1 +1 , . . . , φ2r1 , etc. It is likely that the larger r1 is taken, the better the resulting reduced-order model though this needs to be investigated. One feature of a POD basis that does not hold of the dynamically optimal basis is the preservation of linear properties. For example, when a reduced-order model is built for an incompressible flow, we know that ∇ · φi = 0 for each i as long as this incompressibility condition holds for each element of the snapshot set ∇ · xd (t) = 0. This can be shown using the singular value decomposition of the data. Thus, one might consider imposing this linear property on the basis vectors through additional constraints. However, this would add even more cost and complexity to the method. A possible extension would be to seek basis elements that are themselves linear combinations of the snapshot elements: e.g. φi = Di xd .
IV. NUMERICAL EXAMPLE In this section, we provide preliminary results for a nonlinear distributed parameter system given by the one dimensional Burgers equation. This test example has been used in other studies of reduced-order modeling for problems of this class [17]. A. Burgers Equation As in [17], we consider Burgers equation xt (ξ,t) +
1 2 x (ξ,t) ξ = εxξξ (ξ,t) on (0, 1) × (0, T ) 2
with boundary conditions x(0,t) = 0 = x(1,t) and initial condition sin(2πξ) ξ ≤ 0.5 x(ξ, 0) = . 0 otherwise Additionally, we take ε as 0.01 and use 40 linear finite elements for the simulation. For these conditions, the data, xd is plotted in Figure 2. We now compute the first two KLE/POD and dynamically optimal basis vectors using the approaches above with H1 = L2 (0, 1). As seen in Figure 3, these sets of basis functions are very similar with the following qualitative differences. Note that the dominant basis vector (blue) has a slight curvature change over the portion of the domain corresponding to a sharper traveling wave. The secondary basis vector (green) has a more pronounced dip than the KLE/POD counterpart and overshoots toward positive values near the right hand side of the interval. By studying the simulations in Figure 4, we see very similar models. However, the changes to the dynamically optimal basis vectors allow a sharper wave and smoother transition between the two modes of the solution. For these two dimensional reduced-order models, there was about a 4% improvement in the value of J : J = .1081 with KLE/POD vs. J = .1035 with the dynamically optimal basis. The model errors are plotted in Figure 5. Comparing the model errors, one sees the dynamic basis functions minimize the error on the left portion of the domain over most of the time history. Additionally, the peak error near x = .5 and t = .5 is minimized.
1152
POD Basis Vectors
0.4
2
0.35 0.3
1.5
0.25
1
0.2 0.15
0.5
0.1 0.05
0
0 0.2
0 0
0.4
0.2 0.6
0.4
−0.5
0.6
0.8
0.8 1
1
t
−1 x
−1.5
−2
0
0.1
0.2
0.3
0.4 0.5 0.6 Dynamic Basis Function
0.7
0.8
0.9
1 0.4
2 0.35 0.3
1.5
0.25 0.2
1
0.15 0.1
0.5
0.05
0
0 0.2
0 0
0.4
0.2 0.6
0.4
−0.5
0.6
0.8
0.8 1
−1
1
t
x
−1.5
Fig. 5. −2
0
0.1
Fig. 3.
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
KLE/POD and Dynamically Optimal Vectors
V. CONCLUSIONS We have demonstrated the potential of creating dynamically optimal basis functions for generating reduced-order models for a nonlinear distributed parameter system. While the approach is more computationally intensive than a pure linear algebra-based approach such as KLE/POD, there are a number of applications where generating the data set would dwarf the additional cost. These would include modeling turbulent flow or solving HJB equations [18]). By definition, this approach will generate a model which is at least as good as any other projection-based approach. The amount of improvement will certainly be system and data dependent. This framework allows a number of extensions. These are listed below and are subjects of future work.
1.2 1 0.8 0.6 0.4 0.2 0 −0.2 0
0 0.2
0.2 0.4
Comparison of Absolute Model Errors
1
0.4 0.6
VI. FUTURE WORK
0.6 0.8
0.8 1
1
t
This reduced-order modeling approach holds the promise for improved accuracy and studying different problem formulations. The improvements here were on the order of 4%. This approach needs to be tested on a number of linear and nonlinear examples to find compelling test cases. Aside from improved model accuracy, perhaps the greatest potential for this approach is the extension of the basic idea of balanced truncation, designing a reduced-order model that best approximates measured outputs. To implement this would amount to changing the objective function (8) to
x
1.2 1 0.8 0.6 0.4 0.2 0 −0.2 0
0 0.2
0.2 0.4
0.4 0.6
0.6 0.8
0.8 1
1
Fig. 4. KLE/POD (top) and Dynamic (bottom) Reduced-Order Predictions
Z
1 T yd (t) − CΦa(t)H2 dt T 0 and keep the same constraints. Another desirable property of reduced-order models is robustness, i.e. quality predictions for off-design values of
J (Φ) =
t
x
1153
parameters. One approach for adding robustness is making appropriate choices for the data. However, it may be possible to derive this property from reformulating the basis selection problem as a min-max problem where maximization would be over model changes. The computational example used here was complex enough to demonstrate the effectiveness of the method, but not computationally challenging. To become a promising approach, alternative solution strategies and implementations need to be explored. Possibilities include adding the orthogonality constraints of the vectors and investigating the effectiveness of sequential implementation discussed in Section III. VII. ACKNOWLEDGMENTS This work is supported in part by the Air Force Office of Scientific Research (under contract AFOSR FA9550-051-0449) and the National Science Foundation (under contract NSF DMS-0513542). R EFERENCES [1] S. Acharjee and N. Zabaras, “Uncertainty propagation in finite deformations–a spectral stochastic Lagrangian approach,” Computer Methods in Applied Mechanics and Engineering, vol. 195, pp. 2289– 2312, 2006. [2] A. Antoulas, Approximation of Large-Scale Dynamical Systems. SIAM, 2005. [3] A. Antoulas, D. Sorensen, and S. Gugercin, “A survey of model reduction methods for large-scale systems,” pp. 193–219, 2001. [4] J. Atwell and B. King, “Proper orthogonal decomposition for reduced basis feedback controllers for parabolic equations,” Mathematical and Computer Modeling, vol. 33, pp. 1–19, 2001. [5] G. Berkooz, P. Holmes, and J. Lumley, “The proper orthogonal decomposition in the analysis of turbulent flows,” Annual Review of Fluid Mechanics, vol. 25, pp. 539–575, 1993. [6] T. Bui-Thanh, M. Damodaran, and K. Willcox, “Aerodynamic data reconstruction and inverse design using proper orthogonal decomposition,” AIAA Journal, vol. 42, no. 8, pp. 1505–1516, 2004. [7] J. Burkardt, Q. Du, M. Gunzburger, and H.-C. Lee, “Reduced-order modeling of complex systems,” in Numerical Analysis 2003, 2003, pp. 29–38. [8] J. Burns and B. King, “A reduced basis approach to the design of low order compensators for nonlinear partial differential equation systems,” Journal of Vibrations and Control, vol. 4, pp. 297–323, 1998. [9] K. Camp and B. King, “A comparison of balancing techniques for reduced order controllers for systems of PDEs,” in Proc. 15th International Symposium on Mathematical Theory of Networks and Systems, 2002. [10] R. Curtain and H. Zwart, An introduction to infinite-dimensional linear systems theory, ser. Texts in Applied Mathematics. Springer-Verlag, 1991, vol. 21. [11] S. Ganapathysubramanian and N. Zabaras, “Design across length scales: A reduced-order model of polycrystal plasticity for the control of microstructure-sensitive material properties,” Computer Methods in Applied Mechanics and Engineering, vol. 193, pp. 5017–5034, 2004. [12] M. Graham and I. Kevrekidis, “Alternative approaches to the Karhunen-Lo`eve decomposition for model reduction and data anaylsis,” Computers and Chemical Engineering, vol. 20, pp. 495–, 1996. [13] H. Hotelling, “Analysis of a complex of statistical variables into principal components,” Journal of Educational Psychology, vol. 24, pp. 417–441, 498–520, 1933. [14] W. IJzerman, “Signal representation and modeling of spatial structures in fluids,” Ph.D. dissertation, Universiteit Twente, 2000. [15] K. Ito and S. Ravindran, “A reduced order method for simulation and control of fluid flows,” Journal of Computational Physics, vol. 143, no. 2, pp. 403–425, 1998. [16] K. Karhunen, “Zur spektraltheorie stochastischer prozesse,” Annales Academiae Scientarum Fennicae, vol. 37, 1946.
[17] K. Kunisch and S. Volkwein, “Control of Burgers’ equation by a reduced order approach using proper orthogonal decomposition,” Journal of Optimization Theory and Applications, vol. 102, pp. 345– 371, 1999. [18] K. Kunisch, S. Volkwein, and L. Xie, “HJB-POD-based feedback design for the optimal control of evolution problems,” SIAM Journal on Applied Dynamical Systems, vol. 3, no. 4, pp. 701–722, 2004. [19] S. Lall, J. Marsden, and S. Glavaˇski, “A subspace approach to balanced truncation for model reduction of nonlinear control systems,” International Journal of Robust and Nonlinear Control, vol. 12, pp. 519–535, 2002. [20] E. Lee and L. Markus, Foundations of Optimal Control Theory. Krieger Publishing Company, Inc., 1986. [21] M. Lo`eve, Probability Theory. Princeton, NJ: Van Nostrand, 1955. [22] E. Lorenz, “Empirical orthogonal functions and statistical weather prediction,” Cambridge, MA, 1956. [23] J. Lumley, “The structure of inhomogeneous turbulent flows,” in Atmospheric Turbulence and Radio Propagation. Moscow: Nauka, 1967, pp. 166–178. [24] B. Moore, “Principal component analysis in linear systems: controllability, observability, and model reduction,” IEEE Transactions on Automatic Control, vol. AC-26, 1981. [25] B. Noack, G. Tadmor, and M. Morzy´nski, “A hierarchy of lowdimensional models for the transient and post-transient cylinder wake,” Journal of Fluid Mechanics, vol. 497, pp. 335–363, 2003. [26] J. Peterson, “The reduced basis method for incompressible viscous flow calculations,” SIAM Journal on Scientific and Statistical Computing, vol. 10, no. 4, pp. 777–786, 1989. [27] C. Rowley, T. Colonius, and R. Murray, “Model reduction for compressible flows using POD and Galerkin projection,” Physica D, vol. 189, pp. 115–129, 2004. [28] L. Sirovich, “Turbulence and the dynamics of coherent structures, Part I: Coherent structures,” Quarterly of Applied Mathematics, vol. 45, no. 3, pp. 561–571, 1987. [29] K. Willcox, O. Ghattas, B. van Bloemen Waanders, and B. Bader, “An optimization framework for goal-oriented, model-based reduction of large-scale systems,” in Joint 44th IEEE CDC and ECC conference, Seville, Spain, 2005. [30] K. Willcox and A. Megretski, “Fourier series for accurate, stable, reduced-order models in large-scale applications,” SIAM Journal for Scientific Computing, vol. 26, no. 3, pp. 944–962, 2005.
1154
