(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
A02-13810 AIAA-2002-044 Quaternion-Based Partial Derivative and State Transition Matrix Calculations for Design Optimization James D. Turner, Ph.D.
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
NADS and Simulation Center Oakdale Blvd. Iowa City, Iowa 52242-5003 319.335.4672
[email protected]
Presented to: 40 AIAA Aerospace Sciences Meeting and Exhibit Reno, Nevada 14-17 Jan 2002 th
Session: Multidisciplinary Design Optimization
ABSTRACT A quaternion-embedding algorithm is presented that permits a single function evaluation of a vector-valued function to simultaneously produce the function value and the columns of the Jacobian matrix corresponding to the embedded variables. The algorithm is shown to provide state transition matrix partial derivatives when quaternion embedded vector functions are numerically integrated. The quaternion embedding algorithm transforms Cartesian coordinates by adding a small parameter, h, times a hyper-imaginary direction. An error analysis is presented that demonstrates that the partial derivatives are accurate to O(h2). Several examples are presented to demonstrate the methodology and solution accuracy for scalar, vector, and matrix quaternion functions. Key Words: Quaternions, Jacobian, State Transition Matrix, Parameter Partial Derivatives, Clifford and Geometric Algebra.
INTRODUCTION Many design optimization algorithms require sensitivity partial derivatives for iteratively improving a candidate design. There are two basic approaches for generating the partial derivatives: (1) analytic models, and (2) approximate numerical algorithms. The analytical models are normally preferred, but are not always available. Numerical approaches sample a function to predict its sensitivity. Finite difference algorithms (PDA) are standard approaches for developing approximate numerical sensitivity models.
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
Two or more function samples are generally required. These methods are simple to implement, but sensitive to the perturbation step size used for sampling the function model. A complicating problem with PDA is that a vector difference needs to be computed to evaluate the partial derivative, where the accuracy of the remaining expression is unknown. Alternatively, this paper presents a quaternion-based numerical sensitivity algorithm that does not involve computing a vector difference. A single function call provides both the function evaluation and the partial derivatives. Up to three partial derivatives can be computed for each function call. The quaternion algorithm is shown to extend to state transition matrix and parameter sensitivity calculations. The quaternion algorithm embeds Cartesian coordinates into a space of higher dimension. The immediate benefit of this strategy is that the higher-dimensional space samples the underlying problem sensitivities, no special coding is required. Of course, one must suspend the use of conventional algebraic operations and use quaternion algebraic operations. The embedding algorithm works at the level of the Cartesian coordinates x, y, and z, which are scalar components of a 3d vector. After the transformation, the scalar components become new quantities that consist of both scalar and hyper-complex vector parts. For the sensitivity algorithms presented here, the quaternion embedding introduces a small perturbation, h, into the hyper-complex part of the quaternions. The transformed x, y, and z coordinates become the perturbed quaternion coordinates x + ih, y + jh and z + kh , where /, y, and k the represent hyperimaginary directions. Note that independent perturbed directions are selected for each Cartesian coordinate.
Quaternion Taylor series expansions are used to analytically demonstrate that the introduction of quaternion perturbations allows a single function call to simultaneously provide (1) the desired function value, and (2) recover the sensitivity partial derivative, without computing finite differences, by sampling the hyper-complex part of the quaternion. The paper consists of three parts. The first Section reviews the basic definitions and operational rales for computing quaternion algebra. Both general function and integrated function values are reviewed for quaternion-embedded methods, as well as a detailed Cartesian coordinate embedding example. The next section presents the bulk of the mathematical material for the paper, covering numerical techniques such as Finite Difference Algorithms, a simplified Complex Variable Formulation for sensitivity calculations, the quaternion embedding sensitivity algorithm, and the use of Taylor series for aiding in the interpretation of the quaternion results and establishing the accuracy of the numerical results. Numerical results are presented for computing a nonlinear vector function and numerically integrating a 3D spacecraft trajectory problem. An appendix is provided that presents analytical equations for all the standard quaternion library functions. Also presented is a Macsyma algorithm for handling quaternion linear matrix equation solutions, a 4x4 example inversion is presented.
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
QUATERNION ALGEBRA Hamilton6'7, in 1843, invented quaternion algebra. He was attempting to extend complex variables and analysis to three dimensions. Quaternions belong to a broader class of algebras that are now known as Clifford8'9 and geometric algebras10. Quaternions consist of a scalar part and vector part that are written as follows: q = a + ib + jc + kd where i, j, k represent hyper-imaginary directions that are subject to the operational rules:
i2 = j 2 = k 2 = -1, ij = -ji = k, jk = -kj = i, ik = -ki = -j.
These rules are summarized in the following multiplication table.
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
1 i j k
i -1
-k j
k -j i -1
j k -1 -i
Table 1: Quaternion Multiplication Table
A summary of quaternion forms for standard mathematical operations is presented in Appendix A. These results are easily derived, if one remembers that the product of quaternion unit vectors satisfy the relationship u*u = -1, which follows from A-3. Care must be exercised in building models because a*b # b*a. Quaternion-Embedded Functions and Integrals:
For an engineering design application, assume that one has a scalar function of three variables given by F(x,y,z) = a*. By embedding the x, y, and z components into quaternions, one obtains F( x + ih, y + jh,
= a + ib + jc + kd
(1)
where a, b, c, and d are obtained by evaluating F(*) using quaternion operations. The meaning of a* is clear from conventional experience. Mathematical interpretations of a, b, c, and d, however, are still required. Similarly, when quaternion embedded functions are integrated:
(2) where the initial and integrated vectors are defined by:
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization. /
.7 x0 +. ih
\
/
,.x\
/
. s+ixi
+ jx.j+kx,k
\
(3)
one needs to establish interpretations for a^fi^^O in Eq. (2). To this end, the remainder of the paper is devoted to demonstrating, both analytically and numerically, that the hyper-complex parts of Eqs. (1) and (2) generate the following partial derivatives and state transition matrix partial derivatives
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
dF — =l\b c d\lh J dr dG(t' ) — = \P 7 0]/h
JacobianPartial Derivative (4)
State Transition Matrix
Several numerical examples are presented to demonstrate the accuracy of the method.
MATHEMATICAL MODELS FOR SENSITIVITY The goal of this section is to present algorithms for computing partial derivatives. The classical PDA algorithm is presented first, so that the potential problems with this approach can be highlighted. Next a simplified complex variable algorithm is presented to motivate the more general approach, and provide insight into the potential accuracy available from an embedding class of methods. Last, the complex variable case is generalized to handle quaternion algebra. This section describes the basic quaternion embedding strategy, and Taylor series-based approximation techniques for evaluating the quaternion embedded vector functions. Higher-order quaternion Taylor expansions are examined to analytically establish the accuracy of the approximations for both the real and hyper complex parts of the embedded functions. Series expansions are used to establish that (1) the Jacobian partial derivatives are obtained from function evaluations, and (2) that state transition matrix partial derivatives are generated by integrating vector quaternion embedded functions. Background Material The basic finite difference algorithm is presented, and several numerical issues are discussed. Next a complex variable algorithm is studied for exploring alternative algorithms for the finite difference approaches.
A. Finite Difference Algorithm: Approximations for partial derivatives of a real vector function F(x) at point x0, are obtained by using finite difference formulas, such as:
dF_ dx
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
where h is the perturbed value for XQ. Choosing a reliable value for h is a problem dependent process that is often subject to experimentation. Different variations of the formula above are available for symmetrically computing the partial derivative about x05. Some authors have developed techniques for understanding the behavior of functions as h becomes small3. A principal source of error arises because a difference must be computed between two vector-valued functions, where the number of valid digits in the resulting expression is unknown. The strength of the approach is that it is simple to compute. Nevertheless, the problem remains complicated for many applications. The resulting calculations provide a single column vector of the sensitivity partial derivatives, when F is a vector. For simple problems, one can develop analytic sensitivity models, unfortunately, this process can rapidly get complicated for involved applications. B. Complex Variable Formulation: The idea of using complex variables for computing approximations for scalar functions has been known since the work of Lyness and Moler1, Lyness2, and Squire and Trapp4. In the algorithm, h is allowed to become complex. The resulting calculations are surprisingly simple. Taylor expanding a scalar function, y(x), as a truncated series, demonstrates the method. The key step in the algorithm is to replace the differential change dx with a small complex value ih, where i = sqrt(-l). The resulting Taylor series can be shown to be: 2
-h>d
dx where the real part of the function follows as:
y2
dx
+ itf jcO
dx3
(5)
d2y and the imaginary part of the function follows as:
dx
•*v
xv
For both R and I in Eq. (5) only the first terms survive when h2 is small. The real part becomes the original fiinction evaluated at xO, and the imaginary part becomes h*dy/dx. The partial derivative for dy/dx is obtained by dividing I by h. As a result, one obtains dy/dx without having to compute the difference of two functions, as required by FDA. Both the function and partial derivative calculations are valid to h2. This level of accuracy motivates an exploration for using high-dimensional algebras for recovering sensitivity information for vector-valued functions. The simplest Clifford8'9 and geometric algebra10 are considered in this paper: the quaternion algebra of Hamilton6'7.
Quatertonic Sensitivity Algorithm The original ideas of Lyness and Moler1 are generalized for quaternion algebra, by adding hyper complex perturbations to Cartesian coordinates. The resulting algebraic manipulations require quaternion operations (See Appendix A for a Library of Quaternion operations and functions). As in the complex variable algorithm, the basic
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
idea is to replace the value of the perturbations dx, dy, and dz with a hyper-imaginary quaternion embeddings. This approach has two advantages. First, the Jacobian partial derivatives are recovered using a single function call. Second, no vector difference is required to calculate the partial derivatives. Section A presents the basic algorithm for embedding the Cartesian coordinates into the higher dimensional quaternion coordinates. Section B presents a Taylor series approximation for a standard Cartesian coordinate model. The quaternion-embedding algorithm is presented in Section C. A. Embedding Algorithm: The Cartesian coordinate quaternion embedding algorithm leads to vector coordinate transformations of the form:
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
r=
y + iO+jh + kQ z + iQ + jO+kh)
y
(6)
where the Q,q subscript denotes the quaternion embedded Cartesian vector. B. Standard Taylor Series: In order to understand the behavior of functions containing quaternion embedded Cartesian vectors, one uses standard Taylor expansions of a vectorvalued function for the form: , dF dx + — dy A=0 h=0
dF_ dz
.. + H.O.T.
(1)
where H.O.T denotes higher-order terms in the series. This approach is useful because the quaternion embeddings are being introduced as "small" hyper complex terms. The series allows us to interpret the function values and establish the accuracy limits for the resulting expressions. Equation (7) is generalized for quaternion perturbations in the next Section. C. Quaternion Embedded Taylor Series: A quaternion version of Eq. (7) is obtained by introducing the change of variables defined by (dx, dy, dz) = h(i, /, k];
h\
leading to F (x + ih,y + jh, z + kti)= F
=
+ih
+ kh
•h ———
dx
3 h=0
dy
dF_ dz
+ .. + H.O.T.
(8)
The hyper-imaginary part of Eq. (8), contains the Jacobian as follows: dF
'dF dx
F
u • • s — w hyper-imaginary (——
dr
h
6
.
dF dx
.
dF dz
(9)
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
which appears to be accurate to at least O(h). The only restriction in Eq. (9) is that all of the operations in F(x,y,z) must be computed using quaternion algebra. C.I Accuracy of the Quaternion Solution: To establish the accuracy of the expansion of Eq. (8), it is expanded to 2nd , leading to:
[dx
dydz
dydz
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
I OF 7( d2F ,7 -^- + h\ —---—=2- V jh + \-=- +7 h\ —-——— — 71 \>kh dy dzdx dxdzjj [dz (dxdy dydx)\ where the quaternion multiplication defined in Table 1 has been used to simplify the 2nd order quaternion products. Because F is assumed to be continuous through 2nd order, the order of the partial derivatives is unimportant, leading to the reduced expression
- 2 ^ dx dy dz which demonstrates that the quaternion embedding method yields solution accuracies valid to h2 for both the real and hyper complex parts of the function. These results imply that Eq. (9) is valid to h2. Note, if F is replaced by a gradient expression, then one can obtain elements of the 2nd order partial derivative matrix, which is useful for root solving and optimization algorithms. D. Partial Derivative Algorithm: In real applications the Taylor expansion of Eq. (8) is not used, instead one computes:
+ jh,z + kh) = a + ib + jc + kd
(10)
where a, b, c, and d are evaluated using quaternion operations and the partial derivatives are obtained as follows
dF 9r
L
_...
'
:
II. c//i
:'
dF
^/M \ d/h] = \—— [_dx
5F
:• —— ': dx
5F
~ VvZ,2x —— +O(h ) dz _
x (11)
ni
Observe, however, that it was necessary to investigate the Taylor expansion of Eq. (8) in order to establish Eq. (11). E. State Transition Matrix Algorithm: The state transition matrix is obtained by integrating quaternion-embedded models of the form
(12)
(c)2002 American Institute of Aeronautics & Astronautics or Published with Permission of Author(s) and/or Author(s)' Sponsoring Organization.
and using quaternion Taylor series to identify the resulting functional forms. Assuming that the r,s, and t represent the indices of the subset of variables that have undergone quaternion embedding, then Taylor expansions of Eq. (12) lead to
dx r dx 7 dx , _ dF dx . dF dx 7 dF dx , TT^™ ,-^ x +—ch r +——dx s +—dxt =F +——dxr +————dx s +——dxt+H.OT.(l3) dxr dxs dxt * dx dxr dx dxs dx dxt where the changes in dxp, p = r,s, and t are given by the quaternion perturbations dxr = ih, dxs - jh, dxt = kh Introducing the quaternion perturbations into Eq. (13) and observing that only the r,s, and t components of the partial derivative exist, leads to
Downloaded by James Turner on March 5, 2014 | http://arc.aiaa.org | DOI: 10.2514/6.2002-448
dx .. dx .. dx ., x + ——ih+——jn +—kn = dx, dx,' dx,
dF
dF dxn
dF dx
•kh\ (14)
Equating terms in Eq. (14) leads to
dx dxr
dx dxs
dx dxt j
dF_d^ &, In /2
u-?J
DESCRIPTION Addition Subtraction Multiplication Division Magnitude Conjugation Unit Vector
Angle
Power Natural Log Exponential Cosine Sine Tangent Arc-Cosine Arc-Cosine Arc-Tangent Hyperbolic Cosine Hyperbolic Sine Hyperbolic Tangent Arc-Hyperbolic Cosine Arc-Hyperbolic Sine Arc-Hyperbolic Tangent
All operations assume quaternion algebra for +9 -, /, and * calculations. For example, in A-22 for evaluating sinh'^q), one computes q2 using A-3, q2 +1 using A-l, ^q2 +1 using A-9, q + ^q2 +1 using A-1, and finally In I q + ^q2 +11 using A-10. All functions are available in Macsyma and Matlab formats. Also available are power series, numerical integration, and matrix algebra.
16
