850
ON A‘IITOhfATIC CONTROL, VOL. AC-29, NO.
TRANSACTIONS IEEE
[9] G . Casalino, F. Davoli, R Minaardi, and R.Zoppoli, “Sufficient statistics in team control problems with a common past,” in Proc. 21sl IEEE Con/. Decisron and Conrr.. Orlando, FL. pp. 186-190.1982. [IO] T. Yoshikawq “Decomposition of dynamic team decision problems,” IEEE Trans. Aulomaf.
[email protected]. AC-23, pp. 627-632,1978.
9, S E F ” B E R 1984
In our summary of results for K h a n filtering we draw heavily upon the work of Morf, Kailath, Anderson, and Moore. (See 121 and [3] for our previousreferences to the appropriate literature.) In ourderivation of scaling rules and expressions for rounding error variances we adapt the stationary results of Jackson [4] andMullisandRoberts [5] to our nonstationary problem. SIGNAL MODELSFOR STATIONARY ARMA TME SERIES
Fixed Point Implementationof Fast Kalman
A zero-mean second-order stationary time series {)>(I)}is said to be autoregressive moving average (ARMA) if the entries y ( t ) in the time series obey this recursion for all r:
PrediCt0l-S
P
LOUIS L. SCHARF IWD SIGURDUR SIGURDSSON
n=O
4
b(n)u(t-n)
a(n)y(f-n)= n=O
Eu(t)u(t+n)=u26(n)
Eu(r)=O
Abstract-In this note westudy scaling rules and roundoff noise variancesinafixed-pointimplementationofthe galman predictorforan ARMA time series observed noise free. The galman predictor is realized in a fast form that uses the so-called fast Kalman gain algorithm. The algorithm for the gain is fixed point. Scaling rules and expressions for rounding e m r variances are derived The numerical results show that the fixed-point realization performs very dose to the floating point realization for relatively low-order ARMA time series that are not too narrow band. The predictor has been implemented in 16-bit fixed-point arithmetic on an INTEL 8086 microprocessor, and in Idbit floating-point arithmetic on an INTEL 8080. Fixed-point code was written in Assembly language and floating-pointcode was writtenin Fortran. Experimentalresultswere obtained by running the fixed- and floating-point fiiers on identical data sets. All experiments were carried out on an INTEL MDS 230 develop ment system.
INTRODUCTION
a,, = bo = 1
Markovian Representation
The Markovian representation for { y ( t ) } is summarized in the following equations, where n = ma(’, q), and where for purposes of illustration we have assumed q > p . State Equations: x(t+l/t)=A~(t/t-l)+h(l)u(t) x(O/-
1) : N ( 0 , Q); u ( r) y ( r)
Manuscript received November 21, 1983; revised February 9, 1984 and April 9, 1984. This work was supported by the Office of Naval Research under Contract N0014-82-K-0300. L. L. Scharf is with the Department of Electrical Engineering, University of f i d e Island, Kingston, RI 02881. S. Sigurdsson was with the Department of Electrical Engineering. Colorado State University, Fort Collins. CO 80523.
= C’X(
= N(0,u2)
t / t - I ) + U( t ) .
Unit Puke Response: h(t)= State Covariance:
Finite-dimensionalGaussiantimeserieshave stationary Markovian state-spacedescriptions. In suchdescriptions the initialconditions are multivariate normal and state variables are predicted values of the series based on an infinite past of observations. The linear filtering problemis one of estimating the state at time t based on observations up to time r and the prediction problem is one of predicting the state at time t + 1 based on observations up to time t. Corresponding to theMarkovian representation istheinnovations representation. The essential characteristic of this nonstationary representation is that it may be used to synthesize a time-series,starting from zero initial conditions, whose second-order statistics matchthe statistics of the originaltimeseries.The states arepredictedvalues of thetimeseries based on a finite past of observations. Using this representation,the Kalman predictor may be written down from inspection as the causal and stable inverse of therepresentation.Theso-called K h a n gain in the innovations representation (or equivalently in the K h a n predictor) may be associatedeither with the Levinson recursionsfor factoring the inverse of the correlation matrixfor the time seriesor with the LeRoux-Gueguen latter association recursions for factoring the correlation matrix itself. The leads to afixed-pointalgorithmforcomputingKalman gains. This so-called fast algorithm produces afast Kalman filter. In this note we present results from astudy of fast Kalman predictors, implemented in floating point and in fixed-point arithmetic, for autoregressive moving average time series. More extensive results of this study, for noisyandnoise-freefilteringand prediction, maybe found in the thesis of Sigurdsson [l].
6( n ) : Kronecker delta.
i
0, t 0.
R(t)=A‘Q
Q = AQA’+ u2h(l)h’(1). Output Covariance:
r(r)=c‘R(r)c+u2h(t). Design Equations:
Innovations Representation
The idea behind the innovations representation for { y ( t ) } is to replace the stationary initial conditions, distributed as x@/-1): N ( 0 , Q ) with the nonstationary initialconditions x(O/-l) = 0, toreplacethestationary input vector h(1) by a nonstationary Kalman gain vector k ( t ) , and to replacethe stationary iid sequence ( u ( t ) ) , distributed as u ( t ) : N(0, u2), with the nonstationary iid innovations sequence { u ( t ) } , distributed as u ( t ) : N(0, ~ ( t ) )The . trick is to choose the Kalman gain k ( t ) and the innovations variance v(t) correctly. The innovations representation is summarized below. It isworthnoting that k ( t ) -,h(l), v ( t ) + u2, and Q(1) + Q as t + co.
0018-9286/84/0900-0850$01.00 01984 IEEE
LEEE TRANSACTIONSON AUTOMATIC CONTROL, VOL.
AC-29,NO. 9, SEPTEMBER 1984
851
Stare Equations:
v=diag(u(O);.-,u(t))
x(r+l/t)=Ax(r/t-l)+k(f)u(r) x(O/-1)=0;
u(t):N(O,o(t))
y ( t ) = c'x(t/t -l)+u(t).
hO(1)
h'(0)
hyt)
h'(t-1)
Unit Puke Response: h'(n)=
1
0, n 0.
State Covariance:
R'(n) =A"Q(t) Q(0) = 0.
Output Covariance:
r'( n ) = c'R'( n ) c + c r ( t ) h'( n ) ,
0 0
]
1
0
1
0
A= -
4.1
hl(0)
This means the correlation matrix may be factored with the fast impulse response algorithm of LeRoux-Gueguen [6]to obtain H , and n-dimensional columns of H may be picked off to obtain k ( t). SCALING AND ROUNDING IN THE KALMANPREDICTOR
Q(t+l)=AQ(t)A'+v(r)k(t)k'(t)
Design Equations:
.. .
Let z denote quantization step size and 1~2'"-'[ thebound on the maximum magnitude that may be represented in an m-bit, signed binary representation. The problem in afinite-wordlengthrealization of the Kalman predictor is to scale variablesso that the probability of overflow is small. Scaling: A filter is said to be I , scaled if m
fi(n ) = )€2m-'/612
(12
1
ll=O
c=[o
1
1
-4 )
0
where 6 is a parameter that may be increased to force the norm of j k to be small, and thereby decrease the probability of overflow. Theparameter u2 is the variance of the noisy excitation of the filter. The variablej k ( n ) is the response of the kth state variable for the filter to a unit pulse sequence appliedat the input. In the Kalman predictor equation wemayscale the state vector x( t / t - 1)by a diagonal scaling matrixS
k( t ) tl' ( t ) = A ( Q - Q ( f ) ) c + 0 2 h ( 1 ) ; Q = AQA'+ u 2 h ( l ) h ' ( l ) V ( t ) = r(0)-c'Q(t)c;r(0)
=
'Qc + u2.
In these equations, h'(n) is the response of the system of equations at time r + n to an impulse applied at time t , R'(n) is the state covariance between x(f) and x(f + n ) , and r'(n) is the output covariance between y ( r ) and y(f + n). If r'(n) is to equal r(n), to match the second-order properties, then k ( t ) and u ( t ) must be chosen as above in thedesign eipiltiOns. Kalman Predictor: In the innovations representation, identify c'x( t + l / t ) as theprediction y ( t +l/t). Thenthe Kalman filter equations become simply a rewritingof the innovations representation y(r+l/t)=c'x(r+l/t) x(t+l/r)=Ax(t/f-l)+k(r)[y(t)-c'x(t/t-l)]
The scaled Kalman predictor equations are then
The I, scaling rules for the states of the Kalman predictor are
'
where S - denotes inverse of the transposeof S. The state variance Q ( t ) may be computed recursivelyas outlined previously, or nonrecursively as I
A"-'k(f-n)(A"-'k(t-n))'.
Q(t)=v(t) n =1
Y ( t ) =Y ( f > .
The diagonal terms of Q ( t ) converge monotonically to the upper limit Q = AQA'+ 02h(l)h'(l). A practical procedure is to replace Q(r) by Q under the innovationsequations by solving a system of Riccati equations. to obtain the scaling rule An alternative is to note that thegain is related to thetimevarying impulse responseof the innovations representation S ( k ) = [Q]if6/2"'-'c (k=1,2,-..,n). This result for stationary state space filters is due to Mullis and Roberts h'( n) = c'A"-' k ( f ) , n>0. 151. This means the elements of k ( t) may be read out as follows: The algorithm for the gain need not be scaled becauseit is fixed point: d internal variables are bounded by unity in magnitude. Rounding: The updating of the state vector in the Kalman predictor requires one multiply for the first ( n - 1) elements and ( n + 1) multiplies for the nth element. By associating a sequence of iidvariance 2/12, random variables with eachfixed point multiply, we generate a mean zero, variance N random vector n ( t ) each time we update: Fast Algorithm for k(t): 'The gain k ( t ) may be calculated as outlined
As the correlation matrix for responses, we may write
y(t)
is related to the time varying impulse
En(t)n'(t R = HVHT
where the
tXt
x(t+l/t)-S-'ASx(f/t-l)+S-'k(t)u(t)+n(f)
matrices R , V , and H are defined as follows:
+ k) = N6(k)
-
852
EFE TRANSACTIONS ON AUTOMATIC CONTROL, VOL.
AC-29,NO. 9, SEPTE?dBER 1984
n
m P
1
Fig 1.
Fig. 3.
The state variance is now Q(t+l)= (S-’~S)Q(t)(S-’AS)‘+v(t)k(t)k’(t)+N Q = (S’AS)Q(S-’AS)’+ o2h(1)h’(l)+N.
N
~
c
A
L
E
S
Refer tothe abstract for a summary of how all experiments were conducted. What followsis a brief annotation of Figs. 1-4. Experiments were conducted by generating realizationsof a stationary time series from the model
H( z ) =
1- 1.752-’ + O . ~ Z - ~ 1-1.5~-’+1.212-~ -0.4550z-’
Fig. 1 illustrates k ( t ) , a 3 x1 vector,computed with floating-point arithmetic (curves 1-3) and with fixed-point arithmetic (curves 4-6). Fig. 2 illustrates the innovationvariance u ( t ) computed in floating point (curve 1) and in fixed point. Thesetwo curves illustrate that the data-independent fast Kalman gain calculation maybe practically computed in fixed point using 16-bits. Fig. 3 flustrates predictions y ( r / t - 1) in the floating-point realization (curve 1) and in the scaled fixed-point realization of the Kalman predictor. As y ( t / t - 1) is a linear combinationof state variables, the numerical results provide experimental evidence that state variables are correctly scaled. Fig. 4 illustrates the corresponding innovations sequences. The floating-point and fixed-point predictors were run over the same realization of the time series. CONCLUSIONS The experimental results indicate numericalstability of the fixed-point calculations of the gain k(t) andvariance u ( t ) usingLorinson-type
recursions. The predicted results values for p ( t / t - 1) and innovations u ( t ) indicate that the scaling d es are effective over the nonstatiomy and stationary regimes of the problem and that rounding variances are predicted accurately from the theory. The states are apparently properly scaled and the fixed-point innovations sequence is distributed approxiinnovations. mately exactas the There is more work to be done on this problem. First, balanced as opposed to standard state-space realizations should be investigated for Second, for a variety of signal further reducingroundingvariances. spectra, a parametric study of performance versus number of bits in the fixed-point representation should be conducted. Third, rounding effects on thecalculation of the Kalman gain should be investigated.These effects have not been modeled in the paper. In our discussion of fixed-pointrounding errors we havecorrectly assumed that variables are scaled. Conversely,in our discussion of scaling rules, we have used infinite precision covariance matrices, ignoring the effects of rounding errors on scaling rules. This second-order coupling between scaling and rounding is probably insignificant,but further investigation may be in order. REFERENCLS [I] S.Sigurdsson, “Fast Kalman filtering for ARMA processes: Fixed point implementaColorado State University, Ff. Collins. CO, June 1982. tion,” MS. [2] C. Gueguenand L. L. scharf,‘‘Exact maximum l i k e l i h d identification of ARMA models; A signal processing perspective,”io Proc. EUSIPCO, L a m e . Swieerland, Sept 1980. 131 J. P. L. L. Scharf, and C. J. Gueguen, “Exact likelihood for s t a t i o n a r y vector autoregressive moving averageprocesses.”presentedatthe workshop on fast alge rithms in linear systems.Aussois, France, sept 1981. [4] L. B. Jackson. “On theinteraction of roundoffnoiseanddynamicrange in digital filters,” Bell Sysr. Tech. J.. vol. 49, 1970. [ 5 ] C. T.MuIlis and R. A Roberts, “Synthesis of minimum roundoff noise b e d point digital Nte~,” I E E E Trans. CirnU’tr Syst.,vol. CASZ3. Sept 1976. [6] J. LeRoux and C. J. Gueguen, “A fixed point computation of partialcorrelation coefficients.” IEEE Trans. Accurt., Speech, Signal Processing, voL ASSP-25, pp. 257-259, June 1977.
thesis.
Dugre.
