International Symposium on Signal Processing and its Applications, ISSPA, Gold Coast, Australia, 25-30 August, 1996. Organized by the Signal Processing Research Centre, QUT, Brisbane, Australia.
A FORWARD BACKWARD ALGORITHM FOR ML STATE AND SEQUENCE ESTIMATION* Gary D. Brushe1,2,3
Robert E. Mahony3
John B. Moore2,3
1
Signal Analysis Discipline, Communications Division, ESRL, DSTO, Bldg. 203L, PO Box 1500, Salisbury, SA 5108, AUSTRALIA. Tel. +61 8 259 6662, Fax +61 8 259 6328, E-mail:
[email protected] 2
Department of Systems Engineering, RSISE, Australian National University Canberra, ACT, Australia. 3
Cooperative Research Centre for Robust and Adaptive Systems c/- RSISE, Australian National University, Canberra, ACT, Australia.
be used instead of the inner VA, however it has no respect for the existence of path constrained state sequences.
ABSTRACT The classical Viterbi algorithm is used to estimate the maximum likelihood state sequence from a block of observed data. It achieves this by maximising a forward path probability measure. In an analogous manner a backward path probability measure can be generated which leads to the development of a Viterbi forwardbackward algorithm. This algorithm computes an “a posteriori maximum path probability” for each state at a given time. The resulting probability distribution across all possible state at time t can be used as a soft output for further processing. Maximising a posteriori maximum path probability at each time gives the same state sequence as obtained from the classical Viterbi algorithm. 1.
In order to obtain path constrained state sequence estimates and soft outputs, modifications to the VA have been developed. Forney [1] considered “augmented outputs” for the VA, these included: the depth at which all paths merged; the difference in length between the best two paths; and a limited list of the best paths. Hagenauer and Hoeher [3] developed a soft output VA by modifying the VA to deliver not only the most likely path sequence, but either an APP for each bit (determined from the MAP algorithm) or a reliability value for each bit of the hard decision output. Vucetic and Li [4] provide a survey of soft output algorithms. Since this survey, Nill and Sundberg [5] and Li et. al. [6] have also developed other soft output algorithms.
INTRODUCTION This paper develops a forward backward algorithm for ML state and sequence estimation. The algorithm is derived from the classical VA. This is done by calculating the forward path probability (exactly as in the VA) and in an analogous manner calculating a backward path probability. Combining the forward and backward path probabilities gives an a posteriori probability for each state at each time maximised over all valid state sequences passing through that state. We term this probability the a posteriori maximum path probability (AMPP). A probability value is obtained for each state at each time yielding uncertainty information about each state estimate and providing soft outputs. We demonstrate that by choosing the state estimate which has the maximum AMPP at each time, produces a sequence of state estimates which is the same as would be obtained via the classical VA’s backtracking procedure.
The classical Viterbi algorithm (VA) [1] determines the Maximum Likelihood (ML) state sequence from a block of observed data. It achieves this by maximising a forward path probability and obtaining a path constrained state sequence estimate via a backtracking procedure. The backtracking procedure produces hard decision state estimates. The maximum a posteriori probability (MAP) algorithm [2], [1, Appendix] determines the states which are individually most likely. This is achieved by computing an a posteriori probability (APP) for each state at each time and the most likely state estimate is determined by choosing the state with the maximum APP. The state estimates obtained in this manner may not form a path constrained state sequence as obtained with the VA. However, the APPs do provide a confidence level for each state estimate and these could be used as soft outputs.
2.
Digital communication applications are increasingly using two VAs in a concatenated way, for example in demodulation, decoding and equalisation problems [3]. If the inner VA produces only hard decision state estimates, then the outer VA can not use its capability to accept soft decisions. The MAP algorithm could •
THE VITERBI ALGORITHM
In this section we briefly review the classical VA and introduce the notation used in this paper. A description of the signal model and further information about the notation can be found in [7].
The authors wish to acknowledge the funding of the activities of the Cooperative Research Centre for Robust and Adaptive Systems by the Australian Commonwealth Government under the Cooperative Research Centre Program.
224
International Symposium on Signal Processing and its Applications, ISSPA, Gold Coast, Australia, 25-30 August, 1996. Organized by the Signal Processing Research Centre, QUT, Brisbane, Australia.
K
The VA [1] determines the ML state sequence over the time interval {1,2, , T } given a sequence of observations This is accomplished using forward O = {O1 , O2 , , OT }.
K
γ~t (i ) = max max P[q1 ,K, qt −1 , qt = Si , qt +1 ,K qT , O ] ,
K
K
q1 , , q t −1 q t + 1 , , qT
1 ≤ i ≤ N, 1 ≤ t ≤ T.
dynamic programming [8]. Let δ t (i ) denote the ML probability measure, that is, the probability of being in state Si at time t
This probability measure determines the probability of being in
maximised over all possible paths which end in state Si , with the observed data up to time t:
δ t (i ) = max P[q1 ,K, qt −1 , qt = Si , O1 , O2 ,KOt ] q1 , q 2 ,
Kq
t −1
(7)
state Si at time t maximised over all possible paths which pass through state Si , given the observed data sequence O = {O1 , O2 , , OT }. We will refer to this probability measure as the “a posteriori maximum path probability” (AMPP) measure.
K
(1)
This can be computed via the recursion [9]:
[
To compute Eqn. (7), we split it into two parts.
]
δ t (i ) = max δ t −1 ( j )a ji bi (Ot ) , j
δ 1 (i ) = π(i)
1≤i≤ N
(2)
γ~t (i ) = max P[q1 , K , q t −1 , q t = S i , O1 , K Ot ].
K
2≤t≤T
q1 , , qt −1
K
qt + 1 , , qT
and associated with an estimate of the most likely state at time T, qˆT = arg max[δ T ( j )]
backward probability measure† and can be computed inductively, (analogous to calculating β t (i ) , the backward probability measure described in Rabiner’s tutorial on hidden Markov models (HMMs) [9]) as follows:
qˆ1 ,K, qˆT −1 = arg max P[q1 ,K, qT −1 , qT = qˆT , O ]. (4)
K
q1 , , qT − 1
In practice the maximising state sequence is extracted using a backtracking process. This is done by keeping track of the argument which maximised δ t (i ) at each time,
[
]
1≤ j ≤ N
1≤i≤N
1)
3.
2)
[
]
(10)
T–1≥t≥1 ~ The AMPP, γ t (i ) , is a probability measure for every state at each time, based on path constraints. To obtain a state sequence estimate from the VFBA, simply choose the state with the ~ maximum AMPP, γ t (i ) , at each time t, i.e.,
(6)
qˆt = arg max[γ~t (i )]
In this section we derive a Viterbi forward-backward algorithm (VFBA) which computes an a posteriori probability measure for each state at each time but which is maximised over all valid paths which pass through that state. This probability measure gives a degree of confidence for the state estimate obtained at each time. The probability measures could also be directly used as the soft outputs to a next stage VA, as required in communications systems which use concatenated VAs.
i
1 ≤ t ≤ T.
(11)
Note: That the choice of qˆt (as the state with the maximum AMPP) at any time t, clearly defines only one state sequence estimate - the ML state sequence, see Eqn (7). ~ As stated previously the AMPP measure, γ t (i ) , of the VFBA computes the probability of being in state Si , at time t, maximised over all possible paths which pass through state Si ,
Consider the VA’s ML information probability measure δ t (i ) , Eqn. (1). Assuming that one is given a block of observed data O = {O1 , O2 , , OT } then it is reasonable to consider a new
K
(9)
Recursion:
~ ~ β t (i ) = max β t +1 ( j )aij b j (Ot +1 ) 1 ≤ i ≤ N 1≤ j ≤ N
A VITERBI FORWARD-BACKWARD ALGORITHM.
probability measure at time t ∈ {1,
1≤i≤N
(5)
and then backtracking to obtain the most likely state sequence estimate ie., t = T-1, T-2, ... ,1.
Initialisation: ~ β T (i ) = 1
2≤t≤T
qˆt = ψ t +1 (qˆt +1 ) ,
(8)
Observe that δ t (i ) is a forward probability measure and is exactly ~ the standard VA’s ML probability measure, while β t (i ) is a
there is an estimated state sequence,
ψ t (i ) = arg max δ t −1 ( j )a ji ,
]
~ = δ t (i )β t (i )
(3)
1≤ j ≤ N
[
max P q t +1 , K qT , Ot +1 , K , OT q t = S i
K,T } determined by:
† Observe that the Markov property ensures that once q = S is taken t i as known, all other prior data becomes redundant.
225
International Symposium on Signal Processing and its Applications, ISSPA, Gold Coast, Australia, 25-30 August, 1996. Organized by the Signal Processing Research Centre, QUT, Brisbane, Australia. conditioned on the observed data sequence O = {O1 , O2 ,
K
, OT }. ( ) δ i (Eqn. 1), This compares with the VA’s probability measure, t which determines the probability of being in state Si , at time t, maximised over all possible paths which end in state Si , given the observed data up to time t. When t = T the two measures ~ ( γ t (i ) and δ t (i ) ) are equivalent, however the VFBA is computationally more intensive than the VA in obtaining a state sequence estimate and therefore would only be used if there was a requirement to obtain probability measures (soft outputs) instead of actual hard-decision state estimates as obtained via the VA.
3
To illustrate that the same state sequence estimate is obtained with both the VFBA (when the state with the maximum AMPP is chosen) and the VA, consider the simple four-state trellis covering 5 time units, Forney [1, Fig. 8(a)]. The complete trellis, with each branch labelled with a length is reproduced in Figure 1. The state sequence estimated via the VA [1, Fig. 8(b)] is reproduced in Figure 2. Figure 3, shows the steps of the VFBA. The solid lines show the surviving paths from the forward probability measure, δ t (i ) , while the dotted lines show the ~ surviving paths from the backward probability measure, β t (i ) . At each state the lower left number represents the path length due ~ to δ t (i ) , the lower right number, β t (i ) , while the top centre ~ number represents the combined path lengths, γ t (i ) . The state surrounded by a dotted circle is the state with the shortest path (or maximum AMPP) and hence would be the estimated state. It is easily seen that the states surrounded with dotted circles correspond with the state sequence obtained via the VA shown in Figure 2.
1
1
1
1
0 2
2 0
2
1
0
1
1
1 0
2
Figure 2:
1
1
1
1 Figure 1:
Four-State Trellis, 5 time units with branch lengths labelled.
226
State sequence estimate obtained via the Viterbi Algorithm.
International Symposium on Signal Processing and its Applications, ISSPA, Gold Coast, Australia, 25-30 August, 1996. Organized by the Signal Processing Research Centre, QUT, Brisbane, Australia. probabilities computed using the HMM forward-backward algorithm, rather than via a backtracking procedure as used in the VA. The Viterbi forward-backward algorithm computes an a posteriori probability for each state at each time maximised over all valid state sequences passing through that state. REFERENCES [1] [2]
[3]
[4]
[5]
[6]
[7]
[8] Figure 3:
[9]
Steps to obtain state estimates using the Viterbi forward-backward algorithm.
[10] Remark: It is possible to reduce the computational complexity of the VFBA by applying known HMM based techniques, (i.e. on-line implementation using fixed-lag or sawtooth-lag smoothing [10]). Reduced state Viterbi techniques [11] - [13] may also be applicable.
[11]
The Viterbi forward-backward algorithm is computed in an analogous manner to the HMM forward-backward algorithm. The similarity of these two algorithms is investigated in a companion paper [14] and explained in more detail in [7]. On going research into practical applications of these ideas to areas such as soft-output Viterbi and robust state sequence estimation is being undertaken. 4.
[12]
[13]
CONCLUSION
[14]
This paper derived a Viterbi forward-backward algorithm which can be used to produce the same state sequence estimate as the Viterbi algorithm. This is acheived by maximising probability measures (soft outputs) at each time, analogous to the MAP
227
G.D. Forney, Jr., “The Viterbi Algorithm”, Proc. IEEE, vol. 61, Mar. 1973, pp. 268 - 278. L.R. Bahl, J. Cocke, F. Jelinek, J. Raviv, “Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate”, IEEE Trans. on Inform.. Theory, vol. IT - 20, pp. 284 - 287, Mar. 1974. J. Hagenauer and P. Hoeher, “A Viterbi Algorithm with Soft-Decision Outputs and its Applications”, IEEE Globecom ’89, Dallas, Texas, Nov. 1989, vol. 3, pp 47.1.1 - 47.1.7. B. Vucetic and Y. Li, “A Survey of Soft-Output Algorithms”, Proc. International Symp. on Inform. Theory and its Appl., Sydney, Australia, Nov. 1994, pp. 863 - 867. C. Nill and C-E. W. Sundberg, “List and Soft Symbol Output Viterbi Algorithms: Extensions and Comparisons”, IEEE Trans. on Communications, vol. 43, no. 2, Feb. 1995. Y. Li, B. Vucetic and Y. Sato, “Optimum Soft-Output Detection for Channels with Intersymbol Interference”, IEEE Trans. on Inform.. Theory, vol. 41, no. 3, May 1995. G.D. Brushe, M.E. Mahony and J.B. Moore, “A Soft Output Hybrid Algorithm for ML/MAP Sequence Estimation, submitted for publication to IEEE Trans. on Information Theory. J.K. Omura, “On the Viterbi Algorithm”, IEEE Trans. on Inform.. Theory, Jan, 1969, pp. 177 - 179 L.R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”, Proc. IEEE, v. 77, no. 2, Feb. 1989, pp. 257 - 285. V. Krishnamurthy and J.B. Moore, “On-Line Estimation of Hidden Markov Model Parameters Based on the Kullback-Leibler Information Measure”, IEEE Trans. on Signal Processing, vol. 41. no. 8, Aug. 1993, pp. 2557 - 2573. ( M.V. Eyubo g lu and S.U. Qureshi, “Reduced-State Estimation with Set Partitioning and Decision Feedback”, IEEE Trans. Commun., vol. COM-36, no. 1, Jan. 1988, pp. 13 - 20. A. Duel-Hallen and C. Heegard, “Delayed DecisionFeedback Sequence Estimation”, IEEE Trans. Commun., vol. COM-37, no. 5, May 1989, pp. 428 - 436. G.D. Brushe, V. Krishnamurthy and L.B. White, “A Reduced Complexity On-Line State Sequence and Parameter Estimator for Superimposed Convolutional Coded Signals”, submitted to IEEE Trans. on Communications. M.E. Mahony, G.D. Brushe and J.B. Moore, “An Investigation of the relationship between ML and MAP Sequence Estimation Algorithms”, Proceedings of the 4th. ISSPA, Gold Coast, Aug. 1996.
