Node Synchronization for the Viterbi Decoder - Caltech Authors

524

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-32, NO. 5, MAY 1984

Node Synchronization for the Viterbi Decoder GARY LORDEN, ROBERT J. McELIECE, SENIOR MEMBER,

IEEE, AND LAIF SWANSON

Abstract-Motivated by the needsof NASA’s Voyager 2 mission, in this paper we describe an algorithm which detectsand corrects losses of node synchronization in convolutionally encoded data. This algorithm, which would be implementedas a hardware device external to a Viterbi decoder, makes statistical decisionsshout node synch based on the hard-quantized undecodeddatastream. ,We will show that in a worst-case Voyager environment,ourmethodwilldetect and correct a true loss of synch (thought to be a very rare event) within several hundred bits; many of the resulting outages will be corrected by the outer Reed-Solomon code. At the same time, the mean time between false alarms is on the order of several years, independentof the signal-to-noise ratio.

In thisarticle we shall showthatthisdata loss due to spurious node synch loss in the Viterbi decoder is completely avoidable. Our proposed solution involves disabling the Viterbi decoder’s internal synchronizationhardware and implementing an external node synch algorithm. Our .algorithm is easy t o implement and depends on likelihoodcalculations based on observations of the hard-quantized encoded data stream. In a worst-case Voyager environment, our method will detect and correct a true loss of node synch within several hundred bits; many of these outages will, be corrected by the Reed-Solomon code. On the other hand, the mean time between false alarms for our technique (which is independent of the SNR) is on the order of several years. Thus, for practical purposes our techniI. INTRODUCTION que introduces no false alarms, and the system SNR loss due , no loss of ECENT studies [ 11, [ 21 have shown that at the very low to node synch problems will be e ~ i n a t e d with losses. As anoutboard signal-to-noise ratios NASA’s Voyager- 2 mission will protection against truenodesynch encounter at Uranus in 1986, the performance of the Reed- hardware device, our algorithm could be implemented on a Solomon/Viterbi concatenatedcoding system couldbe seriously single Deep Space Network standard single-board computer degraded by erratic behavior in the node synchronization sub- such as the iSBC 86/12, atleast at data rates up to 20 kbits/s. The paper is divided into three sections (this is Section I). system of NASA’s hardwaredecoder. In thispaper we will show that this problemcan be avoided by disabling the Viterbi In Section 11, we present a functional description of our algodecoder’s internalsynchronization hardware a n d , replacing rithm, together with a summary of the relevant mathematics. it with a simple outboard hardware node synchronizer whose In Section 111, we present some numerical performance results forour technique.They will quantifythe assertions made details we describe in this paper. For definiteness, this paper will deal only with the system parameters relevant to Voyager, above (mean time between false alarms, probability of uncorbut our technique is applicable (with minor modifications) to rectable errors due to true loss of synch, etc.). We also include in the Appendixes some background information. any convolutionally encoded telecommunications system. On Voyager, the high-rate downlink telemetry is protected 11. THE UP-DOWNCOUNTER by a K = 7 , rate 1/2 convolutional code concatenated with a depth-4 interleaved (255, 223) Reed-Solomon code. In princiWe adopt the following model, which has been found to be ple this combination provides. excellent error protection (bit very accurateforcoherent deep-space communication [ 41 . error probability 1.OE-6) for Voyager’s highly sensitive imag- The information to be transmitted via the convolutional code ing data, at bit signal-to-noise ratios as low as 2.9 dB. Since the (which in Voyager is already encoded) is a sequence ..., J L 4 , rate of the outer code is 223/255 = -0.6 dB, when theoverall M - 2 , M o , M z , M4, ..* of independent identically distributed SNR is 2.9 dB, the inner convolutional code is operating at random variables, eachequally likely t o be 0 or 1. The enabout 2.3 dB. coded stream :-, C--3, C - 2 , C- 1 , Co, C1, -..is defined by the However, in practice, the performance of the concatenated encoding equations’ system is significantly worse than theoretical predictions. One problem is carrier-loop jitter, which degrades performance by C2k =M2k +M2k-2 +jg2k-4 +IM2k-6 +’2k-12 0.5 dB or more [ 11. This means that if the system bit SNR remains at the nominal 2.9 dB value, the inner convolutional code must operate at less than 2.0 dB. This is a value much lower than that for which the Deep Space Network’s hardware Viterbidecoders were designed. In thisdemanding .environ(2) 2) (mod ment,theViterbi decoder’s internalnodesynchronization hardware, whose function is to detectandcorrecttrue external losses of node synch, is prone to produce false alarms, We also define the versions of the encoded stream: i.e., spurious losses of node synch, and send useless data to the Reed-Solomon decoder until node synch is reestablished. In [2]it was shownthat thishardware problem can degrade Voyager’s performance by a further 1.O dB or more.

R

*

, ,

matical SocietySummerMeeting, Toronto, Ont., Canada, August 1982. Manuscript received March 11, 1983; revised October 14, 1983. This work was supported by a contract from the National Aeronauticsand Space AdministraI We shill illustrate all of our results for the NASA standard K = 7 , rate 1/2 tion. The authors are with the Jet Propulsion Laboratory, California Institute of convolutional code, but everything generalizes easily to any rate 112 convoluTechnology, Pasadena, CA 91109. tional code.

0090-6778/84/0500-0524$01.00

0 1 9 8 4 IEEE

LORDEN e t a l . : NODE SYNCHRONIZATION FOR

525

VITERBI DECODER

detectiop and demodulation,a sequence { b k } is received, tect loss of synch, i.e., a sudden change making the received where Dk = D k z k . The sequence { z k } is the error se- sequence obey t h e , out-of-synch hypothesis,Amethodfor quence. If the noise process is additive white Gaussian noise, doing thissimply can be based on a general statistical technique [ 7 1 for detecting a change in distribution.’To apply this thenthesequence { z k } is i.i.d.,thecommondistribution being Gaussian, mean zeroand variance u2 = l/p, where p technique it is necessary to simplify the model by assuming X i , are independent.They are not thatthe paritychecks is twice the symbol SNR. widely The Viterbi decoder attempt: to recover the message bits independent for even i, but the dependence between from the noisy code sequence { D k } . It does this by making a separated Xi’s is slight, so calculations based on this assumpvery efficientmaximumlikelihood estimate of each of the tion should be illustrative. message bits { M 2 / } . However, in order to operate, the Viterbi The method for detectingloss of synch is based on acounter decoder must have node synch, i.e., it must know which of the with increments received symbols have even subscripts and which have odd subscripts. Of course, there are only two possibilities, but if the wrong hypothesis is made, the output of the Viterbi decoder will bear no useful relationship to themessage stream { M 2 i ) . Our algorithm will provide node synch information for the Viterbidecoder. It is based onthe hard-quantized received where p(X,) and q ( X , ) are the likelihoods of X,,under the in-synch and out-of-synch hypotheses, respectively. As shown sequence { R k } , where above 0 ifLjk>O if n is odd 1/2 Rk= 1 if ik < 0. if n iseven and X, = 0 n

+

{

+

Clearly R k = Ck Ek (mod 2), where { E k } is i.i.d, and Ek = 0 or 1. The error probability p e = Pr {Ek = l} is given by

.I

1-n

if n is even and X , = I

whereas q(X,) reverses the odd and even cases. Thus,

l r n pe = ~ r ; i e - t 2 / 2 dt

where, as before, p = 2 E s / N o . Associated with the hard-quantized received sequence { R k } is the syndrome, or parity-check sequence {x,} [ 51 :

The counter is defined by

To = 0 T, = max ((T+

n=

1

+ (1 - 2pe)’0 2

The foregoing describes thedistribution of the received sequence and of the parity-check sequence in case node synchronization is maintained.This will be called the in-synch hypothesis. The out-of-synchhypothesis describes the situation when node synchronization is in error. In this case the R i s and Xi’s behave as though the subscripts were shifted by one.Thus,underthe out-of-synch hypothesisit is theodd parity checks that are correct with probability n and the even ones that are purely random. We assume thatnodesynchronization hasbeenacquired and maintained and that the in-synch hypothesis is initially true. We wish to monitor the received sequence so as to de-

+ L(X,)),

O),

n 2 1.

A threshold y > 0 is chosen, and the process stops the first time T , 2 y. Since at this point there is a substantial likelihoodratio infavor of the out-of-synch hypotheses,theinference is made that loss of node synchronization has occurred. This loss of node synch can be remedied by either adding or deleting a channel symbol; the node synchronizer will alternate addinganddeletingsymbolsin order to avoid ruining frame boundaries in case of false loss of synch. The performance of such a counterfora particular y is characterized by two average run lengths (ARL’s). 1) The short A R L . This is the average number of pairs of symbols needed to reach the threshold if the out-of-synch hypothesis is true from the beginning. 2) The long A R L . This is the average number of pairs of symbols needed to reach the threshold if the in-synch hypothesis remains true. The short ARL gives an upper bound on the average time between loss of syhch and its detection, since whenever loss of synch occurs, the counter has a nonnegative value. The long ARL (or its reciprocal) describes the frequency of false detection of loss of node synchronization. An exact determination of the ARL’s will be made in Section 111, for a, very slight modification of the scheme described. It is instructive, however, particularly for later comparisons, to consider their asymptotic behavior as y + m. It turns out that short ARL

-Y

-

I

and long ARL

Cer,

526

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-32, NO. 5 , MAY 1984

so that short ARL

-

TABLE I DEPENDENCE OF INFORMATIONNUMBERS ON SYMBOL ERROR PROBABILITY

log (long ARL)

Z P

Here the constant I , which is much more critical than C, is the Kullback-Leibler information number per pair of symbols, This number is simply the average increment of*the counter per pair of symbols when the out-of-synch hypothesis is true. Since each pair of symbols generates one even X, and one odd

12

I

0.08

0.5875

0.0154

0.0155

0.0309

0.10

0.5537

5.78 X

5.80X

0.0116

0.12

0.5321

2.07 X

2.07 X

4.14 X

X,,

,

TABLE II RELATIONSHIPBETWEEN AVERAGE RUN LENGTHS

i = I1 + I , where

I

P

I , = q(0 I n odd) log 27r

= n log (2n)

1,

T

+ 4(l I n odd) log (2(1 - n))

0.1

0.5537 001157

+ (1 - 77) log (2(1 - n)) 0.08

0.5875 0.03091

and 202

m

Short A R L

Long A R L

IO

118

547

15

205

2,062

20

296

6,693

IO

88

.1.164

IS

145

20

7,496 44.850

check error is quite high. For example, symbol error probability 0.1 corresponds to 7r = 0.45. Thus; in-synch data with Table I shows the dependence of the information numbers p e = 0.1 will fail an even parity check with Probability 0.45, while out-of-synch data will fail the even parity checks with should be noted that on, p e , the symbol error probability. It the information numbers decrease substantially as the symbol probabiiity 0.5. A counter to distinguish between distributions error probability becomes larger. Thus, the ARL’s become less which are so close will either require a long time to react to incorrect synch orhave a large probability of incorrect ,change. favorable as p e increases. If one makes the very slight change of inspecting the counter Performance numbers for such a counterare indicated in Table I1 above; . . ’ onlyat even n, i.e., oncefor each pair of symbols, thena If the pafity checks were independent, there would be no simpler description of the counter is possible. This is because two consecutive X,’s = 0 (successful parity checks) yield a net way to improve thisperformance [7] But the parity checks are change of zero, as do consecutive X,’s = 1. If, however, one of not independent. This ,is because, under the in-synch hypothechanges the value of five even the pair of X,’s is 0 and the other 1, then the total increment sis, one channel symbol error of the counter is easily seen t o be log ( n / l - n),with if parity checks. Under the in-synch hypothesis, for example, an ( n - 12)thchannelsymbol will cause and only if the odd X, = 0 (Le., both a successful check for isolated errorinthe theout-of-synchhypothesisandafailureforthe in-synch parity check failures at time n , n - 2 , n - 6 , h - 8, and n hypothesis). Thus, the counter moves up and down by a fixed 12. Thus, a long sequence of successful parity checks would step size and standard random walk formulas [3, p. 3511 can lead us to believe that another success is on the way, while the sequence X n T l ~= 1, X n - l ~= 0, X n - g = 1, X i - 6 = 1, be used to derive exact formulas forthe ARL’s underthe = Oi. X n P 2 = 1 would lead one to believe that X, is simplifying assumption of independence. Assuming without very likely to be 1. The reason a log-likelihood counter works loss of generality that so well in the independent case is that by adding logarithms we n multiply their arguments. After receiving parity checks Xi = y = m log >xl, X 2 = x 2 , Xi,= x , , where each x i is 0 or 1, the counter 1--n contains m an integer, one has

+

-e,

and long ARL = ( n - 0.5)-

1 - (1 - 7r)/n i=

Table I1 dlustrates the relationship between the two ARL’s as a function of m for two symbol error probabilities, 0.08 and 0.1.

Counter with Memory A simple parity check counter does a fairlygood job of node synchronization in case of high SNR [ 51 . But in the case of high symbolerrorprobability,theprobabilityof parity

1

If the parity checks were independent, this would be exactly

log

(

q(X1 = X I ,

X, =X,, -.*,X, = X , )

@(X1 = X I ,

X , =x,,

...)X ,

=X , )

the log of the ratio of the likelihood of the string (Xl = x l ,

527

LORDEN et al.: NODE SYNCHRONIZATION FOR VITERBI DECODER

X, = x2, X , = X , ) under the two hypotheses, which is In general, the counter using k parity checks has increment statistically optimal for detectingloss of synchronization. Inthe case of noisyconvolutionallyencoded data,the probability of a string of parity checks is not just the product of the probabilities of the individual parity checks. Updating the p probability of a string requires the conditional probability p ( X , = x , I X,- = x , X,- 2 = X , - 2 , ..., X 1 = The system which we have investigated in detail is therefore a system which takes hard-quantized received channel symbols x , ) . Therefore, a counter with increment R l , R 2 , ..., creates parity checks X l 4 , X,5, ... with X i = R i Ri-1 f R i - 3 Ri-4 f R i - 5 + Ri--6 f Ri-7 + R i - 1 0 + Ri+R i- 3 , and keeps a counter whose increment at time n is e*.,

,,

,+

,

-

would, after step r, contain

In orderto design this counter, we need to know P ( X , = x , 1 X n - 2 - x , - 2 , "', X n - 2 k + 2 = X , - 2 k + 2 ) and 4 Of the sameevent for even andodd n. To calculatethese values, remember that for in-synch data, the even parity checks depend only on the sequence of channel symbol errors. so given a probability p e of symbol error, we can calculate the probability that E , = e,, E,- 1 =en- 1 , " ' 9 E n - 2 k - 1 1 = e n - 2 k - 1 1 ..., e n - 2 k - 1 1 , and use these for eachsequence e , ,e n - 1 , probabilities to CalcUlatep(X, = X , , " ' , X n - 2 k + 2 = X n - 2 k t 2 ) and p(X,-2 = X n - 2 , "', X n - 2 k + 2 = X n - 2 k + 2 ) and find p(Xn = x n l X n - 2 = xn-2, X n - Z k + Z - x r 1 - 2 k + 2 ) for even n. For odd n , just as in the case of the simple counter, the X,'s are independent with p ( X , = 0) = 1/2. Thus "'3

P(Xn exactly the log of the quantity we desire. Of course, a real counter will not take into account a past of indefinite length. But a (possibly large) integer m can be chosen, and a counter constructed whose increment at time n is

As might be expected, for large m , the information number approaches theinformationnumberforthehypothetical counter based on the indefinite past of the parity check sequence. This is verified in Appendix A. Moreover, the informationnumbersobtained using theparity checksequence are identical with information numbers obtainable from the hard quantized received sequence. This means that there is no loss of information or efficiency in using the parity check sequence instead of thehardquantized received sequence to detect loss of node synchronization-this is also shown in Appendix A. Also, Appendix A shows thatthecounterincrements depend only on parity checks of the same (odd or even) type, i.e., (8) is unchanged if the given = = ..*)is replaced by ( X n p 2 = x n P 2 , X n - 4 = x , - 4 , ...). We will describe thecounter in terms of thenumber of parity checks used to determine the counter increment, and we will call this k . For example, the "simple" (memoryless) parity check counter corresponds to k = 1, while the counter f o r k = 8 has counter increments -

4 ( X , = x, I X n - 2 = x n - 2 , " ' , X n - 1 4 - x n - 1 4 )

P(x, = X ,

-

IXn-2 =Xn-2j.'**,Xn-14 -xn-14)

,

= x n ~ / x n - 2= x n - 2 , . . ' , X n - 2 k + 2 = X n - 2 k + Z )

= 1/2 for odd

n.

To calculate the probabilities 4 associated with the out-ofsynch hypothesis, just exchange even with odd in the above argument. From these values, we can calculate thecounter increments, We did this assuming a channel symbol error rate of 0.1, which corresponds to a Viterbi decoded bit error rate of 5 X loe3, the standard for imaging data. Voyager's data rate has been adjusted so that this is the largest channel symbol error rate which will be encountered.

Information Numbers and Run Lengths Just as in the case of the up-down counter, the average run lengths are essentially determined by the threshold y and the informationnumber I, i.e., the average increment of the counter when the data are truly out of synch. For large y the run lengths are approximately short ARL and long ARL

- YI

-

-

CeDr

where C , D, and I depend, of course, on k , the number of parity checks used indetermining thecounterincrement. These approximations were borne out by the simulations reported in Section 111, and the constants C and D were determined empirically for each k considered. Z, the information number, is the average increment of the counter when the data are truly out of synch.(Of course, I depends upon k , the number of parity checks used in determining the counter increment, and is an increasing function of it.)

528

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-32, NO. 5 , MAY 1984

TABLE III INFORMATION NUMBERS k

Total

Out-of-sync

16

0.0862

0.0601

0.0261

8

0.0633

0.0406

0.0228

’

In-sync

The information I from a pair of parity checks is the sum of the information number I , from the odd paritychecks (the average increment of the counter at odd n ) and I2 from the even parity checks. Table I11 shows the contributions of these two parity check subsequences, revealing that the odd checks contribute roughly two thirds of the total information. In designing thedetectionalgorithm, we must choose a symbol error probability p e , at which the counter is designed to operatemostefficiently. In practice, as thetruesymbol errorprobabilityvariesfwiththe signal-to-noise ratio, it will generally be smaller than p e , so that fewer parity check failuresoccur. In this case, thecounter will performbetter whether the data are in synch or not, i.e., the short ARL will be reduced and the longARL increased. In case the signal-to-noise ratio is degraded so much that the symbol error probability exceeds p e , however, a problem arises in the performance of the counter, as both the short and long ARL’s become less favorable. In reallife, thesymbol error probability is not constant, and the fact that the long ARL’s shorten duringperiods of low signal-to-noise ratio would introduce the same spurious loss of synch which seems to plague the current Viterbi decoders. If the short ARL gets longer during a period of high symbol error rate, this causes the response time to a loss of synch to increase and may cause a string of data to be lost if a true loss of synch hits at thetime of low signal-to-noise ratio, but a shortening of the long ARL whenever there is such a period will have a far greater effect on the overall behavior of the system. This degradation of long ARL can be totally eliminated by giving up the information I z from the even parity check subsequence. When the counter uses only the odd subsequence, the long ARL is unaffected by the signal-to-noise ratio. This is because the odd checks are completely random (independent, with 50 percent probability of success) whenever the process is in synch. Since most of theinformation comes from the odd subsequence anyway, we believe it is prudent to base the counter on this subsequence alone. The performance parameters in the next section were all calculated for counters based solely on theout-of-synch paritychecks.Itshouldbe repeated, however, that these performance figures are based on the assumption that the data are a sequence of i.i.d. random variables equallylikely to be +1. If theactualdatastream deviates significantly from this model, e.g., if long strings of zeros or ones are probable, then the performance of our algorithm will not be as good as predicted.

111. PERFORMANCE NUMBERS The size of the ROM needed for counter increments is 2 k . As before, we will describe the counter in termsof the number k of parity checks needed to compute the counter increment. Once k is chosen, the log-likelihoodscheme determines the counter increments. The only question left in algorithm design is the threshold at which the system is declared out of synch. If the threshold is high, the probability of false loss of synch is low, or equivalently, the time between false losses of synch is long. On the other hand, a high threshold will also

make the short run length (or the time between loss of synch and detection of that loss of synch) large. We first consider the influence of short run length on system performance. There is an obvious reason to want the short run length to be small: the sooner after aloss of synch that the system gets back ontrack,thebetter. But thereisanother reason as well. Voyager has aconcatenated codingscheme: after the convolutional code is Viterbi decoded, an additional code, the Reed-Solomon code, is decoded, As far as the ReedSolomondecoder is concerned,thedatastream during the short run is just a stream of bad data. (Of course, if the out-ofsynch condition was caused by the deletion of a symbol, and the node-synchronizer solves the problem by deleting another symbol, then a whole bit has been deleted, and frame boundaries are lost as well. In this case, the data during the short run can never be recovered. So we will consider the case in which the total number of channel symbols has not been changed. This will be true when the out-of-synch condition was caused by a spurious loss of synch caused by the node synchronizer, since the synchronizer will alternate adding and deleting channel symbols, and will be true half of the time anyway.) The Reed-Solomon decoder can recover a fair amount of bad data, and so if the short run is short enough, the Reed-Solomon decoder will beable to recover it most of the time. So the length of the average shortrun is not as important as the probabilitythatthe Reed-Solomon decoder will be able to recover thedataintheshortrun.Forthe k = 8 counter, assuming Viterbi burst statistics for 2.3 dB [8] and depth 4 interleaved Reed-Solomon words, Table IV shows the probability of decoding for a word in a frame wholly containing a short run. This same table shows the long run lengths (both in bits and in time, at the reasonable Voyager data rate of 20 000 bits/s) for these same thresholds. Looking, for example, at threshold 15, we see thattheprobabilitythata word containedina short run will be corrected by the Reed-Solomon decoder is 2/3, and that synchis lost incorrectly every two days. Can we do better? In fact, by going to k = 16, we can do much better. Table V shows this same information with k = 16. With k = 16and threshold 14.5, a word which is in a frameattacked by ashortrun will decodecorrectlywith probability 0.86, and the mean time between false losses of synch (average long run time) is 6.7 years. These numbers were obtained by simulation methods explained in Appendix B. Several otherquestions can be asked abouttheperformance of the counter. What if a short run hits more than one frame? With k = 16 and threshold 14.5, the probability that a short run hits more than one frame is 0.033. And even if the short run does intersect more than one frame, the probability that it causes a word error in each frame is less than 0.005. Thus, the probability of a decoder error in each of two consecutive frames because of a spuriousloss of node synchronization is less than 0.0002. Another question is the probability of some loss of data due to a short run. We saw that in the case k = 16, threshold 14.5, theprobabilitythat any one word fails to decode is 0.14, but since decoding failures in the four words are by no means independent, this does not tell us the probability that there is some loss of data-that is, that one or more of the four interleaved Reed-Solomon words fails to decode. In the case k = 16, threshold 14.5, this probability is 0.19. This counter is “tuned” to the channel; that is, counter increments are based on probabilities, calculated from the channel model, of each block under the “in-synch hypothesis” and the “out-of-synch hypothesis.” To make a counter for a channel with other than independent errors, one would make the same calculations using probabilities for that channel.

6.0

529

LORDEN et al.: NODE SYNCHRONIZATION FOR VITERBI DECODER TABLE IV COUNTER PERFORMANCE WITHK = 8

Threshold 116.6

P r o b atbhial ti t y a Reed-Solomon Word a S h o r t Run Will Decode

Mean S h o r t Run ( b i nt s )

Mean Long Run

Bits

&.

5

10

238.6

Time ( a t 20.000 bps)

6.1

.98

x

lo4

3s e c o n d s

5.2

.86

x

lo7

43minutes

363.3

15

4.5

.66

x 1010

2 6d a y s

387.6

16

1.7

.62

x 10"

1 0 0d a y s

438.0

18

2.6

.52

x 1OI2

4 years

487.4

20

.43

x

60 y e a r s

3.8

TABLE V COUNTER PERFORMANCEWITH K = 16

Threshold

Mean S h o r t Run ( b i nt s )

5

88.6

171.4

10

.93

196.4

Probabilitythat a Reed-Solomon Word a S h o r t Run Will Decode .99

x 106

7.9

.95

x

5 minutes

lo9

4d a y s

d x a lyo s lo 221.7

5.0 x 1011

290days 6 . 7y e a r s

.87

4.2 x 1 0 l 2

305.1

.77

6.1 x

20

338.8

.71

1.0 x l 0 l b

APPENDIXA

PROOFS This Appendix gives mathematical proofs of the statements made in Section 11. We use the same random processes to model the situation: 1) .-, M - 2 , M o , M 2 , ... an i.i.d. process, P ( M 0 = 0) = P(M0 = 1) = 1/2, representing the message; 2 ) .-, E - 2 , E - 1 , EO,E l , ... an i.i.d. process, P(E0 = 1) = symbol error probability, representing the errors in the received sequence and independent of the sequence -., M - 2 , M o , M 1 , '-; 3) ..-,R-2, R- 1 , Ro, R , , .-, the convolutionallyencoded Mi's added to the E i s , representing the hard-quantized received channel symbols; and 4) X- 1 , X , , X , , *..the sequence of parity checks derived from the Ri's. = .-, X n - 2 k Proposition: p ( X , = x , , x , - 2 k j = [P(Xn = X,- 2 = x n - 2 , *'.,X n - 2 k = X,-2k).* ..e,

P(Xn-1 = X n - 1 , " ' , X n - 2 k + l - x n - Z k + l ) ] .

Proof: First observe that for rn odd, X,-4,

..., *..E - , ,

Eo, E l , .*.)= 1/2

because X, is a sum of M,+1 and other variables, p(M,+1 = 1) = 1/2, and M,+l is independent of all the random variables on which we are conditioning. This means that for odd m

Xm-21 = x , - ~ , I I . . . , E - ~ , E o , E ~ , . . . )

= 2-1-

1

a.s.

lo1'

960 years 16,000years

for any sequence ( x , , x,- 2 , ..., x , - 2 1 ) of zeros and ones. But the values of the even parity checks are determined entirely by ..., E-.,, E o , E , , -., and so are independent of the odd parity checks. Notice: 1) The fact that we looked at a sequence of odd length was convenient for notation but had no effect on the proof. 2 ) We showed not only that the even and odd parity checksare independent,butthattheoddparity checksare themselves i.i.d. with probability 1/2 of success. 3) The same result holds for the measure q , reversing the roles of even and odd. Corollary: p ( X , = x , 1 X,X n - 2 , .-) = p(X n - x n l X n - 4 , -.), and the same for q . If n is odd, p ( X , = x , l X n - 2 , Xn-4, .*.) = 1/2; for even n , q ( X , = x , JX,-2, - j = 1/2. Definition: The one-sided, pseudoparity checksequence 2 2 , ... is the parity check sequznce based-on the process ..., 0, 0,. R 1 , R 2 ,R-3, That is, X , = R , , X2 = R2 R,, = R , -b R 2 , .-,X, = X , for n 14. Proposition: Every event of the form (X,= x l , X 2 = x 2 , X , = x , , ) corresponds t o exactly one event ( R 1= r l , .-, R , = r, j . Proof: The -2:s are derived from the R;s,Going backwards, knowing X I tells you R 1 , and knowing X i and R 1 , .-, Ri- tells you Ri. Corollary: A log-likelihood counter based on the xis will always containexactlythe same value as one based onthe Ris. Proposition: If I , is the information number of any loglikelihood counter with inputs based on the last rn outputs

,,

xl, x3 1-,

p ( X , = X r n > xm-2 = X r n - 2 , " ' ,

35

.90

18

14.5

P(Xm = 1 I X m - 2 ,

Time2 0(b,a0pt0s 0)

Bits

5.9

11.5 13

246.9

Mean Long Run

*e..

+

530

IEEE TRANSACTIONS COMMUNICATIONS, ON

of any discrete random process and Z the information number with increments based on theindefinite past, then limm-+m I, =

Z. Proof: Forsimplicity, wegive the proof forrandom variables taking on the values 0 and 1. The proof for random variables taking on finitely many values is exactly the same. Let .-, Y-,, ..., Y o , Y l , ... be the stationary process. (In our case, the Y i s are pairs of hard-quantized received channel symbols.) Let

where y is chosen from the probability space on which the Y i s are defined and j is 0 or 1. It isa standardresult of martingale theory [ 10, p. 2371 that lim f n ( Y , i> = f ( Y , i)

P(A) =

dP=

(z)

P withrespect to Q, which in our application is simply the

= i I Y-1, -.)>(Y) (4(Yo = i I y-

-))(v>

3

a.s.

and so by the dominated convergence theorem

.-

r

The long ARL’s were simulated by amodification of a standardtechnique called “importance sampling” 191, in which the process to be analyzed-in thiscase, theparity check sequence-. is generated using a probability distribution Q different from the distribution P for which results are desired. In our case, P specifies independent 50-50 results for the out-of-synch parity checks-so that the time for the counter to reach a distant threshold y is quite large. A distribution Q was chosen to make the counter reach the threshold more quicklynamely, independent parity checks with probability of failure n*, substantially less than 1/2. The method of importance sampling is based on the simple fact that for any event A , the probability of A under P can be obtained from simulations carried out under the distribution Q. The key is provided by the identity

Here the quantity dP/dQ is the Radon-Nikodym derivative of

n+-

= log

VOL. COM-32, NO. 5 , MAY 1984

likelihood ratio P V l )

... P(X,>

Q(xl>Q ( X n > of the parity checks X 1 , -., X, up to the time that A occurs. Since P(Xi) E 1/2, and Q(Xi) = n* if Xi = 1 (failure), = 1 n* if Xi = 0 (success), relation (1) can be made more explicit. Using EQ to denote expectation under Q, it takes the form

and the same for integration with respect to q . But these are just the information numbers for the counters. Notice: A log-likelihood counter using k panty checks based on parity checks 2 will, after time 2k 14, have exactly the same increments as a log-likelihood counter using k parity checks and based on parity checks X , because the values of and X are exactly the same starting at time 14. Theorem: As k approaches infinity, the information in the log-likelihood counter whose increment at time n is

+

x

where F and S are the numbers of failures and successes, respectively,in the out-of-synch-parity checks up to the time that A occurs, and 1{A} = 1 if A occurs, = 0 otherwise. Thesimulation of thelongARL was based onthe definition of a counter cycle. Starting from a given state s* of k zeros and ones, the cycle ends the first time that the counter resets to zero with the same sequence s * in its memory. Let T denote the time (number of symbol pairs) for a cycle to end and let N denote the number of cycles until the threshold y is crossed. Then using a standard result called Wald’s equation [ 3 , vol. 11, p. 6011, we have

kept separately for n even and odd, approaches all the synch informationinthehard-quantized received channel symbol stream R 1 , R 2 , Proof: As k goes to infinity, the information in a counter long ARL = EN ET C( 1) based on past of length 2k of the hard-quantized received channel stream approaches all the information in the stream. (Actually, the right side gives the expected time until the end The values inC(1)are exactlythe same as would be in a of the first cycle on which y is crossed, but the extra time to counter C(2) based onthe past of length 2k of the(non- end the cycle after crossing is negligible compared to the long stationary) parity checks For n > 2k -k 14, the increments ARL.) The quantity ET was easily simulated directly, since checksare randomthe cycles end fairly in counter C(2) are the same as those in a counter C(3) based whentheparity on the probabilities of (X, *.-,X n - z k ) . But, since quickly (and do not depend on y at all). The evaluation of EN even and odd parity checks are independent, this is exactly the was based on same as the counter of the theorem. 1 EN=APPENDIX B P(A )

-

.e..

xi.

SIMULATION OF ARL‘s

The performance figures of Section 111 were obtained by simulation. The short ARL’s were simulated directly by generating independent symbol errors with probability p , computing the resulting parity check stream, andfeeding it to the counter with threshold y. Direct simulation of the long ARL’s is not feasible, however, because (as the results show) the time required to generate a statistically useful sample of long run lengths would be too great.

where A = {y is crossed} for a given cycle, and P(A) was simulated using themethod of importance sampling, as described above. Importance sampling was used to estimate P(A) in several independent sets of simulations.Notsurprisingly, the estimates were more stable for smaller thresholds. After all the data were gathered, a least squares line was drawn through the points representing small thresholds (see Figs. 1 and 2). These lines were used for the longARL’s in Tables IV and V.

53 1

LORDEN et al.: NODE SYNCHRONIZATION FOR VITERBI DECODE:R

H

16

-

14

-

12

-

[7] G. Lorden, “Procedures for reacting to a change indistribution,” Ann. Math. Statist.,vol. 42, no. 6, pp. 1897-1908, 1971. [SI R. L.Miller, L. J. Deutsch, and S. A. Butman, “Ontheerror statisticsof Viterbi decodingand the performance of concatenated codes,’’ Jet Propulsion Lab., Pasadena, CA, Pub. 81-9. [9] D. Siegmund, “Importance sampling in the Monte Carlo studyof sequential tests,”Ann. Statist., vol. 4, pp. 673-684, 1976. [lo] H. G. Tucker, A Graduate Course in Probability. New York: Academic, 1967.

10

-

*

I

2 W

VI -I

c 0 I-

VI

t

m 0

0

s

4

~

I

I

6

4

’

8

~

’

10

*

12

I

~

’

‘

I

’

I

I

20

18

16

14

NODE-SYNC THRESHOLD

Fig. 1 . Mean time to falsealarm, k = 16; least squares line from thresholds4, 5 , 6 , and 7.

*

H

2 W

v)

-I

c 0

VI

c m 0

0

9

1

44

1

1

6

a

1

8

1

1

10

1

1

12 0

1

1

14. 18

1

1

1

1

1

1

16

NODE-SYNCTHRESHOLD

Fig. 2.

Mean time to falsealarm, k = 8; least squares line from thresholds5 , 10. and 15.

REFERENCES

Robert J. McEliece (M’70-SM181)wasborn in Washington, DC, in 1942. He received theB.S. and Ph.D. degrees in mathematics from the California Institute of Technology,Pasadena,in 1964 and 1967, respectively, and attended,TrinityCollege, University of Cambridge, Cambridge,England, during 1964-1965. From 1963 to 1978 he was employed by the California Institute of Technology’s Jet Propulsion Laboratory, where he was Supervisorof the Information Processing Group. From 1972 to 1982 he was Professor of Mathematics and Research Professor at the Coordinated Science Laboratory, University of Illinois, Urbana-Champaign. Since 1982 he has been Professor of Electrical Engineering at Caltech. Heis also a Consultant with the Jet Propulsion Laboratory and with Cyclotornics, Inc. His research interests include telecommunications, computer memories, and applied mathematics.

*

[l] L. J. Deutsch and R. L. Miller, “The effects of Viterbi decoder node synchronization losseson the telemetry receivingsystem,” Jet Propulsion Lab., Pasadena, CA, TDA Progress Rep. 42-68, Aug. 15, 1981. “Viterbi decoder node synchronization losses in the Reed-Solo[2] -, Laif Swanson was born in California on May 13, mon/Viterbi concatenated channel,” Jet Propulsion Lab., Pasadena, 1950. She received the B.A. degreein mathematics CA, TDA Progress Rep. 42-71, Nov. 15, 1982. from the University of California, Irvine, in 1970, [3] W. Feller, An Introduction to Probability Theory and its Applicaand thePh.D. degree from the Universityof Califortions, vols. I and 11. New York: Wiley, 1968, 1971. nia, Berkeley, in 1975. [4] S. Golomb et at., Digital Communications withSpace ApplicaAfter sixyears with the Departmentof Mathemattions. Englewood,Cliffs,NJ: Prentice-Hall, 1964, ch. 7. ics, Texas A&M University, College Station, she [5] C. A. Greenhall and R. L. Miller, “DeSigd of a quick-look decoderfor the DSN (7, 1/2) convolutional code,” Jet Propulsion Lab., Pasadena, joined the Communications Research Section of the CA, DSN Progress Rep. 42-53, July-Aug. 1979. Jet Propulsion Laboratoiy, Pasadena, CA, in 1981. [6] K. Y. Liuand J. J. Lee, “An experimental study of the concatenated Her interests include coding, information theory, ergodic theory, and probability theory, and she iscurReed-Solomon/Viterbi channel coding system performance and its imrently a Visiting Lecturer in the Departmentof Mathematics, California Instipact on space communications,” Jet Propulsion Lab., Pasadena, CA, Pub. 81-58. tute of Technology, Pasadena.