IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 7, JULY 2009
1881
Transactions Letters Iterative Decoding Using Optimum Soft Input – Hard Output Module Michael Kerner and Ofer Amrani, Member, IEEE
Abstract—Maximum likelihood sequence estimation (MLSE) in the shape of the Viterbi algorithm, although a strictly hardoutput approach, is utilized for iterative decoding of turbo codes. It is a practical alternative when decoding complexity and speed may be traded for performance.
La1k
LX k
DeInterleaver
Interleaver
Index Terms—Turbo code, iterative decoding. SISO 1
LY 1
Le1k
Interleaver
Lak2
k
I. I NTRODUCTION
F
OLLOWING the discovery of turbo codes in 1993 [2], numerous papers have been published on the topic. Corresponding to the standard encoder for parallel concatenated convolutional codes (PCCC), a turbo decoder is shown in Figure 1. It consists of two Soft-Input Soft-Output (SISO) decoders linked by an interleaver/deinterlever. The best known SISO algorithm, in terms of performance, is based on the so-called BCJR algorithm [1]. We shall henceforth refer to the logdomain version of this decoder as LogMap (log-domain Maximum A-posteriori Probability) decoder. Other well known SISO algorithms include: MaxLogMap which is a practical approximation of LogMap, and the Soft Output Viterbi Algorithm (SOVA) [4]. The SOVA and the MaxLogMap algorithms have been shown to be equivalent [3]. In the past few years several modifications for the LogMap [5], MaxLogMap [9] and SOVA [6], [10] (and the references therein), have been introduced. In this work we consider the following setting: parallel concatenated convolutional encoders linked by an interleaver, memoryless additive white Gaussian noise (AWGN) channel, BPSK modulation {0 → -1, 1 → +1} and an iterative decoder. The paper is organized as follows. In Section II we show how a soft-input hard-output (SIHO) module can be invoked for decoding PCCC; each of the SISO modules shown in Figure 1 will be replaced by a new module. The great merit of this approach is its low computational complexity. In Section III the performance of the SIHO approach is evaluated via simulations; the performance degradation of this low complexity approach is 1dB and 0.6dB relative to LogMap and MaxLogMap algorithms, respectively, for BER of 10−4 . Finally, Section IV concludes the paper.
Paper approved by L. K. Rasmussen, the Editor for Iterative Detection Decoding and ARQ of the IEEE Communications Society. Manuscript received December 15, 2007; revised January 15, 2009. O. Amrani is with the School of Electrical Engineering, Tel Aviv University, Tel Aviv, Israel (e-mail:
[email protected]). M. Kerner is with Intel Corp., Petach Tiqua, Israel. This work was supported in part by the REMON-4G Consortium, OCS Ministry of Trade and Industry, Israel. Digital Object Identifier 10.1109/TCOMM.2009.07.070167
LY 2
SISO 2
Lek2
k
Fig. 1.
Conventional Turbo Decoding Scheme.
II. A SIHO-BASED I TERATIVE D ECODING A LGORITHMS A low-complexity, sub-optimal, SISO decoder that is based on a SIHO module is derived. This decoder will then be employed as the basis for iterative decoding of PCCC turbo codes. A. Notations Let us denote the received sequence by {Rk }, Rk = {Xk , Yk1 , Yk2 } being a noisy version of the transmitted symbol Ck = {xk , yk1 , yk2 }, where k is the information-bit index. The code sequence {Ck } is transmitted over a binary input additive white Gaussian noise (AWGN) channel. At the output of the channel, the transmitted sequence is thus perturbed by additive noise whose elements are zero-mean Gaussian random variables with variance σ 2 . From the received sequence {Rk }, loglikelihood metric LRk = {LXk , LYk1 , LYk2 } is extracted and stored for future use by the iterative process. In the description of our SISO module we shall be using LSk (systematic bit) to denote LXk , and LPk (parity) to denote LYk1 or LYk2 . A priori and extrinsic information will be denoted by Lak and Lek , respectively. N denotes the length of the information sequence in bits or the length of the interleaver. B. Motivation It is known that the bit error rate (BER) obtained when decoding a convolutional code with the LogMap decoder is only slightly better than that of the Viterbi algorithm (VA), see e.g. [7]. While LogMap is the best known algorithm performance-wise, complexity-wise, LogMap is roughly 4 times more demanding than a conventional Viterbi decoder [8]. This raises the following intriguing question: can one devise a low-complexity module for effectively converting hard-outputs to soft-values; if so, it would seem quite attractive to employ
c 2009 IEEE 0090-6778/09$25.00
1882
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 7, JULY 2009
Lak LSk Trellis Decoder (VA)
LPk
SIHO
Xˆ k
Soft Output Calculator
Lek
Ld k
SISO
Fig. 2.
SIHO-based SISO implementation.
the Viterbi (SIHO) decoder as the basis for a SISO decoder. We shall henceforth refer to this module as a soft-output calculator. This approach is significantly different from SOVA whereby a bit-wise metric is calculated based on the trellis diagram of the code. The proposed SISO structure is presented in Figure 2; indeed, the SIHO module is implemented by means of the VA. Notably, all SISO algorithms currently known are based on calculating soft-output values using some sort of metric differences (ratios). The approach proposed in this work is different. C. Soft Output Calculator For brevity of exposition we shall derive the proposed algorithm in two steps. First, it is assumed that a certain probability, i.e., the bit error probability for each frame, is perfectly known as side information. In the second step a different set of equations is derived thus avoiding the need for side information. Furthermore, the performance of the algorithm detailed first may be used to lower bound any practical approach of the type described as the second step. Experimental results show that the practical approach of Step 2 (Section II-C2) achieves near optimum performance in that sense. 1) Soft-output calculation using side information: The output of the trellis decoder of Figure 2 is a single surviving path. The information bits corresponding to this path are referred to as the decoder’s hard output. Having the exact bit-wise error probability, one could easily obtain the accurate metric required for optimum MAP decoding. Since the proposed approach concerns a hard-output decoder, no such information is available, and hence we must resort to ˆ k denote the estimated (hardpossible approximations. Let X output) of the decoder on the k’th information bit. Also assume that Pe , the bit error probability for each frame, is known. The ˆ k can be converted into a soft value, denoted hard-output X Ldk , using the following transformation:
Ldk = log
ˆk ) ˆ k /xk = 1) P (xk = 1/X P (X = log ˆk ) ˆ k /xk = 0) P (xk = 0/X P (X e ˆk = 1 log 1−P if X Pe = Pe ˆk = 0 if X log 1−P e ˆk − 1)log 1 − Pe . = (2X Pe
(1)
Since the frame size N is finite, the bit error probability per frame, Pe , takes one of the following discrete values: (0, N1 , N2 , ..., 1) . Consequently, it immediately follows from
Equation (1) that Ldk also takes discrete values, for example ˆ k = 1, Ld = log(N − 1) for Pe = 1/N , and given that X k Ldk = log(1 − N ) for Pe = (N − 1)/N . There are two exceptions to Equation (1) that are quite easily resolved as ˆk − follows: for the singular case Pe = 0 we define Ldk = (2X ˆ k −1)log(N ) is 1)log(N ), and similarly the value Ldk = −(2X used for, the very unlikely case, Pe = 1. Thus far the proposed soft-output metric is given by
Ldk
⎧ ˆ k − 1)log(N ) if ⎪ ⎨ (2X ˆ k − 1)log(N ) if = −(2X ⎪ ⎩ (2X ˆ k − 1)log 1−Pe Pe
Pe = 0 Pe = 1 otherwise
(2)
Evidently, using this approach, practically all information bits are assigned with the same (absolute) soft metric. One can hardly expect this to be effective for obtaining good decoding results when used in the framework of iterative decoding. We therefore introduce additional information that is readily available from the VA: the sign-difference between the (soft) input and (hard) output values of the trellis decoder. This is described below. Let us define a flipped bit as a bit for ˆ k ; where HD(x) = 1 if x ≥ 0, which HD(Lak + LSk ) = X and HD(x) = 0 otherwise. Next, the events associated with the probability Pe are divided into two disjoint cases. The probability of a flipped bit being in error is denoted as Pe f , and the probability of a non-flipped bit being in error is denoted Pe nf . It is reasonable to assume that Pe f > Pe nf
(3)
which, in turn, means that a bit that undergoes flipping while being decoded is less reliable than a non-flipped bit. The assumption (3) has been validated by means of simulations. Now assuming that the probabilities Pe f and Pe nf are known, the soft-output metric (2) can be modified to account for this additional information as given by: Ldk = nf e ˆ k − 1)log 1−Pnf (2X Pe ˆ k − 1)log 1−Pfe f (2X Pe
if
ˆk HD(Lak+LSk )=X
(4)
otherwise
The values at the extremes, associated with probabilities Pe nf ∈ {0, 1}, Pe f ∈ {0, 1} are treated in a similar manner to (2). The coarse approximation for the bit-wise error probability given by (2) is refined by (4) from a merely binary soft-output to a 4-level soft-output. This is guaranteed by (3). 2) Soft-output calculation with no side information: The probabilities Pe nf , Pe f can not be easily estimated in practice. Nevertheless, we introduce a much simpler approach that is shown to perform extremely close to the case where these probabilities are perfectly known. According to this approach, instead of estimating the probabilities Pe nf , Pe f , we employ a different, yet related probability Pf lip . Pf lip is the, per-frame, a-posteriori probability for a bit to undergo flipping. Pf lip is straightforward to obtain based on the measured number of flipped bits in a decoded frame: Pf lip =
N 1 J(k), N k=1
(5)
KERNER and AMRANI: ITERATIVE DECODING USING OPTIMUM SOFT INPUT – HARD OUTPUT MODULE
with J(k) being an indicator function: ˆk , J(k) = 1, if HD(Lak + LSk ) = X otherwise J(k) = 0. Based on (5), Ldk can be calculated as Ldk = ˆ k − 1)log 1−Pf lip c(2X Pf lip ˆ k − 1)log 1−Pfflip e(2X Pe
if
ˆk HD(Lak+LSk )=X
(6)
otherwise
Note that the same expression log{(1 − Pf lip )/Pf lip }, is employed for both cases in (6). This clearly simplifies the calculations. Nevertheless, in order to compensate for the loss of accuracy due to employing this approximation, two accompanying coefficients are introduced: c - relevant for nonflipped bits; e - relevant for flipped-bits. As they appear in (6), the coefficients c and e are to be understood as constant per frame. (Note that these coefficients may be optimized per iteration and signal to noise ratio (SNR).) This motivates a search for yet a simpler alternative to (6). Let c˜(i,SN R) , e˜(i,SN R) be defined as follows
nf e c˜(i,SN R) = E log 1−P ; nf P e
(7) f e e˜(i,SN R) = E log 1−P , Pe f where the average is taken for each {iteration, SN R} pair. As an alternative to (6), we replace the logarithmic elements in (4) with their average values, calculated in (7), to obtain the following simple expression for the soft output: L = dk ˆ k −1) if c˜(i,SN R) (2X ˆ k −1) e˜(i,SN R) (2X
ˆk HD(Lak+ LSk ) = X . otherwise
(8)
For simplifying things even further, one may hope to eliminate the dependency on SNR altogether. Surprisingly, for the codes we tested, a single SNR value exists, SNR’, whose corresponding coefficients, c˜(i) = c˜(i,SN R ) , e˜(i) = e˜(i,SN R ) can provide good performance for all SNR values. Note, that obtaining SNR’ is a rather exhaustive procedure, but fortunately it is performed only once, off-line. In that case, one can substitute c˜(i), e˜(i) for c˜(i,SN R) , e˜(i,SN R) , in Equation (8). Finally, we explicitly describe an iterative algorithm, denoted by A, based on the SISO approach derived above. 3) Algorithm A summary: 1) Calculate the log-likelihood values {LXk }, {LYk1 }, {LYk2 }. Set {LSk } = {LXk }, {Lp1k } = {LYk1 } and {Lp2k } = {LYk2 }. Set the (half) iterations counter to i = 1, and initialize all a-priori values as follows {La1k } = 0, 1 ≤ k ≤ N . 2) Decode the first convolutional code RSC1 by means of a trellis decoder T D1 . The branch metrics are to be computed using {LSk + La1k } and {Lp1k }. The output of T D1 in half iteration i is the hard-output sequence ˆ 1 } corresponding to the information bits along the {X k surviving path. (RSC - stands for Recursive Systematic Convolutional)
1883
3) Calculate the soft-output values {Ld1k } using the soft output module by employing (6) and (5), alternatively ˆ 1 }. (8), using {LSk + La1k } and {X k 4) Calculate a priori information for decoder 2: {La2k } = Interleaver{Ld1k − (LSk + La1k )}, and set the iteration counter to: i = i + 1 . 5) Decode the second convolutional code RSC2 by means of employing a trellis decoder T D2 . The branch metrics are to be computed using {Interleaver{LSk } + La2k } and {Lp2k }. The output of T D2 in half iteration i is ˆ 2 } corresponding to the the hard-output sequence {X k information bits along the surviving path. 6) If a predetermined condition for terminating the iterative process is met, go to Step 9. 7) Calculate the soft-output values {Ld2k } by employing (6) and (5), alternatively (8), using {Interleaver{LSk } + ˆ 2 }. La2k } and {X k 8) Calculate a priori information for decoder 1: {La1k } = DeInterleaver{Ld2k − (Interleaver{LSk } + La2k )}; set i = i + 1; and return to Step 2. ˆ 2 } as the 9) Terminate the decoding process taking {X k output sequence. III. P ERFORMANCE C OMPARISON For evaluating the performance of the proposed approach as compared to other known algorithms, we employ the turbo code standardized by the Third-Generation Partnership Project [11]. This code is composed of two parallel concatenated RSC codes of constraint length 4, the feedforward generator is 15, the feedback generator is 13, (in octal representation). In order to study the effect of frame length on the performance of the proposed approach, two different, relatively short frame lengths were simulated; N = 796 and N = 3196. The relevant BER results are shown in Figures 4 and 3. Four curves are depicted for (simulated) LogMap, MaxLogMap, algorithm A*, and finally A. A few results for modified SOVA [10] are also depicted for reference. A* denotes an algorithm having perfect knowledge of the bit error probability for flipped and non-flipped bits as described in Section II-C1 The performance of this approach serves as a lower bound on the actual achievable BER performance. In fact, we conjecture that among all possible soft-output calculators that employ the MLSE (hard) output (possibly in the shape of the VA), A* achieves the best performance. This calculator may be regarded as a first-order approximation for the exact bit-wise error probability in the sense that the metric Ldk is determined using only available information without involving, say, the decoding history of a bit. Indeed, it is evident from our experimental results that the practical approach proposed in Section II-C2, and utilized by algorithm A, provides practically the same results as the approach of Section II-C1. The curve corresponding to algorithm A was obtained using c = 1.4 and e = 0.4, which were optimized empirically. The performance gap to A* could be further tightened by optimizing the values of the coefficients c and e for different channel SNR’s. Note that one possible selection for these values is simply to follow the average values corresponding to
1884
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 7, JULY 2009
BER for N=3196
0
10
LogMap MaxLogMap A A* [10]−improved SOVA (based on report)
−1
10
−2
BER
10
−3
10
−4
10
−5
10
−6
10
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
SNR [db]
Fig. 3.
BER-iterative decoding, N=3196. BER for N=796
0
10
LogMap MaxLogMap A A* [10]−improved SOVA (based on report)
−1
10
−2
BER
10
−3
10
−4
10
−5
10
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
SNR [db]
Fig. 4.
BER-iterative decoding, N=796.
the two cases of (6) as observed at the output of the optimum LogMap decoder, i.e., try to "mimic" the behavior of the LogMap decoder. The simulation results reveal that for BER of 10−4 and frame length of 3196, the proposed decoding approach is about 1.0dB, and 0.6dB, worse than optimal LogMap, and the (sub-optimal) MaxLogMap, respectively. For the short frame length, algorithm A further reduces the gap by roughly 0.1dB. We note that using the soft metric given by (8), for N = 3196, simplifies the decoder for the price of 0.1dB in coding gain as compared to algorithm A (this is not shown in the figures).
IV. C ONCLUDING R EMARKS A low complexity soft-input soft-output algorithm is implemented using maximum likelihood sequence estimation in the shape of the Viterbi algorithm followed by an extremely simple soft-output calculator. This module is successfully employed for iterative decoding of a turbo PCCC code. A few notes regarding implementation of algorithm A. During decoding, the number of flipped bits is calculated using (5). The LLR values are then computed according to Equation (6). These are very simple operations and it is expected that the complexity involved in computing Equation (6) can be
KERNER and AMRANI: ITERATIVE DECODING USING OPTIMUM SOFT INPUT – HARD OUTPUT MODULE
further reduced by employing a predetermined lookup table. Thus, we can safely argue that the complexity of the proposed algorithm (per half iteration) is that of running the Viterbi algorithm. Also note that the number of flipped bits can serve as an effective stopping criterion, e.g., for terminating the decoding process soon after Pf lip = 0. This approach trades performance for computational complexity and can hence be attractive for low cost applications. The performance of the code studied herein, combined with the proposed SIHObased decoding approach, is favorable compared to known convolutional codes of similar decoding complexity. ACKNOWLEDGMENT The authors would like to thank the anonymous reviewers for helpful comments and suggestions that helped improve the manuscript. R EFERENCES [1] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error rates," IEEE Trans. Inform. Theory, vol. IT-20, pp. 284-287, Mar. 1974. [2] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit errorcorrecting coding and decoding: turbo codes," in Proc. 1993 IEEE Int. Conf. Commun., vol. 2, pp. 23-26, May 1993.
1885
[3] M. P. C. Fossorier, F. Burkert, S. Lin, and J. Hagenauer, “On the equivalence between SOVA and max-log-map decodings," IEEE Commun. Lett., vol. 2, no. 5, pp. 137-139, May 1998. [4] J. Hagenauer and P. Hoeher, “A Viterbi algorithm with soft outputs and its applications," in Proc. 1989 IEEE Global Commun. Conf., pp. 47.1.147.1.7, Dallas, TX, Nov., 1989. [5] L. Kocarev, Z. Tacev, and A. Vardy, “Improving turbo codes by control of transient chaos in turbo-decoding algorithms," IEEE Electron. Lett., vol. 38, no. 20, Sept. 2002. [6] L. Papke, P. Robertson, and E. Villerbun, “Improved decoding with the SOVA in a parallel concatenated (turbo-code) scheme," in Proc. IEEE Int. Conf. Commun., pp. 102-106, July 1996. [7] S. S. Pietrobon and A. S. Barbulescu, “A simplification of the modified Bahl decoding algorithm for systematic convolutional codes," in Proc. Inform. Theory Applic., Australia, pp. 1073-1077, Nov. 1994. [8] A. J. Viterbi, “An intuitive justification and a simplified implementation of the MAP decoder for convolutional codes," IEEE J. Select. Areas Commun., vol. 16, no. 2, pp. 260-264, Feb. 1998. [9] J. Vogt and A. Finger, “Improving the MAX-LOG-MAP turbo decoder," Electron. Lett., vol. 36, pp. 1937-1939, Nov. 2000. [10] Z. Wang and K. K. Parhi, “High performance, high throughput turbo/sova decoder design," IEEE Trans. Commun., vol. 51, no. 4, pp. 570-579, Apr. 2003. [11] European Telecommunications Standards Institute, Universal mobile telecommunications system (UMTS): Multiplexing and channel coding (FDD), 3GPP TS 125.212 version 3.4.0, pp.1 4-20, Sept., 2000.
