Is Feedback a Performance Equalizer of Classic and Modern Codes? Tsung-Yi Chen
Nambi Seshadri
Ba-Zhong Shen
Department of Electrical Engineering University of California, Los Angeles Los Angeles, California, 90024, USA Email:
[email protected]
Broadcom Corporation Irvine, California, 92617, USA Email:
[email protected],
Broadcom Corporation Irvine, California, 92617, USA Email:
[email protected],
Abstract—It is well known that in general, feedback cannot increase the capacity of a discrete memoryless channel. However, it can help simplify the complexity of encoding and decoding. Schalkwijk and Kailath (1966) developed a class of block codes for Gaussian channels with ideal feedback. They showed that the probability of decoding error decreases as a second-order exponent in block length for rates below capacity. This paper proposes a coding scheme with ideal feedback and is applied to punctured convolutional codes and punctured Turbo codes. Our results seem to indicate that feedback largely equalizes the performance of classic (convolutional) codes and modern (turbo, LDPC) codes.
I. I NTRODUCTION Shannon showed in 1956 [1] that feedback does not increase the capacity of a discrete memoryless channel. Elias [2] and Chang [3] showed with examples that feedback could greatly simplify error correction at rates below capacity. In 1966, Schalkwijk and Kailath [4] proposed a feedback coding scheme for additive white Gaussian noise (AWGN) channels. They showed that using an ideal feedback link in a fixed-length block-coding scheme for infinite bandwidth AWGN, the probability of error decreases as a second-order exponential in the code constraint length for rates lower than capacity. Schalkwijk then extended the result in the finite bandwidth case [5]. Later, Kramer [6] and Zigangirov [7] showed for the finite and infinite bandwidth case that the above doubly exponential bounds could be replaced by k th-order exponential bounds for any k > 2 in the limit of arbitrarily large block lengths. A recent work done by Gallager [8] derived Schalkwijks result in terms of a result by Elias [2] concerning the minimum meansquare distortion achievable in transmitting a single Gaussian random variable over multiple uses of the same Gaussian channel. A modification of the Schalkwijk-Kailath scheme is shown to have an error probability that decreases with an exponential order which is linearly increasing with block length. Classic codes like convolutional codes are known to have good performance at short block length (N < 1000), but the complexity grows exponentially as the number of memory elements increases. Modern codes like low-density paritycheck(LDPC) and turbo codes are known to have good performance when the block length is long enough (N > 1000). Although the capacity does not increase with feedback, the
error probability could drop rapidly. We conjectured that a classic code can closely approach the performance of modern error correction codes. The main purpose of this work is to demonstrate that with noiseless feedback, a classic codes can have similar performance compared to modern codes while maintaining the same code rate in average. The rest of the paper is organized as follows. Section II provides encoding and decoding protocol proposed by Hagenauer [9] and a modified version of the scheme. Section III gives the method we use to measure feedback coding scheme. Section IV presents the simulation results and finally, concluding remarks are made in section V. II. E NCODING AND D ECODING P ROTOCOL The feedback encoding/decoding protocol is inspired by the feedback coding scheme proposed by Hagenauer [9]. The original scheme and a modified version of it are given in the following subsections. A. Hagenauer’s Incremental Redundancy ARQ scheme As oppose to type-II Hybird ARQ, the principle of the Hagenauer’s Incremental Redundancy (IR) ARQ scheme [9] is not to repeat information or parity bits if the transmission is unsuccessful. The scheme only transmits additional coded bits of a lower rate-compatible code until the code is powerful enough to enable decoding or the lowest available rate is reached. We will simply called this IR scheme in this paper. For a mother code with rate 1/N (N output) and puncturing period P , the rate-compatibility of a code can be achieved by the following restrictions on the puncturing matrix: Let a(l) = (aij (l))N ×P be the puncturing matrix with aij (l) ∈ (0, 1) where 0 implies puncturing. The index l differentiate the punctured rate where l = 1 gives the highest rate and the rate decreases as l increases. The rate-compatibility requires that if aij (l0 ) = 1, then aij (l) = 1 for all l ≥ l0 ≥ 1. For the rest of the paper, we focus on applying feedback scheme to rate-compatible punctured convolutional (RCPC) code and turbo (RCPT) code with terminated encoder/decoder. Suppose we use rate 1/N convolutional code or turbo code as our mother code, the encoding and decoding protocol can be described as follows:
Encoding and transmitting: 1) Add nc parity bits to form an error detection code. For example, a 16 bits CRC code. 2) Encode the output from step 1 and store it at the transmitter. Set the puncturing index l to 0. 3) Transmit the codeword using puncturing matrix a(l) and wait for the feedback from receiver. 4) If ACK is received, then start the procedure from 1 with a new stream of information bits. Otherwise, increase the index l by 1. 5) If lmax is reached, declare an error and start from step 1 with a new stream of information bits. Otherwise, repeat step 3. Decoding and Receiving: 1) Decode the received codeword. 2) Compute the syndrome of the error detection code. 3) If the syndrome is not zero, send NACK to the transmitter and go to step 5. Otherwise, output the information bits. 4) Send ACK to the transmitter. Repeat from step 1. 5) Incorporate the additional coded bits received at the receiver the into the decoding process. 6) Repeat from step 2. Details on computer search of good RCPC codes are described in [9]. The puncturing matrices for RCPT codes [10] are made as symmetric as possible to balance the number of parity bits received for the two constituent decoders. B. Modified Hagenauer’s Incremental Redundancy ARQ Scheme In Hagenauer’s paper, he suggested that if the decoding is still not successful at the lowest code rate, a higher order protocol could take over and ask for a repetition of the whole procedure. However, the repetition procedure would dramatically lower the averaged code rate, especially for time-varying channel. Using the concept of “credit for extra coded bits”, we proposed the following modified scheme, the modified IR scheme, to prevent a significant drop of averaged code rate. Let K be the information length, l be the puncturing index and ri be the punctured code rate using puncturing matrix a(i). Accumulate the credit by 1 1 γ := γ + K × ( − ) r rl
(1)
where r is the desired average code rate. For example, if K = 128, r = 1/2, l = 1 and rl = 4/7 (operating at second highest code rate), then the credit will be accumulated by 32. Encoding and transmitting: 0) Initialize the credit γ to zero. For each concluded transmission, accumulate the credit by (1). 1) Add nc parity bits to form an error detection code. 2) Encode the output that result from step 1 and store it at the transmitter. 3) Set the index l to 0. 4) Transmit the codeword using puncturing matrix a(l) and wait for the feedback from receiver.
5) If ACK is received, then start the procedure from 1 with a new stream of information bits. Otherwise, increase the index l by 1. 6) If l < lmax , start from step 1 with a new stream of information bits. Otherwise, repeat step 3. 7) If l = lmax and the credit γ is larger than the threshold ζ, start from step 3 and decrease γ by zeta. Otherwise, declare an error and start from step 1 with a new stream of information bits. Here we assume that the receiver can recognize whether the received codeword is a repetition of previous codeword, and thus the decoding protocol is the same as in part A. The system can build up credit the when codewords are decodable at a rate higher than, for example, rate 1/2. The system will loose credit when the concluded transmission rate is lower than 1/2. If the channel is too noisy such that even the lowest available rate is decoded incorrectly, the system examine whether there is sufficient credit to retransmit the codeword starting from the highest rate. We are aware that for time-invariant channel like AWGN channel, the credit γ will keep accumulating with a high enough signal-to-noise ratio (SNR). However, the modified scheme will be more effective on maintaining the averaged code rate when we are dealing with time-varying channel. C. Decoding Algorithms: BCJR and List Viterbi Algorithms For RCPC codes, we use L-list Viterbi algorithm [11]. The decoder output the L most probable codewords for every received codeword. Then starting from the most likely codeword, the receiver compute their syndrome with the error detection code and select the first listed codeword that has a zero syndrome. If none of them have a zero syndrome, an error is declared by the list Viterbi decoder and a NACK is sent from the receiver. For RCPT codes, we use traditional BCJR algorithms in log domain (also known as log-MAP algorithm). Note that we assume an ideal feedback link and neglect the probability of undetected error since it is much smaller than the BER level we are interested in. III. M EASURE OF P ERFORMANCE The standard measurement for ARQ schemes is usually throughput. However, we propose different ways to get a better look at the behaviour of (modified) IR scheme. This section introduces practical methods and metrics to measure the performance. To apply the feedback protocol described in section II to RCPC codes and RCPT codes, we need to select the rates for transmission and calculate the the averaged code rate. We also need to account for the rate loss due to CRC bits and termination bits. A. Rate Selection and Puncturing The rates for each transmission should be carefully selected depending on the channel condition, so that at the desired BER level, the averaged transmission rate is close to designated code rate. As the SNR goes higher, the averaged rate rise
since more and more codewords become decodable at higher code rates. For easier demonstration, both RCPC and RCPT codes use the same set of rate for transmission. We use successive punctured rates with a puncturing period of 8: rate 4/6, 4/7, 4/8, 4/9 and 4/10. We seek for the SNR such that the averaged code rate is close to 1/2, which we called the “operating SNR”. We then compare the BER performance with codes without feedback at the SNR. B. Averaged Code Rate Let m be the number of different rates available for transmission, ni be the number of decodable codeword with the i m ∑ th code rate, and n = ni be the total number of codeword i=1
transmitted. The averaged code rate can be calculated by dividing total number of information bits by total number of coded bits. Since the number of information bits is fixed, we can translate the calculation of the averaged rate by ′ Ravg =
1 n
×
1 m ∑ i=1
(2) ni ri
To account for the rate loss in CRC and termination bits, the averaged rate is further multiplied to additional fraction depending on the number of bits used for CRC check (nc ) and number of memory elements (M ) used for RCPC and RCPT codes. Suppose the block length including termination bits and CRC bits is N , the effective averaged code rate Ravg is calculated by nc + M ) (3) N Note that the feedback coding scheme can also be viewed as a variable length coding since the information bits are fixed for each codeword. In this point of view, the scheme is actually allowing a delay to lengthen the code length under bad channel condition. On the other had, the scheme also reduces the delay by shortening the code length. ′ × (1 − Ravg = Ravg
C. Rate Spectrum We collect the number of codewords that are decodable for each available code rate, and plotted as a “rate spectrum” in the simulation section. The number of codewords for each rate are normalized so that the Y-axis of the rate spectrum represents the percentage of the concluded transmissions with rate ri out of total concluded transmissions. D. Instantaneous Operational Capacity Let ηi be the SNR computed by the instantiated Gaussian noise in the i th code block. Define the metric “instantaneous operational capacity” of the i th coded block for AWGN channel as 1 κli = log2 (1 + ηil ) (4) 2 Let σl2 be the variance computed by Gaussian noise within a coded block with length l and assume BPSK modulation,
ηil = 2ri /σl2 , where ri is the code rate of the coded block. The term 1/σl2 can be interpreted as instantaneous Eb /N0 in a coded block. This metric coincide with the discrete memoryless channel capacity when the block length goes to infinity, i.e., when the ergodicity of the Gaussian noise variance holds. IV. S IMULATIONS This section provides the simulation plots and the parameters for each simulation. For RCPC codes, list Viterbi algorithm [11] with three lists is used for decoding. For RCPT codes, we use the trellis and quadratic permutation polynomial defined in the 3GPP LTE turbo code standard. The rates for non-feedback codes are all rate 1/2. Length 1944, Rate 1/2 LDPC code in IEEE 802.11n standard is also simulated as a baseline comparison for modern error correction code. A comparison of rate 1/2 RCPC codes with and without feedback is shown in Fig. 1. The mother code rate is 1/4, and higher rates are obtained with puncturing table described in [9]. The information bit length (including CRC bits and termination bits) for 512 and 64 states RCPC codes are 512 and 128 bits, respectively. Rate loss of 12 bits and 16 bits CRC are considered for 64 states and 512 states RCPC codes, respectively. The two square points are the operating SNR and their BER performance for 64 states and 512 states RCPC codes with IR scheme. The threshold ζ for modified IR scheme is chosen to be equal to the length of information bits for RCPC codes. The two circled points in Fig.1 represent the operating SNR’s for modified IR scheme and no packet error was observed up to more than 107 simulated packets. At the BER level of 10−6 , the gap between 64 states code and 512 states code without feedback is about 2 dB, while the gap between codes using modified IR scheme is only about 0.1 dB. The repetition process reduces the rates as expected. However, the rate loss is merely in the order of 10−3 , while the BER drop significantly. IR scheme with interleaver size of 128 bits and 512 bits RCPT codes are simulated and is shown in Fig.2. The mother code rate is 1/3, and higher rates are obtained with ratecompatible puncturing table. Rate loss of 12 bits and 16 bits CRC are considered for interleaver size 128 and 512 bits RCPT codes, respectively. Since turbo codes have better performance at longer block length, we can see that the operating SNR for RCPT codes go lower and the BER also drops compared to RCPC codes. By comparing Fig. 1 and Fig. 2, we can see that for similar block lengths, 128 and 512 bits, the gap between RCPC and RCPT codes is substantially bridged. Another interesting observation is that the interleaver size 512 bits RCPT code with feedback is better than the turbo code with interleaver size 2112. Fig. 3 to Fig. 5 show the rate spectra of feedback RCPT codes. The rate spectrum is interpreted as follow: the X-axis is the rate used for transmission. In our simulations, the available
0
10
−1.2 dB, R
−3
−4
percentage
10
−5
10
−6
10
10
−2
−1.5
avg
= 0.4769
−1.4 dB, R
avg
= 0.4706
percentage
1
0.5
0
0.5
0
0.4 0.5 0.6 rates −1.5 dB, Ravg = 0.4645
0.4 0.5 0.6 rates −1.6 dB, Ravg = 0.4582
0.4 0.5 0.6 rates −1.7 dB, Ravg = 0.4520
1
1
1
0.5
0.5
0.5
Rate 0.5008
Rate 0.5043
Rate 0.5041
percentage
0.5
0
−7
−1.3 dB, R 1
0
BER
10
= 0.4832
percentage
−2
10
percentage
10
avg
1
percentage
512 States CC 64 States CC 512 States ListCC 64 States ListCC 512 States IR Scheme 64 States IR Scheme 512 States Modified IR Scheme 64 States Modified IR Scheme 802.11n LDPC code
−1
Rate 0.5007 −1 −0.5
0 Es/N0
0.5
1
1.5
0
0.4 0.5 0.6 rates
0
0.4 0.5 0.6 rates
0.4 0.5 0.6 rates
2
Fig. 3. The rate spectrum for interleaver size 128 bits feedback RCPT codes. Rate loss from 12 bits CRC code and 3 termination bits are considered.
Fig. 1. Simulation of feedback and non-feedback RCPC codes. Rate loss from CRC bits and termination bits are included.
−2
10
−3
−1.3 dB, Ravg = 0.5171
−1.4 dB, Ravg = 0.5094
1
1
1
0.5
0
10 BER
−1.2 dB, Ravg = 0.5247
−4
10
percentage
−1
10
percentage
Turbo interleaver size 128 Turbo interleaver size 512 Turbo interleaver size 2112 Turbo interleaver size 6144 IR scheme interleaver size 128 IR scheme interleaver size 512 802.11n LDPC
percentage
0
10
0.5
0
0.5
0
0.4 0.5 0.6 rates −1.5 dB, Ravg = 0.5019
0.4 0.5 0.6 rates −1.6 dB, Ravg = 0.4949
0.4 0.5 0.6 rates −1.7 dB, Ravg = 0.4885
1
1
1
0.5
0.5
0.5
Rate 0.5018
−7
10
0
−8
10
percentage
percentage
Rate 0.4985
−6
10
percentage
−5
10
−2
−1.5
−1
−0.5
0
0
0.4 0.5 0.6 rates
0
0.4 0.5 0.6 rates
0.4 0.5 0.6 rates
0.5
Es/N0
Fig. 4. The rate spectrum for interleaver size 512 bits feedback RCPT codes. Rate loss from 16 bits CRC code and 3 termination bits are considered. Fig. 2. Simulation of feedback and non-feedback RCPT codes. Rate loss from CRC bits and termination bits are included.
−1.2dB, R
avg
= 0.5630
−1.3dB, R
avg
= 0.5484
0.5
0
−1.4dB, R
avg
= 0.5236
1 percentage
1 percentage
0.5
0
0.5
0
0.4 0.5 0.6 rates −1.6dB, Ravg = 0.4982
0.4 0.5 0.6 rates −1.7dB, Ravg = 0.4972
1
1
1
percentage
0.4 0.5 0.6 rates −1.5dB, Ravg = 0.5051
0.5
0
0.4 0.5 0.6 rates
percentage
percentage
1
percentage
rates are, from right to left, 2/3, 4/7, 1/2, 4/9 and 2/5. The Y-axis represents the percentage in the total transmitted codewords. For example, the upper left plot in Fig. 2 tells us that at −1.2dB of SNR, about 30 percent of the codewords are decoded correctly at rate 2/3, and about 45 percent of the codewords are decoded correctly at rate 4/7. The rate spectra show that for the codes with longer block length, the rates used are more concentrated to the averaged code rate. It is interesting to see that when the information block length is 128 for RCPT code, a significant amount of coded blocks are decoded successfully with code rates higher than 1/2. On the other hand, the RCPT code with interleaver size 6144 has a rate spectrum that is tightly concentrated around the averaged code rate. Fig. 6 and Fig. 7 are the instantaneous operational capacity κli compared to the Shannon capacity with rate 1/2 code and Eb /N0 = 0dB. The block length is 256 and 10000,
0.5
0
0.4 0.5 0.6 rates
0.5
0
0.4 0.5 0.6 rates
Fig. 5. The rate spectrum for interleaver size 6144 bits feedback RCPT codes. Rate loss from 32 bits CRC code and 3 termination bits are considered.
l = 256, E /N = 0 dB b
0
0.7
IO Capacity
Shannon capacity IO Capacity 0.6 0.5
length RCPC codes and RCPT codes have been substantially bridged with feedback. However, the same cannot be said for codes with long block length . ACKNOWLEDGMENT
0.4 20
40
60
80
100 i
120
140
160
180
200
The authors would like to thank Professor Richard D. Wesel for his comments and suggestions on this work. R EFERENCES
Fig. 6.
Instantaneous operational capacity with l = 256 l = 10000, E /N = 0 dB b
0
IO Capacity
0.7
Shannon capacity IO Capacity
0.6 0.5 0.4 20
Fig. 7.
40
60
80
100 i
120
140
160
180
200
Instantaneous operational capacity with l = 10000
respectively. There is significant amount of blocks where the operational capacity is higher than the Shannon capacity, and the fluctuation is up to around 0.6 when the block length is small. This statistical phenomenon explains why more codewords are decodable at a higher code rate for codes with shorter block length. V. C ONCLUSION A modified incremental redundancy scheme is proposed to demonstrate how feedback can help improve reliable communication while maintaining the same code rate in average. Various simulation results are shown by applying both IR scheme and modified IR scheme to RCPC and RCPT codes. The BER performance with 1/2 averaged code rates are shown on the plots. These BER performance are then compared to modern error correction codes. We observed in these simulation results that feedback not only improve the BER, but also reduce the performance gap between classic error correction codes and modern error correction codes. The rate spectra show interesting results that for codes with shorter block length, more codewords are decodable at code rate higher than averaged code rate. This phenomenon is then explained by the notion of instantaneous operational capacity κli . The fact that κli fluctuate heavily when the code length is shorter explains why more codewords are decodable at higher code rate. We showed that a simple ARQ scheme with noiseless feedback comes close to equalizing classic and modern codes. More precisely, it comes close to equalizing the performance between codes with long block length and short block length. However, we haven’t shown the case where there is noise on the feedback link. Another future research question would be whether we can build a long classic code (with reasonable decoding complexity) and come close to the capacity with feedback. In other words, the performance gap between short
[1] C. E. Shannon, “The zero error capacity of a noisy channel,” IEEE Trans. Inf. Theory, vol. PGIT-2, pp. 8–19, Sep. 1956. [2] P. Elias, “Channel Capacity Without Coding,” in MIT Res. Lab of Electronics, Cambridge, MA, USA, Sep 1956. [3] S. Chang, “Theory of information feedback systems,” IEEE Trans. Inf. Theory, vol. PGIT-2, pp. 29–40, Sep. 1956. [4] J. Schalkwijk and T. Kailath, “A coding scheme for additive noise channel with feedback–:I No bandwidth constraint,” IEEE Trans. Inf. Theory, vol. IT-12, no.2, pp. 172–182, Apr. 1966. [5] J. Schalkwijk, “A coding scheme for additive noise channel with feedback–:I No bandwidth constraint,” IEEE Trans. Inf. Theory, vol. IT-12, no.2, pp. 183–189, Apr. 1966. [6] A. Kramer, “Improving communication reliability by use of an intermittent feedback channel,” IEEE Trans. Inf. Theory, vol. IT-15, no.1, pp. 52–60, Jan. 1969. [7] K. S. Zigangirov, “Upper bounds for the error probability for channels with feedback,” Probl. Pered. Inform., vol. 6, no.1, pp. 87–92. [8] R. G. Gallager, “Variations on a theme by Schalkwijk and Kailath,” IEEE Trans. Inf. Theory, vol. 56, no.1, pp. 6–17, Jan. 2010. [9] J. Hagenauer, “Rate-compatible punctured convolutional codes (RCPC codes) and their applications,” IEEE Trans. Commun., vol. 36, no.4, pp. 389–400, Apr. 1988. [10] D. N. Rowitch and L. B. Milstein, “On the Performance of Hybrid FEC/ARQ Systems Using Rate Compatible Punctured Turbo (RCPT) Codes,” IEEE Trans. Commun., vol. 48, no.6, pp. 948–959, Jun. 2000. [11] N. Seshadri and C.-E. Sundberg, “List Viterbi decoding algorithms with applications,” IEEE Trans. Commun., vol. 42, pp. 313–323, Apr. 1994.
