2012 International Symposium on Communications and Information Technologies (ISCIT)
A Joint Source and Channel Coding Scheme Base on Simple Protograph Structured Codes Jiguang He1 , Lin Wang1 , Senior Member, IEEE, Pingping Chen2 1. Dept. of Communication Engineering, Xiamen University, Fujian 361005, China 2. Dept. of Electronic Engineering, Xiamen University, Fujian 361005, China Email:
[email protected],
[email protected],
[email protected] of the joint design; 2) how to improve the turbo cliff region as well as lower the error floor region. At present, one sub-type of the LDPC codes, called protograph LDPC codes derived from protograph using copy-andpermute, are described in [6]. The accumulate repeat accumulate (ARA) code has a simple, fast encoder structure and thus allows for high-speed iterative decoding implementation using belief propagation [6], [7]. Employing density evolution, some rate-0.5 ARA codes provide capacity approaching iterative decoding threshold (as low as 0.08dB from channel capacity threshold). Hence, in this paper, the ARA codes are exploited in the JSCC scheme as channel codes to form an up-to-date JSCC scheme. What’s more, the source code also employs a simple protograph structured LDPC code without punctured nodes, which simplifies the encoding and decoding implementation. The differences between our proposed PJSCC scheme with that given in [1], [8] lie in that both the source and channel code of the P-JSCC scheme are composed of protograph codes while the traditional scheme are made up of two regular or irregular LDPC codes. To the best of our knowledge, no JSCC schemes with double protograph LDPC codes have been proposed up to now. Furthermore, it can be observed from the simulation results that the proposed JSCC scheme show a considerable performance gain as compared the JSCC scheme with regular LDPC codes. The rest of the paper is organized as follows. The system description is presented in section II. In section III, the encoding and decoding procedure are described. The simulation results are shown in section IV and section V concludes the paper.
Abstract—A scheme of joint source and channel coding (JSCC) with double Low Density Parity Check (LDPC) structure has been proposed recently, which results in a popular and promising research topic for further improving JSCC design at physical layer. However, it becomes an important problem how to enhance the error performance and reduce the complexity of the whole system. Therefore, we propose a state-of-the-art JSCC scheme, denoted as P-JSCC, based on simple protograph structured codes considering the performance and hardware advantages of such LDPC-like codes. The simulation results with different lengths of source sequence show that the Bit Error Ratio (BER) performance of the proposed scheme is better than that of the previous JSCC scheme base on regular LDPC codes, denoted as traditional [1], in the turbo cliff region. Furthermore, it is found that the performance of the proposed JSCC scheme becomes worse as p1 , the probability of binary bit with value one in source sequence, increases. Eventually, we enlarge the degree of check nodes in source code while keep the channel code invariable, and the simulation results illustrate that a significant improvement is achieved on error floor region. Index Terms—Joint Source and Channel Coding (JSCC), Protograph, BER.
I. I NTRODUCTION Recently, some joint source and channel coding schemes were investigated on correlated sources and binary stationary ergodic Markov sources [2]–[5]. The correlated information between the different sources is exploited at the receiver to estimate the source [2]–[4]. Guang-Chong Zhu et al. studied the design of joint source channel (JSC) turbo coding for binary stationary ergodic Markov sources [5]. Particularly, in recent two years, the joint source and channel coding with independent and identically distributed (i.i.d) binary memoryless source was elaborated in [1]. The novel structure is made up of two LDPC codes, which are independent from each other, a regular or irregular LDPC code as source coding for compressing the source to reduce the redundancy, and another regular or irregular LDPC code as channel code to ensure a reliable transmition. All the compressed bits are supposed to be the same length, which is very suitable for the joint source and channel decoding (JSCD) at the receiver due to no catastrophically affection to the decoded source data. The statistical information of the source is also utilized in the source decoder, which exchanges the updated extrinsic information with the channel decoder iteratively. Nevertheless, there are two problems left to be solved, that is, 1) how to reduce the computational complexity
978-1-4673-1157-1/12/$31.00 © 2012 IEEE
II. S YSTEM D ESCRIPTION The general model of transmission system is shown in Fig. 1. Let s = (s1 , ..., sn ) ∈ {0, 1}n be a binary sequence of length n generated by an i.i.d. nonuniform memoryless source with probability distribution (p0 , p1 = 1−p0 ), where p1 = P r(si = 1) ̸= 0.5. The entropy of the source is calculated as H(s) = −p1 × log2 (p1 ) − (1 − p1 ) × log2 (1 − p1 )
(1)
The sequence s is compressed by an unpunctured Protograph LDPC code of rate Rsc = l/n into the sequence u = (u1 , ..., ul ), which is followed by punctured Protograph LDPC code in order to ensure a reliable transmition, producing a codeword x = (x1 , ..., xm+γ ) of length m + γ including
65
N
Source LDPC
Channel LDPC m! "l
l Protograph LDPC source encoder
u
Protograph LDPC Channel encoder
x
y
s
m
JSC Decoder
m
m
sc ( k )
sc cc ( k ) v
m
c ,v
m
cc sc ( k )
cc ( k ) c ,v
v
m
sc ( k ) v ,c
cc ( k ) v ,c
Ă
Fig. 1. The system model of P-JSCC scheme. The encoding procedures are operated separately, while the decoding is considered to implemented jointly.
1
z
sc
n
v
cc
z
n 1z v n l c
c
v
Punctured nodes
{
s
n γ punctured variable nodes. Consequently, the channel code has a rate of Rcc = l/m, which is modulated by Binary Phase Shifting Keying (BPSK) and then transmited to an additive white Gaussian noise (AWGN) channel. The received signal y is a corrupted version of x because of the Gaussian noise from the channel. The joint decoder runs in parallel through a bipartite graph making use of Belief Propagation (BP) algorithm and the source statistics.
z
cc v
n!m!
m!
Fig. 2. A block diagram of joint decoder scheme. The two decoders are in the form of serial concatenation, but run in parallel actually.
sc(k)
cc(k)
mv,c and mv,c are respectively, the message passed from the vth variable node to the cth check node of the source sc(k) cc(k) code(Csc ) and the channel code (Ccc ). mc,v and mc,v are, respectively, the message passed from the cth check node sc→cc(k) to the vth variable node of (Csc ) and (Ccc ). mv is the message passed from the check node in (Csc ) connected to cc→sc(k) vth variable node in (Ccc ), while mv is the message passed from the vth variable node in (Ccc ) connected to the cth check node in (Csc ). The last two messages are indexed only by v, because there is a single connection between the every check node in (Csc ) with a variable node in (Ccc ). zvsc and zvcc are respectively the Log-Likehood Ratio (LLR) for the variable nodes for v = 1, ..., n and for v = n + 1, ..., n + m + γ. For independent binary sources transmitted over an AWGN channel, zvsc = ln((1 − p1 )/p1 ), p1 denotes the probability of a binary generated source bit of value one, and zvcc = 2yv /σn2 except the punctured variable nodes for which zvcc equals zero, where yv = (1 − 2xv ) + nv (xv is the vth bit from codeword uGcc ) and σn2 is the channel noise variance. All the bits of codewords are transmitted in [1], [8] through the channel while in our proposed scheme they are partly transmitted. The message passing between variable nodes and check nodes follows the same procedure within standard belief propagation algorithm. Firstly, the variable nodes send their LLRs to the check nodes and the corresponding messages are given by ∑ sc(k−1) msc(k) = zvsc + mc′ ,v (2) v,c
III. P ROTOGRAPH J OINT S OURCE AND C HANNEL C ODING A. Encoding of P-JSCC The lifting procedure for protograph LDPC codes is implemented by the progressive edge growth (PEG) algorithm [10] in order to remove all multiple parallel edges and generate a larger low-density matrix. The derived low-density matrix of source from protograph is denoted as Hsc with dimension l × n, while the derived code of channel is denoted as Hcc with size (m + γ − l) × (m + γ). The rate of the source code and channel code are calculated as Rsc = l/n < 1 and Rcc = l/m < 1, respectively. The overall rate of the proposed system is computed via R = Rcc /Rsc = n/m. The encoding operation of the source can be expressed by the equation u = Hsc s, which is followed by a punctured protograph LDPC code for the sake of protecting the compressed data from the noisy channel. The channel coding is done via simply multiplying the output of the source encoder u by the generator matrix Gcc , which satisfies Gcc Hcc = 0. We do not consider an interleaver between the source encoder and channel encoder, since the source decoder needs to know the location of the information nodes. B. Joint Source and Channel Decoding The corresponding decoder for this JSCC scheme consists of two sparse tanner graphs, one is for source decoder, and the other is for channel decoder. Their output extrinsic messages are fed into each other to update. We make some changes to the channel decoder, as shown in Fig. 2, where some variable nodes of channel codes denoted as blank circles are punctured before transmited to the AWGN channel. Each check node of the source code (left) is connected to a single variable node of the channel code (right),the only tunnel for extrinsic information exchange between the source and channel decoder. Thus, this joint decoder can be viewed as a concatenated decoder of two protograph LDPC codes. However, the two decoders run in parallel exchanging updated extrinsic information with each other through the connected edges.
c′ ̸=c
mcc(k) = zvcc + msc→cc(k−1) + v,c v
∑ ′
cc(k−1)
mc′ ,v
(3)
c ̸=c
mcc→sc(k) = zvcc + v
∑ ′
cc(k−1)
mc′ ,v
(4)
c
mcc(k) = zvcc + v,c
∑ ′
cc(k−1)
mc′ ,v
(5)
c ̸=c
where (2) runs for v = 1, ..., n; (3) and (4) run for v = n + 1, ..., n + l; and (5) runs for v = n + l + 1, ..., n + m + γ; sc(0) cc(0) sc→cc(0) and that mc′ ,v = 0, mc′ ,v = 0, mv = 0.
66
The message passing between the check nodes and the variable nodes are given by sc(k)
tanh(
cc→sc(k)
mc,v mv ) = tanh( 2 2 sc→cc(k)
tanh(
mv
2
)=
)
∏
sc(k)
tanh(
mv′ ,c 2
′
v ̸=v
∏
) (6)
sc(k)
tanh(
mv′ ,c 2
v′
)
(7)
cc(k)
tanh(
cc(k) mv′ ,c ∏ mc,v )= tanh( ) 2 2 ′
(8)
v ̸=v
Fig. 3. General protograph of the source with only one check node and four variable nodes connected by twelve edges.
where (6) and (7) run for c = 1, ..., l; while (8) runs for c = l +1, ..., l +m+γ. After K iterations of decoding process, some decisions are made according to the LLR of the source ∑ sc(K) bit sv . LLR(sv ) = zvsc + c mc,v . If LLR(sv ) ≥ 0, sbv = 0, else sbv = 1.
Protograph of ARA family Code rate = (n+1)/(n+2) n=0,1,Ă.
IV. E XPERIMENT R ESULT In this section, three sets of experiments are carried out to illustrate the significant performance gain of our proposed scheme. In all the simulations that follow, we use additive white Gaussian noise (AWGN) channel as transmission medium, and the overall rate R is set to be 2. In subsection A and subsection B, the source code we employ is considered to be the one derived from the protograph shown in Fig. 3, which consists of four variable nodes and one check node, connected by twelve edges. The base matrix presentation of the source code in form of [ ] general Bsc = 3 3 3 3 (9)
Fig. 4.
ARA code family.
dB, respectively. The base matrix of rate 1/2 ARA code and rate 1/2 AR4JA code are shown as follow. 1 2 1 0 0 (10) BARA = 0 2 1 1 1 0 1 2 1 1 1 2 0 0 0 BAR4JA = 0 3 1 1 1 (11) 0 1 2 1 2
Consider all the advantages of ARA codes described in [6] and [7], we introduce ARA code into our proposed P-JSCC scheme. The ARA code family is shown in Fig. 4, where the dark filled circles denote the bits to be transmitted, the dark filled squares denote the check nodes and the blank circles denote the punctured nodes. According to Fig. 4, the rate of the ARA codes is given by (n + 1)/(n + 2). In this paper, we only consider ARA code with code rate 1/2(n = 0).
With the fixed p1 for 0.02, the simulation results of the different JSCC schemes are shown in Fig. 5 for the length of s with 3200 bits and Fig. 6 for the length of s with 6400 bits, respectively. The source code rate and channel code rate are 0.25 and 0.5 (denoted as (0.25, 0.5)), respectively. The BER is evaluated in terms of the Eb /No (dB). The simulation results in Fig. 5 illustrate that the proposed scheme has about 1dB coding gain than the traditional scheme at the level 10−4 of BER. The P-JSCC scheme base on ARA code is the best among the three. However, one can observe from Fig. 5 that the proposed schemes also show as high error floor, i.e., about 1.5 × 10−4 , as the traditional approach. The proposed schemes are better than the traditional scheme as well when the length of input source sequence is enlarged. As shown in Fig. 6, the error floor region drops due to the
A. Comparison Between P-JSCC and Traditional Scheme In the first experiment, we make a comparison between the P-JSCC scheme and the traditional scheme proposed in [1] on error performance. We adopt regular LDPC codes derived from Fig. 3 with three ones per column as source code, which is the same as the one described in reference [5], [8] for the sake of comparison. However, the protograph based source code is computationally simpler on encoding and decoding implementation due to the protograph presentation. In this subsection, with reference to channel code, we introduce two kinds of protograph LDPC code, namely, ARA code and Accumulate repeat by 4 jagged accumulate (AR4JA) code. The iterative decoding threshold of ARA code and AR4JA code under AWGN channel are 0.516 dB and 0.628
67
(0.25,0.5), p1=0.02
−1
Different p1
−1
10
10
traditional ARA AR4JA
−2
10
−2
10
−3
BER
BER
10 −3
10
−4
10 −4
10
−5
10
−6
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
10
2.5
Eb/No (dB)
Fig. 5. BER comparison of P-JSCC schemes and the traditional scheme for the length of input source sequence with 3200.
Fig. 7.
(0.25,0.5), p1=0.02
−1
traditional ARA AR4JA
10
−3
BER
10
−4
10
−5
10
−6
10
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
0 Eb/No (dB)
1
2
BER comparison of P-JSCC scheme with the different P1 .
C. BER of the improved version of P-JSCC
2.5
E /N (dB) b
−1
used in this experiment are with source rate 0.25 and the channel code 0.5. The length of input source sequence is 6400, and the channel code we consider is ARA code. We change p1 from 0.02 to 0.03, which guarantees that the entropy of the source is not larger than Rsc . The results in Fig. 7 show that the lower the entropy is, the better the BER performance is, which is inosculate with the theorem of source coding. One can observe that when p1 comes to 0.03, the cliff turbo region even disappears. The error performance becomes better as the p1 goes down. Our proposed scheme is preferable to high redundancy source, that is, low entropy source.
10
−2
−2
0.02 0.025 0.03
o
The improvement of source code can lead to enhancement on error floor region, which has been proved in the paper [1]. Thus, in this subsection, an improved version of PJSCC scheme is presented. The degree of check nodes in the protograph of source plays an important role in the error floor region of the P-JSCC scheme. Hence, we modify the source code in order to lower the error floor region while we keep the channel codes invariable (ARA code and AR4JA). The modified protograph of source also has a simple structure with one check node and four variable nodes, which is show in Fig. 8. The base matrix of our improved protograph of source is [ ] improved Bsc = 2 3 6 7 (12)
Fig. 6. BER comparison of P-JSCC schemes and the traditional scheme for the length of input bits with 6400.
growth of the length of source sequence compared with that in Fig. 5. What’s more, the gain in turbo cliff region becomes more apparent while the performance on error floor remains the same. As described in [1], the error floor is determined by the source code, thus we can improve the performance of error floor region via modifying the source code, which is discussed in subsection C. The performance of P-JSCC scheme with AR4JA code is worse than that with ARA code, but there are still some coding gains compared with the traditional scheme. Take the performance into consideration, the P-JSCC scheme base on ARA codes is the best candidate for future multimedia communication.
The simulation results presented in Fig. 9 illustrate that the improved versions are better than these in subsection A due to the modification of source code. The ones base on improved source have no error floor when the BER comes to 10−6 while these in subsection A have an error floor when the BER reaches 10−4 . There are two orders of magnitude between the improved versions and these in subsection A. The larger the degree of check nodes, the better the performance of error floor region is. However, there may exist a tradeoff between the encoding complexity and the performance of error floor region.
B. BER of the Proposed P-JSCC Scheme for Different p1 In the second experiment, we show the influence of the entropy of a discrete binary i.i.d source on our proposed p-JSCC scheme. According to Shannon’s Source-Coding: A source that produced independent and identically distributed random variables with entropy H can be encoded with arbitrarily small error probability at any rate Rsc in bits per source output if Rsc ≥ H. Conversely, if Rsc < H, the error probability will be bounded away from zero, independent of the complexity of coder and decoder. The P-JSCC schemes
V. C ONCLUSION In this paper, we have proposed a joint source and channel coding scheme base on protograph-based codes. Simulation
68
R EFERENCES [1] Fresia. M, Perez-Cruz. F, Poor. H. V, “Optimized concatenated LDPC codes for joint source-channel coding,” in Proc. ISIT 2009, pp. 21312135, June 2009. [2] Garcia-Frias. J, “Joint source-channel decoding of correlated sources over noisy channels,” in Proc. DCC 2001, pp. 283-292, 2001. [3] Guang-Chong Zhu, Alajaji. F, “Joint source-channel turbo coding for binary Markov sources,” Wireless Communications, IEEE Transactions on, vol. 5, no. 5, pp. 1065-1075, May 2006. [4] Ali. K, Labeau. F, “Joint source-channel turbo decoding of entropy coded sources,” in Proc. VTC 2005, pp. 1960-1964, Sept, 2005. [5] Garcia-Frias.J, Zhong, W. Zhao.Y, “Iterative decoding schemes for source and joint source-channel coding of correlated sources,” in Proc. Signals, Systems and Computers, 2001, pp. 250-256, vol. 1, Nov. 2002 [6] Abbasfar. A, Divsalar. D, Kung Yao, “Accumulate-Repeat-Accumulate Codes,” Communications, IEEE Transactions on, vol. 55, no. 4, pp. 692702, April 2007. [7] Abbasfar. A, Divsalar. D, Kung Yao, “Accumulate repeat accumulate codes,” in Proc. GLOBECOM 2004, vol. 1, pp. 509- 513, Dec. 2004 [8] Fresia. M, Per´ez-Cruz. F, Poor. H. V, Verd´u. S, “Joint Source and Channel Coding,” Signal Processing Magazine, IEEE, vol. 27, no. 6, pp. 104-113, Nov. 2010. [9] Saeedi. H, Banihashemi. A, “Design of irregular LDPC codes for BIAWGN channels with SNR mismatch,” Communications, IEEE Transactions on, vol. 57, no. 1, pp. 6-11, January 2009. [10] Hu. X. Y, Eleftheriou. E, Arnold. D. M, “Regular and irregular progressive edge-growth tanner graphs,” Information Theory, IEEE Transactions on, vol. 51, no. 1, pp. 386-398, Jan. 2005.
Fig. 8. Improved protograph of source code with only one check node and four variable nodes connected by eighteen edges. (0.25,0.5), p1=0.02
−1
10
ARA AR4JA improved source + ARA improved source + AR4JA
−2
10
−3
BER
10
−4
10
−5
10
−6
10
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
Eb/No (dB)
Fig. 9. BER performance of the improved P-JSCC scheme compared with that in subsection A.
results illustrate that the proposed scheme significantly outperforms the traditional JSCC scheme with regular LDPC codes proposed in [5], [8]. What’s more, the P-JSCC scheme reduces the computational complexity of encoding and decoding implementation by a large scale. The entropy of the source plays a very significant role in the P-JSCC scheme for the reason that the performance becomes worse as the source entropy becomes higher in our proposed JSCC scheme, which is inosculate with the theorem of source coding. In addition, it is found that the performance of P-JSCC scheme on the error floor region can be improved remarkably via increasing the degree of check nodes in source code. However, the tradeoff between computational complexity and performance on error floor region must be taken into consideration. In summary, the P-JSCC scheme stands out as a good candidate or alternative for multimedia communication systems in the near future.
ACKNOWLEDGMENT This work was supported by National Natural Science Foundation of China under Grant 60972053. The authors would like to thank Min Xiao and Sijie Yang for their valuable suggestions.
69
