Optimizing Channel Coding for Orthogonal Multiple Access Schemes

1

Optimizing Channel Coding for Orthogonal Multiple Access Schemes with Correlated Sources A. Abrardo,1 G. Ferrari,2 M. Martal`o,2 M. Franceschini,3 and R. Raheli2 1: Universit`a di Siena, Dipartimento di Ingegneria dell’Informazione viale Roma 56, I-53100 Siena, Italy 2: Universit`a di Parma, Dipartimento di Ingegneria dell’Informazione viale G. P. Usberti 181A, I-43100 Parma, Italy 3: IBM T.J. Watson Research Center 1101 Kitchawan Road, Rte. 134, Yorktown Heights, NY 10598, USA Abstract— In this paper, we consider multiple access schemes with correlated sources. Distributed source coding is not used; rather, the correlation is exploited at the access point (AP). In particular, we assume that each source uses a channel code to transmit, through an additive white Gaussian noise (AWGN) channel, its information to the AP, where component decoders, associated with the sources, iteratively exchange soft information by taking into account the correlation. The key goal of this paper is to investigate whether there exist optimized channel codes for this scenario, i.e., channel codes which guarantee a desired performance level (in terms of average bit error rate, BER) at the lowest possible signal-to-noise ratio (SNR). A twodimensional extrinsic information transfer (EXIT) chart-inspired optimization approach is proposed. Our results suggest that by properly designing serially concatenated convolutional codes (SCCCs), the theoretical performance limits can be approached better than by using parallel concatenated convolutional codes (PCCCs) or low-density parity-check (LDPC) codes. It is also shown that irregular LDPC codes tend to perform better than regular LDPC codes, so that the design of appropriate LDPC codes remains an open issue.

Index Terms – Turbo codes, LDPC codes, correlated sources, joint channel decoding (JCD), wireless sensor networks. I. I NTRODUCTION Wireless sensor networks have recently received a significant attention [1]. The efficient transmission of correlated signals, observed at different nodes, to one or more collectors is one of the main challenges in such networks. In the case of one collector node, this problem is often referred to as reachback channel in the literature [2]–[4]. In its most simple form, the problem can be summarized as follows: two independent nodes have to transmit correlated sensed data to a collector node by using the minimum possible energy, i.e., by exploiting in some way implicit correlation among the data. In the case of separated additive white Gaussian noise (AWGN) channels the separation between source and channel coding is known to be optimal [5], [6]. This means that the theoretical limit can be achieved by first compressing each source up to the Slepian-Wolf (SW) limit and then utilizing two independent capacity achieving channel codes (one for each source) [7]. However, implementing a practical system based on separation, i.e., distributed source coding, is not straightforward [8]–[11]. Moreover, the problem of

designing good practical codes for correlated sources is still an open issue [12] and, moreover, separation between source and channel coding may lead to catastrophic error propagation [13]. Alternative approaches to distributed source coding are represented by cooperative source-channel coding schemes and joint source-channel coding (JSCC). In the JSCC case, no cooperation among nodes is required and the correlated sources are not source encoded but only channel encoded. If one compares a JSCC system with a system based on source/channel separation with the same information rate, the channel codes used in JSCC must be less powerful (i.e., they have higher rate). This weakness can be compensated by exploiting the correlation between the sources at the joint decoder, so that the final performance can approach the theoretical limits. This approach has attracted the attention of several researchers in the recent past, also because of its implementation simplicity [14]–[18]. Note that, in the JSCC approach, the sources are encoded independently of each other (i.e., for a given source neither the realization from the other source nor the correlation model are available at the encoder) and transmitted through the channel. Correlation between the sources must be instead assumed to be known at the (common) receiver which aims at the reconstruction of the two sources. Works dealing with joint source-channel coding have so far considered classical turbo or low-density parity-check (LDPC) codes, where the decoder can exploit the correlation among sources by performing message passing between the various decoders. Although great attention has been given to these topics in recent literature, the problem of designing good codes has not been addressed so far. As an example, in [18] the authors state that for separate channels, that is the case of interest in this paper, the type of turbo-like code utilized for the encoding is not critical, and good results can be obtained provided that powerful codes are applied. In this paper we show that this statement may be considered true only in a limited portion of the capacity region, and that turbo-like codes which present similar behavior in a classical AWGN scenario perform very differently when used for JSCC in presence of correlated sources over separated channels. In this paper, we consider a simple scenario with two correlated sources which transmit to a common access point (AP). The two sources do not make use of source coding, but only of channel coding. At the AP, proper iterative receivers

2

wx x

SN1

sx zx

AP y

SN2

zy

sy

E(|sx,i |2 ) and Ey = E(|sx,i |2 ), i = 0, . . . , k − 1 are the energies per transmitted symbol by the sources; and N0 = E(|wx,i |2 ) = E(|wx,i |2 ) is the variance of the AWGN samples (equal in both links). The SNRs at the AP in the two links are denoted as γx = Ex /N0 and γy = Ey /N0 . Finally, the conditional entropy of the correlated sources is H = −ρ log2 (ρ )− (1 − ρ ) log2 (1 − ρ ). It is well known that distributed source coding allows to reduce the amount of information to be sent by the couple of nodes by exploiting the correlation between the sources. In particular, denoting by rs,x and rs,y the compressing rates we have the following Slepian-Wolf bounds: rs,x > H rs,y > H rs,x + rs,y > 1 + H.

wy Fig. 1. Proposed multi-access communication scenario: two source nodes communicate directly to the AP.

are used to exploit the source correlation. We consider two main classes of channel codes: turbo (either serially or parallely concatenated convolutional codes, SCCCs or PCCCs) and LDPC. By using an extrinsic information transfer (EXIT)inspired approach, we propose an operational approach to evaluate the performance of the iterative receiver for various values of the source correlation. In particular, we analyze the two-dimensional signal-to-noise ratio (SNR) region where the proposed scheme guarantees error-free convergence. We show that the SNR region associated with turbo codes is wider than that associated with LDPC codes. Moreover, in the case of turbo codes, unlike classical results with single channel communication systems, we show that properly optimized SCCCs allow to extend the SNR region in the areas relative to unbalanced communication channels, i.e., areas where the quality of one of the link is significantly better than that of the other. On the other hand, it will be shown that the considered LDPC codes do not allow to efficiently exploit source correlation. The design of proper LDPC (and, more generally, channel) codes tailored for the considered multiple access scheme is an open issue. II. S CENARIO We consider two spatially distributed nodes which transmit two binary information signals x = [x0 , . . . , xk−1 ] and y = [y0 , . . . , yk−1 ], where k is the signal length (equal for both sources). The information signals are assumed to be temporally white with P(xi = 0) = P(xi = 1) = 0.5, P(yi = 0) = P(yi = 1) = 0.5, for i = 0, . . . , k − 1, and we define ρ , P(xi = yi ) > 0.5—the correlation coefficient is 2ρ − 1. The signals x and y are delivered to one collector node (the AP). We denote by s x = [sx,0 , . . . , sx,nx −1 ] and s y = [sy,0 , . . . , sy,ny −1 ] the samples transmitted by the nodes, nx and ny being the number of channel uses. In Fig. 1, we depict the proposed scenario, where SN stands for “Source Node”. The AWGN sequences w x and w y are assumed to be independent, owing to the fact that the sources are assumed to transmit over orthogonal channels (e.g., using time division multiple access). Let us introduce the following quantities: rx = k/nx , ry = k/ny are the transmission rates at the two source nodes; Ex =

(1)

By assuming that source coding (compression) is followed by channel coding within each SN block, the actual channel code rates rc,x and rc,y , expressed as rc,x = rs,x rx rc,y = rs,y ry

(2)

satisfy the following Shannon bounds: rc,x < 12 log2 (1 + γx ) rc,y < 12 log2 (1 + γy ) .

(3)

As discussed in Section I, compressing each source up to the SW limit and then utilizing two independent capacity achieving channel codes allows to achieve the ultimate performance limit. By introducing λx = 21 log2 (1 + γx) and λy = 1 2 log2 (1 + γy ), the achievable rates rx and ry have to satisfy the following inequalities: rx < λHx λ ry < Hy rx ry (1 + H) − λxry − λy rx < 0. III. P ERFORMANCE A NALYSIS

OF

(4)

JSCC S CHEMES

In this section, we propose an analytical approach, based on the use of Tanner graphs and belief propagation (BP) techniques, to the performance evaluation of JSCC schemes. In Subsection III-A, we first present the principle of joint channel decoding (JCD), a decoding strategy which allows to achieve the ultimate performance limits with JSCC schemes. Then, in Subsection III-B we propose a message passing-based performance evaluation of the considered JSCC schemes. A. JCD Principle Consider the case where no distributed compression is performed at the transmitters and correlation is fully exploited at the receiver. In the proposed scheme rs,x = rs,y = 1 and, hence, channel rates are equal to transmission rates. We consider random codes and allow their lengths to grow to infinity, (l) (l) i.e., k → ∞, nx , ny → ∞. Denote by x (l) = [x0 , . . . , xk−1 ], (q) (q) l = 0, . . . , 2k − 1, and y (q) = [y0 , . . . , yk−1 ], q = 0, . . . , 2k − 1, all the possible information sequences and assume that x(0) ,yy (0) are the transmitted sequences. Moreover, denote by (l) (l) (l) (q) (q) (q) s x = [sx,0 , . . . , sx,nx −1 ] and s y = [sy,0 , . . . , sy,ny −1 ] the Gaussian

3

observables associated to the information sequences and by wx and z y = s y +w wy the received vectors (see Fig. 1). z x = s x +w (0) (0) Without loss of generality, denote by s x ,ssy the couple of transmitted codewords. The possible transmitted sequences p √ (l) (q) s x and s y belong to spheres of radius Ex nx and Ey ny , (0) respectively. On the other hand, √ the transmitted vector s x belongs to a sphere of radius N0 nx centered in the observed p (0) vector z x and s y belongs to a sphere of radius N0 ny centered in the observed vector z y . Hence, defining decision region 1 √ (DR1 ) as the intersection of a sphere of radius N0 nx centered (l) in zx with the sphere of the transmitted codeword sx and decision p region 2 (DR2 ) as the intersection of a sphere of radius N0 ny centered in zy with the sphere of the transmitted (q) (0) (0) codewords s x , the transmitted vectors s x and s y belong to the respective decision regions DR1 and DR2 with probability (l) one. The probability Px that a random codeword s x belongs to the decision region DR1 is [19]: nx /2  1 (5) Px = 1 + γx (l)

and the probability Py that a random codeword s y belongs to the decision region DR2 is: ny /2  1 . (6) Py = 1 + γy In classical communication systems without correlation between the information sequences and with orthogonal multiple access, the receiver may independently perform detection for the two received signals. In particular, if only one vector lies (0) in DR1 the receiver detects sx for sure. Otherwise, if more than one vector lies in DR1 an error occurs in the first channel. The same rule can be applied to the second channel, and the well known Shannon capacity bound which allows to perform error free detection can be evaluated for the two channels separately. In presence of correlation the receiver may improve performance by joint detection. To define a proper joint detection rule, observe that, due to correlation, each couple of transmitted information sequences is characterized by an a-priori joint probability. In particular, given the correlation model considered in this paper, the joint probability of any (0) (0) transmitted couple of signals, say s x s y , is:   1 (0) (0) P s x ,ssy = k ρ d (1 − ρ )k−d (7) 2 where d is the Hamming weight of x (0) ⊕ y (0) , i.e., the Hamming distance between the information signals. Note that any couple of transmitted signals is characterized by an average distance d = (1 − ρ )k. We then consider a simple joint decoding scheme based on the following decision strategy: 1) If the joint decision region DR1 ∪ DR2 contains only (0) (0) the couple (ssx , s y ), then (xx(0) , y (0) ) is selected. 2) If the joint decision region DR1 ∪ DR2 contains more than a couple of sequences, consider only those sequences which have distance d = (1 − ρ )k. If there is only one couple with distance d = (1 − ρ )k, then select

it. If there is more than one sequence with d = (1 − ρ )k an error occurs. It can be shown that the proposed JCD scheme allows to achieve system capacity. B. JCD via Message Passing Assume that the information sequences at the source nodes are encoded using linear binary codes, denoted as Cx and Cy , respectively. Binary transmissions are then considered, i.e., sx,i = ±1, i = 0, . . . , nx − 1 and sy, j = ±1, j = 0, . . . , ny − 1. The joint maximum a-posteriori probability (MAP) decoding rule reads: xˆi = argmax ∑ p (zz x ,zz y |xx,yy) p (xx,yy)

i = 0, . . . , nx

yˆ j = argmax ∑ p (zzx ,zz y |xx,yy ) p (xx,yy)

j = 0, . . . , ny

xi =0,1 y ,xx∼xi

y j =0,1 x ,yy∼yi

(8)

where p (xx,yy) denotes the joint probability that the information sequences x and y are transmitted and the notation ∑ denotes ∼

the summation over all variables contained in the expression except the one listed after the operator ∼. Without loss of generality, we focus on the estimate xˆi . From (8), by using the total probability theorem we can write: xˆi

= argmax

∑

xi =0,1 y ,xx∼xi

= argmax

p (zz x ,zz y |xx,yy) p (xx,yy)

∑ ∑ p (zzx ,zzy |ssx ,ssy ,xx,yy) p (ssx ,ssy |xx,yy) p (xx,yy)

xi =0,1 y ,xx∼xi s x ,ssy

= argmax

∑ ∑ p (zzx ,zzy |ssx ,ssy ) p (ssx ,ssy |xx,yy) p (xx,yy)

(9)

xi =0,1 y ,xx∼xi s x ,ssy

where, in the last line, it has been observed that the joint probability density function (pdf) of zx and zy , conditionally on sx ,ssy , does not depend on y,xx. The probability p (ssx ,ssy |xx,yy) is equal to one if sx and sy are the codewords associated with x , y , respectively, or zero otherwise. Since the two information sequences x and y are coded independently, it follows that: p (ss x ,ss y |xx,yy) = p (ssx |xx) p (ssy |yy) .

(10)

On the other hand, since the coded signals are sent over orthogonal AWGN channels, we have: p (zzx ,zz y |ssx ,ssy ) = =

p (zzx |ssx ) p (zzy |ssy ) nx

ny

j=1

j=1

∏ p (zx, j |sx, j ) ∏ p (zy, j |sy, j ) .

(11)

Taking into account the correlation model of the information sequences, one obtains: k

p (xx,yy) = ∏ p (x j , y j ) = j=1

1 2k

k

∏ ρ 1−x j ⊕y j (1 − ρ )x j ⊕y j .

(12)

j=1

At this point, problem (9) admits a Tanner graph representation and a corresponding believe propagation (BP) solution, provided that p (ssx |xx ) and p (ssy |yy) can be expressed as products of factors which depend on restricted subsets of all symbol variables. This is always possible if Cx and Cy are convolutional codes or a serial or parallel combination of convolutional codes (i.e., turbo codes). Indeed, by introducing the state sequence as hidden variables, p (ssx |xx ) (or equivalently

4

Tanner graph of Cx

From hannel: SNR =

G2

γy SNR2out

x0

xk−1

x1

SNR2in (Extrinsi info.)

(For PCCC only)

SNR1in

......

p(x0, y0)

SNR1out

G1

p(xk−1, yk−1)

p(x1, y1) a′y

y0

y1

yk−1

SNRe,y

p(x, y) a′x

Tanner graph of Cy

BSC(ρ)

SNRe,x

Fig. 2.

Tanner Graph for problem (9).

Fig. 3.

p (ssy |yy)) can be written as the product of n factors representing the allowed state transitions [20]. Another situation where problem (9) easily admits a Tanner graph-based representation is when Cx and Cy are LDPC systematic codes. In this case, p (ssx |xx) is the product of n − k parity check equations which involve few parity and systematic bits. In both cases, the solution of (9) can be obtained by using a graphical approach. More precisely, one can consider two separate Tanner Graphs corresponding the codes Cx and Cy used by the two sources. A pictorial description of the global Tanner graph is shown in Fig. 2. The Tanner graphs of Cx and Cy are the usual bipartite graphs, e.g., of turbo or systematic LDPC codes [20]. In Fig. 2, for ease of clarity, the variable nodes x and y are explicitly shown. Each variable node x j of the Tanner graph of Cx is connected to the corresponding node y j of the Tanner graph of Cy through a connection node, denoted by the joint pdf p(x j , y j ) in (12). Note that this probability depends on ρ . The connection nodes, upon receiving the messages, in the form of logarithmic likelihood ratios (LLRs), from one of the two Tanner graphs, send input LLRs to the other Tanner graph. For instance, upon reception of LLRx, j from the j-th variable node of the Tanner graph of Cx , the connection node “extracts,” exploiting the correlation according to (12) and using a standard BP procedure, the following LLR and passes it to the j-th variable node of the Tanner graph of Cy : LLRy, j

p(x j = 0)ρ + p(x j = 1)(1 − ρ ) p(x j = 1)ρ + p(x j = 0)(1 − ρ ) ρ eLLRx, j + (1 − ρ )e−LLRx, j = ln −LLR x, j + (1 − ρ )eLLRx, j ρe = ln

(13)

where ln is the natural logarithm. Note that LLRy, j may be seen as a-priori information of the transmitted bits and can thus easily be taken into account by standard soft-input softoutput decoders. The scheduling of the BP procedure on the overall graph is as follows. We initialize the messages output by the function nodes p(x j , y j ) to zero and we run “internal” BP iterations within one of the componenent Tanner graphs, e.g., Cx . At the end of these BP iterations, messages LLRx, j are fed to the connecting nodes p(x j , y j ) which, in turn, output new LLRs

Density evolution scheme for Cy .

for the component Tanner graph Cy . The iterations between the two Tanner graphs, through the connection nodes, are denoted as “external.” IV. P ERFORMANCE E VALUATION OF T URBO C ODES A. A Two-dimensional EXIT Chart-Inspired Approach We assume that the two transmitters use two identical rate-1/2 linear turbo codes, where with the term “turbo” we generally refer to SCCCs or PCCCs. PCCCs consist of two parallel recursive systematic convolutional (RSC) codes where the lower code receives at its input an interleaved version of the information sequence [21], [22]. The SCCCs consist of a cascade of an outer convolutional code (CC) and an inner CC with a bit interleaver between them [23]. In order to evaluate system performance, we consider a density evolution approach. In particular, in order to characterize the internal behavior of turbo codes, i.e., the evolution of the LLRs, we refer to the density evolution approach proposed in [24]. In Fig. 3, the reference scheme for evaluating density evolution for the Tanner graph relative to Cy code is shown. An identical same scheme applies also for the evaluation of the density evolution for the decoder of code Cx . In order to analyze the density evolution for code Cy , we assume that the corresponding source transmits the all-zero sequence. A turbo decoder is composed of two decoders, referred to as G1 and G2 . In the SCCC scenario, G1 corresponds to the inner decoder and G2 to the outer decoder; in the PCCC scenario, instead, G1 corresponds to the upper decoder and G2 to the lower decoder. Messages (e.g., LLRs) from the channel are fed at the input of both decoders in the PCCC case, while in the SCCC case they are fed only to the input of the inner decoder (in Fig. 3, this is denoted by using a dashed line to connect the channel messages to decoder G1 ). Density consistency is imposed by assuming that each message √ is modeled as a Gaussian random variable with pdf N(µ , 2µ ), and we define SNR , µ /2. Using this assumption, the density evolution approach proposed in [24] allows to evaluate the SNR of the extrinsic information messages at the output of the two component decoders (denoted as SNR1out and SNR2out , respectively), as a function of the SNR of the

5

extrinsic informations at the input of the component decoders (denoted as SNR1in and SNR2in , respectively). These SNRs can be intepreted as “internal quantities,” since they represent SNRs of messages which are internally exchanged between the decoders G1 and G2 . To encompass the presence of apriori information coming from the other component turbo decoder, let us denote by SNRe,x the SNR of external messages LLRx, j entering the connection nodes {p(x j , y j )}. We assume that also messages LLRx, j have a Gaussian distribution with p mean 2 SNRe,x and standard deviation 4 SNRe,x ), so that their pdf is completely determined by SNRe,x . These messages are processed by the connection nodes {p(x j , y j )} to produce apriori information messages {LLRy, j } for decoder 1 of the Tanner graph of Cy . Denote the common pdfs of the messages {LLRx, j } and {LLRy, j } as ax (z) and ay (z), respectively—z is a dummy variable. It is worth noting that a density evolution approach requires that both ax (z) and ay (z) are densities of messages under the assumption that the all-zero information sequence has been transmitted. Hence, since sx and s y are correlated but not identical, for analysis purposes it is expedient to introduce a binary symmetric channel (BSC)-like block with cross-over probability ρ at the input of any function node p(x j , y j ). This channel “flips” a bit at its input with probability ρ and, therefore, the BSC-like block changes the sign of the corresponding LLR with the same probability. The messages at the output of the BSC block, denoted as LLR′x, j , are then characterized by the following pdf: a′x (z) = ρ ax (z) + (1 − ρ )ax(−z). The pdf of LLRy, j can eventually be computed according to (13), where the input messages {LLRx, j } are replaced by {LLR′x, j }. Note that, unlike the other messages, a′x (z) and a′y (z) are not assumed to be Gaussian and, thus, they are indicated explicitly in Fig. 3, instead of using the corresponding SNR value. The behavior of each component decoder (G1 and G2 ) in Fig. 3 is then determined through sampling, as follows. For input messages with a given pdf, we run a SISO algorithm on a very long trellis, which we initialize with independent samples from the given distributions. By collecting a large number of output samples, one can estimate the output SNRs. To formalize in a simple way the decoding procedure for both the decoders relative to codes Cy and Cx , we remove the subscripts x and y in all variables and denote by Z1 the nonlinear function which gives SNR1out as a function of SNR1in . We can then write: SNR1out = Z1 (SNR1in , γ , SNRe ) (14) where γ is the channel SNR which appears, in equation (14), only for the PCCC case. Moreover, if we denote by Z2 the nonlinear function which returns SNR2out as a function of SNR2in , we can write: SNR2out = Z2 (SNR2in , γ ) .

(15)

Imposing SNR2in = SNR1out and SNR1in = SNR2out , one can combine (14) and (15) as follows:     SNR(i+1) = Z2 Z1 SNR(i) , γ , SNRe , γ . (16)

Letting SNR(0) = 0, equation (16) allows to characterize the evolution of the SNR during turbo (inner) decoding iterations between the two component convolutional decoders of the turbo code (either PCCC or SCCC). As in a standard density evolution analysis for turbo codes [24], it can happen that the SNR goes to infinity or there can be a finite equilibrium point. In both cases, we denote by SNRs the final SNR which is achieved by iteratively solving (16). The SNR of the extrinsic information messages which are passed to the other turbo decoder can be finally expressed as SNRe = Z1′ (SNRs , γ , SNRe )

(17)

where Z1′ is the nonlinear function which gives SNRe as a function of SNRe . The overall iterative decoding procedure can then be described as follows. • Let SNRe = 0 and run (16) for decoder Cx . • At the end of the internal decoding operations at decoder Cx , evaluate the new value of SNRe according to (17) and run (16) for decoder Cy . The above steps are repeated until convergence is reached. The values of SNRe corresponding to equilibrium points for the two component decoders are denoted as SNRx and SNRy . We can conclude that the iterative decoding will converge to the correct information sequences x and y if (and only if) both SNRx and SNRy tend to infinity. This approach to the convergence analysis of the iterative decoder in the considered multiple access schemes can be interpreted as a sort of twodimensional EXIT chart-inspired approach. B. Validation of the Proposed Approach In order to validate the accuracy of the proposed approach, we compare the obtained results with classical Monte Carlo simulation results. To this end, we consider two specific turbo codes: • a rate-1/2 PCCC scheme based on two parallel RSC codes; • a rate-1/2 SCCC scheme based on an outer non systematic (and non recursive) convolutional (NSC) code and an inner RSC code. These code structures have been shown to guarantee the best performance in a classical AWGN scenario [21]–[23]. The 8-state rate-1/2 RSC code is characterized by the following generator matrix:   1 + D + D2 + D3 GRSC (D) = 1, . (18) 1 + D2 + D3 The 8-state rate-1/2 NSC code is characterized by the following generator matrix:   GNSC (D) = 1 + D + D3, 1 + D + D2 + D3 .

The PCCC has an overall rate equal to 1/2, obtained via classical puncturing which allows to select the coded bits alternatively from the two encoders. In the SCCC case, the outer code is punctured to obtain a rate equal to 3/4 while

6

5

4

Z

1

4.5

Z1

3.5

4 3

γ = −2.3

1.5

−1 2

3

Z

out

γ = −2 γ = −1.5

SNR

SNR

out

2.5 2

3.5

γ = −1

2.5 γ = −1.3 2 γ = −0.7

Z−1

1.5

2

γ=0

1 1 0.5

0.5

0 1

2

3

4

5 SNRin

6

7

8

0

9

Fig. 4. Density evolution of a single SCCC decoder for SNRe = ∞ and ρ = 0.9.

which deletes one half of parity bits (systematic bits are not punctured). 1) Single Turbo Decoder Convergence in the Presence of A Priori Information: The first scenario refers to the case where one of the two turbo decoders of Fig. 2 has already converged and, hence, it passes a very reliable a priori information to the other decoder (SNRe = ∞). In this case, referring to (16) and (15), we can test the single turbo decoder convergence by setting SNRe = ∞ and by showing the output SNR of decoder G1 versus its input SNR (i.e., the curve Z1 ) and the input SNR of decoder G2 versus its output SNR (i.e., the curve Z2−1 ), for a given γ . In Fig. 4, we show such a plot for the SCCC case with ρ = 0.9 and different values (in dB) of γ . As in usual density evolution for turbo decoding [24], the interpretation of the curves reflects the improvement of the bit error rate (BER) during BP iterations: the farther the curves the faster the convergence to a zero BER. When the two curves touch each other, no further improvement is possible by successive iterations, and the final BER is higher than zero. Note that in the SCCC code the channel observations are only fed at the input of the inner decoder, and hence Z2−1 does not depend on γ . Conversely, by reducing γ , the Z1 curve goes down and the two curves get closer. The lowest value of γ which guarantees convergence is denoted as γSCCC . In the example of Fig. 4, γSCCC = −2.3. The results of the same density evolutionbased analysis in the PCCC case are shown in Fig. 5. From these results, it follows immediately that the threshold SNR is γPCCC = −1.31.

1

1.5

2

2.5 SNR

3

3.5

4

4.5

in

Fig. 5. Density evolution of a single PCCC decoder for SNRe = ∞ and ρ = 0.9.

1

the inner code is punctured to obtain a rate equal to 2/3. The puncturing matrix Po for the rate 3/4 outer code is   1 1 0 Po = 1 1 0

γSCCC

0.5

s

γSCCC

0

γ

−0.5

γ

PCCC s PCCC

−1 critical γ

which punctures (or erases) one third of the parity bits. The puncturing matrix Pi for the rate-2/3 inner code is given by:   1 1 Pi = (19) 1 0

0.5

−1.5 −2 −2.5 −3 −3.5 −4 0.5

Fig. 6.

0.55

0.6

0.65

0.7

ρ

0.75

0.8

0.85

0.9

0.95

Threshold SNRs (for an SCCC and a PCCC) as functions of ρ

Following the same procedure, we have derived the critical channel SNRs γSCCC and γPCCC for different values of ρ . These results are shown in Fig. 6, where for comparison purposes, we also report the threshold SNR values, denoted s s and γPCCC , obtained by simulations. In particular, as γSCCC simulations results have been derived for a scenario with random interleavers and a frame size k equal to 105 . The critical simulation-based SNR values correspond to scenarios with a BER equal to 10−5 . As one can see, the simulation results confirm the analytical predictions. As expected, the results in Fig. 6 show that the PCCC and the SCCC achieve almost the same performance for low values of ρ . Instead, quite surprisingly, the considered SCCC allows to achieve noticeable gain with respect to the PCCC in the presence of significant a priori information. 2) Joint Decoding Convergence for Balanced Channel SNRs: While we have previously considered the decoding convergence of a single turbo decoder (by analyzing the evolution of the a priori information in the single component

7

16

1 γ

Zo

SCCC

−1

14

Zo

γs

SCCC

0.5

γPCCC

12

s

γPCCC 0 critical γ

SNRe,out

10 SCCC

8 PCCC

−0.5

6 −1 4 −1.5 2 0

0

2

4

6

8 SNR

10

12

14

−2 0.5

16

0.55

0.6

e,in

Fig. 7.

Density evolution of joint decoding for γ = −0.7 dB.

decoders), we now analyze the convergence behaviour of the overall receiver at the AP, i.e., a receiver with two component turbo decoders. In particular, we assume that the two channels are characterized by the same SNR γ = γx = γy (balanced channels). In this case, considering (17), we can analyze the joint decoding convergence by drawing, for a given γ , the Zo curve and its inverse Zo−1 . In Fig. 7, the Zo and Zo−1 curves relative to the SCCC and the PCCC are shown for ρ = 0.9 and γ = −0.7 dB. These results allow to characterize the decoding convergence during external iterations (between the component turbo decoders associated with the two sources): the farther the curves, the faster the joint decoding convergence to (theoretically) zero BER. As in the single decoder case, when the two curves touch each other, then global decoding convergence is not reached and the BER is higher than zero. The results in Fig. 7 show that the trend of the Zo (and Zo−1 ) curve is similar in the SCCC and PCCC cases. However, it can be observed that, for γ = −0.7 dB, the “tunnels” in the SCCC and PCCC cases have approximately the same width—although for different values of the SNR of the input reliability sequence. We have then evaluated the critical values of γ , still denoted as γSCCC and γPCCC , for different values of ρ and we have compared these s s and γPCCC obtained by simulations (with values with γSCCC the same set-up as in the single decoder case). These results are shown in Fig. 8. The results in Fig. 8 confirm that the SCCC slightly outperforms PCCC (as predicted by the results in Fig. 7). Moreover, the proposed density evolution approach for evaluating the joint decoding performance is in very good agreement with the simulation results. Another comment is worthwile at this point. As already observed in [18], the results in Fig. 8 confirm that, for balanced channels, the type of turbo-like channel code used at the sources is not critical, since the channel codes which perform well in the AWGN case without correlation are still good in the presence of correlation. On the other end, the results in Fig. 6 suggest that in the unbalanced case, i.e., when the SNRs in the two channels are different, different channel coding

Fig. 8.

0.65

0.7

ρ

0.75

0.8

0.85

0.9

0.95

Critical signal to noise ratios as a function of ρ

strategies entail a different performance. Therefore, the design of optimized channel codes, which allow to approach the theoretical performance limits, is of interest. In the following subsection, we derive an optimized SCCC structure. C. Optimized SCCC We consider the problem of optimizing SCCC decoding convergence in the presence of a priori information, i.e., in the case where the SCCC decoder gets, at its input from the other SCCC decoder, a very reliable (SNRe = ∞) a priori information. In particular, we optimize the generator matrix of the outer code Gouter (D), while we do not change the structure of the inner code (which is still characterized by the generator matrix GRSC (D) given in (18)). The number of states of the outer code is kept fixed to 8. Using the proposed density evolution approach, for a given value of ρ = 0.9 one can run an exhaustive search to determine the code structure which allows to achieve the lowest value of the critical (for decoding convergence) channel SNR γSCCC . As an example, in the case with ρ = 0.9 the optimized outer code generator matrix is the following:   1 + D2 + D3 1 + D + D2 + D3 , Gouter (D) = . 1 + D + D3 1 + D + D3 By using the above code generator polynomial, the critical channel SNR in the balanced channel scenario is γSCCC = −3.2 dB, i.e., there is a gain of almost 1 dB with respect to the classical SCCC with an outer NSC code. In the following, we denote by SCCCopt the optimized SCCC code. It is worth noting that the optimized coding scheme requires the use of an outer recursive non systematic convolutional (RNSC) code. This goes against classical results on optimized SCCC for transmission over a single channel [23]. In order to derive further insights into the performance of the JCD schemes considered in this paper, it is of interest to determine the feasible two-dimensional channel SNR regions as the ensemble of SNR pairs (γx , γy ) which satisfy (4) for given values of the common information rate r = rx = ry

8

3 ZLDPC

3

2.5

2

2

1 PCCC

γ

y

0

out

SNRe-sist 1.5 SCCC

−1

1

SCCCopt

−2

0.5

−3

γ = 0.4 dB γ = 0.3 dB γ = 0.25 dB γ = 0.23 dB

-1

ZLDPC

Theoretical limit −4 −5 −5

Fig. 9.

0 0 −4

−3

−2

−1 γx

0

1

2

3

Feasible two-dimensional regions for ρ = 0.9

LDPC de oder

γ (From the

hannel)

•

p(x, y)

BSC(ρ)

SNRin e−sist

(From the other hypotheti al de oder)

Fig. 10. Scheme for the analysis of the evolution of the a priori information. •

(i.e., no link rate adaptation is considered) and ρ . In Fig. 9, the feasible two-dimensional SNR regions associated with the considered turbo codes (PCCC, SCCC, and optimized SCCC) are shown. As one can see, while in the balanced case scenario (γ = γx = γy ) almost all codes guarantee the same performance (namely, the edge of this region corresponds to γ ≃ −1 dB), the optimized SCCC extends significantly the feasible region in the unbalanced SNR case, i.e., when one of the two channel SNRs is significantly higher than the other. V. P ERFORMANCE E VALUATION

OF

LDPC C ODES

In order to analyze the performance of LDPC codes, we extend the approach proposed for turbo codes as follows. We study the evolution of the SNR of the a priori information sequence, modeled as a Gaussian sequence. In particular, we assume that the all-zero information (and, therefore, coded) sequence is transmitted. In Fig. 10, an illustrative scheme to evaluate the evolution of the a priori information through the LDPC code is shown. The steps of this simulation-based analysis, for each simulation run, can be summarized as follows.

1

1.5 in SNRe-sist

2

2.5

3

Fig. 11. Evolution of extrinsic information evolution with regular LDPC code with ρ = 0.9. Various values for the channel SNR γ are considered.

•

SNRout e−sist

0.5

We assume that the all-zero sequence is transmitted. Therefore, the LDPC decoder receives, at its input, a sequence of Gaussian observables with channel SNR denoted as γ . As considered in Section IV, the input a priori information sequence on the systematic bits (given by a sequence of hypothetical LLRs) √ is modeled as a Gaussian sequence with distribution N(µ , 2µ ), and we denote the SNR µ /2 as SNRin e−sist . In order to take into account the fact that this information is originated by another decoder, it is passed through a BSC-like block with cross-over probability ρ . The sequence at the output, representative of a hypothetical sequence of LLRs originated by the other decoder, is processed taking into account the correlation and input to the LDPC decoder. After a fixed (set to 50) number of message passing decoding operations, from the soft-output information sequence at the output of the decoder, the extrinsic information sequence is extracted and the output SNR, denoted as SNRout e−sist .

For a fixed value of the channel SNR, tha above steps allow to determine numerically the function ZLDPC such that:   in (20) SNRout e−sist = ZLDPC SNRe−sist . In Fig. 11, the behaviour of the function ZLDPC introduced in (20) is shown, for various values of the channel SNR. In all −1 cases, ρ = 0.9. In the same figure, the inverse function ZLDPC is also shown. The decoding trajectory between the two LDPC −1 decoder at the receiver side lies between ZLDPC and ZLDPC , and converge is guaranteed only if there is a “tunnel.” As one can see, the tunnel is closed for relatively high values of the channel SNR (γ = 0.2 dB), where the optimized SCCC converges. Moreover, the slope of the ZLDPC curve, for low values of the channel SNR, is very small. In other words, the a priori information message has to be extremely reliable in order to have a significant (positive) impact on the decoding process of a single decoder.

9

2

3

Double diag.

2.5

1.5

2

1

out

SNRe-sist 1.5

γ ZLDPC

1 0.5 0 0

Regular

(2)

γ = 0.4 dB γ = 0.2 dB γ = 0 dB γ = -0.28 dB

0.5

(1000,2000)

0.5 (10000,20000)

0 -1

ZLDPC

1

1.5 in SNRe-sist

2

2.5

3

Fig. 12. Evolution of extrinsic information evolution with double diagonal LDPC code with ρ = 0.9. Various values for the channel SNR γ are considered.

As suggested by the results in [25], a good channel code, i.e., a code which allows to fully exploit the potential benefits of a-priori information, must associate information sequences with high Hamming distances to codewords with low Hamming distances. To this end, the irregular LDPC code with double diagonal square submatrix in the parity check matrix proposed in [17] provides a sort of implicit “differential encoding” effect which is in agreement with the guidelines presented in [25]. This irregular LDPC code will be referred to as double diagonal (dd) LDPC code. In Fig. 12, the behaviour of the function ZLDPC−dd introduced in (20) is shown, for various values of the channel SNR and fixed value of ρ set to 0.9. The conclusion reached at the end of the previous paragraphs are confirmed by the results shown in Fig. 13, where the feasible two-dimensional SNR regions corresponding to the considered LDPC codes (regular and double diagonal) are shown, considering various values of the information length k (namely, 1000 and 10000) and a fixed value of ρ set to 0.9. As one can see, in each case the borders of the feasible regions are mostly vertical and horizontal, i.e., there is a very limited portion around the balanced channel SNR point (γ , γ ). This means that if one of the two channel SNRs is below a given threshold, then the overall receiver cannot recover the limitations imposed by the channel. Comparing the results in Fig. 13 with those in Figure 9, one can conclude that the feasible regions with turbo codes is wider than that with LDPC codes. However, since we have considered only two possible LDPC codes without any optimization, properly designed LDPC codes might guarantee a performance similar to that of turbo codes. VI. C ONCLUSIONS In this paper, we have considered the optimization of channel coding for a multiple access scheme with two correlated sources and orthogonal channels. Through a novel twodimensional EXIT chart-inspired approach, we have anlyzed the “potential” of specific channel codes to exploit the source correlation at the AP. The system performance has been

-0.5 -0.5

0

1

0.5

1.5

2

(1)

γ

Fig. 13. Feasible two-dimensional SNR regions for the considered LDPC codes (regular and double diagonal) with various values of the codeword length and ρ = 0.9.

evaluated in terms of achievable SNR region. Our results show that properly optimized SCCCs guarantee the widest achievable SNR region, especially when the channel SNRs are unbalanced. On the other hand, the considered LDPC codes (one regular and one irregular) are characterized by smaller achievable SNR regions, underling that correlation can be exploited in a more limited way. However, proper design of LDPC codes is an open issue. R EFERENCES [1] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Caryirci, “A survey on sensor networks,” IEEE Commun. Mag., vol. 40, no. 8, pp. 102–114, August 2002. [2] J. Barros and S. D. Servetto, “On the capacity of the reachback channel in wireless sensor networks,” in IEEE Workshop on Multimedia Signal Processing , US Virgin Islands, USA, December 2002, pp. 480–411. [3] P. Gupta and P. R. Kumar, “The capacity of wireless networks,” IEEE Trans. Inform. Theory, vol. 46, no. 2, pp. 388–404, March 2000. [4] H. El Gamal, “On the scaling laws of dense wireless sensor networks,” IEEE Trans. Inform. Theory, vol. 51, no. 3, pp. 1229–1234, March 2005. [5] S. Shamai and S. Verd`u, “Capacity of channels with uncoded side information,” European Trans. Telecommun., vol. 6, no. 5, pp. 587–600, September-October 1995. [6] J. Barros and S. D. Servetto, “Network information flow with correlated sources,” IEEE Trans. Inform. Theory, vol. 52, no. 1, pp. 155–170, January 2006. [7] D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inform. Theory, vol. 19, no. 4, pp. 471–480, July 1973. [8] A. Aaron and B. Girod, “Compression with side information using turbo codes,” in IEEE Data Compression Conference, Snowbird, UT, USA, April 2002, pp. 252–261. [9] J. Bajcsy and P. Mitran, “Coding for the Slepian-Wolf problem with turbo codes,” in Proc. IEEE Global Telecommun. Conf. (GLOBECOM), San Antonio, TX, USA, November 2001, pp. 1400–1404. [10] I. Deslauriers and J. Bajcsy, “Serial turbo coding for data compression and the Slepian-Wolf problem,” in IEEE Information Theory Workshop, Paris, France, March-April 2003, pp. 296–299. [11] Z. Xiong, A. D. Liveris, and S. Cheng, “Distributed source coding for sensor networks,” IEEE Signal Processing Mag., vol. 21, no. 5, pp. 80– 94, September 2004. [12] Q. Zhao and M. Effros, “Lossless and near-lossless source coding for multiple access networks,” IEEE Trans. Inform. Theory, vol. 49, no. 1, pp. 112–128, January 2003. [13] W. Zhong and J. Garcia-Frias, “LDGM codes for channel coding and joint source-channel coding of correlated sources,” EURASIP Journal on Applied Signal Processing, 2005.

10

[14] J. Garcia-Frias and Y. Zhao, “Compression of correlated binary sources using turbo codes,” IEEE Commun. Lett., vol. 5, no. 10, pp. 417–419, October 2001. [15] F. Daneshgaran, , and M. L. M. Mondin, “Iterative joint channel decoding of correlated sources employing serially concatenated convolutional codes,” IEEE Trans. Inform. Theory, vol. 51, no. 7, pp. 2721–2731, August 2005. [16] J. Muramatsu, T. Uyematsu, and T. Wadayama, “Low-density paritycheck matrices for coding of correlated sources,” IEEE Trans. Inform. Theory, vol. 51, no. 10, pp. 3645–3654, October 2005. [17] F. Daneshgaran, , and M. L. M. Mondin, “LDPC-based channel coding of correlated sources with iterative joint decoding,” IEEE Trans. Inform. Theory, vol. 54, no. 4, pp. 577–582, April 2006. [18] J. Garcia-Frias, Y. Zhao, and W. Zhong, “Turbo-like codes for transmission of correlated sources over noisy channels,” IEEE Signal Processing Mag., vol. 24, no. 5, pp. 58–66, September 2007. [19] C. E. Shannon, “Communication in the presence of noise,” Proceedings of the IRE, vol. 37, no. 1, pp. 10–21, January 1949. [20] T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge,

UK: Cambridge University Press, 2008. [21] D. Divsalar, S. Dolinar, R. J. McEliece, and F. Pollara, “Performance analysis of turbo codes,” in Proc. IEEE Military Comm. Conf. (MILCOM), vol. 1, San Diego, CA, USA, November 1995, pp. 91–96. [22] S. Benedetto and G. Montorsi, “Design of parallel concatenated convolutional codes,” IEEE Trans. Commun., vol. 44, pp. 591–600, May 1996. [23] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “Serial concatenation of interleaved codes: performance analysis, design, and iterative decoding,” IEEE Trans. Inform. Theory, vol. 44, no. 3, pp. 909–926, May 1998. [24] D. Divsalar, S. Dolinar, and F. Pollara, “Iterative turbo decoder analysis based on density evolution,” IEEE J. Select. Areas Commun., vol. 19, no. 5, pp. 891–907, May 2001. [25] A. Abrardo, “Performance bounds and codes design criteria for channel decoding with a-priori information,” IEEE Trans. Wireless Commun., vol. 8, 2009, to appear.