Variations on Gallager's Bounding Techniques: Performance Bounds ...

Variations on Gallager’s Bounding Techniques: Performance Bounds for Turbo Codes in Gaussian and Fading Channels S. Shamai* and I. Sason* *Dept. of Elect. Eng., Technion—Israel Inst. of Technology, Haifa 32000, Israel Tel: (972) 4 8294603, Fax: (972) 4 8323041 E-mail: [email protected], [email protected] Abstract: By generalizing the framework of the second version of the Duman-Salehi bound encompassing also deterministic and random codes, we demonstrate its rather broad features and show that this variation provides the natural bridge between the 1961 and 1965 Gallager bounds. A large class of efficient recent bounds (or their Chernoff versions) is demonstrated to be a special case of the generalized second version of the Duman-Salehi bound. Implications and applications of these observations are pointed out, referring to known bounds as well as a novel extended version of the Shulman-Feder bound. Mismatched decoding metrics are also addressed and some of the results are demonstrated for the fully interleaved fading channel.

1.

INTRODUCTION

Since the error performance of efficiently coded communication systems rarely admits exact expressions, tight analytical bounds emerge as a useful theoretical and engineering tool for assessing performance and for acquiring insight into the effect of the main system parameters. Specific good codes are not easily identified, giving rise to ensembles of codes over which performance should be accurately assessed. The motivation for introducing and applying such bounds has strengthened with the recent introduction of turbo codes and the rediscovery of the low density parity check (LDPC) codes. Clearly the useful bounds must not be subjected to the union bound limitations as the cutoff rate limit for long enough codes, as these families of codes perform considerably beyond the cutoff rate and approach the capacity limit. Useful bounds should also be applicable to either deterministic codes or ensembles of codes relying on basic features such as the distance spectrum or input-output weight enumeration functions (IOWEF) of the examined codes. Among the classical techniques is the Fano-Gallager [11],[12] (referred to as the Fano-Gallager 1961 and the Gallager 1965 [13] bounds. Several such bounds have recently been introduced (see [2]-[12],[16]-[23] and references in [20]). In our work, we focus on the second version of the recently introduced bounds by Duman-Salehi [3],[8],[18],

[19], whose derivation is based on the 1965 Gallager bounding technique [13]. Though originally derived for binary signaling over an additive white Gaussian noise (AWGN) channel, we demonstrate here its rather broad features in a variety of aspects. In our setting, we shall use and build on some of the interesting results in [3], where insightful observations on the Duman-Salehi bounding technique are provided and some efficient and easily applicable bounds are introduced.

2. 2.1.

THE GALLAGER BOUNDS AND DUMAN-SALEHI VARIATION The 1965 Gallager Bound

The maximum likelihood (ML) decoding error probability conditioned on an arbitrary transmitted (lengthN ) codeword xm (Pe|m ), is upper bounded by the 1965 Gallager bound [13]:  Ã !λ ρ m0 X X p (y|x ) N  , pN (y|xm )  Pe|m ≤ m p N (y|x ) 0 y m 6=m

(1) where λ, ρ ≥ 0. Here, y designates the observation vector (with N components) and pN (y|x) is the channel’s transition probability measure. The upper bound (1) is usually not easily evaluated in terms of basic features of particular codes, but for example, orthogonal codes and the special case where ρ = 1 and λ = 12 (yielding the Bhattacharyya-union bound). For ensembles of random codes, with M independent codewords selected with the distribution qN (x), the classical Gallager bound on the average error probability of ML decoding results:  1+ρ X X 1  , Pe ≤ (M − 1)ρ qN (x) pN (y|x) 1+ρ  y

x

(2) where 0 ≤ ρ ≤ 1 and where Pe designates the average decoding error probability (where the average is taken over the randomly and independently selected codewords). For the case of memoryless chanN ´ ³ Y p(yi |xi ) and input distribution nel pN (y|x) = i=1

³

qN (x) =

N Y

i=1

´ q(xi ) , the random coding Gallager

bound (2) admits the form [13]: Pe ≤ e−N ·ERC (R,q) ERC (R, q)

=

E(ρ, R, q)

=

E0 (ρ, q)

Pe ≤ 2h2 (ρ) ·g(s)N ·(1−ρ) ·

max E(ρ, R, q) ,

0≤ρ≤1

E0 (ρ, q) − ρ · R , Ã !1+ρ X X 1 = − ln q(x) p(y|x) 1+ρ , y

x

(3) where R = lnNM is the code rate and ERC (R, q) is the random coding error exponent [13].

2.2.

The 1961 Fano-Gallager Bound

The Gallager 1961 bound on ML decoding [12] relies on the distance spectrum of the N -length block code (or ensemble of codes), and therefore can be also applied to fixed codes or restricted ensembles of codes (differing from the ensemble of the fully random block codes, as in the Gallager’s 1965 bound). The derivation of this bound is based on the Fano’s 1961 bounding technique [11]. The concept of this bounding is as follows: define Let xm be the transmitted codeword ¶ the µ and m f (y) 0 N , where tilted ML metric: D(xm , y) = ln pN (y|xm0 ) 0 m (y) (may dexm is an arbitrary codeword and fN m pend on x ) is an arbitrary function which is posi0 tive if pN (y|xm ) is positive. If the ML decoding is applied, an error occurs if for some m0 6= m: 0

D(xm , y) ≤ D(xm , y) .

where d is an arbitrary real number. The conditioned ML decoding error probability is upper bounded by a sum of two terms:

Pe|m ≤ Prob (y ∈ YbN )+Prob (error , y ∈ YgN ) , (6) being the starting point of many efficient bounds [2],[3],[4],[10],[16],[20] and references therein. We rem strict now our consideration to the case where fN (y) m can be expressed in the product form: fN (y) = N ³ ´ Y f (yi ), where f is an even function f (y) = f (−y) .

Ã

N X l=0

Sl · h(r)l · g(r)N −l

!ρ

,

(7) where 0 ≤ ρ ≤ 1 and h2 (ρ) is the binary entropy function. {Sl }N l=0 designates the distance spectrum of the considered block code. Based on the symmetry of the channel and the function f : g(s)

=

h(r)

=

´ 1 X³ · [p(y|0)]1−s + [p(y|1)]1−s · f (y)s , 2 X y 1−r [p(y|0) · p(y|1)] 2 · f (y)r , y

(8) with s ≥ 0, r ≤ 0, −∞ < d < +∞, 0 ≤ ρ ≤ 1, and where f is optimized to get the tightest upper bound (7).

2.3.

The Duman-Salehi Bound (Version II)

The Duman-Salehi bounding technique [8],[9], originates from the 1965 Gallager bound [13] and it yields:

Pe|m

≤

 

X X ·

1

1

m (y)1− ρ pN (y|xm ) ρ ψN

m0 6=m y

(4)

The received space Y N is divided into two disjoint subspaces: S Y N = YgN YbN , n o YgN = y : D(xm , y) ≤ N d , (5) n o YbN = y : D(xm , y) > N d ,

i=1

For the special case of binary linear block codes and a symmetric memoryless channel, it is demonstrated in [17] by invoking the Chernoff bounds that:

Ã

0

pN (y|xm ) pN (y|xm )

!λ ρ  ,

(9)

m (y) is an where 0 ≤ ρ ≤ 1 and λ ≥ 0, and where ψN arbitrary probability measure (which may also depend on xm ). An alternative form is given by:

Pe|m

·

 1−ρ X m  ≤ Gm N (y) · pN (y|x )

y   X X



m0 6=m

y

pN (y|xm ) · Gm N (y)

1 1− ρ

·

Ã

0

pN (y|xm ) pN (y|xm )

! λ ρ 

(10) where 0 ≤ ρ ≤ 1, λ ≥ 0 and the un-normalized tilting measure Gm N (y) satisfies: m Gm N (y) · pN (y|x ) m . ψN (y) = X m Gm N (y) · pN (y|x ) y

(11)



,

3. 3.1.

INTERCONNECTIONS AND OBSERVATIONS The Duman-Salehi Bound: Random Coding Version

For the ensemble of random codes, where the N length codewords are randomly and independently selected with respect to the input distribution qN (x), the generalized Duman-Salehi bound yields the following upper bound:   X X  qN (x) · pN (y|x)1−λρ  Pe ≤ (M − 1)ρ ·  x y ρ    X  . qN (x0 ) · pN (y|x0 )λ   0 x

(12) with 0 ≤ ρ ≤ 1, λ ≥ 0 and where the optimal selection for the un-normalized tilting measure is:  ρ µ 0 ¶λ X ) p (y|x N   . (13) Gm qN (x0 ) · N (y) = pN (y|xm ) 0 x

1 , one obtains the standard random codFor λ = 1+ρ ing Gallager bound (2). It is hence demonstrated that in the standard random coding setting, no penalty is incurred for invoking the Jensen inequality in the Duman-Salehi (version II) bounding technique, as long as the optimization in (12),(13) is done.

3.2.

The Connection between the Fano-Gallager 1961 Bound with the Duman-Salehi Bounds

Aided by Chernoff inequality, the optimization over the parameter d yields [3]:   ¶s 1−ρ µ m X (y) f N  pN (y|xm ) · Pe|m ≤ 2h2 (ρ) ·  p (y|xm ) N y  !t à 0 X X pN (y|xm ) m · pN (y|x ) · pN (y|xm ) m0 6=m y ³ ´ ρ ¶s 1− ρ1 µ m 0≤ρ≤1, fN (y)  ·  , m m pN (y|x ) t, s ≥ 0 . (14) Divsalar [3] renames t by λ (where t is non-negative parameter and also sets: ¶s µ m fN (y) m , (15) GN (y) = pN (y|xm )

reproducing then the Duman-Salehi upper bound (10) with an additional multiplying factor of 2h2 (ρ) (where

1 ≤ 2h2 (ρ) ≤ 2). This demonstrates the superiority of the Duman-Salehi second version bound over the Fano-Gallager 1961 technique when applied for a particular code or ensemble of codes. It has been demonstrated in [17] that the Fano-Gallager technique equals the 1965 standard random coding Gallager bound (12) up to the 2h2 (ρ) coefficient, where the later as shown before, agrees with the optimized Duman-Salehi bound.

3.3.

A Geometric Interpretation of the Gallager Type Bounds

The connections between the Fano-Gallager tilting measure and the Duman-Salehi normalized and un-normalized tilting measures (which are designated m m (y) and Gm here by fN (y), ψN N (y) respectively) are indicated in Eqs. (11), (15), and they also provide some geometric interpretations of various reported m bounds. The measure fN (y) in the Fano-Gallager 1961 bound, which in general does not imply a product form, yields a geometric interpretation associated with the conditions in the inequalities (5), which specify the disjoint regions YgN , YbN ⊆ Y N . The geometric interpretation of the Fano-Gallager 1961 bound is not necessarily unique. The non-uniqueness emerges due to the shifting and factoring invariance of the inequality µ m ¶ fN (y) ln ≤ Nd (16) pN (y|xm ) and due to fact that the parameter d is also subjected to optimization. We demonstrate this nonuniqueness in [20] focusing on the Divsalar bound [3], which is associated with the decision region (N dimensional sphere with a shifted center) ) ( N X m 2 2 N , (17) (yl − ηγ · xl ) ≤ N · r Yg = y | l=1

q

b where γ = 2RE N0 is the square root of the signal to noise ratio and η, r are parameters subjected to optimization. It is verified [20] that the Fano-Gallager 1961 tilting measures: µ µ ¶ ¶¾ N ½ Y 1 1 m fN (y) = exp · − 1 · (yl )2 ,(18) 2 η

l=1

m fN (y)

=

N n Y

, (19)

³ ´o 2 exp −θ · (yl − φ · xm , l )

(20)

l=1

m (y) fN

=

N n Y l=1

³γ

´o

exp

³ where φ = γ · η +

1−η 2θ

2

´

· (1 − η) · xm l · yl

and θ 6= 0 in (20), yield in

fact all equivalent regions to (17).

4.

GENERALIZATIONS AND SPECIAL CASES

of their second version [8], where the normalized tilting measure in (9) is:

In this section we demonstrate that many reported bounds can be considered as special cases of the generalized (second version) of the Duman-Salehi bound, and we also propose generalizations of the ShulmanFeder bound [21]. Some of these observations have been reported in details in [20].

4.1.

N Y m ψN (y) = ψ(yl ) l=1   !2   Ã r N r  Y α 2E β α s  , = · exp − · yl − · · xm l  2π  2 α N0 l=1

(22) with α, β parameters subjected to optimization. The overall Duman-Salehi bound results by partitioning the code to constant Hamming subcodes and invoking the union bound:

A generalization of the Shulman-Feder bound

In [20] the generalized version of the ShulmanFeder bound [21] is derived as a particular case of the generalized Duman-Salehi bound. Let C be a fixed binary linear block code, where its distance spectrum is designated by {Sl }N l=0 . We get in [20]: ¶ µ ³ ´Q ´ ³P N vl 0 1 1 P −N ·E

Pe|0 ≤ e

ρ , P R+ N · Q ·ln

0 ≤ ρ0 ≤

l=0

bl ·

bl

Pe = Pe|0 ≤

, q= 2

,

1 , P

(21) l , where E(ρ, R, q) is introduced in Eq. (3), vl = MS−1 µ ¶ N , (l = 0, 1, 2 , . . . , N ), P ≥ 1 and bl = 2−N l

1/P + 1/Q = 1. The Shulman-Feder bound [21] is recognized as a special case where P = 1, Q → ∞. The improvement in the tightness of the generalized bound (21) (as compared to the Shulman-Feder bound) is expected to take place for codes whose distance spectrum exponentially deviates from the binomial distribution of the ensemble of fully random block codes, especially at low Hamming weights (e.g, turbo and LDPC codes). For a pure binomial distance spectrum operating over a binary-input and symmetric output channel, P = 1 is optimal and the Shulman-Feder bound matches the Gallager random coding bound. Consider the inequality for the term in the error exponent of (21): N

³X ³ v ´Q ´ 1 1 P l · · ln ≥ · D(v||b) . bl · N Q bl N l=0

The divergence D(v||b) constitutes then a lower bound on the rate loss as compared to pure random codes (with binomial distance spectra) and hence it provides some operative meaning to Battail’s proposition [1] for the design of weakly random-like turbo codes.

4.2.

The Duman-Salehi Bound (First Version)

The first version of the Duman-Salehi bound [9] for a binary-input AWGN channel is a particular case

N X

Pe|0 (d) ,

(23)

d=dmin

where dmin is the minimum Hamming distance of the N -length block code and Pe|0 (d) is the conditioned decoding error probability with respect to the subcode with a constant Hamming weight d. The Duman-Salehi [9] first version bound equals then: ¶− N2ρ α−1 · α− Pe|0 (d) ≤ (Sd ) · α ρ ¶ µ h n REb (β ∗ )2 · (1 − ρ) · −1 + · exp N · N0 α ³ ´2 ¢ ¡ ∗ io ρ · 1 − Nd · β ∗ + 1−β ρ + , (24) α−1 α− ρ N (1−ρ) 2

ρ

1−

µ

d

1 N β ∗ = 1 − d ·(1−ρ) , and where 0 ≤ α ≤ 1−ρ , 0 ≤ ρ ≤ 1, α N are subjected to further optimization.

4.3.

The Viterbi-Viterbi Bound (First Version)

The Viterbi-Viterbi bound [22] on the ML decoding error probability for BPSK modulated block codes operating over a binary-input AWGN channel is given by (23) with: Pe ≤ (Sd )ρ · exp

Ã

−N REb · N0 1−

¡d¢ N d N ·

·ρ

(1 − ρ)

!

,

(25) where 0 ≤ ρ ≤ 1. The optimization over the parameter ρ gives then the Viterbi-Viterbi upper bound [23] which reads: ´ ³ Pe ≤ exp −N · Ev1 (rd ) ,

(26)

where ³

 ¡ ¢ rd d d   N · c − rd , 0 ≤ c ≤ N 1 −       s ³ ´ 2   d rd 1− N  √ Ev1 (rd ) =  ,  c−  d  N     ³ ´    d 1 − d ≤ rd < d/N . N N c 1−d/N rd =

γ2 REb ln(Sd ) , c= = . N 2 N0

d N

´

,

in section 3.3. It has also been indicated there that several alternative geometrical interpretations can be used. This bound given in [3] in closed form reads: Pe

(28)

where

β∗

The Viterbi-Viterbi Bound (Second Version)

³ ´ l ˜ Ev2 (rl ) = max −ρrl + · ln h(ρ) 0≤ρ≤1 N ¶ µ ³ ´ ³ ´o l ˜ · ln g˜(ρ) − (1 − ρ) · ln h(ρ) + g˜(ρ) , + 1− N (29) l) . Based on the symmetry of the where rl = ln(S N binary-input and memoryless channel: n

=

Xh

1

1

p(y|0) 1+ρ + p(−y|0) 1+ρ

y

1 h i 1+ρ · p(y|0) · p(−y|0) ,

g˜(ρ)

=

Xh

p(y|0)

1 1+ρ

+ p(−y|0)

y

2

·p(y|0) 1+ρ .

1 1+ρ

i−(1−ρ)

i−(1−ρ)

(30) This bound is again a special case of the DumanSalehi bound (second version) using the un-normalized tilting measure: Gm N (y)

=

N n³ Y l=1

4.5.

1

1

p(yl |0) 1+ρ + p(yl |1) 1+ρ o ρ ·p(yl |0)− 1+ρ .

´ρ (31)

The Divsalar Bound

The Divsalar bound [3] results also as a particular case of the Fano-Gallager 1961 bounding technique for a binary-input AWGN channel, as demonstrated

h i · ln β + (1 − β) · exp(2 · rd ) ¡d¢ ·β ¡ dN¢ , + 1 − N · (1 − β) (33) " Ã ! 1 − Nd 2 c· · d 1 − exp(−2 · rd ) N 1/2 Ã !2 d h i 1− N (34) + · (1 + c)2 − 1 

ED (c, d, β)

The second version of the Viterbi-Viterbi bounds [23] is based on the Gallager-Fano 1961 bound and it reads: ´ ³ Pe|0 (l) ≤ exp −N · Ev2 (rl ) ,

˜ h(ρ)

d=dmin

³ ³ ´ min exp −N · ED (c, d, β ∗ ) , ´ √ Sd · Q( 2dc) ,

(32)

(27)

This bound turns to be a special case of the DumanSalehi first version bound where α = 1 is substituted in (24) (see [20]).

4.4.

N X

≤

=

=

−rd +

1 2

d N

Ã

1−

d N

!

1 Q(x) = √ · 2π

Z

+∞

−

d N

· (1 + c) , exp

x

µ

−

t2 2

¶

dt ,

(35)

Es d) and the parameters c = N , rd = ln(S N . In fact, it 0 is verified in [20] that the Divsalar bound coincides with the corresponding tilting measure of the first version of the Duman-Salehi bound (22), where: α = (2θ − 1)s + 1, β = (2θ − 1)ηs + 1 and where α and s are properly selected parameters [20]. This is the grounds for the observation in [3] that the Divsalar bound is a closed form version of the Duman-Salehi bound (first version) [9].

4.6.

The Fully Ideally Interleaved Fading Channels with Perfect Channel Side Information (CSI)

The model here is: y = a · x + n, where y stands for the received signal, x stands √ for the BPSK modulated input signal (it is ± 2Es ) and n designates the additive zero mean and N20 variance Gaussian component. The fading a is assumed to be ideally interleaved and perfectly known at the receiver and hence is considered to be real valued as the receiver compensates for any phase rotation. The bounds are based on first decomposing the code to subcodes over which a union bound is applied as in (23). 4.6.1 Optimized Duman-Salehi Bound In [18],[19] the measure: Ψ(y, a) =

N Y l=1

ψ(yl , al ) ,

(36)

is optimized to yield the best possible Duman-Salehi (second version) bound (9), where (y, a) are interpreted as the available measurements at the receiver.

We demonstrate the comparative tightness of several bounds when applied to the repeat-accumulate codes [5] of rate 1/4, operating over the ideally interleaved Rayleigh channel. The system is depicted in Fig4.6.2 Exponential Tilting Measure ure 1. A block of 1024 information bits are encoded In [6], a sub-optimal selection for ψ in (36) is sugin each block, repeated q = 4 times, interleaved by gested which in fact is motivated by the first version a uniform interleaver of length N = 4096 and finally of the Duman-Salehi bound [9], where an exponential differentially encoded. tilting is also applied to the fading sample a (treated A comparison between upper bounds on the bit as a measurement). This gives rise to the following error probability, associated with the ML decoding of exponential tilting measure: the RA coding scheme of Figure 1, operating over a ¸ · q ³ ´ 2 fully interleaved Rayleigh fading channel with perfect pα 2 2 2Es α − αv Na0 Es p(a) side information on the i.i.d fading samples is shown 2π exp − 2 y − au N0 ψ(y, a) = , in Figure 2. The following upper bounds are shown: µ ¶ Z +∞ αv 2 a2 Es The generalized Duman-Salehi second version bound p(a) exp − da N0 0 [18]; the Divsalar-Biglieri bound [4] equivalent also to (37) the bound in [6]; the generalized Engdahl-Zigangirov where α ≥ 0, −∞ < u < +∞, −∞ < v < +∞, and bound [18]; the generalized Viterbi-Viterbi (first verwhere p(a) designates the probability density funcsion) bound [6]; and the union bound. The upper tion of the independent fading samples. That yields bounds on the ML decoding are compared with the a closed form upper bound which reads [6]: computer simulation results of the Log-MAP iterative decoding algorithm with 20 iterations. The relµ ¶− N2ρ −N (1−ρ) α−1 ρ ative tightness of the investigated bounds as well as 2 · α− Pe|0 (d) ≤ (Sd ) · α ρ a marked advantage over the union bound of the opµ ¶N (1−ρ) µ ¶dρ µ ¶(N −d)ρ 1 1 1 timized Duman-Salehi bound [18] is clearly demon· · · , 1+t 1+ε 1+v strated. (38) with: 4.7. The Chernoff version of some t

=

v

=

ε

=

α · ν 2 · Es , N0

(39)

Es h α(u2 + ν 2 ) − 1 · α(u2 + ν 2 ) − N0 ρ ³ ´2 αu−1 i αu − ρ − , α − α−1 ρ α(u2 + ν 2 ) − 1 Es h · α(u2 + ν 2 ) − N0 ρ ³ ´2 αu − αu−1 − 2λ i ρ − . α − α−1 ρ

This bound is in fact equivalent to the DivsalarBiglieri bound [4] which has been derived via a geometric extension of the associated decision region in the Divsalar bound [3] (by rotating the displaced sphere region). The bound in [6] also particularizes to the Viterbi-Viterbi (first version) extended bound to the fading channel [6], which results by setting in (39): p α = 1 , u = 1 − ξρ , ν = 1 − (1 − ξρ)2 , 1 + ξ · (1 − ρ) λ= . 2 (40)

reported bounds: In his paper [3], Divsalar derived simplified Chernoff versions of some upper bounds, which result as particular cases of the Fano-Gallager 1961 bounding technique, and therefore are also special cases of the Duman-Salehi setting. These simplified Chernoff version include: the Chernoff version of the sphere bound ([3],[7]), the Chernoff version of the tangential bound ([2],[3]), the Chernoff version of the tangential sphere bound ([3],[16]). In [20] it is shown that also the Chernoff version of the Engdahl and Zigangirov bound [10] derived for a binary-input AWGN channel, is a particular case of the generalized DumanSalehi bound. The decision region YgN associated with the transmitted codeword xm for the EngdahlZigangirov bound is an N -dimensional plane: ( ) N X YgN = y | yl · x m , (41) l ≥ Nd l=1

where the associated tilting measure is identified to be: m fN (y)

N ½ Y

= q l=1 s β 6= 2E N0 .

µ

1 exp − · yl2 + β · yl · xm l 2

¶¾

,

(42)

5.

THE MISMATCHED DECODING CASE

In this section we use the Duman-Salehi bound for a mismatched decoding metric. The basic setting is as before, that is a codeword x is conveyed via a channel of transition probability pN (y|x). The decoder operates in a maximum likelihood fashion, but may use a mismatched metric QN (y|x). Namely code xj out of the M possible equiprobable codewords is declared if QN (y|xj ) > QN (y|xi ) ,

i = 1, 2 , . . . , M , (43) and ties are randomly resolved. The matched case results when QN (y|x) = pN (y|x). In general, QN (y|x) is not necessarily normalized to yield a probability measure. The Duman-Salehi bound reads now,  X X 1 1 m (y)1− ρ Pe|m ≤  pN (y|xm ) ρ · ψN m0 6=m

∀ i 6= j ,

y

·

Ã

0

QN (y|xm ) QN (y|xm )

(44)

! λ ρ  ,

where λ ≥ 0, 0 ≤ ρ ≤ 1 and where as in section 2.3, m ψN (y) is the Duman-Salehi tilting measure. The advantage of this bound as compared to the original Gallager mismatched bound [15]:  ! λ ρ Ã m0 X X ) Q (y|x N  , Pe|m ≤ pN (y|xm ) ·  QN (y|xm ) 0 y m 6=m

(45) is as in the matched case: the possible utilization for specific codes and ensembles of codes. This is demonstrated in [20] for a fading channel with faulty channel state information. The Gallager bound (45) equaling the optimized m (over ψN (y)) Duman-Salehi bound (44) when applied to a random ensemble is also investigated where the ensembles are appropriately restricted [20]. It is shown in [20] that the technique yields the classical Csiszar-K¨orner-Hui lower bound on the achievable rate: XX RH = sup max q(x) · p(y|x) (46) γ(x) λ≥0



 · ln  X x0

y

x

Qλ (y|x) · eγ(x)

q(x0 ) · Qλ (y|x0 ) · e



  , γ(x )  0

where γ(x) is a real function. This rate is in general advantageous over the generalized mutual information [15] which results by letting γ(x) = 0, and

which governs the performance of the unrestricted ensemble [14].

6.

SUMMARY AND CONCLUSIONS

In addressing the Gallager bounding techniques and their variations, we focus here on the DumanSalehi variation, which originates from the standard 1965 Gallager classical bound [13]. The considered bounds on the block and bit error probabilities rely on the calculable distance spectrum and the inputoutput weight enumerator functions of the codes. By generalizing the framework of the second version of the Duman-Salehi bound (see section 2), we demonstrate its rather broad features and indeed this variation provides the natural bridge between the 1961 and 1965 Gallager bounds ([12] and [13] respectively). It is suitable for both random and specific codes, as well as for either bit or block error analysis. Some observations and interconnections among the Gallager and Duman-Salehi bounds for random and deterministic codes are presented in section 3, which partially rely on the most insightful observations by Divsalar [3]. Focus is put on geometric interpretations of the 1961-Gallager-type bounds as motivated by the Duman-Salehi setting, reflecting the non-uniqueness of their associated tilting measures. We use this unifying framework to generalize the Shulman-Feder bound as well as to demonstrate that this setting accounts for a large class of recently proposed bounds. The Duman-Salehi bound is also examined in the mismatched decoding regime for deterministic and random codes. Full details, extensions and examples are reported in [20].

REFERENCES [1] G. Battail, “A conceptual framework for understanding turbo codes”, Proceedings of the First International Symposium on Turbo Codes and Related Topics, pp. 55–62, Brest, 3–5 September, 1997. [2] E. R. Berlekamp, “The technology of error correction codes”, Proceedings of the IEEE , vol. 68, no. 5, pp. 564–593, May 1980. [3] D. Divsalar, “A simple tight bound on error probability of block codes with application to turbo codes”, TMO Progress Report 42–139, JPL, pp. 1–35, November 1999. [4] D. Divsalar and E. Biglieri, “Upper bounds to error probabilities of coded systems beyond the cutoff rate”, IEEE International Symposium on Information Theory 2000 (ISIT 2000), 25–30 June 2000, Sorrento, Italy. [5] D. Divsalar, H. Jin and R.J. McEliece, “Coding theorems for ‘turbo-like’ codes”, Proceedings of

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

tion Theory, April 2000. [19] I. Sason and S. Shamai, “On improved bounds on coded communications over interleaved fading channels, with application to turbo codes”, this Proceedings. [20] S. Shamai and I. Sason, “Variations on the Gallager bounds, connections and applications”, Technion—Israel Institute of Technology, CC Report No. 316, June 2000. [21] N. Shulman and M. Feder, “Random coding techniques for nonrandom codes”, IEEE Trans. on Information Theory, vol. 45, no. 6, pp. 2101– 2104, September 1999. [22] A. M. Viterbi and A. J. Viterbi, “An improved union bound for binary linear codes on the AWGN channel, with application to turbo decoding”, Proceedings of IEEE Information Theory Workshop, p. 72, San Diego, California, February 1998. [23] A. M. Viterbi and A. J. Viterbi, “Improved union bound on linear codes for the binaryinput AWGN channel, with application to turbo codes”, Proceedings of the IEEE International Symposium on Information Theory (ISIT 1998), Cambridge, MA, p. 29, August 16–21, 1998.

N information bits

a repetition code (q repetitions)

uniform interleaver

differential

of length qN

encoder

qN coded bits

Figure 1: Block diagram of the ensemble of rate- 14 and uniformly interleaved RA (repeat and accumulate) codes. 0

10

3

−1

10

2

−2

10

Bit error probability

[6]

the 1998 Allerton Conference, Monticello, Illinois, pp. 201–210, September 1998. D. Divsalar, I. Sason and S. Shamai, “On simple and tight upper bounds on the ML decoding error probability for block codes operating over interleaved fading channels”, in preparation. S. Dolinar, L. Ekroot and F. Pollara, “Improved error probability bounds for block codes on the Gaussian channel”, Proceedings 1994 IEEE International Symposium on Information Theory, Trondheim, Norway, June 24–July 1, 1994. T.M. Duman, “Turbo codes and turbo coded modulation systems: Analysis and performance bounds”, Ph.D. dissertation, Elect. Comput. Eng. Dep., Northeastern University, Boston, MA USA, May 1998. T.M. Duman and M. Salehi, “New performance bounds for turbo codes”, IEEE Trans. on Communications, vol. 46, no. 6, pp. 717–723, June 1998. K. Engdahl and K. Sh. Zigangirov, “Tighter bounds on the error probability of fixed convolutional codes”, submitted to IEEE Trans. on Information Theory. Available at: ˜ http://www.it.lth.se/karin/artiklar.html. R.M. Fano, Transmission of Information, jointly published by the M.I.T Press and John Wiley & Sons, New York 1961. R.G. Gallager, Low density parity check codes, Cambridge, MA USA, MIT Press, 1963. Available at: http://justice.mit.edu/people/gallager.html. R.G. Gallager, “A simple derivation of the coding theorem and some applications”, IEEE Trans. on Information Theory, vol. 11, no. 1, pp. 3–18, January 1965. A. Ganti, A. Lapidoth and I.E.Telatar, “Mismatched decoding revisited: general alphabets, channels with memory and the wide-band limit”, LIDS Report # 2508, MIT, May 1999. G. Kaplan and S. Shamai, “Information rates and error exponents of compound channels with application to antipodal signaling in a fading environment”, AEU , vol. 47, no. 4, pp. 228–239, 1993. G. Poltyrev, “Bounds on the decoding error probability of binary linear codes via their spectra”, IEEE Trans. on Information Theory, vol. 40, no. 4, pp. 1284–1292, July 1994. I. Sason and S. Shamai, “Gallager’s 1961 bound: extensions and observations”, Technical Report, CC No. 258, Technion, Israel, October 1998. I. Sason and S. Shamai, “On improved bounds on the decoding error probability of block codes over interleaved fading channels, with application to turbo and low density parity check codes”, submitted to IEEE Trans. on Informa-

−3

10

−4

10

−5

10

1

2

3

20 iterations 5

4

−6

10

−7

10

1

1.5

2

2.5

3

3.5

Eb/No [dB]

Figure 2: A comparison between bounds of RA codes for a Rayleigh channel: 1. Generalized DumanSaleh [18]. 2. Divsalar-Biglieri [4],[6]. 3. EngdahlZigangirov [10]. 4. Generalized Viterbi-Viterbi [6]. 5. Union. These bounds are compared to simulation results of the LOG-MAP iterative decoding algorithm (20 iterations).