Improved bounds on the word error probability of RA(2) codes with

Improved bounds on the word error probability of RA(2) codes with linear programming based decoding∗ Guy Even†

Nissim Halabi‡

October 7, 2003

Abstract This paper deals with the linear programming based decoding algorithm of Feldman and Karger [FK02] for Repeat-Accumulate “Turbo-like” codes. We present a new structural characterization that captures the event that decoding fails. Based on this structural characterization, we develop polynomial algorithms that, given an RA(2) code, compute upper and lower bounds on the word error probability P w for the binary symmetric channel and the AWGN channel. Our experiments with an implementation of these algorithms for bounding Pw demonstrate in many interesting cases an improvement in the upper bound on the word error probability by a factor of over 1000 compared to the bounds by Feldman et al. [FK02, FKW02, F03a, F03b]. The experiments also indicate that the improvement in upper bound increases as the codeword length increases and the channel noise decreases. The computed lower bounds on the word error probability in our experiments are roughly 10 times smaller than the upper bound.

Keywords: Turbo Codes; Repeat-Accumulate Codes; Linear-Programming Based Decoding; Efficient Algorithms for Computing Lower and Upper Bounds.

A preliminary version of this paper appeared in the Forty-First Annual Allerton Conference On Communication, Control, And Computing, 2003. † Dept. of Electrical Engineering, Tel-Aviv University, Tel-Aviv 69978, Israel. E-mail:[email protected]. ‡ Dept. of Electrical Engineering, Tel-Aviv University, Tel-Aviv 69978, Israel. E-mail:[email protected]. ∗

1

1 Introduction Turbo codes achieve reliable communication at data rates that are close to the channel capacity and are amenable to practical encoding and decoding [BGT93]. However, there are very few proven theorems that guarantee the empirical results. The first such theorem was proven by Divsalar et al. [DJM98]. They considered Maximum Likelihood (ML) decoding of randomly generated Repeat-Accumulate RA(q) “Turbo-like” codes. They proved, for q > 2, that if the noise in the channel is under a certain constant threshold, then the word error probability is asymptotically bounded by an inverse polynomial in the code length. However, random generation of codes does not point to any specific code with such performance. Moreover, ML decoding of linear codes is NP-Complete [BMT78]. Recently Feldman and Karger [FK02] introduced a breakthrough in the research of Turbo codes. They presented a novel decoding algorithm for RA(q) codes (for every q ≥ 2) called RALP that is based on linear programming. For RA(2) codes, they proved that under the binary symmetric channel (BSC) with a certain constant threshold bound on the noise, the word error probability of RALP is bounded by an inverse polynomial in the code length. A similar claim was also shown for the Additive White Gaussian Noise (AWGN) channel [F03a]. They improved previous results in two important respects: (i) RALP is a provably polynomial time decoding algorithm; when successful RALP provides a certificate that it agrees with ML decoding. (ii) They presented a deterministic polynomial algorithm for constructing RA(2) codes, for every codeword length. The constructed codes achieve the bounds on the word error probability, and, in particular, they do not rely on a random interleaver. We briefly outline the analysis of the RALP decoding algorithm for RA(2) in [FK02] (more details appear in the next section). An auxiliary graph is attached to the code. Edges in the auxiliary graph are assigned costs in {−1, 0, 1} according to the errors introduced by the BSC1 . The analysis considers closed walks, called promenades. The cost of a promenade is the sum of the costs of 1

In an AWGN channel the edge costs are Gaussian random variables.

1

the edges traversed by the promenade. Feldman and Karger [FK02] proved that if RALP decoding of an RA(2) code fails, then there exists a non-positive cost promenade in the auxiliary graph. The coding theorem for RA(2) codes is based on bounding the probability that a non-positive cost promenade exists. Our first contribution is a new structural theorem that characterizes the event that RALP fails. This characterization states that if RALP fails, then there exists a tree-of-cycles subgraph in the auxiliary graph such that every Eulerian tour induced by this tree-of-cycles has non-positive cost. We refer to such Eulerian tours as skeleton promenades. Apart from providing a neat characterization of RALP decoding errors, we employ the structural theorem to compute improved bounds on the word error probability (Pw ). We develop polynomial time algorithms that, given an RA(2) code, compute upper and lower bounds for Pw with respect to the BSC and the AWGN channel. We outline the techniques we use for computing the bounds for P w . Instead of applying a union bound on the probability of a family of events that includes the event that RALP fails, we apply our structural characterization of RALP’s failure. We focus on non-positive cost skeleton promenades and distinguish between long and short such promenades. For the long promenades under the BSC, we bound the probability that RALP fails due to such promenades using a recursive procedure similar to the tree bound of Feldman et al. [FKW02, F03b]. For the long promenades under the AWGN channel, we bound the probability that RALP fails due to such promenades by applying a union bound on the probability of a reduced family of events that includes the event that RALP fails due to such promenades. Our structural theorem implies that the short promenades must be simple cycles. Hence we enumerate all the short cycles. For cycles whose length is far from the girth, we apply the union bound. However, for cycles whose length (nearly) equals the girth, we apply bounds based on the inclusion-exclusion principle. This approach enables us to compute far better upper bounds on Pw as well as a lower bound. We implemented our algorithm for bounding the word error probability of RA(2) codes using RALP decoding. Our experiments show an improvement compared to the bounds of [FK02, FKW02, F03a, F03b] that increases as (i) the codeword length increases and (ii) the crossover 2

probability decreases. In many interesting cases, the improvement is by a factor of over 1000. Moreover, in the case of the BSC our experiments indicate a small gap between the upper and lower bounds. In the case of the AWGN channel, our experiments indicate a small gap between the upper and lower bounds for high SNR. We point out that our algorithm works also for very small values of Pw ; such small values cannot be estimated using standard simulations. Organization. In Section 2, we introduce graph terminology and define RA(2) codes. In addition, RALP decoding and the coding theorem of Feldman and Karger are reviewed. In Section 3, we prove the structural result that states that RALP decoding fails only if there exists a tree-ofcycles in the auxiliary graph in which every Eulerian tour has non-positive cost (Theorem 23). In Section 4, we present algorithms for computing upper and lower bounds on P w . We conclude in Section 5 with empirical results.

2 Preliminaries Graph terminology. Let G = (V, E) denote an undirected graph with edge costs c[e]. A walk W (of length k) in a graph G is a non-empty alternating sequence

W = hv0 , e0 , v1 , e1 , . . . , vk−1 , ek−1 , vk i of vertices and edges in G such that ei = (vi , vi+1 ) for all i < k. If v0 = vk , the walk is closed. A path is a walk with no repeated vertex. A cycle is a closed walk where the only repeated vertex is P the first and last vertex. The cost c[W ] of a walk W is defined by c[W ] = k−1 i=0 c[ei ]. We refer a directed edge (arc) by e, and by e¯ to the corresponding undirected edge. Given a

walk W , we denote by E(W ) the multi-set consisting of undirected edges in W . Namely:

E(W ) = {e¯i : ei ∈ W }.

3

Note that two instances of the same edge in E(W ) may be traversed in opposite directions by W . We denote by G(W ) the subgraph of G induced by E(W ) (In G(W ) we count edges only once). Let A and B be two multi-sets. Inclusion of multisets, A ⊆ B, means that, for every element x ∈ A, the multiplicity of x in A is not greater than the multiplicity of x in B. Definition 1 ([FK02]). A promenade M in a graph G is a closed walk such that the same edge does not appear twice in a row, i.e., e¯i 6= e¯i+1 (mod |W |) , for every i. For simplicity, we often refer to a promenade M as a sequence of vertices (and ignore the edges). Also note that a promenade is a closed walk and therefore it can be cyclicly rotated. We identify the first and last occurrence of v0 = vk in a promenade M as one occurrence. Let A and B denote two walks in a graph G such that there is an edge between the last vertex of A and the first vertex of B. The walk obtained by concatenating the walks A and B is denoted by A ◦ B. Definition 2 (sub-promenade). Let M be a promenade in a graph G. Let hvi , ei , . . . , vj , ej i denote a segment in promenade M . Consider a sequence of ` segments in M :

M0 = hvi0 , ei0 , . . . , vj0 , ej0 i .. .

M`−1 = hvi`−1 , ei`−1 , . . . , vj`−1 , ej`−1 i where i0

j0 < i 1

j1 < . . . < i`−1

Assume that for every 0

j`−1 .

p < `, the last edge ejp of segment Mp satisfies: (1) ejp ends in the first

vertex of segment Mp+1(mod `) (i.e., ejp = (vjp , vip+1(mod `) )), and (2) e−1 jp is not the first edge along segment Mp+1(mod `) (i.e., e¯jp 6= e¯ip+1(mod `) ). Then we can concatenate the segments to obtain a promenade M 0 = M0 ◦ M1 ◦ · · · ◦ M`−1 . Such a promenade M 0 is a sub-promenade of M . RA(2) codes.

Repeat-accumulate codes were defined by Divsalar et al. [DJM98] as examples

of “turbo-like” codes with proven word error probability for random interleavers with respect to 4

maximum likelihood decoding. Following Feldman and Karger [FK02], we focus on RA(2) codes. In an RA(2) code the block size is n = 2k, where k is the information word length. An RA(2) encoder uses the following steps: (i) repeat every information bit twice, (ii) send the n-bit string through an interleaver that permutes the bits, and finally (iii) send the permuted string through an accumulator that outputs the parity of the substring seen so far. RALP decoding.

Feldman and Karger [FK02] attach an auxiliary graph G = (V, E) to an

RA(2) code C as follows. The vertex set V is simply {v1 , . . . , vn }, where vi represents the ith bit in a codeword. The edges are of two kinds: (i) Hamiltonian edges (v i , vi+1 ), for every 1 ≤ i < n, and (ii) matching edges (vi , vj ), where i and j are images of the same information bit position with respect to the interleaver (i.e., π −1 (j) = π −1 (i) + 1 and π −1 (j) is even, where π : [1, n] → [1, n] denotes the permutation implemented by the interleaver). In the BSC the edges of the auxiliary graph are assigned cost c[e] ∈ {−1, 0, 1} as follows: Matching edges always have zero cost. The Hamiltonian edge (v i , vi+1 ) is assigned cost 1 if vi is not flipped by the channel; otherwise it is assigned cost (−1). In the AWGN channel we assume a binary antipodal modulation (e.g. BPSK) in which two √ √ possible signals are s0 = −s1 = Es = R · Eb , where Es is the energy per symbol, R is the √ code rate, and Eb is the energy per information bit. The received signal is y = ± Es + φ where φ ∼ N (0, σ 2 ) is a zero-mean Gaussian random variable with variance σ 2 = 12 N0 . We consider Eb N0

the case of a unit energy per symbol, hence the auxiliary graph are assigned cost c[e] ∈


1 = 10 log10 ( 2Rσ 2 ) [dB]. In such case, the edges of

as follows: Matching edges always have zero cost.

The Hamiltonian edge ei = (vi , vi+1 ) is assigned cost c[ei ] = 1 + φi where φi ∼ N (0, σ 2 ) is a zero-mean Gaussian random variable with variance σ 2 . Feldman and Karger [FK02] introduced a linear programming based decoding algorithm called RALP. When the RALP decoder succeeds it returns the original information word. Moreover, whenever the RALP decoder succeeds, it also provides a proof (called a certificate) that the decoded word agrees with ML decoding. Hence, an upper bound on the probability that the RALP 5

decoder fails is also an upper bound on the probability that ML decoding fails. Feldman and Karger proved the following theorem: Theorem 3 ([FK02]). The RALP decoder for RA(2) codes succeeds if all promenades in G have positive cost. The RALP decoder fails if there is a promenade in G with negative cost. The bounds on the RALP decoder are therefore based on bounding the probability that a nonpositive cost promenade exists. Promenades with many Hamiltonian edges are unlikely to have non-positive cost. Matching edges do not share an endpoint, hence at least every second edge in a promenade is an Hamiltonian edge. Since every promenade contains at least one simple cycle, promenades in graphs with high girth have many Hamiltonian edges. The topic of constructing small cubic graphs with large girth was studied by Erd˝os and Sachs [ES63] and Sauer [S67] (see also Biggs [B98]). These constructions are of cubic graphs with girth g ' log n. An efficient O(n3 ) algorithm for constructing cubic graphs with girth g 

1 2

log n

was presented by Bazzi et al. [BMMS01]. These cubic graphs consist of a path and a matching, and therefore, induce an RA(2) code. Feldman and Karger [FK02] suggested using the above deterministic techniques for constructing specific auxiliary graphs with logarithmic girth. The RA(2) codes that correspond to such auxiliary graphs satisfy/realize the proven bounds. Note that the girth of every cubic graph is bounded by O(log n) [ES63, S67, B98]. Given an RA(2) code, the existence of non-positive cost promenade depends only on the error word introduced by the channel. A naive method for analyzing the word error probability P w is based on considering all the error words with a certain bounded weight. This method is not polynomial since we need to deal with error words with weight bigger than or equal to 41 log n. We suggest a method for computing a lower bound on Pw that, in effect, considers all the error words with weight less than or equal to 14 log n in polynomial time. Feldman and Karger [FK02] proved that the error probability of the RALP decoder is bounded by an inverse polynomial in the code length. They analyzed the probability that there exists a nonpositive cost promenade using bounds on the number of simple paths containing 6

1 2

log n Hamilto-

nian edges in the auxiliary graph that corresponds to the RA(2) code. We suggest a more precise characterization and present methods for computing better bounds for P w .

3 Non-Positive Cost Minimal Promenades In this section we introduce a special class of promenades which we call Non-Positive Cost Minimal Promenades (NPCM-promenades, in short). We also introduce a class of sub-graphs which we call skeleton graphs. The skeleton graphs of G induce a particular set of promenades in G called skeleton promenades. Our main structural theorem is a characterization of all non-positive cost minimal promenades as skeleton promenades (see Theorem 23). Note that the results described in this section do not rely on any assumptions on the properties of the graph G (e.g. degree, structure etc.) or on the weight function c : E → .


NPCM-promenades.

We study minimal promenades since if there exists a non-positive cost

promenade, then there must exist such a minimal promenade. Definition 4 (NPCM-promenade). A promenade M is a non-positive cost minimal promenade if: (i) c[M ] c[M 0 ]

0, and (ii) there is no promenade M 0 in G which satisfies E(M 0 ) 

E(M ) and

0.

Note that the requirement E(M 0 ) 

E(M ) refers to containment of multi-sets.

We often refer to proper sub-promenades M 0 of M , and mean in addition that E(M 0 ) 

E(M ).

Note that it is possible for M 0 to be a promenade with E(M 0 ) ⊆ E(M ) without M 0 being a subpromenade of M (i.e., M 0 traverses edges in M in the opposite direction). Lemma 5 (alternating directions). Let M be an NPCM-promenade. Assume there exists an edge e¯ = (v0 , v1 ) that appears at least twice in E(M ). Then every two consecutive traversals of the edge e¯ in M are in opposite directions. Proof. Let M denote an NPCM-promenade, and let e¯ denote an edge in E(M ). Suppose that two consecutive traversals of e¯ in M are in the same direction. We decompose M as follows: 7

v1 v0

M1

e

M0

u00 6= v1

M0

M 00

v0 v1

u0 6= v1

e

M2 Figure 1: Decomposing the promenade in proof of Lemma 5. Note that the segments between consecutive appearances of the edge e are non-empty.

M = M0 ◦ hv0 , v1 i ◦ M1 ◦ hv0 , v1 i ◦ M2 where (v0 , v1 ) ∈ / E(M1 ) (see Figure 1). Consider the following proper sub-promenades of M : M 0 = hv0 , v1 i ◦ M1 and M 00 = M0 ◦ hv0 , v1 i ◦ M2 . M 0 and M 00 are sub-promenades of M because the predecessor of the occurrences of v0 in M 0 and M 00 (denoted by u0 and u00 respectively in Figure 1) are not v1 . Moreover, c[M ] = c[M 0 ] + c[M 00 ]

0. Thus, M 0 or M 00 is a non-positive cost sub-promenade of M , contradicting

our assumption on M . Corollary 6. Every edge in an NPCM-promenade appears once or an even number of times in the promenade. Proof. If there exists an edge e¯ with odd multiplicity bigger than one in an NPCM-promenade M , then M traverses e¯ twice in a row in the same direction, in contradiction to Lemma 5. Lemma 7 (edge multiplicity ≤ 2). The multiplicity of every edge in an NPCM-promenade is at most two. Proof. From Corollary 6 it suffices to deal with edges of even multiplicity. Assume there exists an edge e¯ = (v0 , v1 ) whose multiplicity in an NPCM-promenade M is 2i, i 

2. Consider four

consecutive appearances of edge e¯ (see Figure 2). Without loss of generality, the first edge in 8

M1 v1

v1

v0 e

e−1

v0

M0

u00 6= v1

M2

M4

M 00

v0

v0 e

−1

v1

v1

u0 6= v1

e

M3 Figure 2: Decomposing the promenade in proof of Lemma 7.

M is e = (v0 , v1 ). Lemma 5 implies that these four appearances of e¯ subdivide M as follows: M = hv0 , v1 i ◦ M1 ◦ hv1 , v0 i ◦ M2 ◦ hv0 , v1 i ◦ M3 ◦ hv1 , v0 i ◦ M4 . We decompose M as follows: M = M 0 ◦ M 00 where M 0 = hv0 , v1 i ◦ M1 ◦ hv1 , v0 i ◦ M2 and M 00 = hv0 , v1 i ◦ M3 ◦ hv1 , v0 i ◦ M4 . Note that M 0 and M 00 are proper sub-promenades of M . At least one of them is of non-positive cost, a contradiction. Lemma 8 (no “8”-decomposition). Assume that vertex v appears at least twice in an NPCMpromenade M . Let u and w denote the predecessor and successor, respectively, of one occurrence of v. Similarly, let x and y denote the predecessor and successor, respectively, of another occurrence of v. Then u, w, x and y are not all distinct. Proof. Consider an NPCM-promenade M . Assume there exists a decomposition of M such that M = M0 ◦ hu, v, wi ◦ M1 ◦ hx, v, yi ◦ M2 where u, w, x & y are all distinct (see Figure 3). Then M 0 = M0 ◦ hu, v, yi ◦ M2 and M 00 = M1 ◦ hx, v, wi are two proper sub-promenades of M where c[M ] = c[M 0 ] + c[M 00 ]

0. Thus, M 0 or M 00 is a non-positive cost sub-promenade of M ,

contradicting M minimality.

9

M0

u

w v

M2

y

M1 x

Figure 3: Decomposing a non simple promenade in proof of Lemma 8 and Corollary 9.

Corollary 9. Consider an NPCM-promenade M . If M is not (vertex) simple, then there is an edge with multiplicity two in E(M ). Proof. Assume v appears more than once in M . Consider two consecutive occurrences of vertex v in M . We decompose M as follows: M = M0 ◦ hu, v, wi ◦ M1 ◦ hx, v, yi ◦ M2 where v ∈ / M1 (see Figure 3). It follows from Lemma 8 that u, w, x & y are not all distinct. If, for example, u = y, then the edge (u, v) = (y, v) appears twice in E(M ). Same holds for other cases. Consider an edge e¯ = (v0 , v1 ) with multiplicity two in E(M ) where M is an NPCM-promenade. Lemma 5 states that these two appearances of e¯ have opposite directions in M . We say (arbitrarily) that edge e¯ = (v0 , v1 ) subdivides M into two segments: (1) right segment which is the segment of M between the two occurrences of the vertex v1 , and (2) left segment which is the segment of M between the two occurrences of the vertex v0 . (see Figure 4 (a)). Lemma 10 (disjoint “ears”). If e¯ = (v0 , v1 ) appears twice in E(M ) and M is an NPCMpromenade, then e¯ subdivides M into two disjoint segments. Proof. Assume that M is an NPCM-promenade. Denote by M 0 and M 00 the right segment and the left segment, respectively, induced by e¯ = (v0 , v1 ) (see Figure 4 (a)). Without loss of generality, v0 is the first vertex in M . We subdivide M as follows: M = hv0 , v1 i ◦ M 0 ◦ hv1 , v0 i ◦ M 00 . Assume that M 0 and M 00 share a vertex v. Let pred(v, M 0 ) = u, i.e., u is the predecessor of v in M 0 (= v1 if v is first in M 0 ), and let succ(v, M 0 ) = w, i.e., w is the successor of v in M 0 (= v1 if v is last in M 0 ). Also let pred(v, M 00 ) = x (= v0 if v is first in M 00 ), and let succ(v, M 00 ) = y (= v0 if v is last 10

in M 00 ). We subdivide M as follows: M = hv0 , v1 i ◦ M0 ◦ hvi ◦ M1 ◦ hv1 , v0 i ◦ M2 ◦ hvi ◦ M3 (see Figure 4 (b)). Note that at least one of the segments in each of the pairs {M 0 , M1 } and {M2 , M3 } is non-empty (for example, if M0 = ∅ then v1 = u, and then v1 6= w, therefore M1 6= ∅). Lemma 8 states that u, w, x & y cannot be all distinct. Since M is a promenade, u 6= w and x 6= y. If u = x then the multiplicity of the edge (u, v) = (x, v) is two. By Lemma 7 there cannot be more occurrences of this edge. Moreover, this edge is traversed twice in the same direction along M , contradicting Lemma 5. It follows that u 6= x and similarly w 6= y. ˜ = hvi◦M −1 ◦hv1 , v0 i◦M2 and M ˆ = hvi◦M3 ◦hv0 , v1 i◦M −1 where Consider the two walks: M 0 1 ˜ and the segment Mi−1 is a reverse walk on segment Mi . Since u 6= x and w 6= y, both walks M ˆ are promenades. Moreover, E(M ) = E(M ˜ ) ∪ E(M ˆ ) and therefore c[M ] = c[M ˜ ] + c[M ˆ] M ˜] Thus, c[M

ˆ] 0 or c[M

0.

0, contradicting our assumption on M . Therefore, M 0 and M 00 cannot

share a vertex v, and the claim follows.

Skeleton graphs.

A skeleton graph is a subgraph that has the structure of a “tree of cycles”. We

are interested in Eulerian tours induced by trees of cycles, and name them skeleton promenades. Our main structural theorem characterizes all NPCM-promenades as skeleton promenades. We first define a finger graph that has the structure of a “lollipop” as illustrated in Figure 5. Then we introduce two equivalent definitions of skeleton graphs. Finally, we define the skeleton promenade. Figure 6 depicts a skeleton and an induced promenade. Definition 11 (finger). A finger is a graph obtained by connecting a simple path to a simple cycle (see Figure 5). We distinguish between two types of edges on a finger: (1) A path-edge is an edge which belongs to the path of a finger, and (2) A cycle-edge is an edge which belongs to the cycle of a finger. Definition 12 (skeleton). A skeleton is defined recursively as follows: [Base] A simple cycle is a skeleton. [Recursion step] The following graph is a skeleton: A graph consisting of a skeleton S 11

v1 v0

e

M0 Right Segment M 00 Left Segment v1 v0

e−1

(a) Subdivision of promenade M induced by edge e. x

w

M1

v M2

y

u

M3

M0 e v0

v1 M 0 = M0 ◦ hvi ◦ M1

M 00 = M2 ◦ hvi ◦ M3

(b) Subdivision of promenade M where the right segment and the left segment induced by edge e, share a vertex v. Figure 4: Decomposing the promenade in proof of Lemma 10.

cycle edge path edge

Figure 5: A finger.

12

and a finger F that share a single vertex v s.t. (1) degF (v) = 1, (2) degS (v) = 2, and (3) v is a vertex in one of the cycles of skeleton S. A finger in a skeleton is a leaf finger if none of the vertices of the finger’s cycle is shared with other fingers. Notice that compressing each cycle C in a skeleton graph into a single node, produces a tree. Hence, we often refer to a skeleton graph as a tree-of-cycles. Lemma 13. If S is a skeleton graph and C is a simple cycle in S, then it is possible to construct S starting from C as a basis. Proof. Since S is a tree-of-cycles, we may “root” this tree at C, and then proceed with the recursive construction. Lemma 14. A graph S is a skeleton graph iff it satisfies one of the following: (i) S is a simple cycle, or (ii) one can decompose S into two disjoint skeletons S1 , S2 and a simple path P such that P shares a single vertex vi with each skeleton Si , and (1) degP (v1 ) = degP (v2 ) = 1, (2) degS1 (v1 ) = degS2 (v2 ) = 2, and (3) vi is a vertex in one of the cycles of skeleton Si for i ∈ {1, 2}. Proof. Every skeleton graph satisfies one of the two conditions. If the skeleton is a simple cycle, then it satisfies condition (i). Otherwise, we translate the recursion step of Definition 12 to follow condition (ii). The translation is obtained by splitting the finger F into a cycle S 2 and a path P . The path P is used in condition (ii) as the “bridge” between S and S2 . We now prove that every graph that satisfies one of the conditions is a skeleton graph. If the graph satisfies condition (i), then it is a skeleton graph. Otherwise, the graph satisfies condition (ii). Denote by Ci the cycle in skeleton Si that shares a vertex with the path P for i ∈ {1, 2}. From Lemma 13 it follows that we can construct skeleton Si starting with Ci as the basis. We show a construction of the graph S according to Definition 12: (1) Construct skeleton S 1 , (2) Connect the leaf of a finger consisting of P ∪ C2 to the cycle C1 in skeleton S1 , and (3) Complete the construction of skeleton S2 . 13

leaf finger

Figure 6: A skeleton graph (tree of cycles) and a skeleton promenade depicted by a dashed line. A skeleton graph G0 with ` cycles induces 2` different skeleton promenades.

Definition 15 (skeleton promenade). A skeleton promenade M S induced by the skeleton S, is a closed Eulerian walk obtained by doubling every path-edge in the skeleton S (see Figure 6).

Bone-decomposition.

We now identify a special decomposition of a promenade that resembles

a contour of a bone. Figure 7 is depiction of the following two definitions. Definition 16 (bone-decomposition). A decomposition of a promenade M into three walks {M 0 , M 00 , Wuw } is called a bone-decomposition of M if it satisfies: (1) M 0 is sub-promenade of M such that the first vertex u in M 0 is of multiplicity one in M 0 . (2) M 00 is sub-promenade of M such that the first vertex w in M 00 is of multiplicity one in M 00 . (3) There exists a (vertex) simple path Puw between −1 vertices u and w in M such that Wuw = Puw ◦ Puw , i.e., Wuw is a closed Eulerian path obtained

by doubling every edge in a simple path between vertices u and w in M . (4) M 0 and M 00 are vertex disjoint, M 0 and Wuw are vertex disjoint except for vertex u, and M 00 and Wuw are vertex disjoint except for vertex w. (see Figure 7) Definition 17 (induced bone-decomposition). Consider an edge e ∈ M . Assume there exists a bone-decomposition {M 0 , M 00 , Wuw } of M such that e ∈ Wuw . Then, we say that the edge e 14

”branching” edge

y

”epiphysis”

M

00

a

e

v1

v0

w

”diaphysis”

x

”epiphysis”

u

Wuw

b

M0

Figure 7: A bone-decomposition induced by edge e.

induces a bone-decomposition of M . We refer to the decomposition {M 0 , M 00 , Wuw } as the bonedecomposition induced by e, and denote it by BeM . Observation 18. The bone-decomposition induced by an edge e is unique. In addition, the same bone-decomposition {M 0 , M 00 , Wuw } of M is induced by all edges e ∈ Wuw . Note that if e induces a bone-decomposition {M 0 , M 00 , Wuw } of M , then obviously e¯ is of multiplicity two in E(Wuw ), and since M 0 , M 00 , Wuw are pairwise edge disjoint, e¯ is also of multiplicity two in E(M ). Hence, we often refer to the bone-decomposition induced by both edges e and e−1 as the bone-decomposition induced by edge e¯. Consider a bone-decomposition {M 0 , M 00 , Wuw } of M induced by edge e¯. Call Wuw the diaphysis of the bone-decomposition. Notice that the underlying simple path of the diaphysis W uw can consist of edge e¯ alone. Call M 0 and M 00 the epiphyses of the bone-decomposition. Denote the edges incident to u in M 0 by (u, a) and (b, u). Similarly, denote the edges incident to w in M 00 by (w, x) and (y, w). The edges (u, a), (b, u), (w, x) and (y, w) are called branching edges, since they are the first branches along M starting from e¯. Notice that each of the branching edges (u, a) and (b, u) are of multiplicity one in E(M 0 ) since u is of multiplicity one in M 0 . By symmetry, each of the branching edges (w, x) and (y, w) are of multiplicity one in E(M 00 ). Thus, all the branching edges are of multiplicity one in E(M ). Lemma 19 (bone-decomposition of NPCM-promenade). Assume M is an NPCM-promenade. Then every edge e¯ with multiplicity two in E(M ) induces a bone-decomposition of M . Proof. Consider an NPCM-promenade M . Let e¯ = (v0 , v1 ) be an edge with multiplicity two in 15

E(M ). Lemmas 5 and 10 imply that the two appearances of e¯ subdivide M as follows: M = M0 ◦ hv0 , v1 i ◦ M1 ◦ hv1 , v0 i ◦ M2 where M1 and M2 ◦ M0 are the right segment and the left segment induced by edge e¯, respectively (see Figure 8 (a)). Define a walk Wv1 u as follows: Consider the extended segment M1e of M1 that includes vertex v1 . Namely, M1e = hu0 , u1 , . . . , uk−1 , uk i where u0 = uk = v1 and hu1 , . . . , uk−1 i = M1 . Let i = max{i | ∀j

i : uj = uk−j }. Denote the last common vertex ui by u. Then, Wv1 u =

hu0 , u1 , . . . , ui = u, uk−i+1 , . . . , uk i (see Figure 8 (b)). Such a vertex u must exist since vi 6= vi+2 mod|M | for all 0

i

|M | − 1. Notice that Wv1 u = Pv1 u ◦ Pv−1 where Pv1 u is the directed 1u

path between vertices v1 and u in M1e . We show that Pv1 u is simple. Assume Pv1 u is not simple, i.e., there exists a vertex v with multiplicity greater than one in Pv1 u . Consider two consecutive occurrences of vertex v in Pv1 u . We distinguish between three cases: (i) If v 6= v1 and v 6= u, then let (q, v), (v, r), (s, v) and (v, t) denote the edges incident to v in these two occurrences of v. Each one of these edges are of multiplicity two in E(Wv1 u ), and therefore is traversed twice in Wv1 u (i.e., back and forth). Hence, Lemma 7 implies that these edges are of multiplicity two in E(M ) since E(W v1 u ) 

E(M ). Since

Lemma 7 bounds the multiplicity of these edges by two, it also implies that q, r, s & t are all distinct, thus contradicting Lemma 8. Hence v cannot be an internal vertex in P v1 u . (ii) If v = v1 , then let (v, r), (s, v) and (v, t) denote the edges incident to v in these two occurrences of v. By the same reasoning as in the previous case, these edges are of multiplicity two in E(M ). Consider also the edge (v0 , v1 ). Lemma 7 implies that r, s, t & v0 are all distinct, thus contradicting Lemma 8. (iii) If v = u, then let (r, v), (v, s) and (t, v) denote the edges incident to v in these two occurrences of v. By the same reasoning used above we conclude that these edges are of multiplicity two in E(M ). Consider also the branching edge (u, a). Lemma 7 implies that r, s, t & a are all distinct, thus contradicting Lemma 8. A contradiction appears in every possible case, and therefore we conclude that Pv1 u is simple. Let M 0 denote the remaining part of M1e after removing Wv1 u . Notice that M 0 = hu, a, . . . , b, ui ⊆ M1e and a 6= b, thus M 0 is proper sub-promenade of M , not necessarily simple. Obviously 16

c[M 0 ] > 0, otherwise M 0 is a non-positive cost sub-promenade of M contradicting our assumption on M . We show that the vertex u must be of multiplicity two in M (one occurrence on the diaphysis and one occurrence on the epiphysis). Note that we already found four edges incident to u in M . Hence, the multiplicity of u ∈ M is at least two. Assume that vertex u is of multiplicity greater than two. Since the path Pv1 u is simple, by Lemma 10 we conclude that all occurrences of vertex u are in M 0 excluding the endpoint of Pv1 u which is shared with M 0 . Therefore, we can ˜ ◦M ˆ where M ˜ = hu, ai ◦ Mau and M ˆ = Mub ◦ hb, ui (see decompose M 0 as follows: M 0 = M ˜ and M ˆ are edge disjoint, c[M 0 ] = c[M ˜ ] + c[M ˆ ] > 0. Therefore, either figure 8 (c)). Since M ˜ ] > 0 or c[M ˆ ] > 0. By chopping off a cycle with positive cost, we get a non-positive cost c[M sub-promenade of M , contradicting our assumption on M . Thus, vertex u must be of multiplicity two in M and of multiplicity one in the sub-promenade M 0 . Lemma 10 states that the right and left segments induced by edge e¯ are vertex disjoint. By symmetry of the left segment M2 ◦ M0 , we can easily identify the bone-decomposition induced by edge e¯, and the claim follows. Since NPCM-promenades have no non-positive cost sub-promenades, we cannot use induction directly to prove properties of NPCM-promenades. In order to be able to use induction, we need to eliminate the non-positive cost requirement, and simultaneously preserve the critical properties that differentiate between general promenades and NPCM-promenades. The next property and the following theorem summarize an hereditary property that eliminates the necessity of the nonpositive cost requirement, while preserving an important unique structure satisfied by all NPCMpromenades. Definition 20 (bone-structure property). A promenade M satisfies the bone-structure property if: (1) The multiplicity of every edge e¯ ∈ E(M ) is at most two, and (2) Every edge e¯ ∈ E(M ) of multiplicity two induces a bone-decomposition of M . Lemma 21 (sub-promenade’s bone-structure). Assume that a promenade M satisfies the bonestructure property. Then every sub-promenade M 0 of M also satisfies the bone-structure property. 17

v1 v0

M1

e

M0 v1 v0

e−1

M2 (a) Subdivision of promenade M by edge e.

M2 ◦ M 0

u

v1

v0

a

M1

e

Wv1 u

M0 b

(b) The bone structure induced by edge e.

M1

e

v0

v1

a

Mau

M0

u b

Mub

(c) Subdivision of epiphysis M 0 Figure 8: Focus on repeated edge in an NPCM-promenade in the proof of Lemma 19.

18

Proof. Let M be a promenade that satisfies the bone-structure property. Consider a sub-promenade ˜ of M . The multiplicity of every edge e¯ ∈ E(M ˜ ) is at most two, since E(M ˜ ) ⊆ E(M ). It M ˜ ) with multiplicity two induces a bone-decomposition remains to show that every edge e¯ ∈ E(M ˜ . Let e¯ = (v0 , v1 ) denote an edge with multiplicity two in E(M ˜ ). Obviously, e¯ is also of of M multiplicity two in E(M ). Therefore, e¯ induces a bone-decomposition B e¯M = {M 0 , M 00 , Wuw } of M . Note that e¯ ∈ E(Wuw ). ˜ , M 00 ∩ M ˜ , Wuw } is the bone-decomposition of M ˜ induced by the We show that {M 0 ∩ M edge e¯. The bone-decomposition Be¯M guarantees that M 0 and M 00 are disjoint. Hence, we can ˜ induced by e¯. Since M ˜ is deal with them separately. Consider the left or right segment of M ˜ is sub-promenade of a promenade, the segment must contain a branching point. Moreover, M ˜ starting from e¯ are the branching edges of the boneM , and therefore the first branches along M ˜ must contain the diaphysis Wuw and all the branching decomposition Be¯M . We conclude that M ˜ is a promenade, thus we conclude that edges of the bone-decomposition Be¯M . Note again that M ˜ , M 00 ∩ M ˜ , Wuw ∩ M ˜ = Wuw } is the bone-decomposition of M ˜ induced by e¯, and the {M 0 ∩ M claim follows. Theorem 22 (sufficient and necessary conditions for skeleton promenades). A promenade M is a skeleton promenade iff (i) M satisfies the bone-structure property, and (ii) There is no decomposition of M = M0 ◦ hu, v, wi ◦ M1 ◦ hx, v, yi ◦ M2 such that u, w, x & y are all distinct (no ”8”-decomposition). Proof. The skeleton obviously satisfies the two conditions. We show the reversed direction. Let M be a promenade that satisfies the bone-structure property. Also assume that there is no decomposition of M = M0 ◦ hu, v, wi ◦ M1 ◦ hx, v, yi ◦ M2 such that u, w, x & y are all distinct. Denote by DM the set of edges with multiplicity two in E(M ), then every edge e¯ ∈ D M induces a bone-decomposition of M . Moreover, both appearances of every edge e¯ ∈ D M in E(M ) are contained in the diaphysis of the bone-decomposition it induces. Obviously, edges that belong to the same diaphysis induce the same bone-decomposition of M . We consider the partitioning 19

of DM into subsets of edges that contained in the same diaphysis. We omit the edges in each diaphysis and replace each subset with an edge connecting the endpoints of the corresponding c c diaphysis. Denote the contracted set of edges by DM . Every edge in DM corresponds to a bone-

decomposition and vice versa. c We prove by induction on the number of diaphyses (i.e., |D M |) that M is a skeleton promec nade. The induction basis is the case where DM = ∅ and therefore DM = ∅. Since there is no

decomposition of M = M0 ◦ hu, v, wi ◦ M1 ◦ hx, v, yi ◦ M2 such that u, w, x & y are all distinct, M is a simple cycle and obviously M is a skeleton promenade. The induction hypothesis c states that the claim holds for a promenade M such that |DM | < n. We proceed to the inducc c tion step. Assume |DM | = n. Consider an edge e¯ = (u, w) ∈ DM , and let {M 0 , M 00 , Wuw }

denote the corresponding bone-decomposition induced by the edges in diaphysis W uw . The sets c c c c Lemma 21 implies that M 0 satisfies the DM 0 and DM 00 satisfy |DM 0 | < n and |DM 00 | < n.

bone-structure property. Since M 0 is sub-promenade of M , then there is no decomposition of M 0 = M00 ◦ hu, v, wi ◦ M10 ◦ hx, v, yi ◦ M20 such that u, w, x & y are all distinct. M 00 is analogous to M 0 , therefore, we can apply the induction hypothesis to both promenades M 0 and M 00 . Denote by S 0 and S 00 the underlying skeleton graphs of the skeleton promenades M 0 and M 00 respectively. The branching edges in the bone-decomposition of M that corresponds to e¯ are cycle-edges, since they are of multiplicity one in E(M ). Notice that the branching edges belong to the skeleton promenades M 0 and M 00 . Therefore, the endpoint u of Wuw belongs to one of the cycles of skeleton S 0 and degS 0 (u) = 2. Similarly, vertex w belongs to one of the cycles of skeleton S 00 and degS 00 (w) = 2. According to Lemma 14, the graph S that consists of skeletons S 0 and S 00 , and the simple path G(Wuw ) is a skeleton graph. Clearly, M is a closed Eulerian path induced by the skeleton graph S, which is obtained by implanting the diaphysis W uw between vertices u and w of the skeleton promenades M 0 and M 00 . We may conclude that M is a skeleton promenade. We are now ready to state our main structural theorem that trivially follows Lemmas 7, 8, 19 and 22.

20

Theorem 23. Every non-positive cost minimal promenade in G is a skeleton promenade. Theorem 23 has several corollaries; we mention two below. Corollary 24. If S is an underlying skeleton graph of an NPCM-promenade, then 1. The cost of every cycle in S is positive. 2. The cost of a skeleton walk induced by every subtree hanging from one of the cycles in S is negative. I.e., every bone-decomposition {M 0 , M 00 , Wuw } of M satisfies: (i)

c[M 0 ] + c[Wuw ] < 0

and (ii) c[M 00 ] + c[Wuw ] < 0.

3. Every leaf finger F in S satisfies c[M S\F ] > 0. Namely, the number of finger leaves in S is minimal so that c[M S ]

0.

Corollary 25. The multiplicity of every vertex in an NPCM-promenade is at most two.

4 Bounding the word error probability In this section we present polynomial algorithms that, given an RA(2) code, compute upper and lower bounds on Pw . The correctness of the algorithms is based on the characterization of nonpositive cost minimal promenades as skeleton promenades. Our experiments show that in the BSC further improvement of the upper bound is limited since the upper bound and lower bound for P w are close. In the AWGN channel further improvement is limited for high SNR. We describe the algorithms and their results with respect to both the BSC and the AWGN channel. Theorem 3 implies that an upper bound on the probability that G contains a promenade with cost less then or equal to zero, is also an upper bound on the probability that the RALP decoder fails. Our goal is to prove an upper bound on the probability that a non-positive cost promenade 21

exists. Since G is cyclic, there are infinitely many promenades (e.g. one may traverse a cycle an arbitrary number of times). Feldman and Karger [FK02] suggested to bypass the problem of infinitely many promenades by bounding the number of non-positive cost simple paths containing b 2g c Hamiltonian edges. This helps because if a non-positive cost promenade exists, then there exists a non-positive simple path containing b g2 c Hamiltonian edges. We propose a different approach for proving an upper bound. This approach is based on the structure of NPCM-promenades formulated in Theorem 23. Recall that there is an NPCMpromenade in G iff there is a non-positive cost promenade in G. From Theorem 23 it follows that in order to bound the probability that the RALP decoder fails, it is sufficient to bound the probability that there is a skeleton promenade in G with non-positive cost. Since the multiplicity of every edge and vertex in a skeleton promenade is at most two, it follows that the number of skeleton promenades in a graph G is finite. Unfortunately, the number of skeleton promenades is exponential. We therefore resort to algorithms that bound the number of skeleton promenades in graphs corresponding to specific codes rather than general bounds. Consider an auxiliary graph G with girth g that corresponds to an RA(2) code C. We consider two cases: 1. There is an NPCM-promenade M in G such that |M | < 2g + 2. 2. There is an NPCM-promenade M in G such that |M | 

2g + 2.

We bound separately the probability that each of these two cases occur. The sum of these probabilities is an upper bound on Pw of the RA(2) code C.

4.1 Short NPCM-Promenades We would like to prove an upper bound on the probability that an NPCM-promenade M of length at most 2g + 1 exists in G. We denote this probability by Pshort . We refer to a promenade as short if it contains at most 2g + 1 edges.

22

Short promenades have a few Hamiltonian edges, and therefore can be non-positive even if few errors occur in the channel. Hence, short promenades can be “easily” made non-positive and have a significant influence on Pw . For this reason, rough estimates on Pshort do not suffice. Claim 26. Pshort equals the probability that there exists a simple short cycle C in G with nonpositive cost. Proof. By Theorem 23 every non-positive cost minimal short promenade M in G is a skeleton promenade. If a skeleton promenade is not a simple cycle, then its length is at least 2g + 2 since it contains at least 2 cycles and a bridge that is traversed on both directions. Let Cycles[h] denote the number of short simple cycles in G with h Hamiltonian edges. Namely,  Cycles[h] = simple cycles C ∈ G : ham(C) = h and |C|

2g + 1 .

Consider a simple cycle C with h Hamiltonian edges. In a BSC, a simple cycle has non-positive cost if at least half of its Hamiltonian edges have negative cost. Note that the costs of edges in C are i.i.d. since C is simple. Therefore,

P r{c[C]

h   X h i 0} = p (1 − p)h−i . i h

(1)

i=d 2 e

In an AWGN channel, a simple cycle has non-positive cost if the sum of its edges costs is nonpositive. Hence,

P r{c[C]

0} = P r{

h−1 X

(1 + φi )

= P r{

h−1 X

φi

0}

(2)

i=0

i=0

−h}

(3)

The sum of independent Gaussian random variables (RVs) with zero mean is a zero-mean 23

Gaussian RV whose variance equals the sum of the variances of the summed variables. Let Φ = Ph−1 2 2 i=0 φi , then Φ ∼ N (0, σ h) is a zero-mean Gaussian RV with variance σ h. Moreover, the

Gaussian distribution function (PDF) is symmetric around 0. Therefore,

P r{c[C]

0} = P r{Φ = P r{Φ 

−h}

(4)

h}

(5)

For a Gaussian RV φ ∼ N (0, σ 2 ) with zero mean and variance σ 2 , the double inequality x2 σ − σ 3 − x22 √ e 2σ x3 2π

P r{φ 

x}

x2 σ √ e− 2σ2 x 2π

(6)

holds for every x > 0 [Fel68]. We conclude that

P r{c[C]

0}

and P r{c[C]

0}

h σ e− 2σ2 2πh hσ − σ 3 − h2 √ e 2σ 2πh3

(7)

√ 

(8)

Since no two matching edges share an endpoint, at least one of every two consecutive edges in a cycle C is Hamiltonian. Therefore, Cycles[h] = 0 for every h, 0

h < d g2 e − 1. By applying a

union bound over all short cycles in G (using Equations 1 or 7), we conclude that 2g+1

Pshort

X

h=d g2 e

Cycles[h] · P r{c[C]

0 : ham(C) = h}.

We now tighten this bound. Note that for small crossover probabilities in a BSC (analogously, high SNR in an AWGN channel), the probability of a cycle having non-positive cost diminishes as the number of Hamiltonian edges in the cycle increases. Therefore, in order to tighten the bound on Pshort we distinguish between the following cases:

24

1. F ewHams: Short cycles C ∈ G that satisfy either ham(C) = b 2g c or ham(C) = b 2g c + 1. Note that for graphs with odd girth Cycles[b 2g c] = 0. Let Pshort0 denote the probability that there exists a cycle C ∈ F ewHams with non-positive cost. 2. M anyHams: Short cycles that satisfy ham(C) 

b g2 c+2. Let Pshort00 denote the probability

that there exists a cycle C ∈ M anyHams with non-positive cost. We bound separately Pshort0 and Pshort00 . Clearly, Pshort Pshort 

Pshort0 + Pshort00 . Note that Pw 

Pshort0 .

Bounds for Pshort0 . Cluster the set F ewHams as follows: (i) Create an auxiliary graph Γ = (VΓ , EΓ ) where VΓ = {vC : C ∈ F ewHams} and EΓ = {(vC1 , vC2 ) : ∃e ∈ E(C1 ) ∩ E(C2 )}. (ii) Decompose VΓ into connected components {Ui } of Γ. (iii) Define a list of cycles per connected component Ui of Γ. We denote by Ki the list corresponding to Ui . We often refer to Ki as a cluster. K Let Pshort 0 denote the probability that there exists a cycle with non-positive cost in cluster K. Ki Since no two cycles of different clusters share an edge, the probabilities {P short 0 } are independent.

Therefore, Pshort0 = 1 −

Y

clusters K

(9)

K (1 − Pshort 0)

K For a BSC we compute both upper and lower bounds on Pshort 0 for every cluster K using

Bonferroni inequalities [J01] with terms that contain products of at most three terms (i.e., inclusionexclusion with only the first terms). K We present an exact expression for Pshort 0 with respect to an AWGN channel. Assume that the

cycles in cluster K share k different edges. Then,

K Pshort 0

=

Z

∞ e1 =−∞

···

Z

∞ ek =−∞

k Y i=1

Y
f (ei ) 1 − (1 − P r{Φ C∈K

−


where I = {e1 , . . . , ek }, Φ ∼ N 0, σ 2 · ham(E(C)\I) , and f (x) = 25

X

ei ∈E(C)∩I


ei }) de1 · · · dek ,

x2

√ 1 e− 2σ 2 . 2πσ

K However, we would like to compute upper and lower bounds for P short 0 while avoiding calcu-

lations of such integrals. Therefore, we apply the union bound over all cycles in cluster K (using K K Equation 7) to obtain an upper bound on Pshort 0 . In order to obtain a lower bound on P short0 , we

choose a cycle C with the fewest Hamiltonian edges in cluster K. The probability that this cycle K has a non-positive cost (using Equation 8) is a lower bound on P short 0 . By substituting the above K upper and lower bounds for Pshort 0 in Equation 9, we obtain upper and lower bounds for P short0 ,

respectively. A bound for Pshort00 .

Cycles that are “abundant” with Hamiltonian edges do not require such a

precise estimate as the cycles in F ewHams. The following two steps are used for computing an upper bound on Pshort00 : (i) compute Cycles[h] for every h, b 2g c + 2

h < 2g + 1. (ii) apply a

union bound (using Equations 1 or 7) to compute an upper bound on P short00 . Time and space complexity. We first construct codes whose auxiliary graphs have large girth. In our experiments we used a modification of Bazzi et al. [BMMS01] to obtain cubic graphs with girth g ' log n. As described in the preliminaries, we assume that the auxiliary graph corresponding to the RA(2) code has girth g = log2 n. Matching edges do not share an endpoint, and therefore there are at most

|VG |·22g+1 2g

3

n = O( log ) n

cycles with length at most 2g + 1. We enumerate all the simple cycles in G with length at most l in O(n · 2l ) time and O(l) space. Therefore, computing Cycles[h] for every h, d 2g e

h < 2g + 1,

requires at most polynomial time and logarithmic space in n. When bounding Pshort0 in the case of a BSC we use Bonferroni inequalities with terms that contain products of at most three terms. That is, we need to compute for every three cycles in a cluster K the probability that each of these cycles has non-positive cost. A simple computation of this probability is by considering all cost assignments to edges that appear in at least two of the three cycles. Given a cost assignment to these joint edges, we can easily compute the probability that the three cycles have non-positive cost. Every cycle C ∈ K satisfies ham(C) 26

b g2 c + 1.

Therefore, the maximum size of a union of all pairwise intersections of three cycles in a cluster K of F ewHams is at most

3g . 4

Hence, the computation of dependent binomial probabilities for every

three cycles in a cluster K requires at most O(n3/4 log n) time. By using a table for every cluster K, one can reduce the running time for computing the dependent binomial probabilities for every three cycles in a cluster K to O(n3/4 ). In addition, |K|

|F ewHams| = O(n ·

3g/2 ) 2g

= O( n

1+(log 3)/2

log n

).

Clustering F ewHams can be done using a DFS, and therefore takes O(|F ewHams|) time. Therefore, the time required for the computation of the third term in Bonferroni inequalities for a cluster K of F ewHams is at most 

 max (C0 ∩C1 )∪(C0 ∩C2 )∪(C1 ∩C2 ) :distinct C0 ,C1 ,C2 ∈K |K| ·2 3 



3

|K|3 · 2 4 g .

We conclude that the computation of the first three terms (at most) in Bonferroni inequalities requires at most O( n

(15+6 log 3)/4

log3 n

) time and O(n1+log 3/2 ) space.

Notice that computing the bounds presented for Pshort0 in the case of the AWGN channel requires clustering the set F ewHams, and therefore, O( n

1+(log 3)/2

log n

) time and space.

In our experiments |F ewHams| and in particular its clusters are very small and the intersections between cycles in the clusters are even smaller (e.g., when n = 1024, we had |F ewHams| = 16 in which the maximum cluster size was 2 and the maximum number of edges that appeared in two cycles in a cluster equaled 2). Therefore, in our experiments the upper and lower bounds for Pshort0 are very close. Moreover, we could even compute Pshort0 precisely very quickly in the case of the BSC.

4.2 Long NPCM-Promenades We would like to prove an upper bound on the probability that a non-positive cost minimal promenade M of length at least 2g + 2 exists in G. We denote this probability by P long . In spite of the characterization of all non-positive cost minimal promenades as skeleton promenades, we do not have small bounds on the number of such long skeleton promenades. We refer 27

to a segment of a skeleton promenade as skeleton walk. Instead of bounding P long by enumerating long skeleton promenades, we apply a reduction to the existence of a non-positive cost skeleton walk that traverses g + 1 Hamiltonian edges in G (with repetitions). The following lemma follows [FK02, Lemma 3] and Theorem 23. Lemma 27. Consider a graph G with girth g > 3. If there exists a non-positive cost minimal promenade M in G where |M | c[W ]

2g + 2, then there exists a skeleton walk W in G with cost 

0 that contains g + 1 Hamiltonian edges (with repetitions).

Let Plong0 denote the probability that there exists a skeleton walk W such that c[W ] ham(W ) = g + 1. Lemma 27 implies that Plong

0 and

Plong0 . We now focus on bounding Plong0 .

Consider a skeleton walk W with g + 1 Hamiltonian edges (with repetitions). Let E i denote the set of edges with multiplicity i in E(W ) for i ∈ {1, 2}. Notice that |E 1 | + 2|E 2 | = g + 1. In a BSC, W has non-positive cost if at least half of the occurrences of Hamiltonian edges have negative cost. Clearly, distinct edges in W have i.i.d. costs. Therefore,

P r{c[W ]

X

0} = P r{

c[e] + 2

e∈E 1

X

=

e d g+1 2 |E 1 |

a+2b 0 a 0 b |E 2 | 









X

c[e]

(10)

0}

e∈E 2

  |E 1 | |E 2 | a+b 1 2 p (1 − p)|E |+|E |−a−b b a

(11)



In an AWGN channel, W has non-positive cost if the sum of its edges costs (with repetitions) is non-positive. Hence,

P r{c[W ]

0} = P r{

X

(1 + φe ) + 2

e∈E 1

= P r{

X

φe +

e∈E 1

X

e∈E 2

X

(1 + φe )

0}

(12)

e∈E 2

2φe

−(g + 1)}

(13)

An α-scaled RV of a zero-mean Gaussian RV is a zero-mean Gaussian RV whose variance equals to the original variance multiplied by the square of the scaling factor α. Also, the sum 28

of independent zero-mean Gaussian RVs is a zero-mean Gaussian RV whose variance equals to P P the sum of the variances of the summed variables. Let Φ = e∈E 1 φe + e∈E 2 2φe , then Φ ∼
N 0, (|E 1 |+4|E 2 |)σ 2 is a Gaussian RV with zero mean and variance (|E 1 |+4|E 2 |)σ 2 . Therefore, P r{c[W ]

0} = P r{Φ = P r{Φ 

−(g + 1)}

(14)

(g + 1)}

(15)

From Equations 6 and 15 it follows that

P r{c[W ] A simple bound on Plong0 :

0}

σ

p (g+1)2 |E 1 | + 4|E 2 | − 2σ2 (|E 1 |+4|E 2 |) √ e (g + 1) 2π

(16)

One could compute all skeleton walks in G with exactly g + 1 Hamil-

tonian edges (with repetitions). For each skeleton walk, compute the probability of having a nonpositive cost (using Equations 11 or 16). Apply a union bound over all counted skeleton walks to obtain an upper bound on Plong0 . Since it is not clear how to efficiently compute all skeleton walks with g +1 Hamiltonian edges, we suggest computing a set of walks that contains the above set. Consider a skeleton walk W that starts at vertex v. In the first step, we may assume without loss of generality that W starts with an Hamiltonian edge. Since the maximum degree in G is three, a sub walk can be extended by at most two edges (recall that a skeleton walk may not traverse e−1 immediately after e). f This motivates the following definition. We say that a walk W is an h-extension of a walk W

if it satisfies the following conditions:

f , i.e., W starts at the last vertex of W f, 1. W is an extension of the walk W

f ◦ W does not traverse an edge twice in the same direction, 2. W f ◦ W does not repeat a vertex more than twice, and 3. W 4. ham(W ) = h.

29

f ) as follows: Define SW (h, W f) = SW (h, W Notice that Plong0

P

v∈VG

X

P r{c[W ]

0}.

W : h-extension of W 

SW (g + 1, hvi) since every non-positive cost skeleton walk with

g + 1 Hamiltonian edges is a (g + 1)-extension of a walk consisting of a single vertex. We present a f ). The algorithm performs a search similar recursive algorithm that bounds the quantity SW (h, W

f. to DFS for every h-extension of W

We define few notations used in Algorithm 1: • Let Adj(w) denote the set of neighbors of w. f denote a walk that does not repeat a vertex more than twice and does not traverse an • Let W edge twice in the same direction.

f ) denote the last vertex in W f. • Let last(W

f ) denote the predecessor of w in W f (if w is the first vertex in W f, then • Let pred(w, W f ) = N U LL). pred(w, W

f ) denote the multiplicity of w in W f. • Let mult(w, W

f for i ∈ {1, 2}. • E i denotes the number of edges with multiplicity i in W

Note that SW (g + 1, hvi) = ComputeSW (g + 1, 0, 0, hvi). We are interested in computing

the quantity SW (g + 1, hvi) for every v ∈ VG where g is the girth of the graph G. Recall that the girth of every cubic graph is bounded by O(log n) [ES63, S67, B98]. We refer to the case where g ' log n. There are at most 3g+1 = O(nlog 3 ) walks W that start at a given vertex v and satisfy Ham(W ) = g + 1. Thus, the algorithm ComputeSW computes this quantity in polynomial time and logarithmic space in n. This technique is applicable both to the BSC and the AWGN channel. We focus on a tighter bound for the BSC. 30

f ) - compute probabilities of all h-extensions of W f Algorithm 1 ComputeSW (h, E 1 , E 2 , W 1: if h = 0 then return P r{c[W ] 0} f) 2: w ← last(W f) 3: u ← pred(w, W 4: r ← 0 f, and (iii) mult(v, W f ) 1 do 5: for all v ∈ Adj(w) s.t. (i) v 6= u, (ii) w → v ∈ /W f then 6: if v → w ∈ W 7: if (w, v) is Hamiltonian edge then f ◦ hvi) 8: r ← r + ComputeSW (h − 1, E 1 − 1, E 2 + 1, W 9: else {(w, v) is a matching edge} f ◦ hvi) 10: r ← r + ComputeSW (h, E 1 , E 2 , W 11: end if f} 12: else {v → w 6∈ W 13: if (w, v) is a Hamiltonian edge then f ◦ hvi) 14: r ← r + ComputeSW (h − 1, E 1 + 1, E 2 , W 15: else {(w, v) is a matching edge} f ) 6= ∅ then 16: if E(W f ◦ hvi) 17: r ← r + ComputeSW (h, E 1 , E 2 , W 18: end if 19: end if 20: end if 21: end for 22: return r

31

A tighter bound on Plong0 for the case of BSC:

We compute a tighter bound based on the tree

bound of Feldman, Karger and Wainwright [FKW02, F03b]. Let Plong0 (v) denote the probability that there exists a skeleton walk W in G that satisfies: (i) W starts at vertex v, (ii) c[W ]

0, and (iii) Ham(W ) = g + 1 (with repetitions). We apply the union

bound on all vertices in G as starting points for the skeleton walks, to obtain an upper bound on P Plong0 , i.e., Plong0 v∈VG Plong 0 (v) .

f ) denote the probability that a walk W in G exists, where the walk W satisfies Let SE(d, k, W

the following conditions:

1. W is a d-pseudo skeleton extension, and 2. W contains at least k occurrences of negative cost Hamiltonian edges. Notice that Plong0 (v)

e, hvi) since every non-positive cost skeleton walk with SE(g + 1, d g+1 2

g + 1 Hamiltonian edges satisfies the above conditions. We present a recursive algorithm that f ) given a distinct cost assignment to edges in W f . The algorithm computes the quantity SE(d, k, W

f with all possible cost assignments performs a search similar to DFS for every legal extension of W

to the extension.

In addition to the notation used in Algorithm 1, we use the following notation in Algorithm 2: • Let p denote the crossover probability of the channel. f. • Let ∆ denote a cost assignment’s vector corresponding to edges in W

We are interested in computing the quantity SE(g + 1, d g+1 e, hvi) for every v ∈ VG where g is 2

the girth of the graph G. We refer to the case where g ' log n. There are at most 3 g+1 = O(nlog 3 ) walks W that start at a given vertex v and satisfy Ham(W ) = g + 1. For each of these walks there are at most 2g+1 = O(n) distinct edge cost assignments. Thus, the algorithm ComputeSE computes this quantity in polynomial time and logarithmic space in n.

32

f , ∆) - compute the probability that W f is “extendible” Algorithm 2 ComputeSE(d, k, W 1: if k > 0 then return 0 f with d Hamiltonian edges exists} 2: if k = 0 then return 1 {assuming a legal extension W of W f) 3: w ← last(W f) 4: u ← pred(w, W 5: r ← 0 f, and (iii) mult(v, W f ) 1 do 6: for all v ∈ Adj(w) s.t. (i) v 6= u, (ii) w → v ∈ /W f then 7: if v → w ∈ W 8: case ∆(v → w) is f ◦ hvi, ∆ ◦ h−1i) {c[(w, v)] = −1} 9: −1 : r ← r + ComputeSE(d − 1, k − 1, W f ◦ hvi, ∆ ◦ h+1i) 10: +1 : r ← r + ComputeSE(d − 1, k, W {c[(w, v)] = +1} f 11: 0 : r ← r + ComputeSE(d, k, W ◦ hvi, ∆ ◦ h0i) {(w, v) is not Hamiltonian} f} 12: else {v → w 6∈ W 13: if (w, v) is a Hamiltonian edge then f ◦ hvi, ∆ ◦ h−1i) 14: r ← r + p · ComputeSE(d − 1, k − 1, W f ◦ hvi, ∆ ◦ h+1i) 15: +(1 − p) · ComputeSE(d − 1, k, W 16: else {(w, v) is a matching edge} f ◦ hvi, ∆ ◦ h0i) 17: r ← r + ComputeSE(d, k, W 18: end if 19: end if 20: end for 21: return r

33

Since Plong0 (v)

SE(g + 1, d g+1 e, hvi) for every v ∈ VG , summing up the quantity SE(g + 2

1, d g+1 e, hvi) for every v ∈ VG is also an upper bound on Plong0 . Thus, 2 Plong0

X

v∈VG

SE(g + 1, d

g+1 e, hvi). 2

5 Experimental Results In this section, we present experimental results of our algorithms for bounding P w with respect to both the BSC and the AWGN channel. These experiments include constructing specific RA(2) codes. Good codes have auxiliary graphs with a large girth. The algorithm of Bazzi et. al. [BMMS01] constructs cubic graphs with a girth that equals

1 2

· log n. We modified this algorithm to construct

auxiliary cubic graphs whose girth nearly equals log n. Using this modified algorithm, we constructed auxiliary graphs corresponding to codes with codeword length n ∈ {128, 256, 512, 1024} (see Tables 1 and 2). For each codeword length we constructed 25 − 50 different auxiliary graphs and picked the graph with the lowest upper bound on Pw . Feldman and Karger [FK02, F03a] presented an upper bound for P w called the path-bound for the cases of the BSC and the AWGN channel. Feldman et al. [FKW02, F03b] also presented a bound for the case of the BSC called the tree-bound. The tree-bound (which is computed by running a recursive procedure) outperforms the path-bound (which is an analytic bound). Both bounds depend only on n (and not on the specific graph). We suggest a modification of the treebound, called the exp-tree-bound; this bound is similar to the tree-bound but applies to a specific code. Our experiments show that the improvement offered by exp-tree-bound compared to the tree-bound is by a factor of approximately 10−0.2 . In Tables 1 and 2 and in Figures 9, 10, 11 and 12 we present our experimental results. We refer to our upper bounds of Pw as the skeleton-bound. Note that the improvement of the skeleton-bound in the case of the BSC increases as n increases and p decreases. Analogously, the improvement of

34

n(g) p skeleton-bound Pw Lower Bound Plong Upper Bound Pshort Upper Bound tree-bound [FKW02] exp-tree-bound [FKW02] n(g) p skeleton-bound Pw Lower Bound Plong Upper Bound Pshort Upper Bound tree-bound [FKW02] exp-tree-bound [FKW02]

−2 −0.51 −3.22 −0.52 −2.63 −0.09 −0.28 −2 −0.48 −5.00 −0.48 −2.82 No Bound No Bound

128(7) −3 −4 −3.91 −6.80 −5.22 −7.22 −3.93 −7.01 −5.12 −7.21 −2.09 −4.09 −2.27 −4.27 512(9) −3 −4 −5.09 −8.82 −8.00 −11.00 −5.16 −9.37 −5.95 −8.96 −0.80 −2.80 −0.98 −2.98

−5 −9.15 −9.22 −10.02 −9.22 −6.09 −6.27

−2 −1.46 −4.00 −1.48 −2.75 No Bound No Bound

−5 −11.95 −14.00 −13.40 −11.97 −4.80 −4.98

−2 −1.42 −3.53 −1.43 −3.13 No Bound No Bound

256(8) −3 −4 −5.61 −8.83 −7.00 −10.00 −6.00 −10.11 −5.85 −8.85 −1.10 −3.10 −1.28 −3.28 1024(10) −3 −4 −6.39 −9.51 −6.52 −9.52 −7.19 −12.49 −6.47 −9.51 −2.75 −5.75 −2.93 −5.93

−5 −11.85 −13.00 −14.13 −11.86 −5.10 −5.28 −5 −12.52 −12.52 −17.53 −12.52 −8.75 −8.93

Table 1: Bounds on Pw for selected RA(2) codes in the case of a BSC. All the quantities are in logarithmic scale (log10 ). n(g) Eb /N0 [dB] skeleton-bound Pw Lower Bound Plong Upper Bound Pshort Upper Bound path-bound [FK02] n(g) Eb /N0 [dB] skeleton-bound Pw Lower Bound Plong Upper Bound Pshort Upper Bound path-bound [FK02]

5 −0.80 −3.74 −0.80 −2.73 No Bound 5 −0.71 −4.47 −0.71 −3.10 No Bound

128(7) 6 7 −2.20 −3.78 −4.50 −5.40 −2.21 −3.82 −3.74 −4.86 −0.23 −1.03 512(9) 6 7 −2.52 −4.61 −5.41 −6.53 −2.53 −4.64 −4.34 −5.72 −0.08 −1.09

8 −5.69 −6.58 −5.84 −6.24 −2.07

5 −0.76 −3.52 −0.76 −2.93 No Bound

8 −7.08 −7.99 −7.32 −7.45 −2.41

5 −0.65 −3.88 −0.65 −3.26 No Bound

256(8) 6 7 −2.37 −4.21 −4.46 −5.57 −2.38 −4.25 −4.06 −5.31 −0.16 −1.06 1024(10) 6 7 −2.66 −4.97 −4.90 −6.09 −2.66 −5.02 −4.54 −5.90 −0.0004 −1.11

8 −6.41 −7.03 −6.59 −6.87 −2.24 8 −7.41 −7.61 −8.04 −7.53 −2.58

Table 2: Bounds on Pw for selected RA(2) codes in the case of an AWGN channel. All the quantities are in logarithmic scale (log10 ). the skeleton-bound in the case of the AWGN channel increases as both n and the SNR increase. We compute also a lower bound on Pw which is that lower bound on Pshort0 . For the tested graphs with 256 and 512 nodes in the BSC, the gap between the upper and lower bounds is roughly 10. For the case of the AWGN channel, in the presence of low SNR there is still a large gap between the upper and lower bounds caused by applying a union bound when dealing with the long promenades. We notice that as the SNR increases this gap diminishes since the portion that affects P w due to long promenades is negligible compared to the portion due to short promenades. The programs used for the experiments are available in [H03].

35

RA(2) LP Based Decoding, n = 256, g = 8

1e0

1e−2

Tree Bound Pshort

1e−6

1e−8

1e−10

1e−12

Skeleton Bound

Upper Bound

Shannon Capacity

Probability of Word Error

1e−4

P

w

Exp. Tree Bound

Lower Bound

Plong

Upper Bound

1e−14

1e−16

1e−18

1e−1

1e−2

1e−3

1e−4

BSC Crossover Probability

1e−5

1e−6

1e−7

Figure 9: Bounds on Pw for an RA(2) code under the BSC. The code corresponds to auxiliary graph of size n = 256 and girth g = 8.

36

RA(2) LP Based Decoding, n = 256, g = 8

1e0

Pshort

Upper Bound

1e−1

Path Bound 1e−2

1e−4

P

Shannon Capacity

Probability of Word Error

1e−3

1e−5 1e−6

w

Lower Bound

Skeleton Bound

1e−7 1e−8

Plong

1e−9 1e−10

Upper Bound 0

1

2

3

4

5

E /N [dB] b

6

7

8

9

10

0

Figure 10: Bounds on Pw for an RA(2) code under the AWGN channel. The code corresponds to auxiliary graph of size n = 256 and girth g = 8.

37

RA(2) LP Based Decoding, n = 1024, g = 10

1e0

1e−2

Pshort

Upper Bound Skeleton Bound

1e−6

P

w

1e−8

1e−10

1e−12

Shannon Capacity

Probability of Word Error

1e−4

Lower Bound

P

Tree Bound

long

Upper Bound

1e−14

Exp. Tree Bound

1e−16

1e−18

1e−1

1e−2

1e−3

1e−4

BSC Crossover Probability

1e−5

1e−6

1e−7

Figure 11: Bounds on Pw for an RA(2) code under the BSC. The code corresponds to auxiliary graph of size n = 1024 and girth g = 10.

38

RA(2) LP Based Decoding, n = 1024, g = 10

1e0

P

short

Upper Bound

1e−1

Path Bound 1e−2

1e−4

P

Shannon Capacity

Probability of Word Error

1e−3

1e−5 1e−6

w

Lower Bound Skeleton Bound

1e−7 1e−8

Plong

Upper Bound

1e−9 1e−10

0

1

2

3

4

5

E /N [dB] b

6

7

8

9

10

0

Figure 12: Bounds on Pw for an RA(2) code under the AWGN channel. The code corresponds to auxiliary graph of size n = 1024 and girth g = 10.

39

Acknowledgments We thank Jon Feldman for continuously sending us versions of his papers and thesis.

References [BMMS01] L. Bazzi, M. Mahdian, S. Miller, and D. Spielman. The minimum distance of turbolike codes. manuscript, 2001. [BMT78] E. R. Berlekamp, R. J. McEliece, and H. C. A. van Tilborg. On the intractability of certain coding problems. IEEE Trans. Inform. Theory, vol. IT-24, pp. 384-386, May 1978. [BGT93] C. Berrou, A. Glavieux, and P. Thitimajashima. Near Shannon limit error-correction coding : Turbo Codes. Proc. IEEE International Conference on Communications (ICC), pp. 1064-1070, Geneva, Switzerland, May 1993. [B98] N. Biggs. Constructions for cubic graphs with large girth. Electronic Journal of Combinatorics, 5(A1), 1998. [DJM98] D. Divsalar, H. Jin, and R. J. McEliece. Coding theorems for ‘turbo-like’ codes. in Proc. 36th Allerton Conf. on Communication, Control, and Computing, pp. 201-210, September 1998. [ES63] P. Erd˝os and H. Sachs. Regul¨are Graphen gegebene Taillenweite mit minimaler Knotenzahl. Natur.Reine 12, 251–257, 1963. [F03a] J. Feldman. Decoding Error-Correcting Codes via Linear Programming. Ph.D. Thesis, Massachusetts Institute of Technology, September 2003. [F03b] J. Feldman, personal communication, 2003.

40

[FK02] J. Feldman and D. R. Karger. Decoding turbo-like codes via linear programming. Proc. 43rd annual IEEE Symposium on Foundations of Computer Science (FOCS), November 2002. [FKW02] J. Feldman, D. R. Karger, and M. J. Wainwright. Linear programming-based decoding of turbo-like codes and its relation to iterative approaches. Proc. 40th Annual Allerton Conference on Comm., Control, and Computing, October 2002. [Fel68] W. Feller. An introduction to probability theory and its applications. 3rd Ed., Vol. I, John Wiley & Sons, Inc., 1968. [H03] N. Halabi, URL: http://www.eng.tau.ac.il/˜nissimh/RA2/ . [J01] S. Jukna. Extremal combinatorics : with applications in computer science. Springer, Berlin, 2001. [S67] N. Sauer. Extremaleigenschaften regul¨arer Graphen gegebener Taillenweite, I & II. ¨ Sitzungsberichte Osterreich. Acad. Wiss. Math. Natur. Kl., S-B II, 176, 9-25, 27-43, 1967.

41