Error Exponents for Recursive Decoding of Reed–Muller ... - IEEE Xplore

4880

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006

Error Exponents for Recursive Decoding of Reed–Muller Codes on a Binary-Symmetric Channel Marat Burnashev and Ilya Dumer, Senior Member, IEEE

Abstract—Error exponents are studied for recursive decoding of Reed–Muller (RM) codes and their subcodes used on a binary-symmetric channel. The decoding process is first decomposed into similar steps, with one new information bit derived in each step. Multiple recursive additions and multiplications of the randomly corrupted channel outputs 61 are performed using a specific order of these two operations in each step. Recalculated random outputs are compared in terms of their exponential moments. As a result, tight analytical bounds are obtained for decoding error probability of the two recursive algorithms considered in the paper. For both algorithms, the derived error exponents almost coincide with simulation results. Comparison of these bounds with similar bounds for bounded distance decoding and majority decoding shows that recursive decoding can reduce the output error probability of the latter two algorithms by five or more orders of magnitude even on the short block length of 256. It is also proven that the error probability of recursive decoding can be exponentially reduced by eliminating one or a few information bits from the original RM code.

given code rate, this threshold was first derived for majority decoding in [1], where it was shown that as . For the second-order codes , the threshold of majority decoding was increased in [2], by designing an algorithm that employs the symmetry group of RM codes. In the sequel, we are mostly interested in recursive decoding algorithms. This is due to two facts. First, general recursive techniques have the lowest decoding complexity order of known for RM codes of an arbitrary order . This compares favorably to the complexity order of of both the majority decoding and the algorithm of [2]. Second, for long RM codes of an arbitrary order , recursive algorithms also achieve the highest thresholds known for nonexponential algorithms. These thresholds derived for two recursive algorithms in [4], [5] are (up to the smaller residual terms) as follows: if

Index Terms—Binary-symmetric channel, Chernoff bound, Plotkin construction, recursive decoding, Reed–Muller (RM) codes.

I. INTRODUCTION ELOW, we consider decoding algorithms for Reed–Muller (RM) codes [3]. These codes—denoted in the paper by —are completely defined by two integers, and , where , and have length , dimension , and Hamming distance as follows:

B

For RM codes used over a binary-symmetric channel, efficient decoding algorithms have been designed that can find the closest codeword with high probability beyond the bounded distance . In this regard, one important characteristic of threshold of the error-correcting capability of a specific algorithm is its de, which represents the number of the received coding symbols that can be corrected almost surely (with probability that tends to for long codes). For general RM codes of a Manuscript received October 16, 2005. This work was supported by the National Science Foundation under Grants CCF-0622242 and CCF-0635339, by INTAS under Grant 00-738, and by the Russian Fund for Fundamental Research under Grant 03-01-00098. M. Burnashev is with the Institute for Information Transmission Problems, Moscow 101447, Russia (e-mail: [email protected]). I. Dumer is with the College of Engineering, University of California, Riverside, CA 92521 USA (e-mail: [email protected]). Communicated by M. Sudan, Associate Editor for Coding Theory. Digital Object Identifier 10.1109/TIT.2006.883557

if

(1)

where is some constant. In particular, for RM codes of the second order, recursive decoding yields the same threshold as that of the algorithm [2], while reducing its complexity. For codes of fixed rate , the recursive decoding doubles the threshold of majority decoding and also reduces decoding complexity. However, no tight analytical bounds are known for the decoding error probability of RM codes when decoding performs beyond the bounded distance threshold. In this paper, our main goal is to tightly estimate error rates achieved by recursive decoding when RM codes are used on the simplest model of a binary-symmetric channel. The problem is especially important since many applications require very low output error probaand less, in which case, simulation techniques bility of become prohibitively time-consuming. In Section II, we introduce recursive design for RM codes and specify one decoding algorithm. Here we show that recursive decoding can be split into many similar steps. Every step—denoted —derives one new information bit from the original channel outputs and consists of multiple recursive additions and multiplications. However, these operations are performed in different order that depends on the specific step . As a result, it turns out that these consecutive steps vary significantly in their decoding error probabilities. Therefore, we first formalize our probabilistic setting in Section III. Channel outputs are first represented as the Bernoulli random variables (RVs). Then we define the information bits, and are obtained which represent the end random variables

0018-9448/$20.00 © 2006 IEEE

BURNASHEV AND DUMER: ERROR EXPONENTS FOR RECURSIVE DECODING OF REED–MULLER CODES

through multiple additions and multiplications performed in different decoding steps . Our goal is to estimate the conventional of each RV , and exponential moment then use the Chernoff bound. We shall see that finding these exponential moments for different decoding steps is an involved analytical problem that includes recursive recalculation of the spreading probability distributions, similarly to iterative algorithms. Therefore, in Section IV, we first establish some partial ranking of different decoding steps. Namely, it is proven in Lemmas 3 and 5 that are obtained in the first decoding steps the highest moments , which begin with multiple multiplications and then proceed with additions. Thus, these steps are the most error prone in recursive decoding and contribute most to the overall error probability. This calculation of the moments of the first paths is done in Lemma 7 of Section V. As a result, we obtain Theorem 8, which gives tight lower and upper bounds on the overall block error rate. Both bounds have almost identical behavior. As a direct by-product of this study, we also obtain similar bounds on the block error probability of majority decoding. In Sections VI and VII, we proceed with further improvements. In Section VI, we study the subcodes of RM codes obtained by eliminating a few information bits that are decoded first in recursive decoding. For good channels, this pruning exponentially reduces the error rate of the entire algorithm even if only one information bit is eliminated. In Section VII, we study another—a more powerful—recursive algorithm, whose threshold is given in (1). The latter algorithm is also more involved than the algorithm considered in Sections IV and V; however, we show that our analysis of mocan be carried over to the new algorithm with some ments alterations. In particular, tight analytical bounds on the error exponent of this algorithm are obtained in Theorem 17. In deriving these exponents, we also refine the bounds on maximum-likelihood (ML) decoding error probability of biorthogonal codes. The numerical results are then exhibited in Fig. 4 for the code , where the newly obtained bounds are compared with those of bounded distance decoding and majority decoding. This comor more parison shows that recursive decoding can reduce times the output error rates of the latter two algorithms, even when codes are used over high-noise channels. II. RECURSIVE DESIGN FOR RM CODES In this section, we briefly discuss recursive design of RM codes and their recursive decoding algorithm of [4]. [3] represents any codeword of in the form , where codewords and RM code belong to the codes and , respectively. We say that is split onto two “paths” and . By splitting both codewords and again, we obtain four paths of length that , and so on. Every splitting lead to RM codes of length doubles the number of paths, until we arrive at the repetition codes or full spaces . This is shown in Fig. 1. Now let denote a block of information bits that encode a . Then is also decomposed into the subblock vector that encodes vector and the subblock that encodes

4881

Fig. 1. Decomposition of RM code

.

. Proceeding in the same way, we see that any codeword can or . Here any be encoded starting from the end nodes left-end (repetition) code gives only one information bit, while any right-end code and can take any value

gives

bits. Also, parameters

Next, consider any specific path

that leads from the origin to the end node . Here we assign if we choose the -component on step , or otherwise. For example, there are two paths and that to in Fig. 1 lead from code Correspondingly, each left-end information bit is now mapped onto some path . Finally, each path is appended with that consists of ones. Now all these paths end at a suffix and have the same length . the node Second, consider any subpath

that leads to a right-end node . We further split subpath into subpaths by adding all binary suffixes of length . Thus, each right-end path also corresponds to one information bit. In the sequel, all extended paths are ordered lexicographically, . as -digital binary vectors, and are denoted Now we turn to the recursive decoding algorithms. Let any . Then any codeword binary symbol be mapped onto and has the form . of RM code belongs to Codeword is transmitted over a binary-symmetric channel with a transition error probability . Thus, our transmission can be represented as

Here the received block consists of two halves and , which are the corrupted versions of vectors and . In subsequent steps, we recalculate using operations over real numbers. is the bitwise product of the two uncorNote that rupted halves and . In our first decoding step, we estimate taking the bitwise product (2)

4882

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006

of the two corrupted halves. Then is decoded by some al, which is specified later. The output is a vector gorithm and its information block . In the second step, we estimate vector . Here we already have its corrupted version . We also take another corrupted from the right half. These two version, which is the vector estimates are combined in their midpoint (3) Then we use some decoding . The output is a vector and its information block . , we do not decode vectors In a more general scheme and , but consider them as our new corrupted inputs. Thus, and are repeatedly updated and decomposed into vectors halves. Here we use (2) in our -recalculation and (3) in an -recalculation. Finally, we perform ML decoding once we reach or full spaces . Given any vector repetition codes of length at any end node (here on full spaces), ML decoding performs majority voting and chooses an information symbol (4) (if the sum equals , we assign with probability ). We that repeats also output the corresponding codeword times symbol . Thus, we have the following algorithm. Algorithm

for an input vector .

1.1. Calculate vector

(6) be the mean of the symbols ML decoding (4) is incorrect if

obtained at this node. Then

(recall that the case is already excluded). For any two paths and , we write if precedes in the lexicographic ordering of all paths. To estimate the block error probability of the algorithm , let us assume that is the first path whose decoding is incorrect. Given correct results on all previous paths , we can replace (5) with a simpler recursion, which is independent of the previous results if if

and

Decode

to Step 1.2

into vector

. . ;

Output decoded components , use ML-decoding (4) at , use ML-decoding (4) at

.

(7)

, we see that the Given any path that ends at some node of (6) is the single output, obtained by performing mean additions (7) on the extended path , . Thus, we can now assume that all our paths give single outputs but end at the nodes , where for all left-end paths. Assuming that the previous paths are correct, let

.

1.2. Calculate vector

3. If

In this section, we formalize the probabilistic setting for the obtained on the various paths . Without loss of genRV erality, let the codeword be transmitted. For any path that ends at some node , let

.

into vector

Decode

2. If

III. PROBABILISTIC SETTING FOR RECURSIVE DECODING

, execute the following.

1. If

Pass

Indeed, the first line of (5) recalculates the current prefix on any -path . After this -path is decoded and returned, . Then is recalculated in the we also use its output second line of (5) on a -path . Thus, the entire decoding process is decomposed into different paths . Our next goal is to analyze decoding error probabilities obtained on these paths.

.

. be the conditional probability that is the first incorrectly decoded path. Note that decoding begins with the leftmost path

.

Next, we wish to specify recalculations (2) and (3) for any given path. Namely, consider any subpath of variable length . Let be its immediate prefix of length

Then we have the following trivial lemma. Lemma 1: The block error probability satisfies inequalities

First, note that any -path always precedes the corre. Therefore, our decoding follows the sponding -path of our paths. Second, we lexicographic order is recursively obtained from its prefix see that any output similarly to (2) and (3) if if

.

(5)

of the algorithm

(8)

To proceed with our probabilistic analysis, note that the origand consist of the different channel bits . inal blocks and are also obConsequently, their descendants tained from the different channel bits. Therefore, all symbols

BURNASHEV AND DUMER: ERROR EXPONENTS FOR RECURSIVE DECODING OF REED–MULLER CODES

of any vector are independent and identically diswill denote tributed (i.i.d.) RVs. Without loss of generality, . any of In the sequel, the decoding error probability will be estimated using the Chernoff upper bound. Namely, we shall consider the obtained on any end path and upper-bound its minRV imal exponential moment

4883

and Proof: We first compare the outputs of the paths . Let be an output of the common prefix , and , , , be four RV identically distributed (ID) with . For brevity, we then write ID- . Then for the path , recalculations (7) give the output

Similarly,

has the output (11)

The entire problem can now be reformulated as follows. where

Given: 1. A vector

of

i.i.d. RV

, where are two ID- RV. Note that classical inequality

2. A path of that consists of zeros. Perform: recalculations (7) on

ones and shows that any number satisfy inequality

and .

Estimate: the minimal exponential moment

of ID (even dependent) RVs

. (12)

IV. PARTIAL ORDERING OF THE PATHS EXPONENTIAL MOMENTS

BY

THEIR

In this section, our goal is to establish some ordering of the exponential moments . In particular, we will show that the first two paths

have the two biggest moments

Combined with Lemma 1, this will yield the following important statement. Lemma 2: The error probabilities , and the block error probability follows:

of all paths , are upper-bounded as

(9)

Thus, exponential moments can only grow if a path placed with

is re-

Next, we show that property (10) remains valid if the subpaths and are extended with any common suffix . First, consider the output of the path . Then we claim of the other path can be written in that the output the form (13) are ID- . Indeed, this property where all — holds at the common prefix , according to (11); and . — can be verified for both one-bit extensions The latter is readily verified by using recalculations (7), which give

To prove the above statement, we first establish some partial ranking of our paths . First, we consider two subpaths

with the same prefix and the same suffix . These subpaths are and be two output symbols called neighbors. Let obtained on these paths by recalculations (7). We begin with the following central lemma. Lemma 3: For any , any two neighbors satisfy the condition

Given property (13), we can again use inequality (12) for exand . Thus, Lemma 3 is proven. tended paths

,

(10)

Now consider all paths that end at the same node . We say that these paths form an -cluster. Lemma 3 immediately leads to the following.

4884

Corollary 4: For any leftmost path

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006

and any -cluster,

, its

(14) has the largest exponential moment in the cluster

Lemma 6: For any subset error probabilities of all paths

Proof: Consider any path of an -cluster with two consecutive bits . Replace these bits by . Then we have two neighbors and that satisfy property (10). These replacements can be performed until all zeros precede all ones, which . gives Thus, within any -cluster, the exponential moments can only grow if we move all zeros of the paths to the first positions. Finally, we compare the paths from and say that leads (or is different clusters. We write a leading path for) another path if has the same or greater minimal exponential moment (15) Lemma 5: The leftmost paths , satisfy the ordering

that has the smallest Hamming weight on each prefix of length , among all paths . In other words, this path serves as the leftmost path for every path in the entire set . Obviously, Lemma 2 can be repeated for such a subset as follows.

of separate -clusters,

with the leftmost path , the are bounded as follows:

Discussion: Note that the rankings of our paths are rather and incomplete. Indeed, we found the two (leading) paths . However, the preceding analysis still admits two possible candidates for the third leading path, namely, the paths

that have four consecutive symbols— and —in disagreement. To circumvent these problems, we shall directly estimate the moments of a few first paths. This will allow us to estimate the error probability for not only RM codes but also their subcodes obtained by eliminating the leading paths. V. EXPONENTIAL MOMENTS OF THE LEADING PATHS Our next goal is to apply Lemma 2 for RM codes and Lemma and use 6 for their subcodes. Namely, we will consider the original Bernoulli distribution of the channel outputs

Proof: Let path end at some -node, . We add a zero and consider the extended path that ends at the -node. In this case, we shall prove that . is the Indeed, according to recalculations (7), any output of two i.i.d. outputs and of the path . Next, product for any given consider conditional expectation . Note that is a convex function of for any . In by its this case, Jensen inequality allows us to replace RV as follows: mean

where

bit

(16) Next, note that by definition of , we may scale our original and consider the normalized RV to verify RV inequality (15). It is also easy to see that the outputs have after any set of recalculations (7). the same means , inequality (16) reads Then for the normalized RV

Our goal is to evaluate the moments of paths for the following three cases: ; A: the leading path B: the second leading path ; C: the th (lexicographically ordered) path

We begin with cases A and B. To use bounds (9), we will deand the exact expression for the error probability rive both . Then we will proceed with case B and find . Here we extensively use the parameter (17) Also, let denote the decoding error probability of a repewith . tition code of length used over the This probability is well known as

Thus, . Next, we prove that . Indeed, consider the path obtained from by replacing its last two digits with . According to Lemma 3, and the proof is completed. Note that is the single path of the -cluster, and is the leftmost path in -cluster. Thus, these two lead all , and we have Lemma 2. other paths. Also, is a bounded subset of paths More generally, we say that if there exists a specific path (considered as a binary vector)

(18) The results are summarized in the following lemma.

BURNASHEV AND DUMER: ERROR EXPONENTS FOR RECURSIVE DECODING OF REED–MULLER CODES

Lemma 7: On a and leading paths

, algorithm decodes the two with error probabilities

4885

Then for any

(19) (20) where

is defined by expressions (17) and (18), and (21)

Proof: We begin with finding . According to (7), subpath

of the leftmost path gives i.i.d.

To minimize the above expression, we take

RVs which gives all of which have the same Bernoulli distribution (22) Then the full path the moments

has the outputs

with

Therefore, we get

Using equality

, this can be rewritten as (20).

Combining Lemmas 2 and 7, we obtain the following bounds. Theorem 8: On the probability , where

Thus, we find

(23) , this proves the inequality in (19). To deSince , note that subpath arrives rive the exact expression for . Its corrupted symbols are repreat the repetition code sented by the Bernoulli RVs of (22). Thus, this decoding is and has error probability performed on the of (19). Case B: We proceed similarly to case A and consider the outputs of the subsequent subpaths. Given original channel outputs , we obtain the following outputs: Subpath

Here

and

Also, and notation

Output

are i.i.d. RV with the same distribution

are i.i.d. RV and so are

and

. Now we use

, algorithm

gives block error

(24) and quantities , , and are defined in (17), (18), and (21). Applications for Good Channels: It is readily verified that bounds (19) and (20) approach each other given a relatively bad . However, if is small, then . channel with and its estimate In this case, both the exact expression have the same order (25) For , estimate (21) gives . Then bound (20) reduces times (25) (here we consider the main exponential term). Thus, our exponential estimates can be substantially reduced on the good channels even when only one is eliminated. path In Figs. 2 and 3, we compare simulation results with both and , respectively. We bounds (24) for the RM codes see that bounds (24) closely approach simulation results and . each other when the output error rate is below Note that majority decoding executes the same calculations . Indeed, each information bit is as those used on the path then derived by taking the majority vote among symbols that and have the same distribution (22). are similar to symbols Thus, we have the following corollary. Corollary 9: On the error probability , where

, majority decoding gives block

(26)

4886

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006

Discussion: It is readily verified that if we can use the estimate

is small, then

Note also that (27) reveals two critical properties of the different . First, the quantities decline in and rapidly appaths proach their limit . Second, the rate of this decline becomes slower as grows. Thus, eliminating the first paths yields the most part of the entire reduction of the output error probability of a subcode. of (27) exceeds its Note also that the estimate exact value (21), which satisfies inequality for all . It is for this reason that we treat our case B separately from the general case C.

9

Fig. 2. Algorithm : Simulation results and bounds on word error rate (WER) of length and dimension . for RM code

128

29

VI. SUBCODES OF RM CODES bounded by the path Consider the maximum subset and let be the corresponding subcode for any . Thus, all other paths and their information symbols are eliminated. Our goal is to find the dimension

of

and estimate its output error probability

Let

be a parameter. Below we consider long codes for and . Note that in this case, code . In particular, rate can still take any value in the interval for and if . Now we show has the same asymptotic code rate as original that subcode code under some mild restrictions.

9

Lemma 11: The subcode dimension

of the RM code

Fig. 3. Algorithm : Simulation results and bounds on WER for RM code of length and dimension .

256

37

(28) Codes

Finally, we proceed with case C and derive the exponential of the path . This will allow us to consider moment is simsubcodes of RM codes in the next section. Finding ilar to (though more involved than) case B. Therefore, we only formulate the result and give the proof in Appendix I. Lemma 10: On the

, algorithm

decodes any path

has

and

have asymptotically equal dimensions if

provided that and satisfy one of the following conditions: if (29) if

const

Proof: The path enters the left-end node . Thus, this path eliminates all the paths that enter the previous left-end nodes. The latter paths have the form

with error probability

where (27)

Here, the prefix has length and includes exactly zeros. This gives (28). Now both conditions (29) are readily employed in the second part by comparing the two binomial and . coefficients

BURNASHEV AND DUMER: ERROR EXPONENTS FOR RECURSIVE DECODING OF REED–MULLER CODES

This lemma combined with Lemma 10 immediately gives the following two corollaries. Corollary 12: On the , algorithm with block error probability subcode

4887

, whereas its decoding complexity in (1)—doubles that of has the same asymptotic order (see [4] for details)

decodes the

(30) where

and

are defined in (27) and (28), respectively.

with Corollary 13: Consider a long RM code on a . For any growing paramsmall error probability reduces the block error probability of the eter , subcode code

or more times, where declines exponentially in . Proof: For growing , we can replace general bound (27) with its approximation

In the previous sections, we first considered the algorithm due to its relative simplicity. In this section, we extend our anal. ysis of error exponents to the more involved algorithm is transmitted, the correct Assuming that the codeword decoding gives the codeword on the node . Let be any other codeword, and be the subset of its positions . Here, we assume that . In this case with symbols if if Finally, given an input vector we define the sum

. at any node

, for any , (32)

Then we compare the exponents in bounds (25) and (30). For , we obtain

, we assume that all previous Similarly to the algorithm are decoded correctly. Thus, for every path , we paths can use the same recursive recalculations (7). By definition (31), for some . decoding on path is incorrect if Then any path is decoded with the error probability (33)

which gives the above statement, due to conditions (29). grows exponentially in , which allows us to Namely, remove the term in the latter expression. Thus, we see that on good channels, recursive decoding of subcodes yields exponential reduction of the output decoding error probability of RM code without any asymptotic reduction in its code rate.

Note that any RV in (32) is the sum of i.i.d. RV . Therefore, for any , we can consider the for any single RV (34) that gives the same error probability for all

VII. A REFINED RECURSIVE ALGORITHM Consider any path of the algorithm that enters any . Instead of moving to the repetition code biorthogonal code , we now perform ML decoding on all these codes. Thus, we choose the code vector that maximizes the inner product with the input vector (31)

Also,

is the sum of i.i.d. RV, in which case . Thus, for any left-end path , we can replace (33) with the union bound (35) Recall also that any right-end path gives information symbols. Each of these symbols has the same error probability . Thus, for we have

Thus, decoding is executed as follows. Algorithm 1. If 2. If

, perform algorithm

.

, use ML decoding (31).

This algorithm compares favorably with the previous algorithm . Namely, the decoding threshold of —presented

(36) Now we see that the total error probability on any path can be evaluated using the sum of symbols . This . Indeed, we gives the same setting as that of the algorithm consider any left-end path that ends at some node and

4888

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006

extend with the suffix . In this case, any RV (34) is the result of recalculations (7) performed on this extension. using the extended paths of Thus, we need to calculate length , which arrive at the end node instead of the former paths of length that ended at the node in the algorithm . In this case, we can again use the Chernoff bound (37) be the dimension of the RM code Let have the following important lemma.

fined paths . We only need to take into account that the leftmost paths are now one bit shorter. In particular, we use the and are the two leading paths. Next, we can fact that estimate their moments similarly to the cases (19) and (20). Correspondingly, the following lemma is identical to Lemmas 7 and 10. Lemma 15: On the

, the paths

. Then we have the moments

Lemma 14: Algorithm decodes RM code block error probability , where

with the (42)

(38)

where the parameters and (21).

and

are defined in (17)

— Proof: The lower bound—similarly to the algorithm is incorrect if it fails on the first path is due to the fact that . The next inequality is the union (upper) bound taken over all paths. To obtain (38), we first combine inequalities (35)–(37) as

We complete our study of bounds (38) by deriving tight lower of the and upper bounds on the error probability biorthogonal code . Here decoding is performed on with transition error probability . the of (42) gives Note that the estimate of (35) combined with the union bound

Here we consider all nonextended paths obtain (38) by proving the equality

To refine this bound, we will use numerical (nonasymptotic) estimate that gives tight results for small block lengths . This is done in the following lemma proven in Appendix II. Our proof also includes a simpler asymptotic estimate.

and

. Below we (39)

First, all paths and are extended with an arbitrary suffix of length and , respectively. In this case, all extended paths represent all possible binary vectors of length . Thus, extended paths. On the other hand, all nonexwe have and tended paths and are now summed with weights , respectively. Thus, the overall number of weighted paths satisfies equality

of distance Lemma 16: Biorthogonal code used on a BSC with a transition error probability has ML decoding error probability , which satisfies inequalities

(43)

(44) (40) where is defined in (18) and is defined in (48). Now we combine Lemmas 14–16 as follows.

Next, we show that (41)

Indeed, all nonextended paths end at the nodes , where . Now let us we append each path with an arbitrary . Then, the extended paths form the ensuffix of length that include or fewer tire set of vectors of length zeros. Therefore, there are extended paths. On the other hand, each path is now counted times. Thus, we obtain equality (41). Now we see that (39) follows from equalities (40) and (41), and the proof is completed. Our next goal is to estimate in (38). Obviously, the earlier analysis of Sections IV and V carries over to the rede-

Theorem 17: On the , the code of order , , and distance is decoded by the length with block error probability algorithm

(45) where functions , , and are defined in (21), (43), and (44). for in Proof: We use the lower bound the left-hand side of (38). In the right-hand side of (38), we count and the remaining paths. Here, separately the first path has weight and all other paths have combined , according to (39). Also, we use the weight

BURNASHEV AND DUMER: ERROR EXPONENTS FOR RECURSIVE DECODING OF REED–MULLER CODES

4889

noise (AWGN) channel, in particular, represents the third important problem in recursive decoding. APPENDIX I Proof of Lemma 10: Similarly to case B, consider the outputs of the subsequent subpaths. Given original channel outputs , we obtain the following outputs: Subpath

Fig. 4. Algorithm 8: Simulation results and bounds on WER for RM code of length 256 and dimension 37. Comparison with bounded distance decoding and majority decoding.

fact that is the maximum moment on the remaining paths . and employ the estimate (42) for Similar upper bounds can also be derived for subcodes using the estimates for . In Fig. 4, we compare simulation results with the bounds (45) . Similarly to the algorithm , both bounds for the RM code (45) closely approach simulation results for . For comparison, we also plot the upper bound (26) for majority decoding and that of the bounded distance decoding.

Output

Here the first step gives the output . In the second step, i.i.d. outputs give the product . Note that outputs , , and are i.i.d. RV, and so are and . We also extend notation (22)

Then for any

, we calculate

VIII. CONCLUDING REMARKS In this paper, we considered two different algorithms for recursive decoding of RM codes on a binary-symmetric channel. We first decomposed the entire decoding process into different paths and estimated their output error probabilities. In particular, we found the leading paths, which give the highest error exponents in the decoding process. In this way, we derived tight bounds on the overall decoding error probability of a given RM code. One important conclusion is that the output decoding probabilities vary significantly for different paths; thus, elimination of the leading paths gives an exponentially lower error probability on the remaining subcodes. Note, however, that this study leaves many open questions. First, our rankings of different paths are rather incomplete. In particular, by using only the two leading paths in our calculations, we assign the same error probability for all remaining paths. Thus, our estimates can exceed the exact bounds up to times. Second, it is yet unclear how to calculate the exponential moments for most of the remaining paths, which is important for choosing the best subcodes. Finally, all our calculations are channel specific and heavily rely on the fact that the channel has only two outputs. Thus, extending these results for general continuous channels, and an additive white Gaussian

To evaluate the last sum, denote the function

and its extremum

,

, and consider

4890

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 11, NOVEMBER 2006

Then

in which case

Now we use the following lemma proven below. Lemma 18: For any , we also have

Therefore, we should set the same value, i.e.

. Both points

and

give

.

and the proof is completed. As an additional remark, it is also is close to constant for easy to show that the function . For any

APPENDIX II

, we now complete the proof as follows:

of length includes the Proof of Lemma 16: Code , the all-one codeword , and all-zero codeword the codewords of weight . Assuming that is transmitted, let denote the event that the received vector is closer to the codeword than to . Then if if

Proof of Lemma 18: Consider the derivatives of the func. Here tion

Since and attained for an internal stationary point equalities

The condition . Since 1)

, the optimum is , which satisfies

holds in two cases.

Then the upper bound (44) is obtained as follows:

Now we proceed with the lower bound. Let

. Now we use the fact that

For any parameter , we can lower-bound the error as follows: probability of the biorthogonal code

and, therefore,

2)

Now let all codewords of weight be partitioned into pairs of opposite codewords, say and , and . Note that for each pair where , the union includes the event . Also,

where

we must have

.

. Then we must also have (46)

BURNASHEV AND DUMER: ERROR EXPONENTS FOR RECURSIVE DECODING OF REED–MULLER CODES

To evaluate , consider three codewords

,

4891

,

By taking the derivative of the last expression, we see that its minimum is attained at Here is the number of ones in the corresponding part of the output vector . The quantities , , are similar. Then any and output vector , such that , satisfies inequalities

Thus, we can use the bound

, where

(47) Thus, we can use the bound

, where In particular, taking bound

in (46), we obtain a simplified lower

ACKNOWLEDGMENT (48) Now we see that the lower bound (44) on the error probability directly follows from (46) and (48). As one final simplification, we also replace the numerical bound (48) with a more feasible asymptotic bound. Here we combine two inequalities (47) into a less stringent condition

Each of the quantities , , is the sum of i.i.d. RVs that take values with probabilities and , respectively. Then applying the Chernoff upper bound for any and , we have using notation

The authors thank K. Shabunov for helpful remarks and assistance with computer simulation. REFERENCES [1] R. E. Krichevskiy, “On the number of Reed-Muller code correctable errors,” Dokl. Soviet Acad. Sci., vol. 191, pp. 541–547, 1970. [2] V. Sidel’nikov and A. Pershakov, “Decoding of Reed-Muller codes with a large number of errors,” Probl. Inf. Transm., vol. 28, no. 3, pp. 80–94, 1992. [3] F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes. Amsterdam, The Netherlands: North-Holland, 1981. [4] I. Dumer, “Recursive decoding and its performance for low-rate Reed–Muller codes,” IEEE Trans. Inf. Theory, vol. 50, no. 5, pp. 811–823, May 2004. [5] I. Dumer and K. Shabunov, “Recursive error correction for general Reed-Muller codes,” Discr. Appl. Math., vol. 154, pp. 253–269, 2006.