Algorithmic Complexity in Coding Theory and the ... - CiteSeerX

Algorithmic

Complexity

and the Minimum Alexander

in Coding

Distance

Theory

Problem

Vardy

Coordinated Science Laboratory University of Illinois 1308 W. Main Street, Urbana, IL 61801 vardy@golay.

Abstract. We start withan overviewof algorithmiccomplexity problems in coding theory We then show that the problem of computing the minimum diktance of a binaryIinwr code is NP-hard,and the correspondingdeci~”onproblemis W-complete. This constitutes a proof of the conjecture Bedekamp, McEliece,vanTilborg, dating back to 1978. Extensionsand applicationsof this result to other problems in codingtheqv are discussed.

1. Introduction. This paper is organized in a manner contrapositive to the talk. In the talk, we give a detailed overview of algorithmic complexity issueain codhg theory. Herein, we start with an overviewof the prominentrole that computational complexity plays in coding theory: we briefly surveythe numerousalgorithmic complexity issuesthat arise in code construction and decoding. Subsequently,we focus on one specific problem. Namely, we consider the problem of computing the minimum distance of a linear code, which is also equivalentto the problem of finding the shortest cycle in a linear matroid over a finite field [90]. A long-standing conjecture [12] says that this problem is NP-hard, and we will settle this conjecture in the ai%rmative. We provide a detailed proof. We hope that our proof illustrates some of the elegant techniques used in coding theory today, such as the construction of MDS codes via Vandermonde matrices [15, 59, 72] and concatenated coding [23, 31], for instance. It is interesting that algebraic techniques of this kind cart be employed to answer an essentially combinatorial question. The complexity of computing the minimum distance and the proof of the conjecture of Berlekamp,McEliece, and vanTilborg [12] will be only briefly mentioned in the talk.

2.

Complexity in coding theory

In discussing the close ties between coding theory and complexity theory, there are two general categories of natural questions: one involving the application of codes to computational complexity and the other focusing on the computational complexity of coding itself. In the first category,codes havebeen usedextensively both to deviseefficientalgorithms in a varietyof contexts and to prove that no such algorithms “This workwas supportedby the Packard Foundation, the National Science Foundation, and the JSEP grant NOOO14-961O129. Permission

10 nmkc digiull/ll:lrd

ct~pics OFall

or

p;lll ol’lhis maleri:ll lilr

pemomd or clmwoonl we is grantixi $vitllwl (LCprovided IIml the coplcs ,arenot made or di.slnlw[cd I’or prolil or cwnmerc ial ;idvoi~[ogc.[he cop},right noliw. lhe title ot Illc p(llll]c:llitm find 11sd:llc nppcar, atld oo[ice is

csl .uiuc

.edu

exist. For example, the elegant theory of probabiliiticrdly checkableproofs [6] uses algebraic codes in an essentialway to resolve Iong-staudIngopen questions about the hardness of approximation for such basic optimization problems as graph-coloring and clique-size. This mea of research has by now accumulated a sizable body of interestingresults. However, we will not even attempt to discuss the use of codes in computational complexity. We refer the reader to [4, 6, 79], and especially [27], for a detailed overview of this subject. On the other hand, in what follows, we briefly survey some of the fascinating computational problems that arise in coding theory itself. We will try to make this overview accessible to as broad an audience as pcsssible,and will not assume any prior knowledge in coding theory. Our overview is by no means comprehensive;when [10] is finally ready, a more detailed survey would hopefully become available. The bkth of the subject of codkg for data transmission occurred at the time of the announcementof Shannon’s coding theorems [73], which not only established the limits of the gains possible with codktg, but also proved the existence of codes that could effectivelyreach these limits. Shannon [73] showed that for every communication channel, there is a constant C, called the copocity of the channel, which has the following fundamentalsignificance: if one wishes to communicate over the channel at a rate R (in bits per channel use), then one can do so as reliably as desired, if and only if R < C. Specifically, for every & >0 there exists a sufficiently long error-correctingcode C of rate R, such that the probability of error in maximum-likelihood decoding of C is at most e. It was recognized early on that the trouble with Shannon’s coding theorems, from a practical point of view, is essentially computational in nature. Although Shannon [73] settled the question “Do good codes exist?” in the atlirmative,his work led to two other questions “How can we find such codes?” and “How can we decode them?”. In a sense, coding the ory is all about these two questions, and both questions are fundamentally computational. It is trivial to find the codes promised by Shannon using a superexponentialsearch, but in practice we would like to construct a code in polynomial time. This leads to the problem of code construction, which is d~cumed in the next sutrsection. Furthermore, Shannon used maximum-likelihood givw

thal

tO republish,

wp~iglll 10

is h! permission ol’llle ,-\~h[. Im. ‘1’0 topy olhcrw isc.

posl011scllws or (0 dislnhulc

permission and/or

[0

Iisls. t’cqllires Spccilic

Lx

.W()(”” ‘ 97 El Paso, “1-cxnsLis g which is nearly MDS. Namely d = (n– k+l) –g, where g = l/2r(r– 1) is the genus of the curve. This is a simple example of a generaf type of codes, known aa algebraic-geometry codes [83, 84].

The error-correction capability of a code has to do with its minimum Hamming distance

where the second equality is true for linear codes. Here, the Hamming distance d(z, y) is the number of positions where z and y differ, while the Hamming weight wt(z) = d(z, O) is the number of nonzero positions in z. Thus an errorcorrecting code can be viewed m a packing of disjoint spheres in the space IF; endowed with the of radius t = l(d–1)/2j Hamming metric. If codewords of C are transmitted over a noisY channel, then errors in any < t positions may be corrected at the receiver end by identifying the unique sphere to which the error-corrupted channel output belongs. Evidently, we would like t, and hence also d, to be large. It should be obvious that attaining a high rate R = log~ 1~/n and a large distance d are conflicting goals. Hence the codes of interest for communications are those that achieve a good tradeoff between these two parameters.

Algebraic-geometry codes are arguably the most powerful codes known today. These codes are constructible in polynomial time, but the complexity of constructing the best codes in this class tends to be too high for practiczd purposea: 0(n30) was originally reported in [60], which was recently improved to 0(n17 ) in [56]. A construction that would produce a generator matrix for such codes in time 0(n3 ), say, would be a significant achievement, and there is reason to believe that a result of this nature is forthcoming [70].

Reed-Solomon codes. Here is an example of a simple polynomial-time construction that produces excellent codes. As is often done in coding theory, we will specify a code C by describing an encoder for C. For each information word u = (Uo,ul,. ... Uk_ ~) c F;, we first consider the polyno-

Another drawback of algebraic-geometry codes is that their symbol alphabet is still quite large. Algebraic-geometry constructions are successful (in particular, better than the Gilbert-Varshamov bound discussed below) only for q >49.

k–1

mid ~.(Z) = Uk-1$ + ~~. + UIZ + UO. Then the codeword c=(cl, cz, ..., ~) E C of length n = g – 1 corresponding to u is the evaluation of /U(z) at all the nonzero elements crl, az, . . . . crg-..l of the field lF~, that is c~ = fw(ai).

BCH codes. On the other hand, here is how one can get binary codes from Reed-Solomon codes: given an (n, k, d) Reed-Solomon code C over I13m, let C be the (n, k*, d*)

93

parameters R and J much better than those of [89]. Since a linear code C is the kernel of its parity-check matrix H by definition, the minimum distance of C is just the minimum number of linearly dependent columns of If, in view of (1). We construct an (n – k) x n binary parity-check matrix H with the property that every d – 1 columns of H are linearly independent, column-by-column, using greedy search. After i columns of H have been already chosen, there are at most

binary subfield subcode of C, that is C’ = Cfl IF;. This produces the important class of binary BCH codes [59, Chapter 9]. These codes are easily constructible in polynomial time, as we can compute a parity-check matrix for the BCH code O in time O(n log n) from a parity-check matrix for the Reed-Solomon code C. It is obvious that d* ~ d but k“ < k, and the question is whether a code with parameters (n, k“, d*) is still a good code. It turns out that for short lengths, say up to n < 128, BCH codes are among the best binary codes known. However, as n -+ cc, either d*/n or k* /n tend to zero. The latter result, due to Lin and Weldon [54], shows that BCH codes are asymptotically bad.

(:)+o+”+(’a distinct linear combinations of these i columns, taken d – 2 or fewer at a time. If this number is less than 2n-k – 1 we can always find another nonzero column, different from these linear combinations, and append it to H. We can keep doing this, and complete the construction of all the n columns in H, provided

Asymptotically good codes. This brings us to the subject of

asymptotic properties of codes. For i = 1, 2, ..., let G be an (~i, ki, di) linear code over a fixed field IFq. The infinite sequence of codes Cl, C2,. . . is said to be asymptotically good if ni + cm with i, while kl/n~ + R and d~/n~ ~ b for some nonzero R and 6. Thus all we require is that the rate and the relative distance are both asymptotically non-vanishing. This defining property being so generous, it is rather surprising that it took nearly 25 years to come up with the first example of an asymptotically good sequence of binary codes that is constructible in polynomial time [46]. The basic idea of Justesen [46] was a clever use of a concatenated coding technique [31], which is described in detail in the next section. Today, many such constructions are known: Zyablov codes [94], codes constructed by Shen [74] from Hermitian curves using a variation of Justesen’s concatenation, and codes constructed using expander graphs by Alon et al. in [3] are just a few examples. Asymptotically good binary codes with polynomial-time complexity of construction and the best known parameters were obtained by Vl&du~, Katsman, and Tsfasman in [89]. The parameters of these codes are depicted in Figure 1. The approach of [89] combines powerful algebraic-geometry codes, constructed from Drinfeld’s modular curves, with concatenated coding.

1+ C1)+(3+”””+(0 0 such that the decoder always finds the closest codeword to a channel output y, provided the distance from y to that codeword is at most t. There is no guarantee on what a bounded-distance decoding algorithm does if the distance from y to the closest codeword c exceeds t:it may still find c, or it may output another codeword, or it may simply halt indicating failure.

channel is characterized by two known probability-density functions fo(.) and fl (), where f,(a) is the probability of receiving a E A given that z ~ {O,1} was transmitted. This is a very general channel model. The most useful special case, known as the additive white Gaussian noise (AWGN) channel, is when fO(.) and fl (.) are the Gaussian dktributions IV(+1, a2) and N(–-1, Oz), respectively. In this case, it is convenient to think of codewords as embedded in the Euclidean space R“ via the mapping {O,1} ~ {+1, –l}. Under this mapping, a binary code C of length n becomes a subset of the 2“ vertices of the hypercube [+1, —1]”. Furthermore, the logarithm of the probability (log-likelihood) of receiving y E R“ becomes proportional to the squared Euclidean disc

(3)

i=l

To illustrate the difference between hard and soft decision decoding we will, for the sake of simplicity, restrict our attention to bhmry codes. That is, we consider the case where the channel input is {O, 1}. In hard-decision decoding, the channel output A is also {O, 1}. The most useful model in this case is the binary symmetric channel: a bit at the channel input is either transmitted as is, or inverted with a fixed probability p. In soft-decision decoding, the channel output A is a large set, most often A is the real line. The

tance from y to the transmitted codeword c E C

= ~(-l)c’pi

–1]’.

Usuafly, soft-decision decoding is a much more challenging task than hard-decision decoding, but the potential rewards are great. Loosely speaking, for a given code, maximumlikelihood soft-decision decoding requires twice less energy per bit than maximum-likelihood hard-decision decoding, to achieve the same probability of decoding error [17].

In hard-decision bounded-distance decoding, we usually have t = l(d– 1)/2], while in soft-decision bounded-distance decoding (of binary codes on the AWGN channel) we usually have t = W. In both cases, t is equal to half the minimum distance between distinct codewords, in the corresponding metric. This means that t is the largest constant for which we can guarantee the correction of all error patterns of

Maximum-likelihood vs. bounded-distance decoding. Given the channel output y E An, the optimal decoding strategy is

95

Euclidean space !Rn. The increase in complexity from harddecision to soft-decision decoding is by a factor of O(d).

weight (norm) at most t, even if we were to use maximumlikelihood decoding. Bounded-distance decoders of this kind are said to achieve the error-correction radius of the code. Algebraic bounded-distance

It is also clear from our discussion of the BerIekarnp-Massey algorithm that it may fail whenever the weight of the error vector exceeds [(d– 1)/2]. A number of papers [30, 40] have been devoted to developing polynomial-time algorithms that guarantee decoding to the closest codeword beyond this bound. For Reed-Solomon codes, the best results have been recently obtained by Sudan [80], using a new approach. The algorithm of [80] decodes to the closest codeword whenever the weight t of the error vector is bounded by

decoding. For many important

families of codes, in particular BCH codes, Reed-Solomon codes, and most algebraic-geometry codes, we now have polynomial-time hard-decision bounded-distance decoding algorithms that achieve the error-correction radius of the code. The availability of such bounded-distance decoders is the result of a considerable body of work in coding theory, see [11, 42, 83] and references therein. The most notable contribution to the development of these decoders is arguably the Berlekamp-Massey [11, 61] algorithm which found applications also outside of coding theory, in such fields as cryptography and dynamic control. We will illustrate the idea for Reed-Solomon codes. Suppose that c= (co, cl, . . ..cl–l ) is a codeword of a Reed-Solomon code C over lF~ = IF~+l, and let e = (eo, el, . .,e~–1) be the channel error-vector, so that y = c + e. Consider the associated polynomials C(Z) = C. – 1z n-1 + ..+ CIZ+CO and e(z) = en–lzn–l +.. + elz + eo. Notice that the sum y(z) = c(z) + e(z) is known to the decoder. Furthermore, it is not difficult to see that the (n, k, d) Reed-Solomon code C, defined in the previous subsection, is precisely the set of all c E IF:, such that the associated polynomial C(X) satisfies c(crl) = C(crz) = . ~. = C(ad-’) = O, where a is a fixed element of order n in lF~. The decoder can thus compute S, ~f

e(ai)

= e(ai) + C(CY;) = y(ai)

t < ‘=a,b:(a+l)(b+l

)+~b(.t-l)(b+l)>n {n-a-b(k-l)-l}

This may be well approximated as t ~ n – (2nk)112. The decoding complexity is still polynomial, and the foregoing upper bound on t becomes far better than the conventional t s l(d–1)/2j for Reed-Solomon codes of rate R < ~. Finally, our discussion of the Berlekamp-Massey afgorithm illustrates the idea of reconstructing the “unknown” syndromes tkom the known ones. Once all the syndromes have been computed, the entire error vector can be recovered. Loosely speaking, this general idea underlies most of the known bounded-distance decoding algorithms for algebraicgeometry codes. These decoding algorithms involve deep r~ suits from algebraic geometry, and we will not even attempt to describe them here. We refer the reader to H@holdt and Pellikaan [42] for an excellent recent survey of this subject.

(4)

The hardness of maximum-likelihood decoding. Maximumlikelihood decoding of a binary linear (n, k, d) code C, both hard-decision and soft-decision, can be trivially accomplished in time 0(2~ ) by simply comparing the channel output y with all the 2k codewords of C. It is not difficult to show that this task can be also accomplished in time 0(2”– k). The following result, due to Berlekarnp, McEliece, and van Tilborg [12], shows that for the general class of binary linear codes, polynomial-time algorithms are unlikely to exist (unless P = NP), even for the simpler case of hard-decision maximum-likelihood decoding. To describe this result, we first need to express the maximum-likelihood decoding problem in slightly different terms. Given the channel output y E IF; and an (n – k) x n parity-check matrix H for a binary code ~ we define the syndrome of y as s = Hyt. Then the most likely codeword, that is the one closest to y in the Hamming metric, is given by c = y + e, where Hei = s and e c IF; is the minimum-weight vector with this property. Hence maximum-likelihood decoding is equivalent to finding e, given H and s. The corresponding decision problem can be stated w follows:

for all i = 1,2, . . . . d– 1. Moreover, observe that the entire error-vector e can be reconstructed from its n syndromes SI, S2,.. ., S., defined by (4). The problem is that only the first d – 1 of these n syndromes are known to the decoder. Fortunately, it can be shown that if wt(e) ~ t then the syndrome sequence S1, S2,. . ., S~ satisfies a linear recurrence of order ~ t. This recurrence, and with it the entire sequence S1, S2, . . . . S., can be recovered from the first d– 1 syndromes S1, S2,... , S,j- 1, provided d – 1 ~ 2t. This is precisely what the Berlekamp-Massey algorithm does: it is a general purpose procedure for efficiently determining the linear recurrence of lowest order that produces a given sequence (over any field). If the sequence S1, %, ..., Sd- 1 of known syndromes satisfies a linear recurrence of order t s L(d– 1)/2], then the Berlekamp-Massey algorithm finds this recurrence in time 0(t2 ). The complexity of the syndrome computation in (4) is O(nt), which is also the complexity of reconstructing the error-vector e given the recurrence generating S1, S2, . . . . Sn. It is obvious that the Berlekamp-Massey algorithm is inherently a hard-decision decoding algorithm. However, Forney and Vardy [36] have recently shown how this algorithm can be employed for polynomial-time bounded-distance decoding in Euclidean space. The results of [36] are actually more general. Forney and Vardy [36] show that whenever an (n, k, d) code C has a hard-decision bounded-distance decoder which achieves the error-correction radius [(d–1)/2j, it can be used to design a soft-decision bounded-distance decoder which achieves the error-correction radius W in the

Problem:

MAXIMUM-LIKELIHOODDECODING

Instance: A binary m x n matrix H, a vector s E lF~, and an integer w >0. Question: Is there a vector z E IF; of weight 0. Bruck and Naor [16] prove that MAXIMUM-LIKELIHOODDECODING remains hard even with unlimited amount of pre-processing on H, in the following sense: the existence of a polynomialtime afgorithm for this version of MAXIMUM-LIKELIHOOD DECODING implies that the polynomial hierarchy collapses at the second level, namely U~l X: = X;. The proof of [16] is baaed on the theorem of Karp and Lipton [47] which shows that the polynomial hierarchy collapses in this way if an NPcomplete problem can be solved in polynomial time with polynomial advice (belongs to the class P/poly). The result of the pre-processing on H constitutes the polynomial advice in the argument of [16].

Trellis decoding and its complexity. As discussed in the foregoing paragraph, it is unlikely that there exist polynomialtime maximum-likelihood decoding algorithms for Iinear codes, either soft-decision or hard-decision. Nevertheless, we can do substantially better than (9(2k ) or 0(2n–k). Namely, it is possible to achieve full maximum-likelihood soft-decision decoding of certain linear codes in exponential time, but with complexity exponent much less than n min{R, 1–R}. Despite their exponential nature, such algorithms are of considerable interest in coding theory, due to the significant gap in performance between hard-decision bounded-distance decoding and soft-decision maximum-likelihood decoding: on many useful channels, a maximum-likelihood soft-decision decoder for a short code might achieve a lower probability of error than a hard-decision bounded-distance decoder for a much longer code.

Downey, Fellows, Vardy, and Whittle [20] recently proved that MAXIMUM-LIKELIHOODDECODING is hard for the parametrized complexity class W[l]. Namely, it is unlikely that there exists an algorithm which solves MAXIMUMLIKELIHOODDECODINGin time /(w)nc, where c is a constant independent of w and f(. ) is an arbitrary function. Many NP-complete problems are fixed-parameter tractable. For example VERTEX COVER, a well-known NP-complete problem [39] which asks whether a graph G on n vertices has a vertex cover of size at most k, can be solved [8] in time O(kn + (4/3) kk2). Loosely speaking, the parametrized complexity hierarchy FPT = WIO] ~ WII] c W[2] c . . . introduced by Downey and Fellows [18, 19] distinguishes between those problems that are fixed-parameter tractable and those that are not. The result of [20] shows that MAXIMUMLIKELIHOODDECODINGis not likely to be fixed-parameter tractable. Furthermore, this result implies that boundeddistance decoding for the class of binary linear codes is hard in the following sense: if a polynomial-time algorithm for bounded-distance decoding exists then the parametrized hierarchy collapses with IV[l] = FPT.

Trellis decoding is a primary example of exponential-time maximum-likelihood decoding algorithms of this type. Trellises can be also used to implement randomized “sequential” decoding algorithms [26, 33], whose performance is close to maximum-likelihood and whose running time is polynomial with appreciably high probability. A trellis T is an edge-labeled directed graph T = (V, E, A) with the following property: the set of vertices V can be partitioned into disjoint subsets Vo, VI,..., Vn, such that every edge e E E begins at a vertex v E Vi and ends at a vertex v’ E V,+l for some i = O,1, . . . . n–l. It is usually assumed that the subsets VO,V. C V each consist of a single vertex, called the root and the toor, respectively. Clearly, the sequence of edge labels along each path from the root to the toor in T defines an n-tuple (al, az, . . . an) over the label alphabet A. We say that T represents a code C over A if the set of afl such rwtuples is equal to the set of codewords of C.

Although we have, by now, accumulated a considerable amount of results on the hardness of MAXIMUM-LIKELIHOOD DECODING,the broad worst-case nature of these results is still somewhat unsatisfactory. For example, as discussed in the previous subsection, it is known [83, p.77] that almost alI linear codes attain the Gilbert-Varshamov bound.

97

Given a code C over Il?gand a trellis T = (V, E, IFq) that represents ~ maximum-likelihood soft-decision decoding of C can be accomplished with exactly 21El – IV I + 1 operations. The decoding procedure that accomplishes this is known as the Viterbi algorithm [32]. Basically, the Viterbi afgorithm is a simple application of dynamic programming [68]. We refer the reader to [63, 86] for a detailed description of the Viterbi algorithm. Here, we are more concerned with the properties of the trellis itself. It is obvious that every trellis T represents a unique code, which can be determined by readng the edge labels of each path in T. However, we usually need to solve the converse

by a trellis with relative complexity s can be decoded in time 0(2nf), using the Viterbi algorithm. A simple upper bound, due to Wolf [93] and Massey [62], is given by

problem: namely, given a code C we wish to construct a trellis T which represents C. It is easy to see that there are always many different trellises representing the same code. Hence, we would generally like to construct the ‘best’ trellis representing a given code. This problem has two important aspects, discussed in what follows.

f ~ min{R, 1 – R}

This bound is true for the minimal trellis of any linear code, for any permutation of the code coordinates. The best known lower bound is due to Lafourcade and Vardy [51, 52]. For n + co this bound takes the following form: for my asymptotically good sequence of codes with rate fixed at R and relative distance d/n iixed at b, we have

If we think of a code C as a fixed set of codewords, then constructing the best trellis for C is relatively straightforward. It is shown in [65, 49] that for every linear code ~ there exists a unique, up to graph isomorphism, minimal trellis representing C. It is now known [63, 65, 87] that the minimal trellis simultaneously minimizes all the conceivable trellis complexity measures, such as 21EI – IVI + 1, IVl, I.El, maxi lV~l, aad lVil for each i = O,1, ..., n, among all possible trellis representations for C. Here is one way to construct the minimal trellis, due to Babl, Cocke, Jelinek, and Raviv [7]. hn] bean (n–k) x n parity-check matrix Let H=[hl, h2,..., for C. Then the set of vertices Vi is given by ~

= {Clhl+...

(6)

where Rm= (.) is the JPL bound (see Figure 1). The bound in (6) holds for all linear codes, and for all possible coordinate permutations. It is depicted in Figure 2 for the case R = 1 – Hz (6), that is for binary codes meeting the GilbertVarshamov bound. The best known upper bound in this

+C{h; :( Cl, Ci, ci+l,i,Cn)EC, Cn)EC fOr SOmeci+l, . . ..~CIFq}

< 0.5

fori=l,2,..., n, with the convention VO= {O}. There is a edge fkom a vertex v c Vi to a vertex v’ 6 Vi+l if and only if there exists a codeword (cl, cz,. ... c~) c C, such that Clhl +-.

+&-1hi-1

+Ciha

=

V

Clhl +...

+ Cihi + Ci+lhi+l

=

v’

(5)

0,45 0.4 0.35 0.3 0.Z5

The label of this edge is ci G lF~. Today, at leaat three other constructions of the minimal trellis are known [62, 34, 49]. In general, it is fair to say that minimal trellises for linear codes are by now well understood, and all the questions pertaining to the minimal trellis have been already answered.

0,2 0,15 0.1 o.r6

Surprisingly, however, the innocuous operation of permuting the symbols in each codeword of C according to a fixed permutation n seems to assume a fundamental significance in the context of trellis complexity, and leads to a number of challenging problems. Two codes C and T(C) that differ by a permutation of coordinates are called equivalent in coding theory. Until recently, such codes were considered aa essentially same. They certainly have the same parameters (n, k, d) and share all other important properties, ezcept trellis complexity. It turns out that a permutation of coordinates can drastically change the number of vertices in the minimal trellis representation of a given code ~ often by an exponential factor. The problem of minimizing the trellis complexity of a code via coordinate permutations, termed the “art of trellis decoding” by Massey [62] has attracted a lot of interest recently (for example, seven papers in [28] are devoted to this problem). Nevertheless, the problem remains essentially unsolved.

o 0

0.1

0.2

0,3

0.4

Figure 2: Asymptotic

0.5

0.6

0.7

0,6

0.9

1

R

bounds on trellis complexity

context was obtained by Dumer [22] and Kudryashov-Zakha-

rova [50]. They show that there exist bhry linear codes on the Gilbert-Varshamov curve R = 1 – Hz(6), whose trellis complexity satisfies 0. Question: Is there a nonzero vector z E IF; of weight ~ w, such that Hzt = O? is also NP-complete. It is easy to see that the NP-completeness of this problem would imply that computing the minimum distance of a binary linear code is NP-hard. Indeed, let C be a linear code defined by the parity-check matrix H, and let d denote the minimum distance of C. If d is known, then one can answer the question of MINIMUMDISTANCEby simply comparing d and w. On the other hand, if one can solve MINIMUMDISTANCEin polynomial time, then one can also find d in polynomial time by successively running an algorithm for MINIMUMDISTANCEwith w = 1,2,..., until the first affirmative answer is obtained.

It is interesting that the Tanner graph for a code C can be afso used for soft-decision decoding of C, using a variant of the Viterbi algorithm known as the sum-product algorithm [35, 81, 91]. It has been recently shown by severaf authors [37, 57, 58] that the sum-product algorithm is actually a version of “probability propagation” or “belief propagation” in Bayesian networks, a topic well-studied in the field of expert systems and artificial intelligence [69]. If the underlying Tanner graph is cycle-free, then the sum-product algorithm converges to the optimal maximum-likelihood solution in time 0(n2). Unfortunately, Trachtenberg and Vardy [82] have recently shown that the question “Which codes have cycle-free Tanner graphs?” has a disappointing answer: essentially, only trivial ones. If the Tanner graph has cycles, the sum-product algorithm generally regresses into iterations. It is presently not clear under which conditions these iterations converge, and if they do, whether they converge to the maximum-likelihood solution. In fact, it is easy to construct pathological counter-examples [64] where no convergence is possible. Nevertheless, iterative decoding techniques based on Tanner graphs and similar graph represen-

The MINIMUM DISTANCEproblem has a long and convoluted history. To the best of our knowledge, it was first mentioned by Welsh [90] at an Oxford Conference on Combinatorial Mathematics in 1969. In the printed version [90] of his paper, Welsh calls for an efficient algorithm to find the shortest cycle in a linear matroid over a field IF. It is easy to see that for IF = IFq this is equivalent to finding the minimum weight codeword in a linear code over IF~. Hence the NP-completeness of MINIMUMDISTANCEimplies that a polynomial-time algorithm for the problem posed by Welsh [90] is unlikely to exist. Following the publication by Berlekamp, McEliece, and van Tilborg [12] of their conjecture, the MINIMUMDISTANCEproblem was mentioned as open in the well-known book by Garey and Johnson [39, p. 280]. Three years later, it was posed by Johnson [44] as

99

an “open problem of the month” in his ongoing guide to NPcompleteness column. The problem remained open despite repeated calls for its resolution by Johnson [45] and others. Determining whether computation of the minimum Hamming distance of a linear code is NP-hard is important not only because this is a long-standing open problem. There are several more compelling reasons. First, for a host of problems in coding theory there is an easy reduction from MINIMUMDISTANCE. A few examples of such problems are presented in the next section. Hence if MINIMUMDISTANCE is computationally intractable, then all these problems are intractable as well. Secondly, as mentioned in the previous section, a polynomial-time procedure for computing the minimum distance of a linear code would imply the existence of polynomial-time randomized algorithms for constructing binary codes that attain the Gilbert-Varshamov bound. Thus it is important to know that such a polynomial-time procedure is unlikely to exist. Due to these and other reasons, the conjecture of Berlekamp, McEliece, and van Tilborg [12] sparked a remarkable amount of work, most of it unpublished. In particular, in an attempt to establish the NP-completeness of MINIMUMDISTANCE,a great number of closely related problems were proved to be NP-complete. For example, the problems of finding the maximum weight of a codeword, finding a codeword of minimum weight which is not a multiple of a given integer, determining the existence of a codeword of weight in/2j, and computing the minimum weight of a codeword which is non-zero in a specified position, were shown to be NP-hard by Ntafos and Hak~mi [67], Calderbank and Shor (see [21]), and Lobstein and Cohen [55], respectively. At least eight different problems of this kind are presently known to be NP-complete. It is pointed out in [55] that all the eight problems are tantalizingly close to MINIMUM DISTANCE,and hence provide further evidence to support the conjecture of [12] that MINIMUMDISTANCEis NP-complete. The ensemble of all these problems, however, does not suffice to prove this conjecture. Our main goal in this paper is to prove that MINIMUMDISTANCEis NP-complete. To this end, we exhibit a polynomial transformation to MINIMUMDISTANCEfrom MAX] MUM-LIKELIHOODDECODING. Thus we settle the conjecture of Berlekamp, McEliece, and van Tilborg [12] in the affirmative, using a reduction from the main result of [12].

3.1.

that the proof in [12] of the NP-completeness of MAXIMUMLIKELIHOODDECODING,based on the transformation from 3-DIMENSIONALMATCHING,uses only the special case where S=(ll. .l)t. Hence the same proof establishes that the minor variation of MAXIMUM-LIKELIHOODDECODING&j-

cussed above is also NP-complete. Next, as pointed out in [12], one may assume w.1.o.g. that the m x n matrix H at the input to MAXIMUM-LIKELIHOOD DECODINGis full-rank. This implies that the columns of H contain a basis for IFzm,and we may further assume w.1.o.g. that w ~ m – 1. Indeed, if H is full-rank and w ~ m, then the answer to the question of MAXIMUM-LIKELIHOOD DECODINGis trivially “Yes.”

We also assume w.1.o.g. that the columns of H are distinct. If this is not so, then we can form in polynomial time an m x n’ matrix H’ by retaining a single representative from each set of equal columns of H. It is easy to see that Hzt = s hm a solution of weight at most w, if and only if so does H’x~ = s, providing s # O. But the case s = O may be safely excluded from the input, as discussed above. The assumption that all the columns of H are distinct further implies that n ~ 2-. These are all the assumptions that we will need. The key idea in the transformation from MAXIMUM-LIKELIHOODDECODINGto MINIMUMDISTANCEis to regard the columns of the m x n parity-check matrix H aa elements al, az, ..., an in the finite field ll?z~. The syndrome vector s E IFzmmay be also regarded as an element ~ in lF2~. With this notation, taking into account the restrictions discussed in the foregoing paragraphs, we may re-phrase MAXIMUMLIKELIHOODDECODINGas the finite-field version of SUBSET SUM, a well-known NP-complete problem [39, p.233]. Specifically, consider the following problem: Problem:

FINITE-FIELDSUBSETSUM

Instance: An integer m ~ 2, a set of n s 2m distinct elements 0s, crz, . . ., an C IFw, a nonzero element @ c IFZWI,and a positive integer w S m – 1. Question:

Is there a subset {~,1 , cri,, . . . . ~,$} of the a2, ..., cYn}, such that CYil+.. .+cu$ = @ and b

o.5–&

~

1 0.5 – ~

>0.5–$ 2m–m3 Z(’p – 1)

where the last inequality holds for all m > 10 (and follows straightforwardly from the fact that 2m ~ m3 + mz + m + 1 for such m). Thus C’ also satisfies property P3. With both & and (7 at hand, we are ready to prove our main result.

Less formally, what we need is a sequence of binary linear codes, whose relative distance approaches the Plotkin bound d*/n* ~ 0.5, and whose rate tends to zero only polynomially fast aa function of their dimension. Furthermore, we should be able to construct each code in the sequence in polynomial time. This rules out codes that attain the Gilbert-Varshamov bound [59, p.557], as well as Zyablov codes [94], since the complexity of Zyablov’s construction [94] becomes exponential at low rates. Nevertheless, many other known constructions of asymptotically good families of low-rate codes have the desired properties. For example, all the polynomial-t ime constructions discussed in Section 2.1, with the exception of Justesen [46] concatenation, would suffice for our purposes. In what follows, however, we shall use a simple construction due to Noga Alon [2], which is concise enough to be described in one paragraph.

Theorem

5. MINIMUMDISTANCEis NP-compIete,

Proof. Clearly, MINIMUM DISTANCE is in NP. A polynomial transformation from FINITE-FIELD SUBSET SUM to MINIMUMDISTANCEcan be described as follows. Given the input al, crz,..., cr~,~ E IFz- and w to FINITE-FIELD SUBSET SUM, we answer the question of FINITE-FIELD SUBSET SUM by exhaustive search if m < 10. Otherwise, we construct in polynomial time the matrix H in (20), with H* = [F’” II ] specified by Alon’s construction. We then query an oracle for MINIMUMDISTANCEfor the existence of a codeword of weight at most Zll

~.

2m–1 Zm–1

Alon’s construction: Given an integer v ~ 2 and a non-negative integer s ~ 2V – 2, consider a concatenation of the (2”, s + 1, 2“ — s) Reed-Solomon

=

1,2”(2”–

l)&

where 11 is defined by (11) and (12), and v = [5 logz ml. By the foregoing discussion, the oracle for MINIMUMDISTANCE

104

NP-completeness of MINIMUMDISTANCEis based on a transformation from MAXIMUM-LIKELIHOODDECODINGand it is known [5, 78] that MAXIMUM-LIKELIHOODDECODING remains RTP-complete under approximation within a constant factor, it is plausible that the same should be true for M] NIMUM DISTANCE. We leave a more rigorous investigation of this question as an open problem.

will return “Yes” if and only if the answer to the question of FINITE-FIELD SUBSETSUM is “Yes.” I This concludes the proof of the conjecture of Berlekamp, McEliece, and van Tilborg [12]. In the next section, we discuss certain extensions and consequences of this result.

4.

Further

results and concluding

remarks

We note that our proof of Theorem 5 can be immediately extended to codes over an arbitrary, fixed, finite field IFq. This is based on the observation [9] that the transformation from 3-DIMENSIONALMATCHINGto MAXIMUM-LIKELIHOOD DECODING in [12] holds without change if the input to MAXIMUM-LIKELIHOODDECODINGis an m x n matrix H over IFQ,rather than a binary matrix. Given the NP-compIeteness of MAXIMUM-LIKELIHOODDECODINGover IFg, one can essentially go through the proof in the previous section, replacing each instance of 2 by q. There are a few intricate points along the way, that require some explanation. First, in rephrasing MAXIMUM-LIKELIHOODDECODINGas FINITE-FIELD SUBSET SUM, one should leave the expression ~i~ + Ct’i~+ . . . + ai~ = /3 in the question of FINITEFIELD SUBSET SUM as is, rather than ask whether @ is a linear combination of cql, ~i2, . . . . ai$. This is certainly not the question that one would be concerned with for decoding purposes, but it is legitimate in an NP-completeness proof given the specific transformation horn 3-DIMENSIONAL MATCHINGto MAXIMUM-LIKELIHOODDECODINGin [12]. (It is easy to see that a vector z 6 IFq”of weight s m/3 satisfies Hzf = (11 . . . I)t for the m x n incidence matrix H constructed in [12] only if all the m/3 nonzero positions in z are equal to 1.) Secondly, the bound in Lemma4 becomes d < n*d#

q

First, we note that determining whether a given linear code is MDS is NP-hard: namely, the following decision problem Problem:

MDS CODE

Instance: A prime p z 2, positive integers m and r, andan Txprn matrix H over lFP_. Question: Is there a nonzero vector x of length pm over lFPm,such that Hxt = O and wt(x) < T? is NP-complete. The fact that MDS CODE is NP-hard, even for p = 2, follows immediately from Lemma 1. The NPcompleteness of MDS CODE then follows from the observation that the phrase “of weight < w“ in the question of MAXIMUM-LIKELIHOODDECODINGcan be changed to the phrase “of weight exactly w,” as discussed in Section 3.2. As another example, consider the problem of determining the trellis complexity of a linear code. More precisely, the computational task is to find a coordinate permutation that minimizes the number of vertices at a given time z in the minimal trellis for a binary linear code. For more details on this problem, and on trellises in general, see Section 2.2. It is not difficult to see that the corresponding decision problem [43] may be posed as follows:

m—1

m–l+qm–2.

9

Problem:

..+q+l

I)q”-l(q”

–S)

PARTITIONRANK

Instance: A binary k x n matrix H, and positive integers i and w.

and one has to modify equation (24) accordingly. Fortunately, Alon’s construction [2] works in this case as well. Here, the columns of G* would be indexed by x, y E lF~., so that equation (26) remains without change, equation (25) becomes n“ = q“ (q” – 1), and equation (27) becomes d* > (q–

Another immediate consequence of our proof is that certain useful computational tasks in coding theory are NP-hard, aa there is an easy reduction from MINIMUMDISTANCEto each of these tasks. There is a large number of computational problems of this kind; we will give just three examples here.

Question: Is there a column permutation that takes H into a matrix H’ = [Ai 113m_a],such that Aa is an k x i matrix and rarrk(Ai) + rank(Bn-i) < w? Horn and Kschischang [43] recently proved that this problem is NP-complete, using an ingenious and elaborate transformation from SIMPLEMAX CUT [39, p.210] which spans over five pages. On the other hand, given the NP-completeness of MINIMUMDISTANCE,this result can be established in a few lines as follows. First, we observe that the least integer i for which

(28)

The key observation in the proof of (28) is as follows: if 0

As a third example, we mention the problem of finding the largest subcode with a prescribed contraction index [88]. Namely, given a k x n generator matrix for a binary linear code C, and a positive integer A, we wish to find the largest subcode ~ ~ C which has a generator matrix with at most A + dlm C’ distinct columns. This problem is of importance in soft-decision and majority-logic decoding (see [88] for an extensive treatment ), and it is possible to show that it is NP-hard using a transformation from MINIMUMDISTANCE. The proof of this is a bit tedious, and we omit the details. Finally, we would like to mention two important problems in coding theory, for which we do not have a polynomial transformation from MINIMUMDISTANCE,but believe that it should be possible to find one.

Question: Is there a nonzero vector z in A, such that IIz112< VJ? is NP-complete. In spite of a considerable amount of work (see [5] for a recent survey), the proof of this conjecture remains elusive. Arora, Babai, Stern, and Sweedyk [5] classify this as a “major open problem.” Moreover, this conjecture becomes particularly significant in view of the celebrated recent result of Ajt ai [1], who showed that hard instantes of the SHORTESTVECTOR problem can be efficiently generated, provided that the problem itself is hard. Thus the NP-completeness of SHORTESTVECTOR is of utmost importance in cryptographic applications. We notice that if the phrase “]IxI12< w“ in the question of SHORTESTVECTOR is replaced with “IIzI12= w,” the problem becomes NP-complete. This result, which was recently established by Fellows and Vardy [29], shows that the counterpart of the WEIGHT DISTRIBUTIONproblem, which was proved to be NP-complete for binary linear codes in [12], is NP-complete for lattices. It is, therefore, all the more plausible that the counterpart of MINIMUMDISTANCEshould be also NP-complete for lattices.

The first problem is that of bounded-distance decoding of binary linear codes. While the intractability of maximumlikelihood decoding has been thoroughly studied (see, for example [5, 9, 12, 16, 20, 78], and the discussion in Section 2.2), most of the decoders used in practice are bounded-distance decoders. It is still not known whether bounded-distance decoding is NP-hard for the general class of binary linear

106

[11]E.R. BERLEKAMP,Algebmic Coding Theory, New York:

Intuitively, finding the shortest vector in a lattice is at least as “difficult” as finding the minimum-weight vector in a binary linear code. Thus it is reasonable to suggest that there would be a polynomial transformation from MINIMUMDISTANCEto the SHORTESTVECTOR. Specifically, we pose the following problem: given a binary linear code C construct, in polynomial time, a lattice A ~ Z“ so that the minimum dktance of C can be determined from the minimum norm of A. In view of our main result, solving this problem would amount to proving that SHORTESTVECTOR is NP-complete.

McGraw-Hill, 1968. [12] E.R. BERLEKAMP, R.J. MCELIECE, and H.C. A. VAN TILBORG, On the inherent intractability of certain coding problems, IEEE !lkans. Inform. Theory, vol. 24, pp. 384-386, 1978. [13] C. BERROU,A. GLAVIEUX,and P. THITIMAJSH]MA,Nar Shannon limit error-correcting coding and decoding: turbo codes, in Proc. IEEE Int. C’onf. on Communications, pp. 1064-1070, Geneva, Switzerland, 1993. [14] M. BLAUM, J. BRUCK, and A. VARDY, On MDS codes and alterants over certain rings, in Proc. 900-th Meeting, American Math. Sot., Chicago, IL., March 1995.

Acknowledgement. I would like to acknowledge helpful dkicusaions with Noga Alon, Alexander Barg, Yoram Bresler, Shuki Bruck, Ilya Dumer, Herbert Edelsbrunner, Mike Fellows, Nabil Kahale, Moni Naor, Ronny Roth, Dilip Sarwate, Leonard Schulman, and Vijay Vazirani. I am especially indebted to Noga Alon for referring me to his construction, which is used in Section 3. Finally, I would like to thank Hagit Itzkowitz for her invaluable help.

[15] M. BLAUM,J. BRUCK, and A. VAR~Y, MDS array codes with independent parity symbols, IEEE Trans. Inform. Theory, vol. 42, pp. 529-542, 1996. [16] J. BRUCK and M. NAOR, The hardness of decoding linear codes with preprocessing, IEEE Ihns. Inform. Theory, vol. 36, pp. 381–385, 1990.

Coding [17] G.C. CLARK and J.B. CAIN, Error-Correction for Digital Communications, New York: Plenum Press, 1981.

References p] M. AJTAI, Generating hard instances of lattice problems, in Proc. 28-th Annual ACM Symp. Theory of Computing, pp. 99-108, Philadelphia, PA, May 1996.

[18] R.G. DOWNEY and M.R. FELLOWS, Fixed parameter tractability and completeness I: b~ic theory, SIAM J. Comput., vol. 24, pp. 873-921, 1995.

[2] N. ALON, Packings with large minimum kissing numbers, preprint, 1996.

[19] R.G. DOWNEY and M.R. FELLOWS, Fixed parameter tractability and completeness II: completeness for W[l], Theoret. Computer Sci. A vol. 141 pp. 109-131, 1995.

[31 N. ALON, J. BRUCK, J. NAOR, M. NAOR, and R. ROTH, Construction of asymptotically good low-rate errorcorrecting codes through pseudo-random graphs, IEEE Trans. Inform. Theory, vol. 38, pp. 509-516, 1992.

[20] R.G. DOWNEY,M. R. FELLOWS,A. VARDY, and G. WHITTLE, On the parametrized complexity of certain fundamental problems in coding theory, preprint in preparation, 1997.

[4] S. ARORA, Probabilistic checking of proofs and hardness of approximation problems, Ph.D. thesis, University of California, Berkeley, CA., 1994.

[21] P. DIACONK and R.L. GRAHAM, The Radon transform on 73$, Pacific J. Math., vol. 118, pp. 176–185, 1985.

[5] S. ARORA, L. BABA1, J. STERN, and Z. SWEEDYK,The hardness of approximate optima in lattices, codes, and systems of linear equations, in Proc. 34-th AnnuaJ Symp. Found. Computer Science, pp. 724-733, Palo Alto, CA, 1993.

[22] 1.1. DUMER, On complexity of maximum-likelihood decoding of the best concatenated codes, in Proc. 8-th AJ1Union Conf. on Coding Theory and Information Theory, Moscow-Kuibishev, pp. 66-69, 1981, (in Russian).

[23] 1.1. DUMER, Concatenated codes and their generalizations, to appear in Handbook of Coding Theory,

[6] S. ARORA and C. LUND, Hardness of approximations, in Approximation Algorithms for NP-Hard Problems, D.S. Hochbaum (Ed.), Boston: PWS Publishing Co., pp. 399-446, 1997.

R.A. Brualdi, W.C. Huffman, V. Plesa (Ed.), dam: Elaevier.

Amster-

[24] S. EVEN and Y. YACOBI, Cryptography and NP-completeness, Lect. Notes Comp. Science, vol. 85, pp. 195207, Springer-Verlag, 1982.

{7] L.R. BAHL, J. COCKE, F. JELtNEK,and J. RAVtV, Optimal decodhg of linear codes for minimizing symbol error rate, IEEE Trans. Inform. Theory, vol. 20, pp. 284– 287, 1974.

[25] J. FANG, G.D. COHEN, PH. GODLEWSKI, and G. BATTAIL, On the inherent intractability of soft decision decoding of linear codes, Lect. Notea Comp. Science, vol. 311, pp. 141–149, Springer-Verlag, 1988.

[8] R. BALASUBRAMANIAN,M. FELLOWS, and V. RAMAN, An improved fixed-parameter algorithm for vertex cover, to appear, 1997.

[26] R.M. FANO, A heuristic discussion of probabilistic decoding, IEEE Tkans. Inform. Theory, vol. 9, pp. 64–73, 1963.

[9] A. BARG, Some new NP-complete coding problems, Prob~. Peredachi Informatsii, vol. 30, pp. 23-28, 1994, (in Russian).

[27] J. FEIGENBAUM, The use of codkig theory in computational complexity, in Proc. Symp. Appl. Math, A.R. Cafderbank (Ed.), Providence, RI: AMS Press, pp. 203-229, 1995.

[10] A. BARG, Complexity issues in coding, survey chapter to appear in Handbook of Coding Theo~, R.A. Brualdi, C. Huffman, and V. Pless (Ed.), Amsterdam: Elsevier.

107

[28] J, FmGmw3Auhi, G.D. FORNEY, B. MARCUS, R.J. Mc ELIECE, and A. VARDY, Special issue on “Codes and Complexity,” IEEE Trans. Inform. Theory, vol. 42, November 1996,

[46] J. JUSTESEN, A class of constructive asymptotically good algebraic codes, IEEE Thans. Inform. Theory, vol. 18, pp. 652–656, 1972. [47] R. KARP and R. LIPTON, Some connections between nonuniform and uniform complexity classes, in Proc. 12-th Annual Symp. Theory of Computing, pp. 302309, 1980.

[29] M.R. FELLOWS and A. VARDY, The hardness of theta series in lattices, preprint in preparation, 1997. [30] G.-L. FENG and K.K. TZENG, A new procedure for de-

coding cyclic and BCH codes up to actual minimum distance, IEEE Trans. Inform. Theory, vol. 40, pp. 1364–

[48] L. KHACHIYAN,On the complexity of approximating extremal determinants in matrices, J. Complexity, vol. 11, pp. 138-153, 1995.

1374, 1994. [31] G .D. FORNEY, JR., Concatenated MA: M.I.T. Press, 1966.

Codes, Cambridge,

[49] F.R. KSCHISCHANGand V. SOROKINE, On the trellis structure of block codes, IEEE Tkans. Inform. Theory, vol. 41, pp. 1924–1937, 1995.

[32] G.D. FORNEY,JR., The Viterbi algorithm, Proc. IEEE, VO].61, pp. 268–278, 1973.

[50] B.D. KUDRYASHOVand T.G. ZAKHAROVA,Block codes from convolutional codes, Problemy Peredachi Infor-

[33] G.D. FORNEY,JR., Convolutional codes III: sequential decoding, Inform. Control, vol. 25, pp. 267-297, 1974.

matsii, vol. 25, pp. 98–102, 1989, (in Russian). [51] A. LAFOURCADEand A. VARDY, Asymptotically good codes have infinite trellis complexity, IEEE Trans. Inform. Theory, vol. 41, pp. 555-559, 1995.

[34] G.D. FORNEY,JR., Coset codes II: Binary lattices and related codes, IEEE ‘Ikms. Inform. Theory, vol. 34,

pp. 1152–1187, 1988.

[52] A. LAFOURCADEand A. VARDY, Lower bounds on trellis complexity of block codes, IEEE 2hns. Inform. Theory, vol. 41, pp. 1938–1954, 1995.

[35] G.D. FORNEY,JR., The forward-backward algorithm, in Proc. 34-th Allerton Conference on Comm., Control, and Computing, Monticello, IL., pp. 432-446, October 1996.

[53] L.A. LEWN, Average case completeness, put., vol. 15, pp. 285-286, 1986.

[36] G.D. FORNEY,JR. and A. VARDY, Generalized minimum distance decoding of Euclidean-space codes and lattices, IEEE Trans. Inform. Theory, vol. 42, pp. 19922026, 1996.

SIAM J. Com-

[54] S. LIN and E.J. WELDON, Long BCH codes are bad, Inform. and Control, vol. 11, pp. 445-451, 1967. [55] A. LOBSTEINand G.D. COHEN, Sur la complexit6 d’un prob16me de codage, Theoretical Informatics AppL, VO1.21, pp. 25–32, 1987.

[37] B.J. FREY and F.R. KSCHtSCHANG,Probability propagation and iterative decoding, in Proc. 34-th Allerton Conference on Comm., Control, and Computing, Monticello, IL., pp. 482–493, October 1996. [38] R.G. GALLAGER,Low Density Parity-Check Codes, Cambridge: M.I.T. Pressl 1962.

[56] B. L6PEZ JIMENEZ, Plane models of Drinfeld modular curves, Ph.D. thesis, University of Complutense, Madrid, March 1996.

[39] M.R. GAREY and D.S. JOHNSON,Computers and htractability: A Guide to the Theoy of NP- Completeness, San l+ancisco: Freeman, 1979.

[57] D.J.C. MACKAY and R.M. NEAL, Near Shannon limit performance of low-density parity-check codes, Elect. Lett., to appear, 1997.

[40] C.R.P. HARTMANand K.K. TZENG, Decoding beyond

[58] D.J.C. MACKAY and R.M. NEAL, Good error-correcting codes based on very sparse matrices, IEEE Tkans. Inform. Theory, submitted for publication, 1996.

the BC!H bound using multiple sets of syndrome sequences, IEEE Tkns. Inform. Theory, vol. 20, pp. 292– 295, 1974.

[59] F.J. MACWILLIAMS and N.J.A. SLOANE, The Theory of Error Correcting Codes, Amsterdam: North-Holland, 1977.

[41] D.S. HOCHBAUM, (Editor), Approximation Algorithms for NP-Hard Problems, Boston: PWS Publishing Co., 1997.

[60] Yu.I. MANIN and S.G. VL.kDUTS, Linear codes and modular curves, J. Soviet Math., vol. 30, pp. 261 1–2643, 1985, (in Russian).

[42] T. H@HOLDT and R. PELLIKAAN, On the decoding of algebraic-geometric codes, IEEE Tkans. Inform. Theory, vol. 41, pp. 1589–1614, 1995.

[61] J.L. MASSEY, Shift-register synthesis and BCH decoding, IEEE fians. Inform. Theory, vol. 15, pp. 122–127, 1969.

[43] G.B. HORN and F.R. KSCHISCHANG,On the intractability of permuting a block code to minimize trellis complexity, IEEE Zkns. Inform. Theory, vol. 42, pp. 2042– 2048, 1996.

[62] J.L. MASSEY, Foundation and methods of channel encodhg, Proc. ht. Conf information Theory and Systems, NTG-Fachberichte, Berlin, 1978.

[44] D.S. JOHNSON,The NP-completeness column: An ongoing guide, J. Algorithms, vol. 3, pp. 182–195, 1982.

[63] R.J. MCELIECE, On the BCJR trellis for linear block codes, IEEE Tkans. Inform. Theory, vol. 42, pp. 1072– 1092, 1996.

[45] D.S. JOHNSON,The NP-completeness column: An ongoing guide, J. Algorithms, vol. 7, pp. 584-601, 1986.

108

[80] M. SUDAN, Decoding of Reed-Solomon codes beyond the error-correction bound, J. Complexity, vol. 1, 1997, to appear.

[64] R.J. MCELtECE, E. R. RoDEM]cH, and J.-F. CHENG, The turbo decision algorithm, in Proc. 33-rd Allerton Conference on Comm., Control, and Computing, Monticello, IL., pp. 366-379, October 1995. [65] D.J. MUDER, Minimal trellises for block codes, IEEE Tkans. Inform. Theory, vol. 34, pp. 1049-1053, 1988.

[81] R.M. TANNER,A recursive approach to low-complexity codes, IEEE Trans. Inform. Theory, vol. 27, pp. 533– 547, 1981.

[66] T. MUIR, Tkentise on the Theory of Determinants, York: Dover, 1960.

New

[82] A. TRACHTENBERGand A. VARDY, Which codes have cycle-free Tanner graphs?, preprint, 1997.

[67] S.C. NTAFOS and S.L. HAKIMI, On the complexity of some coding problems, IEEE Trans. Inform. Theory, vol. 27, pp. 794–796, 1981.

[83] M.A. TSFASMAN and S.G. VLADUTS, Algebmic Geometry Codes, Dodrecht: Kluwer Academic, 1991.

[68] J.K. OMURA, On the Viterbi decoding algorithm, IEEE ?kans. Inform. Theory, vol. 15, pp. 177-179, 1969.

[84] M.A. TSFASMAN, S.G. VLADUTS, and T. ZINK, Modular curves, Shimura curves, and Goppa codes better than the Varshamov-Gilbert bound, Math. IVachrichten, vol. 104, pp. 13–28, 1982.

[69] J. PEARL, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA: Kaufmann, 1988.

[85] P. VAN EMDE BOAS, Another

NP-complete partition problem and the complexity of computing short vectors in a lattice, Tech. Report 81-04, Dept. of Mathematics, Univ. of Amsterdam, 1980.

[70] R. PELLIKAAN, Asymptotically good sequences of curves and codes, in Proc. 34-th Allerton Conference on Comm., Control, and Computing, Monticello, IL., pp. 276–285, October 1996.

[86] A. VARDY, Tkellis structure of block codes, to appear in Handbook of Coding Theory, R. Brualdi, W.C. Huifman, and V. Pless (Ed.), Amsterdam: Elsevier.

[71] 1.S. REED and G. SOLOMON, Polynomial codes over certain finite fields, SIAM J. Appl. Math., vol. 8, pp. 300304, 1960.

[87] A. VARDY and F.R. KSCHISCHANG, Proof of a conjecture of McEliece regarding the expansion index of the minimal trellis, IEEE Tkans. Inform. Theory, vol. 42, pp. 2027-2033, 1996.

[72] R.M. ROTH and A. LEMPEL, A construction of nonReed-Solomon type MDS codes, IEEE Tkans. Inform. Theory, vol. 35, pp. 655-657, 1989.

[88] A. VARIX, J. SNYDERS, and Y. BE’ERY, Bounds on the dimension of codes and subcodes with prescribed contraction index, Linear Algebra Appl., vol. 142, pp. 237– 261, 1990.

[73] C.E. SEIANNON,A mathematical theory of communication, Bell Syst. Tech. J., vol. 27, pp. 379-423 and pp. 623-656, 1948.

[89] S.G. VLADUTS, G.L. KATSMAN, and M.A. TSFASMAN, Modular curves and codes with polynomial complexity of construction, Problemy Peredachi In formatsii, vol. 20, pp. 47–55, 1984, (in Russian).

[74] B.-Z. SHEN, A Justesen construction of binary concatenated codes that asymptotically meet the Zyablov bound for low rate, IEEE Tkans. Inform. Theory, VOI.39, pp. 239–242, 1993.

Combinatorial problems in matroid theory, pp. 291–307 in Combinatorial Mathematics and its Applications, D.J.A. Welsh (Ed.), London: Aca-

[90] D.J.A. WELSH,

[75] V. SHOUP,New algorithms for finding irreducible polynomials over finite fields, Math. Computation, vol. 54, pp. 435-447, 1990.

demic Press, 1971.

[76] M. SIPSERand D.A. SPIELMAN,Expander codes, IEEE Tkans. Inform. Theory, vol. 42, pp. 1710–1722, 1996.

[91] N. WIBERG, Codes and decoding on general graphs, Ph.D. thesis, University of Linkoping, Sweden, 1996.

[77] D.A. SPIELMAN,Linear-time encodable and decodable codes, IEEE Ihms. Inform. Theory, vol. 42, pp. 1723– 1731, 1996.

[92] N. WIBERG, H.-A. LOELIGER, and R. KOTTER, Codes and iterative decoding on general graphs, Euro. Trans. Telecommun., vol. 6, pp. 513-526, 1995.

[78] J. STERN, Approximating the number of error locations within a constant ratio is NP-complete, Lect. Notes Comp. Science, vol. 673, pp. 325-331, Springer, 1993.

decoding of [93] J.K. WOLF, Efficient maximum-likelihood linear block codes using a trellis, IEEE Trans. Inform. Theory, vol. 24, pp. 76-80, 1978.

[79] M. SUDAN,Efficient checking of polynomials and proofs and the hardness of approximation problems, Ph.D. thesis, University of California, Berkeley, CA., 1992.

[94] V.V. ZYABLOV, An estimate of the complexity of constructing binary linear concatenated codes, Problemy Peredachi Informatsii, vol. 7, pp. 5-13, 1971.

109