Systematic Binary Deterministic Rateless Codes

ISIT 2008, Toronto, Canada, July 6 - 11, 2008

Systematic Binary Deterministic Rateless Codes Ming Xiao

Tor Aulin

Muriel M´edard

ACCESS Linnaeus Center Department of Computer Engineering Department of Electrical School of Electrical Engineering, Chalmers University of Technology Engineering and Computer Science, Royal Institute of Technology, Sweden Gothenburg, 412 96 Sweden Massachusetts Institute of Technology, USA [email protected] [email protected] [email protected]

Abstract— We investigate a systematic construction of binary deterministic rateless codes (BDRCs). The codes are for networks with erasure channels. With a maximum distance separable (MDS) property, non-systematic BDRCs were first proposed in [1] with encoding complexity O(K), and decoding complexity is O(K 2 ). Here K is the length of information bits. To reduce complexity, we study systematic-BDRCs (SBDRCs). For SBDRCs, the source first transmits m − 1 uncoded blocks, where m is the number of source blocks. Then, the source produces and transmits coded blocks in a rateless way. These coded blocks are produced using only cyclic-shift and XOR (exclusive or). The SBDRCs can use a large number of information blocks (potentially infinite m). On receiving any m distinct blocks (uncoded or coded), a sink can rebuild the source. The SBDRCs have encoding complexity O(K), and decoding complexity O(2 K 2 ), where  is the source-to-sink block erasure probability.

I. I NTRODUCTION Error control for networks with erasure channels has been an active research topic for many years. A widely used protocol is error detection and retransmission [2], [3]. Another type of error-controlling is forward error coding (FEC), which needs no feedback from receivers. One of the most famous FECs for erasure channels is Reed-Solomon (RS) codes [4]. More recently, rateless codes ([5], [6]) based on digital fountain principles ([7]) have attracted many research efforts and applications. The main advantages of fountain codes over RS codes are low complexity and adaptation to varying environment/channels. The low complexity is because fountain codes are binary and based on sparse matrices ([5], [6]), while RS codes are normally nonbinary. As suggested by the name, rateless codes do not predetermine the degree of redundancy (code rate), and can form arbitrary number of code blocks. Thus, fountain codes are suitable to dynamic environments. However, RS codes also have clear benefits, compared to fountain codes. For instance, RS codes are maximum distance separable (MDS). Thus, the sink can recover source information from the minimum possible received blocks. As random codes, fountain codes cannot be MDS, and some redundant blocks are needed in sinks to rebuild the source. The redundancy is high when source information is finite. Also, the fountain codes often use a large number of blocks. This may not be preferable for delay-sensitive applications. This material is based upon work supported by the National Science Foundation under Grant No. CCR-0325496.

978-1-4244-2571-6/08/$25.00 ©2008 IEEE

From these observations, we see that it is valuable to have rateless codes with an MDS property. The codes work very efficiently for finite blocks. Further, the codes should have low complexity. In our earlier work [1], we propose binary deterministic rateless codes (BDRCs) with most of these merits. The BDRCs have the MDS property in very high probability. Thus, the codes have good performance with a finite length, and favorable to time-sensitive applications. Also, the codes are suitable to dynamic environment, since they are rateless. The BDRCs have encoding complexity O(K), and decoding complexity O(K 2 ), where K is the length of information bits. The encoder only uses cyclic-shift and XOR (exclusive or). Thus, the codes have low complexity. The complexity of the BDRCs can be further reduced, especially for the decoders. One approach is to directly output systematic/uncoded blocks. However, there will be more designing constraints using systematic blocks, especially when the number of systematic blocks becomes quite large. The organization of the paper is as follows: In Section II, we review the nonsystematic BDRCs and introduce notations. In Section III, we investigate construction approaches of systematic BDRCs. In Section IV, we discuss properties of the codes. Finally, the conclusion is given in Section V. II. N ONSYSTEMATIC BDRC S Algorithm 2.1: (Encoding

approach

of

coded

blocks)

1: Pad-zero: In the end of each I i (i = 1, · · · , m), one zero is

appended in blocks. The resulting blocks are Iˆi .

2: Cyclic-shift. For the jth coded block (j = 1, · · · , N ), cyclic-

shift Iˆi (j − 1)(i − 1) times (bits). The resulting block is denoted as  I j,i . 3: Final-bit removal. The final bit of  I j,i is discarded. 4: Binary add (XOR): The jth (j = 1, · · · , N ) coded block C j is formed by binary adding (XOR) all  I j,i (i = 1, · · · , m).

We first review the nonsystematic BDRCs proposed in [1], Chapter 4, [8], since part of results and notations will be used for construction of systematic-BDRCs (SBDRCs). The information bits in the source are put into m blocks. We denote them as I 1 , I 2 , · · · , I m (row vectors). The length of each block is p, and p + 1 is a prime. Here we assume that

2066

ISIT 2008, Toronto, Canada, July 6 - 11, 2008

source information bits are long enough to form these blocks. The jth (j = 1, · · · , N , and m ≤ N ≤ p + 1) coded block C j are produced by Algorithm 2.1. For encoding, we only use cyclic-shift and XOR. Both of them have low complexity and can be easily implemented by a hardware ([2], [9]). C j also has a length p. The source continuously produces and sends out coded block. Each block is sent immediately after being produced. In the encoding process, coded blocks are produced in a block-by-block fashion rather than simultaneously. This property is exploited for our rateless codes. Clearly, BDRCs can produce finite distinct blocks (totaling N blocks). If all these N blocks are sent out and some sinks do not receive enough information (m blocks), the source retransmits sent blocks until all sinks receive enough information. Yet, we show that the probability of retransmitting is extremely low for nontrivial p and m [1], Chapter 4, [8]. By Algorithm 2.1, Cj = M jI

Algorithm 3.1: (Systematic binary deterministic rateless codes: SBDRCs) • • •

•

• •

The source organizes m × p information bits into m blocks with p bits for each. The uncoded blocks are marked as B1 , B2 , · · · , Bm . m − 1 uncoded blocks (B1 to Bm−1 ) are sent to the sinks. Produce and send out coded blocks. Using Algorithm 2.1, the source continuously produces and sends code blocks. One block is sent out immediate after it is produced. The process continues until the termination condition is satisfied. The blocks are marked as Bm+1 , Bm+2 , · · · , Bm+N ∗ and saved. The sink receiving m blocks checks if m blocks are distinct. If they are, the sink sends back a termination signal to the source and decodes information bits. The termination condition is satisfied if the source gets termination signals from all sinks. If the block Bm+N ∗ is sent out and the termination condition is still not satisfied, the source retransmits the saved block (except the block Bm ): starting from B1 , then B2 , · · · , Bm−1 , Bm+1 , · · · . This process repeats until the termination condition is satisfied.

(1)

where I = [I 1 , I 2 , · · · , I m ]T is a pm × 1 column vector denoting the source information bits. M j = M j,1 , M j,2 , · · · , M j,m is a p × pm matrix denoting the encoding matrix for C j . M j,i (i = 1, 2, · · · , m) is ⎛ ⎞ 0 ··· 0 ··· 0 1 0 ··· 0 ⎜ 0 ··· 0 ··· 0 0 1 ··· 0 ⎟ ⎜ ⎟ ⎜ ⎟ ··· ··· M j,i = ⎜ . ⎟ ⎝ 0 ··· 1 0 ··· 0 0 ··· 0 ⎠ 0 ··· 0 1 · · · 0 0 · · · 0 p×p (2) Each column or row of M j,i has at most one 1, and other elements are zeros. Let mk,t (k, t = 1, · · · , p) be the kth row and tth column element of M j,i ,  1 t = ((i − 1)(j − 1) + k) mod(p + 1), mk,t = (3) 0 otherwise.

exactly one 1 and all other elements are zero by (3). Note that is a cyclic-shifting matrix. A (p + 1) × 1 column vector M j,i . is cyclic-shifted (i − 1)(j − 1) times if multiplied with M j,i have the same elements except in the final M j,i and M j,i . We denote σ as an one-time cyclicrow and column of M j,i = σ (i−1)(j−1)mod(p+1) . σ k(p+1) is shifting matrix. Thus, M j,i an identity matrix for any integer k. In the following, we shall omit the mod(p + 1) operation and use (i − 1)(j − 1). Then, ⎞ ⎛ 0 σ σ (z1 −1) · · · σ (z1 −1)(m−1) ⎜ σ 0 σ (z2 −1) · · · σ (z2 −1)(m−1) ⎟ ⎟ .(5)  ⎜ M s,z ⎠ ⎝ ··· 0 (zm −1) (zm −1)(m−1) σ σ ··· σ

From (3), we can see that the kth row of M j,i is all-zero if ((i − 1)(j − 1) + k) mod(p + 1) = 0. On receiving any distinct m coded blocks, a sink can reproduce the source. The proof is as following. We assume m blocks received in a sink z as C z1 , C z2 , · · · , C zm . These m blocks can be any m of N possible blocks. Thus, the generator matrix for them is ⎞ ⎛ M z1 ,1 M z1 ,2 · · · M z1 ,m ⎜ M z ,1 M z ,2 · · · M z ,m ⎟ 2 2 2 ⎟ . (4) Ms,z = ⎜ ⎠ ⎝ ··· M zm ,1 M zm ,2 · · · M zm ,m pm×pm

is a (p+1)m×(p+1)m matrix. σ 0 is a (p+1)×(p+1) M s,z identity matrix. Without loss of generality, we assume 1 ≤ z1 < z2 < · · · < zm ≤ N . Lemma 1. The rank of M s,z is pm + 1. Proof: We omit the proof because of space constraints. Interested readers are referred to Lemma 2.2, [10], Lemma 1, [1], Lemma 4.1, [8] for detailed proofs. Q.E.D. Theorem 1. The rank of Ms,z is pm (full-rank). Proof: We omit the proof because of space constraints. Interested readers are referred to Theorem 1, [1], Theorem 4.1, [8] for detailed proofs. Q.E.D. We also showed in [1] and Chapter 4, [8] that the encoding and decoding complexity are O(K) and O(K 2 ), respectively, where K is the length of information bits.

The decoding seeks to recover the m information blocks from the received m coded blocks. In the encoding and decoding process, all operations are performed in GF(2). The validity of the code then hinges on the invertibility of Ms,z for any m distinct coded blocks. To show that Ms,z is invertible (full-rank), we introduce another matrix M j,i with p + 1 rows and p + 1 columns. The elements of M j,i is (m  k,t , k, t = 1, 2, · · · , p + 1) are gotten by (3). Since M j,i has a (p + 1) × (p + 1) matrix, each row or column of M j,i

In addition to the coded blocks produced by Algorithm 2.1, SBDRCs also use uncoded blocks as outputs. A total of m − 1 uncoded blocks is used for SBDRCs. The other N ∗ (> 1) coded blocks are formed by Algorithm 2.1. N ∗ is a designed parameter, and its range will be discussed later. Then, there is a total of N = N ∗ + m − 1 distinct blocks. Any m of these N blocks can reproduce the m source information blocks at the sink. The SBDRCs work according to Algorithm 3.1.

III. S YSTEMATIC BDRC S

2067

ISIT 2008, Toronto, Canada, July 6 - 11, 2008

For SBDRCs, the generator matrices of any distinct m blocks (coded or uncoded) in a sink (say, sink z) are ⎞ ⎛ 0 ··· U z1 · · · 0 ⎟ ⎜ ··· ⎟ ⎜ ⎟ ⎜ U zk ··· 0 0 ⎟, Mis,s = ⎜ ⎟ ⎜ M z ,1 M z ,2 · · · M zk+1 ,m ⎟ k+1 k+1 ⎜ ⎠ ⎝ ··· M zm ,2 ··· M zm ,m M zm ,1 (6) where U z1 , · · · , U zk are p × p identity matrices, and 0 is the p × p all-zero matrix. k is an integer with k < m. In Mis,s , k upper identity matrices are for uncoded systematic blocks, and m − k bottom rows of submatrices are for nonsystematic blocks. Note that there are at most m − 1 uncoded blocks for Mis,s , i.e., k ≤ m − 1. The reason is given after Theorem 2. Similarly to nonsystematic BDRCs, a key property for SBDRCs is to have full-rank Mis,s for any distinct m blocks. Different from Ms,z , Mis,s is not full-rank for all primes p+1. The condition for Mis,s being full-rank shall be discussed later. Mis,s is similar (with a slight difference) to the parity matrix of binary MDS array codes, e.g., [9], [10], [11]. We use Mis,s as the generator matrix ([2], [12]). Thus, the SBDRCs are similar to the dual codes of array codes. Array codes are systematic codes. There is no limitation on the number of systematic blocks for array codes. However, the number of parity blocks of array codes is small. At most eight parity blocks are verified in [9]. To produce an arbitrary amount of parity blocks for MDS array codes is still an open problem, for our knowledge. Below, we first show the condition for Mis,s being fullrank. The condition can be directly used to find the SBDRCs with finite m. Then we show that for potentially infinite m, there are SBDRCs if p satisfies certain constraints. The same , we extend Mi m rows and m as extending Ms,z to M s,z s,s columns (each submatrix 1 row and 1 column). The elements of the submatrices for nonsystematic parts are evaluated by i (3). Then, we denote the resulting matrix M s,s (It is the parity matrix of MDS array codes). Assuming that σ k is homomorphic ([2], [12]) to the polynomials xk , we use H is i . Thus, to denote the homomorphic matrix ([2], [12]) of M s,s ⎛ ⎞ 0 ··· 1··· 0 ⎜ ⎟ ··· ⎜ ⎟ ⎜ ⎟ 0 1 ··· 0 ⎟ (7) H is = ⎜ (zk+1 −1)(m−1) ⎟ , ⎜ 1 x(zk+1 −1) · · · x ⎜ ⎟ ⎝ ⎠ ··· (zm −1) (zm −1)(m−1) ··· x 1 x where k < m, and zk+1 < zk+2 < · · · < zm ≤ N ∗ . Note that N ∗ is only constrained by p (to be discussed), and there is no direct relation between N ∗ and m. The i nonsystematic part of Ms,s has (m − k) × m submatrices. i if they have 1s in the Then, we delete the columns from M s,s systematic part (upper k(p + 1) rows). Thus, we delete totally k × (p + 1) columns. Then, we delete all the rows in the systematic parts. The remaining matrix in the nonsystematic

part is a (m − k)(p + 1) × (m − k)(p + 1) matrix. We denote . The polynomial this remaining nonsystematic matrix as M s,1 matrix homomorphic to Ms,1 is denoted as Hˆs . We denote Mp (x)  1 + x+ x2 + · · · + xp . Then we have the following lemma used to show the condition for Mis,s being full-rank. Lemma 2. Hˆs is nonsingular over the polynomial ring modulo-Mp (x) if and only if H is is nonsingular over the polynomial ring modulo-Mp (x). Proof: The proof is relatively straightforward, since Hˆs has the same determinant as H is over the polynomial ring moduloMp (x). Note that there is only one 1 in the upper k rows of H is . We omit details for space reasons. The interested readers are referred to Chapter 4, [8]. Q.E.D. With Lemma 2, we show a condition for full-rank Mis,s . Theorem 2. If H is is nonsingular over the polynomial ring modulo-Mp (x), Mis,s is full-rank (for any distinct m blocks). Proof: Since H is is nonsingular over the polynomial ring modulo-Mp (x), Hˆs is also nonsingular by Lemma 2. With a similar approach to Lemma 1, we can show that the rank ˆ of M s,1 is (m − k)p + 1, since H s is the homomorphic polynomial matrix of Ms,1 . By removing the k(p + 1)th , we get (k = 1, 2, · · · , m − k) rows and columns of M s,1 . In the a (m − k)p × (m − k)p matrix, and denote it as M s,2 is same way as Theorem 1, we can show that the rank of M s,2 is a full rank matrix. Then, we extend (m − k)p. Thus, M s,2 M s,2 k × p rows and k × p columns. The resulting matrix .M is denoted as M s,3 s,3 has mp rows and mp columns, as shown in (8). Then the extended kp rows or kp columns of M s,3 has one 1 in each row or each column. ⎞ ⎛ 0 ··· 0 1 ⎜ 0 ··· 1 0 ⎟ ⎟ ⎜ ⎟, ⎜ · · · Ms,3 = ⎜ (8) ⎟ ⎝ 0 1 ··· 0 ⎠ 0 ··· 0 M s,2 where 0 is an all-zero (m − k)(p + 1) × 1 column vector. The right (p + 1)k columns (upper (p + 1)k rows) are extended columns (extended rows). Clearly, M s,3 is also full rank. i in such way: If there is a Then, we rebuild Ms,s from M s,3 unit submatrix U zi in certain p+1 columns of Mis,s , we switch p + 1 columns from the right extended columns to the left of . The switching rules are that the unit submatrix U is M s,3 zi formed. Then, we form the nonsystematic part of the columns of Mis,s having U zi by row transformations. This process continues until all unit submatrices and the nonsystematic parts of Mis,s having the unit submatrices are formed. The resulting matrix is Mis,s . Since all the above transformations on rows . and columns are elementary, Mis,s has the same rank as M s,3 Thus, Mis,s is full-rank. Q.E.D. In the proof process, the leftmost submatrices of M s,1 must be kept as unit matrices. Thus, the unit matrix for the systematic block cannot be in the leftmost part of Mis,s . Otherwise, the unit matrix in the leftmost of nonsystematic

2068

ISIT 2008, Toronto, Canada, July 6 - 11, 2008

part will be removed by linear row transformations. Hence, we have at most m − 1 systematic blocks. From above results, the approach of constructing SBDRCs is to form Mis,s with nonsingular H is (for any distinct m blocks). Yet, not all p (prime p+1) and m can have nonsingular H is . For every m and p, we must manually verify if H is is nonsingular. The verifying complexity increases exponentially with m. There is no analytical result on m and p to guarantee nonsingular H is . In [9], the p and m with nonsingular H is are tabulated for a quite small m (totaling eight). The results are for the parity check matrix of MDS array codes. However, for a larger m, it is prohibitive to verify all p and m if H is of Mis,s is nonsingular [9]. Clearly, the constraint on small m limits applications of SBDRCs. In practice, we might prefer to codes with large m (many uncoded/systematic blocks). Now we prove the existence (also give a construction approach) of SBDRCs with large m. For this purpose, we first define a K × K polynomial matrix with coefficients in GF(2) as following: ⎞ ⎛ 1 xi1 j1 · · · xi1 jK−1 ⎜ 1 xi2 j1 · · · xi2 jK−1 ⎟ ⎟, (9) H us  ⎜ ⎠ ⎝ ··· i K j1 iK jK−1 ··· x 1 x where ik and jn are positive integers, and 1 ≤ j1 < j2 · · · < jK−1 < m, and 1 ≤ i1 < i2 · · · < iK ≤ N ∗ . Here N ∗ ≤ p, and K ≤ m. We denote the determinant of H us as D(Hsu ). Then D(Hsu ) is a polynomial with coefficients in GF(2). The largest power of x for a polynomial is called the degree in [2], [12]. In the following, we have a lemma on the degree of D(Hsu ), which will be used in our analysis. Lemma 3. The degree of D(Hsu ) is Dmax = i2 j1 + i3 j2 + · · ·+iK jK−1 , i.e., the sum of the powers of x for the diagonal elements in H us . Proof: We can use induction to prove this. Clearly, the lemma holds if H us is a 2 × 2 square matrix. In such case

1 xi1 j1  Hsu = with i1 < i2 . D(Hsu ) = xi1 j1 + xi2 j1 . 1 xi2 j1 Thus, the degree is i2 j1 . We assume that for K = L ≥ 2 the lemma holds. Now we prove that the lemma is true for K = L + 1. The matrix becomes ⎞ ⎛ · · · xi1 jm · · · xi1 jL 1 xi1 j1 ⎟ ⎜ 1 xi2 j1 · · · xi2 jm · · · xi2 jL ⎟ . (10) Hsu = ⎜ ⎠ ⎝ ··· iL+1 j1 iL+1 jm iL+1 jL ··· x ··· x 1 x By the Leibniz formula, D(Hsu ) is the sum of all possible products of elements in different rows and columns. Now we show that the largest power of x for these products is by the elements in the diagonal of H us . For the product includes the element xiL+1 jL , the other elements are from the first L rows and the first L columns. Thus, the product from the diagonal element has the largest power, by the induction assumption. Then the product is calculated as ProdL = xi2 j1 · xi3 j2 · · · xim jm−1 · xim+1 jm · · · xiL jL−1 · xiL+1 jL . If a product does not include the element xiL+1 jL , it must include another element in the final row (L + 1th row), say,

xiL+1 jm , for m < L. By removing the L + 1th row and mth column, we get an L×L matrix. By the induction assumption, Prodm,p = xi2 j1 · xi3 j2 · · · xim jm−1 · xim+1 jm+1 · · · xiL−1 jL−1 · xiL jL . Then, the product including xiL+1 jm with the largest power is Prodm = Prodm,p · xiL+1 jm . Now we show that the power of ProdL is larger than that of Prodm . This is the same as to prove im+1 jm + im+2 jm+1 + · · ·

+ iL jL−1 + iL+1 jL > iL+1 jm + im+1 jm+1 + · · · +iL−1 jL−1 + iL jL .(11)

Equivalently, we can prove (im+2 − im+1 )jm+1 + · · · + (iL − iL−1 )jL−1 + (iL+1 − iL )jL

>

(iL+1 − im+1 )jm . (12)

Since jm < jm+1 < jm+2 < · · · < jL−1 < jL , and iL+1 − im+1 = (iL+1 − iL ) + (iL − iL−1 ) + · · · + (im+2 − im+1 ), then (12) is true. Thus, the lemma is true for K = L + 1. Hence, the product of diagonal elements of H us is the degree. Q.E.D. With Lemma 3, we are close to constructing the SBDRCs with a large m. From H is , we remove the columns with 1s in the upper k rows, and then remove upper k rows. Then, the resulting matrix is ⎛ ⎞ 1 x(zk+1 −1)j1 · · · x(zk+1 −1)jm−k−1 ⎝ ⎠, ··· H i,u s = (zm −1)j1 (zm −1)jm−k−1 1 x ··· x (13) where 1 ≤ j1 < · · · < jm−k−1 < m. We denote the determinant of H is as D(H is ), and the determinant of H i,u s i i,u as D(H i,u s ). By Laplace’s formula, D(H s ) = D(H s ). By i ) of D(H is ) equals to Lemma 3, the degree (denoted as Dmax the sum of the power of the diagonal elements of H i,u s . Thus, i Dmax = (zm − 1)jm−k−1 + (zm−1

− 2)jm−k−2 + · · · + (zk+2 − 1)j1 .(14)

i By definition, Dmax is for any received m distinct blocks. u u , i.e., Dmax = The maximum of them is denoted as Dmax i i . Clearly, Dmax have the maximum if all zi and maxi Dmax jk have the maximum values (for all zi , jk in (14)). Thus, u Dmax

= (m − 1)N ∗ + (m − 2)(N ∗ − 1) + · · · (15) + (m − T + 1)(N ∗ − T + 2),

where T = min{N ∗ , m}. Now we are ready to construct our SBDRCs with a large m. In addition to p + 1 being a prime, we add another two constraints on the block length p: (C1), 2 is a primitive element u . From [12], [2], 2 is a primitive in GF(p+1); (C2), p > Dmax element in GF(p + 1) means that the order of 2 in GF(p + 1) is p. The order n of 2 means that n is the smallest positive integer such that 2n = 1 in GF(p + 1). We call the SBDRCs with constraints (C1) and (C2) type II SBDRCs. Without these constraints, the codes having nonsingular H is (for any distinct m blocks) are called type I SBDRCs. Currently, type I codes can only be verified for small m (for nonsingular H is ). We can easily construct Type II SBDRCs with large m: We just

2069

ISIT 2008, Toronto, Canada, July 6 - 11, 2008

design N ∗ and p to meet (C1) and (C2). Now we prove that the generator matrix of type II codes are full rank for any distinct m blocks. First, we show the following lemma. Lemma 4. With constraints (C1) and (C2), the polynomial matrix H is in (7) for any received distinct m blocks is nonsingular over the polynomial ring divided by Mp (x). Proof: With constraint (C1), the polynomial Mp (x) is irreducible [9], [12]. Thus, Mp (x) is not divisible by any polynomial over GF(2) with a degree larger than zero but smaller than p. From constraint (C2), D(Hsi ) has a degree i i u , and 0 < Dmax ≤ Dmax < p. D(Hsi ) is not a zeroDmax divisor for Mp (x) (divisible by Mp (x)). Thus, H is is always nonsingular over Mp (x) for any m distinct blocks. Q.E.D. With Lemma 4, now we can show that the generator matrix of type II SBDRCs is full-rank. Theorem 3. The generator matrix Mis,s of any distinct m blocks for type II SBDRCs is full-rank. Proof: The proof is quite simple now. With Lemma 4, H is is nonsingular over Mp (x). Then, by Theorem 2, Mis,s is full rank. Q.E.D. For type II codes, p is much larger than N ∗ to meet the u