IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
1441
Design of Signature Sequences for Overloaded CDMA and Bounds on the Sum Capacity With Arbitrary Symbol Alphabets K. Alishahi, S. Dashmiz, Student Member, IEEE, P. Pad, Student Member, IEEE, and F. Marvasti, Senior Member, IEEE
Abstract—In this paper, we explore some of the fundamentals of synchronous Code Division Multiple Access (CDMA) as applied to wireless and optical communication systems under very general settings (of any size) for the user symbols and the signature matrix entries. The channel is modeled by real/complex additive noise of arbitrary distribution. Two problems are addressed. The first problem concerns whether uniquely detectable overloaded matrices exist in the absence of additive noise under these general settings, and if so, whether there are any practical optimum detection algorithms. The second one is about the bounds for the sum channel capacity when user data and signature matrices employ any real or complex alphabets (finite or infinite). In response to the first problem, we have developed practical maximum likelihood detection algorithms for overloaded CDMA systems for a large class of alphabets. In response to the second problem, a general theorem has been developed in which the sum capacity lower bounds with respect to the number of users, spreading gain, and signal-to-noise ratio can be derived. To show the power and utility of the main theorem, a number of sum capacity bounds for special cases are evaluated. An important conclusion of this paper is that the lower and upper bounds of the sum capacity for small/medium-size CDMA systems depend on both the input and the signature symbols; this is contrary to the asymptotic results for large-scale systems reported in the literature (also confirmed in this paper) where the signature symbols and statistics disappear for signature matrices and input vectors with i.i.d. entries. Furthermore, upper and asymptotic bounds are derived and compared to other derivations. Index Terms—Multi access channels, optimal signature design, overloaded CDMA, sum capacity, synchronous CDMA.
I. INTRODUCTION ODE DIVISION MULTIPLE ACCESS (CDMA) is an alternative to frequency and time division multiple access (FDMA and TDMA). CDMA has become the standard for
C
Manuscript received March 17, 2011; revised May 30, 2011; accepted September 19, 2011. Date of publication October 18, 2011; date of current version February 29, 2012.The work of K. Alishahi was supported in part by IPM under Grant 89600125. The work of F. Marvasti was supported in part by the Iran National Science Foundation (INSF). K. Alishahi is with the Advanced Communication Research Institute (ACRI) and the Department of Mathematical Sciences, Sharif University of Technology, Tehran 11155, Iran, and the School of Mathematics, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran (e-mail:
[email protected]). S. Dashmiz, P. Pad, and F. Marvasti are with the Advanced Communication Research Institute (ACRI) and the Department of Electrical Engineering, Sharif University of Technology, Tehran 11155, Iran (e-mail:
[email protected];
[email protected];
[email protected]). Communicated by A. Moustakas, Associate Editor for Communications. Digital Object Identifier 10.1109/TIT.2011.2172391
the Universal Mobile Telecommunication Systems (UMTS) and American cellular systems [1]. Also, optical CDMA systems have become an alternative multiple access for fiber optics and optical wireless systems [2]–[4]. The reasons, to name a few, are its simplicity, high loading factor,1 and soft hand-off. In this paper, we will discuss some open issues related to uncoded synchronous CDMA systems of any size with finite input alphabet and matrix elements from arbitrary given alphabet (finite or infinite). The channel is modeled with real/complex additive noise with arbitrary distribution. In Subsections I-A, I-B, and I-C, we will discuss literature review and open issues, the summary of the results, and the structure of this paper, respectively. A. Literature Review and Open Issues 1) Sum Capacity: We start with the main known results. The theoretical developments in CDMA sum capacity have been limited to users with Gaussian inputs where Welch Bound Equality (WBE) matrices achieve the theoretical capacity [5]–[9]. For binary CDMA, only asymptotic results were known [10]–[16] prior to recent papers [17] and [18] (to be discussed in Section II). For nonbinary finite input and signature alphabets, asymptotic sum capacity results were developed in [12]. Again, for finite dimensions (finite number of users and spreading gains), there are no results for the sum capacity. Moreover, in [19], asymptotic expressions for the mutual information in the sparsely spread CDMA systems were derived; besides, a detection scheme based on belief propagation was developed. In addition, in [17], lower and upper bounds were developed for the sum capacity of a binary synchronous CDMA for the noiseless case assuming that users are jointly dependent. In [18], the bounds were extended to a noisy channel with arbitrary distribution. The assumption of the joint probability of multiusers also changed to a more realistic scenario of independence among users. For the noiseless case, the same bounds were derived as the ones in [17] despite the independence assumption. Numerical results again showed that there is a linear region beyond Walsh/Hadamard matrices and there is a degraded region for a highly overloaded CDMA. The asymptotic results for the sum capacity simplified the equations and were compared to that of Tanaka [10]. The main contributions of [18] are tight bounds with closed-form derivations that, unlike the previous papers, depend not on the limiting 1The
number of users (n) divided by the spreading gain (m).
0018-9448/$26.00 © 2011 IEEE
1442
cases but rather on the number of users , spreading gain , and the noise distribution that need not be Gaussian. These sum capacity bounds suggest that there exists a linear region with respect to the number of users far beyond the orthogonal , which implicitly implies Walsh/Hadamard matrices matrices may that uniquely detectable2 overloaded exist. The bounds also suggest that there is a threshold beyond which overloaded uniquely detectable matrices do not exist. 2) Uniquely Detectable Matrices: For binary synchronous CDMA systems, a class of “uniquely detectable” matrices for overloaded CDMA systems were introduced in [17] and [20]. This paper is a continuation of [17] and [18]; the constraints of binary inputs for the multiuser and binary signatures are now relaxed. The uniquely detectable matrices (COW3 and COO4) developed in [17] are also extended to nonbinary cases. Next, we will give a brief summary of the previous results. In [17], [20], and [21], a class of overloaded uniquely detectable matrices for binary multiusers and binary/ternary for COW) and optical (0, 1 for signatures for wireless ( COO) applications were developed. Mow [20] presented a binary and ternary unifying approach to find one-to-one matrices for binary inputs for multiuser applications. He also applied constructive theorems developed by the previous authors [22]–[28] to enlarge such matrices; this paper also discusses asymptotic behavior of such matrices. In [17], we have also developed uniquely detectable matrices for binary inputs independently. In the same paper, we have also suggested ML detection using tensor products. 3) Open Issues: The open issues related to CDMA systems are itemized next. 1) In finite-dimensional CDMA systems and additive noise of any distribution, what is the sum capacity (bounds) for arbitrary input and signature alphabets? 2) For overloaded CDMA systems, are there any uniquely detectable signature matrices with finite input and signature alphabets? 3) What can we say about the aforementioned two questions when we have near--far or active/inactive user scenarios? 4) Can we extend the aforementioned results to an asynchronous case? 5) Can we extend these results to the generalized user inputs5 CDMA [29]? In this paper, we plan to address the first two issues. The importance of both items is related to understanding the theoretical limits of throughputs and practical design of overloaded signature matrices that can be efficiently decoded using maximum likelihood (ML), which can approach the theoretical capacities. B. Summary of Important Results 1) (General Noisy/Noiseless Lower/Upper Bounds for Finite Dimensional Systems): The main results of this paper 2By “uniquely detectable” matrix, we mean an injective matrix, i.e., the inputs and outputs are in one-to-one correspondence. In general, these uniquely detectable matrices depend on the input alphabet. 3Codes
for Overloaded Wireless Systems. for Overloaded Optical Systems. 5Every user has a set of signatures where each signature is chosen and transmitted based on the user data. 4Codes
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
are two general theorems that give lower bounds in either noiseless or noisy cases for the average mutual information for a given number of users and spreading gain. The input vectors and signature matrices are chosen with entries from finite or infinite alphabets and with specified distributions when there is an additive noise with an arbitrary distribution (Theorems 14 and 17). Moreover, general upper bounds are derived based on some reasonable assumptions (Proposition 4 and Theorem 7). From these general bounds, a number of explicit analytic expressions for special cases such as binary, ternary, quaternary, real, and complex signature matrices and inputs are derived; also the results are numerically evaluated and discussed (Subsections VI-A, VI-B, and VI-C). 2) (General Noisy/Noiseless Lower/Upper Bounds for Asymptotic Cases): In either noiseless or noisy cases, when the parameters of the system (number of users , and the spreading gain ) tend to infinity in an approor fixed priate way (respectively, with for noiseless or noisy cases), the normalized bounds reduce to simpler forms that give rise to insights and efficient approximations to the finite but large systems (Theorems 16 and 19). The noisy asymptotic formulas are consistent with those of Tanaka and Guo--Verdu results. Also, the asymptotic results for the noiseless case are new. 3) (Overloaded Uniquely Detectable Codes for Finite Alphabet Sizes): In this paper, a class of highly overloaded signature matrices are designed for various input and signature alphabets that are uniquely detectable, i.e., no two different inputs give rise to the same output, and hence, in the absence of noise, the input can be detected uniquely form the output (Section III-A). Simulation results suggest that these uniquely detectable matrices outperform other known classes of matrices even in the presence of noise (Section III-D). Moreover, a structured subclass of uniquely detectable matrices with low-complexity optimal (ML) detectors is introduced (Section III-C). C. Structure of This Paper Section II on preliminaries covers some definitions and review of the previous relevant topics. Section III deals with the generalization of the COW/COO matrices to the nonbinary finite real/complex CDMA systems (GCO6). Practical algorithms for the construction of GCO matrices and bounds on the loading are introduced in this section. The performance of factor GCO matrices is compared to the WBE matrices, and practical ML detectors are also discussed. Section IV covers the derivation of lower bounds for the sum capacity of an arbitrary CDMA system with no channel noise; a general theorem is developed and the special cases of finite -ary7 CDMA systems are derived from this general theorem. Many examples such as binary/ternary and binary/quaternary CDMA systems are derived with numerical results. The presentation of upper bounds and asymptotic cases is the final parts of this section. 6Generalized
Codes for Overloaded CDMA Systems. we mean that the alphabet sizes of the input user and the signature matrix are q and l , respectively. 7By (q; l)-ary,
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
Section V is similar to the previous section except that additive channel noise with arbitrary distribution is also considered; the noiseless case in Section IV can be derived from the noisy case in the limiting case. In this section, we develop a general theorem for the lower bound of the sum channel capacity that is valid for any type of CDMA systems. In Section VI, we derive the lower bounds from our main theorem for the special cases of -ary and finite/real-complex CDMA systems. As spefinite cial cases, some examples are evaluated for the binary/ternary, ternary/ternary, and binary/real CDMA systems with additive Gaussian noise. Upper bounds and asymptotic limits are also derived in the same section. Finally, a summary of the main results, concluding remarks, and future studies are covered in Section VII. All the proofs of the theorems and examples are provided in the Appendices. II. PRELIMINARIES In order to have a better understanding of the sort of problems we will try to solve in this paper, five topics are reviewed: 1) a unifying mathematical model for all the sections of this paper; 2) power constraint considerations for normalization purposes; 3) the definition of sum capacity and the main known results; 4) review of the binary one-to-one COW/COO matrices; and 5) review of the sum capacity bounds for the binary CDMA. A. General Mathematical Model Let and be two given sets of real or complex numbers which could be finite or infinite. Assume that
1443
discuss detection of a class of such matrices that can be detected optimally with low complexity. is uniquely detectable, 2) If we relax the condition that over all input probabilities and the maximization of signature matrices is the topic of Section IV for general and , where bounds for this capacity are found for various cases. 3) In the case of additive noise, for general and , the mutual information is no longer the same as the maximization of the output entropy and a general theorem and several corollaries are developed for the bounds of the sum capacity in Section V. For finite input but arbitrary signature alphabets, the bounds are given in Section VI. For performance evaluations and channel capacity derivations in the noisy case, we need to normalize the input and the signature powers. Section II-B is related to power normalization before we formally define the channel capacity for a CDMA system. B. Power Constraint Issues Recall that our model for the CDMA channel is (3) Since the total power of users and noise in the aforementioned and model are, respectively, equal to (the symbol * stands for the Hermitian transpose), the multiuser SNR is defined as
(1) (2) where means the set of all matrices with entries from . Consider a CDMA system with users and the spreading as , where is the user gain data vector, is the noise vector, is the resignature matrix, and ceived signal, is the is a gain factor based on the transmitter powers for a desired Signal-to-Noise Ratio (SNR) (these vectors and matrices can be the transmitted signareal or complex).8 We will call ture matrix. The problems we would like to solve are as follows. 1) In the absence of noise, the mutual information reduces to . Therefore, the problem of sum channel over all input capacity becomes maximization of probabilities for a given matrix . For finite , this channel , where stands capacity is upper bounded by for the cardinality of the set. If the signature matrix is uniquely detectable, i.e., induces a one-to-one mapping when it is restricted to input vectors with entries in , this upper bound is achievable. The design of such signature matrices is the topic of Section III. Specifically, we will 8This model is true in the absence of near--far effects. In case users have different powers or alternatively in the presence of near--far effects, the matrix is modified by a diagonal matrix [30].
B
(4) For simplicity, we will assume that , where s are i.i.d. random variables of variance with as the common probability density function (pdf). This assumption implies that , and hence, the SNR definition can be written as (5) Now, if we define the normalized SNR as (6) then (5) can be written in the following form: (7) For a given signature matrix sum capacity is defined as
and normalized SNR , the
(8) where maximization is over all product distributions on the input vector such that (7) is satisfied.
1444
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
With the assumption that s are i.i.d. with the probability , we have , where density and denote the mean and variance, respectively; and are, respectively, the identity matrix and the matrix with all entries equal to one. Therefore, we have
Example 1 (The Sum Channel Capacity for WBE Matrices): Here, we consider real WBE signature matrices since they maximize the capacity in (11) [7]. For the underloaded case and for the normalized WBE matrices, (11) becomes (13)
(9) Hence, the SNR condition (7) can be written in the following form:
and likewise for the overloaded case
, (11) becomes (14)
(10)
C. Definition of Sum Channel Capacity and the Main Known Relevant Results Definition 1 (The Sum Channel Capacity): Define to be the maximum value of in (8) over all signature matrices , where (1), (2), and (10) are ; we use the notation satisfied. For noiseless systems, . Remark 1: In [10], [31], and [32], the signature matrix is taken to be random with Gaussian i.i.d. entries and the average mutual information is evaluated for such a random channel. Being an average rather than the maximum, such capacities are actually less than the one defined in Definition 1. In this paper, we use techniques to give lower bounds for the average mutual information (and hence for the maximum) for a more general case where the entries of are i.i.d. with an arbitrary distribution, e.g., uniform or a discrete distribution on a finite set. Given the aforementioned definition, we are now ready to review the main relevant results from the CDMA literature. Essentially, the previous efforts have been related to either Gaussian inputs or asymptotic results for binary or finite inputs. Our previous works were all related to development of one-to-one signature matrices and bounds for overloaded binary CDMA. In order to extend these results to nonbinary inputs and signature matrices, we need to review the previous known results. Theorem 1 (Main Known Result for the Sum Channel Capacity): When the input vector is real and Gaussian, the noris an real matrix and malized transmitted signature the additive noise is real white Gaussian; the sum channel capacity (8) can be shown to be [5] (11) On the other hand, if complex, we have
, and the additive Gaussian noise are
(12) where
is the normalized SNR as defined in (6).
Proposition 1 (Tanaka’s Asymptotic Average Capacity for Binary User Input): For binary bipolar input and matrix, Tanaka derived an asymptotic average capacity per user using the replica theory from statistical physics [10]: (15) in which
(16)
(17) (18) and is the variance of the noise. where Stating the aforementioned result, Tanaka does not use the term “average capacity” but just “capacity in the large system limit.” This, as he explains, is based on the self-averaging assumption according to which for large enough systems the capacity for almost all signature matrices is very close to the average capacity over all signature matrices. Thus, the results of Tanaka, Guo--Verdu (the following proposition), and also our asymptotic average capacity derivations can be read as the results about the capacity of a typical signature matrix chosen at random. We have shown in [18] that both (15) and our asymptotic upper bound approach the same value obtained by normalized capacity from (13) when increases. The replica theory is a nonrigorous mathematical analysis. Rigorous proofs of Tanaka’s results in special cases are given in [11], [14], and [32]. The extension of Tanaka’s bound to arbitrary input symbols is given in [12]; for complex input symbols, the following asymptotic result can be derived. Proposition 2 (Guo--Verdu’s Asymptotic Average Capacity for Arbitrary User Input [12]): (19)
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
in which
1445
E. Summary of [18]: Bounds on the Sum Capacity of Synchronous Binary CDMA Channels (20)
is the Kullback--Leibler where s is the single-user SNR, distance, is the multiuser efficiency determined from the recursive relationships that depend on the type of detectors, and . It is also shown in [32] that the asymptotic bound does not depend on the distributions of the entries of i.i.d. signature matrices. In this paper, we will show that for the noiseless case, the asymptotic lower bound depends on the input and signature symbols; however, for the noisy case, the asymptotic lower bound does not depend on the signature matrix symbols. Since most of the results of [17] and [18] are needed in this paper, a summary of the results is also given here in the preliminaries.
Theorem 5 (Noiseless Lower Bound): For any sum capacity is lower bounded by
(22) Theorem 6 (Noisy Lower Bound): For a binary system, in an additive white Gaussian Noise (AWGN) channel with the real is equivalent to the normalized noise of variance . For any and and any positive real number , SNR, the sum capacity, as described in Definition 1, is lower bounded by
D. Summary of [17]: Uniquely Detectable Signature Matrices for Overloaded Binary CDMA Systems The work in [17] is on the construction and simple ML detection of uniquely detectable binary signature matrices in the absence of noise for a synchronous CDMA system. The main results of this paper that will be needed in this paper are the following theorems: Definition 2 (COW and COO Matrices): An matrix is called a COW (COO) matrix if with entries in is injective when is restricted to the the mapping . set Theorem 2 (Enlarging COW Matrices): Assume that is an COW matrix and is an invertible matrix; then, is a COW matrix, where denotes the Kronecker product. Moreover, the deteccan be reduced tion of a system with the signature matrix , where to detection systems with the signature matrix can be implemented by Euclidean the detection of distance measurements using ML detection [33]. Theorem 3 (Highly Overloaded COW Matrices): Assume is an COW matrix and is a 2 2 that Walsh/Hadamard matrix. We can add columns to obtain another COW matrix. to Theorem 4 (Upper Bound for the Loading Factor): For a COW/COO matrix, the total number of users is upper bounded by
and , the
(23)
Proposition 3 (Conjectured Noisy Upper Bound): Assumption: we assume that the mutual information is maximized when the binary inputs are equally likely. This is a reasonable assumption that is still a conjecture—[14] and [18]. Under this assumpis symmetric, we have tion, if the noise pdf (24) where
(25) and
is the differential entropy.
Theorem 7 (Asymptotic Noiseless Lower Bound): For largescale systems, the bound in (22) asymptotically behaves as follows: (26)
Theorem 8 (Asymptotic Noisy Lower Bound): Likewise, for large-scale systems when is kept constant, the bound in (23) becomes
(21) (27) where
.
where
.
1446
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Fig. 1. Normalized upper and lower bounds for the sum capacity derived from (23) and (24) and the normalized capacity with hard limiter at the receiver end. For dB, and , the numerical sum capacity evaluated for the overloaded COW matrix is a better lower bound than the one predicted analytically from (23).
m = 8; = 12
>1
Theorem 9 (Asymptotic Noisy Upper Bound): Similarly, the bound in (24) becomes
orems are 11 and 12 and the other theorems and corollaries are special cases. A. Constructing GCO Matrices
(28) Note 1: (Numerical Improvements of the Bounds for Overloaded Binary CDMA): The bounds developed from Theorems 6–9 for the binary CDMA systems are less accurate for underloaded (small ) and small scales . However, for , the Walsh/Hadamard signature matrix is equivalent to the single-user binary phase shift keying (BPSK) and its actual capacity is known analytically and thus can be evaluated. Fig. 1 shows the actual normalized sum capacity based on both hard and soft decisions (soft decision is identical to hard for dB) for . The COW/COO overloaded matrices behave similar to the Walsh/Hadamard matrices that are for the fully/underloaded CDMA systems. The same figure shows the capacity of a specific COW matrix for in the range (1, 13/8) for an SNR of 12 dB. Since the sum capacity of a specific COW matrix based on hard decision is a lower bound for the CDMA peak sum capacity, we have a better numerical bound than that derived in [18] as shown in Theorem 6. Later, we will generalize the COW/COO matrices in Section II-D to nonbinary cases. The rest of the sections are generalizations of the binary sum capacity bounds given in Subsection II-E for finite/infinite, and real/complex symbols.
Suppose that the inputs are from a given set and the signature entries are from a given set . For errorless uncoded comsignature matrices that are munications, we need to find one-to-one over the set . Definition 3 (GCO Matrix): An matrix with entries in is called a matrix if the mapping is injective when it is restricted to the input vectors with entries in . The main problem is to find values of and such that matrices exist. In the following, we will develop a general theorem related to GCO matrices. Consequently, we will give corollaries and examples to construct and detect GCO matrices for overloaded nonbinary CDMA systems. In this to be the first columns of the section, we define identity matrix to be the first columns of the all one matrix to be , and to be the first columns of the zero matrix . We can generalize Theorem 2 for enlarging COW matrices to GCO matrices as shown in the following theorem.
III. GENERALIZED COW/COO (GCO) MATRICES
Theorem 10 (Constructing Larger GCO Matrices From Smaller Ones): Assume that is a matrix invertible matrix with entries belonging to the and is a set . Then, is a matrix where is the set of all products of the elements of and . The proof is very similar to that of Theorem 2 in the section on preliminaries, which is given in [17] and [21]. A generalization of Theorem 3 to GCO matrices is given next.
In this section, we extend the class of COW/COO matrices discussed in Section II-D when the alphabets of the signatures and the user inputs are not limited to binary sets. In the following, we will discuss the construction of GCO matrices and the ML detection of such matrices. In this section, the main the-
Theorem 11 (A General Method for Constructing GCO and , assume that is a Matrices): For integer sets is a Walsh/Hadamard matrix, and is an integer number. Also, suppose that and is the largest nonzero integer such that in which
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
A
TABLE I WHERE “ ” DENOTES AND “ ” DENOTES
+
0
+1
1447
of the aforementioned matrix are a 16 16 Walsh/Hadamard matrix. is a Example 3 ((7, 2)-ary GCO): Note that matrix. Using the aforesaid theorem, we get a matrix. Example 4 ((2, 2)-ary GCO): Using Theorem 11, one can start with a 2 2 Walsh/Hadamard matrix and construct a matrix which is about 200% overloaded. This example is similar to that in [20] and [34]. is a Example 5 ((2, 3)-ary GCO): Suppose ; from Theorem 11, by setting , and , we get and . One can easily check that
01
(29) If
are matrices with entries from , where is any integer, then
.. .
is a
(34)
such that
(30)
matrix if (31)
Thus, is a matrix. If we choose to be the matrix generated from Example 4, we get a matrix which is 300% overloaded. This example is also similar to that in [20]. Corollary 2 (Constructing Nonbinary GCO Matrices): is a Assume that matrix and is a Walsh/Hadamard matrix. Also, assume that and for . Now,
Moreover, when consists of only odd numbers and , then is a matrix if
.. .
(35)
(32) is a matrix, if The proof is given in Appendix A. Corollary 1 (Constructing Binary GCO Matrices): Assume that is a matrix, is Walsh/Hadamard matrix, and . Also, a assume that for and for . Now,
.. .
(33)
is a matrix. The proof is given in Appendix B. Example 2 ((3, 2)-ary GCO): Table I is a matrix which is about 38% overloaded [21]. Using the aforementioned corollary, we matrix which is about get a 60% overloaded. In this case, , and . Notice that the first 16 columns
. The proof is similar to that of Corollary 1.
Example 6 ((3, 3)-ary GCO): It is easy to verify that the matrix shown in Table II is a matrix. Applying the previous theorem twice, we obtain a matrix. Example 7 ((3, 5)-ary GCO): From the previous example, we conclude that there exists a matrix. Applying the previous theorem, we obtain a matrix. Theorem 12 (Generating Overloaded Complex GCO Matrices for Arbitrary Input Integers): Let be a finite set of integers and assume that is a matrix for , and are AINs9(real where or complex). Also, assume that .. .
are
generated as demonstrated in Theorem 11 for
matrices .
9We call a set of numbers algebraically independent numbers (AINs) if linear combinations of the numbers with nonzero integer coefficients do not vanish.
1448
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Fig. 2. Upper bound on the maximum number of users versus the spreading gain for a (3, 2)-ary uniquely detectable signature matrix.
A
TABLE II WHERE “ ” DENOTES AND “ ” DENOTES
+
+1
0
Theorem 13 (A General Upper Bound for Matrices): If there exists a matrix, then
01
for
(36)
Suppose for obtained by replacing each 0 entry of trarily. Then, is a matrix. The proof is given in Appendix C.
by
and or
Corollary 3 (Binary Input With Complex GCO): Assume that s are for and is a complex with minimal polynomial of degree . Also, for that Then,
is arbi-
where is equal to the number of solutions of the equain , divided by . The proof is given tion in Appendix D. , Example 9 (Binary and Ternary Cases): For the upper bound in (36) becomes identical to that given in (21). and On the other hand, for the special case , we get the following upper bound [21]: (37)
Matrix where number assume . is a
matrix. The proof is straightforward from Example 5 and Theorem 12. Example 8 (Binary Input With Quaternary Complex Matrix): For the special case of , and , we get a GCO matrix. In [35], an overloaded matrix is constructed which is much smaller than the matrix derived from the aforementioned theorem. Now that we have given examples of constructing GCO matrices, we can discuss the upper bound for the number of users and practical and optimum detection of such matrices.
(38) The aforementioned upper bound is plotted in Fig. 2. This figure shows that we cannot have errorless communications beyond 230 ternary users (equivalent to bits) for the spreading gain 64. It is interesting to compare this number to the binary case in (21) where the upper bound for the number of users is 268. Example 10 (Quaternary Case): As another example, the following upper bound for the maximum number of users for a system with the spreading gain for which there exist a matrix can be derived:
B. Upper Bound for the Loading Factor The following theorem provides a general upper bound for the loading factor for GCO matrices.
(39)
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
1449
Fig. 3. Symbol error rate versus normalized SNR () for an overloaded CDMA system with the spreading gain 64 and 88 users for the (3, 2)-ary GCO signature (see Table I). matrix
H A
C. ML Detection for a Class of GCO Matrices The direct ML detection of GCO matrices is computationally very expensive for moderate values of and . In this section, we present two lemmas for decreasing the computational complexity of the ML detection for a class of GCO matrices. This is similar to the detection of binary CDMA for COW/COO matrices—[17] and [36]. Lemma 1 (Detection Method for Large Matrices): Suppose is constructed as discussed in Theorem 10. The detection problem of a system with the signature matrix can be reduced to detection problems of a system with the signature matrix . The proof is similar to that of Theorem 2 in Section II. is a full Lemma 2 (Optimum Detection Algorithm): If matrix, then the detection problem for rank a system with the signature matrix can be performed through Euclidean distance measurements. Proof: The proof is similar to that of Theorem 2 in Secfunction, a vector is tion II; however, instead of using the mapped to the nearest -vector. Lemmas 1 and 2 lead to a significant decrease of detection complexity. Since each vector is mapped to the nearest -vector, it is not hard to show that if columns of make a scaled unitary matrix, then the proposed method for detection is an ML detector. , deExample 11: Let picted in Table I, and be a 4 4 Walsh/Hadamard matrix. is a matrix acNow, cording to Theorem 10. The direct implementation of an ML deEuclidean norm calculations of 64-dimentector of needs sional vectors. But using these lemmas, we need only Euclidean norm calculations of 16-dimensional vectors, which is acceptable in practice. D. Numerical Results for GCO Matrices -ary GCO matrix genFig. 3 shows the performance of a erated from the Kronecker product of the matrix represented in
Table I by a 4 4 Walsh/Hadamard matrix in a noisy environment. This GCO matrix has dimensions 64 88; it also confirms that GCO matrices, similar to COW/COO matrices for the binary case, are superior to WBE sequences. The ML detector has been performed by using Lemmas 1 and 2. The WBE sequences are decoded using iterative methods [37]. IV. SUM CAPACITY BOUNDS FOR GENERAL USER INPUTS AND MATRICES—THE NOISELESS CASE Let and be two sets of (real or complex) numbers. In this section, we consider the noiseless CDMA channel where the entries of the user input vector and the signature matrix belong to and , respectively10, i.e., and . In this section, Theorem 14 is the main theorem and the other propositions and corollaries are the special cases. Before stating the theorem, we need the following definition. Definition 4: , the difference set of , is defined as (40) on , we define to be the probaFor a probability law bility law on which is the pdf of the difference of two indepen. dent random variables with the same probability density be a probability density on . The probaMoreover, let is the probability measure on inbility measure duced by choosing entries of the random matrix independently . and with the same probability density A. Sum Capacity Lower Bound In order to obtain lower bounds for the sum capacity for various scenarios, we prove a general theorem. Special cases are derived from this theorem. 10For the noiseless case, the power constraint in (7) is irrelevant, and therefore, r in (3) can be set to 1.
1450
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Theorem 14 (A General Lower Bound for the Noiseless Case): In the absence of additive noise, for any and , we have (41) and with i.i.d. entries with densities where and , respectively. The proof is given in Appendix E. For the case when the input and the signature matrix alphabets are finite, the aforementioned theorem can be simplified as follows. Corollary 4 (Finite Input/Finite Signature Matrix): Sup. Let be nonnegative pose that integers that satisfy . Define to be the set of all vectors such that the number of in are , respectively. occurrences of It is easy to show that if , then and we denote this probability . Thus, (41) becomes by
is the Dirac function. Also, when and where divides , we conjecture that the maximum in (44) happens . when This conjecture is a special case of the conjectured upper bound for the noisy Proposition 7 when the noise variance is zero without restricting the signature to have unity magnitude. An upper bound that does not depend on the symbol alphabets but is simpler to evaluate is given in the following theorem. However, the bound is not as tight as the aforementioned upper bound. and
Theorem 15 (Noiseless Upper Bound): If , then
(46) where
denotes the multinomial distribution
with the probability
at
when are nonnegative integers and The proof is given in Appendix F. (42)
.
Remark 2: It seems reasonable to believe that the maximum value in the aforementioned bound is attained at , and hence, the bound may be simplified as
where (47) (43) C. Asymptotic Lower Bound B. Sum Capacity Upper Bound Below a general conjectured upper bound for the noiseless case is given. Proposition 4 (A General Upper Bound for the Noiseless Case): Assumption: we assume that the mutual information is maximized when the inputs’ product distribution is i.i.d. This is a reasonable assumption that is still a conjecture—[14] and [18]. Under this assumption and in the absence of additive noise, if with density and , we have the following upper bound: (44)
Theorem 16 (Asymptotic Noiseless Lower Bound): Let and be the probability densities on and with and be the distribution induced on as before. Then
(48) and for where a set of numbers denotes the dimension of as a set of vectors over the field of rational numbers , i.e., the maximum size of a subset of such that no nontrivial rational linear combination of which vanishes. For the proof, see Appendix G. D. Examples
in which
Later, we will give many examples and numerical results that are special cases of Corollary 4. The derivations of the following formulas are all given in Appendix H. Example 12: Binary Wireless CDMA, and
(45)
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
Fig. 4. Sum capacity lower bound versus the number of users for binary input and binary signature matrices when all the probabilities are equal to 1/2 for
(49) , the aforementioned equation becomes When identical to (22) in Section II; the evaluation of (49) for the uniform distribution is given in Fig. 4. Example 13: Binary Optical CDMA, and
1451
m = 32.
is slightly greater than the optical case and the optical case is greater than the binary wireless. The fact that a lower bound is greater does not prove that the actual sum capacity is better. The following proposition shows that the AIN symbols, as suggested by the lower bound evaluations, yield the maximum possible capacity. We shall see in Section VI-C.1 that under additive noisy environment, this statement may not be true. Proposition 5 (AIN Achieves the Greatest Sum Capacity in the Absence of Noise): Let be a set of arbitrary numbers and be a set of AINs; then for any we have (52) The proof is given in Appendix I. Example 15: Binary/Ternary, , and
(53) (50) Fig. 4 also shows the aforementioned lower bound when . Example 14: Binary Complex Signature (Uniform Distribuand (AINs as defined in Footnote 9) tion),
See Fig. 5 for the numerical evaluation of the aforementioned equation. A slightly different bound for this system is given in [38]. Example 16: Binary/Ternary With Uniform Distribution, and is a set of ternary AIN
(51) (54) The aforementioned bound is also plotted in Fig. 4. This figure shows that the lower bound for the complex signature matrix
The aforementioned bound is plotted in Fig. 5.
1452
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Fig. 5. Sum capacity lower and upper bounds versus the number of users for binary input and ternary signature matrices.
Example 17: Binary/Ternary CDMA System (Uniform Distribution) ( and
(57) (55) The aforementioned bound is evaluated in Fig. 5. This figure shows that the lower bound for the AIN ternary signature matrix is slightly better than the third roots of unity, which is sigcase. This observation is also nificantly better than the confirmed by Proposition 5. Example 18: Ternary/Binary CDMA System, , and
in which . The aforementioned bound is evaluated in Fig. 6. This figure shows the upper and lower bounds for the ternary/ternary system, which are relatively tight. It also shows the lower bound of the ternary/binary system in Example 18, which is lower than the ternary/ternary system. Example 20: Binary/Quaternary CDMA System (Uniform and is a set of four AINs) Distribution) (
(58)
(56) The aforementioned bound is evaluated in Fig. 6 and is compared to the ternary/ternary case in the next example. Example 19: Ternary Wireless System, , and
The aforementioned formula is evaluated in Fig. 7 and will be compared to the next example with complex signature but the same cardinality. Example 21: Binary/Quaternary System (Uniform Distribuand tion) (
(59)
Notice that the aforementioned formula is similar to (5). This implies that the lower bound for a binary/binary system with
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
1453
Fig. 6. Sum capacity lower and upper bounds versus the number of users for ternary input and binary/ternary signature matrices. The lower bound for the ternary signature matrix is better than that of the binary case.
Fig. 7. Sum capacity lower and upper bounds versus the number of users for binary input and quaternary/pentad signature matrices.
the spreading gain is equivalent to the lower bound for binary/quaternary system with the spreading gain . It is interesting that the same relation holds for the actual sum capacities as shown in Proposition 6. Fig. 7 shows the evaluation of the aforementioned formula. The lower bound for the signature with the AIN quaternary matrix is much better than the dependent case (see Proposition 5). This is again due to the absence of noise; the differences are less pronounced in the case of additive noise. In the same figure, the lower and upper bounds of binary/pentads are plotted. This evaluation implies that our lower bounds are relatively tight. Proposition 6: The following relation between the capacities holds: (60) The proof is given in Appendix J.
Example 22 (Asymptotic Lower Bound for Binary/Binary and be uniform distributions System): Let and , respectively. Therefore, and on and . Now, the maximum with is with , and therefore, the bound in Theorem 16 becomes (61) This is the same result as the binary case given in (26) in Section II. Example 23 (Asymptotic Lower Bound for Binary/Quaternary System): and be uniform distribuLet and tions on and , respectively. Therefore, and . Now, the maximum
1454
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Fig. 8. Asymptotic noiseless lower bound is compared to the normalized finite-scale CDMA systems. This figure shows that small-to-medium-scale systems cannot be accurately estimated by the asymptotic lower bound for the high values of .
with is with the bound in Theorem 16 becomes
, and therefore,
Theorem 17 (General Lower Bound for the Sum Capacity):
(62) The aforementioned asymptotic bound and finite lower are depicted in Fig. 8. This bounds for various values of figure shows that unlike the asymptotic results for the noisy case that will be discussed in the next section, medium-scale systems cannot be accurately estimated by the asymptotic lower bounds. In the case of additive noise, the bounds are modified as shown in the following section. V. SUM CAPACITY LOWER BOUND FOR GENERAL USER INPUTS AND MATRICES—THE NOISY CASE Under
additive noise scenario, the evaluation of is an extremely difficult problem but we will for a derive a family of lower bounds for class of random signature matrices. In the following general theorem, we let as in (3) for a fixed value of and a randomly chosen with distribution . In this case, the power constraint (10) can be written in the following simpler form after taking expectation over :
(64)
and are probability densities on and , rewhere is any arbitrary function. Also, is given by spectively, and (63), and is the first entry of the i.i.d. complex noise as defined in Section II-B. and are, respectively, vectors of length with i.i.d. entries of densities and as explained in Definition 4. This theorem reduces to the general noiseless Theorem 14 when the additive noise disappears. For the proof, refer to Appendix K. In the important special case of Gaussian noise, Theorem 17 can be stated in a more explicit way by substituting into (64); the result is given in the following corollary. Corollary 5 (General Lower Bound for the Sum Capacity for the Gaussian Noise): For a given and , when noise is complex Gaussian with independent real and imaginary parts of vari), (64) becomes ance 1 (hence,
(63)
A. Capacity Lower Bound The following theorem is the most general theorem for the sum capacity lower bound for any given input and signature matrix symbols with real/complex additive noise of arbitrary distribution and other results are special cases of this theorem:
(65)
where vectors of length
and and
are, respectively,
with i.i.d. entries of densities
and
.
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
Remark 3: In the aforementioned corollary, the additive noise is taken as complex Gaussian with independent real and imaginary parts with zero means and equal variances. If the CDMA system was real, the imaginary part of the noise would not affect the capacity bounds. This implies that the bounds would have been the same if the noise was real with half the total complex variance. Having said this, we should note that since we have used the supremum over a family of Jensen’s inequalities, the complex results are not analytically the same as the real ones although the plots are very similar when we use numerical results. In the next section, we will derive and evaluate special cases ) or when the alphabet sizes are either finite ( and infinite (real/complex).
1455
. If
, and ,
and
(65) becomes
VI. RESULTS FOR FINITE USER INPUTS AND ARBITRARY SIGNATURE MATRICES—THE NOISY CASE In this section, we will discuss lower bounds, upper bounds, asymptotics, and numerical results for special input and signature alphabets. A. Capacity Lower Bounds In this section, we will develop bounds for -ary and CDMA systems. 1) Lower Bound for Finite User Inputs and Signature Matrices: For the finite user inputs and signature matrices, the general Theorem 17 reduces to the following two corollaries.
(67) where
. Also, when
, we have
Corollary 6 (General Lower Bound for the Sum Capacity for -ary CDMA Systems): Assume that , and . If and , (65) becomes
(68)
. For the proof, see Appendix L.
Refer to Appendix M for the proof. 2) Lower Bound for Finite Input and Real/Complex CDMA Systems: In this section, we consider the case where the user data are finite and the signature matrix can have real or complex entries.
Corollary 7 (Results for Finite User Inputs and Symmetric Symbols for Signature Matrices): Assume that
Theorem 18 (Lower Bound for ( -ary, Real/Complex) CDMA Systems): For signature matrices with real or complex entries,
(66) where
1456
let
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
, where
, and
. A special case of Corollary
where
7 when the user inputs are binary and the signature matrix is symmetric -ary is given next. Example 25 (Lower Bound for the Sum Capacity for -ary System): For and Binary/Symmetric , if and , (67) becomes
(69)
For signature matrices with complex entries,
(72)
(70)
where
. Also, when
, the bound becomes
See Appendix N for the proof. Special cases of Corollaries 6, 7, and Theorem 18 when the user inputs are either binary or ternary are given next. Example 24 (Lower Bound for the Sum Capacity for Binary/ -ary System): For and , if and , (66) becomes
(73) A special case of Corollary 7 when the user inputs are ternary and the signature matrix has symmetric -ary symbols is given next. Example 26 (Lower Bound for the Sum Capacity for -ary Systems): For and Ternary/Symmetric , if and , (67) becomes
(71)
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
1457
Example 27 (Lower Bound for the Sum Capacity for Binary/ and , if , (69) Real Systems): For becomes
(76)
B. Capacity Upper Bound Next, a general conjectured upper bound is given. (74) where
. Also when
, the bound becomes
Proposition 7 (A General Upper Bound for the Noisy Case): Assumption: we assume that the mutual information is maximized when the input product distribution is i.i.d. This is a reasonable assumption that is still a conjecture—[14] and [18]. with the probability Under this assumption, if and (unit circle), we density have the following upper bound:
(77) in which
(78) (79) and divides , we any complex . Also, when conjecture that the maximum in (77) happens when . For the proof, see Appendix O. C. Numerical Results (75) A special case of Theorem 18 when the user inputs are binary and the signature matrix is real is given next.
1) Binary/Binary: A special case of Example 25 is evaluated in Fig. 9. The evaluation is for (2, 2)-ary and when the . In the evaluations, the input alphaspreading gain is . Also numerical results show that when bets are fixed to , the lower bound is maximum. This evaluation shows that the choice of symbols for the input and the signature
1458
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Fig. 9. Normalized sum capacity upper and lower bounds versus the number of users for binary input and binary signature matrix when all the probabilities are and dB. equal to 1/2 for
m = 16
=8
Fig. 10. Normalized sum capacity upper and lower bounds versus the number of users for (2, 3)-ary when m
matrix affects the sum capacity. The greatest lower bound is for the signature with AIN. 2) Binary/Ternary: The numerical results of Examples 24 and 25 are shown in Fig. 10 for the special case of (2, 3)-ary when dB, and the binary input is equal to . This figure shows a comparison of three different signature symbols; for the signature this evaluation implies that the choice of matrix is not as tight as far as the capacity bound is concerned. 3) Ternary/Ternary: The numerical results of Examples 25 and 26 are shown in Fig. 11 for the special cases of (3, 3)-ary, (2, 3)-ary, and (3, 2)-ary. The input alphabet symbols are for ternary and for binary users. The signature symbol is . As expected, the ternary/ternary system lower ternary bound is the greatest. For the quaternary matrix, the following example is instructive.
= 4 and = 8 dB.
4) Binary/Quaternary: The numerical results of Examples 25 and 26 are shown in Fig. 12 for two special cases of (2, 4)-ary . The systems. The user input alphabet symbols are binary or signature symbols are either that are AINs. This figure shows that the lower bound in the case of the AIN is greater than that of the quaternary signature matrix case. 5) Binary/Arbitrary Matrix: When the user inputs are binary, we have evaluated the bounds given in Examples 24–27 for binary, ternary, and complex quaternary signature matrices as deand dB. This picted in Fig. 13; the plots are for figure shows that by increasing the cardinality of , the lower bound is improved. This is a significant result since by adding extra complexity at the transmitter and receiver sides, we can increase the capacity. On the other hand, our asymptotic results
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
1459
Fig. 11. Normalized sum capacity upper and lower bounds versus the number of users for ternary/ternary, binary/ternary, and ternary/binary systems for and dB.
= 12
Fig. 12. Normalized sum capacity upper and lower bounds versus the number of users for the binary/quaternary system for m
m=4
= 4 and = 8 dB.
as well as those in [12], [32] imply that the sum capacity is independent of the signature alphabets. D. Asymptotic Lower Bound for the Sum Capacity for Additive Gaussian Noise In this section, we consider the problem of estimating the cain the limit when the number of users pacity per user and the spreading gain go to infinity while the loading and the normalized SNR are kept constant. factor We first prove an asymptotic formula for the expression appearing on the right-hand side of Corollary 5. Theorem 19 (Asymptotic Lower Bound for Finite Input and Additive Gaussian Noise): For the given sets and (with ),
(80) and are probability densities on and with , and are eigenvalues of the covariance matrix of real and imaginary parts of an r.v. which has the distribution of the product of two independent variables with densities and , where
1460
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
Fig. 13. Normalized lower bounds for the sum capacity versus the number of users for various signature alphabets. In this figure, I dB.
= 20
Fig. 14. Asymptotic lower bounds for the normalized sum capacity versus for binary, ternary, and quaternary inputs for as far as the asymptotic results are concerned.
and is the Kullback--Leibler distance. Notice that for a binary CDMA system, the aforementioned lower bound becomes identical to that of (27). The proof is given in Appendix P. Remark 4: Note that the asymptotic behavior of the lower bound in the noisy case is considered in a different regime than the noiseless case in Theorem 16. This is because for the noiseless case a much greater loading factor can be achieved and hence the limit in the regime while becomes trivial. This theorem implies that the asymptotic bound does not depend on the symbol sizes, types of the user inputs, or the signaand . This ture matrix but rather on the probabilities of result is plotted in Fig. 14. This figure shows the normalized sum capacity bounds versus for binary, ternary, and quaternary inputs for and 3. Fig. 15 is the evaluation of several asymptotic results. This figure shows a comparison of the actual (soft decision) single-user capacity for QPSK with the Guo--Verdu asymptotic result (11), and our asymptotic lower bound from . For , we can use orthogTheorem 19 for onal Walsh matrices, and hence, the sum capacity is equivalent to the single-user QPSK. Clearly, the asymptotic average mutual information derived by Guo--Verdu [12] for joint decoding
= f61g
; m
= 16, and
= 1 and 3. The matrix is irrelevant
QPSK is a lower bound for the actual sum capacity; however, for the hard decision QPSK, the Guo--Verdu bound is slightly , the Guo--Verdu bound is between our lower better. For and WBE upper bounds. This phenomenon is similar to that of Tanaka’s and our bounds for the binary case [18]. VII. SUMMARY OF THE MAIN RESULTS, CONCLUSION, AND FUTURE WORK In this paper, we have attempted to reveal some of the unknowns regarding CDMA. Our first concern was related to the developments of overloaded uniquely detectable matrices (coined as GCO matrices) for general, finite, and real/complex users and signature matrices. A general theorem (Theorem 11) was developed to construct larger GCO matrices from smaller ones. Various examples were given for special binary/ternary user inputs and signature matrices. The same procedure was ex, tended to complex signatures. For a given spreading gain a general upper bound for the maximum number of users was developed and evaluated for a special case. Also, practical ML detection algorithms were suggested for large-size GCO matrices. Numerical results for special cases showed that GCO matrices outperformed the WBE matrices. Our second concern
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
1461
Fig. 15. Asymptotic results for the actual normalized sum capacity, lower and upper bounds versus for QPSK inputs for bound is compared to that of Guo--Verdu and the single-user sum capacity.
was on the evaluation of the bounds for the sum capacity for arbitrary input symbols and signature matrices. Three sections were devoted to this problem. Section IV was related to the noiseless case. In this section, a general theorem (Theorem 14) was developed to find a lower bound for the sum capacity. From this theorem, a corollary for finite input/matrix and examples for the special cases were derived and evaluated. The evaluations showed that there was a linear region, where the lower bound is very tight up to a point at which the linearity suddenly breaks down in a severely overloaded region. Also, the evaluations showed that by choosing proper symbols for the signature matrix (such as AIN) and a probability law for the symbols, the lower bound could be drastically improved. The evaluations also showed that by choosing matrix entries , we could increase the linear region from larger sets of the lower bound. In Section V, we developed a general theorem (Theorem 17) for the sum capacity lower bound for an arbitrary CDMA system for any types of symbols, sizes, probability distributions, and additive noise. The general noiseless Theorem 14 can be derived from the general noisy Theorem 17 when the additive noise power becomes negligible. The important special case of additive Gaussian noise was given in Corollary 5. Section VI was the special case of finite symbols for the user inputs and signature matrices. Corollaries 24–26 were special cases for binary/ternary user inputs and -ary signature matrices. Many examples and evaluations were given in this section. Again, the dependence of the lower bound on the symbols, sizes, and noise level is apparent. In this section, an asymptotic lower bound was derived, which implies that for large-scale systems the bound does not depend on the signature alphabets but rather on their inputs under the assumption that the entries of the signature matrix are i.i.d. Section VI-A.2 was related to the development of the sum capacity lower bound for binary users but real or complex signature matrices. The evaluations showed that by changing the signature matrix from
= 1 and 3. In this figure, our lower
a finite set to an infinite real/complex number, the sum capacity can be improved significantly. This is a noteworthy result since by adding extra complexity at the transmitter and receiver sides, we can increase the capacity. However, our asymptotic results, as well as those of [12] and [32], imply that the sum capacity is independent of the signature alphabets. Nevertheless, our evaluations for finite and show significant differences for the sum capacity lower bounds for different signature symbols, probabilities, and cardinalities. As for future work, we suggest to study the effects of fading due to multipath on injectivity of GCO matrices and the evaluation of the sum capacity bounds. The problem of asynchronous CDMA would be a productive area of investigation as indicated by several attempts already made [33], [39]–[41]. The extension of our results to near--far effects is another challenging task [30]. Also, the consideration of generalized users [29], [42]–[44] is yet another interesting topic to look into.11 The sum capacity bound can be extended to the -active user with Markov memory; the mathematical model of this problem is similar to compressed sensing and thus could find a potential application for sparse signature matrices.
APPENDIX A. Proof of Theorem 11 (A General Method for Constructing GCO Matrices) The necessary and sufficient condition for to be one-to-one is . Now, if , we claim that . We know that (81) 11A
generalized user chooses its data from a set of arbitrary vectors.
1462
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
and each is an vector for an vector, and the entries of are from . Thus, (81) implies that
is for
all the entries of to (87) and (88).
are 0, and the rest of the proof is similar
B. Proof of Corollary 1 (Constructing Binary GCO Matrices) (82) Multiplying by
consists of only odd numbers . We set and in Theorem 11; note that s are binary matrices such that . Thus, and it suffices to show that
, we get
(91) (83) does not have any
We assume that the first column of Thus, we obtain
.
(84)
The aforementioned equation is true because the minimum is and the maxnonzero positive entry of is , and by our hypothesis, imum positive entry of . C. Proof of Theorem 12 (Generation of Overloaded Complex GCO Matrices for Arbitrary Input Integers) is
Hence
The necessary and sufficient condition for . Now, if , we claim that . Suppose that
to be one-to-one
(92) .. .
..
.
.. . (85)
where and are, respectively, and vecare AINs, the coefficient vector that tors. Since . Thus, is multiplied by is the zero vector for we have
The left-hand side of (85) is an integer vector and because (93) (86) Hence all entries of
are 0; therefore,
(94) (87)
Hence, (88) Since
matrix, for , which implies that . This completes the first part. Now, let consist of only odd numbers; is an vector and (85) becomes then
Because have on
is a for , and thus
matrix, we must ; because is injective .
D. Proof of Theorem 13 (A General Upper Bound for
is a
.. .
..
.
.. .
(89)
The left-hand side of (89) is an integer vector with entries that are either all odd or even and because the last entry of the righthand-side vector in (89) is 0, all the entries are even and again because (90)
Let the input multiuser data be defined by the random , where s are i.i.d. random varivector ables from the set with uniform distribution. Since s . Now, let the are independent, we have , transmitted CDMA random vector be defined by matrix. Since is one-to-one where is a over , we get . But we have , where . . E. Proof of Theorem 14 (A General Lower Bound for the Noiseless Case) For a given
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
1463
be a finite set of numbers Lemma 3: Let and be a nondegenerate probability distribution on with . If are i.i.d. with distribution , then (99)
(95) where the inequality is due to Jensen. Now taking expectation and using Jensen inequality once more, we with respect to have
Proof of Lemma 3: Let and . -dimensional lattice in and Clearly, is a if . But and hence . But since if we obtain and we have iff . Now, has an asymptotically fixed nondegenerate -dimensional Gaussian and hence distribution and is a lattice of codimension is of measure. Proof of Theorem: We know that (100)
(96) However, since the rows of are i.i.d., likewise i.i.d., and thus, we get
are
where Now, let be a probability density on and let the empirical distribution of is . Moreover, assume that is supported on . For large values of may be considered as a sum of i.i.d. r.v.’s with the nondegenerate distribution from and hence according to Lemma 3 (101)
(97)
Note that this probability might dependent on rewrite (101) as
and
so we
Therefore, (102) .
and therefore
. F. Proof of Theorem 15 (Noiseless Upper Bound) Let and capacity is maximum when the numbers case, we have
(103)
. Clearly, the are AINs; for this
But recall that the probability of appearance of ical distribution of is approximately
on the empirand therefore
(98) and if the number of s in the vector , then where has distribution and s are independent. The algebraically independence condition implies that is one-to-one, and therefore, the mapping . On the other hand, according to a theorem from Shepp and Olkin [45] and also from [46], the entropy is concave and of a multinomial distribution and hence attains the maxsymmetric with respect to . . imum at But
is
(104)
But it is easy to see that which is attained where and this completes the proof.
is proportional to
on
. H. Proof of Examples of Corollary 4
G. Proof of Theorem 16 (Asymptotic Noiseless Lower Bound) To prove, we need the following lemma.
1) Proof of Example 12 (Binary Wireless CDMA): Note that with probabilities
1464
and which
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
. Let ; suppose that and has entries and entries in places in is equal to and , respectively. We conclude that or .
The number of such
is equal to
with the ; thus,
probability
We conclude that thus,
because we should choose
is equal to
. Hence, if
,
the bound reduces to the bound in [17]. I. Proof of Proposition 5 (AIN Achieves Maximum Capacity for the Noiseless Case) Let be the matrix with entries from which maximizes over all product distributions and over all matrices with entries from where . be the input vector and Let be the product distribution , which results in the maximum capacity. If we define to be the matrix which is obtained from by . Now, it replacing by for each . Suppose , then . If is easy to observe that if by and if we denote the values of , then in the system . The same computation shows that with , the mutual information the product distribution for is . With the aforementioned argument, each is the sum of different s. The chain of inequalities
entries from
entries in and entries from those entries and those let the corresponding entries in be 1 and let the remaining entries be . Each has the probability and since
and , then due to symthis completes the proof. If metry, it is easy to show that if the number of nonzero entries in is a fix number like then the value of remains behave like a single symbol and . constant, i.e., and , then must be even and Thus, if
, we get . 3) Proof of Example 15 (Binary/Ternary, , and ): If , it is easy to show that if the number is equal to , then the value of of nonzero entries in remains constant. In other words, behave . Now, suppose that like a single symbol and and the nonzero entries of are all and . If has `` " and `` " entries, in positions where is equal to 2, we conclude that . The number of such
is
each with the probability
4) Proof of Example 16 (Binary/Ternary Uniform Distribution): The proof is similar to the proof of example 17. 5) Proof of Example 17 [Binary/Ternary (Uniform Distriwith probabilities bution)]: Note that ; let ; suppose that . Because is the third root of . If we denote as unity, one can deduce that , the sum of entries of , we have which yields . In order to satisfy , one should choose entries such that is equal to , and entries such that is equal to . Let the corresponding entries be and the same argument for and 1 entries of . in Thus, the number of which satisfy is equal to
(108) with the probability . 6) Proof of Example
(106) Therefore (107)
1) Proof of Example 13 (Binary Optical CDMA): The proof is similar to the proof of example 12. 2) Proof of Example 14 [Binary Complex Signature (Uniwith probabilities form Distribution)]: Note that . Let ; supand has `` " and `` " enpose that tries in positions where is equal to and , respectively.
. .
Thus,
(105) shows that for any input distribution
; s is
. The number of such
18
(Ternary/Binary
System, , ): Because and and , it is easy to show that if the number of and entries in is fixed, then the value of remains constant. In other words, and behave like and symbols. Let with the proba. Let bilities and ; from the aforementioned argument, we can assume that the nonzero entries of are all or . If has `` " entries where and `` " en, we get , tries where . The number of such is which yields with the probability Corollary 4, we get (56).
. By applying
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
7) Proof of Example 19 (Ternary Wireless System, , and ): By the same argument in the proof of Example 18, let with the prob. Let abilities and ; we can assume that the are all or . If has `` " and nonzero entries of , respectively, and `` " and `` " entries when , we get `` " entries when ; hence, . The number of such s is
1465
`` ," `` ," `` ," and is equal to 2, we get where , so thus, number of such
``
" entries in positions ; is an even number. The
s is
with the probability
. Hence, . The combinatorial
identities yield
with the proba-
. By applying Corolbility lary 4, we get (57). 8) Proof of Example 20 (Binary/Quaternary System for and Is a Set of Four AINs): Let and note that with the probabilities and . Let and assume that . Because s are independent ; thus, numbers, one has which yields . Note that to , one should choose entries when satisfy , and entries when , and let the corresponding be and the same argument for , and . entries in Thus, the number of s which satisfy is equal to with the probability
. The
following identities yield (58):
.
J. Proof of Proposition 6 Let and be the classes of quaternary and binary matrices, respectively. Let us say that is in relation with if and , where is attained by replacing each entry of by 0 and is attained by replacing each entry of by 0. Note that this relation defines a one-to-one correspondence between elements of and . It is easy to show that if and are related, then for any vector if and only if . Now, let be the matrix , where . Note if we denote which maximizes by and if the values of , then . And by is the matrix which the aforementioned argument if , for any in we have is in relation with if and only if . Thus, in is exactly the same as in . This equality leads to (110) The inequality in the other direction can be proved in a similar . way. K. Proof of Theorem 17 (General Lower Bound for the Sum Capacity) For a fixed
, we have
(109)
9) Proof of Example 21 (Binary/Quaternary System for and ): If and is a uniform distribution, it is easy to show that if the number of nonzero entries in is equal to , the value of remains constant; i.e., behave like the single symbol and . Now, assume that and the are all and . If has nonzero entries of
(111) But clearly
, and thus
1466
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
But
(112) be a random matrix with i.i.d. entries. By taking Now, let expectation with respect to , we get
(117) After substituting into the previous relations, we get
(113) (118)
Using Jensen’s inequality again, we will have Now, since
, we get
(119) (114) Now, note that hence
, and
(115) and and are independent for . Thus, the expectation and product operators commute; we then get For fixed
The right-hand side of the aforementioned inequality is a lower bound for the average mutual information where and are chosen at random with appropriate probability distributions. The choice of guarantees the SNR constraint on the average for this class of random matrices. Since the capacity is the and , the inequality holds. maximization over Note that although in the proof of the aforementioned theorem the SNR condition is only satisfied on the average, because of the measure concentration phenomenon [13], almost all ) result in almost the same power as well matrices (w.r.t. as almost the same mutual information. Thus, a typical matrix simultaneously satisfies both the SNR condition and the bound for mutual information with a high probability. To obtain a true bound for very small values of and , one can start from the equation
(120) where denotes the event that the (random) matrix the SNR condition. If , then
satisfies
(116) (121)
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
Now, it remains to evaluate a lower bound for
. But
1467
a linear combination of independent Gaussian random variables, which is again Gaussian. By using the fact that
(122) which can be controlled using the Markov inequality in the general case or using Chernoff-like inequalities if since is the summation of i.i.d. random variables. L. Proof of Corollary 6 (General Lower Bound for the Sum Capacity for -ary CDMA Systems) number of 0s and number of s. The Suppose has with the probability . number of such s is For such s, suppose is the number of terms in the product ; thus, and if s vectors are fixed numbers, the number of corresponding with the probability . is Therefore, we have
for the real case and and (70) are easily derived.
for the complex case, (69) .
O. Proof of Theorem 7 (A General Upper Bound for the Noisy Case) Our conjecture is based on i.i.d. user input distribution. Note that
When number of in the vector tribution which maximizes terms in of corresponding
is maximum when we have . Let be the i.i.d. product dis. Suppose is the number ; the number of
s is . Therefore,
with the probability has the following probability
density:
(124) where (123) (125)
M. Proof of Corollary 7 (Results for Finite User Inputs and Symmetric Signature Matrices—Noisy Case) Suppose has number of 0s and number of s. One can easily see that from the symmetric distribution on the signature alphabets, the expected value on the vector remains conbehave similar to the single symbol . The rest stant; i.e., of the proof is the same as Corollary 6. The second part is ex. actly the same as the first part and is straightforward.
is the distribution of a complex Gaussian. Noting to the obvious completes the proof. Also, relation and divides , we conjecture that when . . P. Proof of Theorem 19 (Asymptotic Lower Bound for Finite Input and Additive Gaussian Noise) According to Corollary 5, we have (126)
N. Proof of Theorem 18 (Lower Bound for Binary Input and Real/Complex Valued Signature Matrix) We use Corollary 5 where is the probability density of a standard Gaussian real r.v. for the real case, or the distribution of a Gaussian complex r.v. with independent standard Gaussian coordinates for the complex case. In this case, is a vector of i.i.d. entries with values . By noting that the distribution is symmetric and Gaussian, it entries of is fixed, is easy to see that if the number of then the expected value on the vector remains constant. Let be the number of entries of ; the number of such s is with the probability . is
(127)
(128)
1468
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012
But, (133) From the aforementioned equation and (129), we get the desired result. ACKNOWLEDGMENT
(129)
; name the empirical Now, let be a large number and probability measure induced by on as (i.e., a proportion of of entries of is ). In this case, . Therefore, when
is exponen-
, according to a tially small and negligible. When is a complex Gaussian random varicentral limit theorem able . Thus, we have
(130) , the aforementioned expression approaches If which yields a trivial bound. Hence, we assume that thus, we have
;
(131) where and are eigenvalues of the covariance matrix of real and imaginary parts of an r.v. which has the probability density of the product of two independent variables with densities and , and hence
(132) On the other hand, by the Sanov theorem [47] about the large deviation probability of empirical measures, the probability of appearance of is asymptotically equal to , where is the Kullback--Leibler distance of and . Thus, we have
This work would not have been possible without the great efforts of our brilliant and humble undergraduate students, specifically, A. A. Makhdoumi for editing the GCO section and providing Figs. 1–2. We would like to sincerely thank the academic staff and the students of Advanced Communications Research Institute (ACRI) and Center of Excellence on Multi-access Communications Systems of Sharif University of Technology, specially, Prof. J. A. Salehi, Prof. M. R. Aref, Prof. M. Nasiri, Dr. H. Saidi, Dr. B. Seif, and Dr. G. Abed-Hotani. We are also very indebted to our students, specially, M. H. Shafinia, S. M. Mansouri, P. Kabir, V. Montazerhodjat, A. Amini, M. Ferdosizadeh, A. Haghi, and R. Khosravi-Farsani for their helpful comments. We would also like to thank Prof. Arfaei and Prof. Alishahiha from Institute for Research for Fundamental Sciences for providing the facility to the authors for several days and nights to jump start this project. Finally, one of the authors, F. Marvasti, would like to dedicate this paper to his late mother who passed away unexpectedly during the preparation of this paper; she was a great advocate of academic excellence. REFERENCES [1] T. Ojanpera and R. Prasad, Wideband CDMA for Third Generation Mobile Communications: Universal Personal Communications, 1st ed. Norwood, MA: Addison-Wesley, 1998. [2] F. R. K. Chung, J. A. Salehi, and V. K. Wei, “Optical orthogonal codes: Design, analysis and applications,” IEEE Trans. Inf. Theory, vol. 35, no. 3, pp. 595–604, May 1989. [3] S. Mashhadi and J. A. Salehi, “Code-division multiple-access techniques in optical fiber networks part III: Optical AND gate receiver structure with generalized optical orthogonal codes,” IEEE Trans. Commun., vol. 54, no. 6, pp. 1349–1349, Jul. 2006. [4] J. A. Salehi, “Emerging OCDMA communication systems and data networks,” J. Opt. Network., vol. 6, no. 9, pp. 1138–1178, Sept. 2007. [5] S. Verdu, Multiuser Detection. New York: Cambridge Univ. Press, 1998. [6] A. J. Viterbi, CDMA: Principles of Spread Spectrum Communication, 1st ed. New York: Addison-Wesley, 1995. [7] J. L. Massey and T. Mittelholzer, “Welch’s bound and sequence sets for code-division multiple-access systems,” in Sequences II, Methods in Communication, Security, and Computer Sciences, R. Capocelli, A. De Santis, and U. Vaccaro, Eds. New York: Springer-Verlag, 1993. [8] M. Rupf and J. L. Massey, “Optimum sequences multisets for synchronous code-division multiple-access channels,” IEEE Trans. Inf. Theory, vol. 40, no. 4, pp. 1261–1266, Jul. 1994. [9] L. Welch, “Lower bound on the maximum cross correlation of signals,” IEEE Trans. Inf. Theory, vol. 20, no. 3, pp. 397–399, May 1974. [10] T. Tanaka, “A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors,” IEEE Trans. Inf. Theory, vol. 48, no. 11, pp. 2888–2910, Nov. 2002. [11] A. Montanari and D. N. C. Tse, “Analysis of belief propagation for non-linear problems: The example of CDMA (or: How to prove Tanaka’s formula),” in Proc. IEEE Inf. Theory Workshop, Punta del Este, Uruguay, Mar. 2006, pp. 160–164. [12] D. Guo and S. Verdu, “Randomly spread CDMA: Asymptotics via statistical physics,” IEEE Trans. Inf. Theory, vol. 51, no. 6, pp. 1983–2010, Mar. 2005.
ALISHAHI et al.: DESIGN OF SIGNATURE SEQUENCES FOR OVERLOADED CDMA AND BOUNDS
[13] S. B. Korada and N. Macris, “On the concentration of the capacity for a code division multiple access system,” in Proc. IEEE Int. Symp. Inf. Theory, June 2007, pp. 2801–2805. [14] S. B. Korada and N. Macris, “Tight bounds on the capacity of binary input random CDMA systems,” IEEE Trans. Inf. Theory, vol. 56, no. 11, pp. 5590–5613, Nov. 2010. [15] O. Shental, I. Kanter, and A. Weiss, “Capacity of complexity-constrained noise-free CDMA,” IEEE Commun. Lett., vol. 10, no. 1, pp. 10–12, Jan. 2006. [16] R. de Miguel, O. Shental, and R. Muller, “Information and multiaccess interference in a complexity-constrained vector channel,” J. Physics A, Math. Th., vol. 40, no. 20, pp. 5241–5260, 2007. [17] P. Pad, F. Marvasti, K. Alishahi, and S. Akbari, “A class of errorless codes for over-loaded synchronous wireless and optical CDMA systems,” IEEE Trans. Inf. Theory, vol. 55, no. 6, pp. 2705–2715, Jun. 2009. [18] K. Alishahi, F. Marvasti, V. Aref, and P. Pad, “Bounds on the sum capacity of synchronous binary CDMA channels,” IEEE Trans. Inf. Theory, vol. 55, no. 8, pp. 3577–3593, Aug. 2009. [19] D. Guo and C. C. Wang, “Multiuser detection of sparsely spread CDMA,” IEEE J. Sel. Areas Commun., vol. 26, no. 3, pp. 421–431, Apr. 2008. [20] W. H. Mow, “Recursive constructions of detecting matrices for multiuser coding: A unifying approach,” IEEE Trans. Inf. Theory, vol. 55, no. 1, pp. 93–98, Jan. 2009. [21] P. Pad, M. Soltanolkotabi, S. Hadikhanlou, A. Enayati, and F. Marvasti, “Errorless codes for over-loaded CDMA with active user detection,” presented at the ICC’09, Dresden, Germany, Jun. 2009. [22] S. Söderberg and H. S. Shapiro, “A combinatory detection problem,” Amer. Math. Monthly, vol. 70, no. 10, pp. 1066–1070, Dec. 1963. [23] R. Erdös and A. Rényi, “On two problems of information theory,” Magyar Tud. Akad. Mat. Kutato Int. Ko zl., vol. 8A, pp. 229–243, 1963. [24] B. Lindström, “On a combinatory detection problem,” Publ. Hung. Acad. Sci., vol. 9, pp. 195–207, 1964. [25] D. G. Cantor and W. H. Mills, “Determining a subset from certain combinatorial properties,” Can. J. Math., vol. 18, pp. 42–48, 1966. [26] B. Lindström and J. N. Srivastava, “Determining subsets by unramified experiments,” in A Survey of Statistical Design and Linear Models. New York: North-Holland, 1975. [27] S. S. Martirossian and G. H. Khachatrian, “Construction of signature codes and the coin weighing problem,” Probl. Inf. Trans., vol. 25, pp. 334–335, Oct.–Dec. 1989. [28] B. Lindström, “On möbius functions and a problem in combinatorial number theory,” Can. Math. Bull., vol. 14, pp. 513–516, 1971. [29] G. H. Khachatrian and S. S. Martirossian, “Codes for T-user noiseless adder channel,” Probl. Control Inf. Theory, vol. 16, no. 3, pp. 187–192, 1987. [30] M. H. Shafinia, P. Kabir, P. Pad, S. M. Mansouri, and F. Marvasti, “Errorless codes for CDMA system with near-far effect,” presented at the ICC2010, Capetown, South Africa, May 2010. [31] D. Guo, L. K. Rasmussen, S. Sun, and T. J. Lim, “A matrix-algebraic approach to linear parallel interference cancellation in CDMA,” IEEE Trans. Commun., vol. 48, no. 1, pp. 152–161, Jan. 2000. [32] S. B. Korada and A. Montanari, “Applications of Lindeberg principle in communications and statistical learning,” IEEE Trans. Inf. Theory, vol. 57, no. 4, pp. 2440–2450, Apr. 2011. [33] F. Marvasti, M. Ferdowsizadeh, and P. Pad, “Iterative synchronous and asynchronous multi-user detection with optimum soft limiter,” Patent application 12/122668, May 2008. [34] S. Dashmiz, P. Pad, and F. Marvasti, “New bounds for binary and ternary overloaded CDMA,” Jan. 2009, ArXiv:0901.1683v2. [35] M. Akhavan-Bahabadi and M. Shiva, “Double orthogonal codes for increasing capacity in MC-CDMA systems,” in Int. Conf. Wireless Opt. Commun. Netw., Mar. 2005, pp. 468–471. [36] R. Learned, A. Willsky, and D. Boroson, “Low complexity optimal joint decoding for oversaturated multiple access communications,” IEEE Trans. Signal Process., vol. 45, no. 1, pp. 113–123, Jan. 1997. [37] F. Marvasti, Nonuniform Sampling: Theory and Practice. New York: Springer-Verlag, 2001. [38] S. Dashmiz, M. R. Takapoui, P. Pad, and F. Marvasti, “New bounds for the sum capacity of binary and nonbinary synchronous CDMA systems,” in Proc. IEEE Int. Symp. Inf. Theory, June 2010, pp. 2093–2097.
1469
[39] S. Verdu, “The capacity region of the symbol-asynchronous Gaussian multiple-access channel,” IEEE Trans. Inf. Theory, vol. 35, no. 4, pp. 733–751, Jul. 1989. [40] J. Luo, S. Ulukus, and A. Ephremides, “Optimal sequences and sum capacity of symbol asynchronous CDMA systems,” IEEE Trans. Inf. Theory, vol. 51, no. 8, pp. 2760–2769, Aug. 2005. [41] S. Ulukus and R. D. Yates, “User capacity of asynchronous CDMA systems with matched filter receivers and optimum signature sequences,” IEEE Trans. Inf. Theory, vol. 50, no. 5, pp. 903–909, May 2004. [42] S. C. Chang and E. Weldon, “Coding for T-user multiple-access channels,” IEEE Trans. Inf. Theory, vol. 25, no. 6, pp. 684–691, Nov. 1979. [43] S. C. Chang, “Further results on coding for T-user multiple-access channels,” IEEE Trans. Inf. Theory, vol. 30, no. 2, pp. 411–415, Mar. 1984. [44] T. J. Ferguson, “Generalized T-user codes for multiple-access channels,” IEEE Trans. Inf. Theory, vol. 28, no. 5, pp. 775–778, Sep. 1982. [45] L. A. Shepp and I. Olkin, Department of Statistics, Stanford University, Stanford, CA, Entropy of the sum of independent Bernoulli random variables and of the multinominal distribution Tech. Rep. 131, 1978. [46] P. Harremoes, “Binomial and Poisson distributions as maximum entropy distributions,” IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 2039–2041, Jul. 2001. [47] A. Dembo and O. Zeitouni, Large Deviation Techniques and Application, 2nd ed. New York: Springer-Verlag, 2009.
K. Alishahi received his B.S., M.S., and Ph.D. degrees from the Department of Mathematical Sciences, Sharif University of Technology, Tehran, Iran, in 2000, 2002 and 2008 respectively. He has been an Assistant Professor with Sharif University of Technology since 2008 and a resident researcher at Institute for Researches in Foundational Sciences from 2010 to 2011. His research interests are stochastic processes and statistics.
Shayan Dashmiz (S’10) received the silver medal of International Mathematic Olympiad 2007 in Vietnam. He entered Sharif University of Technology as a double major student in Electrical Engineering in Communication and Pure Mathematics. He was an intern student at EPFL under the supervision of V. Cevher in 2011. He won the silver medal of International Math Olympiad for university students 2011 in Bulgaria. He is now a 5th year student at Sharif University, Tehran, Iran. His research interests are communication, signal processing, information theory and financial economy.
Pedram Pad (S’10) was born in Iran in 1986. He received the M.S. degree in electrical engineering, communications branch, and the B.S. degree in electrical engineering and pure mathematics at Sharif University of Technology. He is now a Ph.D. student at Ecole Polytechnique Federale de Lausanne (EPFL). He is a member of the Advanced Communications Research Institute (ACRI) at Sharif University. Mr. Pad received a gold medal in the National Mathematical Olympiad competition in 2003.
Farrokh Marvasti (S’72–M’74–SM’83) received the B.S., M.S. and Ph.D. degrees from Rensselaer Polytechnic Institute in 1970, 1971 and 1973, respectively. He has worked, consulted, and taught in various industries and academic institutions since 1972. Among which are Bell Labs, University of California Davis, Illinois Institute of Technology, University of London, King’s College. He was one of the editors and associate editors of the IEEE TRANSACTIONS ON COMMUNICATIONS AND SIGNAL PROCESSING from 1990-1997. He has about 100 journal publications and has written several reference books; he has also several international patents. His last book is on Nonuniform Sampling: Theory and Practice by Kluwer in 2001. He was also a guest editor for the Special Issue on Nonuniform Sampling for the Sampling Theory & Signal and Image Processing journal, May 2008. Besides being the co-founders of two international conferences (ICT’s and SampTA’s), he has been the organizer and special session chairs of many IEEEE conferences including ICASSP conferences. Dr Marvasti is currently a professor at Sharif University of Technology and the director Advanced Communications Research Institute (ACRI) and head of Center for Multi-Access Communications Systems.