IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 54, NO. 1, JANUARY 2006
51
Loading for Parallel Binary Channels Xin Zhang, Student Member, IEEE, Shengli Zhou, Member, IEEE, and Peter Willett, Fellow, IEEE
Abstract—We develop optimal probability loading for parallel binary channels, subject to a constraint on the total probability of sending ones. The distinctions from the waterfilling power loading for parallel Gaussian channels, particularly the latter’s “dropping” of poor-quality channels, are highlighted. The only binary-input binary-output channel that is never dropped is the Z-channel. Index Terms—Capacity, on–off keying (OOK), parallel binary channels, waterfilling.
I. INTRODUCTION
I
T is well known that the optimal power loading for parallel Gaussian channels obeys the “waterfilling” principle [2]. Specifically, with parallel Gaussian channels , and a total power budget , the individual channel shall be assigned with power [2] (1)
to maximize the total mutual information , where , is the noise variance of the additive Gaussian noise , and the constant is solved by the power constraint . For those channels with positive power allocation, the is a constant (the water level), independent of the sum is low, weak channel index . When the power budget channels will be dropped with zero power assignment. Motivated by capacity maximization over parallel Gaussian channels, in this letter, we investigate capacity maximization over parallel binary channels. Instead of the total power constraint for Gaussian channels, we suitably impose a constraint on the total probability of sending ones over the binary channels. This total probability constraint could be interpreted as an average power constraint for binary channels: if one thinks of the on–off keying (OOK) type of modulation, the power consumption is proportional to the probability of sending ones. Does waterfilling’s dropped-channel phenomenon occur here also, for low-quality (binary) channels? We derive the optimal probability loading for parallel -ary-output (BIMO) channels, and then binary-input specify explicit solutions available for parallel binary-input binary-output (BIBO) channels and parallel binary erasure channels (BECs). We show that the derivative of the mutual information of each channel, with respect to the probability Paper approved by X. Dong, the Editor for Modulation and Signal Design of the IEEE Communications Society. Manuscript received September 14, 2004; revised April 21, 2005. This work was supported by the Office of Naval Research. The authors are with the Department of Electrical and Computer Engineering, University of Connecticut, Storrs, CT 06269 USA (e-mail:
[email protected];
[email protected];
[email protected]). Digital Object Identifier 10.1109/TCOMM.2005.861672
Fig. 1.
Parallel binary channels.
of sending ones, plays a key role. If all parallel channels have infinite derivatives at the origin—such as the “Z” channel and the symmetric BEC—then, perhaps surprisingly, no channels are ever dropped no matter how poor they are. Nondroppable channels share one common feature, that there exists at least one “clean” channel output: this output received means that a one was sent deterministically. II. PROBLEM FORMULATION As depicted in Fig. 1, we consider parallel binary channels. Denote the probability that the th channel sends one as . We impose a total probability constraint as (2) Our objective is to distribute the probability budget in (2) optimally among the parallel channels to maximize the total as the mutual information between capacity. Denote the channel input and output for the th channel. Assuming the channels are independent, to find the ensemble capacity, we must deal with (3) We thus have a constrained optimization problem, formulated as
subject to (4) The formulation in (4) consists of a standard convex optimization problem with inequality constraints, since the mutual information is a concave function of the input probability distribution [2, p. 31]. This problem can be efficiently solved by the
0090-6778/$20.00 © 2006 IEEE
52
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 54, NO. 1, JANUARY 2006
Lagrange method [1]. Specifically, we define an objective function as
(5) where , , are Lagrange multipliers. The optimal solution for the constrained optimization problem in (4) shall satisfy the Karush–Kuhn–Tucker (KKT) conditions [1] as (6)
Fig. 2.
ith binary input M -ary output channel.
from which we obtain
(7) (8) (9) Since the cost function is concave and the constraints are linear, the KKT conditions are necessary and sufficient in solving the primal problem of (4) as well as the corresponding dual problem, according to the strong duality theorem of [1, Prop. 5.2.1]. Based on (5), we simplify (6) to (10) The optimal solution for , , , can then be obtained based on (7)–(10). Lagrange multipliers, there are a total of With combinations of the possibilities each individual multiplier is equal to zero (in this case, the inequality constraint is not tight) or is larger than zero (in this case, the inequality constraint becomes tight). Hence, a brute-force approach to find the optimal solution will be cumbersome. We next simplify the optimization and present explicit solutions in special cases. We will provide insight to the “channel-dropping” effects associated with the waterfilling algorithm. III. PROBABILITY LOADING FOR BIMO CHANNELS We first consider the BIMO channel depicted in Fig. 2, where
(11) denote the cross probabilities of the th channel. For brevity, . The mutual information of channel is denote
(12)
(13) We next prove the following Lemma. Lemma 1: For any BIMO channel, tonically decreasing with , and
is mono-
(14) Proof: It is well known (see, e.g., [2, p. 31]) that is concave with . Hence, is monotonically decreasing. From (13), we define
(15) by Jensen’s inequality [2]. Similarly, we We obtain . have Lemma 1 leads to the following important observations. 1) Even without the total probability constraint, reaches its (unique) peak value at ( stands for unconstrained). Hence, the constraint in (9) is not . tight, with 2) When , the total probability conin (7). Otherwise, the straint is not tight, with should total probability constraint is tight, and be used in (7). Assume that the optimal solution is already obtained with a , and we now add an infinitescurrent probability budget on top of it. Based on imal (16) the channel with maximum derivative at the current probability assignment shall get the increment, to max. Together with Lemma 1, we then imize the increase on have 3) The competency (or, the priority) of the individual channel, determined by , is monotoniof the cally decreasing with the increasing share probability budget.
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 54, NO. 1, JANUARY 2006
53
4)
To decide whether a channel gets nonzero probability defined in (15). allocation, the important quantity is gradually increases from zero to a prescribed As level, the parallel channels get nonzero allocation in a . sequential fashion, in the nonincreasing order of Let us arrange the channels in the nonincreasing order of , i.e., . Observation 4) suggests that we can partition into nonoverlapping intervals , such that when , exactly the first channels get positive assignment. , . The rest of the boundary Obviously, points can be determined as follows. Denote as the optimal power allocation at the first channels, when is right at the boundary point of . Any on top of will lead to infinitesimal budget increment th channel. Thus, we can a positive assignment to the at this boundary point where and set , and rewrite the KKT conditions in (10) as Fig. 3. Illustration of the probability loading algorithm.
(17) Solving each individual we obtain
from (17) and summing them up,
(18) To summarize, the optimal loading solution can be obtained through the following steps. Fig. 4.
Optimal probability loading algorithm (1) Arrange the channels in a nonincreasing order of . , where is found by (2) If , set , and go to step (5); Otherwise, continue to step (3). , , and calculate from (18) for each (3) Set . (4) If falls in the interval , set . For each , substituting (13) into (10) , we have with the corresponding (19) For each given , solving (19) can be efficiently done by bisectional search on the monotonically decreasing func, or by lookup tables. The optimal is tion , such that the solutions searched in the interval of (19) satisfy . (5) This is the optimal probability loading solution. We illustrate the loading process by a simple example with three parallel binary channels, as shown in Fig. 3. In step (2), we . Thus, in step (3), we calculate find that for and, in step (4), we find falls between and , which means only channels 1 and 2 have positive loading,
and that
ith
general BIBO channel.
. We next vary within the interval , such . The optimal loading is then found as , , and . , we shall have at least one positive in For nonzero to be finite, as (18). This necessitates is always finite when . If otherwise, and have to be zero. Hence, we have the following. , then it will Corollary 1: If the th channel has always be used regardless of the probability budget. If all the , they will be used all together, channels have regardless of the probability budget. In this case, the waterfilling behavior, that weak channels are dropped, does not apply. for the BIMO channels. We observe Now let us check from (15) that as long as there exists such that and , then . On necessitates that at least for one , the other hand, and hold true. In accordance with both Corollary 1, we have the following. Corollary 2: In probability loading for parallel binary channels, a BIMO channel is never dropped, if and only if there exists such that and . The conditions and imply that the th output is “clean” corresponding to the transmission of one. In other words, when the th output appears, then without ambiguity we know a one has been sent.
54
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 54, NO. 1, JANUARY 2006
= +1
Fig. 5. Four possible BIBO channels with . Note that (a) is the Z-channel and that (c) is a special case of it with r are alternative representations of cases (a) and (c), with relabeled outputs.
= 0. Further, note that (b) and (d)
Remark: The convex optimization problem in (4) can be also solved by other standard nonlinear optimization techniques, e.g., the interior point, cutting-plane, and ellipsoid methods [1]. In addition, the Blahut–Arimoto algorithm [2, p. 364], a well-known method on channel capacity calculation, can be modified to solve our parallel probability loading problem, as the cost function in (4) retains the convexity of the original capacity problem. These methods could be very efficient in solving (4) numerically. We have presented the Lagrange method here since it provides valuable insight. Specifically, it reveals quite different “channel-dropping” effects from what is seen in power loading for parallel Gaussian channels. Fig. 6. ith BEC.
IV. OPTIMAL PROBABILITY LOADING FOR BIBO AND BEC CHANNELS A. Optimal Loading for BIBO Channels Here we consider the loading for parallel BIBO channels, with each channel depicted in Fig. 4. Define . Solving (17) for BIBO, we obtain (18) explicitly as
where is obtained by the constraint . Simcan be obtained in (21) by letting . Hence, ilarly, the optimal loading for parallel BIBO channels is up to only a one-dimensional numerical search on . are deThe four possible BIBO channels with picted in Fig. 5, with either one “clean” output, or two “clean” outputs, in accordance with Corollary 2. Hence, we have the following. Corollary 3: In probability loading for parallel binary channels, a BIBO channel is never dropped if and only if it is a Z-channel, as plotted in Fig. 5. B. General Binary Erasure Channels
(20) When first channels as
, we obtain positive assignments on
(21)
parallel BECs, with an individual We now consider and are different in channel depicted in Fig. 6, where general. We can view the BEC as a special case of the BIMO channel (denote 0, 1, as the first, second, and third outwith . Notice that when , the puts), and output one is always “clean,” and when , the output “ ” becomes “clean.” Corollary 2 leads to the following. Corollary 4: In probability loading for parallel BECs, the th , or . channel is never dropped if and only if either
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 54, NO. 1, JANUARY 2006
The optimal loading solution can be simplified for parallel symmetric BECs, where . Let us restrict , does not convey information and as the channel with should be dropped. The mutual information on the th channel is (22) By Corollary 4, the symmetric BECs will never be dropped. Hence, in (10). Substituting (22) into (10), we obtain (23) When , (23) becomes the unconstrained solution . To summarize, the optimal loading solution for parallel symmetric BECs is (24)
55
where
is found by
.
V. SUMMARY In this letter, we derived the optimal probability loading for parallel binary channels with a total probability constraint, as might occur when each transmission of a one carries with it an energy cost. We presented the solutions for parallel BIMO, BIBO, and BEC channels. The channel-dropping effects in the optimal loading were highlighted. Nondroppable channels are those for which there exists at least one “clean” channel output: this output being received means that a one was sent. For the binary-output case, the only never-dropped channel is the Z-channel; another example of a never-dropped channel is the symmetric BEC. REFERENCES [1] D. P. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA: Athena Scientific, 1999. [2] T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
