Network Coding for Computing: Cut-Set Bounds - IEEE Xplore

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 2, FEBRUARY 2011

1015

Network Coding for Computing: Cut-Set Bounds Rathinakumar Appuswamy, Student Member, IEEE, Massimo Franceschetti, Member, IEEE, Nikhil Karamchandani, Student Member, IEEE, and Kenneth Zeger, Fellow, IEEE

Abstract—The following network computing problem is considered. Source nodes in a directed acyclic network generate independent messages and a single receiver node computes a target function f of the messages. The objective is to maximize the average number of times f can be computed per network usage, i.e., the “computing capacity”. The network coding problem for a single-receiver network is a special case of the network computing problem in which all of the source messages must be reproduced at the receiver. For network coding with a single receiver, routing is known to achieve the capacity by achieving the network min-cut upper bound. We extend the definition of min-cut to the network computing problem and show that the min-cut is still an upper bound on the maximum achievable rate and is tight for computing (using coding) any target function in multi-edge tree networks. It is also tight for computing linear target functions in any network. We also study the bound’s tightness for different classes of target functions. In particular, we give a lower bound on the computing capacity in terms of the Steiner tree packing number and a different bound for symmetric functions. We also show that for certain networks and target functions, the computing capacity can be less than an arbitrarily small fraction of the min-cut bound. Index Terms—Capacity, cut-set bound, function computation, information theory, network coding, throughput.

I. INTRODUCTION

W

E consider networks where source nodes generate independent messages and a single receiver node computes a target function of these messages. The objective is to characterize the maximum rate of computation, that is, the maximum number of times can be computed per network usage. Giridhar and Kumar [18] have stated: “In its most general form, computing a function in a network involves communicating possibly correlated messages, to a specific destination, at a desired fidelity with respect to a joint distortion criterion dependent on the given function of interest. This combines the complexity of source coding of correlated sources, with rate distortion, different possible network collaborative strategies for computing and communication, and the inapplicability of the separation theorem demarcating source and channel coding.” Manuscript received April 18, 2010; revised August 10, 2010; accepted September 11, 2010. Date of current version January 19, 2011. This work was supported by the National Science Foundation and the UCSD Center for Wireless Communications. This paper is part of the special issue on “Facets of Coding Theory: From Algorithms to Networks,” dedicated to the scientific legacy of Ralf Koetter. The authors are with the Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA 92093-0407 (e-mail: [email protected]; [email protected]; [email protected]; [email protected]). Communicated by M. Effros, Associate Editor for the special issue on “Facets of Coding Theory: From Algorithms to Networks.” Digital Object Identifier 10.1109/TIT.2010.2095070

The overwhelming complexity of network computing suggests that simplifications be examined in order to obtain some understanding of the field. We present a natural model of network computing that is closely related to the network coding model of Ahlswede, Cai, Li, and Yeung [1], [49]. Network coding is a widely studied communication mechanism in the context of network information theory. In network coding, some nodes in the network are labeled as sources and some as receivers. Each receiver needs to reproduce a subset of the messages generated by the source nodes, and all nodes can act as relays and encode the information they receive on in-edges, together with the information they generate if they are sources, into codewords which are sent on their out-edges. In existing computer networks, the encoding operations are purely routing: at each node, the codeword sent over an out-edge consists of a symbol either received by the node, or generated by it if it is a source. It is known that allowing more complex encoding than routing can in general be advantageous in terms of communication rate [1], [22], [38]. Network coding with a single receiver is equivalent to a special case of our function computing problem, namely when the function to be computed is the identity, that is, when the receiver wants to reproduce all the messages generated by the sources. In this paper, we study network computation for target functions different than the identity. Some other approaches to network computation have also appeared in the literature. In [8], [11], [12], [28], [34], [39] network computing was considered as an extension of distributed source coding, allowing the sources to have a joint distribution and requiring that a function be computed with small error probability. For example, [28] considered a network where two correlated uniform binary sources are both connected to the receiver and then determined the maximum rate of computing the parity of the messages generated by the two sources. A rate-distortion approach to the problem has been studied in [10], [15], [47]. However, the complexity of network computing has restricted prior work to the analysis of elementary networks. Networks with noisy links were studied in [3], [14], [16], [17], [19], [26], [35], [37], and [50]. For example, [17] considered broadcast networks where any transmission by a node is received by each of its neighbors via an independent binary symmetric channel. Randomized gossip algorithms [6] have been proposed as practical schemes for information dissemination in large unreliable networks and were studied in the context of distributed computation in [4]–[6], [9], [27], and [36]. In the present paper, our approach is somewhat (tangentially) related to the field of communication complexity [30], [48] which studies the minimum number of messages that two nodes need to exchange in order to compute a function of their inputs with zero error. Other studies of computing in networks have been considered in [18], and [43], but these were restricted

0018-9448/$26.00 © 2011 IEEE

1016


TABLE I EXAMPLES OF TARGET FUNCTIONS

to the wireless communication protocol model of Gupta and Kumar [20]. In contrast, our approach is more closely associated with wired networks with independent noiseless links. Our work is closest in spirit to the recent work of [31], [40]–[42] on computing the sum (over a finite field) of source messages in networks. We note that in independent work, Kowshik and Kumar [29] obtain the asymptotic maximum rate of computation in tree networks and present bounds for computation in networks where all nodes are sources. Our main contributions are summarized in Section I-C, after formally introducing the network model.

network code in a network Definition I.1: A of the following. and any out-edge (i) For any node encoding function

A. Network Model and Definitions

Given a network code, every edge carries a of at most alphabet symbols3, which is obtained vector on the set of vectors by evaluating the encoding function carried by the in-edges to the node and the node’s message vector if it is a source. The objective of the receiver is to compute the target function of the source messages, for any arbitrary message generator . More precisely, the receiver constructs a vector of alphabet symbols such that for each , the -th component of the receiver’s computed vector equals the value of the desired target function applied to the -th components of the source message vectors, for any choice of message generator . Let denote the in-edges of the receiver.

In this paper, a network consists of a finite, directed , a set of source nodes acyclic multigraph , and a receiver . Such a network is denoted by . We will assume that and that the graph1 contains a directed path from , let every node in to the receiver . For each node and denote the set of in-edges and out-edges of respectively. We will also assume (without loss of generality) that if a network node has no in-edges, then it is a source node. An alphabet is a finite set of size at least two. For any pos, and any , itive integer , any vector let denote the -th component of . For any index set with , let denote the vector . The network computing problem consists of a network and a target function of the form

consists , an

if otherwise. (ii) A decoding function:

in

Definition I.2: A network code is said to compute if the decoding function is such that for each and for every message generator (1)

(see Table I for some examples). We will also assume that any target function depends on all network sources (i.e., they cannot be constant functions of any one of their arguments). Let and be positive integers. Given a network with source set and alphabet , a message generator is any mapping

If there exists a code that computes , we say the rational is an achievable computing rate. number In the network coding literature, one definition of the coding capacity of a network is the supremum of all achievable coding rates [7], [13]. We adopt an analogous definition for computing capacity.

For each source components

Definition I.3: The computing capacity of a network respect to target function is

,

is called a message vector and its are called messages.2

1Throughout the paper, we will use “graph” to mean a directed acyclic multigraph, and “network” to mean a single-receiver network. We may sometimes write E ( ) to denote the edges of graph .

G

G

2For

simplicity, we assume that each source has exactly one message vector associated with it, but all of the results in this paper can readily be extended to the more general case.

3By

default, we will assume that edges carry exactly

n symbols.

with

APPUSWAMY et al.: NETWORK CODING FOR COMPUTING: CUT-SET BOUNDS

XY gx Y

x

y f x

Fig. 1. , are two sources with messages and respectively. nicates ( ) to so that can compute a function of and .

Y

y

1017

X commu-

Thus, the computing capacity is the supremum of all achievable computing rates for a given network and a target function . Definition I.4: For any target function , any , and any , , we write index set if for every , , we have whenever , , and for all . It can be verified that is an equivalence relation4 for every and . Definition I.5: For every and , let number of equivalence classes induced by

denote the total and let

Fig. 2. Example of a multi-edge tree.

Since target functions depend on all sources, we have for any cut and any target function . A multi-edge tree is a graph such that for every node , there exists a node such that all the out-edges of are in-edges (e.g., see Fig. 2). to , i.e., B. Classes of Target Functions We study the following four classes of target functions: (1) divisible, (2) symmetric, (3) -exponential, (4) -bounded.

be any function such that iff . That is, assigns a unique index to each equivalence class, and

The value of is independent of the choice of the footprint size of with respect to .

. We call

Remark I.6: Let . The footprint size has the following interpretation (see Fig. 1). Suppose a network has two nodes, and , and both are sources. A single directed edge connects to . Let generate and generate . communicates a function of its where , , input, to so that can compute . Then for any , such that , we need and . Thus, , which implies a lower needs to bound on a certain amount of “information” that send to to ensure that it can compute the function . Note that achieves the lower bound. We will use this intuition to establish a cut-based upper bound on the computing capacity of any network with respect to any target function , and to devise a capacity-achieving scheme for computing any target function in multi-edge tree networks. Definition I.7: A set of edges in network is said to separate sources from the receiver , if for each , every directed path from to contains if it at least one edge in . The set is said to be a cut in separates at least one source from the receiver. For any network , define to be the collection of all cuts in . For any and any target function , define cut

(2) 4Witsenhausen

[46] represented this equivalence relation in terms of the independent sets of a characteristic graph and his representation has been used in various problems related to function computation [11], [12], [39]. Although is defined with respect to a particular index set and a function , we do not make this dependence explicit—the values of and will be clear from the context.

I I

f

f

Definition I.8: A target function is divisible if for every index set , there exists a finite set and a function such that the following hold: ; (1) ; (2) (3) for every partition of , there exists a funcsuch that for every tion , we have . Examples of divisible target functions include the identity, sum, and arithmetic sum. maximum, Divisible functions have been studied previously5 by Giridhar and Kumar [18] and Subramanian, Gupta, and Shakkottai [43]. Divisible target functions can be computed in networks in a divide-and-conquer fashion as follows. For any arbitrary partiof the source indices , the receiver tion can evaluate the target function by combining evaluations . Furthermore, for every , the target of function can be evaluated similarly by partitioning and this process can be repeated until the function value is obtained. Definition I.9: A target function if for any permutation of

is symmetric and any vector ,

That is, the value of a symmetric target function is invariant with respect to the order of its arguments and hence, it suffices to evaluate the histogram target function for computing any symmetric target function. Examples of symmetric funcsum. tions include the arithmetic sum, maximum, and Symmetric functions have been studied in the context of computing in networks by Giridhar et al. [18], Subramanian et al. [43], Ying et al. [50], and [26]. . A target function Definition I.10: Let is said to be -exponential if its footprint size satisfies

5The

definitions in [18], [43] are similar to ours but slightly more restrictive.

1018


Let . A target function -bounded if its footprint size satisfies

is said to be

Example I.11: The following facts are easy to verify: • The identity function is 1-exponential. be an ordered set. The maximum (or minimum) • Let function is 1-bounded. where . The sum • Let is -bounded. target function with Remark I.12: Giridhar and Kumar [18] defined two classes of functions: type-threshold and type-sensitive functions. Both are sub-classes of symmetric functions. In addition, type-threshold functions are also divisible and -bounded, for some constant that is independent of the network size. However, [18] uses a model of interference for simultaneous transmissions and their results do not directly compare with ours. Following the notation in Leighton and Rao [33], the min-cut with unit-capacity edges is of any network -

(3)

A more general version of the network min-cut plays a fundamental role in the field of multi-commodity flow [33], [44]. The min-cut provides an upper bound on the maximum flow for any multi-commodity flow problem. The min-cut is also referred to as “sparsity” by some authors, such as Harvey, Kleinberg, and Lehman [22] and Vazirani [44]. We next generalize the definition in (3) to the network computing problem. Definition I.13: If then

is a network and

II. MIN-CUT UPPER BOUND ON COMPUTING CAPACITY The following shows that the maximum rate of computing a . target function in a network is at most Theorem II.1: If

is a network with target function , then

is a target function,

-

(4)

Example I.14: • If is the identity target function, then Thus for the identity function, the definition of min-cut in (3) and (4) coincide. . If is the arithmetic sum target • Let function, then -

is an upper bound on the function , the quantity . In Section III, we note that the computing capacity computing capacity for any network with respect to the identity target function (Theorem III.1) and linear target functions over finite fields (Theorem III.2) is equal to the min-cut upper bound. We show that the min-cut bound on computing capacity can also be achieved for all multi-edge tree networks with any target function (Theorem III.3). For any network and any target function, a lower bound on the computing capacity is given in terms of the Steiner tree packing number (Theorem III.5). Another lower bound is given for networks with symmetric target functions (Theorem III.7). In Section IV, the tightness of the abovementioned bounds is analyzed for divisible (Theorem IV.2), symmetric (Theorem IV.3), -exponential (Theorem IV.4), and -bounded (Theorem IV.5) target functions. For -exponential target functions, the computing capacity is at least times the min-cut. If every nonreceiver node in a network is a source, then for -bounded target functions the computing capacity is at least a constant times the min-cut divided by . It is also shown, with an example target function, that there are networks for which the computing capacity is less than an arbitrarily small fraction of the min-cut bound (Theorem IV.7). In Section V, we discuss an example network and target function in detail to illustrate the above bounds. In Section VI, conclusions are given and various lemmas are proven in the Appendix.

Proof: Let the network alphabet be and consider any code that computes in . Let be a cut and for each , let , . Suppose is such that , where is the equivalence relation from satsifying: Definition I.4. Then there exist , , , , and for every . The receiver can compute the target function only if, and correfor every such pair sponding to the message vectors generated by the sources in , the edges in cut carry distinct vectors. Since the total number of equivalence classes for the relation equals the footprint size , the edges in cut should carry at least distinct vectors. Thus, we have

(5)

• Let be an ordered set. If tion, then

is the maximum target func-

and hence for any cut

,

-

C. Contributions

Since the cut I.3 and (4).

is arbitrary, the result follows from Definition

The main results of this paper are as follows. In Section II, we show (Theorem II.1) that for any network and any target

The min-cut upper bound has the following intuition. Given , at least units of information any cut


need to be sent across the cut to successfully compute a target function . In subsequent sections, we study the tightness of this bound for different classes of functions and networks. III. LOWER BOUNDS ON THE COMPUTING CAPACITY The following result shows that the computing capacity of any network with respect to the identity target function equals the coding capacity for ordinary network coding. Theorem III.1: If tion , then

is a network with the identity target func-

1019

Every node by (8)) over

sends out a unique index (as guaranteed corresponding to the equivalence classes (9)

If has no in-edges, then by assumption, it is a source node, say . The set of equivalence classes in (9) is a function of its for . On the other hand, if own messages has in-edges, then let be the nodes with out-edges , using the uniqueness of the to . For each index received from , node recovers the equivalence classes (10)

-

Proof: Rasala Lehman and Lehman [32, p.6, Theorem 4.2] showed that for any single-receiver network, the conventional coding capacity (when the receiver demands the messages gen. Since erated by all the sources) always equals the the target function is the identity, the computing capacity is the coding capacity and , so the result follows.

Furthermore, the equivalence classes in (9) can be identified by from the equivalance classes in (10) (and if is a source node) using the fact that for a multi-edge tree network , if is not a source node, then we have a disjoint union

Theorem III.2: If is a network with a finite field alphabet and with a linear target function , then

If each node follows the above steps, then the receiver can identify the equivalence classes for . The receiver can evaluate for each from these equivalence classes. The above network code computes and achieves a computing rate of . From (7), it follows that

Proof: Follows from [41, Theorem 2]. Theorems III.1 and III.2 demonstrate the achievability of the min-cut bound for arbitrary networks with particular target functions. In contrast, the following result demonstrates the achievability of the min-cut bound for arbitrary target functions and a particular class of networks. The following theorem concerns multi-edge tree networks, which were defined in Section I-A. Theorem III.3: If function , then

is a multi-edge tree network with target -

Proof: Let be the network alphabet. From Theorem II.1, . Since it suffices to show that is a cut for node , and using (2), we have Consider any positive integers ,

(6)

(11)

We next establish a general lower bound on the computing capacity for arbitrary target functions (Theorem III.5) and then another lower bound specifically for symmetric target functions (Theorem III.7). with , define For any network to be a minimal (with respect to nodes a Steiner tree6 of and edges) subgraph of containing and such that every source in has a directed path to the receiver . Note that every nonreceiver node in a Steiner tree has exactly one out-edge. Let denote the collection of all Steiner trees in . For each edge , let . The is defined as the fractional Steiner tree packing number linear program

such that (7) .

Then we have

(12) (8)

code that computes in the multi-edge tree We outline a network . Each source generates a message vector . Denote the vector of -th components of the source messages by

Note that for any network , and the maximum value of the sum in (12) is attained at one or more vertices of the closed polytope corresponding to the linear constraints. Since all coefficients in the constraints are rational, the maximum value 6Steiner trees are well known in the literature for undirected graphs. For directed graphs a “Steiner tree problem” has been studied and our definition is consistent with such work (e.g., see [25]).

1020


in (12) can be attained with rational ’s. The following theorem provides a lower bound7 on the computing capacity for any network with respect to a target function and uses the quantity . In the context of computing functions, in the above linear program indicates the fraction of the time the edges in tree are used to compute the desired function. The fact that every . edge in the network has unit capacity implies Lemma III.4: For any Steiner tree of a network , let . Let be a cut in . Then there exists a cut in such that . is the set of indices of sources separated in (Note that by . The set may differ from the indices of sources by .) separated in Proof: Define the cut (13) is the collection of out-edges in of a set of sources disconnected by the cut in . If , then, by (13), disconnects from in , and thus . and let be a path from Let be a source such that to in . From (13), it follows that there exists such that contains at least one edge in . If also lies in and does not contain any edge in , then has a path to in that does not contain any edge in , thus contradicting the . Therefore, either does not lie in or fact that contains an edge in . Thus , i.e., . Theorem III.5: If function , then

is a network with alphabet

by . Let and let denote the quantity on the right hand side of (16). On every Steiner tree , a computing rate of is achievable by (16). Using standard arguments for at least time-sharing between the different Steiner trees of the network , it follows that a computing rate of at least is , the result follows. achievable in , and by letting The lower bound in Theorem III.5 can be readily computed and is sometimes tight. The procedure used in the proof of Theorem III.5 may potentially be improved by maximizing the sum

(17) where is any achievable rate8 for computing in the Steiner . tree network We now obtain a different lower bound on the computing capacity in the special case when the target function is the arithmetic sum. This lower bound is then used to give an alternative lower bound (in Theorem III.7) on the computing capacity for the class of symmetric target functions. The bound obtained in Theorem III.7 is sometimes better than that of Theorem III.5, and sometimes worse (Example III.8 illustrates instances of both cases). is a network with alphabet Theorem III.6: If and the arithmetic sum target function , then

and target where

Proof: Suppose of , and let to be in III.4 (taking

. Consider a Steiner tree . From Lemma ), we have

denotes the smallest prime number greater than . and let denote the same network Proof: Let as but whose alphabet is , the finite field of order . . From Theorem III.2, there exists a code Let that computes the -sum of the source messages in with an achievable computing rate satisfying

(14) Now we lower bound the computing capacity for the network with respect to target function :

-

(15)

(16) The lower bound in (16) is the same for every Steiner tree of . We will use this uniform bound to lower bound the computing capacity for with respect to . Denote the Steiner trees of 7In order to compute the lower bound, the fractional Steiner tree packing number 5( ) can be evaluated using linear programming. Also note that if we construct the reverse multicast network by letting each source in the original network become a receiver, letting the receiver in the become the only source, and reversing the direction of each edge, then it can be verified that the routing capacity for the reverse multicast network is equal to 5( ).

This code can be repeated to derive a code that also computes for any integer (note that edges in the network carry symbols from the alphabet , while those in the network carry symbols from a larger code that computes the -sum in alphabet ). Any can be “simulated” in the network by a code (e.g., see [2]). Furthermore, since and the , the -sum of the source source alphabet is is equal to their arithmetic sum. Thus, messages in network by choosing large enough, the arithmetic sum target function is computed in with an achievable computing rate of at least

N

N

N

N

Since is arbitrary, the result follows. 8From

Theorem III.3, r can be arbitrarily close to min-cut(t ; f ).


1021

Theorem III.7: If is a network with alphabet and a symmetric target function , then

where is the smallest prime number9 greater than . Proof: From Definition I.9, it suffices to evaluate the histogram target function for computing . For any set of source , we have messages

where work

for each . Consider the netwith alphabet . Then for each , can be evaluated by computing the arithmetic sum target function in where every source node is assigned , and 0 otherwise. Since we know that the message 1 if

N

N

• For network the histogram target function can be evaluated by computing the arithmetic sum target function times in the network with alphabet . Let . From Theorem III.6 in code that computes the ariththe Appendix, there exists a with an achievable computing metic sum target function in rate of at least

The above code can be repeated to derive a code . Note that edges in the netthat computes for any integer carry symbols from the alphabet work , while those in the network carry symbols from . Any code that computes the arithmetic sum can be simulated in the network by a target function in code10. Thus by choosing large enough, the above-mentioned code can be simulated in the network to derive a code that computes the histogram target function with an achievable computing rate11 of at least

Since is arbitrary, the result follows. Example III.8: Consider networks and in Fig. 3, each and the (symmetric) arithmetic sum with alphabet target function . Theorem III.7 provides a larger lower bound than Theorem III.5, but on the computing capacity . a smaller lower bound on

Bertrand’s Postulate [21, p.343], we have P (s) 2s. see details of such a simulation, we refer the interested reader to [2]. 11Theorem III.7 provides a uniform lower bound on the achievable computing rate for any symmetric function. Better lower bounds can be found by considering specific functions; for example Theorem III.6 gives a better bound for the arithmetic sum target function. 9From 10To

f A

g f g

has two binary sources ; and Fig. 3. Reverse Butterfly Network has three binary sources ; ; , each with = 0; 1 . network Each network’s receiver computes the arithmetic sum of the source messages.

and when

f

g

(in Fig. 3), we have , both of which occur, for example,

consists of the two in-edges to the receiver . Also, and , so

(18) In fact, we get the upper bound from Theorem II.1, and thus from (18), . , we have • For network

and

, both of which occur when Also,

From Theorem III.3, we have

and

. , so

.

Remark III.9: An open question in network coding, pointed out in [7], is whether the coding capacity of a network can be irrational. Like the coding capacity, the computing capacity is the supremum of ratios for which a code exists that computes the target function. Example III.8 demonstrates that the ) with unit capacity computing capacity of a network (e.g., links can be irrational when the target function is the arithmetic sum function. IV. ON THE TIGHTNESS OF THE MIN-CUT UPPER BOUND In the previous section, Theorems III.1—III.3 demonstrated upper three special instances for which the bound is tight. In this section, we use Theorem III.5 and Theorem III.7 to establish further results on the tightness of upper bound for different classes of target the functions. The following lemma provides a bound on the footprint size for any divisible target function .

1022


Lemma IV.1: For any divisible target function and any index set , the footprint size satisfies

Proof: The result follows immediately from Theorem III.7 and since for any network and any target function -

Proof: From the definition of a divisible target function, , there exist maps , , and such for any that

where . From the definition of the equivalence relation (see Definition I.4), it follows that , belong to the same equivalence class whenever . . We need This fact implies that to complete the proof which follows from Definition I.8(2). Theorem IV.2: If tion , then

The following results provide bounds on the gap between the computing capacity and the min-cut for -exponential and -bounded functions (see Definition I.10). and Theorem IV.4: If ponential target function , then

is a network with a -ex-

-

is a network with a divisible target funcProof: We have -

-

where denotes the set of in-edges of the receiver . Proof: Let be the network alphabet. From Theorem III.5

Therefore

(19) On the other hand, for any network , the set of edges a cut that separates the set of sources from . Thus

-

is where the last inequality follows because a computing rate is achievable for the identity target funcof tion from Theorem III.1, and the computing capacity for is lower bounded by the computing any target function capacity for the identity target function (since any target function can be computed from the identity function), i.e., .

-

(20) Combining (19) and (20) completes the proof. Theorem IV.3: If is a network with alphabet and symmetric target function , then

Theorem IV.5: Let . If is a network with alphabet and a -bounded target function , and all nonreceiver nodes in are sources, then the network -

where

12From

is the smallest prime number greater than and12

^ our assumption, R

2 for any target function f .

Proof: For any network such that all nonreceiver nodes are sources, it follows from Edmond’s Theorem [45, p.405, Theorem 8.4.20] that


1023

Then

-

(21)

On the other hand, -

(22)

N

M

; ; ;

Fig. 4. Network has binary sources f ... g, with A = f0 1g, connected to the receiver node via a relay . Each bold edge denotes parallel capacity-one edges and computes the arithmetic sum of the source messages.

L

;

Combining (21) and (22) gives networks. The proof of Theorem IV.7 uses Lemma VII.1 which is presented in the Appendix.

-

Theorem IV.7: For any , there exists a network that for the arithmetic sum target function

such

Since the maximum and minimum target functions are 1-bounded, and for each one, we get the following corollary. Corollary IV.6: Let be any ordered alphabet and let be any network such that all nonreceiver nodes in the network are sources. If the target function is either the maximum or the minimum function, then Theorems IV.2—IV.5 provide bounds on the tightness of upper bound for different classes of target the functions. In particular, we show that for -exponential (respectively, -bounded) target functions, the computing capacity is at least a constant fraction of the for any constant and any network (respectively, any where all nonreceiver nodes are sources). The network following theorem shows by means of an example target funcupper bound tion and a network , that the cannot always approximate the computing capacity up to a constant fraction. Similar results are known in network coding as well as in multicommodity flow. It was shown in [33] that when source nodes communicate independently with the same number of receiver nodes, there exist networks times a whose maximum multicommodity flow is well known cut-based upper bound. It was shown in [23] that with network coding there exist networks whose maximum times the best known cut bound throughput is (i.e., “meagerness”). Whereas these results do not hold for single-receiver networks (by Theorem III.1), the following similar bound holds for network computing in single-receiver

Proof: Consider the network depicted in Fig. 4 with and the arithmetic sum target function . alphabet Then we have Let be the number of sources disconnected from the receiver by a cut in the network . For each such source , the as well as either the parallel cut must contain the edge edges or the parallel edges . Thus Let

(23) attain the minimum in [23] and define . Then

-

(24)

(25) where (25) follows since the function its maximum value over at by (24)). Let us choose -

attains ( . We have (26) (27)

1024


We now prove a matching upper bound on the computing . Consider any code that computes capacity the arithmetic sum target function in network . For any , let

-

V. AN EXAMPLE NETWORK In this section, we evaluate the computing capacity for an example network and a target function (which is divisible and symmetric) and show that the min-cut bound is not tight. In addition, the example demonstrates that the lower bounds discussed in Section III are not always tight and illustrates the combinatorial nature of the computing problem. Theorem V.1: The computing capacity of network respect to the arithmetic sum target function is

That is, each element of is a possible pair of input edgevectors to the receiver when the function value equals . Let denote the number of components of that are either 0 or 3. Without loss of generality, suppose the first components and define by of belong to if if

with

.

Let

Proof: For any code that computes , let , , denote the message vectors generated by sources , , , respectively, and let , be the vectors and , respectively. carried by edges Consider any positive integers , such that is even and

and notice that (31)

(28) If

Then we have (29) We will describe a network code that computes , by network . Define vectors

in the

if if if if

(i) (ii) (iii)

, then: implies implies implies

; ; or (1,0).

Thus, the elements of consist of -bit vector pairs whose first components are fixed and equal (i.e., both are 0 and both are 1 when ), and whose remaining when components can each be chosen from two possibilities (i.e., either (0, 1) or (1, 0), when ). This observation implies that

. (32)

components of can take on the values 0, 1, The first components can take on the values 0, 1, 2, and the last possible values for , and so there are a total of . From (29), there exists a mapping that assigns similarly for unique values to for each different possible value of , and . This induces a code for that and similarly for computes as summarized below. sends its full message vector to The source each of the two nodes it is connected to. Source (respectively, ) computes the vector (respectively, ), then computes (respectively, ), and finally sends (respecthe vector and tively, ) on its out-edge. The receiver determines from and , respectively, and then computes , whose -th component is , i.e., the arithmetic sum target function . The above code achieves a computing . From [28], it follows that rate of (30)

changes, then the sum Notice that if only changes, and so must change (since is not a function of ) in order for the receiver to compute the target function. changes and does not change, then must Thus, if changes or not. More still change, regardless of whether , changes, then the pair , generally, if the pair must change. Thus (33) and therefore

(34)


1025

We have the following inequalities:

(35)

(36) where (35) follows since the ’s must be disjoint in order for the receiver to compute the target function. Taking logarithms of both sides of (36), gives

N

A f g

Fig. 5. Network ^ has three binary sources, , , and with = 0; 1 and the receiver computes the arithmetic sum of the source messages.

VI. CONCLUSIONS which holds for all

and , and therefore (37)

Combining (30) and (37) concludes the proof. Corollary V.2: For the network target function

with the arithmetic sum

Proof: Consider the network depicted in Fig. 5 with the arithmetic sum target function . It can be shown that the footprint size is for any cut , and thus The result then follows immediately from Theorem V.1. Remark V.3: In light of Theorem V.1, we compare the various lower bounds on the computing capacity of the network derived in Section III with the exact computing capacity. It can be shown that . If is the arithmetic sum target function, then

We examined the problem of network computing. The network coding problem is a special case when the function to be computed is the identity. We have focused on the case when a single receiver node computes a function of the source messages and have shown that while for the identity function the min-cut bound is known to be tight for all networks, a much richer set of cases arises when computing arbitrary functions, as the min-cut bound can range from being tight to arbitrarily loose. One key contribution of the paper is to show the theoretical breadth of the considered topic, which we hope will lead to further research. This work identifies target functions (most notably, the arithmetic sum function) for which the min-cut bound is not always tight (even up to a constant factor) and future work includes deriving more sophisticated bounds for these scenarios. Extensions to computing with multiple receiver nodes, each computing a (possibly different) function of the source messages, are of interest.

APPENDIX Define the function

as follows. For every

Thus, this example demonstrates that the lower bounds obtained in Section III are not always tight and illustrates the combinatorial nature of the problem.

such that each

(38)

1026


We extend

for

From (43), we have

by defining

. We now present Lemma VII.1. The proof uses Lemma VII.2, which is presented thereafter. We define the following function which is used in the next lemma. Let (39) denotes the inverse of the binary entropy function . Note that is an increasing function of . where

Lemma VII.1:

If

(44) The quantity is monotonic increasing from 0 to on the and the right hand side of (44) goes to zero as interval . Thus, the rate can be forced to be arbitrarily close sufficiently large, i.e., . to 1 by making In summary,

, then .

Proof: For any and , a code that computes with send computing rate 1 is obtained by having each source . Hence, its message directly to the receiver on the edge . Now suppose that has a code that computes with computing rate and for each , let

Lemma VII.2: Let , , . For each and let

that

be positive integers such that , let be such . Then

Proof: The result follows from Lemmas VII.4 and VII.7. be the corresponding encoding function on the edge , we have Then for any

.

The remainder of this Appendix is devoted to the proofs of lemmas used in the proof of Lemma VII.2. Before we proceed, we need to define some more notation. For every , define the map

(40) represents a set of possible message vectors of source Each . The left-hand side of (40) is the maximum number of different possible instantiations of the information carried by the possible vectors on each in-edges to the receiver (i.e., edge and possible vectors on the parallel edges ). The right-hand side of (40) is the number of distinct sum vectors that the receiver needs to discriminate, using the information carried by its in-edges. , let be such that For each and choose for each . Also, let

by if otherwise.

(45)

That is, the map subtracts one from the -th component of the input vector (as long as the result is nonnegative) and leaves , all the other components the same. For every define a map

. Then we have (41)

Thus (41) is a necessary condition for the existence of a code that computes in the network . Lemma VII.2 shows that13

by if otherwise for every

and

(46)

. Define

(42) where the function is defined in (39). Combining (41) and code that computes in the network with (42), any rate must satisfy

by (47)

(43) 13One can compare this lower bound to the upper bound j

1) which follows from [38].

Q(U

)j

M+

(

Note that (48)


A set is said to be invariant under the map if the set is is applied to it, in which case from (46) unchanged when , and (47) we would have that for each

1027

It can be verifed that is invariant under . For each invariant under

iff each , let

is

(49) and from (48) note that Lemma VII.3: For any and all integers such that , the set . invariant under the map , we have Proof: For any

and is

(58) Let

(50) The proof of the lemma is by induction on . For the base case , the proof is clear since from (50). Now suppose the lemma is true for all (where ) and let . Since and let from (50), the lemma is true when . In the following arguments, we take . From the induction hypothesis, is invariant under the map , i.e.,

and recall the definition of the function

in (38).

Lemma VII.4:

Proof: We begin by showing that (59)

(51) Consider any vector that

. From (49), we need to show . We have the following cases:

(52) (53)

For every

, let

and note that (60)

(54)

(61)

(55) (56)

where the two unions are in fact disjoint unions. We show that for every (62)

(57) Thus, the lemma is true for complete. Let

and the induction argument is be such that

each . Let

and extend the definition of

which by (60) and (61) implies (59). , then (62) is trivial. Now consider any If such that and let

for in

Then we have

(47) to products by (63) ,

Since

there

exists

such that is said to be invariant under

if (64)

1028


Then from the definition of the map in (46), there are ’s from amongst such that the and . Let be the index set for these vectors and let for each . Then for each , we have

of

Consider any . We show that , which implies by induction that . , then and we are done. Suppose that If . Since , there exists for every such that

and VII.3, Let

(65) From (64) and (65), for every

we have

for some is invariant under and

is an element of

. From Lemma and thus from (49),

.

The lemma below is presented in [3] without proof, as the proof is straightforward. Lemma VII.6: For all positive integers , ,

, and

and thus (67)

(66) Hence, we have

, let denote the Hamming weight of For any , i.e., the number of nonzero components of . The next lemma uses the function defined in (39). Lemma VII.7: and then from (60) and (61), it follows that

For any

and any , we know that . Thus, the same arguments as above can be repeated to show that

.. .

For any ,

, we say that

in

Proof: Let . The number of distinct elements with Hamming weight at most equals

For each there exists

if

for every

. Lemma VII.5: Let . If , then . , it can be obtained by iteratively subProof: Since tracting 1 from the components of , i.e., there exist and such that

, such that

It follows that , and

from (58) and hence . Let

for every

(68)

and

The number of vectors equals also in

in

such that

, and from Lemma VII.5, each such vector is . Therefore


Since

, the result follows. ACKNOWLEDGMENT

The authors would like to thank the reviewers for their comments which have greatly improved the presentation of this paper and would also like to thank Prof. Jacques Verstraete at UC San Diego for his help with the proof of Lemma VII.2. REFERENCES [1] R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung, “Network information flow,” IEEE Trans. Inf. Theory, vol. 46, no. 7, pp. 1204–1216, Jul. 2000. [2] R. Appuswamy, M. Franceschetti, N. Karamchandani, and K. Zeger, “Network computing capacity for the reverse butterfly network,” in Proc. IEEE Int. Symp. Information Theory, 2009, pp. 259–262. [3] O. Ayaso, D. Shah, and M. Dahleh, “Lower bounds on information rates for distributed computation via noisy channels,” in Proc. 45th Allerton Conf. Computation, Communication and Control, 2007. [4] O. Ayaso, D. Shah, and M. Dahleh, “Counting bits for distributed function computation,” in Proc. IEEE Int. Symp. Information Theory, 2008, pp. 652–656. [5] F. Benezit, A. G. Dimakis, P. Thiran, and M. Vetterli, “Gossip along the way: Order-optimal consensus through randomized path averaging,” in Proc. 45th Allerton Conf. Computation, Communication and Control, 2007. [6] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Randomized gossip algorithms,” IEEE Trans. Inf. Theory, vol. 52, no. 6, pp. 2508–2530, Jun. 2006. [7] J. Cannons, R. Dougherty, C. Freiling, and K. Zeger, “Network routing capacity,” IEEE Trans. Inf. Theory, vol. 52, no. 3, pp. 777–788, Mar. 2006. [8] P. Cuff, H. Su, and A. E. Gamal, “Cascade multiterminal source coding,” in Proc. IEEE Int. Symp. Information Theory, 2009, pp. 1199–1203. [9] A. G. Dimakis, A. D. Sarwate, and M. J. Wainwright, “Geographic gossip: Efficient aggregation for sensor networks,” in Proc. 5th Int. Conf. Information Processing in Sensor Networks, 2006, pp. 69–76. [10] V. Doshi, D. Shah, and M. Médard, “Source coding with distortion through graph coloring,” in Proc. IEEE Int. Symp. Information Theory, 2007, pp. 1501–1505. [11] V. Doshi, D. Shah, M. Médard, and S. Jaggi, “Graph coloring and conditional graph entropy,” in Proc. 40th Asilomar Conf. Signals, Systems and Computers, 2006, pp. 2137–2141. [12] V. Doshi, D. Shah, M. Médard, and S. Jaggi, “Distributed functional compression through graph coloring,” in Proc. Data Compression Conf., 2007, pp. 93–102. [13] R. Dougherty, C. Freiling, and K. Zeger, “Unachievability of network coding capacity,” IEEE Trans. Inf. Theory & IEEE/ACM Trans. Netw. (Joint Issue), vol. 52, no. 6, pp. 2365–2372, Jun. 2006. [14] C. Dutta, Y. Kanoria, D. Manjunath, and J. Radhakrishnan, “A tight lower bound for parity in noisy communication networks,” in Proc. 19th Annu. ACM-SIAM Symp. Discrete Algorithms, 2008, pp. 1056–1065. [15] H. Feng, M. Effros, and S. Savari, “Functional source coding for networks with receiver side information,” in Proc. 42nd Allerton Conf. Computation, Communication and Control, 2004, pp. 1419–1427. [16] R. G. Gallager, “Finding parity in a simple broadcast network,” IEEE Trans. Inf. Theory, vol. 34, no. , pp. 176–180, Mar. 1988. [17] A. El Gamal, “Reliable communication of highly distributed information,” in Open Prob. in Commun. and Comput., T. M. Cover and B. Gopinath, Eds. New York: Springer-Verlag, 1987, pp. 60–62. [18] A. Giridhar and P. R. Kumar, “Computing and communicating functions over sensor networks,” IEEE J. Select. Areas Commun., vol. 23, no. 4, pp. 755–764, Apr. 2005. [19] N. Goyal, G. Kindler, and M. Saks, “Lower bounds for the noisy broadcast problem,” SIAM J. Comput., vol. 37, no. 6, pp. 1806–1841, Mar. 2008.

1029

[20] P. Gupta and P. R. Kumar, “The capacity of wireless networks,” IEEE Trans. Inf. Theory, vol. 46, no. 3, pp. 388–404, Mar. 2000. [21] G. H. Hardy and E. M. Wright, An Introduction to the Theory of Numbers, 5th ed. Oxford, U.K.: Oxford University Press, 1979. [22] N. J. A. Harvey, R. Kleinberg, and A. Rasala Lehman, “On the capacity of information networks,” IEEE Trans. Inf. Theory & IEEE/ACM Trans. Netw. (Joint Issue), vol. 52, no. 6, pp. 2345–2364, Jun. 2006. [23] N. J. A. Harvey, R. D. Kleinberg, and A. Rasala Lehman, “Comparing network coding with multicommodity flow for the k-pairs communication problem,” M.I.T. LCS, Tech. Rep. 964, 2004. [24] W. Hoeffding, “Probability inequalities for sums of bounded random variables,” J. Amer. Statist. Assoc., vol. 58, no. 301, pp. 13–30, Mar. 1963. [25] K. Jain, M. Mahdian, and M. R. Salavatipour, “Packing steiner trees,” in Proc. 14th Ann. ACM-SIAM Symp. Discrete Algorithms, 2003, pp. 266–274. [26] N. Karamchandani, R. Appuswamy, and M. Franceschetti, “Distributed computation of symmetric functions with binary inputs,” in Proc. IEEE Information Theory Workshop, 2009, pp. 76–80. [27] D. Kempe, A. Dobra, and J. Gehrke, “Gossip-based computation of aggregate information,” in Proc. 44th Ann. IEEE Symp. Foundations of Computer Science, 2003, pp. 482–491. [28] J. Körner and K. Marton, “How to encode the modulo-two sum of binary sources,” IEEE Trans. Inf. Theory, vol. IT-25, pp. 29–221, Mar. 1979. [29] H. Kowshik and P. R. Kumar, “Zero-error function computation in sensor networks,” in Proc. IEEE Conf. Decision and Control, 2009, pp. 3787–3792. [30] E. Kushilevitz and N. Nisan, Communication Complexity. Cambridge, U.K.: Cambridge University Press, 1997. [31] M. Langberg and A. Ramamoorthy, “Communicating the sum of sources in a 3-sources/3-terminals network,” in Proc. IEEE Int. Symp. Information Theory, 2009, pp. 2121–2125. [32] A. Rasala Lehman and E. Lehman, “Complexity classification of network information flow problems,” in Proc. 15th Annu. ACM-SIAM Symp. Discrete Algorithms, 2003, pp. 142–150. [33] T. Leighton and S. Rao, “Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms,” J. ACM, vol. 46, no. 6, pp. 787–832, Nov. 1999. [34] N. Ma and P. Ishwar, “Two-terminal distributed source coding with alternating messages for function computation,” in Proc. IEEE Int. Symp. Information Theory, 2008, pp. 51–55. [35] N. Ma, P. Ishwar, and P. Gupta, “Information-theoretic bounds for multiround function computation in collocated networks,” in Proc. IEEE Int. Symp. Information Theory, 2009, pp. 2306–2310. [36] D. Mosk-Aoyama and D. Shah, “Fast distributed algorithms for computing separable functions,” IEEE Trans. Inf. Theory, vol. 54, no. 7, pp. 2997–3007, Jul. 2008. [37] B. Nazer and M. Gastpar, “Computing over multiple-access channels,” IEEE Trans. Inf. Theory, vol. 53, no. 10, pp. 3498–3516, Oct. 2007. [38] C. K. Ngai and R. W. Yeung, “Network coding gain of combination networks,” in Proc. IEEE Information Theory Workshop, 2004, pp. 283–287. [39] A. Orlitsky and J. R. Roche, “Coding for computing,” IEEE Trans. Inf. Theory, vol. 47, no. 3, pp. 903–917, Mar. 2001. [40] B. K. Rai and B. K. Dey, “Feasible alphabets for communicating the sum of sources over a network,” in Proc. IEEE Int. Symp. Information Theory, 2009, pp. 1353–1357. [41] B. K. Rai, B. K. Dey, and S. Shenvi, “Some bounds on the capacity of communicating the sum of sources,” in Proc. ITW 2010, Cairo, Egypt, 2010. [42] A. Ramamoorthy, “Communicating the sum of sources over a network,” in Proc. IEEE Int. Symp. Information Theory, 2008, pp. 1646–1650. [43] S. Subramanian, P. Gupta, and S. Shakkottai, “Scaling bounds for function computation over large networks,” in Proc. IEEE Int. Symp. Information Theory, 2007, pp. 136–140. [44] V. V. Vazirani, Approximation Algorithms, 1st ed. New York: Springer, 2004. [45] D. B. West, Introduction to Graph Theory. Upper Saddle River, NJ: Prentice-Hall, 2001. [46] H. Witsenhausen, “The zero-error side information problem and chromatic numbers,” IEEE Trans. Inf. Theory, vol. IT-22, pp. 592–593, Sep. 1976. [47] H. Yamamoto, “Wyner—Ziv theory for a general function of the correlated sources,” IEEE Trans. Inf. Theory, vol. IT-28, pp. 803–807, Sep. 1982.

1030


[48] A. C. Yao, “Some complexity questions related to distributive computing,” in Proc. 11th Annu. ACM Symp. Theory of Computing, 1979, pp. 209–213. [49] R. W. Yeung, A First Course in Information Theory. : Springer, 2002. [50] L. Ying, R. Srikant, and G. E. Dullerud, “Distributed symmetric function computation in noisy wireless sensor networks,” IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4826–4833, Dec. 2007.

Rathinakumar Appuswamy (S’05) received the B.Tech. degree from Anna University, Chennai, India, and the M.Tech. degree from the Indian Institute of Technology, Kanpur, India, both in electrical engineering in 2004, and the M.A. degree in mathematics from the University of California, San Diego, in 2008. He is currently a doctoral student at the University of California, San Diego, where he is a member of the Information and Coding Laboratory, as well as the Advanced Network Sciences Group. His research interests include network coding, communication for computing, and network information theory.

Massimo Franceschetti (M’98) received the Laurea degree (magna cum laude) in computer engineering from the University of Naples, Naples, Italy, in 1997, and the M.S. and Ph.D. degrees in electrical engineering from the California Institute of Technology, Pasadena, CA, in 1999, and 2003, respectively. He is an Associate Professor in the Department of Electrical and Computer Engineering, University of California at San Diego (UCSD). Before joining UCSD, he was a postdoctoral scholar at the University of California at Berkeley for two years. He has held visiting positions at the Vrije Universiteit Amsterdam, the Ecole Polytechnique Federale de Lausanne, and the University of Trento. His research interests are in communication systems theory and include random networks, wave propagation in random media, wireless communication, and control over networks. Dr. Franceschetti is an Associate Editor for Communication Networks of the IEEE TRANSACTIONS ON INFORMATION THEORY (2009–2012) and has served as Guest Editor for two issues of the IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATION. He was awarded the C. H. Wilts Prize in 2003 for

best doctoral thesis in electrical engineering at Caltech; the S.A. Schelkunoff Award in 2005 for best paper in the IEEE TRANSACTIONS ON ANTENNAS AND PROPAGATION; a National Science Foundation (NSF) CAREER award in 2006, an ONR Young Investigator Award in 2007; and the IEEE Communications Society Best Tutorial Paper Award in 2010.

Nikhil Karamchandani (S’05) received the B.Tech. degree in electrical engineering from the Indian Institute of Technology, Bombay, in 2005 and the M.S. degree in electrical engineering in 2007 from the University of California at San Diego (UCSD), where he is currently pursuing the Ph.D. degree in the Department of Electrical and Computer Engineering. His research interests are in communication theory and include network coding, information theory, and random networks. Mr. Karamchandani received the California Institute for Telecommunications and Information Technology (CalIT2) Fellowship in 2005.

Kenneth Zeger (S’85–M’90–SM’95–F’00) was born in Boston, MA, in 1963. He received the B.S. and M.S. degrees in electrical engineering and computer science from the Massachusetts Institute of Technology, Cambridge, in 1984, and both the M.A. degree in mathematics in 1989 and the Ph.D. degree in electrical engineering in 1990 from the University of California at Santa Barbara. He was an Assistant Professor of Electrical Engineering at the University of Hawaii from 1990 to 1992. He was with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, both at the University of Illinois at Urbana-Champaign, first as an Assistant Professor (1992 to 1995) and then as an Associate Professor (1995 to 1996). He has been with the Department of Electrical and Computer Engineering, University of California at San Diego, since 1996, as an Associate Professor (1996 to 1998) and currently as a Professor (since 1998). Dr. Zeger received a National Science Foundation (NSF) Presidential Young Investigator Award in 1991. He served as Associate Editor At-Large for the IEEE TRANSACTIONS ON INFORMATION THEORY during 1995–1998, and as a member of the Board of Governors of the IEEE Information Theory Society during 1998–2000, 2005–2007, and 2008–2010.