Time and Energy Complexity of Function Computation Over Networks

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

7671

Time and Energy Complexity of Function Computation Over Networks Nikhil Karamchandani, Student Member, IEEE, Rathinakumar Appuswamy, Student Member, IEEE, and Massimo Franceschetti, Senior Member, IEEE

p 2p

Abstract—This paper considers the following network computan grid, each node is tion problem: n nodes are placed on a n connected to every other node within distance r (n) of itself, and it is assigned an arbitrary input bit. Nodes communicate with their neighbors and a designated sink node computes a function f of the input bits, where f is either the identity or a symmetric function. We first consider a model where links are interference and noise-free, suitable for modeling wired networks. Then, we consider a model suitable for wireless networks. Due to interference, only nodes which do not share neighbors are allowed to transmit simultaneously, and when a node transmits a bit, all of its neighbors receive an independent noisy copy of the bit. We present lower bounds on the minimum number of transmissions and on the minimum number of time slots required to compute f . We also describe efficient schemes that match both of these lower bounds up to a constant factor and are thus jointly (near) optimal with respect to the number of transmissions and the number of time slots required for computation. At the end of the paper, we extend results on symmetric functions to general network topologies, and obtain a corollary that answers an open question posed by El Gamal in 1987 regarding the computation of the parity function over ring and tree networks. Index Terms—Communication complexity, error correction, function computation, information theory, sensor networks.

I. INTRODUCTION ENSOR networks consist of a large number of small nodes, capable of sensing, processing, and communicating data. These networks are typically required to sample a field of interest, do “in-network” computations, and then communicate a relevant summary of the data to a designated node(s), most often a function of the raw sensor measurements. For example, in environmental monitoring a relevant function can be the average temperature in a region. Another example is an intrusion detection network, where a node switches its value from 0 to 1 if it detects an intrusion and the function to be computed is the maximum of all the node values. For schemes that perform such

S

Manuscript received September 23, 2009; revised May 20, 2011; accepted July 12, 2011. Date of current version December 07, 2011. This work was supported in part by the National Science Foundation (NSF CNS-0916778, CNS 0546235, and CCF-0916465) and in part by the Center for Wireless Communications, University of California San Diego. The authors are with the Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093-0407 USA (e-mail: [email protected]; [email protected]; [email protected]). Communicated by R. A. Berry, Associate Editor for Communication Networks. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIT.2011.2168902

in-network aggregation, two relevant measures of complexity are the latency, i.e., the time it takes to perform the computation, and the corresponding energy spent. Network computation has been studied extensively in the literature, under a wide variety of models. In wired networks with point-to-point noiseless communication links, computation has been traditionally studied in the context of communication complexity [1]. Wireless networks, on the other hand, have three distinguishing features: the inherent broadcast medium, interference, and noise. Due to the broadcast nature of the medium, when a node transmits a message, all of its neighbors receive it. Due to noise, the received message at each neighbor is a noisy copy of the transmitted one. Due to interference, simultaneous transmissions can lead to message collisions. To avoid interference, a simple protocol model introduced in [2] allows only nodes which do not share neighbors to transmit simultaneously. The works in [3]–[5] study computation restricted to the protocol model of operation and assuming noiseless transmissions. A noisy communication model over independent binary symmetric channels was proposed in [6] in which when a node transmits a bit, all of its neighbors receive an independent noisy copy of the bit. Using this model, the works in [7]–[9] consider computation in a complete network where each node is connected to every other node and only one node is allowed to transmit at any given time. An alternative to the complete network is the random geometric network in which nodes are randomly deployed in continuous space inside a square and each node can communicate with all other nodes within a range1 . Computation in such networks under the protocol model of operation and with noisy communication has been studied in [10]–[13]. In these works the connection radius is assumed to be of order2 , which is the threshold required to obtain a connected random geometric network, see [14, Chapter 3]. In this paper, we consider the class of grid geometric networks grid is connected to every in which every node in a other node within distance from it, see Fig. 1. This construction has many useful features. By varying the connection radius we can study a broad variety of networks with contrasting structural properties, ranging from the sparsely connected grid to the densely connected complete network network for 1The connection radius r can be a function of n, but we suppress this dependence in the notation for ease of exposition. 2Throughout the paper we use the following subset of the Bachman-Landau notation for positive functions of the natural numbers: f (n) = O (g (n)) as n if k > 0, n : n > n f (n) kg (n); f (n) = (g (n)) as n if g (n) = O (f (n)); f (n) = 2(g (n)) as n if f (n) = O (g (n)) and f (n) = (g (n)). The intuition is that f is asymptotically bounded up to constant factors from above, below, or both, by g .

!1 9 1

0018-9448/$26.00 © 2011 IEEE

8



!1

!

7672

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

N

Fig. 1. Network (n; r): each node is connected to all nodes within distance r . The (red) node v is the sink that has to compute a function f of the input.

when . This provides intuition about how network properties like the average node degree impact the cost of computation. Above the critical connectivity radius for the random , the grid geometric network geometric network has structural properties similar to its random geometric counterpart and all the results in this paper also hold in that scenario. Thus, our study includes the two network structures studied in previous works as special cases. At the end of the paper, we also present some extensions of our results to arbitrary network topologies. We consider both noiseless wired communication over binary channels and noisy wireless communication over binary symmetric channels using the protocol model. We focus on computing two specific classes of functions with binary inputs, and measure the latency by the number of time slots it takes to compute the function and the energy cost by the total number of transmissions made in the network. The identity function (i.e., recover all source bits) is of interest because it can be used to compute any other function and thus gives a baseline to compare with when considering other functions. The class of symmetric functions includes all functions such that for any input and permutation on

In other words, the value of the function only depends on the . Many functions arithmetic sum of the arguments, i.e., which are useful in the context of sensor networks are symmetric, for example the average, maximum, majority, and parity. A. Statement of Results Under the communication models described above, and for any connection radius , we prove lower bounds on the latency and on the number of transmissions required for computing the identity function (Theorems III.1 and IV.1). We then describe a scheme which matches these bounds up to a constant factor (Theorems III.2 and IV.2). Next, we consider the class of symmetric functions. For a particular symmetric function (parity function), we provide lower bounds on the latency and on the number of transmissions for computing the function (Theorems III.3 and IV.3). We then present a scheme

which can compute any symmetric function while matching the above bounds up to a constant factor (Theorems III.4 and IV.6). These results are summarized in Tables I and II. They illuson the cost trate the effect of the average node degree of computation under both communication models. By comparing the results for the identity function and symmetric functions, we can also quantify the gains in performance that can be achieved by using in-network aggregation for computation, rather than collecting all the data and performing the computation at the sink node. Finally, we extend our schemes for computing symmetric functions to more general network topologies (Theorem V.1) and obtain a lower bound on the number of transmissions required for arbitrary connected networks (Theorem V.2). A corollary of this result answers an open question originally posed by El Gamal in [6] regarding the computation of the parity function over ring and tree networks. We point out that most of previous work ignored the issue of latency and it is only concerned with minimizing the number of transmissions required for computation. Our schemes are latency-optimal, in addition to being efficient in terms of the number of transmissions required. The works in [5], [11] consider the question of latency, but only for the case of . B. Organization of the Paper The rest of the paper is organized as follows. We formally describe the problem and present some preliminary remarks in Section II. Grid geometric networks with noiseless links are considered in Section III and their noisy counterparts are studied in Section IV. Extensions to general network topologies are presented in Section V. In Section VI we draw conclusions and mention some open problems. II. PROBLEM FORMULATION A network of nodes is represented by an undirected graph. Nodes in the network represent communication devices and edges represent communication links. For each node , let denote its set of neighbors. Each node is assigned an input bit . Let denote the vector whose component is . We refer to as the input to the network. The nodes communicate with each other so that a designated sink node can compute a target function of the input bits

where denotes the co-domain of . Time is divided into slots of unit duration. The communication models are as follows. • Noiseless point-to-point model: If a node transmits a bit in a time slot, then node receives the on an edge bit without any error in the same slot. All the edges in the network can be used simultaneously, i.e., there is no interference. • Noisy broadcast model: If a node transmits a bit in time slot , then each neighboring node in receives an independent noisy copy of in the same slot. More precisely, receives where denotes the neighbor is a bernoulli random variable that modulo-2 sum. takes value 1 with probability and 0 with probability .

KARAMCHANDANI et al.: TIME AND ENERGY COMPLEXITY OF FUNCTION COMPUTATION OVER NETWORKS

7673

TABLE I RESULTS FOR NOISELESS GRID GEOMETRIC NETWORKS

TABLE II RESULTS FOR NOISY GRID GEOMETRIC NETWORKS

The noise bits are independent over , , and . A network in the noisy broadcast model with link error probability is called an -noise network. We restrict to the protocol model of operation, namely two nodes and can transmit in the same time slot only if they do not have any . Thus, any node common neighbors, i.e., can receive at most one bit in a time slot. In the protocol model originally introduced in [2], communication is reliable. In our case, even if bits do not collide at the receiver because of the protocol model of operation, there is still a probability of error which models the inherent noise in the wireless communication medium. A scheme for computing a target function specifies the order in which nodes in the network transmit and the procedure for each node to decide what to transmit in its turn. A scheme is defined by the total number of time slots of its execution, and for each slot , by a collection simultaneously transmitting nodes and of . In the corresponding encoding functions any time slot , node computes the function of its input bit and the bits it received before time and then transmits this value. In the noiseless point-to-point case, nodes in the list are repeated for each distinct edge on which they transmit in a rounds of communication, the sink given slot. After the computes an estimate of the value of the funcnode tion . Note that the duration of a scheme and the total are constants for all inputs number of transmissions . Our definition of a computing scheme has a number of desirable properties. First, schemes are oblivious in the sense that in any time slot, the node which transmits is decided ahead of time and does not depend on a particular execution of the scheme. Without this property, the noise in the network may lead to multiple nodes transmitting at the same time, thereby causing collisions and violating the protocol model. Second, the definition rules out communication by silence: when it is a node’s turn to transmit, it must send something. We call a scheme a -error scheme for computing if for any input , . For both the noiseless and noisy broadcast communication models, our objective is to characterize the minimum number of time slots and the minimum number of transmissions required by any -error scheme for computing a target function in a network . We first focus on grid geometric networks of connection ra-

, and then extend our results to more dius , denoted by general network topologies. A. Preliminaries We now present a few useful observations. , every node Remark II.1: For any connection radius is isolated, and hence, in the grid geometric network computation is infeasible. On the other hand, for any , is fully connected. Thus, the interesting the network . regime is when the connection radius Remark II.2: For any connection radius every node in the grid geometric network neighbors.

, has

Theorem II.3: (Gallager’s Coding Theorem), [10, Page 3, Theorem 2], [15]: For any and any integer , there exists a code for sending an -bit message over a binary symtransmissions such that the message metric channel using . is received correctly with probability at least III. NOISELESS GRID GEOMETRIC NETWORKS We begin by considering computation of the identity function. We have the following straightforward lower bound. , Theorem III.1: Let be the identity function, let . Any -error scheme for computing over and let requires at least time slots and transmissions. Proof: To compute the identity function the sink node should receive at least bits. Since has neighbors and can receive at most one bit on each edge in a time slot, time slots to compute the idenit will require at least tity function. Let a cut be any set of edges separating at least one node from the sink . It is easy to verify that there exists a collection of disjoint cuts such that each cut separates nodes from the sink , see Fig. 2 for an example. Thus, to ensure that can compute the identity function, there should be at least transmissions across each cut. The lower bound on the total number of transmissions follows. We now present a simple scheme for computing the identity function which is order-optimal in both the latency and the number of transmissions.

7674

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

transmissions in the network. Since each of bits and node in the left-most column of cells has parallel chains each of length , the there are second phase uses transmissions and time slots. In the final phase, each of the in nodes in the cell with has bits, and hence, transmissions and slots to finish. it requires Adding the costs, the scheme can compute the identity function transmissions and time slots. with Now we consider the computation of symmetric functions. We have the following straightforward lower bound:

N

Fig. 2. Each dashed (magenta) line represents a cut of network (n; r) which separates at least n=4 nodes from the sink v . Since the cuts are separated by a distance of at least 2r , the edges in any two cuts, denoted by the solid (blue) lines, are disjoint.

Fig. 3. Scheme for computing the identity function works in three phases: the solid (blue) lines depict the first horizontal aggregation phase, the dashed (magenta) lines denote the second vertical aggregation phase, and the dotted (red) lines represent the final phase of downloading data to the sink.

Theorem III.2: Let be the identity function and let . There exists a zero-error scheme for computing over which requires at most time slots and transmissions. Proof: Let . Consider a partition of the netinto cells of size , see Fig. 3. Note that work each node is connected to all nodes in its own cell as well as in any neighboring cell. The scheme works in three phases, see Fig. 3. In the first phase, bits are horizontally aggregated towards the left-most column of cells along parallel linear chains. In the second phase, the bits in the left-most cells are vertically aggregated towards the nodes in the cell containing the sink node . In the final phase, all the bits are collected at the sink node. parallel The first phase has bits aggregating along . By pipelining the translinear chains each of length missions, this phase requires time slots and a total

and let . Theorem III.3: Let such that any There exists a symmetric target function -error scheme for computing over requires at least time slots and transmissions. Proof: Let be the parity function. To compute this function, each non-sink node in the network should transmit at least transmissions are required. Since once. Hence, at least the bit of the farthest node requires at least time slots to reach , we have the desired lower bound on the latency of any scheme. Next, we present a matching upper bound. Theorem III.4: Let be any symmetric function and let . There exists a zero-error scheme for computing over which requires at most time slots and transmissions. Proof: We present a scheme which can compute the arithin at most metic sum of the input bits over time slots and transmissions. This suffices to prove the result since is symmetric, and thus, its value only depends on the arithmetic sum of the input bits. Again, consider a partition of the noiseless network into cells of size with . For each cell, pick one node arbitrarily and call it the “cell-center”. For the cell conto be the cell-center. The scheme works taining , choose in two phases, see Fig. 4. First Phase: All the nodes in a cell transmit their input bits to the cell-center. This phase requires only one time-slot and transmissions and at the end of the phase each cell-center knows the arithmetic sum of the input bits in its cell, which is an ele. ment of Second Phase: In this phase, the bits at the cell-centers are aggregated so that can compute the arithmetic sum of all the input bits in the network. There are two cases, depending on the connection radius . : Since each cell-center is connected to the • other cell-centers in its neighboring cells, this phase can be mapped to computing the arithmetic sum over the noisewhere each node observes a less network . See Fig. 4(a) for an illustramessage in tion. In Appendix I we present a scheme to complete this transmissions and time phase using slots. : The messages at cell-centers are aggre• along a tree, see Fig. 4(b). The value at gated towards each cell-center can be viewed as a -length binary

KARAMCHANDANI et al.: TIME AND ENERGY COMPLEXITY OF FUNCTION COMPUTATION OVER NETWORKS

p

7675

p

Fig. 4. Figures (a) and (b) represent the cases r 8 log n and r > 8 log n respectively. The scheme for computing any symmetric function works in two phases: the solid (blue) lines indicate the first phase which is the same in both cases. The second phase differs in the two cases. It is represented by the dashed (magenta) lines in Fig. (a) and the dashed (red) lines in Fig. (b).

vector. To transmit its vector to the parent (cell-center) node in the tree, every leaf node (in parallel) transmits each bit of the vector to a distinct node in the parent cell. In the next time slot, each of these intermediate nodes relays its received bit to the corresponding cell-center. The parent cell-center can then reconstruct the message and aggregate -length binary it with its own value to form another vector. Note that it requires two time slots and transmissions by a cell-center to traverse one level of depth in the aggregation tree. This step is performed rereceives the peatedly (in succession) till the sink node sum of all the input bits in the network. Since the depth , the phase requires of the aggregation tree is time slots. There are transmissions in each cell of the network. Hence, the phase requires a total transmissions. of Adding the costs of the two phases, we conclude that it is transpossible to compute any symmetric function using time slots. missions and IV. NOISY GRID GEOMETRIC NETWORKS We start by considering the computation of the identity function. We have the following lower bound. Theorem IV.1: Let be the identity function. Let , let , and let . Any -error scheme for computing over an -noise grid geometric network requires at least time slots and transmissions. Proof: The lower bound of transmissions follows from the same argument as in the proof of Theorem III.1. transmissions follows The other lower bound of from [8, Corollary 2]. We now turn to the number of time slots required. For computing the identity function, the sink node should receive at bits. However, the sink can receive at most one bit least

in any slot, and hence, any scheme for computing the identity time slots. For the remaining function requires at least lower bound, consider a partition of the network into with . Since the total number cells of size and of transmissions in the network is at least cells, there is at least one cell where the there are number of transmissions is at least . Since all nodes in a cell are connected to each other, at most one of them can transmit in a slot. Thus, any scheme for computing the identime slots. tity function requires at least Next, we present an efficient scheme for computing the identity function in noisy broadcast networks, which matches the above bounds. be the identity function. Let Theorem IV.2: Let , let , and let . over an There exists a -error scheme for computing -noise grid geometric network which requires time slots and at most transmissions. Proof: Consider the usual partition of the network into cells of size with . By the protocol model of operation any two nodes are allowed to transmit in the same time slot only if they do not have any common neighbors. In line with the protocol model of operation, cells are scheduled according to the scheme shown in Fig. 5. Thus, each cell is scheduled once every 7 7 time slots. Within a cell, at most one node can transmit in any given time slot and nodes take turns to transmit one after the other. For each cell, pick one node arbitrarily and call it the “cell-center”. The scheme works in three phases, see Fig. 6. First Phase: There are two different cases, depending on the connection radius . • : In this case, each node in its turn transmits its input bit to the corresponding cell-center using a code-

7676

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

Fig. 5. Cells with the same number (and color) can be active in the same time slot and different numbers (colors) activate one after the other. Each cell is active once in 49 slots.

. Since there are at most cells in at most the network for this case, the probability that the computa. tion fails in at least one cell is bounded by Thus, at the end of the first phase, all cell-centers in the network have the input bits of the nodes in their cells with proba. bility at least Second Phase: In this phase, the messages collected at the cell-centers are aggregated horizontally towards the left-most horizontal chains cells, see Fig. 6. Note that there are input messages. In each such and each cell-center has chain, the rightmost cell-center maps its set of messages into and transmits it to the next a codeword of length cell-center in the horizontal chain. The receiving cell-center decodes the incoming codeword, appends its own input mes, and sages, re-encodes it into a codeword of length then transmits it to the next cell-center, and so on. This phase time slots and a total of requires at most transmissions in the network. From at most Theorem II.3, this step can be executed without error with . probability at least Third Phase: In the final phase, the messages at the cell-centers of the left-most column are aggregated vertically towards the sink node , see Fig. 6. Each cell-center maps its set of input and transmits it messages into a codeword of length to the next cell-center in the chain. The receiving cell-center decodes the incoming message, re-encodes it, and then transmits it to the next node, and so on. By pipelining the transmissions, time slots and this phase requires at most transmissions in the network. This at most phase can also be executed without error with probability at least . It now follows that at the end of the three phases, the sink can compute the identity function with probability of node . Thus, for large enough, we have error at most a -error scheme for computing any symmetric function in the network . Adding the costs of the phases, the scheme time slots and requires at most transmissions.

Fig. 6. Scheme for computing the identity function in a noisy network involves three phases: the solid (blue) lines indicate the first in-cell aggregation phase, the dashed (magenta) lines represent the second horizontal aggregation phase, and the dotted (red) lines represent the final vertical aggregation phase.

We now discuss the computation of symmetric functions in noisy broadcast networks. We begin with a lower bound on the latency and the number of transmissions required.

word of length such that the cell-center decodes . the message correctly with probability at least The existence of such a code is guaranteed by Theorem time slots II.3. This phase requires at most and at most transmissions in the network. Since cells in the network, the probability there are that the computation fails in at least one cell is bounded . by • : In this case, each cell uses the more sophisticated scheme described in [8, Section 7] for recovering all the input messages from the cell at the cell-center. This time slots and a scheme requires at most total of at most transmissions in the network. At the end of the scheme, a cell-center has all the input messages from its cell with probability of error

, let , and let Theorem IV.3: Let for any . There exists a symmetric target function such that any -error scheme for computing over an -noise grid geometric network requires time slots and at least transmissions. We briefly describe the idea of the proof before delving into details. Let be the parity function. First, we notice that [12, Theorem 1.1, page 1057] immediately implies that any -error scheme for computing over requires at least transmissions. So, we only need to establish that transmissions. any such scheme also requires Suppose there exists a -error scheme for computing the parity function in an -noise grid geometric network which requires transmissions. In Lemma IV.5, we translate operating on a “noisy the given scheme into a new scheme

KARAMCHANDANI et al.: TIME AND ENERGY COMPLEXITY OF FUNCTION COMPUTATION OVER NETWORKS

Fig. 7.

n-noisy star network.

star” network (see Fig. 7) whose noise parameter depends on , such that the probability of error for the new scheme is also at most . In Lemma IV.4, we derive a lower bound on the probability of error of the scheme in terms of the noise parameter of the noisy star network. Combining these results we obtain the desired lower bound on the number of transmissions . We remark that while the proof of the lower bound in [12, Theorem 1.1, page 1057] operates a transformation to a problem over “noisy decision trees”, here we need to transform the problem into one over a noisy star network. Hence, the two different transformations lead to different lower bounds on the number of transmissions required for computation. An -noisy star network consists of input nodes and one auxiliary node . Each of the input nodes is connected divia a noisy link, see Fig. 7. We have the following rectly to result for any scheme which computes the parity function in an -noisy star network: Lemma IV.4: Consider an -noisy star network with noise parameter and let the input be distributed uniformly over . For any scheme which computes the parity function (on bits) in the network and in which each input node transmits its input bit only once, the probability of error is at . least Proof: See Appendix II-A. We have the following lemma relating the original network and a noisy star network. Lemma IV.5: Let . If there is a -error scheme for computing the parity function (on input bits) in with transmissions, then there is a -error scheme for cominput bits) in an -noisy star puting the parity function (on network with noise parameter , with each input node transmitting its input bit only once. Proof: See in Appendix II-B. We are now ready to complete the proof of Theorem IV.3. . If there is a Proof (of Theorem IV.3): Let -error scheme for computing the parity function in which requires transmissions, then by combining Lemmas IV.5 and IV.4, the following inequalities must hold:

(1)

7677

follows since for every . Thus, we where have that any -error scheme for computing the parity function requires at least in an -noise network transmissions. We now consider the lower bound on the number of time slots. Since the message of the farthest node requires at least time slots to reach , we have the corresponding lower bound on the duration of any -error scheme. The lower time slots follows from the same arbound of gument as in the proof of Theorem IV.1. We now present an efficient scheme for computing any symmetric function in a noisy broadcast network which matches the above lower bounds. Theorem IV.6: Let be any symmetric function. Let , let , and let . There exists a -error scheme for computing over an -noise grid geometric network which requires time slots and at most transmissions. Proof: We present a scheme which can compute the arith. Note that this suffices metic sum of the input bits over to prove the result since is symmetric, and thus, its value only depends on the arithmetic sum of the input bits. into cells Consider the usual partition of the network of size with . For each cell, we pick one node arbitrarily and call it the “cell-center”. As before, cells are scheduled according to Fig. 5 to prevent interference between simultaneous transmissions. The scheme works in three phases. First Phase: The objective of the first phase is to ensure that each cell-center computes the arithmetic sum of the input messages from the corresponding cell. Depending on the connection radius , this is achieved using two different strategies. : In Appendix III, we describe a • scheme which can compute the partial sums at all cellcenters with probability at least and requires total transmissions and time slots. : In this case, we first divide each • nodes cell into smaller sub-cells with each, see Fig. 8. Each sub-cell has an arbitrarily chosen “head” node. In each sub-cell, we use the Intracell scheme from [10, Section III] to compute the sum of the input bits from the sub-cell at the corresponding head node. transmissions from each node This requires nodes in each cell in the sub-cell. Since there are and only one node in a cell can transmit in a time slot, time slots and a total of this step requires transmissions in the network. The probability that the computation fails in at least one sub-cell is . bounded by Next, each head node encodes the sum of the input bits and from its sub-cell into a codeword of length transmits it to the corresponding cell-center. This step requires a total of transmissions in the network and time slots and can be performed also with . probability of error at most The received values are aggregated so that at the end of the first phase, all cell-centers know the sum of their input

7678

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

N

Fig. 8. Each cell in the network (n; r) is divided into sub-cells. Every subcell has a “head”, denoted by a yellow node. The sum of input messages from each sub-cell is obtained at its head node, depicted by the solid (blue) lines. These partial sums are then aggregated at the cell-center. The latter step is represented by the dashed (magenta) lines.

bits in their cell with probability at least . The phase requires transmissions in the network and time slots to complete. Second Phase: In this phase, the partial sums stored at the cell-centers are aggregated along a tree (see for example, Fig. 6) so that the sink node can compute the sum of all the input bits in the network. We have the following two cases, depending on the connection radius . • : For this regime, our aggregation scheme is similar to the Intercell scheme in [10, Section III]. Each cell-center encodes its message into a codeword of length . Each leaf node in the aggregation tree sends its codeword to the parent node which decodes the message, sums it with its own message and then re-encodes it into a codeword of length . The process continues till the sink node receives the sum of all the input bits in the network. From Theorem II.3, this phase carries a probability of error at most . It requires transmissions in the network and time slots. : In this regime, the above simple ag• gregation scheme does not match the lower bound for the latency in Theorem IV.3. A more sophisticated aggregation scheme is presented in [11, Section V], which uses ideas from [16] to efficiently simulate a scheme for noiseless networks in noisy networks. The phase carries a probability of error at most . It requires transmissions in the network and time slots. Combining the two phases, the above scheme can compute any symmetric function with probability of error at most . Thus, for large enough, we have a -error scheme for computing any symmetric function in the network . It requires at most time slots and transmissions.

Fig. 9. Cell P is composed of fP g smaller cells from the previous level in the hierarchy. Each of the cell-centers c (denoted by the green nodes) holds the sum of the input bits in the corresponding cell P . These partial sums (denoted by the brown are aggregated along the minimum Steiner tree S (denoted by the blue node) can compute bold lines) so that the cell-center c the sum of all the input bits in P .

V. GENERAL NETWORK TOPOLOGIES In the previous sections, we focused on grid geometric networks for their suitable regularity properties and for ease of exposition. The extension to random geometric networks in the is immediate, and we continuum plane when focus here on extensions to more general topologies. First, we discuss extensions of our schemes for computing symmetric functions and then present a generalized lower bound on the number of transmissions required to compute symmetric functions in arbitrary connected networks. A. Computing Symmetric Functions in Noiseless Networks One of the key components for efficiently computing symmetric functions in noiseless networks in Theorem III.4 was the hierarchical scheme proposed for computing the arithmetic sum . The main idea function in the grid geometric network behind the scheme was to consider successively coarser partitions of the network and at any given level aggregate the partial sum of the input messages in each individual cell of the partition using results from the finer partition in the previous level of the hierarchy. Using this idea we extend the hierarchical scheme to any connected noiseless network and derive an upper bound on the number of transmissions required for the scheme. Let each node in the network start with an input bit and denote the set of nodes by . The scheme is defined by the following parameters: • The number of levels . • For each level , a partition of the set of nodes in the network into disjoint cells such that where , i.e., each each cell is composed of one or more cells from the next lower level in the hierarchy. See Fig. 9 for an illustration. and . Here, . Let • For each cell , a designated cell-center be the designated sink node . denote a Steiner tree with the min• For each cell , let imum number of edges which connects the corresponding cell-center with all the cell-centers of its component cells

KARAMCHANDANI et al.: TIME AND ENERGY COMPLEXITY OF FUNCTION COMPUTATION OVER NETWORKS

, i.e., the set of nodes

. Let

denote

the number of edges in . Using the above definitions, the hierarchical scheme from Theorem III.4 can now be easily extended to general network topologies. We start with the first level in the hierarchy and then proceed recursively. At any given level, we compute the partial sums of the input messages in each individual cell of the partition at the corresponding cell-center by aggregating the results from the previous level along the minimum Steiner tree. It is easy to verify that after the final level in the scheme, the sink possesses the arithmetic sum of all the input messages node in the network . The total number of transmissions made by the scheme is at most

Thus, we have a scheme for computing the arithmetic sum function in any arbitrary connected network. In the proof of Theorem III.4, the above bound is evaluated for the grid geometric netwith , , , work , and is shown to be . B. Computing Symmetric Functions in Noisy Networks We generalize the scheme in Theorem IV.6 for computing symmetric functions in a noisy grid geometric network to a more general class of network topologies and derive a corresponding upper bound on the number of transmissions required. The original scheme consists of two phases: an intracell phase where the network is partitioned into smaller cells, each of which is a clique, and partial sums are computed in each individual cell; and an intercell phase where the partial sums in cells are aggregated to compute the arithmetic sum of all input messages at the sink node. We extend the above idea to more , consider the following general topologies. First, for any definition: : a network of nodes is Clique-cover property if the set of nodes said to satisfy the clique-cover property is covered by at most cliques, each of size at most . with For example, a grid geometric network satisfies for . On the other hand, a tree network satisfies only for . . By Note that any connected network satisfies property regarding each disjoint clique in the network as a cell, we can easily extend the analysis in Theorem IV.6 to get the following result, whose proof is omitted. , , and be any Theorem V.1: Let connected network of nodes with . For , if satisfies , then there exists a -error scheme for comwhich requires at most puting any symmetric function over transmissions. C. A Generalized Lower Bound for Symmetric Functions The proof techniques that we use to obtain lower bounds are also applicable to more general network topologies. Recall that

7679

denotes the set of neighbors for any node . For any network, define the average degree as

A slight modification to the proof of Theorem IV.3 leads to the following result: and let . There exists Theorem V.2: Let a symmetric target function such that any -error scheme for computing over any connected network of nodes with average degree , requires at least transmissions. Proof: Let be the parity function. The only difficulty in adapting the proof of Theorem IV.3 arises from the node degree not being necessarily the same for all the nodes. We circumvent this problem as follows: in addition to decomposing the network into the set of source nodes and auxiliary nodes , such that for , as in the proof of Lemma IV.5 (see Appendix II-B); we also let every source node with degree more be an auxiliary node. There can be at most of such than nodes in the network since the average degree is . Thus, we decomposition of the network such obtain an . The rest of the that each source node has degree at most proof then follows in the same way. As an application of the above result, we have the following lower bound for ring or tree networks. , Corollary V.3: Let be the parity function, let . Any -error scheme for computing over and let any ring or tree network of nodes requires at least transmissions. The above result answers an open question, posed originally by El Gamal [6]. VI. CONCLUSION We conclude with some observations and directions for future work. A. Target Functions We considered all symmetric functions as a single class and presented a worst-case characterization (up to a constant) of the number of transmissions and time slots required for computing this class of functions. A natural question to ask is whether it is possible to obtain better performance if one restricts to a particular sub-class of symmetric functions. For example, two subclasses of symmetric functions are considered in [3]: type-sensitive and type-threshold. Since the parity function is a type-sensitive function, the characterization for noiseless networks in Theorems III.3 and III.4, as well as noisy broadcast networks in Theorems IV.3 and IV.6 also holds for the restricted sub-class of type-sensitive functions. A similar general characterization is not possible for type-threshold functions since the trivial funcfor all ) is also in this class and it requires tion ( no transmissions and time slots to compute. The following result, whose proof follows similar lines as the results in previous

7680

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

sections and is omitted, characterizes the number of transmissions and the number of time slots required for computing the maximum function, which is an example type-threshold function. This can be compared with the corresponding results for the whole class of symmetric functions in Theorems IV.3 and IV.6. Theorem VI.1: Let be the maximum function. Let , , and . Any -error scheme for computing over an -noise network requires at least time transmissions. Furslots and which ther, there exists a -error scheme for computing time slots and requires at most transmissions. B. On the Role of and Throughout the paper, the channel error parameter and the threshold on the probability of error are given constants. It is also interesting to study how the cost of computation depends on these parameters. The careful reader might have noticed that our proposed schemes also work when only an upper bound on the channel error parameter is considered, and always achieve a probability of error that is either zero or tends to zero as . It is also clear that the cost of computation should decrease with smaller values of and increase with smaller values of . Indeed, from (1) in the proof of Theorem IV.3, we see that the lower bound on the number of transmissions required for . On computing the parity function depends on as the other hand, from the proof of Theorem IV.6 the upper bound on the number of transmissions required to compute any sym. The two metric function depends on as expressions are close for small values of . C. Network Models We assumed that each node in the network has a single bit value. Our results can be immediately adapted to obtain upper bounds on the latency and the number of transmissions required for the more general scenario where each node observes a block of input messages with each , . However, finding matching lower bounds seems to be more challenging. APPENDIX I COMPUTING THE ARITHMETIC SUM OVER Consider a noiseless network where each node has . We present a scheme an input message which can compute the arithmetic sum of the input messages time slots and using over the network in transmissions. We briefly present the main idea of the scheme before delving into details. Our scheme divides the network into small cells and computes the sum of the input messages in each individual cell at designated cell-centers. We then proceed recursively and in each iteration we double the size of the cells into which the network is partitioned and compute the partial sums by aggregating the computed values from the previous round. This process finally yields the arithmetic sum of all the input messages in the network.

Fig. 10. A is a square cell of size m with regards to A .

2m. This figure illustrates some notation

Before we describe the scheme, we define some notation. square cell in the network, see Fig. 10. DeConsider an and the node in the lower-left corner of note this cell by by . For any which is a power of 2, , can be divided into 4 smaller cells, each of size . Denote these cells by

.

Without loss of generality, let be a power of 4. The scheme has the following steps: . 1) Let 2) Consider the partition of the network into cells each of size

, see Fig. 11.

consists of exactly four cells Note that each cell , see Fig. 12. Each corner node , , 2, 3, 4 possesses the sum of the input messages . corresponding to the nodes in the cell The partial sums stored at aggregated at the node

,

, 2, 3, 4 are

, along the tree shown in

Fig. 12. Each node in the tree makes at most transmissions. has the At the end of this step, each corner node sum of the input messages corresponding to the nodes in the cell . By pipelining the transmissions along the tree, this step takes at most

The total number of transmissions in the network for this step is at most

3) Let . If , return to step 2, else terminate. Note that at the end of the process, the sink node can compute the sum of the input messages for any input . The total number of steps in the

KARAMCHANDANI et al.: TIME AND ENERGY COMPLEXITY OF FUNCTION COMPUTATION OVER NETWORKS

7681

APPENDIX II COMPLETION OF THE PROOF OF THEOREM IV.3 A) Proof of Lemma IV.4: For every , let be the noisy copy of that the auxiliary node receives. is to comDenote the received vector by . The objective of pute the parity of the input bits . Thus, the target function is defined as

Since the input

is uniformly distributed over , we have . In the following, we first show that Maximum Likelihood estimation is equivalent to i.e., as using the parity of the received bits , and then compute the corresponding proban estimate for ability of error. From the definition of Maximum Likelihood estimation, we have

Fig. 11. This figure illustrates the partition of the network cells

, each of size 2

A

22

N (n 1) into smaller ;

if otherwise.

.

Next

where and follows since is uniformly distributed over and from the independence of the channels between the sources and the auxiliary node. Similarly

Fig. 12. Step 2 of the scheme for computing the sum of input messages. The 2 . For any such network is divided into smaller cells, each of size 2 cell A ,j 1; 2; 3; 4 , each corner node u A has the sum of the

2f

2

g

input messages corresponding to the nodes in the cell A input messages corresponding to the cell A along the tree shown in the figure.

scheme is takes is at most

. Then the sum of the

is aggregated at u

A

Putting things together, we have

,

. The number of time slots that the scheme

The total number of transmissions made by the scheme is at most

(2) is the number of components in with value 1. The where can be verified by noting that the product in above equality produces a sum of monomials and that the sign of each

7682

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

monomial is positive if the number of terms of the monomial is even, and negative otherwise. From conditioned on (2), we now have if if

.

Thus, we have shown that Maximum Likelihood estimation is as an estimate for . The correequivalent to using sponding probability of error is given by

for comLemma II.1: If there is a -error scheme with puting the parity function (on input bits) in transmissions, then there is an -decomposition of and a -error, -bounded scheme for bits) in this decomposed computing the parity function (on network. transProof: If all nodes in the network make missions, then the lemma follows trivially. Otherwise, we and decompose the network into the set of input nodes auxiliary nodes as follows. Consider the set of nodes which transmissions each during the execumake more than tion of the scheme . Since requires transmissions, there of such nodes. We let these nodes can be at most be auxiliary nodes and let their input be 0. Thus, we have an -decomposition of the network . The bits) over scheme now reduces to computing the parity (on this decomposed network. By construction, each input node transmissions, and hence, the scheme is makes at most -bounded. The following lemma is stated without proof, as it follows immediately from [8, Section 6, page 1833], or [12, Lemma 5.1, page 1064].

Hence, for any scheme which computes the parity function in an -noisy star network, the probability of error is at least . B) Proof of Lemma IV.5: We borrow some notation from [8] and [12]. Consider the nodes in a network and mark a as auxiliary subset of them as input nodes and the rest nodes. Such a decomposition of the network is called an -decomposition. An input value to this network is an . Consider a scheme on such a network element of which computes a function of the input. The scheme is said -decomposition to be -bounded with respect to an if each node in makes at most transmissions. Recall from Section II that for any scheme in our model, the number of transmissions that any node makes is fixed a priori and does not depend on a particular execution of the scheme. Following [8] and [12] we define the semi-noisy network, in which whenever it is the turn of an input node to transmit, it sends its input bit whose independent -noisy copies are received by its neighbors, while the transmission made by auxiliary nodes are not subject to any noise. The proof now proceeds by combining three lemmas. Suppose there exists a -error scheme for computing the parity which requires transfunction in an -noise network missions. We first show in Lemma II.1 that this implies the existence of a suitable decomposition of the network and a -error, -bounded scheme for computing the parity function in this decomposed network. Lemma II.2 translates the scheme into a scheme for computing in a semi-noisy network and Lemma II.3 translates into a scheme for computing in a noisy star network, while ensuring that the probability of error does not increase at any intermediate step. The proof is completed using the fact that the probability of error for original scheme is at most . . We have the following lemma. Let

Lemma II.2: (FROM NOISY TO SEMI-NOISY) For and any -error, any function -bounded scheme for computing in an -decomposition of , there exists an -decomposed semi-noisy network of nodes neighbors and a such that each input node has at most -error, -bounded scheme for computing in the semi-noisy network. We now present the final lemma needed to complete the proof. Lemma II.3: (FROM SEMI-NOISY TO NOISY STAR) and any -error, For any function -bounded scheme for computing in an -decomposed semi-noisy network where each input node has at neighbors, there exists a -error scheme for most -noisy star network with noise parameter computing in an , with each input node transmitting its input bit only once. Proof: In a semi-noisy network, when it is the turn of an input node to transmit during the execution of , it transmits its input bit. Since the bits sent by the input nodes do not depend on bits that these nodes receive during the execution of the scheme, we can assume that the input nodes make their transmissions at the beginning of the scheme an appropriate number of times, and after that only the auxiliary nodes communicate without any -decomposed noise. Further, since any input node in the neighbors, at most auxiliary network has at most nodes receive independent -noisy copies of each such input is an -bounded scheme, each input node bit. Since makes at most transmissions, and hence, the auxiliary independent -noisy nodes receive a total of at most copies of each input bit. to construct a scheme for Next, we use the scheme computing in an -noisy star network of noise parameter with each input node transmitting its input bit only

KARAMCHANDANI et al.: TIME AND ENERGY COMPLEXITY OF FUNCTION COMPUTATION OVER NETWORKS

once. Lemma II.4 shows that upon receiving an -noisy in the noisy star copy for every input bit, the auxiliary node independent -noisy copies network can generate for each input bit. Then onwards, the auxiliary node can simonly the auxiliary ulate the scheme . This is true since for nodes operate after the initial transmissions by the input nodes, and their transmissions are not subject to any noise.

7683

We can now bound the probability of error for the scheme as follows:

Lemma II.4: [8, Lemma 36, page 1834] Let , , and . There is a randomized algorithm that takes as input a single bit and outputs a sequence of bits such that if the input is a -noisy copy of 0 (respectively of 1), then the output is a sequence of independent -noisy copies of 0 (respectively of 1). where

APPENDIX III SCHEME FOR COMPUTING PARTIAL SUMS AT CELL-CENTERS We describe an adaptation of the scheme in [10, Section III], which requires at most transmissions and time slots. The scheme in [10, Section III], is described for and while the same ideas work for , the parameters need to be chosen carefully so that the scheme can compute efficiently in the new regime. Recall that the network is partitioned into cells of size where . Consider any cell in the network and denote its cell-center by . The scheme has the following steps: takes turn to transmit its input bit , 1) Each node in times, where . Thus, every node in

receives

independent noisy copies of

the entire input. This step requires

transmis-

sions and time slots. 2) Each node in forms an estimate for the input bits of the by taking the majority of the noisy bits other nodes in that it received from each of them. It is easy to verify that the probability that a node has a wrong estimate for any , see for example given input bit to be at most Gallager’s book [15, page 125]. Each node then computes the arithmetic sum of all the decoded bits and thus has an estimate of the sum of all the input bits in the cell . 3) Each node in transmits its estimate to the cell-center using a codeword of length such that is a constant and the cell-center decodes the message with prob. The existence of such a ability of error at most code is guaranteed by Theorem II.3 and from the fact that , the size of the estimate in bits since . At the end of this step, has independent estimates for the sum of the input bits corresponding to the nodes in . The total number of transmissions for this step is at most and it requires at time slots. most takes the mode of these values to 4) The cell-center make the final estimate for the sum of the input bits in .

follows since ; and follows since . Thus, every cell-center can compute the with sum of the input bits corresponding to the nodes in probability of error at most . The total probability of error is then at most . The total number of transmissions , i.e., in the network for this scheme is at most and it takes at most , i.e., time slots. ACKNOWLEDGMENT The authors would like to thank the reviewers for their comments which have greatly improved the presentation of this paper. REFERENCES [1] E. Kushilevitz and N. Nisan, Communication Complexity. Cambridge, U.K.: Cambridge Univ. Press, 1997. [2] P. Gupta and P. R. Kumar, “The capacity of wireless networks,” IEEE Trans. Inf. Theory, vol. 46, no. 2, pp. 388–404, Mar. 2000. [3] A. Giridhar and P. R. Kumar, “Computing and communicating functions over sensor networks,” IEEE J. Sel. Areas Commun., vol. 23, no. 4, pp. 755–764, Apr. 2005. [4] S. Subramanian, P. Gupta, and S. Shakkottai, “Scaling bounds for function computation over large networks,” in Proc. IEEE Int. Symp. Information Theory (ISIT), 2007, pp. 136–140. [5] N. Khude, A. Kumar, and A. Karnik, “Time and energy complexity of distributed computation in wireless sensor networks,” in Proc. IEEE Int. Conf. Computer Communications (INFOCOM), 2005, pp. 2625–2637. [6] A. E. Gamal, “Reliable communication of highly distributed information,” in Open Problems in Communication and Computation, T. M. Cover and B. Gopinath, Eds. New York: Springer-Verlag, 1987, pp. 60–62. [7] R. G. Gallager, “Finding parity in a simple broadcast network,” IEEE Trans. Inf. Theory, vol. 34, no. 2, pp. 176–180, Mar. 1988. [8] N. Goyal, G. Kindler, and M. Saks, “Lower bounds for the noisy broadcast problem,” SIAM J. Comput., vol. 37, no. 6, pp. 1806–1841, Mar. 2008. [9] C. Li, H. Dai, and H. Li, “Finding the k largest metrics in a noisy broadcast network,” in Proc. Annu. Allerton Conf. Communication, Control, and Computing, 2008, pp. 1184–1190. [10] L. Ying, R. Srikant, and G. E. Dullerud, “Distributed symmetric function computation in noisy wireless sensor networks,” IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4826–4833, Dec. 2007. [11] Y. Kanoria and D. Manjunath, “On distributed computation in noisy random planar networks,” in Proc. IEEE Int. Symp. Information Theory (ISIT), 2007, pp. 626–630. [12] C. Dutta, Y. Kanoria, D. Manjunath, and J. Radhakrishnan, “A tight lower bound for parity in noisy communication networks,” in Proc. 19th Annu. ACM-SIAM Symp. Discrete Algorithms (SODA), 2008, pp. 1056–1065.

7684

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 12, DECEMBER 2011

[13] C. Li and H. Dai, “Towards efficient designs for in-network computing with noisy wireless channels,” in Proc. IEEE Int. Conf. Computer Communications (INFOCOM), 2010, pp. 1–8. [14] M. Franceschetti and R. Meester, Random Networks for Communication. Cambridge, U.K.: Cambridge Univ. Press, 2007. [15] R. G. Gallager, Information Theory and Reliable Communication. Hoboken, NJ: Wiley, 1968. [16] S. Rajagopalan and L. J. Schulman, “A coding theorem for distributed computation,” in Proc. Annu. ACM Symp. Theory of Computing (STOC), 1994, pp. 790–799.

Nikhil Karamchandani (S’05) received the B.Tech. degree in electrical engineering from the Indian Institute of Technology, Bombay, in 2005, the M.S. degree in electrical engineering from the University of California at San Diego, La Jolla, in 2007, and is currently pursuing the Ph.D. degree in the Department of Electrical and Computer Engineering, University of California at San Diego. His research interests are in communication theory and include network coding, information theory, and random networks. Mr. Karamchandani received the California Institute for Telecommunications and Information Technology (CalIT2) fellowship in 2005.

Rathinakumar Appuswamy (S’05) received the B.Tech. and M.Tech. degrees in electrical engineering from Anna University, Chennai, and the Indian Institute of Technology, Kanpur, respectively, in 2004. He received the M.A. in mathematics from the University of California at San Diego, La Jolla, in 2008. He is currently a doctoral student at the University of California at San Diego, where he is a member of the Information and Coding Laboratory as well as the Advanced Network Sciences group. His research interests include network coding, communication for computing, and network information theory.

Massimo Franceschetti (SM’98) is an associate professor in the Department of Electrical and Computer Engineering, University of California at San Diego (UCSD), La Jolla. He received the Laurea degree (magna cum laude) in Computer Engineering from the University of Naples in 1997, and the M.S. and Ph.D. degrees in Electrical Engineering from the California Institute of Technology in 1999 and 2003. Before joining UCSD, he was a post-doctoral scholar at University of California at Berkeley for two years. Prof. Franceschetti was awarded the C. H. Wilts Prize in 2003 for best doctoral thesis in Electrical Engineering at Caltech; the S. A. Schelkunoff award in 2005 for best paper in the IEEE TRANSACTIONS ON ANTENNAS AND PROPAGATION; an NSF CAREER award in 2006, an ONR Young Investigator award in 2007; and the IEEE Communications society best tutorial paper award in 2010. He has held visiting positions at the Vrije Universiteit Amsterdam in the Netherlands, the Ecole Polytechnique Federale de Lausanne in Switzerland, and the University of Trento in Italy. He is associate editor for communication networks of the IEEE TRANSACTIONS ON INFORMATION THEORY for 2009–2012 and has served as guest editor for two issues of the IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATION. His research interests are in communication systems theory and include random networks, wave propagation in random media, wireless communication, and control over networks.