Finding the Optimal Communication Structure for Networked Computing Apoorva Jindal
Mingyan Liu
University of Michigan
University of Michigan
Ann Arbor, MI 48109
Ann Arbor, MI 48109
Email:
[email protected]
Email:
[email protected]
Abstract This paper studies the problem of distributed computation over a wireless network of resource constrained nodes. This problem applies to many emerging wireless sensor networks, including those used for body sensing, structural health monitoring, and various cyber-physical systems. Computational tasks in these networks typically require far more sophisticated data processing beyond commonly-studied data aggregation or simple functions like average and max/min, in order to enable real-time diagnosis and control. To determine how to accomplish this in the most energy efficient manner, possibly subject to a delay constraint, we need to first be able to decompose a computational task into components, and then be able to distribute these components over the network taking into account the fact that data input to the computation may originate from different parts of the network. In this paper, we propose a general optimization framework to determine the best communication structure given a fixed decomposition of a computational task represented in a data-flow graph. We do this for two classes of computations, those involving single-stream inputs and those requiring multi-stream inputs. For each category, we prove that finding the optimal communication structure is NP-hard. We then propose centralized integer linear programs to determine the optimal communication structure, and develop polynomial, distributed approximation algorithms to find a nearoptimal communication structure. We analytically derive the approximation factor for each approximation algorithm we propose, and evaluate their performance in realistic engineering applications through simulation.
I. I NTRODUCTION This paper studies the problem of distributed computation over a wireless network of resource constrained nodes. This problem applies to many wireless sensor networks, including those used for body sensing, structural health monitoring, and various cyber-physical systems. Over the past decade there has been extensive studies on innetwork processing in a wireless sensor network, e.g., finding efficient routing strategies when data compression and aggregation are involved. However, computational tasks in many emerging applications, including those cited above, require far more sophisticated data processing beyond commonly-studied data aggregation or simple functions like average and max/min, in order to enable real-time diagnosis and control. This leads to the question of how to perform arbitrary (and likely complex) computational tasks using a distributed network of (resource – energy and processing capability – constrained) nodes, and whose inputs originate from this network of nodes, to achieve energy efficiency and at the same time satisfy certain performance requirements. To accomplish this, one must address two challenges. The first is to develop a general methodology to decompose a complex computational tasks into smaller operations each with its own input and output, and collectively related through a certain data-flow or dependency graph. The second challenge is to distribute or place these operations
among individual sensor nodes so as to incur minimal energy consumption and delay. We focus on the second challenge in this paper. We propose a general optimization framework to determine the best communication structure given a fixed decomposition of a computational task represented in a data-flow graph. We do this for two classes of computations, those involving single-stream inputs and those requiring multi-stream inputs (precisely defined in the next section). For each category, we prove that finding the optimal communication structure is NP-hard. We then propose centralized integer linear programs (ILPs) to determine the optimal communication structure; these yields optimal centralized, albeit exponential, algorithms. We then develop polynomial, distributed approximation algorithms to find near-optimal communication structures. We analytically derive the approximation factor for each approximation algorithm we propose and use simulations to understand their performance for two real engineering applications. The first is a speech detector application [12] and the second is a singular value decomposition (SVD) computation over data collected by a network of sensors. In our numerical examples, the proposed approximation algorithms are always within 3% of the optimal. Our simulations also demonstrate the advantage of in-network computation as it can reduce the number of bytes transmitted over the network by more than half. II. M ODEL
AND
A SSUMPTIONS
In this section, we describe the network model, our assumptions and the metrics used to evaluate the communication structure. A network of nodes is represented with an undirected, unweighted graph G(V, E). Each node in V acts as both a source and a relay. If two nodes can successfully exchange messages with each other, there exists an edge e ∈ E between them. Without loss of generality, we assume node 0 to be the base station. We also assume that sensors are identical in their processor and radio (and hence computational time and energy consumption per bit). This is done to keep the presentation simple and can be easily relaxed. Each node has a local input sensing stream. The objective of the network is to evaluate a multi-input function based on the sensing streams from all nodes and have the output of this function known at the base station. For example, the function in the speech detector application will be to identify if a person is speaking in the room. A function F is defined by a set of operators Oi , 1 ≤ i ≤ |O| and the dependancy relationship between these out bits. For operators. An operator Oi takes one or more input streams of Oin i bits each and outputs a stream of Oi notational convenience, sensing is also assumed to be an operator. In other words, for each sensing stream, the first operator is the sensing operator which generates the sensed data. Finally, let C(Oi ) be the computation time for operator Oi on a node. We assume that the data-flow graph is given, that is, the set of operators comprising the function and their dependency relationship are precisely known. A sensing cycle is defined to be the time duration in which each sensor performs the sensing task to generate a stream of size Oout 1 bits, the function is evaluated based on these data streams and its value is made known at the base station. The sensing rate depends inversely on the duration of one sensing cycle. For a given function, our objective is to determine the optimal communication structure to minimize the energy consumption in a sensing cycle under a constraint on the maximum duration of a sensing cycle. We classify the computation functions into the following two categories. Single-Stream Inputs: The input stream of every operator in the function comes from only one sensor. For example, computing the Fourier transform of a sensor stream requires the data stream of only one sensor. Thus, this category of functions run independently for each sensor stream.
Multiple-Stream Inputs: In functions belonging to this category, there exists at least one operator whose input stream will come from multiple sensors. For example, finding the singular value decomposition (SVD) of the power spectral density matrix of all the data streams requires the data stream of all the sensors. We are interested in the following two performance metrics. The energy consumption is defined to be the total communication energy consumed in the network in one sensing cycle. Let ET x and ERx denote the energy consumption to transmit and receive a bit of data. Then we assume that the energy consumed in transmitting a packet of B bits over one hop is B (ET x + ERx )1 . The computational delay is defined to be the maximum computational delay at a sensor node. As observed in [12], [18], the computational time is the chief contributor to delay as packet sizes in sensor systems tend to be very small. Thus, the duration of a sensing cycle depends chiefly on the maximum computational delay amongst all sensor nodes. In other words, the computational delay constraint imposes a constraint on the maximum duration of a sensing cycle. Our objective is for any given function specified through a collection of operators, to find the optimal communication structure which minimizes the energy consumption under a computational delay constraint at each node. This will be referred to as the delay-constrained energy minimization problem. Note that it is easy to modify our algorithms to minimize the maximum computational delay with a constraint on the energy consumption. Indeed, the dual of the linear programs we propose will optimally solve this alternative formulation. Due to space limit all proofs are omitted in this extended abstract; they will be included in the full version. III. S INGLE -S TREAM I NPUTS We first study functions in which all operators are computed independently for each sensor stream. An example is the speech detector application based on deriving the Mel Frequency Cepstral Coefficients (MFCC) studied by Newton et al. [12]. To determine the optimal communication structure, we have to decide which node should be evaluating which operator for each sensor’s sensed data stream. Thus, finding the optimal communication structure consists of the following steps: (i) decide which operators are to be evaluated on the sensor node, which operators should be evaluated on the base station and which operators should be evaluated on intermediate nodes, and (ii) decide which intermediate nodes should the operators be evaluated on. A decision version of the Delay Constrained Energy Minimization problem for the single-stream input functions is NP-hard as the multiple knapsack problem is a special case of the proposed problem (when |O| = 1). Theorem 1: There is no polynomial time algorithm that solves the Delay Constrained Energy Minimization problem for single-stream input functions, unless P = NP. We next propose an ILP to find the optimal solution. For ease of presentation, we only consider functions which treat each sensor to be equivalent, and hence, the operators evaluated on each sensor stream are the same. It is trivial to apply the proposed algorithms to functions which treat each sensor different without any loss in optimality for the centralized ILPs and without any change in the approximation factor for the proposed approximation algorithms. For notational convenience, we add one extra operator per function - O|O|+1 . O|O|+1 is always evaluated at the out out base station, has Oin |O|+1 = O|O| , O|O|+1 = 0 and C(O|O| ) = 0. We also define the following additional variable. Hi→ j , i, j ∈ V denotes the hop count along the shortest path from node i to j. By definition, Hi→i = 0. The variables to be optimized are denoted xOjki ,n , j, k, n ∈ V, 2 ≤ i ≤ |O| + 1; it is set to 1 only if the operator Oi , evaluated on the data stream of sensor node n, is evaluated at node k ∈ V and the preceding operator Oi−1 for 1 Our algorithms and the corresponding approximation factors do not depend on the exact model used for energy consumption provided the energy consumed remains a function of the number of bits transmitted over a network. Thus, nothing changes if a more complex model for energy consumption which incorporates energy consumed in overhearing is used.
the same sensor stream is evaluated at node j ∈ V . Note that O1 is the sensing operator, hence, it will always be evaluated at the node generating the data stream. To model this constraint, we introduce a variable xOjk1 ,n , j, k, n ∈ V which is always set to 0 except when k = j = n. The following ILP finds the optimal solution to the Delay Constrained Energy Minimization problem for functions belonging to the first category. !
∑
min
∑
n∈V 2≤i≤|O|+1
∑
s.t.
∑
xOjki ,n Oin i H j→k (ET x + ERx )
xOjki ,n = 1, ∀n ∈ V, 1 ≤ i ≤ |O| + 1
j,k∈V O1 ,n xnn = 1, ∀n ∈ V O|O|+1 ,n = 1, ∀n ∈ V x j0 j∈v O ,n xOjki ,n = xk ji+1 , ∀n, k ∈ V, 1 ≤ i ≤ |O| j∈V j∈V xOjki ,nC(Oi ) ≤ C, ∀k ∈ V \{0} j∈V n∈V 1≤i≤|O| xOjki ,n ∈ {0, 1}, ∀n, j, k ∈ V, 1 ≤ i ≤ |O| + 1.
∑
∑
∑
∑∑ ∑
(1)
j,k∈V
(2) (3) (4) (5) (6) (7)
in If xOjki ,n = 1, it implies that Oin i bits need to be sent from node j to node k which consumes Oi H j→k (ET x + ERx ) units of energy. Our objective is to minimize the total energy consumed which yields (1). (2) ensures that each operator is evaluated on one sensor node. (3) ensures that O1 is always evaluated at the node generating the data stream. (4) ensures that O|O|+1 is always evaluated at the base station, thus it ensures that the output of O|O| is sent to the base station. (5) ensures that if operator Oi on the data stream of sensor node n is evaluated at node k, O ,n then ∑ j∈V xk ji+1 is equal to 1, else it is equal to 0. Finally, (6) enforces the computational delay constraint at each sensor node except the base station. Below we present a simple, distributed polynomial time approximation algorithm with a constant approximation factor. We define two new variables. Based on the delay constraint, assume that only operators O1 , . . . Od , 1 ≤ d ≤ |O|, out can be evaluated on a sensor node, that is ∑di=1 C(Oi ) ≤ C and ∑d+1 i=1 C(Oi ) > C. Finally, let dmin = argmini {Oi | 1 ≤ i ≤ d}. In other words, Odmin is the operator whose output stream has the smallest size amongst the operators O1 , . . . Od . Consider the following algorithm. Evaluate the operators O1 , . . . Odmin on the sensor node itself, send the output stream along the shortest path to the base station, and then evaluate operators Odmin+1 , . . . O|O| on the base station. This is a simple, distributed polynomial time algorithm and as Theorem 2 states, yields a constant approximation ratio. Theorem 2: Evaluating O1 , . . . Odmin on the sensor node itself, and operators Odmin+1 , . . . O|O| on the base station Oout
yields an approximation ratio of Odmin out without violating the computational delay constraint at any sensor node. min We evaluate the performance of the proposed approximation algorithm for the speech detector application based on computing the MFCC’s using simulations, and compare it to the performance of the optimal communication structure. We run a number of simulations for topologies generated by randomly distributing nodes in an area of 50 × 50m2 and assuming the transmission range to be 30m, for values of V and C lying between 10 and 30 and 50 and 450 ms respectively. We observe that the approximation algorithm always yields optimal performance for these randomly generated topologies.
IV. M ULTIPLE -S TREAM I NPUTS Functions belonging to this category have at least one operator whose input stream comes from multiple sensors. Such operators can be of two types. The first type of operator does not require any message exchange between two independent computations. Most simple in-network computations like max, min, average etc. belong to this category. Finding the optimal communication structure for this type of operators has been studied in great detail in the literature [1], [4], [14], [16], [17]. Hence, to keep the exposition interesting, we do not study these type of operators here. Instead, we will focus on the second type of operators which require message exchange between two independent computations. We give an example of such an operator to illustrate the associated constraints. Consider finding the SVD of the power spectral density (PSD) matrix of the data streams from all sensors to determine the corresponding eigenvectors. Zimmerman et al. [18] proposed a novel method of decomposing the computation of SVD using in-network computation (graphically represented in Figure 1). Each sensor node is assumed to be aware of the eigenvalues of the PSD matrix and the FFT of its own sensed data Fig. 1. Decomposing the computation of SVD using in-network stream. Now, if a sensor has the FFT of N ⊂ V, |N| > computation. 1 sensors, then it can compute the SVD on the PSD matrix using |N| sets of FFT results and determine |N| eigenvectors. Let another sensor node be in possession of the FFT of N ′ ⊂ V, |N ′ | > 1 sensors. It can do a similar computation to determine |N ′ | eigenvectors. To be able to combine results from these two computations to construct the |N ∪ N ′ | eigenvectors, one needs to be able to determine the appropriate scaling factors. We call two computations combinable if one can determine the appropriate scaling factors to combine them. A computation on N nodes and another computation on N ′ nodes is combinable if and only if either N ∩ N ′ 6= φ (that is, there is at least one common sensor in N and N ′ ), or there exists another computation on N ′′ nodes which is combinable with both N and N ′ . Each independent in-network computation requires message exchange with other in-network computations to make them combinable. While it’s possible to formulate a similar ILP to that presented in the previous section for a generic multistream input problem, below we will focus exclusively on the SVD operator because it illustrates more clearly the hardness of the problem, and its ubiquitousness across several signal processing algorithms implemented in sensor networks [2], [5], [8], [15], [18] makes it an ideal candidate for such a study. Note that the algorithms developed can be easily applied to any operator which requires message exchange between different computations as discussed at the end of this section. Determining the optimal communication structure implies finding the set S of sensor nodes on which the SVD computation will take place, and for each s ∈ S, finding the corresponding set of sensors Ns whose FFT will be made available at s. The computational delay constraint imposes a constraint on the maximum number of FFT’s which can be combined at a sensor node. Assuming C(|Ns |) to be the computational time of evaluating the SVD of the FFT from |Ns | sensors, C(|Ns |) ≤ C, ∀s ∈ S. This constraint is equivalent to |Ns | ≤ d, ∀s ∈ S where d := max {|Ns | | C(|Ns |) ≤ C}.
Our objective is to find the set S and their corresponding Ns , ∀s ∈ S, and the routing structure to minimize the total energy consumed such that |Ns | ≤ d, ∀s ∈ S and the computations on all pairs s1 , s2 ∈ S are combinable. This will be referred to as the In-Network with Exchange problem. A decision version of this problem can be shown to be NP-hard through a reduction from set cover. Theorem 3: There is no polynomial time algorithm that solves the In-Network with Exchange problem, unless P = NP. We next propose an ILP to solve the In-Network with Exchange problem. xi j is set to 1 if the FFT of sensor node i is sent to node j (i.e. i ∈ N j ), otherwise it will be set to 0. xii is set to 1 only if i ∈ S. Thus, xi j is the variable which defines both the set S as well as Ns . pi jk is a variable which will be set to 1 if sensor node k is evaluated at 1 if n = 0 and ∑k∈V pi jk ≥ 1 both nodes i and j. Finally, we define ci jn as ci jn = 1 if 0 < n < |V | and ∑k∈V cik(n−1) .c jk(n−1) + ci j(n−1) ≥ 1. 0 otherwise Thus, ci j(|V |−1) will be equal to 1 if pairs i, j ∈ S are combinable. Below is the ILP to solve the In-Network with Exchange problem. (8) min ∑ xi j (ET x + ERx ) RHi→ j + rH j→0 i∈V, j∈V
s.t.
x ji ≤ xii ≤ j∈V, j6=i V
∑
∑
x ji , ∀i ∈ V
(9)
j∈V, j6=i
∑ xi j ≥ 1, ∀i ∈ V
(10)
j∈V
xki + xk j , ∀i, j, k ∈ V 2 ci j0 ≤ ∑ pi jk , ∀i, j ∈ V pi jk ≤
(11) (12)
k∈V
xii + x j j , ∀i, j ∈ V 2 cik(n−1) + c jk(n−1) ti jkn ≤ , ∀i, j, k ∈ V, 0 < n < |V | 2 ci jn ≤ ci j(n−1) + ∑ ti jk(n−1) , ∀i, j ∈ V, 0 < n < |V | ci j(|V |−1)) ≥
(13) (14) (15)
k∈V
ciin = 0, ∀i ∈ V, 0 ≤ n < |V |
(16)
∑ xi j ≤ d, ∀ j ∈ V
(17)
xi j , ci jk , pi jk ,ti jkn ∈ {0, 1}∀i, j, k ∈ V, 0 ≤ n < |V |
(18)
i∈V
If xi j = 1, the FFT of sensor node i is sent to node j which consumes RHi→ j (ET x + ERx ) units of energy, then node j evaluates the SVD and sends the eigenvector to the base station for putting all the eigenvectors together, which consumes an additional rH j→0 (ET x + ERx ) units of energy. Our objective is to minimize the total energy consumed which yields (8). (9) sets the value of xii to 1 if Ni 6= φ , else it is set to 0. (10) ensures that the FFT of every sensor node is sent to at least one node. (11)-(16) ensure combinability, and (17) imposes the computational delay constraint at each sensor node. We next propose a Θ (log (|V |)) approximation algorithm for the In-Network with Exchange problem. We define a data collection tree for G(V, E) as a spanning tree such that the path from each node v ∈ V to the base station has the minimum weight. Consider a data collection tree with the following property. All the children of a non-leaf node v ∈ V in the
tree cannot be moved to other nodes which have a height less than the height of node v. Thus, this tree has the minimum number of non-leaf nodes. Label this tree TM . Theorem 4: The following solution in the In-Network with Exchange problem is optimal if the computational delay constraint is removed. S consists of all non-leaf nodes in the data collection tree TM , Ns , s ∈ S consists of all the immediate children of s and the data collection tree TM is the routing structure. The solution of Theorem 4 will violate the computation delay constraint if the number of immediate children a node has is greater than d − 1. (Note that a node in S will include the FFT of its own data stream in its computation, hence, having more than d − 1 immediate children will violate the computational delay constraint.) A data collection tree with the additional constraint that no sensor node in the tree has more than d − 1 immediate children will satisfy the computational delay constraint, but may no longer be optimal even if it has the fewest non-leaf nodes. We define a degree-constrained data collection tree for G(V, E) to be such that no sensor node has more than d − 1 immediate children. Constructing such a tree is also NP-hard. It is APX-hard even when weights on edges satisfy the triangle inequality [3]. We propose a modified version of Dijkstra’s shortest path algorithm in Figure 2 to construct a degree-constrained data collection tree. This approximation algorithm can be easily distributed in a manner similar to any shortest path routing algorithm [7]. The tree is build top down from the root with each node choosing its d − 1 children arbitrarily. Hence, like any shortest path algorithm, it can be built by message exchanges only between neighboring nodes. Theorem 5: Under the following assumptions: (i) d ≥ 3, (ii) the height of the unconstrained data collection tree (used to construct the solution in Theorem 4) is O (log(|V |)), and (iii) nodes can transmit to each other only if the distance between them is less than the transmission range, the approximation factor of the algorithm of Figure 2 is Θ (log(|V |)). NV = {0}, hv = ∞, ∀v ∈ V \{0}, h0 = 0, Cv = 0, ∀v ∈ V . (Cv denotes the number of children of node v.) while (NV ! = V ) do For each edge e ∈ E such that e connects nodes v ∈ NV and v′ ∈ V \NV and Cv < d − 1 h′v = min (h′v , hv + 1) vmin = argminv {hv | ∀v ∈ V \NV } Add vmin to NV . Let the parent of vmin be v parent . Update Cv parent = Cv parent + 1 Set hv = ∞, ∀v ∈ V \NV Fig. 2.
Modified Dijkstra’s approximation algorithm for the Degree-Constrained Data Collection Tree problem.
We evaluate the performance of the proposed approximation algorithm for the SVD operator using simulations, and compare it to the performance of the optimal communication structure. We run a number of simulations for topologies generated by randomly distributing nodes in an area of 50 ×50m2 and assuming the transmission range to be 30m, for values of V and d lying between 4 and 200 and 3 and 8 respectively. We observe that the approximation algorithm is always within 3% of the optimal. We also observe that in-network computation reduced the number of bytes transmitted over the network by more than half, which demonstrates the advantage of in-network computation. Algorithms designed for the operator SVD can be directly applied to any multiple-stream input function which requires message exchange between pairs or amongst a constant number of independent computations. Even the approximation factor of the approximation algorithm will not change for these functions. Note that the extra requirement that message exchange is allowed only between a constant number of independent computations is not restrictive, rather it includes all multiple-stream functions we are aware of, including simulated annealing, gradient descent etc.
V. R ELATED W ORK Prior works which study deriving the communication structure with in-network computation/operator placement in a sensing system either consider only single-stream functions or multiple-stream functions which require no message exchange between independent computations and include only a cost function (similar to the energy cost used in this paper) as an objective and do not place a computational delay constraint (for example, see [1], [9]–[11], [13], [16] and references therein), and hence do not represent the complex computational requirements demanded by practical engineering applications, or study scaling laws which do not yield algorithms to determine the optimal communication structure [6]. Prior works which study algorithms to derive the communication structure with data compression/compressed sensing [4], [14], [17] are directly relevant to the study multiple-stream input functions which require no message exchange between independent computations as data compression can be viewed as a computation of this type. To the best of our knowledge, this is the first work to study multiple-input stream functions which require message exchange between independent computations as well as operator placement with a computational delay constraint. R EFERENCES [1] Z. Abrams and J. Liu. Greedy is good: On service tree placement for in-network stream processing. In Proc. of IEEE ICDCS, 2006. [2] L. Balzano and R. Nowak. Blind Calibration of Sensor Networks. In Proc. of IPSN, 2007. [3] B. Brinkman and M. Helmick. Degree-constrained minimum latency trees are apx-hard. Technical Report 2008-03-25, Miami University, 2008. [4] R. Cristescu, B. Beferull-Lozano, M. Vetterli, and R. Wattenhofer. Network correlated data gathering with explicit communication: NP-completeness and algorithms. IEEE/ACM Transactions on Networking, 14(1):41–54, 2006. [5] G. Derveaux, G. Papanicolaou, and C. Tsogka. Time reversal imaging for sensor networks with optimal compensation in time. The Journal of the Acoustical Society of America, 121(4):2071–2085, 2007. [6] A. Giridhar and P. R. Kumar. Towards a Theory of In-Network Computation in Wireless Sensor Networks. IEEE Communications Magazine, 44(4):98–107, 2006. [7] O. Gnawali, R. Fonseca, K. Jamieson, D. Moss, and P. Levis. Collection Tree Protocol. In Proc. of ACM SenSys, 2009. [8] J. Gupchup, R. Burns, A. Terzis, and A. Szalay. Model-Based Event Detection in Wireless Sensor Networks. In Proc. of Workshop on Data Sharing and Interoperability on the World-Wide Sensor Web, 2007. [9] S. Kashyap, S. Deb, K. Naidu, and R. Rastogi. Efficient gossip-based aggregate computation. In Proc. of ACM PODS, 2006. [10] X. Li, Y. Wang, and Y. Wang. Complexity of convergecast and data selection for wireless sensor networks. Technical report, Illinois Institute of Technology, 2009. [11] S. Madden, M. Franklin, J. Hellerstein, and W. Hong. TAG: a Tiny AGgregation service for ad-hoc sensor networks. In Proc. of OSDI, 2002. [12] R. Newton, S. Toledo, L. Girod, H. Balakrishnan, and S. Madden. Wishbone: Profile-based Partitioning for Sensornet Applications. In Proc. of NSDI, 2009. [13] B. Patt-Shamir. A note on efficient aggregate queries in sensor networks. In Proc. of ACM PODC, 2004. [14] S. Pattem, B. Krishnmachari, and R. Govindan. The Impact of Spatial Correlation on Routing with Compression in Wireless Sensor Networks. In Proc. of IPSN, 2004. [15] V. Raykar, I. Kozintsev, and R. Lienhart. Position Calibration of Microphones and Loudspeakers in Distributed Computing Platforms. IEEE Transactions on Speech and Audio Processing, 13(1), 2005. [16] U. Srivastava, K. Munagala, and J. Widom. Operator placement for in-network stream query processing. In Proc. of ACM PODS, 2005. [17] Y. Zhu, K. Sundaresan, and R. Sivakumar. Practical Limits on Achievable Energy Improvements and Useable Delay Tolerance in Correlation Aware Data Gathering in Wireless Sensor Networks. In Proc. of IEEE SECON, 2005. [18] A. Zimmerman, M.Shiraishi, R. Swartz, and J. Lynch. Automated Modal Parameter Estimation by Parallel Processing within Wireless Monitoring Systems. Journal of Infrastructure Systems, 14(1):102–113, 2008.
