Clustered Decentralized Binary Detection: an Information-Theoretic Approach Gianluigi Ferrari, Marco Martal`o and Roberto Pagliari Abstract— This paper presents an information-theoretic approach to decentralized binary detection in sensor networks. In particular, we consider a Bayesian approach for the minimization of the probability of decision error. Two scenarios are considered: (i) a scenario where clusters are identical (uniform clustering) and (ii) a scenario where clusters are different (non-uniform clustering). The performance analysis obtained with a classical “communication-theoretic” approach is extended to the “information-theoretic” realm using the concept of mutual information. We then propose a simplified binary symmetric channel (BSC) model to analyze the clustered schemes, and we show that it allows to accurately predict their realistic performance. Our results show that uniform clustering leads to negligible performance degradation beyond the first clustering subdivision. Moreover, we show that for a given value of the system mutual information, the probability of decision error is uniquely determined. The results predicted by the analytical framework are confirmed by simulations.
I. I NTRODUCTION Distributed detection has been an active research field for a long time [1]. The increasing interest for sensor networks has spurred a significant activity on distributed detection techniques [2]. Classical approaches (see [3]) are not immediately applicable to a real system because they are based on the assumption of ideal communication links between the sensors and the access point (AP). However, in a wireless communication scenario links are likely to be faded. Furthermore, false alarm and correct decision probabilities are usually used as design parameters. In [4], the authors follow a Bayesian approach and study optimal fusion rules in a scenario with Gaussian noise. In [5], the presence of noisy communication links, modeled as binary symmetric channels (BSCs), is considered and a few techniques are proposed in order to make the system more robust against the noise. Information-theoretic approaches have also been proposed to analyze a sensor network. In [6], the authors propose a framework to characterize the system in terms of its entropy and false alarm and missed detection probabilities. In [7], the problem of optimally placing sensors in a network is reformulated as a mutual information minimization problem. In this paper, we consider a scenario where nodes are clustered and there are local fusion centers (FCs) associated with the clusters. Each of the local FCs makes a local decision based on the data collected from the corresponding cluster G. Ferrari, M. Martal`o and R. Pagliari are with the Department of Information Engineering, University of Parma, I-43100 Parma, Italy E-mail:
[email protected], {martalo,pagliari}@tlc.unipr.it. This work was supported in part by Ministero dell’Istruzione, Universit`a e Ricerca (MIUR), Italy, under the PRIN project “CRIMSON” (Cooperative Remote Interconnected Measurement Systems Over Networks).
and then transmits its decision to the final AP. Both uniform and non-uniform clustering configurations are considered. We first propose a communication-theoretic approach, which generalizes that proposed in [5] to clustered scenarios, for the evaluation of the probability of decision error. Then, a novel information-theoretic approach is proposed, based on the characterization of a sensor network with decentralized detection through its mutual information. The key findings of this work are (i) the fact that clustering, as long as uniform, does not entail performance degradation after the first nodes’ subdivision and (ii) the fact that the probability of decision error depends directly on the mutual information, i.e., for a given value of the mutual information, the probability of decision error is fixed. II. P RELIMINARIES
ON
D ECENTRALIZED D ETECTION
We consider a network scenario where N sensors observe a common phenomenon. They are clustered into nc < N groups, and each of them can communicate with only one local AP, referred to as a local FC. The FCs collect decisions from the sensors in the corresponding clusters and make local decisions about the binary phenomenon. Then, each FC sends its local decision to the AP, which makes the final decision. This architecture is shown in Fig. 1. Each sensor observes a common binary phenomenon whose status is defined as H: H0 with probability p0 H= H1 with probability 1 − p0 where p0 , P(H = H0 ). The observed signal at the i-th sensor can be expressed as ri = cE + ni where cE ,
i = 1, . . . , N
0 if H = H0 s if H = H1 .
Assuming that the noise samples {ni } are independent with the same Gaussian distribution N (0, σ 2 ), the common signal-to-noise ratio (SNR) at the sensors can be defined as follows: [E{cE|H1 } − E{cE|H0 }]2 s2 = 2. 2 σ σ Each sensor makes a decision comparing the observation ri with a threshold value τ and computes a local decision ui as 0 if ri < τ ui = γ (ri ) = 1 if ri > τ . SNRsensor =
b H
CLUSTER 1
AP
b1 H
FC1 (1) s1 (1) n1
AP
...
CLUSTER nc
bnc H (1)
sdc
1
(1)
...
(n )
s1 c
...
(n) n1
n dc
1
FC
FC
FCnc FC
(n )
sdcc
FC
nc
(n )
n dc c
nc
cE BINARY PHENOMENON H
Fig. 2.
An uniformly clustered sensor network with N = 16 sensors.
Fig. 1. Block diagram of a clustered sensor network with decentralized detection.
derived in [5] if nc = kf = 1 and dc = N, i.e., there is no clustering.
The relation between τ and s is well known in a standard scenario [4]. However, it needs to be optimized taking into account the presence of clusters. In the following, this optimization is carried out for all considered scenarios by minimizing the probability of decision error.
B. Non-Uniform Clustering
A. Uniform Clustering In a scenario with uniform clustering, the sensors are equally divided into the clusters. An example, with N = 16 nodes, is shown in Fig. 2: there are 4 clusters with 4 sensors each. In general, the j-th FC, j = 1, . . . , nc , computes a local decision using the following majority-like rule [4]: ( ( j) c 0 if ∑dm=1 um < k ( j) ( j) bj = Γ u , . . . , u H = (1) 1 ( j) dc c 1 if ∑dm=1 um ≥ k where the fusion threshold k is the same for all clusters since they have the same dimension. The AP decides with a b j }: majority-like rule over the local FC decisions {H ( c b j < kf H0 if ∑nj=1 H b=Ψ H b1 , . . . , H bnc = H (2) nc b H1 if ∑ j=1 H j ≥ kf
where kf is the AP threshold. Using (2) and (1), the probability of decision error can be expressed as follows: b = H1 |H0 )P(H0 ) + P(H b = H0 |H1 )P(H1 ) = P(H ( )i nc dc dc nc j (dc − j) (τ ) = p0 ∑ ∑ j [1 − Φ(τ )] Φ i i=kf j=k )nc −i ( k−1 dc j (d− j) ∑ j [1 − Φ(τ )] Φ (τ ) j=0 )i ( kf −1 dc dc nc j (dc − j) (τ − s) +p1 ∑ ∑ j [1 − Φ(τ − s)] Φ i i=0 j=k ( )nc −i k−1 dc j (dc − j) (τ − s) (3) ∑ j [1 − Φ(τ − s)] Φ j=0
Pe
R
x √1 where Φ(x) , −∞ exp(−y2 /2) dy. It is possible to show 2π that the probability of decision error (3) reduces to that
We define the cluster size vector D as D = {dc1 , dc2 , . . . , dcnc } where dci is the number of sensors in the i-th cluster and nc dci = N. Furthermore, we define two probability vectors ∑i=1 P 1|1 and P 1|0 as follows: P 1|1 P 1|0 1|1
1|1
1|1
1|1
1|0
1|0
1|0
, {p1 , p2 , . . . , pnc }
, {p1 , p2 , . . . , pnc }
1|0
where pl (pl ) is the probability that FCl decides for H1 when H1 (H0 ) has happened. We want to compute the probability of decision error in a sensor network with a generic topology defined by D, P 1|0 and P 1|1 . Using a combinatorial approach similar to that followed to derive (3) (more precisely, a generalization of the repeated trials formula), one obtains b = H1 |H0 ) P(H b = H0 |H1 ) P(H
nc
=
(nic )
nc
∑ ∑ ∏{string(i, j, l)pl
1|0
i=kf j=1 l=1
l|0
+ (1 − string(i, j, l))(1 − pl )} nc kf −1 ( i ) nc 1|1 = ∑ ∑ ∏{string(i, j, l)pl
(4)
i=0 j=1 l=1
1|1
+ (1 − string(i, j, l))(1 − pl )} (5)
where string(i, j, l) can be either 0 or 1 (i denotes the number of clusters which decide for H1 and j runs over all possible cluster configurations with i clusters in favor of H1 ). In particular, the first value is assumed when FCl decides for H0 , while the second one when FCl decides for H1 . III. M UTUAL I NFORMATION IN S ENSOR N ETWORKS WITH D ECENTRALIZED D ETECTION From an information-theoretic perspective, the network in Fig. 1 can be interpreted as a “probabilistic law” which asb with H. Therefore, one can represent the network sociates H as a binary channel and analyze its mutual information.
0 H 1
p00
p11 (a)
0 p01 H b p10 1
0 H 1
1 − Pe
0 Pe Pe
1 − Pe (b)
1
b H
14-1-1
-1
10
10-2-2-2 -2
10
8-2-2-2-2
Fig. 3. Binary channel models for a sensor network with decentralized detection: (a) realistic model and (b) approximate BSC model.
-3
Pe 10
uniform clustering -4
10
A. Realistic System Model We represent the network using a realistic binary channel model shown in Fig. 3 (a). It can be characterized through the following transition matrix: p00 p01 T= p10 p11 b = Hi |H j ), i, j = 0, 1. Using a well-known where1 pi j , P(H property of the mutual information described in [8, ch. 2], one can write the mutual information as b is the entropy of H b and HeREAL (H|H) b where HeREAL (H) is b b the conditional entropy of H given H. Since H is a binary random variable, it holds that 1 b = P(H b = H0 ) log2 HeREAL (H) b = H0 ) P(H 1 b = H1 ) log2 + P(H b = H1 ) P(H and similarly for the conditional entropy. Based on the model in Fig. 3 (a), after a few manipulations the mutual information can be expressed as −
He (p0 (1 − p10) + (1 − p0)p01 ) p0 He (p10 ) − (1 − p0)He (p01 ) .
(6)
B. Simplified BSC Model of the Sensor Network While, in general, p01 6= p10 in Fig. 3 (a), in order to simplify the binary channel model we consider a simplified BSC model, with cross-over probability equal to Pe . This is shown in Fig. 3 (b). As in Section III-A, one can write the mutual information of the system as b = HeBSC (H) b − HeBSC (H|H) b I BSC (H; H) = He (p0 (1 − Pe) + Pe (1 − p0)) − He (Pe ) .
(7)
Comparing (6) with (7), one can observe that the two expressions are, in general, different. However, if p0 = 12 , i.e., the a priori costs of the phenomenon are equal, it can be verified that b H) = I BSC (H; b H) . I REAL (H;
In other words, the simplified BSC model is accurate if the a priori costs are equal. i| j
1 Note that the definition of p is different from that of p ij l in Section II.
no clustering
-6
10 0
10
5
15
SNRsensor [dB]
Fig. 4. Probability of decision error for equal a priori costs (p0 = p1 = 1/2). Various cases with non-uniform and uniform clustering are considered. The lines correspond to analytical results, whereas symbols are associated with simulations.
IV. A PPLICATIONS
b = HeREAL (H) b − HeREAL(H|H) b I REAL (H; H)
b = I REAL (H; H)
-5
10
introduced
A. Probability of decision error In order to evaluate the probability of decision error, we consider a network with N = 16 sensors and ideal communication links (both from the sensors to the FCs and from the FCs to the AP). The probability of decision error is shown, as a function of the sensor SNR, in Fig. 4. In the figure, three non-uniform configurations and various uniform configurations are considered. • All the scenarios with uniform clustering correspond to a single performance curve. In other words, considering a clustered scenario with configuration given by 2-2-22-2-2-2-2 (8 clusters with 2 sensors each) leads to the same performance of the clustered configuration 4-4-44 (4 clusters with 4 sensors each) or 8-8 (2 clusters with 8 sensors each). Obviously, uniform clustering leads to a performance degradation with respect to a scenario with no clustering (also shown in the figure). However, after the first splitting (from one big cluster to two 8sensor clusters) no further performance degradation is observed with supplementary splitting. This has significant implications in the design of hierarchical sensor networks, and we are currently working on it. • In scenarios with non-uniform clustering, the considered configurations are 8-2-2-2-2 (5 clusters, 4 of which contain 2 nodes and 1 contains 8 nodes), 10-2-2-2, and 14-1-1. As one can see from Fig. 4, the higher is the non-uniformity degree among the clusters, the worse is the system performance. In all cases, both analytical (lines) and simulation (symbols) results are shown. As one can see, there is excellent agreement between them. B. Mutual Information In Fig. 5, the mutual information is shown, as a function of the sensor SNR, in a scenario with N = 8 sensors, uniform
1 -1
10
0,8 -2
10
1 cluster
SNRsensor = 0 dB
0,6
more clusters
I [bits]
SNRsensor = 6 dB
-3
Pe 10
0,4 -4
10
0,2
0 -20
SNRsensor = 10 dB
-5
10
-6
-10
0
10
20
SNRsensor[dB]
Fig. 5. Mutual information for an 8-sensor network with uniform clustering. The realistic binary model (lines) and the BSC model (symbols) of the real system are considered.
clustering, and equal a priori costs (p0 = p1 = 1/2). For comparison the mutual information in a scenario without clustering is also shown. These information-theoretic results confirm the communication-theoretic results in Fig. 4: all the mutual information curves associated with clustered scenarios (either 4-4 or 2-2-2-2) coincide, i.e., the information loss after the first subdivision is negligible. Moreover, in the same figure the results obtained considering the realistic binary model of the sensor network in Fig. 3 (a) (lines in Fig. 5), and the simplified BSC model in Fig. 3 (b) (symbols in Fig. 5), coincide for all values of the sensor SNR. As shown in Section III-B, the simplified BSC model is accurate in a scenario with equal a priori costs. In Fig. 6, the probability of decision error is shown as a function of the mutual information, in the sensor network scenario considered in Fig. 5 (uniform clustering). The curves shown in the figure are parameterized curves obtained by combining probability of decision error curves (such as those in Fig. 4) and mutual information curves (such as those in Fig. 5), through the parameter given by the sensor SNR. As one can see, all curves overlap. In other words, for a given value of the mutual information, the probability of decision error is the same. Note, however, that the same value of mutual information is obtained in clustered (for example, 4-4-4-4 or 2-2-2-2-2-2-2-2) and non-clustered scenarios for different values of the sensor SNR (in the figure, a few SNRs are highlighted). In other words, the presence of clustering leads to an energetic loss for a given level of mutual information. We have also extended our analysis to scenarios where the a priori costs of the phenomenon are not equal, i.e., p0 6= p1 . Our analytical framework is still in excellent agreement with simulation results. Unfortunately, these results are not reported here for lack of space. V. C ONCLUDING R EMARKS In this paper, we have proposed an information-theoretic perspective for the characterization of clustered sensor net-
10 0
0,2
0,4
0,6
0,8
1
I [bits]
Fig. 6. Probability of decision error as a function of the mutual information, in a 8-sensor network with uniform clustering. The simplified BSC model is considered. Circles (◦) correspond to absence of clustering and triangles (△) correspond to clustering.
works. Our results show that uniform clustering leads to a limited information loss with respect to a scenario with no clustering. This translates into a slight performance degradation, for a given sensor SNR, in terms of probability of decision error. In a scenario with non-uniform clustering, the higher is the non-uniformity level, the worse is the performance. In order to characterize this behavior, we have proposed a mutual information-based description of the considered decentralized detection schemes. We have discovered that, for a given value of the mutual information, the probability of decision error is uniquely determined. This motivates the use of the mutual information as a meaningful metric for the analysis of sensor networks. We have also shown that a simplified BSC model for the considered decentralized detection scheme, with cross-over probability equal to the probability of decision error, allows to correctly predict the realistic system performance. R EFERENCES [1] J. N. Tsitsiklis, Adv. Statist. Signal Process., vol. 2, chapter Decentralized detection, pp. 297–344, 1993, Eds.: H. V. Poor and J. B. Thomas. [2] A. R. Reibman and L. W. Nolte, “Detection with distributed sensors,” IEEE Trans. Aerosp. Electron. Syst., pp. 501–510, December 1981. [3] R. Viswanathan and P. K. Varshney, “Distributed detection with multiple sensors–Part I: Fundamentals,” Proc. IEEE, vol. 85, no. 1, pp. 54–63, January 1997. [4] W. Shi, T. W. Sun, and R. D. Wesel, “Quasi-convexity and optimal binary fusion for distributed detection with identical sensors in generalized gaussian noise,” IEEE Trans. Inform. Theory, vol. 47, no. 1, pp. 446–450, January 2001. [5] G. Ferrari and R. Pagliari, “Decentralized detection in sensor networks with noisy communication links,” in Proc. Tyrrhenian Int. Workshop on Digital Commun. (TIWDC’05), Sorrento, Italy, June 2005, available at www.tlc.unipr.it/ferrari/. [6] I. Y. Hoballah and P. K. Varshney, “An information theoretic approach to the distributed detection problem,” IEEE Trans. Inform. Theory, vol. 35, no. 5, pp. 988–994, September 1989. [7] C. Guestrin, A. Krause, and A. P. Singh, “Near-optimal sensor placement in gaussian processes,” in Proc. International Conference on Machine Learning, Bonn, Germany, August 2005. [8] T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley & Sons, Inc., New York, 1991.
