700
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 5, MAY 2008
On the Joint Synchronization of Clock Offset and Skew in RBS-Protocol Ilkay Sari, Erchin Serpedin, Kyoung-Lae Noh, Qasim Chaudhari, and Bruce Suter
Abstract—Motivated by the necessity of having a good clock synchronization amongst the nodes of wireless ad-hoc sensor networks, the joint maximum likelihood (JML) estimator for clock phase offset and skew under exponential noise model for reference broadcast synchronization (RBS) protocol is formulated and found via a direct algorithm. The Gibbs Sampler is also proposed for joint clock phase offset and skew estimation and shown to provide superior performance relative to JMLestimator. Lower and upper bounds for the mean-square errors (MSE) of JML-estimator and Gibbs Sampler are introduced in terms of the MSE of the Uniform Minimum Variance Unbiased (UMVU) estimator and the conventional Best Linear Unbiased Estimator (BLUE), respectively. Index Terms—Clock synchronization, maximum likelihood, Gibbs sampling, sensor networks.
I. I NTRODUCTION N sensor networks, the need for synchronized time arises as a very valuable tool for intra-network coordination among various sensors and for obtaining a coordinated interaction between the sensor network and the physical real world. Getting more accurate and energy-efficient synchronization protocols for wireless sensor networks that achieve the best performance limits represents a fundamental design problem for large-scale sensor networks. As explained in [1], the energy constraints in wireless sensor networks are very strict. Hence, using more overhead for better synchronization is not a good solution. To overcome both of these challenges at the same time, one way is to employ energy efficient protocols specifically designed for wireless sensor networks as [2] and [3]. The other way is to reduce the amount of energy spent on signal transmissions by using sophisticated tools from statistical signal processing as in [4] and [5]. In this letter, we will follow the later strategy by relying on more powerful statistical signal processing algorithms. It has been experimentally demonstrated by Pottie and Kaiser [6] that the energy required to transmit 1 bit over 100 meters (3 Joules) is equivalent to the energy required to execute 3 millions of instructions. Therefore, the strategy we have proposed here aims towards trading the computational energy for reduced communication energy. By developing highly accurate synchronization schemes, we aim to minimize the RF energy consumption (reduction of the number of synchronization beacons) by using slightly more complex computational algorithms. The key to achieve this objective is to employ powerful statistical signal processing techniques for developing highly accurate clock synchronization protocols.
I
Paper approved by H. Minn, the Editor for Synchronization and Equalization of the IEEE Communications Society. Manuscript received March 27, 2006; revised February 14, 2007, June 6, 2007, and September 24, 2007. I. Sari, E. Serpedin (contact author), K.-L. Noh and Q. Chaudhari are with the Dept. of Electrical and Computer Engineering, Texas A&M University, College Station, TX 77843-3128, USA (e-mail: {ilkay, serpedin}@ece.tamu.edu). Dr. B. W. Suter is in New York (e-mail:
[email protected]). Digital Object Identifier 10.1109/TCOMM.2008.060184.
This paper considers the joint maximum likelihood (JML) estimation of clock offset and skew under exponential noise model in the Reference Broadcast Synchronization (RBS) protocol [2]. Gibbs Sampler is also proposed for joint estimation of clock offset and skew. Lower and upper bounds for the performances of JML and Gibbs Sampler are derived in terms of mean-square errors (MSE) of the Uniform Minimum Variance Unbiased (UMVU) estimator and Best Linear Unbiased Estimator (BLUE), respectively. II. RBS-P ROTOCOL AND M ODELING A SSUMPTIONS RBS is a recently proposed receiver-to-receiver synchronization protocol for wireless sensor networks [2]. Roughly speaking, a transmitter-node broadcasts N synchronization signals and the receiver-nodes put time-stamps on these signals. Then, for efficient implementation, the receivers pass the data consisting of the time-stamps to the transmitter where the clock offsets and skews between different pairs of nodes are calculated. By the help of this protocol, two of the main error sources of clock synchronization are eliminated, which are uncertainties at Send Time and Access Time. Furthermore, the difference between propagation times is negligible compared to the uncertainty at Receive Time, which becomes the only error source. In [2], it is experimentally argued that uncertainty at receive time can be modeled in terms of the normal distribution. However, in real-life scenarios, there will be communication signals going around, not just synchronization signals, and nodes will have other jobs, not just the time-stamping. Therefore, we expect that the link from the transmitter to an individual receiver might behave as a regular point-to-point network link (M/M/1), where the receive time uncertainty has exponential distribution (see [7]). In some texts, this type of uncertainty is named as processing delay. Besides, whether the time-stamping is done on Application or Data Link Layer, will change only the mean of the distribution. In short, the ith time-stamps at the receivers X and Y are given by X[i] = T1 + θx + βx τ [i] + vx,λx [i],
(1)
Y [i] = T1 + θy + βy τ [i] + vy,λy [i],
(2)
where T1 stands for the time on the transmitter when it sends the first synchronization signal, θx and βx stand for the offset and skew between the clocks of the receiver X and the transmitter, τ [i] stands for the difference between T1 and the time of ith synchronization signal (with respect to the transmitter’s clock) and vx,λx [i] stands for the exponential iid (independently and identically distributed) noise (with mean 1/λx ), with i = 1, . . . , N . The parameters to be estimated, the offset and skew between the clocks of the nodes X and Y , are given by the following equations: Θ = θx − θy ,
c 2008 IEEE 0090-6778/08$25.00
β = βx − βy .
(3)
SARI et al.: ON THE JOINT SYNCHRONIZATION OF CLOCK OFFSET AND SKEW IN RBS-PROTOCOL
III. JML E STIMATION OF THE O FFSET AND S KEW The estimation of clock skew becomes more important in the context of energy-constrained sensor networks. If nodes have good skew estimates, they could live much longer without dedicating valuable resources for periodic resynchronization. Having these good reasons at hand, now we will consider the JML estimation for the skew and the phase offset. Although important, the joint estimation of clock skew and phase offset might not be easy. Sadler shows that under uniform noise, there are infinite solutions for ML estimation [5]. Besides, the support of likelihood function is not convex which leaves out the possibility of taking the mean of all equally likely solutions. In this letter, we will consider the case described in (1) and (2). As long as the two parameter sets {θx , βx , λx } and {θy , βy , λy } do not have a direct relationship and the noise sources in different nodes are independent (both of which are realistic assumptions), we can find the JML-estimator for Θ and β without loss of any information by estimating the parameters (θx , βx ) and (θy , βy ) separately and plugging these estimates back into (3). Thus, we will concentrate on the estimation of θx and βx . First of all, for simplicity, we will assume that τ [i] = i − 1 and T1 = 0, then the likelihood function becomes N L(θx , βx ) = λx e−λx (X[i]−(θx +(i−1)βx ) I(X[i]≥θx +(i−1)βx ) i=1
=
−λx N (X−f ) λN x e
N
I(X[i]≥fi ) ,
(4)
i=1
where f (θx , βx ) = θx + N2−1 βx , fi (θx , βx ) = θx + (i − 1)βx , X stands for the sample mean of observations X[i] (i = 1, . . . , N ), and I(x≥a) denotes the indicator function, being equal to 1 when x ≥ a and being 0 elsewhere. Note that in (4), the multiplication of indicator functions defines a convex region (S) on the parameter space (θx , βx ), N with S = {(θx , βx ) : i=1 fi (θx , βx ) ≤ X[i]}. S has k k vertices {sj }j=1 and k+1 edges (1 ≤ k ≤ N −1). Specifically, the shape of this region and the value of k will strongly depend on the ordering of X[1], ...X[N ]. On this region, we have to maximize the objective function, f (θx , βx ) = θx + N 2−1 βx . Since 0 < N2−1 < N − 1, the support of the solution is guaranteed to be a closed-convex region on the boundary of S. If N = 2m, the solution will be one of the vertices sj and if N = 2m − 1 the solution will assume possibly a segment of the line fm : θx + (m − 1)βx (or again one of the vertices sj , depending on the ordering of the observations). Fig. 1 illustrates these remarks for N = 2 and X[2] > X[1]. In this illustrative example, since f attains its maximum on s1 amongst all points on S, s1 gives the JML estimation of θx and βx . Before proceeding any further, we have to clarify one more point. In derivations up to now, we assumed that λx and λy were both known. However, if we assume they are unknown and use the reduced likelihood function for (θx , βx ) as in [8], it is straightforward to show that we end up with the same JML solution. IV. A PPLICATION OF G IBBS S AMPLER Although it is possible to find the exact solution for the ML-estimate as explained above, we will also apply the Gibbs
Fig. 1.
701
S and the solution s1 .
Sampler to jointly estimate the parameters. Although by using the Gibbs Sampler it is possible to find an approximate JML estimation which is arbitrarily close to the exact one, there are some more important advantages that the Gibbs Sampler will provide us. First of all, it can be shown that the JML estimation (θˆx,ML , βˆx,ML ) is biased for finite N . (As an example consider the case in Fig. 1, E[θˆx,ML ] = E[X[1]] = E[θx + vx,λx [1]] = θx + 1/λx .) For this reason, we need to look for a uniform minimum variance unbiased (UMVU) estimator. However, the Neyman-Fisher factorization theorem provides mini ((X[i] − θx + (i − 1)βx )) as sufficient statistics, which is not independent of the parameters to be estimated. On the other hand, if we use the Gibbs Sampler at the end we do not have just a single point estimate but the posterior distribution for the parameters to be estimated as the output. Then, we can either find the JML-estimator or set the corresponding estimator as the mean value of the posterior distribution of the parameter, which will automatically perform the marginalization and will give better results with reduced bias and variance. For details on the Gibbs Sampler, please refer to [9]. Another appealing feature of the Gibbs Sampler is its straightforward extendability for additional unknown parameters. For example, it is possible that λx is unknown or in addition to the clock phase offset and skew we could have clock drifts: γx and γy . Although very important, due to the limited space we did not consider such a scenario in this letter. The drifts will be observed on RHS (right-hand side) of (1) and (2) as additional terms: τ 2 [i]γx and τ 2 [i]γy . Definitely, it is straightforward to adapt the Gibbs Sampler to these scenarios. Before applying it, we will briefly give some information about the Gibbs Sampling. Assume that we have the data vector z and we want to estimate some parameters Φ = [φ1 , φ2 , ..., φM ]T . For any kind of statistical inference we want to use the joint posterior distribution of the parameters p(Φ|z) ∝ p(z|Φ)p(Φ) (in point estimation, prior distribution p(Φ) is chosen as noninformative). When it is hard to carry out mathematical derivations on the posterior, we stick to MonteCarlo methods, i.e., to draw as many samples as possible from the posterior so that the inference we make using these samples will be arbitrarily close to the exact solution. When it is hard to draw from the joint posterior directly, MCMC
702
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 56, NO. 5, MAY 2008
(Markov Chain Monte-Carlo) type of iterative methods will be used. That resumes to setting up a Markov chain whose stationary distribution is the joint posterior we need. One convenient way to do this is to use Gibbs Sampling in which we iteratively draw samples from one-dimensional conditionals p(φi |z, Φi ), where Φi is an (M − 1) × 1 vector with entries {φj }j=i . Under mild conditions, these one dimensional conditional distributions uniquely determine the joint posterior distribution [10]. Specifically, the general algorithm for Gibbs Sampling with (0) (0) initial values Φ(0) = [φ1 , ..., φM ] is to iterate the following: (1) (0) (0) • Draw φ1 from p(φ1 |z, φ2 , ..., φM ) (1) (1) (0) (0) • Draw φ2 from p(φ2 |z, φ1 , φ3 , ..., φM ) .. . (1) (1) (1) • Draw φM from p(φM |z, φ1 , ..., φM−1 ). After a threshold value t, the set {Φ(t) , Φ(t+1) , ...} behaves like samples from the joint posterior of the parameters. One important point is that the joint posterior distribution should be proper. Otherwise the Gibbs Sampler always converges to some local points, but not necessarily to a meaningful one [11]. For this reason to assure that the posterior is proper, in application of Gibbs Sampler to the point estimation, priors are not directly chosen as flat, but they are chosen from conjugate families and then their parameters arranged so as to have noninformative priors. However in our case, the likelihood function itself can be used as posterior distribution, since its integral is always bounded and positive-valued which makes it proper. We do not need to use any other type of priors but flat. Then in our case, using (4), the procedure becomes
10
• •
(1)
Draw θx Draw
(1) βx
θxt+1 ,
(0)
from ∝ eλx Nθx I(θx ≤ mini (X[i] − (i − 1)βx )) from ∝ e
λx
N (N −1) βx 2
I(βx ≤
(1)
x mini ( X[i]−θ i−1
)).
For we will draw a sample from the exponential distribution with parameter λx N , multiply it with -1 and add (t) mini (X[i]−(i−1)βx ) to it. The procedure for βxt+1 is similar. Note that if λx were unknown, we would utilize the Gamma (t+1) distribution to draw for λx . V. P ERFORMANCE B OUNDS AND S IMULATIONS In this part, we will look at the performances of the Gibbs Sampler and the JML-estimator. However, it will be useful to have some benchmarks with whom to compare their performances. First we will look for lower bounds. Since the likelihood function does not satisfy the regularity conditions required by CRLB (Cramer-Rao Lower Bound), calculating CRLBs is dropped out of the list. One possible lower bound can be found by assuming that all the parameters are known but the one to be bounded, which reduces the problem to the well-known derivation of the bound for a single unknown parameter in exponential noise. Then we can find the UMVU estimators both for the phase offset and skew in closed forms. For θx , it is derived that the UMVU becomes 1 θˆx,UMV U = mini (X[i] − (i − 1)βx ) − , (5) N λx and the MSE (Mean-Square Error) of the estimator equals 1/(N λx )2 [12]-[13]. For βx , the likelihood function is N (N −1) X[i] − θx ). (6) L(βx ) = Ceλx 2 βx ΠN i=2 I(βx ≤ i−1
−6
BLUE GIBBS JML UMVU GIBBS (for truncated Gaussian priors)
−7
MSE of Estimators
10
10
10
−8
−9
4
Fig. 2.
8
12
16 20 24 N:Number of Synchronization Signals
28
32
36
M SE for θˆx,BLU E , θˆx,J M L , θˆx,GIBBS , and θˆx,U M V U .
x By Factorization Theorem, mini ( X[i]−θ i−1 ) is sufficient statistics and it is straightforward to show that it is also complete. This result can be established by following the similar lines of proof as it is done in [12] for θx . Then, by Lehmann-Scheffe Theorem, the UMVU estimator for the skew when the offset and λx are known takes the form:
2 X[i] − θx βˆx,UMV U = mini ( )− . i−1 λx N (N − 1)
(7)
The MSE of the estimator (7) is equal to the variance of x Z = mini=2,3,...,N ( X[i]−θ i−1 ). Thus, we first need to determine the distribution of Z. From the theory of order statistics, the distribution of the minimum of a sample set is given by F (z) = 1 − (1 − F2 (z))(1 − F3 (z))...(1 − FN (z)), where x Fi (z) = P r( X[i]−θ ≤ z) = P r(vx,λx [i] ≤ (i−1)(z −βx)) = i−1 λx (i−1)(z−βx ) (1 − e )I(z ≥ βx ). Then the distribution becomes F (z) = 1 − eλx (ziβx )(1+2+...+N −1) = 1 − eλx (z−βx )
N (N −1) 2
, (8) which is an exponential distribution with the scale parame−1) and the location parameter βx . The MSE of ter λx N (N 2 βˆx,UMV U equals the variance of Z which is 4/(λx N (N − 1))2 . Therefore, we do not expect the MSE of joint estimator for (θx , βx ) to decay faster than ∝ (1/N 2 , 1/N 4 ). We will also consider the BLUE (Best Linear Unbiased Estimator), since it will represent an upper bound. Here, the same notation is used as [5] except that X is replaced with A (A [1, x], where 1 = [1, 1, · · ·, 1]T and x = [0, 1, · · ·, N − 1]T ) to prevent possible confusion. Since noise is not zero-mean in our model unlike [5], we need to subtract 1/λx from the resulting linear estimate of θx so as to end up with the BLUE-estimator. Then we have [θˆx,BLUE , βˆx,BLUE ]T = (AT A)−1 AT X − [ λ1x , 0]T . It is known that var([θˆx,BLUE , βˆx,BLUE ]T ) 2 T −1 = diag{1/λx (A A) } ∝ [1/N, 1/N 3 ]T . For a detailed discussion on this estimator, the interested readers are referred to [5]. The MSE of the Gibbs Sampler and the JML-estimator for θx = 1 and βx = 0.01 with λx = 103 (which makes var(vx,λx ) = 10−6 ), are presented in Figs. 2 and 3, respectively. In these simulations, the initial values of clock parameters are chosen as zeros. These figures also include the
SARI et al.: ON THE JOINT SYNCHRONIZATION OF CLOCK OFFSET AND SKEW IN RBS-PROTOCOL
10
MSE of Estimators
10
−6
6 LP GIBBS
BLUE GIBBS JML UMVU GIBBS (for truncated Gaussian priors)
−7
5
−8
Relative Number of Computations
10
703
4
10
−9
3
10
10
10
−10
2
−11
−12
Fig. 3.
4
8
12
16 20 24 N:Number of Synchronization Signals
28
32
36
M SE for βˆx,BLU E , βˆx,J M L , βˆx,GIBBS and βˆx,U M V U .
lower and upper bounds presented above. The MSE are plotted against the number of synchronization signals from 4 to 36. It is interesting to note that the MSE of the Gibbs Sampler and the JML-estimator behave like the lower bound, i.e., decay rates on the order of ∝ 1/N 2 and ∝ 1/N 4 , respectively. Note also that the Gibbs Sampler performs better with MSEvalues around 40% for θx and 25% for βx compared to the corresponding values of JML-estimator. We should also note that the convergence of the Gibbs Sampler is achieved after a number of iterations on the order of N . To shed some light on the sensitivity of the Gibbs Sampler to the prior mismatch, we have also provided some simulation results for the mismatched prior knowledge. This is important for engineers and system designers in order to make proper choice of estimator for their considered systems. Fig. 2 and Fig. 3 show the performance of Gibbs Sampler where we have modeled actual prior as a truncated Gaussian while the assumed prior in the Gibbs Sampler is uniform. For prior of offset, truncation points have been chosen as 0 and 10 whereas the mean and Standard Deviation of parent Gaussian distribution as 5 and 2 respectively. And for prior of skew, truncation points are chosen as 0 and 1 whereas the mean and Standard Deviation of parent Gaussian distribution as 0.5 and 0.25 respectively. One drawback of the Gibbs Sampler is definitely its computational complexity. The computational complexity of Gibbs Sampler is affected by the random number generations in each iteration and the number of iterations necessary to converge. Fig. 4 compares the computational complexity of Gibbs Sampler with that of the Linear Programming algorithm through Matlab’s built-in flops function. Although the Gibbs Sampler clearly requires more computations, the required level of precision can be achieved by lesser number of signal transmissions. Hence, there is a tradeoff between the complexity and the gains achieved by Gibbs Sampler. VI. C ONCLUSIONS Under the exponential noise model, we have shown the JML-estimator for the clock skew and phase offset, the JMLestimator is not ill-behaved as opposed to the uniformlydistributed noise case from [5]. JML-estimator of the skew and the phase offset exists and is either unique or a line segment
1
0
4
Fig. 4.
8
12
16 20 24 N: Number of Synchronization Signals
28
32
36
Computational Complexity of the LP and the GIBBS Algorithms.
depending on the magnitudes of the observed data samples. At worst, the support of all equally likely solutions is a closedconvex set (a line segment). The setting was convenient to apply Gibbs Sampler which further increased the performance of JML-estimator. The performances of both estimators (JML and Gibbs Sampler) scale with the same power-law (with respect to the number of synchronization signals: N ). Lower and upper-bounds for the performance of JML and Gibbs Sampler estimators were also presented in terms of the MSEperformances of UMVU estimator and BLUE, respectively. R EFERENCES [1] B. Sundararaman et al., “Clock synchronization in wireless sensor networks: a Survey,” Ad-Hoc Networks, vol. 3, pp. 281-323, May 2005. [2] J. Elson, L. Girod, and D. Estrin, “Fine-grained network time synchronization using reference broadcasts,” in Proc. OSDI, 2002. [3] S. Ganeriwal et al., “Timing-sync protocol for sensor networks,” in Proc. ACM SenSys, 2003, pp. 138-149. [4] N. Khajehnouri and A. H. Sayed, “A distributed broadcasting timesynchronization scheme for wireless sensor networks,” in Proc. ICASSP, 2005, pp. 1053-1056. [5] B. M. Sadler, “Local and broadcast clock sync. in a sensor network,” IEEE Signal Processing Lett., vol. 13, pp. 9-12, Jan. 2006. [6] G. Pottie and W. Kaiser, “Wireless integrated network sensors,” Commun. ACM, vol. 43, no. 5, pp. 51-58, May 2000. [7] H. S. Abdel-Ghaffar, “Analysis of synchronization algortihms with timeout control over networks with exponentially symmetric delays,” IEEE Trans. Commun., vol. 50, pp. 1652-1661, Oct. 2002. [8] D. R. Jeske, “On the maximum likelihood estimation of clock offset,” IEEE Trans. Commun., vol. 53 pp. 53-54, Jan. 2005. [9] A. E. Gelfand and A. F. M. Smith, “Sampling-based approaches to calculating marginal densities,” J. Amer. Stat. Ass., vol. 85, pp. 398409, 1990. [10] J. Besag, “Spatial interaction and the statistical analysis of lattice systems,” J. Roy. Stat. Soc. Series B (Methodological), vol. 36, pp. 192236, 1974. [11] J. P. Hobert and G. Casella, “Functional compatibility, Markov chains, and Gibbs sampling with improper posteriors,” J. Comp. Graph. Stat., vol. 7, pp. 42-60, 1998. [12] E. L. Lehmann and G. Casella, Theory of Point Estimation. Springer, 1998. [13] D. R. Jeske and A. Sampath, “Estimation of clock offset using bootstrap bias-correction techniques,” Technometrics, vol. 45 pp. 256-261, Aug. 2003.
