Causality and the Spatial-Temporal Ordering in Mobile Systems

Baltzer Journals

Causality and the Spatial-Temporal Ordering in Mobile Systems Ravi Prakash1∗ and Roberto Baldoni 1

2

† 2

Department of Computer Science, University of Texas at Dallas, Richardson, TX 75083-0688, USA. E-mail: [email protected]

Dipartimento di Informatica e Sistemistica, University of Rome “La Sapienza”, Via Salaria 113, I-00198, Italy. E-mail: [email protected]

Several mobile computing applications require that both the order and location of occurrence of events be taken into account during decision making. Thus, processes need to track the location of nodes and synchronize their clocks. The Global Positioning System can be employed to mimic a global virtual clock that keeps the local clocks of participating nodes in synchrony with each other. The global virtual clock in conjunction with a space-time vector can track the mobility of nodes. Nodes can prioritize resource requests on the basis of request time as well as the requester’s distance from the resource. Two distributed mutual exclusion algorithms that employ the space-time vector are presented. The error in the estimates of a mobile node, due to clock drift, about the region in which other mobile nodes may be present is formulated. Various resource allocation policies can react differently to such errors leading to performance differences. However, every policy should ensure that resource allocation has the properties of safety, deadlock freedom, liveness, and fairness.

1

Introduction

In the asynchronous model of distributed computation message propagation delay is finite but non-deterministic. All the processes have their own clocks, and individual clocks tend to drift apart. Deterministic clock synchronization algorithms that require communication between participating processes cannot achieve an upper bound on clock skew lower than (dmax −dmin )(1−1/N ) [10]. Here dmax and dmin are the maximum and minimum message delays in the system, respectively and N is the number of processes. As the system size increases this value increases. Alternatively, if all the nodes were to read the time from a single global clock, the information would take a non-deterministic period of time to travel. Due to this non-determinism, the participating processes do not have the notion of global ∗

The author’s work was supported in part by the National Science Foundation grants CCR9796331 and ANI-9805133. † The author’s work was supported in part by the EU IST project MIDAS and by the MURST project MAIS.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

2

time. Probabilistic clock synchronization algorithms [1, 2] can achieve maximum skew values much lower than that specified in [10]. However, they have a non-zero probability of failure resulting in clock skews much greater than the upper bound. In the absence of a global clock, Lamport proposed the idea of a logical scalar clock that captured the happened before relation between events [8]. In several scenarios, a mobile computing system can be considered to be a distributed computing system with the additional notion of node mobility. The actions of a node may be influenced not only by the happened before relationships in the system, but also by the location of the nodes. Thus, besides causality, there is also a need to track the locality of nodes in the system. The problem is that node mobility patterns are not deterministic. Therefore, accurate information about the current location of all nodes may not be available to any one node. Our goal is twofold. First, we wish to synchronize the clocks of all the process in a deterministic fashion with an upper bound independent of the message propagation delay and system size. Such a set of synchronized clocks that is unaffected by system characteristics can be considered as a virtual global clock. Independence from message propagation delay is especially useful for mobile systems that use low bandwidth, high latency wireless links, with possibly high variance in propagation delay, for communication. Independence from system size provides scalability. Second, we wish to continuously refine every node’s knowledge of the position of other nodes in the system. By refinement we mean a reduction in the degree of uncertainty about other nodes’ locations. Knowledge of node location is required for allocation of resources that can be: (i) used only by nodes within a certain distance of the resource, or (ii) can be used concurrently by multiple processes as long as there is some physical separation between them. With the advancement of technology, it has become imperative that we reconsider the traditional assumption regarding the absence of a global clock in a distributed system. Now, we have Global Positioning Systems (GPS) supported by a constellation of satellites that can provide very accurate clock information [5, 9]. So, if all the nodes were to be equipped with GPS receivers, they could all be sharing a virtual global clock. Furthermore, GPS provides each node with its location information (latitude, longitude and elevation) with a high degree of accuracy. In the context of the capabilities provided by GPS, we propose a framework for modeling and tracking locality information of mobile nodes in a distributed system. We consider node mobility restricted to a two-dimensional space. The location of a node can be expressed as an (x, y) co-ordinate with respect to a third dimension, namely time. Thus, if at time 0 the location of a node is (0, 0), then its location at time t will be restricted to the interior of a circle of radius s × t centered at (0, 0), where s is the upper bound on the speed of mobile nodes. This paper is organized as follows. In Section 2, we briefly describe the Navstar Global Positioning System and then we state how GPS can be used to provide a virtual global clock for a distributed mobile system. In Section 3 we revisit the happened before relation [8] and show how GPS synchronized clocks at individual nodes can be used to capture this relation. Such scalar clocks can eliminate vector clocks [3, 11] that have been used until now in distributed computing to capture the symmetric relation between causal precedence and timestamps of events. In Section 3.1 we present the concept of spacetime vectors, enumerate their properties and draw parallels to Lamport’s logical clock. GPS, which provides a node with its own space-time co-ordinates, does not provide any information about the location of other nodes. We explain how the space-time vectors

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

3

can be used to refine a node’s knowledge of the location of other nodes in the system. A process Pi can inform another process Pj of its location only if there exists a chain of messages originating at Pi and terminating at Pj . Processes stamp their own space-time co-ordinates and also piggyback their latest knowledge of the space-time co-ordinates of other processes on the outgoing messages. The location information about other nodes is imprecise as the recipient is only aware of a node’s location in the past and it could have moved since then. We also present distributed mutual exclusion algorithms for singleresource and multi-resource allocation problems in the mobile environment. Space-time vectors are used to prioritize resource requests. In Section 4 we describe how the location information, along with timestamps can be used to determine the spatial-causal relations between events. Finally, the conclusions are presented in Section 5.

2 2.1

The Computation Model The Global Positioning System

The Navstar Global Positioning System (GPS) is funded, designed and controlled by the U. S. Department of Defense [5, 9]. However, there are thousands of civilian users of GPS world-wide. The GPS operational constellation consists of 21 navigational satellites and 3 active spares that orbit the earth in 12 hour orbits ensuring full earth coverage. The satellites repeat the same track and configuration over any point approximately each 24 hours (4 minutes earlier each day). There are six orbital planes (with nominally four satellites in each), equally spaced (60 degrees apart), and inclined at about fifty-five degrees with respect to the equatorial plane. This constellation provides the user with between five and eight satellites visible from any point on the earth. Each GPS satellite has two cesium and two rubidium atomic clocks that can gain or lose about 1 second in 160, 000 years. Fresh clock correction factors are relayed to each satellite at least once every day. So, the clocks on all the satellites can be safely assumed to be perfectly synchronized as far as the granularity of inter mobile node communication latency is concerned. The GPS constellation provides two basic services: a positioning services and a clock service. Both of them have two distinct classes of quality of services: (i) the Standard Positioning Service (SPS) which is used for civilian purposes and (ii) Precise Position Service (PPS) used by military or selected civilian users specifically approved by the U. S. Government. For example, the SPS predictable accuracy is of the order of meters and of hundreds of nanoseconds for positioning and clock value respectively. The PPS predictable accuracy is of the order of centimeters and tens of nanoseconds [5, 9]. The GPS Navigation Message consists of time-tagged data bits marking the time of transmission of each subframe from the satellite. Data frames (1500 bits) are sent every thirty seconds (one bit each 20 msec.). Each frame consists of five subframes. The first subframe contains the amount by which the GPS Time is offset from Universal Coordinated Time. This correction can be used by a user with a GPS receiver to set Universal Coordinated Time (UTC) to within 100 ns. All the subframes contains clock information bits that allows the receiver to have a high degree of accuracy. A GPS receiver is also able to compute its own location (Rx , Ry , Rz ) starting from the coordinates of the four visible satellites (sent with the frames) through the solution of a set of equations [9].

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering 2.2

4

Mobile Nodes

The system is composed of a set of n independent nodes with the ability to move at a maximum admissible speed s1 . No base station exists to coordinate the activities of subsets of nodes. Changes in network topology over time are arbitrary. Nodes communicates only by exchanging messages and all communication is over wireless links. A mobile node has a link with another mobile node if they are within wireless range of each other. We assume that a single or multiple wireless hop path exists between every pair of mobile nodes. However, these paths may change over time. At the logical level paths correspond to channels. Each pair of mobile nodes is connected by an asynchronous logical channel and transmission delays are finite, unpredictable and greater than a minimum transmission delay Tc . Each mobile node is equipped with an internal clock (e.g. quartz crystal) which has a maximum drift rate, σ, with respect to an External Physical Time, EPT, (e.g. atomic time), of 10−5 . The synchronization between the External Physical Time and the internal clock is done every t seconds. So an internal clock of a mobile node may diverge from the EPT by at most σt [6]. Assuming each mobile node is equipped with a GPS receiver that is powered on for the entire duration of operation. Then, the maximum drift of a non-faulty internal clock (without any compensation for errors in crystal frequency) with respect to the Universal Coordinated Time (UTC) 2 , which acts as EPT, is 0.3 msec as the internal clock is synchronized to UTC each 30 seconds. 2.3

Processes

A process is a software entity running on a mobile node. We assume, without loss of generality, that there is one process per mobile node. Execution of a process Pi produces a sequence of events ei0 , ei1 , . . . eiw which can be classified as: send events, delivery events and internal events. An internal event may change only local variables; send or delivery events involve communication. We assume that the minimum time between two successive events in a process is Tp . Finally, the set of all events produced by all processes is denoted by E. Figure ?? shows the evolution of three processes denoting events as black circles. 2.4

Virtual Global Clock

Processes are endowed with a virtual global clock, formed by the set of internal clocks: one for each mobile node. The virtual global clock is a software representation of the EPT. Even though all the internal clocks are periodically synchronized with the EPT, internal clock values are not exactly the same all the time: they differ at most by the clock precision π [17], which represents the maximum difference between any two internal clock values. Indeed, internal clocks of two mobile nodes drifting in opposite directions may diverge by at most 2σt. The latter value represents the upper bound on π for the 1

The maximum admissible speed depends on the underlying network system, e.g., DECT, GSM, PCS, ETACS. For example, in a DECT microcellular system s is less than 40 km/hours (approximately 11m/s). For GSM and PCS systems s is bounded by maximum highway traffic speeds (about 50m/s). 2 Universal Coordinated Time is the world timekeeping. It is provided, by means of shortwaves, by the National Institute of Standard Time from Fort Collins, Colorado.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

5

virtual global clock. A virtual global clock is also characterized by its granularity g (the time between two successive ticks). In order to avoid the overlapping of distinct global ticks in different nodes Kopetz introduced the granularity condition stating that the granularity of a global clock must be greater than or equal to the global clock precision (i.e., g ≥ π) [7]. When each mobile node is equipped with a GPS receiver the maximum drift between two internal (non-faulty) clocks with respect to UTC is 0.6 msec (2 × 10−5 × 30 seconds). So, the granularity of the virtual global clock can be safely assumed to be of the order of a few milliseconds. Many clock synchronization protocols described in the literature are currently used in wired networks and provide a virtual global clock that bounds the value of the sum of the precision and granularity within a few milliseconds (5-10 msec.) [1, 4, 10, 12]. However, the advantage of using GPS based clock synchronization is that the GPS receiver of a node is a passive listening device, and is not required to send messages to other nodes. This results in substantial energy savings compared to the other algorithms wherein energy is spent in communication between nodes. This is of great consequence as mobile nodes have limited energy supply. Not having to communicate also conserves bandwidth. Furthermore, the GPS receiver also provides location information which is critical for several mobile applications, as described in later sections. 2.5

Timestamping of Events

Using the virtual global clock, each event eiw of a node is associated with a global time value (timestamp) at which the event occurred. Granularity and non-zero precision of a virtual global clock could lead to errors like timestamping of two distinct events of a process with the same value or, if communication is involved, the timestamp of the send event of a message could be greater than or equal to the timestamp of the corresponding receive event. In [17] Verissimo observed that given two events a and b at distinct nodes, timestamped by a global clock with precision π and granularity g, their timestamps reflect their relative temporal ordering only if |t(a)−t(b)| ≥ π +g. As a consequence, the quantity π + g represents the minimum inter-event time which allows causality between two events to be represented using a scalar virtual global time. So, in the following discussion we assume that time for communication, Tc ≥ π + g. Concerning events occurring on a process, we assume that the time between successive events of a process Tp ≥ π + g. 3 With these two basic assumptions, we force each event eiw at a process to have a unique timestamp value, and eiw could potentially affect another event ejh in its future such that eiw occurred before ejh with respect to the global clock value. Here, we denote as eih an event that occurred in process Pi at time th . We now state a temporal ordering relation between events, denoted by ≺. Definition 2.1 Let eiw and ejh be two events in E with virtual global timestamps of tw and th , respectively. Then, eiw ≺ ejh iff tw < th . 3

Note that this assumption is not strictly necessary. We only need that the time interval between two successive communication events be greater than π + g. The reason we use that assumption is to assign an unique timestamp to each event in each process.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

6

Two events eiw and ejh are concurrent if ¬(eiw ≺ ejh )∧¬(ejh ≺ eiw ). In other words, if two events are concurrent then tw = th . The relation ≺ defines a partial ordering on events (E, ≺). Thus a scalar clock, based on virtual global time, can be used to order events. 2.6

Positioning Service

Each process may access an internal positioning service provided by the mobile node (using GPS, for example) that provides the three basic coordinates X, Y, Z. We assume that all the nodes are at the same elevation. So, only two coordinates X, and Y of the positioning system need to be considered for location purposes. Li (T ) returns the location of Pi at time T . Overloading the terminology a little, we assume that Li (eiw ) denotes the location of Pi while executing event eiw at time tw . We assume that each process knows the initial position of every other process and the minimum distance between two nodes ∆L is greater than the accuracy provided by the positioning service. Processes get location information with an accuracy of the order of centimeter or meters depending on the type of the GPS receiver.

3

Spatial-Temporal Ordering

In this section we first introduce the space-time vector notion which can be used by distributed algorithms to track mobile entities. Then, we discuss the problem of how to get a total spatial-temporal ordering among events. Then we derive an interval-based spatial temporal ordering that exploits the tradeoff between ordering by location and by time. Sections 3.3 and 3.4 present algorithms where proximity to a resource is an important consideration in resource allocation. 3.1

Space-Time Vectors

We propose to use a space-time vector, ST , to diffuse spatial-temporal information about nodes, and to determine the spatial-temporal ordering among events at these nodes. This vector is an extension of the vector clock proposed by Mattern and Fidge [3, 11] and consists of n 2-tuples, where n is the number of nodes that we need to track in the system. Each tuple corresponds to (time, location) information. Let STi [j] = (tj , lj ). This means that Pi ’s most recent information about Pj corresponds to Pj being at location lj at virtual global clock time tj . The value of STi [i] is always current, while STi [j] for j 6= i may be outdated. Initially, ∀ i, j : STi [j] = (0, initial location of Pj ) A node’s messages are stamped with its space-time vector. We refer to this operation as time-location stamping. A space-time vector is updated in the following manner: 1. Periodically, and definitely before sending a message and on receiving a message, Pi sets the value of STi [i] to be equal to (current time, current location) where these components are obtained using the virtual global clock and the positioning service. 2. On receiving a message m carrying the time-location stamp ml , Pi performs the

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

7

following operation: ∀j 6= i : if ml [j].time > STi [j].time then replace STi [j] with ml [j]. Thus location information about nodes can be propagated through messages. Note that STi [j].location represents the latest position of Pj known to Pi . 3.2

Getting a Total Spatial-Temporal Ordering

Until now the usual approach to define a total order among events has been based on their scalar or vector timestamps. If two events were mutually concurrent the tie was broken using the identities of the processes on which they occurred. In a mobile application the impact of an event occurring in a process also depends on the location of the process at that time. Therefore, in such systems the total order should be influenced by the location of a process, as well as by the virtual clock time when the event occurs in the process and the process identity. For example, let there be a competition for a possibly mobile resource among two processes residing on different nodes, mobile or stationary. Both processes issue a request during the same virtual global clock tick. In some situations a higher priority should be assigned to a request made by the process closer to the resource. In this case a timestamp tie should be broken by using the distance of the requesting node from the location of the mobile resource. This is because once assigned, the shorter the distance the smaller the amount of time for the resource to reach the requester and vice versa. Hence, distance has an impact on resource use latency. Let us now introduce the spatial-temporal total ordering of events with respect to a given reference point R located at position r (denoted ≺r ). Definition 3.1 Let eiw and ejh be any two events belonging to E, and let R be a reference point whose location is r, eiw ≺r ejh iff: (1) tw < th or, (2) tw = th and |L(eiw ) − r| < |L(ejh ) − r| or (3) tw = th and |L(eiw ) − r| = |L(ejh ) − r| and i < j Here, |a − b| denotes the distance between locations a and b. It is easy to see that ≺r is a total order and that ≺ ⇒ ≺r . As an example, Figure 1.a shows the position of three processes Pi , Pj and Pk with respect to the reference R located at coordinate (6, 5) at global virtual time 4. Then, we have ej4 ≺r ei4 ≺r ek4 . 3.3

Mutual Exclusion Algorithm for Single Resource

Let there be a set of n processes competing for one resource. All the entities in the system are mobile with an upper bound on speed equal to s. Processes make requests for the resource to all the other processes and conflicting requests are granted by the other processes according to the ≺r serialization discipline, using the location of the resource as the reference point. For this purpose each process should have the time-location stamp of each request and identical information about the location of the resource. Hence, a space-time vector of one component is used as we need to track the position of the single

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

Y

8

Y Pi

R

L(ej4)=(9,5)

L(e i4)=(1,5) r=(6,5) Pj

Pk L(ek4)=(12,3)

Pi

R

L(e j5)=(11,5) Pj

L(e i5)=(3,5) r=(6,5)

Pk L(ek5)=(10,3)

X (a) Time 4

X (b) Time 5

Figure 1: Positions of processes Pi , Pj andPk at virtual global time 4 and 5. resource.4 An example application that matches this set of requirements is a pick-up express courier service. Persons calling for package pick-ups are equivalent to processes, the cab of the pick-up service is the resource. The pick-up service collects the packages on the basis of request time. If multiple requests have the same timestamp, then packages are picked up on a closest-job-first policy. After each pickup the distance from outstanding requests should be recomputed and the greedy closest-job-first selection should be repeated. Each process Pi owns a queue Q of requests ordered according to ≺r and a pair (Tr , Lr ) which represents Pi ’s latest knowledge of the position of the resource. For each process to be able to sort requests by ≺r each resource request message should be timelocation stamped and the resource release message should carry the time-location stamp of the resource. We assume that all the processes know the initial position of the resource. We also assume that only broadcast messages (resource request and resource release) can update the knowledge of a process about the location of a resource. As stated above, the algorithm serves conflicting requests according to ≺r . Two requests are conflicting iff there exists at least one process in whose queue Q both requests are present at the same time. The algorithm of each process is split into two parts. The first part is executed when the process want to acquire the resource. It consists of the entry to and exit from the critical section. Entry Code: (i) Pi sends a request((ti , li ), (Tr , Lr )) message to all the processes. (ii) wait until Pi has received a grant message from each process access the resource Exit Code: (i) Pi sends a release((Tr , Lr )) message to all the processes. 4 As we defined the problem, a process needs to know only the position of other processes at the time they issued a request. So there is no need to track their positions using additional components in the space-time vector.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

9

The second part is executed upon the arrival of a message: When a requesth(tj , lj ), (tr , lr )i message is received from Pj : (i) If tr > Tr then (Tr , Lr ) := (tr , lr ) (ii) Pi inserts the request (tj , lj ) in Q and also rearranges the queue ( as per ≺r ) based on the latest information about the location, Lr of the resource. (iii) If the request is at the head of Q, and Pi is not in its critical section, then Pi sends a grant message to Pj . When a releaseh(tr , lr )i message is received from Pj : (i) If tr > Tr then (Tr , Lr ) := (tr , lr ). Also, Pi rearranges the queue based on the latest information about the location of the resource. (ii) Pj ’s request is removed from Q. (iii) A grant message is sent to the process Pw , whose request (tw , lw ) is at the head of Q, if that grant was not already sent to Pw . Safety: The resource is used by at most one process at a time. If a request does not conflict, the claim trivially follows. Let ri and rj be two conflicting requests issued by Pi and Pj such that ri ≺r rj . As each request is sent to all the processes, in the worst case, rj reaches all processes, except one (otherwise the requests would be non-conflicting), before ri reaches them. Let P` be the process where ri reaches before rj . P` will then send the grant correctly to Pi and will prevent Pj to get the quorum required to access the resource. As soon as all the other processes receive ri , they, too, send a grant message to Pi (as ri ≺r rj , ri will be inserted in front of rj in the queue of Q in each process), allowing Pi exclusive access to the resource. Deadlock Freedom: There is no deadlock among processes for the shared resource. The proof follows from the result in [15] about Maekawa-type mutual exclusion algorithms. The proposed algorithm has the same conditions for sending grant messages as the algorithm described in [15]. It has been proved in [15] that there will be no deadlock if for every pair of nodes, at least one node in the pair acquires a lock from the other. In the proposed algorithm both Pi and Pj are in each other’s request sets. If they concurrently make requests, the request with higher priority is granted permission while the one with the lower priority is deferred. Liveness: The system makes progress towards the allocation of the resource. Liveness is guaranteed iff each process resolves conflicts among requests according to the same deterministic serialization discipline. Each process sends a grant message only to the request at the head of its queue which is sorted totally by the relation ≺r . This order is consistent in each process as (i) the entry for a request contains the same time-location stamp in each process queue, and (ii) the release message, which triggers the next grant message, is sent to all the processes and contains the same time-location stamp of the resource. Fairness: Each request will be eventually satisfied. This follows directly from the fact that ≺r serialization discipline is a total order.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

10

Let us finally remark that the distance between a process and a resource, as computed in the algorithm, may not be accurate. The time-location stamp informs a process about the location of a resource at time t, which is in the past. The resource may have moved between then and the present time t0 . The maximum possible movement is s×(t0 −t+2σ), where σ is the maximum drift of a clock. The algorithm described above can have two possible versions. In the optimistic version one can assume that the resource has not moved since reporting its location and the information available is correct. This is actually the version we implicitly use in the above algorithm. In the pessimistic version information about the separation between a process and a resource will be weighted by the freshness of the information. Greater the difference between t and t0 , lower the weight associated with the location information. So, the freshness factor will also be considered in queue management. 3.4

Mutual Exclusion in the Presence of more than one Resource

Let there be a set of n processes and m identical resources. All the processes and resources are mobile with an upper bound on speed equal to s. Multiple processes could be making their requests at the same time, and each process should preferably get the resource closest to it. If the same resource is equidistant from two processes that wish to use it, and the two requests are tied on the basis of their timestamp, the process with the lower identity has priority. An example application that matches this set of requirements is a taxi dispatch service. Persons calling for a taxi are equivalent to processes, taxis are resources, and the nearest available taxi should be dispatched to the customer without starving earlier requests that happen to be farther from taxis. Each process maintains m local queues: one per resource. Entries in a queue are sorted according to a variation of the ≺r relation which uses a space-time vector ST with only m components corresponding to the m resources. Component ST [`] represents the space-time coordinates of resource `. At any process, Pi ’s request will be ahead of Pj ’s request in resource k’s queue even if their timestamps are identical if the distance between Pi and k is smaller than the distance between Pj and k, based on the latest knowledge available through the space-time vectors carried by messages. The algorithm is as follows. The first part is executed when the process want to acquire the resource. Entry Code: (i) Pi sends a request((ti , li ), ST i ) message to all the processes. (ii) when n grant messages have been received for accessing the same resource k: for all resources x 6= k Pi sends a release(x, ST i , (ti , li )) message to all the processes, access the resource k (i.e., the request (ti , li ) has been satisfied) Exit Code: Pi sends a release(k, ST i , (ti , li )) message to all the processes. This part is executed upon receiving a message. When a granthh, (ti , li )i message is received from Pj (indicating permission to use resource h):

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

11

(i) if the request (ti , li ) has been already satisfied and no release(h, ST , (ti , li )) message has yet been sent, then Pi sends a release(h, ST i , (ti , li )) message to all the processes. When a requesth(tj , lj ), ST i message is received from Pj : (i) For each ` ∈ {1, . . . , m}, if ST [`].time > STi [`].time then STi [`] := ST [`] (ii) Pi inserts a copy of the request in each of its m queues ordered according to ≺r . In queue Qk requests are ordered using resource k’s location as the reference point (see definition 4.1). (iii) if the request is at the head of some Qk , where k ∈ {1, . . . , m}, Pi sends a grant(k, (tj , lj )) message to Pj . When a releasehk, ST , (tj , lj )i message is received from Pj : (i) For each ` ∈ {1, . . . , m}, if ST [`].time > STi [`].time then STi [`] := ST [`] (ii) the request (tj , lj ) is removed from Qk (iii) if a new request, say (tw , lw ), is now at the head of Qk and a message grant(k, (tw , lw )) has not yet been sent, then a grant(k, (tw , lw )) message is sent to Pw . To handle multiple resources, a process Pi piggybacks on request and release messages the m-component space-time vector of Pi . Note that when a request message is received, if more than one resource is available, grant messages are sent according to a closestresource-next policy. This will ensure that, if channels are FIFO, the first quorum to be formed, by a requesting process, will be the one of the closest resource. Then, once the process has allocated a resource, successive grant messages for accessing another resource will be immediately followed by release messages for that resource. A process may have issued multiple requests for resources. The grant message carries the identity of the request (time and location) to which it corresponds. This ensures a correct match between requests and grants, and avoids any confusion. Finally, the remarks of the previous section about a single resource system are also applicable to this algorithm and the correctness of this algorithm is actually an extension of the one presented in the previous section. The execution of this algorithm is equivalent to m concurrent executions of the single resource algorithm. Even if a process is granted permission to acquire multiple resources, in response to a request, the process actually uses only one instance of the resource. It releases the other resources immediately as if the critical section execution with those resources took zero time to complete. 3.5

Interval-Based Spatial-Temporal Serialization Discipline

In the above examples we use ≺r as the serialization discipline for the requests in queue Q to avoid starvation. This discipline orders by closest-job-first policy only requests which are issued at the same global virtual clock time. To improve the effect of allocation of the resource by closest-job-first policy, requests issued during successive virtual global clock ticks, belonging to a certain interval of time I, could be ordered on a closest-job-first basis. On the other hand, a very important issue that needs to be addressed is monotonicity. Elapsed time since the issuance of a request is monotonically increasing. So, a resource allocation algorithm can guarantee liveness if it uses only temporal ordering. However, as processes and/or resources can move, their mutual separation is not a monotonic function. A resource allocation algorithm that considers both spatial and temporal aspects

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

12

of resource-process relationships needs to employ a monotonic function of spatial-temporal coordinates. With this aim, we assume requests as the only relevant events of the computation and the time provided by the virtual global clock forms a continuous sequence of intervals I0 , . . . , Ii , . . . of duration k global ticks and I0 starts at a given tick known by all processes. i.e., all processes see the same sequence of intervals. As an example Figure ?? shows a sequence of intervals with k = 2 starting at time 0. We denote as eiw ∈ Iw0 (with w0 ≤ w) an event issuing a request produced by process Pi during the interval Iw0 . We can then define an interval-based spatial-temporal ordering (denoted ≺kr ) in the following way: Definition 3.2 Let eiw ∈ Iw0 and ejh ∈ Ih0 be any two events issuing requests, k be the duration of an interval, and let R be a reference point whose location is r, eiw ≺kr ejh iff: (1) w0 < h0 (2) w0 = h0 and |L(eiw ) − r| < |L(ejh ) − r| or (3) w0 = h0 and |L(eiw ) − r| = |L(ejh ) − r| and i < j If k is equal to one ≺kr boils down to ≺r . It is also evident that k can be used as a tradeoff parameter between location and time request ordering. Larger the value of k, closer this interval-based ordering will be to a pure closest-job-first ordering. As an example, Figure 1 shows the changes in the positions of three processes of Figure during interval I3 that includes clock ticks 4 and 5. If we assume Pi and Pj issue requests at time 4 while Pk at time 5, we have ej4 ≺2r ek5 ≺2r ei4 . Hence, if we consider the distributed mutual exclusion algorithm of the previous section where each process resolves conflicts between requests according to ≺kr , we get a new algorithm that guarantees safety, deadlock freedom and liveness. Moreover, as k is finite fairness is also guaranteed. This interval based discipline can be actually modeled and implemented as a gated batch queuing system [16] where intervals can thus be seen as fixed-length batches. In the GPS-based model, the end of an interval can be detected independently by each process passively listening to its virtual common clock.

4

Causality in Mobile Systems

In distributed systems the causal order between events is dependent on the happened before relation between these events. Location of a process is not a factor as its location is assumed to be fixed for the duration of the computation. However, as mentioned earlier, in mobile computations the cause-effect relation between two events will be influenced not only by their happened before relation, but also by their relative separation at the time of occurrence of the events. In this section we first review Lamport’s causality relation for fixed distributed systems, and extend it with the spatial component for mobile systems. 4.1

Lamport’s Causality Relation

The relation ≺ is “temporal” in the sense that it partially orders events according to their timestamp, but it does not take into account cause-effect relation. Causal ordering

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

13

of events in a distributed execution is based on Lamport’s happened-before relation ([8]) denoted →, stated as follows: Definition 4.1 Let e and e0 be two events in E, e→e0 iff one of these conditions is true: (1) e and e0 occurred in the same process with e first; (2) e = send(m) is the send event of a message m and e0 = delivery(m) is the delivery event of the same message; (3) there exists an event e00 such that e→e00 and e00 →e0 . Such a relation allows representation of a distributed execution as a partial order of events (E, →). It is easy to check that → ⇒ ≺. Note that the happened-before relation, as opposed to the partial order structure created by ≺, does take communication events into account by relating only events in distinct processes connected by a chain of messages. 4.2

The Spatial-Causality Relation

Space is also an important aspect to define cause-effect relationship. As an example, let a person be suspected of a crime executed at 10:00 pm and this person has an alibi at 11:00 pm. This person could have committed the crime only if the site of the crime and the alibi site are close enough to be reachable in one hour. Otherwise the person is innocent. In terms of causality, if the alibi event is included in the future s cone of causality starting at the crime event, then that person could be a suspect where s is the maximum speed at which the suspect can move. Figure 2 depicts, in a one-dimensional space, a situation in which the person is innocent. The person could be a suspect only if the alibi event would be included in the interval [a, b] Figure 2: Spatial-causality relation: absence of dependency relation. Processes may be executing events while moving at a maximum speed of s. This means that if a process Pi executes an event eiw at time tw , at a later time t0 the position of Pi is somewhere in a circle centered at L(eiw ) with a diameter of 2s(t0 − tw ). From the point of view of a process Pj , the uncertainty in the position of Pi will depend on Pi ’s event with the highest timestamp that causally precedes an event of Pj (the last event of Pi “seen” by Pj ). Let eiw be that event. The last known position of Pi with respect to Pj will then be L(eiw ). Let us now formally state the concept of uncertainty interval. Definition 4.2 Let t0 be the clock value at Pj and let L(eiw ) be the last known position of Pi with respect to Pj . An uncertainty interval, denoted Ij (eiw , t0 ), represents the region in which Pi will be positioned from Pj ’s viewpoint at time t0 , and is equal to a circle centered at L(eiw ) with a radius of s(t0 − tw + 2σ). The component of 2σ in the radius represents the maximum drift between two synchronized local clocks.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

14

In some applications the cause-effect relationships between events, as the one explained in the previous section, have to be “visible” only if processes producing the events could be within a certain distance of each other. For example, a process Pj would like to know at a certain time which are the processes within radius d of its current location. This is particularly useful when defining group membership in mobile systems [13]. Looking at its past causal relationships, process Pj can infer which processes could be in that space by computing the interval I related to each process. Let eiw and ejh be two events such that eiw → ejh , as shown in Figure 3 (for simplicity this figure is drawn assuming a one-dimensional space). At the time of execution of event ejh , process Pj knows that process Pi is somewhere in the uncertainty interval Ij (L(eiw ), th ). Of course, the larger the difference th − tw is, the larger the interval will be. Uncertainty can be reduced only if a more recent causal relationship, by means of a chain of messages, is established between an event of Pi (later than eiw ) and an event of Pj . Definition 4.3 A process Pi could be within distance d of process Pj at time t0 (from Pj ’s viewpoint), d,t0

denoted Pi u Pj , if Ij (L(eiw ), t0 ) and a circle centered at Lj (t0 ) with radius d intersect. The non-intersection of the two circles rules out the possibility of the two processes being within distance d of each other. However, the intersection between the two circles does not always imply that the two processes are within distance d of each other at time t0 . Definition 4.4 If Ij (L(eiw ), t0 ) is entirely within the circle with center Lj (t0 ) and radius d then the two processes are definitely within distance d of each other at time t0 . d,th

Figure 3 shows an example in which ¬(Pi u Pj ). We are now in the position to d,t

introduce the spatial-causality relation among events denoted →. This relation is based on the happened-before relation ([8]) with the addition of a spatial dimension: Definition 4.5 d,t Let d be an integer, t an instant of time and eiw and ejh be two events in E, eiw →ejh iff: (1) eiw →ejh , and d,t

(2) Pi u Pj This means that event ejh is a causal successor of event eiw and the node Pi , where event eiw occurred could possibly be within distance d of Pj at time t. Such a relation allows us to represent a distributed execution as a partial order of d,t d,t events (E, →). It is easy to see that → ⇒ →. It is interesting to observe that the d,t

structure of the partial order defined by the relation → changes with time (i.e., events d,t

d,t0

which are concurrent with respect to → can be related by → , with t < t0 , and vice versa). d,th

For example, in Figure 3, eiw is concurrent with ejh at time th with respect to the →

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

15

Ij (eiw , th ) ≡ [L(eiw ) − s(th − tw + 2σ), L(eiw ) + s(th − tw + 2σ)] X eiw L(eiw )

ejh

L(ejh )

tw

Ij (ejh , d)

th

t

Figure 3: Distance between process Pj and Pi at time th (case greater than d). relation even though eiw →ejh . In other words, a causal relation may be hidden to the spatial-causality relation for a certain period of time. It takes time for causal relations to propagate. Such relations become visible only if processes are close enough and/or enough time has elapsed. Let us d,t

finally say that if d is equal to infinite → boils down to →. 4.3

A Property of Space-Time Vectors

According to properties of Mattern-Fidge vector times [3, 11], we have: eiw → ejh iff STiw < STjh where STiw is the space-time vector associated with event eiw , and (∀`) :: STiw [`].time ≤ STjh [`].time STiw < STjh ⇔ and ∃y : STiw [y].time < STjh [y].time Hence, we can show that: d,t

d,t

eiw → ejh iff STiw < STjh where STiw < STjh d,t STiw < STjh ⇔ and d,t Pi u Pj

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering 4.4

16

An Implementation of Spatial-Temporal Ordering Constraint

Location stamping can be used to perform actions that are location and time dependent. For example, let node Pj originate a request to perform actions with one of the following types of location constraints: C1: If node Pi is within distance d of the position where Pj was located at the time of originating the request, Pi has to perform the action. C2: If node Pi is within distance d of Pj at the time of receiving the request, Pi has to perform the action. The request might have propagated from Pj to Pi either as a direct message or through a sequence of messages passing through other nodes. Let us assume that the following requirements need to be met: R1: The action should be performed by Pi at most once. R2: If Pi is causally dependent on action(s) of Pj subsequent to making this request, Pi should ignore the request. Here is how Pi can act on Pj ’s request: Let the message that brings Pj ’s request to Pi be m. Let Pj ’s event for the message send be msend and the corresponding space-time vector be STj,msend . Also, let Pi ’s event for the message receive be mrecv with the corresponding space-time vector being STi,mrecv .5 If STj,msend [j].time ≤ STi,mrecv [j].time then Pi ignores the request because otherwise requirement R2 will be violated. If STj,msend [j].time > STi,mrecv [j].time then, obviously, there is no causal dependency from Pj ’s request to Pi ’s state prior to the message reception. Let t be STi,mrecv [i].time, so, Pi proceeds in the following manner. 1. Pi updates its location vector in the manner described in Section 3.1. 2. If | STi,mrecv [i].location−STj,msend [j].location |≤ d (i.e., the request is constrained by C1) then Pi performs the action specified in the request. d,t

3. If Pj u Pi (i.e., the request is constrained by C2) then the following tests are performed by Pi : (a) If I(msend, t) is entirely within the circle centered at Li (t) and radius d then Pi performs the action. (b) If I(msend, t) and the circle centered at Li (t) and radius d intersect and one is not contained within the other, then Pi cannot be sure if it is within distance d of Pj . So, depending on whether an optimistic or a pessimistic policy is being employed, the action is either performed or the request is ignored, respectively. d,t

Else (i.e., ¬(Pj u Pi )) Pi does not perform the action. 5

Note that here STi,mrecv represents the space-time vector at Pi prior to its update using the space-time vector received in the message.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

17

As the request gets older the difference betwen STi,mrecv [i].time and STj,msend [j].time increases. As a result the probability that I(msend, t) is entirely within the circle centered in Li (t) and radius d declines. Therefore, we see a natural aging of requests. If a pessimistic approach is being followed, at most ds time units after a request is generated, it will become inconsequential.

5

Conclusion

Mobile computing applications require that both the order of occurrence of events, and the location of their occurrence, be taken into account during decision making. Thus, processes need to track the location of nodes and synchronize their clocks. Clock synchronization through message passing is an expensive operation and does not scale well. We have proposed using the Global Positioning System to accurately determine time, and to keep the local clocks of participating nodes in synchrony with each other. This does not require any message transmission by the mobile nodes and is energy efficient. We have also introduced the concept of space-time vector to track the mobility of nodes. Using this vector nodes can prioritize resource requests on the basis of request time as well as the requester’s distance from the resource. We have presented two distributed mutual exclusion algorithms that employ the space-time information. The intrinsic clock drifts in the mobile nodes introduce some asynchrony among their local clocks. This contributes to the error in the estimates of a mobile node about the region in which other mobile nodes may be present. We have presented a formulation that captures the magnitude of this error. We have also described how various resource allocation policies (optimistic versus pessimistic) react to such errors. References [1] K. Arvind. Probabilistic Clock Synchronization in Distributed Systems. IEEE Transactions on Parallel and Distributed Systems, 5(5):474–487, May 1994. [2] F. Cristian. A Probabilistic Approach to Distributed Clock Synchronization. Distributed Computing, 3:146–158, 1989. [3] C. Fidge. Logical time in distributed computing systems. IEEE Computer: 28–33, August 1991. [4] R. Gusella and S. Zatti, “The accuracy of the clock synchronization achieved by TEMPO in Berkeley UNIX 4.3BSD,” IEEE Transactions on Software Engineering, 15(7):847–853, 1989. [5] Hoffmann-Wellenhof, B. H. Lichtenegger, and J. Collins. GPS: Theory and Practice. 3rd ed.New York: Springer-Verlag, 1994. [6] H. Kopetz and W. Ochsenreiter, “Clock synchronization in distributed real-time systems,” IEEE Transactions on Computers, (8):933–940, 1987. [7] H. Kopetz. Sparse Time versus Dense Time in Real-Time Systems, Proceedings of the International Conference on Distributed Computing Systems, pages 460–467, IEEE press, 1992. [8] L. Lamport. Time, Clocks and the Ordering of Events in a Distributed System. Communications of the ACM, 21(7):558–565, July 1978. [9] T. Logsdon. The Navstar Global Positioning System. Van Nostrand Reinhold, New York, 1992. ISBN 0-442-01040-0.

R. Prakash and R. Baldoni /Causality and Spatial-Temporal Ordering

18

[10] L. Lundelius and N. Lynch. An Upper and Lower Bound for Clock Synchronization. Inform. Control, 62:190–204, 1984. [11] F. Mattern. Virtual time and global states of distributed systems. In Cosnard, Quinton, Raynal and Robert Editors, Proceedings of the International Workshop on Parallel and Distributed Algorithms, LNCS series, pages 215–226, North Holland, 1989. [12] A. Olson and K.G. Shin. Fault-Tolerant Clock Synchronization in Large Multicomputer Systems. IEEE Transactions on Parallel and Distributed Systems, 5(9):912–923, September 1994. [13] R. Prakash and R. Baldoni. Architecture for group communication in mobile systems. Proceedings of the Symposium on Reliable Distributed Systems, pages 235–242 IEEE press, 1998). [14] R. Schwarz and F. Mattern, Detecting Causal Relations in Distributed Computing: in Search of the Holy Grail, Distributed Computing, 7(3),149–174,1994. [15] M. Singhal. A class of deadlock-free Maekawa-type algorithms for mutual exclusion in distributed systems. Distributed Computing, 4:131–138, 1991. [16] H. Takagi. Queuing analysis volume 1. vacation and priority systems. North-Holland, 1991. [17] P. Verissimo, “Ordering and timeliness requirements of dependable real-time programs,” Journal of Real-Time Systems, Kluwer Eds., 7:105–128, 1994.