Bristle: A Mobile Structured Peer-to-Peer Architecture Hung-Chang Hsiao
[email protected]
Chung-Ta King*
[email protected]
Department of Computer Science National Tsing-Hua University Hsinchu, Taiwan 300
Abstract Most hash-based structured peer-to-peer (HS-P2P) architectures proposed previously focus on stationary peer nodes. When peers move to new locations in the network, these schemes become cumbersome and inefficient in delivering messages. This paper proposes a mobile HS-P2P architecture, called Bristle, which allows nodes to move freely. A node can disseminate its location information through the P2P network, and other nodes can learn its movement or query its location also through the P2P network. Consequently, an overlay infrastructure based on Bristle can be efficiently maintained over mobile nodes. The scalability, reliability and performance of the HS-P2P are thus leveraged. We believe that the concept proposed in Bristle can be applied to existing HS-P2P overlays.
1. Introduction Peer-to-peer (P2P) computing intends to provide a universal utility space that is distributed among the participating end computers (peers) across the Internet, to facilitate accesses from anywhere at anytime. All peers assume equal role and there is no centralized server in the space. Such a space is collectively referred to as the P2P architecture. The often-cited features of the P2P architecture include scalability, dynamic adaptation, permanency, fault tolerance, selfconfiguration and anonymity. The key to realizing these features lies in an effective way of naming and locating peers and data items. A common strategy is to use hashing [2], [8], [9], [12], [13], which represents peers and data items with unique hashing keys to control the topology and data placement in a P2P overlay network. Such a naming scheme not only protects the anonymity of nodes and data items but also prevents attacks such as denial of services. This is because there is no correlation among node key, data key, network location and topology, and data ownership. The resultant system is denoted as a hash-based structured P2P architecture (abbreviated as HS-P2P). A common strategy of naming and routing in HS-P2P is to store a data item with a hash key k in a peer node whose hash key is the closest to k. To fetch the data item, the request message is routed through the intermediate peers whose hash keys are closer and closer to k. This requires each peer node keep * This work was supported in part by the National Science Council, R.O.C., under Grant NSC 90-2213-E-007-076 and by the Ministry of Education, R.O.C., under Grant MOE 89-E-FA04-1-4.
some state information for routing. The state information may appear as a list of entries; each is called a state and associates the hash key of a known peer and its network address, e.g., the IP address and port number. A state gives the location information of a peer node in the network, which allows the local node to communicate with that node directly. We will call the tuple as a state-pair. When a node receives a message, it knows the next peer node to forward the message to according to the states that it maintains. Obviously, states in P2P architecture are distributed, and a robust and scalable HS-P2P relies on sophisticated state management. Previously proposed HS-P2Ps [2], [8], [9], [12], [13] assume that the participating peers are stationary. The problem is that when a node in the HS-P2P moves to another network attachment point, the state-pairs regarding this node that are distributed across the P2P network become invalid. A straightforward solution is to treat that node as leaving the HS-P2P and then joining as a new peer in the new location. The peers in the HSP2P periodically update their states to preserve the freshness. The old states associated with the mobile node can then be removed gradually from the system once their states expire. Such a scheme will be referred to as a Type A HS-P2P. Apparently, this approach cannot guarantee end-to-end semantics for applications running on top of it. The mobility of nodes also incurs extra maintenance overhead and unavailability of stored data. These all turn into degradation of robustness and scalability. It is possible to deploy a HS-P2P over a mobile IP infrastructure [6] (called a Type B HS-P2P). Mobile IP provides a transparent view of the underlying network to the HS-P2P. The HSP2P thus does not need to consider mobility. However, mobile IP assumes that home and foreign agents are reliable and administrative support is available. These agents may introduce critical points of failure and performance bottlenecks, and they require sophisticated management. In addition, the networklayer mobility provided by mobile IP cannot exploit the semantics of the applications. Perhaps the most serious problem with mobile IP is the triangular route that it introduces. Although mobile IPv6 [7] solves the problem by allowing a host to cache the binding of the address of the mobile node and the address of its network attachment point, it requires that the correspondent host be mobile-IPv6 capable. In addition, it still needs the help of a reliable home agent to resolve unknown network addresses of mobile hosts. Table 1 summarizes the design choices for handling node mobility in HS-P2P. Type A HS-P2P needs only the IP infrastructure support and can be more scalable, reliable, efficient and
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
Table 1. A summary of various design choices Type Infrastructure Scalability Reliability Performance Deployment End-to-end semantic
A IP Fair Fair Good Easy
B Mobile IP Poor Poor Poor Sophisticated
Bristle IP Good Good Good Easy
No
Transparent
Transparent
deployable than Type B HS-P2P, although it does not guarantee the end-to-end semantics for applications. Is it then possible to design an efficient HS-P2P that can preserve states for mobile nodes? In this paper, we propose a novel P2P architecture with mobility support, called Bristle, which can attain this goal. Bristle is based on the P2P infrastructure Tornado [2]. It classifies the participating nodes as stationary and mobile nodes. Stationary nodes form a location information repository, which is robust and is used to discover the states that a node may be interested. A straightforward routing scheme in Bristle requires O (log N )2 hops to send a message from its source to the destination, where N is the number of nodes in the system. Since a route between two stationary nodes may involve mobile nodes to help message forwarding, this thus may require O (log N ) address resolution operations for mobile nodes. We thus propose a clustered naming scheme, in which the routing of messages between two stationary nodes does not need the help of mobile nodes. This boosts the routing performance to O (log N ) . In addition to reactively performing location queries, a Bristle node can proactively receives the updated location of a mobile node via a location dissemination tree. The tree consists of the set of nodes that are interested in the movement of that mobile node. It is adaptive by taking workload and capabilities of participating nodes into consideration. We show that a mobile node can disseminate its new network attachment point to interested nodes in O (log(log N )) hops.
2. Bristle 2.1 Overview X
L o c a tio n U p d a te
P a c ke t F o rw ard in g
Z
M o b ile L a y e r
Suppose a node X in the HS-P2P would like to forward a packet to node Y that is the next node in the routing path. If Y’s network address is known to X, the packet can be directly sent to Y. However, if Y’s network address is unknown or invalidated, the system needs to (1) resolve Y’s address and (2) forward the packet. As shown in Figure 1, node X is in the mobile layer and does not know the network address of Y. X simply injects the packet into the stationary layer to resolve Y’s network address. The packet is routed to a node Z that knows the exact location of Y and how to forward the packet to Y. This is done by requiring the mobile node Y update (or publish) its state with the newly attached network point to the stationary layer. With HS-P2P, this is accomplished by sending the information to a node (i.e., Z) in the stationary layer whose hash key is the closest to Y’s. Other nodes in the mobile layer can also proactively “subscribe” to the mobile nodes of interest for updated network addresses. Note that the stationary and mobile layers are deployed in two separate HS-P2Ps. They, however, can share the number of nodes (N) that the two HS-P2Ps exploit. Note also that the advertisement operation performed by a mobile node is scalable, since most HS-P2Ps distribute a node’s state to O (log N ) nodes in the system.
2.2 Routing _route (node i, key j, payload d) // Does there exist a node closer to the designated key j? if ( ∃p ∈ state[i ] such that p.key is closer to j) // Is the network address valid? if (p.addr = null) or (p.addr is invalid) // Resolve the address for p.key and forward packets // by a node in location management layer p.addr = _discovery (p.key); else // node i forwards packets to node p.key _forward (p.addr, j, d); Figure 2. The routing in the Bristle mobile layer, in which node i forwards a packet towards a node with the hash key closest to j, where the state-pair p =
A d d re s s R e so lu tio n
Y
The stationary layer can be any HS-P2P, e.g., CAN [8], Chord [12], Pastry [9], Tapestry [13], Tornado [2], etc. It is responsible for resolving network addresses for mobile nodes in the mobile layer and handles their mobility.
S ta tio n a ry L a y e r
Figure 1. The Bristle architecture The Bristle architecture is depicted in Figure 1, which consists of two HS-P2Ps: the stationary layer and the mobile layer. The nodes in the stationary layer have fixed locations, while those in the mobile layer can change their network attachment points.
As aforementioned, a node’s state-pair in the mobile layer is represented as . The network address field may be invalid (denoted by null) if the network address is not resolved. Suppose a node X requests a data item with a hash key k and finds in its local list a state-pair , it then seeks the help of the stationary layer by sending a discovery message with the hash key k to a node, say Z, which can resolve the network address of k. Once Z determines the network address of k, said a, it forwards the message to the destination node Y to retrieve the data. In the mean time, Z replies the resolved network address to X, which then updates its local state-pair from to .
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
Figure 2 shows the routing operations in the Bristle mobile layer to forward a message in a node with the hash key i (denoted by node i) to request a data item with a hash key of j, where state[i ] denotes the set of state-pairs maintained by node i. Note that different HS-P2Ps have different definitions for the “closeness” in Figure 2. Also, a discovery message is only routed in the stationary layer.
2.3 Location Management Bristle supports four operations, namely register, update, join and leave, for location management of mobile nodes. As mentioned in Section 2.1, a node X, which is interested in the state of a mobile node Y, “registers” its interest to Y. When Y moves to a new network attachment point, it “updates” its new network address to the stationary layer and informs X of the change. Since Bristle allows nodes constituting the mobile layer to “join” and “leave” dynamically, it thus should maintain the robustness of the mobile layer1. Bristle organizes those nodes interested in a given Y into a location advertisement tree (denoted by LDT) in the mobile layer. The tree supports multicast communication for states advertisement. This allows Y to send an address update message to all the nodes in the tree. A LDT will dynamically change its structure according to the load of the participating nodes (described later). There are two unique features of Bristle by adopting LDTs: • •
Each mobile node is associated with a LDT. The number of members in a LDT is O (log N ) , i.e., when the size of a HS-P2P increases, the size of a LDT is increased in logarithmical fashion.
There are two design alternatives for a LDT: member-only or non-member-only. A member-only LDT consists only of Y and those interested nodes. A non-member-only2 LDT may contain other nodes in addition to Y and those interested nodes. Bristle uses the member-only LDT. Consider the non-member-only LDT protocol. In the worse case, the number of nodes (denoted by S (τ ) ) participating in a non-member-only LDT τ is the number of nodes from the tree leaves to the tree root (i.e., the average number of hops for a route from the leave to the root) times the number of tree leaves (i.e., the number of interested nodes). In a HS-P2P such as Chord [12], Pastry [9], Tapestry [13] and Tornado [2], S (τ ) = O (log N ) × O (log N ) . Assuming a Bristle with M mobile nodes ( M < N ), the total number of nodes in a LDT is 2 thus O (M × S (τ )) = O M × (log N ) . To prevent recursively resolving the network addresses of the forwarding nodes when advertising states from a LDT root to a LDT leave, the nonmember nodes can be elected from the other N − M nodes in
(
1
)
The nodes participating in the stationary layer may also join and leave dynamically. HS-P2Ps such as CAN [8], Chord [12], Pastry [9], Tapestry [13] and Tornado [2] have been designed with such a self-configuration feature. 2 A non-member-only LDT shares several similar aspects with the IP-multicast [5] and the Scribe protocols, which organize the tree by utilizing the nodes along the routes from the leaves to the root.
Figure 3. The responsibility value for varying M and N for the member-only and non-member-only approaches
(
)
the stationary layer. As a result, each stationary node should × (log N )2 handle the location information for O N M −M (called the responsibility) nodes in average. For a member-only approach, however, the responsibility is reduced to O N M− M × log N .
(
)
Figure 3 shows the responsibility values for the member-only and non-member-only approaches, where N is set to 1048576. is increased linearly (i.e., the number of It shows that when M N mobile nodes is linearly increased), the responsibility value of the non-member-only approach increases exponentially. On the other hand, the member-only approach can drastically reduce the responsibility for handing the mobility of mobile nodes.
2.3.1 Register and Update A node X of a HS-P2P sets up and maintains a list of states in two situations. (1) X newly joins the HS-P2P and collects a set of states from other nodes appeared in the same HS-P2P to construct its own states. (2) X becomes an active node and receives the periodical states refreshment advertised by other peer nodes. In either case, X registers itself to nodes whose state-pairs are replicated in X. As mentioned in Section 2.1, X performs the registrations to O (log N ) nodes. If M of N nodes are mobile, X can register itself to those mobile nodes only and × log N . Consethe number of registrations becomes O M N quently, the total number of registrations issued from all active × log N = O (M × log N ) . Each mobile nodes is O N × M N node thus updates its new network attachment point to O M × log N nodes, i.e., each mobile node only maintains N O M × log N members for the update. Note that O M log N N N < O (log N ) since M < N .
(
(
( (
)
)
) )
(
)
Note that when X registers itself to the nodes it is interested in, it also reports its capacity (denoted by C X ) to these nodes. The capacity represents a node’s ability, which can be the maximum network bandwidth, the number of maximum network connections, the computational power, etc. Scheduling the location update to the registry nodes is based on the capacity values collected by a mobile node. Figure 4 illustrates the location update algorithm for a mobile node i to update its new network address to the set of nodes
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
registered in node i (denoted by R (i ) ). The idea is that node i firstly estimates its remaining capacity. If it is overloaded, it only updates its new state to the registry node with the maximum capacity. Then, that updated registry node behaving as node i performs the state advertisement for the registry nodes that are not being updated. Otherwise, according to its available capacity (denoted by Availi ), node i advertises its state to the k nodes with the maximum capacities such that k × v ≤ Availi < (k + 1) × v (where v is the unit cost to send an update message).
_advertise (node i) // sort R (i ) in decreasing order list(i ) = sort R (i ) according to Ct ' s for ∀t ∈ R(i ) ; Availi = Ci − Used i ; // the remaining capacity of i if ( Availi − v ≤ 0 ) // node i is overloaded? // report node i’s location to the registry node with // the maximum capacity h = list (i ).head _send ( h , i’s network address, list (i ) − {h} ); else Availi // equally partition list(i ) to decreasing lists v empty) while ( list(i ) is notAvail for ( k = 1 ; k ≤ v i ; k = k + 1 ) h = list (i ).head ; partition(k ) = partition(k ) ∪ {h} ; list(i ) = list(i ) − h ; Avail i // update node i’s location to registry nodes v // with the maximum capacity Avail for ( k = 1 ; k ≤ v i ; k = k + 1 ) h = partition(k ).head ; _send ( h , i’s network address, partition(k ) − {h} );
¬
¬
¼
¼
¬
¬
¼
¼
Note that (1) in addition to receive the new network address, each of the k nodes also receives a disjoint subset comprising of the nodes registering in node i. The algorithm guarantees that the numbers of registry nodes of different disjoint subsets are nearly equal. Consequently, a LDT constructed by the algorithm shown in Figure 4 exposes the following features: •
It exploits the heterogeneity of nodes by accessing a node’s capacity.
•
A LDT associated with a mobile node is dynamically structured based on the participating nodes’ workloads. The workload depends on the consumption of a node’s local resources such as the network bandwidth and the memory used by other processes running on that node.
•
2.3.2 State Discovery Although a mobile node can broadcast its state to the registry N hops, a registry node may not receive the nodes in O loglog k updated location issued from the mobile node. This is because the registry node may also change its network attachment point. Each state stored in a Bristle node appeared in the mobile layer is thus associated with a time-to-live (TTL) value, which indicates the valid lifetime of the state. Once the contract of a state expires, the state is no longer valid.
(
To prevent skewing a LDT, a tree node tends to advertise states to the registry nodes evenly. This can greatly reduce the height of a LDT.
)
Based on the lease concept, two state binding approaches are used by Bristle: •
Early binding: Each mobile periodically publishes its state to the registry nodes and each registry node also periodically registers itself to the mobile node it interested in.
•
Late binding: During the movement of a registry node, it is possible that the registry node does not receive the state periodically advertised by the mobile node even though the registry node periodically registers itself to the mobile node. The registry node can thus issue a discovery message to the location management layer to resolve the network address of the mobile node.
¼
Figure 4. The state advertisement algorithm, where Used i denotes the present workload of node i, partition(k) ⊆ list(i) Avail for 1 ≤ k ≤ v i and * k partition(k ) = list (i )
¬
(2) In addition to advertise a node’s state to registry nodes, the node also publishes its state to the location management layer (this is not shown in Figure 4). Publishing states in the location management layer allows a registry node query the up-to-date states it does not receive (see Section 2.3.2).
To discover a node Y’s network address, a discovery message with the designated hash key of Y is issued (i.e., the _discovery operation shown in Figure 2) to the stationary layer. The stationary layer can adopt any of HS-P2Ps mentioned above that are designed with the scalability, reliability, availability and responsiveness in mind. (1) For scalability, each node in a HS-P2P takes O (log N ) memory overheads to maintain the states except O (log D ) in CAN [8], where each node needs to maintain 2 D neighbors. (2) For reliability, each node periodically monitors its connectivity to other O (log N ) nodes in the system except that a CAN node only uses O (log D ) connections. In addition, a route towards its destination can be adaptive by maintaining multiple paths to the neighbors. (3) For availability, a data item published to a HS-P2P can simply be replicated to k nodes clustered with the hash keys closest to the one represented the data item. Once one of these nodes fails, the requested data item can be rapidly accessed in the remaining k − 1 nodes. (4) For responsiveness, discovering a data item takes O (log N ) hops in these systems. CAN, however, needs O log N 1 D ones.
(
)
2.3.3 Join and Leave When a Bristle node joins and leaves, the overlay structure is changed since the state of a HS-P2P node needs to be hosted by “appropriate” nodes 3 to sustain the overlay structure and thus its effectiveness.
Ideally, if a LDT is a k-way complete tree, then perform a state
(
)
N hops. advertisement takes O loglog k
3
The nodes are varied according to the naming and routing protocol used.
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
_join (node i) // each k in the path traveled by i’s join message for (each node k visited) // can i become k’s neighbor? if ( ∃p ∈ state[k ] such that i.key is closer to k than p.key) state[k ] = state[k ] ∪ {i} ; // i becomes k’s neighbor _register (i, k); // i registers itself to k // can each of k and state[k ] become i’s neighbor? for (each r ∈ {k ∪ state[k ]}) if ( ∃q ∈ state[i ] such that r.key is closer to i than q.key) and ( distance(r,i ) < distance(q,i ) ) state[i ] = state[i ] ∪ {r} ; // r becomes i’s neighbor _register (r, i); // r registers itself to i Figure 5. A Bristle node joining operation Consider a node i joins Bristle. It publishes its state to O (log N ) nodes and then these nodes return their registrations to Y. Note that the LDT with the root node i is not yet constructed in this phase if node i is mobile. This at most takes 2 × O (log N ) messages4 sent and received by node i. Figure 5 presents the algorithm. Note that the newly joining node i will collect the states maintained by each visited node and determine whether each of these collected states (i.e., r in Figure 5) should be included into the set of states it maintains. This depends on the closeness of r’s and i’s keys, and the network distance between r and i. The network distance can be the measured network latency of sending a packet between r and i, 1 between r and i. Maintaining states representor the bandwidth ing the geographical closed nodes allows a node to exploit the network proximity. Consequently, a node can help forward a route to a geographical closed node in the next hop. As well as a typical HS-P2P, since a node may leave the system at any time, it needs to periodically refresh its state to the associated nodes to maintain the entire system’s reliability. This is accomplished by adopting the operations similar to those shown in Figure 5. The traffic overhead (i.e., 2 × O (log N ) messages) thus will be repeatedly introduced to the root of each LDT.
3. System-Dependent Optimization Bristle is implemented on top of Tornado [2]. A route between two stationary nodes in a HS-P2P normally takes O (log(N − M )) hops except CAN mentioned above. Unfortunately, in Bristle the number of hops between two stationary nodes can be up to O (log(N − M ) × log N ) since it is possible that the network address of each intermediate node visited in each hop needs to be resolved. The frequent “flip-flop” between the stationary and mobile layers will greatly degrade Bristle’s performance. Two potential optimizations can be performed to boost the performance of a route in Bristle. (1) Reduce the routing overhead of each hop by exploiting the network proximity. This is accomplished by forwarding the route to a neighboring node 4
The number of messages becomes O (log N ) + 1 if Pastry [9] or Tapestry [13] is used. A newly joining node in these systems simply sends a joining message to sequentially visit O (log N ) nodes.
whose hash key is closer to the destination and the cost of the network link to the neighbor is minimal. Although this optimization still needs O (log N ) hops to discover a requested state, each hop can greedily follow the network link with the minimal cost. (2) It is expected that communication between nodes in the stationary layer should reduce the help of nodes in the mobile layer. In Bristle, minimizing the frequency of flip-flop during a route between two stationary nodes is through clustering the hash keys identifying the nodes in the stationary layer. Figure 6 illustrates an example of that a stationary node a.b.c.d wants to perform a route towards another stationary node w.x.y.z. If hash keys are scrambled to each Bristle node, a route may frequently need state discovery for resolving network addresses of mobile nodes. As shown in Figure 6(a), two state discovery operations are required. However, if hash keys are assigned based on node mobility, a route can possibly utilize the paths comprising of stationary nodes (see Figure 6(b)). Mobile Node
Stationary Node
a.b.c.d
w.x.y.z d 2
d 4
d 8
Home Address Space
(a) a.b.c.d
w.x.y.z d' 4
d' 8
Home Address Space
(b)
Figure 6. (a) The scrambled and (b) the clustered naming schemes In Bristle, a stationary node is assigned with hash key k S such that 0 < L ≤ k S ≤ U < ρ , where L and U are the pre-defined system parameters. In contrast, a mobile node is designated to the key k M such that k M < L or k M > U . Let U −L U − L = ∇ × ρ , where ∇ = ρ ≈ N −NM . If ∇ decreases, the probability of a routing path comprising of mobile nodes will be increased and a route thus cannot exploit stationary nodes in its routing path (we assume that a uniform hash function such as SHA-1 is adopted). However, if ∇ enlarges, a route will try to exploit nodes in the stationary layer to help the forwarding. Consider a stationary node x1 sending a message towards another stationary node x2 . Note that x represents the hash key of x. If x1 < x2 , then the route must visit the nodes in the stationary layer and thus there needs not any address resolution. Otherwise, in the worst if the route can be forwarded by stationary nodes, then we have
§¨ x + 1 ©
ρ − x1 − x 2 2
·¸ mod ρ ≥ L . ¹
(1)
Equation (1) can be rewritten as x1 + x2 ≥ 2 × L + ρ . Since x1 ≤ U and x2 ≤ U , we can have ∇ = U ρ− L ≥ 12 . This concludes that in the worst case when the number of stationary
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
nodes is greater or equal to the number of mobile nodes, a route issued from a stationary node towards another stationary node can fully utilize stationary nodes only.
4. Performance Evaluation
30
Scrambled Clustered 25
20
Application-level Hops
We evaluate Bristle via simulation. In default, there are N − M = 2,000 stationary nodes and M = 0 ~ 8,000 mobile = 0 ~ 80% . The hash key representnodes simulated, i.e., M N ing each node is randomly generated and each Bristle node is randomly placed to the network modeled as the Transit-Stub topology. The topology is generated by the GT-ITM topology generator. The Transit-Stub topology models a 2-level hierarchy of routing domains, where the higher-level transit domains bridge the lower-level stub domains.
15
10
5
0
Note that we do not present the measurements for scalability, reliability and availability of Tornado that Bristle is based on. The details can be found in [2].
0
10
20
30
40
50
60
70
80
70
80
M/N (%)
(a)
4.1 State Discovery
Figure 7(a) presents the number of average application-level hops of each route for the scrambled and clustered naming schemes. Figure 7(b) depicts the relative delay penalty (RDP), where the RDP is the ratio of the average path cost (or the application-level hops) required by the scrambled scheme and those required by the clustered scheme. We can see that (1) the clustered naming scheme is superior than the scrambled scheme since the clustered scheme can greatly reduce the number of address resolution for those mobile nodes appeared in a route. (2) As we estimate in Section 3.2, when the number of stationary nodes is greater or equal to the number of mobile ≤ 50% ), the clustered scheme can sustain the nodes (i.e., M N performance of performing address resolution in the stationary layer (see Figure 7(b)). (3) The RDP ratios for application-level hops and the path costs are closed.
4.2 State Advertisement Figure 8 illustrates the performance of adopting LDTs for state advertisement. In this experiment, the capacity value used by the advertisement algorithm shown in Figure 4 is the number of network connections. Each node simulated is randomly assigned the number of available network connections from 1 to MAX, where MAX is 1,2,3,",15 . Note that since the maximum number of nodes simulated is 25,000, the average num-
3.0
Application-level Hops Actual Path Cost 2.5
Relative Delay Penalty
We firstly measure the performance of the stationary layer in Bristle (assume that a mobile node only advertises its updated location to the stationary layer). There are 10,000 sample routes between two randomly picked stationary nodes generated, and the average application-level hops and the path costs for these routes are averaged. Consider a particular route (denoted by r), the path cost takes the path weights into considerap (r ) q ( r , j ) tion from the source to the destination, i.e., ¦i =1 ¦ j =1 Wij , where p (r ) is the number of application-level hops for the route r. q(r, j ) is the path length for the j-th hop of the route r. Wij is the path weight of the j-th path of the i-th hop in the route r. Note that the shortest path weight for the i-th hop of the route r is calculated by adopting Dijkstra’s shortest path q (r , j ) algorithm, i.e., ¦ j =1 Wij is minimal for the i-hop of the route r.
2.0
Knee
1.5
1.0
0
10
20
30
40
50
60
M/N (%)
(b) Figure 7. (a) The number of application-level hops and (b) the relative delay penalty measured for the scrambled and clustered naming schemes ber of nodes interested in a given node’s location is thus ªO (log 25,000)º = 15 , i.e., the maximum MAX value. Different numbers of the available network connections nodes have represents different workloads experiences by nodes. We measure all LDTs in the system. Figure 8(a) presents the tree structures according to the maximum number of the network connections a tree node have in each tree. Tree nodes in different tree levels are labeled with different colors (the tree root is the level-1 node of a tree). Obviously, a LDT tree is dynamically structured according to each tree node’s workload and thus a LDT tree’s depth is varied from 1 to 14. When each node in a tree encounters heavy workload, the tree depth becomes lengthened. In contrast, the tree depth is reduced if each tree experiences light load and thus a node’s updated location can be rapidly propagated to nodes in a few hops. Figure 8(b) further explores how state advertisements are evenly distributed to each tree node of a LDT and how a LDT exploits node heterogeneity to accelerate updated state propa-
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
6
Average Per Tree Per Edge Cost (x10 )
105
100
Percentage (%)
80 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
60
40
20
100 95 90 85 80 75 70 65 60
With Locality Without Locality
55
0 0
1
2
3
4
5
6
7
8
9
0
10 11 12 13 14 15 16 17
10
20
30
40
50
60
70
80
90
100
M/N (%)
Maximum Available Capacity
Figure 9. The relative delay penalty of the LDT tree without and with network locality
(a)
Quantity
Number of Nodes Assigned
Available Capacity
15 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0 10 5 0
C = 15 C = 14 C = 13 C = 12 C = 11 C = 10 C=9 C=8 C=7 C=6 C=5 C=4 C=3 C=2 C=1 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Node ID (in Decreasing Order According to Available Capacity)
(b) Figure 8. (a) The effect of dynamic LDT adaptation and (b) the effect of node heterogeneity, where “Number of Nodes Assigned” for node ID i (from 1 to 15) is the number of elements in the set of partition(i) gation. There are 15 trees randomly picked from the simulation. Each is comprised of nodes with different capabilities (i.e., the number of available network connections). A node with ID equal to 1 represents a tree root. Firstly, we can observe that a LDT can exploit the super nodes in the tree to help perform state advertisement (see the gray bar in Figure 8(b)). Secondly, giving a set of state advertisement tasks, they can be evenly distributed to super nodes (see the dark bar).
4.3 Advertisement and Network Proximity
Exploiting the network proximity can greatly help a route in a HS-P2P. Tornado where Bristle is based on presents such a feature and thus the state discovery can benefit such support. The interesting readers can further refer to the results reported in [2]. In the experiment, we present how the network proximity impacts the overhead of state advertisement in Bristle. Each node simulated is randomly assigned the number of available network connections and the maximum number of connections is M t (i , j ) 15. The costs of all LDTs (i.e., ¦i =1 ¦ j =1 Eij ) in the network are measured and the average cost per tree per edge, (i.e., M ¦iM=1 ¦tj(=i ,1j ) E ij M ¦k =1 t (i , j ) ) is calculated, where t (i, j ) denotes the number of edges of the i-th LDT tree and E ij is the j-th edge cost of the i-th LDT tree. Note that given the j-th edge in the i-th LDT tree, Eij is the minimal sum of path weights for the network links assembling the edge.
((
) )(
)
Figure 9 presents the simulation results, where Bristle nodes are dynamically increased and randomly assigned to a network comprising of 10,000 nodes. We can see that (1) a LDT can be constructed by exploiting network locality. This can reduce the cost of a LDT. (2) The performance of a LDT without considering locality will be rapidly degraded since nodes comprising of a LDT may sparsely scatter in the network. This is independent of the number of mobile nodes increased. (3) When the number of mobile nodes is increased, LDTs with locality allow a newly joining node has the greater alternative in picking those nodes it is interested in. Those picked nodes have relatively closer network distance to the newly participating node. This gradually improves the quality of a LDT. A node had joined Bristle can periodically re-perform joining operations to refresh its local state and registrations to those nodes it is likely interested in.
5. Related Works Mysore et al. proposed to use the IP multicast infrastructure to handle Internet host mobility [5]. A mobile host is uniquely identified by a multicast IP address. Packets destined to the mobile host are routed to the address by using the IP multicast
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
infrastructure. Bristle can though adopt the IP multicast infrastructure to support node mobility for a HS-P2P. The scalability of a HS-P2P, however, is restricted by the available multicast IP addresses. In addition, due to lack of widely deployment of IP multicast, a HS-P2P cannot rapidly support node mobility based on IP multicast. In contrast to rely on IP multicast to provide HS-P2Ps with mobility support, Bristle distribute the location management by exploiting resources in endhosts while guaranteeing reliability and scalability. This allows Bristle to be rapidly and incrementally deployed by using ubiquitous Internet infrastructure only without the sophisticated support of IP multicast. Rather than handling host mobility in the IP network layer, Maltz et al. proposed to leverage mobility management in transport [4] and application-layer [10] in end systems, respectively. Handling mobility in transport- and application-layer allows distinguishing packets from various sessions, i.e., each packet will not be identically treated. The feature enables an application to further perform additional operations such as filtering over various communication streams. [4] needed to modify the standard TCP protocol and [10] relied on the tailored DNS infrastructure. Bristle, however, manages the mobility in end systems’ application-layer with the rudimentary networking protocols (i.e., TCP/IP). Li et al. proposed an architecture called Grid to provide GPS location information for aiding routes in a wireless ad hoc networking environment [3]. Grid pre-organizes geographical location regions in a hierarchical fashion and distributes mobile nodes’ location to each mobile node in the system. To maintain location information for each mobile node in a dynamic mobile environment, Grid makes the following assumptions: (1) each mobile node is GPS-capable. (2) Each node is aware of its location in the hierarchy. (3) The hierarchical region where a mobile node appears should be identified in advance. (4) Each node is aware of the identities of other nodes appeared in the same geographical unit-region. The above assumptions allow a mobile node to update its location and perform the location query to the mobile nodes in the geographical regions following the hierarchy. In contrast, Bristle does not structure nodes participating in the location management layer via auxiliary information such as GPS location and it poses no assumptions that each node participating in the location management layer is aware of its position in the network. Recently, an infrastructure i3 [11] based on Chord [12] was proposed, which generalize point-to-point communication using rendezvous abstraction to provide services such as multicast, anycast and mobility for Internet. To send a packet to a mobile node, the sender simply injects the packet to i3 towards a rendezvous point that helps forward the packet to the receiver. Once the receiver changes its network attachment point, it updates its location to the rendezvous point. Note that, the sender and receiver are not the constituent members of i3. The mobility support by i3 is primarily for the non-i3 members. Unlike i3, Bristle intends to extend a HS-P2P with mobility capability for participating members assembling the HS-P2P.
6. Conclusions In this paper, we propose a design for a mobile HS-P2P, called Bristle. Bristle allows nodes to dynamically change their network attachment points without re-associating new states. The old state of a node can be retained. This leverages the entire system reliability, scalability and performance. Applications based on Bristle can thus guarantee the end-to-end semantics. Bristle decouples stationary and mobile nodes into the stationary and mobile layers, respectively. Each mobile node is associated with a LDT that can rapidly update its state to those nodes that are interested in its movements. Nodes can also reactively discover missing states with the help from the stationary nodes. We show that a node can rapidly advertise its updated state to those nodes interested in its movement in O (log(log N )) hops via the associated LDT. A LDT is self-organized and can adapt to node heterogeneity and workload. It can evenly distribute the task of location update to those nodes that are relatively powerful. LDTs are locality-aware, i.e., nodes in a LDT tend to have closer network locations. We have also presented that proactive state discovery can be efficiently supported by the clustered naming scheme. This naming allows a discovery request to be routed in the stationary layer only if the number of stationary nodes is greater than or equal to the number of mobile nodes. This can greatly re2 duce the address resolution overhead from O (log N ) to O (log N ) .
References [1]
[2] [3]
[4] [5]
[6] [7] [8] [9]
[10] [11] [12] [13]
I. Clarke, O. Sandberg, B. Wiley, and T. W. Hong. “Freenet: A Distributed Anonymous Information Storage and Retrieval System,” In Workshop on Design Issues in Anonymity and Unobservability, pages 311-320, July 2000. H.-C. Hsiao and C.-T. King. “Capability-Aware Peer-to-Peer Storage Networks,” In Int’l Parallel and Distributed Processing Symp., April 2003. J. Li, J. Jannotti, D. De Couto, D. R. Karger, and R. Morris. “A Scalable Location Service for Geographic Ad Hoc Routing,” In Proc. of the ACM/IEEE Int’l Conf. on Mobile Computing and Networking, pages 120-130, Auguest 2000. D. A. Maltz and P. Bhagwat. “MSOCKS: an architecture for transport layer mobility,” In IEEE INFOCOM, pages 1037-1045, March 1998. J. Mysore and V. Bharghavan. “A New Multicasting-based Architecture for Internet Host Mobility,” In Proc. of the ACM/IEEE Int’l Conf. on Mobile Computing and Networking, pages 161-172, September 1997. C. E. Perkins. IP Mobility Support. RFC 2002, October 1996. C. E. Perkins and D. B. Johnson. “Mobility Support in IPv6,” In ACM SIGCOMM, pages 27-37, August 1996. S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker. “A Scalable Content-Addressable Network,” In ACM SIGCOMM, pages 161-172, August 2001. A. Rowstron and P. Druschel. “Pastry: Scalable, Distributed Object Location and Routing for Large-Scale Peer-to-Peer Systems,” In Proc. of the IFIP/ACM Int’l Conf. on Distributed Systems Platforms (Middleware 2001), Nov. 2001. A. C. Snoeren and H. Balakrishnan. “An End-to-End Approach to Host Mobility,” In Proc. of the ACM/IEEE Int’l Conf. on Mobile Computing and Networking, pages 155-166, Auguest 2000. I. Stoica, D. Adkins, S. Zhuang, S. Shenker, and S. Surana. “Internet Indirection Infrastructure,” In ACM SIGCOMM, August 2002. I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. “Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications,” In ACM SIGCOMM, pages 149-160, August 2001. B. Y. Zhao, J. D. Kubiatowicz, and A. D. Joseph. “Tapestry: An Infrastructure for Fault-Tolerant Wide-Area Location and Routing,” Technical Report UCB/CSD-01-1141, April 2000.
0-7695-1926-1/03/$17.00 (C) 2003 IEEE
Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’03)
