Non-Euclidean Internet Coordinates Embedding Alexander Allan, Ross Humphrey and Giuseppe Di Fatta School of Systems Engineering The University of Reading Whiteknights, Reading, Berkshire, RG6 6AY, UK
[email protected],
[email protected],
[email protected]
Abstract—For many applications it is desirable to be able to estimate latency in a decentralised network when it is not practical to explicitly measure it. It has previously been shown that latency can be approximated by assigning hosts coordinates in some geometric space such that the Euclidean distance between two hosts in this space is equivalent to latency, a method known as a Network Coordinate (NC) system. This is commonly achieved by a large scale distributed optimisation which seeks to minimise the error between latency and Euclidean distance. In this work we challenge the assumption of Euclidean space as a satisfactory model for embedding Internet-like networks, due to the curved nature of network distances. We present a novel distributed optimisation methodology: Non-Euclidean Internet Coordinates Embedding (NICE). NICE uses a polynomial regression model to explicitly learn the most effective distance function for latency estimation within a geometric space, in addition to a distributed non linear dimensionality reduction method. Dimensionality reduction is achieved via a variant of Landmark Multi Dimensional Scaling (LMDS) and a distributed optimisation algorithm. This allows the distributed system to create a set of coordinates for each of the participating hosts that can be used to accurately estimate latency. The system is implemented within the Java based PeerSim network simulator using both real and artificially generated input topologies and then compared to two of the most widely implemented NC systems: GNP and Vivaldi. By experimental simulation we show that NICE is significantly more accurate than either method while still remaining robust in the face of real network conditions. Index Terms—Non-Euclidean Distance; Multidimensional Scaling; Regression; Network Coordinates;
I. I NTRODUCTION Network Coordinate (NC) systems provide an effective way of exploiting locality in a decentralised network by allowing two nodes to estimate the latency between each other without the need to send a ping request. This brings benefits to many diverse applications including efficient routing in overlay topologies [1], reducing the load on ISPs due to Peer-to-Peer (P2P) networks [2], geographical location based services for social media [3] and disaster reporting [4]. NC systems are a form of decentralised global optimisation which attempts to build a space within which the distances between hosts represent latency. Most NC systems work by randomly or heuristically assigning initial coordinates to hosts and then attempting to minimise the error in estimated latency (usually the Euclidean distance) and the measured latency between pairs of hosts. Two of the most widely implemented NC systems are GNP [5] and Vivaldi [6] which both use Euclidean distance
to represent latency between sets of coordinates. However a fundamental inaccuracy is introduced when using the Euclidean model for representing Internet-like networks, due to the centralised topology of the Internet which can cause distances to be curved. This is due to packets needing to be routed through the centre of the network. If two nodes are geographically adjacent but on different ISPs for example, the packets from the sender will have to travel in towards the network centre, through each ISPs Border Gateway Protocol (BGP) then back out to the receiver [7]. It is due to this property that the application of nonEuclidean distance metrics have been considered in an attempt to more accurately represent this curved nature of the network space [8]. In particular Hyperbolic space, in which lines curve towards the origin, has been applied to Vivaldi, with the result that it improved accuracy for short distances only to decrease it for longer distances [9]. In this work we build upon the notion that Non-Euclidean distances can be used to accurately represent latency and try to explicitly model the error between Euclidean distance and latency using a polynomial function. This idea is used as a basis for a novel NC system, Non-Euclidean Internet Coordinates Embedding (NICE), that calculates this distance model over a sample of nodes. NICE then uses the distance model as part of a distributed dimensionality reduction and optimisation function to create a set of coordinates for every host in a large scale distributed network that can be used accurately and reliably to estimate latency. In order to test the validity of the method it was implemented as a protocol in PeerSim (a Java based network simulator) [10] and tested on a variety of real and simulated network topologies in realist network conditions. To provide comparison, GNP and Vivaldi are implemented alongside NICE and a variety of tests are conducted to establish the effectiveness of the three NC systems. Two measures were used: Relative Rank Loss (RRL) and relative error, which measure how accurately node and neighbourhood orderings are preserved respectively. We find that for all topologies tested, in terms of both RRL and relative error, NICE outperforms GNP and Vivaldi significantly. Additionally we found that NICE and Vivaldi were also able to maintain accuracy during realistic network conditions which was not the case for GNP. The rest of the paper is organised as follows: section II gives an overview of related work in this field and section III
explains the difficulty in using Euclidean space to represent Internet-like networks. Section IV outlines the method used in the NICE protocol, section V gives details of the experimental simulation and section VI gives the results. Conclusive remarks are given in section VII. II. R ELATED W ORK NC systems have existed since the early 2000s and tend to be either semi centralised approaches which require a subset of nodes to be used as landmarks or decentralised approaches which do not. Both types of NC system predominantly employ the Euclidean distance function between vectors of Cartesian coordinates at each node. A. Euclidean NC systems Global Network Positioning (GNP) [5] is the archetypical NC system on which several subsequent landmark based approaches were modelled: NCS [11], ICS [12], Lighthouses [13] and Virtual Landmarks [14]. GNP requires n nodes be elected landmark nodes and measure the round trip time between each other. The value of n used in the GNP paper was 15. These nodes then choose random points in k-dimensional space and run a simplex downhill function to minimise the squared error between the nodes Euclidean distance and the measured latency for every pair of nodes. When subsequent nodes join the network they measure their latency to these landmarks and use simplex downhill to find their coordinates by minimising the sum of squared error between Euclidean distance and the latency to each landmark. GNP has been incorporated into the Google Content Distribution Network (CDN) and has been shown to correctly choose the closest replica 86% of the time when conducting client-to-replica redirection [15]. Costa et.al.’s 2004 approach, Practical Internet Coordinates (PIC), [16] is similar to GNP in that it uses a simplex downhill algorithm but differs in that it allows any node to act as a landmark. It could be thought of therefore as a decentralized version of GNP. When a node joins the network it updates its network coordinates in the same manner as GNP except instead of a fixed set of landmarks it takes a sample of nodes (the selection of which is biased to have a proportion of neighbours and a portion of random nodes) which act as landmarks. Dabek et.al’s 2004 Vivaldi algorithm [6] is also similar to GNP, except it requires no landmarks and is entirely distributed. It uses a three dimensional Euclidean coordinate space to attempt to predict the Round Trip Times (RTT) between nodes. Nodes periodically ping other nodes, and when this happens, they update their own co-ordinates using the error between their real and predicted RTTs. Vivaldi was inspired by a 3-d mass-spring system and the distributed function to update coordinates is based on Hooke’s law. Vivaldi is to our knowledge the most widely implemented NC system being used in Peerwise Overlay [17], Census [18], Bamboo DHT [19], Stream-Based Overlay Network (SBON) [20] and the Azureus BitTorrent client [21]. This is likely due to its
robustness against node failures and its simplicity which it achieves while still maintaining comparable accuracy to GNP. B. Non-Euclidean NC systems Vivaldi has been extended to work in hyperbolic space [9]. It is found that hyperbolic distance is more accurate for estimating short latencies than Euclidean distance but less accurate for estimating large (> 100ms) latencies. A heuristic T hresholdHyperbolic, is proposed, which uses both Euclidean and hyperbolic distances to try and get the best of both worlds. Chen et al.’s 2011 work [22] Phoenix is the latest in a line of newer approaches to network distance estimation which uses matrix factorisation in lieu of Euclidean distance. Previous work included IDES (2006) [23] and DMF (2010) [24]. The intuition behind matrix factorisation lies in the fact that a distance matrix can be approximately factorised by two smaller matrices using a technique such as Singular Value Decomposition (SVD). The main advantage over Euclidean space methods is that matrix factorisation does not require the triangular inequality to hold, so can represent distances where the route A-B-C is shorter than A-C. III. N ON -E UCLIDEAN PROPERTIES OF I NTERNET- LIKE NETWORKS
Many NC systems are built on the basic premise that network latencies can be accurately modelled by an embedding in a Euclidean space. As Lumezanu and Spring [9] and Shavitt and Tankel [8] argue, this is an inaccurate assumption. The basis of this argument stems from the fact that in most networks, messages tend to get routed through the center of the network. This means hosts communicating with each other on two different tendrils of the network will have to send packets towards the centre and back out again in a majority of cases. This can be illustrated by Figure 1 which shows a simple network of nodes distributed on a 2 − d Euclidean plane, with latencies x and y between them. If a straight line (i.e. the Euclidean distance) were to be drawn between A and C, it would be a fair approximation of the latency. Draw the same line between A and D and the latency will be underestimated considerably. Lumezanu and Spring (2004) and Shavitt and Tankel (2008) have suggested that hyperbolic distances are more appropriate as they curve towards the origin and so are able to provide better estimates of the latency. This is visualised via the dotted lines on Figure 1. However in the case of Lumezanu and Spring they find that, while it is more accurate for estimating small latencies, hyperbolic embedding is actually worse for latencies above a certain threshold. Shavitt and Tankel find that hyperbolic embeddings are better than Euclidean for estimating number of hops, but find there is no clear advantage in using them for estimating latencies. Both approaches find that there is a large amount of sensitivity to the hyperbolic curvature parameter, which defines to what extent distances curve towards the origin.
D
C
x
x y
y
y y
x
x
(r2 ) of the regression model has converged. The final model is stored at each landmark. The landmark nodes then accept requests for a data bundle (Table I) containing the landmark MDS embeddings, landmark NICE coordinates, landmark latency matrix and the regression model to the remaining nodes in the network. A node computes an initial estimate of its embedding as determined by Landmark Multi Dimensional Scaling (LMDS) given the landmark embeddings and distance matrix [26]. It then updates these initial coordinates by minimising the error between the output of the regression model and the measured latency to each of the landmarks. The distance between any two nodes can then be estimated via taking the Euclidean distance between their coordinates and applying the regression model. A. Landmark Initialisation
However while both sets of authors to some extent inspired this work, it can be argued that representing distances with a hyperbolic distance function is no different than doing so with a Euclidean distance function. They are both likely to introduce error by attempting to generalise a space which is neither Euclidean or hyperbolic. The approach taken in this paper is to derive a distance function which best represents the underlying topology by explicitly modelling the error between Euclidean distance and latency in a sample of the network. A high order polynomial regression model is used in order to learn the contortions of the specific network space distances are estimated in.
The network needs to be bootstrapped by randomly choosing a subset of m nodes to act as landmarks L. The more landmarks are chosen the more accurate and robust the embedding will be, however NICE will work provided there are at least more than twice k landmarks, formally: |L| = m, m > 2k. This is due to the number of positive eigenvalues needed by the LMDS method. These initial landmarks ping each other and distribute a latency distance matrix between themselves. This distance matrix is used at each landmark to compute a k dimensional MDS embedding space, lM DS . The number of dimensions, k, was chosen to be five as this had been suggested to be optimal for embedding Internet-like topologies [27]. Each landmark determines a set of 6th order polynomial regression coefficients, B, using the least squares method. The order of the polynomial was determined experimentally, with order 6 yielding the most accurate results. This polynomial model attempts to describe the relationship between Euclidean distance δ(i, j) and measured latency d(i, j) for each pair of landmarks li , lj ∈ L. If x = δ(i, j) then this regression model takes the form:
IV. N ON -E UCLIDEAN I NTERNET C OORDINATES E MBEDDING
d0 (i, j) = β0 + β1 x + β2 x2 + β3 x3 + β4 x4 + β5 x5 + β6 x6 (1)
A
B
Fig. 1. A simple 8 node network embedded in 2−d Euclidean space. If x = 3 and y = 1, the network distance d(A, C) is ≈ 14% greater than network distance d(A, D). The Euclidean distance δ(A, C) however is ≈ 41% greater than δ(A, D). This demonstrates the error introduced by modelling networks in Euclidean space.
Non-Euclidean Internet Coordinate Embedding (NICE) is a novel NC system based on learning a non-Euclidean distance function. NICE is fundamentally a landmark based approach which provides node coordinates via two stages. In the first stage the set of landmarks measure a latency matrix between themselves and use classical Multi Dimensional Scaling (MDS) [25] to produce an initial k-dimensional embedding. This initial embedding is used to build a polynomial regression model between pairwise distances in the embedding and the measured latencies in the distance matrix. Each landmark then adjusts its coordinates by minimising the difference between the latency estimated via the regression model (with the input to the model being the Euclidean distance) and the latency. The regression model is then rebuilt using the new updated embeddings to give an updated model. This process is iterated until the coefficient of determination
Each landmark, i, in the set of landmarks L then updates its initial coordinates, lM DS , by minimising the following objective function: f (i) =
X
(d0 (i, j) − d(i, j))2
(2)
j∈L\{i}
To minimise the function, a line search algorithm is used [28]. When all landmarks have updated their coordinates, the regression model is recalculated using the new positions of the landmarks which results in a higher coefficient of determination (r2 ). The resulting set of coefficients, B, is then used in equation (1) to provide a more accurate estimation within objective function (2). This process of minimisation and regression is then iterated until the r2 has converged on
Data: Latency matrix between landmarks ∆, embedding dim k. Result: Regression model d0 , landmark coordinates l. t = convergence threshold; = smallest representable difference; l = computeM DS(∆).getT opKEigenV alues; 2 2 while abs(r2 − r−1 ) ≥ (t × × abs(r−1 ) do 0 d = computeP olynomialLeastSquares(∆, l); for each landmark in l do update l by minimising eq. (2) end r2 = calculateCoef f icientOf Determination(d0 ); end Algorithm 1: Landmark initialisation (performed independently at each landmark)
some value (see algorithm (1)). At this point the landmarks are embedded in a space which most accurately estimates latency given a polynomial function of Euclidean distance. The final landmark coordinates l can now be used to create a 1-d node index via Hilbert curve, similarly to [27]. B. Non-Landmark Setup After the initial landmark setup phase, any non landmark node can calculate its NICE coordinates in the following steps. A landmark (or any other node which has already calculated its NICE coordinates) is contacted and provides the node with a distance matrix between all landmarks, the landmark MDS initial embeddings, the landmark NICE coordinates and set of regression coefficients B. The node will then measure the latency to all landmarks and create a vector of latencies. This can then be used to create an initial estimate of its coordinates by performing LMDS in the same manner as [27]. This uses the Nystr¨om approximation of the eigenvectors and eigenvalues of a matrix to estimate a position in the landmark MDS embedding space [29]. As shown in algorithm (2), given these initial coordinates, the vector of distances to landmarks, the regression coefficients B and the set of landmark NICE coordinates, the node can minimise objective function (equation 2) using a line search algorithm. This yields the nodes NICE coordinates and allows it to create a Hilbert node index. Data: Data bundle P (see Table I), embedding dim k, set of landmarks L. Result: Node coordinates c. vector g = latency from i to each landmark in L ; c0 = computeLM DS(∆, lM DS , g); c = minimise eq (2) to L with starting val c0 and inputs l, g; Algorithm 2: Initialisation for a node i
C. Landmark Replacement Mechanism As nodes join and leave the network, landmarks may go offline and so be unavailable. NICE handles this by using the
Notation ∆ lM DS l d0
Description Landmark latency matrix MDS embedding of landmarks Coordinates of landmarks Regression model
TABLE I C ONTENTS OF DATA BUNDLE P
nearest neighbour of a missing landmark as a substitute. This is made possible by the using the node index of every host within the network as the key to a DHT (in this implementation CHORD is used [30]). When a missing landmark is encountered by a node, the DHT is queried for the k nearest online nodes. This list of IDs is then used to estimate the nearest neighbour. This is achieved by converting the missing landmarks ID and its k nearest online neighbours in CHORD’s IDs from a 1-d integer to an array of 5d coordinates. This is made possible by the properties of the Hilbert curve, as Hilbert curve functions are fully bijective. Using these coordinates the latency from the missing landmark to its k neighbours can be approximated using the NICE NC system. The closest node to the missing landmark is chosen to be the substitute. The data which is needed for node setup can also be given from this node, providing it has completed its setup itself. This method of handling landmark failure has some desirable properties. The accuracy of the estimation increases with the number of nodes in the network (n). This is due to the probability of a node falling close to missing landmark increasing. This method also allows the network to keep functioning even if 100% of the landmarks have failed without significant drop in accuracy. The number of nearest neighbours to check, k, is a tuning parameter. Increases in k were proportional to increase in accuracy until k = 10 at which point there was little measurable increase. V. E XPERIMENTAL A NALYSIS The following section outlines the experimental comparison conducted between the NICE, Vivaldi and GNP NC systems. All three systems were implemented in the Java based P2P simulator PeerSim [10] with an underlying BRITE or KING routing topology. The simulator was run with a fixed churn fraction to monitor the effects of nodes joining and leaving on the effectiveness of the NC systems. This effectiveness was established by periodically applying two quality metrics on the current state of the network: Relative Error and Relative Rank Loss (RRL). Optimal values for tuning parameters per topology for all three NC systems were based on the values used by the respective authors and by feedback from results. The simulation was run over all topologies 5 times with different random seeds. A. Simulation Topologies Two different types of topologies were used for the basis of the PeerSim simulator: Generated BRITE topologies and
measured KING topologies. 1) BRITE: The Internet topology simulator BRITE [31] was used to generate several different topologies using the Waxman connectivity model [32]. Topologies were created that simulated single Autonomous System (AS) and multi AS hierarchical networks with Internet-like properties. The size of the networks ranged from 1000 to 10,000 nodes. 2) KING: The KING measurement technique allows a distance matrix between DNS servers to be created [33]. Two such distance matrices were used in the experimental comparison. Set ”mking-t” which is used in the original Vivaldi paper and another additional set ”uking-t”. Both sets were cleaned to make sure only nodes which had latency information to all other nodes were kept which yielded topologies of size 364 for ”uking-t” and 355 for ”mking-t”. For both topologies there was a need to simulate the effect of churn (nodes joining and leaving) on the structure of the network. This was achieved by assigning nodes to certain percentage of underlying routers, leaving some free to be assigned by joining nodes. Nodes leaving the network put their router back into the pool of free routers to be reassigned. In this way the shape of the underlying network would slowly change as nodes join and leave.
the probability of an incorrect ranking at node i can therefore be described by: xrrl (i) =
P
j=1..k
P dˆx (i, j) − j=1..k d(i, j) P j=1..k d(i, j)
(3)
For each NC system x the mean µxre and standard deviation x σre of the relative error were recorded across all nodes. 2) Relative Rank Loss: A nodes Relative Rank Loss (RRL) is a measure proposed by Lua et al. in their 2005 paper on the accuracy of embeddings for NC systems [34]. RRL is a measure of how accurately relative distances are preserved based on the notion that a common application for NC systems is based on the ability to accurately answer the query ”is node A closer to me than node B?”. For each node i we consider two other nodes a and b which are sampled randomly from the network. We consider swapped(i, a, b) = true if the order is not preserved when comparing estimated latency to real latency (i.e. a is closer than b in the network, but the NC system estimates b is closer than a). If the order is preserved correctly swapped(i, a, b) = f alse. The RRL gives
(4)
For each NC system x the mean µxrrl and standard deviation of the relative rank loss were recorded across all nodes.
C. Missing Landmarks and GNP Due to the effect of churn, there is a chance that landmarks may be unavailable. We have outlined a technique in section IV-C to deal with missing landmarks using Hilbert curves to find nearest neighbours to offline landmarks. This allows NICE to be resilient to landmark failure. However GNP has no such mechanism in their original paper. In order to study the long term effects of churn on GNP accuracy we implemented the same method for GNP as we used for NICE. When this method is not used, GNP coordinates can not be calculated past the initial few cycles of simulation as the landmarks it relies upon go offline. This must be considered when interpreting the results. 0.55
NICE GNP Vivaldi
0.5 Mean RRL Over All Nodes
In order to assess the quality of the NC systems two quality metrics were used: Relative Error and Relative Rank Loss (RRL). 1) Relative Error: A nodes Relative Error considers k nearest neighbours as predicted by the NC system and compares them to the k nearest neighbours found within the actual topology. This is done by comparing the sum of latencies in the following way. For each node of the network of size n, let d(i, j) be the measured latency and dˆx (i, j) be the estimated latency found by NC system x. The relative error xre (i) for node i is then given by: =
(n−1)(n−2) 2
x σrrl
B. Quality Metrics
xre (i)
|{(a, b)|a 6= b & swapped(i, a, b)}|
0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0
10000
20000
30000
40000
50000
60000
70000
80000
Total Nodes Joined + Total Nodes Left
Fig. 2. A two level AS network generated in BRITE with 10,000 routers. The network was divided into 100 AS with 100 nodes in each AS. A PeerSim network of 900 nodes was created using this underlying topology. The mean RRL µx rrl was recorded periodically as nodes joined and left the network for each of the three NC systems. The x-axis shows the total number of nodes joined and left, the y-axis the µx rrl .
VI. R ESULTS A full table of results is shown in Table II. BRITE allowed the creation of larger topologies than is possible using the KING files, this allowed the observation of the effect of large amounts of network drift by having a relatively small overlay network on a large underlying set of ’free’ routers which were assigned during the churn process. Figure 2 shows a plot of µxrrl for each of the NC systems during a simulation where 80,000 nodes joined and left the network. This topology had 900 nodes with 10,000 possible routers. Due to the this the shape of the topology changed significantly as nodes joined and left as a majority of available routers were unassigned. On this topology Vivaldi performed
Measurement µx rrl
x σrrl
µx re
x σre
NC System NICE Vivaldi GNP Random NICE Vivaldi GNP Random NICE Vivaldi GNP Random NICE Vivaldi GNP Random
Total Churn
uKing 0.140 0.161 0.199 0.500 0.058 0.067 0.055 0.029 0.679 0.882 1.043 5.047 0.535 0.727 0.851 3.946 77240
Mean Simulation Results mKing Brite 900N 1KR 0.137 0.167 0.157 0.230 0.245 0.227 0.505 0.501 0.057 0.069 0.068 0.091 0.066 0.086 0.029 0.029 0.665 0.756 0.880 1.035 1.437 0.842 5.061 2.399 0.495 0.373 0.786 0.437 1.325 0.326 3.606 1.218 86240 29000
Brite AS 900N 10KR 0.210 0.367 0.300 0.500 0.057 0.073 0.077 0.029 0.440 2.080 0.959 2.957 0.411 0.852 0.662 0.798 84800
Brite 4KN 10KR 0.247 0.392 0.341 0.500 0.060 0.392 0.073 0.029 0.461 1.100 0.713 0.170 0.168 0.859 0.170 0.308 59000
TABLE II TABLE SHOWING THE MEAN RESULTS OVER EVERY MEASUREMENT ON EACH SIMULATION ON EACH TOPOLOGY
0.35
NICE GNP Vivaldi
2.5
NICE GNP Vivaldi
0.3 Mean RRL Over All Nodes
Mean Relative Error Over All Nodes
3
2
1.5
1
0.5
0.25
0.2
0.15
0.1
0
10000
20000
30000
40000
50000
60000
70000
Total Nodes Joined + Total Nodes Left
0
10000
20000
30000
40000
50000
60000
70000
Total Nodes Joined + Total Nodes Left
Fig. 3. The uKing topology with 320 nodes and 364 routers. The x-axis shows the number of nodes left and joined total since the network started, the y shows the µx re .
Fig. 4. The uKing topology with 320 nodes and 364 routers. The x-axis shows the number of nodes left and joined total since the network started, the y shows the µx rrl .
x the worst with its mean µxrrl just under 0.37 with a mean σrrl of around .38. GNP performed somewhat better with a mean x µxrrl of .3 and mean σrrl of 0.077. Both GNP and Vivaldi’s x µrrl were more effected by network churn than was NICE’s. NICE was the most accurate NC system in terms of mean µxrrl x which was 0.21 with a mean σrrl of 0.057. Figure 4 shows a plot of the µxrrl for the uking topology with 320 nodes and 365 routers. The churn rate was set to be high, with the x-axis of the graph showing how many nodes had joined and left since beginning of the simulation and the y-axis showing µxrrl . These results clearly show that both Vivaldi and NICE are much less susceptible to churn than GNP in this topology. It is possible to see that while the µxrrl for GNP started at a little over 0.17, the effect of churn caused this figure to increase dramatically during the simulation. As was the case
with Figure 2 NICE shows greater stability in the face of churn. Vivaldi performs significantly better on this data set than on the 900 node 10,000 router AS topology, with a µxrrl of x 0.16 on average and with a mean σrrl of approximately 0.055. Although GNP was highly effected by the churn it produced a x lower σrrl across nodes with a mean of 0.06. NICE performed the most accurately in terms of mean µxrrl which was .14. It is possible to see the effect of having to estimate landmarks on µxrrl , with the lowest value of 0.11 coming at the simulation start when 100% of landmarks are online. Figure 3 shows the same topology as figure 4, but instead of µxrrl shows the µxre . The relative effectiveness of the three NC systems are similar to the µxre shown in figure 4. The mean x σre for Vivaldi, GNP and NICE were 0.679, 1.043 and 0.882 respectively. In the simulation shown in figures 4 and 3, GNP suffered
100% landmark failure after 1120 nodes had joined and left. NICE suffered 100% landmark failure after 2920 nodes joined and left. The effects on both methods of this landmark failure can be observed in both graphs as a sharp reduction in accuracy at the start of the simulation. The graph shows however that using the landmark approximation method outlined both GNP and NICE are able to function in this 100% landmark failure state. VII. C ONCLUSION In this work we challenge the notion of embedding NC systems in Euclidean space under the rational that distances in Internet-like topologies may be curved towards the centre. We outline a method of deriving a distance function directly from measurement of the underlying topology and show how this could be integrated into a NC system. We implement this method and present a novel NC system, Non-Euclidean Internet Coordinates Embedding (NICE). The robustness and accuracy of this novel NC system are compared to two well known NC systems which use the Euclidean model of space: Vivaldi and GNP. This comparison is conducted by implementing all three methods in the Java based P2P simulator PeerSim. NICE is shown by experimental simulation to be more accurate than either GNP or Vivaldi in terms of relative rank loss and relative error in both generated BRITE topologies and measured KING topologies. It is also shown to be robust to network churn to a large degree and to the underlying topology changing shape. We additionally describe a method for dealing with landmark failures which allows 100% of landmarks to fail and still achieve a useful level of accuracy. We implement this method for GNP and NICE and show that reasonable levels of accuracy can be achieved even after they experience the failure of all landmark nodes. However, there are still advantages to methods like Vivaldi over NICE in terms of simplicity and decentralisation. Future work will look at creating a fully decentralised implementation of NICE using epidemic protocols to estimate regression coefficients in a distributed manner. The work of this paper is of fundamental nature but has several possible applications to address other important issues. Examples include the family of network clustering problems (p-median, p-center) [35] and the optimal service facility location problem [36], which both rely critically on network latency estimations. R EFERENCES [1] S. Ratnasamy, M. Handley, R. Karp, and S. Shenker, “Topologicallyaware overlay construction and server selection,” in Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies Proceedings, New York, USA, Jun. 2002, pp. 1190–1199. [2] J. Li, “On peer-to-peer (P2P) content delivery,” Peer-to-Peer Networking and Applications, vol. 1, no. 1, pp. 45–63, Jan. 2008. [3] B. Chun, D. Culler, T. Roscoe, A. Bavier, L. Peterson, M. Wawrzoniak, and M. Bowman, “Planetlab: an overlay testbed for broad-coverage services,” IEEE Transactions on Computers, vol. 33, no. 3, pp. 3–12, jul 2003.
[4] N. Bessis, E. Assimakopoulou, M. Aydin, and F. Xhafa, “Utilizing next generation emerging technologies for enabling collective computational intelligence in disaster management,” Next Generation Data Technologies for Collective Computational Intelligence, pp. 503–526, 2011. [5] T. Ng, “Predicting Internet network distance with coordinates-based approaches,” INFOCOM 2002. Twenty-First Annual Joint, vol. 00, no. c, 2002. [6] F. Dabek, R. Cox, and F. Kaashoek, “Vivaldi: A Decentralized Network Coordinate System,” ACM SIGCOMM Computer, 2004. [Online]. Available: http://dl.acm.org/citation.cfm?id=1046017 [7] M. Caesar and J. Rexford, “Bgp routing policies in isp networks,” Network, IEEE, vol. 19, no. 6, pp. 5–11, 2005. [8] Y. Shavitt and T. Tankel, “Hyperbolic embedding of internet graph for distance estimation and overlay construction,” IEEE/ACM Transactions on Networking (TON), vol. 16, no. 1, pp. 25–36, 2008. [9] C. Lumezanu and N. Spring, “Playing vivaldi in hyperbolic space,” in In Proc. of SIGCOMM-IMC. Citeseer, 2006. [10] A. Montresor and M. Jelasity, “Peersim: A scalable p2p simulator,” in Peer-to-Peer Computing, 2009. P2P’09. IEEE Ninth International Conference on. IEEE, 2009, pp. 99–100. [11] T. E. Ng and H. Zhang, “A network positioning system for the internet.” in USENIX Annual Technical Conference, General Track, 2004, pp. 141– 154. [12] H. Lim, J. C. Hou, and C.-H. Choi, “Constructing internet coordinate system based on delay measurement,” in Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement. ACM, 2003, pp. 129– 142. [13] M. Pias, J. Crowcroft, S. Wilbur, T. Harris, and S. Bhatti, “Lighthouses for scalable distributed location,” Peer-to-Peer Systems II, 2003. [Online]. Available: http://link.springer.com/chapter/10.1007/978-3-54045172-3 26 [14] L. Tang and M. Crovella, “Virtual landmarks for the internet,” in Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement. ACM, 2003, pp. 143–152. [15] M. Szymaniak, D. Presotto, G. Pierre, and M. van Steen, “Practical large-scale latency estimation,” Elsevier Computer Networks, vol. 52, no. 7, pp. 1343–1364. [16] M. Costa, M. Castro, R. Rowstron, and P. Key, “PIC: practical Internet coordinates for distance estimation,” 24th International Conference on Distributed Computing Systems, 2004. Proceedings., pp. 178–187, 2004. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1281582 [17] C. Lumezanu, R. Baden, D. Levin, N. Spring, and B. Bhattacharjee, “Symbiotic relationships in internet routing overlays.” in NSDI, 2009, pp. 467–480. [18] J. Cowling, D. R. Ports, B. Liskov, R. A. Popa, and A. Gaikwad, “Census: Location-aware membership management for large-scale distributed systems,” USENIX09, 2009. [19] S. Rhea, D. Geels, T. Roscoe, and J. Kubiatowicz, “Handling churn in a dht,” in Proceedings of the USENIX Annual Technical Conference. Boston, MA, USA, 2004, pp. 127–140. [20] P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, M. Welsh, and M. Seltzer, “Network-aware operator placement for stream-processing systems,” in Data Engineering, 2006. ICDE’06. Proceedings of the 22nd International Conference on. IEEE, 2006, pp. 49–49. [21] M. Steiner and E. W. Biersack, “Where is my peer? evaluation of the vivaldi network coordinate system in azureus,” in NETWORKING 2009. Springer, 2009, pp. 145–156. [22] Y. Chen, X. Wang, C. Shi, E. K. Lua, X. Fu, B. Deng, and X. Li, “Phoenix: A weight-based network coordinate system using matrix factorization,” Network and Service Management, IEEE Transactions on, vol. 8, no. 4, pp. 334–347, 2011. [23] Y. Mao, L. K. Saul, and J. M. Smith, “Ides: An internet distance estimation service for large networks,” Selected Areas in Communications, IEEE Journal on, vol. 24, no. 12, pp. 2273–2284, 2006. [24] Y. Liao, P. Geurts, and G. Leduc, “Network distance prediction based on decentralized matrix factorization,” in NETWORKING 2010. Springer, 2010, pp. 15–26. [25] W. Torgeson, “Multidimensional scaling of similarity,” Psychometrika, vol. 30, pp. 379–393, 1965. [26] V. de Silva and J. B. Tenenbaum, “Sparse multidimensional scaling using landmark points,” University of Stanford, Standford, USA, Tech. Rep. CSE-TR-456-02, jun 2004.
[27] A. Allan, J. Bradbury, and G. Di Fatta, “Virtual landmarking for locality aware peer ids,” in AP2PS 2011, The Third International Conference on Advances in P2P Systems, 2011, pp. 7–12. [28] M. Box, D. Davies, W. H. Swann, and I. Australia, Non-linear optimization techniques. Oliver & Boyd Edinburgh, 1969, no. 5. [29] J. C. Platt, “Fastmap, metricmap, and landmark mds are all nystrom algorithms,” in In Proceedings of 10th International Workshop on Artificial Intelligence and Statistics. IEEE, 2005, pp. 261–268. [30] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan, “Chord: A scalable peer-to-peer lookup service for internet applications,” in ACM SIGCOMM Computer Communication Review, vol. 31, no. 4. ACM, 2001, pp. 149–160. [31] A. Medina, A. Lakhina, I. Matta, and J. Byers, “BRITE: An approach to universal topology generation,” in Proc. of the International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunications Systems (MASCOTS’01), 2001, pp. 346–353. [32] B. Waxman, “Routing of multipoint connections,” Selected Areas in Communications, IEEE Journal on, vol. 6, no. 9, pp. 1617–1622, 1988. [33] K. P. Gummadi, S. Saroiu, and S. D. Gribble, “King: Estimating latency between arbitrary internet end hosts,” in Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment. ACM, 2002, pp. 5–18. [34] E. K. Lua, T. Griffin, M. Pias, H. Zheng, and J. Crowcroft, “On the accuracy of embeddings for internet coordinate systems,” in Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement. USENIX Association, 2005, pp. 11–11. [35] C. Ragusa, A. Liotta, and G. Pavlou, “An adaptive clustering approach for the management of dynamic systems,” Selected Areas in Communications, IEEE Journal on, vol. 23, no. 12, pp. 2223–2235, 2005. [36] A. Liotta, C. Ragusa, and G. Pavlou, “Near-optimal service facility location in dynamic communication networks,” Communications Letters, IEEE, vol. 9, no. 9, pp. 862–864, 2005.