and L3 compose an MIH at hop 7. Hop 8 is also an MIH since it contains four interfaces (A, B, C and D). We now present our probing strategy to find all load-.
Exhaustive path tracing with Paris traceroute Brice Augustin, Timur Friedman, and Renata Teixeira Laboratoire d’Informatique de Paris 6, Universite´ Pierre et Marie Curie Paris, France {brice.augustin,
renata.teixeira, timur.friedman}@lip6.fr
TTL = 6
Categories and Subject Descriptors
TTL = 7
Hop #6
C.2.3 [Computer Communication Networks]: Network Operations
Hop #7 2 0
S
0
2
L1 1
General Terms
Hop #6
L21
Hop #8 0
A
1
Hop #9
E1 B1 2 2 0 0 C 10 1 L31 F 2 0 D 1
Hop #7
Hop #10
Possible traceroute outcome:
0
L20
0
Hop #8
0
G 2
1
L1 0
C0
G2 F0
Hop #9 Hop #10
TTL = 8
Measurement
TTL = 9 TTL = 10
Keywords
Figure 1: Traceroute under load-balanced paths
Traceroute, Load balancing
1.
PARIS TRACEROUTE
Traceroute [2] is used to learn the path between two machines in the internet. Uses range from the diagnosis of network problems to the assemblage of internet maps. Unfortunately, traceroute measurements can be inaccurate and incomplete when the measured route traverses a load balancer. Consider the example in Fig. 1, where L1, L2 and L3 are load balancers. The left side represents the actual network topology. Routers are represented as circles. We also numbered each of their interfaces. Probes are represented by black squares, either above the topology if they traverse L2, or below if they traverse L3. The right side is a possible classic traceroute outcome. This example illustrates the two problems with classic traceroute under load balancing. First, the discovered path is inaccurate since it reports a false link (the link (L20 ,C0 ) does not exist is the real topology). Second, it is incomplete since traceroute missed half of the nodes and their respective links. As a result, one may diagnose a wrong path or build an incomplete map of the network. Recent work [1] proposes a new traceroute implementation called Paris traceroute 1 , which, by controlling the probe 1 Paris traceroute is available http://www.paris-traceroute.net
for
download
at
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright 2006 ACM 1-59593-456-1/06/0012 ...$5.00.
content, solves the problem of inacuracy under load balancing. Two types of load balancers cause problems with traceroute. Per-flow load balancers ascribe each packet to a flow (defined by the five-tuple), and each flow to an outgoing interface. They use a hash of various packet fields to decide to which flow a packet belongs to. On the other hand, perpacket load balancers may choose different paths for packets belonging to the same flow. Traceroute cannot control the effect of per-packet load balancing since we cannot control the path packets will take in the network. Nevertheless, we observe the same effects with per-flow load balancing because traceroute varies some packet fields which are used to define a flow. Thus, probes belong to different flows and may take different paths in the network. When it measures a path, Paris traceroute maintains the flow identifier in all the probes it sends. This control of the probe content avoids a significant amount of the measurement artifacts (loops, cycles, and diamonds) that can be observed in traces. In this work, we modify Paris traceroute to solve the problem of path incompleteness. We propose a new probing algorithm, called Exhaustive algorithm, which sends probes varying the flow identifier to discover all the possible loadbalanced paths between two Internet hosts.
2. FINDING ALL PATHS Let us call a traceroute hop that presents multiple interfaces, a Multi-Interface Hop, or MIH for short. In Fig. 1, L2 and L3 compose an MIH at hop 7. Hop 8 is also an MIH since it contains four interfaces (A, B, C and D). We now present our probing strategy to find all loadbalanced interfaces at a hop h when tracing towards a given destination. The algorithm proceeds iteratively. At each iteration, it readjusts the number of probes to send, according to the number of interfaces it has already found. For each probe to send, it selects the flow identifier to use for
this probe, issues the probe, and waits for the discovered interface. The algorithm keeps sending probes until it does not find any new interface. Before terminating, and if h is an MIH, an additional test determines the type of load balancing that caused this MIH. We now discuss each of the main building blocks of this algorithm in more detail. Interface enumeration. We use a statistical, adaptive approach to compute a confidence level of finding all the interfaces at a given hop h. We model the probing of h as a sample from a random variable uniformly distributed over the set of interfaces at that hop. At any step, having found d interfaces, we hypothesize that n, the total number of interfaces of h, is greater than d. We then adapt k, the number of probes to send, such that the probability of discovering all of the n interfaces is greater than 1 − α, our confidence level. Flow selection. For each probe we send, we vary the flow identifier in a controlled manner. This allows us to explore and discover all possible paths with a high confidence level. In practice, we randomly vary the UDP destination port or the ICMP checksum, for UDP and ICMP probes respectively. Probe retransmission. Unlike classic algorithms, in our case it is crucial to have responses from all the probes we send, to reach the desired level of confidence. We add a mechanism to retransmit probes which do not receive a response before a timeout. MIH classification. If h is an MIH, we then determine the type of load balancing that caused this MIH. We achieve this by sending six more probes having the same flow identifier. If we receive responses from a single interface that composes the MIH, we deduce that this MIH is caused by per-flow load balancing. Otherwise, it might be caused by per-packet load balancing or by a routing change during the measurement. Output. We enrich the output of Paris traceroute to display informations on flows. For each interface, it prints the list of the flows that traverse it. This allows us to easily track a flow hop by hop, and build an accurate map by distinguishing real links from false ones.
3.
RESULTS
We ran experiments from a single source located at the Laboratoire d’Informatique de Paris 6. Our source has only one link to the Internet, via the french academic backbone, Renater. We built a destination list of 5, 000 randomlyselected IP addresses that responded to a ping at the time of the construction of the list. For each destination d in the list, we issue a Paris traceroute to d, using the Exhaustive algorithm, immediately followed by a classic traceroute with three probes per hop. This side-by-side measurement allows us to compare the number of interfaces found by the two traceroutes and evaluate the cost of our algorithm. The routes to approximately 30% of the 5, 000 destina-
tions are affected by per-flow load balancing in the measurements, whereas per-packet load balancing affects less than 2% of the routes. This is mostly due to the fact that the vast majority of load balancers observed (85%) use the per-flow technique, and are located in the networks that compose the core of the Internet. Thus, more paths are likely to traverse a single per-flow load balancer. The prevalence of load-balanced paths in our traces has an impact on the completeness of the paths discovered by classic traceroute. We evaluate the ability of Paris traceroute to discover additionnal interfaces, compared to a classic traceroute. For 1,005 destinations, Paris traceroute finds at least 2 more interfaces than classic traceroute. We found 598 cases where a classic traceroute misses at least 5 interfaces. Nevertheless, in 201 cases classic traceroute also finds interfaces that Paris traceroute misses. In most of the cases, Paris traceroute misses a single interface because of a temporary outage that our simplistic retransmission algorithm cannot handle. Over 1664 distinct MIHs found by Paris traceroute, 850 MIHs (51%) have only 2 interfaces, and 445 (27%) present more than 3 interfaces. Our tool finds additional interfaces with the price of a higher number of probes sent. For instance, for a confidence level of 95% at a given hop, the algorithm has to send at least 6 probes in presence of a single interface. We need 33 probes to discover an MIH of 6 interfaces. In our traces the maximum number of interfaces found in an MIH is 16. Interestingly, it is also the maximum number of load-balanced paths in Juniper routers.
4. PERSPECTIVES This work should inspire research in many directions. First, the Exhaustive algorithm will allow us to build more complete and accurate maps of the Internet. We also plan to study the impact of our tool on previous studies based on classic traceroute measurements. For instance, our view of routing dynamics might be significantly biased by the measurement artifacts that arise when using classic traceroute in load-balanced paths. Finally, we plan to study the original topologies that our algorithm uncovers. This will allow us to evaluate the path diversity that the network natively provides, with possible innovations in the optimization of end-to-end connections.
5. REFERENCES [1] B. Augustin, X. Cuvellier, B. Orgogozo, F. Viger, T. Friedman, M. Latapy, C. Magnien, and R. Teixeira. Avoiding traceroute anomalies with paris traceroute. In Proc. ACM SIGCOMM Internet Measurement Conference, Oct. 2006. [2] V. Jacobsen. traceroute, Feb. 1989. The most recent version is available at: ftp://ftp.ee.lbl.gov/traceroute.tar.gz.
