Fault Tolerant Algorithms for Network-On-Chip Interconnect - CiteSeerX

Fault Tolerant Algorithms for Network-On-Chip Interconnect M. Pirretti, G. M. Link, R. R. Brooks, N. Vijaykrishnan, M. Kandemir, M. J. Irwin The Pennsylvania State University University Park, PA 16802 email: {vijay, pirretti, link}@cse.psu.edu This paper appeared in the proceedings of ISVLSI, February 2004 Abstract As technology scales, fault tolerance is becoming a key concern in on-chip communication. Consequently, this work examines fault tolerant communication algorithms for use in the NoC domain. Two different flooding algorithms and a random walk algorithm are investigated. We show that the flood-based fault tolerant algorithms have an exceedingly high communication overhead. We find that the redundant random walk algorithm offers significantly reduced overhead while maintaining useful levels of fault tolerance. We then compare the implementation costs of these algorithms, both in terms of area as well as in energy consumption, and show that the flooding algorithms consume an order of magnitude more energy per message transmitted.1 Index Terms: Fault Tolerance, Network on Chip, Random Walk, Flooding.

1. Introduction As technology scales, errors and faults are becoming increasingly common. Crosstalk interferes with signal transmission, while soft errors result in random bit-flips throughout the design [3]. Manufacturing faults can result in entirely non-functional segments of the circuit. The ITRS notes that relaxing the 100% correctness requirement for designs can result in lowered costs of manufacturing, verification, and testing [1]. Designing systems that operate in the presence of transient or permanent failures has been studied for years, but research has focused mostly on largescale systems and their interconnect. The Network-On-Chip (NoC) design paradigm has been proposed as the future of ASIC design [2]. The NoC design methodology connects different IP blocks by a packet-based on-chip network. This is markedly different from traditional large-scale interconnects, due to the limited area resources available on the chip, as well as the increased dependence on low-latency communication. Implementing traditional fault tolerant algorithms in the NoC domain is infeasible due to these area restrictions, so 1 This research is sponsored by the Defense Advance Research Projects Agency (DARPA), and administered by the Army Research Office under Emergent Surveillance Plexus MURI Award No. DAAD19-01-1-0504. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the sponsoring agencies.

other techniques must be developed if fault tolerant ASICs are to become possible. Previous work in this area is extremely limited as NoC design is still in its infancy. Dumitras et al. [4] have focused on stochastic communication for a number of reasons. Stochastic communication has been shown to cope well with random faults, requires no large lookup tables or reconfiguration, and requires very few retransmissions overall. Dumitras et al. propose a probabilistic flooding scheme based upon well known gossip techniques as a possible fault tolerant solution [6] [7]. Flooding is an effective fault tolerant technique because it is highly fault tolerant. If a path exists to the destination, a message will almost certainly arrive. In practice, however, this level of fault tolerance may not be necessary. Resilience to a much smaller number of faults still offers increased chip yields, as well as resistance to transient failures, but also reduces unnecessary packet transmissions. Lowering the number of unnecessary packet transmissions allows for higher network throughput, and a more efficient use of the interconnect. This paper investigates the previously proposed probabilistic flooding algorithm, as as well as two new fault tolerant algorithms, the directed flood and the N-random walk. To more accurately gauge the effectiveness of these algorithms in the NoC design space, we designed the components in Verilog, and used synthesis tools to determine the area and energy requirements of the hardware. The remainder of this paper begins by discussing the algorithms and their implementation. We then compare the performance of the algorithms using a NoC simulator, and show that the directed flood has better fault tolerance and lower overhead than the probabilistic flooding algorithm. The N-random walk algorithm has nearly the same level of fault tolerance, but requires an order of magnitude fewer packet transmissions. The implementation costs of the algorithms are presented, and we find that all three algorithms are similar in terms of area cost, but that the flooding algorithms consume an order of magnitude more energy than the N-random walk algorithm proposed.

2

Fault Tolerant Algorithms

A number of different fault tolerant methods of communication for large-scale systems have been proposed [5] [12]. These algorithms are not amenable for NoC implementation due to significant area and storage overhead. Recently, work has begun on fault tolerant algorithms specifically for the NoC space, with most consisting of various forms of gossip algorithms [4]. We investigate two new fault tolerant algorithms, and compare them to the previ-

ously proposed probabilistic flood [4]. We have utlized the same uniform communication pattern as was used in the probabilistic flooding algorithm for the sake of making fair comparisons.

2.1

Probabilistic Flooding

The probabilistic flooding algorithm is a variant on a simple flooding protocol. In this algorithm, every time a message is generated it will be passed to all of its neighbors with probability p, and will be dropped with probability 1p. p must be carefully chosen to obtain near flooding performance without sending as many redundant copies of the message [9]. Given enough time every node in the mesh will have received multiple copies of the message, with a high probability. This mode of communication is most susceptible to failure during its initial phase of transmission, when only a limited amount of nodes need not pass on the message in order for message propagation to halt. To be certain that the message does not die off during this period, an initial flooding phase of two cycles is performed to ensure that all functional nodes within two hops receive a copy of the message. Implementing this algorithm in hardware requires a pseudo-random number generator to determine whether or not to replicate a given message. This, and further implementation details, are discussed later in the paper.

2.2

Directed Flooding

The probabilistic flooding algorithm is destinationignorant in that packets get routed irrespective of where they are in the network. This results in significant network resources being used to transmit the packet to areas of the chip where it will not be needed. The directed flooding algorithm makes use of an NoC’s highly regular structure to direct a flood towards the destination. In this algorithm, the probability p of passing a message to each outgoing link is not fixed, but instead varies based on the destination of the packet. At the algorithm’s onset packets have a high probability of being routed towards the destination node, allowing for packets to get in the general vicinity of the destination node quickly. As the packets approach the destination node it is desirable for them to take varied paths, to mitigate the risk of all packets encountering the same defective nodes on their way to the destination. More exactly the probabilities of a message being sent out on a given output port are calculated as follows: First, the manhattan distance Dc between the current node and the destination node is computed. This distance is also computed for all nodes adjacent to the current node, giving DN ,DS ,DW , and DE . A multiplicative factor Mx is set to 1 for any direction where Dx is greater than Dc . For the remaining nodes (where Dx ≤ Dc ), the multiplicative factor Mx is equal to the min(Dx ,4). This min function prevents the scenario where a distant destination weights the multiplicative factors heavily, resulting in all packets following similar paths. Using a maximum multiplicative factor of 4 allows the most heavily favored ports to be followed more often and results in lower latencies, while still forwarding the packet on less favored ports an appreciable amount of the time. Finally, each of the multiplicative factors Mx is multiplied by the gossip rate, G to find the probability of transmission to that particular link. G is a user-defined con-

stant that can be varied to control the level of performance versus fault-tolerance. This algorithm, like the probabilistic flood, has a statistical chance of not replicating a given message received. Therefore, there is an initial two cycle flood to seed the area, resulting in all functional nodes within two hops receiving a copy of the message.

2.3

Redundant Random Walk

The two previous algorithms result in significant extraneous transmission overhead, and as a result, limit overall network performance. The random walk algorithm we propose sends a finite, predetermined number of copies of a message into the network. These messages follow a nondeterministic path, where every node that receives one of these messages must forward it to one of its output links. To simplify the creation of these additional redundant packets, an initial flooding stage can be used, just as in the other proposed algorithms. This initial flooding stage allows for a finite number of packets to be generated easily, while limiting the impact of any local faults. The choice of output link is determined by a set of random probabilities pN , pS , pW , pE , where the sum of all probabilities is 1. Therefore, every incoming message is paired with exactly one outgoing message, as in a standard network. Though normalization is often considered to be a large and slow operation, due to the limited number of possible values for the multiplicative factors, a reasonable approximation can be fashioned out of combinatorial hardware. The values for all probabilities (pN etc.) are computed as follows. First, the multiplicative factors Mx are computed as in the discussion of directed flooding above. The multiplicative factors are then normalized to create the probabilities pN , pS , pW , pE . A random number is then generated, choosing one output port of the four to follow. Only one random decision unit is required per input port, rather than the multiple required for the directed flooding algorithm. This reduction in random decision units helps compensate for the additional logic required for the normalization process. In general, each message in the network will tend towards the same destination, but will follow slightly different paths to reach it. Thus, even if a few links on the way are broken, at least one message should arrive intact.

2.4

How to Determine Number of Redundant Walks

When implementing the N-random walk algorithm, n redundant messages must be sent. It is possible to calculate a reasonable value for n using methods derived from Markov chain and random walk literature [10] [11] [13]. One such method is to model the NoC communication behavior using a two-dimensional discrete time Markov chain, with one state in the Markov chain for each node in the NoC. In our NoC example this model indicates the probability that a packet will reach its destination by time t. Our Markov chain model requires a transition probability matrix to describe the probability of transitioning from any given state to any other state in the Markov chain, requiring a matrix of size x2 by x2 for an x by x NoC. A node located at point (i,j) in the NoC corresponds to row and column x*(i-1) + j in the transition probability matrix. There is one row and one column in the matrix for each node. Each column tells us which nodes can directly transition into this

node, and how probable these transitions are. Similarly, the rows of the matrix indicate the probability of transitioning out of one node and into another. We must now define our desired model a bit further. We would like to know how the N-random walk algorithm behaves in the worst case communication scenario. This corresponds to when a node in one corner is communicating with a node in the opposite corner. To further define our model, once a packet has arrived at its destination we do not want it to continue propagating throughout the network. In order to achieve this goal we can make modifications to the destination node’s row in the transition probability matrix. We do this by making the probability of transitioning out of this node to be 0 (i.e. once the model enters this state it will never leave). To develop the transition probability matrix for the N-random walk one has to take into consideration how the algorithm will attempt to route packets. Intermediate routers are biased in that they tend to route packets towards the destination. Also recall that as packets get closer to the destination this bias decreases. Additionally if a node determines that a packet is destined for one of its adjacent neighbors it will route to this neighbor with a probability of 1. We would also like to model transient faults using this model, where a packet can be randomly dropped at any point in time with probability E. To do this we now consider each entry in the transition probability matrix to be the probability of transitioning to another state without experiencing an error (i.e. it is the intersection of these two events). We make the claim that these two events are independent, and thus to calculate the probability of moving in a certain direction we can simply add a multiplicative factor of (1-E) to each entry in the transition probability matrix. We can now obtain a cumulative probability distribution describing the relationship between nodes in opposite corners. Utilizing this distribution we make one last refinement so that we can model the effect of multiple redundant packets, based on the claim that the behavior of the individual packets are independent. We are interested in the probability of any of the redundant packets reaching the destination node (i.e. the union of all the redundant packets). Manipulating some basic axioms of probability gives us the following equation for extending the probability distribution of one packet to an arbitrary number of redundant packets N. PN = 1 − (1 − P1 )N where PN is the probability distribution for N redundant packets and P1 is the probability distribution for 1 packet. This model is sufficient to illustrate how the probabilities scale for different quantities of redundant packets and different error probabilities. We show data that we have collected using this method in Table 1. In this table we have used a maximum lifetime parameter of 50 in order to be consistent with the results from the rest of the paper. Notice that there are a few differences between the calculated values and the values obtained in practice. First, the calculated values are for the worst case scenario, which is why they have lower probabilities than what is seen in Figure 1. Additionally when execution of the simulations that were used to derive the results in Figure 1 were stopped there were still packets in transit. If the simulation was to be executed to allow these packets to clear out of the network the probabilities obtained would be slightly higher (this effect is most noticeable in how 100% is never obtained in the figures).

Errors (%) 0 1 2 3 4 5

N=1 39.4 25.3 16.2 10.3 6.6 4.2

N=4 86.5 68.8 50.6 35.3 23.8 15.6

N=8 98.2 90.3 75.6 58.1 41.9 28.8

N=12 99.8 97.0 87.9 72.9 55.7 40.0

N=16 100 99.1 94.0 82.4 66.2 49.4

N=64 100 100 100 99.9 98.7 93.4

Table 1. Message Arrival using N-Random Walk Equations(%)

3

Implementation Details

Simulation of network performance is done using a packet level network simulator. The network is composed of 256 routing nodes arranged as a 16 by 16 mesh. Each packet is composed of a 32-bit header and a 256-bit payload. These packets are transmitted across the network in two 144-bit flits in a store-and-forward fashion. Each routing node contains both input and output buffers on each port, for a total of 12 packets of storage per routing node. The crossbar is a fully connected 5 x 5 design, with a bit width of 144 bits. Arbitration is done using a simple FIFO organization. The node to node interconnect was modeled in HSPICE to obtain energy consumption values. Interconnection is assumed to be done on metal 3, and wires are assumed to traverse a distance equal to a single routing node. Determination of random variables is accomplished using linear-feedback shift registers (LFSRs) [8]. The LSFRs are composed of a series of D flip-flops followed by a pattern of XOR gates. Though these provide only an approximation of truly random output, outfitting routing nodes with LSFRs of differing sizes can ensure that the entire network array does not follow the same pattern. A single 8-bit LSFR was synthesized to obtain an approximate energy consumption per cycle, and this number was used wherever an LSFR operation was called for. The LSFR used has a period of 255 cycles. The calculation process that determines whether or not to replicate a packet was not done using double precision in hardware. Instead, an unsigned fixed point 8-bit representation was used. This allows for accurate operation of the algorithm, and hardware results reasonably match those obtained in higher precision. Arithmetic units used in the operations accommodate 8-bit operands as well. Components for each node were design in Verilog and synthesized to obtain area and energy consumption information for each algorithm.

4

Experimental Results

To evaluate the performance of the algorithms, we have used three separate metrics: fault tolerance, latency, and implementation costs. In each simulation all non-faulty tiles communicate uniformly with all other non-faulty tiles. All message recipients were non-faulty tiles so that the algorithms would be evaluated based upon their fault-tolerance and not on how fortunate they were to pick a working destination. For all simulations, an injection rate (i.e. the proportion of nodes that generate messages per iteration) of 0.05% was used. The injection rate was chosen to be this low to allow for meaningful results for the flooding algorithms. Higher injection rates quickly saturate the network when high levels of redundancy and flooding algorithms are used.

A time to live of 50 hops was used for all simulations unless otherwise noted. The choice of this value is explained in detail later.

4.1

of all tiles in the NoC that are not functioning properly due to manufacturing faults.

Fault Tolerance

Fault tolerance evaluates how reliably each algorithm can route messages in despite transient and permanent faults. Transient faults are considered to have a percentage chance of disrupting a message during each hop, while permanent faults indicate nodes that are unable to send any packets. We first vary the transient fault rate, which is a generic representation of transient errors encountered in the system [4]. These errors include any alterations to the packet bit stream (e.g. crosstalk), as well as buffer overflow. In both cases a message that encounters an error is dropped from the network. This was implemented in the simulations by dropping a fixed percentage of all packets in the network for every iteration of the algorithms. Figure 1 shows the fault tolerance of the various algorithms for transient faults, such as crosstalk and soft errors.

(a) Fault tolerance for N-random walk

(b) Fault tolerance for gossip flooding

(a) Fault tolerance for N-random walk (c) Fault tolerance for directed flooding Figure 2. Effect of permanent faults on successful transfers (2% transient fault rate)

(b) Fault tolerance for gossip flooding

(c) Fault tolerance for directed flooding Figure 1. Effect of varying transient errors on successful transfers(0% permanent fault rate) We see that the N-random walk and the probabilistic flood offer similar levels of fault tolerance, while the directed flood algorithm is slightly more fault tolerant. Increasing the number of redundant packets sent significantly affects the transient fault tolerance of the N-random walk, as with a 5% transient fault rate, a packet has less than a 50% chance of being intact after 14 hops. We then evaluate the effect of varying the permanent fault rate. This parameter models the effect of faulty tiles in the NoC. Permanent fault rate corresponds to the percent

Figure 2 shows the effect of permanently faulty nodes on successful transmission rate. For low levels of permanent faults, N-random walk performs well, but as the number of fault nodes increases beyond 10%, flooding algorithms become the only reasonable option. In practice, however, a die with 10 to 20% of routing nodes faulty is most likely binned as a bad part. As the algorithms presented are not deterministic, there is no fixed time by which a packet will reach the destination. The flooding algorithms exacerbate this problem, as each hop can generate a number of new packets. To prevent saturation of the network, a maximum lifetime is imposed on each message, such that all packets relating to that message will die off after a fixed number of cycles. A high value for this time to live results in a larger number of packets in the network for longer, and as such, higher fault tolerance. On the other hand, these same packets also congest the network, so we should keep the lifetime as small as possible while still maintaining high levels of fault tolerance. Figure 3 shows the effect of lifetime on fault tolerance. Note that extremely small lifetimes result in very poor performance. This is not necessarily due to faults, but instead because the message dies off before any of the packets reach the destination. It is evident that increasing the lifetime of packets beyond 50 has little difference on the level of fault tolerance within the network. As such, the lifetime for all simulations was set to 50 for this paper.

(a) Fault tolerance for N-random walk

Figure 4. Comparison of latency across algorithms (2% permanent, 1% transient fault rate) Algorithm Area per Node in 70nm (sq. mm.) Energy per Hop in 160nm (pJ)

GFlood 0.121 927.22

DFlood 0.127 971.47

RWalk 0.131 975.16

Table 2. Area and energy comparison (b) Fault tolerance for gossip flooding

(c) Fault tolerance for directed flooding Figure 3. Effect of lifetime on successful transfers (2% permanent, 1% transient fault rate)

4.2

Latency

Latency measures how quickly a message arrives at its destination. This is measured as the number of hops that are required for a message to reach its destination. Since these algorithms utilize redundant packets, only the first time a message arrives at its destination is included in this metric. We have compared the latencies of the three algorithms, as shown in Figure 4. This figure indicates that directed flooding tends to find the shortest paths, while the other two algorithms have comparable latencies. We can also see that the effect of increasing redundancy in N-random walk and probabilistic flood is to reduce the algorithms’ latency. It is notable that for the parameter settings we have chosen for directed flooding, increasing redundancy has little affect on latency. This is due to directed flooding’s tendency to push the flood towards the destination early, then let the flood effect find the destination once in the general area. The N-random walk provides comparable performance to the previously proposed probabilistic flood, while the directed flood provides superior performance in all cases.

4.3

Energy and Area Cost

This metric evaluates the viability of the three algorithms based upon the implementation cost of each node. Table 2 shows the relative area cost of each type of routing node. The majority of the area in each node is consumed by the packet buffering and crossbar. Thus, the additional stochastic hardware required for the directed flood and N-random

walk are not a significant contributor to overall routing node area. In a 70nm technology using the redundant random walk algorithm, 256 routing nodes will consume less than 35 percent of a 1 sq. cm. die. As energy consumption and power consumption are increasing in importance with technology scaling, we must consider the energy consumption of these algorithms. The hardware for each algorithm was implemented in Verilog, and synthesized using a commercial 160nm library to obtain energy consumption information. Table 2 summarizes the energy consumption of the different algorithms. Note that the energy per hop is very similar among the three algorithms, and that the additional stochastic hardware is insignificant. We then compare the total energy consumed by each algorithm, based on the number of hops required by each unique message sent. As different network sizes require different packet lifetimes, we vary the lifetime of the packet to demonstrate how the algorithms scale. Figure 5 shows the total number of hops generated by each simulation, as the lifetime of the packets varies. From these results, we see that N-random walk clearly has the fewest total hops. Typically, the hop count is at least an order of magnitude smaller than what it is for the flooding algorithms (with the exception of the N=64 case, which has about a third of the hops). The directed flooding algorithm typically generates 10% fewer messages than the probabilistic flooding algorithm. As the number of messages in each simulation is fixed, this graph also represents the number of hops per unique message sent. Thus, the total energy consumption per unique message can be calculated, as shown in table 3. This shows that though the per packet consumption may

Algorithm Gossip Flood

Directed Flood

Redundant Random Walk

Configuration and Energy Consumption (uJ) Chance of Node Continuing Flood 40% 60% 80% 1.17 3.89 4.49 Chance of Output Port Continuing Flood 25% 40% 60% 3.31 4.31 4.53 Number of Redundant Packets Sent (N) 1 4 8 12 16 64 0.025 0.10 0.20 0.29 0.39 1.60

Table 3. Energy consumption per message

(a) Total hops for N-random walk

(b) Total hops for gossip flooding

(c) Total hops for directed flooding Figure 5. Effect of lifetime on total hops (Note: scales are different for each chart) be similar, the overall energy consumption differs significantly. The flooding algorithms have a dynamic power consumption per message roughly an order of magnitude larger than that of the N-rwalk. This additional energy consumption provides little additional fault-tolerance for nominal error rates. For example, with 2% of nodes permanently faulty, as well as a 2% chance of packet disruption per hop, the N-rwalk consumes only 0.39 uJ to achieve a 95% successful transmission rate, while Gflood consumes nearly 3.89 uJ to obtain a similar level of fault-tolerance. For levels of permanent faults below 5% and transient fault rates below 10%, the N-random walk provides similar levels of fault tolerance, while consuming an order of magnitude less energy per successful message. Even when sending 64 duplicate copies of a message, the N-random walk still consumes less energy per message than the flooding algorithms.

5

Conclusions

As technology scales, reliability of circuits becomes a significant concern. Implementing fault tolerance in future ASIC designs insures manufacturability and reliable operation at reasonable costs. Implementing traditional fault tolerant communication algorithms in NoC’s is infeasible due to area constraints, so new approaches to fault tolerant communication must be found. Previous works have suggested stochastic flooding algorithms as a means of reliably transmitting messages over a NoC interconnect. In this work, we compared three different fault tolerant communication algorithms for the NoC domain. We find that flooding algorithms cause significant communication

overhead, which limits the feasibility of such algorithms to a very low message injection rate. The random walk algorithm typically required an order of magnitude less communication overhead than would be possible for the flooding algorithms. In comparison to each other, the directed flooding required 90% of the overhead that the previously proposed probabilistic flooding required. We have also observed that the flooding algorithms tend to be more fault resistant. As error rates go beyond 10% random walk throughput begins to drop. Note that this is a very high number of errors in practice. This can be combated by increasing the quantity of redundant messages beyond what was attempted in this paper at the cost of increased overhead. Further, improvements could be made to the random walk algorithm, such as dynamically adjusting the quantity of redundant messages to cope with current network conditions. The flooding algorithms, especially the directed flood, tended to be quite error resistant for higher error rates. Directed flooding also appears to operate quite well with a lower gossip rate than what would be possible for probabilistic flooding. This fact can help to keep directed flooding’s overhead lower. It is important to note that since the flooding algorithms have a much higher quantity of message hops, they are likely to have higher packet drop rates than would be likely for random walk, due to buffer overflows. Our results indicate that flooding protocols, while simple and lightweight, rely on too many message replications to obtain their fault tolerance. While this can be compensated in part by using a directed flood with a lower gossip rate, a much more drastic savings in overhead can be obtained by using fault tolerant algorithms that focus on limited message replication following individual non-uniform paths such as the random walk.

References [1] The international technology roadmap for semiconductors, 2002. [2] L. Benini and G. D. Micheli. Networks on chip: a new paradigm for systems on chip design. In Proceedings Design, Automation and Test in Europe Conference and Exhibition, pages 418 –419, 2002. [3] W. Dally and J. Poulton. Digital Systems Engineering. Cambridge University Press, 1998. [4] T. Dumitras, S. Kerner, and R. Marculescu. Towards onchip fault-tolerant communication. In Proc. Asia and South Pacific Design Automation Conference, 2003. [5] Y. Hatanaka, M. Nakamura, Y. Kakuda, and T. Kikuno. A synthesis method for fault-tolerant and flexible multipath routing protocols. pages 96–105, 1997. [6] S. M. Hedetniemi, T. Hedetniemi, and A. L. Liestman. A survey of gossiping and broadcasting in communication networks. NETWORKS, 18:319–349, 1988. [7] D. W. Krumme, G. Cybenko, and K. N. Venkataraman. Gossiping in minimal time. SIAM J. Comput., 21(1):111–139, 1992. [8] P. L’Ecuyer and F. Panneton. A new class of linear feedback shift register generators. In Proc. 2000 Winter Simulation Conference, 2000. [9] L. Li, J. Halpern, and Z. Haas. Gossip-based ad hoc routing. [10] J. R. Norris. Markov Chains. Cambridge University Press, 1997. [11] A. Papoulis and S. U. Pillai. Probability, Random Variables and Stochastic Processes. McGraw Hill, 2002. [12] W. Stallings. Data and Computer Communications. Prentice Hall, 1999. [13] W. J. Stewart. Introduction to the Numerical Solutions of Markov Chains. Princeton University Press, 1994.