Modular Self-stabilization of Network Protocols

Modular Self-stabilization of Network Protocols

A case-study in O'Haskell Johan Nordlander

[email protected]

1 Introduction Self-stabilization is a quality attributed to network protocols that are guaranteed to exhibit correct behavior in a nite amount of time, even if started in an erroneous state. Such protocols clearly have some very desirable fault-tolerant properties, e.g. the ability to automatically recover from arbitrary state-variable corruption, in any number of nodes, once memory failures stop. Selfstabilizing protocols also avoid some of the initialization problems that are inherent in distributed and dynamic systems, since, in eect, any network state is an acceptable initial state. The concept of self-stabilization was originally conceived by Dijkstra [Dij74]. While the ensuing work has mostly been concerned with the construction of self-stabilizing protocols for particular tasks like token passing, network reset, and spanning tree construction [BP89, GM91, KP93, DIM93, IJ90, AG94, AKY90], some general methods for transforming existing protocols into selfstabilizing counterparts have also been devised. The scheme proposed by Katz and Perry can convert an arbitrary non-stabilizing protocol into a stabilizing one; however, this generality comes at the expense of a quite substantial space and processing overhead that must be paid even during normal operation [KP93]. In contrast, Awerbuch, Patt, and Varghese are able to construct highly ecient stabilizing protocols by their transformations, but their method is limited to protocols that are amenable to what is called local checking and correction [APSV91, Var93]. Unfortunately, both these approaches are described using a mixture of formal and informal arguments, and it is not entirely clear from the sources available how to put the ideas into practice, given an implementation of some particular non-stabilizing algorithm. In this paper we will take some steps towards a practical application of self-stabilization, by means of a concrete implementation of the stabilizing method of Awerbuch, Patt, and Varghese. The implementation is coded in the programming language O'Haskell, which is an object-oriented extension of Haskell [PH96], oering state encapsulation, reactive concurrency, and subtyping in addition to Haskell's non-strict, purely functional core. We present this implementation primarily as an illustration of the usefulness of O'Haskell as a network programming language, but besides that, we also think that our coding exercise makes some small contributions to the eld of selfstabilization:  It sheds some light on coding issues that are either just outlined in [APSV91], or obscured by the rather abstract formalism of I/O Automata used in [Var93].  It provides modularity, in the sense that the self-stabilizing code is encapsulated and can be plugged into the signal path of any protocol implementation that meets the requirements of local checking and correction. This stands in contrast to the original method description, which is speci ed as stepwise modi cations that must be made to the non-stabilizing protocol code. On another account, we also note that working in a non-strict language puts some focus on the assumption that underlies the self-stabilization model: only program state can be corrupted, not program code. An analysis of the validity of this assumption is however outside the scope of the present work.

The paper now continues with an introduction to the ideas behind local checking and correction, before we give swift overview of O'Haskell in section 3, especially its relation to I/O Automata. Section 4 describes an object template stabilizer, which is an O'Haskell encoding of the general self-stabilizing mechanisms that implement local checking and correction. In section 5 we provide an implementation of a protocol example from [APSV91], and show how it is made self-stabilizing utilizing the stabilizer from section 4. The paper rounds up with our conclusions in section 6

2 Self-stabilization by local checking and correction In the literature on local checking and correction, a network is a directed, symmetric graph, where each node is modeled as a node automaton (in the I/O Automata formalism), and each link is a unit storage data link (UDL), that is also modeled as a simple I/O automaton. A link subsystem is de ned as the composition of two connected node automata, together with the two unidirectional UDLs that connect them. Thus, a link subsystem is uniquely identi ed by a pair of node identi ers. The key idea behind local checking is now this: for many protocols it can be shown that whenever the protocol is in an illegal state, some link subsystem must also be in an illegal state. So instead of collecting and checking state information for the whole network at some central node, a protocol amenable local checking can examine each link subsystem in parallel. A corresponding conception underlies the theory of local correction: many protocols can return to a legal state by letting each link subsystem independently correct itself to a legal state. An intuitive precondition here is that correcting one link subsystem must not invalidate the correctness of others, however, it can be shown that a sucient condition is to require that any such invalidation dependencies be acyclic. The goal of the transformation scheme of Awerbuch, Patt, and Varghese is to convert any locally checkable and correctable protocol into a self-stabilizing variant. To this end it is assumed that for the given, non-stabilizing protocol, there is a set of local predicates = that can be used for checking every link subsystem ( ), and a correction function for resetting the state of a node automaton w.r.t. the link subsystem ( ). To simplify the model it is assumed that in each link subsystem there is a designated leader node that is responsible for periodically checking, and eventually correcting the link subsystem. As a further simpli cation it is prescribed that the UDLs of a link subsystem be empty during checking and correction, hence ordinary trac must be suspended whenever the subsystem is in a checking/correction phase. In [Var93] this is achieved by means of boolean ags in the node automata, under the supervision of code added by the transformation scheme. We will discuss an alternative solution to this problem in the next section. The essence of the checking and correction extension is quite simple. For a particular link subsystem ( ), the leader node (say ) sends out a distinguished snapshot request packet to its neighbour node ( ), which in turn replies with a snapshot response packet containing its current state. Node then uses local predicate to determine whether correction of ( ) is needed. If so, a reset request packet is sent out by , which causes to reset itself using and send back a reset response packet. When node receives the response, it also resets its state using . The correctness of this snapshot/reset algorithm relies on the assumption that the given protocol is sound in the sense that legal states are only followed by legal states (in the absence of memory faults). Hence and can be used as if they were applied to the whole link subsystem ( ) at once, even though the individual node states are sampled at dierent times. In order to make the snapshot/reset algorithm insensitive to garbage packets on the links (i.e. making the stabilizing code itself self-stabilizing), counter- ushing is used (see [Var93]). Since the communication channels are UDLs, there can only be 3 distinct packets hidden in a link subsystem; hence counters can wrap around when they reach 4. Further details on the theory of local checking and correction, as well as proofs of the formal claims made, can be found in [Var93]. We will now proceed with the encoding of these ideas in O'Haskell. Lu;v

u; v

u

Lv;u

fu;v

u; v

u; v

u

v

u

Lu;v u

u

Lu;v

u; v

fu;v

u; v

v

fv;u

fu;v

3 An overview of O'Haskell O'Haskell is an object-oriented extension of the lazy, purely functional language Haskell [PH96]. The only major omission from Haskell is its record-syntax, which is replaced by a more powerful mechanism that supports subtyping. The central notion in O'Haskell is the state-encapsulating, concurrently executing object. Objects are de ned in a template construct, which, when executed, creates a new object with its own unique instance variables. To preserve the pure semantics of Haskell, state-variables are not generally visible, they can only be accessed (and assigned to) within procedures (a.k.a. do-expressions), and their message-passing counterparts, the action and request constructs. Actions and requests are collectively called methods; they dier only in the semantics of method invocation, which is asynchronous for actions, and synchronous in the case of a request. On the type level, templates, actions, and requests are matched by the prede ned constants Template a, Action, and Request a, respectively. For readers familiar with Haskell, the upper bound of these three types constitutes a monad, which is the established method of introducing computational eects in a pure functional language. An object is an implicit critical section, that is, at most one of its methods can be active at any time. Methods are furthermore guaranteed to be executed in the order they are invoked, although concurrent execution of multiple objects can make the actual invocation order nondeterministic. Another important characteristic of O'Haskell is that the methods of an object cannot be disabled, nor can a method choose to wait for anything else than the termination of another, synchronous method invocation. The net result of these restrictions is that in the absence of deadlocks and in nite loops, objects are only transiently active, and can be guaranteed to react to method invocation within any prescribed time quantum, just given a suciently fast processor. For this reason, we characterize O'Haskell as a reactive language. Record types are introduced using struct de nitions, and the subtype relation is de ned by explicit declaration. This makes record subtyping in O'Haskell look quite similar to interface extension in Java, with the notable dierence that structs in O'Haskell can also take type arguments. Moreover, O'Haskell is supported by a powerful partial type inference algorithm, which eliminates the need for explicit type annotations in most situations that arise in practice. O'Haskell also contains a number of minor, mostly syntactic extensions to Haskell, but these will be introduced whenever called for, as we proceed through the code examples. The reader is referred to [NC97] and [Nor98] for an indepth explanation of the novel features of the language.

3.1 Some basic de nitions

As an introductory example of the subtyping system of O'Haskell, let us de ne a small hierarchy of struct types that will form the basis of the object interfaces to follow. type Link = Int struct Client a = receive :: Link -> a -> Action struct PolledClient a < Client a = poll :: Link -> Request (Maybe a) struct AckClient a < Client a = ack :: Link -> Action

These de nitions are intended to capture the operations exported by various kinds of communication protocols, as seen from an immediate lower layer in the protocol stack. Because of subtyping, both PolledClients and AckClients also support the receive operation. For the particular application of this paper, the addressing needs are rather simple, thus we just parameterize each operation with respect to the index number of a particular link.

It turns out that most protocols intuitively expose a quite dierent interface to their upper clients, from what is visible from below. We can express this by means of another, unrelated set of struct de nitions: struct NullNetwork struct Network a < NullNetwork = send :: Link -> a -> Action

Here we see an example of a struct type without any selectors. This de nition is useful since there are protocols (e.g. those that wish to poll their client for outgoing data) that do not export any operations at all for the upper layers to invoke. In O'Haskell, the interface to an object can be of any type, in particular, it can be a pair of sub-interfaces intended for two dierent kinds of users. We establish this as a convention for network programming in O'Haskell, and de ne a general template combinator , for stacking two protocols and connecting their respective interfaces: ta tb client network = new (a_up,a_down) (a' -> nw -> Template (b',b)) -> (cl -> nw -> Template (a,b))

3.2 O'Haskell vs. I/O Automata

Before we proceed to the actual encoding of self-stabilization, we need to make a few comments on the relationship between O'Haskell and the formalism used in the original description of the algorithm: Input/Output Automata, or IOA for short [LT89]. First, we notice that there are indeed striking similarities between the formalisms: objects in O'Haskell resemble automata both in terms of state encapsulation and atomicity of operations, and the constantly enabled input actions of IOA have a direct counterpart in O'Haskell. For this reason, it will be clear to readers familiar with the literature on self-stabilization, that our implementation resembles the original notation in many ways. However, closer scrutiny reveals a crucial dierence between the two models in the way actions are triggered. In the case of IOA, an input action is executed whenever the preconditions of a corresponding output action makes that action enabled. Normally, output actions are de ned to become disabled as an eect of their execution, but nothing in the IOA model really prevents an output action from being continuously enabled. Such a state will actually result in an arbitrary number of action activations, as long as the precondition holds for the output action in question. In O'Haskell there are no output actions, the closest equivalent must instead be the hypothetical actions we can introduce by lambda abstraction (c.f. the parameters client and network in the de nition of . These actions, as well as any action for that matter, are only invoked explicitly in response to some speci c event. In particular, an action cannot be activated by the static condition that some predicate is true; it must instead be triggered by the state-changing event that makes the predicate become true.1

1 For an analogy, we can think of an IOA as a resemblance to edge-triggered circuit technology.

gated

digital circuit, while an O'Haskell system bears more

This semantic dierence has a direct consequence to our encoding of the UDL interface. In [Var93], a UDL is modeled as an automaton with an input action Send and an output action Free, which is enabled whenever the UDL is ready to accept new packets. In order to know when a packet can be sent, a client of the UDL is required to input the Free action as an acknowledgment, so it can keep an internal ag that mirrors the state of the UDL. This scheme works even if the ag is erroneously set to false, since Free is continuously enabled by the UDL. However, a direct encoding of this UDL model in O'Haskell (e.g. as a template of type AckClient a -> Template (Network a)) would not be self-stabilizing, because a memory fault in the client that resets the free ag just after an acknowledgment has been received irrevocably leads to a passive deadlock. Moreover, mimicking the IOA semantics by having the UDL repeatedly send acknowledgments (invoking ack) when it is ready, implies race conditions that may result in loss of data. An alternative would be to turn both send and ack into synchronous requests, but then active deadlock can occur, and its proper handling would unduly obscure the client code. Instead, in a self-stabilizing context it is arguably better to simply make the UDL repeatedly ask for packets when it is ready, i.e. to use polling. The frequency of these requests can be controlled by a timer, to enable ne-tuning of the load that the polling mechanism incurs on the system. Indeed, such a polling implementation should have essentially the same dynamic behavior as an imagined implementation of an IOA, where a scheduler would need to assert that a continuously enabled action is repeatedly executed at some suitable interval. A detailed, self-stabilizing implementation of the UDL layer itself is beyond the scope of this paper, thus we will only consider its interface henceforth (see e.g. [APSV91] for some implementation details). In doing so we will in fact abstract a bit further, and assume that the interface to the UDL layer is bidirectional, and that it also takes care of routing packets to and from the actual objects controlling the individual links. This makes it possible to treat the UDL layer as a stackable protocol, with the following type: udl :: HW -> PolledClient a -> Template (NullNetwork,HWClient)

Here we assume that HW and HWClient are some abstract types suitable for interfacing with the communication hardware.

4 Implementing self-stabilization We are now ready to discuss the actual self-stabilizing code. In our implementation, it is built up as a separate protocol layer, that can be inserted between any stabilizable protocol and the bottom UDL layer. Since messages to and from the network this way are intercepted by the stabilizer, trac can eectively be subsumed when the stabilizer enters a checking/correction phase. To simplify matters, we assume that the given protocol is prepared to be polled for outgoing data, as it would have to be in a non-stabilizing setting when stacked on top of the UDL abstraction. This restriction is not very serious, since it is straightforward to implement buering in an upper layer should an asynchronous sending interface be desired (we will actually see an example of this in section 5). However, an imperative demand on the protocol in question is that it is locally checkable and correctable, in the sense discussed in section 2. This is captured in O'Haskell by requiring the client of our stabilizer to export the following interface: struct StabilizableClient a s < current_state :: Link -> check_invariants :: Link -> local_reset :: Link ->

PolledClient a = Request s s -> Request Bool Action

The local predicate and the local correction function are here called check_invariants and local_reset, respectively. Injection of request and response packets into the data stream is achieved by means of a datatype: Lu;v

fu;v

data StablePkt a s = DataPkt a | RequestPkt Int SnapMode | ResponsePkt Int SnapMode s data SnapMode

= Snap | Reset

deriving Eq

The Int parameters in these packets are the counter values that implement counter ushing. To enable local checking and eventual correction at regular intervals, we will de ne the lower interface to the stabilizer layer to be of type PeriodicClient (which in turn is de ned using a type Periodic, since the periodic property might be desirable to add other types in the interface hierarchy as well). struct Periodic = tick :: Link -> Action struct PeriodicClient a < PolledClient a, Periodic

It is implied here that the constructor of a particular protocol stack will connect the tick method of our stabilizer to a properly initialized array of timers, one for each link. The actual code for the stabilizer looks as follows: stabilizer :: Int -> Array Link Bool -> StabilizableClient a s -> NullNetwork -> Template (NullNetwork, PeriodicClient (StablePkt a s)) stabilizer dim leader client network = template active := array' (1,dim) False count := array' (1,dim) 0 phase := array' (1,dim) Snap in let tick i = action active!i := True poll i = request if active!i then if leader!i then return (Just (RequestPkt (count!i) (phase!i))) else if phase!i == Reset then client.local_reset i s NullNetwork -> Template (ResetNetwork, PeriodicClient (StablePkt (ResetMsg a) LocalState))

Thanks to subtyping of interfaces, the following protocol stacks are also con gurable: stack = stable_global_reset udl dim stack' = global_reset dim udl dim stack'' = stabilizer dim leader udl dim

Note, though, that we have avoided the question of how the tick method of the stabilizer protocol is connected to an external timer in these examples. In general, initializing a timer is not possible without triggering an action of some sort, so if we want to be able to do this while still using the stacking combinator, it must be generalized to the type of commands instead of templates. We have not done so here, however, in order to keep the presentation simple.

5.2 What about inheritance?

In object-oriented languages it is common to encode extension and modi cation of existing code by means of an inheritance-mechanism. The exact de nition of what inheritance should mean varies, some standpoints would even include the subtyping system of O'Haskell in the concept. In any case, it is unquestionably so that in languages like Java and C++, the natural way to add the methods current_state, check_invariants, and local_reset to an existing, non-stabilizable implementation of global reset, would be to implement them in a separate subclass, related to the original code by implementation inheritance. This style of programming is not supported in O'Haskell. Since state-variables are not rstclass, there is no automatic way of extending a given template with state-dependent code without modifying it. However, in this particular case it appears like modifying instead of inheriting code is just a stylistic dierence, since in order to even write down the implementation of a method like local_reset, the programmer must already have gained access to (not to mention gained a full understanding of) the original source code in any way. We argue that the lack of an inheritance mechanism in O'Haskell is acceptable, especially considering the semantic complexity that is avoided by this restriction. On the other hand, once we have a client protocol that is stabilizable, the stabilizer implementation shows how we can (1) construct a new interface that has all the methods of the client plus a new one (tick), and (2) \override" the old method bodies with new code that occasionally makes calls to the old method implementations (client.poll and client.receive). Indeed this is a form of inheritance (inheritance by delegation, as it should be called according to some de nitions [AC96]), although we are probably much more used to think of this kind of program composition in other terms. However, when combined with the subtyping system of O'Haskell, inheritance by delegation seems like an interesting programming technique, worthy of further exploration.

6 Conclusions In this paper we have shown how a non-trivial network protocol, or more precisely, a protocol transformation scheme, is transcribed from an abstract notation suitable for formal reasoning, into a concrete program written in programming language O'Haskell. The exercise has been relatively painless, which we take as an indication of the merits of O'Haskell as a network programming language. We summarize our conclusions from this undertaking in the following paragraphs.  O'Haskell is expressive enough to allow faithful transcription of both concurrency issues in the form of IOA notation, and informal mathematical language (c.f. especially the encoding of local predicates in the global reset protocol). Our impression is that one does not lose much in clarity and succinctness by going from an abstract formalism to a concrete implementation in O'Haskell, at least not in this particular case. We have not checked whether the formal proofs of self-stabilization still go through as easily using O'Haskell notation, though.  The higher-order nature of O'Haskell allows the original protocol transformation scheme to be replaced by code that is parameterized over an unknown protocol. This modular construction actually leads to increased clarity in the concrete implementation, compared to the original de nition.  Imperative constructs like loops, assignments, and array updates are as easy to use in O'Haskell as they are in any traditional language, despite the fact that O'Haskell retains the property of being what is generally called a pure functional language.  The ability to separate protocol interfaces in an upper and and a lower part makes speci cation and implementation of protocol stacks both natural and exible. The convention we use also blends well with the intuition that a more re ned (smaller) interface plug should t into the hole of more general (larger) socket.



Type inference works perfectly well in this quite realistic example, in spite of an incomplete inference algorithm and several examples non-trivial interaction between polymorphism and subtyping. In fact, all type annotations given in this text are completely super uous, and serve only as a reading aid.

Acknowledgments We are thankful to Philippas Tsigas and Mariana Papatrianta lou for invaluable help on understanding the subject of self-stabilization.

References [AC96] [AG94]

M. Abadi and L. Cardelli. A Theory of Objects. Springer Verlag, 1996. A Arora and MG Gouda. Distributed reset. IEEE Transactions on Computers, 43:1026{ 1038, 1994. [AKY90] Y Afek, S Kutten, and M Yung. Memory-ecient self-stabilization on general networks. In WDAG90 Distributed Algorithms 4th International Workshop Proceedings, SpringerVerlag LNCS:486, pages 15{28, 1990. [APSV91] B Awerbuch, B Patt-Shamir, and G Varghese. Self-stabilization by local checking and correction. In FOCS91 Proceedings of the 31st Annual IEEE Symposium on Foundations of Computer Science, pages 268{277, 1991. [BP89] JE Burns and J Pachl. Uniform self-stabilizing rings. ACM Transactions on Programming Languages and Systems, 11:330{344, 1989. [Dij74] EW Dijkstra. Self stabilizing systems in spite of distributed control. Communications of the ACM, 17:643{644, 1974. [DIM93] S Dolev, A Israeli, and S Moran. Self-stabilization of dynamic systems assuming only read/write atomicity. Distributed Computing, 7:3{16, 1993. [GM91] MG Gouda and N Multari. Stabilizing communication protocols. IEEE Transactions on Computers, 40:448{458, 1991. [IJ90] A Israeli and M Jalfon. Self-stabilizing ring orientation. In WDAG90 Distributed Algorithms 4th International Workshop Proceedings, Springer-Verlag LNCS:486, pages 1{14, 1990. [KP93] S Katz and KJ Perry. Self-stabilizing extensions for message-passing systems. Distributed Computing, 7:17{26, 1993. [LT89] Nancy A. Lynch and Mark R. Tuttle. An introduction to input/output automata. CWI Quarterly, 2(3):219{246, 1989. [NC97] Johan Nordlander and Magnus Carlsson. Reactive Objects in a Functional Language { An escape from the evil \I". In Proceedings of the Haskell Workshop, Amsterdam, Holland, 1997. [Nor98] Johan Nordlander. Pragmatic Subtyping in Polymorphic Languages. In ICFP, Baltimore,MD, September 1998. [PH96] J. Peterson and K. Hammond, editors. The Haskell 1.3 Report. YALEU/DCS/RR1106. Yale University Research Report, 1996. [Var93] G Varghese. Self-stabilization by local checking and correction (Ph.D. thesis). Technical Report MIT/LCS/TR-583, MIT, 1993.

A A stabilizable global reset protocol struct Resettable = reset :: Action struct ResetNetwork a < Network a, Resettable struct ResetClient a < Client a, Resettable data ResetMsg a = | | |

DataMsg AbortMsg AckMsg ReadyMsg

a Int

instance Eq (ResetMsg a) where DataMsg _ == DataMsg _ = AbortMsg d == AbortMsg d' = AckMsg == AckMsg = ReadyMsg == ReadyMsg = _ == _ =

True d == d' True True False

data ResetMode = AbortMode | ConvergeMode | ReadyMode deriving Eq global_reset :: Int -> ResetClient a -> NullNetwork -> Template (ResetNetwork a, StabilizableClient (ResetMsg a) LocalState) global_reset dim client network = template ack_pend := [] parent := Nothing distance := 0 buffer := array' (1,dim) [] queue := array' (1,dim) [] in let mode [] Nothing = ReadyMode mode [] (Just _) = ConvergeMode mode _ _ = AbortMode send i m = action if mode ack_pend parent == ReadyMode then enqueue i (DataMsg m) enqueue i m = do queue!i := queue!i ++ [m] poll i = request case queue!i of m:ms -> queue!i := ms return (Just m) [] -> return Nothing

receive i m = action do_receive i m do_receive i (DataMsg m) = do if mode ack_pend parent == ReadyMode then client.receive i m else do buffer!i := buffer!i ++ [m] do_receive i (AbortMsg dist) = do if mode ack_pend parent == ReadyMode then propagate (Just i) dist if dist == 0 || mode ack_pend parent /= ReadyMode then enqueue i AckMsg buffer!i := [] do_receive i AckMsg = do if i `elem` ack_pend then do ack_pend := delete i ack_pend case mode ack_pend parent of ConvergeMode -> enqueue (the parent) AckMsg ReadyMode -> reset_completed AbortMode -> done do_receive i ReadyMsg = do if mode ack_pend parent == ConvergeMode && parent == Just i then parent := Nothing reset_completed reset_completed = do client.reset forall [1..dim] (\j -> do enqueue j ReadyMsg) forall [1..dim] (\i -> do forall (buffer!i) (\m -> do client.receive i m)) propagate par dist = do parent := if dist /= 0 then par else Nothing distance := dist forall [1..dim] (\j -> do enqueue j (AbortMsg (distance+1))) ack_pend := [1..dim] reset = action propagate Nothing 0 current_state i = request return (struct mode you_parent you_pending ack_in_q rdy_in_q distance

= = = = = =

mode ack_pend parent parent == Just i i `elem` ack_pend AckMsg `elem` queue!i ReadyMsg `elem` queue!i distance)

check_invariants i remote = request let p0 = i `elem` ack_pend p1 = AbortMsg (distance+1) `elem` queue!i p2 = remote.mode == AbortMode && remote.you_parent p3 = remote.ack_in_q p4 = parent == Just i p5 = mode ack_pend parent == ConvergeMode p6 = AckMsg `elem` queue!i p7 = not remote.you_pending && remote.mode /= ReadyMode p8 = remote.rdy_in_q p9 = f1 (queue!i) p10 = mode ack_pend parent == ReadyMode p11 = distance == remote.distance + 1 f1 f1 f1 f1

[] [ReadyMsg] [DataMsg _] (m:ms)

= = = =

False True True f1 ms

f2 f2 f2 f2 f2 f2

[AbortMsg _,AckMsg] = True [AckMsg,ReadyMsg,AbortMsg _] = True [ReadyMsg,AbortMsg _] = True (AckMsg:ReadyMsg:ms) = f3 ms (ReadyMsg:ms) = f3 ms ms = f3 ms

f3 [] = True f3 (DataMsg _:ms) = f3 ms f3 _ = False pA pB pC pD pR pP pQ

= = = = = = =

p0 == (p1 || p2 || p3) length (filter id [p1,p2,p3])