M.S., Kansas State University, 1989. A DISSERTATION submitted in partial ful llment of the requirements for the degree. DOCTOR OF PHILOSOPHY.
Quorum Structures in Distributed Systems by Mitchell L. Neilsen M.S., Kansas State University, 1989
A DISSERTATION submitted in partial ful llment of the requirements for the degree
DOCTOR OF PHILOSOPHY Department of Computing and Information Sciences KANSAS STATE UNIVERSITY Manhattan, Kansas 1992 Approved by: Major Professor
Acknowledgements First of all, I would like to thank my major professor, Dr. Masaaki Mizuno, for being a continual source of inspiration, encouragement, and friendship. During the many hours we spent doing research together, I only learned a few words in Japanese. However, I learned much more than I can ever acknowledge from my \sensei". Next, I would like to acknowledge the many insightful comments provided by my committee members: Dr. Rodney Howell, Dr. Virgil Wallentine, Dr. Todd Cochrane, and Dr. Victor Wallace. During the past two years, it has been my good fortune to have had the opportunity to discuss these ideas with many of the authors to which I refer in the dissertation. In particular, I am indebted to the following researchers: Dr. Michel Raynal (at IRISA), Dr. Akhil Kumar (at Cornell), Dr. Tiko Kameda (at Simon Fraser), and Dr. Toshi Ibaraki (at Kyoto). I eagerly look forward to working in this area with these outstanding researchers. Also, I want to thank my dear wife, Rebecca, and daughters, Anne and Beth, for their constant encouragement and love. I thank my parents, Thomas and Rosalin, for always believing in me and for allowing me to take chances, and I thank my twin brother, Michael, for challenging me to attend graduate school. Finally and foremost, I thank God for giving me the opportunity to pursue a dream, and for the blessing of all who made that dream come true.
ii
Contents 1 Introduction
1
2 De nitions and Properties
7
1.1 Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Quorum Structures . . . . . . . . 2.2 Properties . . . . . . . . . . . . . 2.2.1 Quorum sets and coteries . 2.2.2 Bicoteries . . . . . . . . .
3 Applications
. . . .
. . . .
. . . .
. . . .
. . . .
3.1 Decentralized Consensus Protocols . . . . 3.1.1 Generalized consensus protocols . . 3.1.2 Quorum-based consensus protocols 3.2 Distributed Mutual Exclusion Algorithms . 3.3 Replica Control Protocols . . . . . . . . .
4 Related Research
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
4.1 Game Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Reliability Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iii
4 5
7 9 9 13
15 15 16 19 24 26
30 30 34 38
5 Simple Quorum Structures 5.1 Quorum Consensus . . 5.2 Grid Protocols . . . . . 5.3 Disjoint Set Protocols . 5.3.1 Coteries . . . . 5.3.2 Bicoteries . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
42 42 43 45 45 47
6 Composition
50
7 Composite Structures
62
6.1 Composition of Quorum Sets . . . . . . . . . . . . . . . . . . . . . 6.2 Properties of Composition . . . . . . . . . . . . . . . . . . . . . .
7.1 7.2 7.3 7.4 7.5
Tree Protocol . . . . . . . . . . . Hierarchical Quorum Consensus . Hybrid Replica Control Protocols RST Protocol . . . . . . . . . . . Arbitrary Network Protocol . . .
8 Composite Structure Evaluation 8.1 Quorum Containment Test . 8.1.1 Method . . . . . . . 8.1.2 Example . . . . . . . 8.1.3 Complexity . . . . . 8.2 Availability . . . . . . . . . 8.2.1 Method . . . . . . . 8.2.2 Correctness . . . . . 8.2.3 Example . . . . . . . 8.2.4 Complexity . . . . . 8.2.5 Bicoteries . . . . . . 8.3 Measure of Power . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
iv
. . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
50 52
62 66 68 70 70
72 72 72 73 74 75 76 77 82 83 85 85
8.3.1 8.3.2 8.3.3 8.3.4
Banzhaf index . Method . . . . Correctness . . Example . . . .
9 Conclusion
9.1 Contributions . . . . 9.2 Open Questions . . . 9.2.1 Optimization 9.2.2 Enumeration
. . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
v
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
86 87 87 91
93 93 94 95 99
List of Figures Figure 3.1 Communication Graphs . . . . . . . . . . . . . . . . . . . . Figure 3.2 Distributed Mutual Exclusion . . . . . . . . . . . . . . . . .
17 25
Figure 4.1 Bridge Structure . . . . . . . . . . . . . . . . . . . . . . . . Figure 4.2 Finite Projective Plane . . . . . . . . . . . . . . . . . . . . . Figure 4.3 Hypergraph . . . . . . . . . . . . . . . . . . . . . . . . . . .
36 38 40
Figure 5.1 Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
Figure 7.1 Tree . . . . . . . . Figure 7.2 Tree (Q1) . . . . . Figure 7.3 Tree (Qa) . . . . . Figure 7.4 Tree (Qb) . . . . . Figure 7.5 Hierarchical Tree . Figure 7.6 Grid-set Protocol . Figure 7.7 Arbitrary Network
. . . . . . .
63 64 64 64 66 69 71
Figure 8.1 Quorum Containment Test . . . . . . . . . . . . . . . . . . . Figure 8.2 Availability of a Composite Quorum Set . . . . . . . . . . . Figure 8.3 Arbitrary Network Node Reliability . . . . . . . . . . . . . .
72 76 82
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
vi
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
List of Tables Table 3.1 Quorums (Li) . . . . . . . . . . . . . . . . . . . . . . . . . . Table 3.2 Quorums (Ci) . . . . . . . . . . . . . . . . . . . . . . . . . . Table 3.3 Quorums (Hi ) . . . . . . . . . . . . . . . . . . . . . . . . . .
21 22 23
Table 4.1 Game Theory Structures . . . . . . . . . . . . . . . . . . . . Table 4.2 Reliability Theory Structures . . . . . . . . . . . . . . . . . .
33 37
Table 7.1 Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
Table 8.1 Banzhaf Index . . . . . . . . . . . . . . . . . . . . . . . . . .
92
Table 9.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Table 9.2 Enumeration . . . . . . . . . . . . . . . . . . . . . . . . . . .
93 99
vii
Chapter 1 Introduction A distributed system is an interconnected collection of several autonomous computers [38]. The models generally used to model distributed systems are process models. In a process model, computation is represented as the concurrent execution of sequential processes. These models can be further subdivided by the mechanism they use for interprocess communication. One of the most commonly used mechanisms is message-passing. Implicitly, there are several assumptions made in the message-passing model: 1. Message transmission is the main cost of execution. 2. Processes are independent; that is, a process can continue to operate even if other processes fail. Message-passing models can be further classi ed by four distinguishing features: network topology, synchrony, types of failure, and message bu ering. 1. The network topology is de ned by a communication graph. For example, the processes may be fully connected or arranged in a ring. Secondly, processes may or may not have knowledge of all other processes in the system. 2. The model is completely asynchronous if messages are eventually delivered and processes respond to the messages, but no assumption is made 1
about how long message transmission might take. At the other extreme, the model is synchronous if the computation proceeds in a sequence of rounds. 3. There are two types of failures: process failures and communication line failures. A common assumption is that communication lines do not fail. There are various assumptions that can be made about process failures. We will list them from the least restrictive to the most restrictive:
Byzantine failures: The failure of one process cannot a ect the communication between other processes.
omission failures: Processes can only fail to send or receive some messages when they fail.
halting failures: A failed process simply halts and does nothing. fail-stop failures: A failed process simply halts, and other processes can detect the failure.
4. Various assumptions can be made about message bu ering. A common assumption is that the bu er size is in nite and message ordering is FIFO; that is, messages, which are not lost, are received in the same order in which they are sent. When designing a distributed algorithm, it is important to clearly de ne the assumptions made about the underlying distributed system. In many distributed algorithms, there are two kinds of fundamental properties that are of concern: safety and liveness. A safety property ensures that an undesirable outcome does not occur; whereas, a liveness property ensures that a desirable outcome eventually occurs. For instance, in a distributed mutual exclusion algorithm, the property that at most one node may enter the critical section is a safety property. The property that at least one node must be able 2
to enter the critical section in nite time, if there are no failures, is a liveness property. Quorum-based algorithms are an important class of distributed algorithms because they gracefully tolerate node and communication line failures. Maekawa's distributed mutual exclusion algorithm is a classical example of a quorum-based algorithm [40]. If a node receives permission from a quorum of nodes, then the node may enter the critical section. In order to ensure safety, any two quorums must have a non-empty intersection and a node may only give permission to one node at a time. Such a collection of quorums is called a coterie [24]. GarciaMolina and Barbara de ned a class of coteries that exhibit better performance than other coteries. The coteries in this class are called nondominated coteries. Several authors have shown that many other problems can be solved by using quorum-based algorithms, including decentralized consensus, leader election, and replica control. For instance, in a replica control protocol, instead of having a single set of quorums, two types of quorums are used: read quorums and write quorums. The safety aspect of quorum-based protocols can be formalized in terms of the data structures that are used by the protocols [2, 7, 21, 28]. These data structures are a generalization of an idea which was rst proposed by Lamport [36]. In this dissertation, we formally study these structures, which we call quorum structures. In the distributed systems area, there have been several methods proposed to construct quorum structures. As observed by Ibaraki and Kameda, several equivalent methods have developed independently in other areas of research, including: game theory, reliability theory, and graph theory [28]. One very well known method is to use quorum consensus (weighted voting) [25]. However, Garcia-Molina and Barbara have shown that quorum consensus cannot be used to construct all nondominated coteries. Many algorithms use the simplifying assumption that the distributed system is fully connected and links 3
do not fail. In such a case, Tong and Kain have shown that quorum consensus can be used to construct optimal coteries (in terms of availability) [77]. However, Tang and Natarajan have shown that for any real system (not fully connected or not having perfect links), this may not be the case [75]. Several authors have tried to optimize quorums in general, either by formulating the problem as an integer linear programming problem [75] or by using an iterative method, such as simulated annealing [31, 34, 53]. However, they have only been successful for less than ten nodes. Also, most of these studies have concentrated on optimizing availability, without considering other measures, such as communication cost. Some authors have tried to structure the nodes to improve performance, not only in terms of availability, but also in terms of communication cost. For instance, Kumar proposed hierarchical quorum consensus [32]. The nodes are partitioned into a multi-level hierarchy, and quorum consensus is used on each level. Agrawal and El Abbadi proposed two methods to construct quorum structures: a grid protocol and a tree protocol [2]. They also proposed protocols that use a twolevel hierarchy, called hybrid replica control protocols. On the rst level, quorum consensus is used; and on the second level, either the grid protocol or the tree protocol is used. Recently, Rangarajan proposed another protocol that uses a two-level hierarchy [56]. On the rst level, Maekawa's protocol, based on nite projective planes, is used [40]; and on the second level, quorum consensus is used.
1.1 Thesis In this dissertation, a more general method is presented. We de ne two di erent types of quorum structures: simple and composite. Composite structures are quorum structures constructed by using composition. Simple structures are all other non-empty quorum structures. We show that composition can be used to construct a wide range of di erent quorum structures. In particular, the tree protocol, hierarchical quorum consen4
sus, the hybrid replica control protocols, and the RST protocol can all be generalized by using composition. In general, any structures, simple or composite, can be used to construct composite structures. Other protocols impose restrictions on the types of simple structures that can be used to construct composite structures. Finally, we present several methods to eciently evaluate composite structures. These include: the quorum containment test to determine if a given set contains a quorum, a method to compute the availability of a composite quorum set, and a method to compute the importance of each node in the system, called the absolute Banzhaf index of a node. In summary, the thesis of this dissertation is three-fold: 1. Composition provides a natural method to construct quorum structures in an arbitrary network or even in a collection of interconnected networks. 2. Composition generalizes several previous protocols. 3. Composite structures can be easily and eciently evaluated.
1.2 Organization The organization of this dissertation is as follows:
Chapter 2: Formal de nitions of quorum structures are presented, including quorum sets, coteries, and bicoteries. Also, de nitions of nondominated quorum structures are presented. Then, some properties that belong to folklore or that follow directly from previous work are presented.
Chapter 3: Applications of quorum structures are brie y surveyed, including
decentralized consensus protocols, distributed mutual exclusion algorithms, and replica control protocols.
Chapter 4: Related research concepts are presented. These concepts are
taken from game theory, reliability theory, and graph theory. Theoretical 5
developments have taken place independently in these areas. One goal of this dissertation is to unify some of these concepts and developments.
Chapter 5: Various existing methods to construct quorum structures are reviewed, including: quorum consensus (weighted voting), grid protocols, and disjoint set protocols.
Chapter 6: The composition of quorum structures is introduced. Then, properties satis ed by composition are presented.
Chapter 7: Several methods to construct composite structures are presented. Some previously published protocols are presented and generalized by using composition, including: the tree protocol, hierarchical quorum consensus, the hybrid replica control protocols, and the RST protocol.
Chapter 8: Several methods to eciently evaluate composite structures are presented. These include:
{ a test to determine if a given set contains a quorum, called the quorum containment test, { a method to compute the availability of a composite quorum set, and { a method to compute the importance of each node in the system. Chapter 9: A summary of the results and open problems are presented.
6
Chapter 2 De nitions and Properties In this chapter, formal de nitions of quorum structures are presented. Then, properties that will be used in subsequent proofs are presented. These properties belong to folklore or follow directly from previous work.
2.1 Quorum Structures Several authors have de ned quorum structures that can be used in a wide variety of distributed protocols [7, 21, 24, 28]. In this section, these structures are de ned. Let U denote a non-empty set of nodes. The term node refers to a computer in a network or a copy of some data object in a replicated database. A collection of sets, Q, is a quorum set under U if 1. (8G 2 Q) [ G 6= ; and G U ]. 2. (Minimality): (8G; H 2 Q) [ G 6 H ]. The sets G 2 Q are called quorums. For example, let U = fa; b; c; dg. Then, Q = ffa; bg; fb; cgg is a quorum set under U . Note that not all nodes must appear in some quorum; in particular, node d does not appear in either quorum of Q. Nodes that appear in some quorum are called used nodes. 7
A similar notion was rst proposed by Sperner [71]. A family Q of subsets of U is a Sperner family if Q is a quorum set and Q 6= ;. Thus, the only di erence in de nitions is that the empty set is not a Sperner family. For any set S , let jS j denote the cardinality of S . Sperner proved that the number of quorums in a Sperner family is bounded above by bN=N 2c , where N = jU j. A quorum set Q can be represented by the set of all subsets of U containing a quorum of Q. We call such a set the acceptance set corresponding to Q, and denote it by A(Q).
A(Q) = fH U j G H for some G 2 Qg In the above example, with Q = ffa; bg; fb; cgg under U = fa; b; c; dg, the corresponding acceptance set A(Q) is given by:
A(Q) = ffa; bg; fb; cg; fa; b; cg; fa; b; dg; fb; c; dg; fa; b; c; dgg De ne min(S ) = fG 2 S j (8H 2 S ) [ H 6 G ] g for any set S of non-empty subsets of U . Then, Q = min(A(Q)) for any quorum set Q. There is a one-to-one correspondence between acceptance sets and quorum sets. Two quorum sets under U , say Q1 and Q2, are isomorphic if there exists an isomorphism : U ! U such that Q2 = f f(x) j x 2 Gg j G 2 Q1g. Let Q1 and Q2 be quorum sets under U . Then, Q1 dominates Q2 if 1. Q1 6= Q2. 2. (8H 2 Q2) [ 9G 2 Q1 such that G H ]. A quorum set, Q under U , is dominated if there is another quorum set under U that dominates Q. If there is no such quorum set, then Q is nondominated. 8
A quorum set, Q, is a coterie under U if the intersection property is satis ed; that is, (8G; H 2 Q) [ G \ H 6= ; ]. A coterie Q = fGg, containing a single quorum, is called a singleton coterie. A coterie, Q under U , is dominated if there is another coterie under U that dominates Q. If there is no such coterie, then Q is nondominated. Let Q be a quorum set under U . Then, a complimentary quorum set, Qc, is another quorum set under U such that (8G 2 Q) (8H 2 Qc) [ G \ H 6= ; ]. The pair B = (Q; Qc) is called a bicoterie under U . If Q or Qc is a coterie, then the pair B is called a semicoterie. Suppose that B1 = (Q1; Qc1) and B2 = (Q2; Qc2) are bicoteries under U . Then, B1 dominates B2 if 1. B1 6= B2; that is, Q1 6= Q2 or Qc1 6= Qc2. 2. (8H 2 Q2) [ 9G 2 Q1 such that G H ]. 3. (8H 2 Qc2) [ 9G 2 Qc1 such that G H ]. A bicoterie, B under U , is dominated if there is another bicoterie under U that dominates B. If there is no such bicoterie, then B is nondominated.
2.2 Properties We start by considering properties satis ed by quorum sets and coteries.
2.2.1 Quorum sets and coteries First, we show that there is a one-to-one correspondence between quorum sets and acceptance sets. 9
Lemma 2.1: Let Q1 and Q2 denote quorum sets under U . Then, A(Q1) = A(Q2)
if and only if Q1 = Q2.
Proof: Let Q1 and Q2 denote quorum sets under U . Suppose that A(Q1) = A(Q2). We will assume that Q1 = 6 Q2 and derive a contradiction. Since Q1 6= Q2,
either
1. (9G1 2 Q1) such that G1 62 Q2, or 2. (9G2 2 Q2) such that G2 62 Q1. Without loss of generality, we will assume that there is a quorum G1 2 Q1 such that G1 62 Q2. By the de nition of an acceptance set, G1 2 A(Q1). Since A(Q1) = A(Q2), it follows that G1 2 A(Q2) as well. So, there is a quorum G2 2 Q2 such that G2 G1. Since G1 62 Q2, it follows that G2 6= G1. Thus, G2 G1 . Since G2 2 Q2, it is also in A(Q2). However, G2 62 A(Q1), for otherwise there would exist G3 2 Q1 such that G3 G2 G1 and this contradicts the minimality of Q1. Since G2 2 A(Q2) and G2 62 A(Q1), we obtain A(Q1) 6= A(Q2), and this is a contradiction. Thus, Q1 = Q2. On the other hand, suppose that Q1 = Q2. Let H1 2 A(Q1). By the de nition of an acceptance set, there is a quorum G1 2 Q1 such that G1 H1. Since Q1 = Q2, G1 is also in Q2, and it follows that H1 2 A(Q2). Similarily, if H2 2 A(Q2), then H2 2 A(Q1). Thus, A(Q1) = A(Q2). Therefore, A(Q1) = A(Q2) if and only if Q1 = Q2. 2 Let Q be a quorum set under U . A transversal of Q is a subset of U that intersects with all quorums in Q; that is, H U is a transversal of Q if G \ H 6= ; for any G 2 Q. A minimal transversal is a transversal, H , such that any subset of H is not a transversal. The set of all minimal transversals of Q, denoted by 10
Tr(Q), is a complementary quorum set. Tr(Q) = min(fH U j (8G 2 Q) [ G \ H 6= ; ] g) In the literature, Tr(Q) is also called the antiquorum set of Q [7].
Lemma 2.2: De ne P (U ) to be the powerset of U . If Q is a quorum set under U , then
A(Tr(Q)) = P (U ) ? fU ? G j G 2 A(Q)g
Proof: Let H 2 A(Tr(Q)). Then, there is a G 2 Tr(Q) such that G H . Assume that H 62 P (U ) ? fU ? G j G 2 A(Q)g. Then, H 2 fU ? G j G 2 A(Q)g. So, there is a K 2 A(Q) such that H = U ? K . Since G H and H \ K = ;, it follows that G \ K = ;. Also, since K 2 A(Q), there is an L 2 Q such that L K . Finally, since L K and G \ K = ;, it follows that G \ L = ;. This is a contradiction, because G 2 Tr(Q) and L 2 Q. Thus, our assumption is false, and H 2 P (U ) ? fU ? G j G 2 A(Q)g. On the other hand, suppose that H 2 P (U ) ? fU ? G j G 2 A(Q)g. Assume that H 62 A(Tr(Q)). Then, there is no G 2 Tr(Q) such that G H . Since Tr(Q) is the set of all minimal transversals, there must be some K 2 Q such that H \ K = ;. Let L = U ? H . Then, K L, so L 2 A(Q), and it follows that H 2 fU ? G j G 2 A(Q)g because H = U ? L. This is a contradiction. Thus, H 2 A(Tr(Q)). Therefore, A(Tr(Q)) = P (U ) ? fU ? G j G 2 A(Q)g. 2
11
Lemma 2.3 [7]: Let Q be a quorum set under U . Then, any other complementary quorum set is dominated by Tr(Q). Nondominated quorum sets are not very interesting. If U is a non-empty set of nodes, then the only nondominated quorum set under U is Q = ffxg j x 2 U g. However, there may be many nondominated coteries under U . The following theorem makes it easier to determine if a coterie is dominated.
Theorem 2.1 [24]: Let Q be a coterie under a non-empty set U . Then, Q is dominated if and only if there exists a set H U such that: 1. G 2 Q ) G 6 H . 2. G 2 Q ) G \ H 6= ;. For example, let Q2 = ffa; cg; fb; cgg. The set H = fa; bg satis es the properties in Theorem 2.1. Informally, we say that H dominates Q2. Note that Q2 is dominated by Q1 = ffa; bg; fa; cg; fb; cgg. The coterie Q1 is nondominated. Coterie Q2 is also dominated by the coterie ffcgg. Finally, note that if H satis es the properties in Theorem 2.1, then so does the set U ? H . In the above example, U ? H = fcg.
Theorem 2.2: Let Q be a coterie under a non-empty set U , where jU j = N . Then, the following statements are equivalent: 1. Q is nondominated. 2. jA(Q)j = 2N ?1. 3. Tr(Q) = Q. 12
Proof: By Theorem 2.1, Q is nondominated if and only if Tr(Q) = Q. Suppose that Q is nondominated, and hence Tr(Q) = Q. By Lemma 2.2, A(Tr(Q)) = P (U ) ?fU ? G j G 2 A(Q)g, and so it follows that jA(Tr(Q))j = jP (U )j?jA(Q)j. Thus, jA(Tr(Q))j + jA(Q)j = 2N . By Lemma 2.1, A(Q) = A(Tr(Q)), and so it follows that jA(Q)j = 2N ?1. Finally, suppose that jA(Q)j = 2N ?1. If H 2 A(Q), then U ? H 62 A(Q) because of the intersection property imposed on coteries. So, A(Q) is the largest possible acceptance set; that is, given any H 2 P (U ), either H or U ? H must be in A(Q). Assume that Q is dominated. By Theorem 2.1, there must exist some H U that dominates Q. Thus, H 62 A(Q). Recall that U ? H also dominates Q, so U ? H 62 A(Q). This is a contradiction. Therefore, all three statements are equivalent. 2
2.2.2 Bicoteries Let Q be a quorum set under U . Let Qc be a complimentary quorum set. The pair B = (Q; Qc) is called a quorum agreement if Qc = Tr(Q). It is easy to show that quorum agreements are the same as nondominated bicoteries. To avoid confusion, we will use the term \nondominated bicoterie" in the rest of the paper. For any nondominated bicoterie, (Q; Qc), there are only three possibilities [7, 28]: 1. Q and Qc are nondominated coteries, and Q = Qc; 2. Q is a dominated coterie, and Qc is not a coterie (or equivalently, Qc is a dominated coterie, and Q is not a coterie); or 3. neither Q nor Qc is a coterie. Next, we state, without proof, the analog of Theorem 2.1 for bicoteries. The proof is essentially the same as the proof of Theorem 2.1. 13
Theorem 2.3: Let B = (Q; Qc) be a bicoterie under a non-empty set U . Then, B is dominated if and only if there exists a set H U such that: 1. G 2 Q ) G \ H 6= ;. 2. G 2 Qc ) G 6 H .
Theorem 2.4: Let B = (Q; Qc) be a bicoterie under a non-empty set U , where jU j = N . Then, the following statements are equivalent: 1. B is nondominated. 2. jA(Q)j + jA(Qc)j = 2N . 3. B is a quorum agreement; that is, Qc = Tr(Q).
Proof: By Lemma 2.3 and Theorem 2.3, B is a nondominated bicoterie if and only if B is a quorum agreement. Suppose that B = (Q; Qc) is nondominated, and hence Qc = Tr(Q). Then, by Lemma 2.2, A(Qc) = P (U ) ? fU ? G j G 2 A(Q)g. Since jP (U )j = 2N , it follows that jA(Q)j + jA(Qc)j = 2N . On the other hand, suppose that jA(Q)j+jA(Qc)j = 2N . Assume that Qc 6= Tr(Q). By Lemma 2.1, A(Qc) 6= A(Tr(Q)), and it follows that there must exist some H 2 A(Tr(Q)) such that H 62 A(Qc). So, jA(Qc)j < jA(Tr(Q))j. However, if G 2 A(Q), then U ? G 62 A(Tr(Q)) because of the intersection property. So, jA(Q)j + jA(Tr(Q))j 2N . Hence, jA(Q)j + jA(Qc)j < 2N , and this is a contradiction. Therefore, all three statements are equivalent. 2 14
Chapter 3 Applications In this chapter, several applications based on quorum structures are described. Such applications include: decentralized consensus, distributed mutual exclusion, replica control, leader election, and termination detection protocols [7]. We will brie y describe a few of these protocols.
3.1 Decentralized Consensus Protocols Consider a distributed system in which N computers (also called nodes) are connected by a point-to-point, fully connected, reliable network. Message exchange is completely asynchronous. The goal of decentralized consensus is to exchange information between nodes so that each node acquires the information originally held by every other node in the system. Applications of such protocols include extrema nding, coordination of distributed check points, and maintenance of transaction atomicity [35]. If one process is faulty, distributed consensus may be impossible [20]. Unlike other consensus protocols [15, 18], we assume that there are no faulty processes executing the protocol. Instead, our objective is to minimize the number of messages required to achieve consensus. A naive approach to attain decentralized consensus is to require each node to send its information to every other node. This approach requires one round of 15
message exchange. Since there are N nodes in the system and each node sends (N ? 1) messages, a total of N (N ? 1) messages are required. Lakshman and p Agrawala presented a protocol which requires O(N N ) messages and two rounds of message exchange [35]. The protocol is based on nite projective planes. Yuan and Agrawala proposed a protocol based on k-dimensional arrays, where k is the number of rounds of message exchange [80] . For k rounds of message exchange, the protocol requires a total of kN (N 1=k ? 1) messages. We introduce a unifying framework which generalizes decentralized consensus protocols which use k rounds of message exchange. A necessary and sucient property for these consensus protocols is presented. Yuan and Agrawala's kdimensional array based protocol can be viewed in this unifying framework. We introduce a new class of protocols within the unifying framework, called quorumbased protocols [48]. Quorum-based protocols are obtained by observing only the necessary properties which must be maintained. They use two rounds of message exchange and both rounds of message exchange can use the same connections. This property can be exploited if the underlying network provides connection-oriented service. By applying nite projective planes directly, we obtain a new protocol which reduces the number of required messages to half the number required in Lakshman and Agrawala's protocol.
3.1.1 Generalized consensus protocols All decentralized consensus protocols, which use k rounds of message exchange, can be generalized by viewing each round of message exchange as a directed graph. Let Rp denote the incidence matrix of the pth round of message exchange. 8 > < Rp[i; j ] = > :
1 if (i = j ) or node i sends a message to node j in the pth round of message exchange 0 otherwise 16
Decentralized consensus protocols, which use k rounds of message exchange, can be described as follows: 1. In the rst round of message exchange, node i sends its own information to all nodes in fj j R1[i; j ] = 1; i 6= j g. 2. In subsequent rounds, 2 p k, node i waits to receive information from all nodes in fj j Rp?1[j; i] = 1; i 6= j g. After receiving this information, node i combines the information and sends the combined information to all nodes in fj j Rp[i; j ] = 1; i 6= j g. 3. After the nal round of message exchange, node i waits to receive information from all nodes in fj j Rk [j; i] = 1; i 6= j g. After receiving this information, node i combines the information and terminates the protocol. For example, consider the message exchange denoted by the graphs given below in Figure 3.1. In the rst round of message exchange, node 1 sends its own information to node 2, node 2 sends its own information to node 3, and node 3 sends its own information to node 1.
1 S o S S
S
2
1
-
3
S
/
2 R1 = 64
1 1 0 0 1 1 1 0 1
S S
3 7 5
Figure 3.1 Communication Graphs
17
7
3
S S w S
2
2 R2 = 64
1 0 1 1 1 0 0 1 1
3 7 5
Lemma 3.1: Let denote the usual matrix product. In a decentralized consensus
protocol, which uses k rounds of message exchange, (mp=1Rp)[i; j ] = r, for some m, 2 m k and r 1, if and only if information ows through exactly r di erent paths of length m, between nodes i and j , in m rounds of message exchange.
Proof: Let nodes i and j be any two nodes in the system. We proceed by induction on m.
Basis Step: Suppose that m = 1. Then, (mp=1Rp)[i; j ] = R1[i; j ]. From the
de nition of R1, it follows that R1[i; j ] = 1 if and only if information ows through exactly one path of length one between nodes i and j , in one round of message exchange. Note that the trivial path from node i to itself is counted. Induction Step: Let 2 n k. Suppose that the lemma is satis ed for ?1 R ). Then, (n R )[i; j ] = r if and only all m < n. Let R[1;n?1] = (np=1 p p=1 p if ((R[1;n?1])(Rn))[i; j ] = r. Computing the usual matrix product, we see that PN 0 a=1((R[1;n?1] [i; a])(Rn[a; j ])) = r if and only if there exist exactly r di erent nodes, say fa1; a2; ar0 g, such that R[1;n?1][i; as] = rs and Rn[as; j ] = 1 for 0 1 s r0, and Prs=1 ((R[1;n?1][i; as])(Rn[as; j ])) = r. By induction, this is true if and only if there are exactly rs di erent paths from node i to node as and one path from node as to node j for 1 s r0. Therefore, (np=1Rp)[i; j ] = r if and 0 only if information ows through exactly r = Prs=1 rs di erent paths of length n, from node i to node j , in n rounds of message exchange. 2
Theorem 3.1: Let R = kp=1Rp. A necessary and sucient property for all decentralized consensus protocols, which use k rounds of message exchange, is that R JN , where JN [i; j ] = 1 for 1 i; j N . We call the matrix R the reachability matrix.
Proof: To derive a contradiction, assume that R < JN . Then, there exist nodes i and j such that R[i; j ] = 0. By Lemma 3.1, information does not pass from node 18
i to node j . This is a contradiction, because all consensus protocols require every node to send and receive information from every other node in the system. On the other hand, suppose that R JN . For any pair of nodes i and j , R[i; j ] 1. By Lemma 3.1, there is at least one path of information ow from node i to node j . Since this property holds for each pair of nodes, all nodes will receive information from all other nodes in the system. 2
3.1.2 Quorum-based consensus protocols In this section, we present a new class of decentralized consensus protocols called quorum-based protocols. First, a quorum is assigned to each node. Let Gi be the quorum assigned to node i for 1 i N . Note that the same quorum can be assigned to more than one node. Let Hi = fj j i 2 Gj g. The following two rounds of message exchange are performed: 1. In the rst round of message exchange, node i sends its information to all nodes in Gi, excluding node i. Node i receives messages from all nodes in Hi, excluding from node i. 2. In the second round of message exchange, node i sends messages to all nodes in Hi, excluding node i; that is, node i sends messages to all nodes that node i has just received messages from. The message contains all of the information node i has received in the rst round and its own information. Node i receives messages from all nodes in Gi, excluding node i.
Theorem 3.2: Using a quorum-based protocol, after the second round of message
exchange, each node holds the information originally held by every other node in the system. 19
Proof: De ne a two dimensional array R1 such that R1[i; j ] = 1 if (j 2 Gi or
i = j ), and R1[i; j ] = 0 otherwise. Let R2 = RT1 , where RT1 is the transpose of R1. From the de nition of Hi , it is obvious that R2[i; j ] = 1 if (j 2 Hi or i = j ), and R2[i; j ] = 0 otherwise. Let R = R1 R2. Because of the intersection property, for any pair of nodes i and j , there is at least one node k such that R1[i; k] = 1 and R2[k; j ] = 1. Thus, R[i; j ] 1 for 1 i; j N . From Theorem 3.1, after the second round of message exchange, each node holds the information originally held by every other node. 2 Note that in this protocol, we only require the intersection property, and not minimality. If the network provides bidirectional connection-oriented service, it may be desirable to have both the rst and second rounds of message exchange use the same connection. The quorum-based protocols satisfy this property. In the rst round, node i sends messages to the nodes in Gi and receives messages from the nodes in Hi . Thus, node i needs to open connections with all of the nodes in Gi [ Hi . In the second round, node i sends messages to the nodes in Hi and receives messages from the nodes in Gi . Therefore, the second round of message exchange requires the same connections as the rst round. Obviously, the network must allow the opening of a sucient number of connections to make this a viable alternative.
Lakshman and Agrawala's protocol Lakshman and Agrawala's protocol uses a communication scheme based on nite projective planes. The N nodes are divided (with overlap) into N quorums: L1; : : :; LN , satisfying the following properties [35, 40]: 1. Each i 2 Li for 1 i N . 2. Li \ Lj 6= ; for 1 i; j N . 20
3. jLij = m + 1 for 1 i N , where m is called the order of the nite p projective plane and N = m2 + m + 1. Thus, m is approximately N . 4. Each i, 1 i N , is contained in (m + 1) quorums. A nite projective plane does not exist for all N . However, it is known that a nite projective plane of order m exists if the relation m = pk holds for a prime number p and a positive integer k. As an example, some possible quorums for m = 2 (thus, N = 7) are shown in Table 3.1.
Table 3.1 Quorums (Li)
i Li 1 f1; 2; 4g 2 f2; 3; 5g 3 f3; 4; 6g 4 f4; 5; 7g 5 f5; 6; 1g 6 f6; 7; 2g 7 f7; 1; 3g Each quorum contains 3 nodes, and each node appears in 3 quorums, including its own quorum. For other values of m, properties 3 and 4 above can be relaxed to create quorums satisfying properties 1 and 2. Based on the quorums L1; L2; ; LN , Lakshman and Agrawala de ned a new set Ci for each 1 i N , by setting Ci = fj j (j 2 Li or i 2 Lj ); i 6= j g. Then, jCij = 2m, for 1 i N , because jLij = m + 1, i is contained in (m + 1) quorums, and i 2 Li . Table 3.2 shows the sets for m = 2, based on the quorums given above in Table 3.1. Lakshman and Agrawala's protocol is described as follows: 1. In the rst round of the protocol, each node i sends its information to all nodes in Ci. 2. In the second round of the protocol, each node i sends all the information it has received in the rst round to all nodes in Ci. 21
After the second round of the protocol has completed, each node obtains information originally held by every other node in the system. Table 3.2 Quorums (Ci) i Ci 1 f2; 4; 5; 7g 2 f3; 5; 6; 1g 3 f4; 6; 7; 2g 4 f5; 7; 1; 3g 5 f6; 1; 2; 4g 6 f7; 2; 3; 5g 7 f1; 3; 4; 6g p Since (m N ) and each node sends 2m messages in each round, a total of p 4mN ( = O(N N )) messages are required. In the above example, with N = 7 nodes, a total of 56 messages are required. From the de nition of Ci and the properties of nite projective planes, it is clear that the sets satisfy the intersection property. Thus, Lakshman and Agrawala's protocol is a quorum-based protocol. In this view, Gi = Hi = Ci for 1 i N . In the above example, R1 = R2 and the reachability matrix R are as follows: 3 2 1 1 0 1 1 0 1 6 6 1 1 1 0 1 1 0 777 6 6 0 1 1 1 0 1 1 777 6 6 R1 = 66 1 0 1 1 1 0 1 77 6 1 1 0 1 1 1 0 77 6 6 0 1 1 0 1 1 1 7 5 4 1 0 1 1 0 1 1 3
2 6 6 6 6 6 R = 666 6 6 6 4
5 3 3 4 4 3 37 3 5 3 3 4 4 3 77 3 3 5 3 3 4 4 77 4 3 3 5 3 3 4 777 4 4 3 3 5 3 3 77 3 4 4 3 3 5 3 75 3 3 4 4 3 3 5 Note, there is a redundancy in information ow because information passes through at least three di erent paths between any two di erent nodes. 22
Finite projective plane protocol Any grouping based on nite projective planes satis es the intersection property [72]. Direct application of nite projective planes yields the following protocol: Let fGi j 1 i N g be a coterie based on a nite projective plane. We de ne Hi = fj j i 2 Gj g for 1 i N . Since i is in (m +1) of the quorums, jHi j = m +1 for 1 i N . Also, since i 2 Gi , i 2 Hi . For example, Table 3.3 shows Hi for 1 i 7, corresponding to the quorums given in Table 3.1.
Table 3.3 Quorums (Hi) i 1 2 3 4 5 6 7
Hi f1; 5; 7g f2; 6; 1g f3; 7; 2g f4; 1; 3g f5; 2; 4g f6; 3; 5g f7; 4; 6g
Let R1[i; j ] = 1 if j 2 Gi or i = j . In the example, R1 is: 2 6 6 6 6 6 R1 = 666 6 6 6 4
1 0 0 0 1 0 1
1 1 0 0 0 1 0
0 1 1 0 0 0 1
1 0 1 1 0 0 0
0 1 0 1 1 0 0
0 0 1 0 1 1 0
0 0 0 1 0 1 1
3 7 7 7 7 7 7 7 7 7 7 7 5
Let R2[i; j ] = 1 if j 2 Hi or i = j . Note that R2 = RT1 . In the example, R2 is: 2 6 6 6 6 6 R2 = 666 6 6 6 4
1 1 0 1 0 0 0
0 1 1 0 1 0 0
0 0 1 1 0 1 0 23
0 0 0 1 1 0 1
1 0 0 0 1 1 0
0 1 0 0 0 1 1
1 0 1 0 0 0 1
3 7 7 7 7 7 7 7 7 7 7 7 5
The resulting reachability matrix is given by: 2 3 3 1 1 1 1 1 1 6 6 1 3 1 1 1 1 1 777 6 6 1 1 3 1 1 1 1 77 6 6 R = 66 1 1 1 3 1 1 1 777 6 1 1 1 1 3 1 17 6 6 1 1 1 1 1 3 1 7 7 4 5 1 1 1 1 1 1 3 p Since each node sends m messages in each round, a total of 2mN = O(N N ) messages are required. This is approximately the same performance as Yuan and Agrawala's protocol in the case of two rounds of message exchange (i.e., k = 2). In this example, there are a total of 28 messages exchanged for 7 nodes. Unlike Yuan and Agrawala's protocol, this protocol satis es the property that the rst and the second rounds of message exchange can use the same connection.
3.2 Distributed Mutual Exclusion Algorithms A classical problem in distributed operating systems is mutual exclusion. There have been many distributed mutual exclusion algorithms proposed in the last decade [3, 12, 16, 26, 37, 40, 41, 43, 45, 47, 49, 51, 57, 58, 59, 62, 63, 64, 67, 68, 69, 74, 78, 79]. A nice summary can be found in a book by Raynal [61]. These algorithms have been classi ed into two types: token-based and permission-based [60, 68]. In token-based algorithms, a node may enter the critical section only after receiving the token. In permission-based algorithms, a node may enter the critical section only after receiving permission from a quorum of nodes. Token-based algorithms can be further subdivided into those that impose a logical structure on the nodes and those that do not. Of those that impose a logical structure, we have shown that the logical structure can be either static or dynamic [47]. We have presented a token-based algorithm, which uses a static logical structure, and reduces the number of messages required per critical section to three. This is the same number of messages required by a centralized algorithm. 24
In general, token-based algorithms require fewer messages than permission-based algorithms. However, token-based algorithms are not very fault-tolerant. Also, such algorithms must deal with the problem of token loss. In order to circumvent these problems, a class of permission-based algorithms, called quorum-based algorithms, have been proposed [3, 40]. Let U = fa; b; cg denote the set of nodes in a distributed system. Then, Q1 = ffa; bg; fa; cg; fb; cgg is a nondominated coterie under U . Such a coterie can be used in a mutual exclusion algorithm that survives some node and communication line failures [7]. In order to enter the critical section, a node must receive permission from all nodes in a quorum of Q1. Because of the intersection property, mutual exclusion is guaranteed.
a
b
c Figure 3.2 Distributed Mutual Exclusion To see the advantage of using a nondominated coterie, let Q2 = ffa; cg; fb; cgg be another coterie under U . Note that coterie Q2 is dominated by Q1. As shown in Figure 3.2, if a network partition occurs between node c and the other nodes, or if node c fails, then a quorum can still be formed in Q1, but not in Q2. In general, algorithms that use nondominated coteries are more fault tolerant. Recently, Mizuno, Neilsen, and Rao proposed a token-based algorithm which is based on nondominated bicoteries (quorum agreements) [43]. The quorum sets and complementary quorum sets are used to propagate requests to the token holder or to a node which will hold the token. The algorithm generalizes the 25
class of algorithms that do not impose a logical structure on the nodes. However, unlike quorum-based algorithms, the algorithm does not survive communication line failures.
3.3 Replica Control Protocols Consider a distributed system in which N sites (computers) are connected by a communication network. There is no assumption made about the topology, except that every site can communicate with every other site when there are no failures. Communication lines can fail, resulting in partitioning failures. Sites in a partition can only communicate with other sites in the same partition. Message exchange is completely asynchronous. Processes are fail-stop. In a distributed database system, several copies (replicas) of a data object may be maintained at di erent sites to improve fault tolerance. Maintaining several replicas allows the system to gracefully tolerate node and communication line failures. A replica control protocol is used to ensure that di erent copies of a data object appear to the user as a single nonreplicated object; that is, objects are one-copy equivalent [2]. A replica control protocol is used in conjunction with a copy-level concurrency control protocol to ensure correct transaction executions. The correctness criterion is called one-copy serializability [14]. One well known protocol is based on quorum consensus (weighted voting) [25]. Agrawal and El Abbadi generalized quorum consensus in terms of read and write quorum sets [2]. Associated with each data object, read and write quorum sets are formed. A read operation accesses all of the copies in a read quorum, and a copy with the highest version number is returned. A write operation writes to all of the copies in a write quorum and assigns each copy the version number that is one more than the maximum version number encountered in the write quorum. Let Qc and Q be sets of read and write quorum sets, respectively. In order to ensure one-copy equivalence, the read and write quorums must satisfy the following two 26
intersection properties: 1. Read-write : (8G 2 Q); (8H 2 Qc) [ G \ H 6= ; ]. 2. Write-write : (8G; H 2 Q) [ G \ H 6= ; ]. Thus, the pair (Q; Qc) must be a semicoterie in this protocol. However, this protocol is overly restrictive. Bernstein and Goodman have decomposed the concurrency control problem into seperate synchronization problems: read/write and write/write synchronization [13, 14]. These problems can be solved by independent mechanisms and at di erent times. They presented twelve di erent protocols by combining the rst four locking techniques shown below. Barbara and Garcia-Molina have shown how bicoteries can be used to represent these locking techniques [7]. For example, let U = fa; b; c; dg; that is, there are four replicas of some data object in the distributed database. 1. Basic two-phase locking: When a transaction wants to write, it must obtain write locks on all copies of the data object. On the other hand, a read operation only requires a read lock on any copy. In this case, the bicoterie B = (Q; Qc) is given by Q = ffa; b; c; dgg Qc = ffag; fbg; fcg; fdgg 2. Centralized two-phase locking [5, 22]: Before accessing a data object, locks must be obtained at a single site. In this case, the bicoterie B = (Q; Qc) may be given by Q = ffagg Qc = ffagg 3. Primary copy two-phase locking [73]: Before accessing a data object, locks must be obtained on a primary copy. In this case, the bicoterie B = (Q; Qc) may be given by 27
Q = ffagg Qc = ffagg 4. Voting (majority consensus) two-phase locking [76]: Each copy is assigned a number of votes. Transactions request locks from all copies and wait for a majority of votes before accessing the data object. For example, if replicas a, b, and c are assigned a single vote, and replica d is assigned two votes, then the bicoterie B = (Q; Qc) is given by
Q = ffa; b; cg; fa; dg; fb; dg; fc; dgg Qc = Q 5. Quorum consensus (weighted voting) two-phase locking [25]: Each copy is assigned a number of votes. Transactions request locks from all copies and wait for a prede ned number of votes (called a read or write threshold) before accessing the data object. Quorum consensus is described in more detail in Section 5.1. For example, suppose that replicas a, b, and c are assigned a single vote, and replica d is assigned two votes. Further, suppose that the write threshold is four and the read threshold is two. Then, the bicoterie B = (Q; Qc) is given by
Q = ffa; b; dg; fa; c; dg; fb; c; dgg Qc = ffdg; fa; bg; fa; cg; fb; cgg Various combinations of the above locking mechanisms can be used [13]. For example, we could use basic two-phase locking for read/write synchronization and voting two-phase locking for write/write synchronization.
28
Maintaining replicas may a ect not only the reliability, but also the security of the system. Security is concerned with the following two principal issues [19]: 1. secrecy (privacy) - to prevent unauthorized disclosure of data, and 2. integrity (authenticity) - to prevent unauthorized modi cation of data. Maintaining replicas may improve the integrity of the data object. As long as an intruder has not modi ed all of the copies and an authorized user can detect which copies have been modi ed by the intruder, the user can still access a correct copy of the data object. However, maintaining replicas may decrease the secrecy of the data. In order to obtain con dential data, an intruder can access any copy of the data object. Since reliability and security are closely related in a replicated database system, it is natural to integrate one-copy equivalence and security issues in a replica control protocol. However, relatively few such attempts have been made. Three such protocols have been proposed by (1) Herlihy and Tygar [27], (2) Agrawal and El Abbadi [1], and (3) Mizuno and Neilsen [42]. There are many other quorum-based protocols. The interested reader can refer to numerous other papers [2, 7, 21].
29
Chapter 4 Related Research In this chapter, we review related research concepts that are equivalent to quorum structures. Ibaraki and Kameda were the rst researchers to note the equivalence between quorum structures and Boolean functions [28]. It is interesting to note the many equivalent ideas that have developed independently in di erent areas of research.
4.1 Game Theory Simple games are used to model organizational and group decision processes [55, 66]. The basic idea behind a simple game is similar to an election. In order to win the election, a candidate must receive votes from a quorum of players. Let U be a non-empty set of N players. Formally, a simple game on U is a function : P (U ) ! f0; 1g such that 1. (;) = 0. 2. (U ) = 1. 3. G H ) (G) (H ). A set of players G is a winning (losing) coalition if (G) = 1 (0). A coalition G is a blocking coalition if (U ? G) = 0; that is, if all players in G refuse 30
to vote, then no one can win. A winning (blocking) coalition G is a minimal winning (blocking) coalition if H G implies (H ) = 0 ((U ? H ) = 1). Let () and () denote the families of all minimal winning and blocking coalitions, respectively. Then, () is a clutter on U , and () is a blocking clutter on U . It is easy to see that a clutter is just a quorum set. Also, a blocking clutter is a complementary quorum set; in fact, () = Tr( ()). The pair ( (); ()) is called a blocking system under U . Thus, blocking systems are nondominated bicoteries. A player i is a dictator in if fig 2 (). On the other hand, if fig 2 (), then player i is a veto-player in . A player i is a dummy player if (G [fig) = (G ?fig) for every G U . If player i is a dummy player, then i does not appear in any minimal winning coalition. Thus, nondummy players correspond to used nodes. The dual D of a simple game is another simple game on U de ned by
D (G) = 1 ? (U ? G) for each G U . Note that () = (D ) and () = (D ). Simple games can be classi ed as follows [66]: 1. A simple game is proper if every winning coalition is a blocking coalition; that is, only one candidate can win. For example, the simple game 1, with
(1) = ff1; 3g; f2; 3gg and (1) = ff1; 2g; f3gg is proper. Thus, (1) is a coterie. 2. A simple game is strong if every blocking coalition is a winning coalition. For example, the simple game 2, with
(2) = ff1; 2g; f3gg and (2) = ff1; 3g; f2; 3gg is strong. 31
3. A simple game is decisive if it is both proper and strong. For example, the simple game 3, with
(3 ) = (3) = ff1; 2g; f1; 3g; f2; 3gg is decisive. Thus, (3) is a nondominated coterie. 4. A simple game is symmetric if there is a positive integer k such that a coalition G is winning if and only if jGj k. Thus, all of the quorums in the corresponding quorum set have the same size k. 5. A weighted majority game is a simple game on U = f1; 2; ; N g de ned by assigning weights to each player. The weight assigned to player i, denoted wi, is a nonnegative real number. The quota, denoted q, is a nonnegative real number such that:
q
N X i=1
wi
A set of players G U is a winning coalition if and only if: X
i2G
wi q
A weighted majority game is denoted by [ q : w1; w2; ; wN ]. For example, the voting rules of the United Nations Security Council de ne a weighted majority game. The ve permanent members are each assigned a weight of 7. The other ten members are each assigned a weight of 1. Thus, there are a total of 45 votes. It takes q = 39 votes to pass an issue. So, the resulting weighted majority game is denoted by: [ 39 : 7; 7; 7; 7; 7; 1; 1; 1; 1; 1; 1; 1; 1; 1; 1 ] Note that each permanent member is a veto-player. 32
6. A simple game on U is square if the number of nondummy players in U is equal to j () [ () j. Square games are an important subset of decisive games. They correspond to nondominated coteries in which the number of quorums in the coterie is the same as the number of used nodes. 7. Let V1; V2; ; VM be a partition of U = f1; 2; ; N g such that 2 M N , jV1j = 1, and jVij 2 for 2 i M . A partition game is a simple game with:
() = fG1; G2; ; GN g; where [ G1 = Vk where ? = fk j k M mod 2g Gj = Ai =
k2? Ai [ fj g for j 2 Vi ; where 2 i M [ Vk where = fk j k < i and k (i + 1) mod 2g k2
For example, let V1 = f1g; V2 = f2; 3g; and V3 = f4; 5g. Then, the quorums are computed as follows:
G1 G2 G3 G4 G5
= = = = =
f1; 4; 5g f1; 2g f1; 3g f2; 3; 4g f2; 3; 5g
Isbell showed that a square game is either a partition game or the simple game associated with the seven point projective plane shown in Table 3.1 [30]. The results of this section are summarized below in Table 4.1.
33
Table 4.1 Game Theory Structures Quorum Structure Game Theory Structure
node player used node nondummy player quorum minimal winning coalition complementary quorum minimal blocking coalition acceptance set simple game () quorum set clutter ( ()) complementary quorum set blocking clutter ( ()) coterie proper clutter nondominated coterie decisive clutter nondominated bicoterie blocking system
4.2 Reliability Theory Any complex device or structure is comprised of several components. The failure of some of those components may lead to the failure of the entire structure. The probability that a structure is able to perform is called the reliability of the structure. To simplify the analysis, a common assumption is that a structure can either perform or fail; thus, we only consider dichotomic reliability. It is important to understand how individual component failures a ect the reliability of the entire structure. Boolean functions are used to model the structure. In reliability theory, Boolean functions are called structure functions. Recently, Ibaraki and Kameda observed the correspondence between coteries and Boolean functions [28]. Let U = f1; 2; ; N g denote the set of components. A binary indicator variable xi is used to indicate the state of each component; that is, (
i is functioning xi = 10 ifif component component i has failed
34
If we assume that the state of the system is completely determined by the state of its components, then we can de ne a Boolean function f : BN ! B by (
structure is functioning f (x) = 10 ifif the the structure has failed where x = (x1; x2; ; xN ). Since knowledge of f is equivalent to knowledge of the structure having structure function f , we will simply refer to the structure having structure function f as structure f . Let f be a structure on U . The dual structure f D is another structure on U de ned by f D (x) = 1 ? f (1 ? x) for all x 2 BN , where 1 = (1; 1; ; 1). If x y in BN implies f (x) f (y), then f is monotone (or positive). A structure f on U is semi-coherent if 1. f (0) = 0. 2. f (1) = 1. 3. f is monotone. Note that a semi-coherent structure corresponds to a simple game. Let f be a structure on U , G U , and H = U ? G. Then, G is a path (cut) set of f if f (1G ; 0H ) = 1 (f (0G ; 1H ) = 0) where vector (1G ; 0H ) = (x1; x2; ; xN ) such that (
xi = 10 ifif ii 22 G H Let (f ) ( (f )) denote the set of minimal path (cut) sets. If f is a semicoherent structure, then (f ) is a quorum set. 35
For example consider the bridge structure f shown in Figure 4.1 [55].
1 4 3 2 5
A
Z
Z
Z Z Z
Z Z
Z Z Z
Z Z
Z Z
Z Z
B
Figure 4.1 Bridge Structure Path sets are formed by selecting elements on a path from A to B. The resulting minimal path and cut sets are:
(f ) = ff1; 4g; f2; 5g; f1; 3; 5g; f2; 3; 4gg (f ) = ff1; 2g; f4; 5g; f1; 3; 5g; f2; 3; 4gg A k-out-of-N structure f is a structure that functions if and only if at least k components function; that is, (
PN 1 if i=1 xi k f (x) = 0 otherwise
for all x 2 BN . Thus, a k-out-of-N structure corresponds to a symmetric simple game. If k = 1, then f is a parallel structure. On the other hand, if k = N , then f is a series structure.
36
Suppose that f is a non-trivial structure function; that is, f (0) = 0 and f (1) = 1. Ibaraki and Kameda observed the following equivalences between structures and quorum structures: 1. Function f is monotone , (f ) is a quorum set. 2. Function f is dual-minor (f D f ) , (f ) is a coterie. 3. Function f is dual-major (f f D ) , (f ) is a coterie. 4. Function f is self-dual (f = f D ) , (f ) = (f ) is a nondominated coterie. Let UN denote a non-empty set of size N . Using Shannon's decompostion, they showed that there is a one-to-one correspondence between nondominated coteries under UN and coteries under UN ?1. Also, they noted that the number of quorum sets under UN is equal to the number of nondominated bicoteries under UN . The results of this section are summarized below in Table 4.2.
Table 4.2 Reliability Theory Structures Quorum Structure Reliability Theory Structure
node component used node relevant component quorum minimal path set complementary quorum minimal cut set acceptance set semi-coherent structure (f ) quorum set (f ) complementary quorum set (f ) coterie (f ) with dual-minor f (f D f ) nondominated coterie (f ) with self-dual f (f = f D ) nondominated bicoterie ( (f ); (f ))
37
4.3 Graph Theory A quorum set, Q = fG1; G2; ; GM g, can be represented by a simple nite hypergraph, H = (V; E ), where the quorums are the hyperedges of H and the nodes that appear in some quorum are the vertices of H; that is V is the set of all used nodes and E = Q [11]. For example, the hypergraph shown below in Figure 4.2 is the seven point, nite projective plane. Note that this hypergraph corresponding to the coterie given in Table 3.1. 1
2
3 6
7
5
4
Figure 4.2 Finite Projective Plane Let k be a positive integer. Then, H is k-colorable if there exists a mapping : V ! f1; 2; ; kg such that for each G 2 E , there are at least two di erent colors assigned to the nodes in G; that is, j(G)j 2: The mapping is called a k-coloring of H. A hypergraph is singular if it is not k-colorable for any positive integer k. If a hypergraph H is singular, then there must be some edge G in H such that jGj 1. The chromatic number of a hypergraph H, denoted (H), is the smallest positive integer, k, such that H is k-colorable. If H is singular, then (H) = 1: The chromatic number of the empty hypergraph is de ned to be one; 38
that is, ((;; ;)) = 1: Hypergraphs can be partially ordered in several di erent ways. Let H = (V; E ) and H0 = (V 0; E 0) be two hypergraphs. 1. The usual partial ordering is de ned as follows:
H H0 if [V = V 0 and E E 0]: 2. Another partial ordering was de ned by Benzaken [10] as follows:
H H0 if [V = V 0 and (E = E 0 or E 0 dominates E ) ]: Finally, by using these two di erent partial orders, there are two di erent types of critical hypergraphs: 1. A hypergraph H is edge-critical if (8H0) [ H0 H ) (H0) < (H) ]: 2. A hypergraph H is Benzaken-critical if (8H0) [ H0 < H ) (H0) < (H) ]: Benzaken observed that if a hypergraph is Benzaken-critical, then it is also edge-critical. The converse is not true. This is easy to see by considering the graph H = (V; E ) (and hence hypergraph) shown below in Figure 4.3. In this example, the hypergraph H = (V; E ), where:
V = f1; 2; 3; 4; 5g E = ff1; 2g; f2; 3g; f3; 4g; f4; 5g; f5; 1gg
39
2 1 3 5 4 B B
B
B
B B
Z
Z
Z Z Z
Figure 4.3 Hypergraph
So, (H) = 3. Also, it is easy to see that H is edge-critical. However, H is not Benzaken-critical. For example, let H0 = (V 0; E 0), where
V0 =V E 0 = fG V j jGj = 3g Then, H0 < H, but (H0) = (H) = 3.
Lemma 4.1 [7]: A non-empty coterie C under U is dominated if and only if the
corresponding hypergraph is 2-colorable.
Theorem 4.1: Let U be a non-empty set of nodes. Let C be any coterie under
U , such that C is not a singleton coterie. Then, C is nondominated if and only if C corresponds to a 3-chromatic, Benzaken-critical hypergraph.
Proof: Suppose that C is a nondominated coterie under a non-empty set U .
Without loss of generality, we may assume that all nodes in U appear in some quorum in C . Further, suppose that C is not a singleton coterie. Since U is non-empty, C is also non-empty. It is easy to see that the corresponding hypergraph HC = (U; C ) is 3-chromatic. 40
First, to see that HC is 3-colorable, let G 2 C . Since C is not a singleton coterie, jGj 2. Let fG1; G2g be any partitioning of G, and let G3 = U ? G. Then, consider the 3-coloring : U ! f1; 2; 3g de ned by (x) = i if x 2 Gi . This part of the proof is essentially the same as the proof of Lemma 2.1 in [24]. Finally, by Lemma 4.1, HC is not 2-colorable. Hence, HC is 3-chromatic. Let D be any non-empty coterie under U that C dominates. By Lemma 4.1, the corresponding hypergraph HD is such that (HD) = 2. Thus, HC is a 3chromatic, Benzaken-critical hypergraph. On the other hand, let H = (U; C ) be a 3-chromatic, Benzaken-critical hypergraph. By Lemma 4.1, C is a nondominated coterie. Therefore, C is nondominated if and only if C corresponds to a 3-chromatic, Benzaken-critical hypergraph. 2 Garcia-Molina and Barbara have shown that nondominated coteries (except for singleton coteries) are 3-chromatic, edge-critical hypergraphs (see Theorem 2.5 in [24]). Theorem 4.1 strengthens that result. Benzaken-critical hypergraphs are edge-critical, but the converse is not true. Thus, Theorem 4.1 gives a more precise characterization of nondominated coteries in terms of hypergraphs. Since these methods of representing quorum structures are equivalent, properties that hold in one domain, generally work equally well in the other domains.
41
Chapter 5 Simple Quorum Structures In this chapter, we review several protocols which may be used to construct quorum structures, including majority consensus, quorum consensus, and several grid protocols. In addition, we present several new protocols.
5.1 Quorum Consensus Quorum sets may be constructed by using quorum consensus (weighted voting) [14, 25]. Each node is assigned a speci c number of votes. A quorum is formed by obtaining at least a threshold of votes. Note that quorum consensus corresponds to the weighted majority game presented in Section 4.1. Formally, quorum consensus is de ned as follows. Let U denote a non-empty set of nodes. A vote assignment is a function v : U ! f0; 1; 2; g. The total number of votes is TOT(v) = Pa2U v(a). The majority of votes is given by MAJ(v) = d(TOT(v) + 1)=2e. Given thresholds q; qc 1, such that q + qc TOT(v) + 1, the corresponding quorum set Q and complimentary quorum set Qc are given by: X Q = min(fG U j v(a) qg)
Qc = min(fG U; j
42
a2G X
a2G
v(a) qcg)
If q MAJ(v), then Q is a coterie. Note that either q or qc must be greater than MAJ(v), so either Q or Qc must be a coterie. For example, if q = qc = MAJ(v), the resulting quorum sets correspond to majority consensus [76].
5.2 Grid Protocols As an alternative to constructing nite projective planes, Maekawa suggested constructing coteries by using a square grid [40]. First, each node is assigned a location on a square k k grid. A quorum is formed by choosing all elements in any one row and any one column. Grids may also be used to construct bicoteries. Several such methods have been proposed [2, 17, 21]. However, some of these methods result in bicoteries which are dominated. In this section, we present several new methods that result in bicoteries which dominate the bicoteries produced by these methods. Recall that nondominated structures exhibit better performance than the structures which they dominate [7]. 1. Fu's rectangular bicoteries [21]: A quorum is formed by choosing all elements in any one column. A complementary quorum is formed by choosing one element from each column. The resulting bicoteries are nondominated. 2. Cheung, Ammar, and Ahamad's grid protocol [17]: A quorum is formed by choosing all elements in any one column and one element from each of the remaining columns. A complementary quorums is formed by choosing one element from each column. The resulting bicoteries are dominated. 3. Grid protocol A: Quorums are formed by using Cheung, Ammar, and Ahamad's grid protocol. A complementary quorum is formed by choosing one element from each column or by choosing all elements in any one column. 43
The resulting bicoteries are nondominated, and they dominate the bicoteries that result from Cheung, Ammar, and Ahamad's grid protocol. 4. Agrawal and El Abbadi's grid protocol [2]: A quorum is formed by using Maekawa's grid protocol; that is, by choosing all elements in any row, along with all elements in any column. A complementary quorum is formed by choosing all elements is any one row or by choosing all elements in any one column. The resulting bicoteries are dominated. 5. Grid protocol B: Quorums are formed by using Maekawa's grid protocol. A complementary quorum is formed by choosing one element from each row or by choosing one element from each column. The resulting bicoteries are nondominated, and they dominate the bicoteries that result from Agrawal and El Abbadi's grid protocol. For example, consider the following simple grid shown in Figure 5.1. 1 2 3 4 5 6 7 8 9
Figure 5.1 Grid In the rst case, the resulting pair, (Q1; Qc1), is a nondominated bicoterie.
Q1 = ff1; 4; 7g; f2; 5; 8g; f3; 6; 9gg Qc1 = ff1; 2; 3g; f1; 2; 6g; f1; 2; 9g; f1; 3; 5g; f1; 3; 8g; f1; 5; 6g; ; f7; 8; 9gg In the second case, the resulting pair, (Q2; Qc2), is a dominated bicoterie.
Q2 = ff1; 2; 3; 4; 7g; f1; 2; 4; 6; 7g; f1; 2; 4; 7; 9g; f1; 3; 4; 5; 7g; f1; 3; 4; 7; 8g; f1; 4; 5; 6; 7g; f1; 4; 5; 7; 9g; f1; 4; 6; 7; 8g; f1; 4; 7; 8; 9g; ; f3; 6; 7; 8; 9gg 44
Qc2 = Qc1 De ne Q Qc = fG [ Gc j G 2 Q; Gc 2 Qcg and Q Qc = Q [ Qc. In the third case, the quorum set is the same coterie as above, and the resulting pair is a nondominated bicoterie.
Q3 = Q1 Qc1 = Q2 Qc3 = Q1 Qc1 In the fourth case, the resulting pair is a dominated bicoterie.
Q4 = ff1; 2; 3; 4; 7g; f1; 4; 5; 6; 7g; f1; 4; 7; 8; 9g; ; f3; 6; 7; 8; 9gg Qc4 = ff1; 2; 3g; f4; 5; 6g; f7; 8; 9g; f1; 4; 7g; f2; 5; 8g; f3; 6; 9gg Finally, in the fth case, the quorum set is the same as above, and the resulting pair is a nondominated bicoterie.
Q5 = Q4 Qc5 = Qc4 [ ff1; 2; 6g; f1; 2; 9g; f1; 3; 5g; f1; 3; 8g; f1; 4; 8g; f6; 7; 8gg
5.3 Disjoint Set Protocols As a generalization of the above grid protocols, nondominated coteries and bicoteries can be constructed by using any collection of disjoint sets.
5.3.1 Coteries An interesting method to construct k-uniform strange hypergraphs was proposed by Lovasz [39]. This method can be used to construct nondominated coteries. Let U be a non-empty set of nodes. Let V = fV1; V2; ; Vk g be a collection of pairwise disjoint subsets of U such that jVij = i. Construct a quorum, G, 45
by selecting all elements in Vi and one element from each Vj for all j > i. For example, a quorum can be constructed by choosing the one element in V1, and one element from each of the other disjoint sets. At the other extreme, a quorum can be constructed by choosing all elements in Vk ; that is, Vk is a quorum. Note that the resulting quorums all have exactly k elements. For example, let:
V = ff1g; f2; 3g; f4; 5; 6gg Then the resulting nondominated coterie, Q1, is given by:
Q1 = ff1; 2; 4g; f1; 2; 5g; f1; 2; 6g; f1; 3; 4g; f1; 3; 5g; f1; 3; 6g; f2; 3; 4g; f2; 3; 5g; f2; 3; 6g; f4; 5; 6gg The above method can be easily generalized. In particular, we only need to require jV1j = 1 and jVij > 1 for i 6= 1. We will refer the resulting coteries as Lovasz coteries. For example, let V = ff1g; f2; 3; 4gg. Then the resulting nondominated coterie, Q2, is given by Q2 = ff1; 2g; f1; 3g; f1; 4g; f2; 3; 4gg.
Theorem 5.1: Let V = fV1; V2; ; Vk g be a collection of pairwise disjoint subsets of U such that jV1j = 1 and jVi j > 1 for i = 6 1. Let Q be the collection of sets formed by using the above construction. Then, the resulting Lovasz coterie Q is a nondominated coterie under U .
Proof: Let V = fV1 ; V2; ; Vk g be a collection of pairwise disjoint subsets of U such that jV1j = 1 and jVi j > 1 for i = 6 1. Let Q be the collection of sets formed
by using the above construction. It is easy to see that Q is a coterie. We need to show that Q is nondominated. Assume that Q is dominated. By Theorem 2.1, there exists a set H U such that H \ G 6= ; for all G 2 Q, and G 6 H for any G 2 Q. First, we will show that Vi 6 H for 1 i k. Assume that Vj H for some Vj 2 V. Since H \ G 6= ;, there must exist xi 2 Vi such that xi 2 H for each 46
i > j . But, Vj [ fxj+1; ; xk g 2 Q, so this is a contradiction. Thus, Vi 6 H for 1 i k. So, there must exist some element in each Vi that is not in H ; that is, there exist yi 2 Vi such that yi 62 H for 1 i k. However, fy1; y2; ; yk g 2 Q, and this is a contradiction because fy1; y2; ; yk g \ H = ;. Therefore, Q is a nondominated coterie under U . 2
5.3.2 Bicoteries Let U be a non-empty set of nodes. Let Q = fG1; G2; ; Gk g be a collection of non-empty, pairwise disjoint subsets of U . Let Qc = ffx1; x2; ; xk g j xi 2 Gig. Then, Qc = Tr(Q). So, the pair (Q; Qc) is a nondominated bicoterie under U . For example, let:
Q = ff1; 2; 3g; f4; 5gg Then, Qc is given by:
Qc = ff1; 4g; f1; 5g; f2; 4g; f2; 5g; f3; 4g; f3; 5gg In general, the quorum sets, Q and Qc, are not coteries. However, if jQj = 1, then Q is a coterie. Also, if jGj = 1 for some G 2 Q, then Qc is a coterie because all quorums in Qc contain G. Suppose that the quorum sets, Q and Qc, are not coteries. Then, a simple transformation may be used to transform the pair (Q; Qc) into a nondominated semicoterie; that is, one of the quorum sets is a coterie. First, we need the following lemma which follows directly from Proposition 5, on page 423, in Berge's text on graphs and hypergraphs [11].
Lemma 5.1 [11]: Let Q and Qc be quorum sets under U . Then, Tr(Q Qc) = min(Tr(Q) Tr(Qc)): 47
Now, we are ready to describe the transformation.
Theorem 5.2: Let U be a non-empty set of nodes. Let Q = fG1; G2; ; Gk g
be any collection of non-empty, pairwise disjoint subsets of U such that k > 1 and jGij > 1 for 1 i k. Let Qc = ffx1; x2; ; xkg j xi 2 Gi g. Then, (Q Qc; Q Qc) is a nondominated bicoterie under U and Q Qc is a coterie under U .
Proof: First, we will show that Q Qc is a coterie under U . Let G 2 Q Qc. Then, G = H [ H c for some H 2 Q and some H c 2 Qc. Thus, G 6= ; and G U . Since any quorum in Q intersects with all quorums in Qc, the intersection property follows immediately. Finally, we show that minimality holds. Assume that there exist H1; H2 2 Q and H1c; H2c 2 Qc such that H1 [ H1c H2 [ H2c. There are two cases to consider: 1. Suppose that H1 6= H2 . Then, there exists x 2 H1 such that x 62 H2 [ H2c because jH1j > 1, H1 \ H2 = ;, and jH1 \ H2cj = 1. Thus, H1 [ H1c 6 H2 [ H2c. 2. Suppose that H1 = H2. Thus, H1c 6= H2c , and it follows that there is some quorum Gj 2 Q such that Gj 6= H1 and H1c \ Gj 6= H2c \ Gj . Thus, H1 [ H1c 6 H2 [ H2c. Therefore, Q Qc is a coterie under U . It is easy to see that the pair (Q Qc; Q Qc) is a bicoterie under U . We only need to show that the pair is nondominated. Since (Q; Qc) is a nondominated bicoterie, Qc = Tr(Q) and Q = Tr(Qc). So, Tr(Q Qc) = min(Q Qc) by Lemma 5.1. 48
Now, we only need to show that min(Q Qc) = (Q Qc). Assume that there exists a quorum H 2 Q Qc that is not minimal. Then, there exists G 2 Q Qc such that G H . Suppose that G = G1 [ Gc1 and H = H1 [ H1c for some G1; H1 2 Q and some Gc1; H1c 2 Qc. There are two cases to consider. 1. Suppose that G1 = H1. Since jGc1j = jH1c j = k and jG1 \Gc1 j = jH1 \H1c j = 1, it follows that jGj = jH j. Thus, G 6 H . 2. Suppose that G1 6= H1. Then, G1 \ H1 = ; and jG1 \ H1c j = 1. Since jG1j > 1, there is at least one other element in G1 that is not in H1 [ H1c. Thus, G 6 H . Thus, min(Q Qc) = Q Qc. Therefore, (Q Qc; Q Qc) is a nondominated bicoterie under U . 2 For example, if we apply the transformation to the pair (Q; Qc) given above, we obtain:
Q Qc = ff1; 2; 3; 4g; f1; 2; 3; 5g; f1; 4; 5g; f2; 4; 5g; f3; 4; 5gg Q Qc = ff1; 2; 3g; f4; 5g; f1; 4g; f1; 5g; f2; 4g; f2; 5g; f3; 4g; f3; 5gg
49
Chapter 6 Composition Composition provides a simple way of combining non-empty quorum structures to construct new, larger structures. First, we discuss the composition of quorum sets. Then, we disscuss properties satis ed by composition and show that composition can be used to construct any type of quorum structure.
6.1 Composition of Quorum Sets Let U1 be a non-empty set of nodes and x 2 U1. Let U2 be a non-empty set of nodes such that U1 \ U2 = ;. Let U3 = (U1 ? fxg) [ U2. Given a quorum set Q1 under U1 and a quorum set Q2 under U2, a new quorum set Q3 under U3 can be constructed by replacing each occurrence of x in quorums of Q1 by nodes in a quorum of Q2. More formally, let QUi denote the set of all non-empty quorum sets under Ui for i = 1; 2; 3, and de ne a function, Tx : QU QU ! QU , by 1
2
3
Tx(Q1; Q2) = fQTx(G1; G2) j G1 2 Q1; G2 2 Q2g where
(
if x 2 G1 QTx(G1; G2) = (GG1 ? fxg) [ G2 otherwise 1
The function, Tx, is called a composition function. A quorum set constructed by using a composition function is called a composite quorum set; that is, 50
Q3 = Tx(Q1; Q2) is a composite quorum set. All other non-empty quorum sets are called simple quorum sets. For instance, simple quorum sets can be constructed by using quorum consensus [25], a grid protocol [2, 17], a tree protocol [3, 46], or some other method described in Chapter 5 [72, 76]. The input quorum sets, Q1 and Q2, can be either simple or composite. Node x is called a logical node because it is used to logically represent nodes in a quorum of Q2. For example, let U1 = f1; 2; ag, x = a, and U2 = f3; 4; 5g. De ne the input quorum sets, Q1 (under U1) and Q2 (under U2), as follows: Q1 = ff1; 2g; f1; ag; f2; agg Q2 = ff3; 4g; f3; 5g; f4; 5gg Then, Ta(Q1; Q2) = Q3, where Q3 is a quorum set under U3 = f1; 2; 3; 4; 5g, and Q3 is constructed by replacing each occurrence of a in quorums of Q1 by nodes in a quorum of Q2. Q3 = ff1; 2g; f1; 3; 4g; f1; 3; 5g; f1; 4; 5g; f2; 3; 4g; f2; 3; 5g; f2; 4; 5gg
1
3 a 4
2
5
Figure 6.1 Composition Note that the above quorum sets, Q1; Q2, and Q3, are all nondominated coteries. This is no accident. In the following section, we prove some properties that composition satis es and show that composition can be used to construct all other types of quorum structures. 51
6.2 Properties of Composition Let U1 be a non-empty set of nodes and let x 2 U1. Let U2 be a non-empty set of nodes such that U1 \ U2 = ;. Let U3 = (U1 ? fxg) [ U2. Let Qi be a non-empty quorum set under Ui for i = 1; 2. Let Q3 = Tx(Q1; Q2).
Theorem 6.1: Q3 is a quorum set under U3. Proof: First, we show that G3 6= ; and G3 U3 for any quorum G3 2 Q3. Let G3 2 Q3. There are two cases to consider: 1. Suppose G3 = G1 for some quorum G1 2 Q1 where x 62 G1 . Since Q1 is a quorum set, G1 6= ; and G1 U1. Thus, G3 6= ; and G3 U3. 2. Suppose G3 = (G1 ? fxg) [ G2 for some quorum G1 2 Q1 and some quorum G2 2 Q2. Since Q2 is a quorum set, G2 6= ;, and it follows that G3 6= ;. Also, since (G1 ? fxg) (U1 ? fxg) and G2 U2, it follows that G3 U3. Next, we show that minimality is satis ed. Let G3; H3 2 Q3. There are four cases to consider: 1. Suppose G3 = G1 for some quorum G1 2 Q1, and H3 = H1 for some quorum H1 2 Q1. Since Q1 is a quorum set, G1 6 H1 . Thus, G3 6 H3. 2. Suppose G3 = G1 for some quorum G1 2 Q1 where x 62 G1, and H3 = (H1 ?fxg) [ H2 for some quorum H1 2 Q1 where x 2 H1, and some quorum H2 2 Q2. Since Q1 is a quorum set, G1 6 H1. Also, G1 6= H1 because x 2 H1, but x 62 G1. Thus, there exists y 2 G1 such that y 62 H1, and y 6= x. Since G1 U1, H2 U2, and U1 \ U2 = ;, it follows that y 62 H2. Thus, y 62 H3. So, G3 6 H3 because y 2 G3, but y 62 H3. 52
3. Suppose G3 = (G1 ? fxg) [ G2 for some quorum G1 2 Q1 and some quorum G2 2 Q2, and H3 = H1 for some quorum H1 2 Q1. Since Q2 is a quorum set under U2, G2 6= ; and G2 U2. Since Q1 is a quorum set under U1, H1 U1. Thus, H3 U1. Since G2 U2, H3 U1, and U1 \ U2 = ;, it follows that G2 \ H3 = ;. So, there exists y 2 G2 such that y 62 H3. Since G2 G3, y is also in G3. Thus, G3 6 H3 because y 2 G3 , but y 62 H3. 4. Suppose G3 = (G1 ? fxg) [ G2 for some quorum G1 2 Q1 and some quorum G2 2 Q2, and H3 = (H1 ? fxg) [ H2 for some quorum H1 2 Q1 where x 2 H1, and some quorum H2 2 Q2. Since Q1 is a quorum set, G1 6 H1. Thus, G1 = H1 or there exists y 2 G1 such that y 62 H1. Note that y 6= x because x 2 H1. Since Q2 is a quorum set, G2 6 H2. Thus, G2 = H2 or there exists z 2 G2 such that z 62 H2. By comparing G1 with H1 and G2 with H2, there are four cases to consider: (a) Suppose G1 = H1 and G2 = H2. Since G3 = (G1 ? fxg) [ G2 and H3 = (H1 ? fxg) [ H2, it follows that G3 = H3. Thus, G3 6 H3. (b) Suppose G1 = H1 and there exists z 2 G2 such that z 62 H2. Then, z 2 G3 and z 62 H3. Thus, G3 6 H3. (c) Suppose G2 = H2 and there exists y 2 G1 such that y 62 H1. Since y 6= x, y 2 G3. Also, y 62 H2 , because y 2 G1 U1, H2 U2, and U1 \ U2 = ;. Since y 62 H1 and y 62 H2, it follows that y 62 H3. Thus, G3 6 H3. (d) Suppose there exists y 2 G1 such that y 62 H1 and there exists z 2 G2 such that z 62 H2 . Then, z 2 G3 and z 62 H3. Thus, G3 6 H3. Therefore, minimality holds and Q3 is a quorum set under U3. 2 53
Theorem 6.2: If Q1 and Q2 are coteries, then Q3 is a coterie. Proof: By Theorem 6.1, Q3 is a quorum set under U3. So, we only need to show that the intersection property is satis ed. Let G3; H3 2 Q3. There are four cases
to consider:
1. Suppose G3 = G1 for some quorum G1 2 Q1, and H3 = H1 for some quorum H1 2 Q1. Since Q1 is a coterie, G1 \ H1 6= ;. Thus, G3 \ H3 6= ;. 2. Suppose G3 = G1 for some quorum G1 2 Q1 where x 62 G1, and H3 = (H1 ? fxg) [ H2 for some quorum H1 2 Q1 and some quorum H2 2 Q2. Since Q1 is a coterie, G1 \ H1 6= ;. Thus, there exists y 2 G1 \ H1. Since x 62 G1, y 6= x, and it follows that y 2 (H1 ? fxg). Thus, G3 \ H3 6= ;. 3. Suppose G3 = (G1 ? fxg) [ G2 for some quorum G1 2 Q1 and some quorum G2 2 Q2, and H3 = H1 for some quorum H1 2 Q1. This case follows directly from Case 2 (above). 4. Suppose G3 = (G1 ? fxg) [ G2 for some quorum G1 2 Q1 and some quorum G2 2 Q2, and H3 = (H1 ? fxg) [ H2 for some quorum H1 2 Q1 and some quorum H2 2 Q2. Since Q2 is a coterie, G2 \ H2 6= ;. Thus, G3 \ H3 6= ;. Therefore, the intersection property holds and Q3 is a coterie. 2
Theorem 6.3: If Q1 and Q2 are both nondominated coteries, then Q3 is a nondominated coterie.
Proof: Assume that Q3 is a dominated coterie. By Theorem 2.1, there must exist a set H3 U3 such that G3 \ H3 6= ; and G3 6 H3 for all quorums G3 2 Q3. We consider the relation between H3 and the quorums in Q2. There are two cases to consider: either H3 has at least one node in common with each quorum in Q2 54
or there is a quorum G2 in Q2 such that G2 \ H3 = ;. In each case, we nd a quorum G03 2 Q3 such that G03 H3 to obtain a contradiction. 1. Suppose G2 \ H3 6= ; for all quorums G2 2 Q2. Let H2 = H3 \ U2. Since G2 U2, G2 \ H2 6= ; for all quorums G2 2 Q2. So, there must exist a quorum G02 2 Q2 such that G02 H2. Otherwise, H2 would satisfy both properties in Theorem 2.1, and Q2 would be dominated. Let H1 = (H3 [ fxg) \ U1. We start by showing that G1 \ H1 6= ; for all quorums G1 2 Q1. Let G1 2 Q1. Then, either x 2 G1 or x 62 G1. If x 2 G1, then G1 \ H1 6= ; because x 2 H1. On the other hand, if x 62 G1, then G1 = G3 for some quorum G3 2 Q3. Since G3 \ H3 6= ; and G3 = G1 U1, G1 \ H1 6= ;. Thus, G1 \ H1 6= ; for all quorums G1 2 Q1. Since Q1 is nondominated, there must exist a quorum G01 2 Q1 such that G01 H1. Finally, by using quorum G01 2 Q1 and quorum G02 2 Q2, we show that there exists a quorum G03 2 Q3 such that G03 H3 to obtain a contradiction. There are only two possible cases to consider: either x 2 G01 or x 62 G01.
Suppose x 2 G01. Let G03 = (G01 ? fxg) [ G02. Then, G03 2 Q3. Since G01 H1, G02 H2 , and (H1 ?fxg) [ H2 H3, it follows that G03 H3.
This is a contradiction. Suppose x 62 G01. Let G03 = G01. Then, G03 2 Q3. Since G03 (H1 ?fxg) and (H1 ?fxg) H3, it follows that G03 H3. This is a contradiction.
2. Suppose there exists G2 2 Q2 such that G2 \ H3 = ;. Let H1 = H3 \ U1. We start by showing that G1 \ H1 6= ; for all quorums G1 2 Q1. Assume there is a quorum G1 2 Q1 such that G1 \ H1 = ;. There are two cases to consider: either x 2 G1 or x 62 G1. In each case, we nd a quorum G3 2 Q3 such that G3 \ H3 = ; to obtain a contradiction. 55
Suppose x 2 G1. Let G3 = (G1 ? fxg) [ G2. Then, G3 2 Q3. Since G1\H1 = ; and G1 U1, it follows that G1\H3 = ;. Thus, G3 \H3 = ; because G1 \ H3 = ; and G2 \ H3 = ;. This is a contradiction. Suppose x 62 G1. Let G3 = G1. Then, G3 2 Q3. Since G1 \ H1 = ; and G1 U1, it follows that G1 \ H3 = ;. Thus, G3 \ H3 = ;. This is a contradiction.
Thus, G1 \ H1 6= ; for all quorums G1 2 Q1. Since Q1 is nondominated, there exists a quorum G01 2 Q1 such that G01 H1. Otherwise, H1 would satisfy both properties in Theorem 2.1, and Q1 would be dominated. Since x 62 H1, it follows that x 62 G01. Let G03 = G01. Then, G03 2 Q3. Since G03 H1 and H1 H3, it follows that G03 H3. This is a contradiction. Therefore, Q3 is a nondominated coterie. 2
Theorem 6.4: If Q1 and Q2 are coteries and Q1 is a dominated coterie, then Q3 is a dominated coterie.
Proof: To show that Q3 is dominated, we construct a set H3 satisfying the properties in Theorem 2.1, that is G3 \ H3 6= ; and G3 6 H3 for all quorums G3 2 Q3. Since Q1 is a dominated coterie, by Theorem 2.1, there exists a set H1 U1 such that G1 \ H1 6= ; and G1 6 H1 for all quorums G1 2 Q1. There are only two possible cases to consider: either x 2 H1 or x 62 H1. 1. Suppose x 2 H1. Let H3 = (H1 ? fxg) [ H2 for some quorum H2 2 Q2. We show that G3 \ H3 6= ; and G3 6 H3 for all quorums G3 2 Q3. Let G3 2 Q3. There are two cases to consider: 56
Suppose G3 = G1 for some quorum G1 2 Q1 where x 62 G1. First, we show that G3 \ H3 = 6 ;. By the de nition of H1, G1 \ H1 6= ;. Since x 62 G1, it follows that G1 \ (H1 ? fxg) = 6 ;. Thus, G3 \ H3 =6 ;. Next, we show that G3 6 H3. By the de nition of H1, G1 6 H1 . So, there exists y 2 G1 such that y 62 H1, and y = 6 x. Also, y 62 H2 because y 2 G1 U1, H2 U2, and U1 \ U2 = ;. Thus, G3 6 H3 because y 2 G3 , but y 62 H3. Suppose G3 = (G1 ?fxg) [ G2 for some quorum G1 2 Q1 where x 2 G1, and some quorum G2 2 Q2. First, we note that G3 \ H3 = 6 ; because G2 \ H2 =6 ;. Next, we show that G3 6 H3. Since G1 6 H1, there exists y 2 G1 such that y 62 H1, and y = 6 x. Thus, G3 6 H3 because y 2 G3, but y 62 H3. 2. Suppose x 62 H1. Let H3 = H1. We show that G3 \ H3 6= ; and G3 6 H3 for all G3 2 Q3. Let G3 2 Q3. There are two cases to consider:
Suppose G3 = G1 for some quorum G1 2 Q1 where x 62 G1. By the de nition of H1, G1 \ H1 = 6 ; and G1 6 H1. Thus, G3 \ H3 6= ; and G3 6 H3 . Suppose G3 = (G1 ?fxg) [ G2 for some quorum G1 2 Q1 where x 2 G1, and some quorum G2 2 Q2. First, we show that G3 \ H3 = 6 ;. Since x 62 H1 and G1 \ H1 =6 ;, it follows that (G1 ? fxg) \ H1 = 6 ;. Thus, G3 \ H3 =6 ;. Next, we show that G3 6 H3. Since x 62 H1 and G1 6 H1, it follows that (G1 ? fxg) 6 H1. Thus, G3 6 H3. Thus, G3 \ H3 6= ; and G3 6 H3 for all G3 2 Q3. Therefore, by Theorem 2.1, Q3 is a dominated coterie. 2 57
Theorem 6.5: If Q2 is a dominated coterie and x 2 G for some quorum G in coterie Q1, then Q3 is a dominated coterie.
Proof: Since Q2 is a dominated coterie, by Theorem 2.1, there exists a set H2 U2 such that G2 \ H2 = 6 ; and G2 6 H2 for all G2 2 Q2. Let H1 be a quorum in Q1 such that x 2 H1 and let H3 = (H1 ? fxg) [ H2. We show that H3 satis es the properties in Theorem 2.1, that is G3 \ H3 = 6 ; and G3 6 H3 for all G3 2 Q3. Let G3 2 Q3. There are two cases to consider: 1. Suppose G3 = G1 for some quorum G1 2 Q1 where x 62 G1 . First, we show that G3 \ H3 6= ;. Since H1 2 Q1 and Q1 is a coterie, G1 \ H1 6= ;. Furthermore, G1 \ (H1 ? fxg) 6= ; because x 62 G1. Thus, G3 \ H3 6= ;. Next, we show that G3 6 H3. Since x 2 H1 and x 62 G1, H1 6= G1. Since Q1 is a coterie, G1 6 H1. Thus, G1 6 H1. So, there exists a y 2 G1 such that y 62 (H1 ? fxg). Since G3 = G1 and H3 = (H1 ? fxg) [ H2 , it follows that G3 6 H3 because y 2 G3, but y 62 H3. 2. Suppose G3 = (G1 ? fxg) [ G2 for some quorum G1 2 Q1 where x 2 G1, and some quorum G2 2 Q2. First, we note that G3 \ H3 6= ; because G2 \ H2 6= ;. Next, we show that G3 6 H3. From the de nition of H2, we have G2 6 H2. So, there exists y 2 G2 such that y 62 H2. Since G3 = (G1 ? fxg) [ G2 and H3 = (H1 ? fxg) [ H2, it follows that G3 6 H3 . Thus, G3 \ H3 6= ; and G3 6 H3 for all G3 2 Q3. Therefore, by Theorem 2.1, Q3 is dominated. 2 58
Next, we turn our attention to bicoteries. Let B1 = (Q1; Qc1) be a bicoterie under U1 and B2 = (Q2; Qc2) be a bicoterie under U2. Let Q3 = Tx(Q1; Q2) and Qc3 = Tx(Qc1; Qc2).
Theorem 6.6: B3 = (Q3; Qc3) is a bicoterie under U3. Proof: By Theorem 6.1, Q3 and Qc3 are quorum sets under U3. So, we only need to show that the intersection property is satis ed. Let G3 2 Q3 and H3 2 Qc3. There are four cases to consider:
1. Suppose G3 = G1 for some G1 2 Q1, and H3 = H1 for some H1 2 Qc1. Since G1 \ H1 6= ;, it follows that G3 \ H3 6= ;. 2. Suppose G3 = G1 for some G1 2 Q1 where x 62 G1, and H3 = (H1 ?fxg) [ H2 for some H1 2 Qc1 and some H2 2 Qc2. There exists y 2 G1 \ H1 because G1 \ H1 6= ;. Since x 62 G1, y 6= x and y 2 (H1 ? fxg). Thus, G3 \ H3 6= ;. 3. Suppose G3 = (G1 ? fxg) [ G2 for some G1 2 Q1 and some G2 2 Q2, and H3 = H1 for some H1 2 Qc1 where x 62 H1. There exists y 2 G1 \ H1 because G1 \ H1 6= ;. Since x 62 H1, y 6= x and y 2 (G1 ? fxg). Thus, G3 \ H3 6= ;. 4. Suppose G3 = (G1 ? fxg) [ G2 for some G1 2 Q1 and some G2 2 Q2, and H3 = (H1 ? fxg) [ H2 for some H1 2 Qc1 and some H2 2 Qc2. Since G2 \ H2 6= ;, it follows that G3 \ H3 6= ;. Therefore, the intersection property holds and B3 is a bicoterie under U3. 2
59
Theorem 6.7: If B1 and B2 are nondominated bicoteries, then B3 is also a
nondominated bicoterie.
Proof: By Theorem 6.6, B3 is a bicoterie under U3. We only need to show that B3 = (Q3; Qc3) is nondominated. Assume that B3 is dominated. By Theorem 2.3, there is a set H3 U3 such that: G3 2 Q3 ) G3 \ H3 =6 ;, and H30 2 Qc3 ) H30 6 H3: We consider the relation between H3 and quorums in Q2. There are two cases to consider: either H3 has at least one node in common with each quorum in Q2 or there is a quorum G2 in Q2 such that G2 \ H3 = ;. In each case, we nd a quorum H30 2 Qc3 such that H30 H3 to obtain a contradiction. 1. Suppose G2 \ H3 6= ; for all G2 2 Q2. Let H2 = H3 \ U2. Since G2 U2, it follows that G2 \ H2 6= ; for all G2 2 Q2. Since Qc2 = Tr(Q2), there must exist a quorum H20 2 Qc2 such that H20 H2. Let H1 = (H3 [ fxg) \ U1. We start by showing that G1 \ H1 6= ; for all G1 2 Q1. Let G1 2 Q1. Then, either x 2 G1 or x 62 G1. If x 2 G1, then G1 \ H1 6= ; because x 2 H1. On the other hand, if x 62 G1 , then G3 = G1 for some G3 2 Q3. Since G3 \ H3 6= ; and G3 = G1 U1, G1 \ H1 6= ;. Thus, G1 \ H1 6= ; for all G1 2 Q1. Since Qc1 = Tr(Q1), there exists H10 2 Qc1 such that H10 H1. Finally, by using H10 2 Qc1 and H20 2 Qc2, we show that there exists H30 2 Qc3 such that H30 H3 to obtain a contradiction. There are only two possible cases to consider: either x 2 H10 or x 62 H10 . 60
Suppose x 2 H10 . Let H30 = (H10 ? fxg) [ H20 . Then, H30 2 Qc3, and H30 H3 because H10 H1, H20 H2, and (H1 ?fxg) [ H2 H3. This is a contradiction.
Suppose x 62 H10 . Let H30 = H10 . Then, H30 2 Qc3, and H30 H3 because H30 (H1 ? fxg) and (H1 ? fxg) H3. This is a contradiction. 2. Suppose that there exists G2 2 Q2 such that G2 \ H3 = ;. Let H1 = H3 \ U1. We start by showing that G1 \ H1 6= ; for all G1 2 Q1. Assume that there exists G1 2 Q1 such that G1 \ H1 = ;. There are two cases to consider: either x 2 G1 or x 62 G1. In each case, we nd G3 2 Q3 such that G3 \ H3 = ; to obtain a contradiction.
Suppose x 2 G1. Let G3 = (G1 ? fxg) [ G2. Then, G3 2 Q3. Since G1\H1 = ; and G1 U1, it follows that G1\H3 = ;. Thus, G3 \H3 = ; because G1 \ H3 = ; and G2 \ H3 = ;. This is a contradiction. Suppose x 62 G1. Let G3 = G1. Then, G3 2 Q3. Since G1 \ H1 = ; and G1 U1, it follows that G1 \ H3 = ;. Thus, G3 \ H3 = ;. This is a contradiction.
Thus, G1 \ H1 6= ; for all G1 2 Q1. Since Qc1 = Tr(Q1), there exists H10 2 Qc1 such that H10 H1. Since x 62 H1, it follows that x 62 H10 . Let H30 = H10 . Then, H30 2 Qc3. Since H30 H1 and H1 H3, it follows that H30 H3. This is a contradiction. Therefore, Qc3 = Tr(Q3), and B3 is a nondominated bicoterie under U3. 2 In summary, composition can be used to construct any type of quorum structure. Furthermore, if the input quorum structures are nondominated, then the resulting composite structure is also nondominated. 61
Chapter 7 Composite Structures In this chapter, we review several methods which may be used to construct composite structures: the tree protocol, hierarchical quorum consensus, and hybrid replica control protocols. Then, we show that all of these protocols may be obtained by using composition. Finally, we show that composition provides a natural method to construct quorum structures in an arbitrary network.
7.1 Tree Protocol The tree protocol was proposed as a method to construct coteries to be used in a distributed mutual exclusion algorithm [3]. The set of N nodes are logically arranged in a complete binary tree. A path in the tree is a sequence of nodes a1; a2; ; ai; ai+1; ; aj such that ai+1 is a child of ai. A quorum is constructed by grouping all nodes on a path from the root node to a leaf node. If a node on the path is not available, paths that start at both children and terminate at the leaves may be used instead. Agrawal and El Abbadi suggested that any k-ary tree, with k 2, could be used. In fact, we show that the protocol can be applied to any tree in which each nonleaf node has at least two children. Furthermore, we show that coteries constructed by using the protocol are always nondominated [46]. The resulting coteries are called tree coteries. 62
1
3 2 4 5 6 7 8
Z
Z Z
Z
S S
S
S
S S
S
S
Figure 7.1 Tree
Consider the tree shown in Figure 7.1. If all nodes are available, then any of the following sets are quorums: f1,2,4g, f1,2,5g, f1,2,6g, f1,3,7g, and f1,3,8g. If node 1 is unavailable, then paths from both children, nodes 2 and 3, may be used instead. Thus, the sets f2,3,4,7g, f2,3,4,8g, f2,3,5,7g, f2,3,5,8g, f2,3,6,7g, and f2,3,6,8g are quorums. If node 2 is unavailable, the set f1,4,5,6g is a quorum. Likewise if node 3 is unavailable, the set f1,7,8g is a quorum. If both nodes 1 and 2 are unavailable, the sets f3,4,5,6,7g and f3,4,5,6,8g are quorums. Likewise, if both nodes 1 and 3 are unavailable, the sets f2,4,7,8g, f2,5,7,8g, and f2,6,7,8g are quorums. Finally, if nodes 1, 2, and 3 are unavailable, the set f4,5,6,7,8g is a quorum. The collection of all quorums is called a tree coterie. Composition can be used to construct tree coteries. Let U = fa1; a2; ; ang be a set of n 3 nodes. We de ne a tree coterie of depth two over U by
Q = ffa1; aj g j 2 j ng [ ffa2; a3; ; angg Node a1 is viewed as the root node and the remaining nodes are viewed as leaf nodes in the tree. Tree coteries are constructed by repeatedly composing tree coteries of depth two together at one of the leaf nodes. Thus, any tree in which each nonleaf node has at least two children can be constructed.
63
1
b a Figure 7.2 Tree (Q ) 2 6 5 4 Figure 7.3 Tree (Q ) 3 7 8
Z
Z Z
Z
1
S
S
S
S
a
S S
S
S
Figure 7.4 Tree (Qb)
For example, the above tree coterie can be constructed by composing the three tree coteries resulting from the trees shown above:
U1 = f1; a; bg Q1 = ff1; ag; f1; bg; fa; bgg Ua = f2; 4; 5; 6g Qa = ff2; 4g; f2; 5g; f2; 6g; f4; 5; 6gg Ub = f3; 7; 8g Qb = ff3; 7g; f3; 8g; f7; 8gg 64
Let Q2 = Ta(Q1; Qa), and Q = Tb(Q2; Qb). Then, Q under U = f1; 2; ; 8g is the tree coterie corresponding to the tree shown in Figure 7.1.
Theorem 7.1: Tree coteries are nondominated coteries. Proof: It is easy to see that tree coteries of depth two are Lovasz coteries. Hence,
by Theorem 5.1, they are nondominated. Since tree coteries are constructed by composing tree coteries of depth two, it follows from Theorem 6.3 that tree coteries are nondominated. 2
Note that de ning a large composite structure becomes quite cumbersome. Thus, we will simplify our notation. Without loss of generality, we may assume that all of the input quorum sets are de ned under disjoint sets. Let Ux ; Ux ; ; UxN be pairwise disjoint nite sets of nodes, and let Qxi be a quorum set under Uxi for 1 i N . Let Q be a quorum set under 1
2
fx1; x2; ; xN ; y1; y2; ; yM g The additional nodes, y1; ; yM , are not modi ed by the composition. De ne,
Q(Qx ; Qx ; ; QxN ) = TxN (TxN ? ( Tx (Q; Qx ); ); QxN ? ); QxN ) 1
2
1
1
1
1
For example, the above tree coterie Q is given by Q = Q1(Qa; Qb). Note that Q is formed by replacing:
node a in Q1 by nodes in a quorum of Qa; node b in Q1 by nodes in a quorum of Qb; and node 1 is not changed. 65
7.2 Hierarchical Quorum Consensus In order to improve on the performance exhibited by quorum consensus, Kumar proposed hierarchical quorum consensus [32, 33]. A complete tree of depth n is formed with the root at level 0. All non-leaf nodes are logical nodes. A single vote is assigned to each node, except for the root. A pair of thresholds is assigned to each level, except for level 0. Let qi (qic) denote the quorum set (complementary quorum set) threshold assigned to level i. A quorum (complementary quorum) at level i is obtained by collecting at least qi+1 (qic+1) votes from nodes at level (i + 1). By applying this method recursively from level 0 (the root) down to level (n ? 1), a quorum (complementary quorum) of the system is constructed. For example, consider the 9 nodes organized into a tree of depth 2 shown in Figure 7.5.
t
Z Z Z Z
level 0
ta tb t c level 1 level 2 1 2 3 4 5 6 7 8 9
S S
S S
Z Z
Z Z Z
S S S
S S
S
S S
Figure 7.5 Hierarchical Tree
Table 7.1 is a list of possible threshold values and the resulting quorum sizes for a quorum G 2 Q or a complimentary quorum Gc 2 Qc.
66
Table 7.1 Thresholds q1 q1c q2 q2c jGj jGcj 3 1 3 1
9
2 2 3 1 2 2 2 2
6 4
3 1 2 2 6
1
2 2 4
Suppose that q1 = 3, q1c = 1, q2 = 2, and q2c = 2. Then, the corresponding quorum set and complementary quorum set are given by:
Q = ff1; 2; 4; 5; 7; 8g; f1; 2; 4; 5; 7; 9g; f1; 2; 4; 5; 8; 9g; ; f2; 3; 5; 6; 8; 9gg Qc = ff1; 2g; f1; 3g; f2; 3g; f4; 5g; f4; 6g; f5; 6g; f7; 8g; f7; 9g; f8; 9gg For example, f1; 2; 4; 5; 7; 8g 2 Q is formed by collecting fa,b,cg at level 1 (q1 = 3), and then by collecting f1,2g, f4,5g, and f7,8g for a, b, and c, respectively, at level 2 (q2 = 2). Since each node is assigned a single vote, the size of each quorum in the (complementary) quorum set, jGj (jGcj), is equal to the product of the thresholds. Hierarchical quorum consensus can be generalized by using composition; the quorum sets are formed by repeatedly applying composition to quorum sets de ned by quorum consensus. For instance, consider the example given above. Since q1 = 3, q1c = 1, and q2 = q2c = 2, let:
U1 = fa; b; cg Q1 = ffa; b; cgg Qc1 = ffag; fbg; fcgg Ua = f1; 2; 3g Qa = Qca = ff1; 2g; f1; 3g; f2; 3gg Ub = f4; 5; 6g Qb = Qcb = ff4; 5g; f4; 6g; f5; 6gg 67
Uc = f7; 8; 9g Qc = Qcc = ff7; 8g; f7; 9g; f8; 9gg Then, apply composition to construct the quorum sets, Q and Qc, shown above; that is, Q = Q1(Qa; Qb; Qc) and Qc = Qc1(Qca; Qcb; Qcc). Note that this method can be easily generalized. In particular, the tree does not have to be complete, and each node does not have to be assigned a single vote.
7.3 Hybrid Replica Control Protocols Hybrid replica control protocols (or integrated protocols) are essentially methods to construct quorum sets by combining quorum consensus with a structured quorum protocol, such as the grid protocol or the tree protocol [2, 3]. If the grid protocol is used, the resulting protocol is called the grid-set protocol. On the other hand, if the tree protocol is used, the resulting protocol is called the forest protocol. In the grid-set protocol, the nodes are organized as a set of square grids. Suppose there are n grids and m nodes on each grid. Then, there are a total of n m nodes. The quorum sets, in terms of grids, are de ned by using quorums consensus. Suppose, q is the quorum set threshold and qc is the complementary quorum set threshold such that
q + qc (n + 1) and q d((1=2)(n + 1))e: In order to form a quorum in Q (Qc), quorums of nodes must be obtained from at least q (qc) grids. This protocol can also be easily generalized. In particular, it is not necessary to have the same number of nodes on each grid. For instance, consider the example shown in Figure 7.6. Suppose that we use the same quorum sets based on quorum consensus, as in Section 7.2; that is: 68
U1 = fa; b; cg Q1 = ffa; b; cgg Qc1 = ffag; fbg; fcgg
t1 2a
3 4
t
Z Z Z Z
t5 6b 7 8
Z Z
t9 c
Z Z Z
Figure 7.6 Grid-set Protocol Next, the grid protocol, described by Agrawal and El Abbadi, is applied to obtain:
Ua = f1; 2; 3; 4g Qa = ff1; 2; 3g; f1; 2; 4g; f1; 3; 4g; f2; 3; 4gg Qca = ff1; 2g; f3; 4g; f1; 3g; f2; 4gg, Ub = f5; 6; 7; 8g Qb = ff5; 6; 7g; f5; 6; 8g; f5; 7; 8g; f6; 7; 8gg Qcb = ff5; 6g; f7; 8g; f5; 7g; f6; 8gg Uc = f9g, Qc = ff9gg Qcc = ff9gg The resulting quorum set and complementary quorum set are given by:
Q = ff1; 2; 3; 5; 6; 7; 9g; f1; 2; 3; 5; 6; 8; 9g; f1; 2; 3; 5; 7; 8; 9g; ; f2; 3; 4; 6; 7; 8; 9gg Qc = ff1; 2g; f3; 4g; f1; 3g; f2; 4g; f5; 6g; f7; 8g; f5; 7g; f6; 8g; f9gg 69
For example, f1,2,3,5,6,7,9g in Q is formed by collecting fa,b,cg using quorum consensus, and then by collecting f1,2,3g, f5,6,7g, and f9g for a, b, and c, respectively, by using the grid protocol. Using composition, Q and Qc are obtained as follows:
Q = Q1(Qa; Qb; Qc) Qc = Qc1(Qca; Qcb; Qcc) Note that Qc 6= Tr(Q); for instance, f1; 4g 62 Qc, but f1; 4g\ G 6= ; for all G 2 Q. Thus, (Q; Qc) is a dominated bicoterie. However, if Grid Protocol B was used instead of Agrawal and El Abbadi's Grid Protocol, then the resulting bicoterie would be nondominated.
7.4 RST Protocol Recently, Rangarajan proposed the RST Protocol for distributed mutual exclusion [56]. This approach is quite similar to the above method because it also uses a two-level hierarchy. Coteries at the rst level are constructed by using nite projective planes, as used in Maekawa's algorithm[40]. Coteries at the second level are constructed by using quorum consensus.
7.5 Arbitrary Network Protocol Composition provides a natural method for combining structures in an arbitrary network or collection of interconnected networks. For example, consider the graph depicting interconnected networks shown in Figure 7.7.
70
1 2 4 5 t t a b 3 6 7 c t 8 9 10 J
J J J J
J
Figure 7.7 Arbitrary Network
There are three interconnected networks: a, b, and c. Suppose that locally, each network administrator has decided on a coterie to be used in a mutual exclusion algorithm. In order to construct quorums for a mutual exclusion algorithm over the entire collection of networks, composition can be used. For example, suppose that the coteries on each of the networks are de ned as follows:
Qa = ff1; 2g; f1; 3g; f2; 3gg Qb = ff4; 5g; f4; 6g; f4; 7g; f5; 6; 7gg Qc = ff8; 9g; f8; 10g; f9; 10gg Further, suppose that the coterie for the networks is given by:
Qnet = ffa; bg; fa; cg; fb; cgg That is, if a process requires mutually exclusive access to an object over the collection of networks, then permission must be obtained from any two of the three networks. Thus, the coterie for the entire collection of networks (in terms of nodes) is given by Q = Qnet(Qa; Qb; Qc). For example, f1; 2; 4; 5g 2 Q is constructed by selecting f1; 2g 2 Qa, f4; 5g 2 Qb, and fa; bg 2 Qnet. 71
Chapter 8 Composite Structure Evaluation In this chapter, we present several methods to evaluate quorum structures.
8.1 Quorum Containment Test When designing a distributed algorithm, based on quorum structures, it is essential to have a method to determine if a given set contains a quorum. For instance, a simple distributed mutual exclusion algorithm is to broadcast requests to all other nodes and then wait to receive permission from at least a quorum of nodes. In practice, to determine if a given set contains a quorum, it is not necessary to actually compute and store all of the quorums in advance. Instead, we only need to store the input quorum sets used to construct the composite quorum set and information about how the composite quorum set was constructed. This yields an ecient method to determine if a given set of nodes, say S , contains a quorum of the composite quorum set Q.
8.1.1 Method The following function, qct, called the quorum containment test, returns true if there exists a quorum G 2 Q such that G S , and false otherwise.
72
Figure 8.1 Quorum Containment Test function qct(S : set of nodes, Q : quorum set) : Boolean; begin if Q = Q1(Qx ; Qx ; ; Qx ) then /* Q is a composite quorum set */ begin for i := 1 to M do if qct(S; Qx ) then S := (S ? Ux ) [ fxig else S := (S ? Ux ); return qct(S; Q1) end else /* Q is a simple quorum set */ if G S for some G 2 Q then return true else return false end 1
M
2
i
i
i
8.1.2 Example For example, consider the graph depicting interconnected networks shown in Figure 7.7. Recall that there are three interconnected networks: a, b, and c, and the coteries on each of the networks are de ned as follows:
Qa = ff1; 2g; f1; 3g; f2; 3gg Qb = ff4; 5g; f4; 6g; f4; 7g; f5; 6; 7gg Qc = ff8; 9g; f8; 10g; f9; 10gg The coterie for the networks is given by:
Qnet = ffa; bg; fa; cg; fb; cgg Finally, recall that Q = Qnet(Qa; Qb; Qc). Suppose that we want to know if the set S = f1; 3; 5; 8; 9g contains a quorum of Q. 73
qct(S; Q) = if Q = Qnet(Qa; Qb; Qc) then qct(S; Qa) = true ) S := S ? f1; 2; 3g [ fag qct(S; Qb) = false ) S := S ? f4; 5; 6; 7g qct(S; Qc) = true ) S := S ? f8; 9; 10g [ fcg = qct(fa; cg; Qnet) = true, because fa; cg 2 Qnet. Thus, S contains a quorum of Q.
8.1.3 Complexity In the complexity analysis, we will only consider nondominated coteries. By being a little bit clever, it is easy to see that nondominated coteries constitute the worst case in terms of complexity. In particular, for any coterie Q under a non-empty set U of size N , the size of the acceptance set A(Q) is bounded by 2N ?1 . The bounding case only occurs for nondominated coteries. Also, if we consider a quorum set Q under U , then the corresponding acceptance set may be larger than 2N ?1, but then the number of subsets of U not in A(Q) will be less than 2N ?1. Let Q be a nondominated coterie under a non-empty set of nodes U , with N = jU j. Since jA(Q)j = 2N ?1, the number of comparisons required to directly determine if a given set contains a quorum is 2N ?1 = O(2N ). On the other hand, suppose that Q is a composite coterie constructed by composing M coteries Q1; Q2; ; QM under pairwise disjoint sets U1; U2; ; UM , respectively. Each of the input coteries must be evaluated directly. To evaluate coterie Qi requires at most 2jUij?1 comparisons. So, the number of comparisons required to evaluate Q, by using the above method, is at most PMi=1 2jUi j?1. Note that the number of comparisons required does not depend on how the M coteries are composed. 74
Since we are only interested in starting with relatively small coteries, we assume that all coteries are de ned under sets of size at most k for some constant k; that is, 2 jUij k for 1 i M , and k
