A Scalable Architecture for Highly Reliable Certification

A Scalable Architecture for Highly Reliable Certification Xavier Bonnaire1 , Rudyar Cortés1 , Fabrice Kordon2, Olivier Marin2,3 1

Department of Computer Science, Universidad Técnica Federico Santa María, Valparaíso, Chile 2 LIP6, CNRS UMR 7606, Université Pierre & Marie Curie, Paris, France 3 Inria, Domaine de Voluceau, BP 105, 78153 Le Chesnay, France

Abstract—Building a certification authority (CA) that is both decentralized and fully reliable is impossible. However, the limitation thus imposed on scalability is unacceptable for many types of information systems, such as e-government services. This paper proposes a solution to build an highly reliable CA, based on a DHT and a dedicated protocol ensuring a very low probability of arbitrary failure. Thus, in practice, false positives should never occur.

section III. We demonstrate the high reliability of our solution by means of a statistical failure assessment in section IV which shows that its probability of failure can be as low as 1×10−14, as shown in Table IV. We also compute the message complexity of our system to show that it is scalable. Finally, sections V and VI present a discussion around our solution and a comparison to the state of the art.

Keywords—DHT, Certification, peer-to-peer

II. I.

I NTRODUCTION

Building a certification authority (CA) that is both decentralized and fully reliable is impossible [1]. However, the limitation thus imposed on scalability is unacceptable for many types of large scale information systems, such as egovernment [2] services. This type of service requires an infrastructure able to support a heavy load whilst remaining highly reliable. A typical example is income declaration, also known in the US as tax return, where a certification authority must be able to attest that a form was filed in time by the requested actors. In that context, Distributed Hash Tables (DHTs) such as Pastry [3] provide a basis for scalable solutions as opposed to those involving centralized servers. By integrating redundancy, they offer fault tolerance and a high availability. They also allow to locate a service by name in a timely manner. However, a certificate delivered by a single node is only as reliable as the node that delivers it. Therefore a DHT alone cannot constitute a satisfactory solution for problems, such as income declaration, where trust is essential. In this paper, we propose a quasi-certification authority built by coalescing multiple server nodes on top of a DHT. Our solution can deliver and store timestamped certificates ensuring that a service has been executed for a given client around time t. This is the key result of this work since, to the best of our knowledge, there is presently no other decentralized solution that can deliver and store reliable certificates with a timestamp. We call our service “quasi-certification” because, due to the impossibility of distributed consensus presented in [4], we cannot ensure a fully trustable certificate. Instead, we opt for a scalable solution that guarantees a very high degree of trustworthiness. We use the building blocks and notations introduced in section II to detail the principle of our system in

B UILDING B LOCKS

AND

N OTATION

This paper presents a solution built above two existing blocks which provide some necessary services for scalable trust management: a Distributed Hash Table (DHT), and a way to generate a pair of public / private keys for every node to use an asymmetrical encryption mechanism [5]. For simplicity, we assume in the following that the underlying DHT is Pastry[3]. Every node in Pastry is assigned a unique nodeID in a space of identifiers of 160-bits, generated using a cryptographic hash. The nodeID determines the position of the node in the circular namespace, that is the Pastry ring. In Pastry, a key k is a value in the same namespace as the nodeIDs. Every node in Pastry maintains a set of its numerically closest neighbors in the ring, called the leaf set. A typical value for the leafset size is 16 nodes. Please see section V for details about the portability of our approach above other overlays. We suppose that any node X of the DHT possesses a pair of public and privates keys (X pub , X priv ) to be used with an asymmetrical encryption algorithm[5] known by all the nodes. For scalability considerations, we suppose that every node of the DHT self-generates its own pair of keys, and that the size of the keys used in the whole system is both fixed and large enough to make it computationally infeasible to decrypt an information without knowing the corresponding private keys. Throughout this paper, SHA stands for the Secure Hash Algorithm One (SHA-1 – 160 bits hash). We also assume that all communications among nodes are made using the TCP protocol, and that all transmissions errors are handle at the transport level. Our final assumption is that the underlying operating system of every node which participates to the DHT is running the Network Time Protocol. Thus, we consider that every node has a good global time, with a small deviation from the Coordinated Universal Time (UTC).

A 3 - transaction ack 1 - request init

A

answers 2 - transaction

S

C

}

A requests leafset

A & S secure exchanges

receive leafset

C

A requests cert. service

S

4 - certificate generation

ack cert. service

Certificate

End ack

nodes request leafset

Log Entry

nodes receive leafset nodes ack transaction

Fig. 1.

Certification Scheme

Legend

III.

} }

S performs the service for A

the service

S get leafset from C C elaborates the certificate

}

Single message request to the leafset messages from leafset to leafset request L/2+1 answers of the same value

A Q UASI -C ERTIFICATION AUTHORITY Fig. 2.

This section details the protocol we propose for building a quasi certification authority (CA) over DHTs. [1] shows that it is impossible to get a 100% reliable system over a DHT where malicious nodes are present. Hence the goal of our approach is to drastically reduce the probability of establishing a corrupt certificate for a transaction between a node A and a service S.

Structure of the protocol

KROOT will not suffice to acquire a reliable leafset associated with K. Our solution for mitigating this issue is to obtain and compare the leafsets of at least 3 nodes within the leafset of node KROOT . For this purpose we use the Diversity Routing proposed in [6]. This technique takes different nodes as starting points in order to reach the same root node KROOT . The diversity of the routing tables on each node guarantees a very high probability of reaching root node KROOT using different paths, and therefore of entering the leafset of KROOT via different nodes. This makes for a more secure way of acquiring the leafset associated with a key K.

Let A be a node of the DHT, and S = {S1 , S2 , · · · , Sn } a set of n nodes that cooperate to provide a service. Let NameofS be the name of service S; every node that needs to use the service knows NameofS. Ks = SHA(N ameof S) is the key that identifies S in DHT, and Sroot is the node closest to Ks in the DHT. It follows that S = {S1 , S2 , · · · , Sn } is composed of node Sroot and of all the nodes in its leafset. Hence every existing service in the DHT is associated with a different set of nodes which handle the service. This induces natural load balancing among services. Let C be a certification authority such that C = {C1 , C2 , · · · , Cn } is a group of n nodes belonging to the DHT. Figure 1 shows the general interaction scheme between A, S, and C.

B. Service Request To find the nodes associated with a given service, node A computes the hash of the service name: Ks = SHA(N ameof S). Let Sroot be the node that is closest to Ks in the DHT. A uses diversity routing to acquire the leafset of Sroot . A thus obtains access to the set S of nodes that process requests for service ServiceN ame. Let L be the number of nodes in a leafset: card(S) = L + 1.

Finally, we assume that two identical requests (same version number, same client identifier, same content) from a client A to a server S are in fact duplicates of the same request. For instance if a student repeatedly submits electronic versions of her essay, two exact same submissions of the essay will be considered by the server as duplicates of the same request. If the essay has been modified in-between two submissions then these two messages will be considered as different requests.

Once A has acquired S, it proceeds to the inception of its service request by sending a RequestInit message to every node in S. A RequestInit message contains the public key Apub of client A and a sequence number SEQ used as a transaction identifier (see section III-D1). In reply to such a message, every node Si sends its own public key Sipub . In order to avoid a manin-the-middle attack, all the ensuing communications between A and nodes that belong to S will use a public/private key encryption scheme to ensure that no other node can read the content of the messages.

Figure 2 presents the overall structure of the protocol. A is the client which initiates the service with S, and C is the quasicertification authority. S gets executed on a set of servers, and so does C on a different set. As the figure shows, the protocol can be decomposed into four main steps that are detailed in the next sections: securing exchanges via the acquisition of the involved leafset, the service itself (request and progression), and the certification itself.

If A receives at least L2 + 1 answers from the nodes in S, then A considers that the transaction will proceed. Since S is composed of common nodes from the DHT, it is possible for some of these nodes to be malicious. However, as explained in our evaluation of the quasi certification protocol (Section IV), a situation where S contains more than L2 malicious nodes is highly improbable.

A. Securing the nodes of a leafset The first step to requesting a certification/service is to acquire the set of nodes that compose the leafset for a given root key K. Supposing that K is associated with node KROOT , there remains a probability for KROOT to be malicious. KROOT might lie and provide references to other malicious nodes in order to collude against the requester, or it might deny any service by remaining silent. Therefore, a simple request to

A combination of network failures and/or malicious nodes that choose to remain silent may prevent A from receiving at least L2 + 1 answers. Since S retains the properties of Pastry’s leafset, the numerical closeness of its nodes in the DHT implies 2

1) Certificate Generation: Upon reception of a RequestInit message from a client A, every node in S starts looking for the set C of nodes that will constitute the quasi certification authority for the transaction. To determine which nodes belong to C, S computes its key K such that K = SHA(N odeIDA ⊕ ServiceN ame). Croot is the node closest to K in the DHT, and the quasi certification authority C = {C1 , C2 , · · · , CL+1 } for the transaction between A and S is composed of Croot and of its leafset secured through diversity routing.

that they will very likely be geographically far on the Internet. Therefore, the probability of L2 + 1 simultaneous IP routing failures or node failures, that is the probability for more than L 2 answers to not reach A, is very close to zero. To account for this type of situation, A aborts the transaction if it has not received a sufficient amount of identical answers after a timeout △t. This is a very rare and highly improbable case, as shown by our evaluation in Section IV-B. C. Service Progress

To prevent man-in-the-middle attacks, we use an asymmetric key encryption scheme for the communications between S and C. This induces that every node in S must exchange public keys with every node in C. The initial cost expressed in number of messages is 2L2 + 4L + 2, but this exchange only occurs the first time a client A requests a service from S. Every node in S maintains a local cache with the public keys of C, and key exchanges only occur again if a new node replaces one of the original nodes from C.

After the initialization phase, A and S proceed to a service progress phase. A sends all its messages to every node of S and expects to receive at least L2 + 1 identical answers for each message. A signs every message to the nodes of S with its private key Apriv , and then encrypts every message using the public key of the recipient. All messages sent to A by a node of S are signed using its private key Sipriv ) and encrypted using the public key Apub of A.

Upon reception of the ACK that denotes the end of a transaction, a node of S sends a notification to every node Ci in C, encrypted with the public key of the recipient. The notification message contains the hash signature hashSIGN of the transaction, computed by and signed with the private key of the sender in order to prevent a man-in-the-middle attack.

Upon receiving the last transactional message from A, every node of S waits for an ACK message from A and responds with an ACK to A. If A does not receive at least L 2 + 1 ACKs, it considers that the transaction has failed. Similarly to the service request, A may fail to receive at least L2 + 1 answers because of malicious behaviours or because of network partitions. In such cases, A considers that the transaction has failed and aborts after a time △t.

When a node Ci of C receives at least L2 + 1 identical notifications from S reagrding a transaction with A (same hash signature), then Ci assumes that a transaction between A and S has occurred indeed, emits a certificate, and logs the latter locally in an append-only file. Ci uses the key K = SHA(N odeIDA ⊕ServiceN ame) to identify the log. It follows that every log file is associated with a specific service and a specific client.

Figure 3 sketches the exchange of messages between A and S during the service progress phase.

Service S

A

S1

S2

Progress Ack

Fig. 3.

Service progress (A_Signature, SEQ,Data)

Since a client can submit different requests to the same service, we define an order relation to sort transactions in the same log.

Sn Ansvers ([S1_Sign, SEQ, Ans_S1]...[Sn_Sign, SEQ, Ans_Sn])

Let Tx = [SEQx , hashxSIGN ] identify a transaction. For two transaction identifiers Ta and Tb :     T < T a b     Ta = Tb

Transaction progress between a client A and a service S

In order to identify request duplicates, every node Si of S computes a recursive hash signature hashSIGN for the transaction with A. Equation 1 shows the computation of the hash signature for the transaction with A, where ⊕ denotes the concatenation operation. (

hash0SIGN = SHA(SEQ ⊕ F irstData) hashiSIGN = SHA(hashi−1 SIGN ⊕ SEQ ⊕ CurrentData)

   SEQa < SEQb if OR   SEQ = SEQ a b otherwise

b ∧ hasha SIGN < hashSIGN

If two transactions are identical, C considers them as duplicates and stores only one certificate into the corresponding log. Every node in C maintains the same log concurrently, which means that at any given time several versions of the same log may coexist. However, although some certification nodes may lie when exchanging log updates, it is not necessary to impose total ordering among nodes when handling the certification process. In order to avoid such a strong constraint, we designed a lazy consistency protocol that we describe in Subsection III-D2. The protocol relies on the concept of log entries. Every node in C arranges log entries in the log file so that, if Ey (Tx ) is the log entry of index y associated with transaction Tx :

(1)

D. Certification Concurrently to carrying out a transaction for a client A, S proceeds to obtain its certification from another set C of nodes. At the end of the transaction, the objective is to return a certificate to A and to store the certificate on the nodes that belong to C.

∀a, b

3

∀i, j

Ei (Ta ) appears bef ore Ej (Tb )

if Ta < Tb

log(C)

Every log entry follows the format: K Ei (T ) = {SEQT , hashT SIGN , hashi , t}

where SEQT is the sequence number of the transaction provided by the client, hashTSIGN is the transaction signature, hashK i is the recursive hash of the log entry as defined below, and t is a timestamp generated with the local UTC time of the node. In order to distinguish successive versions of a log, we use a recursive hashing method computed as follows: (

hashK 0 = 0 hashK i+1 = SHA(SEQi+1

log(A)

log(B)

log(B)

log(B)

log(B)

X-4

X-3

X-2

X-1

[

interval I

X

JOIN

X+1

X+2

log(A)

log(A)

log(B)

log(C)

]

X+3

X+4

log(C)

log(C)

log(C)

Fig. 4. Certificate logs maintenance (brackets delimit interval I, grey nodes are those in the leafset and white ones those outside the leafset)

⊕ hashiSIGN ⊕ hashK i )

this log, then X must ask Z for the entire log. Otherwise, X sends the last q recursive hashes it has for the log. The recursive hashes act as checkpoints for the log. The purpose of the checkpoints is to prevent nodes from downloading entire logs when they already store parts of logs. This may happen when a node re-enters a leafset it had previously left. Our checkpointing scheme allows reentrant nodes to identify the missing parts of the log. Upon reception of the last q recursive hashes from X, Z looks for matches in its log entries. If the search is successfull, Z sends back all its entries that are subsequent to the most recent match. If Z cannot find any match, then it requests the next q hashes from X. If Z states that it cannot find any match, then X can consider that Z is lying, and X must contact another node in N .

2) Lazy Log Consistency Maintenance: Insertions and departures of nodes in the DHT cause modifications of the leafsets, and it is therefore crucial to maintain the consistency of the certificates logs on every node. Upon entering a leafset, a node X must update its own view of the certificates logs. It starts by following the steps below to find which logs must be updated.

Upon reception of the log entries from Z, X adds the entries to its log and computes the recursive hash. If the recursive hash differs from those provided by at least L2 + 1 nodes in set Y then X must consider that Z is lying, and can proceed to retrieve the log entries from another node in N .

Step 1: X constructs the set of nodes N = {X−L/2 , X−L/2+1 , . . . , X, . . . , XL/2−1 , XL/2 } composed of its leafset, and of the L2 + 1 nodes both on the right and on the left of its leafset. Figure 4 gives an example for a leafset of size L = 2. In order to build this set of nodes, X can ask the farthest nodes X−L/2 and XL/2 of its leafset for their own respective leafsets. If one of these nodes fails to answer in a timely manner, X can ask the second to farthest node, and so on until its immediate neighbors in the leafset. Having L2 + 1 malicious neighbors on both sides is highly improbable, and practically infeasible: for more details about this statement, please refer to our probabilistic evaluation in section (IV)). Step 2: X computes interval I such that X−L/2 − X−L/2+1 2

log(A)

interval I

This recursive hashing method provides a secure identification for the state of the log on each node of C. It allows to determine with absolute certainty whether a node is indeed in possession of a given version, and induces neither false positives nor false negatives. A single missing entry prevents a node from pretending it stores a valid log. It also guarantees that the recursive hash of the last entry of the log corresponds to the recursive hash of the whole log, and allows to compare two versions efficiently.

I = [X−L/2 −

log(A)

, XL/2 +

XL/2 − XL/2−1 2

It is essential for X to handle log synchronization as an atomic operation, as delivering a transaction notification may modify a log entry and invalidate log consistency. Therefore X postpones all transaction notification deliveries until log synchronization is over. The last recursive hash will suffice for almost every log entry retrieval. In some rare cases, X may receive a transaction notification from service S immediately after entering the leafset and before the beginning of the log synchronization process. This is possible if A makes a very short transaction with S when X is part of the leafset of the corresponding quasi certification authority, but just before the synchronization of the logs on X. In this case, this will produce a shift in the log of X, and its last recursive hash will not correspond to the ones of the other nodes that manage the same certification log. Since this recursive hash is useless, X discards it and proceeds to a new log synchronization.

]

and sends it to every node in set N . Upon reception of this message, every node replies with a set of all the log identifiers with key K ∈ I it stores, along with the last recursive hash corresponding to each log. Thus the format of a response is a variable-size set of pairs {K, hashK }.

3) Node Failures and Node Departures: Two situations call for the removal of a node. A normal departure corresponds to an honest node that notifies its departure to the nodes of its leafset. In a crash departure situation, the node fails silently. Pastry exchanges periodical heartbeats among nodes in the same leafset in order to detect such failures.

Step 3: X builds set Y = {{Ki , hashKi }, . . . , {Kj , hashKj }} the union of all received log identifiers and recursive hashes. X then discards all pairs {Ki , hashKi } that appear less than L2 + 1 times in Y , since these may have been generated by colluding nodes. Once this is done, X synchronizes all the logs whose identifier matches one of the remaining identifiers in set Y .

Upon removal of a peer in its leafset, a node X will look for a replacement. X starts by identifying which side of its leafset it must fix: the left side or the right side, depending on the nodeId of the node that was removed.

For every log that X must synchronize, X selects a node Z in N that holds this log. If X does not have any entry for 4

Let N = {Nα , Nβ , . . . , Nω } be the set of L2 − 1 nodes corresponding to the side of the leafset in need of fixing. In order to find the next node to add to its leafset, X requests the leafset of the farthest node Nω in set N . Upon reception of the answer, X then inserts in its own leafset the reference to the node closest to Nω that does not yet belong to N . If Nω remains silent, X must repeat the operation with the second to farthest node in N , and so on until reaching node Nα if no timely answer ever comes back If X fails to receive any answer from the nodes in its leafset, then it cannot repair its leafset.

Size of Trusted Set (L) 8 16 32

TABLE I.

0.0081 6.56 × 10−5 4.3 × 10−9

6.25 × 10−6 3.9 × 10−11 1.52 × 10−21

P ROBABILITY OF FAILURE WHEN SECURING A LEAFSET

DHT. We conduct every assessment with two different values for p, namely p = 0, 3 and p = 0, 05. The former corresponds to the limit above which our relaxed version of the byzantine agreement collapses. The latter corresponds to the value most commonly found in the literature about malicious attacks in P2P systems.

Once it has successfully fixed its leafset, X must synchronize its logs. X uses diversity routing to acquire the leafset LS of the node it has just added. X then synchronizes its logs with the nodes in LS by means of the scheme described in section III-D2.

A. Securing a Leafset 1) Probability of failure: Table (I) shows the probability of failure when securing a leafset. The algorithm fails to get the leafset of a given node KROOT if L2 consecutive nodes are malicious in a leafset. This probability is given by

E. Verifying a certificate In order to verify whether a transaction has occurred between client A and service S, a node Z must first acquire the leafset associated with key K = SHA(N odeIDA ⊕ N ameof S) through diversity routing. Depending on the application, Z may be interested in the last transaction that occurred between A and S, or the last nth transaction, or even in checking whether a transaction occured between two dates. In order to simplify the explanation of the protocol, let us assume that Z wants to check if the last transaction between A and S has occurred before time t0 .

L

P=L/2 = p 2

(2)

2) Message complexity: According to equation (2), the probability of having more than 4 consecutive malicious nodes when using the Diversity Trusted Routing is around 6.25 × 10−6 . We can thus consider it highly improbable that more than 4 routing attempts will be necessary to secure a leafset of a node near KROOT . An upper bound of the cost to get the leafset of KROOT is given by

Z sends a request to every node in C for the last certificate delivered to A. The last certificate is the last entry in the log file associated with A and stored on C. When Z receives at least N 2 + 1 answers from the nodes in C with the same recursive hash value, then Z considers this answer as correct. After a time △t, if there is no answer from C, or if there are less than N 2 + 1 answers from C, then Z considers that there is no such certificate.

n=

O(log2b (N )) + 4 × O(log2b (N )) {z } | F irst and Diversity Routing

+

Q+4 | {z } Direct IP

n = 5 × O(log2b (N )) + Q + 4 = O(log2b (N ))

(3)

where Q is the maximum number of tries to get a leafset from a node that belongs to the leafset of KROOT . The probability of not being able to retrieve a leafset from a node that belongs to the leafset of KROOT after Q = 16 tries for a leafset size L = 16 is 1.52 × 10−21. This is highly improbable, so we can consider that L/2 = 8 is a reasonable upper bound for Q.

Let W = {Cert1 , Cert2 , . . . , CertN } be the set of certificates received by Z. Z builds the time interval T = [tmin , tmax ] with tmin the oldest certificate timestamp in W and tmax the most recent. Since we assume that every node of the DHT uses an NTP server to synchronize its clock, timestamping certificates with the UTC time ensures that T will be relatively small. Z then computes the average tavg between tmax and tmin . Z considers that the transaction between A and S occurred before the deadline t0 if tavg < t0 or if |t0 − tavg | ≤ α, α being a tolerance threshold which depends on the application. Otherwise, Z can still attest that a transaction did occur between A and S, but that it was not carried out within the required time. IV.

Probability to fail p = 0.3 p = 0.05

In the best case, the cost is reduced to n = O(log(N )) when the root node KROOT is honest. Thus, the message complexity of our leafset securing algorithm easily scales when the size of the DHT increases. B. Transactions between A and S 1) Probability of failure: A transaction between A and S can fail (a) if S does not respond to A during the service request or during the transaction progress, or (b) if S does not send the ACKs to A to terminate the transaction. In both cases, this corresponds to the probability that more than L/2 nodes of S are malicious. The probability of encountering exactly k malicious node among L+1 is given by the binomial distribution

E VALUATION

In this section, we present a probability assessment of our algorithms, and show that it is highly improbable for our quasi certification authority to fail. We also evaluate the message complexity of our solution, and deduce that it is both scalable and efficient.

PKmalicious =

L + 1 pk (1 − p)L+1−k k

(4)

where p is the probability that a single node is malicious. Hence the probability of facing at most k malicious nodes is

In this section, p represents the probability that a single node is malicious, and N the total number of nodes in the

P≤k =

5

k  X L + 1 i p (1 − p)L+1−i k i=1

(5)

Case 2 Case 1

0.007 1e-06

Probability

4.1e-14

TABLE III.

p = 0.3 0.1960 0.07437 0.0138 0.216 0.0745 0.0138

Ptotal p = 0.05 3.72 × 10−4 3.5 × 10−7 4.24 × 10−13 4.97 × 10−4 3.5 × 10−7 4.24 × 10−13

P ROBABILITY OF FAILURE FOR THE CERTIFICATE LOGS MAINTENANCE

30% of faulty nodes 15% of faulty nodes 5% of faulty nodes

1.16e-43 1

4

8

12

n = (r + 3)(L + 1) 16

20

24

28

where r corresponds to the number of data messages sent by A to S, and fully depends on the transaction. The total cost is then

32

Amount of faulty nodes [k]

Fig. 5.

L 8 16 32 8 16 32

ntotal = 5 × O(log2b (N )) + Q + 4 + (r + 3)(L + 1)

Probability of facing more than k malicious nodes

The total cost only depends on the size L of the leafset, which is a constant, and O(log(N )). In the best case,

Therefore, the probability of facing more than k malicious nodes among n is given by P>k = P≤n − P≤k

ntotal = O(log(N )) + (r + 3)(L + 1)

(6)

Therefore, the cost of a transaction between A and service S remains scalable when the size N of the DHT increases.

and the probability that S will not respond is P>L/2 = P≤L+1 − P≤L/2 =

−

L+1 X

i=1 L 2  X i=1

L + 1 i p (1 − p)L+1−i i

C. Lazy Log Consistency Maintenance We must consider two cases for certificate log maintenance: when a new nodes enters the leafset (case 1) and when a node leaves the leafset (case 2).

(7)

L + 1 i p (1 − p)L+1−i i

Case 1 - A new node enters the leafset. The maintenance of the certificates logs fails if it encounters L2 consecutive malicious nodes when building the node interval to retrieve the logs, or if more than L2 nodes are malicious (impossibility to have more than L2 + 1 identical answers).

Figure 5 represents the likeliness of dealing with an increasing number k of malicious nodes among a leafset comprising 32 nodes. Then the probability that S will not respond to the RequestInit or to not send the final ACKs is PAS = (1 − P>L/2 )P>L/2 + P>L/2 PAS = 2P>L/2 − P>2 L/2

The probability P1 of having L2 consecutive malicious L nodes is given by equation (2): P1 = p /2 , and the probability P2 of not being able to retrieve at least L2 + 1 identical answers is given by equation (7): P2 = P>L/2 . Hence the total probability that this maintenance operation will fail is

(8)

Ptotal = P1 + (1 − P1 )P2

Table II shows the probability of failure between A and S, for a leafset size of L = {8, 16, 32}. 2) Message complexity: First, A must get the leafset of S. The associated cost is n = 5 × O(log2b (N )) + Q + 4 (see section IV-A2). The number n of message inherent to the transaction itself is given by n=

TABLE II.

2(L + 1) | {z } Init

+

r(L + 1) | {z } Data

+

L

p = 0.3

p = 0.05

8 16 32

0.188 0.079 0.016

6.64 × 10 6.57 × 10−8 8.24 × 10−14

Case 2 - A node is leaving the leafset. The certificate logs cannot be repaired if the node cannot repair its leafset (adding a node on the left or right side), or if it is impossible to find at least L2 + 1 identical answers to download the logs. A node cannot repair its leafset if L2 − 1 consecutive nodes are malicious with a probability P1 = pL/2−1 and the download of the logs fails with a probability P2 = P>L/2 .

L+1 | {z } ACKs

Hence the total probability of failure is Ptotal = P1 + (1 − P1 )P2

−5

P ROBABILITY OF FAILURE OF THE TRANSACTION BETWEEN A AND S

Table III computes the probability that a maintenance operation will fail, given increasing leafset sizes L = {8, 16, 32}. 6

TABLE IV.

L

p = 0.3

p = 0.05

8 16 32

0.188 0.079 0.016

6.64 × 10−5 6.57 × 10−8 8.24 × 10−14

P ROBABILITY OF

the (group of) server(s) it runs on. Well-established centralized CAs have been corrupted before [7], [8]. In a decentralized approach, the main issue is that the supporting hosts are not accountable for their decisions and therefore cannot be trusted. That is precisely the point of our solution: no CA node is accountable on its own, since we build sets of nodes to cooperate on certificate validation. Overall in a P2P system with a 5% proportion of malicious nodes distributed uniformly, the resulting probability of a false positive, where a decision made by malicious nodes is adopted, is of the order of 10−13 . This value is far lower than any we have encountered in the literature about decentralized CAs. It is obtained with a leafset containing 32 nodes which, as shown in Section IV, remains very affordable in terms of communications overhead.

FAILURE OF THE CERTIFICATION

D. Certificate Generation 1) Probability of failure: The generation of a certificate fails if S fails to send a sufficient amount of ACKs to C, or if an insufficient number of nodes from C confirms that the certificate exists. Either S or C fails if at least more than L/2 of its nodes are malicious. We have P>L/2 = P≤L+1 − P≤L/2

Scope of Applicability. We propose two examples of application for our approach, explain why a centralized solution is not fully satisfactory, and give an overview of how they might be implemented over our solution. (i) Digital filing for tax purposes. Many governments expect their citizens to file a revenue declaration form every year before a fixed date, and accept digital filing. However the servers that process filings must handle huge peak workloads as the deadline approaches, and full server crashes are common. Our approach offers a simple load balancing solution for this application. Since the data size of a declaration is very small, it is perfectly affordable to store the encrypted declaration directly with the timestamped certificate. The scalable service will remain available despite intense activity from last minute filers, and the taxation authority may then process declarations in its own time well after the deadline. (ii) Online game refereeing. Online games regroup large numbers of players that interact in the same virtual universe. Besides their huge maintenance cost, centralized gaming servers do not scale so well and are not failproof. [9] uses a reputation system to decentralize the refereeing process, thus cutting the costs and removing the limit on the size of the virtual universe. Yet it cannot detect complex cheating behaviours such as goldf arming, the gathering of virtual resources in order to sell them for profit in the real world. Every game action could lead to a certificate in our approach, and the logs would allow to analyze player behaviors over long periods of time.

The probability PCF of failure of the certification is then given by PCF = 1 − (1 − P>L/2 )²

=

PCF

1−

1−

L+1 X i=1

L + 1 i p (1 − p)L+1−i i 2

L/2 L + 1 X pi (1 − p)L+1−i  + i i=1

(9)

Table IV shows the evaluation of the probability of failure of our certification protocol with an increasing leafset size L. The results seem identical to those in table II, but differ between the 6th and 13th decimals, even if they have the same order of magnitude. 2) Message complexity: To generate a certificate, every node in S must first secure the leafset associated with C, then get the public keys of each node in C, and finally send a notification (ACK) to every node in C. Let nLEAF be the cost of getting the leafset associated with C. The total cost is then given by ntotal =

(L + 1)nLEAF {z } | Getleaf set

+

2(L + 1)2 + (L + 1)2 {z } | P ublic Keys + ACK

ntotal = (L + 1) (3(L + 1) + 5 × O(log2b (N )) + Q + 4)

and in the best case

Portability. We wish to emphasize that approach is modular, and therefore portable. Although our prototype relies on Pastry as its DHT, our approach can rely on other building block implementations. For example it is very simple to switch the underlying DHT from Pastry to Chord [10], as it maintains structures that are equivalent to the leafset and the routing table in Pastry.

ntotal = (L + 1) (3(L + 1) + O(log(N )))

This demonstrates that the certification protocol scales well as the size N of the DHT increases. V.

D ISCUSSION VI.

Similarly to all scientific contributions, our approach may raise several debatable issues such as its overall usefulness. In this section we try to address some of these issues, namely: (a) the reliability of our solution, (b) its applicability, and (c) its portability.

R ELATED W ORK

A considerable amount of former research addresses the issue of creating a scalable certification system. [11] and [12] attempt to extend the scalability of hierarchic approaches by adding DHT support. Both use Chord [10] to store copies of the certificates issued by a fully trusted CA. While this partially mitigates the formation of a bottleneck around the CA, the reconstruction and validation of a certificate from its copies requires a byzantine consensus which is extremely costly over an asynchronous network.

Reliability in Practical Terms. The reliability of a centralized certification authority depends mostly on the supporting host: its availability, its securedness and its capacity to handle significant amounts of concurrent requests. In practice this entails that a centralized CA is not foolproof: it is as trustworthy as 7

Another type of fully decentralized solution, distributed public-key infrastructures (PKIs) constitutes a potent way of delivering certificates. [13] identifies three main sub-classes of decentralized PKIs : web of trust, statistical, and hybrid approaches.

2012). We also thank Maricel Nuñez for her participation in the elaboration of a preliminary version of some of the algorithms presented in this paper.

Web of trust approaches. They rely on transitive trust models to coalesce nodes into chains of certification. For instance, KeyChains [14] combines an unstructured P2P lookup protocol with a PGP-like web of trust model to build certificate chains. However, the strength of a chain is determined by the weakest link. This makes the simple transitive trust assumed by KeyChains highly vulnerable, and thus unreliable.

[1] J. R. Douceur, “The sybil attack,” in Revised Papers from the First International Workshop on Peer-to-Peer Systems. Springer-Verlag, 2002, pp. 251–260. [2] United Nations Department of Economic and Social Affairs, “United Nations E-Government Survey 2012, E-Government for the People,” United Nations, Tech. Rep., 2012. [3] A. I. T. Rowstron and P. Druschel, “Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems,” in IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg. Springer-Verlag, 2001, pp. 329–350. [4] M. J. Fischer, N. A. Lynch, and M. S. Paterson, “Impossibility of distributed consensus with one faulty process,” Journal of the ACM, vol. 32, no. 2, pp. 374–382, Apr. 1985. [5] G. J. Simmons, “Symmetric and asymmetric encryption,” ACM Comput. Surv., vol. 11, no. 4, pp. 305–330, Dec. 1979. [6] M. Castro, P. Druschel, A. Ganesh, A. Rowstron, and D. S. Wallach, “Secure routing for structured peer-to-peer overlay networks,” in 5th symposium on Operating systems design and implementation. ACM, 2002, pp. 299–314. [7] S. Richmond and C. Williams, “Millions of internet users hit by massive sony playstation data theft,” 2011, the Telegraph. [8] J. Menn, “Key internet operator verisign hit by hackers,” 2012, Reuters. [9] M. Veron, O. Marin, S. Monnet, and Z. Guessoum, “Towards a scalable refereeing system for online gaming,” in 11th International Workshop on Network and Systems Support for Games (NetGames’2012), 2012. [10] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan, “Chord: A scalable peer-to-peer lookup service for internet applications,” in conference on Applications, technologies, architectures, and protocols for computer communications. ACM, 2001, pp. 149–160. [11] S. Ajmani, D. E. Clarke, C.-H. Moh, and S. Richman, “Conchord: Cooperative sdsi certificate storage and name resolution.” in International workshop on Peer-To-Peer Systems, vol. 2429, 2002, pp. 141–154. [12] A. Tassanaviboon and G. Gong, “A framework toward a self-organizing and self-healing certificate authority group in a content addressable network,” in 6th Int. Conference on Wireless and Mobile Computing, Networking and Communications. IEEE, 2010, pp. 614–621. [13] A. Datta, M. Hauswirth, and K. Aberer, “Beyond "web of trust": enabling p2p e-commerce,” in IEEE International Conference on ECommerce, 2003, pp. 303–312. [14] R. Morselli, B. Bhattacharjee, J. Katz, and M. A. Marsh, “Exploiting approximate transitivity of trust,” in International Conference on Broadband Communications, Networks, and Systems, 2007, pp. 515–524. [15] F. Lesueur, L. Mé, and V. V. T. Tong, “An efficient distributed PKI for structured P2P networks,” in 9th International Conference on Peer-toPeer Computing (P2P). IEEE, 2009, pp. 1–10. [16] L. Zhou, F. B. Schneider, and R. Van Renesse, “Coca: A secure distributed online certification authority,” ACM Trans. Comput. Syst., vol. 20, no. 4, pp. 329–368, 2002. [17] M. K. Reiter, M. K. Franklin, J. B. Lacy, and R. N. Wright, “The omega key management service,” Journal of Computer Security, vol. 4, no. 4, pp. 267–288, 1996. [18] R. Guerraoui, “Genuine atomic multicast in asynchronous distributed systems,” Theoretical Computer Science, vol. 254, pp. 297–316, 2001. [19] M. Reiter and S. Stubblebine, “Authentication metric analysis and design,” ACM Transactions on Information and System Security, vol. 2, no. 2, 1999. [20] V. Pathak and L. Iftode, “Byzantine fault tolerant public key authentication in peer-to-peer systems,” Computer Networks, vol. 50, no. 4, pp. 579–596, 2006. [21] E. Rosas, O. Marin, and X. Bonnaire, “CORPS: building a Community Of Reputable PeerS in distributed hash tables,” The Computer Journal, vol. 54, no. 10, pp. 1721–1735, 2011.

R EFERENCES

Statistical approaches. They form quorums made of multiple random and thus presumably independent peers. As in our approach, signing a certificate thus requires the cooperation of a minimal number of honest nodes. Many statistical solutions that scale, such as [15], [16] and [13], split private keys among the nodes and maintain virtual distributed search trees on top of structured P2P overlays to index the partial keys. The main problem with quorum-based approaches that disregard node reputation is that they exhibit a high probability of certification failure. Our solution compensates by coalescing nodes very efficiently; it allows to involve a high number of nodes and thus to reduce the probability of failure drastically. Similarly to our approach, Ω [17] relies on state-machine replication to provide a powerful and extensive PKI service. However, Ω requires atomic multicast which is impossible to implement with unreliable failure detectors over an asynchronous network [18]. Hybrid approaches. They form weighted quorums dependent on the relative trust that is acquired among peers by exchanging public key information [19], [20]. Despite the multiplication of alternate trust paths, guaranteeing the emergence of a fully trustworthy path remains impossible since a single malicious node can invalidate one or more paths. Besides, the authentication metrics used to quantify the reliability of a path induce a costly consumption of network and computation resources on every node. VII.

C ONCLUSION

This paper presents a quasi-certification authority on top of a DHT. It replicates transactions over multiple peers and retains identical results from a qualified majority to certify that a service has been carried out for a given client at a given time. As any system, be it centralized or distributed, our authority is subject to failures. Yet our solution remains highly reliable: under typical circumstances where 5% of the peers are malicious, its probability of failure can be as low as 1 × 10−14 (see Table IV). Our solution also inherits its scalability from the DHT it relies on, and is portable over various DHT implementations. As an extension of the contributions presented in this paper, we are currently working on reducing even further the probability of failure of our quasi-certification authority through the integration of a scalable trust mechanism [21] associated with the current protocol. We are also working on proving the liveness and the safety of our protocols through formal verification techniques. Acknowledgements. The work presented in this paper was partially funded by a CNRS/CONICYT grant (no. 25289 8