5th IFAC Workshop on Distributed Estimation and 5th on Distributed Control Networked 5th IFAC IFACinWorkshop Workshop onSystems Distributed Estimation Estimation and and 5th IFAC Workshop on Distributed Estimation and Control in Networked Systems Available online at www.sciencedirect.com Control in Networked Systems September 10-11, 2015. Philadelphia, USA, Control in Networked Systems September 10-11, 2015. Philadelphia, USA, September September 10-11, 10-11, 2015. 2015. Philadelphia, Philadelphia, USA, USA,
ScienceDirect
IFAC-PapersOnLine 48-22 (2015) 180–185
Secure cloud computing algorithms for Secure cloud computing algorithms for Secure cloud computing algorithms for Secure cloud computing algorithms for discrete constrained potential games discrete constrained potential games discrete constrained potential games discrete constrained potential games
∗ ∗ ∗ ∗ Yang Lu ∗ , Minghui Zhu ∗ ∗ , Minghui Zhu ∗ Yang Lu ∗ , Minghui Zhu ∗ Yang Lu Yang Lu , Minghui Zhu ∗ ∗ ∗ Department of Electrical Engineering, Pennsylvania State University, ∗ Department of Electrical Engineering, Pennsylvania State University, ∗ Department Electrical Engineering, Pennsylvania State University, 201 Oldof University Park, PA, 16802 USA (e-mail: Department ofMain, Electrical Engineering, Pennsylvania State University, 201 Old Main, University Park, PA, 16802 USA (e-mail: 201 Old Main, University Park, PA, 16802 USA (e-mail:
[email protected] (Y. Lu),
[email protected] (M. Zhu)). 201 Old Main, University Park, PA, 16802 USA (e-mail:
[email protected] [email protected] (Y. (Y. Lu), Lu),
[email protected] [email protected] (M. (M. Zhu)). Zhu)).
[email protected] (Y. Lu),
[email protected] (M. Zhu)). Abstract: In In this this paper, paper, we we study study secure secure cloud cloud computing computing problem problem for for aa class class of of discrete discrete Abstract: Abstract: In this paper, we study secure cloud computing problem for aa class of discrete constrained potential games. In the games, certain functions are confidential for the system Abstract: In this paper, we study secure cloud computing problem for class of discrete constrained games. the games, certain functions are confidential for the system constrained potential games. In the games, certain functions are confidential for the system operator andpotential not disclosed toIn any other participant. Meanwhile, each agent is unwilling to constrained potential games. In the games, certain functions are confidential for the system operator and not disclosed to any other participant. Meanwhile, each agent is unwilling to operator and not disclosed to any other participant. Meanwhile, each agent is unwilling to disclose its private functions and states to any other participant. By utilizing reinforcement operator and not disclosed toand anystates othertoparticipant. Meanwhile, each agent isreinforcement unwilling to disclose its private functions any other participant. By utilizing disclose its private functions and states to any other participant. By utilizing reinforcement learning and homomorphic encryption, we propose a distributed algorithm where (i) both both the the disclose its private functions and states to any other participant. By utilizing reinforcement learning and and homomorphic homomorphic encryption, encryption, we we propose propose aa distributed distributed algorithm algorithm where where (i) (i) learning both the confidentiality for the system operator and the privacy for the agents are protected; (ii) the learning and homomorphic encryption, we propose a distributed algorithm where (i) both confidentiality the system operator and the privacy for the agents are protected; (ii) the confidentiality for the system and the convergence to for Nash equilibria is formally ensured. confidentiality for theequilibria system operator operator and the privacy privacy for for the the agents agents are are protected; protected; (ii) (ii) the the convergence to Nash is formally ensured. convergence to Nash equilibria is formally ensured. convergence to Nash equilibria is formally ensured. © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Secure Secure computation, computation, cloud cloud computing, computing, potential potential games, games, reinforcement reinforcement learning, learning, Keywords: Keywords: Secure Secure computation, cloud cloud computing, computing, potential potential games, games, reinforcement reinforcement learning, learning, homomorphic encryption. Keywords: computation, homomorphic encryption. homomorphic encryption. homomorphic encryption. 1. INTRODUCTION We show that the proposed algorithm is able to protect 1. We show that the proposed algorithm able to protect 1. INTRODUCTION INTRODUCTION We show that the proposed algorithm is able to protect both the confidentiality for the system is operator and the 1. INTRODUCTION We show that the proposed algorithm is able to protect both the confidentiality for the system operator and the both the confidentiality for the system operator and the privacy for the agents and ensure the convergence of Nash theforconfidentiality forensure the system operator and the Distributed computation computation has has been been widely widely applied applied to to concon- both privacy the agents and the convergence of Nash Distributed privacy for the agents and ensure the convergence of Nash Distributed computation has been widely applied to conequilibrium with probability one. Due to the space limit, trol multi-agent networks due to a number of advantages; privacy for the agents and ensure the convergence of Nash Distributed computation has been widely applied to conequilibrium with probability one. Due to the space limit, trol multi-agent networks due aa number of with probability one. Due to the limit, trol multi-agent networks due to to In number of advantages; advantages; we omit most analysis in this paper. The equilibrium with probability one. Due to complete the space spaceresults limit, i.e., multi-agent scalability and and robustness. distributed computa- equilibrium trol networks due to a number of advantages; we omit most analysis in this paper. The complete results i.e., scalability robustness. In distributed computawe omit most analysis in this paper. The complete results of the paper are included in the technical report [Lu and i.e., scalability and robustness. In distributed distributed computawe omit most analysis in this paper. The complete results tion, distributed data sharing among the participants is i.e., scalability and robustness. In computaof the paper are included in the technical report [Lu and tion, distributed data sharing among the participants is of the paper are included in the technical report [Lu tion, distributed data sharing among the participants is Zhu, 2015]. the2015]. paper are included in the technical report [Lu and and necessary to achieve achieve network-wide goals. However, it also also tion, distributed datanetwork-wide sharing among theHowever, participants is of Zhu, necessary to goals. it Zhu, 2015]. 2015]. necessary toconcern achievethat network-wide goals. However, it also also Zhu, causes theto private andgoals. confidential informanecessary achieve network-wide However, it Literature review: causes the concern that and confidential informa- Literature review: causes the concern entities that private private and confidential information of of the legitimate couldand be confidential leaked to to unauthounauthocauses concern that private informa- Literature review: tion legitimate entities could be leaked Literature review: tion of legitimate entities could be leaked to unauthopaper studies aa class of potential games. Potential rized of entities. Such entities concerncould demands for new newtodistributed distributed tion legitimate be leaked unautho- This This paper studies potential games. Potential rized entities. Such concern demands for This paper studies aa class class of potential games. Potential rized entities. Such concern demands for new games were first proposed in of [Monderer and Shapley, 1996]. algorithms that can accomplish the tasks of distributed This paper studies class of potential games. Potential rized entities. Such concern demands for new distributed games were first proposed in [Monderer and Shapley, 1996]. algorithms that can accomplish the tasks of distributed games were first proposed in [Monderer and Shapley, 1996]. algorithms that can accomplish the tasks of distributed computation and simultaneously protect the information Several types of potential games have been studied, e.g., were first proposed ingames [Monderer and Shapley, 1996]. algorithms that can accomplish the tasksthe of information distributed games Several types of potential have been studied, e.g., computation and simultaneously protect Several types of potential games have been studied, e.g., computation and simultaneously protect the information ordinal potential games, cardinal potential games and security of legitimate entities. Several types of potential games have been studied, computation and simultaneously protect the information ordinal potential games, cardinal potential games e.g., and security of legitimate entities. ordinal potential games, cardinal potential games and security of legitimate entities. pseudo-potential games [Kukushkin, 1999], [Jehiel et al., ordinal potential games, cardinal potential games and security of legitimate entities. games [Kukushkin, 1999], [Jehiel et al., Contribution. This paper studies the secure cloud comput- pseudo-potential pseudo-potential games [Kukushkin, 1999], [Jehiel et al., 2004], [Dubey et al., 2006] and different algorithms for poContribution. This paper studies the secure cloud computpseudo-potential games [Kukushkin, 1999], [Jehiel et al., Contribution. This paper studies the secure cloud comput2004], [Dubey et al., 2006] and different algorithms for poing problem for a class of discrete constrained potential Contribution. This paper studies the secure cloud comput2004], [Dubey et al., 2006] and different algorithms for potential games have been developed such as best response, ing problem for aa class of discrete constrained potential 2004], [Dubey et al., 2006] and different algorithms for poing problem for class of discrete constrained potential games been developed such response, games. In the theforgames of of interest, there are certain confi- tential ing problem a class discretethere constrained potential tential response games have have been developed such as as best best response, better and MaxLogit [Voorneveld, 2000], [Song games. of interest, are confitential games have been developed such as best response, games. In the games games of the interest, there are certain certain confibetter response and MaxLogit [Voorneveld, 2000], [Song dential In functions which system operator is unwilling games. In the games of interest, there are certain confibetter response and MaxLogit [Voorneveld, 2000], [Song et al., 2011], [Candogan et al., 2013]. Recently, potential dential functions which the system operator is unwilling better response and MaxLogit [Voorneveld, 2000], [Song dential functions which the system operator is unwilling et al., 2011], [Candogan et al., 2013]. Recently, potential to disclose. In addition, each agent is unwilling to disclose dential functions which the system operator is unwilling et al., 2011], [Candogan et al., 2013]. Recently, potential games have been broadly applied to cooperative control, to disclose. In addition, each agent is unwilling to disclose et al., 2011], [Candogan et al., 2013]. Recently, potential to disclose. In addition, addition, each agent is unwilling unwilling to propose disclose games have been broadly applied to cooperative control, its disclose. private functions andeach states to any any others. We We to In agent is to disclose games have broadly applied to et 2009], [Zhu, its private and to games have been been broadly to cooperative cooperative control, control, its private functions functions and states states tointegrates any others. others. We propose propose e.g., e.g., [Marden [Marden et al., al., 2009],applied [Zhu, 2014]. 2014]. a distributed algorithm which reinforcement its private functions and states to any others. We propose e.g., [Marden et al., 2009], [Zhu, 2014]. aalearning distributed algorithm which integrates reinforcement e.g., [Marden et al., 2009], [Zhu, 2014]. distributed algorithm which integrates reinforcement Another set of relevant work is information security. There [Zhu and Mart´ ınez, 2013] and homomorphic ena distributed algorithm which integrates reinforcement Another set of relevant work is information security. There learning [Zhu and Mart´ ınez, 2013] and homomorphic enAnother set of relevant work is information security. There learning [Zhu and Mart´ ınez, 2013] and homomorphic enhave been different secure computing techniques, e.g., dcryption [Sen, 2013]. At each iteration, each agent, on one setdifferent of relevant workcomputing is information security.e.g., There learning [Zhu and Mart´ ınez, 2013] and each homomorphic en- Another have been secure techniques, dcryption [Sen, 2013]. At each iteration, agent, on one have been different secure computing techniques, e.g., dcryption [Sen, 2013]. 2013]. At each iteration, iteration, each agent, on one have ifferential privacy [Dwork et al., 2006], secure multiparhand, reinforces the successful state in the most recent been different secure computing techniques, e.g., dcryption [Sen, At each each agent, on one privacy [Dwork et al., 2006], secure multiparhand, reinforces the successful state in in the the most most recent recent ifferential ifferential privacy [Dwork et al., 2006], secure multiparhand, reinforces the successful state ty computation (SMC) [Shamir, 1979] and homomorphic history, and on the other hand, randomly explores feasible ifferential privacy [Dwork et al., 2006], secure multiparhand, reinforces the successful state in the most recent computation 1979] and homomorphic history, and on hand, randomly explores feasible ty ty computation (SMC) [Shamir, 1979] and history, and decision on the the other other hand, randomly feasible [Sen,(SMC) 2013]. [Shamir, Due to the probabilistic nature, states. The making is only basedexplores on the realized ty computation (SMC) [Shamir, 1979] and homomorphic homomorphic history, and on the other hand, randomly explores feasible encryption encryption [Sen, 2013]. Due to the probabilistic nature, states. The decision making is only based on the realized encryption [Sen, 2013]. Due to the probabilistic nature, states. The decision making is only based on the realized differential privacy cannot guarantee perfect correctness values of its private objective function and the realized [Sen, 2013]. Dueguarantee to the probabilistic nature, states. Theitsdecision making is only basedand on the the realized encryption differential privacy cannot perfect correctness values of private objective function differential privacy cannot guarantee perfect correctness values are of its its privatecomputed objective by function and the realized differential of computation. In existing SMC literature, the parties securely using a new homomorprivacy cannot guarantee perfect correctness values of private objective function and the realized of computation. In existing SMC literature, the parties values are securely computed by using a new homomorof computation. In existing SMC literature, the parties values are securely securely computed by computation using aa new new algorithm. homomor- collectively compute a function, where it requires that all phic encryption based polynomial of computation. In existing SMC literature, the parties values are computed by using homomorcollectively compute a function, where it requires that all phic encryption based polynomial computation algorithm. collectively compute a function, where it requires that phic encryption based polynomial computation algorithm. the parties know the structure of the function. HomomorFirst, the system operator computes each multiplication compute astructure function,ofwhere it requires that all all phic encryption based polynomial computation algorithm. collectively the parties know the the function. HomomorFirst, the system operator computes each multiplication the parties know the structure of the function. HomomorFirst, the system operator computes each multiplication phic encryption has been used in secure cloud computing, term of the polynomial by operating on the data encrypted the parties know the structure of the function. HomomorFirst, the system operator computes each multiplication phic encryption has been used in secure cloud computing, term of the polynomial by by operating operating on on the the data data encrypted phic encryption has been used in secure cloud computing, term ofagents, the polynomial e.g., encryption [L´ oopez-Alt et al., current work on by the theof who then thenbydecrypt decrypt theon computing results to to phic has been2012]. used However, in secure cloud computing, term the polynomial operating the data encrypted encrypted [L´ et al., 2012]. However, current work on by agents, who the computing results e.g., [L´ oopez-Alt pez-Alt et al., 2012]. However, current work on by the agents, who then decrypt the computing results to e.g., homomorphic encryption only considers the privacy for the obtain the true values of the terms. After that, together e.g., [L´ pez-Alt et al., 2012]. However, current work on by the agents, who then decrypt the computing results to homomorphic encryption only considers the privacy for the obtain the true values of the terms. After that, together homomorphic encryption only considers the privacy for the obtain the true values of the terms. After that, together with a third party, by using a non-singular masking matrix, agents but does not take into account the confidentiality homomorphic encryption only considers the privacy for the obtain the true values of the terms. After that, together but does not with aa third party, using non-singular masking matrix, agents but doesoperator. not take take into into account account the the confidentiality confidentiality with third party, by bycollectively using a non-singular masking matrix, agents for the system all the the participants perform an an inner product product agents but does not take into account the confidentiality with a third party, by using aa non-singular masking matrix, for the system operator. all participants collectively perform inner for the system operator. all the participants collectively perform an inner product operation to obtain the true value of the polynomial. for the system operator. all the participants collectively perform an inner product operation above short analysis indicates that it still lacks algooperation to to obtain obtain the the true true value value of of the the polynomial. polynomial. The The short indicates that it lacks operation to obtain the true value of the polynomial. The above above short analysis analysis indicates that it still still manner. lacks algoalgorithmic frameworks to solve games in a secure The above short analysis indicates that it still lacks algo This work was partially supported by ARO W911NF-13-1-0421 rithmic frameworks to solve games in aa secure manner. rithmic frameworks to solve games in secure manner. work was partially supported by ARO W911NF-13-1-0421 This rithmic frameworks to solve games in a secure manner. (MURI) and NSF grant CNS-1505664. This work was partially supported by ARO W911NF-13-1-0421 This work was partially supported by ARO W911NF-13-1-0421
(MURI) and NSF grant CNS-1505664. (MURI) (MURI) and and NSF NSF grant grant CNS-1505664. CNS-1505664. Copyright 2015 IFAC 180 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2015, IFAC (International Federation of Automatic Control) Copyright © 2015 IFAC 180 Copyright 2015 IFAC 180 Peer review© of International Federation of Automatic Copyright ©under 2015 responsibility IFAC 180Control. 10.1016/ifacol.2015.10.327
IFAC NecSys 2015 Yang Lu et al. / IFAC-PapersOnLine 48-22 (2015) 180–185 Sept 10-11, 2015. Philadelphia, USA
181
Notations: In this paper, the following notations will be used. 0n : zero vector of size n. Rn+ : set of non-negative real column vectors of size n. Rn++ : set of positive real column vectors of size n. [x]+ : projection of vector x = [x1 , · · · , xn ]T onto the nonnegative space, i.e., [x]+ = [¯ x1 , · · · , x ¯n ]T where for each i = 1, · · · , n, x ¯i = xi if xi ≥ 0 and x ¯i = 0 if xi < 0. |x|: entry-wise absolute operation on vector x, i.e., |x| = [|x1 |, · · · , |xn |]T .
xa : entry-wise exponent operation with exponent a, i.e., xa = [xa1 , · · · , xan ]T . P(E): probability that an event E happens.
x[−i] = [x[1]T , · · · , x[i−1]T , x[i+1]T , · · · , x[N ]T ]T ∈ Rn−ni the vector of all the other agents’ states except that of agent i, and by X−i = j=i Xj ⊆ Rn−ni the space of x[−i] . Given the joint state x[−i] , each agent i ∈ V aims to solve the following constrained optimization problem: minx[i] ∈Xi fi (x[i] , x[−i] )
s.t.
w.p.: abbreviation for the term “with probability”. PZ : projection operator onto a convex and compact set Z ⊆ Rn , i.e., for z ∈ Rn , PZ [z] = argminx∈Z x − z.
Given a non-negative scalar sequence {α(k)}k≥0 , it is ∞ α(k) < +∞ and square summable if summable if k=0 ∞ 2 α(k) < +∞. k=0 2. PROBLEM STATEMENT
In this section, we formalize the class of discrete constrained potential games and the security issues concerned in this paper. 2.1 Problem formulation ∆
Consider a set of agents V = {1, · · · , N }, a system operator (SO) and a third party (TP). The interconnections among them are represented by an undirected communication graph G = (V ∪ {SO, TP}, E) where E is the set of edges. If there is an (undirected) communication channel between i and j, where i, j ∈ V ∪ {SO, TP} and i = j, then (i, j), (j, i) ∈ E. Since G is undirected, we consider (i, j) and (j, i) as the same channel. Assumption 2.1. In G, (SO, TP) ∈ E. For each i ∈ V, (SO, i), (TP, i) ∈ E. For any pair i, j ∈ V, (i, j) ∈ / E. Figure 1 illustrates the interconnections between V ∪ {SO, TP}. The solid lines represent the communication channels. Many networked engineering systems operate in hierarchical structures as Figure 1, e.g., Internet, power grid and transportation systems, where an SO is placed at the top layer and agents are placed at the bottom layer [Wu et al., 2005], [Chiang et al., 2007]. Assumption 2.1 is widely used in, e.g., network flow control [Low and Lapsley, 1999]. The operation conducted by the SO can be viewed as cloud computing, where the SO is the cloud provider while the agents are remote data sources. The TP is introduced due to the security concern, as will be illustrated later. We point out that the TP is untrustful and could be malicious. [i]
Fig. 1. Interconnections between the agents, the system operator and the third party
[i]
Each agent i ∈ V has a state x[i] = [x1 , · · · , xni ]T ∈ Xi ⊆ Rni . Denote by x = [x[1]T , · · · , x[N ]T ]T ∈ Rn the overall N vector of all agents’ states where n = =1 n , and by X = N n i=1 Xi ⊆ R the space of x. For each i ∈ V, denote by
181
g [i] (x[i] , x[−i] ) ≤ 0pi ,
h[i] (x[i] , x[−i] ) = 0qi
(1)
where fi : Rn → R, g [i] : Rn → Rpi and h[i] : Rn → Rqi . [i] [i] [i] For each i ∈ V, denote g [i] = [g1 , · · · , gpi ]T with g : [i] [i] Rn → R for = 1, · · · , pi , and h[i] = [h1 , · · · , hqi ]T with [i] h : Rn → R for = 1, · · · , qi . Assumption 2.2. For all i ∈ V, Xi includes a finite number of states, pi = p and qi = q. Assumption 2.3. There exist functions F : Rn → R, G : Rn → Rp and H : Rn → Rq such that, for any i ∈ V, any x[−i] ∈ X−i and any x[i] , x ¯[i] ∈ Xi , it holds that [i] [−i] [i] [−i] [i] fi (x , x ) − fi (¯ x , x ) = F (x , x[−i] ) − F (¯ x[i] , x[−i] ), x[i] , x[−i] ) = G(x[i] , x[−i] )−G(¯ x[i] , x[−i] ) g [i] (x[i] , x[−i] )−g [i] (¯ [i] [i] [−i] [i] [i] [−i] [i] x , x ) = H(x , x[−i] ) − and h (x , x ) − h (¯ [i] [−i] H(¯ x , x ). By Assumptions 2.2 and 2.3, the collection of the coupled optimization problems (1) consists of a discrete constrained potential (DCP) game. The definition of Nash equilibrium (NE) of the DCP game is given as follows. Definition 2.1. An NE of the DCP game is a state x ˜∈X such that for all i ∈ V, g [i] (˜ x) ≤ 0p , h[i] (˜ x) = 0q , and for all x[i] ∈ Xi such that g [i] (x[i] , x ˜[−i] ) ≤ 0p , h[i] (x[i] , x ˜[−i] ) = x) ≤ fi (x[i] , x ˜[−i] ). 0q , it holds that fi (˜ Denote by XDC the set of NEs of the DCP game. 2.2 Confidentiality and privacy issues The distributed NE computation of the DCP game requires distributed data sharing between the participants which causes concerns of information leakage. This motivates us to study the security issue in cloud computing. In this paper, we are concerned with semi-honest adversaries and information-theoretic security. A participant is semihonest if it correctly follows the algorithm but attempts to use the received messages to infer private information of legitimate participants ([Hazay and Lindell, 2010], pp20). A cryptosystem is information-theoretically secure if its security derives purely from information theory, i.e., if an adversary does not have enough information, then it cannot break the system even if it has unlimited computing power [Shannon, 1949]. Assume that in G, each (i, j) ∈ E is secure, that is, only i and j can access the message sent via (i, j) and no other participant can access the message.
IFAC NecSys 2015 Yang Lu et al. / IFAC-PapersOnLine 48-22 (2015) 180–185 Sept 10-11, 2015. Philadelphia, USA
182
In this paper, we consider both the confidentiality issue for the SO and the privacy issue for the agents. Confidentiality. For each agent i ∈ V, there are ri subfunctions in fi , g [i] and h[i] which are private pieces of information of the SO and the SO is unwilling to disclose the structures of these subfunctions to V ∪ {TP}. This is the confidentiality issue for the SO. Denote by [i] [i] F [i] = [F1 , · · · , Fri ]T : Rn → Rri the vector of such subfunctions in agent i’s optimization problem defined by (1). Given any state x, the SO is willing to publicize the value of F [i] (x) to agent i. Denote by F = [F [1]T , · · · , F [N ]T ]T the overall confidential functions for the SO. Privacy. Each agent i is not willing to share the structures of its cost function fi , constraints g [i] , h[i] and the value of its state x[i] with (V\{i}) ∪ {SO, TP}. This is the privacy issue for the agents. For each i ∈ V, denote W [i] = [fi , g [i]T , h[i]T ]T : Rn → R1+p+q . Assumption 2.4. For each i ∈ V: 1) For = 1, · · · , ri , [i] F is a scalar polynomial function of x; 2) There exists a ˜ [i] : Rni × Rri → R1+p+q such that W [i] (x) = function W [i] [i] ˜ [i] . ˜ W (x , F [i] (x)) and agent i knows the structure of W By Assumption 2.4, the confidentiality and privacy issues concerned in this paper reduce to the problem of secure polynomial computation. The polynomial assumption is valid in many applications, e.g., the sum-of-squares (SOS) optimization, where the cost function is linear and the constraints are polynomials with SOS property [Prajna et al., 2004], and the model predictive control, where the optimization problem therein usually adopts a quadratic cost function with linear constraints [Lu et al., 2014]. 2.3 Objective The objective of this paper is to design a distributed cloud computing algorithm to solve the DCP game such that the convergence of NE is ensured and both the confidentiality for the SO and the privacy for the agents are protected. 3. SECURE POLYNOMIAL COMPUTATION ALGORITHM We pointed out in the last section that, by Assumption 2.4, the security issues concerned in this paper reduce to the problem of secure polynomial computation. In this section, we propose a homomorphic encryption based secure polynomial computation algorithm. First, we illustrate some preliminaries for homomorphic encryption. After that, we present the algorithm and its analysis. 3.1 Preliminaries for homomorphic encryption Homomorphic encryption is a form of encryption that allows computations to be carried out on ciphertext, thus generating an encrypted result which, when decrypted, matches the result of operations performed on the plaintext. Based on different encryption methods, a homomorphic encryption scheme can be classified as deterministic or probabilistic. Roughly speaking, if some random parameters are used in the encryption operation, then the scheme 182
is probabilistic, otherwise, it is deterministic. In this paper, because we want to achieve perfect computation correctness, we are concerned with deterministic homomorphic encryption scheme which is defined as follows. Definition 3.1. Let S be the message space and be a binary operation. A deterministic homomorphic encryption scheme on (S, ) is a quadruple (K, E, D, T ), where K is the key-generation algorithm, E is the encryption algorithm, D is the decryption algorithm and T is a binary operation with desired homomorphic property, such that, for any key ke generated by K, for any c, c ∈ S, it holds that D(T (E(c, ke ), E(c , ke )), ke ) = c c . The intuition of the above definition is that homomorphic encryption allows computations to be carried out on ciphertext, thus generating an encrypted result which, when decrypted, matches the result of operations performed on the plaintext. In this paper, we use the multiplicatively homomorphic encryption scheme, that is, in Definition 3.1 is the multiplication operation. There are several classical multiplicatively homomorphic encryption schemes such as RSA (named by the initials of the surnames of the inventors, Ron Rivest, Adi Shamir and Leonard Adleman) and ElGamal. In this paper, we choose RSA [Rivest et al., 1978] as our implementing scheme. In RSA, the key ke is a positive scalar a ∈ R++ . The encryption operation of a message c ∈ R is E(c, a) = ca . The decryption operation 1 of an encrypted value d ∈ R is D(d, a) = d a . The binary operation is the multiplication operation. We then have, for any c, c ∈ R, D(T (E(c, a), E(c , a)), a) = cc . In this way, if two agents send their encrypted inputs ca and c a to the SO and ask the SO to compute and send back the product of the two encrypted inputs, (cc )a , they can decrypt (cc )a and obtain the desired product cc . During this process, since the SO does not know the value of a, it cannot infer the values of c or c . 3.2 Secure polynomial computation algorithm For convenience of notation, for a polynomial of x, we call each term without the coefficient a multiplication term. [1] [2] [1] [3] For example, for the polynomial 6x1 x2 + 4x2 x3 + 5, [1] [2] [1] [3] x1 x2 , x2 x3 and 1 are multiplication terms with coefficients 6, 4 and 5, respectively. For i ∈ V, for j = 1, · · · , ri , mi,j [i] [i] [i] let Fj (x) = =1 Cj, Qj, , where for = 1, · · · , mi,j , [i]
[i]
Qj, is a multiplication term with coefficient Cj, . Denote [i]
[i]
[i]
[i]
[i]
[i]
Qj = [Qj,1 , · · · , Qj,mi,j ]T and Cj = [Cj,1 , · · · , Cj,mi,j ]T . Before running Algorithm 1, all the agents agree on a RSA key a ∈ R++ which is unknown to {SO, TP}. By running (OUT[1] , · · · , OUT[N ] ) =alg3(IN[1] , · · · , IN[N ] ), the algorithm takes IN[i] as the input for agent i and outputs OUT[i] to agent i. Denote IN= [IN[1]T , · · · , IN[N ]T ]T .
To help understand the steps of Algorithm 1, we first provide a simple example for concreteness. Consider the polynomial 6x[1] x[2] + 4x[1] x[3] + 5 where x[1] , x[2] and x[3] are scalars. First, for i = 1, 2, 3, each agent i sends (x[i] )a to the SO. The SO computes the encrypted multiplication terms (x[1] x[2] )a and (x[1] x[3] )a . Notice that the coefficients are not included in the computation at this step. This is because that the RSA cannot deal with multiplication
IFAC NecSys 2015 Yang Lu et al. / IFAC-PapersOnLine 48-22 (2015) 180–185 Sept 10-11, 2015. Philadelphia, USA
by coefficients. We will deal with the multiplication by coefficients and the addition of the terms by using the TP and a masking matrix. The SO generates an arbitrary nonsingular 3-by-3 masking matrix B, say, B1 = [1, 2, 3]T , B2 = [2, 1, 0]T and B3 = [3, 0, 1]T , where Bi is the i-th column of B for i = 1, 2, 3. The SO then sends ((x[1] x[2] )a , B1 ) to agent 3, ((x[1] x[3] )a , B2 ) to agent 2 and (1, B3 ) to agent 1, respectively (the way of distributing the encrypted multiplications terms is not unique). Agent 1 decrypts 1, which is still 1, and sends B3 to the TP. Agent 2 decrypts (x[1] x[3] )a to obtain x[1] x[3] and sends x[1] x[3] B2 to the TP. Agent 3 decrypts (x[1] x[2] )a to obtain x[1] x[2] and sends x[1] x[2] B1 to the TP. The SO sends [6, 4, 5]B −1 to the TP. The TP then computes [6, 4, 5]B −1 (x[1] x[2] B1 + x[1] x[3] B2 + B3 ) = [6, 4, 5]B −1 B[x[1] x[2] , x[1] x[3] , 1]T = [6, 4, 5][x[1] x[2] , x[1] x[3] , 1]T = 6x[1] x[2] + 4x[1] x[3] + 5, which is the desired polynomial. Notice that the step of multiplication by coefficients and the addition of the terms is an inner product operation, that is, for the example here, to compute [6, 4, 5][x[1] x[2] , x[1] x[3] , 1]T . This inner product operation cannot be performed by the SO, because otherwise the SO has to know the true values of x[1] x[2] and x[1] x[3] . Since the SO also knows the encrypted values (x[1] x[2] )a and (x[1] x[3] )a , it will then be able to infer the value of a and further infer the values of x[1] , x[2] and x[3] . The inner product operation cannot be performed by any of the agents either, since otherwise the agent has to know the coefficients of the polynomial. Therefore, to protect both the confidentiality for the SO and the privacy for the agents, we introduce the TP to perform the inner product operation. The usage of the non-singular matrix B is to mask the values of the multiplication terms and the coefficients from the TP. Since the multiplication terms sent to the TP are multiplied by B and the coefficients sent to the TP are multiplied by B −1 , and the TP does not know B or B −1 , it cannot infer the values of the multiplication terms or the coefficients. When the TP performs the multiplication operation, B and B −1 cancels and the value of the desired polynomial is correctly computed. The general steps of Algorithm 1 are informally stated as follows. First, each agent i ∈ V encrypts its input by RSA, i.e., s[i] = (IN[i] )a , and sends s[i] to the SO. Then, for each agent i, there are ri rounds such that, at each round j, [i] [i] OUTj = Fj (IN) is computed and sent to agent i. For each agent i ∈ V, at round j, first, the SO computes each [i] encrypted version of Qj, by treating s[l] as IN[l] for l ∈ V. [i] [1] [2] That is, e.g., if Qj, = (x1 )2 x2 , then the SO computes ¯ [i] = (IN[1] )2 IN[2] . The SO then generates a non-singular Q 1 2 j, [i] [i] [i] [i] mi,j -by-mi,j matrix Bj = [Bj,1 , · · · , Bj,mi,j ], where Bj, [i] is the -th column of Bj . After that, the SO sends each [I] [i] ¯ , B ) to the agents in the way described in the pair (Q j, j,
algorithm. The specific way of data sending at this step is to ensure that no individual agent can infer the structure of the processed polynomial, as will be shown in the following ¯ [i] it analysis part. Then, each agent l ∈ V decrypts each Q j, [i] [i] 1 ˆ ¯ a receives from the SO by computing Q = (Q ) . Each [i]
agent i then computes Yj, =
j, [i] ˆ [i] Bj, Q j,
j,
and sends it to
183
183
[i]
[i]
[i]
the TP, while the SO sends Zj = (Cj )T (Bj )−1 to the [i] [i] mi,j [i] TP. The TP then computes OUTj = Zj =1 Yj, and [i]
sends it to agent i. Here, the matrix Bj has the masking ˆ [i] and C [i] from function, i.e., it masks the values of Q j j the TP. We will show in the following analysis part that [i] [i] OUTj = Fj (IN). After all the ri rounds, agent i forms [i]
T [i] the vector OUT[i] = [OUT1 , · · · , OUT[i] ri ] = F (IN).
Algorithm 1: Secure polynomial computation algorithm Syntax: (OUT[1] , · · · , OUT[N ] ) =alg3(IN[1] , · · · , IN[N ] ) for i ∈ V do Agent i sends s[i] = E(IN[i] , a) = (IN[i] )a to the SO; for i ∈ V do for j = 1, · · · , ri do [i] ¯ [i] , The SO computes each encrypted Qj, , denoted by Q j, for = 1, · · · , mi,j , by treating s[l] as IN[l] for l ∈ V; The SO generates a non-singular mi,j -by-mi,j matrix [i]
[i]
[i]
Bj = [Bj,1 , · · · , Bj,m
i,j
];
¯ [i] , B [i] ) (Q j, j,
for = 1, · · · , mi,j to the The SO sends agents such that, each pair is sent to only one agent, not ¯ [i] all the pairs are sent to a single agent, and if some Q j, only includes the data of a single agent l ∈ V, then this pair is not sent to agent l; for l ∈ V do ¯ [i] it holds by computing Agent l decrypts each Q j, ¯ [i] , a) = (Q ¯ [i] ) a1 ; ˆ [i] = D(Q Q j, j, j, [i] [i] ˆ [i] ˆ [i] it for each Q Agent l computes Yj, = Bj, Q j, j, [i]
holds and sends each Yj, to the TP; [i]
[i]
[i]
The SO sends Zj = (Cj )T (Bj )−1 to the TP; The TP sends
[i] OUTj
=
[i] Zj [i]
mi,j =1
[i]
Yj, to agent i; [i]
Agent i forms OUT[i] = [OUT1 , · · · , OUTri ]T ;
3.3 Analysis Theorem 3.1. Under Assumptions 2.1 and 2.4, by Algorithm 1, the following claims hold: 1) Correctness: For each i ∈ V, OUT[i] = F [i] (IN).
2) Confidentiality: For the SO, the structure of F is unknown to V ∪ {TP}.
3) Privacy: For each agent i, the value of its input IN[i] is unknown to (V\{i}) ∪ {SO, TP}. Remark 3.1. In existing secure cloud computing literature, only the privacy for the agents is considered. Algorithm 1 is able to protect both the confidentiality for the SO and the privacy for the agents. Now we analyze the online computational complexity of [i] computing one polynomial by Algorithm 1. We choose Fj for concreteness, where i ∈ V and j ∈ {1, · · · , ri }. Assume [i] [i] that in Fj , there are totally oj times of multiplications. [i]
[i]
The generation of Bj and the computation of Zj can be done off-line by the SO beforehand, thus, we do not count these operations into the online computational complexity. [i] To compute Fj , it collectively takes the participants
IFAC NecSys 2015 Yang Lu et al. / IFAC-PapersOnLine 48-22 (2015) 180–185 Sept 10-11, 2015. Philadelphia, USA
184
[i]
2N times of exponent computations, oj + mi,j times of multiplications, m2i,j + 2mi,j − 2 times of additions and N + 2mi,j + 2 times of communications. 4. DISCRETE POTENTIAL GAMES In this section, by integrating reinforcement learning and homomorphic encryption, we propose a distributed secure cloud computing algorithm to solve the DCP game. 4.1 Game transformation
[i]
[i]
Xi , x[−i] ∈ X−i |h (x[i] , x[−i] ) = 0, |h (x [i] , x[−i] )| < [i] |h (x[i] , x[−i] )|}. Lemma 4.1. If µ > σσµmax for all = 1, · · · , p and ,min σmax λ > σλ ,min for all = 1, · · · , q, then any NE of the DUP(µ, λ) game is an NE of the DCP game. By Assumption 2.4, for each i ∈ V, there exists a function P˜i : Rni × Rri × Rp+ × Rq+ → R such that Pi (x, µ, λ) = P˜i (x[i] , F [i] (x), µ, λ) and agent i knows the structure of P˜i . 4.2 Distributed computation algorithm
To relax the constraints, we adopt the approach of penalty functions. For each i ∈ V, define the local penalty function ∆ Pi : Rn × Rp+ × Rq+ → R by Pi (x, µ, λ) = fi (x) + T [i] + T [i] µ [g (x)] + λ |h (x)|, where µ = [µ1 , · · · , µp ]T ∈ Rp+ and λ = [λ1 , · · · , λq ]T ∈ Rq+ . Fix µ and λ. Then, it can be checked that H(x, µ, λ) = F (x)+µT [G(x)]+ +λT |H(x)| is a potential function such that for any i ∈ V, any x[−i] ∈ X−i and any x[i] , x ¯[i] ∈ Xi , it holds that Pi (x[i] , x[−i] , µ, λ) − [i] [−i] x , x , µ, λ) = H(x[i] , x[−i] , µ, λ)−H(¯ x[i] , x[−i] , µ, λ). Pi (¯
For each i ∈ V, consider the following discrete unconstrained optimization problem parameterized by µ and λ: minx[i] ∈Xi Pi (x[i] , x[−i] , µ, λ).
(2)
Due to the existence of potential function H, the collection of the coupled optimization problems (2) consists of a discrete unconstrained potential game parameterized by µ and λ (DUP(µ, λ) game). The definition of NE of the DUP(µ, λ) game is presented as follows. Definition 4.1. A joint state x ˜ ∈ X is an NE of the DUP(µ, λ) game if it holds that Pi (˜ x, µ, λ) ≤ ˜[−i] , µ, λ), ∀x[i] ∈ Xi , ∀i ∈ V. Pi (x[i] , x Denote by XDU (µ, λ) the set of NEs of the DUP(µ, λ) game. Since the DUP(µ, λ) game is a potential game, XDU (µ, λ) = ∅. We now proceed to study the relations between the DCP game and the DUP(µ, λ) game. Before that, we have the following assumption. Assumption 4.1. For any infeasible x ¯ ∈ X, there exists i ∈ V for which x ¯ ∈ / {x ∈ X|g [i] (x) ≤ 0p , h[i] (x) = 0q } and agent i can unilaterally deviate from x ¯[i] to another [i] [i] x [i] , x ¯[−i] )]+ ≤ [g (¯ x)]+ state x ¯ [i] ∈ Xi such that [g (¯ [i] [i] [i] [−i] for all = 1, · · · , p, |h (¯ x ,x ¯ )| ≤ |h (¯ x)| for all = 1, · · · , q, and: (i) there exists at least one j ∈ {1, · · · , p} [i] [i] [i] [i] such that gj (¯ x) > 0 and gj (¯ x ,x ¯[−i] ) < gj (¯ x); (ii) or there exists at least one j ∈ {1, · · · , q} such that [i] [i] [i] [i] hj (¯ x) = 0 and |hj (¯ x ,x ¯[−i] )| < |hj (¯ x)|; (iii) or both. For convenience of notation, denote: σmax = maxi∈V maxΘi (fi (x[i] , x[−i] ) − fi (x [i] , x[−i] )), [i] [i] σµ ,min = mini∈V minΞi (g (x[i] , x[−i] )−[g (x [i] , x[−i] )]+ ) for = 1, · · · , p, [i] [i] σλ ,min = mini∈V minΨi (|h (x[i] , x[−i] )|−|h (x [i] , x[−i] )|) for = 1, · · · , q, where Θi = {x[i] , x [i] ∈ Xi , x[−i] ∈ X−i }, Ξi = [i] {x[i] , x [i] ∈ Xi , x[−i] ∈ X−i | g (x[i] , x[−i] ) > 0, [i] [i] g (x [i] , x[−i] ) < g (x[i] , x[−i] )} and Ψi = {x[i] , x [i] ∈ 184
In this subsection, we propose a distributed algorithm by which the participants collectively compute NE of the DCP game and the confidentiality for the SO and the privacy for the agents are both protected. Due to the security issues, we cannot adopt the bestresponse or better-response methods which are classical ways to solve potential games. Because, to compute best or better responses, each agent needs to completely know its own objective function and other agents’ states [Monderer and Shapley, 1996]. We address this problem by using reinforcement learning and homomorphic encryption. ˜ Before running Algorithm 2, the agents agree on µ ˜, λ whose values are fixed during the algorithm. The way of ˜ will be illustrated by Theorem 4.1 in choosing µ ˜ and λ the next subsection. Algorithm 2 is informally stated as follows. At each iteration k, each agent i first computes (k) whose value is small and diminishing as k increases to infinity. Then, each agent i chooses τi (k) to be the iteration number with the lower cost within the last two iterations. Here, x[i] (τi (k)) is the more successful action of agent i in the last two iterations. Then, each agent i uniformly picks any state from Xi \{x[i] (τi (k))} as the new state x[i] (k + 1) with probability (k), and with probability 1−(k), chooses the new state x[i] (k + 1) by reinforcing the more successful decision x[i] (τi (k)) in the last two iterations. Then, the participants collectively run Algorithm 1 which outputs v [i] (k + 1) = F [i] (x(k + 1)) to agent i. After that, with v [i] (k + 1), each agent i computes the cost P˜i at the next iteration k +1. Note that, since each agent i does not know the structure of F [i] , it cannot directly compute P˜i . Also, due to the privacy issue, the agents cannot directly send their states to the SO and let the SO do the computation. 4.3 Analysis ∆
Observe that z(k) = (x(k − 1), x(k)) in Algorithm 2 constitutes a Markov chain on the space X ×X. The following theorem states that the algorithm asymptotically con∆ ¯ DC = {(s, s)|s ∈ XDC } verges in probability to the set X and at the same time the confidentiality for the system operator and the privacy for the agents are protected. Theorem 4.1. Under Assumptions 2.1, 2.3, 2.4 and 4.1, max ˜ > σmax by choosing µ ˜ > σσµ,min for all = 1, · · · , p and λ σλ,min for all = 1, · · · , q, by Algorithm 2, the following claims hold: ¯ DC ) = 1. 1) Correctness: limk→+∞ P(z(k) ∈ X
IFAC NecSys 2015 Yang Lu et al. / IFAC-PapersOnLine 48-22 (2015) 180–185 Sept 10-11, 2015. Philadelphia, USA
Algorithm 2: Distributed secure cloud computing algorithm for i ∈ V do Agent i uniformly picks x[i] (0) ∈ Xi ; The participants collectively run (v [1] (0), · · · , v [N ] (0)) = alg3(x[1] (0), · · · , x[N ] (0)); for i ∈ V do ˜ ˜i (x[i] (0), v [i] (0), µ Agent i computes ui (0) = P ˜, λ); for i ∈ V do Agent i uniformly picks x[i] (1) ∈ Xi ; The participants collectively run (v [1] (1), · · · , v [N ] (1)) = alg3(x[1] (1), · · · , x[N ] (1)); for i ∈ V do ˜ ˜i (x[i] (1), v [i] (1), µ Agent i computes ui (1) = P ˜, λ); while k ≥ 1 do for i ∈ V do
−
N (|
N1
i=1 Agent i computes (k) = k Agent i computes τi (k) as follows: if ui (k) ≤ ui (k − 1) then τi (k) = k;
else τi (k) = k − 1;
Xi |+1)
;
[i]
Agent i uniformly picks xtp ∈ Xi \ {x[i] (τi (k))} w.p. (k) [i]
and chooses xtp = x[i] (τi (k)) w.p. 1 − (k), and sets
x[i] (k
[i]
+ 1) = xtp ;
The participants collectively run (v [1] (k + 1), · · · , v [N ] (k + 1)) = alg3(x[1] (k + 1), · · · , x[N ] (k + 1)); for i ∈ V do Agent i computes ˜ ˜i (x[i] (k + 1), v [i] (k + 1), µ ui (k + 1) = P ˜, λ); k = k + 1;
2) Confidentiality: For the SO, the structure of F is unknown to V ∪ {TP}.
3) Privacy: For each agent i, the structures of fi , g [i] and h[i] and the values of x[i] (k) at each iteration k are unknown to (V\{i}) ∪ {SO, TP}. 5. CONCLUSION
We have proposed a secure cloud computing algorithm for discrete constrained potential games. The algorithm is able to protect both the confidentiality for the system operator and the privacy for the agents via homomorphic encryption, and its convergence is ensured by reinforcement learning. One future direction is to investigate secure computation for continuous convex games. REFERENCES Candogan, O., Ozdaglar, A., and Parrilo, P.A. (2013). Dynamics in near-potential games. Games and Economic Behavior, 82, 66–90. Chiang, M., Low, S.H., Calderbank, A.R., and Doyle, J.C. (2007). Layering as optimization decomposition: A mathematical theory of network architectures. Proceedings of IEEE, 95(1), 255–312. Dubey, P., Haimanko, O., and Zapechelnyuk, A. (2006). Strategic complements and substitutes, and potential games. Games and Economics Behvior, 54(1), 77–94. 185
185
Dwork, C., McSherry, F., Nissim, K., and Smith, A. (2006). Calibrating noise to sensitivity in private data analysis. Lecture Notes in Computer Science, 3876, 265– 284. Hazay, C. and Lindell, Y. (2010). Efficient Secure Two-Party Protocols–Techniques and Constructions. Springer. Jehiel, P., ter Vehn, M.M., and Moldovanu, B. (2004). Potential and implementation. University of Bonn. Kukushkin, N.S. (1999). Potential games: a purely ordinal approach. Economics Letters, 64(3), 279–283. L´opez-Alt, A., Tromer, E., and Vaikuntanathan, V. (2012). On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption. Proceedings of the 44th annual ACM symposium on Theory of computing, 1219–1234. Low, S. and Lapsley, D. (1999). Optimization flow control, I: Basic algorithm and convergence. IEEE/ACM Transactions on Networking, 7(6), 861–874. Lu, Y., Li, D., Xu, Z., and Xi, Y. (2014). Convergence analysis and digital implementation of a discrete-time neural network for model predictive control. IEEE Transactions on Industrial Electronics, 61(12), 7035– 7045. Lu, Y. and Zhu, M. (2015). Secure cloud computing algorithms for discrete constrained potential games. Technical report, Pennsylvania State University. Marden, J.R., Arslan, G., and Shamma, J.S. (2009). Cooperative control and potential games. IEEE Transactons on Systems, Man, and Cybemetics–Part B: Cybernetics, 39(6), 1393–1407. Monderer, D. and Shapley, L. (1996). Potential games. Games and Economic Behavior, 14, 124 – 143. Prajna, S., Parrilo, P.A., and Rantzer, A. (2004). Nonlinear control synthesis by convex optimization. IEEE Transactions on Automatic Control, 49(2), 310–314. Rivest, R.L., Shamir, A., and Adleman, L. (1978). A method for obtaining digital signatures and public-key cryptosystems. Communications of the ACM, 21(2), 120–126. Sen, J. (2013). Homomorphic encryption: Theory and application. [Online] http://arxiv.org/ftp/arxiv/papers/1305/1305.5886.pdf. Shamir, A. (1979). How to share a secret. Communications of the ACM, 22(11), 612–613. Shannon, C.E. (1949). Communication theory of secrecy systems. Bell System Technical Journal, 28(4), 656–715. Song, Y., Wong, S.H.Y., and Lee, K.W. (2011). Optimal gateway selection in multi-domain wireless networks: a potential game perspective. Proceedings of the 17th annual international conference on Mobile computing and networking, 325–336. Voorneveld, M. (2000). Best-response potential games. Economics Letters, 66(3), 289–295. Wu, F., Moslehi, K., and Bose, A. (2005). Power system control centers: Past, present and future. Proceedings of the IEEE, 93(11), 1890 – 1908. Zhu, M. (2014). Distributed demand response algorithms against semi-honest adversaries. In IEEE PES General Meeting. No. 943. Zhu, M. and Mart´ınez, S. (2013). Distributed coverage games for energy-aware mobile sensor networks. SIAM Journal on Control and Optimization, 51(1), 1–27.
