Demand Response Aggregator Stackelberg ...

Demand Response Aggregator Stackelberg Competition for Selling Stored Energy Mahdi Motalleb, John Branigan, Reza Ghorbani Renewable Energy Design Laboratory (REDLab), Department of Mechanical Engineering University of Hawaii at Manoa Honolulu, Hawai'i [email protected], [email protected], [email protected]. Abstract-This work is concerned with the application of gametheoretic principles in a power market consisting of demand response aggregators which compete to sell energy stored in batteries to other aggregators. The amount and price of power transacted is controlled by the utility (as leader), and the sellers do not share information about their bidding strategy, or parameters determining their cost of providing the power, hence the competition takes the form of an incomplete-information Stackelberg game. The optimal bidding strategy for maximizing payoff is determined by the Nash equilibrium. The model is used to examine the effect of price-sensitive demand response scheduling by comparison with a similar game without scheduling. In an application to case study data from the island of Maui, USA, we show that the player who implements it markedly increases payoff compared to a player who does not. Index Terms—Demand Response Aggregator (DRA), Stackelberg game, Price-sensitive demand response scheduling, Nash equilibrium

I.

INTRODUCTION

Increasing prevalence of intermittent renewable energy generation has motivated interest in Demand Response (DR) technologies which may serve to provide ancillary services [1], curtail usage during peak demand times [2], allow customers to save money by adjusting usage patterns in response to fluctuating electricity prices [3]. Game theoretic approaches have been successful in modeling market interactions between Demand Response Aggregators (DRAs) and power generators [4]. Games considering price based DR mechanisms designed to motivate the consumers to adjust their electricity consumption may be formulated to effectively predict rational selfish player’s decisions [5], and provide efficient scheduling of controllable loads [6]. Optimal bidding strategies have been derived using game-theory for trading in various power markets including virtual power plant market cooperation [7]. Research done in-house at the REDlab has analyzed the effectiveness of employing energy storage systems for peak load shaving and smoothing under conditions of high penetration of renewable energy generation [8] and determined their optimal sizing and location [9]. Our work has also shown that control of loads such as Water Heaters (WHs), and battery systems coupled with dynamic supply pricing strategies provide a feasible comprehensive system for DR [10, 11]. A Stackelberg game-based DR algorithm proved effective for achieving optimal load control of devices [12]. Stackelberg games have been used to model DR management in the smart-

grid where utility companies compete [13]. In [14], some rules have been produced for learning Nash equilibrium strategies of opponents over multiple periods by simply observing their own sales and without knowing anything about each other. Building on these works, we now propose a Stackelberg competition model of a market structure where DRAs (as the players of the game) which oversee and manage the resources of a number of buildings, may compete to buy and sell power stored in batteries directly to each other. This framework is an initial step in development of a functional market for trading energy produced by independent renewable energy resources intermittently and stored to be sold on demand. In the model presented in this paper, batteries are initially charged from the grid. The cost payoff to a DRA depends on this price. In a Stackelberg game, the market contains a leader empowered by the utility to regulate the price and size of transactions. We consider two variants of game: with Price-Sensitive (P-S) DR scheduling where large load device operating time is set in response to the grid price of electricity; and without, where scheduling is set following typical use patterns. II.

AGGREGATOR COST, PAYOFF, AND BIDDING

A. Aggregator’s Cost In our game, the only energy available to a DRA for sale is that which is stored in its batteries, so that the cost to the DRA is the discharging cost. Each aggregator controls numerous houses. We assume that the cost of discharging the batteries in house 𝑖, Chbat , is a function of the energy stored, ΔEhi , i adequately represented by a logarithmic barrier function [15]: ΔEh

i Chbat (ΔEhi ) = −a hi log(1 − ) (1) i B where B is a parameter which serves as the maximum typical value of ΔEhi , and a hi is a pricing coefficient set by the utility. We may find the coefficient values by writing the cost, Chbat , i in terms of known operating plus capital and maintenance cost data. Truncating the Taylor series approximation of (1) after the second order term, for each house managed by the DRA, gives quadratic cost functions. Rewriting cost as a function of power (updated every 15 min. and assumed constant during the interim), we may express the DRA 𝑔’s cost, Cg (Pg ), of selling stored power, Pg , as the sum of the cost functions of the Nh houses it manages, which is also quadratic : Nh bat Cg (Pg ) = ∑i=1 Chi (ΔPhi ) = vg Pg + ug Pg2 (2)

where vg =

0.25 BNh

N

h ∑i=1 a hi and ug =

0.252 B 2 Nh

N

h ∑i=1 a hi .

B. Aggregator’s Payoff For a given spot market price, 𝜙, the 𝑔th DRA’s payoff is: R g = −ΔCg + ϕTg (3) where ΔCg is the difference between the discharging cost of the batteries managed by aggregator 𝑔 before and after power transactions, and Tg is the net power transacted [16]. For each power transaction,T , consider the pair of aggregator payoffs (R g1 , R g2 ) for arbitrary DRAs, g1 , g 2 , with the pair (0,0) corresponds to no transaction taking place. We seek the payoff at the Nash equilibrium, (R∗g1 , R∗g2 ) , i.e. the pair of payoffs given R∗g1 > 0 and R∗g2 > 0, such that: R∗g1 R∗g2 > R g1 R g2 for all possible payoff combinations (𝑅𝑔1 , 𝑅𝑔2 ). At Nash equilibrium neither rational participant has any incentive to alter strategy as it provides the greatest minimum payoff that can be guaranteed despite other’s strategy. If a DRA participates in 𝑘 ∈ 𝐾 contracts, the objective function to maximize becomes: 𝑘 ∑𝑘∈𝐾 ∏𝜙𝑖𝑗 ,𝑇𝑖𝑗 𝑅𝑖𝑗 (𝜙𝑖𝑗 , 𝑇𝑖𝑗 ) (4) where 𝑘 𝑅𝑖𝑗 (𝜙𝑖𝑗 , 𝑇𝑖𝑗 ) = 𝜙𝑖𝑗 𝑇𝑖𝑗∗ − [𝐶𝑖 (𝑃𝑔𝑖 +𝑇𝑖𝑗 ) − 𝐶𝑖 (𝑃𝑔𝑖 )] (5) 𝑘 𝑅𝑖𝑗 is the payoff of the 𝑖th DRA after transmitting 𝑇𝑖𝑗 to the 𝑗th DRA ( and receiving 𝑇𝑖𝑗∗ ) in contract 𝑘. Any difference between 𝑇𝑖𝑗∗ and 𝑇𝑖𝑗 is due to transmission losses, with negligible transmission losses 𝑇𝑖𝑗∗ = 𝑇𝑖𝑗 . 𝜙𝑖𝑗 , is the transaction price and 𝑃𝑔𝑖 is the aggregate of local loads, of the DRA 𝑖, fed prior to the transaction. The first term in (5) is the gross revenue of the aggregator due to the transaction and the second (bracketed) term is the change in the aggregator’s batteries discharging cost due to the transaction. C. Bidding Process From the cost function described above, the marginal cost of the 𝑔th DRA is a linear function of 𝑃𝑔 : 𝜆𝑔 =

𝑑𝐶𝑔 𝑑𝑃𝑔

= 𝑣𝑔 + 2𝑢𝑔 𝑃𝑔

(6)

Since the marginal cost of power, 𝜆𝑔 , is a linear function of stored power, 𝑃𝑔 , we may assume that the bid price, 𝛽𝑔 , is also a linear function of 𝑃𝑔 : 𝛽𝑔 = 𝜆𝑔0 + 𝑚𝑔 𝑃𝑔 (7) where 𝜆𝑔0 is the marginal cost of power at the starting point of the bid curve and 𝑚𝑔 is a measure of 𝑔’s bidding strategy. A market coordinator receives bids from each DRA and pairs buyers with sellers. III.

GAME THEORETIC MARKET FRAMEWORK

We consider two variants of Stackelberg game: without PS DR scheduling of large load devices, and with scheduling for one of the competitors, but not the other. Fig. 1 shows a visual schematic of the proposed market framework. Each DRA manages a number of buildings. Each house contains various devices possibly including WHs and batteries. The DRA’s objective in the game to decide their optimal bidding strategy.

A Dynamic Programming (DP) scheduling algorithm outputs signals to determine when a WH will run to satisfy the consumers demand for hot water. The DP algorithm is also used to output operation times in the case that P-S DR scheduling is considered. The DRAs communicate through a wireless network with a server which arranges transactions while enforcing utility policy constraints. The game without scheduling (not shown) has an identical framework except that electricity price is not an input of the DP scheduler because the demand is inelastic- the same amount of hot water is demanded by the customer at any price.

Fig. 1: Diagram of the market framework with scheduling

The objective of the game is to choose the bidding strategy, 𝑚𝑔 , that maximizes payoff, but each DRA’s payoff is a function of discharging cost, spot market price, and power transacted, and a given DRA has no knowledge of its competitors’ bids, or parameters, 𝐵, 𝑎𝑖 , associated with operating their equipment. To decide which bidding strategy to use, we classify the DRAs into a finite number of types, each with a characteristic cost, which account for the unknown information such as: other aggregators’ pay off functions, grid electricity prices, availability of charged batteries, etc. and use a Bayesian approach to deal with incomplete information where the unknown parameters are represented by a probability distribution, 𝚷, which may be estimated from available information such as electricity prices, demand data, water usage, and prevalence of storage devices. The expected value of the payoffs of the aggregators are maximized over 𝚷 for the random variable “type”. We consider competition between DRAs 𝐴 and 𝐵 to sell power to 𝐶 who is willing to buy. For our purposes, the goal of the game is to calculate the expected payoff, 𝐸𝑃𝐴𝑚 , for different strategies for DRA A of type m (and similarly for the B of type n) to find the Nash equilibrium and to choose the strategy that maximizes the expected value of the payoff, which is just the sum (over N possible types of B) of the conditional probabilities that A is type m and B is type n given that A is of type m, 𝜂𝐴𝑚 (𝑛), multiplied by the conditional payoff, 𝐻𝐴𝑚 : 𝑚 𝑚 𝑚 𝑛 𝐸𝑃𝐴𝑚 = ∑𝑁 (8) 𝑛=1[ 𝜂𝐴 (𝑛). 𝐻𝐴 (𝑠𝐴 , 𝑠𝐵 , 𝑚, 𝑛)] 𝜋𝑚𝑛 𝑚 𝜂𝐴 (𝑛) = ∑𝑁 (9) 𝑛=1 𝜋𝑚𝑛

where and 𝜋𝑚𝑛 is the probability that A is type m and B is type n, and 𝑠𝐴𝑚 is a vector of bidding strategies for 𝐴’s type 𝑚 and 𝑠𝐵𝑛 for 𝐵’s type 𝑛 . The conditional payoff, 𝐻𝐴𝑚 , is a function of 𝑠𝐴𝑚 , 𝑠𝐵𝑛 , as well as 𝑚, and 𝑛.

IV.

DRA DECISION MAKING PROCEDURE

Assuming that DRAs 𝐴, and 𝐵, know 𝐶’s payoff function, we determine Nash equilibrium following the procedure outlined in Fig. 2.

Fig. 2: Procedure for determining optimal bidding strategy

Step1: Define aggregators’ types: The participant’s type is determined largely by electricity price (we use the symbol 𝑒> when the price is greater than the median, and 𝑒< when less) and availability of the batteries for discharging. Availability of stored energy in the batteries may be estimated by considering use patterns for high power consumption devices such as WHs since when consuming power most of the energy stored in the batteries is being consumed locally to power these devices and thus not available for sale (we use the symbols 𝑆𝐼 when WHs are running and 𝑆𝑂 when powered off). Considering these two factors we may classify an aggregator into one of four scenarios, 𝜎: 𝜎1 ↔ {𝑒> , 𝑆𝑂 }; 𝜎2 ↔ {𝑒> , 𝑆𝐼 }; 𝜎3 ↔ {𝑒< , 𝑆𝑂 }; 𝜎4 ↔ {𝑒< , 𝑆𝐼 }. Before considering P-S DR, scheduling depends entirely on water usage and is completely independent of the price of electricity, thus the probability of both events occurring is the product of the probabilities of each event. For example, the probability of the event 𝜎1 is: 𝑃(𝜎1 ) = 𝑃(𝑒> ∩ 𝑆𝑂 ) = 𝑃(𝑒> ) ∗ 𝑃(𝑆𝑂 ) (10) such that ∑4𝑓=1 𝑃(𝜎𝑓 ) = 1. When we consider P-S DR scheduling depends on both water usage and electricity price so that in calculating the probability of the joint events we use conditional probabilities instead, e.g., 𝑃(𝜎1 ) = 𝑃(𝑒> ∩ 𝑆𝑂 ) = 𝑃(𝑒> ) ∗ 𝑃(𝑆𝑂 /𝑒> ) (11) In either case, the probability of the WHs’ operating state, 𝑃(𝑆𝐼/𝑂 ), is output by the DP scheduler. Step2: Define the probability distribution of the game: 𝑓 Let Ψ𝐴 (𝑚) be the probability that aggregator A is of type 𝑓 m in scenario f ; and Ψ𝐵 (𝑛) the probability that aggregator B is in type n. These probabilities depend on the type of market. For example, in the ancillary services market (like reserves), ramp rates of discharging of batteries are really important and aggregators probably choose the more expensive discharging coefficients to discharge the batteries faster. The probability that A is type m and B is type n, 𝜋𝑚𝑛 , may be expressed as: 𝑓 𝑓 𝜋𝑚𝑛 = ∑4𝑓=1(𝑃(𝜎𝑓 )𝜗(𝜎𝑓 )Ψ𝐴 (𝑚)Ψ𝐵 (𝑛)) (12) where 𝜗(𝜎𝑓 ) is a participation coefficient for scenario f. 𝜗(𝜎2 ) = 0 because when the electricity is expensive and the water usage is high, energy stored in electrochemical cells will be used preferably for powering local loads. 𝜗(𝜎𝑓 ) = 1 otherwise: because if the price of electricity is cheap, the

stored energy is available for sale as loads may be supplied directly from the grid at lower cost; whereas even if the price is high, absence of local demand leaves energy available for sale. Given this scheme for predicting 𝜋𝑚𝑛 , any DRA may compute its own discharging costs without knowing the opponent’s discharging cost precisely. Step 3: Define aggregators’ strategies We assume that each aggregator has a vector of three strategies 𝜀𝑠 = [𝜀𝑠1 , 𝜀𝑠2 , 𝜀𝑠3 ]. The slope of the bid curve (7) is 𝑚𝑔 = 𝜀𝑆 × 𝑢𝑔 where 𝑢𝑔 was defined with (2). Step 4: Calculate conditional payoff Next we must solve the system of equations 𝜆𝐴 = 𝜆𝐵 = 𝜙 𝑇 ; 𝑃𝐴 + 𝑃𝐵 = 𝐿𝐶 where 𝑃𝐴 and 𝑃𝐵 are the power remaining in the storage cells of DRA A and B, after feeding their local loads. This power is sold to aggregator C (𝐿𝐶 is the power purchased by aggregator C), and 𝜙 𝑇 is the transaction price. After determining 𝑃𝐴 , 𝑃𝐵 , and 𝜙 𝑇 , the values must be checked against the market controllers constraints and if they are consistent the conditional payoffs of DRA A and B may be calculated with (5). The conditional payoff matrix, 𝐻𝐴𝑚 (𝑛), takes the form: ℎ𝐴,11 ℎ𝐴,12 ℎ𝐴,13 𝑚 (𝑛) 𝐻𝐴 = [ℎ𝐴,21 ℎ𝐴,22 ℎ𝐴,23 ] (13) ℎ𝐴,31 ℎ𝐴,32 ℎ𝐴,33 where each row corresponds to a strategy of A for type m and each column corresponds to a strategy of B for type n, e.g. ℎ𝐴,23 in 𝐻𝐴2 (1) corresponds to A’s type 2 payoff if A decides to bid marginal cost when B is type 1 and bids above. 𝐻𝐵𝑛 (𝑚) is defined analogously. Step 5: Define expected payoff matrices. Next we calculate 𝐸𝑃𝐴𝑚 using (8); it takes the form: 𝐸𝑃𝐴𝑚

1 𝜅11 2 = [𝜅11 3 𝜅11

1 𝜅12 2 𝜅12 3 𝜅12

1 1 𝜅13 𝜅21 2 2 𝜅13 𝜅21 3 3 𝜅13 𝜅21

1 𝜅22 2 𝜅22 3 𝜅22

1 𝜅23 2 ] 𝜅23 3 𝜅23

(14)

Each row corresponds to the strategy of participant A. Each column corresponds to the presumed strategy of participant A’s 3 opponent e.g. the 𝜅21 in 𝐸𝑃𝐴1 is A’s type 1 payoff when A uses strategy 3 and B uses strategy 1 in its type 2. 𝐸𝑃𝐵𝑛 is defined analogously. Step 6: Obtain the Nash equilibrium strategies We may obtain the Nash equilibrium by inspection of the expected payoff matrices. We look for the strategy that produced the highest payoff regardless of the other players choice. The Nash equilibrium is a “consistent” prediction of how the game will be played by rational players; each play may guarantee at least the payoff at equilibrium. All participants predict that a particular Nash equilibrium will occur: there is no incentive to play differently. V.

CASE STUDY RESULTS

A. Game without P-S Scheduling For illustration purposes, consider a system with three DRAs labeled 𝐴, 𝐵, and C; load data for these three was taken from a real grid model on the island of Maui (Hawaii-United

States) [17]. 𝐴 manages 200 houses, all of which contain electrochemical storage cells. 𝐵 manages 240 house, but only 180 have storage devices. 𝐶 manages 240 houses of which only 160 are equipped with storage devices. All houses have WHs that are assumed to draw the majority of the load (60%) [17]. The nominal power capable of being generated by inverters connected to a given storage device 3.3 kW. Thus, the maximum generation capacity of any DRA is 𝑃𝐴𝐺𝑒𝑛 = 𝑔𝑒𝑛 𝑔𝑒𝑛 660𝑘𝑊, 𝑃𝐵 = 594𝑘𝑊, 𝑃𝐶 = 528𝑘𝑊. We assume that the nominal power of each WH is 4.5 kW. Thus, power demands for the DRAs are: 10 𝑔𝑒𝑛 𝑃𝐴𝑑𝑒𝑚𝑎𝑛𝑑 = 200 ∗ 4.5 ∗ + 𝑃𝐴 = 2160𝑘𝑊; 6 𝑃𝐵𝑑𝑒𝑚𝑎𝑛𝑑 = 2394𝑘𝑊; 𝑃𝐶𝑑𝑒𝑚𝑎𝑛𝑑 = 2478𝑘𝑊. Since the marginal cost of 𝐶 is greatest, 𝐴 and 𝐵 will compete to sell power to 𝐶. We assume 𝑀 = 2, 𝑁 = 2, 𝐶 has only a single type, and that the sellers know the buyers cost function, and that the discharging cost coefficients, {[𝑎𝑡1 , 𝑎𝑡2 ], [𝐵𝑡1 , 𝐵𝑡2 ]}, associated with each DRA of each type, 𝑡1 ,𝑡2 , are: 𝐴 ⟷ {[4,4.3], [25,23.5]}; 𝐵 ⟷ {[4.2,4.5], [24,23.5]}; 𝐶 ⟷ {[5], [21.5]}. Fig. 3 and Fig. 4 show the normalized price of electricity and average water consumption from the Maui case study. For our purposes, we take prices over half of the normalized price of electricity to be “expensive”. From the normalized price data (Fig. 3), 𝑃(𝑒> ) = 0.7083, and 𝑃(𝑒< ) = 0.2917. The status of the WHs may be determined from the water consumption data (Fig. 4). In order to keep water temperatures in the range of 110 − 130℉, a typical WH runs during 18 of the 96 fifteen minute intervals of the day. The probabilities 𝑃(𝑆𝐼 ) = 0.1875 and 𝑃(𝑆𝑂 ) = 0.8125 follow immediately from the known ratio of on to off intervals. Fig. 5 is the output of the DP scheduler for the sellers in the game without P-S scheduling. The dark bars represent intervals when the WH is running. Following the example in (10), the probabilities of the four scenarios are: 𝑃(𝜎1 ) = 0.5755, 𝑃(𝜎2 ) = 0.1328, 𝑃(𝜎3 ) = 0.2370, 𝑃(𝜎4 ) = 0.0547. Probability distributions for both DRAs 𝐴 and 𝐵 in the above mentioned scenarios are: Ψ𝐴1 = [0.16,0.84], Ψ𝐵1 = [0.21,0.79], Ψ𝐴2 = [0.11,0.89], Ψ𝐵2 = [0.18,0.82], Ψ𝐴3 = [0.75,0.25],Ψ𝐵3 = [0.67,0.33], Ψ𝐴4 = [0.69,0.31], Ψ𝐴4 = [0.60,0.40]. The probabilities, 𝜋𝑚𝑛 , from (12) are: 𝜋11 = 0.0193, 𝜋12 = 0.0727, 𝜋21 = 0.1015, 𝜋22 = 0.3819. Conditional probability vectors for A and B are calculated using (9): 𝜂1𝐴 = [0.21,0.79], 𝜂1𝐵 = [0.16,0.84], 𝜂𝐴2 = [0.21,0.79], 𝜂𝐵2 = [0.16,0.84]. We assume that each DRA chooses one of the following strategies: bidding 80% of marginal cost (𝜀𝑆1 = 1.6) , bidding marginal cost (𝜀𝑆2 = 2) , and bidding 120% of marginal cost (𝜀𝑆3 = 2.4), the strategies are: 𝑠𝐴1 = [0.00064,0.00080,0.00096], 𝑠𝐴2 = [0.00078,0.00097,0.00117], 𝑠𝐵1 = [0.00073,0.00091,0.00109], 𝑠𝐵2 = [0.00081,0.00102,0.00122].

Fig. 3: Normalized electricity price data for a typical day [17]

Fig. 4: Normalized water consumption data for a typical day [17]

Fig 5. : On (black)/Off (white) status of WHs without P-S scheduling

The expected payoff matrices are calculated as in (14): 9.52 12.28 14.62 41.32 51.93 60.71 𝐸𝑃𝐴1 : [14.89 19.13 21.75 64.38 80.81 81.40] 16.94 20.83 21.42 73.77 79.93 81.41 7.42 10.09 11.87 33.17 43.63 41.24 𝐸𝑃𝐴2 : [11.87 14.25 13.90 52.47 52.91 50.82] 13.23 14.05 14.28 51.76 53.57 53.69 8.16 24.87 37.39 4.06 5.86 7.47 𝐸𝑃𝐵1 : [6.96 9.71 11.23 −3.26 13.25 23.91] 7.76 10.57 11.03 −13.47 1.85 12.96 16.85 25.70 33.83 −9.28 87.52 147.5 𝐸𝑃𝐵2 : [30.21 43.73 43.75 −75.6 12.19 74.09] 33.18 42.42 44.09 −137 −48.8 15.45

Inspecting the data, we see that 𝐴 will choose strategy 3“bidding above marginal cost” (row 3 is highlighted) for either of its types. 𝐵 will also conclude that 𝐴 will bid this way need only consider the best response to this strategy i.e. considering the 3rd and 6th columns of 𝐸𝑃𝐵1 and 𝐸𝑃𝐵2 which represents the payoffs for B’s possible strategies when competing against a player A who bids above marginal cost against any type of opponent: if B is type 1, it will receive the greatest payoff by bidding marginal cost (red number) in type 1 of A and bidding below marginal cost (blue number) in type 2 of A; and if B is type 2, it will receive the greatest payoff by bidding above marginal cost (purple number) in type 1 of A and bidding below marginal cost (green number) in type 2 of A. Player A knows that B will play this way, but has no incentive to change its strategy. Same colors show the corresponding payoffs for player A. The colorful numbers are the payoffs corresponding to the Nash equilibrium bidding strategies (optimal decision). B. Game Including P-S scheduling Now consider the same game with P-S DR WH scheduling implemented by 𝐴. The DP scheduler now considers the electricity price as a factor for 𝐴 but not 𝐵. The output is shown in Fig. 6 (the highlighted sections show 68/96 intervals when

the electricity price is greater than the median). The calculations follow the same general procedure and we use the same data with the single exception that when calculating the probabilities of the scenarios we use conditional probabilities as described in (11). They are: 𝑃(𝜎1 ) = 0.6041, 𝑃(𝜎2 ) = 0.1042, 𝑃(𝜎3 ) = 0.2292, 𝑃(𝜎4 ) = 0.0625.

Fig. 6: On (blue)/Off(white) times of WHs with P-S scheduling

Comparison of Fig. 5 with Fig. 6 shows DR scheduling can curtail load of WHs by 11.2%. A’s 𝑃𝐴𝑑𝑒𝑚𝑎𝑛𝑑 becomes 10 𝑔𝑒𝑛 𝑃𝐴𝑑𝑒𝑚𝑎𝑛𝑑 = 0.888 ∗ 200 ∗ 4.5 ∗ + 𝑃𝐴 = 1992𝑘𝑊, while 6 𝐵’s is unchanged. The expected payoffs are: 10.25 13.07 15.44 44.19 55.01 63.92 𝐸𝑃𝐴1 : [15.65 19.94 23.22 67.35 83.95 87.06] 17.76 22.18 22.81 76.98 85.10 86.73 8.18

10.94 13.33 36.21 46.95 47.21 15.59 15.29 55.67 58.06 56.19] 14.09 15.31 15.58 56.40 58.39 58.69

𝐸𝑃𝐴2 : [12.66

3.89 5.64 7.20 6.12 22.71 35.14 𝐸𝑃𝐵1 : [6.72 9.40 11.24 −5.24 11.53 22.20] 7.49 10.49 10.98 −15.16 −0.18 11.29 16.05 24.64 32.54 −20.6 75.59 137.7 𝐸𝑃𝐵2 : [29.08 42.22 43.93 −86.5 2.65 64.60] 31.94 42.13 43.94 −146.6 −58.2 6.17

The Nash equilibrium strategies for 𝐴 is to bid above marginal cost regardless of its type (3rd row). For 𝐵 in type 1, is to bid marginal cost (red) if A is in type 1 and bid below marginal cost (blue) if A in in type 2. For 𝐵 in type 2, is to bid above marginal cost (purple) if A is in type 1 and below marginal cost if A in in type 2 (green). Same colors show the corresponding payoffs of A. Comparing with the previous section, payoff to 𝐴 has increased through implementation of P-S DR scheduling while the payoff to 𝐵 has decreased. VI.

CONCLUSION

Using a game-theoretic market competition framework we formulated competition between DRAs to sell power stored in batteries under restricted trading conditions as an incompleteinformation Stackelberg game and derived an optimal bidding strategies for rational players involving the Nash equilibrium of the game. We applied our procedure to case study data taken from an aggregator network on the Island of Maui. We then consider the same game again allowing one player to implement P-S DR WH scheduling. Our results demonstrate that implementation of P-S DR WH scheduling increases payoff to the aggregator implementing it while decreasing payoff to the aggregator competing against it. The proposed model in this article is comprehensive enough to use any distributed energy resources instead of batteries. Using various sources in aggregators can be an extension of this work.

VII. ACKNOWLEDGEMENT This research is sponsored by United States National Science Foundation under award number: 1310709. REFERENCES [1] M. Motalleb, M. Thornton, E. Reihani and R. Ghorbani, "A nascent market for contingency reserve services using demand response," Applied Energy, pp. 985-995, 2016. [2] P. Srikantha and D. Kundur, "A Novel Evolutionary Game Theoretic Approach to Real-Time Distributed Demand Response," in Power & Energy Society General Meeting, IEEE, 2015. [3] F. B. Saghezchi, F. B. Saghezchi, A. Nascimento and J. Rodriguez, "Game-Theoretic Based Scheduling for Demand-Side Management in 5G Smart Grids," in IEEE Symposium on Computers and Communication (ISCC), Larnaca, 2015. [4] E. Nekouei, T. Alpcan and D. Chattopadhyay, "A Game-Theoretic Analysis of Demand Response in Electricity Markets," in IEEE PES General Meeting, Conference & Exposition, 2014. [5] M. Ye and G. Hu, "Game Design for Price based Demand Response," in Power & Energy Society General Meeting, IEEE, 2015. [6] M. Motalleb, M. Thornton, E. Reihani and R. Ghorbani, "Providing frequency regulation reserve services using demand response scheduling," Energy Conversion and Management, vol. 124, pp. 439452, 2016. [7] Y. Wang, X. Ai, Z. Tan, L. Yan and S. Liu, "Interactive Dispatch Modes and Bidding Strategy of Multiple Virtual Power Plants Based on Demand Response and Game Theory," IEEE Transaction on Smart Grid, vol. 7, no. 1, Jan 2016. [8] E. Reihani, M. Motalleb, R. Ghorbani and L. Saoud, "Load peak shaving and power smoothing of a distribution grid with high renewable energy penetration," Renewable Energy, vol. 86, pp. 1372-1379, 2016. [9] M. Motalleb, E. Reihani and R. Ghorbani, "Optimal placement and sizing of the storage supporting transmission and distribution networks," Renewable Energy, vol. 94, pp. 651-659, 2016. [10] E. Reihani, M. Motalleb, M. Thornton and R. Ghorbani, "A novel approach using flexible scheduling and aggregation to optimize demand response in the developing interactive grid market architecture," Applied Energy, 183, 445-455., vol. 183, pp. 445-455, 2016. [11] A. Eshraghi, R. Ghorbani, "Islanding detection and over voltage mitigation using controllable loads," Sustainable Energy, Grids and Networks, vol. 6, pp. 125-135, 2016. [12] S. H. H. M. Yu, "A Real-Time Demand-Response Algorithm for Smart Grids: A Stackelberg Game Approach," IEEE Transactions on Smart Grid, vol. 7, no. 2, pp. 879 - 888, March 2016. [13] K. Alshehri, J. Liu, X. Chen, T. Basar, "A Stackelberg Game for MultiPeriod Demand Response Management in the Smart Grid," in IEEE 54th Annual Conference on Decision and Control, Japan, 2015. [14] A. A. G. A. A. Zeinalzadeh, "Distribution-Free Learning in Inventory Competition," in 4th World Congress of the Game Theory Society, Istanbul, Turkey, 2012. [15] H. M. Soliman and A. Leon-Garcia, "Game-Theoretic Demand-Side Management With Storage Devices for the Future Smart Grid," IEEE Transaction on Smart Grid, vol. 5, no. 3, May 2014. [16] M. Shahidehpour, H. Yamin and Zuyi Li, Market Operations in Electric Power Systems: Forecasting, Scheduling, and Risk Management, New York: by John Wiley & Sons, Inc., 2002. [17] GE Energy Consulting, "Hawaii Solar Integration Study: Final Technical Report for Maui," Hawai‘i Natural Energy Institute (HNEI), 2013.