A Game-Theoretic Demand Response Market with Networked Competition Model Mahdi Motalleb , Alireza Eshraghi , Ehsan Reihani , Hossein Sangrody , Reza Ghorbani University of Hawaii at Manoa, Emails: {motalleb, rezag}@hawaii.edu University of California, San Diego, Email:
[email protected] California State University, Bakersfield, Email:
[email protected] Binghamton University, New York, Email:
[email protected]
Abstract—In classical competition models, each firm decides on the amount of production to maximize its own profit and only one market is shared among all firms. In the case with more than one market which has been proposed in recent years, Networked Cournot Competition (NCC) models the relation between firms and markets. This paper describes a model of competition between demand response aggregators (as firms) which sell aggregated energy stored in residential storage devices (as a homogeneous good) in a networked environment with a market maker. This game is called Networked Stackelberg Competition (NSC). In this paper, for each firm, the optimal bidding plan and Nash equilibrium are obtained in an incomplete information game. Demand response aggregators submit their bids and the market maker (system operator) controls the transaction powers and price subject to grid’s constraints and policies. Also, the effect of pricebased demand scheduling has been studied on the firms’ payoffs in a real world case study. Keywords—Networked Cournot Competition (NCC), Networked Stackelberg Competition (NSC), Demand Response Aggregator (DRA), Game theory, Optimal bidding plan, Nash equilibrium
I.
INTRODUCTION
As more distributed generations including renewable resources are introduced into the power grid infrastructure, understanding how these new resources make changes into the grid becomes more important [1, 2, 3]. A lack of adequate network protocols and regulations can cause network instabilities such as islanding, transient over voltage, lightning, and frequency change [4, 5, 6]. In order to solve these issues, maximizing the flexibility of the power system is desirable from system designer viewpoint. Demand Response (DR) program is an applicable method for increasing the flexibility into the modern grid [7, 8, 9, 10, 11]. One type of DR resources is residential storage device which can participate in a DR market for providing different types of ancillary services [12, 13] especially in a grid with higher penetration level of uncertain renewable energies [14, 15, 16, 17].
In terms of the number of markets, DR markets can be classified in two models: the first one where the competition between firms is among one market [18, 19] while in the second model firms compete to each other across several different markets. The first model covers the majority of the proposed DR market models in literature. But the concepts of the second model have been proposed in recent years which is called networked competition models. Considering this type of DR model in power system from one side and different forms of cost and price functions from another side increase the complexity of computing the Nash equilibrium for this type of markets. [20] proposes an algorithm for finding pure Nash equilibrium of Network Cournot Competition (NCC). [21, 22] present possible methods used by market maker in order to balance the supply and demand in NCC. Social welfare and consumer surplus are different objective functions for the market maker to optimize. [23] provides a characterization of the unique equilibrium with highlighting the production quantities and supply paths relationship in the underlying networked structure. It concludes that it is not right to generalize the result from analyzing a single market competition to multiple markets one. [24] studies the impact of strategic anticipated behaviors in networked markets. It compares the efficiency of a networked Stackelberg equilibrium with a network Cournot one. This work is based on Networked Stackelberg Competition (NSC) with two main objectives: the first one is to propose a market framework based on Stackelberg model for competition between Demand Response Aggregators (DRAs) to sell aggregated energy stored in residential storage devices using networked competition models instead of classical ones. This type of models have not been studied in literature for DRAs. The second objective is to model the participants of DRAs in an incomplete information game (i-game). Unlike complete information game (c-game) which all players have information about other players in terms of cost function, bidding plans, etc., in an i-game each player does not have any information about other players. While most of the related works in literature
consider c-game, this assumption does not perfectly reflect the situation in the real market. In addition, studying the effects of scheduling DR resources on players’ payoffs and providing contingency reserve services for frequency regulation in abnormal situations are other contributions of this paper. II.
NETWORKED COMPETITION
III.
GAME PLAYERS
DRAs and market maker are participants of electricity market presented in this paper. In each node (k) of the network, there is a DRA that is shown with DRAk which has two parts: Aggregated charged batteries ( DRAkB ) and aggregated load of
Fig. 1 shows the diagram for networked Cournot competition where there is more than one markets. F = { f1 ,..., f n } and
end-users ( DRAkL ).
M = {m1 ,..., mm } show the set of firms and markets,
DRAkB is the power source of each DRAk and it has the
respectively. Firms produce commodity and compete for the markets. G = ( F ∪ M , E ) is the graph where E shows the set
responsibility of selling the power in the game.
of edges from the firms set (F) to the markets (M). the quantity that firms i supplies to the market
qik shows
mk .
cost function ( ck (.) ) in eq. 4.
Markets (M) 1
.
qik
DRAkL shows the aggregated load of end-users which is the
.
consumer part of DRAk . It consists of the aggregated loads in node k and its role in the game is to buy the power as a consumer (market). This load is a fraction of total load of the node which can be fed either with generation companies or local batteries. The model in this paper considers this load to be equal or smaller than available power of batteries in each DRA k . A
k
.
.
n
m
Cost: ci(.) Fig. 1. Networked Cournot competition model (with more than one market)
The goal for each firm is to maximize its profit which can be defined as follows: qik ≥ 0
mk ∈Fi
pk which are the
quantity offer and price, respectively. It insures the aggregator’s
1
max
available power capacity of this aggregator. As mentioned in the introduction, this paper considers an i-game where DRAs do not have any information about others. After processing the game, each DRAkB submits qkm ≥ 0 and
Firms (F)
i
Bk shows the
qik . pk q jk − ci qij j∈M k j∈Fi
(1)
qik . pk q jk is the revenue of firm i and j∈M k ci qij calculates its production cost. The profit of firm i j∈Fi is separable across the market if ci is linear. On the other hand
Where
a unique Nash equilibrium exists for this optimization problem when: 1) pk is twice differentiable, concave, and strictly decreasing and 2) ci is twice differentiable, convex, and increasing [23, 25]. Based on [20], it is not too hard to compute this optimization problem and there are different situations with convex cost function which one of them is NCC with linear inverse demand functions. This situation forms an ordinal potential game.
widely used linear demand function in litarture ( ak − bk .d k ) has been used in this work.
dk is a part of total demand in node
k which is not committed to be fed by generation companies and also d k ≤ qk . Market maker (M) is the other participant of the market model in this paper. Market maker is an Independent System Operator (ISO) that chooses r := (r1 ,..., rn ) in order to rebalance element of this vector shows the net power quantities. The injection into node k. Decisions for rebalancing quantities are made by market maker based on the market mechanism that assigns demands and prices in order to balance the supply and demand through feasible flows over the network. 1Τ r = 0 where 1 is a vector of ones with dimension n and n is the number of nodes. This is because the market maker neither consumes nor produces power in the network and only transports the power among DRAs [20, 22]. Fig. 2 demonstrates the proposed model of this paper. In this model, it has been considered that each DRA sells the excess stored energy after feeding its local load and hence there is no power transmission between each firm and its own market in the figure. This means that local loads are the first priority of each aggregator and they have to be fed first. After feeding the
local loads, the extra power, if there is any, can be transferred to the other nodes (markets). As fig. 2 shows, all firms and markets are equipped with communication devices with the goal of data transmission with market maker. This data includes bids, offers, power quantities, price, available charged batteries, demands, and etc. Different communication methods and standards such as ZigBee, RFID, and power line can be used based on available infrastructure, size of the market, communication channel condition, and etc. [26, 27].
Ci is the cost function of the
th
aggregator when selling the
stored power of Pi (kW). Since the load data is available for every 15 minutes in this study, the stored energy in eq. 3 is converted to stored power in eq. 4 using the number of 0.25 hour. For the coefficients of eq. 4 we have:
0.25ai a (0.25)2 , ui = i 2 2b b N 1 ai = ah , Nh is the number of houses in the N h h=1 vi =
h
(5) th
aggregator.
The payoff function for a participant DRA in the market that should be maximized is:
(
,
)
(6)
,
is the payoff of DRAiB after transmitting power of L j
DRA with the transaction price of
to
. It can be defined as the
is the aggregate of local loads in following equation where the th aggregator that has been fed prior to the transaction:
,
=
.
−
+
−
( )
(7)
Fig. 2. Proposed NSC model including DRAs and market maker
IV.
COST FUNCTION AND BIDDING
The cost of discharging of the batteries is the cost function of the aggregators in each node since energy stored in batteries is the only energy resources of DRAs in this model. Eq. 2 is the logarithmic discharging cost function for the batteries as a function of stored energy [7, 28]:
Chbat (ΔEh ) = −ah log(1 −
ΔEh ) b
(2)
bat
In this equation C h is discharging cost of the battery in house h and ΔEh (KWh) is the stored energy for the same house. ah and b are pricing coefficients and depend on the grid’ electricity price for charging the battery and capital/maintenance costs of the battery. Eq. 2 can be expanded using Taylor series as follows:
C hbat (ΔEh ) ≈
a h ΔE h a h ΔE h + b 2b 2
2
The marginal cost of the using eq. 4:
=
=
th
aggregator ( ) can be derived
+2
(8)
As eq. 8 shows, marginal cost is a linear function of stored power ( ). Therefore, it can be assumed that the bidding price ( ) is a linear function of stored power. Eq. 9 shows this linear relationship. In this equation, shows the marginal cost at the starting point of bidding curve where is equal to zero. Also is the slope of the bidding curve for the th aggregator. This slope will be used for bidding plan of players in section V.
=
+
(9)
2
(3)
Eq. 4 shows the quadratic cost function for each aggregator. This conclusion comes from this fact that since each individual house has a quadratic cost function, aggregation of them also produces a quadratic one:
Ci (Pi ) = vi Pi + ui Pi
In this equation the term of ( . ) is the revenue of the aggregator through power transaction. On the other hand, + − ( ) calculates the change in the aggregator’s cost before and after transaction.
(4)
V.
PROPOSED GAME MODEL
Fig. 3 shows the visual schematic of the proposed networked market framework where each aggregator manages a predefined number of houses. The goal is to maximize each aggregator’s payoff in this i-game through finding the optimal bidding plan (slope of in eq. 9) for each DRA. Fig. 3 shows that all DRAs are connected to the market maker using a communication network. As mentioned before, this market maker controls the power transactions in the grid (considering
the constraints of the connecting lines) and transaction price (based on the policies dictated by the utility). DP scheduler in fig. 3 shows price-based demand scheduling based on electricity price and hot water consumption for the WHs as the majority of the load. Dynamic programming has been used for this goal [29].
(10) = ∑ [ ( ) ( , , , )] represents the conditional payoff of aggregator A and depends on the bidding plans of aggregator A ( ) and ( ) is the aggregator B ( ). The conditional probability probability that aggregator A is in type m for aggregator B of shows the type n and is defined using eq. 11 where probability that A and B are in type m and n, respectively:
( ) = ∑
,
∈
,
∈
(11)
Fig. 4 depicts the steps of the proposed decision making process to find the optimal bidding plan and Nash equilibrium.
A network with three firms and three markets is considered named A, B, and C. In this case, A and B compete with each other to sell power to C and they know the buyer’s payoff function. The methodology and results are extendable to any number of firms and markets. The goal of the game is to calculate the expected payoffs for different bidding plans of each firm and then choosing the optimal bidding plan that maximizes the expected payoffs. Finally, Nash equilibrium will be obtained. Eq. 10 calculates the expected payoff of firm A in its type m. Types of the players are different possible sets of coefficients in discharging cost function (different possible sets of [ , ] in eq. 4 and 5). Different factors involve in determining the player’s type including electricity price and availability of batteries for discharging. Availability of charged batteries is estimated using the power consumption pattern focusing on high-power consumption devices such as WHs. The reason is that when these devices consume power, most of the energy stored in the batteries has to be consumed for them and thus there is less available power to sell in the market.
Step 6
Optimal bidding plan
Firms’ expected payoffs
Conditional payoff
Firms’ types
For finding the optimal bidding plan, other aggregators must be classified first into a limited number of types based on a characteristic cost which account unknown information including other aggregators’ pay off functions and availability of charged batteries. In this paper a Bayesian approach has been used for dealing with incomplete information. A probability distribution ( ) represents the unknown parameters. Available information such as electricity prices and historical data of demand and hot water usage may be used in order to estimate . In the case study of this paper, the majority of the loads are thermostatic storages such as Water Hearts (WHs) and Air Conditioners (ACs). Therefore historical data of hot water usage can make a good estimation about the opponent players’ demand and other information. This model maximizes the expected value of the payoffs of the aggregators over .
Step 5 Step 3 Step 4
Firms’ bidding plan
Step 1
Fig. 3: Proposed NSC framework for competition between DRAs
Probability distribution
Step 2
Fig. 4: Steps of determining optimal bidding plan in the game
= × is the slope of the bidding curve in eq. 9 as bidding plan. In this study, we assumed that each firm has three different bidding plans = [ , , ]. Bidding less than < 2), bidding equal to the marginal cost marginal cost ( ( = 2), and bidding more than marginal cost ( > 2) are three possible plans for each firm. Using eq. 10, the final form of expected payoff matrix of firm A in type m is can be defined as follows. is defined similarly. =
(12)
Rows in this matrix show the bidding plan corresponding to type m for firm A. The columns, on the other hand, correspond to the presumed bidding plan of the firm’s opponent. As an in is A’s type 2 payoff when A uses bidding example, plan 3 and B uses bidding plan 2 in its type 1. Optimal bidding , ). The plans are obtained based on these matrices ( goal of each firm is to choose the best bidding plan to make the highest payoff regardless of the other firms’ choices. In this study, the price intercepts of the inverse demand function is spatially homogeneous. Hence based on [22], any equilibrium of Stackelberg game is also an equilibrium of a classical nonnetworked Cournot game which has an inverse demand function of aggregated demand. Accordingly, the existence results of classical Cournot game can be used for the case study of this work. [24, 30, 31] provide the proof for the existence of
Nash equilibrium in a market with firm’s cost function
ck and
inverse demand function in classical non-networked game. CASE STUDY AND RESULTS VI. Fig. 5 shows the schematic of the case study. A real world network with three aggregators including , , and ( , , as firms and , , as markets) has been considered. Load data has been taken from real grid model (Island of Maui in Hawaii-United States). Aggregator A includes 200 houses and all of them equipped with batteries. B and C manage 240 and 260 houses, respectively. While in B 180 houses have storage devices and in C 160 houses. Also, all houses in this system have their own WHs that are assumed to draw the majority of the load around 60% [32]. In the case without price-based WHs scheduling, a typical WH runs during 18 of the 96 fifteen minute intervals of the day in order to keep water temperatures in the range of 110 − 130℉. WHs scheduling curtails WHs load by 11.2% (18 time intervals have been decreased to 16 after scheduling).
Fig. 5. Networked graph of the Maui grid including firms and markets
The nominal power of each residential battery and WH are 3.3 and 4.5 kW, respectively. and compete with each other to sell the power to since the marginal cost of is greater than the other two. {[ , ], [ , ]} shows the set of discharging cost coefficients (firms’ types). This set represents coefficients a and b in eq. 5 for both types. For A, B and C we have:
: {[4 , 25], [4.3 , 23.5]},
: {[4.2 , 24], [4.5 , 23.5]};
: {[5], [21.5]}. After calculating the expected payoffs for different types and bidding plans, transaction prices are in the range of $0.54 and $0.74. This study sets the maximum permissible price at $0.65 which means that the prices more than this value will be set into $0.65. This is because of market maker (M) which makes a sequential-move in the NSC model based on enviable constrains before all firms participated in a simultaneous game
and made the expected payoffs. After affecting market maker’s role, the updated expected payoffs are: 3.03 : 5.66 .
4.32 7.57 .
5.53 15.48 9.35 27.46 . .
1.10 : 2.82 .
1.99 4.17 .
2.88 8.10 4.16 17.67 . .
3.60 5.16 : 5.96 8.15 6.68 9.24 8.54 13.43 : 15.62 22.49 17.24 24.99
.
20.87 35.40 . 12.77 19.57 .
25.79 42.21 . 13.55 18.53 .
18.66 32.25 8.61 22.81 −0.55 12.40
. . . . .
35.92 98.61 −10.58 48.56 −54.20 3.55
. . . . .
.
Inspecting the data, we see that firm will chose its 3rd bidding plan- “bidding above marginal cost” (row 3) for either of its types which these numbers have been bolded. will also conclude that will bid this way needs only consider the best response to this bidding plan i.e. considering the 3rd and 6th columns of and (bolded) which represents the payoffs for B’s bidding plans when competing against a player A who bids above marginal cost for any type of opponent: if B is type 1, it will receive the greatest payoff by bidding above marginal cost (orange number) in type 1 of A and bidding below marginal cost (blue number) in type 2 of A; and if B is type 2, it will receive the greatest payoff by bidding above marginal cost (red number) in type 1 of A and bidding below marginal cost (green number) in type 2 of A. Player A knows that B will play this way, but has no incentive to change its bidding plan. Same colors show the corresponding payoffs for player A. The colorful numbers are the payoffs corresponding to the optimal bidding plans of the players. These optimal bidding plans lead to the maximum expected payoffs in all firms shown in fig. 5 in an i-game.
VII.
CONCLUSIONS
In classical competition models, only one market is shared among all firms and each firm tries to maximize its own payoff by deciding on the amount of production. In this research paper, networked competition models were described in demand response market where different demand response aggregators (as firms) compete across a variety of markets simultaneously. The commodity of this market is aggregated energy stored in residential storage devices. A game with incomplete information was proposed to model a NSC (with a market maker) between the aggregators. In addition, price-based scheduling of demand was considered in this market model. The optimal bidding plan and Nash equilibrium were obtained in this i-game. As a real world case study, the proposed networked competition model was examined on Maui’ power grid located in Hawaii, United States.
ACKNOWLEDGEMENT This project is sponsored by the United States National Science Foundation (NSF) under award number: 1310709. REFERENCES
[15] M. Motalleb, E. Reihani, R. Ghorbani, "Optimal placement and sizing of the storage supporting transmission and distribution networks," Renewable Energy, vol. 94, pp. 651-659, 2016. [16] M. R. Khalghani, S. Khushalani-Solanki, J. Solanki, "Optimal integration and location of PHEV aggregators in power distribution systems," in North American Power Symposium (NAPS), Denver, CO USA, 2016. [17] H. Sangrody, M. Sarailoo, N. Zhou, N. Tran, M. Motalleb, E. Foruzan, "Weather Forecasting Error in Solar Energy Forecasting," IET Renewable Power Generation, 2017. [18] J. Lin, L. Gan, E. Foruzan, "Towards a voltage regulation market," in North American Power Symposium (NAPS), Denver, CO USA, 2016.
[1] H. Khazaei, Y. Zhao, "Ex-post Stable and Fair Payoff Allocation for Renewable Energy Aggregation," in Innovative Smart Grid Technologies (ISGT ), Washington D.C., 2017. [2] S. Akhlaghi, H. Sangrody, M. Sarailoo, M. Rezaeiahari, "Efficient Operation of Residential Solar Panels with Determination of the Optimal Tilt Angle and Optimal Intervals Based on Forecasting Model," IET Renewable Power Generation, 2017. [3] M. R. Khalghani, M. Ramezani, M. Rajabi-Mashhadi, "Demonstrating the Importance of Applying a New Probabilistic Power Flow Strategy to Evaluate Power Systems with High Penetration of Wind Farms," Journal of Energy Engineering, vol. 142, no. 4, p. 04016002, 2016. [4] A. Eshraghi, R. Ghorbani, "Islanding detection and transient overvoltage mitigation using wireless sensor network," in Power & EnergySociety General Meeting, Denver, 2015. [5] A. Eshraghi, R. Ghorbani, "Islanding detection and over voltage mitigation using controllable loads," Sustainable Energy, Grids and Networks, vol. 6, pp. 125-135, 2016. [6] A. D. Banadaki, F. D. Mohammadi, A. Feliachi, "State Space Modeling of Inverter Based Microgrids Considering Distributed Secondary Voltage Control," in North American Power Symposium (NAPS), WV, USA, 2017. [7] Reihani, E., Motalleb, M., Thornton, M., Ghorbani, R., "A novel approach using flexible scheduling and aggregation to optimize demand response in the developing interactive grid market architecture," Applied Energy, vol. 183, pp. 445-455, 2016. [8] M. Motalleb, M. Thornton, E. Reihani, R. Ghorbani, "A nascent market for contingency reserve services using demand response," Applied Energy, vol. 179, pp. 985-995, 2016. [9] M. Sabounchi, H. Khazaei, S. K. H. Sani, "A new residential demand response management method based on a social welfare optimization framework," in 2nd International Conference on Electrical, Electronics and System Engineering (ICEESE), Kuala Lumpur, 2014. [10] J. Solanki, N. Venkatesan, S. K. Solanki, "Market optimization for microgrid with demand response model," in North American Power Symposium (NAPS), Boston, MA, 2011. [11] R. Fazal, J. Solanki, S. K. Solanki, "Demand response using multi-agent system," in North American Power Symposium (NAPS), Champaign, IL, 2012. [12] M. Motalleb, R. Ghorbani, "Non-cooperative game-theoretic model of demand response aggregator competition for selling stored energy in storage devices," Applied Energy, vol. 202, pp. 581-596, 2017. [13] A. Shahsavari, A. Sadeghi-Mobarakeh, E. Stewart and H. MohsenianRad, "Distribution grid reliability analysis considering regulation down load resources via micro-PMU data," in IEEE International Conference on Smart Grid Communications (SmartGridComm), Sydney, NSW, 2016. [14] E. Reihani, M. Motalleb, R. Ghorbani, L. S. Saoud, "Load peak shaving and power smoothing of a distribution grid with high renewable energy penetration," Renewable Energy, vol. 86, pp. 1372-1379, 2016.
[19] E. Foruzan, S. D. Scott, J. Lin, "A comparative study of different machine learning methods for electricity prices forecasting of an electricity market," in North American Power Symposium (NAPS), Charlotte, NC, USA, 2015. [20] Abolhassani, M., Bateni, M. H., Hajiaghayi, M. T., Mahini, H., Sawant, A., "Network Cournot Competition," arXiv:1405.1794. [21] Bose, S., Cai, D. W. H. , Low, S., Wierman, A., "The Role of a Market Maker in Networked Cournot Competition," in 53rd IEEE Conference on Decision and Control, California, USA, 2014. [22] Cai, D., Bose, S., Wierman, A., "On the role of a market maker in networked Cournot competition," Preprint available at http://boses. ece.illinois.edu/files/NwCournot.pdf, 2016. [23] Bimpikis, K., Ehsani, S., Ilkilic, R., "Cournot competition in networked markets," EC , p. 733, 2014. [24] Xu, Y., Cai, D., Bose, S., Wierman, A., "On the Efficiency of Networked Stackelberg Competition," preprint available at http://boses.ece.illinois .edu/files/stackelberg.pdf, 2016. [25] Wierman, A., Bose, S., Cai, D., Chen, N., Ruhi, N., Xu, Y., "Networked Markets: Market makers & market power," [Online]. Available: http://helper.ipam.ucla.edu/publications/gss2015/gss2015_12619.pdf. [26] A. Eshraghi, B. Maham, Z. Han, M. Banagar, "Efficiency and coverage improvement of active RFID two-hop relay systems," in IEEE Wireless Communications and Networking Conference (WCNC14), Istanbul, Turkey, 2014. [27] R. Belkacemi, A. Babalola, F. Ariyo, A. Feliachi, "Restoration of smart grid distribution system using two-way communication capability," in North American Power Symposium (NAPS), Manhattan, KS, 2013. [28] Soliman, H. M., Leon-Garcia, A., "Game-theoretic demand-side management with storage devices for the future smart grid," IEEE Transactions on Smart Grid, vol. 5, no. 3, pp. 1475-1485, 2014. [29] M. Motalleb, M. Thornton, E. Reihani, R. Ghorbani, "Providing frequency regulation reserve services using demand response scheduling," Energy Conversion and Management, vol. 124, pp. 439452, 2016. [30] Mas-Colell, A., Whinston, M. D., Green, J. R. , Microeconomic theory, New York: Oxford university press, 1995. [31] Novshek, W., "On the existence of cournot equilibrium," Review of Economic Study, vol. 52, no. 1, pp. 85-98, 1985. [32] GE Energy Consulting, "Hawaii Solar Integration Study: Final Technical Report for Maui," Hawai‘i Natural Energy Institute (HNEI), University of Hawai‘i, 2013.
