Stochastic equilibrium programming for dynamic ... - Springer Link

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 66, No, 2, AUGUST 1990

Stochastic Equilibrium Programming for Dynamic Oligopolistic Markets t A.

HAURIE,

2

G.

ZACCOUR,

3 AND

Y.

SMEERS 4

Communicated by G. Leitmann

Abstract. This paper deals with the concept of stochastic equilibrium programming (SEP), which has recently been proposed for the modeling of imperfect competition on uncertain dynamic markets. We show that the equilibria computed via SEP correspond to an information structure, called S-adapted open-loop, which is not common in the dynamic game literature. We compare the single-player case with the many-player case using a simple two-stage dynamical system. An illustration of the use of the approach for the modeling of imperfect dynamic markets is also provided. Key Words. Stochastic equilibrium programming, uncertain dynamical systems, imperfect dynamic markets, dynamic oligopolistic markets.

1. Introduction The aim of this paper is to clarify the information structure which is implicit in the stochastic equilibrium-programming approach, an optimization concept to be elucidated in Section 4, for the modeling of dynamic equilibria in a class of multi-agent uncertain systems. Stochastic programming is an active area of research, motivated by the modeling of economic systems in a dynamic and stochastic environment (e.g., energy planning, natural resources exploitation, etc.). The domain has benefited from important theoretical and applied developments (Refs. 1, 2). Often the modeling of economic systems leads to the consideration of a multiplicity of agents or players who are competing on an imperfect This research was supported by FCAR, Qu6bec, Canada, by CRSNG, Canada, by DG XII, European Commission, and by SPPS, Belgium. 2 Professor, Ecole des Hautes Etudes Commerciales, GERAD, Montr6al, Qu6bec, Canada. 3 Professor, Ecole des Hautes Etudes Commerciales, GERAD, Montr6al, Qu6bec, Canada. 4 Professor, Universit~ de Louvain-ta-Neuve,CORE, Louvain, Belgium. 243 0022-3239/90/0800-0243506.00/0

© 1990 Plenum Publishing Corporation

244

JOTA: VOL. 66, NO. 2, AUGUST 1990

market. This is particularly the case when one tries to model the energy markets (e.g., the electricity market in the Northeast American States and Canadian Provinces, the gas market in Europe, the world oil market, etc.). The modeling of such markets may be attempted by extending the stochastic programming approach. One thus replaces the single optimization criterion with a Nash-Cournot equilibrium computation usually performed by using recent advances in the numerical treatment of variational inequalities (Refs. 3-7). We call this approach the stochastic equilibrium-programming approach (SEP). Since this approach deals with a game-theoretic concept (viz., the Nash equilibrium solution) in a dynami~ and stochastic setting, there should be a relationship with the dynamic theory of games (Refs. 8-10). The central concept in dynamic games is the information structure which describes the way the players use the information in their strategies. Usually, one distinguishes at least between the open-loop, feedback, and closed-loop information structures. In the first case, the players only use the information related to the stage (i.e., time) index; in the second case, the players use a Markovian strategy based on the observation of the current state of the system; in the third case, the players use all the available information about past state and action values. Actually, the feedback and closed-loop equilibria are very hard to compute (see Ref. 11 for a recent survey of the equilibrium algorithms in stochastic games). In this paper, we show that the stochastic equilibrium-programming approach deals with a particular class of strategies, which we call S-adapted open-loop strategies. In the case of a single player system, it can be easily shown that, for the class of systems considered, any closed-loop strategy has a representation through an S-adapted open-loop strategy. This means in particular that the stochastic programming approach gives the same solution (in terms of the optimal value for the performance criterion) as the dynamic programming approach. In the case of an m-player system, the stochastic equilibrium-programming approach leads to an interesting solution concept which ties halfway between the completely adaptive closedloop equilibrium solution and the completely nonadaptive open-loop. Although this equilibrium is not subgame perfect ~ la Selten (Ref. 12), it can be a useful representation of the outcome of supply and exchange contracts between energy-producing and energy-consuming countries over a long time horizon and under uncertainty. These contracts often reflect the competition between the possible suppliers and also include provisions under which the contracts will adapt to random modifications of the economic environment. The paper is organized as follows. In Section 2, we present the class of systems considered, in the simplified framework of a two-stage dynamical structure which nevertheless retains the essential ingredients of a dynamic


245

game. In Section 3, we first deal with the single player case and we show that the stochastic p r o g r a m m i n g approach is locally equivalent to the dynamic p r o g r a m m i n g approach. In Section 4, we define the equilibrium concept and we show that, in the class of S-adapted open-loop strategies, the characterization of an equilibrium is obtained through the stochastic equilibrium-programming approach. In Section 5, we show how this approach can be used for the modeling of a dynamic oligopoly model ~ la Cournot, with investment activities and r a n d o m perturbations on the demand laws. A multi-stage model is considered, and the existence of a unique equilibrium solution is guaranteed. In conclusion, we discuss the appropriateness of this solution concept in the energy modeling area.

2. Dynamical System We consider a two-stage dynamical system where M = { 1 ..... , m} is the set of players. In two-stage systems, the feedback and closed-loop information structures coincide. Let U be the joint decision set of the m players at stage 1. This decision set is the Cartesian product of the rn individual decision sets pertaining to each of the m players, respectively; i.e., U = U~ × . . • × Urn. Let L: U-* ~m be the vector reward function at stage 1. Here again, the reward function L = ( L ~ , . . . , Lm) relates to the m rewards accrued in the first stage to the rn players. In the stochastic programming framework, one assumes that a random event with a finite sample set is perturbing the system in its transition from stage 1 to stage 2. Let S = { s ~ , . . . , sn} be the sample set of a r a n d o m event, and let ~r = ( ~ - ~ , . . . , ~r,),

7rk >-0,

~ 7rk = 1, k=l

be the probability distribution over this sample set. At stage 2, there is a new subproblem defined for each sample value Sk:. It will be convenient to represent the random perturbation as an event tree and to associate a node k with each sample value sk. Let V k= V~ x . . . x V~ represent the decision set at stage 2 and node k; and let Qk : U x V k ~ R " be the vector reward function at stage 2 and node k.

Definition 2.1. A closed-loop strategy is a pair v = (u, 7), where u ~ U and 7 = (Y 1. . . . , y n) is a vector of mappings, 3,k: U ~ V k,

s.t. yk(u)cKk(u);

here, for each u ~ U, the constraint set Kk(u) is a given subset of V k.

246


A closed-loop strategy is a way to adapt the decisions taken at stage 2 by the m players to the decisions previously taken at stage 1 and to the sample value observed for the perturbing random event. A closed-loop strategy permits the players to delay their commitment to use a particular control, to after the end of stage 1, i.e., after they know the choice of decisions made at stage 1 and the sample value of the random perturbation affecting the system transition at stage 1. The class of all possible closed-loop strategies is called Y. We will now consider a subclass of Y which is related to the stochastic programming approach.

Definition 2.2. A strategy v = (u, 3') is called an S-adapted open-loop strategy (where S stands for sample), if 3' is such that there exist vk ~ Kk(u) C V k, k = 1 , . . . , n, such that one has 3'k(u')~ v k for any u'~ U for which vk c Kk(u'). In such a strategy, the decisions taken at stage 2 can be adapted to the sample value observed for the random event; however, they cannot be adapted to the first-stage decision u. This corresponds to a situation where the players commit themselves to adapt their decision at stage 2 only to the realization of the random perturbation. The class of S-adapted open-loop strategies is called A. With any strategy (u, 3') in Y, we associated the value

W(u, 3') = (W,(u, 3'),..,, w,,(u, 3'))= L(u)+ E ~-~O~(u, 3'~(u)),

(1)

k=l

which represents the expected reward vector for the m players. Similarly, with any strategy (u, 3') in A, we associate the value

W(u, 3') =(W~(u, 3'),..., w,,(u, 3')) = L(u)+ E ~r~Ok(u, vk).

(2)

k=l

This simple system captures the essential stochastic dynamical structure of the class of games considered, as well as the two-information structures which will be discussed.

3. One-Player Case: Stochastic Programming Approach Consider the single-player case (m = 1). Let FCf~ be a given subset of the class of strategies defined in Section 2, called the set of admissible strategies. The single agent who controls the system seeks a strategy v* ~ F such that, for any other admissible strategy v ~ F,

w(v*) >- w(v). The strategy v* is then said to be optimal in F.

(3)

JOTA: VOL 66, NO. 2, AUGUST 1990

247

The following result gives a representation theorem that establishes a link between the stochastic programming approach and the computation of optimal closed-loop strategies. Proposition 3.1. If a strategy v* is optimal in A with an optimal value W*, then any optimal strategy in Y gives also the same optimal value W*. Proof.

Since A CY, one has

Sup{ W(v):

vc Y} >-Sup{ W(v): v~ A}.

If v* is in 4, it is represented by the vectors u* and v *k, k = 1 , . . . , n, where v*k~ Kk(u*). Therefore, the optimality condition (3) can be rewritten as

W* = W(v*)=max{L(u)+ ~ ~rkQk(u' vk): uc U' vk cKk(u)}

(4)

Assume that the pair v = (u, y) ~ Y satisfies

w(u, ~,)> w*.

(5)

It suffices to define

v k = yk(u)

(6)

to obtain a contradiction between (5) and (6). Hence, the result is obtained. [] This result establishes that, for the single-player case, any optimal pair in the class of closed-loop strategies has a representation in the class of S-adapted open-loop trajectories. The characterization given in (6) of the optimal value gives also the foundations of the stochastic programming approach which has received a considerable attention in the operations research literature (Refs. 1 and 2). For single-player deterministic systems, it is well known that the open-loop and closed-loop approaches a,:e equivalent in terms of the optimal value of the criterion. For the particular class of stochastic systems described in Section 3, Proposition 3.1 shows that there is a similar equivalence between the S-adapted open-loop and closed-loop approaches. The stochastic programming approach permits the solution of large-scale problems which, because of their size, do not admit of analysis through the dynamic programming approach. There is nevertheless an important distinction between closed-loop and S-adapted open-loop optimal strategies, in the sense that the closed-loop strategy describes the optimal play at stage 2 even in the case where the actions taken at stage 1 are not optimal. This property of closed-loop strategies is related to the concept of perfectness discussed in the next section. This is why we could say that the equivalence exhibited by Proposition 3.1 is, in a sense, local.

248


4. M-Player Case: Stochastic Equilibrium-Programming Approach

In this section, we define the Nash equilibrium concept and we compare the equilibria obtained under the two information structures considered, namely, closed-loop vs S-adapted open-loop. Definition 4.1. The strategy v* is a Nash equilibrium in F C Y if the following holds for any j = 1 , . . . , m :

(i)

v* c F,

(7a)

(ii)

Wj(v*) >- Wj(v*(J)),

(7b)

where

(s)

v *(j) = ( v * , . . . , v * , , vj, v T ÷ , , . . . , v*),

for any vi such that v *(j) ~ F. Assume that the pair v* = (u*, y*) is a Nash equilibrium in A. Since v is then an S-adapted open-loop strategy, it has a representation through the vectors u* and v* = (v'k: k = 1,..., n). Then according to Definition 2.2 the following holds for each player j = I , . . . , m:

Wj(v*)

=

max ( Lj(u*(J) + ~ 7rkQ~(u *(j), /

VNk(J)):

k=l

Uj~ Uj, v*k(J)~Kk(u*(J)),k=l,...,n},

(9)

where UzR(J)-- (Ulg¢, "' ., "~-1, blj, U~+I, . . . , urn), @

v*~°') = (v~*k, . . . ,

v*~l,v), vj+l*k,. .. , v'k).

(10a) (10b)

The equilibrium conditions (9), (10) correspond to the so-called stochastic equilibrium programming concept. A solution of these conditions can be obtained, under sufficient regularity conditions, via the solution of an auxiliary mathematical programming problem (e.g., Ref. 13) or via the solution of a variational inequality (Refs. 3-7). Therefore, the equilibria in A are computable even for systems described by many decision variables a n d / o r constraints. O f course, it is well known (see Ref. 10) that, in general, an equilibrium in Y cannot be represented by an equilibrium in A, since, even when the sample space reduces to a singleton (i.e., when the system is deterministic), there is an important distinction between closed-loop and open-loop equilibria. Therefore, we cannot expect a result similar to Proposition 3.1 to hold for the many-player case.


249

The interesting feature of relation (9) is that it gives a formal definition of an equilibrium which has not often been considered in the literature on dynamic game theory, which lies halfway between the open-loop and the closed-loop concepts and which inherits the computational simplicity of the open-loop concept. The stochastic equilibrium programming approach is an adaptation to this context of the algorithms developed by different authors for the solution of static Nash-Cournot games (Refs. 3, 5, and 6). In Section 5, we illustrate this equilibrium concept on a model of imperfect competition. The closed-loop or feedback equilibria (or equilibria in Y) possess the property of being subgame perfect. This means that, even if the players do not play correctly at stage 1, the actions y*(u) at stage 2 are an equilibrium for the subgame defined at stage 2. This property does not hold for the S-adapted open-loop equilibrium. The same phenomenon has been mentioned for the single-player case; however, in the many-player case, the perfectness property of the closed-loop equilibrium implies in general a different criterion value compared with the S-adapted open-loop equilibrium.

5. Two-Stage Model of Imperfect Competition with Uncertainty The example developed in this section is intended to provide an illustration of the type of modeling permitted by the use of S-adapted open-loop strategies. We consider a two-player three-stage game. The players are producers of a homogenous commodity sold on a competitive market. The market is represented by a stochastic inverse demand law

p( t, s') = P( ql( t, s ~) + q2( t, s'), st);

(11)

here, s' is the sample value at period t of a random perturbation of the market at period t; ql(t, s') + qz(t, s') is the total quantity put on the market; and p(t, s') is the clearing market price at period t for the realization s ~ of the random perturbation. The function P ( . , • ) is assumed to be affine, with negative slope w.r.t, its first argument, and the random perturbations are described by an event tree. The players' actions correspond to the quantities they put on the market at each of the three stages, together with their investment decisions to increase their production capacities. Each ptayer j = 1, 2 is described by the following data: (i) the production capacity Kj(t, s'), t = 0, 1, 2; (ii) a production cost function Cj(qj(t, s')), where qj(t, s') denotes the quantity put on the market at period t and for sample value s'; C~(.)

250


is assumed to be strictly convex, increasing, and twice continuously differentiable; (iii) an investment cost function Fj(/~(t, s~)), where /j(t, s t) denotes the physical capacity installed at period t and for sample value s~; we also assume that Fj(-) is strictly convex, increasing, and twice continuously differentiable. Let S' denote the set of possible realizations of the random perturbation at period t; let a(s ~) ~ S ~-1 denote the unique predecessor of s' c S', t = 1, 2; and let B ( s ' ) C S '+~, t = 0, t, denote the set of successors of s' on the event tree. Let O(s']a(s'))>-0 be the conditional probability associated with the arc (a(s'), s') in the event tree, with

2

O(s'+'l s') = 1.

s'+l~B(s ')

The set S Oreduces to the singleton s o called the root of the event tree. One assumes that player j strives to maximize a discounted stream of profits, with a discount factor/3j, over the 3-period time horizon. Player j thus considers the following payoff:

Jj = qj(O, s°)P(q,(O, s °) + q2(O, s°), s °) - C)(qi(0, s°)) - Fj (/j (0, s°)) +fl~ 20(S~tsO){(qj(1, s~)P(q,(1, S~k)+q2(1, S~),S lk) s~.~S ~

- ~(qj(1, s ~ ) ) - Fj(/j(1, s~))+fij ~ ( ~ b

O(s21s~)

2 q~(2, s~), s~)- Cj(qj(2, s~))] } , x [(q~(2, s~)P(q,(2, s~)+

(12)

which is strictly concave w.r.t, the decision variables. The optimization of (12) is performed subject to the following constraints: expansion of production capacity constraint,

Kj( t, s') = Kj( t - 1, a( s') ) + Ij( t, a(s'));

(13)

capacity constraint,

qj( t, s') -O,

s'~S',

t = 0 , 1.

(16)

JOTA: VOL. 66, NO. 2, A U G U S T 1990

251

Proposition 5.1. There exists a unique S-adapted open-loop equilibrium for the market game defined by (11)-(16). Proof. The S-adapted open-loop equilibrium satisfies (9). In the context of the market game (11)-(16), this condition admits a unique solution according to Theorems 7.1 and 7.7 in Friedman (Ref. 14). [] As a numerical illustration, consider the case where the cost functions are defined as follows: Cl(ql )

=3q~,

C2(q2) = 2q 2, FI(I,) = 8I~, F2( h ) = 7 I~. Figure 1 gives a representation of the demand laws at each node of the event tree. Tables 1 and 2 give the results of the computation of S-adapted stochastic equilibria in the cases where /3j is equal to 1/1.1 and 1/1.6, respectively. One should notice the contingency in production and investment embedded in this equilibrium solution.

.6o

.15 P

= s o - 2Q ]

.45 P

P

100

= 11o- 3Q ]l

2Q .2O

t=o

P = 150 - 3Q [ l

?=.o_sQ t=l

.20 P = 120 - 5Q [

t=2 Fig. l.

Representation of the d e m a n d laws at each node of the event tree. N u m b e r above boxes denote probability.

252

JOTA: VOL. 66, NO. 2, A U G U S T 1990

T a b l e 1.

N u m e r i c a l r e s u l t s f o r / 3 = 1/1.1.

Node

(0, s °)

(1, s 1)

(1, s 2 )

(2, S l)

(2, S2)

(2, s3)

(2, s 4)

p ql q2 K~ K2 Il 12 ~'l 7r2

82.000 5.000 4.000 5.000 4.000 0.808 1.945 329.777 269.519

84.741 5.808 5.945 5.808 5.945 0.035 0.578 355.425 391.601

85.085 5.038 5.945 5.808 5.945 0.787 1.249 315.963 385.658

56.192 5.381 6.523 5.843 6.523

72.902 5.843 6.523 5.843 6.523

108.633 6.595 7.194 6.595 7.194

71.720 3.914 5.742 6.595 7.194

178.102 232.596

267.392 322.678

484.258 560.060

194.011 285.847

Profits ~'1, ~'2 are in present values.

6. Conclusions The optimal control of stochastic systems is usually performed via the dynamic programming approach (see, e.g., Refs. 8-11). The stochastic programming approach gives an alternative way for the computation of a representation of the optimal policy, in the case where the decision maker actions do not influence the discrete probability distribution of the stochastic perturbations. This property does not translate to the case of Nash equilibria in many-player systems. We have shown that the stochastic equilibriumprogramming approach corresponds to an information structure which is different from the feedback or closed-loop ones. The example sketched in Section 5 shows the usefulness of this concept for the modeling of dynamic Table 2. Numericalresults for/3 =

1/1.6.

Node

(0, s °)

(1, s l )

(1, s 2)

(2, s 1)

(2, s 2 )

(2, s 3 )

(2, s 4)

p ql q2 K1 K2 11 I2 7rI 77"2

82.000 5.000 4.000 5.000 4.000 0.658 1.486 331.536 280.543

85.568 5.658 5.486 5.658 5.486 0.119 0.584 242.495 254.279

85.995 5.315 5.486 5.658 5.486 0.673 1.083 230.433 252.521

56.882 5.489 6.070 5.777 6.070

74.459 5.777 6.070 5.777 6.070

111.435 6.331 6.524 6.331 6.524

71.760 4.101 5.547 6.331 6.524

86.655 106.087

128.917 147.764

228.613 250.733

95.247 131.451

Profits ~'1, ~'2 are in present values.


253

stochastic markets. In Ref. 15, a detailed example dealing with the modeling of the competition in the European gas market is fully developed.

References 1. WETS, R. B., Stochastic Programming: Solution Techniques and Approximation Schemes, Mathematical Programming, The State of the Art, Edited by A. Bachen, M. Groetschel, and B. Korte, Springer-Verlag, Berlin, Germany, 1983. 2. ERMOLIEV, Y., and WETS, R. B., Numerical Techniques for Stochastic Optimization Problems, International Institute for Applied System Analysis, Laxemburg, Austria, 1985. 3. COHEN, G., and CHAPLAIS, F., Algorithmes Numdriques pour les Equilibres de Nash, RAIRO, Recherche Op6rationnelle, Vol. 20, pp. 273-293, 1986. 4. DAFERMOS, S. C., An Iterative Scheme for VariationaI Inequalities, Mathematical Programming, Vol. 29, pp. 40-47, 1983. 5. HARKER, P. T., A Variational Inequality Approach for the Determination of Oligopolistic Market Equilibrium, Mathematical Programming, Vol. 30, pp. 105111, 1984. 6. MARCOTTE, P., Quelques Notes et R~sultats Nouveaux sur le Probl~me d'Equilibre d'un OligopoIe, RAIRO, Recherche Op6rationnelle, Vol. 18, pp. 147171, 1984. 7. PANG, J. S., and CHAN, D., Iterative Methods for Variational and Complementarity Problems, Mathematical Programming, Vol. 24, pp. 284-313, 1982. 8. ISAACS, R., Differential Games, Wiley, New York, New York, 1965. 9. LEITMANN, G., Cooperative and Noncooperative Many-Player Differential Games, Springer-Verlag, New York, New York, 1974. 10. BASAR, T., and OLSDER, G. J., Dynamic Noncooperative Games, Academic Press, New York, New York, 1982. 11. BRETON, M., FILAR, J. A., HAURIE, A., and SCHULTZ, T. A., On the Computation of Equilibria in Discounted Stochastic Dynamic Games, Dynamic Games and Applications in Economics, Edited by T. Basar, Springer-Verlag, Berlin, Germany, pp. 64-87, 1986. 12. SELTEN, R., Reexamination of the Perfectness Concept for Equilibrium Points in Extensive Games, International Journal of Game Theory, Vol. 4, pp. 25-55, 1975. 13. MURPHY, F. H., SHERALI, H. D., and SOYSTER, A. L., A Mathematical Programming Approach for Determining Oligopolistic Market Equilibria, Mathematical Programming, Vol. 24, pp. 92-106, 1982. 14. FRIEDMAN, J. W., Oligopoly and the Theory of Games, North-Holland, Amsterdam, Holland, 1977. 15. HAURIE, A., LEGRAND, J. SMEERS, Y., and ZACCOUR, G., A Dynamic Stochastic Nash-Cournot Model for the European Gas Market (to appear).