Energy Modelling Research Group Department of Management University of Canterbury
A Constructive Dual DP for a Reservoir Model with Correlation
EMRG-WP-99-01
Miao Yang and E. Grant Read Department of Management University of Canterbury
email:
[email protected]
A Constructive Dual DP for a Reservoir Model with Correlation
Abstract The reservoir management problem for a hydrothermal power system is well suited to modelling via Dynamic Programming. In this paper we describe a dual approach which we term “Constructive Dynamic Programming” (CDP) which has been successfully applied to optimise releases in a stochastic two reservoir model of the New Zealand power system. That model ignores serial correlations of inflows, though, and hence assumes that current inflow observations do not have any impact on future release decisions. These show, however, that better decision rules can be produced by accounting for inflow correlation. Hence we have developed an extension to the standard CDP to explicitly deal with serial correlation of reservoir inflows, and we report on those extensions also.
Energy Modelling Research Group
i
Contents 1
INTRODUCTION................................................................................................................................. 1
2
DUALITY IN DP .................................................................................................................................. 2 2.1 CSDP FOR THE NON-CORRELATED CASE ............................................................................................... 2 2.1.1 The guidelines ........................................................................................................................... 2 2.1.2 Augmentation of the guidelines ................................................................................................. 3 2.1.3 Uncertainty adjustment ............................................................................................................. 5 2.1.4 Implemented CSDP algorithm ................................................................................................... 5 2.2 ADVANTAGE OF CONSTRUCTIVE DP ..................................................................................................... 5 2.3 RELATED METHODS ............................................................................................................................. 6
3
CORRELATED INFLOWS ................................................................................................................. 7 3.1 REPRESENTATION OF THE CORRELATED INFLOWS .................................................................................. 7 3.2 CSDP FOR THE CORRELATED CASE....................................................................................................... 8 3.2.1 Augmentation of the guidelines ................................................................................................. 9 3.2.2 Uncertainty adjustment ........................................................................................................... 11
4
REVISED CSDP ................................................................................................................................. 12 4.1 DIAGONAL REPRESENTATION OF THE REVISED CSDP........................................................................... 12 4.1.1 Representing a point on a diagonal line .................................................................................. 12 4.1.2 Guidelines on the diagonal line............................................................................................... 13 4.1.3 Augmenting the guidelines on diagonal lines .......................................................................... 13 4.1.4 Beginning-of-period MWV curves ........................................................................................... 16 4.1.5 MWV curves along diagonal grid lines .................................................................................... 16 4.1.6 CSDP algorithm using a diagonal grid representation ............................................................ 17 4.2 PERFORMANCE OF THE REVISED CSDP ............................................................................................... 18 4.2.1 The form of guidelines............................................................................................................. 18 4.3 SIMULATED COST SAVINGS ................................................................................................................. 21
5
CONCLUSIONS ................................................................................................................................. 21
REFERENCES............................................................................................................................................. 22
Energy Modelling Research Group
ii
List of Figures Figure 1: Typical end-of-period MWV curve .......................................................................... 2 Figure 2: Typical marginal water value surface with guidelines ............................................... 3 Figure 3: The augmentation .................................................................................................... 4 Figure 4: Typical end-of-period MWV surface ....................................................................... 8 Figure 5: The guidelines for correlated case ............................................................................ 9 Figure 6: Points on diagonal line........................................................................................... 12 Figure 7: Transformation in diagonal representation ............................................................. 14 Figure 8: The augmented guidelines ..................................................................................... 15 Figure 9: Beginning-of-period MWV along new diagonal line............................................... 17 Figure 10: Guidelines for correlated case .............................................................................. 18 Figure 11: The equilibrium slope with the correlation coefficient .......................................... 20 Figure 12: Shortage guideline ............................................................................................... 20
Energy Modelling Research Group
iii
1
Introduction
The reservoir management problem for a hydrothermal power system is to decide how much water should be released in each period so as to minimise the expected operational cost, including the fuel cost of thermal plants, and the cost of shortages, should they occur. In economic terms, the basic principle of reservoir management is to continue generation until the marginal value of releasing water falls to the marginal value of storing water. The marginal value of releasing water is equal to the fuel cost of the most expensive thermal station used to meet residual demand after hydro generation, or during a shortage, the value of meeting loads which would otherwise not be serviced. The marginal value of the water in stock depends on the optimal utilisation of that stock in future periods. Because all reservoirs have limited storage capacities, and because the inflows are uncertain, water management becomes quite complex. Dynamic Programming (DP) has been widely used in reservoir management. This is mainly due to its ability to handle non-linear and stochastic features which characterise many reservoir systems. A drawback with traditional DP is that it can only provide a discrete approximation for problems with a continuous state space. Problems involving several "state variables" (in this case reservoir storage) also run into serious computational difficulties, the so called "curse of dimensionality". A dual approach to Dynamic Programming was formulated by Read and George [1986, 1990], and has the advantage of constructing the optimal solution directly and hence reducing the computational requirements, and a very similar technique was developed by Bannister and Kaye [1991] and further developed by Kaye and Travers [1997]. Kaye and Travers refer to this approach as a “Constructive Dynamic Programming” (CDP) technique, and we will use that term here, in order to distinguish it from the rather different technique which has been developed by Pereira and Pinto [1991] under the name of Stochastic Dual DP. CDP is more accurate than primal DP when finding a solution using the same number of grid points, because CDP deals directly with the critical values, which determine the form of the optimal strategy, rather than trying to infer them by interpolating on an arbitrary grid. This also eliminates any need to approximate the solution by successive refinement of the grids as is commonly done in DP. Stochastic CDP (CSDP) retains much of this advantage, and has been successfully applied to several problems, including a two reservoir model, RESOP, (Read [1985, 1990], which has been used in a planning model for the New Zealand system since 1985, and to manage reservoir release since 1990. In that model, flows are assumed to be independent for successive periods, so that recent inflow observations can not impact on release decisions. If inflows are correlated, though, then current inflows should impact on release decisions. For example, if current inflows are high, then we expect the inflows in the next period to be higher than average. This means that the marginal value of storing water must be lower than otherwise, so relatively more water should therefore be released. If inflows are highly correlated, then current flows may have a high impact on release decisions. If that impact is large, then without taking account of correlated inflows, the model may not provide suitable release schedules. For the New Zealand system which Yang [1995] used as a case study, the weekly correlation coefficient is between 0.6 and 0.8. The impact of correlation on release decisions Energy Modelling Research Group
1
has been tested using heuristic methods on the basis of the existing model, RESOP. Results suggest that significantly better decision rules could be made by accounting for the correlated inflows. Thus the purpose of this paper has been to generalise the CSDP approach to handle correlated inflows.
2 2.1
Duality in DP CSDP for the non-correlated case
In CSDP, the optimal release rules are expressed by a set of guidelines consisting of the locus t over time of storage levels (S ) whose marginal water values (MWV) are equal to the fuel costs of particular thermal stations. This discussion summarises that in Read [1985, 1990] and in Read and George [1986, 1990]. For simplicity we assume a single reservoir, although Read's model actually has two reservoirs. We also assume a continuous state space and convex differentiable cost, and hence value, functions. 2.1.1 The guidelines A typical end-of-period MWV curve is shown in Figure 1. If thermal stations are indexed by b = 1, ... B, in order of increasing fuel costs, MCb (ie. in "merit order"), the guideline level for t+1 thermal station "b" at the end of the period "t", Lb is determined by: Ltb+ 1 = MIN{St + 1: MWV(S t + 1) = MC bt }
(1)
MWV t
MWV curve
MWVbt
0
Ltb
St
Figure 1: Typical end-of-period MWV curve If the end-of-period storage is expected to lie below the guideline b at the end of the period, t+1 Lb , then all thermal stations up to, and including, b should be base-loaded, used before hydro generation to meet residual demand. The guidelines are the level contours of the marginal water value surface (Figure 2). The MWV surface gives the MWV for any given storage level in any given period.
Energy Modelling Research Group
2
Title: wvsc.eps Creator: MATLAB, The Mathworks, Inc. Preview: This EPS picture was not saved with a preview included in it. Comment: This EPS picture will print to a PostScript printer, but not to other types of printers.
Figure 2: Typical marginal water value surface with guidelines 2.1.2 Augmentation of the guidelines However, decisions must be made at the beginning of each period, when the end of period storage is not yet known. It is therefore necessary to determine how low the beginning of period storage can be before stations are required to be base-loaded up to b. The lower augmented guideline level for the beginning of period t, LMIN tb , is defined as the storage t
level such that, starting from LMIN tb with expected inflow EF and base loading level R tb , we would expect to end up at the guideline Ltb+ 1 . It is therefore necessary to find LMIN tb such that: LMINb + EFt − R tb = L t+b1 t
(2)
That is: t
LMINb = Lbt+1 - EFt + R bt
Energy Modelling Research Group
(3)
3
t
Similarly, the upper augmented guideline level for the beginning of period t, LMAXb , is t t defined as the storage level such that, starting from LMAXb with the expected inflow EF and t t+1 base loading level Rb-1 , we would expect to end up at the guideline Lb . That is: t t t t+1 LMAX b + EF - R b-1 = L b
(4)
t t+1 t t LMAX b = L b - EF + R b-1
(5)
t+1
Since Lb is defined as the level below which full base-loading of b is desirable, partial baset t+1 loading of b is definitely desirable below LMAXb , to achieve Lb . The release decision rules can now be re-expressed in terms of the beginning-of-period t "augmented guidelines" shown in Figure 3. If the storage level is above LMAXb , the optimal t release decision is to only base-load thermal stations up to b, while if it is below LMINb t station b should be base-loaded as far as possible. If the storage level lies between LMAXb t and LMINb , then thermal station b should be only partially base-loaded to achieve a storage t+1 "target" of Lb . The precise loading level can be found by linear interpolation. For example, t t if the storage level is half way between LMAXb and LMINb , then half loading of station b t would be appropriate, if the storage level is below LMINb , then thermal station b should be fully base-loaded. St + 1
St
base-load b-1 only
LMAX bt Ltb+ 1
partially base-load b LMIN bt
fully base-load b as well partially base-load b+1
Ltb+ +11
LMAX bt + 1
Figure 3: The augmentation The augmented guidelines also define the MWV curve for the beginning of the period. We ignore discounting, wastage and holding cost. The beginning-of-period MWV is zero for (hypothetical) storage levels above the “full level”, and if the reservoir is empty, it will equal Energy Modelling Research Group
4
the marginal cost of using thermal stations to meet loads in the period with current inflows, or the marginal ‘shortage cost’if this is not possible. t The MWV is constant at the marginal cost of thermal station b in the range [LMAXb , t LMINb ]. That is: t t MWV ( S ) = MCb ,
[
for S t ∈ LMAX bt , LMIN bt
]
t
(6) t
Between the augmented guidelines LMINb and LMAXb+1 , the MWV lies between MCb and MCb+1. In fact, Read and George [1986, 1990] argue that, since the release is constant over that range, each storage level in that range will map to a corresponding point on the end of period curve, so that this section of the beginning-of-period curve is identical to the corresponding section of the end of period curve, appropriately translated. That is: t t t+1 t+1 t t t MWV (S ) = MWV (S (S , EF , R b )) = MWV t+1 (St + EFt - R tb ),
[
for S t ∈ LMAX bt + 1 , LMIN bt
]
(7)
2.1.3 Uncertainty adjustment In stochastic problems, the inflow is still unknown when the release decision is made at the beginning of the period. For this reason, the release decision must be based on the expected MWV for the end of the period. This can be modelled by saying that we aim at an end-ofperiod storage "target" assuming the expected inflow. At the end of the period, the actual inflows become apparent, producing a range of possible storage levels around the target. Alternative models are possible, but this approximation is simpler, and no less accurate than the other simple alternative, that of assuming that the inflows are known at the beginning of the week. The expected MWV for any end-of-period target can be generated by averaging the MWV for these storage levels. Read [1985] refers to the procedure for calculating the expected MWV as "Uncertainty Adjustment". 2.1.4 Implemented CSDP algorithm The process of CSDP starts at the final period (ie, t=T) and assumes that the end-of-period expected MWV (for which t=T+1) is known. The guideline for thermal station b at the end of period t is first determined by searching for the storage level whose expected MWV is equal to the marginal fuel cost of this thermal station. The beginning-of-period MWV curve for period t is then determined from the end-of-period expected MWV curve using Equations 6 and 7, in which the lower and upper augmented guideline are defined by Equations 3 and 5. Next the end-of-period expected MWV curve for period t-1 is determined from the beginning of period MWV for period t and the inflow distribution, via the uncertainty adjustment. The whole process is complete when the beginning of the horizon is reached. 2.2
Advantage of Constructive DP
For deterministic problems with a continuous state space, DP is fundamentally more accurate Energy Modelling Research Group
5
than primal DP, because it does not require the state space to be discretised, and so can produce a solution which is as accurate as the assumed end-of-horizon MWV function will allow. Primal DP can only approximate this solution by choosing finer and finer grids and placing them as close to the expected guideline positions as possible. Analytical Dual DP is also more efficient than primal DP, because the MWVs at the beginning of a period are constructed directly from the MWVs at the end of that period by an algebraic transformation, requiring considerably less computation time than the recursive optimisation required by primal DP. Also the number of points needed to describe the decision rules is normally lower than the number of points needed to give a reasonably accurate description of the water value curves, reducing the number of decisions that have to be considered in each period. For stochastic problems, Analytical Dual DP can also produce exact solutions, but the number of corners in the MWV curve can explode as the algorithm works backwards through time. Thus in practice, it loses some accuracy if approximations are needed to keep the number of points in the MWV solution to a reasonable level. Although this loss of accuracy can be controlled, it is not clear, a priori, whether the net result will be more or less accurate than that for primal DP. Also, when solving stochastic problems, the computational efficiency of both Analytical Dual DP and primal DP depends heavily on the complexity of the expected MWV calculations. The relative merits of the two approaches are therefore dependent on the characteristics of particular applications. Macgregor [1991] studied the efficiency of both primal and Analytical Dual DP algorithms for a linear production/inventory problem, concluding that the Analytical Dual DP algorithm was considerably more efficient than the primal DP algorithm in both deterministic and stochastic cases. Yang [1995] came to the same conclusion, and also showed that Analytical Dual DP algorithm was more accurate in both deterministic and stochastic cases. 2.3
Related methods
Yang [1995] surveys some earlier approaches to duality in DP by Iwamoto [1977], Ben Israel and Flam [1989]. Here we discuss the two approaches which have been implemented for large scale reservoir management. Pereira and Pinto [1991] develop a somewhat similar technique which they call Stochastic Dual DP (SDDP), but from a mathematical programming base. This technique also constructs a marginal value function without discretising the continuous state-space, but the value function is assumed to be piece-wise linear, so that the marginal value function is represented as a set of level planes. Technically, these are formed as "cuts" in a Benders decomposition of the Stochastic Programming problem. This model has been applied to systems with a large number of reservoirs for which it would be impossible to define a complete operating strategy using the approach of Read and George [1986, 1990]. Rather than produce exact decision rules for the entire state-space and planning horizon, it focuses on producing a good solution for the first period, and only forms approximate decision rules for later periods to the extent that this seems likely to significantly improve the initial decision for a small set of inflow scenarios. Their algorithm performs several iterations. Each iteration consists of a forward pass, then a backward pass. When working forward for the first time, it ignores the future at each time step. Hence at each step it will use as much water as possible, and will not keep any for the future when it might be more valuable. On the backward pass, information is passed back Energy Modelling Research Group
6
which allows the overall solution to be improved by taking account of the future. This serves to refine the representation of the marginal value curve around the simulated trajectories, that is around the region where storage is likely to lie in future. The forward pass provides an upper bound on fuel cost while the backward pass provides a lower bound on solution value, and the algorithm continues until these are acceptably close. Thus, as formulated by Read and George [1986, 1990], CSDP is an essentially exact method for relatively low dimensional problems, while SDDP is an approximate method for higher dimensional problems. CSDP defines the whole marginal value surface exactly, while SDDP constructs a locally accurate approximation, using a simplified model of future uncertainty.
3
Correlated Inflows
3.1
Representation of the correlated inflows
Assuming that the inflows in the current period are correlated with those in the previous period, they can be expressed as follows:
(
)
F t = AF t + α × F t − 1 − AF t − 1 + e t
(8)
where Ft
:
The actual inflow in period t;
AF
:
The average inflow in period t;
a
:
A positive correlation coefficient;
t
:
The random perturbation experienced in period t, with mean zero.
t
e t
Let f be the inflow deviation from the average inflow in period t, ie: f t = F t − AF t
(9)
= α ×f t − 1 + e t t
t-1
t
t-1
We can also define Ef (f ) and EF (f ) to be the expected inflow deviation and the expected flow in period t, respectively, given the observed inflow deviation in period t-1. Clearly:
( )
EF t f t − 1 = α ×f t − 1
(10)
t t-1 t t t-1 EF (f ) = AF + Ef (f )
(11)
t
t
t
t
Often it is more convenient to work in terms of f and Ef , rather than F and EF , and to draw t guidelines etc. with a horizontal axis centred on f =0. This means that guidelines or MWVs measured on the vertical axis apply for the average flows expected in each period (AF), and are comparable to those for the non-correlated case. It also means that certain transformations are reduced to simple vertical shifts on that axis. Energy Modelling Research Group
7
3.2
CSDP for the correlated case
In the non-correlated case, the system status at the beginning of period t is described by only one state variable, the reservoir storage level. When inflows are modelled by Equation 8, two state variables are required to describe the system status at the beginning of period t, the reservoir inflow level in the previous period t-1, and the reservoir storage level at the beginning of period t. For the non-correlated case, the MWV is a monotonic decreasing function of the storage level, so that extra water is worth less when the level of the reservoir is high. For the correlated case, the MWV surface is also a monotonic decreasing function of the inflow level. That is, the marginal value of water in stock is lower if current flows are high. A typical endof-period MWV surface is shown in Figure 4 MWV t + 1
ESt + 1 MWV t + 1 = MC bt + 1
EF t
guideline
Figure 4: Typical end-of-period MWV surface The guideline for each thermal station is again defined as the locus of inflow/storage pairs whose MWV is equal to the marginal fuel cost of that particular thermal station. Mathematically, the guideline for thermal station b, in the correlated case, can be expressed by: t t t t t t-1 t-1 Lb ( f ) = MIN { S : MWV ( f , S ) = MCb}
(12)
t
As for the non-correlated case, choosing Lb as the minimum of this set is arbitrary, but t adequate for our purposes. Thus Lb represents the storage level below which thermal station b must definitely be base-loaded. In the non-correlated case, it was shown that CSDP, can produce a MWV curve for a Energy Modelling Research Group
8
continuous storage state variable, rather than just for discrete values of storage state. In principle, the same is true for the correlated case. Because the functional form of the two dimensional MWV function is not known, though, we prefer to approximate the MWV surface by a set of MWV curves, each defined for a particular inflow level chosen from a discrete grid. Thus, we solve Equation 12 for each of these discrete flow levels, to define t t-1 “guidelines”, Lb (f ), as shown in Figure 5. All thermal stations up to, and including, b should be base-loaded if the end of period inflow/storage pair is expected to lie below the guideline Lb t-1 at the end of the period. The guideline levels for any given f correspond to the guideline levels for a single period in the non-correlated case. St
SMAX
B
A
Χ
Χ
( ) (f )
Ltb f t − 1 Ltb + 1 SMAX
fA
0
fB
t− 1
f t− 1
Figure 5: The guidelines for correlated case A detailed justification for the shape of these guidelines is discussed later, but, intuitively, these guidelines should be decreasing as a function of the inflow level, but must lie within the maximum and minimum storage levels, as shown. 3.2.1 Augmentation of the guidelines As for the non-correlated case, decisions must be made at the beginning of each period, when the end-of-period state is not yet known. The release decision rules must be re-expressed in t+1 t terms of beginning-of-period "augmented guidelines". Lb (Ef ), the guideline for thermal t station b at the end of period t, and for expected inflow deviation Ef , can be augmented in the same way as that in the non-correlated case. We need to ascertain how low the beginning-of-period storage can be before we need to t-1 base-load station b, given the observed inflow f . We define the lower augmented guideline t t-1 at the beginning of the period, LMINb (f ), to be the storage level such that, starting from Energy Modelling Research Group
9
t
t-1
t-1
t
LMINb (f ) with the observed inflow f and base-loading level Rb , we would expect to end t+1 t t-1 t t-1 up at Lb (Ef (f )). Thus, we want to find LMINb (f ) such that: LMIN bt ( f t-1 ) + AFt + Ef t (f t-1) - R tb = L t+b1 (Ef t (f t-1))
(13)
t t-1 t+1 t t-1 t t t-1 t LMINb ( f ) = L b (Ef (f )) - AF - Ef (f ) + R b = L bt+1 (α f t-1) - α f t-1 + (R tb - AF t)
(14)
That is:
Similarly, we want to know how high the beginning of period storage can be before we can t-1 dispense with station b (except for peaking purposes), given the observed inflow f . We t t-1 define the upper augmented guideline at the beginning of the period, LMAXb (f ), as the t t-1 t-1 storage level such that, starting from LMAXb (f ) with observed inflow f and base-loading t t+1 t t-1 t t-1 level Rb-1 , we would expect to end up at Lb (Ef (f )). Thus, we want to find LMAXb (f ) such that: LMAX tb ( f t-1 ) + AFt + Ef t (f t-1) - R tb-1 = L t+b1 (Ef t (f t-1))
(15)
t t-1 t+1 t t-1 t t t-1 t LMAX b ( f ) = L b (Ef (f )) - AF - Ef (f ) + R b-1 = L t+b1 (α f t-1) - α f t-1 + (R b-t 1 - AF t)
(16)
That is:
t
This transformation shifts the guideline up or down the storage axis by an amount Rb-1 t AF , corresponding to the transformation required for the non-correlated case. It also shears t the guideline parallel to the storage axis by an amount -Ef , representing the impact of flow on storage. Finally, it stretches the guideline along the inflow axis by 1/a, because one unit of inflow variation in the current period, will produce a units of expected inflow variation in the following period. Thus, in order to aim at end-of-period points on the vertical line through, say, Ef1, the points at the beginning of the period must be on a vertical line through Ef1/a. t t The upper augmented guideline is shifted by Rb-1 -AF , while the lower augmented t t t guideline is shifted by Rb -AF . Thus the lower augmented guideline, LMINb , is parallel to the t upper augmented guideline, LMAXb , but shifted down Rb-1 - Rb along the storage axis, just as for the non-correlated case. The augmentation process is similar to that for the non-correlated case, but now should be done for several observed inflow levels. The augmented guidelines define release decisions in terms of the beginning-of-period storage level for each inflow level, just as for the non-correlated case. As for the noncorrelated case, the augmentation process also defines the MWV surface for the beginning of the period at these observed inflow levels. If the storage is in the range between the upper and t t lower augmented guidelines for the thermal station b, LMAXb (Ef1/a) and LMINb (Ef1/a), then the MWV is constant at the marginal cost of thermal station b, MCb. This produces a “flat section” in the beginning-of-period MWV curve, ie: t t MWV (Ef 1 / α , S ) = MCb t t t for S ∈ [ LMINb ( Ef 1 / α ), LMAX b ( Ef 1 / α )]
Energy Modelling Research Group
(17)
10
t
Or, equivalently, if f represents the observed flow in t, then t t t MWV (f , S ) = MC b t t t t t for S ∈ [LMINb (f ), LMAX b (f )]
(18)
If the storage is between the augmented guidelines for b and b+1, the MWV should be between MCb and MCb+1. Since the release is constant over that range, any storage level in that range will map to a corresponding point on the end-of-period curve. Thus this section of the beginning-of-period curve is identical to the corresponding section of the end-of-period curve, appropriately translated. That is: t t t+1 t+1 t t MWV (Ef 1 / α , S ) = MWV (Ef 1 , ES (S , Ef 1 / α , R b )) = MWV t+1 (Ef 1 , St + AFt + Ef 1t - R bt )
(19)
for St ∈ (LMAX tb+1 (Ef 1 / α ), LMINtb (Ef 1 / α ))
Or, equivalently, t t t t+1 t t t t t MWV (f , S ) = MWV ( α f , S + AF + α f - R b ) t t t t t for S ∈ (LMINb+1 (f ), LMINb (f ))
(20)
As for the non-correlated case, the above discussion ignores discounting, wastage and holding cost. 3.2.2 Uncertainty adjustment For a stochastic problem, the inflow is not known when the release decision is made at the beginning of period, so the decision will be based on the expected MWV. As in the noncorrelated case, this can be modelled by saying that we aim at an end of period inflow/storage "target", assuming the inflows which we expect, given the observed inflows for the previous period. At the end of the period the actual inflows become apparent, producing a range of possible inflow/storage pairs around the target. Thus the expected MWV for the end-ofperiod target can be generated by averaging the MWVs for these inflow/storage pairs. Because the beginning-of-period MWV curves are only constructed for discrete inflow levels, the MWVs for intermediate inflow levels have to be determined by linear interpolations. In particular, since the curve for Ef1 in period t+1 maps to a curve for Ef1/a in period t, the spacing of the transformed copies of the grid lines will be increasing by a factor of 1/a at each period. If a reasonable representation is to be preserved, new grid lines will need to be inserted at some stage. This is achieved here by using the same spacing for each period, and interpolating to find MWVs on these lines. Alternatively, we could have let the spacing increase from ? , say, to approximately 2? , then inserted new lines to approximately preserve the original spacing. That approach, while slightly more complex, would probably yield a substantial reduction in computation time.
Energy Modelling Research Group
11
4 4.1
Revised CSDP Diagonal representation of the revised CSDP
We note, though, that although a single reservoir with serial correlations requires a two dimensional Stochastic DP, the stochasticity of the problem is really only one dimensional. t This is due to the fact that an unexpected extra inflow, e , in Equation 8, becomes an t unexpected extra storage of e , so that the actual inflow/storage pairs around a target will all lie somewhere along the 45 degree diagonal line. In order to exploit this observation computationally, we now choose to define the guidelines along such diagonal lines, and would like to adapt the augmentation process, so as to form the MWV curve along diagonal lines directly from the expected end-of-period MWV surface, also defined on diagonal lines. However, as shown below, the augmentation process actually transforms end-of-period diagonal lines into horizontal lines for the beginning of the period. The beginning-of-period augmented guidelines then have to be determined by interpolation, to form beginning-ofperiod MWV curves along a new set of diagonal lines. 4.1.1 Representing a point on a diagonal line In our previous discussion, we adopted an orthogonal representation. Thus a point in the two dimensional state space, say A, has been represented as a height of SA, along the vertical line fA, as shown in Figure 6. Alternatively we could refer to point A as being at a height of SA, along the 45 degree diagonal line, passing through the axis at flow level dA. Now we choose this representation in our model. St
SA
A
45º
0
dA
fA
f t+ 1
Figure 6: Points on diagonal line Clearly, fA=dA+SA, and M, the transformation between this representation and the standard one is given by:
M (d, S) = (f (d, S), S) = (d + S, S)
(21)
The reverse transformation, R can be represented as:
R (f , S) = (d(f , S), S) = (f − S, S) Energy Modelling Research Group
(22) 12
4.1.2 Guidelines on the diagonal line Using this new representation of a point in the two dimensional state space, the guideline, defined as a locus of inflow/storage pairs whose MWVs are equal to the marginal fuel cost of a particular thermal station in Section 4, is now expressed in a different way. To distinguish t t these two, we now use DLb , instead of Lb to refer to the guideline of thermal station b in period t, as defined on the diagonal lines. The guideline levels, DL, can be determined by searching along diagonal lines for the storage levels whose MWVs equal the marginal fuel costs of particular thermal stations. We can also define MWV curves along diagonal lines, DMWV and then define DL as:
{
DLtb (d) = MIN S t : DMWV t (d, S t ) = MC tb
}
(23)
t
Again, we choose DLb as the minimum of this set so that it is the level below which b must definitely be fully base-loaded. The MWV function will be monotone along this diagonal line if it is monotone in both S and f. Clearly: DMWV t (d, S) = MWV t (d + S, S)
(24)
DLtb (d) = Ltb (d + DLtb (d ))
(25)
and
4.1.3 Augmenting the guidelines on diagonal lines The augmentation procedure for the diagonal case will be described in two steps. First, we consider the transformation from the end-of-period state space into the beginning-of-period state space, then the insertion of flats. First, assume that water releases are the same for all inflow/storage pairs at the t+1 beginning of the period, and consider a point in the end-of-period state space (d, ES ). This t t+1 point can also be represented in the orthogonal representation (via Equation 21) as (Ef , ES ) t+1 t+1 = (d+ES , ES ). The corresponding beginning-of-period point can be described in the t-1 t t t t+1 orthogonal representation as (f (Ef ), S (Ef , ES )), where: Ef d + ES = = f t-1 (d, ESt+1) α α t
t-1 t f (Ef ) =
t+1
(26)
t t t+1 t+1 t t t S (Ef , ES ) = ES - AF - Ef + R = ESt+1 - AFt - (d + ESt+1) + R t
(27)
= R t - AFt - d = St (d, R ) t
t+1
t
It will be observed that the ES term disappears from the expression for S in Equation t-1 27, but appears in the expression for f in Equation 26. If end-of-period points A, B and C are on the diagonal line d (d 1, Let t = t-1 and Go to step "Derive Guidelines", otherwise stop. 4.2
Performance of the revised CSDP
4.2.1 The form of guidelines Recursively applying the augmentation transformation has an impact on the ultimate form of the guidelines. An example of actual guidelines produced by Dual DP for the correlated inflow case is shown in Figure 10. Title: BGI Graphics Creator: BGI by Borland International Preview: This EPS picture was not saved with a preview included in it. Comment: This EPS picture will print to a PostScript printer, but not to other types of printers.
Figure 10: Guidelines for correlated case
Energy Modelling Research Group
18
As expected, for extreme high/low inflows, the guidelines lie along the spill/shortage bounds. For moderate inflows the guidelines curve around to form a diagonal line, sloping downwards. The form of the guidelines results from the following factors. The "Equilibrium" Slope of the Guidelines t-1
First, note that the slope of the (upper/lower) augmented guideline, Slope , may be different t from the slope of the initial guideline, Slope . In fact : Slope = α ×(Slope - 1) t-1
t
(36)
If the storage bounds SMAX and SMIN are assumed to be non-binding, and the uncertainty adjustment is ignored, the augmented guidelines for period t will become the endof-period guidelines for period t-1. They will then be augmented again to give a new slope: t-1 t Slope = α ×(Slope - 1)
= α ×(α ×( Slope t+1 - 1) - 1) (37)Eventually, as T -> 4 = α 2 ×Slope t+1 - α 2 - α successive augmentations will cause all guidelines to assume an "equilibrium slope", ESlope, providing that the correlation coefficient is less than 1. That is: (T)
ESlope =
α α -1
(38)
The "equilibrium slope" depends on the correlation coefficient a. It becomes steeper as the correlation coefficient increases, as shown Figure 11. In the non-correlated case, the guidelines are independent of inflows, or in other words, a is equal to zero, so the "equilibrium slope" is equal to zero and the guideline is horizontal. Conversely, a =1 corresponds to the extreme case of deterministic inflows, for which the "guidelines" become meaningless. The Guidelines for Spillage and Shortage In practice, the storage bounds, SMIN and SMAX have an impact on all guidelines, as shown in Figure 12. t We could assume that SMAX formed a horizontal spill guideline for the end of the horizon. If storage is above this guideline, then spill occurs immediately in this period. Ignoring uncertainty, this spill guideline for period t will be transformed into a guideline, for period t-1, with a slope of (-a), as shown in Figure 12. This guideline acts as an "induced constraint" on the state space for period t-1, indicating that when storage is above that guideline, spill is expected in the next period. This guideline is, however, bounded by the t-1 storage bound SMAX , producing an effective spill guideline with two sections, the slopes being 0 and (-a).
Energy Modelling Research Group
19
St SMAX
α = 0 (ESlope = 0)
L b (0)
α = 0.7 (ESlope = -1) α = 0 .7
α = 1 (ESlope = -4) SMIN
f t− 1
0
Figure 11: The equilibrium slope with the correlation coefficient
St
Spill for period t slope = 0 SMAX
Spill for period t + 1 slope = - α Spill for period t + 2 slope = - α (α + 1)
Shortage for period t + 2 slope = - α (α + 1) Shortage for period t + 2 slope = - α
Shortage for period t slope = 0
SMIN
0
f t− 1
Figure 12: Shortage guideline Energy Modelling Research Group
20
When this guideline is again augmented, it has three sections with slopes of 0, (-a), and a(a+1), representing immediate spill, and spill in one and two periods time, respectively. Thus we have an effective spill guideline as shown in Figure 12, and an effective shortage guideline can be formed similarly. The uncertainty adjustment, however, will moderate the above effects, because even when spill is projected on the basis of expected flows, there is a significant probability that it will not occur. The result is that guidelines tend to a reverse sigmoid shape, sloping down linearly at the equilibrium slope for moderate inflows, and becoming parallel to spill and shortage bounds for extreme high or low inflows, as shown in Figure 12. 4.3
Simulated cost savings
Yang and Read [1990] describe how the revised CSDP algorithm has been applied to the New Zealand power system, in which all reservoirs are aggregated into one equivalent "reservoir". Up to 75% of the electricity generated in New Zealand comes from hydro sources. The long term storage capacity totals about 3700 kWh, roughly 10% of the annual generation. It appears that accounting for correlation explicitly can yield a 5.5% reduction in the simulated cost, which means more than $11.72 million could be saved annually for the New Zealand system by comparison with a method which ignores correlation entirely. Actual savings would be less, however, as the modelling system implemented does already include heuristics to deal with correlation, and because it may not be politically acceptable in practice, to allow storage to fall in situations where higher than average flows were expected. The new method is also shown to be more robust than the non-correlated model in that it is relatively insensitive to mis-estimation of the hydrological distribution. It should be understood, though, that these simulated savings could be obtained using any stochastic dynamic programming approaches which accounted for inflow correlation. The advantage claimed for CSDP is primarily computational, as noted earlier. We should also observe that the insights derived from this analysis may also have proved useful in deriving heuristic adjustments to other models.
5
Conclusions
We have described in detail the CDP algorithm, both in its original form due to Read and George [1986, 1990] and in the context of a system with serial inflow correlation. For the correlated model we have shown how to take advantage of the symmetry of the state space (inflow variation translates directly into storage variation for the following period) to greatly reduce the computational complexity. Computational results (Yang [1995], Macgregor [1991] demonstrate that CDP is both more efficient and more accurate than primal DP. Further, Yang [1995] demonstrates that explicitly modelling the correlation can achieve cost savings of more than five percent over the non-correlated model.
Energy Modelling Research Group
21
References Bannister, C.H. and R.J. Kaye, A rapid method for optimisation of linear systems with storage, Operations Research, Vol. 39, No. 2, pp 220-232, 1991. Ben-Israel, A. and S.D. Flam, Input optimisation for infinite horizon discounted programs, Journal of Optimisation Theory and Applications, Vol. 61, No. 3, pp 347-357, 1989. Iwamoto, S., Inverse theorems in dynamic programming, Journal of Mathematical Analysis and Applications, Vol. 58, pp 113-134, 1977. J. Kaye and D. Travers “Constructive Dynamic Programming” submitted to Operations Research 1997. Macgregor, A.D., Dual Dynamic Programming, A Computational Study, M.Sc. Thesis, University of Canterbury, 1991. Pereira, M.V.F. and L.M.V.G. Pinto, Multi-stage stochastic optimisation applied to energy planning, Mathematical Programming, 52, pp 359-375, 1991. Read, E.G. and J.A. George, Dual dynamic programming for linear production/inventory systems, New Zealand Operational Research, 14, 2, pp 133-136, 1986. Read, E.G. and J.A. George, Dual dynamic programming for linear production/inventory systems, Computers & Mathematics With Applications, 19, pp 29-42, 1990. Read, E.G., A dual approach to stochastic dynamic programming for reservoir release scheduling, In Dynamic Programming For Optimal Water Resource Systems Analysis (Ed. A.O. Esogbue), Prentice-Hall Englewood Cliffs, N.J., pp 361-372, 1990. Read, E.G., A new variant of stochastic DP for multi-reservoir release scheduling, in Proceedings of the 21st Operational Research Society Of New Zealand Conference, pp 4-87, Wellington, New Zealand, September 1985. Yang, M. and E.G. Read, A dual dynamic programming approach to reservoir management with correlated inflows, in Proceedings of the Operational Research Society Of New Zealand Conference, pp 21-25, Hamilton, New Zealand, August 1990. Yang, M., Dual dynamic programming for reservoir management with correlated inflows, Ph.D. Thesis, University Of Canterbury, New Zealand, 1995.
Energy Modelling Research Group
22
