Introduction. The cyclic steam injection process has become the most widely applied and most successful thermal re- covery technique in use today.1-4 ...
A Dynamic Programming Model of the Cyclic Steam Injection Process R. G. Bentsen, SPE-AIME, The U. of Alberta
D. A. T. Donohue, SPE-AIME, The Pennsylvania State U.*
Introduction The cyclic steam injection process has become the most widely applied and most successful thermal recovery technique in use today.1-4 Normally, steam stimulation is repeated several times during the life of a project. The problem is to select the optimum number and size of steam treatments so that the profits realized over the entire life of the project are maximum. Obviously, operating costs rise directly with the frequency and amount of steam injection. Operating efficiency, on the other hand, can be adversely affected if the amount and frequency of stimulation are insufficient. Thus arises a multistage decision problem; that is, a decision must be made at each sub-period (day, for example), as to whether steam should be injected, and, if so, how much. It is desirable that these decisions be made in such a way as to secure maximum net profit before taxes. This can be achieved by dynamic programming. To use dynamic programming, we must be able to predict not only the production rate-time curve resulting from a given stimulation, but also how changes in stimulation policy will affect this curve. In this study, the effect of steam injection on well performance is estimated on the basis of a simple model that takes into account both steady-state and transient flow. Oil production rate is calculated for both types of flow, and the larger of the two values is used. This allows the calculation to change to radial steady-state * Presently at Boston U.
flow at such time as this mechanism becomes more effective than the transient mechanism. We propose to show how dynamic programming can be used to optimize the steam soak process with respect to net profit. For this, a mathematical representation of the steam soak technique is necessary. A further objective, therefore, is the formulation of a mathematical model capable of realistically simulating the physical process.
Oil Production Response Model The advantage of using a mathematical model is its generality and ease of manipulation. However, the model must be an accurate representation of the physical process, for a solution derived from a model can be no better than the model itself. The question now arises as to how well the model should be expected to simulate reality. In general, the simplest model should be tried first, for additional accuracy is apt to require additional cost and time. Also, the search for a high degree of accuracy may yield only greater complexity and difficulty without yielding significantly better results. With this in mind, we kept our model as simple as possible, yet generally representative of the actual physical process. The number of state variables is limited to three, and stochastic variations in the parameters are ignored. As a consequence, use of this model should be restricted to cases in which the underlying assumptions are not severely violated. In
Because cyclic steam injection is a multistage-decision process, dynamic programmingcharacterized by a criterion function to be satisfied, a number of stages in time and space, and decisions to be made at each stage to satisfy the criterion function - may be used to predict an optimum operating policy for the specific injection project. 1582
JOURNAL OF PETROLEUM TECHNOLOGY
particular, it should not be applied to reservoirs wherein the crude is very viscous and the pressure very low. The idealized model used in this study divides the reservoir around a wellbore into two concentric cylinders of productive formation, sealed at top and bottom with impermeable material. The inner cylinder, which extends from the wellbore radius to the heated radius, is termed the hot zone; and the outer cylinder, which extends from the outer boundary of the hot zone to the outer drainage boundary, is called the cold zone. In developing the oil production response model, the following assumptions are made: 1. The radial extent of the heated zone can be estimated using the equation of Marx and Langenheim. 5 2. The temperature history of the heated zone can be calculated using the method of Boberg and Lantz. 6 3. The backflow process is characterized initially by a transient flow period, followed by a period of steady-state flow. 4. The production rate during the period of transient flow can be estimated using the method of van Everdingen and Hurst. 7 5. The production rate during the period of steadystate flow can be calculated using the method of Boberg and Lantz. 6. Transient flow occurs until the rate predicted by the steady-state model is greater than that of the transient model. At this time the flow becomes steadystate in nature. Assumption 6 is admittedly arbitrary. However, the exact point in time at which the flow becomes steady-state in nature has very little effect on results. Consequently, the development of a more sophisticated method for determining this point was not felt to be warranted. Because the flow model presented here is an extension of a previously published model, only the transient flow portion will be discussed fully. Details concerning the steady-state portion of the model are available elsewhere. 6 However, several points should be reviewed here. These include the effects of the prestimulation WOR and GOR, the importance of backpressuring the well early in the producing cycle, and the consequences of taking into account the effect upon subsequent cycles of heat lost to the formations surrounding the heated zone. As noted by Boberg and Lantz, both high WOR's and high GOR's have a detrimental effect on stimulated performance. The detrimental effect of a high pre stimulation WOR is a consequence of the high heat capacity of the produced water (approximately twice that of oil), while that of a high GOR is due to a large amount of water in the vapor phase, accompanied by a high latent heat of vaporization. In either case, the result is a more rapid decline in reservoir temperature because of the high rate of thermal energy removal. High rates of energy removal also result if produced water is allowed to flash to steam in the wellbore. Backpressuring of the well early in the producing cycle can prevent or minimize such flashing and avoid the loss of large quantities of energy. As a DECEMBER, 1969
result, more incremental oil will be obtained than would be the case if drawdown were maximum throughout the entire cycle. In estimating the performance of succeeding cycles, two points should be kept in mind. 1. The possible effect of heat lost to formations surrounding the heated zone is neglected. Consequently, it might be thought that estimates of production rates for cycles subsequent to the first would be conservative since under this assumption calculated heat losses for cycles after the first would be higher than the actual heat losses. However, Closmann8 has shown that, for the injection times that normally pertain in the cyclic steam injection process, the size of the steam zone in a preheated reservoir is dependent primarily upon the amount of heat left in the heated zone, and is virtually independent of the temperature existing in the overburden and underburden. In CIosmann's analysis, it is assumed that the temperature of the injection zone prior to steam injection is at some uniform value Tavg that is less than T s , but greater than or equal to Tn and that it has this value to an indefinite extent in the radial direction in the reservoir. In this study, however, Tav~ is assumed to exist only out to the heated radius of the reservoir. 2. The production response model is to be used in conjunction with dynamic programming, and hence the response not only from adjacent stages within a cycle but also from different cycles must be separable. Provided that the optimization problem is formulated as an "initial state problem"9 in which the multistage analysis is carried out in the "backward" direction, the separability of adjacent stages within a given cycle is readily apparent. However, the separability of adjacent cycles is not so obvious, since once the reservoir is restimulated, the same system no longer exists. Nevertheless, if the backward formulation is used, and if there is a transformation function relating the state of the reservoir at the beginning of a given cycle to that at the end of the previous cycle, decomposition can be achieved. This follows directly from the fact that in the backward formulation, the decision to stimulate affects only the current and following stages in time, and in no way affects the previous cycle. Moreover, knowledge of the state of the reservoir at the end of the previous cycle, together with the appropriate transformation function, makes it possible to predict the state at the beginning of the subsequent cycle. Further details concerning these aspects of the production response model are given in the section on dynamic programming and in Appendix A.
Pressure History During the Soak Period The development of a model capable of simulating the period of transient flow presupposes a knowledge not only of the pressure distribution immediately following injection, but also of how this distribution changes during the soak period. An incomplete understanding of the physics of the steam soak process precludes an exact description of the pressure history of the reservoir. However, the formulation of several assumptions has made possible the development of a simplified model of the pressure history during the 1583
soak period. Aside from the usual assumptions, these include the following: 1. The pressure throughout the hot zone is constant and equal to the saturated vapor pressure of water at Tavg until such time as the vapor pressure falls to the pressure existing in the reservoir prior to stimulation. From that point, the pressure of the heated region remains constant and equal to Pe. 2. The pressure distribution existing within the cold portion of the reservoir immediately subsequent to injection can be approximated by a logarithmic distribution varying between rh, where the pressure is postulated to be the same as that in the heated region, and ret, where the pressure is assumed to be the same as that which existed prior to stimulation. 3. During the soak period, the pressure distribution within the cold region can be estimated by superimposing on the original distribution the daily drops in pressure that occur at ril because of energy conducted from the hot region. The validity of the first assumption is debatable; nevertheless, it is considered to be reasonable, when argued on the basis of the magnitude of the Prandtl number for the system. Physically, this number is a measure of the relative speeds at which momentum and thermal energy are transferred through the system. For the fluids saturating both the heated and the cold region, the kinematic viscosity is considerably greater than the thermal diffusivity (calculated on the basis of a weighted average of both fluids and rock), and consequently momentum travels more rapidly through the system than does thermal energy. That is to say, pressure will change more rapidly than temperature, and pressure in the cold region should tend to drop more rapidly than the vapor pressure of the steam at Tav~' It does not seem likely that a discontinuity in pressure will occur at rho Therefore, it seems reasonable to suppose that the pressure existing at rh is that of the vapor pressure of water at Tavg. When the vapor pressure of the water saturating the heated region falls below Pe, the pressure in the cold region can no longer decrease. Hence, the steam in the hot region will likely condense at this point (if not before), resulting in an influx of cold oil into the hot region; and thus the pressure existing at rh will tend to remain near Pe. Alternatively, if we assume that the pressure at rll immediately drops to p" we find that although this will result in changes in the shape of the pressuretime curve, it will not significantly affect the average pressure under the curve. Therefore, the hypothesis that the pressure at rh is equal to that of the vapor pressure of water at Tavg is proposed since it seems to be the more reasonable assumption. Given the foregoing assumptions, the pressure distribution (in the cold region) at any time during the soak period can be calculated by _
P( r, t8 ) -
~p(1)
[1 - In (r/rh)] F(O)
I
L K=!
~P
(K
+
1) [1 - In (r/rh)] F(l+l-K) (1)
1584
The volumetric average pressure existing in the reservoir at the end of the soak period, Pas. can then be estimated by
+1
ret
Pas =
r
et
P (r, ts) rdr
(2)
rh
This integral can be evaluated using any of the standard numerical integration techniques. In this study, it was evaluated using Simpson's % rule. A more detailed description of the steps leading to Eq. 1 is given in Appendix B. Transient Flow Model
The model used to predict the amount and rate of transient flow is based on that developed by van Everdingen and Hurst for determining aquifer water movement into a reservoir. It might be supposed that the use of this solution limits the usefulness of the proposed model to reservoirs in which only one fluid flows, but this is not so. The results of Weller10 and the theoretical analysis of Martinl l indicate that single-fluid methods are applicable (at least approximately) in the multiple-fluid case, provided that the total compressibility and total mobility factors are used instead of the single-fluid quantities. The transient model is concerned only with flow in the cold portion of the reservoir; but, since the volume of the cold portion is so much greater than that of the hot zone, it seems unlikely that changes in saturations and properties of fluids in the hot zone can have any great effect on fluid flow in the cold zone. However, as time goes on, more and more heat will be transferred to the cold zone, and this could have a significant effect on cold zone flow rates, as small decreases in viscosity can cause significant increases in rate. In order that the van Everdingen and Hurst model be applicable here, several additional assumptions are made. 1. The actual pressure distribution within the reservoir can be replaced by the volumetric average of the reservoir pressure. 2. The radial volumetric flow rate is equal at all points within and on the boundary of the hot zone. There is no flow at the drainage boundary of the formation. 3. The pressure at rh can be estimated on the basis of the steady-state pressure drop occurring between rh and rw. These assumptions lead to the following equation for calculating cumulative transient production:
K=!
The rate of transient oil production can then be determined from (4)
Since qoht must be known in order to calculate Ne(J), the solution must be obtained using an iteration JOURNAL OF PETROLEUM TECHNOLOGY
scheme. This presents no further complication since the solution for the steady-state rate is also obtained iteratively. (See Appendix C for the development of Eq.3.) Normally, a well experiences a temporary increase in WOR following steam injection. To take this into account, the oil production during stimulated production has been modified as follows: qoh =
+ Rwo) + R",
qoht (1
1
(5)
where Rw=
1 at (Wp/Wi)
+ a + R",o
(6)
2
To determine how realistically the proposed model simulates the steam soak process, predicted results have been compared with an actual field test. This comparison is given in Fig. 1. The field example used to test the model is one selected from McLaren and Price's paper. 12 (The data used in making the prediction are presented in Table 1.) Assumed values have been assigned to those parameters for which no information was given in their paper. Note that the calculated curves are matches of the observed data, rather than true predictions, since the WOR curves have been read in as input data.
made whether or not to stimulate. To evaluate the "no stimulation" possibility, the return from the nth stage is added to that of the n - 1 remaining stages. In tum, the return from the n - 1 remaining stages is dependent upon the decisions made during each of these subsequent stages. Dynamic programming is a mathematical tool, based on the notion of recursion, which can be used to optimize mathematical representations of processes. This technique. developed by Richard E. Bellman's during the early 1950's, is particularly well adapted to the optimization of multistage decision processes. It is characterized by a criterion function to be satisfied, a multiplicity of stages in time and space, and decisions to be made at each stage to satisfy the criterion function. A wide variety of both management and scientific problems can be solved by dynamic programming. In particular, this method provides a uniform approach to a class of problems known generally as replacement problems. In this type of problem, the productivity and efficiency of materials decreases with time, making the expense of using them great enough to warrant their replacement. Typical of this class are (1) the equipment replacement problem in which the
TABLE I-DATA USED IN SIMULATION OF FIELD TEST
Dynamic Programming Model Since the objective is now to optimize the steam soak process for maximum net profit before taxes (hereafter referred to simply as "profit"), attention must be given not only to how, but also to when the reservoir should be restimulated. Thus, at each stage in the life of a project, the operator must decide whether the reservoir should be shut in and restimulated, or continue to be produced. If he decides to restimulate, he must make decisions about the amount of steam to be injected, the temperature, pressure, and rate of injection, and so forth. All of these decisions should be made in the light of the entire process, rather than of each particular stage, since any action taken will affect the project over the remainder of its life. For instance, at any given stage n, a decision must be
120
-
~
rd. Similar equations can be written for the case in which the reservoir is bounded. Determination of Oil Production Rate
15b 6
-2 -16-64
1024 - ...
+ In(rh/rw)
Sr
exp( -b"y2)J2l(y)ydy
The stimulated oil production rate, qoh, can be calculated using _ JJc(Pe - Pw) (1 + Rwo) qoh (1 + Rw)
(A-20)
where, for the purposes of the present study, (A-21)
- 8) - 8J,
NNv) Peo pp
(A-l 2) where
(A-22)
Estimation of Performance of Succeeding Cycles
(A-13)
At the beginning of any step in time, the heat remaining in the oil sand can be estimated by
Because of simplifying assumptions made in deriving Eq. A-12, Tavg can become less than T r • When this occurs, Tayg should be set equal to T r • The rate of thermal energy removal, H" is defined by
(A-23)
Ii
(A-14) where Hog = [5.61(pC)o
If, at that time, it is decided to restimulate the reservoir, the performance of the succeeding cycle can be estimated by adding Qr to the heat injected and assuming that the steam is injected into a reservoir at its original temperature, T r • APPENDIX B
+ RgCgJ (Tavg
- T r),
Derivation of Equations for Soak Period (A-15) (A-16)
RlOv = 0.0001356(
PlOV )R g Pw - Pwv
,
(A-17)
if Pw > Pwv and Rwv < Rw. If Pw < Pwv and/or if Rwv (as calculated by Eq. A-17) > Rw. Rwv = Rw. Because H f is a function of Tavg , Eq. A-12 must be solved in a step-wise manner. Moreover, if rather large steps in time are used, it seems advisable to average Ta,-g over time as well as distance. In this study, the mean temperature (arithmetically averaged over time) is used in calculating the amount of oil produced during a given step in time. Steady-State Approximation of the Productivity Index
The ratio of stimulated to unstimulated productivity indices, J, can be approximated by
Temperature History of Heated Region During Soak Period
Normally, a well is shut in and allowed to soak for a specified period of time before being put back on production. During this period of time, the temperature of the heated region declines. The sole mechanism of temperature decay is thermal conduction, since no fluid convection takes place while the well is soaking. A simplified form of Eq. A-12 can be used, therefore, to describe the temperature history of the heated region during this stage of the process. Setting HI equal to zero gives Tayg = Tr
+ (T.
- Tr)Vrvz
(B-l)
Pressure History of the Reservoir
When fluid is injected into a well, a pressure transient travels outward into the reservoir. Moreover, this transient may be closely represented by a series of steady-state incompressible fluid pressure distributions, as given by Eq. B-2, in which the pressured radius takes on increasingly larger values.
(A-18) (B-2) For the case where the skin factor S is reduced to Sr following stimulation, DECEMBER, 1969
The value of the pressured radius at any time after the start of injection may be approximated by 1595
ret =
2 -V 1]ti
(B-3)
During each day the well is allowed to soak, the temperature of the heated region declines. The extent of the decline can be predicted by Eq. B-l. Since it is assumed that the pressure of the heated region is that of the vapor pressure of steam at Tavg , the daily pressure drop can also be predicted. The effect which these daily drops in pressure have on the cold region pressure distribution can be estimated by the method of superposition. It should be noted that the pressure transient developed during the injection period continues to move outward into the reservoir during the soak period. Thus, the pressure distribution (in the cold region) at any time during the soak period is given by _
p ( r, t8 ) -
~p(l)
[1 - In(r/rh)] F(O)
I
~p(K
K:!
+ 1) [1 + 1-
F(/
In(r/rh)]
K)
,
Assuming that the steady-state flow equations apply, the pressure drop between rh and r w can be calculated using (C-2)
Cumulative transient production can then be calculated by means of a modified form of the van Everdingen and Hurst 7 equation: Ne(J) =
B[~paCO)QtD(1) J
-
1: ~paCK)QtD(1 - K
+ 1)]
(C-3)
K=!
where ~pa(O) =
Pas
~Pd(K) =
paCK
Q (1)
(C-4)
,
= rD -
+ 1) 1_ 2
2
tD
(C-5)
paCK) ,
n:! 00
exp( -antD)Jl(anrD) a 2 n[J2 o(a n) - J2 l (a nrD)]
(B-4)
(C-6)
where ~p(K
+
1) = Pwv(K) - Pwv(K
+
1)
F(O) = In(retlrh)
(B-6)
F(K) = In(rts/rh)
(B-7)
rts = 2Y'fJts
(B-8)
tD =
6.323(10-3)kt
(C-8)
2'
