Hemodynamic Control using Direct Model Reference Adaptive Control

European Journal of Control (2005)11:558–571 # 2005 EUCA

Hemodynamic Control using Direct Model Reference Adaptive Control – Experimental Results Cesar C. Palerm and B. Wayne Bequette Department of Chemical and Biological Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180-3590, USA

Regulation of hemodynamic parameters is a challenging problem, due to inter- and intra-patient variability, drug interactions, evolving patient condition, and in the case of simultaneous control of mean arterial pressure and cardiac output, tight coupling between the two variables. This requires a controller that can adapt to conditions as they evolve, and without the need to tune the controller for each individual patient. To address these problems, direct model reference adaptive control (DMRAC) is extended by incorporating mechanisms to handle rate and saturation constraints, allowing a range in the value of a regulated variable instead of fixed setpoint tracking, and a design method that directly considers time delay uncertainty. In this paper we present experimental results using extended DMRAC on canines, for a variety of conditions. These results demonstrate the feasibility of using DMRAC, with the advantage of not requiring prior model identification for the individual patient. Keywords: Biomedical Control Systems; Hemodynamic Regulation; Canine Experiments; Drug Infusion; Adaptive Control

1. Introduction Control systems theory has been extending into many fields, from the household thermostat to complex controllers for flight dynamics of fighter aircraft. Medicine is not an exception, although the progress E-mail: [email protected]

has been slow in some cases due to particular challenges encountered by the inherent complexity of biological systems. Whereas Westenskow [37] seems to attribute the lack of actual clinical use of such systems mostly to ‘‘our natural distrust of assigning such delicate tasks to a machine,’’ Packer [26], in his review of closed-loop control in patient care, reflects the growing realization in the late 1980s of the difficulties presented by such applications. Still, after more than three decades of research in this field, there is little to point to in the actual clinical environment today. The main motivation of this research is to give this area another push towards this goal, in particular hemodynamic regulation. Critical care patients such as those in intensive care or undergoing surgery require close monitoring of all their vital signs. The anesthesiologist or critical care physician must monitor and regulate a wide range of physiological states such as mean arterial pressure (MAP), cardiac output (CO) (amount of blood pumped by the heart each minute), carbon dioxide and oxygen levels, blood acidity, fluid levels, heart contractility, renal function and more. Unfortunately, many physiological states cannot be measured directly and must be inferred by the physician. Physicians maintain patient states within acceptable operating ranges by infusing different drugs and intravenous fluids. In addition, during surgical procedures physicians must administer anesthetics and monitor the depth of anesthesia. Current clinical practice involves manual regulation of drip IV lines to Received 12 June 2004; Accepted 11 July 2005. Recommended by G. Dumont and D.W. Clarke.

Hemodynamic Control using DMRAC

infuse drugs. Programmable pumps are also used to either deliver the drugs at a constant rate or a variable rate to achieve a desired concentration. Control of such pumps is based on averaged pharmacokinetic data and is essentially open-loop, requiring regular intervention by the attending physician or nurse to adjust the drug flow rates. It is desirable to have an automated system that closes the loop on primary variables, but monitors secondary variables and helps the physician perform diagnosis. This would allow the physician to spend more time monitoring the patient for conditions that are not easily measured and assure that the physician is always ‘‘in the loop.’’ The physician would use her expertise to diagnose the patient, specify setpoints or ranges of values for the states to be regulated, choose the drugs best suited to obtain the objective and mandate permissible infusion rates; this information would then be explicitly used by the controller to automate the regulation of physiological states. In the case of patients with congestive heart failure (CHF) the measured variables that are of primary importance are MAP and CO. Secondary variables that are monitored, but not regulated as tightly as the primary variables, include heart rate and pulmonary capillary wedge pressure. Regulation of MAP and CO is a challenging problem, as the two are tightly coupled, and modifying one will affect the other. Although there seem to be no clinical trials for this specific application, other closely related areas can give us an idea of the potential benefits. Chitwood et al. [6] present the results of a multicenter study of automated drug infusion for postoperative hypertension. They enrolled 1089 patients, of whom 51% were treated using closed-loop titration while the rest received the standard treatment with a nurse manually adjusting the infusion rate. They considered multiple end points, including length of intensive care unit (ICU) stay, frequency and amount of blood products transfused and the number of hypertensive and hypotensive episodes. For the automated group they found a significant reduction in the number of hypertensive episodes per patient and a slight reduction in the number of hypotensive ones. Percentage of patients receiving transfusions, and the total amount transfused, were also lower in the automated group. Although not evaluated, they suggest that such a system could improve overall patient outcome, in part by reducing the time and stress loads on the nursing staff. Obviously, a shorter ICU stay and lower needs for transfusions also translate into economic savings. Around the same time, two other groups [3,22] published results of clinical evaluations of the same

559

system. They concur that the automated infusion is superior to manual titration. More recently Hoeksel and Blom [11] present the results of a trial involving 160 patients, comparing manual and computer control of systemic hypertension during cardiac surgery. Their conclusion is that closed-loop control significantly improves hemodynamic stability. At this point though, there has been no clinical trial with the objective of evaluating the impact of such a system on patient outcome. The vast amount of research on blood pressure control was initiated by Slate et al. [34] who used a proportional-integral-derivative (PID) controller with empirical tuning rules to control MAP using sodium nitroprusside (SNP). Many implemented such controllers and tested them in clinical environments involving postoperative care [3,7,22,31,32]. Since then, more complex control schemes have been used in the automation of hemodynamic regulation. Isaka and Sebald [14] provide a comprehensive review of the single-input single-output (SISO) system controlling MAP with SNP; they highlight the still unresolved issues of safety and robustness, which are even more important when dealing with multiple drugs to control more than one output. Martin et al. [21] and Kwok et al. [19] have used adaptive control methodologies for blood pressure regulation during surgery. In the specific application of cardiac surgery some of the more thoroughly clinically-tested strategies use a combination of proportional-plus-integral (PI) controllers and a supervisory, rule-based, expert system. [4,11–13,23] Additional research efforts have been made in the simultaneous regulation of MAP and CO. Serna et al. [33] reported on the simultaneous control of CO and MAP using dopamine (DP) and SNP. As the CO measurements were available at a rate much slower than MAP, they decoupled the DP–CO loop from the MAP–SNP loop. Voss et al. [36] used the control advance moving average controller to regulate MAP and CO in canine experiments. Yu et al. [39] used a multiple model adaptive control approach for regulating MAP and CO in canine experiments. Held and Roy [9,10] developed an expert system that used a fuzzy controller for controlling MAP and CO using SNP and DP. The application of direct model reference-adaptive control (DMRAC) to the simultaneous regulation of MAP and CO is presented in this paper, focusing on experimental results. In work done in parallel in our group, Rao et al. [29] present the application of a strategy that uses a model predictive control (MPC) framework and adaptation using large sets of simple first-order plus time-delay (FOTD) models.

560

2. System Overview The main motivation for this research is the particular need for blood pressure and CO regulation for patients with CHF. Usually the physician requires that MAP be kept within a certain range, or at a given target. CO must be kept above a minimum as well, in order to guarantee adequate perfusion of all organs. It is important to highlight this last requirement, as due to the tight coupling, it is nearly impossible to achieve perfect setpoint tracking for MAP and cardiac output simultaneously (in many cases it could even be an infeasible problem). Relaxing the condition for CO from maintaining a specific target to keeping it above a minimum (in essence within a defined range) makes this a manageable problem. In Frankel and Fifer [8] (congestive) heart failure is defined as the inability of the heart to pump blood at a sufficient rate in order to meet the body’s metabolic demands, or the ability to do so only when the cardiac pressures are abnormally high, or both. Heart failure can result from diverse forms of cardiac disease. There are two main categorizations of heart failure: acute and chronic. Acute heart failure usually stems from myocardial ischemia, infarction or mitral valvular dysfunction. It is generally treated in an emergency care unit and the initial management includes the administration of pharmacologic agents for hemodynamic stabilization and oxygen administration. The underlying cause is often a reparable condition or lesion [20]. Chronic heart failure most commonly results from impaired left ventricular function, and is the focus of the models developed in this area (described later). Due to the aging population and the availability of interventions that significantly prolong survival after acute cardiac insults, the incidence of heart failure is increasing in the United States [8]. For a normal heart, CO follows the Frank–Starling relationship. In summary, reducing afterload (the pressure the heart must overcome in order to get blood out of the heart and flowing, i.e. MAP) increases stroke volume (amount of blood expelled in one heart contraction) and therefore the CO. Increasing the preload (the amount of blood returning to the heart, venous return) leads to increased stretching and a stronger contraction. A reduction in CO results from both an increase in afterload and a decrease in preload. In the case of CHF, the heart is unable to pump all the blood in the venous return. An increase in preload only decreases the efficiency of the heart and leads to greater congestion, in direct opposition to what is expected for a normal heart.

C.C. Palerm and B.W. Bequette

The venous return is dependent on venous compliance, which is the ease with which the veins can stretch like a balloon. The greater the venous compliance is, the larger the amount of blood that is retained in the veins, and the lower the venous return to the heart. In this study three drugs are used. Phenylephrine (PNP) is used to increase blood pressure through arterial vasoconstriction (which increases the systemic vascular resistance). SNP is used to lower blood pressure, as it reduces the venous compliance and the systemic vascular resistance through arterial vasodilation. For CHF, it is an ideal drug since it decreases both the preload and afterload, so the heart has less blood to pump at a lower resistance. DP is used in its inotropic range to enhance cardiac performance by improving heart contractility. The beneficial effects of the SNP–DP combined therapy in CHF for humans are described by Miller et al. [24] and Stemple et al. [35]. Safety constraints for the controller translate in part to limits on the infusion rates of these drugs. Saturation limits must be handled, as drugs can be infused but not removed from the bloodstream. Upper limits are also necessary to prevent overdosing and toxic effects. For example, SNP breaks down into cyanide, thus a maximum rate of 10 mg/(kg  min) is used to prevent accumulation of this toxin [15]. In the case of DP the inotropic range used for alleviating CHF is from 2 to 7 mg/(kg  min)1. The sensitivity to these drugs varies from patient to patient, as well as for the same patient over time. In the case of SNP, there can be as much as two orders of magnitude difference. The recommendation is that infusion start at 0.25–0.3 mg/(kg  min) and gradually titrated upward every few minutes until adequate blood pressure control is achieved [1]. The limits on dosing remain as outlined above, as toxicity remains, regardless of the sensitivity to the drug as related to blood pressure. For the application to patients undergoing surgery the effects of the chosen anesthetic are also important. These have direct consequences in the animal experiments conducted to test the controller performance. In our study, two volatile anesthetic agents are used, isoflurane (Forane1) or fluothane (Halothane1) combined with 50% nitrous oxide. Testing with different anesthetics is important, as the dynamics of the system will change depending on the agent used. Both anesthetics will decrease the systemic vascular resistance, thus dropping blood pressure. Fluothane also depresses the heart muscle, thus decreasing the heart 1

This range is for dogs, in humans the range is 4–7 mg/(kg  min).

561

Hemodynamic Control using DMRAC

contractility; it is this characteristic that is used in our experimental protocol to mimic CHF. Kawasaki et al. [18] show that in dogs the effect of fluothane depresses the left ventricular contractility more than the arterial system. The baroreceptor reflex is also affected by each anesthetic in different ways. Under isoflurane the baroreflex remains strong, thus the system is capable of counteracting the controller if the desired MAP is too high or too low compared with normal body function. With fluothane, the baroreceptor reflex is affected to the point that it no longer has a significant effect on the blood pressure. Modeling is always a challenging problem. Whatever model is used must capture enough of the dynamics of the system to be useful, but be simple enough to be practical. In the case of the cardiovascular system we have two models. The first is a simple, linear first order plus dead time model; the second one is a complex nonlinear model. Yu et al. [39] modelled the hemodynamic system by a two-input two-output first order system with delays. They proposed the following equation to represent the plant model: 2 3 K11 eT11 s K12 eT12 s     6 11 s þ 1 DP CO 12 s þ 1 7 6 7 , ¼4 MAP K21 eT21 s K22 eT22 s 5 SNP 21 s þ 1 22 s þ 1 ð1Þ with the gains Kij representing the patient sensitivity to the drug,  ij the corresponding time constant and Tij the corresponding time delay between drug infusion and the response of the system. The infusion rates are given in mg/(kg  min), MAP is in mmHg and CO is in ml/(min  kg). Typical values and ranges are shown in Table 1. This is the model used for the design of the controller in this work. Yu et al. [38] also developed a complex nonlinear Table 1. Nominal values and ranges of model parameters. Parameter

Typical

Range

Units

K11  11 T11 K12  12 T12 K21  21 T21 K22  22 T22

5 300 60 12 150 50 3 40 60 15 40 50

1–12 70–600 15–60 15–25 70–600 15–60 0–9 30–60 15–60 1–50 30–60 15–60

ml/mg s s ml/mg s s mmHg  kg  min/mg s s mmHg  kg  min/mg s s

model. Its purpose is to approximate the hemodynamic responses of DP and SNP in acute left ventricular pump failure. It is a lumped parameter model which uses an electrical circuit analog to describe the circulatory system. The two variables of interest are MAP and CO. It includes the effects of the baroreflex system as well. This is the model that was used to validate the controller prior to the in vivo experiments.

3. DMRAC The linear time invariant model reference adaptive control problem is considered for the plant x_ ðtÞ ¼ AxðtÞ þ BuðtÞ yðtÞ ¼ CxðtÞ,

ð2Þ

where x(t) is the (n  1) state vector, u(t) is the (m  1) control vector, y(t) is the (q  1) plant output vector, and A, B and C are matrices with appropriate dimensions. The range of the plant parameters is assumed to be known and bounded by aij  aði, jÞ  aij b  bði, jÞ  bij ij

i, j ¼ 1, . . . , n i ¼ 1, . . . , n; j ¼ 1, . . . , m ð3Þ

The objective is to find, without explicit knowledge of A and B, a control u(t) such that the plant output vector y(t) follows the reference model x_ m ðtÞ ¼ Am xm ðtÞ þ Bm rðtÞ ym ðtÞ ¼ Cm xm ðtÞ

ð4Þ

The model incorporates the desired behavior of the plant, but its choice is not restricted. In particular, the order of the plant may be much larger than the order of the reference model. The adaptive control algorithm presented is based upon the command generator tracker (CGT) concept developed by Broussard and O’Brien [5]. In the CGT method, it is assumed that there exists an ideal plant with ideal state and control trajectories, x(t) and u(t) respectively, which corresponds to perfect output tracking (i.e. when yðtÞ ¼ ym ðtÞ8t  0). By definition, this ideal plant satisfies the same dynamics as the real plant, and the ideal plant output is identically equal to the model output. Thus, x_  ðtÞ ¼ Ax ðtÞ þ Bu ðtÞ

8t  0

ð5Þ

and y ðtÞ ¼ ym ðtÞ ¼ Cx ðtÞ ¼ Cm xm ðtÞ:

ð6Þ

562

C.C. Palerm and B.W. Bequette

Hence, when perfect tracking occurs, the real plant trajectories become the ideal plant trajectories, and the real plant output becomes the ideal plant output, which is defined to be the model output. The ideal control law u(t) which generates perfect output tracking and the ideal state trajectories x(t) is assumed to be a linear combination of the model states and model input:       S11 S12 xm ðtÞ x ðtÞ ¼ , ð7Þ S21 S22 u ðtÞ rðtÞ where the Sij submatrices satisfy the following conditions: S11 Am ¼ AS11 þ BS21

ð8aÞ

S11 Bm ¼ AS12 þ BS22

ð8bÞ

Cm ¼ CS11 0 ¼ CS12 :

ð10Þ

uðtÞ ¼ Ke ðtÞ½ ym ðtÞ  yðtÞ þ Kx ðtÞxm ðtÞ þ Kr ðtÞrðtÞ ð11Þ where Ke(t), Kx(t), and Kr(t) are adaptive gains and concatenated into the matrix K(t) as follows Kx ðtÞ

Kr ðtÞ

ð12Þ

Defining the vector v(t) as 3 ym ðtÞ  yðtÞ 6 7 xm ðtÞ vðtÞ ¼ 4 5,

2

ð13Þ

rðtÞ the control u(t) is written in a compact form as follows uðtÞ ¼ KðtÞvðtÞ:

ð15bÞ

K_ I ðtÞ ¼ ½ ym ðtÞ  yðtÞvT ðtÞT,

T > 0:

ð15cÞ

The sufficiency conditions for asymptotic tracking are (1) There exists a solution to the CGT problem (Eq. 8) (2) The plant is ASPR; this is, there exists a positive definite constant gain matrix KE, not needed for implementation, such that the closed loop transfer function

is strictly positive real (SPR).

Then the adaptive control law based on this CGT approach is given as [16]

KðtÞ ¼ ½Ke ðtÞ

T 0

ð8dÞ

ð9Þ

ð14Þ

ð15aÞ

KP ðtÞ ¼ ½ ym ðtÞ  yðtÞvT ðtÞT ,

Gcl ðsÞ ¼ ½I þ GðsÞKE 1 GðsÞ

If when perfect output tracking does not occur, y(t) 6¼ ym(t), asymptotic tracking is achievable provided stabilizing output feedback is included in the control law uðtÞ ¼ S21 xm ðtÞ þ S22 rðtÞ þ Ke ð ym ðtÞ  yðtÞÞ:

KðtÞ ¼ KP ðtÞ þ KI ðtÞ

ð8cÞ

In summary, when perfect output tracking occurs, x(t) ¼ x(t), and the ideal control is given by u ðtÞ ¼ S21 xm ðtÞ þ S22 rðtÞ:

The adaptive gains are obtained as a combination of an integral gain and a proportional gain as shown below [16]

ð16Þ

In general, the ASPR condition is not satisfied by most real systems. Bar-Kana and Kaufman [2] have shown that a non-ASPR plant of the form G(s) ¼ C(sI  A)  1B can be augmented with a feedforward compensator H(s) such that the augmented plant transfer function Ga ðsÞ ¼ Gp ðsÞ þ HðsÞ

ð17Þ

is ASPR. However, the resulting adaptive controller will in general result in a model following error that is bounded but not zero in steady state. To eliminate this problem, a modification that incorporates the supplementary feedforward into the reference model output as well as the plant output has been developed by Kaufman and Neat [17]. This configuration is shown in Fig. 1. Since many applications deal with plants that are not ASPR, we need a way to design the required feedforward compensator. In Kaufman et al. [16, Chapter 6] such a procedure is outlined for SISO and MIMO plants, from both time and frequency domain point of view. The extension of the design method to directly include uncertainty in the time delays and the derivation of the feedforward compensator used in this study, together with the corresponding simulation results can be found in Ref. [25] and Ref. [27]. One of the main drawbacks of the standard DMRAC algorithm is its inability to handle input constraints, which is critical in the case of drug infusion control for safety reasons. The controller must be able to handle saturation limits, as no drug can be removed from the bloodstream; an upper bound is also set to avoid overdosing and toxic side effects.

563

Hemodynamic Control using DMRAC

Fig. 1. DMRAC with plant and reference model feedforward.

To this end, we expanded the algorithm in order to handle such cases. In Ref. [28] we present the details, but we outline the changes to the algorithm below, as they were used in this study. Whenever the command saturates, the gains keep changing according to the dynamics of (15). If ym(t)  y(t) 6¼ 0, the gains will keep increasing or decreasing, even though these changes are not having any effect on the system’s output. A solution is to leak the integral part (15c) to minimize (or eliminate) the windup. In this case the adaptation law is changed to KðtÞ ¼ KP ðtÞ þ KI ðtÞ

ð18aÞ

KP ðtÞ ¼ ½ym ðtÞ  yðtÞvT ðtÞT,

T  0 ð18bÞ

K_ I ðtÞ ¼ ½ym ðtÞ  yðtÞvT ðtÞT  ðtÞKI ðtÞ, T > 0: ð18cÞ The (t) term is added to leak the gains when the saturation constraints are hit. A basic form for this function is

with

_ ðtÞ ¼ K1 juðtÞ  usat ðuðtÞÞj  K2 ðtÞ

ð19Þ

8 > < ulb usat ðuðtÞÞ ¼ uðtÞ > : uub

ð20Þ

for uðtÞ < ulb for ulb  uðtÞ  uub for uðtÞ > uub

where K1 > 0 and K2 > 0 are tuning parameters, and ulb and uub are the lower and upper saturation limits respectively. Thus, when the command is within the

saturation limits (t) ! 0 and increases when they are violated. For the MIMO case (t) can be defined as a diagonal matrix with elements _ ii ðtÞ ¼ K1 jui ðtÞ  uisat ðui ðtÞÞj  K2 ii ðtÞ:

ð21Þ

Rate limiting will also result in windup of the adaptation gains if they are violated for an extended period of time. A cause of fast changes in the control input is related directly to the adaptation gains T and T. If these gains are too large, then the controller gains will react too aggressively to the model reference dynamics and the error between the model and plant outputs. Tuning of these gains is done off-line, for what would be expected to be nominal system dynamics. In cases such as drug infusion control, there is really no ‘‘nominal system’’ that we can talk about to start with. Thus this leads to the dilemma if we should make the controller gains change very slowly at the cost of lower performance. A possible solution is to allow these gains to vary on-line. One solution to this is to not allow the adaptation gains to change directly, but to introduce an additional variable. If we modify the adaptation law as follows: KðtÞ ¼ KP ðtÞ þ KI ðtÞ

ð22aÞ

KP ðtÞ ¼ ½ym ðtÞ  yðtÞv ðtÞðtÞT ,

T 0

ð22bÞ

K_ I ðtÞ ¼ ½ ym ðtÞ  yðtÞvT ðtÞðtÞT,

T > 0,

ð22cÞ

T

564

C.C. Palerm and B.W. Bequette

where (t) is a diagonal matrix with elements 0 < ii(t)  1. Then, as long as the rate limits are not being violated, we can keep ii(t) ¼ 1, thus maintaining the adaptation gains originally selected. But if the rate limits are violated, then (t) decreases in proportion to the magnitude of the violation, therefore slowing down the adaptation. Preserving the directionality of the control signal in MIMO systems is important, particularly when the system is tightly coupled. This is quite simple to accomplish, and just consists of scaling all control inputs proportionally, using the most severely saturated signal as the base. For each control signal we can find a constant %, 0  %  1 such that uisat ¼ %i ui

ð23Þ

then scale the control input vector using usat ¼ minð%i Þu

ð24Þ

There is also a need with the drug infusion control problem to handle a range of allowable output values instead of forcing a specific setpoint; this is the case in particular for CO. DMRAC is based on the CGT structure, which means that we have a reference model output following scheme. Thus we need to modify the structure of the controller in a way that this reference model will provide us with an allowable range instead of a fixed setpoint. The easiest way to accomplish this is to introduce a dead-zone in the error signal as follows 8 > < ymin  yðtÞ if yðtÞ < ymin , if ymin  yðtÞ  ymax ð25Þ ey ðtÞ ¼ 0 > : ymax  yðtÞ if yðtÞ > ymax , where ymin and ymax are, respectively, the lower and upper bounds in which we want to maintain the plant output y(t). To preserve the dynamics of the reference model for these responses, we can set the values of the bounds as deviations from the output of the reference model as ymin ðtÞ ¼ ym ðtÞ  min

ð26Þ

ymax ðtÞ ¼ ym ðtÞ þ max

ð27Þ

For the specific case of drug infusion control, it is best if the total amount of drug infused is kept to a minimum. With the above implementation, particularly if we let ymax ! 1, we can stay within the bounds with a wide range of drug infusions, thus a way to minimize the control action is warranted. This can be easily accomplished by leaking the integral term in the

adaptation law as follows KðtÞ ¼ KP ðtÞ þ KI ðtÞ KP ðtÞ ¼ ½ ym ðtÞ  yðtÞvT ðtÞT ,

ð28aÞ T  0 ð28bÞ

K_ I ðtÞ ¼ ½ ym ðtÞ  yðtÞvT ðtÞT  b KI ðtÞ, T > 0 ð28cÞ where b is a small, positive term. Once the plant output approaches a bound, this term can be set to zero to stop the leaking. Even though all of the above theory for DMRAC is based on continuous time systems, digital implementation is straightforward. For the hemodynamic control problem the sample time of the system is 30 s. The reference model is integrated numerically between the sample times. The same goes for the feedforward dynamics. The integral part of the gain update is also computed at this frequency, using simple Euler type integration. Measurements are kept constant between sampling times.

4. Experimental Setup The testing of the controller was done on four mongrel dogs under an IACUC approved protocol. The experiments were performed on 11 experiment days with 3–5 runs each day. A schematic diagram of the experimental setup is shown in Fig. 2. Following induction of a surgical plane of anesthesia with sodium pentobarbithal, the animal was intubated and mechanically ventilated (Siemens– Elena 900C Servo–Ventilator) with isoflurane (Forane1) or fluothane (Halothane1) with 50% Nitrous oxide anesthetic. An arterial line was placed in the femoral artery to provide continuous arterial pressure measurement through a Mennen Horizon monitor. A Swan–Ganz catheter (Baxter Edwards Swan–Ganz Intellicath CCO/VIP Thermodilution), connected to a Baxter Vigilance monitor, was introduced in the pulmonary arterial tree to provide continuous CO measurement. Control calculations were performed on a Dell Pentium II PC running a custom built Windows-based GUI. The pressure and flow measurements were received from the monitors through RS-232 ports. The control loop was closed with rotary infusion pumps (Critikon Simplicity 2100A) modified to accept digital inputs via a digital output card. The sampling time for the controller was set at 30 s. The canines were pharmacologically altered to exhibit hyper- or hypotension and depressed CO. Both SNP and PNP were used to affect the baseline

565

Hemodynamic Control using DMRAC

Fig. 2. Schematic diagram of the experimental setup [30].

blood pressure. High levels of fluothane were used to significantly depress heart contractility and thus mimic CHF.

5. Results Previous to the dog experiments, simulations were done using linear [39] and nonlinear [38] models of the hemodynamic system. The controller was tested on mongrel dogs, using different drug combinations and different anesthetics. The results below are presented in chronological order. 5.1. Hypotensive Canine Under Isoflurane This run is on a 19 kg female, using isoflurane as the anesthetic agent. For these runs the (t) parameter is active but (t) is not, allowing (t) to handle both rate and saturation limits. The results of this run (shown in Fig. 3) is for MAP regulation using PNP as the manipulated input. The canine starts being slightly hypotensive, with an MAP of 67 mmHg. The MAP setpoint is started at 75 mmHg. The controller’s action of infusing PNP actually relaxes the baroreflex which was trying to increase MAP, as can be seen from the decrease in the heart rate. After 20 min the infusion rate of PNP required to maintain the desired setpoint is practically zero, thus at t ¼ 22.6 min the MAP setpoint is increased to 85 mmHg. It is increased once more at t ¼ 31.2 min to 100 mmHg. In all cases it takes the controller 4 min to bring the MAP to within 5% of the setpoint. Overall the controller does a very good job in regulating MAP.

Note that the controller has to keep adjusting the infusion rate of PNP constantly to maintain the MAP at the setpoint, which clearly shows the disadvantage of maintaining hemodynamic variables by manual adjustment. 5.2. Hypertensive Canine Under Isoflurane This run is on the same day and canine as the one in Section 5.1. In Fig. 4 MAP is being regulated by SNP. It can be seen that the controller does a satisfactory job, even when other disturbances are introduced. Initially the setpoint was set to 80 mmHg, and although the controller is doing a good job, the infusion rate of SNP is very low. At t ¼ 12.7 min the setpoint was dropped to 70 mmHg, and the controller responded by increasing the amount of SNP infused. But the baroreflex (which remains strong under isoflurane) also reacted to this, as can be seen by the sudden increase in heart rate. Thus at t ¼ 16.4 min the setpoint was changed once again to 80 mmHg rather than have the controller fight the baroreflex, which in our experience from other runs would have meant possible SNP infusion rates of close to the upper limit. Instead, the hypertensive tendency of the canine was increased by infusing PNP. At t ¼ 17.7 min the infusion rate for PNP was set to 1 mg/(kg  min), at t ¼ 20 min it was increased again to 1 mg/(kg  min) and at t ¼ 22.3 min it was set at 3 mg/(kg  min). Changes in the concentration of isoflurane administered were another source of disturbances. These changes are indicated by the arrows in the MAP plot. At t ¼ 30 min the container for the anesthetic had to

566

C.C. Palerm and B.W. Bequette

Fig. 3. SISO Run 1: MAP control of a hypotensive canine using PNP [30].

be refilled, which requires turning off the anesthetic. At t ¼ 31 min, the concentration of isoflurane was set back to 1%. Only moments later the canine started to wake up (as attested by the sharp increase in heart rate). The concentration was then set to 5% from t ¼ 31.2 min to t ¼ 32 min, at which time it was brought down to 2% and again at t ¼ 32.2 min to the maintenance level of 1%. The sudden increase in anesthetic concentration to 5% caused the sharp decrease in MAP, at which point the controller started to back down the amount of SNP infused. Once the concentration was brought down to 1% this caused the MAP to swing to the opposite extreme, causing the overshoot around t ¼ 35 min; again, the controller responded by increasing the infusion of SNP. Note that as part of the safety constraints the rate of change in the infusion rate is limited, and thus large changes in SNP infusion are not possible. There is one more large transient disturbance, in this case caused by changes in the infusion of PNP as well as the anesthetic. At t ¼ 45.8 min the infusion rate of PNP was set back to 2 mg/(kg  min), practically at the same time the canine started to shiver (an indication that the animal is once again waking up), and thus at t ¼ 46.2 min the concentration of isoflurane was increased to 3%. At t ¼ 47.9 min isoflurane was set once more at 1%. The controller once more does a good job in handling the disturbance.

5.3. Hypertensive and Low CO Canine Under Fluothane – Run 1 This run is on a 17 kg male, using fluothane with 50% nitrous oxide as the anesthetic agent. The concentration of fluothane is kept at 2% throughout the run. This run and the one in the Section 5.4 use the following extensions to the DMRAC algorithm. When the controller hits rate limit constraints, the ðtÞ term is used. In addition, the adaptation is stopped when a saturation constraint is hit. The tracking for CO is relaxed, using b ¼ 0.005 when within the desired range, and the bounds are fixed. Figure 5 shows the results for this run. The canine starts with an MAP of 78 mmHg, and the MAP setpoint is 60 mmHg. CO is slightly > 90 ml/(kg  min) to start. The controller is engaged 2 min into this run. DP starts infusion at the lower limit of 3 mg/(kg  min). It takes the controller 15 min to bring the CO close to the setpoint. It is interesting to note that around minute 15 the canine’s heart rate starts to increase; this is most likely a secondary effect of the DP infusion which at this point has hit the upper limit of 7 mg/(kg  min). This rise in heart rate tends to raise the pressure, thus the controller has to keep increasing the SNP infusion, and it is clear it is having some trouble keeping good control.

567

Hemodynamic Control using DMRAC

Fig. 4. SISO Run 2: MAP control of a hypertensive canine using SNP, PNP is used to induce hypertension; significant disturbances are due to changes in the anesthetic concentration [30].

Fig. 5. MIMO Run 1: MAP and CO control of a hypertensive canine using SNP and DP.

The CO in the meantime rises very slowly; it is not until minute 30 that it reaches the lower limit for the desired range. After this point it stays within the range until the run is terminated.

There are some sudden changes in the blood pressure around minutes 37, 43 and 47, for which there is no clear explanation. During the last 5 min the dog started to wake up; the increase in heart rate at this

568

C.C. Palerm and B.W. Bequette

Fig. 6. MIMO Run 2: MAP & CO control of a hypertensive canine using SNP and DP.

point is a good indication of this as well. At this point the run was terminated in order to stabilize the animal once more. 5.4. Hypertensive and Low CO Canine Under Fluothane – Run 2 This is the same animal on the same day as the run described in Section 5.3. This run is later in the day and the anesthetic concentration is now at 3% (Fig. 6). Once again, the animal starts out with an MAP slightly > 78 mmHg, with the setpoint at 65 mmHg. CO starts at 130 ml/(kg  min). This time the controller does a better job in controlling MAP. Again, the infusion of DP hits the upper limit. In this run the CO never goes above the lower limit of CO. Again the heart rate goes up slightly, but not as much as the previous run that day, most likely due to the higher anesthetic concentration. At minute 37 a 2 mg/(kg  min) step of PNP is introduced to perturb the arterial pressure. Its effect can be seen 1 min after the infusion is initiated. The controller does a good job in bringing it back down, even though a slightly higher heart rate at this point is also pushing the pressure up. At minute 52 the infusion of PNP is lowered to 1 mg/(kg  min), and then stopped completely at minute 55. The controller is

able to keep the pressure close to the setpoint at this point, most likely aided by the increasing heart rate. Unfortunately, this was the last run of the day, and the run had to be halted in order to wrap up for the day. It is interesting to note, that although there was only one degree of freedom due to DP staying at the saturation point for most of the run, the regulation of MAP remains quite good. From a clinical point of view this is a desirable outcome, as it is more important to maintain MAP under tighter control. Certainly the inability of the controller to get CO within the range would require the intervention of the physician, and either allow greater DP infusion rates, or the concomitant administration of some other drug or intervention. 5.5. Hypotensive and Low CO Canine Under Fluothane – Manual Run Often the question is asked, how much better can a controller perform than an experienced physician? In reality the controller does not have to outperform the physician, as one of the motivations is to provide a tool to aid her and allow her to give more attention to many other vital signs. Thus, the minimum standard would be to have the controller do a comparable job. On this day we were not able to get the canine into a hypertensive state, therefore we had to content

569

Hemodynamic Control using DMRAC

Fig. 7. Manual run: MAP and CO control of a hypotensive canine with depressed CO using PNP and DP.

ourselves with the hypotensive scenario. Unfortunately this means that we can not do a direct comparison with the controller runs presented in Sections 5.3 and 5.4. With a direct statistical comparison of the performance of the controller versus the manual run not justifiable, this leaves us in a position to do only a qualitative assessment. Even though there was no experienced physician to do the manual run, B. Aufderheide, C. Palerm and R. Rao were the ‘‘experts.’’ Our approach was to control MAP and CO in a similar fashion to what a physician would do; namely, no rate limiting is enforced and we started with infusion rates that were our best estimate of the ones required to get us to the desired setpoints. The feature of allowing the physician to use his expertise to jump start the infusion at a higher rate is certainly easy to do. The run presented in Fig. 7 is the best of three performed. As seen, we were able to keep good control of the arterial pressure; CO control leaves more to be desired. This was mainly because our focus was more on doing the best we could on regulating MAP. It is important to note that the three of us were completely focused on the task at hand. We discussed what the next move should be before making any changes. This is something that the physician does not have the time to do. With so many other tasks they are responsible for, it would be difficult to devote as

much time as we did. We feel that this makes up for our lack of a high level of expertise. It is clear that the controller is able to regulate MAP, even when left with a single degree of freedom as in the run in Section 5.4. The ability of the controller to perform unattended is a significant advantage over manual control, freeing the physician to attend to other aspects of patient care.

6. Conclusions and Recommendations DMRAC is an attractive option for use in drug infusion control, as it does not require on-line identification of the process parameters. The basic algorithm had to be expanded in order to address issues important to its use in patient care. Safety considerations, including limits on drug infusion rates are important, and to this end have been addressed. Allowing a range of values for a controlled output also makes the controller feasible for such a tightly coupled system as is the simultaneous regulation of MAP and CO. Tuning of the controller remains one of its main drawbacks, and this is made worse by the additional tuning parameters introduced in order to handle constraints. If this algorithm is to be considered further for clinical application, then this is an issue that must be addressed in more detail. At the same time,

570

the robust design procedures for the feedforward compensators, including the uncertainty in the time delay elements, make this a more reliable alternative than other adaptive algorithms. For deployment of such a system into a clinical environment, other features can and should be incorporated. For example, alarms that warn the physician that something is not evolving as expected (e.g. the saturation of a drug infusion for a prolonged period of time). The physician should also have the ability to override the controller calculated infusion rate, as they can use their expertise to jump start the infusion rate or even make quick adjustments during operation. We have demonstrated the feasibility of using DMRAC to regulate MAP and CO in experiments on canines, using different drugs combinations and anesthetics. DMRAC has the advantage of not requiring prior model identification for the individual patient, and is able to evolve as patient’s condition evolves. Its applicability has been extended, allowing the simultaneous regulation of MAP and CO, which is a challenging problem for any controller.

Acknowledgments The authors would like to thank W. Hartz and M. Smith for veterinary support. They would also like to thank R. Rao, B. Aufderheide and M. Ayele for their assistance with various aspects of the experiments. They are grateful to Prof J. Newell for his time, support and advice. This paper is dedicated to the memory of Prof H. Kaufman. This paper is based upon research performed under NSF Grant BES9522639. C. Palerm’s work is supported in part by the Consejo Nacional de Ciencia y Tecnologı´ a (CONACYT) of Mexico.

References 1. American Hospital Formulary Service. Sodium nitroprusside. Dialog File 229, Access Number 02096581, 2000 2. Bar-Kana I, Kaufman H. Global stability and performance of a simplified adaptive algorithm 1985; 42(6): 1491–1505 3. Bednarski P, Siclari F, Voigt A, Demertzis S, Lau G. Use of a computerized closed-loop sodium nitroprusside titration system for antihypertensive treatment after open heart surgery. Crit Care Med 1990; 18(10): 1061–1065 4. Blom JA, Hoeksel SAAP. Control aspects of clinical support systems. Trans Inst Meas Control 2000; 22(4): 331–352

C.C. Palerm and B.W. Bequette

5. Broussard J, O’Brien O. Feedforward control to track the output of a forced model. In 17th IEEE Conference on Decision and Control. San Diego, CA, 1979, pp. 1149–1155 6. Chitwood WR, III DMC, Lust RM. Multicenter trial of automated nitroprusside infusion for postoperative hypertension. Ann Thorac Surg 1992; 54: 517–522 7. Cosgrove III DM, Petre JH, Waller JL, Roth JV, Shepherd C, Cohn LH. Automated control of postoperative hypertension: a prospective, randomized multicenter study. Ann Thorac Surg 1989; 47: 678–683 8. Frankel SK, Fifer MA. Heart failure 2nd edn., Ch. 9, pp. 193–216, Lippincott Williams & Wilkins, Baltimore, MD, 1997 9. Held CM, Roy RJ. Multiple drug hemodynamic control by means of a supervisory-fuzzy rule-based adaptive control system: validation on a model. IEEE Trans Biomed Eng 1995; 42(4): 371–385 10. Held CM, Roy RJ. Hemodynamic management of congestive heart gailure by means of a multiple mode rule-based control system using fuzzy logic. IEEE Trans Biomed Eng 2000; 47(1): 115–123 11. Hoeksel SAAP, Blom JA. Computer control versus manual control of systemic hypertension during cardiac surgery. Acta Anaesthesiol Scand 2001; 45: 553–557 12. Hoeksel SAAP, Blom JA, Jansen JRC, Maessen JG, Schreuder JJ. Automated infusion of vasoactive and inotropic drugs to control arterial and pulmonary pressures during cardiac surgery. Crit Care Med 1999; 27(12): 2792–2798 13. Hoeksel SAAP, Schreuder JJ, Blom JA, Maessen JG, Penn OCKM. Automated infusion of nitroglycerin to control arterial hypertension during cardiac surgery. Intensive Care Med 1996; 22(7): 688–693 14. Isaka S, Sebald AV. Control strategies for arterial blood pressure regulation. IEEE Trans Biomed Eng 1993; 40(4): 353–363 15. Johanning RJ, Zaske DE, Tschida SJ, Johnson SV, Hoey LL, Vance-Bryan K. A retrospective study of sodium nitroprusside use and assessment of the potential risk of cyanide poisoning. Pharmacotherapy 1995; 15(6): 773–777 16. Kaufman H, Bar-Kana I, Sobel K. Direct adaptive control algorithms 2nd edn., Springer Verlag, 1998 17. Kaufman H, Neat GW. Asymptotically stable multipleinput multiple-output direct model reference adaptive controller for processes not necessarily satisfying a positive real constraint 1993; 58(5): 1011–1031 18. Kawasaki T, Hoka S, Okamoto H, Okuyama T, Takahashi S. The difference of isoflurane and halothane in ventriculoarterial coupling in dogs. Anesth Analg 1994; 79(4): 681–686 19. Kwok KE, Shah SL, Finegan BA, Kwong GK. An observational trial of a computerized drug delivery system on two patients. IEEE Trans Contr Syst Technol 1997; 5(4): 385–393 20. Leier CV. Acute heart failure: stategies for initial management, part 2. Consultant 1998; 38(12): 2847–2855 21. Martin JF, Smith NT, Quinn ML, Schneider AM. Supervisory adaptive control of arterial pressure during cardiac surgery. IEEE Trans Biomed Eng 1992; 39(4): 389–393 22. McKinley S, Cade JF, Siganporia R, Evans OM, Mason DG, Packer JS. Clinical evaluation of

Hemodynamic Control using DMRAC

23.

24.

25.

26. 27. 28.

29.

30.

closed-loop control of blood pressure in seriously ill patients. Crit Care Med 1991; 19(2): 166–170 Meijers RH, Schmartz D, Cantraine FR, Barvais L, d’Hollander AA, Blom JA. Clinical evaluation of an automatic blood pressure controller during cardiac surgery. J Clin Monit 1997; 13(4): 261–268 Miller RR, Awan NA, Joye JA, Maxwell KS, DeMaria AN, Amsterdam EA, Mason DT. Combined dopamine and nitroprusside therapy in congestive heart failure: Greater augmentation of cardiac performance by addition of inotropic stimulation to afterload reduction. Circulation 1977; 55(6): 881–884 Ozcelik S, Palerm CC, Kaufman H. Multi-drug infusion control using a robust direct adaptive controller for plants with time delays. In: Proceedings 7th IEEE Mediterranean Conference on Control and Automation. Haifa, Israel, 1999, paper MP5 Packer JS. Patient care using closed-loop computer control. Comput Control Eng J 1990; 1(1): 23–28 Palerm CC. Drug infusion control: an extended direct model reference adaptive control strategy. Ph.D. thesis, Rensselaer Polytechnic Institute, Troy, NY, USA, 2003 Palerm CC, Bequette BW. Direct model reference adaptive control and saturation constraints. In: Proceedings of the 15th IFAC World Congress, Barcelona, Spain, 2002, pp. 2714–2719 Rao RR, Aufderheide B, Bequette BW. Experimental studies on multiple-model predictive control for automated regulation of hemodynamic variables. IEEE Trans Biomed Eng 2003; 50(3): 277–288 Rao RR, Palerm CC, Aufderheide B, Bequette BW. Automated regulation of hemodynamic variables. IEEE Eng Med Biol Mag 2001; 20(1): 24–38

571 31. Reid JA, Kenny GNC. Evaluation of closed-loop control of arterial pressure after cardiopulmonary bypass. Br J Anaesth 1987; 59: 247–255 32. Rosenfeldt FL, Chang V, Grigg M, Parker S, Cearns R, Rabinov M, Xu WG. A closed loop microprocessor controller for treatment of hypertension after cardiac surgery. Anaesth Intensive Care 1986; 14(2): 158–162 33. Serna V, Roy R, Kaufman H. Adaptive control of multiple drug infusion. In: 1983 American control conference. San Francisco, CA, 1983, pp. 22–26 34. Slate JB, Sheppard LC, Rideout VC, Blackstone EH. A model for design of a blood pressure controller for hypertensive patients. In: IEEE Engineering in medicine and biology conference. Denver, 1979, pp. 867–872 35. Stemple DR, Kleiman JH, Harrison DC. Combined nitroprusside-dopamine therapy in severe chronic congestive heart failure: Dose-related hemodynamic advantages over single drug infusions. Am J Cardiol 1978; 42(2): 267–275 36. Voss GI, Katona PG, Chizeck HJ. Adaptive multivariable drug delivery: Control of arterial pressure and cardiac output in anesthetized dogs. IEEE Trans Biomed Eng 1987; 34(8): 617–623 37. Westenskow DR. Automating patient care with closedloop control. MD Computing 1986; 3(2): 14–20 38. Yu C, Roy R, Kaufman H. A circulatory model for combined nitroprusside-dopamine therapy in acute heart failure. Med Prog Technol 1990; 16: 77–88 39. Yu CL, Roy RJ, Kaufman H, Bequette BW. Multiplemodel adaptive predictive control of mean arterial pressure and cardiac output. IEEE Trans Biomed Eng 1992; 39(8): 765–778