Real-time implementation of Model Predictive Control ...

Real-time implementation of Model Predictive Control Reference Prediction for Radiotherapy Patient Support System Daniel P. Paluszczyszyn ∗ , Piotr Skworcow ∗ , Olivier C.L. Haas ∗ , Keith J. Burnham ∗ , John A. Mills ∗∗ ∗

Control Theory and Applications Centre, Coventry University, Priory Street, Coventry, CV1 5FB, UK (e-mail: [email protected], [email protected], [email protected], [email protected]). ∗∗ Arden Cancer Centre, University Hospitals Coventry and Warwickshire, Coventry, CV2 2DX, UK (e-mail: [email protected])

Abstract: This paper presents the real-time implementation of a new control system combining model predictive control (MPC) and Kalman filter (KF) predictor to automatically adjust the patient support system (PSS) position, thereby compensating for tumour motion caused by respiration and patient movements during radiotherapy treatment. The control scheme utilises an observer to estimate the PSS state-feedback, and a prediction algorithm to predict future tumour position as the reference for the MPC. The real-time control algorithm was implemented using the MATLAB and Simulink environment, with the communication with the clinical PSS performed through the dSPACE real-time system. The controller was shown to be able to position the patient accurately and be able to follow a pre-planned trajectory and track organ motion with an accuracy of 0.6mm RMSE and 1.1mm RMSE, respectively. Keywords: implementation, model-based control, predictive control, radiotherapy, real-time 1. INTRODUCTION Radiotherapy aims to destroy the cancer cells in the treated area whilst minimising the radiation dose to the surrounding healthy tissues. The organ and tumour motion which occurs during the treatment (during the beamon period) is currently one of the major challenges in radiotherapy treatment (Murphy, 2004). Such intrafractional organ motion, mostly due to respiration, have been reported to be of the order of a few centimetres (Alasti et al., 2001; van Herk, 2004). Several methods have been developed to accommodate the uncertainty of tumour location due to respiration e.g. breath-holding (Mageras and Yorke, 2004), respiratory gating (Murphy, 2004), immobilization (Negoro et al., 2001), tumour tracking (Keall et al., 2001). An approach proposed in (Skworcow, 2008) and (Putra et al., 2007) is based on an accommodation of the tumour motion during irradiation via tumour tracking. In that tumour-tracking technique the position of the tumour is monitored and used as a reference signal to reposition the patient support system (PSS), such that the tumour location is fixed with respect to the beam. The developed control strategy was based on the model predictive control (MPC) algorithm, which is widely implemented in advanced process control technology (Qin and Badgwell, 2003; Camacho and Bordons, 2004). The MPC is an optimisation based control strategy that uses a system model to predict the effect of a control action applied on the system (Camacho and Bordons, 2004).

The advantage of MPC is the ability to handle multivariable constrained control problems in an optimal way (Maciejowski, 2002). The computational requirements of MPC, where typically a quadratic optimisation problem is solved online at every sample, have previously prohibited its application in areas where fast sampling is required. Therefore, MPC has traditionally only been applied to slow processes, mainly in the chemical industry. However, the evolution of computational power and the development of more efficient optimisation algorithms, has led to applications of MPC to processes governed by faster dynamics, e.g. (Dunbar et al., 2002; Castillo et al., 2007). (Haugan, 2001) implemented a real-time MPC to control a ball & plate tracking problem. The aim was to extend the MPC toolbox for real-time use. A linearised model of the ball & plate was developed and the MPC algorithm was coded in C and interfaced with Simulink by use of a Sfunction structure. The real-time implementation of MPC was done using xPC Target, which is a Simulink toolbox that enables to connect Simulink models to physical systems and execute them in real time on PC-compatible hardware. The xPC Target was compared against the Real-Time Toolbox. (The Real-Time Toolbox is a package created for connecting MATLAB and Simulink to the physical systems.) The control sequence was computed using a quadratic programming solver written in C. Simulations and tests shown the possibility to extend the MPC toolbox for real-time use. However, it was also revealed that solving a quadratic programming problem demanded

Fig. 1. MPC with predicted reference and output-feedback. z(k) is actual tumour position at time k while z(k − τd |k) represents the most recent measurement of target position available at time k, Tˆ (k) is a vector of predicted target positions, y(k) and u(k) are PSS output and input, respectively, and x ˆ(k) denotes estimated state for state variable feedback. an excessive amounts of a processor power that restricted possibilities of tuning the MPC; e.g complexity of calculations had limited the prediction horizon to maximum of 30 steps ahead. An application of MPC to control a Caltech ducted fan, a 3 degree of freedom (DOF) thrust-vectored flight experiment was reported in (Dunbar et al., 2002). The fan was controlled by commanding a current to an electric motor for fan thrust and by steering remote control servos to control the thrust vectoring mechanism. For the fan driving and data acquisition a dSPACE multi-processor system was used. The method utilised a real-time trajectory generation package for the considered constrained mechanical system. Sequential quadratic programming (SQP) was used to solve the B-spline coefficients that optimise the performance objective, while respecting dynamics and constrains. For the implementation of the receding horizon approach a planar model of the fan was considered. The proposed control approach was a hybrid of MPC and control Lyapunov functions. The timing was set-up in such a way that computation times varied each optimization as the current state of the ducted fan changed. Examples of the related work have demonstrated different approaches to implement the MPC algorithms and also have highlighted an issue related to the computational requirements for the optimisations associated with the MPC. However, approaches adopted for this paper differ from the mentioned due to specific requirements of this implementation and available tools. The main contribution of this paper is a real-time implementation of MPC with constraints on the input on a clinical radiotherapy treatment (RT) machine. The realtime implementation of the control algorithm was developed using MATLAB, Simulink, and dSPACE real-time system based on the DS1104 Controller Board, which is used to provide a hardware connection to the clinical PSS interface. The remaining parts of this paper are divided as follows: Section 2 describes the control strategy. Section 3 presents the tools used and the process of implementation. Section 4 discusses the results of experiments on the clinical PSS. Section 5 provides a conclusion. 2. CONTROL STRATEGY The scheme to control the motion of the PSS in order to track the respiratory-induced tumour motion was de-

veloped by the CTAC (Control Theory and Applications Centre) team in collaboration with UHCW (University Hospitals Coventry and Warwickshire, NHS Trust), see (Putra et al., 2007; Skworcow, 2008). The control algorithm consists of the MPC controller to compute an optimal control signal for the PSS to follow the trajectory predicted by a Kalman filter predictor. The control scheme contains an observer designed to obtain estimated statefeedback from input-output measurements. This configuration has allowed the use of the MPC look-ahead feature to compensate for delays in the control loop as well as the PSS dynamics, without knowing of the desired trajectory in advance. In addition it provided the means to explicitly use constraints, such as restriction in the allowed velocity of the PSS. The overall control scheme for the outputfeedback case is shown in Figure 1. The controller design was based on a state-space model of the PSS, with the states being position and velocity of the PSS, due to two reasons: (i) if the PSS is equipped with not only position sensors, but also velocity sensors, inclusion of such additional feedback is straightforward, (ii) the constraints being a function of states are easy to implement. When designing a controller for an electromechanical system, it is important to take into account the actuator limitations. MPC with constraints allowed to explicitly restrict the input to the plant and the plant response when computing the control action, such that the constraints were not violated. The constraint considered was a maximum voltage, which corresponded to the maximum speed. In MPC, a sequence of control actions {ˆ u(k|k) . . . u ˆ(k + Hu − 1|k)} is computed such that the plant model follows the future reference trajectory provided by the predictor whilst minimising a cost function subject to defined constraints. The cost function to be minimised is: Hp X ¯¯ ¯¯ ¯¯yˆ(k + i|k) − zˆ(k + i|k)¯¯2 V (k) =

Q(i)

+

Hu X

i=1

¯¯ ¯¯2 ¯¯∆ˆ u(k + i − 1|k)¯¯

(1)

R(i)

i=1

where: Hu is the control horizon such that Hu ≤ Hp (where Hp is the prediction horizon); yˆ and u ˆ are the predicted output and input, respectively; ∆ = 1 − q −1 with q −1 denoting the backward shift operator defined by q −1 u ˆ(k + i|k) = u ˆ(k + i − 1|k), and Q(i) and R(i) are controller weights, such that Q(i) ≥ 0 and R(i) > 0. At each sample step k the predictor calculates a vector of predicted target positions Tˆ (k) = {ˆ z (k + 1|k), . . . , zˆ(k + i|k), . . . , zˆ(k + Hp |k)}. The vector Tˆ (k) is fed to the MPC as a reference trajectory to be tracked. Numerous studies, using a variety of prediction methods have reported that the accuracy of the prediction degrades with the increase in the prediction horizon, see e.g. (Goodband and Haas, 2008). Thus the prediction of tumour position zˆ(k+i|k) is typically more accurate than zˆ(k+j|k) for j > i, and yˆ(k + i|k) − zˆ(k + i|k) shall be of greater importance than yˆ(k + j|k) − zˆ(k + j|k). To reflect the confidence in the relative accuracy of the predictor the

controller weights were chosen such that Q(i) = (Hp − i + 1)2 . It was also assumed that ∆ˆ u should be equally penalised for the whole control horizon, and the ratio of penalisation of predicted tracking error and penalisation of ∆ˆ u should remain constant regardless of Hp and Hu , leading to: PHp i=1 Q(i) = constant (2) PHu i=1 R(i) ∀i, j ∈ h1, Hu i R(i) = R(j) therefore

(3)

PHp R(j) = ρQR ·

Q(i) , j ∈ h1, Hu i Hu

i=1

(4)

where ρQR is a scalar valued tuning parameter. In constrained MPC an optimal sequence of control actions is computed using quadratic programming (QP) solver. 3. REAL-TIME IMPLEMENTATION 3.1 Real-time implementation approaches

longitudinal (Y-axis) and vertical (Z-axis); and one rotational (around the vertical axis) called the I-rotation. Note, that all practical experiments in this project were performed on the clinical Elekta Synergy, in which the position of the PSS along its axes is measured by potentiometers. A dSPACE DS1104 Controller Board is placed in a host PC and provides the interface to connect to external systems. The DS1104 consists of the PowerPC microprocessor combined with input/output digital and analog interfaces. The DS1104 Controller Board has eight 16-bit D/A converters (DAC), each with an output range of ±10 V . The DS1104 also contains eight A/D channels. The first four channels share a single 16-bit A/D converter (ADC) through an analog multiplexer. The remaining four channels each have dedicated 12-bit A/D converters. The input range for all eight channels is ±10 V . The resolution of the 16-bit converters is 0.305 mV , while the resolution for the 12-bit converters is 4.9 mV . In this work the analog to digital converters (ADC) are used to establish the position readings from the PSS. The DACs are connected to the PSS interface to drive the X, Y and Z axes. Reference from external devices such as position sensor or video tracking system is connected to the ADC channels.

Initial work found that the control strategy could not be implemented using the MPC Toolbox due to its limitations when using the look-ahead feature and incapability to specify the required form of the controller weights Q and R. An embedded MATLAB function allowed only to implement the unconstrained MPC, where solving the QP problem is not required. Therefore, to accommodate the velocity constraints an S-function approach was chosen. The S-function allowed the constrained MPC tp be implemented, however, it required to code the MPC algorithm, including the QP solver, in C.

The positions of the 4 axes of the PSS are read from a purpose built interface designed and constructed by the UHCW technicians. The interface to drive the PSS was linked with the manual joystick controller in such way, that the voltages sent to the interface imitated the displacement of the manual controller joysticks, thus causing the motion of the PSS. It was observed that the positions read out were highly corrupted by noise.

Finally, after a review of potential solutions it was decided to use the following software, tools and devices:

A graphical user interface (GUI) was designed and developed using ControlDesk, a software part of the dSPACE system. The designed layout provides the user with the means to control and tune online the overall prediction and control strategy parameters as well as to monitor the process while the experiment is in progress. A series of selectors enable the user to specify the initial experiment conditions and save the data for further analysis. dSPACE software provides also the appropriate libraries for MATLAB and Simulink, which allows a Simulink block diagram to be converted to real time C code, cross compiled and downloaded to the DS1104 board (dSPACE GmbH, 2005).

MATLAB 7.1.0.246 Release 14 Service Pack 3 Simulink Real-Time Workshop Toolbox Release 6.3 dSPACE ControlDesk 2.7 with associated libraries for MATLAB • host PC connected to PSS via dSPACE DS1104 Controller Board. • special interface to send control signals to PSS and read feedback. • • • •

3.2 Hardware All simulations and experimental tests were done using standard PCs in the following configuration: • Processor Intel® Pentium® 4 1500MHz, 768MB RAM, • dSPACE DS1104 R&D Controller Board in PCI Slot • Microsoft® Windows— XP Professional SP 2 A standard (RT) machine consists of a gantry-mounted megavoltage linear accelerator (linac), which produces a radiation beam. A 4 DOF manipulator, referred as the PSS, positions the patient under the radiation beam. The DOFs are: three translational, namely lateral (X-axis),

3.3 Software

The code generation, packaging and compilation is performed with the Real-Time Workshop (RTW) Toolbox. The process of generating source code from Simulink models using RTW is shown in the following diagram (Figure 2). The outcome of the process is a real-time application, running on the DS1104, of the control strategies implemented in Simulink. One of the main advantages of RTW consists of its fully configurable code generator called TLC, which specifies how to transform a Simulink block model into a C code, allowing the production of a control software for virtually any operating system and platform on which MATLAB can be run. In this paper the rti1104.tlc library was used. It controls the means by which code is generated by

Fig. 3. Illustrating typical set up for experiments with the clinical PSS at UHCW. position is converted from pixel to millimetres and then to volts before being sent to the PSS controller PC as a reference to be tracked. The Camera controller PC is equipped with a NI6024 PCI board which offers two 12bits DAC wich is connected to the DS1104. This provides a facility to track and compensate for patient motion in along the vertical and longitudinal axes. Fig. 2. Illustrating a process of generating source code from Simulink models using the RTW (The MathWorks, 1994-2005).

A position sensor can also be used as an independent measurement device or as a reference trajectory.

RTW from individual Simulink blocks to be interpreted for the DS1104 Controller Board.

4.2 Evaluation of reference tracking

For a more detailed description of the implementation see (Paluszczyszyn, 2008).

The controller performance was evaluated using a series of tests. Initially the controller was evaluated for its ability to follow a user defined trajectory. Step changes were applied to simulate the process of positioning the patient before each fraction. An accuracy (root mean square error RMSE) of the order of 0.3 mm was achieved making the system able to reposition the PSS consistently to the treatment isocentre without any visual discrepancy. Regular signals such as sinewave were then used to evaluate the ability of the predictor to predict smooth trajectories. The next set of tests was aimed to demonstrate the ability of the patient positioning system to move the PSS to follow recorded patient motion, along the X,Y and Z axes. It was found that an accuracy better than 1 mm in terms of RMSE could be achieved. Finally, trajectories provided by external sensors were used as a reference, i.e. video camera, phantom or the position sensor.

4. RESULTS OF PRACTICAL EXPERIMENTS 4.1 Clinical PSS experiment set-up A typical set up for the experiments is shown in Figure 3. The implemented control strategy was loaded to the dSPACE DS1104 Controller Board installed in the PC indicated in Figure 3 as PSS controller. The output of the DS1104 board was connected via a connector box to the PSS. A host PC was used to provide a GUI to run the control application, control the system components switching such as observer, predictor or reference, provide diagnostic signal plots, change the system parameters online, and collect test results for further analysis. To evaluate the performance of the overall closed loop system to track and compensate patient motion, a phantom replicating chest motion as well as internal organ motion was developed (Land et al., 2008). It is able to simulate arbitrary motion with a drive system controlled by another PC. The phantom was placed on the PSS table in front of a video camera to record the movement of a marker located on the phantom drive system. The motion was detected by a video camera connected to the Camera controller PC. The video tracking system was implemented in LabVIEW and is able to recognise and track active as well as passive markers that can be located on the phantom. The marker

The set up illustrated in Figure 3 was used to assess the ability of the controller to track patient motion trajectories from the phantom drive system. The motion was generated by the phantom, measured by the position sensor positioned along the drive system and then sent to the controller. Figures 4 and 5 show that the controller followed the recorded motion trajectory with an accuracy of RMSE = 0.57 mm. Figure 5 illustrates the ability of the controller to adapt in the presence of drift (signal moving downwards) as well as amplitude change from 10 mm to less than 3 mm.

Response of PSS to the true motion generated by phantom model

180

Response of the PSS to trajectory generated by pressing a sensor

Reference PSS response

Response of the lateral axis.

Position [mm]

175

170

165 5

Controlled Input [V]

2

RMSE = 0.57 [mm]

10

470 460

Reference PSS response

RMSE(lateral) = 1.09 [mm] 15 20 Time [s] Controlled input forwarded to PSS

25

30

450 0

|∆ˆ u(k)| = 0.176 [V]

5

10

0 −1

15

20 25 Time [s]

30

35

40

45

35

40

45

35

40

45

Response of the longitudinal axis.

310

1

Position [mm]

Position [mm]

480

300 290 RMSE(long.) = 1.09 [mm]

−2 5

10

15

20

25

280 0

30

Time [s]

5

10

Position [mm]

20 25 Time [s]

30

Response of the vertical axis

880

Fig. 4. Illustrating the ability of the model predictive controller to track patient motion generated by the phantom.

15

870 860 RMSE(vertical) = 1.10 [mm] 850 0

Response of PSS to the true motion generated by phantom model

Position [mm]

170 Reference PSS response

5

10

15

20 25 Time [s]

30

160

150 RMSE = 0.57 [mm]

140 5

10

15 20 Time [s] Controlled input forwarded to PSS

25

30

Fig. 7. Illustrating the ability of the controller to follow a position request given by a position sensor moved manually.

Controlled Input [V]

4 2 0 −2

|∆u(k)| = 0.098 [V]

−4 5

10

15

20

25

30

Time [s]

Fig. 5. Illustrating the ability of the model predictive controller to track irregular patient motion with drift generated by the phantom. Response of the PSS vertical axis.

Position [mm]

825

Reference PSS response

820 815 810 805

RMSE = 1.13 [mm]

800 12

14

16

18

20

22 24 26 28 Time [s] Controlled input forwarded to the PSS

∆|ˆ u(k)| = 0.351[V ] 12 14 16 18

20

30

32

30

32

Controlled Input [V]

2 1 0 −1 −2

22 24 Time [s]

26

28

Fig. 6. Illustrating the ability of the controller to follow a marker motion recorded by the video camera. The tests with the another external source of reference involved a marker that was moving along the PSS axis. The aim was to track the marker.

It was observed that when the marker moved forward, the PSS followed the marker. Figure 6 illustrates that the controller was able to take into account the latencies in the reference signal due to communication delay in the control loop and controlled the PSS, such that it followed the trajectory with the accuracy of 1.13 mm in terms of RMSE. The aim of experiments with a position sensor was to check if the approach adopted for the longitudinal axis was applicable to other axes. It was decided to use a hand held position sensor to request position changes along the longitudinal and then the vertical and lateral axes. The position sensor was a spring mounted rod, in which the length could be changed over the range of 0 mm to 50 mm by depressing the rod manually. Operating the sensor resulted in the PSS moving all three axes concurrently forward by 50 mm when the sensor was fully depressed and moving back by 50 mm when the sensor was fully released. Figure 7 shows the performance of the controller for the three axes. The PSS was able to follow the position requested by the sensor with a RMSE of 1.09 mm, 1.09 mm and 1.10 mm for the lateral, longitudinal and vertical axis, respectively. The developed controller has been tested with all three axes considered, i.e. longitudinal, lateral and vertical. The main issue was the selection of the controller parameters and controller tuning is remaining work in progress. The results achieved have demonstrated that the approach is applicable to control a standard RT PSS not initially designed for motion tracking or compensation.

5. CONCLUSIONS The real-time implementation of the constrained MPC was carried out utilising the S-function approach. The S-function structure has required MPC algorithm to be coded in C. In constrained MPC an optimal sequence of control actions is computed using quadratic programming solver, which also was implemented in C. The real-time implementation platform was based on the dSPACE real-time system, which provided hardware components to connect to the PSS interface. A graphical user interface was designed to operate the individual control system components, such as observer, predictor, or reference, and to provide a diagnostic plots of the real-time signals. The GUI allows the online tuning of the controller parameters as well as the collection of data for analysis. The controller has been tested on all three translational PSS axes, i.e. longitudinal, lateral and vertical. The results from the experiments, where three axes have been controlled simultaneously have confirmed the ability of the controller to drive all the PSS axes. Additionally, experiments with the video camera tracking system have demonstrated that the controller is able to take into account latencies in the reference signal due to image processing and communication delay and still achieve tracking accuracy of 1.13 mm in terms of RMSE. The results of experiments involving the clinical PSS has confirmed the theoretical benefits of the developed control strategy demonstrated previously by simulations and also validate its real-time implementation. ACKNOWLEDGEMENTS This work is sponsored by the Framework 6 European integrated project Methods and Advanced Equipment for Simulation and Treatment in Radiation Oncology (MAESTRO) CE LSHC CT 2004 503564. The authors would like to acknowledge all the staff of the Radiotherapy department at the University Hospitals Coventry and Warwickshire NHS Trust for their help and technical support. REFERENCES Alasti, H., Petric, M., Catton, C., and Warde, P. (2001). Portal imaging for evaluation of daily on-line setup errors and off-lineorgan motion during conformal irradiation of carcinoma of the prostate. International Journal of Radiation Oncology Biology Physics, 49(3), 869–884. Camacho, E. and Bordons, C. (2004). Model predictive control. Springer, 2nd Edition ,London. Castillo, C.L., Moreno, W., and Valavanis, K. (2007). Unmanned helicopter waypoint trajectory tracking using model predictive control. In Proceedings of the 15th Mediterranean Conference on Control & Automation. dSPACE GmbH (2005). ControlDesk Reference Release 5.0. Dunbar, W.B., William, M.B., Franz, R., and Murray, R.M. (2002). Model predictive control of a thrustvectored flight control experiment. In Proceedings of the 15th IFAC World Congress on Automatic Control. Goodband, J.H. and Haas, O.C.L. (2008). Artificial neural networks in radiation therapy. In O.C. Haas and K.J.

Burnham (eds.), Intelligent and Adaptive Systems in Medicine, 213–258. Taylor & Francis. Haugan, T. (2001). Real-Time Model Predictive Control. Master’s thesis, Automatic Control Laboratory Swiss Federal Institute of Technology. Keall, P., Kini, V., Vedam, S., and Mohan, R. (2001). Motion adaptive x-ray therapy: a feasibility study. Phys. Med. Biol., 46, 1–10. Land, I., Mills, J.A., Haas, O.C., Burnham, K.J., and Wilson, A. (2008). Deliverable no 62, report on thorax phantom performance and role in pre-clinical evaluation. Maciejowski, J.M. (2002). Predictive Control With Constraints. Pearson Education Ltd. Mageras, G. and Yorke, E. (2004). Deep inspiration breath hold and respiratory gating strategies for reducingorgan motion in radiation treatment. Semin. Radiat. Oncol., 14, 65–75. Murphy, M. (2004). Tracking moving organs in real time. Semin. Radiat. Oncol., 14, 91–100. Negoro, Y., Nagata, Y., Aoki, T., Mizowaki, T., Araki, N., Takayama, K., Kokubo, M., Yano, S., Koga, S., Sasai, K., Shibamoto, Y., and Hiraoka, M. (2001). The effectiveness of an immobilization device in conformal radiotherapy for lung tumor: reduction of respiratory tumor movement and evaluation of the daily setup accuracy. Int. J. Radiat. Oncol. Biol. Phys., 50(4), 889– 898. Paluszczyszyn, D. (2008). Real-Time implementation of a new control system utilising the Model Predictive Control and the Kalman Filter predictor for the Radiotherapy Patient Support System. Master’s thesis, Coventry University, Control Theory and Applications Centre, UK. Putra, D., Skworcow, P., Haas, O., Burnham, K., and Mills, J. (2007). Output-feedback tracking for tumour motion compensation in adaptive radiotherapy. In Proceedings of American Control Conference 2007, New York, USA. Qin, S.J. and Badgwell, T.A. (2003). A survey of industrial model predictive control technology. Control Engineering Practice 11, 733-764. Skworcow, P. (2008). Modelling and predictive control of radiotherapy treatment machines. Ph.D. thesis, Coventry University, Control Theory and Applications Centre, UK. The MathWorks, I. (1994-2005). Real-Time Workshop Users Guide. van Herk, M. (2004). Errors and margins in radiotherapy. Seminars in Radiation Oncology, 14(1), 52–64.