Proceedings of the 2003 IEEE International Conference on Robotics & Automation Taipei, Taiwan, September 14-19, 2003
Modeling and Vision-based Control of a Micro Catheter Head for Teleoperated In-Pipe Inspection Saliha Boudjabi1, Antoine Ferreira1, Alexandre Krupa2 1)
Laboratoire Vision et Robotique (LVR), ENSIB- University of Orléans 10, Bd. Lahitolle, 18020 Bourges, France 2) Laboratoire des Sciences de l’Image, de l’Informatique et de la Télédétection (LSIIT), ULP Strasbourg, France
[email protected]
Abstract The objective of this project, named MESIA (Multifunctional Micro-Endoscope Head System for Industrial Applications) is to develop an integrated multisensor system and to create intelligent sensor fusion for detection of micro-cracks. This paper is focused on the modeling of a 2-DOF pan-tilt platform actuated by Ni-Ti shape memory alloy (SMA) wires and antagonistic mechanical springs in order to control visually the CCD camera motion. A Preisach model is used and experimentally identified. The derived model is then exploited to design a position controller which compensates for the hysteretic nonlinearity. Finally, by controlling the catheter head orientation through a head motion tracker, in-pipe visual tracking has been experimentally verified from a remote site.
1. INTRODUCTION The needs for inspection and diagnosis of industrial intra-tubular structures of millimeter sized diameter, i.e., vapor generator, electric turbines, nuclear stations, requires the development of active catheter devices with multifunctional abilities able to detect, measures and intervene in flooded pipes. The objective of this project (Fig.1b), named MESIA (Multifunctional Micro-Endoscope Head System for Industrial Applications) is to develop multi-sensor system and to create intelligent sensor fusion for detection of micro-cracks. It is envisaged that a CCD micro-camera would measure the surface geometry of the drained part of a sewer, while an ultrasonic sonar scanner measures the flooded part. Up to now, various technologies have been investigated in the actuation of micro endoscope head systems, such as electrostatic [1], pneumatic [2] and shape memory alloy (SMA) [3] microactuators. It appears that considering its interesting characteristics for lowpowered and miniaturized micro head endoscopes with coarse and fine motion capabilities, SMA actuation is the mostly employed. As illustration of this technology for real in-pipe industrial inspection, a 2-DOF pan-tilt platform prototype actuated by two antagonistic Ni-Ti shape memory alloy wires has been developed (Fig.1a). Its goal is to control visually the CCD camera motion. Practically, the extreme environmental operating conditions of flooded pipes (inert gas, variable temperature, high pressure), lead to strong dynamic nonlinearities of SMA actuators, i.e. hysteresis, mechanical stress and thermal behavior. For
0-7803-7736-2/03/$17.00 ©2003 IEEE
(a)
(b)
Figure 1. a) Current developed prototype and (b) schematic representation of active catheter head end-effector with multifunctional possibilities, e.g., vision, ultrasonic sonar, light, micro-tool.
accurate motion control, linearization based inverse model control methods should be employed. In [4], thermomechanical models with a limited number of physical parameters are provided, and used for closed loop control. A number of authors also use Preisach [5] or KP[6] modeling as input-output static mapping. These models do not account for the physics of the system but provides a generic way of modeling hysteretic systems. They use simple identification methods, and can be exploited in a control loop. Also, a number of convenient properties for control design, such as positivity, passivity, etc.., have been formally proven from Preisach models in [7]. In the first part, we briefly present the 2-DOF pan-tilt platform actuated by two antagonistic Ni-Ti shape memory alloy (SMA) wires. In Sections 3 and 4 we show how generalized Preisach model can be used for modeling and for vision-based control of the micro catheter head prototype. Then, Section 5 shows experiments of visual inspection in a non-flooded pipe made by a remote operator through a teleoperation system assisted by a graphical user interface (GUI) before to conclude in Section 6.
2. SYSTEM DESIGN The micro catheter head end-effector is a 12 mm diameter system which includes (Fig.1): - a two degree of freedom pan-tilt device, each degree of freedom being actuated by one Ni-Ti
4282
-
shape memory alloy (SMA) wire and an antagonistic mechanical spring; a miniature CCD camera sensor integrating an achromatic lens, a piezoelectric ultrasonic sensor of 5 mm diameter for micro-crack characterization. a micro-tool acting as a marquer and/or a pointer (e.g.,micro-drilling tool) [8].
Once heated through Joule effect, an SMA wire changes from martensite phase to austenite phase. It is thus contracted, which generates a rotation of the mobile plate around the extremity of the fixed joint. Once cooled (through natural convection) it can recover its initial length (back rotation) if a force is exerted by the antagonistic spring. The electrical connection of the CCD sensor is possible through micro holes that have been drilled through the in-pipe base and the mobile part. The CCD sensor is connected to 7×25mm board which pre-amplifies the CCD signal. The ultrasonic sensor is connected to a signal conditioner board integrated into the catheter. The maximum deviation angle is approximately 60 degrees for a maximum input current of 400mA.
From Fig.2, it is clear that the SMA actuators hysteresis loop is only defined in the first quadrant of the u(t)-f(t) plane. So the value of the hysteresis operator γαβ[u(t)] are selected as switching between 0 and 1. Knowing what the maximum and minimum values of the input are, the condition α ≥ β leads to a limiting triangle T on the α−β plane (Fig.3) which is defined such that the function µ(α,β) is equal to zero outside T. On the α−β half-plane, there is a one-to-one correspondence between operators γαβ[u(t)] and points (α,β) which implies at each pair of values defines a unique operator γαβ[.] with switching values α and β [9].
3. APPLICATION OF THE PREISACH MODEL TO SMA ACTUATORS 3.1. Generalized Preisach Model The Preisach model is a phenomenological model which has recently been applied for modeling hysteresis in smart actuators. The SMA wires are current-controlled through the linear relation i(t)=u(t)/R with R=20Ω. In the following model, we assume that the electrical input is given by the applied voltage u(t). Figure 2 shows experimental hysteresis curves describing the nonlinear angular motion versus applied voltage behavior of the SMA-driven platform. Not modeling hysteresis in SMA material can lead to inaccuracy in open-loop control, and can generate amplitude-dependent phase shifts and harmonic distortion that reduce the effectiveness of feedback control. In order to implement an efficient visual tracking control, hysteresis have to be taken into account. The classical Preisach model equation relating the shape memory alloy deformation f(t) to the electric current u(t) is [9]:
[
]
f (t ) = ∫∫ µ (α , β ) ⋅ γ αβ (u (t )) ⋅ dα ⋅ dβ
(1)
P
where γαβ[.] are elementary hysteresis operators with switching values α and β and whose values are determined by the input current signal u(t). The output of these operators is multiplied by the corresponding Preisach functions µ(α,β) , and then summed continuously over all possible values of α and β. The function µ(α,β) is a weighting function estimated from the measured data and is defined as : 2 ∂ f (α, β ) µ(α, β ) = 1 ⋅ 2 ∂α∂β
Figure 2. Angular position versus voltage curves showing hysteresis behavior of SMA wire actuators during martinsite and austenite phases.
Figure 3. Mechanism of storage of input history in the Preisach plane. Graphs (a)-(c) depict the time trajectory of voltage input and graphs (d)-(f) show the corresponding subdivisions of the triangle T and the formation of the interface L(t).
At each instant of time and as a result of applying an input u(ti), the limiting triangle in the half-plane can be divided into two areas, P+(t) and P-(t) (Fig.3a). All operators that belong to P+(t) are equal to 1 and those that belong P-(t) are equal to 0. For detailed explanations, the reader may refer to [9]. As the input starts to monotonically increase, the hysteresis operators γαβ[u(t)] with switching values less than the current input u(t1) switch to “+1” position (P+ region). The demarcation between the two sets is a horizontal line given by the equation α = α1 = u(t1), and
4283
shown in Fig.3(d). Next, when the input starts to monotonically decrease from u(t1) to u(t2) all the hysteresis operator γαβ[u(t)] with switching values larger than the current input value u(t2) switch to the “0” position (Pregion). Geometrically, this means that the previous subdivision of the limiting triangle T is changed, and the interface link L(t), which express the boundary between the regions P+(t) and P-(t), moves from right to left (Fig.3b-e). Based on the above, the latter two displacements are denoted F(ut1,ut2) and are defined as:
F (u t1 , u t 2 ) = f ut 1 − f ut1ut 2 = ∫∫ µ (α , β ) ⋅ dα ⋅ dβ
(2)
P
The function F(ut1,ut2) is a double integral of the weighting function µ(α,β) over the region P+(ut1,ut2). In the third step, voltage is again increased to a value α2 at time t3, which is less than α1. This increase the input results in the formation of a new horizontal link in L(t) at α = α2 = u(t3). Next, as voltage is decreased again to β2 greater than β1, a new vertical link appears on the interface function L(t). The figure of the final two links in L(t) is shown in Fig.3(f). 3.2. Identification and Numerical Implementation The Preisach model can be implemented by using Eq.(2) to compute the output f(t). However, the computation of the weighting function µ(α,β) requires double differentiation which amplifies errors always present in experimentation. In general, the expression of the SMA expansion, in the case of a monotonically increasing input, is: n −1
f (t ) = ∑ [F (M k , mk −1 ) − F (M k , m k )] + F [u (t ), m n −1 ]
(3)
k =1
where f(t) stands for the SMA expansion estimated using the classical Preisach model, (n-1) represents the number of maxima/minima stored (not wiped-out) and the pairs {(Mk,mk)} represent the sequence of maximum and minimum values of the input signal constituting the coordinates interface L(t). The corresponding triangles are depicted in Fig.4(a-b). By considering
F (M k , mk ) = f M k − f M k mk
(4)
F (M k , mk −1 ) = f M k − f M k mk −1 Eq.(3) can be rewritten as (Fig.4a) : n−1
[
]
u& (t ) > 0 : f (t ) = ∑ f M k mk − f M k mk −1 + f u (t ) − f u (t ) mk −1
(c) Figure 4. Triangle T for numerical implementation of Preisach model when (a) input is decreasing, (b) input is increasing and (c) square mesh in α−β plane.
subject to a known arbitrary input voltage sequence u(t). Numerical implementation of the above form of the Preisach model requires the experimental determination of F(α,β) at a finite number of points within the triangle T. For this purpose, a square mesh covering the triangle T is created, divided into a number of squares and triangles, and the corresponding values of f(α,β) are experimentally determined (Fig.4c). The parameters of the Preisach model F(α,β) are calculated using Eq.(5) and (6). The alternating series of dominant extrema (Mk, mk) are calculated from the time history of the input voltage, to determine the vertices of the interface L(t). This series is continuously updated at each instant of time. Finally, the output f(t) of the hysteresis nonlinearity is calculated using the Preisach model parameters {Fα,β}, in (5) and (6). In the experiments reported in this article, α0=β0=1.2V. A grid of points with ∆α=∆β=0.1V in the complete range of variation of α and β on the triangle T is created. Then, fα,β are experimentally determined for these grid points, and the Preisach model parameters model are calculated. For points of the (α,β) plane lying within of the squares, the interpolation Eq. (7) is used, and for points lying within any of the triangles, the linear interpolation Eq.(8) is used.
(5)
k =1
and similarly, when the input is monotonically decreasing (Fig.4b): n−1 (6)
[
]
u& (t ) < 0 : f (t ) = ∑ f M k mk − f M k mk −1 + f M nu ( t ) − f M nmk −1 k =1
From this, it is possible to explicit expressions that can be used to calculate the response f(t) of a SMA actuator
fα 'β ' = c0 + c1α '+c2 β '+c3α ' β '
(7)
fα 'β ' = c0 + c1α '+c2 β '
(8)
The ci-coefficients in (7)-(8) are found for each square and triangle using the experimental values fα’β’ at the corners of the squares and triangles, respectively. In order to verify the validity of the identified model, an arbitrary sequence was applied to the experimental system. Figure 5 shows the good agreement between the real and simulated outputs.
4284
θd[k]
θd[k] > θm[k − 1]
no
yes u[k ] = u[k − 1] + 0.1
no
θd[k] < θm[k − 1]
y
θm[k] = f (u[k])
u[k] = u[k −1] − 0.1
no
θm[k] ≥ θd[k]
θm[k] = f (u[k])
yes u[k −1] = u[k]
Figure 5. Actual and predicted hysteresis response under an arbitrary input excitation.
θm[k − 1] = θm[k]
no θm[k] ≤ θd[k]
yes
u[k]
4. VISION-BASED CONTROL
u[k −1] = u[k]
θm[k −1] = θm[k]
u[k]
4.1. Inversion of the Preisach Model Hysteresis As explained in the previous section, the modeling of hysteresis allows to design controllers that correct these hysteretic effects and improve accuracy of the orientation of the micro catheter head. Essential to the synthesis of such controller architectures is the development of the inverse Preisach model. Given the Preisach model parameters {Fα’,β’} and the associated interface L(t) for the triangle T, the inverse Preisach model determines the voltage u(t+∆t) that will result in a desired orientation angle θd (t+∆t) that at the next instant. In the formulation of the inverse Preisach model, two distinct cases corresponding to decreasing θd (t+∆t) < θd (t) and increasing θd (t+∆t) >θd (t) need to be considered.When θd (t+∆t)=θd (t), then u(t+∆t)=u(t) as shown in Fig. 6. The terms θd, θm and u are the signal entry of the Preisach model, the output of the Preisach model and the inverse Preisach model, respectively. Consequently, it is possible to pursue modelbased compensation of hysteresis nonlinearity in these actuators using an open-loop compensation strategy depicted in Fig.7. The final inverse model consists of two blocks: an inverse linear transfer function which approximates the heating process occurred on SMA wires and an inverse nonlinear Preisach block. The performance of the compensation scheme is experimentally investigated for variations of angular orientation (Fig.8). The experimental angular position trajectory follows accurately the first rising ramp for slowly varying inputs which is not the case at the end of the input profile due to strong dynamical effects and variable dynamic behavior of the SMA heating process. A significant error exists between the reference and the actual signal after a long period of operation.
u[k]
Figure 6. Algorithm of Preisach model inversion.
Figure 7. Model for open-loop compensation of hysteresis.
Figure 8. Tracking control using open-loop control.
gravity of the target: P=(X,Y)T and we control the camera in order to see it centered in the image : Pd=(Xd,Yd)T =(0,0)T. The image Jacobian is given by [10]:
XY L = 2 1 + Y
4.2. 2-D Visual Servoing Approach We define as visual features the projection of the center of
u[k] = u[k −1] θm[k] = θm[k −1]
(
)
− 1+ X 2 − XY
and the resulting control law is simply estimated from :
4285
(9)
θ θ
act X act Y
Y 2 2 1 X Y + + + µI , = −λ X − 1+ X 2 + Y 2
angle (deg) versus time (s)
angle (deg) versus time (s)
(10)
where I, λ and µ are an integral term to attenuate the tracking errors, and constant parameters, respectively.
2
23
1.5
22.5
1
22
0.5
21.5
0
21
4.3. Internal-based Model Control Different closed-loop controllers have been developed for control of SMA actuators based on inverse Preisach model u=f-1(y). In [11], the inverse Preisach model is used in a feedforward compensation, in addition to a PI feedback control loop. In, [12], a model reference adaptive control (MRAC) using the inverse Preisach model as inverse compensator is used. These methods can be poorly applied on our prototype taking into account the variability of the in-pipe operating conditions. A more robust approach based on feedback linearization with an internal model has been studied (Fig.9).
0
4
2
6
8
10
0
2
4
6
8
10
Figure 10. Closed loop response with Preisach model based controller. 70 60
θ=40°
50
θ=30°
40
θ=20°
Y [mm] 30 20 10 0
10
20
30
40
50
60
70
X [mm] Figure 11. Vision-based tracking closed loop response of a circular trajectory.
Figure 9. Scheme of the internal model based control.
The desired input and output position vectors are defined by θd =[θdx,θdy] and θact=[θactx,θacty], respectively. The vector position θm=[θmx,θmy] defines the internal model output. The function G(s) is used to compensate the phase-lag due to dynamics of the plant by placing a slow pole (-1/τH). The main characteristics of the internal model based control is to guarantee the closed-loop stability by adjusting a simple corrector C(s) and to ensure the robustness against perturbations. The filter H(s) describes the ideal transfer function between the actual angular position and the input of the inverse Preisach model. In order to verify the efficiency of the proposed approach, Fig.10 shows the tracking results for different desired positions by considering only one degree of freedom of the micro catheter head. It is shown that the closed loop behaviour using the inverse Preisach model linearizes greatly the system dynamics. Figure11 shows the a two-dimensional closed-loop tracking of a circular trajectory. From this experiment, it can be seen that the vision-based closed tracking confirms that the generalized inverse Preisach model linearizes the system dynamics. We can conclude that off-line identification can reduce the computational overhead for real time control implementation.
5. TELEOPERATED VISUAL IN-PIPE INSPECTION 5.1. Head Motion Tracking System Figure 12 shows the input and output devices for teleoperated in-pipe inspection. A joystick is used as an input device for manoeuvering of the microcatheter locomotion, whereas the pan and tilt motions of the camera are controlled by the output of a motion tracking system (Flick of Birds, Ascension Technology Corp.) attached to an head mounted display (Seam Team Corp.).
cone of visibility
HMD
Display HMD
attitude HMD motion tracker
camera angles
Joystick
Joystick
Remote Site
velocity command
In-Pipe
Figure 12. Vision-based head motion tracking system for in-pipe inspection.
4286
The camera control commands, which are calculated on the basis of the head attitude and direction detected by the head motion tracking sensor, are sent to the camera angle controller. The latter calculates the scaled operator attitude in order to be within the cone of visibility. This cone corresponds to the scaled orientation limits of the catheter head system. Similarly, the motion commands that are calculated on the basis of the joystick position are sent. The joystick has 2-DOF for forward-backward and left-right, and a button which is pushed when the operator wants to select a reference image point. The real image captured by micro-vision camera is displayed on the HMD. 5.2. Strategy for Visual Inspection The main strategy for in-pipe inspection process flow is described as follows: Step1: The catheter head is inserted in the pipe and visual inspection is performed through the operator head motion attitude. Step2: The supervisor guides a cursor on a monitor (using a computer mouse) which the visual tracking system accepts as a control input to a visual servoing marquer. Step3: Focus on the detected crack. Step4: A gradient method is applied in order to extract effect regions from the pipe surface image. The main interest is to identify the points where the variation in intensity levels are high, regardless of the lighting conditions. Step5: Ultrasonic characterization of the crack is performed under visual tracking due to instabilities of the catheter head through antagonistic effects (pressure, temperature, liquid flow). Step6: Reconstruction of a 3D-graphic representation of the failure (2D location and depth) through sensor fusion at the pixel level.
(a)
(d)
(b)
(e)
(c)
(f)
Figure 14. Images of a pipe with a 1mm hole. The location of the hole is marked with a small square. Images from (a) to (c) are acquired with a standard lighting system when the catheter head is inserted. The crack can be easily identified (white spot). Note that the spot next to the white spot is created by light reflection and it is not a defect. Images from (d) to (f) shows the efficiency of visual tracking. The white spot is efficiently tracked during attitude motion of the HMD system for further visual inspection.
5.3. Experiments Some experiments have been performed in an nonflooded metallic pipe (20mm diameter) to test the efficiency and robustness of the remote visual tracking system in real operating conditions (see Fig.14 ).
5. CONCLUSION This paper has shown the first implementation of an integrated design of a multifunctional micro catheter head for in-pipe industrial inspection. The first step of integration of modeling and vision-based control issues have addressed. An extended Preisach methodology has proven to be very efficient for both modeling and control aspects. However, further work will be carried out in order to implement adaptive control of hysteretic systems in order to match with the in-pipe variation of operating conditions. Visual tracking experiments made in a non-flooded pipe has shown that it was possible to decouple attitude head motion of the operator when the visual tracker is switch on. The next-term objective (Step 5 and 6) of this research is to assess partially flooded millimeter-sized pipes using the described multi-sensor system.
6. REFERENCES [1]: A. Sadamoto et al. “Wireless micromachine for in-pipe visual inspection and the possibility of biomedical applications”, 32nd Int. Symposium on Robotics,19-21 April, Seoul, 2001, pp.433-438. [2]: S. Kumar, I. M Kassim, V.K. Asari, “Design of a vision-based microrobotic colonoscopy system”, Advanced Robotics, Vol.14, No.2, 2000, pp.87-104. [3]: K. Ikuta, M. Tsukamoto, S. Hirose, ”Mathematical Model and Experimental Verification of Shape Memory Alloy for Designing Microactuator”, IEEE Int. Conf. on Rob. and Automation, 1988. [4]: H.Benzaoui, “Modélisation thermomécanique et commande d’actionneurs en alliage à mémoire de forme pour la microrobotique”, PhD dissertation of university of FrancheComté, 1998. [5]: R.B. Gorbet, D.W.L. Wang, K.A. Morris, “Preisach Model Identification of a Two-Wire SMA Actuator”, IEEE Int. Conf. On Robotics and Automation, 1998. [6]: G.V.Webb, D.C. Lagoudas, A. Kurdial “Hysteresis Modeling of SMA Actuators for Control Applications”, Jour. of Intelligent Material Systems and Structures, Vol.9, 1998, pp.432-448. [7]: R.B. Gorbet, “Control of Hysteretic Systems with Preisach Representation”, PhD thesis, University of Waterloo, 1997. [8]: J. Forêt, A. Ferreira, J-G. Fontaine, “GA-based control of a binary-continuous joints of a multi-link micromanipulator using SMA actuators”, IEEE Int. Conference on Robots and Intelligent Systems, Oct. 1-4, Lausanne, Switzerland, 2002, pp.1736-1741. [9]: I.D. Mayergoyz, “Mathematical models of hysteresis”, Springer-Verlag, New-York, 1991. [10] : B. Espiau, F. Chaumette, and P. Rives, “A new approach to visual servoing in robotics”, IEEE Trans. on Robotics and Automation, 8(3) June 1992, pp. 313-326. [11]: P. Ge, M. Jouaneh, “Generalized Preisach model for hysteresis nonlinearity of piezoceramic actuators”, Precision Engineering, Vol.20, 1997, pp.91-111. [12]: D. Song, C.J. Li, “Modeling of piezoactuator’s nonlinear and frequency dynamics”, Mechatronics, Vol.9, pp.391-410, 1999.
4287
