Nonlinear Image-Based Visual Servo controller for automatic landing guidance of a fixed-wing Aircraft Florent Le Bras, Tarek Hamel, Christian Barat and Robert Mahony
Abstract— This paper describes a nonlinear Image-Based Visual Servo controller for automatic landing of a fixed wing aircraft in presence of a wind gust. Only 2D image features are exploited from the image of the runway to perform a typical path for landing maneuver (alignment, glide and flare maneuvers). The visual measurements for the alignment are the two lines delimitating the runway using the so-called bi-normalised Plucker ¨ Coordinates parametrisation. For the glide and flare maneuvers, additional 2D features such as the coordinates of the first two corners (or some specific markers) of runway are used. The controller is unique for the alignment and the glide maneuver; it exploits the full nonlinear dynamics of the airplane guidance problem to derive a control Lyapunov function along with an estimate of the wind gust. Finally simulation results are presented to illustrate the performance of the control approach.
I. I NTRODUCTION Recent advances in technology have lead to a growing interest in aerial robotic applications where, historically, payload constraints have severely limited autonomy and range of small scale Unmanned Aerial Vehicles (UAV). There has been a significant effort placed on the development of autonomous fixed-wing and rotary-wing aircrafts in the last ten years for a wide variety of aircraft sizes and characteristics going from micro to jet sized [1], [2], [3], [4], [5]. The highly coupled dynamics and implementation of such systems provide an ideal testing ground for sophisticated nonlinear control techniques. A key issues arising is the difficulty of accurately measuring position of the vehicle with respect to its local environment. A vision system is a cheap, light, passive and adaptable sensor that can be used to provide robust relative pose information. Using an airborne camera as the primary sensor for relative position leads to a visual servo control problem [1], a distinct approach from typical Unmanned Aerial Vehicle tasks that involve navigation between GPS way-points with little regard to the local topography of the environment. Navigation and control of vehicles in partially known or unknown environments is a central topic in the field of area of sensor-based control. A typical aircraft is classically equipped with an inertial measurement unit (IMU) consisting of accelerometers and rate gyros, along with barometric pressure (or air speed) measurements and, from few years, a GPS system. Using a camera to perform visual servo control has the potential to become an important enabling technology for aerial robotic vehicles and in particular for automatic landing and take-off F. Le Bras is with the Laboratoire de Recherches Balistiques et A´erodynamiques (LRBA) of the French D´el´egation G´en´erale pour l’Armement, Vernon, France (e-mail:
[email protected]) T. Hamel and C. Barat are with I3S UNSA-CNRS, Nice-Sophia Antipolis, France, (e-mail:
[email protected] and
[email protected]) R. Mahony is with the Australian National University, Department of Engineering, ANU, ACT, 0200, Australia, (e-mail:
[email protected])
of an aircraft on any helipad/runway without any external assistance or ground equipment. For the control design using visual features two main classes have been identified [6]. The first one, termed Position-based visual servo (PBVS), involves reconstruction of the system position based on visual information and leads to a classical dual estimation and control problem. The second class, termed Image-Based Visual Servoing (IBVS) is posed directly in terms of image features. The controller is designed to maneuver the image features towards a goal configuration that implicitly solves the original Cartesian motion-planning problem. More robustness is expected using IBVS processes, however, the control problem is more complex due to the nonlinearities of image features dynamics that incorporate unknown image depth. Moreover, most aerial robotic systems may be modeled as under-actuated rigid-body dynamics, along with aerodynamic effects and possibly additional dynamics due to actuators. Such systems are inherently difficult to control and increase significantly the complexity of image-based visual servo control design. Visual servoing based on linear features allows interesting capabilities such as following accurately a road during landing approach or an observation mission. The feasibility of tracking lines extracted from the video-feed has been studied and experimented by a number of authors [7], [8], [2]. In [9] IBVS strategies along with complementary sensor information have been proposed to stabilize a blimp along a linear path. In [5] an IBVS control scheme, along with measurement of inertial velocity, has been developed to stabilize the lateral and the vertical motions of an aircraft along a runway. By linearizing the dynamics around a desired flight position, two linear and decoupled controllers are introduced in [5] to stabilize the longitudinal and the lateral motion respectively. The stability of the system is however guaranteed only in a limited domain and the control approach appears to be sensible to wind disturbance and initial position w.r.t the desired set point. This paper proposes a new IBVS controller for autolanding of an aircraft in airports without any external assistance or ground equipment. Autolanding maneuver is the most demanding phase of a flight due to the precision required and the increased workload. The control architecture is decoupled into an inner and outer controller. The outer controller stabilizes the translational dynamics (or guidance dynamics) by using visual data along with angle of attack and roll angle set point as control inputs. The inner controller provides high gain stabilization of the vehicle angle of attack, and sideslip angle, and roll angle based on direct measurement of the IMU and pitot tubes, along with actuation of the aircraft control surfaces. For a typical aircraft, the time-scale separation between the two loops is sufficient that the interaction terms can be ignored
in the control design. In this paper, only the outer controller is considered. The image features used are the bi-normalized Pl¨ucker coordinates of the lines delimitating a runway [10]. The paper is arranged into four sections. Section II presents the dynamic model of an airplane, and introduces the autolanding maneuvers (alignment, glide and flare). Section III presents the image features and derives a Lyapunov based controller for the considered problem. Section IV presents simulation results for the full nonlinear dynamics of an aircraft. The final section provides a short summary of conclusions and an indication of future research directions.
Fig. 1. Representation of the aircraft along with the frames and velocity notations
II. M ODELING B. Guidance, Control and Navigation structure
A. Aircraft dynamics We consider a fixed wing business jet (mass m = 16000kg, wing span b = 20 m) studied in the frame of the European project PEGASE (http://dassault.ddo.net/pegase/). To describe the motion of the aircraft two reference frames are used: a local North-East-Down inertial frame, denoted I = (EN , EE , ED ), located on the runway and a body-fixed frame B = (Ex , Ey , Ez ) attached to the center of mass of the aircraft (see Fig.1). The position of the aircraft center of mass in I is denoted ξ. The inertial velocity, denoted v ∈ B, is the sum of the wind velocity vw and the so-called airspeed va : v = va + vw
(1)
The wind is assumed to be constant w.r.t the inertial frame. The orientation of the aircraft is given by the orientation matrix R ∈ SO(3) from B to I, usually parameterized by Euler0 s angles ψ, θ, φ (yaw, pitch, roll). Finally let Ω = (p, q, r)T be the angular velocity of the aircraft defined in B. Denoting I the inertial matrix of the vehicle, the flight dynamics are1 ([12]) ξ˙ = R(va + vw ) mv˙ = −sk(Ω)mv + Fearth + Faero + Fengine R˙ = Rsk(Ω)
(2) (3)
˙ = −sk(Ω)IΩ + Γaero IΩ
(5)
(4)
where Fearth = mgRT ED denotes the gravitational force, (Faero , Γaero ) denotes the aerodynamic effects, and finally Fengine = T.Ex , where T is the thrust of the engines turbines. The aerodynamic forces are expressed as a function of the ¯ and of (α, β), the angle of attack and dynamic pressure, Q, the sideslip angle ([12]): ¯ (CX (α, β), CY,β β, CZ,α (α − α0 )) Faero = QS
(6)
where S is the reference surface of the airplane, and (CX , CY,β , CZ,α ) are the so-called aerodynamic coefficients expressed in the body-fixed frame (α0 is the angle-of-attack nullifying the lift force). 1 sk(Ω)
denotes the skew-symmetric matrix such that sk(Ω)x = Ω × x.
The control actuators of dynamics (2-5) are the thrust of engines turbines T and orientations of control surfaces (δl , δm , δn ) that allow to design the torque Γaero as desired. The classical approach used to stabilize a fixed-wing aircraft along a desired path consists in [13]: 1) regulating the norm of the airspeed Va = kva k on a desired forward velocity Vad (Vad is a constant value for each phase of the landing procedure maneuver), 2) regulating the sideslip angle β to zero, this implies that the turn manoeuvres should be coordinated by banking the airplane while turning, 3) stabilizing the aircraft translational dynamics (or Guidance dynamics) (2-3) by considering that the sideslip angle is null and that Va is constant along with (φ, α) are used as guidance control inputs. 4) stabilizing the orientation dynamics (also called Control dynamics) (4-5) through a fast inner-loop controller such that assignments in (φ, α, β) be correctly realized. The first item requires that the propeller thrust regulates the airspeed Va towards the desired value Vad . Note that the desired airspeed should be larger than the maximum between the wind amplitude (kvw k) and the lower speed threshold of the aircraft (Val ), so that dynamical system of the airplane, Eq.2-5 is controllable: Vad > max{kvw k, Val } In practice, this limitation comes from the engine capability and, the airplane shall not be used when the wind conditions are higher than a limit, identified through the airplane conception. In the sequel, the following assumption is done on wind velocity: Assumption 2.1: There exist ² ∈ (0, 1) such that: kvw k < ²Vad (7) Items 2) and 3) are realized through a standard inner-loop, whose description is omitted from this paper (further details on the control architecture are given in [14], [13]). Finally the guidance dynamics, item 4), can be simplified by considering that corresponding time constant are larger than those of the inner-loop controller and of the airspeed regulation. It allows to consider that the airspeed is constant and that the side-slip angle is null for the guidance control
problem, whose dynamics also become: ξ˙ = R(va + vw )
(8)
v˙ w = −sk(Ω)vw v˙ a = −sk(Ω)va + πva ua (α, φ)
(9) (10)
along with: va vaT (11) Va2 ua (α, φ) = mg cos θ sin φEy (12) ¯ Z,α (α − α0 ) + mg cos α cos θ cos φ)Eα + (QSC πva =Id −
3) Flare maneuver: when the airplane comes near to the runway (about 20 meters for a Jet sized aircraft), a specific flare maneuver begins to lower the glide-path angle and to align the airplane w.r.t the runway axis. 4) Taxiing: this last phase begins when the airplane tires touch the runway, the airplane is then rolling while reducing its velocity to zero. In the sequel, we focus on the two first maneuvers (the flare phase is considered, in this paper, as particular situation of the glide-slope maneuver). We describe first the visual features used and then we propose a unique controller for the considered maneuvers.
v ×E
where Ey = (0, 1, 0)T and Eα = aVa y (note that hEy , va i = 0 when β = 0). The corresponding assignment in (α, φ) can be calculated through a Nonlinear Dynamic Inversion of the velocity dynamics (3), which incorporates the aerodynamic model of the aircraft (6) within its architecture [14]. Finally, the proposed controlled strategy will depend only on the measurement of the following variables: • the Euler angles (φ, θ, ψ) and the angular velocity Ω, both provided by an Inertial Measurement Unit, • the norm of the airspeed, angle of attack and sideslip angle (|Va |, α, β), measured by pitot tubes and pressure intakes, and providing a direct measurement of va , • visual linear features extracted by a vision system. The main unknown parameters are the aircraft position and the wind velocity. The latter will be estimated by the proposed control algorithm, while the visual features used allow to stabilise the aircraft along the landing path without complementary position measurement. C. Automatic landing The control task considered consists in performing an automatic landing of the airplane using a vision-based controller.
III. IBVS C ONTROL STRATEGY A. Visual features The visual features considered are the images of a collection of n ≥ 2 parallel lines along with some specific marks on the runway. The unit direction vector of the lines is denoted u (resp. U ) in I (resp. B). We assume that camera-fixed frame coincides with B and that the image features remain in the camera field of view throughout the flight. Two different position-like error terms are derived from the visual features, the first one is based on the so-called binormalised Pl¨ucker coordinates of the observed parallel lines in order to perform the alignment phase; the second one is based (in addition to the measurements of the border lines) on the measurement of two landmarks on the runway to perform the glid-slope maneuver (or the flare maneuver). Analogously to the alignment phase, the bi-normalised Pl¨ucker coordinates of two virtual lines, derived from the observed landmarks, are used for the remaining maneuvers (glid-slope and flare phases). Pl¨ucker coordinates representation is an explicit model of the lines, which will avoid technical complications in the development of the proposed control approach. How to model image features using the Pl¨ucker coordinates (see Figure 3) is described in [11]. The approach chosen for the alignment phase lies on the fact that hi ∈ B, a vector of the Pl¨ucker coordinates, can be measured directly with the vision system [10]: hi =
Fig. 2.
Representation of the different phases of the landing trajectory
The landing maneuver is composed of four phases (Fig. 2): 1) Alignment: In this phase, the aircraft has to be aligned with the runway at a fixed desired altitude from the ground. For this maneuver, the border-lines of the runway will be used as the primary pose information. 2) Glide-slope maneuver: In this phase, the airplane follows a straight-line descending path, while keeping the alignment with respect to the runway axis. For this maneuver, the image features are the coordinates of some specific marks on the runway and the runway axis.
Xi × U xi × u = ; |Xi × U | |xi × u|
(13)
where Xi (resp. xi ) denotes the vector between the camera and the i-th parallel line (resp. the image of this line) and u is the unit direction of the line’s image. For the glide-slop (or the flare) phase, the image features proposed are the coordinates of some specific marks on the runway and the runway axis. For instance, we propose to use the first corners of the runway and the desired descending direction. More precisely, the image features used are the Pl¨ucker-like coordinates of the virtual line derived from the image coordinates of the first corners of the runway and the desired descending direction. Let gi ∈ B be the following image feature: gi =
pi × UD Pi × UD = |Pi × UD | |pi × UD |
(14)
be not touched by the airplane. For practical application [15], the corresponding area will be defined a priori for a set of interesting configurations and then, a basin of attraction can be proposed so that closed-loop trajectories satisfy (18). B. Problem formulation The IBVS stabilization task consists in driving exponentially the centroid vector q to a desired one q d (i.e δ := q − q d → 0) q d = R T bd
Fig. 3.
where bd is a constant vector that encodes the desired position information. Combining the dynamics of the above equation, (16), and an estimation of the wind velocity, dynamics of the error term δ can be expressed as:
Pluecker coordinate h1 of a line.
where Pi (resp. pi ) denotes the vector between the camera and the i-th corner of the runway (resp. the image of this point). The vector UD is the unit direction of the glideslop in the body-fixed frame. This direction UD belongs to the plane spanned by the two vectors RT Ez and U (UD ∈ span{RT Ez , U }). Given that the lines are horizontal, their images provide direct measurements of U (formally obtained ×hj by U = hhii ×hj for i 6= j) and RT Ez . Note that each vector gi benefits from all properties of the vector hi . They will be termed the virtual Pl¨ucker coordinates. In the sequel, definitions and properties are presented for the alignment phase; they can be directly transposed to the remaining maneuvers by using gi instead of hi . The positioning error is dealt through the so-called centroid vectors q, derived from the vision-based measurements (13) (or (14)), in bijection with ξ × RU (resp. ξ × RU d ). The centroid vector is the sum of the Pl¨ucker coordinates: n X q= hi (15) i=1
Camera-fixed frame being aligned with the body-fixed frame, the dynamics of q is given as proven in [11]: q˙ = −sk(Ω)q − Q(v × U ) where Q is the Jacobian matrix: X Q= (Id − hi hTi )/|Xi × U |
(16)
(17)
This matrix is definite positive (Q > 0) and can not be measured since no measurement are available for |Xi ×U |. The definite positiveness of Q is exploited in the control design; this avoids the necessity of estimating or approximating it. However, some bounds are required on Q for the trajectories considered to avoid an ill-conditioning into the control design. In particular, the following assumption is done: Assumption 3.1: Assume that there exists two positive scalars (qm , qx ) > 0 such that: ∀ξ 0 ∈ D, qm Id < Q(ξ 0 ) < qx Id (18) This assumption is classic for IBVS control schemes; recalling (17), qm limits the distance between the airplane and the runway, while qx , the upper-bound, implies that the ground
δ˙ = −sk(Ω)δ − Q [(va + vw ) × U ] = −sk(Ω)δ + Q [sk(U )va − vˆw − v˜w ]
(19)
where v˜w = vw × U − vˆw and vˆw is an an estimate of vw × U . Choose the following dynamics for vˆw : vˆ˙ w = −sk(Ω)ˆ vw + Pvˆw πU uw ,
vˆw (0) = 0
(20)
where uw acts as the innovation of the wind estimator and: s µ ¶ µ ¶ T kˆ vw k2 vˆw vˆw 1+² 0 0 Pvˆw = ² Va 1 − 02 2 Id − 02 2 , ² ∈ ,1 ² Va ² Va 2 (21) This estimation of the wind velocity remains in the plan orthogonal to U and insures that the norm of the wind estimate kˆ vw k is strictly lower than ²0 Va . Indeed it can be verified that: y vˆw = ²0 Va p , where y˙ = −sk(Ω)y + πU uw 1 + kyk2 y(0) = 0 C. Control design First, consider the following storage function, where k1 is a positive control gain: S1 = |δ|2 +
4 2 T v˜w δ + 2 |˜ vw |2 k1 k1
(22)
Choosing the innovation term of the wind estimate, uw , as follows: uw = k2 δ, k2 > 0 (23) let’s define vad the virtual commanded airspeed as: p vad = sk(U )(k1 δ − vˆw ) + Va2 − kk1 δ − vˆw k2 U
(24)
where it is straightforward to verify that kvad k = Va . Knowing that |δ| < 2n and |ˆ vw | < ²0 Va , and choosing k1 such that: k1
0
| 1 − cos(va , vad ) < 2 − ²a
Specify the control input as follows: ua (α, φ, ) =
k3 πva vad kva + vad k2
(30)
along with (23) as innovation of the wind estimation. Then there exists positive gains (k1 , k2 , k3 , K) such that the function L = S1 + KS2 is a Lyapunov function for the guidance dynamics that guarantees that the closed-loop solutions exist for all time and the error signals (δ, δ2 , V˜w ) converge exponentially to zero. Proof: First consider the first term of of S˙ 2 , (29), and introduce control (30). Given that kvad k = Va , it can be written as : −k3
vad T πva vad k3 (Va4 − hvad , va i2 ) = − = −k3 kva − vad k2 kva + vad k2 Va2 kva + vad k2
Then using Scharwz inequality X T Y ≤ 12 (|x|2 + |y|2 ), the derivative of L can be written as: µ ¶T µ ¶µ ¶ v˜w Kk12 4k2 v˜w ˙ L≤− δ+ (k1 − )Q − Pvˆw δ+ k 2 k1 k1 µ 1 ¶ 2 K − k2 − δ T Pvˆw δ (31) k1 2 µ 1 − k3 Kkδ2 k2 + Kδ2T − sk(U )Qsk(U )+ Kk1 ´¶ ³ k2 T T δ2 M d Pvˆ M d − k1 Mvad Q sk(U ) − k1 Mvad 2 va w va
To determine an appropriate gain scheduling, it is of interest to consider the following relationship: kMvad k ≤ kMvad k∞ = 1 +
Va (1 − ²0 )Va − 2nk1
(32)
where kMvad k∞ has been obtained through tedious but straightforward calculations. Recalling assumption 3.1, one can ensure that L˙ is upper-bounded by a definite negative expression of the (Pvˆw δ, δ −k1−1 v˜w , δ2 ) as soon as the control gains satisfy: 2 k1 µ ¶ Kk1 qm k 2 k2 < 0 1 1 − 4² Va 2 µ ¶ qx k 2 ² 0 Va k3 > + qx k12 kMvad k∞ + qx k1 + kMvad k2∞ Kk1 2 K
