Aerial Robotics Vision-based control for Vertical Take-Off and Landing UAVs Toulouse, October, 2 nd, 2014 Henry de Plinval (Onera - DCSD) collaborations with P. Morin (UPMC-ISIR), P. Mouyon (Onera), T. Hamel (I3S), L. Burlion (Onera), C. Samson (Inria), Onera DCSD 2
Plan I Introduction II Problem setting & models III Results 1 Principles 2 Linear approaches 3 Nonlinear approaches 4 Complements Figure 1 : Onera Rmax UAV IV Conclusion VTOL UAV Visual servoing - 10/02/2014-2/19
I - Context : why vision? Reasons : Rich information Sensors unavailibility (e.g. GPS jamming) Cost, mass, size efficiency Relative navigation Figure 2 : The bee, Portelli 2011 Applications examples : Target following Building surveillance / IR cartography Refueling Urban navigation Figure 3 : Vehicle tracking (ONERA-DCSD) How to compute relevant information from image for use in the control loops VTOL UAV Visual servoing - 10/02/2014-3/19
I - Objective & Difficulties Objective : control a VTOL UAV based on a single videocamera and inertial measurements (angular velocity, acceleration) only, knowing an image of the observed scene. Difficulties : Lack of knowledge : UAV position and orientation, target distance and orientation = "extreme case" Wealth of the visual information ; Under-actuation : 4 control inputs (thrust, torques), 6 deg. of freedom (rotation, translation) Robustness : wind, measurement noise, etc. Nonlinearities : rotations, saturations Semi-global stabilization Figure 4 : UAV Hovereye (Bertin) Each problem considered appart from the others has been studied ; the complete problem much less. VTOL UAV Visual servoing - 10/02/2014-4/19
I - State of the art Visual servoing for VTOLs UAVs : Pose estimation (Shakernia, 1999) Vision based orders (Franceschini, 2004) Direct use : Bounded orientation control (Metni, 2003) Quadrirotor control with image moments (Ozawa & Chaumette, 2011) Aircraft landing based on homography (Gonçalvès, 2009) Observation and nonlinear control (Le Bras, 2010) Discussion : Assumptions, computations Low level control : which sensors? Complementary measurements Local result Linear assumption knowledge about the environment VTOL UAV Visual servoing - 10/02/2014-5/19
II - Considered scénario The problem UAV flying in front of a scene Scene assumed to be locally planar Goal : join an equilibrium point Reference / current image No more information : target, size, distance unknown Only image, inertia : sometimes translational velocity position, orientation unmeasured T d * P Z * x* y* z* x y z Fig. 5 : problem illustration, Fig. 6 : Chaumette & Hutchinson, 2006 VTOL UAV Visual servoing - 10/02/2014-6/19
II - UAV dynamic model Model with aerodyn. effects and mass induced torque neglected : ṗ = R v (1) Ṙ = R S(ω) (2) m v = m S(ω)v Tb 3 + m g R T b 3 (3) J ω = S(ω) J ω + Γ (4) Figure 7 : helicopter with p position error in inertial frame v, ω linear and angular velocities in body frame R rotation matrix from body to inertial frame Thrust T and torque Γ inputs m mass, J inertia, g gravity, b 3 = [0, 0, 1] T VTOL UAV Visual servoing - 10/02/2014-7/19
II - Measurement model Available sensors : Thus : One videocamera Inertial measurements Depending on the case : translational velocity Reference image / current one ; Planar target assumption ; Comparaison : homography H = R T 1 d RT p n T y* z*, x* z x d * y P y* * z* R rotational error p position error in inertial frame d distance to target object n normal vector to the object n * x VTOL UAV Visual servoing - 10/02/2014-8/19
III.1 - Philosophy of the approach Dynamics + sensors + processing input-output relationship. How to exploit this relationship? Usually : jacobian, but : Linear assumption ; Lack of knowledge of these variables ; Following approach retained : Extract from image a measurement close to position ; Exact position cannot be obtained in this case : σ(p, δ) p with σ(0, δ) = 0 ; Define a control law robust, which manages this positioning uncertainty ; Robust approach using an uncertain position measurement VTOL UAV Visual servoing - 10/02/2014-9/19
III.2 - Linear approach Principle and difficulties : Movements around equilibrium : the models are linearized ; Maintained difficulties : lack of knowledge of distance and orientation of the target, UAV position and orientation ; Retained appproach : Define an error vector close to position / orientation ; Law mimics classical control law, when the state is known ; Error vector in the image inspired by Benhimane & Malis 2007 : e = [e ν, e ω ] T e ν = (H I )m, e ω = vex(h H T ) from homography H and a direction m = [0, 0, 1] T ; To cope with dynamics, ( new error vector needed : 2I3 S(m ē = Me, M = ) ( ) ) ep S(m, ē = ) I 3 e Θ VTOL UAV Visual servoing - 10/02/2014-10/19
III.2 - Linear approach Control law definition (unique measurements : H, ω) : Dynamic extension to replace the unmeasured velocity Nested loops : vertical, yaw, horizontal T = m (g + k 1 ē 3 + k 2 ν 3 ) ν = K 7 ν ē p γ d = K 5 ē p K 6 ν ω d = K ( 4 g gēθ + S(b 3 )γ d) ( Γ = JK 3 ω ω d ) Results : 1 For all bound lower than the distance to target at the reference pose, exist gains which stabilize (locally exponentially) for all greater distance 2 Tuning heuristics for the gains for performance VTOL UAV Visual servoing - 10/02/2014-11/19
III.2 - Linear approach : simulations 1.5 1 0.5 0-0.5 Figures legends -1 0 10 20 30 40 50 15 10 5 0-5 Position (m) Orientation ( ) -10 0 10 20 30 40 50 Temps (s) Obtained result for d = 1, 5m 1 0-1 1.5 1 0.5 0 Position (m) 0 10 20 30 40 50 Orientation ( ) -0.5 0 10 20 30 40 50 Temps (s) Obtained result for d = 15m The same gains stabilize over a large window of values for d VTOL UAV Visual servoing - 10/02/2014-12/19
III.3 - Nonlinear approach : principle Principle and difficulties : Define an error vector adapted to the nonlinear context ; Consider fully-actuated then under-actuated ; Measurements model : σ = S (Hb 2 ) Hb 3 Hb 1 = R T Mp γ = ghb 3 = gr T b 3 v = R T ṗ ω With : σ uncertain position meas. M n 1 d I + S( n 2 d b 3 ) uncertain γ vertical meas. in body frame v, ω linear/angular velocities Reminder : R p unknown Vertical target assumption u = k 1 m sat δ (σ) k 2 m sat δ(v) VTOL UAV Visual servoing - 10/02/2014-13/19
III.3 - Nonlinear approach : principle Principle and difficulties : Define an error vector adapted to the nonlinear context ; Consider fully-actuated then under-actuated ; Measurements model : σ = S (Hb 2 ) Hb 3 Hb 1 = R T Mp γ = ghb 3 = gr T b 3 v = R T ṗ ω With : σ uncertain position meas. M n 1 d I + S( n 2 d b 3 ) uncertain γ vertical meas. in body frame v, ω linear/angular velocities Reminder : R p unknown Vertical target assumption u = k 1 m sat δ (σ) k 2 m sat δ(v) VTOL UAV Visual servoing - 10/02/2014-13/19
III.3 - Under-actuated case, results Result : one control design, with complex formulation so that Exist gains which render the system asymptotically stable and locally exp. stable, domain { µ(t = 0) µ(t = 0) b 3 } ; Proof by Lyapunov function : stability & robustness. Let us define the law, with k 4 >> 0 (measurements : H, σ, γ, v, ω) : µ γ + k 2 sat δ(v) + k 1 sat δ (σ) ω1 d = k 4 µ 2 ω 2 d = k 4 µ 1 ω3 d = k 5 H 21 { ( Γ = k6 ω d ω ) T = m µ 3 Similar properties (singular perturbations theory) ; VTOL UAV Visual servoing - 10/02/2014-14/19
III.3 - Under-actuated case, results Result : one control design, with complex formulation so that Exist gains which render the system asymptotically stable and locally exp. stable, domain { µ(t = 0) µ(t = 0) b 3 } ; Proof by Lyapunov function : stability & robustness. Let us define the law, with k 4 >> 0 (measurements : H, σ, γ, v, ω) : µ γ + k 2 sat δ(v) + k 1 sat δ (σ) ω1 d = k 4 µ 2 ω 2 d = k 4 µ 1 ω3 d = k 5 H 21 { ( Γ = k6 ω d ω ) T = m µ 3 Similar properties (singular perturbations theory) ; VTOL UAV Visual servoing - 10/02/2014-14/19
III.3 - Simulation results 1 0.5 0-0.5-1 { φ0 = 78, θ 0 = 67, ψ 0 = 78 p 0 = [0.7, 0.2, 1] m -1.5 0 5 10 15 20 25 100 50 0-50 Position (m) Orientation ( ) -100 0 5 10 15 20 25 Temps (s) Result obtained with the complex control law Both control law stabilize the nonlinear system. VTOL UAV Visual servoing - 10/02/2014-15/19 1 0.5 0-0.5-1 Figures legends -1.5 0 5 10 15 20 25 100 50 0-50 Position (m) Orientation ( ) -100 0 5 10 15 20 25 Temps (s) Result obtained with the simplified control law
III.4 - Linear complements Problematics : Unmodeled dynamic rejection (e.g. wind) ; Videocamera misalignment ; Proposed solutions : Accelerometers measurements exploitation ; Modified error vector definition ; Numerical approaches to robustness ; Robust synthesis & analysis techniques VTOL UAV Visual servoing - 10/02/2014-16/19
III.4 - Nonlinear complements Problematics : Avoid the use of a velocity measurement ; Avoid the vertical target assumption ; Improve performances ; Follow a trajectory ; Proposed solutions : Exploit the optical flow ; Vertical direction estimation techniques ; Robustness to the lack of verticality ; Adaptation through online estimation of the normal vector to the plane ; Extension of the model to trajectory following ; VTOL UAV Visual servoing - 10/02/2014-17/19
IV - Conclusion : prospectives Summary : Almost global, nonlinear stabilization ; Sensors context minimalist (vision, inertia) ; Generalization to trajectory following ; Improvements prospectives : Refined UAV modelling ; Exhaustive use of the tools for robust synthesis & analysis ; Maintain target inside the videocamera field of view ; Related projects at Onera : ANR project Visioland European FP7 project Aeroceptor Onera internal project Azur Figure 8 : Onera Turbo UAV VTOL UAV Visual servoing - 10/02/2014-18/19
IV - Références Références : H. de Plinval, P. Morin, P. Mouyon, T. Hamel Visual servoing for underactuated VTOL UAVs : a linear, Homography-based approach., IEEE International Conference on Robotics and Automation, 2011. H. de Plinval, P. Morin, P. Mouyon. Nonlinear control of underactuated vehicles with uncertain position measurements and application to visual, American Control Conference, 2012. H. de Plinval, P. Morin, P. Mouyon, T. Hamel Visual servoing for underactuated VTOL UAVs : a linear, Homography-based framework, International Journal of Robust and Nonlinear Control, 2013. H. de Plinval, A. Eudes and P. Morin, Control and estimation algorithms for the stabilization of VTOL UAVs from mono-cameras measurements, to appear in Aerospace Lab Journal, 2014. VTOL UAV Visual servoing - 10/02/2014-19/19