Adaptive Control. Bo Bernhardsson and K. J. Åström. Department of Automatic Control LTH, Lund University

Transcription

1 Department of Automatic Control LTH, Lund University

2 1 Introduction 2 Model Reference 3 Recursive Least Squares Estimation 4 The Self-Tuning Regulator 5 Real lers 6 Summary

3 Introduction Adapt to adjust to a specified use or situation Tune to adjust for proper response Autonomous independence, self-governing Learn to acquire knowledge or skill by study, instruction or experience Reason the intellectual process of seeking truth or knowledge by infering from either fact of logic Intelligence the capacity to acquire and apply knowledge In Automatic Control Automatic tuning - tuning on demand Gain scheduling Adaptation - continuous adjustment

4 Parameter Variations Robust control Find a control law that is insensitive to parameter variations Gain scheduling Measure variable that is well correlated with the parameter variations and change controller parameters Adaptive control Design a controller that can adapt to parameter variations Many different schemes Model reference adaptive control The self-tuning regulator L 1 adaptive control (later in LCCC) Dual control Control should be directing as well as investigating!

5 A Brief History of Early work driven adaptive flight control The brave period: Develop an idea, hack a system and make flight tests. Several adaptive schemes emerged no analysis Disasters in flight tests Emergence of adaptive theory Model reference adaptive control emerged from stability theory The self tuning regulator emerged from stochastic control theory Microprocessor based products 1980 Novatune Auto-tuners for PID control 1980 Robustness 1990 L1-adaptive control - Flight control 2010

6 Adaptive Schemes Parameter adjustment Controller parameters Setpoint Controller Control signal Plant Output Two loops Regular feedback loop Parameter adjustment loop Schemes Model Reference Adaptive Control MRAS Self-tuning Regulator STR Dual Control

7 Dual Control - Feldbaum u c Nonlinear control law u Process y Calculation of hyperstate Hyperstate No certainty equivalence Control should be directing as well as investigating! Intentional perturbation to obtain better information Conceptually very interesting Unfortunately very complicated - state is conditional probability distribution of states and parameters Helmerson KJA, worth a second look?

8 1 Introduction 2 Model Reference Model reference - command signal following The MIT rule -sensitivity derivatives Rules derived from Lyapunov theory Various fixes 3 Recursive Least Squares Estimation 4 The Self-Tuning Regulator 5 Real lers 6 Summary

9 Model Reference Model y m Controller parameters Adjustment mechanism u c Controller u Plant y Linear feedback frome=y y m is not adequate for parameter adjustment! The MIT rule Many other versions dθ dt = γe e θ

10 An Example Process Model Controller ideal parameters dy m dt dy dt = ay+bu = a m y m +b m u c u(t)= θ 1 u c (t) θ 2 y(t) θ 1 = θ 0 1 =b m b θ 2 = θ 0 2 =a m a b

11 MIT Rule The error Approximate bθ 1 e=y y m, y= u c p= dxt p+a+bθ 2 dt e θ 1 = b p+a+bθ 2 u c e b 2 θ 1 b = θ 2 (p+a+bθ 2 ) 2u c= y p+a+bθ 2 p+a+bθ 2 p+a m Hence dθ 1 dt ( ) am = γ u c e, p+a m dθ 2 dt ( ) am = γ y e p+a m

12 Block Diagram u c Π G m (s) + Σ u G(s) y Σ + e θ 1 Π γ s θ 2 γ s a m a m s + a Π Π m s + a m dθ 1 dt ( ) am = γ u c e, p+a m dθ 2 dt ( ) am = γ y e p+a m Examplea=1,b=0.5,a m =b m =2.

13 y m y Simulation Input and output θ 1 θ 2 Time u Time Parameters γ =5 γ =1 γ =0.2 Time γ =5 γ =1 γ =0.2 Time

14 Adaptation Law from Lyapunov Theory The idea Determine a controller structure Derive the Error Equation Find a Lyapunov function Determine an adaptation law A first order system State feedback Output feedback Passivity Error augmentation>

15 Process model Desired response Controller The error First Order System dy m dt dy dt = ay+bu = a m y m +b m u c u= θ 1 u c θ 2 y e=y y m de dt = a me (bθ 2 +a a m )y+(bθ 1 b m )u c Candidate for Lyapunov function V(e,θ 1,θ 2 )= 1 ( e bγ (bθ 2+a a m ) ) bγ (bθ 1 b m ) 2

16 Derivative of Lyapunov Function V(e,θ 1,θ 2 )= 1 2 ( e bγ (bθ 2+a a m ) ) bγ (bθ 1 b m ) 2 Derivative of Lyapunov function dv dt =ede dt +1 γ (bθ 2+a a m ) dθ 2 dt +1 γ (bθ 1 b m ) dθ 1 dt = a m e ( ) γ (bθ dθ2 2+a a m ) Adaptation law + 1 ( γ (bθ dθ1 1 b m ) dt + γu ce dt γye ) dθ 1 dt = γu c e, dθ 2 dt = γye

17 Lyapunov (left) vs MIT Rule (right) u c θ 1 Π G m (s) + Σ u G(s) Π y Σ + e u c θ 1 Π G m (s) + Σ u G(s) Π y Σ + e γ s θ 2 γ s γ s θ 2 γ s Π Π, a m a m s + a Π Π m s + a m Do the filters matter?

18 Simulation Process inputs and y outputs (a) m y (b) Time u Time Parameters θ 1 θ 2 Time γ =5 γ =1 γ =0.2 γ =5 γ =1 γ =0.2 Time

19 Bo Bernhardsson A A and K. m J. Åström =BL, Adaptive B m Control =BM State Feedback dx dt =Ax+Bu Desired response to command signals Control law dx m dt =A m x m +B m u c u=mu c Lx The closed-loop system dx dt =(A BL)x+BMu c=a c (θ)x+b c (θ)u c Parametrization Compatibility conditions A c (θ 0 )=A m, B c (θ 0 )=B m

20 The Error Equation Desired response Control law Error Hence dx m dt de dt =dx dt dx m dt dx dt =Ax+Bu =A m x m +B m u c u=mu c Lx e=x x m =Ax+Bu A m x m B m u c de dt =A me+(a A m BL)x+(BM B m )u c =A m e+(a c (θ) A m )x+(b c (θ) B m )u c ( ) =A m e+ Ψ θ θ 0

21 The Lyapunov Function The error equation de ( ) dt =A me+ Ψ θ θ 0 Try V(e,θ)= 1 ( γe T Pe+(θ θ 0 ) T (θ θ )) 0 2 dv dt = γ 2 et Qe+ γ(θ θ 0 )Ψ T Pe+(θ θ 0 ) Tdθ dt = γ ( ) dθ 2 et Qe+(θ θ 0 ) T dt + γ ΨT Pe whereqpositive definite and A T m P+PA m= Q Adaptation law and derivative of Lyapunov function dθ dt = γ ΨT Pe, dv dt = γ 2 et Qe

22 Extensions Output feedback Kalman Yakubovich Lemma - Strictly positive real systems Various tricks Augmented error Error normalization dθ dt = γe e θ L1 Adaptive control

23 1 Introduction 2 Model Reference 3 Recursive Least Squares Estimation 4 The Self-Tuning Regulator 5 Real lers 6 Summary

24 The Least Squares Method The problem: The Orbit of Ceres The problem solver: Karl Friedrich Gauss The principle: Therefore, that will be the most probable system of values of the unknown quantities, in which the sum of the squares of the differences between the observed and computed values, multiplied by numbers that measure the degree of precision, is a minimum. In conclusion, the principle that the sum of the squares of the differences between the observed and computed quantities must be a minimum, may be considered independently of the calculus of probabilities. An observation: Other criteria could be used. But of all these principles ours is the most simple; by the others we should be led into the most complicated calculations.

25 The Book

26 Recursive Least Squares u y S θˆ y t+1 = a 1 y t a 2 y t 1 + +b 1 u t + +e t+1 = ϕt T θ+e t+1 θˆ t = θˆ t 1 +K t (y t ϕ tˆ θ t 1 ) K t =P t 1 ϕ t (λ+ ϕ T t P t 1ϕ t ) 1 =P t ϕ t Many versions: directional forgetting, resetting,... Square-root filtering (good numerics!)

27 Persistent Excitation PE Introduce 1 c(k)=lim t t t u(i)u(i k) i=1 A signaluis called persistently exciting (PE) of ordernif the matrixc n is positive definite. c(0) c(1)... c(n 1) c(1) c(0)... c(n 2) C n =. c(n 1) c(n 2)... c(0) A signaluis persistently exciting of ordernif and only if 1 U=lim t t t (A(q)u(k)) 2 >0 k=1 for all nonzero polynomials A of degree n 1 or less.

28 Persistent Excitation - Examples A step is PE of order 1 (q 1)u(t)=0 A sinusoid is PE of order 2 (q 2 2qcosωh+1)u(t)=0 White noise PRBS Physical meaning Mathematical meaning

29 Lack of Identifiability due to Feedback y(t)=ay(t 1)+bu(t 1)+e(t), u(t)= ky(t) Multiply by α and add, hence y(t)=(a+ αk)y(t 1)+(b+ α)u(t 1)+e(t) Same I/O relation for allâandˆb such that â=a+ αk, ˆb=b+ α ˆ b Slope 1 k b True value a a ˆ

30 Lack of Identifiability due to Feedback Consider for example the standard feedback loop. add fig If the only perturbation on the system is the signaldwe have Y(s)= P(s) C(s)P(s) d, U(s)= 1+P(s)C(s) 1+P(s)C(s) d, and it thus follows thaty(s)= 1 C(s) U(s) any attempt to find a model relatinguandy will thus result in the negative inverse of the controller transfer function. However, ifd=0we have instead Y(s)= P(s)C(s) 1+P(s)C(s) (F(s)R(s) N(s)) U(s)= C(s) 1+P(s)C(s) (F(s)R(s) N(s)d, hencey(s)=p(s)u(s) and the process model can indeed be estimated.

31 1 Introduction 2 Model Reference 3 Recursive Least Squares Estimation 4 The Self-Tuning Regulator 5 Summary Basic indirect self-tuner Direct self-tuner Minimum variance control

32 The Self-Tuning Regulator Specification Self-tuning regulator Process parameters Controller design Estimation Reference Controller Controller parameters Input Process Output Certainty Equivalence - Design as if the estimates were correct (Simon) Many control and estimation schemes

33 The Self-Tuning Regulator Estimate parameters recursively Compute control law from estimated parameters Apply control signal Many different choices Model structure Parameterization Criterion A simple choice: Design an adaptive minimum variance controller.

34 Process and controller Command Signal Following Ay(t)=B(u(t)+v(t)), Ru(t)=Tu c (t) Sy(t) u c Controller Ru = Tu c Sy u v Σ Process B A y Closed loop system BT y(t)= AR+BS u BR c(t)+ AR+BS v(t) AT u(t)= AR+BS u BS c(t) AR+BS v(t) Closed loop characteristic polynomial AR+BS=A c

35 Closed loop response y(t)= BT AR+BS u c(t) Desired response A m y m (t)=b m u c (t) Perfect model following BT AR+BS =BT A c = B m A m Avoid cancelation of unstable process zeros :B=B + B Hence B m =B B m, A c=a o A m B +, R=R B + AR +B S=A o A m =A c, T=A ob m

36 Example Sampling with h = 0.5 G(s)= 1 s(s+1) H(q)= b 0q+b 1 q 2 +a 1 q+a 2 = q q q B m (q) A m (q) = b m0 q q 2 +a m1 q+a m2 = q q q B + (q)=q+b 1 /b 0, B (q)=b 0, B m(q)=b m0 q/b 0 ChooseA o =1 u(t)+r 1 u(t 1)=t 0 u c (t) s 0 y(t) s 1 y(t 1)

37 Example u c y Time u Time â 2 â 1 Time ˆb 1 ˆb 0 Time

38 Redesign with no Cancellation H(q)= b 0q+b 1 q 2 +a 1 q+a 2 = q q q We haveb + =1 andb =B=b 0 q+b 1 b 0 q+b 1 H m (q)=β q 2 = b m0q+b m1 +a m1 q+a m2 q 2 +a m1 q+a m2 Diophantine equation Control law (q 2 +a 1 q+a 2 )(q+r 1 )+(b 0 q+b 1 )(s 0 q+s 1 ) =(q 2 +a m1 q+a m2 )(q+a o ) u(t)+r 1 u(t 1)=t 0 u c (t) s 0 y(t) s 1 y(t 1)

39 u c y Time u Time â 2 â 1 Time ˆb 1 ˆb 0 Time

40 Load Disturbances u c y Time u Time

41 Process and controller Direct and Indirect STR Ay=Bu, Ru=Tu c Sy Direct: EstimateA andb, computer ands from Indirect: Form AR+BS=A o A m A o A m y=ary+bsy=bru+bsy=b(ru+sy) Estimate R and S by least squares from A o A m y=b(ru+sy) Combine with estimate ofbfrom the direct method and use control law ˆRu=A o u c Ŝy

42 Adaptation of Feedforward Gains Process model Ay=Bu+Cv wherevis a measured disturbance. Estimate R, S and Q by least squares from A o A m y=b(ru Sy Qv) Use control law ˆRu=A o u c Ŝy ˆQv

43 STR and Minimum Variance Control - Björn y t+1 +ay t =bu t +e t+1 +ce t u t = 1 b (ay t ce t )= a c b y t In generala(q) t y=b(q)u t +C(q)e t u t = a(q) c(q) (y t r t ) b(q) The self-tuning controller y t = β(u t θy t )+ǫ ) u t =sat(ˆ θ(y t r t ) Estimate θ by least squares for fixed β,0.5<β/b<,b(z) stable + order conditions. Local stability: real part ofc(z) positive for all zeros ofb(z)

44 An Example Consider a process governed by y(t+1)+ay(t)=u(t)+e(t+1) Assume that we would like to keep the output as close to zero as possible in the least squares sense. Estimate parameteraby least squares and use the control lawu=ây(t). The estimate is given by the normal equation y(t+1)+ây(t)=u(t) by least squares. The normal equation is t 1 k=1 t 1 y 2 (k)â(t)= (u(k) y(k+1))y(k) k=1

45 An Example... Using recursive computations of the estimate the control algorithm becomes u(t) = â(t)y(t) â(t)=â(t 1)+K(t)(y(t) u(t 1)+â(t 1)y(t 1)) K(t)=P(t)y(t 1) P(t)=P(t 1) P2 (t 1)y 2 (t 1) 1+y 2 (t 1)P(t 1) Clearly a nonlinear time-varying controller. The algorithm behaves as expected when applied to a system with the right model structure. What happens if it is applied to the system y(t+1)+ay(t)=bu(t)+e(t+1)+ce(t) Does the algorithm have some general properties?

46 An Example... Consider the system y(t+1) 0,9y(t)=3u(t)+e(t+1) 0.3e(t) The minimum variance controller is u(t)= c a b y(t)= 0.2y(t) Control the process by a self-tuner that estimates parameters in y(t+1)+ay(t)=u(t)+e(t+1) and use the control algorithmu=ây(t). Notice that the model structure is wrong!

47 Example... ŝ 0 Time Self-tuning control Minimum variance control Time

48 Example... Consider a process governed by y(t+1)+ay(t)=bu(t)+e(t+1)+ce(t) Estimate the parameterain y(t+1)+ay(t)=u(t) by least squares. The normal equation is t 1 k=1 t 1 y 2 (k)â(t)= (u(k) y(k+1))y(k) k=1 The control law is u(t) = â(t)y(t)

49 Example... t 1 k=1 t 1 y 2 (k)â(t)= (u(k) y(k+1))y(k) k=1 t 1 = (â(k)y(k) y(k + 1))y(k) k=1 1 t t 1 k=1 y(k+1)y(k)= 1 t t 1 (â(k) â(t) ) y 2 (k) k=1 Ify(k) is bounded in the mean square and if the estimate converges we find that the closed loop system has the property 1 t 1 lim y(k+1)y(k)=0 t t k=1

50 Example... If the parameters converges and the signals remain mean square bounded the simple self tuner drives the correlation of the output to zero. 1 t 1 lim y(k+1)y(k)=r y (1)=0 t t k=1 Compare with a controller with integral action which drives the error to zero if the closed loop system is stable. What does the conditionr y (1)=0imply?

51 Example... Consider the system y(t+1)+ay(t)=bu(t)+e(t+1)+ce(t) with the feedback u(t) = ky(t) The closed loop system is y(t)= q+c q+a+bk e(t)=e(t)+c a+bk q+a+bk e(t 1) The conditionr y (1)=0implies thatc a+bk=0 and that r y (τ)=0for all τ =0. The simple self-tuner converges to the minimum variance controller

52 1 Introduction 2 Model Reference 3 Recursive Least Squares Estimation 4 The Self-Tuning Regulator 5 Real lers 6 Summary

53 ASEA Novatune

54 ASEA Novatune Adaptation of feedforward from measured disturbances

55 First Control - Gunnar Bengtsson

56 Ship Steering - Clas Källström

57 Ship Steering - Performance

58 Steermaster

59 Relay Auto-Tuner Tore One-button tuning Automatic generation of gain schedules Adaptation of feedback and feedforward gains Many versions Robust Single loop controllers DCS systems Excellent industrial experience Large numbers

60 1 Introduction 2 Model Reference 3 Recursive Least Squares Estimation 4 The Self-Tuning Regulator 5 Real lers 6 Summary

61 Summary Adaptive control has had a turbulent history Adaptive systems are now reasonably well understood They are nonlinear and not trivial to analyse and design Interesting ideas Stability theory, passivity Chaotic behavior Averaging based on the assumption that parameters are slow Lennart s differential equations A large field we have just given a sketch Important issues still unresolved There are a number of adaptive systems running in industry Initialization, safety nets and guards Excitation and load disturbances