Spectral Properties of Linear- Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 2018! Stability margins of single-input/singleoutput (SISO) systems! Characterizations of frequency response! Loop transfer function! Return difference function! Kalman inequality! Stability margins of scalar linearquadratic regulators Copyright 2018 by Robert Stengel. All rights reserved. For educational use only. http://www.princeton.edu/~stengel/mae546.html http://www.princeton.edu/~stengel/optconest.html 1 Return Difference Function and Closed-Loop Roots Single-Input/Single-Output Control Systems 2
SISO Transfer Function and Return Difference Function Unit feedback control law Block diagram algebra 1+ A s Δy ( s ) = A( s ) Δy C ( s ) Δy ( s ) Δy ( s ) = A( s )Δy C ( s ) Δy s Δy C s = A( s) 1+ A( s) : Open - Loop Transfer Function : Return Difference Function A s 1+ A s : Closed - Loop Transfer Function 3 Return Difference Function and Root Locus A( s ) = kn( s ) d ( s ) 1+ A( s ) = 1+ kn( s ) = 0 defines locus of roots d ( s ) d ( s ) + kn( s ) = 0 defines locus of closed-loop roots 4
Return Difference Example = kn( s) d( s) = k( s z) A s 1+ A( s) = 1+ 2 s 2 + 2ζω n s + ω = 1.25 s + 40 2 n s 2 + 2( 0.3) ( 7)s + 7 k( s z) = 0 2 s 2 + 2ζω n s + ω = 1+ 1.25 s + 40 2 n s 2 + 2( 0.3) ( 7)s + 7 s 2 + 2( 0.3) ( 7)s + ( 7) + 1.25 s + 40 ) = 0 5 Closed-Loop Transfer Function Example A s 1+ A s = A s 1+ A s 1.25 s + 40 s 2 + 2( 0.3) ( 7)s + ( 7) 2 1.25( s + 40) 1+ s 2 + 2( 0.3) ( 7)s + ( 7) 2 = 1.25( s + 40) s 2 + 2( 0.3) ( 7)s + ( 7) 2 +1.25 s + 40 1.25 s + 40 = s 2 + 2( 0.3) ( 7) +1.25 s + 7 kn( s ) = d s + kn( s ) 2 +1.25( 40) 6
Open-Loop Frequency Response: Bode Plot A( jω ) = Two plots 20 log A jω A jω jω + ( 7) 2 K jω z ( jω ) 2 + 2ζω n jω + ω = 1.25 jω + 40 2 n ( jω ) 2 + 2( 0.3) 7 vs. logω vs. logω Gain Margin Referenced to 0 db line Evaluated where phase angle = 180 Phase Margin Referenced to 180 Evaluated where amplitude ratio = 0 db 7 Open-Loop Frequency Response: Nyquist Plot vs. Imag( A( jω )) Re A( jω ) Single plot; input frequency not shown explicitly Gain and Phase Margins referenced to ( 1) point GM and PM represented as length and angle Dotted lines are M-Circles denoting the amplitude of closed-loop frequency response (db) Δy jω 20log Δy C jω = 20log A( jω ) 1+ A( jω ) Only positive frequencies need be considered 8
Algebraic Riccati Equation in the Frequency Domain 9 Linear-Quadratic Control Quadratic cost function for infinite final time J = 1 2 = 1 2 0 0 Δx T (t) Δu T (t) Q 0 0 R Δx T (t)qδx(t) + Δu(t)RΔu T (t) dt Linear, time-invariant dynamic system Constant-gain optimal control law Δx(t) dt Δu(t) Δ x(t) = FΔx(t) + GΔu(t) Δu( t ) = R 1 G T PΔx( t ) = CΔx( t ) C! R 1 G T P Algebraic Riccati equation 0 = Q P PF + PGR 1 G T P Q = F T P PF + C T RC 10
Frequency Characteristics of the Algebraic Riccati Equation Add and subtract sp such that P( F) + ( F T )P + C T RC = Q P( si n ) + ( si n )P + C T RC = Q State Characteristic Matrix Adjoint Characteristic Matrix 11 Frequency Characteristics of the Algebraic Riccati Equation P( si n ) + si n P + C T RC = Q Pre-multiply each term by G T ( si n ) 1 Post-multiply each term by ( si n ) 1 G G T ( si n ) 1 PG + G T P si n 1 G + G T si n 1 Q si n = G T si n 1 C T RC si n 1 G 1 G 12
Frequency Characteristics of the Algebraic Riccati Equation Substitute on left using the control gain matrix C = R 1 G T P G T P = RC G T ( si n ) 1 C T R + RC si n 1 G + G T si n 1 Q si n = G T si n Add R to both sides R + G T ( si n ) 1 C T R + RC si n 1 C T RC si n 1 G 1 G + G T si n 1 Q si n = R + G T si n 1 G 1 C T RC si n 1 G 1 G 13 Frequency Characteristics of the Algebraic Riccati Equation R + G T ( si n ) 1 C T R + RC si n I m + G T 1 G + G T si n 1 Q si n = R + G T si n 1 C T RC si n 1 G The left side can be factored as* ( si n ) 1 C T 1 G R I + C si m n = R + G T ( si n ) 1 Q( si n ) 1 G = R + G T ( si n ) 1 H T 1 G H si n 1 G * Verify by multiplying the factored form 14
Modal Expression of Algebraic Riccati Equation Define the loop transfer function matrix A( s)! C( si n ) 1 G 15 Y 1 Modal Expression of Algebraic Riccati Equation Recall the cost function transfer matrix ( s)! Y s = H( si n ) 1 G, where Q = H T H Laplace transform of algebraic Riccati equation becomes I m + A s T R I m + A( s) = R + ( YT s)y( s) 16
Algebraic Riccati Equation I m + A s T R I m + A( s) = R + ( YT s)y( s) Cost function transfer matrix, Y(s) Reflects control-induced state variations in the cost function Governs closed-loop modal properties as R becomes small Does not depend on R or P Y( s) = H( si n ) 1 G Loop transfer function matrix, A(s) Defines the modal control vector when s = s i 1 GΔu i Δu i = C s i I n = A( s i )Δu i, i = 1,n 17 LQ Regulator Portrayed as a Unit-Feedback System A( s) = C( si n ) 1 G I m + A( s) = I m + C( si n ) 1 G : Return Difference Matrix 18
Determinant of Return Difference Matrix Defines Closed-Loop Eigenvalues I m + A( s) = I m + C( si n ) 1 G = I m + CAdj ( si n )G si n = I m + CAdj ( si n )G s Δ OL Δ CL Closed-Loop Characteristic Equation ( s ) = Δ OL ( s ) I m + A( s ) = Δ OL s = Δ OL s Δ OL G I m + CAdj si n Δ OL ( s ) ( s ) Δ OL s I m + CAdj ( si n )G = 0 19 Stability Margins and Robustness of Scalar LQ Regulators 20
Scalar Case Multivariable algebraic Riccati equation I m + A s T R I m + A( s) = R + ( YT s)y( s) Algebraic Riccati equation with scalar control 1 + A s r 1 + A( s) = r + Y T ( s)y ( s) where = C( si n ) 1 G ( 1 1) dim( C) = ( 1 n) dim( F) = ( n n) dim( G) = ( n 1) A s = H( si n ) 1 G dim Y ( s) = ( p 1) dim( H) = ( p n) Y s 21 Scalar Case 1 + A jω 1+ A jω Let s = jω r 1+ A ( jω ) = r + Y T ( jω )Y ( jω ) or = 1 + Y T ( jω )Y jω r 1 + A( jω ) A( jω ) is a complex variable 1+ A jω 1+ A jω = 1+ c ω jd ω = 1+ c ω 2 + d 2 ω { }{ 1+ c ( ω ) + jd ( ω )} { } = 1+ A( jω ) 2 (absolute value) 22
Kalman Inequality In frequency domain, cost transfer function becomes Y i ( jω ) = l i ( ω ) + jm i ( ω ), i = 1, p 1+ Y T ( jω )Y jω r = 1+ Consequently, the return difference function magnitude is greater than one p i=1 l i 2 ( ω ) + m 2 i ( ω ) 1+ A( jω ) 1 Kalman Inequality r 23 Nyquist Plot Showing Consequences of Kalman Inequality 1+ A( jω ) stays outside of unit circle centered on ( 1,0) 24
Uncertain Gain and Phase Modifications to the LQ Feedback Loop Uncertain Element How large an uncertainty in loop gain or phase angle can be tolerated by the LQ regulator? 25 LQ Gain Margin Revealed by Kalman Inequality Nyquist Stability Criteria! Stability is preserved if! No encirclements of the 1 point, or! Number of counterclockwise encirclements of the 1 point equals the number of unstable open-loop roots! Nominal LQR satisfies the criteria! Loop gain change expands or shrinks entire Nyquist plot 26
A optimal A non optimal A non optimal Gain Change Expands or Shrinks Entire Plot k U! Uncertain gain ( jω ) = C LQ ( jωi n ) 1 G ( jω ) = k U A optimal ( jω ) = k U C LQ ( jωi n ) 1 G ( jω ) = k U A optimal ( jω )! Closed-Loop LQ system is stable until 1 point is reached, and # of encirclements changes 27 Scalar LQ Regulator Gain Margin Increased gain margin = Infinity Decreased gain margin = 50% 28
LQ Phase Margin Revealed by Kalman Inequality! Stability is preserved if! No encirclements of the 1 point! Number of counterclockwise encirclements of the 1 point equals the number of unstable open-loop roots Return Difference Function, 1 + A( jω ), is excluded from a unit circle centered at ( 1,0) A( jω ) = 1 intersects a unit circle centered at the origin Intersection of the unit circles occurs where the phase angle of A( jω ) = ( 180 o ± 60 o ) Therefore, Phase Margin of LQ regulator 60 29 LQ Regulator Preserves Stability with Phase Uncertainties of At Least 60 A optimal A non optimal ϕ U! Uncertain phase angle, deg ( jω ) = C LQ ( jωi n ) 1 G ( jω ) = e jϕ U A optimal ( jω ) = e jϕ U C ( jωi LQ n ) 1 G ϕ non optimal = ϕ optimal +ϕ U! Phase-angle change rotates entire Nyquist plot! Closed-Loop LQ system is stable until 1 point is reached 30
Reduced-Gain-/Phase-Margin Tradeoff Reduced loop gain decreases allowable phase lag while retaining closed-loop stability (see OCE, Fig. 6.5-5) 31 Control Design for Increased Gain Margin! Obtain lowest possible LQ control gain matrix, C, by choosing large R! Gain margin is 1/2 of these gains! Speed of response (e.g., bandwidth) may be too slow! Increase gains to restore desired bandwidth! Control system is sub-optimal but has higher gain margin than LQ system designed for same bandwidth 32
Control Design for Increased Gain Margin! High R, low-gain optimal controller! Increased gain to restore bandwidth R! ρ 2 R o F T P + PF + Q PGR 1 G T P = 0 C opt = R 1 G T P C sub opt = R 1 o G T P = ρ 2 C opt! Increased gain margin for high-bandwidth controller K sub opt = 1 2ρ, 2 33 Example: Control Design for Increased Robustness (Ray, Stengel, 1991) Open-loop longitudinal eigenvalues λ 1 4 = 0.1 ± 0.057 j, 5.15, 3.35! Three controllers! a) Q = diag(1 1 1 0) and R = 1! b) R = 1000! c) Case (b) with gains multiplied by 5 34
Loop Transfer Function Frequency Response with Elevator Control = C( jωi ) 1 G H jω Q = 1 0 1 0, R = 1 or 1000 35 Loop Transfer Function Nyquist Plots with Elevator Control H ( jω ) = C( jωi ) 1 G 36
Next Time: Singular Value Analysis of LQ Systems 37 Supplemental Material 38
Root Loci for Three Cases Case a Case b Transmission zeros z 1,2 = 0 1.2 Case c! Closed-loop system roots! Originate at stable images of open-loop poles! 2 roots to transmission zeros! 2 roots to, multiple Butterworth spacing 39 Open-Loop Frequency Response: Nichols Chart 20 log 10 A jω vs. A jω! Single plot! Gain and Phase Margins shown directly 40
Loop Transfer Function Nichols Charts with Elevator Control H ( jω ) = C( jωi ) 1 G 41 Probability of Instability Describes Robustness to Parameter Uncertainty (Ray, Stengel, 1991)! Distribution of closed-loop roots with! Probability of instability! Gaussian uncertainty in 10 parameters! a) Pr = 0.072! Uniform uncertainty in velocity and air density! b) Pr = 0.021! 25,000 Monte Carlo evaluations! c) Pr = 0.0076 Stochastic Root Locus 42
Effect of Time Delay 43 Time Delay Example: DC Motor Control Control command delayed by τ sec 44
Effect of Time Delay on Step Response With no delay Phase lag due to time delay reduces closed-loop stability 45 Bode Plot of Pure Time Delay = 1 = τω AR e jτω φ e jτω As input frequency increases, phase angle eventually exceeds 180 46
Effect of Pure Time Delay on LQ Regulator Loop Transfer Function Laplace transform of time-delayed signal: L u t τ = e τ s L u t = e τ s u s τ U! Uncertain time delay, sec A optimal A non optimal ( jω ) = C LQ ( jωi n ) 1 G ( jω ) = e τ U jω A optimal ( jω ) = e τ U jω C LQ ( jωi n ) 1 G 47 Effect of Pure Time Delay on LQ Regulator Loop Transfer Function Crossover frequency, ω cross, is frequency for which A optimal = C LQ ( jω cross I n ) 1 G = 1 jω cross Time delay that produces 60 phase lag τ U = 60 ω cross π 180 = π 3ω cross, sec 48