SYSTEMS MIN-MAX Harry G. Kwatny Department of Mechanical Engineering & Mechanics Drexel University
OUTLINE MIN-MAX CONTROL Problem Definition HJB Equation Example GAME THEORY Differential Games Isaacs Equations EXAMPLES Heating Tank
PROBLEM DEFINITION PROBLEM SETUP Consider the control system where x R n is the state u U R m is the control ẋ = f (x, u, w w W R q is an external disturbance We seek a control u that minimizes the cost J for the worst possible disturbance w and T J = l (x (T + L (x (t, u (t, w (t dt 0
HJB EQUATION HJB EQUATION We seek both u (t and w (t such that {u, w } = arg min Define the cost to go ( V (x, t = min max l (x (T + u U w W The principle of optimality leads to V (x, t t + H ( x, max u U w W T t J (u, w L (x (τ, u (τ, w (τ dτ V (x, t = 0 x where H (x, λ = min max u U w W { L (x, u, w + λ T f (x, u, w }
EXAMPLE EXAMPLE Consider the linear dynamics and cost 1 J(u, w = lim T 2T T Suppose (A, B and (A, E are stabilizable Then (A, C is detectable 0 ẋ = Ax + Bu + Ew y = Cx u (t = 1 ρ BT Sx (t, [ ] y T (ty(t + ρu T (tu(t γ 2 w T (tw(t dt w(t = 1 γ 2 ET Sx(t ( 1 A T S + SA S ρ BBT 1 γ 2 EET S = C T C
EXAMPLE EXAMPLE CONT D: SOME CALCULATIONS { 1 H ( = min max x T C T Cx + ρu T u γ 2 w T w } + λ T (Ax + Bu + Ew u w 2 u = 1 ρ BT λ, w = 1 γ 2 ET λ H = 1 2 xt C T Cx + λ T Ax 1 2ρ λt BB T λ + 1 2γ 2 λt EE T λ ẋ = H λ = Ax + 1 γ 2 EET λ 1 ρ BBT λ [ ẋ λ λ = H x = CT Cx + A T λ ] [ 1 A = γ EE T 1 ] [ 2 ρ BBT x C T C A T λ ]
EXAMPLE EXAMPLE CONT D Consider the Hamiltonian matrix [ A H = C T C 1 EE T 1 ] γ 2 ρ BBT A T Now, under the above assumptions: There exists a γ min such that so that there are no eigenvalues of H on the imaginary axis provided γ > γ min. When γ we approach the standard LQR solution. For γ = γ min, the controller is the full state feedback H controller. All other values of γ produce valid min-max controllers. The condition Reλ(H 0 is equivalent to stability of the matrix The closed loop system matrix is A + 1 γ 2 EET 1 ρ BBT A 1 ρ BBT The term EE T /γ 2 is destabilizing, so the system is guaranteed some margin of stability.
DIFFERENTIAL GAMES TWO PERSON DIFFERENTIAL GAME Once again consider the system ẋ = f (x, u, w However, now consider both u and w to be controls associated, respectively, with two players A and B. Player A wants to minimize the cost T J = l (x (T + L (x (t, u (t, w (t dt 0 and player B wants to maximize it. This is a two person, zero-sum, differential game.
DIFFERENTIAL GAMES DEFINITIONS Instead of the cost to go define two value functions: DEFINITION (VALUE FUNCTIONS The lower value function is ( T l (x (T + V A (x, t = min u U max w W and the upper value function is ( V B (x, t = max min l (x (T + w W u U t T t L (x (τ, u (τ, w (τ dτ L (x (τ, u (τ, w (τ dτ if V A = V B, a unique solution exists. This solution is called a pure strategy. If V A V B, a unique strategy does not exist. In general, the player who plays second has an advantage.
ISAACS EQUATIONS ISAACS EQUATIONS Define H A (x, λ = min H B (x, λ = min Then we get Isaacs equations H A ( x, VA (x, t x V A (x, t t V B (x, t t max u U w W max u U w W { L (x, u, w + λ T f (x, u, w } { L (x, u, w + λ T f (x, u, w } + H A ( x, + H B ( x, H B ( x, VA (x, t = 0 x VB (x, t = 0 x V A (x, t V B (x, t VB (x, t x and the game is said to satisfy the Isaacs condition. Notice that if u,w are optimal,the the pair (u, w is a saddle point in the sense that J (u, w J (u, w J (u, w
HEATING TANK WELL-STIRRED HEATING TANK STARTUP ẋ = x + w + u, x (0 = x 0, 0 u 2, w 1 x, dimensionless tank temperature u, dimensionless heat input w, dimensionless input water flow T J = (x x 2 dt, T fixed 0 H = (x x 2 + λ ( x + w + u {u, w } = arg min max H (x, u, w, λ u w u = (1 sgn (λ, w = sgn (λ λ = H x λ = λ 2 (x x, λ (T = 0
HEATING TANK EXAMPLE, CONT D ẋ = x + 1 λ = λ 2 (x x x (T = x, λ (T = 0 0.5 0.0 x,λ 0.5 1.0 with x = 0.8 0.0 0.5 1.0 1.5 2.0