Lecture 5: Control Over Lossy Networks

Transcription

1 Lecture 5: Control Over Lossy Networks Yilin Mo July 2, Classical LQG Control The system: x k+1 = Ax k + Bu k + w k, y k = Cx k + v k x 0 N (0, Σ), w k N (0, Q), v k N (0, R). Information available for the controller at time k: Y k = (y 0,..., y k ). The control at time k is a function of the information Y k : u k (Y k ). The goal of a finite horizon LQG problem is to find a controller that minimizes the following quadratic cost: J(N) = min E u 0,...,u N N ( x T k W x k + u T ) k Uu k. k=0 1.1 Optimal Estimator Design Since the system is linear, the following Kalman filtering equations holds: 1. Initialization: 2. Prediction: 3. Correction: ˆx 0 1 = 0, P 0 1 = Σ. (1) ˆx k+1 k = Aˆx k + Bu k, P k+1 k = AP k A T + Q. (2) ˆx k+1 = ˆx k+1 k + P k+1 k C T (CP k+1 k C T + R) 1 (y k+1 C ˆx k+1 k ), (3) P k+1 = P k+1 k P k+1 k C T (CP k+1 k C T + R) 1 CP k+1 k. (4) And we have that E(x k Y k ) = ˆx k, Cov(x k Y k ) = P k. One important thing to notice: the P k is independent from u k. This is because the system is linear and hence we can subtract u k. 1

2 1.2 Optimal Controller Design and Separation Principle Now we can try to solve the optimal control design problem by dynamical programming. Define the value function [ N ( V (t) = min E x T k W x k + u T ) ] k Uu k u t,...,u N Clearly and Now, by Bellman equation We will guess that k=t J(N) = V (0), V (N) = Ex T NW x N. V (t) = min u t E ( x T t W x t + u T t Uu t + V (t + 1) ) (5) V (t) = Ex T t S t x t + c t (6) To prove this, we will use induction. Clearly S N = W and c N = 0. Now suppose (6) holds for t + 1, and we look at the following quantity: E ( x T t W x t + u T t Uu t + V (t + 1) ) = Ex T (W + A T S t+1 A)x + tr(s t+1 Q) + c t+1 + E [ u T t (U + B T S t+1 B)u t + x T t A T S t+1 Bu t + u T t B T S t+1 Ax t ] Notice that the controller do not know x t. Hence, let us rewrite x t as x t = ˆx t + x t ˆx t = ˆx t + e t. Theorem e k is independent of Y k and hence ˆx k and u k. 2. Proof. 1. Notice that Eˆx T k Sˆx k = Ex T k Sx k tr(sp k ) E(e k Y k ) = E(x k Y k ) E(ˆx k Y k ) = ˆx k ˆx k = 0. Hence, e k is linearly independent from Y k. Since e k and Y k are jointly Gaussian, e k and Y k are independent. 2. Since Ex T k Sx k = Eˆx T k Sˆx k + Eˆx T k Se k + Ee T k Sˆx k + Ee T k Se k = Eˆx T k Sˆx k tr(sp k ) 2

3 Now let us look at E [ u T t (U + B T S t+1 B)u t + x T t A T S t+1 Bu t + u T t B T ] S t+1 Ax t = E [ u T t (U + B T S t+1 B)u t + ˆx T t A T S t+1 Bu t + u T t B T ] S t+1 Aˆx t = E [ (u t u t ) T (U + B T S t+1 B)(u t u t ) ˆx T t A T S t+1 B(U + B T S t+1 B) 1 B T ] S t+1 Aˆx t where u t = (U + B T S t+1 B) 1 B T S t+1 Aˆx t. Hence V (t) = Ex T t (W + A T S t+1 A A T S t+1 B(U + B T S t+1 B) 1 B T S t+1 A)x t + c t+1 + tr(a T S t+1 B(U + B T S t+1 B) 1 B T S t+1 AP k ) + tr(s t+1 Q) Therefore S t = W + A T S t+1 A A T S t+1 B(U + B T S t+1 B) 1 B T S t+1 A, (7) and c t = c t+1 + tr(a T S t+1 B(U + B T S t+1 B) 1 B T S t+1 AP k ) + tr(s t+1 Q). Thus, J(N) = E(x T 0 S 0 x 0 ) + c 0 = tr(s 0 Σ) + c Infinite Horizon LQG problem Define J as J(N) J = lim N N. We consider the problem of finding a controller that minimizes the infinite horizon cost J. Notice that (7) is a Riccati equation. Hence, if N, then S k converges to S, which is the fixed solution of S = W + A T SA A T SB(U + B T SB) 1 B T SA, (8) The optimal controller is given by u k = (U + B T SB) 1 B T SAˆx k. 2 Witsenhausen s Counterexample Consider x 0 N (0, σ 2 ). 1. The first player knows x 0 and he computes an which is a function of x 0. x 1 = f(x 0 ), 3

4 2. The first player sends x 1 to the second player though a noisy channel. Therefore, the second player receives where v N (0, 1). y 2 = x 1 + v, 3. The second player then computes x 2 = g(y 2 ). The goal is to minimize the following cost function J = min f,g E k2 (x 0 x 1 ) 2 + (x 1 x 2 ) 2 Alternatively, one can consider the following equivalent scheme: 1. The controller knows x 0 and it computes an control u which is a function of x 0. u = f(x 0 ), 2. The state of the system satisfies the following update equation: x 1 = x 0 + u. 3. The second player observe the system via a noisy sensor: where v N (0, 1). y 2 = x 1 + v, 4. The second player then computes the state estimate x 2 = g(y 2 ). The goal is to minimize the following cost function 2.1 Optimal linear strategy J = min f,g E k2 u 2 + (x 1 x 2 ) 2 We adopt the first setting. Consider that both f(x) = λx and g(x) = µx are linear, then J = min λ,µ E k2 (1 λ) 2 x (λx 0 µ(λx 0 + v)) 2 Therefore, and λ = arg min λ µ = λ2 σ λ 2 σ 2, If k 2 σ 2 = 1 and k 0, then λ 1 and J 1. k 2 σ 2 (1 λ) 2 + λ2 σ λ 2 σ 2. 4

5 3 Nonlinear strategy One can prove that for small k and k 2 σ 2 = 0, the following design is better than the linear design: f(x) = σsgn(x), g(x) = σ 1 e 2σx 1 + e 2σx. 4 Control Over Lossy Networks The system: x k+1 = Ax k + ν k Bu k + w k, y k = Cx k + v k x 0 N (0, Σ), w k N (0, Q), v k N (0, R). ν k is an i.i.d. Bernouli process with P (ν k = 1) = λ. The goal of a finite horizon LQG problem is to find a controller that minimizes the following quadratic cost: J = 4.1 TCP case min u 0,...,u N lim N 1 N N E k=0 Information available for the controller at time k: ( x T k W x k + ν k u T k Uu k ). I k = (y 0,..., y k, ν 0,..., ν k ). J is finite if and only if the following Riccati equation has a positive semidefinite solution: Optimal Filter: S = W + A T SA λa T SB(U + B T SB) 1 B T SA, (9) 1. Initialization: ˆx 0 1 = 0, P 0 1 = Σ. (10) 2. Prediction: ˆx k+1 k = Aˆx k + ν k Bu k, P k+1 k = AP k A T + Q. (11) 3. Correction: ˆx k+1 = ˆx k+1 k + P k+1 k C T (CP k+1 k C T + R) 1 (y k+1 C ˆx k+1 k ), (12) P k+1 = P k+1 k P k+1 k C T (CP k+1 k C T + R) 1 CP k+1 k. (13) 5

6 Optimal Control: The optimal controller is given by where S is the solution of (9). 4.2 UDP case u k = (U + B T SB) 1 B T SAˆx k, Information available for the controller at time k: Y k = (y 0,..., y k ). We do not know whether u k has been applied to the system or not. The control actually affect the estimation performance. The optimal control law and the stability of the system is unknown. 6