Decentralized and distributed control

Transcription

1 Decentralized and distributed control Constrained distributed control for discrete-time systems M. Farina 1 G. Ferrari Trecate 2 1 Dipartimento di Elettronica e Informazione (DEI) Politecnico di Milano, Italy farina@elet.polimi.it 2 Dipartimento di Informatica e Sistemistica (DIS) Università degli Studi di Pavia, Italy giancarlo.ferrari@unipv.it EECI-HYCON2 Graduate School on Control 2012 Supélec, France Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

2 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

3 Schedule of the course Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

4 Suggested readings Books 1. T. Başar and G. J. Olsder. Dynamic Noncooperative Game Theory. Academic Press, 2nd edition, J. B. Rawlings and D. Q. Mayne. Model Predictive Control: Theory and Design. Nob Hill Publishing, Madison, WI, Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

5 Suggested readings Papers 3. W. B. Dunbar. Distributed receding horizon control of dynamically coupled nonlinear systems. IEEE Transactions on Automatic Control, 52(7): , July M. Farina and R. Scattolini. Distributed predictive control: a non-cooperative algorithm with neighbor-to-neighbor communication for linear systems. Automatica. In press., W. Langson, I. Chrissochoos, S.V. Rakovic, and Mayne D. Q. Robust model predictive control using tubes. Automatica, 40(1), L. Magni and R. Scattolini. Stabilizing decentralized model predictive control of nonlinear systems. Automatica, 42(7): , Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

6 Suggested readings Papers 7. D.Q. Mayne, M.M. Seron, and V. Rakovic. Robust model predictive control of constrained linear systems with bounded disturbances. Automatica, 41: , R. Scattolini. Architectures for distributed and hierarchical Model Predictive Control A review. Journal of Process Control, 19(5): , May B. T. Stewart, A. N. Venkat, J. B. Rawlings, S. J. Wright, and G. Pannocchia. Cooperative distributed model predictive control. System & Control Letters, 59: , Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

7 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

8 Introduction to MPC MPC is an on-line optimization-based control approach which allows to account for operational constraints, allows to account for multi-variable systems, allows to account for non linear systems, can be extended to deal with continuous and discrete decision variables and to include logic relations, has been recently used for control of large-scale systems. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

9 Introduction to MPC Nominal MPC - Ingredients The system where x R n, u R m. x(k + 1) = Ax(k) + Bu(k) The constraints { x X R n u U R m where X and U are convex neighborhoods of the origin. The auxiliary control law u(k) = Kx(k) (its properties will be later specified) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

10 Introduction to MPC Nominal MPC Terminal set The positively invariant terminal set X f X defined in such a way that, if x(k) X f, then { x(k + i) X f Kx(k + i) U for all i 0, if the state is controlled with the auxiliary control law x(k + 1) = (A + BK)x(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

11 Introduction to MPC Nominal MPC - Ingredients Remark: the constraint sets X and U, as well as the terminal set X f, can be, e.g., polytopic, i.e., described as an intersection of a finite number of half spaces (the sets are convex), e.g., given scalars c i and vectors f i X f = {x R n : f T i x c i, for all i} ellipsoidal, i.e., described by, for a given scalar c > 0: where H = H T 0. X f = {x R n : x T Hx c} Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

12 Introduction to MPC Nominal MPC - Ingredients For example, an ellipsoidal positively invariant terminal set for the system (controlled with a stabilizing auxiliary control law) x(k + 1) = (A + BK)x(k) is where P is such that X f = {x R n : x T Px c} (A + BK) T P(A + BK) P < 0 V(x) = x T Px is a Lyapunov function for the system and, if x(k) X f V(x(k+1))=x(k) T (A+BK) T P(A+BK)x(k) T <x(k) T Px(k) T =V(x(k)) c and then x(k + 1) X f Methods for computing polytopic invariant sets have also been developed. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

13 Introduction to MPC Nominal MPC - Ingredients The cost function V(x(t : t + N),u(t : t + N 1)) := t+n 1 k=t 1 2 { x(k) 2 Q + u(k) 2 R } + }{{} V f (x(t + N)) }{{} stage cost arrival cost where x 2 H = xt Hx Q > 0, R > 0, V f is a positive definite function, (i.e., V f (0) = 0 and V f (x) > 0 if x 0). Furthermore x(t : t + N),u(t : t + N 1) denote the sequences {x(t),...,x(t + N)} and {u(t),...,u(t + N 1)}, respectively. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

14 Introduction to MPC Nominal MPC - Ingredients The MPC optimization problem The MPC problem consists in the following optimization, at each time step t V (x(t)) = min V(x(t : t + N),u(t : t + N 1)) subject to u(t:t+n 1) x(k + 1) = Ax(k) + Bu(k) x(k) X, for k = t,...,t + N 1 u(k) U, for k = t,...,t + N 1 x(t + N) X f Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

15 Introduction to MPC Nominal MPC - Ingredients Result of the MPC optimization problem The result of the MPC problem (solved at each time step t) is the optimal input sequence u(t : t + N 1 t) = u(t t),...,u(t + N 1 t) According to the receding horizon criterion, at instant t only the first element u(t t) is applied. This implicitly defines a time-invariant MPC control law u(t) = u(t t) = K MPC (x(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

16 Introduction to MPC Nominal MPC - Ingredients Remark: MPC is a close-loop control method, i.e., 1. at time step t x(t) (or its estimate, obtained through an observer from the measurement y(t)) is evaluated; 2. the MPC optimization problem is solved on-line; 3. the control input u(t/t) is computed and applied at time t; 4. t + 1 t, and go to step 1. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

17 Introduction to MPC Nominal MPC - Assumptions Results of MPC-controlled systems can be established. Two possible solutions can be adopted. I) Zero terminal constraint auxiliary control law: u(k) = 0, terminal constraint: X f = {0} positively invariant under the auxiliary control law, i.e.,, x(t + 1) = Ax(t) arrival cost: V f 0 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

18 Introduction to MPC Nominal MPC - Assumptions II) Stabilizing auxiliary control law auxiliary control law: terminal constraint: arrival cost: u(k) = Kx(k) such that A + BK is as. stable, X f = {x : x 2 P α}, (positively invariant under the auxiliary control law) V f (x) = 1 2 x 2 P Matrix P is selected in such a way that x(k + 1) 2 P x(k) 2 P ( x(k) 2 Q + u(k) 2 R ) under the auxiliary control law, i.e. u(k) = Kx(k) and x(k + 1) = (A + BK)x(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

19 Introduction to MPC Nominal MPC - Assumptions For computing P (method II)): a typical choice is to set K = (R + B T PB) 1 B T PA (LQ control), where P solves the algebraic Riccati equation P = A T PA + Q A T PB(R + B T PB) 1 B T PA the simplest choice is, first to find K with alternative methods (e.g., eigenvalue assignment), and then to let P be the solution of the discrete-time Lyapunov equation (remark that K is given) A T PA P = (Q + K T RK) if A is stable a simple solution is to set K = 0, and to let P be the solution of the discrete-time Lyapunov equation A T PA P = Q Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

20 Introduction to MPC Nominal MPC - Main results Main results Under the stated assumptions, it is possible to prove: (i) recursive feasibility, i.e., if the MPC problem has a solution at time t, then has a solution at time t + 1; (ii) convergence to the origin, i.e., x(t) 0 as t +. For simplicity, we now consider only the case II) (stabilizing auxiliary control law). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

21 Introduction to MPC Nominal MPC - Main results We briefly sketch the proof of the recursive feasibility. Assume that, at step t, the MPC problem is feasible: there exists an optimal trajectory u(t : t + N 1 t), denote with x(t + 1 : t + N t) the trajectory computed with model x(t + 1) = Ax(t) + Bu(t) with x(t) as initial condition and with u(t : t + N 1 t) as input sequence, Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

22 Introduction to MPC Nominal MPC - Main results We briefly sketch the proof of the recursive feasibility. Assume that, at step t, the MPC problem is feasible: there exists an optimal trajectory u(t : t + N 1 t), denote with x(t + 1 : t + N t) the trajectory computed with model x(t + 1) = Ax(t) + Bu(t) with x(t) as initial condition and with u(t : t + N 1 t) as input sequence, in view of the feasibility at time t: a) x(k t) X for all k = t,...,t + N 1, b) u(k t) U for all k = t,...,t + N 1, c) x(t + N t) X f X, Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

23 Introduction to MPC Nominal MPC - Main results We briefly sketch the proof of the recursive feasibility. Assume that, at step t, the MPC problem is feasible: there exists an optimal trajectory u(t : t + N 1 t), denote with x(t + 1 : t + N t) the trajectory computed with model x(t + 1) = Ax(t) + Bu(t) with x(t) as initial condition and with u(t : t + N 1 t) as input sequence, in view of the feasibility at time t: a) x(k t) X for all k = t,...,t + N 1, b) u(k t) U for all k = t,...,t + N 1, c) x(t + N t) X f X, in view of a), and of the fact that X f is invariant with respect to the auxiliary control law: u(t + N t) = Kx(t + N t) U, x(t + N + 1 t) = (A + BK)x(t + N t) X f Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

24 Introduction to MPC Nominal MPC - Main results We briefly sketch the proof of the recursive feasibility. Assume that, at step t, the MPC problem is feasible: there exists an optimal trajectory u(t : t + N 1 t), denote with x(t + 1 : t + N t) the trajectory computed with model x(t + 1) = Ax(t) + Bu(t) with x(t) as initial condition and with u(t : t + N 1 t) as input sequence, in view of the feasibility at time t: a) x(k t) X for all k = t,...,t + N 1, b) u(k t) U for all k = t,...,t + N 1, c) x(t + N t) X f X, in view of a), and of the fact that X f is invariant with respect to the auxiliary control law: u(t + N t) = Kx(t + N t) U, x(t + N + 1 t) = (A + BK)x(t + N t) X f This proves that u(t + 1 : t + N t) is a feasible (although not optimal) trajectory for the MPC problem at time t + 1 (where the state of the system is x(t + 1) = x(t + 1 t)). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

25 Introduction to MPC Nominal MPC - Main results A sketch of the proof of convergence is the following. As we have already proved, u(t + 1 : t + N t) is a feasible (non-optimal) solution to the MPC problem at time t + 1, where u(t + 1 : t + N 1 t) is given by the solution of the MPC problem at time t, u(t + N t) = Kx(t + N t), furthermore x(t + N + 1 t) = (A + BK)x(t + N t), Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

26 Introduction to MPC Nominal MPC - Main results A sketch of the proof of convergence is the following. As we have already proved, u(t + 1 : t + N t) is a feasible (non-optimal) solution to the MPC problem at time t + 1, where u(t + 1 : t + N 1 t) is given by the solution of the MPC problem at time t, u(t + N t) = Kx(t + N t), furthermore x(t + N + 1 t) = (A + BK)x(t + N t), we compute the value of V with respect to such a feasible non-optimal solution: V (x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) = = t+n 1 k=t+1 2 { x(k t) 2 Q + u(k t) 2 R } + Vf (x(t + N + 1 t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

27 Introduction to MPC Nominal MPC - Main results A sketch of the proof of convergence is the following. As we have already proved, u(t + 1 : t + N t) is a feasible (non-optimal) solution to the MPC problem at time t + 1, where u(t + 1 : t + N 1 t) is given by the solution of the MPC problem at time t, u(t + N t) = Kx(t + N t), furthermore x(t + N + 1 t) = (A + BK)x(t + N t), we compute the value of V with respect to such a feasible non-optimal solution: V (x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) = = t+n 1 k=t+1 2 { x(k t) 2 Q + u(k t) 2 R } + Vf (x(t + N + 1 t)) = t+n 1 1 k=t 2 { x(k t) 2 Q + u(k t) 2 R } + Vf (x(t + N t))+ 2 1 { x(t t) 2 Q + u(t t) 2 R } { x(t + N t) 2 Q + u(t + N t) 2 R }+ V f (x(t + N t)) + V f (x(t + N + 1 t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

28 Introduction to MPC Nominal MPC - Main results A sketch of the proof of convergence is the following. As we have already proved, u(t + 1 : t + N t) is a feasible (non-optimal) solution to the MPC problem at time t + 1, where u(t + 1 : t + N 1 t) is given by the solution of the MPC problem at time t, u(t + N t) = Kx(t + N t), furthermore x(t + N + 1 t) = (A + BK)x(t + N t), we compute the value of V with respect to such a feasible non-optimal solution: V (x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) = = t+n 1 k=t+1 2 { x(k t) 2 Q + u(k t) 2 R } + Vf (x(t + N + 1 t)) = t+n 1 1 k=t 2 { x(k t) 2 Q + u(k t) 2 R } + Vf (x(t + N t))+ 2 1 { x(t t) 2 Q + u(t t) 2 R } { x(t + N t) 2 Q + u(t + N t) 2 R }+ V f (x(t + N t)) + V f (x(t + N + 1 t)) = V(x(t : t + N t),x(t : t + N 1 t))+ 1 2 { x(t t) 2 Q + u(t t) 2 R } { x(t + N t) 2 Q + Kx(t + N t) 2 R }+ V f (x(t + N t)) + V f (x(t + N + 1 t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

29 Introduction to MPC Nominal MPC - Main results in view of the assumption on the terminal constraint: Therefore V Recalling that V f (x(t + N + 1 t)) V f (x(t + N t)) 1 2 { x(t + N t) 2 Q + Kx(t + N t) 2 R } (x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) V(x(t : t + N t),u(t : t + N 1 t))+ 1 2 { x(t t) 2 Q + u(t t) 2 R } V(x(t : t + N t),u(t : t + N 1 t)) = V (x(t)) and that, in view of the sub-optimality of x(t + 1 : t + N + 1 t) and u(t + 1 : t + N t) V (x(t + 1)) V(x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) we obtain that V (x(t + 1)) V (x(t)) 1 2 { x(t t) 2 Q + u(t t) 2 R } Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

30 Introduction to MPC Nominal MPC - Main results in view of the assumption on the terminal constraint: Therefore V Recalling that V f (x(t + N + 1 t)) V f (x(t + N t)) 1 2 { x(t + N t) 2 Q + Kx(t + N t) 2 R } (x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) V(x(t : t + N t),u(t : t + N 1 t))+ 1 2 { x(t t) 2 Q + u(t t) 2 R } V(x(t : t + N t),u(t : t + N 1 t)) = V (x(t)) and that, in view of the sub-optimality of x(t + 1 : t + N + 1 t) and u(t + 1 : t + N t) V (x(t + 1)) V(x(t + 1 : t + N + 1 t),u(t + 1 : t + N t)) we obtain that V (x(t + 1)) V (x(t)) 1 2 { x(t t) 2 Q + u(t t) 2 R } from the latter it follows that V (x(t + 1)) is decreasing, which implies that x(t t) 2 Q 0 as t + which, in view of the positive-definiteness of Q, implies that x(t) 0 as t +. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

31 Introduction to MPC Nominal MPC - Convergence An extension The assumption Q > 0 can be relaxed to Q 0 provided that the pair (A, Q) is detectable. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

32 Introduction to MPC Nominal MPC - Remarks Remark: MPC is computationally demanding, i.e., on-line optimization; many applications involve non-linear systems; many applications require small sampling time; many application involve large-scale systems - large scale optimization problems; even explicit methods (off-line computation of the MPC control law K MPC (x(t)) or an approximation of it) are demanding - memory and computational power. There is a strong need to develop distributed and/or decentralized MPC methods to cope with large-scale systems. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

33 Introduction to MPC Nominal MPC - Remarks Main issues: scalability: as the order of the system grows, the main goal is to divide the problem into small-scale subproblems and to keep the computational/memory burden, and the transmission/communication load as limited as possible; reliability and robustness: large-scale systems involve relevant model uncertainties and disturbances, and the possibility that parts or subsystems are removed, added, or replaced: adaptivity to structural changes: large-scale plants require that parts or subsystems are removed, added, or replaced, without the necessity to re-design the overall control system and architecture. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

34 Introduction to MPC Robust MPC Perturbed systems x(k + 1) = Ax(k) + Bu(k) + w(k) where w(k) is a bounded disturbance, i.e., w(k) W, where W is compact and contains the origin. Problem To devise an MPC controller that provides convergence, (worst-case) optimality, and constraint satisfaction for all possible realizations of the bounded disturbance w(k). Two main approaches: min-max approach, which leads to burdensome optimization problems; tube-based approach. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

35 Introduction to MPC Robust MPC - tube-based approach Perturbed systems x(k + 1) = Ax(k) + Bu(k) + w(k) where w(k) is a bounded disturbance, i.e., w(k) W, where W is compact and contains the origin. Nominal model ^x(k + 1) = A^x(k) + B^u(k) Robust control law u(k) = ^u(k) + K(x(k) ^x(k)) Denote z(k) = x(k) ^x(k). The variable z(k) evolves according to z(k + 1) = (A + BK)z(k) + w(k) irrespective of ^u(k). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

36 Introduction to MPC Robust MPC - tube-based approach Robust positively invariant (RPI) set z(k + 1) = (A + BK)z(k) + w(k) If A + BK is as. stable, then there exists a robust positively invariant (RPI) set Z such that, if z(t) Z and w(k) W for all k t, then z(t + i) Z for all i 0, i.e., (A + BK)Z W Z Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

37 Introduction to MPC Robust MPC - tube-based approach Minkowski sum: C = A B = {c = a + b : a A,b B} Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

38 Introduction to MPC Robust MPC - tube-based approach Minkowski sum: C = A B = {c = a + b : a A,b B} Minkowski difference: C = A B = {c : c B A} Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

39 Introduction to MPC Robust MPC - tube-based approach Since z(k) = x(k) ^x(k) Z, in order to meet the state and input constraints { x X R n u U R m where X and U are convex neighborhoods of the origin, it is sufficient to satisfy the following tightened constraints where ^X = X Z and ^U = U KZ. { ^x ^X ^u ^U Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

40 Introduction to MPC Robust MPC - tube-based approach Auxiliary control law ^u(k) = K^x(k) where A + BK is asymptotically stable. Positively invariant terminal set It is the invariant set ^X f ^X for the nominal model such that, if ^x(k) ^X f, then { ^x(k + i) ^X f K^x(k + i) ^U for all i 0, if the nominal state evolves according to ^x(k + 1) = (A + BK)^x(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

41 Introduction to MPC Robust MPC - tube-based approach The cost function V(^x(t : t + N), ^u(t : t + N 1)) := t+n 1 k=t 1 2 { ^x(k) 2 Q + ^u(k) 2 R } + }{{} V f (^x(t + N)) }{{} stage cost arrival cost where Q > 0, R > 0, and V f is a positive definite function. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

42 Introduction to MPC Robust MPC - tube-based approach The tube-based robust MPC optimization problem The tube-based robust MPC problem consists in the following optimization, at time t V (^x(t)) = min V(^x(t : t + N), ^u(t : t + N 1)) ^u(t:t+n 1) subject to ^x(k + 1) = A^x(k) + B^u(k) ^x(k) ^X, for k = t,...,t + N 1 ^u(k) ^U, for k = t,...,t + N 1 ^x(t + N) ^X f Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

43 Introduction to MPC Robust MPC - tube-based approach Result of the MPC optimization problem The result of the MPC problem (solved at each time step t) is optimal nominal input sequence ^u(t : t + N 1 t) = ^u(t/t),..., ^u(t + N 1/t) According to the receding horizon criterion at instant t only the first element ^u(t/t) is applied to the nominal model in such a way that ^x(t + 1) is computed; ^x(t + 1) = A^x(t) + B^u(t) the robust MPC control input for the perturbed system is, at time t u(t) = ^u(t/t) + K(x(t) ^x(t)) the nominal state trajectory ^x(t) is independent of the perturbed state trajectory x(t), but the invariance property guarantees that z(t) = x(t) ^x(t) remains bounded, i.e., x(t) = ^x(t) Z Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

44 Introduction to MPC Robust MPC - tube-based approach Main idea: at time t = 0, x(0) = x 0 and set ^x(0) such that x(0) ^x(0) Z at time step t 1. solve the nominal MPC problem with tightened constraints; 2. the nominal input ^u(t/t) is computed and applied to the nominal model: ^x(t + 1) is computed; 3. the robust input u(t) = ^u(t/t) + K(x(t) ^x(t)) is applied to the real system; 4. t + 1 t and go to step 1. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

45 Introduction to MPC Robust MPC - tube-based approach Remarks: Since the MPC method is actually applied to the nominal system, it is apparent that ^x(t) 0 as t in view of the invariance property for all t; it follows that x(t) ^x(t) Z x(t) Z as t for better performance: find Z as small as possible! Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

46 Introduction to MPC Robust MPC - improved tube-based approach In the previously discussed tube based approach, the evolution of the nominal system is not affected by the evolution of the real system. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

47 Introduction to MPC Robust MPC - improved tube-based approach In the previously discussed tube based approach, the evolution of the nominal system is not affected by the evolution of the real system. Problem How to introduce a feedback from the real system to the nominal controller? Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

48 Introduction to MPC Robust MPC - improved tube-based approach The optimization problem is reformulated. The improved tube-based MPC problem The tube-based improved robust MPC problem consists in the following optimization, at time t V (x(t)) = min V(^x(t : t + N), ^u(t : t + N 1)) ^x(t),^u(t:t+n 1) subject to with the additional constraint ^x(k + 1) = A^x(k) + B^u(k) ^x(k) ^X, for k = t,...,t + N 1 ^u(k) ^U, for k = t,...,t + N 1 ^x(t + N) ^X f x(t) ^x(t) Z Note that a degree of freedom has been added: ^x(t) is now an argument of the optimization problem. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

49 Introduction to MPC Robust MPC - improved tube-based approach Result of the MPC optimization problem The result of the MPC problem (solved at each time step t) is the optimal nominal input sequence ^u(t : t + N 1 t) = ^u(t/t),..., ^u(t + N 1/t) and the nominal state (at instant t): ^x(t/t) the robust MPC control input for the perturbed system is, at time t u(t) = ^u(t/t) + K(x(t) ^x(t)) the nominal state trajectory ^x(t) now depends on the perturbed state trajectory x(t),in view of the additional constraint x(t) = ^x(t) Z Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

50 Introduction to MPC Robust MPC - improved tube-based approach Main idea: at time t = 0, x(0) = x 0 ; at time step t 1. solve the nominal MPC problem with tightened constraints and with the additional constraint on the initial value ^x(t); 2. the robust input u(t) = ^u(t/t) + K(x(t) ^x(t)) is applied to the real system; 3. t + 1 t and go to step 1. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

51 Introduction to MPC Robust MPC - improved tube-based approach Properties Also for this approach, recursive feasibility and stability properties can be established. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

52 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

53 Introduction to dynamic non-cooperative games To classify and understand the main available distributed optimization-based control algorithms, we need to introduce non-cooperative dynamic games. Game theory It is the study of the interactions among different agents, involving multi-person decision-making Dynamic games A game is dynamic (or differential) if the order in which decisions are taken is relevant. I.e., the decision taken by an agent at instant t may depend on the state of the system (the environment), which in turn depends on the decision taken also by the competing agents at previous time instants. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

54 Introduction to dynamic non-cooperative games Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

55 Introduction to dynamic non-cooperative games Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

56 Introduction to dynamic non-cooperative games Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

57 Introduction to dynamic non-cooperative games Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

58 Introduction to dynamic non-cooperative games Non-cooperative game A game is said to be non-cooperative when each player pursues its own interests. This can lead to conflicting goals among players. In fact a player has to take a decision based on its own utility, or payback, agents must take decisions with (in general) different utility functions, then a conflicting situation can be produced. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

59 Introduction to dynamic non-cooperative games Definition A normal-form game is a tuple (N,A,g), where N is a finite set of M players, indexed by i; A = A 1 A M, where A i is the (finite) set of actions available to player i; any vector a = (a 1,...,a M ) A is an action profile; g = (g 1,...,g M ), where g i = A R is the utility (gain or payoff) function for player i. Remark: in general the payoff g i of an agent depends both: on the action of agent i, a i A i, the action of the competing agents, a j A j, j i, we define by A i the set of actions of the competing agents: A i = A 1 A i 1 A i+1 A M a i is the action profile of the competing agents of i, a i A i. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

60 Introduction to dynamic non-cooperative games Strategy A strategy is a set of decision rules, defining the actions to be taken by a player in each situation. It can depend on the state of the system (especially in dynamic games e.g., a control law!), can be I) fixed (i.e., a pure strategy), II) probabilistic (i.e., mixed strategy), when decisions in A i are not deterministic, but are taken according to a given probability distribution. s i and S i denote a strategy and a set of strategies, respectively, for agent i; s = (s 1,...,s M ) is strategy profile, and S = S 1 S M ; s i = (s 1,...,s i 1,s i+1,...,s M ) is the strategy profile of the competing agents to i, and S i is the set where s i lies; note that s = (s i,s i ) S. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

61 Introduction to dynamic non-cooperative games Optimality in a single-player framework The optimal strategy is the strategy that defines the action (a) that maximizes the utility function g for a given environment where the (single) agent operates, i.e., g(a) g(a ) for all a A Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

62 Introduction to dynamic non-cooperative games Optimality in a single-player framework The optimal strategy is the strategy that defines the action (a) that maximizes the utility function g for a given environment where the (single) agent operates, i.e., g(a) g(a ) for all a A... and in a multi-player framework what is an optimal strategy? Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

63 Introduction to dynamic non-cooperative games Optimality in a single-player framework The optimal strategy is the strategy that defines the action (a) that maximizes the utility function g for a given environment where the (single) agent operates, i.e., g(a) g(a ) for all a A... and in a multi-player framework Desired properties: an optimal strategy what is an optimal strategy? optimizes the system-wide outcome of a game, should be invariant with respect to additive or scaling operations on the single player s utility functions, Different solution concepts have been defined, i.e., different definitions of optimality. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

64 Introduction to dynamic non-cooperative games Solution concepts Pareto-optimal strategies A given strategy profile s is said to Pareto-dominate the strategy profile s if, for all i, g i (s) g i (s ), and if this inequality is strict for at least a value of i N. A strategy profile s is Pareto-optimal if there does not exist any other strategy profile s S that Pareto-dominates s. Pareto-optimality defines an unambiguous way to establish that a given strategy is globally dominating. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

65 Introduction to dynamic non-cooperative games Solution concepts Pareto-optimal strategies A given strategy profile s is said to Pareto-dominate the strategy profile s if, for all i, g i (s) g i (s ), and if this inequality is strict for at least a value of i N. A strategy profile s is Pareto-optimal if there does not exist any other strategy profile s S that Pareto-dominates s. Pareto-optimality defines an unambiguous way to establish that a given strategy is globally dominating. Nash equilibria A strategy profile s = (s 1,...,s M ) is a Nash equilibrium if, for all agents i, s i is i s best response to s i, meaning that g i (s i,s i ) g i (s i,s i) for all s i S i. A Nash equilibrium defines optimality from a single player s point of view, with respect to the states of all the other agents. It is possible to prove that every game has at least one (possibly mixed) Nash equilibrium. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

66 Introduction to dynamic non-cooperative games Solution concepts Max-min strategies The maxmin strategy of player i is a (not necessarily unique or fixed) strategy that maximizes i s worst case utility. For player i, it is defined as argmax si S i min s i g i (s i,s i ) It is the choice taken with the aim of maximizing one s expected utility without having to make any assumption on the other player s adopted strategy. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

67 Introduction to dynamic non-cooperative games Game theory and distributed control Why basics of game theory are useful for study of distributed optimization-based control? to classify and understand the main rationale underlying the methods proposed in the literature: basically all the proposed methods have a clear game-theoretical characterization; to provide stimulating starting points for the development of novel control schemes: many recent control schemes are explicitly inspired by game theoretical solution concepts. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

68 Introduction to dynamic non-cooperative games Game theory and distributed control Type of games in distributed MPC: I) dynamic infinite games, where players have an infinite number of actions to make, i.e., u i R m i ; II) a pure (fixed) strategy, for each player, is represented as a real input vector u i for the subsystem i; III) the cost functions V i = g i, i = 1,...,M, are strictly convex; IV) the utility functions g i (u 1,...,u M ), i = 1,...,M are jointly continuous in all its arguments and strictly concave - generally quadratic in u i for every u j, j i; V) mixed strategies are not likely to be implemented, since this would mean to use control laws with statistically changing parameters. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

69 Introduction to dynamic non-cooperative games Game theory and distributed control Three types of algorithms are studied: Nash equilibrium solutions of general non-cooperative games where the utility functions of the players differ from each other: Nash solution of a non-cooperative game; Maxmin solutions of general non-cooperative games where the utility functions of the players differ from each other: robust solution of a non-cooperative game; Pareto-optimal (i.e., Nash) solution of non-cooperative games where the utility functions are the same for all players (g i = g for all i = 1,...,M): solution of a cooperative game. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

70 Introduction to dynamic non-cooperative games Example: the prisoner s dilemma The players are two prisoners (P 1 and P 2 ) suspected of a crime; P 1 and P 2 undergo two interrogations into separate rooms. Each prisoner has two choices: confess (i.e., cooperate, C) and deny (i.e., defect, D). The utilities g i, i = 1,2 are defined as g i = cost i, where which are indicated in the table cost i = month in prison for i C D C 1,1 0,4 D 4,0 3,3 the maxmin solution is the choice each prisoner makes to minimize the months in prison, in face of the worst choice that the other prisoner can make the Nash equilibrium is the choice such that, for both i = 1 and i = 2, if i changes its mind, then number of month in prison for i is greater than now. In both cases, the solution is DD. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

71 Introduction to dynamic non-cooperative games Example: the prisoner s dilemma To change the problem into a cooperative one, we assume that the cost cost i = cost for both prisoners is the same, cost is the total number of prison years for P 1 and P 2. The corresponding table is C D C 2,2 4,4 D 4,4 6,6 Apparently, the Nash equilibrium (actually corresponding both to the Pareto-optimal solution and to the maxmin solution) corresponds to the choices CC. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

72 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

73 Models The design of decentralized and distributed MPC control systems requires the availability of partitioned models of large-scale systems into a number M of sub-models S i. Here we deal with discrete-time systems. Unstructured discrete-time model Partitioning methods In this part we focus on: x o (k + 1) = A o x o (k) + B o u(k) y(k) = C o x o (k) non-overlapping decompositions; completely overlapping decompositions. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

74 Models Non-overlapping decomposition Interaction-oriented model { xi (k + 1) = A ii x i (k) + B ii u i (k) + M j i A ijx j (k) + M j i B iju j (k) y i (k) = C i x i (k) where x i R n i, u i R m i, and y i R p i. Recall that, possibly under a permutation: x 1 x M u 1 u M y 1 x o =., u =., y =. i.e., the input/state/output vectors are completely partitioned. y M Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

75 Models Non-overlapping decomposition Features of non-overlapping decomposition The main objectives are: to obtain small scale sub-models (minimality of the representation), to identify cascade configurations (simplicity of the control structure), to identify subsystems that are weakly connected together (allow decentralized/distributed controllers), to minimize the communication/interaction links (minimization of the communication). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

76 Models Completely overlapping decomposition Interaction-oriented model { xi (k + 1) = A ii x i (k) + M j=1 B iju j (k) y i (k) = C i x i (k) i.e., the dynamically-decoupled form. where x i R n i, u i R m i, and y i R p i. Recall that, possibly under a permutation: u 1 u M y 1 u =., y =. i.e., the input/output vectors are completely partitioned, while n i is generally greater that n. y M Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

77 Models Completely overlapping decomposition Features of completely overlapping decomposition The main objectives are: to obtain sub-models with local (and low-order) input and output variables, to avoid the dependance of the local equations upon the state variables of neighboring subsystems. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

78 Models Main requirements Many features of the distributed MPC algorithms that will be later presented strongly depend on the type and the characteristics of the used partition. Minimality of the representation needed to limit the algorithm computational burden, the number of involved state variables (for each subsystem) generally affects the computational load of the algorithm (at least when state constraints are accounted for). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

79 Models Reduction of the communication burden Reducing the communication burden reduces the probability of transmission delays, overloads and package losses. It is strongly dependent on I) the number of neighbors for each subsystem: this is mostly dependent on the type of decomposition; II) the required information to be transmitted on-line: this mostly depends on the selected distributed control algorithm. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

80 Models Reduction of the required information to be stored and robustness Minimal non-overlapping representations require that only the knowledge of local dynamics (and of how the neighboring variables affect them) is available to each local controller scalable memory load; robustness with respect to uncertainties on the dynamics of other subsystems; overlapping decompositions imply that possibly the dynamical model of the overall system be stored by local control units this requires non-scalable memory load (it increases as the order of the system increases); this makes the local control system affected by uncertainties and perturbations. Remark that globally optimal performances can be generally obtained with non-minimal representations, such as the completely overlapping ones. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

81 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

82 MPC decentralized and distributed algorithms In the following we consider the case where the system is partitioned using a non-specified interaction-oriented decomposition and M = 2: Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

83 MPC decentralized and distributed algorithms Interacting subsystems: { x1 (k + 1) = A S 1 = 11 x 1 (k) + B 11 u 1 (k) + A 12 x 2 (k) + B 12 u 2 (k) { y 1 (k) = C 1 x 1 (k) x2 (k + 1) = A S 2 = 22 x 2 (k) + B 22 u 2 (k) + A 21 x 1 (k) + B 21 u 1 (k) y 2 (k) = C 2 x 2 (k) Collecting together the interaction oriented models, we obtain a (possibly non-minimal) collective model: { x(k + 1) = Ax(k) + Bu(k) where and x(k) = y(k) = Cx(k) [ ] x1 (k), y(k) = x 2 (k) [ ] y1 (k), u(k) = y 2 (k) [ ] u1 (k) u 2 (k) [ ] [ ] [ ] A11 A A = 12 B11 B, B = 12 C1 0, C = A 21 A 22 B 21 B 22 0 C 2 Remark: A, B, and C are different, in general, from the minimal ones A o, B o, and C o. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

84 MPC decentralized and distributed algorithms Cost function The cost function is V = t+n { x(k) 2 Q + u(k) 2 R } x(t + N) 2 P k=t Separability of the cost function The cost function is assumed to be formally separable, i.e., V = ρ 1 V 1 + ρ 2 V 2 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

85 MPC decentralized and distributed algorithms Separability of the cost function is obtained by setting block diagonal weighting matrices [ ] [ ] [ ] ρ1 Q Q = 1 0 ρ1 R, R = 1 0 ρ1 P, P = ρ 2 Q 2 0 ρ 2 R 2 0 ρ 2 P 2 in such a way that ][ ] V = t+n 1 1 k=t 2 {[ x1 T (k) xt 2 (k)][ ρ 1 Q 1 0 x1 (k) + 0 ρ 2 Q 2 x 2 (k) + [ ][ ] u1 T (k) ut 2 (k)][ ρ 1 R 1 0 u1 (k) }+ 0 ρ 2 R 2 u 2 (k) ][ ] [ 1 2 x T 1 (t + N) x2 T (t + N)][ ρ 1 P 1 0 x1 (t + N) 0 ρ 2 P 2 x 2 (t + N) = ρ 1 V 1 + ρ 2 V 2 where V 1 = t+n 1 k=t 1 2 { x 1 (k) 2 Q 1 + u 1 (k) 2 R 1 } x 1(t + N) 2 P 1 V 2 = t+n 1 k=t 1 2 { x 2 (k) 2 Q 2 + u 2 (k) 2 R 2 } x 2(t + N) 2 P 2 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

86 MPC decentralized and distributed algorithms Centralized MPC (cmpc) Minimization problem min u 1 (t:t+n 1),u 2 (t:t+n 1) ρ 1V 1 + ρ 2 V 2 System model S : x(k + 1) = Ax(k) + Bu(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

87 MPC decentralized and distributed algorithms Decentralized MPC (dmpc) Decentralized control local regulators designed to control x i (k) using input u i (k) independently of x j (k) and u j (k), j i; mutual interactions neglected (model error); design of local regulators trivial (low order problems); if interactions between S 1 and S 2 are sufficiently weak (i.e., matrices A 12, B 12, A 21, and B 21 are small), then closed loop stability; strong interactions may prevent stability and/or acceptable performances; stability analysis of the closed loop system not trivial. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

88 MPC decentralized and distributed algorithms Decentralized MPC (dmpc) Regulator R 1 Minimization problem: Model of S 1 ( wrong ) min u 1 (t:t+n 1) V 1 x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k) Regulator R 2 Minimization problem: Model of S 2 ( wrong ) min u 2 (t:t+n 1) V 2 x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

89 MPC decentralized and distributed algorithms Distributed MPC (DMPC) Distributed control information is transmitted among local regulators, e.g., future predicted state and control variables computed locally, to predict the interaction effects; transmitted data is strictly is related to the employed model decomposition. E.g., I. non-overlapping partition: S 1 needs the predictions of both x 2 (k) and u 2 (k) to compute predictions of the evolution of x 1 (k); II. completely overlapping partitions: S 1 needs u 2 (k) to compute predictions of the evolution of x 2 (k). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

90 MPC decentralized and distributed algorithms Distributed MPC (DMPC) Classification of DMPC algorithms: Topology of the transmission network fully connected transmission networks (all-to-all communication): information is transmitted from any local regulator to the others; partially connected transmission networks (neighbor-to-neighbor communication): information is transmitted among the local regulators of subsystems only with direct dynamic/input coupling (it allows to significantly reduce the transmission load especially with non-overlapping - sparse - partitions). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

91 MPC decentralized and distributed algorithms Distributed MPC (DMPC) Classification of DMPC algorithms: Exchange of information iterative algorithms:information is transmitted (and received) more than once within each sampling time (e.g., optimality is obtained when the algorithm steps reach convergence); non-iterative algorithms:information is transmitted (and received) once within each sampling time. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

92 MPC decentralized and distributed algorithms Distributed MPC (DMPC) Classification of DMPC algorithms: Rationale of the algorithm The distributed MPC problems described in this section can all be cast as dynamic non-cooperative games: Nash solution of a non-cooperative game where the utility functions of the players differ from each other (independent DMPC, idmpc); robust solution of a non-cooperative game where the utility functions of the players differ from each other (robust DMPC, rdmpc); solution of a cooperative game where the utility functions are the same for all players (cooperative DMPC, cdmpc). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

93 MPC decentralized and distributed algorithms Independent DMPC (idmpc) Regulator R 1 Minimization problem: min u 1 (t:t+n 1) V 1 Model of S 1 x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k) + A 12 x 2 (k) + B 12u 2 (k) where predictions u2 (t : t + N 1), x 2 (t : t + N 1) are available Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

94 MPC decentralized and distributed algorithms Independent DMPC (idmpc) Regulator R 2 Minimization problem: min u 2 (t:t+n 1) V 2 Model of S 2 x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k) + A 21 x 1 (k) + B 21u 1 (k) where predictions u1 (t : t + N 1), x 1 (t : t + N 1) are available Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

95 MPC decentralized and distributed algorithms Independent DMPC (idmpc) Remarks: idmpc methods can be either iterative or non-iterative: predicted trajectories can be computed locally and transmitted iteratively during each sampling time to obtain more reliable predictions; the needed transmission network is partially connected. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

96 MPC decentralized and distributed algorithms Cooperative DMPC (cdmpc) Note that, cooperative solutions have been proposed for dynamically decoupled models, i.e., in case A 12 = 0 and A 21 = 0. Regulator R 1 Minimization problem: Model of S min u 1 (t:t+n 1) ρ 1V 1 + ρ 2 V 2 x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k) + B 12 u2 (k) x 2 (k + 1) = A 22 x 2 (k) + B 21 u 1 (k) + B 22 u2 (k) where predictions u2 (t : t + N 1) are available. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

97 MPC decentralized and distributed algorithms Cooperative DMPC (cdmpc) Note that, cooperative solutions have been proposed for dynamically decoupled models, i.e., in case A 12 = 0 and A 21 = 0. Regulator R 2 Minimization problem: Model of S min u 2 (t:t+n 1) ρ 1V 1 + ρ 2 V 2 x 1 (k + 1) = A 11 x 1 (k) + B 12 u 2 (k) + B 11 u1 (k) x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k) + B 21 u1 (k) where predictions u1 (t : t + N 1) are available. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

98 MPC decentralized and distributed algorithms Cooperative DMPC (cdmpc) Remarks: cdmpc methods are iterative: predicted trajectories must be computed locally and transmitted iteratively during each sampling time to obtain more reliable predictions; the needed transmission network is fully connected: predictions of the overall system response to u i (k) must be computed; some available algorithms guarantee stability even in the non-iterative formulation, while global optimality is guaranteed only when iterations converge. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

99 MPC decentralized and distributed algorithms Robust DMPC (rdmpc) Main idea At time t, S 1 sends to S 2 the nominal predicted reference trajectories u1 (t : t + N 1) and x 1 (t : t + N 1), guarantees (through suitable constraints in the optimization problem) that the actual trajectories u 1 (t : t + N 1) and x 1 (t : t + N 1) lie in suitable neighborhoods of u1 (t : t + N 1) and x1 (t : t + N 1), respectively, i.e., u 1 (k) u 1 (k) + U 1 x 1 (k) x 1 (k) + Z 1 In this way, the regulator R 2 has the following model of S 2 : x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k) + A 21 x 1 (k) + B 21 u 1 (k) = A 22 x 2 (k) + B 22 u 2 (k) + A 21 x 1 (k) + B 21u 1 (k)+ +A 21 (x 1 (k) x 1 (k)) + B 21(u 1 (k) u 1 (k)) = A 22 x 2 (k) + B 22 u 2 (k) + A 21 x 1 (k) + B 21u 1 (k) + w 2(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

100 MPC decentralized and distributed algorithms Robust DMPC (rdmpc) Main idea Therefore, at time t, the regulator R 2 has the following model of S 2 : x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k) + A 21 x 1 (k)+ +B 21 u 1 (k) + w 2(k) The disturbance w 2 (k) is bounded: a robust (tube-based) approach is used for control of this system. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

101 MPC decentralized and distributed algorithms Robust DMPC (rdmpc) Regulator R 1 Minimization problem: min u 1 (t:t+n 1) V 1 Model of S 1 x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k)+ +A 12 x2 (k) + B 12u2 (k) + w 1(k) where predictions (reference trajectories) u2 (t : t + N 1), x2 (t : t + N 1) are available, and w 1(k) is unknown but bounded. Further constraints: u 1 (k) u 1 (k) + U 1 x 1 (k) x 1 (k) + Z 1 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

102 MPC decentralized and distributed algorithms Robust DMPC (rdmpc) Regulator R 2 Minimization problem: min u 2 (t:t+n 1) V 2 Model of S 2 x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k)+ +A 21 x1 (k) + B 21u1 (k) + w 2(k) where predictions (reference trajectories) u1 (t : t + N 1), x1 (t : t + N 1) are available, and w 2(k) is unknown but bounded. Further constraints: u 2 (k) u 2 (k) + U 2 x 2 (k) x 2 (k) + Z 2 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

103 MPC decentralized and distributed algorithms Robust DMPC (rdmpc) Remarks: rdmpc methods are non-iterative: predicted trajectories must be computed locally and transmitted once during each sampling time; the needed transmission network is partially connected. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

104 MPC decentralized and distributed algorithms Distributed solution of centralized MPC optimization (DcMPC) Distributed solution of a centralized MPC problem from a control-theoretical perspective, it is equivalent to solve cmpc; from an optimization perspective, the one-step MPC problem is: R 1 and R 2 solve small-scale optimization problems; R 1 and R 2 exchange information; this procedure is iterated until convergence; there exist both hierarchical and distributed architectures. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

105 MPC decentralized and distributed algorithms Distributed solution of centralized MPC optimization (DcMPC) Minimization problem min u 1 (t:t+n 1),u 2 (t:t+n 1) ρ 1V 1 + ρ 2 V 2 System model S : x(k + 1) = Ax(k) + Bu(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

106 MPC decentralized and distributed algorithms Distributed solution of centralized MPC optimization (DcMPC) Remark: price coordination, primal and dual decomposition methods are widespread DcMPC algorithms; optimality and/or feasibility are generally attained only when the convergence of the decomposition algorithm (within each sampling time) is achieved. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

107 MPC decentralized and distributed algorithms Summary ALGO cmpc dmpc idmpc cdmpc rdmpc DcMPC #CSs 1 M M M M M Model S S i S i S i S i S i Cost fcn V V i V i V V i V tnw CS to all no n2n all-to-all n2n all-to-all CS load high small small small small small iters no no yes/no yes no yes opt yes no no yes no yes Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

108 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

109 Analysis of prototypical algorithms MAIN ASSUMPTIONS the system is partitioned in M = 2 subsystems; we have dynamically decoupled subsystems { x1 (k + 1) = A S 1 = 11 x 1 (k) + B 11 u 1 (k) + B 12 u 2 (k) { y 1 (k) = C 1 x 1 (k) x2 (k + 1) = A S 2 = 22 x 2 (k) + B 22 u 2 (k) + B 21 u 1 (k) y 2 (k) = C 2 x 2 (k) A 11 and A 22 are asymptotically stable; The cost functions are: V 1 = t+n 1 k=t = t+n 1 k=t V 2 = t+n 1 k=t = t+n 1 k=t 1 2 { y 1(k) 2 Q1 + u 1 (k) 2 R 1 } x 1(t + N) 2 P { x 1(k) 2 Q 1 + u 1 (k) 2 R 1 } x 1(t + N) 2 P { y 2(k) 2 Q2 + u 2 (k) 2 R 2 } x 2(t + N) 2 P { x 2(k) 2 Q 2 + u 2 (k) 2 R 2 } x 2(t + N) 2 P 2 where Q 1 = C T 1 Q 1 C 1 0 and Q 2 = C T 2 Q 2 C 2 0; Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

110 Analysis of prototypical algorithms MAIN ASSUMPTIONS constraints on input and state variables are neglected; being A 11 and A 22 asymptotically stable, the decentralized stabilizing auxiliary control law is u 1 = 0 and u 2 = 0 (i.e., K 1 = 0 and K 2 = 0); The final cost matrices P 1 and P 2 are chosen in such a way that A T 11 P 1A 11 P 1 = Q 1 A T 22 P 2A 22 P 2 = Q 2 i.e., 2 1 x 1 2 P 1 and 2 1 x 2 2 P 2 are the infinite-horizon costs-to-go under zero control; the global cost function is V = ρ 1 V 1 + ρ 2 V 2 where ρ 1 + ρ 2 = 1. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

111 Analysis of prototypical algorithms Centralized control (cmpc) cmpc problem subject to where where and V = minv u x(k + 1) = Ax(k) + Bu(k) t+n { x(k) 2 Q + u(k) 2 R } x(t + N) 2 P k=t [ ] [ ] A11 0 B11 B A =, B = 12 0 A 22 B 21 B 22 [ ] [ ] [ ] ρ1 Q Q = 1 0 ρ1 R, R = 1 0 ρ1 P, P = ρ 2 Q 2 0 ρ 2 R 2 0 ρ 2 P 2 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

112 Analysis of prototypical algorithms Centralized control (cmpc) Since: A is assumed to be asymptotically stable, u(k) = 0 (with K = 0) is a suitable stabilizing auxiliary control law, A T PA P = Q, the pair (A,Q) is detectable, then, according to the theory of MPC stability of cmpc is guaranteed! Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

113 Analysis of prototypical algorithms Decentralized control (dmpc) dmpc problems The unconstrained minimization problems solved by regulators R 1 and R 2 are min u1 (t:t+n 1) V 1 min u2 (t:t+n 1) V 2 subject to subject to x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k) x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k) The explicit solutions these problems can be found, since state and input constraints are not present: u1 o(k) = K 1 d x 1(k) u2 o(k) = K 2 d x 2(k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

114 Analysis of prototypical algorithms Decentralized control (dmpc) In fact, consider for example the model of S 1 x 1 (t) A u 1 (t) x 1 (t + 1) A 1 B u 1 (t + 1) = x. 1 (t) x 1 (t + N) }{{} A N 11 }{{} A N 1 11 B 11 } A N 2 11 B 11 {{... B 11 } u 1 (t + N 1) }{{} x 1 (t : t + N) A 11 B 11 u 1 (t : t + N 1) Denoting Q 1 =diag(q 1,...,Q 1,P 1 ) and R 1 =diag(r 1,...,R 1 ), V 1 can be written as V 1 = 1 2 A 11x 1 (t) + B 11 u 1 (t : t + N 1) 2 Q u 1(t : t + N 1) 2 R 1 The problem min u1 (t:t+n 1) V 1 leads to the solution u o 1 (t : t + N 1) = (R 1 + B T 11 Q 1B 11 ) 1 B T 11 Q 1A 11 x 1 (t) Denoting E i (i = 1,2) the matrix that selects from the vector u o i (t : t + N 1) the subvector u o i (t), i.e., E i = [ I ] we obtain that the gain K d 1 is K d 1 = E 1 (R 1 + B T 11 Q 1B 11 ) 1 B T 11 Q 1A 11 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

115 Analysis of prototypical algorithms Decentralized control (dmpc) Therefore K d 1 = E 1 (R 1 + B T 11 Q 1B 11 ) 1 B T 11 Q 1A 11 K d 2 = E 1 (R 2 + B T 22 Q 2B 22 ) 1 B T 22 Q 2A 22 Since K d 1 and K d 2 result from separated well-posed MPC problems: A 11 + B 11 K d 1 and A 22 + B 22 K d 2 are asymptotically stable. However, the collective close-loop system results in [ ] [ x1 (t + 1) A11 + B = 11 K1 d B 12 K d ] [ ] 2 x1 (t) x 2 (t + 1) B 21 K1 d A 22 + B 22 K2 2 x 2 (t) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

116 Analysis of prototypical algorithms Decentralized control (dmpc) Remarks [ ] x1 (t + 1) = x 2 (t + 1) [ A11 + B 11 K d 1 B 12 K d 2 B 21 K d 1 A 22 + B 22 K 2 2 ] [ ] x1 (t) x 2 (t) The stability of the system is not guaranteed, due to the non block-diagonal elements B 12 K d 2 and B 21K d 1. B 12 = 0 and/or B 21 = 0 if the two subsystems are non-interacting (when both B12 = 0 and B 21 = 0) the two subsystems are arranged in a cascaded setting (when only one of the two matrices is non-zero). For continuity arguments, under the assumption of small interaction ( small B 12 and/or B 21 ) the stability of the closed loop system is preserved. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

117 Analysis of prototypical algorithms Distributed control with independent agents (idmpc) idmpc problems Given predictions Given predictions u (p 1) 2 (t : t + N 1) u (p 1) 1 (t : t + N 1) Model of S 1 : Model of S 2 : x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k)+ x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k)+ +B 12 u (p 1) 2 (k) +B 21 u (p 1) 1 (k) Minimization problem for R 1 : Minimization problem for R 2 : min u1 (t:t+n 1) V 1 min u2 (t:t+n 1) V 2 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

118 Analysis of prototypical algorithms Distributed control with independent agents (idmpc) idmpc problems Given predictions Given predictions u (p 1) 2 (t : t + N 1) u (p 1) 1 (t : t + N 1) Model of S 1 : Model of S 2 : x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k)+ x 2 (k + 1) = A 22 x 2 (k) + B 22 u 2 (k)+ +B 12 u (p 1) 2 (k) +B 21 u (p 1) 1 (k) Minimization problem for R 1 : Minimization problem for R 2 : min u1 (t:t+n 1) V 1 min u2 (t:t+n 1) V 2 During each sampling period, at iteration p 0 of the negotiation algorithm : R 1 knows the predicted trajectory u (p 1) 2 (t : t + N 1), generated by R 2 (at step p 1), the result of the optimization carried out by R 1 is u1 (t : t + N 1 t), u (p) 1 (t : t + N 1) is generated as where w 1 [0,1]. u (p) 1 (t : t + N 1) = (1 w 1)u (p 1) 1 (t : t + N 1) + w 1 u1 (t : t + N 1 t) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

119 Analysis of prototypical algorithms Distributed control with independent agents (idmpc) Focusing for example on R 1 x 1 (t : t + N) = A 11 x 1 (t) + B 11 u 1 (t : t + N 1) B A N 1 11 B 12 A N 2 11 B B 12 }{{} B 12 u (p 1) 2 (t) u (p 1) 2 (t + 1). u (p 1) 2 (t + N 1) }{{} u (p 1) 2 (t : t + N 1) one obtains that u 1 (t : t + N 1 t) = (R 1 + B T 11 Q 1B 11 ) 1 B T 11 Q 1(A 11 x 1 (t) + B 12 u (p 1) 2 (t : t + N 1)) We denote and K i,n 1 = (R 1 + B T 11 Q 1B 11 ) 1 B T 11 Q 1A 11 L i,n 1 = (R 1 + B T 11 Q 1B 11 ) 1 B T 11 Q 1B 12 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

120 Analysis of prototypical algorithms Distributed control with independent agents (idmpc) Collectively we can write and [ ] [ u 1 (t : t + N 1 t) u2 (t : t + N 1 t) = K i,n K i,n 2 ] [x1 ] [ (t) + x 2 (t) 0 L i,n 1 L i,n 2 0 ][ u (p 1) 1 (t : t + N 1) u (p 1) 2 (t : t + N 1) ] [ u (p) ] 1 (t : t + N 1) u (p) = 2 (t : t + N 1) [ w 1 K i,n ] w 2 K i,n 2 }{{} [ ] x1 (t) + x 2 (t) [ (1 w 1 )I w 1 L i,n ] 1 w 2 L i,n 2 (1 w 2 )I }{{} [ u (p 1) ] 1 (t : t + N 1) u (p 1) 2 (t : t + N 1) }{{} K i,n L i,n u (p 1) (t : t + N 1) leading to the iterative equation u (p) (t : t + N 1) = K i,n x(k) + L i,n u (p 1) (t : t + N 1) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

121 Analysis of prototypical algorithms Distributed control with independent agents (idmpc) u (p) (t : t + N 1) = K i,n x(k) + L i,n u (p 1) (t : t + N 1) Convergence of the inter-sampling iterations Only if the matrix L i,n has all the eigenvalues inside the unit circle, then u (p) (t : t + N 1) p + ū(t : t + N 1) = (I L i,n ) 1 K i,n x(k) If such a solution exists K D = E(I L i,n ) 1 K i,n is the resulting (Nash solution) control gain, being E =diag(e 1,E 2 ). The collective close-loop system results in However x(k + 1) = (A + B K D )x(k) there is no guarantee that A + B K D is asymptotically stable! Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

122 Analysis of prototypical algorithms Distributed control with independent agents (idmpc) Different cases I) the Nash equilibrium does not exist (i.e., L i,n is an unstable matrix): no control law can be found (at least after an infinite number of negotiation steps); II) the Nash equilibrium exists (i.e., L i,n is an asymptotically stable matrix), but the closed loop is unstable: a control law is found after an infinite number of negotiation steps, but it leads to instability of the closed loop system; III) the Nash equilibrium is stable (i.e., L i,n is an asymptotically stable matrix), and the closed loop is stable (A + B K D is stable): a stabilizing control law can be found after an infinite number of negotiation steps. However, global optimality of the controlled system is generally not obtained. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

123 Analysis of prototypical algorithms Distributed control with cooperative agents (cdmpc) cdmpc problems Given predictions Given predictions u (p 1) 2 (t : t + N 1) u (p 1) 1 (t : t + N 1) min u1 (t:t+n 1) ρ 1 V 1 + ρ 2 V 2 min u2 (t:t+n 1) ρ 1 V 1 + ρ 2 V 2 subject to x 1 (k+1)=a 11 x 1 (k)+b 11 u 1 (k)+b 12 u (p 1) 2 (k) x 2 (k+1)=a 22 x 2 (k)+b 21 u 1 (k)+b 22 u (p 1) 2 (k) subject to x 2 (k+1)=a 22 x 2 (k)+b 22 u 2 (k)+b 21 u (p 1) 1 (k) x 1 (k+1)=a 11 x 1 (k)+b 12 u 2 (k)+b 11 u (p 1) 1 (k) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

124 Analysis of prototypical algorithms Distributed control with cooperative agents (cdmpc) cdmpc problems Given predictions Given predictions u (p 1) 2 (t : t + N 1) u (p 1) 1 (t : t + N 1) min u1 (t:t+n 1) ρ 1 V 1 + ρ 2 V 2 min u2 (t:t+n 1) ρ 1 V 1 + ρ 2 V 2 subject to x 1 (k+1)=a 11 x 1 (k)+b 11 u 1 (k)+b 12 u (p 1) 2 (k) x 2 (k+1)=a 22 x 2 (k)+b 21 u 1 (k)+b 22 u (p 1) 2 (k) subject to x 2 (k+1)=a 22 x 2 (k)+b 22 u 2 (k)+b 21 u (p 1) 1 (k) x 1 (k+1)=a 11 x 1 (k)+b 12 u 2 (k)+b 11 u (p 1) 1 (k) During each sampling period, at iteration p 0 of the negotiation algorithm : R 1 knows the predicted trajectory u (p 1) 2 (t : t + N 1), generated by R 2 (at step p 1), the result of the optimization carried out by R 1 is u1 (t : t + N 1 t), u (p) 1 (t : t + N 1) is generated as where w 1 [0,1]. u (p) 1 (t : t + N 1) = (1 w 1)u (p 1) 1 (t : t + N 1) + w 1 u1 (t : t + N 1 t) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

125 Analysis of prototypical algorithms Distributed control with cooperative agents (cdmpc) Define V(x(t),u 1 (t : t + N 1),u 2 (t : t + N 1)) as the value assumed by V subject to system x 1 (k + 1) = A 11 x 1 (k) + B 11 u 1 (k) + B 12 u 2 (k) x 2 (k + 1) = A 22 x 1 (k) + B 21 u 1 (k) + B 22 u 2 (k) where the inputs of the previous system are set as the argument of the functions V, i.e., u 1 (k) = u1 (k), k = t,...,t + N 1 u 2 (k) = u2 (k), k = t,...,t + N 1 and with initial condition (x 1 (t),x 2 (t)) = x(t). Results It is possible to prove that I) the optimal cost function V decreases from the time instant t 1 to the time instant t, if no negotiation steps are performed; II) the optimal cost function V decreases at each negotiation step. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

126 Analysis of prototypical algorithms Distributed control with cooperative agents (cdmpc) Main result From I) and II), the decrease of the cost function V is proved. Stability of cdmpc is guaranteed for an arbitrary number of iterations (i.e., negotiation steps) of the algorithm within each sampling interval, and that (see in particular item II)), for p + u (p) i (t : t + N 1) u (opt) i (t : t + N 1) for i = 1,2. That is, cdmpc leads to the optimal solution obtainable with centralized MPC. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

127 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Assume that reference (predicted) trajectories u 1 (t) and u 2 (t) are defined for u 1(t) and u 2 (t), respectively. x 1 (t + 1) = A 11 x 1 (t) + B 11 u 1 (t) + B 12 u2 (t) + w 1(t) x 2 (t + 1) = A 22 x 2 (t) + B 22 u 2 (t) + B 21 u1 (t) + w 2(t) Remarks through suitable constraints in the optimization problem we guarantee then the terms u 1 (t) = u 1 (t) + U 1 u 2 (t) = u 2 (t) + U 2 w 1 (t) = B 12 (u 2 (t) u2 (t)) w 2 (t) = B 21 (u 1 (t) u1 (t)) can be treated as bounded unknown disturbances for subsystems S 1 and S 2, specifically w 1 (t) W 1 = B 12 U 2 w 2 (t) W 2 = B 21 U 1 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

128 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Considering the equations: x 1 (t + 1) = A 11 x 1 (t) + B 11 u 1 (t) + B 12 u2 (t) + w 1(t) x 2 (t + 1) = A 22 x 2 (t) + B 22 u 2 (t) + B 21 u1 (t) + w 2(t) Exogenous signals and noise the effect of w 1 (t) and w 2 (t) must be rejected by the respective regulators; u1 (k) and u 2 (k) are known exogenous input, whose presence is to be compensated (i.e., explicitly taken into account in the control law). We use tube-based robust MPC to reject w 1 (t) and w 2 (t). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

129 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Perturbed models Nominal models where x 1 (t + 1) = A 11 x 1 (t) + B 11 u 1 (t) + B 12 u2 (t) + w 1(t) x 2 (t + 1) = A 22 x 2 (t) + B 22 u 2 (t) + B 21 u1 (t) + w 2(t) ^x 1 (t + 1) = A 11^x 1 (t) + B 11^u 1 (t) + B 12 u2 (t) ^x 2 (t + 1) = A 22^x 2 (t) + B 22^u 2 (t) + B 21 u1 (t) u 1 (t) = ^u 1 (t) + K 1 (x 1 (t) ^x 1 (t)) u 2 (t) = ^u 2 (t) + K 2 (x 2 (t) ^x 2 (t)) and K i is defined in such a way that A ii + B ii K i is asymptotically stable for i = 1,2. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

130 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) rdmpc problems Given u2 (t : t + N 1) Given u 1 (t : t + N 1) min^u1 (t:t+n 1) V 1 min^u2 (t:t+n 1) V 2 subject to subject to ^x 1 (k + 1) = A 11^x 1 (k) + B 11^u 1 (k)+ ^x 2 (k + 1) = A 22^x 2 (k) + B 22^u 2 (k)+ +B 12 u 2 (k) +B 21u 1 (k) ^u 1 (k) u 1 (k) ΔU 1, k = t,...,t + N 1 ^u 2 (k) u 2 (k) ΔU 1, k = t,...,t + N 1 Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

131 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Denoting z 1 (t) = x 1 (t) ^x 1 (t) and z 2 (t) = x 2 (t) ^x 2 (t): z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) Robust positively invariant sets Define Z 1 and Z 2 as an robust positively invariant sets for z 1 and z 2, respectively, such that guarantees that for all t 0. x i (0) ^x i (0) Z i, w i W i x i (t) ^x i (t) Z i and u i (t) ^u i (t) K i Z i = ΔU i Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

132 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Additional constraints in the optimization problems In the MPC problems, the following constraints are present: for k = t,...,t + N 1. ^u 1 (k) u1 (k) ΔU 1 ^u 2 (k) u2 (k) ΔU 1 Boundedness of the disturbance The robust positive invariant set and the additional constraints guarantee the boundedness of w 1 and w 2. In fact u i (t) ui (t) = u i(t) ^u i (t) }{{} + ^u i (t) ui }{{ (t) } U i K i Z i ΔU i U i Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

133 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Outputs of the rdmpc problem The result of the optimization problems are the values ^x 1 (t t) and ^x 2 (t t), and the trajectories ^u 1 (t : t + N 1 t) and ^u 2 (t : t + N 1 t), the inputs to the real systems are: u 1 (t) = ^u 1 (t t) + K 1 (x 1 (t) ^x 1 (t t)) u 2 (t) = ^u 2 (t t) + K 2 (x 2 (t) ^x 2 (t t)) the state trajectories stemming from these inputs are denoted ^x 1 (t + 1 : t + N t) and ^x 2 (t + 1 : t + N t), the reference trajectories are updated by setting u1 (t + N) = K1^x 1(t + N t) u2 (t + N) = K2^x 2(t + N t) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

134 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Main result Convergence of rdmpc is guaranteed if I) for all i = 1,2, A ii + B ii K i is stable, II) for all i = 1,2, there exist U i and Z i such that K i Z i U i. This is needed to guarantee the existence of ΔU i = U i K i Z i ; III) the gains K i must stabilize the system in a decentralized fashion. In fact, denoting K =diag(k 1,K 2 ), A + BK is stable; IV) the auxiliary control must guarantee that x(k + 1) 2 P x(k) 2 P ( x(k) 2 Q + u(k) 2 R ) where P, Q, and R are block-diagonal matrices and where the state is updated using the decentralized auxiliary control law K: u(k) = Kx(k) and x(k + 1) = (A + BK)x(k). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

135 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. WHY? Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

136 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. WHY? Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

137 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. WHY? Equations 1. Given U 2, then W 1 = B 12 U 2 is defined. Given U 1, then W 2 = B 21 U 1 is defined. z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

138 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. WHY? Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) 1. Given U 2, then W 1 = B 12 U 2 is defined. Given U 1, then W 2 = B 21 U 1 is defined. 2. Given W 1, then Z 1 is defined. Given W 1, then Z 1 is defined. where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

139 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. WHY? Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) 1. Given U 2, then W 1 = B 12 U 2 is defined. Given U 1, then W 2 = B 21 U 1 is defined. 2. Given W 1, then Z 1 is defined. Given W 1, then Z 1 is defined. 3. u 1 (t) ^u 1 (t) K 1 Z 1. u 2 (t) ^u 2 (t) K 2 Z 2. where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

140 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. WHY? Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) 1. Given U 2, then W 1 = B 12 U 2 is defined. Given U 1, then W 2 = B 21 U 1 is defined. 2. Given W 1, then Z 1 is defined. Given W 1, then Z 1 is defined. 3. u 1 (t) ^u 1 (t) K 1 Z 1. u 2 (t) ^u 2 (t) K 2 Z Only if K 1 Z 1 U 1 we can define ΔU 1 = U 1 K 1 Z 1 to constrain u 1 (t) ^u 1 (t) in the MPC problem. Only if K 2 Z 2 U 2 we can define ΔU 2 = U 2 K 2 Z 2 to constrain u 2 (t) ^u 2 (t) in the MPC problem. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

141 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. IT IS A SMALL GAIN CONDITION! Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

142 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. IT IS A SMALL GAIN CONDITION! Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

143 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. IT IS A SMALL GAIN CONDITION! Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

144 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Remark on II) For all i = 1,2, there exist U i and Z i such that K i Z i U i. IT IS A SMALL GAIN CONDITION! Equations z 1 (t + 1) = (A 11 + B 11 K 1 )z 1 (t) + w 1 (t) z 2 (t + 1) = (A 22 + B 22 K 2 )z 2 (t) + w 2 (t) where w 1 (t) = B 12 (u2 (t) u 2(t)) w 2 (t) = B 21 (u1 (t) u 1(t)) Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

145 Analysis of prototypical algorithms Distributed control with robust agents (rdmpc) Application of the main result to the simple case Under the given assumptions, the choice K 1 = 0, K 2 = 0 is sufficient to guarantee that all the assumptions (I-IV) are satisfied. In fact I) for all i = 1,2, A ii + B ii K i = A ii is stable; II) define arbitrary neighborhoods of the origin U 1 and U 2. In turn, this defines W 1 = B 12 U 2 and W 2 = B 21 U 1 : Since A 11 + B 11 K 1 and A 22 + B 22 K 2 are stable, RPI sets Z 1 and Z 2 can be automatically defined. Since K 1 = 0 and K 2 = 0, then III) A + BK = A is stable; IV) matrix P =diag(p 1,P 2 ) satisfies K 1 Z 1 U 1 and K 2 Z 2 U 2 A T PA P = Q Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

146 Analysis of prototypical algorithms Summary Among the presented algorithms: Stability can be guaranteed a-priori for centralized MPC (cmpc); distributed control with cooperative agents (cdmpc); distributed control with robust agents (rdmpc). Optimality is guaranteed by: centralized MPC (cmpc); distributed control with cooperative agents (cdmpc), when inter-sampling iterations reach the convergence. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

147 Analysis of prototypical algorithms A simple example The systems x 1 (t + 1) = a 1 x 1 (t) + b 11 u 1 (t) + b 12 u 2 (t) x 2 (t + 1) = a 2 x 2 (t) + b 22 u 2 (t) + b 12 u 1 (t) Model parameters a 1 = 0.99 and a 2 = 0.9. MPC parameters: N = 1, R 1 = R 2 = 0.1, Q 1 = Q 2 = 1 and P i = Q i /(1 a i ), i = 1,2. idmpc and cdmpc parameters: w 1 = w 2 = 0.5 and ρ 1 = ρ 2 = 0.5. Three cases: I) b 11 = 1, b 12 =.5, b 21 =.2, and b 22 = 1; II) b 11 = 0.001, b 12 = 1.1, b 21 = 0.9, and b 22 = 10; III) b 11 = 1, b 12 = 5, b 21 = 2, and b 22 = 1. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

148 Analysis of prototypical algorithms A simple example Case I cmpc: the spectral radius of the controlled system is dmpc: b 12 and b 21 are sufficiently small to preserve stability of the closed-loop system. Spectral radius: idmpc: The matrix L i,n is stable, then there exists a Nash equilibrium. If p the obtained controlled system has a spectral radius cdmpc: If p the obtained controlled system has a spectral radius (i.e., it is equal to the cmpc). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

149 Analysis of prototypical algorithms A simple example Case I Solutions of the optimization problems using different distributed control approaches. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

150 Analysis of prototypical algorithms A simple example Case I Solutions of the optimization problems using different distributed control approaches. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

151 Analysis of prototypical algorithms A simple example Case I Simulations Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

152 Analysis of prototypical algorithms A simple example Case II cmpc: the spectral radius of the controlled system is dmpc: b 12 and b 21 are not sufficiently small to preserve stability, and the closed-loop system is unstable (spectral radius 1.071). idmpc: The matrix L i,n is stable, then there exists a Nash equilibrium. If p the obtained controlled system has a spectral radius cdmpc: If p the obtained controlled system has a spectral radius (i.e., it is equal to the cmpc). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

153 Analysis of prototypical algorithms A simple example Case II Solutions of the optimization problems using different distributed control approaches. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

154 Analysis of prototypical algorithms A simple example Case II Simulations Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

155 Analysis of prototypical algorithms A simple example Case III cmpc: the spectral radius of the controlled system is dmpc: b 12 and b 21 are not sufficiently small to preserve stability of the closed-loop system. Spectral radius: idmpc: The matrix L i,n is unstable (spectral radius equal to 2.073). No control. cdmpc: If p the obtained controlled system has a spectral radius (i.e., it is equal to the cmpc). Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

156 Analysis of prototypical algorithms A simple example Case III Solutions of the optimization problems using different distributed control approaches. Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

157 Analysis of prototypical algorithms A simple example Case III Simulations Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

158 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

159 Outline 1 Information on the lecture 2 Introduction to MPC 3 Introduction to dynamic non-cooperative games 4 Models 5 MPC decentralized and distributed algorithms 6 Analysis of prototypical algorithms 7 Examples Temperature control Three-tank system 8 Conclusions Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147

160 Examples Temperature control Consider the problem of controlling the temperature of the building Farina, Ferrari Trecate () Decentralized and distributed control EECI-HYCON2 School / 147