Team Decision Theory and Information Structures in Optimal Control Problems

Size: px

Start display at page:

Download "Team Decision Theory and Information Structures in Optimal Control Problems"

Marcus King
5 years ago
Views:

1 Team Decision Theory and Information Structures in Optimal Control Problems YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH October 4, 2011 YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

2 Problem Setup Team: A set of decision makers which have a common goal, I = {1, 2,, 3}. The member i I receives information z i and makes a decision u i = γ i (z i ), where γ i Γ i. A common pay-off: J(γ 1, γ 2,, γ N ). Problem 1: Find γ i Γ i for all i such that J is minimized. Static vs Dynamic Teams. YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

3 Information Structures Let ξ R n represent uncertainties of the external world, with pdf ξ N (0, X ) known to all members. The information variable z i = H i ξ + j D iju j. Interested in causal systems: D ij 0 D ji = 0. Definition 1: We say j is related to i, jri, if D ij 0. Definition 2: We say j is precedent of i, j{i, iff (a) jri or (b) there exist distinct r, s,, k I such that jrr and rrs,..., kri. Precedence diagram to graphical represent the idea of precedence. YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

4 Information Structures-Example 1 Example 1: D ij = 0, i and j z i = H i ξ, i (1) Static or dynamic team? Precedence diagram. No causal precedence relation among DMs, therefore actual time instants of observation and decisions are not important. YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

5 Information Structures-Example 2 Example 2 (Classical multi-stage stochastic control) : A dynamic LTI system (organization) x i+1 = Fx i + Gu i + w i, i = 1, 2,, N, (2) y i = Hx i + v i, i = 1, 2,, N, (3) where x i is the state, u i the control, and w i, v i independent sequences. Perfect memory system Each DM remembers it s own past observations and decisions. Express information at the i th DM in the following form: z i = H i ξ + j D iju j. Draw the precedence diagram with. YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

6 Pay-off Function and Control Laws Problem P1: The common goal for all members is to minimize J(γ 1,, γ N ) = E[J ] = E[u T Qu + u T Sξ + u T c], where u = [u 1 u 2 u N ] T, u i = γ i (z i ), γ i Γ i, i {1, 2,, N}. Problem P2 (Member-by-member optimality): Find γi such that Γ i for all i J(γ 1,, γ i 1, γ i, γ i+1,, γ N ) J(γ 1,, γ i 1, γ i, γ i+1,, γ N ) for all γ i Γ i and for all i. Problem P3: For fixed γ j (z j), j i, and any z i min E[u T Qu + u T Sξ + u T c] =: min J i (4) u i u i YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

7 Optimal control laws for STATIC TEAMS A linear information structure: z i = H i ξ for all i. Theorem (due to Radner): The optimal control law for each DM is unique and affine in it s observation variable. That is of the form, u i = γ i (z i ) + b i. YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

8 Dynamic Teams with Partially Nested Information Structure An information structure is called partially nested if j{i implies Z j Z i for all i, j, and γ. Theorem 1: In a dynamic team with partially nested information structure, z i = H i ξ + j{i D ij u j (5) is equivalent to an information structure in static form for any fixed set of control laws: ẑ i = [{H i ξ j{i or j = i}]. (6) YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

9 Partially Nested Information Structure-Example Example: YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

10 Optimal control laws for Partially Nested Information Structure Theorem 2: In a dynamic team with partially nested information structure, the optimal control for each member exists, is unique and linear. How to prove? Application 1: LQG Control Problem. YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

11 Application 2: One-step Communication Delay Control System Two coupled linear discrete-time dynamic systems controlled by u 1 (t) and u 2 (t), t = 1, 2,, N The two controllers observe: z 1 (t) = {y 1 (τ), y 2 (k) τ = 1, 2,, t; k = 1, 2,, t 1} z 2 (t) = {y 2 (τ), y 1 (k) τ = 1, 2,, t; k = 1, 2,, t 1} YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

12 Further applications: Interconnected/ Hierarchical System YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

13 Take-home message In a decentralized decision-making environment, if a DM s action effects our information, then knowing what he knows will yield linear optimal solutions. Thank you for your attention! YU-CHI HO and K AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH () October 4, / 13

McGill University Department of Electrical and Computer Engineering

McGill University Department of Electrical and Computer Engineering ECSE 56 - Stochastic Control Project Report Professor Aditya Mahajan Team Decision Theory and Information Structures in Optimal Control