Utility Design for Distributed Engineering Systems

Transcription

1 Utility Design for Distributed Engineering Systems Jason R. Marden University of Colorado at Boulder (joint work with Na Li, Caltech) University of Oxford June 7, 2011

2 Cooperative control Goal: Derive desirable collective behavior through local control algorithms Sensor coverage Appeal Local processing (manageable) Reduces communication Robustness Network Coding Wind farms Challenges Characterization Coordination Efficiency Trend: Game theory popular tool for cooperative control

3 Game theory social system Descriptive Agenda: Modeling model as game Decision Makers game theory Global Behavior Metrics: Reasonable description of sociocultural phenomena? Matches available experimental/observational data?

4 Game theory social system Prescriptive Agenda: Distributed robust optimization engineering system model as game Decision Makers game theory Global Behavior desired global behavior Metrics: distributed control Design parameters: Asymptotic global behavior? Communication/Information requirement? Computation requirement? Convergence rates? Decision makers Objective/Utility functions Decision/Learning rule

5 Why game theory? Reason #1: Constrained distributed optimization Global objective: Agent control policies: G( ) π i ( ) Goal: Design admissible control policies Wind farms Conventional distributed optimization: G( ) π 1 ( )... π n ( ) Structure of policies a byproduct of structure of global objective

6 Why game theory? Reason #1: Constrained distributed optimization Global objective: Agent control policies: G( ) π i ( ) Goal: Design admissible control policies Wind farms Game theoretic distributed optimization: U 1 ( ) π 1 ( ) Policies now depend on utility functions G( ) heterogeneity admissibility U n ( ) π n ( ) heterogeneity = game theory

7 Why game theory? Reason #1: Constrained distributed optimization Global objective: Agent control policies: G( ) π i ( ) Goal: Design admissible control policies Wind farms Reason #2: Hierarchical decomposition between game design and learning design Game design game structure Learning design Ex: Potential games Modularization of design Wide array of existing learning algorithms Robustness to decision rules [Young, 2005] [Gopalakrishnan, JRM, and Wierman, 2010]

8 Game theoretic control Setup Dynamics Model interactions as game decision makers / players possible choices local objective functions Potential games Local agent decision rules informational dependencies processing requirements Architecture common to many designs [Zhu and Martinez, 2009] [Kaumann et al. 2007] [Marden et al. 2007, 2008] [Mhatre et al. 2007] [Komali and MacKenzie 2007] [Zou and Chakrabarty 2004] [Campos-Nanez 2008] and many others

9 Game theoretic control Setup Model interactions as game decision makers / players possible choices local objective functions Desirable Properties Existence of (pure) NE Efficiency of NE Locality of information Potential game? Potential games GOAL Emergent global behavior desirable Dynamics Local agent decision rules informational dependencies processing requirements Desirable Properties Asymptotic behavior Limited information Fast convergence Equilibrium selection...

10 Game theoretic control Setup Model interactions as game decision makers / players possible choices local objective functions Desirable Properties Existence of (pure) NE Efficiency of NE Locality of information Potential game? Potential games GOAL Emergent global behavior desirable Dynamics Local agent decision rules informational dependencies processing requirements Learning in games [Young, ] [Marden et al., ] [Leslie et al., ] [Shah and Shin,2009] [Montanari and Saberi, 2009] and many others

11 Game theoretic control Setup Model interactions as game decision makers / players possible choices local objective functions Limited work [Wolpert and Tumer, 1999] [Arslan et al., 2007] [Marden and Wierman, 2009,10] many unresolved questions Potential games GOAL Emergent global behavior desirable Dynamics Local agent decision rules informational dependencies processing requirements Learning in games [Young, ] [Marden et al., ] [Leslie et al., ] [Shah and Shin,2009] [Montanari and Saberi, 2009] and many others

12 Game design game structure Learning design What is the viability of normal form games as a mediating layer for this decomposition? Goal: Develop underlying theory for game design to meet specifications: Constrained utility functions Efficiency of NE with respect to system level objective Game possesses desirable structure

13 Game design game structure Learning design What is the viability of normal form games as a mediating layer for this decomposition? Goal: Develop underlying theory for game design to meet specifications: Constrained utility functions Efficiency of NE with respect to system level objective Game possesses desirable structure Limitations: Normal form games not rich enough to meet objectives Coupled constraints in system level objective [N. Li and JRM, 2011] NE + Local + BB = computationally prohibitive [JRM and A. Wierman, 2011] NE + Local + BB = price of stability < 1 [JRM and A. Wierman, 2011]

14 Game design game structure Learning design What is the viability of normal form games as a mediating layer for this decomposition? Goal: Develop underlying theory for game design to meet specifications: Constrained utility functions Efficiency of NE with respect to system level objective Game possesses desirable structure Limitations: Normal form games not rich enough to meet objectives Coupled constraints in system level objective [N. Li and JRM, 2011] NE + Local + BB = computationally prohibitive [JRM and A. Wierman, 2011] NE + Local + BB = price of stability < 1 [JRM and A. Wierman, 2011]

15 Example: Consensus Global objective: Reach consensus on average of initial values using admissible controllers Setup: Player set: Information set: Value set: Initial values: N N i N V i v i (0) [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

16 Example: Consensus System level objective v i v j i N min v V Admissible controllers v i (t) =Π i Info about j at time t s.t. i N v i = i N v i (0) [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

17 Example: Consensus System level objective v i v j i N min v V s.t. i N v i = i N v i (0) Admissible controllers v i (t) =Π i Info about j at time t coupled constraint [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

18 Example: Consensus System level objective v i v j i N min v V s.t. i N v i = i N v i (0) Admissible controllers v i (t) =Π i Info about j at time t coupled constraint locality constraint [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

19 Example: Consensus System level objective v i v j i N min v V s.t. i N v i = i N v i (0) Admissible controllers v i (t) =Π i Info about j at time t coupled constraint locality constraint Game theoretic goal Define local and scalable cost functions such that all NE minimize system level objective while satisfying the coupled constraint irrespective of initial setup. U i (v i,v i )=F {v j,v j (0)} j Ni [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

20 Example: Consensus System level objective v i v j i N min v V s.t. i N v i = i N v i (0) Admissible controllers v i (t) =Π i Info about j at time t coupled constraint locality constraint Game theoretic goal Define local and scalable cost functions such that all NE minimize system level objective while satisfying the coupled constraint irrespective of initial setup. U i (v i,v i )=F {v j,v j (0)} j Ni locality + scalability constraint embedded into cost functions [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

21 Example: Consensus System level objective v i v j i N min v V s.t. i N v i = i N v i (0) Admissible controllers v i (t) =Π i Info about j at time t coupled constraint locality constraint locality + scalability constraint embedded into cost functions Game theoretic goal Define local and scalable cost functions such that all NE minimize system level objective while satisfying the coupled constraint irrespective of initial setup. U i (v i,v i )=F {v j,v j (0)} j Ni IMPOSSIBLE [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

22 Impossibility Setting #1: Setting #2: v i (0) = 0 v i (0) = 1 v i (0) = 0 v i (0) = 1 [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

23 Impossibility Setting #1: Setting #2: v i (0) = 0 v i (0) = 1 v i (0) = 0 v i (0) = 1 U i (v i,v i )=F FIXED {v j,v j (0)} j Ni [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

24 Impossibility Setting #1: Setting #2: v i (0) = 0 v i (0) = 1 v i (0) = 0 v i (0) = 1 U i (v i,v i )=F FIXED {v j,v j (0)} j Ni Problem: (v, v, v, v) (v, v, v, v, v) is a NE setting #1 is a NE setting #2 [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

25 Impossibility Setting #1: Setting #2: v i (0) = 0 v i (0) = 1 v i (0) = 0 v i (0) = 1 U i (v i,v i )=F FIXED {v j,v j (0)} j Ni Problem: (v, v, v, v) (v, v, v, v, v) is a NE setting #1 is a NE setting #2 [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

26 Goal: Develop underlying theory for utility design Goal: Develop underlying theory for utility design This talk Social systems: Normal form games is natural choice Engineering systems: Normal form games is one possible design choice Limitations: Provide analytical justification for moving beyond normal form games Game design game structure This talk: introduces state into game environment additional design freedom Learning design introduces state into learning environment [Young, 2010] [Pradelski & Young, 2011] [Arslan & Shamma, 2007] [JRM, State based potential games, 2011]

27 Goal: Develop underlying theory for utility design Goal: Develop underlying theory for utility design This talk Part 1: Formally introduce state base games as a new mediating layer Normal form game {1,...,n} A i U i : A R Players Actions Utilities [JRM, State based potential games, 2011]

28 Goal: Develop underlying theory for utility design Goal: Develop underlying theory for utility design This talk Part 1: Formally introduce state base games as a new mediating layer Normal form game {1,...,n} A i U i : A R Players Actions Utilities States State Transition State based game {1,...,n} A i U i : X A R X P : X A X [JRM, State based potential games, 2011]

29 Goal: Develop underlying theory for utility design Goal: Develop underlying theory for utility design This talk Part 1: Formally introduce state base games as a new mediating layer Myopic players Static equilibrium concepts Potential game extension Players Actions Utilities States State Transition State based game {1,...,n} A i U i : X A R X P : X A X [JRM, State based potential games, 2011]

30 Goal: Develop underlying theory for utility design Goal: Develop underlying theory for utility design This talk Part 1: Formally introduce state base games as a new mediating layer Myopic players Static equilibrium concepts Potential game extension Players Actions Utilities States State Transition State based game {1,...,n} A i U i : X A R X P : X A X Part 2: Develop theory for utility design in state based games Local utility design System level objectives with coupled constraints [JRM, State based potential games, 2011]

31 State based games Two simplifications: Myopic players Static equilibrium concepts Repeated play: State at time t: x(t) Each player myopically updates action: a i (t) =F i x(t), {x(τ),a(τ)} τ=0,1,...,t 1 State based game {1,...,n} A i U i : X A R X P : X A X a i (t) arg max a i A i U i (x(t),a i,a i (t 1)) myopic Cournot adjustment process One-shot payoff: State at time t+1: U i (x(t),a(t)) x(t + 1) = f(x(t),a(t)) [JRM, State based potential games, 2011]

32 Recurrent state equilibrium Action invariant state trajectory: X(x 0,a 0 )={x 1,x 2,x 3,...} x k+1 = P (x k,a 0 ) Definition: A state action pair [x 0,a 0 ] is a recurrent state equilibrium if U i (x, a 0 ) = max U i (x, a i,a 0 a i A i i) for all x X(x 0,a 0 ) x 0 X(x, a 0 ) for all x X(x 0,a 0 ) Recurrent state equilibrium fixed point of myopic Cournot adjustment process [JRM, State based potential games, 2011]

33 State based games Definition: A state based game is a state based potential game if there exists a potential function such that for any state action pair φ : X A R U i (x, a i,a i ) U i (x, a i,a i )=φ(x, a i,a i ) φ(x, a i,a i ) φ(x,a) φ(x, a) x = f(x, a) for [x, a] [JRM, State based potential games, 2011]

34 State based games Definition: A state based game is a state based potential game if there exists a potential function such that for any state action pair φ : X A R U i (x, a i,a i ) U i (x, a i,a i )=φ(x, a i,a i ) φ(x, a i,a i ) φ(x,a) φ(x, a) x = f(x, a) for [x, a] USUAL CONDITION alignment with potential function for unilateral deviations [JRM, State based potential games, 2011]

35 State based games Definition: A state based game is a state based potential game if there exists a potential function such that for any state action pair φ : X A R U i (x, a i,a i ) U i (x, a i,a i )=φ(x, a i,a i ) φ(x, a i,a i ) φ(x,a) φ(x, a) x = f(x, a) for [x, a] NEW CONDITION potential function nondecreasing along action invariant state trajectory USUAL CONDITION alignment with potential function for unilateral deviations [JRM, State based potential games, 2011]

36 State based games Definition: A state based game is a state based potential game if there exists a potential function such that for any state action pair φ : X A R U i (x, a i,a i ) U i (x, a i,a i )=φ(x, a i,a i ) φ(x, a i,a i ) φ(x,a) φ(x, a) x = f(x, a) for [x, a] NEW CONDITION potential function nondecreasing along action invariant state trajectory USUAL CONDITION alignment with potential function for unilateral deviations Fact: A recurrent state equilibrium exists in any state based potential game [JRM, State based potential games, 2011]

37 State based games Definition: A state based game is a state based potential game if there exists a potential function such that for any state action pair φ : X A R U i (x, a i,a i ) U i (x, a i,a i )=φ(x, a i,a i ) φ(x, a i,a i ) φ(x,a) φ(x, a) x = f(x, a) for [x, a] NEW CONDITION potential function nondecreasing along action invariant state trajectory USUAL CONDITION alignment with potential function for unilateral deviations Fact: A recurrent state equilibrium exists in any state based potential game Fact: Many learning algorithms for potential games extend to SBPG Gradient play Log-linear learning Finite memory better reply process extend to state based potential games [JRM, State based potential games, 2011]

38 State based games Definition: A state based game is a state based potential game if there exists a potential function such that for any state action pair φ : X A R U i (x, a i,a i ) U i (x, a i,a i )=φ(x, a i,a i ) φ(x, a i,a i ) φ(x,a) φ(x, a) x = f(x, a) for [x, a] NEW CONDITION potential function nondecreasing along action invariant state trajectory USUAL CONDITION alignment with potential function for unilateral deviations Fact: A recurrent state equilibrium exists in any state based potential game Fact: Many learning algorithms for potential games extend to SBPG Gradient play Log-linear learning Finite memory better reply process ROBUSTNESS extend to state based potential games [JRM, State based potential games, 2011]

39 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions [Na Li and JRM, Designing games for distributed optimization, 2011]

40 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions flavor of design introduce state as estimation parameter State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

41 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions flavor of design introduce state as estimation parameter State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

42 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

43 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions x =(x 1,...,x n ) x i =(v i,e 1 i,e 2 i,...,e n i ) local state variables State based game {1,...,n} X v i e k i actual value agent i agent i s estimate of agent k s value A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

44 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

45 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions a i = ˆv i ê k i j any player ˆv i, ê k i j,k N change in value any neighbor change in estimate of agent k s value State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

46 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

47 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions x =(v, e) a =(ˆv, ê) local value update ṽ i = v i +ˆv i local estimate update ẽ k i = e k i ê k i j + passed to neighbors ê k j i passed from neighbors k = i State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

48 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions x =(v, e) a =(ˆv, ê) local value update ṽ i = v i +ˆv i local estimate update ẽ k i = e k i ê k i j + ẽ i i = e i i + nˆv i ê k j i ê i i j + k = i ê i j i State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

49 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions x =(v, e) a =(ˆv, ê) local value update ṽ i = v i +ˆv i local estimate update ẽ k i = e k i ê k i j + ẽ i i = e i i + nˆv i ê k j i ê i i j + k = i ê i j i conservation Initial estimation terms satisfy e k i (0) = n v k (0) i N State dynamics preserve e k i (t) =n v k (t) i N [Na Li and JRM, Designing games for distributed optimization, 2011]

50 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions x =(v, e) a =(ˆv, ê) local value update ṽ i = v i +ˆv i local estimate update ẽ k i = e k i ê k i j + ẽ i i = e i i + nˆv i ê k j i ê i i j + k = i ê i j i conservation Initial estimation terms satisfy e k i (0) = n v k (0) i N State dynamics preserve e k i (t) =n v k (t) i N [Na Li and JRM, Designing games for distributed optimization, 2011]

51 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

52 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions J i (x, a) =J G i (x, a)+j e i (x, a) J G i (x, a) = G J e i (x, a) = k N ẽ1 j,...,ẽ n j ẽk i ẽ k j ] 2 State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Designing games for distributed optimization, 2011]

53 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions Theorem: The game is a state based potential game with potential function φ(x, a) = G ẽ 1 i,...,ẽ n ẽk i + i ẽ k 2 j i N where x =(ṽ, ẽ) =P (x, a) i N k N [Na Li and JRM, Designing games for distributed optimization, 2011]

54 Given: System level objective Desired interaction graph min G(v) v V N i N Goal: Design local cost functions J i : Recurrent state equilibrium (convex, continuously differentiable) (undirected, connected) X j A j R such that Optimal solutions Theorem: An action state pair [x,a] is a recurrent state equilibrium if and only if Value profile v =(v 1,...,v n ) is optimal Estimation is accurate e k i = v k Change in value profile satisfies Change in estimation profile satisfies ˆv i =0 êk i j ê k j i =0 [Na Li and JRM, Designing games for distributed optimization, 2011]

55 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

56 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions flavor of design use penalty functions to relax constraints min v V G(v)+µ α(v) α(v) = k M max 0, i N µ>0 tradeoff parameter A k i v i C k State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

57 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions x =(x 1,...,x n ) v i e k i local state variables x i =(v i,e 1 i,...,e n i,c 1 i,...,c m i ) actual value agent i agent i s estimate of agent k s value c i C Av agent i s estimate of constraints State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

58 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

59 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions J i (x, a) =J G i (x, a)+j e i (x, a)+µ J c i (x, a) J c i (x, a) = i N same as before max 0, c k 2 i k M State based game {1,...,n} X A i P : X A X J i : X A R [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

60 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions Theorem: The game is a state based potential game with potential function max 0, c k i 2 X φ(x, a) =φ prev (x, a)+µ i N where (ṽ, ẽ, c) =P (x, a) k M A i [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

61 Given: System level objective Desired interaction graph m linear coupled constraint Goal: Design local cost functions Recurrent state equilibrium min G(v) v V N i N Av C (convex, continuously differentiable) (undirected, connected) J i : X j A j R such that Optimal solutions Theorem: An action state pair [x,a] is a recurrent state equilibrium if and only if Value profile Estimation is accurate v =(v 1,...,v n ) Change in value profile satisfies Change in estimation profile satisfies optimizes ˆv i =0 G(v)+µ α(v) e k i = v k, c i = max(0, Av C) êk i j ê k j i =0 [Na Li and JRM, Decoupling coupled constraints through utility design, 2011]

62 Example: Illustration Global objective: Reach consensus on average of initial values using admissible controllers Setup: Player set: Information set: Value set: Initial values: N N i N V i v i (0) System level objective v i v j i N min v V s.t. i N v i = i N v i (0) Admissible controllers v i (t) =Π i Info about j at time t

63 Global objective: Example: Illustration Reach consensus on average of initial values using admissible controllers Approach: Model as state based game Use algorithm gradient play cost value Φ (l) φ(x, Φ a) (b) Φ (x) value optimization constraint estimation total time step, t max 0, ẽ k i 2 φ(x, a) =φ prev (x, a)+µ i N k M

64 Conclusion Setup Model interactions as game decision makers / players possible choices local objective functions State based potential? games localizing utility functions localizing coupled constraints improving efficiency? improving scalability? exploit decomposition for MAS Dynamics Local agent decision rules informational dependencies processing requirements Finite memory better response log-linear learning gradient play improving convergence rates?