Decentralized Control of Stochastic Systems
|
|
- Eugene Jasper Berry
- 5 years ago
- Views:
Transcription
1 Decentralized Control of Stochastic Systems Sanjay Lall Stanford University CDC-ECC Workshop, December 11, 2005
2 2 S. Lall, Stanford Decentralized Control G 1 G 2 G 3 G 4 G 5 y 1 u 1 y 2 u 2 y 3 u 3 y 4 u 4 y 5 u 5 K 1 K 2 K 3 K 4 K 5 Control design problem is to find K which is block diagonal, whose diagonal blocks are 5 separate controllers. u 1 u 2 u 3 u 4 = u 5 K y 1 0 K y K y K 3 0 y K 3 y 5
3 3 S. Lall, Stanford Example: Communicating Controllers Often we do not want perfectly decentralized control; we may achieve better performance by allowing controllers to communicate, both to receive measurements from other aircraft, and to exchange data with other controllers. G 1 G 2 G 3 D D D D K 1 K 2 K 3 Here, we need to design 3 separate controllers.
4 4 S. Lall, Stanford Research directions Spacetime models Infinite and finite spatial extent Synthesis procedures via SDP Decentralization structure via information passing Structured synthesis Decentralized structure specified in advance Quadratic invariance conditions on solvability See talk by Mike Rotkowitz Dynamic programming Stochastic finite state systems Relaxation approach
5 5 S. Lall, Stanford Finite-State Stochastic Systems In this talk Look at dynamics specified by Markov Decision Processes. Not linear, but finite state New semidefinite programming relaxations Randy Cogill, Mike Rotkowitz, Ben Van Roy Many applications: Decision problems: e.g., task allocation problems for vehicles Detection problems: e.g., physical sensing, communication
6 6 S. Lall, Stanford Example: Medium-Access Control Two transmitters, each with a queue that can hold up to 3 packets p a k = probability that k 1 packets arrive at queue a p 1 = [ ] p 2 = [ ] At each time step, each transmitter sees how many packets are in its queue, and sends some of them; then new packets arrive Packets are lost when queues overflow, or when there is a collision, i.e., both transmit at the same time
7 7 S. Lall, Stanford Example: Medium-Access Control We would like a control policy for each queue, i.e., a function mapping number of packets in the queue number of packets sent One possible policy; transmit all packets in the queue. Causes large packet loss due to collisions. The other extreme; wait until the queue is full Causes large packet loss due to overflow. We d like to find the policy that minimizes the expected number of packets lost per period.
8 8 S. Lall, Stanford Centralized Control Each transmitter can see how many packets are in the other queue In this case, we look for a single policy, mapping pair of queue occupancies pair of transmission lengths Decentralized Control Each transmitter can only see the number of packets in its own queue In this case, we look for two policies, each mapping queue occupancy transmission length
9 9 S. Lall, Stanford Medium Access: Centralized Solution The optimal policy is Transmitter 2 state Transmitter 1 state (0,0) (0,1) (0,2) (0,3) 1 (1,0) (1,0) (0,2) (0,3) 2 (2,0) (2,0) (2,0) (0,3) 3 (3,0) (3,0) (3,0) (3,0) Easy to compute via dynamic programming Each transmitter needs to know the state of the other We would like a decentralized control law
10 10 S. Lall, Stanford Markov Decision Processes The above medium-access control problem is an example of a Markov Decision Process (MDP) n states, and m actions, hence m n possible centralized policies However, the centralized problem is solvable by linear programming Decentralized control NP-hard, even with just two policies, even non-dynamic case Would like efficient algorithms Hence focus on suboptimal solutions
11 11 S. Lall, Stanford Static Problems Called classification or hypothesis testing e.g., a radar system sends out n pulses, and measures y reflections The cost depends on the number of false positives/negatives prob(y X 1 ) prob(y X 2 ) p(y X 1 ) = prob. of receiving y reflections given no aircraft present p(y X 2 ) = prob. of receiving y reflections given an aircraft present
12 12 S. Lall, Stanford Static Centralized Formulation x is the state, with prior pdf Prob(x) = p(x) y = h(x, w) is the measurement w is sensor noise Choosing control action u results in cost C(u, x) Represent sensing by transition matrix A(y, x) = Prob(y x) Problem: find γ : Y U to minimize the expected cost: p(x)a(y, x)c ( γ(y), x ) x,y
13 13 S. Lall, Stanford Static Decentralized Formulation x is the state, with prior pdf Prob(x) = p(x) y i = h i (x, w i ) is the i th measurement w i is noise at i th sensor Choosing control actions u 1,..., u m results in cost C(u 1,..., u m, x) y = (y 1,..., y m ) given by transitions A(y, x) = Prob(y x) Problem: find γ 1,..., γ m to minimize the expected cost: p(x)a(y, x)c ( γ 1 (y 1 ),..., γ m (y m ), x ) x,y
14 14 S. Lall, Stanford Change of variables Represent the policy by a matrix { 1 if γ(y) = u K yu = 0 otherwise This matrix K satisfies K1 = 1, that is K is a stochastic matrix. K is Boolean Then the expected cost is p x A yx C ( γ(y), x ) = p x A yx C ux K yu x,y,u x,y
15 15 S. Lall, Stanford Optimization Let W yu = x C uxa yx p x, the cost of decision u when y is measured. minimize W yu K yu y,u subject to K1 = 1 K yu {0, 1} for all y, u nm variables K ij, with both linear and Boolean constraints Just m easy problems; pick γ(y) = arg min u W yu
16 16 S. Lall, Stanford Linear Programming Relaxation Another solution: relax the Boolean constraints: For estimation: interpret K yu as belief in the estimate u For control: we have a mixed policy K yu = Prob(u y) Relaxation gives the linear program: minimize W yu K yu y,u subject to K1 = 1 K 0 It is easy to show that this gives the optimal cost and rounding gives the optimal controller.
17 17 S. Lall, Stanford Decentralization Constraints Decentralization imposes constraints on K(y 1,..., y m, u 1,..., u m ). Measurement constraint: controller i can only measure measurement i Prob(u i y) = Prob(u i y i ) Independence constraint: for mixed policies, controllers i and j use independent sources of randomness to generate u i and u j Prob(u y) = m Prob(u i y i ) i=1
18 18 S. Lall, Stanford Decentralization Constraints We call a controller K decentralized if K(y 1,..., y m, u 1,..., u m ) = K 1 (y 1, u 1 )... K m (y m, u m ) for some K 1,..., K m If K is a deterministic controller y = γ(u), this is equivalent to γ(u 1, u 2 ) = ( γ 1 (u 1 ), γ 2 (u 2 ) )
19 19 S. Lall, Stanford The Decentralized Static Control Problem minimize y,u W y1 y 2 u 1 u 2 K y1 y 2 u 1 u 2 subject to K y1 y 2 u 1 u 2 = K 1 y 1 u 1 K 2 y 2 u 2 K i 0 K i 1 = 1 K i ab {0, 1} for all a, b This problem is NP-hard In addition to linear constraints, we have bilinear constraints
20 20 S. Lall, Stanford Boolean Constraints One can show the following facts Removing the Boolean constraints does not change the optimal value i.e., we can simplify the constraint set Then there exists an optimal solution which is Boolean i.e., using a mixed policy does not achieve better performance than a deterministic policy
21 21 S. Lall, Stanford Relaxing the Constraints Now we would like to solve minimize y,u W y1 y 2 u 1 u 2 K y1 y 2 u 1 u 2 subject to K y1 y 2 u 1 u 2 = K 1 y 1 u 1 K 2 y 2 u 2 K i 0 K i 1 = 1 Still NP-hard, because it would solve the original problem. Removing the bilinear constraint gives the centralized problem. This is a trivial lower bound on the achievable cost
22 22 S. Lall, Stanford Valid Constraints For any decentralized controller, we know K y1 y 2 u 1 u 2 = Ky 2 2 u 2 u 1 The interpretation is that, given measurements y 1 and y 2, averaging over the decisions of controller 1 does not affect the distribution of the decisions of controller 2 Easy to see, since K 1 is a stochastic matrix, and K y1 y 2 u 1 u 2 = K 1 y 1 u 1 K 2 y 2 u 2
23 23 S. Lall, Stanford Valid Constraints Adding these new constraints leaves the optimization unchanged. But it changes the relaxation. Dropping the bilinear constraint gives a tighter lower bound than the centralized problem. minimize y,u W y1 y 2 u 1 u 2 K y1 y 2 u 1 u 2 subject to K y1 y 2 u 1 u 2 = K 1 y 1 u 1 K 2 y 2 u 2 u 1 K y1 y 2 u 1 u 2 = K 2 y 2 u 2 u 2 K y1 y 2 u 1 u 2 = K 1 y 1 u 1 new valid constraints K i 0, K i 1 = 1
24 24 S. Lall, Stanford Synthesis Procedure 1. Solve the linear program minimize subject to W y1 y 2 u 1 u 2 K y1 y 2 u 1 u 2 y,u u 1 K y1 y 2 u 1 u 2 = K 2 y 2 u 2 K y1 y 2 u 1 u 2 = Ky 1 1 u 1 u 2 K i 0, K i 1 = 1 The optimal value gives a lower bound on the achievable cost 2. If K y1 y 2 u 1 u 2 = Ky 1 1 u 1 Ky 2 2 u 2 then the solution is optimal. If not, use K 1 and K 2, as approximate solutions.
25 25 S. Lall, Stanford Lifting Lifting is a general approach for constructing primal relaxations; the idea is Introduce new variables Y which are polynomial in x This embeds the problem in a higher dimensional space Write valid constraints in the new variables The feasible set of the original problem is the projection of the lifted feasible set
26 26 S. Lall, Stanford Example: Minimizing a Polynomial We d like to find the minimum of f = 6 k=0 a kx k Pick new variables Y = g(x) where 1 x x 2 x 3 g(x) = x x 2 x 3 x 4 x 2 x 3 x 4 x 5 and C = x 3 x 4 x 5 x 6 Then an equivalent problem is a a 1 a 2 a a a 5 2 a 6 minimize trace CY subject to Y 0 Y 11 = 1 Y 24 = Y 33 Y 22 = Y 13 Y 14 = Y 23 Y = g(x) Dropping the constraint Y = g(x) gives an SDP relaxation of the problem
27 27 S. Lall, Stanford The Dual SDP Relaxation The SDP relaxation has a dual, which is also an SDP. Example Suppose f = x 6 + 4x 2 + 1, then the SDP dual relaxation is maximize subject to t 1 t λ 2 λ 3 0 2λ 2 λ 3 λ λ 2 λ 3 2λ 1 0 λ 3 λ this is exactly the condition that f t be sum of squares
28 28 S. Lall, Stanford Lifting for General Polynomial Programs When minimizing a polynomial, lifting gives an SDP relaxation of whose dual is an SOS condition When solving a general polynomial program with multiple constraints, there is a similar lifting This gives an SDP, whose feasible set is a relaxation of the feasible set of the original problem The corresponding dual SDP is a Positivstellensatz refutation Solving the dual certifies a lower bound on the original problem
29 29 S. Lall, Stanford Example: The Cut Polytope The feasible set of the MAXCUT problem is C = { X S n X = vv T, v { 1, 1} n } A simple relaxation gives the outer approximation to its convex hull. Here n = 11; the set has affine dimension 55; a projection is shown below
30 30 S. Lall, Stanford Example Suppose the sample space is Ω = {f 1, f 2, f 3, f 4 } {g 1, g 2, g 3, g 4 } The unnormalized probabilities of (f, g) Ω are given by g 1 g 2 g 3 g 4 f f f f Player 1 measures f, i.e., Y 1 is the set of horizontal strips and would like to estimate g, i.e, X 1 is the set of vertical strips Player 2 measures g and would like to estimate f
31 31 S. Lall, Stanford Example Objective: maximize the expected number of correct estimates g 1 g 2 g 3 g 4 f f f f Optimal decision rules are K 1 = Y 1 f 1 f 2 f 3 f Xest 1 g 2 g 4 g 1 g K 2 = Y 2 g 1 g 2 g 3 g 4 X 2 est f 3 f 1 f 2 f 2 The optimal is These are simply the maximum a-posteriori probability classifiers
32 32 S. Lall, Stanford Example Objective: maximize the probability that both estimates are correct Optimal decision rules are g 1 g 2 g 3 g 4 f f f f Y 1 f 1 f 2 f 3 f 4 X 1 est g 2 g 4 g 1 g 3 Y 2 g 1 g 2 g 3 g 4 X 2 est f 3 f 1 f 4 f 2 The relaxation of the lifted problem is tight The optimal probability that both estimates are correct is MAP estimates are not optimal; they achieve 0.5
33 33 S. Lall, Stanford Example Objective: maximize the probability that at least one estimate is correct g 1 g 2 g 3 g 4 f f f f The relaxation of the lifted problem is not tight; it gives upper bound of The following decision rules (constructed by projection) achieve Y 1 f 1 f 2 f 3 f 4 X 1 est g 2 g 4 g 1 g 1 Y 2 g 1 g 2 g 3 g 4 X 2 est f 1 f 3 f 1 f 4 MAP estimates achieve
34 34 S. Lall, Stanford Heuristic 1: MAP f 1 1 f g 1 g 2 g 3 g f f The optimal probability that both estimates are correct is 4 7, achieved by Player 1 sees: f 1 f 2 f 3 f 4 Player 1 guesses: g 1 g 2 g 3 g 4 Player 2 sees: g 1 g 2 g 3 g 4 Player 2 guesses: f 1 f 2 f 3 f 4 If both just guess most likely outcome, achieved probability is 1 4.
35 35 S. Lall, Stanford Heuristic 2: Person by Person Optimality 1. Choose arbitrary initial policies γ 1 and γ Hold γ 2 fixed. Choose best γ 1 given γ Hold new γ 1 fixed. Choose best γ 2 given this γ Continue until no more improvement. u 1 = 0 u 1 = 1 u 2 = u 2 = PBPO value is 1000 times the optimal! Almost as bad as the worst policy.
36 36 S. Lall, Stanford Lift and Project Performance Guarantee We would like to maximize reward; all rewards are nonnegative. Use the above LP relaxation together with a projection procedure The computed policies always achieve a a reward of at least optimal reward min{ U 1, U 2 } This gives an upper bound, a feasible policy, and a performance guarantee
37 37 S. Lall, Stanford Markov Decision Processes We will now consider a Markov Decision Process where X i (t) is the event that the system is in state i at time t A j (t) is the event that action j is taken at time t We assume for simplicity that for every stationary policy the chain is irreducible and aperiodic Transition probabilities: A ijk = Prob(X i (t + 1) X j (t) A k (t)) Mixed policy: Cost function: K jk = Prob(X j (t) A k (t)) W jk = cost of action k in state j
38 38 S. Lall, Stanford Markov Decision Processes The centralized solution minimize subject to W jk K jk j,k K ir = r j,k K 0 F jk = 1 j,k A ijk K jk
39 39 S. Lall, Stanford Decentralized Markov Decision Processes Two sets of states X p = {X p 1,..., Xp n} Two transition matrices A p ijk = Prob(Xp i (t + 1) Xp j (t) Ap k (t)) Two controllers K p jk = Prob(Xp j (t) Ap k (t)) Cost function W j1 j 2 k 1 k 2 = cost of actions k 1, k 2 in states j 1, j 2
40 40 S. Lall, Stanford Decentralized Markov Decision Processes minimize W j1 j 1 k 1 k 2 K j1 j 2 k 1 k 2 j 1,j 1,k 1,k 2 subject to K j1 j 2 k 1 k 2 = Kj 1 1 k 1 Kj 2 2 k 2 K p ir = A p ijk Kp jk (1) r j,k K p 0 (2) K p jk = 1 (3) j,k Each of constraints (1) (3) can be multiplied by K 3 p to construct a valid constraint in lifted variables K The resulting linear program gives a lower bound on the optimal cost
41 41 S. Lall, Stanford Exact Solution If the solution K to the lifted linear program has the form K j1 j 2 k 1 k 2 = K 1 j 1 k 1 K 2 j 2 k 2 then the controller is an optimal decentralized controller. This corresponds to the usual rank conditions in e.g., MAXCUT. Projection If not, we need to project the solution K defines a pdf on X 1 X 2 U 1 U 2 We project by constructing the marginal pdf on X p U p
42 42 S. Lall, Stanford Example: Medium-Access Control Two transmitters, each with a queue that can hold up to 3 packets p a k = probability that k 1 packets arrive at queue a p 1 = [ ] p 2 = [ ] At each time step, each transmitter sees how many packets are in its queue, and sends some of them; then new packets arrive Packets are lost when queues overflow, or when there is a collision, i.e., both transmit at the same time
43 43 S. Lall, Stanford Example: Medium Access This is a Decentralized Markov Decision Process, where Each MDP has 4 states; the no. of packets in the queue Each MDP has 4 actions; transmit 0, 1, 2, 3 packets State transitions are determined by arrival probabilities and actions Cost is total number of packets lost; Each queue loses all packets sent if there is a collision Each queue loses packets due to overflows
44 44 S. Lall, Stanford Example: Medium Access Optimal policies for each player are queue occupancy number sent queue occupancy number sent Expected number of packets lost per period is The policy always transmit loses per period queue 2 successful collisions overflows
45 45 S. Lall, Stanford Summary Decentralized control problems are inherently computationally hard We have developed Efficient algorithms to construct suboptimal solutions Upper bounds on achievable performance Guaranteed performance ratio Much more topology independent controllers algorithms that exploit structure peer-to-peer techniques other applications: robotics, sensor networks
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationSection Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018
Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections
More information9. Interpretations, Lifting, SOS and Moments
9-1 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC 2003 2003.12.07.04 9. Interpretations, Lifting, SOS and Moments Polynomial nonnegativity Sum of squares (SOS) decomposition Eample
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationParameterization of Stabilizing Controllers for Interconnected Systems
Parameterization of Stabilizing Controllers for Interconnected Systems Michael Rotkowitz G D p D p D p D 1 G 2 G 3 G 4 p G 5 K 1 K 2 K 3 K 4 Contributors: Sanjay Lall, Randy Cogill K 5 Department of Signals,
More informationMathematical Optimization Models and Applications
Mathematical Optimization Models and Applications Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 1, 2.1-2,
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationAnalysis and synthesis: a complexity perspective
Analysis and synthesis: a complexity perspective Pablo A. Parrilo ETH ZürichZ control.ee.ethz.ch/~parrilo Outline System analysis/design Formal and informal methods SOS/SDP techniques and applications
More informationConvex relaxation. In example below, we have N = 6, and the cut we are considering
Convex relaxation The art and science of convex relaxation revolves around taking a non-convex problem that you want to solve, and replacing it with a convex problem which you can actually solve the solution
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationConvex relaxation. In example below, we have N = 6, and the cut we are considering
Convex relaxation The art and science of convex relaxation revolves around taking a non-convex problem that you want to solve, and replacing it with a convex problem which you can actually solve the solution
More informationMIT Algebraic techniques and semidefinite optimization February 14, Lecture 3
MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications
More informationDistributed Optimization. Song Chong EE, KAIST
Distributed Optimization Song Chong EE, KAIST songchong@kaist.edu Dynamic Programming for Path Planning A path-planning problem consists of a weighted directed graph with a set of n nodes N, directed links
More informationOptimization in Wireless Communication
Zhi-Quan (Tom) Luo Department of Electrical and Computer Engineering University of Minnesota 200 Union Street SE Minneapolis, MN 55455 2007 NSF Workshop Challenges Optimization problems from wireless applications
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationMachine Learning 4771
Machine Learning 477 Instructor: Tony Jebara Topic 5 Generalization Guarantees VC-Dimension Nearest Neighbor Classification (infinite VC dimension) Structural Risk Minimization Support Vector Machines
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationLECTURE 3. Last time:
LECTURE 3 Last time: Mutual Information. Convexity and concavity Jensen s inequality Information Inequality Data processing theorem Fano s Inequality Lecture outline Stochastic processes, Entropy rate
More information8 Approximation Algorithms and Max-Cut
8 Approximation Algorithms and Max-Cut 8. The Max-Cut problem Unless the widely believed P N P conjecture is false, there is no polynomial algorithm that can solve all instances of an NP-hard problem.
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationSDP Relaxations for MAXCUT
SDP Relaxations for MAXCUT from Random Hyperplanes to Sum-of-Squares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP
More informationSelected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.
. Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,
More informationHidden Markov Models (HMM) and Support Vector Machine (SVM)
Hidden Markov Models (HMM) and Support Vector Machine (SVM) Professor Joongheon Kim School of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea 1 Hidden Markov Models (HMM)
More informationAgenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization
Agenda Applications of semidefinite programming 1 Control and system theory 2 Combinatorial and nonconvex optimization 3 Spectral estimation & super-resolution Control and system theory SDP in wide use
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a two-class classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationConvex sets, conic matrix factorizations and conic rank lower bounds
Convex sets, conic matrix factorizations and conic rank lower bounds Pablo A. Parrilo Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute
More informationOPTIMALITY OF RANDOMIZED TRUNK RESERVATION FOR A PROBLEM WITH MULTIPLE CONSTRAINTS
OPTIMALITY OF RANDOMIZED TRUNK RESERVATION FOR A PROBLEM WITH MULTIPLE CONSTRAINTS Xiaofei Fan-Orzechowski Department of Applied Mathematics and Statistics State University of New York at Stony Brook Stony
More informationDimension reduction for semidefinite programming
1 / 22 Dimension reduction for semidefinite programming Pablo A. Parrilo Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute of Technology
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationSupport Vector Machine
Support Vector Machine Kernel: Kernel is defined as a function returning the inner product between the images of the two arguments k(x 1, x 2 ) = ϕ(x 1 ), ϕ(x 2 ) k(x 1, x 2 ) = k(x 2, x 1 ) modularity-
More information15-780: LinearProgramming
15-780: LinearProgramming J. Zico Kolter February 1-3, 2016 1 Outline Introduction Some linear algebra review Linear programming Simplex algorithm Duality and dual simplex 2 Outline Introduction Some linear
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationComputational Game Theory Spring Semester, 2005/6. Lecturer: Yishay Mansour Scribe: Ilan Cohen, Natan Rubin, Ophir Bleiberg*
Computational Game Theory Spring Semester, 2005/6 Lecture 5: 2-Player Zero Sum Games Lecturer: Yishay Mansour Scribe: Ilan Cohen, Natan Rubin, Ophir Bleiberg* 1 5.1 2-Player Zero Sum Games In this lecture
More informationPositive semidefinite rank
1/15 Positive semidefinite rank Hamza Fawzi (MIT, LIDS) Joint work with João Gouveia (Coimbra), Pablo Parrilo (MIT), Richard Robinson (Microsoft), James Saunderson (Monash), Rekha Thomas (UW) DIMACS Workshop
More informationLinear and conic programming relaxations: Graph structure and message-passing
Linear and conic programming relaxations: Graph structure and message-passing Martin Wainwright UC Berkeley Departments of EECS and Statistics Banff Workshop Partially supported by grants from: National
More informationFinal Exam December 12, 2017
Introduction to Artificial Intelligence CSE 473, Autumn 2017 Dieter Fox Final Exam December 12, 2017 Directions This exam has 7 problems with 111 points shown in the table below, and you have 110 minutes
More informationMethodology for Computer Science Research Lecture 4: Mathematical Modeling
Methodology for Computer Science Research Andrey Lukyanenko Department of Computer Science and Engineering Aalto University, School of Science and Technology andrey.lukyanenko@tkk.fi Definitions and Goals
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationFinal Exam December 12, 2017
Introduction to Artificial Intelligence CSE 473, Autumn 2017 Dieter Fox Final Exam December 12, 2017 Directions This exam has 7 problems with 111 points shown in the table below, and you have 110 minutes
More informationMarkov Processes Hamid R. Rabiee
Markov Processes Hamid R. Rabiee Overview Markov Property Markov Chains Definition Stationary Property Paths in Markov Chains Classification of States Steady States in MCs. 2 Markov Property A discrete
More informationMARKOV DECISION PROCESSES (MDP) AND REINFORCEMENT LEARNING (RL) Versione originale delle slide fornita dal Prof. Francesco Lo Presti
1 MARKOV DECISION PROCESSES (MDP) AND REINFORCEMENT LEARNING (RL) Versione originale delle slide fornita dal Prof. Francesco Lo Presti Historical background 2 Original motivation: animal learning Early
More informationWeek 5: Markov chains Random access in communication networks Solutions
Week 5: Markov chains Random access in communication networks Solutions A Markov chain model. The model described in the homework defines the following probabilities: P [a terminal receives a packet in
More informationOPPORTUNISTIC Spectrum Access (OSA) is emerging
Optimal and Low-complexity Algorithms for Dynamic Spectrum Access in Centralized Cognitive Radio Networks with Fading Channels Mario Bkassiny, Sudharman K. Jayaweera, Yang Li Dept. of Electrical and Computer
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.85J / 8.5J Advanced Algorithms Fall 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 8.5/6.85 Advanced Algorithms
More information16.1 L.P. Duality Applied to the Minimax Theorem
CS787: Advanced Algorithms Scribe: David Malec and Xiaoyong Chai Lecturer: Shuchi Chawla Topic: Minimax Theorem and Semi-Definite Programming Date: October 22 2007 In this lecture, we first conclude our
More informationRobust and Optimal Control, Spring 2015
Robust and Optimal Control, Spring 2015 Instructor: Prof. Masayuki Fujita (S5-303B) G. Sum of Squares (SOS) G.1 SOS Program: SOS/PSD and SDP G.2 Duality, valid ineqalities and Cone G.3 Feasibility/Optimization
More informationSupport Vector Machines
Wien, June, 2010 Paul Hofmarcher, Stefan Theussl, WU Wien Hofmarcher/Theussl SVM 1/21 Linear Separable Separating Hyperplanes Non-Linear Separable Soft-Margin Hyperplanes Hofmarcher/Theussl SVM 2/21 (SVM)
More informationPower Allocation over Two Identical Gilbert-Elliott Channels
Power Allocation over Two Identical Gilbert-Elliott Channels Junhua Tang School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University, China Email: junhuatang@sjtu.edu.cn Parisa
More informationDenis ARZELIER arzelier
COURSE ON LMI OPTIMIZATION WITH APPLICATIONS IN CONTROL PART II.2 LMIs IN SYSTEMS CONTROL STATE-SPACE METHODS PERFORMANCE ANALYSIS and SYNTHESIS Denis ARZELIER www.laas.fr/ arzelier arzelier@laas.fr 15
More informationCSC Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method Notes taken by Stefan Mathe April 28, 2007 Summary: Throughout the course, we have seen the importance
More informationLecture notes for Analysis of Algorithms : Markov decision processes
Lecture notes for Analysis of Algorithms : Markov decision processes Lecturer: Thomas Dueholm Hansen June 6, 013 Abstract We give an introduction to infinite-horizon Markov decision processes (MDPs) with
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationMachine Learning. Support Vector Machines. Manfred Huber
Machine Learning Support Vector Machines Manfred Huber 2015 1 Support Vector Machines Both logistic regression and linear discriminant analysis learn a linear discriminant function to separate the data
More informationRecap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks
Recap Probability, stochastic processes, Markov chains ELEC-C7210 Modeling and analysis of communication networks 1 Recap: Probability theory important distributions Discrete distributions Geometric distribution
More informationNetwork Utility Maximization With Nonconcave Utilities Using Sum-of-Squares Method
Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 2005 Seville, Spain, December 2-5, 2005 MoC4.6 Network Utility Maximization With Nonconcave Utilities
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationMarkov Chain Model for ALOHA protocol
Markov Chain Model for ALOHA protocol Laila Daniel and Krishnan Narayanan April 22, 2012 Outline of the talk A Markov chain (MC) model for Slotted ALOHA Basic properties of Discrete-time Markov Chain Stability
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Formal models of interaction Daniel Hennes 27.11.2017 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Taxonomy of domains Models of
More informationMarkov decision processes and interval Markov chains: exploiting the connection
Markov decision processes and interval Markov chains: exploiting the connection Mingmei Teo Supervisors: Prof. Nigel Bean, Dr Joshua Ross University of Adelaide July 10, 2013 Intervals and interval arithmetic
More informationThe quest for finding Hamiltonian cycles
The quest for finding Hamiltonian cycles Giang Nguyen School of Mathematical Sciences University of Adelaide Travelling Salesman Problem Given a list of cities and distances between cities, what is the
More informationA Polynomial-time Nash Equilibrium Algorithm for Repeated Games
A Polynomial-time Nash Equilibrium Algorithm for Repeated Games Michael L. Littman mlittman@cs.rutgers.edu Rutgers University Peter Stone pstone@cs.utexas.edu The University of Texas at Austin Main Result
More informationMIT Algebraic techniques and semidefinite optimization May 9, Lecture 21. Lecturer: Pablo A. Parrilo Scribe:???
MIT 6.972 Algebraic techniques and semidefinite optimization May 9, 2006 Lecture 2 Lecturer: Pablo A. Parrilo Scribe:??? In this lecture we study techniques to exploit the symmetry that can be present
More information16.4 Multiattribute Utility Functions
285 Normalized utilities The scale of utilities reaches from the best possible prize u to the worst possible catastrophe u Normalized utilities use a scale with u = 0 and u = 1 Utilities of intermediate
More informationProbabilistic Model Checking and Strategy Synthesis for Robot Navigation
Probabilistic Model Checking and Strategy Synthesis for Robot Navigation Dave Parker University of Birmingham (joint work with Bruno Lacerda, Nick Hawes) AIMS CDT, Oxford, May 2015 Overview Probabilistic
More informationAlgorithmic Game Theory and Applications. Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games
Algorithmic Game Theory and Applications Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games Kousha Etessami warning 1 The subjects we will touch on today are so interesting and
More information1 Strict local optimality in unconstrained optimization
ORF 53 Lecture 14 Spring 016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, April 14, 016 When in doubt on the accuracy of these notes, please cross check with the instructor s
More informationPolynomial-Time Verification of PCTL Properties of MDPs with Convex Uncertainties and its Application to Cyber-Physical Systems
Polynomial-Time Verification of PCTL Properties of MDPs with Convex Uncertainties and its Application to Cyber-Physical Systems Alberto Puggelli DREAM Seminar - November 26, 2013 Collaborators and PIs:
More informationOn the Approximate Linear Programming Approach for Network Revenue Management Problems
On the Approximate Linear Programming Approach for Network Revenue Management Problems Chaoxu Tong School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853,
More informationChapter 16 focused on decision making in the face of uncertainty about one future
9 C H A P T E R Markov Chains Chapter 6 focused on decision making in the face of uncertainty about one future event (learning the true state of nature). However, some decisions need to take into account
More informationapproximation algorithms I
SUM-OF-SQUARES method and approximation algorithms I David Steurer Cornell Cargese Workshop, 201 meta-task encoded as low-degree polynomial in R x example: f(x) = i,j n w ij x i x j 2 given: functions
More informationMulti-channel Opportunistic Access: A Case of Restless Bandits with Multiple Plays
Multi-channel Opportunistic Access: A Case of Restless Bandits with Multiple Plays Sahand Haji Ali Ahmad, Mingyan Liu Abstract This paper considers the following stochastic control problem that arises
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationNear-Optimal Control of Queueing Systems via Approximate One-Step Policy Improvement
Near-Optimal Control of Queueing Systems via Approximate One-Step Policy Improvement Jefferson Huang March 21, 2018 Reinforcement Learning for Processing Networks Seminar Cornell University Performance
More informationc 2000 Society for Industrial and Applied Mathematics
SIAM J. OPIM. Vol. 10, No. 3, pp. 750 778 c 2000 Society for Industrial and Applied Mathematics CONES OF MARICES AND SUCCESSIVE CONVEX RELAXAIONS OF NONCONVEX SES MASAKAZU KOJIMA AND LEVEN UNÇEL Abstract.
More informationBBM402-Lecture 20: LP Duality
BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More informationPrimal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization
Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization Robert M. Freund M.I.T. June, 2010 from papers in SIOPT, Mathematics
More informationNonconvex Quadratic Programming: Return of the Boolean Quadric Polytope
Nonconvex Quadratic Programming: Return of the Boolean Quadric Polytope Kurt M. Anstreicher Dept. of Management Sciences University of Iowa Seminar, Chinese University of Hong Kong, October 2009 We consider
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More informationGenerating Valid 4 4 Correlation Matrices
Generating Valid 4 4 Correlation Matrices Mark Budden, Paul Hadavas, Lorrie Hoffman, Chris Pretz Date submitted: 10 March 2006 Abstract In this article, we provide an algorithm for generating valid 4 4
More informationApproximation Algorithms
Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA
More informationPart 7: Structured Prediction and Energy Minimization (2/2)
Part 7: Structured Prediction and Energy Minimization (2/2) Colorado Springs, 25th June 2011 G: Worst-case Complexity Hard problem Generality Optimality Worst-case complexity Integrality Determinism G:
More informationOn the Throughput, Capacity and Stability Regions of Random Multiple Access over Standard Multi-Packet Reception Channels
On the Throughput, Capacity and Stability Regions of Random Multiple Access over Standard Multi-Packet Reception Channels Jie Luo, Anthony Ephremides ECE Dept. Univ. of Maryland College Park, MD 20742
More information1 Markov decision processes
2.997 Decision-Making in Large-Scale Systems February 4 MI, Spring 2004 Handout #1 Lecture Note 1 1 Markov decision processes In this class we will study discrete-time stochastic systems. We can describe
More informationL5 Support Vector Classification
L5 Support Vector Classification Support Vector Machine Problem definition Geometrical picture Optimization problem Optimization Problem Hard margin Convexity Dual problem Soft margin problem Alexander
More informationChapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)
Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis
More informationLow-ranksemidefiniteprogrammingforthe. MAX2SAT problem. Bosch Center for Artificial Intelligence
Low-ranksemidefiniteprogrammingforthe MAX2SAT problem Po-Wei Wang J. Zico Kolter Machine Learning Department Carnegie Mellon University School of Computer Science, Carnegie Mellon University, and Bosch
More informationSubmodularity in Machine Learning
Saifuddin Syed MLRG Summer 2016 1 / 39 What are submodular functions Outline 1 What are submodular functions Motivation Submodularity and Concavity Examples 2 Properties of submodular functions Submodularity
More informationCapacity of the Discrete Memoryless Energy Harvesting Channel with Side Information
204 IEEE International Symposium on Information Theory Capacity of the Discrete Memoryless Energy Harvesting Channel with Side Information Omur Ozel, Kaya Tutuncuoglu 2, Sennur Ulukus, and Aylin Yener
More informationA POMDP Framework for Cognitive MAC Based on Primary Feedback Exploitation
A POMDP Framework for Cognitive MAC Based on Primary Feedback Exploitation Karim G. Seddik and Amr A. El-Sherif 2 Electronics and Communications Engineering Department, American University in Cairo, New
More informationInformation in Aloha Networks
Achieving Proportional Fairness using Local Information in Aloha Networks Koushik Kar, Saswati Sarkar, Leandros Tassiulas Abstract We address the problem of attaining proportionally fair rates using Aloha
More informationThe Graph Realization Problem
The Graph Realization Problem via Semi-Definite Programming A. Y. Alfakih alfakih@uwindsor.ca Mathematics and Statistics University of Windsor The Graph Realization Problem p.1/21 The Graph Realization
More informationMarkovian Decision Process (MDP): theory and applications to wireless networks
Markovian Decision Process (MDP): theory and applications to wireless networks Philippe Ciblat Joint work with I. Fawaz, N. Ksairi, C. Le Martret, M. Sarkiss Outline Examples MDP Applications A few examples
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationGame Theory. Greg Plaxton Theory in Programming Practice, Spring 2004 Department of Computer Science University of Texas at Austin
Game Theory Greg Plaxton Theory in Programming Practice, Spring 2004 Department of Computer Science University of Texas at Austin Bimatrix Games We are given two real m n matrices A = (a ij ), B = (b ij
More informationOptimality of Walrand-Varaiya Type Policies and. Approximation Results for Zero-Delay Coding of. Markov Sources. Richard G. Wood
Optimality of Walrand-Varaiya Type Policies and Approximation Results for Zero-Delay Coding of Markov Sources by Richard G. Wood A thesis submitted to the Department of Mathematics & Statistics in conformity
More informationThe Markov Decision Process (MDP) model
Decision Making in Robots and Autonomous Agents The Markov Decision Process (MDP) model Subramanian Ramamoorthy School of Informatics 25 January, 2013 In the MAB Model We were in a single casino and the
More information