Rate Control in Communication Networks
|
|
- Timothy Townsend
- 5 years ago
- Views:
Transcription
1 From Models to Algorithms Department of Computer Science & Engineering The Chinese University of Hong Kong February 29, 2008
2 Outline Preliminaries 1 Preliminaries Convex Optimization TCP Congestion Control 2 Motivation Basic Model Optimization Decomposition 3 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence 4 5
3 Convex Optimization Convex Optimization TCP Congestion Control Convex Set Set C is a convex set if the line segment between any two points in C lies in C, i.e., if for any x 1, x 2 C and any θ [0, 1], we have θx 1 + (1 θ)x 2 C Convex Hull Convex hull of C is the set of all convex combinations of points in C: { k θ i x i x i C,θ i 0, i = 1, 2,, k, i=1 k θ i = 1}. i=1
4 Convex Optimization Convex Optimization TCP Congestion Control Convex Function : Jensen s inequality f : R n R is a convex function if domf is a convex set and for all x, y dom f and t [0, 1], we have f(tx + (1 t)y) tf(x) + (1 t)f(y) f is strictly convex if above strict inequality holds for all x y and 0 < t < 1.
5 Convex Optimization Convex Optimization TCP Congestion Control Standard Form A convex optimization problem with variables x: minimize subject to f 0 (x) f i (x) 0, i = 1, 2,, m h i (x) = 0, i = 1, 2,, p. where f 0, f 1,, f m are convex functions; h i (x) are linear. Objective Function: Minimize convex objective function (or maximize concave objective function). Inequality Constraints: Upper bound inequality constraints on convex functions. Equality Constraints: Equality constraints must be affine.
6 Convex Optimization Convex Optimization TCP Congestion Control Lagrangian Function Absorb the constraints as the penalties to the objective. Lagrangian function: L(x,λ,ν) = f 0 (x) + m λ i f i (x) + i=1 p ν i h i (x) i=1 where Lagrange multipliers (dual variables): λ 0, ν. Dual Problem Perform unconstrained maximization on L(x, λ, ν), thus obtaining Lagrangian dual function: g(λ,ν) = inf x L(x,λ,ν).
7 Convex Optimization Convex Optimization TCP Congestion Control Example: Primal problem: minimize f 0 (x) = x 1 log x 1 x 2 log x 2 subject to x 1 + 2x 2 2, x 1, x 2 > 0; Lag Function: L(x,λ) = x 1 log x 1 x 2 log x 2 +λ(x 1 +2x 2 2);(λ 0) Optimal x: x 1 = e λ 1 ; x 2 = e 2λ 1 ; Dual Function: maximize D(λ) = e λ 1 + e 2λ 1 2λ; over λ 0.
8 Convex Optimization Convex Optimization TCP Congestion Control Economics Interpretation Primal objective (f 0 (x)): cost of operation Primal constraints (f i (x)): can be violated Dual variables (λ, v): price for violating the corresponding constraint (dollar per unit violation). For the same price, can sell unused violation for revenue Lagrangian (L(x, λ, v)): total cost Lagrange dual problem (g(λ, v)): optimal cost as a function of violation prices (Lagrangian multipliers) Question Optimal Solution of Primal Problem = Optimal Solution of Dual Problem?
9 Convex Optimization Convex Optimization TCP Congestion Control KKT Optimal Conditions Karush-Kuhn-Tucker (KKT) conditions for a standard convex optimization problem: Primal constraints: f i (x) 0 and h i (x) = 0 Dual constraints: λ 0 Complementary slackness: λ i f i (x) = 0 Gradient of Lagrangian with respect to x vanishes: f 0 (x) + m i=1 λ i f i (x) + p i=1 v i h i (x) = 0 If strong duality holds and x, λ, v are optimal, then they must satisfy the KKT conditions.
10 TCP Congestion Control Convex Optimization TCP Congestion Control Problem Description Congestion is the conflict between demand and capacity. Congestion control is a problem of resource management. Congestion leads to buffer overflow, large delay, bandwidth underutilization. Current Solutions Rate Adaption in the source (e.g. TCP) Controllers in the buffer (e.g. AQM) Question Is current TCP merely a heuristic algorithm?
11 TCP Congestion Control Convex Optimization TCP Congestion Control TCP Versions Tahoe (Jacobson 1988) Slow Start, Congestion Avoidance, Fast Retransmit Reno (Jacobson 1990) Further Adding Fast Recovery Vegas (Brakmo & Peterson 1994) Delay(RTT)-based Congestion Avoidance Active Queue Management Random Early Detection (Floyd & Jacobson 1993) Proportional Integral (Hollot,Misra & Towsley 2001) Random Exponential Marking (Athuraliya & Low 2000)
12 TCP Congestion Control Convex Optimization TCP Congestion Control TCP Reno/RED Dynamics: window marking/dropping 1 host time router B Avg queue
13 Motivation Basic Model Optimization Decomposition Motivating Example Given a network with Two Links that have the capacities c 1 and c 2. Three end-to-end flows x 1, x 2 and x 3. Question: How to allocate bandwidth for the end-to-end flows? x 1 c 1 c 2 x 2 x 3
14 Motivation Basic Model Optimization Decomposition Potential Solutions x 1 = 0, x 2 = c 1, x 3 = c 2. x 1 = c 1 /2, x 2 = c 1 /2, x 3 = c 2 c 1 /2. Which solution is the BEST? It depends on our objective! Definition In economics, utility is a measure of the relative happiness or satisfaction (gratification) gained by consuming different bundles of goods and services. Wikipedia
15 Motivation Basic Model Optimization Decomposition Notation C j,(j J): the finite capacity of link j; r,(r R): a router that has non-empty link set; x s,(s S): the flow rate allocated to user s; A,(A js, j J, s S): defines a 0-1 matrix that depicts the routing; set A js = 1 if s uses the resource j. U,(U s ( ), s S): the utility of user s with the rate x s.
16 Motivation Basic Model Optimization Decomposition Assumption The utility function U s (x s ) is an increasing, strictly concave and continuously differentiable function of x s. (The traffic that leads to such a utility function is called elastic traffic by S. Shenker.) System Model SYSTEM(U,A,C): subject to over max s S U s (x s ) Ax C x 0.
17 Motivation Basic Model Optimization Decomposition Example 1 Let U s (x s ) = log x s, what is the optimal solution? Example 2 Let U s (x s ) = x 1, what is the optimal solution? x 1 1 A = Mbps 2Mbps x 2 x 3
18 Motivation Basic Model Optimization Decomposition Solving SYSTEM(U, A, C) Lagrangian form: L(x;µ) = s S U s (x s ) + µ T (C Ax) = (U s (x s ) x s µ j ) + µ j C j, s S j s j J where µ = (µ j, j J) are Lagrangian multipliers. Then, L = U x s(x s ) µ j. s j s Optimize the dual function over the feasibility region.
19 Motivation Basic Model Optimization Decomposition Solving SYSTEM(U, A, C) Using KKT conditions, we can express these conditions more compactly: (x) solves SYSTEM(U, A, C) if and only if these exists multipliers (µ) such that: Ax C, x 0; µ 0; µ T (C Ax) = 0,( j s µ j U (x)) T x = 0; The first row is primal feasibility; the second row is dual feasibility; and the third row comprises complementary slackness.
20 Motivation Basic Model Optimization Decomposition Discussion THE GOOD: Mathematically tractable due to the convexity; THE BAD: Utilities are unlikely to be known by the network; THE UGLY: You derive the flow rates in a centralized manner. Question How can we obtain a distributed algorithm to allocate rates? Road Map Solve THE BAD first, then the THE UGLY.
21 Motivation Basic Model Optimization Decomposition Definition Shadow price is the change in the objective value of the optimal solution of an optimization problem obtained by relaxing the constraint by one unit. In a business application, a shadow price is the maximum price that management is willing to pay for an extra unit of a given limited resource. Shadow price is the value of the Lagrange multiplier at the optimal solution. Wikipedia Decomposition From the perspective of economic theory, the original problem is replaced by two simpler problems for users and network.
22 Motivation Basic Model Optimization Decomposition User s Angle USER s (U s ;λ s ): over max U s ( w s λ s ) w s w s 0. where w s is the amount to pay per-unit time and λ s is regarded as a charge per unit flow for user s. Hence, the flow rate x s is exactly ws λ s. The Important Idea: User s wants to maximize profit by choosing optimal w s to pay.
23 Motivation Basic Model Optimization Decomposition Network s Angle Network(A, C; w): subject to over max s S w s log x s Ax C, x 0. The Important Idea: Network knows payments w s from all users s and chooses rate allocation to maximize the revenue.
24 Motivation Basic Model Optimization Decomposition Question Is the optimization decomposition true? Theorem 1 There always exist vectors w,λ, and x, satisfying w s = λ s x s, such that w s solves USER s (U s ;λ s ) and x solves NETWORK(A, C; w); further, the vector x is then the unique solution to SYSTEM(U, A, C). Proof The combinations of KKT conditions of the USER and NETWORK problems are identical to the SYSTEM problem.
25 Motivation Basic Model Optimization Decomposition Physical Meaning The users and the network optimize their individual benefit, and the social welfare is automatically achieved. Recap SYSTEM is decomposed into many local USER problems and one global NETWORK problem where local utility functions are not needed. Fairness? What are the relationships between optimal rate allocation and fairness?
26 Motivation Basic Model Optimization Decomposition Definition Max-Min Fairness : A vector of rates x is max-min fair if it is feasible, and if for each s, x s cannot be increased without decreasing x s for some s for which x s x s. Proportional Fairness : Feasible x is proportionally fair (per unit charge) if for any other feasible x, Theorem 2 s S x s x s x s 0. x solves NETWORK(A, c; w) if and only if it is proportionally fair (per unit charge).
27 Motivation Basic Model Optimization Decomposition Physical Meaning The network cannot achieve a better social revenue by changing the rate vector x. A tradeoff between maximum capacity and max-min throughput. Generally, their relationship can be depicted by Fair Unfair Max-min throughput Proportional Fairness Maximum Capacity Road Map THE BAD is solved, and the remaining problem is THE UGLY.
28 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Global Picture Design distributed algorithms using the gradient-based method; The congestion indications (link prices) can be generated by considering different performance goals (e.g. loss rate, delay, robustness etc.); The congestion indications (link prices) can be feedback to the source in several ways; Rate control system is globally stable without regard to time delay.
29 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence NETWORK(A, C; w) Problem with Variable x max r R w s log x s subject to Ax C, x 0. Lagrangian Function L(x; w) = r R w s log x s + µ T (C Ax) Unique Optimum : x s = w s j r µ. j
30 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Primal Algorithm dx s (t) dt = κ(w s x s µ j (t)) µ j (t) = p j ( s:j s x s (t)) j r where κ is a small constant. Interpretation Link j charges p j (y) per unit flow, when total flow on link j is y. Each source tries to equalize the total cost with target value w s.
31 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Another Interpretation Link j generates feedback signal p j (y), when total flow on link j is y. Each source linearly increase its rate (proportional to w s ) and multiplicatively decrease its rate (proportional to total feedback). Implementation x s (t + 1) = x s (t) + κ(w s x s µ j (t)) µ j (t + 1) = p j ( s:j s x s (t + 1)) j r
32 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Dual Algorithm dµ j (t) dt x s (t) = ( = κ s:j s w s j r µ j(t). ) x s (t) q j (µ j (t)) where q j (µ j (t)) is the amount of flow on link j that would generate price µ j (t). Implementation Link algorithm: µ j (t + 1) = µ j (t)+κ( s:j s x s(t) q j (µ j (t))) Source algorithm: x s (t + 1) = Us 1 (p s (t))
33 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Primal Algorithm VS Dual Algorithm Primal algorithm: a system where rates vary gradually, and shadow prices are given as functions of the rates. Dual algorithm: a system where shadow prices vary gradually, with rates given as functions of the shadow prices. In the primal algorithm, sender adjusts rate according to the feedback of the congestion signals. In the dual algorithm, the network computes the shadow prices directly and send them back to the sender.
34 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Mapping to TCP/AQM A TCP scheme may be mapped into a specific utility function. Major TCP schemes approximately carrying out primal or dual algorithm. Congestion Measures Price Queueing delay Queue Length Packet Loss
35 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence TCP Utility Functions TCP Reno: TCP Vegas: Queue Management U reno s (x s ) = 2 tan 1( x s D s ) D s 2 Us vegas (x s ) = α s d s log x s FIFO: ṗ l = 1 c l [(y l (t) c l )] + RED: ḃ l = (y l (t) c l ) + ; ṙ l = α l c l (r l (t) b l (t)); p l = m l (r l ); REM: p l (t + 1) = [p l (t) + γ(y l (t) c l )] +
36 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence System Block Diagram
37 Question Primal Problem Dual Algorithm Relation to TCP Stability and Convergence What is the most important property for a rate controller? Stability! Basic Notions An insensitivity to small perturbations, where perturbations are modeling errors of system, environment, noise etc.
38 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Lyapunov Stability Consider an autonomous nonlinear dynamical system ẋ = f(x(t)), x(0) = x 0, where x(t) D R n denotes the system state vector, D an open set containing the origin, and f : D R n continuous on D. Without loss of generality, we may assume that the origin is an equilibrium. The origin of the above system is said to be Lyapunov stable, if, for every ǫ > 0, there exists a δ = δ(ǫ) > 0 such that, if x(0) < δ, then x(t) < ǫ, for every t 0. from Wikipedia
39 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Lyapunov second theorem on stability Consider a function V(x) : R n R such that V(x) 0 with equality if and only if x = 0 (positive definite). V(x(t)) < 0 (negative definite). Then V(x) is called a Lyapunov function candidate and the system is asymptotically stable in the sense of Lyapunov (i.s.l.). Interpretation The kinesthetic energy of an autonomous dynamic system will vanish eventually.
40 Stability of Primal Algorithms Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Establish a Lyapunov function under mild regularity conditions: U(x) = s S w s log x s j J P s:j s xs 0 p j (y)dy Stability of Dual Algorithms Establish a Lyapunov function under mild regularity conditions: V(x) = w s log( µ j ) s S j s j J µj 0 q j (η)dη Prove the global stability under the Lyapunov second theorem.
41 Primal Problem Dual Algorithm Relation to TCP Stability and Convergence Related Issues Rate of convergence Stochastic perturbation Time-delay systems Routing Other possible decomposition
42 Logical Network: Link 1 Link 2 S1 Switch Switch Switch D3 S2 S3 D1 D2 Note Note: We use an alternative dual algorithm (REM) in the simulation since F.P.Kelly s work presents a theoretic framework instead of an implementable algorithm in the real network. We evaluate the price update in REM algorithm. Each PC was equipped with 64 MB of RAM and 100-MB/s PCI ethernet cards. The packets are 500B long, containing 489B data payload. Utility function: U i (x i ) = α i log x i. γ =
43 Homogeneous Case:Each source transmitted data for a total of 120 s, with their starting times staggered by intervals of 40 s: source 1 started transmitting at time 0, source 2 at time 40 s, and source 3 at time 80 s. α 1 = α 2 = α 3 =
44 Heterogeneous Case:The setup in this experiment is the same as in Experiment 1, except that the utility function of source 3 has α 3 = , double that of sources 1 and 2.
45 Examples The assumption of above network is that C is a fixed vector, which is not true in wireless networks, e.g. the link capacity is a function of scheduler or transmission power. The original maximization is then decomposed into a flow rate control subproblem and a lower-layer subproblem maximize µ j c j subject to j J MAC or PHY layer constraints
46 Appendix Key Reference Key Reference I F.P. Kelly, A.K. Maulloo, et.al, Rate control for communication networks: shadow prices, proportional fairness and stability, Journal of the Operational Research Society, 1998 S.H. Low, D.E. Lapsley, Optimization Flow Control a I: Basic Algorithm and Convergence, IEEE/ACM Trans. Networking, M. Chiang, S.H. Low, et.al, Layering as optimization decomposition, Proceeding of IEEE, 2007.
Motivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationUtility, Fairness and Rate Allocation
Utility, Fairness and Rate Allocation Laila Daniel and Krishnan Narayanan 11th March 2013 Outline of the talk A rate allocation example Fairness criteria and their formulation as utilities Convex optimization
More informationSolving Dual Problems
Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More information9. Dual decomposition and dual algorithms
EE 546, Univ of Washington, Spring 2016 9. Dual decomposition and dual algorithms dual gradient ascent example: network rate control dual decomposition and the proximal gradient method examples with simple
More informationCharging and rate control for elastic traffic
Charging and rate control for elastic traffic Frank Kelly University of Cambridge Abstract This paper addresses the issues of charging, rate control and routing for a communication network carrying elastic
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationNetwork Optimization and Control
Foundations and Trends R in Networking Vol. 2, No. 3 (2007) 271 379 c 2008 S. Shakkottai and R. Srikant DOI: 10.1561/1300000007 Network Optimization and Control Srinivas Shakkottai 1 and R. Srikant 2 1
More informationOn the Method of Lagrange Multipliers
On the Method of Lagrange Multipliers Reza Nasiri Mahalati November 6, 2016 Most of what is in this note is taken from the Convex Optimization book by Stephen Boyd and Lieven Vandenberghe. This should
More informationUnderstanding TCP Vegas: A Duality Model
Understanding TCP Vegas: A Duality Model STEVEN H. LOW Caltech, Pasadena, California AND LARRY L. PETERSON AND LIMIN WANG Princeton University, Princeton, New Jersey Abstract. We view congestion control
More informationUnderstanding TCP Vegas: A Duality Model
Understanding TCP Vegas: A Duality Model Steven Low Departments of CS and EE, Caltech, USA slow@caltech.edu Larry Peterson Limin Wang Department of CS, Princeton University, USA {llp,lmwang}@cs.princeton.edu
More informationCongestion Control. Topics
Congestion Control Topics Congestion control what & why? Current congestion control algorithms TCP and UDP Ideal congestion control Resource allocation Distributed algorithms Relation current algorithms
More informationInternet Congestion Control: Equilibrium and Dynamics
Internet Congestion Control: Equilibrium and Dynamics A. Kevin Tang Cornell University ISS Seminar, Princeton University, February 21, 2008 Networks and Corresponding Theories Power networks (Maxwell Theory)
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationAn event-triggered distributed primal-dual algorithm for Network Utility Maximization
An event-triggered distributed primal-dual algorithm for Network Utility Maximization Pu Wan and Michael D. Lemmon Abstract Many problems associated with networked systems can be formulated as network
More information384Y Project June 5, Stability of Congestion Control Algorithms Using Control Theory with an application to XCP
384Y Project June 5, 00 Stability of Congestion Control Algorithms Using Control Theory with an application to XCP . Introduction During recent years, a lot of work has been done towards the theoretical
More informationLagrangian Duality and Convex Optimization
Lagrangian Duality and Convex Optimization David Rosenberg New York University February 11, 2015 David Rosenberg (New York University) DS-GA 1003 February 11, 2015 1 / 24 Introduction Why Convex Optimization?
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationcs/ee/ids 143 Communication Networks
cs/ee/ids 143 Communication Networks Chapter 4 Transport Text: Walrand & Parakh, 2010 Steven Low CMS, EE, Caltech Agenda Internetworking n Routing across LANs, layer2-layer3 n DHCP n NAT Transport layer
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationResource Allocation and Pricing. R. Srikant University of Illinois
Resource Allocation and Pricing R. Srikant University of Illinois References The Mathematics of Internet Congestion Control, Birkhauser, 2004. Pricing: Kelly Distributed Resource Allocation: Kelly, Mauloo
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationA Time-Scale Decomposition Approach to Adaptive Explicit Congestion Notification (ECN) Marking
University of Pennsylvania ScholarlyCommons Departmental Papers (ESE) Department of Electrical & Systems Engineering June 2002 A Time-Scale Decomposition Approach to Adaptive Explicit Congestion Notification
More informationTHE Internet is increasingly being used in the conduct of
94 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 14, NO. 1, FEBRUARY 2006 Global Stability Conditions for Rate Control With Arbitrary Communication Delays Priya Ranjan, Member, IEEE, Richard J. La, Member,
More informationSingular perturbation analysis of an additive increase multiplicative decrease control algorithm under time-varying buffering delays.
Singular perturbation analysis of an additive increase multiplicative decrease control algorithm under time-varying buffering delays. V. Guffens 1 and G. Bastin 2 Intelligent Systems and Networks Research
More informationTopics in Data Mining Fall Bruno Ribeiro
Network Utility Maximization Topics in Data Mining Fall 2015 Bruno Ribeiro 2015 Bruno Ribeiro Data Mining for Smar t Cities Need congestion control 2 Supply and Demand (A Dating Website [China]) Males
More informationUniqueness of Generalized Equilibrium for Box Constrained Problems and Applications
Uniqueness of Generalized Equilibrium for Box Constrained Problems and Applications Alp Simsek Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology Asuman E.
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationPrimal/Dual Decomposition Methods
Primal/Dual Decomposition Methods Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Subgradients
More informationAlternative Decompositions for Distributed Maximization of Network Utility: Framework and Applications
Alternative Decompositions for Distributed Maximization of Network Utility: Framework and Applications Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization
More informationMore on Lagrange multipliers
More on Lagrange multipliers CE 377K April 21, 2015 REVIEW The standard form for a nonlinear optimization problem is min x f (x) s.t. g 1 (x) 0. g l (x) 0 h 1 (x) = 0. h m (x) = 0 The objective function
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationMathematical Foundations -1- Constrained Optimization. Constrained Optimization. An intuitive approach 2. First Order Conditions (FOC) 7
Mathematical Foundations -- Constrained Optimization Constrained Optimization An intuitive approach First Order Conditions (FOC) 7 Constraint qualifications 9 Formal statement of the FOC for a maximum
More informationA Generalized FAST TCP Scheme
A Generalized FAST TCP Scheme Cao Yuan a, Liansheng Tan a,b, Lachlan L. H. Andrew c, Wei Zhang a, Moshe Zukerman d,, a Department of Computer Science, Central China Normal University, Wuhan 430079, P.R.
More informationThe Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem:
HT05: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford Convex Optimization and slides based on Arthur Gretton s Advanced Topics in Machine Learning course
More informationThis Dissertation. entitled. Event-triggered distributed algorithms for network optimization. Pu Wan
This Dissertation entitled Event-triggered distributed algorithms for network optimization typeset with nddiss2ε v3.0 (2005/07/27) on November 30, 2009 for Pu Wan This L A TEX2ε classfile conforms to the
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationNEW CONGESTION CONTROL SCHEMES OVER WIRELESS NETWORKS: STABILITY ANALYSIS. Minghua Chen Alessandro Abate Shankar Sastry
NEW CONGESTION CONTROL SCHEMES OVER WIRELESS NETWORKS: STABILITY ANALYSIS Minghua Chen Alessandro Abate Shankar Sastry Department of Electrical Engineering and Computer Science University of California
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationEfficiency and Braess Paradox under Pricing
Efficiency and Braess Paradox under Pricing Asuman Ozdaglar Joint work with Xin Huang, [EECS, MIT], Daron Acemoglu [Economics, MIT] October, 2004 Electrical Engineering and Computer Science Dept. Massachusetts
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationDistributed Approaches for Proportional and Max-Min Fairness in Random Access Ad Hoc Networks
Distributed Approaches for Proportional and Max-Min Fairness in Random Access Ad Hoc Networks Xin Wang, Koushik Kar Department of Electrical, Computer and Systems Engineering, Rensselaer Polytechnic Institute,
More informationNumerical Optimization
Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,
More informationELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications
ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February
More informationA Mechanism for Pricing Service Guarantees
A Mechanism for Pricing Service Guarantees Bruce Hajek Department of Electrical and Computer Engineering and the Coordinated Science Laboratory University of Illinois at Urbana-Champaign Sichao Yang Qualcomm
More informationLagrangian Duality. Richard Lusby. Department of Management Engineering Technical University of Denmark
Lagrangian Duality Richard Lusby Department of Management Engineering Technical University of Denmark Today s Topics (jg Lagrange Multipliers Lagrangian Relaxation Lagrangian Duality R Lusby (42111) Lagrangian
More informationRobustness of Real and Virtual Queue based Active Queue Management Schemes
Robustness of Real and Virtual Queue based Active Queue Management Schemes Ashvin Lakshmikantha, C. L. Beck and R. Srikant Department of General Engineering University of Illinois lkshmknt@uiuc.edu, rsrikant@uiuc.edu,
More informationOptimization and Stability of TCP/IP with Delay-Sensitive Utility Functions
Optimization and Stability of TCP/IP with Delay-Sensitive Utility Functions Thesis by John Pongsajapan In Partial Fulfillment of the Requirements for the Degree of Master of Science California Institute
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationUtility Maximization for Communication Networks with Multi-path Routing
Utility Maximization for Communication Networks with Multi-path Routing Xiaojun Lin and Ness B. Shroff School of Electrical and Computer Engineering Purdue University, West Lafayette, IN 47906 {linx,shroff}@ecn.purdue.edu
More informationOptimization. A first course on mathematics for economists
Optimization. A first course on mathematics for economists Xavier Martinez-Giralt Universitat Autònoma de Barcelona xavier.martinez.giralt@uab.eu II.3 Static optimization - Non-Linear programming OPT p.1/45
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationGeometric Programming for Communication Systems
Geometric Programming for Communication Systems Mung Chiang Electrical Engineering Department, Princeton University Tutorial Presentation August 2005 Outline Part I: Geometric Programming What s GP? Why
More informationStability of Primal-Dual Gradient Dynamics and Applications to Network Optimization
Stability of Primal-Dual Gradient Dynamics and Applications to Network Optimization Diego Feijer Fernando Paganini Department of Electrical Engineering, Universidad ORT Cuareim 1451, 11.100 Montevideo,
More informationThe Kuhn-Tucker Problem
Natalia Lazzati Mathematics for Economics (Part I) Note 8: Nonlinear Programming - The Kuhn-Tucker Problem Note 8 is based on de la Fuente (2000, Ch. 7) and Simon and Blume (1994, Ch. 18 and 19). The Kuhn-Tucker
More informationDuality. Geoff Gordon & Ryan Tibshirani Optimization /
Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider
More informationNetwork Utility Maximization With Nonconcave Utilities Using Sum-of-Squares Method
Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 2005 Seville, Spain, December 2-5, 2005 MoC4.6 Network Utility Maximization With Nonconcave Utilities
More informationFairness and Optimal Stochastic Control for Heterogeneous Networks
λ 91 λ 93 Fairness and Optimal Stochastic Control for Heterogeneous Networks sensor network wired network wireless 9 8 7 6 5 λ 48 λ 42 4 3 0 1 2 λ n R n U n Michael J. Neely (USC) Eytan Modiano (MIT) Chih-Ping
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More informationUtility Maximization for Communication Networks with Multi-path Routing
Utility Maximization for Communication Networks with Multi-path Routing Xiaojun Lin and Ness B. Shroff Abstract In this paper, we study utility maximization problems for communication networks where each
More informationInformation in Aloha Networks
Achieving Proportional Fairness using Local Information in Aloha Networks Koushik Kar, Saswati Sarkar, Leandros Tassiulas Abstract We address the problem of attaining proportionally fair rates using Aloha
More informationPrimal-dual Subgradient Method for Convex Problems with Functional Constraints
Primal-dual Subgradient Method for Convex Problems with Functional Constraints Yurii Nesterov, CORE/INMA (UCL) Workshop on embedded optimization EMBOPT2014 September 9, 2014 (Lucca) Yu. Nesterov Primal-dual
More informationA Globally Stable Adaptive Congestion Control Scheme for Internet-Style Networks with Delay 1
A Globally Stable Adaptive ongestion ontrol Scheme for Internet-Style Networks with Delay Tansu Alpcan 2 and Tamer Başar 2 (alpcan, tbasar)@control.csl.uiuc.edu Abstract In this paper, we develop, analyze
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More informationLagrange Relaxation and Duality
Lagrange Relaxation and Duality As we have already known, constrained optimization problems are harder to solve than unconstrained problems. By relaxation we can solve a more difficult problem by a simpler
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More informationRouting. Topics: 6.976/ESD.937 1
Routing Topics: Definition Architecture for routing data plane algorithm Current routing algorithm control plane algorithm Optimal routing algorithm known algorithms and implementation issues new solution
More informationPrice Competition with Elastic Traffic
Price Competition with Elastic Traffic Asuman Ozdaglar Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology August 7, 2006 Abstract In this paper, we present
More informationEfficient Nonlinear Optimizations of Queuing Systems
Efficient Nonlinear Optimizations of Queuing Systems Mung Chiang, Arak Sutivong, and Stephen Boyd Electrical Engineering Department, Stanford University, CA 9435 Abstract We present a systematic treatment
More informationFairness Considerations in Network Flow Problems
2015 IEEE 54th Annual Conference on Decision and Control (CDC) December 15-18, 2015. Osaka, Japan Fairness Considerations in Network Flow Problems Er Wei, Chaithanya Bandi Abstract In most of the physical
More informationMicroeconomic Algorithms for Flow Control in Virtual Circuit Networks (Subset in Infocom 1989)
Microeconomic Algorithms for Flow Control in Virtual Circuit Networks (Subset in Infocom 1989) September 13th, 1995 Donald Ferguson*,** Christos Nikolaou* Yechiam Yemini** *IBM T.J. Watson Research Center
More informationECE Optimization for wireless networks Final. minimize f o (x) s.t. Ax = b,
ECE 788 - Optimization for wireless networks Final Please provide clear and complete answers. PART I: Questions - Q.. Discuss an iterative algorithm that converges to the solution of the problem minimize
More informationOutline. Roadmap for the NPP segment: 1 Preliminaries: role of convexity. 2 Existence of a solution
Outline Roadmap for the NPP segment: 1 Preliminaries: role of convexity 2 Existence of a solution 3 Necessary conditions for a solution: inequality constraints 4 The constraint qualification 5 The Lagrangian
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationIn view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written
11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationA SIMPLE PARALLEL ALGORITHM WITH AN O(1/T ) CONVERGENCE RATE FOR GENERAL CONVEX PROGRAMS
A SIMPLE PARALLEL ALGORITHM WITH AN O(/T ) CONVERGENCE RATE FOR GENERAL CONVEX PROGRAMS HAO YU AND MICHAEL J. NEELY Abstract. This paper considers convex programs with a general (possibly non-differentiable)
More informationLagrangian Duality. Evelien van der Hurk. DTU Management Engineering
Lagrangian Duality Evelien van der Hurk DTU Management Engineering Topics Lagrange Multipliers Lagrangian Relaxation Lagrangian Duality 2 DTU Management Engineering 42111: Static and Dynamic Optimization
More informationCan Shortest-path Routing and TCP Maximize Utility
Can Shortest-path Routing and TCP Maximize Utility Jiantao Wang Lun Li Steven H. Low John C. Doyle California Institute of Technology, Pasadena, USA {jiantao@cds., lun@cds., slow@, doyle@cds.}caltech.edu
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationDual Decomposition.
1/34 Dual Decomposition http://bicmr.pku.edu.cn/~wenzw/opt-2017-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/34 1 Conjugate function 2 introduction:
More informationOptimization. Charles J. Geyer School of Statistics University of Minnesota. Stat 8054 Lecture Notes
Optimization Charles J. Geyer School of Statistics University of Minnesota Stat 8054 Lecture Notes 1 One-Dimensional Optimization Look at a graph. Grid search. 2 One-Dimensional Zero Finding Zero finding
More informationLocal and Global Stability of Symmetric Heterogeneously-Delayed Control Systems
Local and Global Stability of Symmetric Heterogeneously-Delayed Control Systems Yueping Zhang and Dmitri Loguinov Texas A&M University, College Station, TX 77843 Email: {yueping, dmitri}@cs.tamu.edu Abstract
More informationThe Multi-Path Utility Maximization Problem
The Multi-Path Utility Maximization Problem Xiaojun Lin and Ness B. Shroff School of Electrical and Computer Engineering Purdue University, West Lafayette, IN 47906 {linx,shroff}@ecn.purdue.edu Abstract
More information