Stochastic resource allocation Boyd, Akuiyibo

Transcription

1 Stochastic resource allocation Boyd, Akuiyibo ACHIEVEMENT DESCRIPTION STATUS QUO Current resource allocation research focus on iterative methods. These automatically adapt to changing data assuming they are held constant. MAIN RESULT: Explicit optimal control laws for resource allocation in a system with quadratic cost, linear dynamics, and random linear constraints. averaging input algorithm Target value, x =6 IMPACT Stochastic allocation of competing network resources i.e., bandwidth, power, flow rates, etc. Simple control laws (linear coefficients can be computed ahead of time). greedy algorithm trajectory NEW INSIGHTS Formulate as stochastic control problem Resource limits are random Allocate resources based on availability and system state optimal trajectory ASSUMPTIONS AND LIMITATIONS: Assumes that the first and second moments of the resources are know n Utility is quadratic; dynamics must be linear NEXT-PHASE GOALS Utility maximizing estimation techniques Decentralized solutions Optimal dynamic resource allocation with heterogeneous flows

2 Multi-Period Stochastic Control of Networks Stephen Boyd and Ekine Akuiyibo ITMANET PI meeting 03/5 6/09

3 The problem wireless networks are characterized by extreme variation in availability of resources, e.g., bandwidth, power, link capacities, connectivity, etc. These issues have been dealt with in an ad hoc way in the past (iterative methods) the goal: multi-period resource (e.g., flow rate, power) allocation resources (e.g., link capacities, channel states) vary randomly ITMANET PI meeting 03/5 6/09 1

4 Our approach allocate resources (random resource limits) based on availability and system state we form a (nonstandard) stochastic control problem to handle resource allocation and dynamic utilites maximize utility (or minimize cost) that reflects different weights (priorities) desired/required target levels averaging time scales for different flows unifies what we called before WNUM, DNUM, SNUM ITMANET PI meeting 03/5 6/09 2

5 Dynamic utility and averaging time scale f t is flow in period t, t = 0,1,... U : R R gives utility derived for a flow value f t = (1 θ) t τ=0 θτ f t τ is (first order) smoothed or averaged flow T avg = 1/ log(1/θ) gives smoothing time scale average smoothed utility is Ū = lim t (1/t)E t τ=0 U( f τ ) when U is not linear, Ū depends on smoothing time scale T avg our claim: flow utility should be judged dynamically, i.e., by a utility function and an averaging time scale ITMANET PI meeting 03/5 6/09 3

6 Stochastic flow control must have f t R t (rate region) in general, R t is random process on sets; we ll assume R t are IID stochastic flow control: choose f t as function (policy) of R t, x t, to maximize Ū can solve in principle via DP can solve exactly in only a few special cases lots of approximate solution methods ITMANET PI meeting 03/5 6/09 4

7 Linear quadratic formulation: (nonstandard) stochastic control problem maximize Ū subject to x τ+1 = A τ x τ + B τ f τ, 1 T f τ = c τ, τ = 0, 1,...,T 1 with variables f 0,...,f T 1, x 1,x 2,...,x T where Ū = E T 1 τ=0 U(x τ), U(x τ ) are concave quadratic functions x τ R n are utility states f τ R m are flows c τ R p are random capacities rate region is defined by (random) capacity c τ : R τ = {z 1 T z = c τ } problem data are A τ R n n, B τ R n m ; first and second moments of c τ ITMANET PI meeting 03/5 6/09 5

8 Solution via dynamic programming let V t (z) be expected utility to go, in state x t = z, at time t, before c t is revealed V t is quadratic, defined by a (nonstandard) backward recursion expectation is over c t optimal policies are affine in x t, c t : f t = Kx t + wc t + s K, w, s found by backward recursion 1 T K = 0, 1 T w = 1, 1 T s = 0, so we have 1 T f t = c t ITMANET PI meeting 03/5 6/09 6

9 2 flows share one link (Simplest possible) example U(a) = (a 1) 2 (i.e., target flow values are one) smoothing times T avg = 1, T avg = 30 link capacity c t is exponential with mean 1.5 optimal policy: [ ] [ f t = x t [ 0.5 we ll compare with simple sharing: f t = 0.5 ] c t + ] c t [ ] ITMANET PI meeting 03/5 6/09 7

10 Sample trajectories smoothed flow (xt) (xt) t ITMANET PI meeting 03/5 6/09 8

11 Sample trajectories flow 4 3 (ut) (ut) t ITMANET PI meeting 03/5 6/09 9

12 What s next handling inequality constraints (e.g., f t 0, Rf t c t ) via control Lyapunov methods extension to multi-flow, multi-hop, store-and-forward (easy) approximating general concave utilities as quadratic decentralization ITMANET PI meeting 03/5 6/09 10