Online Convex Optimization with Ramp Constraints

Transcription

1 015 IEEE 54th Aual Coferece o Decisio ad Cotrol CDC) December 15-18, 015. Osaka, Japa Olie Covex Optimizatio with Ramp Costraits Masoud Badiei 1, Na Li 1, Adam Wierma 1 Harvard Uiversity, {mbadieik, ali}@seas.harvard.edu Califoria Istitute of Techology, adamw@caltech.edu Abstract We study a ovel variatio of olie covex optimizatio where the algorithm is subject to ramp costraits limitig the distace betwee cosecutive actios. Our cotributio is results providig asymptotically tight bouds o the worstcase performace, as measured by the competitive differece, of a variat of Model Predictive Cotrol termed Averagig Fixed Horizo Cotrol A). Additioally, we prove that A achieves the asymptotically optimal achievable competitive differece withi a geeral class of forward lookig olie algorithms. Furthermore, we illustrate that the performace of A i practice is ofte much better tha idicated by the worst-case) competitive differece usig a case study i the cotext of the ecoomic dispatch problem. I. INTRODUCTION I a olie covex optimizatio OCO) problem, a olie learer iteracts with a eviromet i a sequece of rouds. Durig each roud t: i) the learer must choose a actio x t = x 1 t, x t,, x N t ) from a covex set X ; ii) the eviromet reveals a cost loss fuctio c t ) C ; ad iii) the learer experieces cost c t x t ). The goal is to desig learig algorithms that miimize the cost experieced over a log) horizo of T rouds. Olie covex optimizatio has emerged as a fudametal tool i applicatios ragig from commuicatio etworks [1], [] to cloud computig [3], [4] to portfolio optimizatio [5], [6]. As such, there is a large literature spaig machie learig, olie algorithms, ad operatios research that seeks to develop ad aalyze algorithms for this settig [7], [8], [9]. Further, motivated by practical cosideratios, a large umber of variatios of olie covex optimizatio have emerged, e.g., the additio of switchig costs [3], the cosideratio of predictios [10], badit formulatios [11]. I this paper, we cosider a ovel variatio of olie covex optimizatio where the learer algorithm) is subject to costraits boudig the distace betwee cosecutive actios, i.e., is subject to ramp costraits. I particular, we cosider a settig where the learer s actios durig each roud are additioally costraied by X i x i t x i t 1 X i, for give ramp costraits X i ad X i. Ramp costraits are commo i a variety of practical settigs. For example, i the ecoomic dispatch problem, which plas the geeratio trajectories of power suppliers for the electricity grid, the rampig costraits of geerators are of crucial importace [1], [13]. These limits are due to the thermal ad mechaical iertia that has to be overtake i order to ramp the output up or dow, e.g., uclear plats have strict The complete versio of this paper ca be foud olie at rampig costraits that limit their use i providig respose to fast timescale demad fluctuatios. Aother example i the eergy domai is eergy storage maagemet, where the output power of a battery is subject to a hard costrait. Outside of eergy, examples where ramp costraits are bidig ca be foud i, e.g., portfolio optimizatio ad etwork routig. The additio of ramp costraits to the classical olie covex optimizatio formulatio adds cosiderable difficulty. I particular, without ramp costraits the curret actio ad cost ca be decoupled from future cost fuctios; however whe ramp costraits are preset it is ecessary to esure that the curret actio does ot costrai future actios i a maer that leads to high costs. Thus, oe ca expect that it is difficult to attai olie algorithms with ear-optimal performace without allowig the algorithm some access to predictios of future cost fuctios. Luckily, i may applicatios e.g., ecoomic dispatch ad storage maagemet) the learer has predictios/forecasts available. Cosequetly, i this paper, we cosider a situatio where the learer has access to a limited umber of future cost fuctios. Specifically, we allow the learer perfect lookahead for W time steps, i.e., whe decidig x t the learer has access to c t,..., c t+w 1. This model is obviously overly optimistic about the quality of predictios available to the algorithm, but it is a commo choice for studyig the value of predictios i the olie algorithms commuity, e.g., [14], [15], [3] so it is a atural startig poit for the ivestigatio of olie covex optimizatio with ramp costraits. Note that it is typically straightforward to exted competitive aalysis of olie algorithms with perfect lookahead to the case of lookahead with bouded error. Cotributios of this paper: This paper iitiates the study of olie covex optimizatio with ramp costraits. Specifically, we focus our study o the aalysis of a promisig algorithm, amed Averagig Fixed Horizo Cotrol A), ad derive asymptotically) tight bouds o the competitive differece, i.e., the additive worst-case gap betwee the cost of obtaied uder A ad that obtaied by the offlie, static optimal solutio. Additioally, we itroduce a geeral class of forward lookig algorithms that iclude Model Predictive Cotrol MPC) ad its variats, amog other algorithms. We show that A provides the asymptotically optimal competitive differece achievable by forward lookig algorithms ad, i fact, has competitive differece withi a factor of 4 of ay forward lookig algorithm. Fially, we illustrate that A ofte performs much better /15/$ IEEE 6730

2 tha the worst-case) bouds o competitive differece idicate. To do this, we perform a case study usig the ecoomic dispatch problem. Related literature: Olie covex optimizatio has bee studied extesively, see [7] for a recet survey. While the additio of ramp costraits is ovel, may variatios that cosider the additio of costraits of various forms to the optimizatios have bee studied previously. For example, [16] cosiders a olie learig problem wherei the adversary reveals a costrait ad a cost fuctio at each step; [17] cosiders a variatio i which the adversary is costraied i the actios that it ca play at each roud; ad [18] illustrates that allowig the costrait violatios results i a sigificat improvemet i the achievable performace. Perhaps the most closely related prior work is the literature studyig olie covex optimizatio with switchig costs, which are ofte called smoothed olie covex optimizatio problems [3], [19]. I such problems, there are o costraits, but there is a extra cost of x t x t 1 icurred each roud. Thus, like i the case of ramp costraits, whe switchig costs are preset, the curret actio must cosider future cost fuctios i order to avoid payig a avoidable switchig cost i the ext time step. Clearly, the problems of olie covex optimizatio with switchig costs ad with ramp costraits are related. But, ote that the case of ramp costraits requires ew aalytic tools ad leads to structurally differet results compared to the case of switchig costs. However, the literature studyig switchig costs provides us motivatio for cosiderig A i the curret paper. See the discussio i Sectio IV. II. PROBLEM FORMULATION I this paper we cosider a variatio of olie covex optimizatio where the algorithm is subject to ramp costraits that limit the maximal deviatio i actios from oe time step to the ext. Clearly, oe caot hope for a olie algorithm to perform well i this cotext if othig is kow about future cost fuctios, sice the curret choice places costraits o feasible choices i the ext stage. Thus, we cosider a situatio where the algorithm has iformatio about both the curret cost fuctio ad a limited umber of future cost fuctios. Specifically, the algorithm has perfect lookahead for W steps. Formally, we cosider olie covex optimizatio over a fiite time horizo t [T ]. At each time step t [T ], a sequece of cost fuctios c t, c t+1,, c t+w 1 ) from a set C fiite or ifiite) is revealed to the olie algorithm. This W -lookahead widow is perfect i the sese that these are the true cost fuctios the algorithm will experiece i future time steps. We assume that each cost fuctio c t ) : R N R i C is differetiable ad covex. Furthermore, we suppose that the subgradiet of the cost fuctio is bouded o the whole actio space X. I particular, we assume that there exists L t R 0 for all t [T ] such that c t x t ) L t, for all x t X, 1) where c t x t ) = c t x t )/ x i t) N is the gradiet of c t ). These assumptios are commo i the literature, e.g., see [7]. We ca prove stroger results whe the cost fuctios further satisfy the strog covexity criterio, i.e., for all the vectors x t, y t X it satisfies c t y t ) c t x t ) c t x t ) y t x t ) T + σ t y t x t, ) for some σ t > 0. I the followig, we ie C C as the set of all strogly covex cost fuctios, i.e., for all c t ) C, the coditio i ) holds for some σ t > 0. At each time t, the olie-decisio maker takes a actio x t = x 1 t,..., x N t ) R N from the actio set X R N after observig the cost fuctios c t,..., c t+w 1 ), i.e., x t X, 3) where X is assumed to be covex. I the case that the actio space X is bouded, we let r > 0 to be the radius of the largest l 1 -ball that cotais it, i.e., X B 1 r) = {x R N : x 1 r}. We impose ramp costraits o cosecutive actios. These take the followig form X i x i t x i t 1 X i, 4) where, the iitial coditio for 5) is give by x t = x 0 X for t < 1. Moreover, whe t > T, we assume x t is costat ad equal to the value of x T. The objective is to fid a sequece of olie actios x t ) T that yields the miimum cost subject to 3), 4). I particular, we ie the olie cost as T C olie = c t x t ), 5) where x t ) T are the actios that are take by a olie decisio maker ad satisfy 4). Further, we ie c t ) 0 for t < 1, t > T. A. Performace measures I the absece of the ramp rate costrait imposed o the actios, a greedy algorithm that optimizes the curret cost fuctio at each time step is optimal, i the sese that it achieves the optimal offlie cost. However, i the presece of ramp costraits, takig a actio that has immediate payoff ca limit the actio set for future decisios, which ca result i poor performace. The focus of this paper is to uderstad the achievable performace of olie algorithms relative to the cost of the optimal offlie solutio. To that ed, we ie the followig optimal offlie cost as our performace bechmark. C offlie = mi x 1,,x T T c t x t ), 6) s.t. : 3), 4). Throughout, x t ) T specifies the optimal offlie solutio. As is stadard i the olie algorithms commuity, we use the competitive ratio i order to quatify the iefficiecy of a olie algorithm relative to the optimal offlie solutio. 6731

3 Defiitio 1. A olie algorithm is α T -competitive, if there exists a α T R 0 such that C olie sup α T. 7) c 1,,c T C C offlie We achieve bouds o the competitive ratio through the derivatio of bouds o the competitive differece. Defiitio. A olie algorithm has a competitive differece of β T R 0 if sup C olie C offlie ) β T. 8) c 1,,c T C Though the competitive differece is ot as commoly used as the competitive ratio, it is more coveiet for our cotext ad it is straightforward to traslate oe ito the other. Further, the competitive differece is more comparable to regret, which is aother commo measure used to evaluate algorithms for olie covex optimizatio, e.g., see [7]. Thus, we state our results i terms of competitive differece throughout this paper ad leave it to the reader to iterpret the results i terms of the competitive ratio. Note that, typically, the offlie optimal cost is ΩT ), e.g., whe there exists a e 0 such that c t 0) > e 0 for all t as is imposed i [3]. III. AN ONLINE ALGORITHM A wide variety of algorithms have bee proposed for olie covex optimizatio problems. The key differetiators of the cotext we cosider i this paper are i) ramp costraits ad ii) lookahead. Give these features of the model, perhaps the most atural cadidates for cosideratio are algorithms from the cotrol commuity such as Model Predictive Cotrol MPC) ad its variatios. MPC, i its stadard form, has bee show to perform poorly i the worst-case i the related problem of smoothed olie covex optimizatio, which does ot iclude ramp costraits but, istead, adds a regularizer to the objective. Specifically, MPC has a competitive ratio that does ot improve as the lookahead widow grows i high-dimesioal smoothed olie covex optimizatio problems [3]. However, a variatio of MPC termed Averagig Fixed Horizo Cotrol A) has much stroger worst-case guaratees. I particular, the competitive ratio of A with lookahead W is 1 + O1/W ). 1 Thus, A is a promisig algorithm with which to begi the exploratio of olie covex optimizatio with ramp costraits. A. Defiig Averagig Fixed Horizo Cotrol A) As the ame implies, A averages the choices made by Fixed Horizo Cotrol ) algorithms. I particular, the decisio x t at the time slot [t, t + W 1] is made by averagig differet solutios of the algorithms correspodig to differet iitializatios, i.e. t 0 W = [ W, 1]. 1 Note that this result assumes that the actio set is bouded, i.e., r <, ad that there exists e 0 > 0 s.t. c t0) e 0 for all t. The results we prove here do ot make these assumptios. Specifically, i a algorithm, the actio i the th roud is characterized as the solutio of the followig optimizatio t+w 1 t ) ˆx t0 = arg mi x t,,x t+w 1 c t x t ) 9) s.t. : 3), 4), where the superscript i ˆx t0 t idicates the choice of parameter t 0. The iitial coditio for solvig 9) is give by ˆx t0 t, 1 which is determied recursively by solvig the optimizatio problem for [t 1, t 1 + W 1]. By takig the average of ˆx t0 t over differet values of t 0 W, we obtai the A solutio. More precisely, ˆx t = 1 ˆx t0 t, 10) W t 0 W for all t [T ]. Let C A deote the overall cost of usig the A solutio, C A = T c t ˆx t ). B. Aalyzig Averagig Fixed Horizo Cotrol A) I this sectio, we seek to uderstad how the performace of A compares to that of the optimal offlie solutio. To that ed, we prove upper ad lower bouds o the competitive differece of A. 1) Upper bouds o the competitive differece: Our first boud is stated i terms of the KKT multipliers associated with the costraits 3) ad 4) i 9), which are deoted by λ t0 t = λ t0 t, λ t0 t ), µ t0 t = t, t ), respectively. Theorem 1. The competitive differece of A is upper bouded by T { C A C offlie γ mi r, T/W + 1 W W X + 1 }, X 11) where γ = max t0 W max{ t, t } T. Furthermore, whe c t ) C, the competitive differece is upper bouded by C A C offlie 1) { NT γ T T/W T/W + 1) γ mi, r, W W σ mi W X + 1 }, X where σ mi = mi{σ 1,, σ T }. A few remarks o Theorem 1 are i order. First, if we assume that the optimal offlie solutio has cost ΩT ) ad we cosider a bouded actio space, i.e., r <, the the geeral upper boud i 11) yields a boud o the competitive differece that scales as OT/W ) ad o the competitive ratio that scales as O1/W ). Iterestigly, this is the same scalig that A achieves for the competitive ratio uder these assumptios i the related problem of smoothed olie covex optimizatio [3], [19]. Furthermore, if strog covexity of the cost fuctios is assumed, the upper boud improves to O1/W ). I cotrast, if the actio space is 673

4 ubouded, r = ad the the boud o the competitive differece scales as OT /W ). Secod, ote that whe the ramp costraits are looseed, i.e., Xi, X i, i [N], γ 0. Therefore, C A = C offlie. This highlights the tightess of the boud sice, without the couplig betwee the actios provided by ramp costraits, the optimizatio problem i 6) is separable, ad the decisio vector x t ca be computed idepedetly at each time t [1, T ] by optimizig c t x t ) with respect to the costrait 3). Third, it is ot immediately obvious whether the KKT multipliers of A are bouded. I fact, without a upper boud o γ the competitive differece bouds i 11) ad 1) ca potetially be very loose. I the ext theorem, we demostrate that γ is ideed bouded, ad the upper boud depeds o the widow size W as well as the parameter L t i 1). Theorem. For all t [t, t + W 1], the KKT multipliers of the optimizatio problem i 9) with the set of costraits i 4) are bouded as follows λ t0 t, λ t0 t L t I, t, t L τ I. τ=t The cosequece of the above theorem is that γ = max t 0 W max t 1,,t M max { τ=t ) Lower bouds o the competitive differece: We ow tur out attetio from upper bouds to lower bouds ad address the questio of whether the upper bouds we have obtaied are tight. Theorem 3. Suppose the actio space is ubouded, i.e., X = R N. The, the competitive differece of A has a lower boud as γ T/W T/W + 1) L τ }. W X sup c 1,,c T C C A C offlie ). Moreover, i the special case of W = 1, the lower boud takes the followig form γ T/ T/ + 1) mi{ X 1, X 1 } sup C A C offlie ). c 1,,c T C Note that the lower boud assumes that the actio space is ubouded, i.e., r =. Thus, compared to the upper boud i 11), the lower boud is tight to withi a costat factor of 1/. This highlights that the asymptotic scalig of the upper boud is correct. Iterestigly, to obtai this theorem we costruct a sequece of liear cost fuctios that exploits the structure of A to eforce a large cost o the algorithm. Thus, the cost fuctios i the costructio of the lower boud are ot strogly covex. IV. A GENERAL LOWER BOUND Our goal i this sectio is to uderstad whether there are olie algorithms that ca sigificatly outperform A. With this motivatio, we seek a geeral lower boud o the competitive differece of ay olie algorithm. While we caot achieve this goal i full, what we ca achieve is a lower boud o the competitive differece of a broad class of forward lookig algorithms that icludes, amog others, MPC ad its may variats. A. A class of forward lookig algorithms I geeral, at each time t, a algorithm has iformatio about the etire set of reveled cost fuctios, i.e., c 1, c,, c t+w 1 ). However, the class of forward lookig algorithms we cosider is limited to usig oly a subset of these cost fuctios i its decisio makig. I particular, at each time t, the algorithm may oly use cost fuctios t+w 1 τ=t M {c τ }. Thus, forward lookig algorithms make use of oly a limited umber M of past cost fuctios. The iformatioal costrait above is the key piece of the iitio of forward lookig algorithms. However, for techical reasos, we eed to additioally impose a mior costrait o how the algorithms may use the cost fuctios. Let x t to deote the actios a forward lookig algorithm. Let a be ay time slot such that t M a t, b be ay time slot such that t b t + W 1, x a 1 be ay actio i X, t+w 1 τ=t M c a, c a+1,..., c b be a series of cost fuctios from {c τ } for all τ [a, b]. The cosider the followig optimizatio problem, mi x τ,τ=a,...,b s.t. 3), 4). c t x τ ) 13) If the optimal solutios for all a, b, x a 1, c have the same t, the the decisio maker will take actio x t = x t 1 + t. This costrait esures that, o matter how the algorithm t+w 1 uses the cost fuctio {c τ } τ=t M, i.e., either by permutig or throwig away cost fuctio, if the optimal actios have the same rampig decisio, i.e., t = x t x t 1, the the decisio maker will take x t = x t 1 + t. Note that this coditio does ot costrai the class of olie algorithms dramatically sice it is very ulikely to be bidig. I particular, it oly applies if all the optimal solutios for 13) have the same t. B. A lower boud o the competitive differece of forward lookig algorithms Though the class of forward lookig algorithms is broad, e.g., icludig MPC ad its variats, we ca prove a boud o the competitive differece for all such algorithms that asymptotically matches the upper boud i Theorem 1 o the competitive differece of A. Theorem 4. For ay forward lookig olie algorithm ied o X = R N, the competitive differece is lower bouded by: MT ρ M + W ) + T T ) M + W ) + 1 X sup c 1,...,c T C C olie C offlie ), 14) 6733

5 where ρ max t [T ] L t. Importatly, this geeral lower boud is withi 1/4 of the upper boud i Theorem 1, which highlights that o forward lookig algorithm ca sigificatly outperform A. Moreover, i most situatios the boud is eve tighter. I particular, ρ = max t [T ] L t while γ is the maximum sum of L t over a widow of size W, which whe divided by the extra factor of 1/W i 11), results i the maximum average of L t. Cosequetly, γ is potetially much smaller tha ρ whe L t has a large variatio with respect to t ad/or W is very large. V. A CASE STUDY: ECONOMIC DISPATCH I order to groud the theoretical work i this paper, we cosider a case study with which to illustrate the performace of A i practice. I particular, we use the ecoomic dispatch problem as a illustrative example. The objective of the ecoomic dispatch problem is to schedule the committed geeratig uits to supply the load demad with miimum cost while satisfyig the operatioal costraits, see [0], [1]. The challege i derives from the fact that geerators have hard ramp costraits which limit the chage i geeratio feasible betwee epochs. Note that the focus of this paper is ot o the ecoomic dispatch problem, rather it is o the study of geeral olie covex optimizatio with ramp costraits. Thus, the goal of this case study is i) to emphasize the importace of ramp costraits i real-world example ad ii) to illustrate that A ofte performs much better tha the worst-case boud o the competitive differece suggest. Cosequetly, we cosider a relatively simple variatio of the ecoomic dispatch problem ad igore may issues that oe would eed to cosider before actually applyig the ideas here i practice. A. Experimetal Setup We cosider a power etwork that has a wid power geerator as a supplemet to three hydro-thermal power geerators N = 3). We deote the power outputs of the hydro-thermal uits by x i t, i {1,, 3}. I lie with the other works o the ecoomic dispatch problem [1], [], [3] we choose a polyomial cost fuctio with time idepedet coefficiets for the power geeratio uits, i.e., c t x t ) = N a i x i t) + b i x i t + c i). Table I shows the bouds o each power statio. The iitial values x i 0, i {1,, 3} are chose i a way that the load demad at t = 1 ca be satisfied. We use d t to deote the aggregated load demad at time t [T ] ad the et amout of power geerated by wid power statios by r t. The objective is to supply sufficiet power to meet the demad at all the times, i.e., N x i t + r t = d t, 15) where x i t, i [N] are bouded variables as i 3) ad satisfy the ramp costraits as i 4). Sice r t ad d t are both radom processes, we ca combie their et effect ito a uified process, called the residual demad process ˆd t = max{0, d t r t }. To satisfy 15), we thus cosider the followig optimizatio problem mi x 1,,x T s.t. : 3), 4), T N a i x i t) + b i x i t + c i) T N + η t ˆd t where η t R 0 is the regularizer factor. For simplicity let η t = η for all t [T ]. B. Numerical Results For our umerical results, we apply A to the problem specified i the previous sectio. Our experimetal results cosider the case of symmetric ramp rate limits, i.e., the case whe icreasig or decreasig the power output is subject to the same ramp rate limits X i = X i = X i. For this sceario, we ie the ratio of the ramp costrait to the rage of each variable by R i = X i / x i + x i ). I the case of R 1 = R = R 3 = R, Fig. 1c) shows the ratio of C A /C offlie as a fuctio of R for all 0 R 0.1. For all values of R that are larger tha 0.1, the ratio is uity. To plot the figure, we assumed that A has a predictio widow size of W = 1 hour. It is evidet from Fig. 1c) that for striget ramp rate limits, the performace of A deteriorates. This ca be explaied by the fact that whe the ramp rate limit is strict, it becomes more difficult for ay olie algorithm to adjust its trajectory after takig a suboptimal actio. We additioally ote that i the special case of R = 0, both olie ad offlie solutios are costat ad equal to their iitial value. As a result C A /C offlie = 1 whe R = 0. Lastly, Fig. 1d) shows the trajectories of each algorithm to satisfy the residual demad ˆd t for the worst case value of the ramp costraits i.e., R = 0.009). The similarity of the two trajectories is immediately evidet. VI. CONCLUSION I this paper we have itroduced a variatio of olie covex optimizatio where the actios are costraied by limits o the ramp rate ad where the olie algorithm has a W - step perfect) lookahead widow. This problem is motivated by, amog other applicatios, the ecoomic dispatch problem, for which ramp costraits are a fudametal challege. Our mai results focus o a relatively ew algorithm, Averagig Fixed Horizo Cotrol A). We prove upper ad lower bouds o the competitive differece of A that are tight up to a factor of. Additioally, we prove a geeral lower boud o forward lookig algorithms which shows that o such algorithm ca have a competitive differece more tha a factor of 4 better tha A, ad that i most cases this factor is eve smaller. Further, we have illustrated the performace of A i the cotext of the ecoomic dispatch problem, which highlights that the typical performace of A is much better tha idicated by the worst-case) competitive differece. Our results are a first step toward the desig of algorithms for olie covex optimizatio problems that are subject to x i t ) 6734

6 Table I: The hypothetical cost ad operatioal capacities of the hydro-thermal uits. Dispatcher x i MW) x i MW) a i $ MW ) b i $ MW 1 ) c i $) Gas: x 1 t Hydro: x t Nuclear: x 3 t Load Demad MW). x Solar Power MW) Ratio C A /C Offlie ) Power Dispatch MW) x 104 offlie A Days Days R Days a) b) c) d) Figure 1: a) The load demad profile of Otario, Caada durig March 1st to March 5th [4] b): The solar power geeratio profile of a solar power plat c): The ratio C A /C offlie for differet values of the ramp costrait, i.e., differet values of R d): The trajectories of the offlie ad A solutios. ramp costraits, ad there are may ope questios that remai. For example, we have adopted the model of perfect lookahead here due to its popularity i the olie algorithms commuity but, of course, this model misses may importat features of predictios. Cosiderig a more realistic predictio model, such as [10], is a importat ext step. Additioally, while A has a competitive differece that is asymptotically tight amog forward lookig algorithms, there may be other algorithms that make use of historical cost fuctios to improve o the competitive differece ad there may be forward lookig algorithms that ca improve o A by a factor of up to) four. VII. ACKNOWLEDGMENT Adam Wierma s research is supported by NSF CNS , NSF NETS , ad NSF grat as part of the NSF/DHS/DOT/NASA/NIH Cyber-Physical Systems Program. Na Li is supported by Harvard Ceter for Gree Buildigs ad Cities. REFERENCES [1] N. Basal, A. Blum, S. Chawla, ad A. Meyerso, Olie oblivious routig, i Proceedigs of the fifteeth aual ACM symposium o Parallel algorithms ad architectures. ACM, 003, pp [] B. Awerbuch ad R. Kleiberg, Olie liear optimizatio ad adaptive routig, Joural of Computer ad System Scieces, vol. 74, o. 1, pp , 008. [3] M. Li, Z. Liu, A. Wierma, ad L. L. Adrew, Olie algorithms for geographical load balacig, i Gree Computig Coferece IGCC), 01 Iteratioal. IEEE, 01, pp [4] M. Li, A. Wierma, L. L. Adrew, ad E. Thereska, Dyamic rightsizig for power-proportioal data ceters, IEEE/ACM Trasactios o Networkig TON), vol. 1, o. 5, pp , 013. [5] T. M. Cover, Uiversal portfolios, Mathematical fiace, vol. 1, o. 1, pp. 1 9, [6] A. Blum, V. Kumar, A. Rudra, ad F. Wu, Olie learig i olie auctios, Theoretical Computer Sciece, vol. 34, o., pp , 004. [7] S. Shalev-Shwartz, Olie learig ad olie covex optimizatio, Foudatios ad Treds i Machie Learig, vol. 4, o., pp , 011. [8] E. Haza ad S. Kale, Beyod the regret miimizatio barrier: Optimal algorithms for stochastic strogly-covex optimizatio, Joural of Machie Learig Research, vol. 15, pp , 014. [Olie]. Available: [9] E. Haza, 10 the covex optimizatio approach to regret miimizatio, Optimizatio for machie learig, p. 87, 01. [10] N. Che, A. Agarwal, A. Wierma, S. Barma, ad A. Lachla, Olie covex optimizatio usig predictios, i Proceedig of ACM SIGMETRICS Performace Evaluatio Review. ACM, 015. [11] S. Bubeck ad N. Cesa-Biachi, Regret aalysis of stochastic ad ostochastic multi-armed badit problems, Foudatios ad Treds i Machie Learig, vol. 5, o. 1, pp. 1 1, 01. [1] C. Wag ad S. Shahidehpour, Effects of ramp-rate limits o uit commitmet ad ecoomic dispatch, Power Systems, IEEE Trasactios o, vol. 8, o. 3, pp , [13] A. Fragioi, C. Getile, ad F. Lacaladra, Solvig uit commitmet problems with geeral ramp costraits, Iteratioal Joural of Electrical Power & Eergy Systems, vol. 30, o. 5, pp , 008. [14] A. Borodi, N. Liial, ad M. E. Saks, A optimal o-lie algorithm for metrical task system, Joural of the ACM JACM), vol. 39, o. 4, pp , 199. [15] J. Camacho, Y. Zhag, M. Che, ad D. M. Chiu, Balace your bids before your bits: The ecoomics of geographic load-balacig, i Proceedigs of the 5th iteratioal coferece o Future eergy systems. ACM, 014, pp [16] B. Kveto, J. Y. Yu, G. Theocharous, ad S. Maor, A lazy approach to olie learig with costraits. i ISAIM. Citeseer, 008. [17] S. Maor, J. N. Tsitsiklis, ad J. Y. Yu, Olie learig with sample path costraits, The Joural of Machie Learig Research, vol. 10, pp , 009. [18] M. Mahdavi, R. Ji, ad T. Yag, Tradig regret for efficiecy: olie covex optimizatio with log term costraits, The Joural of Machie Learig Research, vol. 13, o. 1, pp , 01. [19] L. Adrew, S. Barma, K. Ligett, M. Li, A. Meyerso, A. Roytma, ad A. Wierma, A tale of two metrics: simultaeous bouds o competitiveess ad regret, i Proceedigs of COLT, 014. [0] B. H. Chowdhury ad S. Rahma, A review of recet advaces i ecoomic dispatch. Istitute of Electrical ad Electroics Egieers, [1] H. Happ, Optimal power dispatch: A comprehesive survey, Power Apparatus ad Systems, IEEE Trasactios o, vol. 96, o. 3, pp , [] S. Mei, Y. Wag, ad Z. Su, Robust ecoomic dispatch cosiderig reewable geeratio, i Iovative Smart Grid Techologies Asia ISGT), 011 IEEE PES. IEEE, 011, pp [3] L. dos Satos Coelho ad C.-S. Lee, Solvig ecoomic load dispatch problems i power systems usig chaotic ad gaussia particle swarm optimizatio approaches, Iteratioal Joural of Electrical Power ad Eergy Systems, vol. 30, o. 5, pp , 008. [4] [olie]. Available: [5] M. A. Epelma. 01) Barrier methods for costraied optimizatio. [Olie]. Available: mepelma/teachig/ioe511/hadouts/511otes07-11.pdf 6735

7 A. Proof of Theorem 1 VIII. APPENDIX First we state a techical lemma: Lemma 1. Lets, u, v, x, y, z R +. The, we have the followig iequality x y z) + + y x u) + v y z) + + y v u) + where x + = mi{x, 0}. + x v, Proof: Based o the triagle iequality we have that x y z) + v y z) + + x v) + 16) y x u) + y v u) + + v x) +. 17) Addig 16) ad 17) ad usig the fact that x v) + + v x) + = x v, proves the claim. Next, ow ie the cost fuctio ) = c t ˆx t0 t ), where ˆx i,t0 t is the solutio give by 9). We use the pealty fuctio method to ie a relaxed versio of the optimizatio problem i 9) as below γ t 0 where γ t0 px t ) = ) = mi x t,,x t+w 1 ct x t ) + γ t0 px t ) ), 18) R 0, ad N x i t x i t 1 X N i ) + + x i t 1 x i t X i ) +. The followig lemma is show i [5, p. 63]: } Lemma. Suppose γ t0 t0 t+w max { µ t, 1 t. The, with the same iitial coditio, the set of optimal solutios of 9) ad 18) coicide. Specifically, we have ) = Ct0 ). γ t 0 Based o Lemma we ow obtai that ) = Ct0 ) 19) γ t γ t0 N c t x t ) + γ t0 x i, t +1 px t ) 0) ˆx i,t0 t 1 X i ) + + ˆx i,t0 t 1 xi, t X i ) +), where the iequality follows from the fact that the offlie trajectory x t ) T for the optimizatio problem i 9) with the iitial coditio ˆx t0 t 1 is sub-optimal. Usig Lemma 1, we further derive x i, t x i, t ˆx i,t0 t 1 X i ) + + ˆx i,t0 t 1 xi, t X i ) + x i, t 1 X i ) + + x i, t 1 xi, t X i ) + + ˆx i,t0 t 1 xi, t 1. 1) I the case of = 0, we have ˆx i,t0 t = 0 1 xi, t = 0 1 xi 0 ad thus, the above iequality ca be replaced by the equality without the third term. We substitute 1) ito 0) to obtai, t+w 1 ) 1 a) = c t x t ) + γ t0 px t ) + γ t0 ˆx t0 c t x t ) + γ t0 ˆx t0 t 1 x t 1 1, t 1 x t 1 1 where i a) comes from the fact that the pealty fuctio for the offlie solutio is zero, i.e. px t ) = 0. We use the two upper bouds to boud ˆx t0 t 1 x t 1 1. The first boud is trivially give by ad the secod boud is ˆx t0 t 1 x t 1 1 r, ) ˆx t0 t 1 x t 1 1 W X + X 1. 3) Lets ie = M =0 Ct0 ). By usig the first boud i ), we ca proceed as below C A c) = 1 W 1 W t 0 W M t 0 W =0 d) C offlie + 1 W ) 4) M t 0 W =1 γ t0 ˆx t0 t 1 x t 1 1 5) e) = C offlie + γmr, 6) where c) follows from the covexity of the cost fuctio c t ) as well as the remark after 1), d) is due to the followig iequality M =0 c t x t ) T c t x t ) = C offlie, ad e) follows by iig γ } t0 T = max t0 W max { µ t, t. Usig the alterative upper boud i 3) ad followig a similar argumet as above gives C A C offlie + γ W t 0 W =1 M W X + 1 X M + 1)M = C offlie + γ W X ) X The fial boud is the miimum of 6), 7). We omit the argumet for the case of strog covexity due to space costraits. 6736