Notes on online convex optimization
|
|
- Griffin Bryant
- 5 years ago
- Views:
Transcription
1 Noes on online convex opimizaion Karl Sraos Online convex opimizaion (OCO) is a principled framework for online learning: OnlineConvexOpimizaion Inpu: convex se S, number of seps T For =, 2,..., T : Selec w S. Receive a convex loss f : S R chosen adversarially. Suffer loss f (w ). Each hypohesis is a vecor in some convex se S. The loss funcion f : S R is convex and defined for each ime sep individually. Our goal is o have small regre wih respec o a hypohesis space U, namely Regre T (U) := max u U Regre T (u) where Regre T (u) := f (w ) f (u) Unregularized aggregae loss minimizaion A ime, we have observed losses f... f, so a naural choice of w is one ha minimizes he sum of all pas losses. This is known as Follow-he-Leader (FTL): f i (w) () Lemma.. If we use Eq. () in OCO, we have Regre T (S) f (w ) f (w + ) 2 Regularized aggregae loss minimizaion Lemma. suggess a need for conaining f (w ) f (w + ). If we assume f is L -Lipschiz wih respec o S and some norm, we have f (w ) f (w + ) L w w + This is a bird s eye view of he incredible uorial by Shai Shalev-Shwarz (20). For full deails, see he original uorial.
2 which in urn suggess a need for conaining w w +. If he objecive in Eq. () F (w) := f i (w) happens o be σ-srongly-convex, w w + canno be arbirarily large: by he definiion of w and w + and srong convexiy, Adding hese wo inequaliies, we ge: F (w + ) F (w ) σ 2 w w + 2 F + (w ) F + (w + ) σ 2 w w + 2 w w + f (w ) f (w + ) L σ σ We can always endow σ-srong-convexiy on F by adding a σ-srongly-convex regularizer R : S R. This is known as Follow-he-Regularized-Leader (FoReL): R(w) + f i (w) (2) By reaing R as he (convex) loss a ime = 0, we ge he following corollary from Lemma.. Corollary 2.. If we use Eq. (2) in OCO, for all u S we have Regre T (u) R(u) min R(v) + T f (w ) f (w + ) v S Theorem 2.2. Le f : S R be convex loss funcions ha are L -Lipschiz over convex S wih respec o. Le L R be a consan such ha L 2 (/T ) T L2, and le R : S R be a σ-srongly-convex regularizer. Then he regre of FoReL wih respec o u S is bounded above as: T L2 Regre T (u) R(u) min R(v) + v S σ 3 Linearizaion of convex losses Theorem 2.2 assumes an oracle ha solves Eq. (2), so i s no very useful for deriving concree algorihms. Bu a echnique known as linearizaion of convex losses grealy simplifies his ask. Since S is a convex se and f is convex, a each round of OCO we can selec z f (w ) so ha f (w ) f (w + ) z, w z, w + (3) Thus given a general convex loss f, we can preend ha i s a linear loss g (u) := z, u where z is a sub-gradien of f a w. In ligh of Corollary 2. and Eq. (3), running FoReL on hese linearized losses: enjoys he same regre bound in Theorem 2.2. R(w) + w, z i (4) 2
3 3. Online mirror descen Eq. (4) can be addiionally analyzed in a dual framework known as online mirror descen (OMD). OMD frames Eq. (4) as wo separae seps: saring wih θ := 0, w = g(θ ) θ = θ z where g(θ) := arg max w, θ R(w) is known as he link funcion. The paricular form of he link funcion comes from he convex conjugae of R (R is assumed o be closed and convex): R (θ) := max w, θ R(w) A propery of R is ha if z R (θ), hen R (θ) = z, θ R(z). Thus g(θ ) = z R (θ ). This framework can be used o show ha OMD achieves Regre T (u) R(u) + min R(v) + T v S D R ( ) z i z i where D R (u v) is he Bregman divergence beween u and v under R. If R is (/η)- srongly-convex wih respec o, hen R is η-srongly-smooh wih respec o he dual norm : in his case, 3.2 Example algorihms Regre T (u) R(u) + min v S R(v) + η 2 (5) z 2 (6) We can now crank ou algorihms under he OMD framework. All hese algorihms enjoy he bound in Theorem 2.2 (or Eq. (6)). Online gradien descen (OGD): Assumes an unconsrained domain S = R d and an l 2 regularizer R(w) = 2η w 2 2. We have g(θ) = ηθ and w = w ηz (7) Online gradien descen wih lazy projecions (OGDLP): Assumes a general convex se S and an l 2 regularizer R(w) = 2η w 2 2. Noe ha Thus he link funcion g(θ) projecs ηθ ono S. 2η w 2 2 w, θ = arg min w ηθ 2 2 (8) Unnormalized exponeniaed gradien descen (UEG): Assumes an unconsrained domain S = R d and a shifed enropy regularizer R(w) = η i w i(log w i log λ) where λ > 0. We have g i (θ) = λ exp(ηθ i ), hus w = (λ... λ) and for i > : [w ] i = [w ] i exp( η[z ] i ) (9) 3
4 Normalized exponeniaed gradien descen (NEG): Assumes a probabiliy simplex S = {w R d : w 0, i w i = } and an enropy regularizer R(w) = i w i log w i. We have g i (θ) = exp(ηθi) j exp(ηθj), hus w = (/d... /d) and for i > : η [w ] i = [w ] i exp( η[z ] i ) j [w ] j exp( η[z ] j ) (0) 4 Applicaions o classificaion problems The cenral sep in applying OCO o a classificaion problem is finding he righ convex surrogae of he problem. 4. Percepron A each round, we re given a poin x R d. We predic p {, +} and receive he rue class y {, +}. The (non-convex) loss is given by { if p y l(p, y ) := 0 if p = y Noe ha he cumulaive loss M := l(p, y ) is he number of misakes. Convex surrogae: We mainain a vecor w R d ha defines p := sign w, x. We use a hinge loss f (w ) := max(0, y w, x ) which by he paricular consrucion is convex and upperbounds he original loss l(p, y ). Using a sub-gradien z f (w ) where z = y x if y w, x and z = 0 oherwise, we can now run OGD using some η > 0: w := 0 and { w + ηy w + := x if y w, x w if y w, x > Le L := max z. I s possible o apply Eq. (6) and show ha for any u R d M f (u) + u 2 L f (u) + L 2 u 2 2 In paricular, if here exiss u R d such ha f (u) = 0, we have M L 2 u Weighed majoriy A each round, we re given a poin x X and d hypoheses H = {h,..., h d } where h i : X {0, }. We make a choice p [d] and receive he rue class y {0, }. The (non-convex) loss is given by { if hp (x l(p, y ) := ) y 0 if h p (x ) = y 4
5 Convex surrogae: We mainain a vecor w {w R d : w 0, i w i = }. This vecor defines weighed majoriy : p = if d [w ] i h i (x ) /2 and p = 0 oherwise. We use he convex loss funcion: f (w ) := d [w ] i h i (x ) y = w, z where [z ] i := h i (x ) y (hus z is also he gradien of f ). Hence we have an online linear problem suiable for NEG. I s possible o show ha if here exiss some h H such ha T h(x ) y = 0, hen NEG achieves f (w ) 4 log d Muli-armed bandi A problem closely relaed o weighed majoriy is he so-called muli-armed bandi problem. A each round, here d slo machines ( one-armed bandis ) o choose from. We make a choice p [d] and receive he cos of playing ha machine: [y ] p [0, ]. A crucial aspec of he problem is he exisence of unobserved coss [y ] i [0, ] for i p, because if we observe all y [0, ] d we can jus formulae i as an online linear problem by minimizing he expeced loss f (w ) := w, y where w {w R d : w 0, i w i = } again defines weighed majoriy over d machines. Since y is he gradien of f, anoher way of saing he difficuly is ha gradiens are no fully observed. A soluion is o use a p -dependen esimaor z (p) [z (p) ] i { [y ] i /[w ] i if i = p 0 if i p of he gradien y as follows: This is indeed an unbiased esimaor of y over he randomness of p since E[z (p) ] i := d p p(p )[z (p) [y ] i ] i = w i + 0 = [y ] i [w ] i p i Thus we can run NEG by subsiuing he unobserved gradien y wih z (p). Noe ha he algorihm will be slighly differen from he weighed majoriy algorihm since we need o acually make he predicion p w which is required for compuing. I s possible o derive regre bounds where he regre is defined as he difference beween he algorihm s expeced cumulaive cos (over he randomness of p ) and he cumulaive cos of he bes machine: [ T ] E [y ] p min [y ] i z (p) i [d] Reference Shalev-Shwarz, S. (20). Online Learning and Online Convex Opimizaion. 5
Lecture 4: November 13
Compuaional Learning Theory Fall Semeser, 2017/18 Lecure 4: November 13 Lecurer: Yishay Mansour Scribe: Guy Dolinsky, Yogev Bar-On, Yuval Lewi 4.1 Fenchel-Conjugae 4.1.1 Moivaion Unil his lecure we saw
More informationOnline Convex Optimization Example And Follow-The-Leader
CSE599s, Spring 2014, Online Learning Lecure 2-04/03/2014 Online Convex Opimizaion Example And Follow-The-Leader Lecurer: Brendan McMahan Scribe: Sephen Joe Jonany 1 Review of Online Convex Opimizaion
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Noes for EE7C Spring 018: Convex Opimizaion and Approximaion Insrucor: Moriz Hard Email: hard+ee7c@berkeley.edu Graduae Insrucor: Max Simchowiz Email: msimchow+ee7c@berkeley.edu Ocober 15, 018 3
More information1 Review of Zero-Sum Games
COS 5: heoreical Machine Learning Lecurer: Rob Schapire Lecure #23 Scribe: Eugene Brevdo April 30, 2008 Review of Zero-Sum Games Las ime we inroduced a mahemaical model for wo player zero-sum games. Any
More informationOnline Learning with Partial Feedback. 1 Online Mirror Descent with Estimated Gradient
Avance Course in Machine Learning Spring 2010 Online Learning wih Parial Feeback Hanous are joinly prepare by Shie Mannor an Shai Shalev-Shwarz In previous lecures we alke abou he general framework of
More informationEnsamble methods: Boosting
Lecure 21 Ensamble mehods: Boosing Milos Hauskrech milos@cs.pi.edu 5329 Senno Square Schedule Final exam: April 18: 1:00-2:15pm, in-class Term projecs April 23 & April 25: a 1:00-2:30pm in CS seminar room
More informationEnsamble methods: Bagging and Boosting
Lecure 21 Ensamble mehods: Bagging and Boosing Milos Hauskrech milos@cs.pi.edu 5329 Senno Square Ensemble mehods Mixure of expers Muliple base models (classifiers, regressors), each covers a differen par
More informationLecture 9: September 25
0-725: Opimizaion Fall 202 Lecure 9: Sepember 25 Lecurer: Geoff Gordon/Ryan Tibshirani Scribes: Xuezhi Wang, Subhodeep Moira, Abhimanu Kumar Noe: LaTeX emplae couresy of UC Berkeley EECS dep. Disclaimer:
More informationA Primal-Dual Type Algorithm with the O(1/t) Convergence Rate for Large Scale Constrained Convex Programs
PROC. IEEE CONFERENCE ON DECISION AND CONTROL, 06 A Primal-Dual Type Algorihm wih he O(/) Convergence Rae for Large Scale Consrained Convex Programs Hao Yu and Michael J. Neely Absrac This paper considers
More informationSupplement for Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence
Supplemen for Sochasic Convex Opimizaion: Faser Local Growh Implies Faser Global Convergence Yi Xu Qihang Lin ianbao Yang Proof of heorem heorem Suppose Assumpion holds and F (w) obeys he LGC (6) Given
More informationArticle from. Predictive Analytics and Futurism. July 2016 Issue 13
Aricle from Predicive Analyics and Fuurism July 6 Issue An Inroducion o Incremenal Learning By Qiang Wu and Dave Snell Machine learning provides useful ools for predicive analyics The ypical machine learning
More informationPENALIZED LEAST SQUARES AND PENALIZED LIKELIHOOD
PENALIZED LEAST SQUARES AND PENALIZED LIKELIHOOD HAN XIAO 1. Penalized Leas Squares Lasso solves he following opimizaion problem, ˆβ lasso = arg max β R p+1 1 N y i β 0 N x ij β j β j (1.1) for some 0.
More informationLecture 20: Riccati Equations and Least Squares Feedback Control
34-5 LINEAR SYSTEMS Lecure : Riccai Equaions and Leas Squares Feedback Conrol 5.6.4 Sae Feedback via Riccai Equaions A recursive approach in generaing he marix-valued funcion W ( ) equaion for i for he
More informationLearning a Class from Examples. Training set X. Class C 1. Class C of a family car. Output: Input representation: x 1 : price, x 2 : engine power
Alpaydin Chaper, Michell Chaper 7 Alpaydin slides are in urquoise. Ehem Alpaydin, copyrigh: The MIT Press, 010. alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/ ehem/imle All oher slides are based on Michell.
More informationFinish reading Chapter 2 of Spivak, rereading earlier sections as necessary. handout and fill in some missing details!
MAT 257, Handou 6: Ocober 7-2, 20. I. Assignmen. Finish reading Chaper 2 of Spiva, rereading earlier secions as necessary. handou and fill in some missing deails! II. Higher derivaives. Also, read his
More informationHamilton- J acobi Equation: Explicit Formulas In this lecture we try to apply the method of characteristics to the Hamilton-Jacobi equation: u t
M ah 5 2 7 Fall 2 0 0 9 L ecure 1 0 O c. 7, 2 0 0 9 Hamilon- J acobi Equaion: Explici Formulas In his lecure we ry o apply he mehod of characerisics o he Hamilon-Jacobi equaion: u + H D u, x = 0 in R n
More informationMATH 5720: Gradient Methods Hung Phan, UMass Lowell October 4, 2018
MATH 5720: Gradien Mehods Hung Phan, UMass Lowell Ocober 4, 208 Descen Direcion Mehods Consider he problem min { f(x) x R n}. The general descen direcions mehod is x k+ = x k + k d k where x k is he curren
More informationPhysics 235 Chapter 2. Chapter 2 Newtonian Mechanics Single Particle
Chaper 2 Newonian Mechanics Single Paricle In his Chaper we will review wha Newon s laws of mechanics ell us abou he moion of a single paricle. Newon s laws are only valid in suiable reference frames,
More informationInventory Analysis and Management. Multi-Period Stochastic Models: Optimality of (s, S) Policy for K-Convex Objective Functions
Muli-Period Sochasic Models: Opimali of (s, S) Polic for -Convex Objecive Funcions Consider a seing similar o he N-sage newsvendor problem excep ha now here is a fixed re-ordering cos (> 0) for each (re-)order.
More informationHamilton- J acobi Equation: Weak S olution We continue the study of the Hamilton-Jacobi equation:
M ah 5 7 Fall 9 L ecure O c. 4, 9 ) Hamilon- J acobi Equaion: Weak S oluion We coninue he sudy of he Hamilon-Jacobi equaion: We have shown ha u + H D u) = R n, ) ; u = g R n { = }. ). In general we canno
More informationHamilton Jacobi equations
Hamilon Jacobi equaions Inoducion o PDE The rigorous suff from Evans, mosly. We discuss firs u + H( u = 0, (1 where H(p is convex, and superlinear a infiniy, H(p lim p p = + This by comes by inegraion
More informationMatrix Versions of Some Refinements of the Arithmetic-Geometric Mean Inequality
Marix Versions of Some Refinemens of he Arihmeic-Geomeric Mean Inequaliy Bao Qi Feng and Andrew Tonge Absrac. We esablish marix versions of refinemens due o Alzer ], Carwrigh and Field 4], and Mercer 5]
More informationLie Derivatives operator vector field flow push back Lie derivative of
Lie Derivaives The Lie derivaive is a mehod of compuing he direcional derivaive of a vecor field wih respec o anoher vecor field We already know how o make sense of a direcional derivaive of real valued
More information1 Widrow-Hoff Algorithm
COS 511: heoreical Machine Learning Lecurer: Rob Schapire Lecure # 18 Scribe: Shaoqing Yang April 10, 014 1 Widrow-Hoff Algorih Firs le s review he Widrow-Hoff algorih ha was covered fro las lecure: Algorih
More informationHomework sheet Exercises done during the lecture of March 12, 2014
EXERCISE SESSION 2A FOR THE COURSE GÉOMÉTRIE EUCLIDIENNE, NON EUCLIDIENNE ET PROJECTIVE MATTEO TOMMASINI Homework shee 3-4 - Exercises done during he lecure of March 2, 204 Exercise 2 Is i rue ha he parameerized
More informationINTRODUCTION TO MACHINE LEARNING 3RD EDITION
ETHEM ALPAYDIN The MIT Press, 2014 Lecure Slides for INTRODUCTION TO MACHINE LEARNING 3RD EDITION alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/i2ml3e CHAPTER 2: SUPERVISED LEARNING Learning a Class
More informationA Forward-Backward Splitting Method with Component-wise Lazy Evaluation for Online Structured Convex Optimization
A Forward-Backward Spliing Mehod wih Componen-wise Lazy Evaluaion for Online Srucured Convex Opimizaion Yukihiro Togari and Nobuo Yamashia March 28, 2016 Absrac: We consider large-scale opimizaion problems
More informationLecture 33: November 29
36-705: Inermediae Saisics Fall 2017 Lecurer: Siva Balakrishnan Lecure 33: November 29 Today we will coninue discussing he boosrap, and hen ry o undersand why i works in a simple case. In he las lecure
More informationBoosting with Online Binary Learners for the Multiclass Bandit Problem
Shang-Tse Chen School of Compuer Science, Georgia Insiue of Technology, Alana, GA Hsuan-Tien Lin Deparmen of Compuer Science and Informaion Engineering Naional Taiwan Universiy, Taipei, Taiwan Chi-Jen
More informationDifferential Geometry: Numerical Integration and Surface Flow
Differenial Geomery: Numerical Inegraion and Surface Flow [Implici Fairing of Irregular Meshes using Diffusion and Curaure Flow. Desbrun e al., 1999] Energy Minimizaion Recall: We hae been considering
More informationFinal Spring 2007
.615 Final Spring 7 Overview The purpose of he final exam is o calculae he MHD β limi in a high-bea oroidal okamak agains he dangerous n = 1 exernal ballooning-kink mode. Effecively, his corresponds o
More informationLearning a Class from Examples. Training set X. Class C 1. Class C of a family car. Output: Input representation: x 1 : price, x 2 : engine power
Alpaydin Chaper, Michell Chaper 7 Alpaydin slides are in urquoise. Ehem Alpaydin, copyrigh: The MIT Press, 010. alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/ ehem/imle All oher slides are based on Michell.
More information1 Solutions to selected problems
1 Soluions o seleced problems 1. Le A B R n. Show ha in A in B bu in general bd A bd B. Soluion. Le x in A. Then here is ɛ > 0 such ha B ɛ (x) A B. This shows x in B. If A = [0, 1] and B = [0, 2], hen
More informationNon-parametric techniques. Instance Based Learning. NN Decision Boundaries. Nearest Neighbor Algorithm. Distance metric important
on-parameric echniques Insance Based Learning AKA: neares neighbor mehods, non-parameric, lazy, memorybased, or case-based learning Copyrigh 2005 by David Helmbold 1 Do no fi a model (as do LDA, logisic
More informationCSE/NB 528 Lecture 14: From Supervised to Reinforcement Learning (Chapter 9) R. Rao, 528: Lecture 14
CSE/NB 58 Lecure 14: From Supervised o Reinforcemen Learning Chaper 9 1 Recall from las ime: Sigmoid Neworks Oupu v T g w u g wiui w Inpu nodes u = u 1 u u 3 T i Sigmoid oupu funcion: 1 g a 1 a e 1 ga
More informationChapter 21. Reinforcement Learning. The Reinforcement Learning Agent
CSE 47 Chaper Reinforcemen Learning The Reinforcemen Learning Agen Agen Sae u Reward r Acion a Enironmen CSE AI Faculy Why reinforcemen learning Programming an agen o drie a car or fly a helicoper is ery
More informationSections 2.2 & 2.3 Limit of a Function and Limit Laws
Mah 80 www.imeodare.com Secions. &. Limi of a Funcion and Limi Laws In secion. we saw how is arise when we wan o find he angen o a curve or he velociy of an objec. Now we urn our aenion o is in general
More informationWe just finished the Erdős-Stone Theorem, and ex(n, F ) (1 1/(χ(F ) 1)) ( n
Lecure 3 - Kövari-Sós-Turán Theorem Jacques Versraëe jacques@ucsd.edu We jus finished he Erdős-Sone Theorem, and ex(n, F ) ( /(χ(f ) )) ( n 2). So we have asympoics when χ(f ) 3 bu no when χ(f ) = 2 i.e.
More informationNotes for Lecture 17-18
U.C. Berkeley CS278: Compuaional Complexiy Handou N7-8 Professor Luca Trevisan April 3-8, 2008 Noes for Lecure 7-8 In hese wo lecures we prove he firs half of he PCP Theorem, he Amplificaion Lemma, up
More informationMath Week 15: Section 7.4, mass-spring systems. These are notes for Monday. There will also be course review notes for Tuesday, posted later.
Mah 50-004 Week 5: Secion 7.4, mass-spring sysems. These are noes for Monday. There will also be course review noes for Tuesday, posed laer. Mon Apr 3 7.4 mass-spring sysems. Announcemens: Warm up exercise:
More informationIntroduction to Probability and Statistics Slides 4 Chapter 4
Inroducion o Probabiliy and Saisics Slides 4 Chaper 4 Ammar M. Sarhan, asarhan@mahsa.dal.ca Deparmen of Mahemaics and Saisics, Dalhousie Universiy Fall Semeser 8 Dr. Ammar Sarhan Chaper 4 Coninuous Random
More informationTransform Techniques. Moment Generating Function
Transform Techniques A convenien way of finding he momens of a random variable is he momen generaing funcion (MGF). Oher ransform echniques are characerisic funcion, z-ransform, and Laplace ransform. Momen
More informationCSE/NB 528 Lecture 14: Reinforcement Learning (Chapter 9)
CSE/NB 528 Lecure 14: Reinforcemen Learning Chaper 9 Image from hp://clasdean.la.asu.edu/news/images/ubep2001/neuron3.jpg Lecure figures are from Dayan & Abbo s book hp://people.brandeis.edu/~abbo/book/index.hml
More informationLongest Common Prefixes
Longes Common Prefixes The sandard ordering for srings is he lexicographical order. I is induced by an order over he alphabe. We will use he same symbols (,
More informationNon-parametric techniques. Instance Based Learning. NN Decision Boundaries. Nearest Neighbor Algorithm. Distance metric important
on-parameric echniques Insance Based Learning AKA: neares neighbor mehods, non-parameric, lazy, memorybased, or case-based learning Copyrigh 2005 by David Helmbold 1 Do no fi a model (as do LTU, decision
More informationAppendix to Online l 1 -Dictionary Learning with Application to Novel Document Detection
Appendix o Online l -Dicionary Learning wih Applicaion o Novel Documen Deecion Shiva Prasad Kasiviswanahan Huahua Wang Arindam Banerjee Prem Melville A Background abou ADMM In his secion, we give a brief
More informationarxiv: v1 [math.oc] 11 Sep 2017
Online Learning in Weakly Coupled Markov Decision Processes: A Convergence ime Sudy Xiaohan Wei, Hao Yu and Michael J. Neely arxiv:709.03465v [mah.oc] Sep 07. Inroducion Absrac: We consider muliple parallel
More informationEchocardiography Project and Finite Fourier Series
Echocardiography Projec and Finie Fourier Series 1 U M An echocardiagram is a plo of how a porion of he hear moves as he funcion of ime over he one or more hearbea cycles If he hearbea repeas iself every
More informationarxiv: v2 [cs.lg] 28 Dec 2018
Dynamic Online Gradien Descen wih Improved Query Complexiy: A Theoreical Revisi arxiv:8.086v [cs.lg] 8 Dec 08 Yawei Zhao, En Zhu, Xinwang Liu, and Jianping Yin Naional Universiy of Defense Technology,
More informationState-Space Models. Initialization, Estimation and Smoothing of the Kalman Filter
Sae-Space Models Iniializaion, Esimaion and Smoohing of he Kalman Filer Iniializaion of he Kalman Filer The Kalman filer shows how o updae pas predicors and he corresponding predicion error variances when
More informationGames Against Nature
Advanced Course in Machine Learning Spring 2010 Games Agains Naure Handous are joinly prepared by Shie Mannor and Shai Shalev-Shwarz In he previous lecures we alked abou expers in differen seups and analyzed
More informationBeating the Adaptive Bandit with High Probability
Beaing he Adapive Bandi wih High Probabiliy Jacob Abernehy Compuer Science Division UC Berkeley jake@cs.berkeley.edu Alexander Rakhlin Deparmen of Saisics Universiy of Pennsylvania rakhlin@wharon.upenn.edu
More informationSolutions from Chapter 9.1 and 9.2
Soluions from Chaper 9 and 92 Secion 9 Problem # This basically boils down o an exercise in he chain rule from calculus We are looking for soluions of he form: u( x) = f( k x c) where k x R 3 and k is
More informationdt = C exp (3 ln t 4 ). t 4 W = C exp ( ln(4 t) 3) = C(4 t) 3.
Mah Rahman Exam Review Soluions () Consider he IVP: ( 4)y 3y + 4y = ; y(3) = 0, y (3) =. (a) Please deermine he longes inerval for which he IVP is guaraneed o have a unique soluion. Soluion: The disconinuiies
More informationEcon107 Applied Econometrics Topic 7: Multicollinearity (Studenmund, Chapter 8)
I. Definiions and Problems A. Perfec Mulicollineariy Econ7 Applied Economerics Topic 7: Mulicollineariy (Sudenmund, Chaper 8) Definiion: Perfec mulicollineariy exiss in a following K-variable regression
More information2.7. Some common engineering functions. Introduction. Prerequisites. Learning Outcomes
Some common engineering funcions 2.7 Inroducion This secion provides a caalogue of some common funcions ofen used in Science and Engineering. These include polynomials, raional funcions, he modulus funcion
More informationTwo Coupled Oscillators / Normal Modes
Lecure 3 Phys 3750 Two Coupled Oscillaors / Normal Modes Overview and Moivaion: Today we ake a small, bu significan, sep owards wave moion. We will no ye observe waves, bu his sep is imporan in is own
More informationZápadočeská Univerzita v Plzni, Czech Republic and Groupe ESIEE Paris, France
ADAPTIVE SIGNAL PROCESSING USING MAXIMUM ENTROPY ON THE MEAN METHOD AND MONTE CARLO ANALYSIS Pavla Holejšovsá, Ing. *), Z. Peroua, Ing. **), J.-F. Bercher, Prof. Assis. ***) Západočesá Univerzia v Plzni,
More informationChapter 2. First Order Scalar Equations
Chaper. Firs Order Scalar Equaions We sar our sudy of differenial equaions in he same way he pioneers in his field did. We show paricular echniques o solve paricular ypes of firs order differenial equaions.
More informationELE 538B: Large-Scale Optimization for Data Science. Quasi-Newton methods. Yuxin Chen Princeton University, Spring 2018
ELE 538B: Large-Scale Opimizaion for Daa Science Quasi-Newon mehods Yuxin Chen Princeon Universiy, Spring 208 00 op ff(x (x)(k)) f p 2 L µ f 05 k f (xk ) k f (xk ) =) f op ieraions converges in only 5
More informationAsymptotic Equipartition Property - Seminar 3, part 1
Asympoic Equipariion Propery - Seminar 3, par 1 Ocober 22, 2013 Problem 1 (Calculaion of ypical se) To clarify he noion of a ypical se A (n) ε and he smalles se of high probabiliy B (n), we will calculae
More informationLecture 2-1 Kinematics in One Dimension Displacement, Velocity and Acceleration Everything in the world is moving. Nothing stays still.
Lecure - Kinemaics in One Dimension Displacemen, Velociy and Acceleraion Everyhing in he world is moving. Nohing says sill. Moion occurs a all scales of he universe, saring from he moion of elecrons in
More informationE β t log (C t ) + M t M t 1. = Y t + B t 1 P t. B t 0 (3) v t = P tc t M t Question 1. Find the FOC s for an optimum in the agent s problem.
Noes, M. Krause.. Problem Se 9: Exercise on FTPL Same model as in paper and lecure, only ha one-period govenmen bonds are replaced by consols, which are bonds ha pay one dollar forever. I has curren marke
More informationStationary Distribution. Design and Analysis of Algorithms Andrei Bulatov
Saionary Disribuion Design and Analysis of Algorihms Andrei Bulaov Algorihms Markov Chains 34-2 Classificaion of Saes k By P we denoe he (i,j)-enry of i, j Sae is accessible from sae if 0 for some k 0
More informationSome Ramsey results for the n-cube
Some Ramsey resuls for he n-cube Ron Graham Universiy of California, San Diego Jozsef Solymosi Universiy of Briish Columbia, Vancouver, Canada Absrac In his noe we esablish a Ramsey-ype resul for cerain
More informationEXERCISES FOR SECTION 1.5
1.5 Exisence and Uniqueness of Soluions 43 20. 1 v c 21. 1 v c 1 2 4 6 8 10 1 2 2 4 6 8 10 Graph of approximae soluion obained using Euler s mehod wih = 0.1. Graph of approximae soluion obained using Euler
More informationL07. KALMAN FILTERING FOR NON-LINEAR SYSTEMS. NA568 Mobile Robotics: Methods & Algorithms
L07. KALMAN FILTERING FOR NON-LINEAR SYSTEMS NA568 Mobile Roboics: Mehods & Algorihms Today s Topic Quick review on (Linear) Kalman Filer Kalman Filering for Non-Linear Sysems Exended Kalman Filer (EKF)
More informationLet us start with a two dimensional case. We consider a vector ( x,
Roaion marices We consider now roaion marices in wo and hree dimensions. We sar wih wo dimensions since wo dimensions are easier han hree o undersand, and one dimension is a lile oo simple. However, our
More informationt dt t SCLP Bellman (1953) CLP (Dantzig, Tyndall, Grinold, Perold, Anstreicher 60's-80's) Anderson (1978) SCLP
Coninuous Linear Programming. Separaed Coninuous Linear Programming Bellman (1953) max c () u() d H () u () + Gsusds (,) () a () u (), < < CLP (Danzig, yndall, Grinold, Perold, Ansreicher 6's-8's) Anderson
More informationPrimal-Dual Splitting: Recent Improvements and Variants
Primal-Dual Spliing: Recen Improvemens and Varians 1 Thomas Pock and 2 Anonin Chambolle 1 Insiue for Compuer Graphics and Vision, TU Graz, Ausria 2 CMAP & CNRS École Polyechnique, France The proximal poin
More informationApproximation Algorithms for Unique Games via Orthogonal Separators
Approximaion Algorihms for Unique Games via Orhogonal Separaors Lecure noes by Konsanin Makarychev. Lecure noes are based on he papers [CMM06a, CMM06b, LM4]. Unique Games In hese lecure noes, we define
More informationMinimizing Regret on Reflexive Banach Spaces and Nash Equilibria in Continuous Zero-Sum Games
Minimizing Regre on Reflexive Banach Spaces and Nash Equilibria in Coninuous Zero-Sum Games Maximilian Balanda, Walid Krichene, Claire Tomlin, Alexandre Bayen Elecrical Engineering and Compuer Sciences,
More informationOn Boundedness of Q-Learning Iterates for Stochastic Shortest Path Problems
MATHEMATICS OF OPERATIONS RESEARCH Vol. 38, No. 2, May 2013, pp. 209 227 ISSN 0364-765X (prin) ISSN 1526-5471 (online) hp://dx.doi.org/10.1287/moor.1120.0562 2013 INFORMS On Boundedness of Q-Learning Ieraes
More informationELE 538B: Large-Scale Optimization for Data Science. Introduction. Yuxin Chen Princeton University, Spring 2018
ELE 538B: Large-Scale Opimizaion for Daa Science Inroducion Yuxin Chen Princeon Universiy, Spring 2018 Surge of daa-inensive applicaions Widespread applicaions in large-scale daa science and learning 2.5
More informationHomogenization of random Hamilton Jacobi Bellman Equations
Probabiliy, Geomery and Inegrable Sysems MSRI Publicaions Volume 55, 28 Homogenizaion of random Hamilon Jacobi Bellman Equaions S. R. SRINIVASA VARADHAN ABSTRACT. We consider nonlinear parabolic equaions
More informationCongruent Numbers and Elliptic Curves
Congruen Numbers and Ellipic Curves Pan Yan pyan@oksaeedu Sepember 30, 014 1 Problem In an Arab manuscrip of he 10h cenury, a mahemaician saed ha he principal objec of raional righ riangles is he following
More informationMath 527 Lecture 6: Hamilton-Jacobi Equation: Explicit Formulas
Mah 527 Lecure 6: Hamilon-Jacobi Equaion: Explici Formulas Sep. 23, 2 Mehod of characerisics. We r o appl he mehod of characerisics o he Hamilon-Jacobi equaion: u +Hx, Du = in R n, u = g on R n =. 2 To
More informationDual Averaging Methods for Regularized Stochastic Learning and Online Optimization
Dual Averaging Mehods for Regularized Sochasic Learning and Online Opimizaion Lin Xiao Microsof Research Microsof Way Redmond, WA 985, USA lin.xiao@microsof.com Revised March 8, Absrac We consider regularized
More informationOptimality Conditions for Unconstrained Problems
62 CHAPTER 6 Opimaliy Condiions for Unconsrained Problems 1 Unconsrained Opimizaion 11 Exisence Consider he problem of minimizing he funcion f : R n R where f is coninuous on all of R n : P min f(x) x
More informationGMM - Generalized Method of Moments
GMM - Generalized Mehod of Momens Conens GMM esimaion, shor inroducion 2 GMM inuiion: Maching momens 2 3 General overview of GMM esimaion. 3 3. Weighing marix...........................................
More informationLecture 16: FTRL and Online Mirror Descent
Lecture 6: FTRL and Online Mirror Descent Akshay Krishnamurthy akshay@cs.umass.edu November, 07 Recap Last time we saw two online learning algorithms. First we saw the Weighted Majority algorithm, which
More informationSlide03 Historical Overview Haykin Chapter 3 (Chap 1, 3, 3rd Ed): Single-Layer Perceptrons Multiple Faces of a Single Neuron Part I: Adaptive Filter
Slide3 Haykin Chaper 3 (Chap, 3, 3rd Ed): Single-Layer Perceprons CPSC 636-6 Insrucor: Yoonsuck Choe Hisorical Overview McCulloch and Pis (943): neural neworks as compuing machines. Hebb (949): posulaed
More informationChapter 3 Boundary Value Problem
Chaper 3 Boundary Value Problem A boundary value problem (BVP) is a problem, ypically an ODE or a PDE, which has values assigned on he physical boundary of he domain in which he problem is specified. Le
More informationSOLUTIONS TO ECE 3084
SOLUTIONS TO ECE 384 PROBLEM 2.. For each sysem below, specify wheher or no i is: (i) memoryless; (ii) causal; (iii) inverible; (iv) linear; (v) ime invarian; Explain your reasoning. If he propery is no
More informationEECE 301 Signals & Systems Prof. Mark Fowler
EECE 31 Signals & Sysems Prof. Mark Fowler Noe Se #1 C-T Sysems: Convoluion Represenaion Reading Assignmen: Secion 2.6 of Kamen and Heck 1/11 Course Flow Diagram The arrows here show concepual flow beween
More informationSection 3.5 Nonhomogeneous Equations; Method of Undetermined Coefficients
Secion 3.5 Nonhomogeneous Equaions; Mehod of Undeermined Coefficiens Key Terms/Ideas: Linear Differenial operaor Nonlinear operaor Second order homogeneous DE Second order nonhomogeneous DE Soluion o homogeneous
More information3.1 More on model selection
3. More on Model selecion 3. Comparing models AIC, BIC, Adjused R squared. 3. Over Fiing problem. 3.3 Sample spliing. 3. More on model selecion crieria Ofen afer model fiing you are lef wih a handful of
More informationDemodulation of Digitally Modulated Signals
Addiional maerial for TSKS1 Digial Communicaion and TSKS2 Telecommunicaion Demodulaion of Digially Modulaed Signals Mikael Olofsson Insiuionen för sysemeknik Linköpings universie, 581 83 Linköping November
More informationMultiarmed Bandits With Limited Expert Advice
uliarmed Bandis Wih Limied Exper Advice Sayen Kale Yahoo Labs ew York sayen@yahoo-inc.com Absrac We consider he problem of minimizing regre in he seing of advice-efficien muliarmed bandis wih exper advice.
More informationTasty Coffee example
Lecure Slides for (Binary) Classificaion: Learning a Class from labeled Examples ITRODUCTIO TO Machine Learning ETHEM ALPAYDI The MIT Press, 00 (modified by dph, 0000) CHAPTER : Supervised Learning Things
More informationHW6: MRI Imaging Pulse Sequences (7 Problems for 100 pts)
HW6: MRI Imaging Pulse Sequences (7 Problems for 100 ps) GOAL The overall goal of HW6 is o beer undersand pulse sequences for MRI image reconsrucion. OBJECTIVES 1) Design a spin echo pulse sequence o image
More informationActa Mathematica Academiae Paedagogicae Nyíregyháziensis 32 (2016), ISSN
Aca Mahemaica Academiae Paedagogicae Nyíregyháziensis 3 6, 79 7 www.emis.de/journals ISSN 76-9 INTEGRAL INEQUALITIES OF HERMITE HADAMARD TYPE FOR FUNCTIONS WHOSE DERIVATIVES ARE STRONGLY α-preinvex YAN
More information4.5 Constant Acceleration
4.5 Consan Acceleraion v() v() = v 0 + a a() a a() = a v 0 Area = a (a) (b) Figure 4.8 Consan acceleraion: (a) velociy, (b) acceleraion When he x -componen of he velociy is a linear funcion (Figure 4.8(a)),
More information6. Stochastic calculus with jump processes
A) Trading sraegies (1/3) Marke wih d asses S = (S 1,, S d ) A rading sraegy can be modelled wih a vecor φ describing he quaniies invesed in each asse a each insan : φ = (φ 1,, φ d ) The value a of a porfolio
More informationUnit Root Time Series. Univariate random walk
Uni Roo ime Series Univariae random walk Consider he regression y y where ~ iid N 0, he leas squares esimae of is: ˆ yy y y yy Now wha if = If y y hen le y 0 =0 so ha y j j If ~ iid N 0, hen y ~ N 0, he
More informationMath 10B: Mock Mid II. April 13, 2016
Name: Soluions Mah 10B: Mock Mid II April 13, 016 1. ( poins) Sae, wih jusificaion, wheher he following saemens are rue or false. (a) If a 3 3 marix A saisfies A 3 A = 0, hen i canno be inverible. True.
More informationBias in Conditional and Unconditional Fixed Effects Logit Estimation: a Correction * Tom Coupé
Bias in Condiional and Uncondiional Fixed Effecs Logi Esimaion: a Correcion * Tom Coupé Economics Educaion and Research Consorium, Naional Universiy of Kyiv Mohyla Academy Address: Vul Voloska 10, 04070
More informationRandom variables. A random variable X is a function that assigns a real number, X(ζ), to each outcome ζ in the sample space of a random experiment.
Random variables Some random eperimens may yield a sample space whose elemens evens are numbers, bu some do no or mahemaical purposes, i is desirable o have numbers associaed wih he oucomes A random variable
More informationSimulation-Solving Dynamic Models ABE 5646 Week 2, Spring 2010
Simulaion-Solving Dynamic Models ABE 5646 Week 2, Spring 2010 Week Descripion Reading Maerial 2 Compuer Simulaion of Dynamic Models Finie Difference, coninuous saes, discree ime Simple Mehods Euler Trapezoid
More informationDeep Learning: Theory, Techniques & Applications - Recurrent Neural Networks -
Deep Learning: Theory, Techniques & Applicaions - Recurren Neural Neworks - Prof. Maeo Maeucci maeo.maeucci@polimi.i Deparmen of Elecronics, Informaion and Bioengineering Arificial Inelligence and Roboics
More information