1 Review of Zero-Sum Games

Size: px
Start display at page:

Download "1 Review of Zero-Sum Games"

Transcription

1 COS 5: heoreical Machine Learning Lecurer: Rob Schapire Lecure #23 Scribe: Eugene Brevdo April 30, 2008 Review of Zero-Sum Games Las ime we inroduced a mahemaical model for wo player zero-sum games. Any such game can in principle be formulaed via is Game Marix M. We saw wo possible models for such games: deerminisic play and randomized play. In deerminisic play, players make decisions according o pure sraegies, and in randomized play, players ac according o mixed (random) sraegies. A he end of each game we have an oucome. For pure sraegy games, he oucome is a scalar. For mixed sraegy games, we can calculae he expecaion of he oucome (he expeced oucome). Deerminisic Game: Game marix M Row player (min) chooses row i (he min oucome) Column player (max) chooses column j (he max oucome) oucome = M(i, j) Randomized Game: Game marix M Row chosen according o disribuion over he rows Column chosen according o disribuion over he columns (expeced) oucome = i,j (i)m(i, j)(j) = M where for randomized games, we use he noaion M(, ) = M o denoe he expeced oucome as a funcion of he disribuions chosen by he players. here are many connecions beween game heory and machine learning, and in oday s class we will ry o bring ogeher opics from boh of hese fields and unify hem.. A Basic Inequaliy for Randomized Games Las ime we also inroduced a basic analysis for he game of Rock-aper-Scissors (R,,S). In his game, boh sides play simulaneously. Suppose he game is modified slighly, and insead play is sequenial. Mindy (he row player) goes firs, and Max (he column player) can decide wha o do. In a deerminisic seing his game is no very ineresing. Le s consider randomized play: min chooses firs

2 max chooses knowing Since Max knows ha Mindy has chosen and knows ha for a given, he oucome is M(, ), hen Max will wan o choose o maximize M(, ), giving oucome max M(, ). Since Mindy knows wha Max will do and will know he given oucome, she chooses o minimize his oucome. hus, he final oucome will be min max M(, ) where again, in his case Mindy has gone firs. he minimizing above is called he min max sraegy. Example: Consider he game where Mindy prefers rock and has chosen = (/2, /4, /4): R wih probabiliy /2 wih probabiliy /4 S wih probabiliy /4 hen Max can always choose paper and = (0,, 0). he game marix M for his game is: R S R /2 0 0 /2 S 0 /2 and hus we have M(, ) = 5/8. In fac, if Mindy chooses a which is no uniform, he payoff will always be greaer han /2. Now, if Max chooses he column disribuion firs, and Mindy chooses he o minimize he oucome, we would have an ou come of max min M(, ) where he seleced is also known as he max min sraegy. We have discussed his in oher conexs, and noe ha he player going second has he advanage. More specifically, i is sraighforward o show min max M(, ) max min M(, ). 2 Fundamenal heorem of Zero-Sum Games I urns ou ha for all zero-sum games wih finie moves, he above inequaliy can be urned ino an equaliy. hus, regardless of who goes firs, in a game of opimal players, he expeced oucome is always he same. We denoe he oucome value v, his is he Value of he game. So, v = min max M(, ) = max min M(, ). his heorem was proved by von Neumann (who was a IAS in rinceon), and is called he von Neumann min max heorem. We will aim o prove his heorem via an online learning algorihm and argumens we have seen previously. 2

3 heorem 2. (von Neumann min max heorem) For randomized zero-sum games of wo players: min max M(, ) = max min M(, ). he heorem implies ha even if Max knows Mindy s sraegy, Max canno ge a beer oucome han v; v is he bes possible value: min max sraegy such ha : M(, ) v. I also implies ha no maer wha sraegy Mindy uses, he oucome is a wors v: max min sraegy such ha : M(, ) v. and are he opimal sraegies, assuming ha he opponen is also using an opimal sraegy. hus for a wo person zero sum game agains a good opponen, your bes be is o find he min max sraegy and o always play i. his is no he end of he sory. here are many oher issues o consider in such games, usually due o limied informaion abou he game or he opponen. Here are some examples:. We don always know M. 2. M can be very large (and compuing compuaionally difficul). 3. he sraegy is only he bes when playing agains an opimal opponen; we migh do beer agains a non-opimal opponen (i.e., dumb, no mean, ec...). Bar Simpson, for example, always plays Rock insead of choosing he uniform disribuion. hus you can play aper and always bea Bar, insead of only abou /3 of he ime. As mos games are played over and over again, here is an opporuniy for learning eiher he game rules M or he opponen s sraegy, or boh, even wihou any knowledge of eiher a he beginning. Le s consider an online version of ieraions of he game: n = # rows for =,..., row (learner) chooses column (environmen) chooses learner observes M(i, ) for each row i learner suffers loss M(, ) and we define he oal loss = = M(, ). Noe ha a each ieraion, he learner is able o observe par of he game marix M. Here, M(i, ) is he oucome of M given ha Mindy chose row i and he disribuion over he columns is. Similarly M(, j) is he oucome of M given ha max chose column i and he disribuion over he rows was. If is concenraed on a paricular column, hen he learner sees ha enire column. he opposie exreme is when he learner can see only a paricular elemen of he game marix M. Such cases are more complicaed o 3

4 analyze. Now, he learner wishes o minimize he oal loss, as compared o he bes loss possible, had he learner jus chosen he bes fixed sraegy for he ieraions and suck wih i. We wan: M(, ) min M(, ) + small. = Noe ha min = M(, ) v, wih equaliy if =. his happens when he opponen is an opimal opponen, leading o =. In general his doesn have o be he case and we can do significanly beer. hus we can do almos as well as if we had known ahead of ime. 2. Muliplicaive Updaes Le s consider a simple muliplicaive weigh updae algorihm, which oupus disribuions over rows, hus elling us how o play: (i) = n i + (i) = (i)β M(i, ) Z where 0 < β < and Z is a normalizing consan. Noe ha M(i, ) is he loss for exper i. We can prove he following bound for his algorihm (which is a direc generalizaion of he weighed majoriy voe algorihm we sudied earlier): heorem 2.2 (erformance of Muliplicaive Weigh Updaes) Using he algorihm above, we have M(, ) a β min M(, ) + c β ln n = where a β and c β are funcions of β. We are no going o prove his heorem, bu we have seen similar poenial-based argumens of his sor of proof before. Corollary 2.3 I is possible o se β so ha, when dividing boh sides by o calculae he average per-round loss: ( ) ln n where = O. M(, ) min = M(, ) + Noe ha he small error erm falls off o 0 as, for n fixed. hus he average per-round loss approaches he bes possible. 4

5 In he heorem above, depends on, bu we keep his same on he righ hand side. hus, we don ge everyhing we waned; however i is sufficien o prove he min max heorem. o prove he heorem, firs assume is chosen adversarially o maximize he loss: and define = arg max M(, ) = arg max M(, j) j = =. Noe ha he average of several disribuions is also a disribuion. Now, we know ha max min M(, ) min max M(, ). We wish o show ha max min M(, ) min max M(, ). Consider he following sring of inequaliies: min max M max M () = max = min M (2) max M (3) M (4) M + (5) = = min M + (6) max M +. (7) Here, () is by definiion of he minimum, (2) is by definiion of, (3) is by convexiy: max avg avg max, (4) is by definiion of he adversary, (5) is by he bound of he online algorihm, (6) is by definiion of, (7) is because we maximize over. hus we have ha he min max max min +, wih 0 as. his proves he von Neumann min max heorem. We also learn somehing useful from he algorihm. If we skip he firs inequaliy, hen we see ha: max M(, ) v + where v = max min M(, ). hus, aking he average of he s compued during he algorihm, we ge a disribuion ha is wihin of opimal. If = 0, hen is opimal. 5 (8)

6 hus we ge as close o he maximum as we wish by running he game for more seps. is called he approximae min max sraegy. Similarly we can skip he las inequaliy and show ha is he approximae max min sraegy. As an aside, games ha are no zero-sum are much harder o analyze. In general, one wishes o find a sraegy for finding he Nash equilibrium. he sraegies we are considering are relaed o a class of sraegies ha find anoher equilibrium poin called he correlaed equilibrium. 3 Relaion o Online Learning Le us consider he following learning problem: for =,..., observe x X predic ŷ {0, } observe he rue label c(x ) (we made a misake if ŷ c(x )) Suppose ha we associae expers wih each hypohesis h H, and we wish o do almos as well as he bes hypohesis. We consider X and H o be finie ses. And we would like, similarly o wha we ve seen in he pas: # misakes # misakes of bes h + small. For he given learning problem, we can se up an equivalen game by formulaing a game marix M: M = h h 2. x x 2 where {x i }, i =,..., X are simply indices ino he se of possible insances (no he observaions ha we see in any given round of learning). Similarly we have {h j }, j =,..., H. We fill in his misake marix: { if h(x) c(x) M(h, x) = 0 oherwise and apply he game playing algorihm o he game marix. Given a paricular observaion x, he algorihm chooses a disribuion from he rows. his is a disribuion on he hypoheses. he algorihm chooses a random hypohesis according o his disribuion and applies i o he example given. For he analysis, we have: M(, X ) } {{ } E[# misakes] min M(, x ) + small = min M(h, x ) +small h }{{} # misakes of bes h which, afer all he definiions are properly plugged in, is he same bound as we achieved in he online learning model. 6

7 4 Relaion o Boosing We can consider he basic Boosing learning problem as a game beween wo players: he Boosing algorihm and he weak learning algorihm. for =,..., H = {weak hypoheses} X = raining examples he booser chooses a disribuion D over X he WL algorihm selecs h H such ha: r x D [h (x) c(x)] 2 γ. We canno urn his procedure ino a game readily using he M we derived in he las secion, because he hypohesis space is no longer over he examples. However, by flipping he game board, convering he row player ino he min player, and renormalizing so ha he resuling M values are in [0, ], we can apply he game playing algorihm o he Boosing problem. hus, we use he game marix Now M (x, h) = M = M { if h(x) = c(x) 0 oherwise Our reducion becomes, for boosing, o le D = and is he disribuion concenraed on he h given o us: all he weigh is on one paricular column. By applying he game playing algorihm, we have M (, ) = M (, h ) = r x D [h (x) = c(x)] + γ. (0) 2 And in hree seps we have he guaranees of a Boosing algorihm:. Using he bound given in corollary (2.3): 2 + γ M (, h ) min (9) M (x, h ) + () where he firs inequaliy is due o (0) applied o he individual ieraions. 2. Since he second inequaliy in () applies o he minimum over all (all rows), i applies o all rows: x : = M (x, h ) 2 + γ > /2 where he second inequaliy is rue provided is large enough (so ha < γ). Noe ha M (x, h ) is he fracion of weak hypoheses ha correcly classify x. 7

8 3. Because for any x, he fracion of hypoheses ha correcly classify x approaches a value sricly greaer han /2 we have ha: on all x. MAJ(h (x),, h (x)) = c(x) hus he game formulaion M (x, h) as given in (9) solves he problem of choosing sample weighs in he simple, non-adapive case of Boosing. 8

Online Convex Optimization Example And Follow-The-Leader

Online Convex Optimization Example And Follow-The-Leader CSE599s, Spring 2014, Online Learning Lecure 2-04/03/2014 Online Convex Opimizaion Example And Follow-The-Leader Lecurer: Brendan McMahan Scribe: Sephen Joe Jonany 1 Review of Online Convex Opimizaion

More information

Lecture 2 October ε-approximation of 2-player zero-sum games

Lecture 2 October ε-approximation of 2-player zero-sum games Opimizaion II Winer 009/10 Lecurer: Khaled Elbassioni Lecure Ocober 19 1 ε-approximaion of -player zero-sum games In his lecure we give a randomized ficiious play algorihm for obaining an approximae soluion

More information

COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture 24 Scribe: Sachin Ravi May 2, 2013

COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture 24 Scribe: Sachin Ravi May 2, 2013 COS 5: heoretical Machine Learning Lecturer: Rob Schapire Lecture 24 Scribe: Sachin Ravi May 2, 203 Review of Zero-Sum Games At the end of last lecture, we discussed a model for two player games (call

More information

Ensamble methods: Boosting

Ensamble methods: Boosting Lecure 21 Ensamble mehods: Boosing Milos Hauskrech milos@cs.pi.edu 5329 Senno Square Schedule Final exam: April 18: 1:00-2:15pm, in-class Term projecs April 23 & April 25: a 1:00-2:30pm in CS seminar room

More information

Ensamble methods: Bagging and Boosting

Ensamble methods: Bagging and Boosting Lecure 21 Ensamble mehods: Bagging and Boosing Milos Hauskrech milos@cs.pi.edu 5329 Senno Square Ensemble mehods Mixure of expers Muliple base models (classifiers, regressors), each covers a differen par

More information

Lecture 2-1 Kinematics in One Dimension Displacement, Velocity and Acceleration Everything in the world is moving. Nothing stays still.

Lecture 2-1 Kinematics in One Dimension Displacement, Velocity and Acceleration Everything in the world is moving. Nothing stays still. Lecure - Kinemaics in One Dimension Displacemen, Velociy and Acceleraion Everyhing in he world is moving. Nohing says sill. Moion occurs a all scales of he universe, saring from he moion of elecrons in

More information

Notes for Lecture 17-18

Notes for Lecture 17-18 U.C. Berkeley CS278: Compuaional Complexiy Handou N7-8 Professor Luca Trevisan April 3-8, 2008 Noes for Lecure 7-8 In hese wo lecures we prove he firs half of he PCP Theorem, he Amplificaion Lemma, up

More information

Linear Response Theory: The connection between QFT and experiments

Linear Response Theory: The connection between QFT and experiments Phys540.nb 39 3 Linear Response Theory: The connecion beween QFT and experimens 3.1. Basic conceps and ideas Q: How do we measure he conduciviy of a meal? A: we firs inroduce a weak elecric field E, and

More information

t is a basis for the solution space to this system, then the matrix having these solutions as columns, t x 1 t, x 2 t,... x n t x 2 t...

t is a basis for the solution space to this system, then the matrix having these solutions as columns, t x 1 t, x 2 t,... x n t x 2 t... Mah 228- Fri Mar 24 5.6 Marix exponenials and linear sysems: The analogy beween firs order sysems of linear differenial equaions (Chaper 5) and scalar linear differenial equaions (Chaper ) is much sronger

More information

Finish reading Chapter 2 of Spivak, rereading earlier sections as necessary. handout and fill in some missing details!

Finish reading Chapter 2 of Spivak, rereading earlier sections as necessary. handout and fill in some missing details! MAT 257, Handou 6: Ocober 7-2, 20. I. Assignmen. Finish reading Chaper 2 of Spiva, rereading earlier secions as necessary. handou and fill in some missing deails! II. Higher derivaives. Also, read his

More information

1 Widrow-Hoff Algorithm

1 Widrow-Hoff Algorithm COS 511: heoreical Machine Learning Lecurer: Rob Schapire Lecure # 18 Scribe: Shaoqing Yang April 10, 014 1 Widrow-Hoff Algorih Firs le s review he Widrow-Hoff algorih ha was covered fro las lecure: Algorih

More information

Diebold, Chapter 7. Francis X. Diebold, Elements of Forecasting, 4th Edition (Mason, Ohio: Cengage Learning, 2006). Chapter 7. Characterizing Cycles

Diebold, Chapter 7. Francis X. Diebold, Elements of Forecasting, 4th Edition (Mason, Ohio: Cengage Learning, 2006). Chapter 7. Characterizing Cycles Diebold, Chaper 7 Francis X. Diebold, Elemens of Forecasing, 4h Ediion (Mason, Ohio: Cengage Learning, 006). Chaper 7. Characerizing Cycles Afer compleing his reading you should be able o: Define covariance

More information

Chapter 7: Solving Trig Equations

Chapter 7: Solving Trig Equations Haberman MTH Secion I: The Trigonomeric Funcions Chaper 7: Solving Trig Equaions Le s sar by solving a couple of equaions ha involve he sine funcion EXAMPLE a: Solve he equaion sin( ) The inverse funcions

More information

Lecture 33: November 29

Lecture 33: November 29 36-705: Inermediae Saisics Fall 2017 Lecurer: Siva Balakrishnan Lecure 33: November 29 Today we will coninue discussing he boosrap, and hen ry o undersand why i works in a simple case. In he las lecure

More information

Learning a Class from Examples. Training set X. Class C 1. Class C of a family car. Output: Input representation: x 1 : price, x 2 : engine power

Learning a Class from Examples. Training set X. Class C 1. Class C of a family car. Output: Input representation: x 1 : price, x 2 : engine power Alpaydin Chaper, Michell Chaper 7 Alpaydin slides are in urquoise. Ehem Alpaydin, copyrigh: The MIT Press, 010. alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/ ehem/imle All oher slides are based on Michell.

More information

Online Learning Applications

Online Learning Applications Online Learning Applicaions Sepember 19, 2016 In he las lecure we saw he following guaranee for minimizing misakes wih Randomized Weighed Majoriy (RWM). Theorem 1 Le M be misakes of RWM and M i he misakes

More information

Final Spring 2007

Final Spring 2007 .615 Final Spring 7 Overview The purpose of he final exam is o calculae he MHD β limi in a high-bea oroidal okamak agains he dangerous n = 1 exernal ballooning-kink mode. Effecively, his corresponds o

More information

Notes on online convex optimization

Notes on online convex optimization Noes on online convex opimizaion Karl Sraos Online convex opimizaion (OCO) is a principled framework for online learning: OnlineConvexOpimizaion Inpu: convex se S, number of seps T For =, 2,..., T : Selec

More information

COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture #24 Scribe: Jad Bechara May 2, 2018

COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture #24 Scribe: Jad Bechara May 2, 2018 COS 5: heoretical Machine Learning Lecturer: Rob Schapire Lecture #24 Scribe: Jad Bechara May 2, 208 Review of Game heory he games we will discuss are two-player games that can be modeled by a game matrix

More information

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Course Noes for EE7C Spring 018: Convex Opimizaion and Approximaion Insrucor: Moriz Hard Email: hard+ee7c@berkeley.edu Graduae Insrucor: Max Simchowiz Email: msimchow+ee7c@berkeley.edu Ocober 15, 018 3

More information

Two Coupled Oscillators / Normal Modes

Two Coupled Oscillators / Normal Modes Lecure 3 Phys 3750 Two Coupled Oscillaors / Normal Modes Overview and Moivaion: Today we ake a small, bu significan, sep owards wave moion. We will no ye observe waves, bu his sep is imporan in is own

More information

Learning a Class from Examples. Training set X. Class C 1. Class C of a family car. Output: Input representation: x 1 : price, x 2 : engine power

Learning a Class from Examples. Training set X. Class C 1. Class C of a family car. Output: Input representation: x 1 : price, x 2 : engine power Alpaydin Chaper, Michell Chaper 7 Alpaydin slides are in urquoise. Ehem Alpaydin, copyrigh: The MIT Press, 010. alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/ ehem/imle All oher slides are based on Michell.

More information

Comparing Means: t-tests for One Sample & Two Related Samples

Comparing Means: t-tests for One Sample & Two Related Samples Comparing Means: -Tess for One Sample & Two Relaed Samples Using he z-tes: Assumpions -Tess for One Sample & Two Relaed Samples The z-es (of a sample mean agains a populaion mean) is based on he assumpion

More information

T L. t=1. Proof of Lemma 1. Using the marginal cost accounting in Equation(4) and standard arguments. t )+Π RB. t )+K 1(Q RB

T L. t=1. Proof of Lemma 1. Using the marginal cost accounting in Equation(4) and standard arguments. t )+Π RB. t )+K 1(Q RB Elecronic Companion EC.1. Proofs of Technical Lemmas and Theorems LEMMA 1. Le C(RB) be he oal cos incurred by he RB policy. Then we have, T L E[C(RB)] 3 E[Z RB ]. (EC.1) Proof of Lemma 1. Using he marginal

More information

ACE 562 Fall Lecture 5: The Simple Linear Regression Model: Sampling Properties of the Least Squares Estimators. by Professor Scott H.

ACE 562 Fall Lecture 5: The Simple Linear Regression Model: Sampling Properties of the Least Squares Estimators. by Professor Scott H. ACE 56 Fall 005 Lecure 5: he Simple Linear Regression Model: Sampling Properies of he Leas Squares Esimaors by Professor Sco H. Irwin Required Reading: Griffihs, Hill and Judge. "Inference in he Simple

More information

Inventory Analysis and Management. Multi-Period Stochastic Models: Optimality of (s, S) Policy for K-Convex Objective Functions

Inventory Analysis and Management. Multi-Period Stochastic Models: Optimality of (s, S) Policy for K-Convex Objective Functions Muli-Period Sochasic Models: Opimali of (s, S) Polic for -Convex Objecive Funcions Consider a seing similar o he N-sage newsvendor problem excep ha now here is a fixed re-ordering cos (> 0) for each (re-)order.

More information

Lecture 2 April 04, 2018

Lecture 2 April 04, 2018 Sas 300C: Theory of Saisics Spring 208 Lecure 2 April 04, 208 Prof. Emmanuel Candes Scribe: Paulo Orensein; edied by Sephen Baes, XY Han Ouline Agenda: Global esing. Needle in a Haysack Problem 2. Threshold

More information

Matrix Versions of Some Refinements of the Arithmetic-Geometric Mean Inequality

Matrix Versions of Some Refinements of the Arithmetic-Geometric Mean Inequality Marix Versions of Some Refinemens of he Arihmeic-Geomeric Mean Inequaliy Bao Qi Feng and Andrew Tonge Absrac. We esablish marix versions of refinemens due o Alzer ], Carwrigh and Field 4], and Mercer 5]

More information

Some Ramsey results for the n-cube

Some Ramsey results for the n-cube Some Ramsey resuls for he n-cube Ron Graham Universiy of California, San Diego Jozsef Solymosi Universiy of Briish Columbia, Vancouver, Canada Absrac In his noe we esablish a Ramsey-ype resul for cerain

More information

Solutions from Chapter 9.1 and 9.2

Solutions from Chapter 9.1 and 9.2 Soluions from Chaper 9 and 92 Secion 9 Problem # This basically boils down o an exercise in he chain rule from calculus We are looking for soluions of he form: u( x) = f( k x c) where k x R 3 and k is

More information

Physics 235 Chapter 2. Chapter 2 Newtonian Mechanics Single Particle

Physics 235 Chapter 2. Chapter 2 Newtonian Mechanics Single Particle Chaper 2 Newonian Mechanics Single Paricle In his Chaper we will review wha Newon s laws of mechanics ell us abou he moion of a single paricle. Newon s laws are only valid in suiable reference frames,

More information

Math 10B: Mock Mid II. April 13, 2016

Math 10B: Mock Mid II. April 13, 2016 Name: Soluions Mah 10B: Mock Mid II April 13, 016 1. ( poins) Sae, wih jusificaion, wheher he following saemens are rue or false. (a) If a 3 3 marix A saisfies A 3 A = 0, hen i canno be inverible. True.

More information

Fishing limits and the Logistic Equation. 1

Fishing limits and the Logistic Equation. 1 Fishing limis and he Logisic Equaion. 1 1. The Logisic Equaion. The logisic equaion is an equaion governing populaion growh for populaions in an environmen wih a limied amoun of resources (for insance,

More information

5. Stochastic processes (1)

5. Stochastic processes (1) Lec05.pp S-38.45 - Inroducion o Teleraffic Theory Spring 2005 Conens Basic conceps Poisson process 2 Sochasic processes () Consider some quaniy in a eleraffic (or any) sysem I ypically evolves in ime randomly

More information

Chapter 2. First Order Scalar Equations

Chapter 2. First Order Scalar Equations Chaper. Firs Order Scalar Equaions We sar our sudy of differenial equaions in he same way he pioneers in his field did. We show paricular echniques o solve paricular ypes of firs order differenial equaions.

More information

Chapter 6. Systems of First Order Linear Differential Equations

Chapter 6. Systems of First Order Linear Differential Equations Chaper 6 Sysems of Firs Order Linear Differenial Equaions We will only discuss firs order sysems However higher order sysems may be made ino firs order sysems by a rick shown below We will have a sligh

More information

23.2. Representing Periodic Functions by Fourier Series. Introduction. Prerequisites. Learning Outcomes

23.2. Representing Periodic Functions by Fourier Series. Introduction. Prerequisites. Learning Outcomes Represening Periodic Funcions by Fourier Series 3. Inroducion In his Secion we show how a periodic funcion can be expressed as a series of sines and cosines. We begin by obaining some sandard inegrals

More information

non -negative cone Population dynamics motivates the study of linear models whose coefficient matrices are non-negative or positive.

non -negative cone Population dynamics motivates the study of linear models whose coefficient matrices are non-negative or positive. LECTURE 3 Linear/Nonnegaive Marix Models x ( = Px ( A= m m marix, x= m vecor Linear sysems of difference equaions arise in several difference conexs: Linear approximaions (linearizaion Perurbaion analysis

More information

Article from. Predictive Analytics and Futurism. July 2016 Issue 13

Article from. Predictive Analytics and Futurism. July 2016 Issue 13 Aricle from Predicive Analyics and Fuurism July 6 Issue An Inroducion o Incremenal Learning By Qiang Wu and Dave Snell Machine learning provides useful ools for predicive analyics The ypical machine learning

More information

RL Lecture 7: Eligibility Traces. R. S. Sutton and A. G. Barto: Reinforcement Learning: An Introduction 1

RL Lecture 7: Eligibility Traces. R. S. Sutton and A. G. Barto: Reinforcement Learning: An Introduction 1 RL Lecure 7: Eligibiliy Traces R. S. Suon and A. G. Baro: Reinforcemen Learning: An Inroducion 1 N-sep TD Predicion Idea: Look farher ino he fuure when you do TD backup (1, 2, 3,, n seps) R. S. Suon and

More information

Online Learning, Regret Minimization, Minimax Optimality, and Correlated Equilibrium

Online Learning, Regret Minimization, Minimax Optimality, and Correlated Equilibrium Algorihm Online Learning, Regre Minimizaion, Minimax Opimaliy, and Correlaed Equilibrium High level Las ime we discussed noion of Nash equilibrium Saic concep: se of prob Disribuions (p,q, ) such ha nobody

More information

Christos Papadimitriou & Luca Trevisan November 22, 2016

Christos Papadimitriou & Luca Trevisan November 22, 2016 U.C. Bereley CS170: Algorihms Handou LN-11-22 Chrisos Papadimiriou & Luca Trevisan November 22, 2016 Sreaming algorihms In his lecure and he nex one we sudy memory-efficien algorihms ha process a sream

More information

IMPLICIT AND INVERSE FUNCTION THEOREMS PAUL SCHRIMPF 1 OCTOBER 25, 2013

IMPLICIT AND INVERSE FUNCTION THEOREMS PAUL SCHRIMPF 1 OCTOBER 25, 2013 IMPLICI AND INVERSE FUNCION HEOREMS PAUL SCHRIMPF 1 OCOBER 25, 213 UNIVERSIY OF BRIISH COLUMBIA ECONOMICS 526 We have exensively sudied how o solve sysems of linear equaions. We know how o check wheher

More information

Lecture 12: Multiple Hypothesis Testing

Lecture 12: Multiple Hypothesis Testing ECE 830 Fall 00 Saisical Signal Processing insrucor: R. Nowak, scribe: Xinjue Yu Lecure : Muliple Hypohesis Tesing Inroducion In many applicaions we consider muliple hypohesis es a he same ime. Example

More information

20. Applications of the Genetic-Drift Model

20. Applications of the Genetic-Drift Model 0. Applicaions of he Geneic-Drif Model 1) Deermining he probabiliy of forming any paricular combinaion of genoypes in he nex generaion: Example: If he parenal allele frequencies are p 0 = 0.35 and q 0

More information

GMM - Generalized Method of Moments

GMM - Generalized Method of Moments GMM - Generalized Mehod of Momens Conens GMM esimaion, shor inroducion 2 GMM inuiion: Maching momens 2 3 General overview of GMM esimaion. 3 3. Weighing marix...........................................

More information

INTRODUCTION TO MACHINE LEARNING 3RD EDITION

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN The MIT Press, 2014 Lecure Slides for INTRODUCTION TO MACHINE LEARNING 3RD EDITION alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/i2ml3e CHAPTER 2: SUPERVISED LEARNING Learning a Class

More information

23.5. Half-Range Series. Introduction. Prerequisites. Learning Outcomes

23.5. Half-Range Series. Introduction. Prerequisites. Learning Outcomes Half-Range Series 2.5 Inroducion In his Secion we address he following problem: Can we find a Fourier series expansion of a funcion defined over a finie inerval? Of course we recognise ha such a funcion

More information

Seminar 4: Hotelling 2

Seminar 4: Hotelling 2 Seminar 4: Hoelling 2 November 3, 211 1 Exercise Par 1 Iso-elasic demand A non renewable resource of a known sock S can be exraced a zero cos. Demand for he resource is of he form: D(p ) = p ε ε > A a

More information

MATH 4330/5330, Fourier Analysis Section 6, Proof of Fourier s Theorem for Pointwise Convergence

MATH 4330/5330, Fourier Analysis Section 6, Proof of Fourier s Theorem for Pointwise Convergence MATH 433/533, Fourier Analysis Secion 6, Proof of Fourier s Theorem for Poinwise Convergence Firs, some commens abou inegraing periodic funcions. If g is a periodic funcion, g(x + ) g(x) for all real x,

More information

Instructor: Barry McQuarrie Page 1 of 5

Instructor: Barry McQuarrie Page 1 of 5 Procedure for Solving radical equaions 1. Algebraically isolae one radical by iself on one side of equal sign. 2. Raise each side of he equaion o an appropriae power o remove he radical. 3. Simplify. 4.

More information

Two Popular Bayesian Estimators: Particle and Kalman Filters. McGill COMP 765 Sept 14 th, 2017

Two Popular Bayesian Estimators: Particle and Kalman Filters. McGill COMP 765 Sept 14 th, 2017 Two Popular Bayesian Esimaors: Paricle and Kalman Filers McGill COMP 765 Sep 14 h, 2017 1 1 1, dx x Bel x u x P x z P Recall: Bayes Filers,,,,,,, 1 1 1 1 u z u x P u z u x z P Bayes z = observaion u =

More information

Asymptotic Equipartition Property - Seminar 3, part 1

Asymptotic Equipartition Property - Seminar 3, part 1 Asympoic Equipariion Propery - Seminar 3, par 1 Ocober 22, 2013 Problem 1 (Calculaion of ypical se) To clarify he noion of a ypical se A (n) ε and he smalles se of high probabiliy B (n), we will calculae

More information

Supplement for Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence

Supplement for Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence Supplemen for Sochasic Convex Opimizaion: Faser Local Growh Implies Faser Global Convergence Yi Xu Qihang Lin ianbao Yang Proof of heorem heorem Suppose Assumpion holds and F (w) obeys he LGC (6) Given

More information

PENALIZED LEAST SQUARES AND PENALIZED LIKELIHOOD

PENALIZED LEAST SQUARES AND PENALIZED LIKELIHOOD PENALIZED LEAST SQUARES AND PENALIZED LIKELIHOOD HAN XIAO 1. Penalized Leas Squares Lasso solves he following opimizaion problem, ˆβ lasso = arg max β R p+1 1 N y i β 0 N x ij β j β j (1.1) for some 0.

More information

Randomized Perfect Bipartite Matching

Randomized Perfect Bipartite Matching Inenive Algorihm Lecure 24 Randomized Perfec Biparie Maching Lecurer: Daniel A. Spielman April 9, 208 24. Inroducion We explain a randomized algorihm by Ahih Goel, Michael Kapralov and Sanjeev Khanna for

More information

Supplementary Material

Supplementary Material Dynamic Global Games of Regime Change: Learning, Mulipliciy and iming of Aacks Supplemenary Maerial George-Marios Angeleos MI and NBER Chrisian Hellwig UCLA Alessandro Pavan Norhwesern Universiy Ocober

More information

3.1 More on model selection

3.1 More on model selection 3. More on Model selecion 3. Comparing models AIC, BIC, Adjused R squared. 3. Over Fiing problem. 3.3 Sample spliing. 3. More on model selecion crieria Ofen afer model fiing you are lef wih a handful of

More information

KINEMATICS IN ONE DIMENSION

KINEMATICS IN ONE DIMENSION KINEMATICS IN ONE DIMENSION PREVIEW Kinemaics is he sudy of how hings move how far (disance and displacemen), how fas (speed and velociy), and how fas ha how fas changes (acceleraion). We say ha an objec

More information

Mon Apr 9 EP 7.6 Convolutions and Laplace transforms. Announcements: Warm-up Exercise:

Mon Apr 9 EP 7.6 Convolutions and Laplace transforms. Announcements: Warm-up Exercise: Mah 225-4 Week 3 April 9-3 EP 7.6 - convoluions; 6.-6.2 - eigenvalues, eigenvecors and diagonalizabiliy; 7. - sysems of differenial equaions. Mon Apr 9 EP 7.6 Convoluions and Laplace ransforms. Announcemens:

More information

On Measuring Pro-Poor Growth. 1. On Various Ways of Measuring Pro-Poor Growth: A Short Review of the Literature

On Measuring Pro-Poor Growth. 1. On Various Ways of Measuring Pro-Poor Growth: A Short Review of the Literature On Measuring Pro-Poor Growh 1. On Various Ways of Measuring Pro-Poor Growh: A Shor eview of he Lieraure During he pas en years or so here have been various suggesions concerning he way one should check

More information

Longest Common Prefixes

Longest Common Prefixes Longes Common Prefixes The sandard ordering for srings is he lexicographical order. I is induced by an order over he alphabe. We will use he same symbols (,

More information

Guest Lectures for Dr. MacFarlane s EE3350 Part Deux

Guest Lectures for Dr. MacFarlane s EE3350 Part Deux Gues Lecures for Dr. MacFarlane s EE3350 Par Deux Michael Plane Mon., 08-30-2010 Wrie name in corner. Poin ou his is a review, so I will go faser. Remind hem o go lisen o online lecure abou geing an A

More information

Explaining Total Factor Productivity. Ulrich Kohli University of Geneva December 2015

Explaining Total Factor Productivity. Ulrich Kohli University of Geneva December 2015 Explaining Toal Facor Produciviy Ulrich Kohli Universiy of Geneva December 2015 Needed: A Theory of Toal Facor Produciviy Edward C. Presco (1998) 2 1. Inroducion Toal Facor Produciviy (TFP) has become

More information

Vehicle Arrival Models : Headway

Vehicle Arrival Models : Headway Chaper 12 Vehicle Arrival Models : Headway 12.1 Inroducion Modelling arrival of vehicle a secion of road is an imporan sep in raffic flow modelling. I has imporan applicaion in raffic flow simulaion where

More information

Math 2142 Exam 1 Review Problems. x 2 + f (0) 3! for the 3rd Taylor polynomial at x = 0. To calculate the various quantities:

Math 2142 Exam 1 Review Problems. x 2 + f (0) 3! for the 3rd Taylor polynomial at x = 0. To calculate the various quantities: Mah 4 Eam Review Problems Problem. Calculae he 3rd Taylor polynomial for arcsin a =. Soluion. Le f() = arcsin. For his problem, we use he formula f() + f () + f ()! + f () 3! for he 3rd Taylor polynomial

More information

Approximation Algorithms for Unique Games via Orthogonal Separators

Approximation Algorithms for Unique Games via Orthogonal Separators Approximaion Algorihms for Unique Games via Orhogonal Separaors Lecure noes by Konsanin Makarychev. Lecure noes are based on he papers [CMM06a, CMM06b, LM4]. Unique Games In hese lecure noes, we define

More information

Random Walk with Anti-Correlated Steps

Random Walk with Anti-Correlated Steps Random Walk wih Ani-Correlaed Seps John Noga Dirk Wagner 2 Absrac We conjecure he expeced value of random walks wih ani-correlaed seps o be exacly. We suppor his conjecure wih 2 plausibiliy argumens and

More information

SOLUTIONS TO ECE 3084

SOLUTIONS TO ECE 3084 SOLUTIONS TO ECE 384 PROBLEM 2.. For each sysem below, specify wheher or no i is: (i) memoryless; (ii) causal; (iii) inverible; (iv) linear; (v) ime invarian; Explain your reasoning. If he propery is no

More information

Online Learning with Queries

Online Learning with Queries Online Learning wih Queries Chao-Kai Chiang Chi-Jen Lu Absrac The online learning problem requires a player o ieraively choose an acion in an unknown and changing environmen. In he sandard seing of his

More information

More Digital Logic. t p output. Low-to-high and high-to-low transitions could have different t p. V in (t)

More Digital Logic. t p output. Low-to-high and high-to-low transitions could have different t p. V in (t) EECS 4 Spring 23 Lecure 2 EECS 4 Spring 23 Lecure 2 More igial Logic Gae delay and signal propagaion Clocked circui elemens (flip-flop) Wriing a word o memory Simplifying digial circuis: Karnaugh maps

More information

Online Learning with Partial Feedback. 1 Online Mirror Descent with Estimated Gradient

Online Learning with Partial Feedback. 1 Online Mirror Descent with Estimated Gradient Avance Course in Machine Learning Spring 2010 Online Learning wih Parial Feeback Hanous are joinly prepare by Shie Mannor an Shai Shalev-Shwarz In previous lecures we alke abou he general framework of

More information

Assignment 6. Tyler Shendruk December 6, 2010

Assignment 6. Tyler Shendruk December 6, 2010 Assignmen 6 Tyler Shendruk December 6, 1 1 Harden Problem 1 Le K be he coupling and h he exernal field in a 1D Ising model. From he lecures hese can be ransformed ino effecive coupling and fields K and

More information

We just finished the Erdős-Stone Theorem, and ex(n, F ) (1 1/(χ(F ) 1)) ( n

We just finished the Erdős-Stone Theorem, and ex(n, F ) (1 1/(χ(F ) 1)) ( n Lecure 3 - Kövari-Sós-Turán Theorem Jacques Versraëe jacques@ucsd.edu We jus finished he Erdős-Sone Theorem, and ex(n, F ) ( /(χ(f ) )) ( n 2). So we have asympoics when χ(f ) 3 bu no when χ(f ) = 2 i.e.

More information

Section 5: Chain Rule

Section 5: Chain Rule Chaper The Derivaive Applie Calculus 11 Secion 5: Chain Rule There is one more ype of complicae funcion ha we will wan o know how o iffereniae: composiion. The Chain Rule will le us fin he erivaive of

More information

Math Week 14 April 16-20: sections first order systems of linear differential equations; 7.4 mass-spring systems.

Math Week 14 April 16-20: sections first order systems of linear differential equations; 7.4 mass-spring systems. Mah 2250-004 Week 4 April 6-20 secions 7.-7.3 firs order sysems of linear differenial equaions; 7.4 mass-spring sysems. Mon Apr 6 7.-7.2 Sysems of differenial equaions (7.), and he vecor Calculus we need

More information

On Boundedness of Q-Learning Iterates for Stochastic Shortest Path Problems

On Boundedness of Q-Learning Iterates for Stochastic Shortest Path Problems MATHEMATICS OF OPERATIONS RESEARCH Vol. 38, No. 2, May 2013, pp. 209 227 ISSN 0364-765X (prin) ISSN 1526-5471 (online) hp://dx.doi.org/10.1287/moor.1120.0562 2013 INFORMS On Boundedness of Q-Learning Ieraes

More information

Topics in Machine Learning Theory

Topics in Machine Learning Theory Topics in Machine Learning Theory The Adversarial Muli-armed Bandi Problem, Inernal Regre, and Correlaed Equilibria Avrim Blum 10/8/14 Plan for oday Online game playing / combining exper advice bu: Wha

More information

( ) ( ) if t = t. It must satisfy the identity. So, bulkiness of the unit impulse (hyper)function is equal to 1. The defining characteristic is

( ) ( ) if t = t. It must satisfy the identity. So, bulkiness of the unit impulse (hyper)function is equal to 1. The defining characteristic is UNIT IMPULSE RESPONSE, UNIT STEP RESPONSE, STABILITY. Uni impulse funcion (Dirac dela funcion, dela funcion) rigorously defined is no sricly a funcion, bu disribuion (or measure), precise reamen requires

More information

Unit Root Time Series. Univariate random walk

Unit Root Time Series. Univariate random walk Uni Roo ime Series Univariae random walk Consider he regression y y where ~ iid N 0, he leas squares esimae of is: ˆ yy y y yy Now wha if = If y y hen le y 0 =0 so ha y j j If ~ iid N 0, hen y ~ N 0, he

More information

Designing Information Devices and Systems I Spring 2019 Lecture Notes Note 17

Designing Information Devices and Systems I Spring 2019 Lecture Notes Note 17 EES 16A Designing Informaion Devices and Sysems I Spring 019 Lecure Noes Noe 17 17.1 apaciive ouchscreen In he las noe, we saw ha a capacior consiss of wo pieces on conducive maerial separaed by a nonconducive

More information

Hamilton- J acobi Equation: Explicit Formulas In this lecture we try to apply the method of characteristics to the Hamilton-Jacobi equation: u t

Hamilton- J acobi Equation: Explicit Formulas In this lecture we try to apply the method of characteristics to the Hamilton-Jacobi equation: u t M ah 5 2 7 Fall 2 0 0 9 L ecure 1 0 O c. 7, 2 0 0 9 Hamilon- J acobi Equaion: Explici Formulas In his lecure we ry o apply he mehod of characerisics o he Hamilon-Jacobi equaion: u + H D u, x = 0 in R n

More information

Econ107 Applied Econometrics Topic 7: Multicollinearity (Studenmund, Chapter 8)

Econ107 Applied Econometrics Topic 7: Multicollinearity (Studenmund, Chapter 8) I. Definiions and Problems A. Perfec Mulicollineariy Econ7 Applied Economerics Topic 7: Mulicollineariy (Sudenmund, Chaper 8) Definiion: Perfec mulicollineariy exiss in a following K-variable regression

More information

Online Appendix for "Customer Recognition in. Experience versus Inspection Good Markets"

Online Appendix for Customer Recognition in. Experience versus Inspection Good Markets Online Appendix for "Cusomer Recogniion in Experience versus Inspecion Good Markes" Bing Jing Cheong Kong Graduae School of Business Beijing, 0078, People s Republic of China, bjing@ckgsbeducn November

More information

Zürich. ETH Master Course: L Autonomous Mobile Robots Localization II

Zürich. ETH Master Course: L Autonomous Mobile Robots Localization II Roland Siegwar Margaria Chli Paul Furgale Marco Huer Marin Rufli Davide Scaramuzza ETH Maser Course: 151-0854-00L Auonomous Mobile Robos Localizaion II ACT and SEE For all do, (predicion updae / ACT),

More information

Cash Flow Valuation Mode Lin Discrete Time

Cash Flow Valuation Mode Lin Discrete Time IOSR Journal of Mahemaics (IOSR-JM) e-issn: 2278-5728,p-ISSN: 2319-765X, 6, Issue 6 (May. - Jun. 2013), PP 35-41 Cash Flow Valuaion Mode Lin Discree Time Olayiwola. M. A. and Oni, N. O. Deparmen of Mahemaics

More information

Stochastic models and their distributions

Stochastic models and their distributions Sochasic models and heir disribuions Couning cusomers Suppose ha n cusomers arrive a a grocery a imes, say T 1,, T n, each of which akes any real number in he inerval (, ) equally likely The values T 1,,

More information

Games Against Nature

Games Against Nature Advanced Course in Machine Learning Spring 2010 Games Agains Naure Handous are joinly prepared by Shie Mannor and Shai Shalev-Shwarz In he previous lecures we alked abou expers in differen seups and analyzed

More information

EECE 301 Signals & Systems Prof. Mark Fowler

EECE 301 Signals & Systems Prof. Mark Fowler EECE 31 Signals & Sysems Prof. Mark Fowler Noe Se #1 C-T Sysems: Convoluion Represenaion Reading Assignmen: Secion 2.6 of Kamen and Heck 1/11 Course Flow Diagram The arrows here show concepual flow beween

More information

STATE-SPACE MODELLING. A mass balance across the tank gives:

STATE-SPACE MODELLING. A mass balance across the tank gives: B. Lennox and N.F. Thornhill, 9, Sae Space Modelling, IChemE Process Managemen and Conrol Subjec Group Newsleer STE-SPACE MODELLING Inroducion: Over he pas decade or so here has been an ever increasing

More information

Ensemble Confidence Estimates Posterior Probability

Ensemble Confidence Estimates Posterior Probability Ensemble Esimaes Poserior Probabiliy Michael Muhlbaier, Aposolos Topalis, and Robi Polikar Rowan Universiy, Elecrical and Compuer Engineering, Mullica Hill Rd., Glassboro, NJ 88, USA {muhlba6, opali5}@sudens.rowan.edu

More information

Expert Advice for Amateurs

Expert Advice for Amateurs Exper Advice for Amaeurs Ernes K. Lai Online Appendix - Exisence of Equilibria The analysis in his secion is performed under more general payoff funcions. Wihou aking an explici form, he payoffs of he

More information

An random variable is a quantity that assumes different values with certain probabilities.

An random variable is a quantity that assumes different values with certain probabilities. Probabiliy The probabiliy PrA) of an even A is a number in [, ] ha represens how likely A is o occur. The larger he value of PrA), he more likely he even is o occur. PrA) means he even mus occur. PrA)

More information

Stationary Distribution. Design and Analysis of Algorithms Andrei Bulatov

Stationary Distribution. Design and Analysis of Algorithms Andrei Bulatov Saionary Disribuion Design and Analysis of Algorihms Andrei Bulaov Algorihms Markov Chains 34-2 Classificaion of Saes k By P we denoe he (i,j)-enry of i, j Sae is accessible from sae if 0 for some k 0

More information

The Arcsine Distribution

The Arcsine Distribution The Arcsine Disribuion Chris H. Rycrof Ocober 6, 006 A common heme of he class has been ha he saisics of single walker are ofen very differen from hose of an ensemble of walkers. On he firs homework, we

More information

Notes on Kalman Filtering

Notes on Kalman Filtering Noes on Kalman Filering Brian Borchers and Rick Aser November 7, Inroducion Daa Assimilaion is he problem of merging model predicions wih acual measuremens of a sysem o produce an opimal esimae of he curren

More information

Homework sheet Exercises done during the lecture of March 12, 2014

Homework sheet Exercises done during the lecture of March 12, 2014 EXERCISE SESSION 2A FOR THE COURSE GÉOMÉTRIE EUCLIDIENNE, NON EUCLIDIENNE ET PROJECTIVE MATTEO TOMMASINI Homework shee 3-4 - Exercises done during he lecure of March 2, 204 Exercise 2 Is i rue ha he parameerized

More information

Sections 2.2 & 2.3 Limit of a Function and Limit Laws

Sections 2.2 & 2.3 Limit of a Function and Limit Laws Mah 80 www.imeodare.com Secions. &. Limi of a Funcion and Limi Laws In secion. we saw how is arise when we wan o find he angen o a curve or he velociy of an objec. Now we urn our aenion o is in general

More information

Lecture Notes 2. The Hilbert Space Approach to Time Series

Lecture Notes 2. The Hilbert Space Approach to Time Series Time Series Seven N. Durlauf Universiy of Wisconsin. Basic ideas Lecure Noes. The Hilber Space Approach o Time Series The Hilber space framework provides a very powerful language for discussing he relaionship

More information

Predator - Prey Model Trajectories and the nonlinear conservation law

Predator - Prey Model Trajectories and the nonlinear conservation law Predaor - Prey Model Trajecories and he nonlinear conservaion law James K. Peerson Deparmen of Biological Sciences and Deparmen of Mahemaical Sciences Clemson Universiy Ocober 28, 213 Ouline Drawing Trajecories

More information