# 1 Online Learning and Regret Minimization

Size: px
Start display at page:

Transcription

1 2.997 Decision-Mking in Lrge-Scle Systems My 10 MIT, Spring 2004 Hndout #29 Lecture Note 24 1 Online Lerning nd Regret Minimiztion In this lecture, we consider the problem of sequentil decision mking in n online environment. At ech time stge, the decision mker must choose n ction nd receives cost or rewrd tht is function of its ction nd of the ction of the environment. We ssume tht nothing is known priori bout the evolution lw for the ctions of the environment, which in prticulr my depend on the ctions of the decision mker nd/or on n unobservble stte of the environment, nd be nonsttionry. The problem of online sequentil decision mking cn be cst s two-plyer repeted gme, where the environment is seen s the opponent. Although this seems restrictive, from the stndpoint of the lgorithms we study, two-plyer repeted gmes ctully encompss lrge clss of problems: how to ply clssicl two-plyer gme such s repeted prisoner s dilemm, rock-pper-scissors, or mtching pennies, ginst humn; how to ply gme ginst multiple other plyers, such s investing in the stock mrket; how to control system tht is difficult to model nd possibly nonsttionry, such s systems tht interct with other systems nd/or humns. Formlly, we describe our problem s follows. At ech time stge t, the decision mker must choose n ction t tking vlues in finite set A, with A K, nd receives rewrd R( t, b t ) tht is function of its ction nd of the ction b t of the environment, tking vlues in (possibly infinite) set B. We ssume tht nothing is known bout the evolution lw for the ction of the environment b t, which in prticulr my depend on the ctions of the decision mker nd/or on n unobservble stte of the environment, nd be vrint in time. The decision mker s objective is, of course, to ccumulte s much rewrd s possible. However, due to the lck of ssumptions bout the opponent, certin objectives such s mximizing expected rewrd rise some controversy s to how the expecttion should be defined. A populr criterion in online lerning is regret minimiztion. Regret is defined s the difference between the rewrd tht could hve been chieved, given the choices of the opponent, nd wht ws ctully chieved. Specificlly, we define the regret of n lgorithm A t time T for the decision mker s R A (T ) mx 1 T r(, b t ) 1 T r( t, b t ). Hving no regret t time T implies tht, in retrospect, given the opponent s ctions b 1,..., b T, no single ction could hve chieved higher rewrd thn the lgorithm s sequence of ctions 1,..., T the lgorithm performs s well s the best ction. Note tht the best ction is chosen with full knowledge of the opponent s whole sequence of ctions, wheres lgorithm A must choose ction t bsed solely on the pst history b 1,..., b t 1, or on the vector of rewrds r(, b 1 ),..., r(, b t 1 ), or in some cses only on the rewrds 1

2 effectively chieved r( 1, b 1 ),..., r( t 1, b t 1 ). Nonetheless, we will see tht in ech of these cses there is reltively simple lgorithm tht chieves no regret, symptoticlly. We strt with wht we cll the full informtion cse the sitution where the opponent s ctions or, equivlently, the vector of rewrds r(, b t ). Note tht, in the full informtion cse, before choosing ction t one is ble to compute how well it ction would do reltive to the previous sequence of moves of the opponent; explicitly, we hve t 1 G (t) r(, b t ). t 1 An intuitive lgorithm could choose ction t with probbility proportionl to G (t). In the sequel, we consider the following lgorithm. Full-Informtion Algorithm: Tke η > 0. Let G (0) 0, A. For t 1, 2,..., 1. choose t with probbility P (t), where P (t) exp (ηg (t 1)) exp (ηg (t 1)). 2. For ll ctions, compute G (t) G (t 1) + r(, b t ). We cn prove the following result bout the expected rewrd chieved by the full informtion lgorithm. It follows from the theorem tht, for every T, the difference in totl rewrd chieved by the lgorithm nd by the best ction is on the order of O( T ). Therefore the lgorithm chieves no regret symptoticlly. Theorem 1 For ll b 1, b 2,..., b T nd 1, 2,..., T, generted ccording to P (t), we hve E r( t, b t ) η mx T r(, b t) ln K e η. 1 Proof: Let Then we hve K exp (ηg (t)) K 1 exp(ηg (t) exp(ηr(, b t ))) K P (t) exp (ηr (, b t )) η P (t)r(, b t ) + (e η 1 η) P (t)r(, b t ) 1 + (e η 1) P (t)r(, b t ). 2

3 It follows tht ln +1 W 1 ln +1 ( ln 1 + (e η 1) ) P (t)r(, b t ) (e η 1) P (t)r(, b t ). (1) On the other hnd, we hve Combining (1) nd (2), the theorem follows. ln W t+1 ln exp (ηg (T + 1)) W 1 K η mx G (T + 1) ln K. [ Corollry 1 If η ln 1 + 2(ln K)/T, we hve E r( t, b t ) 1.1 Prtil Informtion Cse mx r(, b t ) 2T ln K We now consider the sitution where we only observe sequence of rewrds r( 1, b 1 ),..., r( t 1, b t 1 ) before choosing ction t. The following lgorithm is slight modifiction of the full informtion lgorithm. Prtil Informtion Algorithm: Tke η > 0 nd γ (0, 1. Let G (0) 0, A. For t 1, 2, Choose ction t with probbility 2. Let Note tht P (t) (1 γ) exp (ηg (t 1)) exp (ηg + γ (t 1)) K G t (t) G t (t 1) + r( t, b t ) P (t) G (t) G (t 1), t. E P(t) [G (t) G (t 1) + r(, b t ), so tht in expected vlue the prtil informtion lgorithm performs the sme updtes in G (t) s the full informtion lgorithm. However, the probbility of choosing ech ction differs slightly, s ctions re not chosen solely bsed on their expected rewrds, but rther every ction is chosen with probbility t lest γ/k. This introduces extr explortion in the lgorithm, which is necessry in the prtil informtion cse to ensure tht ll ctions re tested often enough. The following theorem shows tht the prtil informtion lgorithm lso chieves no regret symptoticlly, nd the totl loss t time T is still on the order of O( T ). 3

4 } Theorem 2 Suppose tht η γ K {1, nd γ min K ln K (e 1)T. Then, b 1, b 2,..., b t, E r( t, b t ) mx r(, b t ) 2.63 T K ln K The proof of Theorem 2 is bsed on Theorem 1. The min difference in the results is tht the rewrd loss grows fster with the number of ctions. The previous results, s well s lower bounds on the rewrd loss, re summrized in the tble below. Full Informtion Prtil Informtion Upper Bound O( T ln K) O( T K ln K) Lower Bound Ω( T ln K) Ω( T K) It is still n open question whether the lower bound of O( T K) cn be mtched, in the prtil informtion cse. Experts Algorithms. In the previous results, we compre the performnce of the online lerning lgorithm with tht of the best ction. It is conceivble tht, in mny prcticl circumstnces, resonble decision-mking strtegies would not consist of plying single ction ll the time, but rther choose n ction (possibly with rndomiztion) bsed on the whole history. The previous nlysis cn be extended to llow for comprison of the online lerning lgorithm with ny strtegy in fixed set of strtegies. We cll ech such strtegy n expert. Experts lgorithms try to decide, bsed on the ction e t suggested by ech expert e t time t, which ction to choose next. We cn extend the full or prtil informtion lgorithms to choose mong experts in trivil wy, by keeping trck of the rewrd G e (t) ssocited with ech expert, rther thn G (t). It cn be shown tht, if there re N experts, the loss in rewrd is t most 2 e 1 T K ln N. (2) Note tht we could pply Theorem 2 directly to obtin bound on the order of O(sqrtT N log N). However, it is possible to exploit the fct tht there re only K underlying ctions to chieve the upper bound (2) insted. (2) is especilly ttrctive considering tht, in mny prcticl situtions, the number of ctions K my be reltively smll, but one my wnt consider lrge number of strtegies/experts. The notion of regret minimiztion is n interesting one nd prticulrly meningful when one must mke sequentil decisions in n environment tht is not ffected by one s own choices s is the cse, for instnce, when smll investor is trding stock in the stock mrket, nd the volume of his trnsctions is not lrge enough to ffect prices. However, in environments tht my be ffected by the decision mker, hving zero regret my not be s meningful, or even desirble, s illustrted by the following exmple. The Prisoner s Dilemm. In the single-stge Prisoner s Dilemm (PD) gme, ech plyer cn either cooperte (C) or defect (D). Defecting is better thn cooperting regrdless of wht the opponent does, but it is better for both plyers if both cooperte thn if both defect. Consider the repeted PD. One possible pyoff mtrix for the prisoner s dilemm gme is given below: Suppose the row plyer consults with set of experts, including the defecting expert, who recommends defection ll the time. Let the strtegy of the column plyer in the repeted gme be fixed. In prticulr, the column plyer my be very ptient nd coopertive, willing to wit for the row plyer to become coopertive, but eventully becoming noncoopertive if the row plyer does not seem to cooperte. Since defection is dominnt strtegy in the stge 4

5 D C D (1,1) (4,0) C (0,4) (3,3) gme, the defecting expert chieves in ech step rewrd s high s ny other expert ginst ny sequence of choices of the column plyer, so the row plyer lerns with the experts lgorithm to defect ll the time. Obviously, in retrospect, this seems to minimize regret, since for ny fixed sequence of ctions by the column plyer, constnt defection is the best response. Obviously, constnt defection is not the best response in the repeted gme ginst mny possible strtegies of the column plyer. For instnce, the row plyer would regret very much using the experts lgorithm if he were told lter tht the column plyer hd been plying strtegy such s Tit-for-Tt, which repets t time t the sme ction plyed by the row plyer t time t 1. Aginst Tit-for-Tt, the defecting expert induces defection in every stge of the gme, chieving verge rewrd equl to 1. The best strtegy ginst Tit-for-Tt is the cooperting expert, which induces coopertion in every stge nd chieves rewrd equl to 3. 5

### Reinforcement Learning

Reinforcement Lerning Tom Mitchell, Mchine Lerning, chpter 13 Outline Introduction Comprison with inductive lerning Mrkov Decision Processes: the model Optiml policy: The tsk Q Lerning: Q function Algorithm

CSE 547/Stt 548: Mchine Lerning for Big Dt Lecture Multi-Armed Bndits: Non-dptive nd Adptive Smpling Instructor: Shm Kkde 1 The (stochstic) multi-rmed bndit problem The bsic prdigm is s follows: K Independent

### CS 188 Introduction to Artificial Intelligence Fall 2018 Note 7

CS 188 Introduction to Artificil Intelligence Fll 2018 Note 7 These lecture notes re hevily bsed on notes originlly written by Nikhil Shrm. Decision Networks In the third note, we lerned bout gme trees

### 5.7 Improper Integrals

458 pplictions of definite integrls 5.7 Improper Integrls In Section 5.4, we computed the work required to lift pylod of mss m from the surfce of moon of mss nd rdius R to height H bove the surfce of the

### Chapter 0. What is the Lebesgue integral about?

Chpter 0. Wht is the Lebesgue integrl bout? The pln is to hve tutoril sheet ech week, most often on Fridy, (to be done during the clss) where you will try to get used to the ides introduced in the previous

### Reinforcement learning II

CS 1675 Introduction to Mchine Lerning Lecture 26 Reinforcement lerning II Milos Huskrecht milos@cs.pitt.edu 5329 Sennott Squre Reinforcement lerning Bsics: Input x Lerner Output Reinforcement r Critic

### Lecture 3. In this lecture, we will discuss algorithms for solving systems of linear equations.

Lecture 3 3 Solving liner equtions In this lecture we will discuss lgorithms for solving systems of liner equtions Multiplictive identity Let us restrict ourselves to considering squre mtrices since one

### Lecture 3: Equivalence Relations

Mthcmp Crsh Course Instructor: Pdric Brtlett Lecture 3: Equivlence Reltions Week 1 Mthcmp 2014 In our lst three tlks of this clss, we shift the focus of our tlks from proof techniques to proof concepts

### 1 Probability Density Functions

Lis Yn CS 9 Continuous Distributions Lecture Notes #9 July 6, 28 Bsed on chpter by Chris Piech So fr, ll rndom vribles we hve seen hve been discrete. In ll the cses we hve seen in CS 9, this ment tht our

### Math 1B, lecture 4: Error bounds for numerical methods

Mth B, lecture 4: Error bounds for numericl methods Nthn Pflueger 4 September 0 Introduction The five numericl methods descried in the previous lecture ll operte by the sme principle: they pproximte the

### How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?

XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk bout solving systems of liner equtions. These re problems tht give couple of equtions with couple of unknowns, like: 6 2 3 7 4

### Math Lecture 23

Mth 8 - Lecture 3 Dyln Zwick Fll 3 In our lst lecture we delt with solutions to the system: x = Ax where A is n n n mtrix with n distinct eigenvlues. As promised, tody we will del with the question of

### Riemann is the Mann! (But Lebesgue may besgue to differ.)

Riemnn is the Mnn! (But Lebesgue my besgue to differ.) Leo Livshits My 2, 2008 1 For finite intervls in R We hve seen in clss tht every continuous function f : [, b] R hs the property tht for every ɛ >

### A signalling model of school grades: centralized versus decentralized examinations

A signlling model of school grdes: centrlized versus decentrlized exmintions Mri De Pol nd Vincenzo Scopp Diprtimento di Economi e Sttistic, Università dell Clbri m.depol@unicl.it; v.scopp@unicl.it 1 The

### 19 Optimal behavior: Game theory

Intro. to Artificil Intelligence: Dle Schuurmns, Relu Ptrscu 1 19 Optiml behvior: Gme theory Adversril stte dynmics hve to ccount for worst cse Compute policy π : S A tht mximizes minimum rewrd Let S (,

### MAA 4212 Improper Integrals

Notes by Dvid Groisser, Copyright c 1995; revised 2002, 2009, 2014 MAA 4212 Improper Integrls The Riemnn integrl, while perfectly well-defined, is too restrictive for mny purposes; there re functions which

### Recitation 3: More Applications of the Derivative

Mth 1c TA: Pdric Brtlett Recittion 3: More Applictions of the Derivtive Week 3 Cltech 2012 1 Rndom Question Question 1 A grph consists of the following: A set V of vertices. A set E of edges where ech

### Solution for Assignment 1 : Intro to Probability and Statistics, PAC learning

Solution for Assignment 1 : Intro to Probbility nd Sttistics, PAC lerning 10-701/15-781: Mchine Lerning (Fll 004) Due: Sept. 30th 004, Thursdy, Strt of clss Question 1. Bsic Probbility ( 18 pts) 1.1 (

### Lecture 1: Introduction to integration theory and bounded variation

Lecture 1: Introduction to integrtion theory nd bounded vrition Wht is this course bout? Integrtion theory. The first question you might hve is why there is nything you need to lern bout integrtion. You

p-adic Egyptin Frctions Contents 1 Introduction 1 2 Trditionl Egyptin Frctions nd Greedy Algorithm 2 3 Set-up 3 4 p-greedy Algorithm 5 5 p-egyptin Trditionl 10 6 Conclusion 1 Introduction An Egyptin frction

### Intermediate Math Circles Wednesday, November 14, 2018 Finite Automata II. Nickolas Rollick a b b. a b 4

Intermedite Mth Circles Wednesdy, Novemer 14, 2018 Finite Automt II Nickols Rollick nrollick@uwterloo.c Regulr Lnguges Lst time, we were introduced to the ide of DFA (deterministic finite utomton), one

### Duality # Second iteration for HW problem. Recall our LP example problem we have been working on, in equality form, is given below.

Dulity #. Second itertion for HW problem Recll our LP emple problem we hve been working on, in equlity form, is given below.,,,, 8 m F which, when written in slightly different form, is 8 F Recll tht we

### Improper Integrals, and Differential Equations

Improper Integrls, nd Differentil Equtions October 22, 204 5.3 Improper Integrls Previously, we discussed how integrls correspond to res. More specificlly, we sid tht for function f(x), the region creted

### 1B40 Practical Skills

B40 Prcticl Skills Comining uncertinties from severl quntities error propgtion We usully encounter situtions where the result of n experiment is given in terms of two (or more) quntities. We then need

### The Wave Equation I. MA 436 Kurt Bryan

1 Introduction The Wve Eqution I MA 436 Kurt Bryn Consider string stretching long the x xis, of indeterminte (or even infinite!) length. We wnt to derive n eqution which models the motion of the string

### Chapter 14. Matrix Representations of Linear Transformations

Chpter 4 Mtrix Representtions of Liner Trnsformtions When considering the Het Stte Evolution, we found tht we could describe this process using multipliction by mtrix. This ws nice becuse computers cn

### Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018

Finite Automt Theory nd Forml Lnguges TMV027/DIT321 LP4 2018 Lecture 10 An Bove April 23rd 2018 Recp: Regulr Lnguges We cn convert between FA nd RE; Hence both FA nd RE ccept/generte regulr lnguges; More

### Advanced Calculus: MATH 410 Notes on Integrals and Integrability Professor David Levermore 17 October 2004

Advnced Clculus: MATH 410 Notes on Integrls nd Integrbility Professor Dvid Levermore 17 October 2004 1. Definite Integrls In this section we revisit the definite integrl tht you were introduced to when

### New data structures to reduce data size and search time

New dt structures to reduce dt size nd serch time Tsuneo Kuwbr Deprtment of Informtion Sciences, Fculty of Science, Kngw University, Hirtsuk-shi, Jpn FIT2018 1D-1, No2, pp1-4 Copyright (c)2018 by The Institute

### Lecture 1. Functional series. Pointwise and uniform convergence.

1 Introduction. Lecture 1. Functionl series. Pointwise nd uniform convergence. In this course we study mongst other things Fourier series. The Fourier series for periodic function f(x) with period 2π is

### Goals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite

Unit #8 : The Integrl Gols: Determine how to clculte the re described by function. Define the definite integrl. Eplore the reltionship between the definite integrl nd re. Eplore wys to estimte the definite

### Continuous Random Variables

STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 217 Néhémy Lim Continuous Rndom Vribles Nottion. The indictor function of set S is rel-vlued function defined by : { 1 if x S 1 S (x) if x S Suppose tht

### State space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies

Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response

### Review of Calculus, cont d

Jim Lmbers MAT 460 Fll Semester 2009-10 Lecture 3 Notes These notes correspond to Section 1.1 in the text. Review of Clculus, cont d Riemnn Sums nd the Definite Integrl There re mny cses in which some

### CMDA 4604: Intermediate Topics in Mathematical Modeling Lecture 19: Interpolation and Quadrature

CMDA 4604: Intermedite Topics in Mthemticl Modeling Lecture 19: Interpoltion nd Qudrture In this lecture we mke brief diversion into the res of interpoltion nd qudrture. Given function f C[, b], we sy

### Lecture Note 9: Orthogonal Reduction

MATH : Computtionl Methods of Liner Algebr 1 The Row Echelon Form Lecture Note 9: Orthogonl Reduction Our trget is to solve the norml eution: Xinyi Zeng Deprtment of Mthemticl Sciences, UTEP A t Ax = A

### Chapter 4 Contravariance, Covariance, and Spacetime Diagrams

Chpter 4 Contrvrince, Covrince, nd Spcetime Digrms 4. The Components of Vector in Skewed Coordintes We hve seen in Chpter 3; figure 3.9, tht in order to show inertil motion tht is consistent with the Lorentz

### UNIFORM CONVERGENCE. Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3

UNIFORM CONVERGENCE Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3 Suppose f n : Ω R or f n : Ω C is sequence of rel or complex functions, nd f n f s n in some sense. Furthermore,

### ECO 317 Economics of Uncertainty Fall Term 2007 Notes for lectures 4. Stochastic Dominance

Generl structure ECO 37 Economics of Uncertinty Fll Term 007 Notes for lectures 4. Stochstic Dominnce Here we suppose tht the consequences re welth mounts denoted by W, which cn tke on ny vlue between

### f(x) dx, If one of these two conditions is not met, we call the integral improper. Our usual definition for the value for the definite integral

Improper Integrls Every time tht we hve evluted definite integrl such s f(x) dx, we hve mde two implicit ssumptions bout the integrl:. The intervl [, b] is finite, nd. f(x) is continuous on [, b]. If one

### 2D1431 Machine Learning Lab 3: Reinforcement Learning

2D1431 Mchine Lerning Lb 3: Reinforcement Lerning Frnk Hoffmnn modified by Örjn Ekeberg December 7, 2004 1 Introduction In this lb you will lern bout dynmic progrmming nd reinforcement lerning. It is ssumed

### THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS.

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS RADON ROSBOROUGH https://intuitiveexplntionscom/picrd-lindelof-theorem/ This document is proof of the existence-uniqueness theorem

### Lecture 3 ( ) (translated and slightly adapted from lecture notes by Martin Klazar)

Lecture 3 (5.3.2018) (trnslted nd slightly dpted from lecture notes by Mrtin Klzr) Riemnn integrl Now we define precisely the concept of the re, in prticulr, the re of figure U(, b, f) under the grph of

### Problem Set 7: Monopoly and Game Theory

ECON 000 Problem Set 7: Monopoly nd Gme Theory. () The monopolist will choose the production level tht mximizes its profits: The FOC of monopolist s problem is: So, the monopolist would set the quntity

### Jim Lambers MAT 169 Fall Semester Lecture 4 Notes

Jim Lmbers MAT 169 Fll Semester 2009-10 Lecture 4 Notes These notes correspond to Section 8.2 in the text. Series Wht is Series? An infinte series, usully referred to simply s series, is n sum of ll of

### CS 188: Artificial Intelligence Spring 2007

CS 188: Artificil Intelligence Spring 2007 Lecture 3: Queue-Bsed Serch 1/23/2007 Srini Nrynn UC Berkeley Mny slides over the course dpted from Dn Klein, Sturt Russell or Andrew Moore Announcements Assignment

### c n φ n (x), 0 < x < L, (1) n=1

SECTION : Fourier Series. MATH4. In section 4, we will study method clled Seprtion of Vribles for finding exct solutions to certin clss of prtil differentil equtions (PDEs. To do this, it will be necessry

### Advanced Calculus: MATH 410 Uniform Convergence of Functions Professor David Levermore 11 December 2015

Advnced Clculus: MATH 410 Uniform Convergence of Functions Professor Dvid Levermore 11 December 2015 12. Sequences of Functions We now explore two notions of wht it mens for sequence of functions {f n

### How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?

XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk out solving systems of liner equtions. These re prolems tht give couple of equtions with couple of unknowns, like: 6= x + x 7=

### The Regulated and Riemann Integrals

Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue

### Non-Linear & Logistic Regression

Non-Liner & Logistic Regression If the sttistics re boring, then you've got the wrong numbers. Edwrd R. Tufte (Sttistics Professor, Yle University) Regression Anlyses When do we use these? PART 1: find

### Improper Integrals. Type I Improper Integrals How do we evaluate an integral such as

Improper Integrls Two different types of integrls cn qulify s improper. The first type of improper integrl (which we will refer to s Type I) involves evluting n integrl over n infinite region. In the grph

Qudrtic Forms Recll the Simon & Blume excerpt from n erlier lecture which sid tht the min tsk of clculus is to pproximte nonliner functions with liner functions. It s ctully more ccurte to sy tht we pproximte

### Best Approximation. Chapter The General Case

Chpter 4 Best Approximtion 4.1 The Generl Cse In the previous chpter, we hve seen how n interpolting polynomil cn be used s n pproximtion to given function. We now wnt to find the best pproximtion to given

### Introduction to the Calculus of Variations

Introduction to the Clculus of Vritions Jim Fischer Mrch 20, 1999 Abstrct This is self-contined pper which introduces fundmentl problem in the clculus of vritions, the problem of finding extreme vlues

### Strong Bisimulation. Overview. References. Actions Labeled transition system Transition semantics Simulation Bisimulation

Strong Bisimultion Overview Actions Lbeled trnsition system Trnsition semntics Simultion Bisimultion References Robin Milner, Communiction nd Concurrency Robin Milner, Communicting nd Mobil Systems 32

### Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus

Unit #9 : Definite Integrl Properties; Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl

### Parse trees, ambiguity, and Chomsky normal form

Prse trees, miguity, nd Chomsky norml form In this lecture we will discuss few importnt notions connected with contextfree grmmrs, including prse trees, miguity, nd specil form for context-free grmmrs

### CS 275 Automata and Formal Language Theory

CS 275 Automt nd Forml Lnguge Theory Course Notes Prt II: The Recognition Problem (II) Chpter II.5.: Properties of Context Free Grmmrs (14) Anton Setzer (Bsed on book drft by J. V. Tucker nd K. Stephenson)

### Student Activity 3: Single Factor ANOVA

MATH 40 Student Activity 3: Single Fctor ANOVA Some Bsic Concepts In designed experiment, two or more tretments, or combintions of tretments, is pplied to experimentl units The number of tretments, whether

### Coalgebra, Lecture 15: Equations for Deterministic Automata

Colger, Lecture 15: Equtions for Deterministic Automt Julin Slmnc (nd Jurrin Rot) Decemer 19, 2016 In this lecture, we will study the concept of equtions for deterministic utomt. The notes re self contined

### Infinite Geometric Series

Infinite Geometric Series Finite Geometric Series ( finite SUM) Let 0 < r < 1, nd let n be positive integer. Consider the finite sum It turns out there is simple lgebric expression tht is equivlent to

### I1 = I2 I1 = I2 + I3 I1 + I2 = I3 + I4 I 3

2 The Prllel Circuit Electric Circuits: Figure 2- elow show ttery nd multiple resistors rrnged in prllel. Ech resistor receives portion of the current from the ttery sed on its resistnce. The split is

### Math 426: Probability Final Exam Practice

Mth 46: Probbility Finl Exm Prctice. Computtionl problems 4. Let T k (n) denote the number of prtitions of the set {,..., n} into k nonempty subsets, where k n. Argue tht T k (n) kt k (n ) + T k (n ) by

### 7.2 The Definite Integral

7.2 The Definite Integrl the definite integrl In the previous section, it ws found tht if function f is continuous nd nonnegtive, then the re under the grph of f on [, b] is given by F (b) F (), where

Lecture 14: Qudrture This lecture is concerned with the evlution of integrls fx)dx 1) over finite intervl [, b] The integrnd fx) is ssumed to be rel-vlues nd smooth The pproximtion of n integrl by numericl

### Properties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives

Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn

### #6A&B Magnetic Field Mapping

#6A& Mgnetic Field Mpping Gol y performing this lb experiment, you will: 1. use mgnetic field mesurement technique bsed on Frdy s Lw (see the previous experiment),. study the mgnetic fields generted by

### State Minimization for DFAs

Stte Minimiztion for DFAs Red K & S 2.7 Do Homework 10. Consider: Stte Minimiztion 4 5 Is this miniml mchine? Step (1): Get rid of unrechle sttes. Stte Minimiztion 6, Stte is unrechle. Step (2): Get rid

### Acceptance Sampling by Attributes

Introduction Acceptnce Smpling by Attributes Acceptnce smpling is concerned with inspection nd decision mking regrding products. Three spects of smpling re importnt: o Involves rndom smpling of n entire

### Math 8 Winter 2015 Applications of Integration

Mth 8 Winter 205 Applictions of Integrtion Here re few importnt pplictions of integrtion. The pplictions you my see on n exm in this course include only the Net Chnge Theorem (which is relly just the Fundmentl

### fractions Let s Learn to

5 simple lgebric frctions corne lens pupil retin Norml vision light focused on the retin concve lens Shortsightedness (myopi) light focused in front of the retin Corrected myopi light focused on the retin

### Review of Gaussian Quadrature method

Review of Gussin Qudrture method Nsser M. Asi Spring 006 compiled on Sundy Decemer 1, 017 t 09:1 PM 1 The prolem To find numericl vlue for the integrl of rel vlued function of rel vrile over specific rnge

### LECTURE NOTE #12 PROF. ALAN YUILLE

LECTURE NOTE #12 PROF. ALAN YUILLE 1. Clustering, K-mens, nd EM Tsk: set of unlbeled dt D = {x 1,..., x n } Decompose into clsses w 1,..., w M where M is unknown. Lern clss models p(x w)) Discovery of

### Physics 202H - Introductory Quantum Physics I Homework #08 - Solutions Fall 2004 Due 5:01 PM, Monday 2004/11/15

Physics H - Introductory Quntum Physics I Homework #8 - Solutions Fll 4 Due 5:1 PM, Mondy 4/11/15 [55 points totl] Journl questions. Briefly shre your thoughts on the following questions: Of the mteril

### 1 The Lagrange interpolation formula

Notes on Qudrture 1 The Lgrnge interpoltion formul We briefly recll the Lgrnge interpoltion formul. The strting point is collection of N + 1 rel points (x 0, y 0 ), (x 1, y 1 ),..., (x N, y N ), with x

### Applying Q-Learning to Flappy Bird

Applying Q-Lerning to Flppy Bird Moritz Ebeling-Rump, Mnfred Ko, Zchry Hervieux-Moore Abstrct The field of mchine lerning is n interesting nd reltively new re of reserch in rtificil intelligence. In this

### 13: Diffusion in 2 Energy Groups

3: Diffusion in Energy Groups B. Rouben McMster University Course EP 4D3/6D3 Nucler Rector Anlysis (Rector Physics) 5 Sept.-Dec. 5 September Contents We study the diffusion eqution in two energy groups

### Arithmetic & Algebra. NCTM National Conference, 2017

NCTM Ntionl Conference, 2017 Arithmetic & Algebr Hether Dlls, UCLA Mthemtics & The Curtis Center Roger Howe, Yle Mthemtics & Texs A & M School of Eduction Relted Common Core Stndrds First instnce of vrible

### Lecture 7 notes Nodal Analysis

Lecture 7 notes Nodl Anlysis Generl Network Anlysis In mny cses you hve multiple unknowns in circuit, sy the voltges cross multiple resistors. Network nlysis is systemtic wy to generte multiple equtions

### Administrivia CSE 190: Reinforcement Learning: An Introduction

Administrivi CSE 190: Reinforcement Lerning: An Introduction Any emil sent to me bout the course should hve CSE 190 in the subject line! Chpter 4: Dynmic Progrmming Acknowledgment: A good number of these

### Heat flux and total heat

Het flux nd totl het John McCun Mrch 14, 2017 1 Introduction Yesterdy (if I remember correctly) Ms. Prsd sked me question bout the condition of insulted boundry for the 1D het eqution, nd (bsed on glnce

### Chapter 5 : Continuous Random Variables

STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 216 Néhémy Lim Chpter 5 : Continuous Rndom Vribles Nottions. N {, 1, 2,...}, set of nturl numbers (i.e. ll nonnegtive integers); N {1, 2,...}, set of ll

### 8 Laplace s Method and Local Limit Theorems

8 Lplce s Method nd Locl Limit Theorems 8. Fourier Anlysis in Higher DImensions Most of the theorems of Fourier nlysis tht we hve proved hve nturl generliztions to higher dimensions, nd these cn be proved

### The First Fundamental Theorem of Calculus. If f(x) is continuous on [a, b] and F (x) is any antiderivative. f(x) dx = F (b) F (a).

The Fundmentl Theorems of Clculus Mth 4, Section 0, Spring 009 We now know enough bout definite integrls to give precise formultions of the Fundmentl Theorems of Clculus. We will lso look t some bsic emples

### and that at t = 0 the object is at position 5. Find the position of the object at t = 2.

7.2 The Fundmentl Theorem of Clculus 49 re mny, mny problems tht pper much different on the surfce but tht turn out to be the sme s these problems, in the sense tht when we try to pproimte solutions we

### Jack Simons, Henry Eyring Scientist and Professor Chemistry Department University of Utah

1. Born-Oppenheimer pprox.- energy surfces 2. Men-field (Hrtree-Fock) theory- orbitls 3. Pros nd cons of HF- RHF, UHF 4. Beyond HF- why? 5. First, one usully does HF-how? 6. Bsis sets nd nottions 7. MPn,

### Quantum Physics II (8.05) Fall 2013 Assignment 2

Quntum Physics II (8.05) Fll 2013 Assignment 2 Msschusetts Institute of Technology Physics Deprtment Due Fridy September 20, 2013 September 13, 2013 3:00 pm Suggested Reding Continued from lst week: 1.

### W. We shall do so one by one, starting with I 1, and we shall do it greedily, trying

Vitli covers 1 Definition. A Vitli cover of set E R is set V of closed intervls with positive length so tht, for every δ > 0 nd every x E, there is some I V with λ(i ) < δ nd x I. 2 Lemm (Vitli covering)

### Riemann Sums and Riemann Integrals

Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 2013 Outline 1 Riemnn Sums 2 Riemnn Integrls 3 Properties

### Where did dynamic programming come from?

Where did dynmic progrmming come from? String lgorithms Dvid Kuchk cs302 Spring 2012 Richrd ellmn On the irth of Dynmic Progrmming Sturt Dreyfus http://www.eng.tu.c.il/~mi/cd/ or50/1526-5463-2002-50-01-0048.pdf

### Improper Integrals. The First Fundamental Theorem of Calculus, as we ve discussed in class, goes as follows:

Improper Integrls The First Fundmentl Theorem of Clculus, s we ve discussed in clss, goes s follows: If f is continuous on the intervl [, ] nd F is function for which F t = ft, then ftdt = F F. An integrl

### UNIT 1 FUNCTIONS AND THEIR INVERSES Lesson 1.4: Logarithmic Functions as Inverses Instruction

Lesson : Logrithmic Functions s Inverses Prerequisite Skills This lesson requires the use of the following skills: determining the dependent nd independent vribles in n exponentil function bsed on dt from

### Finite Automata. Informatics 2A: Lecture 3. John Longley. 22 September School of Informatics University of Edinburgh

Lnguges nd Automt Finite Automt Informtics 2A: Lecture 3 John Longley School of Informtics University of Edinburgh jrl@inf.ed.c.uk 22 September 2017 1 / 30 Lnguges nd Automt 1 Lnguges nd Automt Wht is

### Ordinary differential equations

Ordinry differentil equtions Introduction to Synthetic Biology E Nvrro A Montgud P Fernndez de Cordob JF Urchueguí Overview Introduction-Modelling Bsic concepts to understnd n ODE. Description nd properties

### For the percentage of full time students at RCC the symbols would be:

Mth 17/171 Chpter 7- ypothesis Testing with One Smple This chpter is s simple s the previous one, except it is more interesting In this chpter we will test clims concerning the sme prmeters tht we worked

### Population bottleneck : dramatic reduction of population size followed by rapid expansion,

Selection We hve defined nucleotide diversity denoted by π s the proportion of nucleotides tht differ between two rndomly chosen sequences. We hve shown tht E[π] = θ = 4 e µ where µ cn be estimted directly.

### Riemann Sums and Riemann Integrals

Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 203 Outline Riemnn Sums Riemnn Integrls Properties Abstrct