Markov Decision Processes
|
|
- Stewart Daniel
- 6 years ago
- Views:
Transcription
1 Markov Decisio Processes Defiitios; Statioary policies; Value improvemet algorithm, Policy improvemet algorithm, ad liear programmig for discouted cost ad average cost criteria. Markov Decisio Processes 1
2 Markov Decisio Process Let X = {X 0, X 1, } be a system descriptio process o state space E ad let D = {D 0, D 1, } be a decisio process with actio space A. The process (X, D) is a Markov decisio process if, for j E ad = 0, 1,, + 1 =,,..., 0, 0 = + 1 =, Furthermore, for each k A, let f k be a cost vector ad P k be a oe-step trasitio probability matrix. The the cost f k (i) is icurred wheever X = i ad D = k, ad { } { } P X j X D X D P X j X D { + 1 = =, = } = k (, ) P X j X i D k P i j The problem is to determie how to choose a sequece of actios i order to miimize cost. Markov Decisio Processes 2
3 Policies A policy is a rule that specifies which actio to take at each poit i time. Let D deote the set of all policies. I geeral, the decisios specified by a policy may deped o the curret state of the system descriptio process be radomized (deped o some exteral radom evet) also deped o past states ad/or decisios A statioary policy is defied by a (determiistic) actio fuctio that assigs a actio to each state, idepedet of previous states, previous actios, ad time. Uder a statioary policy, the MDP is a Markov chai. Markov Decisio Processes 3
4 Cost Miimizatio Criteria Sice a MDP goes o idefiitely, it is likely that the total cost will be ifiite. I order to meaigfully compare policies, two criteria are commoly used: 1. Expected total discouted cost computes the preset worth of future costs usig a discout factor < 1, such that oe dollar obtaied at time = 1 has a preset value of at time = 0. Typically, if r is the rate of retur, the = 1/(1 + r). The expected total discouted cost is 0 ( ) E f D X = 2. The log ru average cost is 1 lim m m 1 m = 0 f D ( X ) Markov Decisio Processes 4
5 Optimizatio with Statioary Policies If the state space E is fiite, there exists a statioary policy that solves the problem to miimize the discouted cost: v () i mi v () i d, where v d () i E = = d f ( X ) X i 0 D 0 = d D = If every statioary policy results i a irreducible Markov chai, there exists a statioary policy that solves the problem to miimize the average cost: * 1 m 1 ϕ = mi ϕd, where ϕd = lim f ( ) 0 D X d D m = m Markov Decisio Processes 5
6 Computig Expected Discouted Costs Let X = {X 0, X 1, } be a Markov chai with oe-step trasitio probability matrix P, let f be a cost fuctio that assigs a cost to each state of the M.C., ad let (0 < < 1) be a discout factor. The the expected total discouted cost is 1 g() i = E f ( X ) ( ) ( ) 0 X0 = i = f i I P = Why? Startig from state i, the expected discouted cost ca be foud recursively as g i = f i + P g j ( ) ( ) ( ), or g= f + Pg Note that the expected discouted cost always depeds o the iitial state, while for the average cost criterio the iitial state is uimportat. j ij Markov Decisio Processes 6
7 Solutio Procedures for Discouted Costs Let v be the (vector) optimal value fuctio whose ith compoet is v ( i) = mi v d ( i) d D For each i E, v i = mi fk i + Pk i, j v j k A j E These equatios uiquely determie v. { } () () ( ) ( ) If we ca somehow obtai the values v that satisfy the above equatios, the the optimal policy is the vector a, where { } k k () arg mi () (, ) ( ) a i = f i + P i j v j k A j E arg mi is the argumet that miimizes Markov Decisio Processes 7
8 Value Iteratio for Discouted Costs Make a guess keep applyig the optimal value equatios util the fixed poit is reached. Step 1. Choose ε > 0, set = 0, let v 0 (i) = 0 for each i i E. Step 2. For each i i E, fid v +1 (i) as { } () () ( ) ( ) v 1 i = mi fk i + Pk i, j v j + k A j E { ( ) ( )} Step 3. Let δ = max v + 1 i v i i E Step 4. If δ < ε, stop with v = v +1. Otherwise, set = +1 ad retur to Step 2. Markov Decisio Processes 8
9 Policy Improvemet for Discouted Costs Start myopic, the cosider loger-term cosequeces. Step 1. Set = 0 ad let a 0 (i) = arg mi k A f k ( i) Step 2. Adopt the cost vector ad trasitio matrix: f ( i) = f ()( i) P( i, j) = P ()( i, j a ) i a i Step 3. Fid the value fuctio v= ( I P) 1 f Step 4. Re-optimize: a 1 i = arg mi fk i + Pk i, j v j { } () () ( ) ( ) + j E k A Step 5. If a +1 (i) = a (i), the stop with v = v ad a = a (i). Otherwise, set = + 1 ad retur to Step 2. Markov Decisio Processes 9
10 Liear Programmig for Discouted Costs Cosider the liear program: max u( i) i E () () + ( ) ( ) s.t. u i f i P i, j u j for each i, k k j E The optimal value of u(i) will be v (i), ad the optimal policy is idetified by the costraits that hold as equalities i the optimal solutio (slack variables equal 0). k Note: the decisio variables are urestricted i sig! Markov Decisio Processes 10
11 Log Ru Average Cost per Period For a give policy d, its log ru average cost could be foud from its cost vector f d ad oe-step trasitio probability matrix P d : First, fid the limitig probabilities by solvig The ϕ ( ) = = π π P i, j, j E; π 1 j i d j i E j E m 1 f = 0 d X d = lim = fd ( j) π j m m j E ( )( X ) So, i priciple we could simply eumerate all policies ad choose the oe with the smallest average cost ot practical if A ad E are large. Markov Decisio Processes 11
12 Recursive Equatio for Average Cost Assume that every statioary policy yields a irreducible Markov chai. There exists a scalar ϕ ad a vector h such that for all states i i E, () { () ( ) ( )} * ϕ + hi = mi f i k + P ijh, j k k A j E The scalar ϕ is the optimal average cost ad the optimal policy is foud by choosig for each state the actio that achieves the miimum o the right-had-side. The vector h is uique up to a additive costat as we will see, the differece betwee h(i) - h(j) represets the icrease i total cost from startig out i state i rather tha j. Markov Decisio Processes 12
13 Relatioships betwee Discouted Cost ad Log Ru Average Cost If a cost of c is icurred each period ad is the discout factor, the the total discouted cost is c v= c = = 0 1 Therefore, a total discouted cost v is equivalet to a * average cost of c = (1-)v per period, so lim( 1 ) v ( i) = ϕ Let v 1 be the optimal discouted cost vector, ϕ* be the optimal average cost ad h be the mystery vector from the previous slide. 1 ( ) ( ) = ( ) ( ) lim v i v j h i h j Markov Decisio Processes 13
14 Policy Improvemet for Average Costs Desigate oe state i E to be state umber 1 Step 1. Set = 0 ad let a 0 (i) = arg mi k A f k ( i) Step 2. Adopt the cost vector ad trasitio matrix: f i = f i P i, j = P i, j ( ) ()( ) ( ) ()( ) a i a i Step 3. With h(1) = 0, solve ϕ + h= f + Ph Step 4. Re-optimize: a 1 i = arg mi fk i + Pk i, j h j Step 5. If a +1 (i) = a (i), the stop with ϕ * = ϕ ad a * (i) = a (i). Otherwise, set = + 1 ad retur to Step 2. { } () () ( ) ( ) + j E k A Markov Decisio Processes 14
15 Liear Programmig for Average Costs Cosider radomized policies: let w i (k) = P{D = k X = i}. A statioary policy has w i (k) = 1 for each k=a(i) ad 0 otherwise. The decisio variables are x(i,k) = w i (k)π(i). The objective is to miimize the expected value of the average cost (expectatio take over the radomized policy): x( i k) fk ( i) ( ) ( ) k ( ) x( i, k) = 1 mi =, ϕ i E k A s.t. x j, k = x i, k P i, j for each j E k A i E k A i E k A Note that oe costrait will be redudat ad may be dropped. Markov Decisio Processes 15
Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.
Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More informationDifferentiable Convex Functions
Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for
More informationSequences, Mathematical Induction, and Recursion. CSE 2353 Discrete Computational Structures Spring 2018
CSE 353 Discrete Computatioal Structures Sprig 08 Sequeces, Mathematical Iductio, ad Recursio (Chapter 5, Epp) Note: some course slides adopted from publisher-provided material Overview May mathematical
More informationTHE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.
THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of
More informationUnit 6: Sequences and Series
AMHS Hoors Algebra 2 - Uit 6 Uit 6: Sequeces ad Series 26 Sequeces Defiitio: A sequece is a ordered list of umbers ad is formally defied as a fuctio whose domai is the set of positive itegers. It is commo
More informationOn forward improvement iteration for stopping problems
O forward improvemet iteratio for stoppig problems Mathematical Istitute, Uiversity of Kiel, Ludewig-Mey-Str. 4, D-24098 Kiel, Germay irle@math.ui-iel.de Albrecht Irle Abstract. We cosider the optimal
More informationTCOM 501: Networking Theory & Fundamentals. Lecture 3 January 29, 2003 Prof. Yannis A. Korilis
TCOM 5: Networkig Theory & Fudametals Lecture 3 Jauary 29, 23 Prof. Yais A. Korilis 3-2 Topics Markov Chais Discrete-Time Markov Chais Calculatig Statioary Distributio Global Balace Equatios Detailed Balace
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationROLL CUTTING PROBLEMS UNDER STOCHASTIC DEMAND
Pacific-Asia Joural of Mathematics, Volume 5, No., Jauary-Jue 20 ROLL CUTTING PROBLEMS UNDER STOCHASTIC DEMAND SHAKEEL JAVAID, Z. H. BAKHSHI & M. M. KHALID ABSTRACT: I this paper, the roll cuttig problem
More information9 - Markov processes and Burt & Allison 1963 AGEC
This documet was geerated at 4:51 PM o Wedesday, October 09, 2013 Copyright 2013 Richard T. Woodward 9 - Markov processes ad Burt & Alliso 1963 AGEC 637-2013 I. What is a Markov Chai? A Markov chai is
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationIntroduction to Optimization Techniques. How to Solve Equations
Itroductio to Optimizatio Techiques How to Solve Equatios Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually
More informationChapter 9: Numerical Differentiation
178 Chapter 9: Numerical Differetiatio Numerical Differetiatio Formulatio of equatios for physical problems ofte ivolve derivatives (rate-of-chage quatities, such as velocity ad acceleratio). Numerical
More informationECE-S352 Introduction to Digital Signal Processing Lecture 3A Direct Solution of Difference Equations
ECE-S352 Itroductio to Digital Sigal Processig Lecture 3A Direct Solutio of Differece Equatios Discrete Time Systems Described by Differece Equatios Uit impulse (sample) respose h() of a DT system allows
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationA New Solution Method for the Finite-Horizon Discrete-Time EOQ Problem
This is the Pre-Published Versio. A New Solutio Method for the Fiite-Horizo Discrete-Time EOQ Problem Chug-Lu Li Departmet of Logistics The Hog Kog Polytechic Uiversity Hug Hom, Kowloo, Hog Kog Phoe: +852-2766-7410
More informationOptimization Methods MIT 2.098/6.255/ Final exam
Optimizatio Methods MIT 2.098/6.255/15.093 Fial exam Date Give: December 19th, 2006 P1. [30 pts] Classify the followig statemets as true or false. All aswers must be well-justified, either through a short
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationw (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.
2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For
More informationsubject to A 1 x + A 2 y b x j 0, j = 1,,n 1 y j = 0 or 1, j = 1,,n 2
Additioal Brach ad Boud Algorithms 0-1 Mixed-Iteger Liear Programmig The brach ad boud algorithm described i the previous sectios ca be used to solve virtually all optimizatio problems cotaiig iteger variables,
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationVector Quantization: a Limiting Case of EM
. Itroductio & defiitios Assume that you are give a data set X = { x j }, j { 2,,, }, of d -dimesioal vectors. The vector quatizatio (VQ) problem requires that we fid a set of prototype vectors Z = { z
More informationFind a formula for the exponential function whose graph is given , 1 2,16 1, 6
Math 4 Activity (Due by EOC Apr. ) Graph the followig epoetial fuctios by modifyig the graph of f. Fid the rage of each fuctio.. g. g. g 4. g. g 6. g Fid a formula for the epoetial fuctio whose graph is
More informationThe Method of Least Squares. To understand least squares fitting of data.
The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve
More informationRandom Models. Tusheng Zhang. February 14, 2013
Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the
More informationApproximate Dynamic Programming by Linear Programming for Stochastic Scheduling
Approximate Dyamic Programmig by Liear Programmig for Stochastic Schedulig Mohamed Mostagir Nelso Uha 1 Itroductio I stochastic schedulig, we wat to allocate a limited amout of resources to a set of jobs
More informationThe Maximum-Likelihood Decoding Performance of Error-Correcting Codes
The Maximum-Lielihood Decodig Performace of Error-Correctig Codes Hery D. Pfister ECE Departmet Texas A&M Uiversity August 27th, 2007 (rev. 0) November 2st, 203 (rev. ) Performace of Codes. Notatio X,
More informationPC5215 Numerical Recipes with Applications - Review Problems
PC55 Numerical Recipes with Applicatios - Review Problems Give the IEEE 754 sigle precisio bit patter (biary or he format) of the followig umbers: 0 0 05 00 0 00 Note that it has 8 bits for the epoet,
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationClassification of problem & problem solving strategies. classification of time complexities (linear, logarithmic etc)
Classificatio of problem & problem solvig strategies classificatio of time complexities (liear, arithmic etc) Problem subdivisio Divide ad Coquer strategy. Asymptotic otatios, lower boud ad upper boud:
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationOptimization Methods: Linear Programming Applications Assignment Problem 1. Module 4 Lecture Notes 3. Assignment Problem
Optimizatio Methods: Liear Programmig Applicatios Assigmet Problem Itroductio Module 4 Lecture Notes 3 Assigmet Problem I the previous lecture, we discussed about oe of the bech mark problems called trasportatio
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More information2.4 - Sequences and Series
2.4 - Sequeces ad Series Sequeces A sequece is a ordered list of elemets. Defiitio 1 A sequece is a fuctio from a subset of the set of itegers (usually either the set 80, 1, 2, 3,... < or the set 81, 2,
More informationLecture 7: October 18, 2017
Iformatio ad Codig Theory Autum 207 Lecturer: Madhur Tulsiai Lecture 7: October 8, 207 Biary hypothesis testig I this lecture, we apply the tools developed i the past few lectures to uderstad the problem
More informationCHAPTER 10 INFINITE SEQUENCES AND SERIES
CHAPTER 10 INFINITE SEQUENCES AND SERIES 10.1 Sequeces 10.2 Ifiite Series 10.3 The Itegral Tests 10.4 Compariso Tests 10.5 The Ratio ad Root Tests 10.6 Alteratig Series: Absolute ad Coditioal Covergece
More informationMassachusetts Institute of Technology
6.0/6.3: Probabilistic Systems Aalysis (Fall 00) Problem Set 8: Solutios. (a) We cosider a Markov chai with states 0,,, 3,, 5, where state i idicates that there are i shoes available at the frot door i
More informationMath 312 Lecture Notes One Dimensional Maps
Math 312 Lecture Notes Oe Dimesioal Maps Warre Weckesser Departmet of Mathematics Colgate Uiversity 21-23 February 25 A Example We begi with the simplest model of populatio growth. Suppose, for example,
More informationIntroduction to Machine Learning DIS10
CS 189 Fall 017 Itroductio to Machie Learig DIS10 1 Fu with Lagrage Multipliers (a) Miimize the fuctio such that f (x,y) = x + y x + y = 3. Solutio: The Lagragia is: L(x,y,λ) = x + y + λ(x + y 3) Takig
More informationChapter 2: Numerical Methods
Chapter : Numerical Methods. Some Numerical Methods for st Order ODEs I this sectio, a summar of essetial features of umerical methods related to solutios of ordiar differetial equatios is give. I geeral,
More informationA NEW APPROACH TO SOLVE AN UNBALANCED ASSIGNMENT PROBLEM
A NEW APPROACH TO SOLVE AN UNBALANCED ASSIGNMENT PROBLEM *Kore B. G. Departmet Of Statistics, Balwat College, VITA - 415 311, Dist.: Sagli (M. S.). Idia *Author for Correspodece ABSTRACT I this paper I
More informationRecurrence Relations
Recurrece Relatios Aalysis of recursive algorithms, such as: it factorial (it ) { if (==0) retur ; else retur ( * factorial(-)); } Let t be the umber of multiplicatios eeded to calculate factorial(). The
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationGeometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT
OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca
More informationDesign and Analysis of Algorithms
Desig ad Aalysis of Algorithms Probabilistic aalysis ad Radomized algorithms Referece: CLRS Chapter 5 Topics: Hirig problem Idicatio radom variables Radomized algorithms Huo Hogwei 1 The hirig problem
More informationCHAPTER 1 SEQUENCES AND INFINITE SERIES
CHAPTER SEQUENCES AND INFINITE SERIES SEQUENCES AND INFINITE SERIES (0 meetigs) Sequeces ad limit of a sequece Mootoic ad bouded sequece Ifiite series of costat terms Ifiite series of positive terms Alteratig
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More information(b) What is the probability that a particle reaches the upper boundary n before the lower boundary m?
MATH 529 The Boudary Problem The drukard s walk (or boudary problem) is oe of the most famous problems i the theory of radom walks. Oe versio of the problem is described as follows: Suppose a particle
More informationDynamic Programming. Sequence Of Decisions
Dyamic Programmig Sequece of decisios. Problem state. Priciple of optimality. Dyamic Programmig Recurrece Equatios. Solutio of recurrece equatios. Sequece Of Decisios As i the greedy method, the solutio
More informationDynamic Programming. Sequence Of Decisions. 0/1 Knapsack Problem. Sequence Of Decisions
Dyamic Programmig Sequece Of Decisios Sequece of decisios. Problem state. Priciple of optimality. Dyamic Programmig Recurrece Equatios. Solutio of recurrece equatios. As i the greedy method, the solutio
More information2 Markov Chain Monte Carlo Sampling
22 Part I. Markov Chais ad Stochastic Samplig Figure 10: Hard-core colourig of a lattice. 2 Markov Chai Mote Carlo Samplig We ow itroduce Markov chai Mote Carlo (MCMC) samplig, which is a extremely importat
More informationNYU Center for Data Science: DS-GA 1003 Machine Learning and Computational Statistics (Spring 2018)
NYU Ceter for Data Sciece: DS-GA 003 Machie Learig ad Computatioal Statistics (Sprig 208) Brett Berstei, David Roseberg, Be Jakubowski Jauary 20, 208 Istructios: Followig most lab ad lecture sectios, we
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationInformation Theory and Statistics Lecture 4: Lempel-Ziv code
Iformatio Theory ad Statistics Lecture 4: Lempel-Ziv code Łukasz Dębowski ldebowsk@ipipa.waw.pl Ph. D. Programme 203/204 Etropy rate is the limitig compressio rate Theorem For a statioary process (X i)
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationNotes on iteration and Newton s method. Iteration
Notes o iteratio ad Newto s method Iteratio Iteratio meas doig somethig over ad over. I our cotet, a iteratio is a sequece of umbers, vectors, fuctios, etc. geerated by a iteratio rule of the type 1 f
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationTopics. Homework Problems. MATH 301 Introduction to Analysis Chapter Four Sequences. 1. Definition of convergence of sequences.
MATH 301 Itroductio to Aalysis Chapter Four Sequeces Topics 1. Defiitio of covergece of sequeces. 2. Fidig ad provig the limit of sequeces. 3. Bouded covergece theorem: Theorem 4.1.8. 4. Theorems 4.1.13
More informationConstraint Satisfaction. Algorithm Design (3) Constraint Satisfaction and Optimization. Formalization of Constraint Satisfaction Problems
Algorithm Desig (3) Costrait Satisfactio ad Optimizatio Taashi Chiayama School of Egieerig The Uiversity of Toyo Costrait Satisfactio Fidig a way that satisfies give coditios e.g. How to obtai eough credits
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More information5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.
40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig
More information15.081J/6.251J Introduction to Mathematical Programming. Lecture 21: Primal Barrier Interior Point Algorithm
508J/65J Itroductio to Mathematical Programmig Lecture : Primal Barrier Iterior Poit Algorithm Outlie Barrier Methods Slide The Cetral Path 3 Approximatig the Cetral Path 4 The Primal Barrier Algorithm
More informationDynamic Policy Programming with Function Approximation: Supplementary Material
Dyamic Policy Programmig with Fuctio pproximatio: Supplemetary Material Mohammad Gheshlaghi zar Radboud Uiversity Nijmege Geert Grooteplei Noord 21 6525 EZ Nijmege Netherlads m.azar@sciece.ru.l Viceç Gómez
More informationSequences. A Sequence is a list of numbers written in order.
Sequeces A Sequece is a list of umbers writte i order. {a, a 2, a 3,... } The sequece may be ifiite. The th term of the sequece is the th umber o the list. O the list above a = st term, a 2 = 2 d term,
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationRoger Apéry's proof that zeta(3) is irrational
Cliff Bott cliffbott@hotmail.com 11 October 2011 Roger Apéry's proof that zeta(3) is irratioal Roger Apéry developed a method for searchig for cotiued fractio represetatios of umbers that have a form such
More informationis also known as the general term of the sequence
Lesso : Sequeces ad Series Outlie Objectives: I ca determie whether a sequece has a patter. I ca determie whether a sequece ca be geeralized to fid a formula for the geeral term i the sequece. I ca determie
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 3
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture 3 Tolstikhi Ilya Abstract I this lecture we will prove the VC-boud, which provides a high-probability excess risk boud for the ERM algorithm whe
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationMathematical Notation Math Finite Mathematics
Mathematical Notatio Math 60 - Fiite Mathematics Use Word or WordPerfect to recreate the followig documets. Each article is worth 0 poits ad should be emailed to the istructor at james@richlad.edu. If
More information( ) = p and P( i = b) = q.
MATH 540 Radom Walks Part 1 A radom walk X is special stochastic process that measures the height (or value) of a particle that radomly moves upward or dowward certai fixed amouts o each uit icremet of
More informationAlternating Series. 1 n 0 2 n n THEOREM 9.14 Alternating Series Test Let a n > 0. The alternating series. 1 n a n.
0_0905.qxd //0 :7 PM Page SECTION 9.5 Alteratig Series Sectio 9.5 Alteratig Series Use the Alteratig Series Test to determie whether a ifiite series coverges. Use the Alteratig Series Remaider to approximate
More informationRecursive Algorithms. Recurrences. Recursive Algorithms Analysis
Recursive Algorithms Recurreces Computer Sciece & Egieerig 35: Discrete Mathematics Christopher M Bourke cbourke@cseuledu A recursive algorithm is oe i which objects are defied i terms of other objects
More informationThe target reliability and design working life
Safety ad Security Egieerig IV 161 The target reliability ad desig workig life M. Holický Kloker Istitute, CTU i Prague, Czech Republic Abstract Desig workig life ad target reliability levels recommeded
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationChapter 11 Output Analysis for a Single Model. Banks, Carson, Nelson & Nicol Discrete-Event System Simulation
Chapter Output Aalysis for a Sigle Model Baks, Carso, Nelso & Nicol Discrete-Evet System Simulatio Error Estimatio If {,, } are ot statistically idepedet, the S / is a biased estimator of the true variace.
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationCS321. Numerical Analysis and Computing
CS Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 September 8 5 What is the Root May physical system ca
More informationDe Moivre s Theorem - ALL
De Moivre s Theorem - ALL. Let x ad y be real umbers, ad be oe of the complex solutios of the equatio =. Evaluate: (a) + + ; (b) ( x + y)( x + y). [6]. (a) Sice is a complex umber which satisfies = 0,.
More informationMa 530 Infinite Series I
Ma 50 Ifiite Series I Please ote that i additio to the material below this lecture icorporated material from the Visual Calculus web site. The material o sequeces is at Visual Sequeces. (To use this li
More informationLinear Programming! References! Introduction to Algorithms.! Dasgupta, Papadimitriou, Vazirani. Algorithms.! Cormen, Leiserson, Rivest, and Stein.
Liear Programmig! Refereces! Dasgupta, Papadimitriou, Vazirai. Algorithms.! Corme, Leiserso, Rivest, ad Stei. Itroductio to Algorithms.! Slack form! For each costrait i, defie a oegative slack variable
More informationMixed Acceptance Sampling Plans for Multiple Products Indexed by Cost of Inspection
Mied ace Samplig Plas for Multiple Products Ideed by Cost of Ispectio Paitoo Howyig ), Prapaisri Sudasa Na - Ayudthya ) ) Dhurakijpudit Uiversity, Faculty of Egieerig (howyig@yahoo.com) ) Kasetsart Uiversity,
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationHOMEWORK 2 SOLUTIONS
HOMEWORK SOLUTIONS CSE 55 RANDOMIZED AND APPROXIMATION ALGORITHMS 1. Questio 1. a) The larger the value of k is, the smaller the expected umber of days util we get all the coupos we eed. I fact if = k
More informationAnalysis of Algorithms. Introduction. Contents
Itroductio The focus of this module is mathematical aspects of algorithms. Our mai focus is aalysis of algorithms, which meas evaluatig efficiecy of algorithms by aalytical ad mathematical methods. We
More informationMATH 304: MIDTERM EXAM SOLUTIONS
MATH 304: MIDTERM EXAM SOLUTIONS [The problems are each worth five poits, except for problem 8, which is worth 8 poits. Thus there are 43 possible poits.] 1. Use the Euclidea algorithm to fid the greatest
More informationExpectation-Maximization Algorithm.
Expectatio-Maximizatio Algorithm. Petr Pošík Czech Techical Uiversity i Prague Faculty of Electrical Egieerig Dept. of Cyberetics MLE 2 Likelihood.........................................................................................................
More informationLinear Classifiers III
Uiversität Potsdam Istitut für Iformatik Lehrstuhl Maschielles Lere Liear Classifiers III Blaie Nelso, Tobias Scheffer Cotets Classificatio Problem Bayesia Classifier Decisio Liear Classifiers, MAP Models
More informationThe Simplex algorithm: Introductory example. The Simplex algorithm: Introductory example (2)
Discrete Mathematics for Bioiformatics WS 07/08, G. W. Klau, 23. Oktober 2007, 12:21 1 The Simplex algorithm: Itroductory example The followig itroductio to the Simplex algorithm is from the book Liear
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More information1 Duality revisited. AM 221: Advanced Optimization Spring 2016
AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationOPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE
Joural of ELECTRICAL EGIEERIG, VOL. 56, O. 7-8, 2005, 200 204 OPTIMAL PIECEWISE UIFORM VECTOR QUATIZATIO OF THE MEMORYLESS LAPLACIA SOURCE Zora H. Perić Veljo Lj. Staović Alesadra Z. Jovaović Srdja M.
More informationSupport Vector Machines and Kernel Methods
Support Vector Machies ad Kerel Methods Daiel Khashabi Fall 202 Last Update: September 26, 206 Itroductio I Support Vector Machies the goal is to fid a separator betwee data which has the largest margi,
More informationTime-Domain Representations of LTI Systems
2.1 Itroductio Objectives: 1. Impulse resposes of LTI systems 2. Liear costat-coefficiets differetial or differece equatios of LTI systems 3. Bloc diagram represetatios of LTI systems 4. State-variable
More information