Reinforcement learning
|
|
- Philomena Jordan
- 6 years ago
- Views:
Transcription
1 Lecue 3 Reinfocemen leaning Milos Hauskech milos@cs.pi.edu 539 Senno Squae Reinfocemen leaning We wan o lean he conol policy: : X A We see examples of x (bu oupus a ae no given) Insead of a we ge a feedback (einfocemen, ewad) fom a ciic quanifying how good he seleced oupu was Inpu x Leane Oupu a Reinfocemen Ciic The einfocemens may no be deeminisic Goal: find : X A wih he bes expeced einfocemens 1
2 Gambling example. Game: 3 diffeen biased coins ae ossed The coin o be ossed is seleced andomly fom he hee opions and I always see which coin I am going o play nex I make bes on head o ail and I always wage $1 If I win I ge $1, ohewise I lose my be RL model: Inpu: X a coin chosen fo he nex oss, Acion: A choice of head o ail, Reinfocemens: {1, -1} A policy : X A Example: : Coin1 head Coin ail Coin3 head Gambling example RL model: Inpu: X a coin chosen fo he nex oss, Acion: A choice of head o ail, Reinfocemens: {1, -1} A policy : Coin1 head Coin ail Coin3 head Leaning goal: find : X A maximizing fuue expeced pofis : Coin1? Coin? Coin3? 0 E ( ) a discoun faco = pesen value of money
3 Agen navigaion example. Agen navigaion in he Maze: 4 moves in compass diecions Effecs of moves ae sochasic we may wind up in ohe han inended locaion wih non-zeo pobabiliy Objecive: each he goal sae in he shoes expeced ime moves G Agen navigaion example The RL model: Inpu: X posiion of an agen Oupu: A a move Reinfocemens: R -1 fo each move +100 fo eaching he goal A policy: : X A : Posiion 1 Posiion Posiion 0 G igh igh lef moves Goal: find he policy maximizing fuue expeced ewads E ( ) 0 3
4 Objecives of RL leaning Objecive: * Find a mapping : X A Tha maximizes some combinaion of fuue einfocemens (ewads) eceived ove ime Valuaion models (quanify how good he mapping is): Finie hoizon model E ( T 0 Infinie hoizon discouned model 0 Aveage ewad T 1 lim E ( ) T T ) E ( ) Discoun faco: Time hoizon: T 0 Exploaion vs. Exploiaion The (leane) acively ineacs wih he envionmen: A he beginning he leane does no know anyhing abou he envionmen I gadually gains he expeience and leans how o eac o he envionmen Dilemma (exploaion-exploiaion): Afe some numbe of seps, should I selec he bes cuen choice (exploiaion) o y o lean moe abou he envionmen (exploaion)? Exploiaion may involve he selecion of a sub-opimal acion and peven he leaning of he opimal choice Exploaion may spend o much ime on ying bad cuenly subopimal acions 4
5 Effecs of acions on he envionmen Effec of acions on he envionmen (nex inpu x o be seen) No effec, he disibuion ove possible x is fixed; acion consequences (ewads) ae seen immediaely, Ohewise, disibuion of x can change; he ewads elaed o he acion can be seen wih some delay. Leads o wo foms of einfocemen leaning: Leaning wih immediae ewads Gambling example Leaning wih delayed ewads Agen navigaion example; move choices affec he sae of he envionmen (posiion changes), a big ewad a he goal sae is delayed RL wih immediae ewads Game: 3 diffeen biased coins ae ossed The coin o be ossed is seleced andomly fom he hee opions and I always see which coin I am going o play nex I make bes on head o ail and I always wage $1 If I win I ge $1, ohewise I lose my be RL model: Inpu: X a coin chosen fo he nex oss Acion: A head o ail be Reinfocemens: {1, -1} Leaning goal: find : X A maximizing he fuue expeced pofis ove ime 0 E ( ) a discoun faco = pesen value of money 5
6 Expeced ewad 0 RL wih immediae ewads E ( ) - a discoun faco = pesen value of money Immediae ewad case: Rewad fo he choice becomes available immediaely Ou choice does no affec envionmen and hus fuue ewads 0 E ( ) E ( ) E ( ) E (, 1, Expeced one sep ewad fo inpu x and he choice a : R ( x, a ) 1 Rewads fo evey sep )... RL wih immediae ewads Immediae ewad case: Rewad fo he choice a becomes available immediaely Expeced ewad fo he inpu x and choice a: R ( x, a ) Fo he gambling poblem i can be defined as: R ( x, a ) ( a, x ) P ( j x, a i ) i j j j- a hidden oucome of he coin oss Recall he definiion of he expeced loss Expeced one sep ewad fo a saegy : X A R ( ) R ( ) R ( x, ( x )) P ( x ) x is he expeced ewad fo i, 1,
7 Expeced ewad RL wih immediae ewads Opimizing he expeced ewad : max E( 0 E ( ) E ( 0 ) E ( 1 ) E ( 0 ) max 0 E( ) max 0 )... R( ) max R( )( 0 ) ( 0 ) max R( ) max R ( ) max R ( x, ( x)) P ( x) x Opimal saegy: * : X A * ( x ) ag max R ( x, a ) a x P ( x)[ max ( x ) R ( x, ( x))] RL wih immediae ewads We know ha * ( x) ag max R( x, a Poblem: In he RL famewok we do no know R ( x, a ) The expeced ewad fo pefoming acion a a inpu x How o ge R ( x, a )? 7
8 RL wih immediae ewads Poblem: In he RL famewok we do no know R ( x, a ) The expeced ewad fo pefoming acion a a inpu x Soluion: Fo each inpu x y diffeen acions a Esimae R ( x, a ) using he aveage of obseved ewads ~ R ( x, a ) 1 N x, a, ~ Acion choice ( x) ag max R ( x, a Accuacy of he esimae: saisics (Hoeffding s bound) ~ N x, a P R ( x, a ) R ( x, a ) exp ( max min ) Numbe of samples: ( max min ) 1 N x, a ln N x i 1 a x, a i RL wih immediae ewads On-line (sochasic appoximaion) An alenaive way o esimae R ( x, a ) Idea: choose acion a fo inpu x and obseve a ewad Updae an esimae R ~ ( x, a ) (1 ) R ~ ( x, a ) Convegence popey: The appoximaion conveges in he limi fo an appopiae leaning ae schedule. Assume: ( n ( x, a )) - is a leaning ae fo nh ial of (x, pai Then he convege is assued if: i 1 1. ( i ). x, a i 1 (i) x a, - a leaning ae 8
9 Exploaion vs. Exploiaion In he RL famewok he (leane) acively ineacs wih he envionmen. ~ A any poin in ime i has an esimae of R ( x, fo any inpu acion pai Dilemma: Should he leane use he cuen bes choice of acion (exploiaion) ˆ ( x) ag max R ~ ( x, a A O choose ohe acion a and fuhe impove is esimae (exploaion) Diffeen exploaion/exploiaion saegies exis Exploaion vs. Exploiaion Unifom exploaion Choose he cuen bes choice ~ wih pobabiliy 1 ˆ ( x) ag max R ( x, a A All ohe choices ae seleced wih a unifom pobabiliy A 1 Bolzman exploaion The acion is chosen andomly bu popoionally o is cuen expeced ewad esimae exp R ~ ( x, / T p( a x) ~ exp R ( x, a' ) / T a ' A T is empeaue paamee. Wha does i do? 9
Reinforcement learning
CS 75 Mchine Lening Lecue b einfocemen lening Milos Huskech milos@cs.pi.edu 539 Senno Sque einfocemen lening We wn o len conol policy: : X A We see emples of bu oupus e no given Insed of we ge feedbck
More informationChapter 21. Reinforcement Learning. The Reinforcement Learning Agent
CSE 47 Chaper Reinforcemen Learning The Reinforcemen Learning Agen Agen Sae u Reward r Acion a Enironmen CSE AI Faculy Why reinforcemen learning Programming an agen o drie a car or fly a helicoper is ery
More informationCS 188: Artificial Intelligence Fall Probabilistic Models
CS 188: Aificial Inelligence Fall 2007 Lecue 15: Bayes Nes 10/18/2007 Dan Klein UC Bekeley Pobabilisic Models A pobabilisic model is a join disibuion ove a se of vaiables Given a join disibuion, we can
More informationCSE/NB 528 Lecture 14: Reinforcement Learning (Chapter 9)
CSE/NB 528 Lecure 14: Reinforcemen Learning Chaper 9 Image from hp://clasdean.la.asu.edu/news/images/ubep2001/neuron3.jpg Lecure figures are from Dayan & Abbo s book hp://people.brandeis.edu/~abbo/book/index.hml
More informationRepresenting Knowledge. CS 188: Artificial Intelligence Fall Properties of BNs. Independence? Reachability (the Bayes Ball) Example
C 188: Aificial Inelligence Fall 2007 epesening Knowledge ecue 17: ayes Nes III 10/25/2007 an Klein UC ekeley Popeies of Ns Independence? ayes nes: pecify complex join disibuions using simple local condiional
More informationRadiation Therapy Treatment Decision Making for Prostate Cancer Patients Based on PSA Dynamics
adiaion Theapy Teamen Decision Making fo Posae Cance Paiens Based on PSA Dynamics Maiel S. Laiei Main L. Pueman Sco Tyldesley Seen Sheche Ouline Backgound Infomaion Model Descipion Nex Seps Moiaion Tadeoffs
More informationSections 3.1 and 3.4 Exponential Functions (Growth and Decay)
Secions 3.1 and 3.4 Eponenial Funcions (Gowh and Decay) Chape 3. Secions 1 and 4 Page 1 of 5 Wha Would You Rahe Have... $1million, o double you money evey day fo 31 days saing wih 1cen? Day Cens Day Cens
More informationLow-complexity Algorithms for MIMO Multiplexing Systems
Low-complexiy Algoihms fo MIMO Muliplexing Sysems Ouline Inoducion QRD-M M algoihm Algoihm I: : o educe he numbe of suviving pahs. Algoihm II: : o educe he numbe of candidaes fo each ansmied signal. :
More informationCombinatorial Approach to M/M/1 Queues. Using Hypergeometric Functions
Inenaional Mahemaical Foum, Vol 8, 03, no 0, 463-47 HIKARI Ld, wwwm-hikaicom Combinaoial Appoach o M/M/ Queues Using Hypegeomeic Funcions Jagdish Saan and Kamal Nain Depamen of Saisics, Univesiy of Delhi,
More informationZürich. ETH Master Course: L Autonomous Mobile Robots Localization II
Roland Siegwar Margaria Chli Paul Furgale Marco Huer Marin Rufli Davide Scaramuzza ETH Maser Course: 151-0854-00L Auonomous Mobile Robos Localizaion II ACT and SEE For all do, (predicion updae / ACT),
More informationProbabilistic Models. CS 188: Artificial Intelligence Fall Independence. Example: Independence. Example: Independence? Conditional Independence
C 188: Aificial Inelligence Fall 2007 obabilisic Models A pobabilisic model is a join disibuion ove a se of vaiables Lecue 15: Bayes Nes 10/18/2007 Given a join disibuion, we can eason abou unobseved vaiables
More informationCSE/NB 528 Lecture 14: From Supervised to Reinforcement Learning (Chapter 9) R. Rao, 528: Lecture 14
CSE/NB 58 Lecure 14: From Supervised o Reinforcemen Learning Chaper 9 1 Recall from las ime: Sigmoid Neworks Oupu v T g w u g wiui w Inpu nodes u = u 1 u u 3 T i Sigmoid oupu funcion: 1 g a 1 a e 1 ga
More informationSTUDY OF THE STRESS-STRENGTH RELIABILITY AMONG THE PARAMETERS OF GENERALIZED INVERSE WEIBULL DISTRIBUTION
Inenaional Jounal of Science, Technology & Managemen Volume No 04, Special Issue No. 0, Mach 205 ISSN (online): 2394-537 STUDY OF THE STRESS-STRENGTH RELIABILITY AMONG THE PARAMETERS OF GENERALIZED INVERSE
More information20. Applications of the Genetic-Drift Model
0. Applicaions of he Geneic-Drif Model 1) Deermining he probabiliy of forming any paricular combinaion of genoypes in he nex generaion: Example: If he parenal allele frequencies are p 0 = 0.35 and q 0
More informationAn random variable is a quantity that assumes different values with certain probabilities.
Probabiliy The probabiliy PrA) of an even A is a number in [, ] ha represens how likely A is o occur. The larger he value of PrA), he more likely he even is o occur. PrA) means he even mus occur. PrA)
More informationPresentation Overview
Acion Refinemen in Reinforcemen Learning by Probabiliy Smoohing By Thomas G. Dieerich & Didac Busques Speaer: Kai Xu Presenaion Overview Bacground The Probabiliy Smoohing Mehod Experimenal Sudy of Acion
More informationComputer Propagation Analysis Tools
Compue Popagaion Analysis Tools. Compue Popagaion Analysis Tools Inoducion By now you ae pobably geing he idea ha pedicing eceived signal sengh is a eally impoan as in he design of a wieless communicaion
More informationGeneral Non-Arbitrage Model. I. Partial Differential Equation for Pricing A. Traded Underlying Security
1 Geneal Non-Abiage Model I. Paial Diffeenial Equaion fo Picing A. aded Undelying Secuiy 1. Dynamics of he Asse Given by: a. ds = µ (S, )d + σ (S, )dz b. he asse can be eihe a sock, o a cuency, an index,
More informationPHYS PRACTICE EXAM 2
PHYS 1800 PRACTICE EXAM Pa I Muliple Choice Quesions [ ps each] Diecions: Cicle he one alenaive ha bes complees he saemen o answes he quesion. Unless ohewise saed, assume ideal condiions (no ai esisance,
More informationRelative and Circular Motion
Relaie and Cicula Moion a) Relaie moion b) Cenipeal acceleaion Mechanics Lecue 3 Slide 1 Mechanics Lecue 3 Slide 2 Time on Video Pelecue Looks like mosly eeyone hee has iewed enie pelecue GOOD! Thank you
More informationLecture-V Stochastic Processes and the Basic Term-Structure Equation 1 Stochastic Processes Any variable whose value changes over time in an uncertain
Lecue-V Sochasic Pocesses and he Basic Tem-Sucue Equaion 1 Sochasic Pocesses Any vaiable whose value changes ove ime in an unceain way is called a Sochasic Pocess. Sochasic Pocesses can be classied as
More informationToday - Lecture 13. Today s lecture continue with rotations, torque, Note that chapters 11, 12, 13 all involve rotations
Today - Lecue 13 Today s lecue coninue wih oaions, oque, Noe ha chapes 11, 1, 13 all inole oaions slide 1 eiew Roaions Chapes 11 & 1 Viewed fom aboe (+z) Roaional, o angula elociy, gies angenial elociy
More informationLecture 18: Kinetics of Phase Growth in a Two-component System: general kinetics analysis based on the dilute-solution approximation
Lecue 8: Kineics of Phase Gowh in a Two-componen Sysem: geneal kineics analysis based on he dilue-soluion appoximaion Today s opics: In he las Lecues, we leaned hee diffeen ways o descibe he diffusion
More informationRisk tolerance and optimal portfolio choice
Risk oleance and opimal pofolio choice Maek Musiela BNP Paibas London Copoae and Invesmen Join wok wih T. Zaiphopoulou (UT usin) Invesmens and fowad uiliies Pepin 6 Backwad and fowad dynamic uiliies and
More informationSMT 2014 Calculus Test Solutions February 15, 2014 = 3 5 = 15.
SMT Calculus Tes Soluions February 5,. Le f() = and le g() =. Compue f ()g (). Answer: 5 Soluion: We noe ha f () = and g () = 6. Then f ()g () =. Plugging in = we ge f ()g () = 6 = 3 5 = 5.. There is a
More informationMATHEMATICAL FOUNDATIONS FOR APPROXIMATING PARTICLE BEHAVIOUR AT RADIUS OF THE PLANCK LENGTH
Fundamenal Jounal of Mahemaical Phsics Vol 3 Issue 013 Pages 55-6 Published online a hp://wwwfdincom/ MATHEMATICAL FOUNDATIONS FOR APPROXIMATING PARTICLE BEHAVIOUR AT RADIUS OF THE PLANCK LENGTH Univesias
More informationThe sudden release of a large amount of energy E into a background fluid of density
10 Poin explosion The sudden elease of a lage amoun of enegy E ino a backgound fluid of densiy ceaes a song explosion, chaaceized by a song shock wave (a blas wave ) emanaing fom he poin whee he enegy
More informationPolicy regimes Theory
Advanced Moneary Theory and Policy EPOS 2012/13 Policy regimes Theory Giovanni Di Barolomeo giovanni.dibarolomeo@uniroma1.i The moneary policy regime The simple model: x = - s (i - p e ) + x e + e D p
More information, on the power of the transmitter P t fed to it, and on the distance R between the antenna and the observation point as. r r t
Lecue 6: Fiis Tansmission Equaion and Rada Range Equaion (Fiis equaion. Maximum ange of a wieless link. Rada coss secion. Rada equaion. Maximum ange of a ada. 1. Fiis ansmission equaion Fiis ansmission
More informationDistributed Search Systems with Self-Adaptive Organizational Setups
Inenaional Jounal of Ineacive Mulimedia and Aificial Inelligence, Vol. 4, Nº4 Disibued Seach Sysems wih Self-Adapive Oganizaional Seups Fiedeike Wall Univesiae Klagenfu, Depamen of Conolling and Saegic
More informationKalman Filter: an instance of Bayes Filter. Kalman Filter: an instance of Bayes Filter. Kalman Filter. Linear dynamics with Gaussian noise
COM47 Inoducion o Roboics and Inelligen ysems he alman File alman File: an insance of Bayes File alman File: an insance of Bayes File Linea dynamics wih Gaussian noise alman File Linea dynamics wih Gaussian
More informationComparing Means: t-tests for One Sample & Two Related Samples
Comparing Means: -Tess for One Sample & Two Relaed Samples Using he z-tes: Assumpions -Tess for One Sample & Two Relaed Samples The z-es (of a sample mean agains a populaion mean) is based on he assumpion
More informationVariance and Covariance Processes
Vaiance and Covaiance Pocesses Pakash Balachandan Depamen of Mahemaics Duke Univesiy May 26, 2008 These noes ae based on Due s Sochasic Calculus, Revuz and Yo s Coninuous Maingales and Bownian Moion, Kaazas
More informationThe Production of Polarization
Physics 36: Waves Lecue 13 3/31/211 The Poducion of Polaizaion Today we will alk abou he poducion of polaized ligh. We aleady inoduced he concep of he polaizaion of ligh, a ansvese EM wave. To biefly eview
More information1 Review of Zero-Sum Games
COS 5: heoreical Machine Learning Lecurer: Rob Schapire Lecure #23 Scribe: Eugene Brevdo April 30, 2008 Review of Zero-Sum Games Las ime we inroduced a mahemaical model for wo player zero-sum games. Any
More informationLinear Response Theory: The connection between QFT and experiments
Phys540.nb 39 3 Linear Response Theory: The connecion beween QFT and experimens 3.1. Basic conceps and ideas Q: How do we measure he conduciviy of a meal? A: we firs inroduce a weak elecric field E, and
More informationThe Global Trade and Environment Model: GTEM
The Global Tade and Envionmen Model: A pojecion of non-seady sae daa using Ineempoal GTEM Hom Pan, Vivek Tulpulé and Bian S. Fishe Ausalian Bueau of Agiculual and Resouce Economics OBJECTIVES Deive an
More informationINSTANTANEOUS VELOCITY
INSTANTANEOUS VELOCITY I claim ha ha if acceleraion is consan, hen he elociy is a linear funcion of ime and he posiion a quadraic funcion of ime. We wan o inesigae hose claims, and a he same ime, work
More informationControl Volume Derivation
School of eospace Engineeing Conol Volume -1 Copyigh 1 by Jey M. Seizman. ll ighs esee. Conol Volume Deiaion How o cone ou elaionships fo a close sysem (conol mass) o an open sysem (conol olume) Fo mass
More informationLecture 17: Kinetics of Phase Growth in a Two-component System:
Lecue 17: Kineics of Phase Gowh in a Two-componen Sysem: descipion of diffusion flux acoss he α/ ineface Today s opics Majo asks of oday s Lecue: how o deive he diffusion flux of aoms. Once an incipien
More informationThe k-filtering Applied to Wave Electric and Magnetic Field Measurements from Cluster
The -fileing pplied o Wave lecic and Magneic Field Measuemens fom Cluse Jean-Louis PINÇON and ndes TJULIN LPC-CNRS 3 av. de la Recheche Scienifique 4507 Oléans Fance jlpincon@cns-oleans.f OUTLINS The -fileing
More informationExponential and Logarithmic Equations and Properties of Logarithms. Properties. Properties. log. Exponential. Logarithmic.
Eponenial and Logaihmic Equaions and Popeies of Logaihms Popeies Eponenial a a s = a +s a /a s = a -s (a ) s = a s a b = (ab) Logaihmic log s = log + logs log/s = log - logs log s = s log log a b = loga
More informationNon-sinusoidal Signal Generators
Non-sinusoidal Signal Geneaos ecangle, iangle, saw ooh, pulse, ec. Muliibao cicuis: asable no sable saes (wo quasi-sable saes; i emains in each sae fo pedeemined imes) monosable one sable sae, one unsable
More information2-d Motion: Constant Acceleration
-d Moion: Consan Acceleaion Kinemaic Equaions o Moion (eco Fom Acceleaion eco (consan eloci eco (uncion o Posiion eco (uncion o The eloci eco and posiion eco ae a uncion o he ime. eloci eco a ime. Posiion
More informationOBJECTIVES OF TIME SERIES ANALYSIS
OBJECTIVES OF TIME SERIES ANALYSIS Undersanding he dynamic or imedependen srucure of he observaions of a single series (univariae analysis) Forecasing of fuure observaions Asceraining he leading, lagging
More informationLinear Time-invariant systems, Convolution, and Cross-correlation
Linear Time-invarian sysems, Convoluion, and Cross-correlaion (1) Linear Time-invarian (LTI) sysem A sysem akes in an inpu funcion and reurns an oupu funcion. x() T y() Inpu Sysem Oupu y() = T[x()] An
More informationUnit Root Time Series. Univariate random walk
Uni Roo ime Series Univariae random walk Consider he regression y y where ~ iid N 0, he leas squares esimae of is: ˆ yy y y yy Now wha if = If y y hen le y 0 =0 so ha y j j If ~ iid N 0, hen y ~ N 0, he
More informationTopic Astable Circuits. Recall that an astable circuit has two unstable states;
Topic 2.2. Asable Circuis. Learning Objecives: A he end o his opic you will be able o; Recall ha an asable circui has wo unsable saes; Explain he operaion o a circui based on a Schmi inverer, and esimae
More informationFinal Exam. Tuesday, December hours, 30 minutes
an Faniso ae Univesi Mihael Ba ECON 30 Fall 04 Final Exam Tuesda, Deembe 6 hous, 30 minues Name: Insuions. This is losed book, losed noes exam.. No alulaos of an kind ae allowed. 3. how all he alulaions.
More informationHomework-8(1) P8.3-1, 3, 8, 10, 17, 21, 24, 28,29 P8.4-1, 2, 5
Homework-8() P8.3-, 3, 8, 0, 7, 2, 24, 28,29 P8.4-, 2, 5 Secion 8.3: The Response of a Firs Order Circui o a Consan Inpu P 8.3- The circui shown in Figure P 8.3- is a seady sae before he swich closes a
More informationCompetitive and Cooperative Inventory Policies in a Two-Stage Supply-Chain
Compeiive and Cooperaive Invenory Policies in a Two-Sage Supply-Chain (G. P. Cachon and P. H. Zipkin) Presened by Shruivandana Sharma IOE 64, Supply Chain Managemen, Winer 2009 Universiy of Michigan, Ann
More informationMacroeconomic Theory Ph.D. Qualifying Examination Fall 2005 ANSWER EACH PART IN A SEPARATE BLUE BOOK. PART ONE: ANSWER IN BOOK 1 WEIGHT 1/3
Macroeconomic Theory Ph.D. Qualifying Examinaion Fall 2005 Comprehensive Examinaion UCLA Dep. of Economics You have 4 hours o complee he exam. There are hree pars o he exam. Answer all pars. Each par has
More informationServomechanism Design
Sevomechanism Design Sevomechanism (sevo-sysem) is a conol sysem in which he efeence () (age, Se poin) changes as ime passes. Design mehods PID Conol u () Ke P () + K I ed () + KDe () Sae Feedback u()
More informationAN EVOLUTIONARY APPROACH FOR SOLVING DIFFERENTIAL EQUATIONS
AN EVOLUTIONARY APPROACH FOR SOLVING DIFFERENTIAL EQUATIONS M. KAMESWAR RAO AND K.P. RAVINDRAN Depamen of Mechanical Engineeing, Calicu Regional Engineeing College, Keala-67 6, INDIA. Absac:- We eploe
More informationExam 3 Review (Sections Covered: , )
19 Exam Review (Secions Covered: 776 8184) 1 Adieisloadedandihasbeendeerminedhaheprobabiliydisribuionassociaedwih he experimen of rolling he die and observing which number falls uppermos is given by he
More informationSolutions Problem Set 3 Macro II (14.452)
Soluions Problem Se 3 Macro II (14.452) Francisco A. Gallego 04/27/2005 1 Q heory of invesmen in coninuous ime and no uncerainy Consider he in nie horizon model of a rm facing adjusmen coss o invesmen.
More informationDiebold, Chapter 7. Francis X. Diebold, Elements of Forecasting, 4th Edition (Mason, Ohio: Cengage Learning, 2006). Chapter 7. Characterizing Cycles
Diebold, Chaper 7 Francis X. Diebold, Elemens of Forecasing, 4h Ediion (Mason, Ohio: Cengage Learning, 006). Chaper 7. Characerizing Cycles Afer compleing his reading you should be able o: Define covariance
More informationSimulation-Solving Dynamic Models ABE 5646 Week 2, Spring 2010
Simulaion-Solving Dynamic Models ABE 5646 Week 2, Spring 2010 Week Descripion Reading Maerial 2 Compuer Simulaion of Dynamic Models Finie Difference, coninuous saes, discree ime Simple Mehods Euler Trapezoid
More informationInventory Analysis and Management. Multi-Period Stochastic Models: Optimality of (s, S) Policy for K-Convex Objective Functions
Muli-Period Sochasic Models: Opimali of (s, S) Polic for -Convex Objecive Funcions Consider a seing similar o he N-sage newsvendor problem excep ha now here is a fixed re-ordering cos (> 0) for each (re-)order.
More informationE β t log (C t ) + M t M t 1. = Y t + B t 1 P t. B t 0 (3) v t = P tc t M t Question 1. Find the FOC s for an optimum in the agent s problem.
Noes, M. Krause.. Problem Se 9: Exercise on FTPL Same model as in paper and lecure, only ha one-period govenmen bonds are replaced by consols, which are bonds ha pay one dollar forever. I has curren marke
More informationThe Brock-Mirman Stochastic Growth Model
c December 3, 208, Chrisopher D. Carroll BrockMirman The Brock-Mirman Sochasic Growh Model Brock and Mirman (972) provided he firs opimizing growh model wih unpredicable (sochasic) shocks. The social planner
More informationSystem Processes input signal (excitation) and produces output signal (response)
Signal A funcion of ime Sysem Processes inpu signal (exciaion) and produces oupu signal (response) Exciaion Inpu Sysem Oupu Response 1. Types of signals 2. Going from analog o digial world 3. An example
More informationReserves measures have an economic component eg. what could be extracted at current prices?
3.2 Non-renewable esources A. Are socks of non-renewable resources fixed? eserves measures have an economic componen eg. wha could be exraced a curren prices? - Locaion and quaniies of reserves of resources
More informationChapter 12: Velocity, acceleration, and forces
To Feel a Force Chaper Spring, Chaper : A. Saes of moion For moion on or near he surface of he earh, i is naural o measure moion wih respec o objecs fixed o he earh. The 4 hr. roaion of he earh has a measurable
More informationLecture 22 Electromagnetic Waves
Lecue Elecomagneic Waves Pogam: 1. Enegy caied by he wave (Poyning veco).. Maxwell s equaions and Bounday condiions a inefaces. 3. Maeials boundaies: eflecion and efacion. Snell s Law. Quesions you should
More informationViterbi Algorithm: Background
Vierbi Algorihm: Background Jean Mark Gawron March 24, 2014 1 The Key propery of an HMM Wha is an HMM. Formally, i has he following ingrediens: 1. a se of saes: S 2. a se of final saes: F 3. an iniial
More information!!"#"$%&#'()!"#&'(*%)+,&',-)./0)1-*23)
"#"$%&#'()"#&'(*%)+,&',-)./)1-*) #$%&'()*+,&',-.%,/)*+,-&1*#$)()5*6$+$%*,7&*-'-&1*(,-&*6&,7.$%$+*&%'(*8$&',-,%'-&1*(,-&*6&,79*(&,%: ;..,*&1$&$.$%&'()*1$$.,'&',-9*(&,%)?%*,('&5
More information( ) ( ) if t = t. It must satisfy the identity. So, bulkiness of the unit impulse (hyper)function is equal to 1. The defining characteristic is
UNIT IMPULSE RESPONSE, UNIT STEP RESPONSE, STABILITY. Uni impulse funcion (Dirac dela funcion, dela funcion) rigorously defined is no sricly a funcion, bu disribuion (or measure), precise reamen requires
More informationf(x) dx with An integral having either an infinite limit of integration or an unbounded integrand is called improper. Here are two examples dx x x 2
Impope Inegls To his poin we hve only consideed inegls f() wih he is of inegion nd b finie nd he inegnd f() bounded (nd in fc coninuous ecep possibly fo finiely mny jump disconinuiies) An inegl hving eihe
More informationENGI 4430 Advanced Calculus for Engineering Faculty of Engineering and Applied Science Problem Set 9 Solutions [Theorems of Gauss and Stokes]
ENGI 44 Avance alculus fo Engineeing Faculy of Engineeing an Applie cience Poblem e 9 oluions [Theoems of Gauss an okes]. A fla aea A is boune by he iangle whose veices ae he poins P(,, ), Q(,, ) an R(,,
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON4325 Moneary Policy Dae of exam: Tuesday, May 24, 206 Grades are given: June 4, 206 Time for exam: 2.30 p.m. 5.30 p.m. The problem se covers 5 pages
More information7 Wave Equation in Higher Dimensions
7 Wave Equaion in Highe Dimensions We now conside he iniial-value poblem fo he wave equaion in n dimensions, u c u x R n u(x, φ(x u (x, ψ(x whee u n i u x i x i. (7. 7. Mehod of Spheical Means Ref: Evans,
More informationCS 4495 Computer Vision Tracking 1- Kalman,Gaussian
CS 4495 Compuer Vision A. Bobick CS 4495 Compuer Vision - KalmanGaussian Aaron Bobick School of Ineracive Compuing CS 4495 Compuer Vision A. Bobick Adminisrivia S5 will be ou his Thurs Due Sun Nov h :55pm
More informationFinal Exam. Tuesday, December hours
San Francisco Sae Universiy Michael Bar ECON 560 Fall 03 Final Exam Tuesday, December 7 hours Name: Insrucions. This is closed book, closed noes exam.. No calculaors of any kind are allowed. 3. Show all
More informationTwo-dimensional Effects on the CSR Interaction Forces for an Energy-Chirped Bunch. Rui Li, J. Bisognano, R. Legg, and R. Bosch
Two-dimensional Effecs on he CS Ineacion Foces fo an Enegy-Chiped Bunch ui Li, J. Bisognano,. Legg, and. Bosch Ouline 1. Inoducion 2. Pevious 1D and 2D esuls fo Effecive CS Foce 3. Bunch Disibuion Vaiaion
More informationProblem Set 5. Graduate Macro II, Spring 2017 The University of Notre Dame Professor Sims
Problem Se 5 Graduae Macro II, Spring 2017 The Universiy of Nore Dame Professor Sims Insrucions: You may consul wih oher members of he class, bu please make sure o urn in your own work. Where applicable,
More informationBayes Nets. CS 188: Artificial Intelligence Spring Example: Alarm Network. Building the (Entire) Joint
C 188: Aificial Inelligence ping 2008 Bayes Nes 2/5/08, 2/7/08 Dan Klein UC Bekeley Bayes Nes A Bayes ne is an efficien encoding of a pobabilisic model of a domain Quesions we can ask: Infeence: given
More informationNotes on Kalman Filtering
Noes on Kalman Filering Brian Borchers and Rick Aser November 7, Inroducion Daa Assimilaion is he problem of merging model predicions wih acual measuremens of a sysem o produce an opimal esimae of he curren
More informationHamilton- J acobi Equation: Explicit Formulas In this lecture we try to apply the method of characteristics to the Hamilton-Jacobi equation: u t
M ah 5 2 7 Fall 2 0 0 9 L ecure 1 0 O c. 7, 2 0 0 9 Hamilon- J acobi Equaion: Explici Formulas In his lecure we ry o apply he mehod of characerisics o he Hamilon-Jacobi equaion: u + H D u, x = 0 in R n
More informationTournament selection in zeroth-level classifier systems based on. average reward reinforcement learning
ournamen selecion in zeroh-level classifier sysems based on average reward reinforcemen learning Zang Zhaoxiang, Li Zhao, Wang Junying, Dan Zhiping zxzang@gmail.com; zangzx@hus.edu.cn (Hubei Key Laboraory
More information2.7. Some common engineering functions. Introduction. Prerequisites. Learning Outcomes
Some common engineering funcions 2.7 Inroducion This secion provides a caalogue of some common funcions ofen used in Science and Engineering. These include polynomials, raional funcions, he modulus funcion
More informationMEEN 617 Handout #11 MODAL ANALYSIS OF MDOF Systems with VISCOUS DAMPING
MEEN 67 Handou # MODAL ANALYSIS OF MDOF Sysems wih VISCOS DAMPING ^ Symmeic Moion of a n-dof linea sysem is descibed by he second ode diffeenial equaions M+C+K=F whee () and F () ae n ows vecos of displacemens
More informationTwo Coupled Oscillators / Normal Modes
Lecure 3 Phys 3750 Two Coupled Oscillaors / Normal Modes Overview and Moivaion: Today we ake a small, bu significan, sep owards wave moion. We will no ye observe waves, bu his sep is imporan in is own
More informationIntroduction D P. r = constant discount rate, g = Gordon Model (1962): constant dividend growth rate.
Inroducion Gordon Model (1962): D P = r g r = consan discoun rae, g = consan dividend growh rae. If raional expecaions of fuure discoun raes and dividend growh vary over ime, so should he D/P raio. Since
More informationStatistics versus mean-field limit for Hawkes process. with Sylvain Delattre (P7)
Saisics versus mean-field limi for Hawkes process wih Sylvain Delare (P7) The model We have individuals. Z i, := number of acions of he i-h individual unil ime. Z i, jumps (is increased by 1) a rae λ i,
More informationThe general Solow model
The general Solow model Back o a closed economy In he basic Solow model: no growh in GDP per worker in seady sae This conradics he empirics for he Wesern world (sylized fac #5) In he general Solow model:
More informationInternational Journal of Pure and Applied Sciences and Technology
In. J. Pue Appl. Sci. Technol., 4 (211, pp. 23-29 Inenaional Jounal of Pue and Applied Sciences and Technology ISS 2229-617 Available online a www.ijopaasa.in eseach Pape Opizaion of he Uiliy of a Sucual
More information[ ] 0. = (2) = a q dimensional vector of observable instrumental variables that are in the information set m constituents of u
Genealized Mehods of Momens he genealized mehod momens (GMM) appoach of Hansen (98) can be hough of a geneal pocedue fo esing economics and financial models. he GMM is especially appopiae fo models ha
More informationPower of Random Processes 1/40
Power of Random Processes 40 Power of a Random Process Recall : For deerminisic signals insananeous power is For a random signal, is a random variable for each ime. hus here is no single # o associae wih
More informationLaplace Transforms. Examples. Is this equation differential? y 2 2y + 1 = 0, y 2 2y + 1 = 0, (y ) 2 2y + 1 = cos x,
Laplace Transforms Definiion. An ordinary differenial equaion is an equaion ha conains one or several derivaives of an unknown funcion which we call y and which we wan o deermine from he equaion. The equaion
More informationFishing limits and the Logistic Equation. 1
Fishing limis and he Logisic Equaion. 1 1. The Logisic Equaion. The logisic equaion is an equaion governing populaion growh for populaions in an environmen wih a limied amoun of resources (for insance,
More informationChapter Finite Difference Method for Ordinary Differential Equations
Chape 8.7 Finie Diffeence Mehod fo Odinay Diffeenial Eqaions Afe eading his chape, yo shold be able o. Undesand wha he finie diffeence mehod is and how o se i o solve poblems. Wha is he finie diffeence
More informationResearch on the Algorithm of Evaluating and Analyzing Stationary Operational Availability Based on Mission Requirement
Reseach on he Algoihm of Evaluaing and Analyzing Saionay Opeaional Availabiliy Based on ission Requiemen Wang Naichao, Jia Zhiyu, Wang Yan, ao Yilan, Depamen of Sysem Engineeing of Engineeing Technology,
More informationSophisticated Monetary Policies. Andrew Atkeson. V.V. Chari. Patrick Kehoe
Sophisicaed Moneary Policies Andrew Akeson UCLA V.V. Chari Universiy of Minnesoa Parick Kehoe Federal Reserve Bank of Minneapolis and Universiy of Minnesoa Barro, Lucas-Sokey Approach o Policy Solve Ramsey
More informationMachine Learning 4771
ony Jebara, Columbia Universiy achine Learning 4771 Insrucor: ony Jebara ony Jebara, Columbia Universiy opic 20 Hs wih Evidence H Collec H Evaluae H Disribue H Decode H Parameer Learning via JA & E ony
More informationWORK POWER AND ENERGY Consevaive foce a) A foce is said o be consevaive if he wok done by i is independen of pah followed by he body b) Wok done by a consevaive foce fo a closed pah is zeo c) Wok done
More informationLecture 2 October ε-approximation of 2-player zero-sum games
Opimizaion II Winer 009/10 Lecurer: Khaled Elbassioni Lecure Ocober 19 1 ε-approximaion of -player zero-sum games In his lecure we give a randomized ficiious play algorihm for obaining an approximae soluion
More informationTwo Popular Bayesian Estimators: Particle and Kalman Filters. McGill COMP 765 Sept 14 th, 2017
Two Popular Bayesian Esimaors: Paricle and Kalman Filers McGill COMP 765 Sep 14 h, 2017 1 1 1, dx x Bel x u x P x z P Recall: Bayes Filers,,,,,,, 1 1 1 1 u z u x P u z u x z P Bayes z = observaion u =
More informationFinite-Sample Effects on the Standardized Returns of the Tokyo Stock Exchange
Available online a www.sciencediec.com Pocedia - Social and Behavioal Sciences 65 ( 01 ) 968 973 Inenaional Congess on Inedisciplinay Business and Social Science 01 (ICIBSoS 01) Finie-Sample Effecs on
More informationr r r r r EE334 Electromagnetic Theory I Todd Kaiser
334 lecoagneic Theoy I Todd Kaise Maxwell s quaions: Maxwell s equaions wee developed on expeienal evidence and have been found o goven all classical elecoagneic phenoena. They can be wien in diffeenial
More information