The Winding Path to RL
|
|
- Hugo Park
- 5 years ago
- Views:
Transcription
1 Markov Deciion Procee MDP) Ron Parr ComSci 70 Deartment of Comuter Science Duke Univerity With thank to Kri Hauer for ome lide The Winding Path to RL Deciion Theory Decritive theory of otimal behavior Markov Deciion Procee Mathematical/Algorithmic realization of Deciion Theory Reinforcement Learning Alication of learning technique to challenge of MDP with numerou or unknown arameter
2 Deciion Theory Review MDP Algorithm for MDP Value Determination Otimal Policy Selection Value Iteration Policy Iteration Covered Today Swet under the rug today Utility of money aumed :) How to determine cot/utilitie How to determine robabilitie
3 Playing a Game Show Aume erie of quetion Increaing difficulty Increaing ayoff Choice: Accet accumulated earning and quit Continue and rik loing everything Who want to be a millionaire? State Rereentation Dollar amount indicate the ayoff for getting the quetion right Probabilitic Tranition on Attemt to Anwer Start $00 correct $,000 correct $0K correct $50K $0 $0 $0 $0 $6,00 Downward green arrow indicate the choice to exit the game $00 $,00 $,00 N.B.: Thee exit tranition hould actually correond to tate Green indicate rofit at exit from game
4 Making Otimal Deciion Work backward from future to reent Conider $50,000 quetion Suoe Pcorrect) = /0 Vto)=$,00 Vcontinue) = 0.9*$0 + 0.*$6.K = $6.K Otimal deciion to Working Backward V=$,749 V=$4,66 V=$5,555 V=$.K 9/0 /4 / $00 $K $0K $50K X /0 $0 X $0 X $0 $0 Red X indicate bad choice $00 $,00 $,00 4
5 Deciion Theory Review Provide theory of otimal deciion Princile of maximizing utility Eay for mall, tree tructured ace with Known utilitie Known robabilitie Deciion Theory MDP Covered in Today Algorithm for MDP Value Determination Otimal Policy Selection Value Iteration Policy Iteration 5
6 Dealing with Loo Suoe you can ay $000 from any loing tate) to lay again 9/0 /4 / /0 $0 $0 $0 $0 $-000 $00 $,00 $,00 From Policie to Linear Sytem Suoe we alway ay until we win. What i value of following thi olicy? V 0 ) = V 0 )) V ) V ) = V 0 )) V ) V ) = V 0 )) V )!! V ) = V 0 )) ) Return to Start Continue 6
7 And the olution i V=$,749 V=$4,66 V=$5,555 V=$.K w/o cheat V=$.47K V=$.58K V=$.95K V=$4.4K 9/0 /4 / /0 $-000 I thi otimal? How do we find the otimal olicy? State ace: S The MDP Framework Action ace: A Tranition function: P Reward function: R,a, ) or R,a) or R) Dicount factor: g Policy: ) a Objective: Maximize exected, dicounted return deciion theoretic otimal behavior) 7
8 Alication of MDP AI/Comuter Science Robotic control Koenig & Simmon, Thrun et al., Kaelbling et al.) Air Camaign Planning Meuleau et al.) Elevator Control Barto & Crite) Comutation Scheduling Zilbertein et al.) Control and Automation Moore et al.) Soken dialogue management Singh et al.) Cellular channel allocation Singh & Berteka) Alication of MDP Economic/Oeration Reearch Fleet maintenance Howard, Rut) Road maintenance Golabi et al.) Packet Retranmiion Feinberg et al.) Nuclear lant management Rothwell & Rut) Debt collection trategie Abe et al.) Data center management DeeMind) 8
9 EE/Control Alication of MDP Miile defene Berteka et al.) Inventory management Van Roy et al.) Football lay election Patek & Berteka) Agriculture Herd management Kritenen, Toft) Other Sort trategie Video game The Markov Aumtion Let S t be a random variable for the tate at time t PS t A t- S t-,,a 0 S 0 ) = PS t A t- S t- ) Markov i ecial kind of conditional indeendence Future i indeendent of at given current tate 9
10 Undertanding Dicounting Mathematical motivation Kee value bounded What if I romie you $0.0 every day you viit me? Economic motivation Dicount come from inflation Promie of $.00 in future i worth $0.99 today Probability of dying Suoe e robability of dying at each deciion interval Tranition w/rob e to tate with value 0 Equivalent to - e dicount factor Dicounting in Practice Often choen unrealitically low Fater convergence of the algorithm we ll ee later Lead to lightly myoic olicie Can reformulate mot alg. for avg. reward Mathematically uglier Somewhat lower run time 0
11 Deciion Theory MDP Algorithm for MDP Value Determination Otimal Policy Selection Value Iteration Policy Iteration Covered Today V Value Determination Determine the value of each tate under olicy ) = R, )) + g å ' P ', )) V Bellman Equation for a fixed olicy ') R= S S S V ) = + g 0.4V ) + 0.6V ))
12 Matrix Form ø ö ç ç ç è æ = )), )), )), )), )), )), )), )), )), P P P P P P P P P P g R V P V + = Thi i a generalization of the game how examle from earlier How do we olve thi ytem efficient? Doe it even have a olution? Solving for Value g R V P V + = For moderate number of tate we can olve thi ytem exacty: g R P I V ) - - = Guaranteed invertible becaue ha ectral radiu < gp
13 Iteratively Solving for Value V = gp V + R For larger number of tate we can olve thi ytem indirectly: V i+ = gp Vi + R Guaranteed convergent becaue ha ectral radiu < gp Etablihing Convergence Eigenvalue analyi Monotonicity Aume all value tart eimitic One value mut alway increae Can never overetimate Eay to rove Contraction analyi
14 Contraction Analyi Define maximum norm Conider two value function V a and V b each at iteration : WLOG ay V a V -V V a V b =max i V [i ] b = e + e! Vector of all e ) Contraction Analyi Contd. At next iteration for V b : V b = R +γpv b For V a V a = R+γPV a ) R+γPV b + ε)=! R+γPV b +γp! ε = R+γPV b +γ! ε Conclude: Ditribute V a V b γε 4
15 Imortance of Contraction Any two value function get cloer True value function V* i a fixed oint value doen t change with iteration) Max norm ditance from V* decreae dramatically quickly with iteration V 0 V * =ε V n V * γ n ε Deciion Theory MDP Algorithm for MDP Value Determination Otimal Policy Selection Value Iteration Policy Iteration Covered Today 5
16 Finding Good Policie Suoe an exert told you the true value of each tate: VS) = 0 VS) = S 0.7 S 0.5 S 0. S Action Action Imroving Policie How do we get the otimal olicy? If we knew the value under the otimal olicy, then jut take the otimal action in every tate How do we define thee value? Fixed oint equation with choice Bellman equation): V * ) =max a R,a)+γ ' P',a)V * ') Deciion theoretic otimal choice given V* If we know V*, icking the otimal action i eay If we know the otimal action, comuting V* i eay How do we comute both at the ame time? 6
17 Value Iteration We can t olve the ytem directly with a max in the equation Can we olve it by iteration? V å ) = maxa R, a) + g P ', a) Vi ' ) i+ ' Called value iteration or imly ucceive aroximation Same a value determination, but we can change action Convergence: Can t do eigenvalue analyi not linear) Still monotonic Still a contraction in max norm exercie) Converge quickly Robot Navigation Examle quare,) 4 The robot hown ) live in a world decribed by a 4x grid of quare with quare,) occuied by an obtacle A tate i defined by the quare in which the robot i located:,) in the above figure tate 7
18 Action Tranition) Model U bring the robot to:,) with robability 0.8,) with robability 0.,) with robability 0. 4 In each tate, the robot oible action are {U, D, R, L} For each action: With robability 0.8 the robot doe the right thing move u, down, right, or left by one quare) With robability 0. it move in a direction erendicular to the intended one If the robot can t move, it tay in the ame quare [Thi model atifie the Markov condition] Action Tranition) Model L bring the robot to:,) with robability = 0.9,) with robability 0. 4 In each tate, the robot oible action are {U, D, R, L} For each action: With robability 0.8 the robot doe the right thing move u, down, right, or left by one quare) With robability 0. it move in a direction erendicular to the intended one If the robot can t move, it tay in the ame quare [Thi model atifie the Markov condition] 8
19 Terminal State, Reward, and Cot Two terminal tate: 4,) and 4,) Reward: R4,) = + [The robot find gold] R4,) = - [The robot get traed in quickand] R) = in all other tate Thi examle from the textbook) aume no dicounting g=) Dicuion: I thi a good modeling deciion? Stationary) Policy A tationary olicy i a comlete ma!: tate action For each non-terminal tate it recommend an action, indeendent of when and how the tate i reached Under the Markov and infinite horizon aumtion, the otimal olicy! i necearily a tationary olicy [The bet action in a tate doe not deend on the at] 4 9
20 Stationary) Policy A tationary olicy i a comlete ma!: tate action For each non-terminal tate it recommend an action, indeendent of when and how the tate The i reached otimal olicy trie to avoid Under the Markov and infinite dangerou horizon tate aumtion,,) the otimal olicy! i necearily a tationary olicy [The bet action in a tate doe not deend on the at] Finding! i called an obervable Markov Deciion Problem MDP) 4 Otimal Policie for Variou R) R) = R) = R) = -0.0 R) > 0 0
21 If i terminal: Bellman Equation If i non-terminal: + - 4! # = & # + max The equation!#) = &#) are non-linear + -../0) ,+) The utility of deend on the utility of other tate oibly, including ), and vice vera 7 # #, 8!# ) 9 # = arg max + -../0) ,+) 7 # #, 8!# ) [Bellman equation] Value Iteration Alied Initialize the utility of each non-terminal tate to V 0 ) = 0. For t = 0,,,... do! "#$ % = ' % + max for each non-terminal tate 4,.//0) ,,) : % 5 %, ;! " % 5 )
22 State Utilitie The utility of a tate i the maximal exected amount of reward that the robot will collect from and future tate by executing ome action in each encountered tate, until it reache a terminal tate infinite horizon) Under the Markov and infinite horizon aumtion, the utility of i indeendent of when and how i reached [It only deend on the oible equence of tate after, not on the oible equence before ] Convergence of Value Iteration
23 Proertie of Value Iteration VI converge to V*. " from V* hrink by g factor each iteration) Converge to otimal olicy Why? Becaue we figure out V*, otimal olicy i argmax) Otimal olicy i tationary i.e. Markovian deend only on current tate) Why? Becaue we are umming utilitie. Thought exeriment: Suoe you think it better to change action the econd time you viit a tate. Why didn t you jut take the bet action the firt time?) Deciion Theory MDP Algorithm for MDP Value Determination Otimal Policy Selection Value Iteration Policy Iteration Covered Today
24 Greedy Policy Contruction Let name the action that look bet WRT V:!! π v ) = argmax a R,a) + γ P',a)V') '!! π = greedyv) v Exectation over next-tate value Boottraing: Policy Iteration Idea: Greedy election i ueful even with ubotimal V Gue v = 0 V = value of acting on olve linear ytem) v greedyv ) Reeat until olicy doen t change Guaranteed to find otimal olicy Uually take very mall number of iteration Comuting the value function i the exenive art 4
25 Comaring VI and PI VI Value change at every te Policy may change before exact value of olicy i comuted Many chea iteration PI Alternate olicy/value udate Solve for value of each olicy exactly Fewer, lower iteration need to invert matrix) Convergence Both are contraction in max norm PI i hockingly fat in ractice Comutational Comlexity VI and PI are both contraction maing w/rate g we didn t rove thi for PI in cla) VI cot le er iteration For n tate, a action PI tend to take On) iteration in ractice Recent reult indicate ~On a/-g) wort cae Intereting aide: Bigget inight into PI came ~50 year after the algorithm wa introduced 5
26 A Unified View of Value Iteration and Policy Iteration Notation Udate for for a fixed olicy definition of T oerator:! " # % " + ' " # Udate with olicy imrovement definition of the T oerator:!#*) = max 0 *, + ' 4 56 * 6 *, #* 6 ) 6
27 Value Determination For 0 te! " = $ % For i te! & = ' %! &) = ' % &$ % Infinite horizon lim! & = ' %.$ % = % ) ) $ % &. Value Iteration For 0 te! " = $ If R deend on a, ick a with the highet immediate reward) For i te! % = &! %' = & ) $ Infinite horizon lim! % = & ) $ = &! =! % ) 7
28 Modified Policy Iteration Gue V 0 uually jut R), and i= Reeat until convergence* For j= to n V i = T V i- i = i+ =greedyv i- ) Secial cae: n= VI), n PI) MDP Limitation Reinforcement Learning MDP oerate at the level of tate State = atomic event We uually have exonentially or infinitely) many of thee We aume P and R are known Machine learning to the recue! Infer P and R imlicitly or exlicitly from data) Generalize from mall number of tate/olicie 8
Reinforcement Learning
Reinforcement Learning Yihay Manour Google Inc. & Tel-Aviv Univerity Outline Goal of Reinforcement Learning Mathematical Model (MDP) Planning Learning Current Reearch iue 2 Goal of Reinforcement Learning
More informationFigure 1 Siemens PSSE Web Site
Stability Analyi of Dynamic Sytem. In the lat few lecture we have een how mall ignal Lalace domain model may be contructed of the dynamic erformance of ower ytem. The tability of uch ytem i a matter of
More information66 Lecture 3 Random Search Tree i unique. Lemma 3. Let X and Y be totally ordered et, and let be a function aigning a ditinct riority in Y to each ele
Lecture 3 Random Search Tree In thi lecture we will decribe a very imle robabilitic data tructure that allow inert, delete, and memberhi tet (among other oeration) in exected logarithmic time. Thee reult
More informationskipping section 6.6 / 5.6 (generating permutations and combinations) concludes basic counting in Chapter 6 / 5
kiing ection 6.6 / 5.6 generating ermutation and combination conclude baic counting in Chater 6 / 5 on to Chater 7 / 6: Dicrete robability before we go to trickier counting in Chater 8 / 7 age 431-475
More informationProblem Set 8 Solutions
Deign and Analyi of Algorithm April 29, 2015 Maachuett Intitute of Technology 6.046J/18.410J Prof. Erik Demaine, Srini Devada, and Nancy Lynch Problem Set 8 Solution Problem Set 8 Solution Thi problem
More informationPrice Protection with Consumer s Policy Behavior Beibei LI, Huipo WANG and Yunfu HUO
8 3rd International Conference on Society Science and Economic Develoment (ICSSED 8) ISBN: 978--6595-3- Price Protection with Conumer Policy Behavior Beibei LI, Huio WANG and Yunfu HUO No., Xuefu Street,
More informationIEOR 3106: Fall 2013, Professor Whitt Topics for Discussion: Tuesday, November 19 Alternating Renewal Processes and The Renewal Equation
IEOR 316: Fall 213, Profeor Whitt Topic for Dicuion: Tueday, November 19 Alternating Renewal Procee and The Renewal Equation 1 Alternating Renewal Procee An alternating renewal proce alternate between
More informationDesign of Two-Channel Low-Delay FIR Filter Banks Using Constrained Optimization
contrained otimization, CIT Journal of Comuting and Information Technology, vol. 8, no 4,. 34 348, 2. Deign of Two-Channel Low-Delay FIR Filter Bank Uing Contrained Otimization Abtract Robert Bregović
More informationSystems Analysis. Prof. Cesar de Prada ISA-UVA
Sytem Analyi Prof. Cear de Prada ISAUVA rada@autom.uva.e Aim Learn how to infer the dynamic behaviour of a cloed loo ytem from it model. Learn how to infer the change in the dynamic of a cloed loo ytem
More informationUsing Maple to Evaluate the Partial Derivatives of Two-Variables Functions
Available Online at wwwijcmccom International Journal of Comuter Science and Mobile Comuting A Monthly Journal of Comuter Science and Information Technology ISSN 30 088X IJCSMC, Vol, Iue 6, June 013, g5
More informationAssignment for Mathematics for Economists Fall 2016
Due date: Mon. Nov. 1. Reading: CSZ, Ch. 5, Ch. 8.1 Aignment for Mathematic for Economit Fall 016 We now turn to finihing our coverage of concavity/convexity. There are two part: Jenen inequality for concave/convex
More informationEstimating Conditional Mean and Difference Between Conditional Mean and Conditional Median
Etimating Conditional Mean and Difference Between Conditional Mean and Conditional Median Liang Peng Deartment of Ri Management and Inurance Georgia State Univerity and Qiwei Yao Deartment of Statitic,
More informationCMSC 474, Introduction to Game Theory Maxmin and Minmax Strategies
CMSC 474, Introduction to Game Theory Maxmin and Minmax Strategie Mohammad T. Hajiaghayi Univerity of Maryland Wort-Cae Expected Utility For agent i, the wort-cae expected utility of a trategy i i the
More informationDo Dogs Know Bifurcations?
Do Dog Know Bifurcation? Roland Minton Roanoke College Salem, VA 4153 Timothy J. Penning Hoe College Holland, MI 4943 Elvi burt uon the mathematical cene in May, 003. The econd author article "Do Dog Know
More informationClustering Methods without Given Number of Clusters
Clutering Method without Given Number of Cluter Peng Xu, Fei Liu Introduction A we now, mean method i a very effective algorithm of clutering. It mot powerful feature i the calability and implicity. However,
More informationLecture 3. Dispersion and waves in cold plasma. Review and extension of the previous lecture. Basic ideas. Kramers-Kronig relations
Lecture 3 Dierion and wave in cold lama Review and extenion of the reviou lecture Baic idea At the reviou lecture, we dicued how to roerly earch for eigenmode (or quai-eigenmode) of a dierive medium. In
More informationBogoliubov Transformation in Classical Mechanics
Bogoliubov Tranformation in Claical Mechanic Canonical Tranformation Suppoe we have a et of complex canonical variable, {a j }, and would like to conider another et of variable, {b }, b b ({a j }). How
More informationMemoryle Strategie in Concurrent Game with Reachability Objective Λ Krihnendu Chatterjee y Luca de Alfaro x Thoma A. Henzinger y;z y EECS, Univerity o
Memoryle Strategie in Concurrent Game with Reachability Objective Krihnendu Chatterjee, Luca de Alfaro and Thoma A. Henzinger Report No. UCB/CSD-5-1406 Augut 2005 Computer Science Diviion (EECS) Univerity
More informationSociology 376 Exam 1 Spring 2011 Prof Montgomery
Sociology 76 Exam Spring Prof Montgomery Anwer all quetion. 6 point poible. You may be time-contrained, o pleae allocate your time carefully. [HINT: Somewhere on thi exam, it may be ueful to know that
More information11.5 MAP Estimator MAP avoids this Computational Problem!
.5 MAP timator ecall that the hit-or-mi cot function gave the MAP etimator it maimize the a oteriori PDF Q: Given that the MMS etimator i the mot natural one why would we conider the MAP etimator? A: If
More informationNCAAPMT Calculus Challenge Challenge #3 Due: October 26, 2011
NCAAPMT Calculu Challenge 011 01 Challenge #3 Due: October 6, 011 A Model of Traffic Flow Everyone ha at ome time been on a multi-lane highway and encountered road contruction that required the traffic
More informationChapter Landscape of an Optimization Problem. Local Search. Coping With NP-Hardness. Gradient Descent: Vertex Cover
Coping With NP-Hardne Chapter 12 Local Search Q Suppoe I need to olve an NP-hard problem What hould I do? A Theory ay you're unlikely to find poly-time algorithm Mut acrifice one of three deired feature
More informationMARKOV DECISION PROCESSES (MDP) AND REINFORCEMENT LEARNING (RL) Versione originale delle slide fornita dal Prof. Francesco Lo Presti
1 MARKOV DECISION PROCESSES (MDP) AND REINFORCEMENT LEARNING (RL) Versione originale delle slide fornita dal Prof. Francesco Lo Presti Historical background 2 Original motivation: animal learning Early
More informationMarch 18, 2014 Academic Year 2013/14
POLITONG - SHANGHAI BASIC AUTOMATIC CONTROL Exam grade March 8, 4 Academic Year 3/4 NAME (Pinyin/Italian)... STUDENT ID Ue only thee page (including the back) for anwer. Do not ue additional heet. Ue of
More informationLaplace Transformation
Univerity of Technology Electromechanical Department Energy Branch Advance Mathematic Laplace Tranformation nd Cla Lecture 6 Page of 7 Laplace Tranformation Definition Suppoe that f(t) i a piecewie continuou
More informationRELIABILITY OF REPAIRABLE k out of n: F SYSTEM HAVING DISCRETE REPAIR AND FAILURE TIMES DISTRIBUTIONS
www.arpapre.com/volume/vol29iue1/ijrras_29_1_01.pdf RELIABILITY OF REPAIRABLE k out of n: F SYSTEM HAVING DISCRETE REPAIR AND FAILURE TIMES DISTRIBUTIONS Sevcan Demir Atalay 1,* & Özge Elmataş Gültekin
More informationIterative Decoding of Trellis-Constrained Codes inspired by Amplitude Amplification (Preliminary Version)
Iterative Decoding of Trelli-ontrained ode inired by Amlitude Amlification (Preliminary Verion hritian Franck arxiv:190406473v1 [cit] 13 Ar 2019 Abtract We invetigate a novel aroach for the iterative decoding
More informationRADIATION THERMOMETRY OF METAL IN HIGH TEMPERATURE FURNACE
XVII IMEKO World Congre Metrology in the 3rd Millennium June 22 27, 2003, Dubrovnik, Croatia RADIATION THERMOMETRY OF METAL IN HIGH TEMPERATURE FURNACE Tohru Iuchi, Tohru Furukawa and Nobuharu Sato Deartment
More informationStochastic Perishable Inventory Control in a Service Facility System Maintaining Inventory for Service: Semi Markov Decision Problem
Stochatic Perihable Inventory Control in a Service Facility Sytem Maintaining Inventory for Service: Semi Markov Deciion Problem R.Mugeh 1,S.Krihnakumar 2, and C.Elango 3 1 mugehrengawamy@gmail.com 2 krihmathew@gmail.com
More informationImproved Adaptive Time Delay Estimation Algorithm Based on Fourth-order Cumulants
Available online www.jocr.com Journal of hemical and Pharmaceutical Reearch, 016, 8(5):889-894 Reearch Article ISSN : 0975-784 ODEN(USA) : JPR5 Imroved Adative Time Delay Etimation Algorithm Baed on Fourth-order
More informationROOT LOCUS. Poles and Zeros
Automatic Control Sytem, 343 Deartment of Mechatronic Engineering, German Jordanian Univerity ROOT LOCUS The Root Locu i the ath of the root of the characteritic equation traced out in the - lane a a ytem
More informationEfficiency Optimal of Inductive Power Transfer System Using the Genetic Algorithms Jikun Zhou *, Rong Zhang, Yi Zhang
International Conference on echanical Science and Engineering (ICSE5 Efficiency Otimal of Inductive Power Tranfer Sytem Uing the Genetic Algorithm Jikun Zhou *, ong Zhang, Yi Zhang Intitute of ytem engineering,
More informationonline learning Unit Workbook 4 RLC Transients
online learning Pearon BTC Higher National in lectrical and lectronic ngineering (QCF) Unit 5: lectrical & lectronic Principle Unit Workbook 4 in a erie of 4 for thi unit Learning Outcome: RLC Tranient
More informationSocial Studies 201 Notes for November 14, 2003
1 Social Studie 201 Note for November 14, 2003 Etimation of a mean, mall ample ize Section 8.4, p. 501. When a reearcher ha only a mall ample ize available, the central limit theorem doe not apply to the
More informationThe simplex method is strongly polynomial for deterministic Markov decision processes
The implex method i trongly polynomial for determinitic Markov deciion procee Ian Pot Yinyu Ye May 31, 2013 Abtract We prove that the implex method with the highet gain/mot-negative-reduced cot pivoting
More informationRisk reducing actions: efficiency evaluation
E TRNCTION on BUINE and ECONOMIC Joel Pereira Jr., ougla Barboa, Patricia Bernarde, Thiago Bruzadelli, Petr Eel, Tiago araiva Ri reducing action: efficiency evaluation JOE PEREIR JR.,2,*, OUG BRBO,4, PTRICI
More informationRELIABILITY ANALYSIS OF A COMPLEX REPAIRABLE SYSTEM COMPOSED OF TWO 2-OUT-OF-3: G SUBSYSTEMS CONNECTED IN PARALLEL
Journal of Reliability and Statitical Studie; ISSN (Print: 97-8, (Online:9-5666 Vol. 7, Iue Secial (: 89- RELIILITY NLYSIS OF COMPLEX REPIRLE SYSTEM COMPOSE OF TWO -OUT-OF-: G SUSYSTEMS CONNECTE IN PRLLEL
More informationThe Hassenpflug Matrix Tensor Notation
The Haenpflug Matrix Tenor Notation D.N.J. El Dept of Mech Mechatron Eng Univ of Stellenboch, South Africa e-mail: dnjel@un.ac.za 2009/09/01 Abtract Thi i a ample document to illutrate the typeetting of
More informationMoment of Inertia of an Equilateral Triangle with Pivot at one Vertex
oment of nertia of an Equilateral Triangle with Pivot at one Vertex There are two wa (at leat) to derive the expreion f an equilateral triangle that i rotated about one vertex, and ll how ou both here.
More informationTo describe a queuing system, an input process and an output process has to be specified.
5. Queue (aiting Line) Queuing terminology Input Service Output To decribe a ueuing ytem, an input proce and an output proce ha to be pecified. For example ituation input proce output proce Bank Cutomer
More informationChapter 4. The Laplace Transform Method
Chapter 4. The Laplace Tranform Method The Laplace Tranform i a tranformation, meaning that it change a function into a new function. Actually, it i a linear tranformation, becaue it convert a linear combination
More informationDepartment of Mechanical Engineering Massachusetts Institute of Technology Modeling, Dynamics and Control III Spring 2002
Department of Mechanical Engineering Maachuett Intitute of Technology 2.010 Modeling, Dynamic and Control III Spring 2002 SOLUTIONS: Problem Set # 10 Problem 1 Etimating tranfer function from Bode Plot.
More informationAn Inequality for Nonnegative Matrices and the Inverse Eigenvalue Problem
An Inequality for Nonnegative Matrice and the Invere Eigenvalue Problem Robert Ream Program in Mathematical Science The Univerity of Texa at Dalla Box 83688, Richardon, Texa 7583-688 Abtract We preent
More informationLecture 10 Filtering: Applied Concepts
Lecture Filtering: Applied Concept In the previou two lecture, you have learned about finite-impule-repone (FIR) and infinite-impule-repone (IIR) filter. In thee lecture, we introduced the concept of filtering
More informationCorrection for Simple System Example and Notes on Laplace Transforms / Deviation Variables ECHE 550 Fall 2002
Correction for Simple Sytem Example and Note on Laplace Tranform / Deviation Variable ECHE 55 Fall 22 Conider a tank draining from an initial height of h o at time t =. With no flow into the tank (F in
More informationMarkov Decision Processes Infinite Horizon Problems
Markov Decision Processes Infinite Horizon Problems Alan Fern * * Based in part on slides by Craig Boutilier and Daniel Weld 1 What is a solution to an MDP? MDP Planning Problem: Input: an MDP (S,A,R,T)
More informationLinear Motion, Speed & Velocity
Add Important Linear Motion, Speed & Velocity Page: 136 Linear Motion, Speed & Velocity NGSS Standard: N/A MA Curriculum Framework (006): 1.1, 1. AP Phyic 1 Learning Objective: 3.A.1.1, 3.A.1.3 Knowledge/Undertanding
More informationPreemptive scheduling on a small number of hierarchical machines
Available online at www.ciencedirect.com Information and Computation 06 (008) 60 619 www.elevier.com/locate/ic Preemptive cheduling on a mall number of hierarchical machine György Dóa a, Leah Eptein b,
More informationÜbung zu Globale Geophysik I 05 Answers
Übung zu Globale Geohyik I 05 Anwer Übung zu Globale Geohyik I: Wedneday, 6:00 8:00, Thereientr. 4, Room C49 Lecturer: Heather McCreadie Comanion cla to Globale Geohyik I: Monday, :00 4:00, Thereientr.
More informationLecture #5: Introduction to Continuum Mechanics Three-dimensional Rate-independent Plasticity. by Dirk Mohr
Lecture #5: 5-0735: Dynamic behavior of material and tructure Introduction to Continuum Mechanic Three-dimenional Rate-indeendent Platicity by Dirk Mohr ETH Zurich, Deartment of Mechanical and Proce Engineering,
More informationLecture 10. Erbium-doped fiber amplifier (EDFA) Raman amplifiers Have replaced semiconductor optical amplifiers in the course
ecture 1 Two tye of otical amlifier: Erbium-doed fiber amlifier (EDFA) Raman amlifier Have relaced emiconductor otical amlifier in the coure Fiber Otical Communication ecture 1, Slide 1 Benefit and requirement
More informationCS788 Dialogue Management Systems Lecture #2: Markov Decision Processes
CS788 Dialogue Management Systems Lecture #2: Markov Decision Processes Kee-Eung Kim KAIST EECS Department Computer Science Division Markov Decision Processes (MDPs) A popular model for sequential decision
More informationMidterm 3 Review Solutions by CC
Midterm Review Solution by CC Problem Set u (but do not evaluate) the iterated integral to rereent each of the following. (a) The volume of the olid encloed by the arabaloid z x + y and the lane z, x :
More informationμ + = σ = D 4 σ = D 3 σ = σ = All units in parts (a) and (b) are in V. (1) x chart: Center = μ = 0.75 UCL =
Our online Tutor are available 4*7 to provide Help with Proce control ytem Homework/Aignment or a long term Graduate/Undergraduate Proce control ytem Project. Our Tutor being experienced and proficient
More informationTwo-echelon supply chain coordination under information asymmetry with multiple types
Two-echelon uly chain coordination under information aymmetry with multile tye.b.o. Kerkkam & W. van den Heuvel & A.P.M. Wagelman Econometric Intitute eort EI206-8 Abtract We analye a rincial-agent contracting
More informationLecture 8: Period Finding: Simon s Problem over Z N
Quantum Computation (CMU 8-859BB, Fall 205) Lecture 8: Period Finding: Simon Problem over Z October 5, 205 Lecturer: John Wright Scribe: icola Rech Problem A mentioned previouly, period finding i a rephraing
More informationUSE OF INTERNET TO DO EXPERIMENTS IN DYNAMICS AND CONTROL FROM ZACATECAS MEXICO IN THE LABORATORY OF THE UNIVERSITY OF TENNESSEE AT CHATANOOGAA.
USE OF INTERNET TO DO EXPERIMENTS IN DYNAMICS AND CONTROL FROM ZACATECAS MEXICO IN TE LABORATORY OF TE UNIVERSITY OF TENNESSEE AT CATANOOGAA. Jim enry *, Joé Alberto González Guerrero, Benito Serrano Roale..
More informationAnalysis the Transient Process of Wind Power Resources when there are Voltage Sags in Distribution Grid
Analyi the Tranient Proce of Wind Power Reource when there are Voltage Sag in Ditribution Grid Do Nhu Y 1,* 1 Hanoi Univerity of ining and Geology, Deartment of Electrification, Electromechanic Faculty,
More informationChapter 3- Answers to selected exercises
Chater 3- Anwer to elected exercie. he chemical otential of a imle uid of a ingle comonent i gien by the exreion o ( ) + k B ln o ( ) ; where i the temerature, i the reure, k B i the Boltzmann contant,
More informationSolving Differential Equations by the Laplace Transform and by Numerical Methods
36CH_PHCalter_TechMath_95099 3//007 :8 PM Page Solving Differential Equation by the Laplace Tranform and by Numerical Method OBJECTIVES When you have completed thi chapter, you hould be able to: Find the
More informationTRANSITION PROBABILITY MATRIX OF BRIDGE MEMBERS DAMAGE RATING
TRANSITION PROBABILITY MATRIX OF BRIDGE MEMBERS DAMAGE RATING Hirohi Sato and Ryoji Hagiwara 2 Abtract Bridge member damage characteritic were tudied uing the inpection record. Damage can be claified into
More information(b) Is the game below solvable by iterated strict dominance? Does it have a unique Nash equilibrium?
14.1 Final Exam Anwer all quetion. You have 3 hour in which to complete the exam. 1. (60 Minute 40 Point) Anwer each of the following ubquetion briefly. Pleae how your calculation and provide rough explanation
More informationSuggested Answers To Exercises. estimates variability in a sampling distribution of random means. About 68% of means fall
Beyond Significance Teting ( nd Edition), Rex B. Kline Suggeted Anwer To Exercie Chapter. The tatitic meaure variability among core at the cae level. In a normal ditribution, about 68% of the core fall
More informationCS 170: Midterm Exam II University of California at Berkeley Department of Electrical Engineering and Computer Sciences Computer Science Division
1 1 April 000 Demmel / Shewchuk CS 170: Midterm Exam II Univerity of California at Berkeley Department of Electrical Engineering and Computer Science Computer Science Diviion hi i a cloed book, cloed calculator,
More informationarxiv: v2 [math.nt] 1 Jan 2018
A CONTINUOUS ANALOGUE OF LATTICE PATH ENUMERATION: PART II TANAY WAKHARE AND CHRISTOPHE VIGNAT arxiv:66986v [mathnt] Jan 8 Abtract Following the work of Cano and Díaz, we tudy continuou binomial coefficient
More informationOVERSHOOT FREE PI CONTROLLER TUNING BASED ON POLE ASSIGNMENT
OVERSHOO FREE PI CONROER UNING BASED ON POE ASSIGNMEN Nevra Bayhan * Mehmet uran Söylemez ** uğba Botan ** e-mail: nevra@itanbul.edu.tr e-mail: oylemez@el.itu.edu.tr e-mail: botan@itu.edu.tr * Itanbul
More informationSource slideplayer.com/fundamentals of Analytical Chemistry, F.J. Holler, S.R.Crouch. Chapter 6: Random Errors in Chemical Analysis
Source lideplayer.com/fundamental of Analytical Chemitry, F.J. Holler, S.R.Crouch Chapter 6: Random Error in Chemical Analyi Random error are preent in every meaurement no matter how careful the experimenter.
More information7.2 INVERSE TRANSFORMS AND TRANSFORMS OF DERIVATIVES 281
72 INVERSE TRANSFORMS AND TRANSFORMS OF DERIVATIVES 28 and i 2 Show how Euler formula (page 33) can then be ued to deduce the reult a ( a) 2 b 2 {e at co bt} {e at in bt} b ( a) 2 b 2 5 Under what condition
More informationON THE APPROXIMATION ERROR IN HIGH DIMENSIONAL MODEL REPRESENTATION. Xiaoqun Wang
Proceeding of the 2008 Winter Simulation Conference S. J. Maon, R. R. Hill, L. Mönch, O. Roe, T. Jefferon, J. W. Fowler ed. ON THE APPROXIMATION ERROR IN HIGH DIMENSIONAL MODEL REPRESENTATION Xiaoqun Wang
More informationHalliday/Resnick/Walker 7e Chapter 6
HRW 7e Chapter 6 Page of Halliday/Renick/Walker 7e Chapter 6 3. We do not conider the poibility that the bureau might tip, and treat thi a a purely horizontal motion problem (with the peron puh F in the
More informationSTOCHASTIC GENERALIZED TRANSPORTATION PROBLEM WITH DISCRETE DISTRIBUTION OF DEMAND
OPERATIONS RESEARCH AND DECISIONS No. 4 203 DOI: 0.5277/ord30402 Marcin ANHOLCER STOCHASTIC GENERALIZED TRANSPORTATION PROBLEM WITH DISCRETE DISTRIBUTION OF DEMAND The generalized tranportation problem
More informationPerformance Evaluation
Performance Evaluation 95 (206) 40 Content lit available at ScienceDirect Performance Evaluation journal homepage: www.elevier.com/locate/peva Optimal cheduling in call center with a callback option Benjamin
More informationSolving Radical Equations
10. Solving Radical Equation Eential Quetion How can you olve an equation that contain quare root? Analyzing a Free-Falling Object MODELING WITH MATHEMATICS To be proficient in math, you need to routinely
More informationDiscovery Mass Reach for Excited Quarks at Hadron Colliders
Dicovery Ma Reach for Excited Quark at Hadron Collider Robert M. Harri Fermilab, Batavia, IL 60510 ABSTRACT If quark are comoite article then excited tate are exected. We etimate the dicovery ma reach
More informationOrbitals, Shapes and Polarity Quiz
Orbital, Shae and Polarity Quiz Name: /17 Knowledge. Anwer the following quetion on foolca. /2 1. Exlain why the ub-level can aear to be herical like the ub-level? /2 2.a) What i the maximum number of
More informationPhysics 741 Graduate Quantum Mechanics 1 Solutions to Final Exam, Fall 2014
Phyic 7 Graduate Quantum Mechanic Solution to inal Eam all 0 Each quetion i worth 5 point with point for each part marked eparately Some poibly ueful formula appear at the end of the tet In four dimenion
More informationReinforcement Learning and Control
CS9 Lecture notes Andrew Ng Part XIII Reinforcement Learning and Control We now begin our study of reinforcement learning and adaptive control. In supervised learning, we saw algorithms that tried to make
More informationFactor Analysis with Poisson Output
Factor Analyi with Poion Output Gopal Santhanam Byron Yu Krihna V. Shenoy, Department of Electrical Engineering, Neurocience Program Stanford Univerity Stanford, CA 94305, USA {gopal,byronyu,henoy}@tanford.edu
More informationSingular perturbation theory
Singular perturbation theory Marc R. Rouel June 21, 2004 1 Introduction When we apply the teady-tate approximation (SSA) in chemical kinetic, we typically argue that ome of the intermediate are highly
More informationCHAPTER 5. The Operational Amplifier 1
EECE22 NETWORK ANALYSIS I Dr. Charle J. Kim Cla Note 9: Oerational Amlifier (OP Am) CHAPTER. The Oerational Amlifier A. INTRODUCTION. The oerational amlifier or o am for hort, i a eratile circuit building
More informationSocial Studies 201 Notes for March 18, 2005
1 Social Studie 201 Note for March 18, 2005 Etimation of a mean, mall ample ize Section 8.4, p. 501. When a reearcher ha only a mall ample ize available, the central limit theorem doe not apply to the
More informationGiven the following circuit with unknown initial capacitor voltage v(0): X(s) Immediately, we know that the transfer function H(s) is
EE 4G Note: Chapter 6 Intructor: Cheung More about ZSR and ZIR. Finding unknown initial condition: Given the following circuit with unknown initial capacitor voltage v0: F v0/ / Input xt 0Ω Output yt -
More informationChapter 2 Sampling and Quantization. In order to investigate sampling and quantization, the difference between analog
Chapter Sampling and Quantization.1 Analog and Digital Signal In order to invetigate ampling and quantization, the difference between analog and digital ignal mut be undertood. Analog ignal conit of continuou
More informationOne Class of Splitting Iterative Schemes
One Cla of Splitting Iterative Scheme v Ciegi and V. Pakalnytė Vilniu Gedimina Technical Univerity Saulėtekio al. 11, 2054, Vilniu, Lithuania rc@fm.vtu.lt Abtract. Thi paper deal with the tability analyi
More informationUSEFUL TECHNIQUES FOR FIELD ANALYSTS IN THE DESIGN AND OPTIMIZATION OF LINEAR INDUCTION MOTORS
USEFUL TECHNIQUES FOR FIELD ANALYSTS IN THE DESIGN AND OPTIMIZATION OF LINEAR INDUCTION MOTORS By: K.R. Davey R.C. Zowarka Twelfth Biennial IEEE Conference on Electromagnetic Field Comutation (CEFC 006),
More informationarxiv: v1 [math.mg] 25 Aug 2011
ABSORBING ANGLES, STEINER MINIMAL TREES, AND ANTIPODALITY HORST MARTINI, KONRAD J. SWANEPOEL, AND P. OLOFF DE WET arxiv:08.5046v [math.mg] 25 Aug 20 Abtract. We give a new proof that a tar {op i : i =,...,
More informationQuestion 1 Equivalent Circuits
MAE 40 inear ircuit Fall 2007 Final Intruction ) Thi exam i open book You may ue whatever written material you chooe, including your cla note and textbook You may ue a hand calculator with no communication
More informationHigh Precision Feedback Control Design for Dual-Actuator Systems
Proceeding of the 25 IEEE Conference on Control Alication Toronto, Canada, Augut 28-31, 25 TC1.2 High Preciion Feedback Control Deign for Dual-Actuator Sytem Tielong Shen and Minyue Fu Abtract Dual-actuator
More informationLecture 17: Analytic Functions and Integrals (See Chapter 14 in Boas)
Lecture 7: Analytic Function and Integral (See Chapter 4 in Boa) Thi i a good point to take a brief detour and expand on our previou dicuion of complex variable and complex function of complex variable.
More informationChapter 5 Consistency, Zero Stability, and the Dahlquist Equivalence Theorem
Chapter 5 Conitency, Zero Stability, and the Dahlquit Equivalence Theorem In Chapter 2 we dicued convergence of numerical method and gave an experimental method for finding the rate of convergence (aka,
More informationAdministration, Department of Statistics and Econometrics, Sofia, 1113, bul. Tzarigradsko shose 125, bl.3, Bulgaria,
Adanced Studie in Contemorary Mathematic, (006), No, 47-54 DISTRIBUTIONS OF JOINT SAMPLE CORRELATION COEFFICIENTS OF INDEPEENDENT NORMALLY DISTRIBUTED RANDOM VARIABLES Eelina I Velea, Tzetan G Ignato Roue
More informationFrames of Reference and Relative Velocity
1.5 frame of reference coordinate ytem relative to which motion i oberved Frame of Reference and Relative Velocity Air how provide element of both excitement and danger. When high-peed airplane fly in
More informationMATEMATIK Datum: Tid: eftermiddag. A.Heintz Telefonvakt: Anders Martinsson Tel.:
MATEMATIK Datum: 20-08-25 Tid: eftermiddag GU, Chalmer Hjälpmedel: inga A.Heintz Telefonvakt: Ander Martinon Tel.: 073-07926. Löningar till tenta i ODE och matematik modellering, MMG5, MVE6. Define what
More informationLinear System Fundamentals
Linear Sytem Fundamental MEM 355 Performance Enhancement of Dynamical Sytem Harry G. Kwatny Department of Mechanical Engineering & Mechanic Drexel Univerity Content Sytem Repreentation Stability Concept
More informationConfusion matrices. True / False positives / negatives. INF 4300 Classification III Anne Solberg The agenda today: E.g., testing for cancer
INF 4300 Claification III Anne Solberg 29.10.14 The agenda today: More on etimating claifier accuracy Cure of dimenionality knn-claification K-mean clutering x i feature vector for pixel i i- The cla label
More informationOnline Appendix for Managerial Attention and Worker Performance by Marina Halac and Andrea Prat
Online Appendix for Managerial Attention and Worker Performance by Marina Halac and Andrea Prat Thi Online Appendix contain the proof of our reult for the undicounted limit dicued in Section 2 of the paper,
More informationarxiv: v2 [math.nt] 30 Apr 2015
A THEOREM FOR DISTINCT ZEROS OF L-FUNCTIONS École Normale Supérieure arxiv:54.6556v [math.nt] 3 Apr 5 943 Cachan November 9, 7 Abtract In thi paper, we etablih a imple criterion for two L-function L and
More informationDesign By Emulation (Indirect Method)
Deign By Emulation (Indirect Method he baic trategy here i, that Given a continuou tranfer function, it i required to find the bet dicrete equivalent uch that the ignal produced by paing an input ignal
More informationME 3560 Fluid Mechanics
Sring 018 ME 3560 Fluid Mechanic Chater III. Elementary Fluid Dynamic The Bernoulli Equation 1 Sring 018 3.1 Newton Second Law A fluid article can exerience acceleration or deceleration a it move from
More informationJul 4, 2005 turbo_code_primer Revision 0.0. Turbo Code Primer
Jul 4, 5 turbo_code_primer Reviion. Turbo Code Primer. Introduction Thi document give a quick tutorial on MAP baed turbo coder. Section develop the background theory. Section work through a imple numerical
More information