Reinforcement Learning" CMPSCI 383 Nov 29, 2011!
|
|
- Dustin Neal
- 6 years ago
- Views:
Transcription
1 Reinfrcement Learning" CMPSCI 383 Nv 29, 2011! 1
2 Tdayʼs lecture" Review f Chapter 17: Making Cmple Decisins! Sequential decisin prblems! The mtivatin and advantages f reinfrcement learning.! Passive learning! Plicy evaluatin! Direct utility estimatin! Adaptive dynamic prgramming! Tempral Difference (TD) learning! 2
3 Check this ut" MIT's Technlgy Review has an in-depth interview with Peter Nrvig, Ggle'sDirectr f Research, and Eric Hrvitz, a Distinguished Scientist at MicrsftResearch abut their ptimism fr the futuref AI.! 3
4 A Simple Eample" Gridwrld with 2 Gal states! Actins: Up, Dwn, Left, Right! Fully bservable: Agent knws where it is! 4
5 Transitin Mdel" 5
6 Markv Assumptin" 6
7 Agentʼs Utility Functin" Perfrmance depends n the entire sequence f states and actins.! Envirnment histry! In each state, the agent receives a reward R(s).! The reward is real-valued. It may be psitive r negative.! Utility f envirnment histry = sum f reward received.! 7
8 Reward functin"
9 Decisin Rules" Decisin rules say what t d in each state.! Often called plicies, π. Actin fr state s is given by π(s).! 9
10 Our Gal" Find the plicy that maimizes the epected sum f rewards.! Called an ptimal plicy.! 10
11 Markv Decisin Prcess (MDP)" M=(S,A,P,R)! S = set f pssible states! A = set f pssible actins! P(sʼ s,a) gives transitin prbabilities! R = reward functin! Gal: find an ptimal plicy, π*.! 11
12 Finite/Infinite Hrizn" Finite hrizn: the game ends after N steps.! Infinite hrizn: the game never ends! With a finite hrizn, the ptimal actin in a given state culd change ver time.! The ptimal plicy is nnstatinary.! With infinite hrizn, the ptimal plicy is statinary.! 12
13 Utilities ver time" Additive rewards:! Discunted rewards:! Discunt factr:! 13
14 Discunted Rewards" Wuld yu rather have a marshmallw nw, r tw in 20 minutes?! Infinite sums!! 14
15 Utility f States" Given a plicy, we can define the utility f a state:! 15
16 Plicy Evaluatin" Finding the utility f states fr a given plicy.! Slve a system f linear equatins:! π π An instance f a Bellman Equatin.! 16
17 Plicy Evaluatin" The n linear equatins can be slved in O(n 3 ) with standard linear algebra methds.! If O(n 3 ) is still t much, we can d it iteratively:! π π 17
18 Optimal Plicy" Optimal plicy desnʼt depend n what state yu start in (fr infinite hrizn discunted case).! Optimal plicy:! π * True utility f a state:! ptimal value functin! 18
19 Selecting ptimal actins" Given the true U(s) values, hw can we select actins? (Maimum epected utility MEU)! 19
20 Utility and Rewards" Utility = lng term ttal reward frm s nwards! Reward = shrt term reward frm s! 20
21 Utility" 21
22 Value-Guided Optimal Cntrl" Predicted minimum time t gal (negated) Get t the tp f the hill as quickly as pssible (rughly) Muns & Mre Variable reslutin discretizatin fr high-accuracy slutins f ptimal cntrl prblems, IJCAI
23 Searching fr Optimal Plicies" Bellman Equatin! If we write ut the Bellman equatin fr all n states, we get n equatins, with n unknwns: U(s).! We can slve this system f equatins t determine the Utility f every state.! 23
24 Value Iteratin" The equatins are nn-linear, s we canʼt use standard linear algebra methds.! Value iteratin: start with randm initial values fr each U(s), iteratively update each value t fit the fight-hand side f the equatin:! 24
25 Value Iteratin" The update is applied simultaneusly t every state.! If this update is applied infinitely ften, we are guaranteed t find the true U(s) values.! There is ne unique slutin! Given the true U(s) values, hw can we select actins? (Maimum epected utility MEU)! 25
26 Plicy Iteratin" Plicy iteratin interleaves tw steps:! Plicy evaluatin: Given a plicy, cmpute the utility f each state fr that plicy! Plicy imprvement: Calculate a new MEU plicy! Terminate when the plicy desnʼt change the utilities.! Guaranteed t cnverge t an ptimal plicy! 26
27 Asynchrnus Plicy Iteratin" We said the utility f every state is updated simultaneusly. This isnʼt necessary.! Yu can pick and subset f the states and apply either plicy imprvement r value iteratin t that subset.! Given certain cnditins, this is als guaranteed t cnverge t an ptimal plicy.! 27
28 Reinfrcement Learning" 28
29 Edward L. Thrndike ( )" Learning by Trial-and-Errr! puzzle b! 29
30 Law f Effect " Of several respnses made t the same situatin, thse which are accmpanied r clsely fllwed by satisfactin t the animal will, ther things being equal, be mre firmly cnnected with the situatin, s that, when it recurs, they will be mre likely t recur; thse which are accmpanied r clsely fllwed by discmfrt t the animal will, ther things being equal, have their cnnectins with that situatin weakened, s that when it recurs, they will be less likely t ccur. Edward Thrndike,
31 RL = Search + Memry" Search: Trial-and-Errr, Generate-and-Test, Variatin-and-Selectin! Memry: remember what wrked best fr each situatin and start frm there net time! 31
32 32 MENACE (Michie, 1961)" Matchb Educable Nughts and Crsses Engine
33 Supervised learning" Training Inf = desired (target) utputs! Inputs! Supervised Learning " System! Outputs! Errr = (target utput actual utput)! 33
34 Reinfrcement learning" Training Inf = evaluatins ( rewards, penalties, etc.)! RL" Inputs! Outputs! System! ( actins, cntrls )! Objective: get as much reward as pssible! 34
35 Reinfrcement learning" Learning what t d hw t map situatins t actins s as t maimize a numerical reward signal! Rewards may be prvided fllwing each actin, r nly when the agent reaches a gal state.! The credit assignment prblem.! 35
36 Why reinfrcement learning?" Mre realistic mdel f the kind f learning that animals d.! Supervised learning is smetimes unrealistic: where will crrect eamples cme frm?! Envirnments change, and s the agent must adjust its actin chices.! 36
37 Stick t this kind f RL prblem here:" Envirnment is fully bservable: agent knws what state the envirnment is in! But: agent des nt knw hw the envirnment wrks r what its actins d! 37
38 Sme kinds f RL agents" Utility-based agent: learns utility functin n states and uses it t select actins! Needs an envirnment mdel t decide n actins! Q-learning agent: leans and actin-utility functins, r Q-functin, giving epected utility fr taking each actin in each state.! Des nt need an envirnment mdel.! Refle agent: learn a plicy withut first learning a state-utility functin r a Q-functin! 38
39 Passive versus active learning " A passive learner simply watches the wrld ging by and tries t learn the utility f being in varius states.! An active learner must als act using the learned infrmatin, and can use its prblem generatr t suggest eplratins f unknwn prtins f the envirnment.! 39
40 Passive learning" Given (but agent desnʼt knw this):" A Markv mdel f the envirnment.! States, with prbabilistic actins.! Terminal states have rewards/utilities.! Prblem:" Learn epected utility f each state.! Nte: if agent knws hw the envirnment and its actins wrk, can slve the relevant Bellman equatin (which wuld be linear).! 40
41 Eample" Percepts tell agent: [State, Reward, Terminal?] 41
42 Learning utility functins" A training sequence (r episde) is an instance f wrld transitins frm an initial state t a terminal state.! The additive utility assumptin: utility f a sequence is the sum f the rewards ver the states f the sequence.! Under this assumptin, the utility f a state is the epected reward-t-g f that state.! 42
43 Direct Utility Estimatin" Fr each training sequence, cmpute the reward-t-g fr each state in the sequence and update the utilities.! This is just learning the utility functin frm eamples.! Generates utility estimates that minimize the mean square errr (LMS-update).! 43
44 Direct Utility Estimatin" U(i) (1 α)u(i) + α REWARD(training sequence) state i! T T T T T T T T T T 44
45 Prblems with direct utility estimatin" Cnverges slwly because it ignres the relatinship between neighbring states:! 1 New U=? Old U=.8 p=.9 p=
46 Key Observatin: Tempral Cnsistency" Utilities f states are nt independent! The utility f each state equals its wn reward plus the epected utility f its successr states:! U π (s) = R(s) +γ s ʹ P( s ʹ s, π(s)) U π ( s ʹ ) The key fact:! U t = r t +1 +γ r t +2 +γ 2 r t +3 +γ 3 r t +4 L = r t +1 +γ( r t +2 +γ r t +3 +γ 2 r t +4 L) = r t +1 +γu t +1 46
47 Adaptive dynamic prgramming" Learn a mdel: transitin prbabilities, reward functin! D plicy evaluatin! Slve the Bellman equatin either directly r iteratively (value iteratin withut the ma)! Learn mdel while ding iterative plicy evaluatin:! Update the mdel f the envirnment after each step. Since the mdel changes nly slightly after each step, plicy evaluatin will cnverge quickly.! 47
48 Adaptive dynamic prg (ADP)" j U(i) R(i) + P ij U( j) state i R(i) pssible states j T T T T T T T T T T 48
49 Passive ADP learning curves" 49
50 Eact utility values fr the eample" 50
51 Tempral difference (TD) learning" Apprimate the cnstraint equatins withut slving them fr all states.! Mdify U(i) whenever we see a transitin frm i t j using the fllwing rule:! U(i) U(i) +α[ R(i) + U( j) U(i) ] The mdificatin mves U(i) clser t satisfying the riginal equatin.! Q. Why des it wrk?! 51
52 TD learning cntd." U(i) (1 α)u(i) + α[ R(i) + U( j) ] state i R(i) state j T T T T T T T T T T 52
53 TD learning curves" 53
54 Net Class" Active Reinfrcement Learning! Sec. 21.3! 54
55 Net Class" Reinfrcement Learning! Secs ! 55
Admin. MDP Search Trees. Optimal Quantities. Reinforcement Learning
Admin Reinfrcement Learning Cntent adapted frm Berkeley CS188 MDP Search Trees Each MDP state prjects an expectimax-like search tree Optimal Quantities The value (utility) f a state s: V*(s) = expected
More informationAn Introduction to COMPUTATIONAL REINFORCEMENT LEARING. Andrew G. Barto. Department of Computer Science University of Massachusetts Amherst
An Intrductin t COMPUTATIONAL REINFORCEMENT LEARING Andrew G. Bart Department f Cmputer Science University f Massachusetts Amherst UPF Lecture 1 Autnmus Learning Labratry Department f Cmputer Science Artificial
More informationYou need to be able to define the following terms and answer basic questions about them:
CS440/ECE448 Sectin Q Fall 2017 Midterm Review Yu need t be able t define the fllwing terms and answer basic questins abut them: Intr t AI, agents and envirnments Pssible definitins f AI, prs and cns f
More informationAdministrativia. Assignment 1 due thursday 9/23/2004 BEFORE midnight. Midterm exam 10/07/2003 in class. CS 460, Sessions 8-9 1
Administrativia Assignment 1 due thursday 9/23/2004 BEFORE midnight Midterm eam 10/07/2003 in class CS 460, Sessins 8-9 1 Last time: search strategies Uninfrmed: Use nly infrmatin available in the prblem
More informationDepartment of Electrical Engineering, University of Waterloo. Introduction
Sectin 4: Sequential Circuits Majr Tpics Types f sequential circuits Flip-flps Analysis f clcked sequential circuits Mre and Mealy machines Design f clcked sequential circuits State transitin design methd
More informationPart 3 Introduction to statistical classification techniques
Part 3 Intrductin t statistical classificatin techniques Machine Learning, Part 3, March 07 Fabi Rli Preamble ØIn Part we have seen that if we knw: Psterir prbabilities P(ω i / ) Or the equivalent terms
More informationPattern Recognition 2014 Support Vector Machines
Pattern Recgnitin 2014 Supprt Vectr Machines Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Pattern Recgnitin 1 / 55 Overview 1 Separable Case 2 Kernel Functins 3 Allwing Errrs (Sft
More informationAssessment Primer: Writing Instructional Objectives
Assessment Primer: Writing Instructinal Objectives (Based n Preparing Instructinal Objectives by Mager 1962 and Preparing Instructinal Objectives: A critical tl in the develpment f effective instructin
More informationDifferentiation Applications 1: Related Rates
Differentiatin Applicatins 1: Related Rates 151 Differentiatin Applicatins 1: Related Rates Mdel 1: Sliding Ladder 10 ladder y 10 ladder 10 ladder A 10 ft ladder is leaning against a wall when the bttm
More informationCAUSAL INFERENCE. Technical Track Session I. Phillippe Leite. The World Bank
CAUSAL INFERENCE Technical Track Sessin I Phillippe Leite The Wrld Bank These slides were develped by Christel Vermeersch and mdified by Phillippe Leite fr the purpse f this wrkshp Plicy questins are causal
More informationIAML: Support Vector Machines
1 / 22 IAML: Supprt Vectr Machines Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester 1 2 / 22 Outline Separating hyperplane with maimum margin Nn-separable training data Epanding the input int
More informationMath Foundations 20 Work Plan
Math Fundatins 20 Wrk Plan Units / Tpics 20.8 Demnstrate understanding f systems f linear inequalities in tw variables. Time Frame December 1-3 weeks 6-10 Majr Learning Indicatrs Identify situatins relevant
More informationx 1 Outline IAML: Logistic Regression Decision Boundaries Example Data
Outline IAML: Lgistic Regressin Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester Lgistic functin Lgistic regressin Learning lgistic regressin Optimizatin The pwer f nn-linear basis functins Least-squares
More informationSimple Linear Regression (single variable)
Simple Linear Regressin (single variable) Intrductin t Machine Learning Marek Petrik January 31, 2017 Sme f the figures in this presentatin are taken frm An Intrductin t Statistical Learning, with applicatins
More informationCOMP9414/ 9814/ 3411: Artificial Intelligence. 14. Course Review. COMP3411 c UNSW, 2014
COMP9414/ 9814/ 3411: Artificial Intelligence 14. Curse Review COMP9414/9814/3411 14s1 Review 1 Assessment Assessable cmpnents f the curse: Assignment 1 10% Assignment 2 8% Assignment 3 12% Written Eam
More informationCHAPTER 3 INEQUALITIES. Copyright -The Institute of Chartered Accountants of India
CHAPTER 3 INEQUALITIES Cpyright -The Institute f Chartered Accuntants f India INEQUALITIES LEARNING OBJECTIVES One f the widely used decisin making prblems, nwadays, is t decide n the ptimal mix f scarce
More informationMath 105: Review for Exam I - Solutions
1. Let f(x) = 3 + x + 5. Math 105: Review fr Exam I - Slutins (a) What is the natural dmain f f? [ 5, ), which means all reals greater than r equal t 5 (b) What is the range f f? [3, ), which means all
More informationCHAPTER 24: INFERENCE IN REGRESSION. Chapter 24: Make inferences about the population from which the sample data came.
MATH 1342 Ch. 24 April 25 and 27, 2013 Page 1 f 5 CHAPTER 24: INFERENCE IN REGRESSION Chapters 4 and 5: Relatinships between tw quantitative variables. Be able t Make a graph (scatterplt) Summarize the
More informationKinetic Model Completeness
5.68J/10.652J Spring 2003 Lecture Ntes Tuesday April 15, 2003 Kinetic Mdel Cmpleteness We say a chemical kinetic mdel is cmplete fr a particular reactin cnditin when it cntains all the species and reactins
More informationFall 2013 Physics 172 Recitation 3 Momentum and Springs
Fall 03 Physics 7 Recitatin 3 Mmentum and Springs Purpse: The purpse f this recitatin is t give yu experience wrking with mmentum and the mmentum update frmula. Readings: Chapter.3-.5 Learning Objectives:.3.
More informationDataflow Analysis and Abstract Interpretation
Dataflw Analysis and Abstract Interpretatin Cmputer Science and Artificial Intelligence Labratry MIT Nvember 9, 2015 Recap Last time we develped frm first principles an algrithm t derive invariants. Key
More informationA New Evaluation Measure. J. Joiner and L. Werner. The problems of evaluation and the needed criteria of evaluation
III-l III. A New Evaluatin Measure J. Jiner and L. Werner Abstract The prblems f evaluatin and the needed criteria f evaluatin measures in the SMART system f infrmatin retrieval are reviewed and discussed.
More informationInternal vs. external validity. External validity. This section is based on Stock and Watson s Chapter 9.
Sectin 7 Mdel Assessment This sectin is based n Stck and Watsn s Chapter 9. Internal vs. external validity Internal validity refers t whether the analysis is valid fr the ppulatin and sample being studied.
More informationSequential Allocation with Minimal Switching
In Cmputing Science and Statistics 28 (1996), pp. 567 572 Sequential Allcatin with Minimal Switching Quentin F. Stut 1 Janis Hardwick 1 EECS Dept., University f Michigan Statistics Dept., Purdue University
More informationGetting Involved O. Responsibilities of a Member. People Are Depending On You. Participation Is Important. Think It Through
f Getting Invlved O Literature Circles can be fun. It is exciting t be part f a grup that shares smething. S get invlved, read, think, and talk abut bks! Respnsibilities f a Member Remember a Literature
More informationk-nearest Neighbor How to choose k Average of k points more reliable when: Large k: noise in attributes +o o noise in class labels
Mtivating Example Memry-Based Learning Instance-Based Learning K-earest eighbr Inductive Assumptin Similar inputs map t similar utputs If nt true => learning is impssible If true => learning reduces t
More informationTrigonometric Ratios Unit 5 Tentative TEST date
1 U n i t 5 11U Date: Name: Trignmetric Ratis Unit 5 Tentative TEST date Big idea/learning Gals In this unit yu will extend yur knwledge f SOH CAH TOA t wrk with btuse and reflex angles. This extensin
More informationChapter 3: Cluster Analysis
Chapter 3: Cluster Analysis } 3.1 Basic Cncepts f Clustering 3.1.1 Cluster Analysis 3.1. Clustering Categries } 3. Partitining Methds 3..1 The principle 3.. K-Means Methd 3..3 K-Medids Methd 3..4 CLARA
More informationLecture 13: Markov Chain Monte Carlo. Gibbs sampling
Lecture 13: Markv hain Mnte arl Gibbs sampling Gibbs sampling Markv chains 1 Recall: Apprximate inference using samples Main idea: we generate samples frm ur Bayes net, then cmpute prbabilities using (weighted)
More informationPart a: Writing the nodal equations and solving for v o gives the magnitude and phase response: tan ( 0.25 )
+ - Hmewrk 0 Slutin ) In the circuit belw: a. Find the magnitude and phase respnse. b. What kind f filter is it? c. At what frequency is the respnse 0.707 if the generatr has a ltage f? d. What is the
More informationCOMP 551 Applied Machine Learning Lecture 11: Support Vector Machines
COMP 551 Applied Machine Learning Lecture 11: Supprt Vectr Machines Instructr: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted fr this curse
More informationCS:4420 Artificial Intelligence
CS:4420 Artificial Intelligence Spring 2017 Learning frm Examples Cesare Tinelli The University f Iwa Cpyright 2004 17, Cesare Tinelli and Stuart Russell a a These ntes were riginally develped by Stuart
More informationElements of Machine Intelligence - I
ECE-175A Elements f Machine Intelligence - I Ken Kreutz-Delgad Nun Vascncels ECE Department, UCSD Winter 2011 The curse The curse will cver basic, but imprtant, aspects f machine learning and pattern recgnitin
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationThis section is primarily focused on tools to aid us in finding roots/zeros/ -intercepts of polynomials. Essentially, our focus turns to solving.
Sectin 3.2: Many f yu WILL need t watch the crrespnding vides fr this sectin n MyOpenMath! This sectin is primarily fcused n tls t aid us in finding rts/zers/ -intercepts f plynmials. Essentially, ur fcus
More informationFive Whys How To Do It Better
Five Whys Definitin. As explained in the previus article, we define rt cause as simply the uncvering f hw the current prblem came int being. Fr a simple causal chain, it is the entire chain. Fr a cmplex
More informationLab 1 The Scientific Method
INTRODUCTION The fllwing labratry exercise is designed t give yu, the student, an pprtunity t explre unknwn systems, r universes, and hypthesize pssible rules which may gvern the behavir within them. Scientific
More informationCOMP 551 Applied Machine Learning Lecture 4: Linear classification
COMP 551 Applied Machine Learning Lecture 4: Linear classificatin Instructr: Jelle Pineau (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted
More informationMATCHING TECHNIQUES. Technical Track Session VI. Emanuela Galasso. The World Bank
MATCHING TECHNIQUES Technical Track Sessin VI Emanuela Galass The Wrld Bank These slides were develped by Christel Vermeersch and mdified by Emanuela Galass fr the purpse f this wrkshp When can we use
More informationActivity Guide Loops and Random Numbers
Unit 3 Lessn 7 Name(s) Perid Date Activity Guide Lps and Randm Numbers CS Cntent Lps are a relatively straightfrward idea in prgramming - yu want a certain chunk f cde t run repeatedly - but it takes a
More informationx x
Mdeling the Dynamics f Life: Calculus and Prbability fr Life Scientists Frederick R. Adler cfrederick R. Adler, Department f Mathematics and Department f Bilgy, University f Utah, Salt Lake City, Utah
More informationNUMBERS, MATHEMATICS AND EQUATIONS
AUSTRALIAN CURRICULUM PHYSICS GETTING STARTED WITH PHYSICS NUMBERS, MATHEMATICS AND EQUATIONS An integral part t the understanding f ur physical wrld is the use f mathematical mdels which can be used t
More informationCOMP9444 Neural Networks and Deep Learning 3. Backpropagation
COMP9444 Neural Netwrks and Deep Learning 3. Backprpagatin Tetbk, Sectins 4.3, 5.2, 6.5.2 COMP9444 17s2 Backprpagatin 1 Outline Supervised Learning Ockham s Razr (5.2) Multi-Layer Netwrks Gradient Descent
More informationPhysics 2010 Motion with Constant Acceleration Experiment 1
. Physics 00 Mtin with Cnstant Acceleratin Experiment In this lab, we will study the mtin f a glider as it accelerates dwnhill n a tilted air track. The glider is supprted ver the air track by a cushin
More informationMaterials Engineering 272-C Fall 2001, Lecture 7 & 8 Fundamentals of Diffusion
Materials Engineering 272-C Fall 2001, Lecture 7 & 8 Fundamentals f Diffusin Diffusin: Transprt in a slid, liquid, r gas driven by a cncentratin gradient (r, in the case f mass transprt, a chemical ptential
More informationSupport-Vector Machines
Supprt-Vectr Machines Intrductin Supprt vectr machine is a linear machine with sme very nice prperties. Haykin chapter 6. See Alpaydin chapter 13 fr similar cntent. Nte: Part f this lecture drew material
More informationComputational modeling techniques
Cmputatinal mdeling techniques Lecture 4: Mdel checing fr ODE mdels In Petre Department f IT, Åb Aademi http://www.users.ab.fi/ipetre/cmpmd/ Cntent Stichimetric matrix Calculating the mass cnservatin relatins
More informationResampling Methods. Cross-validation, Bootstrapping. Marek Petrik 2/21/2017
Resampling Methds Crss-validatin, Btstrapping Marek Petrik 2/21/2017 Sme f the figures in this presentatin are taken frm An Intrductin t Statistical Learning, with applicatins in R (Springer, 2013) with
More informationMedium Scale Integrated (MSI) devices [Sections 2.9 and 2.10]
EECS 270, Winter 2017, Lecture 3 Page 1 f 6 Medium Scale Integrated (MSI) devices [Sectins 2.9 and 2.10] As we ve seen, it s smetimes nt reasnable t d all the design wrk at the gate-level smetimes we just
More informationHypothesis Tests for One Population Mean
Hypthesis Tests fr One Ppulatin Mean Chapter 9 Ala Abdelbaki Objective Objective: T estimate the value f ne ppulatin mean Inferential statistics using statistics in rder t estimate parameters We will be
More informationA Scalable Recurrent Neural Network Framework for Model-free
A Scalable Recurrent Neural Netwrk Framewrk fr Mdel-free POMDPs April 3, 2007 Zhenzhen Liu, Itamar Elhanany Machine Intelligence Lab Department f Electrical and Cmputer Engineering The University f Tennessee
More informationThe Law of Total Probability, Bayes Rule, and Random Variables (Oh My!)
The Law f Ttal Prbability, Bayes Rule, and Randm Variables (Oh My!) Administrivia Hmewrk 2 is psted and is due tw Friday s frm nw If yu didn t start early last time, please d s this time. Gd Milestnes:
More informationThermodynamics Partial Outline of Topics
Thermdynamics Partial Outline f Tpics I. The secnd law f thermdynamics addresses the issue f spntaneity and invlves a functin called entrpy (S): If a prcess is spntaneus, then Suniverse > 0 (2 nd Law!)
More informationFebruary 28, 2013 COMMENTS ON DIFFUSION, DIFFUSIVITY AND DERIVATION OF HYPERBOLIC EQUATIONS DESCRIBING THE DIFFUSION PHENOMENA
February 28, 2013 COMMENTS ON DIFFUSION, DIFFUSIVITY AND DERIVATION OF HYPERBOLIC EQUATIONS DESCRIBING THE DIFFUSION PHENOMENA Mental Experiment regarding 1D randm walk Cnsider a cntainer f gas in thermal
More informationTuring Machines. Human-aware Robotics. 2017/10/17 & 19 Chapter 3.2 & 3.3 in Sipser Ø Announcement:
Turing Machines Human-aware Rbtics 2017/10/17 & 19 Chapter 3.2 & 3.3 in Sipser Ø Annuncement: q q q q Slides fr this lecture are here: http://www.public.asu.edu/~yzhan442/teaching/cse355/lectures/tm-ii.pdf
More informationINSTRUMENTAL VARIABLES
INSTRUMENTAL VARIABLES Technical Track Sessin IV Sergi Urzua University f Maryland Instrumental Variables and IE Tw main uses f IV in impact evaluatin: 1. Crrect fr difference between assignment f treatment
More informationMATCHING TECHNIQUES Technical Track Session VI Céline Ferré The World Bank
MATCHING TECHNIQUES Technical Track Sessin VI Céline Ferré The Wrld Bank When can we use matching? What if the assignment t the treatment is nt dne randmly r based n an eligibility index, but n the basis
More informationResampling Methods. Chapter 5. Chapter 5 1 / 52
Resampling Methds Chapter 5 Chapter 5 1 / 52 1 51 Validatin set apprach 2 52 Crss validatin 3 53 Btstrap Chapter 5 2 / 52 Abut Resampling An imprtant statistical tl Pretending the data as ppulatin and
More informationWe can see from the graph above that the intersection is, i.e., [ ).
MTH 111 Cllege Algebra Lecture Ntes July 2, 2014 Functin Arithmetic: With nt t much difficulty, we ntice that inputs f functins are numbers, and utputs f functins are numbers. S whatever we can d with
More informationCOMP 551 Applied Machine Learning Lecture 5: Generative models for linear classification
COMP 551 Applied Machine Learning Lecture 5: Generative mdels fr linear classificatin Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Jelle Pineau Class web page: www.cs.mcgill.ca/~hvanh2/cmp551
More informationCONSTRUCTING STATECHART DIAGRAMS
CONSTRUCTING STATECHART DIAGRAMS The fllwing checklist shws the necessary steps fr cnstructing the statechart diagrams f a class. Subsequently, we will explain the individual steps further. Checklist 4.6
More informationLab #3: Pendulum Period and Proportionalities
Physics 144 Chwdary Hw Things Wrk Spring 2006 Name: Partners Name(s): Intrductin Lab #3: Pendulum Perid and Prprtinalities Smetimes, it is useful t knw the dependence f ne quantity n anther, like hw the
More informationThermodynamics and Equilibrium
Thermdynamics and Equilibrium Thermdynamics Thermdynamics is the study f the relatinship between heat and ther frms f energy in a chemical r physical prcess. We intrduced the thermdynamic prperty f enthalpy,
More informationPetrel TIPS&TRICKS from SCM
Petrel TIPS&TRICKS frm SCM Knwledge Wrth Sharing Planning a Petrel Prject Ding a Petrel prject is like digging a hle in yur backyard. Yu start knwing hw big the hle is t be, hw deep yu will g, and that
More informationBootstrap Method > # Purpose: understand how bootstrap method works > obs=c(11.96, 5.03, 67.40, 16.07, 31.50, 7.73, 11.10, 22.38) > n=length(obs) >
Btstrap Methd > # Purpse: understand hw btstrap methd wrks > bs=c(11.96, 5.03, 67.40, 16.07, 31.50, 7.73, 11.10, 22.38) > n=length(bs) > mean(bs) [1] 21.64625 > # estimate f lambda > lambda = 1/mean(bs);
More informationWhat is Statistical Learning?
What is Statistical Learning? Sales 5 10 15 20 25 Sales 5 10 15 20 25 Sales 5 10 15 20 25 0 50 100 200 300 TV 0 10 20 30 40 50 Radi 0 20 40 60 80 100 Newspaper Shwn are Sales vs TV, Radi and Newspaper,
More informationLeast Squares Optimal Filtering with Multirate Observations
Prc. 36th Asilmar Cnf. n Signals, Systems, and Cmputers, Pacific Grve, CA, Nvember 2002 Least Squares Optimal Filtering with Multirate Observatins Charles W. herrien and Anthny H. Hawes Department f Electrical
More information, which yields. where z1. and z2
The Gaussian r Nrmal PDF, Page 1 The Gaussian r Nrmal Prbability Density Functin Authr: Jhn M Cimbala, Penn State University Latest revisin: 11 September 13 The Gaussian r Nrmal Prbability Density Functin
More informationPlan o o. I(t) Divide problem into sub-problems Modify schematic and coordinate system (if needed) Write general equations
STAPLE Physics 201 Name Final Exam May 14, 2013 This is a clsed bk examinatin but during the exam yu may refer t a 5 x7 nte card with wrds f wisdm yu have written n it. There is extra scratch paper available.
More informationCOMP 551 Applied Machine Learning Lecture 9: Support Vector Machines (cont d)
COMP 551 Applied Machine Learning Lecture 9: Supprt Vectr Machines (cnt d) Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Class web page: www.cs.mcgill.ca/~hvanh2/cmp551 Unless therwise
More informationA Few Basic Facts About Isothermal Mass Transfer in a Binary Mixture
Few asic Facts but Isthermal Mass Transfer in a inary Miture David Keffer Department f Chemical Engineering University f Tennessee first begun: pril 22, 2004 last updated: January 13, 2006 dkeffer@utk.edu
More informationEcology 302 Lecture III. Exponential Growth (Gotelli, Chapter 1; Ricklefs, Chapter 11, pp )
Eclgy 302 Lecture III. Expnential Grwth (Gtelli, Chapter 1; Ricklefs, Chapter 11, pp. 222-227) Apcalypse nw. The Santa Ana Watershed Prject Authrity pulls n punches in prtraying its missin in apcalyptic
More informationEvaluating enterprise support: state of the art and future challenges. Dirk Czarnitzki KU Leuven, Belgium, and ZEW Mannheim, Germany
Evaluating enterprise supprt: state f the art and future challenges Dirk Czarnitzki KU Leuven, Belgium, and ZEW Mannheim, Germany Intrductin During the last decade, mircecnmetric ecnmetric cunterfactual
More information[COLLEGE ALGEBRA EXAM I REVIEW TOPICS] ( u s e t h i s t o m a k e s u r e y o u a r e r e a d y )
(Abut the final) [COLLEGE ALGEBRA EXAM I REVIEW TOPICS] ( u s e t h i s t m a k e s u r e y u a r e r e a d y ) The department writes the final exam s I dn't really knw what's n it and I can't very well
More informationPublic Key Cryptography. Tim van der Horst & Kent Seamons
Public Key Cryptgraphy Tim van der Hrst & Kent Seamns Last Updated: Oct 5, 2017 Asymmetric Encryptin Why Public Key Crypt is Cl Has a linear slutin t the key distributin prblem Symmetric crypt has an expnential
More informationOptimization Programming Problems For Control And Management Of Bacterial Disease With Two Stage Growth/Spread Among Plants
Internatinal Jurnal f Engineering Science Inventin ISSN (Online): 9 67, ISSN (Print): 9 676 www.ijesi.rg Vlume 5 Issue 8 ugust 06 PP.0-07 Optimizatin Prgramming Prblems Fr Cntrl nd Management Of Bacterial
More informationCS 477/677 Analysis of Algorithms Fall 2007 Dr. George Bebis Course Project Due Date: 11/29/2007
CS 477/677 Analysis f Algrithms Fall 2007 Dr. Gerge Bebis Curse Prject Due Date: 11/29/2007 Part1: Cmparisn f Srting Algrithms (70% f the prject grade) The bjective f the first part f the assignment is
More informationThe standards are taught in the following sequence.
B L U E V A L L E Y D I S T R I C T C U R R I C U L U M MATHEMATICS Third Grade In grade 3, instructinal time shuld fcus n fur critical areas: (1) develping understanding f multiplicatin and divisin and
More informationhttps://goo.gl/eaqvfo SUMMER REV: Half-Life DUE DATE: JULY 2 nd
NAME: DUE DATE: JULY 2 nd AP Chemistry SUMMER REV: Half-Life Why? Every radiistpe has a characteristic rate f decay measured by its half-life. Half-lives can be as shrt as a fractin f a secnd r as lng
More informationWYSE Academic Challenge Regional Mathematics 2007 Solution Set
WYSE Academic Challenge Reginal Mathematics 007 Slutin Set 1. Crrect answer: C. ( ) ( ) 1 + y y = ( + ) + ( y y + 1 ) = + 1 1 ( ) ( 1 + y ) = s *1/ = 1. Crrect answer: A. The determinant is ( 1 ( 1) )
More informationIn SMV I. IAML: Support Vector Machines II. This Time. The SVM optimization problem. We saw:
In SMV I IAML: Supprt Vectr Machines II Nigel Gddard Schl f Infrmatics Semester 1 We sa: Ma margin trick Gemetry f the margin and h t cmpute it Finding the ma margin hyperplane using a cnstrained ptimizatin
More informationMaximum A Posteriori (MAP) CS 109 Lecture 22 May 16th, 2016
Maximum A Psteriri (MAP) CS 109 Lecture 22 May 16th, 2016 Previusly in CS109 Game f Estimatrs Maximum Likelihd Nn spiler: this didn t happen Side Plt argmax argmax f lg Mther f ptimizatins? Reviving an
More informationMODULE 1. e x + c. [You can t separate a demominator, but you can divide a single denominator into each numerator term] a + b a(a + b)+1 = a + b
. REVIEW OF SOME BASIC ALGEBRA MODULE () Slving Equatins Yu shuld be able t slve fr x: a + b = c a d + e x + c and get x = e(ba +) b(c a) d(ba +) c Cmmn mistakes and strategies:. a b + c a b + a c, but
More informationENSC Discrete Time Systems. Project Outline. Semester
ENSC 49 - iscrete Time Systems Prject Outline Semester 006-1. Objectives The gal f the prject is t design a channel fading simulatr. Upn successful cmpletin f the prject, yu will reinfrce yur understanding
More informationGMM with Latent Variables
GMM with Latent Variables A. Rnald Gallant Penn State University Raffaella Giacmini University Cllege Lndn Giuseppe Ragusa Luiss University Cntributin The cntributin f GMM (Hansen and Singletn, 1982) was
More informationDead-beat controller design
J. Hetthéssy, A. Barta, R. Bars: Dead beat cntrller design Nvember, 4 Dead-beat cntrller design In sampled data cntrl systems the cntrller is realised by an intelligent device, typically by a PLC (Prgrammable
More informationSIZE BIAS IN LINE TRANSECT SAMPLING: A FIELD TEST. Mark C. Otto Statistics Research Division, Bureau of the Census Washington, D.C , U.S.A.
SIZE BIAS IN LINE TRANSECT SAMPLING: A FIELD TEST Mark C. Ott Statistics Research Divisin, Bureau f the Census Washingtn, D.C. 20233, U.S.A. and Kenneth H. Pllck Department f Statistics, Nrth Carlina State
More informationINSTRUCTIONAL PLAN Day 2
INSTRUCTIONAL PLAN Day 2 Subject: Trignmetry Tpic: Other Trignmetric Ratis, Relatinships between Trignmetric Ratis, and Inverses Target Learners: Cllege Students Objectives: At the end f the lessn, students
More informationLecture 5: Equilibrium and Oscillations
Lecture 5: Equilibrium and Oscillatins Energy and Mtin Last time, we fund that fr a system with energy cnserved, v = ± E U m ( ) ( ) One result we see immediately is that there is n slutin fr velcity if
More informationToday s s Lecture. Applicability of Neural Networks. Back-propagation. Review of Neural Networks. Lecture 20: Learning -4. Markov-Decision Processes
Today s s Lecture Lecture 20: Learning -4 Review of Neural Networks Markov-Decision Processes Victor Lesser CMPSCI 683 Fall 2004 Reinforcement learning 2 Back-propagation Applicability of Neural Networks
More informationLesson Plan. Recode: They will do a graphic organizer to sequence the steps of scientific method.
Lessn Plan Reach: Ask the students if they ever ppped a bag f micrwave ppcrn and nticed hw many kernels were unppped at the bttm f the bag which made yu wnder if ther brands pp better than the ne yu are
More informationTree Structured Classifier
Tree Structured Classifier Reference: Classificatin and Regressin Trees by L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stne, Chapman & Hall, 98. A Medical Eample (CART): Predict high risk patients
More informationDepartment of Economics, University of California, Davis Ecn 200C Micro Theory Professor Giacomo Bonanno. Insurance Markets
Department f Ecnmics, University f alifrnia, Davis Ecn 200 Micr Thery Prfessr Giacm Bnann Insurance Markets nsider an individual wh has an initial wealth f. ith sme prbability p he faces a lss f x (0
More informationQuery Expansion. Lecture Objectives. Text Technologies for Data Science INFR Learn about Query Expansion. Implement: 10/24/2017
Tet Technlgies fr Data Science INFR11145 Query Epansin Instructr: Walid Magdy 24-Oct-2017 Lecture Objectives Learn abut Query Epansin Query epansin methds Relevance feedback in IR Rcchi s algrithm PRF
More informationHiding in plain sight
Hiding in plain sight Principles f stegangraphy CS349 Cryptgraphy Department f Cmputer Science Wellesley Cllege The prisners prblem Stegangraphy 1-2 1 Secret writing Lemn juice is very nearly clear s it
More informationMODULE FOUR. This module addresses functions. SC Academic Elementary Algebra Standards:
MODULE FOUR This mdule addresses functins SC Academic Standards: EA-3.1 Classify a relatinship as being either a functin r nt a functin when given data as a table, set f rdered pairs, r graph. EA-3.2 Use
More informationANSWER KEY FOR MATH 10 SAMPLE EXAMINATION. Instructions: If asked to label the axes please use real world (contextual) labels
ANSWER KEY FOR MATH 10 SAMPLE EXAMINATION Instructins: If asked t label the axes please use real wrld (cntextual) labels Multiple Chice Answers: 0 questins x 1.5 = 30 Pints ttal Questin Answer Number 1
More informationPurchase Order Workflow Processing
P a g e 1 Purchase Order Wrkflw Prcessing P a g e 2 Table f Cntents PO Wrkflw Prcessing...3 Create a Purchase Order...3 Submit a Purchase Order...4 Review/Apprve the PO...4 Prcess the PO...6 P a g e 3
More informationAccelerated Chemistry POGIL: Half-life
Name: Date: Perid: Accelerated Chemistry POGIL: Half-life Why? Every radiistpe has a characteristic rate f decay measured by its half-life. Half-lives can be as shrt as a fractin f a secnd r as lng as
More information