Admin. MDP Search Trees. Optimal Quantities. Reinforcement Learning
|
|
- Nathan McDowell
- 6 years ago
- Views:
Transcription
1 Admin Reinfrcement Learning Cntent adapted frm Berkeley CS188 MDP Search Trees Each MDP state prjects an expectimax-like search tree Optimal Quantities The value (utility) f a state s: V*(s) = expected utility starting in s and acting ptimally The value (utility) f a q-state (s,a): Q*(s,a) = expected utility starting ut having taken actin a frm state s and (thereafter) acting ptimally The ptimal plicy: π*(s) = ptimal actin frm state s
2 Bellman Equatins Bellman Equatins Definitin f ptimal utility via expectimax recurrence gives a simple nestep lkahead relatinship amngst ptimal utility values V * (s) = max a Q* (s, a) Q* (s, a) = T (s, a, s')"# R(s, a, s') + γ V * (s')$% s' V (s) = max a T (s, a, s')"# R(s, a, s') + γ V * (s')$% * Value Iteratin Bellman equatins characterize the ptimal values: Value Iteratin Cnvergence* Value iteratin cmputes them: Value iteratin is just a fixed pint slutin methd. thugh the Vk vectrs are als interpretable as time-limited values s' The are Bellman equatins, and they characterize ptimal behavir in a way we ll use ver and ver Hw d we knw the Vk vectrs are ging t cnverge? Case 1: If the tree has maximum depth M, then VM hlds the actual untruncated values Case 2: If the discunt is less than 1 Sketch: fr any state Vk and Vk+1 can be viewed as depth k+1 expectimax results in nearly identical search trees The difference is that n the bttm later, Vk+1 has actual rewards while Vk has zers That last layer is at best all RMAX It is at wrst RMIN But everything is discunted by γk that far ut S Vk and Vk+1 are at mst γk max R different S as k increases, the values cnverge
3 Value Iteratin Cnvergence Plicy Lss [Dem] Plicy Iteratin Alternative apprach fr ptimal plicies: Step 1: Plicy evaluatin: calculate utilities fr sme fixed plicy (nt ptimal utilities!) until cnvergence Step 2: Plicy imprvement: update plicy using ne-step lk-ahead with resulting cnverged (but nt ptimal!) utilities as future values This is plicy iteratin It s still ptimal! Can cnverge (much) faster under sme cnditins Plicy Iteratin (EM) Evaluatin: Fr fixed current plicy π, find values with plicy evaluatin: Iterate until values cnverge: Imprvement: Fr fixed values, get a better plicy using plicy extractin One-step lk-ahead:
4 Plicy Iteratin Generalized Plicy Iteratin [Suttn and Bart] Plicy Iteratin Prf (Sketch) Duble Bandits Guaranteed t cnverge: In every step the plicy imprves (because therwise we return nce we get the same plicy twice). Thus every iteratin generates a new plicy. There are a finite number f plicies. In the wrst-case, we may have t iterate thrugh all (num actins) (num states) plicies befre we terminate. Optimality at Cnvergence: k+1 (s) = k (s) By definitin f cnvergence, This means: 8s, V k (s) = max a Ps T (s, a, 0 s0 )[R(s, a, s 0 )+ V k i (s 0 )] Thus, V k (s) satisfies the Bellman equatin, which means V k (s) is a fixed-pint pint slutin t the Bellman equatin, V (s)
5 Duble-Bandit MDP Actins: Blue, Red States: Win, Lse Offline Planning Slving MDPs is ffline planning Yu determine all quantities thrugh cmputatin Yu need t knw the details f the MDP Yu d nt actually play the game! Online Planning New rules! Red s win chance is different. Let s Play! $0 $0 $0 $2 $0 $2 $0 $0 $0
6 What Just Happened? Reinfrcement Learning That wasn t planning, it was learning! Specifically, reinfrcement learning There was an MDP, but yu culdn t slve it with just cmputatin Yu needed t actually act t figure it ut Reinfrcement Learning Example: Learning t Walk Basic idea: Receive feedback in the frm f rewards Agent s utility is defined by the reward functin Must (learn t) act s as t maximize expected rewards All learning is based n bserved samples f utcmes! Initial Training Finished [Khl and Stne, ICRA 2004]
7 Example: Learning t Walk The Crawler! [Tedrake, Zhang, and Seung 2005] [Yu, Prject 3] Reinfrcement Learning Offline (MDPs) vs. Online (RL) Still assume a Markv decisin prcess (MDP): A set f states s 2 S A set f actins (per state) A mdel T (s, a, s 0 ) A reward functin A R(s, a, s 0 ) Still lking fr a plicy (s) New twist: we dn t knw T r R! I.e., we dn t knw which states are gd r what the actins d Must actually try actins and states ut t learn
8 Quiz 1: Reinfrcement Learning Mdel-Based Learning The difference between planning in a knwn Markv Decisin Prcess and reinfrcement learning (RL) is that: In RL the agent desn t knw the transitin mdel T r the reward functin R. In RL the agent desn t knw what its current state is (e.g., desn t knw its wn psitin when acting in a gridwrld). A) T/T B) T/F C) F/T D) F/F Mdel-Based Learning Mdel-Based Idea: Learn an apprximate mdel based n experiences Slve fr values as if the learned mdel were crrect Step 1: Learn empirical MDP mdel Cunt utcmes s fr each s,a Nrmalize t give an estimate f T (s, a, s0 ) Discver each R (s, a, s0 ) when we experience (s,a,s ) Step 2: Slve the learned MDP Fr example, use value iteratin, as befre Example: Mdel-Based Learning
9 Example: Expected Age Gal: Cmpute expected age f CS 151 students Withut P(A), instead cllect samples [a 1,a 2, a N ] Try it! Get in a grup f 3-6 students. Based n yur sample, build a mdel fr the expected graduatin year f students in the class. Why des this wrk? Yu eventually learn the right mdel. Why des this wrk? Samples appear with the right frequencies. What s yur predictin? Quiz 2: Mdel-based Learning Quiz 2: Rapid-Fire Click-in T(A,suth,C)= T(B,east,C)= T(C,suth,E)= T(C,suth,D)= A) 1.0 B) 0.75 C) 0.5 D) 0.25 E) 0.0 What mdel wuld be learned frm the abve bserved episdes?
10 Mdel-Free Learning Passive Reinfrcement Learning Passive Reinfrcement Learning Direct Evaluatin Simplified task: plicy evaluatin Input: a fixed plicy π(s) Yu dn t knw the transitins T(s,a,s ) Yu dn t knw the rewards R(s,a,s ) Gal: learn the state values In this case: Learner is alng fr the ride N chice abut what actins t take Just execute the plicy and learn frm experience This is NOT ffline planning! Yu actually take actins in the wrld. Gal: Cmpute values fr each state under π Idea: Average tgether bserved sample values Act accrding t π Every time yu visit a state, write dwn what the sum f discunted rewards turned ut t be Average thse samples This is called direct evaluatin
11 Example: Direct Evaluatin Prblems with Direct Evaluatin What s gd abut direct evaluatin? It s easy t understand It desn t require any knwledge f T, R It eventually cmputes the crrect average values, using just sample transitins What s bad abut it? It wastes infrmatin abut state cnnectins Each state must be learned separately S, it takes a lng time t learn Why Nt Use Plicy Evaluatin? Simplified Bellman updates calculate V fr a fixed plicy: Each rund, replace V with a ne-step-lk-ahead layer ver V Quiz 3: Passive Reinfrcement Learning Estimate the utput values f the fllwing: What gives? This apprach fully explited the cnnectins between the states Unfrtunately, we need T and R t d it! Key questin: hw can we d this update t V withut knwing T and R? In ther wrds, hw d we take a weighted average withut knwing the weights?
12 Quiz 4: Rapid-Fire Click-in V π (A)= V π (B)= V π (C)= V π (D)= V π (E)= A) 10 B) 8 C) 4 D) -2 E) -10 Sample-Based Plicy Evaluatin? We want t imprve ur estimate f V by cmputing these averages: Idea: Take samples f utcmes s (by ding the actin!) and average Sample-Based Plicy Evaluatin? We want t imprve ur estimate f V by cmputing these averages: Idea: Take samples f utcmes s (by ding the actin!) and average Tempral Difference Learning Big idea: learn frm every experience! Update V(s) each time we experience a transitin (s,a,s;,r) Likely utcmes s will cntribute updates mre ften Tempral difference learning f values Plicy still fixed, still ding evaluatin! Mve values tward value f whatever successr ccurs: running average
13 TD Dem! Expnential Mving Average Expnential mving average The running interplatin update: Makes recent samples mre imprtant: Frgets abut the past (distant past values were wrng anyway) Decreasing learning rate (alpha) can give cnverging averages Prblems with TD Value Learning Quiz 4: TD Learning TD value learning is a mdel-free way t d plicy evaluatin, mimicking Bellman updates with running sample averages Hwever, if we want t turn values int a (new) plicy, we re sunk: Idea: learn Q-values, nt values Makes actin selectin mdel-free t!
14 Active Reinfrcement Learning Active Reinfrcement Learning Full reinfrcement learning: ptimal plicies (like value iteratin) Yu dn t knw the transitins T(s,a,s ) Yu dn t knw the rewards R(s,a,s ) Yu chse the actins nw Gal: learn the ptimal plicy / values In this case: Learner makes chices! Fundamental tradeff: explratin vs. explitatin This is NOT ffline planning! Yu actually take actins in the wrld and find ut what happens Detur Q-Value Iteratin Value iteratin: find successive (depth-limited) values Q-Learning Q-Learning: sample-based Q-value iteratin Learn Q(s,a) values as yu g Start with V0(s)=0, which we knw is right Given Vk, calculate the depth k+1 values fr all states: But Q-values are mre useful, s cmpute them instead! Start with Q0(s,a)=0, which we knw is right Given Qk, calculate the depth k+1 q-values fr all q-states: Receive a sample (s,a,s,r) Cnsider yur ld estimate: Cnsider yur new sample estimate: Incrprate the new estimate int a running average:
15 Q-Learning Dem: Gridwrld Q-Learning Dem: Crawler Q-Learning Prperties Amazing result: Q-learning cnverges t ptimal plicy even if yu re acting subptimally! Quiz 5: Q-Learning Which f the fllwing equatins is the Q-value iteratin update? This is called ff-plicy learning Caveats: Yu have t explre enugh Yu have t eventually make the learning rate small enugh but nt decrease it t quickly Basically, in the limit, it desn t matter hw yu select actins(!) T/F If α=1, n averaging will happen --- instead simply the value frm the sample will be used. T/F If α=0, then the sample will nt influence the update.
16 Next time Amazing result: Q-learning cnverges t ptimal plicy even if yu re acting subptimally! but hw d we select actins? what if ur state/actin space is t large t maintain?
Reinforcement Learning" CMPSCI 383 Nov 29, 2011!
Reinfrcement Learning" CMPSCI 383 Nv 29, 2011! 1 Tdayʼs lecture" Review f Chapter 17: Making Cmple Decisins! Sequential decisin prblems! The mtivatin and advantages f reinfrcement learning.! Passive learning!
More informationFive Whys How To Do It Better
Five Whys Definitin. As explained in the previus article, we define rt cause as simply the uncvering f hw the current prblem came int being. Fr a simple causal chain, it is the entire chain. Fr a cmplex
More informationTrigonometric Ratios Unit 5 Tentative TEST date
1 U n i t 5 11U Date: Name: Trignmetric Ratis Unit 5 Tentative TEST date Big idea/learning Gals In this unit yu will extend yur knwledge f SOH CAH TOA t wrk with btuse and reflex angles. This extensin
More informationFall 2013 Physics 172 Recitation 3 Momentum and Springs
Fall 03 Physics 7 Recitatin 3 Mmentum and Springs Purpse: The purpse f this recitatin is t give yu experience wrking with mmentum and the mmentum update frmula. Readings: Chapter.3-.5 Learning Objectives:.3.
More informationThis section is primarily focused on tools to aid us in finding roots/zeros/ -intercepts of polynomials. Essentially, our focus turns to solving.
Sectin 3.2: Many f yu WILL need t watch the crrespnding vides fr this sectin n MyOpenMath! This sectin is primarily fcused n tls t aid us in finding rts/zers/ -intercepts f plynmials. Essentially, ur fcus
More informationYou need to be able to define the following terms and answer basic questions about them:
CS440/ECE448 Sectin Q Fall 2017 Midterm Review Yu need t be able t define the fllwing terms and answer basic questins abut them: Intr t AI, agents and envirnments Pssible definitins f AI, prs and cns f
More informationAdministrativia. Assignment 1 due thursday 9/23/2004 BEFORE midnight. Midterm exam 10/07/2003 in class. CS 460, Sessions 8-9 1
Administrativia Assignment 1 due thursday 9/23/2004 BEFORE midnight Midterm eam 10/07/2003 in class CS 460, Sessins 8-9 1 Last time: search strategies Uninfrmed: Use nly infrmatin available in the prblem
More informationGetting Involved O. Responsibilities of a Member. People Are Depending On You. Participation Is Important. Think It Through
f Getting Invlved O Literature Circles can be fun. It is exciting t be part f a grup that shares smething. S get invlved, read, think, and talk abut bks! Respnsibilities f a Member Remember a Literature
More informationLecture 7: Damped and Driven Oscillations
Lecture 7: Damped and Driven Oscillatins Last time, we fund fr underdamped scillatrs: βt x t = e A1 + A csω1t + i A1 A sinω1t A 1 and A are cmplex numbers, but ur answer must be real Implies that A 1 and
More informationCAUSAL INFERENCE. Technical Track Session I. Phillippe Leite. The World Bank
CAUSAL INFERENCE Technical Track Sessin I Phillippe Leite The Wrld Bank These slides were develped by Christel Vermeersch and mdified by Phillippe Leite fr the purpse f this wrkshp Plicy questins are causal
More informationComputational modeling techniques
Cmputatinal mdeling techniques Lecture 2: Mdeling change. In Petre Department f IT, Åb Akademi http://users.ab.fi/ipetre/cmpmd/ Cntent f the lecture Basic paradigm f mdeling change Examples Linear dynamical
More informationSequential Allocation with Minimal Switching
In Cmputing Science and Statistics 28 (1996), pp. 567 572 Sequential Allcatin with Minimal Switching Quentin F. Stut 1 Janis Hardwick 1 EECS Dept., University f Michigan Statistics Dept., Purdue University
More information[COLLEGE ALGEBRA EXAM I REVIEW TOPICS] ( u s e t h i s t o m a k e s u r e y o u a r e r e a d y )
(Abut the final) [COLLEGE ALGEBRA EXAM I REVIEW TOPICS] ( u s e t h i s t m a k e s u r e y u a r e r e a d y ) The department writes the final exam s I dn't really knw what's n it and I can't very well
More informationNUMBERS, MATHEMATICS AND EQUATIONS
AUSTRALIAN CURRICULUM PHYSICS GETTING STARTED WITH PHYSICS NUMBERS, MATHEMATICS AND EQUATIONS An integral part t the understanding f ur physical wrld is the use f mathematical mdels which can be used t
More informationKinetic Model Completeness
5.68J/10.652J Spring 2003 Lecture Ntes Tuesday April 15, 2003 Kinetic Mdel Cmpleteness We say a chemical kinetic mdel is cmplete fr a particular reactin cnditin when it cntains all the species and reactins
More informationSPH3U1 Lesson 06 Kinematics
PROJECTILE MOTION LEARNING GOALS Students will: Describe the mtin f an bject thrwn at arbitrary angles thrugh the air. Describe the hrizntal and vertical mtins f a prjectile. Slve prjectile mtin prblems.
More informationInternal vs. external validity. External validity. This section is based on Stock and Watson s Chapter 9.
Sectin 7 Mdel Assessment This sectin is based n Stck and Watsn s Chapter 9. Internal vs. external validity Internal validity refers t whether the analysis is valid fr the ppulatin and sample being studied.
More informationLab 1 The Scientific Method
INTRODUCTION The fllwing labratry exercise is designed t give yu, the student, an pprtunity t explre unknwn systems, r universes, and hypthesize pssible rules which may gvern the behavir within them. Scientific
More informationWe can see from the graph above that the intersection is, i.e., [ ).
MTH 111 Cllege Algebra Lecture Ntes July 2, 2014 Functin Arithmetic: With nt t much difficulty, we ntice that inputs f functins are numbers, and utputs f functins are numbers. S whatever we can d with
More informationCOMP 551 Applied Machine Learning Lecture 4: Linear classification
COMP 551 Applied Machine Learning Lecture 4: Linear classificatin Instructr: Jelle Pineau (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted
More informationDifferentiation Applications 1: Related Rates
Differentiatin Applicatins 1: Related Rates 151 Differentiatin Applicatins 1: Related Rates Mdel 1: Sliding Ladder 10 ladder y 10 ladder 10 ladder A 10 ft ladder is leaning against a wall when the bttm
More informationAP Physics Kinematic Wrap Up
AP Physics Kinematic Wrap Up S what d yu need t knw abut this mtin in tw-dimensin stuff t get a gd scre n the ld AP Physics Test? First ff, here are the equatins that yu ll have t wrk with: v v at x x
More informationTuring Machines. Human-aware Robotics. 2017/10/17 & 19 Chapter 3.2 & 3.3 in Sipser Ø Announcement:
Turing Machines Human-aware Rbtics 2017/10/17 & 19 Chapter 3.2 & 3.3 in Sipser Ø Annuncement: q q q q Slides fr this lecture are here: http://www.public.asu.edu/~yzhan442/teaching/cse355/lectures/tm-ii.pdf
More informationCOMP 551 Applied Machine Learning Lecture 11: Support Vector Machines
COMP 551 Applied Machine Learning Lecture 11: Supprt Vectr Machines Instructr: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted fr this curse
More informationLab #3: Pendulum Period and Proportionalities
Physics 144 Chwdary Hw Things Wrk Spring 2006 Name: Partners Name(s): Intrductin Lab #3: Pendulum Perid and Prprtinalities Smetimes, it is useful t knw the dependence f ne quantity n anther, like hw the
More informationCorrections for the textbook answers: Sec 6.1 #8h)covert angle to a positive by adding period #9b) # rad/sec
U n i t 6 AdvF Date: Name: Trignmetric Functins Unit 6 Tentative TEST date Big idea/learning Gals In this unit yu will study trignmetric functins frm grade, hwever everything will be dne in radian measure.
More informationSection 5.8 Notes Page Exponential Growth and Decay Models; Newton s Law
Sectin 5.8 Ntes Page 1 5.8 Expnential Grwth and Decay Mdels; Newtn s Law There are many applicatins t expnential functins that we will fcus n in this sectin. First let s lk at the expnential mdel. Expnential
More informationPhysics 212. Lecture 12. Today's Concept: Magnetic Force on moving charges. Physics 212 Lecture 12, Slide 1
Physics 1 Lecture 1 Tday's Cncept: Magnetic Frce n mving charges F qv Physics 1 Lecture 1, Slide 1 Music Wh is the Artist? A) The Meters ) The Neville rthers C) Trmbne Shrty D) Michael Franti E) Radiatrs
More informationI. Analytical Potential and Field of a Uniform Rod. V E d. The definition of electric potential difference is
Length L>>a,b,c Phys 232 Lab 4 Ch 17 Electric Ptential Difference Materials: whitebards & pens, cmputers with VPythn, pwer supply & cables, multimeter, crkbard, thumbtacks, individual prbes and jined prbes,
More informationMaximum A Posteriori (MAP) CS 109 Lecture 22 May 16th, 2016
Maximum A Psteriri (MAP) CS 109 Lecture 22 May 16th, 2016 Previusly in CS109 Game f Estimatrs Maximum Likelihd Nn spiler: this didn t happen Side Plt argmax argmax f lg Mther f ptimizatins? Reviving an
More informationActivity Guide Loops and Random Numbers
Unit 3 Lessn 7 Name(s) Perid Date Activity Guide Lps and Randm Numbers CS Cntent Lps are a relatively straightfrward idea in prgramming - yu want a certain chunk f cde t run repeatedly - but it takes a
More informationCOMP 551 Applied Machine Learning Lecture 5: Generative models for linear classification
COMP 551 Applied Machine Learning Lecture 5: Generative mdels fr linear classificatin Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Jelle Pineau Class web page: www.cs.mcgill.ca/~hvanh2/cmp551
More informationMath Foundations 20 Work Plan
Math Fundatins 20 Wrk Plan Units / Tpics 20.8 Demnstrate understanding f systems f linear inequalities in tw variables. Time Frame December 1-3 weeks 6-10 Majr Learning Indicatrs Identify situatins relevant
More informationCS 477/677 Analysis of Algorithms Fall 2007 Dr. George Bebis Course Project Due Date: 11/29/2007
CS 477/677 Analysis f Algrithms Fall 2007 Dr. Gerge Bebis Curse Prject Due Date: 11/29/2007 Part1: Cmparisn f Srting Algrithms (70% f the prject grade) The bjective f the first part f the assignment is
More informationMath 105: Review for Exam I - Solutions
1. Let f(x) = 3 + x + 5. Math 105: Review fr Exam I - Slutins (a) What is the natural dmain f f? [ 5, ), which means all reals greater than r equal t 5 (b) What is the range f f? [3, ), which means all
More information, which yields. where z1. and z2
The Gaussian r Nrmal PDF, Page 1 The Gaussian r Nrmal Prbability Density Functin Authr: Jhn M Cimbala, Penn State University Latest revisin: 11 September 13 The Gaussian r Nrmal Prbability Density Functin
More informationDepartment of Economics, University of California, Davis Ecn 200C Micro Theory Professor Giacomo Bonanno. Insurance Markets
Department f Ecnmics, University f alifrnia, Davis Ecn 200 Micr Thery Prfessr Giacm Bnann Insurance Markets nsider an individual wh has an initial wealth f. ith sme prbability p he faces a lss f x (0
More informationMedium Scale Integrated (MSI) devices [Sections 2.9 and 2.10]
EECS 270, Winter 2017, Lecture 3 Page 1 f 6 Medium Scale Integrated (MSI) devices [Sectins 2.9 and 2.10] As we ve seen, it s smetimes nt reasnable t d all the design wrk at the gate-level smetimes we just
More informationModelling of Clock Behaviour. Don Percival. Applied Physics Laboratory University of Washington Seattle, Washington, USA
Mdelling f Clck Behaviur Dn Percival Applied Physics Labratry University f Washingtn Seattle, Washingtn, USA verheads and paper fr talk available at http://faculty.washingtn.edu/dbp/talks.html 1 Overview
More informationCHAPTER 3 INEQUALITIES. Copyright -The Institute of Chartered Accountants of India
CHAPTER 3 INEQUALITIES Cpyright -The Institute f Chartered Accuntants f India INEQUALITIES LEARNING OBJECTIVES One f the widely used decisin making prblems, nwadays, is t decide n the ptimal mix f scarce
More informationDataflow Analysis and Abstract Interpretation
Dataflw Analysis and Abstract Interpretatin Cmputer Science and Artificial Intelligence Labratry MIT Nvember 9, 2015 Recap Last time we develped frm first principles an algrithm t derive invariants. Key
More informationLecture 13: Markov Chain Monte Carlo. Gibbs sampling
Lecture 13: Markv hain Mnte arl Gibbs sampling Gibbs sampling Markv chains 1 Recall: Apprximate inference using samples Main idea: we generate samples frm ur Bayes net, then cmpute prbabilities using (weighted)
More informationLecture 5: Equilibrium and Oscillations
Lecture 5: Equilibrium and Oscillatins Energy and Mtin Last time, we fund that fr a system with energy cnserved, v = ± E U m ( ) ( ) One result we see immediately is that there is n slutin fr velcity if
More informationBuilding Consensus The Art of Getting to Yes
Building Cnsensus The Art f Getting t Yes An interview with Michael Wilkinsn, Certified Master Facilitatr and authr f The Secrets f Facilitatin and The Secrets t Masterful Meetings Abut Michael: Mr. Wilkinsn
More informationPattern Recognition 2014 Support Vector Machines
Pattern Recgnitin 2014 Supprt Vectr Machines Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Pattern Recgnitin 1 / 55 Overview 1 Separable Case 2 Kernel Functins 3 Allwing Errrs (Sft
More informationCHAPTER 24: INFERENCE IN REGRESSION. Chapter 24: Make inferences about the population from which the sample data came.
MATH 1342 Ch. 24 April 25 and 27, 2013 Page 1 f 5 CHAPTER 24: INFERENCE IN REGRESSION Chapters 4 and 5: Relatinships between tw quantitative variables. Be able t Make a graph (scatterplt) Summarize the
More informationIntroduction to Models and Properties
Intrductin t Mdels and Prperties Cmputer Science and Artificial Intelligence Labratry MIT Armand Slar-Lezama Nv 23, 2015 Nvember 23, 2015 1 Recap Prperties Prperties f variables Prperties at prgram pints
More informationInstructional Plan. Representational/Drawing Level
Instructinal Plan Representatinal/Drawing Level Name f Math Skill/Cncept: Divisin Prcess and Divisin with Remainders Prerequisite Skills Needed: 1.) Mastery f dividing cncrete bjects int equal grups. 2.)
More informationk-nearest Neighbor How to choose k Average of k points more reliable when: Large k: noise in attributes +o o noise in class labels
Mtivating Example Memry-Based Learning Instance-Based Learning K-earest eighbr Inductive Assumptin Similar inputs map t similar utputs If nt true => learning is impssible If true => learning reduces t
More informationName: Block: Date: Science 10: The Great Geyser Experiment A controlled experiment
Science 10: The Great Geyser Experiment A cntrlled experiment Yu will prduce a GEYSER by drpping Ments int a bttle f diet pp Sme questins t think abut are: What are yu ging t test? What are yu ging t measure?
More informationAn Introduction to COMPUTATIONAL REINFORCEMENT LEARING. Andrew G. Barto. Department of Computer Science University of Massachusetts Amherst
An Intrductin t COMPUTATIONAL REINFORCEMENT LEARING Andrew G. Bart Department f Cmputer Science University f Massachusetts Amherst UPF Lecture 1 Autnmus Learning Labratry Department f Cmputer Science Artificial
More informationPHYS 314 HOMEWORK #3
PHYS 34 HOMEWORK #3 Due : 8 Feb. 07. A unifrm chain f mass M, lenth L and density λ (measured in k/m) hans s that its bttm link is just tuchin a scale. The chain is drpped frm rest nt the scale. What des
More informationENSC Discrete Time Systems. Project Outline. Semester
ENSC 49 - iscrete Time Systems Prject Outline Semester 006-1. Objectives The gal f the prject is t design a channel fading simulatr. Upn successful cmpletin f the prject, yu will reinfrce yur understanding
More informationMODULE 1. e x + c. [You can t separate a demominator, but you can divide a single denominator into each numerator term] a + b a(a + b)+1 = a + b
. REVIEW OF SOME BASIC ALGEBRA MODULE () Slving Equatins Yu shuld be able t slve fr x: a + b = c a d + e x + c and get x = e(ba +) b(c a) d(ba +) c Cmmn mistakes and strategies:. a b + c a b + a c, but
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationIn the OLG model, agents live for two periods. they work and divide their labour income between consumption and
1 The Overlapping Generatins Mdel (OLG) In the OLG mdel, agents live fr tw perids. When ung the wrk and divide their labur incme between cnsumptin and savings. When ld the cnsume their savings. As the
More informationAP Literature and Composition. Summer Reading Packet. Instructions and Guidelines
AP Literature and Cmpsitin Summer Reading Packet Instructins and Guidelines Accrding t the Cllege Bard Advanced Placement prgram: "The AP English curse in Literature and Cmpsitin shuld engage students
More informationHow do scientists measure trees? What is DBH?
Hw d scientists measure trees? What is DBH? Purpse Students develp an understanding f tree size and hw scientists measure trees. Students bserve and measure tree ckies and explre the relatinship between
More informationResampling Methods. Cross-validation, Bootstrapping. Marek Petrik 2/21/2017
Resampling Methds Crss-validatin, Btstrapping Marek Petrik 2/21/2017 Sme f the figures in this presentatin are taken frm An Intrductin t Statistical Learning, with applicatins in R (Springer, 2013) with
More informationx x
Mdeling the Dynamics f Life: Calculus and Prbability fr Life Scientists Frederick R. Adler cfrederick R. Adler, Department f Mathematics and Department f Bilgy, University f Utah, Salt Lake City, Utah
More informationAssociated Students Flacks Internship
Assciated Students Flacks Internship 2016-2017 Applicatin Persnal Infrmatin: Name: Address: Phne #: Years at UCSB: Cumulative GPA: E-mail: Majr(s)/Minr(s): Units Cmpleted: Tw persnal references (Different
More informationPhysics 2010 Motion with Constant Acceleration Experiment 1
. Physics 00 Mtin with Cnstant Acceleratin Experiment In this lab, we will study the mtin f a glider as it accelerates dwnhill n a tilted air track. The glider is supprted ver the air track by a cushin
More informationLesson Plan. Recode: They will do a graphic organizer to sequence the steps of scientific method.
Lessn Plan Reach: Ask the students if they ever ppped a bag f micrwave ppcrn and nticed hw many kernels were unppped at the bttm f the bag which made yu wnder if ther brands pp better than the ne yu are
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationCESAR Science Case The differential rotation of the Sun and its Chromosphere. Introduction. Material that is necessary during the laboratory
Teacher s guide CESAR Science Case The differential rtatin f the Sun and its Chrmsphere Material that is necessary during the labratry CESAR Astrnmical wrd list CESAR Bklet CESAR Frmula sheet CESAR Student
More informationThe Law of Total Probability, Bayes Rule, and Random Variables (Oh My!)
The Law f Ttal Prbability, Bayes Rule, and Randm Variables (Oh My!) Administrivia Hmewrk 2 is psted and is due tw Friday s frm nw If yu didn t start early last time, please d s this time. Gd Milestnes:
More informationBASD HIGH SCHOOL FORMAL LAB REPORT
BASD HIGH SCHOOL FORMAL LAB REPORT *WARNING: After an explanatin f what t include in each sectin, there is an example f hw the sectin might lk using a sample experiment Keep in mind, the sample lab used
More informationPart a: Writing the nodal equations and solving for v o gives the magnitude and phase response: tan ( 0.25 )
+ - Hmewrk 0 Slutin ) In the circuit belw: a. Find the magnitude and phase respnse. b. What kind f filter is it? c. At what frequency is the respnse 0.707 if the generatr has a ltage f? d. What is the
More informationProfessional Development. Implementing the NGSS: High School Physics
Prfessinal Develpment Implementing the NGSS: High Schl Physics This is a dem. The 30-min vide webinar is available in the full PD. Get it here. Tday s Learning Objectives NGSS key cncepts why this is different
More informationDepartment of Electrical Engineering, University of Waterloo. Introduction
Sectin 4: Sequential Circuits Majr Tpics Types f sequential circuits Flip-flps Analysis f clcked sequential circuits Mre and Mealy machines Design f clcked sequential circuits State transitin design methd
More informationLim f (x) e. Find the largest possible domain and its discontinuity points. Why is it discontinuous at those points (if any)?
THESE ARE SAMPLE QUESTIONS FOR EACH OF THE STUDENT LEARNING OUTCOMES (SLO) SET FOR THIS COURSE. SLO 1: Understand and use the cncept f the limit f a functin i. Use prperties f limits and ther techniques,
More informationA New Evaluation Measure. J. Joiner and L. Werner. The problems of evaluation and the needed criteria of evaluation
III-l III. A New Evaluatin Measure J. Jiner and L. Werner Abstract The prblems f evaluatin and the needed criteria f evaluatin measures in the SMART system f infrmatin retrieval are reviewed and discussed.
More informationhttps://goo.gl/eaqvfo SUMMER REV: Half-Life DUE DATE: JULY 2 nd
NAME: DUE DATE: JULY 2 nd AP Chemistry SUMMER REV: Half-Life Why? Every radiistpe has a characteristic rate f decay measured by its half-life. Half-lives can be as shrt as a fractin f a secnd r as lng
More information1 Course Notes in Introductory Physics Jeffrey Seguritan
Intrductin & Kinematics I Intrductin Quickie Cncepts Units SI is standard system f units used t measure physical quantities. Base units that we use: meter (m) is standard unit f length kilgram (kg) is
More informationChurn Prediction using Dynamic RFM-Augmented node2vec
Churn Predictin using Dynamic RFM-Augmented nde2vec Sandra Mitrvić, Jchen de Weerdt, Bart Baesens & Wilfried Lemahieu Department f Decisin Sciences and Infrmatin Management, KU Leuven 18 September 2017,
More informationThe steps of the engineering design process are to:
The engineering design prcess is a series f steps that engineers fllw t cme up with a slutin t a prblem. Many times the slutin invlves designing a prduct (like a machine r cmputer cde) that meets certain
More informationCHAPTER 6 -- ENERGY. Approach #2: Using the component of mg along the line of d:
Slutins--Ch. 6 (Energy) CHAPTER 6 -- ENERGY 6.) The f.b.d. shwn t the right has been prvided t identify all the frces acting n the bdy as it mves up the incline. a.) T determine the wrk dne by gravity
More informationKepler's Laws of Planetary Motion
Writing Assignment Essay n Kepler s Laws. Yu have been prvided tw shrt articles n Kepler s Three Laws f Planetary Mtin. Yu are t first read the articles t better understand what these laws are, what they
More informationDetermining Optimum Path in Synthesis of Organic Compounds using Branch and Bound Algorithm
Determining Optimum Path in Synthesis f Organic Cmpunds using Branch and Bund Algrithm Diastuti Utami 13514071 Prgram Studi Teknik Infrmatika Seklah Teknik Elektr dan Infrmatika Institut Teknlgi Bandung,
More informationx 1 Outline IAML: Logistic Regression Decision Boundaries Example Data
Outline IAML: Lgistic Regressin Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester Lgistic functin Lgistic regressin Learning lgistic regressin Optimizatin The pwer f nn-linear basis functins Least-squares
More informationWho is the Holy Spirit?
ill at w w this h t h in SS est abut erence u O q L G ka iff hink : As m t t es a d K S k A the n ma. wn help rmati ur Jesus. y f t u inf e life ab h iple in t alk a disc f T : RE ce as ece t i A p SH
More informationCS 188: Artificial Intelligence
CS 188: Artificial Intelligence Reinforcement Learning Instructor: Fabrice Popineau [These slides adapted from Stuart Russell, Dan Klein and Pieter Abbeel @ai.berkeley.edu] Reinforcement Learning Double
More informationAssessment Primer: Writing Instructional Objectives
Assessment Primer: Writing Instructinal Objectives (Based n Preparing Instructinal Objectives by Mager 1962 and Preparing Instructinal Objectives: A critical tl in the develpment f effective instructin
More informationLab 11 LRC Circuits, Damped Forced Harmonic Motion
Physics 6 ab ab 11 ircuits, Damped Frced Harmnic Mtin What Yu Need T Knw: The Physics OK this is basically a recap f what yu ve dne s far with circuits and circuits. Nw we get t put everything tgether
More informationIntroduction to Spacetime Geometry
Intrductin t Spacetime Gemetry Let s start with a review f a basic feature f Euclidean gemetry, the Pythagrean therem. In a twdimensinal crdinate system we can relate the length f a line segment t the
More informationPreparation work for A2 Mathematics [2017]
Preparatin wrk fr A2 Mathematics [2017] The wrk studied in Y12 after the return frm study leave is frm the Cre 3 mdule f the A2 Mathematics curse. This wrk will nly be reviewed during Year 13, it will
More informationPlan o o. I(t) Divide problem into sub-problems Modify schematic and coordinate system (if needed) Write general equations
STAPLE Physics 201 Name Final Exam May 14, 2013 This is a clsed bk examinatin but during the exam yu may refer t a 5 x7 nte card with wrds f wisdm yu have written n it. There is extra scratch paper available.
More informationWhy Don t They Get It??
Why Dn t They Get It?? A 60-minute Webinar NEURO LINGUISTIC PROGRAMMING NLP is the way we stre and prcess infrmatin in ur brains, and then frm the wrds we use t cmmunicate. By learning abut NLP, yu can
More informationINSTRUCTIONAL PLAN Day 2
INSTRUCTIONAL PLAN Day 2 Subject: Trignmetry Tpic: Other Trignmetric Ratis, Relatinships between Trignmetric Ratis, and Inverses Target Learners: Cllege Students Objectives: At the end f the lessn, students
More informationSlide04 (supplemental) Haykin Chapter 4 (both 2nd and 3rd ed): Multi-Layer Perceptrons
Slide04 supplemental) Haykin Chapter 4 bth 2nd and 3rd ed): Multi-Layer Perceptrns CPSC 636-600 Instructr: Ynsuck Che Heuristic fr Making Backprp Perfrm Better 1. Sequential vs. batch update: fr large
More information1 PreCalculus AP Unit G Rotational Trig (MCR) Name:
1 PreCalculus AP Unit G Rtatinal Trig (MCR) Name: Big idea In this unit yu will extend yur knwledge f SOH CAH TOA t wrk with btuse and reflex angles. This extensin will invlve the unit circle which will
More informationINSTRUMENTAL VARIABLES
INSTRUMENTAL VARIABLES Technical Track Sessin IV Sergi Urzua University f Maryland Instrumental Variables and IE Tw main uses f IV in impact evaluatin: 1. Crrect fr difference between assignment f treatment
More information2004 AP CHEMISTRY FREE-RESPONSE QUESTIONS
2004 AP CHEMISTRY FREE-RESPONSE QUESTIONS 6. An electrchemical cell is cnstructed with an pen switch, as shwn in the diagram abve. A strip f Sn and a strip f an unknwn metal, X, are used as electrdes.
More informationEngineering Decision Methods
GSOE9210 vicj@cse.unsw.edu.au www.cse.unsw.edu.au/~gs9210 Maximin and minimax regret 1 2 Indifference; equal preference 3 Graphing decisin prblems 4 Dminance The Maximin principle Maximin and minimax Regret
More informationNAME: Prof. Ruiz. 1. [5 points] What is the difference between simple random sampling and stratified random sampling?
CS4445 ata Mining and Kwledge iscery in atabases. B Term 2014 Exam 1 Nember 24, 2014 Prf. Carlina Ruiz epartment f Cmputer Science Wrcester Plytechnic Institute NAME: Prf. Ruiz Prblem I: Prblem II: Prblem
More informationTP1 - Introduction to ArcGIS
TP1 - Intrductin t ArcGIS During this practical, we will use ArcGIS (ArcMap and ArcCatalg) t create maps f predictrs that culd explain the bserved bird richness in Switzerland. ArcMap is principally used
More informationExperiment #3. Graphing with Excel
Experiment #3. Graphing with Excel Study the "Graphing with Excel" instructins that have been prvided. Additinal help with learning t use Excel can be fund n several web sites, including http://www.ncsu.edu/labwrite/res/gt/gt-
More informationRelationships Between Frequency, Capacitance, Inductance and Reactance.
P Physics Relatinships between f,, and. Relatinships Between Frequency, apacitance, nductance and Reactance. Purpse: T experimentally verify the relatinships between f, and. The data cllected will lead
More informationMaterials Engineering 272-C Fall 2001, Lecture 7 & 8 Fundamentals of Diffusion
Materials Engineering 272-C Fall 2001, Lecture 7 & 8 Fundamentals f Diffusin Diffusin: Transprt in a slid, liquid, r gas driven by a cncentratin gradient (r, in the case f mass transprt, a chemical ptential
More informationA Quick Overview of the. Framework for K 12 Science Education
A Quick Overview f the NGSS EQuIP MODULE 1 Framewrk fr K 12 Science Educatin Mdule 1: A Quick Overview f the Framewrk fr K 12 Science Educatin This mdule prvides a brief backgrund n the Framewrk fr K-12
More information