Reinforcement learning


 Milo Conley
 4 years ago
 Views:
Transcription
1 CS 75 Mchine Lening Lecue b einfocemen lening Milos Huskech 539 Senno Sque einfocemen lening We wn o len conol policy: : X A We see emples of bu oupus e no given Insed of we ge feedbck einfocemen ewd fom ciic qunifying how good he seleced oupu ws Inpu Lene Oupu einfocemen Ciic he einfocemens my no be deeminisic Gol: find : X A wih he bes epeced einfocemens
2 Gmbling emple Gme: 3 bised coins 3 he coin o be ossed is seleced ndomly fom he hee coin opions. he gen lwys sees which coin is going o be plyed ne. he gen mkes be on eihe hed o il wih wge of $. If fe he coin oss he oucome gees wih he be he gen wins $ ohewise i looses $ L model: Inpu: X coin chosen fo he ne oss Acion: A choice of hed o il he gen bes on einfocemens: { } A policy : X A mple: Coin : Coin Coin3 hed il hed : 3 hed il hed Gmbling emple L model: Inpu: X coin chosen fo he ne oss Acion: A choice of hed o il he gen bes on einfocemens: { } A policy : Coin hed Lening gol: find he opiml policy *: X A mimizing fuue epeced pofis Coin Coin3 il hed *: discoun fco = pesen vlue of money 3???
3 peced ewds peced ewds fo : X A un ime un ime un 3 ime pecion ove mny possible ewd jecoies fo : X A peced discouned ewds peced discouning ewds fo : X A Discouning wih fuue vlue of money No discouning: un ime un Discouning ime pecion ove mny possible discouned ewd jecoies fo : X A 3
4 L lening: objecive funcions Objecive: * Find mpping : X A h mimizes some combinion of fuue einfocemens ewds eceived ove ime Vluion models qunify how good he mpping is: Finie hoizon models Infinie hoizon discouned model Avege ewd ime hoizon: Discoun fco: lim Discoun fco: Agen nvigion emple Agen nvigion in he mze: 4 moves in compss diecions ffecs of moves e sochsic we my wind up in ohe hn inended locion wih nonzeo pobbiliy Objecive: len how o ech he gol se in he shoes epeced ime moves G 4
5 Agen nvigion emple he L model: Inpu: X posiion of n gen Oupu: A he ne move einfocemens:  fo ech move + fo eching he gol A policy: : X A Gol: find he policy mimizing fuue epeced ewds : Posiion Posiion Posiion G igh igh lef moves ploion vs. ploiion in L he lene cively inecs wih he envionmen: A he beginning he lene does no know nyhing bou he envionmen I gdully gins he epeience nd lens how o ec o he envionmen Dilemm eploioneploiion: Afe some numbe of seps should I selec he bes cuen choice eploiion o y o len moe bou he envionmen eploion? ploiion my involve he selecion of subopiml cion nd peven he lening of he opiml choice ploion my spend o much ime on ying bd cuenly subopiml cions 5
6 ffecs of cions on he envionmen ffec of cions on he envionmen ne inpu o be seen No effec. he disibuion ove possible is fied nd independen of ps cions. he ewds eceived depend only on he se nd cion chosen. he e seen fe he cion. Acions my effec he envionmen nd ne inpus. he disibuion of cn chnge due o ps cions; he ewds eled o he cion cn be seen wih some dely. Leds o wo foms of einfocemen lening: Lening wih immedie ewds 3 coin emple 3 Lening wih delyed ewds Agen nvigion emple; move choices ffec he se of he envionmen posiion chnges big ewd he gol se is delyed L wih immedie ewds Gme: 3 bised coins 3 he coin o be ossed is seleced ndomly fom he hee coin opions. he gen lwys sees which coin is going o be plyed ne. he gen mkes be on eihe hed o il wih wge of $. If fe he coin oss he oucome gees wih he be he gen wins $ ohewise i looses $ L model: Inpu: X coin chosen fo he ne oss Acion: A hed o il he gen bes on einfocemens: { } $ eihe won o los Lening gol: find he opiml policy *: X A mimizing he fuue epeced pofis ove ime discoun fco 6
7 L wih immedie ewds peced ewd Immedie ewd cse: ewd depends only on nd he cion choice he cion does no ffec he envionmen nd hence fuue inpus ses nd fuue ewds: peced one sep ewd fo inpu coin o ply ne nd he choice : ewds fo evey sep of he gme j L wih immedie ewds Immedie ewd cse: ewd fo inpu nd he cion choice my vy peced ewd fo he inpu nd choice : Fo he coin be poblem i is: i j i P j i j : n oucome of he coin oss : ewd fo n oucome nd he be mde on j i peced one sep ewd fo segy P is he epeced ewd fo : X A... 7
8 8 L wih immedie ewds peced ewd Opimizing he epeced ewd : Opiml segy: m m m m... X A *: ] [m m m P P g m * m L wih immedie ewds We know h Poblem: In he L fmewok we do no know he epeced ewd fo pefoming cion inpu How o esime? g m *
9 L wih immedie ewds Poblem: In he L fmewok we do no know he epeced ewd fo pefoming cion inpu Soluion: Fo ech inpu y diffeen cions sime using he vege of obseved ewds N Acion choice g m Accucy of he esime: sisics Hoeffding s bound N P ep Numbe of smples: N i i N m m min min ln L wih immedie ewds Online sochsic ppoimion An lenive wy o esime Ide: choose cion fo inpu nd obseve ewd Upde n esime in evey sep i i i i i i i  lening e Convegence popey: he ppoimion conveges in he limi fo n ppopie lening e schedule. Assume: n  is lening e fo nh il of pi hen he convege is ssued if: i. i. i i 9
10 L wih immedie ewds A ny sep in ime i duing he epeimen we hve esimes of epeced ewds fo ech coin cion pi: i coin hed i coin il i coin hed i coin il i coin3 hed i coin3 il Assume he ne coin o ply in sep i+ is coin nd we pick hed s ou be. hen we upde i coin hed using he obseved ewd nd one of he upde segy bove nd keep he ewd esimes fo he emining coin cion pis unchnged e.g. i coin il coin il i ploion vs. ploiion In he L fmewok he lene cively inecs wih he envionmen nd choses he cion o ply fo he cuen inpu Also ny poin in ime i hs n esime of fo ny inpucion pi Dilemm fo choosing he cion o ply fo : Should he lene choose he cuen bes choice of cion eploiion ˆ g m A O choose some ohe cion which my help o impove is esime eploion his dilemm is clled eploion/eploiion dilemm Diffeen eploion/eploiion segies eis
11 ploion vs. ploiion Unifom eploion: ploion pmee Choose he cuen bes choice wih pobbiliy ˆ g m A A All ohe choices e seleced wih unifom pobbiliy Bolzmn eploion he cion is chosen ndomly bu popoionlly o is cuen epeced ewd esime ep / p ep ' / ' A is empeue pmee. Wh does i do?
Reinforcement learning
Lecue 3 Reinfocemen leaning Milos Hauskech milos@cs.pi.edu 539 Senno Squae Reinfocemen leaning We wan o lean he conol policy: : X A We see examples of x (bu oupus a ae no given) Insead of a we ge a feedback
More informationf(x) dx with An integral having either an infinite limit of integration or an unbounded integrand is called improper. Here are two examples dx x x 2
Impope Inegls To his poin we hve only consideed inegls f() wih he is of inegion nd b finie nd he inegnd f() bounded (nd in fc coninuous ecep possibly fo finiely mny jump disconinuiies) An inegl hving eihe
More informationChapter 2: Evaluative Feedback
Chper 2: Evluive Feedbck Evluing cions vs. insrucing by giving correc cions Pure evluive feedbck depends olly on he cion ken. Pure insrucive feedbck depends no ll on he cion ken. Supervised lerning is
More informationCircuits 24/08/2010. Question. Question. Practice Questions QV CV. Review Formula s RC R R R V IR ... Charging P IV I R ... E Pt.
4/08/00 eview Fomul s icuis cice s BL B A B I I I I E...... s n n hging Q Q 0 e... n... Q Q n 0 e Q I I0e Dischging Q U Q A wie mde of bss nd nohe wie mde of silve hve he sme lengh, bu he dimee of he bss
More informationReinforcement learning II
CS 1675 Introduction to Mchine Lerning Lecture 26 Reinforcement lerning II Milos Huskrecht milos@cs.pitt.edu 5329 Sennott Squre Reinforcement lerning Bsics: Input x Lerner Output Reinforcement r Critic
More information() t. () t r () t or v. ( t) () () ( ) = ( ) or ( ) () () () t or dv () () Section 10.4 Motion in Space: Velocity and Acceleration
Secion 1.4 Moion in Spce: Velociy nd Acceleion We e going o dive lile deepe ino somehing we ve ledy inoduced, nmely () nd (). Discuss wih you neighbo he elionships beween posiion, velociy nd cceleion you
More informationScience Advertisement Intergovernmental Panel on Climate Change: The Physical Science Basis 2/3/2007 Physics 253
Science Adeisemen Inegoenmenl Pnel on Clime Chnge: The Phsicl Science Bsis hp://www.ipcc.ch/spmfeb7.pdf /3/7 Phsics 53 hp://www.fonews.com/pojecs/pdf/spmfeb7.pdf /3/7 Phsics 53 3 Sus: Uni, Chpe 3 Vecos
More informationME 141. Engineering Mechanics
ME 141 Engineeing Mechnics Lecue 13: Kinemics of igid bodies hmd Shhedi Shkil Lecue, ep. of Mechnicl Engg, UET Emil: sshkil@me.bue.c.bd, shkil6791@gmil.com Websie: eche.bue.c.bd/sshkil Couesy: Veco Mechnics
More information4.8 Improper Integrals
4.8 Improper Inegrls Well you ve mde i hrough ll he inegrion echniques. Congrs! Unforunely for us, we sill need o cover one more inegrl. They re clled Improper Inegrls. A his poin, we ve only del wih inegrls
More informatione t dt e t dt = lim e t dt T (1 e T ) = 1
Improper Inegrls There re wo ypes of improper inegrls  hose wih infinie limis of inegrion, nd hose wih inegrnds h pproch some poin wihin he limis of inegrion. Firs we will consider inegrls wih infinie
More informationHomework 5 for BST 631: Statistical Theory I Solutions, 09/21/2006
Homewok 5 fo BST 63: Sisicl Theoy I Soluions, 9//6 Due Time: 5:PM Thusy, on 9/8/6. Polem ( oins). Book olem.8. Soluion: E = x f ( x) = ( x) f ( x) + ( x ) f ( x) = xf ( x) + xf ( x) + f ( x) f ( x) Accoing
More informationFaraday s Law. To be able to find. motional emf transformer and motional emf. Motional emf
Objecie F s w Tnsfome Moionl To be ble o fin nsfome. moionl nsfome n moionl. 331 1 331 Mwell s quion: ic Fiel D: Guss lw :KV : Guss lw H: Ampee s w Poin Fom Inegl Fom D D Q sufce loop H sufce H I enclose
More informationCh.4 Motion in 2D. Ch.4 Motion in 2D
Moion in plne, such s in he sceen, is clled 2dimensionl (2D) moion. 1. Posiion, displcemen nd eloci ecos If he picle s posiion is ( 1, 1 ) 1, nd ( 2, 2 ) 2, he posiions ecos e 1 = 1 1 2 = 2 2 Aege eloci
More informationBipartite Matching. Matching. Bipartite Matching. Maxflow Formulation
Mching Inpu: undireced grph G = (V, E). Biprie Mching Inpu: undireced, biprie grph G = (, E).. Mching Ern Myr, Hrld äcke Biprie Mching Inpu: undireced, biprie grph G = (, E). Mflow Formulion Inpu: undireced,
More informationf t f a f x dx By Lin McMullin f x dx= f b f a. 2
Accumulion: Thoughs On () By Lin McMullin f f f d = + The gols of he AP* Clculus progrm include he semen, Sudens should undersnd he definie inegrl s he ne ccumulion of chnge. 1 The Topicl Ouline includes
More informationLECTURE 5. is defined by the position vectors r, 1. and. The displacement vector (from P 1 to P 2 ) is defined through r and 1.
LECTURE 5 ] DESCRIPTION OF PARTICLE MOTION IN SPACE The displcemen, veloci nd cceleion in D moion evel hei veco nue (diecion) houh he cuion h one mus p o hei sin. Thei full veco menin ppes when he picle
More informationPhysics 201, Lecture 5
Phsics 1 Lecue 5 Tod s Topics n Moion in D (Chp 4.14.3): n D Kinemicl Quniies (sec. 4.1) n D Kinemics wih Consn Acceleion (sec. 4.) n D Pojecile (Sec 4.3) n Epeced fom Peiew: n Displcemen eloci cceleion
More informationSections 3.1 and 3.4 Exponential Functions (Growth and Decay)
Secions 3.1 and 3.4 Eponenial Funcions (Gowh and Decay) Chape 3. Secions 1 and 4 Page 1 of 5 Wha Would You Rahe Have... $1million, o double you money evey day fo 31 days saing wih 1cen? Day Cens Day Cens
More information#6: Double Directional Spatial Channel Model
2011 1 s semese MIMO Communicion Sysems #6: Doube Diecion Spi Chnne Mode Kei Skguchi ee c My 24 2011 Schedue 1 s hf De Tex Conens #1 Ap. 12 A1 B1 Inoducion #2 Ap. 19 B5
More informationgraph of unit step function t
.5 Piecewie coninuou forcing funcion...e.g. urning he forcing on nd off. The following Lplce rnform meril i ueful in yem where we urn forcing funcion on nd off, nd when we hve righ hnd ide "forcing funcion"
More informationENGR 1990 Engineering Mathematics The Integral of a Function as a Function
ENGR 1990 Engineering Mhemics The Inegrl of Funcion s Funcion Previously, we lerned how o esime he inegrl of funcion f( ) over some inervl y dding he res of finie se of rpezoids h represen he re under
More informationChapter 2. Motion along a straight line. 9/9/2015 Physics 218
Chper Moion long srigh line 9/9/05 Physics 8 Gols for Chper How o describe srigh line moion in erms of displcemen nd erge elociy. The mening of insnneous elociy nd speed. Aerge elociy/insnneous elociy
More informationINTEGRALS. Exercise 1. Let f : [a, b] R be bounded, and let P and Q be partitions of [a, b]. Prove that if P Q then U(P ) U(Q) and L(P ) L(Q).
INTEGRALS JOHN QUIGG Eercise. Le f : [, b] R be bounded, nd le P nd Q be priions of [, b]. Prove h if P Q hen U(P ) U(Q) nd L(P ) L(Q). Soluion: Le P = {,..., n }. Since Q is obined from P by dding finiely
More informationCalculus 241, section 12.2 Limits/Continuity & 12.3 Derivatives/Integrals notes by Tim Pilachowski r r r =, with a domain of real ( )
Clculu 4, econ Lm/Connuy & Devve/Inel noe y Tm Plchow, wh domn o el Wh we hve o : vecovlued uncon, ( ) ( ) ( ) j ( ) nume nd ne o veco The uncon, nd A w done wh eul uncon ( x) nd connuy e he componen
More informationMagnetostatics Bar Magnet. Magnetostatics Oersted s Experiment
Mgneosics Br Mgne As fr bck s 4500 yers go, he Chinese discovered h cerin ypes of iron ore could rc ech oher nd cerin mels. Iron filings "mp" of br mgne s field Crefully suspended slivers of his mel were
More informationAverage & instantaneous velocity and acceleration Motion with constant acceleration
Physics 7: Lecure Reminders Discussion nd Lb secions sr meeing ne week Fill ou Pink dd/drop form if you need o swich o differen secion h is FULL. Do i TODAY. Homework Ch. : 5, 7,, 3,, nd 6 Ch.: 6,, 3 Submission
More informationv T Pressure Extra Molecular Stresses Constitutive equations for Stress v t Observation: the stress tensor is symmetric
Momenum Blnce (coninued Momenum Blnce (coninued Now, wh o do wih Π? Pessue is p of i. bck o ou quesion, Now, wh o do wih? Π Pessue is p of i. Thee e ohe, nonisoopic sesses Pessue E Molecul Sesses definiion:
More informationMinimum Squared Error
Minimum Squred Error LDF: Minimum SquredError Procedures Ide: conver o esier nd eer undersood prolem Percepron y i > for ll smples y i solve sysem of liner inequliies MSE procedure y i = i for ll smples
More informationAddition & Subtraction of Polynomials
Addiion & Sucion of Polynomil Addiion of Polynomil: Adding wo o moe olynomil i imly me of dding like em. The following ocedue hould e ued o dd olynomil 1. Remove enhee if hee e enhee. Add imil em. Wie
More informationMinimum Squared Error
Minimum Squred Error LDF: Minimum SquredError Procedures Ide: conver o esier nd eer undersood prolem Percepron y i > 0 for ll smples y i solve sysem of liner inequliies MSE procedure y i i for ll smples
More informationContraction Mapping Principle Approach to Differential Equations
epl Journl of Science echnology 0 (009) 4953 Conrcion pping Principle pproch o Differenil Equions Bishnu P. Dhungn Deprmen of hemics, hendr Rn Cmpus ribhuvn Universiy, Khmu epl bsrc Using n eension of
More information0 for t < 0 1 for t > 0
8.0 Sep nd del funcions Auhor: Jeremy Orloff The uni Sep Funcion We define he uni sep funcion by u() = 0 for < 0 for > 0 I is clled he uni sep funcion becuse i kes uni sep = 0. I is someimes clled he Heviside
More informationTopics for Review for Final Exam in Calculus 16A
Topics fo Review fo Finl Em in Clculus 16A Instucto: Zvezdelin Stnkov Contents 1. Definitions 1. Theoems nd Poblem Solving Techniques 1 3. Eecises to Review 5 4. Chet Sheet 5 1. Definitions Undestnd the
More informationData Structures. Element Uniqueness Problem. Hash Tables. Example. Hash Tables. Dana Shapira. 19 x 1. ) h(x 4. ) h(x 2. ) h(x 3. h(x 1. x 4. x 2.
Element Uniqueness Poblem Dt Stuctues Let x,..., xn < m Detemine whethe thee exist i j such tht x i =x j Sot Algoithm Bucket Sot Dn Shpi Hsh Tbles fo (i=;i
More information10 Statistical Distributions Solutions
Communictions Engineeing MSc  Peliminy Reding 1 Sttisticl Distiutions Solutions 1) Pove tht the vince of unifom distiution with minimum vlue nd mximum vlue ( is ) 1. The vince is the men of the sques
More informationMTH 146 Class 11 Notes
8. Are of Surfce of Revoluion MTH 6 Clss Noes Suppose we wish o revolve curve C round n is nd find he surfce re of he resuling solid. Suppose f( ) is nonnegive funcion wih coninuous firs derivive on he
More informationD zone schemes
Ch. 5. Enegy Bnds in Cysls 5.. D zone schemes Fee elecons E k m h Fee elecons in cysl sinα P + cosα cosk α cos α cos k cos( k + π n α k + πn mv ob P 0 h cos α cos k n α k + π m h k E Enegy is peiodic
More informationS Radio transmission and network access Exercise 12
S7.330 Rdio rnsmission nd nework ccess Exercise 1  P1 In foursymbol digil sysem wih eqully probble symbols he pulses in he figure re used in rnsmission over AWGNchnnel. s () s () s () s () 1 3 4 )
More informationREAL ANALYSIS I HOMEWORK 3. Chapter 1
REAL ANALYSIS I HOMEWORK 3 CİHAN BAHRAN The quesions re from Sein nd Shkrchi s e. Chper 1 18. Prove he following sserion: Every mesurble funcion is he limi.e. of sequence of coninuous funcions. We firs
More informationA Kalman filtering simulation
A Klmn filering simulion The performnce of Klmn filering hs been esed on he bsis of wo differen dynmicl models, ssuming eiher moion wih consn elociy or wih consn ccelerion. The former is epeced o beer
More informationReinforcement Learning
Reiforceme Corol lerig Corol polices h choose opiml cios Q lerig Covergece Chper 13 Reiforceme 1 Corol Cosider lerig o choose cios, e.g., Robo lerig o dock o bery chrger o choose cios o opimize fcory oupu
More informationPHYS PRACTICE EXAM 2
PHYS 1800 PRACTICE EXAM Pa I Muliple Choice Quesions [ ps each] Diecions: Cicle he one alenaive ha bes complees he saemen o answes he quesion. Unless ohewise saed, assume ideal condiions (no ai esisance,
More informationChapter 21. Reinforcement Learning. The Reinforcement Learning Agent
CSE 47 Chaper Reinforcemen Learning The Reinforcemen Learning Agen Agen Sae u Reward r Acion a Enironmen CSE AI Faculy Why reinforcemen learning Programming an agen o drie a car or fly a helicoper is ery
More information156 There are 9 books stacked on a shelf. The thickness of each book is either 1 inch or 2
156 Thee ae 9 books sacked on a shelf. The hickness of each book is eihe 1 inch o 2 F inches. The heigh of he sack of 9 books is 14 inches. Which sysem of equaions can be used o deemine x, he numbe of
More informationCombinatorial Approach to M/M/1 Queues. Using Hypergeometric Functions
Inenaional Mahemaical Foum, Vol 8, 03, no 0, 46347 HIKARI Ld, wwwmhikaicom Combinaoial Appoach o M/M/ Queues Using Hypegeomeic Funcions Jagdish Saan and Kamal Nain Depamen of Saisics, Univesiy of Delhi,
More informationProbability, Estimators, and Stationarity
Chper Probbiliy, Esimors, nd Sionriy Consider signl genered by dynmicl process, R, R. Considering s funcion of ime, we re opering in he ime domin. A fundmenl wy o chrcerize he dynmics using he ime domin
More informationMotion on a Curve and Curvature
Moion on Cue nd Cuue his uni is bsed on Secions 9. & 9.3, Chpe 9. All ssigned edings nd execises e fom he exbook Objecies: Mke cein h you cn define, nd use in conex, he ems, conceps nd fomuls lised below:
More informationChapter 2. Kinematics in One Dimension. Kinematics deals with the concepts that are needed to describe motion.
Chpe Kinemic in One Dimenin Kinemic del wih he cncep h e needed decibe min. Dynmic del wih he effec h fce he n min. Tgehe, kinemic nd dynmic fm he bnch f phyic knwn Mechnic.. Diplcemen. Diplcemen.0 m 5.0
More informationRotations.
oons j.lbb@phscs.o.c.uk To s summ Fmes of efeence Invnce une nsfomons oon of wve funcon: funcons Eule s ngles Emple: e e   Angul momenum s oon geneo Genec nslons n Noehe s heoem Fmes of efeence Conse
More informationMotion. Part 2: Constant Acceleration. Acceleration. October Lab Physics. Ms. Levine 1. Acceleration. Acceleration. Units for Acceleration.
Moion Accelerion Pr : Consn Accelerion Accelerion Accelerion Accelerion is he re of chnge of velociy. = v  vo = Δv Δ ccelerion = = v  vo chnge of velociy elpsed ime Accelerion is vecor, lhough in onedimensionl
More informationToday  Lecture 13. Today s lecture continue with rotations, torque, Note that chapters 11, 12, 13 all involve rotations
Today  Lecue 13 Today s lecue coninue wih oaions, oque, Noe ha chapes 11, 1, 13 all inole oaions slide 1 eiew Roaions Chapes 11 & 1 Viewed fom aboe (+z) Roaional, o angula elociy, gies angenial elociy
More informationAns: In the rectangular loop with the assigned direction for i2: di L dt , (1) where (2) a) At t = 0, i1(t) = I1U(t) is applied and (1) becomes
omewok # P73 ecngul loop of widh w nd heigh h is siued ne ve long wie cing cuen i s in Fig 7 ssume i o e ecngul pulse s shown in Fig 7 Find he induced cuen i in he ecngul loop whose selfinducnce is
More informationEECE 260 Electrical Circuits Prof. Mark Fowler
EECE 60 Electicl Cicuits Pof. Mk Fowle Complex Numbe Review /6 Complex Numbes Complex numbes ise s oots of polynomils. Definition of imginy # nd some esulting popeties: ( ( )( ) )( ) Recll tht the solution
More informationElectric Potential. and Equipotentials
Electic Potentil nd Euipotentils U Electicl Potentil Review: W wok done y foce in going fom to long pth. l d E dl F W dl F θ Δ l d E W U U U Δ Δ l d E W U U U U potentil enegy electic potentil Potentil
More informationZürich. ETH Master Course: L Autonomous Mobile Robots Localization II
Roland Siegwar Margaria Chli Paul Furgale Marco Huer Marin Rufli Davide Scaramuzza ETH Maser Course: 151085400L Auonomous Mobile Robos Localizaion II ACT and SEE For all do, (predicion updae / ACT),
More informationLowcomplexity Algorithms for MIMO Multiplexing Systems
Lowcomplexiy Algoihms fo MIMO Muliplexing Sysems Ouline Inoducion QRDM M algoihm Algoihm I: : o educe he numbe of suviving pahs. Algoihm II: : o educe he numbe of candidaes fo each ansmied signal. :
More informationHomework 3 MAE 118C Problems 2, 5, 7, 10, 14, 15, 18, 23, 30, 31 from Chapter 5, Lamarsh & Baratta. The flux for a point source is:
. Homewok 3 MAE 8C Poblems, 5, 7, 0, 4, 5, 8, 3, 30, 3 fom Chpte 5, msh & Btt Point souces emit nuetons/sec t points,,, n 3 fin the flux cuent hlf wy between one sie of the tingle (blck ot). The flux fo
More informationThe solution is often represented as a vector: 2xI + 4X2 + 2X3 + 4X4 + 2X5 = 4 2xI + 4X2 + 3X3 + 3X4 + 3X5 = 4. 3xI + 6X2 + 6X3 + 3X4 + 6X5 = 6.
[~ o o : o o ill] i 1. Mrices, Vecors, nd GussJordn Eliminion 1 x y = =  z= The soluion is ofen represened s vecor: n his exmple, he process of eliminion works very smoohly. We cn elimine ll enries
More informationMATH 124 AND 125 FINAL EXAM REVIEW PACKET (Revised spring 2008)
MATH 14 AND 15 FINAL EXAM REVIEW PACKET (Revised spring 8) The following quesions cn be used s review for Mh 14/ 15 These quesions re no cul smples of quesions h will pper on he finl em, bu hey will provide
More information2d Motion: Constant Acceleration
d Moion: Consan Acceleaion Kinemaic Equaions o Moion (eco Fom Acceleaion eco (consan eloci eco (uncion o Posiion eco (uncion o The eloci eco and posiion eco ae a uncion o he ime. eloci eco a ime. Posiion
More information( ) ( ) ( ) ( ) ( ) ( y )
8. Lengh of Plne Curve The mos fmous heorem in ll of mhemics is he Pyhgoren Theorem. I s formulion s he disnce formul is used o find he lenghs of line segmens in he coordine plne. In his secion you ll
More informationMaking Complex Decisions Markov Decision Processes. Making Complex Decisions: Markov Decision Problem
Mking Comple Decisions Mrkov Decision Processes Vsn Honvr Bioinformics nd Compuionl Biology Progrm Cener for Compuionl Inelligence, Lerning, & Discovery honvr@cs.ise.edu www.cs.ise.edu/~honvr/ www.cild.ise.edu/
More informationPreviously. Extensions to backstepping controller designs. Tracking using backstepping Suppose we consider the general system
436459 Advnced contol nd utomtion Extensions to bckstepping contolle designs Tcking Obseves (nonline dmping) Peviously Lst lectue we looked t designing nonline contolles using the bckstepping technique
More informationComputer Propagation Analysis Tools
Compue Popagaion Analysis Tools. Compue Popagaion Analysis Tools Inoducion By now you ae pobably geing he idea ha pedicing eceived signal sengh is a eally impoan as in he design of a wieless communicaion
More informationDerivation of the differential equation of motion
Divion of h iffnil quion of oion Fis h noions fin h will us fo h ivion of h iffnil quion of oion. Rollo is hough o insionl isk. xnl ius of h ll isnc cn of ll (O)  IDU s cn of gviy (M) θ ngl of inclinion
More informationOptimality of Myopic Policy for a Class of Monotone Affine Restless MultiArmed Bandit
Univeriy of Souhern Cliforni Opimliy of Myopic Policy for Cl of Monoone Affine Rele MuliArmed Bndi Pri Mnourifrd USC Tr Jvidi UCSD Bhkr Krihnmchri USC Dec 0, 202 Univeriy of Souhern Cliforni Inroducion
More informationCS 188: Artificial Intelligence Fall Probabilistic Models
CS 188: Aificial Inelligence Fall 2007 Lecue 15: Bayes Nes 10/18/2007 Dan Klein UC Bekeley Pobabilisic Models A pobabilisic model is a join disibuion ove a se of vaiables Given a join disibuion, we can
More informationLectureV Stochastic Processes and the Basic TermStructure Equation 1 Stochastic Processes Any variable whose value changes over time in an uncertain
LecueV Sochasic Pocesses and he Basic TemSucue Equaion 1 Sochasic Pocesses Any vaiable whose value changes ove ime in an unceain way is called a Sochasic Pocess. Sochasic Pocesses can be classied as
More information2D Motion WS. A horizontally launched projectile s initial vertical velocity is zero. Solve the following problems with this information.
Nme D Moion WS The equions of moion h rele o projeciles were discussed in he Projecile Moion Anlsis Acii. ou found h projecile moes wih consn eloci in he horizonl direcion nd consn ccelerion in he ericl
More informationLaplace Transforms. Examples. Is this equation differential? y 2 2y + 1 = 0, y 2 2y + 1 = 0, (y ) 2 2y + 1 = cos x,
Laplace Transforms Definiion. An ordinary differenial equaion is an equaion ha conains one or several derivaives of an unknown funcion which we call y and which we wan o deermine from he equaion. The equaion
More information5.1The InitialValue Problems For Ordinary Differential Equations
5.The IniilVlue Problems For Ordinry Differenil Equions Consider solving iniilvlue problems for ordinry differenil equions: (*) y f, y, b, y. If we know he generl soluion y of he ordinry differenil
More informationMathematics 805 Final Examination Answers
. 5 poins Se he Weiersrss Mes. Mhemics 85 Finl Eminion Answers Answer: Suppose h A R, nd f n : A R. Suppose furher h f n M n for ll A, nd h Mn converges. Then f n converges uniformly on A.. 5 poins Se
More informationLecture 10. Solution of Nonlinear Equations  II
Fied point Poblems Lectue Solution o Nonline Equtions  II Given unction g : R R, vlue such tht gis clled ied point o the unction g, since is unchnged when g is pplied to it. Whees with nonline eqution
More informationControl Volume Derivation
School of eospace Engineeing Conol Volume 1 Copyigh 1 by Jey M. Seizman. ll ighs esee. Conol Volume Deiaion How o cone ou elaionships fo a close sysem (conol mass) o an open sysem (conol olume) Fo mass
More informationClass Summary. be functions and f( D) , we define the composition of f with g, denoted g f by
Clss Summy.5 Eponentil Functions.6 Invese Functions nd Logithms A function f is ule tht ssigns to ech element D ectly one element, clled f( ), in. Fo emple : function not function Given functions f, g:
More informationProperties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives
Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums  1 Riemnn
More informationANSWERS TO EVEN NUMBERED EXERCISES IN CHAPTER 2
ANSWERS TO EVEN NUMBERED EXERCISES IN CHAPTER Seion Eerise : Coninuiy of he uiliy funion Le λ ( ) be he monooni uiliy funion defined in he proof of eisene of uiliy funion If his funion is oninuous y hen
More informationReinforcement Learning. Markov Decision Processes
einforcemen Lerning Mrkov Decision rocesses Mnfred Huber 2014 1 equenil Decision Mking Nrmed bi problems re no good wy o model sequenil decision problem Only dels wih sic decision sequences Could be miiged
More informationWORK POWER AND ENERGY Consevaive foce a) A foce is said o be consevaive if he wok done by i is independen of pah followed by he body b) Wok done by a consevaive foce fo a closed pah is zeo c) Wok done
More informationCS103B Handout 18 Winter 2007 February 28, 2007 Finite Automata
CS103B ndout 18 Winter 2007 Ferury 28, 2007 Finite Automt Initil text y Mggie Johnson. Introduction Severl childrens gmes fit the following description: Pieces re set up on plying ord; dice re thrown or
More informationProbabilistic Models. CS 188: Artificial Intelligence Fall Independence. Example: Independence. Example: Independence? Conditional Independence
C 188: Aificial Inelligence Fall 2007 obabilisic Models A pobabilisic model is a join disibuion ove a se of vaiables Lecue 15: Bayes Nes 10/18/2007 Given a join disibuion, we can eason abou unobseved vaiables
More informationSeptember 20 Homework Solutions
College of Engineering nd Compuer Science Mechnicl Engineering Deprmen Mechnicl Engineering A Seminr in Engineering Anlysis Fll 7 Number 66 Insrucor: Lrry Creo Sepember Homework Soluions Find he specrum
More informationEquations from The Four Principal Kinetic States of Material Bodies. Copyright 2005 Joseph A. Rybczyk
Equions fom he Fou Pinipl Kinei Ses of Meil Bodies Copyigh 005 Joseph A. Rybzyk Following is omplee lis of ll of he equions used in o deied in he Fou Pinipl Kinei Ses of Meil Bodies. Eh equion is idenified
More informationThe Production of Polarization
Physics 36: Waves Lecue 13 3/31/211 The Poducion of Polaizaion Today we will alk abou he poducion of polaized ligh. We aleady inoduced he concep of he polaizaion of ligh, a ansvese EM wave. To biefly eview
More informationWeek 8. Topic 2 Properties of Logarithms
Week 8 Topic 2 Popeties of Logithms 1 Week 8 Topic 2 Popeties of Logithms Intoduction Since the esult of ithm is n eponent, we hve mny popeties of ithms tht e elted to the popeties of eponents. They e
More informationRepresenting Knowledge. CS 188: Artificial Intelligence Fall Properties of BNs. Independence? Reachability (the Bayes Ball) Example
C 188: Aificial Inelligence Fall 2007 epesening Knowledge ecue 17: ayes Nes III 10/25/2007 an Klein UC ekeley Popeies of Ns Independence? ayes nes: pecify complex join disibuions using simple local condiional
More informationGoals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite
Unit #8 : The Integrl Gols: Determine how to clculte the re described by function. Define the definite integrl. Eplore the reltionship between the definite integrl nd re. Eplore wys to estimte the definite
More informationChapter Direct Method of Interpolation
Chper 5. Direc Mehod of Inerpolion Afer reding his chper, you should be ble o:. pply he direc mehod of inerpolion,. sole problems using he direc mehod of inerpolion, nd. use he direc mehod inerpolns o
More informationAn random variable is a quantity that assumes different values with certain probabilities.
Probabiliy The probabiliy PrA) of an even A is a number in [, ] ha represens how likely A is o occur. The larger he value of PrA), he more likely he even is o occur. PrA) means he even mus occur. PrA)
More informationAnswers to test yourself questions
Answes to test youself questions opic Descibing fields Gm Gm Gm Gm he net field t is: g ( d / ) ( 4d / ) d d Gm Gm Gm Gm Gm Gm b he net potentil t is: V d / 4d / d 4d d d V e 4 7 9 49 J kg 7 7 Gm d b E
More informationSignals and Systems Profs. Byron Yu and Pulkit Grover Fall Midterm 1 Solutions
890 Signals and Sysems Profs. Byron Yu and Pulki Grover Fall 07 Miderm Soluions Name: Andrew ID: Problem Score Max 0 8 4 6 5 0 6 0 7 8 9 0 6 Toal 00 Miderm Soluions. (0 poins) Deermine wheher he following
More informationMATHEMATICAL FOUNDATIONS FOR APPROXIMATING PARTICLE BEHAVIOUR AT RADIUS OF THE PLANCK LENGTH
Fundamenal Jounal of Mahemaical Phsics Vol 3 Issue 013 Pages 556 Published online a hp://wwwfdincom/ MATHEMATICAL FOUNDATIONS FOR APPROXIMATING PARTICLE BEHAVIOUR AT RADIUS OF THE PLANCK LENGTH Univesias
More informationEquations and Inequalities
Equtions nd Inequlities Equtions nd Inequlities Curriculum Redy ACMNA: 4, 5, 6, 7, 40 www.mthletics.com Equtions EQUATIONS & Inequlities & INEQUALITIES Sometimes just writing vribles or pronumerls in
More informationServomechanism Design
Sevomechanism Design Sevomechanism (sevosysem) is a conol sysem in which he efeence () (age, Se poin) changes as ime passes. Design mehods PID Conol u () Ke P () + K I ed () + KDe () Sae Feedback u()
More informationProperties of Logarithms. Solving Exponential and Logarithmic Equations. Properties of Logarithms. Properties of Logarithms. ( x)
Properies of Logrihms Solving Eponenil nd Logrihmic Equions Properies of Logrihms Produc Rule ( ) log mn = log m + log n ( ) log = log + log Properies of Logrihms Quoien Rule log m = logm logn n log7 =
More informationPhysics 101 Lecture 4 Motion in 2D and 3D
Phsics 11 Lecure 4 Moion in D nd 3D Dr. Ali ÖVGÜN EMU Phsics Deprmen www.ogun.com Vecor nd is componens The componens re he legs of he righ ringle whose hpoenuse is A A A A A n ( θ ) A Acos( θ) A A A nd
More informationRelative and Circular Motion
Relaie and Cicula Moion a) Relaie moion b) Cenipeal acceleaion Mechanics Lecue 3 Slide 1 Mechanics Lecue 3 Slide 2 Time on Video Pelecue Looks like mosly eeyone hee has iewed enie pelecue GOOD! Thank you
More informationGeneral NonArbitrage Model. I. Partial Differential Equation for Pricing A. Traded Underlying Security
1 Geneal NonAbiage Model I. Paial Diffeenial Equaion fo Picing A. aded Undelying Secuiy 1. Dynamics of he Asse Given by: a. ds = µ (S, )d + σ (S, )dz b. he asse can be eihe a sock, o a cuency, an index,
More informationTechnical Vibration  text 2  forced vibration, rotational vibration
Technicl Viion  e  foced viion, oionl viion 4. oced viion, viion unde he consn eenl foce The viion unde he eenl foce. eenl The quesion is if he eenl foce e is consn o vying. If vying, wh is he foce funcion.
More informationNonsinusoidal Signal Generators
Nonsinusoidal Signal Geneaos ecangle, iangle, saw ooh, pulse, ec. Muliibao cicuis: asable no sable saes (wo quasisable saes; i emains in each sae fo pedeemined imes) monosable one sable sae, one unsable
More information