Non-Myopic Multi-Aspect Sensing with Partially Observable Markov Decision Processes
|
|
|
- Gavin Barrett
- 5 years ago
- Views:
Transcription
1 Non-Myopic Multi-Apect Sening with Prtilly Oervle Mrkov Deciion Procee Shiho Ji 2 Ronld Prr nd Lwrence Crin Deprtment of Electricl & Computer Engineering 2 Deprtment of Computer Engineering Duke Univerity Durhm NC
2 Outline Summry of the underlying prtilly-oerved Mrkov model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
3 Bic Contruct S S2 trget S3 S4 Scttering Dt Cn e Segmented into Angulr Bin Chrcterized y prticulr phyic Ech uch ngulr rnge termed tte S S2... SN
4 Hidden Mrkov Model π π 2 π
5 = K j j i j j i j i j d w d w p φ φ φ = 2 2 / 2 exp 2 j j j w σ φ σ π φ 2 / = j j j φ φ σ Action-Dependent Stte-rnition Mtrix Let d ij repreent the ngulr ditnce etween the center of tte i nd j in precried direction e.g. clockwie he proility of trnition from tte i to tte j fter moving ngulr ditnce φ i he tndrd devition σ j i dictted y the width of tte j i j d ij
6 Outline Summry of the underlying prtilly-oerved model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
7 o Pr = o o Pro o Pro Pr Pr Pro Pro Pr Pro = = = p p Belief Stte Sufficient Sttitic he elief tte quntifie the proility tht the enor i in tte given equence of ction nd correponding oervtion he elief tte t time i ufficient ttitic for ll ction nd oervtion up to tht point Very importnt for prcticl implementtion: Needn t tore ll previou ction & oervtion Belief tte computed redily uing underlying trget POMDP model
8 Belief Stte nd Bye Rik Belief tte my lo e ued to compute the proility tht trget n i eing interrogted ed on previou ction nd oervtion p n o...o... = p n = hi fct ply key role in uequent policy deign which mp elief tte to correponding ction ecue the elief tte my e ued to compute the Bye rik of clifiction deciion S n rget = rg min u N v= C uv p v = rg min u N v= C uv S v
9 Action nd Sening Cot wo type of ction: - Sening ction tht elect next ngle of oervtion nd/or frequency of opertion - Deciion ction â for which ening i topped nd clifiction deciion i mde Cot for ening ction: c independent of wht trget tte i viited thi repreent the cot of performing meurement poily enor dependent Introduce rik-ed terminl rewrd for mking deciion thi termed ction â
10 Clifiction Cot Upon performing clifiction ction â we move into new tte ij correponding to declring trget i when the ctul trget i trget j he cot ocited with tte ij i repreented C ij he proility of interrogting trget j given elief tte where re the underlying tte of the trget i p j = S j he expected immedite cot of tking terminl clifiction ction â in elief tte my therefore e repreented C = mx ˆ i j C ij p j mx = ˆ i j S j C ij Immedite expected vlue of terminting ening nd declring trget i ction â i driven y Bye rik
11 POMDP Formultion Summry Action Stte Cot Sening Action: Move pltform ngle φ Perform meurement with one of M enor Clifiction Action: Stop ening declre oject under tet to e one memer from et { 2 N} S = { n k k n } rget tte k cro ll trget n={ 2 N} uv correponding to declring trget u when in relity trget v i eing ened; oth u nd v memer of the et { 2 N} cm m repreenting one of the M poile enor independent of trget tte viited C uv for clifiction tte uv In term of trget tte in S c=u=c uv for ll ocited with trget v
12 POMDP Summry Algorithm h two type of tte: underlying tte of the trget plu terminl tte ij fter mking clifiction deciion Optiml policy lern wht ening ction to tke given elief tte well policy to when to mke deciion top ening function of the elief tte My include different cot for different enor modlitie while lo ccounting vi Bye rik for cot of different miclifiction C ij Optiml policy determined vi point-ed lgorithm tht preerve the locl lope of vlue function
13 Outline Summry of the underlying prtilly-oerved model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
14 Implementtion Iue he cot of tking ction when in elief tte t tep from the horizon i Dicounted Expected Future Cot χ t = min C + γ p χ t B Immedite Expected Cot Become dynmic-progrmming prolem for lerning the optiml policy which mp elief tte to ction dicounted infinite-horizon prolem Vlue-itertion dynmic progrmming tilize when fixed ction i defined for ech elief tte defining the optiml dicounted infinite-horizon policy
15 min S C t t α χ α = + = O S S C A t p p C t o o min min α γ χ α he cot function i liner in the elief tte which implie tht the cot function i piecewie liner concve prolem in the elief-pce implex Belief pce χ For locl region in elief pce we trck nd updte the lope α tht minimize the locl cot Implementtion Iue - 2 Vlue itertion ecome prolem of lerning the elief-tte locl lope α for ech of which there i n optiml ction policy Policy lerned y trcking lope pproximtely
16 Outline Summry of the underlying prtilly-oerved model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
17 wo Ditinct POMDP Formultion Infinite horizon with reet upon ech clifiction - Approprite when we wih to perform equence of mny clifiction - Multi-trget ening within udget - Policy h the opportunity to opt out of difficult ening ce trget miguity Algorithm trnition into n oring tte fter clifiction - Finite-horizon policy with horizon dictted y difficulty of initil elief tte - Doe not hve opportunity to opt out of difficult clifiction ce
18 Rndom Reet rget rget 2 Sening cot cm for enor m â â â 2 â 2 Actully rget Actully rget 2 Actully rget Actully rget 2 Cot: C C 2 C 2 C 22 Aoring Stte
19 Outline Summry of the underlying prtilly-oerved model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
20 + + = + = O o S S N v uv u E v p o p C R min [ ] ˆ = E R R c C = = min C R S v N v uv u Myopic Sening with Stop Criterion Given elief tte the expected rik fter tking ening ction + my e expreed We compute the difference etween the cot of ction + nd the correponding expected reduction in rik We terminte ening when thi difference ecome poitive cot exceed expected reduction in rik
21 Outline Summry of the underlying prtilly-oerved model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
22 rget rget rget 3 rget 4 φ Ri rget 5 Internl Ocilltor
23 Multi-Apect Dt 50 Reopne for rget 50 Reopne for rget 2 50 Reopne for rget Frequency khz Frequency khz Frequency khz Angle Deg Reopne for rget Angle Deg Reopne for rget 5 Angle Deg Frequency khz Frequency khz Angle Deg Angle Deg
24 Full-Bnd Dt Clifiction Accurcy v. Averge Numer of Action C = C uu =-0 nd C uv =C c with C c vrile from 5 to Clifiction Accurcy POMDP - oring POMDP - reet Greedy - topping Greedy - no topping verge numer of ction
25 Full-Bnd Dt Clifiction Accurcy v. Cot of Miclifiction C = C uu =-0 nd C uv =C c with C c vrile from 5 to Clifiction Accurcy POMDP - oring POMDP - reet Greedy - topping Cot of Miclifiction
26 Full-Bnd Dt Cot Per Action v. Cot of Miclifiction -0.4 C = C uu =-0 nd C uv =C c with C c vrile from 5 to Cot per Action POMDP - oring POMDP - reet Greedy - topping Cot of Miclifiction
27 Outline Summry of the underlying prtilly-oerved model with correponding ction Prtilly oerved Mrkov deciion procee POMDP nd elief tte cot nd Bye rik Lerning POMDP policy vi vlue itertion with policy defining the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic wo POMDP implementtion trtegie for multi-trget cttering dt Myopic or greedy ening lterntive with top criterion Exmple reult on cttering dt meured y NRL - Action: Selection of optiml trget-enor orienttion fullnd dt - Action: Selection of optiml trget-enor orienttion nd frequency und
28 Action: Select Sund nd Angle C = C uu =-0 nd C uv =C c with C c =40 Fixed ngulr mpling 5 o Angle Selection Fixed und: LL Fixed und: HL Fixed und: LH Fixed und: HH Fixed nd: Fullnd Sund Selection 86.% 72.67% 73.72% 77.72% 76.50% 90.72% % % % % % % % % % % % % % 2.5 Blck: HMM with fixed ngulr mpling of 5 o five ction Blue: Myopic POMDP with fixed numer of five ction Red: Non-myopic POMDP with reet verge numer of ction
29 Summry nd Future Work Hve developed POMDP formultion for generl ening prolem with policy deigned to define the optiml ction for given elief tte ccounting for dicounted infinite horizon non-myopic Algorithm operte in rel time nd optimlly integrte the ening nd ignl proceing tk perfect mtch for UUV for exmple Key point: he POMDP formultion ume cce to model for the trget to lern the optiml policy; my not e relitic in mny etting Reinforcement lerning RL i generliztion of POMDP wherein the ening ction re not performed imply to exploit n underlying model optimlly ut the ction lo ddre explortion to lern more out given environment/trget tht it my not hve een previouly RL POMDP optimlly execute ction in non-myopic etting to ddre the exploittionexplortion trdeoff; now extending the reerch towrd RL POMDP
Markov Decision Processes
Mrkov Deciion Procee A Brief Introduction nd Overview Jck L. King Ph.D. Geno UK Limited Preenttion Outline Introduction to MDP Motivtion for Study Definition Key Point of Interet Solution Technique Prtilly
Reinforcement Learning and Policy Reuse
Reinforcement Lerning nd Policy Reue Mnuel M. Veloo PEL Fll 206 Reding: Reinforcement Lerning: An Introduction R. Sutton nd A. Brto Probbilitic policy reue in reinforcement lerning gent Fernndo Fernndez
Reinforcement learning
Reinforcement lerning Regulr MDP Given: Trnition model P Rewrd function R Find: Policy π Reinforcement lerning Trnition model nd rewrd function initilly unknown Still need to find the right policy Lern
Artificial Intelligence Markov Decision Problems
rtificil Intelligence Mrkov eciion Problem ilon - briefly mentioned in hpter Ruell nd orvig - hpter 7 Mrkov eciion Problem; pge of Mrkov eciion Problem; pge of exmple: probbilitic blockworld ction outcome
Module 6 Value Iteration. CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo
Module 6 Vlue Itertion CS 886 Sequentil Decision Mking nd Reinforcement Lerning University of Wterloo Mrkov Decision Process Definition Set of sttes: S Set of ctions (i.e., decisions): A Trnsition model:
Reinforcement Learning
Reinforcement Lerning Tom Mitchell, Mchine Lerning, chpter 13 Outline Introduction Comprison with inductive lerning Mrkov Decision Processes: the model Optiml policy: The tsk Q Lerning: Q function Algorithm
Reinforcement learning II
CS 1675 Introduction to Mchine Lerning Lecture 26 Reinforcement lerning II Milos Huskrecht milos@cs.pitt.edu 5329 Sennott Squre Reinforcement lerning Bsics: Input x Lerner Output Reinforcement r Critic
Bellman Optimality Equation for V*
Bellmn Optimlity Eqution for V* The vlue of stte under n optiml policy must equl the expected return for the best ction from tht stte: V (s) mx Q (s,) A(s) mx A(s) mx A(s) Er t 1 V (s t 1 ) s t s, t s
CS 188: Artificial Intelligence Fall Announcements
CS 188: Artificil Intelligence Fll 2009 Lecture 20: Prticle Filtering 11/5/2009 Dn Klein UC Berkeley Announcements Written 3 out: due 10/12 Project 4 out: due 10/19 Written 4 proly xed, Project 5 moving
19 Optimal behavior: Game theory
Intro. to Artificil Intelligence: Dle Schuurmns, Relu Ptrscu 1 19 Optiml behvior: Gme theory Adversril stte dynmics hve to ccount for worst cse Compute policy π : S A tht mximizes minimum rewrd Let S (,
Administrivia CSE 190: Reinforcement Learning: An Introduction
Administrivi CSE 190: Reinforcement Lerning: An Introduction Any emil sent to me bout the course should hve CSE 190 in the subject line! Chpter 4: Dynmic Progrmming Acknowledgment: A good number of these
2D1431 Machine Learning Lab 3: Reinforcement Learning
2D1431 Mchine Lerning Lb 3: Reinforcement Lerning Frnk Hoffmnn modified by Örjn Ekeberg December 7, 2004 1 Introduction In this lb you will lern bout dynmic progrmming nd reinforcement lerning. It is ssumed
Policy Gradient Methods for Reinforcement Learning with Function Approximation
Policy Grdient Method for Reinforcement Lerning with Function Approximtion Richrd S. Sutton, Dvid McAlleter, Stinder Singh, Yihy Mnour AT&T Lb Reerch, 180 Prk Avenue, Florhm Prk, NJ 07932 Abtrct Function
Bayesian Networks: Approximate Inference
pproches to inference yesin Networks: pproximte Inference xct inference Vrillimintion Join tree lgorithm pproximte inference Simplify the structure of the network to mkxct inferencfficient (vritionl methods,
APPENDIX 2 LAPLACE TRANSFORMS
APPENDIX LAPLACE TRANSFORMS Thi ppendix preent hort introduction to Lplce trnform, the bic tool ued in nlyzing continuou ytem in the frequency domin. The Lplce trnform convert liner ordinry differentil
{ } = E! & $ " k r t +k +1
Chpter 4: Dynmic Progrmming Objectives of this chpter: Overview of collection of clssicl solution methods for MDPs known s dynmic progrmming (DP) Show how DP cn be used to compute vlue functions, nd hence,
Chapter 4: Dynamic Programming
Chpter 4: Dynmic Progrmming Objectives of this chpter: Overview of collection of clssicl solution methods for MDPs known s dynmic progrmming (DP) Show how DP cn be used to compute vlue functions, nd hence,
1 Online Learning and Regret Minimization
2.997 Decision-Mking in Lrge-Scle Systems My 10 MIT, Spring 2004 Hndout #29 Lecture Note 24 1 Online Lerning nd Regret Minimiztion In this lecture, we consider the problem of sequentil decision mking in
Genetic Programming. Outline. Evolutionary Strategies. Evolutionary strategies Genetic programming Summary
Outline Genetic Progrmming Evolutionry strtegies Genetic progrmming Summry Bsed on the mteril provided y Professor Michel Negnevitsky Evolutionry Strtegies An pproch simulting nturl evolution ws proposed
TP 10:Importance Sampling-The Metropolis Algorithm-The Ising Model-The Jackknife Method
TP 0:Importnce Smpling-The Metropoli Algorithm-The Iing Model-The Jckknife Method June, 200 The Cnonicl Enemble We conider phyicl ytem which re in therml contct with n environment. The environment i uully
CS103B Handout 18 Winter 2007 February 28, 2007 Finite Automata
CS103B ndout 18 Winter 2007 Ferury 28, 2007 Finite Automt Initil text y Mggie Johnson. Introduction Severl childrens gmes fit the following description: Pieces re set up on plying ord; dice re thrown or
Robot Planning in Partially Observable Continuous Domains
Robot Plnning in Prtilly Obervble Continuou Domin Joep M. Port Intitut de Robòtic i Informàtic Indutril (UPC-CSIC) Lloren i Artig 4-6, 828, Brcelon Spin Emil: port@iri.upc.edu Mtthij T. J. Spn Informtic
20.2. The Transform and its Inverse. Introduction. Prerequisites. Learning Outcomes
The Trnform nd it Invere 2.2 Introduction In thi Section we formlly introduce the Lplce trnform. The trnform i only pplied to cul function which were introduced in Section 2.1. We find the Lplce trnform
Robot Planning in Partially Observable Continuous Domains
Robot Plnning in Prtilly Obervble Continuou Domin Joep M. Port Intitut de Robòtic i Informàtic Indutril (UPC-CSIC) Lloren i Artig 4-6, 828, Brcelon Spin Emil: port@iri.upc.edu Mtthij T. J. Spn Informtic
Exploring parametric representation with the TI-84 Plus CE graphing calculator
Exploring prmetric representtion with the TI-84 Plus CE grphing clcultor Richrd Prr Executive Director Rice University School Mthemtics Project rprr@rice.edu Alice Fisher Director of Director of Technology
1B40 Practical Skills
B40 Prcticl Skills Comining uncertinties from severl quntities error propgtion We usully encounter situtions where the result of n experiment is given in terms of two (or more) quntities. We then need
Review of Gaussian Quadrature method
Review of Gussin Qudrture method Nsser M. Asi Spring 006 compiled on Sundy Decemer 1, 017 t 09:1 PM 1 The prolem To find numericl vlue for the integrl of rel vlued function of rel vrile over specific rnge
Reinforcement Learning for Robotic Locomotions
Reinforcement Lerning for Robotic Locomotion Bo Liu Stnford Univerity 121 Cmpu Drive Stnford, CA 94305, USA bliuxix@tnford.edu Hunzhong Xu Stnford Univerity 121 Cmpu Drive Stnford, CA 94305, USA xuhunvc@tnford.edu
CSCI 340: Computational Models. Transition Graphs. Department of Computer Science
CSCI 340: Computtionl Models Trnsition Grphs Chpter 6 Deprtment of Computer Science Relxing Restrints on Inputs We cn uild n FA tht ccepts only the word! 5 sttes ecuse n FA cn only process one letter t
We will see what is meant by standard form very shortly
THEOREM: For fesible liner progrm in its stndrd form, the optimum vlue of the objective over its nonempty fesible region is () either unbounded or (b) is chievble t lest t one extreme point of the fesible
p-adic Egyptian Fractions
p-adic Egyptin Frctions Contents 1 Introduction 1 2 Trditionl Egyptin Frctions nd Greedy Algorithm 2 3 Set-up 3 4 p-greedy Algorithm 5 5 p-egyptin Trditionl 10 6 Conclusion 1 Introduction An Egyptin frction
MA123, Chapter 10: Formulas for integrals: integrals, antiderivatives, and the Fundamental Theorem of Calculus (pp.
MA123, Chpter 1: Formuls for integrls: integrls, ntiderivtives, nd the Fundmentl Theorem of Clculus (pp. 27-233, Gootmn) Chpter Gols: Assignments: Understnd the sttement of the Fundmentl Theorem of Clculus.
Decision Networks. CS 188: Artificial Intelligence Fall Example: Decision Networks. Decision Networks. Decisions as Outcome Trees
CS 188: Artificil Intelligence Fll 2011 Decision Networks ME: choose the ction which mximizes the expected utility given the evidence mbrell Lecture 17: Decision Digrms 10/27/2011 Cn directly opertionlize
Accelerator Physics. G. A. Krafft Jefferson Lab Old Dominion University Lecture 5
Accelertor Phyic G. A. Krfft Jefferon L Old Dominion Univerity Lecture 5 ODU Accelertor Phyic Spring 15 Inhomogeneou Hill Eqution Fundmentl trnvere eqution of motion in prticle ccelertor for mll devition
CSCI565 - Compiler Design
CSCI565 - Compiler Deign Spring 6 Due Dte: Fe. 5, 6 t : PM in Cl Prolem [ point]: Regulr Expreion nd Finite Automt Develop regulr expreion (RE) tht detet the longet tring over the lphet {-} with the following
PRACTICE EXAM 2 SOLUTIONS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Deprtment of Phyic Phyic 8.01x Fll Term 00 PRACTICE EXAM SOLUTIONS Proble: Thi i reltively trihtforwrd Newton Second Lw problem. We et up coordinte ytem which i poitive
Cf. Linn Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability & Statistics, 1999.
Cf. Linn Sennott, Stochstic Dynmic Progrmming nd the Control of Queueing Systems, Wiley Series in Probbility & Sttistics, 1999. D.L.Bricker, 2001 Dept of Industril Engineering The University of Iow MDP
Chapter 3 Polynomials
Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling
Connected-components. Summary of lecture 9. Algorithms and Data Structures Disjoint sets. Example: connected components in graphs
Prm University, Mth. Deprtment Summry of lecture 9 Algorithms nd Dt Structures Disjoint sets Summry of this lecture: (CLR.1-3) Dt Structures for Disjoint sets: Union opertion Find opertion Mrco Pellegrini
Oracular Partially Observable Markov Decision Processes: A Very Special Case
Orculr Prtilly Obervble Mrkov Deciion Procee: A Very Specil Ce Nichol Armtrong-Crew nd Mnuel Veloo Robotic Intitute, Crnegie Mellon Univerity {nrmtro,veloo}@c.cmu.edu Abtrct We introduce the Orculr Prtilly
Evaluating Definite Integrals. There are a few properties that you should remember in order to assist you in evaluating definite integrals.
Evluting Definite Integrls There re few properties tht you should rememer in order to ssist you in evluting definite integrls. f x dx= ; where k is ny rel constnt k f x dx= k f x dx ± = ± f x g x dx f
Definite Integrals. The area under a curve can be approximated by adding up the areas of rectangles = 1 1 +
Definite Integrls --5 The re under curve cn e pproximted y dding up the res of rectngles. Exmple. Approximte the re under y = from x = to x = using equl suintervls nd + x evluting the function t the left-hnd
Today. Recap: Reasoning Over Time. Demo Bonanza! CS 188: Artificial Intelligence. Advanced HMMs. Speech recognition. HMMs. Start machine learning
CS 188: Artificil Intelligence Advnced HMMs Dn Klein, Pieter Aeel University of Cliforni, Berkeley Demo Bonnz! Tody HMMs Demo onnz! Most likely explntion queries Speech recognition A mssive HMM! Detils
ODE: Existence and Uniqueness of a Solution
Mth 22 Fll 213 Jerry Kzdn ODE: Existence nd Uniqueness of Solution The Fundmentl Theorem of Clculus tells us how to solve the ordinry differentil eqution (ODE) du = f(t) dt with initil condition u() =
2. The Laplace Transform
. The Lplce Trnform. Review of Lplce Trnform Theory Pierre Simon Mrqui de Lplce (749-87 French tronomer, mthemticin nd politicin, Miniter of Interior for 6 wee under Npoleon, Preident of Acdemie Frncie
Calculus Module C21. Areas by Integration. Copyright This publication The Northern Alberta Institute of Technology All Rights Reserved.
Clculus Module C Ares Integrtion Copright This puliction The Northern Alert Institute of Technolog 7. All Rights Reserved. LAST REVISED Mrch, 9 Introduction to Ares Integrtion Sttement of Prerequisite
DEFINITION The inner product of two functions f 1 and f 2 on an interval [a, b] is the number. ( f 1, f 2 ) b DEFINITION 11.1.
![DEFINITION The inner product of two functions f 1 and f 2 on an interval [a, b] is the number. ( f 1, f 2 ) b DEFINITION 11.1.](/thumbs/81/83893993.jpg "DEFINITION The inner product of two functions f 1 and f 2 on an interval [a, b] is the number. ( f 1, f 2 ) b DEFINITION 11.1.")
398 CHAPTER 11 ORTHOGONAL FUNCTIONS AND FOURIER SERIES 11.1 ORTHOGONAL FUNCTIONS REVIEW MATERIAL The notions of generlized vectors nd vector spces cn e found in ny liner lger text. INTRODUCTION The concepts
Efficient Planning in R-max
Efficient Plnning in R-mx Mrek Grześ nd Jee Hoey Dvid R. Cheriton School of Computer Science, Univerity of Wterloo 200 Univerity Avenue Wet, Wterloo, ON, N2L 3G1, Cnd {mgrze, jhoey}@c.uwterloo.c ABSTRACT
2. VECTORS AND MATRICES IN 3 DIMENSIONS
2 VECTORS AND MATRICES IN 3 DIMENSIONS 21 Extending the Theory of 2-dimensionl Vectors x A point in 3-dimensionl spce cn e represented y column vector of the form y z z-xis y-xis z x y x-xis Most of the
Discrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 17
EECS 70 Discrete Mthemtics nd Proility Theory Spring 2013 Annt Shi Lecture 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion,
Lecture Note 9: Orthogonal Reduction
MATH : Computtionl Methods of Liner Algebr 1 The Row Echelon Form Lecture Note 9: Orthogonl Reduction Our trget is to solve the norml eution: Xinyi Zeng Deprtment of Mthemticl Sciences, UTEP A t Ax = A
CS 310 (sec 20) - Winter Final Exam (solutions) SOLUTIONS
CS 310 (sec 20) - Winter 2003 - Finl Exm (solutions) SOLUTIONS 1. (Logic) Use truth tles to prove the following logicl equivlences: () p q (p p) (q q) () p q (p q) (p q) () p q p q p p q q (q q) (p p)
Decision Networks. CS 188: Artificial Intelligence. Decision Networks. Decision Networks. Decision Networks and Value of Information
CS 188: Artificil Intelligence nd Vlue of Informtion Instructors: Dn Klein nd Pieter Abbeel niversity of Cliforni, Berkeley [These slides were creted by Dn Klein nd Pieter Abbeel for CS188 Intro to AI
Flexible Beam. Objectives
Flexile Bem Ojectives The ojective of this l is to lern out the chllenges posed y resonnces in feedck systems. An intuitive understnding will e gined through the mnul control of flexile em resemling lrge
Jim Lambers MAT 169 Fall Semester Lecture 4 Notes
Jim Lmbers MAT 169 Fll Semester 2009-10 Lecture 4 Notes These notes correspond to Section 8.2 in the text. Series Wht is Series? An infinte series, usully referred to simply s series, is n sum of ll of
The Quest for Perfect and Compact Symmetry Breaking for Graph Problems
The Quest for Perfect nd Compct Symmetry Breking for Grph Prolems Mrijn J.H. Heule SYNASC Septemer 25, 2016 1/19 Stisfiility (SAT) solving hs mny pplictions... forml verifiction grph theory ioinformtics
4.1. Probability Density Functions
STT 1 4.1-4. 4.1. Proility Density Functions Ojectives. Continuous rndom vrile - vers - discrete rndom vrile. Proility density function. Uniform distriution nd its properties. Expected vlue nd vrince of
10 Vector Integral Calculus
Vector Integrl lculus Vector integrl clculus extends integrls s known from clculus to integrls over curves ("line integrls"), surfces ("surfce integrls") nd solids ("volume integrls"). These integrls hve
CONTROL SYSTEMS LABORATORY ECE311 LAB 3: Control Design Using the Root Locus
CONTROL SYSTEMS LABORATORY ECE311 LAB 3: Control Deign Uing the Root Locu 1 Purpoe The purpoe of thi lbortory i to deign cruie control ytem for cr uing the root locu. 2 Introduction Diturbnce D( ) = d
Surface maps into free groups
Surfce mps into free groups lden Wlker Novemer 10, 2014 Free groups wedge X of two circles: Set F = π 1 (X ) =,. We write cpitl letters for inverse, so = 1. e.g. () 1 = Commuttors Let x nd y e loops. The
Hidden Markov Models
Hidden Mrkov Models Huptseminr Mchine Lerning 18.11.2003 Referent: Nikols Dörfler 1 Overview Mrkov Models Hidden Mrkov Models Types of Hidden Mrkov Models Applictions using HMMs Three centrl problems:
AT100 - Introductory Algebra. Section 2.7: Inequalities. x a. x a. x < a
Section 2.7: Inequlities In this section, we will Determine if given vlue is solution to n inequlity Solve given inequlity or compound inequlity; give the solution in intervl nottion nd the solution 2.7
Today. CS 188: Artificial Intelligence. Recap: Reasoning Over Time. Particle Filters and Applications of HMMs. HMMs
CS 188: Artificil Intelligence Prticle Filters nd Applictions of HMMs Tody HMMs Prticle filters Demo onnz! Most-likely-explntion queries Instructors: Jco Andres nd Dvis Foote University of Cliforni, Berkeley
MArkov decision processes (MDPs) have been widely
Spre Mrkov Deciion Procee with Cul Spre Tlli Entropy Regulriztion for Reinforcement Lerning yungje Lee, Sungjoon Choi, nd Songhwi Oh rxiv:709.0693v3 [c.lg] 3 Oct 07 Abtrct In thi pper, re Mrkov deciion
An Optimal Best-First Search Algorithm for Solving Infinite Horizon DEC-POMDPs
An Optiml Best-First Serch Algorithm for Solving Infinite Horizon DEC-POMDPs Dniel Szer nd Frnçois Chrpillet INRIA Lorrine - LORIA, MAIA Group, 54506 Vndœuvre-lès-Nncy, Frnce {szer, chrp}@lori.fr http://mi.lori.fr
Discrete Mathematics and Probability Theory Summer 2014 James Cook Note 17
CS 70 Discrete Mthemtics nd Proility Theory Summer 2014 Jmes Cook Note 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion, y tking
Much ado about nothing: the mixed models controversy revisited
Much do out nothing: the mixed model controvery reviited Vivin etriz Lencin eprtmento de Metodologí e Invetigción, FM Univeridd Ncionl de Tucumán Julio d Mott Singer eprtmento de Ettític, IME Univeridde
Bases for Vector Spaces
Bses for Vector Spces 2-26-25 A set is independent if, roughly speking, there is no redundncy in the set: You cn t uild ny vector in the set s liner comintion of the others A set spns if you cn uild everything
MATRIX DEFINITION A matrix is any doubly subscripted array of elements arranged in rows and columns.
4.5 THEORETICL SOIL MECHNICS Vector nd Mtrix lger Review MTRIX DEFINITION mtrix is ny douly suscripted rry of elements rrnged in rows nd columns. m - Column Revised /0 n -Row m,,,,,, n n mn ij nd Order
Lecture 3. In this lecture, we will discuss algorithms for solving systems of linear equations.
Lecture 3 3 Solving liner equtions In this lecture we will discuss lgorithms for solving systems of liner equtions Multiplictive identity Let us restrict ourselves to considering squre mtrices since one
Thomas Whitham Sixth Form
Thoms Whithm Sith Form Pure Mthemtics Unit C Alger Trigonometry Geometry Clculus Vectors Trigonometry Compound ngle formule sin sin cos cos Pge A B sin Acos B cos Asin B A B sin Acos B cos Asin B A B cos
Transfer Functions. Chapter 5. Transfer Functions. Derivation of a Transfer Function. Transfer Functions
5/4/6 PM : Trnfer Function Chpter 5 Trnfer Function Defined G() = Y()/U() preent normlized model of proce, i.e., cn be ued with n input. Y() nd U() re both written in devition vrible form. The form of
CS 373, Spring Solutions to Mock midterm 1 (Based on first midterm in CS 273, Fall 2008.)
CS 373, Spring 29. Solutions to Mock midterm (sed on first midterm in CS 273, Fll 28.) Prolem : Short nswer (8 points) The nswers to these prolems should e short nd not complicted. () If n NF M ccepts
Generalized Fano and non-fano networks
Generlized Fno nd non-fno networks Nildri Ds nd Brijesh Kumr Ri Deprtment of Electronics nd Electricl Engineering Indin Institute of Technology Guwhti, Guwhti, Assm, Indi Emil: {d.nildri, bkri}@iitg.ernet.in
Chapter 2 Organizing and Summarizing Data. Chapter 3 Numerically Summarizing Data. Chapter 4 Describing the Relation between Two Variables
Copyright 013 Peron Eduction, Inc. Tble nd Formul for Sullivn, Sttitic: Informed Deciion Uing Dt 013 Peron Eduction, Inc Chpter Orgnizing nd Summrizing Dt Reltive frequency = frequency um of ll frequencie
CS 188: Artificial Intelligence Spring 2007
CS 188: Artificil Intelligence Spring 2007 Lecture 3: Queue-Bsed Serch 1/23/2007 Srini Nrynn UC Berkeley Mny slides over the course dpted from Dn Klein, Sturt Russell or Andrew Moore Announcements Assignment
Strong Bisimulation. Overview. References. Actions Labeled transition system Transition semantics Simulation Bisimulation
Strong Bisimultion Overview Actions Lbeled trnsition system Trnsition semntics Simultion Bisimultion References Robin Milner, Communiction nd Concurrency Robin Milner, Communicting nd Mobil Systems 32
Monte-Carlo-Based Partially Observable Markov Decision Process Approximations for Adaptive Sensing
Proceedings of the 9th Interntionl Workshop on Discrete Event Systems Göteborg, Sweden, My 28-30, 2008 WeE1.5 Monte-Crlo-Bsed Prtilly Observble Mrkov Decision Process Approximtions for Adptive Sensing
NFA DFA Example 3 CMSC 330: Organization of Programming Languages. Equivalence of DFAs and NFAs. Equivalence of DFAs and NFAs (cont.
NFA DFA Exmple 3 CMSC 330: Orgniztion of Progrmming Lnguges NFA {B,D,E {A,E {C,D {E Finite Automt, con't. R = { {A,E, {B,D,E, {C,D, {E 2 Equivlence of DFAs nd NFAs Any string from {A to either {D or {CD
Greedy regular expression matching
Alin Frisch INRIA Luc Crdelli MSRC 2004-05-15 ICALP The mtching prolem The prolem Project the structure of regulr expression on flt sequence. R = ( ) w = 1 2 1 2 3 v = [1 : [ 1 ; 2 ]; 2 : 1 ; 2 : 2 ; 1
Analysis of Variance and Design of Experiments-II
Anlyi of Vrince nd Deign of Experiment-II MODULE VI LECTURE - 7 SPLIT-PLOT AND STRIP-PLOT DESIGNS Dr. Shlbh Deprtment of Mthemtic & Sttitic Indin Intitute of Technology Knpur Anlyi of covrince ith one
CS 188: Artificial Intelligence
CS 188: Artificil Intelligence Lecture 19: Decision Digrms Pieter Abbeel --- C Berkeley Mny slides over this course dpted from Dn Klein, Sturt Russell, Andrew Moore Decision Networks ME: choose the ction
1 The Riemann Integral
The Riemnn Integrl. An exmple leding to the notion of integrl (res) We know how to find (i.e. define) the re of rectngle (bse height), tringle ( (sum of res of tringles). But how do we find/define n re
Calculus of variations with fractional derivatives and fractional integrals
Anis do CNMAC v.2 ISSN 1984-820X Clculus of vritions with frctionl derivtives nd frctionl integrls Ricrdo Almeid, Delfim F. M. Torres Deprtment of Mthemtics, University of Aveiro 3810-193 Aveiro, Portugl
The Value 1 Problem for Probabilistic Automata
The Vlue 1 Prolem for Proilistic Automt Bruxelles Nthnël Fijlkow LIAFA, Université Denis Diderot - Pris 7, Frnce Institute of Informtics, Wrsw University, Polnd nth@lif.univ-pris-diderot.fr June 20th,
10. AREAS BETWEEN CURVES
. AREAS BETWEEN CURVES.. Ares etween curves So res ove the x-xis re positive nd res elow re negtive, right? Wrong! We lied! Well, when you first lern out integrtion it s convenient fiction tht s true in
C/CS/Phys C191 Bell Inequalities, No Cloning, Teleportation 9/13/07 Fall 2007 Lecture 6
C/CS/Phys C9 Bell Inequlities, o Cloning, Teleporttion 9/3/7 Fll 7 Lecture 6 Redings Benenti, Csti, nd Strini: o Cloning Ch.4. Teleporttion Ch. 4.5 Bell inequlities See lecture notes from H. Muchi, Cltech,
Preview 11/1/2017. Greedy Algorithms. Coin Change. Coin Change. Coin Change. Coin Change. Greedy algorithms. Greedy Algorithms
Preview Greed Algorithms Greed Algorithms Coin Chnge Huffmn Code Greed lgorithms end to e simple nd strightforwrd. Are often used to solve optimiztion prolems. Alws mke the choice tht looks est t the moment,
Stochastic Programming Project Konrad Borys. Model for Optical Fiber Manufacturing
Stochstic Progrmming Project Konrd Borys Model for Opticl Fiber Mnufcturing. Introduction Opticl fibers re mde of solid rods of glss clled preforms. he s of the preforms re heted nd fibers re drwn from
8Similarity UNCORRECTED PAGE PROOFS. 8.1 Kick off with CAS 8.2 Similar objects 8.3 Linear scale factors. 8.4 Area and volume scale factors 8.
8.1 Kick off with S 8. Similr ojects 8. Liner scle fctors 8Similrity 8. re nd volume scle fctors 8. Review U N O R R E TE D P G E PR O O FS 8.1 Kick off with S Plese refer to the Resources t in the Prelims
First Midterm Examination
Çnky University Deprtment of Computer Engineering 203-204 Fll Semester First Midterm Exmintion ) Design DFA for ll strings over the lphet Σ = {,, c} in which there is no, no nd no cc. 2) Wht lnguge does
CS 188 Introduction to Artificial Intelligence Fall 2018 Note 7
CS 188 Introduction to Artificil Intelligence Fll 2018 Note 7 These lecture notes re hevily bsed on notes originlly written by Nikhil Shrm. Decision Networks In the third note, we lerned bout gme trees
Resources. Introduction: Binding. Resource Types. Resource Sharing. The type of a resource denotes its ability to perform different operations
Introduction: Binding Prt of 4-lecture introduction Scheduling Resource inding Are nd performnce estimtion Control unit synthesis This lecture covers Resources nd resource types Resource shring nd inding
STABILITY and Routh-Hurwitz Stability Criterion
Krdeniz Technicl Univerity Deprtment of Electricl nd Electronic Engineering 6080 Trbzon, Turkey Chpter 8- nd Routh-Hurwitz Stbility Criterion Bu der notlrı dece bu deri ln öğrencilerin kullnımın çık olup,
SIMULATION OF TRANSIENT EQUILIBRIUM DECAY USING ANALOGUE CIRCUIT
Bjop ol. o. Decemer 008 Byero Journl of Pure nd Applied Science, ():70 75 Received: Octoer, 008 Accepted: Decemer, 008 SIMULATIO OF TRASIET EQUILIBRIUM DECAY USIG AALOGUE CIRCUIT *Adullhi,.., Ango U.S.
FORM FIVE ADDITIONAL MATHEMATIC NOTE. ar 3 = (1) ar 5 = = (2) (2) (1) a = T 8 = 81
FORM FIVE ADDITIONAL MATHEMATIC NOTE CHAPTER : PROGRESSION Arithmetic Progression T n = + (n ) d S n = n [ + (n )d] = n [ + Tn ] S = T = T = S S Emple : The th term of n A.P. is 86 nd the sum of the first
Where did dynamic programming come from?
Where did dynmic progrmming come from? String lgorithms Dvid Kuchk cs302 Spring 2012 Richrd ellmn On the irth of Dynmic Progrmming Sturt Dreyfus http://www.eng.tu.c.il/~mi/cd/ or50/1526-5463-2002-50-01-0048.pdf
CS 188: Artificial Intelligence Fall 2010
CS 188: Artificil Intelligence Fll 2010 Lecture 18: Decision Digrms 10/28/2010 Dn Klein C Berkeley Vlue of Informtion 1 Decision Networks ME: choose the ction which mximizes the expected utility given
Math Advanced Calculus II
Mth 452 - Advnced Clculus II Line Integrls nd Green s Theorem The min gol of this chpter is to prove Stoke s theorem, which is the multivrible version of the fundmentl theorem of clculus. We will be focused
PHYS 601 HW 5 Solution. We wish to find a Fourier expansion of e sin ψ so that the solution can be written in the form
5 Solving Kepler eqution Conider the Kepler eqution ωt = ψ e in ψ We wih to find Fourier expnion of e in ψ o tht the olution cn be written in the form ψωt = ωt + A n innωt, n= where A n re the Fourier