Reasoning over Time or Space. CS 188: Artificial Intelligence. Outline. Markov Models. Conditional Independence. Query: P(X 4 )
|
|
- Hugo Grant
- 6 years ago
- Views:
Transcription
1 CS 88: Artificil Intelligence Lecture 7: HMMs nd Prticle Filtering Resoning over Time or Spce Often, we wnt to reson out sequence of oservtions Speech recognition Root locliztion User ttention Medicl monitoring Pieter Aeel --- UC Berkeley Mny slides over this course dpted from Dn Klein, Sturt Russell, Andrew Moore Need to introduce time (or spce) into our models 5 Outline Mrkov Models Mrkov Models ( = prticulr Byes net) Hidden Mrkov Models (HMMs) Representtion ( = nother prticulr Byes net) Inference Forwrd lgorithm ( = vrile elimintion) Prticle filtering ( = likelihood weighting with some tweks) Viteri (= vrile elimintion, ut replce sum y mx = grph serch) Dynmic Byes Nets Representtion (= yet nother prticulr Byes net) Inference: forwrd lgorithm nd prticle filtering 6 A Mrkov model is chin-structured BN Ech node is identiclly distriuted (sttionrity) Vlue of X t given time is clled the stte As BN: X X 2 X 3 X 4 Prmeters: clled trnsition proilities or dynmics, specify how the stte evolves over time (lso, initil stte proilities) Sme s MDP trnsition model, ut no choice of ction Conditionl Independence X X 2 X 3 X 4 Bsic conditionl independence: Pst nd future independent of the present Ech time step only depends on the previous This is clled the (first order) Mrkov property Note tht the chin is just (growing) BN We cn lwys use generic BN resoning on it if we truncte the chin t fixed length X X 2 X 3 X 4 Query: P(X 4 ) Slow nswer: inference y enumertion Enumerte ll sequences of length t which end in s Add up their proilities 8 = join on X, X 2, X 3, then sum over x, x 2, x 3 9
2 X X 2 X 3 X 4 Query: P(X 4 ) Query P(X_t) Fst nswer: vrile elimintion Order: X, X 2, X 3 X X 2 X 3 X 4 Vrile elimintion in order X, X 2,, X t- computes for k = 2, 3,, t Forwrd simultion = mini-forwrd lgorithm Note: common thred in this lecture: specil cses of lgorithms we lredy know, nd they hve specil nme in the context of HMMs for historicl resons. Exmple Mrkov Chin: Wether Sttes: X = {rin, sun} CPT P(X t X t- ): X t- X t P(X t X t- ) sun sun.9 sun rin. rin sun.3 rin rin.7 Exmple Run of Mini-Forwrd Algorithm From initil oservtion of sun P(X ) P(X 2 ) P(X 3 ) P(X 4 ) P(X ) From initil oservtion of rin P(X ) P(X 2 ) P(X 3 ) P(X 4 ) P(X ) Two new wys of representing the sme CPT, tht re often used for Mrkov models (These re not BNs!).7 rin.3. sun.9 sun rin sun rin 2 From yet nother initil distriution P(X ): P(X ) P(X ) 4 Sttionry Distriutions For most chins: influence of initil distriution gets less nd less over time. the distriution we end up in is independent of the initil distriution Sttionry distriution: Distriution we end up with is clled the sttionry distriution P of the chin It stisfies Appliction of Mrkov Chin Sttionry Distriution: We Link Anlysis PgeRnk over we grph Ech we pge is stte Initil distriution: uniform over pges Trnsitions: With pro. c, uniform jump to rndom pge (dotted lines, not ll shown) With pro. -c, follow rndom outlink (solid lines) Sttionry distriution Will spend more time on highly rechle pges E.g. mny wys to get to the Acrot Reder downlod pge Somewht roust to link spm Google. returned the set of pges contining ll your keywords in decresing rnk, now ll serch engines use link nlysis long with mny other fctors (rnk ctully getting less importnt over time) 6 2
3 Appliction of Mrkov Chin Sttionry Distriution: Gis Smpling* Ech joint instntition over ll hidden nd query vriles is stte. Let X = H \union Q Trnsitions: With proility /n resmple vrile X j ccording to P(X j x, x 2,, x j-, x j+,, x n, e,, e m ) Sttionry distriution: = conditionl distriution P(X, X 2,, X n e,, e m ) à When running Gis smpling long enough we get smple from the desired distriution! We did not prove this, ll we did is stting this result. 7 Outline Mrkov Models ( = prticulr Byes net) Hidden Mrkov Models (HMMs) Representtion ( = nother prticulr Byes net) Inference Forwrd lgorithm ( = vrile elimintion) Prticle filtering ( = likelihood weighting with some tweks) Viteri (= vrile elimintion, ut replce sum y mx = grph serch) Dynmic Byes Nets Representtion (= yet nother prticulr Byes net) Inference: forwrd lgorithm nd prticle filtering 8 Hidden Mrkov Models Exmple Mrkov chins not so useful for most gents Need oservtions to updte your eliefs Hidden Mrkov models (HMMs) Underlying Mrkov chin over sttes S You oserve outputs (effects) t ech time step As Byes net: X X 2 X 3 X 4 X 5 E E 2 E 3 E 4 E 5 An HMM is defined y: Initil distriution: Trnsitions: Emissions: Ghostusters HMM Conditionl Independence P(X ) = uniform P(X X ) = usully move clockwise, ut sometimes move in rndom direction or sty in plce P(R ij X) = sme sensor model s efore: red mens close, green mens fr wy. /9 /9 /9 /9 /9 /9 /9 /9 /9 P(X ) HMMs hve two importnt independence properties: Mrkov hidden process, future depends on pst vi the present Current oservtion independent of ll else given current stte X X 2 X 3 X 4 X 5 /6 /6 /2 X X 2 X 3 X 4 /6 E E 2 E 3 E 4 E 5 R i,j R i,j R i,j R i,j X 5 P(X X =<,2>) Quiz: does this men tht evidence vriles re gurnteed to e independent? [No, they tend to correlted y the hidden stte] E 5 3
4 Rel HMM Exmples Speech recognition HMMs: Oservtions re coustic signls (continuous vlued) Sttes re specific positions in specific words (so, tens of thousnds) Mchine trnsltion HMMs: Oservtions re words (tens of thousnds) Sttes re trnsltion options Root trcking: Oservtions re rnge redings (continuous) Sttes re positions on mp (continuous) Filtering / Monitoring Filtering, or monitoring, is the tsk of trcking the distriution B t (X) = P t (X t e,, e t ) (the elief stte) over time We strt with B (X) in n initil setting, usully uniform As time psses, or we get oservtions, we updte B(X) The Klmn filter ws invented in the 6 s nd first implemented s method of trjectory estimtion for the Apollo progrm Exmple: Root Locliztion Exmple: Root Locliztion Exmple from Michel Pfeiffer Pro t= Sensor model: cn red in which directions there is wll, never more thn mistke Motion model: my not execute ction with smll pro. Pro t= Lighter grey: ws possile to get the reding, ut less likely /c required mistke Exmple: Root Locliztion Exmple: Root Locliztion Pro Pro t=2 t=3 4
5 Exmple: Root Locliztion Exmple: Root Locliztion Pro Pro t=4 t=5 Query: P(X 4 e,e 2,e 3,e 4 ) --- Vrile Elimintion, X, X 2, X 3 P (X4 e,e2,e3,e4) P (X4,e,e2,e3,e4) = P (x,x2,x3,x4,e,e2,e3,e4) x,x2,x3 = P (e4 X4)P (X4 x3)p (e3 x3)p (x3 x2)p (e2 x2)p (x2 x)p (e x)p (x) x3 x2 x = P (e4 X4)P (X4 x3)p (e3 x3)p (x3 x2)p (e2 x2)p (x2 x)p (x,e) x3 x2 x = P (e4 X4)P (X4 x3)p (e3 x3)p (x3 x2)p (e2 x2) P (x2 x)p (x,e) x3 x2 x = P (e4 X4)P (X4 x3)p (e3 x3)p (x3 x2)p (e2 x2)p (x2,e) x3 x2 = P (e4 X4)P (X4 x3)p (e3 x3)p (x3 x2)p (x2,e,e2) x3 The Forwrd Algorithm We re given evidence t ech time nd wnt to know We cn derive the following updtes We cn normlize s we go if we wnt to hve P(x e) t ech time step, or just once t the end x2 = P (e4 X4)P (X4 x3)p (e3 x3) P (x3 x2)p (x2,e,e2) x3 x2 = P (e4 X4)P (X4 x3)p (e3 x3)p (x3,e,e2) x3 = P (e4 X4)P (X4 x3)p (x3,e,e2,e3) x3 Re-occurring computtion: = P (e4 X4) P (X4 x3)p (x3,e,e2,e3) x3 = P (e4 X4)P (x4,e,e2,e3) = P (X4,e,e2,e3,e4) 32 = exctly vrile elimintion in order X, X 2, Belief Updting = the forwrd lgorithm roken down into two steps nd with normliztion Belief updtes cn lso esily e derived from sic proility Forwrd lgorithm: Cn rek this down into: Time updte: Oservtion updte: Pssge of Time Given: P(X t ), P(X t+ X t ) Query: P(x t+ ) 8 x t+ Oservtion Given: P(X t+ ), P(e t+ X t+ ) Query: P(x t+ e t+ ) 8 x t+ X t+ Normlizing in the oservtion updte gives: Time updte: X t X t+ E t+ Oservtion updte: Nottion: Time updte: Oservtion updte: 5
6 Exmple: Pssge of Time As time psses, uncertinty ccumultes Exmple: Oservtion As we get oservtions, eliefs get reweighted, uncertinty decreses T = T = 2 T = 5 Before oservtion After oservtion Trnsition model: ghosts usully go clockwise Exmple HMM Outline Mrkov Models ( = prticulr Byes net) Hidden Mrkov Models (HMMs) Representtion ( = nother prticulr Byes net) Inference Forwrd lgorithm ( = vrile elimintion) Prticle filtering ( = likelihood weighting with some tweks) Viteri (= vrile elimintion, ut replce sum y mx = grph serch) Dynmic Byes Nets Representtion (= yet nother prticulr Byes net) Inference: forwrd lgorithm nd prticle filtering 43 Prticle Filtering Representtion: Prticles Filtering: pproximte solution Sometimes X is too ig to use exct inference X my e too ig to even store B(X) E.g. X is continuous Solution: pproximte inference Trck smples of X, not ll vlues Smples re clled prticles Time per step is liner in the numer of smples But: numer needed my e lrge In memory: list of prticles, not sttes This is how root locliztion works in prctice Prticle is just new nme for smple Our representtion of P(X) is now list of N prticles (smples) Generlly, N << X Storing mp from X to counts would defet the point P(x) pproximted y numer of prticles with vlue x So, mny x will hve P(x) =! More prticles, more ccurcy For now, ll prticles hve weight of Prticles: (,2) 45 6
7 Prticle Filtering: Elpse Time Prticle Filtering: Oserve Ech prticle is moved y smpling its next position from the trnsition model This is like prior smpling smples frequencies reflect the trnsition pros Here, most smples move clockwise, ut some move in nother direction or sty in plce This cptures the pssge of time If enough smples, close to exct vlues efore nd fter (consistent) Prticles: (,2) Prticles: (3,) (,3) (2,2) Slightly trickier: Don t smple oservtion, fix it Similr to likelihood weighting, downweight smples sed on the evidence As efore, the proilities don t sum to one, since most hve een downweighted (in fct they sum to n pproximtion of P(e)) Prticles: (3,) (,3) (2,2) Prticles: w=.9 w=.2 w=.9 (3,) w=.4 w=.4 w=.9 (,3) w=. w=.2 w=.9 (2,2) w=.4 Prticle Filtering: Resmple Recp: Prticle Filtering Rther thn trcking weighted smples, we resmple N times, we choose from our weighted smple distriution (i.e. drw with replcement) Prticles: w=.9 w=.2 w=.9 (3,) w=.4 w=.4 w=.9 (,3) w=. w=.2 w=.9 (2,2) w=.4 Prticles: trck smples of sttes rther thn n explicit distriution Elpse Weight Resmple This is equivlent to renormlizing the distriution Now the updte is complete for this time step, continue with the next one (New) Prticles: (2,2) (,3) Prticles: (,2) Prticles: (3,) (,3) (2,2) Prticles: w=.9 w=.2 w=.9 (3,) w=.4 w=.4 w=.9 (,3) w=. w=.2 w=.9 (2,2) w=.4 (New) Prticles: (2,2) (,3) 49 Outline Mrkov Models ( = prticulr Byes net) Hidden Mrkov Models (HMMs) Representtion ( = nother prticulr Byes net) Inference Forwrd lgorithm ( = vrile elimintion) Prticle filtering ( = likelihood weighting with some tweks) Viteri (= vrile elimintion, ut replce sum y mx = grph serch) Dynmic Byes Nets Representtion (= yet nother prticulr Byes net) Inference: forwrd lgorithm nd prticle filtering 5 Dynmic Byes Nets (DBNs) We wnt to trck multiple vriles over time, using multiple sources of evidence Ide: Repet fixed Byes net structure t ech time Vriles from time t cn condition on those from t- G E t = t =2 G E G 2 E 2 Discrete vlued dynmic Byes nets re lso HMMs G 2 E 2 G 3 E 3 t =3 G 3 E 3 7
8 Exct Inference in DBNs Vrile elimintion pplies to dynmic Byes nets Procedure: unroll the network for T time steps, then eliminte vriles until P(X T e :T ) is computed G E t = t =2 t =3 G E G 2 E 2 Online elief updtes: Eliminte ll vriles from the previous time step; store fctors for current time only G 2 E 2 G 3 E 3 G 3 E 3 52 DBN Prticle Filters A prticle is complete smple for time step Initilize: Generte prior smples for the t= Byes net Exmple prticle: G = G = (5,3) Elpse time: Smple successor for ech prticle Exmple successor: G 2 = G 2 = (6,3) Oserve: Weight ech entire smple y the likelihood of the evidence conditioned on the smple Likelihood: P(E G ) * P(E G ) Resmple: Select prior smples (tuples of vlues) in proportion to their likelihood 53 Trick I to Improve Prticle Filtering Performnce: Low Vrince Resmpling Trick II to Improve Prticle Filtering Performnce: Regulriztion If no or little noise in trnsitions model, ll prticles will strt to coincide Advntges: More systemtic coverge of spce of smples If ll smples hve sme importnce weight, no smples re lost Lower computtionl complexity à regulriztion: introduce dditionl (rtificil) noise into the trnsition model Root Locliztion In root locliztion: We know the mp, ut not the root s position Oservtions my e vectors of rnge finder redings Stte spce nd redings re typiclly continuous (works siclly like very fine grid) nd so we cnnot store B(X) Prticle filtering is min technique SLAM SLAM = Simultneous Locliztion And Mpping We do not know the mp or our loction Stte consists of position AND mp! Min techniques: Klmn filtering (Gussin HMMs) nd prticle methods Demos: glol-floor.gif 8
9 Prticle Filter Exmple 3 prticles SLAM DEMOS Intel-l-rw-odo.wmv Intel-l-scn-mtching.wmv visionslm_helioffice.wmv mp of prticle mp of prticle 3 mp of prticle 2 6 P4: Ghostusters 2. (et) Outline Plot: Pcmn's grndfther, Grndpc, lerned to hunt ghosts for sport. He ws linded y his power, ut could her the ghosts nging nd clnging. Trnsition Model: All ghosts move rndomly, ut re sometimes ised Emission Model: Pcmn knows noisy distnce to ech ghost Noisy distnce pro True distnce = Mrkov Models ( = prticulr Byes net) Hidden Mrkov Models (HMMs) Representtion ( = nother prticulr Byes net) Inference Forwrd lgorithm ( = vrile elimintion) Prticle filtering ( = likelihood weighting with some tweks) Viteri (= vrile elimintion, ut replce sum y mx = grph serch) Dynmic Byes Nets Representtion (= yet nother prticulr Byes net) Inference: forwrd lgorithm nd prticle filtering 63 Best Explntion Queries Best Explntion Query Solution Method : Serch slight use of nottion, ssuming P(x x ) = P(x ) X X 2 X 3 X 4 X 5 E E 2 E 3 E 4 E 5 Sttes: {(), +x, -x, +x 2, -x 2,, +x t, -x t } Strt stte: () Query: most likely seq: 64 Actions: in stte x k, choose ny ssignment for stte x k+ Cost: Gol test: gol(x k ) = true iff k == t à Cn run uniform cost grph serch to find solution à Uniform cost grph serch will tke O( t d 2 ). Think out this! 9
10 Best Explntion Query Solution Method 2: Viteri Algorithm (= mx-product version of forwrd lgorithm) Further redings We re done with Prt II Proilistic Resoning To lern more (eyond scope of 88): Koller nd Friedmn, Proilistic Grphicl Models (CS28A) Thrun, Burgrd nd Fox, Proilistic Rootics (CS287) Viteri computtionl complexity: O(t d 2 ) Compre to forwrd lgorithm: 66
CS 188: Artificial Intelligence Fall Announcements
CS 188: Artificil Intelligence Fll 2009 Lecture 20: Prticle Filtering 11/5/2009 Dn Klein UC Berkeley Announcements Written 3 out: due 10/12 Project 4 out: due 10/19 Written 4 proly xed, Project 5 moving
More informationToday. CS 188: Artificial Intelligence. Recap: Reasoning Over Time. Particle Filters and Applications of HMMs. HMMs
CS 188: Artificil Intelligence Prticle Filters nd Applictions of HMMs Tody HMMs Prticle filters Demo onnz! Most-likely-explntion queries Instructors: Jco Andres nd Dvis Foote University of Cliforni, Berkeley
More informationDecision Networks. CS 188: Artificial Intelligence Fall Example: Decision Networks. Decision Networks. Decisions as Outcome Trees
CS 188: Artificil Intelligence Fll 2011 Decision Networks ME: choose the ction which mximizes the expected utility given the evidence mbrell Lecture 17: Decision Digrms 10/27/2011 Cn directly opertionlize
More informationToday. Recap: Reasoning Over Time. Demo Bonanza! CS 188: Artificial Intelligence. Advanced HMMs. Speech recognition. HMMs. Start machine learning
CS 188: Artificil Intelligence Advnced HMMs Dn Klein, Pieter Aeel University of Cliforni, Berkeley Demo Bonnz! Tody HMMs Demo onnz! Most likely explntion queries Speech recognition A mssive HMM! Detils
More informationCS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2011 Lecture 18: HMMs and Particle Filtering 4/4/2011 Pieter Abbeel --- UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore
More informationPacman in the News. Announcements. Pacman in the News. CS 188: Ar7ficial Intelligence Par7cle Filters and A pplica7ons of HMMs
Announcements Surveys reminder Pcmn in the News HW 8 edx / grdescope consolid7on Mid- semester survey P4 Due Mondy 4/6 t 11:59pm P5 Ghostusters Due Fridy 4/10 t 5pm Pcmn in the News CS 188: Ar7ficil Intelligence
More informationBayesian Networks: Approximate Inference
pproches to inference yesin Networks: pproximte Inference xct inference Vrillimintion Join tree lgorithm pproximte inference Simplify the structure of the network to mkxct inferencfficient (vritionl methods,
More informationProbabilistic Reasoning. CS 188: Artificial Intelligence Spring Inference by Enumeration. Probability recap. Chain Rule à Bayes net
CS 188: Artificil Intelligence Spring 2011 Finl Review 5/2/2011 Pieter Aeel UC Berkeley Proilistic Resoning Proility Rndom Vriles Joint nd Mrginl Distriutions Conditionl Distriution Inference y Enumertion
More informationCSEP 573: Artificial Intelligence
CSEP 573: Artificial Intelligence Hidden Markov Models Luke Zettlemoyer Many slides over the course adapted from either Dan Klein, Stuart Russell, Andrew Moore, Ali Farhadi, or Dan Weld 1 Outline Probabilistic
More informationConvert the NFA into DFA
Convert the NF into F For ech NF we cn find F ccepting the sme lnguge. The numer of sttes of the F could e exponentil in the numer of sttes of the NF, ut in prctice this worst cse occurs rrely. lgorithm:
More informationMarkov Models. CS 188: Artificial Intelligence Fall Example. Mini-Forward Algorithm. Stationary Distributions.
CS 88: Artificial Intelligence Fall 27 Lecture 2: HMMs /6/27 Markov Models A Markov model is a chain-structured BN Each node is identically distributed (stationarity) Value of X at a given time is called
More informationReinforcement learning II
CS 1675 Introduction to Mchine Lerning Lecture 26 Reinforcement lerning II Milos Huskrecht milos@cs.pitt.edu 5329 Sennott Squre Reinforcement lerning Bsics: Input x Lerner Output Reinforcement r Critic
More informationAnnouncements. CS 188: Artificial Intelligence Fall Markov Models. Example: Markov Chain. Mini-Forward Algorithm. Example
CS 88: Artificial Intelligence Fall 29 Lecture 9: Hidden Markov Models /3/29 Announcements Written 3 is up! Due on /2 (i.e. under two weeks) Project 4 up very soon! Due on /9 (i.e. a little over two weeks)
More informationMarkov Chains and Hidden Markov Models
Markov Chains and Hidden Markov Models CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2018 Soleymani Slides are based on Klein and Abdeel, CS188, UC Berkeley. Reasoning
More informationNondeterminism and Nodeterministic Automata
Nondeterminism nd Nodeterministic Automt 61 Nondeterminism nd Nondeterministic Automt The computtionl mchine models tht we lerned in the clss re deterministic in the sense tht the next move is uniquely
More informationCS 188: Artificial Intelligence
CS 188: Artificil Intelligence Lecture 19: Decision Digrms Pieter Abbeel --- C Berkeley Mny slides over this course dpted from Dn Klein, Sturt Russell, Andrew Moore Decision Networks ME: choose the ction
More informationHidden Markov Models. Hal Daumé III. Computer Science University of Maryland CS 421: Introduction to Artificial Intelligence 19 Apr 2012
Hidden Markov Models Hal Daumé III Computer Science University of Maryland me@hal3.name CS 421: Introduction to Artificial Intelligence 19 Apr 2012 Many slides courtesy of Dan Klein, Stuart Russell, or
More informationCS 188: Artificial Intelligence Spring 2007
CS 188: Artificil Intelligence Spring 2007 Lecture 3: Queue-Bsed Serch 1/23/2007 Srini Nrynn UC Berkeley Mny slides over the course dpted from Dn Klein, Sturt Russell or Andrew Moore Announcements Assignment
More informationCS 343: Artificial Intelligence
CS 343: Artificial Intelligence Hidden Markov Models Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley.
More informationCS 188: Artificial Intelligence Spring 2009
CS 188: Artificial Intelligence Spring 2009 Lecture 21: Hidden Markov Models 4/7/2009 John DeNero UC Berkeley Slides adapted from Dan Klein Announcements Written 3 deadline extended! Posted last Friday
More informationDecision Networks. CS 188: Artificial Intelligence. Decision Networks. Decision Networks. Decision Networks and Value of Information
CS 188: Artificil Intelligence nd Vlue of Informtion Instructors: Dn Klein nd Pieter Abbeel niversity of Cliforni, Berkeley [These slides were creted by Dn Klein nd Pieter Abbeel for CS188 Intro to AI
More informationAnnouncements. CS 188: Artificial Intelligence Fall VPI Example. VPI Properties. Reasoning over Time. Markov Models. Lecture 19: HMMs 11/4/2008
CS 88: Artificial Intelligence Fall 28 Lecture 9: HMMs /4/28 Announcements Midterm solutions up, submit regrade requests within a week Midterm course evaluation up on web, please fill out! Dan Klein UC
More informationDiscrete Mathematics and Probability Theory Summer 2014 James Cook Note 17
CS 70 Discrete Mthemtics nd Proility Theory Summer 2014 Jmes Cook Note 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion, y tking
More informationCS 188: Artificial Intelligence
CS 188: Artificial Intelligence Hidden Markov Models Instructor: Anca Dragan --- University of California, Berkeley [These slides were created by Dan Klein, Pieter Abbeel, and Anca. http://ai.berkeley.edu.]
More informationCS 188 Introduction to Artificial Intelligence Fall 2018 Note 7
CS 188 Introduction to Artificil Intelligence Fll 2018 Note 7 These lecture notes re hevily bsed on notes originlly written by Nikhil Shrm. Decision Networks In the third note, we lerned bout gme trees
More informationDiscrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 17
EECS 70 Discrete Mthemtics nd Proility Theory Spring 2013 Annt Shi Lecture 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion,
More informationCSE 473: Ar+ficial Intelligence
CSE 473: Ar+ficial Intelligence Hidden Markov Models Luke Ze@lemoyer - University of Washington [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188
More information19 Optimal behavior: Game theory
Intro. to Artificil Intelligence: Dle Schuurmns, Relu Ptrscu 1 19 Optiml behvior: Gme theory Adversril stte dynmics hve to ccount for worst cse Compute policy π : S A tht mximizes minimum rewrd Let S (,
More informationCS 188: Artificial Intelligence Fall 2010
CS 188: Artificil Intelligence Fll 2010 Lecture 18: Decision Digrms 10/28/2010 Dn Klein C Berkeley Vlue of Informtion 1 Decision Networks ME: choose the ction which mximizes the expected utility given
More information2D1431 Machine Learning Lab 3: Reinforcement Learning
2D1431 Mchine Lerning Lb 3: Reinforcement Lerning Frnk Hoffmnn modified by Örjn Ekeberg December 7, 2004 1 Introduction In this lb you will lern bout dynmic progrmming nd reinforcement lerning. It is ssumed
More informationCSE 473: Artificial Intelligence
CSE 473: Artificial Intelligence Hidden Markov Models Dieter Fox --- University of Washington [Most slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials
More informationMinimal DFA. minimal DFA for L starting from any other
Miniml DFA Among the mny DFAs ccepting the sme regulr lnguge L, there is exctly one (up to renming of sttes) which hs the smllest possile numer of sttes. Moreover, it is possile to otin tht miniml DFA
More informationIntermediate Math Circles Wednesday, November 14, 2018 Finite Automata II. Nickolas Rollick a b b. a b 4
Intermedite Mth Circles Wednesdy, Novemer 14, 2018 Finite Automt II Nickols Rollick nrollick@uwterloo.c Regulr Lnguges Lst time, we were introduced to the ide of DFA (deterministic finite utomton), one
More informationGenetic Programming. Outline. Evolutionary Strategies. Evolutionary strategies Genetic programming Summary
Outline Genetic Progrmming Evolutionry strtegies Genetic progrmming Summry Bsed on the mteril provided y Professor Michel Negnevitsky Evolutionry Strtegies An pproch simulting nturl evolution ws proposed
More informationLECTURE NOTE #12 PROF. ALAN YUILLE
LECTURE NOTE #12 PROF. ALAN YUILLE 1. Clustering, K-mens, nd EM Tsk: set of unlbeled dt D = {x 1,..., x n } Decompose into clsses w 1,..., w M where M is unknown. Lern clss models p(x w)) Discovery of
More informationLecture 3. In this lecture, we will discuss algorithms for solving systems of linear equations.
Lecture 3 3 Solving liner equtions In this lecture we will discuss lgorithms for solving systems of liner equtions Multiplictive identity Let us restrict ourselves to considering squre mtrices since one
More informationCSE 473: Ar+ficial Intelligence. Probability Recap. Markov Models - II. Condi+onal probability. Product rule. Chain rule.
CSE 473: Ar+ficial Intelligence Markov Models - II Daniel S. Weld - - - University of Washington [Most slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188
More informationParse trees, ambiguity, and Chomsky normal form
Prse trees, miguity, nd Chomsky norml form In this lecture we will discuss few importnt notions connected with contextfree grmmrs, including prse trees, miguity, nd specil form for context-free grmmrs
More informationCMPSCI 250: Introduction to Computation. Lecture #31: What DFA s Can and Can t Do David Mix Barrington 9 April 2014
CMPSCI 250: Introduction to Computtion Lecture #31: Wht DFA s Cn nd Cn t Do Dvid Mix Brrington 9 April 2014 Wht DFA s Cn nd Cn t Do Deterministic Finite Automt Forml Definition of DFA s Exmples of DFA
More informationHidden Markov Models. Vibhav Gogate The University of Texas at Dallas
Hidden Markov Models Vibhav Gogate The University of Texas at Dallas Intro to AI (CS 4365) Many slides over the course adapted from either Dan Klein, Luke Zettlemoyer, Stuart Russell or Andrew Moore 1
More informationAUTOMATA AND LANGUAGES. Definition 1.5: Finite Automaton
25. Finite Automt AUTOMATA AND LANGUAGES A system of computtion tht only hs finite numer of possile sttes cn e modeled using finite utomton A finite utomton is often illustrted s stte digrm d d d. d q
More informationCSCI 340: Computational Models. Kleene s Theorem. Department of Computer Science
CSCI 340: Computtionl Models Kleene s Theorem Chpter 7 Deprtment of Computer Science Unifiction In 1954, Kleene presented (nd proved) theorem which (in our version) sttes tht if lnguge cn e defined y ny
More informationMA123, Chapter 10: Formulas for integrals: integrals, antiderivatives, and the Fundamental Theorem of Calculus (pp.
MA123, Chpter 1: Formuls for integrls: integrls, ntiderivtives, nd the Fundmentl Theorem of Clculus (pp. 27-233, Gootmn) Chpter Gols: Assignments: Understnd the sttement of the Fundmentl Theorem of Clculus.
More informationCS103B Handout 18 Winter 2007 February 28, 2007 Finite Automata
CS103B ndout 18 Winter 2007 Ferury 28, 2007 Finite Automt Initil text y Mggie Johnson. Introduction Severl childrens gmes fit the following description: Pieces re set up on plying ord; dice re thrown or
More informationCS 343: Artificial Intelligence
CS 343: Artificial Intelligence Particle Filters and Applications of HMMs Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro
More informationReview of Gaussian Quadrature method
Review of Gussin Qudrture method Nsser M. Asi Spring 006 compiled on Sundy Decemer 1, 017 t 09:1 PM 1 The prolem To find numericl vlue for the integrl of rel vlued function of rel vrile over specific rnge
More information4.1. Probability Density Functions
STT 1 4.1-4. 4.1. Proility Density Functions Ojectives. Continuous rndom vrile - vers - discrete rndom vrile. Proility density function. Uniform distriution nd its properties. Expected vlue nd vrince of
More informationLecture 3: Equivalence Relations
Mthcmp Crsh Course Instructor: Pdric Brtlett Lecture 3: Equivlence Reltions Week 1 Mthcmp 2014 In our lst three tlks of this clss, we shift the focus of our tlks from proof techniques to proof concepts
More informationContinuous Random Variables Class 5, Jeremy Orloff and Jonathan Bloom
Lerning Gols Continuous Rndom Vriles Clss 5, 8.05 Jeremy Orloff nd Jonthn Bloom. Know the definition of continuous rndom vrile. 2. Know the definition of the proility density function (pdf) nd cumultive
More informationCS415 Compilers. Lexical Analysis and. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University
CS415 Compilers Lexicl Anlysis nd These slides re sed on slides copyrighted y Keith Cooper, Ken Kennedy & Lind Torczon t Rice University First Progrmming Project Instruction Scheduling Project hs een posted
More informationThe Regulated and Riemann Integrals
Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue
More informationModel Reduction of Finite State Machines by Contraction
Model Reduction of Finite Stte Mchines y Contrction Alessndro Giu Dip. di Ingegneri Elettric ed Elettronic, Università di Cgliri, Pizz d Armi, 09123 Cgliri, Itly Phone: +39-070-675-5892 Fx: +39-070-675-5900
More informationProperties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives
Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn
More informationMath& 152 Section Integration by Parts
Mth& 5 Section 7. - Integrtion by Prts Integrtion by prts is rule tht trnsforms the integrl of the product of two functions into other (idelly simpler) integrls. Recll from Clculus I tht given two differentible
More informationLexical Analysis Finite Automate
Lexicl Anlysis Finite Automte CMPSC 470 Lecture 04 Topics: Deterministic Finite Automt (DFA) Nondeterministic Finite Automt (NFA) Regulr Expression NFA DFA A. Finite Automt (FA) FA re grph, like trnsition
More informationConnected-components. Summary of lecture 9. Algorithms and Data Structures Disjoint sets. Example: connected components in graphs
Prm University, Mth. Deprtment Summry of lecture 9 Algorithms nd Dt Structures Disjoint sets Summry of this lecture: (CLR.1-3) Dt Structures for Disjoint sets: Union opertion Find opertion Mrco Pellegrini
More informationDATA Search I 魏忠钰. 复旦大学大数据学院 School of Data Science, Fudan University. March 7 th, 2018
DATA620006 魏忠钰 Serch I Mrch 7 th, 2018 Outline Serch Problems Uninformed Serch Depth-First Serch Bredth-First Serch Uniform-Cost Serch Rel world tsk - Pc-mn Serch problems A serch problem consists of:
More informationQuantum Nonlocality Pt. 2: No-Signaling and Local Hidden Variables May 1, / 16
Quntum Nonloclity Pt. 2: No-Signling nd Locl Hidden Vriles My 1, 2018 Quntum Nonloclity Pt. 2: No-Signling nd Locl Hidden Vriles My 1, 2018 1 / 16 Non-Signling Boxes The primry lesson from lst lecture
More information1.4 Nonregular Languages
74 1.4 Nonregulr Lnguges The number of forml lnguges over ny lphbet (= decision/recognition problems) is uncountble On the other hnd, the number of regulr expressions (= strings) is countble Hence, ll
More informationp-adic Egyptian Fractions
p-adic Egyptin Frctions Contents 1 Introduction 1 2 Trditionl Egyptin Frctions nd Greedy Algorithm 2 3 Set-up 3 4 p-greedy Algorithm 5 5 p-egyptin Trditionl 10 6 Conclusion 1 Introduction An Egyptin frction
More informationFinite Automata-cont d
Automt Theory nd Forml Lnguges Professor Leslie Lnder Lecture # 6 Finite Automt-cont d The Pumping Lemm WEB SITE: http://ingwe.inghmton.edu/ ~lnder/cs573.html Septemer 18, 2000 Exmple 1 Consider L = {ww
More informationMonte Carlo method in solving numerical integration and differential equation
Monte Crlo method in solving numericl integrtion nd differentil eqution Ye Jin Chemistry Deprtment Duke University yj66@duke.edu Abstrct: Monte Crlo method is commonly used in rel physics problem. The
More information1 Nondeterministic Finite Automata
1 Nondeterministic Finite Automt Suppose in life, whenever you hd choice, you could try oth possiilities nd live your life. At the end, you would go ck nd choose the one tht worked out the est. Then you
More informationHomework 3 Solutions
CS 341: Foundtions of Computer Science II Prof. Mrvin Nkym Homework 3 Solutions 1. Give NFAs with the specified numer of sttes recognizing ech of the following lnguges. In ll cses, the lphet is Σ = {,1}.
More informationHidden Markov Models
Hidden Mrkov Models Huptseminr Mchine Lerning 18.11.2003 Referent: Nikols Dörfler 1 Overview Mrkov Models Hidden Mrkov Models Types of Hidden Mrkov Models Applictions using HMMs Three centrl problems:
More informationModule 9: Tries and String Matching
Module 9: Tries nd String Mtching CS 240 - Dt Structures nd Dt Mngement Sjed Hque Veronik Irvine Tylor Smith Bsed on lecture notes by mny previous cs240 instructors Dvid R. Cheriton School of Computer
More informationModule 9: Tries and String Matching
Module 9: Tries nd String Mtching CS 240 - Dt Structures nd Dt Mngement Sjed Hque Veronik Irvine Tylor Smith Bsed on lecture notes by mny previous cs240 instructors Dvid R. Cheriton School of Computer
More informationLecture 2e Orthogonal Complement (pages )
Lecture 2e Orthogonl Complement (pges -) We hve now seen tht n orthonorml sis is nice wy to descrie suspce, ut knowing tht we wnt n orthonorml sis doesn t mke one fll into our lp. In theory, the process
More informationBases for Vector Spaces
Bses for Vector Spces 2-26-25 A set is independent if, roughly speking, there is no redundncy in the set: You cn t uild ny vector in the set s liner comintion of the others A set spns if you cn uild everything
More information20 MATHEMATICS POLYNOMIALS
0 MATHEMATICS POLYNOMIALS.1 Introduction In Clss IX, you hve studied polynomils in one vrible nd their degrees. Recll tht if p(x) is polynomil in x, the highest power of x in p(x) is clled the degree of
More informationModule 6 Value Iteration. CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo
Module 6 Vlue Itertion CS 886 Sequentil Decision Mking nd Reinforcement Lerning University of Wterloo Mrkov Decision Process Definition Set of sttes: S Set of ctions (i.e., decisions): A Trnsition model:
More information1 The Lagrange interpolation formula
Notes on Qudrture 1 The Lgrnge interpoltion formul We briefly recll the Lgrnge interpoltion formul. The strting point is collection of N + 1 rel points (x 0, y 0 ), (x 1, y 1 ),..., (x N, y N ), with x
More informationCS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Particle Filters and Applications of HMMs Instructor: Wei Xu Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley.] Recap: Reasoning
More informationCS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Particle Filters and Applications of HMMs Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials
More information1B40 Practical Skills
B40 Prcticl Skills Comining uncertinties from severl quntities error propgtion We usully encounter situtions where the result of n experiment is given in terms of two (or more) quntities. We then need
More information1 Probability Density Functions
Lis Yn CS 9 Continuous Distributions Lecture Notes #9 July 6, 28 Bsed on chpter by Chris Piech So fr, ll rndom vribles we hve seen hve been discrete. In ll the cses we hve seen in CS 9, this ment tht our
More informationConcepts of Concurrent Computation Spring 2015 Lecture 9: Petri Nets
Concepts of Concurrent Computtion Spring 205 Lecture 9: Petri Nets Sebstin Nnz Chris Poskitt Chir of Softwre Engineering Petri nets Petri nets re mthemticl models for describing systems with concurrency
More informationFingerprint idea. Assume:
Fingerprint ide Assume: We cn compute fingerprint f(p) of P in O(m) time. If f(p) f(t[s.. s+m 1]), then P T[s.. s+m 1] We cn compre fingerprints in O(1) We cn compute f = f(t[s+1.. s+m]) from f(t[s.. s+m
More information8 Laplace s Method and Local Limit Theorems
8 Lplce s Method nd Locl Limit Theorems 8. Fourier Anlysis in Higher DImensions Most of the theorems of Fourier nlysis tht we hve proved hve nturl generliztions to higher dimensions, nd these cn be proved
More informationCSE 473: Artificial Intelligence Spring 2014
CSE 473: Artificial Intelligence Spring 2014 Hidden Markov Models Hanna Hajishirzi Many slides adapted from Dan Weld, Pieter Abbeel, Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1 Outline
More informationDesigning finite automata II
Designing finite utomt II Prolem: Design DFA A such tht L(A) consists of ll strings of nd which re of length 3n, for n = 0, 1, 2, (1) Determine wht to rememer out the input string Assign stte to ech of
More informationCS 343: Artificial Intelligence
CS 343: Artificial Intelligence Particle Filters and Applications of HMMs Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro
More informationCS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Hidden Markov Models Instructor: Wei Xu Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley.] Pacman Sonar (P4) [Demo: Pacman Sonar
More informationApproximate Inference
Approximate Inference Simulation has a name: sampling Sampling is a hot topic in machine learning, and it s really simple Basic idea: Draw N samples from a sampling distribution S Compute an approximate
More informationState space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies
Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response
More informationFinite Automata. Informatics 2A: Lecture 3. John Longley. 22 September School of Informatics University of Edinburgh
Lnguges nd Automt Finite Automt Informtics 2A: Lecture 3 John Longley School of Informtics University of Edinburgh jrl@inf.ed.c.uk 22 September 2017 1 / 30 Lnguges nd Automt 1 Lnguges nd Automt Wht is
More informationAPPROXIMATE INTEGRATION
APPROXIMATE INTEGRATION. Introduction We hve seen tht there re functions whose nti-derivtives cnnot be expressed in closed form. For these resons ny definite integrl involving these integrnds cnnot be
More informationCS5371 Theory of Computation. Lecture 20: Complexity V (Polynomial-Time Reducibility)
CS5371 Theory of Computtion Lecture 20: Complexity V (Polynomil-Time Reducibility) Objectives Polynomil Time Reducibility Prove Cook-Levin Theorem Polynomil Time Reducibility Previously, we lernt tht if
More informationAdministrivia CSE 190: Reinforcement Learning: An Introduction
Administrivi CSE 190: Reinforcement Lerning: An Introduction Any emil sent to me bout the course should hve CSE 190 in the subject line! Chpter 4: Dynmic Progrmming Acknowledgment: A good number of these
More informationCS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Hidden Markov Models Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials available at http://ai.berkeley.edu.]
More informationSurface maps into free groups
Surfce mps into free groups lden Wlker Novemer 10, 2014 Free groups wedge X of two circles: Set F = π 1 (X ) =,. We write cpitl letters for inverse, so = 1. e.g. () 1 = Commuttors Let x nd y e loops. The
More informationLecture 9: LTL and Büchi Automata
Lecture 9: LTL nd Büchi Automt 1 LTL Property Ptterns Quite often the requirements of system follow some simple ptterns. Sometimes we wnt to specify tht property should only hold in certin context, clled
More informationMath 1B, lecture 4: Error bounds for numerical methods
Mth B, lecture 4: Error bounds for numericl methods Nthn Pflueger 4 September 0 Introduction The five numericl methods descried in the previous lecture ll operte by the sme principle: they pproximte the
More informationTheoretical foundations of Gaussian quadrature
Theoreticl foundtions of Gussin qudrture 1 Inner product vector spce Definition 1. A vector spce (or liner spce) is set V = {u, v, w,...} in which the following two opertions re defined: (A) Addition of
More informationRandom subgroups of a free group
Rndom sugroups of free group Frédérique Bssino LIPN - Lortoire d Informtique de Pris Nord, Université Pris 13 - CNRS Joint work with Armndo Mrtino, Cyril Nicud, Enric Ventur et Pscl Weil LIX My, 2015 Introduction
More informationNFAs continued, Closure Properties of Regular Languages
Algorithms & Models of Computtion CS/ECE 374, Fll 2017 NFAs continued, Closure Properties of Regulr Lnguges Lecture 5 Tuesdy, Septemer 12, 2017 Sriel Hr-Peled (UIUC) CS374 1 Fll 2017 1 / 31 Regulr Lnguges,
More informationNFAs continued, Closure Properties of Regular Languages
lgorithms & Models of omputtion S/EE 374, Spring 209 NFs continued, losure Properties of Regulr Lnguges Lecture 5 Tuesdy, Jnury 29, 209 Regulr Lnguges, DFs, NFs Lnguges ccepted y DFs, NFs, nd regulr expressions
More informationCoalgebra, Lecture 15: Equations for Deterministic Automata
Colger, Lecture 15: Equtions for Deterministic Automt Julin Slmnc (nd Jurrin Rot) Decemer 19, 2016 In this lecture, we will study the concept of equtions for deterministic utomt. The notes re self contined
More informationCS 373, Spring Solutions to Mock midterm 1 (Based on first midterm in CS 273, Fall 2008.)
CS 373, Spring 29. Solutions to Mock midterm (sed on first midterm in CS 273, Fll 28.) Prolem : Short nswer (8 points) The nswers to these prolems should e short nd not complicted. () If n NF M ccepts
More informationReinforcement Learning
Reinforcement Lerning Tom Mitchell, Mchine Lerning, chpter 13 Outline Introduction Comprison with inductive lerning Mrkov Decision Processes: the model Optiml policy: The tsk Q Lerning: Q function Algorithm
More information