Computer Science 188 Artificial Intelligence. Introduction to Probability. Probability Ryan Waliany

Transcription

1 Computer Siee 88 Artifiial Itelligee Rya Waliay Note: this is meat to be a simple referee sheet ad help studets uder the derivatios. If there s aythig that seems shaky or iorret do t hesitate to me at rwaliay@gmail.om Itrodutio to Radom Variables A radom variable is a arbitrary measuremet of the world whih may have some degree uertaity. For example, we a have the radom variable H = will the ext oi flip be heads or tails?, C = will the robot rash geerally deoted by apital letters where as a realized value are deoted by a lower-ase letter. Assume we are avigatig a robot i the Urba DARPA Grad Challege, robots are drive autoomously o real roads with traffi sigals. We a the represet probabilities depedig o what evidee we observe. For example, if we saw that P C = ras ACION = tur left, we ould probably argue that P(C = ras ACION = tur rigt) is greater sie, left turs are ofte more omplex ad more diffiult agaist traffi. If we also had evidee that there was a ar i your way, we kow that there s a higher likelihood of you rashig. hus, P C = ras ACION = tur left, CAR = true > P(C = ras ACION = tur_left ) Normalizatio x P x = Margializatio P X = P(X, X 2 = x 2 ) x 2 Coditioal Probabilities P a b = P a, b P b Normalizatio rik P x x 2 = P x, x 2 P x 2 = P x, x 2 P x, x 2 x Chai Rule P(A... Z) = P(A B... Z) P(B C... Z)... P(Y Z) P(Z)

2 Computer Siee 88 Artifiial Itelligee Rya Waliay Bayes Rule Idepedee P x y = P y x P x P y P A, B = P A P(B) Direted Graphial Models Represetatio Node Eah ode represets a radom variable Edges Represet a relatio of ausality A direted arrow from A to B represets that A auses B. C A B I this ase, B is oditioally idepedet of C or P(B A, C) = P(B A). More formally, we a represet the probability of a give ode oditioed o the world. P B π B, = P(B π B ) π X is a term that referees the parets of X ad π B = A i our example. hus, we a represet the joit distributio of ay diret graphial model as P x, x 2, = i= P(x i π x i What is the direted graphial model represetig N idepedet oi flips?

3 Computer Siee 88 Artifiial Itelligee Rya Waliay Probabilisti Part of Speeh aggig What is taggig? aggig is the proess of markig the words i a text orrespodig to their part of speeh. A tag a be a ou, verb, adjetive, adverb or a eve more desriptive subtype suh as a proper ou. he aïve Bayes probabilisti model C F F 2 F We eed to determie the probability that the lass is a ou give a set of features desribig the text, P(C F,, F ) P C F,, F = P C, F,, F P F, F = P C P F,, F C P F,, F However, sie we kow that P F,, F is ostat, we a drop the term whe takig a max over lasses. max P C F,, F = P C P F,, F C Sie F,, F are oditioally idepedet give C, max P C F,, F = P C i= P(F i C) Beause we do t kow the exat probabilities, we a estimate them. max P C F,, F = P C i= P(F i C)

4 Computer Siee 88 Artifiial Itelligee Rya Waliay P C = C C C P F i C = C F i, C C C Where we defie P as simply the frequey. I this example, we assume that C is the tag that we are tryig to lassify to a word. Ad the features that are otaied i the word are self-determied by us. his a simply be a Boolea flag suh that the first letter is apital. Or it a be simply the letters otaied i the word, or pairs of letters (bigram), or triples of letters (trigram). Some drawbaks of usig higher-order features is that it requires a substatial amout of more traiig data ad it requires more memory to store examples. w w 2 w Hidde Markov-Models A Markov model is distiguished by two properties.. Limited horizo: P X i+ = x X,, X = P X i+ = x X i 2. ime ivariat: P X i+ = x X i = P(X 2 = x X ) A hidde Markov model is a form of a markov hai whih otais a set of states X ad some observable harateristi Y. 2 3 W W 2 W 3 W 4

5 Computer Siee 88 Artifiial Itelligee Rya Waliay We wish to solve the equatio: arg max P, W, t, P W,, P, = arg max t, P W, Sie, P(W, ) is ostat for maximizig,, we a simply remove it from the maximizig equatio = arg max P W,, P, = arg max ( P(W i i )) P, = arg max P W i i P( i i ) For simpliity sake, we assume that P 0 =, ad we a simply alulate the maximum likelihood estimate (MLE) arg max P, W, = arg max t, P W i i P( i i ) We ould evaluate this equatio for all possible taggig, however, that would make taggig of expoetial legth. Oe might look at a more effiiet algorithm for HMM s suh as Viterbi to perform this omputatio. Viterbi is a dyami programmig algorithm that is for fidig the most likely sequee of hidde states.