Outline. L7: Probability Basics. Probability. Probability Theory. Bayes Law for Diagnosis. Which Hypothesis To Prefer? p(a,b) = p(b A) " p(a)

Transcription

1 Outlie L7: Probability Basics CS 344R/393R: Robotics Bejami Kuipers. Bayes Law 2. Probability distributios 3. Decisios uder ucertaity Probability For a propositio A, the probability p(a is your degree of belief i the truth of A. By covetio, 0 p(a. This is the Bayesia view of probability. It cotrasts with the view that probability is the frequecy that A is true, over some large populatio of experimets. The frequetist view makes it awkward to use data to estimate the value of a costat. Probability Theory p(a,b is the joit probability of A ad B. p(a B is the coditioal probability of A give B. p(a B + p( A B = Bayes Law: p(a,b = p(b A " p(a p(b A = p(a, B p(a = p(a B " p(b p(a Bayes Law for Diagosis Let H be a hypothesis, E be evidece. p(e H " p(h p(h E = p(e p(e H is the likelihood of the data, give the hypothesis. p(h is prior probability of hypothesis. p(e is prior probability of the evidece (but acts as a ormalizig costat. p(h E is what you really wat to kow (posterior probability of hypothesis. Which Hypothesis To Prefer? Maximum Likelihood (ML max H p(e H The model that makes the data most likely Maximum a posteriori (MAP max H p(e H p(h The model that is the most probable explaatio (Story: perfect match to rare disease

2 Bayes Law The deomiator i Bayes Law acts as a ormalizig costat: p(e H p(h p(h E = = " p(e H p(h p(e " = p(e # = p(e H p(h H It esures that the probabilities sum to across all the hypotheses H. Idepedece Two radom variables are idepedet if p(x,y = p(x p(y p(x Y = p(x p(y X = p(y These are all equivalet. X ad Y are coditioally idepedet give Z if p(x,y Z = p(x Z p(y Z p(x Y, Z = p(x Z p(y X, Z = p(y Z Idepedece simplifies iferece. Accumulatig Evidece (aïve Bayes p(h d Ld = p(h p(d H p(d p(h d Ld = p(h * " i= p(d 2 H L p(d H p(d 2 p(d p(d i H p(d i p(h d Ld = " p(h * p(d i H # log p(h d Ld = log p(h + " log p(d i H + # i= i= Bayes ets Represet Depedece The odes are radom variables. The liks represet depedece. p(x i parets(x i Idepedece ca be iferred from etwork The etwork represets how the joit probability distributio ca be decomposed. " p(x,l X = p(x i parets(x i i= There are effective propagatio algorithms. Simple Bayes et Example Outlie. Bayes Law 2. Probability distributios 3. Decisios uder ucertaity 2

3 Expectatios Let x be a radom variable. The expected value E[x] is the mea: E[x] = " x p(x dx # x = The probability-weighted mea of all possible values. The sample mea approaches it. Expected value of a vector x is by compoet. E[x] = x = [x,lx ] T x i Variace ad Covariace The variace is E[ (x-e[x] 2 ] " 2 = E[(x # x 2 ] = (x i # x 2 Covariace matrix is E[ (x-e[x](x-e[x] T ] C ij = # k = Divide by to make the sample variace a ubiased estimator for the populatio variace. (x ik " x i (x jk " x j Biased ad Ubiased Estimators Strictly speakig, the sample variace " 2 = E[(x # x 2 ] = (x i # x 2 is a biased estimate of the populatio variace. A ubiased estimator is: s 2 = #(x i " x 2 " But: If the differece betwee ad ever matters to you, the you are probably up to o good ayway [Press, et al] Covariace Matrix Alog the diagoal, C ii are variaces. Off-diagoal C ij are essetially correlatios. # 2 C, = " C,2 C, & 2 ( C 2, C 2,2 = " 2 ( O M ( 2 ( C, L C, = " ' Idepedet Variatio x ad y are Gaussia radom variables (=00 Geerated with σ x = σ y =3 Covariace matrix: " C xy = ' # & c ad d are radom variables. Geerated with c=x+y d=x-y Covariace matrix: Depedet Variatio # 0.62 "7.93& C cd = ( " ' 3

4 Estimates ad Ucertaity Coditioal probability desity fuctio Gaussia (ormal Distributio Completely described by (µ,σ Mea µ Stadard deviatio σ, variace σ 2 " 2# e( x µ 2 / 2" 2 The Cetral Limit Theorem Illustratig the Cetral Limit Thm Add, 2, 3, 4 variables from the same distributio. The sum of may radom variables with the same mea, but with arbitrary coditioal desity fuctios, coverges to a Gaussia desity fuctio. If a model omits may small umodeled effects, the the resultig error should coverge to a Gaussia desity fuctio. Detectig Modelig Error Every model is icomplete. If the omitted factors are all small, the resultig errors should add up to a Gaussia. If the error betwee a model ad the data is ot Gaussia, The some omitted factor is ot small. Oe should fid the domiat source of error ad add it to the model. Outlie. Bayes Law 2. Probability distributios 3. Decisios uder ucertaity 4

5 Diagostic Errors ad Sesor Iterpretatio Iterpretig sesor values is like diagosis. Disease preset Disease abset Test=Pos True Positive False Positive Test=eg False egative True egative hit false alarm miss correct reject Tests: Sesor oise ad Decisio Thresholds Overlappig respose to differet cases: o Yes Every test has false positives ad egatives. Soar(fwd=d implies Obstacle-at-distace(d?? The Test Threshold Requires a Trade-Off You ca t elimiate all error. Choose which errors are importat ROC Curves The overlap d cotrols the trade-off betwee types of errors. For more, search o Sigal Detectio Theory. d'= separatio spread Bayesia Reasoig Oe stregth of Bayesia methods is that they reaso with probability distributios, ot just the most likely idividual case. For more, see Adrew Moore s tutorial slides Comig up: Regressio to fid models from data Kalma filters to track dyamical systems Visual object trackers. 5