robablty reew dopted from notes of ndrew W. Moore and Erc Xng from CMU Copyrght ndrew W. Moore Slde So far our classfers are determnstc! For a gen X, the classfers we learned so far ge a sngle predcted y alue In contrast, a probablstc predcton returns a probablty oer the output space y X, y X We can easly thnk of stuatons when ths would be ery useful! Gen y X.49, y- X.5, how would you predct? What f I tell you t s much more costly to mss an poste example than the other way around? Copyrght ndrew W. Moore Slde
Dscrete Random Varables s a oolean-alued random arable f denotes an eent, and there s some degree of uncertanty as to whether occurs. Examples The US presdent n 3 wll be male You wake up tomorrow wth a headache You hae Ebola Copyrght ndrew W. Moore Slde 3 robabltes We wrte as the fracton of possble worlds n whch s true We could at ths pont spend hours on the phlosophy of ths. ut we won t. Copyrght ndrew W. Moore Slde 4
Vsualzng Eent space of all possble worlds Worlds n whch s true rea of reddsh oal Its area s Worlds n whch s False Copyrght ndrew W. Moore Slde 5 asc axoms < < True False or + - and Worlds n whch s False or and Smple addton and subtracton Copyrght ndrew W. Moore Slde 6 3
Elementary robablty Theorems ~ + ^ + ^ ~ Copyrght ndrew W. Moore Slde 7 Multalued Random Varables Suppose can take on more than alues s a random arable wth arty k f t can take on exactly one alue out of {,,.. k } Thus f k Copyrght ndrew W. Moore Slde 8 4
5 Copyrght ndrew W. Moore Slde 9 n easy fact about Multalued Random Varables: Usng the axoms of probablty < <, True, False or + - and nd assumng that obeys It s easy to proe that f k Copyrght ndrew W. Moore Slde n easy fact about Multalued Random Varables: Usng the axoms of probablty < <, True, False or + - and nd assumng that obeys It s easy to proe that f k nd thus we can proe k
6 Copyrght ndrew W. Moore Slde nother fact about Multalued Random Varables: Usng the axoms of probablty < <, True, False or + - and nd assumng that obeys It s easy to proe that f k ] [ Copyrght ndrew W. Moore Slde nother fact about Multalued Random Varables: Usng the axoms of probablty < <, True, False or + - and nd assumng that obeys It s easy to proe that f k ] [ nd thus we can proe k
Elementary robablty n ctures k k Copyrght ndrew W. Moore Slde 3 Condtonal robablty Fracton of worlds n whch s true that also hae true H Hae a headache F Comng down wth Flu F H / F /4 H F / H Headaches are rare and flu s rarer, but f you re comng down wth flu there s a 5-5 chance you ll hae a headache. Copyrght ndrew W. Moore Slde 4 7
Condtonal robablty F H F Fracton of flu-nflcted worlds n whch you hae a headache H Hae a headache F Comng down wth Flu H / F /4 H F / H #worlds wth flu and headache ------------------------------------ #worlds wth flu rea of H and F regon ------------------------------ rea of F regon H ^ F ----------- F Copyrght ndrew W. Moore Slde 5 Defnton of Condtonal robablty ^ ----------- Corollary: The Chan Rule ^ Copyrght ndrew W. Moore Slde 6 8
robablstc Inference F H Hae a headache F Comng down wth Flu H H / F /4 H F / One day you wake up wth a headache. You thnk: Drat! 5% of flus are assocated wth headaches so I must hae a 5-5 chance of comng down wth flu Is ths reasonng good? Copyrght ndrew W. Moore Slde 7 robablstc Inference F H Hae a headache F Comng down wth Flu H H / F /4 H F / F ^ H F H Copyrght ndrew W. Moore Slde 8 9
What we ust dd ^ ----------- --------------- Ths s ayes Rule ayes, Thomas 763 n essay towards solng a problem n the doctrne of chances. hlosophcal Transactons of the Royal Socety of London, 53:37-48 Copyrght ndrew W. Moore Slde 9 Usng ayes Rule to Gamble $. The Wn enelope has a dollar and four beads n t The Lose enelope has three beads and no money Tral queston: someone draws an enelope at random and offers to sell t to you. How much should you pay? Copyrght ndrew W. Moore Slde
Usng ayes Rule to Gamble $. The Wn enelope has a dollar and four beads n t The Lose enelope has three beads and no money Interestng queston: before decdng, you are allowed to see one bead drawn from the enelope. Suppose t s black: How much should you pay? Suppose t s red: How much should you pay? Copyrght ndrew W. Moore Slde Calculaton $. Copyrght ndrew W. Moore Slde
Contnuous robablty Dstrbuton contnuous random arable x can take any alue n an nteral on the real lne X usually corresponds to some real-alued measurements, e.g., today s lowest temperature It s not possble to talk about the probablty of a contnuous random arable takng an exact alue --- x56. Instead we talk about the probablty of the random arable takng a alue wthn a gen nteral x [5, 6] Copyrght ndrew W. Moore Slde 3 DF: probablty densty functon The probablty of X takng alue n a gen range [x, x] s defned to be the area under the DF cure between x and x We use fx to represent the DF of x Note: fx fx can be larger than f x dx X [ x, x] x x f x dx Copyrght ndrew W. Moore Slde 4
What s the ntute meanng of fx? If f xα*a and f xa Then when x s sampled from ths dstrbuton, you are α tmes more lkely to see that x s ery close to x than that x s ery close to x Copyrght ndrew W. Moore Slde 5 Some commonly used dstrbutons nomal dstrbuton: nomaln, p the probablty to see x heads out of n flps x n n L n x + x! x n x p p Multnomal dstrbuton: Multnomaln, [x, x,, x k ] The probablty to see x ones, x twos, etc, out of n dce rolls n! x x x [ x x x k,,..., k ] θ θ Lθ k x! x! Lx! k Copyrght ndrew W. Moore Slde 6 3
Contnuous Dstrbutons f f f Copyrght ndrew W. Moore Slde 7 The Jont Dstrbuton Recpe for makng a ont dstrbuton of M arables: Example: oolean arables,, C Copyrght ndrew W. Moore Slde 8 4
The Jont Dstrbuton Recpe for makng a ont dstrbuton of M arables:. Make a truth table lstng all combnatons of alues of your arables f there are M oolean arables then the table wll hae M rows. Example: oolean arables,, C C Copyrght ndrew W. Moore Slde 9 The Jont Dstrbuton Recpe for makng a ont dstrbuton of M arables:. Make a truth table lstng all combnatons of alues of your arables f there are M oolean arables then the table wll hae M rows.. For each combnaton of alues, say how probable t s. Example: oolean arables,, C C rob.3.5..5.5..5. Copyrght ndrew W. Moore Slde 3 5
The Jont Dstrbuton Recpe for makng a ont dstrbuton of M arables:. Make a truth table lstng all combnatons of alues of your arables f there are M oolean arables then the table wll hae M rows.. For each combnaton of alues, say how probable t s. 3. If you subscrbe to the axoms of probablty, those numbers must sum to. Example: oolean arables,, C.5 C..5.5 rob.3.5..5.5..5...5 C.3. Copyrght ndrew W. Moore Slde 3 Usng the Jont One you hae the JD you can ask for the probablty of any logcal expresson nolng your attrbute E rows matchng E row Copyrght ndrew W. Moore Slde 3 6
Usng the Jont oor Male.4654 E rows matchng E row Copyrght ndrew W. Moore Slde 33 Inference wth the Jont E E E E E rows matchng E and E rows matchng E row row Copyrght ndrew W. Moore Slde 34 7
Inference wth the Jont E E E E E rows matchng E and E rows matchng E row row Male oor.4654 /.764.6 Copyrght ndrew W. Moore Slde 35 So we hae learned that Jont dstrbuton s extremely useful! we can do all knds of cool nference I e got a sore neck: how lkely am I to hae menngts? Many ndustres grow around eyesan Inference: examples nclude medcne, pharma, Engne dagnoss etc. ut, HOW do we get them? We can learn from data Copyrght ndrew W. Moore Slde 36 8
Learnng a ont dstrbuton uld a JD table for your attrbutes n whch the probabltes are unspecfed C rob???????? Fracton of all records n whch and are True but C s False The fll n each row wth records matchng row ˆ row total number of records C rob.3.5..5.5..5. Copyrght ndrew W. Moore Slde 37 9