Sets and Probabilistic Models

Transcription

1 ets ad Probabilistic Models Berli Che Departmet of Computer ciece & Iformatio Egieerig Natioal Taiwa Normal iversity Referece: - D. P. Bertsekas, J. N. Tsitsiklis, Itroductio to Probability, ectios

2 ets (1/2) A set is a collectio of objects which are the elemets of the set If x is a elemet of set, deoted by x Otherwise deoted by x A set that has o elemets is called empty set is deoted by Ø et specificatio Coutably fiite: { 1,2,3,4,5,6} Coutably ifiite: { 0,2, 2,4, 4,... } With a certai property: { k k 2 is iteger} (coutably ifiite) x 0 x 1 (ucoutable) { } { x x satisfies P} such that Probability-Berli Che 2

3 ets (2/2) If every elemet of a set is also a elemet of a set T, the is a subset of T Deoted by T or T If T ad T, the the two sets are equal Deoted by T The uiversal set, deoted by Ω, which cotais all objects of iterest i a particular cotext Ω cosider sets that are subsets of Ω After specifyig the cotext i terms of uiversal set, we oly Probability-Berli Che 3

4 et Operatios (1/3) Complemet The complemet of a set with respect to the uiverse, is the set { x Ω x },amely, the set of all elemets that do ot c belog to, deoted by The complemet of the uiverse Ø io The uio of two sets ad is the set of all elemets that belog to or T, deoted by T T x x or x T Itersectio { } The itersectio of two sets ad is the set of all elemets that belog to both ad T, deoted by I T I T x x ad x T { } T Ω c T Ω Probability-Berli Che 4

5 et Operatios (2/3) The uio or the itersectio of several (or eve ifiite may) sets 1 I 1 Disjoit Two sets are disjoit if their itersectio is empty (e.g., I T Ø) Partitio 1 1 I 2 2 L I L { x x for some } A collectio of sets is said to be a partitio of a set if the sets i the collectio are disjoit ad their uio is { x x for all } Probability-Berli Che 5

6 et Operatios (3/3) Visualizatio of set operatios with Ve diagrams Probability-Berli Che 6

7 Probability-Berli Che 7 The Algebra of ets The followig equatios are the elemetary cosequeces of the set defiitios ad operatios De Morga s law I c c I c c ( ) ( ) ( ) ( ),,,, Ω Ω I I I T T T T c c ( ) ( ) ( ) ( ) ( )., T T T T c Ω I I I I Ø commutative distributive associative distributive

8 Probabilistic Models (1/2) A probabilistic model is a mathematical descriptio of a ucertaity situatio It has to be i accordace with a fudametal framework to be discussed shortly Elemets of a probabilistic model The sample space The set of all possible outcomes of a experimet The probability law Assig to a set A of possible outcomes (also called a evet) a oegative umber P ( A ) (called the probability of A ) that ecodes our kowledge or belief about the collective likelihood of the elemets of A Probability-Berli Che 8

9 Probabilistic Models (2/2) The mai igrediets of a probabilistic model Probability-Berli Che 9

10 ample paces ad Evets (1/2) Each probabilistic model ivolves a uderlyig process, called the experimet That produces exactly oe out of several possible outcomes The set of all possible outcomes is called the sample space of the experimet, deoted by Ω A subset of the sample space (a collectio of possible outcomes) is called a evet Examples of the experimet A sigle toss of a coi (fiite outcomes) Three tosses of two dice (fiite outcomes) A ifiite sequeces of tosses of a coi (ifiite outcomes) Throwig a dart o a square (ifiite outcomes), etc. Probability-Berli Che 10

11 ample paces ad Evets (2/2) Properties of the sample space Elemets of the sample space must be mutually exclusive The sample space must be collectively exhaustive The sample space should be at the right graularity (avoidig irrelevat details) Probability-Berli Che 11

12 Graularity of the ample pace Example 1.1. Cosider two alterative games, both ivolvig te successive coi tosses: Game 1: We receive $1 each time a head comes up Game 2: We receive $1 for every coi toss, up to ad icludig the first time a head comes up. The, we receive $2 for every coi toss, up to the secod time a head comes up. More geerally, the dollar amout per toss is doubled each time a head comes up >> Game 1 cosists of 11 (0,1,..,10) possible outcomes (of moey received) >> Game 2 cosists of?? possible outcomes (of moey received) A fier descriptio is eeded 1, 1, 1, 2, 2, 2, 4, 4, 4, 8 E.g., each outcome correspods to a possible te-log sequece of heads ad tails (will each sequece have a distict outcome?) Probability-Berli Che 12

13 equetial Probabilistic Models May experimets have a iheret sequetial character Tossig a coi three times Observig the value of stock o five successive days Receivig eight successive digits at a commuicatio receiver >> They ca be described by meas of a tree-based sequetial descriptio Probability-Berli Che 13

14 Probability Laws Give the sample space associated with a experimet is settled o, a probability law pecify the likelihood of ay outcome, ad/or of ay set of possible outcomes (a evet) Or alteratively, assig to every evet A, a umber P( A), called the probability of, satisfyig the followig axioms: A Ω A B P P ( Ω ) ( Ø) 0 1 Probability-Berli Che 14

15 Probability Laws for Discrete Models Discrete Probability Law If the sample space cosists of a fiite umber of possible outcomes, the the probability law is specified by the probabilities of the evets that cosist of a sigle elemet. I particular, the probability of ay evet { s 1, s2, K, s } is the sum of the probabilities of its elemets: P ({ s1, s2, K, s} ) P( { s1} ) + P( { s2} ) + L + P( { s} ) P( s ) + P( s ) + L + P( s ) Discrete iform Probability Law 1 If the sample space cosists of possible outcomes which are equally likely (i.e., all sigle-elemet evets have the same probability), the the probability of ay evet A is give by P ( A) umber of 2 elemet of A Probability-Berli Che 15

16 A Example of ample pace ad Probability Law The experimet of rollig a pair of 4-sided dice Probability-Berli Che 16

17 Cotiuous Models Probabilistic models with cotiuous sample spaces It is iappropriate to assig probability to each sigle-elemet evet (?) Istead, it makes sese to assig probability to ay iterval (oedimesioal) or area (two-dimesioal) of the sample space Example: a wheel of fortue P P P K ({ 0.3} ) ({ 0.33} ) ({ 0.333} )??? c d b a P ({ x a x b} )? Probability-Berli Che 17

18 Aother Example for Cotiuous Models Example 1.5: Romeo ad Juliet have a date at a give time, ad each will arrive at the meetig place with a delay betwee 0 ad 1 hour, with all pairs of delays beig equally likely. The first to arrive will wait for 15 miutes ad will leave if the other has ot yet arrived. What is the probability that they will meet? y x : arrivig time for Romeo 1 y : arrivig time for Juliet M M : the evet that Romeo ad Juliet will meet {( x, y) x y 1/ 4, 0 x 1, 0 1 } y 1/4 M ( 3 )( ) /4 1 x Probability-Berli Che 18

19 Properties of Probability Laws (1/2) Probability laws have a umber of properties, which ca be deduced from the axioms. ome of them are summarized below Probability-Berli Che 19

20 Properties of Probability Laws (2/2) Visualizatio ad verificatio usig Ve diagrams Probability-Berli Che 20

21 Model ad Reality (1/2) sig the framework of probability theory to aalyze ucertaity i a wide variety of cotexts ivolves two distict stages I the first stage, we costruct a probabilistic model, by specifyig a probability law o a suitably defied sample space. A ope-eded task! I the secod stage, we work withi a fully specified probabilistic model ad derive the probabilities of certai evets, or deduce some iterestig properties Tightly regulated by rules of ordiary logic ad the axioms of probability. Probability-Berli Che 21

22 Model ad Reality (2/2) Example Bertrad s Paradox Probability-Berli Che 22

23 Recitatio ECTION 1.1 et Problem 3 ECTION 1.2 Probabilistic Models Problems 5, 8 ad 9 Probability-Berli Che 23