Statistika pro informatiku

Size: px

Start display at page:

Download "Statistika pro informatiku"

Maximillian Jayson Simon
5 years ago
Views:

1 Statistika pro informatiku prof. RNDr. Roman Kotecký DrSc., Dr. Rudolf Blažek, PhD Katedra teoretické informatiky FIT České vysoké učení technické v Praze MI-SPI, ZS 2011/12, Přednáška 1 Evropský sociální fond. Praha & EU: Investujeme do vaší budoucnosti Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 1 / 17

2 Organizace Organizace předmětu Statistika pro informatiku Cíle: základní pojmy teorie pravděpodobnosti (úvod a sjednocení znalostí) základy teorie markovských řetězců a jejich aplikací (metody Monte Carlo, náhodné algoritmy) aplikace ve statistice: teorie front, bootstrap metody Cvičení: seminární, u tabule, 8 krátkých písemek za celkem 20b, domácí úkoly 20b. K zápočtu: účast na aspoň šesti písemkách, odevzdané úkoly, celkem aspoň 20b (z možných 40). Zkouška: Písemná (povinná), 60b, min 30b. Body dosažené ze cvičení a ze zkoušky se sčítají. Nebudete-li spokojeni s celkovým hodnocením, můžete si polepšit až o 5b u nepovinné ústní zkoušky. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 2 / 17

3 Organizace Literatura pro první polovinu semestru Používám (často sleduji tyto zdroje dosti věrně i když velmi selektivně) čtyři základní knihy: Grimmett, Stirzaker: Probability and Random Processes, Oxford University Press 2009 Georgii: Stochastics, de Gruyter 2007 Bertsekas, Tsitsiklis: Introduction to Probability, Athena Scientific 2008 Grinstead, Snell: Introduction to Probability, AMS 1997 Poslední citovaná kniha je asi nejvhodnější pro náš kurs. Je volně ke stažení (plus další materiály jako prográmky v Mathematice, řešení ke všem lichým cvičením, atd) na stránce articles/probability_book/book.html Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 3 / 17

4 Main Players: (Ω, P) Space of outcomes Ω Space of outcomes Ω The set of possible outcomes (of an experiment, trial, series of trials,... ): sample space Ω prostor elementárních jevů ω Ω: outcome, elementary event. Examples tossing of a coin, Ω c = {H, T} (head, tail) rolling a die, Ω d = {1, 2, 3, 4, 5, 6} rolling a die n-times, Ω = {1, 2, 3, 4, 5, 6} n rolling a die n-times, interested in how often a given side appears, Ω = {(k 1, k 2, k 3, k 4, k 5, k 6 ) Z 6 + : 6 l=1 k l = n} tossing a coin repeatedly until first head appears, Ω = {ω 1, ω 2, ω 3,... }, ω i : outcome where first i 1 tosses are tails and the ith is head tossing a coin infinitely many times, Ω = {H, T} N orel, panna Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 4 / 17

5 Main Players: (Ω, P) Exclusivity and Exhaustivity Exclusivity and Exhaustivity The first step always consists in a decision which possibilities we observe and distinguish. This determines Ω. Outcomes should always be mutually exclusive and exhaustive ve svém souhrnu vyčerpávající Mutually exclusive outcomes: with a die, try 1 or 2, 1 or 3,...? If the result of a roll is 1, it is not clear to which one it belongs! Exhaustive: each result of an experiment can be interpreted as an elementary outcome. While tossing a coin, we should actually have Ω = {H, T, E}, where E denotes an outcome when the coin rests at an edge: Set of outcomes should be sufficiently detailed to allow to distinguish results that we view as different, while ignoring unimportant details. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 5 / 17

6 Main Players: (Ω, P) Graphical representation Depiction of outcomes for a series of experiments For example for two rolls of a die: a coordinate description and a sketch in the form of a tree where each sequence of results of particular rolls corresponds to a single leaf that is uniquely determined by the path from the root to the leaf (in the illustration, only 3 leafs corresponding to 3 outcomes are explicitly marked) (1,5) second roll first roll (5,4) (6,6) Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 6 / 17

7 Main Players: (Ω, P) Events Events jevy We are interested in likelihoods of events Example jevy rolling a die 3-times, Ω = {1, 2, 3, 4, 5, 6} 3, A: 6 rolled at least once, A = {ω = (ω 1, ω 2, ω 3 ) Ω : 3 l=1 δ ω l,6 1} Events are subsets of Ω, A Ω For Ω finite or at most countable (discrete set of outcomes), it makes sense to consider any A Ω (any A P(Ω)) For Ω uncoutable, one should be careful: we will consider only a subset F P(Ω). First, we will recall all the basic notions in the simplest case of at most countable Ω. The set of events is the set P(Ω) of all subsets of Ω. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 7 / 17

8 Main Players: (Ω, P) Probability Probability pravděpodobnost Definition Probability distribution is a function P pravděp. rozdělení attributing to each event A a real number P(A) such that (P) Nonnegativity: P(A) 0 for every A Ω. (N) Normalization: probability of the collection of all outcomes is one, P(Ω) = 1. The set Ω exhausts all possibilities. (A) Additivity: if A 1, A 2, P(Ω) are pairwise disjoint events (A i A j = for i j), then the probability of their union equals the sum of their probabilities, P( l 1 A l ) = P(A l ) l 1 Think about P(A) = N(A)/N. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 8 / 17

9 Main Players: (Ω, P) Uniform distributions Uniform probability distributions Stejnoměrná pravd. rozd. Remark The function p : Ω [0, 1], defined by p(ω) = P({ω}) for each ω Ω, determines uniquely the function P. Indeed, P(A) = ω A p(ω) for any A P(Ω). Notice that, necessarily, ω Ω p(ω) = P(Ω) = 1. For Ω finite it makes sence to assume that the distribution is uniform (like fair coin toss, rolling of an unbiased die, etc.). Then, p(ω) = 1 Ω and P(A) = A # of favourable outcomes = Ω # of possible outcomes for any A Ω. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 9 / 17

10 Main Players: (Ω, P) Uniform distributions Example Rolling a die two times: Ω = {1, 2, 3, 4, 5, 6} 2 = {(ω 1, ω 2 ) : ω 1, ω 2 {1, 2, 3, 4, 5, 6}}. For any (ω 1, ω 2 ) Ω, we have P({(ω 1, ω 2 )}) = p(ω 1, ω 2 ) = Consider the events A n = {(ω 1, ω 2 ) Ω : ω 1 + ω 2 = n}, n = 2,..., 12 and B m = {(ω 1, ω 2 ) Ω : max(ω 1, ω 2 ) = m}, m = 1, 2,..., 6. A8 6 B Check that we have: P(A n )= min(n 1,13 n), n = 2,..., 12, and P(B 36 m )= 2m 1, m = 1,..., Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 10 / 17

11 Properties of probability distributions Probability rules Implications of the additivity (A) The pair (Ω, P) is called a probability space pravděp. prostor Theorem Let (Ω, P) be a probability space (with at most countable Ω). Then, for arbitrary events A, B, A 1, A 2, Ω, we have (i) P( ) = 0. (ii) Monotonicity. If A B then P(A) P(B). (iii) Finite additivity. P(A B) + P(A B) = P(A) + P(B) (and thus also, P(A) + P(A c ) = 1). (iv) Subadditivity. P( n 1 A n ) n 1 P(A n). (v) Inclusion-exclusion principle. P( n k=1 A k) = n k=1 P(A k) i<j P(A i A j ) + + ( 1) n+1 P(A 1 A n ). Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 11 / 17

12 Properties of probability distributions Probability rules Proof. (i) P( ) = 0: Ω and are disjoint sets (Ω = ). Hence, P(Ω) = P(Ω ) = P(Ω) + P( ) which is possible only if P( ) = 0. (ii) B A, then P(B) = P(A) + P(B \ A) P(A). (iii) P(A B) + P(A B) = P(A \ B) + P(B \ A) + 2P(A B) = = P(A) + P(B). A A B B (iv) n 1 A n = A 1 (A 2 \ A 1 ) (A 3 \ (A 1 A 2 )).... Hence, P( n 1 A n ) = P(A 1 ) + P(A 2 \ A 1 ) + P(A 3 \ (A 1 A 2 )) + = = n 1 P(A n \ m<n A m ) n 1 P(A n). Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 12 / 17

13 Properties of probability distributions Probability rules Proof. (Continuation) (v) By induction in n. Hint: C A C B C A B C A A B B A very easy proof will come later. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 13 / 17

14 Conditional probability and independence Conditional probability Conditional probability Podmíněná pravděp. Conditional probability allows to make claims about an experiment that are based on an incomplete information. We know that an event B occurred. How the probability of the event A changed by this additional information? Conditional probability of an event A under the condition B: P(A B). Let us repeat an experiment N times. If we are interested only in the outcomes when B occurs (and disregard the others), the proportion of cases when A also occurs (among those when B occurs) is N(A B) = N(B) This motivates the definition: Definition N(A B)/N N(B)/N. If P(B) > 0 then the conditional probability that A occurs under the condition B is defined to be P(A B) = P(A B). P(B) Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 14 / 17

15 Conditional probability and independence Conditional probability Example A dice rolled twice. P(sum > 6 first = 3)? Clearly 1/2: The value of the second die must be 4, 5, or 6. Formally: Ω = {1, 2, 3, 4, 5, 6} 2, any P(A) = A /36 for any A Ω. Take, B = {(3, ω 2 ) : 1 ω 2 6}, A = {(ω 1, ω 2 ) : ω 1 + ω 2 > 6}. Then, P(A B) = P(A B) P(B) = A B B = 3 6 = A B Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 15 / 17

16 Conditional probability and independence Conditional probability Example A family with two children. P(both boys at least one is boy)? Ω = {GG, GB, BG, BB}. Correct: P(BB GB BG BB) = P(BB (GB BG BB)) P(GB BG BB) = P(BB) P(GB BG BB) = 1/4 3/4 = 1 3. Wrong: P(BB younger is a boy) = P(BB GB BB) = P(BB (GB BB)) P(GB BB) = 1 2. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 16 / 17

17 Conditional probability and independence Basic properties Basic properties of conditional probability Conditional probability is a probability. Let an event B such that P(B) > 0 be fixed. Then, conditional probability P(A B) satisfies axioms of probability: P(Ω B) = P(Ω B) P(B) = P(B) P(B) = 1. P(A 1 A 2 B) = P((A1 A 2) B) = P(A 1 B) P(B) + P(A2 B) P(B) P(B) P((A1 B) (A2 B) P(B) = P(A 1 B) + P(A 2 B). = P(A1 B)+P(A2 B) P(B) = Hence, other properties of the probability law follow. For example: P(A c B) = 1 P(A B). P(A 1 A 2 B) = P(A 1 B) + P(A 2 B) P(A 1 A 2 B). Moreover, the probability P(A B) lives on B: for A B =, we have P(A B) = 0. Roman Kotecký, Rudolf Blažek (FIT ČVUT) Statistika pro informatiku MI-SPI, ZS 2011/12, Přednáška 1 17 / 17

Statistika pro informatiku

Statistika pro informatiku prof. RNDr. Roman Kotecký DrSc., Dr. Rudolf Blažek, PhD Katedra teoretické informatiky FIT České vysoké učení technické v Praze MI-SPI, ZS 2011/12, Přednáška 2 Evropský sociální