Set Theory A, B sets e.g. A = {ζ 1,...,ζ n } A = { c x y d} S space (universe) A,B S Outline Pattern Recognition III Michal Haindl Faculty of Information Technology, KTI Czech Technical University in Prague Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Prague, Czech Republic Evropský sociální fond. Praha & EU: Investujeme do vaší budoucnosti MI-ROZ 2011-2012/Z Set Operations c M. Haindl MI-ROZ - 03 3/17 Outline c M. Haindl MI-ROZ - 03 1/17 January 16, 2012 Outline A = {1,2,3} B = {3,2,1} sum (union) product (intersection) A 1,...,A n are iff 1 Set Theory 2 Probability c M. Haindl MI-ROZ - 03 2/17
Set Operations Set Operations sum (union) product (intersection) A 1,...,A n are iff sum (union) A = {1,2,3} B = {3,4,5} A B = {1,2,3,4,5} product (intersection) A 1,...,A n are iff Set Operations 2 Set Operations complement Ā = S, S =, AĀ =, A+Ā = S difference A B,A\B A B = A B = A AB De Morgan law A B = Ā B A B = Ā B sum (union) product (intersection) A B = {3} A 1,...,A n are iff c M. Haindl MI-ROZ - 03 5/17
Classical Set Operations 2 P(A) = N A N N A no of favourable outcomes complement Ā = S, S =, AĀ =, A+Ā = S if A i i are disjoint P( A i ) = difference A\B = {1,2} A B,A\B A B = A B = A AB De Morgan law 1654 Blaise Pascal, 1812 Pierre-Simon Laplace Théorie analytique des probabilités A B = Ā B A B = Ā B c M. Haindl MI-ROZ - 03 7/17 c M. Haindl MI-ROZ - 03 5/17 Axiomatic Probability 1 P(A) is positive: P(A) 0 2 Probability of certain events equals 1: P(S) = 1 3 If A and B are mutually exclusive, then: P(A+B) = P(A)+P(B) (otherwise P(A+B) = P(A)+P(B) P(AB) n A no of event A appearance Relative Frequency n A P(A) = lim n n definitions: Classical (A priori definition as a ratio of favourable to total number of alternatives.) Axiomatic (measure, A. Kolmogoroff - 1933) Relative frequency (Richard von Mieses - 1936) Probability as a measure of belief (inductive reasoning) A, B events, S space (certain event), impossible event, 0 P(.) 1 A, B mutually exclusive events, A Ā = S c M. Haindl MI-ROZ - 03 8/17 c M. Haindl MI-ROZ - 03 6/17
Conditional Probability Axiomatic given P(B) > 0 P(A B) = P(AB) P(B) Total Probability A i i mutually exclusive events n A i = S 1 P(A) is positive: P(A) 0 2 Probability of certain events equals 1: P(S) = 1 3 If A and B are mutually exclusive, then: P(A+B) = P(A)+P(B) (otherwise P(A+B) = P(A)+P(B) P(AB) P(B) = P(B A i ) = P(B A i ) Relative Frequency Independent Events P(A, B) = P(A)P(B) def. P(A B) = P(A), P(A 1,...,A n ) = i n A no of event A appearance n A P(A) = lim n n c M. Haindl MI-ROZ - 03 9/17 c M. Haindl MI-ROZ - 03 8/17 Conditional Probability Conditional Probability given P(B) > 0 given P(B) > 0 P(A B) = P(AB) P(B) Total Probability A i i mutually exclusive events n A i = S P(A B) = P(AB) P(B) Total Probability A i i mutually exclusive events n A i = S P(B) = P(B A i ) = P(B A i ) P(B) = P(B A i ) = P(B A i ) P(A, B) = P(A)P(B) def. Independent Events P(A, B) = P(A)P(B) def. Independent Events P(A B) = P(A), P(A 1,...,A n ) = i P(A B) = P(A), P(A 1,...,A n ) = i c M. Haindl MI-ROZ - 03 9/17 c M. Haindl MI-ROZ - 03 9/17
Random Variable Bayes Theorem X : ζ (real /complex)number ζ experiment outcome distribution function of the r.v. X F X (x) = P(X x) F( ) = 0 F(+ ) = 1 nondecreasing function of x F(x 1 ) F(x 2 ) for x 1 < x 2 continuous from the right F(x + ) = F(x) A i i mutually exclusive events P(A i B) = n A i = S P(B A i ) n P(B A i) density function F(x) = x f(t)dt nonnegativity f(x) 0 f(x) = df(x) Thomas Bayes: An Essay Toward Solving a Problem in the Doctrine of Chances, 1764 expected value E{X} = xf(x) = xdf(x) c M. Haindl MI-ROZ - 03 11/17 c M. Haindl MI-ROZ - 03 10/17 Random Variable Random Variable X : ζ (real /complex)number ζ experiment outcome distribution function of the r.v. X F X (x) = P(X x) F( ) = 0 F(+ ) = 1 X : ζ (real /complex)number ζ experiment outcome distribution function of the r.v. X F X (x) = P(X x) nondecreasing function of x F(x 1 ) F(x 2 ) for x 1 < x 2 continuous from the right F(x + ) = F(x) density function F(x) = x f(t)dt nonnegativity f(x) 0 expected value f(x) = df(x) E{X} = xf(x) = xdf(x) c M. Haindl MI-ROZ - 03 11/17 F( ) = 0 F(+ ) = 1 nondecreasing function of x F(x 1 ) F(x 2 ) for x 1 < x 2 continuous from the right F(x + ) = F(x) density function c M. Haindl MI-ROZ - 03 f(x) = df(x) 11/17
Dicrete Random Variable Random Variable 2 F X (x) = i P(X = x i) i : x i x (staircase form) expected value E{X} = i x i P(X = x i ) moments µ k = E{X k } central moments µ k = E { (X E{X}) k} F X (x) Mixed Random Variable discontinuous but not of a staircase form µ 2 = σ 2 variance (dispersion) σ = µ 2 standard deviation median x : F( x) 1 2 F( x +0) 1 2 c M. Haindl MI-ROZ - 03 13/17 c M. Haindl MI-ROZ - 03 12/17 Conditional Distribution F X (x z) = P(X x z) = f(x z) = df(x z) total probability ( n i P(z i ) = 1) F X (x) = P(X x, Z = z) P(z) F X (x z i )P(z i ) i Dicrete Random Variable F X (x) = i P(X = x i) i : x i x (staircase form) expected value E{X} = i x i P(X = x i ) conditional expected value E{X z} = xf(x z) F X (x) Mixed Random Variable discontinuous but not of a staircase form c M. Haindl MI-ROZ - 03 14/17 c M. Haindl MI-ROZ - 03 13/17
Normal Distribution Joint Distribution Gaussian N(µ,Σ) f(y) = (2π) n 2 Σ 1 2 exp { 1 } 2 (y µ)t Σ 1 (y µ) Y = {Y 1,...,Y n } E{y} = µ E{(y µ) T (y µ)} = Σ F Y (y) = P(Y 1 y 1,...,Y n y n ) f Y (y) = n F Y (y) y 1,..., y n E{Y} = (E{Y 1 },...,E{Y n }) cov{y i,y j } = E {(Y i E{Y i })(Y j E{Y j })} c M. Haindl MI-ROZ - 03 17/17 c M. Haindl MI-ROZ - 03 15/17 Normal Distribution Marginal Distribution Gaussian N(µ,Σ) f(y) = (2π) n 2 Σ 1 2 exp { 1 } 2 (y µ)t Σ 1 (y µ) F Yi (y i ) = F Y (,...,,y i,,...) E{y} = µ E{(y µ) T (y µ)} = Σ Σ regular, positive definite matrix if Σ = diag{σ 1,1,...,Σ n,n } then y 1,...,y n independent ỹ y ỹ N conditional distribution N any lin. combination of y i N F Yi (y i ) = f k (y 1,...,y k ) = yi... f(y 1,...,y i 1,t,y i+1,...,y n ) dy 1,...,dy i 1 dtdy i+1,...,dy n R n k f(y 1,...,y n )dy k+1...dy n c M. Haindl MI-ROZ - 03 17/17 c M. Haindl MI-ROZ - 03 16/17