Introduction to probability theory
|
|
- Rachel Blair
- 5 years ago
- Views:
Transcription
1 Introduction to probability theory Fátima Sánchez Cabo Institute for Genomics and Bioinformatics, TUGraz 07/03/ p. 1/35
2 Outline Random and conditional probability (7 March) From Probability Theory to Statistics (14 March) Hypothesis testing (14 March) Stochastic processes and Markov chains (21 March) 07/03/ p. 2/35
3 Introduction Sample space and events Probability Space Algebra of Sets Probability measure 07/03/ p. 3/35
4 Introduction Introduction Sample space and events Probability Space Algebra of Sets Probability measure Life is full of unpredictable events Probabilistic models are used to make inference from this type of : their outcome cannot be completely determined, but we can get some hints about the most likely situation that will occur Randomness does not mean chaos: for example the most likely value in a normally distributed random variable is the mode; very small or very large values are very unlikely to occur 0.4 Normal(0,1) /03/ p. 4/35
5 Random : Definition Introduction Sample space and events Probability Space Algebra of Sets Probability measure All outcomes of the experiment are known in advance But, it is a priori unknown which will be the outcome of each performance of the experiment: Systematic and random errors Complex processes, result of many combined processes The experiment can be repeated under identical conditions 07/03/ p. 5/35
6 Examples Introduction Sample space and events Probability Space Algebra of Sets Probability measure Throwing a die: Possible outcomes: {1,2,3,4,5,6} It is unknown what we will get next if we throw the die We can throw the die n times and all are independent trials Tossing a coin: Possible outcomes: {head, tail} It is unknown what we will get next if we toss the coin We can toss the coin n independent times The length life of a light bulb produced by a manufacturer: Possible outcomes: Any number between 0 and The life-time of a bulb is not known before hand We assume that the life-time of n bulbs can be measured under the same conditions 07/03/ p. 6/35
7 Sample space, events, σ algebra Introduction Sample space and events Probability Space Algebra of Sets Probability measure Sample space: Collection of possible elementary outcomes from a random experiment. 1. Throwing a die: Ω={1,2,3,4,5,6} 2. Tossing a coin: Ω={head, tail} 3. Life-time of a bulb: Ω = [0, ) Event: A set of outcomes of the experiment. 1. Throwing a die: "To obtain a 6" ({6}) or "Not to obtain a 6" ({1,2,3,4,5}) 2. Tossing a coin: "To obtain a head" 3. Life-time of a bulb: "The life-time of the bulb is greater than 3 years" A σ-field (or σ-algebra) is a non-empty collection of subsets of Ω that satisfy: ǫ F if A ǫ F then A c ǫ F,and if A i ǫ F is a countable sequence of sets then i A i ǫ F 07/03/ p. 7/35
8 Probability space Introduction Sample space and events Probability Space Algebra of Sets Probability measure The pair (Ω, F) is called under the previous conditions a sample space A measure is a nonnegative countably additive set function, i.e. a function µ : F R such as: 1. µ(a) µ( ) for all AǫF, and 2. if A i ǫ F and i A i = then µ( i A i) = i µ(a i) a If µ(ω) = 1 the measure is called a probability measure a countable: the set of the natural number, uncountable: the set of the real numbers 07/03/ p. 8/35
9 A little on Algebra of Sets Introduction Sample space and events Probability Space Algebra of Sets Probability measure Sets algebra is equivalent to elementary algebra (arithmetic): Intersection Multiplication; Union Addition Differently from arithmetic operations (only distributive for the product) union and intersection have both the distributive property Uniqueness of complements: Given A, B such as A B = Ω and A B =, then B = A c B\A = B A c Morgan s Laws: (A B) c = A c B c (A B) c = A c B c A cc = A c = Ω; Ω c = 07/03/ p. 9/35
10 Probability measure Introduction Sample space and events Probability Space Algebra of Sets Probability measure Some properties of the probability measure: 1. P(A B) = P(A) + P(B) P(A B) 2. P(A c ) = 1 P(A) 3. If B A P(B) = P(A) + P(B\A) P(A) 4. In general, if A 1,...,A n ǫf P( A i ) = = P(A i ) i<j P(A i A j )... + ( 1) n+1 P(A 1... A n ) Exercise: Proof /03/ p. 10/35
11 Example Introduction Sample space and events Probability Space Algebra of Sets Probability measure The probability of a chicken from Steiermark to be infected with the Bird flu is 0.2 a. At the same time a chicken may have another lethal disease called Y that appears with probability 0.5. Calculate the probability of chicken dying due to any disease: 1. If it is not possible that both diseases appear simultaneously; 2. If the probability of having both diseases is 0.6. a Unreal number 07/03/ p. 11/35
12 Dependent events Bayes Theorem 07/03/ p. 12/35
13 Dependent events Dependent events Bayes Theorem P(B)=2/3 P(W)=1/3 P(B)=1/3 P(W)=2/3 Which is the probability of getting a white ball? Is the same in both? 07/03/ p. 13/35
14 Dependent events Dependent events Bayes Theorem Definition: Two events A, B are called independent if P(A B) = P(A) P(B). Otherwise, P(A B) = P(A) P(B A) = P(B) P(A B) Multiplication rule: Given a probability space (Ω, F, P) such as A 1,...,A n ǫf and P( n 1 i=1 A i) > 0, then P( n i=1a i ) = P(A 1 )P(A 2 A 1 )...P(A n n 1 i=1 A i) 07/03/ p. 14/35
15 Dependent events Bayes Theorem Definition: (Ω, F, P) is a probability space such as the events A and B ǫ F. If P(B) > 0 it can be defined the probability of A conditional on B as: P(A B) = P(A B) P(B) Total probability theorem: Let Ω be the sample space of a random experiment and {A i, i = 1, 2,...} ǫ F such as A i Aj = i j and i A i = Ω. Then, for all B ǫ F P(B) = i P(B A i )P(A i ) 07/03/ p. 15/35
16 Example Dependent events Bayes Theorem The probability of a chicken from Steiermark to be infected with the Bird flu is 0.2 a. At the same time a chicken may have another lethal disease called Y that appears with probability 0.5. The death rate if a chicken has the Bird Flu is 0.8. The probability of death for a chicken with disease Y is 0.1. Both diseases cannot appear simultaneously. Additionally, a chicken might die due to natural causes with a probability 0.1. Calculate the probability of dying for a chicken in Steiermark. a Unreal number 07/03/ p. 16/35
17 Bayes Theorem Dependent events Bayes Theorem Let Ω be the sample space of a random experiment and {A i, i = 1, 2,...} ǫ F such as A i Aj = i j and i A i = Ω. Let B ǫ F with P(B)>0. Then P(A i B) = P(B A i) P(B) Equivalently, P(A i B) = P(B A i)p(a i ) j P(B A j)p(a j ) 07/03/ p. 17/35
18 Example Dependent events 1 Door A Door B Door C Bayes Theorem /03/ p. 18/35
19 Example 1 Door A Door B Door C Dependent events 2 3 Bayes Theorem Naive approach: Regardless to the initial situation now there are only two doors from which I could choose. Hence, Pr(car is behind A)=Pr(car is not behind A)= 1 2 It is not an advantage to switch the door. By Bayes Theorem: 07/03/ p. 19/35
20 Example (cont.) We define the event A as "car is behind door A" (the same for other doors) Dependent events Bayes Theorem Prior : Pr(A) = Pr(B) = Pr(C) = 1, i = 1, 2, 3 3 Pr(open C A) = 1 2 Pr(open C B) = 1 Pr(open C C) = 0 By the Total Probability Theorem: Pr(op C) = = P(op C A) P(A) + P(op C B) P(B) + P(op C C) P(C) = = /03/ p. 20/35
21 Example (cont.) By Bayes Theorem: Dependent events Bayes Theorem Pr(A open C) = Pr(B open C) = Pr(open C A) Pr(A) Pr(open C) Pr(open C B) Pr(B) Pr(open C) = = = 1 3 = 2 3 Conclusion: The probability of winning the car is bigger if you change the door!!! 07/03/ p. 21/35
22 Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem 07/03/ p. 22/35
23 Random variable Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem The probability measure P is a set function and hence difficult to work with it. We define a random variable on a probability space as a real valued function X defined in Ω such as: X : Ω R and X 1 (B) = {ω : X(ω)ǫB}ǫF, for all Borel set B R In other words, if the inverse of any Borel subset of R (semi-open intervals) belongs to the σ-algebra Example: The random experiment "To toss a coin" can be represented with the random variable { 1 if Head( X 1 (1) = HeadǫF) X = 0 if Tail( X 1 (0) = TailǫF) 07/03/ p. 23/35
24 Example 3.1. Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem A roulette wheel has 38 slots -18 red, 18 black and 2 green. A gambler bets 1 $ to red each time. We define the random variable X i as the monetary gain of the gambler at game i. (From Durret (1996)) Ω = {Red, Black, Green} Events: F = {{Red}, {Black}, {Green}, {Red, Black}, {Red, Green}, {Black, Green} {Red, Black, Green}, } P(Red) = = 9 19, P(no Red) = = Random variable: X = { +1 Red 1 Red 07/03/ p. 24/35
25 Discrete random variables Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem A discrete random variable can take a countable number of predetermined values Example: "To toss a coin", "To throw a die", "Number of cars crossing a line during a certain time interval" Mass function: For discrete random variables, the mass function determines the probability of each element of the sample space. Example: for the random experiment "To throw a die" p(1) =... = p(6) = Mass probability function of 100 random numbers from a binomial distribution (n=100, p=0.5) 07/03/ p. 25/35
26 Discrete random variables Typical mass functions of discrete random variables: Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem 1. Bernoulli: X = { 1 p 0 (1 p) 2. Binomial: Number of successes within n Bernoulli trials: P(X = k) = C(n, k)p k (1 p) n k 3. Poisson, P(X = k) = e λ λ k k! 07/03/ p. 26/35
27 Continuous random variables Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem For continuous random variables (that can take any real value) the way the probability is distributed within the sample space is more difficult to define. The probability density function (pdf) has the following properties: 1. P[a X b] = b a f(x)dx 2. f(x) 0, xǫr 3. f(x)dx = Histogram of frequencies for a normal random variable 07/03/ p. 27/35
28 Continuous random variables Most usual pdf Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem 1. Normal distribution f(x) = 1 2πσ 2 exp{ (x µ)2 2σ 2 } Random errors are normally distributed 2. Uniform f(x) = 1 b a I [a,b] 3. Gamma f(x) = βα Γ(α) xα 1 e βx 07/03/ p. 28/35
29 Distribution function F(x) = P(X x) = { x f(x)dx cont r.v. a P(X = k) discrete r.v. x a Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem where a is the smallest value that the r.v. can take. Empirical CDF Empirical CDF F(x) 0.5 F(x) x x Probability distribution of 100 random numbers from a binomial distribution (n=10, p=0.5) and cumulative probability distribution of 1000 random numbers from a normal distribution (µ=0, σ 2 =1) 07/03/ p. 29/35
30 Example 3.2. Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem A fair coin (p=0.5) is tossed twice. For each one of the possible outcomes of the experiment we define the random variable that indicates the number of heads, i.e.: X = 2 HH 1 HT,TH 0 TT Calculate and plot the probability distribution for this random variable. 07/03/ p. 30/35
31 Distribution function and probability Properties of the distribution function: Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem 1. lim x = 0; lim x + = 1 2. If x < y F(x) F(y) 3. F is continuous from the right, i.e. F(x + h) F(x) as h 0 Distribution function and probability: P(X > x) = 1 F(x) P(x < X y) = F(y) F(x) 07/03/ p. 31/35
32 Expectation and variance Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem Expectation: Continuous random variable: E[X] = xf(x)dx Discrete random variable: E[X] = x i P(X = x i ) Variance:V [X] = E[(X E(X)) 2 ] = E[X 2 ] (E[X]) 2 Probability Statistic Base Population Sample Central tendency Expectation Average Dispersion Variance Sample variance 07/03/ p. 32/35
33 Central Limit Theorem Random variable Continuous random variables Distribution function Expectation and variance Central Limit Theorem Given X 1,...,X n a set of random variables independents and with common distribution f, such as E[X i ] = µ and V [X i ] = σ 2 for all i. Then it is true that: If n is big enough. X E[ X] V [ X] N(0, 1) 07/03/ p. 33/35
34 07/03/ p. 34/35
35 [1] Durbin, R., Eddy, S., Krogh, A. and Mitchison, G. (1996) Biological sequence analysis, Cambridge University Press. [2] Durret, R. (1996) Probability: Theory and examples, Duxbury Press, Second edition. [3] Rohatgi, V.K. and Ehsanes Saleh, A.K.Md. (1988) An introduction to probability and statistics, Wiley, Second Edition. [4] Tuckwell, H.C. (1988) Elementary applications of probability theory, Chapman and Hall. [5] tsirel/courses/introprob/syl1a.html [Engineering statistics] 07/03/ p. 35/35
Introduction to Statistical Inference
Introduction to Statistical Inference Dr. Fatima Sanchez-Cabo f.sanchezcabo@tugraz.at http://www.genome.tugraz.at Institute for Genomics and Bioinformatics, Graz University of Technology, Austria Introduction
More informationStatistical Inference
Statistical Inference Lecture 1: Probability Theory MING GAO DASE @ ECNU (for course related communications) mgao@dase.ecnu.edu.cn Sep. 11, 2018 Outline Introduction Set Theory Basics of Probability Theory
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationLECTURE 1. 1 Introduction. 1.1 Sample spaces and events
LECTURE 1 1 Introduction The first part of our adventure is a highly selective review of probability theory, focusing especially on things that are most useful in statistics. 1.1 Sample spaces and events
More informationLecture 1: Basics of Probability
Lecture 1: Basics of Probability (Luise-Vitetta, Chapter 8) Why probability in data science? Data acquisition is noisy Sampling/quantization external factors: If you record your voice saying machine learning
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationPreliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More informationChapter 2 Random Variables
Stochastic Processes Chapter 2 Random Variables Prof. Jernan Juang Dept. of Engineering Science National Cheng Kung University Prof. Chun-Hung Liu Dept. of Electrical and Computer Eng. National Chiao Tung
More informationRecap of Basic Probability Theory
02407 Stochastic Processes? Recap of Basic Probability Theory Uffe Høgsbro Thygesen Informatics and Mathematical Modelling Technical University of Denmark 2800 Kgs. Lyngby Denmark Email: uht@imm.dtu.dk
More informationProbability. Lecture Notes. Adolfo J. Rumbos
Probability Lecture Notes Adolfo J. Rumbos October 20, 204 2 Contents Introduction 5. An example from statistical inference................ 5 2 Probability Spaces 9 2. Sample Spaces and σ fields.....................
More informationRecap of Basic Probability Theory
02407 Stochastic Processes Recap of Basic Probability Theory Uffe Høgsbro Thygesen Informatics and Mathematical Modelling Technical University of Denmark 2800 Kgs. Lyngby Denmark Email: uht@imm.dtu.dk
More informationBasics on Probability. Jingrui He 09/11/2007
Basics on Probability Jingrui He 09/11/2007 Coin Flips You flip a coin Head with probability 0.5 You flip 100 coins How many heads would you expect Coin Flips cont. You flip a coin Head with probability
More informationELEG 3143 Probability & Stochastic Process Ch. 1 Probability
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 1 Probability Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Applications Elementary Set Theory Random
More informationDeep Learning for Computer Vision
Deep Learning for Computer Vision Lecture 3: Probability, Bayes Theorem, and Bayes Classification Peter Belhumeur Computer Science Columbia University Probability Should you play this game? Game: A fair
More informationIEOR 3106: Introduction to Operations Research: Stochastic Models. Professor Whitt. SOLUTIONS to Homework Assignment 1
IEOR 3106: Introduction to Operations Research: Stochastic Models Professor Whitt SOLUTIONS to Homework Assignment 1 Probability Review: Read Chapters 1 and 2 in the textbook, Introduction to Probability
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More informationSample Spaces, Random Variables
Sample Spaces, Random Variables Moulinath Banerjee University of Michigan August 3, 22 Probabilities In talking about probabilities, the fundamental object is Ω, the sample space. (elements) in Ω are denoted
More informationPROBABILITY THEORY. Prof. S. J. Soni. Assistant Professor Computer Engg. Department SPCE, Visnagar
PROBABILITY THEORY By Prof. S. J. Soni Assistant Professor Computer Engg. Department SPCE, Visnagar Introduction Signals whose values at any instant t are determined by their analytical or graphical description
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More information1.1 Review of Probability Theory
1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More informationWeek 2. Review of Probability, Random Variables and Univariate Distributions
Week 2 Review of Probability, Random Variables and Univariate Distributions Probability Probability Probability Motivation What use is Probability Theory? Probability models Basis for statistical inference
More informationLecture 3 Probability Basics
Lecture 3 Probability Basics Thais Paiva STA 111 - Summer 2013 Term II July 3, 2013 Lecture Plan 1 Definitions of probability 2 Rules of probability 3 Conditional probability What is Probability? Probability
More informationELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Random Variable Discrete Random
More informationMath 105 Course Outline
Math 105 Course Outline Week 9 Overview This week we give a very brief introduction to random variables and probability theory. Most observable phenomena have at least some element of randomness associated
More informationChapter 1. Sets and probability. 1.3 Probability space
Random processes - Chapter 1. Sets and probability 1 Random processes Chapter 1. Sets and probability 1.3 Probability space 1.3 Probability space Random processes - Chapter 1. Sets and probability 2 Probability
More informationNotes 1 : Measure-theoretic foundations I
Notes 1 : Measure-theoretic foundations I Math 733-734: Theory of Probability Lecturer: Sebastien Roch References: [Wil91, Section 1.0-1.8, 2.1-2.3, 3.1-3.11], [Fel68, Sections 7.2, 8.1, 9.6], [Dur10,
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationProbability and Statistics Concepts
University of Central Florida Computer Science Division COT 5611 - Operating Systems. Spring 014 - dcm Probability and Statistics Concepts Random Variable: a rule that assigns a numerical value to each
More informationEE514A Information Theory I Fall 2013
EE514A Information Theory I Fall 2013 K. Mohan, Prof. J. Bilmes University of Washington, Seattle Department of Electrical Engineering Fall Quarter, 2013 http://j.ee.washington.edu/~bilmes/classes/ee514a_fall_2013/
More informationStatistics and Econometrics I
Statistics and Econometrics I Random Variables Shiu-Sheng Chen Department of Economics National Taiwan University October 5, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 5, 2016
More information18.175: Lecture 2 Extension theorems, random variables, distributions
18.175: Lecture 2 Extension theorems, random variables, distributions Scott Sheffield MIT Outline Extension theorems Characterizing measures on R d Random variables Outline Extension theorems Characterizing
More informationProbability theory for Networks (Part 1) CS 249B: Science of Networks Week 02: Monday, 02/04/08 Daniel Bilar Wellesley College Spring 2008
Probability theory for Networks (Part 1) CS 249B: Science of Networks Week 02: Monday, 02/04/08 Daniel Bilar Wellesley College Spring 2008 1 Review We saw some basic metrics that helped us characterize
More informationName: Firas Rassoul-Agha
Midterm 1 - Math 5010 - Spring 016 Name: Firas Rassoul-Agha Solve the following 4 problems. You have to clearly explain your solution. The answer carries no points. Only the work does. CALCULATORS ARE
More informationBandits, Experts, and Games
Bandits, Experts, and Games CMSC 858G Fall 2016 University of Maryland Intro to Probability* Alex Slivkins Microsoft Research NYC * Many of the slides adopted from Ron Jin and Mohammad Hajiaghayi Outline
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationLecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019
Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial
More informationRandom variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
More informationDiscrete Random Variables
CPSC 53 Systems Modeling and Simulation Discrete Random Variables Dr. Anirban Mahanti Department of Computer Science University of Calgary mahanti@cpsc.ucalgary.ca Random Variables A random variable is
More informationChapter 3. Chapter 3 sections
sections 3.1 Random Variables and Discrete Distributions 3.2 Continuous Distributions 3.4 Bivariate Distributions 3.5 Marginal Distributions 3.6 Conditional Distributions 3.7 Multivariate Distributions
More information5. Conditional Distributions
1 of 12 7/16/2009 5:36 AM Virtual Laboratories > 3. Distributions > 1 2 3 4 5 6 7 8 5. Conditional Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an
More informationIntroduction to Probability Theory and Statistics
Introduction to Probability Theory and Statistics Copyright c 1996,1998, 2002 Javier R. Movellan. This is an open source document. Permission is granted to copy, distribute and/or modify this document
More informationRS Chapter 1 Random Variables 6/5/2017. Chapter 1. Probability Theory: Introduction
Chapter 1 Probability Theory: Introduction Basic Probability General In a probability space (Ω, Σ, P), the set Ω is the set of all possible outcomes of a probability experiment. Mathematically, Ω is just
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationLecture 9: Conditional Probability and Independence
EE5110: Probability Foundations July-November 2015 Lecture 9: Conditional Probability and Independence Lecturer: Dr. Krishna Jagannathan Scribe: Vishakh Hegde 9.1 Conditional Probability Definition 9.1
More informationLecture Notes 1 Basic Probability. Elements of Probability. Conditional probability. Sequential Calculation of Probability
Lecture Notes 1 Basic Probability Set Theory Elements of Probability Conditional probability Sequential Calculation of Probability Total Probability and Bayes Rule Independence Counting EE 178/278A: Basic
More informationProbability, Random Processes and Inference
INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx
More informationV. RANDOM VARIABLES, PROBABILITY DISTRIBUTIONS, EXPECTED VALUE
V. RANDOM VARIABLES, PROBABILITY DISTRIBUTIONS, EXPECTED VALUE A game of chance featured at an amusement park is played as follows: You pay $ to play. A penny a nickel are flipped. You win $ if either
More informationElementary Discrete Probability
Elementary Discrete Probability MATH 472 Financial Mathematics J Robert Buchanan 2018 Objectives In this lesson we will learn: the terminology of elementary probability, elementary rules of probability,
More informationMATH Notebook 5 Fall 2018/2019
MATH442601 2 Notebook 5 Fall 2018/2019 prepared by Professor Jenny Baglivo c Copyright 2004-2019 by Jenny A. Baglivo. All Rights Reserved. 5 MATH442601 2 Notebook 5 3 5.1 Sequences of IID Random Variables.............................
More informationProbability Space: Formalism Simplest physical model of a uniform probability space:
Lecture 16: Continuing Probability Probability Space: Formalism Simplest physical model of a uniform probability space: Probability Space: Formalism Simplest physical model of a non-uniform probability
More informationIntro to Stats Lecture 11
Outliers and influential points Intro to Stats Lecture 11 Collect data this week! Midterm is coming! Terms X outliers: observations outlying the overall pattern of the X- variable Y outliers: observations
More informationBasic Probability and Information Theory: quick revision
Basic Probability and Information Theory: quick revision ML for NLP Lecturer: S Luz http://www.scss.tcd.ie/~luzs/t/cs4ll4/ February 17, 2015 In these notes we review the basics of probability theory and
More informationRandom Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping
Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 20, 2012 Introduction Introduction The world of the model-builder
More informationChapter 1: Introduction to Probability Theory
ECE5: Stochastic Signals and Systems Fall 8 Lecture - September 6, 8 Prof. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter : Introduction to Probability Theory Axioms of Probability
More informationEcon 325: Introduction to Empirical Economics
Econ 325: Introduction to Empirical Economics Lecture 2 Probability Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 3-1 3.1 Definition Random Experiment a process leading to an uncertain
More informationCS 361: Probability & Statistics
February 12, 2018 CS 361: Probability & Statistics Random Variables Monty hall problem Recall the setup, there are 3 doors, behind two of them are indistinguishable goats, behind one is a car. You pick
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14
CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten
More informationProbability Theory and Applications
Probability Theory and Applications Videos of the topics covered in this manual are available at the following links: Lesson 4 Probability I http://faculty.citadel.edu/silver/ba205/online course/lesson
More informationProbability Basics Review
CS70: Jean Walrand: Lecture 16 Events, Conditional Probability, Independence, Bayes Rule Probability Basics Review Setup: Set notation review A B A [ B A \ B 1 Probability Basics Review 2 Events 3 Conditional
More informationStatistika pro informatiku
Statistika pro informatiku prof. RNDr. Roman Kotecký DrSc., Dr. Rudolf Blažek, PhD Katedra teoretické informatiky FIT České vysoké učení technické v Praze MI-SPI, ZS 2011/12, Přednáška 2 Evropský sociální
More information1 Probability and Random Variables
1 Probability and Random Variables The models that you have seen thus far are deterministic models. For any time t, there is a unique solution X(t). On the other hand, stochastic models will result in
More informationPROBABILITY CHAPTER LEARNING OBJECTIVES UNIT OVERVIEW
CHAPTER 16 PROBABILITY LEARNING OBJECTIVES Concept of probability is used in accounting and finance to understand the likelihood of occurrence or non-occurrence of a variable. It helps in developing financial
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationIntroduction to Information Entropy Adapted from Papoulis (1991)
Introduction to Information Entropy Adapted from Papoulis (1991) Federico Lombardo Papoulis, A., Probability, Random Variables and Stochastic Processes, 3rd edition, McGraw ill, 1991. 1 1. INTRODUCTION
More informationCSC Discrete Math I, Spring Discrete Probability
CSC 125 - Discrete Math I, Spring 2017 Discrete Probability Probability of an Event Pierre-Simon Laplace s classical theory of probability: Definition of terms: An experiment is a procedure that yields
More informationMath 151. Rumbos Fall Solutions to Review Problems for Exam 2. Pr(X = 1) = ) = Pr(X = 2) = Pr(X = 3) = p X. (k) =
Math 5. Rumbos Fall 07 Solutions to Review Problems for Exam. A bowl contains 5 chips of the same size and shape. Two chips are red and the other three are blue. Draw three chips from the bowl at random,
More informationFundamental Tools - Probability Theory II
Fundamental Tools - Probability Theory II MSc Financial Mathematics The University of Warwick September 29, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory II 1 / 22 Measurable random
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 1 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationProbabilistic models
Probabilistic models Kolmogorov (Andrei Nikolaevich, 1903 1987) put forward an axiomatic system for probability theory. Foundations of the Calculus of Probabilities, published in 1933, immediately became
More information1 Probability theory. 2 Random variables and probability theory.
Probability theory Here we summarize some of the probability theory we need. If this is totally unfamiliar to you, you should look at one of the sources given in the readings. In essence, for the major
More informationOrigins of Probability Theory
1 16.584: INTRODUCTION Theory and Tools of Probability required to analyze and design systems subject to uncertain outcomes/unpredictability/randomness. Such systems more generally referred to as Experiments.
More informationChapter 2: Random Variables
ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:
More informationStatistics 100A Homework 5 Solutions
Chapter 5 Statistics 1A Homework 5 Solutions Ryan Rosario 1. Let X be a random variable with probability density function a What is the value of c? fx { c1 x 1 < x < 1 otherwise We know that for fx to
More informationRelationship between probability set function and random variable - 2 -
2.0 Random Variables A rat is selected at random from a cage and its sex is determined. The set of possible outcomes is female and male. Thus outcome space is S = {female, male} = {F, M}. If we let X be
More informationLecture Lecture 5
Lecture 4 --- Lecture 5 A. Basic Concepts (4.1-4.2) 1. Experiment: A process of observing a phenomenon that has variation in its outcome. Examples: (E1). Rolling a die, (E2). Drawing a card form a shuffled
More informationThe main results about probability measures are the following two facts:
Chapter 2 Probability measures The main results about probability measures are the following two facts: Theorem 2.1 (extension). If P is a (continuous) probability measure on a field F 0 then it has a
More informationMathematical Statistics 1 Math A 6330
Mathematical Statistics 1 Math A 6330 Chapter 3 Common Families of Distributions Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 28, 2015 Outline 1 Subjects of Lecture 04
More informationCourse: ESO-209 Home Work: 1 Instructor: Debasis Kundu
Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear
More informationMTH 202 : Probability and Statistics
MTH 202 : Probability and Statistics Lecture 5-8 : 15, 20, 21, 23 January, 2013 Random Variables and their Probability Distributions 3.1 : Random Variables Often while we need to deal with probability
More informationProbability Review I
Probability Review I Harvard Math Camp - Econometrics Ashesh Rambachan Summer 2018 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability
More informationChapter 4 : Discrete Random Variables
STAT/MATH 394 A - PROBABILITY I UW Autumn Quarter 2015 Néhémy Lim Chapter 4 : Discrete Random Variables 1 Random variables Objectives of this section. To learn the formal definition of a random variable.
More informationFundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner
Fundamentals CS 281A: Statistical Learning Theory Yangqing Jia Based on tutorial slides by Lester Mackey and Ariel Kleiner August, 2011 Outline 1 Probability 2 Statistics 3 Linear Algebra 4 Optimization
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationPROBABILITY VITTORIA SILVESTRI
PROBABILITY VITTORIA SILVESTRI Contents Preface. Introduction 2 2. Combinatorial analysis 5 3. Stirling s formula 8 4. Properties of Probability measures Preface These lecture notes are for the course
More informationMarkov Branching Process
MODULE 8: BRANCHING PROCESSES 11 Lecture 3 Markov Branching Process Let Z(t) be number of particles at time t. Let δ 1k + a k h + o(h), k = 0, 1,... represent the probability that a single particle will
More informationSingle Maths B: Introduction to Probability
Single Maths B: Introduction to Probability Overview Lecturer Email Office Homework Webpage Dr Jonathan Cumming j.a.cumming@durham.ac.uk CM233 None! http://maths.dur.ac.uk/stats/people/jac/singleb/ 1 Introduction
More informationProbability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014
Probability Machine Learning and Pattern Recognition Chris Williams School of Informatics, University of Edinburgh August 2014 (All of the slides in this course have been adapted from previous versions
More informationMath-Stat-491-Fall2014-Notes-I
Math-Stat-491-Fall2014-Notes-I Hariharan Narayanan October 2, 2014 1 Introduction This writeup is intended to supplement material in the prescribed texts: Introduction to Probability Models, 10th Edition,
More informationGreat Theoretical Ideas in Computer Science
15-251 Great Theoretical Ideas in Computer Science Probability Theory: Counting in Terms of Proportions Lecture 10 (September 27, 2007) Some Puzzles Teams A and B are equally good In any one game, each
More informationChapter 2: The Random Variable
Chapter : The Random Variable The outcome of a random eperiment need not be a number, for eample tossing a coin or selecting a color ball from a bo. However we are usually interested not in the outcome
More informationPart (A): Review of Probability [Statistics I revision]
Part (A): Review of Probability [Statistics I revision] 1 Definition of Probability 1.1 Experiment An experiment is any procedure whose outcome is uncertain ffl toss a coin ffl throw a die ffl buy a lottery
More informationSTAT 7032 Probability Spring Wlodek Bryc
STAT 7032 Probability Spring 2018 Wlodek Bryc Created: Friday, Jan 2, 2014 Revised for Spring 2018 Printed: January 9, 2018 File: Grad-Prob-2018.TEX Department of Mathematical Sciences, University of Cincinnati,
More informationProbability and random variables. Sept 2018
Probability and random variables Sept 2018 2 The sample space Consider an experiment with an uncertain outcome. The set of all possible outcomes is called the sample space. Example: I toss a coin twice,
More informationCS626 Data Analysis and Simulation
CS626 Data Analysis and Simulation Instructor: Peter Kemper R 104A, phone 221-3462, email:kemper@cs.wm.edu Today: Probability Primer Quick Reference: Sheldon Ross: Introduction to Probability Models 9th
More information