Mathematical Foundations
|
|
|
- Cody Cross
- 5 years ago
- Views:
Transcription
1 Mathematical Foundations Introduction to Data Science Algorithms Michael Paul and Jordan Boyd-Graber JANUARY 23, 2017 Introduction to Data Science Algorithms Boyd-Graber and Paul Mathematical Foundations 1 of 3
2 By the end of today... You ll be able to apply the concepts of distributions, independence, and conditional probabilities You ll be able to derive joint, marginal, and conditional probabilites from each other You ll be able to compute expectations and entropies Introduction to Data Science Algorithms Boyd-Graber and Paul Mathematical Foundations 2 of 3
3 Preface: Why make us do this? Probabilities are the language we use to describe data A reasonable (but geeky) definition of data science is how to get probabilities we care about from data Later classes will be about how to do this for different probability models But first, we need key definitions of probability (and it makes more sense to do it all at once) Introduction to Data Science Algorithms Boyd-Graber and Paul Mathematical Foundations 3 of 3
4 Preface: Why make us do this? Probabilities are the language we use to describe data A reasonable (but geeky) definition of data science is how to get probabilities we care about from data Later classes will be about how to do this for different probability models But first, we need key definitions of probability (and it makes more sense to do it all at once) So pay attention! Introduction to Data Science Algorithms Boyd-Graber and Paul Mathematical Foundations 3 of 3
5 Mathematical Foundations INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber JANUARY 23, 2017 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 1 of 4
6 Preface: Why make us do this? Probabilities (along with statistics) are the language we use to describe data A reasonable (but geeky) definition of data science is how to get probabilities we care about from data Later classes will be about how to do this for different probability models But first, we need key definitions of probability (and it makes more sense to do it all at once) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 4
7 Preface: Why make us do this? Probabilities (along with statistics) are the language we use to describe data A reasonable (but geeky) definition of data science is how to get probabilities we care about from data Later classes will be about how to do this for different probability models But first, we need key definitions of probability (and it makes more sense to do it all at once) So pay attention! INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 4
8 By the end of today... You ll be able to apply the concepts of distributions, independence, and conditional probabilities You ll be able to derive joint, marginal, and conditional probabilities from each other You ll be able to compute expectations and entropies INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 4
9 Engineering rationale behind probabilities Encoding uncertainty Data are variables We don t always know the values of variables Probabilities let us reason about variables even when we are uncertain INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 4
10 Engineering rationale behind probabilities Encoding uncertainty Data are variables We don t always know the values of variables Probabilities let us reason about variables even when we are uncertain Encoding confidence The flip side of uncertainty Useful for decision making: should we trust our conclusion? We can construct probabilistic models to boost our confidence E.g., combining polls INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 4
11 Mathematical Foundations INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 1 of 7
12 Random variable Probability is about random variables. A random variable is any probabilistic outcome. Examples of variables: Yesterday s high temperature The height of someone Examples of random variables: Tomorrow s high temperature The height of someone chosen randomly from a population INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 7
13 Random variable Probability is about random variables. A random variable is any probabilistic outcome. Examples of variables: Yesterday s high temperature The height of someone Examples of random variables: Tomorrow s high temperature The height of someone chosen randomly from a population We ll see that it s sometimes useful to think of quantities that are not strictly probabilistic as random variables. The high temperature on 03/04/1905 The number of times streetlight appears in a document INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 7
14 Random variable Random variables take on values in a sample space. They can be discrete or continuous: Coin flip: {H,T } Height: positive real values (0, ) Temperature: real values (, ) Number of words in a document: Positive integers {1,2,...} We call the outcomes events. Denote the random variable with a capital letter; denote a realization of the random variable with a lower case letter. E.g., X is a coin flip, x is the value (H or T ) of that coin flip. INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 7
15 Discrete distribution A discrete distribution assigns a probability to every event in the sample space For example, if X is a coin, then P(X = H) = 0.5 P(X = T) = 0.5 And probabilities have to be greater than or equal to 0 Probabilities of disjunctions are sums over part of the space. E.g., the probability that a die is bigger than 3: P(D > 3) = P(D = 4) + P(D = 5) + P(D = 6) The probabilities over the entire space must sum to one INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 7
16 Discrete distribution A discrete distribution assigns a probability to every event in the sample space For example, if X is a coin, then P(X = H) = 0.5 P(X = T) = 0.5 And probabilities have to be greater than or equal to 0 Probabilities of disjunctions are sums over part of the space. E.g., the probability that a die is bigger than 3: P(D > 3) = P(D = 4) + P(D = 5) + P(D = 6) The probabilities over the entire space must sum to one INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 7
17 Discrete distribution A discrete distribution assigns a probability to every event in the sample space For example, if X is a coin, then P(X = H) = 0.5 P(X = T) = 0.5 And probabilities have to be greater than or equal to 0 Probabilities of disjunctions are sums over part of the space. E.g., the probability that a die is bigger than 3: P(D > 3) = P(D = 4) + P(D = 5) + P(D = 6) The probabilities over the entire space must sum to one P(X = x) = 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 7
18 Discrete distribution A discrete distribution assigns a probability to every event in the sample space For example, if X is a coin, then P(X = H) = 0.5 P(X = T) = 0.5 And probabilities have to be greater than or equal to 0 Probabilities of disjunctions are sums over part of the space. E.g., the probability that a die is bigger than 3: P(D > 3) = P(D = 4) + P(D = 5) + P(D = 6) The probabilities over the entire space must sum to one P(X = x) = 1 x INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 7
19 Events An event is a set of outcomes to which a probability is assigned drawing a black card from a deck of cards drawing a King of Hearts Intersections and unions: Intersection: drawing a red and a King P(A B) (1) Union: drawing a spade or a King P(A B) = P(A) + P(B) P(A B) (2) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 7
20 Events An event is a set of outcomes to which a probability is assigned drawing a black card from a deck of cards drawing a King of Hearts Intersections and unions: Intersection: drawing a red and a King P(A B) (1) Union: drawing a spade or a King B Intersection of A and B A P(A B) = P(A) + P(B) P(A B) (2) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 7
21 Events An event is a set of outcomes to which a probability is assigned drawing a black card from a deck of cards drawing a King of Hearts Intersections and unions: Intersection: drawing a red and a King P(A B) (1) Union: drawing a spade or a King B Union of A and B A P(A B) = P(A) + P(B) P(A B) (2) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 7
22 Joint distribution Typically, we consider collections of random variables. The joint distribution is a distribution over the configuration of all the random variables in the ensemble. For example, imagine flipping 4 coins. The joint distribution is over the space of all possible outcomes of the four coins. P(HHHH) = P(HHHT) = P(HHTH) = You can think of it as a single random variable with 16 values. INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 6 of 7
23 Visualizing a joint distribution ~x x ~x, ~y ~x, y x, y x, ~y INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 7 of 7
24 Mathematical Foundations INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 1 of 3
25 Marginalization If we know a joint distribution of multiple variables, what if we want to know the distribution of only one of the variables? We can compute the distribution of P(X) from P(X,Y,Z) through marginalization: P(X,Y = y,z = z) = P(X)P(Y = y,z = z X) y z y z = P(X) P(Y = y,z = z X) = P(X) y z INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 3
26 Marginalization If we know a joint distribution of multiple variables, what if we want to know the distribution of only one of the variables? We can compute the distribution of P(X) from P(X,Y,Z) through marginalization: P(X,Y = y,z = z) = P(X)P(Y = y,z = z X) y z y z = P(X) P(Y = y,z = z X) = P(X) We ll explain this notation more next week for now the formula is the most important part. y z INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 3
27 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: Marginalize out weather Marginalize out temperature P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
28 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
29 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
30 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold.15 Marginalize out temperature New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
31 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
32 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature W=Sunny W=Cloudy New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
33 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature W=Sunny W=Cloudy New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
34 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature W=Sunny.40 W=Cloudy New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
35 Marginalization (from Leyton-Brown) Joint distribution temperature (T) and weather (W) T=Hot T=Mild T=Cold W=Sunny W=Cloudy Marginalization allows us to compute distributions over smaller sets of variables: P(X,Y) = z P(X,Y,Z = z) Corresponds to summing out a table dimension Marginalize out weather T=Hot T=Mild T=Cold Marginalize out temperature W=Sunny.40 W=Cloudy.60 New table still sums to 1 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 3
36 Mathematical Foundations INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 1 of 4
37 Independence Random variables X and Y are independent if and only if P(X = x,y = y) = P(X = x)p(y = y). Mathematical examples: If I flip a coin twice, is the second outcome independent from the first outcome? INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 4
38 Independence Random variables X and Y are independent if and only if P(X = x,y = y) = P(X = x)p(y = y). Mathematical examples: If I flip a coin twice, is the second outcome independent from the first outcome? If I draw two socks from my (multicolored) laundry, is the color of the first sock independent from the color of the second sock? INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 4
39 Independence Intuitive Examples: Independent: you use a Mac / the Hop bus is on schedule snowfall in the Himalayas / your favorite color is blue INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 4
40 Independence Intuitive Examples: Independent: you use a Mac / the Hop bus is on schedule snowfall in the Himalayas / your favorite color is blue Not independent: you vote for Mitt Romney / you are a Republican there is a traffic jam on 25 / the Broncos are playing INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 4
41 Independence Sometimes we make convenient assumptions. the values of two dice (ignoring gravity!) the value of the first die and the sum of the values whether it is raining and the number of taxi cabs whether it is raining and the amount of time it takes me to hail a cab the first two words in a sentence INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 4
42 Mathematical Foundations INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 1 of 5
43 Expectation An expectation of a random variable is a weighted average: E[f(X)] = f(x) p(x) (discrete) = x f(x) p(x) dx (continuous) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 5
44 Expectation Expectations of constants or known values: E[a] = a E[Y Y = y] = y INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 5
45 Expectation Intuition Average outcome (might not be an event: 2.4 children) Center of mass INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 5
46 Expectation of die / dice What is the expectation of the roll of die? INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
47 Expectation of die / dice What is the expectation of the roll of die? One die = INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
48 Expectation of die / dice What is the expectation of the roll of die? One die = 3.5 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
49 Expectation of die / dice What is the expectation of the roll of die? One die = 3.5 What is the expectation of the sum of two dice? INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
50 Expectation of die / dice What is the expectation of the roll of die? One die = 3.5 What is the expectation of the sum of two dice? Two die = INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
51 Expectation of die / dice What is the expectation of the roll of die? One die = 3.5 What is the expectation of the sum of two dice? Two die = 7 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
52 Mathematical Foundations INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 1 of 5
53 Entropy Measure of disorder in a system In the real world, entroy in a system tends to increase Can also be applied to probabilities: Is one (or a few) outcomes certain (low entropy) Are things equiprobable (high entropy) In data science We look for features that allow us to reduce entropy (decision trees) All else being equal, we seek models that have maximum entropy (Occam s razor) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 2 of 5
54 Aside: Logarithms lg(1)=0 lg(2)=1 lg(x) = b 2 b = x Makes big numbers small Way to think about them: cutting a carrot lg(4)=2 lg(8)=3 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 5
55 Aside: Logarithms lg(1)=0 lg(2)=1 lg(x) = b 2 b = x Makes big numbers small Way to think about them: cutting a carrot lg(4)=2 Negative numbers? lg(8)=3 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 5
56 Aside: Logarithms lg(1)=0 lg(x) = b 2 b = x lg(2)=1 Makes big numbers small Way to think about them: cutting a carrot Negative numbers? lg(4)=2 Non-integers? lg(8)=3 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 3 of 5
57 Entropy Entropy is a measure of uncertainty that is associated with the distribution of a random variable: H(X) = E[lg(p(X))] = p(x) lg(p(x)) = x p(x) lg(p(x)) dx (discrete) (continuous) INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 5
58 Entropy Entropy is a measure of uncertainty that is associated with the distribution of a random variable: H(X) = E[lg(p(X))] = p(x) lg(p(x)) = x p(x) lg(p(x)) dx (discrete) (continuous) Does not account for the values of the random variable, only the spread of the distribution. H(X) 0 uniform distribution = highest entropy, point mass = lowest suppose P(X = 1) = p, P(X = 0) = 1 p and P(Y = 100) = p, P(Y = 0) = 1 p: X and Y have the same entropy INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 4 of 5
59 Wrap up Probabilities are the language of data science You ll need to manipulate probabilities and understand marginalization and independence Thursday: Working through probability examples Next week: Conditional probabilities INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Mathematical Foundations 5 of 5
Expectations and Entropy
Expectations and Entropy Data Science: Jordan Boyd-Graber University of Maryland SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH Data Science: Jordan Boyd-Graber UMD Expectations and Entropy 1 / 9 Expectation
Random Variables and Events
Random Variables and Events Data Science: Jordan Boyd-Graber University of Maryland SLIDES ADAPTED FROM DAVE BLEI AND LAUREN HANNAH Data Science: Jordan Boyd-Graber UMD Random Variables and Events 1 /
Some Probability and Statistics
Some Probability and Statistics David M. Blei COS424 Princeton University February 13, 2012 Card problem There are three cards Red/Red Red/Black Black/Black I go through the following process. Close my
Some Probability and Statistics
Some Probability and Statistics David M. Blei COS424 Princeton University February 12, 2007 D. Blei ProbStat 01 1 / 42 Who wants to scribe? D. Blei ProbStat 01 2 / 42 Random variable Probability is about
Discrete Probability Distributions
Discrete Probability Distributions Data Science: Jordan Boyd-Graber University of Maryland JANUARY 18, 2018 Data Science: Jordan Boyd-Graber UMD Discrete Probability Distributions 1 / 1 Refresher: Random
COS 424: Interacting with Data. Lecturer: Dave Blei Lecture #2 Scribe: Ellen Kim February 7, 2008
COS 424: Interacting with Data Lecturer: Dave Blei Lecture #2 Scribe: Ellen Kim February 7, 2008 1 Probability Review 1.1 Random Variables A random variable can be used to model any probabilistic outcome.
Recitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
CMPSCI 240: Reasoning about Uncertainty
CMPSCI 240: Reasoning about Uncertainty Lecture 2: Sets and Events Andrew McGregor University of Massachusetts Last Compiled: January 27, 2017 Outline 1 Recap 2 Experiments and Events 3 Probabilistic Models
Chapter 6: Probability The Study of Randomness
Chapter 6: Probability The Study of Randomness 6.1 The Idea of Probability 6.2 Probability Models 6.3 General Probability Rules 1 Simple Question: If tossing a coin, what is the probability of the coin
Fundamentals of Probability CE 311S
Fundamentals of Probability CE 311S OUTLINE Review Elementary set theory Probability fundamentals: outcomes, sample spaces, events Outline ELEMENTARY SET THEORY Basic probability concepts can be cast in
Probability Year 10. Terminology
Probability Year 10 Terminology Probability measures the chance something happens. Formally, we say it measures how likely is the outcome of an event. We write P(result) as a shorthand. An event is some
Formalizing Probability. Choosing the Sample Space. Probability Measures
Formalizing Probability Choosing the Sample Space What do we assign probability to? Intuitively, we assign them to possible events (things that might happen, outcomes of an experiment) Formally, we take
Probability Year 9. Terminology
Probability Year 9 Terminology Probability measures the chance something happens. Formally, we say it measures how likely is the outcome of an event. We write P(result) as a shorthand. An event is some
The probability of an event is viewed as a numerical measure of the chance that the event will occur.
Chapter 5 This chapter introduces probability to quantify randomness. Section 5.1: How Can Probability Quantify Randomness? The probability of an event is viewed as a numerical measure of the chance that
Introduction to probability and statistics
Introduction to probability and statistics Alireza Fotuhi Siahpirani & Brittany Baur sroy@biostat.wisc.edu Computational Network Biology Biostatistics & Medical Informatics 826 https://compnetbiocourse.discovery.wisc.edu
Lecture 11: Information theory THURSDAY, FEBRUARY 21, 2019
Lecture 11: Information theory DANIEL WELLER THURSDAY, FEBRUARY 21, 2019 Agenda Information and probability Entropy and coding Mutual information and capacity Both images contain the same fraction of black
CS 361: Probability & Statistics
February 12, 2018 CS 361: Probability & Statistics Random Variables Monty hall problem Recall the setup, there are 3 doors, behind two of them are indistinguishable goats, behind one is a car. You pick
Probability Distributions: Continuous
Probability Distributions: Continuous INFO-2301: Quantitative Reasoning 2 Michael Paul and Jordan Boyd-Graber FEBRUARY 28, 2017 INFO-2301: Quantitative Reasoning 2 Paul and Boyd-Graber Probability Distributions:
Review Basic Probability Concept
Economic Risk and Decision Analysis for Oil and Gas Industry CE81.9008 School of Engineering and Technology Asian Institute of Technology January Semester Presented by Dr. Thitisak Boonpramote Department
Markov Chains. Chapter 16. Markov Chains - 1
Markov Chains Chapter 16 Markov Chains - 1 Why Study Markov Chains? Decision Analysis focuses on decision making in the face of uncertainty about one future event. However, many decisions need to consider
Econ 325: Introduction to Empirical Economics
Econ 325: Introduction to Empirical Economics Lecture 2 Probability Copyright 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 3-1 3.1 Definition Random Experiment a process leading to an uncertain
Outline Conditional Probability The Law of Total Probability and Bayes Theorem Independent Events. Week 4 Classical Probability, Part II
Week 4 Classical Probability, Part II Week 4 Objectives This week we continue covering topics from classical probability. The notion of conditional probability is presented first. Important results/tools
I - Probability. What is Probability? the chance of an event occuring. 1classical probability. 2empirical probability. 3subjective probability
What is Probability? the chance of an event occuring eg 1classical probability 2empirical probability 3subjective probability Section 2 - Probability (1) Probability - Terminology random (probability)
Lecture 1: Probability Fundamentals
Lecture 1: Probability Fundamentals IB Paper 7: Probability and Statistics Carl Edward Rasmussen Department of Engineering, University of Cambridge January 22nd, 2008 Rasmussen (CUED) Lecture 1: Probability
MATH MW Elementary Probability Course Notes Part I: Models and Counting
MATH 2030 3.00MW Elementary Probability Course Notes Part I: Models and Counting Tom Salisbury salt@yorku.ca York University Winter 2010 Introduction [Jan 5] Probability: the mathematics used for Statistics
Basic Concepts of Probability
Probability Probability theory is the branch of math that deals with unpredictable or random events Probability is used to describe how likely a particular outcome is in a random event the probability
Probability Experiments, Trials, Outcomes, Sample Spaces Example 1 Example 2
Probability Probability is the study of uncertain events or outcomes. Games of chance that involve rolling dice or dealing cards are one obvious area of application. However, probability models underlie
UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis
![UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis](/thumbs/90/103872711.jpg "UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis")
UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis Lecture 10 Class URL: http://vlsicad.ucsd.edu/courses/cse21-s14/ Lecture 10 Notes Midterm Good job overall! = 81; =
Probability - Lecture 4
1 Introduction Probability - Lecture 4 Many methods of computation physics and the comparison of data to a mathematical representation, apply stochastic methods. These ideas were first introduced in the
2011 Pearson Education, Inc
Statistics for Business and Economics Chapter 3 Probability Contents 1. Events, Sample Spaces, and Probability 2. Unions and Intersections 3. Complementary Events 4. The Additive Rule and Mutually Exclusive
Probability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3
Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................
3.2 Probability Rules
3.2 Probability Rules The idea of probability rests on the fact that chance behavior is predictable in the long run. In the last section, we used simulation to imitate chance behavior. Do we always need
Review of probability
Review of probability Computer Sciences 760 Spring 2014 http://pages.cs.wisc.edu/~dpage/cs760/ Goals for the lecture you should understand the following concepts definition of probability random variables
Generative Learning. INFO-4604, Applied Machine Learning University of Colorado Boulder. November 29, 2018 Prof. Michael Paul
Generative Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November 29, 2018 Prof. Michael Paul Generative vs Discriminative The classification algorithms we have seen so far
Ch 14 Randomness and Probability
Ch 14 Randomness and Probability We ll begin a new part: randomness and probability. This part contain 4 chapters: 14-17. Why we need to learn this part? Probability is not a portion of statistics. Instead
Discrete Mathematics and Probability Theory Fall 2014 Anant Sahai Note 15. Random Variables: Distributions, Independence, and Expectations
EECS 70 Discrete Mathematics and Probability Theory Fall 204 Anant Sahai Note 5 Random Variables: Distributions, Independence, and Expectations In the last note, we saw how useful it is to have a way of
Brief Review of Probability
Brief Review of Probability Nuno Vasconcelos (Ken Kreutz-Delgado) ECE Department, UCSD Probability Probability theory is a mathematical language to deal with processes or experiments that are non-deterministic
Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information.
L65 Dept. of Linguistics, Indiana University Fall 205 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission rate
Dept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 28 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission
Decision Trees. Nicholas Ruozzi University of Texas at Dallas. Based on the slides of Vibhav Gogate and David Sontag
Decision Trees Nicholas Ruozzi University of Texas at Dallas Based on the slides of Vibhav Gogate and David Sontag Supervised Learning Input: labelled training data i.e., data plus desired output Assumption:
Probability Basics. Part 3: Types of Probability. INFO-1301, Quantitative Reasoning 1 University of Colorado Boulder
Probability Basics Part 3: Types of Probability INFO-1301, Quantitative Reasoning 1 University of Colorado Boulder September 30, 2016 Prof. Michael Paul Prof. William Aspray Example A large government
Review: Probability. BM1: Advanced Natural Language Processing. University of Potsdam. Tatjana Scheffler
Review: Probability BM1: Advanced Natural Language Processing University of Potsdam Tatjana Scheffler tatjana.scheffler@uni-potsdam.de October 21, 2016 Today probability random variables Bayes rule expectation
V7 Foundations of Probability Theory
V7 Foundations of Probability Theory Probability : degree of confidence that an event of an uncertain nature will occur. Events : we will assume that there is an agreed upon space of possible outcomes
Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019
Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial
Basic Probability and Statistics
Basic Probability and Statistics Yingyu Liang yliang@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison [based on slides from Jerry Zhu, Mark Craven] slide 1 Reasoning with Uncertainty
1 Probability Theory. 1.1 Introduction
1 Probability Theory Probability theory is used as a tool in statistics. It helps to evaluate the reliability of our conclusions about the population when we have only information about a sample. Probability
Basic Concepts of Probability
Probability Probability theory is the branch of math that deals with random events Probability is used to describe how likely a particular outcome is in a random event the probability of obtaining heads
Properties of Probability
Econ 325 Notes on Probability 1 By Hiro Kasahara Properties of Probability In statistics, we consider random experiments, experiments for which the outcome is random, i.e., cannot be predicted with certainty.
MATH 3C: MIDTERM 1 REVIEW. 1. Counting
MATH 3C: MIDTERM REVIEW JOE HUGHES. Counting. Imagine that a sports betting pool is run in the following way: there are 20 teams, 2 weeks, and each week you pick a team to win. However, you can t pick
1. Consider a random independent sample of size 712 from a distribution with the following pdf. c 1+x. f(x) =
1. Consider a random independent sample of size 712 from a distribution with the following pdf f(x) = c 1+x 0
Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2017 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
Probability & Random Variables
& Random Variables Probability Probability theory is the branch of math that deals with random events, processes, and variables What does randomness mean to you? How would you define probability in your
18.05 Practice Final Exam
No calculators. 18.05 Practice Final Exam Number of problems 16 concept questions, 16 problems. Simplifying expressions Unless asked to explicitly, you don t need to simplify complicated expressions. For
UNIT 5 ~ Probability: What Are the Chances? 1
UNIT 5 ~ Probability: What Are the Chances? 1 6.1: Simulation Simulation: The of chance behavior, based on a that accurately reflects the phenomenon under consideration. (ex 1) Suppose we are interested
Machine Learning Recitation 8 Oct 21, Oznur Tastan
Machine Learning 10601 Recitation 8 Oct 21, 2009 Oznur Tastan Outline Tree representation Brief information theory Learning decision trees Bagging Random forests Decision trees Non linear classifier Easy
10601 Machine Learning
10601 Machine Learning September 2, 2009 Recitation 2 Öznur Taştan 1 Logistics Homework 2 is going to be out tomorrow. It is due on Sep 16, Wed. There is no class on Monday Sep 7 th (Labor day) Those who
Mathematical Probability
Mathematical Probability STA 281 Fall 2011 1 Introduction Engineers and scientists are always exposed to data, both in their professional capacities and in everyday activities. The discipline of statistics
Information Theory and Communication
Information Theory and Communication Ritwik Banerjee rbanerjee@cs.stonybrook.edu c Ritwik Banerjee Information Theory and Communication 1/8 General Chain Rules Definition Conditional mutual information
Outline. Probability. Math 143. Department of Mathematics and Statistics Calvin College. Spring 2010
Outline Math 143 Department of Mathematics and Statistics Calvin College Spring 2010 Outline Outline 1 Review Basics Random Variables Mean, Variance and Standard Deviation of Random Variables 2 More Review
Basic Probability. Introduction
Basic Probability Introduction The world is an uncertain place. Making predictions about something as seemingly mundane as tomorrow s weather, for example, is actually quite a difficult task. Even with
4/17/2012. NE ( ) # of ways an event can happen NS ( ) # of events in the sample space
I. Vocabulary: A. Outcomes: the things that can happen in a probability experiment B. Sample Space (S): all possible outcomes C. Event (E): one outcome D. Probability of an Event (P(E)): the likelihood
Notes on Probability
Notes on Probability Mark Schmidt January 7, 2017 1 Probabilites Consider an event A that may or may not happen. For example, if we roll a dice then we may or may not roll a 6. We use the notation p(a)
Topic 2 Probability. Basic probability Conditional probability and independence Bayes rule Basic reliability
Topic 2 Probability Basic probability Conditional probability and independence Bayes rule Basic reliability Random process: a process whose outcome can not be predicted with certainty Examples: rolling
18.05 Final Exam. Good luck! Name. No calculators. Number of problems 16 concept questions, 16 problems, 21 pages
Name No calculators. 18.05 Final Exam Number of problems 16 concept questions, 16 problems, 21 pages Extra paper If you need more space we will provide some blank paper. Indicate clearly that your solution
Probabilistic Systems Analysis Spring 2018 Lecture 6. Random Variables: Probability Mass Function and Expectation
EE 178 Probabilistic Systems Analysis Spring 2018 Lecture 6 Random Variables: Probability Mass Function and Expectation Probability Mass Function When we introduce the basic probability model in Note 1,
7.1 What is it and why should we care?
Chapter 7 Probability In this section, we go over some simple concepts from probability theory. We integrate these with ideas from formal language theory in the next chapter. 7.1 What is it and why should
What is a random variable
OKAN UNIVERSITY FACULTY OF ENGINEERING AND ARCHITECTURE MATH 256 Probability and Random Processes 04 Random Variables Fall 20 Yrd. Doç. Dr. Didem Kivanc Tureli didemk@ieee.org didem.kivanc@okan.edu.tr
Probability theory basics
Probability theory basics Michael Franke Basics of probability theory: axiomatic definition, interpretation, joint distributions, marginalization, conditional probability & Bayes rule. Random variables:
HW2 Solutions, for MATH441, STAT461, STAT561, due September 9th
HW2 Solutions, for MATH44, STAT46, STAT56, due September 9th. You flip a coin until you get tails. Describe the sample space. How many points are in the sample space? The sample space consists of sequences
02 Background Minimum background on probability. Random process
0 Background 0.03 Minimum background on probability Random processes Probability Conditional probability Bayes theorem Random variables Sampling and estimation Variance, covariance and correlation Probability
Probability the chance that an uncertain event will occur (always between 0 and 1)
Quantitative Methods 2013 1 Probability as a Numerical Measure of the Likelihood of Occurrence Probability the chance that an uncertain event will occur (always between 0 and 1) Increasing Likelihood of
STAT 111 Recitation 1
STAT 111 Recitation 1 Linjun Zhang January 20, 2017 What s in the recitation This class, and the exam of this class, is a mix of statistical concepts and calculations. We are going to do a little bit of
Exercises. Class choose one. 1.11: The birthday problem. 1.12: First to fail: Weibull.
Physics 4488/6562: Statistical Mechanics http://www.physics.cornell.edu/sethna/teaching/562/ Material for Week 1 Exercises due Monday Jan 29 Five Days From Today Last correction at February 2, 2018, 9:28
Multivariate probability distributions and linear regression
Multivariate probability distributions and linear regression Patrik Hoyer 1 Contents: Random variable, probability distribution Joint distribution Marginal distribution Conditional distribution Independence,
Discrete Random Variables
Discrete Random Variables An Undergraduate Introduction to Financial Mathematics J. Robert Buchanan 2014 Introduction The markets can be thought of as a complex interaction of a large number of random
STAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
Human-Oriented Robotics. Probability Refresher. Kai Arras Social Robotics Lab, University of Freiburg Winter term 2014/2015
Probability Refresher Kai Arras, University of Freiburg Winter term 2014/2015 Probability Refresher Introduction to Probability Random variables Joint distribution Marginalization Conditional probability
Joint Probability Distributions and Random Samples (Devore Chapter Five)
Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 1 Joint Probability Distributions 2 1.1 Two Discrete
UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis
![UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis](/thumbs/83/87821691.jpg "UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis")
UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis Lecture 10 Class URL: http://vlsicad.ucsd.edu/courses/cse21-s14/ Lecture 10 Notes Midterm Good job overall! = 81; =
4. Suppose that we roll two die and let X be equal to the maximum of the two rolls. Find P (X {1, 3, 5}) and draw the PMF for X.
Math 10B with Professor Stankova Worksheet, Midterm #2; Wednesday, 3/21/2018 GSI name: Roy Zhao 1 Problems 1.1 Bayes Theorem 1. Suppose a test is 99% accurate and 1% of people have a disease. What is the
1. Basics of Information
1. Basics of Information 6.004x Computation Structures Part 1 Digital Circuits Copyright 2015 MIT EECS 6.004 Computation Structures L1: Basics of Information, Slide #1 What is Information? Information,
Independence. P(A) = P(B) = 3 6 = 1 2, and P(C) = 4 6 = 2 3.
Example: A fair die is tossed and we want to guess the outcome. The outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 1 6 each. If we are interested in getting the following results: A = {1, 3,
Lecture #13 Tuesday, October 4, 2016 Textbook: Sections 7.3, 7.4, 8.1, 8.2, 8.3
STATISTICS 200 Lecture #13 Tuesday, October 4, 2016 Textbook: Sections 7.3, 7.4, 8.1, 8.2, 8.3 Objectives: Identify, and resist the temptation to fall for, the gambler s fallacy Define random variable
STAT Chapter 3: Probability
Basic Definitions STAT 515 --- Chapter 3: Probability Experiment: A process which leads to a single outcome (called a sample point) that cannot be predicted with certainty. Sample Space (of an experiment):
Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides
Probability Review Lecturer: Ji Liu Thank Jerry Zhu for sharing his slides slide 1 Inference with Bayes rule: Example In a bag there are two envelopes one has a red ball (worth $100) and a black ball one
Chapter 2: Entropy and Mutual Information. University of Illinois at Chicago ECE 534, Natasha Devroye
Chapter 2: Entropy and Mutual Information Chapter 2 outline Definitions Entropy Joint entropy, conditional entropy Relative entropy, mutual information Chain rules Jensen s inequality Log-sum inequality
Review of Probability Mark Craven and David Page Computer Sciences 760.
Review of Probability Mark Craven and David Page Computer Sciences 760 www.biostat.wisc.edu/~craven/cs760/ Goals for the lecture you should understand the following concepts definition of probability random
Notes slides from before lecture. CSE 21, Winter 2017, Section A00. Lecture 16 Notes. Class URL:
Notes slides from before lecture CSE 21, Winter 2017, Section A00 Lecture 16 Notes Class URL: http://vlsicad.ucsd.edu/courses/cse21-w17/ Notes slides from before lecture Notes March 8 (1) This week: Days
Problem # Number of points 1 /20 2 /20 3 /20 4 /20 5 /20 6 /20 7 /20 8 /20 Total /150
Name Student ID # Instructor: SOLUTION Sergey Kirshner STAT 516 Fall 09 Practice Midterm #1 January 31, 2010 You are not allowed to use books or notes. Non-programmable non-graphic calculators are permitted.
Introduction to probability
Introduction to probability 4.1 The Basics of Probability Probability The chance that a particular event will occur The probability value will be in the range 0 to 1 Experiment A process that produces
Probability Theory Review
Cogsci 118A: Natural Computation I Lecture 2 (01/07/10) Lecturer: Angela Yu Probability Theory Review Scribe: Joseph Schilz Lecture Summary 1. Set theory: terms and operators In this section, we provide
Discrete Probability. Chemistry & Physics. Medicine
Discrete Probability The existence of gambling for many centuries is evidence of long-running interest in probability. But a good understanding of probability transcends mere gambling. The mathematics
Statistics for Managers Using Microsoft Excel (3 rd Edition)
Statistics for Managers Using Microsoft Excel (3 rd Edition) Chapter 4 Basic Probability and Discrete Probability Distributions 2002 Prentice-Hall, Inc. Chap 4-1 Chapter Topics Basic probability concepts
Statistics for Business and Economics
Statistics for Business and Economics Basic Probability Learning Objectives In this lecture(s), you learn: Basic probability concepts Conditional probability To use Bayes Theorem to revise probabilities
Today we ll discuss ways to learn how to think about events that are influenced by chance.
Overview Today we ll discuss ways to learn how to think about events that are influenced by chance. Basic probability: cards, coins and dice Definitions and rules: mutually exclusive events and independent
More on Distribution Function
More on Distribution Function The distribution of a random variable X can be determined directly from its cumulative distribution function F X. Theorem: Let X be any random variable, with cumulative distribution
Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
Motivation. Stat Camp for the MBA Program. Probability. Experiments and Outcomes. Daniel Solow 5/10/2017
Stat Camp for the MBA Program Daniel Solow Lecture 2 Probability Motivation You often need to make decisions under uncertainty, that is, facing an unknown future. Examples: How many computers should I
Physics 403. Segev BenZvi. Choosing Priors and the Principle of Maximum Entropy. Department of Physics and Astronomy University of Rochester
Physics 403 Choosing Priors and the Principle of Maximum Entropy Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Odds Ratio Occam Factors
What does independence look like?
What does independence look like? Independence S AB A Independence Definition 1: P (AB) =P (A)P (B) AB S = A S B S B Independence Definition 2: P (A B) =P (A) AB B = A S Independence? S A Independence