Soc500: Applied Social Statistics Week 1: Introduction and Probability

Transcription

1 Soc500: Applied Social Statistics Week 1: Introduction and Probability Brandon Stewart 1 Princeton September 14, These slides are heavily influenced by Matt Blackwell, Adam Glynn and Matt Salganik. The spam filter segment is adapted from Justin Grimmer and Dan Jurafsky. Illustrations by Shay O Brien. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

2 Where We ve Been and Where We re Going... Last Week methods camp pre-grad school life This Week Wednesday welcome basics of probability Next Week random variables joint distributions Long Run probability inference regression Questions? Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

3 Welcome and Introductions Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

4 Welcome and Introductions Soc500: Applied Social Statistics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

5 Welcome and Introductions Soc500: Applied Social Statistics I Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

6 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

7 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

8 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

9 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis... love doing collaborative research Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

10 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis... love doing collaborative research... talk very quickly Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

11 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis... love doing collaborative research... talk very quickly Your Preceptors Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

12 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis... love doing collaborative research... talk very quickly Your Preceptors sage guides of all things Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

13 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis... love doing collaborative research... talk very quickly Your Preceptors sage guides of all things Ian Lundberg Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

14 Welcome and Introductions Soc500: Applied Social Statistics I... am an Assistant Professor in Sociology.... am trained in political science and statistics... do research in methods and statistical text analysis... love doing collaborative research... talk very quickly Your Preceptors sage guides of all things Ian Lundberg Simone Zhang Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

15 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

16 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

17 The Core Strategy Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

18 The Core Strategy Goal: get you ready to quantitative work Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

19 The Core Strategy Goal: get you ready to quantitative work First in a two course sequence replication project Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

20 The Core Strategy Goal: get you ready to quantitative work First in a two course sequence replication project (for graduate students, part of a longer arc) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

21 The Core Strategy Goal: get you ready to quantitative work First in a two course sequence replication project (for graduate students, part of a longer arc) Difficult course but with many resources to support you. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

22 The Core Strategy Goal: get you ready to quantitative work First in a two course sequence replication project (for graduate students, part of a longer arc) Difficult course but with many resources to support you. When we are done you will be able to teach yourself many things Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

23 Specific Goals Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

24 Specific Goals For the semester Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

25 Specific Goals For the semester critically read, interpret and replicate the quantitative content of many articles in the quantitative social sciences Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

26 Specific Goals For the semester critically read, interpret and replicate the quantitative content of many articles in the quantitative social sciences conduct, interpret, and communicate results from analysis using multiple regression Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

27 Specific Goals For the semester critically read, interpret and replicate the quantitative content of many articles in the quantitative social sciences conduct, interpret, and communicate results from analysis using multiple regression explain the limitations of observational data for making causal claims Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

28 Specific Goals For the semester critically read, interpret and replicate the quantitative content of many articles in the quantitative social sciences conduct, interpret, and communicate results from analysis using multiple regression explain the limitations of observational data for making causal claims write clean, reusable, and reliable R code. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

29 Specific Goals For the semester critically read, interpret and replicate the quantitative content of many articles in the quantitative social sciences conduct, interpret, and communicate results from analysis using multiple regression explain the limitations of observational data for making causal claims write clean, reusable, and reliable R code. feel empowered working with data Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

30 Specific Goals Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

31 Specific Goals For the year Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

32 Specific Goals For the year conduct, interpret, and communicate results from analysis using generalized linear models Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

33 Specific Goals For the year conduct, interpret, and communicate results from analysis using generalized linear models understand the fundamental ideas of missing data, modern causal inference, and hierarchical models Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

34 Specific Goals For the year conduct, interpret, and communicate results from analysis using generalized linear models understand the fundamental ideas of missing data, modern causal inference, and hierarchical models build a solid, reproducible research pipeline to go from raw data to final paper Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

35 Specific Goals For the year conduct, interpret, and communicate results from analysis using generalized linear models understand the fundamental ideas of missing data, modern causal inference, and hierarchical models build a solid, reproducible research pipeline to go from raw data to final paper provide you with the tools to produce your own research (e.g. second year empirical paper). Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

36 Why R? It s free Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

37 Why R? It s free It s extremely powerful, but relatively simple to do basic stats Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

38 Why R? It s free It s extremely powerful, but relatively simple to do basic stats It s the de facto standard in many applied statistical fields Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

39 Why R? It s free It s extremely powerful, but relatively simple to do basic stats It s the de facto standard in many applied statistical fields great community support Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

40 Why R? It s free It s extremely powerful, but relatively simple to do basic stats It s the de facto standard in many applied statistical fields great community support continuing development Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

41 Why R? It s free It s extremely powerful, but relatively simple to do basic stats It s the de facto standard in many applied statistical fields great community support continuing development massive number of supporting packages Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

42 Why R? It s free It s extremely powerful, but relatively simple to do basic stats It s the de facto standard in many applied statistical fields great community support continuing development massive number of supporting packages It will help you do research Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

43 Why RMarkdown? What you ve done before Baumer et al (2014) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

44 Why RMarkdown? RMarkdown Baumer et al (2014) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

45 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

46 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

47 Mathematical Prerequisites Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

48 Mathematical Prerequisites No formal pre-requisites Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

49 Mathematical Prerequisites No formal pre-requisites Balancing rigor and intuition Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

50 Mathematical Prerequisites No formal pre-requisites Balancing rigor and intuition no rigor for rigor s sake Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

51 Mathematical Prerequisites No formal pre-requisites Balancing rigor and intuition no rigor for rigor s sake we will tell you why you need the math, but also feel free to ask Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

52 Mathematical Prerequisites No formal pre-requisites Balancing rigor and intuition no rigor for rigor s sake we will tell you why you need the math, but also feel free to ask We will teach you any math you need as we go along Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

53 Mathematical Prerequisites No formal pre-requisites Balancing rigor and intuition no rigor for rigor s sake we will tell you why you need the math, but also feel free to ask We will teach you any math you need as we go along Crucially though- this class is not about statistical aptitude, it is about effort Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

54 Ways to Learn Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

55 Ways to Learn Lecture learn broad topics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

56 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

57 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

58 Reading Required reading: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

59 Reading Required reading: Fox (2016) Applied Regression Analysis and Generalized Linear Models Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

60 Reading Required reading: Fox (2016) Applied Regression Analysis and Generalized Linear Models Angrist and Pischke (2008) Mostly Harmless Econometrics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

61 Reading Required reading: Fox (2016) Applied Regression Analysis and Generalized Linear Models Angrist and Pischke (2008) Mostly Harmless Econometrics Imai (2017) A First Course in Quantitative Social Science* Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

62 Reading Required reading: Fox (2016) Applied Regression Analysis and Generalized Linear Models Angrist and Pischke (2008) Mostly Harmless Econometrics Imai (2017) A First Course in Quantitative Social Science* Aronow and Miller (2017) Theory of Agnostic Statistics* Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

63 Reading Required reading: Fox (2016) Applied Regression Analysis and Generalized Linear Models Angrist and Pischke (2008) Mostly Harmless Econometrics Imai (2017) A First Course in Quantitative Social Science* Aronow and Miller (2017) Theory of Agnostic Statistics* Suggested reading Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

64 Reading Required reading: Fox (2016) Applied Regression Analysis and Generalized Linear Models Angrist and Pischke (2008) Mostly Harmless Econometrics Imai (2017) A First Course in Quantitative Social Science* Aronow and Miller (2017) Theory of Agnostic Statistics* Suggested reading When and how to do the reading Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

65 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

66 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Problem Sets reinforce understanding of material, practice Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

67 Problem Sets Schedule (available Wednesday, due 8 days later at precept) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

68 Problem Sets Schedule (available Wednesday, due 8 days later at precept) Grading and solutions Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

69 Problem Sets Schedule (available Wednesday, due 8 days later at precept) Grading and solutions Code conventions Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

70 Problem Sets Schedule (available Wednesday, due 8 days later at precept) Grading and solutions Code conventions Collaboration policy Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

71 Problem Sets Schedule (available Wednesday, due 8 days later at precept) Grading and solutions Code conventions Collaboration policy Note: You may find these difficult. Start early and seek help! Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

72 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Problem Sets reinforce understanding of material, practice Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

73 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Problem Sets reinforce understanding of material, practice Piazza ask questions of us and your classmates Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

74 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Problem Sets reinforce understanding of material, practice Piazza ask questions of us and your classmates Office Hours ask even more questions. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

75 Ways to Learn Lecture learn broad topics Precept learn data analysis skills, get targeted help on assignments Readings support materials for lecture and precept Problem Sets reinforce understanding of material, practice Piazza ask questions of us and your classmates Office Hours ask even more questions. Your Job: get help when you need it! Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

76 Attribution and Thanks Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

77 Attribution and Thanks My philosophy on teaching: don t reinvent the wheel- customize, refine, improve. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

78 Attribution and Thanks My philosophy on teaching: don t reinvent the wheel- customize, refine, improve. Huge thanks to those who have provided slides particularly: Matt Blackwell, Adam Glynn, Justin Grimmer, Jens Hainmueller, Kevin Quinn Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

79 Attribution and Thanks My philosophy on teaching: don t reinvent the wheel- customize, refine, improve. Huge thanks to those who have provided slides particularly: Matt Blackwell, Adam Glynn, Justin Grimmer, Jens Hainmueller, Kevin Quinn Also thanks to those who have discussed with me at length including Dalton Conley, Chad Hazlett, Gary King, Kosuke Imai, Matt Salganik and Teppei Yamamoto. Shay O Brien produced the hand-drawn illustrations used throughout. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

80 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

81 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

82 Outline of Topics Outline in reverse order: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

83 Outline of Topics Outline in reverse order: Regression: how to determine the relationship between variables. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

84 Outline of Topics Outline in reverse order: Regression: how to determine the relationship between variables. Inference: how to learn about things we don t know from the things we do know Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

85 Outline of Topics Outline in reverse order: Regression: how to determine the relationship between variables. Inference: how to learn about things we don t know from the things we do know Probability: learn what data we would expect if we did know the truth. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

86 Outline of Topics Outline in reverse order: Regression: how to determine the relationship between variables. Inference: how to learn about things we don t know from the things we do know Probability: learn what data we would expect if we did know the truth. Probability Inference Regression Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

87 What is Statistics? branch of mathematics studying collection and analysis of data Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

88 What is Statistics? branch of mathematics studying collection and analysis of data the name statistic comes from the word state Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

89 What is Statistics? branch of mathematics studying collection and analysis of data the name statistic comes from the word state the arc of developments in statistics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

90 What is Statistics? branch of mathematics studying collection and analysis of data the name statistic comes from the word state the arc of developments in statistics 1 an applied scholar has a problem Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

91 What is Statistics? branch of mathematics studying collection and analysis of data the name statistic comes from the word state the arc of developments in statistics 1 an applied scholar has a problem 2 they solve the problem by inventing a specific method Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

92 What is Statistics? branch of mathematics studying collection and analysis of data the name statistic comes from the word state the arc of developments in statistics 1 an applied scholar has a problem 2 they solve the problem by inventing a specific method 3 statisticians generalize and export the best of these methods Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

93 Quantitative Research in Theory Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

94 Quantitative Research in Theory Inspiration: Hadley Wickham, Image: Matt Salganik Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

95 Quantitative Research in Practice Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

96 Quantitative Research in Practice Inspiration: Hadley Wickham, Image: Matt Salganik Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

97 Traditional Statistics Class Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

98 Traditional Statistics Class Inspiration: Hadley Wickham, Image: Matt Salganik Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

99 Time Actually Spent Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

100 Time Actually Spent Inspiration: Hadley Wickham, Image: Matt Salganik Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

101 This Class Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

102 This Class Strike a balance between practice and theory Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

103 This Class Strike a balance between practice and theory Heavy emphasis on applied data analysis Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

104 This Class Strike a balance between practice and theory Heavy emphasis on applied data analysis problem sets with real data Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

105 This Class Strike a balance between practice and theory Heavy emphasis on applied data analysis problem sets with real data replication project next semester Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

106 This Class Strike a balance between practice and theory Heavy emphasis on applied data analysis problem sets with real data replication project next semester Teaching select key principles from statistics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

107 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

108 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

109 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

110 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

111 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

112 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). but that s impossible Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

113 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). but that s impossible A better approach Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

114 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). but that s impossible A better approach Instead treat other factors as stochastic Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

115 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). but that s impossible A better approach Instead treat other factors as stochastic Thus we often write it as performance i = f (hours i ) + ɛ i Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

116 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). but that s impossible A better approach Instead treat other factors as stochastic Thus we often write it as performance i = f (hours i ) + ɛ i This allows us to have uncertainty over outcomes given our inputs Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

117 Deterministic vs. Stochastic what is the relationship between hours spent studying and performance in Soc500? One way to approach this: generate a deterministic account of performance performance i = f (hours i ) but studying isn t the only indicator of performance! we could try to account for everything performance i = f (hours i ) + g(other i ). but that s impossible A better approach Instead treat other factors as stochastic Thus we often write it as performance i = f (hours i ) + ɛ i This allows us to have uncertainty over outcomes given our inputs Our way of talking about stochastic outcomes is probability. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

118 In a rasa ± I. a**roao.. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

119 In Picture Form. :..arrf ± HE rasa a :. a**roao.. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

120 In Picture Form probability Data generating process Observed data inference Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

121 Statistical Thought Experiments Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

122 Statistical Thought Experiments Start with probability Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

123 Statistical Thought Experiments Start with probability Allows us to contemplate world under hypothetical scenarios Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

124 Statistical Thought Experiments Start with probability Allows us to contemplate world under hypothetical scenarios hypotheticals let us ask- is the observed relationship happening by chance or is it systematic? Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

125 Statistical Thought Experiments Start with probability Allows us to contemplate world under hypothetical scenarios hypotheticals let us ask- is the observed relationship happening by chance or is it systematic? it tells us what the world would look like under a certain assumption Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

126 Statistical Thought Experiments Start with probability Allows us to contemplate world under hypothetical scenarios hypotheticals let us ask- is the observed relationship happening by chance or is it systematic? it tells us what the world would look like under a certain assumption We will review probability today, but feel free to ask questions as needed. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

127 Example: Fisher s Lady Tasting Tea Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

128 Example: Fisher s Lady Tasting Tea The Story Setup Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

129 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

130 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

131 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment (perform a taste test) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

132 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment (perform a taste test) The Hypothetical Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

133 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment (perform a taste test) The Hypothetical (count possibilities) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

134 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment (perform a taste test) The Hypothetical (count possibilities) The Result Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

135 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment (perform a taste test) The Hypothetical (count possibilities) The Result (boom she was right) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

136 Example: Fisher s Lady Tasting Tea The Story Setup (lady discerning about tea) The Experiment (perform a taste test) The Hypothetical (count possibilities) The Result (boom she was right) This became the Fisher Exact Test. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

137 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

138 1 Welcome 2 Goals 3 Ways to Learn 4 Structure of Course 5 Introduction to Probability What is Probability? Sample Spaces and Events Probability Functions Marginal, Joint and Conditional Probability Bayes Rule Independence 6 Fun With History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

139 Why Probability? Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

140 Why Probability? Helps us envision hypotheticals Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

141 Why Probability? Helps us envision hypotheticals Describes uncertainty in how the data is generated Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

142 Why Probability? Helps us envision hypotheticals Describes uncertainty in how the data is generated Data Analysis: estimate probability that something will happen Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

143 Why Probability? Helps us envision hypotheticals Describes uncertainty in how the data is generated Data Analysis: estimate probability that something will happen Thus: we need to know how probability gives rise to data Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

144 Intuitive Definition of Probability Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

145 Intuitive Definition of Probability While there are several interpretations of what probability is, most modern (post 1935 or so) researchers agree on an axiomatic definition of probability. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

146 Intuitive Definition of Probability While there are several interpretations of what probability is, most modern (post 1935 or so) researchers agree on an axiomatic definition of probability. 3 Axioms (Intuitive Version): Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

147 Intuitive Definition of Probability While there are several interpretations of what probability is, most modern (post 1935 or so) researchers agree on an axiomatic definition of probability. 3 Axioms (Intuitive Version): 1 The probability of any particular event must be non-negative. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

148 Intuitive Definition of Probability While there are several interpretations of what probability is, most modern (post 1935 or so) researchers agree on an axiomatic definition of probability. 3 Axioms (Intuitive Version): 1 The probability of any particular event must be non-negative. 2 The probability of anything occurring among all possible events must be 1. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

149 Intuitive Definition of Probability While there are several interpretations of what probability is, most modern (post 1935 or so) researchers agree on an axiomatic definition of probability. 3 Axioms (Intuitive Version): 1 The probability of any particular event must be non-negative. 2 The probability of anything occurring among all possible events must be 1. 3 The probability of one of many mutually exclusive events happening is the sum of the individual probabilities. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

150 Intuitive Definition of Probability While there are several interpretations of what probability is, most modern (post 1935 or so) researchers agree on an axiomatic definition of probability. 3 Axioms (Intuitive Version): 1 The probability of any particular event must be non-negative. 2 The probability of anything occurring among all possible events must be 1. 3 The probability of one of many mutually exclusive events happening is the sum of the individual probabilities. All the rules of probability can be derived from these axioms. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

151 Sample Spaces Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

152 Sample Spaces To define probability we need to define the set of possible outcomes. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

153 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

154 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

155 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

156 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = { {heads, heads}, Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

157 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = { {heads, heads}, {heads, tails}, Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

158 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = { {heads, heads}, {heads, tails}, {tails, heads}, Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

159 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = { {heads, heads}, {heads, tails}, {tails, heads}, {tails, tails} } Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

160 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = { {heads, heads}, {heads, tails}, {tails, heads}, {tails, tails} } Thus the table in Lady Tasting Tea was defining the sample space. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

161 Sample Spaces To define probability we need to define the set of possible outcomes. The sample space is the set of all possible outcomes, and is often written as S or Ω. For example, if we flip a coin twice, there are four possible outcomes, S = { {heads, heads}, {heads, tails}, {tails, heads}, {tails, tails} } Thus the table in Lady Tasting Tea was defining the sample space. (Note we defined illogical guesses to be prob= 0) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

162 A Running Visual Metaphor Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

163 A Running Visual Metaphor Imagine that we sample an apple from a bag. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

164 A Running Visual Metaphor Imagine that we sample an apple from a bag. Looking in the bag we see: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

165 A Running Visual Metaphor Imagine that we sample an apple from a bag. Looking in the bag we see: The sample space is: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

166 Events Events are subsets of the sample space. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

167 Events Events are subsets of the sample space. For Example, if Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

168 . Events Events are subsets of the sample space. For Example, if then a given }, ) and { rain } are both. T.B.DMBB.GG. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

169 Events Are a Kind of Set Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

170 Events Are a Kind of Set Sets are collections of things, in this case collections of outcomes Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

171 Events Are a Kind of Set Sets are collections of things, in this case collections of outcomes One way to define an event is to describe the common property that all of the outcomes share. We write this as {ω ω satisfies P}, where P is the property that they all share. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

172 Events Are a Kind of Set Sets are collections of things, in this case collections of outcomes One way to define an event is to describe the common property that all of the outcomes share. We write this as {ω ω satisfies P}, where P is the property that they all share. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

173 Complement A complement of event A is a set: A c, is collection of all of the outcomes not in A. That is, it is everything else in the sample space. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

174 . Complement A complement of event A is a set: A c, is collection of all of the outcomes not in A. That is, it is everything else in the sample space. a given }, ) and again { } g.a.oa.baaqog@rwaaaa.brb.s are A c = {ω Ω ω / A}. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

175 . Complement A complement of event A is a set: A c, is collection of all of the outcomes not in A. That is, it is everything else in the sample space. a given }, ) and again { } g.a.oa.baaqog@rwaaaa.brb.s are A c = {ω Ω ω / A}. Important complement: Ω c =, where is the empty set it s just the event that nothing happens. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

176 Operations on Events Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

177 Operations on Events The union of two events, A and B is the event that A or B occurs: { air. u * on #Mina,, = air } A B = {ω ω A or ω B}. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

178 Operations on Events The union of two events, A and B is the event that A or B occurs: { air. u * on #Mina,, = air } A B = {ω ω A or ω B}. The intersection of two events, A and B is the event that both A and B occur: s.no#go= A B = {ω ω A and ω B}. {ease } Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

179 Operations on Events We say that two events A and B are disjoint or mutually exclusive if they don t share any elements or that A B =. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

180 Operations on Events We say that two events A and B are disjoint or mutually exclusive if they don t share any elements or that A B =. An event and its complement A and A c are disjoint. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

181 Operations on Events We say that two events A and B are disjoint or mutually exclusive if they don t share any elements or that A B =. An event and its complement A and A c are disjoint. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

182 Operations on Events We say that two events A and B are disjoint or mutually exclusive if they don t share any elements or that A B =. An event and its complement A and A c are disjoint. Sample spaces can have infinite outcomes A 1, A 2,.... Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

183 Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

184 . P( - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. B 89 ) =. 5 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

185 . P( - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity B 89 ) =. 5 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

186 . P. - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity 2 P(S) = 1 B 399k (89 ) = P ( { air, raise. = Bin } ), 1 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

187 . P. - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity 2 P(S) = 1 normalization B 399k (89 ) = P ( { air, raise. = Bin } ), 1 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

188 . P. - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity 2 P(S) = 1 normalization 3 if events A 1, A 2,... are mutually exclusive then P( i=1 A i) = i=1 P(A i). B 399k B.. (89 ) =. 5 (given an} = ), Praa) P = ) P ( Aip ) + {give P (.ba when nq a and he# are mutually exclusive. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

189 . P. - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity 2 P(S) = 1 normalization 3 if events A 1, A 2,... are mutually exclusive then P( i=1 A i) = i=1 P(A i). additivity B 399k B.. (89 ) =. 5 (given an} = ), Praa) P = ) P ( Aip ) + {give P (.ba when nq a and he# are mutually exclusive. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

190 . P. - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity 2 P(S) = 1 normalization 3 if events A 1, A 2,... are mutually exclusive then P( i=1 A i) = i=1 P(A i). additivity B 399k B.. (89 ) =. 5 (given an} = ), Praa) P = ) P ( Aip ) + {give P (.ba when nq a and he# are mutually exclusive. All the rules of probability can be derived from these axioms. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

191 . P. - Probability Function A probability function P( ) is a function defined over all subsets of a sample space S that satisfies the following three axioms: 1 P(A) 0 for all A in the set of all events. nonnegativity 2 P(S) = 1 normalization 3 if events A 1, A 2,... are mutually exclusive then P( i=1 A i) = i=1 P(A i). additivity B 399k B.. (89 ) =. 5 (given an} = ), Praa) P = ) P ( Aip ) + {give P (.ba when nq a and he# are mutually exclusive. All the rules of probability can be derived from these axioms. Intuition: probability as allocating chunks of a unit-long stick. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

192 A Brief Word on Interpretation Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

193 A Brief Word on Interpretation Massive debate on interpretation: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

194 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

195 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

196 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. But... Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

197 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. But... If you don t follow the axioms, a bookie can beat you Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

198 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. But... If you don t follow the axioms, a bookie can beat you There is a correct way to update your beliefs with data. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

199 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. But... If you don t follow the axioms, a bookie can beat you There is a correct way to update your beliefs with data. Frequency Interpretation Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

200 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. But... If you don t follow the axioms, a bookie can beat you There is a correct way to update your beliefs with data. Frequency Interpretation Probability of is the relative frequency with which an event would occur if the process were repeated a large number of times under similar conditions. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

201 A Brief Word on Interpretation Massive debate on interpretation: Subjective Interpretation Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is whatever you want it to be. But... If you don t follow the axioms, a bookie can beat you There is a correct way to update your beliefs with data. Frequency Interpretation Probability of is the relative frequency with which an event would occur if the process were repeated a large number of times under similar conditions. Example: The probability of drawing 5 red cards out of 10 drawn from a deck of cards is the frequency with which this event occurs in repeated samples of 10 cards. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

202 Marginal and Joint Probability So far we have only considered situations where we are interested in the probability of a single event A occurring. We ve denoted this P(A). P(A) is sometimes called a marginal probability. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

203 Marginal and Joint Probability So far we have only considered situations where we are interested in the probability of a single event A occurring. We ve denoted this P(A). P(A) is sometimes called a marginal probability. Suppose we are now in a situation where we would like to express the probability that an event A and an event B occur. This quantity is written as P(A B), P(B A), P(A, B), or P(B, A) and is the joint probability of A and B. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

204 p( a.) =? qq.pe#)= p( a *air.. #*@a rain. ringing. 589? a a.. µ Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

205 Conditional Probability Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

206 Conditional Probability The soul of statistics Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

207 Conditional Probability The soul of statistics If P(A) > 0 then the probability of B conditional on A can be written as Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

208 Conditional Probability The soul of statistics If P(A) > 0 then the probability of B conditional on A can be written as P(B A) = P(A, B) P(A) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

209 Conditional Probability The soul of statistics If P(A) > 0 then the probability of B conditional on A can be written as P(B A) = P(A, B) P(A) This implies that Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

210 Conditional Probability The soul of statistics If P(A) > 0 then the probability of B conditional on A can be written as P(B A) = P(A, B) P(A) This implies that P(A, B) = P(A) P(B A) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

211 Conditional Probability: A Visual Example. ) Parsa = t.io bids PC.ae#dpl*a grain. a gear a * or oaivoiv Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

212 Conditional Probability: A Visual Example. ) Parsa = t.io bids PC.ae#dpl*a grain. a gear a * or oaivoiv Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

213 Conditional Probability: A Visual Example. ) Parsa = t.io bids PC.ae#dpl*a grain. a gear a * or oaivoiv Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

214 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

215 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

216 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = 4/52 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

217 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = 4/52 P(B A) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

218 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = 4/52 P(B A) = 3/51 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

219 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = 4/52 P(B A) = 3/51 P(A, B) = P(A) P(B A) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

220 A Card Player s Example If we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = 4/52 P(B A) = 3/51 P(A, B) = P(A) P(B A) = 4/52 3/ Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

221 Law of Total Probability (LTP) With 2 Events: P(B) = P(B, A) + P(B, A c ) = P(B A) P(A) + P(B A c ) P(A c ) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

222 Recall, if we randomly draw two cards from a standard 52 card deck and define the events A = {King on Draw 1} and B = {King on Draw 2}, then P(A) = 4/52 P(B A) = 3/51 P(A, B) = P(A) P(B A) = 4/52 3/51 Question: P(B) =? Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

223 Confirming Intuition with the LTP Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

224 Confirming Intuition with the LTP P(B) = P(B, A) + P(B, A c ) = P(B A) P(A) + P(B A c ) P(A c ) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

225 Confirming Intuition with the LTP P(B) = P(B, A) + P(B, A c ) = P(B A) P(A) + P(B A c ) P(A c ) P(B) = 3/51 1/13 + 4/51 12/13 = = 1 13 = 4 52 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

226 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

227 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

228 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

229 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

230 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Pr(mobilized) = 0.6 and so Pr( not mobilized) = 0.4 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

231 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Pr(mobilized) = 0.6 and so Pr( not mobilized) = 0.4 Note that mobilization partitions the data. Everyone is either mobilized or not. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

232 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Pr(mobilized) = 0.6 and so Pr( not mobilized) = 0.4 Note that mobilization partitions the data. Everyone is either mobilized or not. Thus, we can apply the LTP: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

233 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Pr(mobilized) = 0.6 and so Pr( not mobilized) = 0.4 Note that mobilization partitions the data. Everyone is either mobilized or not. Thus, we can apply the LTP: Pr(vote) = Pr(vote mobilized) Pr(mobilized)+ Pr(vote not mobilized) Pr(not mobilized) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

234 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Pr(mobilized) = 0.6 and so Pr( not mobilized) = 0.4 Note that mobilization partitions the data. Everyone is either mobilized or not. Thus, we can apply the LTP: Pr(vote) = Pr(vote mobilized) Pr(mobilized)+ Pr(vote not mobilized) Pr(not mobilized) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

235 Example: Voter Mobilization Suppose that we have put together a voter mobilization campaign and we want to know what the probability of voting is after the campaign: Pr[vote]. We know the following: Pr(vote mobilized) = 0.75 Pr(vote not mobilized) = 0.15 Pr(mobilized) = 0.6 and so Pr( not mobilized) = 0.4 Note that mobilization partitions the data. Everyone is either mobilized or not. Thus, we can apply the LTP: Pr(vote) = Pr(vote mobilized) Pr(mobilized)+ Pr(vote not mobilized) Pr(not mobilized) = =.51 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

236 Bayes Rule Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

237 Bayes Rule Often we have information about Pr(B A), but require Pr(A B) instead. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

238 Bayes Rule Often we have information about Pr(B A), but require Pr(A B) instead. When this happens, always think: Bayes rule Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

239 Bayes Rule Often we have information about Pr(B A), but require Pr(A B) instead. When this happens, always think: Bayes rule Bayes rule: if Pr(B) > 0, then: Pr(A B) = Pr(B A) Pr(A) Pr(B) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

240 Bayes Rule Often we have information about Pr(B A), but require Pr(A B) instead. When this happens, always think: Bayes rule Bayes rule: if Pr(B) > 0, then: Pr(A B) = Pr(B A) Pr(A) Pr(B) Proof: combine the multiplication rule Pr(B A) Pr(A) = P(A B), and the definition of conditional probability Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

241 Bayes Rule Mechanics )posed... ) I Pl = birds go.nu. areas aired aimed Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

242 Bayes Rule Mechanics )posed... ) I Pl = birds go.nu. areas aired aimed Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

243 Bayes Rule Mechanics )posed... ) I Pl = birds go.nu. areas aired aimed Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

244 Bayes Rule Mechanics )posed... ) I Pl = birds go.nu. areas aired aimed Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

245 Bayes Rule Example b.. Baird.. am r g. B... p. ) 76.5% of female their - billionaires inherited Women Men. fortunes, compared 24.5% of male billionaires So is p( woman ) inherited billions ) than greater Plmanlinherited billions )? to Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

246 Bayes Rule Example ) % of female - billionaires inherited Women Men. fortunes, compared to their 24.5% of male billionaires So is P( woman ) inherited billions ), than greater Plmanlinherited billions ) ei ml tet. ad.q.boibboioaaa.r@g.zaoap.o.76.s P( WII )=P(I1W)P(w)P( I ) = } * Data source : Billionaires characteristics database Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

247 Example: Race and Names Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

248 Example: Race and Names Enos (2015): how do we identify a person s race from their name? Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

249 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

250 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

251 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Pr(AfAm) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

252 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Pr(AfAm) = Pr(not AfAm) = 1 Pr(AfAm) =.868 Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

253 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Pr(AfAm) = Pr(not AfAm) = 1 Pr(AfAm) =.868 Pr(Washington AfAm) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

254 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Pr(AfAm) = Pr(not AfAm) = 1 Pr(AfAm) =.868 Pr(Washington AfAm) = Pr(Washington not AfAm) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

255 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Pr(AfAm) = Pr(not AfAm) = 1 Pr(AfAm) =.868 Pr(Washington AfAm) = Pr(Washington not AfAm) = We can now use Bayes Rule Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

256 Example: Race and Names Enos (2015): how do we identify a person s race from their name? First, note that the Census collects information on the distribution of names by race. For example, Washington is the most common last name among African-Americans in America: Pr(AfAm) = Pr(not AfAm) = 1 Pr(AfAm) =.868 Pr(Washington AfAm) = Pr(Washington not AfAm) = We can now use Bayes Rule Pr(AfAm Wash) = Pr(Wash AfAm) Pr(AfAm) Pr(Wash) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

257 Example: Race and Names Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

258 Example: Race and Names Note we don t have the probability of the name Washington. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

259 Example: Race and Names Note we don t have the probability of the name Washington. Remember that we can calculate it from the LTP since the sets African-American and not African-American partition the sample space: Pr(Wash AfAm) Pr(AfAm) Pr(Wash) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

260 Example: Race and Names Note we don t have the probability of the name Washington. Remember that we can calculate it from the LTP since the sets African-American and not African-American partition the sample space: Pr(Wash AfAm) Pr(AfAm) Pr(Wash) = Pr(Wash AfAm) Pr(AfAm) Pr(Wash AfAm) Pr(AfAm) + Pr(Wash not AfAm) Pr(not AfAm) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

261 Example: Race and Names Note we don t have the probability of the name Washington. Remember that we can calculate it from the LTP since the sets African-American and not African-American partition the sample space: Pr(Wash AfAm) Pr(AfAm) Pr(Wash) Pr(Wash AfAm) Pr(AfAm) = Pr(Wash AfAm) Pr(AfAm) + Pr(Wash not AfAm) Pr(not AfAm) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

262 Example: Race and Names Note we don t have the probability of the name Washington. Remember that we can calculate it from the LTP since the sets African-American and not African-American partition the sample space: Pr(Wash AfAm) Pr(AfAm) Pr(Wash) Pr(Wash AfAm) Pr(AfAm) = Pr(Wash AfAm) Pr(AfAm) + Pr(Wash not AfAm) Pr(not AfAm) = Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

263 Independence Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

264 Independence Intuitive Definition Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

265 Independence Intuitive Definition Events A and B are independent if knowing whether A occurred provides no information about whether B occurred. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

266 Independence Intuitive Definition Events A and B are independent if knowing whether A occurred provides no information about whether B occurred. Formal Definition Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

267 Independence Intuitive Definition Events A and B are independent if knowing whether A occurred provides no information about whether B occurred. Formal Definition P(A, B) = P(A)P(B) = A B Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

268 Independence Intuitive Definition Events A and B are independent if knowing whether A occurred provides no information about whether B occurred. Formal Definition P(A, B) = P(A)P(B) = A B With all the usual > 0 restrictions, this implies Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

269 Independence Intuitive Definition Events A and B are independent if knowing whether A occurred provides no information about whether B occurred. Formal Definition P(A, B) = P(A)P(B) = A B With all the usual > 0 restrictions, this implies P(A B) = P(A) P(B A) = P(B) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

270 Independence Intuitive Definition Events A and B are independent if knowing whether A occurred provides no information about whether B occurred. Formal Definition P(A, B) = P(A)P(B) = A B With all the usual > 0 restrictions, this implies P(A B) = P(A) P(B A) = P(B) Independence is a massively important concept in statistics. Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

271 Next Week Random Variables Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

272 Next Week Random Variables Reading Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

273 Next Week Random Variables Reading Aronow and Miller (1.1) on Random Events Aronow and Miller ( ) on Probability Theory, Summarizing Distributions Optional: Blitzstein and Hwang Chapters (probability), (conditional probability), (random variables), (expectation), (indicator rv, LOTUS, variance), (joint distributions) Optional: Imai Chapter 6 (probability) Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

274 Next Week Random Variables Reading Aronow and Miller (1.1) on Random Events Aronow and Miller ( ) on Probability Theory, Summarizing Distributions Optional: Blitzstein and Hwang Chapters (probability), (conditional probability), (random variables), (expectation), (indicator rv, LOTUS, variance), (joint distributions) Optional: Imai Chapter 6 (probability) A word from your preceptors Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

275 Fun With Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

276 Fun with Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

277 Fun with History Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

278 Fun with History Legendre Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

279 Fun with History Gauss Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

280 Fun with History Quetelet Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62

281 Fun with History Galton Stewart (Princeton) Week 1: Introduction and Probability September 14, / 62