Statistical Theory; Why is the Gaussian Distribution so popular?
|
|
|
- Chloe Gray
- 5 years ago
- Views:
Transcription
1 Statistical Theory; Why is the Gaussia Distributio so popular? Rob Nicholls MRC LMB Statistics Course 2014
2 Cotets Cotiuous Radom Variables Expectatio ad Variace Momets The Law of Large Numbers (LLN) The Cetral Limit Theorem (CLT)
3 Cotiuous Radom Variables A Radom Variable is a object whose value is determied by chace, i.e. radom evets Probability that the radom variable X adopts a particular value x: " $ P(X = x) = # $ % $ p [0,1] : discrete : cotiuous X 0 X
4 Cotiuous Radom Variables Cotiuous Uiform Distributio: Probability Desity Fuctio: $ & f X (x) = % & '& X ~ U(a, b) (b a) 1 x [a, b] 0 x [a, b]
5 Cotiuous Radom Variables Example: X ~ U(0,1) P(X [0,1]) =1
6 Cotiuous Radom Variables Example: X ~ U(0,1) P(X [0, 1 2 ]) = 1 2
7 Cotiuous Radom Variables Example: X ~ U(0,1) P(X [0, 1 3 ]) = 1 3
8 Cotiuous Radom Variables Example: X ~ U(0,1) P(X [0, 1 10 ]) = 1 10
9 Cotiuous Radom Variables Example: X ~ U(0,1) P(X [0, ]) = 1 100
10 Cotiuous Radom Variables Example: X ~ U(0,1) P(X [0, 1 ]) = 1 Lim Lim P(X [0, 1 ]) = 0 P(0 X 1 ) = 0 I geeral, for ay cotiuous radom variable X: Lim ε 0 P(0 X ε) = 0 Lim ε 0 P(α X α +ε) = 0
11 Cotiuous Radom Variables P(X = x) =! # " # $ # : discrete p X (x) X 0 X : cotiuous Why do I observe a value if there s o probability of observig it?! Aswers: Data are discrete You do t actually observe the value precisio error Some value must occur eve though the probability of observig ay particular value is ifiitely small
12 Cotiuous Radom Variables For a radom variable: The Cumulative Distributio Fuctio (CDF) is defied as: (discrete/cotiuous) Properties: No-decreasig Lim F X (x) = 0 x Lim F X (x) =1 x + X :Ω A F X (x) = P(X x)
13 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: f X (x) F X (x) = P(X x) Boxplot:
14 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: f X (x) F X (x) = P(X x) Boxplot:
15 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x f X (y)dy
16 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
17 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
18 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
19 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
20 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
21 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
22 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
23 Cotiuous Radom Variables Probability Desity fuctio: Cumulative Distributio fuctio: Boxplot: f X (x) F X (x) = P(X x) F X (x) = x d dx F X (x) = f X (x) f X (y)dy
24 Cotiuous Radom Variables Cumulative Distributio Fuctio (CDF): Discrete: Cotiuous: F X (x) = P(X x) = Probability Desity Fuctio (PDF): Discrete: Cotiuous: F X (x) = P(X x) = p X (x) = P(X = x) f X (x) = d dx F X (x) x x y= p X (y) f X (y)dy x= f X (x)dx =1 p X (x) =1
25 Expectatio ad Variace Motivatioal Example: Experimet o Plat Growth (ibuilt R dataset) - compares yields obtaied uder differet coditios
26 Expectatio ad Variace Motivatioal Example: Experimet o Plat Growth (ibuilt R dataset) - compares yields obtaied uder differet coditios Compare meas to test for differeces Cosider variace (ad shape) of the distributios help choose appropriate prior/protocol Assess ucertaity of parameter estimates allow hypothesis testig
27 Expectatio ad Variace Motivatioal Example: Experimet o Plat Growth (ibuilt R dataset) - compares yields obtaied uder differet coditios Compare meas to test for differeces Cosider variace (ad shape) of the distributios help choose appropriate prior/protocol Assess ucertaity of parameter estimates allow hypothesis testig I order to do ay of this, we eed to kow how to describe distributios i.e. we eed to kow how to work with descriptive statistics
28 Expectatio ad Variace Discrete RV: Sample (empirical): (explicit weightig ot required) Cotiuous RV: E(X) = E(X) = E(X) = 1 xp X (x) x i xf X (x) dx
29 Expectatio ad Variace Normal Distributio: X ~ N(µ,σ 2 ) f X (x) = 1 2πσ e ( x µ ) 2 2σ 2
30 Expectatio ad Variace Normal Distributio: X ~ N(µ,σ 2 ) f X (x) = 1 2πσ e x µ ( ) 2 2σ 2 E(X) = xf X (x) dx = µ
31 Expectatio ad Variace Stadard Cauchy Distributio: (also called Loretz) f X (x) = 1 1 π 1+ x 2 E(X) = xf X (x) dx = udefied
32 Expectatio ad Variace Expectatio of a fuctio of radom variables: E(g(X)) = g(x) f X (x) dx Liearity: E(αX + β) = (αx + β) f X (x) dx = α xf X (x) dx + β f X (x) dx = αe(x)+ β
33 Expectatio ad Variace Variace: Var(X) = E ((X µ) 2 ) = E ((X E(X)) 2 ) = E(X 2 ) E(X) 2 X ~ N(0,1)
34 Expectatio ad Variace Variace: Var(X) = E ((X µ) 2 ) = E ((X E(X)) 2 ) = E(X 2 ) E(X) 2 X ~ N(0,1)
35 Expectatio ad Variace Variace: Var(X) = E ((X µ) 2 ) = E ((X E(X)) 2 ) = E(X 2 ) E(X) 2 X ~ N(0,1) X ~ N(0,2)
36 Expectatio ad Variace Variace: Var(X) = E ((X µ) 2 ) = E ((X E(X)) 2 ) = E(X 2 ) E(X) 2 X ~ N(0,1) Populatio Variace: σ 2 = 1 Ubiased Sample Variace: s 2 = 1 1 ( x i µ ) 2 ( x i x) 2 X ~ N(0,2)
37 Expectatio ad Variace Variace: Var(X) = E(X 2 ) E(X) 2 No-liearity: Stadard deviatio (s.d.): Var(X) Var( αx + β) = E ((αx + β) 2 ) E ( αx + β) 2
38 Expectatio ad Variace Variace: Var(X) = E(X 2 ) E(X) 2 No-liearity: Stadard deviatio (s.d.): Var(X) Var( αx + β) = E ((αx + β) 2 ) E ( αx + β) 2 = E ( α 2 X 2 + 2αβX + β 2 ) ( αe(x)+ β) 2
39 Expectatio ad Variace Variace: Var(X) = E(X 2 ) E(X) 2 No-liearity: Stadard deviatio (s.d.): Var(X) Var( αx + β) = E ((αx + β) 2 ) E ( αx + β) 2 = E ( α 2 X 2 + 2αβX + β 2 ) ( αe(x)+ β) 2 = ( α 2 E(X 2 )+ 2αβE(X)+ β 2 ) ( α 2 E(X) 2 + 2αβE(X)+ β 2 )
40 Expectatio ad Variace Variace: Var(X) = E(X 2 ) E(X) 2 No-liearity: Stadard deviatio (s.d.): Var(X) Var( αx + β) = E ((αx + β) 2 ) E ( αx + β) 2 = E ( α 2 X 2 + 2αβX + β 2 ) ( αe(x)+ β) 2 = ( α 2 E(X 2 )+ 2αβE(X)+ β 2 ) ( α 2 E(X) 2 + 2αβE(X)+ β 2 )
41 Expectatio ad Variace Variace: Var(X) = E(X 2 ) E(X) 2 No-liearity: Stadard deviatio (s.d.): Var(X) Var( αx + β) = E ((αx + β) 2 ) E ( αx + β) 2 = E ( α 2 X 2 + 2αβX + β 2 ) ( αe(x)+ β) 2 = ( α 2 E(X 2 )+ 2αβE(X)+ β 2 ) ( α 2 E(X) 2 + 2αβE(X)+ β 2 ) = α 2 ( E(X 2 ) E(X) 2 ) = α 2 Var(X)
42 Expectatio ad Variace Ofte data are stadardised/ormalised Z-score/value: Example: Z = X µ σ X ~ N(µ,σ 2 ) f X (x) = 1 2πσ e ( x µ ) 2 2σ 2 Z ~ N(0,1) f Z (x) = 1 2π e x2 2
43 Momets Shape descriptors Li ad Hartley (2006) Computer Visio Saupe ad Vraic (2001) Spriger
44 Momets Shape descriptors Li ad Hartley (2006) Computer Visio Saupe ad Vraic (2001) Spriger
45 Momets Shape descriptors Li ad Hartley (2006) Computer Visio Saupe ad Vraic (2001) Spriger
46 Momets Momets provide a descriptio of the shape of a distributio Raw momets Cetral momets Stadardised momets µ 1 = E(X) = µ : mea E (X µ) 2 ( ) = σ 2 : variace 1 (( σ ) 3 ) E X µ : skewess (( σ ) 4 ) E X µ : kurtosis µ = E(X ) E (X µ) ( ) ( ) E ( X µ σ )
47 Momets Stadard Normal: Stadard Log-Normal:
48 Momets Momet geeratig fuctio (MGF): M X (t) = E(e Xt ) =1+ te(x)+ t 2 2! E(X 2 ) t! E(X )+... Alterative represetatio of a probability distributio. µ = E(X ) = d dt M (0) X
49 Momets Momet geeratig fuctio (MGF): M X (t) = E(e Xt ) =1+ te(x)+ t 2 2! E(X 2 ) t! E(X )+... Alterative represetatio of a probability distributio. µ = E(X ) = d dt M (0) X Example: X ~ N(µ,σ 2 ) M X (t) = e tµ+1 2 σ 2 t 2 X ~ N(0,1) M X (t) = e 1 2 t2
50 Momets However, MGF oly exists if E(X ) exists M X (t) = E(e Xt ) Characteristic fuctio always exists: ϕ X (t) = M ix (t) = M X (it) = E(e itx ) = e itx f X (x)dx Related to the probability desity fuctio via Fourier trasform Example: X ~ N(0,1) ϕ X (t) = e t2 2
51 The Law of Large Numbers (LLN) Motivatioal Example: Experimet o Plat Growth (ibuilt R dataset) - compare yields obtaied uder differet coditios Wat to estimate the populatio mea usig the sample mea. How ca we be sure that the sample mea reliably estimates the populatio mea?
52 The Law of Large Numbers (LLN) Does the sample mea reliably estimate the populatio mea? The Law of Large Numbers: X = 1 X i ### µ Providig X i : i.i.d.
53 The Law of Large Numbers (LLN) Does the sample mea reliably estimate the populatio mea? The Law of Large Numbers: X = 1 X i ### µ Providig X i : i.i.d. X ~ U(0,1) µ = 0.5
54 The Cetral Limit Theorem (CLT) Questio - give a particular sample, thus kow sample mea, how reliable is the sample mea as a estimator of the populatio mea? Furthermore, how much will gettig more data improve the estimate of the populatio mea? Related questio - give that we wat the estimate of the mea to have a certai degree of reliability (i.e. sufficietly low S.E.), how may observatios do we eed to collect? The Cetral Limit Theorem helps aswer these questios by lookig at the distributio of stochastic fluctuatios about the mea as
55 The Cetral Limit Theorem (CLT) The Cetral Limit Theorem states: For large : Or equivaletly: More formally: Coditios: E(X i ) = µ Var(X i ) = σ 2 < X i ( X µ ) ~ N(0,σ 2 )! X ~ N µ, σ 2 $ # & " % # 1 & d % X i µ ( ) ) N(0,σ 2 ) $ ' : i.i.d. RVs (ay distributio)
56 The Cetral Limit Theorem (CLT) The Cetral Limit Theorem states: For large : ( ) ~ N(0,σ 2 ) X ~ N µ, σ 2 # X µ X ~ U(0,1) µ = 0.5 σ 2 = 1 12! " $ & %
57 The Cetral Limit Theorem (CLT) The Cetral Limit Theorem states: For large : ( ) ~ N(0,σ 2 ) X ~ N µ, σ 2 # X µ X ~ U(0,1) µ = 0.5 σ 2 = 1 12! " $ & %
58 The Cetral Limit Theorem (CLT) The Cetral Limit Theorem states: For large : ( ) ~ N(0,σ 2 ) X ~ N µ, σ 2 # X µ X ~ U(0,1) µ = 0.5 σ 2 = 1 12! " $ & %
59 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: # 1 & d % X i µ ( ) ) N(0,σ 2 ) $ '
60 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: # 1 & d % X i µ ( ) ) N(0,σ 2 ) $ ' 1 # % $ X i µ & ( ' d ) ) N(0,σ 2 )
61 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: σ # 1 & d % X i µ ( ) ) N(0,σ 2 ) $ ' 1 1 # % $ # % $ X i µ & ( ' X i µ d ) ) N(0,σ 2 ) & ( ' d ) ) N(0,1)
62 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: σ # 1 & d % X i µ ( ) ) N(0,σ 2 ) $ ' # % $ # % $ X i µ & ( ' X i µ d ) ) N(0,σ 2 ) & ( ' d ) ) N(0,1) " X i µ % d $ ' ) ) N(0,1) # σ &
63 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: σ # 1 & d % X i µ ( ) ) N(0,σ 2 ) $ ' # % $ # % $ X i µ Z i & ( ' X i µ d ) ) N(0,σ 2 ) & ( ' d ) ) N(0,1) " X i µ % d $ ' ) ) N(0,1) # σ & d " " N(0,1) Z i = X i µ σ
64 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: Z i d " " N(0,1)
65 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: Z i d " " N(0,1) E(e i! # " Z $ &t ) = E(e iz %! # " t $ & % ) ϕ (t) = ϕ Zi Z i " $ # t % ' &
66 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: Z i d " " N(0,1) ϕ (t) = ϕ " Zi $ Z i # t % ' & E(e i! # " Z $ &t ) = E(e iz %! # " t $ & % ) E(e it ( Z 1+Z ) 2 ) = E(e itz 1 )E(eitZ 2 )! t = ϕ Zi # " $ & %
67 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: Z i d " " N(0,1) ϕ (t) = ϕ " Zi $ Z i # t % ' & E(e i! # " Z $ &t ) = E(e iz % t $ & % ) E(e it ( Z 1+Z ) 2 ) = E(e itz 1 )E(eitZ 2 )! # "! t = ϕ Zi # " $ & %!! t = # ϕ Z # " " $ $ && %%
68 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: Z i d " " N(0,1) ϕ (t) = ϕ " Zi $ Z i # t % ' & E(e i! # " Z $ &t ) = E(e iz % t $ & % ) E(e it ( Z 1+Z ) 2 ) = E(e itz 1 )E(eitZ 2 )! # "! t $ = ϕ Zi # & " %!! t $ $ = # ϕ Z # && " " %% " = 1 t o " t 2 $ $ # # %% '' &&
69 The Cetral Limit Theorem (CLT) Proof of the Cetral Limit Theorem: Z i d " " N(0,1) ϕ (t) = ϕ " Zi $ Z i # t % ' &! t $ = ϕ Zi # & " %!! t $ $ = # ϕ Z # && " " %% " = 1 t o " t 2 %% $ $ '' # # && e x = Lim!! e t2 2 = characteristic fuctio of a E(e i! # " Z $ &t ) = E(e iz %! # " # 1+ x & % ( $ ' N(0,1) t $ & % ) E(e it ( Z 1+Z ) 2 ) = E(e itz 1 )E(eitZ 2 )
70 Summary Cosidered how: Probability Desity Fuctios (PDFs) ad Cumulative Distributio Fuctios (CDFs) are related, ad how they differ i the discrete ad cotiuous cases Expectatio is at the core of Statistical theory, ad Momets ca be used to describe distributios The Cetral Limit Theorem idetifies how/why the Normal distributio is fudametal The Normal distributio is also popular for other reasos: Maximum etropy distributio (give mea ad variace) Itrisically related to other distributios (t, F, χ 2, Cauchy, ) Also, it is easy to work with
71 Refereces Coutless books + olie resources! Probability ad Statistical theory: Grimmett ad Stirzker (2001) Probability ad Radom Processes. Oxford Uiversity Press. Geeral comprehesive itroductio to (almost) everythig mathematics: Garrity (2002) All the mathematics you missed: but eed to kow for graduate school. Cambridge Uiversity Press.
This section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
AMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
Distribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
Parameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
Random Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
![Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].](/thumbs/90/101606445.jpg "Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].")
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
Lecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
Topic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
Topic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
Lecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
STAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
Expectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
Lecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
AMS 216 Stochastic Differential Equations Lecture 02 Copyright by Hongyun Wang, UCSC ( ( )) 2 = E X 2 ( ( )) 2
AMS 216 Stochastic Differetial Equatios Lecture 02 Copyright by Hogyu Wag, UCSC Review of probability theory (Cotiued) Variace: var X We obtai: = E X E( X ) 2 = E( X 2 ) 2E ( X )E X var( X ) = E X 2 Stadard
LECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
Discrete Probability Functions
Discrete Probability Fuctios Daiel B. Rowe, Ph.D. Professor Departmet of Mathematics, Statistics, ad Computer Sciece Copyright 017 by 1 Outlie Discrete RVs, PMFs, CDFs Discrete Expectatios Discrete Momets
7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
Topic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
Statisticians use the word population to refer the total number of (potential) observations under consideration
6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space
7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
Mathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
Estimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
EE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
Hypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance
Hypothesis Testig Empirically evaluatig accuracy of hypotheses: importat activity i ML. Three questios: Give observed accuracy over a sample set, how well does this estimate apply over additioal samples?
Advanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
Discrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
Exam II Review. CEE 3710 November 15, /16/2017. EXAM II Friday, November 17, in class. Open book and open notes.
Exam II Review CEE 3710 November 15, 017 EXAM II Friday, November 17, i class. Ope book ad ope otes. Focus o material covered i Homeworks #5 #8, Note Packets #10 19 1 Exam II Topics **Will emphasize material
Lecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
Lecture 2: Poisson Sta*s*cs Probability Density Func*ons Expecta*on and Variance Es*mators
Lecture 2: Poisso Sta*s*cs Probability Desity Fuc*os Expecta*o ad Variace Es*mators Biomial Distribu*o: P (k successes i attempts) =! k!( k)! p k s( p s ) k prob of each success Poisso Distributio Note
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
Notes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
4. Basic probability theory
Cotets Basic cocepts Discrete radom variables Discrete distributios (br distributios) Cotiuous radom variables Cotiuous distributios (time distributios) Other radom variables Lect04.ppt S-38.45 - Itroductio
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
SDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
Elements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
Lecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
HOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
Parameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
Quick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig
Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
Quick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.
Module 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
The central limit theorem for Student s distribution. Problem Karim M. Abadir and Jan R. Magnus. Econometric Theory, 19, 1195 (2003)
The cetral limit theorem for Studet s distributio Problem 03.6.1 Karim M. Abadir ad Ja R. Magus Ecoometric Theory, 19, 1195 (003) Z Ecoometric Theory, 19, 003, 1195 1198+ Prited i the Uited States of America+
Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
Probability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
Convergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
Stat 400: Georgios Fellouris Homework 5 Due: Friday 24 th, 2017
Stat 400: Georgios Fellouris Homework 5 Due: Friday 4 th, 017 1. A exam has multiple choice questios ad each of them has 4 possible aswers, oly oe of which is correct. A studet will aswer all questios
Frequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
Review Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
Probability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
Lecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
Central Limit Theorem using Characteristic functions
Cetral Limit Theorem usig Characteristic fuctios RogXi Guo MAT 477 Jauary 20, 2014 RogXi Guo (2014 Cetral Limit Theorem usig Characteristic fuctios Jauary 20, 2014 1 / 15 Itroductio study a radom variable
Lecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
f(x)dx = 1 and f(x) 0 for all x.
OCR Statistics 2 Module Revisio Sheet The S2 exam is 1 hour 30 miutes log. You are allowed a graphics calculator. Before you go ito the exam make sureyou are fully aware of the cotets of theformula booklet
Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
An Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
The standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
Lecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
STAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
IE 230 Seat # Name < KEY > Please read these directions. Closed book and notes. 60 minutes.
IE 230 Seat # Name < KEY > Please read these directios. Closed book ad otes. 60 miutes. Covers through the ormal distributio, Sectio 4.7 of Motgomery ad Ruger, fourth editio. Cover page ad four pages of
Lecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
Probability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
Binomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
Chapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS
NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS STRUCTURE OF EXAMINATION PAPER. There will be oe 2-hour paper cosistig of 4 questios.
Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
![Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.](/thumbs/91/105380228.jpg "Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.")
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
Direction: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
Lecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
Lecture 5. Random variable and distribution of probability
Itroductio to theory of probability ad statistics Lecture 5. Radom variable ad distributio of probability prof. dr hab.iż. Katarzya Zarzewsa Katedra Eletroii, AGH e-mail: za@agh.edu.pl http://home.agh.edu.pl/~za
Direction: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
[ ] ( ) ( ) [ ] ( ) 1 [ ] [ ] Sums of Random Variables Y = a 1 X 1 + a 2 X 2 + +a n X n The expected value of Y is:
![[ ] ( ) ( ) [ ] ( ) 1 [ ] [ ] Sums of Random Variables Y = a 1 X 1 + a 2 X 2 + +a n X n The expected value of Y is:](/thumbs/89/98900182.jpg "[ ] ( ) ( ) [ ] ( ) 1 [ ] [ ] Sums of Random Variables Y = a 1 X 1 + a 2 X 2 + +a n X n The expected value of Y is:")
PROBABILITY FUNCTIONS A radom variable X has a probabilit associated with each of its possible values. The probabilit is termed a discrete probabilit if X ca assume ol discrete values, or X = x, x, x 3,,
Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;
Lecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
Chapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
Lecture 4. Random variable and distribution of probability
Itroductio to theory of probability ad statistics Lecture. Radom variable ad distributio of probability dr hab.iż. Katarzya Zarzewsa, prof.agh Katedra Eletroii, AGH e-mail: za@agh.edu.pl http://home.agh.edu.pl/~za
STATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
IE 230 Probability & Statistics in Engineering I. Closed book and notes. No calculators. 120 minutes.
Closed book ad otes. No calculators. 120 miutes. Cover page, five pages of exam, ad tables for discrete ad cotiuous distributios. Score X i =1 X i / S X 2 i =1 (X i X ) 2 / ( 1) = [i =1 X i 2 X 2 ] / (
Discrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
Chapter 13: Tests of Hypothesis Section 13.1 Introduction
Chapter 13: Tests of Hypothesis Sectio 13.1 Itroductio RECAP: Chapter 1 discussed the Likelihood Ratio Method as a geeral approach to fid good test procedures. Testig for the Normal Mea Example, discussed
1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
Statistical inference: example 1. Inferential Statistics
Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either
PRACTICE PROBLEMS FOR THE FINAL
PRACTICE PROBLEMS FOR THE FINAL Math 36Q Fall 25 Professor Hoh Below is a list of practice questios for the Fial Exam. I would suggest also goig over the practice problems ad exams for Exam ad Exam 2 to
Lecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
Lecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
Notation List. For Cambridge International Mathematics Qualifications. For use from 2020
Notatio List For Cambridge Iteratioal Mathematics Qualificatios For use from 2020 Notatio List for Cambridge Iteratioal Mathematics Qualificatios (For use from 2020) Mathematical otatio Eamiatios for CIE
Stat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
Lecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.