Time series models 2007

Size: px
Start display at page:

Download "Time series models 2007"

Transcription

1 Norwegia Uiversity of Sciece ad Techology Departmet of Mathematical Scieces Solutios to problem sheet 1, 2007 Exercise 1.1 a Let Sc = E[Y c 2 ]. The This gives Sc = EY 2 2cEY + c 2 ds dc = 2EY + 2c = 0 for c = EY, which leads to a global miimum sice d2 S dc 2 = 2 > 0 for all c. b E[ Y fx 2 X] = E[ Y EY X + EY X fx 2 X] = E[ Y EY X 2 X] + 2E[ Y EY X EY X fx X] + E[ EY X fx 2 X] = E[ Y EY X 2 X] + 2 X] 2 EY X fx E[ Y EY X X] + E[ EY X fx = E[ Y EY X 2 X] + 2 X] 2 X] E[ EY X fx E[ Y EY X because EY X is a fuctio of X ad EgXY X = gxey X for ay fuctio g such that EgXY exists. It follows that E[ Y EY X 2 X] E[ Y fx 2 X] for ay fuctio f. Hece E[ Y fx 2 X] is miimized whe fx = EY X. c Sice E[ Y EY X 2 ] = E E[ Y EY X 2 X] E E[ Y fx 2 X] = E[ Y fx 2 ] it follows immediately that the radom variable fx that miimizes E[ Y fx 2 ] is fx = EY X. problemsheet1 August 30, 2007 Side 1

2 Exercise 1.2 a Let X = X 1, X 2,..., X. The E[ X +1 fx 2 X] = E[ X+1 EX +1 X + EX +1 X fx 2 X] = E[ X +1 EX +1 X 2 X] + 2E[ X+1 EX +1 X EX +1 X fx X] + E[ EX +1 X fx 2 X] = E[ X +1 EX +1 X 2 X] + 2 EX+1 X fx E[ X +1 EX +1 X X] + E[ EX +1 X fx 2 X] = E[ X +1 EX +1 X 2 X] + E[ EX+1 X fx 2 X] E[ X+1 EX +1 X 2 X] because EX +1 X is a fuctio of X ad EgXX +1 X = gxex +1 X for ay fuctio g such that EgXX +1 exists. It follows that E[ X +1 EX +1 X 2 X] E[ X+1 fx 2 X] for ay fuctio f. Hece E[ X +1 EX +1 X 2 X] is miimized whe fx = EX+1 X. b Sice E[ X +1 EX +1 X 2 ] = E E[ X+1 EX +1 X 2 X] E E[ X +1 fx 2 X] = E[ X+1 fx 2 ] it follows immediately that the radom variable fx that miimizes E[ X +1 fx 2 ] is agai fx = EX +1 X. c By b the miimum mea-squared error predictor of X +1 i terms of X = X 1, X 2,..., X whe X t IIDµ, σ 2 is EX +1 X = EX +1 = µ d Suppose that i=1 α ix i is a ubiased estimator for µ, that is, i=1 α i = 1. The E[ α i X i µ 2 ] = E[ α i X i X 2 ] + 2E[ α i X i X X µ ] + E[ X µ 2 2] ] E[ X µ i=1 i=1 i=1 sice the secod term is zero: E[ i=1 α ix i X X µ ] = Cov i=1 α ix i X, X = Cov i=1 α ix i, i=1 1 X i Cov i=1 1 X i, i=1 1 X i = i=1 α i σ2 i=1 1 σ 2 = 0. 2 problemsheet1 August 30, 2007 Side 2

3 e Agai, suppose that i=1 α ix i is a ubiased estimator for µ, that is, i=1 α i = 1. The E[ X +1 2] α i X i = E[ X+1 X 2 ] + 2E[ X+1 X X i=1 E[ X +1 X 2 ] 2] α i X i ] + E[ X α i X i sice the secod term is zero: Cov X +1 X, X i=1 α ix i = CovX, X+CovX, i=1 α ix i = 0 as i d. i=1 i=1 f E S +1 S 1,..., S = E S + X +1 S 1,..., S = S + E X +1 S 1,..., S = S + µ sice X +1 is idepedet of S 1,..., S. Exercise 1.3 i EX t is idepedet of t sice the distributio of X t is idepedet of t ad EX t exists. ii Sice E[X t+h X t ] 2 E[X 2 t+h ]E[X2 t ] for all itegers t, h, ad the joit distributio of X t+h ad X t is idepedet of t, it follows that E[X t+h X t ] exists ad is idepedet of t for every iteger h. Combiig i ad ii it follows that X t is weakly statioary. Exercise 1.4 a EX t = a is idepedet of t. b Cov 2 + c 2 σ 2 ; h = 0 0 ; h = ±1 X t+h, X t = bcσ 2 ; h = ±2 0 ; h > 2 which is idepedet of t. That is, X t is statioary. b problemsheet1 August 30, 2007 Side 3

4 EX t = 0 is idepedet of t. Cov X t+h, X t = Cov Z1 cos ct + h + Z 2 si ct + h, Z 1 cos ct + Z 2 si ct = σ 2 cos ct + h cos ct + si ct + h si ct = σ 2 cos ch which is idepedet of t. That is, X t is statioary. c EX t = 0 is idepedet of t. Cov X t+1, X t = σ 2 cos ct + 1 si ct which is ot idepedet of t. That is, X t is ot statioary except i the special case whe c is a iteger multiple of 2π. d EX t = a is idepedet of t. Cov X t+h, X t = b 2 σ 2 which is idepedet of t. That is, X t is statioary. e EX t = 0 is idepedet of t. Cov X t+h, X t = σ 2 cos ct + h cos ct which is ot idepedet of t. That is, X t is ot statioary except i the special case whe c is a iteger multiple of 2π. f EX t = 0 is idepedet of t. Cov X t+h, X t = E[Xt+h X t ] = E[Z t+h Z t+h 1 Z t Z t 1 ] = { σ 4 ; h = 0 0 ; h > 0 which is idepedet of t. That is, X t is statioary, ad it is see that i fact X t W N0, σ 4. problemsheet1 August 30, 2007 Side 4

5 Exercise 1.5 a The autocovariace fuctio The autocorrelatio fuctio For θ = 0.8 it is obtaied that 1 + θ 2 ; h = 0 γ X h = θ ; h = ±2 0 ; otherwise 1 ; h = 0 θ ρ X h = ; h = ±2 1+θ 2 0 ; otherwise 1.64 ; h = 0 γ X h = 0.8 ; h = ±2 0 ; otherwise 1 ; h = 0 ρ X h = ; h = ±2 0 ; otherwise b Let X 4 = 1 4 X X 4. The V arx 4 = Cov X 4, X 4 = i=1 i=1 4 Cov X i, X j = 1 γx 0 + γ X 2 = = c V arx 4 = Cov 1 X 4, X 4 = γx 0 + γ X 2 4 = = The egative lag 2 correlatio i c meas that positive deviatios of X t from zero ted to be followed two time uits later by a compesatig egative deviatio, resultig i smaller variability i the sample mea tha i b ad also smaller tha if the time series X t were IID0, 1.64 i which case V arx 4 = problemsheet1 August 30, 2007 Side 5

6 Exercise 1.15 a Sice s t has period 12 so that The E[W t ] = 0 ad 12 X t = 12 a + bt + s t + Y t = 12b + Y t Y t 12 W t := 12 X t = Y t Y t 1 Y t 12 Y t 13. Cov[W t+h, W t ] = Cov[Y t+h Y t+h 1 Y t+h 12 Y t+h 13, Y t Y t 1 Y t 12 Y t 13 ] = 4γh 2γh 1 2γh γh 11 + γh γh 12 2γh γh γh 13 where γ is the ACVF of Y t. Sice E[W t ] ad Cov[W t+h, W t ] are idepedet of t, W t is statioary. Also ote that 12 X t is statioary. b Usig X t = a + bts t + Y t it is obtaied that 12 X t = bts t bt 12s t 12 + Y t Y t 12 = 12bs t 12 + Y t Y t 12. Now let U t = 2 12 X t = Y t 2Y t 12 + Y t 24. The E[U t ] = 0 ad Cov[U t+h, U t ] = Cov[Y t+h 2Y t+h 12 + Y t+h 24, Y t 2Y t 12 + Y t 24 ] = 6γh 4γh γh 12 + γh γh 24, which is idepedet of t. Hece U t is statioary. Exercise 2.1 Sa, b = E[X +h ax b 2 ] to be miimized wrt a ad b. Now Sa, b = E[ X +h µ ax µ b aµ + µ ] = γ0 + a 2 γ0 + b + aµ µ 2 2aγh This gives S = 2aγ0 + 2µb + aµ µ 2γh a S = 2b + aµ µ b S is clearly miimized wrt b whe for b = µ1 a. Substitutig this value ito S a ad equatig to zero leads to the result a = γh γ0 = ρh Hece, Sa, b is miimized whe a = ρh, b = µ1 ρh The BLP best liear predictor of X +h i terms of X is therefore µ + ρhx µ. problemsheet1 August 30, 2007 Side 6

7 Exercise 2.3 a X t = Z t + 0.3Z t 1 0.4Z t 2 γ0 = = 1.25 γ1 = = 0.18 γ2 = 0.4 γh = 0, h > 2 γ h = γh b Y t = Z t 1.2 Z t Z t 2 γ0 = = 1.25 γ1 = = 0.18 γ2 = = 0.4 γh = 0, h > 2 γ h = γh That is, we obtai the same ACVF as i a. Exercise 2.5 j=1 θj X j coverges absolutely with probability 1 sice E[ θ j X j ] j=1 θ j E[ X j ] j=1 θ j γ0 + µ 2 j=1 < sice θ < 1 by Cauchy-Schwartz iequality That is, j=1 θ j X j < with probability 1. Mea square covergece of S m = m j=1 θj X j as m ca be veried by ivokig Cauchy's criterio. For m > k E[ S m S k 2 ] = E[ θ j X j 2 ] = j=k+1 i=k+1 j=k+1 θ i+j E[X i X j ] problemsheet1 August 30, 2007 Side 7

8 E[ S m S k 2 ] = E[ = j=k+1 i=k+1 j=k+1 i=k+1 j=k+1 θ j X j 2 ] = 0 as k, m i=k+1 j=k+1 θ i+j γi j + µ 2 θ i+j E[X i X j ] θ i+j γ0 + µ 2 = γ0 + µ 2 m j=k+1 θ j 2 sice j=1 θ j <. Hece, by Cauchy's mutual covergece criterio, mea square covergece is guarateed. Exercise 2.7 sice φz > φz = 1 φz 1 1 φz = φz φz + 1 φz = φz j j=1 Exercise 2.8 X t = φx t 1 + Z t X t = φx t 1 + Z t = Z t + φz t 1 + φx t 2 =... = Z t + φz t φ Z t + φ +1 X t 1 That is X t φ +1 X t 1 = Z t + φz t φ Z t First we calculate VarX t φ +1 X t 1 = γ01 + φ 2+2 2φ +1 γ + 1 γ01 + φ φ +1 = 4γ0 problemsheet1 August 30, 2007 Side 8

9 if X t is statioary ad φ = 1 Next we calculate VarZ t + φz t φ Z t = σ 2 if φ = 1 Sice clearly σ 2 > 4γ0 for sucietly large, we have reached a cotradictio. Hece X t caot be statioary if φ = 1. Exercise 2.10 X t φx t 1 = Z t + θz t 1 where φ = θ = 0.5 Accordig to Sectio 2.3, equatio 2.3.3, we obtai that X t = ψ j Z t j j=0 where ψ 0 = 1, ψ j = φ + θφ j 1 = 0.5 j 1 for j = 1, 2,.... >From Sectio 2.3, equatio 2.3.5, we get Z t = π j X t j where π 0 = 1, π j = φ + θ θ j 1 = 0.5 j 1 for j = 1, 2,.... Agrees with the results from ITSM. j=0 Exercise 2.12 The give MA1-model is where Z t WN0, 1. Observed that x 100 = The variace of x 100 : X t = Z t 0.6Z t 1 Var[x 100 ] = 1 1 h γh h= = 1 99 γ γ1 = = problemsheet1 August 30, 2007 Side 9

10 That is, 95% codece bouds for µ are approximately x 100 ± = ± = ± = 0.076, Reject H 0 : µ = 0 i favour of the alterative hypothesis H 1 : µ 0 at sigicace level 0.05 sice the 95% bouds for µ do ot iclude the value 0. Note: The coclusio would dier if the time series X t IID0, Exercise 2.13 a Assume a AR1-model X t = φx t 1 + Z t Sice ρh = φ h h > 0 for a AR1-model, ad it has bee observed that ρ2 ˆ = 0.145, we shall assume that φ 2 << 1. Usig Bartlett's formula, the followig approximate relatios are obtaied: ad Var[ ρ1] ˆ 1 1 φ 2 Var[ ρ2] ˆ 1 1 φ φ 2 That is, 95% codece bouds for ρ1 are approximately ρ1 ˆ ± φ 2 Correspodigly, 95% codece bouds for ρ2 are approximately ρ2 ˆ ± φ φ 2 With φ = ˆφ = ρ1, ˆ = 100, ρ1 ˆ = 0.438, ρ2 ˆ = 0.145, these bouds become for ρ1: 0.262, 0.614, ad for ρ2: , These values are ot cosistet with φ = 0.8, sice both ρ1 = 0.8 ad ρ2 = 0.64 are outside these bouds. b Assume a MA1-model X t = Z t + θz t 1 Bartlett's formula gives the followig approximate relatios Var[ ρ1] ˆ 1 1 3ρ ρ1 4 problemsheet1 August 30, 2007 Side 10

11 ad Var[ ρ2] ˆ ρ1 2 That is, 95% codece bouds for ρ1 are approximately ρ1 ˆ ± ρ ρ1 4 Correspodigly, 95% codece bouds for ρ2 are approximately ρ2 ˆ ± ρ1 2 With the umbers as i a, it is ow obtaied that these bouds become for ρ1: 0.290, 0.586, ad for ρ2: , θ = 0.6 leads to ρ1 = θ = , ρ2 = 0. It follows that the codece bouds are cosistet 1+θ 2 with these two values, ad the data are therefore cosistet with the MA1-model X t = Z t + 0.6Z t 1 Exercise 2.14 X t = A cosωt + B siωt, t Z where A ad B are ucorrelated radom variables with zero mea ad variace 1. statioary with ACF ρh = cosωh. This process is a P 1 X 2 = φ 11 X 1 where γ0φ 11 = γ1, which gives φ 11 = ρ1 = cos ω. Hece Also P 1 X 2 = cosωx 1 E[X 2 P 1 X 2 2 ] = γ0 φ 11 γ1 = γ01 cos 2 ω = si 2 ω Note: 2.14 is a example i which the matrix Γ i the equatio Γ φ = γ is sigular for 3. This is because X 3 = 2 cos ωx 2 X 1. b where P 2 X 3 = φ 21 X 2 + φ 22 X 1 γ0φ 21 + γ1φ 22 = γ1 γ1φ 21 + γ0φ 22 = γ2 problemsheet1 August 30, 2007 Side 11

12 that is φ 21 + cos ωφ 22 = cos ω cos ωφ 21 + φ 22 = cos 2ω Solvig these equatios give φ 22 cos 2 ω 1 = cos 2 ω 2 cos 2 ω + 1 = cos 2 ω + 1, that is, φ 22 = 1, ad the, φ 21 = cos ω φ 22 cos ω = 2 cos ω. Hece ad P 2 X 3 = 2 cos ωx 2 X 1 E[X 3 P 2 X 3 2 ] = γ0 φ 2 γ 2 = 1 2 cos ω, 1cos ω, cos 2ω = 1 2 cos 2 ω + cos 2ω = 0 c >From b ad statioarity, it follows that P X +1 X, X 1 = 2 cos ωx X 1 with MSE = 0. Sice 2 cos ωx X 1 is a liear combiatio of X s, < s, ad sice it is impossible to d a predictor of this form with smaller MSE, we coclude that P X +1 = 2 cos ωx X 1 with MSE = 0. Exercise 2.18 Give the MA1 process X t = Z t θz t 1 where θ < 1, ad Z t W N0, σ 2. Represeted as a AR process, it assumes the form Z t = X t + θx t 1 + θ 2 X t Settig t = + 1 i the last equatio ad applyig P to each side, leads to the result P X +1 = θ j X +1 j = θz Predictio error = X +1 P X +1 = Z +1. Hece, MSE = E[Z 2 +1 ] = σ2. j=1 problemsheet1 August 30, 2007 Side 12

13 Exercise 2.19 The give MA1-model is X t = Z t Z t 1 ; where Z t WN0, σ 2. The vector a = a 1,..., a of the coeciets that provide the best liear predictor BLP of X +1 i terms of X = X,..., X 1 satises the equatio Γ a = γ where the covariace matrix Γ = CovX, X ad γ = CovX +1, X = γ1,..., γ. Sice γ0 = 2σ 2, γ1 = σ 2, γh = 0 for h > 1, it follows that Γ = σ 2. t Z ad γ = σ 2 1, 0,..., 0. It ca be show, e.g. by iductio, that the equatios to be solved ca be rewritte as follows a a a 3 1 a 4 = a a 0 The solutio is foud to be give as follows a j = 1 j + 1 j + 1 Hece it is obtaied that P X +1 = j=1 1 j + 1 j + 1 X +1 j The mea square error is E[X +1 P X +1 2 ] = γ0 a γ = 2σ 2 + a 1 σ 2 = σ problemsheet1 August 30, 2007 Side 13

14 Exercise 2.20 We have to prove that CovX ˆX, X j = E[X ˆX X j ] = 0 for j = 1,..., 1. This follows from equatios for suitable values of ad h with a 0 = 0 sice we may assume that E[X ] = 0. This clearly implies that E[X ˆX X k ˆX k ] = 0 for k = 1,..., 1, sice ˆX k is a liear combiatio of X 1,..., X k 1. Exercise 2.21 I this exercise we shall determie the best liear predictor BLP P X 3 W α wrt three dieret vector variables W α, α = a, b, c. Let Γ α = CovW α, W α ad γ α = CovX 3, W α. The give MA1-model is X t = Z t + θz t 1 ; t Z where Z t WN0, σ 2. a I this case we have W a = W 1, W 2 = X 2, X 1. Hece Γ a = σ θ 2 θ θ 1 + θ 2 ad γ a = CovX 3, W a = γ1, γ2 = σ 2 θ, 0. The equatio Γ a a 1, a 2 = γ a, or has the solutio We obtai the BLP The mea square error 1 + θ 2 a 1 + θa 2 = θ θa θ 2 a 2 = 0 a 1 = θ1 + θ2 1 + θ 2 2 θ 2 a 2 = P X 3 X 2, X 1 = θ θ 2 2 θ 2 θ 1 + θ θ 2 2 θ 2 X 2 θx 1 E[X 3 P X 3 X 2, X 1 2 ] = VarX 3 a 1, a 2 γ a = σ θ 2 a 1 σ 2 θ = σ θ 2 θ θ 2 2 θ 2 b problemsheet1 August 30, 2007 Side 14

15 Here W b = W 1, W 2 = X 4, X 5. With this choice, it follows that Γ b = Γ a, ad γ b = γ a. It follows immediately that the BLP is give by P X 3 X 4, X 5 = Ad the mea square error is the same as i a E[X 3 P X 3 X 4, X 5 2 ] = σ θ 2 θ 1 + θ θ 2 2 θ 2 X 4 θx 5 θ θ 2 2 θ 2 c Now, W b = W 1, W 2, W 3, W 4 = X 2, X 1, X 4, X 5. It the follows that Γ c = σ 2 Γa 0 0 Γ a where 0 deotes a 2 2 zero-matrix. Also, γ c = γ a, γ a. Hece, it follows that the solutio to the equatio Γ c a 1,..., a 4 = γ c is give by a 3 = a 1 ad a 4 = a 2, where a 1 ad a 2 are as give i a or b. The BLP is therefore with mea square error P X 3 X 2, X 1, X 4, X 5 = θ 1 + θ θ 2 2 θ 2 [X 2 + X 4 ] θ[x 1 + X 5 ] E[X 3 P X 3 X 2, X 1, X 4, X 5 2 ] = V arx 3 a 1, a 2, a 3, a 4 γ c = σ θ 2 2a 1 σ 2 θ = σ θ 2 2θ θ 2 2 θ 2 d See above. Exercise 2.22 We shall determie the best liear predictor BLP P X 3 W α wrt three dieret vector variables W α, α = a, b, c. Let Γ α = CovW α, W α ad γ α = CovX 3, W α. The give causal statioary AR1-model is X t = φx t 1 + Z t ; t Z where Z t WN0, σ 2. Causality implies that φ < 1. Hece, the ACVF γh = σ 2 1 φ 2 1 φ h. a I this case we have W a = W 1, W 2 = X 2, X 1. Hece Γ a = σ2 1 φ 1 φ 2 φ 1 problemsheet1 August 30, 2007 Side 15

16 ad γ a = CovX 3, W a = γ1, γ2 = σ2 φ, φ 2. The equatio Γ 1 φ 2 a a 1, a 2 = γ a, or a 1 + φa 2 = φ has the solutio We obtai the BLP The mea square error φa 1 + a 2 = φ 2 a 1 = φ a 2 = 0 P X 3 X 2, X 1 = φx 2 E[X 3 P X 3 X 2, X 1 2 ] = VarX 3 a 1, a 2 γ a = σ2 1 φ 2 σ2 φ 2 1 φ 2 = σ2 b Here W b = W 1, W 2 = X 4, X 5. With this choice, it follows that Γ b = Γ a, ad γ b = γ a. It follows immediately that the BLP is give by Ad the mea square error is P X 3 X 4, X 5 = φx 4 E[X 3 P X 3 X 4, X 5 2 ] = V arx 3 a 1, a 2 γ b = σ2 1 φ 2 σ2 φ 2 1 φ 2 = σ2 c Now, W c = W 1, W 2, W 3, W 4 = X 2, X 1, X 4, X 5. It the follows that Γ c = σ2 1 φ 2 1 φ φ 2 φ 3 φ 1 φ 3 φ 4 φ 2 φ 3 1 φ φ 3 φ 4 φ 1 where γ c = γ a, γ a. Hece, the followig set of equatios is obtaied a 1 + φa 2 + φ 2 a 3 + φ 3 a 4 = φ φa 1 + a 2 + φ 3 a 3 + φ 4 a 4 = φ 2 φ 2 a 1 + φ 3 a 2 + a 3 + φa 4 = φ φ 3 a 1 + φ 4 a 2 + φa 3 + a 4 = φ 2 It is see that the rst two equatios give a 2 = 0, while the last two equatios give a 4 = 0. The it is foud that a 1 = a 3 = φ 1 + φ 2 problemsheet1 August 30, 2007 Side 16

17 The BLP is therefore with mea square error P X 3 X 2, X 1, X 4, X 5 = φ 1 + φ 2 [X 2 + X 4 ] E[X 3 P X 3 X 2, X 1, X 4, X 5 2 ] = V arx 3 a 1, a 2, a 3, a 4 γ c = = σ2 1 + φ 2 σ2 1 φ 2 σ2 2φ 2 1 φ φ 2 d See above. problemsheet1 August 30, 2007 Side 17

Estimation of the Mean and the ACVF

Estimation of the Mean and the ACVF Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].

Probability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)]. Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x

More information

Homework 3 Solutions

Homework 3 Solutions Math 4506 Sprig 04 Homework 3 Solutios. a The ACF of a MA process has a o-zero value oly at lags, 0, ad. Problem 4.3 from the textbook which you did t do, so I did t expect you to metio this shows that

More information

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons Statistical Aalysis o Ucertaity for Autocorrelated Measuremets ad its Applicatios to Key Comparisos Nie Fa Zhag Natioal Istitute of Stadards ad Techology Gaithersburg, MD 0899, USA Outlies. Itroductio.

More information

Unbiased Estimation. February 7-12, 2008

Unbiased Estimation. February 7-12, 2008 Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

Solution to Chapter 2 Analytical Exercises

Solution to Chapter 2 Analytical Exercises Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =

More information

Probability and Random Processes

Probability and Random Processes Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems

More information

Problems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:

Problems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman: Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio

More information

Lecture 6 Ecient estimators. Rao-Cramer bound.

Lecture 6 Ecient estimators. Rao-Cramer bound. Lecture 6 Eciet estimators. Rao-Cramer boud. 1 MSE ad Suciecy Let X (X 1,..., X) be a radom sample from distributio f θ. Let θ ˆ δ(x) be a estimator of θ. Let T (X) be a suciet statistic for θ. As we have

More information

x = Pr ( X (n) βx ) =

x = Pr ( X (n) βx ) = Exercise 93 / page 45 The desity of a variable X i i 1 is fx α α a For α kow let say equal to α α > fx α α x α Pr X i x < x < Usig a Pivotal Quatity: x α 1 < x < α > x α 1 ad We solve i a similar way as

More information

Lecture 10 October Minimaxity and least favorable prior sequences

Lecture 10 October Minimaxity and least favorable prior sequences STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least

More information

ST5215: Advanced Statistical Theory

ST5215: Advanced Statistical Theory ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The

More information

Introduction to Probability. Ariel Yadin

Introduction to Probability. Ariel Yadin Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

Statistical and Mathematical Methods DS-GA 1002 December 8, Sample Final Problems Solutions

Statistical and Mathematical Methods DS-GA 1002 December 8, Sample Final Problems Solutions Statistical ad Mathematical Methods DS-GA 00 December 8, 05. Short questios Sample Fial Problems Solutios a. Ax b has a solutio if b is i the rage of A. The dimesio of the rage of A is because A has liearly-idepedet

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should be doe

More information

Mathematics 170B Selected HW Solutions.

Mathematics 170B Selected HW Solutions. Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Solutions: Homework 3

Solutions: Homework 3 Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Linear Regression Demystified

Linear Regression Demystified Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Sample Correlation. Mathematics 47: Lecture 5. Dan Sloughter. Furman University. March 10, 2006

Sample Correlation. Mathematics 47: Lecture 5. Dan Sloughter. Furman University. March 10, 2006 Sample Correlatio Mathematics 47: Lecture 5 Da Sloughter Furma Uiversity March 10, 2006 Da Sloughter (Furma Uiversity) Sample Correlatio March 10, 2006 1 / 8 Defiitio If X ad Y are radom variables with

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze

More information

Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables

Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;

More information

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we

More information

Linear Regression Models

Linear Regression Models Liear Regressio Models Dr. Joh Mellor-Crummey Departmet of Computer Sciece Rice Uiversity johmc@cs.rice.edu COMP 528 Lecture 9 15 February 2005 Goals for Today Uderstad how to Use scatter diagrams to ispect

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig

More information

x iu i E(x u) 0. In order to obtain a consistent estimator of β, we find the instrumental variable z which satisfies E(z u) = 0. z iu i E(z u) = 0.

x iu i E(x u) 0. In order to obtain a consistent estimator of β, we find the instrumental variable z which satisfies E(z u) = 0. z iu i E(z u) = 0. 27 However, β MM is icosistet whe E(x u) 0, i.e., β MM = (X X) X y = β + (X X) X u = β + ( X X ) ( X u ) \ β. Note as follows: X u = x iu i E(x u) 0. I order to obtai a cosistet estimator of β, we fid

More information

ECE534, Spring 2018: Final Exam

ECE534, Spring 2018: Final Exam ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute

More information

University of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!

University of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck! Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad

More information

Supplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate

Supplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We

More information

Optimally Sparse SVMs

Optimally Sparse SVMs A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but

More information

Stat 139 Homework 7 Solutions, Fall 2015

Stat 139 Homework 7 Solutions, Fall 2015 Stat 139 Homework 7 Solutios, Fall 2015 Problem 1. I class we leared that the classical simple liear regressio model assumes the followig distributio of resposes: Y i = β 0 + β 1 X i + ɛ i, i = 1,...,,

More information

Ma 530 Introduction to Power Series

Ma 530 Introduction to Power Series Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power

More information

SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker

SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker CHAPTER 9. POINT ESTIMATION 9. Covergece i Probability. The bases of poit estimatio have already bee laid out i previous chapters. I chapter 5

More information

2.2. Central limit theorem.

2.2. Central limit theorem. 36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard

More information

TMA4245 Statistics. Corrected 30 May and 4 June Norwegian University of Science and Technology Department of Mathematical Sciences.

TMA4245 Statistics. Corrected 30 May and 4 June Norwegian University of Science and Technology Department of Mathematical Sciences. Norwegia Uiversity of Sciece ad Techology Departmet of Mathematical Scieces Corrected 3 May ad 4 Jue Solutios TMA445 Statistics Saturday 6 May 9: 3: Problem Sow desity a The probability is.9.5 6x x dx

More information

32 estimating the cumulative distribution function

32 estimating the cumulative distribution function 32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

More information

A Risk Comparison of Ordinary Least Squares vs Ridge Regression

A Risk Comparison of Ordinary Least Squares vs Ridge Regression Joural of Machie Learig Research 14 (2013) 1505-1511 Submitted 5/12; Revised 3/13; Published 6/13 A Risk Compariso of Ordiary Least Squares vs Ridge Regressio Paramveer S. Dhillo Departmet of Computer

More information

IIT JAM Mathematical Statistics (MS) 2006 SECTION A

IIT JAM Mathematical Statistics (MS) 2006 SECTION A IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Sequences and Limits

Sequences and Limits Chapter Sequeces ad Limits Let { a } be a sequece of real or complex umbers A ecessary ad sufficiet coditio for the sequece to coverge is that for ay ɛ > 0 there exists a iteger N > 0 such that a p a q

More information

This section is optional.

This section is optional. 4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore

More information

MATH 324 Summer 2006 Elementary Number Theory Solutions to Assignment 2 Due: Thursday July 27, 2006

MATH 324 Summer 2006 Elementary Number Theory Solutions to Assignment 2 Due: Thursday July 27, 2006 MATH 34 Summer 006 Elemetary Number Theory Solutios to Assigmet Due: Thursday July 7, 006 Departmet of Mathematical ad Statistical Scieces Uiversity of Alberta Questio [p 74 #6] Show that o iteger of the

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

Lecture 8: Convergence of transformations and law of large numbers

Lecture 8: Convergence of transformations and law of large numbers Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Statistical Properties of OLS estimators

Statistical Properties of OLS estimators 1 Statistical Properties of OLS estimators Liear Model: Y i = β 0 + β 1 X i + u i OLS estimators: β 0 = Y β 1X β 1 = Best Liear Ubiased Estimator (BLUE) Liear Estimator: β 0 ad β 1 are liear fuctio of

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

Mathematical Statistics - MS

Mathematical Statistics - MS Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios

More information

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments: Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal

More information

MATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1

MATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1 MATH 2: HOMEWORK 6 SOLUTIONS CA PRO JIRADILOK Problem. If s = 2, ad Problem : Rudi, Chapter 3, Problem 3. s + = 2 + s ( =, 2, 3,... ), prove that {s } coverges, ad that s < 2 for =, 2, 3,.... Proof. The

More information

It is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.

It is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial. Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable

More information

Machine Learning Brett Bernstein

Machine Learning Brett Bernstein Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio

More information

MATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED

MATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas

More information

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + 62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of

More information

Lecture 12: September 27

Lecture 12: September 27 36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

More information

Distribution of Random Samples & Limit theorems

Distribution of Random Samples & Limit theorems STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to

More information

Section 14. Simple linear regression.

Section 14. Simple linear regression. Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo

More information

Math 451: Euclidean and Non-Euclidean Geometry MWF 3pm, Gasson 204 Homework 3 Solutions

Math 451: Euclidean and Non-Euclidean Geometry MWF 3pm, Gasson 204 Homework 3 Solutions Math 451: Euclidea ad No-Euclidea Geometry MWF 3pm, Gasso 204 Homework 3 Solutios Exercises from 1.4 ad 1.5 of the otes: 4.3, 4.10, 4.12, 4.14, 4.15, 5.3, 5.4, 5.5 Exercise 4.3. Explai why Hp, q) = {x

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

We are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n

We are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at

More information

Math 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency

Math 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the

More information

3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N.

3/3/2014. CDS M Phil Econometrics. Types of Relationships. Types of Relationships. Types of Relationships. Vijayamohanan Pillai N. 3/3/04 CDS M Phil Old Least Squares (OLS) Vijayamohaa Pillai N CDS M Phil Vijayamoha CDS M Phil Vijayamoha Types of Relatioships Oly oe idepedet variable, Relatioship betwee ad is Liear relatioships Curviliear

More information

Random Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices

Random Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio

More information

Notes 19 : Martingale CLT

Notes 19 : Martingale CLT Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15 17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

Proof of Goldbach s Conjecture. Reza Javaherdashti

Proof of Goldbach s Conjecture. Reza Javaherdashti Proof of Goldbach s Cojecture Reza Javaherdashti farzijavaherdashti@gmail.com Abstract After certai subsets of Natural umbers called Rage ad Row are defied, we assume (1) there is a fuctio that ca produce

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

f(x)g(x) dx is an inner product on D.

f(x)g(x) dx is an inner product on D. Ark9: Exercises for MAT2400 Fourier series The exercises o this sheet cover the sectios 4.9 to 4.13. They are iteded for the groups o Thursday, April 12 ad Friday, March 30 ad April 13. NB: No group o

More information

http://www.xelca.l/articles/ufo_ladigsbaa_houte.aspx imulatio Output aalysis 3/4/06 This lecture Output: A simulatio determies the value of some performace measures, e.g. productio per hour, average queue

More information

MA131 - Analysis 1. Workbook 9 Series III

MA131 - Analysis 1. Workbook 9 Series III MA3 - Aalysis Workbook 9 Series III Autum 004 Cotets 4.4 Series with Positive ad Negative Terms.............. 4.5 Alteratig Series.......................... 4.6 Geeral Series.............................

More information

TAMS24: Notations and Formulas

TAMS24: Notations and Formulas TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =

More information

1+x 1 + α+x. x = 2(α x2 ) 1+x

1+x 1 + α+x. x = 2(α x2 ) 1+x Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem

More information

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

Seunghee Ye Ma 8: Week 5 Oct 28

Seunghee Ye Ma 8: Week 5 Oct 28 Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value

More information

Supplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate

Supplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We

More information

Songklanakarin Journal of Science and Technology SJST R1 Teerapabolarn

Songklanakarin Journal of Science and Technology SJST R1 Teerapabolarn Soglaaari Joural of Sciece ad Techology SJST--.R Teeraabolar A No-uiform Boud o Biomial Aroimatio to the Beta Biomial Cumulative Distributio Fuctio Joural: Soglaaari Joural of Sciece ad Techology For Review

More information

BHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13

BHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13 BHW # /5 ENGR Probabilistic Aalysis Beautiful Homework # Three differet roads feed ito a particular freeway etrace. Suppose that durig a fixed time period, the umber of cars comig from each road oto the

More information

(all terms are scalars).the minimization is clearer in sum notation:

(all terms are scalars).the minimization is clearer in sum notation: 7 Multiple liear regressio: with predictors) Depedet data set: y i i = 1, oe predictad, predictors x i,k i = 1,, k = 1, ' The forecast equatio is ŷ i = b + Use matrix otatio: k =1 b k x ik Y = y 1 y 1

More information

Machine Learning Brett Bernstein

Machine Learning Brett Bernstein Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete

More information

Topics. Homework Problems. MATH 301 Introduction to Analysis Chapter Four Sequences. 1. Definition of convergence of sequences.

Topics. Homework Problems. MATH 301 Introduction to Analysis Chapter Four Sequences. 1. Definition of convergence of sequences. MATH 301 Itroductio to Aalysis Chapter Four Sequeces Topics 1. Defiitio of covergece of sequeces. 2. Fidig ad provig the limit of sequeces. 3. Bouded covergece theorem: Theorem 4.1.8. 4. Theorems 4.1.13

More information

DETERMINATION OF MECHANICAL PROPERTIES OF A NON- UNIFORM BEAM USING THE MEASUREMENT OF THE EXCITED LONGITUDINAL ELASTIC VIBRATIONS.

DETERMINATION OF MECHANICAL PROPERTIES OF A NON- UNIFORM BEAM USING THE MEASUREMENT OF THE EXCITED LONGITUDINAL ELASTIC VIBRATIONS. ICSV4 Cairs Australia 9- July 7 DTRMINATION OF MCHANICAL PROPRTIS OF A NON- UNIFORM BAM USING TH MASURMNT OF TH XCITD LONGITUDINAL LASTIC VIBRATIONS Pavel Aokhi ad Vladimir Gordo Departmet of the mathematics

More information

( a) ( ) 1 ( ) 2 ( ) ( ) 3 3 ( ) =!

( a) ( ) 1 ( ) 2 ( ) ( ) 3 3 ( ) =! .8,.9: Taylor ad Maclauri Series.8. Although we were able to fid power series represetatios for a limited group of fuctios i the previous sectio, it is ot immediately obvious whether ay give fuctio has

More information

ANSWERS TO MIDTERM EXAM # 2

ANSWERS TO MIDTERM EXAM # 2 MATH 03, FALL 003 ANSWERS TO MIDTERM EXAM # PENN STATE UNIVERSITY Problem 1 (18 pts). State ad prove the Itermediate Value Theorem. Solutio See class otes or Theorem 5.6.1 from our textbook. Problem (18

More information