Univerza na Primorskem FAMNIT, MFI STATISTIKA 2 Seminarska naloga
|
|
- Augustine Powers
- 5 years ago
- Views:
Transcription
1 Univerza na Primorskem FAMNIT, MFI STATISTIKA 2 Seminarska naloga Naloge so edini način preverjanja znanja pri predmetu Statistika. Vsaka naloga je vredna 10 točk, natančna pravila ocenjevanja pa so navedena na spletni strani predmeta.
2 1. Naj bodo ξ 1, ξ 2,..., ξ n neodvisne z ξ k exp1) za k = 1, 2,..., n. Označite ξ = ξ ξ n. Za 1 k n definirajte η k = Za fiksen 1 k n 1 naj bo ξ k ξ ξ n. η i = ξ i ξ k + ξ k ξ n za i = k, k + 1,..., n. a. Izračunajte gostoto vektorja η 1, η 2,..., η n 1, ξ). b. Pokažite, da sta slučajna vektorja η 1,..., η k 1 ) in η k,..., η n ) neodvisna. 2. Slučajne spremenljivke X, Y in Z naj imajo gosototo 1 πγ n 1 ) Γ n xy z2 ) n e x/2 e y/2 2 2 )2n za x > 0, y > 0 in xy z 2 > 0. a. Pokažite, da sta slučajni spremenljivki X in Y neodvisni. Kakšni sta porazdelitvi X in Y? b. Definirajte slučajne spremenljivke U, V in W s predpisom ) ) 1 U W X Z =. W V Z Y Ali sta slučajni spremenljivki U in V neodvisni? Kot znano privzemite, da je Jacobijeva determinanta preslikave x, y, z) Φ 1 y, x, z) xy z2 enaka xy z 2 ) 3. Preverite to slednje. 2
3 3. Suppose Z is a random vector whose components are independent standard normal random variables and let A be a rectangular matrix such that AA T is invertible. Prove that the density of X = AZ + µ is still given by the formula 1 f X x) = exp 2π) detaa 1 ) n/2 T 2 x µ)t AA T ) 1 x µ). ) 4. Suppose X is a multivariate normal vector with expectation µ and variance Σ. Write ) ) X1 Σ11 Σ X = and Σ = 12. X 2 Σ 21 Σ 22 Assume Σ is invertible. Compute the conditional density of X 2 given X 1 = x 1 by using the usual formula Hint: Use the inversion lemma Σ 1 = f X2 X 1 =x 1 x 2 ) = f Xx) f X1 x 1 ). Σ 11 Σ 12 Σ 1 22 Σ 21 ) 1 Σ 11 Σ 12 Σ 1 22 Σ 21 ) 1 Σ 12 Σ 1 22 Σ 1 22 Σ 21 Σ 11 Σ 12 Σ 1 22 Σ 21 ) 1 Σ 22 Σ 21 Σ 1 11 Σ 12 ) 1 Compare this proof to the slicker one using independence of linear transformations of multivariate normal vectors. Comment. 5. Let X be a n p data matrix from N p µ, Σ). This means that the rows of X are independent p-dimensional vectors with the above distribution. Let a, b be fixed p dimensional vectors and c a fixed n-dimensional vector. Find the conditional distribution of Y 2 = Xb given Y 1 = c T Xa. Hint: You can either do it the hard way computing all the covariances, or show that ) I cct Xa c T c is independent of Y 1. ) 3
4 6. Suppose X 1, X 2,..., X n are i.i.d. observations from a multivariate normal distribution Nµ, Σ) where Σ is known. Further assume that R is a given matrix and r a given vector. For a given vector r use the likelihood ratio procedure to produce a test statistic for H 0 : Rµ = r vs. H 1 : Rµ r. Find the exact distribution of the likelihood ratio test statistic under H Vrstice v matriki X naj bodo med sabo neodvisni večrazsežni normalni vektorji z matematičnim upanjem 0 in varianco Σ. Predpostavite, da je vektor ɛ neodvisen od X z Eɛ) = 0 in varɛ) = σ 2 I. Predpostavite, da je Y = Xβ + ɛ. a. Pokažite, da sta običajni cenilki ˆβ in ˆσ 2 iz linearne regresije še vedno nepristranski. b. Privzemite, da je ɛ večrazsežen normalen. Izračunajte var ˆβ i ). Namig: Oglejte si Theorem v K. V. Mardia, J. T. Kent and J. M. Bibby, Multivariate Analysis, Academic Press, c. Privzemite, da je ɛ večrazsežen normalen. Kakšna je porazdelitev cenilke ˆσ 2? 8. Naj bosta θ i ) i 1 in ξ i ) i 1 zaporedji slučajnih vektorjev dimenzij k in l, ki zadoščajo enačbam θ n+1 = a 1,n + A 1,n θ n + B 1,n η 1,n+1 + C 1,n η 2,n+1 ξ n+1 = a 2,n + A 2,n θ n + B 2,n η 1,n+1 + C 2,n η 2,n+1, kjer so a 1,n, a 2,n vektorji dimenzij k in l, A 1,n, A 2,n matrike domenzij k k in l k, B 1,n, B 2,n matrike dimenzij k k and l k, C 1,n in C 2,n matrike dimenzij k l and l l, ter η 1,n and η 2,n i.i.d. zaporedji z porazdelitvama N0, I k ) in N0, I l ), ki sta neodvisni. Privzemite, da je θ 0, ξ 0 ) večrazsežen normalen vektor neodvisen od zaporedij η i,j ) j 1, i = 1, 2,..., tako da je ) ) )) θ0 µ0 Σ11 Σ N, 12. ξ 0 ν 0 Σ 21 Σ 22 4
5 a. Pokažite, da je pogojna porazdelitev θ n in θ n, ξ n+1 ) T in θ n+1, ξ n+1 ) T glede na ξ 1, ξ 2,..., ξ n ) večrazsežna normalna. Porazdelitve ni potrebno eksplicitno opisati. b. Označite µ n = Eθ n ξ 1, ξ 2,..., ξ n ) in γ n = var{θ n ξ 1, ξ 2,..., ξ n }. Vemo, da je µ n najboljša napoved θ n na podlagi ξ 1, ξ 2,..., ξ n. Pokažite, da veljajo naslednje rekurzijske formule, pri čemer vedno privzemite, da lahko vedno obrnete matrike, ki jih je potrebno obrniti. µ n+1 = [a 1,n + A 1,n µ n ] + [B 1,n B T 2,n + C 1,n C T 2,n + A 1,n γ n A T 2,n] [B 2,n B T 2,n + C 2,n C T 2,n + A 2,n γ n A T 2,n] 1 [ξ n+1 a 2,n A 2,n µ n ] γ n+1 = [A 1,n γ n A T 1,n + B 1,n B T 1,n + C 1,n C T 1,n] [B 1,n B T 2,n + C 1,n C T 2,n + A 1,n γ n A T 2,n][B 2,n B T 2,n + C 2,n C T 2,n + A 2,n γ n A T 2,n] 1 [B 1,n B T 2,n + C 1,n C T 2,n + A 1,n γ n A T 2,n] T. Opomba: Te rekurzijske formule so znane kot Kalman-Bucyjev filter. Namigi: i) Če je Z = X, Y)T slučajni vektor, tak da je pogojna porazdelitev Z glede na W večrazsežna normalna s parametri ) )) µ0 Σ11 Σ N, 12. ν 0 Σ 21 Σ 22 Pokažite, da je pogojna porazdelitev Y glede na X, W večrazsežna normalna s parametroma EY X, W) = EY W) + Σ 21 Σ 1 11 X EX W)) var{y X, W} = Σ 22 Σ 21 Σ 1 11 Σ 12. 5
6 ii) Naj bo Eθ n+1 ξ 1, ξ 2,..., ξ n+1 ) = Eθ n+1 ξ 1, ξ 2,..., ξ n ), ξ n+1 ) Uporabite i). c. Zakaj mislite, je uporabljena beseda filter? d. Kako bi simulirali večrazsežne normalne vektorje z danimi parametri? e. Predpostavite, da so vse matrike neodvisne od n. Simulirajte ustrezna zaporedja in ocene µ n za majhne dimenzije, recimo 2. Prikažite razultate z ustreznimi grafi. 9. Pogosto srečamo v statistiki problem manjkajočih opazovanih vrednosti. Obstaja mnogo metod, kako korektno oceniti parametre. Ogledali si bomo poseben primer EM expectation maximization) algoritma, ki je eden od pristopov. a. Prepostavite, da so vaše opazovane vrednosti neodvisni p-razsežni normalni vektorji X 1, X 2,..., X n s parametri µ in Σ. Ocenite parametra po metodi največjega verjetja, če ni manjkajočih podatkov. b. Predpostavite, da nekatere komponente opazovanih vektorjev manjkajo. Prepostavite, da so podatki manjkajo naključno in neodvisno od X 1,..., X n, vendar tako, da nikoli ne manjkajo vse komponente. Označimo z x 1, x 2,..., x n opazovane vrednosti z manjkajočimi podatki). EM algoritem ima dva koraka: i) E-korak: Naj bo l c µ, Σ x 1, x 2,..., x n ) logaritemska funkcija verjetja, če imamo vse podatke. Indeks c pomeni complete. Te funkcije ne moremo izračunati, če kakšen podatek manjka. Kaj storiti? Označimo z y 1, y 2,..., y n dejansko opazovane škrbaste vektorje. Izberimo začeten približek za parametra µ in Σ, recimo µ 0 in Σ 0. Izračunajmo pogojno matematično upanje Qµ, Σ, µ 0, Σ 0 ) = E l c µ, Σ X 1, X 2,..., X n ) ) y 1,..., y n. Pri tem privzemamo, da so X 1, X 2,..., X n porazdeljeni večrazsežno normalno s parametroma µ 0 in Σ 0. 6
7 ii) M-korak: Naslednja približka µ 1 in Σ 1 za neznana parametra dobimo tako, da maksimiziramo funkcijo Qµ, Σ, µ 0, Σ 0 ) po µ in Σ. Koraka E in M potem ponavljamo. Ponovimo E-korak z novimi približki za parameter in pridelamo nove približke z M-korakom. V mnogo primerih glej Dempster, A. P., Laird, N. M., and Rubin, D. B. 1977). Maximum likelihood from incomplete data via the EM algorithm with discussion), Journal of the Royal Statistical Society B, 39, 1-38) zaporedni približki konvergirajo proti neki limiti, ki je potem naša ocena za parametre. Na kratko komentirajte, kaj mislite o tem postopku? Se vam zdi smiseln? Zakaj? c. Opišite, s čim se nadomestijo manjkajoče vrednosti v primeru večrazsežne normalne porazdelitve. Utemeljite vaše izjave. Lahko se omejite na primer p = 2. Kako smiseln se vam zdi zdaj EM algoritem? Na kratko komentirajte. d. Naj bo p = 2. Generirajte vzorec velikosti n = 400. Za vsak k = 1, 2,..., n naj manjka ena od komponent z verjetnostjo 1/10 in sicer manjkajoči podatek izberite naključno z verjetnostjo 1/2. Sprogramirajte EM algoritem in ugotovite ali zaporedni približki res konvergirajo. Primerjajte limitne ocene s tistimi, ki bi jih dobili z metodo največjega verjetja samo na podlagi podatkov, kjer ne manjka nobena komponenta. Komentar? Literatura: Geoffrey J. McLachlan, Thriyambakam Krishnan, The EM Algorithm and Extensions, Wiley Series in Probability and Statistics, Suppose a linear model Y = Xβ + ɛ where ɛ N0, σ 2 I). Write X = [X 1 ; X 2 ] where X 1 are the first p 1 columns of X and X 2 are the last p 2 columns. Similarly split β T = β T 1 ; β T 2 ). a. If X 1X 2 = 0 show that the two subcomponents ˆβ 1 and ˆβ 2 of the BLUE ˆβ are independent. 7
8 b. Find the distribution of U = Y X 1 ˆβ1 ) T Y X 1 ˆβ1 ). 11. Suppose Y = Xβ+ɛ where ɛ N0, σ 2 I). Show that for the internally studentised residuals r i defined as r i = ˆɛ i ˆσ 1 h ii. the expression r 2 i /n p) has the Beta1/2, n p 1)/2) distribution where p = rankx). Hints: i) Recall that the Betap, q) distribution is defined as the distribution of U/U + V ) where U and V are independent, and U Gammap, λ), V Gammaq, λ). ii) Let e i be the n-dimensional vector whose only nonzero component is a 1 in the i th position, and note that ˆɛ i = e T i I H)Y. iii) Define P = I H and P 1 = Pe i e T i P/1 h ii ), and U = Y P 1 Y, V = Y P P 1 )Y. Show that U and V are independent and have Gamma distributions. iv) Show that r 2 i /n p) = U/U + V ) to conclude the proof. 12. Suppose Y = Xβ + ɛ with Eɛ) = 0 and varɛ) = σ 2 I. Assume X is of full rank. To test the hypothesis H 0 : β i = 0 vs. H 1 : β i 0 one has two possibilities: i) Use the t = ˆβ i /ˆσ c ii where c ii is the i-th diagonal element of X T X) 1 and reject H 0 if t t n p 1 α/2) for a given α 0, 1). ii) Test H 0 : Xβ ω vs. H 1 : Xβ RangeX)\ω 8
9 where ω is the subspace spanned by columns of X other than the i-th one and then use the quadratic forms approach based on Cochran s theorem, and reject H 0 if the appropriate F F 1,n p 1 α) for a given α 0, 1). Prove that for a given α the two tests are the same: more precisely, you will need to show that Q 1 Q 0 = ˆβ 2 i /c ii. Hints: i) Prove that the estimate of β i does not change if you replace the i-th column in X by its orthogonal projection onto the orthogonal complement of the space spanned by the other columns while the estimates of other β s will in general change). ii) Express the quadratic forms Q 0 and Q 1 in terms of these new estimates using orthogonality. 13. Assume that the data x 1, x 2,..., x n are an i.i.d. sample from the multivariate normal distribution of the form ) )) µ 1) Σ11 Σ X 1 N µ 2), 12. Σ 21 Σ 22 Assume that the parameters µ and Σ are unknown. Assume the following theorem: If Ap p) is a given symmetric positive definite matrix then the positive definite matrix Σ that maximizes the expression 1 exp 12 detσ) Tr Σ 1 A )) n/2 is the matrix Σ = 1 n A. The testing problem is H 0 : Σ 12 = 0 versus H 1 : Σ a. Find the maximum likelihood estimates of µ and Σ in the unconstrained case. b. Find the maximum likelihood estimates of µ and Σ in the constrained case. 9
10 c. Write the likelihood ratio statistic for the testing problem as explicitly as possible. d. What can you say about the distribution of the likelihood ratio statistic if H 0 is true. e. Simulate by computer the distribution of the test statistics under H 0 in case when the vectors are 2-dimensional. Comment. 14. Raschev model za analizo binarnih podatkov specificira, da velja P X ij = x ij, 1 i m, 1 j n) = i,j e α i δ j )x ij 1 + e α i δ j ) za parametre α = α 1,..., α m ) in δ = δ 1,..., δ n ). a. Na predavanjih smo našli nabor zadostnih statistik. Pokažite, da so te statistike zadostne brez uporabe faktorizacijskega izreka. b. V porazdelitvi spremenjivk X ij se nič ne spremeni, če vsakemu parametru prištejemo isto konstanto. Rečemo, da sta paramtera α in δ nedoločljiva. Ali lahko govorimo o zadostnih statistikah tudi v tem primeru? Utemeljite! c. Naj bo za vse 1 j n x j = m x ij. i=1 Pokažite, da je pogojna porazdelitev {X ij : 1 i m, 1 j n} glede na X 1,..., X n ) neodvisna od parametra δ. d. Ali lahko uporabite ugotovitev v točki c., da ocenite parameter α? Pomislite na metodo največjega verjetja. Ker so parametri nedoločljivi, lahko umetno dodate zahtevo, da je α 1 + +α m = 0. e. Napišite program, ki bo na podatkih v rasch-data.txt ocenil parameter α. 15. Abstraktno lahko formuliramo, da je vektorska) statistika T X) zadostna, če za vsak θ Ω in za vsako omejeno Borelovo funkcijo φ velja E θ φx) T X) ) = Φφ T X) ) 10
11 za neko funkcijo Φ φ, ki ni odvisna od paramtera θ. Predpostavite, da ima X gostoto fx, θ) in da velja fx, θ) = gt x), θ) hx) za neki ustrezni funkciji g in h. Pokazati želimo, da je ta pogoj zadosten za to, da je statistika T X) zadostna. a. Privzemite najprej, da je h gostota neke porazdelitve. Naj bo Y slučajna spremenljivka s to gostoto. Utemeljite, da velja za neko Borelovo funkcijo Φ φ. E φy ) T Y ) ) = Φ φ T Y )) b. Utemeljite, da je za zadostnost dovolj, če pokažemo, da je za vsako omejeno Borelovo funkcijo ψ in za vsak θ Ω c. Utemeljite, da je E θ φx)ψt X)) )) = Eθ Φφ T X))ψT X) ). E θ Φφ T X))ψT X)) = E Φ φ T Y )) ψt Y )) gt Y ), θ) ). Namig: Napišite prvo upanje kot integral in upoštevajte, da je h gostota spremenljivke Y. d. Utemeljite, da je E Φ φ T Y )) ψt Y )) gt Y ), θ) ) = E φy ) ψt Y )) gt Y ), θ) ). e. Pokažite, da je E φy ) ψt Y )) gt Y ), θ) ) = E θ φx) ψt X)) ) in sklepajte, da je T X) zadostna statistika. f. Prevedite primer, ko h ni gostota na primer, ko h je gostota. 16. Eksponentne družine porazdelitev opisujejo gostote glede na neko referenčno mero ν v našem primeru ali Lebesgueovo mero ali mero, ki šteje) r ) fx, θ) = cθ)hx) exp c i θ)t i x). 11 i=1
12 a. Pokažite, da je T X) = T 1 X),..., T r X)) zadostna statistika. b. Privzemite, da je r = 1 in naj bo c 1 θ) = θ za enodimenzionalni parameter θ a, b). Kot znano privzemite, da obstaja od θ neodvisna σ-končna mera ν T na R, da velja P θ T X) A) = cθ)e θt dν T t). Kot znano tudi privzemite, da je funkcija z gt)e zt dν T t) holomorfna na traku Rez) a, b) za vsako funkcijo g, za katero je gt) e zt dν T t) < za vsak θ a, b). Če je za tako funkcijo E θ gt X)) = 0 za vse θ a, b), lahko enakost prepišemo v gt)e zt dν T t) = 0, A iz česar sledi gx) + e zt dν T t) = gx) e zt dν T t). Za fiksen θ 0 a, b) definirajte verjetnostni meri P in Q s predpisom P A) = 1 g + x) e θ0t dν T t) in QA) = 1 g x) e θ0t dν T t) a a kjer je a ustrezna končna konstanta, ki obstaja po predpostavki. Na podlagi holomorfnosti in s pomočjo izreka o edinosti za karakteristične funkcije sklepajte, da sta meri P in Q enaki. c. Na podlagi b. sklepajte, da je g + = g skoraj gotovo glede na ν T, torej je gt X)) = 0 skoraj gotovo glede na ν. e. Posplošite izrek z indukcijo na primer, ko je r > 1 in velja a i < θ i < b i. 12
13 f. Posplošite izrek na primer, ko so funkcije c 1,..., c r splošne, vendar take, da ima zaloga vrednosti funkcije c = c 1,..., c r ), ki slika Ω v R r, neprazno notranjost. g. Preverite, da ta izrek da kompletnost statistik za izbran nabor standardnih porazdelitev Poisson, normalna, gama) Suppose {px, θ), θ Θ R k } is a regular) family of distributions. Define the vector valued score function s as the column vector with components sx, θ) = θ logpx, θ)) = gradlogpx, θ)). and the Fisher information matrix as Iθ) = vars). Remark: If px, θ) = 0 define log px, θ)) = 0. a. Let tx) be an unbiased estimator of θ based on the likelihood function, i.e. E θ tx)) = θ. Prove that Deduce that covs, t) = I. Es) = 0 and Est T ) = I. Remark: Make liberal assumptions about interchanging integration and differentiation. b. Let a, c be two arbitrary k-dimensional vectors. Prove that corr 2 a T t, c T s ) = a T c) 2 a T vart)a c T Iθ)c. The correlation coefficient squared is always less or equal 1. Maximize the expression for the correlation coefficient over c and deduce the Rao-Cramèr inequality. 1 Povzeto po: Mark J. Shervish, Theory of Statistics, Springer, 1996, str
14 18. Naj bodo X 1, X 2,..., X n neodvisni večrazsežni normalni vektorji za enako porazdelitvijo Nµ, I). Poiščite najboljšo nepristransko cenilko za izraz µ T µ + 1 T µ, kjer je 1 T = 1, 1,..., 1). Utemeljite vsak korak. 19. Assume the data pairs y 1, z 1 ),..., y n, z n ) are an i.i.d. sample from the distribution with density for y > 0 and σ > 0. fy, z, θ, σ) = e y 1 e z θy) 2yσ 2 2πyσ a. Find the maximum likelihood estimators of θ and σ 2. Are the estimators unbiased? b. Find the exact standard errors of ˆθ and ˆσ 2. c. Compute the Fisher information matrix. d. Find the standard errors of the maximum likelihood estimators using the Fisher information matrix. Comment on your findings. e. Can you find a sufficient statistic for the parameter θ, σ). Can you use them to improve your unbiased estimators? Can you prove that the two unbiased estimators have the least variance among all unbiased estimators? Compare the variance to the Rao-Cramèr bound. 2 14
Statement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Take-home final examination February 1 st -February 8 th, 019 Instructions You do not need to edit the solutions Just make sure the handwriting is legible The final solutions should
More informationSTAT 730 Chapter 4: Estimation
STAT 730 Chapter 4: Estimation Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 23 The likelihood We have iid data, at least initially. Each datum
More informationMaximum Likelihood Estimation
Maximum Likelihood Estimation Merlise Clyde STA721 Linear Models Duke University August 31, 2017 Outline Topics Likelihood Function Projections Maximum Likelihood Estimates Readings: Christensen Chapter
More informationPart IB Statistics. Theorems with proof. Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua. Lent 2015
Part IB Statistics Theorems with proof Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
More information5.1 Consistency of least squares estimates. We begin with a few consistency results that stand on their own and do not depend on normality.
88 Chapter 5 Distribution Theory In this chapter, we summarize the distributions related to the normal distribution that occur in linear models. Before turning to this general problem that assumes normal
More informationRegression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationProblem Selected Scores
Statistics Ph.D. Qualifying Exam: Part II November 20, 2010 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. Problem 1 2 3 4 5 6 7 8 9 10 11 12 Selected
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationMathematical statistics
October 4 th, 2018 Lecture 12: Information Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation Chapter
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More information3. For a given dataset and linear model, what do you think is true about least squares estimates? Is Ŷ always unique? Yes. Is ˆβ always unique? No.
7. LEAST SQUARES ESTIMATION 1 EXERCISE: Least-Squares Estimation and Uniqueness of Estimates 1. For n real numbers a 1,...,a n, what value of a minimizes the sum of squared distances from a to each of
More informationECE 275B Homework # 1 Solutions Winter 2018
ECE 275B Homework # 1 Solutions Winter 2018 1. (a) Because x i are assumed to be independent realizations of a continuous random variable, it is almost surely (a.s.) 1 the case that x 1 < x 2 < < x n Thus,
More informationMatrix Approach to Simple Linear Regression: An Overview
Matrix Approach to Simple Linear Regression: An Overview Aspects of matrices that you should know: Definition of a matrix Addition/subtraction/multiplication of matrices Symmetric/diagonal/identity matrix
More informationECE 275B Homework # 1 Solutions Version Winter 2015
ECE 275B Homework # 1 Solutions Version Winter 2015 1. (a) Because x i are assumed to be independent realizations of a continuous random variable, it is almost surely (a.s.) 1 the case that x 1 < x 2
More informationDA Freedman Notes on the MLE Fall 2003
DA Freedman Notes on the MLE Fall 2003 The object here is to provide a sketch of the theory of the MLE. Rigorous presentations can be found in the references cited below. Calculus. Let f be a smooth, scalar
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationMathematical Statistics
Mathematical Statistics Chapter Three. Point Estimation 3.4 Uniformly Minimum Variance Unbiased Estimator(UMVUE) Criteria for Best Estimators MSE Criterion Let F = {p(x; θ) : θ Θ} be a parametric distribution
More informationSolutions. Name and surname: Instructions
Uiversity of Ljubljaa, Faculty of Ecoomics Quatitative fiace ad actuarial sciece Probability ad statistics Writte examiatio September 4 th, 217 Name ad surame: Istructios Read the problems carefull before
More informationChapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationSpring 2012 Math 541A Exam 1. X i, S 2 = 1 n. n 1. X i I(X i < c), T n =
Spring 2012 Math 541A Exam 1 1. (a) Let Z i be independent N(0, 1), i = 1, 2,, n. Are Z = 1 n n Z i and S 2 Z = 1 n 1 n (Z i Z) 2 independent? Prove your claim. (b) Let X 1, X 2,, X n be independent identically
More information1. Fisher Information
1. Fisher Information Let f(x θ) be a density function with the property that log f(x θ) is differentiable in θ throughout the open p-dimensional parameter set Θ R p ; then the score statistic (or score
More informationUNIVERZA NA PRIMORSKEM FAKULTETA ZA MATEMATIKO, NARAVOSLOVJE IN INFORMACIJSKE TEHNOLOGIJE. Ekstremne porazdelitve za odvisne spremenljivke
UNIVERZA NA PRIMORSKEM FAKULTETA ZA MATEMATIKO, NARAVOSLOVJE IN INFORMACIJSKE TEHNOLOGIJE Zaključna naloga Ekstremne porazdelitve za odvisne spremenljivke (Extremal Distributions for Dependent Variables)
More informationChapter 2: Fundamentals of Statistics Lecture 15: Models and statistics
Chapter 2: Fundamentals of Statistics Lecture 15: Models and statistics Data from one or a series of random experiments are collected. Planning experiments and collecting data (not discussed here). Analysis:
More informationExercises and Answers to Chapter 1
Exercises and Answers to Chapter The continuous type of random variable X has the following density function: a x, if < x < a, f (x), otherwise. Answer the following questions. () Find a. () Obtain mean
More informationLecture 13: Simple Linear Regression in Matrix Format. 1 Expectations and Variances with Vectors and Matrices
Lecture 3: Simple Linear Regression in Matrix Format To move beyond simple regression we need to use matrix algebra We ll start by re-expressing simple linear regression in matrix form Linear algebra is
More informationBrief Review on Estimation Theory
Brief Review on Estimation Theory K. Abed-Meraim ENST PARIS, Signal and Image Processing Dept. abed@tsi.enst.fr This presentation is essentially based on the course BASTA by E. Moulines Brief review on
More informationPh.D. Qualifying Exam Friday Saturday, January 6 7, 2017
Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a
More informationLinear Models and Estimation by Least Squares
Linear Models and Estimation by Least Squares Jin-Lung Lin 1 Introduction Causal relation investigation lies in the heart of economics. Effect (Dependent variable) cause (Independent variable) Example:
More informationUnbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
More informationSTAT 135 Lab 13 (Review) Linear Regression, Multivariate Random Variables, Prediction, Logistic Regression and the δ-method.
STAT 135 Lab 13 (Review) Linear Regression, Multivariate Random Variables, Prediction, Logistic Regression and the δ-method. Rebecca Barter May 5, 2015 Linear Regression Review Linear Regression Review
More informationPeter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
More informationFinal Examination Statistics 200C. T. Ferguson June 11, 2009
Final Examination Statistics 00C T. Ferguson June, 009. (a) Define: X n converges in probability to X. (b) Define: X m converges in quadratic mean to X. (c) Show that if X n converges in quadratic mean
More informationIntroduction to Estimation Methods for Time Series models. Lecture 1
Introduction to Estimation Methods for Time Series models Lecture 1 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 1 SNS Pisa 1 / 19 Estimation
More informationMultivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
More informationInverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1
Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is
More information. Find E(V ) and var(v ).
Math 6382/6383: Probability Models and Mathematical Statistics Sample Preliminary Exam Questions 1. A person tosses a fair coin until she obtains 2 heads in a row. She then tosses a fair die the same number
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
More informationFinal Exam. 1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given.
1. (6 points) True/False. Please read the statements carefully, as no partial credit will be given. (a) If X and Y are independent, Corr(X, Y ) = 0. (b) (c) (d) (e) A consistent estimator must be asymptotically
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 05 Full points may be obtained for correct answers to eight questions Each numbered question (which may have several parts) is worth
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationChapter 7. Hypothesis Testing
Chapter 7. Hypothesis Testing Joonpyo Kim June 24, 2017 Joonpyo Kim Ch7 June 24, 2017 1 / 63 Basic Concepts of Testing Suppose that our interest centers on a random variable X which has density function
More information2018 2019 1 9 sei@mistiu-tokyoacjp http://wwwstattu-tokyoacjp/~sei/lec-jhtml 11 552 3 0 1 2 3 4 5 6 7 13 14 33 4 1 4 4 2 1 1 2 2 1 1 12 13 R?boxplot boxplotstats which does the computation?boxplotstats
More informationMAS223 Statistical Inference and Modelling Exercises
MAS223 Statistical Inference and Modelling Exercises The exercises are grouped into sections, corresponding to chapters of the lecture notes Within each section exercises are divided into warm-up questions,
More informationTesting a Normal Covariance Matrix for Small Samples with Monotone Missing Data
Applied Mathematical Sciences, Vol 3, 009, no 54, 695-70 Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data Evelina Veleva Rousse University A Kanchev Department of Numerical
More informationLinear Models A linear model is defined by the expression
Linear Models A linear model is defined by the expression x = F β + ɛ. where x = (x 1, x 2,..., x n ) is vector of size n usually known as the response vector. β = (β 1, β 2,..., β p ) is the transpose
More informationSTA 2101/442 Assignment 3 1
STA 2101/442 Assignment 3 1 These questions are practice for the midterm and final exam, and are not to be handed in. 1. Suppose X 1,..., X n are a random sample from a distribution with mean µ and variance
More informationUnbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.
Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it
More informationMLES & Multivariate Normal Theory
Merlise Clyde September 6, 2016 Outline Expectations of Quadratic Forms Distribution Linear Transformations Distribution of estimates under normality Properties of MLE s Recap Ŷ = ˆµ is an unbiased estimate
More informationLIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
More informationStatistics Ph.D. Qualifying Exam: Part II November 3, 2001
Statistics Ph.D. Qualifying Exam: Part II November 3, 2001 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. 1 2 3 4 5 6 7 8 9 10 11 12 2. Write your
More informationEM Algorithm II. September 11, 2018
EM Algorithm II September 11, 2018 Review EM 1/27 (Y obs, Y mis ) f (y obs, y mis θ), we observe Y obs but not Y mis Complete-data log likelihood: l C (θ Y obs, Y mis ) = log { f (Y obs, Y mis θ) Observed-data
More informationPrinciples of Statistics
Part II Year 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2018 81 Paper 4, Section II 28K Let g : R R be an unknown function, twice continuously differentiable with g (x) M for
More informationGaussian Models (9/9/13)
STA561: Probabilistic machine learning Gaussian Models (9/9/13) Lecturer: Barbara Engelhardt Scribes: Xi He, Jiangwei Pan, Ali Razeen, Animesh Srivastava 1 Multivariate Normal Distribution The multivariate
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
More informationMathematics Ph.D. Qualifying Examination Stat Probability, January 2018
Mathematics Ph.D. Qualifying Examination Stat 52800 Probability, January 2018 NOTE: Answers all questions completely. Justify every step. Time allowed: 3 hours. 1. Let X 1,..., X n be a random sample from
More informationMEI Exam Review. June 7, 2002
MEI Exam Review June 7, 2002 1 Final Exam Revision Notes 1.1 Random Rules and Formulas Linear transformations of random variables. f y (Y ) = f x (X) dx. dg Inverse Proof. (AB)(AB) 1 = I. (B 1 A 1 )(AB)(AB)
More informationCanonical Correlation Analysis of Longitudinal Data
Biometrics Section JSM 2008 Canonical Correlation Analysis of Longitudinal Data Jayesh Srivastava Dayanand N Naik Abstract Studying the relationship between two sets of variables is an important multivariate
More informationStatistics 910, #15 1. Kalman Filter
Statistics 910, #15 1 Overview 1. Summary of Kalman filter 2. Derivations 3. ARMA likelihoods 4. Recursions for the variance Kalman Filter Summary of Kalman filter Simplifications To make the derivations
More informationSTAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
More informationA matrix over a field F is a rectangular array of elements from F. The symbol
Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F ) denotes the collection of all m n matrices over F Matrices will usually be denoted
More informationChapter 3 : Likelihood function and inference
Chapter 3 : Likelihood function and inference 4 Likelihood function and inference The likelihood Information and curvature Sufficiency and ancilarity Maximum likelihood estimation Non-regular models EM
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More informationREGRESSION WITH SPATIALLY MISALIGNED DATA. Lisa Madsen Oregon State University David Ruppert Cornell University
REGRESSION ITH SPATIALL MISALIGNED DATA Lisa Madsen Oregon State University David Ruppert Cornell University SPATIALL MISALIGNED DATA 10 X X X X X X X X 5 X X X X X 0 X 0 5 10 OUTLINE 1. Introduction 2.
More informationLecture 1: Introduction
Principles of Statistics Part II - Michaelmas 208 Lecturer: Quentin Berthet Lecture : Introduction This course is concerned with presenting some of the mathematical principles of statistical theory. One
More informationSTA 2201/442 Assignment 2
STA 2201/442 Assignment 2 1. This is about how to simulate from a continuous univariate distribution. Let the random variable X have a continuous distribution with density f X (x) and cumulative distribution
More informationBIOS 2083 Linear Models c Abdus S. Wahed
Chapter 5 206 Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter
More informationProperties of the least squares estimates
Properties of the least squares estimates 2019-01-18 Warmup Let a and b be scalar constants, and X be a scalar random variable. Fill in the blanks E ax + b) = Var ax + b) = Goal Recall that the least squares
More informationAn Introduction to Signal Detection and Estimation - Second Edition Chapter III: Selected Solutions
An Introduction to Signal Detection and Estimation - Second Edition Chapter III: Selected Solutions H. V. Poor Princeton University March 17, 5 Exercise 1: Let {h k,l } denote the impulse response of a
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationConvergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit
Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit Evan Kwiatkowski, Jan Mandel University of Colorado Denver December 11, 2014 OUTLINE 2 Data Assimilation Bayesian Estimation
More information1 Complete Statistics
Complete Statistics February 4, 2016 Debdeep Pati 1 Complete Statistics Suppose X P θ, θ Θ. Let (X (1),..., X (n) ) denote the order statistics. Definition 1. A statistic T = T (X) is complete if E θ g(t
More informationFor more information about how to cite these materials visit
Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
More informationMaster s Written Examination - Solution
Master s Written Examination - Solution Spring 204 Problem Stat 40 Suppose X and X 2 have the joint pdf f X,X 2 (x, x 2 ) = 2e (x +x 2 ), 0 < x < x 2
More informationGaussian vectors and central limit theorem
Gaussian vectors and central limit theorem Samy Tindel Purdue University Probability Theory 2 - MA 539 Samy T. Gaussian vectors & CLT Probability Theory 1 / 86 Outline 1 Real Gaussian random variables
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationLinear models. Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark. October 5, 2016
Linear models Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark October 5, 2016 1 / 16 Outline for today linear models least squares estimation orthogonal projections estimation
More informationST 740: Linear Models and Multivariate Normal Inference
ST 740: Linear Models and Multivariate Normal Inference Alyson Wilson Department of Statistics North Carolina State University November 4, 2013 A. Wilson (NCSU STAT) Linear Models November 4, 2013 1 /
More informationLast lecture 1/35. General optimization problems Newton Raphson Fisher scoring Quasi Newton
EM Algorithm Last lecture 1/35 General optimization problems Newton Raphson Fisher scoring Quasi Newton Nonlinear regression models Gauss-Newton Generalized linear models Iteratively reweighted least squares
More informationMiscellaneous Errors in the Chapter 6 Solutions
Miscellaneous Errors in the Chapter 6 Solutions 3.30(b In this problem, early printings of the second edition use the beta(a, b distribution, but later versions use the Poisson(λ distribution. If your
More informationHypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA
Hypothesis Testing Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA An Example Mardia et al. (979, p. ) reprint data from Frets (9) giving the length and breadth (in
More informationLinear Models Review
Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth
More informationBayesian Inference. Chapter 9. Linear models and regression
Bayesian Inference Chapter 9. Linear models and regression M. Concepcion Ausin Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master in Mathematical Engineering
More informationLecture 11. Multivariate Normal theory
10. Lecture 11. Multivariate Normal theory Lecture 11. Multivariate Normal theory 1 (1 1) 11. Multivariate Normal theory 11.1. Properties of means and covariances of vectors Properties of means and covariances
More informationLarge Sample Properties of Estimators in the Classical Linear Regression Model
Large Sample Properties of Estimators in the Classical Linear Regression Model 7 October 004 A. Statement of the classical linear regression model The classical linear regression model can be written in
More informationStochastic Design Criteria in Linear Models
AUSTRIAN JOURNAL OF STATISTICS Volume 34 (2005), Number 2, 211 223 Stochastic Design Criteria in Linear Models Alexander Zaigraev N. Copernicus University, Toruń, Poland Abstract: Within the framework
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationChapter 4 Euclid Space
Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,
More informationQualifying Exam in Probability and Statistics. https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf
Part : Sample Problems for the Elementary Section of Qualifying Exam in Probability and Statistics https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf Part 2: Sample Problems for the Advanced Section
More informationLecture 3. Inference about multivariate normal distribution
Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates
More information10. Linear Models and Maximum Likelihood Estimation
10. Linear Models and Maximum Likelihood Estimation ECE 830, Spring 2017 Rebecca Willett 1 / 34 Primary Goal General problem statement: We observe y i iid pθ, θ Θ and the goal is to determine the θ that
More informationWLS and BLUE (prelude to BLUP) Prediction
WLS and BLUE (prelude to BLUP) Prediction Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark April 21, 2018 Suppose that Y has mean X β and known covariance matrix V (but Y need
More informationECE 275A Homework 6 Solutions
ECE 275A Homework 6 Solutions. The notation used in the solutions for the concentration (hyper) ellipsoid problems is defined in the lecture supplement on concentration ellipsoids. Note that θ T Σ θ =
More informationEconomics 573 Problem Set 5 Fall 2002 Due: 4 October b. The sample mean converges in probability to the population mean.
Economics 573 Problem Set 5 Fall 00 Due: 4 October 00 1. In random sampling from any population with E(X) = and Var(X) =, show (using Chebyshev's inequality) that sample mean converges in probability to..
More informationF2E5216/TS1002 Adaptive Filtering and Change Detection. Course Organization. Lecture plan. The Books. Lecture 1
Adaptive Filtering and Change Detection Bo Wahlberg (KTH and Fredrik Gustafsson (LiTH Course Organization Lectures and compendium: Theory, Algorithms, Applications, Evaluation Toolbox and manual: Algorithms,
More information