Maximum Likelihood Estimation
|
|
- Hilary Randall
- 6 years ago
- Views:
Transcription
1 ECE 645: Estimatio Theory Sprig 2015 Istructor: Prof. Staley H. Cha Maximum Likelihood Estimatio (LaTeX prepared by Shaobo Fag) April 14, 2015 This lecture ote is based o ECE 645(Sprig 2015) by Prof. Staley H. Cha i the School of Electrical ad Computer Egieerig at Purdue Uiversity. 1 Itroductio For may families besides expoetial family, Miimum Variace Ubiased Estimator (MVUE) could be very difficult to fid, or it may ot eve exist. For such models, we eed a alterative method to obtai good estimators. With the absece of the prior iformatio, the maximum likelihood estimatio might be a viable alterative. (Poor IV.D) Defiitio 1. Maximum likelihood estimate (MLE) The maximum likelihood estimator is defied as: θ ML (y) def = argmax θ f θ (y) (1) where f θ (y) = f Y (y;θ). Here, the fuctio f θ (y) is called the likelihood fuctio. We ca also take log o f θ (y) ad yield the same maximizer: θ ML (y) def =argmax logf θ (y). (2) θ The fuctio logf θ (y) is called the log-likelihood fuctio. Example 1. Let Y = [Y 1,...,Y ] be a sequece of iid radom variables such that Assume that σ 2 is kow, fid θ ML for µ. Solutio: First of all, the likelihood fuctio is f θ (y) = Y k N(µ,σ 2 ). ( 1 exp 1 (2πσ 2 ) /2 2σ 2 ) (y k µ) 2. Takig the log o both sides we have the log-likelihood fuctio logf θ (y) = 1 2σ 2 (y k µ) 2 2 log(2πσ2 ). I order to fid the maximizer of the log-likelihood fuctio, we take the first order derivative ad set it to zero. This yields µ ML (y) = 1 y k.
2 We ca also show that E[ µ ML (Y )] = 1 which says that the estimator is ubiased. E[Y k ] = µ, Example 2. Nowwecosiderthepreviousexamplewithbothµadσ ukow. Ourgoalistodetermie θ def ML = [ θ 1, θ 2 ] T for θ 1 = µ ad θ 2 = σ 2. Solutio: Same as the previous problem, the log-likelihood fuctio is logf θ (y) = 1 2θ 2 Takig the partial derivative wrt to θ 1 yields which gives (y k θ 1 ) 2 2 log(2πθ 2). θ 1 logf θ (y) = 1 2θ 2 θ 1 (y) = 1 Similarly, takig the partial derivative wrt to θ 2 yields which gives θ 2 logf θ (y) = 1 θ 2 2 θ 2 (y) = 1 Note that E[ θ 2 (Y )] = 1 σ2 σ 2. So θ 2 is biased. 2(y k θ 1 ) = 0, y k. 2(y k θ 1 ) 2 2θ 2 = 0, (y k θ 1 ) 2. Remark: I order to obtai a ubiased estimator for the populatio variace, it is preferred to use the sample variace defie as (Zwilliger 1995, p. 603): S 1 = 1 1 (Y i Y) 2, where Y = 1 Y i is the sample mea. I fact, the fuctio var i MATLAB is the sample variace. Example 3. Beroulli (Statistical Iferece: Example 7.2.7, Casella ad Berger) Let Y = [Y 1,...,Y ] be a sequece of i.i.d. Beroulli radom variables of parameter θ. We would like to fid the MLE θ ML for θ. Solutio: 2
3 First of all, we defie the likelihood fuctio: f θ (y) = θ y k (1 θ) 1 y k. Lettig y = y k, we ca rewrite the likelihood fuctio as Hece, the log-likelihood fuctio is Takig the derivative ad settig it to zero yields f θ (y) = θ y (1 θ) 1 y. logf θ (y) = ylogθ +( y)log(1 θ). θ logf θ(y) = y θ y 1 θ = 0. Therefore, θ ML (y) is θ ML (y) = y k. Example 4. Biomial Let Y = [Y 1,...,Y ] be a sequece of iid radom variables of a Biomial distributio of biomial(k,θ). We would like to fid θ ML for θ. Solutio: The likelihood fuctio is f θ (y) = ( ) k θ yi (1 θ) k yi. By lettig y = y i, we ca rewrite the likelihood fuctio as: The log-likelihood fuctio is y i ( ) k f θ (y) = θ y (1 θ) 1 y. logf θ (y) = ylogθ +(k y)log(1 θ)+ Takig the first order derivative ad settig to zero yields θ ML (y) = y k = 1 k y i ( ) k log y i }{{} This term does ot cotai θ y i. Example 5. Poisso Let Y = [Y 1,...,Y ] be a sequece of i.i.d. Poisso radom variables of parameter λ. Recall that Poisso distributio is: P(Y i = y i ) = e λ λ y i y i!. We would like to fid λ ML for parameter λ. 3
4 Solutio: Similarly as i previous examples, first we fid the likelihood fuctio: Thus the log-likelihood fuctio is logf λ (y) = λ+ Settig the first-order derivative to 0 yields f λ (y) = e λ λ yi y i! = e λ λ yi y. i! y i logθ λ ML (y) = 1 log y i! }{{} This term does ot cotai λ y i. 2 Bias v.s. Variace I geeral, MLE could be both biased or ubiased. To take a closer look at this property, we write the MLE as a sum of bias ad variace terms as below: MSE θ = E Y [( θ ML (Y ) θ) 2 ] = E[( θ ML E[ θ ML ]+E[ θ ML ] θ) 2 ] = E[( θ ML E[ θ ML ]) 2 ]+E[(E[ θ ML ] θ) 2 ]+2E[( θ ML E[ θ ML ])(E[ θ ML ] θ)] = E Y [( θ ML (Y ) E[ θ ML (Y )]) 2 ] +(E[ θ ML (Y )] θ) 2. }{{}}{{} variace bias (3) Example 6. Image Deoisig Let z be a clea sigal ad let be a oise vector such that N(0,σ 2 I). Suppose that we are give the oisy observatio y = z +, our goal is to estimate z from y. I this example, let us cosidera liear deoisigmethod. We wouldliketo fid aw suchthat the estimator ẑ = Wy would be optimal i some sese. We shall call W as a smoothig filter. To determie what W would be good, we first cosider the MSE: MSE = E[ ẑ z 2 ] = E[ Wy z 2 ] = E[ W(z +) z 2 ] = E[ (W I)z +W 2 ] = (W I)z 2 +E[ W 2 ] }{{}}{{} bias variace 4
5 Now, by usig eige-decompostio we ca write W as W = UΛU T. The, the bias ca be computed as bias = (W I)z 2 = E[ ẑ z 2 ] = (UΛU T I)z 2 = U(Λ I)U T z 2 = z T U(Λ I) 2 U T z = (λ i 1) 2 vi 2, where v = U T z. Similarly, the variace ca be computed as Therefore, the MSE ca be writte as: variace = E[ W 2 ] MSE = = E[ T W T W] { } = σ 2 Tr W T W = σ 2 λ 2 i (λ i 1) 2 vi 2 +σ2 To miimize MSE, λ i should be chose such that which is λ 2 i λ i MSE = 2v 2 i (λ i 1)+2σ 2 λ i = 0, λ i = v2 i v 2 i +σ2. Thus far we have come across may examples where the estimators are ubiased. So are biased estimators bad? The aswer is o. Here is a example. Let us cosider a radom variable Y N(0,σ 2 ). Now, cosider the followig two estimators: Estimator 1: θ 1 (Y) = Y 2. The E[ θ 1 (Y)] = E[Y 2 ] = σ 2, thus it is ubiased. Estimator 2: θ 2 (Y) = ay 2,a 1, the E[ θ 2 (Y)] = aσ 2. Thus it is biased. Let us ow cosider the MSE of θ 2. (Note that the MSE of θ 1 ca be foud by lettig a = 1.) Therefore, the MSE attais its miimum at: which is MSE = E[( θ 2 (Y) σ 2 ) 2 ] = E[(aY 2 σ 2 ) 2 ] = E[a 2 Y 4 ] 2σ 2 E[aY 2 ]+σ 4 = 3a 2 σ 4 2aσ 4 +σ 4 = σ 4 (3a 2 2a+1) a MSE = σ4 (6a 2) = 0, a = 1 3. This result says: although θ 2 is biased, it actually attais a lower MSE! 5
6 3 Fisher Iformatio 3.1 Variace ad Curvature of log-likelihood For ubiased estimators, the variace ca provide extremely importat iformatio about the performace of the estimators. I order to study the variace more carefully, we first study its relatioship with regard to the log-likelihood as demostrated i the example below. Example 7. Let Y N(θ,σ 2 ), where σ is kow. Accordigly, f θ (y) = 1 θ)2 exp( (y 2πσ 2 2σ 2 ) logf θ (y) = log 2πσ 2 1 θ)2 2σ2(y logf θ(y) θ = 1 σ2(y θ) 2 logf θ (y) } θ {{ 2 = 1 } σ 2 curvature of log-likelihood Therefore, as σ 2 icreases, we ca easily coclude that 2 θ 2 logf θ (y) will decrease. Thus, we coclude that with the variace icreasig, the curvature will be decreasig. 3.2 Fisher-Iformatio Defiitio 2. Fisher Iformatio The Fisher-iformatio is defied as: [ I(θ) def 2 ] logf θ (Y ) = E Y θ 2, (4) where [ 2 ] logf θ (Y ) 2 logf θ (y) E Y θ 2 = θ 2 f θ (y)dy (5) We will try to estimate the fisher iformatio i the examples below. Example 8. Let Y = [Y 1,...,Y ] be a sequece of iid radom variables such that Y i N(θ,σ 2 ). We would like to determie I(θ). First, we kow that the log-likelihood is The first order derivative is logf θ (y) = 2 log(2πσ2 ) logf θ (y) θ = σ2(y θ), (y i θ) 2σ 2. 6
7 where Cosequetly, the secod order derivative is Fially, the fisher iformatio is y = 1 y i. 2 logf θ (y) θ 2 I(θ) = E Y [ 2 logf θ (Y ) θ 2 = σ 2. ] = E Y [ σ ] 2 = σ 2. Example 9. Let Y = [Y 1,...,Y ] be a sequece of iid radom variables such that where N k N(0,σ 2 ). Fid I(θ). The likelihood fuctio is f θ (y) = Y k = Acos(w 0 k +θ)+n k, 1 exp( (y i Acos(w 0 k +θ)) 2 2πσ 2 2σ 2 ) 1 1 = exp( /2 2πσ 2 2σ 2 (y i Acos(w 0 k +θ)) 2 ) The, the first order derivative of the log-likelihood is [ ] θ logf θ(y) = 1 θ σ 2 (y i Acos(w 0 k +θ)) 2 = 1 σ 2 (y i Acos(w 0 k +θ))(asi(w 0 k +θ)) The secod order derivative is = A σ 2 2 θ 2 logf θ(y) = A σ 2 (y i si(w 0 k +θ) A 2 si(2w 0k +2θ)) [y i cos(w 0 k +θ) Acos(2w 0 k +2θ)] Accordigly, the E Y [ 2 θ logf 2 θ (Y )] ca be estimated as below: [ ] 2 E Y θ 2 logf θ(y ) = A σ 2 (E[Y i ]cos(w 0 k +θ) Acos(2w 0 k +2θ)) = A 2σ 2 = A2 σ 2 [Acos 2 (w 0 k +θ) Acos(2w 0 k +2θ)] ( cos(2w 0k +2θ) cos(2w 0 k +2θ)) = A2 2σ 2 + A2 2σ 2 1 cos(2w 0 k +2θ) 7
8 By usig the fact that 1 cos(2w 0k +2θ) 0, we have: I(θ) = A2 2σ Fisher-Iformatio ad KL Divergece There is a iterestig relatioship betwee the Fisher-Iformatio ad the KL divergece, which we shall ow discuss. To begi with, let us first list out two assumptios. Assumptio: θ θ f θ (y)dy = θ(y)f θ (y)dy = θ f θ(y)dy θ f θ(y) θ(y)dy Basically, the two assumptios say that we ca iterchage the order of itegratio ad the differetiatio. If the assumptio holds, we ca show the followig result: [ (logfθ ) ] 2 (Y) I(θ) = E Y. (6) θ Proof. By the assumptios ad itegratio by part, we have [ ] ( I(θ) = E Y 2 logf θ (Y) f θ 2 = = f θ (y)dy 1 + f }{{} θ (y) (f θ (y))2 dy = 0 by (1) θ (y)f θ(y) (f θ (y))2 f 2 θ (y) ( ) [ 2 (logfθ ) ] 2 logfθ (y) (y) = f θ (y) dy = E Y. θ θ ) f θ (y)dy The followig propositio liks KL divergece ad I(θ). Propositio 1. Let θ = θ 0 +δ for some small deviatio δ, the D(f θ0 f θ ) I(θ 0) 2 (θ θ 0) 2 +O(θ θ 0 ) 3. (7) Iterpretatio: If I(θ) is large, the D(f θ0 f θ ) is large. Accordigly, it would be easier to differetiate θ 0 ad θ. Proof. First, recall that the KL divergece is defied as D(f θ0 f θ ) = f θ0 (y)log f θ 0 (y) f θ (y) dy. 8
9 Cosider Taylor expasio o θ 0, we estimate the first two terms as below. First-order derivative: θ D(f θ 0 f θ ) = f θ0 (y) θ=θ0 θ [logf θ 0 (y) logf θ (y)] dy θ=θ0 [ ] 1 = f θ0 (y) f θ (y) θ f dy θ(y) θ=θ 0 = θ f θ(y)dy = f θ (y)dy = 0. θ (8) Secod-order derivative: 2 θ 2D(f θ 0 f θ ) θ=θ0 = f θ0 (y) 2 θ 2 [ logf θ(y)]dy ] = E [ 2 θ 2 logf θ(y) = I(θ 0 ). θ=θ 0 (9) Substitute the above terms ito Taylor expasio ad igore the higher order terms: D(f θ0 f θ ) = D(f θ0 f θ0 )+(θ θ 0 ) θ D(f θ 0 f θ )+ (θ θ 0) 2 = (θ θ 0) 2 I(θ 0 )+O(θ θ 0 ) θ 2D(f θ 0 f θ )+O(θ θ) 3 (10) 4 Cramer-Rao Lower Boud (CRLB) Theorem The CRLB is a fudametal result that characterizes the performace of a estimator. Theorem 1. Uder the assumptios (1) ad (2): Var( θ(y)) ( θ E[ θ(y)]) 2 I(θ) (11) for ay estimator θ(y). Proof. To prove the iequality, we first ote that Lettig Var( θ(y))i(θ) = ) 2fθ ( ) 2 ( θ(y) E[ θ(y)] (y)dy θ logf θ(y) f θ (y)dy. A = θ(y) E[ θ(y)], B = θ logf θ(y), 9
10 the above equatio ca be simplified as Var( θ(y))i(θ) = E[A 2 ]E[B 2 ] E[AB] 2, where the iequality is due to Cauchy. We ca also show that [ E[AB] 2 = [ = [ = = ( θ(y) E[ θ(y)])( ] 2 θ logf θ(y))f θ (y)dy ( θ(y) E[ θ(y)]) ] 2 θ f θ(y)dy θ(y) θ f θ(y)dy E[ θ(y)] θ f θ(y)dy [ ] 2 ( ) 2 θ E[ θ(y)] 0 = θ E[ θ(y)]. ] 2 Propositio 2. A estimator θ(y) achieves CRLB equality if ad oly if θ(y) is a sufficiet statistic of a oe-parameter expoetial family. Proof. Suppose that CRLB equality holds, the we must have for some fuctio k(θ). This implies that Thus, logf θ (y) = θ logf θ(y) = k(θ)( θ(y) E[ θ(y)]), θ = a θ which is a oe-parameter expoetial family. k(θ )( θ(y) E[ θ(y)])dθ +H(y) k(θ )E[ θ(y)]dθ } a {{ } log C(θ) + H(y) }{{} log h(y) f θ (y) = C(θ)exp(Q(θ) θ(y)) h(y), θ + θ(y) k(θ )dθ. } a {{ } Q(θ) Coversely, suppose that θ(y) is a sufficiet statistic of a oe-parameter expoetial family, the, f θ (y) = C(θ)exp(Q(θ)T(y)) h(y), where T(y) = θ(y), ad ( C(θ) = exp(q(θ)t(y)) h(y)dy) 1. I order to show that Var{T(Y)} attais the CRLB, we eed to obtai the Fisher Iformatio: { ( ) } 2 I(θ) = E θ logf θ(y). 10
11 Note that sice logf θ (y) = Q(θ)T(y)+logh(y) log exp(q(θ)t(y))h(y)dy, we must have ( θ logf θ(y) = Q (θ)t(y) = Q (θ){t(y) E{T(Y)}}. ) exp(q(θ)t(y))h(y) T(y) dy Q (θ) exp(q(θ)t(y))h(y)dy Therefore, The Cramer Rao Lower boud is ( ) 2 I(θ) = E θ logf θ(y) = (Q (θ)) 2 Var{T(Y)}. Var( θ(y)) ( θ E[ θ(y)]) 2. I(θ) Thus we eed to determie ( θ E[ θ(y)]) 2. Suppose that θ(y) = T(Y), Therefore, θ {E{ θ(y)}} = T(y)exp(Q(θ)T(y))h(y)dy θ exp(q(θ)t(y))h(y)dy T(y) 2 exp(q(θ)t(y))h(y)dy exp(q(θ)t(y))h(y)dy ( T(y)exp(Q(θ)T(y))h(y)dy) 2 = Q (θ) = Q (θ)var{t(y)}. ( exp(q(θ)t(y))h(y)dy) 2 ( θ E[ θ(y)]) 2 I(θ) = Q (θ) 2 Var{T(Y)} 2 (Q (θ)) 2 Var{T(Y)} = Var(T(Y)) = Var( θ(y)), which shows that CRLB equality is attaied. Example 10. Let Y = [Y 1,...,Y ] be a sequece of iid radom variables such that Y i N(θ,σ 2 ). Cosider the estimator θ(y ) = 1 Y i. Is θ(y ) a MVUE? Solutio: The CRLB is Var( θ) ( θ E[ θ(y )]) 2, I(θ) where it is ot difficult to show that I(θ) = σ 2 ad E[Y i ] = θ. Therefore, CRLB becomes Var( θ) 1 I(θ) = σ2. 11
12 O the other had, we ca show that Var( θ) = Var ( 1 ) Y i = 1 2 Var(Y i ) = σ2 = 1 I(θ), which meas that CRLB equality is achieved. Therefore, the estimator is a MVUE. θ(y ) = 1 Y i Example 11. Let Y = [Y 1,...,Y ] be a sequece of iid radom variables such that Y k s k (θ) + N k where s k (θ) is a fuctio of k, ad N k N(0,σ 2 ). Fid CRLB for ay ubiased estimator θ. Solutio: The log-likelihood is Cosequetly, we ca show that 2 θ 2 logf θ(y) = 1 σ 2 logf θ (y) = 2 log(2πσ2 ) 1 2σ 2 (y i s k (θ)) 2 [ ] (y k s k (θ)) 2 s k (θ) θ 2 1 ( ) 2 sk (θ) σ 2. θ Accordigly, Therefore, ] E [ 2 θ 2 θ(y ) = 1 ( ) 2 sk (θ) σ 2. θ Var( θ) 1 I(θ) = σ 2 ( θ s k(θ)) 2. For example, if s k = θ, the Var( θ) σ2. If s k(θ) = Acos(w 0 k +θ), the Var( θ) 2σ2 A 2. 12
Unbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationAn Introduction to Signal Detection and Estimation - Second Edition Chapter IV: Selected Solutions
A Itroductio to Sigal Detectio Estimatio - Secod Editio Chapter IV: Selected Solutios H V Poor Priceto Uiversity April 6 5 Exercise 1: a b ] ˆθ MAP (y) = arg max log θ θ y log θ =1 1 θ e ˆθ MMSE (y) =
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationLecture 13: Maximum Likelihood Estimation
ECE90 Sprig 007 Statistical Learig Theory Istructor: R. Nowak Lecture 3: Maximum Likelihood Estimatio Summary of Lecture I the last lecture we derived a risk (MSE) boud for regressio problems; i.e., select
More informationECE 901 Lecture 13: Maximum Likelihood Estimation
ECE 90 Lecture 3: Maximum Likelihood Estimatio R. Nowak 5/7/009 The focus of this lecture is to cosider aother approach to learig based o maximum likelihood estimatio. Ulike earlier approaches cosidered
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationLast Lecture. Unbiased Test
Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationMATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1
MATH 2: HOMEWORK 6 SOLUTIONS CA PRO JIRADILOK Problem. If s = 2, ad Problem : Rudi, Chapter 3, Problem 3. s + = 2 + s ( =, 2, 3,... ), prove that {s } coverges, ad that s < 2 for =, 2, 3,.... Proof. The
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationStatistics for Applications. Chapter 3: Maximum Likelihood Estimation 1/23
18.650 Statistics for Applicatios Chapter 3: Maximum Likelihood Estimatio 1/23 Total variatio distace (1) ( ) Let E,(IPθ ) θ Θ be a statistical model associated with a sample of i.i.d. r.v. X 1,...,X.
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationG. R. Pasha Department of Statistics Bahauddin Zakariya University Multan, Pakistan
Deviatio of the Variaces of Classical Estimators ad Negative Iteger Momet Estimator from Miimum Variace Boud with Referece to Maxwell Distributio G. R. Pasha Departmet of Statistics Bahauddi Zakariya Uiversity
More informationLecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationEstimation Theory Chapter 3
stimatio Theory Chater 3 Likelihood Fuctio Higher deedece of data PDF o ukow arameter results i higher estimatio accuracy amle : If ˆ If large, W, Choose  P  small,  W POOR GOOD i Oly data samle Data
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationChapter 7 Maximum Likelihood Estimate (MLE)
Chapter 7 aimum Likelihood Estimate (LE) otivatio for LE Problems:. VUE ofte does ot eist or ca t be foud . BLUE may ot be applicable ( Hθ w) Solutio: If the PDF
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationHomework for 2/3. 1. Determine the values of the following quantities: a. t 0.1,15 b. t 0.05,15 c. t 0.1,25 d. t 0.05,40 e. t 0.
Name: ID: Homework for /3. Determie the values of the followig quatities: a. t 0.5 b. t 0.055 c. t 0.5 d. t 0.0540 e. t 0.00540 f. χ 0.0 g. χ 0.0 h. χ 0.00 i. χ 0.0050 j. χ 0.990 a. t 0.5.34 b. t 0.055.753
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationQuestions and Answers on Maximum Likelihood
Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,
More informationLecture 3: MLE and Regression
STAT/Q SCI 403: Itroductio to Resamplig Methods Sprig 207 Istructor: Ye-Chi Che Lecture 3: MLE ad Regressio 3. Parameters ad Distributios Some distributios are idexed by their uderlyig parameters. Thus,
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationThe Bilateral Laplace Transform of the Positive Even Functions and a Proof of Riemann Hypothesis
The Bilateral Laplace Trasform of the Positive Eve Fuctios ad a Proof of Riema Hypothesis Seog Wo Cha Ph.D. swcha@dgu.edu Abstract We show that some iterestig properties of the bilateral Laplace trasform
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationREVIEW 1, MATH n=1 is convergent. (b) Determine whether a n is convergent.
REVIEW, MATH 00. Let a = +. a) Determie whether the sequece a ) is coverget. b) Determie whether a is coverget.. Determie whether the series is coverget or diverget. If it is coverget, fid its sum. a)
More informationMa 4121: Introduction to Lebesgue Integration Solutions to Homework Assignment 5
Ma 42: Itroductio to Lebesgue Itegratio Solutios to Homework Assigmet 5 Prof. Wickerhauser Due Thursday, April th, 23 Please retur your solutios to the istructor by the ed of class o the due date. You
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationMaximum Likelihood Estimation and Complexity Regularization
ECE90 Sprig 004 Statistical Regularizatio ad Learig Theory Lecture: 4 Maximum Likelihood Estimatio ad Complexity Regularizatio Lecturer: Rob Nowak Scribe: Pam Limpiti Review : Maximum Likelihood Estimatio
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture Stat Maximum Likelihood Estimation
Lecture Stat 461-561 Maximum Likelihood Estimatio A.D. Jauary 2008 A.D. () Jauary 2008 1 / 63 Maximum Likelihood Estimatio Ivariace Cosistecy E ciecy Nuisace Parameters A.D. () Jauary 2008 2 / 63 Parametric
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More informationIn this section we derive some finite-sample properties of the OLS estimator. b is an estimator of β. It is a function of the random sample data.
17 3. OLS Part III I this sectio we derive some fiite-sample properties of the OLS estimator. 3.1 The Samplig Distributio of the OLS Estimator y = Xβ + ε ; ε ~ N[0, σ 2 I ] b = (X X) 1 X y = f(y) ε is
More informationRiemann Sums y = f (x)
Riema Sums Recall that we have previously discussed the area problem I its simplest form we ca state it this way: The Area Problem Let f be a cotiuous, o-egative fuctio o the closed iterval [a, b] Fid
More informationRegression and generalization
Regressio ad geeralizatio CE-717: Machie Learig Sharif Uiversity of Techology M. Soleymai Fall 2016 Curve fittig: probabilistic perspective Describig ucertaity over value of target variable as a probability
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More informationSection 14. Simple linear regression.
Sectio 14 Simple liear regressio. Let us look at the cigarette dataset from [1] (available to dowload from joural s website) ad []. The cigarette dataset cotais measuremets of tar, icotie, weight ad carbo
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationSOLUTION FOR HOMEWORK 7, STAT np(1 p) (α + β + n) + ( np + α
SOLUTION FOR HOMEWORK 7, STAT 6331 1 Exerc733 Here we just recall that MSE(ˆp B ) = p(1 p) (α + β + ) + ( p + α 2 α + β + p) 2 The you plug i α = β = (/4) 1/2 After simplificatios MSE(ˆp B ) = 4( 1/2 +
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationEstimation of the Mean and the ACVF
Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators
More informationSolutions to Homework 2 - Probability Review
Solutios to Homework 2 - Probability Review Beroulli, biomial, Poisso ad ormal distributios. A Biomial distributio. Sice X is a biomial RV with parameters, p), it ca be writte as X = B i ) where B,...,
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More informationMath 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency
Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationDouble Stage Shrinkage Estimator of Two Parameters. Generalized Exponential Distribution
Iteratioal Mathematical Forum, Vol., 3, o. 3, 3-53 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.9/imf.3.335 Double Stage Shrikage Estimator of Two Parameters Geeralized Expoetial Distributio Alaa M.
More informationThe Sampling Distribution of the Maximum. Likelihood Estimators for the Parameters of. Beta-Binomial Distribution
Iteratioal Mathematical Forum, Vol. 8, 2013, o. 26, 1263-1277 HIKARI Ltd, www.m-hikari.com http://d.doi.org/10.12988/imf.2013.3475 The Samplig Distributio of the Maimum Likelihood Estimators for the Parameters
More informationLecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting
Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationPROBLEM SET 5 SOLUTIONS 126 = , 37 = , 15 = , 7 = 7 1.
Math 7 Sprig 06 PROBLEM SET 5 SOLUTIONS Notatios. Give a real umber x, we will defie sequeces (a k ), (x k ), (p k ), (q k ) as i lecture.. (a) (5 pts) Fid the simple cotiued fractio represetatios of 6
More information2. The volume of the solid of revolution generated by revolving the area bounded by the
IIT JAM Mathematical Statistics (MS) Solved Paper. A eigevector of the matrix M= ( ) is (a) ( ) (b) ( ) (c) ( ) (d) ( ) Solutio: (a) Eigevalue of M = ( ) is. x So, let x = ( y) be the eigevector. z (M
More informationLecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound
Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More information