4. Understand the key properties of maximum likelihood estimation. 6. Understand how to compute the uncertainty in maximum likelihood estimates.
|
|
- Tamsyn Porter
- 5 years ago
- Views:
Transcription
1 9.07 Itroductio to Statistics for Brai ad Cogitive Scieces Emery N. Brow Lecture 9: Likelihood Methods I. Objectives 1. Review the cocept of the joit distributio of radom variables.. Uderstad the cocept of the likelihood fuctio. 3. Uderstad the cocept of maximum likelihood estimatio. 4. Uderstad the key properties of maximum likelihood estimatio. 5. Uderstad the cocept of Fisher iformatio. 6. Uderstad how to compute the ucertaity i maximum likelihood estimates. II. Joit Probability Desity or Distributio of a Set of Radom Variables We recall that two evets E 1 ad E are idepedet if Ituitively, this statemet says that kowledge about E 1 geeral a set of evets E 1, K, E is idepedet if Pr( E 1 I E ) = Pr( E 1 ) Pr( E ) (9.1) Pr( E 1 I E K I gives o iformatio about E. I E ) = Pr( E i ) (9.) It follows from Lecture 1 ad Lecture 5 that if x= x 1, K, x is a sample of idepedet, idetically distributed observatios from a pmf or a pdf f( x i θ) the the joit probability desity of x= x 1, K,x is f( x θ ) = f( x i θ ). (9.3) Equatio 9.3 follows directly from the defiitio of a pmf for a discrete radom variable. Usig a cdf of a cotiuous radom variable which is differetiable, it is also easy to show. Equatio 9.3 is essetial for our defiitio of the likelihood fuctio of a sample of idepedet observatios. III. The Likelihood Fuctio A. Defiitio Defiitio 9.1 (Likelihood Fuctio). Give x= x1, K,x is a idepedet, idetically distributed sample from a pdf or pmf f( xi θ ). Let f( x θ ) deote the joit pdf or pmf of the sample x = ( x, K, x ) defied i Eq Give X = x is observed, the fuctio of θ defied as 1
2 page : 9.07 Lecture 9: Likelihood Methods L( θ ) = f ( x θ ) = f ( x k θ ) (9.4) k =1 is called the likelihood fuctio for the parameter θ. The likelihood fuctio or likelihood provides a objective meas of assessig the iformatio i a sample of data about the model parameter θ. We view the data as fixed ad ow study L( θ ) = f ( x θ ) as a fuctio ofθ. For a give model it summarizes all the iformatio i the sample about θ. Use of the likelihood will allow us to overcome some of the ambiguities we faced with the method-of-momets. B. Examples of Likelihoods Example.1 (cotiued). Biomial Likelihood Fuctio. I our learig experimet, we observed correct resposes from the aimal i 40 trials. We derived the pmf of the observatios i 40 trials as the biomial distributio f ( p) = p (1 p). (9.5) (Notice that this distributio does ot have exactly the same form as Eq. 9.3, because we allowed for all the possible ways the sample could be realized. It turs out that i our likelihood aalysis, this distictio will ot matter because the biomial coefficiet does ot alter the likelihood). Sice the ukow parameter is p we have the likelihood of p is Lp ( ) = f ( p) = p (1 p). (9.6) I geeral, for the biomial pmf with trials ad probability of a correct respose p, the likelihood x x Lp ( )= f( x p ) = p ( 1 p). (9.7) x
3 page 3: 9.07 Lecture 9: Likelihood Methods Figure 9A. Biomial Likelihood Fuctio L(p) for = 40 ad k = 5, 10,, 30, 35. Plottig these elemetary likelihood fuctios is useful to gai isight ito what type of iformatio they provide. We ca predict the shape of the biomial likelihood fuctio based o discussios of the beta distributio i Lecture 3 (Eq. 3.30, Figure 3J). Example 3. Quatile Release Hypothesis (Poisso Likelihood). We used a Poisso pmf to model the quatile release of acetylcholie at the syapse. If we record the umber of quata released i o-overlappig time itervals of a specified legth, the the likelihood fuctio for the Poisso parameter λ, which is the expected umber of quata released, is x λ i λ e L ( λ )= f ( x λ ) = x i! λ = λ x i (9.8) e x i! We see i this example, a very importat feature of the likelihood. Namely, that it has reduced or summarized the data from all observatios to simply S = x i, the sum of the observatios. For a large class of probability models, this is a typical feature.
4 page 4: 9.07 Lecture 9: Likelihood Methods Figure 9B Poisso Likelihood Fuctios L( λ ). The shape of the Poisso likelihood could be predicted based o results i Lecture 3. Note that the essetial features of this likelihood are described by S λ L( λ ) λ e (9.9) which is proportioal to a gamma probability desity (Eq. 3.6) with α = (S 1) ad β = (Figure 3I). Example 3. (cotiued) Magetoecephalogram Noise Data (Gaussia Likelihood Fuctio). If we view the MEG data as a radom sample from a Gaussia probability model the i this problem there are two ukow parameters: µ ad σ.
5 page 5: 9.07 Lecture 9: Likelihood Methods L ( µ, σ ) = f( x µ,σ ) = f( x µ,σ ) i i 1 1 (x µ) = exp πσ σ (9.10) 1 (x i µ) = exp πσ σ If we expad the sum i the expoet, we see that the data have bee compressed ito two distict terms: x i ad x i. That is, we ca rewrite Eq as x i + x i log(πσ ) + 1 µ µ L( µ, σ ) = exp σ σ σ A crucial feature is that the coefficiets o these two statistics ivolve the ukow parameters. This data reductio is a importat cosequece of the likelihood aalysis. These statistics are called sufficiet statistics i that it is possible to show that they cotai all the iformatio i the data about the parameters µ ad σ. We state this cocept formally. Defiitio 9. A statistic T( X ) is sufficiet for a parameter θ if the probability desity of the data coditioal o the statistic is idepedet of θ. That is, if f( X1,..,. X T( X ),θ) = h( X ), the T( X) is sufficiet. If there are two parameters, the the smallest umber of sufficiet statistics is two. Similarly, i the case of the Poisso likelihood above, the umber of sufficiet statistics was oe cosistet with the fact that there is oly oe parameter to estimate. There are very precise theorems that tell us how to idetify the sufficiet statistics for parameters i a probability model. We will ot discuss them here. Details ca be foud i Pawita (001), DeGroot ad Schervish (00) ad i Rice (007). Likelihood aalyses i geeral are based o the sufficiet statistics. Whe method-ofmomets ad likelihood aalyses differ, it is usually because the sample momets are ot the sufficiet statistics for the parameters to be estimated. We will illustrate this poit below whe we defie maximum likelihood estimatio. C. Maximum Likelihood Estimatio Ideally, we would like to aalyze the likelihood fuctio for every problem. This gets to be especially challegig whe the umber of parameters is large. Therefore, we would like to derive a summary that gives us a sese about the shape of the likelihood. This is what the maximum likelihood estimate ad the Fisher iformatio provide. We describe them i tur.
6 page 6: 9.07 Lecture 9: Likelihood Methods Defiitio 9.3. Maximum Likelihood Estimatio. For each sample poit x = ( x 1,..., x ) let θˆ( x ) be a parameter value of which L( θ ) = L( θ x ) attais a maximum as a fuctio of θ for fixed x. ˆ θ ( x ) is a maximum likelihood (ML) estimator (MLE) of the parameter θ. I particular, if L( θ ) is differetiable, we ca cosider L ( θ) θ = 0 ad check the coditios L( θ ) o to be sure that the estimate defies a maximum. Remember that this would mea θ that for a oe-dimesioal problem, verifyig that the secod derivative was egative ad for the d dimesioal problem verifyig the coditio that the determiat of the Hessia is egative defiite. I likelihood aalyses, it is usually easier to work with log L( θ ) istead of L ( θ ). log L( θ) The fuctio is called the score fuctio. Hece, computig the MLE ca ofte be θ formulated as fidig θˆ which solves the score equatio log L ( θ ) θ = 0. (9.11) Two challegig problems for fidig maximum likelihood estimates of parameters i complex high dimesioal models is fidig the solutio to Eq umerically ad establishig that this solutio achieves a global maximum. Example 9.1. Suppose that x 1, K, x are idepedet, idetically distributed observatios from a gamma distributio with parameters α ad β. If α is kow the we have the likelihood fuctio is β α α 1 β x L ( α, β ) = k x k e Γ( α ) k =1 α = β exp{( α 1) log x k β x k } Γ( ) α k =1 k =1 (9.1) ad the log likelihood fuctio is log L( αβ, ) = log Γ( ) α α x (9.13) α + log β + ( 1) log( x k ) β k k=1 k=1 Differetiatig with respect to β log L(α, β) α = x k (9.14) β β k =1 ad solvig yields
7 page 7: 9.07 Lecture 9: Likelihood Methods α 0 = x (9.15) β βˆ = α. (9.16) x Notice also that if α is kow, the the secod derivative of the log likelihood evaluated at βˆ is log L( αβ), α x = = ˆ β βˆ β α This is clearly egative siceα is positive ad x is positive. Hece, the log likelihood (ad the likelihood) is maximized at βˆ. If α = 1 we have βˆ = x 1 is the maximum likelihood estimate for a expoetial model. If α is ukow, the there is o closed form solutio for either α or β. The estimates must the be foud umerically. To see this we differetiate Eq with respect to α to obtai the log likelihood for α Substitutig for β from Eq we obtai log L( αβ, ) Γ ( α ) = + log β + log( x ) (9.17) α Γ( α) k k=1 log L( α ) Γ ( α ) α = + log + log( x k ) (9.18) α Γ( α ) x k =1 Settig the left had side of Eq equal to zero ad solvig for the MLE of α requires a umerical procedure such as Newto s method. Plausible startig values for computig the ML estimates ca be obtaied from the method-of-momets estimates. We ca see i this case that the method-of-momets estimates ad the maximum likelihood estimates are differet. The distictio betwee the maximum likelihood ad the method-of-momets estimates comes about because for the gamma distributio, the sufficiet statistics for α ad β are log( x k ) ad x k. This is suggested by how the likelihood summarizes the data sample as idicated by k =1 the terms i the log likelihood fuctio i Eq Give the defiitio of the sufficiet statistics, this shows that the simple method-of-momets estimates are ot efficiet (i.e., they must have some loss of iformatio). We will make this cocept more precise below. Example 3. (cotiued) Let,, x be a radom sample of MEG backgroud oise believed x 1 K to obey a Gaussia probability desity with ukow µ ad σ. It is easy to show that 1 1 = x k is the maximum likelihood (ML) estimate of µ ad ˆ σ = (x k x) is the x k=1 k =1 maximum likelihood estimate of σ. This is straightforward to show by differetiatig the k =1
8 page 8: 9.07 Lecture 9: Likelihood Methods Gaussia log likelihood, equatig the gradiet to zero ad solvig for µ ad σ. Here the method-of-momets ad the maximum likelihood estimates are the same. I this case, the sufficiet statistics are the first two sample momets. Example 9.. Iverse Gaussia Distributio. If x 1, K, x is a radom sample from a iverse Gaussia distributio with parameters µ ad λ, the the likelihood is 1 λ λ(x k µ) 3 exp L(µ,λ) = k =1 π x k x k µ k λ 1 λ(x µ) = exp (9.19) 3 k =1 π x k k =1 x k µ 1 λ 1 λ λ 3 µ π µ k =1 k =1 k =1 = exp x k + λ x k log + + log x k. Recall that for this probability model the mea is µ ad the variace is µ 3 / λ. What are the maximum likelihood estimates of µ ad λ? Are they the same as the method-of-momets estimate? The maximum likelihood estimates are µˆml = 1 x i (9.0) whereas the method-of-momets estimates are λˆml = ( x µˆ ) (9.1) i ML µˆmm = 1 x i (9.) ˆ 3 λ = x /σˆ. (9.3) MM Ca you veture a guess as to what are the sufficiet statistics for µ ad λ? (Remember the maximum likelihood estimates are always fuctios of the sufficiet statistics. If you ca derive the last lie i Eq. 9.19, the the extra credit problem o Homework Assigmet 7 is straightforward). Ivariace Property of the Maximum Likelihood Estimator. Suppose that a distributio is idexed by a parameter θ. Let T ( θ ) be a fuctio of θ, the if ˆ is the maximum likelihood estimator of θ the T(θˆML ) is the maximum likelihood estimate of T ( θ ). For example, if θ = λ, the scale parameter of the iverse Gaussia distributio the MLE ˆ λ ML is give i Eq If we are iterested i T( λ ) = co s( λ ) the the MLE of T ( λ ) is T ( ˆ λ ML ). This is oe of the most importat properties of maximum likelihood estimates. θ ML
9 page 9: 9.07 Lecture 9: Likelihood Methods D. Fisher Iformatio (Pawita, 001). The MLE provides a poit (or sigle umber summary) estimate. I geeral a sigle umber is ot sufficiet to represet a likelihood fuctio. If the log likelihood is well approximated by a quadratic fuctio, the we eed at least two quatities to represet it: the locatio of its maximum the (MLE) ad the curvature at the maximum. The curvature will be the Fisher iformatio. Whe this quadratic approximatio is valid we call the likelihood fuctio regular. Fortuately, whe sample sizes are sufficietly large, the likelihood fuctio geerally does become regular. To restate this critical requiremet, regular problems are those i which we ca approximate the log likelihood fuctio aroud the MLE by a quadratic fuctio. To defie the Fisher iformatio we first recall how we compute the Taylor series expasio of a fuctio. The Taylor series expasio of a fuctio f( x) about a poit x 0 is defied as f f ''( x 0 )( x x 0 ) ( x ) = f ( x 0 ) + f ' ( x 0 )( x x 0 ) (9.4)! Let us assume that our log likelihood fuctio is differetiable ad let us expad the log likelihood fuctio i a Taylor series about the ML estimate ˆ θ ML. This gives ˆ ˆ log L(θ ML )''(θ θ ML ) log L( θ ) = log L(θˆ ˆ ˆ ML ) + log L(θ ML )'(θ θ ML ) (9.5)! Because log L(θˆML )' = 0 sice θˆml is the ML estimate of θ, we obtai ˆ ˆ ˆ log L(θ ML )''(θ θ ML ) log L( θ ) log L( θ ML ) +. (9.6)! Therefore, we ca approximate L( θ ) for θ close to θˆ as L( θ ) L ( ˆML θ )e x p { 1 I(θˆML ) ( θ θˆml ) } (9.7) where I (θˆ ) = log L(θˆ )". (9.8) ML is the observed Fisher iformatio. It is a umber because it is evaluated at the MLE. I(θˆML ) measures the curvature of the likelihood aroud the MLE. Eq. 9.7 implies that ear the MLE the likelihood ca be approximated as a Gaussia distributio. We ca gai ituitio about the meaig of the Fisher iformatio by cosiderig the followig Gaussia example. Example 3. (cotiued). If we assume σ is kow, the igorig irrelevat costats we get ML ad 1 log L( µ ) = (x i µ) (9.9) σ
10 page 10: 9.07 Lecture 9: Likelihood Methods log L( µ ) 1 = (x i µ) (9.30) µ σ log L( µ ) = µ σ (9.31) 1 1 Thus I ( ˆ µ ) = Hece, µ ( ˆ) Thus a key poit is: A large amout. V ar( ˆ ) = σ = I µ. of σ iformatio implies smaller variace. Example.1 (cotiued). For the biomial model of the learig experimet, the log likelihood ad score fuctios (igorig irrelevat costats) are log Lp ( ) = k log p + ( k )log(1 p) (9.3) The MLE is log L( p) k k = (9.33) p p 1 p k pˆ = (9.34) ad so that at the MLE the observed Fisher iformatio is log L( p) k k I( p) = = + (9.35) p p (1 p) I( p ˆ ) =. (9.36) pˆ(1 pˆ) See the Addedum 1 to Lecture 9 to uderstad the details of the last step. Remark 9.1. I priciple, we ca judge the quadratic approximatio to the log likelihood by plottig the true log likelihood with the quadratic approximatio superimposed. Remark 9.. For the Gaussia distributio from Example 3., the quadratic approximatio is exact. A practical rule for early all likelihood applicatios: a reasoably regular likelihood meas ˆ is approximately Gaussia. θ ML Remark 9.3. The result i Eq. 9.7 suggests that a approximate 95% cofidece iterval for θ may be costructed as ˆ θ ± se(θˆ ) (9.37) ML ML
11 page 11: 9.07 Lecture 9: Likelihood Methods 1 where se( ˆML θ ) = [ I ( θˆml )]. Remark 9.4. Reportig θˆml ad I(θˆML ) is a practical alterative to plottig the likelihood fuctio i every problem. The quatity we derived above was the observed Fisher iformatio. It is a estimate of the Fisher iformatio which we defie below. Defiitio 9.4. If x= x 1,..., x is a sample with joit probability desity f( x θ), the the Fisher iformatio i x for the parameter θ is log f( x θ ) I( θ ) = E[ ] θ log f( x θ ) = [ ] f ( x θ )dx. θ (9.38) Equivaletly, log f( x θ ) I( θ ) = E[ ] θ log f( x θ ) = θ f( x θ ) d x. (9.39) Now that we have defied the Fisher iformatio we ca use it to make a importat geeral statemet about the performace of a estimator. Theorem 9.1 (Cramer-Rao Lower Boud). Suppose x= x 1, K, x is a sample from f( x θ ), ad T( x) is a estimator of θ ad E [( T( x))] is a differetiable fuctio of θ. Suppose also that θ d d f( x θ) hx ( ) f ( x θ ) d x = hx ( ) d x, (9.40) dθ dθ for all hx ( ) with E h()< x. The θ V ar( T( x )) de θ ( T ( x )) ( ) dθ. (9.41) log f( x θ ) E θ ( ) θ Equatio 9.41 is the Cramer-Rao Lower Boud (CRLB) for the estimator Tx. () For a proof see Casella ad Berger (1990). CRLB gives the lowest boud o the variace of a estimator. If the estimate is ubiased, the the umerator is 1 ad the deomiator is the Fisher iformatio. If θ is a d 1 vector the the Fisher iformatio is a d d matrix give by
12 page 1: 9.07 Lecture 9: Likelihood Methods log f( x θ) T log f( x θ) log f( x θ) I( θ ) = E θ [( ] = E θ [ ]. (9.4) θ θ θθ T We will make extesive use of the Fisher iformatio to derive cofidece itervals for our estimates ad to make ifereces i our problems. E. Criteria for Evaluatig a Estimator If T( x) is a estimator of θ, the there are several criteria that ca be used to evaluate its performace. Four commoly used criteria to evaluate the performace of estimators are: 1. Mea-Squared Error (MSE) E T x θ θ [ ( ) ] The smaller the MSE the better the estimator.. Ubiasedess E θ ( T( x))=θ The bias of a estimator is b T ( θ ) = E( T ( x )) θ. A estimator is ubiased if its expected value is the parameter it estimates. 3. Cosistecy p Tx ( ) θ as Cosistecy meas that as the sample size icreases the estimator coverges i mea-squared error, probability or almost surely to the true value of the parameter. 4. Efficiecy Achieves a miimum variace (Cramer-Rao Lower Boud). The efficiecy of a estimator is usually characterized i terms of its variace relative to aother estimator. The Cramer-Rao Lower boud gives a characterizatio of how small the variace of a estimator ca be. F. Summary of Key Properties of Maximum Likelihood Estimatio 1. Maximum likelihood estimates are geerally biased.. Maximum likelihood estimates are cosistet, hece, they are asymptotically ubiased. 3. Maximum likelihood estimates are asymptotically efficiet. 4. The variace of the maximum likelihood estimate may be approximated by the reciprocal (iverse) of the Fisher iformatio. 5. The maximum likelihood estimate θˆml is asymptotically Gaussia with mea θ ad variace I( θ) 1.
13 page 13: 9.07 Lecture 9: Likelihood Methods 6. If θˆml is the maximum likelihood estimate of θ the h(θˆml ) is the maximum likelihood estimate of h( θ ). 7. Give that θˆ ML : N( θ, I 1 ( ) ), expasio we have θ by the Ivariace Priciple for the MLE ad a Taylor Series ad a approximate 95% cofidece iterval for h( θ ) is 1 h(θˆ ML ) N( h ( θ ), h' ( θ ) [ I ( θ )] ) (9.43) h( ˆ θ ) ±1.96h ' θˆml )[ ( ˆML )] 1 ML ( I θ. (9.44) See Addedum to Lecture 9 for a explaatio ad illustratio of this result. III. Summary The likelihood fuctio ad maximum likelihood estimatio are two of the most importat theoretical ad practical cocepts i statistics. Ackowledgmets I am grateful to Uri Ede for makig the figures, to Jim Mutch for careful proofreadig ad commets ad to Julie Scott for techical assistace. Textbook Refereces Casella, G, Berger RL. Statistical Iferece. Belmot, CA: Duxbury, DeGroot MH, Schervish MJ. Probability ad Statistics, 3rd editio. Bosto, MA: Addiso Wesley, 00. Pawita Y. I All Likelihood: Statistical Modelig ad Iferece Usig Likelihood. Lodo: Oxford, 001. Rice JA. Mathematical Statistics ad Data Aalysis, 3 rd editio. Bosto, MA, 007. Literature Referece Kass RE, Vetura V, Brow EN. Statistical issues i the aalysis of euroal data. Joural of Neurophysiology 005, 94: 8-5.
14 MIT OpeCourseWare Statistics for Brai ad Cogitive Sciece Fall 016 For iformatio about citig these materials or our Terms of Use, visit:
Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationLecture 12: Hypothesis Testing
9.07 Itroductio to Statistics for Brai ad Cogitive Scieces Emery N. Brow Lecture : Hypothesis Testig I. Objectives. Uderstad the hypothesis testig paradigm.. Uderstad how hypothesis testig procedures are
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationx = Pr ( X (n) βx ) =
Exercise 93 / page 45 The desity of a variable X i i 1 is fx α α a For α kow let say equal to α α > fx α α x α Pr X i x < x < Usig a Pivotal Quatity: x α 1 < x < α > x α 1 ad We solve i a similar way as
More informationEXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY
EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 016 MODULE : Statistical Iferece Time allowed: Three hours Cadidates should aswer FIVE questios. All questios carry equal marks. The umber
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationMaximum Likelihood Estimation
ECE 645: Estimatio Theory Sprig 2015 Istructor: Prof. Staley H. Cha Maximum Likelihood Estimatio (LaTeX prepared by Shaobo Fag) April 14, 2015 This lecture ote is based o ECE 645(Sprig 2015) by Prof. Staley
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationLecture 9: September 19
36-700: Probability ad Mathematical Statistics I Fall 206 Lecturer: Siva Balakrisha Lecture 9: September 9 9. Review ad Outlie Last class we discussed: Statistical estimatio broadly Pot estimatio Bias-Variace
More informationKLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions
We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationProbability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More informationReview Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn
Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationStatistics for Applications. Chapter 3: Maximum Likelihood Estimation 1/23
18.650 Statistics for Applicatios Chapter 3: Maximum Likelihood Estimatio 1/23 Total variatio distace (1) ( ) Let E,(IPθ ) θ Θ be a statistical model associated with a sample of i.i.d. r.v. X 1,...,X.
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationGoodness-of-Fit Tests and Categorical Data Analysis (Devore Chapter Fourteen)
Goodess-of-Fit Tests ad Categorical Data Aalysis (Devore Chapter Fourtee) MATH-252-01: Probability ad Statistics II Sprig 2019 Cotets 1 Chi-Squared Tests with Kow Probabilities 1 1.1 Chi-Squared Testig................
More informationLecture Stat Maximum Likelihood Estimation
Lecture Stat 461-561 Maximum Likelihood Estimatio A.D. Jauary 2008 A.D. () Jauary 2008 1 / 63 Maximum Likelihood Estimatio Ivariace Cosistecy E ciecy Nuisace Parameters A.D. () Jauary 2008 2 / 63 Parametric
More informationECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015
ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationCSE 527, Additional notes on MLE & EM
CSE 57 Lecture Notes: MLE & EM CSE 57, Additioal otes o MLE & EM Based o earlier otes by C. Grat & M. Narasimha Itroductio Last lecture we bega a examiatio of model based clusterig. This lecture will be
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More informationProperties of Point Estimators and Methods of Estimation
CHAPTER 9 Properties of Poit Estimators ad Methods of Estimatio 9.1 Itroductio 9. Relative Efficiecy 9.3 Cosistecy 9.4 Sufficiecy 9.5 The Rao Blackwell Theorem ad Miimum-Variace Ubiased Estimatio 9.6 The
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationLECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if
LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio
More informationQuestions and Answers on Maximum Likelihood
Questios ad Aswers o Maximum Likelihood L. Magee Fall, 2008 1. Give: a observatio-specific log likelihood fuctio l i (θ) = l f(y i x i, θ) the log likelihood fuctio l(θ y, X) = l i(θ) a data set (x i,
More informationDepartment of Mathematics
Departmet of Mathematics Ma 3/103 KC Border Itroductio to Probability ad Statistics Witer 2017 Lecture 19: Estimatio II Relevat textbook passages: Larse Marx [1]: Sectios 5.2 5.7 19.1 The method of momets
More informationLecture 7 Limit Theorems: Law of Large Numbers and the Central Limit Theorem
9.07 Itroductio to Statistics for Brai ad Cogitive Scieces Emery N. Brow Lecture 7 Limit Theorems: Law of Large Numbers ad the Cetral Limit Theorem I. Objectives Uderstad the logic behid the Law of Large
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationVariance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom
Variace of Discrete Radom Variables Class 5, 18.05 Jeremy Orloff ad Joatha Bloom 1 Learig Goals 1. Be able to compute the variace ad stadard deviatio of a radom variable.. Uderstad that stadard deviatio
More informationThe Method of Least Squares. To understand least squares fitting of data.
The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve
More information( θ. sup θ Θ f X (x θ) = L. sup Pr (Λ (X) < c) = α. x : Λ (x) = sup θ H 0. sup θ Θ f X (x θ) = ) < c. NH : θ 1 = θ 2 against AH : θ 1 θ 2
82 CHAPTER 4. MAXIMUM IKEIHOOD ESTIMATION Defiitio: et X be a radom sample with joit p.m/d.f. f X x θ. The geeralised likelihood ratio test g.l.r.t. of the NH : θ H 0 agaist the alterative AH : θ H 1,
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationLecture 6 Ecient estimators. Rao-Cramer bound.
Lecture 6 Eciet estimators. Rao-Cramer boud. 1 MSE ad Suciecy Let X (X 1,..., X) be a radom sample from distributio f θ. Let θ ˆ δ(x) be a estimator of θ. Let T (X) be a suciet statistic for θ. As we have
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationf(x i ; ) L(x; p) = i=1 To estimate the value of that maximizes L or equivalently ln L we will set =0, for i =1, 2,...,m p x i (1 p) 1 x i i=1
Parameter Estimatio Samples from a probability distributio F () are: [,,..., ] T.Theprobabilitydistributio has a parameter vector [,,..., m ] T. Estimator: Statistic used to estimate ukow. Estimate: Observed
More informationMaximum likelihood estimation from record-breaking data for the generalized Pareto distribution
METRON - Iteratioal Joural of Statistics 004, vol. LXII,. 3, pp. 377-389 NAGI S. ABD-EL-HAKIM KHALAF S. SULTAN Maximum likelihood estimatio from record-breakig data for the geeralized Pareto distributio
More informationDouble Stage Shrinkage Estimator of Two Parameters. Generalized Exponential Distribution
Iteratioal Mathematical Forum, Vol., 3, o. 3, 3-53 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/.9/imf.3.335 Double Stage Shrikage Estimator of Two Parameters Geeralized Expoetial Distributio Alaa M.
More informationLecture 3: MLE and Regression
STAT/Q SCI 403: Itroductio to Resamplig Methods Sprig 207 Istructor: Ye-Chi Che Lecture 3: MLE ad Regressio 3. Parameters ad Distributios Some distributios are idexed by their uderlyig parameters. Thus,
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More informationStat 319 Theory of Statistics (2) Exercises
Kig Saud Uiversity College of Sciece Statistics ad Operatios Research Departmet Stat 39 Theory of Statistics () Exercises Refereces:. Itroductio to Mathematical Statistics, Sixth Editio, by R. Hogg, J.
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More informationBayesian Methods: Introduction to Multi-parameter Models
Bayesia Methods: Itroductio to Multi-parameter Models Parameter: θ = ( θ, θ) Give Likelihood p(y θ) ad prior p(θ ), the posterior p proportioal to p(y θ) x p(θ ) Margial posterior ( θ, θ y) is Iterested
More informationG. R. Pasha Department of Statistics Bahauddin Zakariya University Multan, Pakistan
Deviatio of the Variaces of Classical Estimators ad Negative Iteger Momet Estimator from Miimum Variace Boud with Referece to Maxwell Distributio G. R. Pasha Departmet of Statistics Bahauddi Zakariya Uiversity
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationSTATISTICAL INFERENCE
STATISTICAL INFERENCE POPULATION AND SAMPLE Populatio = all elemets of iterest Characterized by a distributio F with some parameter θ Sample = the data X 1,..., X, selected subset of the populatio = sample
More informationAAEC/ECON 5126 FINAL EXAM: SOLUTIONS
AAEC/ECON 5126 FINAL EXAM: SOLUTIONS SPRING 2015 / INSTRUCTOR: KLAUS MOELTNER This exam is ope-book, ope-otes, but please work strictly o your ow. Please make sure your ame is o every sheet you re hadig
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationA statistical method to determine sample size to estimate characteristic value of soil parameters
A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationn n i=1 Often we also need to estimate the variance. Below are three estimators each of which is optimal in some sense: n 1 i=1 k=1 i=1 k=1 i=1 k=1
MATH88T Maria Camero Cotets Basic cocepts of statistics Estimators, estimates ad samplig distributios 2 Ordiary least squares estimate 3 3 Maximum lielihood estimator 3 4 Bayesia estimatio Refereces 9
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationAccess to the published version may require journal subscription. Published with permission from: Elsevier.
This is a author produced versio of a paper published i Statistics ad Probability Letters. This paper has bee peer-reviewed, it does ot iclude the joural pagiatio. Citatio for the published paper: Forkma,
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More information5 : Exponential Family and Generalized Linear Models
0-708: Probabilistic Graphical Models 0-708, Sprig 206 5 : Expoetial Family ad Geeralized Liear Models Lecturer: Matthew Gormley Scribes: Yua Li, Yichog Xu, Silu Wag Expoetial Family Probability desity
More informationClass 23. Daniel B. Rowe, Ph.D. Department of Mathematics, Statistics, and Computer Science. Marquette University MATH 1700
Class 23 Daiel B. Rowe, Ph.D. Departmet of Mathematics, Statistics, ad Computer Sciece Copyright 2017 by D.B. Rowe 1 Ageda: Recap Chapter 9.1 Lecture Chapter 9.2 Review Exam 6 Problem Solvig Sessio. 2
More informationStochastic Simulation
Stochastic Simulatio 1 Itroductio Readig Assigmet: Read Chapter 1 of text. We shall itroduce may of the key issues to be discussed i this course via a couple of model problems. Model Problem 1 (Jackso
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationDiscrete Mathematics for CS Spring 2005 Clancy/Wagner Notes 21. Some Important Distributions
CS 70 Discrete Mathematics for CS Sprig 2005 Clacy/Wager Notes 21 Some Importat Distributios Questio: A biased coi with Heads probability p is tossed repeatedly util the first Head appears. What is the
More informationSome Properties of the Exact and Score Methods for Binomial Proportion and Sample Size Calculation
Some Properties of the Exact ad Score Methods for Biomial Proportio ad Sample Size Calculatio K. KRISHNAMOORTHY AND JIE PENG Departmet of Mathematics, Uiversity of Louisiaa at Lafayette Lafayette, LA 70504-1010,
More informationECE 901 Lecture 13: Maximum Likelihood Estimation
ECE 90 Lecture 3: Maximum Likelihood Estimatio R. Nowak 5/7/009 The focus of this lecture is to cosider aother approach to learig based o maximum likelihood estimatio. Ulike earlier approaches cosidered
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More information