10 Simulation-Assisted Estimation

Size: px
Start display at page:

Download "10 Simulation-Assisted Estimation"

Transcription

1 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 10 Simulatio-Assisted Estimatio 10.1 Motivatio So far we have examied how to simulate choice probabilities but have ot ivestigated the properties of the parameter estimators that are based o these simulated probabilities. I the applicatios we have preseted, we simply iserted the simulated probabilities ito the log-likelihood fuctio ad maximized this fuctio, the same as if the probabilities were exact. This procedure seems ituitively reasoable. However, we have ot actually show, at least so far, that the resultig estimator has ay desirable properties, such as cosistecy, asymptotic ormality, or efficiecy. We have also ot explored the possibility that other forms of estimatio might perhaps be preferable whe simulatio is used rather tha exact probabilities. The purpose of this chapter is to examie various methods of estimatio i the cotext of simulatio. We derive the properties of these estimators ad show the coditios uder which each estimator is cosistet ad asymptotically equivalet to the estimator that would arise with exact values rather tha simulatio. These coditios provide guidace to the researcher o how the simulatio eeds to be performed to obtai desirable properties of the resultat estimator. The aalysis also illumiates the advatages ad limitatios of each form of estimatio, thereby facilitatig the researcher s choice amog methods. We cosider three methods of estimatio: 1. Maximum Simulated Likelihood: MSL. This procedure is the same as maximum likelihood (ML) except that simulated probabilities are used i lieu of the exact probabilities. The properties of MSL have bee derived by, for example, Gourieroux ad Mofort,(1993), Lee(1995), ad Hajivassiliou ad Ruud (1994). 2. Method of Simulated Momets: MSM. This procedure, suggested by McFadde (1989), is a simulated aalog to the traditioal method of momets (MOM). Uder traditioal MOM for discrete choice, residuals are defied as the differece betwee 240

2 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 241 the 0 1 depedet variable that idetifies the chose alterative ad the probability of the alterative. Exogeous variables are idetified that are ucorrelated with the model residuals i the populatio. The estimates are the parameter values that make the variables ad residuals ucorrelated i the sample. The simulated versio of this procedure calculates residuals with the simulated probabilities rather tha the exact probabilities. 3. Method of Simulated Scores: MSS. As discussed i Chapter 8, the gradiet of the log likelihood of a observatio is called the score of the observatio. The method of scores fids the parameter values that set the average score to zero. Whe exact probabilities are used, the method of scores is the same as maximum likelihood, sice the log-likelihood fuctio is maximized whe the average score is zero. Hajivassiliou ad McFadde (1998) suggested usig simulated scores istead of the exact oes. They showed that, depedig o how the scores are simulated, MSS ca differ from MSL ad, importatly, ca attai cosistecy ad efficiecy uder more relaxed coditios. I the ext sectio we defie these estimators more formally ad relate them to their osimulated couterparts. We the describe the properties of each estimator i two stages. First, we derive the properties of the traditioal estimator based o exact values. Secod, we show how the derivatio chages whe simulated values are used rather tha exact values. We show that the simulatio adds extra elemets to the samplig distributio of the estimator. The aalysis allows us to idetify the coditios uder which these extra elemets disappear asymptotically so that the estimator is asymptotically equivalet to its osimulated aalog. We also idetify more relaxed coditios uder which the estimator, though ot asymptotically equivalet to its osimulated couterpart, is evertheless cosistet Defiitio of Estimators Maximum Simulated Likelihood The log-likelihood fuctio is LL(θ) = l P (θ), where θ is a vector of parameters, P (θ) is the (exact) probability of the observed choice of observatio, ad the summatio is over a sample

3 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio of N idepedet observatios. The ML estimator is the value of θ that maximizes LL(θ). Sice the gradiet of LL(θ) is zero at the maximum, the ML estimator ca also be defied as the value of θ at which s (θ) = 0, where s (θ) = l P (θ)/ θ is the score for observatio. Let ˇP (θ) be a simulated approximatio to P (θ). The simulated log-likelihood fuctio is SLL(θ) = l ˇP (θ), ad the MSL estimator is the value of θ that maximizes SLL(θ). Stated equivaletly, the estimator is the value of θ at which š(θ) = 0, where š (θ) = l ˇP (θ)/ θ. A preview of the properties of MSL ca be give ow, with a full explaatio reserved for the ext sectio. The mai issue with MSL arises because of the log trasformatio. Suppose ˇP (θ) is a ubiased simulator of P (θ), so that E r ˇP (θ) = P (θ), where the expectatio is over draws used i the simulatio. All of the simulators that we have cosidered are ubiased for the true probability. However, sice the log operatio is a oliear trasformatio, l ˇP (θ) is ot ubiased for l P (θ) eve though ˇP (θ) is ubiased for P (θ). The bias i the simulator of l P (θ) traslates ito bias i the MSL estimator. This bias dimiishes as more draws are used i the simulatio. To determie the asymptotic properties of the MSL estimator, the questio arises of how the simulatio bias behaves whe the sample size rises. The aswer depeds critically o the relatioship betwee the umber of draws that are used i the simulatio, labeled R, ad the sample size, N. IfR is cosidered fixed, the the MSL estimator does ot coverge to the true parameters, because of the simulatio bias i l ˇP (θ). Suppose istead that R rises with N; that is, the umber of draws rises with sample size. I this case, the simulatio bias disappears as N (ad hece R) rises without boud. MSL is cosistet i this case. As we will see, if R rises faster tha N, MSL is ot oly cosistet but also efficiet, asymptotically equivalet to maximum likelihood o the exact probabilities. I summary, if R is fixed, the MSL is icosistet. If R rises at ay rate with N, the MSL is cosistet. If R rises faster tha N, the MSL is asymptotically equivalet to ML. The primary limitatio of MSL is that it is icosistet for fixed R. The other estimators that we cosider are motivated by the desire for a simulatio-based estimator that is cosistet for fixed R. Both MSM ad MSS, if structured appropriately, attai this goal. This beefit comes at a price, however, as we see i the followig discussio.

4 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Method of Simulated Momets Simulatio-Assisted Estimatio 243 The traditioal MOM is motivated by the recogitio that the residuals of a model are ecessarily ucorrelated i the populatio with factors that are exogeous to the behavior beig modeled. The MOM estimator is the value of the parameters that make the residuals i the sample ucorrelated with the exogeous variables. For discrete choice models, MOM is defied as the parameters that solve the equatio (10.1) where [d j P j (θ)]z j = 0, j d j is the depedet variable that idetifies the chose alterative: d j = 1if chose j, ad = 0 otherwise, ad is a vector of exogeous variables called istrumets. z j The residuals are d j P j (θ), ad the MOM estimator is the parameter values at which the residuals are ucorrelated with the istrumets i the sample. This MOM estimator is aalogous to MOM estimators for stadard regressio models. A regressio model takes the form y = x β + ε. The MOM estimator for this regressio is the β at which (y x β)z = 0 for a vector of exogeous istrumets z. Whe the explaatory variables i the model are exogeous, the they serve as the istrumets. The MOM estimator i this case becomes the ordiary least squares estimator: (y x β)x = 0, x y = x x β, ( ) 1 ( ˆβ = x x x y ), which is the formula for the least squares estimator. Whe istrumets are specified to be somethig other tha the explaatory variables, the

5 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio MOM estimator becomes the stadard istrumetal variables estimator: (y x β)z = 0, z y = z x β, ( ) 1 ( ˆβ = z x z y ), which is the formula for the istrumetal variables estimator. This estimator is cosistet if the istrumets are idepedet of ε i the populatio. The estimator is more efficiet the more highly correlated the istrumets are with the explaatory variables i the model. Whe the explaatory variables, x, are themselves exogeous, the the ideal istrumets (i.e., those that give the highest efficiecy) are the explaatory variables themselves, z = x. For discrete choice models, MOM is defied aalogously ad has a similar relatio to other estimators, especially ML. The researcher idetifies istrumets z j that are exogeous ad hece idepedet i the populatio of the residuals [d j P j (θ)]. The MOM estimator is the value of θ at which the sample correlatio betwee istrumets ad residuals is zero. Ulike the liear case, equatio (10.1) caot be solved explicitly for ˆθ. Istead, umerical procedures are used to fid the value of θ that solves this equatio. As with regressio, ML for a discrete choice model is a special case of MOM. Let the istrumets be the scores: z j = l P j (θ)/ θ. With these istrumets, MOM is the same as ML: [d j P j (θ)]z j = 0, ( ) ( l P j (θ) d j β j j lp i (θ) β j j P j (θ) l P j(θ) β ) = 0, 1 P j (θ) P j (θ) = 0, P j (θ) θ s (θ) P j (θ) = 0, θ j s (θ) = 0, which is the defiig coditio for ML. I the third lie, i is the chose alterative, recogizig that d j = 0 for all j i. The fourth

6 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 245 lie uses the fact that the sum of P j / θ over alteratives is zero, sice the probabilities must sum to 1 before ad after the chage i θ. Sice MOM becomes ML ad hece is fully efficiet whe the istrumets are the scores, the scores are called the ideal istrumets. MOM is cosistet wheever the istrumets are idepedet of the model residuals. It is more efficiet the higher the correlatio betwee the istrumets ad the ideal istrumets. A iterestig simplificatio arises with stadard logit. For the stadard logit model, the ideal istrumets are the explaatory variables themselves. As show i Sectio 3.7.1, the ML estimator for stadard j [d j P j (θ)]x j = 0, where logit is the value of θ that solves x j are the explaatory variables. This is a MOM estimator with the explaatory variables as istrumets. A simulated versio of MOM, called the method of simulated momets (MSM), is obtaied by replacig the exact probabilities P j (θ) with simulated probabilities ˇP j (θ). The MSM estimator is the value of θ that solves [d j ˇP j (θ)]z j = 0 j for istrumets z j. As with its osimulated aalog, MSM is cosistet if z j is idepedet of d j ˇP j (θ). The importat feature of this estimator is that ˇP j (θ) eters the equatio liearly. As a result, if ˇP j (θ) is ubiased for P j (θ), the [d j ˇP j (θ)]z j is ubiased for [d j P j (θ)]z j. Sice there is o simulatio bias i the estimatio coditio, the MSM estimator is cosistet, eve whe the umber of draws R is fixed. I cotrast, MSL cotais simulatio bias due to the log trasformatio of the simulated probabilities. By ot takig a oliear trasformatio of the simulated probabilities, MSM avoids simulatio bias. MSM still cotais simulatio oise (variace due to simulatio). This oise becomes smaller as R rises ad disappears whe R rises without boud. As a result, MSM is asymptotically equivalet to MOM if R rises with N. Just like its usimulated aalog, MSM is less efficiet tha MSL uless the ideal istrumets are used. However, the ideal istrumets are fuctios of l P j. They caot be calculated exactly for ay but the simplest models, ad, if they are simulated usig the simulated probability, simulatio bias is itroduced by the log operatio. MSM is usually applied with oideal weights, which meas that there is a loss of

7 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio efficiecy. MSM with ideal weights that are simulated without bias becomes MSS, which we discuss i the ext sectio. I summary, MSM has the advatage over MSL of beig cosistet with a fixed umber of draws. However, there is o free luch, ad the cost of this advatage is a loss of efficiecy whe oideal weights are used Method of Simulated Scores MSS provides a possibility of attaiig cosistecy without a loss of efficiecy. The cost of this double advatage is umerical: the versios of MSS that provide efficiecy have fairly poor umerical properties such that calculatio of the estimator ca be difficult. The method of scores is defied by the coditio s (θ) = 0, where s (θ) = l P (θ)/ θ is the score for observatio. This is the same defiig coditio as ML: whe exact probabilities are used, the method of scores is simply ML. The method of simulated scores replaces the exact score with a simulated couterpart. The MSS estimator is the value of θ that solves š (θ) = 0, where š (θ) is a simulator of the score. If š (θ) is calculated as the derivative of the log of the simulated probability; that is, š (θ) = l ˇP (θ)/ θ, the MSS is the same as MSL. However, the score ca be simulated i other ways. Whe the score is simulated i other ways, MSS differs from MSL ad has differet properties. Suppose that a ubiased simulator of the score ca be costructed. With this simulator, the defiig equatio š(θ) = 0 does ot icorporate ay simulatio bias, sice the simulator eters the equatio liearly. MSS is therefore cosistet with a fixed R. The simulatio oise decreases as R rises, such that MSS is asymptotically efficiet, equivalet to MSL, whe R rises with N. I cotrast, MSL uses the biased score simulator š (θ) = l ˇP (θ)/ θ, which is biased due to the log operator. MSS with a ubiased score simulator is therefore better tha MSL with its biased score simulator i two regards: it is cosistet uder less striget coditios (with fixed R rather tha R risig with N) ad is efficiet uder less striget coditios (R risig at ay rate with N rather tha R risig faster tha N).

8 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 247 The difficulty with MSS comes i fidig a ubiased score simulator. The score ca be rewritte as s (θ) = l P j(θ) θ = 1 P j P j (θ) θ. A ubiased simulator for the secod term P j / θ is easily obtaied by takig the derivative of the simulated probability. Sice differetiatio is a liear operatio, ˇP j / θ is ubiased for P j / θ if ˇP j (θ) is ubiased for P j (θ). Sice the secod term i the score ca be simulated without bias, the difficulty arises i fidig a ubiased simulator for the first term 1/P j (θ). Of course, simply takig the iverse of the simulated probability does ot provide a ubiased simulator, sice E r (1/ ˇP j (θ)) 1/P j (θ). Like the log operatio, a iverse itroduces bias. Oe proposal is based o the recogitio that 1/P j (θ) is the expected umber of draws of the radom terms that are eeded before a accept is obtaied. Cosider drawig balls from a ur that cotais may balls of differet colors. Suppose the probability of obtaiig a red ball is That is, oe-fifth of the balls are red. How may draws would it take, o average, to obtai a red ball? The aswer is 1/0.2 = 5. The same idea ca be applied to choice probabilities. P j (θ) is the probability that a draw of the radom terms of the model will result i alterative j havig the highest utility. The iverse 1/P j (θ) ca be simulated as follows: 1. Take a draw of the radom terms from their desity. 2. Calculate the utility of each alterative with this draw. 3. Determie whether alterative j has the highest utility. 4. If so, call the draw a accept. If ot, the call the draw a reject ad repeat steps 1 to 3 with a ew draw. Defie B r as the umber of draws that are take util the first accept is obtaied. 5. Perform steps 1 to 4 R times, obtaiig B r for r = 1,...,R. The simulator of 1/P j (θ) is(1/r) R r=1 Br. This simulator is ubiased for 1/P j (θ). The product of this simulator with the simulator ˇP j / θ provides a ubiased simulator of the score. MSS based o this ubiased score simulator is cosistet for fixed R ad asymptotically efficiet whe R rises with N. Ufortuately, the simulator of 1/P j (θ) has the same difficulties as the accept reject simulators that we discussed i Sectio 5.6. There is o guaratee tha a accept will be obtaied withi ay give umber of draws. Also, the simulator is ot cotiuous i parameters. The

9 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio discotiuity hiders the umerical procedures that are used to locate the parameters that solve the MSS equatio. I summary, there are advatages ad disadvatages of MSS relative to MSL, just as there are of MSM. Uderstadig the capabilities of each estimator allows the researcher to make a iformed choice amog them The Cetral Limit Theorem Prior to derivig the properties of our estimators, it is useful to review the cetral limit theorem. This theorem provides the basis for the distributios of the estimators. Oe of the most basic results i statistics is that, if we take draws from a distributio with mea µ ad variace σ, the mea of these draws will be ormally distributed with mea µ ad variace σ/n, where N is a large umber of draws. This result is the cetral limit theorem, stated ituitively rather tha precisely. We will provide a more complete ad precise expressio of these ideas. Let t = (1/N) t, where each t is a draw a from a distributio with mea µ ad variace σ. The realizatio of the draws are called the sample, ad t is the sample mea. If we take a differet sample (i.e., obtai differet values for the draws of each t ), the we will get a differet value for the statistic t. Our goal is to derive the samplig distributio of t. For most statistics, we caot determie the samplig distributio exactly for a give sample size. Istead, we examie how the samplig distributio behaves as sample size rises without boud. A distictio is made betwee the limitig distributio ad the asymptotic distributio of a statistic. Suppose that, as sample size rises, the samplig distributio of statistic t coverges to a fixed distributio. For example, the samplig distributio of t might become arbitrarily close to a ormal with mea t ad variace σ. I this case, we say that N(t,σ) is the limitig distributio of t ad that t coverges i distributio to N(t,σ). We d deote this situatio as t N(t,σ). I may cases, a statistic will ot have a limitig distributio. As N rises, the samplig distributio keeps chagig. The mea of a sample of draws is a example of a statistic without a limitig distributio. As stated, if t is the mea of a sample of draws from a distributio with mea µ ad variace σ, the t is ormally distributed with mea µ ad variace σ/n. The variace decreases as N rises. The distributio chages as N rises, becomig more ad more tightly dispersed aroud

10 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 249 the mea. If a limitig distributio were to be defied i this case, it would have to be the degeerate distributio at µ: asn rises without boud, the distributio of t collapses o µ. This limitig distributio is useless i uderstadig the variace of the statistic, sice the variace of this limitig distributio is zero. What do we do i this case to uderstad the properties of the statistic? If our origial statistic does ot have a limitig distributio, the we ofte ca trasform the statistic i such a way that the trasformed statistic has a limitig distributio. Suppose, as i i our example of a sample mea, that the statistic we are iterested i does ot have a limitig distributio because its variace decreases as N rises. I that case, we ca cosider a trasformatio of the statistic that ormalizes for sample size. I particular, we ca cosider N(t µ). Suppose that this statistic does ideed have a limitig distributio, for example d N(t µ) N(0,σ). I this case, we ca derive the properties of our origial statistic from the limitig distributio of the trasformed statistic. Recall from basic priciples of probabilities that, for fixed a ad b, ifa(t b) is distributed ormal with zero mea ad variace σ, the t itself is distributed ormal with mea b ad variace σ/a 2. This statemet ca be applied to our limitig distributio. For large eough N, N(t µ) is distributed approximately N(0,σ). Therefore, for large eough N, t is distributed approximately N(µ, σ/n). We deote this a as t N(µ, σ/n). Note that this is ot the limitig distributio of t, sice t has o odegeerate limitig distributio. Rather, it is called the asymptotic distributio of t, derived from the limitig distributio of N(t µ). We ca ow restate precisely our cocepts about the samplig distributio of a sample mea. The cetral limit theorem states the followig. Suppose t is the mea of a sample of N draws from a distributio with mea µ ad variace σ. The N(t µ) d N(0,σ). With this limitig a distributio, we ca say that t N(µ, σ/n). There is aother, more geeral versio of the cetral limit theorem. I the versio just stated, each t is a draw from the same distributio. Suppose t is a draw from a distributio with mea µ ad variace σ, for = 1,...,N. That is, each t is from a differet distributio; the distributios have the same mea but differet variaces. The geeralized versio of the cetral limit theorem states that N(t µ) d N(0,σ), where σ is ow the average variace: σ = (1/N) σ.give a this limitig distributio, we ca say that t N(µ, σ/n). We will use both versios of the cetral limit theorem whe derivig the distributios of our estimators.

11 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio 10.4 Properties of Traditioal Estimators I this sectio, we review the procedure for derivig the properties of estimators ad apply that procedure to the traditioal, o-simulatiobased estimators. This discussio provides the basis for aalyzig the properties of the simulatio-based estimators i the ext sectio. The true value of the parameters is deoted θ. The ML ad MOM estimators are roots of a equatio that takes the form (10.2) g ( ˆθ)/N = 0. That is, the estimator ˆθ is the value of the parameters that solve this equatio. We divide by N, eve though this divisio does ot affect the root of the equatio, because doig so facilitates our derivatio of the properties of the estimators. The coditio states that the average value of g (θ) i the sample is zero at the parameter estimates. For ML, g (θ) is the score l P (θ)/ θ. For MOM, g (θ) is the set of first momets of residuals with a vector of istrumets, j (d j P j )z j. Equatio (10.2) is ofte called the momet coditio. I its osimulated form, the method of scores is the same as ML ad therefore eed ot be cosidered separately i this sectio. Note that we call (10.2) a equatio eve though it is actually a set of equatios, sice g (θ) is a vector. The parameters that solve these equatios are the estimators. At ay particular value of θ, the mea ad variace of g (θ) ca be calculated for the sample. Label the mea as g(θ) ad the variace as W (θ). We are particularly iterested i the sample mea ad variace of g (θ) at the true parameters, θ, sice our goal is to estimate these parameters. The key to uderstadig the properties of a estimator comes i realizig that each g (θ ) is a draw from a distributio of g (θ ) s i the populatio. We do ot kow the true parameters, but we kow that each observatio has a value of g (θ ) at the true parameters. The value of g (θ ) varies over people i the populatio. So, by drawig a perso ito our sample, we are essetially drawig a value of g (θ ) from its distributio i the populatio. The distributio of g (θ ) i the populatio has a mea ad variace. Label the mea of g (θ ) i the populatio as g ad its variace i the populatio as W. The sample mea ad variace at the true parameters, g(θ ) ad W (θ ), are the sample couterparts to the populatio mea ad variace, g ad W.

12 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 251 We assume that g = 0. That is, we assume that the average of g (θ )i the populatio is zero at the true parameters. Uder this assumptio, the estimator provides a sample aalog to this populatio expectatio: ˆθ is the value of the parameters at which the sample average of g (θ) equals zero, as give i the defiig coditio (10.2). For ML, the assumptio that g = 0 simply states that the average score i the populatio is zero, whe evaluated at the true parameters. I a sese, this ca be cosidered the defiitio of the true parameters, amely, θ are the parameters at which the log-likelihood fuctio for the etire populatio obtais its maximum ad hece has zero slope. The estimated parameters are the values that make the slope of the likelihood fuctio i the sample zero. For MOM, the assumptio is satisfied if the istrumets are idepedet of the residuals. I a sese, the assumptio with MOM is simply a reiteratio that the istrumets are exogeous. The estimated parameters are the values that make the istrumets ad residuals ucorrelated i the sample. We ow cosider the populatio variace of g (θ ), which we have deoted W. Whe g (θ) is the score, as i ML, this variace takes a special meaig. As show i Sectio 8.7, the iformatio idetity states that V = H, where ( 2 l P (θ ) ) H = E θ θ is the iformatio matrix ad V is the variace of the scores evaluated at the true parameters: V = Var( l P (θ )/ θ). Whe g (θ) is the score, we have W = V by defiitio ad hece W = H by the iformatio idetity. That is, whe g (θ) is the score, W is the iformatio matrix. For MOM with oideal istrumets, W H, so that W does ot equal the iformatio matrix. Why does this distictio matter? We will see that kowig whether W equals the iformatio matrix allows us to determie whether the estimator is efficiet. The lowest variace that ay estimator ca achieve is H 1 /N. For a proof, see, for example, Greee (2000) or Ruud (2000). A estimator is efficiet if its variace attais this lower boud. As we will see, this lower boud is achieved whe W = H but ot whe W H. Our goal is to determie the properties of ˆθ. We derive these properties i a two-step process. First, we examie the distributio of g(θ ), which, as stated earlier, is the sample mea of g (θ ). Secod, the distributio of ˆθ is derived from the distributio of g(θ ). This two-step process is ot ecessarily the most direct way of examiig traditioal estimators.

13 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio However, as we will see i the ext sectio, it provides a very coveiet way for geeralizig to simulatio-based estimators. Step 1: The Distributio of g(θ ) Recall that the value of g (θ ) varies over decisio makers i the populatio. Whe takig a sample, the researcher is drawig values of g (θ ) from its distributio i the populatio. This distributio has zero mea by assumptio ad variace deoted W. The researcher calculates the sample mea of these draws, g(θ ). By the cetral limit theorem, N(g(θ ) 0) d N(0, W) such that the sample mea has distributio g(θ ) a N(0, W/N). Step 2: Derive the Distributio of ˆθ from the Distributio of g(θ ) We ca relate the estimator ˆθ to its defiig term g(θ) as follows. Take a first-order Taylor s expasio of g( ˆθ) aroud g(θ ): (10.3) g( ˆθ) = g(θ ) + D[ ˆθ θ ], where D = g(θ )/ θ.bydefiitio of ˆθ (that is, by defiig coditio (10.2)), g( ˆθ) = 0 so that the right-had side of this expasio is 0. The (10.4) 0 = g(θ ) + D[ ˆθ θ ], ˆθ θ = D 1 g(θ ), N( ˆθ θ ) = N( D 1 )g(θ ). Deote the mea of g (θ )/ θ i the populatio as D. The sample mea of g (θ )/ θ is D, asdefied for equatio (10.3). The sample mea D coverges to the populatio mea D as the sample size rises. We kow from step 1 that Ng(θ ) d N(0, W). Usig this fact i (10.4), we have (10.5) N( ˆθ θ ) d N(0, D 1 WD 1 ). This limitig distributio tells us that ˆθ a N(θ, D 1 WD 1 /N). We ca ow observe the properties of the estimator. The asymptotic distributio of ˆθ is cetered o the true value, ad its variace decreases as the sample size rises. As a result, ˆθ coverges i probability to θ as the sample sise rises without boud: ˆθ p θ. The estimator is therefore cosistet. The estimator is asymptotically ormal. Ad its variace is D 1 WD 1 /N, which ca be compared with the lowest possible variace, H 1 /N, to determie whether it is efficiet. For ML, g ( ) is the score, so that the variace of g (θ )isthe variace of the scores: W = V. Also, the mea derivative of g (θ )

14 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 253 is the mea derivative of the scores: D = H = E( 2 l P (θ )/ θ θ ), where the expectatio is over the populatio. By the iformatio idetity, V = H. The asymptotic variace of ˆθ becomes D 1 WD 1 /N = H 1 VH 1 /N = H 1 ( H)H 1 /N = H 1 /N, which is the lowest possible variace of ay estimator. ML is therefore efficiet. Sice V = H, the variace of the ML estimator ca also be expressed as V 1 /N, which has a readily iterpretable meaig: the variace of the estimator is equal to the variace of the scores evaluated at the true parameters, divided by sample size. For MOM, g ( ) is a set of momets. If the ideal istrumets are used, the MOM becomes ML ad is efficiet. If ay other istrumets are used, the MOM is ot ML. I this case, W is the populatio variace of the momets ad D is the mea derivatives of the momets, rather tha the variace ad mea derivatives of the scores. The asymptotic variace of ˆθ does ot equal H 1 /N. MOM without ideal weights is therefore ot efficiet Properties of Simulatio-Based Estimators Suppose that the terms that eter the defiig equatio for a estimator are simulated rather tha calculated exactly. Let ǧ (θ) deote the simulated value of g (θ), ad ǧ(θ) the sample mea of these simulated values, so that ǧ(θ) is the simulated versio of g(θ). Deote the umber of draws used i simulatio for each as R, ad assume that idepedet draws are used for each (e.g., separate draws are take for each ). Assume further that the same draws are used for each value of θ whe calculatig ǧ (θ). This procedure prevets chatter i the simulatio: the differece betwee ǧ(θ 1 ) ad ǧ(θ 2 ) for two differet values of θ is ot due to differet draws. These assumptios o the simulatio draws are easy for the researcher to implemet ad simplify our aalysis cosiderably. For iterested readers, Lee (1992) examies the situatio whe the same draws are used for all observatios. Pakes ad Pollard (1989) provide a way to characterize a equicotiuity coditio that, whe satisfied, facilitates aalysis of simulatio-based estimators. McFadde (1989) characterizes this coditio i a differet way ad shows that it ca be met by usig the same draws for each value of θ, which is the assumptio that we make. McFadde (1996) provides a helpful sythesis that icludes a discussio of the eed to prevet chatter. The estimator is defied by the coditio ǧ( ˆθ) = 0. We derive the properties of ˆθ i the same two steps as for the traditioal estimators.

15 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio Step 1: The Distributio of ǧ(θ ) To idetify the various compoets of this distributio, let us reexpress ǧ(θ ) by addig ad subtractig some terms ad rearragig: ǧ(θ ) = ǧ(θ ) + g(θ ) g(θ ) + E r ǧ(θ ) E r ǧ(θ ) = g(θ ) + [E r ǧ(θ ) g(θ )] + [ǧ(θ ) E r ǧ(θ )], where g(θ ) is the osimulated value ad E r ǧ(θ ) is the expectatio of the simulated value over the draws used i the simulatio. Addig ad subtractig terms obviously does ot chage ǧ(θ ). Yet, the subsequet rearragemet of the terms allows us to idetify compoets that have ituitive meaig. The first term g(θ ) is the same as arises for the traditioal estimator. The other two terms are extra elemets that arise because of the simulatio. The term E r ǧ(θ ) g(θ ) captures the bias, if ay, i the simulator of g(θ ). It is the differece betwee the true value of g(θ ) ad the expectatio of the simulated value. If the simulator is ubiased for g(θ ), the E r ǧ(θ ) = g(θ ) ad this term drops out. Ofte, however, the simulator will ot be ubiased for g(θ ). For example, with MSL, ǧ (θ) = l ˇP (θ)/ θ, where ˇP (θ) is a ubiased simulator of P (θ). Sice ˇP (θ) eters oliearly via the log operator, ǧ (θ) is ot ubiased. The third term, ǧ(θ ) E r ǧ(θ ), captures simulatio oise, that is, the deviatio of the simulator for give draws from its expectatio over all possible draws. Combiig these cocepts, we have (10.6) ǧ(θ) = A + B + C, where A is the same as i the traditioal estimator, B is simulatio bias, C is simulatio oise. To see how the simulatio-based estimators differ from their traditioal couterparts, we examie the simulatio bias B ad oise C. Cosider first the oise. This term ca be reexpressed as C = ǧ(θ ) E r ǧ(θ ) = 1 ǧ N (θ ) E r ǧ (θ ) = d /N,

16 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 255 where d is the deviatio of the simulated value for observatio from its expectatio. The key to uderstadig the behavior of the simulatio oise comes i otig that d is simply a statistic for observatio. The sample costitutes N draws of this statistic, oe for each observatio: d, = 1,...,N. The simulatio oise C is the average of these N draws. Thus, the cetral limit theorem gives us the distributio of C. I particular, for a give observatio, the draws that are used i simulatio provide a particular value of d. If differet draws had bee obtaied, the a differet value of d would have bee obtaied. There is a distributio of values of d over the possible realizatios of the draws used i simulatio. The distributio has zero mea, sice the expectatio over draws is subtracted out whe creatig d. Label the variace of the distributio as S /R, where S is the variace whe oe draw is used i simulatio. There are two thigs to ote about this variace. First, S /R is iversely related to R, the umber of draws that are used i simulatio. Secod, the variace is differet for differet. Sice g (θ ) is differet for differet, the variace of the simulatio deviatio also differs. We take a draw of d for each of N observatios; the overall simulatio oise, C, is the average of these N draws of observatio-specific simulatio oise. As just stated, each d is a draw from a distributio with zero mea ad variace S /R. The geeralized versio of the cetral limit theorem tells us the distributio of a sample average of draws from distributios that have the same mea but differet variaces. I our case, d NC N(0, S/R), where S is the populatio mea of S. The C a N(0, S/NR). The most relevat characteristic of the asymptotic distributio of C is that it decreases as N icreases, eve whe R is fixed. Simulatio oise disappears as sample size icreases, eve without icreasig the umber of draws used i simulatio. This is a very importat ad powerful fact. It meas that icreasig the sample size is a way to decrease the effects of simulatio o the estimator. The result is ituitively meaigful. Essetially, simulatio oise cacels out over observatios. The simulatio for oe observatio might, by chace, make that observatio s ǧ (θ) too large. However, the simulatio for aother observatio is likely, by chace, to be too small. By averagig the simulatios over observatios, the errors ted to cacel each other. As sample size rises, this cacelig out property becomes more powerful util, with large eough samples, simulatio oise is egligible.

17 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio Cosider ow the bias. If the simulator ǧ(θ) is ubiased for g(θ), the the bias term B i (10.6) is zero. However, if the simulator is biased, as with MSL, the the effect of this bias o the distributio of ǧ(θ ) must be cosidered. Usually, the defiig term g (θ) is a fuctio of a statistic, l, that ca be simulated without bias. For example, with MSL, g (θ) is a fuctio of the choice probability, which ca be simulated without bias; i this case l is the probability. More geerally, l ca be ay statistic that is simulated without bias ad serves to defie g (θ). We ca write the depedece i geeral as g (θ) = g(l (θ)) ad the ubiased simulator of l (θ)asľ (θ) where E r ľ (θ) = l (θ). We ca ow reexpress ǧ (θ) by takig a Taylor s expasio aroud the usimulated value g (θ): ǧ (θ) = g (θ) + g(l (θ)) [ľ (θ) l (θ)] l g(l (θ)) [ľ 2 l 2 (θ) l (θ] 2, ǧ (θ) g (θ) = g [ľ (θ) l (θ)] g [ľ (θ) l (θ)] 2, where g ad g are simply shorthad ways to deote the first ad secod derivatives of g (l( )) with respect to l. Sice ľ (θ) is ubiased for l (θ), we kow E r g [ľ (θ) l (θ)] = g [E rľ(θ) l (θ)] = 0. As a result, oly the variace term remais i the expectatio: E r ǧ (θ) g (θ) = 1 2 g E r[ľ (θ) l (θ)] 2 = 1 2 g Var r ľ(θ). Deote Var r ľ (θ) = Q /R to reflect the fact that the variace is iversely proportioal to the umber of draws used i the simulatio. The simulatio bias is the E r ǧ(θ) g(θ) = 1 E r ǧ N (θ) g (θ) = 1 g Q N 2R = Z R, where Z is the sample average of g Q /2.

18 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 257 Sice B = Z/R, the value of this statistic ormalized for sample size is (10.7) NB = N R Z. If R is fixed, the B is ozero. Eve worse, NB rises with N, i such a way that it has o limitig value. Suppose that R is cosidered to rise with N. The bias term the disappears asymptotically: B = Z/R p 0. However, the ormalized bias term does ot ecessarily disappear. Sice N eters the umerator of this term, NB = ( N/R)Z p 0 oly if R rises faster tha N, so that the ratio N/R approaches zero as N icreases. If R rises slower tha N, the ratio N/R rises, such that the ormalized bias term does ot disappear but i fact gets larger ad larger as sample size icreases. We ca ow collect our results for the distributio of the defiig term ormalized by sample size: (10.8) N ǧ(θ ) = N(A + B + C), where d NA N(0, W), N NB = R Z, d NC N(0, S/R), the same as i the traditioal estimator, capturig simulatio bias, capturig simulatio oise. Step 2: Derive Distributio of ˆθ from Distributio of ǧ(θ ) As with the traditioal estimators, the distributio of ˆθ is directly related to the distributio of ǧ(θ ). Usig the same Taylor s expasio as i (10.3), we have (10.9) N( ˆθ θ ) = Ď 1 N ǧ(θ ) = Ď 1 N(A + B + C), where Ď is the derivative of ǧ(θ ) with respect to the parameters, which coverges to its expectatio Ď as sample size rises. The estimator itself is expressed as (10.10) ˆθ = θ Ď 1 (A + B + C). We ca ow examie the properties of our estimators.

19 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio Maximum Simulated Likelihood For MSL, ǧ (θ) is ot ubiased for g (θ). The bias term i (10.9) is NB = ( N/R)Z. Suppose R rises with N. IfR rises faster tha N, the NB = ( N/R)Z p 0, sice the ratio N/R falls to zero. Cosider ow the third term i (10.9), which captures simulatio oise: NC d N(0, S/R). Sice S/R decreases as R rises, we have S/R p 0asN whe R rises with N. The secod ad third terms disappear, leavig oly the first term. This first term is the same as appears for the osimulated estimator. We have N( ˆθ θ ) = D 1 NA d N(0, D 1 WD 1 ) = N(0, H 1 VH 1 ) = N(0, H 1 ), where the ext-to-last equality occurs because g (θ) is the score, ad the last equality is due to the iformatio idetity. The estimator is distributed ˆθ a N(θ, H 1 /N). This is the same asymptotic distributio as ML. Whe R rises faster tha N, MSL is cosistet, asymptotically ormal ad efficiet, ad asymptotically equivalet to ML. Suppose that R rises with N but at a rate that is slower tha N.I this case, the ratio N/R grows larger as N rises. There is o limitig distributio for N( ˆθ θ ), because the bias term, ( N/R)Z, rises with N. However, the estimator itself coverges o the true value. ˆθ depeds o (1/R)Z, ot multiplied by N. This bias term disappears whe R rises at ay rate. Therefore, the estimator coverges o the true value, just like its osimulated couterpart, which meas that ˆθ is cosistet. However, the estimator is ot asymptotically ormal, sice N( ˆθ θ ) has o limitig distributio. Stadard errors caot be calculated, ad cofidece itervals caot be costructed. Whe R is fixed, the bias rises as N rises. N( ˆθ θ ) does ot have a limitig distributio. Moreover, the estimator itself, ˆθ, cotais a bias B = (1/R)Z that does ot disappear as sample size rises with fixed R. The MSL estimator is either cosistet or asymptotically ormal whe R is fixed.

20 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 259 The properties of MSL ca be summarized as follows: 1. If R is fixed, MSL is icosistet. 2. If R rises slower tha N, MSL is cosistet but ot asymptotically ormal. 3. If R rises faster tha N, MSL is cosistet, asymptotically ormal ad efficiet, ad equivalet to ML Method of Simulated Momets For MSM with fixed istrumets, ǧ (θ) = j [d j ˇP j (θ)]z j, which is ubiased for g (θ), sice the simulated probability eters liearly. The bias term is zero. The distributio of the estimator is determied oly by term A, which is the same as i the traditioal MOM without simulatio, ad term C, which reflects simulatio oise: N( ˆθ θ ) = Ď 1 N(A + C). Suppose that R is fixed. Sice Ď coverges to its expectatio D, we have N Ď 1 A d N(0, D 1 WD 1 ) ad N Ď 1 C d N(0, D 1 (S/R)D 1 ), so that N( ˆθ θ ) d N(0, D 1 [W + S/R]D 1 ). The asymptotic distributio of the estimator is the ˆθ a N(θ, D 1 [W + S/R]D 1 /N). The estimator is cosistet ad asymptotically ormal. Its variace is greater tha its osimulated couterpart by D 1 SD 1 /RN,reflectig simulatio oise. Suppose ow that R rises with N at ay rate. The extra variace due to simulatio oise disappears, so that ˆθ a N(θ, D 1 WD 1 /N), the same as its osimulated couterpart. Whe oideal istrumets are used, D 1 WD 1 H 1 ad so the estimator (i either its simulated or osimulated form) is less efficiet tha ML. If simulated istrumets are used i MSM, the the properties of the estimator deped o how the istrumets are simulated. If the istrumets are simulated without bias ad idepedetly of the probability that eters the residual, the this MSM has the same properties as MSM with fixed weights. If the istrumets are simulated with bias ad the istrumets are ot ideal, the the estimator has the same properties as MSL except that it is ot asymptotically efficiet, sice the iformatio

21 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= Estimatio idetity does ot apply. MSM with simulated ideal istrumets is MSS, which we discuss ext Method of Simulated Scores With MSS usig ubiased score simulators, ǧ (θ) is ubiased for g (θ), ad, moreover, g (θ) is the score such that the iformatio idetity applies. The aalysis is the same as for MSM except that the iformatio idetity makes the estimator efficiet whe R rises with N. As with MSM, we have ˆθ a N(θ, D 1 [W + S/R]D 1 /N), which, sice g (θ) is the score, becomes ˆθ a N (θ, H 1 [V + S/R]H 1 ) = N (θ, H 1 N N + H 1 SH 1 ). RN Whe R is fixed, the estimator is cosistet ad asymptotically ormal, but its covariace is larger tha with ML because of simulatio oise. If R rises at ay rate with N, the we have ˆθ a N(0, H 1 /N). MSS with ubiased score simulators is asymptotically equivalet to ML whe R rises at ay rate with N. This aalysis shows that MSS with ubiased score simulators has better properties tha MSL i two regards. First, for fixed R, MSS is cosistet ad asymptotically ormal, while MSL is either. Secod, for R risig with N, MSS is equivalet to ML o matter how fast R is risig, while MSL is equivalet to ML oly if the rate is faster tha N. As we discussed i Sectio , fidig ubiased score simulators with good umerical properties is difficult. MSS is sometimes applied with biased score simulators. I this case, the properties of the estimator are the same as with MSL: the bias i the simulated scores traslates ito bias i the estimator, which disappears from the limitig distributio oly if R rises faster tha N Numerical Solutio The estimators are defied as the value of θ that solves ǧ(θ) = 0, where ǧ(θ) = ǧ(θ)/n is the sample average of a simulated statistic ǧ (θ). Sice ǧ (θ) is a vector, we eed to solve the set of equatios for the

22 CB495-10DRV CB495/Trai KEY BOARDED August 20, :43 Char Cout= 0 Simulatio-Assisted Estimatio 261 parameters. The questio arises: how are these equatios solved umerically to obtai the estimates? Chapter 8 describes umerical methods for maximizig a fuctio. These procedures ca also be used for solvig a set of equatios. Let T be the egative of the ier product of the defiig term for a estimator: T = ǧ(θ) ǧ(θ) = ( ǧ(θ)) ( ǧ(θ))/n 2. T is ecessarily less tha or equal to zero, sice it is the egative of a sum of squares. T has a highest value of 0, which is attaied oly whe the squared terms that compose it are all 0. That is, the maximum of T is attaied whe ǧ(θ) = 0. Maximizig T is equivalet to solvig the equatio ǧ(θ) = 0. The approaches described i Chapter 8, with the exceptio of BHHH, ca be used for this maximizatio. BHHH caot be used, because that method assumes that the fuctio beig maximized is a sum of observatio-specific terms, whereas T takes the square of each sum of observatio-specific terms. The other approaches, especially BFGS ad DFP, have prove very effective at locatig the parameters at which ǧ(θ) = 0. With MSL, it is usually easier to maximize the simulated likelihood fuctio rather tha T. BHHH ca be used i this case, as well as the other methods.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Discrete Mathematics for CS Spring 2008 David Wagner Note 22

Discrete Mathematics for CS Spring 2008 David Wagner Note 22 CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig

More information

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4 MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.

More information

Stat 421-SP2012 Interval Estimation Section

Stat 421-SP2012 Interval Estimation Section Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.

Econ 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1. Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio

More information

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample. Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized

More information

1 Inferential Methods for Correlation and Regression Analysis

1 Inferential Methods for Correlation and Regression Analysis 1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

11 THE GMM ESTIMATION

11 THE GMM ESTIMATION Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.

The variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2. SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara

Econ 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio

More information

Chapter 6 Sampling Distributions

Chapter 6 Sampling Distributions Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

More information

KLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions

KLMED8004 Medical statistics. Part I, autumn Estimation. We have previously learned: Population and sample. New questions We have previously leared: KLMED8004 Medical statistics Part I, autum 00 How kow probability distributios (e.g. biomial distributio, ormal distributio) with kow populatio parameters (mea, variace) ca give

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures

FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals

More information

The standard deviation of the mean

The standard deviation of the mean Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider

More information

Lecture 2: Monte Carlo Simulation

Lecture 2: Monte Carlo Simulation STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

1.010 Uncertainty in Engineering Fall 2008

1.010 Uncertainty in Engineering Fall 2008 MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval

More information

A statistical method to determine sample size to estimate characteristic value of soil parameters

A statistical method to determine sample size to estimate characteristic value of soil parameters A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig

More information

Statistics 511 Additional Materials

Statistics 511 Additional Materials Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability

More information

AAEC/ECON 5126 FINAL EXAM: SOLUTIONS

AAEC/ECON 5126 FINAL EXAM: SOLUTIONS AAEC/ECON 5126 FINAL EXAM: SOLUTIONS SPRING 2015 / INSTRUCTOR: KLAUS MOELTNER This exam is ope-book, ope-otes, but please work strictly o your ow. Please make sure your ame is o every sheet you re hadig

More information

Understanding Samples

Understanding Samples 1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

Linear Regression Demystified

Linear Regression Demystified Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to

More information

Lecture 3. Properties of Summary Statistics: Sampling Distribution

Lecture 3. Properties of Summary Statistics: Sampling Distribution Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

11 Correlation and Regression

11 Correlation and Regression 11 Correlatio Regressio 11.1 Multivariate Data Ofte we look at data where several variables are recorded for the same idividuals or samplig uits. For example, at a coastal weather statio, we might record

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

Infinite Sequences and Series

Infinite Sequences and Series Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

More information

Frequentist Inference

Frequentist Inference Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for

More information

Element sampling: Part 2

Element sampling: Part 2 Chapter 4 Elemet samplig: Part 2 4.1 Itroductio We ow cosider uequal probability samplig desigs which is very popular i practice. I the uequal probability samplig, we ca improve the efficiecy of the resultig

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

Lecture 33: Bootstrap

Lecture 33: Bootstrap Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

More information

Chapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc.

Chapter 22. Comparing Two Proportions. Copyright 2010 Pearson Education, Inc. Chapter 22 Comparig Two Proportios Copyright 2010 Pearso Educatio, Ic. Comparig Two Proportios Comparisos betwee two percetages are much more commo tha questios about isolated percetages. Ad they are more

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5 CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

Estimation of the Mean and the ACVF

Estimation of the Mean and the ACVF Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators

More information

Data Analysis and Statistical Methods Statistics 651

Data Analysis and Statistical Methods Statistics 651 Data Aalysis ad Statistical Methods Statistics 651 http://www.stat.tamu.edu/~suhasii/teachig.html Suhasii Subba Rao Review of testig: Example The admistrator of a ursig home wats to do a time ad motio

More information

Session 5. (1) Principal component analysis and Karhunen-Loève transformation

Session 5. (1) Principal component analysis and Karhunen-Loève transformation 200 Autum semester Patter Iformatio Processig Topic 2 Image compressio by orthogoal trasformatio Sessio 5 () Pricipal compoet aalysis ad Karhue-Loève trasformatio Topic 2 of this course explais the image

More information

Introductory statistics

Introductory statistics CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key

More information

Problem Set 4 Due Oct, 12

Problem Set 4 Due Oct, 12 EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios

More information

Binomial Distribution

Binomial Distribution 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible

More information

Confidence intervals summary Conservative and approximate confidence intervals for a binomial p Examples. MATH1005 Statistics. Lecture 24. M.

Confidence intervals summary Conservative and approximate confidence intervals for a binomial p Examples. MATH1005 Statistics. Lecture 24. M. MATH1005 Statistics Lecture 24 M. Stewart School of Mathematics ad Statistics Uiversity of Sydey Outlie Cofidece itervals summary Coservative ad approximate cofidece itervals for a biomial p The aïve iterval

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

Chapter 8: Estimating with Confidence

Chapter 8: Estimating with Confidence Chapter 8: Estimatig with Cofidece Sectio 8.2 The Practice of Statistics, 4 th editio For AP* STARNES, YATES, MOORE Chapter 8 Estimatig with Cofidece 8.1 Cofidece Itervals: The Basics 8.2 8.3 Estimatig

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

This is an introductory course in Analysis of Variance and Design of Experiments.

This is an introductory course in Analysis of Variance and Design of Experiments. 1 Notes for M 384E, Wedesday, Jauary 21, 2009 (Please ote: I will ot pass out hard-copy class otes i future classes. If there are writte class otes, they will be posted o the web by the ight before class

More information

ENGI 4421 Confidence Intervals (Two Samples) Page 12-01

ENGI 4421 Confidence Intervals (Two Samples) Page 12-01 ENGI 44 Cofidece Itervals (Two Samples) Page -0 Two Sample Cofidece Iterval for a Differece i Populatio Meas [Navidi sectios 5.4-5.7; Devore chapter 9] From the cetral limit theorem, we kow that, for sufficietly

More information

Chapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.

Chapter 22. Comparing Two Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc. Chapter 22 Comparig Two Proportios Copyright 2010, 2007, 2004 Pearso Educatio, Ic. Comparig Two Proportios Read the first two paragraphs of pg 504. Comparisos betwee two percetages are much more commo

More information

6 Sample Size Calculations

6 Sample Size Calculations 6 Sample Size Calculatios Oe of the major resposibilities of a cliical trial statisticia is to aid the ivestigators i determiig the sample size required to coduct a study The most commo procedure for determiig

More information

Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett

Lecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets

More information

32 estimating the cumulative distribution function

32 estimating the cumulative distribution function 32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

More information

If, for instance, we were required to test whether the population mean μ could be equal to a certain value μ

If, for instance, we were required to test whether the population mean μ could be equal to a certain value μ STATISTICAL INFERENCE INTRODUCTION Statistical iferece is that brach of Statistics i which oe typically makes a statemet about a populatio based upo the results of a sample. I oesample testig, we essetially

More information

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10

DS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10 DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Estimation for Complete Data

Estimation for Complete Data Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of

More information

NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS

NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS STRUCTURE OF EXAMINATION PAPER. There will be oe 2-hour paper cosistig of 4 questios.

More information

CS284A: Representations and Algorithms in Molecular Biology

CS284A: Representations and Algorithms in Molecular Biology CS284A: Represetatios ad Algorithms i Molecular Biology Scribe Notes o Lectures 3 & 4: Motif Discovery via Eumeratio & Motif Represetatio Usig Positio Weight Matrix Joshua Gervi Based o presetatios by

More information

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons Statistical Aalysis o Ucertaity for Autocorrelated Measuremets ad its Applicatios to Key Comparisos Nie Fa Zhag Natioal Istitute of Stadards ad Techology Gaithersburg, MD 0899, USA Outlies. Itroductio.

More information

Review Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn

Review Questions, Chapters 8, 9. f(y) = 0, elsewhere. F (y) = f Y(1) = n ( e y/θ) n 1 1 θ e y/θ = n θ e yn Stat 366 Lab 2 Solutios (September 2, 2006) page TA: Yury Petracheko, CAB 484, yuryp@ualberta.ca, http://www.ualberta.ca/ yuryp/ Review Questios, Chapters 8, 9 8.5 Suppose that Y, Y 2,..., Y deote a radom

More information

Confidence Intervals for the Population Proportion p

Confidence Intervals for the Population Proportion p Cofidece Itervals for the Populatio Proportio p The cocept of cofidece itervals for the populatio proportio p is the same as the oe for, the samplig distributio of the mea, x. The structure is idetical:

More information

Lecture 12: September 27

Lecture 12: September 27 36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

More information

7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals

7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals 7-1 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7- Sectio 1. Samplig Distributio 7-3 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses

More information

Statistical Properties of OLS estimators

Statistical Properties of OLS estimators 1 Statistical Properties of OLS estimators Liear Model: Y i = β 0 + β 1 X i + u i OLS estimators: β 0 = Y β 1X β 1 = Best Liear Ubiased Estimator (BLUE) Liear Estimator: β 0 ad β 1 are liear fuctio of

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

ECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015

ECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015 ECE 8527: Itroductio to Machie Learig ad Patter Recogitio Midterm # 1 Vaishali Ami Fall, 2015 tue39624@temple.edu Problem No. 1: Cosider a two-class discrete distributio problem: ω 1 :{[0,0], [2,0], [2,2],

More information

10-701/ Machine Learning Mid-term Exam Solution

10-701/ Machine Learning Mid-term Exam Solution 0-70/5-78 Machie Learig Mid-term Exam Solutio Your Name: Your Adrew ID: True or False (Give oe setece explaatio) (20%). (F) For a cotiuous radom variable x ad its probability distributio fuctio p(x), it

More information

Machine Learning Brett Bernstein

Machine Learning Brett Bernstein Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio

More information

Simulation. Two Rule For Inverting A Distribution Function

Simulation. Two Rule For Inverting A Distribution Function Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump

More information

Tests of Hypotheses Based on a Single Sample (Devore Chapter Eight)

Tests of Hypotheses Based on a Single Sample (Devore Chapter Eight) Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........

More information

CSE 527, Additional notes on MLE & EM

CSE 527, Additional notes on MLE & EM CSE 57 Lecture Notes: MLE & EM CSE 57, Additioal otes o MLE & EM Based o earlier otes by C. Grat & M. Narasimha Itroductio Last lecture we bega a examiatio of model based clusterig. This lecture will be

More information

An Introduction to Asymptotic Theory

An Introduction to Asymptotic Theory A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by

More information

The Method of Least Squares. To understand least squares fitting of data.

The Method of Least Squares. To understand least squares fitting of data. The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve

More information

Big Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates.

Big Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates. 5. Data, Estimates, ad Models: quatifyig the accuracy of estimates. 5. Estimatig a Normal Mea 5.2 The Distributio of the Normal Sample Mea 5.3 Normal data, cofidece iterval for, kow 5.4 Normal data, cofidece

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should be doe

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

Output Analysis and Run-Length Control

Output Analysis and Run-Length Control IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%

More information

CEU Department of Economics Econometrics 1, Problem Set 1 - Solutions

CEU Department of Economics Econometrics 1, Problem Set 1 - Solutions CEU Departmet of Ecoomics Ecoometrics, Problem Set - Solutios Part A. Exogeeity - edogeeity The liear coditioal expectatio (CE) model has the followig form: We would like to estimate the effect of some

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 9

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 9 Hypothesis testig PSYCHOLOGICAL RESEARCH (PYC 34-C Lecture 9 Statistical iferece is that brach of Statistics i which oe typically makes a statemet about a populatio based upo the results of a sample. I

More information

Estimation of a population proportion March 23,

Estimation of a population proportion March 23, 1 Social Studies 201 Notes for March 23, 2005 Estimatio of a populatio proportio Sectio 8.5, p. 521. For the most part, we have dealt with meas ad stadard deviatios this semester. This sectio of the otes

More information

Module 1 Fundamentals in statistics

Module 1 Fundamentals in statistics Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly

More information

3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense,

3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense, 3. Z Trasform Referece: Etire Chapter 3 of text. Recall that the Fourier trasform (FT) of a DT sigal x [ ] is ω ( ) [ ] X e = j jω k = xe I order for the FT to exist i the fiite magitude sese, S = x [

More information

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the

More information

Expectation and Variance of a random variable

Expectation and Variance of a random variable Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio

More information

4. Partial Sums and the Central Limit Theorem

4. Partial Sums and the Central Limit Theorem 1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems

More information

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio

More information

Statisticians use the word population to refer the total number of (potential) observations under consideration

Statisticians use the word population to refer the total number of (potential) observations under consideration 6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space

More information

Math 10A final exam, December 16, 2016

Math 10A final exam, December 16, 2016 Please put away all books, calculators, cell phoes ad other devices. You may cosult a sigle two-sided sheet of otes. Please write carefully ad clearly, USING WORDS (ot just symbols). Remember that the

More information

POLS, GLS, FGLS, GMM. Outline of Linear Systems of Equations. Common Coefficients, Panel Data Model. Preliminaries

POLS, GLS, FGLS, GMM. Outline of Linear Systems of Equations. Common Coefficients, Panel Data Model. Preliminaries Outlie of Liear Systems of Equatios POLS, GLS, FGLS, GMM Commo Coefficiets, Pael Data Model Prelimiaries he liear pael data model is a static model because all explaatory variables are dated cotemporaeously

More information