Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference


 Marian Cole
 11 months ago
 Views:
Transcription
1 1 / 171 Bootstrap inference Francisco CribariNeto Departamento de Estatística Universidade Federal de Pernambuco Recife / PE, Brazil October 2013
2 2 / 171 Unpaid advertisement Graduate program in Statistics: Masters and PhD Graduate Program in Statistics at Federal University of Pernambuco: CAPES: 5 Research areas: asymptotic theory, econometrics, game theory, multivariate analysis, probability theory, regression analysis, signal processing, time series.
3 3 / 171 Figure 1 : Boa Viagem beach.
4 4 / 171 Figure 2 : Boa Viagem beach (at night).
5 5 / 171 Figure 3 : Recife ( the Brazilian Venice ).
6 6 / 171 Figure 4 : Porto de Galinhas beach (near Recife).
7 7 / 171 In a world in which the price of calculation continues to decrease rapidly, but the price of theorem proving continues to hold steady or increase, elementary economics indicates that we ought to spend a larger fraction of our time on calculation. John W. Tukey, 1986
8 8 / 171 Figure 5 : This is the man.
9 9 / 171 Some references General 1 Chernick, M.R. (1999). Bootstrap Methods: A Practitioner s Guide. New York: Wiley. 2 Davison, A.C. & Hinkley, D.V. (1997). Bootstrap Methods and their Application. Cambridge: Cambridge University Press. 3 Efon, B. (1982) The Jackknife, the Bootstrap and Other Resampling Plans. Philadelphia: Society for Industrial and Applied Mathematics. 4 Efron, B. & Tibshirani, R.J. (1993). An Introduction to the Bootstrap. New York: Chapman & Hall.
10 10 / 171 Some references General 5 Godfrey, L. (2009). Bootstrap Tests for Regression Models. New York: Palgrave MacMillan. 6 Hall, P. (1992). The Bootstrap and Edgeworth Expansion. New York: Springer Verlag. 7 Shao, J. & Tu, D. (1995). The Jackknife and Bootstrap. New York: Springer.
11 11 / 171 Some references Specific 1 Booth, J.G. & Hall, P. (1994). Monte Carlo approximation and the iterated bootstrap. Biometrika, 81, CribariNeto, F. & Zarkos, S.G. (1999). Bootstrap methods for heteroskedastic regression models: evidence on estimation and testing. Econometric Reviews, 18, CribariNeto, F. & Zarkos, S.G. (2001). Heteroskedasticityconsistent covariance matrix estimation: White s estimator and the bootstrap. Journal of Statistical Computation and Simulation, 68,
12 12 / 171 Some references Specific (cont.) 4 CribariNeto, F. (2004). Asymptotic inference under heteroskedasticity of unknown form. Computational Statistics and Data Analysis, 45, CribariNeto, F.; Frery, A.C. & Silva, M.F. (2002). Improved estimation of clutter properties in speckled imagery. Computational Statistics and Data Analysis, 40, Davidson, R. & MacKinnon, J.G. (2000). Bootstrap tests: how many bootstraps? Econometric Reviews, 19,
13 13 / 171 Some references Specific (cont.) 7 Ferrari, S.L.P. & CribariNeto, F. (1997). On bootstrap and analytical bias corrections. Economics Letters, 58, Ferrari, S.L.P. & CribariNeto, F. (1999). On the robustness of analytical and bootstrap corrections to score tests in regression models. Journal of Statistical Computation and Simulation, 64,
14 14 / 171 Some references Specific (cont.) 9 Lemonte, A.J.; Simas, A.B.; CribariNeto, F. (2008). Bootstrapbased improved estimators for the twoparameter BirnbaumSaunders distribution. Journal of Statistical Computation and Simulation, 78, MacKinnon, J.G. & Smith, Jr., A.A. (1998). Approximate bias correction in econometrics. Journal of Econometrics, 85, Wu, C.F.J. (1996). Jackknife, bootstrap and other resampling methods in regression analysis (with discussion). Annals of Statistics, 14,
15 15 / 171 A fundamental equation Figure 6 : C.R. Rao C.R. Rao: uncertain knowledge + knowledge about the uncertainty = useful knowledge
16 15 / 171 A fundamental equation Figure 6 : C.R. Rao C.R. Rao: uncertain knowledge + knowledge about the uncertainty = useful knowledge
17 15 / 171 A fundamental equation Figure 6 : C.R. Rao C.R. Rao: uncertain knowledge + knowledge about the uncertainty = useful knowledge
18 15 / 171 A fundamental equation Figure 6 : C.R. Rao C.R. Rao: uncertain knowledge + knowledge about the uncertainty = useful knowledge
19 16 / 171 As Anthony Davison and David Hinkley remind us... The explicity recognition of uncertainty is central to statistical sciences. Notions such as prior information, probability models, likelihood, standard errors and confidence limits are all intended to formalize uncertainty and thereby make allowance for it. Davison & Hinkley
20 16 / 171 As Anthony Davison and David Hinkley remind us... The explicity recognition of uncertainty is central to statistical sciences. Notions such as prior information, probability models, likelihood, standard errors and confidence limits are all intended to formalize uncertainty and thereby make allowance for it. Davison & Hinkley
21 17 / 171 The big picture (the grand scheme of things) Sampling POPULATION DATA model = f(parameters) The great scheme of things.
22 18 / 171 What is the bootstrap? The bootstrap is a computerbased method for assessing the accuracy of statistical estimates and tests. It was first proposed by Bradley Efron in a 1979 Annals of Statistics paper. Main idea: Treat the data as if they were the (true, unknown) population, and draw samples (with replacement) from the data as if you were sampling from the population. Repeat the procedure a large number of times (say, B), each time computing the quantity of interest. Then, use the B values of the quantity of interest to estimate its unknown distribution.
23 18 / 171 What is the bootstrap? The bootstrap is a computerbased method for assessing the accuracy of statistical estimates and tests. It was first proposed by Bradley Efron in a 1979 Annals of Statistics paper. Main idea: Treat the data as if they were the (true, unknown) population, and draw samples (with replacement) from the data as if you were sampling from the population. Repeat the procedure a large number of times (say, B), each time computing the quantity of interest. Then, use the B values of the quantity of interest to estimate its unknown distribution.
24 18 / 171 What is the bootstrap? The bootstrap is a computerbased method for assessing the accuracy of statistical estimates and tests. It was first proposed by Bradley Efron in a 1979 Annals of Statistics paper. Main idea: Treat the data as if they were the (true, unknown) population, and draw samples (with replacement) from the data as if you were sampling from the population. Repeat the procedure a large number of times (say, B), each time computing the quantity of interest. Then, use the B values of the quantity of interest to estimate its unknown distribution.
25 19 / 171 In a nutshell... population sample }{{} real world bootstrap samples }{{} virtual world
26 20 / 171 Does it work? Question: Does it work well? Answer: Yes (most of the time). In the simplest nonparametric problems we do literally sample from the data, and a common initial reaction is that this is a fraud. In fact it is not. Davison and Hinkley, 1997
27 20 / 171 Does it work? Question: Does it work well? Answer: Yes (most of the time). In the simplest nonparametric problems we do literally sample from the data, and a common initial reaction is that this is a fraud. In fact it is not. Davison and Hinkley, 1997
28 21 / 171 Asymptotic refinement Question: When does the bootstrap provide an asymptotic refinement? The quantity being bootstrapped must be asymptotically pivotal. That is: It must have a limiting distribution free of unknown parameters.
29 21 / 171 Asymptotic refinement Question: When does the bootstrap provide an asymptotic refinement? The quantity being bootstrapped must be asymptotically pivotal. That is: It must have a limiting distribution free of unknown parameters.
30 21 / 171 Asymptotic refinement Question: When does the bootstrap provide an asymptotic refinement? The quantity being bootstrapped must be asymptotically pivotal. That is: It must have a limiting distribution free of unknown parameters.
31 22 / 171 Point estimation in a nutshell Suppose that model that represents the population is indexed by the parameter θ = (θ 1,..., θ p ) Θ, where Θ is the parameter space. Estimator: A statistic used to estimate θ. The estimator, say θ, is typically obtained from the minimization of some undesirable quantity (e.g., sum of squared errors) or from the maximization of some desirable quantity.
32 22 / 171 Point estimation in a nutshell Suppose that model that represents the population is indexed by the parameter θ = (θ 1,..., θ p ) Θ, where Θ is the parameter space. Estimator: A statistic used to estimate θ. The estimator, say θ, is typically obtained from the minimization of some undesirable quantity (e.g., sum of squared errors) or from the maximization of some desirable quantity.
33 22 / 171 Point estimation in a nutshell Suppose that model that represents the population is indexed by the parameter θ = (θ 1,..., θ p ) Θ, where Θ is the parameter space. Estimator: A statistic used to estimate θ. The estimator, say θ, is typically obtained from the minimization of some undesirable quantity (e.g., sum of squared errors) or from the maximization of some desirable quantity.
34 23 / 171 Point estimation in a nutshell (cont.) Some of the most important properties an estimator can enjoy are: Unbiasedness: IE( θ) = θ θ Θ; Consistency ( ): θ p θ; Asymptotic normality: When n is large, θ approx normally distributed; Efficiency: (more generally, optimality in some class; e.g., GaussMarkov Theorem).
35 23 / 171 Point estimation in a nutshell (cont.) Some of the most important properties an estimator can enjoy are: Unbiasedness: IE( θ) = θ θ Θ; Consistency ( ): θ p θ; Asymptotic normality: When n is large, θ approx normally distributed; Efficiency: (more generally, optimality in some class; e.g., GaussMarkov Theorem).
36 23 / 171 Point estimation in a nutshell (cont.) Some of the most important properties an estimator can enjoy are: Unbiasedness: IE( θ) = θ θ Θ; Consistency ( ): θ p θ; Asymptotic normality: When n is large, θ approx normally distributed; Efficiency: (more generally, optimality in some class; e.g., GaussMarkov Theorem).
37 23 / 171 Point estimation in a nutshell (cont.) Some of the most important properties an estimator can enjoy are: Unbiasedness: IE( θ) = θ θ Θ; Consistency ( ): θ p θ; Asymptotic normality: When n is large, θ approx normally distributed; Efficiency: (more generally, optimality in some class; e.g., GaussMarkov Theorem).
38 23 / 171 Point estimation in a nutshell (cont.) Some of the most important properties an estimator can enjoy are: Unbiasedness: IE( θ) = θ θ Θ; Consistency ( ): θ p θ; Asymptotic normality: When n is large, θ approx normally distributed; Efficiency: (more generally, optimality in some class; e.g., GaussMarkov Theorem).
39 24 / 171 Setup Y 1,..., Y n i.i.d. F 0 (θ), where θ Θ IR p. We can write the unknown parameter θ as a functional of F 0 : θ = θ(f 0 ). We can denote an estimator of θ (say, the MLE) as θ, which can be written as the functional θ = θ( F), where F is the empirical c.d.f. of Y 1,..., Y n.
40 24 / 171 Setup Y 1,..., Y n i.i.d. F 0 (θ), where θ Θ IR p. We can write the unknown parameter θ as a functional of F 0 : θ = θ(f 0 ). We can denote an estimator of θ (say, the MLE) as θ, which can be written as the functional θ = θ( F), where F is the empirical c.d.f. of Y 1,..., Y n.
41 24 / 171 Setup Y 1,..., Y n i.i.d. F 0 (θ), where θ Θ IR p. We can write the unknown parameter θ as a functional of F 0 : θ = θ(f 0 ). We can denote an estimator of θ (say, the MLE) as θ, which can be written as the functional θ = θ( F), where F is the empirical c.d.f. of Y 1,..., Y n.
42 25 / 171 Plugin principle Y 1,..., Y n iid F0. Plugin Write the parameter as θ = θ(f 0 ). Estimator: θ = θ( F). Example: mean Parameter: θ(f 0 ) = y df 0 = IE(Y). Estimator: θ = y d F = n 1 n i=1 y i = y.
43 25 / 171 Plugin principle Y 1,..., Y n iid F0. Plugin Write the parameter as θ = θ(f 0 ). Estimator: θ = θ( F). Example: mean Parameter: θ(f 0 ) = y df 0 = IE(Y). Estimator: θ = y d F = n 1 n i=1 y i = y.
44 25 / 171 Plugin principle Y 1,..., Y n iid F0. Plugin Write the parameter as θ = θ(f 0 ). Estimator: θ = θ( F). Example: mean Parameter: θ(f 0 ) = y df 0 = IE(Y). Estimator: θ = y d F = n 1 n i=1 y i = y.
45 25 / 171 Plugin principle Y 1,..., Y n iid F0. Plugin Write the parameter as θ = θ(f 0 ). Estimator: θ = θ( F). Example: mean Parameter: θ(f 0 ) = y df 0 = IE(Y). Estimator: θ = y d F = n 1 n i=1 y i = y.
46 25 / 171 Plugin principle Y 1,..., Y n iid F0. Plugin Write the parameter as θ = θ(f 0 ). Estimator: θ = θ( F). Example: mean Parameter: θ(f 0 ) = y df 0 = IE(Y). Estimator: θ = y d F = n 1 n i=1 y i = y.
47 Main idea: Plugin principle. Example: Let Y = n 1 n i=1 Y i. We know that if Y i (µ, σ 2 ), then Y (µ, n 1 σ 2 ), so that n 1 σ 2 gives us an indication of the accuracy of the estimate Y. In particular, the standard error of the estimate can be obtained as s.e.(y) = σ 2 n, σ2 = 1 n 1 n (y i ȳ) 2, ( ) for a given observed sample Y 1 = y 1,..., Y n = y n. Bootstrap approach: Write σ 2 = σ 2 (F 0 ), and replace F 0 by F to obtain: σ 2 b.s.e.(y) = n, σ2 σ 2 ( F) = 1 n (y i ȳ) 2. n This is the bootstrap estimate. i=1 i=1 26 / 171
48 Main idea: Plugin principle. Example: Let Y = n 1 n i=1 Y i. We know that if Y i (µ, σ 2 ), then Y (µ, n 1 σ 2 ), so that n 1 σ 2 gives us an indication of the accuracy of the estimate Y. In particular, the standard error of the estimate can be obtained as s.e.(y) = σ 2 n, σ2 = 1 n 1 n (y i ȳ) 2, ( ) for a given observed sample Y 1 = y 1,..., Y n = y n. Bootstrap approach: Write σ 2 = σ 2 (F 0 ), and replace F 0 by F to obtain: σ 2 b.s.e.(y) = n, σ2 σ 2 ( F) = 1 n (y i ȳ) 2. n This is the bootstrap estimate. i=1 i=1 26 / 171
49 Main idea: Plugin principle. Example: Let Y = n 1 n i=1 Y i. We know that if Y i (µ, σ 2 ), then Y (µ, n 1 σ 2 ), so that n 1 σ 2 gives us an indication of the accuracy of the estimate Y. In particular, the standard error of the estimate can be obtained as s.e.(y) = σ 2 n, σ2 = 1 n 1 n (y i ȳ) 2, ( ) for a given observed sample Y 1 = y 1,..., Y n = y n. Bootstrap approach: Write σ 2 = σ 2 (F 0 ), and replace F 0 by F to obtain: σ 2 b.s.e.(y) = n, σ2 σ 2 ( F) = 1 n (y i ȳ) 2. n This is the bootstrap estimate. i=1 i=1 26 / 171
50 27 / 171 Noteworthy Note: (i) the difference between the two estimates is minor and vanishes as n ; (ii) F places probability mass 1/n on y 1,..., y n.
51 27 / 171 Noteworthy Note: (i) the difference between the two estimates is minor and vanishes as n ; (ii) F places probability mass 1/n on y 1,..., y n.
52 28 / 171 Problem: We are usually interested in estimates more complicated than the sample mean, and for such statistics we may not have a directly available formula like ( ). [E.g., we may be interested in the correlation coefficient, the median, a given quantile, the coefficients of a quantile regression, etc.] Solution: The bootstrap approach allows us to numerically evaluate σ 2 = σ 2 ( F).
53 28 / 171 Problem: We are usually interested in estimates more complicated than the sample mean, and for such statistics we may not have a directly available formula like ( ). [E.g., we may be interested in the correlation coefficient, the median, a given quantile, the coefficients of a quantile regression, etc.] Solution: The bootstrap approach allows us to numerically evaluate σ 2 = σ 2 ( F).
54 29 / 171 Bootstrap standard error Question: How can we use bootstrap resampling to obtain a standard error for a given estimate?
55 30 / 171 What s in a number?
56 31 / 171 The basic bootstrap algorithm Suppose we wish to obtain an standard error for θ = θ( F), an estimate of θ = θ(f 0 ), from an i.i.d. sample of size n. Here is how we proceed: 1 Compute θ for our sample. 2 Sample from the data with replacement and construct a new sample of size n, say y = (y 1,..., y n). 3 Compute θ for the bootstrap sample obtained in (ii). 4 Repeat steps (ii) and (iii) B times. 5 Use the B realizations of θ to obtain an estimate for the standard error of θ.
57 32 / 171 The basic bootstrap algorithm That is, where 1 b.s.e.( θ ) = B B 1 b=1 { θ b θ ( )} 2, θ ( ) = 1 B B θ b. b=1
58 33 / 171 It is important to notice that... Note that the bootstrap generalizes the jackknife in the sense that resampling is carried out in a random fashion, and not in a deterministic and systematic way ( leave one out ).
59 34 / 171 Parametric versus nonparametric bootstrap The bootstrap may be performed parametrically or nonparametrically. Nonpametric bootstrap: Resampling from F. That is, sample from the data (with replacement). Parametric bootstrap: Sample from F( θ ). The nonparametric bootstrap is more robust against distributional assumptions whereas the parametric bootstrap is expected to be more efficient when the parametric assumptions are true.
60 34 / 171 Parametric versus nonparametric bootstrap The bootstrap may be performed parametrically or nonparametrically. Nonpametric bootstrap: Resampling from F. That is, sample from the data (with replacement). Parametric bootstrap: Sample from F( θ ). The nonparametric bootstrap is more robust against distributional assumptions whereas the parametric bootstrap is expected to be more efficient when the parametric assumptions are true.
61 34 / 171 Parametric versus nonparametric bootstrap The bootstrap may be performed parametrically or nonparametrically. Nonpametric bootstrap: Resampling from F. That is, sample from the data (with replacement). Parametric bootstrap: Sample from F( θ ). The nonparametric bootstrap is more robust against distributional assumptions whereas the parametric bootstrap is expected to be more efficient when the parametric assumptions are true.
62 34 / 171 Parametric versus nonparametric bootstrap The bootstrap may be performed parametrically or nonparametrically. Nonpametric bootstrap: Resampling from F. That is, sample from the data (with replacement). Parametric bootstrap: Sample from F( θ ). The nonparametric bootstrap is more robust against distributional assumptions whereas the parametric bootstrap is expected to be more efficient when the parametric assumptions are true.
63 35 / 171 Noparametric bootstrap sampling EMPIRICAL DISTRIBUTION: Puts equal weight probabilities n 1 at each sample value y i. EMPIRICAL DISTRIBUTION FUNCTION (EDF): F(y) = #{y i y}. n Notice that the values of the EDF are fixed: (0, 1/n, 2/n,..., n/n). Hence, the EDF is equivalent to its points of increase: y (1) y (n) (ordered sample values).
64 35 / 171 Noparametric bootstrap sampling EMPIRICAL DISTRIBUTION: Puts equal weight probabilities n 1 at each sample value y i. EMPIRICAL DISTRIBUTION FUNCTION (EDF): F(y) = #{y i y}. n Notice that the values of the EDF are fixed: (0, 1/n, 2/n,..., n/n). Hence, the EDF is equivalent to its points of increase: y (1) y (n) (ordered sample values).
65 35 / 171 Noparametric bootstrap sampling EMPIRICAL DISTRIBUTION: Puts equal weight probabilities n 1 at each sample value y i. EMPIRICAL DISTRIBUTION FUNCTION (EDF): F(y) = #{y i y}. n Notice that the values of the EDF are fixed: (0, 1/n, 2/n,..., n/n). Hence, the EDF is equivalent to its points of increase: y (1) y (n) (ordered sample values).
66 36 / 171 Noparametric bootstrap sampling (cont.) Since the EDF puts equal probabilities on the data values y 1,..., y n, each Y is independently sampled at random from those data values. Hence, the bootstrap sample is a random sample taken with replacement from the original data.
67 36 / 171 Noparametric bootstrap sampling (cont.) Since the EDF puts equal probabilities on the data values y 1,..., y n, each Y is independently sampled at random from those data values. Hence, the bootstrap sample is a random sample taken with replacement from the original data.
68 37 / 171 Smoothed bootstrap The empirical distribution function ( F) is discrete. Sampling from it boils down to sampling from data with replacement. An interesting idea: To sample from a smoothed distribution function. We replace F by a smooth distribution based on, e.g., a kernel density estimate of F (the derivative of F with respect to y). An example using the correlation coefficient can be found in Efron s 1982 monograph: Efron, B. (1982). The Jackknife, the Bootstrap, and Other Resampling Plans. Philadelphia: SIAM.
69 37 / 171 Smoothed bootstrap The empirical distribution function ( F) is discrete. Sampling from it boils down to sampling from data with replacement. An interesting idea: To sample from a smoothed distribution function. We replace F by a smooth distribution based on, e.g., a kernel density estimate of F (the derivative of F with respect to y). An example using the correlation coefficient can be found in Efron s 1982 monograph: Efron, B. (1982). The Jackknife, the Bootstrap, and Other Resampling Plans. Philadelphia: SIAM.
70 37 / 171 Smoothed bootstrap The empirical distribution function ( F) is discrete. Sampling from it boils down to sampling from data with replacement. An interesting idea: To sample from a smoothed distribution function. We replace F by a smooth distribution based on, e.g., a kernel density estimate of F (the derivative of F with respect to y). An example using the correlation coefficient can be found in Efron s 1982 monograph: Efron, B. (1982). The Jackknife, the Bootstrap, and Other Resampling Plans. Philadelphia: SIAM.
71 37 / 171 Smoothed bootstrap The empirical distribution function ( F) is discrete. Sampling from it boils down to sampling from data with replacement. An interesting idea: To sample from a smoothed distribution function. We replace F by a smooth distribution based on, e.g., a kernel density estimate of F (the derivative of F with respect to y). An example using the correlation coefficient can be found in Efron s 1982 monograph: Efron, B. (1982). The Jackknife, the Bootstrap, and Other Resampling Plans. Philadelphia: SIAM.
72 38 / 171 Bayesian bootstrap Suppose y 1,..., y n are i.i.d. realizations of Y which has distribution function F(θ), where θ is scalar. Let θ be an estimator of θ. We know that the bootstrap can be used to construct an estimate of the distribution of such an estimator. Instead of sampling from the data with replacement (i.e., sampling each y i with probability 1/n), the Bayesian bootstrap uses a posterior probability distribution for y i. The posterior probability distribution is centered at 1/n, but varies for each y i. How is that done?
73 38 / 171 Bayesian bootstrap Suppose y 1,..., y n are i.i.d. realizations of Y which has distribution function F(θ), where θ is scalar. Let θ be an estimator of θ. We know that the bootstrap can be used to construct an estimate of the distribution of such an estimator. Instead of sampling from the data with replacement (i.e., sampling each y i with probability 1/n), the Bayesian bootstrap uses a posterior probability distribution for y i. The posterior probability distribution is centered at 1/n, but varies for each y i. How is that done?
74 38 / 171 Bayesian bootstrap Suppose y 1,..., y n are i.i.d. realizations of Y which has distribution function F(θ), where θ is scalar. Let θ be an estimator of θ. We know that the bootstrap can be used to construct an estimate of the distribution of such an estimator. Instead of sampling from the data with replacement (i.e., sampling each y i with probability 1/n), the Bayesian bootstrap uses a posterior probability distribution for y i. The posterior probability distribution is centered at 1/n, but varies for each y i. How is that done?
75 38 / 171 Bayesian bootstrap Suppose y 1,..., y n are i.i.d. realizations of Y which has distribution function F(θ), where θ is scalar. Let θ be an estimator of θ. We know that the bootstrap can be used to construct an estimate of the distribution of such an estimator. Instead of sampling from the data with replacement (i.e., sampling each y i with probability 1/n), the Bayesian bootstrap uses a posterior probability distribution for y i. The posterior probability distribution is centered at 1/n, but varies for each y i. How is that done?
76 39 / 171 Bayesian bootstrap Draw a random sample of size n 1 from the standard uniform distribution. Order the sampled values: u (1),..., u (n 1). Let u (0) = 0 and u (n) = 1. Compute g i = u (i) u (i 1), i = 1,..., n. The g i s are called the gaps between the uniform order statistics. The vector g = (g 1,..., g n ) is used to assign probabilities to the Bayesian bootstrap. That is, sample y i with probability g i (not 1/n). Note that we obtain a different g in each bootstrap replication.
77 39 / 171 Bayesian bootstrap Draw a random sample of size n 1 from the standard uniform distribution. Order the sampled values: u (1),..., u (n 1). Let u (0) = 0 and u (n) = 1. Compute g i = u (i) u (i 1), i = 1,..., n. The g i s are called the gaps between the uniform order statistics. The vector g = (g 1,..., g n ) is used to assign probabilities to the Bayesian bootstrap. That is, sample y i with probability g i (not 1/n). Note that we obtain a different g in each bootstrap replication.
78 39 / 171 Bayesian bootstrap Draw a random sample of size n 1 from the standard uniform distribution. Order the sampled values: u (1),..., u (n 1). Let u (0) = 0 and u (n) = 1. Compute g i = u (i) u (i 1), i = 1,..., n. The g i s are called the gaps between the uniform order statistics. The vector g = (g 1,..., g n ) is used to assign probabilities to the Bayesian bootstrap. That is, sample y i with probability g i (not 1/n). Note that we obtain a different g in each bootstrap replication.
79 39 / 171 Bayesian bootstrap Draw a random sample of size n 1 from the standard uniform distribution. Order the sampled values: u (1),..., u (n 1). Let u (0) = 0 and u (n) = 1. Compute g i = u (i) u (i 1), i = 1,..., n. The g i s are called the gaps between the uniform order statistics. The vector g = (g 1,..., g n ) is used to assign probabilities to the Bayesian bootstrap. That is, sample y i with probability g i (not 1/n). Note that we obtain a different g in each bootstrap replication.
80 39 / 171 Bayesian bootstrap Draw a random sample of size n 1 from the standard uniform distribution. Order the sampled values: u (1),..., u (n 1). Let u (0) = 0 and u (n) = 1. Compute g i = u (i) u (i 1), i = 1,..., n. The g i s are called the gaps between the uniform order statistics. The vector g = (g 1,..., g n ) is used to assign probabilities to the Bayesian bootstrap. That is, sample y i with probability g i (not 1/n). Note that we obtain a different g in each bootstrap replication.
81 39 / 171 Bayesian bootstrap Draw a random sample of size n 1 from the standard uniform distribution. Order the sampled values: u (1),..., u (n 1). Let u (0) = 0 and u (n) = 1. Compute g i = u (i) u (i 1), i = 1,..., n. The g i s are called the gaps between the uniform order statistics. The vector g = (g 1,..., g n ) is used to assign probabilities to the Bayesian bootstrap. That is, sample y i with probability g i (not 1/n). Note that we obtain a different g in each bootstrap replication.
82 40 / 171 Bayesian bootstrap ADVANTAGE: It can used to make Bayesian inference on θ based on the estimated posterior distribution for θ. The bootstrap distribution of θ and the Bayesian bootstrap posterior distribution for θ will be similar in many applications. Reference: Rubin, D.B. (1981). The Bayesian bootstrap. Annals of Statistics, 9,
83 40 / 171 Bayesian bootstrap ADVANTAGE: It can used to make Bayesian inference on θ based on the estimated posterior distribution for θ. The bootstrap distribution of θ and the Bayesian bootstrap posterior distribution for θ will be similar in many applications. Reference: Rubin, D.B. (1981). The Bayesian bootstrap. Annals of Statistics, 9,
84 40 / 171 Bayesian bootstrap ADVANTAGE: It can used to make Bayesian inference on θ based on the estimated posterior distribution for θ. The bootstrap distribution of θ and the Bayesian bootstrap posterior distribution for θ will be similar in many applications. Reference: Rubin, D.B. (1981). The Bayesian bootstrap. Annals of Statistics, 9,
85 41 / 171 Revisiting the big picture Sampling POPULATION DATA model = f(parameters) The great scheme of things.
86 42 / 171 Software Programming Programming bootstrap resampling is easy: (i) PARAMETRIC: Sample from F( θ), (ii) NONPARAMETRIC: Sample from F (empirical distribution function). Sampling from F: 1) Obtain a standard uniform draw, i.e., obtain u from U(0, 1). 2) Generate a random integer (sau, i ) from {1,..., n} as u n + 1. The ith observation in the bootstrap sample is i th observation in the original sample.
87 42 / 171 Software Programming Programming bootstrap resampling is easy: (i) PARAMETRIC: Sample from F( θ), (ii) NONPARAMETRIC: Sample from F (empirical distribution function). Sampling from F: 1) Obtain a standard uniform draw, i.e., obtain u from U(0, 1). 2) Generate a random integer (sau, i ) from {1,..., n} as u n + 1. The ith observation in the bootstrap sample is i th observation in the original sample.
88 43 / 171 Software R package boot: functions and datasets for bootstrapping from the book Bootstrap Methods and Their Applications by A.C. Davison and D.V. Hinkley (1997, Cambridge University Press).
89 44 / 171 Bootstrap bias correction Suppose that θ is biased (although consistent) for θ, and that we would like to obtain a new estimate which is biascorrected up to some order of accuracy. The bias of θ is: IE( θ) θ (systematic error). Ideally, we would like to have: θ bias, but this is not feasible (since the bias depends on θ). Define the biascorrected estimate as: θ = θ bias. We then take bias to be bias B = θ ( ) θ, which implies that θ = θ { θ ( ) θ} = 2 θ θ ( ).
90 44 / 171 Bootstrap bias correction Suppose that θ is biased (although consistent) for θ, and that we would like to obtain a new estimate which is biascorrected up to some order of accuracy. The bias of θ is: IE( θ) θ (systematic error). Ideally, we would like to have: θ bias, but this is not feasible (since the bias depends on θ). Define the biascorrected estimate as: θ = θ bias. We then take bias to be bias B = θ ( ) θ, which implies that θ = θ { θ ( ) θ} = 2 θ θ ( ).
91 44 / 171 Bootstrap bias correction Suppose that θ is biased (although consistent) for θ, and that we would like to obtain a new estimate which is biascorrected up to some order of accuracy. The bias of θ is: IE( θ) θ (systematic error). Ideally, we would like to have: θ bias, but this is not feasible (since the bias depends on θ). Define the biascorrected estimate as: θ = θ bias. We then take bias to be bias B = θ ( ) θ, which implies that θ = θ { θ ( ) θ} = 2 θ θ ( ).
92 44 / 171 Bootstrap bias correction Suppose that θ is biased (although consistent) for θ, and that we would like to obtain a new estimate which is biascorrected up to some order of accuracy. The bias of θ is: IE( θ) θ (systematic error). Ideally, we would like to have: θ bias, but this is not feasible (since the bias depends on θ). Define the biascorrected estimate as: θ = θ bias. We then take bias to be bias B = θ ( ) θ, which implies that θ = θ { θ ( ) θ} = 2 θ θ ( ).
93 44 / 171 Bootstrap bias correction Suppose that θ is biased (although consistent) for θ, and that we would like to obtain a new estimate which is biascorrected up to some order of accuracy. The bias of θ is: IE( θ) θ (systematic error). Ideally, we would like to have: θ bias, but this is not feasible (since the bias depends on θ). Define the biascorrected estimate as: θ = θ bias. We then take bias to be bias B = θ ( ) θ, which implies that θ = θ { θ ( ) θ} = 2 θ θ ( ).
94 45 / 171 We shall call the above bias correction BC1. NOTE: θ ( ) is not itself the bootstrap biascorrected estimate. Let s look into that. (For futher details, see, e.g., MacKinnon & Smith, Journal of Econometrics, 1998). Assuming that IE( θ) exists, write θ = θ 0 + B(θ 0, n) + R(θ 0, n), where B(θ 0, n) = IE( θ) θ 0 (i.e., B(, ) is the bias function) and R(θ 0, n) is defined so that the above equation holds. Assume we know the distribution of Y i up to the unknown parameter θ (so that we can use the parametric bootstrap).
95 45 / 171 We shall call the above bias correction BC1. NOTE: θ ( ) is not itself the bootstrap biascorrected estimate. Let s look into that. (For futher details, see, e.g., MacKinnon & Smith, Journal of Econometrics, 1998). Assuming that IE( θ) exists, write θ = θ 0 + B(θ 0, n) + R(θ 0, n), where B(θ 0, n) = IE( θ) θ 0 (i.e., B(, ) is the bias function) and R(θ 0, n) is defined so that the above equation holds. Assume we know the distribution of Y i up to the unknown parameter θ (so that we can use the parametric bootstrap).
96 45 / 171 We shall call the above bias correction BC1. NOTE: θ ( ) is not itself the bootstrap biascorrected estimate. Let s look into that. (For futher details, see, e.g., MacKinnon & Smith, Journal of Econometrics, 1998). Assuming that IE( θ) exists, write θ = θ 0 + B(θ 0, n) + R(θ 0, n), where B(θ 0, n) = IE( θ) θ 0 (i.e., B(, ) is the bias function) and R(θ 0, n) is defined so that the above equation holds. Assume we know the distribution of Y i up to the unknown parameter θ (so that we can use the parametric bootstrap).
97 45 / 171 We shall call the above bias correction BC1. NOTE: θ ( ) is not itself the bootstrap biascorrected estimate. Let s look into that. (For futher details, see, e.g., MacKinnon & Smith, Journal of Econometrics, 1998). Assuming that IE( θ) exists, write θ = θ 0 + B(θ 0, n) + R(θ 0, n), where B(θ 0, n) = IE( θ) θ 0 (i.e., B(, ) is the bias function) and R(θ 0, n) is defined so that the above equation holds. Assume we know the distribution of Y i up to the unknown parameter θ (so that we can use the parametric bootstrap).
98 46 / 171 Noteworthy If θ is n consistent and asymptotically normal, the bias will be typically be O(n 1 ). (Otherwise, n( θ θ 0 ) would not have mean zero asymptotically.)
99 47 / 171 Suppose that B(θ, n) = B(n) for all θ. That is, suppose the bias function if flat. Here, it does not matter at which value of θ we evaluate the bias function since it is flat. An obvious candidate, however, is θ, the MLE. And what we get here is exactly our bias correction BC1: θ = θ B( θ, n) = θ { θ ( ) θ} = 2 θ θ ( ). NOTE: In many (most?) cases, however, the bias function is not flat.
100 47 / 171 Suppose that B(θ, n) = B(n) for all θ. That is, suppose the bias function if flat. Here, it does not matter at which value of θ we evaluate the bias function since it is flat. An obvious candidate, however, is θ, the MLE. And what we get here is exactly our bias correction BC1: θ = θ B( θ, n) = θ { θ ( ) θ} = 2 θ θ ( ). NOTE: In many (most?) cases, however, the bias function is not flat.
101 47 / 171 Suppose that B(θ, n) = B(n) for all θ. That is, suppose the bias function if flat. Here, it does not matter at which value of θ we evaluate the bias function since it is flat. An obvious candidate, however, is θ, the MLE. And what we get here is exactly our bias correction BC1: θ = θ B( θ, n) = θ { θ ( ) θ} = 2 θ θ ( ). NOTE: In many (most?) cases, however, the bias function is not flat.
102 48 / 171 Suppose now that the bias function is linear in θ, that is, B(θ, n) = α 0 + α 1 θ. ( ) The main idea is to evaluate ( ) at two points and then solve for α 0 and α 1. (Note that this will require two sets of simulations!) Obvious choices for the two points at which we evaluate the DGP are θ and θ. The solution is α 0 = B B B θ θ θ, α 1 = B B θ θ. (NOTE: Here we are using a shorthand notation.) The estimated α s will converge to the true ones as B increases.
103 48 / 171 Suppose now that the bias function is linear in θ, that is, B(θ, n) = α 0 + α 1 θ. ( ) The main idea is to evaluate ( ) at two points and then solve for α 0 and α 1. (Note that this will require two sets of simulations!) Obvious choices for the two points at which we evaluate the DGP are θ and θ. The solution is α 0 = B B B θ θ θ, α 1 = B B θ θ. (NOTE: Here we are using a shorthand notation.) The estimated α s will converge to the true ones as B increases.
104 48 / 171 Suppose now that the bias function is linear in θ, that is, B(θ, n) = α 0 + α 1 θ. ( ) The main idea is to evaluate ( ) at two points and then solve for α 0 and α 1. (Note that this will require two sets of simulations!) Obvious choices for the two points at which we evaluate the DGP are θ and θ. The solution is α 0 = B B B θ θ θ, α 1 = B B θ θ. (NOTE: Here we are using a shorthand notation.) The estimated α s will converge to the true ones as B increases.
105 48 / 171 Suppose now that the bias function is linear in θ, that is, B(θ, n) = α 0 + α 1 θ. ( ) The main idea is to evaluate ( ) at two points and then solve for α 0 and α 1. (Note that this will require two sets of simulations!) Obvious choices for the two points at which we evaluate the DGP are θ and θ. The solution is α 0 = B B B θ θ θ, α 1 = B B θ θ. (NOTE: Here we are using a shorthand notation.) The estimated α s will converge to the true ones as B increases.
106 48 / 171 Suppose now that the bias function is linear in θ, that is, B(θ, n) = α 0 + α 1 θ. ( ) The main idea is to evaluate ( ) at two points and then solve for α 0 and α 1. (Note that this will require two sets of simulations!) Obvious choices for the two points at which we evaluate the DGP are θ and θ. The solution is α 0 = B B B θ θ θ, α 1 = B B θ θ. (NOTE: Here we are using a shorthand notation.) The estimated α s will converge to the true ones as B increases.
107 48 / 171 Suppose now that the bias function is linear in θ, that is, B(θ, n) = α 0 + α 1 θ. ( ) The main idea is to evaluate ( ) at two points and then solve for α 0 and α 1. (Note that this will require two sets of simulations!) Obvious choices for the two points at which we evaluate the DGP are θ and θ. The solution is α 0 = B B B θ θ θ, α 1 = B B θ θ. (NOTE: Here we are using a shorthand notation.) The estimated α s will converge to the true ones as B increases.
108 49 / 171 The biascorrected estimator can then be defined as θ = θ α 0 α 1 θ. (Here we are evaluating the bias function at θ itself.) The solution, therefore, is θ = α 1 ( θ α 0 ). We can call the above bias correction BC2.
109 49 / 171 The biascorrected estimator can then be defined as θ = θ α 0 α 1 θ. (Here we are evaluating the bias function at θ itself.) The solution, therefore, is θ = α 1 ( θ α 0 ). We can call the above bias correction BC2.
110 49 / 171 The biascorrected estimator can then be defined as θ = θ α 0 α 1 θ. (Here we are evaluating the bias function at θ itself.) The solution, therefore, is θ = α 1 ( θ α 0 ). We can call the above bias correction BC2.
111 What if the bias function is nonlinear? In that case, we define a biascorrected estimator as θ = θ B( θ, n). One way of implement this is as follows. Start w/ B obtained as in the BC1, i.e., B = θ ( ) θ. Now, compute (sequentially) θ (j) = (1 λ) θ (j 1) + λ( θ B( θ (j 1), n)), where θ (0) = θ and 0 < λ 1. Stop when θ (j) θ (j 1) < ɛ for a sufficiently small ɛ. Suggestion: Start with λ = 1. If the procedure does not converge, try smaller values of λ. 50 / 171
112 What if the bias function is nonlinear? In that case, we define a biascorrected estimator as θ = θ B( θ, n). One way of implement this is as follows. Start w/ B obtained as in the BC1, i.e., B = θ ( ) θ. Now, compute (sequentially) θ (j) = (1 λ) θ (j 1) + λ( θ B( θ (j 1), n)), where θ (0) = θ and 0 < λ 1. Stop when θ (j) θ (j 1) < ɛ for a sufficiently small ɛ. Suggestion: Start with λ = 1. If the procedure does not converge, try smaller values of λ. 50 / 171
113 What if the bias function is nonlinear? In that case, we define a biascorrected estimator as θ = θ B( θ, n). One way of implement this is as follows. Start w/ B obtained as in the BC1, i.e., B = θ ( ) θ. Now, compute (sequentially) θ (j) = (1 λ) θ (j 1) + λ( θ B( θ (j 1), n)), where θ (0) = θ and 0 < λ 1. Stop when θ (j) θ (j 1) < ɛ for a sufficiently small ɛ. Suggestion: Start with λ = 1. If the procedure does not converge, try smaller values of λ. 50 / 171
114 What if the bias function is nonlinear? In that case, we define a biascorrected estimator as θ = θ B( θ, n). One way of implement this is as follows. Start w/ B obtained as in the BC1, i.e., B = θ ( ) θ. Now, compute (sequentially) θ (j) = (1 λ) θ (j 1) + λ( θ B( θ (j 1), n)), where θ (0) = θ and 0 < λ 1. Stop when θ (j) θ (j 1) < ɛ for a sufficiently small ɛ. Suggestion: Start with λ = 1. If the procedure does not converge, try smaller values of λ. 50 / 171
115 What if the bias function is nonlinear? In that case, we define a biascorrected estimator as θ = θ B( θ, n). One way of implement this is as follows. Start w/ B obtained as in the BC1, i.e., B = θ ( ) θ. Now, compute (sequentially) θ (j) = (1 λ) θ (j 1) + λ( θ B( θ (j 1), n)), where θ (0) = θ and 0 < λ 1. Stop when θ (j) θ (j 1) < ɛ for a sufficiently small ɛ. Suggestion: Start with λ = 1. If the procedure does not converge, try smaller values of λ. 50 / 171
116 What if the bias function is nonlinear? In that case, we define a biascorrected estimator as θ = θ B( θ, n). One way of implement this is as follows. Start w/ B obtained as in the BC1, i.e., B = θ ( ) θ. Now, compute (sequentially) θ (j) = (1 λ) θ (j 1) + λ( θ B( θ (j 1), n)), where θ (0) = θ and 0 < λ 1. Stop when θ (j) θ (j 1) < ɛ for a sufficiently small ɛ. Suggestion: Start with λ = 1. If the procedure does not converge, try smaller values of λ. 50 / 171
117 51 / 171 An alternative bootstrap bias estimate was introduced by Efron (1990). It is carried out nonparametrically and uses an auxiliary (n 1) resampling vector, whose elements are the proportions of observations in the original sample y = (y 1,..., y n ) that were included in the bootstrap sample. Let P = (P1, P 2,..., P n) be the resampling vector. Its jth element (j = 1, 2,..., n), Pj, is defined with respect to a given bootstrap sample y = (y 1,..., y n) as P j = n 1( #{y k = y j} ). It is important to note that the vector P 0 = (1/n, 1/n,..., 1/n) corresponds to the original sample.
118 51 / 171 An alternative bootstrap bias estimate was introduced by Efron (1990). It is carried out nonparametrically and uses an auxiliary (n 1) resampling vector, whose elements are the proportions of observations in the original sample y = (y 1,..., y n ) that were included in the bootstrap sample. Let P = (P1, P 2,..., P n) be the resampling vector. Its jth element (j = 1, 2,..., n), Pj, is defined with respect to a given bootstrap sample y = (y 1,..., y n) as P j = n 1( #{y k = y j} ). It is important to note that the vector P 0 = (1/n, 1/n,..., 1/n) corresponds to the original sample.
119 51 / 171 An alternative bootstrap bias estimate was introduced by Efron (1990). It is carried out nonparametrically and uses an auxiliary (n 1) resampling vector, whose elements are the proportions of observations in the original sample y = (y 1,..., y n ) that were included in the bootstrap sample. Let P = (P1, P 2,..., P n) be the resampling vector. Its jth element (j = 1, 2,..., n), Pj, is defined with respect to a given bootstrap sample y = (y 1,..., y n) as P j = n 1( #{y k = y j} ). It is important to note that the vector P 0 = (1/n, 1/n,..., 1/n) corresponds to the original sample.
120 51 / 171 An alternative bootstrap bias estimate was introduced by Efron (1990). It is carried out nonparametrically and uses an auxiliary (n 1) resampling vector, whose elements are the proportions of observations in the original sample y = (y 1,..., y n ) that were included in the bootstrap sample. Let P = (P1, P 2,..., P n) be the resampling vector. Its jth element (j = 1, 2,..., n), Pj, is defined with respect to a given bootstrap sample y = (y 1,..., y n) as P j = n 1( #{y k = y j} ). It is important to note that the vector P 0 = (1/n, 1/n,..., 1/n) corresponds to the original sample.
121 51 / 171 An alternative bootstrap bias estimate was introduced by Efron (1990). It is carried out nonparametrically and uses an auxiliary (n 1) resampling vector, whose elements are the proportions of observations in the original sample y = (y 1,..., y n ) that were included in the bootstrap sample. Let P = (P1, P 2,..., P n) be the resampling vector. Its jth element (j = 1, 2,..., n), Pj, is defined with respect to a given bootstrap sample y = (y 1,..., y n) as P j = n 1( #{y k = y j} ). It is important to note that the vector P 0 = (1/n, 1/n,..., 1/n) corresponds to the original sample.
122 52 / 171 Also, any bootstrap replicate θ can be defined as a function of the resampling vector. For example, if θ = s(y) = y = n 1 n i=1 y i, then θ = y 1 + y y n n = (np 1 )y (np n)y n n = #{y k = y 1}y #{y k = y n}y n n = P y.
123 52 / 171 Also, any bootstrap replicate θ can be defined as a function of the resampling vector. For example, if θ = s(y) = y = n 1 n i=1 y i, then θ = y 1 + y y n n = (np 1 )y (np n)y n n = #{y k = y 1}y #{y k = y n}y n n = P y.
124 53 / 171 Suppose we can write the estimate of interest, obtained from the original sample y, as G(P 0 ). It is now possible to obtain bootstrap estimates θ b using the resampling vectors P b, b = 1, 2,..., R, as G(P b ). Efron s (1990) bootstrap bias estimate, BˆF(ˆθ, θ), is defined as BˆF(ˆθ, θ) = ˆθ ( ) G(P ( ) ), where P ( ) = 1 R R P b, b=1 which differs from ˆBˆF(ˆθ, θ), since ˆBˆF(ˆθ, θ) = ˆθ ( ) G(P 0 ). Notice that this bias estimate uses an additional information, namely: the proportions of the n observations that were selected each nonparametric resampling.
125 53 / 171 Suppose we can write the estimate of interest, obtained from the original sample y, as G(P 0 ). It is now possible to obtain bootstrap estimates θ b using the resampling vectors P b, b = 1, 2,..., R, as G(P b ). Efron s (1990) bootstrap bias estimate, BˆF(ˆθ, θ), is defined as BˆF(ˆθ, θ) = ˆθ ( ) G(P ( ) ), where P ( ) = 1 R R P b, b=1 which differs from ˆBˆF(ˆθ, θ), since ˆBˆF(ˆθ, θ) = ˆθ ( ) G(P 0 ). Notice that this bias estimate uses an additional information, namely: the proportions of the n observations that were selected each nonparametric resampling.
126 53 / 171 Suppose we can write the estimate of interest, obtained from the original sample y, as G(P 0 ). It is now possible to obtain bootstrap estimates θ b using the resampling vectors P b, b = 1, 2,..., R, as G(P b ). Efron s (1990) bootstrap bias estimate, BˆF(ˆθ, θ), is defined as BˆF(ˆθ, θ) = ˆθ ( ) G(P ( ) ), where P ( ) = 1 R R P b, b=1 which differs from ˆBˆF(ˆθ, θ), since ˆBˆF(ˆθ, θ) = ˆθ ( ) G(P 0 ). Notice that this bias estimate uses an additional information, namely: the proportions of the n observations that were selected each nonparametric resampling.
127 53 / 171 Suppose we can write the estimate of interest, obtained from the original sample y, as G(P 0 ). It is now possible to obtain bootstrap estimates θ b using the resampling vectors P b, b = 1, 2,..., R, as G(P b ). Efron s (1990) bootstrap bias estimate, BˆF(ˆθ, θ), is defined as BˆF(ˆθ, θ) = ˆθ ( ) G(P ( ) ), where P ( ) = 1 R R P b, b=1 which differs from ˆBˆF(ˆθ, θ), since ˆBˆF(ˆθ, θ) = ˆθ ( ) G(P 0 ). Notice that this bias estimate uses an additional information, namely: the proportions of the n observations that were selected each nonparametric resampling.
128 54 / 171 After obtaining an estimate for the bias, it is easy to obtain a biasadjusted estimator: θ = s(y) BˆF(ˆθ, θ) = ˆθ ˆθ ( ) + G(P ( ) ).
129 55 / 171 It is important to note that the bias estimation procedure proposed by Efron (1990) requires the estimator ˆθ to have closed form. However, oftentimes the maximum likelihood estimator of θ, the parameter that indexes the model used to represent the population, does not have a closed form. Rather, it needs to be obtained by numerically maximizing the log likelihood function using a nonlinear optimization algorithm, such as a Newton or quasinewton algorithm. CribariNeto, Frery and Silva (2002) proposed an adaptation of Efron s method that can be used with estimators that cannot be written in closed form.
130 55 / 171 It is important to note that the bias estimation procedure proposed by Efron (1990) requires the estimator ˆθ to have closed form. However, oftentimes the maximum likelihood estimator of θ, the parameter that indexes the model used to represent the population, does not have a closed form. Rather, it needs to be obtained by numerically maximizing the log likelihood function using a nonlinear optimization algorithm, such as a Newton or quasinewton algorithm. CribariNeto, Frery and Silva (2002) proposed an adaptation of Efron s method that can be used with estimators that cannot be written in closed form.
131 55 / 171 It is important to note that the bias estimation procedure proposed by Efron (1990) requires the estimator ˆθ to have closed form. However, oftentimes the maximum likelihood estimator of θ, the parameter that indexes the model used to represent the population, does not have a closed form. Rather, it needs to be obtained by numerically maximizing the log likelihood function using a nonlinear optimization algorithm, such as a Newton or quasinewton algorithm. CribariNeto, Frery and Silva (2002) proposed an adaptation of Efron s method that can be used with estimators that cannot be written in closed form.
132 56 / 171 They use the resampling vector to modify the log likelihood function, and then maximize the modified log likelihood. The main idea is to write the log likelihood function in terms of P 0, replace this vector by P ( ), and then maximize the resulting (modified) log likelihood function. The maximizer of such a function is a biascorrected maximum likelihood estimator.
133 56 / 171 They use the resampling vector to modify the log likelihood function, and then maximize the modified log likelihood. The main idea is to write the log likelihood function in terms of P 0, replace this vector by P ( ), and then maximize the resulting (modified) log likelihood function. The maximizer of such a function is a biascorrected maximum likelihood estimator.
134 56 / 171 They use the resampling vector to modify the log likelihood function, and then maximize the modified log likelihood. The main idea is to write the log likelihood function in terms of P 0, replace this vector by P ( ), and then maximize the resulting (modified) log likelihood function. The maximizer of such a function is a biascorrected maximum likelihood estimator.
Preliminaries The bootstrap Bias reduction Hypothesis tests Regression Confidence intervals Time series Final remark. Bootstrap inference
1 / 172 Bootstrap inference Francisco CribariNeto Departamento de Estatística Universidade Federal de Pernambuco Recife / PE, Brazil email: cribari@gmail.com October 2014 2 / 172 Unpaid advertisement
More informationHeteroskedasticityRobust Inference in Finite Samples
HeteroskedasticityRobust Inference in Finite Samples Jerry Hausman and Christopher Palmer Massachusetts Institute of Technology December 011 Abstract Since the advent of heteroskedasticityrobust standard
More informationBayesian Interpretations of Heteroskedastic Consistent Covariance Estimators Using the Informed Bayesian Bootstrap
Bayesian Interpretations of Heteroskedastic Consistent Covariance Estimators Using the Informed Bayesian Bootstrap Dale J. Poirier University of California, Irvine September 1, 2008 Abstract This paper
More informationPolitical Science 236 Hypothesis Testing: Review and Bootstrapping
Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall 2007 1 Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The
More informationBootstrapBased Improvements for Inference with Clustered Errors
BootstrapBased Improvements for Inference with Clustered Errors Colin Cameron, Jonah Gelbach, Doug Miller U.C.  Davis, U. Maryland, U.C.  Davis May, 2008 May, 2008 1 / 41 1. Introduction OLS regression
More informationNew heteroskedasticityrobust standard errors for the linear regression model
Brazilian Journal of Probability and Statistics 2014, Vol. 28, No. 1, 83 95 DOI: 10.1214/12BJPS196 Brazilian Statistical Association, 2014 New heteroskedasticityrobust standard errors for the linear
More informationUNIVERSITÄT POTSDAM Institut für Mathematik
UNIVERSITÄT POTSDAM Institut für Mathematik Testing the Acceleration Function in Life Time Models Hannelore Liero Matthias Liero Mathematische Statistik und Wahrscheinlichkeitstheorie Universität Potsdam
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationInference via Kernel Smoothing of Bootstrap P Values
Queen s Economics Department Working Paper No. 1054 Inference via Kernel Smoothing of Bootstrap P Values Jeff Racine McMaster University James G. MacKinnon Queen s University Department of Economics Queen
More informationLESLIE GODFREY LIST OF PUBLICATIONS
LESLIE GODFREY LIST OF PUBLICATIONS This list is in two parts. First, there is a set of selected publications for the period 19711996. Second, there are details of more recent outputs. SELECTED PUBLICATIONS,
More informationSequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk
Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy Department of Mathematics Imperial College London a.gandy@imperial.ac.uk user! 2009, Rennes July 810, 2009
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationBiascorrected Estimators of Scalar Skew Normal
Biascorrected Estimators of Scalar Skew Normal Guoyi Zhang and Rong Liu Abstract One problem of skew normal model is the difficulty in estimating the shape parameter, for which the maximum likelihood
More informationOnline Appendix to Correcting Estimation Bias in Dynamic Term Structure Models
Online Appendix to Correcting Estimation Bias in Dynamic Term Structure Models Michael D. Bauer, Glenn D. Rudebusch, Jing Cynthia Wu May 4, 2012 A Bootstrap bias correction The bootstrap has become a common
More informationQuantile regression and heteroskedasticity
Quantile regression and heteroskedasticity José A. F. Machado J.M.C. Santos Silva June 18, 2013 Abstract This note introduces a wrapper for qreg which reports standard errors and t statistics that are
More informationOn the Accuracy of Bootstrap Confidence Intervals for Efficiency Levels in Stochastic Frontier Models with Panel Data
On the Accuracy of Bootstrap Confidence Intervals for Efficiency Levels in Stochastic Frontier Models with Panel Data Myungsup Kim University of North Texas Yangseon Kim EastWest Center Peter Schmidt
More informationTheory and Methods of Statistical Inference. PART I Frequentist likelihood methods
PhD School in Statistics XXV cycle, 2010 Theory and Methods of Statistical Inference PART I Frequentist likelihood methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution
More informationTheory and Methods of Statistical Inference. PART I Frequentist theory and methods
PhD School in Statistics cycle XXVI, 2011 Theory and Methods of Statistical Inference PART I Frequentist theory and methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution
More informationA Simple, Graphical Procedure for Comparing Multiple Treatment Effects
A Simple, Graphical Procedure for Comparing Multiple Treatment Effects Brennan S. Thompson and Matthew D. Webb May 15, 2015 > Abstract In this paper, we utilize a new graphical
More informationStatistics  Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation
Statistics  Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence
More informationON THE NUMBER OF BOOTSTRAP REPETITIONS FOR BC a CONFIDENCE INTERVALS. DONALD W. K. ANDREWS and MOSHE BUCHINSKY COWLES FOUNDATION PAPER NO.
ON THE NUMBER OF BOOTSTRAP REPETITIONS FOR BC a CONFIDENCE INTERVALS BY DONALD W. K. ANDREWS and MOSHE BUCHINSKY COWLES FOUNDATION PAPER NO. 1069 COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY
More informationDay 3B Nonparametrics and Bootstrap
Day 3B Nonparametrics and Bootstrap c A. Colin Cameron Univ. of Calif. Davis Frontiers in Econometrics Bavarian Graduate Program in Economics. Based on A. Colin Cameron and Pravin K. Trivedi (2009,2010),
More informationTheory and Methods of Statistical Inference
PhD School in Statistics cycle XXIX, 2014 Theory and Methods of Statistical Inference Instructors: B. Liseo, L. Pace, A. Salvan (course coordinator), N. Sartori, A. Tancredi, L. Ventura Syllabus Some prerequisites:
More informationTesting Statistical Hypotheses
E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I SmallSample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions
More informationAN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY
Econometrics Working Paper EWP0401 ISSN 14856441 Department of Economics AN EMPIRICAL LIKELIHOOD RATIO TEST FOR NORMALITY Lauren Bin Dong & David E. A. Giles Department of Economics, University of Victoria
More informationDouble Bootstrap Confidence Interval Estimates with Censored and Truncated Data
Journal of Modern Applied Statistical Methods Volume 13 Issue 2 Article 22 112014 Double Bootstrap Confidence Interval Estimates with Censored and Truncated Data Jayanthi Arasan University Putra Malaysia,
More informationBootstrapping heteroskedastic regression models: wild bootstrap vs. pairs bootstrap
Bootstrapping heteroskedastic regression models: wild bootstrap vs. pairs bootstrap Emmanuel Flachaire To cite this version: Emmanuel Flachaire. Bootstrapping heteroskedastic regression models: wild bootstrap
More informationBias evaluation and reduction for samplepath optimization
Bias evaluation and reduction for samplepath optimization 1 1 Department of Computing Science and Operational Research Université de Montréal; CIRRELT Québec, Canada Eigth IMACS Seminar on Monte Carlo
More informationAdvanced Econometrics
Advanced Econometrics Dr. Andrea Beccarini Center for Quantitative Economics Winter 2013/2014 Andrea Beccarini (CQE) Econometrics Winter 2013/2014 1 / 156 General information Aims and prerequisites Objective:
More informationBagging During Markov Chain Monte Carlo for Smoother Predictions
Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods
More informationThe assumptions are needed to give us... valid standard errors valid confidence intervals valid hypothesis tests and pvalues
Statistical Consulting Topics The Bootstrap... The bootstrap is a computerbased method for assigning measures of accuracy to statistical estimates. (Efron and Tibshrani, 1998.) What do we do when our
More informationBootstrap Methods and their Application
Bootstrap Methods and their Application Anthony Davison c 2006 A short course based on the book Bootstrap Methods and their Application, by A C Davison and D V Hinkley c Cambridge University Press, 1997
More informationFULL LIKELIHOOD INFERENCES IN THE COX MODEL
October 20, 2007 FULL LIKELIHOOD INFERENCES IN THE COX MODEL BY JIANJIAN REN 1 AND MAI ZHOU 2 University of Central Florida and University of Kentucky Abstract We use the empirical likelihood approach
More informationA Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,
A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. HeckmanType
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongamro, Namgu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationDiscussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon
Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics
More informationResampling and the Bootstrap
Resampling and the Bootstrap Axel Benner Biostatistics, German Cancer Research Center INF 280, D69120 Heidelberg benner@dkfz.de Resampling and the Bootstrap 2 Topics Estimation and Statistical Testing
More informationA COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES
A COMPARISON OF HETEROSCEDASTICITY ROBUST STANDARD ERRORS AND NONPARAMETRIC GENERALIZED LEAST SQUARES MICHAEL O HARA AND CHRISTOPHER F. PARMETER Abstract. This paper presents a Monte Carlo comparison of
More informationOneSample Numerical Data
OneSample Numerical Data quantiles, boxplot, histogram, bootstrap confidence intervals, goodnessoffit tests University of California, San Diego Instructor: Ery AriasCastro http://math.ucsd.edu/~eariasca/teaching.html
More informationThe Bayesian Approach to Multiequation Econometric Model Estimation
Journal of Statistical and Econometric Methods, vol.3, no.1, 2014, 8596 ISSN: 22410384 (print), 22410376 (online) Scienpress Ltd, 2014 The Bayesian Approach to Multiequation Econometric Model Estimation
More informationBootstrap Confidence Intervals for the Correlation Coefficient
Bootstrap Confidence Intervals for the Correlation Coefficient Homer S. White 12 November, 1993 1 Introduction We will discuss the general principles of the bootstrap, and describe two applications of
More informationA Simulation Comparison Study for Estimating the Process Capability Index C pm with Asymmetric Tolerances
Available online at ijims.ms.tku.edu.tw/list.asp International Journal of Information and Management Sciences 20 (2009), 243253 A Simulation Comparison Study for Estimating the Process Capability Index
More informationBayesian Quadrature: Modelbased Approximate Integration. David Duvenaud University of Cambridge
Bayesian Quadrature: Modelbased Approimate Integration David Duvenaud University of Cambridge The Quadrature Problem ˆ We want to estimate an integral Z = f ()p()d ˆ Most computational problems in inference
More informationDepartment of Criminology
Department of Criminology Working Paper No. 201513.01 Calibrated Percentile Double Bootstrap for Robust Linear Regression Inference Daniel McCarthy University of Pennsylvania Kai Zhang University of North
More informationlarge number of i.i.d. observations from P. For concreteness, suppose
1 Subsampling Suppose X i, i = 1,..., n is an i.i.d. sequence of random variables with distribution P. Let θ(p ) be some realvalued parameter of interest, and let ˆθ n = ˆθ n (X 1,..., X n ) be some estimate
More informationEstimation of StressStrength Reliability Using Record Ranked Set Sampling Scheme from the Exponential Distribution
Filomat 9:5 015, 1149 116 DOI 10.98/FIL1505149S Published by Faculty of Sciences and Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat Estimation of StressStrength eliability
More informationNumerical Analysis for Statisticians
Kenneth Lange Numerical Analysis for Statisticians Springer Contents Preface v 1 Recurrence Relations 1 1.1 Introduction 1 1.2 Binomial CoefRcients 1 1.3 Number of Partitions of a Set 2 1.4 Horner's Method
More informationInversion Base Height. Daggot Pressure Gradient Visibility (miles)
Stanford University June 2, 1998 Bayesian Backtting: 1 Bayesian Backtting Trevor Hastie Stanford University Rob Tibshirani University of Toronto Email: trevor@stat.stanford.edu Ftp: stat.stanford.edu:
More informationUniversity of California San Diego and Stanford University and
First International Workshop on Functional and Operatorial Statistics. Toulouse, June 1921, 2008 Ksample Subsampling Dimitris N. olitis andjoseph.romano University of California San Diego and Stanford
More informationInference Based on the Wild Bootstrap
Inference Based on the Wild Bootstrap James G. MacKinnon Department of Economics Queen s University Kingston, Ontario, Canada K7L 3N6 email: jgm@econ.queensu.ca Ottawa September 14, 2012 The Wild Bootstrap
More informationNONINFORMATIVE NONPARAMETRIC BAYESIAN ESTIMATION OF QUANTILES
NONINFORMATIVE NONPARAMETRIC BAYESIAN ESTIMATION OF QUANTILES Glen Meeden School of Statistics University of Minnesota Minneapolis, MN 55455 Appeared in Statistics & Probability Letters Volume 16 (1993)
More informationBeyond stochastic gradient descent for largescale machine learning
Beyond stochastic gradient descent for largescale machine learning Francis Bach INRIA  Ecole Normale Supérieure, Paris, France Joint work with Eric Moulines  October 2014 Big data revolution? A new
More informationSupplement to QuantileBased Nonparametric Inference for FirstPrice Auctions
Supplement to QuantileBased Nonparametric Inference for FirstPrice Auctions Vadim Marmer University of British Columbia Artyom Shneyerov CIRANO, CIREQ, and Concordia University August 30, 2010 Abstract
More informationEstimation, Inference, and Hypothesis Testing
Chapter 2 Estimation, Inference, and Hypothesis Testing Note: The primary reference for these notes is Ch. 7 and 8 of Casella & Berger 2. This text may be challenging if new to this topic and Ch. 7 of
More informationTesting Statistical Hypotheses
E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........
More informationTolerance limits for a ratio of normal random variables
Tolerance limits for a ratio of normal random variables Lanju Zhang 1, Thomas Mathew 2, Harry Yang 1, K. Krishnamoorthy 3 and Iksung Cho 1 1 Department of Biostatistics MedImmune, Inc. One MedImmune Way,
More informationEstimation of the Conditional Variance in Paired Experiments
Estimation of the Conditional Variance in Paired Experiments Alberto Abadie & Guido W. Imbens Harvard University and BER June 008 Abstract In paired randomized experiments units are grouped in pairs, often
More informationRobust Performance Hypothesis Testing with the Variance. Institute for Empirical Research in Economics University of Zurich
Institute for Empirical Research in Economics University of Zurich Working Paper Series ISSN 14240459 Working Paper No. 516 Robust Performance Hypothesis Testing with the Variance Olivier Ledoit and Michael
More informationInterval Estimation III: Fisher's Information & Bootstrapping
Interval Estimation III: Fisher's Information & Bootstrapping Frequentist Confidence Interval Will consider four approaches to estimating confidence interval Standard Error (+/ 1.96 se) Likelihood Profile
More informationA Bootstrap Test for Causality with Endogenous Lag Length Choice.  theory and application in finance
CESIS Electronic Working Paper Series Paper No. 223 A Bootstrap Test for Causality with Endogenous Lag Length Choice  theory and application in finance R. Scott Hacker and Abdulnasser HatemiJ April 200
More informationA Bootstrap Test for Conditional Symmetry
ANNALS OF ECONOMICS AND FINANCE 6, 51 61 005) A Bootstrap Test for Conditional Symmetry Liangjun Su Guanghua School of Management, Peking University Email: lsu@gsm.pku.edu.cn and Sainan Jin Guanghua School
More informationA multiple testing procedure for input variable selection in neural networks
A multiple testing procedure for input variable selection in neural networks MicheleLaRoccaandCiraPerna Department of Economics and Statistics  University of Salerno Via Ponte Don Melillo, 84084, Fisciano
More informationThe Exact Distribution of the tratio with Robust and Clustered Standard Errors
The Exact Distribution of the tratio with Robust and Clustered Standard Errors by Bruce E. Hansen Department of Economics University of Wisconsin June 2017 Bruce Hansen (University of Wisconsin) Exact
More informationMCMC algorithms for fitting Bayesian models
MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models
More informationBayesian Inference and the Parametric Bootstrap. Bradley Efron Stanford University
Bayesian Inference and the Parametric Bootstrap Bradley Efron Stanford University Importance Sampling for Bayes Posterior Distribution Newton and Raftery (1994 JRSSB) Nonparametric Bootstrap: good choice
More informationarxiv: v2 [stat.co] 23 Nov 2017
Inference in a bimodal BirnbaumSaunders model Rodney V. Fonseca a,, Francisco CribariNeto a arxiv:1609.00339v2 [stat.co] 23 Nov 2017 a Departamento de Estatística, Universidade Federal de Pernambuco,
More informationINFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS
Pak. J. Statist. 2016 Vol. 32(2), 8196 INFERENCE FOR MULTIPLE LINEAR REGRESSION MODEL WITH EXTENDED SKEW NORMAL ERRORS A.A. Alhamide 1, K. Ibrahim 1 M.T. Alodat 2 1 Statistics Program, School of Mathematical
More informationgpriors for Linear Regression
Stat60: Bayesian Modeling and Inference Lecture Date: March 15, 010 gpriors for Linear Regression Lecturer: Michael I. Jordan Scribe: Andrew H. Chan 1 Linear regression and gpriors In the last lecture,
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and NonParametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationLongRun Covariability
LongRun Covariability Ulrich K. Müller and Mark W. Watson Princeton University October 2016 Motivation Study the longrun covariability/relationship between economic variables great ratios, longrun Phillips
More informationBootstrap Testing in Nonlinear Models
!" # $%% %%% Bootstrap Testing in Nonlinear Models GREQAM Centre de la Vieille Charité 2 rue de la Charité 13002 Marseille, France by Russell Davidson russell@ehesscnrsmrsfr and James G MacKinnon Department
More informationSome Monte Carlo Evidence for Adaptive Estimation of UnitTime Varying Heteroscedastic Panel Data Models
Some Monte Carlo Evidence for Adaptive Estimation of UnitTime Varying Heteroscedastic Panel Data Models G. R. Pasha Department of Statistics, Bahauddin Zakariya University Multan, Pakistan Email: drpasha@bzu.edu.pk
More informationModified Simes Critical Values Under Positive Dependence
Modified Simes Critical Values Under Positive Dependence Gengqian Cai, Sanat K. Sarkar Clinical Pharmacology Statistics & Programming, BDS, GlaxoSmithKline Statistics Department, Temple University, Philadelphia
More informationTesting Goodness Of Fit Of The Geometric Distribution: An Application To Human Fecundability Data
Journal of Modern Applied Statistical Methods Volume 4 Issue Article 8 5 Testing Goodness Of Fit Of The Geometric Distribution: An Application To Human Fecundability Data Sudhir R. Paul University of
More informationRejection sampling  Acceptance probability. Review: How to sample from a multivariate normal in R. Review: Rejection sampling. Weighted resampling
Rejection sampling  Acceptance probability Review: How to sample from a multivariate normal in R Goal: Simulate from N d (µ,σ)? Note: For c to be small, g() must be similar to f(). The art of rejection
More informationPostexam 2 practice questions 18.05, Spring 2014
Postexam 2 practice questions 18.05, Spring 2014 Note: This is a set of practice problems for the material that came after exam 2. In preparing for the final you should use the previous review materials,
More informationarxiv: v4 [stat.ml] 26 Jun 2017
A nonparametric sequential test for online randomized experiments arxiv:1612490v4 [stat.ml] 26 Jun 2017 Vineet Abhishek Walmart Labs CA, USA vabhishek@walmartlabs.com Abstract Shie Mannor Technion Haifa,
More informationA Note on Bayesian Inference After Multiple Imputation
A Note on Bayesian Inference After Multiple Imputation Xiang Zhou and Jerome P. Reiter Abstract This article is aimed at practitioners who plan to use Bayesian inference on multiplyimputed datasets in
More informationEconometrics. Week 4. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague
Econometrics Week 4 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 23 Recommended Reading For the today Serial correlation and heteroskedasticity in
More informationBayesian HeteroskedasticityRobust Regression. Richard Startz * revised February Abstract
Bayesian HeteroskedasticityRobust Regression Richard Startz * revised February 2015 Abstract I offer here a method for Bayesian heteroskedasticityrobust regression. The Bayesian version is derived by
More informationMarkovSwitching Models with Endogenous Explanatory Variables. ChangJin Kim 1
MarkovSwitching Models with Endogenous Explanatory Variables by ChangJin Kim 1 Dept. of Economics, Korea University and Dept. of Economics, University of Washington First draft: August, 2002 This version:
More informationAFT Models and Empirical Likelihood
AFT Models and Empirical Likelihood Mai Zhou Department of Statistics, University of Kentucky Collaborators: Gang Li (UCLA); A. Bathke; M. Kim (Kentucky) Accelerated Failure Time (AFT) models: Y = log(t
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationBayesian Inference. Chapter 1. Introduction and basic concepts
Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master
More informationOn robust and efficient estimation of the center of. Symmetry.
On robust and efficient estimation of the center of symmetry Howard D. Bondell Department of Statistics, North Carolina State University Raleigh, NC 276958203, U.S.A (email: bondell@stat.ncsu.edu) Abstract
More informationConfidence Distribution
Confidence Distribution Xie and Singh (2013): Confidence distribution, the frequentist distribution estimator of a parameter: A Review Céline Cunen, 15/09/2014 Outline of Article Introduction The concept
More informationOutline of GLMs. Definitions
Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density
More informationLecture 1: Random number generation, permutation test, and the bootstrap. August 25, 2016
Lecture 1: Random number generation, permutation test, and the bootstrap August 25, 2016 Statistical simulation 1/21 Statistical simulation (Monte Carlo) is an important part of statistical method research.
More informationLecture 1: Supervised Learning
Lecture 1: Supervised Learning Tuo Zhao Schools of ISYE and CSE, Georgia Tech ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine from Portland, Learning Oregon: pervised learning (Supervised)
More informationA COMPARISON OF POISSON AND BINOMIAL EMPIRICAL LIKELIHOOD Mai Zhou and Hui Fang University of Kentucky
A COMPARISON OF POISSON AND BINOMIAL EMPIRICAL LIKELIHOOD Mai Zhou and Hui Fang University of Kentucky Empirical likelihood with right censored data were studied by Thomas and Grunkmier (1975), Li (1995),
More informationWEIGHTED LIKELIHOOD NEGATIVE BINOMIAL REGRESSION
WEIGHTED LIKELIHOOD NEGATIVE BINOMIAL REGRESSION Michael Amiguet 1, Alfio Marazzi 1, Victor Yohai 2 1  University of Lausanne, Institute for Social and Preventive Medicine, Lausanne, Switzerland 2  University
More informationwhere x i and u i are iid N (0; 1) random variates and are mutually independent, ff =0; and fi =1. ff(x i )=fl0 + fl1x i with fl0 =1. We examine the e
Inference on the Quantile Regression Process Electronic Appendix Roger Koenker and Zhijie Xiao 1 Asymptotic Critical Values Like many other KolmogorovSmirnov type tests (see, e.g. Andrews (1993)), the
More informationSequential Bayesian Updating
BS2 Statistical Inference, Lectures 14 and 15, Hilary Term 2009 May 28, 2009 We consider data arriving sequentially X 1,..., X n,... and wish to update inference on an unknown parameter θ online. In a
More informationNonlinear Regression
Nonlinear Regression James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) Nonlinear Regression 1 / 36 Nonlinear Regression 1 Introduction
More informationThe Bootstrap in Econometrics
The Bootstrap in Econometrics by Russell Davidson Department of Economics and CIREQ McGill University Montréal, Québec, Canada H3A 2T7 AMSE and GREQAM Centre de la Vieille Charité 2 rue de la Charité 13236
More informationSequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process
Applied Mathematical Sciences, Vol. 4, 2010, no. 62, 30833093 Sequential Procedure for Testing Hypothesis about Mean of Latent Gaussian Process Julia Bondarenko HelmutSchmidt University Hamburg University
More informationEconomics 536 Lecture 7. Introduction to Specification Testing in Dynamic Econometric Models
University of Illinois Fall 2016 Department of Economics Roger Koenker Economics 536 Lecture 7 Introduction to Specification Testing in Dynamic Econometric Models In this lecture I want to briefly describe
More informationSmooth nonparametric estimation of a quantile function under right censoring using beta kernels
Smooth nonparametric estimation of a quantile function under right censoring using beta kernels Chanseok Park 1 Department of Mathematical Sciences, Clemson University, Clemson, SC 29634 Short Title: Smooth
More informationSmooth simultaneous confidence bands for cumulative distribution functions
Journal of Nonparametric Statistics, 2013 Vol. 25, No. 2, 395 407, http://dx.doi.org/10.1080/10485252.2012.759219 Smooth simultaneous confidence bands for cumulative distribution functions Jiangyan Wang
More informationCOS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION
COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION SEAN GERRISH AND CHONG WANG 1. WAYS OF ORGANIZING MODELS In probabilistic modeling, there are several ways of organizing models:
More information