Assignment # 1. STA 355 (K. Knight) February 8, (a) by the Central Limit Theorem. n(g( Xn ) g(µ)) by the delta method ( ) (b) i.

Size: px

Start display at page:

Download "Assignment # 1. STA 355 (K. Knight) February 8, (a) by the Central Limit Theorem. n(g( Xn ) g(µ)) by the delta method ( ) (b) i."

Clemence Berry
5 years ago
Views:

1 STA 355 (K. Knight) February 8, 2014 Assignment # 1 Saša Milić (a) n( n ) d N (0, σ 2 ()) by the Central Limit Theorem n(g( n ) g()) d g (θ)n (0, σ 2 ()) by the delta method σ 2 ()g (θ) 2 1 g (θ)y N (0, g (θ) 2 σ 2 ()), where Y N (0, σ 2 ()) g () ± 1 σ() (b) i. ii. ( ) 1 1/2 ( ) 1 1/2 g () ± σ 2 ± () g() ± 1/2 d ±2 1/2 + c where c is some constant. ( ) 1 1/2 ( ) 1 1/2 g () ± σ 2 ± () 2 ± 1 1 g() ± d ± ln + c where c is some constant. 2. (a) θ(f ) E F (F ) ln (F ) [ ] ( [ ]) E F ln E F (F ) (F ) E F () (F ) ln ( ) EF () (F ) by Jensen s inequality, g(x) x ln(x) is convex. ln (1) Since (F ) E F (). 0 1

2 (b) Let c be the income of everyone in the population. Then let be a constant random variable with the following probability distribution: 1 x c f(x) 0 x c Now we calculate θ(f ). θ(f ) E F (F ) ln (F ) c c (F ) ln f ( c) since we have f ( c) 0 (F ) [ c ( c c c)] ln (F ) E F () c ln (1) 0 (c) We have the following probability distribution for (the income of a member of the population): ɛ x > a f(x) 1 ɛ x 0 We first calculate (F ). (F ) E F () 0 a 0 a a xf(x) dx xf(x) dx + a xf(x) dx xf(x) dx Since x [0, a] x 0. xɛ dx So, (F ) 0 as ɛ 0. 2

3 We now look at how θ(f ) behaves as ɛ 0. θ(f ) E F (F ) ln (F ) 1 (F ) E F ln (F ) 1 (F ) E F [ ln () ln ((F ))] as ɛ 0 As ɛ 0, 1, and ln((f )). (F ) (d) Let the new distribution (once everyone in the population has had their income multiplied by k) be G kf. θ(g) E G [ Y E F (Y ) ln ( Y )] E F (Y ) [ ( )] k k E F E F (k) ln E F (k) [ ( )] k k E F ke F () ln ke F () E F E F () ln E F () θ(f ) (e) Let (F ) (G). (1 ɛ)θ(f ) + ɛθ(g) (1 ɛ)e F ln ( (1 ɛ) ln ( (1 ɛ) ln )] + ɛe G ln ( )] f(x) dx + ɛ ln ( )] f(x) + ɛ ln ln ((1 ɛ)f(x) + ɛg(x)) dx E (1 ɛ)f +ɛg ln ( )] ( )] g(x) dx ( )] ) g(x) dx θ ((1 ɛ)f + ɛg) 3

4 3. (a) Approximately n data points are expected to be outliers when n data points are drawn from a standard normal distribution. > iqr qnorm(.75) - qnorm(.25) > pnorm(qnorm(.25) * iqr) pnorm(qnorm(.75) * iqr) [1] (b) We solve analytically. 0 x < 0 (1+x) x 0 F (x) 1 (1+x) < x > x We have F 1 (1/4) 1/ , F 1 (3/4) 1 1/ , and IQR 2( 1/2 1) And so the probability of an outlier is F ( ) + 1 F ( ) F ( ) + 1 F (1.1715) We expect no outliers for a triangular distribution. (c) Approximately n, n, and n data points are expected to be outliers when n data points are drawn from t distributions with 2, 5 and 10 degrees of freedom, respectively. > v c(2,5,10) > iqr qt(.75,v) - qt(.25,v) > pt(qt(.25,v) - 1.5*iqr, v) pt(qt(.75,v) + 1.5*iqr, v) [1] (d) As the tails of the distributions in (a) (c) become heavier, the proportion of outliers increases, which should match our intuition. 4. (a) Figure 1 shows a histogram of our data. The data appears to be bimodal. The data does not appear to come from a single normal distribution, although it seems it could potentially come from a mixture of two normal distributions. 4

5 Figure 1: A histogram of 272 eruptions of the Old Faithful geyser in Yellowstone National Park. (b) As bandwidth increases, the density plot becomes increasingly smoother (see Figure 2). In particular, the curves with bandwidths 0.1 and 1.7 are undersmoothed and oversmoothed, respectively. (c) We now assume the data comes from a mixture of two normal distributions. Although not a very sophisticated method, we can look at the density plots from before (in part (b)), and use them to make good guesses as to the parameters of the mixture distribution. In particular, we observe that the two mounds of the distribution are centred at approximately 2 and 4.5, and both have a width of about 2, corresponding to a standard deviation of 0.5. It also appears the data is about twice as likely to lie somewhere in the rightmost mound, corresponding to a θ value of This initial guess corresponds to values 1 2, 2 4.5, σ 1 0.5, σ 2 0.5, θ 0.67; the resulting density for this set of parameter estimates is shown in Figure 3a. After adjusting these parameters further, we have the density in Figure 3b, which more correctly matches the kernel density estimates obtained before. 5

6 N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth N 272 Bandwidth 1.7 Figure 2: Estimates of the distribution of the geyser data using the R density function with varying bandwidths. 6

7 density.default(x dat) N 272 Bandwidth (a) 1 2, 2 4.5, σ 1 0.5, σ 2 0.5, θ 0.67 density.default(x dat) N 272 Bandwidth (b) 1 1.9, 2 4.4, σ 1 0.4, σ 2 0.5, θ 0.64 Figure 3: Estimating parameters for a mixture of two normal distributions. The kernel density estimation (with the default bandwidth in R) is the dotted line. 7

Curve Fitting Re-visited, Bishop1.2.5

Curve Fitting Re-visited, Bishop1.2.5 Maximum Likelihood Bishop 1.2.5 Model Likelihood differentiation p(t x, w, β) = Maximum Likelihood N N ( t n y(x n, w), β 1). (1.61) n=1 As we did in the case of the