Lund Institute of Technology Centre for Mathematical Sciences Mathematical Statistics

Size: px

Start display at page:

Download "Lund Institute of Technology Centre for Mathematical Sciences Mathematical Statistics"

Derick Cummings
5 years ago
Views:

1 Lund Institute of Technology Centre for Mathematical Sciences Mathematical Statistics STATISTICAL METHODS FOR SAFETY ANALYSIS FMS065 ÓÑÔÙØ Ö Ü Ö Ì ÓÓØ ØÖ Ô Ð ÓÖ Ø Ñ Ò Ý Ò Ò ÐÝ In this exercise we will focus on two different approaches to study uncertainty of estimates, the bootstrap algorithm and Bayesian analysis. In the first exercise we will analyse times between earthquakes and in the second exercise life times of ball bearings. The earthquake data and ball bearings data are stored in the files ÕÙ Ô ÖºÑ Ø and ÐÐ Ö Ò ºÑ Ø. 1 Preparatory exercises 1. Read the instructions for the computer exercise and chapter and 6 in the book. 2. Write down Bayes formula when updating the prior density to the posterior density.. 2 Earthquake data Time intervals in days between successive serious earthquakes (>7.5 on Richter scale or more than 1000 casualties) worldwide have been recorded. In all 63 serious earthquakes are recorded, i.e. 62 waiting times. This particular set covers the period from the 16th December 1902 to 4th March Our objective is to get an opinion about the probability of a period of more than 1500 days ( 4 years) between serious earthquakes. We will estimate this probability and plot a histogram to describe the uncertainty of the estimation. A confidence interval will also be given. Load the earthquake data into MATLAB. Ñ Ò Ø Ñ ¼ Ò ØÛ Ó ÐÓ ÕÙ Ô Ö Ü ÕÙ Ô Ö Plot the data in a histogram. Û ØÓ Üµ ÜÐ Ð ³ Ý ³µ From previous analysis, we believe that the exponential distribution is a good candidate to model periods between earthquakes (judging from probability papers and χ 2 tests). The exponential distribution depends on the parameter θ, (θ =return period of earthquakes) F X (x;θ) = 1 e x/θ. (1) and we search the probability, p = P(X > 1500) = 1 F X (1500;θ) = e 1500/θ. (2)

2 ii COMPUTER EXERCISE 3, FMS065 First we will analytically write down an estimate of p and (try to) analyse the variability: Consider n random variables X k, k = 1,... n, independent and distributed as given by Equation??. The MLestimate of θ is Θ = 1 n n k=1 X k. Equation?? gives, P = e 1500/Θ. (3) Note that, since the variables X k are random, Θ and P are also random. The uncertainty of the estimator P is measured by its variance. Try to compute V [P ] = This variance is clearly difficult to compute. Instead we will use bootstrap to overcome this problem. 2.1 Point estimate of p To obtain an estimate p of p, first calculate the ML-estimate of θ Ø Ø Ø Ö Ñ Ò Üµ Ô Ø Ö ÜÔ ¹½ ¼¼»Ø Ø Ø Öµ Write down the estimate of p: p = (How accurate is this result?) Another method to estimate the probability is to use the empirical distribution. Plot the empirical distribution and use the zoom button in the figure window to get the value at ÑÔ ØÖ Üµ The empirical estimate can also be obtained by the following line, Ô ÑÔ ½ ¹ Ñ Ü Ò ½µ ½ ¼¼µµ ¾µ Are the estimates Ô Ø Ö and Ô ÑÔ close to each other? Now we have estimated the searched probability. We will next use bootstrap to evaluate the uncertainty of p. 2.2 Estimating the uncertainty of p with bootstrap In the past decades, the use of bootstrap techniques has attracted a lot of interest, from scientists in different fields handling data, as well as researchers in statistical theory. Roughly speaking, bootstrap techniques combine notions from classical inference with computer intensive methods. We will here only point out some of the basic ideas and demonstrate how to use bootstrap to derive the distribution of the estimation error E = p p. Of course we cannot calculate the distribution straight-forwardly since we only have one estimate p and p is unknown. But with bootstrap we will be able to estimate the distribution of the error E. To understand the basic idea of the bootstrap, we first test the basic commands on a short vector. The following three lines creates a bootstrap sample of vector y. Ý ¾ ½ ¾ ½ ½ Ö Ò ÓÑ Ò Ð Ö Ò ½ µ µ Ñ Ý Ö Ò ÓÑ Ò µ

3 COMPUTER EXERCISE 3, FMS065 iii Convince yourself that Ð Ö Ò ½ µ µ produce 6 random numbers from 1 to 6, and that they occur with equal probability (i.e. comparable to throwing a dice 6 times). Thus, one index can occur more than once. Repeat the command Ð Ö Ò ½ µ µ to get new results. The line Ñ Ý Ö Ò ÓÑ Ò µ gives a bootstrap sample of Ý. The vector Ñ contains 6 values, each element in Ñ takes one of the values: ¾ ½ ¾ ½ ½ with equal probability. Now we consider the 62 samples in vector Ü. Recall that we have assumed that they are independent. From the values in Ü we obtain the estimate p of p. Now to see how p varies we randomly pick with replacement 62 numbers from the sample Ü: Ö Ò ÓÑ Ò Ð Ö Ò ¾ ½µ ¾µ Ñ Ü Ö Ò ÓÑ Ò µ Ô ÓÓØ ÜÔ ¹½ ¼¼»Ñ Ò Ñµµ ± ÓÓØ ØÖ Ô ÑÔÐ The value of Ô ÓÓØ is a new estimate of p, estimated from the bootstrap sample Ñ. Recall that the purpose to bootstrap Ü was that we want to obtain the distribution of E = p p. One can prove that the error E = p p is approximately equally distributed with the difference Ô Ø Ö¹Ô ÓÓØ. So if we repeat the lines above to produce several estimates Ô ÓÓØ, then we can make a histogram of the difference Ô Ø Ö¹Ô ÓÓØ, which will describe the variability of the error p p and hence the uncertainty of the estimate p, Æ ½¼¼¼ ÓÖ ½ Æ Ñ µ Ü Ð Ö Ò ¾ ½µ ¾µµ Ò Ô ÓÓØ ÜÔ ¹½ ¼¼º»Ñ Ò Ñµµ This is the crucial step in the algorithm. Explain what the Matlab code does. Then plot the bootstrap estimates, to visualise the variability Û ØÓ Ô ÓÓØ ¾¼ ½ ½µ Ö ÓÒ Estimate the errors and plot a histogram, Ø ÖÖÓÖ Ô Ø Ö ¹ Ô ÓÓØ Û ØÓ Ø ÖÖÓÖ ¾¼ ½ ½µ 2.3 Bootstrap confidence intervals Finally, we will construct a confidence interval of our estimate based on the distribution of the error. Chapter 4.5.1, in the course compendium gives a more thorough explanation on how to construct a confidence interval based on Bootstrap. Here, we will only give the MATLAB-code. From the distribution of the error we obtain 95%-quantiles of the error. Ø ÖÖÓÖ ÓÖØ Ø ÖÖÓÖµ Õ Ø ÖÖÓÖ ÖÓÙÒ Æ ¼º¼¾ Æ ¼º µµ

4 iv COMPUTER EXERCISE 3, FMS065 We then have that 95% of the errors belong to the interval [Õ ½µ, Õ ¾µ]. Since the error E = p p, the confidence interval for p is ÓÒ ÒØ Ô Ø Ö Õ ½µ Ô Ø Ö Õ ¾µ Write down the confidence interval of p: I p = 3 Bayesian analysis of earthquake data Instead of bootstrap techniques we will now analyse the same data set with a Bayesian approach. We study the intensity Λ = 1 Θ instead of the return period Θ. The probability we are interested in is p = P(X > 1500) = exp( Λ 1500). Similarly as in the bootstrap exercise, we want to obtain the distribution of p. In the Bayesian approach, Λ (Capital letter "lambda") is a random variable, therefore we use the capital version of λ in this chapter. (Recall that in classical approach λ is a fixed unknown constant, but an estimator Λ is random.) So in the Bayesian approach, p is a function of a random variable Λ, and thus also a random variable. However to not confuse p with the probability function P we do not use the capital letter. We will first calculate the posterior distribution of Λ and then calculate the posterior distribution of p. We assume as in previous chapter that the time between earthquakes is exponentially distributed. This implies that the number of earthquakes, N, in a time interval [0,t] is Poisson distributed given that we know the value of Λ. The probability of k earthquakes is, P(N = k Λ = λ) = e λt(λt)k. k! The Bayesian technique requires a prior distribution of Λ. Then using the information that we have 63 earthquakes during the time period, we get the posterior distribution. The updating formula is f post Λ prior (λ) = c P(N = 63 Λ = λ)f (λ), (4) where c is a constant (i.e. not depending on λ). Λ 3.1 Choice of prior The first step in the Bayesian analysis is to choose the prior distribution. The prior distribution should reflect our knowledge about Λ prior to the measurement. If we know very little about Λ, we should choose a flat distribution. Chapter in the course book discusses how to choose suitable priors for intensities. Here we will choose an improper prior, f prior Λ (λ) = λ 1. This gives a non-informative prior. Explain why the prior is called non-informative and improper.

5 COMPUTER EXERCISE 3, FMS065 v 3.2 Calculating the posterior density To calculate the posterior density we need to take into account the information of 63 earthquakes during days. More precisely, we want to calculate the posterior distribution using Equation??. Since N is a Poisson variable, we have that P(N = 63 Λ = λ) = e λ 27120(λ 27120)63 63! So the posterior density is f post Λ (λ) = c 1 e λ λ 63 }{{} λ 1 }{{} updating probability prior density = constant e λ λ 63 (5) = c 1 λ 63 1 e λ (6) which is a gamma density but with parameters a = 63 and b = (In MATLAB the parameters are a = 63 and 1/b = 1/27120). Use ÑÔ to plot the posterior density and distribution. Ñ ÙÖ ÜÜ ¼ ¼º¼¼¼¼½ ¼º¼½ ÔÐÓØ ÜÜ ÑÔ ÜÜ ½»¾ ½¾¼µ ³ ³µ For simplicity we can approximate this distribution with a normal density. Using a table of the gamma distribution we get the expectation, m and standard deviation, σ, Ñ»¾ ½¾¼ ± ÜÔ Ø Ø ÓÒ ¾»¾ ½¾¼ ¾ ±Î Ö Ò ÔÔÖÓÜ ÒÓÖÑÔ ÜÜ Ñ ÕÖØ ¾µµ Ñ µ ÓÐ ÓÒ ÔÐÓØ ÜÜ ÔÔÖÓÜ ³Ö³µ Ð Ò ³ Ü Ø³ ³ ÔÔÖÓÜ Ñ Ø ÓÒ³µ From the figure we see that the posterior density of Λ is approximately a Normal variable, which is easier to handle, Λ N(m,σ 2 ) (7) However we are interested in the posterior density of p = exp( Λ 1500). Thus if we take the logarithm we obtain, log(p) = 1500 Λ and since Λ is (approximately) Normal also log(p) is Normal, log(p) N( 1500 m, σ 2 ). (8) Fill in the numerical values of expectation and variance of log(p) E(log(p)) = 1500 m = V(log(p)) = σ 2 = Now we can plot the posterior density and distribution of p, ÐÓ ÔÑ Ò ¹½ ¼¼ Ñ ÐÓ ÔÚ Ö ½ ¼¼ ¾ ¾ ÔÔ ¼ ¼º¼¼¼½ ¼º½¾ Ô ÐÓ ÒÔ ÔÔ ÐÓ ÔÑ Ò ÕÖØ ÐÓ ÔÚ Öµµ

6 vi COMPUTER EXERCISE 3, FMS065 Ô ÐÓ Ò ÔÔ ÐÓ ÔÑ Ò ÕÖØ ÐÓ ÔÚ Öµµ Ù ÔÐÓØ ¾½½µ ÔÐÓØ ÔÔ Ôµ Ø ØÐ ³ÈÓ Ø Ö ÓÖ Ò ØÝ Ó Ô³µ Ù ÔÐÓØ ¾½¾µ ÔÐÓØ ÔÔ Ôµ Ø ØÐ ³ÈÓ Ø Ö ÓÖ ØÖ ÙØ ÓÒ Ó Ô³µ Write down the credibility interval of p with help from the figure you just plotted. [p 0.975, p ] = Compare the credibility interval with the confidence interval in previous chapter. Are they similar? 4 Ball bearings Now we will study measurements of life times of ball bearings. We will assume a parametric model for the distribution and study the uncertainty of an estimate of the quantile x 0.9. Load the data set into MATLAB. ÐÓ ÐÐ Ö Ò Ü ÐÐ The unit is million cycles. By experience we know that the Weibull distribution often is a suitable model for this kind of data. Plot the data on a Weibull-paper to graphically check the assumption. ÛÛ ÔÐÓØ Üµ Is it a reasonable assumption? The Weibull distribution has two parameters, a and c, F X (x) = 1 e ( x a) c (9) Now an expert claims that only 10% has a lifetime less than 40 million cycles. We will now investigate this statement with bootstrap. 5 Ball bearings, a bootstrap study We will now analyse the quantile of interest, x 0.9. More precisely, we will estimate this quantile and analyse the variability of the quantile by means of bootstrap. First we estimate the parameters a and c in the Weibull distribution and plot the empirical distribution. Ô Ö ÛÛ Ø Üµ Ô Ö ½µ Ô Ö ¾µ Get the the empirical estimate of x 0.9 from the empirical distribution function, the blue line in the figure. Use the zoom button if necessary x emp 0.9 =

7 COMPUTER EXERCISE 3, FMS065 vii It is also possible to get an estimate of the quantile using the ML estimates, Ü¼ Ø Ö ¹ÐÓ ¼º µµ ½»µ x 0.9 = To get a notion about the variability of these estimates we bootstrap the data set to get more estimates of x 0.9. Æ ½¼¼¼ ÓÖ ½ Æ Ò Ü Ü Ð Ö Ò ¾¾ ½µ ¾¾µµ Ô Ö ½ ¾µ ÛÛ Ø Ü µ Ü¼ µ Ô Ö ½µ ¹ÐÓ ¼º µµ ½»Ô Ö ¾µµ Then plot a histogram of the obtained estimates of the quantile. Û ØÓ Ü¼ ½µ Does this confirm the experts statement or not?

Estimation of Quantiles

9 Estimation of Quantiles The notion of quantiles was introduced in Section 3.2: recall that a quantile x α for an r.v. X is a constant such that P(X x α )=1 α. (9.1) In this chapter we examine quantiles