Location Scale Distributions

Transcription

1 Location Scale Distriutions Linear Estimation and Proaility Plotting Using MATLAB Horst Rinne

2 Copyright: Prof. em. Dr. Horst Rinne Department of Economics and Management Science Justus Lieig University, Giessen, Germany

3 Contents Preface List of Figures List of Tales VII IX XII 1 The family of location scale distriutions Properties of location scale distriutions Genuine location scale distriutions A short listing Distriutions transformale to location scale type Order statistics Distriutional concepts Moments of order statistics Definitions and asic formulas Identities, recurrence relations and approximations Functions of order statistics Statistical graphics Some historical remarks The role of graphical methods in statistics Graphical versus numerical techniques Manipulation with graphs and graphical perception Graphical displays in statistics Distriution assessment y graphs PP plots and QQ plots Proaility paper and plotting positions Hazard plot TTT plot Linear estimation Theory and methods Types of sampling data

4 IV Contents 4.2 Estimators ased on moments of order statistics GLS estimators GLS for a general location scale distriution GLS for a symmetric location scale distriution GLS and censored samples Approximations to GLS B approximated y a diagonal matrix B approximated y an identity matrix BLOM s estimator DOWNTON s estimator Approximations to GLS with approximated moments of order statistics Estimators ased on empirical percentiles Grouped data Randomly and multiply censored data Goodness of fit Proaility plotting and linear estimation Applications What will e given for each distriution? The case of genuine location scale distriutions Arc sine distriution X AS(a, CAUCHY distriution X CA(a, Cosine distriutions Ordinary cosine distriution X COO(a, Raised cosine distriution X COR(a, Exponential distriution X EX(a, Extreme value distriutions of type I Maximum distriution (GUMBEL distriution X EMX1(a, Minimum distriution (Log WEIBULL X EMN1(a, Half distriutions Half CAUCHY distriution X HC(a,

5 Contents V Half logistic distriution X HL(a, Half normal distriution X HN(a, Hyperolic secant distriution X HS(a, LAPLACE distriution X LA(a, Logistic distriution X LO(a, MAXWELL BOLTZMANN distriution X MB(a, Normal distriution X NO(a, Paraolic distriutions of order U shaped paraolic distriution X PAU(a, Inverted U shaped paraolic distriution X PAI(a, RAYLEIGH distriution X RA(a, Reflected exponential distriution X RE(a, Semi elliptical distriution X SE(a, TEISSIER distriution X T E(a, Triangular distriutions Symmetric version X T S(a, Right angled and negatively skew version X T N(a, Right angled and positively skew version X T P (a, Uniform or rectangular distriution X UN(a, U shaped eta distriution X UB(a, V shaped distriution X VS(a, The case of ln transformale distriutions Estimation of the shift parameter Extreme value distriutions of types II and III Type II maximum or inverse WEIBULL distriution X EMX2(a,, c Type III maximum or reflected WEIBULL distriution X EMX3(a,, c Type II minimum or FRÉCHET distriution X EMN2(a,, c

6 VI Contents Type III minimum or WEIBULL distriution X EMN3(a,, c Lognormal distriutions Lognormal distriution with lower threshold X LNL(a, ã, Lognormal distriution with upper threshold X LNU(a, ã, PARETO distriution X PA(a,, c Power function distriution X P O(a,, c The program LEPP Structure of LEPP Input to LEPP Directions for using LEPP Mathematical and statistical notations 237 Biliography 245 Author Index 255 Suject Index 257

7 Preface Preface VII Statistical distriutions can e grouped into families or systems. Such groupings are descried in JOHNSON/KOTZ/KEMP (1992, Chapter 2, JOHNSON/KOTZ/BALAKRISHNAN (1994, Chapter 12 or PATEL(KAPADIA/OWEN (1976, Chapter 4. The most popular families are those of PEARSON, JOHNSON and BURR, the exponential, the stale and the infinitely divisile distriutions or those with a monotone likelihood ratio or with a monotone failure rate. All these categories have attracted the attention of statisticians and they are fully discussed in the statistical literature. But there is one family, the location scale family, which hitherto has not een discussed in greater detail. To my knowledge this ook is the first comprehensive monograph on one dimensional continuous location scale distriutions and it is organized as follows. Chapter 1 goes into the details of location scale distriutions and gives their properties along with a short list of those distriutions which are genuinely location scale and which after a suitale transformation of its variale ecome memer of this class. We will only consider the ln transformation. Location scale distriutions easily lend themselves to an assessment y graphical methods. On a suitaly chosen proaility paper the cumulative distriution function of the universe gives a straight line and the cumulative distriution of a sample only deviates y chance from a straight line. Thus we can realize an informal goodness of fit test. When we fit the straight line free hand or y eye we may read off the location and scale parameters as percentiles. Another and ojective method is to find the straight line on proaility paper y a least squares technique. Then, the estimates of the location and scale parameters will e the parameters of that straight line. Because proaility plotting heavily relies on ordered oservations Chapter 2 gives as a prerequisite a detailed representation of the theory of order statistics. Proaility plotting is a graphical assessment of statistical distriutions. To see how this kind of graphics fits into the framework of statistical graphics we have written Chapter 3. A first core chapter is Chapter 4. It presents the theory and the methods of linear estimating the location and scale parameters. The methods to e implemented depend on the type of sample, i.e. grouped or non grouped, censored or uncensored, the type of censoring and also whether the moments of the order statistics are easily calculale or are readily availale in taulated form or not. In the latter case we will give various approximations to the optimal method of general least squares. Applications of the exact or approximate linear estimation procedures to a great numer of location scale distriutions will e presented in Chapter 5, which is central to this ook. For each of 35 distriutions we give a warrant of arrest enumerating the characteristics, the underlying stochastic model and the fields of application together with the pertinent proaility paper and the estimators of the location parameter and the scale parameter. Distriutions which have to e transformed to location scale type sometimes have a third parameter which has to e pre estimated efore applying proaility plotting and the linear estimation procedure. We will show how to estimate this third parameter.

8 VIII Preface The calculations and graphics of Chapter 5 have een done using MATLAB, 1 Version 7.4 (R2007a. The accompanying CD contains the MATLAB script M file LEPP and all the function files to e used y the reader when he wants to do inference on location scale distriutions. Hints how to handle the menu driven program LEPP and how to organize the data input will e given in Chapter 6 as well as in the comments in the files on the CD. Dr. HORST RINNE, Professor Emeritus of Statistics and Econometrics Department of Economics and Management Science Justus Lieig University, Giessen, Germany 1 MATLAB R is a registered trade mark of The MathWorks, Inc.

9 List of Figures 2/1 Structure of the variance covariance matrix of order statistics from a distriution symmetric around zero /1 Explanation of the QQ plot and the PP plot /2 PP plot comparing two normal distriutions: X NO(0, 1, Y N O( 1, /3 Normal and CAUCHY densities and the corresponding PP plot /4 Theoretical and empirical QQ plots /5 Exponential and PARETO proaility papers /6 Hazard paper for the maximum extreme value distriution of type I /7 TTT plots for several WEIBULL samples of size n = /1 Types of sampling data /2 Proaility plot and estimated regression line (uncensored sample of size n = 10 from an exponential distriution /3 Proaility plot and estimated regression line (uncensored sample of size n = 6 from a normal distriution /4 Proaility plot and estimated regression line n = 50, grouped, from an exponential distriution /5 Proaility plot and estimated regression line n = 20 randomly censored from an exponential distriution /1 Several functions for the reduced arc sine distriution /2 Arc sine proaility paper with data and OLS line (left and GLS line (right109 5/3 Several functions for the reduced CAUCHY distriution /4 CAUCHY proaility paper with data and regression line /5 Cosine and sine functions /6 Several functions for the reduced ordinary cosine distriution /7 Ordinary cosine proaility paper with data and regression line /8 Several functions for the reduced raised cosine distriution /9 Raised cosine proaility paper with data and regression line /10 Several functions for the reduced exponential distriution /11 Exponential proaility paper with data and regression line /12 Densities of the reduced extreme value distriutions of type I for the maximum and minimum /13 Several functions of the reduced extreme value distriution of type I for the maximum /14 Type I maximum extreme value proaility paper with data and regression line

10 X List of Figures 5/15 Several functions for the reduced extreme value distriution of type I for the minimum /16 Type I minimum extreme value proaility paper with data and regression line /17 Several functions for the reduced half CAUCHY distriution /18 Half CAUCHY proaility paper with data and regression line /19 Several functions for the reduced half logistic distriution /20 Half logistic proaility paper with data and regression line /21 Several functions for the reduced half normal distriution /22 Half normal proaility paper with data and regression line /23 Several functions for the reduced hyperolic secant distriution /24 Hyperolic secant proaility paper with data and regression line /25 Several functions for the reduced LAPLACE distriution /26 LAPLACE proaility paper with data and regression line /27 Several functions for the reduced logistic distriution /28 Logistic proaility paper with data and regression line /29 Several functions for the reduced MAXWELL BOLTZMANN distriution /30 MAXWELL BOLTZMANN proaility paper with data and regression line /31 Several functions for the reduced (standardized normal distriution /32 Normal proaility paper with data and regression line /33 Several functions for the reduced U shaped paraolic distriution /34 U shaped paraolic proaility paper with data and regression line /35 Several functions for the reduced inverted U shaped paraolic distriution 170 5/36 Inverted U shaped paraolic proaility paper with data and regression line 171 5/37 Several functions for the reduced RAYLEIGH distriution /38 RAYLEIGH proaility paper with data and regression line /39 Several functions for the reduced reflected exponential distriution /40 Reflected exponential proaility paper with data and regression line /41 Densities of semi elliptical distriutions /42 Several functions for the reduced semi elliptical distriution /43 Semi elliptical proaility paper with data and regression line /44 Several functions for the reduced TEISSIER distriution /45 TEISSIER proaility paper with data and regression line /46 Densities of several triangular distriutions /47 Several functions for the reduced symmetric triangular distriution /48 Symmetric triangular proaility paper with data and regression line /49 Several functions for the reduced right angled and negatively skew triangular distriution /50 Right angled, negatively skew triangular proaility paper with data and regression line

11 List of Figures XI 5/51 Several functions for the reduced right angled and positively skew triangular distriution /52 Right angled, positively skew triangular proaility paper with data and regression line /53 Several functions for the reduced uniform or rectangular distriution /54 Uniform or rectangular proaility paper with data and regression line /55 Density functions of several U shaped reduced eta distriutions /56 Several functions for the reduced U shaped eta distriution /57 U shaped eta proaility paper with data and regression line /58 Several functions for the reduced V shaped distriution /59 V shaped proaility paper with data and regression line /60 Several functions for the reduced type II maximum extreme value distriution208 5/61 Type II maximum extreme value proaility paper with data and regression line /62 Several functions for the reduced type III maximum extreme value distriution210 5/63 Type III maximum extreme value proaility paper with data and regression line /64 Several functions for the reduced type II minimum extreme value distriution212 5/65 Type II minimum extreme value proaility paper with data and regression line /66 Several functions for the reduced type III minimum extreme value or WEIBULL distriution /67 Type III minimum extreme value or WEIBULL proaility paper with data and regression line /68 Several functions for the reduced lognormal distriution with lower threshold221 5/69 Lognormal (lower threshold proaility paper with data and regression line 223 5/70 Several functions for the reduced lognormal distriution with upper threshold224 5/71 Lognormal (upper threshold proaility paper with data and regression line 225 5/72 Several functions for the reduced PARETO distriution /73 PARETO proaility paper with data and regression line /74 Several functions for the reduced power function distriution /75 Power function proaility paper with data and regression line

12 List of Tales 1/1 Relations etween functions and parameters of the reduced and the general variates of a continuous location scale distriution /2 Transformation of extreme value distriutions /1 Exact and approximate means, variances and covariances of reduced exponential order statistics when n = /1 Plotting positions /1 Elements of the unscaled variance covariance matrix ( A Ω A 1 for different modes of type II censoring (exponential distriution, n =

13 1 The family of location scale distriutions Statistical distriutions can e grouped into families or systems such that all memers of a family share the same properties and/or are constructed according to the same principles and/or possess the same structure. For example, we have the PEARSON system, the JOHNSON system and the BURR system, the exponential family, the family of stale distriutions and especially the location scale family. The latter family is a good candidate for applying graphical procedures and linear estimation when the variate is continuous. 1.1 Properties of location scale distriutions A random variale X is said to elong to the location scale family when its cumulative distriution (= CDF F X (x a, := Pr(X x a, (1.1a is a function only of (x a/: ( x a F X (x a, = F ; a R, > 0; (1.1 where F ( is a distriution having no other parameters. Different F ( s correspond to different memers of the family. The two dimensional parameter (a, is called a location scale parameter, a eing the location parameter and eing the scale parameter. For fixed = 1 we have a sufamily which is a location family with parameter a, and for fixed a = 0 we have a scale family with parameter. The variale Y := X a (1.1c is called the reduced variale, 1 y eing a realization of Y. The reduced variale Y has a = 0 and = 1, and we will write the reduced CDF as ( x a F Y (y := F. (1.1d 1 Some authors call it the standardized variale. We will refrain from using this name ecause, conventionally, a standardized variale is defined as Z = [ X E(X ]/ Var(X and thus has mean E(Z = 0 and variance Var(Z = 1. The normal distriution, which is a memer of the location scale family, is the only distriution with E(X = a and Var(X = 2. So, in this case, reducing and standardizing are the same.

14 2 1 The family of location scale distriutions If the distriution of X is asolutely continuous with the density function (= DF f X (x a, = df X(x a, d x then (a, is a location scale parameter for the distriution of X if (and only if f X (x a, = 1 ( x a f Y (1.2a (1.2 for some density f Y (, called reduced DF. The location parameter a R is responsile for the distriution s position on the ascissa. An enlargement (reduction of a causes a movement of the distriution to the right (left. The location parameter is either a measure of central tendency of a distriution, e.g.: a is the mean, median and mode for a symmetric and unimodal distriution as for example with the normal distriution having DF f X (x a, = 1 } { 2 π exp (x a2, x R, a R, > 0. ( a is the median and mode for a symmetric distriution as for example with the CAUCHY distriution having DF { [ ( ]} 2 1 x a f X (x a, = π 1 +, x R, a R, > 0. (1.4 a is the mode for an asymmetric and unimodal distriution as for example with the extreme value distriution of type I for the maximum having DF f X (x a, = 1 { exp x a ( exp x a }, x R, a R, > 0. (1.5 or it is the threshold parameter of a distriution, e.g. a is the lower threshold of the exponential distriution having DF f X (x a, = 1 ( exp x a, x a, a R, > 0. (1.6 a is the upper threshold of the reflected exponential distriution having DF f X (x a, = 1 ( exp a x, x a, a R, > 0. (1.7 The parameter, > 0, is the scale parameter. It is responsile for the dispersion or variation of the variate X. Increasing (decreasing results in an enlargement (reduction of the spread and a corresponding reduction (enlargement of the density. may e

15 1.1 Properties of location scale distriutions 3 the standard deviation of X as for example with the normal distriution, the length of the support 2 of X as for example with the uniform distriution having DF f X (x a, = 1, a x a +, a R, > 0, (1.8 the length of a central (1 α interval 3 for X as for example with the extreme value distriution of type I for the maximum, see (1.5, x x , α (1.9 All distriutions in a given family have the same shape, i.e. the same skewness and the same kurtosis. When the reduced variale Y has mean µ Y = E(Y and standard deviation σ Y = Var(Y then, in the general case, the mean of X is E(X = a + µ Y, and the standard deviation of X is Var(X = σy. (1.10a (1.10 For µ Y = 0 and σ Y = 1 we have the goal expectation a and the goal standard deviation, ut this is not necessarily the case. It is possile, for example, that µ Y and σ Y may not exist, as in the case of the CAUCHY distriution. We have a lot of functions and parameters descriing and measuring certain features of a variate. When we know such a function or parameter for the reduced variate Y, the corresponding function and parameter for the general variale X = a + Y follow in an easy way as is depicted in Ta. 1/1. Tale 1/1: Relations etween functions and parameters of the reduced and the general variates of a continuous location scale distriution Name Definition for Y Relation for X = a + Y density function (DF f Y (y := df Y (y f X (x = 1 ( x a d y f Y ( y x a cumulative distriution function (CDF F Y (y := f Y (u du F X (x = F Y ( x a reliaility function R Y (y := 1 F Y (y R X (x = 1 R Y (CCDF hazard function (HF h Y (y := f Y (y h X (x = 1 ( x a R Y (y h Y 2 For some symmetric location scale distriutions it is more convenient to have a parametrization which results in a length of 2 or a multiple of 2. 3 Such an interval excludes the α/2 smallest and the α/2 largest realizations of X, where α < 1.

16 4 1 The family of location scale distriutions Name Definition for Y Relation for X = a + Y ( y x a cumulative hazard H Y (y := h Y (u du H X (x = H Y function (CHF percentile, generally: 0 P 1 percentile distance, 0 P 1 <P 2 1 y P := F 1 Y (P x P = a + y P PD Y (P 2 P 1 := y P2 y P1 PD X (P 2 P 1 = PD Y (P 2 P 1 mode y M such that f Y (y M = max y f Y (y x M = a + y M crude moments generating function crude moments, r N 0 M Y (t := E ( e t Y M X (t = exp(a t M Y ( t µ ( r(y := E Y r = dr M Y (t d t r µ r(x = r t=0 j=0 ( r j µ r j (Y r j a j mean µ Y := µ 1 (Y := E(Y µ X := µ 1 (X = a + µ Y central moments generating function central moments [ ] Z Y (t := E e t (Y µ Y µ r (Y :=E [ (Y µ Y r] = dr Z Y (t d t r Z X (t = Z Y ( t µ r (X = r µ r (Y t=0 variance σ 2 Y := Var(Y := E[ (Y µ Y 2] σ 2 X := Var(X = 2 σ 2 Y standard deviation σ Y := Var(Y σ X = σ Y index of skewness α 3 (Y := µ 3(Y [ µ2 (Y ] 3/2 α 3 (X = α 3 (Y index of kurtosis α 4 (Y := µ 4(Y [ µ2 (Y ] 2 α 4 (X = α 4 (Y cumulants generating function cumulants characteristic function K Y (t := ln M Y (t K X (t = a t + K Y ( t κ r (Y := dk Y (t dt r κ 1 (X = a + κ 1 (Y t=0 κ r (X = r κ r (Y, r = 2, 3,... C Y (t := E ( e i t Y, i := 1 C X (t = exp (i t a C Y ( t entropy I(Y := E { ld [ f Y (y ]} I(X = ld( + I(Y LAPLACE transform L Y (t := E ( e t Y L X (t = exp ( a t L Y ( t ld(. is the inary logarithm (logarithm with ase 2. Some authors give the entropy in terms of the natural logarithm. Binary and natural logarithms are related as ld(x = ln(x / ln ln(x.

17 1.2 Genuine location scale distriutions A short listing Genuine location scale distriutions A short listing 4 In this section we list in alphaetic order those continuous distriutions which are directly of location scale type. There also exist distriutions which after suitale transformation are of location scale type. They will e presented in Sect The following listing only gives the DF of the non reduced variate X. A complete description of each distriution including all the parameters and functions of Ta. 1/1 will e given in Chapter 5 where we present the accompanying proaility paper together with some auxiliary functions which are useful for the linear estimation procedure of the location scale parameter. Arc sine distriution f(x a, = = 1 π 2 (x a 2 1 (, a x a +, a R, > 0. (1.11a 2 x a π 1 The name of this distriution is given y the fact that its CDF can e expressed among others y the arc sine function arcsin(x = sin 1 (x: ( x a F (x a, = 1 arcsin 2 +. (1.11 π Beta distriution (Special cases The eta distriution in its general form has DF f(x a,, c, d = ( x a c 1 ( 1 x a B(c, d = (x ac 1 (a + x d 1 c+d 1 B(c, d d 1 a x a +, a R, > 0, c > 0, d > 0, (1.12a with the complete eta function 1 B(c, d := u c 1 (1 u d 1 du = 0 Γ(c Γ(d Γ(c + d, ( Suggested reading for this and the following section: JOHNSON/KOTZ/BALAKRISHNAN (1994, 1995, PATEL/KAPADIA/OWEN (1976.

18 6 1 The family of location scale distriutions is not of location scale type ecause it depends on two extra parameters c and d which are responsile for the shape of the DF. When these parameters are given special values we will arrive at a location scale distriution, e.g. c = d = 1 gives the uniform distriution or rectangular distriution, see (1.33. c = 1, d = 2 gives the positively skew right angled triangular distriution, see (1.32c. c = 2, d = 1 gives the negatively skew right angled triangular distriution, see (1.32d. c = d = 0.5 gives an U shaped distriution, see (5.52a. c = 1 and d = 3, 4,... gives a power function distriution of order d 1. d = 1 and c = 3, 4,... gives a power function distriution of order c 1, see (1.27a. CAUCHY distriution The CAUCHY distriution has DF { [ ( ]} 2 1 x a f(x a, = π 1 +, x R, a R, > 0. (1.13 Chi distriution (Special cases The χ distriution with DF f(x a,, ν= 1 2 (ν/2 1 Γ(ν/2 is not of location scale type, ut for given ν it is. We get the half normal distriution for ν = 1, see (1.20, the RAYLEIGH distriution for ν = 2, see (1.28, ( ν 1 x a exp{ 1 ( } 2 x a x R, a R, 2 > 0, ν N, (1.14 the MAXWELL BOLTZMANN distriution for ν = 3, see (1.24. Cosine distriution The cosine distriution has DF f(x a, = 1 ( x a 2 cos, a π 2 x a + π, a R, > 0. (1.15a 2

19 1.2 Genuine location scale distriutions A short listing 7 ( Because cos u = sin 1 + π we can write the cosine distriution as sine distriution: 2 f(x a, = 1 ( π 2 sin 2 + x a, a π 2 x a + π 2. (1.15 Exponential distriution The DF of this very popular distriution is given y f(x a, = 1 ( exp x a, x a, a R, > 0. (1.16 Extreme value distriutions Extreme value distriutions are the limiting distriutions of either the largest or the smallest value in a sample of size n for n. We have three types for each of the two cases (largest or smallest oservation. Only the type I distriutions, which are of GUMBEL type (EMIL JULIUS GUMBEL, are of location scale type: Extreme value distriution of type I for the maximum f(x a, = 1 { exp x a ( exp x a }, x R, a R, > 0. (1.17a This is often called the extreme value distriution y some authors. Extreme value distriution of type I for the minimum f(x a, = 1 { ( } x a x a exp exp, x R, a R, > 0. (1.17 The type II and the type III extreme value distriutions can e transformed to type I distriutions, see Sect Half CAUCHY distriution This distriution results when the CAUCHY distriution (1.13 is folded around its location parameter a so that the left hand part for x < a is added to the right hand part (x > a. { [ ( ]} 1 x a f(x a, = 2 π 1 +, x a, a R, > 0. (1.18 Half logistic distriution The half logistic distriution results from the logistic distriution (1.23 in the same way as the half CAUCHY distriution is derived from the CAUCHY distriution: ( 2 exp x a f(x a, = [ ( 1 + exp x a ] 2, x a, a R, > 0. (1.19

20 8 1 The family of location scale distriutions Half normal distriution The half normal distriution results from normal distriution (1.25 like the half CAUCHY distriution from the CAUCHY distriution as f(x a, = 1 { 2 π exp 1 ( } 2 x a, x a, a R, > 0. ( Hyperolic secant distriution The DF is LAPLACE distriution f(x a, = 1 π sech ( x a, x R, a R, > 0. (1.21 This distriution is a ilateral or two tailed exponential distriution with DF Logistic distriution f(x a, = 1 { 2 exp x a }, x R, a R, > 0. (1.22 This distriution can e written in different ways, see Sect , one eing f(x a, = ( x a exp [ ( ] 2, x R, a R, > 0. (1.23 x a 1 + exp MAXWELL BOLTZMANN distriution This distriution is a special case of the χ distriution with ν = 3: f(x a, = 1 2 π ( 2 x a exp{ 1 ( } 2 x a, x a, a R, > 0. ( Normal distriution This well known distriution has DF 5 { f(x a, = 1 2 π exp 1 ( } 2 x a, x R, a R, > 0. ( We remark the a = µ(x = E(X and 2 = σ 2 X = Var(X.

21 1.2 Genuine location scale distriutions A short listing 9 Paraolic distriutions of order 2 We have two types, one has an U form with DF f(x a, = 3 ( 2 x a, a x a +, a R, > 0, (1.26a 2 the other one has an inverted U form with DF [ f(x a, = 3 ( ] 2 x a 1, a x a +, a R, > 0. ( Power function distriution (Special cases The power function distriution with DF ( c 1 x a, a x a +, a R, > 0, c > 0, (1.27a f(x a,, c = c and CDF ( c x a F (x a,, c = (1.27 is not of location scale type. We can transform (1.27a, to a location scale DF y considering X = ln(x a to arrive at the reflected exponential distriution, see (1.7 and (1.29. We also have a location scale distriution when c is known, e.g.: c = 1 gives an uniform distriution or rectangular distriution. c = 2 gives a right angled negatively skew triangular distriution. c = 3 gives a paraolic distriution of order 2, ut with a support of length. RAYLEIGH distriution This distriution is a special case of the WEIBULL distriution, see (1.45a, with shape parameter c = 2: f(x a, = 1 ( { x a exp 1 ( } 2 x a x a, a R, > 0. ( Reflected exponential distriution When the exponential distriution is reflected around x = a we get the DF f(x a, = 1 ( a x, x a, a R, > 0. (1.29

22 10 1 The family of location scale distriutions Semi elliptical distriution The graph of the DF f(x a, = 1 1 π is a semi ellipse centered at (a, 0. For = TEISSIER distriution ( 2 x a, a x a +, a R, > 0 ( / π we will have a semi circle. The DF of this distriution, named after the French iologist G. TEISSIER and pulished in 1934, is given y f(x a, = 1 { exp ( x a Triangular Distriutions 1 } { exp 1+ x a exp We have several types of triangular distriutions: a symmetric version with DF f(x a, = or equivalently written as ( x a x a + 2, a x a, a + x 2, a x a +, }, x a, a R, > 0 (1.31, a R, > 0, (1.32a f(x a, = x a 2, a x a +, (1.32 a right angled and positively skew version with DF f(x a, = 2 ( a + x, a x a +, a R, > 0, (1.32c a right angled and negatively skew version with DF f(x a, = 2 ( x a +, a x a, a R, > 0, (1.32d asymmetric versions which have esides a and a third parameter indicating the mode, thus they are not of location scale type.

23 1.3 Distriutions transformale to location scale type 11 Uniform or rectangular distriution This rather simple, ut very important distriution see Sect has DF f(x a, = 1, a x a +, a R, > 0. (1.33 U shaped and inverted U shaped distriutions All paraolic distriutions of order k = 2 m, m N, have an U shape and are of location scale type when m is known. The DF is f(x a,, m = 2 m ( 2 m x a, a x a +, a R, > 0. (1.34 We arrive at a distriution whose graph has an inverted U shape with DF [ f(x a, = 2 m + 1 ( ] 2 m x a 1, a x a +, a R, > 0. ( m We will only study the case m = 1 which gives the paraolic distriutions of order 2, see (1.26a,. V shaped distriution A symmetric V shaped DF is given y f(x a, = or equivalently written as a x, 2 a x a, x a, 2 a x a +, a R, > 0, (1.36a f(x a, = x a 2, a x a +. ( Distriutions transformale to location scale type We start y giving the rules governing the transformation of a variate X to another random variale X. Theorem: Let X e continuously distriuted with DF f X (x, CDF F X (x, mean µ X = E(X, variance σx 2 = Var(X and percentiles x P, 0 P 1, and let x = g(x e a transforming function which is a one to one mapping over the entire range of X and thus is monotonic so that x = g 1 ( x exists. Furthermore g(x has to e differentiale twice. Then X = g(x

24 12 1 The family of location scale distriutions has the DF the CDF the percentiles the approximative mean f X( x = f X [ g 1 ( x ] dg 1 ( x d x, [ F X g 1 ( x ] for g (x > 0 F X( x = [ 1 F X g 1 ( x ] for g (x < 0 g(x P for g (x > 0 x P = g(x 1 P for g (x < 0,, (1.37a (1.37 (1.37c and the approximative variance (1.37e should e used only when σ X / µx 1. µ X = E( X g(µ X + σ2 X 2 g (µ X, (1.37d σ 2 X = Var( X σ 2 X[ g (µ X ] 2. (1.37e We will concentrate on the most popular transformation, the ln transformation, for rendering a non location scale distriution to a location scale distriution. In this case, when the original variale X has a location parameter a 0, we either have to know its value, what rarely is the case, or we have to estimate it efore forming ln(x a or ln(a x. In Sect we will give several estimators of a. Extreme value distriutions of type II and type III The extreme value distriution of type II for the maximum, sometimes referred to as FRÉCHET type distriution has DF f(x a,, c = c ( c 1 ( } x a exp{ c x a x a, a R, > 0, c > 0, (1.38a and CDF 0 for x < a, { ( } F (x a,, c = c x a exp for x a. (1.38 Forming x = g(x = ln(x a (1.39a

25 1.3 Distriutions transformale to location scale type 13 we first have x = g 1 ( x, = exp( x + a (1.39 and dg 1 ( x d x The CDF of X easily follows from (1.37 with (1.38 as = exp( x. (1.39c { ( } c exp( x F ( x, c = exp, x = ln(x a R. (1.40a Using the identity exp(ln, (1.40a first can e written as { [ ] } c exp( x F ( x, c = exp exp(ln { [ ] c } = exp exp( x ln = exp { exp[ c ( x ln ] }. (1.40 Introducing the transformed location scale parameter (ã,, where (1.40 results in ã = ln, (1.40c = 1 c, (1.40d { ( F ( x ã, = exp exp x ã }, x R, ã R, > 0, (1.40e which is recognized as the CDF of the extreme value distriution of type I for the maximum. The DF elonging to (1.40e can e found either y differentiating (1.40e with respect to x or y applying (1.37a together with (1.39,c. The result is f( x ã, = 1 { exp x ã ( exp x ã }. (1.40f Thus, when X is of maximum type II, ln(x a is of maximum type I with location scale parameter (ã, = (ln, 1 / c. The transformations of the other extreme value distriutions follow along the same line.

26 14 1 The family of location scale distriutions The extreme value distriution of type III for the maximum, sometimes referred to as WEIBULL type distriution, has DF f(x a,, c= c ( c 1 ( c } a x a x exp{, x a, a R, > 0, c > 0 (1.41a and CDF F (x a,, c = { exp ( c } a x for x a, 1 for x > a. (1.41 The transformed variale X = ln(a X (1.42a has the type I distriution for the maximum (1.40e,f with scale parameter = 1 / c, whereas the new location parameter is ã = ln. (1.42 The extreme value distriution of type II for the minimum, the FRÉCHET distriution, has DF f(x a,, c= c ( c 1 ( } c a x a x exp{, x a, a R, > 0, c > 0 (1.43a and CDF { ( } c a x 1 exp for x a, F (x a,, c = 1 for x > a. (1.43 Introducing once more X = ln(a X (1.44a transforms this distriution to the extreme value distriution of type I for the minimum, see (1.17, with DF f( x ã, = 1 { ( } x ã x ã exp exp, x R, ã R, > 0, (1.44 and CDF { F ( x ã, = 1 exp exp ( x ã }, (1.44c where = 1 / c and ã = ln.

27 1.3 Distriutions transformale to location scale type 15 The extreme value distriution of type III for the minimum, the WEIBULL distriution, 6 has DF f(x a,, c= c ( c 1 ( c } x a x a exp{, x a, a R, > 0, c > 0 (1.45a and CDF F (x a,, c = With { 1 exp 0 for x < a, ( c } x a for x a. X = ln(x a (1.45 we find the extreme value distriution of type I for the minimum, see (1.44,c, where = 1 / c ut ã = ln. The extreme value of type I for the minimum is often referred to as Log WEIBULL distriution. Comparing (1.45a with (1.38a we may also call the extreme value distriution of type II for the maximum an inverse WEIBULL distriution. Comparing (1.45a with (1.41a we see that the extreme value distriution of type III for the maximum may e called reflected WEIBULL distriution. Ta. 1/2 summarizes the transformation procedures just descried in a concise manner. Tale 1/2: Transformation of extreme value distriutions Original Transformed Transformed Transformed distriution variale distriution parameters maximum of type II x = ln(x a maximum of type I ã = ln (1.38a, (1.40e,f = 1 / c maximum of type III x = ln(a x maximum of type I ã = ln (1.41a, (1.40e,f = 1 / c minimum of type II x = ln(a x minimum of type I ã = ln (1.43a, (1.44,c = 1 / c minimum of Type III x = ln(x a minimum of type I ã = ln (1.45a, (1.44,c ã = 1 / c Lognormal distriution If there is a real numer a such that X = ln(x a (1.46a 6 For more details of this distriution see RINNE (2009.

28 16 1 The family of location scale distriutions is normally distriuted, the distriution of X is said to e lognormal. The DF of X is { 1 f(x a, ã, = (x a 2 π exp 1 ( } 2 ln(x a ã x a, a R, 2 ã R, > 0. (1.46 The graph of (1.46 is positively skew. The distriution of X is normal with DF { f( x ã, = 1 exp 1 ( } 2 x ã x R, 2 π 2 ã R, > 0. (1.46c The meaning of the three parameters in (1.46 is as follows: a is the location parameter of X and gives a lower threshold. 7 ã = E[ln(X a] is the location parameter of the transformed variate. = Var[ln(X a] is the scale parameter of X. In Sections and we will show that the variance of X is dependent on ã and and that the shape, i.e. the skewness and the kurtosis, of X is only dependent on. PARETO distriution The PARETO distriution (of the first kind has a negatively skew DF: ( c 1 x a, x a +, a R, > 0, c > 0, (1.47a and CDF Introducing f(x a,, c = c ( c x a F (x a,, c = 1. (1.47 X = ln(x a leads to f( x ã, = 1 ( exp x ã ( F ( x ã, = 1 exp x ã, x ã, ã R, > 0, (1.48a (1.48 with ã = ln, = 1 / c. (1.48c So X has an exponential distriution. 7 The term lognormal can also e applied to the distriution of X if ln(a X is normally distriuted. In this case a is an upper threshold and X has zero proaility of exceeding a.

29 1.3 Distriutions transformale to location scale type 17 Power function distriution The power function distriution with DF ( c 1 x a, a x a +, a R, > 0, c > 0, (1.49a f(x a,, c = c and CDF y introducing transforms to ( c x a F (x a,, c = (1.49 X = ln(x a f( x ã, = 1 ( x ã exp, x ã, ã R, > 0, (1.50a F ( x ã, = ( x ã exp (1.50 with parameters ã = ln, = 1 / c. (1.50c (1.50a is recognized as the reflected exponential distriution. Because the PARETO and the power function distriutions are related y a reciprocal transformation of their variales, the logarithms of these variales differ y sign and their distriutions are related y a reflection.

30 2 Order statistics 1 Order statistics and their functions play an important role in proaility plotting and linear estimation of location scale distriutions. Plotting positions, see Sect , and regressors as well as the elements of the variance covariance matrix in the general least squares (GLS approach, see Sect , are moments of the reduced order statistics. In most sampling situations the oservations have to e ordered after the sampling, ut in life testing, when failed items are not replaced, order statistics will arise in a natural way. 2.1 Distriutional concepts Let X 1, X 2,..., X n e independently identically distriuted (iid with CDF F (x. The variales X i eing arranged in ascending order and written as X 1:n X 2:n... X n:n are called order statistics. The CDF of X r:n is given y F r:n (x = Pr ( X r:n x = Pr(at least r of the X i are less than or equal to x = n ( n [F ] i [ ] n i, (x 1 F (x i (2.1a i=r since the term in the summand is the inomial proaility that exactly i of X 1,..., X n are less than or equal to x. F r:n (x can e written as the incomplete eta function ratio or eta distriution function: F r:n (x = F (x 0 u r 1 (1 u n r du B(r, n r + 1. (2.1 Because the parameters r and n of the complete eta function are integers we have B(r, n r + 1 = (r 1! (n r! n!. (2.1c (2.1a, hold whether X is discrete or continuous. In the following text we will always assume that X is asolutely continuous with DF f(x. Then differentiation of (2.1 and 1 Suggested reading for this chapter: ARNOLD/BALAKRISHNAN/NAGARAJA (1992, BALAKRIS- NAN/COHEN (1991, BALAKRISHNAN/RAO (1998a,, DAVID (1981, GALAMBOS (1978, SARHAN/GREENBERG (1962.

31 2.1 Distriutional concepts 19 regarding (2.1c gives the DF of X r:n : f r:n (x = Two values of r are of special interest: n! [ ] r 1 [ ] n r F (x 1 F (x f(x. (2.1d (r 1! (n r! r = 1 X 1:n is the sample minimum with DF and CDF f 1:n (x = n f(x [ 1 F (x ] n 1, (2.2a F 1:n (x = n i=1 ( n [F ] i [ ] n i [ ] n. (x 1 F (x = 1 1 F (x (2.2 i r = n X n:n is the sample maximum with DF and CDF f n:n (x = n f(x [ F (x ] n 1, (2.3a F n:n (x = [ F (x ] n. (2.3 There are only a few variates whose order statistics DF can e given in a simple and handy form. The most important example is the reduced uniform variate, denoted as U in the following text, with DF and CDF f(u = 1, 0 u 1, (2.4a F (u = u. (2.4 Upon inserting (2.4a, into (2.1a and (2.1d we find for U r:n, 1 r n: F r:n (u = = n i=r ( n u i (1 u n i i n! (r 1! (n r! u 0 t r 1 (1 t n r dt, 0 u 1, (2.4c and f r:n (u = Thus, U r:n has a reduced eta distriution. n! (r 1! (n r! ur 1 (1 u n r. (2.4d Another example is the reduced power function distriution with DF and CDF f(y = c y c 1, 0 y 1, (2.5a F (y = y c. (2.5

32 20 2 Order statistics With (2.5a, the CDF and DF of Y r:n, 1 r n, follow from (2.1a,d as F r:n (y = = f n:r (y = n i=r ( n (y c i ( 1 y c n i i n! (r 1! (n r! yc 0 t r 1 (1 t n r dt, 0 y 1, (2.5c n! (r 1! (n r! c yr c 1 ( 1 y c n r. (2.5d The joint DF of X r:n and X s:n, 1 r < s n, is f r,s:n (x, y = n! [ ] r 1 [ ] s r 1 F (x F (y F (x (r 1! (s r 1! (n s! [ 1 F (y ] n s f(x f(y, x < y. (2.6a Even if X 1,..., X n are independent, their order statistics are not independent random variales. The joint CDF of X r:n and X s:n may e otained y integration of (2.6a as well as y a direct argument valid also in the discrete case. For x < y we have: F r,s:n (x, y = Pr(at least r X i x and at least s X i y = n j Pr(exactly k X i x and exactly j X i y = j=s k=r n j n! [ ] k [ ] j k F (x F (y F (x k! (j k! (n j! j=s k=r [ ] n j, 1 F (y x < y. (2.6 For x y the inequality X s:n y implies X r:n x, so that F r,s:n (x, y = F s:n (y, x y. (2.6c By using the identity n j j=s k=r = n! ( j k ( n j k! (j k! (n j! pk 1 p2 p 1 1 p2 n! (r 1!(s r 1!(n s! p 1 p 2 0 t 1 ( t r 1 s r 1 ( n s 1 t2 t 1 1 t2 dt2 dt 1, 0 < p 1 < p 2 < 1,

33 2.2 Moments of order statistics 21 we can write the joint CDF of X r:n and X s:n in (2.6 equivalently as F (x F (y n! ( F r,s:n (x, y = t r 1 s r 1 1 t2 t 1 (r 1! (s r 1!(n s! 0 t ( 1 n s 1 t2 dt2 dt 1, 0 < x < y <, (2.7 which is the CDF of the reduced ivariate eta distriution. The joint DF of X r:n and X s:n may e derived from (2.7 y differentiating with respect to oth x and y. Let U 1, U 2,..., U n e a sample of the reduced uniform distriution and X 1, X 2,..., X n e a random sample from a population with CDF F (x. Furthermore, let U 1:n U 2:n... U n:n and X 1:n X 2:n... X n:n e the corresponding order statistics. Specifically, when F (x is continuous the proaility integral transformation U = F (X produces a reduced uniform distriution. Thus, when F (x is continuous we have F (X r:n d = U r:n, r = 1, 2,..., n. (2.8a d where = reads as has the same distriution as. Furthermore, with the inverse CDF F 1 (, it is easy to verify that F 1 (U r d = X r, r = 1, 2,..., n for an aritrary F (. Since F 1 ( is also order preserving, it immediately follows that F 1 (U r:n d = X r:n, r = 1, 2,..., n. (2.8 (2.8c We will apply the distriutional relation (2.8c in Sect in order to develop some series approximations for the moments of reduced order statistics Y r:n in terms of moments of the uniform order statistics U r:n. 2.2 Moments of order statistics For proaility plotting and linear estimation we will need the first and second moments of the reduced order statistics Definitions and asic formulas We will denote the crude single moments of the reduced order statistics, E(Y k α (k as r:n, 1 r n. They follow with α (k r:n := E(Y k r:n = = f(y := f Y (y and F (y := F Y (y n! (r 1! (n r! y k f r:n (y dy y k [F (y] r 1 [1 F (y] n r f(y dy. r:n, y (2.9a

34 22 2 Order statistics The most import case of (2.9a is the mean, shortly denoted y α r:n : ( n 1 α r:n := E(Y r:n = n r 1 Since 0 F (y 1, it follows that ( n 1 α r:n n r 1 y [F (y] r 1 [1 F (y] n r df (y. y df (y, (2.9 (2.9c showing that α r:n exists provided E(Y exists, although the converse is not necessarily true. We mention that for the general variate X = a + Y the crude single moments of X r:n are given y k ( k µ (k r:n := E(Xr:n k = α r:n (k j k j a j, j especially j=0 µ r:n := E(X r:n = a + α r:n. An alternative formula for α r:n may e otained y integration y parts in α r:n = y df r:n (y. To this end, note that for any CDF F (y the existence of E(Y implies so that we have lim y F (y = 0 and lim y E(Y = y df (y y [1 F (y] = 0, x = 0 y df (y y d[1 F (y] = 0 [1 F (y] dy 0 F (y dy. (2.10a 0 This general formula gives α r:n = E(Y r:n if F (y is replaced y F r:n (y : α r:n = [ 1 Fr:n (y ] dy 0 F r:n (y dy. (2.10 0

35 2.2 Moments of order statistics 23 We may also write α r:n = [ 1 Fr:n (y F r:n ( y ] dy, (2.10c and when f(y is symmetric aout y = 0 we have 0 α r:n = 0 [ Fn r+1 (y F r:n (y ] dy. (2.10d Crude product moments of reduced order statistics may e defined similarly: ( α r,s:n (k,l := E Yr:n k Ys:n l = n! (r 1! (s r 1! (n s! <t<v< t k v l [ F (t ] r 1 [ F (v F (t ] s r 1 [ 1 F (v ] n s f(t f(v dt dv. (2.11a The most important case of(2.11a has k = l = 1 and leads to the covariance of Y r:n and Y s:n : Cov ( Y r:n, Y s:n = E ( Yr:n Y s:n E ( Yr:n E ( Ys:n = α r,s:n α r:n α s:n. (2.11 We introduce the notation β r,s:n := Cov(Y r:n, Y s:n. (2.12a A special case of (2.12a for r = s is the variance of Y r:n : β r,r:n = Var(Y r:n = α (2 r:n ( α r:n 2. (2.12 We collect all the variances and covariances for a given sample size in the so called variance covariance matrix B: B := ( β r,s:n ; r, s = 1,..., n. (2.12c which is a symmetric matrix: B = B or β r,s:n = β s,r:n r, s. (2.12d For the general order statistics we have σ r,s:n = 2 β r,s:n Σ = ( σ r,s:n = 2 B.

36 24 2 Order statistics Generally, the moments of order statistics cannot e given in a handy and closed form. They have either to e evaluated y numeric integration of (2.9a and (2.11a in comination with some recurrence relation or to e approximated y expanding Y r:n = F 1 (U r:n in a TAYLOR series around the point E(U r:n. Furthermore, when the distriution of Y is symmetric around zero we may save time in computing the moments of order statistics making use of the following relations which are ased on the distriutional equivalences resulting in: (Y r:n, Y s:n Y r:n d = ( Y n r+1:n, 1 r n, d = ( ( Y n s+1:n, ( Y n r+1:n, 1 r s n, α (k n r+1:n = ( 1 k α (k r:n, 1 r n, k 1, (2.13a α n s+1,n r+1:n = α r,s:n, 1 r s n, (2.13 β n s+1,n r+1:n:n = α n s+1,n r+1:n α n s+1:n α n r+1:n = α r,s:n α r:n α s:n = β r,s:n, 1 r s n. (2.13c Figure 2/1: Structure of the variance covariance matrix of order statistics from a distriution symmetric around zero A special case of (2.13a occurs when n is odd, n = 2 l + 1, l N: α l+1:2 l+1 = α l+1:2 l+1 = 0, i.e. the mean of the sample median is equal to zero. (2.13a means that we have to evaluate [ n+1 ] means instead of n, where [z] is the integer part of z. Looking at (2.13c we have 2 esides β r,s:n = β s,r:n a second symmetry in the variance covariance matrix of order

37 2.2 Moments of order statistics 25 statistics, see Fig. 2/1. The greater upper triangle ABD results from a reflection of the smaller triangle ABC at the line BC. We only have to evaluate the elements in the upper triangle ABC. Thus, instead of evaluating (n + 1 / 2 elements β r,s:n (r = 1,..., n; s r we only have to evaluate n (n + 2 / 4 elements for n = 2 l and (n + 1 2/ 4 elements for n = 2 l + 1. Closed form expressions for moments of order statistics exist among others for the following distriutions: 1. Reduced uniform distriution: f(u = 1, 0 u 1 E ( Ur:n k n! (r + k 1! = (n + k! (r 1! α r:n = β r,r:n = (2.14a r n + 1 =: p r (2.14 r (n r + 1 (n (n + 2 = p r q r n + 2, q r := 1 p r (2.14c E ( U r:n U s:n = r (s + 1 (n + 1 (n + 2, 1 1 < r < s n (2.14d β r,s:n = r (n s + 1 (n (n + 2 = p r q s n + 2, q s := 1 p s (2.14e 2. Reduced power function distriution: f(y = c y c 1, 0 y 1 E ( Yr:n k Γ(n + 1 = Γ(n k / Γ(r + k / c c Γ(r = n! (r 1! Γ(r + k / c Γ(n k / c (2.15a α r:n = β r,r:n = n! (r 1! n! (r 1! Γ(r + 1 / c Γ(n / c { / Γ(r + 2 c Γ(n / c n! (r 1! (2.15 Γ 2 (r / } c Γ 2 (n / (2.15c c E(Y r:n Y s:n = β r,s:n = n! (r 1! Γ(r + 1 / c Γ(s + 2 / c Γ(s + 1 / c Γ(n / c, 1 r < s n { / / n! Γ(s + 2 c Γ(r + 1 c (r 1! Γ(s + 1 / c Γ(n / c n! Γ(s + 1 / c Γ(r + 1 / } c (s 1! Γ 2 (n / c (2.15d (2.15e