Exact Linear Likelihood Inference for Laplace

Transcription

1 Exact Linear Likelihood Inference for Laplace Prof. N. Balakrishnan McMaster University, Hamilton, Canada p. 1/52

2 Pierre-Simon Laplace p. 2/52

3 Laplace s Biography Born: On March 23, 1749, in Normandy, France p. 3/52

4 Laplace s Biography Born: On March 23, 1749, in Normandy, France Died: On March 5, 1827, in Paris, France p. 3/52

5 Laplace s Biography Born: On March 23, 1749, in Normandy, France Died: On March 5, 1827, in Paris, France Nationality: French p. 3/52

6 Laplace s Biography Born: On March 23, 1749, in Normandy, France Died: On March 5, 1827, in Paris, France Nationality: French Alma mater: University of Caen, France p. 3/52

7 Laplace s Biography Born: On March 23, 1749, in Normandy, France Died: On March 5, 1827, in Paris, France Nationality: French Alma mater: University of Caen, France Advisors: Jean d Alembert, Christophe Gadbled p. 3/52

8 Laplace s Biography Born: On March 23, 1749, in Normandy, France Died: On March 5, 1827, in Paris, France Nationality: French Alma mater: University of Caen, France Advisors: Jean d Alembert, Christophe Gadbled Student: Siméon Denis Poisson p. 3/52

9 Laplace s Biography Born: On March 23, 1749, in Normandy, France Died: On March 5, 1827, in Paris, France Nationality: French Alma mater: University of Caen, France Advisors: Jean d Alembert, Christophe Gadbled Student: Siméon Denis Poisson p. 3/52

10 In collaboration with Colleen Cutler (Univ. of Waterloo) G. Iliopoulos (Univ. of Piraeus) Aaron Childs (McMaster Univ.) X. Zhu (Xian-Jiaotong Liverpool Univ) R. Ambagaspitiya (Univ. of Calgary) Kai Liu (McMaster Univ.) p. 4/52

11 Goals 1. To provide historical details on Laplace distribution; p. 5/52

12 Goals 1. To provide historical details on Laplace distribution; 2. To describe results on order statistics; p. 5/52

13 Goals 1. To provide historical details on Laplace distribution; 2. To describe results on order statistics; 3. To present results for Type-II censoring; p. 5/52

14 Goals 1. To provide historical details on Laplace distribution; 2. To describe results on order statistics; 3. To present results for Type-II censoring; 4. To present results on outlier-models and robustness; p. 5/52

15 Goals 1. To provide historical details on Laplace distribution; 2. To describe results on order statistics; 3. To present results for Type-II censoring; 4. To present results on outlier-models and robustness; 5. To present results for Type-I censoring; p. 5/52

16 Goals 1. To provide historical details on Laplace distribution; 2. To describe results on order statistics; 3. To present results for Type-II censoring; 4. To present results on outlier-models and robustness; 5. To present results for Type-I censoring; 6. To present some numerical results and examples; p. 5/52

17 Goals 1. To provide historical details on Laplace distribution; 2. To describe results on order statistics; 3. To present results for Type-II censoring; 4. To present results on outlier-models and robustness; 5. To present results for Type-I censoring; 6. To present some numerical results and examples; 7. To mention briefly some other work carried out recently. p. 5/52

18 Historical Details! p. 6/52

19 Distribution The Laplace distribution has its pdf as f(x) = 1 2σ e x µ /σ, < x <, where µ and σ are the location and scale parameters. p. 7/52

20 Distribution The Laplace distribution has its pdf as f(x) = 1 2σ e x µ /σ, < x <, where µ and σ are the location and scale parameters. It is also known as Double Exponential Distribution. p. 7/52

21 Distribution The Laplace distribution has its pdf as f(x) = 1 2σ e x µ /σ, < x <, where µ and σ are the location and scale parameters. It is also known as Double Exponential Distribution. The distribution is symmetric about µ. p. 7/52

22 Distribution The Laplace distribution has its pdf as f(x) = 1 2σ e x µ /σ, < x <, where µ and σ are the location and scale parameters. It is also known as Double Exponential Distribution. The distribution is symmetric about µ. The corresponding cdf is 1 2 F(x) = e(x µ)/σ, x µ e (x µ)/σ, x > µ. p. 7/52

23 Distribution The quantile function is µ+σln(2u), 0 < u 1 2 Q(u) = 1 µ σln{2(1 u)}, < u < 1. 2 p. 8/52

24 Distribution The quantile function is µ+σln(2u), 0 < u 1 2 Q(u) = 1 µ σln{2(1 u)}, < u < 1. 2 Chapter 24 of Johnson, Kotz and Balakrishnan (1995) and the book of Kotz, Kozubowski and Podgórski (2001) both provide detailed overviews of various developments on theory, methods and applications of the Laplace distribution and its generalizations. p. 8/52

25 ML Estimation Let x 1,,x n be a random sample from the Laplace distribution. p. 9/52

26 ML Estimation Let x 1,,x n be a random sample from the Laplace distribution. Then, the likelihood function is { n L = f(x i ) = 1 2 n σ exp 1 n σ i=1 n i=1 x i µ }. p. 9/52

27 ML Estimation Let x 1,,x n be a random sample from the Laplace distribution. Then, the likelihood function is { n L = f(x i ) = 1 2 n σ exp 1 n σ i=1 n i=1 x i µ }. Evidently, L is maximized with respect to µ when n i=1 x i µ is minimized. p. 9/52

28 ML Estimation Let x 1,,x n be a random sample from the Laplace distribution. Then, the likelihood function is { n L = f(x i ) = 1 2 n σ exp 1 n σ i=1 n i=1 x i µ }. Evidently, L is maximized with respect to µ when n i=1 x i µ is minimized. This happens when µ is the central most value (if n is odd), or any value in the interval of the two middle most values (if n is even). p. 9/52

29 ML Estimation Thus, the MLE of µ is taken to be the sample median x m+1:n, n = 2m+1 ˆµ = 1 (x 2 m:n +x m+1:n ), n = 2m. p. 10/52

30 ML Estimation Thus, the MLE of µ is taken to be the sample median x m+1:n, n = 2m+1 ˆµ = 1 (x 2 m:n +x m+1:n ), n = 2m. Next, upon maximizing L with respect to σ, we obtain the MLE of σ to be ˆσ = 1 n n i=1 x i Median. p. 10/52

31 ML Estimation Thus, the MLE of µ is taken to be the sample median x m+1:n, n = 2m+1 ˆµ = 1 (x 2 m:n +x m+1:n ), n = 2m. Next, upon maximizing L with respect to σ, we obtain the MLE of σ to be ˆσ = 1 n n i=1 x i Median. Note that the MLEs in this case naturally turn out to be linear estimators, meaning they are linear functions of order statistics. p. 10/52

32 Historical Note In fact, the Laplace distribution was discovered by Laplace (1774) as the distribution form for which the likelihood function is maximized by setting the location parameter µ equal to the median of the observed values of an odd number of i.i.d. observations. p. 11/52

33 Historical Note In fact, the Laplace distribution was discovered by Laplace (1774) as the distribution form for which the likelihood function is maximized by setting the location parameter µ equal to the median of the observed values of an odd number of i.i.d. observations. Furthermore, Laplace went on by replacing the median by the arithmetic mean as the value maximizing the likelihood function and derived the corresponding distribution function to be the normal distribution; see Stigler (1975). p. 11/52

34 Here comes censoring! p. 12/52

35 Symmetric Censoring Suppose we have available only a Type-II symmetrically censored sample of the form x r+1:n < < x n r:n, with the smallest r and the largest r order statistics having been censored. p. 13/52

36 Symmetric Censoring Suppose we have available only a Type-II symmetrically censored sample of the form x r+1:n < < x n r:n, with the smallest r and the largest r order statistics having been censored. Then, the corresponding likelihood function is L = n! (r!) 2 [F(x r+1:n){1 F(x n r:n )}] r n r i=r+1 f(x i:n ). p. 13/52

37 Symmetric Censoring Suppose we have available only a Type-II symmetrically censored sample of the form x r+1:n < < x n r:n, with the smallest r and the largest r order statistics having been censored. Then, the corresponding likelihood function is L = n! (r!) 2 [F(x r+1:n){1 F(x n r:n )}] r n r i=r+1 f(x i:n ). Now, let us first maximize L with respect to µ. p. 13/52

38 Symmetric Censoring 1. Case µ < x r+1:n : In this case, the likelihood is L = [ C 1 σ n 2r 2 e(µ x n r:n)/σ 1 4 e(2µ x r+1:n x n r:n)/σ [e { ] n r i=r+1 x i:n+(n 2r)µ}/σ. ] r p. 14/52

39 Symmetric Censoring 1. Case µ < x r+1:n : In this case, the likelihood is L = [ C 1 σ n 2r 2 e(µ x n r:n)/σ 1 4 e(2µ x r+1:n x n r:n)/σ [e { ] n r i=r+1 x i:n+(n 2r)µ}/σ. ] r It is easy to show that the second and third terms are both increasing functions of µ for < µ < x r+1:n. p. 14/52

40 Symmetric Censoring 1. Case µ < x r+1:n : In this case, the likelihood is L = [ C 1 σ n 2r 2 e(µ x n r:n)/σ 1 4 e(2µ x r+1:n x n r:n)/σ [e { ] n r i=r+1 x i:n+(n 2r)µ}/σ. ] r It is easy to show that the second and third terms are both increasing functions of µ for < µ < x r+1:n. So, L is increasing over the range < µ < x r+1:n. p. 14/52

41 Symmetric Censoring 2. Case µ > x n r:n : In this case, the likelihood is L = C σ n 2r [ 1 2 e(x r+1:n µ)/σ 1 4 e(x r+1:n+x n r:n 2µ)/σ [e { n r i=r+1 x i:n (n 2r)µ}/σ ]. ] r p. 15/52

42 Symmetric Censoring 2. Case µ > x n r:n : In this case, the likelihood is L = C σ n 2r [ 1 2 e(x r+1:n µ)/σ 1 4 e(x r+1:n+x n r:n 2µ)/σ [e { n r i=r+1 x i:n (n 2r)µ}/σ ]. ] r In this case, it is easy to show that second and the third terms are both decreasing functions of µ for x n r:n < µ <. p. 15/52

43 Symmetric Censoring 2. Case µ > x n r:n : In this case, the likelihood is L = C σ n 2r [ 1 2 e(x r+1:n µ)/σ 1 4 e(x r+1:n+x n r:n 2µ)/σ [e { n r i=r+1 x i:n (n 2r)µ}/σ ]. ] r In this case, it is easy to show that second and the third terms are both decreasing functions of µ for x n r:n < µ <. So, L is decreasing over the range x n r:n < µ <. p. 15/52

44 Symmetric Censoring 3. Case x r+1: µ x n r:n : In this case, L = C n r:n x r+1:n)/σ e n r σ n 2re r(x i=r+1 xi:n µ /σ. p. 16/52

45 Symmetric Censoring 3. Case x r+1: µ x n r:n : In this case, L = C n r:n x r+1:n)/σ e n r σ n 2re r(x i=r+1 xi:n µ /σ. From this expression, it is clear that the MLE of µ is once again the sample median given by x m+1:n, n = 2m+1 ˆµ = any value in [x m:n,x m+1:n ], n = 2m. p. 16/52

46 Symmetric Censoring 3. Case x r+1: µ x n r:n : In this case, L = C n r:n x r+1:n)/σ e n r σ n 2re r(x i=r+1 xi:n µ /σ. From this expression, it is clear that the MLE of µ is once again the sample median given by x m+1:n, n = 2m+1 ˆµ = any value in [x m:n,x m+1:n ], n = 2m. So, the MLE of µ in the case of symmetric censoring is exactly the same as that based on complete sample. p. 16/52

47 Symmetric Censoring Next, upon maximizing L with respect to σ, we obtain p. 17/52

48 Symmetric Censoring Next, upon maximizing L with respect to σ, we obtain [ 2m+1 r 1 ˆσ = x i:2m+1 +rx 2m+1 r:2m+1 2m+1 2r i=m+2 ] m x i:2m+1 rx r+1:2m+1, n = 2m+1; ˆσ = i=r+1 1 2m 2r m i=r+1 [ 2m r i=m+1 x i:2m +rx 2m r:2m x i:2m rx r+1:2m ], n = 2m. p. 17/52

49 Symmetric Censoring Next, upon maximizing L with respect to σ, we obtain [ 2m+1 r 1 ˆσ = x i:2m+1 +rx 2m+1 r:2m+1 2m+1 2r i=m+2 ] m x i:2m+1 rx r+1:2m+1, n = 2m+1; ˆσ = i=r+1 1 2m 2r m i=r+1 [ 2m r i=m+1 x i:2m +rx 2m r:2m x i:2m rx r+1:2m ], n = 2m. Interestingly, the MLEs of µ and σ in this case are also linear estimators [Balakrishnan and Cutler (1996)]. p. 17/52

50 Right Censoring Let us now consider a Type-II right censored sample of the form x 1:n < < x n r:n, with largest r order statistics having been censored. p. 18/52

51 Right Censoring Let us now consider a Type-II right censored sample of the form x 1:n < < x n r:n, with largest r order statistics having been censored. Then, the corresponding likelihood function is L = n! r! {1 F(x n r:n)} r n r i=1 f(x i:n ). p. 18/52

52 Right Censoring Let us now consider a Type-II right censored sample of the form x 1:n < < x n r:n, with largest r order statistics having been censored. Then, the corresponding likelihood function is L = n! r! {1 F(x n r:n)} r n r i=1 f(x i:n ). In this case, the MLEs depend on whether r n 2 or r > n 2. p. 18/52

53 Right Censoring 1. Case r n : In this case, note that the median of the 2 sample is available. So, the MLE of µ is once again the sample median. p. 19/52

54 Right Censoring 1. Case r n : In this case, note that the median of the 2 sample is available. So, the MLE of µ is once again the sample median. 2. Case r > n : In this case, the median of the sample 2 is not available. p. 19/52

55 Right Censoring 1. Case r n : In this case, note that the median of the 2 sample is available. So, the MLE of µ is once again the sample median. 2. Case r > n : In this case, the median of the sample 2 is not available. It is easy to show in this case that the likelihood function is increasing over µ in the interval (,x 1:n ) and also increasing in the interval (x 1:n,x n r:n ]. p. 19/52

56 Right Censoring 1. Case r n : In this case, note that the median of the 2 sample is available. So, the MLE of µ is once again the sample median. 2. Case r > n : In this case, the median of the sample 2 is not available. It is easy to show in this case that the likelihood function is increasing over µ in the interval (,x 1:n ) and also increasing in the interval (x 1:n,x n r:n ]. So, the MLE of µ must be in the interval (x n r:n, ). p. 19/52

57 Right Censoring The likelihood function in this case is { e(x n r:n µ)/σ L = C σ n r } r e n r i=1 (µ x i:n)/σ. p. 20/52

58 Right Censoring The likelihood function in this case is { e(x n r:n µ)/σ L = C σ n r } r e n r i=1 (µ x i:n)/σ. When this function is maximized for µ, we determine the MLE of µ (when r > n 2 ) as ˆµ = x n r:n + ˆσ ( log n/2 ). n r p. 20/52

59 Right Censoring The likelihood function in this case is { e(x n r:n µ)/σ L = C σ n r } r e n r i=1 (µ x i:n)/σ. When this function is maximized for µ, we determine the MLE of µ (when r > n 2 ) as ˆµ = x n r:n + ˆσ ( log n/2 ). n r Substituting these expressions of MLEs for µ in L and then maximizing it with respect to σ, we get expressions for the MLE of σ. p. 20/52

60 Right Censoring For example, in the case when r n, we obtain the 2 MLE of σ as follows: p. 21/52

61 Right Censoring For example, in the case when r n, we obtain the 2 MLE of σ as follows: ˆσ = ˆσ = 1 n r 1 n r [ n r i=m+2 [ n r i=m+1 x i:n x i:n ] m x i:n +rx n r:n, n = 2m+1; i=1 ] m x i:n +rx n r:n, n = 2m. i=1 p. 21/52

62 Right Censoring For example, in the case when r n, we obtain the 2 MLE of σ as follows: ˆσ = ˆσ = 1 n r 1 n r [ n r i=m+2 [ n r i=m+1 x i:n x i:n ] m x i:n +rx n r:n, n = 2m+1; i=1 ] m x i:n +rx n r:n, n = 2m. i=1 Similar expressions for ˆσ can be presented for the case when r > n by using the MLE of µ presented earlier 2 [Balakrishnan and Cutler (1996)]. p. 21/52

63 How about order statistics and moments? p. 22/52

64 Order Statistics Let X 1:n < < X n:n be the order statistics from a standard Laplace distribution. p. 23/52

65 Order Statistics Let X 1:n < < X n:n be the order statistics from a standard Laplace distribution. Let Y 1:n < < Y n:n be the order statistics from the folded distribution with pdf and cdf as p(x) = 2f(x) and P(x) = 2F(x) 1, x > 0. p. 23/52

66 Order Statistics Let X 1:n < < X n:n be the order statistics from a standard Laplace distribution. Let Y 1:n < < Y n:n be the order statistics from the folded distribution with pdf and cdf as p(x) = 2f(x) and P(x) = 2F(x) 1, x > 0. In other words, these order statistics are from the standard exponential distribution. p. 23/52

67 Order Statistics Let X 1:n < < X n:n be the order statistics from a standard Laplace distribution. Let Y 1:n < < Y n:n be the order statistics from the folded distribution with pdf and cdf as p(x) = 2f(x) and P(x) = 2F(x) 1, x > 0. In other words, these order statistics are from the standard exponential distribution. Let (µ (k) i:n,µ i,j:n) and (ν (k) i:n,ν i,j:n) denote the single and product moments of order statistics from Laplace and exponential distributions, respectively. p. 23/52

68 Order Statistics Then, Govindarajulu (1963) established the following relationships between these two sets of moments: p. 24/52

69 Order Statistics Then, Govindarajulu (1963) established the following relationships between these two sets of moments: { µ (k) i:n = 1 i 1 2 n r=0 ( ) n ν (k) i r:n r r +( 1)k n r=i ( ) } n ν (k) r i+1:r r ; p. 24/52

70 Order Statistics Then, Govindarajulu (1963) established the following relationships between these two sets of moments: { µ (k) i:n = 1 i 1 2 n r=0 { µ i,j:n = 1 i 1 2 n ( ) n ν (k) i r:n r r +( 1)k r=0 j 1 r=i n r=i ( ) n ν i r,j r:n r + r r=j } )ν r i+1:r ν j r:n r ( n r n ( ) } n ν (k) r i+1:r r ( ) n ν r j+1,r i+1:r r. ; p. 24/52

71 Order Statistics Then, Govindarajulu (1963) established the following relationships between these two sets of moments: { µ (k) i:n = 1 i 1 2 n r=0 { µ i,j:n = 1 i 1 2 n ( ) n ν (k) i r:n r r +( 1)k r=0 j 1 r=i n r=i ( ) n ν i r,j r:n r + r r=j } )ν r i+1:r ν j r:n r ( n r n ( ) } n ν (k) r i+1:r r ( ) n ν r j+1,r i+1:r r The proof provided by Govindarajulu (1963) is purely an algebraic one through integration. p. 24/52. ;

72 Order Statistics A simple and elegant probabilistic proof was given by Balakrishnan, Govindarajulu and Balasubramanian (1993). p. 25/52

73 Order Statistics A simple and elegant probabilistic proof was given by Balakrishnan, Govindarajulu and Balasubramanian (1993). Let D denote the number of X i s that are 0. p. 25/52

74 Order Statistics A simple and elegant probabilistic proof was given by Balakrishnan, Govindarajulu and Balasubramanian (1993). Let D denote the number of X i s that are 0. Then, the following properties hold: p. 25/52

75 Order Statistics A simple and elegant probabilistic proof was given by Balakrishnan, Govindarajulu and Balasubramanian (1993). Let D denote the number of X i s that are 0. Then, the following properties hold: 1. D Bin ( n, 2) 1 ; p. 25/52

76 Order Statistics A simple and elegant probabilistic proof was given by Balakrishnan, Govindarajulu and Balasubramanian (1993). Let D denote the number of X i s that are 0. Then, the following properties hold: 1. D Bin ( n, 2) 1 ; 2. Given D = i 1, X i:n < < X n:n are distributed exactly as order statistics from a sample of size n i+1 from exponential distribution; p. 25/52

77 Order Statistics 3. Also, given D = i 1, X i 1:n < < X 1:n are distributed exactly as order statistics from a sample of size i 1 from exponential distribution; p. 26/52

78 Order Statistics 3. Also, given D = i 1, X i 1:n < < X 1:n are distributed exactly as order statistics from a sample of size i 1 from exponential distribution; 4. Furthermore, given D = i 1, these two sets of order statistics are mutually independent. p. 26/52

79 Order Statistics 3. Also, given D = i 1, X i 1:n < < X 1:n are distributed exactly as order statistics from a sample of size i 1 from exponential distribution; 4. Furthermore, given D = i 1, these two sets of order statistics are mutually independent. Then, the above four properties readily yield the relations presented earlier; in addition, this approach leads to some generalizations of Govindarajulu s results as well. p. 26/52

80 What about BLUEs? p. 27/52

81 Bias, MSE & JRE BLUEs of µ and σ were tabulated by Govindarajulu (1966) for Type-II symmetrically censored samples. p. 28/52

82 Bias, MSE & JRE BLUEs of µ and σ were tabulated by Govindarajulu (1966) for Type-II symmetrically censored samples. The MLEs ˆµ and ˆσ are symmetric and skew-symmetric estimators, respectively. p. 28/52

83 Bias, MSE & JRE BLUEs of µ and σ were tabulated by Govindarajulu (1966) for Type-II symmetrically censored samples. The MLEs ˆµ and ˆσ are symmetric and skew-symmetric estimators, respectively. The same is true for the BLUEs µ and σ. p. 28/52

84 Bias, MSE & JRE BLUEs of µ and σ were tabulated by Govindarajulu (1966) for Type-II symmetrically censored samples. The MLEs ˆµ and ˆσ are symmetric and skew-symmetric estimators, respectively. The same is true for the BLUEs µ and σ. So, both pairs of estimators are uncorrelated. p. 28/52

85 Bias, MSE & JRE BLUEs of µ and σ were tabulated by Govindarajulu (1966) for Type-II symmetrically censored samples. The MLEs ˆµ and ˆσ are symmetric and skew-symmetric estimators, respectively. The same is true for the BLUEs µ and σ. So, both pairs of estimators are uncorrelated. While ˆµ, µ and σ are all unbiased, ˆσ is biased. p. 28/52

86 Bias, MSE & JRE BLUEs of µ and σ were tabulated by Govindarajulu (1966) for Type-II symmetrically censored samples. The MLEs ˆµ and ˆσ are symmetric and skew-symmetric estimators, respectively. The same is true for the BLUEs µ and σ. So, both pairs of estimators are uncorrelated. While ˆµ, µ and σ are all unbiased, ˆσ is biased. So, we may define Joint Relative Efficiency as JRE = 100 Var(µ )+Var(σ ) Var(ˆµ)+MSE(ˆσ). p. 28/52

87 Bias, MSE & JRE n r V(ˆµ)/σ 2 B(ˆσ)/σ MSE(ˆσ)/σ 2 JRE p. 29/52

88 Bias, MSE & JRE Bias in ˆσ decreases as n increases, for fixed r. p. 30/52

89 Bias, MSE & JRE Bias in ˆσ decreases as n increases, for fixed r. (ˆµ, ˆσ) is jointly more efficient than (µ, σ ) for small censored sample sizes considered here. p. 30/52

90 Bias, MSE & JRE Bias in ˆσ decreases as n increases, for fixed r. (ˆµ, ˆσ) is jointly more efficient than (µ, σ ) for small censored sample sizes considered here. Furthermore, unlike the BLUEs µ and σ, the MLEs ˆµ and ˆσ are explicit linear estimators. p. 30/52

91 Bias, MSE & JRE Bias in ˆσ decreases as n increases, for fixed r. (ˆµ, ˆσ) is jointly more efficient than (µ, σ ) for small censored sample sizes considered here. Furthermore, unlike the BLUEs µ and σ, the MLEs ˆµ and ˆσ are explicit linear estimators. JRE of the MLEs increases in general as the censoring proportion increases. p. 30/52

92 Outliers? No problem!! p. 31/52

93 Outlier-Model Let X 1:n < < X n:n denote the order statistics from a single-outlier Laplace model, with the outlier having a different scale parameter α. p. 32/52

94 Outlier-Model Let X 1:n < < X n:n denote the order statistics from a single-outlier Laplace model, with the outlier having a different scale parameter α. Let Y 1:n < < Y n:n denote the order statistics from a single-outlier exponential model, with the outlier having the same scale parameter α. p. 32/52

95 Outlier-Model Let X 1:n < < X n:n denote the order statistics from a single-outlier Laplace model, with the outlier having a different scale parameter α. Let Y 1:n < < Y n:n denote the order statistics from a single-outlier exponential model, with the outlier having the same scale parameter α. Let Z 1:n < < Z n:n denote the order statistics from the standard exponential distribution. p. 32/52

96 Outlier-Model Let X 1:n < < X n:n denote the order statistics from a single-outlier Laplace model, with the outlier having a different scale parameter α. Let Y 1:n < < Y n:n denote the order statistics from a single-outlier exponential model, with the outlier having the same scale parameter α. Let Z 1:n < < Z n:n denote the order statistics from the standard exponential distribution. Let (µ (k) i:n,µ i,j:n), (ν (k) i:n,ν i,j:n) and (ν (k) i:n,ν i,j:n) denote the single and product moments of these three sets of order statistics, respectively. p. 32/52

97 Outlier-Model Then, Balakrishnan (1989) established the following generalized relations between these three sets of moments of order statistics: p. 33/52

98 Outlier-Model Then, Balakrishnan (1989) established the following generalized relations between these three sets of moments of order statistics: µ (k) i:n = 1 2 n + { i 1 r=1 i 1 r=0 ( n 1 ) r 1 ( n 1 ) r ν (k) i r:n r +( 1)k ν (k) i r:n r +( 1)k n 1 r=i ( n 1 ) r n ( n 1 ) r=i r 1 ν (k) r i+1:r ν (k) r i+1:r } ; p. 33/52

99 Outlier-Model Then, Balakrishnan (1989) established the following generalized relations between these three sets of moments of order statistics: µ (k) i:n = µ (k) i,j:n = 1 2 n n { i 1 r=1 ( n 1 ) r 1 ν (k) i r:n r +( 1)k n 1 r=i ( n 1 ) i 1 ( n 1 ) n ( + ν (k) n 1 ) i r:n r r +( 1)k r 1 r=0 r=i i 1 ( n 1 n 1 )νi r,j r:n r + ( n 1 ) j 1 r=i i 1 r=0 r=1 r 1 r=j ( n 1 ) νj r:n r r 1 ν r i+1:r ( n 1 ) ν i r,j r:n r + r j 1 r=i r n ( n 1 ) r=j r 1 r ν (k) r i+1:r ν (k) r i+1:r } ν r j+1,r i+1:r ( n 1 ) νr i+1:r r ν j r:n r ν r j+1,r i+1:r ;. p. 33/52

100 Outlier-Model These results were used by Balakrishnan and Ambagaspitiya (1988) to study the robustness features of various linear estimators of both location and scale parameters of the Laplace distribution with respect to the presence of a scale-outlier in the sample. p. 34/52

101 Outlier-Model These results were used by Balakrishnan and Ambagaspitiya (1988) to study the robustness features of various linear estimators of both location and scale parameters of the Laplace distribution with respect to the presence of a scale-outlier in the sample. Of course, these developments can also be extended to the case of multiple outliers. p. 34/52

102 Outlier-Model These results were used by Balakrishnan and Ambagaspitiya (1988) to study the robustness features of various linear estimators of both location and scale parameters of the Laplace distribution with respect to the presence of a scale-outlier in the sample. Of course, these developments can also be extended to the case of multiple outliers. They compared various estimators of location and scale parameters in the presence of a single-outlier with its scale parameter as ασ, and these results are presented next for the case when n = 10. p. 34/52

103 Outlier-Model 1 σ 2 (Variance of ˆµ), when n = 10 ˆµ Mean Median BLUE(0) BLUE(1) BLUE(2) TrimMean(1) TrimMean(2) WinsMean(1) WinsMean(2) LinWMean(1) LinWMean(2) α GastMean p. 35/52

104 Outlier-Model 1 σ 2 (MSE of ˆσ), when n = 10 α ˆσ BLUE(0) BLUE(1) BLUE(2) RSE(0) RSE(1) RSE(2) RSE(3) p. 36/52

105 Exact Inference for Type-II Right Censoring p. 37/52

106 Exact Inference 1. Case r n : In this case, the joint MGF of (ˆµ, ˆσ) can 2 be derived as follows [Balakrishnan and Zhu (2016)]: p. 38/52

107 Exact Inference 1. Case r n : In this case, the joint MGF of (ˆµ, ˆσ) can 2 be derived as follows [Balakrishnan and Zhu (2016)]: E ( e t 1ˆµ+t 2ˆσ ) = r 1 j=0 + r 1 j n r l=0 l=0 p 1 e t 1µ (1 s 2 σ) j (1+s 2 σ) (r 1 j) { 1 (s 1 s 2 l)σ r(n r +1+l) } 1 p 2 e t 1µ (1 s 2 σ) (r 1) ( 1+ t 1σ l+r ) 1, p. 38/52

108 Exact Inference 1. Case r n : In this case, the joint MGF of (ˆµ, ˆσ) can 2 be derived as follows [Balakrishnan and Zhu (2016)]: E ( e t 1ˆµ+t 2ˆσ ) = r 1 j=0 + r 1 j n r l=0 l=0 p 1 e t 1µ (1 s 2 σ) j (1+s 2 σ) (r 1 j) { 1 (s 1 s 2 l)σ r(n r +1+l) } 1 p 2 e t 1µ (1 s 2 σ) (r 1) ( 1+ t 1σ l+r ) 1, where p. 38/52

109 Exact Inference p 1 p 2 = ( 1) l c jr 2 n r 1 j (n r +1+l) 1, l = ( 1) l c jr 2 r l n r (l+r) 1, l c jr = c jr = s 1 = n! j!(r 1 j)!(n r)!, n! (r 1)!(n r)!, ( r 1 ln n ) r 2r +1 t 1 + r 1 r t 2, s 2 = 1 r ln n 2r t r t 2. p. 39/52

110 Exact Inference p 1 p 2 = ( 1) l c jr 2 n r 1 j (n r +1+l) 1, l = ( 1) l c jr 2 r l n r (l+r) 1, l c jr = c jr = s 1 = n! j!(r 1 j)!(n r)!, n! (r 1)!(n r)!, ( r 1 ln n ) r 2r +1 t 1 + r 1 r t 2, s 2 = 1 r ln n 2r t r t 2. Analogous results for other cases have also been derived by Balakrishnan and Zhu (2016). p. 39/52

111 Exact Inference From the joint MGF of (ˆµ, ˆσ), the Bias and MSE of the estimators ˆµ and ˆσ can be easily obtained. p. 40/52

112 Exact Inference From the joint MGF of (ˆµ, ˆσ), the Bias and MSE of the estimators ˆµ and ˆσ can be easily obtained. Also, the correlation between the estimators ˆµ and ˆσ can be obtained. p. 40/52

113 Exact Inference From the joint MGF of (ˆµ, ˆσ), the Bias and MSE of the estimators ˆµ and ˆσ can be easily obtained. Also, the correlation between the estimators ˆµ and ˆσ can be obtained. From the joint MGF of (ˆµ, ˆσ), upon setting t 2 = 0 and t 1 = 0, respectively, the marginal MGFs of ˆµ and ˆσ can be deduced. p. 40/52

114 Exact Inference From the joint MGF of (ˆµ, ˆσ), the Bias and MSE of the estimators ˆµ and ˆσ can be easily obtained. Also, the correlation between the estimators ˆµ and ˆσ can be obtained. From the joint MGF of (ˆµ, ˆσ), upon setting t 2 = 0 and t 1 = 0, respectively, the marginal MGFs of ˆµ and ˆσ can be deduced. Upon inverting these marginal MGFs, the exact distributions of the estimators ˆµ and ˆσ can be derived. p. 40/52

115 Exact Inference From the joint MGF of (ˆµ, ˆσ), the Bias and MSE of the estimators ˆµ and ˆσ can be easily obtained. Also, the correlation between the estimators ˆµ and ˆσ can be obtained. From the joint MGF of (ˆµ, ˆσ), upon setting t 2 = 0 and t 1 = 0, respectively, the marginal MGFs of ˆµ and ˆσ can be deduced. Upon inverting these marginal MGFs, the exact distributions of the estimators ˆµ and ˆσ can be derived. In particular, it can be shown that p. 40/52

116 Exact Inference ˆµ ˆσ r 1 d = µ+ d = + r 1 j=0 + j=0 n r l=0 r 1 j l=0 n r l=0 r 1 j l=0 p 2 [Γ p 1 [Γ p 2 Γ p 1 [Γ ( j, bσ r ( r 1, bσ r ) )+Γ ( r 1 j, bσ r ( )] σ +E, l+r ) ( )] r +b(r l 1)σ +E r(n r+l+1) ( j, σ ) +Γ ( r 1 j, σ ) ( )] (r l 1)σ +E r r r(n r +l+1) ( r 1, σ ) ; r p. 41/52

117 Exact Inference ˆµ ˆσ r 1 d = µ+ d = + r 1 j=0 j=0 n r l=0 r 1 j l=0 r 1 j l=0 p 2 [Γ p 1 [Γ p 1 [Γ ( j, bσ r ( r 1, bσ r ) )+Γ ( r 1 j, bσ r ( )] σ +E, l+r ) ( )] r +b(r l 1)σ +E r(n r+l+1) ( j, σ ) +Γ ( r 1 j, σ ) ( )] (r l 1)σ +E r r r(n r +l+1) + n r l=0 p 2 Γ ( r 1, σ ) ; r in the above, Γ and E denote Gamma and Exponential random variables, Γ and E denote the negative of these variables, and Γ(0, σ r ), Γ (0, σ r ), E(0), E (0) are all degenerate at the point 0. p. 41/52

118 Example The data below, given by Mann and Fertig (1973), are the lifetimes of 13 aeroplane components with the last 3 components having been censored: p. 42/52

119 Example The data below, given by Mann and Fertig (1973), are the lifetimes of 13 aeroplane components with the last 3 components having been censored: p. 42/52

120 Example The data below, given by Mann and Fertig (1973), are the lifetimes of 13 aeroplane components with the last 3 components having been censored: Here, we analyze these data by assuming a Laplace distribution. p. 42/52

121 Example The data below, given by Mann and Fertig (1973), are the lifetimes of 13 aeroplane components with the last 3 components having been censored: Here, we analyze these data by assuming a Laplace distribution. We computed the MLEs based on this Type-II censored sample, their mean square errors (MSE) and correlation, and the 95% confidence intervals based on the exact formulae presented earlier. p. 42/52

122 Example MLEs of µ and σ and their MSEs and correlation coefficient r ˆµ ˆσ MSE(ˆµ) MSE(ˆσ) Corr(ˆµ,ˆσ) p. 43/52

123 Example MLEs of µ and σ and their MSEs and correlation coefficient r ˆµ ˆσ MSE(ˆµ) MSE(ˆσ) Corr(ˆµ,ˆσ) Exact and simulated 95% CIs for µ and σ Exact results Simulated results r µ σ µ σ 10 (0.7828, ) (0.4827, ) (0.7843, ) (0.4837, ) p. 43/52

124 How about Type-I Censoring? p. 44/52

125 Likelihood Function Let x 1:n < x 2:n < < x n:n denote the ordered lifetimes of n units under a life-test. p. 45/52

126 Likelihood Function Let x 1:n < x 2:n < < x n:n denote the ordered lifetimes of n units under a life-test. Suppose the life-test is terminated at a fixed time T. p. 45/52

127 Likelihood Function Let x 1:n < x 2:n < < x n:n denote the ordered lifetimes of n units under a life-test. Suppose the life-test is terminated at a fixed time T. Then, the data observed will be (x 1:n < < x D:n ), where D is the random number of failures until T. p. 45/52

128 Likelihood Function Let x 1:n < x 2:n < < x n:n denote the ordered lifetimes of n units under a life-test. Suppose the life-test is terminated at a fixed time T. Then, the data observed will be (x 1:n < < x D:n ), where D is the random number of failures until T. The corresponding likelihood function is d L = C d f(x i:n ){1 F(T)} n d, x 1:n < < x d:n < T. i=1 p. 45/52

129 Likelihood Function Let x 1:n < x 2:n < < x n:n denote the ordered lifetimes of n units under a life-test. Suppose the life-test is terminated at a fixed time T. Then, the data observed will be (x 1:n < < x D:n ), where D is the random number of failures until T. The corresponding likelihood function is d L = C d f(x i:n ){1 F(T)} n d, x 1:n < < x d:n < T. i=1 The MLEs of µ and σ exist only when D 1, and so all inferential results are based on this condition of at least one failure. p. 45/52

130 MLEs By maximizing L, the MLEs of (µ,σ) have been derived by Zhu and Balakrishnan (2016) as follows: p. 46/52

131 MLEs By maximizing L, the MLEs of (µ,σ) have been derived by Zhu and Balakrishnan (2016) as follows: ˆµ = ˆσ = [X m:n,x m+1:n ], n = 2m,d m+1; X m+1:n, n = 2m+1,d m+1; [X m:n,t], n = 2m,d = m; T + ˆσlog( n 2d ), d < n 2 ; [ 1 (n d)t + d d i=m+1 X i:n ] m i=1 X i:n, n = 2m,d m; [ 1 (n d)t + d d i=m+2 X i:n ] m i=1 X i:n, n = 2m+1,d m+1; 1 di=1 (T X d i:n ), d < n 2. p. 46/52

132 MLEs By maximizing L, the MLEs of (µ,σ) have been derived by Zhu and Balakrishnan (2016) as follows: ˆµ = ˆσ = [X m:n,x m+1:n ], n = 2m,d m+1; X m+1:n, n = 2m+1,d m+1; [X m:n,t], n = 2m,d = m; T + ˆσlog( n 2d ), d < n 2 ; [ 1 (n d)t + d d i=m+1 X i:n ] m i=1 X i:n, n = 2m,d m; [ 1 (n d)t + d d i=m+2 X i:n ] m i=1 X i:n, n = 2m+1,d m+1; 1 di=1 (T X d i:n ), d < n 2. They have then derived the exact conditional joint MGF, marginal MGF and marginal distributions of the MLEs ˆµ and ˆσ. p. 46/52

133 Example Let us again consider the data of Mann and Fertig (1973), but with the life-test terminating at time T = 2.75: p. 47/52

134 Example Let us again consider the data of Mann and Fertig (1973), but with the life-test terminating at time T = 2.75: p. 47/52

135 Example Let us again consider the data of Mann and Fertig (1973), but with the life-test terminating at time T = 2.75: Let us analyze these data by assuming a Laplace distribution. p. 47/52

136 Example Let us again consider the data of Mann and Fertig (1973), but with the life-test terminating at time T = 2.75: Let us analyze these data by assuming a Laplace distribution. We computed the MLEs based on this Type-I censored sample, their mean square errors (MSE) and covariance, and the 95% confidence intervals based on the exact results. p. 47/52

137 Example MLEs of µ and σ and their MSEs and covariance T ˆµ ˆσ MSE(ˆµ) MSE(ˆσ) Cov(ˆµ,ˆσ) p. 48/52

138 Example MLEs of µ and σ and their MSEs and covariance T ˆµ ˆσ MSE(ˆµ) MSE(ˆσ) Cov(ˆµ,ˆσ) Exact and simulated 90% CIs for µ and σ Exact results Simulated results T µ σ µ σ 2.75 (0.9228, ) (0.6053, ) (0.9188, ) (0.6066, ) p. 48/52

139 What else? p. 49/52

140 Other Work Results for Type-II symmetric censoring developed by Iliopoulos and Balakrishnan (2011) are simpler; p. 50/52

141 Other Work Results for Type-II symmetric censoring developed by Iliopoulos and Balakrishnan (2011) are simpler; The results on Type-II and Type-I censoring have been generalized to hybrid Type-II and Type-I censoring; p. 50/52

142 Other Work Results for Type-II symmetric censoring developed by Iliopoulos and Balakrishnan (2011) are simpler; The results on Type-II and Type-I censoring have been generalized to hybrid Type-II and Type-I censoring; From the joint MGF of the MLEs, exact inference has been developed for reliability, quantile and cumulative hazard functions, and likelihood predictors; p. 50/52

143 Other Work Results for Type-II symmetric censoring developed by Iliopoulos and Balakrishnan (2011) are simpler; The results on Type-II and Type-I censoring have been generalized to hybrid Type-II and Type-I censoring; From the joint MGF of the MLEs, exact inference has been developed for reliability, quantile and cumulative hazard functions, and likelihood predictors; Exact inference under progressive Type-II censoring based on BLUEs has been developed by Liu, Zhu and Balakrishnan (2018). p. 50/52

144 Other Work Results for Type-II symmetric censoring developed by Iliopoulos and Balakrishnan (2011) are simpler; The results on Type-II and Type-I censoring have been generalized to hybrid Type-II and Type-I censoring; From the joint MGF of the MLEs, exact inference has been developed for reliability, quantile and cumulative hazard functions, and likelihood predictors; Exact inference under progressive Type-II censoring based on BLUEs has been developed by Liu, Zhu and Balakrishnan (2018). Exact predictive likelihood inference has been developed by Zhu and Balakrishnan (2018). p. 50/52

145 References Balakrishnan, N. (1989). Ann. Inst. Stat. Math., 41. Balakrishnan, N. and Ambagaspitiya, R.S. (1988). Commun. Stat. - Theor. Meth., 17. Balakrishnan, N. and Cramer, E. (2014). The Art of Progressive Censoring, Birkhäuser. Balakrishnan, N. and Cutler, C.D. (1996). Festschrift Volume for H.A. David, Springer. Balakrishnan, N., Govindarajulu, Z. and Balasubramanian, K. (1993). Ann. Inst. Stat. Math., 17. Balakrishnan, N. and Zhu, X. (2016). J. Stat. Comp. Simul., 86. Govindarajulu, Z. (1963). Technometrics, 5. Govindarajulu, Z. (1966). J. Amer. Stat. Assoc., 61. Iliopoulos, G. and Balakrishnan, N. (2011). J. Stat. Plann. Inf., 141. Johnson, N., Kotz, S. and Balakrishnan, N. (1995). Continuous Univariate Distributions Vol. 2, Wiley. p. 51/52

146 References Kotz, S., Kozubowski, T.J. and Podgórski, K. (2001). The Laplace Distribution and Generalizations, Birkhäuser. Laplace, P.S. (1774). Mémories de mathématique et de physique presentés à l Academie royale des scienes, 6. Liu, K., Zhu, X. and Balakrishnan, N. (2018). Metrika, 81. Mann, N.R. and Fertig, K.W. (1973). Technometrics, 15. Stigler, S.M. (1975). Biometrika, 62. Zhu, X. and Balakrishnan, N. (2016). IEEE Trans. Reliab., 65. p. 52/52