Repairable Systems Reliability Trend Tests and Evaluation

Repairable Systems Reliability Trend Tests and Evaluation Peng Wang, Ph.D., United Technologies Research Center David W. Coit, Ph.D., Rutgers University Keywords: repairable system, reliability trend test, counting failure process, homogeneous Poisson process, nonhomogeneous Poisson process, renewal process SUMMARY & CONCLUSIONS Repairable systems reliability trend tests are reviewed, extensively tested and compared to evaluate their effectiveness over diverse data patterns. A repairable system is often modeled as a counting failure process. For a counting failure process, successive inter-arrival failure times will tend to become larger (smaller) for an improving (deteriorating) system. During testing and development of new systems, reliability trend analysis is needed to evaluate the progress of the design development and improvement process. Often a program of testing and modification, followed by more testing and modification, is required to achieve a desired system reliability goal. Reliability trend tests can be an important part of this program. The objective of system reliability trend tests is to determine whether and how the pattern of failures is significantly changing with time. This paper reviews the following four trend tests: (1) /AMSAA Test, (2) (pairwise comparison nonparametric test), (3) Laplace Test, and (4) Lewis-Robinson Test. These tests are extensively tested, evaluated and compared for diverse repairable system reliability trends. Particular emphasis focused on comparisons with low sample sizes. Simulation models for trend tests are presented and discussed; and simulation results are summarized and compared. Based on these comparisons, it is concluded that the /AMSAA test is the most robust trend test. 1. INTRODUCTION A repairable system is often modeled as a counting failure process. Analysis of repairable system reliability must consider the effects of successive repair actions. When there is no trend in the system failure data, the failure process can often be modeled as a renewal process where successive repair actions render the system to be in good as new condition. The two principal classes of systems where this is not appropriate is (1) reliability growth where design flaws are removed and the failure intensity is decreasing over time as the design evolves and improves, and (2) reliability deterioration when a system ages. 1

Reliability trend tests constitute a major tool during the system development or monitoring process. They are also very helpful when reliability evaluation is needed. They can be used to identify reliability growth by the observed data. Two general types of reliability trend tests for repairable systems, with null (H o ) and alternative (H a ) hypotheses, as follows: Trend Test 1. H o : Homogeneous Poisson Process (HPP) H a : Non-homogeneous Poisson Process (NHPP) Trend Test 2. H o : Renewal Process H a : Non-Renewal Process For systems undergoing reliability growth testing, it is critically important to identify whether significant improvement (i.e., real reliability growth) is occurring. System reliability growth can be detected by observing a significant trend of increasing successive time-between-failures, i.e., system failure inter-arrival times. For fielded systems, it is very important to detect when the system reliability is deteriorating. Decisions for preventive maintenance and over-haul require this information. System reliability deterioration can be detected by observing a significant trend of decreasing successive time-between-failures. A non-homogeneous Poisson process (NHPP) is capable of modeling these situations. If the failure intensity function, u(t), is decreasing over time, the times between failures tend to be longer, and if it is increasing, the times between failures tend to be shorter. Often in system early design phase, a formalized testing program is developed to identify design flaws and implement improvements. Reliability growth testing provides a systematic method to conduct developmental testing, to track the progress of reliability improvement efforts and to predict system reliability given the observed (or anticipated) rate of improvement. Reliability growth testing has been adopted by the many industries, including defense [1], automotive [2] and cellular telephone industries [3]. If a system in service can be repaired to "good as new" condition following each failure, then the failure process is called a renewal process. For renewal processes, the times between failures are independent and identically distributed. A special case of this is the Homogeneous Poisson Process (HPP) which has independent and exponential times between failures. Once reliability data has been collected, it is important to select an appropriate trend test. Power, 2

computational ease and simplicity of interpretation of trend test results are factors that should be considered in the decision of selecting an appropriate trend test. Another practical problem is that the available sample sizes are often small, making it critical to be able to evaluate the performance of trend tests for various and differing sample sizes and select the most robust one. There are many trend tests that will be effective when there is plentiful data. However, a truly useful trend test will continue to be effective for smaller sample sizes. This paper is composed of five sections. Section 2 provides formal definitions of HPP, NHPP and Renewal Process. Section 3 introduces four trend tests. In section 4, simulation models for trend tests are presented and discussed; simulation results are summarized and compared. Section 5 makes conclusions. Notation HPP Homogeneous Poisson Process NHPP Non-homogeneous Poisson process N(t) number of observed failures in (0, t] u(t) failure intensity (sometimes called "instantaneous failure rate") Λ(t) expected number of failures by time t λ, β model parameters (λ >0, β > 0) t development test time 2. HPP, NHPP AND RENEWAL PROCESS 2.1 Homogeneous Poisson Process (HPP) A counting process, N(t), is a homogenous Poisson process with parameter λ>0 if N(0)=0 the process has independent increments the number of failures in any interval of length t is distributed as a Poisson distribution with parameter λt There are several implications to this definition of Poisson process. First, the distribution of the number of events in (0, t] has the Poisson distribution with parameter λt. Second, the expected number of failures by time t, is Λ(t) = E[N(t)] = λt, where λ is often called the failure intensity or rate of occurrence of failures (ROCOF). Therefore, the probability that N(t) is a given integer n is expressed by n λt ( λt) e Pr{ N( t) = n} =, n = 0, 1, 2, (1) n! 3

The intensity function is u(t) = Λ (t) = λ. Therefore, if the inter-arrival times are independent and identically distributed exponential random variables, then N(t) corresponds to a Poisson process. 2.2 Non-homogeneous Process (NHPP) A counting process, N(t), is a nonhomogenous Poisson process if N(0) = 0 the process has independent increments the number of failures in any interval of length t is distributed as a Poisson distribution with parameter Λ(t) Pr{N(t+h) - N(t) = 1} = u(t) + o(h) Pr{N(t+h) - N(t) 2} = o(h) Λ(t) is the mean value function which describes the expected cumulative number of failures. u(t) is the failure intensity function. o(h) denotes a quantity which tends to zero for small h. Given u(t), the mean value function Λ(t)=E[N(t)] satisfies Λ( t) = t 0 u( s) ds (2) d Inversely, knowing Λ(t), the failure intensity at time t can be obtained as u(t)= Λ () t. As a general class of well-developed stochastic process models in reliability engineering, nonhomogeneous Poisson process models have been successfully used in studying hardware and software reliability problems. NHPP models are especially useful to describe failure processes which possess trends such as reliability growth or deterioration. The cumulative number of failures to time t, N(t), follows a Poisson distribution with parameter Λ(t). The probability that N(t) is a given integer n is expressed by Pr n [ Λ( t)] =, n = 0, 1, 2, (3) n! Λ( t) { N( t) n} = e Reliability growth testing, also known as Test-Analyze-and-Fix (TAAF) testing, involves the testing of a system early in the development cycle when the design is immature and design changes can be implemented more readily. At this point the design is still evolving and system reliability is improving as design changes are made in response to observed failures. Reliability growth testing has many advantages compared to reliability qualification and validation testing. One advantage is that it is not necessary to wait until all design efforts have been completed to initiate a reliability testing program. Another attractive feature of reliability growth testing is that the emphasis is on 4 dt

reliability improvement as opposed to reliability measurement. The concept of reliability growth testing was introduced by Duane [4] who was involved in developmental testing of aircraft engines. [5] further studied reliability growth and proposed that the improvement in reliability can be modeled by a NHPP. He formalized his findings with the popular /AMSAA model with corresponding maximum likelihood estimators for model parameters and goodness-of-fit tests. The /AMSAA reliability growth model is as follows: [ N( t ] = λ() t β E ) u( t) = λβ t β () 1 where N(t) = number of observed failures in (0, t] u(t) = failure intensity λ, β = model parameters (λ >0, β > 0) t = development test time For 0 < β < 1, failures during development testing occur as a NHPP with a decreasing failure intensity. When development testing is concluded at time t, subsequent failures in actual or field conditions at time τ, arrive in accordance with a HPP at a constant rate of u(τ;t) = λβ(t) β 1. In other words, failure intensity is decreasing during development testing and constant thereafter. This is logical because design fixes are being implemented to prevent or minimize the occurrences of observed failures during the testing program. Then, improvement ceases at the conclusion of testing because it is no longer practical or cost effective to incorporate design changes in response to each failure. Other reliability growth models have also been proposed by Lloyd [6], Robinson and Dietrich [7, 8], and [9 11]. 2.3 Renewal Process A renewal process is more general than the HPP for describing system failure processes where there is no trend. For renewal processes, times between failures can be distributed according to any lifetime distribution [12]. If observed system failure patterns follow a renewal process, then there is no reliability growth or deterioration. If the inter-arrival times are distributed as independent and identically distributed exponential random variables, then the renewal process is also a HPP. 3. TREND TESTS During testing and development of new systems, reliability trend analysis is needed to evaluate (4) (5) 5

the progress of the development process. Often a program of testing and modification followed by more testing is required in order to meet a pre-determined reliability specification [13]. Reliability trend test can be an important part of this program. A stochastic point process exhibits monotonic trend if FX ( x) > FX ( x) or FX ( x) < FX ( x) for every i 1, j > i and x > 0, where and are independent random variables [14]. If we assume that we have a sequence of independent inter-arrival times, then this is an improving process if (x) > (x) for every i 1, j > i and x > 0. Similarly, the process is deteriorating if FX i F X j X i X j the former inequality is reversed. This implies that successive inter-arrival failure times will tend to become larger (smaller) for an improving (deteriorating) system. The objective of system reliability trend tests is to determine whether the pattern of failures is significantly changing with time. This can be conducted by testing a null hypothesis that the system failure pattern is a renewal process. If this hypothesis can be rejected at some appropriate significance level, then, it can be concluded that some level of reliability improvement or deterioration is occurring. In practice, it can be difficult to test the renewal process null hypothesis. It is often more convenient to test a Poisson process (HPP) null hypothesis. The danger with this approach is that renewal processes with non-exponential failure inter-arrival times may lead to a rejection of the null hypothesis even when there is no trend. 3.1 Trend Tests This paper focus on the following four quantitative trend tests: /AMSAA Test (pair-wise comparison nonparametric test) Laplace Test Lewis-Robinson Test 3.1.1 /AMSAA Test The /AMSAA test [5] is based on the assumption that a failure intensity of u ( t) = λβt is appropriate. When β=1, the failure intensity reduces to u(t)=λ, which means the failure process follows a HPP. Then, the test involves whether an estimate of β is significantly different from 1. The hypothesis test is: H o : β =1 (HPP) i j i j β 1 H a : β 1 (NHPP) For one system on test, the maximum likelihood estimate (MLE) for β is, 6

β= ˆ N 1 i= 1 N ( T T ) ln / N where N = number of observed failures T i = i th failure arrival time i The test statistic is 2 Nβ. According to, it is distributed as a chi-squared random variable. β ˆ So considering the null hypothesis, the rejection criteria is given by: 2N Reject H 0 if <χ βˆ 2 2 N, 1 α/2 or 2N >χ βˆ 2 2 N, α /2 3.1.2 (pair-wise comparison nonparametric test) [14] The hypothesis test is: H o : renewal process H a : not a renewal process is nonparametric in nature because it does not require any assumption about the failure process. To conduct the test, compare all the inter-arrival times X j and X i for j > i, count the number of times that X j > X i. for j > i and define this number as U. Under H 0, the mean value of U is E( U ) = N( N 1) / 4 where N is the number of failures. The variance of U can be estimated as Var(U) = (2N+5)(N-1)N/72. The test statistic is U p = U N( N 1) / 4 (2N + 5)( N 1) N (7) 72 For large N, the test statistic is approximately distributed as a standard normal distribution according to the central limit theorem. Therefore, reject H o if U p > z α / 2 or U p < zα / 2. 3.1.3 Laplace Test [14] The hypothesis test is: H o : HPP H a : NHPP Under H o and conditioning on T N, T 1, T 2,, T N-1, are uniformly distributed on (0, T N ). The test statistic is (6) 7

by, N 1 TN Ti ( N 1) i= 1 2 U L = N 1 TN 12 where N= number of failures T i = i th failure arrival time The rejection criteria is based on a standard normal distribution assumption for U L. It is given Reject H 0 if U L > z or UL < z. α / 2 α / 2 The Laplace test corresponds to a Poisson process (HPP) null hypothesis. Therefore, there is the danger that H o is rejected when the underlying failure process is a non-exponential renewal process. 3.1.4 Lewis-Robinson Test [14] As mentioned in previous section, The Laplace test corresponds to a Poisson process null hypothesis. When the Laplace test is used, there is the danger that H o is rejected when the underlying failure process is a non-exponential renewal process. Lewis-Robinson (L-R) test is a modification of Laplace test that attempts to overcome this deficiency. The Lewis-Robinson test of renewal hypothesis uses the numerical values of inter-arrival times. The hypothesis test is: H o : renewal process H a : not a renewal process The Lewis-Robinson test statistics U LR is formed by dividing the Laplace test statistic U L by the coefficient of variation (CV) for the observed inter-arrival times. U L U = LR CV (9) where CV is the estimated coefficient of the variation of the inter-arrival times. CV can be calculated by Var ˆ [ X ] CV[ X ] = (10) X where X represent the variable of inter-arrival times. Reject H o if U LR > z α / 2 or U LR < zα / 2. 4. EVALUATION 4.1 Simulation Models To compare these four different trend tests, numerous simulations were conducted. Two NHPP 8 (8)

models and various renewal processes were assumed to be appropriate and data simulated. All four tests are applied to the simulated data to observe the result compared to the underlying known model. The probability of a Type I and Type II error was then empirically observed and tabulated. The simulation was run for a selected time T. Simulated data was generated from models with and without a trend. The two NHPP models are a /AMSAA and a linear model as follows. β 1 model: u( t) = λβt, 0 t T, λ, β > 0 Linear model: u(t)= λ + βt, 0 t T, λ, u(t) > 0 HPP models (no trend) were considered as a special case of both NHPP model. Additionally, other non-hpp renewal process models were used to simulate failure data. For the model, simulation was run 1,000 times for every combination of β = {0.5, 0.75, 1.0, 1.25, 1.5, 2, 3, 5}, E[N(t)] = {5, 10, 20, 30, 50, 70, 100}, and α = {.01,.1}. For the linear model, simulation was run 1,000 times for every combination of u(t)/u(0) = {0.5, 0.75, 1.0, 1.25, 1.5, 2, 3, 5} and E[N(t)] = {5, 10, 20, 30, 50, 70, 100}, and α = {.01,.1}. α is the test confidence level. For a sound test, α should approximate the probability of a Type I error when the null hypothesis is true. The /AMASS β parameter and the linear model u(t)/u(0) represent the respective model s degree of time dependence. (u(t)/u(0) = 1 + βt for linear model.) When β or u(t)/u(0) is less than one, u(t) is decreasing, and when β or u(t)/u(0) is greater than one, u(t) is increasing. As β or u(t)/u(0) increases, a sound test will consistently reject with a higher probability. When either β or u(t)/u(0) equals one, the model is a HPP. 4.2 Simulation Results The simulation results are presented numerically in Table 1 and 2 for the two NHPP cases. Also Figures 4-1 through 4-8 present several important cases. In the figures, the y-axis, labeled probability, is the observed proportion of simulations that the hypothesis test results in a fail-toreject decision. So, for any non-renewal model, it would be desirable for the probability (y-axis) to be low. For a renewal process, it would be desirable for the probability to be high. 9

Alpha=0.1,Beta=0.5 1.0 Probability 0.8 0.6 0.4 0.2 Laplace L-R 0.0 0 20 40 60 80 100 E[N(t)] Figure 4-1: AMSAA/ Model (β =.5) Probability Fail-to-Reject vs. E[N(t)] Alpha=0.1,Beta=2 1.0 Probability 0.8 0.6 0.4 0.2 0.0 0 20 40 60 80 100 LaPlace L-R E[N(t)] Figure 4-2: AMSAA/ Model (β = 2) Probability Fail-to-Reject vs. E[N(t)] Alpha=0.1,E[N(t)]=10 1.0 Probabili 0.8 0.6 0.4 0.2 0.0 0 1 2 3 4 5 Beta LaPlace L-R Figure 4-3: AMSAA/ Model Probability Fail-to-Reject vs. β; sparse data (E[N(t)]=10) Alpha=0.1,E[N(t)]=50 1.0 Probabilit 0.8 0.6 0.4 0.2 0.0 0 1 2 3 4 5 Beta LaPlace L-R Figure 4-4: AMSAA/ Model Probability Fail-to-Reject vs. β; plentiful data (E[N(t)]=50) 10

Alpha=0.1,u(T)/u(0)=0.5 Probability 1.0 0.8 0.6 0.4 0.2 0.0 0 50 100 E[N(t)] LapLace L-R Figure 4-5: Linear Model (u(t)/u(0) =.5) Probability Fail-to-Reject vs. E[N(t)] Alpha=0.1,u(T)/u(0)=5 1.0 Probability 0.8 0.6 0.4 0.2 0.0 0 50 100 E[N(t)] LapLace L-R Figure 4-6: Linear Model (u(t)/u(0) = 5) Probability Fail-to-Reject vs. E[N(t)] Alpha=0.1,E[N(t)]=10 1.00 Probability 0.80 0.60 0.40 0.20 0.00 0 1 2 3 4 5 u(t)/u(0) LapLace ModLapLace Figure 4-7: Linear Model Probability Fail-to-Reject vs. u(t)/u(0), sparse data (E[N(t)]=10) Alpha=0.1,E[N(t)]=50 1.00 Probability 0.80 0.60 0.40 0.20 LapLace ModLapLace 0.00 0 1 2 3 4 5 u(t)/u(0) Figure 4-8: Linear Model Probability Fail-to-Reject vs. u(t)/u(0), plentiful data (E[N(t)]=50) 11

Figures 4-1 and 4-5 correspond to decreasing u(t) while Figures 4-2 and 4-6 correspond to increasing u(t). For all four of these examples, an HPP or renewal process is not appropriate (except when β = 1 or u(t)/u(0) = 1) and a sound trend test will reject with a high probability. As would be expected, the performance improves as more data is available. Figures 4-3 and 4-7 correspond to cases where there is sparse data available to test the trend, while Figures 4-4 and 4-8 correspond to the case where there is significant data available. For these eight graphs, it is interesting to observe the test performance. A perfect test would have probability equal to 1-α when β or u(t)/u(0) is one, and low everywhere else. Of course, when β or u(t)/u(0) are close to one, it is not surprising that it is difficult to detect a trend, particularly, when data is sparse. Various renewal processes were also simulated. Renewal processes with Weibull inter-arrival times were selected for the simulation. These results are presented in Table 3. Figures 4-9 and 4-10 present sample graphical output. For these analyses and graphs, β represents the Weibull distribution shape parameter for the inter-arrival time distribution. Alpha=0.1, Beta=0.5 1.0 Probability 0.8 0.6 0.4 0.2 LapLace L-R 0.0 0 20 40 60 80 100 E[N(t)] Figure 4-9: Renewal Process Probability (β=.5) Fail-to-Reject vs. E[N(t)] Alpha=0.1, Beta=2 1.0 Probability 0.8 0.6 0.4 0.2 LapLace L-R 0.0 0 20 40 60 80 100 E[N(t)] Figure 4-10: Renewal Process Probability (β=2) Fail-to-Reject vs. E[N(t)] 4.3 Summary of Trend Test Comparisons When the /AMSAA model is the assumed model, as expected the /AMSAA test is consistently the most powerful test among the four tests (Figures 4-1 to 4-4). When β>1, Test is the least powerful, and the Laplace test is more powerful than L-R test. When β<1, there is 12

no observable difference in power between Laplace Test and L-R Test. When E[N(t)] is large, Laplace test has similar power as /AMSAA test. So in general, when β>1,, L-R, Laplace and /AMSAA test are in an increasing order of performance. When β is close to 1 (but not equal to 1), all tests are weak in power as expected. It is hardly surprising that the /AMSAA model has the best performance when it is the assumed model. It is still an important observation because it is the model most often believed to be appropriate. However, it is necessary to observe its performance when there is some other NHPP model or a non-exponential renewal process. For the linear model (Figures 4-5 to 4-8), when u(t)/u(0)<1, the /AMSAA Test is the least powerful among the four tests. However, when u(t)/u(0)>1, it is observed to be the most powerful, while Test is the least powerful. It is interesting to note that the performance for the L-R Test becomes very poor when u(t) is increasing rapidly. Specifically, when u(t)/u(0)<1, all tests are weak and /AMSAA is the weakest. But when u(t)/u(0)>1, it becomes the strongest while test becomes the weakest. L-R and Laplace test are similar in power. When E[N(t)] is large, /AMSAA, L-R and Laplace have similar power. Generally, when u(t)/u(0) is close to 1, no test performs good. When u(t)/u(0)>1,, L-R, Laplace and /AMSAA tests are in an increasing order of performance. For the renewal process, when the Weibull shape parameter (β) is less than one, the system hazard function is decreasing. For this case, the and L-R tests are superior. This is not too surprising because these are the two tests that are specifically used to test renewal rather than homogeneous Poisson processes. When the Weibull shape parameter (β) is greater than one, the system hazard function is increasing. For these cases, all tests perform quite well. This is an important result because many mechanical systems will perform as a renewal process in the early age with time dependent and increasing hazard function for inter-arrival times. Then, as these systems age, they are more likely to follow a NHPP with increasing u(t). 5. CONCLUSIONS The reliability growth philosophy is to begin reliability testing early in the design and development process, generally exposing the test item to environmental conditions simulating actual usage conditions. When failures are observed, failure analyses are conducted and the appropriate failure mechanisms and root causes are identified. Then, the design is revised or improved to prevent or minimize future occurrence of the same failure mechanisms. In practice, it is generally impossible to implement design changes in response to 100% of observed failures. However, the reliability growth philosophy suggests the investigation of all observed failures and 13

the implementation of fixes for those failure modes which demonstrate a failure rate that does not support attainment of the system reliability goal. In this paper, for important reliability trend tests are evaluated using simulated data. The comparison of these tests is enlightening. One interesting result is that the offers very little advantages over the other tests. This is surprising because, it is the most general test, and it has asymptotically sound properties. The /AMSAA test was observed to have the best performance or comparable performance in all cases but two: linearly decreasing u(t) and renewal processes with decreasing hazard functions. Overall, however, the /AMSAA test is the most robust. 14

Table 1: Trend Test Results (NHPP-Linear) Table 2: Trend Test Results (NHPP-) u(t)/u(0) E[N(T)] Laplace L-R Beta E[N(T)] Laplace L-R 0.5 5 0.87 0.83 0.84 0.80 0.5 5 0.64 0.71 0.62 0.72 0.5 10 0.90 0.81 0.82 0.78 0.5 10 0.43 0.38 0.42 0.35 0.5 20 0.84 0.71 0.76 0.74 0.5 20 0.12 0.16 0.18 0.12 0.5 30 0.79 0.69 0.67 0.67 0.5 30 0.03 0.05 0.07 0.05 0.5 50 0.73 0.62 0.59 0.59 0.5 50 0 0.01 0.01 0.01 0.5 70 0.66 0.55 0.51 0.51 0.5 70 0 0 0 0 0.5 100 0.59 0.47 0.39 0.40 0.5 100 0 0 0 0 0.75 5 0.88 0.87 0.86 0.84 0.75 5 0.84 0.86 0.87 0.85 0.75 10 0.89 0.83 0.84 0.82 0.75 10 0.81 0.82 0.8 0.8 0.75 20 0.88 0.78 0.81 0.77 0.75 20 0.67 0.71 0.68 0.71 0.75 30 0.86 0.74 0.78 0.76 0.75 30 0.61 0.63 0.7 0.68 0.75 50 0.88 0.83 0.84 0.84 0.75 50 0.4 0.43 0.47 0.49 0.75 70 0.86 0.80 0.83 0.83 0.75 70 0.22 0.3 0.3 0.33 0.75 100 0.84 0.79 0.81 0.81 0.75 100 0.09 0.2 0.16 0.2 1.25 5 0.91 0.90 0.90 0.88 1.25 5 0.86 0.95 0.92 0.9 1.25 10 0.89 0.87 0.89 0.85 1.25 10 0.83 0.93 0.89 0.87 1.25 20 0.85 0.86 0.87 0.85 1.25 20 0.77 0.91 0.85 0.87 1.25 30 0.85 0.86 0.88 0.86 1.25 30 0.7 0.89 0.8 0.8 1.25 50 0.90 0.95 0.92 0.92 1.25 50 0.56 0.87 0.72 0.74 1.25 70 0.88 0.95 0.91 0.92 1.25 70 0.42 0.8 0.6 0.6 1.25 100 0.86 0.96 0.89 0.90 1.25 100 0.29 0.68 0.44 0.45 1.5 5 0.86 0.92 0.90 0.88 1.5 5 0.78 0.94 0.92 0.89 1.5 10 0.85 0.89 0.89 0.86 1.5 10 0.69 0.93 0.83 0.83 1.5 20 0.81 0.88 0.85 0.86 1.5 20 0.46 0.85 0.63 0.69 1.5 30 0.82 0.87 0.86 0.85 1.5 30 0.29 0.72 0.45 0.49 1.5 50 0.79 0.91 0.84 0.85 1.5 50 0.1 0.57 0.23 0.26 1.5 70 0.76 0.87 0.80 0.81 1.5 70 0.03 0.39 0.1 0.12 1.5 100 0.75 0.88 0.76 0.79 1.5 100 0.01 0.24 0.04 0.04 2 5 0.86 0.93 0.91 0.90 2 5 0.54 0.91 0.82 0.85 2 10 0.82 0.91 0.89 0.88 2 10 0.38 0.82 0.6 0.62 2 20 0.75 0.88 0.83 0.84 2 20 0.1 0.69 0.25 0.33 2 30 0.73 0.86 0.79 0.80 2 30 0.01 0.46 0.07 0.1 2 50 0.64 0.82 0.68 0.69 2 50 0 0.25 0 0.01 2 70 0.56 0.74 0.58 0.59 2 70 0 0.11 0 0 2 100 0.41 0.60 0.40 0.41 2 100 0 0.03 0 0 3 5 0.79 0.93 0.91 0.89 3 5 0.22 0.85 0.55 0.71 3 10 0.73 0.92 0.85 0.86 3 10 0.05 0.69 0.17 0.32 3 20 0.63 0.85 0.72 0.74 3 20 0 0.4 0.01 0.04 3 30 0.52 0.79 0.62 0.63 3 30 0 0.22 0 0.01 3 50 0.39 0.66 0.42 0.43 3 50 0 0.06 0 0 3 70 0.23 0.48 0.24 0.27 3 70 0 0.03 0 0 3 100 0.14 0.31 0.12 0.12 3 100 0 0 0 0 5 5 0.74 0.93 0.91 0.89 5 5 0.03 0.76 0.21 0.54 5 10 0.62 0.90 0.77 0.80 5 10 0 0.59 0.02 0.16 5 20 0.46 0.80 0.59 0.63 5 20 0 0.31 0 0.01 5 30 0.30 0.67 0.40 0.42 5 30 0 0.1 0 0 5 50 0.13 0.44 0.17 0.19 5 50 0 0.02 0 0 5 70 0.10 0.28 0.09 0.09 5 70 0 0.01 0 0 5 100 0.01 0.12 0.01 0.01 5 100 0 0 0 0 15

Table 3: Renewal Process Trend Tests Weibull inter-arrival times Table 4: HPP Trend Test Results Shape, γ E[N(T)] Laplace L-R E[N(T)] Laplace L-R 0.5 5 0.55 0.88 0.43 0.81 5 0.91 0.85 0.87 0.86 0.5 10 0.56 0.89 0.48 0.79 Linear 10 0.89 0.79 0.85 0.80 0.5 20 0.53 0.88 0.5 0.79 Model 20 0.87 0.80 0.84 0.80 0.5 30 0.52 0.9 0.48 0.81 u(t)/u(0) 30 0.86 0.78 0.84 0.81 0.5 50 0.54 0.89 0.52 0.82 1 50 0.91 0.89 0.91 0.91 0.5 70 0.51 0.88 0.5 0.84 70 0.92 0.94 0.95 0.95 0.5 100 0.52 0.9 0.52 0.86 100 0.93 0.94 0.95 0.95 0.75 5 0.83 0.92 0.74 0.87 5 0.90 0.84 0.86 0.87 0.75 10 0.77 0.89 0.7 0.82 10 0.88 0.80 0.85 0.80 0.75 20 0.78 0.9 0.74 0.85 Model 20 0.87 0.79 0.83 0.81 0.75 30 0.77 0.89 0.75 0.87 Beta=1 30 0.84 0.76 0.83 0.79 0.75 50 0.78 0.9 0.72 0.87 50 0.91 0.88 0.89 0.88 0.75 70 0.79 0.91 0.77 0.88 70 0.91 0.92 0.93 0.94 0.75 100 0.79 0.9 0.77 0.88 100 0.94 0.95 0.95 0.96 1.25 5 0.95 0.89 0.93 0.86 5 0.90 0.89 0.84 0.86 1.25 10 0.95 0.89 0.94 0.87 10 0.89 0.90 0.87 0.86 1.25 20 0.96 0.91 0.96 0.89 Weibull 20 0.90 0.88 0.87 0.85 1.25 30 0.95 0.91 0.95 0.9 Model 30 0.90 0.90 0.87 0.87 1.25 50 0.96 0.9 0.96 0.89 Shape 50 0.91 0.89 0.87 0.87 1.25 70 0.94 0.89 0.94 0.87 γ=1 70 0.90 0.90 0.89 0.88 1.25 100 0.94 0.9 0.93 0.9 100 0.90 0.90 0.88 0.88 1.5 5 0.96 0.91 0.97 0.87 1.5 10 0.97 0.89 0.97 0.87 1.5 20 0.98 0.9 0.98 0.89 1.5 30 0.98 0.91 0.99 0.89 1.5 50 0.98 0.9 0.99 0.9 1.5 70 0.98 0.91 0.97 0.89 1.5 100 0.97 0.9 0.97 0.9 2 5 0.99 0.91 1 0.88 2 10 0.99 0.9 1 0.89 2 20 0.99 0.9 1 0.88 2 30 0.99 0.91 1 0.91 2 50 1 0.9 1 0.89 2 70 1 0.89 1 0.9 2 100 1 0.9 1 0.91 3 5 1 0.9 1 0.86 3 10 1 0.92 1 0.9 3 20 1 0.91 1 0.88 3 30 1 0.89 1 0.88 3 50 1 0.91 1 0.91 3 70 1 0.89 1 0.89 3 100 1 0.9 1 0.9 5 5 1 0.91 1 0.88 5 10 1 0.89 1 0.88 5 20 1 0.9 1 0.87 5 30 1 0.9 1 0.89 5 50 1 0.89 1 0.89 5 70 1 0.9 1 0.9 5 100 1 0.9 1 0.9 16

REFERENCES 1. Military Standard (1981), MIL-HDBK-189, Reliability Growth Management. 2. Freind, H. (1995) Reliability growth test planning, in Proceedings of the 41st Technical Meeting, Institute of Environmental Sciences, IES, pp. 104-110. 3. Bothwell, R., Donthamsetty, R., Kania, Z. and Wesoloski, R. (1996), Reliability evaluation: field experience from Motorola s cellular base transceiver systems, in Proceedings Annual Reliability and Maintainability Symposium, IEEE, pp. 348-359. 4. Duane, J. T. (1964), Learning curve approach to reliability monitoring, IEEE Transactions on Aerospace, 2, 563-566. 5., L. H. (1974) Reliability analysis for complex, repairable systems, SIAM Reliability and Biometry, 379-410. 6. Lloyd, D. K. (1986) Forecasting reliability growth, Quality and Reliability Engineering Journal, John Wiley & Sons, 2, 19-23. 7. Robinson, D. and Dietrich, D. (1987) A new nonparametric growth model, IEEE Transactions on Reliability, 36, 411-418. 8. Robinson, D. and Dietrich, D. (1988) A system-level reliability growth model, in Proceedings Annual Reliability and Maintainability Symposium, IEEE, pp. 243-247. 9., L. H. (1995) Reliability growth projections with applications to integrated testing, in Proceedings of the 41st Technical Meeting, Institute of Environmental Sciences, IES, pp. 93-103. 10., L. H. (1993) Confidence intervals on the reliability of repairable systems, in Proceedings Annual Reliability and Maintainability Symposium, IEEE, pp. 126-133. 11., L. H. (1990) Evaluating the reliability of repairable systems, in Proceedings Annual Reliability and Maintainability Symposium, IEEE, pp. 275-279. 12. Leemis, Lawrence M. "Reliability Probabilistic Models and Statistical Methods", Prentice-Hall International, Inc., 1995. 13. Dhillon, Balbir S. "Reliability Engineering in Systems Design and Operation", Van Nostrand Reinhold Company Inc., 1983. 14. Ascher, Harold and Harry Feingold, "Repairable Systems Reliability: Modeling, Inference, Misconceptions and Their Causes", Marcel Dekker, Inc., 1984. 15. Bain, Lee J. and Max Engelhardt, "Statistical Analysis of Reliability and Life-tesing Models: Theory and Methods", 2nd ed., Marcel Dekker, Inc., 1991. 16. Bain, Lee J. and Max Engelhardt, "Inference on the Parameters and Current System Reliability for a Time Truncated Weibull Process", Technometrics, Vol. 22, No. 3, pp 421-426, August 1980. 17., Larry H., "Confidence Interval Procedures for the Weibull Process with Application to Reliability Growth", Technometrics, Vol. 24, No. 1, pp 67-72, February 1980. 18. Miller, Grady W., "Confidence Interval for the Reliability of a Future System Configuration", AMSAA Technical Report, No. 343, pp 5-13, September 1981. 19. Tsokos, Chris P., Reliability Growth: Non-homogeneous Poisson Process, Recent Advances in Life Testing and Reliability (N. Balakrishnan, editor), CRD Press, 1995. 17

20. Campbell, C. L., Subsystem reliability growth allocation, in Proceedings of the 36th Technical Meeting, Institute of Environmental Sciences, IES, 1990, pp. 748-751. 21. Coit, David W. "Economic Allocation of Test Times for Subsystem-Level Reliability Growth Testing," IIE Transactions, vol. 30, no. 12, December 1998, pp. 1143-1151. 22. Cohen, Arthur, Sackrowitz, H. B. "Evaluating Tests for Increasing Intensity of a Poisson Process", Technometrics, vol. 35, no. 4, November 1993, pp446-448 23. Bain, L. J. Max E. and Wright, F. T. "Tests for an increasing Trend in the intensity of a Poisson Process: A Power Study", Journal of the American Statistical Association. vol. 80, no. 390, June 1985. 18