Non-stationary functional series modeling and analysis of hardware reliability series: a comparative study using rail vehicle interfailure times

Transcription

1 Reliability Engineering and System Safety 68 (2000) Non-stationary functional series modeling and analysis of hardware reliability series: a comparative study using rail vehicle interfailure times Ch.N. Stavropoulos, S.D. Fassois* Department of Mechanical and Aeronautical Engineering, University of Patras, GR Patras, Greece Received 2 October 1999; accepted 31 January 2000 Abstract A novel reliability modeling and analysis framework based upon the distinct class of non-stationary Functional Series (FS) models is introduced. This framework allows for non-stationary reliability modeling, evolution assessment, analysis (including non-stationarity assessment, dependency assessment, as well as cycle detection), and prediction. The Functional Series framework is used for the modeling and analysis of two rail vehicle reliability series (Times Between Failures, TBFs), while comparisons with alternative (ARIMA, adaptive RARMA RML, and Bayesian) modeling approaches are also made. The results indicate the advantages and usefulness of the Functional Series framework, as the TBF modeling accuracy is improved, its non-stationarity and serial dependency are established, the presence of cyclic patterns is revealed, and reliability evolution is assessed. It is conjectured that the cycles revealed in the TBF series may be related to maintenance policies. Finally, reliability prediction is shown to be feasible, although the larger excursions in the TBF series are difficult to accurately predict Published by Elsevier Science Ltd. All rights reserved. Keywords: Reliability; Repairable systems; Interfailure times; Non-stationary time series; Functional series models; ARIMA models; Adaptive models; Bayesian models 1. Introduction Time series models constitute mathematical representations, which are useful for describing the evolution of reliability measures in repairable systems. They have been pioneered by authors such as Singpurwalla [1,2], Walls and Bendell [3], and Soyer [4], and have been demonstrated to be capable of describing the stochastic dependencies present in reliability data series obtained from repairable systems. Time series models thus constitute useful tools for reliability modeling, evolution assessment, underlying data structure analysis, exogenous effect analysis, as well as prediction. The reliability of a repairable system is a expectedly deteriorating (for hardware systems) or improving (for software systems) stochastic function of time, although these trends may be reversed by human intervention, including maintenance related actions. Coupled with dependencies, which are inevitably present in reliability data series, such trends necessitate the use of non-stationary time series models for the proper representation of reliability data. * Corresponding author. Tel./fax: address: fassois@mech.upatras.gr (S.D. Fassois). Non-stationary time series models may be broadly used for the following practical purposes: (i) Gaining insight into the reliability evolution. Assessment and characterization of random effects and dependencies over time. Detection of the presence of cycles that may be associated with exogenous conditions. Detection of reliability trends. (ii) Prediction. Prediction of reliability evolution for a given system based upon retrospective data. In the study of the times between failures (TBFs), the time to the next failure is to be predicted. Prediction is important for planning purposes. (iii) Detection and assessment of exogenous effects. Detection and assessment of environmental effects (depending upon the type of hardware this may be weather effects, working conditions, and so on). Detection and assessment of scheduled maintenance effects. Appraisal of the effectiveness of maintenance policies. The present study focuses on the following specific issues which are of fundamental importance within the context of non-stationary time series reliability models: (a) Modeling. What model classes are appropriate? What /00/$ - see front matter 2000 Published by Elsevier Science Ltd. All rights reserved. PII: S (00)

2 170 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Nomenclature AIC Akaike Information Criterion AR AutoRegressive MA Moving Average ARMA AutoRegressive Moving Average ARIMA Integrated AutoRegressive Moving Average RARMA Recursive AutoRegressive Moving Average TARMA Time-dependent AutoRegressive Moving Average TAR Time-dependent AutoRegressive FS Functional Series P-A Polynomial-Algebraic PE Prediction Error RAT Reverse Arrangements Test RML Recursive Maximum Likelihood ROCOF Rate of OCcurrence Of Failures RSS Residual Sum of Squares SSS Series Sum of Squares S&S Singpurwalla and Soyer (model) TBF Time Between Failures is the required model structure? What is the achievable model quality? (b) Reliability evolution assessment. How to assess reliability evolution? How to detect the presence of trends? (c) Analysis. How to assess the degree and type of nonstationarity? How to formally assess the dependency structure? How to detect cyclic patterns that may be present? (d) Prediction. Is prediction possible? What is the achievable prediction accuracy? Parametric non-stationary time series models that may be potentially suitable in addressing the above issues include the class of Integrated AutoRegressive Moving Average (ARIMA) models of Box and Jenkins [5], the adaptive model class (of either the recursive model or the stochastic parameter evolution subclasses) [4,6], and the class of Functional Series models [7,8]. The ARIMA model class has been the first one used [1 3,9], but is known to be limited to a rather restrictive type of non-stationarity (referred to as homogeneous [5]), and has been thus criticized for failing to capture a sufficiently large portion of the series variability [3]. The adaptive model class is significantly wider, as adaptive models allow for the evolution (over time) of the model parameters. In a notable subclass (referred to as the recursive model subclass) parameter evolution is essentially unstructured ( unrestricted ), with the parameter estimates being updated via Kalman Filter type recursions every time a new data sample becomes available [6]. In an alternative subclass (referred to as stochastic parameter evolution model subclass), parameter evolution is more structured with the parameters being bound to obey certain stochastic smoothness constraints. With a single recent exception [10], the former subclass remains essentially unexplored in reliability data analysis, whereas the latter has been developed in a series of studies [4,11,12] focusing on software reliability and using a Bayesian framework which places increased emphasis on prior assumptions. The main goal of this paper is the introduction and assessment of a reliability modeling and analysis framework based upon the distinct class of non-stationary Functional Series (FS) models [7,8]. Functional Series models constitute conceptual extensions of conventional stationary models, such as the ARMA models, in which their parameters are allowed to evolve with time while remaining subject to certain functional constraints. The latter are typically enforced by requiring the model parameters to belong to a functional subspace spanned by a suitably selected set of functions (basis functions). Compared to adaptive models, Functional Series models are characterized by significantly more structured parameter evolution, but are flexible enough to capture a broad spectrum of non-stationarities while also leading to a degree of statistical parsimony significantly higher than that of adaptive models. An additional characteristic of Functional Series models is that, in contrast to their adaptive counterparts, they may be estimated via batch (non-recursive) schemes. The potential usefulness of a restricted (first-order with specific parameter form) Functional Series model in reliability analysis has been conjectured by Singh [13], but no further studies or actual applications have been reported. The Functional Series models used in this study are referred to as Time-dependent ARMA (TARMA) models, and are of the conventional ARMA form with parameters belonging to a proper functional subspace, while their estimation is based upon the recently introduced Polynomial- Algebraic (P-A) method [8]. The proposed Functional Series reliability modeling and analysis framework includes tools for: (a) reliability modeling; (b) evolution assessment; (c) model-based analysis; and (d) prediction. The focus of the study is on hardware reliability, and the fundamental issues (a) (d) posed earlier, by using the paradigm of rail vehicle reliability. The reliability data employed are chronologically ordered retrospective series of interfailure times (Times Between Failures, TBFs, with time expressed in km traveled) for vehicles of the Athens Electric Railways. Beyond the assessment of the Functional Series framework, the study has a significant comparative component in which the effectiveness of alternative modeling approaches is also assessed. These include the ARIMA, adaptive Recursive ARMA Recursive Maximum Likelihood (RARMA RML) [6], and the Bayesian stochastic parameter evolution approach of Singpurwalla and Soyer [11] (S&S). Model assessment and comparison are primarily based upon the ability of the various models to predict the time to next failure, although issues such as model

3 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Fig. 1. TBF series A: (a) Plot of the series; (b) Cumulative number of failures N(t) versus time (km traveled); (c) Non-parametric estimate of the ROCOF; (d) Sample correlogram and partial correlogram (horizontal lines indicate statistical significance at the a ˆ 0:05 level); (e) Laplace factor versus failure number (the horizontal lines indicate statistical significance at the a ˆ 0:05 level). parsimony are also considered through the Akaike Information Criterion (AIC) [6]. The rest of this paper is organized as follows. A description of the TBF series used in the study is, along with preliminary analysis, presented in Section 2. The Functional Series reliability modeling and analysis framework is presented in Section 3, while the alternative non-stationary time series reliability modeling approaches used are briefly reviewed in Section 4. Two cases of rail vehicle reliability modeling and analysis, using the Functional Series framework and the aforementioned alternative approaches, are presented in Section 5, and the conclusions of the study are summarized in Section Description of the reliability series and preliminary analysis Two retrospective series of Times (distance traveled in km) Between Failures (TBFs), each one corresponding to successive electrical failures incurred within the January 1995 to October 1998 period in a different vehicle of the Athens Electric Railways, are used. Series A consists of 74, and Series B of 71 TBFs (Figs. 1(a) and 2(a), respectively). The objective of the preliminary analysis is a first assessment of the system reliability evolution (including reliability trend detection) and the detection of the correlation (dependency) structure that may be present in the series Reliability estimators The cumulative number of failures N(t) versus time (km traveled), a rough non-parametric estimate of the function H t W E N t Š [14, p. 115], with E[ ] denoting statistical expectation, is presented in Figs. 1(b) and 2(b) for Series A and B, respectively. The almost linear form of the curve in the Series A case indicates potential stationarity, whereas the concave down form observed in the Series B case

4 172 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Fig. 2. TBF series B: (a) Plot of the series; (b) Cumulative number of failures N(t) versus time (km traveled); (c) Non-parametric estimate of the ROCOF; (d) Sample correlogram and partial correlogram (horizontal lines indicate statistical significance at the a ˆ 0:05 level); (e) Laplace factor versus failure number (the horizontal lines indicate statistical significance at the a ˆ 0:05 level). indicates potential reliability improvement [15]. A nonparametric estimate of the ROCOF (Rate of OCcurrence Of Failures) [14, p. 136] is presented in Figs. 1(c) and 2(c) for Series A and B, respectively. The estimate is obtained by dividing the total distance traveled (in km) by each vehicle in 10 equal subintervals (thus the number of failures per 40,938 and 39,861 km is depicted for Series A and B, respectively). The estimates obtained support the previous observations, since an approximately constant ROCOF (with the exception of the second subinterval) is observed for Series A, and an overall decreasing ROCOF is observed for Series B TBF series correlation and trend analysis The detection of the correlation (serial dependency) structure that may be present in each series is based upon the sample correlogram and the sample partial correlogram [5]. In the Series A case (Fig. 1(d)) the sample autocorrelation and partial autocorrelation are both statistically significant, at the a ˆ 0:05 level, at lag one. Furthermore, the cyclic structure appearing in the correlogram indicates the potential presence of a cycle in the TBF series. For its proper modeling, an autoregressive order of two may be thus necessary. In the Series B case (Fig. 2(d)) the sample autocorrelation and partial autocorrelation are statistically significant, at the a ˆ 0:05 level, at lags one and two. A cyclic structure, stronger than that of the previous case, appears in the Series B correlogram. Trend detection may be based upon the Reverse Arrangements Test (RAT) [15, pp ] and the Laplace-factor evolution [16]. In the Series A case no trend is detected by the RAT (at the a ˆ 0:05 level), but the Laplace factor evolution (Fig. 1(e)) indicates a dynamic, though overall stationary, behavior with an initial non-stationary (improvement) phase. In the Series B case a trend (reliability

5 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) {G 1 t ; G 2 t ; G p t } the model s AR/MA parameters are: Fig. 3. Pictorial representation of the Functional Series reliability modeling and analysis framework. f i t W Xp a i;j G j t 1 i n jˆ1 u i t W Xp c i;j G j t 1 i m jˆ Modeling and prediction 4 improvement) is detected by both the RAT and Laplace tests (Fig. 2(e)). The results of these tests appear to be in qualitative agreement with the previous remarks on the H(t) and ROCOF estimates. It should be nevertheless noted that these preliminary results are to be interpreted with caution. This is due to the fact that the correlogram and partial correlogram tests presuppose stationarity, while the RAT and Laplace tests independence. It is evident that both may be presently missing, so that alternative procedures that permit arrival at more concrete conclusions are necessary. This is the subject of the subsequent sections. 3. The functional series modeling and analysis framework The Functional Series framework postulates modeling via Time-dependent ARMA (TARMA) models. These are of the conventional ARMA form, but with parameters being explicit functions of time. These functions are restricted to a functional subspace spanned by orthogonal time-domain functions, which compose the model s functional basis. A TARMA(n,m) p model, with n, m indicating its AR and MA orders, respectively, and p the functional basis dimensionality, is thus of the form: A B; t X t ˆ C B; t W t t t o 1 with t indicating discrete time (presently failure number), t o the original time, X t the zero-mean non-stationary series modeled, and W t an innovations (uncorrelated) sequence with zero mean and possibly varying variance. A(B,t) and C(B,t) represent the time-dependent AutoRegressive (AR) and Moving Average (MA) polynomials, which are of the forms: A B; t W 1 f 1 t B f n t B n C B; t W 1 u 1 t B u m t B m with B indicating the backshift operator BX t W X t 1 ; and f n t 0; u m t 0; for some t t o : Expressed in terms of the basis functions 2 3 TARMA model estimation is based upon the modeling framework of Ben Mrad et al. [8], which accounts for parameter estimation, functional basis dimensionality, basis function, and model order selection. According to the cross-validation principle, a reliability data series is divided into two disjoint sets: The first, longer, set is used for model estimation (estimation set), whereas the latter is reserved for model validation purposes (validation set). Parameter estimation is achieved via the Polynomial-Algebraic (P-A) method, followed by Prediction Error (PE) refinement [8] (PE estimation is based on minimization of the model s onestep-ahead prediction error sequence using the Levenberg Marquadt optimization scheme). Model selection is based upon minimization of the Residual (prediction error) Sum of Squares (RSS) and the Akaike Information Criterion (AIC) [6, p. 419]. The final acceptance of the selected model depends upon successful validation, which is based upon the a posteriori examination of the zero-mean, stationarity, and uncorrelatedness hypotheses for the model residuals, as well as upon examination of the model s predictive performance within the validation set. One of the main goals achieved through TARMA modeling is the transformation of the serially correlated TBF series into an uncorrelated residual series (see Fig. 3). This allows for the proper and formal detection of reliability trends Reliability evolution The detection of reliability trends is closely coupled with model validation and is based upon the analysis of model residuals. In case that no trend is present in the TBF series, the residuals are to reflect that by forming a zero-mean, stationary, and independent (uncorrelated from a second order analysis viewpoint) series. This is to say that the inverted TARMA model generating them effectively transforms the autocorrelated TBF series into an uncorrelated one. In case the trends are present in the reliability series, they are carried on to the model residuals. Therefore, monotonic (as well as other) reliability trends may be detected by two types of tests: (a) a t-test [17] examining the zero-mean hypothesis on the model residuals, and, (b) the Reverse Arrangements Test (RAT) and the Laplace factor evolution test examining the lack of trend hypothesis on the residuals. The residual series uncorrelatedness, in the null case,

6 174 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) warrants that the RAT and Laplace tests are now properly applied Analysis: non-stationarity, dependency, and cycle detection Even in the absence of a trend, the TBF series may (and is generally expected to) be non-stationary. Within the proposed framework, non-stationarity assessment is based upon formal establishment of statistical significance (through a t-test) for the model s non-stationary coefficients of projection (that is the coefficients that introduce time-dependence in the model expression). Dependency (serial correlation) assessment is similarly based upon formal establishment of statistical significance for any one of the TARMA model coefficients of projection. The presence of cyclic patterns ( cycles ) in the series may be established based upon further model analysis, as cycles are related to the existence of complex conjugate pairs of time-varying frozen model poles. The undamped period T n (t) and normalized damping ( damping ratio ) z(t) of a cycle corresponding to the pole pair [l(t),l * (t)] (the asterisk indicating complex conjugation) may be obtained as: "! ln l t l 2 t Š T n t ˆ2p 2!! cos 1 l t l 2 # 5 1=2 t p 2 samples l t l t v ln l t l t Š 2 z t ˆ!! u ln l t l t Š 2 4 cos 1 l t l 2 t t p 2 l t l t and the cycle s observed (damped) period as: T d ˆ T n 1 z 2 1=2 samples 7 The cycle s importance in the data series is assessed by its contribution to the series variability via the (normalized) frozen dispersion: d t ˆE T t =E t % in which E T (t) represents the series variability associated with the cyclic component and E(t) the total series variability [18]. The estimated TARMA-based frozen timedependent spectrum of the series may be also obtained as: X m 2 u i t e jvi iˆ0 S v; t ˆ X n sw 2 9 f i t e jvi iˆ0 with p v representing frequency and j the imaginary number, 1 : Remark The presented Functional Series framework is pictorially depicted in Fig. 3. Modeling is based upon the (autocorrelated) reliability series. Analysis and prediction are based upon the obtained TARMA model. Reliability evolution (trend detection) is based upon the uncorrelated (in the null case) model residual series, which is obtained as the response of the inverted model to the original reliability series. The inverted TARMA model thus acts on the original series as a decorrelating filter, or, equivalently, as a transformation from serial dependence to independence. 4. Alternative modeling and analysis approaches In this section the Integrated ARMA (ARIMA), Recursive ARMA Recursive Maximum Likelihood (RARMA RML), and the Bayesian stochastic parameter evolution approach of Singpurwalla and Soyer [11] (S&S modeling) are briefly reviewed ARIMA modeling An ARIMA(n,d,m) model is of the form [5]: A B 1 B d X t ˆ C B W t 10 with d being a positive integer, the AR and MA polynomials of the forms (2), (3), respectively, but with time-invariant parameters, and W t a zero-mean and stationary innovations sequence. ARIMA estimation is based upon the conditional Maximum Likelihood method [5]. Model selection and validation, as well as reliability evolution assessment and analysis are presently accomplished via procedures similar to those presented in the previous section Adaptive RARMA RML modeling In the adaptive RARMA RML modeling Recursive ARMA (RARMA) models of a form similar to (1) but with AR/MA parameters characterized by unstructured time-dependency (that is they are allowed to vary freely with time) are used. At each time instant they are estimated by the Recursive Maximum Likelihood (RML) method [6, p. 314]. For an ARMA(n,m) model with parameter vector (lower-case/capital bold face symbols indicate vector/ matrix quantities, respectively): p W f 1 f n.. u 1 u m Š T the RARMA RML algorithm is: ^p t ˆ ^p t 1 k t X t ^X t t 1 k t ˆ P t 1 c t l c T t P t 1 c t

7 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) P t ˆ 1 ( l P t 1 P t 1 c t c ) T t P t 1 follows [11]: l c t T 14 P t 1 c t l y t N M t ; S t 21 ^C B; t 1 c t ˆf t f t W 15 " X t 1 X t n. # T ^W t 1 ^W t m 16 where: M t ˆ St 1Y t Y t 1 M t 1 r t S t 1 Y 2 t 1 r t r t ˆ s 2 2 Y 2 t 1 s 2 1 ; S t ˆ S t 1 r t S t 1 Yt 1 2 r ; t ^W t W X t f T t ^p t a posteriori error 17 with M 0 ˆ m and S 0 ˆ s 2 3 : The posterior distribution of u t given y t is: ^W t W X t ^X t t 1 ˆ x t f T t ^p t 1 prediction error 18 with ^X t t 1 indicating one-step-ahead prediction of the series made at time t 1, and l the forgetting factor. Model selection and validation, as well as reliability evolution assessment and analysis are presently accomplished via procedures similar to those previously presented Stochastic parameter evolution S&S modeling In the Singpurwalla and Soyer [11] (S&S) modeling each TBF series sample, X t, is assumed to be related to the previous sample, X t 1, through an expression of the form: X t ˆ X t 1 u t d t 19 with u t being a random, time-varying, exponent, and d t a multiplicative error series which is independent of u t. Assuming, without loss of generality, that X t 1 t ; a value of u t 1 indicates reliability growth, whereas a value of u t 1 indicates reliability deterioration. Assuming that X t is log-normallly distributed, the model (19) may be expressed as [11]: Y t ˆ u t Y t 1 e t with 20 e t N 0; s 2 1 ; s 2 1 known u t N l; s 2 2 ; s 2 2 known and l N m; s 2 3 ; m and s 2 3 known and Y t W lnx t ; e t W lnd t : It is useful to note that the conditional expectation E u t y t with y t ˆ Y 1 ; ; Y t ; gives information about reliability growth or decay from stage (t 1) to stage t, whereas E l y t gives information about the overall growth or decay in reliability. Thus values of E l y t greater than unity indicate reliability growth, whereas those smaller indicate reliability deterioration. The objective of the model is to make inference about l; u t ; Y t 1 given y t ˆ Y 1 ; ; Y t : The mean of the predictive distribution p Y t 1 y t is the point estimator of the next time to failure, and the main results may be summarized as u t y t N ^u t ; ^S t where: ^u t ˆ s 2 1 M t s 2 2 Y t Y t 1 r t ^S t ˆ s 2 1 s 2 1 S t s 2 2 r t r 2 t The predictive distribution of Y t y t 1 is: Y t y t 1 N M t 1 Y t 1 ; Y 2 t 1S t 1 r t : It should be noted that the S&S model is not dataoriented, as it is based upon the power law assumption (19) and first order dynamics. 5. Modeling and analysis of rail vehicle reliability In this section the non-stationary time series modeling and analysis of the two retrospective series of Times (km traveled) Between Failures (TBFs) (Series A and B) presented in Section 2 is examined based upon the Functional Series, ARIMA, RARMA RML and S&S approaches Modeling and analysis of TBF series A Following constant mean correction, the 74-sample-long Series A is divided into two disjoint sets: A 67-sample-long estimation set and a 7-sample-long validaton set TARMA modeling and analysis (a) Modeling and reliability trend detection. Two types of functional bases are examined: The first consists of sine/ cosine functions and the second of Chebyshev II polynomials. The sine/cosine functions are of the form: C i t ˆcos ip N t ; S i t ˆsin ip N t ; 1 t N with: C 0 t ˆ1; S 0 t ˆ1; 1 t N On the other hand, the Chebyshev II polynomials are defined over the discrete time interval [1,N] N ˆ 80 through the recursive expression: U i 1 t ˆ 4 t N 2 U i t U i 1 t 1 t N

8 176 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Table 1 Samples Per Parameter (SPP), Akaike Information Criterion (AIC), and RSS/SSS (estimation and validation sets) for selected models (Series A; minimal values in boldface) Model SPP AIC 10 3 RSS/SSS (%) with: Estimation U 1 t ˆ0; U 0 t ˆ1; 1 t N Validation TARMA(1,2) 2 Chebyshev ARIMA(3,1,3) RARMA(3,2) l ˆ 0: S&S TAR(2) 3 cosine TARMA(1,2) 2 cosine TARMA(2,1) 2 cosine Pure TAR, as well as TARMA, models of various orders and functional subspaces of dimensions ranging from p ˆ 2 to p ˆ 6; are fitted to the constant-mean-corrected TBF series A. The number of estimated parameters is maintained smaller than 7, in order to ensure a Samples Per Parameter (SPP) value greater than 10 within the estimation set. Candidate models are examined in terms of their achieved RSS/SSS (Residual Sum of Squares normalized by the mean-corrected Series Sum of Squares) and AIC values within the estimation set. Certain candidate TAR and TARMA models are shown in Table 1, from which a Chebyshev TARMA(1,2) 2 model, with the 0th and 39th Chebyshev II polynomials included in its functional basis, is selected as best. The estimated Chebyshev II TARMA(1,2) 2 model is: X t 0:393 U 0 t 1:035 U 39 t ŠX t 1 ˆ W t 0:173 U 0 t 1:376 U 39 t ŠW t 1 0:098 U 0 t 0:588 U 39 t ŠW t 2 It is characterized by minimum RSS/SSS and AIC values within the estimation set, although the other candidate TAR/ TARMA models are close to it (lower part of Table 1). The model residuals are presented in Fig. 4(a). Visual inspection indicates no trends or other patterns, while almost all residuals are within their statistical significance limits (at the a ˆ 0:05 level). In addition, no deviation from the zero-mean, stationarity, or uncorrelatedness hypotheses is observed. Residual zero-mean is formally confirmed via the one-sample t-test, stationarity (lack of trend) via the RAT and Laplace tests, and uncorrelatedness via the corresponding correlogram (which is insignificant at the a ˆ 0:05 level) (see Table 2 and Fig. 5(a)). The model is thus formally validated and accepted, while the presence of a monotonic trend in the series is rejected. The model s RSS/SSS within the validation set is at 90.21% (Table 1). (b) Analysis. The original TBF series non-stationarity is confirmed from the statistical significance of non-stationary coefficients of projection, while dependency (serial correlation) through that of several estimated coefficients. Furthermore, while the Chebyshev TARMA(1,2) 2 model has a real frozen pole, a TARMA(2,1) 2 (cosine functional basis) model which is very close to the former in terms of the various criteria used (see lower part of Table 1) has a pair of complex conjugate poles giving rise to a damped (damping ratio 0:8 z 1) cycle with period mainly Fig. 4. Model-based one-step-ahead prediction errors for Series A: (a) TARMA(1,2) 2 model; (b) ARIMA(3,1,3) model; (c) RARMA(3,2) model; (d) S&S model (the horizontal lines indicate statistical significance at the a ˆ 0:05 level).

9 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Table 2 RAT and t-test results using the model residual series (critical values at the a ˆ 0:05 level; Series A) Critical value TARMA(1,2) 2 ARIMA(3,1,3) RARMA(3,2) S&S RAT ^ t-test ^ varying in the interval T d ˆ 3 3:5 samples (Fig. 5(b)). A similar cycle is encountered in other alternative (such as the cosine TAR(2) 3 ) models. (c) Prediction. TARMA(1,2) 2 model based one-stepahead predictions are, along with the actual series, presented in Fig. 6(a) for part of the estimation and the validation sets. The predictions tend to follow the general pattern of the actual series, although not its larger excursions ARIMA modeling and analysis (a) Modeling and reliability trend detection. The modeling procedure leads to the ARIMA(3,1,3) model: X t 0:887X t 1 0:571X t 2 0:198X t 3 ˆ W t 0:289W t 1 0:161W t 2 0:937W t 3 characterized by minimum RSS/SSS and AIC values within the estimation set. The attained values are, nevertheless, significantly higher than those of the Chebyshev TARMA(1,2) 2 model (Table 1). It should be noted that attempted Box Cox type series transformations (logarithmic, square root, and others [3]) have led to decreased predictive accuracy and are thus not adopted. The model residuals are presented in Fig. 4(b), where no trends or deviations from the zero-mean, stationarity, and uncorrelatedness hypotheses are observed or formally (at the a ˆ 0:05 level) confirmed (Table 2). The model is thus validated and accepted, while the presence of a monotonic trend in the series is rejected. The model s RSS/SSS within the validation set is at % (Table 1), which indicates that the model has difficulty in predicting the TBF series within the validation set. (b) Analysis. The TBF series non-stationarity is confirmed from the first order differencing transformation required (pole at unity), while dependency through the statistical significance of the model parameters. Model analysis indicates a pair of complex conjugate poles, giving rise to a lightly damped z ˆ 0:04 cycle with period of T d ˆ 2:811 samples. This suggests the presence of a cycle in Series A, and is in agreement with the remark made on the series correlogram (Fig. 1(d)). (c) Prediction. ARIMA(3,1,3) model based one-stepahead predictions are, along with the actual series, presented in Fig. 6(b) for part of the estimation and the validation sets. The predictions tend to follow the general pattern of the actual series, but are, overall, less accurate in both the estimation and validation sets than the TARMA(1,2) 2 predictions (also see Table 1). Fig. 5. TBF Series A: (a) Laplace factor applied on the TARMA(1,2) 2 residuals (the horizontal lines indicate statistical significance at the a ˆ 0:05 level); (b) Damped period T d versus failure number (TARMA(2,1) 2 model); (c) E l y t versus failure number (S&S model).

10 178 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Fig. 6. Actual TBF Series A (-W-) and model-based one-step-ahead predictions (- -) (estimation and validation sets): (a) TARMA(1,2) 2 model; (b) ARIMA(3,1,3) model; (c) RARMA(3,2) model; (d) S&S model RARMA modeling and analysis (a) Modeling and reliability trend detection. RARMA models of various orders are estimated using a variety of forgetting factors [6]. The final estimates are obtained following a forward and a backward pass through the estimation set. A RARMA(3,2) model, estimated with a forgetting factor of l ˆ 0:995; is selected as best from both the RSS/SSS and AIC points of view. Nevertheless, the attained values are somewhat inferior than those of the ARIMA(3,1,3) model (Table 1). The model residuals are presented in Fig. 4(c), where no trends or deviations from the zero-mean, stationarity, and uncorrelatedness hypotheses are observed or are formally (at the a ˆ 0:05 level) confirmed (Table 2). The model is thus validated and accepted, while the presence of a monotonic trend in the series is rejected. The RSS/SSS within the validation set is at % (Table 1); this is mainly due to its inability to predict the large excursion of sample 69. (b) Analysis. Model analysis indicates that two of the model s poles form (at all times) a complex conjugate pair giving rise to a lightly damped (damping ratio 0:40 z 0:46 for most of the time) cycle with period varying in the interval T d ˆ 2:6 3 samples. (c) Prediction. RARMA(3,2) model based one-stepahead predictions are, along with the actual series, presented in Fig. 6(c) for part of the estimation and validation sets. The predictions tend to follow the general pattern of the actual series, but appear less accurate in both the estimation and validation sets than the TARMA(1,2) 2 predictions (also see Table 1) S&S modeling and analysis (a) Modeling. In this case the series A is assumed to obey a log-normal distribution, so that the transformed series Y t W lnx t is modeled as Gaussian. The distribution values are [11] s 2 1 ˆ 1:0; s 2 2 ˆ 1:0; s 2 3 ˆ 0:25; and m ˆ 1:0: The model predictions are exponentiated so that comparisons with the rest of the models can be made. It is evident that the model s RSS/SSS and AIC values are inferior to those of all previous models (Table 1). The model residuals (Fig. 4(d)) are evidently larger than those of the previous models, although no trends or deviations from the zero-mean, stationarity, and uncorrelatedness hypotheses may be observed or are formally confirmed (at the a ˆ 0:05 level) (Table 2). The model s RSS/SSS within the validation set is at %, which is also inferior to that of all previous models (Table 1). (b) Reliability trend detection. The conditional expectation E l y t which provides information on reliability growth or deterioration, indicates a very slight improvement in the system reliability (Fig. 5(c)). Although marginal, this improvement appears not to be in agreement with the estimates of Fig. 1(b) and (e). (c) Prediction. The model based one-step-ahead predictions are, along with the actual series, presented in Fig. 6(d) for part of the estimation and validation sets. The predictions tend to follow the general pattern of the actual series, but appear less accurate in both the estimation and validation sets than those of alternative models (also see Table 1) Modeling and analysis of the TBF series B Following constant mean correction, the 71-sample-long Series B is divided into two disjoint sets: A 63-sample-long estimation set and an 8-sample-long validaton set.

11 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Table 3 Samples Per Parameter (SPP), Akaike Information Criterion (AIC), and RSS/SSS (estimation and validation sets) for selected models (Series B; minimal values in boldface) Model SPP AIC 10 3 RSS/SSS (%) TARMA modeling and analysis (a) Modeling and reliability trend detection. Pure TAR, as well as TARMA, models of various orders, and functional subspaces spanned by Chebyshev II polynomials or sine/cosine functions and of dimensions ranging from p ˆ 2 to p ˆ 6; are fitted to the constant-mean-corrected TBF series B. Certain candidate models are (along with their RSS/SSS and AIC values) presented in Table 3, from which a Chebyshev TAR(3) 2 model, with the 0th and 28th polynomials included in its functional basis, is selected as best. The estimated Chebyshev TAR(3) 2 model is: X t 0:102 U 0 t 0:135 U 28 t ŠX t 1 0:205 U 0 t 0:305 U 28 t ŠX t 2 0:348 U 0 t 0:348 U 28 t ŠX t 3 ˆ W t Estimation Validation TAR(3) 2 Chebyshev ARIMA(5,1,1) RARMA(2,1) l ˆ 0: S&S TARMA(2,1) 2 Chebyshev TAR(3) 2 cosine TARMA(2,1) 2 cosine and is characterized by minimum RSS/SSS and AIC values within the estimation set, although the other candidate models are close to it (lower part of Table 3). The model residuals are presented in Fig. 7(a). Visual inspection of the residual series indicates no trends or other patterns, while almost all series samples are within their statistical significance limits (at the a ˆ 0:05 level). In addition, no deviation from the zero-mean, stationarity, or uncorrelatedness hypotheses is observed. Residual zero-mean is formally confirmed via the one-sample t-test, stationarity (lack of trend) via the RAT and Laplace tests, and uncorrelatedness via the corresponding correlogram test (which is insignificant at the a ˆ 0:05 level) (see Table 4 and Fig. 8(a)). The model is thus formally validated and accepted, while the presence of a monotonic trend in the series is rejected. The model s RSS/SSS within the validation set is as low as 50.49% (Table 3). (b) Analysis. The TBF series non-stationarity is confirmed from the statistical significance of non-stationary coefficients of projection, while dependency (serial correlation) through that of several estimated coefficients. Furthermore, model analysis indicates that two of the model s frozen poles are characterized by dominant dispersions and form (at almost all times) a complex conjugate pair giving rise to a damped cycle with period varying mostly within the interval T d ˆ 2 3 samples (Fig. 8(b)). The corresponding damping ratio remains smaller than 0.60 for most of the time. Such a cycle is present in the other candidate TAR and TARMA models as well, and is in agreement with the remark made on the correlogram of Fig. 2(d). (c) Prediction. TAR(3) 2 model based one-step-ahead Fig. 7. Model-based one-step-ahead prediction errors for Series B: (a) TAR(3) 2 model; (b) ARIMA(5,1,1) model; (c) RARMA(2,1) model; (d) S&S model (the horizontal lines indicate statistical significance at the a ˆ 0:05 level).

12 180 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Table 4 RAT and t-test results using the model residual series (critical values at the a ˆ 0:05 level; Series B) Critical value TAR(3) 2 ARIMA(5,1,1) RARMA(2,1) S&S RAT ^ t-test ^ predictions are, along with the actual series, presented in Fig. 9(a) for part of the estimation and validation sets. Like in the Series A case, the predictions tend to follow the general pattern of the actual series, although not its larger excursions ARIMA modeling and analysis (a) Modeling and reliability trend detection. The modeling procedure leads to the ARIMA(5,1,1) model: X t 0:200 X t 1 0:273 X t 2 0:115 X t 3 0:216 X t 4 0:110 X t 5 ˆ W t 0:959 W t 1 characterized by minimum RSS/SSS and AIC values within the estimation set. It is worth noting that the attained values are, in this case, somewhat lower than those of the TAR(3) 2 model (Table 3). The model residuals are presented in Fig. 7(b), where no trends or deviations from the zero-mean, stationarity, or uncorrelatedness hypotheses are observed or formally (at the a ˆ 0:05 level) confirmed (Table 4). The model is thus validated and accepted, while the presence of a monotonic trend in the series is rejected. It is interesting that the RSS/SSS within the validation set is (in contrast to its behavior within the estimation set) higher, at % (Table 3). Because of this the Chebyshev TAR(3) 2 model is considered superior. (b) Analysis. The TBF series non-stationarity is confirmed from the first order differencing transformation required (pole at unity), while dependency through the statistical significance of the model parameters. Model analysis indicates two pairs of complex conjugate poles, giving rise to damped z ˆ 0:192; 0:430 cycles with periods of T d ˆ 2:75; 10:25 samples, respectively. This result is in agreement with that of the TAR(3) 2 model and the nonparametric analysis (Section 2). (c) Prediction. ARIMA(5,1,1) model based one-stepahead predictions are, along with the actual series, presented in Fig. 9(b) for part of the estimation and validation sets. The predictions tend to follow the general pattern of the actual series, but not its larger excursions. The TAR(3) 2 model accuracy is clearly superior within the validation set RARMA modeling and analysis (a) Modeling, reliability trend detection, and analysis. The modeling procedure leads to a RARMA(2,1) model Fig. 8. TBF Series B: (a) Laplace factor applied on the TAR(3) 2 residuals (the horizontal lines indicate statistical significance at the a ˆ 0:05 level); (b) Damped period T d versus failure number (TAR(3) 2 model); (c) E l y t versus failure number (S&S model).

13 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) Fig. 9. Actual TBF Series B (-W-) and model-based one-step-ahead predictions (- -) (estimation and validation sets): (a) TAR(3) 2 model; (b) ARIMA(5,1,1) model; (c) RARMA(2,1) model; (d) S&S model. with forgetting factor l ˆ 0:995: The attained RSS/SSS and AIC values are inferior to those of the ARIMA(5,1,1) and TAR(3) 2 models (Table 3). The model residuals are presented in Fig. 7(c), where no trends or deviations from the zero-mean, stationarity, or uncorrelatedness hypotheses are observed. These are also formally confirmed (see Table 4) and the model is validated and accepted, while the presence of a monotonic trend in the series is rejected. The RSS/SSS within the validation set is at % (Table 3). Model analysis indicates no complex conjugate poles in this case. (b) Prediction. RARMA(2,1) model based one-stepahead predictions are, along with the actual series, presented in Fig. 9(c). The predictions are less accurate in both the estimation and validation sets than their TAR(3) 2 based counterparts S&S modeling and analysis (a) Modeling. The procedure reported in the Series A case is used. The resulting model s RSS/SSS and AIC values are inferior to those of all previous models (Table 3); this is interpreted as a result of the model s fixed structure. The model residuals (Fig. 7(d)) are evidently larger than those of the previous models, although no deviations from the zero mean, stationarity, or uncorrelatedness hypotheses are observed or are formally confirmed (at the a ˆ 0:05 level) (see Table 4). The model s RSS/SSS within the validation set is at %, which is inferior to those of all previous models (Table 3). (b) Reliability trend detection. The conditional expectation E l y t indicates a very slight improvement in the system reliability after the 30th failure (Fig. 8(c)). Although marginal, this appears to be in agreement with the indication provided by the non-parametric analysis (Fig. 2(b) and (e)). (c) Prediction. The model based one-step-ahead predictions are, along with the actual series, presented in Fig. 9(d), and appear less accurate than those of alternative models Discussion The following remarks may be made on the non-stationary time series modeling and analysis of the two TBF series: Modeling (a1) The Functional Series TAR and TARMA models proved appropriate for TBF series representation. 2-dimensional Chebyshev II functional subspaces were selected as best, although models with cosine based subspaces provided similar results. The TARMA(1,2) 2 (Series A) and TAR(3) 2 (Series B) models attained low (66.17% and 73.24%, respectively) RSS/SSS values within the estimation set, and were capable of explaining a significant portion (but not all) of the series variability. With the exception of the Series B case where the ARIMA(5,1,1) model achieved lower values, the Functional Series models systematically (for the various examined TBF series [19]) achieved optimal performance, that is lower RSS/SSS and AIC values than their ARIMA, RARMA RML, and S&S counterparts. The results thus indicated the suitability and improved fit of the Functional Series models to the non-stationary TBF series. (a2) The ARIMA, RARMA RML, and S&S models generally lagged, in terms of achievable performance, behind the Functional Series models in the order mentioned. The generally poorer performance of the S&S model is attributed to its restricted first order structure.

14 182 Ch.N. Stavropoulos, S.D. Fassois / Reliability Engineering and System Safety 68 (2000) (a3) With the obvious exception of the S&S model, various structures were investigated within the context of the examined models. Mixed autoregressive moving average representations with AR order of two or higher were generally found necessary in most cases. This is consistent with the non-parametric correlogram analysis of Section 2 (Figs. 1(d) and 2(d)), and is obviously related to the presence of cycles in the TBF series Reliability evolution assessment (b1) None of the TAR/TARMA, ARIMA, and RARMA RML based approaches supported (at the a ˆ 0:05 level) the presence of a monotonic trend in any of the two TBF series. The S&S based approach, on the other hand, indicated a very marginal reliability improvement Analysis (c1) The TBF series non-stationarity (also indicated by the preliminary analysis of Section 2) was confirmed through the statistical significance of the non-stationary parameters. The generally inferior fit of the ARIMA models over their Functional Series counterparts indicates that the non-stationarity is not of the homogeneous type. (c2) The presence of a dependency structure in the TBF series, which was indicated by the non-parametric analysis of Section 2 (see Figs. 1(d) and 2(d)), was formally confirmed through the fitted models. (c3) The presence of cycles in the two TBF series was also confirmed. It is quite interesting that the cycle observed in both series is characterized by a period in the neighborhood of 2.8 samples. The presence of such cycles in reliability data series has been noted by a number of researchers, and is generally attributed to exogenous causes such as weather factors or maintenance policies [3,13]. In the present case it is conjectured that the cycles present may be associated with vehicle aintenance policy. Support for this comes from the fact that the mean number of failures between successive maintenance actions is m ˆ 1:97 (standard deviation s ˆ 2:14) for TBF Series A, and m ˆ 2:13 (standard deviation s ˆ 1:68) for TBF Series B. Further research is nevertheless needed before this conjecture may be substantiated Prediction (d1) The presented results indicated that prediction of the time to next failure is possible, although the larger excursions in the series are difficult to accurately predict. 6. Concluding remarks In this paper a novel Functional Series reliability modeling and analysis framework was introduced. This framework allows for the non-stationary reliability modeling, evolution assessment (including trend detection), analysis (including non-stationarity assessment, dependency assessment, as well as cycle detection), and prediction. It thus enables the formulation of answers to the corresponding fundamental issues posed in the introduction (issues (a) (d)) concerning non-stationary time series reliability models. The Functional Series framework was used for the modeling and analysis of two rail vehicle reliability series (Times Between Failures), while comparisons with alternative non-stationary reliability modeling approaches (ARIMA, RARMA RML, S&S) were also made. The results of the study indicated the improved capabilities and advantages of the Functional Series framework. The main conclusions reached on each one of the fundamental issues examined may be summarized as follows: (a) Modeling. The improved accuracy of the Functional Series (TAR/TARMA) models in representing the reliability series was demonstrated. The alternative ARIMA, RARMA RML, and S&S representations consistently lagged behind their Functional Series counterparts in the indicated performance order. Relatively high model orders were necessary for good representations, and Chebyshev functional subspaces were found to be best within the Functional Series framework. (b) Reliability evolution assessment. The Functional Series models indicated no reliability trend at the a ˆ 0:05 level. A very slight reliability improvement was suggested by S&S modeling, but was not detected as significant by either the ARIMA or RARMA RML approaches. The Functional Series based non-stationary filtering scheme appeared useful in monitoring reliability evolution. (c) Analysis. The TBF series non-stationarity and dependency (autocorrelation) was clearly confirmed by the Functional Series framework. The non-stationarity appeared broader than the special homogeneous type implied by the ARIMA approach. Cyclic patterns, detected by the Functional Series framework and the ARIMA and RARMA RML approaches, may be related to maintenance policies. (d) Prediction. Functional Series based prediction of the time to next failure was found to be feasible and generally best among those of the considered approaches, although the larger excursions in the reliability series are difficult to accurately predict. Acknowledgements The authors are grateful to Mr. N. Skoulariotis and the engineers of the DEA Division of the Athens Electric Railways for providing the vehicle failure data and also for useful discussions. The financial support of the study by the General Secretariat for Research and Technology, Greece, through the YPER 95 Program (Project #441) is also acknowledged.