Enhanced Trajectory Based Similarity Prediction with Uncertainty Quantification

Transcription

1 Enhanced Trajectory Based Simiarity Prediction with Uncertainty Quantification Jack Lam 1, Shankar Sankararaman 2, and Bryan Stewart 3 1,3 Nava Surface Warfare Center Port Hueneme Division, Port Hueneme, CA, 93043, USA jack.am@navy.mi bryan.stewart2@navy.mi 2 SGT Inc., NASA Ames Research Center, Moffett Fied, CA 94035, USA shankar.sankararaman@nasa.gov ABSTRACT Today, data driven prognostics acquires historic data to generate degradation path and estimate the Remaining Usefu Life (RUL) of a system. A successfu methodoogy, Trajectory Simiarity Based Prediction (TSBP) that detais the process of predicting the system RUL and evauating the performance metrics of the estimate was proposed in Two essentia components of TSBP identified for potentia improvement incude 1) a distance or simiarity measure that is capabe of determining which degradation mode the testing data is most simiar to and 2) computation of uncertainty in the remaining usefu ife prediction, instead of a point estimate. In this paper, the Trajectory Based Simiarity Prediction approach is evauated to incude Simiarity Linear Regression (SLR) based on Pearson Correation and Dynamic Time Warping (DTW) for determining the degradation modes that are most simiar to the testing data. A computationa approach for uncertainty quantification is impemented using the principe of weighted kerne density estimation in order to quantify the uncertainty in the remaining usefu ife prediction. The revised approach is measured against the same dataset and performance metrics evauation method used in the origina TBSP approach. The resut is documented and discussed in the paper. Future research is expected to augment TSBP methodoogy with higher accuracy and stronger anticipation of uncertainty quantification. 1. INTRODUCTION Data driven prognostics acquires historic data to generate degradation path and estimate the Remaining Usefu Life (RUL) of a system. In 2008, a new approached known as Jack Lam et a. This is an open-access artice distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the origina author and source are credited. the Trajectory Simiarity Based Prediction (TSBP) methodoogy was proposed in (Wang T., 2013), and was successfuy demonstrated during the NASA AMES 2008 Prognostics Heath Management (PHM) chaenge by obtaining the highest score by using a data-driven prognostics method to predict the RUL of a turbofan engine (Saxena & Goebe, PHM08 Chaenge Data Description, 2008). Whie the TSBP is a proven technique, (Wang T., 2013) does not address imbaanced data (Gouriveau, Ramasso, & Zerhouni, 2013), the effectiveness of different dissimiarity measure (Giusti, 2013), and uncertainty of the mode (Daachiesa, Nushi, Miryenka, & Papanas, 2012). These considerations are required to minimize the variation that exists in the data driven prognostics method, and systematicay quantify the uncertainty in the RUL prediction. In (Wang T., 2013), the author deveoped a nove RUL prediction method based on the Instance Based Learning methodoogy caed TSBP. In TSBP, the historica instances of a system with ife-time condition data and known faiure time from the training data are used to create a ibrary of degradation modes; these modes are then compared against the testing data in order to compute a simiarity measure and predict an RUL corresponding to each of the degradation modes. The fina RUL estimate can be obtained by aggregating the mutipe RUL estimates using a density estimation method. Whie (Wang T., 2013) focused on the basic TSBP methodoogy, there are sti severa areas for improvement. For exampe, in (Yu, Yong, Datong, & Xiyuan, 2012), the authors investigated sensor seection as a critica research topic for prognostics. In their research, the authors stated that incusion of irreevant or redundant variabes during data fusion may ead to over-fitting or ess sensitivity of prognostics mode, which woud ead to adverse prediction performance.

2 In (Guo, Gerokostopouos, Liao, & Niu, 2013), the authors proposed to incorporate degradation initiation time into the genera degradation path modeing. Their paper argued that there is a degradation free period, i.e., degradation starts ony after an initiation time and that a product faiure is a combined effect of the initiation time and the degradation growth. In (Gouriveau, Ramasso, & Zerhouni, 2013), the authors suggested the need to dea with 1) data whose reative number of instances in each cass evoves with time and 2) data whose significance is not known by the user. In (Giusti, 2013)and (Otey & Parthasarathy, 2004), both authors examined the notion of quantifying the dissimiarity between different mutivariate time series. Their argument suggested that cacuating the Eucidean distance between the centroids of two data sets is ineffective because it ignores the correations present in the data sets. Finay, in (Daachiesa, Nushi, Miryenka, & Papanas, 2012), the authors summarized the uncertainty in time series and suggested two main approaches to mode these uncertain time series. Given a these factors, it can be easiy seen that the TSBP method proposed in (Wang T., 2013) can be reviewed and be improved. In (Lei & Govindaraju, 2004), authors proposed the use of Simpe Linear Regression (SLR) as a simiarity measure technique for on-ine signature recognition appications in comparison with the traditiona approach of computing the Eucidean Distance, whie having ower time compexity (O(n)) than Dynamic Time Warping (DTW) (O(n 2 )). The SLR method utiized the mean-deviation normaization to circumvent the probem of scaing and shifting, which, in genera, impacts the performance of the DTW method. Further, SLR can be adapted to muti-dimensiona sequences, where most rea-ife appications are reevant. In this paper, we examine the use of SLR and DTW within the TSBP method for simiarity prediction and address the various shortcomings of the origina TBSP approach that were expained in the previous paragraphs. Further, we test the resut on the origina dataset (Saxena & Goebe, 2008) and use the origina performance evauation metrics (Saxena, Ceaya, Saha, Saha, & Goebe, 2009) against the origina TBSP approach described by (Wang T., 2013). We aso compare the resuts using different density estimation approaches. The TSBP method with SLR and DTW as the simiarity measure with the use of the kerne density estimation provide us with more insight into the probem. The motivation for this work is to improve further the TSBP method by incorporating different simiarity measures and deveop a better understanding for uncertainty quaification. Athough more work is needed to compare the resuts of TBSP methodoogy against the state-of-the-art data driven technique used by the industry, our study produced a survey of reated areas that can be experimented to serve as an improved TSBP method. The target appication is highy compex systems where physica modeing wi be difficut and state of the operating condition can be observed. In this case, TSBP method can generate different degradation modes against each regime from the different operating condition to generate an aggregation of RUL estimation. Unike (Wang T., 2013), this paper 1) anticipates imbaanced data, 2) evauates the SLR and DTW simiarity measures, and 3) incorporates the uncertainty modeing done in (Daachiesa, Nushi, Miryenka, & Papanas, 2012). These capabiities further support the practica feasibiity of the proposed method used in rea appications. We envision more interest and study in the TBSP approach wi drive academic community and industry into maturing the methodoogy to provide more accurate RUL estimation. The rest of this paper is organized as foows. In Section 2, we review the muti-regime partitioning and normaization method used in (Wang T., 2013). In Section 3, we briefy review the techniques for degradation modeing expained in (Wang & Coit, 2007) and (Guo, Gerokostopouos, Liao, & Niu, 2013). In Section 4, we describe the simiarity/dissimiarity measure used in (Daachiesa, Nushi, Miryenka, & Papanas, 2012), (Yu, Yong, Datong, & Xiyuan, 2012), (Giusti, 2013), (Otey & Parthasarathy, 2004), and (Lei & Govindaraju, 2004). In Section 5, we describe uncertainty quantification in RUL estimation and review the density estimation methods. In Section 6, we incude the discussion of the performance metrics described in (Saxena, et a., 2008). In Section 7, we review the dataset (Saxena & Goebe, 2008) and describe the procedures for the experiment. In Section 8 and 9 we present resuts and findings then concude the paper in Section 10. Figure 1. Process for muti-regime heath assessment. 2

3 2. MULTI-REGIME PARTITIONING AND NORMALIZATION When a system is operating under mutipe operating conditions, the sensor measurements can behave differenty in those unique environments, thereby causing difficuty in identifying faiure trends. It is beneficia to identify the unique operating conditions or regimes from which sensors can be normaized or features can be extracted. Figure 1shows the high eve process for muti-regime heath assessment. To iustrate muti-regime partitioning, the Turbofan Engine Degradation simuation data set from (Saxena & Goebe, PHM08 Chaenge Data Description, 2008) wi be examined. Within this data set, there are 21 sensor measurements and three other measurements that describe the operationa conditions the system was operated under. The operating conditions change for each measurement (cyce). Figure 2 shows a seect number of sensor measurements for the ife time of one particuar system Regime Identification The first step in the process for muti-regime heath assessment is to identify the unique, non-overapping regimes. In this paper, mutipe regimes are found using k- means custering. The k-means custering agorithm finds the optimum number of custers, k, where each observation beongs to the nearest custer s mean, hence the name k- means. Figure 3 shows the resuts of the k-means custering agorithm on the Turbofan Engine Degradation simuation data set. As seen in Figure 3, the data was found to have 6 nicey separated and non-overapping regimes. (1) where p represents the regime the sensor measurement beongs at time instance i. y i = x i p μ p (1) σ p The mean-variance normaized data becomes the time series heath indices as depicted in Figure 1. In continuation of the iustration, the progression from Figure 2 to Figure 4 shows a more reveaing portraya of the system behavior once the operating conditions are taken into consideration. Figure 3. Muti-regime partitioning of the Turbofan Engine Degradation simuation data set. This figure represents a the operationa condition that was performed. Figure 2. sensor measurement from Turbofan Engine Degradation simuation data set Mean-Variance Normaization The next step is to normaize the sensor data according to the regime the measurement was taken under. This is done by performing mean-variance normaization. Simiar to Eq. Figure 4. Mean variance normaized data (bue ine) from the Turbofan Engine Degradation simuation data set for a singe system or unit. The red ine shows the degradation mode for each sensor. 3

4 2.3. Variabe Weighting and Dimensionaity Reduction At this point, the system data has been prepared and normaized for training the degradation modes. However, there are additiona techniques that can be used to further emphasize and refine the data to produce more accurate and timey resuts. Variabe/feature weighting is used to emphasis certain sensor measurements over other variabe/features and is often used in the feature seection process. In (Wang T., 2013), an Empirica Signa-to-Noise Ratio (esnr) is used for variabe reevance evauation. The esnr is defined as esnr(s i ) = var(s i) (2) var(s i ) where s i is a one dimensiona time series representing the features of the system evoving over time. Let s i be a smoothed version of s i fitered by a certain fitering or smoothing agorithm. The idea is that, in the event the goba variance (variance of the entire time series) is highy correated to the oca variance (variance within a shorter period of the time series), the smoothed time series wi have a much smaer variance compared to the origina. Therefore, the feature seection or emphasis can be performed from the ranking of the esnr. The feature weighting is y n = y n esnr(y n ) (3) where n represents the n th feature. This approach effectivey de-emphasizes the features with arge oca variance. Once the feature has been weighted, the next step is to uncorreated the features. In this case, (Wang T., 2013) suggests the use of Principa Component Anaysis (PCA). PCA is a common technique used to transform the features into a smaer set of uncorreated features. The uncorreated feature wi contain minimum redundancy and is important to combat the so-caed curse of dimensionaity. The method transforms the data into another coordinate system where the first coordinate or principa component (PC) represents the direction of the greatest variance of the origina data with the second, third, etc. PC represents decreasing variance of the origina data. The transformed features are cacuated as z = V M T (y y ) (4) Where y is the mean of y, and V M consist of the eigenvectors from the covariance matrix of y. The top M principa components that make up 90% of the tota variance are retained. The resutant PCs form a new time series z for each training and testing instance. An exampe of variabe weighting and dimensionaity reduction of the origina data can be seen in Figure 5. With the PCA competed, the origina data is now ready for Degradation Trajectory Abstraction. The data is Figure 5 show how the system is degrading through time with the red ine showing the degradation trajectory abstraction mode discussed in the foowing section. 3. DEGRADATION MODELING/REGRESSION The degradation modes are buit from the M Principa Components (PC) extracted from the normaized data as described in Section 2. These modes describe the PCs of z as a function of time t: G: z = g(t) + ε, 0 t ti (5) where ε is the noise term and in many cases is modeed as Gaussian. (Wang, 2010) Figure 5. Exampe Trajectory Abstraction mode from the Turbofan Engine Degradation simuation data set. The bue ine is the variabe weighting and dimensionaity reduction of the origina data, z. The red ine is the degradation trajectory abstraction modes. There are many parametric and non-parametric methods that can be used to buid the degradation modes, a of which shoud be considered based on their abiity to address the goba degradation pattern, short-period characteristics, amount of avaiabe data, data noise eve, and many other infuentia system characteristics. For this type of RUL estimation, ong-term degradation behavior and the operating setting of the system are important, whereas the oca fuctuations in the degradation trajectory can argey be considered noise. For these types of appications a smoothing operation of the time series such as a inear interpoation can be used. In (Wang, 2010), an exponentia curve fitting, moving average fiter and interpoation, Kerne regression smoothing, and reevance vector machines were expored. Based on the resuts found in (Wang, 2010) the kerne regression smoothing approach was used for degradation Trajectory Abstraction in this paper; see Eq. (6)-(7). 4

5 E z(t) = i=1 K G(t, t i ) z i E K G (t, t i ) i=1 (6) x y 2 K G (x, y) = exp 2ρ 2 (7) Where ρ is the kerne width and is a free parameter usuay chosen based on the data. An exampe output is shown in Figure 5 as the red ine. 4. REMAINING USEFUL LIFE ESTIMATION Once a the modes have been trained, the testing data wi need to be compared to every mode and a simiarity measure computed. The simiarity measure is used to determine which mode the system under test is most simiar too. This can be done by computing a distance or simiarity measure. In (Wang T., 2013), the Minimum Eucidean Distance with Degradation Acceeration (MED-DA), Minimum Eucidean Distance with Time Lag (MED-TL), and Minimum Eucidean Distance with Time Lag and Degradation Acceeration (MED-TL-DA) was proposed. It was found that the MED-DA performed the best on the CMAPSS dataset evauated. The remaining of this section, we briefy review MED-DA distance measure and provide an overview of two new simiarity/distance measures we propose in this paper: Pearson s Correation and Dynamic Time Warping Minimum Eucidean Distance with Degradation Acceeration In (Wang T., 2013), the Minimum Eucidean Distance with Degradation Acceeration (MED-DA) is the same as computing the Minimum Eucidean Distance between the training and testing modes except the MED-DA uses a scaing factor for time diation. This scaing factor is to accommodate the degradation rate differences between testing and training systems. I D 2 (λ) max (λ, 1 ) λ (z mi g(λ t i )) 2 I M i=1 m=1 2σ m 2 (8) where max(λ, 1/λ) is the pentaty term for the difference in degradation rate. The RUL prediction using this distance measure is cacuated as: t E r I = arg min D 2 (λ) t I (9) λ Additionay, in (Wang T., 2013) it was assumed that the most recent cyces provided more vaue to the simiarity measure than the earier cyces. Therefore (Wang T., 2013) used a non-uniform weighting scheme to emphasis the most recent cyces of the system under test. Eq. (8) then becomes I 2 max (λ, 1 D ) DA (λ) λ I I υ i (z mi g(λ t i )) 2 2σ2 m i=1 υ i i=1 M m=1 where υ i is the non-uniform weighting of each cyce i. (10) υ i = exp (t i t) 2 2ρ 2 (11) ρ = γ r E The non-uniform weighting is controed by the spread parameter which is a percentage of the ife r E of the degradation mode G and is controed by the spread ratioγ. In (Wang T., 2013), through cross-vauation, a spread parameter of 0.3 was found to produce the best resuts. Since MED-DA is a squared distance measure, a simiarity measure is computed as foows: S DA = exp ( 2 D DA ) (12) In (Wang T., 2013), the best reported performance score on the evauation set was This score is based the optimum vaues for the kerne width parameter ρ used for the kerne regression smoothing and spread ratio γ used in the MED-DA simiarity evauation. The optimum parameters were found by a 5-fod cross-vaidation of the training set where ρ = 7 and γ = Simiarity based on Pearson s correation In (Lei & Govindaraju, 2004), a simpe inear regression was used to assess the strength of a inear reationship between sequences X = (x 1, x 2,, x n ) and Y = (y 1, y 2,, y n ). A goodness-of-fit measures ca R 2 was used and is defined as: R 2 = n i=1 u i 2 n(y i Y) i (13) where u is the error term and Y is the mean of Y. R 2 is aso caed the coefficient of determination. It is interpreted as the fraction of the variation in Y that is expained by X. After further evauation it is found that R 2 is exacty the square of Pearson s correation (Lei & Govindaraju, 2004). n S SLR i (x i X)(y i Y) = r = 2 n(x i X) (yi Y) 2 i n i (14) As r approaches 1, the inear reation between the two sequences becomes stronger. Therefore the Pearson s correation of X and Y wi have simiarity r. The RUL prediction using this simiarity measure is a direct cacuation between the test system and the mode with the highest Pearson s correation. r I = t E t I (15) 5

6 4.3. Simiarity based on Dynamic Time Warping Dynamic Time Warping (DTW) is an aternative approach to determine the distance between two time-series signas where the two tempora sequences may vary in time or speed. It attempts to match two time series by stretching and contracting subsequences of the series so the difference between the series is minimized. (Giusti, 2013) The distance is then measured as the square root of the sum of the differences between the matched observations. Technicay, DTW (Savador & Chan, 2007) constructs a warp path between the two time series. A dynamic programming approach is first used to find the warp path and create a cost matrix. A singe point in the origina time series can be warped to mutipe points in the comparing time series. Every ce of the cost matrix is fied and the minimum-distance warp path can be evauated by reversey foowing the smaest cost of each move unti the origina point is reached. If both series were identica, the warp path through the matrix woud aong the diagona. DTW can aso adapt a constrained version by incorporating a window size parameter. This parameter imits the number of observations a matching can occur ahead or behind any given observation. It is noted in (Giusti, 2013) that the constrained version may sometimes improve the cassification accuracy by avoiding pathoogica warping. The RUL prediction using this simiarity measure is a direct cacuation between the test system and the mode with the highest Pearson s correation. r I = t E t I (16) Since DTW is a squared distance measure, a simiarity measure is computed as foows: S DTW 2 = exp ( D DTW ) (17) 4.4. Mode Aggregation A RUL estimates and simiarity scores are used to form a hypothesis set and the goa of mode aggregation is to use mutipe estimates in the hypothesis set and sum them up to create a fina prediction. The simpest method of aggregation is to use the simiarity-weighted sum, which provides a Point Estimate of the RUL. r I L =1 L =1 S I S I r I (18) This approach is inadequate for uncertainty management in prognostics. A probabiity distribution or confidence interva for the predicted RUL is desired in order to aid riskinformed decision-making in the context of prognostics and heath management. (Wang, 2010) 5. UNCERTAINTY QUANTIFICATION IN RUL PREDICTION The computation of uncertainty in the remaining usefu ife prediction is an important, essentia, and chaenging issue. Since prognostics deas with the prediction of the future behavior of engineering systems, it is necessary to understand that it is amost impossibe to make predictions regarding the future. That is why it is important to quantify the various sources of uncertainty in prognostics and quantify their combined effect on the remaining usefu ife prediction. Some recent research efforts in (Sankararaman, Daige, & Goebe, 2014) and (Sankararaman & Goebe, 2013) have been focusing on the topic of quantifying the uncertainty in prognostics and the remaining usefu ife prediction. At any given instant of time at which prediction needs to be performed, the uncertainty in the RUL prediction depends on three important factors: Heath state estimate at the time of prediction (initia state) Future operating and oading conditions Degradation mode that predicts heath state degradation from the initia state, based on the future operating and oading conditions It has been demonstrated that the computation of the uncertainty in the RUL, based on the uncertainty in the above quantities is a non-trivia probem and needs to be soved using statistica methods (Sankararaman, 2014). In this context, the goa is to cacuate the probabiity distribution of the remaining usefu ife prediction continuousy as a function of time; note that this probabiity distribution varies as a function of time and therefore, needs to be recacuated at every time instant. This probabiity distribution needs to systematicay account for the different sources of uncertainty in the aforementioned ist of quantities and quantify their combined effect on prognostics and remaining usefu ife prediction. Most of the previous efforts have focused on such uncertainty quantification ony in the context of modebased prognostics where physics-based modes are used to represent heath state degradation. Uncertainty quantification and management in the context of data-driven prognostics has not been studied in the detai, and since, different types of data-driven techniques have been used by severa researchers, the interpretation, quantification, and management of uncertainty may be different for different data-driven approaches. Hence, uncertainty quantification needs to be discussed in the context of the data-driven approach being pursued, and hence, this paper focuses ony on uncertainty quantification in the TBSP approach. 6

7 5.1. Uncertainty in Simiarity-Based Prediction Technique In the context of simiarity-based prediction, it is first essentia to understand the importance of uncertainty quantification. In this methodoogy, the focus is on finding out the simiarity between the desired testing data set and the entire training data set. The remaining usefu ife of the testing data set can be predicted through some sort of meaningfu interpoation in the domain of the training data set, where the interpoation procedure attempts to identify where the testing data set ies, with respect to the training data set. An important underying assumption here is that, at any point of prediction, the future operating conditions and oading conditions in the testing data set can aso be interpoated based on that of the training data set; in many practica appications, this assumption may be incorrect and therefore, this method may not be appicabe. Therefore, if there is exact simiarity between a testing data set and a particuar training data set, then there is no uncertainty regarding the prediction of remaining usefu ife. This is because the remaining usefu ife of the desired testing data set is equa to the remaining usefu ife of the corresponding training data set. This can be easiy expained by understanding data-driven earning agorithms such as Gaussian process earning where the variance of the prediction at any training point is exacty equa to zero. Therefore, if the testing point is identica to a training point, the variance of the prediction is zero and hence, there is no uncertainty regarding the remaining usefu ife. (Note that, the simiarity-based comparison is performed ony unti the time of prediction. There may be significant differences between the testing set and the training set after the time of prediction; such differences ead to uncertainty in the remaining usefu ife prediction but cannot be quantified without knowedge regarding the future operating/oading conditions of the testing data set.) Typicay, the testing data set may be significanty different from the training data set, and the TBSP approach computes a simiarity between the training and testing data set. This simiarity measure is simpy refective of the probabiistic weightage that is given to each of the remaining usefu ife vaues of the training data set. Therefore, Eq. (18) impies that the remaining usefu ife is cacuated ony using a weighted averaging approach, and therefore, is refective ony of the mean behavior. Other statistics of the remaining usefu ife prediction can aso be cacuated. For exampe, the standard deviation can be cacuated as: L σ r = =1 S I( ri r I ) 2 L L L 1 =1 S I (19) where L' denotes the number of non-zero simiarity measures. Note that the weighted mean and weighted standard deviation are centra measures. Whie such centra measures are important, they do not sufficienty capture the information regarding the uncertainty in the remaining usefu ife prediction. In order to achieve this goa, it is necessary to cacuate the entire probabiity distribution (either in terms of the probabiity density function or in terms of the cumuative distribution function). This cacuation is faciitated through the use of kerne density estimation, as expained ater in this section Uncertainty Quantification through Maximum Likeihood Estimation In (Fonseca, Friswe, Mottershead, Lees, & Adhikari, 2005), the authors describe that the key to the maximum ikeihood (ML) approach is to parameterize the probabiity density functions (PDFs) of the parameters. The uncertainty quantification incudes cacuating the probabiity that the measurements occur given the PDF of the parameters. Suppose that the physica parameters, x, foow a certain probabiity distribution beonging to a probabiity distribution famiy parameterized by Ѳ (for exampe the mean, μ, and covariance matrix, Σ). For a given Ѳ, the output PDF, f(x Ѳ), can be approximated using the uncertainty propagation method. Let the measurements be x 1, x 2,, x N. The measurements are assumed to be independent, therefore the measurements ikeihood is L(Ѳ) = f(x 1, x 2,, x N Ѳ) = f(x i Ѳ) N i=1 (20) The maximum ikeihood estimator is vaue of Ѳ that corresponds to the maximum of L(Ѳ). Note that the maximum ikeihood estimate is aso a centra measure. Two important changes need to be made in order to adapt this methodoogy for the purpose of uncertainty quantification in TBSP. First, it is necessary to infer information regarding the uncertainty; such uncertainty can be expressed either in terms of the PDF f(x) or in terms of confidence intervas. Secondy, and more importanty, the PDF f(x Ѳ) corresponds a parametric probabiity distribution (with parameters Ѳ), and such a distribution may not be avaiabe. So, it may be necessary to use non-parametric distribution and directy estimate the PDF f(x) without empoying the use parameters Ѳ. In this paper, both of these goas are accompished through the use of a weighted kerne density function that is not ony parametric but aso can directy compute confidence intervas on the quantity of interest, x in this case Uncertainty Quantification through Kerne Density Estimation A non-parametric approach for mode aggregation is used which is caed Kerne Density Estimation or KDE using a 7

8 Parzen window method. (Wang T., 2013) The kerne density approximation is given by: n f (x) ĥ = 1 n 1 h K x x i h i=1 (21) where K is the Gaussian kerne function and h is the bandwidth for density estimation. The Gaussian kerne function is defined as: K(u) = 1 2 π exp u2 (22) 2 In (Wang T., 2013) and in this paper, the KDE method via diffusion with automatic bandwidth seection as proposed in (Botev, Grotowski, & Kroese, 2010) was used. (cyces) progresses the most simiar degradation modes can be readiy observed. The pot in Figure 7 shows the density estimation of the RUL prediction at each cyce based on the SLR weighted KDE mode aggregation. 6. PERFORMANCE METRICS The evauation of the proposed enhancements to TBSP wi be based on the work in (Saxena, Ceaya, Saha, Saha, & Goebe, 2009): Prediction Horizon, Rate of Acceptabe Predictions, Reative Accuracy, and Convergence. A brief description of the metric wi be provided in this section but the reader is referred to (Saxena, Ceaya, Saha, Saha, & Goebe, 2009) and (Wang T., 2013) for further information Prediction Horizon Prediction Horizon (PH) is the time difference between the EoL faiure and the time from which the RUL prediction first met the specified performance criteria,i. PH = t E t ia (23) Figure 6. Exampe of SLR simiarity between testing data and a degradation mode for each cyce of the test system Rate of Acceptabe Predictions This metric quantifies the prediction quaity. This is done by determining whether the prediction fas within a specified percentage of the true RUL for each RUL prediction. AP = Mean({δ i t H t i t EoUP }) (24) The specified percentage can be thought of as a cone of accuracy since as the true RUL decreases the accuracy requirement for the prediction become more stringent. 1 if(1 )r i r i (1+ )r i δ i = 0 Otherwise (25) r δ i = 1 i + t E π(r i )dr i β r i t E 0 Otherwise (26) Figure 7. Kerne Density Estimation approach for RUL prediction using mode aggregation. Figure 6 shows an exampe of the simiarity between test data and the trained modes for over 200+ cyces. As can be seen in Figure 6, at the beginning the testing unit is very simiar to a the degradation modes, however as time 6.3. Reative Accuracy Reative accuracy quantitativey evauates the absoute percentage error of a prediction at a time within the prediction horizon, t H, if the agorithm has met the requirements of the previous metrics. RA = 1 Mean r i r i r i t H t i t EoUP (27) 8

9 6.4. Convergence Convergence evauates how fast the prediction performance (any accuracy based metric) improves towards the end ife of the instance, if the agorithm has met the requirements of the previous metrics. 1 CG = EoUP 2 t i+1 2 t 2 i=p i M i EoUP t 2 i+1 t i 6.5. Performance Score i=p M i t p 1 t EoUP t p (28) The fina evauation metric or performance score used in (Wang T., 2013) wi be used in this paper. The performance score is a weighted sum of the Rate of Acceptabe Predictions, Reative Accuracy, and Convergence. PH = Median({ k PH}) (29) AP = Median({ k AR}) (30) RA = Median({ k RA}) (31) CG = Median({ k CG}) (32) Prediction Horizon is the ony metric with a unit of time whie the others have a vaue between 0 and 1, where 1 impies perfect. Since PH wi be used as a preiminary requirement for the performance of RA, a weighted sum of the other three wi be used as the overa performance score. score = w 1 AP + w 2 RA + w 3 CG (33) where w 1 = 0.6, w 2 =.3, w 3 =.1. (Wang T., 2013) 7. DATA SET& EXPERIMENT To compare the performance of the proposed enhancements to the baseine TBSP in (Wang T., 2013), this paper wi use the same data set and experiment as outined in (Wang T., 2013). The Commercia Moduar Aero-Propusion System Simuation (C-MAPSS) is used in this paper. C-MAPSS is a too for simuating a reaistic arge commercia turbofan engine which simuates an engine mode of a 90,000 b thrust cass turbofan engine that was written using MATLAB and Simuink. (Saxena A., Goebe, Simon, & Ekund, 2008) There are four data sets of the run-to-faiure data acquired from the C-MAPSS simuation (Saxena & Goebe, 2008). However, ony the fourth data set, FD004, was used in (Wang T., 2013) and wi be used in this paper. The data set FD004, has 2 faut modes, 6 operating condition regimes, 249 training units, and 248 testing units. There are 25 fieds in the data set: cyce number, 3 condition settings, and 21 sensor measurements. Though FD004 provides a training and testing set, (Wang T., 2013) determined that the testing set contained instances with incompete run-to-faiure data and woud not be suitabe for the performance evauation method described in Section 6. Therefore, in (Wang T., 2013) and in this paper the 249 training units are partitioned in to a training set of 150 randomy seected units with the remaining 99 units being used for evauation. For the experiment, the regime identification, mean variance normaization, and regression modeing foow the same procedure described in (Wang T., 2013). For the RUL estimation, the SLR, MED-DA, and DTW are used to determine the simiarity between the test system and the degradation modes. The RUL of the test system is cacuated based on four different approaches: 1) minimum distance (point estimation), 2) mode aggregation (point estimation), 3) KDE (probabiity interva), and 4) MLE - Maximum Likeihood Estimation (confidence interva). In Summary, there are 12 different RUL predictions being evauating for this paper; each simiarity measure wi have 2 point estimation, KDE, and a MLE. 8. RESULTS This paper compares the RUL prediction using the simiarity measure MED-DA from (Wang T., 2013) to the Pearson s inear correation coefficient and Dynamic Time Warping measures based on the FD004 data of the C- MAPSS data set. Figure 8. TSBP high eve process fow (Wang T., 2013) The resuts are quite different than the ones report in (Wang T., 2013). However, in (Wang T., 2013) a singe tria of 150 randomy seected units were used for training and the remaining 99 were used for testing. In this paper we performed our anaysis using 20 independent trias. Figure 9 shows a boxpot of the 20 tria scores as defined in Eq. (33) showing the median performance of the 20 trias with the 25 th and 75 th percenties as the edges of the box. The whiskers of the box extend to the most extreme data points not considered outiers, and the outiers are potted individuay. 9

10 Figure 9. Boxpot of the 20 Tria Scores of 150 randomy seected units used for training and the remaining 99 used for testing. The resuts in Figure 9 show that DA and SLR simiarity measures are not significanty different for Point Estimation (PE), Aggregated (Aggr), Kerne Density Estimation (KDE), or Maximum Likeihood (ML) predictors. What is interesting is that the DTW measure performed worse than the DA and SLR measure using PE but outperformed them using a ML predictor. It is very difficut to form a concusion based on the experiment performed by (Wang T., 2013), because the resuts wi be greaty dependent upon the randomy seected training and testing dataset. Hence, without knowedge of the specific randomy seected training mode used for the resuts in (Wang T., 2013), it is not feasibe to perform anaysis of a possibe training mode configuration to verify resuts. 9. BASELINE EXPERIMENT Based on the above resuts we have decided to perform an additiona experiment with the intent to baseine these measures and predictors for the FD004 dataset. In the baseine experiment we wi make predictions for each of the 249 unit in the dataset. For each unit under test we wi use the remaining 248 unit for training. This wi aow the experiment to have maximum knowedge of the Feet but without overapping the degradation modes and unit under test. There are two faut modes identified in the FD004 dataset and can be seen in Figure 10. For simpicity, we wi identify faut mode 1 as the red dashed ines and faut mode 2 as the soid back ines which have 101 and 148 degradation modes, respectivey. The ony computationa difference between the simiarity evauating of (Wang T., 2013) and our baseine experiment is that we use ony the degradation modes for a given faut mode once it has been identified for the unit under test (UUT). The initia RUL predictions are based on a of the 248 degradation modes, however after 30 or ess cyces the UUT s faut mode is identified and the simiarity comparisons is reduced to 101 or 148 degradation modes. Of course at no time wi the UUT degradation mode be incuded in simiarity computation of the training degradation modes. Figure 11. Boxpot of the baseine experiment. Tabe 1. Median score performance of RUL simiiaritypredictor combinations. Pointe Estimate DA-PE SLR-PE DTW-PE Kerne Density Estimation 20 Trias Trias Baseine Baseine DA-KDE SLR-KDE DTW-KDE Figure Unit of the FD004 dataset. There were two faut modes identify in the dataset description fie and can be ceary seen by the first principa component of the degradation mode. Each ine is 1 st PC for the 249 units in the FD004 dataset. Mode Aggregation DA-Aggr SLR-Aggr DTW-Aggr Maximum Likeihood 20 Trias Trias Baseine Baseine From Tabe 1, the SLR PE showed the best performance for both the 20 tria experiment adapted from (Wang T., 2013) DA-ML SLR-ML DTW-ML 10

11 and our estabished baseine experiment. However, these scores are based on point estimation performance metrics (Saxena, Ceaya, Saha, Saha, & Goebe, 2009) and do not take advantage of the probabiity or confidence intervas of the ML or KDE predictors. 10. CONCLUSION This paper examined aternative approaches to measure simiarity in a Trajectory Based Simiarity Prediction framework. Additionay, we evauated a simiarity weighted Kerne Density Estimation RUL predictor and simiarity weight maximum ikeihood RUL predictor. The use of these weighted KDE and ML predictors aows the RUL prediction to be defined over a probabiity and confidence interva. The two experiments presented show that the point estimation predictor using the Simpe Linear Regression measure performed the best for each experiment, but further research wi be needed to examine the benefit of the KDE and ML predictors that are not fuy evauated by the performance metrics Some sources of error and uncertainty for TBSP approach incude muti-regime normaization and sensor aggregation through principa component anaysis, see Section 2. The regime normaization assumes uniform system degradation within and across the operationa regimes which may greaty impact the simiarity-predictor performance. Additionay, it is very difficut and impractica to make predictions of RUL for systems that have an unknown operationa profie. It is anticipate that rea word systems wi have a known operationa profie with a desired maintenance free period for a given system. Therefore, predictions and prediction accuracies shoud be based on a faiure occurring within a maintenance free period and known operationa profie for certain appications. We envision future research wi be focused with these restrictions in mind. NOMENCLATURE t i z i The time stamp of the i th measurement cyce The sampe of PC vector at the i th measurement cyce E The index of the End-of-Life measurement cyce for an instance. P The index of the Start-of-Prediction cyce for an instance. EoUP The index of End-of-Usefu-Prediction cyce for an instance. r I The estimated RUL at measurement cyce I r I The ground-truth RUL at measurement cyce I. A eft super script appied to any of the above symbos, indicating the symbo corresponding to the th training instance or degradation mode. G The th degradation mode extracted from the th training instance. D 2 Squared distance to the th degradation mode trajectory. S Simiarity to the th degradation mode trajectory. esnr( ) The Empirica Signa/Noise Ratio computed from 1-D time series data. α Percentage of RUL prediction error bound, e.g i α The index of the first RUL prediction that satisfies the α-bound criteria. REFERENCES Botev, Z. I., Grotowski, J. F., & Kroese, D. P. (2010). Kerne density estimation via diffusion. The Annas of Statistics, 39(5), Daachiesa, M., Nushi, B., Miryenka, K., & Papanas, T. (2012). Uncertain Time-Series Simiarity: Return to the Basics. University of Trento. Fonseca, J., Friswe, M. I., Mottershead, J. E., Lees, A. W., & Adhikari, S. (2005). Uncertainty Quantification using Maximum Likeihood: Experimenta Vaidation. 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structura Dynamics & Materias Conference. Austin. Giusti, G. R. (2013). An Empirica Comparison of Dissimiarity Measures for Time Series Cassification. Braziian Conference on Inteigent Systems (BRACIS). Fortaeza. Gouriveau, R., Ramasso, E., & Zerhouni, N. (2013). Strategies to face imbaanced and unabeed data in PHM appications. Chemica Engineering Transactions(33), Guo, H., Gerokostopouos, A., Liao, H., & Niu, P. (2013). Modeing and Anaysis for Degradation with an Initiation Time. Reiabiity and Maintainabiity Symposium. Lei, H., & Govindaraju, V. (2004). Matching and Retrieving Sequentia Patterns Under Regression. IEEE/WIC/ACM Internationa Conference on Web Inteigence. Otey, M. E., & Parthasarathy, S. (2004). A Dissimiarity Measure for Comparing Subsets of Data: Appication to Mutivariate Time Series. Department of Computer Science and Engineering, The Ohio State University. Savador, S., & Chan, P. (2007). FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space. Inteigent Data Anaysis. Sankararaman, S. (2014). Significance, interpretation, and quantification of uncertainty in prognostics and remaining usefu ife prediction. Mechanica Systems and Signa Processing, In press. 11

12 Sankararaman, S., & Goebe, K. (2013). Why is the Remaining Usefu Life Prediction Uncertain? Annua Conference of the Prognostics and Heath Management Society Sankararaman, S., Daige, M., & Goebe, K. (2014). Uncertainty Quantification in Remaining Usefu Life Prediction Using First-Order Reiabiity Methods. IEEE Transactions on Reiabiity, Saxena, A., & Goebe, K. (2008). C-MAPSS Data Set. NASA Ames Prognostics Data Repository. Saxena, A., & Goebe, K. (2008). PHM08 Chaenge Data Description. Denver: 1st Internationa Conference on Prognostics and Heath Management. Saxena, A., Ceaya, J., Baaban, E., Goebe, K., Saha, B., Saha, S., et a. (2008). Metrics for Evauating Performance of Prognostic Techniques. Prognostics and Heath Management (PHM). Saxena, A., Ceaya, J., Saha, B., Saha, S., & Goebe, K. (2009). On appying the prognostic performance metrics. Prognostics and Heath Management Society. Saxena, A., Goebe, K., Simon, D., & Ekund, N. (2008). Damage Propagation Modeing for Aircraft Engine Run-to-Faiure Simuation. Ist Internationa Conference on Prognostics and Heath Management (PHM08). Denver. Wang, P., & Coit, D. W. (2007). Reiabiity Assessment Based on Degradation Modeing with Random or Uncertain Faiure Threshod. Reiabiity and Maintainabiity Symposium. Orando, FL. Wang, T. (2013). Trajectory Based prediction for Remaining Usefu Life Estimation. Cincinnati: University of Cincinnati. Yu, P., Yong, X., Datong, L., & Xiyuan, P. (2012). Sensor Seection with Grey Correation Anaysis for Remaining Usefu Life Evauation. PHM Society Conference. 12