DL Approaches to Time Series Data Miro Enev, DL Solution Architect Jeff Weiss, Director West SAs
Agenda Define Time Series [ Examples & Brief Summary of Considerations ] Semi-supervised Anomaly Detection [ with Deep Autoencoders ] Ensemble event detection and classification [ with MLPs & CNN-MLPs ] Prediction [ with Dual Attentional RNNs (DA-RNNs) ]
Time Series [ One Definition ] A time series is a series of values recorded over equally spaced time intervals. The amount of time between observations is the sampling interval or sampling rate. The time series represent some underlying, partially-random ( stochastic ) process which generated the data. We want to use the data to make guesses ( inferences ) about the process, and want to make reliable guesses while being clear about the uncertainty involved. The complication is that each observation is dependent on all the other observations, and in fact its this dependence that we want to learn and draw inferences about.
Popularity of French Names
NVIDIA Stock https://www.google.com/finance?chdnp=0&chdd=0&chds=1& chdv=1&chvs=linear&chdeh=0&chfdeh=0&chdet=1494360000 000&chddm=98532&chls=IntervalBasedLine&q=NASDAQ:NVD A&ntsp=0&ei=N_kRWemBAoqxjAHRgZDwBA
Categories of Time Series Analysis Prediction/Forecasting Weather, Sales, Inventory, Financial Markets Anomaly Detection Predictive Maintenance Reconstruction Virtual Sensors Classification Event/Activity Recognition
Time Series Best Practices Think hard about the input(s) Time and frequency domain representations Lags and windowing (multi-time scale) Study correlational structure Use semi/unsupervised approaches [ free ] Feature Building, Data Buckets, Anomaly Detection Gather and/or generate ground truth Class imbalance & sample weighting Try DL fabrics with increasing complexity & Ensemble Use a non DL baseline [ ARIMA, SVMs, random forests ] Use forward chained cross-validation for evaluation Train attentional mechanism*
DL for Anomaly Detection
Deep Autoencoder Anomaly Detection 13 Sensors, 100Hz, NASA Dataset,.5 seconds window, 650 dimensions per sample [ 256, 196, 136, 76, 14 ] Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16)
Sample Reconstructions Reconstruction & Feature Learning Exceptionally low normalized RMS reconstruction error (0.04 0.09) Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16)
Anomaly Analysis 11-layer 14-dimensional bottleneck DAE yields 97.8% true positive detection rate with 0.0% false alarm Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16) Artifcially created anomalies [ Spall Fault ; Ballscrew Jam ]
Anomaly Analysis/Interpretation 11-layer 14-dimensional bottleneck DAE yields 97.8% true positive detection rate with 0.0% false alarm Anomaly Detection and Fault Disambiguation in Large Flight Data: A Multi-modal Deep Auto-encoder Approach, K. Reddy et al, United Technologies Research Center (PHM16) Artificially created anomalies [ Spall Fault ; Ballscrew Jam ]
DL for Time Series Classification
IoT Case Study Water Disaggregation
Frequency Domain Utilization
Automated Ground Truth Building Open Edges Close Edges
Event/Edge Detection
Edge Characterization, Example 1 Open Edge Close edge
Edge Characterization, Example 2 Close edge Open Edge
DL for Time Series Prediction
Dual Attention RNNs
Performance @ NASDAQ 100 In NASDAQ 100 Stock dataset, we collect the stock prices of 81 major corporations under NASDAQ 100, which are used as the driving time series. The index value of NASDAQ 100 is used as the target series. The frequency of the data collection is one-minute. This data covers the period from July 26, 2016 to December 26, 2016, in total 104 days. Each day contains 390 data points from the opening to closing of the market. In our experiments, we use the first 90 days as the training set and the following seven days as the validation.
Performance @ SML 2010 SML 2010 is a public dataset used for indoor temperature forecasting. This dataset is collected from a monitor system mounted in a domestic house. We use room temperature as the target series and select 16 relevant driving series which contains approximately 40 days of monitoring data. The data was sampled every minute and was smoothed with 15 minute means. In our experiment, we use the first 3200 data points as the training set, the following 400 data points as the validation set, and the last 537 data points as the test set.
Attention Mechanism at Work Train Set [ NASDAQ ] Test Set [ NASDAQ ]
Thanks!