Dynamic Time Warping

Similar documents
Time-Series Analysis Prediction Similarity between Time-series Symbolic Approximation SAX References. Time-Series Streams

Anticipatory DTW for Efficient Similarity Search in Time Series Databases

Time Series Classification

Efficient search of the best warping window for Dynamic Time Warping

University of Florida CISE department Gator Engineering. Clustering Part 1

GLR-Entropy Model for ECG Arrhythmia Detection

On-line Dynamic Time Warping for Streaming Time Series

Extracting Optimal Performance from Dynamic Time Warping

MANAGING UNCERTAINTY IN SPATIO-TEMPORAL SERIES

Data Preprocessing. Cluster Similarity

Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.

SBASS: Segment based approach for subsequence searches in sequence databases

Option 1: Landmark Registration We can try to align specific points. The Registration Problem. Landmark registration. Time-Warping Functions

The University of Texas at Austin Department of Electrical and Computer Engineering. EE381V: Large Scale Learning Spring 2013.

Improving time-series rule matching performance for detecting energy consumption patterns

Università di Pisa A.A Data Mining II June 13th, < {A} {B,F} {E} {A,B} {A,C,D} {F} {B,E} {C,D} > t=0 t=1 t=2 t=3 t=4 t=5 t=6 t=7

Recent advances in Time Series Classification

Data Mining 4. Cluster Analysis

Introduction to Time Series Mining. Slides edited from Keogh Eamonn s tutorial:

Uncertain Time-Series Similarity: Return to the Basics

Learning Time-Series Shapelets

Math 110 Test # 1. The set of real numbers in both of the intervals [0, 2) and ( 1, 0] is equal to. Question 1. (F) [ 1, 2) (G) (2, ) (H) [ 1, 2]

Chapter 3. Measuring data

What is Statistics? Statistics is the science of understanding data and of making decisions in the face of variability and uncertainty.

CS6220: DATA MINING TECHNIQUES

HOT SAX: Efficiently Finding the Most Unusual Time Series Subsequence

Michael Lechner Causal Analysis RDD 2014 page 1. Lecture 7. The Regression Discontinuity Design. RDD fuzzy and sharp

Data Mining: Data. Lecture Notes for Chapter 2. Introduction to Data Mining

Cluster Analysis (Sect. 9.6/Chap. 14 of Wilks) Notes by Hong Li

Chapter 17 Current and Resistance

Generalized Gradient Learning on Time Series under Elastic Transformations

Nearest-Neighbor Searching Under Uncertainty

Temporal and Frequential Metric Learning for Time Series knn Classication

10.1 Vectors. c Kun Wang. Math 150, Fall 2017

The Normal Distribution. Chapter 6

Probabilistic classification CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2016

Retrieval by Content. Part 2: Text Retrieval Term Frequency and Inverse Document Frequency. Srihari: CSE 626 1

Neural Networks Lecture 2:Single Layer Classifiers

Lecture Notes 1: Vector spaces

4. Numerical Quadrature. Where analytical abilities end... continued

Searching Dimension Incomplete Databases

CSE446: non-parametric methods Spring 2017

Chapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)

Research Article Dynamic Time Warping Distance Method for Similarity Test of Multipoint Ground Motion Field

Lecture 3. Econ August 12

Active and Semi-supervised Kernel Classification

Lower Bounds on the Graphical Complexity of Finite-Length LDPC Codes

Supplementary Information

Grade 6 Curriculum Map Key: Math in Focus Course 1 (MIF)

Year 10 Higher Easter 2017 Revision

Clustering by Mixture Models. General background on clustering Example method: k-means Mixture model based clustering Model estimation

Chapter 17. Current and Resistance

Absolute Value Inequalities 2.5, Functions 3.6

Support Vector Machines

CSE 546 Final Exam, Autumn 2013

The problem of classification in Content-Based Image Retrieval systems

BREVET BLANC N 1 JANUARY 2012

MAC Learning Objectives. Learning Objectives (Cont.) Module 10 System of Equations and Inequalities II

STATISTICAL PERFORMANCE

Outline. Background on time series mining Similarity Measures Normalization

Cheng Soon Ong & Christian Walder. Canberra February June 2018

Time Series. Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech. Mining and Forecasting

Dynamic Time Warping and FFT: A Data Preprocessing Method for Electrical Load Forecasting

Computational Intelligence Lecture 3: Simple Neural Networks for Pattern Classification

A FINE-GRAINED APPROACH TO

Time: 1 hour 30 minutes

Decision Trees (Cont.)

Definitions and Properties of R N

Shape, Outliers, Center, Spread Frequency and Relative Histograms Related to other types of graphical displays

Max-Sum Diversification, Monotone Submodular Functions and Semi-metric Spaces

Machine Learning on temporal data

CS6220: DATA MINING TECHNIQUES

SYDE 372 Introduction to Pattern Recognition. Probability Measures for Classification: Part I

Modeling Periodic Phenomenon with Sinusoidal Functions Lesson 33

Bayesian Decision Theory

Math 117: Infinite Sequences

Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata

Number of weeks in n days. Age in human years of your dog when she has lived n years. Your actual height h when you are wearing 2 inch shoes.

f (x) f (a) f (a) = lim x a f (a) x a

Econ Lecture 3. Outline. 1. Metric Spaces and Normed Spaces 2. Convergence of Sequences in Metric Spaces 3. Sequences in R and R n

a) 3 cm b) 3 cm c) cm d) cm

Lecture 5. Ch. 5, Norms for vectors and matrices. Norms for vectors and matrices Why?

Lecture 10: Dimension Reduction Techniques

Knowledge Discovery in Databases II Summer Term 2017

Nonlinear Support Vector Machines through Iterative Majorization and I-Splines

ICML - Kernels & RKHS Workshop. Distances and Kernels for Structured Objects

Transforming Hierarchical Trees on Metric Spaces

THE CRYSTAL BALL FORECAST CHART

Lecture 9: March 26, 2014

Measures of Morphological Complexity of Gray Matter on Magnetic Resonance Imaging for Control Age Grouping

A Comparison of HRV Techniques: The Lomb Periodogram versus The Smoothed Pseudo Wigner-Ville Distribution

An invariance result for Hammersley s process with sources and sinks

Data Mining Stat 588

Consider teaching Geometry throughout the school year. Geometry Friday is a suggestion. See a member of the pacing committee for information.

Unsupervised Learning

Decision Trees. Machine Learning CSEP546 Carlos Guestrin University of Washington. February 3, 2014

Machine Learning Linear Classification. Prof. Matteo Matteucci

Write down all the 3-digit positive integers that are squares and whose individual digits are non-zero squares. Answer: Relay

Probabilistic Similarity Search for Uncertain Time Series

Day-ahead time series forecasting: application to capacity planning

Transcription:

Dynamic Time Warping Steven Elsworth School of Mathematics, University of Manchester Friday 29 th September, 2017 S. Elsworth Dynamic Time Warping 1 / 19

What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. S. Elsworth Dynamic Time Warping 2 / 19

What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. We do not need equally spaced time points. S. Elsworth Dynamic Time Warping 2 / 19

Temperature ( / C) What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. We do not need equally spaced time points. Temperature at 12:00 in Manchester during Nov 2016 14 12 10 8 6 4 2 0 5 10 15 20 25 30 Day in November S. Elsworth Dynamic Time Warping 2 / 19

Temperature ( / C) ON or OFF What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. We do not need equally spaced time points. Temperature at 12:00 in Manchester during Nov 2016 14 12 10 1 0.8 Light switch over 24 hour period 8 0.6 6 0.4 4 0.2 2 0 5 10 15 20 25 30 Day in November 0 2 4 6 8 10 12 14 16 18 20 22 24 Hour S. Elsworth Dynamic Time Warping 2 / 19

What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: S. Elsworth Dynamic Time Warping 3 / 19

What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) S. Elsworth Dynamic Time Warping 3 / 19

What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) Dynamic Time Warping S. Elsworth Dynamic Time Warping 3 / 19

What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) Dynamic Time Warping Longest Common Subsequence model S. Elsworth Dynamic Time Warping 3 / 19

What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) Dynamic Time Warping Longest Common Subsequence model Threshold based distance measures S. Elsworth Dynamic Time Warping 3 / 19

Example data Consider the following time series: 5 4 3 2 1 0-1 0 5 10 15 20 25 S. Elsworth Dynamic Time Warping 4 / 19

Euclidean distance Defined by d(x, y) = n i=0 d(x i, y i ) 2 where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. S. Elsworth Dynamic Time Warping 5 / 19

Euclidean distance Defined by d(x, y) = n i=0 d(x i, y i ) 2 where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. Fast to compute Satisfies the triangle inequality Requires n = m Performs poorly when the time series have outliers. S. Elsworth Dynamic Time Warping 5 / 19

Euclidean distance Defined by d(x, y) = n i=0 d(x i, y i ) 2 where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. Fast to compute Satisfies the triangle inequality Requires n = m Performs poorly when the time series have outliers. S. Elsworth Dynamic Time Warping 5 / 19

Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. S. Elsworth Dynamic Time Warping 6 / 19

Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: S. Elsworth Dynamic Time Warping 6 / 19

Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: 1 Boundary conditions: w 1 = (1, 1) and w k = (m, n) where k is the length of the warping path. S. Elsworth Dynamic Time Warping 6 / 19

Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: 1 Boundary conditions: w 1 = (1, 1) and w k = (m, n) where k is the length of the warping path. 2 Continuity: if w i = (a, b) then w i 1 = (a, b ) where a a 1 and b b 1. S. Elsworth Dynamic Time Warping 6 / 19

Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: 1 Boundary conditions: w 1 = (1, 1) and w k = (m, n) where k is the length of the warping path. 2 Continuity: if w i = (a, b) then w i 1 = (a, b ) where a a 1 and b b 1. 3 Monotonicity: if w i = (a, b) then w i 1 = (a, b ) where a a 0 and b b 0. S. Elsworth Dynamic Time Warping 6 / 19

Cont. Boundary Continuity Monotonicity We want d DTW (x, y) = min k i=1 D(w i). S. Elsworth Dynamic Time Warping 7 / 19

Cont. Boundary Continuity Monotonicity We want d DTW (x, y) = min k i=1 D(w i). In practise we use dynamic programming to prevent construction of the whole matrix. Define γ(i, j) to be the cumulative distance then γ(i, j) = d(x i, y j ) + min{γ(i 1, j 1), γ(i 1, j), γ(i, j 1)}. S. Elsworth Dynamic Time Warping 7 / 19

Cont. Doesnt satisfy triangle inequality Slow to compute (O(nm) time and space complexity) S. Elsworth Dynamic Time Warping 8 / 19

Cont. Doesnt satisfy triangle inequality Slow to compute (O(nm) time and space complexity) S. Elsworth Dynamic Time Warping 8 / 19

3D graphical view of Dynamic Time Warping S. Elsworth Dynamic Time Warping 9 / 19

What are distance measures used for? S. Elsworth Dynamic Time Warping 10 / 19

What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. S. Elsworth Dynamic Time Warping 10 / 19

What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. Clustering - Split database DB into groups using d(, ). S. Elsworth Dynamic Time Warping 10 / 19

What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. Clustering - Split database DB into groups using d(, ). Classification - Given clusters DB 1,, DB k, which cluster does timeseries x belong to? S. Elsworth Dynamic Time Warping 10 / 19

What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. Clustering - Split database DB into groups using d(, ). Classification - Given clusters DB 1,, DB k, which cluster does timeseries x belong to? Query search - Given large time series x and small query time series q and distance d, find subsequences of x which are less than d away from q. S. Elsworth Dynamic Time Warping 10 / 19

How do we speed it up? S. Elsworth Dynamic Time Warping 11 / 19

How do we speed it up? 1 Global Constraints S. Elsworth Dynamic Time Warping 11 / 19

How do we speed it up? 1 Global Constraints 2 Lower bounds S. Elsworth Dynamic Time Warping 11 / 19

How do we speed it up? 1 Global Constraints 2 Lower bounds 3 FASTDTW S. Elsworth Dynamic Time Warping 11 / 19

Global Constraints Figure: Sakoe-Chimba band S. Elsworth Dynamic Time Warping 12 / 19

Global Constraints Figure: Sakoe-Chimba band S. Elsworth Figure: Itakura parallelogram Dynamic Time Warping 12 / 19

Lower Bounds Many lower bounds have been developed. These are cheaper to compute but prone off unnecessary DTW distance measures. Two most popular are LB KIM and LB KEOGH. LB kimfl is the simplest and of O(1). It consists of finding the Eulidean distance of the first and last point in time series. LB kim is of O(n) and consists of distance between maximum and minimum values of time series. S. Elsworth Dynamic Time Warping 13 / 19

Tightness of bound Lower Bounds Many lower bounds have been developed. These are cheaper to compute but prone off unnecessary DTW distance measures. Two most popular are LB KIM and LB KEOGH. LB kimfl is the simplest and of O(1). It consists of finding the Eulidean distance of the first and last point in time series. LB kim is of O(n) and consists of distance between maximum and minimum values of time series. 1 max(lb KeoghEQ,LB KeoghEC )! A LB KeoghEQ A LB KimFL A LB LB Yi Kim 0 O(1) O(n) Order of bound S. Elsworth Dynamic Time Warping 13 / 19

FASTDTW FASTDTW was developed by Salvador and Chan. It achieves linear time and space complexity. Consists of three stages: Coarsening, projection, refinement. S. Elsworth Dynamic Time Warping 14 / 19

FASTDTW FASTDTW was developed by Salvador and Chan. It achieves linear time and space complexity. Consists of three stages: Coarsening, projection, refinement. Figure: 1/4 Figure: 1/2 Figure: 1/1 S. Elsworth Dynamic Time Warping 14 / 19

FASTDTW FASTDTW was developed by Salvador and Chan. It achieves linear time and space complexity. Consists of three stages: Coarsening, projection, refinement. Figure: 1/4 Figure: 1/2 Figure: 1/1 Doesnt always find optimal solution. S. Elsworth Dynamic Time Warping 14 / 19

Query search example Consider the following time series. 8 Time Series 6 4 2 0-2 -4-6 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 #10 7 This is an electrocardiogram of an 84-year old female. Recording is 22 hours and 23 minutes long containing 110,087 beats. End cropped for visual purposes. S. Elsworth Dynamic Time Warping 15 / 19

Time series: x 250 samples per second giving a total length of 20,145,000. Here is a small 4s clippet (1000 data points). 0.4 Time Series clippet 0.3 0.2 0.1 0-0.1-0.2-0.3-0.4 0 100 200 300 400 500 600 700 800 900 1000 S. Elsworth Dynamic Time Warping 16 / 19

Query: q Premature venticular contraction. These are premature heartbeats originating from the ventricles of the heart, possibly due to high blood pressure or heart disease. Query containing 421 data points, about 1.7 seconds. 4 Query 2 0-2 -4-6 0 50 100 150 200 250 300 350 400 S. Elsworth Dynamic Time Warping 17 / 19

Results Find the following two closest matches. 4 2 Query search Query 1st Match 2nd Match 0-2 -4-6 0 50 100 150 200 250 300 350 400 S. Elsworth Dynamic Time Warping 18 / 19

References DTW : Berndt and Clifford (1994) Sakoe-Chimba band: Sakoe and Chimba (1978) Itakura parallelogram: Itakura (1975) Bounds and UCR suite: Rakthanmanon et al. (2012) FastDTW: Salvador and Chan (2007) Last worked example: http://www.cs.ucr.edu/$\sim$eamonn/ucrsuite.html S. Elsworth Dynamic Time Warping 19 / 19

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback