Dynamic Time Warping

Transcription

1 Dynamic Time Warping Steven Elsworth School of Mathematics, University of Manchester Friday 29 th September, 2017 S. Elsworth Dynamic Time Warping 1 / 19

2 What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. S. Elsworth Dynamic Time Warping 2 / 19

3 What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. We do not need equally spaced time points. S. Elsworth Dynamic Time Warping 2 / 19

4 Temperature ( / C) What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. We do not need equally spaced time points. Temperature at 12:00 in Manchester during Nov Day in November S. Elsworth Dynamic Time Warping 2 / 19

5 Temperature ( / C) ON or OFF What is a time series? A time series is a sequence of measurements of a quantity (e.g. temperature, pressure) over time. We do not need equally spaced time points. Temperature at 12:00 in Manchester during Nov Light switch over 24 hour period Day in November Hour S. Elsworth Dynamic Time Warping 2 / 19

6 What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: S. Elsworth Dynamic Time Warping 3 / 19

7 What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) S. Elsworth Dynamic Time Warping 3 / 19

8 What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) Dynamic Time Warping S. Elsworth Dynamic Time Warping 3 / 19

9 What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) Dynamic Time Warping Longest Common Subsequence model S. Elsworth Dynamic Time Warping 3 / 19

10 What is a distance measure? Consider two time series x = {x i } m i=1 and y = {y i} n i=1. A distance/ similarity measure d(x, y) gives the distance/ similarity between the time series x and y. Some examples include: Euclidean distance (on time series or transformed representation of time series) Dynamic Time Warping Longest Common Subsequence model Threshold based distance measures S. Elsworth Dynamic Time Warping 3 / 19

11 Example data Consider the following time series: S. Elsworth Dynamic Time Warping 4 / 19

12 Euclidean distance Defined by d(x, y) = n i=0 d(x i, y i ) 2 where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. S. Elsworth Dynamic Time Warping 5 / 19

13 Euclidean distance Defined by d(x, y) = n i=0 d(x i, y i ) 2 where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. Fast to compute Satisfies the triangle inequality Requires n = m Performs poorly when the time series have outliers. S. Elsworth Dynamic Time Warping 5 / 19

14 Euclidean distance Defined by d(x, y) = n i=0 d(x i, y i ) 2 where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. Fast to compute Satisfies the triangle inequality Requires n = m Performs poorly when the time series have outliers. S. Elsworth Dynamic Time Warping 5 / 19

15 Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. S. Elsworth Dynamic Time Warping 6 / 19

16 Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: S. Elsworth Dynamic Time Warping 6 / 19

17 Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: 1 Boundary conditions: w 1 = (1, 1) and w k = (m, n) where k is the length of the warping path. S. Elsworth Dynamic Time Warping 6 / 19

18 Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: 1 Boundary conditions: w 1 = (1, 1) and w k = (m, n) where k is the length of the warping path. 2 Continuity: if w i = (a, b) then w i 1 = (a, b ) where a a 1 and b b 1. S. Elsworth Dynamic Time Warping 6 / 19

19 Dynamic Time Warping First proposed by Berndt and Clifford (1994). Start by constructing n m matrix D, where D i,j = d(x i, y j ) where d(x i, y i ) = x i y i or d(x i, y i ) = (x i y i ) 2. A warping path w is a contiguous set of matrix elements which defines a mapping between x and y that satisfies the following conditions: 1 Boundary conditions: w 1 = (1, 1) and w k = (m, n) where k is the length of the warping path. 2 Continuity: if w i = (a, b) then w i 1 = (a, b ) where a a 1 and b b 1. 3 Monotonicity: if w i = (a, b) then w i 1 = (a, b ) where a a 0 and b b 0. S. Elsworth Dynamic Time Warping 6 / 19

20 Cont. Boundary Continuity Monotonicity We want d DTW (x, y) = min k i=1 D(w i). S. Elsworth Dynamic Time Warping 7 / 19

21 Cont. Boundary Continuity Monotonicity We want d DTW (x, y) = min k i=1 D(w i). In practise we use dynamic programming to prevent construction of the whole matrix. Define γ(i, j) to be the cumulative distance then γ(i, j) = d(x i, y j ) + min{γ(i 1, j 1), γ(i 1, j), γ(i, j 1)}. S. Elsworth Dynamic Time Warping 7 / 19

22 Cont. Doesnt satisfy triangle inequality Slow to compute (O(nm) time and space complexity) S. Elsworth Dynamic Time Warping 8 / 19

23 Cont. Doesnt satisfy triangle inequality Slow to compute (O(nm) time and space complexity) S. Elsworth Dynamic Time Warping 8 / 19

24 3D graphical view of Dynamic Time Warping S. Elsworth Dynamic Time Warping 9 / 19

25 What are distance measures used for? S. Elsworth Dynamic Time Warping 10 / 19

26 What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. S. Elsworth Dynamic Time Warping 10 / 19

27 What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. Clustering - Split database DB into groups using d(, ). S. Elsworth Dynamic Time Warping 10 / 19

28 What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. Clustering - Split database DB into groups using d(, ). Classification - Given clusters DB 1,, DB k, which cluster does timeseries x belong to? S. Elsworth Dynamic Time Warping 10 / 19

29 What are distance measures used for? Indexing - Given time series x, find time series y in database DB such that d(x, y) is minimal. Clustering - Split database DB into groups using d(, ). Classification - Given clusters DB 1,, DB k, which cluster does timeseries x belong to? Query search - Given large time series x and small query time series q and distance d, find subsequences of x which are less than d away from q. S. Elsworth Dynamic Time Warping 10 / 19

30 How do we speed it up? S. Elsworth Dynamic Time Warping 11 / 19

31 How do we speed it up? 1 Global Constraints S. Elsworth Dynamic Time Warping 11 / 19

32 How do we speed it up? 1 Global Constraints 2 Lower bounds S. Elsworth Dynamic Time Warping 11 / 19

33 How do we speed it up? 1 Global Constraints 2 Lower bounds 3 FASTDTW S. Elsworth Dynamic Time Warping 11 / 19

34 Global Constraints Figure: Sakoe-Chimba band S. Elsworth Dynamic Time Warping 12 / 19

35 Global Constraints Figure: Sakoe-Chimba band S. Elsworth Figure: Itakura parallelogram Dynamic Time Warping 12 / 19

36 Lower Bounds Many lower bounds have been developed. These are cheaper to compute but prone off unnecessary DTW distance measures. Two most popular are LB KIM and LB KEOGH. LB kimfl is the simplest and of O(1). It consists of finding the Eulidean distance of the first and last point in time series. LB kim is of O(n) and consists of distance between maximum and minimum values of time series. S. Elsworth Dynamic Time Warping 13 / 19

37 Tightness of bound Lower Bounds Many lower bounds have been developed. These are cheaper to compute but prone off unnecessary DTW distance measures. Two most popular are LB KIM and LB KEOGH. LB kimfl is the simplest and of O(1). It consists of finding the Eulidean distance of the first and last point in time series. LB kim is of O(n) and consists of distance between maximum and minimum values of time series. 1 max(lb KeoghEQ,LB KeoghEC )! A LB KeoghEQ A LB KimFL A LB LB Yi Kim 0 O(1) O(n) Order of bound S. Elsworth Dynamic Time Warping 13 / 19

38 FASTDTW FASTDTW was developed by Salvador and Chan. It achieves linear time and space complexity. Consists of three stages: Coarsening, projection, refinement. S. Elsworth Dynamic Time Warping 14 / 19

39 FASTDTW FASTDTW was developed by Salvador and Chan. It achieves linear time and space complexity. Consists of three stages: Coarsening, projection, refinement. Figure: 1/4 Figure: 1/2 Figure: 1/1 S. Elsworth Dynamic Time Warping 14 / 19

40 FASTDTW FASTDTW was developed by Salvador and Chan. It achieves linear time and space complexity. Consists of three stages: Coarsening, projection, refinement. Figure: 1/4 Figure: 1/2 Figure: 1/1 Doesnt always find optimal solution. S. Elsworth Dynamic Time Warping 14 / 19

41 Query search example Consider the following time series. 8 Time Series #10 7 This is an electrocardiogram of an 84-year old female. Recording is 22 hours and 23 minutes long containing 110,087 beats. End cropped for visual purposes. S. Elsworth Dynamic Time Warping 15 / 19

42 Time series: x 250 samples per second giving a total length of 20,145,000. Here is a small 4s clippet (1000 data points). 0.4 Time Series clippet S. Elsworth Dynamic Time Warping 16 / 19

43 Query: q Premature venticular contraction. These are premature heartbeats originating from the ventricles of the heart, possibly due to high blood pressure or heart disease. Query containing 421 data points, about 1.7 seconds. 4 Query S. Elsworth Dynamic Time Warping 17 / 19

44 Results Find the following two closest matches. 4 2 Query search Query 1st Match 2nd Match S. Elsworth Dynamic Time Warping 18 / 19

45 References DTW : Berndt and Clifford (1994) Sakoe-Chimba band: Sakoe and Chimba (1978) Itakura parallelogram: Itakura (1975) Bounds and UCR suite: Rakthanmanon et al. (2012) FastDTW: Salvador and Chan (2007) Last worked example: S. Elsworth Dynamic Time Warping 19 / 19