Wavelet decomposition of data streams. by Dragana Veljkovic

Transcription

1 Wavelet decomposition of data streams by Dragana Veljkovic

2 Motivation Continuous data streams arise naturally in: telecommunication and internet traffic retail and banking transactions web server log records etc. Many applications need this data to be processed on a 24*7 basis in only one pass

3 Motivation cont. Usually this data is accumulated and archived for later use, but not always (e.g. network security) The ability to make decisions and interpret interesting patterns online can be crucial and has real dollar value for large corporations (e.g. fraud detection)

4 Our motivation Currently working on data collected from 100 electrodes receiving electrical potential of monkey brain over long periods of time We want to look at this data in real time and seek patterns, trends and surprises

5 Outline Background streams wavelets sketches error analysis Results Implementation details Strengths and weaknesses of this approach

6 Data streams Sequence of unbounded, real time data with high rate that can only be read once by an application Problems: Unbounded memory requirements High data rate

7 Underlying signal Signal is one dimensional function a: [0,, N-1]? Z + Data item that arrives in time is an ordered pair: <domain, value> Example: voting results <Texas, 60> Example: phone call records < , 12>

8 Data model Two different data models used for rendering the underlying signal: Cash register Aggregate Example: cash register model < ,10>, < ,13>, < , 20>, < , 5>, < , 2>, < , 30> where the underlying signal is < , 32>, < , 13>, < , 35>

9 Stream format Two distinct formats for the stream Ordered Unordered Example: Aggregate ordered stream any time series Example: Unordered cash-register stream phone call records Ordered cash-register is trivial to convert to order aggregate

10 Wavelets Basis functions of limited duration and average value of zero Basis functions are shifted and scaled versions of the original wavelet

11 Discrete wavelet transform Uses only fixed values for wavelet scales based on powers of two Wavelet positions are also fixed and non overlapping Wavelets form a set of wavelet basis vectors of length N Haar wavelets for signal of size 8 Example: Haar wavelets on signal of length N = 8 j = 1,, logn levels k = 0,, 2 j -1 spaces for each level

12 Wavelet decomposition Wavelet decomposition can be regarded as projection of the signal on the set of wavelet basis vectors Each wavelet coefficient can be computed as the dot product of the signal with the corresponding basis vector Example: Table 1. from Gilbert et al

13 Best B-term decomposition The signal can be fully recovered from the wavelet decomposition Best B-term decomposition uses only a small number of coefficients, B, that carry the highest energy The signal reconstructed using the B-term coefficients and the corresponding vectors is called the best B-term approximation Most signals that occur in nature can be well approximated using only a small number of coefficients (5-10).

14 Computing best B-term decomposition in runtime For the ordered aggregate model Maintain two sets of items Highest B wavelet basis coefficients for the signal seen so far logn straddling coefficients, one for each level When the data item is read the affected straddling coefficients get updated. If a coefficient is no longer straddling it is compared to existing highest B coefficient and the set is updated if necessary. New straddling coefficient is initialized. Takes O(B + logn) storage and time for the ordered aggregate model

15 Sketches Sketch is made by projecting a signal onto several different low dimensional spaces which are chosen at random Many properties of the signal, such as histograms, can be accurately estimated by looking at the sketch

16 Definition of a sketch Atomic sketch of signal a is the dot product <a, r> where r is a random vector of ±1 valued random variables A sketch of a signal is k independent atomic sketches, each with a different random vector r j Sketch size is small compared to the signal size

17 Sketches Maintaining the sketch is easy as we are receiving the data If element <i, a(i)> arrives, add a(i)*r ij to the sketch corresponding to random vector r j Example: In cash-register receive <5, 10>, need to add 10* r 5j to each atomic sketch corresponding to the random vector r j

18 Error metrics SSE (sum squared error) if R is a representation of the signal a then SSE is defined as Pseudoenergy of the representation R is computed as

19 Query processing Batched queries are posed at certain periodic intervals Ad hoc a query may be posed at any time

20 Batch query using best B-term approximation for day 0 of call records Figure 2. from Gilbert et al

21 Batch query using best B-term approximation for all 7 days of call records Figure 3. from Gilbert et al

22 Estimating a point query Answer to point query i is a(i) Direct point estimate directly estimating a(i) using the sketch Direct wavelet estimate use the sketch to estimate the wavelet coefficients whose support intersects i and reconstruct a(i) using these coefficients Another way is to compute a(i) using only the high wavelet coefficients (like the known B-term approximation) whose support intersects a(i)

23 Using sketches to estimate dot product Following parameters characterize how well the sketch does e distortion parameter d failure probability? failure threshold Sketch of a signal is independent atomic sketches, each with a different random vector If the cosine between vectors a and b is greater than? we estimate the dot product within (1±e) with probability at least 1- d

24 Sketches and random vectors If element <i, a(i)> arrives, add a(i)*r ij to the sketch corresponding to random vector r j In order to use the sketches we need to get the elements r j quickly. r j is of size N, it can not be stored explicitly

25 Generating random vectors The paper shows that r ij can be generated by a pseudorandom number generator using a seed s j of size log O(1) N Generator G is based on second order Reed-Muller codes The generator G takes s j and i and outputs r j i = G(s j, i) quickly

26 Estimation of dot products using sketches Lemma: Let X be a O(logN/ d)-wise median of e 2 )-wise means of independent copies of O(1/ then we have with probability of 1-1 d Note: : use b=a to estimate energy of a using this lemma

27 Example: Want to estimate dot product of vectors a and b with no more than 30% error with probability of 80%, assuming the cosine between these two vectors is greater then 0.25 That is e = 0.3,? = 0.25 and d = 0.2 and for a signal of size N=1024 we would need about 30 atomic sketches

28 Theorem There is a streaming algorithm, A, such that, given a signal a[1,, N] with energy a 22 if there is a B-term representation with energy at least?* a 22, then, with probability at least (1-d) A finds a representation of at most B terms with pseudoenergy at least (1-e)?* a 22. If there is no such B-term representation with energy?* a 22, A reports no good representation. In any case A uses space and per item time while processing the stream. This holds with both aggregate and cash-register models Example: take?=0.3, d=0.2, e=0.3 and B=10. Then if there exists a 10 terms representation of the signal that captures at least 30% of the signal s energy the algorithm will output a 10 term representation with energy at least 21% of the signal with 80% probability

29 Strengths and weaknesses Good example how to work with cashregister models Shows several ways to estimate the signal using a sketch Time requirements seem higher than the paper claims On-line algorithms do not seem as promising as batch algorithms

30 References 1. A. C. Gilbert, Y. Kotidis, S. Muthukrishnan and M. J. Strauss, "Onepass wavelet decomposition of data streams," IEEE transactions on knowledge and data engineering, Vol. 15, No. 3, May/June A. C. Gilbert, Y. Kotidis, S. Muthukrishnan and M. J. Strauss, "Surfing wavelets on streams: one-pass summaries for approximate aggregate queries," Proceedings of the 27th VLDB Conference, Roma, Italy A. C. Gilbert, S. Guha, P. Indyk, Y. Kotidis, S. Muthukrishnan and M. J. Strauss, "Fast, small-space algorithms for approximate histogram maintenance," STOC 02, May 19-21, 2002, Montreal, Quebec, Canada.

31 Answering queries on-line Comparison of sse/energy of top B wavelets against direct estimates Table 1. from Gilbert et al Table 2. from Gilbert et al

32 Direct estimates for the top 10 heavy hitters Figure 6. from Gilbert et al

33 Direct estimates for the top 10 heavy hitters using the greedy algorithm Figure 7. from Gilbert et al

34 Adaptive greedy pursuit for heavy hitters Obtain a very accurate estimate for the first heavy hitter Get a new sketch by subtracting this value from the original sketch. This can be done because sketches are linear New sketch is a good estimation of the residual distribution in which the second heavy hitter is the peak value Use the new sketch to estimate the second heavy hitter Repeat procedure for more heavy hitters Each estimate introduces an error and after many iterations the errors tend to overwhelm the benefits