CAP representations (The right(?) way for generic MR analysis of Internet data ) Amos Ron University of Wisconsin Madison WISP: UCSD, November 2004 : Breaking news, 06/11/04: Wisconsin routed Minnesota 38:14, on its way to the national title. 1
Outline Possible goals behind generic analysis on Internet signals Why is that a non-trivial task? Predictability and pyramidal algorithms Performance of pyramidal representation CAMP and my favorite pyramidal representation What parameters to extract from the representation? 2
A mathematical view of Internet signals Main features in the signal: burst types rate of their appearances This is non-trivial (why? After all, nothing is easier than 1D signals...) the amount of data may be overwhelming there is no clear way to judge success It is also a cultural problem. really? 3
maybe it is time to show some images? 4
5
d 4 d 3 d 2 d 1 6
7
Pyramidal algorithms I: MR representation h : ZZ IR is a symmetric, normalized, filter: h(k) = h( k), k ZZ h(k) = 1., are downsampling & upsampling: (y j ) j= CZZ s.t: y (k) = y(2k), k ZZ { 2y(k/2), k even, y (k) = 0, otherwise. y j = Cy j+1 := (h y j ), j. C is Compression or Coarsification y j+1 is then predicted from y j by y j+1 Py j := h (y j ). P is Prediction or subdivision 8
Pyramidal algorithms II: the detail coefficients Define the detail coefficients: d j := (I PC) y j = y j P y j 1. Replace y j by the pair (y j 1, d j ). Continue iteratively. y m C y m 1 C y m 2... C y 1 y 0 I PC I PC I PC I PC d m d m 1 d m 2 d 1 Reconstruction. Recovering y m from y 0,d 1,d 2,...,d m is trivial: y 1 = d 1 +Py 0, y 2 = d 2 +Py 1 and so on. 9
Wavelet pyramids, Mallat, 1987 Decompose the detail map I PC: I PC = RD D : y (h 1 y) =: w 1,j 1, R : y h 1 y with h 1 a real, symmetric, highpass: k ZZ h 1(k) = 0. y m C y m 1 C y m 2... C y 1 y 0 D 7 D 7 D 7 w 1,m 1 w 1,m 2 w 1,0 Note that we can recover y m from y 0,w 1,0,w 1,1,...,w 1,m 1 since y 1 = Rw 1,0 +Py 0, y 2 = Rw 1,1 +Py 1 and so on. 10
Performance Ability to predict. The best prediction are based on local averaging, and on nothing else =spline predictors Time blurring: good prediction requires long averaging filter. That blurs spontaneous events. Internet data exhibit different behaviour at small scales than other scales. Hence: non-stationary representation Standard wavelet systems are mediocre for Internet data: they blur time, and create artifacts, in order to gain unnecessary properties (orthonormality). 11
Poor prediction 40 40 35 35 30 30 25 25 20 20 15 15 10 10 5 5 0 0 5 10 15 20 25 30 35 40 0 0 5 10 15 20 25 30 35 40 12
My favorite representation well, before we conducted any numerical tests Step I: Build an MR representation based on h 1 = 1 (1 2 1) 4 Step II: Define the detail coefficients by: y j (k+1)+2y j (k) y j (k 1) 4, k even, d j (k) = y j (k 3) 9y j (k 1)+16y j (k) 9y j (k+1) y j (k+3) 16, k odd. The performance grade here is 2 in the strict sense. (To compare, Haar s grade is 1 in the non-strict sense, and 0 in the strict sense.) This is an example of a new class of high-performance representations called CAMP 13
for p 1, the p-norm is what to analyse? what to extract? a p = ( k a k p ) 1/p the best thing to analyse is the compressibility of the detail coefficients: choose a number N, then (1) replace the N most important detail coefficients by 0, to obtain a signal e N. (2) reconstruct using e N to obtain Y N. (3) define e p (N) := Y N p. (4) find the a parameter α such that e p (N) CN α. 14
α(p) = the predictability parameter in the p-norm most important =? (1) Non-linear: choose the largest ones (2) Linear: go from coarse scale to fine scale. Output: this way we have two functions p α(p). Goal: learn how to judge properties of your signal based on these two functions it might be that the detail coefficients behave rather differently at different scale (small scale vs. large scale). 15
CAP representations Choose: two refinable functions φ c,φ r with refinement filters h c,h r. A third (Auxiliary-Alignment) lowpass filter h a. Decompose: Fix f : IR C. For all k,j ZZ, define y j (k) := 2 j/2 f,(φ c ) j,k. The CAP operators are: C : y (h c y), A : y Ay := h a y, P : y Py := h r (y ), (Coarsification-Compression), (Alignment), (Prediction-subdivision). Then Cy j+1 = y j, j. 16
The detail coefficients are: d j := (A PAC)y j = Ay j PAy j 1. This is the CAP representation with (d j ) the CAP coefficients. C C C y m 2... y 1 y m y m 1 y 0 A PAC A PAC A PAC A PAC d m d m 1 d m 2 y m is recovered from y 0,d 1,d 2,...,d m since Ay 1 = d 1 +PAy 0, Ay 2 = d 2 +PAy 1,..., Ay m = d m +PAy m 1 and deconvolving A from Ay m. d 1 17
Summary Do they W F CAP implemented by fast pyramid algorithms? provides good function space characterizations? avoid mother wavelets? very short filters, with no artifacts? have simple constructions? avoid redundant representations? Wavelet are non-redundant. Caplets are only slightly redundant in high dimensions. Their redundancy is non-essential. 18
CAMP representations: Compression-Alignment-Modified Prediction With CAP in hand, one can modify the process s.t.: The filters are shorter The performance (:= function space characterization) is the same Example: Assume h is interpolatory. Define the details as: d j := { yj h y j, on 2ZZ d, y j h (y j ), otherwise. Let φ be the refinable function of h. If φ C s+ǫ c, then the above detail characterize L s p. 19
Example (2D): h = 0 1/8 1/8 1/8 1/4 1/8 1/8 1/8 0. There are four (hidden) filters, for computing d j : 0 1/8 1/8 0 0 0 1/8 +3/4 1/8, 1/2 +1 1/2 1/8 1/8 0 0 0 0 0 1/2 0 0 0 1/2 0 +1 0, 0 +1 0 0 1/2 0 1/2 0 0 Those are 7,3,3,3-tap. There are four (hidden) CAMPlets, whose average area of support is about 2. The performance is on par with tensor 3/5, whose filters are 25, 15, 15-tap. Each supported in 3 3 square. 20
y j 21
d V j d D j d * j d H j Figure 1: First level d CAMP coefficients, organized by cosets. 22
Wisconsin From left, 1st row: Julia Velikina, Youngmi Hur, Yeon Kim, Narfi Stefansson. 2nd row: Thomas Hangelbroek, Sangnam Nam, Jeff Kline, Steven Parker. 23
Julia Velikina: undersampled MRI data Schepp Logan phantom Conventional recon. from 90 projections, acceptable quality Conventional recon. from 23 projections, unacceptable quality TV based recon. from 23 projections 24
Jeff Kline: new data representation in NMR 25
Steven Parker: redundant representation of acoustic signals Adpative framelet based representation of a vibraphone recording coefs scaled to V0 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 1,1 2,1 5,7 5,6 6,11 7,21 7,20 6,9 6,8 8,31 8,30 8,29 8,28 6,6 7,11 7,10 7,9 7,8 7,7 7,6 8,11 8,10 8,9 8,8 8,7 8,6 8,5 8,4 7,1 8,1 8,0 original signal (time) 131072 26
Narfi Stefansson: sparse framelet representations 27
6/10 61440 coefficients quartic spline 34452 coefficients cubic spline 34608 coefficients box15,box17,box18 35085 coefficients 28
FrameNet: on-line interactive framelet and wavelet analysis 29
30
31
32