Haar wavelets The Haar wavelet basis for L (R) breaks down a signal by looking at the difference between piecewise constant approximations at different scales. It is the simplest example of a wavelet transform, and is very easy to understand. Let V be the space of signals that are piecewise constant between the integers. Example member: 3 3 3 4 Set φ (t) = { t < otherwise.5.5.5 It is clear that {φ (t n), n Z} is an orthobasis for V. Given an arbitrary x (t), we can find the best piecewise constant (between the integers) approximation by projecting onto the span of the {φ (t n)}. To keep the notation compact, we use φ,n (t) = φ (t n), and then the closest signal in V to x(t) is ˆx (t) = x, φ,n φ,n (t). 7 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
We will use P V [ ] for the projection operator onto V. P V [ ] takes a signal and returns the signal in V closest to the input. Then Example: ˆx (t) = P V [x(t)]..8.6.4...4.6.8 x(t).8.6.4...4.6.8 ˆx (t) = P V [x(t)] 3 4 5 6 3 4 5 6 We can do something similar for the space of piecewise constant functions at higher resolution. Let V j be the space of signals that are piecewise constant on intervals of length j..5 signal in V signal in V.5.5.5.5.5.5.5.5.75.5.5.5.5.75.5.5.75.5.5 We can get an orthobasis for V j simply by contracting the basis for V by a dyadic factor. Set φ j (t) = j/ φ ( j t), 8 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
and φ j,n (t) = φ j (t j n) = j/ φ ( j t n). Then it is easy to see that {φ j,n (t), n Z} is an orthobasis for V j. Examples: φ, (t) φ, (t) φ, (t) φ, (t).5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5 φ, (t) φ, (t) φ, (t) φ, (t) φ, (t) φ,3 (t) φ,4 (t) φ,5 (t) The best approximation to an arbitrary signal x(t) L (R) is again given by ˆx j (t) = P Vj [x(t)] = x, φ j,n φ j,n (t) As j increases this approximation becomes better and better: 9 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
.8.6.4...4.6.8 x(t) ˆx (t) (in V ).8.6.4...4.6.8 3 4 5 6 3 4 5 6 ˆx (t) (in V ) ˆx (t) (in V ).8.6.4...4.6.8.8.6.4...4.6.8 3 4 5 6 3 4 5 6 ˆx 3 (t) (in V 3 ) ˆx 4 (t) (in V 4 ).8.6.4...4.6.8 3 4 5 6.8.6.4...4.6.8 3 4 5 6 In the end, at infinitely fine resolution, we get the original signal back: lim ˆx j(t) = lim P Vj [x(t)] = x(t). j j The Haar wavelet basis represents the differences between these approximations. Set W = V V. This is the space of signals that are in V but are orthogonal to everything in V. Signals in W are piecewise constant between the Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
half integers (just like everything else in V ) but also have zero mean between the integers. Example member of W :.8.6.4...4.6.8 3 4 5 6 Similarly, we can set W j = V j+ V j. We can generate an orthobasis for the W j using the following function: ψ (t) = With t < / / t <. otherwise.5.5.5.5.5 then ψ j (t) = j/ ψ( j t), is an orthobasis for W j. ψ j,n (t) = ψ j (t j n) = j/ ψ ( j t n), {ψ j,n (t), n Z} We now have the following multiscale decomposition for an arbitrary Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
x(t): x(t) = P V [x(t)] + P W [x(t)] }{{} =P V [x(t)] }{{} =P V [x(t)] +P W [x(t)] +P W [x(t)] + } {{ } =P V3 [x(t)] The corresponding reproducing formula for the Haar wavelet system is x(t) = x, φ,n φ,n (t) + x, ψ j,n ψ j,n (t) } {{ } =P V [x(t)] j= } {{ } =P Wj [x(t)] Here are plots of the φ j,n (t) and ψ j,n (t) at the first three scales: (From Burrus et al, Intro. to Wavelets and Wavelet Transforms) Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
Nomenclature We call the V j the scaling spaces or approximation spaces. The {φ j,n, n Z} are an orthobasis for V j, and are called scaling functions at scale j. The W j are called wavelet spaces or detail spaces. The {ψ j,n (t), n Z} are an orthobasis for W j, and are called wavelets at scale j. The expansion coefficients that specify the approximation ˆx j in terms of the φ j,n are called scaling coefficients: in the expression ˆx j = s j,n φ j,n (t), s j,n = x, φ j,n, the s j,n are scaling coefficients at scale j. The expansion coefficients for the projection onto W j are called wavelet coefficients w j,n at scale j: w j,n = x, ψ j,n, and P Wj [x(t)] = w j,n ψ j,n (t). 3 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
Moving between scales We have seen that conceptually, the Haar transform analyzes a signal through a series of multiscale approximations onto nested subspaces V j, with V V V L (R). We can move between different approximation scales by adding in the detail spaces W j, V j+ = V j W j. P V [x(t)] + P W [x(t)] = P V3 [x(t)].5.8.4.8.6.3.6.4.. +... =.4...4..4.6.3.6.8.4.8 3 4 5 6.5 3 4 5 6 3 4 5 6 At a fixed scale J, there are multiple ways to write the approximation ˆx J in V J. The first is using just the scaling function at scale J: ˆx J (t) = P VJ [x(t)] = s J,n φ J,n (t). Another way is to break up the approximation at two scales: approximate x(t) in V J then add in details in W J to take you to the approximation in V J : ˆx J (t) = P VJ [x(t)] + P WJ [x(t)] = s J,n φ J,n (t) + w J,n ψ J,n (t) 4 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
Yet another way is to break up the approximation at three scales: approximate x(t) in V J, then add in the details in W J to take you to the approximation in V J, then add in the details in W J to take you to the approximation in V J : ˆx J (t) = P VJ [x(t)] + P WJ [x(t)] + P WJ [x(t)]. Of course we could continue this all the way up to scale : ˆx J (t) = P V [x(t)] + P W [x(t)] + + P WJ [x(t)] J = s,n φ,n (t) + w j,n ψ j,n (t). j= One of the key properties of the Haar wavelet transform (and all wavelet transforms, as we will see later) is that there is an efficient way to compute the multiscale approximation coefficients {s,n, w,n, w,n,..., w J,n } from the single scale approximation coefficients {s J,n }. What lets us do this is the simple fact that the scaling functions φ J,n and wavelet functions ψ J,n at scale J both can be built up out of scaling functions φ J,n at scale J. To start, let us look at scale V = V W. Now, as we illustrate below, φ (t) = (φ (t) + φ (t /)), = (φ, (t) + φ, (t)), and ψ (t) = (φ, (t) φ, (t)). 5 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
φ, (t) = φ, (t) + φ, (t).5.5.5.5 =.5 +.5.5.5.5.5.5 ψ, (t) = φ, (t) - φ, (t).5.5.5.5.5 =.5 -.5.5.5.5.5.5.5 We can generalize this relation to different shifts at scale : φ,n (t) = (φ,n (t) + φ,n+ (t)), ψ,n (t) = (φ,n (t) φ,n+ (t)), as well as between general scales j and j + : φ j,n (t) = (φ j+,n (t) + φ j+,n+ (t)), ψ j,n (t) = (φ j+,n (t) φ j+,n+ (t)). This immediately shows us how to compute the scaling coefficients s j,n and wavelet coefficients w j,n at scale j from the scaling coefficients 6 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
s j+,n at scale j + : and s j,n = x, φ j,n = x, φ j+,n + x, φ j+,n+ = s j+,n + s j+,n+, w j,n = x, ψ j,n = x, φ j+,n x, φ j+,n+ = s j+,n s j+,n+. If we think of the scaling/wavelet coefficients at scale j as a discrete time sequence, so s j [n] := s j,n and w j [n] := w j,n, then the expressions above suggest that the scaling coefficients at scale j can be broken down scaling and wavelet coefficients at scale j using the following architecture: h [n] s j [n] s j+ [n] h [n] w j [n] 7 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
where the block means downsample by and the impulse responses for the filters are.8 h [n] = { n =, otherwise.7.6.5.4.3.. 3 3 h [n] = n = n = otherwise.8.6.4...4.6.8 3 3 Of course, we can continue on and break up the {s j,n } into scaling and wavelet coefficients at the next coarsest scale. Now we can associate a filter bank structure with each of the ways we can write the approximation at scale J, ˆx J (t) = P VJ [x(t)]. ˆx J (t) = P VJ [x(t)] + P WJ [x(t)] = s J,n φ J,n (t) + w J,n ψ J,n (t) h [n] s J [n] s J [n] h [n] w J [n] 8 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
ˆx J (t) = P VJ [x(t)] + P WJ [x(t)] + P WJ [x(t)] = s J,n φ J,n (t) + w J,n ψ J,n (t) + w J,n ψ J,n (t) h [n] s J [n] h [n] s J [n] h [n] w J [n] h [n] w J [n] ˆx J (t) = P V [x(t)] + P W [x(t)] + + P WJ [x(t)] J = s,n φ,n (t) + w j,n ψ j,n (t) j= h [n] h [n] h [n] s [n] w [n] h [n] s J [n] h [n] w J [n] h [n] w J [n] 9 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
The discrete Haar transform The connection to filter banks above gives us a natural way to define a wavelet transform for discrete-time signals. Basically, we just treat x[n] like it was a sequence of scaling coefficients at fine scale, then apply as many levels of the filter bank as we like. So the following structure: h [n] s J [n] x[n] h [n] w J [n] takes x[n] and transforms it into two sequences, s J [n] and w J [n], each of which have half the rate of the input. How do we invert this particular transform? With another filter filter bank. Consider the following structure: h [n] s J [n] g [n] u[n] x[n] x[n] h [n] w J [n] g [n] v[n] 3 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
If we take g [n] = h [ n] = g [n] = h [ n] = { n =, otherwise n = n = otherwise, then we will have x[n] = x[n]. To see this, recall that and so s J [n] = (x[n] + x[n + ]), u[n] = { (x[n] + x[n + ]) n even, (x[n ] + x[n]) n odd that is, the values in u[n] appear in pairs, u[] = u[] = (x[] + x[]), u[] = u[3] = (x[] + x[3]), etc. Similarly, since w J [n] = (x[n] x[n + ]), we have v[n] = { (x[n] x[n + ]) n even, ( x[n ] + x[n]) n odd that is, the values in v[n] appear in pairs of ± terms, v[] = v[] = (x[] x[]), v[] = v[3] = (x[] x[3]), etc. 3 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7
Now it is easy to see that x[n] = u[n] + v[n] = x[n] for all n Z. We can repeat this to as many levels as we desire. For example, the following structure h [n] s J [n] h [n] x[n] h [n] w J [n] h [n] w J [n] takes x[n] and transforms it into three sequences, one of which is at half the rate of x[n], and the other two are at a quarter of the rate. To invert it, we simply apply the inverse filter bank twice: s J [n] g [n] g [n] w J [n] g [n] x[n] w J [n] g [n] It should be clear how to extend this to an arbitrary number of levels. 3 Georgia Tech ECE 65 Fall 7; Notes by J. Romberg and M. Davenport. Last updated :33, September 5, 7