Lecture 27. Wavelets and multiresolution analysis (cont d) Analysis and synthesis algorithms for wavelet expansions

Transcription

1 Lecture 7 Wavelets and multiresolution analysis (cont d) Analysis and synthesis algorithms for wavelet expansions We now return to the general case of square-integrable functions supported on the entire real line, i.e., f L (R). And to begin, we consider the following relations, V 0 V 1, V 1 = V 0 W 0. (1) Let f 1 V 1. We may consider f 1 to be the projection of a function f L (R). As time goes on, we shall consider approximations f j of higher resolution. Since f 1 V 1, it admits an expansion in the functions φ 1k (t), k Z, which span V 1. Recall that these functions are piecewise-constant over intervals of length 1/: f 1 = k Za 1k φ 1k, a 1k = f,φ 1k. () But from the relation V 1 = V 0 W 0, V 0 is also spanned by the basis functions φ 0k,ψ 0k, k Z. Therefore f 1 also admits the expansion f 1 = k Z a 0k φ 0k + k Zb 0k ψ 0k, a 0k = f,φ 0k, b 0k = f,ψ 0k. (3) The expansions in () and (3) represent the same function. The natural question is: Can we relate the coefficients a 1k to the coefficients a 0k and b 0k? The answer is Yes, as we now show. The key to providing a relation between these two sets of expansion coefficients lies in the scaling relations that are satisfied by the Haar scaling function φ and the associated Haar wavelet function ψ. Recall that these relations are given by φ(t) = φ(t)+φ(t 1), (4) and ψ(t) = φ(t) φ(t 1). (5) 318

2 We ll come back to these relations in a moment. To see how they will be useful, we shall first rewrite these two equations in terms of the scaling function φ(t): f 1 (t) = k Za 1k φ(t k), (6) and f 1 (t) = a 0k φ(t k)+ 0k ψ(t k). (7) k Z k Zb It is now a matter of comparing Eqs. (6) and (7). But first, we shall make another minor temporary change, for the purpose of having to work explicitly with the normalization factors j/. We introduce the following modified coefficients, whenever needed: A jk = j/ a jk, B jk = j/ b jk. (8) Eqs. (6) and (7) become, respectively, f 1 (t) = k ZA 1k φ(t k) (9) and f 1 (t) = A 0k φ(t k)+ 0k ψ(t k). (10) k Z k ZB The reader will note that the introduction of the A jk and B jk coefficients is analogous to the standard Fourier series expansion a n cosnx+b n sinnx, (11) where the 1/ π normalization factors are not explicitly written, but rather absorbed into the coefficients a n and b n. Important note: The A jk and B jk coefficients are used in the textbook by Boggess and Narcowich, but are denoted as a jk and b jk, respectively. We now return to the scaling equations (4) and (5). Solving for φ(t) and φ(t 1) by first adding the two and then subtracting them, we obtain φ(t) = 1 φ(t)+ 1 ψ(t) φ(t 1) = 1 φ(t) 1 ψ(t). (1) 319

3 Now replace t with t 1 in the above equations to obtain φ(t ) = 1 φ(t 1)+ 1 ψ(t 1) φ(t 3) = 1 φ(t 1) 1 ψ(t 1). (13) Do you see the pattern? Let s replace t in the first pair with t k to obtain φ(t k) = 1 φ(t k)+ 1 ψ(t k) φ(t k 1) = 1 φ(t k) 1 ψ(t k). (14) This pair of equations represents two cases, φ(t even) and φ(t odd). We shall now apply these results to Eq. (9). But we ll first change the index k to l and then split the single sum into separate sums over odd and even l: f 1 (t) = A 1,l φ(t l)+ A 1,l φ(t l). (15) l odd l even And now we ll express odd l as l = k+1 and even l as l = k, k Z: f 1 (t) = k Z We now use the equations in (14) to rewrite the above equation as A 1,k φ(t k)+ k ZA 1,k+1 φ(t k 1). (16) f 1 (t) = 1 A 1,k [φ(t k)+ψ(t k)]+ 1 A 1,k+1 [φ(t k) ψ(t k)] k Z k Z = 1 [A 1,k +A 1,k+1 ]φ(t k)+ 1 [A 1,k A 1,k+1 ]ψ(t k). (17) k Z k Z A comparison of this equation with Eq. (10) yields the following result A 0k = 1 [A 1,k +A 1,k+1 ] B 0k = 1 [A 1,k A 1,k+1 ]. (18) Once again, these are the relations presented in the book by Boggess and Narcowich, but using lower-case variables, i.e., a jk and b jk. We now return to the a jk and b jk expansion coefficients, recalling the definitions in (8): a 0k = 1 [a 1,k +a 1,k+1 ] b 0k = 1 [a 1,k a 1,k+1 ]. (19) 30

4 These relations define the so-called analysis algorithm: Given the set of expansion coefficients a 1k for the representation of a function f 1 V 1 in the V 1 basis set φ 1k, we may compute the coefficients a 0k and b 0k of the V 0 W 0 decomposition of V 1. For this reason, the analysis algorithm is also called the decomposition algorithm. We re well overdue for an example that will clarify all of these notions. Example: We now return to the special case of a function defined on a finite interval, namely, [0,1]. Consider the function f(x) = x on [0,1] and f(x) = 0 elsewhere. (Note that we ll use x to denote the independent variable in this example.) Here we shall examine its two expansions in V 1 and show that the relations in Eq. (19) are satisfied. You will recall that this function was examined earlier in the course (Lecture 6, Set ) in the context of projections onto subspaces of Hilbert spaces. You also examined this function in Problem Set No. 1. First of all, since we are considering the space V 1, we shall be examining the projection of f on V 1, which is the best approximation to f in V 1. We denote this projection as f 1. On [0,1], f 1 (x) will have two expansions, f 1 (x) = a 10 φ 10 (x)+a 11 φ 11 (x), (expansion in V 1 basis) (0) and f 1 (x) = a 00 φ 00 (x)+b 00 ψ 00 (x), (expansion in V 0 W 0 basis) (1) We shall work upwards in refinement, starting with the second expansion in V 0 W The first term in Eq. (1) corresponds to f 0, the projection of f onto V 0. f 0 (x) is the best constant approximation to f(x) on [0,1]. Thus f 0 (x) = a 00 φ 00 (x) = a 00 φ(x), () where Since φ(x) = 1 on [0,1], a 00 = x,φ = 1 0 x dx = 1 3. (3) f 0 (x) = 1 (4) 3 is the best constant approximation to f(x) = x on [0,1], as you found in Problem Set No

5 . The second term in Eq. (31) corresponds to f 0,d, the detail that must be added to f 0 in order to produce f 1 V 1. We compute the coefficient b 00 as follows, b 00 = x,ψ = = 1/ 0 [ 1 3 x3 1 x dx ] 1/ 0 1/ ] 1 [ 1 3 x3 x dx 1/ = 1 4. (5) 3. In summary, our function f 1 (x) is given by f 1 (x) = 1 3 φ(x) 1 ψ(x), (6) 4 from which we easily compute that f 1 (x) = = 1 1, 0 x < 1, (7) = 7 1, 1 x < 1. This is the result that you found in Problem Set No. 1, and which was also presented in Lecture We now compute the expansion of f 1 (x) in Eq. (0): a 10 = x,φ 10 = a 11 = x,φ 11 = 1/ 0 1 1/ x dx = 4 x dx = 7 4. (8) Substitution into Eq. (0) yields (acknowledging that φ 1k = on appropriate half-intervals), 4 = 1 1 f 1 (x) =, 0 x < 1, 7 (9) 4 = 7 1, 1 x < 1. in agreement with the previous computation. As you found in Problem Set 1, the values 1/1 and 7/1 correspond to the mean values of f(x) = x on the subintervals [0,1/] and [1/,1], respectively. This is, of course, what the V 1 decomposition performs. We now show that the analysis algorithm described earlier applies to this example. We start with the coefficients a 10 and a 11 in the V 1 decomposition of f. From these coefficients we should be able to 3

6 compute a 00 and b 00 of the V 0 W 0 decomposition using Eq. (19), where k = 0: [ ] a 00 = 1 [a 10 +a 11 ] = [ b 00 = 1 [a 10 a 11 ] = in agreement with the results of Steps 1 and. = 1 3 ] = 1 4, (30) The above calculation in which the analysis algorithm was used represents a situation generally encountered in practice, although at higher resolution levels. Generally, one is presented with digital data, which represent high-resolution samples of a function. From this data, it is desired to produce lower-resolution samples, along with the detail functions at each level. We ll discuss the reason for performing such decompositions in a later section. That being said, you may wonder, If we can decompose a signal in this way, can we reconstruct it? In the context of the above example, the question is as follows: Given the coefficients a 00 and b 00 (V 0 W 0 expansion) can we construct a 10 and a 11 (V 1 expansion)? The answer is Yes. It is just a matter of solving for the latter coefficients in Eqs. (18) and/or (19). With very little work we find, A 1,k = A 0k +B 0k A 1,k+1 = A 0k B 0k, (31) and a 1,k = 1 [a 0k +b 0k ] a 1,k+1 = 1 [a 0k b 0k ]. (3) These equations correspond to the synthesis or construction algorithm, in which you construct higherresolution approximations from lower-resolution data. It is easy to verify that these relations are satisfied for the example studied. 33

7 in agreement with the results of Step 4 above. a 10 = 1 [a 00 +b 00 ] = 1 [ ] = 4 4 a 11 = 1 [a 00 b 00 ] = 1 [ ] = 7 4 4, (33) Let us now revisit the example studied above but in another manner. Instead of focussing on the expansion coefficients a jk and b jk, however, we now consider the values of the function f 1 (x) on each interval, namely 1/1 and 7/1. Recall that each of these values is obtained by multiplying the appropriate coefficient a 0k by, which is the value of the basis function φ 0k (x). This comes from Eq. (0). But let s rewrite this equation in terms of the original Haar scaling function φ(x), first in terms of the a 0k and then in terms of the coefficients A 0k introduced earlier: f 1 (x) = a 10 φ 10 (x)+a 11 φ 11 (x) = a 10 φ(x)+a11 φ(x 1) = A 10 φ(x)+a 11 φ(x 1). (34) Since the Haar scaling function φ(x) = 1 on the interval [0, 1/), and is zero everywhere else, it follows that f 1 (x) = A 10 on [0,1/). Likewise, it follows that f 1 (x) = A 11 on [1/,1). We ll see later that this holds in general: The constant A jk corresponds to the value of the projection f j (x) on the subinterval [k/ j,(k + 1)/ j ). In the special case of the Haar scaling function, it can also be shown (Exercise) that A jk corresponds to the mean value of the function f(x) over this subinterval. Analysis and synthesis algorithms at general resolutions Previously, we showed how the two expansions of a function f 1 V 1 are related to each other. We can generalize this result to the decomposition V j = V j 1 W j 1 (35) for any j Z. Let f j V j, which may be the projection of a function f L (R) on V j. It can be written in two 34

8 ways, f j = k Za jk φ jk (expansion in V j basis), (36) and f j = k Z a j 1,k φ j 1,k + k Zb j 1,k ψ j 1,k (expansion in V j 1 W j 1 basis) = f j 1 +w j 1. (37) We shall proceed in the same way as we did in the previous lecture, first by expressing all functions in the above expansions in terms of the Haar scaling function φ(t) and wavelet function ψ(t): f j (t) = k Za jk j/ φ( j t k) = k ZA jk φ( j t k), (38) and f j (t) = a j 1,k (j 1)/ φ( (j 1) t k)+ j 1,k k Z k Zb (j 1)/ ψ( (j 1) t k) = A j 1,k φ( j 1 t k)+ j 1,k ψ( k Z k ZB j 1 t k), (39) where, as before, we have introduced the modified coefficients, A jk = j/ a jk, B jk = j/ b jk. (40) We now return to the following equations, derived in the previous lecture, φ(t k) = 1 φ(t k)+ 1 ψ(t k) φ(t k 1) = 1 φ(t k) 1 ψ(t k), (41) which, you will recall, represents two cases, φ(t even) and φ(t odd). In order to apply these equations to the expansions for f j (t), we ll have to make one change, i.e., to replace t with j 1 t in (41). Then split the sum in (38) into sums over even and odd indices, substitute the modified results for (41) into this split equation, collect terms and compare with (39). The final result is given in the following algorithms (Exercise): 35

9 Analysis or deconstruction algorithm for V j = V j 1 W j 1 (Haar wavelet basis) A j 1,k = 1 [A j,k +A j,k+1 ] B j 1,k = 1 [A j,k A j,k+1 ], (4) or, in terms of the a jk and b jk, a j 1,k = 1 [a j,k +a j,k+1 ] b j 1,k = 1 [a j,k a j,k+1 ]. (43) These results can easily be rewritten in order to express the higher resolution coefficients A jk in terms of the lower resolution coefficients A j 1,k and B j 1,k, etc.. The result is Synthesis or construction algorithm for V j = V j 1 W j 1 (Haar wavelet basis) A j,k = A j 1,k +B j 1,k A j,k+1 = A j 1,k B j 1,k. (44) or, in terms of the a jk and b jk, a j,k = 1 [a j 1,k +b j 1,k ] a j,k+1 = 1 [a j 1,k b j 1,k ]. (45) Note that we have written Haar wavelet basis in the above subtitles, in order to emphasize that the above results apply to this particular wavelet system. Analysis/synthesis algorithms will exist for other wavelet bases, but the forms of the equations, i.e., the coefficients, the number of coefficients on the RHS, will be different. 36

10 Practical application of analysis/synthesis algorithms We have arrived at one of the most important topics of this course regarding wavelet methods the use of the analysis/synthesis algorithms to (1) analyze signals and possibly () enhance (i.e., denoise, deblur) them. The following discussion is quite simplified, but it does provide a general picture of wavelet-based methods of signal/image analysis and processing. In what follows, we assume that we are presented with some kind of digital data, possibly obtained from the equal-time sampling of a continuous signal f(t). Because of the dyadic (i.e., powers of two) nature of the wavelet spaces, we ll also assume that the number of data points is N = J for some J > 0. So, as before, we assume that this digital sampling has produced the data points, f[0],f[1],,f[ J 1]. (46) Without loss of generality, we assume that these data points define the following piecewise-constant approximation to f(t) on [0,1]. f J (t) = J 1 k=0 f[k]φ( J t k), (47) where, once again, φ(t) denotes the Haar scaling function. Note our choice of the scaled functions φ( J t k) since φ( J 1, t [ k, k+1 ) t k) = J, J 0, otherwise. (48) By construction f J V J, the space of piecewise-constant functions on intervals of width J. In the previous section, we denoted the expansion of such functions as f J (t) = k ZA Jk φ( J t k). (49) In fact, at any level 0 j J, the projection of our original continuous function f on the space V j will have the form f j (t) = k ZA jk φ( j t k). (50) The main point is as follows: From the given V J resolution of our digital data, we may, using the analysis/deconstruction algorithm to progressively compute construct lower resolution projections V j 37

11 along with accompanying detail functions W j, until we stop at V 0 : f J (t) = f J 1 (t)+w J 1 (t) = f J (t)+w J (t)+w J 1 (t)... = f 0 (t)+w 0 (t)+w 1 (t)+ +w J 1 (t). (51) This may be done by performing the analysis algorithm on the A jk /B jk coefficients, starting with the input data, A Jk = f[k], k = 0,1,, J 1, (5) and producing A jk /B jk coefficients for j = J 1, then j = J, and ending at j = 0. At each step, the coefficients A jk define the projection f j V j via Eq. (50). The B jk coefficient define the detail function w j W j that must be added to f j to produce the approximation V j+1. This is essentially what was done in the figures presented in Lecture 3, where we started with the f 5 projection of a function f(t) and computed lower resolution projections f j and associated detail functions w j. But we want to go furtherthan that we would like to work with the wavelet coefficients a jk that define the detail functions w j. So we can proceed in two ways: 1. Starting with the values A Jk = f[k], 0 k J 1, perform the analysis algorithm in the A jk /B jk scheme to produce the coefficients A jk, 0 k j 1 for j = J 1,J,,0. Then construct the wavelet coefficients a jk from these coefficients from their connection: a jk = j/ A jk. (53). Start with the values a Jk = J/ A Jk = J/ f[k], perform the analysis algorithm in the a jk /b jk scheme to produce directly the wavelet coefficients a jk, 0 k j 1 for j = J 1,J,,0. It s really not a big deal which approach you apply the first involves the factor 1/ and the second 1/. The important point is that you produce the wavelet coefficients a jk. It is on these coefficients that wavelet-based signal/image processing schemes can then be applied. At each stepj, theanalysisalgorithm takes the j scalingcoefficients a jk andproduces j 1 scaling coefficients a j 1,k and j 1 wavelet coefficients b j 1,k, for a total of j coefficients. The total number of 38

12 coefficients remains unchanged. You then repeat this procedure for the next smallest j value, until you must stop at j = 1, where you have produced a 00 and b 00. This analysis or deconstruction procedure is illustrated schematically in the figure on the next page. {a Jk } {a J 1,k } {b J 1,k } {a J,k } {b J,k } {a 1,k } {a 0,0 } {b 0,0 } Schematic to illustrate the analysis or deconstruction procedure, starting with the scaling coefficients a Jk, 0 k J 1, to produce the wavelet coefficients b jk. The result of the analysis procedure is to produce a 00 along with the wavelet coefficients b jk, j = 0,,J 1, k = 0,, j 1, which may be arranged as follows: a 00 b 00 b 10 b 11 b 0 b 1 b b b J 1,0 b J 1,1 b J 1, J 1 1 Note that the final row is comprised of the wavelet coefficients {b J 1,k}: You cannot construct the level b Jk from the f J resolution. Once again, at the risk of over-repetition: This tree of coefficients represents the following decomposition of a function f J V J : V J = V 0 W 0 1 W J 1. (54) 39

13 The element a 00 represents the function f 0 V 0 since, as you will recall, a 00 = f J,φ = the mean value of f J on [0,1]. 1 0 f J (t) dt, (55) 330

14 Lecture 8 Wavelets and multiresolution analysis (cont d) Of course, the question remains, So what can we do with these coefficients? As mentioned earlier, they may be useful in analyzing a signal/image, for example, its frequency components. In general, wavelet functions are localized, so the analysis of signals can be performed locally, unlike the situation with Fourier transforms (unless we break up the data into blocks). Secondly, various enhancement procedures may be performed in the wavelet domain, e.g., denoising, deblurring, edge enhancement. Once again, the localized nature of wavelet basis functions can be advantageous here. One of the fundamental properties of wavelet expansions that permits analysis and/or enhancement is the decay of (the magnitude of) wavelet coefficients a jk as the resolution level j increases. The resolution level is also referred to as the frequency since the wavelet functions ψ jk necessarily oscillate more rapidly with increasing j this follows from the scaling relation ψ jk = j/ ψ( j t k). That being said, some wavelet coefficients will decay more slowly than others, because of particular features in the signal/image being represented. To illustrate this, let us return to the example presented in Lecture : the function f(x) on [0,1] defined as follows, 8(x 0.6) +1, 0 x < 0.6, f(x) = 8(x 0.6) +3, 0.6 x < 1. (56) A plot of f(x) along with its approximation f 5 V 5 is shown in the next figure. Obviously, f(x) is discontinuous at x = 0.6. The wavelet coefficient tree constructed from f 5 originally presented in Lecture 3 is presented below the graph. The decay of the wavelet coefficients with increasing frequency is quite evident from this display. But another noteworthy feature of this coefficient tree is how the coefficients b 49 = 0.0, b 34 = 0.13 stand out from the other coefficients in the levels j = 4 and j = 3, respectively. Recalling that for the Haar wavelet basis, the location of the box enclosing the wavelet coefficient b jk actually corresponds to the support of the associated wavelet function ψ jk, we see that these coefficients are associated with wavelets that overlap with the discontinuity at 0.6. The support of ψ 49 is [9/16,10/16) = [0.565,0.65), and contains the discontinuity. 331

15 x Graph of f(x) defined in text, along with its approximation f 5 (x). a 00 =.55 b 00 = 0.40 b 10 = 0.49 b 11 = 0.49 b 0 = 0.4 b 1 = 0.11 b = 0.41 b 3 = Wavelet coefficient tree associated with f 5 approximation of f(x) above. These two coefficients, along with b = 0.41 are said to comprise a significant (wavelet) tree. Significant trees are produced by discontinuities or cusps, i.e., irregularities, of the function f(t). In the two-dimensional case, i.e., images, they are associated primarily with edges. In the case of images, it is generally desired that an enhancement scheme preserve edges, i.e., that it not alter them significantly, for example, by blurring. This is because the edges of an image contain a great deal of information in the image, being associated with boundaries of objects, etc.. The human visual system is also quite sensitive to degradations in edges. In some cases, observers are known to prefer noisy images over denoised images in which significant edges (e.g., the outline of a face) have been distorted. Wavelet-based compression As mentioned earlier, signals and images generally possess a great deal of redundancy because of correlation within the data. For example, if a signal is continuous, then its value at f(t + t), for t sufficiently small, will be close to f(t). As such, we might not have to store the two signal values: 33

16 storing f(t) and f(t) = f(t+ t) f(t) will suffice to reconstruct f(t+ t), possibly reducing the storage requirements, especially if we can do this for a number of consecutive signal values. This is one example of data compression. In our discussion of the discrete Fourier transform, we saw that a good number of DFT coefficients are small in magnitude, essentially insignificant. If they are discarded, there is little error incurred in the modified signal. The same idea holds with wavelet expansions. There can be many coefficients that are insignificant and which can be discarded without affecting the quality of the signal appreciably. Of course, the more compression desired, the more coefficients we must discard, resulting in an increased error, or decreased fidelity. For example, in the wavelet coefficient table above, we could decide to discard all coefficients that are zero to two decimals. Wavelet-based denoising Denoising in the wavelet domain is based upon the same general principle as Fourier-based methods discussed earlier. Very loosely put, in the case of additive noise, the noise will appear over the entire wavelet coefficient tree not just at high frequencies but at all frequencies. But since the wavelet coefficients decay in magnitude with increasing frequency, the noise will be more dominant at higher frequencies, until it essentially makes up most, if not all, of the signal there. A simple strategy, therefore, is to eliminate these higher frequency coefficients. The thresholding of wavelet coefficients using a single threshold T > 0 can be performed, but it is not optimal. More sophisticated schemes employ different thresholds T j at the different resolution levels, acknowledging the general decay in magnitudes with j. Hopefully we can return to a discussion of some methods later in the course. Reconstruction/synthesis from the wavelet expansion After having constructed the wavelet coefficient tree, and possibly making modifications to it for the purpose of denoising or whatever, the question remains, How do we reconstruct our possibly modified function f(t)? The answer is, via the synthesis/construction algorithm. You simply work backwards. If you go back to the equations, you ll see that the synthesis algorithm produces only a jk coefficients. But it needs a j 1,k and b j 1,k coefficients. Very quickly, the procedure goes as follows: 1. Step j = 1: Start with a 00 and b 00, representing V 0 and W 0, and construct a 10 and a 11. You ve now constructed the V 1 representation. 333

17 . Step j = : Take the a 1k coefficients and, along with b 10 and b 11, compute the coefficients a,k, 0 k 3. This is the V representation. 3. Step j: Take the a j 1,k coefficients computed in the previous step and, along with the b jk, compute the coefficients a jk, 0 k j 1. This is the V j representation. 4. Step J: Thisis thefinalstep, in whichthecoefficients a Jk arecomputed. From thesecoefficients you compute A Jk = J/ a Jk, the discrete values of the function f J. Example: The first four coefficients in the table presented earlier, to five digit precision, are a 00 =.54666, b 00 = , b 10 = , b 11 = (57) This corresponds to the V projection of f(t), i.e., J =, since V = V 0 W 0 W 1. (58) The above coefficients may be used to compute this projection f (t), as defined by the coefficients a k, 0 k 3. Using the synthesis algorithm, we must first compute a 1k (Step j = 1): We now use these to compute a k (Step j = ): a 10 = 1 [a 00 +b 00 ] = , a 11 = 1 [a 00 b 00 ] = (59) a 0 = 1 [a 10 +b 10 ] = , a 1 = 1 [a 10 b 10 ] = , a = 1 [a 11 +b 11 ] = , (60) a 3 = 1 [a 11 b 11 ] = (61) The four function values f [k] (J = ) assumed by f on [k/4,(k +1)/4) are A 0 = a 0 J/ = a 0 = A 1 = a 1 J/ = a 1 = A = a J/ = a =.466 A 3 = a 3 J/ = a 3 = (6) 334

18 These results are in agreement with the plot of the projection f V of this function presented in last week s lecture. The plot is presented below x f V Note that this is as far as we can go! We were given four initial data values in Eq. (57). From this data set, we can construct only four function values. We cannot construct f 3, the projection of f onto V 3, since this would require 8 initial data values. We would need the additional four wavelet coefficients b 0, b 1, b andb 3 inordertoconstruct, withthehelpofthea i coefficients, thecoefficients a 3j defining f 3 V

19 Multiresolution analysis: A general treatment We now turn to a more general mathematical treatment of multiresolution analysis (MRA) and wavelets. You have seen most of the ideas presented below in the context of the Haar system, a particular example of an MRA. Indeed, the Haar system was a perfect starting point since the main ideas of MRA could be presented in a rather easy way. A more general mathematical treatment will show that many wavelet systems are possible. Multiresolution analysis Let V j, j =,, 1,0,1,,, be an infinite sequence of closed subspaces of the function space L (R). This collection {V j } is called a multiresolution analysis with scaling function φ if the following conditions hold: 1. nesting: V j V j+1 for all j Z. You ve already seen this property in the context of the Haar system, where V j is the set of all L (R) functions that are piecewise-constant on intervals [k/ j,(k +1)/ j ). For other MRA s, the definition of the V j will be different.. density: V j = L (R). j Z This essentially states, in proper set-theoretic language, that lim V j = L (R). j 3. separation: V j = {0}. j Z The only element common to all subspaces V j is the zero function. Think about it in the context of the Haar system: Here, the space V j consists of all piecewise-constant functions over intervals of length j. As j, these intervals get larger and larger. The only L (R) function that is piecewise-constant over the entire real line R is the zero function. 4. scaling: f(x) V j f(x) V j+1. This provides a connection between consecutive subspaces in terms of scaling: When you contract a function f V j in the x-direction toward the y-axis by a factor of, 336

20 you have produced a new function that lies in the next higher refinement space V j+1. A mathematical statement equivalent to the above is (Exercise): f(x) V j f( j x) V 0. Subspaces V j satisfying 1-4 are known as approximation spaces. 5. orthonormal basis: The function φ V 0, and the set of functions {φ(x k),k Z} is an orthonormal basis of V 0 (with respect to the inner product in L (R). Different MRAs will have different scaling functions, φ, and different systems of subspaces V j. In fact, as we shall see below, we need only to specify a scaling function φ and the rest will follow. Once again, we remind ourselves of the Haar MRA which is determined by the scaling function 1, 0 x < 1. φ Haar (x) = 0, otherwise. (63) The space V 0 spanned by integer translates {φ Haar (x k),k Z} is naturally the space of L (R) functions that are piecewise-constant over the intervals [k,k + 1), k Z. The fact that the space V j is composed of functions that are piecewise-constant over the intervals [k/ j,(k+1)/ j ) naturally follows. In what follows, we consider a general multiresolution analysis (MRA). Theorem: Let {V j } be an MRA with scaling function φ. Then for any j Z, the set of functions, {φ jk = j/ φ( j x k),k Z}, (64) forms an orthonormal basis of the subspace V j. Proof: We must show that any function f V j is expressible as a linear combination of functions φ jk definedabove. Letf V j. Fromthescalingproperty(4)forMRA,itfollowsthatg(x) = f( j x) V 0. But from property (5) for MRA, the functions φ(x k) form an orthonormal basis of V 0, implying that g(x) admits an expansion of the form g(x) = k Za 0k φ(x k), a 0k = g(x),φ(x k). (65) 337

21 We rewrite this result as follows, f( j x) = k Za 0k φ(x k). (66) Now let x = j t, t R, to give f(t) = k Z a 0k φ( j t k) = k Za 0k j/ φ jk. (67) Thus, f is a linear combination of the functions φ jk. It remains to show that the φ jk form an orthonormal set: For k,l Z, φ jk,φ jl = j/ φ( j x k) j/ φ( j x l) dx R = j φ( j x k)φ( j x l) dx. (68) Now let t = j x, so that dt = j dx, etc.. Then φ jk,φ jl = R R φ(x k)φ(x l) dx = δ kl, (69) since the {φ(x k)} form an orthonormal basis in V 0. The orthonormality of the {φ jk } has thus been established. Note: Yes, the above result is quite simple. Nevertheless, it is a very powerful one, and will be used to establish other important results. It characterizes the fundamental property of scaling for MRA s. The Scaling Equation Sincethescalingfunction φ V 0 V 1 and, fromtheprevioustheorem, theset{φ 1k (x) = φ(x k)} forms an orthonormal basis of V 1, it follows that φ admits an expansion in this basis: φ(x) = k Zh k φ 1k (x), (70) where h k = φ,φ 1k. (71) 338

22 This equation is usually expressed in the following form, φ(x) = k Zh k φ(x k), (7) known as the scaling equation or two-scale relation for the scaling function φ of the MRA. Since the expansion is in the L -sense, the coefficients h = {h k } form a square-summable sequence in l. In fact, it is easy to see, from the orthonormality of the φ 1k functions, that φ,φ = 1 = h k. (73) k Z Recall that for the Haar system h 0 = h 1 = 1, h k = 0 otherwise, (74) which leads to the familiar scaling equation, φ(x) = φ(x)+φ(x 1). (75) The scaling equation is a fundamental equation for MRA. In essence, it may be viewed as a definition of the scaling function φ via the expansion coefficients h k. Not any set of coefficients {h k } will do recall that there is the requirement that the integer translates of φ(x) are orthogonal to each other. There are some other constraints that we shall discuss later. Just to give you an idea of another MRA, here are the coefficients that define the so-called Daubechies-4 wavelets, h 0 = , h 1 = , h = 3 3 4, h 3 = 1 3 4, h k = 0 otherwise. (76) They are named after Ingrid Daubechies, a mathematician at Princeton University who is well-known for her many fundamental contributions to wavelet theory, including the construction of wavelets with prescribed regularity (i.e., continuous, k-times differentiable), including the Daubechies-4 MRA. (There is a family of Daubechies-n wavelets, where signifies the number of non-zero expansion coefficients h k.) In the figure below is plotted the Daubechies-4 scaling function φ(x) the scaling function that satisfies Eq. (7) for the four non-zero h k coefficients in (76). At first sight, it must look rather 339

23 x Scaling function φ(x) for the Daubechies-4 MRA, with coefficients h k given in (76). strange, with its jaggedness. That being said, it is, in some ways, an enormous improvement over the Haar scaling function since it is a continuous function! What may appear to be even more remarkable is that integer translates of this function, i.e., φ(x k) are orthogonal to each other. We plot the graphs of φ(x) and its translate φ(x 1) on the same set of axes below the reader is invited to reason geometrically how the inner product of these two functions is zero x Scaling function φ(x) for the Daubechies-4 MRA and its translate φ(x 1). 340

24 Lecture 9 Multiresolution analysis: A general treatment (cont d) Wavelets with finite support You will also note that the support of the Daubechies-4 scaling function appears to be finite, which indeed it is. (In other words, there is a finite interval [a,b] such that φ(x) = 0 for all x / [a,b].) This is, of course, also the case for the Haar scaling function. Wavelets with finite support are very useful in the analysis and processing of signals and images. The localization of these functions increases with frequency, i.e., their supports become smaller, permitting finer detection of important features. You have already seen an example of such detection in the case of Haar wavelets, where the magnitudes of the coefficients in a given band exhibited local maxima in the vicinity of a discontinuity. We shall return to this phenomenon later. An important result: Theorem: If the support of the scaling function φ(x) is finite, then only a finite number of the coefficients h k can be nonzero. Proof: Supposethat φ(x) = 0outside the interval [ a,a], wherea > 0 is finite. Also let k 1 < k < be an infinite sequence of integers for which h ki 0. Now suppose that φ(p) 0 for some p [ a,a]. Then from the scaling equation, φ(x) = k Zh k φ(x k), (77) it follows that there will be nonzero contributions to the right hand side at the points x i R defined by x i k i = p, i = 1,,, implying that the values φ(x i ) are nonzero. But a rearrangement yields x i = 1 (p+k i), implying that x i as i. This contradicts the assumption that φ(x) is zero outside the interval [ a, a]. The scaling equation and self-similarity The scaling equation (7) shows that the graph of φ(x) may be expressed as a linear combination of contracted and translated copies of itself (in the x direction) which are also modified by the coefficients 341

25 h k. This may be viewed as a kind of self-similarity property of the scaling function φ(x). Self-similarity is a well-known concept in the context of fractal sets, which you may have seen in popular expositions of mathematics. For example, the Sierpinski gasket, shown below, is an example of a fractal set. (Its topological dimension is, since it is a union of continuous curves, but it s Hausdorff dimension is log3/log because of its scaling property. In other words, it is thicker than a curve but not as thick as the plane.) This set is self-similar in that parts of it are contracted copies of the entire set. Moreover, it may be viewed as a union of contracted copies of itself. For example, it may be expressed as a union of three copies that are contracted by factors of in the x and y direction and positioned so that appropriate vertices touch. The Sierpinski gasket fractal set. The scaling equation (7) is a functional version of self-similarity you take a function such as φ(x), make contracted copies of its graph in the x-direction, vary the y-values of each of these graphs, then translate them, and finally add them up. If the result is φ(x), then φ(x) is said to be self-similar. Wavelet spaces Previously, we discussed the scaling equation for the scaling function φ(x). We now discuss an analogous equation for the associated wavelet function ψ(x). Once again, we start with the facts that φ V 0 and V 0 V 1. As we did last week, we define the 34

26 subspace W 0 V 1 that is orthogonal to V 0, or the orthogonal complement of V 0, defined as follows, W 0 = V 0 = {x V 1 x,y = 0 for all y V 0 }. (78) It can be shown that W 0 is a closed linear subspace if {x n } is a Cauchy sequence in W 0, then it has a limit x W 0. Recall that integer translates φ(x k) of the scaling function formed an orthonormal basis of V 0. We now look for a function ψ W 0 such that its integer translates ψ(x k) form an orthonormal basis or W 0. Since ψ W 0 V 1, it will admit an expansion in the φ 1k basis of V 1. We shall first write this expansion as follows, ψ(x) = k Zg k φ 1k, (79) and then write it in two-scale form, ψ(x) = k Zg k φ(x k). (80) It is helpful to recall the associated expansions for the scaling function φ(x), i.e., φ(x) = k Zh k φ 1k (81) and Note the similarity in form of these two sets of equations. φ(x) = k Zh k φ(x k), (8) Returning to the wavelet function, since ψ W 0, we must have that ψ,φ = 0. (83) The above inner product is computed easily, using the expansions in (79) and (81) in the V 1 basis: ψ,φ = k φ 1k k Zg, h l φ 1l l Z = g k hl φ 1k,φ 1l k Z l Z = k Zg k hk, (84) 343

27 where the final line follows from the orthonormality of the φ 1k basis functions. From the orthogonality constraint in Eq. (83), the g and h sequences must obey the following condition, g k hk = +g 1 h 1 +g 0 h0 +g 1 h1 +g h + = 0. (85) k Z Here is a very cheap trick that works: Set g k = ( 1) k h1 k, k Z, (86) so that, for example, g 1 = h, g 0 = h 1, g 1 = h 0, g = h 1, etc.. (87) Substitution into (85) yields h h 1 + h 1 h0 h 0 h1 + h h 1 + = 0. (88) In other words, this clever trick produces a cancellation of pairs of terms in the sum. To review, given the scaling function with expansion φ(x) = k Zh k φ(x k), (89) we have produced a function ψ(x) that is orthogonal to φ(x) with expansion ψ(x) = k Z( 1) k h1 k φ(x k). (90) The function ψ(x) is known as the mother wavelet function or, simply, the wavelet function of the MRA defined by the h k coefficients. For the remainder of the course, we shall be working with real-valued expansions, in which case it is not necessary to use the complex-conjugate notation. As such, we shall simply write ψ(x) as follows, ψ(x) = k Z( 1) k h 1 k φ(x k). (91) Recall that for the Haar MRA case, the h k scaling coefficients are h 0 = 1, h 1 = 1, h k = 0 otherwise, (9) 344

28 implying that the g k coefficients for the wavelet function are g 0 = ( 1) 0 h 1 = 1, g 1 = ( 1) 1 h 0 = 1, g k = 0 otherwise. (93) This leads to the following equation for the Haar mother wavelet, which we have already encountered. ψ(x) = φ(x) φ(x 1), (94) We now return to the Daubechies-4 multiresolution analysis system defined by the h k coefficients in Eq. (76). The associated wavelet function ψ(x) is plotted in the figure below. Because the scaling function φ(x) and its dilations/translations φ(x k) are continuous, the function ψ(x), a linear combination of the latter, is continuous x Mother wavelet function ψ(x) for the Daubechies-4 MRA, with coefficients h k given in (76). A note on the cheap trick given above: We must acknowledge here that the method given above produced a particular solution to Eq. (85), namely Eq. (86). Clearly, Eq. (85) represents only one equation in several, and possibly an infinite number of, unknowns, g k, so a unique (nonzero) solution will not exist. In fact, we may multiply all of the g k in Eq. (86) by a nonzero constant C to produce solutions that satisfy (85), leading to constant multiples of the wavelet function ψ(x). In fact, it can be shown (and will be a little later) that the following is also a solution, g k = ( 1) k hp+1 k, k Z, (95) for any integer p, representing even-integer translations of the vector g k. These solutions correspond to integer translations of the wavelet function ψ which, as we will discover, are also orthogonal to φ(x). 345

29 Are there any other nontrivial (i.e., nonzero) solutions? There may be but they are probably specific to the set of h k coefficients of concern. The cheap trick producesa solution that is applicable to all MRA s, producing the desired wavelet function ψ(x). We now continue with our discussion of multiresolution analysis. Theorem: Thesetoffunctions{ψ 0k = ψ(x k)}, whereψ isdefinedineq. (90), formsanorthonormal basis of W 0. Proof: We must divide the proof into several steps. 1. We first prove that the set {ψ 0k = ψ(x k),k Z} is orthonormal. For any k,l Z, we ll compute the inner product, ψ 0k,ψ 0l = ψ(x k),ψ(x l). (96) It will be convenient to expand these functions in terms of the orthogonal functions φ(x l). From Eq. (80), we have ψ(x k) = m Zg m φ(x k m). (97) Substitution into Eq. (96) yields ψ(x k),ψ(x l) = g m φ(x k m), g n φ(x l n) m Z n Z = m Zg m g m+k l (k +m = l +n n = m+k l) = m Z( 1) m h 1 m ( 1) m h 1 m k+l = m Zh 1 m h 1 m k+l. (98) We shall now show that the above expression is zero for k l and 1 for k = l. To do this, we need to look at the corresponding orthogonality relations for the translated scaling functions, φ(x k),φ(x l) = h m φ(x k m), h n φ(x l n) m Z n Z = m Zh m h m+k l (k +m = l +n n = m+k l) = δ kl. (99) 346

30 Replacement of m with 1 m and substitution into (98) yields the desired result ψ(x k),ψ(x l) = δ kl. (100). We now prove that the set {ψ 0k (x) = ψ(x k), k Z} is orthogonal to V 0. It is sufficient to show that φ(x k),ψ(x l) = 0. (101) From the scaling results, φ(x k),ψ(x l) = h m φ(t k m), g n φ(t l n) m Z n Z = m Zh m g m+k l (k +m = l+n n = m+k l) = m Z( 1) m+k l h m h 1 m k+l = m Z( 1) m h m h 1 m k+l. (10) We shall need one more result, ( 1) m h m h 1 m+p = 0, for any p Z. (103) m Z We have already shown that the above is true for p = 0: The series becomes a cancelling sum about the terms h 0 h 1 and h 1 h 0. Recall that this was the basis of the definition of the g k in terms of the h k. But for a shift of p in one of the arguments, it is also a cancelling sum about m = p and m = p+1: ( 1) m h m h 1 m+p = +( 1) p h p h 1+p +( 1) p+1 h p+1 h p + m Z = 0. (104) Since k+l is an even number, it follows that φ(x k),ψ(x l) = 0. (105) 3. We now prove that any element y W 0 admits an expansion in the functions ψ(x k). The proof is performed in a somewhat roundabout way. We ll show that the space V 1 is spanned by integer translates of φ and corresponding translates of ψ. Recall the fact that the functions φ 1k = 1/ φ(x k) span V 1. We must show that for each j, φ(x j) = k a k φ(x k)+b k ψ(x k), (106) 347

31 for an appropriate set of constants a k and b k. (These constants will also depend on j, but we ll omit this index for simplicity of notation.) From the orthogonality of the φ(x k), it follows that Likewise, we find that a k = φ(x j),φ(x k) = φ(x j), h l φ(x k l) l = 1 h j k (j = k +l l = j k). (107) b k = φ(x j),ψ(x k) = φ(x j), g l φ(x k l) l = 1 g j k = 1 ( 1) j h 1 j+k. (108) If the equality in (106) holds, in the L -sense, then the following result must hold: φ(x j),φ(x j) = 1 = k [ ak + b k ]. (109) From the previous equations, [ ak + b k ] = 1 [ h j k + h 1 j+k ]. (110) k k Regarding the right-hand side, if j is odd, then 1 j is even; if j is even, then 1 j is odd. As a result, But from the scaling equation (7), we have 1 [ h j k + h 1 j+k ] = 1 h l. (111) k l φ(x),φ(x) = k h k = 1. (11) Therefore Eq. (109) is verified and the expansion in (106) holds. In summary, any element u V 1 admits a unique expansion in terms of the functions φ(x j) which, in turn, admit unique expansions in terms of the φ(x k) and ψ(x k) functions. Since the φ(x k) span V 0, it follows that the ψ(x k) span W

32 The above result is now easily extended to the space W j by replacing x by j x in the above equations and employing the orthogonality of the φ( j x l) functions. We then have the following important result: Theorem: For any j Z, the set of functions ψ jk (x) = j/ ψ( j x k) forms an orthonormal basis of the subspace W j. Summary of results: 1. The scaling function φ(x) satisfies the relation φ(x) = k Zh k φ(x k). (113). By assumption, the set of functions φ 0k (x) = φ(x k), i.e., the set of all integer translates of φ(x), span the space V 0. From this result, and the scaling property of MRA s, it follows that the set of functions φ jk = j/ φ( j x k) forms an orthonormal basis of V j. 3. The space W 0 V 1, W 0 = V0, is spanned by the function ψ(x) = k Zg k φ(x k), (114) where g k = ( 1) k h 1 k. 4. In general, V j+1 = V j W j and the set of functions ψ jk = j/ ψ( j k) forms an orthonormal basis of W j. 349