The organization of the paper is as follows. First, in sections 2-4 we give some necessary background on IFS fractals and wavelets. Then in section 5

Size: px

Start display at page:

Download "The organization of the paper is as follows. First, in sections 2-4 we give some necessary background on IFS fractals and wavelets. Then in section 5"

Virgil Mason
5 years ago
Views:

1 Chaos Games for Wavelet Analysis and Wavelet Synthesis Franklin Mendivil Jason Silver Applied Mathematics University of Waterloo January 8, 999 Abstract In this paper, we consider Chaos Games to generate wavelet decompositions and wavelet synthesis of functions. The primary technique involved is mixing the several Chaos Games for the individual wavelets in order to get the overall wavelet expansion. Generalizations to dilation factors other than two and to multidimensional wavelets are given. Keywords: Wavelets, Iterated Function Systems, Chaos Game, Ergodic Theorem. Introduction There are many strong connections between the study of wavelets and fractals. For example, the scaling function satises a dilation equation (equation (8) which is a type of fractal self-similarity. Using this connection, Berger [2, 3] adapted the classical Chaos Game algorithm from IFS fractal theory to render the scaling function and mother wavelet. This allows one to compute function values for the mother wavelet. This paper starts with Berger's Chaos Game algorithm and modies and extends it. The basic modication is to use the algorithm to compute integrals rather than function values. These integrals can then be used to compute the wavelet expansion coecients for an arbitrary L 2 function f. This is the Wavelet Analysis. On the other hand, by carefully \mixing" appropriate Chaos Game algorithms, we can start with the wavelet expansion coecients and produce a rendering or approximation of the function f. This is the Wavelet Synthesis.

2 The organization of the paper is as follows. First, in sections 2-4 we give some necessary background on IFS fractals and wavelets. Then in section 5 we discuss the vectorized version of the dilation equation and the Chaos Game for generating the scaling function. In section 6 we modify this Chaos Game and obtain a Chaos Game for rendering the wavelets. Section 7 presents the parallel Chaos Games necessary to generate the coecients in a wavelet expansion of a function while section 8 discusses \mixing" Chaos Games to generate a wavelet synthesis of a function. Finally, in section 9 we briey discuss the extension of these ideas to more general dilation equations { to dilation equations with dilation factors other than 2 and to multidimensional dilation equations and wavelet expansions. 2 Iterated Function Systems Here we briey review the central concepts of IFS in order to provide a framework in which to the discuss the main issues of this article. Let (; d) be a compact metric space and fw i g N i= be a nite collection of contractive maps with w i :!. The pair (; fw i g) is called an Iterated Function System (or IFS), though usually we refer to the maps alone as an IFS. Dene the parallel action of the maps w i by w(s) = N[ i= w i (S): () Letting h denote the Hausdor distance on the metric space, we have ([, ]) Proposition There is a unique compact set A, the attractor of the IFS, such that A = N[ i= w i (A) (2) and h(w n (S); A)! 0 as n!, for every non-empty compact set S. If we associate with each w i a non-zero probability, p i, then the triad f; fw i g N ; fp i= ig N g i= is known as an Iterated Function System with Probabilities, or IFSP. Let B be the algebra of Borel subsets of and let P be the set of all probability measure on. The Monge-Kantorovich metric (often called the Hutchinson metric in the IFS literature) on P is dened as d(; ) = sup f2lip f fd fdg for all ; 2 P (3) where Lip = ff :! IR : jf(x) f(y)j d(x; y)g is the set of Lipschitz functions with Lipschitz factor. 2

3 Now we dene the following Markov operator on the space P, M() = N i= p i w i : (4) For this operator we have the following proposition (see [, ]). Proposition 2 There is a unique measure 2 P, called the invariant measure, such that M() = (5) Further, supp() = A 3 Rendering and approximating attractors of IFS There are two basic algorithms for generating the attractor of a given IFS. One is a deterministic algorithm, the other is a stochastic algorithm, known as the Chaos Game (see []). Given an IFS with N contractive maps fw i g N i=, we may approximate the attractor deterministically as follows:. Pick x 0 2 and form the set S 0 = fx 0 g and x an > 0 2. Form the set S n = w(s n ), 3. If h(s n ; S n ), goto step Plot S n. This algorithm is essentially computing the xed the point of the set-valued function w through a direct iteration process. At the n th iteration there are N n points. When rendering attractors on a computer screen the algorithm could terminate once resolution of the screen prevents any further detail from being revealed; it is this consideration which typically sets. Given an IFSP with N contractive maps fw i g N i=, with associated probabilities fp i g N i= we may approximate the attractor stochastically as follows:. Pick x 0 2 to be the xed point of w (this guarantees that all points determined henceforth in this algorithm will lie on the attractor). 2. Pick n 2 f; : : : ; Ng at random according to the probabilities fp i g N i= and set x n = w n (x n ) 3. Plot x n 4. If n is less than the maximum number of iterations, go to step 2. 3

4 Elton [9] proved the following theorem, commonly known as Elton's Ergodic Theorem, related to the Chaos Game. Proposition 3 Let f; fw i g N, fp i= ig N i=g be an IFSP with invariant measure. Then for every starting point x 0 2 and almost every sequence f n g generated by the Chaos Game algorithm, we have lim n! n n k= f(x k ) = for all continuous and simple functions f :! IR. f(x) d(x) (6) Suppose that we set f = B, the characteristic function of B 2 B. Then from Elton's theorem, # fx n 2 Bg lim = lim n! n n! n n k= B (x k ) = (B): (7) This explains why the Chaos Game generates an approximation to the attractor. For each set B so that (B) > 0, we know that eventually x n 2 B so we will eventually plot a point in B. Thus, the Chaos Game will eventually plot each \pixel" in A. 4 Necessary wavelets background In this section we discuss those parts of wavelet theory that will be needed for the purposes of this paper. For a more complete introduction, see the very nice book [7]. Let be a scaling function for a MultiResolution Analysis. Then f( i)ji 2 g is an orthonormal set and satises a dilation equation of the form (x) = n2 h n (2x n): (8) We will assume that the only non-zero coecients h n are h 0 ; h ; : : : ; h N. In this case, it can be shown that is supported on the interval [0; N]. Associated with the scaling function there is a function, called the mother wavelet, which is dened by (x) = k ( ) k h N k (2x k): (9) One can show that is also supported on [0; N] with this denition. Dene i;j(x) = 2 i=2 (2 i x j). Then it turns out that the set f i;j j i; j 2 g forms an orthonormal basis for L 2 (IR). The important feature for us is that this basis is generated by the dilations and translations of a single function { the mother wavelet. 4

5 5 Rendering a Compactly Supported Scaling Function In general, a compactly supported scaling function as dened above does not have a closed analytic form. In the following we will discuss an IFS type algorithm to approximate the graph of to arbitrary accuracy. Let us suppose that we have chosen a sequence of h n 's, subject to the our assumption that h n = 0 for n < 0 and n > N, in such a way that induces a multiresolution analysis on L 2 (IR). Daubechies and Lagarias [8] and Micchelli and Prautzsch [3] independently noticed that one could vectorize the dilation equation. Dene V : [0; ]! IR N as V (x) = 0 (x) (x + ). (x + N ) C A (0) Dene the N N matrices T 0 and T by (T 0 ) i;j = h 2i j and (T ) i;j = h 2i j. That is, 0 h T 0 h 2 h h = C. A h N h N and T = 0 C h h h 3 h 2 h A : h N Let : [0; ]! [0; ] be dened as (x) = 2x mod. Then the dilation equation becomes (assuming that is continuous) V (x) = T! V (x) () where! is the rst digit in the binary expansion of x. As an example, we compute the transformation T 0 corresponding to the Daubechies-4 wavelet. In this case only h 0 ; h ; h 2, and h 3 are non-zero, so is supported on the interval [0; 3]. Thus V (x) = (x) (x + ) (x + 2) A (2) 5

6 From the dilation equation, we have the equations (x) = h 0 (2x) + h (2x ) + h 2 (2x 2) + h 3 (2x 3) (x + ) = h 0 (2x + 2) + h (2x + ) + h 2 (2x) + h 3 (2x ) (x + 2) = h 0 (2x + 4) + h (2x + 3) + h 2 (2x + 2) + h 3 (2x + ): After rejecting the contributions from outside the support of it follows that (for 0 < x < =2) (x) (x + ) (x + 2) A = h h 2 h h 0 0 h 3 h 2 0 A@ (2x) (2x + ) (2x + 2) Clearly the matrix is T 0. The computations for x > =2 are similar. In the papers [2, 3], Berger rst noticed that this IFS approach to the dilation equation naturally leads to a Chaos Game. We now set up a vectorized IFSP. The graph of the scaling function can be approximated through the following modied version of the Chaos Game. A :. Initialize x = 0, and the vector V (x) to be the xed point of T 0 2. Pick! 2 f0; g with equal probability. 3. x 7! x=2 +!=2 4. V (x) 7! T! V (x) 5. Plot the points (x; (x)); (x + ; (x + )); : : : ; (x + N ; (x + N )) 6. If number of iterations is less than maximum, go to step 2. The basic reason that this simple algorithm works is that the vectorized IFS is non-overlapping, that is that m(w 0 ([0; ]) T w ([0; ])) = 0. See [0] for more discussion of Chaos Games for IFS on functions. 6 Modied Chaos Game Algorithm for Wavelet Generation In this section we modify the Chaos Game which generates the scaling function,, in order to generate the wavelet. This algorithm is a slight variation of Berger's algorithm given in [2, 3] in that we generate a piecewise constant approximation to by calculating the average value of on some partition. The algorithm is as follows. Let B i be a partition of [0; N] into Borel sets and for each B i we have S i, an accumulation variable. Let x = 0 and y 2 IR N be the (normalized) xed point of T 0. Initialize S j = 0:. Choose! 2 f0; g with equal probability. 6

7 2. x 7! x=2 +!=2 3. y 7! T! (y) 4. for each k = 0; ; : : : ; 2N, if x=2 + k=2 2, then update + = min(n;k) i=max(0;k N+) ( ) i h N i y k i 5. If number of iterations is less than maximum, go to step. The approximation to on is 2 m( ) numits (3) Basically what this algorithm does is generate the scaling function via the previous Chaos Game and take the appropriate dilation and linear combinations (from equation (9)) in order to form. The extra factor of 2 in the denominator is because is dened in terms of (2x i) rather than (x). Another way to write step 4 of the algorithm is the following. We have the values y j = (x + j) for j = 0; ; : : : ; N and we have the coecients h i for i = 0; ; : : : ; N. For z = (x + i + j)=2, we know that (z) depends on the term ( ) i h N i y j. So, for each i; j so that (x + i + j)=2 2, we update + = ( ) i h N i y j : In our algorithm in step 4, we simply group all those i; j so that i + j = k and loop through k. Proposition 4 For each m 2 numits m( )! m( ) (x) dx as numits! : Proof: The process is a non-overlapping vector valued IFSM on [0; ] (see [0]). We map the vector valued function g : [0; ]! IR N to the function ^g : [0; N]! IR by ^g(x) = N i= [i ;i] g(x) i : Through this mapping, the process is equivalent to a non-overlapping IFSM on [0; N]. Now at each stage y represents ((x); (x + ); : : : ; (x + N )): 7

8 To make the notation simpler, we just set (x + i) = 0 if i < 0 or i > N in the formulas below. So letting x n be the orbit generated by the Chaos Game, we have numits 2 =! =2 = = = 2 numits 2N n N 0 k=0 i=0 2N (k+)=2 k=0 N 0 i k=2 2N k=0 N i=0 ( ) i h N i (x n + k i) Bm ((x n + k)=2) ( ) i h N i (x + k i) Bm ((x + k)=2) dx N i=0 h N i (2y i) Bm (y) dy h N i (2y i) Bm (y) dy (y) dy: Thus, we have the desired result. R Notice that =m( ) (x) dx is the average value of over. Dene (x) = =m( ) Bm (x) dx: (4) m If we rene the partition B j, we will get a rened estimate of. The next proposition shows that in the limit, we recover exactly. Proposition 5 Let P n be a nested sequence of Borel partitions of [0; N] whose \sizes" go to zero as n!. Let n be the average value function of associated with P n. Then n converges to pointwise almost everywhere. Proof: Just notice that n is the conditional expectation of given P n. Since 2 L ( is a continuous compactly supported function), we have the result by the Martingale Convergence Theorem ([4]). Suppose that we want to generate the wavelet i;j instead of the mother wavelet. The only change to the above algorithm necessary is to replace Step 4 with 8

9 if w i;j (x=2 + k=2) 2, then update + = 2 i=2 min(n;k) l=max(0;k N+) and then the approximation to i;j on is since ( ) l h N l y k l 2 numits m(w i;j ()) = numits m( ) 2 i+ The scaling factor 2 i=2 is necessary since i;j (x) = 2 i=2 (2 i x j). With the modied Step 4, we have 2 numits m(w i;j ())! =m() =m( ) i;j(x) dx = 2 i 2 i=2 m( ) wi;j (Bm) 7 Chaos Game for Wavelet Analysis Given an f in L 2, we can represent f = c i;j i;j i;j(x) dx (5) (x) dx: since the wavelets i;j form an orthonormal basis. This is the wavelet analysis of f. Suppose we modify the algorithm of the previous section by replacing step 4 with the following: + = f(x=2 + k=2) min(n;k) i=max(0;k N+) ( ) i h N i y k i (6) Then, by similar arguments to those in the proof of Proposition 4, we would have that 2 numits m(b j )! m(b j ) B j f(x) (x) dx as numits! : (7) Now, in wavelet analysis, we wish to compute integrals of the form c i;j = f(x) i;j (x) dx: 9

10 So, we now discuss how to modify the algorithm of the previous section in such a way as to do this in parallel for many dierent pairs (i; j). We use as motivation equation (7). Suppose that we want to calculate c i;j for (i; j) 2, some collection of coecients. For each 2, let S be an accumulation variable, initialized to 0. Let w (x) = w i;j (x) = x=2 i + j=2 i ; the map that maps the mother wavelet to the wavelet i;j (in that i;j = 2 i=2 wi;j ). For = (i; j) 2, dene scale() = i. Our modied algorithm is as follows:. Initialize x = 0 and y 2 IR N to be the xed point of T Choose! 2 f0; g with equal probability. 3. x 7! x=2 +!=2 4. y 7! T! (y) 5. For each 2, we do for each k = 0; ; : : : 2N { Update S + = f(w (x=2 + k=2)) min(n;k) i=max(0;k N+) 6. If number of iterations is less than maximum, go to step 2. The approximation to c is ( ) i h N i y k i S 2 2 scale() numits : (8) Proposition 6 For each 2 S 2 2 scale() numits! f(x) (x) dx as numits! : Proof: The proof is similar to the proof of Proposition 4. In this case, we are evaluating the function g(x) = (x)f(x) along the trajectory. Thus, S 2 2 scale() numits! (x)f(x) dx which is what we wished to show. 0

11 What if we do not have the true function f but only have some approximation to f? The typical case would be where we have a piece-wise constant approximation to f (as in the case of a function that we only sample at certain values or a function which is the result of a Chaos Game approximation). Call this approximation ^f. Then what we can compute is ^c i;j = ^f(x) i;j (x) dx: As long as we know that ^f is close to f, then we know that ^c i;j is close to c i;j. A special case of potential interest is when the function f is also the attractor for a related IFS. Then we can run two Chaos Games simultaneously to obtain a wavelet decomposition of f. 8 Chaos Game for Wavelet Synthesis Let f = i;j c i;j i;j be the decomposition of a function in a wavelet basis. We wish to recover the function f from the coecients c i;j. For the moment, we assume that there are only nitely many non-zero coecients. Let = f(i; j)jc i;j 6= 0g. Since is nite, the support of f is compact, say the interval [A; B]. Let E = P i;j jc i;jj and p i;j = jc i;j j=e. Using the notation from previous sections, we have the following algorithm. Initialize x = 0 and y 2 IR N to be the xed point of T 0. Let B j be a partition of [A; B] into Borel sets with associated accumulation variables S j initialized to be 0. Let w i;j (x) = x=2 i + j=2 i, the map that maps the mother wavelet to the wavelet i;j.. Choose! 2 f0; g with equal probability. 2. x 7! x=2 +!=2 3. y 7! T! (y) 4. Choose (i; j) 2 with probability p i;j. 5. For k = 0; ; : : : ; 2N if w i;j (x=2 + k=2) 2, then update 0 + = SGN(c i;j ) 2 i min(n;k) l=max(0;k N+) ( ) l h N l y k l A

12 6. If number of iterations is less than maximum, go to step. The approximation to f on is E 2 m( ) numits (9) Proposition 7 For each m 2 numits m( )! E m( ) f(x) dx as numits! : Proof: For each 2, let Sm be an accumulation variable corresponding to. Suppose we modify the above algorithm so that when we are in state, we modify only Sm. Then, by Proposition 4, we know that Now, S m 2 numits m( )! SGN(c i;j)p =m( ) = 2 S m (x) dx: so 2 numits m( ) = 2 S m 2 numits m( )! SGN(c i;j )p =m( ) 2 (x) dx and 2 SGN(c i;j )p =m( ) (x) dx = =m( )=E f(x) dx: Thus, we have proved the proposition. As in the case of the Chaos Game for, we have that if we rene the partition, we will get a rened estimate of. For completeness, we record the next proposition. The proof is just like the proof of Proposition 5. Proposition 8 Let P n be a nested sequence of Borel partitions of [A; B] whose \sizes" go to zero as n!. Let f n be the average value function of f associated with P n. Then f n converges to f pointwise almost everywhere. 2

13 9 Extensions 9. Arbitrary function in L 2 (IR) Let us take f 2 L 2 (IR), then in the wavelet basis f = ij c ij ij : If we x an > 0, then there is a nite subset F of so that if we dene the \truncation" of f, f tr, as f tr = it follows that (i;j)2f c ij ij kf f tr k L 2 < 3 : Notice that f tr is a nite linear combination of compactly supported functions, and consequently must be compactly supported itself. Let B k be a Borel partition of supp(f tr ), and let f be the average value function associated with this partition (see equation 4). In addition let us suppose that that partition B k is suciently rened that kf tr fk L 2 < 3 Suppose we generate a Chaos Game approximation to the function f tr, call it f cg. According to Proposition 7 by choosing a maximal number of Chaos Game iterations suciently large then almost certainly. This in turn implies that k f f cg k L 2 < 3 kf f cg k L 2 which proves the following proposition. Proposition 9 Let f 2 L 2 (IR), and > 0, then there exists a Chaos Game dened by the algorithm in section 8 which produces a function f cg which almost surely satises kf f cg k L 2 < 3

14 9.2 General dilations and higher dimensions The results in this paper can be extended to more general dilation equations and more general wavelets. For dilation factor N (instead of 2), we simply get maps T 0 ; T ; : : : ; T N corresponding to the N map vector IFSM T (f) = N i=0 T i f(nx i) with f being the vectorized version of the scaling function as usual. Then the Chaos Game for the scaling function is a simple modication of our Chaos Game from section 5. Once we have a Chaos Game for the scaling function, the Chaos Game for the wavelet and for wavelet analysis and wavelet synthesis follow. In order to extend our results to multidimensional scaling functions (that is, : IR n! IR), wavelets and wavelet expansions, we need a formulation of the vector IFS in this, more general, context. We briey indicate the theory here. For a more complete discussion see the references [4, 5, 6, 2, 5] Given a dilation matrix A 2 M n () and a set of coecients c n, we have the dilation equation (x) = n2 c n (Ax n) (20) where n is a nite set. The scaling function will be the solution to this equation. For n, we let K be the attractor of the IFS W = fa x+a d : d 2 g. Then if is a compactly supported solution to equation (20) supp() K. In one dimension, the vectorized function V is based covering the support of the scaling function with copies of [0; ]. In IR n, we need to base the vectorized function on a self-ane tiling of IR n. Under certain conditions (see [5]), there is a digit set D n (necessarily a set of complete representatives of n =A n ) so that K D is a tile for IR n (i.e. that n + K D tiles IR n ). So, we convert the dilation equation (20) into the vectorized equation V (x) = T! V (x): (2) Here V : K D! IR S, where S n is a \covering set" (see [2], pg 85), and for each! 2 D we have T! is an jsj jsj matrix dened by (T! ) m;n = c!+am n for m; n 2 S and : K D! K D is the shift map (see [2], pg 83). This gives a nonoverlapping vector IFS, so we can use the Chaos Game to generate the attractor. The following algorithm will accomplish this (assuming that the dilation equation has a solution). Given c n : n 2 n, the dilation matrix A, D n a digit set for a tiling based on A, S n a covering set. Let B n be a partition of compact sets 4

15 so that K S B i. For each B i, let S i be an accumulation variable initialized to 0.. Choose d 2 D and initialize x as the xed point of the map x! A x + A d. Initialize y 2 IR S as the xed point of T d 2. Choose! 2 D randomly (with equal probability). 3. Update y = T! y and x = A x + A!. 4. For each m 2 S, if x + m 2 B i update S i + = y m : 5. If the number of iterations is less than the maximum, go to step 2. The approximation to on B i is S i m(b i ) numits (22) and the proof of the convergence to the average value of on B i is the same as in the one dimensional case. It is relatively straightforward to modify this basic algorithm to obtain a Chaos Game algorithm for generating the mother wavelet, for doing wavelet analysis or for doing wavelet synthesis. 0 Acknowledgments This work was supported in part by a Natural Sciences and Engineering Research Council of Canada (NSERC) Collaborative Grant (under C. Tricot and E.R. Vrscay) in the form of a Postdoctoral Fellowship (F.M.) and Graduate Research Assistantship (J.S.). The authors would like to thank E.R. Vrscay for helpful discussions and guidance on this project. The current address for the rst author is: School of Mathematics, Georgia Institute of Technology, Atlanta, GA References [] M. F. Barnsley, Fractals Everywhere, Academic Press, New York, 988. [2] M. Berger, Wavelets as Attractors of Random Dynamical Systems, in Stochastic Analysis: Liber Amicorum for Moshe akai, edited by E. Mayer-Wolf, Ely Merzbach and Adam Shwartz, Academic Press, Boston, 99, pp

16 [3] M. Berger, Random Ane Iterated Function Systems: Curve Generation and Wavelets, SIAM Review, 34 No. 3, pp [4] M. Berger and Yang Wang, Multidimensional two-scale dilation equations, in Wavelets - A Tutorial in Theory and Applications, C. K. Chui, ed, Academic Press, 992, pp [5] C. Cabrelli, C. Heil, and U. Molter, Accuracy of several multidimensional renable distributions. preprint, April 997. [6] C. Cabrelli, C. Heil, and U. Molter, Accuracy of lattice translates of several multidimensional renable functions, to appear in J. Approximation Theory. [7] I. Daubechies, Ten Lectures on Wavelets, SIAM Press, Philadelphia, PA, 992. [8] I. Daubechies and J. Lagarias, Two-scale dierence equations II. Local regularity, innite products of matrices and fractals, SIAM J. Math. Anal., 23 (992), pp [9] J. Elton, An ergodic theorem for iterated maps, J. Erg. Th. Dyn. Sys. 7, (987). [0] B. Forte, F. Mendivil and E.R. Vrscay, `Chaos Games' for Iterated Function Systems with Grey Level Maps, to appear in SIAM Journal of Mathematical Analysis. [] J. E. Hutchinson, Fractals and Self-similarity, Indiana Univ. Math. J. 30 (98), [2] Kevin Leeds, Dilation Equations with Matrix Dilations, PhD Thesis, Department of Math, Georgia Tech, 997. [3] C. A. Micchelli and H. Prautzsch, Uniform renement of curves, Linear Algebra Appl., 4/5 (989), pp [4] K. Stromberg, Probability for Analysts, Chapman and Hall, New York, 994. [5] Yang Wang, Self-Ane Tiles, preprint, to appear in Proc. Chinese Univ. of Hong Kong Workshop on Wavelet,

Mathematical Theory of Generalized Fractal. Transforms and Associated Inverse Problems. Edward R. Vrscay. Department of Applied Mathematics

Mathematical Theory of Generalized Fractal Transforms and Associated Inverse Problems Edward R. Vrscay Department of Applied Mathematics Faculty of Mathematics University of Waterloo Waterloo, Ontario,