Comuting with Haar Functions Sami Khuri Deartment of Mathematics and Comuter Science San Jose State University One Washington Square San Jose, CA 9519-0103, USA khuri@juiter.sjsu.edu Fax: (40)94-500 Keywords: decetive function, fast transforms, tness functions, Haar functions, Walsh functions Abstract Walsh functions are orthogonal, rectangular functions that take values 1 and form a convenient basis for the exansion of genetic algorithm tness functions. Since their introduction into genetic algorithms [, ], they have been used to comute the average tness values of schemata, to decide whether functions are hard or easy for genetic algorithms, and to design decetive functions for the genetic algorithm. In [10], Haar functions were introduced as an alternative to Walsh functions and it was shown that Haar functions are in general more comutationally advantageous. This aer revisits Haar functions, albeit with a slight variation to [10]'s denition, uses the functions to construct fully decetive functions for the genetic algorithm (as was done with Walsh functions [5]), and studies fast Haar transforms and fast Walsh-Haar transforms. 1 Introduction Orthonormal bases have catured the attention of researchers in a variety of elds. In wavelet theory, \orthonormal wavelet bases are used as a tool to describe mathematically the `increment in information' needed to go from a coarse aroximation to a higher resolution aroximation [3]. Digital electronics brought the necessity for more ractical bases than Fourier's sine/cosine functions. Orthogonal, rectangular functions are better suited for digital electronics and have also been introduced as theoretical tools for the analysis of genetic algorithms [, ]. One of the reasons behind the success of Walsh's rectangular functions as a theoretical tool in genetic algorithms is related to the exansion of schemata in terms of Walsh coecients. More recisely, the shorter the order of the schema (the smaller the number of xed ositions), the fewer Walsh coecients it needs in the exansion. In [5], Walsh functions were used to construct fully-decetive functions. After the introduction of Haar functions, in which we use a variation to the denition found in [10], we make use of these functions to construct, in a more ecient way than with Walsh functions, similar decetive functions. We then study two fast transforms, the fast Haar transform, and the fast Walsh-Haar transform. Because of the interdeendent nature of fast transforms, the comutation of a single term is built uon the values of revious levels: many more levels for Walsh than Haar. We conclude the aer by reiterating the comutational advantage that Haar functions have over their Walsh counterarts, and observe that it might be advantageous to use Haar functions rather than Walsh's for some alications. In some cases, such as with symmetrical functions and for very small roblem sizes, it might be referable to use Walsh functions. Haar Functions The Haar functions [6] form a comlete set of orthogonal rectangular basis functions. They take values of 1, 0 and?1, multilied by owers of and are usually dened on an interval normalized to [0; 1) [1]. In the following, the Haar functions are dened on [0; `), for any ositive integer `, and are normalized, H 0(x) 1 for 0 x < ` 1 for 0 x < `?1 H 1(x)?1 for `?1 x < ` H (x) H 3(x). H q m(x). H `?1(x) < : < : >< >: >< >: for 0 x < `?? for `? x < `?1 0 elsewhere in [0; `) for ()`? x < (3)`?? for (3)`? x < (4)`? 0 elsewhere in [0; `) ( ) q for (m)`?q?1 x and x < (m 1)`?q?1?( ) q for (m 1)`?q?1 x (1) and x < (m )`?q?1 0 elsewhere in [0; `) ( )`?1 for (`?1? 1) x?( )`?1 and x < `? 1 for `? 1 x < ` 0 elsewhere in [0; `)
For q 0; 1; : : : ; `? 1, we have m 0; 1; : : : ; q? 1. The set fh j(x) : j 0; 1; ; : : : ; `? 1g forms a basis for the tness functions dened on the integers in [0; `): That is: `?1 j0 hj Hj(x); () where the h j's, for j q m, are the Haar coecients given by: h j 1 ` `?1 H j(x): (3) The Haar function, H q m(x), has degree q and order m. Functions with the same degree are such that each one is just a \right-shift of the revious function. The osition of the function is given by the order m. The higher the degree q, the smaller the subinterval with non-zero values for H j(x). Haar functions of degree q contain q more nonzero arts than Haar functions of degree q?1. Consequently, each Haar coecient deends only on the local behavior of. More recisely, the following two results (equivalent to Beaucham's [1]) were obtained in [10]: Result 1 Every Haar coecient of degree q has at most `?q non-zero terms. Each term corresonds to a oint in an interval of the form [(i)`?q ; (i 1)`?q ): Consequently, the linear combination of each Haar coecient h j, where j q m; has at most `?q non-zero terms. In addition, h 0 has at most ` non-zero terms. Result For any xed value x [0; `), has at most ` 1 non-zero terms. In the next section, we use the Haar transform to build fully decetive functions. 3 Designing Fully Decetive 3-Bit Functions In his quest to identify functions that are suitable for genetic algorithms, Goldberg, using Walsh coecients, constructed fully decetive 3-bit functions [5]. We use Haar functions to construct such a function. If 111 is the otimal oint in the search sace of a maximization roblem, then all order-one schemata should lead away from it. In other words, we have: f(?? 1) < f(?? 0), f(?1?) < f(?0?), and f(1??) < f(0??). By using Equation and taking averages, the inequalities roduce: h 4 h 5 h 6 h 7 > 0, h h 3 > 0, and h 1 > 0, resectively. Similarly, for order-two schemata, we want them to lead away from the best. In other words, we should have: f(?00) > f(?01), f(?00) > f(?10), and f(?00) > f(?11); which roduce the following inequalities: h 4 h 6 > 0, (h h 3) >?h 4 h 5? h 6 h 7, and (h h 3) >?(h 4 h 5 h 6 h 7). f(0? 0) > f(0? 1), f(0? 0) > f(1? 0), and f(0? 0) > f(1? 1); h 4 h 5 > 0, h 1 (h 4 h 5? h 6? h 7) > 0, and h 1 (h 4 h 5 h 6 h 7) > 0, resectively. f(00?) > f(01?), f(00?) > f(10?), and f(00?) > f(11?); h > 0, h 1 (h? h 3) > 0, and h 1 (h h 3) > 0, resectively. Finally, since 111 is the otimal oint, we must have: f(111) > f(000), f(111) > f(001), f(111) > f(010), and f(111) > f(011), (h 1 h 4 h 7) (h h 3) < 0, (h 1? h 4 h 7) (h h 3) < 0, (h 1 h 5 h 7)? (h h 3) < 0, and (h 1? h 5 h 7)? (h? h 3) < 0, resectively. f(111) > f(100), f(111) > f(101), and f(111) > f(110), which roduces the following inequalities: h 6 h 7 h 3 < 0,?h 6 h 7 h 3 < 0, and h 7 < 0, resectively. Various solutions for the Haar coecients can be obtained by solving the simultaneous set of inequalities, thus roducing many fully decetive functions. A ossible solution to the inequalities is: h 0 h 1 1; h ; h 3? ; h 4 10; h 5?1; h 6 ; h 7?16: By relacing these values in Equation, we obtain the fully decetive 3-bit function given in Table 1. x 000 001 010 011 100 101 110 111 6?14?4 0 14?1?30 34 Table 1: A fully decetive 3-bit function. The average tness of?10, for instance, can be exressed as a linear combination of at most four non-zero Walsh coecients: f(?10) w 0 w 1? w? w 3: The comutation of w i, for i 1; ; 3; 4, has terms, requiring the values of at all oints in the interval [0; ), thus bringing the total number of comutations to 3. The average tness of the same schema exressed as a linear combination of Haar coecients yields: f(?10) h 0? (h h3) h5 h7: According to Equation 3 and Result 1 from the revious section, the comutation of h and of h 3 has 3?1 terms each (see Result 1, where q 1 since 1 0, and 3 1 1), the comutation of h 5 and h 7 have 3? terms each (since 5 1 and 7 3), and h 0 requires the comutation of terms. Thus, the total number of comutations is 0 with the Haar exansion, a substantial savings over Walsh's exansion. To take advantage of the many reetitive comutations erformed with orthogonal transformations, fast Walsh and fast Haar transforms have been roosed. In the next section, we exlore and comare fast Haar and Walsh transforms.
4 Fast Transforms Fast transforms are reresented by layered owcharts where an intermediate result at a certain stage is obtained by adding (or subtracting) two intermediate results from the revious layer. Thus, when dealing with fast transforms, it is more aroriate to count the number of oerations (additions or subtractions), which is equivalent to counting the number of terms [13]. These are in-lace algorithms, and so memory storage for intermediate-stage comutations is not needed. The calculated air of outut values can relace the corresonding air of data in the receding stage. We study the fast Haar and the fast Walsh-Haar transforms, which are modied Cooley and Tukey algorithms, modeled after the fast Fourier transform. f(000) f(001) f(010) h? f(011)? h 5 f(100)?1 h 1 f(101)? h 6 f(110) f(111) multiliers f(000)?? h 4 h 7? h 3 Figure 1: Fast Haar transform. B A D h 0 w 0 C 1 1 h 0 4.1 Fast Haar Transform Fast Haar transforms can be imlemented with on the order of at most ` comutations. More recisely, Roeser and Jernigan's fast Haar transform requires (N? 1) oerations [13], where N `. A ow diagram of their algorithm, with ` 3, is shown in Figure 1. Each term from the leftmost column is either added to (solid line) or subtracted from (dashed line) its adjacent term to roduce a new term for a subsequent stage of the algorithm. Some oerations have multilying factors greater than one and their values can be seen over the corresonding joining lines. The ow diagram going from left to right roduces the Haar coecients (after multilying by 1 `, according to Equation 3), while the oosite direction roduces the value of the function for any oint x [0; `? 1]. For examle, to comute h, we locate it in the gure, and read from right to left to obtain: h 1 [ (f(000) f(001))? (f(010) f(011))]: To comute f(010), we read from left to right (thick line): f(010) h 0 h 1? h h 5: Note that the numbers of terms in the exansions of h and f(010) are consistent with Results 1 and, resectively, from the revious section. We also note that the total number of oerations (additions and subtractions) in Figure 1, is (going from left to right), on the rst layer, 4 on the second, and on the last, i.e., 14, which is equal to (`? 1) for ` 3. Note that the coecients do not occur in sequential order. When ` 3, reading from to to bottom, the order is: h 0; h 4; h ; h 5, h 1; h 6; h 3; h 7. To locate a coecient, the following equation is used [13]: (`?q) (?1 m) for 0 < m < q? 1 and 0 < q < ` where q is the degree, and m the member to be determined. For instance, if ` 3, then the location of h 5, i.e., h 1, is (3?) (?1 1) 3. In the next section, we show an algorithm that gives both Walsh and Haar transforms. f(001) f(010) f(011) f(100) h 4 h h 3 h 1 w 4 w w 6 w 1 Strings Fitness 1111 0000 0 1110 0001 16 1101 0010 31 1011 0100 9 1010 0101 5 1001 0110 39 1000 0111 3 f(101) h 5 w 5 Table : Binary strings with their tness values. f(110) h 6 f(111) h7 Figure : Fast Walsh-Haar transform. w 3 w 7 4. The Fast Walsh-Haar Transform Either the Walsh or the Haar exansion can be obtained from Fino's comosite Walsh-Haar transform [4]. The transform is resented by the ow diagram of Figure for ` 3. The multilying constants emloyed in the routine for the Haar transform are shown in the diagram as the constants
j 0 1 3 4 5 6 7 9 10 11 1 13 14 15 w j 39 0 0?46 0??6 0 0??11 0?66 0 0 h j 39 0?33 33?10?16 16 10?3?10 3?3? 10 3 Table 3: Binary strings with their tness values. associated with the levels. At level B, the multilier is, at level C the multilier is and at level D, the multilier is 1. Each term from the leftmost column is either added to (solid line) or subtracted from (dashed line) its adjacent term to roduce a new term of a subsequent stage of the algorithm. The ow diagram going from left to right roduces the Haar coecients (after multilying by 1 and the ` aroriate level multilier) or the Walsh coecients (after multilying by 1 ). Going in the oosite direction roduces the value of the function for any oint x [0; `? 1]. ` For instance, going from right to left: And h 7 [f(110)? f(111)] 1 : h [f(000) f(001)? f(010)? f(011)] 1 : Going from left to right, we can comute f(110) in terms of Haar coecients: f(110) h 0? h 1? h3 h 7; or as a linear combination of Walsh coecients: f(110) w 0? w 4? w w 6 w 1? w 5? w 3 w 7: We note that the total number of oerations to get Walsh coecients, in Figure, (additions and subtractions), is 3 in each of the 3 layers. i.e., 4, which is equal to 3(`) for ` 3. The Haar coecients are obtained much faster than the Walsh. Indeed, Haar requires (`? 1) oerations comared to Walsh's `(`). Thus, ``? `1 more oerations are needed with Walsh than with Haar functions. For instance, for ` 10, we need to erform ; 19 more oerations when the Walsh fast transform is used instead of the Haar transform. It is generally more advantageous to use the Haar transform rather than its Walsh counterart. However, the next section examines a secial case for which the latter might be referable. 5 Symmetrical Fitness Functions If is a symmetrical function, then many Walsh coecients will vanish while the Haar coecients will have nonzero values. This is due to the symmetry roerties of Walsh transforms. More recisely, the roof of the following result [14] can be found in the Aendix. Result 3 Let be a symmetric function, i.e.,, where x i 1? x i. Then every Walsh coecients w j that contains an odd number of ones in its index j, is zero. The maximum cut roblem illustrates this situation. Given a weighted grah G (V; E) where V f1; : : : ; ng is the set of vertices, E the set of edges, and w ij the weight of the edge between vertices i and j, the roblem consists in artitioning V into two disjoint sets V 0 and V 1, such that the sum of the weights of the edges from E that have one endoint in V 0 and the other in V 1 is maximized. Examle 1 Consider the instance of the maximum cut roblem with four vertices, and ve edges with the following weights: w 1 w 1 6, w 13 w 31 10, w 3 w 3 11, w 4 w 4 15, w 34 w 43, and w ij 0 for all other values of i and j. If we let x i 0 when vertex i is in V 0 and x i 1 when i is in V 1, then each string in f0; 1g 4 will reresent a ossible artition of the vertices. Since w ij w ji for 1 i; j 4, the tness function to maximize is given by ([11]): n?1 n ji1 [x i(1? x j) x j(1? x i)] w ij with n 4. The function is symmetrical and every binary string has the same tness value as its comlement. For instance, f(1010) f(0101) 5. Table gives all ossible solutions with their corresonding tness values. The Walsh and Haar coecients tabulated in Table 3 have to be multilied by 1. 16 We note that although it is more ecient to comute the Haar coecients, it might still be referable to use the Walsh transform in this roblem, since half of the coecients dro out while all but one of the Haar coecients have non-zero value, and since, more imortantly, ` is very small. 6 Conclusion The aim of this article is to bring to the attention of researchers involved in theoretical asects of genetic algorithms the existence and some of the roerties of Haar functions. More recisely, these functions can be used as a tool, to comute tness averages, to comute schemata, and to construct decetive functions. Because of the comutational advantages, Haar functions might be used for certain alications instead of Walsh functions. Using Haar functions a basis, the exansion of the tness function requires substantially fewer terms than with Walsh functions. The Haar coecients too can be written as linear combinations of very few oints that are \close together and do not require the comutation of all oints, as is the case with Walsh coecients. The comutational dierence between Walsh and Haar functions can also be seen by considering fast transforms. With these imlementations, modeled after the fast Fourier transform, Haar coecients require O(N) transformations, whereas O(N log N) transformations are required for Walsh, where N `. We showed that the dierence between the total number of terms required for the comutation of the exansions of Walsh and of Haar remains exonential in ` (of order ``).
Acknowledgments The author would like to thank Dr. Frederick Stern for many discussions on the toic, and one of the reviewers' suggestions that have imroved the clarity and style of this work. He would also like to thank Stefan Bischof and Helko Lehmann for getting the aer in camera{ready form. References [1] Beaucham K. G., (194), Alications of Walsh and Related Functions, Academic Press. [] Bethke, A. D., (190), Genetic Algorithms as Function Otimizers, Doctoral dissertation, University of Michigan. [3] Daubechies, I., (199), Ten Lectures on Wavelets, Philadelhia: SIAM, 199. [4] Fino, B.J., (197), Relations Between Haar and Walsh- Hadamard Transforms, Proc. IEEE (Letters), vol 60,. 647-64. [5] Goldberg, D.E., (199), Genetic Algorithms and Walsh Functions Part II: Decetion and its Analysis, Comlex Systems 3,. 153-171. [6] Haar, A., (1910), Zur Theorie der Orthogonalen Funktionensysteme, Math. Annalen 69:331-371. [7] Harmuth, H. F., (196), A Generalized Concet of Frequency and Some Alications, IEEE Transactions on Information Theory, IT-14:375-3. [] Holland, J.H., (1975), Adatation in Natural and Arti- cial Systems, The University of Michigan Press, Ann Arbor. [9] Karovsky, M.G., (195), Sectral Techniques and Fault Detection, Academic Press. [10] Khuri S (1994) Walsh and Haar Functions in Genetic Algorithms. Proceedings of the 1994 ACM Symosium of Alied Comuting (New York: ACM Press) 01{5. [11] Khuri, S., Back, T., and Heitkotter, J., (1994), An Evolutionary Aroach to Combinatorial Otimization Problems, Proc. of the nd. ACM Comuter Science Conf., Phoenix, Arizona. [1] Kremer, H., (1973), On Theory of Fast Walsh Transform Algorithms, Colloquium on the Theory and Alications of Walsh and Other Non-Sinusoidal Functions, Hateld, U.K. [13] Roeser, P. R., and Jernigan, M. E., (19), Fast Haar Transform Algorithms, IEEE Transactions on Comuters C-31 no :175-177. [14] Rudolf, G., (1994), Private communication reroduced in the Aendix. Aendix Result 3 Let be a symmetric function, i.e.,, where x i 1? x i. Then every Walsh coecients w j that contains an odd number of ones in its index j, is zero. Note that 1? x i 1? (1? x i) (?1) (1? x i). The Walsh coecient w j is calculated by w j l?1 l?1 x l?1 ( Y l l Y Y l (?1) j i! 1! (?1) j i! (1? kjk 1 mod ) P Q l (1? xi)j i if kjk 1 even 0 if kjk 1 odd P l where kjk 1 ji denotes the l1{norm of j in binary notation.