DISCRETE HAAR WAVELET TRANSFORMS
|
|
|
- Benjamin Griffin
- 6 years ago
- Views:
Transcription
1 DISCRETE HAAR WAVELET TRANSFORMS Catherine Bénéteau University of South Florida Tampa, FL USA UNM - PNM Statewide Mathematics Contest, 2011 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 1 / 17
2 A MINI HISTORY OF WAVELETS A MINI HISTORY OF WAVELETS In the late 1970s, Morlet (trained at the Ecole Polytechnique), a geophysicist, became interested in signals that carried information related to geological layers. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 2 / 17
3 A MINI HISTORY OF WAVELETS A MINI HISTORY OF WAVELETS In the late 1970s, Morlet (trained at the Ecole Polytechnique), a geophysicist, became interested in signals that carried information related to geological layers. The problem: Fourier analysis techniques lacked the ability to zoom in on a signal and find short varying high frequencies. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 2 / 17
4 A MINI HISTORY OF WAVELETS A MINI HISTORY OF WAVELETS In the late 1970s, Morlet (trained at the Ecole Polytechnique), a geophysicist, became interested in signals that carried information related to geological layers. The problem: Fourier analysis techniques lacked the ability to zoom in on a signal and find short varying high frequencies. Morlet represented his signals via a special kind of function that is an ancestor of what we would call a wavelet today. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 2 / 17
5 A MINI HISTORY OF WAVELETS A MINI HISTORY OF WAVELETS In the late 1970s, Morlet (trained at the Ecole Polytechnique), a geophysicist, became interested in signals that carried information related to geological layers. The problem: Fourier analysis techniques lacked the ability to zoom in on a signal and find short varying high frequencies. Morlet represented his signals via a special kind of function that is an ancestor of what we would call a wavelet today. But Morlet was a geophysicist, and wanted to make sure his work made sense mathematically, so he contacted Grossman (a quantum physicist), who gave an elegant proof that Morlet s representation worked. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 2 / 17
6 A MINI HISTORY OF WAVELETS A MINI HISTORY OF WAVELETS In the late 1970s, Morlet (trained at the Ecole Polytechnique), a geophysicist, became interested in signals that carried information related to geological layers. The problem: Fourier analysis techniques lacked the ability to zoom in on a signal and find short varying high frequencies. Morlet represented his signals via a special kind of function that is an ancestor of what we would call a wavelet today. But Morlet was a geophysicist, and wanted to make sure his work made sense mathematically, so he contacted Grossman (a quantum physicist), who gave an elegant proof that Morlet s representation worked. In 1984, Yves Meyer(a mathematician) became acquainted with Morlet s work, and noticed right away that Morlet s functions were connected to some deep mathematics that had been worked on in the 1960s, and the subject took off from there. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 2 / 17
7 WHY WAVELETS? ENOUGH OF THIS HISTORY! What would a math lecture be without an exam? It s time to take a test! You are going to see three images. One is the original image consisting of bytes of information. A second image is a wavelet-compressed version of the original using bytes (about 8% of the original size), and another image is a wavelet-compressed version of the original using only 4452 bytes (about 3% of the original size)! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 3 / 17
8 WHY WAVELETS? ENOUGH OF THIS HISTORY! Question: Which is which? The following images come from a really neat site developed and maintained by Osmar R. Zaïane at Simon Fraser University in Canada. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 3 / 17
9 WHY WAVELETS? ENOUGH OF THIS HISTORY! Image A SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 3 / 17
10 WHY WAVELETS? ENOUGH OF THIS HISTORY! Image B SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 3 / 17
11 WHY WAVELETS? ENOUGH OF THIS HISTORY! Image C SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 3 / 17
12 WHY WAVELETS? ENOUGH OF THIS HISTORY! The Answer Key: Image Number of Bytes Transfer Time* Image A seconds Image B seconds Image C seconds *This rate assumes a standard 56K dial up modem that typically connects at about 30K. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 3 / 17
13 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? It is definitely new math - the subject took off in the late 80 s/early 90 s. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
14 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? It is definitely new math - the subject took off in the late 80 s/early 90 s. If you ask engineering or physics students to name some of the top accomplishments in their respective fields in the past 100 years, they can. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
15 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? It is definitely new math - the subject took off in the late 80 s/early 90 s. If you ask engineering or physics students to name some of the top accomplishments in their respective fields in the past 100 years, they can. Mathematics students, when asked the same question, typically struggle to name one or two. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
16 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? It is definitely new math - the subject took off in the late 80 s/early 90 s. If you ask engineering or physics students to name some of the top accomplishments in their respective fields in the past 100 years, they can. Mathematics students, when asked the same question, typically struggle to name one or two. The reason is most of the math we cover was developed hundreds of years ago and most of those famous mathematicians are... SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
17 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Isaac Newton - Dead... for a VERY long time. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
18 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Joseph Fourier - Dead. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
19 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Archimedes - Fossilized. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
20 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? C.F. Gauss - Dead... But Brain is Still Around. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
21 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Alfred Haar - Dead... Ok, so we re still talking about him. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
22 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? But wavelet theory is new math and the major players are... SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
23 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Ingrid Daubechies - Still Alive! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
24 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? David Donoho - Still Alive! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
25 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Yves Meyer - Blurry, But Still Alive! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
26 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Gil Strang, Still Alive! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
27 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Applications involving wavelets are numerous. Here are two: The FBI uses wavelets to digitally compress fingerprints. The compression factor is between 40 and 100 times the original size of the fingerprint! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
28 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? Applications involving wavelets are numerous. Here are two: The FBI uses wavelets to digitally compress fingerprints. The compression factor is between 40 and 100 times the original size of the fingerprint! Starting in 2000, the Joint Photographic Experts Group, designers of the popular.jpg image format used on the web, began implementing wavelet techniques in their format scheme. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
29 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? And two more: Kodak Polychrome Graphics, a Twin Cities based company, produces extremely high resolution digital images for advertisements that appear in publications like Time Magazine. They use wavelets to allow their clients to quickly analyze parts of the image. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
30 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? And two more: Kodak Polychrome Graphics, a Twin Cities based company, produces extremely high resolution digital images for advertisements that appear in publications like Time Magazine. They use wavelets to allow their clients to quickly analyze parts of the image. Wavelets have been heavily utilized in the modeling of distant galaxies. Wavelets have helped astronomers locate a subcluster of galaxies in the Coma supercluster of 1,400 galaxies! (Quantum Vol. 15 No. 3) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
31 WAVELETS ARE EXCITING BECAUSE... SO WHY WAVELETS? And one more: Musicologists restore an 1889 recording of Brahms playing his Hungarian Dance Number 1 from a wax cylinder recording in such bad shape that many listeners failed to recognize that a piano was even being played. (Quantum Vol. 15 No. 3) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 4 / 17
32 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
33 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS A bit is a fundamental unit on a computer - either 0 or 1. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
34 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS A bit is a fundamental unit on a computer - either 0 or 1. A byte is 8 bits. There are 2 8 = 256 possible bytes. There are also 256 characters on a standard keyboard! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
35 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS A bit is a fundamental unit on a computer - either 0 or 1. A byte is 8 bits. There are 2 8 = 256 possible bytes. There are also 256 characters on a standard keyboard! The American Standard Code for Information Interchange (ASCII) assigns to each byte a character. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
36 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS A bit is a fundamental unit on a computer - either 0 or 1. A byte is 8 bits. There are 2 8 = 256 possible bytes. There are also 256 characters on a standard keyboard! The American Standard Code for Information Interchange (ASCII) assigns to each byte a character. Some of these characters are visible on a standard computer keyboard (eg., y is 121 and 0 is 48). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
37 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
38 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
39 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS An 8-bit digital image can be viewed as a matrix whose entries (known as pixels) range from 0 (black) to 255 (white) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
40 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS Thus if we store an intensity value, say 121, to disk, we don t store 121, we store its ASCII character y. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
41 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS Thus if we store an intensity value, say 121, to disk, we don t store 121, we store its ASCII character y. y also has a binary representation: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
42 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS Thus if we store an intensity value, say 121, to disk, we don t store 121, we store its ASCII character y. y also has a binary representation: Thus if an image has dimensions N M, we need 8MN bits to store it on disk (modulo some header information). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
43 DIGITAL IMAGES GRAYSCALE IMAGE BASICS DIGITAL IMAGES GRAYSCALE IMAGE BASICS Thus if we store an intensity value, say 121, to disk, we don t store 121, we store its ASCII character y. y also has a binary representation: Thus if an image has dimensions N M, we need 8MN bits to store it on disk (modulo some header information). We will refer to the stored image as the bitstream and note that the bits per pixel (bpp) is 8. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 5 / 17
44 HUFFMAN CODING HUFFMAN CODING In 1952, David Huffman made a simple observation: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
45 HUFFMAN CODING HUFFMAN CODING In 1952, David Huffman made a simple observation: Rather than use the same number of bits to represent each character, why not use a short bit stream for characters that appear often in an image and a longer bit stream for characters that appear infrequently in the image? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
46 HUFFMAN CODING HUFFMAN CODING In 1952, David Huffman made a simple observation: Rather than use the same number of bits to represent each character, why not use a short bit stream for characters that appear often in an image and a longer bit stream for characters that appear infrequently in the image? He then developed an algorithm to do just that. We refer to his simple algorithm as Huffman coding. We will illustrate the algorithm via an example. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
47 HUFFMAN CODING HUFFMAN CODING Suppose you want to perform Huffman coding on the word seesaws. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
48 HUFFMAN CODING HUFFMAN CODING Suppose you want to perform Huffman coding on the word seesaws. First observe that s appears three times (24 bits), e appears twice (16 bits), and a and w each appear once (16 bits) so the total number of bits needed to represent seesaws is 56. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
49 HUFFMAN CODING HUFFMAN CODING Char. ASCII Binary Frequency s e a w So in terms of bits, the word seesaws is SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
50 HUFFMAN CODING HUFFMAN CODING The first step in Huffman coding is as follows: Assign probabilities to each character and then sort from smallest to largest. We will put the probabilities in circles called nodes and connect them with lines (branches). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
51 HUFFMAN CODING HUFFMAN CODING Now simply add the two smallest probabilities to create a new node with probability 2/7. Branch the two small nodes off this one and resort the three remaining nodes: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
52 HUFFMAN CODING HUFFMAN CODING Again we add the smallest two probabilities on the top row (2/7 + 2/7 = 4/7), create a new node with everything below these nodes as branches and sort again: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
53 HUFFMAN CODING HUFFMAN CODING Since only two nodes remain on top, we simply add the probabilities of these nodes together to get 1 and obtain our finished tree: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
54 HUFFMAN CODING HUFFMAN CODING SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
55 HUFFMAN CODING HUFFMAN CODING Now assign to each left branch the value 0 and to each right branch the value 1: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
56 HUFFMAN CODING HUFFMAN CODING SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
57 HUFFMAN CODING HUFFMAN CODING We can read the new bit stream for each character right off the tree! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
58 HUFFMAN CODING HUFFMAN CODING We can read the new bit stream for each character right off the tree! Here are the new bit streams for the four characters: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
59 HUFFMAN CODING HUFFMAN CODING Char. Binary s 0 2 e 11 2 a w SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
60 HUFFMAN CODING HUFFMAN CODING Since s appears three times in seesaws, we need 3 bits to represent them. The character e appears twice (4 bits), and a and w each appear once (3 bits each). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
61 HUFFMAN CODING HUFFMAN CODING Since s appears three times in seesaws, we need 3 bits to represent them. The character e appears twice (4 bits), and a and w each appear once (3 bits each). The total number of bits we need to represent the word seesaws is 13 bits! Recall without Huffman coding, we needed 56 bits so we have reduced the number of bits needed by a factor of 4! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
62 HUFFMAN CODING HUFFMAN CODING Since s appears three times in seesaws, we need 3 bits to represent them. The character e appears twice (4 bits), and a and w each appear once (3 bits each). The total number of bits we need to represent the word seesaws is 13 bits! Recall without Huffman coding, we needed 56 bits so we have reduced the number of bits needed by a factor of 4! Here is the word seesaws using the Huffman codes for each character: SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 6 / 17
63 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE The example is a bit of a sales job - of course we will enjoy great savings with only 4 distinct characters. What happens when we apply Huffman coding to a digital image? Consider the image SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
64 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE Unencoded, we need bits or 8 bits per pixel (bpp) to represent the image. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
65 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE Unencoded, we need bits or 8 bits per pixel (bpp) to represent the image. Using Huffman encoding, we only need bits to store the image. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
66 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE Unencoded, we need bits or 8 bits per pixel (bpp) to represent the image. Using Huffman encoding, we only need bits to store the image. This constitutes a 16.5% savings or an average of 6.67bpp. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
67 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE Unencoded, we need bits or 8 bits per pixel (bpp) to represent the image. Using Huffman encoding, we only need bits to store the image. This constitutes a 16.5% savings or an average of 6.67bpp. We should be able to do better... SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
68 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE Unencoded, we need bits or 8 bits per pixel (bpp) to represent the image. Using Huffman encoding, we only need bits to store the image. This constitutes a 16.5% savings or an average of 6.67bpp. We should be able to do better... What really helps an encoding method is a preprocessor that transforms the image to a setting that is a bit more amenable to the encoding scheme. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
69 HUFFMAN CODING HUFFMAN NEEDS HELP ENCODING AN IMAGE Unencoded, we need bits or 8 bits per pixel (bpp) to represent the image. Using Huffman encoding, we only need bits to store the image. This constitutes a 16.5% savings or an average of 6.67bpp. We should be able to do better... What really helps an encoding method is a preprocessor that transforms the image to a setting that is a bit more amenable to the encoding scheme. That s where the discrete wavelet transform comes in! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 7 / 17
70 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Get with a partner, and each partner should choose a word or short phrase and determine the Huffman codes for each letter. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 8 / 17
71 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Get with a partner, and each partner should choose a word or short phrase and determine the Huffman codes for each letter. Write the word as a binary string using the Huffman codes. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 8 / 17
72 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Get with a partner, and each partner should choose a word or short phrase and determine the Huffman codes for each letter. Write the word as a binary string using the Huffman codes. Give the string and the codes (dictionary) to your partner. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 8 / 17
73 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Get with a partner, and each partner should choose a word or short phrase and determine the Huffman codes for each letter. Write the word as a binary string using the Huffman codes. Give the string and the codes (dictionary) to your partner. The partner should determine the word or phrase. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 8 / 17
74 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Get with a partner, and each partner should choose a word or short phrase and determine the Huffman codes for each letter. Write the word as a binary string using the Huffman codes. Give the string and the codes (dictionary) to your partner. The partner should determine the word or phrase. Given the Huffman codes g = 10, n = 01, o = 00, space key=110, e = 1110, and i = 1111, decode the bit stream SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 8 / 17
75 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Are Huffman codes unique? In other words, given a word or a phrase, is it possible to have more than one Huffman code for that word or phrase? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 9 / 17
76 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Are Huffman codes unique? In other words, given a word or a phrase, is it possible to have more than one Huffman code for that word or phrase? Are Huffman codes invertible? In other words, given a bit stream and a Huffman code tree or dictionary, can I always translate the bit stream? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 9 / 17
77 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Are Huffman codes unique? In other words, given a word or a phrase, is it possible to have more than one Huffman code for that word or phrase? Are Huffman codes invertible? In other words, given a bit stream and a Huffman code tree or dictionary, can I always translate the bit stream? Is it possible to design a word so that one letter has as its code 0 and the other letter has as its code 1? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 9 / 17
78 HUFFMAN CODING CHALLENGE PROBLEMS CHALLENGE PROBLEMS Are Huffman codes unique? In other words, given a word or a phrase, is it possible to have more than one Huffman code for that word or phrase? Are Huffman codes invertible? In other words, given a bit stream and a Huffman code tree or dictionary, can I always translate the bit stream? Is it possible to design a word so that one letter has as its code 0 and the other letter has as its code 1? For a word of length greater than 2, is it possible for all the letters to have the same code length? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 9 / 17
79 NAIVE DATA APPROXIMATION THE PROBLEM NAIVE DATA APPROXIMATION Suppose you are given N values where N is even. x = (x 1, x 2,..., x N ) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 10 / 17
80 NAIVE DATA APPROXIMATION THE PROBLEM NAIVE DATA APPROXIMATION Suppose you are given N values where N is even. x = (x 1, x 2,..., x N ) Your task: Send an approximation s (a list of numbers) of this data via the internet to a colleague. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 10 / 17
81 NAIVE DATA APPROXIMATION THE PROBLEM NAIVE DATA APPROXIMATION Suppose you are given N values where N is even. x = (x 1, x 2,..., x N ) Your task: Send an approximation s (a list of numbers) of this data via the internet to a colleague. In order to reduce transfer time, the length of your approximation must be N/2. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 10 / 17
82 NAIVE DATA APPROXIMATION THE PROBLEM NAIVE DATA APPROXIMATION Suppose you are given N values where N is even. x = (x 1, x 2,..., x N ) Your task: Send an approximation s (a list of numbers) of this data via the internet to a colleague. In order to reduce transfer time, the length of your approximation must be N/2. How do you suggest we do it? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 10 / 17
83 NAIVE DATA APPROXIMATION THE PROBLEM NAIVE DATA APPROXIMATION One solution is to pair-wise average the numbers: s k = x 2k 1 + x 2k, k = 1,..., N/2 2 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 10 / 17
84 NAIVE DATA APPROXIMATION THE PROBLEM NAIVE DATA APPROXIMATION One solution is to pair-wise average the numbers: For example: s k = x 2k 1 + x 2k, k = 1,..., N/2 2 x = (6, 12, 15, 15, 14, 12, 120, 116) s = (9, 15, 13, 118) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 10 / 17
85 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
86 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION Suppose now you were allowed to send extra data in addition to the pair-wise averages list s. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
87 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION Suppose now you were allowed to send extra data in addition to the pair-wise averages list s. The idea is to send a second list of data d so that the original list x can be recovered from s and d. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
88 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION Suppose now you were allowed to send extra data in addition to the pair-wise averages list s. The idea is to send a second list of data d so that the original list x can be recovered from s and d. How would you do it? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
89 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION There are a couple of choices for d k (called directed distances): SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
90 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION There are a couple of choices for d k (called directed distances): We could set d k = x 2k 1 x 2k, k = 1,..., N/2 2 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
91 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION There are a couple of choices for d k (called directed distances): We could set or d k = x 2k 1 x 2k, k = 1,..., N/2 2 d k = x 2k x 2k 1, k = 1,..., N/2 2 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
92 NAIVE DATA APPROXIMATION ADDING MORE INFORMATION There are a couple of choices for d k (called directed distances): We could set d k = x 2k 1 x 2k, k = 1,..., N/2 2 or d k = x 2k x 2k 1, k = 1,..., N/2 2 We will use the second formula. d k = x 2k x 2k 1, k = 1,..., N/2 2 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 11 / 17
93 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS The process is invertible since and s k + d k = x 2k 1 + x 2k 2 s k d k = x 2k 1 + x 2k 2 + x 2k x 2k 1 2 x 2k x 2k 1 2 = x 2k = x 2k 1 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
94 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS The process is invertible since s k + d k = x 2k 1 + x 2k 2 + x 2k x 2k 1 2 = x 2k and s k d k = x 2k 1 + x 2k x 2k x 2k 1 = x 2k So we map x = (x 1, x 2,..., x N ) to (s d) = (s 1,..., s N/2 d 1,..., d N/2 ). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
95 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS The process is invertible since s k + d k = x 2k 1 + x 2k 2 + x 2k x 2k 1 2 = x 2k and s k d k = x 2k 1 + x 2k x 2k x 2k 1 = x 2k So we map x = (x 1, x 2,..., x N ) to (s d) = (s 1,..., s N/2 d 1,..., d N/2 ). Using our example values we have (6, 12, 15, 15, 14, 12, 120, 116) (9, 15, 13, 118 3, 0, 1, 2) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
96 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS The process is invertible since s k + d k = x 2k 1 + x 2k 2 + x 2k x 2k 1 2 = x 2k and s k d k = x 2k 1 + x 2k x 2k x 2k 1 = x 2k So we map x = (x 1, x 2,..., x N ) to (s d) = (s 1,..., s N/2 d 1,..., d N/2 ). Using our example values we have (6, 12, 15, 15, 14, 12, 120, 116) (9, 15, 13, 118 3, 0, 1, 2) Why might people prefer the data in this form? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
97 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS We can identify large changes in the the differences portion d of the transform. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
98 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS We can identify large changes in the the differences portion d of the transform. It is easier to quantize the data in this form. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
99 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS We can identify large changes in the the differences portion d of the transform. It is easier to quantize the data in this form. The transform concentrates the information (energy) in the signal in fewer values. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
100 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS We can identify large changes in the the differences portion d of the transform. It is easier to quantize the data in this form. The transform concentrates the information (energy) in the signal in fewer values. And the obvious answer: fewer digits!! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
101 NAIVE DATA APPROXIMATION ANALYZING THE PROCESS We can identify large changes in the the differences portion d of the transform. It is easier to quantize the data in this form. The transform concentrates the information (energy) in the signal in fewer values. And the obvious answer: fewer digits!! We will talk about quantizing and image compression a little later. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 12 / 17
102 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION The transformation x = (x 1,..., x N ) (s d) = (s 1,..., s N/2 d 1,..., d N/2 ) is (almost!) called the (1-dimensional) Discrete Haar Wavelet Transformation. (Actually, usually we would multiply by 2 - why do you think we might do that?) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 13 / 17
103 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION The transformation x = (x 1,..., x N ) (s d) = (s 1,..., s N/2 d 1,..., d N/2 ) is (almost!) called the (1-dimensional) Discrete Haar Wavelet Transformation. (Actually, usually we would multiply by 2 - why do you think we might do that?) What does the transform look like as a matrix? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 13 / 17
104 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION Consider applying the transform to an 8-vector. What is the matrix that works? We will denote the transform matrix by W 8. x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 = 1 2 x 1 + x 2 x 3 + x 4 x 5 + x 6 x 7 + x 8 x 2 x 1 x 4 x 3 x 6 x 5 x 8 x 7 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 13 / 17
105 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION Consider applying the transform to an 8-vector. What is the matrix that works? We will denote the transform matrix by W 8. x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 = 1 2 x 1 + x 2 x 3 + x 4 x 5 + x 6 x 7 + x 8 x 2 x 1 x 4 x 3 x 6 x 5 x 8 x 7 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 13 / 17
106 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION What about W 1 8? That is, what matrix solves x 1 + x 2 x 3 + x 4 x 5 + x 6 x 7 + x 8 x 2 x 1 x 4 x 3 x 6 x 5 x 8 x 7 = x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 13 / 17
107 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION What about W 1 8? That is, what matrix solves x 1 + x 2 x 3 + x 4 x 5 + x 6 x 7 + x 8 x 2 x 1 x 4 x 3 x 6 x 5 x 8 x 7 = x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 13 / 17
108 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION The matrices W 1 8 and W T 8 are very closely connected! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 14 / 17
109 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION The matrices W 1 8 and W8 T In fact, 2W8 T = W 1 8. are very closely connected! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 14 / 17
110 THE DISCRETE HAAR WAVELET TRANSFORM - 1D MATRIX FORMULATION The matrices W 1 8 and W8 T are very closely connected! In fact, 2W8 T = W 1 8. This makes W 8 very close to being what we call an orthogonal matrix. (That s why we usually throw in a 2!) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 14 / 17
111 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION Consider the image (call it A) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
112 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION If a 1,..., a 640 are the columns of A, then computing W 480 A is the same as applying the HWT to each column of A: W 480 A = (W 480 a 1,..., W 480 a 640 ) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
113 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION Graphically, we have SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
114 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION C = W 480 A processes the columns of A. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
115 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION C = W 480 A processes the columns of A. How would we process the rows of C? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
116 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION C = W 480 A processes the columns of A. How would we process the rows of C? We compute CW T 640 = W 480AW T 640. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
117 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION C = W 480 A processes the columns of A. How would we process the rows of C? We compute CW640 T = W 480AW640 T. The two-dimensional Haar transform of the M N matrix A is B = W M AW T N SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
118 THE DISCRETE HAAR WAVELET TRANSFORM - 2D CONSTRUCTION Graphically, we have SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 15 / 17
119 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS Can we interpret what the transformation does to the image? SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
120 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS Can we interpret what the transformation does to the image? Suppose A is the 4 4 matrix A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
121 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS Can we interpret what the transformation does to the image? Suppose A is the 4 4 matrix A = Partitioning W 4 = H G a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44, we have SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
122 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS W 4 AW T 4 = = = H G A [H ] T G T GAH T Let s look at each 2 2 block individually: HA [ ] H T G T GA [ HAH T HAG T ] GAG T SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
123 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS [ a11 + a 12 + a 21 + a 22 a 13 + a 14 + a 23 + a 24 ] HAH T = 1 4 a 31 + a 32 + a 41 + a 42 a 33 + a 34 + a 43 + a 44 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
124 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS [ a11 + a 12 + a 21 + a 22 a 13 + a 14 + a 23 + a 24 ] HAH T = 1 4 a 31 + a 32 + a 41 + a 42 a 33 + a 34 + a 43 + a 44 Partition A in 2 2 blocks as a 11 a 12 a 13 a 14 [ ] A = a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 = A11 A 12 A 21 A 22 a 41 a 42 a 43 a 44 SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
125 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS [ a11 + a 12 + a 21 + a 22 a 13 + a 14 + a 23 + a 24 ] HAH T = 1 4 a 31 + a 32 + a 41 + a 42 a 33 + a 34 + a 43 + a 44 Partition A in 2 2 blocks as a 11 a 12 a 13 a 14 [ ] A = a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 = A11 A 12 A 21 A 22 a 41 a 42 a 43 a 44 Then the (i, j) element of HAH T is simply the average of the elements in A ij! SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
126 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS [ a11 + a 12 + a 21 + a 22 a 13 + a 14 + a 23 + a 24 ] HAH T = 1 4 a 31 + a 32 + a 41 + a 42 a 33 + a 34 + a 43 + a 44 Partition A in 2 2 blocks as a 11 a 12 a 13 a 14 [ ] A = a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 = A11 A 12 A 21 A 22 a 41 a 42 a 43 a 44 Then the (i, j) element of HAH T is simply the average of the elements in A ij! So HAH T is an approximation or blur of the original image. We will denote HAH T as B. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
127 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The upper right hand corner is HAG T = 1 4 [ (a12 + a 22 ) (a 11 + a 21 ) (a 14 + a 24 ) (a 13 + a 23 ) (a 32 + a 42 ) (a 31 + a 41 ) (a 34 + a 44 ) (a 33 + a 43 ) ] SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
128 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The upper right hand corner is HAG T = 1 4 [ (a12 + a 22 ) (a 11 + a 21 ) (a 14 + a 24 ) (a 13 + a 23 ) (a 32 + a 42 ) (a 31 + a 41 ) (a 34 + a 44 ) (a 33 + a 43 ) ] Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
129 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The upper right hand corner is HAG T = 1 4 [ (a12 + a 22 ) (a 11 + a 21 ) (a 14 + a 24 ) (a 13 + a 23 ) (a 32 + a 42 ) (a 31 + a 41 ) (a 34 + a 44 ) (a 33 + a 43 ) ] Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] The (i, j) element of HAG T can be viewed as a difference between columns of A ij. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
130 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The upper right hand corner is HAG T = 1 4 [ (a12 + a 22 ) (a 11 + a 21 ) (a 14 + a 24 ) (a 13 + a 23 ) (a 32 + a 42 ) (a 31 + a 41 ) (a 34 + a 44 ) (a 33 + a 43 ) ] Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] The (i, j) element of HAG T can be viewed as a difference between columns of A ij. We will denote HAG T as V (for vertical differences). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
131 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower left hand corner is GAH T = 1 [ ] (a21 + a 22 ) (a 12 + a 11 ) (a 23 + a 24 ) (a 13 + a 14 ) 4 (a 31 + a 32 ) (a 42 + a 41 ) (a 43 + a 44 ) (a 33 + a 34 ) SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
132 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower left hand corner is GAH T = 1 [ ] (a21 + a 22 ) (a 12 + a 11 ) (a 23 + a 24 ) (a 13 + a 14 ) 4 (a 31 + a 32 ) (a 42 + a 41 ) (a 43 + a 44 ) (a 33 + a 34 ) Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
133 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower left hand corner is GAH T = 1 [ ] (a21 + a 22 ) (a 12 + a 11 ) (a 23 + a 24 ) (a 13 + a 14 ) 4 (a 31 + a 32 ) (a 42 + a 41 ) (a 43 + a 44 ) (a 33 + a 34 ) Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] The (i, j) element of HAG T can be viewed as a difference between rows of A ij. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
134 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower left hand corner is GAH T = 1 [ ] (a21 + a 22 ) (a 12 + a 11 ) (a 23 + a 24 ) (a 13 + a 14 ) 4 (a 31 + a 32 ) (a 42 + a 41 ) (a 43 + a 44 ) (a 33 + a 34 ) Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] The (i, j) element of HAG T can be viewed as a difference between rows of A ij. We will denote GAH T as H (for horizontal differences). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
135 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower right hand corner is GAG T = 1 4 [ (a11 + a 22 ) (a 12 + a 21 ) (a 13 + a 24 ) (a 23 + a 14 ) (a 31 + a 42 ) (a 32 + a 41 ) (a 33 + a 44 ) (a 43 + a 34 ) ] SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
136 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower right hand corner is GAG T = 1 4 [ (a11 + a 22 ) (a 12 + a 21 ) (a 13 + a 24 ) (a 23 + a 14 ) (a 31 + a 42 ) (a 32 + a 41 ) (a 33 + a 44 ) (a 43 + a 34 ) ] Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
137 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower right hand corner is GAG T = 1 4 [ (a11 + a 22 ) (a 12 + a 21 ) (a 13 + a 24 ) (a 23 + a 14 ) (a 31 + a 42 ) (a 32 + a 41 ) (a 33 + a 44 ) (a 43 + a 34 ) ] Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] The (i, j) element of GAG T can be viewed as a difference between the diagonals of A ij. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
138 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS The lower right hand corner is GAG T = 1 4 [ (a11 + a 22 ) (a 12 + a 21 ) (a 13 + a 24 ) (a 23 + a 14 ) (a 31 + a 42 ) (a 32 + a 41 ) (a 33 + a 44 ) (a 43 + a 34 ) ] Partition A in 2 2 blocks as A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 = [ A11 A 12 A 21 A 22 ] The (i, j) element of GAG T can be viewed as a difference between the diagonals of A ij. We will denote GAG T as D (for diagonal differences). SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
139 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS So again the transform of our image is SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
140 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS We can apply the HWT to the blur. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
141 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS We can apply the HWT to the blur. This is called the iterated HWT. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 16 / 17
142 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS THIS IS JUST A TASTE! All the examples discussed today are examples of orthogonal wavelets - in many applications (such as JPEG2000), wavelet transforms used are biorthogonal. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 17 / 17
143 THE DISCRETE HAAR WAVELET TRANSFORM - 2D ANALYSIS THIS IS JUST A TASTE! All the examples discussed today are examples of orthogonal wavelets - in many applications (such as JPEG2000), wavelet transforms used are biorthogonal. There are variations of the wavelet algorithm - FBI fingerprint compression uses what is called wavelet packets. SATURDAY, FEBRUARY 5, 2011 (UNM) DISCRETE HAAR WAVELET TRANSFORMS UNM - PNM MATH CONTEST 17 / 17
The Haar Wavelet Transform: Compression and. Reconstruction
The Haar Wavelet Transform: Compression and Damien Adams and Halsey Patterson December 14, 2006 Abstract The Haar Wavelet Transformation is a simple form of compression involved in averaging and differencing
17.1 Binary Codes Normal numbers we use are in base 10, which are called decimal numbers. Each digit can be 10 possible numbers: 0, 1, 2, 9.
( c ) E p s t e i n, C a r t e r, B o l l i n g e r, A u r i s p a C h a p t e r 17: I n f o r m a t i o n S c i e n c e P a g e 1 CHAPTER 17: Information Science 17.1 Binary Codes Normal numbers we use
Information and Entropy. Professor Kevin Gold
Information and Entropy Professor Kevin Gold What s Information? Informally, when I communicate a message to you, that s information. Your grade is 100/100 Information can be encoded as a signal. Words
( c ) E p s t e i n, C a r t e r a n d B o l l i n g e r C h a p t e r 1 7 : I n f o r m a t i o n S c i e n c e P a g e 1
( c ) E p s t e i n, C a r t e r a n d B o l l i n g e r 2 0 1 6 C h a p t e r 1 7 : I n f o r m a t i o n S c i e n c e P a g e 1 CHAPTER 17: Information Science In this chapter, we learn how data can
Image Compression. 1. Introduction. Greg Ames Dec 07, 2002
Image Compression Greg Ames Dec 07, 2002 Abstract Digital images require large amounts of memory to store and, when retrieved from the internet, can take a considerable amount of time to download. The
Image Compression Using the Haar Wavelet Transform
College of the Redwoods http://online.redwoods.cc.ca.us/instruct/darnold/laproj/fall2002/ames/ 1/33 Image Compression Using the Haar Wavelet Transform Greg Ames College of the Redwoods Math 45 Linear Algebra
Learning goals: students learn to use the SVD to find good approximations to matrices and to compute the pseudoinverse.
Application of the SVD: Compression and Pseudoinverse Learning goals: students learn to use the SVD to find good approximations to matrices and to compute the pseudoinverse. Low rank approximation One
Image and Multidimensional Signal Processing
Image and Multidimensional Signal Processing Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ Image Compression 2 Image Compression Goal: Reduce amount
Signal Processing With Wavelets
Signal Processing With Wavelets JAMES MONK Niels Bohr Institute, University of Copenhagen. Self-Similarity Benoît B.* Mandlebrot: Clouds are not spheres, mountains are not cones, coastlines are not circles,
Proyecto final de carrera
UPC-ETSETB Proyecto final de carrera A comparison of scalar and vector quantization of wavelet decomposed images Author : Albane Delos Adviser: Luis Torres 2 P a g e Table of contents Table of figures...
( nonlinear constraints)
Wavelet Design & Applications Basic requirements: Admissibility (single constraint) Orthogonality ( nonlinear constraints) Sparse Representation Smooth functions well approx. by Fourier High-frequency
! Where are we on course map? ! What we did in lab last week. " How it relates to this week. ! Compression. " What is it, examples, classifications
Lecture #3 Compression! Where are we on course map?! What we did in lab last week " How it relates to this week! Compression " What is it, examples, classifications " Probability based compression # Huffman
Image Data Compression
Image Data Compression Image data compression is important for - image archiving e.g. satellite data - image transmission e.g. web data - multimedia applications e.g. desk-top editing Image data compression
Wavelets in Image Compression
Wavelets in Image Compression M. Victor WICKERHAUSER Washington University in St. Louis, Missouri victor@math.wustl.edu http://www.math.wustl.edu/~victor THEORY AND APPLICATIONS OF WAVELETS A Workshop
Review for Exam 1. Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA
Review for Exam Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA 0003 March 26, 204 Abstract Here are some things you need to know for the in-class
Complex Matrix Transformations
Gama Network Presents: Complex Matrix Transformations By By Scott Johnson Gamasutra May 17, 2002 URL: http://www.gamasutra.com/features/20020510/johnson_01.htm Matrix transforms are a ubiquitous aspect
1 Introduction to Wavelet Analysis
Jim Lambers ENERGY 281 Spring Quarter 2007-08 Lecture 9 Notes 1 Introduction to Wavelet Analysis Wavelets were developed in the 80 s and 90 s as an alternative to Fourier analysis of signals. Some of the
- An Image Coding Algorithm
- An Image Coding Algorithm Shufang Wu http://www.sfu.ca/~vswu vswu@cs.sfu.ca Friday, June 14, 2002 22-1 Agenda Overview Discrete Wavelet Transform Zerotree Coding of Wavelet Coefficients Successive-Approximation
Lecture Notes. Advanced Discrete Structures COT S
Lecture Notes Advanced Discrete Structures COT 4115.001 S15 2015-01-27 Recap ADFGX Cipher Block Cipher Modes of Operation Hill Cipher Inverting a Matrix (mod n) Encryption: Hill Cipher Example Multiple
Module 4. Multi-Resolution Analysis. Version 2 ECE IIT, Kharagpur
Module 4 Multi-Resolution Analysis Lesson Multi-resolution Analysis: Discrete avelet Transforms Instructional Objectives At the end of this lesson, the students should be able to:. Define Discrete avelet
Welcome to Comp 411! 2) Course Objectives. 1) Course Mechanics. 3) Information. I thought this course was called Computer Organization
Welcome to Comp 4! I thought this course was called Computer Organization David Macaulay ) Course Mechanics 2) Course Objectives 3) Information L - Introduction Meet the Crew Lectures: Leonard McMillan
What is Image Deblurring?
What is Image Deblurring? When we use a camera, we want the recorded image to be a faithful representation of the scene that we see but every image is more or less blurry, depending on the circumstances.
Coping with disk crashes
Lecture 04.03 Coping with disk crashes By Marina Barsky Winter 2016, University of Toronto Disk failure types Intermittent failure Disk crash the entire disk becomes unreadable, suddenly and permanently
22A-2 SUMMER 2014 LECTURE 5
A- SUMMER 0 LECTURE 5 NATHANIEL GALLUP Agenda Elimination to the identity matrix Inverse matrices LU factorization Elimination to the identity matrix Previously, we have used elimination to get a system
6.02 Fall 2012 Lecture #1
6.02 Fall 2012 Lecture #1 Digital vs. analog communication The birth of modern digital communication Information and entropy Codes, Huffman coding 6.02 Fall 2012 Lecture 1, Slide #1 6.02 Fall 2012 Lecture
2.6 Complexity Theory for Map-Reduce. Star Joins 2.6. COMPLEXITY THEORY FOR MAP-REDUCE 51
2.6. COMPLEXITY THEORY FOR MAP-REDUCE 51 Star Joins A common structure for data mining of commercial data is the star join. For example, a chain store like Walmart keeps a fact table whose tuples each
Lecture 6: Introducing Complexity
COMP26120: Algorithms and Imperative Programming Lecture 6: Introducing Complexity Ian Pratt-Hartmann Room KB2.38: email: ipratt@cs.man.ac.uk 2015 16 You need this book: Make sure you use the up-to-date
Math 31 Lesson Plan. Day 2: Sets; Binary Operations. Elizabeth Gillaspy. September 23, 2011
Math 31 Lesson Plan Day 2: Sets; Binary Operations Elizabeth Gillaspy September 23, 2011 Supplies needed: 30 worksheets. Scratch paper? Sign in sheet Goals for myself: Tell them what you re going to tell
4. Quantization and Data Compression. ECE 302 Spring 2012 Purdue University, School of ECE Prof. Ilya Pollak
4. Quantization and Data Compression ECE 32 Spring 22 Purdue University, School of ECE Prof. What is data compression? Reducing the file size without compromising the quality of the data stored in the
A Mathematical Theory of Communication
A Mathematical Theory of Communication Ben Eggers Abstract This paper defines information-theoretic entropy and proves some elementary results about it. Notably, we prove that given a few basic assumptions
L. Yaroslavsky. Fundamentals of Digital Image Processing. Course
L. Yaroslavsky. Fundamentals of Digital Image Processing. Course 0555.330 Lec. 6. Principles of image coding The term image coding or image compression refers to processing image digital data aimed at
We are here. Assembly Language. Processors Arithmetic Logic Units. Finite State Machines. Circuits Gates. Transistors
CSC258 Week 3 1 Logistics If you cannot login to MarkUs, email me your UTORID and name. Check lab marks on MarkUs, if it s recorded wrong, contact Larry within a week after the lab. Quiz 1 average: 86%
LMS Popular Lectures. Codes. Peter J. Cameron
LMS Popular Lectures Codes Peter J. Cameron p.j.cameron@qmul.ac.uk June/July 2001 Think of a number... Think of a number between 0 and 15. Now answer the following questions. You are allowed to lie once.
2018/5/3. YU Xiangyu
2018/5/3 YU Xiangyu yuxy@scut.edu.cn Entropy Huffman Code Entropy of Discrete Source Definition of entropy: If an information source X can generate n different messages x 1, x 2,, x i,, x n, then the
MITOCW R11. Double Pendulum System
MITOCW R11. Double Pendulum System The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for
c 1 v 1 + c 2 v 2 = 0 c 1 λ 1 v 1 + c 2 λ 1 v 2 = 0
LECTURE LECTURE 2 0. Distinct eigenvalues I haven t gotten around to stating the following important theorem: Theorem: A matrix with n distinct eigenvalues is diagonalizable. Proof (Sketch) Suppose n =
SIGNAL COMPRESSION. 8. Lossy image compression: Principle of embedding
SIGNAL COMPRESSION 8. Lossy image compression: Principle of embedding 8.1 Lossy compression 8.2 Embedded Zerotree Coder 161 8.1 Lossy compression - many degrees of freedom and many viewpoints The fundamental
CS4800: Algorithms & Data Jonathan Ullman
CS4800: Algorithms & Data Jonathan Ullman Lecture 22: Greedy Algorithms: Huffman Codes Data Compression and Entropy Apr 5, 2018 Data Compression How do we store strings of text compactly? A (binary) code
MITOCW ocw f99-lec31_300k
MITOCW ocw-18.06-f99-lec31_300k OK. So, coming nearer the end of the course, this lecture will be a mixture of the linear algebra that comes with a change of basis. And a change of basis from one basis
Wavelets and Signal Processing
Wavelets and Signal Processing John E. Gilbert Mathematics in Science Lecture April 30, 2002. Publicity Mathematics In Science* A LECTURE SERIES FOR UNDERGRADUATES Wavelets Professor John Gilbert Mathematics
RLE = [ ; ], with compression ratio (CR) = 4/8. RLE actually increases the size of the compressed image.
![RLE = [ ; ], with compression ratio (CR) = 4/8. RLE actually increases the size of the compressed image.](/thumbs/89/98396699.jpg "RLE = [ ; ], with compression ratio (CR) = 4/8. RLE actually increases the size of the compressed image.")
MP/BME 574 Application Solutions. (2 pts) a) From first principles in class, we expect the entropy of the checkerboard image to be since this is the bit depth of the image and the frequency of each value
Lecture 1 : Data Compression and Entropy
CPS290: Algorithmic Foundations of Data Science January 8, 207 Lecture : Data Compression and Entropy Lecturer: Kamesh Munagala Scribe: Kamesh Munagala In this lecture, we will study a simple model for
Compression. Reality Check 11 on page 527 explores implementation of the MDCT into a simple, working algorithm to compress audio.
C H A P T E R 11 Compression The increasingly rapid movement of information around the world relies on ingenious methods of data representation, which are in turn made possible by orthogonal transformations.the
How to write maths (well)
How to write maths (well) Dr Euan Spence 29 September 2017 These are the slides from a talk I gave to the new first-year students at Bath, annotated with some of the things I said (which appear in boxes
Nearest Neighbor Search with Keywords: Compression
Nearest Neighbor Search with Keywords: Compression Yufei Tao KAIST June 3, 2013 In this lecture, we will continue our discussion on: Problem (Nearest Neighbor Search with Keywords) Let P be a set of points
Algorithms for Data Science
Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Tuesday, December 1, 2015 Outline 1 Recap Balls and bins 2 On randomized algorithms 3 Saving space: hashing-based
Next topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
Data Structures in Java
Data Structures in Java Lecture 20: Algorithm Design Techniques 12/2/2015 Daniel Bauer 1 Algorithms and Problem Solving Purpose of algorithms: find solutions to problems. Data Structures provide ways of
Exam 2 Solutions. x 1 x. x 4 The generating function for the problem is the fourth power of this, (1 x). 4
Math 5366 Fall 015 Exam Solutions 1. (0 points) Find the appropriate generating function (in closed form) for each of the following problems. Do not find the coefficient of x n. (a) In how many ways can
March 19 - Solving Linear Systems
March 19 - Solving Linear Systems Welcome to linear algebra! Linear algebra is the study of vectors, vector spaces, and maps between vector spaces. It has applications across data analysis, computer graphics,
Chapter 2: Source coding
Chapter 2: meghdadi@ensil.unilim.fr University of Limoges Chapter 2: Entropy of Markov Source Chapter 2: Entropy of Markov Source Markov model for information sources Given the present, the future is independent
Intro to Information Theory
Intro to Information Theory Math Circle February 11, 2018 1. Random variables Let us review discrete random variables and some notation. A random variable X takes value a A with probability P (a) 0. Here
Algorithms: COMP3121/3821/9101/9801
NEW SOUTH WALES Algorithms: COMP3121/3821/9101/9801 Aleks Ignjatović School of Computer Science and Engineering University of New South Wales TOPIC 4: THE GREEDY METHOD COMP3121/3821/9101/9801 1 / 23 The
MATH 320, WEEK 7: Matrices, Matrix Operations
MATH 320, WEEK 7: Matrices, Matrix Operations 1 Matrices We have introduced ourselves to the notion of the grid-like coefficient matrix as a short-hand coefficient place-keeper for performing Gaussian
Math 31 Lesson Plan. Day 5: Intro to Groups. Elizabeth Gillaspy. September 28, 2011
Math 31 Lesson Plan Day 5: Intro to Groups Elizabeth Gillaspy September 28, 2011 Supplies needed: Sign in sheet Goals for students: Students will: Improve the clarity of their proof-writing. Gain confidence
2. Polynomials. 19 points. 3/3/3/3/3/4 Clearly indicate your correctly formatted answer: this is what is to be graded. No need to justify!
1. Short Modular Arithmetic/RSA. 16 points: 3/3/3/3/4 For each question, please answer in the correct format. When an expression is asked for, it may simply be a number, or an expression involving variables
Functions - Definitions and first examples and properties
Functions - Definitions and first examples and properties Jean-Luc Bouchot jean-luc.bouchot@drexel.edu February 4, 2013 1 Definition and examples 1.1 Preliminaries What is a function? Besides its mathematical
Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim
Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its
3.4 Pascal s Pride. A Solidify Understanding Task
3.4 Pascal s Pride A Solidify Understanding Task Multiplying polynomials can require a bit of skill in the algebra department, but since polynomials are structured like numbers, multiplication works very
The PROMYS Math Circle Problem of the Week #3 February 3, 2017
The PROMYS Math Circle Problem of the Week #3 February 3, 2017 You can use rods of positive integer lengths to build trains that all have a common length. For instance, a train of length 12 is a row of
Introduction to Algebra: The First Week
Introduction to Algebra: The First Week Background: According to the thermostat on the wall, the temperature in the classroom right now is 72 degrees Fahrenheit. I want to write to my friend in Europe,
Dynamics of Ocean Structures Prof. Dr. Srinivasan Chandrasekaran Department of Ocean Engineering Indian Institute of Technology, Madras
Dynamics of Ocean Structures Prof. Dr. Srinivasan Chandrasekaran Department of Ocean Engineering Indian Institute of Technology, Madras Module - 1 Lecture - 20 Orthogonality of modes (Refer Slide Time:
6.080 / Great Ideas in Theoretical Computer Science Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Counting in Different Number Systems
Counting in Different Number Systems Base 1 (Decimal) is important because that is the base that we first learn in our culture. Base 2 (Binary) is important because that is the base used for computer codes
Analog Computing: a different way to think about building a (quantum) computer
Analog Computing: a different way to think about building a (quantum) computer November 24, 2016 1 What is an analog computer? Most of the computers we have around us today, such as desktops, laptops,
Signal Processing With Wavelets
Signal Processing With Wavelets JAMES MONK Niels Bohr Institute, University of Copenhagen. Reminder of the Fourier Transform g(!) = 1 p 2 Z 1 1 f(t)e i!t dt Tells you the frequency components in a signal
CSE 421 Greedy: Huffman Codes
CSE 421 Greedy: Huffman Codes Yin Tat Lee 1 Compression Example 100k file, 6 letter alphabet: File Size: ASCII, 8 bits/char: 800kbits 2 3 > 6; 3 bits/char: 300kbits better: 2.52 bits/char 74%*2 +26%*4:
1 Maintaining a Dictionary
15-451/651: Design & Analysis of Algorithms February 1, 2016 Lecture #7: Hashing last changed: January 29, 2016 Hashing is a great practical tool, with an interesting and subtle theory too. In addition
Communication Engineering Prof. Surendra Prasad Department of Electrical Engineering Indian Institute of Technology, Delhi
Communication Engineering Prof. Surendra Prasad Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 41 Pulse Code Modulation (PCM) So, if you remember we have been talking
State of the art Image Compression Techniques
Chapter 4 State of the art Image Compression Techniques In this thesis we focus mainly on the adaption of state of the art wavelet based image compression techniques to programmable hardware. Thus, an
NEWS ENGLISH LESSONS.com
NEWS ENGLISH LESSONS.com Photo shows a billion stars MANY FLASH AND ONLINE ACTIVITIES FOR THIS LESSON, PLUS A LISTENING, AT: http://www.newsenglishlessons.com/1203/120330-stars.html IN THIS LESSON: Photo:
18.310A Final exam practice questions
18.310A Final exam practice questions This is a collection of practice questions, gathered randomly from previous exams and quizzes. They may not be representative of what will be on the final. In particular,
Tutorial Three: Loops and Conditionals
Tutorial Three: Loops and Conditionals Imad Pasha Chris Agostino February 18, 2015 1 Introduction In lecture Monday we learned that combinations of conditionals and loops could make our code much more
CHAPTER 3: THE INTEGERS Z
CHAPTER 3: THE INTEGERS Z MATH 378, CSUSM. SPRING 2009. AITKEN 1. Introduction The natural numbers are designed for measuring the size of finite sets, but what if you want to compare the sizes of two sets?
Math Circle at FAU 10/27/2018 SOLUTIONS
Math Circle at FAU 10/27/2018 SOLUTIONS 1. At the grocery store last week, small boxes of facial tissue were priced at 4 boxes for $5. This week they are on sale at 5 boxes for $4. Find the percent decrease
UNM - PNM STATEWIDE MATHEMATICS CONTEST XLII. November 7, 2009 First Round Three Hours
UNM - PNM STATEWIDE MATHEMATICS CONTEST XLII November 7, 009 First Round Three Hours 1. Let f(n) be the sum of n and its digits. Find a number n such that f(n) = 009. Answer: 1990 If x, y, and z are digits,
Lecture 16 Oct 21, 2014
CS 395T: Sublinear Algorithms Fall 24 Prof. Eric Price Lecture 6 Oct 2, 24 Scribe: Chi-Kit Lam Overview In this lecture we will talk about information and compression, which the Huffman coding can achieve
8 The SVD Applied to Signal and Image Deblurring
8 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an
Number Representation and Waveform Quantization
1 Number Representation and Waveform Quantization 1 Introduction This lab presents two important concepts for working with digital signals. The first section discusses how numbers are stored in memory.
Discrete Wavelet Transform
Discrete Wavelet Transform [11] Kartik Mehra July 2017 Math 190s Duke University "1 Introduction Wavelets break signals up and then analyse them separately with a resolution that is matched with scale.
CHAPTER 8: MATRICES and DETERMINANTS
(Section 8.1: Matrices and Determinants) 8.01 CHAPTER 8: MATRICES and DETERMINANTS The material in this chapter will be covered in your Linear Algebra class (Math 254 at Mesa). SECTION 8.1: MATRICES and
Designing Information Devices and Systems I Fall 2018 Lecture Notes Note Introduction to Linear Algebra the EECS Way
EECS 16A Designing Information Devices and Systems I Fall 018 Lecture Notes Note 1 1.1 Introduction to Linear Algebra the EECS Way In this note, we will teach the basics of linear algebra and relate it
1 Last time: linear systems and row operations
1 Last time: linear systems and row operations Here s what we did last time: a system of linear equations or linear system is a list of equations a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22
BASIC COMPRESSION TECHNIQUES
BASIC COMPRESSION TECHNIQUES N. C. State University CSC557 Multimedia Computing and Networking Fall 2001 Lectures # 05 Questions / Problems / Announcements? 2 Matlab demo of DFT Low-pass windowed-sinc
Lab 2 Worksheet. Problems. Problem 1: Geometry and Linear Equations
Lab 2 Worksheet Problems Problem : Geometry and Linear Equations Linear algebra is, first and foremost, the study of systems of linear equations. You are going to encounter linear systems frequently in
Singular Value Decomposition and Digital Image Compression
Singular Value Decomposition and Digital Image Compression Chris Bingham December 1, 016 Page 1 of Abstract The purpose of this document is to be a very basic introduction to the singular value decomposition
Linear Algebra. Introduction. Marek Petrik 3/23/2017. Many slides adapted from Linear Algebra Lectures by Martin Scharlemann
Linear Algebra Introduction Marek Petrik 3/23/2017 Many slides adapted from Linear Algebra Lectures by Martin Scharlemann Midterm Results Highest score on the non-r part: 67 / 77 Score scaling: Additive
Lecture 4: Constructing the Integers, Rationals and Reals
Math/CS 20: Intro. to Math Professor: Padraic Bartlett Lecture 4: Constructing the Integers, Rationals and Reals Week 5 UCSB 204 The Integers Normally, using the natural numbers, you can easily define
Shannon-Fano-Elias coding
Shannon-Fano-Elias coding Suppose that we have a memoryless source X t taking values in the alphabet {1, 2,..., L}. Suppose that the probabilities for all symbols are strictly positive: p(i) > 0, i. The
Pr[X = s Y = t] = Pr[X = s] Pr[Y = t]
![Pr[X = s Y = t] = Pr[X = s] Pr[Y = t]](/thumbs/94/120114683.jpg "Pr[X = s Y = t] = Pr[X = s] Pr[Y = t]")
Homework 4 By: John Steinberger Problem 1. Recall that a real n n matrix A is positive semidefinite if A is symmetric and x T Ax 0 for all x R n. Assume A is a real n n matrix. Show TFAE 1 : (a) A is positive
MATRIX MULTIPLICATION AND INVERSION
MATRIX MULTIPLICATION AND INVERSION MATH 196, SECTION 57 (VIPUL NAIK) Corresponding material in the book: Sections 2.3 and 2.4. Executive summary Note: The summary does not include some material from the
Designing Information Devices and Systems I Spring 2018 Lecture Notes Note Introduction to Linear Algebra the EECS Way
EECS 16A Designing Information Devices and Systems I Spring 018 Lecture Notes Note 1 1.1 Introduction to Linear Algebra the EECS Way In this note, we will teach the basics of linear algebra and relate
Designing Information Devices and Systems I Spring 2018 Homework 13
EECS 16A Designing Information Devices and Systems I Spring 2018 Homework 13 This homework is due April 30, 2018, at 23:59. Self-grades are due May 3, 2018, at 23:59. Submission Format Your homework submission
A Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
Digital Image Processing Lectures 25 & 26
Lectures 25 & 26, Professor Department of Electrical and Computer Engineering Colorado State University Spring 2015 Area 4: Image Encoding and Compression Goal: To exploit the redundancies in the image
Fast Progressive Wavelet Coding
PRESENTED AT THE IEEE DCC 99 CONFERENCE SNOWBIRD, UTAH, MARCH/APRIL 1999 Fast Progressive Wavelet Coding Henrique S. Malvar Microsoft Research One Microsoft Way, Redmond, WA 98052 E-mail: malvar@microsoft.com
Image Registration Lecture 2: Vectors and Matrices
Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this
University of the Virgin Islands, St. Thomas January 14, 2015 Algorithms and Programming for High Schoolers. Lecture 5
University of the Virgin Islands, St. Thomas January 14, 2015 Algorithms and Programming for High Schoolers Numerical algorithms: Lecture 5 Today we ll cover algorithms for various numerical problems:
ECE 521. Lecture 11 (not on midterm material) 13 February K-means clustering, Dimensionality reduction
ECE 521 Lecture 11 (not on midterm material) 13 February 2017 K-means clustering, Dimensionality reduction With thanks to Ruslan Salakhutdinov for an earlier version of the slides Overview K-means clustering
Singular Value Decompsition
Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost