Machine Learning: Basis and Wavelet 32 157 146 204 + + + + + - + - 김화평 (CSE ) Medical Image computing lab 서진근교수연구실 7 22 38 191 17 83 188 211 71 167 194 207 135 46 40-17 18 42 20 44 31 7 13-32 + + - - + - - + 159 187 201 216 27 4-17 1 Haar DWT in 2 levels
Mission: Find a feed-forward function from labeled training data,, :,,, such that,,,. Machine learning is the field of study that gives computers the ability to learn the feed-forward function without being explicitly programmed. Supervised learning is the machine learning technique of finding a feed-forward function iteratively from labeled training data,, : 1,,, such that, 1,,. Machine learning: Why it is and why it matters. Humans can typically create one or two good models a week; machine learning can create thousands of models a week
Basis: Fourier Transform Every function can be expressed as a linear combination of basis functions, 1, where,, is a set of orthonormal basis, 0. Basis The Fourier transform of is defined by. Each fourier transform acts as a basis to demonstrate the ability to distinguish different signals.
Approximation by 4 principal components (basis) only Slide Credit: Vaclav
Why wavelets? Scattering convolution network For appropriate wavelets, such a dreamlike kernel Φ can be represented by scattering coefficients using wavelet transform. Wavelets are uniformly stable to deformations. Wavelets separate multiscale information. What is wavelet? Wavelets provide sparse representations.,,,,,,, Wavelet coefficients:,, : average : higher frequencies, and,,
Review on Wavelet 계산과학공학과통합과정김화평 Medical Image computing lab 서진근교수연구실 7 22 38 191 17 83 188 211 32 157 146 204 135 40 46-17 20 44 31 7 + + + + + + - - + - + - + - - + 71 167 194 207 18 42 13-32 159 187 201 216 27 4-17 1 Haar DWT in 2 levels
Discrete Haar wavelet Transform Wavelet basis functions: The family of functions, :,, dyadic translations and dilations of a mother wavelet function, construct a complete orthonormal Hilbert basis.,,, where,,,., / for., 2 / 2, 2 / 2 1, 2 2 1, 22 2, 2 2 3, 2 2 4
Approximate the signal from wavelet coefficients,,.,,,,,,,,,,
Wavelet filter bank g 2 6 6 level 3 coefficients g 2 6 2 0 8 g 2 4 8 2 2 2 level 2 coefficients 9 4 7 3 5 6 10 2 2 1-1 -2-2 level 1 coefficients Low pass filter g, High pass filter, 6
Example of discrete Haar Wavelet Transform for sound signal Scattering convolution network : average : detail(backward difference) Wavelet coefficients:
Scattering convolution network Example of continuous Wavelet Transform for EEG signal EEG 10-20 System x x
Scattering convolution network,, lim Φ Φ is a diffeomorphism,
Example of Scattering transform for EEG signal Scattering convolution network,,
Subspace Methods: PCA, ICA www.deeplearningbook.org Written By Ian Goodfellow Yoshua Bengio Aaron Courville
Basics in Principal Component Analysis Suppose we would like to apply lossy compression to a collection of m points,,. Lossy compression means storing the points in a way that requires less memory but may lose some precision. Slide Credit: Vaclav
Approximation by 4 principal components only High-dimensional data s often lies on or near a much lower dimensional, curved manifold. A good way to represent data points is by low-dimensional coordinates. The lowdimensional representation of the data should capture information about high-dimensional pairwise distance.
Approximation by 4 principal components only Slide Credit: Vaclav
Encoding/Decoding function Let f: R ln be an encoding function which represents each data point x by a point c fx in the lowdimensional space R. PCA is defined by our choice of the decoding function g: R such that g f.letgc Dc where D R defines the decoding. PCA constraints the columns of D to be orthonormal vectors in R. =,,,
Let where defines the decoding. [ ].... 1 ST column 2 nd column 3 rd column 4 th column Slide Credit: Vaclav
PCA constraints the columns of vectors in. to be orthonormal To generate from, one may use It is easy to see that. This optimization problem can be solve by..
How to choose encoding matrix By defining the encoding function, we can define the PCA reconstruction operation An encoding matrix canbechosenby subject to.
How to extract the first principle component In the case when, can be simplified in a single vector and. Denoting,,, the first principle component can be obtained by A simple computation shows that.. This optimization problem may be solved using eigenvalue decomposition. Specifically, is given by the eigenvector of corresponding to the largest eigenvalue.
The first principle component 1 st row 32 nd row Slide Credit: Vaclav
More detailed explanation in computing the first principle component. =..
Subspace Methods Slide Credit: Vaclav