Machine Learning: Basis and Wavelet 김화평 (CSE ) Medical Image computing lab 서진근교수연구실 Haar DWT in 2 levels

Size: px

Start display at page:

Download "Machine Learning: Basis and Wavelet 김화평 (CSE ) Medical Image computing lab 서진근교수연구실 Haar DWT in 2 levels"

Terence Lucas
6 years ago
Views:

Machine Learning: Basis and Wavelet 32 157 146 204 + + + + + -

17 83 188 211 71 167 194 207 135 46 40-17 18 42 20 44 31 7

1 Machine Learning: Basis and Wavelet 김화평 (CSE ) Medical Image computing lab 서진근교수연구실 Haar DWT in 2 levels

2 Mission: Find a feed-forward function from labeled training data,, :,,, such that,,,. Machine learning is the field of study that gives computers the ability to learn the feed-forward function without being explicitly programmed. Supervised learning is the machine learning technique of finding a feed-forward function iteratively from labeled training data,, : 1,,, such that, 1,,. Machine learning: Why it is and why it matters. Humans can typically create one or two good models a week; machine learning can create thousands of models a week

3 Basis: Fourier Transform Every function can be expressed as a linear combination of basis functions, 1, where,, is a set of orthonormal basis, 0. Basis The Fourier transform of is defined by. Each fourier transform acts as a basis to demonstrate the ability to distinguish different signals.

4 Approximation by 4 principal components (basis) only Slide Credit: Vaclav

Why wavelets? Scattering convolution network For appropriate wavelets, such a dreamlike kernel Φ can be represented by scattering coefficients using wavelet transform.

5 Why wavelets? Scattering convolution network For appropriate wavelets, such a dreamlike kernel Φ can be represented by scattering coefficients using wavelet transform. Wavelets are uniformly stable to deformations. Wavelets separate multiscale information. What is wavelet? Wavelets provide sparse representations.,,,,,,, Wavelet coefficients:,, : average : higher frequencies, and,,

6 Review on Wavelet 계산과학공학과통합과정김화평 Medical Image computing lab 서진근교수연구실 Haar DWT in 2 levels

7 Discrete Haar wavelet Transform Wavelet basis functions: The family of functions, :,, dyadic translations and dilations of a mother wavelet function, construct a complete orthonormal Hilbert basis.,,, where,,,., / for., 2 / 2, 2 / 2 1, 2 2 1, 22 2, 2 2 3, 2 2 4

8 Approximate the signal from wavelet coefficients,,.,,,,,,,,,,

9 Wavelet filter bank g level 3 coefficients g g level 2 coefficients level 1 coefficients Low pass filter g, High pass filter, 6

10 Example of discrete Haar Wavelet Transform for sound signal Scattering convolution network : average : detail(backward difference) Wavelet coefficients:

11 Scattering convolution network Example of continuous Wavelet Transform for EEG signal EEG System x x

12 Scattering convolution network,, lim Φ Φ is a diffeomorphism,

13 Example of Scattering transform for EEG signal Scattering convolution network,,

14 Subspace Methods: PCA, ICA Written By Ian Goodfellow Yoshua Bengio Aaron Courville

15 Basics in Principal Component Analysis Suppose we would like to apply lossy compression to a collection of m points,,. Lossy compression means storing the points in a way that requires less memory but may lose some precision. Slide Credit: Vaclav

16 Approximation by 4 principal components only High-dimensional data s often lies on or near a much lower dimensional, curved manifold. A good way to represent data points is by low-dimensional coordinates. The lowdimensional representation of the data should capture information about high-dimensional pairwise distance.

17 Approximation by 4 principal components only Slide Credit: Vaclav

18 Encoding/Decoding function Let f: R ln be an encoding function which represents each data point x by a point c fx in the lowdimensional space R. PCA is defined by our choice of the decoding function g: R such that g f.letgc Dc where D R defines the decoding. PCA constraints the columns of D to be orthonormal vectors in R. =,,,

19 Let where defines the decoding. [ ] ST column 2 nd column 3 rd column 4 th column Slide Credit: Vaclav

20 PCA constraints the columns of vectors in. to be orthonormal To generate from, one may use It is easy to see that. This optimization problem can be solve by..

21 How to choose encoding matrix By defining the encoding function, we can define the PCA reconstruction operation An encoding matrix canbechosenby subject to.

22 How to extract the first principle component In the case when, can be simplified in a single vector and. Denoting,,, the first principle component can be obtained by A simple computation shows that.. This optimization problem may be solved using eigenvalue decomposition. Specifically, is given by the eigenvector of corresponding to the largest eigenvalue.

23 The first principle component 1 st row 32 nd row Slide Credit: Vaclav

24 More detailed explanation in computing the first principle component. =..

25 Subspace Methods Slide Credit: Vaclav

Invariant Scattering Convolution Networks

Invariant Scattering Convolution Networks Joan Bruna and Stephane Mallat Submitted to PAMI, Feb. 2012 Presented by Bo Chen Other important related papers: [1] S. Mallat, A Theory for Multiresolution Signal