Introduction to Machine learning

Size: px

Start display at page:

Download "Introduction to Machine learning"

Christine Shelton
5 years ago
Views:

1 Introduction to Machine learning Some slides and images are taken from: David Wolfe Corne Wikipedia Geoffrey A. Hinton

2 Examples 1

3 Examples 1 WaveNet

4 Examples 1 WaveNet

5 Examples 2

6 Basics of Machine Learning Feedforward nets Backpropagation Convolutional networks Boltzmann machines Deep Belief Nets

7 Supervised learning 1 Function approximation Shallow ML methods Deep ML methods Regression Kernel Regression x y = F(x) Feedforward networks Deep tensor networks ConvNets

8 Supervised learning 2 Classification

9 Supervised learning 2 Classification Shallow ML methods Deep ML methods Support Vector Machines (SVMs) Kernel SVMs Inputs Classes

10 Unsupervised learning 1 Learning distributions

11 Unsupervised learning 1 Learning distributions Shallow ML methods Deep ML methods Markov random fields Boltzmann machines Deep Belief Nets variational Autoencoders

12 Unsupervised learning 2 Classification Shallow ML methods Deep ML methods Clustering Deep transformation + clustering

13 Plan for the first talk 1. Linear least squares regression Example: Markov state models and Koopman models 2. Validation, cross-validation and hyperparameter selection 3. Regularization and sparsity Example: Sindy 4. Kernel ridge regression

14 Linear least squares regression

15 Regression

16 Regression

17 Regression

18 Linear least squares

19 Normal equations

20 Normal equations Solution methods: 1) do not literally do this!

21 Normal equations Solution methods: 2) Cholesky decomposition of CXX (still numerically unstable, but sometimes useful) 3) Orthogonolization of X (e.g. via Householder reflections) stable 4) SVD of X to perform the pseudoinversion stable

22 Markov state models and Koopman models of molecular dynamics

23 Markov state models and Koopman models of molecular dynamics

24 Markov state models and Koopman models of molecular dynamics

25 Markov state models

26 Koopman models Koopman model: Wu et al: JCP 146, (2017) Eigenvalues and eigenvectors of K have been used to perform dimension reduction in: VAC (variational approach of conformation dynamics) Noé and Nüske: SIAM MMS 11, (2013) Nüske et al: JCTC 10, (2014) EDMD (Extended dynamic mode decomposition) Williams, Kevrekidis and Rowley: J. Nonlinear Sci. 25, (2015) TICA (time-lagged independent component analysis) Perez-Hernandez et al: JCP 139, (2013) Schwantes and Pande: JCTC 9, (2013)

27 Validation, cross-validation and hyperparameter selection

28 Validation, cross-validation and hyperparameter selection

29 Validation

30 Validation

31 Validation

32 Cross-validation

33 Cross-validation

34 Hyperparameter selection

35 Regularization and sparsity

36 L2 / Tihonov regularization / Ridge regression

37 Sparsity-inducing regularization L0 (too hard) L1 (LASSO) Elastic net Good approach in practice: - Use L1 regularization and annihilate small elements with thresholding. - Select regularization hyperparameters with cross-validation.

38 Sindy (sparse identification of nonlinear dynamics) Brunton, Proctor and Kutz: PNAS 113, (2016)

39 Kernel ridge regression

40 Kernel ridge regression Reminder: ridge regression Kernel formulation

41 Kernel ridge regression Kernel formulation

42 Kernel ridge regression Kernel formulation

43 Kernel ridge regression Key ideas

44 Example: polynomial Kernel

Introduction to Deep Learning

Introduction to Deep Learning Some slides and images are taken from: David Wolfe Corne Wikipedia Geoffrey A. Hinton https://www.macs.hw.ac.uk/~dwcorne/teaching/introdl.ppt Feedforward networks for function