Mixture of metrics optimization for machine learning problems

Transcription

1 machine learning and Marek mieja Faculty of Mathematics and Computer Science, Jagiellonian University TFML 2015 B dlewo, February 16-21

2 How to select data representation and metric for a given data set?

3 How to select data representation and metric for a given data set? Combining various data representations and.

4 How to select data representation and metric for a given data set? Combining various data representations and. Optimizing a linear combination of selected distance measures.

5

6 real - life problem of chemoinformatics

7

8 Representation of molecules Fingerprints are binary strings where a given bit indicates the absence or presence of particular pattern.

9 Representation of molecules Fingerprints are binary strings where a given bit indicates the absence or presence of particular pattern. Problems: high dimensionality

10 Representation of molecules Fingerprints are binary strings where a given bit indicates the absence or presence of particular pattern. Problems: high dimensionality they are not unique

11 Biological activity IC 50, EC 50, K d

12 Biological activity IC 50, EC 50, K d a binding constant K i was used

13 Biological activity IC 50, EC 50, K d a binding constant K i was used prediction of molecule's activity is repeated several times

14 Biological activity IC 50, EC 50, K d a binding constant K i was used prediction of molecule's activity is repeated several times the chemical compound were considered as active if K i 100 while for K i inactive

15 Intuitively: design a measure which gives low values for compounds with similar activities while high values are assigned for compounds with dierent values of K i.

16 Intuitively: design a measure which gives low values for compounds with similar activities while high values are assigned for compounds with dierent values of K i. METRIC LEARNING

17 Multidimensional Scaling (1994) Locally Linear Embedding (Roweis and Saul, 2000) learning a Mahalanobis metric by Xing et al. (2003) kernel regression (Takeda et al., 2006)...

18 Our aims optimize existing and representations use of combination distance measures coecients improve classication and clustering

19 Our aims optimize existing and representations use of combination distance measures coecients improve classication and clustering a(x, y) ω 1 d 1 (x, y) ω n d n (x, y)

20 Optimization X - data set a : X X [0, )

21 Optimization X - data set a : X X [0, ) d : X X R

22 Optimization X - data set a : X X [0, ) d : X X R d ω (x, y) := ω 1 d 1 (x, y) ω n d n (x, y)

23 Optimization X - data set a : X X [0, ) d : X X R d ω (x, y) := ω 1 d 1 (x, y) ω n d n (x, y) in practice d ω (x, y) := ω 0 + ω 1 d 1 (x, y) ω n d n (x, y)

24 Optimization X - data set a : X X [0, ) d : X X R d ω (x, y) := ω 1 d 1 (x, y) ω n d n (x, y) in practice d ω (x, y) := ω 0 + ω 1 d 1 (x, y) ω n d n (x, y) or less formally K i (x) K i (y) = ω 0 + ω 1 d(x, y) ω n d(x, y) + ɛ,

25 Optimization X - data set a : X X [0, ) d : X X R d ω (x, y) := ω 1 d 1 (x, y) ω n d n (x, y) in practice d ω (x, y) := ω 0 + ω 1 d 1 (x, y) ω n d n (x, y) or less formally K i (x) K i (y) = ω 0 + ω 1 d(x, y) ω n d(x, y) + ɛ, x,y X (a(x, y) d ω(x, y)) 2

26 Data sets

27 Dissimilarity Buser: Tanimoto: cd+c cd+a+b c c a+b c

28 knn

29 k-means

30 hierarchical clustering

31 after optimization process

32 after optimization process

33 more explanatory variables

34 more explanatory variables

35 metric learning problem a single function which combines data representation-metric pairs can improve the performance of metric-based algorithms

36 Aczel A., Sounderpandian J.; Complete Business Statistics, McGraw Hill, NY, Atkeson C., Moore A., Schaal S; Locally weighted learning, Articial Intelligence Review 11, 1997, pp Bar-Hillel A., Hertz T., Shental N., Weinshall D.; Learning a Mahalanobis metric from equivalence constraints, Journal of Machine Learning Research 6, 2005, pp Roweis S.T., Saul L.K.; Nonlinear dimensionality reduction by locally linear embedding, Science 290, 2000, pp Takeda, H., Farsiu, S. and Milanfar, P.; Robust kernel regression for restoration and reconstruction of images from sparse noisy data, IEEE International Conference on Image Processing, 2006, pp Xing E.P., and Ng A.Y. and Jordan M.I., Russell S.; Distance Metric Learning, With Application To Clustering With Side-Information, Advances in Neural Information Processing Systems 15, 2003, pp