Tensor network vs Machine learning. Song Cheng ( 程嵩 ) IOP, CAS

Transcription

1 Tensor network vs Machine learning Song Cheng ( 程嵩 ) IOP, CAS physichengsong@iphy.ac.cn

2 Outline Tensor network in a nutshell TN concepts in machine learning TN methods in machine learning

3 Outline Tensor network in a nutshell TN concepts in machine learning TN methods in machine learning

4 Orus, R. (2014). A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States. Annals of Physics, 349,

5 Ψ v = c ijlk ijlk φ i (v 1 ) φ j (v 2 ) φ k (v 3 ) φ l (v 4 )... Tensors are local building blocks for the quantum state(like LEGO) Locality lead to low rank approximation Orus, R. (2014). A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States. Annals of Physics, 349,

6

7 1, Find right TN representation 2, Contract TN to calculate physics quantity

8

9 Coarse-graining Projection Variation

10 Outline Tensor network in a nutshell TN concepts in machine learning TN methods in machine learning

11 First contact: 2013

12 Bény, C. (2013). Deep learning and the renormalization group. arxiv: [Quant-Ph]. Retrieved from Mehta, P., & Schwab, D. J. (2014). An exact mapping between the variational renormalization group and deep learning. arxiv Preprint arxiv: Retrieved from

13 Cichocki, A. (2014). Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems. arxiv: [Cs, Math]. Retrieved from

14 1. Cichocki, A. et al. Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1. Foundations and Trends in Machine Learning 9, (2016).

15 Fashionable: 2015

16

17 Carrasquilla, J., & Melko, R. G. (2016). Machine learning phases of matter. arxiv Preprint arxiv: Retrieved from

18 Stoudenmire, E. M., & Schwab, D. J. (2016). Supervised Learning with Quantum-Inspired Tensor Networks. arxiv: [Cond-Mat, Stat]. Retrieved from

19 Carleo, G., & Troyer, M. (2017). Solving the quantum many-body problem with artificial neural networks. Science, 355(6325),

20 Reasoning: 2016-

21 Number of atoms in universe: Number of possible samples for 28 * 28 gray image: Number of parameters in RBM : 10 4 ~10 8 Why they succeed? Possible space of image Meaningful image is limited by law of physics H. W. Lin and M. Tegmark, Why does deep and cheap learning work so well,arxiv:

22 Lin, H. W., & Tegmark, M. (2016). Why does deep and cheap learning work so well? arxiv: [Cond-Mat, Stat]. Retrieved from

23 Number of atoms in universe: Number of states in manybody Hilbert space: tremendous! Number of parameters in manybody numerical algorithm: negligible Why they succeed? Many body Hilbert space Most of physical quantum many body states fulfill the area law. H. W. Lin and M. Tegmark, Why does deep and cheap learning work so well,arxiv:

24 Similarities between machine learning and quantum many body physics: Using few parameters to approximate exponentially large number of probabilities of data. H. W. Lin and M. Tegmark, Why does deep and cheap learning work so well,arxiv:

25 Deng, D.-L., Li, X., & Sarma, S. D. (2017). Quantum Entanglement in Neural Network States. Physical Review X, 7(2).

26 Gao, X., & Duan, L.-M. (2017). Efficient Representation of Quantum Many-body States with Deep Neural Networks. arxiv: [Cond-Mat, Physics:quant-Ph]. Retrieved from

27 Chen, J., Cheng, S., Xie, H., Wang, L., & Xiang, T. (2017). On the Equivalence of Restricted Boltzmann Machines and Tensor Network States. arxiv: [Cond-Mat, Physics:quant-Ph, Stat]. Retrieved from

28 Cheng, S., Chen, J. & Wang, L. Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines. arxiv: [cond-mat, physics:physics, physics:quant-ph, stat] (2017). 1.

29 Levine, Y., Yakira, D., Cohen, N., & Shashua, A. (2017). Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design. arxiv: [Quant-Ph]. Retrieved from

30 Neural network quantum states vs tensor network states

31 Huang, Y., & Moore, J. E. (2017). Neural network representation of tensor network and chiral states. arxiv: [Cond-Mat]. Retrieved from

32 Glasser, I., Pancotti, N., August, M., Rodriguez, I. D., & Cirac, J. I. (2017). Neural Networks Quantum States, String- Bond States and chiral topological states. arxiv: [Cond-Mat, Physics:quant-Ph, Stat]. Retrieved from

33 Tensor concepts in language model

34 Gallego, A. J., & Orus, R. (2017). The physical structure of grammatical correlations: equivalences, formalizations and consequences. arxiv: [Cond-Mat, Physics:physics, Physics:quant-Ph]. Retrieved from

35 Benefit Entanglement spectrum Gauge invariance Exact tensor decomposition math Expression power evaluation Tensor Algorithm

36 Disadvantage High cost Non-local terms are excluded

37 Outline Tensor network in a nutshell TN concepts in machine learning TN method in machine learning

38 representation Stoudenmire, E. M., & Schwab, D. J. (2016). Supervised Learning with Quantum-Inspired Tensor Networks. arxiv: [Cond-Mat, Stat]. Retrieved from

39 target

40 (tensor) gradient descent

41

42

43

44 Zhao-Yu Han, Jun Wang, Heng Fan, Lei Wang, Pan Zhang, Unsupervised Generative Modeling Using Matrix Product States,

45 Training Algorithm: Generative Algorithm:

46 Liu, D., Ran, S.-J., Wittek, P., Peng, C., García, R. B., Su, G., & Lewenstein, M. (2017). Machine Learning by Two-Dimensional Hierarchical Tensor Networks: A Quantum Information Theoretic Perspective on Deep Architectures. arxiv: [Cond- Mat, Physics:physics, Physics:quant-Ph, Stat]. Retrieved from

47 Liu, D., Ran, S.-J., Wittek, P., Peng, C., García, R. B., Su, G., & Lewenstein, M. (2017). Machine Learning by Two-Dimensional Hierarchical Tensor Networks: A Quantum Information Theoretic Perspective on Deep Architectures. arxiv: [Cond-Mat, Physics:physics, Physics:quant-Ph, Stat]. Retrieved from

48 Koch-Janusz, M., & Ringel, Z. (2017). Mutual Information, Neural Networks and the Renormalization Group. arxiv: [Cond-Mat]. Retrieved from

49 Stoudenmire, E. M. Learning Relevant Features of Data with Multi-scale Tensor Networks. arxiv: [condmat, stat] (2017). 1.

50

51 Gao, X., Zhang, Z. & Duan, L. An efficient quantum algorithm for generative machine learning. arxiv: [quant-ph, stat] (2017).

52 Thanks

53 Take home message Tensor networks correlated to many ML architecture Datasets in ML are not totally unfamiliar to physicists Tensor viewpoints/techniques could be transferred to ML Lots of work to be done.