Data Mining & Machine Learning

Similar documents
Deep Learning Basics Lecture 10: Neural Language Models. Princeton University COS 495 Instructor: Yingyu Liang

CS230: Lecture 8 Word2Vec applications + Recurrent Neural Networks with Attention

word2vec Parameter Learning Explained

DISTRIBUTIONAL SEMANTICS

Bayesian Paragraph Vectors

Deep Learning for NLP Part 2

Semantics with Dense Vectors. Reference: D. Jurafsky and J. Martin, Speech and Language Processing

RaRE: Social Rank Regulated Large-scale Network Embedding

Sparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation.

ANLP Lecture 22 Lexical Semantics with Dense Vectors

arxiv: v2 [cs.cl] 1 Jan 2019

An overview of word2vec

Generative Clustering, Topic Modeling, & Bayesian Inference

Homework 3 COMS 4705 Fall 2017 Prof. Kathleen McKeown

Deep Learning Basics Lecture 7: Factor Analysis. Princeton University COS 495 Instructor: Yingyu Liang

Notes on Noise Contrastive Estimation (NCE)

text classification 3: neural networks

SUPPORT VECTOR MACHINE

Word2Vec Embedding. Embedding. Word Embedding 1.1 BEDORE. Word Embedding. 1.2 Embedding. Word Embedding. Embedding.

Machine Learning Linear Models

Neural Networks for NLP. COMP-599 Nov 30, 2016

Multivariate statistical methods and data mining in particle physics

Conditional Language Modeling. Chris Dyer

Natural Language Processing with Deep Learning CS224N/Ling284. Richard Socher Lecture 2: Word Vectors

Word Embeddings 2 - Class Discussions

arxiv: v3 [cs.cl] 30 Jan 2016

GloVe: Global Vectors for Word Representation 1

COMP 551 Applied Machine Learning Lecture 13: Dimension reduction and feature selection

Part-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287

CS Lecture 18. Topic Models and LDA

Joint Emotion Analysis via Multi-task Gaussian Processes

Natural Language Processing

NEURAL LANGUAGE MODELS

CME323 Distributed Algorithms and Optimization. GloVe on Spark. Alex Adamson SUNet ID: aadamson. June 6, 2016

Learning Features from Co-occurrences: A Theoretical Analysis

Support Vector Machines: Kernels

An Introduction to Statistical and Probabilistic Linear Models

Hidden Markov Models

Logic and machine learning review. CS 540 Yingyu Liang

Sequence Models. Ji Yang. Department of Computing Science, University of Alberta. February 14, 2018

Towards Universal Sentence Embeddings

Support Vector Machines. CAP 5610: Machine Learning Instructor: Guo-Jun QI

Lecture 5 Neural models for NLP

Methods and Criteria for Model Selection. CS57300 Data Mining Fall Instructor: Bruno Ribeiro

Regularization Introduction to Machine Learning. Matt Gormley Lecture 10 Feb. 19, 2018

MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October,

Data Mining. CS57300 Purdue University. Bruno Ribeiro. February 8, 2018

Dimension Reduction (PCA, ICA, CCA, FLD,

Machine Learning. B. Unsupervised Learning B.2 Dimensionality Reduction. Lars Schmidt-Thieme, Nicolas Schilling

Online Videos FERPA. Sign waiver or sit on the sides or in the back. Off camera question time before and after lecture. Questions?

Lecture 6: Neural Networks for Representing Word Meaning

UVA CS / Introduc8on to Machine Learning and Data Mining. Lecture 9: Classifica8on with Support Vector Machine (cont.

Recap from previous lecture

Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones

Machine Learning for Data Science (CS4786) Lecture 12

Topics in Natural Language Processing

ESANN'2003 proceedings - European Symposium on Artificial Neural Networks Bruges (Belgium), April 2003, d-side publi., ISBN X, pp.

Information Extraction from Text

CS 6375 Machine Learning

Artificial Neural Networks D B M G. Data Base and Data Mining Group of Politecnico di Torino. Elena Baralis. Politecnico di Torino

Neural Word Embeddings from Scratch

BANA 7046 Data Mining I Lecture 6. Other Data Mining Algorithms 1

Algorithms for Classification: The Basic Methods

Logistic Regression & Neural Networks

Unsupervised Learning: Projections

Prepositional Phrase Attachment over Word Embedding Products

Lecture 3 Classification, Logistic Regression

Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany

CS 540: Machine Learning Lecture 1: Introduction

Machine Learning for natural language processing

Learning with multiple models. Boosting.

Security Analytics. Topic 6: Perceptron and Support Vector Machine

A Randomized Approach for Crowdsourcing in the Presence of Multiple Views

Lecture 7: Kernels for Classification and Regression

Machine Learning Practice Page 2 of 2 10/28/13

Machine Learning for natural language processing

Deep Learning for NLP

Modeling Data with Linear Combinations of Basis Functions. Read Chapter 3 in the text by Bishop

Instructions for NLP Practical (Units of Assessment) SVM-based Sentiment Detection of Reviews (Part 2)

Support Vector Machines for Classification: A Statistical Portrait

Fast classification using sparsely active spiking networks. Hesham Mostafa Institute of neural computation, UCSD

Support Vector Machine & Its Applications

11/3/15. Deep Learning for NLP. Deep Learning and its Architectures. What is Deep Learning? Advantages of Deep Learning (Part 1)

arxiv: v3 [stat.ml] 1 Nov 2015

From perceptrons to word embeddings. Simon Šuster University of Groningen

Stat 406: Algorithms for classification and prediction. Lecture 1: Introduction. Kevin Murphy. Mon 7 January,

Data Mining & Machine Learning

a) b) (Natural Language Processing; NLP) (Deep Learning) Bag of words White House RGB [1] IBM

Microarray Data Analysis: Discovery

1 Inference in Dirichlet Mixture Models Introduction Problem Statement Dirichlet Process Mixtures... 3

Classification 2: Linear discriminant analysis (continued); logistic regression

Generative Learning. INFO-4604, Applied Machine Learning University of Colorado Boulder. November 29, 2018 Prof. Michael Paul

CS534 Machine Learning - Spring Final Exam

Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University

Hidden Markov Models

Machine Learning. Classification, Discriminative learning. Marc Toussaint University of Stuttgart Summer 2015

Collaborative Filtering. Radek Pelánek

Making Deep Learning Understandable for Analyzing Sequential Data about Gene Regulation

Midterm Review CS 6375: Machine Learning. Vibhav Gogate The University of Texas at Dallas

Learning the Semantic Correlation: An Alternative Way to Gain from Unlabeled Text

Transcription:

Data Mining & Machine Learning CS57300 Purdue University April 10, 2018 1

Predicting Sequences 2

But first, a detour to Noise Contrastive Estimation 3

} Machine learning methods are much better at classifying examples than generating new ones In classification tasks, we use the exact derivatives to find a solution that maximizes the likelihood In generative tasks, we can only compute an estimate of the derivative } Because we have better techniques to classify data than to generate data Can we make generative tasks look more like classification tasks? Bruno Ribeiro 4

Learn a Classifier to Distinguishing Noise from Data This is key idea behind noise contrastive estimation (NCE): make generative tasks look like classification tasks Pioneered by Hastie, Tibshirani, Friedman in The Elements of Statistical Learning in 2008, Section 14.2.4 Unsupervised Learning as Supervised Learning The idea is quite simple, consider the task of learning to distinguish noise from the data, i.e., search for a good classifier f(x ;W) Label y 1 =1 (= true data) Training Data Examples Classifier. x 1 = x 2 = Random Noise Examples Label y=0 (= false data) True example generator: : noise generator 5

Generation Task Using Classifier Now use the learned classifier (which distinguishes noise from the data) to generate new data but how? Naïve approach: Generate examples from the random noise whatever gets classified as real data will be our generated examples x 1 = Random Noise Examples x 2 = What is the problem with this naïve approach? In very high dimensions (e.g., images), true random noise will not generate any interesting examples x 1 x 2 x 3 6

Go to ipython notebook 7

Back to sequences 8

Sequences } In this lecture we will focus on word sequences (a.k.a. text) } The techniques we see are applicable to any type of sequence } A sequence is a succession of elements from a set (likely finite) } We will write a sequence of n elements as x 1,, x n } The temporal ordering is key to learning the sequence 9

2017 Bruno Ribeiro Word Sequences Rank-3 word embedding } Embed words w.r.t. their sentences Bring me a constant woman to her husband Forgetting, like a good man your late censure of his wife www.tensorflow.org } Under the Markov assumption P[bring, me, a, constant, woman, to, her, husband] = P[bring]P[me bring] P[a me] P[constant a] P[husband her] Go to ipython notebook

2017 Bruno Ribeiro Word Sequences & Embeddings Rank-3 word embedding } Embed words w.r.t. their sentences Bring me a constant woman to her husband Forgetting, like a good man your late censure of his wife husband man wife woman www.tensorflow.org } Initial idea by (Chen et al. 2012) was named Latent Markov Embedding, rediscovered by (Mikolov et al. 2013) named word2vec } Main difference: Application: (Mikolov et al. 2013) paid attention to the composition of latent vectors in sentences Otherwise, techniques equivalent Chen, S., Moore, J. L., Turnbull, D., & Joachims, T. (2012). Playlist prediction via metric embedding. In ACM SIGKDD. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In NIPS

2017 Bruno Ribeiro Word2vec Embeddings Rank-3 word embedding How the math works Bring me a constant woman to her husband Forgetting, like a good man your late censure of his wife husband man woman wife www.tensorflow.org } Conditional bag-of-words assumption (SKIPGRAM): All words are independent given the target word P[bring, constant, husband woman] = P[bring woman] P[constant woman] P[husband woman] P[forgetting, like, good, late, censure, wife man] = P[forgetting man] P[like man] P[good man] P[late man] P[censure man] P[wife man]

Word2vec Type Embeddings III 1 st coordinate of output words 2 nd coord. of output words exp X = 1 st coordinate of conditional word σ 1 u 1 woman v 1 husband + Embedding of woman as conditional word 2 nd coordinate of conditional word σ 2 u 2 woman v 2 husband The machine learning challenge is not summing over all words 2017 Bruno Ribeiro

<latexit sha1_base64="cipmw6+acjie9pp8tt1pqfh35g4=">aaahohic3vvnbxmxen0uskr4auhixw0vkrqabsiqschsjs6vemgrmryiu428xiexyq9xtrdtau3p4mfxozhyyfadkdrtbycbpdu+z8zz2g9gdhbzpo3rfqss3bu//kc68rd26pgtp89w154fa5koqrtecqloa6wpzxhtgmy4py0vxslg9cqyf8r8j+duasajizojqs/wmgidrrabu3/16ynycbwreymsly9nfq3tbvprfmi1hipko9t7ziycp9o4gcc0rcfp2rhqie8nom/hnvz6tocwuedpyf3xxjpk1pcvdw6ws6p+6qbbdpobfkgrbjtooq76a8vfvvcsrndiei617rxc2pgwk8mip2nnszsnmrnjie2f5yzwerzu+/yyz52iovhdnjakvsig3dplslhoprebrapsrvq6lzpe5oslzvdbtyyke0mjuiqajbwzibj6ojapsgyfambemdguiiosmdfqtbks2dpgsq7t2lwggcdussl8s3bekj93w8pgv6upg5wfcquj1smcu90utbnqjz8vp2amzdb/rb9z1wmos0fqrc+ifajh4zb19gbqgkgghqm1noadx0azk1rzsiq9ttcnhbfz8m2txq8jgg4pnczu+kjnrizyuap9xn2ftrdrw/0n2gvoa/gipasdc4jmmqpgesdrkzfzned+ysdmqz1ro9wodxbn4qhpinaafpnk9ujp+qvodooxu7qdubcv9g37smvpgpbqeae6/ikhp65bfb3jzp+ctjd1kwj717v0f9j0rllrytuudg4ht8zuysnvdpfemdpitdcjbemfr0rbw5d308b/nrayetu4k14gezhdwjj7hq1c4zloxx8ufsfxu9mcgh++3dzzlx+mfeels+e0njbz3tlxdp0dp+sq50dlvbjvev3dqo5w96uhrehspes8cozg9ctptzacjq==</latexit> <latexit sha1_base64="cipmw6+acjie9pp8tt1pqfh35g4=">aaahohic3vvnbxmxen0uskr4auhixw0vkrqabsiqschsjs6vemgrmryiu428xiexyq9xtrdtau3p4mfxozhyyfadkdrtbycbpdu+z8zz2g9gdhbzpo3rfqss3bu//kc68rd26pgtp89w154fa5koqrtecqloa6wpzxhtgmy4py0vxslg9cqyf8r8j+duasajizojqs/wmgidrrabu3/16ynycbwreymsly9nfq3tbvprfmi1hipko9t7ziycp9o4gcc0rcfp2rhqie8nom/hnvz6tocwuedpyf3xxjpk1pcvdw6ws6p+6qbbdpobfkgrbjtooq76a8vfvvcsrndiei617rxc2pgwk8mip2nnszsnmrnjie2f5yzwerzu+/yyz52iovhdnjakvsig3dplslhoprebrapsrvq6lzpe5oslzvdbtyyke0mjuiqajbwzibj6ojapsgyfambemdguiiosmdfqtbks2dpgsq7t2lwggcdussl8s3bekj93w8pgv6upg5wfcquj1smcu90utbnqjz8vp2amzdb/rb9z1wmos0fqrc+ifajh4zb19gbqgkgghqm1noadx0azk1rzsiq9ttcnhbfz8m2txq8jgg4pnczu+kjnrizyuap9xn2ftrdrw/0n2gvoa/gipasdc4jmmqpgesdrkzfzned+ysdmqz1ro9wodxbn4qhpinaafpnk9ujp+qvodooxu7qdubcv9g37smvpgpbqeae6/ikhp65bfb3jzp+ctjd1kwj717v0f9j0rllrytuudg4ht8zuysnvdpfemdpitdcjbemfr0rbw5d308b/nrayetu4k14gezhdwjj7hq1c4zloxx8ufsfxu9mcgh++3dzzlx+mfeels+e0njbz3tlxdp0dp+sq50dlvbjvev3dqo5w96uhrehspes8cozg9ctptzacjq==</latexit> <latexit sha1_base64="cipmw6+acjie9pp8tt1pqfh35g4=">aaahohic3vvnbxmxen0uskr4auhixw0vkrqabsiqschsjs6vemgrmryiu428xiexyq9xtrdtau3p4mfxozhyyfadkdrtbycbpdu+z8zz2g9gdhbzpo3rfqss3bu//kc68rd26pgtp89w154fa5koqrtecqloa6wpzxhtgmy4py0vxslg9cqyf8r8j+duasajizojqs/wmgidrrabu3/16ynycbwreymsly9nfq3tbvprfmi1hipko9t7ziycp9o4gcc0rcfp2rhqie8nom/hnvz6tocwuedpyf3xxjpk1pcvdw6ws6p+6qbbdpobfkgrbjtooq76a8vfvvcsrndiei617rxc2pgwk8mip2nnszsnmrnjie2f5yzwerzu+/yyz52iovhdnjakvsig3dplslhoprebrapsrvq6lzpe5oslzvdbtyyke0mjuiqajbwzibj6ojapsgyfambemdguiiosmdfqtbks2dpgsq7t2lwggcdussl8s3bekj93w8pgv6upg5wfcquj1smcu90utbnqjz8vp2amzdb/rb9z1wmos0fqrc+ifajh4zb19gbqgkgghqm1noadx0azk1rzsiq9ttcnhbfz8m2txq8jgg4pnczu+kjnrizyuap9xn2ftrdrw/0n2gvoa/gipasdc4jmmqpgesdrkzfzned+ysdmqz1ro9wodxbn4qhpinaafpnk9ujp+qvodooxu7qdubcv9g37smvpgpbqeae6/ikhp65bfb3jzp+ctjd1kwj717v0f9j0rllrytuudg4ht8zuysnvdpfemdpitdcjbemfr0rbw5d308b/nrayetu4k14gezhdwjj7hq1c4zloxx8ufsfxu9mcgh++3dzzlx+mfeels+e0njbz3tlxdp0dp+sq50dlvbjvev3dqo5w96uhrehspes8cozg9ctptzacjq==</latexit> <latexit sha1_base64="cipmw6+acjie9pp8tt1pqfh35g4=">aaahohic3vvnbxmxen0uskr4auhixw0vkrqabsiqschsjs6vemgrmryiu428xiexyq9xtrdtau3p4mfxozhyyfadkdrtbycbpdu+z8zz2g9gdhbzpo3rfqss3bu//kc68rd26pgtp89w154fa5koqrtecqloa6wpzxhtgmy4py0vxslg9cqyf8r8j+duasajizojqs/wmgidrrabu3/16ynycbwreymsly9nfq3tbvprfmi1hipko9t7ziycp9o4gcc0rcfp2rhqie8nom/hnvz6tocwuedpyf3xxjpk1pcvdw6ws6p+6qbbdpobfkgrbjtooq76a8vfvvcsrndiei617rxc2pgwk8mip2nnszsnmrnjie2f5yzwerzu+/yyz52iovhdnjakvsig3dplslhoprebrapsrvq6lzpe5oslzvdbtyyke0mjuiqajbwzibj6ojapsgyfambemdguiiosmdfqtbks2dpgsq7t2lwggcdussl8s3bekj93w8pgv6upg5wfcquj1smcu90utbnqjz8vp2amzdb/rb9z1wmos0fqrc+ifajh4zb19gbqgkgghqm1noadx0azk1rzsiq9ttcnhbfz8m2txq8jgg4pnczu+kjnrizyuap9xn2ftrdrw/0n2gvoa/gipasdc4jmmqpgesdrkzfzned+ysdmqz1ro9wodxbn4qhpinaafpnk9ujp+qvodooxu7qdubcv9g37smvpgpbqeae6/ikhp65bfb3jzp+ctjd1kwj717v0f9j0rllrytuudg4ht8zuysnvdpfemdpitdcjbemfr0rbw5d308b/nrayetu4k14gezhdwjj7hq1c4zloxx8ufsfxu9mcgh++3dzzlx+mfeels+e0njbz3tlxdp0dp+sq50dlvbjvev3dqo5w96uhrehspes8cozg9ctptzacjq==</latexit> <latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> <latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> <latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> <latexit sha1_base64="sm8sxrapgkh1g+uhjbid190wsxc=">aaahjhic3vxdahqxfe7vrnx9a/vkvektc6u0y8yikeih4i3ghrxtd+wmjzm5oxs2mqxjpu02dd6nxuqlecdeeonl+ajmdoz1t9v2tkedw3w553w5j98jszrxpo3nfv+4dpnkyupq0rxm9rs3b91exrmzq2wukoxqyaxaj4ggzllymcxw2m8uebfx2iugl0r/3ieozwt6zowycavjutznlbhnoli+z4pxijymapgoglqi3rjekojgec3reoob54ffgzvuj+2dlcwfqsxplia1lbote76xmdaszrjludsdxeng6jak0ispwaztikch9nhcqyfbzh/jvltusw0ew6djlgitryjykykygt7tk41n+xq56t8lluuz3ebkq0t9ngmjcakjjpkcavjiauivc+vioickuooum8lsrm2k5lpozmqqzahkshfaslikfnztdrue1kyschypokzwd0gguioypk6d8x45mklw2pfv9ym3gng3latk4yhkiugap7lbq747blgkkslaqlvkm6pzctsdmklpu8jh0+v4ow22wi0mcqjun7eirz0zygmi29117yhe2mbt7zfovqdrwhphw+70oeemzd7jfdpioayx0zekaz5lyvuzt/1uzhx3kl5bxiu7enzjzo+e7l/itkzhvdufzlm6umfuvtslosfd44xt8u8joxgdq+gfe/7ntd3rldpt//op/z80nzlubkqlgrnbizazkgkkki/si2ygnalmtk39s52oaot4yprzn6zfzgcglm2lji/lc1jyolk5updm+kcfhxmw2+34rsdvhq9tva4fjcv0hz1abesjp2glvutbaadr9b59qj/q58bhxpfg18a3kvtsqs25i2zg48cv7ehvaw==</latexit> <latexit sha1_base64="sm8sxrapgkh1g+uhjbid190wsxc=">aaahjhic3vxdahqxfe7vrnx9a/vkvektc6u0y8yikeih4i3ghrxtd+wmjzm5oxs2mqxjpu02dd6nxuqlecdeeonl+ajmdoz1t9v2tkedw3w553w5j98jszrxpo3nfv+4dpnkyupq0rxm9rs3b91exrmzq2wukoxqyaxaj4ggzllymcxw2m8uebfx2iugl0r/3ieozwt6zowycavjutznlbhnoli+z4pxijymapgoglqi3rjekojgec3reoob54ffgzvuj+2dlcwfqsxplia1lbote76xmdaszrjludsdxeng6jak0ispwaztikch9nhcqyfbzh/jvltusw0ew6djlgitryjykykygt7tk41n+xq56t8lluuz3ebkq0t9ngmjcakjjpkcavjiauivc+vioickuooum8lsrm2k5lpozmqqzahkshfaslikfnztdrue1kyschypokzwd0gguioypk6d8x45mklw2pfv9ym3gng3latk4yhkiugap7lbq747blgkkslaqlvkm6pzctsdmklpu8jh0+v4ow22wi0mcqjun7eirz0zygmi29117yhe2mbt7zfovqdrwhphw+70oeemzd7jfdpioayx0zekaz5lyvuzt/1uzhx3kl5bxiu7enzjzo+e7l/itkzhvdufzlm6umfuvtslosfd44xt8u8joxgdq+gfe/7ntd3rldpt//op/z80nzlubkqlgrnbizazkgkkki/si2ygnalmtk39s52oaot4yprzn6zfzgcglm2lji/lc1jyolk5updm+kcfhxmw2+34rsdvhq9tva4fjcv0hz1abesjp2glvutbaadr9b59qj/q58bhxpfg18a3kvtsqs25i2zg48cv7ehvaw==</latexit> <latexit sha1_base64="sm8sxrapgkh1g+uhjbid190wsxc=">aaahjhic3vxdahqxfe7vrnx9a/vkvektc6u0y8yikeih4i3ghrxtd+wmjzm5oxs2mqxjpu02dd6nxuqlecdeeonl+ajmdoz1t9v2tkedw3w553w5j98jszrxpo3nfv+4dpnkyupq0rxm9rs3b91exrmzq2wukoxqyaxaj4ggzllymcxw2m8uebfx2iugl0r/3ieozwt6zowycavjutznlbhnoli+z4pxijymapgoglqi3rjekojgec3reoob54ffgzvuj+2dlcwfqsxplia1lbote76xmdaszrjludsdxeng6jak0ispwaztikch9nhcqyfbzh/jvltusw0ew6djlgitryjykykygt7tk41n+xq56t8lluuz3ebkq0t9ngmjcakjjpkcavjiauivc+vioickuooum8lsrm2k5lpozmqqzahkshfaslikfnztdrue1kyschypokzwd0gguioypk6d8x45mklw2pfv9ym3gng3latk4yhkiugap7lbq747blgkkslaqlvkm6pzctsdmklpu8jh0+v4ow22wi0mcqjun7eirz0zygmi29117yhe2mbt7zfovqdrwhphw+70oeemzd7jfdpioayx0zekaz5lyvuzt/1uzhx3kl5bxiu7enzjzo+e7l/itkzhvdufzlm6umfuvtslosfd44xt8u8joxgdq+gfe/7ntd3rldpt//op/z80nzlubkqlgrnbizazkgkkki/si2ygnalmtk39s52oaot4yprzn6zfzgcglm2lji/lc1jyolk5updm+kcfhxmw2+34rsdvhq9tva4fjcv0hz1abesjp2glvutbaadr9b59qj/q58bhxpfg18a3kvtsqs25i2zg48cv7ehvaw==</latexit> <latexit sha1_base64="sm8sxrapgkh1g+uhjbid190wsxc=">aaahjhic3vxdahqxfe7vrnx9a/vkvektc6u0y8yikeih4i3ghrxtd+wmjzm5oxs2mqxjpu02dd6nxuqlecdeeonl+ajmdoz1t9v2tkedw3w553w5j98jszrxpo3nfv+4dpnkyupq0rxm9rs3b91exrmzq2wukoxqyaxaj4ggzllymcxw2m8uebfx2iugl0r/3ieozwt6zowycavjutznlbhnoli+z4pxijymapgoglqi3rjekojgec3reoob54ffgzvuj+2dlcwfqsxplia1lbote76xmdaszrjludsdxeng6jak0ispwaztikch9nhcqyfbzh/jvltusw0ew6djlgitryjykykygt7tk41n+xq56t8lluuz3ebkq0t9ngmjcakjjpkcavjiauivc+vioickuooum8lsrm2k5lpozmqqzahkshfaslikfnztdrue1kyschypokzwd0gguioypk6d8x45mklw2pfv9ym3gng3latk4yhkiugap7lbq747blgkkslaqlvkm6pzctsdmklpu8jh0+v4ow22wi0mcqjun7eirz0zygmi29117yhe2mbt7zfovqdrwhphw+70oeemzd7jfdpioayx0zekaz5lyvuzt/1uzhx3kl5bxiu7enzjzo+e7l/itkzhvdufzlm6umfuvtslosfd44xt8u8joxgdq+gfe/7ntd3rldpt//op/z80nzlubkqlgrnbizazkgkkki/si2ygnalmtk39s52oaot4yprzn6zfzgcglm2lji/lc1jyolk5updm+kcfhxmw2+34rsdvhq9tva4fjcv0hz1abesjp2glvutbaadr9b59qj/q58bhxpfg18a3kvtsqs25i2zg48cv7ehvaw==</latexit> <latexit sha1_base64="6jplcbwszurx8pqdw+oaf+hhnae=">aaahcxic3vvlbxmxehylbup4txbcxnxwkqk00w5uvcquqrixja4uit6k7bj5vu5irb27sr1tu2v/ix+av0gpcgd2ozd0dqmjlewznw8+z/ibkr2kgmvjon8wfu/cxvpurnxr3n/w8nhj1bunhzrjfguhnbgjog6izolh7mbwi9hxqhirgwbhqfsuwi9omni8it+bscp8suyxh3jkdlggq0ppcbeye5wprlu1m+me9nqmbzbqufmxhwy9mocniypgire4b5ypjmkh0rrulmyzcomonzodkbh8bly74b2sbjodp1z4quhwxiaq1/5gbenccxoasrybkojwfddjjw+jmpwklje9tlou0iimwd884amoiwtat2dl9hy3aa/xmfhwiw0uvbmks6twexlapcrmrc9jhfm6rj+z4rvf8jjnditplwiycwwsxaimq64ynwicbqgkq7myjoki1eab5riue5sketpvzmwqjgiqsarvkykpe/owntw6r12fkxigijpypsyp0x3jnqv1yvmkzmyaz8u/9dfoeibgkwik2slnpcrx+nj6h4ywu0ggrrm1nobku6p5owt6rzk+pkswntnxfdtstvqyhsmgpwkvoe2bmy993o5uos/w9jzuo7+nbmv0wxglpayjcijmmumurk8ylk3is7t8mk+5gj7jtttndxbn4mhaqnaw/drjzdht81ec6adfvxul80pd3wvqypv5fsgh8rj0/fnctqebfb3lzp+cttdnkwj716f0f9j0rllretwshg4hl0mziizrxuv6ys1ycmmntju+34mkrs5upwf+mraymtokaf6qilc4gbphvt4ihwfcvfwoxduoux0xevxpz3pvy/1grkdnaao1kyt20r56j/braalok/qofqcfyxenzw3c2khcfxdqzlm0txqvfgg5j3d0</latexit> <latexit sha1_base64="6jplcbwszurx8pqdw+oaf+hhnae=">aaahcxic3vvlbxmxehylbup4txbcxnxwkqk00w5uvcquqrixja4uit6k7bj5vu5irb27sr1tu2v/ix+av0gpcgd2ozd0dqmjlewznw8+z/ibkr2kgmvjon8wfu/cxvpurnxr3n/w8nhj1bunhzrjfguhnbgjog6izolh7mbwi9hxqhirgwbhqfsuwi9omni8it+bscp8suyxh3jkdlggq0ppcbeye5wprlu1m+me9nqmbzbqufmxhwy9mocniypgire4b5ypjmkh0rrulmyzcomonzodkbh8bly74b2sbjodp1z4quhwxiaq1/5gbenccxoasrybkojwfddjjw+jmpwklje9tlou0iimwd884amoiwtat2dl9hy3aa/xmfhwiw0uvbmks6twexlapcrmrc9jhfm6rj+z4rvf8jjnditplwiycwwsxaimq64ynwicbqgkq7myjoki1eab5riue5sketpvzmwqjgiqsarvkykpe/owntw6r12fkxigijpypsyp0x3jnqv1yvmkzmyaz8u/9dfoeibgkwik2slnpcrx+nj6h4ywu0ggrrm1nobku6p5owt6rzk+pkswntnxfdtstvqyhsmgpwkvoe2bmy993o5uos/w9jzuo7+nbmv0wxglpayjcijmmumurk8ylk3is7t8mk+5gj7jtttndxbn4mhaqnaw/drjzdht81ec6adfvxul80pd3wvqypv5fsgh8rj0/fnctqebfb3lzp+cttdnkwj716f0f9j0rllretwshg4hl0mziizrxuv6ys1ycmmntju+34mkrs5upwf+mraymtokaf6qilc4gbphvt4ihwfcvfwoxduoux0xevxpz3pvy/1grkdnaao1kyt20r56j/braalok/qofqcfyxenzw3c2khcfxdqzlm0txqvfgg5j3d0</latexit> <latexit sha1_base64="6jplcbwszurx8pqdw+oaf+hhnae=">aaahcxic3vvlbxmxehylbup4txbcxnxwkqk00w5uvcquqrixja4uit6k7bj5vu5irb27sr1tu2v/ix+av0gpcgd2ozd0dqmjlewznw8+z/ibkr2kgmvjon8wfu/cxvpurnxr3n/w8nhj1bunhzrjfguhnbgjog6izolh7mbwi9hxqhirgwbhqfsuwi9omni8it+bscp8suyxh3jkdlggq0ppcbeye5wprlu1m+me9nqmbzbqufmxhwy9mocniypgire4b5ypjmkh0rrulmyzcomonzodkbh8bly74b2sbjodp1z4quhwxiaq1/5gbenccxoasrybkojwfddjjw+jmpwklje9tlou0iimwd884amoiwtat2dl9hy3aa/xmfhwiw0uvbmks6twexlapcrmrc9jhfm6rj+z4rvf8jjnditplwiycwwsxaimq64ynwicbqgkq7myjoki1eab5riue5sketpvzmwqjgiqsarvkykpe/owntw6r12fkxigijpypsyp0x3jnqv1yvmkzmyaz8u/9dfoeibgkwik2slnpcrx+nj6h4ywu0ggrrm1nobku6p5owt6rzk+pkswntnxfdtstvqyhsmgpwkvoe2bmy993o5uos/w9jzuo7+nbmv0wxglpayjcijmmumurk8ylk3is7t8mk+5gj7jtttndxbn4mhaqnaw/drjzdht81ec6adfvxul80pd3wvqypv5fsgh8rj0/fnctqebfb3lzp+cttdnkwj716f0f9j0rllretwshg4hl0mziizrxuv6ys1ycmmntju+34mkrs5upwf+mraymtokaf6qilc4gbphvt4ihwfcvfwoxduoux0xevxpz3pvy/1grkdnaao1kyt20r56j/braalok/qofqcfyxenzw3c2khcfxdqzlm0txqvfgg5j3d0</latexit> <latexit sha1_base64="6jplcbwszurx8pqdw+oaf+hhnae=">aaahcxic3vvlbxmxehylbup4txbcxnxwkqk00w5uvcquqrixja4uit6k7bj5vu5irb27sr1tu2v/ix+av0gpcgd2ozd0dqmjlewznw8+z/ibkr2kgmvjon8wfu/cxvpurnxr3n/w8nhj1bunhzrjfguhnbgjog6izolh7mbwi9hxqhirgwbhqfsuwi9omni8it+bscp8suyxh3jkdlggq0ppcbeye5wprlu1m+me9nqmbzbqufmxhwy9mocniypgire4b5ypjmkh0rrulmyzcomonzodkbh8bly74b2sbjodp1z4quhwxiaq1/5gbenccxoasrybkojwfddjjw+jmpwklje9tlou0iimwd884amoiwtat2dl9hy3aa/xmfhwiw0uvbmks6twexlapcrmrc9jhfm6rj+z4rvf8jjnditplwiycwwsxaimq64ynwicbqgkq7myjoki1eab5riue5sketpvzmwqjgiqsarvkykpe/owntw6r12fkxigijpypsyp0x3jnqv1yvmkzmyaz8u/9dfoeibgkwik2slnpcrx+nj6h4ywu0ggrrm1nobku6p5owt6rzk+pkswntnxfdtstvqyhsmgpwkvoe2bmy993o5uos/w9jzuo7+nbmv0wxglpayjcijmmumurk8ylk3is7t8mk+5gj7jtttndxbn4mhaqnaw/drjzdht81ec6adfvxul80pd3wvqypv5fsgh8rj0/fnctqebfb3lzp+cttdnkwj716f0f9j0rllretwshg4hl0mziizrxuv6ys1ycmmntju+34mkrs5upwf+mraymtokaf6qilc4gbphvt4ihwfcvfwoxduoux0xevxpz3pvy/1grkdnaao1kyt20r56j/braalok/qofqcfyxenzw3c2khcfxdqzlm0txqvfgg5j3d0</latexit> <latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> <latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> <latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> Matrix Factorization as a Neural Network (v2) But neural networks can have isomorphic network representations KX Matrix factorization: X U V T = Neural network representation: k=1 ku k v T k 4X X 1,7 = ku 1,k v 7,k k=1 Element-wise product Linear activation Linear activation u 1 v 1 U<latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> V<latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 14 One-hot encoding of user One-hot encoding of product

<latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> <latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> <latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> <latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> <latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> <latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> Word2vec Embedding as Neural Network Neural network representation of word2vec: Linear activation Linear activation u 1 v 1 U<latexit sha1_base64="20r+kfvojzrfz60s1mdkdrhzzoq=">aaahf3ic3vxnbhmxehyldsx8tiujly1vpidaabobqekvknfb4kcrsfspu1re7ysxyq9xtrdtau17wbfehbviyphn4awyza5c07s9gqswvvt5zj7p+bvljllbjfx9hwul164v1w4s36zfun3n7srq2r09ozlnomuuupogogyet6brurvwkgqgmhkwh41efv79i9cgq+sdhacqsjpiej8zath03gwtjzygoo/mh6sbfsufdg8etcuwqaqxe7i29doifcskjjyjakyv7ac2dfrbzgtk9sazkfi2ogpoxuc8nqmvyej3mkmbew30x15fafws602sz0mosmpgmsjise3qnpcvxot8vcz2n4eoj2lmiwflon4mpku8qgyv5hqyfwmelgmo5xpssdvlfswayvksbzusjq/pzjb0bfopgtpgewzi1u0sh51wpgikhmmqx84maqqmjblhqm5kvwksi6kedtrakrvjb/2pvx4jxaqdejhmskqaxi9d8lqpny8ipcidcwxwnlrdt6eefel6hleb236rhbp6o9hwib4a7ibw9lhnhzwjvwzn03/kbw15tf836jsgg+af8jw8mcjilnnnqmxhiopnyjpbycvnxgofvwa7k1rknonhe1wgiwirjjdht/dfcqbtskzqcuzcxz0l6srr+zyq2hijov4piaeusxw9ys//nlyxnvlu9q+f0v9j0zosuodqger4owqqbu2t0svfesztuhdjrxgvf7ytjy2exe1d/3lazidgadg8sim4uicvceobhuiz0t7/kmydvu6rjt1++2rj5031ycytb+qhazi2euz2ycuys7qeeu0+ke/kc+1j7uvta+1bgbq4uhhuk5lr+/4lktzp9a==</latexit> V<latexit sha1_base64="rjx2viy0zv0wbbosgmdnfnknehm=">aaahg3ic3vxnbhmxehyldsx8pxdk4ljfcqindimqskhsjs5ihcgs/zgyq8rrnsrw7pxk9qznrx0tomklcencofacvadeh4wmaxsdcsyt9vpmfj7xn5ydpzxp43k/lpavxv9p3fi92bx1+87de621+/tazorchpvcqsoiaoasgt3ddifdvaereyedapyq8b9mqgkmk/dmmkioydbha0ajcaajvssg5si24hnked8/am14xa8cebh4ndha9dg9wlv5gcsszgisqznruu97qqktuyzrdnkzydskhi7jeprxhku6iqj0ae/kxdluo3+mb1k5lzg4tj4lwsk0norirqpirvq8rzbe5otnzvaitcxjmwmjrrinmo6nxiuuogykqoftbwhvzjwl6ygoqo0tbc5lsbarkuu8ozdbkdeokuvokuko8hm3nwx8wpskyfmkijpapsip6k5gmjp1yv1ymdymajxuaeluy9/6m2894m4pf5tamzvckcr+yom3a9f9kfld3npau8pto9kpnimisv9twmhb6/qhbbbbbqzxenxuykwo+2bekhb3epvey7y1htveb9crqm+bl46h3afxgsyya8b5dnfgnotpickat5lyfcydv5caxz0trycuwh0i6ysxr8/2x3fm07cq6grmql29c+rkm/mckk7xwun4p4scus5v9io9/3paxnrknbz//zt+t5qekdqgra0fc7ddifnqxehvxkthziw4e8xow/vno1hrymnvnoc/t4vmcawpmhdjhhcxsorbzxoh7pnwzz8ki2c/1/vdj9893dh5wz8yq+gheoq6yefp0q56jxbrhqjogj6gt+hz42pjs+nr41svurxucx6gudh4/gtsildh</latexit> 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 15 One-hot encoding of output word One-hot encoding of input word

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback