Applications of Deep Learning

Similar documents
Google s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Natural Language Processing (CSEP 517): Machine Translation (Continued), Summarization, & Finale

Factored Neural Machine Translation Architectures

Neural Hidden Markov Model for Machine Translation

Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

arxiv: v1 [cs.cl] 22 Jun 2015

arxiv: v1 [cs.cl] 21 May 2017

Learning to translate with neural networks. Michael Auli

CS885 Reinforcement Learning Lecture 7a: May 23, 2018

Multi-Source Neural Translation

Introduction of Reinforcement Learning

Word Attention for Sequence to Sequence Text Understanding

REINFORCEMENT LEARNING

CSC321 Lecture 15: Recurrent Neural Networks

CS230: Lecture 9 Deep Reinforcement Learning

Coverage Embedding Models for Neural Machine Translation

Multi-Source Neural Translation

arxiv: v2 [cs.cl] 1 Jan 2019

Reinforcement Learning

Utilizing Portion of Patent Families with No Parallel Sentences Extracted in Estimating Translation of Technical Terms

Jakub Hajic Artificial Intelligence Seminar I

Deep Reinforcement Learning. Scratching the surface

A Little History of Machine Learning

What Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1

Edinburgh Research Explorer

Deep Learning and Information Theory

Deep Reinforcement Learning SISL. Jeremy Morton (jmorton2) November 7, Stanford Intelligent Systems Laboratory

Recurrent Neural Networks (Part - 2) Sumit Chopra Facebook

CS230: Lecture 8 Word2Vec applications + Recurrent Neural Networks with Attention

Analysis of techniques for coarse-to-fine decoding in neural machine translation

WaveNet: A Generative Model for Raw Audio

Fast and Scalable Decoding with Language Model Look-Ahead for Phrase-based Statistical Machine Translation

Machine Learning for Physicists Lecture 1

Deep Reinforcement Learning. STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 19, 2017

Deep Learning Sequence to Sequence models: Attention Models. 17 March 2018

Multi-Task Word Alignment Triangulation for Low-Resource Languages

Improved Learning through Augmenting the Loss

CSC321 Lecture 10 Training RNNs

Self-Attention with Relative Position Representations

Phrase Table Pruning via Submodular Function Maximization

Algorithms for NLP. Machine Translation II. Taylor Berg-Kirkpatrick CMU Slides: Dan Klein UC Berkeley

Out of GIZA Efficient Word Alignment Models for SMT

Deep Learning for NLP

CSC321 Lecture 16: ResNets and Attention

Feature Design. Feature Design. Feature Design. & Deep Learning

statistical machine translation

TTIC 31230, Fundamentals of Deep Learning David McAllester, April Sequence to Sequence Models and Attention

An overview of word2vec

Deep Neural Machine Translation with Linear Associative Unit

arxiv: v1 [cs.cl] 22 Jun 2017

Conditional Language modeling with attention

Latent Variable Models in NLP

Payments System Design Using Reinforcement Learning: A Progress Report

Tuning as Linear Regression

Conquering the Complexity of Time: Machine Learning for Big Time Series Data

David Silver, Google DeepMind

Anticipating Visual Representations from Unlabeled Data. Carl Vondrick, Hamed Pirsiavash, Antonio Torralba

Approximate Q-Learning. Dan Weld / University of Washington

UAlacant word-level machine translation quality estimation system at WMT 2015

Lecture 1: March 7, 2018

Identifying QCD transition using Deep Learning

CSC321 Lecture 22: Q-Learning

Deep Reinforcement Learning

What s so Hard about Natural Language Understanding?

Machine Translation. 10: Advanced Neural Machine Translation Architectures. Rico Sennrich. University of Edinburgh. R. Sennrich MT / 26

A Video from Google DeepMind.

From perceptrons to word embeddings. Simon Šuster University of Groningen

ACS Introduction to NLP Lecture 2: Part of Speech (POS) Tagging

Improving Lexical Choice in Neural Machine Translation. Toan Q. Nguyen & David Chiang

Quantum Artificial Intelligence and Machine Learning: The Path to Enterprise Deployments. Randall Correll. +1 (703) Palo Alto, CA

Backpropagation Through

Human-level control through deep reinforcement. Liia Butler

Chapter 8: Generalization and Function Approximation

Deep Learning for Natural Language Processing

The Noisy Channel Model and Markov Models

A phrase-based hidden Markov model approach to machine translation

Maja Popović Humboldt University of Berlin Berlin, Germany 2 CHRF and WORDF scores

NEAL: A Neurally Enhanced Approach to Linking Citation and Reference

a) b) (Natural Language Processing; NLP) (Deep Learning) Bag of words White House RGB [1] IBM

TTIC 31230, Fundamentals of Deep Learning, Winter David McAllester. The Fundamental Equations of Deep Learning

Introduction to Neural Networks

Be able to define the following terms and answer basic questions about them:

Overview (Fall 2007) Machine Translation Part III. Roadmap for the Next Few Lectures. Phrase-Based Models. Learning phrases from alignments

If Mathematical Proof is a Game, What are the States and Moves? David McAllester

Information Extraction from Text

Generating Sequences with Recurrent Neural Networks

On Some Mathematical Results of Neural Networks

The Geometry of Statistical Machine Translation

INF5820/INF9820 LANGUAGE TECHNOLOGICAL APPLICATIONS. Jan Tore Lønning, Lecture 3, 7 Sep., 2016

Making Deep Learning Understandable for Analyzing Sequential Data about Gene Regulation

Sequence Modeling with Neural Networks

Investigating Connectivity and Consistency Criteria for Phrase Pair Extraction in Statistical Machine Translation

Today s Lecture. Dropout

Better Conditional Language Modeling. Chris Dyer

Reinforcement Learning as Classification Leveraging Modern Classifiers

Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction

Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks

Natural Language Processing (CSEP 517): Machine Translation

Advances in Neural Machine Translation

Natural Language Processing with Deep Learning CS224N/Ling284

Transcription:

Applications of Deep Learning Alpha Go Google Translate Data Center Optimisation Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid November 23, 2016 Template by Philipp Arndt

Applications of Deep Learning Introduction November 23, 2016 FFR141 - Complex Systems Seminar Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid Applications of Deep Learning 2

Applications of Deep Learning Introduction AlphaGo Google s Neural Machine Translation (GNMT) Deep learning to control data centre cooling Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 3

Solving boardgames November 23, 2016 FFR141 - Complex Systems Seminar Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid Applications of Deep Learning 4

Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 5

Deep neural network Supervised learning Reinforcement learning from games of self-play Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 6

Deep neural network Supervised learning Reinforcement learning from games of self-play Monte Carlo simulation Policy network Value network Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 6

Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 7

Policy network: classifies promising positions Value Network: calculate estimates of winning Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 7

Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 8

99.8% Winratio against other go programs Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 8

Applications of Deep Learning Google s Neural Machine Translation (GNMT) Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 9

Google s Neural Machine Translation (GNMT) Introduction Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 10

Google s Neural Machine Translation (GNMT) Introduction There are flaws, BUT... September 27, 2016 GNMT announced Error reduction by 60% Bridging the Gap between Human and Machine Translation How? Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 11

Google s Neural Machine Translation (GNMT) Introduction Overview Models used so far GNMT Model Experiments and Results Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 12

Google s Neural Machine Translation (GNMT) Models used so far Overview Models used so far GNMT Model Experiments and Results Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 13

Google s Neural Machine Translation (GNMT) Models used so far Phrase-Based Machine Translation (PBMT) Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 14

Google s Neural Machine Translation (GNMT) Models used so far Phrase-Based Machine Translation (PBMT) Probability tables Linguistic properties Neural Machine Translation (NMT) Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 15

Google s Neural Machine Translation (GNMT) Models used so far Flaws of NMT Accuracy Speed / Computation Robustness Coverage Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 16

Google s Neural Machine Translation (GNMT) GNMT Model Overview Models used so far GNMT Model Experiments and Results Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 17

Google s Neural Machine Translation (GNMT) GNMT Model How does GNMT handle these problems? Speed / Computation Robustness Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 18

Google s Neural Machine Translation (GNMT) GNMT Model How does GNMT handle these problems? Speed / Computation Robustness Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 19

Google s Neural Machine Translation (GNMT) GNMT Model Architecture Parallelism Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 20

Google s Neural Machine Translation (GNMT) GNMT Model How does GNMT handle these problems? Speed / Computation Robustness Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 21

Google s Neural Machine Translation (GNMT) GNMT Model Segmentation: WordPiece Model (WPM) Abwasserbehandlungsanlage Abwasser behandlungs anlage sewage water treatment plant Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 22

Google s Neural Machine Translation (GNMT) Experiments and Results Overview Models used so far GNMT Model Experiments and Results Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 23

Google s Neural Machine Translation (GNMT) Experiments and Results Tests on Benchmark Sentence Pairs Workshop on Machine Translation (WMT) data set BiLingual Evaluation Understudy (BLEU) metric Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 24

Google s Neural Machine Translation (GNMT) Experiments and Results Tests on Benchmark Sentence Pairs Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 25

Google s Neural Machine Translation (GNMT) Experiments and Results Human Evaluation Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 26

Google s Neural Machine Translation (GNMT) Experiments and Results Bridging the Gap between Human and Machine Translation Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 27

Deep learning to optimize cooling of data centres Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 28

Deep learning to control data centre cooling Introduction Facts and definitions 1 Google search = keep a lightbulb going for 25s 40.000 searches/s PUE = Power Usage Efficiency Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 29

Deep learning to control data centre cooling Predicting PUE Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 30

Deep learning to control data centre cooling Predicting PUE 99.6% accuracy Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 31

Deep learning to control data centre cooling Predicting PUE Difficult to optimize efficiency Non-linear interactions between machines and environment Systems ability to adapt to operational changes Each facility has unique architechture Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 32

Deep learning to control data centre cooling Controling the data centre Results: 40% reduction in energy usage for cooling Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 33

Conclusions Beating human intuition in board games Solving Language translation tasks Outperforming human engineering abilities Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 34

Thank you for listening! Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 35

Discussion Questions What are the limitations of deep learning? Are there tasks for which the technique cannot be applied? Are there areas where deep learning should be used, but isn t? Who is responsible when a machine makes a critical error? For example: Who is responsible if an AI or machine causes a train to derail or fails to properly diagnose a patient? Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 36

Applications of Deep Learning References Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi. Google s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. (2016) D. Bahdanau, K. H. Cho, Y. Bengio. Neral Machine Translation by Jointly Learning to Align and Translate. (2015) P. Koehn, F. J. Och, D. Marcu. Statistical Phrase-Based Translation. (2003) R. Sennrich, B. Haddow, A. Birch. Neural Machine Translation of Rare Words with Subword Units. (2016) S. Jean, K. Cho, R. Memisevic, Y. Bengio. On Using Very Large Target Vocabulary for Neural Machine Translation (2015) http://www.slideshare.net/nlab_utokyo/machine-translation-introduction http://slideplayer.com/slide/9202214/ https://research.googleblog.com/2016/09/a-neural-network-for-machine.html Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 37

Applications of Deep Learning References https://www.bloomberg.com/news/articles/2016-07-19/ google-cuts-its-giant-electricity-bill-with-deepmind-powered-ai http://www.theverge.com/2016/7/21/12246258/google-deepmind-ai-data-center-cooling https://googleblog.blogspot.se/2014/05/better-data-centers-through-machine.html https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/ https: //docs.google.com/a/google.com/viewer?url=www.google.com/about/datacenters/efficiency/ internal/assets/machine-learning-applicationsfor-datacenter-optimization-finalv2.pdf https://googleblog.blogspot.se/2009/01/powering-google-search.html http://www.internetlivestats.com/google-search-statistics/ Robin Sigurdson, Yvonne Krumbeck, Henrik Arnelid 38

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback