GANs, GANs everywhere
|
|
|
- Irene French
- 6 years ago
- Views:
Transcription
1 GANs, GANs everywhere particularly, in High Energy Physics Maxim Borisyak Yandex, NRU Higher School of Economics
2 Generative
3 Generative models Given samples of a random variable X find X such as: P X P X Approaches: probability density function; sampling procedure. Maxim Borisyak 3
4 Generator vs probability function Learning directly a probability density function P X is hard: normalization is the main issue; sampling might be computationally costly (usually, long MC); P X is usually heavily restricted (e.g. RBM). Learning a generator is easier: X = G(Z) where: G - a parametrized deterministic function; Z - predetermined and easy to sample. Maxim Borisyak 4
5 Generator Generator might be any program: a neural network (traditionally); a Monte-Carlo simulation. Maxim Borisyak 5
6 Generative Adversarial
7 Adversarial training Generator is trained to maximize goodness of produced samples. GAN defines goodness of a generator via a classifier D: learns to discriminate X against X ; if quality is close to a random guess: X is similar to X; if quality is high: G should be improved. Maxim Borisyak 7
8 Discriminator usually called adversary or critic; traditionally, also a neural network; discriminator defines goodness of generated samples: rich set of methods for classification; easy to identify important properties of good generator and use inside discriminator; produces interpretable quality metric. Maxim Borisyak 8
9 Generative Adversarial Networks
10 GANs Generative Adversarial Networks: introduced recently; a lot of promising results and development; adoptable: conditional GANs; GANs as auxiliary loss; Maxim Borisyak 10
11 GANs in the wild Source: 11
12 GANs in the wild Source: 12
13 Summary Generative Adversarial Networks: generator training; performance of trained classifier as quality measure; easily adoptable for problems beyond pure generation. Maxim Borisyak 13
14 GANs in HEP
15 GANs IN High Energy Physics CaloGAN: fast simulation; PartyGAN: learning (almost) black-box physical process; MC tuning: selecting best parameters for simulation to match data. Maxim Borisyak 15
16 CaloGAN Samples EM calorimeter showers. Source: 16
17 CaloGAN direct sampling from the response space; no intermediate steps involved; significant speedup. Source: 17
18 PartyGAN Simulation of cosmic rays interaction with a CMOS sensor: partially simulated by GEANT, without readout; generator transforms GEANT output into realistic images: generator is to learn readout-process; matching to the real data. Maxim Borisyak 18
19 PartyGAN
20 PartyGAN: toy readout model Maxim Borisyak 20
21 PartyGAN: quality test Trigger is training on D 1, evaluated on D 2. Maxim Borisyak 21
22 MC tuning A MC simulation as generator: parameter tuning to match real data; non-differentiable generator; gradient-free optimization methods. Maxim Borisyak 22
23 Summary
24 Summary Generative Adversarial Networks is powerful tool for modeling distributions; easily adoptable for various settings, even beyond pure generative tasks; GANs in HEP: fast simulation: e.g. CaloGAN: possible applications: RICH-GAN, VeLo-GAN; learning behavior of an almost black-box system: e.g. PartyGAN; tuning Monte-Carlo parameters: Adversarial Optimization. Maxim Borisyak 24
25 References, GAN Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y., Generative adversarial nets. In Advances in neural information processing systems (pp ). Isola, P., Zhu, J.Y., Zhou, T. and Efros, A.A., Image-to-image translation with conditional adversarial networks. arxiv preprint arxiv: Maxim Borisyak 25
26 References, GAN in HEP Paganini, M., de Oliveira, L. and Nachman, B., CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks. arxiv preprint arxiv: PartyGAN at ACAT Louppe, G. and Cranmer, K., Adversarial Variational Optimization of Non-Differentiable Simulators. arxiv preprint arxiv: Maxim Borisyak 26
27 Backup
28 PartyGAN details GEANT output as generator's input; real data as reference data; event rate adjustment; Cycle-GAN to enforce reversibility of the generator; restricted (3 3) receptive field of the generator. Maxim Borisyak 28
29 Party-GAN details generator discriminator Real observations GEANT simulation discriminator reverse tranformation Maxim Borisyak 29
30 Different flavors of GANs, part I GAN as auxiliary loss: restricts e.g. regression to realistic results; Conditional GAN: replaces Z N m (0, 1) by another dataset; Cycle-GAN: a conditional GAN that learns reversible generator; staked GANs, ensembles of GANs, Maxim Borisyak 30
31 Different flavors of GANs, part II Classical GAN: cross-entropy discriminator: proxy to KL distance; vanishing/exploding gradients. EB-GAN: energy-based discriminator: proxy to total variation distance; rarely vanishing, smooth gradients; Wasserstein-GAN: critic, a L 1 function: proxy for earth-moving distance; never vanishing, smooth gradients; difficulties training: keeping critic in L 1. Maxim Borisyak 31
32 PartyGAN
Singing Voice Separation using Generative Adversarial Networks
Singing Voice Separation using Generative Adversarial Networks Hyeong-seok Choi, Kyogu Lee Music and Audio Research Group Graduate School of Convergence Science and Technology Seoul National University
Deep Generative Models. (Unsupervised Learning)
Deep Generative Models (Unsupervised Learning) CEng 783 Deep Learning Fall 2017 Emre Akbaş Reminders Next week: project progress demos in class Describe your problem/goal What you have done so far What
Top Tagging with Lorentz Boost Networks and Simulation of Electromagnetic Showers with a Wasserstein GAN
Top Tagging with Lorentz Boost Networks and Simulation of Electromagnetic Showers with a Wasserstein GAN Y. Rath, M. Erdmann, B. Fischer, L. Geiger, E. Geiser, J.Glombitza, D. Noll, T. Quast, M. Rieger,
Notes on Adversarial Examples
Notes on Adversarial Examples David Meyer dmm@{1-4-5.net,uoregon.edu,...} March 14, 2017 1 Introduction The surprising discovery of adversarial examples by Szegedy et al. [6] has led to new ways of thinking
Some theoretical properties of GANs. Gérard Biau Toulouse, September 2018
Some theoretical properties of GANs Gérard Biau Toulouse, September 2018 Coauthors Benoît Cadre (ENS Rennes) Maxime Sangnier (Sorbonne University) Ugo Tanielian (Sorbonne University & Criteo) 1 video Source:
Learning Particle Physics by Example:
Learning Particle Physics by Example: Accelerating Science with Generative Adversarial Networks arxiv:1701.05927, arxiv:1705.02355 @lukede0 @lukedeo lukedeo@manifold.ai https://ldo.io Luke de Oliveira
arxiv: v1 [cs.lg] 20 Apr 2017
![arxiv: v1 [cs.lg] 20 Apr 2017](/thumbs/78/78348074.jpg "arxiv: v1 [cs.lg] 20 Apr 2017")
Softmax GAN Min Lin Qihoo 360 Technology co. ltd Beijing, China, 0087 mavenlin@gmail.com arxiv:704.069v [cs.lg] 0 Apr 07 Abstract Softmax GAN is a novel variant of Generative Adversarial Network (GAN).
Importance Reweighting Using Adversarial-Collaborative Training
Importance Reweighting Using Adversarial-Collaborative Training Yifan Wu yw4@andrew.cmu.edu Tianshu Ren tren@andrew.cmu.edu Lidan Mu lmu@andrew.cmu.edu Abstract We consider the problem of reweighting a
GENERATIVE ADVERSARIAL LEARNING
GENERATIVE ADVERSARIAL LEARNING OF MARKOV CHAINS Jiaming Song, Shengjia Zhao & Stefano Ermon Computer Science Department Stanford University {tsong,zhaosj12,ermon}@cs.stanford.edu ABSTRACT We investigate
Generative Adversarial Networks
Generative Adversarial Networks SIBGRAPI 2017 Tutorial Everything you wanted to know about Deep Learning for Computer Vision but were afraid to ask Presentation content inspired by Ian Goodfellow s tutorial
A QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS
A QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS Dan Hendrycks University of Chicago dan@ttic.edu Steven Basart University of Chicago xksteven@uchicago.edu ABSTRACT We introduce a
ON ADVERSARIAL TRAINING AND LOSS FUNCTIONS FOR SPEECH ENHANCEMENT. Ashutosh Pandey 1 and Deliang Wang 1,2. {pandey.99, wang.5664,
ON ADVERSARIAL TRAINING AND LOSS FUNCTIONS FOR SPEECH ENHANCEMENT Ashutosh Pandey and Deliang Wang,2 Department of Computer Science and Engineering, The Ohio State University, USA 2 Center for Cognitive
Negative Momentum for Improved Game Dynamics
Negative Momentum for Improved Game Dynamics Gauthier Gidel Reyhane Askari Hemmat Mohammad Pezeshki Gabriel Huang Rémi Lepriol Simon Lacoste-Julien Ioannis Mitliagkas Mila & DIRO, Université de Montréal
Training Generative Adversarial Networks Via Turing Test
raining enerative Adversarial Networks Via uring est Jianlin Su School of Mathematics Sun Yat-sen University uangdong, China bojone@spaces.ac.cn Abstract In this article, we introduce a new mode for training
Deep generative models for fast shower simulation in ATLAS
Deep generative models for fast shower simulation in ATLAS Dalila Salamani University of Geneva dalila.salamani@cern.ch On behalf of the ATLAS collaboration 14th e-science IEEE International Conference
arxiv: v1 [eess.iv] 28 May 2018
![arxiv: v1 [eess.iv] 28 May 2018](/thumbs/92/108638989.jpg "arxiv: v1 [eess.iv] 28 May 2018")
Versatile Auxiliary Regressor with Generative Adversarial network (VAR+GAN) arxiv:1805.10864v1 [eess.iv] 28 May 2018 Abstract Shabab Bazrafkan, Peter Corcoran National University of Ireland Galway Being
Nishant Gurnani. GAN Reading Group. April 14th, / 107
Nishant Gurnani GAN Reading Group April 14th, 2017 1 / 107 Why are these Papers Important? 2 / 107 Why are these Papers Important? Recently a large number of GAN frameworks have been proposed - BGAN, LSGAN,
Adversarial Sequential Monte Carlo
Adversarial Sequential Monte Carlo Kira Kempinska Department of Security and Crime Science University College London London, WC1E 6BT kira.kowalska.13@ucl.ac.uk John Shawe-Taylor Department of Computer
Quark/Gluon Discrimination with Jet-Images and Deep Learning
Quark/Gluon Discrimination with Jet-Images and Deep Learning BOOST 2017 Patrick T. Komiske Center for Theoretical Physics, Massachusetts Institute of Technology Based on arxiv:1612.01551 PTK, Eric M. Metodiev,
GAN Applications in High Energy Particle Physics
75.2355 7.5927 GAN Applications in High Energy Particle Physics Benjamin Nachman Lawrence Berkeley National Laboratory with collaborators Michela Paganini and Luke de Oliveira Outline: DNN with HEP images
arxiv: v1 [stat.ml] 25 May 2017
![arxiv: v1 [stat.ml] 25 May 2017](/thumbs/80/81897727.jpg "arxiv: v1 [stat.ml] 25 May 2017")
Lat-Net: Compressing Lattice Boltzmann Flow Simulations using Deep Neural Networks arxiv:1705.09036v1 [stat.ml] 25 May 2017 Oliver Hennigh Mexico loliverhennigh101@gmail.com Abstract Computational Fluid
Wasserstein GAN. Juho Lee. Jan 23, 2017
Wasserstein GAN Juho Lee Jan 23, 2017 Wasserstein GAN (WGAN) Arxiv submission Martin Arjovsky, Soumith Chintala, and Léon Bottou A new GAN model minimizing the Earth-Mover s distance (Wasserstein-1 distance)
Calorimetry with Deep Learning: Particle Classification, Energy Regression, and Simulation for High-Energy Physics
Calorimetry with Deep Learning: Particle Classification, Energy Regression, and Simulation for High-Energy Physics Federico Carminati, Gulrukh Khattak, Maurizio Pierini CERN Amir Farbin Univ. of Texas
Neural networks and optimization
Neural networks and optimization Nicolas Le Roux Criteo 18/05/15 Nicolas Le Roux (Criteo) Neural networks and optimization 18/05/15 1 / 85 1 Introduction 2 Deep networks 3 Optimization 4 Convolutional
Improving Visual Semantic Embedding By Adversarial Contrastive Estimation
Improving Visual Semantic Embedding By Adversarial Contrastive Estimation Huan Ling Department of Computer Science University of Toronto huan.ling@mail.utoronto.ca Avishek Bose Department of Electrical
arxiv: v1 [cs.lg] 6 Dec 2018
![arxiv: v1 [cs.lg] 6 Dec 2018](/thumbs/91/107120580.jpg "arxiv: v1 [cs.lg] 6 Dec 2018")
Embedding-reparameterization procedure for manifold-valued latent variables in generative models arxiv:1812.02769v1 [cs.lg] 6 Dec 2018 Eugene Golikov Neural Networks and Deep Learning Lab Moscow Institute
Experiments on the Consciousness Prior
Yoshua Bengio and William Fedus UNIVERSITÉ DE MONTRÉAL, MILA Abstract Experiments are proposed to explore a novel prior for representation learning, which can be combined with other priors in order to
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Emily Denton 1, Soumith Chintala 2, Arthur Szlam 2, Rob Fergus 2 1 New York University 2 Facebook AI Research Denotes equal
Wafer Pattern Recognition Using Tucker Decomposition
Wafer Pattern Recognition Using Tucker Decomposition Ahmed Wahba, Li-C. Wang, Zheng Zhang UC Santa Barbara Nik Sumikawa NXP Semiconductors Abstract In production test data analytics, it is often that an
Open Set Learning with Counterfactual Images
Open Set Learning with Counterfactual Images Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, Fuxin Li Collaborative Robotics and Intelligent Systems Institute Oregon State University Abstract.
Learning to Sample Using Stein Discrepancy
Learning to Sample Using Stein Discrepancy Dilin Wang Yihao Feng Qiang Liu Department of Computer Science Dartmouth College Hanover, NH 03755 {dilin.wang.gr, yihao.feng.gr, qiang.liu}@dartmouth.edu Abstract
Variational Autoencoders for Classical Spin Models
Variational Autoencoders for Classical Spin Models Benjamin Nosarzewski Stanford University bln@stanford.edu Abstract Lattice spin models are used to study the magnetic behavior of interacting electrons
arxiv: v1 [cs.lg] 22 Jan 2018
![arxiv: v1 [cs.lg] 22 Jan 2018](/thumbs/90/104480573.jpg "arxiv: v1 [cs.lg] 22 Jan 2018")
Mitigating Unwanted Biases with Adversarial Learning Brian Hu Zhang Stanford University Stanford, CA bhz@stanford.edu Blake Lemoine Google Mountain View, CA lemoine@google.com Margaret Mitchell Google
Generative Adversarial Networks for Real-time Stability of Inverter-based Systems
Generative Adversarial Networks for Real-time Stability of Inverter-based Systems Xilei Cao, Gurupraanesh Raman, Gururaghav Raman, and Jimmy Chih-Hsien Peng Department of Electrical and Computer Engineering
Supplementary Materials for: f-gan: Training Generative Neural Samplers using Variational Divergence Minimization
Supplementary Materials for: f-gan: Training Generative Neural Samplers using Variational Divergence Minimization Sebastian Nowozin, Botond Cseke, Ryota Tomioka Machine Intelligence and Perception Group
LEARNING TO SAMPLE WITH ADVERSARIALLY LEARNED LIKELIHOOD-RATIO
LEARNING TO SAMPLE WITH ADVERSARIALLY LEARNED LIKELIHOOD-RATIO Chunyuan Li, Jianqiao Li, Guoyin Wang & Lawrence Carin Duke University {chunyuan.li,jianqiao.li,guoyin.wang,lcarin}@duke.edu ABSTRACT We link
Adversarial Examples Generation and Defense Based on Generative Adversarial Network
Adversarial Examples Generation and Defense Based on Generative Adversarial Network Fei Xia (06082760), Ruishan Liu (06119690) December 15, 2016 1 Abstract We propose a novel generative adversarial network
Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks
Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks Panos Stinis (joint work with T. Hagge, A.M. Tartakovsky and E. Yeung) Pacific Northwest National Laboratory
Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels
Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels (possibly selecting from a set, i.e. kernel learning)
Compressing deep neural networks
From Data to Decisions - M.Sc. Data Science Compressing deep neural networks Challenges and theoretical foundations Presenter: Simone Scardapane University of Exeter, UK Table of contents Introduction
From perceptrons to word embeddings. Simon Šuster University of Groningen
From perceptrons to word embeddings Simon Šuster University of Groningen Outline A basic computational unit Weighting some input to produce an output: classification Perceptron Classify tweets Written
Generative Adversarial Networks, and Applications
Generative Adversarial Networks, and Applications Ali Mirzaei Nimish Srivastava Kwonjoon Lee Songting Xu CSE 252C 4/12/17 2/44 Outline: Generative Models vs Discriminative Models (Background) Generative
Dynamic Data Modeling, Recognition, and Synthesis. Rui Zhao Thesis Defense Advisor: Professor Qiang Ji
Dynamic Data Modeling, Recognition, and Synthesis Rui Zhao Thesis Defense Advisor: Professor Qiang Ji Contents Introduction Related Work Dynamic Data Modeling & Analysis Temporal localization Insufficient
Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net
Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net Supplementary Material Xingang Pan 1, Ping Luo 1, Jianping Shi 2, and Xiaoou Tang 1 1 CUHK-SenseTime Joint Lab, The Chinese University
Machine learning in ALICE
Activities in ALICE and heavy-ion physics (CERN) (05.04.2018) Outline This lecture consists of two parts 1) Overview on machine learning activities in ALICE Jets Particle identification Charmed baryons
Generative adversarial networks
14-1: Generative adversarial networks Prof. J.C. Kao, UCLA Generative adversarial networks Why GANs? GAN intuition GAN equilibrium GAN implementation Practical considerations Much of these notes are based
Adversarial learning to eliminate systematic errors: a case study in High Energy Physics
Adversarial learning to eliminate systematic errors: a case study in High Energy Physics Victor Estrade 1, Cécile Germain 1, Isabelle Guyon 1, David Rousseau 2 1. LRI, TAU, UPSud. 2. IN2P3, LAL. Université
Text2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to Action Hyemin Ahn, Timothy Ha*, Yunho Choi*, Hwiyeon Yoo*, and Songhwai Oh Abstract In this paper, we propose a generative model which learns
arxiv: v2 [cs.lg] 21 Aug 2018
![arxiv: v2 [cs.lg] 21 Aug 2018](/thumbs/87/96459185.jpg "arxiv: v2 [cs.lg] 21 Aug 2018")
CoT: Cooperative Training for Generative Modeling of Discrete Data arxiv:1804.03782v2 [cs.lg] 21 Aug 2018 Sidi Lu Shanghai Jiao Tong University steve_lu@apex.sjtu.edu.cn Weinan Zhang Shanghai Jiao Tong
Wave-dynamics simulation using deep neural networks
Wave-dynamics simulation using deep neural networks Weiqiang Zhu Stanford University zhuwq@stanford.edu Yixiao Sheng Stanford University yixiao2@stanford.edu Yi Sun Stanford University ysun4@stanford.edu
Boosted Top Tagging with Deep Neural Networks
Boosted Top Tagging with Deep Neural Networks Jannicke Pearkes University of British Columbia, Engineering Physics Wojtek Fedorko, Alison Lister, Colin Gay Inter-Experimental Machine Learning Workshop
arxiv: v1 [cs.lg] 28 Dec 2017
![arxiv: v1 [cs.lg] 28 Dec 2017](/thumbs/80/82263025.jpg "arxiv: v1 [cs.lg] 28 Dec 2017")
PixelSNAIL: An Improved Autoregressive Generative Model arxiv:1712.09763v1 [cs.lg] 28 Dec 2017 Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, Pieter Abbeel Embodied Intelligence UC Berkeley, Department of
Lecture 14: Deep Generative Learning
Generative Modeling CSED703R: Deep Learning for Visual Recognition (2017F) Lecture 14: Deep Generative Learning Density estimation Reconstructing probability density function using samples Bohyung Han
Classifaction of gg gh against qg qh
Jet Variables, Machine Learning, Box vs Triangle Diagrams Joshua Lin + Ben Nachman joshua.z.lin@gmail.com LBNL, Berkeley 2018 January 8-12 Conference, LBNL Introduction 2 These days, we understand very
Introduction to Support Vector Machines
Introduction to Support Vector Machines Hsuan-Tien Lin Learning Systems Group, California Institute of Technology Talk in NTU EE/CS Speech Lab, November 16, 2005 H.-T. Lin (Learning Systems Group) Introduction
Multiplicative Noise Channel in Generative Adversarial Networks
Multiplicative Noise Channel in Generative Adversarial Networks Xinhan Di Deepearthgo Deepearthgo@gmail.com Pengqian Yu National University of Singapore yupengqian@u.nus.edu Abstract Additive Gaussian
Very Deep Residual Networks with Maxout for Plant Identification in the Wild Milan Šulc, Dmytro Mishkin, Jiří Matas
Very Deep Residual Networks with Maxout for Plant Identification in the Wild Milan Šulc, Dmytro Mishkin, Jiří Matas Center for Machine Perception Department of Cybernetics Faculty of Electrical Engineering
Large-Scale Feature Learning with Spike-and-Slab Sparse Coding
Large-Scale Feature Learning with Spike-and-Slab Sparse Coding Ian J. Goodfellow, Aaron Courville, Yoshua Bengio ICML 2012 Presented by Xin Yuan January 17, 2013 1 Outline Contributions Spike-and-Slab
arxiv: v1 [cs.lg] 18 Dec 2017
![arxiv: v1 [cs.lg] 18 Dec 2017](/thumbs/85/92121711.jpg "arxiv: v1 [cs.lg] 18 Dec 2017")
Squeezed Convolutional Variational AutoEncoder for Unsupervised Anomaly Detection in Edge Device Industrial Internet of Things arxiv:1712.06343v1 [cs.lg] 18 Dec 2017 Dohyung Kim, Hyochang Yang, Minki Chung,
arxiv: v1 [cs.lg] 10 Jun 2016
![arxiv: v1 [cs.lg] 10 Jun 2016](/thumbs/89/99233164.jpg "arxiv: v1 [cs.lg] 10 Jun 2016")
Deep Directed Generative Models with Energy-Based Probability Estimation arxiv:1606.03439v1 [cs.lg] 10 Jun 2016 Taesup Kim, Yoshua Bengio Department of Computer Science and Operations Research Université
Wild Variational Approximations
Wild Variational Approximations Yingzhen Li University of Cambridge Cambridge, CB2 1PZ, UK yl494@cam.ac.uk Qiang Liu Dartmouth College Hanover, NH 03755, USA qiang.liu@dartmouth.edu Abstract We formalise
arxiv: v1 [cs.lg] 24 Jul 2018
![arxiv: v1 [cs.lg] 24 Jul 2018](/thumbs/91/106034493.jpg "arxiv: v1 [cs.lg] 24 Jul 2018")
Learning Plannable Representations with Causal InfoGAN Thanard Kurutach UC Berkeley Aviv Tamar UC Berkeley Ge Yang University of Chicago arxiv:1807.09341v1 [cs.lg] 24 Jul 2018 Stuart Russell UC Berkeley
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE GRAVIS 2016 BASEL. Logistic Regression. Pattern Recognition 2016 Sandro Schönborn University of Basel
Logistic Regression Pattern Recognition 2016 Sandro Schönborn University of Basel Two Worlds: Probabilistic & Algorithmic We have seen two conceptual approaches to classification: data class density estimation
CS60010: Deep Learning
CS60010: Deep Learning Sudeshna Sarkar Spring 2018 16 Jan 2018 FFN Goal: Approximate some unknown ideal function f : X! Y Ideal classifier: y = f*(x) with x and category y Feedforward Network: Define parametric
Introduction to Deep Learning CMPT 733. Steven Bergner
Introduction to Deep Learning CMPT 733 Steven Bergner Overview Renaissance of artificial neural networks Representation learning vs feature engineering Background Linear Algebra, Optimization Regularization
Lecture 12. Neural Networks Bastian Leibe RWTH Aachen
Advanced Machine Learning Lecture 12 Neural Networks 10.12.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de This Lecture: Advanced Machine Learning Regression
The Success of Deep Generative Models
The Success of Deep Generative Models Jakub Tomczak AMLAB, University of Amsterdam CERN, 2018 What is AI about? What is AI about? Decision making: What is AI about? Decision making: new data High probability
SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation
SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation Moustafa Alzantot University of California, Los Angeles malzantot@ucla.edu Supriyo Chakraborty IBM T. J. Watson Research Center
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
OPTIMIZATION METHODS IN DEEP LEARNING
Tutorial outline OPTIMIZATION METHODS IN DEEP LEARNING Based on Deep Learning, chapter 8 by Ian Goodfellow, Yoshua Bengio and Aaron Courville Presented By Nadav Bhonker Optimization vs Learning Surrogate
arxiv: v3 [stat.ml] 20 Feb 2018
![arxiv: v3 [stat.ml] 20 Feb 2018](/thumbs/96/127732243.jpg "arxiv: v3 [stat.ml] 20 Feb 2018")
MANY PATHS TO EQUILIBRIUM: GANS DO NOT NEED TO DECREASE A DIVERGENCE AT EVERY STEP William Fedus 1, Mihaela Rosca 2, Balaji Lakshminarayanan 2, Andrew M. Dai 1, Shakir Mohamed 2 and Ian Goodfellow 1 1
ECE521 Lecture 7/8. Logistic Regression
ECE521 Lecture 7/8 Logistic Regression Outline Logistic regression (Continue) A single neuron Learning neural networks Multi-class classification 2 Logistic regression The output of a logistic regression
arxiv: v1 [cs.lg] 7 Sep 2017
![arxiv: v1 [cs.lg] 7 Sep 2017](/thumbs/84/89101857.jpg "arxiv: v1 [cs.lg] 7 Sep 2017")
Deep Learning the Physics of Transport Phenomena Amir Barati Farimani, Joseph Gomes, and Vijay S. Pande Department of Chemistry, Stanford University, Stanford, California 94305 (Dated: September 7, 2017)
Domain adaptation for deep learning
What you saw is not what you get Domain adaptation for deep learning Kate Saenko Successes of Deep Learning in AI A Learning Advance in Artificial Intelligence Rivals Human Abilities Deep Learning for
arxiv: v3 [stat.ml] 14 Mar 2018
![arxiv: v3 [stat.ml] 14 Mar 2018](/thumbs/89/100778205.jpg "arxiv: v3 [stat.ml] 14 Mar 2018")
A-NICE-MC: Adversarial Training for MCMC Jiaming Song Stanford University tsong@cs.stanford.edu Shengjia Zhao Stanford University zhaosj1@cs.stanford.edu Stefano Ermon Stanford University ermon@cs.stanford.edu
Variational Autoencoders (VAEs)
September 26 & October 3, 2017 Section 1 Preliminaries Kullback-Leibler divergence KL divergence (continuous case) p(x) andq(x) are two density distributions. Then the KL-divergence is defined as Z KL(p
Adversarial Examples
Adversarial Examples presentation by Ian Goodfellow Deep Learning Summer School Montreal August 9, 2015 In this presentation. - Intriguing Properties of Neural Networks. Szegedy et al., ICLR 2014. - Explaining
ON THE LIMITATIONS OF FIRST-ORDER APPROXIMATION IN GAN DYNAMICS
ON THE LIMITATIONS OF FIRST-ORDER APPROXIMATION IN GAN DYNAMICS Anonymous authors Paper under double-blind review ABSTRACT Generative Adversarial Networks (GANs) have been proposed as an approach to learning
SOLVING LINEAR INVERSE PROBLEMS USING GAN PRIORS: AN ALGORITHM WITH PROVABLE GUARANTEES. Viraj Shah and Chinmay Hegde
SOLVING LINEAR INVERSE PROBLEMS USING GAN PRIORS: AN ALGORITHM WITH PROVABLE GUARANTEES Viraj Shah and Chinmay Hegde ECpE Department, Iowa State University, Ames, IA, 5000 In this paper, we propose and
Generative Adversarial Networks
Generative Adversarial Networks Stefano Ermon, Aditya Grover Stanford University Lecture 10 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 10 1 / 17 Selected GANs https://github.com/hindupuravinash/the-gan-zoo
CSC321 Lecture 4: Learning a Classifier
CSC321 Lecture 4: Learning a Classifier Roger Grosse Roger Grosse CSC321 Lecture 4: Learning a Classifier 1 / 28 Overview Last time: binary classification, perceptron algorithm Limitations of the perceptron
Dynamic Prediction Length for Time Series with Sequence to Sequence Networks
Dynamic Prediction Length for Time Series with Sequence to Sequence Networks arxiv:1807.00425v1 [cs.lg] 2 Jul 2018 Mark Harmon, 1 Diego Klabjan 2 1 Department of Engineering Sciences and Applied Mathematics
Stochastic Video Prediction with Deep Conditional Generative Models
Stochastic Video Prediction with Deep Conditional Generative Models Rui Shu Stanford University ruishu@stanford.edu Abstract Frame-to-frame stochasticity remains a big challenge for video prediction. The
Recurrent Neural Network
Recurrent Neural Network Xiaogang Wang xgwang@ee..edu.hk March 2, 2017 Xiaogang Wang (linux) Recurrent Neural Network March 2, 2017 1 / 48 Outline 1 Recurrent neural networks Recurrent neural networks
Deep Feedforward Networks. Han Shao, Hou Pong Chan, and Hongyi Zhang
Deep Feedforward Networks Han Shao, Hou Pong Chan, and Hongyi Zhang Deep Feedforward Networks Goal: approximate some function f e.g., a classifier, maps input to a class y = f (x) x y Defines a mapping
Learning Adversarially Fair and Transferable Representations
David Madras 2 Elliot Creager 2 Toniann Pitassi 2 Richard Zemel 2 Abstract In this paper, we advocate for representation learning as the key to mitigating unfair prediction outcomes downstream. Motivated
ECE521 Lectures 9 Fully Connected Neural Networks
ECE521 Lectures 9 Fully Connected Neural Networks Outline Multi-class classification Learning multi-layer neural networks 2 Measuring distance in probability space We learnt that the squared L2 distance
A Unified View of Deep Generative Models
SAILING LAB Laboratory for Statistical Artificial InteLigence & INtegreative Genomics A Unified View of Deep Generative Models Zhiting Hu and Eric Xing Petuum Inc. Carnegie Mellon University 1 Deep generative
arxiv: v1 [cs.lg] 24 Jan 2019
![arxiv: v1 [cs.lg] 24 Jan 2019](/thumbs/88/115323077.jpg "arxiv: v1 [cs.lg] 24 Jan 2019")
Rithesh Kumar 1 Anirudh Goyal 1 Aaron Courville 1 2 Yoshua Bengio 1 2 3 arxiv:1901.08508v1 [cs.lg] 24 Jan 2019 Abstract Unsupervised learning is about capturing dependencies between variables and is driven
Provable Non-Convex Min-Max Optimization
Provable Non-Convex Min-Max Optimization Mingrui Liu, Hassan Rafique, Qihang Lin, Tianbao Yang Department of Computer Science, The University of Iowa, Iowa City, IA, 52242 Department of Mathematics, The
Generative models for missing value completion
Generative models for missing value completion Kousuke Ariga Department of Computer Science and Engineering University of Washington Seattle, WA 98105 koar8470@cs.washington.edu Abstract Deep generative
arxiv: v1 [cs.lg] 7 Nov 2017
![arxiv: v1 [cs.lg] 7 Nov 2017](/thumbs/84/89307865.jpg "arxiv: v1 [cs.lg] 7 Nov 2017")
Theoretical Limitations of Encoder-Decoder GAN architectures Sanjeev Arora, Andrej Risteski, Yi Zhang November 8, 2017 arxiv:1711.02651v1 [cs.lg] 7 Nov 2017 Abstract Encoder-decoder GANs architectures
NON-TRIVIAL APPLICATIONS OF BOOSTING:
NON-TRIVIAL APPLICATIONS OF BOOSTING: REWEIGHTING DISTRIBUTIONS WITH BDT, BOOSTING TO UNIFORMITY Alex Rogozhnikov YSDA, NRU HSE Heavy Flavour Data Mining workshop, Zurich, 2016 1 BOOSTING a family of machine
What Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1
What Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1 Multi-layer networks Steve Renals Machine Learning Practical MLP Lecture 3 7 October 2015 MLP Lecture 3 Multi-layer networks 2 What Do Single
Bayesian Deep Learning
Bayesian Deep Learning Mohammad Emtiyaz Khan AIP (RIKEN), Tokyo http://emtiyaz.github.io emtiyaz.khan@riken.jp June 06, 2018 Mohammad Emtiyaz Khan 2018 1 What will you learn? Why is Bayesian inference
Deep Variational Inference. FLARE Reading Group Presentation Wesley Tansey 9/28/2016
Deep Variational Inference FLARE Reading Group Presentation Wesley Tansey 9/28/2016 What is Variational Inference? What is Variational Inference? Want to estimate some distribution, p*(x) p*(x) What is
ACTIVATION MAXIMIZATION GENERATIVE ADVER-
ACTIVATION MAXIMIZATION GENERATIVE ADVER- SARIAL NETS Zhiming Zhou, an Cai Shanghai Jiao Tong University heyohai,hcai@apex.sjtu.edu.cn Shu Rong Yitu Tech shu.rong@yitu-inc.com Yuxuan Song, Kan Ren Shanghai
Vote. Vote on timing for night section: Option 1 (what we have now) Option 2. Lecture, 6:10-7:50 25 minute dinner break Tutorial, 8:15-9
Vote Vote on timing for night section: Option 1 (what we have now) Lecture, 6:10-7:50 25 minute dinner break Tutorial, 8:15-9 Option 2 Lecture, 6:10-7 10 minute break Lecture, 7:10-8 10 minute break Tutorial,
Latent Variable Models
Latent Variable Models Stefano Ermon, Aditya Grover Stanford University Lecture 5 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 5 1 / 31 Recap of last lecture 1 Autoregressive models:
arxiv: v1 [cs.lg] 6 Nov 2016
![arxiv: v1 [cs.lg] 6 Nov 2016](/thumbs/86/94094733.jpg "arxiv: v1 [cs.lg] 6 Nov 2016")
GENERATIVE ADVERSARIAL NETWORKS AS VARIA- TIONAL TRAINING OF ENERGY BASED MODELS Shuangfei Zhai Binghamton University Vestal, NY 13902, USA szhai2@binghamton.edu Yu Cheng IBM T.J. Watson Research Center