Singing Voice Separation using Generative Adversarial Networks
|
|
- Joshua Wilson
- 6 years ago
- Views:
Transcription
1 Singing Voice Separation using Generative Adversarial Networks Hyeong-seok Choi, Kyogu Lee Music and Audio Research Group Graduate School of Convergence Science and Technology Seoul National University {kekepa15, Ju-heon Lee College of Liberal Studies Seoul National University Abstract In this paper, we propose a novel approach extending Wasserstein generative adversarial networks (GANs) [3] to separate singing voice from the mixture signal. We used the mixture signal as a condition to generate singing voices and applied the U-net style network for the stable training of the model. Experiments with the DSD100 dataset show the promising results with the potential of using the GANs for music source separation. 1 Introduction Music source separation is the process of separating a specific source from a music signal. Separating the source from the mixture signal can be interpreted as maximizing the likelihood of the source from a given mixture. Our task is to perform this task using GANs [1] which are classified as a method to maximize the likelihood by using implicit density. GANs are usually used to produce samples from noise, but in recent years, research [7,8] has been under way to better tailor the desired sample with a certain constraint. In this paper, our research aims to generate singing voice signals using mixture signals as a condition. Background GANs are generative model that learns a function generator G θ to map noise samples z p(z) into the real data space. The main idea of training GANs is often described as a mini-max game between two players which are discriminator (D) and generator (G) [1]. The input of D is either real sample x P r or fake sample x x x P g and the mission of D is to classify x x x as fake and to classify x as real. Many improved GANs model was attempted [,3,5] and one of the notable GANs studies that provides both theoretic background and practical result is the Wasserstein GANs. It is a model that tries to reduce the Wasserstein distance between the data distribution (P r ) and the generated sample distribution (P g ). Using the Wasserstein distance, the GANs training can be done as follows. Note that, x = G(z), z p(z) and D is a set of function that holds 1-Lipschitz condition. min max E x P r [D(x)] E x Pg [D( x)] (1) G D D In order to enforce D to be a function that holds 1-Lipschitz condition, [] suggests to regularize objective function by adding a gradient penalty term. Note that Pˆx is a sampling distribution that samples from the straight line between x P r and x P g, that is, ˆx = ɛ x + (1 ɛ) x, when 0 ɛ 1 and λ g is a gradient penalty coefficient. L = E x Pg [D( x)] E x Pr [D(x)] + λ g Eˆx Pˆx [( ˆx D(ˆx) 1) ] () 31st Conference on Neural Information Processing Systems (NIPS 017), Long Beach, CA, USA.
2 3 Model setup 3.1 Objective function We define x m, x s, and x s as mixture, real source paired with mixture, and fake (generated) source paired with mixture respectively. In our setting, the goal of G is to transform x m into x s as similar as possible to x s and the goal of D is to distinguish real source x s from the fake source x s conditioned on x m. To formulate this, we changed the aforementioned objective () into conditional GANs fashion [7, 8]. Thus, the input of D becomes the concatenation of either (x m, x s ) or (x m, x s ). For the gradient penalty term, we uniformly sampled ˆx ms Pˆxms from the straight line between the concatenation of (x m, x s ) and (x m, x s ) []. L = E xm P data, x s P g [D(x m, x s )] E (xm,x s) P data [D(x m, x s )] + λ g E (xm,x s) P data, x s P g, ˆx ms Pˆxms [( ˆxms D(x m, ˆx ms ) 1) ] (3) As a final objective for generator, we added l1 loss term to check the effect of more conventional loss and experimented with three cases including the objective containing only l1 loss, only generative adversarial loss and finally a case that adds both terms together. Therefore, our final objective for each generator (L G ) and discriminator (L D ) is as follows. The coefficients for adversarial loss, gradient penalty loss and l1 loss are denoted as λ D, λ g and λ l1. L G = λ D E xm P data, x s P g [D(x m, x s )] + λ l1 E xs P r x s P g [ x s x s 1 ] () L D = λ D (E xm P data, x s P g [D(x m, x s )] E (xm,x s) P data [D(x m, x s )]) + λ g E (xm,x s) P data, x s P g, ˆx ms Pˆxms [( ˆxms D(x m, ˆx ms ) 1) ] (5) 3. Network structure for generator Our generator model is constructed as follows. First, as a deep neural network we adapt U-net structure [6]. U-net consists of encoding and decoding stages and the layers in each stage are composed of convolutional layers. In the encoding stage, inputs are encoded with convolutional layers followed by batch normalization. The inputs are encoded until it becomes a vector with a length of 08. Then, by using a fully connected layer(fc layer) we encode it to a vector with a length of 51. In the following decoding stage, the input of each layer is concatenated in channel axis by using skip connection from each corresponding layer of encoding stage. Then, the concatenated layers are decoded by deconvolutional layers followed by batch normalization. For the non-linearity functions of each convolutional and deconvolutional layer, we used leaky Relu except the last layer using Relu. The more details are described in Figure Network structure for discriminator Our discriminator model is constructed as follows. First, input either (x m, x s ) or (x m, x s ) is concatenated in channel axis. We use 5 layers of convolutional layers without batch normalization since it is invalid in gradient penalty setting []. After each batch normalization layer, we used leaky Relu as a non-linearity function except the last layer that we didn t use any non-linearity. The more details are described in Figure. One noticeable aspect of our discriminator model is the fact that we intended to make the output to have the size of [7]. This allows the each pixel value of the output to equally contribute to the Wasserstein distance we compute by simply taking the mean of the output pixel values. In this way, each pixel of the output corresponds to the each different receptive region with the same receptive size of Intuitively, we assume that this is a better idea than having a full receptive size of input size(51 18), since the receptive size is roughly a quarter of the input size, and hence, each pixel is able to make a decision over a different time-frequency region of input. Also, in practice, we found out that it is not only time consuming to train but also the train fails when the receptive size becomes bigger as the layer of discriminator becomes deeper.
3 Encode (Convolution layers) Height : 51 F 7 7 C : Width : 18 Channel: 1 C : C : Skip connections 8 reshape 1 1 F 7 7 C : 1 C : C : 18 C : 56 Decode (Deconvolution layers) Figure 1: Network structure for generator. It consists of two stages, encoding and decoding with skip connections from encoding layers. F denotes filter size, S_h denotes strides over height, S_w denotes strides over width and C denotes the output channel for the next layer. C : C : 18 C : 56 C : 1 Height : Width : 18 Channel : Figure : Network structure for discriminator. Preliminary experiments.1 Dataset The DSD100 data set was used for model training. The DSD100 consists of 50 songs as a development set and 50 songs as a test set, each consisting of mixture and four sources (vocal, bass, drums and others). All recordings are digitized with a sampling frequency of,100hz.. Mini-batch composition To train our conditional GANs model, we composed our mini-batch having two parts, one as a condition part and the other one as a target source part. It might seem natural to include only the mixtures into the condition part, but we composed the condition part to include some proportion of singing voice sources as well as the mixtures. This technique was tried due to the nature of common popular music that includes intro, interlude and outro composed only with accompaniment, and hence the term "mixture" itself does not say much about the music signal composed of both singing voice and accompaniment. Because of this reason, lots of time the target source turns out to be a zero matrix which is not good for the training of the model. Moreover, in the music of the real world, there is also a chance of singing voice appearing only by itself (e.g., Acappella). Therefore we thought 3
4 there is also a need to prepare for this situation. In most experiments, the ratio between the mixtures and the singing voices in condition part was adjusted to 7:1. This is illustrated in Figure 3. Condition Target source x " x$ " Condition Target source x " x$ " x " xx$ " " x " x " Mixture Mixture Vocal Figure 3: Composition of mini-batch used in training. Mixture, true vocal, and fake vocal from generator is denoted as x m, x s and x s, respectively..3 Pre- & post-processing As a preprocessing, the songs in the dataset are split into audio segments to have a time length of seconds with a overlap of 1 second between each audio segment. Then, we converted each stereo audio segment to a mono by taking the mean of two channels. Next, we down-sampled the audio segment to,00hz and then performed short time Fourier transform on this waveform with a window size of 10 frames and hop length of 56 frames. This setting in turn makes a segment of audio into a matrix with a size of As a post-processing, to change the final extracted vocal spectrogram as a waveform, we simply applied inverse spectrogram Fourier transform using a phase of input mixture spectrogram.. Results In Figure, we show four log magnitude spectrograms to compare the effect of generative adversarial loss and l1 loss. We found out that by using generative adversarial loss, the network tries to remove the accompaniment part more aggressively compared to the case when we only use l1 loss. Thus, we assume that one of the keys to train this model is to adjust the coefficients of l1 loss (λ l1 ) and generative adversarial loss term (λ D ). Still, we have not evaluated our algorithm with the common metrics in the music source separation task which are SDR (Source to Distortion Ratio), SIR (Source to Interference Ratio), and SAR (Source to Artifact Ratio). However, for a fair quantitative evaluation, we are planning to compare our model with the algorithm evaluation results in Signal Separation Evaluation Campaign(SiSEC) 0. The generated vocal samples of our model are available on the demo website 1. (a) (b) (c) (d) (e) Figure : Log magnitude spectrograms of (a) mixture, (b) true vocal, (c) estimated vocal using generative adversarial loss only, (d) estimated vocal using l1 loss only, and (e) estimated vocal using both generative adversarial loss and l1 loss. 1 Demo audio samples for our model are available on the website :
5 References [1] Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. & Bengio, Y. (01). Generative adversarial nets. Advances in neural information processing systems, [] Mao, X., Li, Q., Xie, H., Lau, R. Y., Wang, Z., & Smolley, S. P. (0). Least squares generative adversarial networks. arxiv preprint ArXiv: [3] Arjovsky, M., Chintala, S., & Bottou, L. (017). Wasserstein generative adversarial networks. In International Conference on Machine Learning, 1-3. [] Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (017). Improved Training of Wasserstein GANs. arxiv preprint arxiv: [5] Kodali, N., Abernethy, J., Hays, J., & Kira, Z. (017). How to Train Your DRAGAN. arxiv preprint arxiv: [6] Ronneberger, O. (017). Invited Talk: U-Net Convolutional Networks for Biomedical Image Segmentation. Informatik aktuell Bildverarbeitung für die Medizin 017,3-3. [7] Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (0). Image-to-image translation with conditional adversarial networks.arxiv preprint arxiv: [8] Mirza, M., & Osindero, S. (01). Conditional generative adversarial nets. arxiv preprint arxiv:
GANs, GANs everywhere
GANs, GANs everywhere particularly, in High Energy Physics Maxim Borisyak Yandex, NRU Higher School of Economics Generative Generative models Given samples of a random variable X find X such as: P X P
More informationDeep Generative Models. (Unsupervised Learning)
Deep Generative Models (Unsupervised Learning) CEng 783 Deep Learning Fall 2017 Emre Akbaş Reminders Next week: project progress demos in class Describe your problem/goal What you have done so far What
More informationON ADVERSARIAL TRAINING AND LOSS FUNCTIONS FOR SPEECH ENHANCEMENT. Ashutosh Pandey 1 and Deliang Wang 1,2. {pandey.99, wang.5664,
ON ADVERSARIAL TRAINING AND LOSS FUNCTIONS FOR SPEECH ENHANCEMENT Ashutosh Pandey and Deliang Wang,2 Department of Computer Science and Engineering, The Ohio State University, USA 2 Center for Cognitive
More informationarxiv: v1 [cs.lg] 20 Apr 2017
Softmax GAN Min Lin Qihoo 360 Technology co. ltd Beijing, China, 0087 mavenlin@gmail.com arxiv:704.069v [cs.lg] 0 Apr 07 Abstract Softmax GAN is a novel variant of Generative Adversarial Network (GAN).
More informationGENERATIVE ADVERSARIAL LEARNING
GENERATIVE ADVERSARIAL LEARNING OF MARKOV CHAINS Jiaming Song, Shengjia Zhao & Stefano Ermon Computer Science Department Stanford University {tsong,zhaosj12,ermon}@cs.stanford.edu ABSTRACT We investigate
More informationNegative Momentum for Improved Game Dynamics
Negative Momentum for Improved Game Dynamics Gauthier Gidel Reyhane Askari Hemmat Mohammad Pezeshki Gabriel Huang Rémi Lepriol Simon Lacoste-Julien Ioannis Mitliagkas Mila & DIRO, Université de Montréal
More informationGenerative Adversarial Networks
Generative Adversarial Networks SIBGRAPI 2017 Tutorial Everything you wanted to know about Deep Learning for Computer Vision but were afraid to ask Presentation content inspired by Ian Goodfellow s tutorial
More informationLecture 14: Deep Generative Learning
Generative Modeling CSED703R: Deep Learning for Visual Recognition (2017F) Lecture 14: Deep Generative Learning Density estimation Reconstructing probability density function using samples Bohyung Han
More informationarxiv: v3 [stat.ml] 20 Feb 2018
MANY PATHS TO EQUILIBRIUM: GANS DO NOT NEED TO DECREASE A DIVERGENCE AT EVERY STEP William Fedus 1, Mihaela Rosca 2, Balaji Lakshminarayanan 2, Andrew M. Dai 1, Shakir Mohamed 2 and Ian Goodfellow 1 1
More informationNotes on Adversarial Examples
Notes on Adversarial Examples David Meyer dmm@{1-4-5.net,uoregon.edu,...} March 14, 2017 1 Introduction The surprising discovery of adversarial examples by Szegedy et al. [6] has led to new ways of thinking
More informationTraining Generative Adversarial Networks Via Turing Test
raining enerative Adversarial Networks Via uring est Jianlin Su School of Mathematics Sun Yat-sen University uangdong, China bojone@spaces.ac.cn Abstract In this article, we introduce a new mode for training
More informationImportance Reweighting Using Adversarial-Collaborative Training
Importance Reweighting Using Adversarial-Collaborative Training Yifan Wu yw4@andrew.cmu.edu Tianshu Ren tren@andrew.cmu.edu Lidan Mu lmu@andrew.cmu.edu Abstract We consider the problem of reweighting a
More informationGenerative Adversarial Networks, and Applications
Generative Adversarial Networks, and Applications Ali Mirzaei Nimish Srivastava Kwonjoon Lee Songting Xu CSE 252C 4/12/17 2/44 Outline: Generative Models vs Discriminative Models (Background) Generative
More informationGenerative adversarial networks
14-1: Generative adversarial networks Prof. J.C. Kao, UCLA Generative adversarial networks Why GANs? GAN intuition GAN equilibrium GAN implementation Practical considerations Much of these notes are based
More informationDeep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Emily Denton 1, Soumith Chintala 2, Arthur Szlam 2, Rob Fergus 2 1 New York University 2 Facebook AI Research Denotes equal
More informationText2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to Action Hyemin Ahn, Timothy Ha*, Yunho Choi*, Hwiyeon Yoo*, and Songhwai Oh Abstract In this paper, we propose a generative model which learns
More informationSome theoretical properties of GANs. Gérard Biau Toulouse, September 2018
Some theoretical properties of GANs Gérard Biau Toulouse, September 2018 Coauthors Benoît Cadre (ENS Rennes) Maxime Sangnier (Sorbonne University) Ugo Tanielian (Sorbonne University & Criteo) 1 video Source:
More informationEnforcing constraints for interpolation and extrapolation in Generative Adversarial Networks
Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks Panos Stinis (joint work with T. Hagge, A.M. Tartakovsky and E. Yeung) Pacific Northwest National Laboratory
More informationGenerative Adversarial Networks. Presented by Yi Zhang
Generative Adversarial Networks Presented by Yi Zhang Deep Generative Models N(O, I) Variational Auto-Encoders GANs Unreasonable Effectiveness of GANs GANs Discriminator tries to distinguish genuine data
More informationarxiv: v1 [eess.iv] 28 May 2018
Versatile Auxiliary Regressor with Generative Adversarial network (VAR+GAN) arxiv:1805.10864v1 [eess.iv] 28 May 2018 Abstract Shabab Bazrafkan, Peter Corcoran National University of Ireland Galway Being
More informationA QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS
A QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS Dan Hendrycks University of Chicago dan@ttic.edu Steven Basart University of Chicago xksteven@uchicago.edu ABSTRACT We introduce a
More informationarxiv: v3 [cs.lg] 11 Jun 2018
Lars Mescheder 1 Andreas Geiger 1 2 Sebastian Nowozin 3 arxiv:1801.04406v3 [cs.lg] 11 Jun 2018 Abstract Recent work has shown local convergence of GAN training for absolutely continuous data and generator
More informationOpen Set Learning with Counterfactual Images
Open Set Learning with Counterfactual Images Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, Fuxin Li Collaborative Robotics and Intelligent Systems Institute Oregon State University Abstract.
More informationImproving Visual Semantic Embedding By Adversarial Contrastive Estimation
Improving Visual Semantic Embedding By Adversarial Contrastive Estimation Huan Ling Department of Computer Science University of Toronto huan.ling@mail.utoronto.ca Avishek Bose Department of Electrical
More informationWhich Training Methods for GANs do actually Converge?
Lars Mescheder 1 Andreas Geiger 1 2 Sebastian Nowozin 3 Abstract Recent work has shown local convergence of GAN training for absolutely continuous data and generator distributions. In this paper, we show
More informationWasserstein GAN. Juho Lee. Jan 23, 2017
Wasserstein GAN Juho Lee Jan 23, 2017 Wasserstein GAN (WGAN) Arxiv submission Martin Arjovsky, Soumith Chintala, and Léon Bottou A new GAN model minimizing the Earth-Mover s distance (Wasserstein-1 distance)
More informationThe Success of Deep Generative Models
The Success of Deep Generative Models Jakub Tomczak AMLAB, University of Amsterdam CERN, 2018 What is AI about? What is AI about? Decision making: What is AI about? Decision making: new data High probability
More informationarxiv: v1 [cs.sd] 30 Oct 2017
GENERATIVE ADVERSARIAL SOURCE SEPARATION Y.Cem Subaan, Paris Smaragdis, UIUC, Adobe Systems {subaan, paris}@illinois.edu arxiv:17.779v1 [cs.sd] 30 Oct 017 ABSTRACT Generative source separation methods
More informationMultiplicative Noise Channel in Generative Adversarial Networks
Multiplicative Noise Channel in Generative Adversarial Networks Xinhan Di Deepearthgo Deepearthgo@gmail.com Pengqian Yu National University of Singapore yupengqian@u.nus.edu Abstract Additive Gaussian
More informationNishant Gurnani. GAN Reading Group. April 14th, / 107
Nishant Gurnani GAN Reading Group April 14th, 2017 1 / 107 Why are these Papers Important? 2 / 107 Why are these Papers Important? Recently a large number of GAN frameworks have been proposed - BGAN, LSGAN,
More informationON THE DIFFERENCE BETWEEN BUILDING AND EX-
ON THE DIFFERENCE BETWEEN BUILDING AND EX- TRACTING PATTERNS: A CAUSAL ANALYSIS OF DEEP GENERATIVE MODELS. Anonymous authors Paper under double-blind review ABSTRACT Generative models are important tools
More informationGenerative Adversarial Networks
Generative Adversarial Networks Stefano Ermon, Aditya Grover Stanford University Lecture 10 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 10 1 / 17 Selected GANs https://github.com/hindupuravinash/the-gan-zoo
More informationCS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS
CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS LAST TIME Intro to cudnn Deep neural nets using cublas and cudnn TODAY Building a better model for image classification Overfitting
More informationGenerative Adversarial Networks for Real-time Stability of Inverter-based Systems
Generative Adversarial Networks for Real-time Stability of Inverter-based Systems Xilei Cao, Gurupraanesh Raman, Gururaghav Raman, and Jimmy Chih-Hsien Peng Department of Electrical and Computer Engineering
More informationTwo at Once: Enhancing Learning and Generalization Capacities via IBN-Net
Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net Supplementary Material Xingang Pan 1, Ping Luo 1, Jianping Shi 2, and Xiaoou Tang 1 1 CUHK-SenseTime Joint Lab, The Chinese University
More informationarxiv: v4 [cs.cv] 5 Sep 2018
Wasserstein Divergence for GANs Jiqing Wu 1, Zhiwu Huang 1, Janine Thoma 1, Dinesh Acharya 1, and Luc Van Gool 1,2 arxiv:1712.01026v4 [cs.cv] 5 Sep 2018 1 Computer Vision Lab, ETH Zurich, Switzerland {jwu,zhiwu.huang,jthoma,vangool}@vision.ee.ethz.ch,
More informationarxiv: v4 [stat.ml] 16 Sep 2018
Relaxed Wasserstein with Applications to GANs in Guo Johnny Hong Tianyi Lin Nan Yang September 9, 2018 ariv:1705.07164v4 [stat.ml] 16 Sep 2018 Abstract We propose a novel class of statistical divergences
More informationarxiv: v1 [cs.lg] 18 Dec 2017
Squeezed Convolutional Variational AutoEncoder for Unsupervised Anomaly Detection in Edge Device Industrial Internet of Things arxiv:1712.06343v1 [cs.lg] 18 Dec 2017 Dohyung Kim, Hyochang Yang, Minki Chung,
More informationDeep Feedforward Networks
Deep Feedforward Networks Liu Yang March 30, 2017 Liu Yang Short title March 30, 2017 1 / 24 Overview 1 Background A general introduction Example 2 Gradient based learning Cost functions Output Units 3
More informationMatching Adversarial Networks
Matching Adversarial Networks Gellért Máttyus and Raquel Urtasun Uber Advanced Technologies Group and University of Toronto gmattyus@uber.com, urtasun@uber.com Abstract Generative Adversarial Nets (GANs)
More informationarxiv: v1 [cs.lg] 7 Nov 2017
Theoretical Limitations of Encoder-Decoder GAN architectures Sanjeev Arora, Andrej Risteski, Yi Zhang November 8, 2017 arxiv:1711.02651v1 [cs.lg] 7 Nov 2017 Abstract Encoder-decoder GANs architectures
More informationarxiv: v3 [cs.lg] 25 Dec 2017
Improved Training of Wasserstein GANs arxiv:1704.00028v3 [cs.lg] 25 Dec 2017 Ishaan Gulrajani 1, Faruk Ahmed 1, Martin Arjovsky 2, Vincent Dumoulin 1, Aaron Courville 1,3 1 Montreal Institute for Learning
More informationDo you like to be successful? Able to see the big picture
Do you like to be successful? Able to see the big picture 1 Are you able to recognise a scientific GEM 2 How to recognise good work? suggestions please item#1 1st of its kind item#2 solve problem item#3
More informationCompressing deep neural networks
From Data to Decisions - M.Sc. Data Science Compressing deep neural networks Challenges and theoretical foundations Presenter: Simone Scardapane University of Exeter, UK Table of contents Introduction
More informationSupplementary Materials for: f-gan: Training Generative Neural Samplers using Variational Divergence Minimization
Supplementary Materials for: f-gan: Training Generative Neural Samplers using Variational Divergence Minimization Sebastian Nowozin, Botond Cseke, Ryota Tomioka Machine Intelligence and Perception Group
More informationEE-559 Deep learning 10. Generative Adversarial Networks
EE-559 Deep learning 10. Generative Adversarial Networks François Fleuret https://fleuret.org/dlc/ [version of: May 17, 2018] ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE Adversarial generative models François
More informationAn overview of deep learning methods for genomics
An overview of deep learning methods for genomics Matthew Ploenzke STAT115/215/BIO/BIST282 Harvard University April 19, 218 1 Snapshot 1. Brief introduction to convolutional neural networks What is deep
More informationarxiv: v3 [cs.lg] 10 Sep 2018
The relativistic discriminator: a key element missing from standard GAN arxiv:1807.00734v3 [cs.lg] 10 Sep 2018 Alexia Jolicoeur-Martineau Lady Davis Institute Montreal, Canada alexia.jolicoeur-martineau@mail.mcgill.ca
More informationPredicting Deeper into the Future of Semantic Segmentation Supplementary Material
Predicting Deeper into the Future of Semantic Segmentation Supplementary Material Pauline Luc 1,2 Natalia Neverova 1 Camille Couprie 1 Jakob Verbeek 2 Yann LeCun 1,3 1 Facebook AI Research 2 Inria Grenoble,
More informationWaveNet: A Generative Model for Raw Audio
WaveNet: A Generative Model for Raw Audio Ido Guy & Daniel Brodeski Deep Learning Seminar 2017 TAU Outline Introduction WaveNet Experiments Introduction WaveNet is a deep generative model of raw audio
More informationarxiv: v2 [cs.lg] 21 Aug 2018
CoT: Cooperative Training for Generative Modeling of Discrete Data arxiv:1804.03782v2 [cs.lg] 21 Aug 2018 Sidi Lu Shanghai Jiao Tong University steve_lu@apex.sjtu.edu.cn Weinan Zhang Shanghai Jiao Tong
More informationarxiv: v2 [cs.sd] 4 Feb 2019
SEMI-SUPERVISED MONAURAL SINGING VOICE SEPARATION WITH A MASKING NETWORK TRAINED ON SYNTHETIC MIXTURES Michael Michelashvili 1, Sagie Benaim 1, Lior Wolf 1,2 1 Tel Aviv University 2 Facebook AI Research
More informationDeep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
More informationA Variance Modeling Framework Based on Variational Autoencoders for Speech Enhancement
A Variance Modeling Framework Based on Variational Autoencoders for Speech Enhancement Simon Leglaive 1 Laurent Girin 1,2 Radu Horaud 1 1: Inria Grenoble Rhône-Alpes 2: Univ. Grenoble Alpes, Grenoble INP,
More informationFirst Order Generative Adversarial Networks
Calvin Seward 1 2 Thomas Unterthiner 2 Urs Bergmann 1 Nikolay Jetchev 1 Sepp Hochreiter 2 Abstract GANs excel at learning high dimensional distributions, but they can update generator parameters in directions
More informationWafer Pattern Recognition Using Tucker Decomposition
Wafer Pattern Recognition Using Tucker Decomposition Ahmed Wahba, Li-C. Wang, Zheng Zhang UC Santa Barbara Nik Sumikawa NXP Semiconductors Abstract In production test data analytics, it is often that an
More informationarxiv: v1 [cs.lg] 6 Nov 2016
GENERATIVE ADVERSARIAL NETWORKS AS VARIA- TIONAL TRAINING OF ENERGY BASED MODELS Shuangfei Zhai Binghamton University Vestal, NY 13902, USA szhai2@binghamton.edu Yu Cheng IBM T.J. Watson Research Center
More informationCAUSAL GAN: LEARNING CAUSAL IMPLICIT GENERATIVE MODELS WITH ADVERSARIAL TRAINING
CAUSAL GAN: LEARNING CAUSAL IMPLICIT GENERATIVE MODELS WITH ADVERSARIAL TRAINING (Murat Kocaoglu, Christopher Snyder, Alexandros G. Dimakis & Sriram Vishwanath, 2017) Summer Term 2018 Created for the Seminar
More informationOPTIMIZATION METHODS IN DEEP LEARNING
Tutorial outline OPTIMIZATION METHODS IN DEEP LEARNING Based on Deep Learning, chapter 8 by Ian Goodfellow, Yoshua Bengio and Aaron Courville Presented By Nadav Bhonker Optimization vs Learning Surrogate
More informationProvable Non-Convex Min-Max Optimization
Provable Non-Convex Min-Max Optimization Mingrui Liu, Hassan Rafique, Qihang Lin, Tianbao Yang Department of Computer Science, The University of Iowa, Iowa City, IA, 52242 Department of Mathematics, The
More informationarxiv: v1 [cs.lg] 7 Sep 2017
Deep Learning the Physics of Transport Phenomena Amir Barati Farimani, Joseph Gomes, and Vijay S. Pande Department of Chemistry, Stanford University, Stanford, California 94305 (Dated: September 7, 2017)
More informationExperiments on the Consciousness Prior
Yoshua Bengio and William Fedus UNIVERSITÉ DE MONTRÉAL, MILA Abstract Experiments are proposed to explore a novel prior for representation learning, which can be combined with other priors in order to
More informationBiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN Jogendra Nath Kundu Maharshi Gor R. Venkatesh Babu Video Analytics Lab, Department of Computational and Data Sciences Indian Institute of Science,
More informationUnderstanding GANs: Back to the basics
Understanding GANs: Back to the basics David Tse Stanford University Princeton University May 15, 2018 Joint work with Soheil Feizi, Farzan Farnia, Tony Ginart, Changho Suh and Fei Xia. GANs at NIPS 2017
More informationSegmentation of Cell Membrane and Nucleus using Branches with Different Roles in Deep Neural Network
Segmentation of Cell Membrane and Nucleus using Branches with Different Roles in Deep Neural Network Tomokazu Murata 1, Kazuhiro Hotta 1, Ayako Imanishi 2, Michiyuki Matsuda 2 and Kenta Terai 2 1 Meijo
More informationTop Tagging with Lorentz Boost Networks and Simulation of Electromagnetic Showers with a Wasserstein GAN
Top Tagging with Lorentz Boost Networks and Simulation of Electromagnetic Showers with a Wasserstein GAN Y. Rath, M. Erdmann, B. Fischer, L. Geiger, E. Geiser, J.Glombitza, D. Noll, T. Quast, M. Rieger,
More informationarxiv: v3 [stat.ml] 14 Mar 2018
A-NICE-MC: Adversarial Training for MCMC Jiaming Song Stanford University tsong@cs.stanford.edu Shengjia Zhao Stanford University zhaosj1@cs.stanford.edu Stefano Ermon Stanford University ermon@cs.stanford.edu
More informationCONTINUOUS-TIME FLOWS FOR EFFICIENT INFER-
CONTINUOUS-TIME FLOWS FOR EFFICIENT INFER- ENCE AND DENSITY ESTIMATION Anonymous authors Paper under double-blind review ABSTRACT Two fundamental problems in unsupervised learning are efficient inference
More informationVariational Autoencoders for Classical Spin Models
Variational Autoencoders for Classical Spin Models Benjamin Nosarzewski Stanford University bln@stanford.edu Abstract Lattice spin models are used to study the magnetic behavior of interacting electrons
More informationNeural networks and optimization
Neural networks and optimization Nicolas Le Roux Criteo 18/05/15 Nicolas Le Roux (Criteo) Neural networks and optimization 18/05/15 1 / 85 1 Introduction 2 Deep networks 3 Optimization 4 Convolutional
More informationVariational Autoencoders (VAEs)
September 26 & October 3, 2017 Section 1 Preliminaries Kullback-Leibler divergence KL divergence (continuous case) p(x) andq(x) are two density distributions. Then the KL-divergence is defined as Z KL(p
More informationarxiv: v1 [cs.lg] 6 Dec 2018
Embedding-reparameterization procedure for manifold-valued latent variables in generative models arxiv:1812.02769v1 [cs.lg] 6 Dec 2018 Eugene Golikov Neural Networks and Deep Learning Lab Moscow Institute
More informationThe Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems
The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems Weinan E 1 and Bing Yu 2 arxiv:1710.00211v1 [cs.lg] 30 Sep 2017 1 The Beijing Institute of Big Data Research,
More informationUnderstanding GANs: the LQG Setting
Understanding GANs: the LQG Setting Soheil Feizi 1, Changho Suh 2, Fei Xia 1 and David Tse 1 1 Stanford University 2 Korea Advanced Institute of Science and Technology arxiv:1710.10793v1 [stat.ml] 30 Oct
More informationImproved Training of Wasserstein GANs
Improved Training of Wasserstein GANs Ishaan Gulrajani 1, Faruk Ahmed 1, Martin Arjovsky 2, Vincent Dumoulin 1, Aaron Courville 1,3 1 Montreal Institute for Learning Algorithms 2 Courant Institute of Mathematical
More informationIntroduction to Deep Neural Networks
Introduction to Deep Neural Networks Presenter: Chunyuan Li Pattern Classification and Recognition (ECE 681.01) Duke University April, 2016 Outline 1 Background and Preliminaries Why DNNs? Model: Logistic
More informationarxiv: v1 [cs.lg] 28 Dec 2017
PixelSNAIL: An Improved Autoregressive Generative Model arxiv:1712.09763v1 [cs.lg] 28 Dec 2017 Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, Pieter Abbeel Embodied Intelligence UC Berkeley, Department of
More informationMachine Learning for Computer Vision 8. Neural Networks and Deep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Machine Learning for Computer Vision 8. Neural Networks and Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group INTRODUCTION Nonlinear Coordinate Transformation http://cs.stanford.edu/people/karpathy/convnetjs/
More informationSource Separation Tutorial Mini-Series III: Extensions and Interpretations to Non-Negative Matrix Factorization
Source Separation Tutorial Mini-Series III: Extensions and Interpretations to Non-Negative Matrix Factorization Nicholas Bryan Dennis Sun Center for Computer Research in Music and Acoustics, Stanford University
More informationDeep Generative Models for Graph Generation. Jian Tang HEC Montreal CIFAR AI Chair, Mila
Deep Generative Models for Graph Generation Jian Tang HEC Montreal CIFAR AI Chair, Mila Email: jian.tang@hec.ca Deep Generative Models Goal: model data distribution p(x) explicitly or implicitly, where
More informationDynamic Prediction Length for Time Series with Sequence to Sequence Networks
Dynamic Prediction Length for Time Series with Sequence to Sequence Networks arxiv:1807.00425v1 [cs.lg] 2 Jul 2018 Mark Harmon, 1 Diego Klabjan 2 1 Department of Engineering Sciences and Applied Mathematics
More informationComposite Functional Gradient Learning of Generative Adversarial Models. Appendix
A. Main theorem and its proof Appendix Theorem A.1 below, our main theorem, analyzes the extended KL-divergence for some β (0.5, 1] defined as follows: L β (p) := (βp (x) + (1 β)p(x)) ln βp (x) + (1 β)p(x)
More informationConvolutional Neural Networks. Srikumar Ramalingam
Convolutional Neural Networks Srikumar Ramalingam Reference Many of the slides are prepared using the following resources: neuralnetworksanddeeplearning.com (mainly Chapter 6) http://cs231n.github.io/convolutional-networks/
More informationConvolutional Neural Network. Hung-yi Lee
al Neural Network Hung-yi Lee Why CNN for Image? [Zeiler, M. D., ECCV 2014] x 1 x 2 Represented as pixels x N The most basic classifiers Use 1 st layer as module to build classifiers Use 2 nd layer as
More informationBounded Information Rate Variational Autoencoders
Bounded Information Rate Variational Autoencoders ABSTRACT Daniel Braithwaite Victoria University of Wellington School of Engineering and Computer Science daniel.braithwaite@ecs.vuw.ac.nz This paper introduces
More informationarxiv: v1 [cs.lg] 8 Dec 2016
Improved generator objectives for GANs Ben Poole Stanford University poole@cs.stanford.edu Alexander A. Alemi, Jascha Sohl-Dickstein, Anelia Angelova Google Brain {alemi, jaschasd, anelia}@google.com arxiv:1612.02780v1
More informationCS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning Lei Lei Ruoxuan Xiong December 16, 2017 1 Introduction Deep Neural Network
More informationLEARNING TO SAMPLE WITH ADVERSARIALLY LEARNED LIKELIHOOD-RATIO
LEARNING TO SAMPLE WITH ADVERSARIALLY LEARNED LIKELIHOOD-RATIO Chunyuan Li, Jianqiao Li, Guoyin Wang & Lawrence Carin Duke University {chunyuan.li,jianqiao.li,guoyin.wang,lcarin}@duke.edu ABSTRACT We link
More informationarxiv: v3 [stat.ml] 5 Apr 2018
Adversarial Variational Optimization of Non-Differentiable Simulators Gilles Louppe University of Liège, Belgium Kyle Cranmer New York University, USA G.LOUPPE@ULIEGE.BE KYLE.CRANMER@NYU.EDU arxiv:1707.07113v3
More informationTUTORIAL PART 1 Unsupervised Learning
TUTORIAL PART 1 Unsupervised Learning Marc'Aurelio Ranzato Department of Computer Science Univ. of Toronto ranzato@cs.toronto.edu Co-organizers: Honglak Lee, Yoshua Bengio, Geoff Hinton, Yann LeCun, Andrew
More informationFrom perceptrons to word embeddings. Simon Šuster University of Groningen
From perceptrons to word embeddings Simon Šuster University of Groningen Outline A basic computational unit Weighting some input to produce an output: classification Perceptron Classify tweets Written
More informationWhen Harmonic Analysis Meets Machine Learning: Lipschitz Analysis of Deep Convolution Networks
When Harmonic Analysis Meets Machine Learning: Lipschitz Analysis of Deep Convolution Networks Radu Balan Department of Mathematics, AMSC, CSCAMM and NWC University of Maryland, College Park, MD Joint
More informationStochastic Video Prediction with Deep Conditional Generative Models
Stochastic Video Prediction with Deep Conditional Generative Models Rui Shu Stanford University ruishu@stanford.edu Abstract Frame-to-frame stochasticity remains a big challenge for video prediction. The
More informationTheories of Deep Learning
Theories of Deep Learning Lecture 02 Donoho, Monajemi, Papyan Department of Statistics Stanford Oct. 4, 2017 1 / 50 Stats 385 Fall 2017 2 / 50 Stats 285 Fall 2017 3 / 50 Course info Wed 3:00-4:20 PM in
More informationarxiv: v2 [stat.ml] 23 Mar 2018 Abstract
Published as a conference paper at CVPR 2018 Structured Uncertainty Prediction Networks Garoe Dorta 1,2 Sara Vicente 2 Lourdes Agapito 3 Neill D.F. Campbell 1 Ivor Simpson 2 1 University of Bath 2 Anthropics
More informationCSC321 Lecture 20: Reversible and Autoregressive Models
CSC321 Lecture 20: Reversible and Autoregressive Models Roger Grosse Roger Grosse CSC321 Lecture 20: Reversible and Autoregressive Models 1 / 23 Overview Four modern approaches to generative modeling:
More informationGenerating Text via Adversarial Training
Generating Text via Adversarial Training Yizhe Zhang, Zhe Gan, Lawrence Carin Department of Electronical and Computer Engineering Duke University, Durham, NC 27708 {yizhe.zhang,zhe.gan,lcarin}@duke.edu
More informationDomain adaptation for deep learning
What you saw is not what you get Domain adaptation for deep learning Kate Saenko Successes of Deep Learning in AI A Learning Advance in Artificial Intelligence Rivals Human Abilities Deep Learning for
More informationAdversarial Examples Generation and Defense Based on Generative Adversarial Network
Adversarial Examples Generation and Defense Based on Generative Adversarial Network Fei Xia (06082760), Ruishan Liu (06119690) December 15, 2016 1 Abstract We propose a novel generative adversarial network
More informationStatistical Machine Learning
Statistical Machine Learning Lecture 9 Numerical optimization and deep learning Niklas Wahlström Division of Systems and Control Department of Information Technology Uppsala University niklas.wahlstrom@it.uu.se
More information