Tutorial on Methods for Interpreting and Understanding Deep Neural Networks. Part 3: Applications & Discussion
|
|
|
- Nancy Dorthy Burke
- 5 years ago
- Views:
Transcription
1 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks W. Samek, G. Montavon, K.-R. Müller Part 3: Applications & Discussion ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller
2 Recap: Layer-wise Relevance Propagation (LRP) Layer-wise Relevance Propagation (LRP) (Bach et al. 2015) every neuron gets it s share of relevance depending on activation and strength of connection. Theoretical interpretation Deep Taylor Decomposition (Montavon et al., 2017) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 2/64
3 Recap: Layer-wise Relevance Propagation (LRP) Classification cat rooster dog ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 3/64
4 Recap: Layer-wise Relevance Propagation (LRP) Explanation cat rooster dog Initialization Idea: Backpropagate relevances = ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 4/64
5 Recap: Layer-wise Relevance Propagation (LRP) Explanation? cat rooster dog ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 5/64
6 Recap: Layer-wise Relevance Propagation (LRP) Explanation? cat rooster dog Theoretical interpretation Deep Taylor Decomposition (Montavon et al., 2017) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 6/64
7 Recap: Layer-wise Relevance Propagation (LRP) Explanation cat rooster dog ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 7/64
8 Explanations and now? Data Information Interpretable for human How good is the explanation? What can we do with it? ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 8/64
9 Explanations and now? How good is the explanation? What can we do with it? - Objective measure of quality - Compare explanation methods - Compare classifiers - Detect biases and flaws - Quantify use of context - Novel representation - Application in the sciences ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 9/64
10 Measuring Quality of Explanations Heatmap quality depends on classifier (better performance -> better heatmap?) - no guarantee that meaningful for humans explanation method - need objective measure to compare explanation methods Pixel flipping - good heatmap assigns high relevance to truly relevant pixels. - destroying these relevant pixels will strongly decrease classification score. - by measuring this decrease we can assess the quality of heatmaps. ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 10/64
11 Compare Explanation Methods LRP ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 11/64
12 Compare Explanation Methods LRP ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 12/64
13 Compare Explanation Methods LRP ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 13/64
14 Compare Explanation Methods LRP ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 14/64
15 Compare Explanation Methods Sensitivity ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 15/64
16 Compare Explanation Methods Sensitivity ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 16/64
17 Compare Explanation Methods Sensitivity ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 17/64
18 Compare Explanation Methods Random ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 18/64
19 Compare Explanation Methods Random ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 19/64
20 Compare Explanation Methods Random ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 20/64
21 Compare Explanation Methods LRP: Sensitivity: Random: LRP produces quantitatively better heatmaps than sensitivity analysis and random. What about more complex datasets? SUN397 ILSVRC2012 MIT Places 397 scene categories (108,754 images in total) 1000 categories (1.2 million training images) 205 scene categories (2.5 millions of images) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 21/64
22 Compare Explanation Methods Sensitivity Analysis (Simonyan et al. 2014) Deconvolution Method (Zeiler & Fergus 2014) LRP Algorithm (Bach et al. 2015) (Samek et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 22 /64
23 Compare Explanation Methods LRP produces quantitatively better heatmaps. (Samek et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 23/64
24 Compare Explanation Methods Same idea can be applied for other domains (e.g. text document classification) Pixel flipping = Word deleting Text classified as sci.med > LRP identifies most relevant words. (Arras et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 24/64
25 Compare Explanation Methods Deleting relevant words leads to a quick decrease of classifier accuracy. The decrease is much steeper for LRP than for random word deletion and deletion according to sensitivity. LRP better identifies relevant words. (Arras et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 25/64
26 Explanations and now? How good is the explanation? What can we do with it? - Objective measure of quality - Compare explanation methods - Compare classifiers - Detect biases and flaws - Quantify use of context - Novel representation - Application in the sciences ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 26/64
27 Application: Compare Classifiers 20 Newsgroups data set man king woman Test set performance word2vec / CNN model: 80.19% BoW/SVM model: 80.10% same performance > same strategy? man woman king queen? ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 27/64
28 Application: Compare Classifiers word2vec / CNN model BoW/SVM model (Arras et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 28/64
29 Application: Compare Classifiers word2vec / CNN model BoW/SVM model Words with maximum relevance (Arras et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 29/64
30 Application: Compare Classifiers Fisher Vector / SVM Classifier LRP general method for non-linear classifiers (Lapuschkin et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 30/64
31 Application: Compare Classifiers Fisher Vector / SVM Classifier Deep Neural Network - BVLC reference model + fine tuning. Dataset - PASCAL VOC 2007 (Lapuschkin et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 31/64
32 Application: Compare Classifiers Fisher Vector / SVM Classifier Deep Neural Network - BVLC reference model + fine tuning. Dataset - PASCAL VOC 2007 (Lapuschkin et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 32 /64
33 Application: Compare Classifiers same performance > same strategy? (Lapuschkin et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 33 /64
34 Application: Compare Classifiers horse images in PASCAL VOC 2007 ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 34 /64
35 Application: Compare Classifiers BVLC: - 8 Layers - ILSRCV: 16.4% GoogleNet: - 22 Layers - ILSRCV: 6.7% - Inception layers ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 35/64
36 Application: Compare Classifiers GoogleNet focuses on faces of animal. > suppresses background noise (Binder et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 36/64
37 Application: Measure Context Use how important is context? classifier how important is context? LRP decomposition allows meaningful pooling over bbox! importance = of context relevance outside bbox relevance inside bbox ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 37/64
38 Application: Measure Context Use Image Fisher Vector DNN airplane Context use boat chair sofa airplane chair sofa boat Large values indicate importance of context (Lapuschkin et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 38/64
39 Application: Measure Context Use 333 classes low acc Context use GoogleNet uses less context than BVLC model. Context use anti-correlated with performance. 333 classes medium acc 333 classes top acc BVLC Caffe Model 333 classes low acc 333 classes medium acc 333 classes top acc GoogleNet Model (Lapuschkin et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 39/64
40 Application: Novel Representation word2vec word2vec word2vec relevance relevance relevance = + + document vector (Arras et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 40/64
41 Application: Novel Representation 2D PCA projection of document vectors uniform TFIDF LRP Document vector computation is unsupervised. KNN-Performance on document vectors (Explanatory Power Index) LRP: TFIDF: uniform: (Arras et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 41/64
42 Application: Face Analysis Emotions Identifying age-related features Attractivity (Arbabzadah et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 42/64
43 Application: Video Analysis MVs assigned by encoder from RD-perspective > not necessarily real motion Session: IVMSP-P7: Image/Video Analysis & Applications Location: Churchill: Poster Area D Time: Tuesday, March 7, 13:30-15:30 (Srinivasan et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 43/64
44 Application: Video Analysis class chew Session: IVMSP-P7: Image/Video Analysis & Applications Location: Churchill: Poster Area D Time: Tuesday, March 7, 13:30-15:30 (Srinivasan et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 44/64
45 Application: Interpretability in the Sciences Brain-Computer Interfacing interpret interpret variance computation ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 45/64
46 Application: Interpretability in the Sciences Brain-Computer Interfacing (linear) spatial filtering interpret variance computation interpret (linear) classifier (LDA) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 46/64
47 Application: Interpretability in the Sciences Brain-Computer Interfacing How brain works subject-dependent > individual explanations CNN explain LRP DNN (Sturm et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 47/64
48 Application: Interpretability in the Sciences With LRP we can analyze relevant activity at each time point and channel. desynchronization over left sensorimotor cortex desynchronization over right sensorimotor cortex Allows to spatially & temporally identify important activity in EEG data. (Sturm et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 48/64
49 Application: Interpretability in the Sciences With LRP we can analyze what made a trial being misclassified. (Sturm et al. 2016) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 49/64
50 Application: Interpretability in the Sciences ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 50/64
51 Application: Interpretability in the Sciences Basic simulations do not exploit similarity structure between molecules. Our approach can be seed up analysis significantly. ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 51/64
52 Application: Interpretability in the Sciences Deep Tensor Network predicts molecular properties with state-of-the-art accuracy. Effect of energy of test charge > interpretable for human expert (Schütt et al. 2017) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 52/64
53 Discussion Individual Explanations vs. Feature Selection explain prediction of individual data sample What makes these images belong to category boat average explanations What are the most important features of boat images or how does a typical boat image look like Medical context: Patient s view vs. Population view ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 53/64
54 Discussion Post-hoc Interpretability vs. Interpretability as Model Prior Suboptimal or biased results if prior does not fit the data. Train best model you can get and explain it afterwards Include interpretability aspect into the model from the beginning Examples: Linearity, sparsity, hierarchy, disentangled representations ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 54/64
55 Discussion Post-hoc Interpretability vs. Interpretability as Model Prior Suboptimal or biased results if prior does not fit the data. Train best model you can get and explain it afterwards Include interpretability aspect into the model from the beginning Examples: Linearity, sparsity, hierarchy, disentangled representations Decision trees are highly interpretable. ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 55/64
56 Discussion Post-hoc Interpretability vs. Interpretability as Model Prior Suboptimal or biased results if prior does not fit the data. Train best model you can get and explain it afterwards Include interpretability aspect into the model from the beginning Examples: Linearity, sparsity, hierarchy, disentangled representations Disentangled representations are highly interpretable, but it may be hard to assign meaning to factors (often mixed). ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 56/64
57 Discussion Post-hoc Interpretability vs. Interpretability as Model Prior Suboptimal or biased results if prior does not fit the data. Train best model you can get and explain it afterwards Include interpretability aspect into the model from the beginning Examples: Linearity, sparsity, hierarchy, disentangled representations Sparse features can lie in non-interpretable domain (e.g. hidden layer activations) ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 57/64
58 Different aspects of interpretability What about negative evidence? Distinguish between positive evidence supporting a prediction (red) and negative evidence speaking against it (blue). LRP naturally distinguishes between positive and negative evidence. -> Sensitivity analysis and deconvolution approach do not ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 58/64
59 Different aspects of interpretability Explanation relative to what? - what makes this car a car (i.e. distinguishes it from other categories) - what distinguishes this car from another car - what changes to the image would make it belong less / more to category car ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 59/64
60 Summary - In many problems interpretability as important as prediction. - Explaining individual predictions is key. - We have powerful, mathematically well-founded methods (LRP / deep Taylor) to explain individual predictions. - More research needed on how to compare and evaluation all the different aspects of interpretability. - Many interesting applications with interpretable deep nets > more to come soon! ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 60/64
61 Take Home Message Don t be afraid of complex models. There are powerful, mathematically well-founded tools to explain individual predictions. ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 61/64
62 Thank you for your attention ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 62/64
63 References F Arbabzadeh, G Montavon, KR Müller, W Samek. Identifying Individual Facial Expressions by Deconstructing a Neural Network. Pattern Recognition - 38th German Conference, GCPR 2016, Lecture Notes in Computer Science, 9796:344-54, Springer International Publishing, L Arras, F Horn, G Montavon, KR Müller, W Samek. Explaining Predictions of Non-Linear Classifiers in NLP. Workshop on Representation Learning for NLP, Association for Computational Linguistics, 1-7, L Arras, F Horn, G Montavon, KR Müller, W Samek. "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach. arxiv: , S Bach, A Binder, G Montavon, F Klauschen, KR Müller, W Samek. On Pixel-wise Explanations for Non-Linear Classifier Decisions by Layer-wise Relevance Propagation. PLOS ONE, 10(7):e , A Binder, G Montavon, S Lapuschkin, KR Müller, W Samek. Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers. Artificial Neural Networks and Machine Learning ICANN 2016, Part II, Lecture Notes in Computer Science, Springer-Verlag, 9887:63-71, A Binder, W Samek, G Montavon, S Bach, KR Müller. Analyzing and Validating Neural Networks Predictions. ICML Workshop on Visualization for Deep Learning, 2016 S Lapuschkin, A Binder, G Montavon, KR Müller, Wojciech Samek. Analyzing Classifiers: Fisher Vectors and Deep Neural Networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), , 2016 ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 63/64
64 References G Montavon, S Bach, A Binder, W Samek, KR Müller. DeepTaylor Decomposition of Neural Networks. ICML Workshop on Visualization for Deep Learning, G Montavon, S Bach, A Binder, W Samek, KR Müller. Explaining NonLinear Classification Decisions with Deep Taylor Decomposition. Pattern Recognition, 65: , A Nguyen, A Dosovitskiy, J Yosinski, T Brox, J Clune. Synthesizing the Preferred Inputs for Neurons in Neural Networks via Deep Generator Networks. Conference on Neural Information Processing Systems (NIPS), , W Samek, A Binder, G Montavon, S Lapuschkin, KR Müller. Evaluating the visualization of what a Deep Neural Network has learned. IEEE Transactions on Neural Networks and Learning Systems, K Schütt, F Arbabzadah, S Chmiela, KR Müller, A Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature Communications, 13890, 2017 K Simonyan, A Vedaldi, A Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arxiv: , V Srinivasan, S Lapuschkin, C Hellge, KR Müller, W Samek. Interpretable human action recognition in compressed domain. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), I Sturm, S Lapuschkin, W Samek, KR Müller. Interpretable Deep Neural Networks for Single-Trial EEG Classification. Journal of Neuroscience Methods, 274: , ICASSP 2017 Tutorial W. Samek, G. Montavon & K.-R. Müller 64/64
Explaining and Interpreting Deep Neural Networks. Klaus-Robert Müller, Wojciech Samek, Gregoire Montavon, Sebastian Lapuschkin, Leila Arras et al.
Explaining and Interpreting Deep Neural Networks Klaus-Robert Müller, Wojciech Samek, Gregoire Montavon, Sebastian Lapuschkin, Leila Arras et al. Based on: ICASSP 2017 Tutorial Acknowledgements Outline:
Explaining Machine Learning Decisions
Explaining Machine Learning Decisions Grégoire Montavon, TU Berlin Joint work with: Wojciech Samek, Klaus-Robert Müller, Sebastian Lapuschkin, Alexander Binder 18/09/2018 Intl. Workshop ML & AI, Telecom
Explaining and Interpreting Deep Neural Networks. Klaus-Robert Müller, Wojciech Samek, Gregoire Montavon, Sebastian Lapuschkin, Kristof Schütt et al.
Explaining and Interpreting Deep Neural Networks Klaus-Robert Müller, Wojciech Samek, Gregoire Montavon, Sebastian Lapuschkin, Kristof Schütt et al. Outline general remarks on ML, also on explaining and
Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers
Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers Alexander Binder 1, Grégoire Montavon 2, Sebastian Lapuschkin 3, Klaus-Robert Müller 2,4, and Wojciech Samek 3 1 ISTD
Explaining Predictions of Non-Linear Classifiers in NLP
Explaining Predictions of Non-Linear Classifiers in NLP Leila Arras 1, Franziska Horn 2, Grégoire Montavon 2, Klaus-Robert Müller 2,3, and Wojciech Samek 1 1 Machine Learning Group, Fraunhofer Heinrich
Machine Learning and AI for the sciences Towards Understanding. Klaus-Robert Müller!!et al.!!
Machine Learning and AI for the sciences Towards Understanding Klaus-Robert Müller!!et al.!! Outline understanding single decisions of nonlinear learners Layer-wise Relevance Propagation (LRP) Applications
Explaining Predictions of Non-Linear Classifiers in NLP
Explaining Predictions of Non-Linear Classifiers in NLP Leila Arras 1, Franziska Horn 2, Grégoire Montavon 2, Klaus-Robert Müller 2,3, and Wojciech Samek 1 1 Machine Learning Group, Fraunhofer Heinrich
Tutorial on Interpreting and Explaining Deep Models in Computer Vision
Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek (Fraunhofer HHI) Grégoire Montavon (TU Berlin) Klaus-Robert Müller (TU Berlin) 08:30-09:15 Introduction KRM 09:15-10:00
Supervised Machine Learning: Learning SVMs and Deep Learning. Klaus-Robert Müller!!et al.!!
Supervised Machine Learning: Learning SVMs and Deep Learning Klaus-Robert Müller!!et al.!! Today s Tutorial Machine Learning introduction: ingredients for ML Kernel Methods and Deep networks with explaining
arxiv: v1 [cs.lg] 24 Jun 2017
![arxiv: v1 [cs.lg] 24 Jun 2017](/thumbs/74/70021686.jpg "arxiv: v1 [cs.lg] 24 Jun 2017")
Methods for Interpreting and Understanding Deep Neural Networks Grégoire Montavon a,, Wojciech Samek b,, Klaus-Robert Müller a,c,d, a Department of Electrical Engineering & Computer Science, Technische
Machine Learning for Computer Vision 8. Neural Networks and Deep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Machine Learning for Computer Vision 8. Neural Networks and Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group INTRODUCTION Nonlinear Coordinate Transformation http://cs.stanford.edu/people/karpathy/convnetjs/
Machine Learning and Sparsity. Klaus-Robert Müller!!et al.!!
Machine Learning and Sparsity Klaus-Robert Müller!!et al.!! Today s Talk sensing, sparse models and generalization interpretabilty and sparse methods explaining for nonlinear methods Sparse Models & Generalization?
Asaf Bar Zvi Adi Hayat. Semantic Segmentation
Asaf Bar Zvi Adi Hayat Semantic Segmentation Today s Topics Fully Convolutional Networks (FCN) (CVPR 2015) Conditional Random Fields as Recurrent Neural Networks (ICCV 2015) Gaussian Conditional random
Understanding How ConvNets See
Understanding How ConvNets See Slides from Andrej Karpathy Springerberg et al, Striving for Simplicity: The All Convolutional Net (ICLR 2015 workshops) CSC321: Intro to Machine Learning and Neural Networks,
Introduction to Deep Learning
Introduction to Deep Learning Some slides and images are taken from: David Wolfe Corne Wikipedia Geoffrey A. Hinton https://www.macs.hw.ac.uk/~dwcorne/teaching/introdl.ppt Feedforward networks for function
arxiv: v1 [cs.cv] 14 Sep 2016
![arxiv: v1 [cs.cv] 14 Sep 2016](/thumbs/89/100152104.jpg "arxiv: v1 [cs.cv] 14 Sep 2016")
Understanding Convolutional Neural Networks with A Mathematical Model arxiv:1609.04112v1 [cs.cv] 14 Sep 2016 Abstract C.-C. Jay Kuo Ming-Hsieh Department of Electrical Engineering University of Southern
Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
1 Explaining NonLinear Classification Decisions with Deep Taylor Decomposition Grégoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, and Klaus-Robert Müller arxiv:1512.2479v1 [cs.lg] 8 Dec
Feature Design. Feature Design. Feature Design. & Deep Learning
Artificial Intelligence and its applications Lecture 9 & Deep Learning Professor Daniel Yeung danyeung@ieee.org Dr. Patrick Chan patrickchan@ieee.org South China University of Technology, China Appropriately
Tasks ADAS. Self Driving. Non-machine Learning. Traditional MLP. Machine-Learning based method. Supervised CNN. Methods. Deep-Learning based
UNDERSTANDING CNN ADAS Tasks Self Driving Localizati on Perception Planning/ Control Driver state Vehicle Diagnosis Smart factory Methods Traditional Deep-Learning based Non-machine Learning Machine-Learning
Artificial Neural Networks D B M G. Data Base and Data Mining Group of Politecnico di Torino. Elena Baralis. Politecnico di Torino
Artificial Neural Networks Data Base and Data Mining Group of Politecnico di Torino Elena Baralis Politecnico di Torino Artificial Neural Networks Inspired to the structure of the human brain Neurons as
Convolutional neural networks
11-1: Convolutional neural networks Prof. J.C. Kao, UCLA Convolutional neural networks Motivation Biological inspiration Convolution operation Convolutional layer Padding and stride CNN architecture 11-2:
Fine-grained Classification
Fine-grained Classification Marcel Simon Department of Mathematics and Computer Science, Germany marcel.simon@uni-jena.de http://www.inf-cv.uni-jena.de/ Seminar Talk 23.06.2015 Outline 1 Motivation 2 3
An overview of deep learning methods for genomics
An overview of deep learning methods for genomics Matthew Ploenzke STAT115/215/BIO/BIST282 Harvard University April 19, 218 1 Snapshot 1. Brief introduction to convolutional neural networks What is deep
Artificial Intelligence
Artificial Intelligence Explainable AI Marc Toussaint University of Stuttgart Winter 2018/19 Explainable AI General Concept of Explaination Data, Objective, Method, & Input Counterfactuals & Pearl Fitting
How to do backpropagation in a brain
How to do backpropagation in a brain Geoffrey Hinton Canadian Institute for Advanced Research & University of Toronto & Google Inc. Prelude I will start with three slides explaining a popular type of deep
Spatial Transformer Networks
BIL722 - Deep Learning for Computer Vision Spatial Transformer Networks Max Jaderberg Andrew Zisserman Karen Simonyan Koray Kavukcuoglu Contents Introduction to Spatial Transformers Related Works Spatial
arxiv: v1 [cs.cv] 11 May 2015 Abstract
![arxiv: v1 [cs.cv] 11 May 2015 Abstract](/thumbs/80/80674557.jpg "arxiv: v1 [cs.cv] 11 May 2015 Abstract")
Training Deeper Convolutional Networks with Deep Supervision Liwei Wang Computer Science Dept UIUC lwang97@illinois.edu Chen-Yu Lee ECE Dept UCSD chl260@ucsd.edu Zhuowen Tu CogSci Dept UCSD ztu0@ucsd.edu
TUTORIAL PART 1 Unsupervised Learning
TUTORIAL PART 1 Unsupervised Learning Marc'Aurelio Ranzato Department of Computer Science Univ. of Toronto ranzato@cs.toronto.edu Co-organizers: Honglak Lee, Yoshua Bengio, Geoff Hinton, Yann LeCun, Andrew
Introduction to Convolutional Neural Networks (CNNs)
Introduction to Convolutional Neural Networks (CNNs) nojunk@snu.ac.kr http://mipal.snu.ac.kr Department of Transdisciplinary Studies Seoul National University, Korea Jan. 2016 Many slides are from Fei-Fei
Sajid Anwar, Kyuyeon Hwang and Wonyong Sung
Sajid Anwar, Kyuyeon Hwang and Wonyong Sung Department of Electrical and Computer Engineering Seoul National University Seoul, 08826 Korea Email: sajid@dsp.snu.ac.kr, khwang@dsp.snu.ac.kr, wysung@snu.ac.kr
Machine Learning for Physicists Lecture 1
Machine Learning for Physicists Lecture 1 Summer 2017 University of Erlangen-Nuremberg Florian Marquardt (Image generated by a net with 20 hidden layers) OUTPUT INPUT (Picture: Wikimedia Commons) OUTPUT
Convolutional Neural Networks. Srikumar Ramalingam
Convolutional Neural Networks Srikumar Ramalingam Reference Many of the slides are prepared using the following resources: neuralnetworksanddeeplearning.com (mainly Chapter 6) http://cs231n.github.io/convolutional-networks/
Jakub Hajic Artificial Intelligence Seminar I
Jakub Hajic Artificial Intelligence Seminar I. 11. 11. 2014 Outline Key concepts Deep Belief Networks Convolutional Neural Networks A couple of questions Convolution Perceptron Feedforward Neural Network
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
Pattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016/2017 Lessons 9 11 Jan 2017 Outline Artificial Neural networks Notation...2 Convolutional Neural Networks...3
Understanding CNNs using visualisation and transformation analysis
Understanding CNNs using visualisation and transformation analysis Andrea Vedaldi Medical Imaging Summer School August 2016 Image representations 2 encoder Φ representation An encoder maps the data into
UNSUPERVISED LEARNING
UNSUPERVISED LEARNING Topics Layer-wise (unsupervised) pre-training Restricted Boltzmann Machines Auto-encoders LAYER-WISE (UNSUPERVISED) PRE-TRAINING Breakthrough in 2006 Layer-wise (unsupervised) pre-training
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) Human Brain Neurons Input-Output Transformation Input Spikes Output Spike Spike (= a brief pulse) (Excitatory Post-Synaptic Potential)
Kai Yu NEC Laboratories America, Cupertino, California, USA
Kai Yu NEC Laboratories America, Cupertino, California, USA Joint work with Jinjun Wang, Fengjun Lv, Wei Xu, Yihong Gong Xi Zhou, Jianchao Yang, Thomas Huang, Tong Zhang Chen Wu NEC Laboratories America
Machine Learning for Signal Processing Neural Networks Continue. Instructor: Bhiksha Raj Slides by Najim Dehak 1 Dec 2016
Machine Learning for Signal Processing Neural Networks Continue Instructor: Bhiksha Raj Slides by Najim Dehak 1 Dec 2016 1 So what are neural networks?? Voice signal N.Net Transcription Image N.Net Text
Classification goals: Make 1 guess about the label (Top-1 error) Make 5 guesses about the label (Top-5 error) No Bounding Box
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton Motivation Classification goals: Make 1 guess about the label (Top-1 error) Make 5 guesses
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
Convolutional Neural Network Architecture
Convolutional Neural Network Architecture Zhisheng Zhong Feburary 2nd, 2018 Zhisheng Zhong Convolutional Neural Network Architecture Feburary 2nd, 2018 1 / 55 Outline 1 Introduction of Convolution Motivation
Loss Functions and Optimization. Lecture 3-1
Lecture 3: Loss Functions and Optimization Lecture 3-1 Administrative: Live Questions We ll use Zoom to take questions from remote students live-streaming the lecture Check Piazza for instructions and
Dimensionality Reduction and Principle Components Analysis
Dimensionality Reduction and Principle Components Analysis 1 Outline What is dimensionality reduction? Principle Components Analysis (PCA) Example (Bishop, ch 12) PCA vs linear regression PCA as a mixture
Convolutional Neural Networks
Convolutional Neural Networks Books» http://www.deeplearningbook.org/ Books http://neuralnetworksanddeeplearning.com/.org/ reviews» http://www.deeplearningbook.org/contents/linear_algebra.html» http://www.deeplearningbook.org/contents/prob.html»
Lecture 8: Introduction to Deep Learning: Part 2 (More on backpropagation, and ConvNets)
COS 402 Machine Learning and Artificial Intelligence Fall 2016 Lecture 8: Introduction to Deep Learning: Part 2 (More on backpropagation, and ConvNets) Sanjeev Arora Elad Hazan Recap: Structure of a deep
Analyzing the Performance of Multilayer Neural Networks for Object Recognition
Analyzing the Performance of Multilayer Neural Networks for Object Recognition Pulkit Agrawal, Ross Girshick, Jitendra Malik {pulkitag,rbg,malik}@eecs.berkeley.edu University of California Berkeley Supplementary
Course Structure. Psychology 452 Week 12: Deep Learning. Chapter 8 Discussion. Part I: Deep Learning: What and Why? Rufus. Rufus Processed By Fetch
Psychology 452 Week 12: Deep Learning What Is Deep Learning? Preliminary Ideas (that we already know!) The Restricted Boltzmann Machine (RBM) Many Layers of RBMs Pros and Cons of Deep Learning Course Structure
Convolutional Neural Networks II. Slides from Dr. Vlad Morariu
Convolutional Neural Networks II Slides from Dr. Vlad Morariu 1 Optimization Example of optimization progress while training a neural network. (Loss over mini-batches goes down over time.) 2 Learning rate
Deep learning on 3D geometries. Hope Yao Design Informatics Lab Department of Mechanical and Aerospace Engineering
Deep learning on 3D geometries Hope Yao Design Informatics Lab Department of Mechanical and Aerospace Engineering Overview Background Methods Numerical Result Future improvements Conclusion Background
LOCAL INTERPRETABLE MODEL-AGNOSTIC EXPLANATIONS FOR MUSIC CONTENT ANALYSIS
LOCAL INTERPRETABLE MODEL-AGNOSTIC EXPLANATIONS FOR MUSIC CONTENT ANALYSIS Saumitra Mishra, Bob L. Sturm, Simon Dixon Centre for Digital Music, Queen Mary University of London, United Kingdom { saumitra.mishra,
APPLIED DEEP LEARNING PROF ALEXIEI DINGLI
APPLIED DEEP LEARNING PROF ALEXIEI DINGLI TECH NEWS TECH NEWS HOW TO DO IT? TECH NEWS APPLICATIONS TECH NEWS TECH NEWS NEURAL NETWORKS Interconnected set of nodes and edges Designed to perform complex
Improved Local Coordinate Coding using Local Tangents
Improved Local Coordinate Coding using Local Tangents Kai Yu NEC Laboratories America, 10081 N. Wolfe Road, Cupertino, CA 95129 Tong Zhang Rutgers University, 110 Frelinghuysen Road, Piscataway, NJ 08854
Neural networks and optimization
Neural networks and optimization Nicolas Le Roux INRIA 8 Nov 2011 Nicolas Le Roux (INRIA) Neural networks and optimization 8 Nov 2011 1 / 80 1 Introduction 2 Linear classifier 3 Convolutional neural networks
Abstention Protocol for Accuracy and Speed
Abstention Protocol for Accuracy and Speed Abstract Amirata Ghorbani EE Dept. Stanford University amiratag@stanford.edu In order to confidently rely on machines to decide and perform tasks for us, there
Compressed Fisher vectors for LSVR
XRCE@ILSVRC2011 Compressed Fisher vectors for LSVR Florent Perronnin and Jorge Sánchez* Xerox Research Centre Europe (XRCE) *Now with CIII, Cordoba University, Argentina Our system in a nutshell High-dimensional
Neural Network Back-propagation HYUNG IL KOO
Neural Network Back-propagation HYUNG IL KOO Hidden Layer Representations Backpropagation has an ability to discover useful intermediate representations at the hidden unit layers inside the networks which
Stable and Efficient Representation Learning with Nonnegativity Constraints. Tsung-Han Lin and H.T. Kung
Stable and Efficient Representation Learning with Nonnegativity Constraints Tsung-Han Lin and H.T. Kung Unsupervised Representation Learning Layer 3 Representation Encoding Sparse encoder Layer 2 Representation
CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS
CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS LAST TIME Intro to cudnn Deep neural nets using cublas and cudnn TODAY Building a better model for image classification Overfitting
Regularization Introduction to Machine Learning. Matt Gormley Lecture 10 Feb. 19, 2018
1-61 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Regularization Matt Gormley Lecture 1 Feb. 19, 218 1 Reminders Homework 4: Logistic
Deep Learning for NLP
Deep Learning for NLP CS224N Christopher Manning (Many slides borrowed from ACL 2012/NAACL 2013 Tutorials by me, Richard Socher and Yoshua Bengio) Machine Learning and NLP NER WordNet Usually machine learning
Lecture 17: Neural Networks and Deep Learning
UVA CS 6316 / CS 4501-004 Machine Learning Fall 2016 Lecture 17: Neural Networks and Deep Learning Jack Lanchantin Dr. Yanjun Qi 1 Neurons 1-Layer Neural Network Multi-layer Neural Network Loss Functions
11/3/15. Deep Learning for NLP. Deep Learning and its Architectures. What is Deep Learning? Advantages of Deep Learning (Part 1)
11/3/15 Machine Learning and NLP Deep Learning for NLP Usually machine learning works well because of human-designed representations and input features CS224N WordNet SRL Parser Machine learning becomes
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning Lei Lei Ruoxuan Xiong December 16, 2017 1 Introduction Deep Neural Network
Introduction to Machine Learning Midterm Exam
10-701 Introduction to Machine Learning Midterm Exam Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes, but
arxiv: v2 [cs.sd] 7 Feb 2018
![arxiv: v2 [cs.sd] 7 Feb 2018](/thumbs/92/108747184.jpg "arxiv: v2 [cs.sd] 7 Feb 2018")
AUDIO SET CLASSIFICATION WITH ATTENTION MODEL: A PROBABILISTIC PERSPECTIVE Qiuqiang ong*, Yong Xu*, Wenwu Wang, Mark D. Plumbley Center for Vision, Speech and Signal Processing, University of Surrey, U
Neural Networks. David Rosenberg. July 26, New York University. David Rosenberg (New York University) DS-GA 1003 July 26, / 35
Neural Networks David Rosenberg New York University July 26, 2017 David Rosenberg (New York University) DS-GA 1003 July 26, 2017 1 / 35 Neural Networks Overview Objectives What are neural networks? How
Deep Learning Basics Lecture 7: Factor Analysis. Princeton University COS 495 Instructor: Yingyu Liang
Deep Learning Basics Lecture 7: Factor Analysis Princeton University COS 495 Instructor: Yingyu Liang Supervised v.s. Unsupervised Math formulation for supervised learning Given training data x i, y i
Learning and Memory in Neural Networks
Learning and Memory in Neural Networks Guy Billings, Neuroinformatics Doctoral Training Centre, The School of Informatics, The University of Edinburgh, UK. Neural networks consist of computational units
Making Deep Learning Understandable for Analyzing Sequential Data about Gene Regulation
Making Deep Learning Understandable for Analyzing Sequential Data about Gene Regulation Dr. Yanjun Qi Department of Computer Science University of Virginia Tutorial @ ACM BCB-2018 8/29/18 Yanjun Qi / UVA
Discriminative Direction for Kernel Classifiers
Discriminative Direction for Kernel Classifiers Polina Golland Artificial Intelligence Lab Massachusetts Institute of Technology Cambridge, MA 02139 polina@ai.mit.edu Abstract In many scientific and engineering
AUDIO SET CLASSIFICATION WITH ATTENTION MODEL: A PROBABILISTIC PERSPECTIVE. Qiuqiang Kong*, Yong Xu*, Wenwu Wang, Mark D. Plumbley
AUDIO SET CLASSIFICATION WITH ATTENTION MODEL: A PROBABILISTIC PERSPECTIVE Qiuqiang ong*, Yong Xu*, Wenwu Wang, Mark D. Plumbley Center for Vision, Speech and Signal Processing, University of Surrey, U
Domain adaptation for deep learning
What you saw is not what you get Domain adaptation for deep learning Kate Saenko Successes of Deep Learning in AI A Learning Advance in Artificial Intelligence Rivals Human Abilities Deep Learning for
Final Examination CS540-2: Introduction to Artificial Intelligence
Final Examination CS540-2: Introduction to Artificial Intelligence May 9, 2018 LAST NAME: SOLUTIONS FIRST NAME: Directions 1. This exam contains 33 questions worth a total of 100 points 2. Fill in your
Artificial Intelligence
Artificial Intelligence Jeff Clune Assistant Professor Evolving Artificial Intelligence Laboratory Announcements Be making progress on your projects! Three Types of Learning Unsupervised Supervised Reinforcement
RAGAV VENKATESAN VIJETHA GATUPALLI BAOXIN LI NEURAL DATASET GENERALITY
RAGAV VENKATESAN VIJETHA GATUPALLI BAOXIN LI NEURAL DATASET GENERALITY SIFT HOG ALL ABOUT THE FEATURES DAISY GABOR AlexNet GoogleNet CONVOLUTIONAL NEURAL NETWORKS VGG-19 ResNet FEATURES COMES FROM DATA
Course 10. Kernel methods. Classical and deep neural networks.
Course 10 Kernel methods. Classical and deep neural networks. Kernel methods in similarity-based learning Following (Ionescu, 2018) The Vector Space Model ò The representation of a set of objects as vectors
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
Deep Learning for Gravitational Wave Analysis Results with LIGO Data
Link to these slides: http://tiny.cc/nips arxiv:1711.03121 Deep Learning for Gravitational Wave Analysis Results with LIGO Data Daniel George & E. A. Huerta NCSA Gravity Group - http://gravity.ncsa.illinois.edu/
Deep Recurrent Neural Networks
Deep Recurrent Neural Networks Artem Chernodub e-mail: a.chernodub@gmail.com web: http://zzphoto.me ZZ Photo IMMSP NASU 2 / 28 Neuroscience Biological-inspired models Machine Learning p x y = p y x p(x)/p(y)
arxiv: v3 [cs.lg] 18 Mar 2013
![arxiv: v3 [cs.lg] 18 Mar 2013](/thumbs/72/67390553.jpg "arxiv: v3 [cs.lg] 18 Mar 2013")
Hierarchical Data Representation Model - Multi-layer NMF arxiv:1301.6316v3 [cs.lg] 18 Mar 2013 Hyun Ah Song Department of Electrical Engineering KAIST Daejeon, 305-701 hyunahsong@kaist.ac.kr Abstract Soo-Young
Speaker Representation and Verification Part II. by Vasileios Vasilakakis
Speaker Representation and Verification Part II by Vasileios Vasilakakis Outline -Approaches of Neural Networks in Speaker/Speech Recognition -Feed-Forward Neural Networks -Training with Back-propagation
Online Videos FERPA. Sign waiver or sit on the sides or in the back. Off camera question time before and after lecture. Questions?
Online Videos FERPA Sign waiver or sit on the sides or in the back Off camera question time before and after lecture Questions? Lecture 1, Slide 1 CS224d Deep NLP Lecture 4: Word Window Classification
Deep Learning. Hung-yi Lee 李宏毅
Deep Learning Hung-yi Lee 李宏毅 Deep learning attracts lots of attention. I believe you have seen lots of exciting results before. Deep learning trends at Google. Source: SIGMOD 206/Jeff Dean 958: Perceptron
CSC321 Lecture 16: ResNets and Attention
CSC321 Lecture 16: ResNets and Attention Roger Grosse Roger Grosse CSC321 Lecture 16: ResNets and Attention 1 / 24 Overview Two topics for today: Topic 1: Deep Residual Networks (ResNets) This is the state-of-the
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY 1 On-line Resources http://neuralnetworksanddeeplearning.com/index.html Online book by Michael Nielsen http://matlabtricks.com/post-5/3x3-convolution-kernelswith-online-demo
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
Neural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /9/17
3/9/7 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/9/7 Perceptron as a neural
Lecture 7 Convolutional Neural Networks
Lecture 7 Convolutional Neural Networks CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago April 17, 2017 We saw before: ŷ x 1 x 2 x 3 x 4 A series of matrix multiplications:
Cheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression
Neural Networks and Deep Learning
Neural Networks and Deep Learning Professor Ameet Talwalkar November 12, 2015 Professor Ameet Talwalkar Neural Networks and Deep Learning November 12, 2015 1 / 16 Outline 1 Review of last lecture AdaBoost
ECE 521. Lecture 11 (not on midterm material) 13 February K-means clustering, Dimensionality reduction
ECE 521 Lecture 11 (not on midterm material) 13 February 2017 K-means clustering, Dimensionality reduction With thanks to Ruslan Salakhutdinov for an earlier version of the slides Overview K-means clustering
Global Optimality in Matrix and Tensor Factorization, Deep Learning & Beyond
Global Optimality in Matrix and Tensor Factorization, Deep Learning & Beyond Ben Haeffele and René Vidal Center for Imaging Science Mathematical Institute for Data Science Johns Hopkins University This
Analysis of Feature Maps Selection in Supervised Learning Using Convolutional Neural Networks
Analysis of Selection in Supervised Learning Using Convolutional Neural Networks Joseph Lin Chu and Adam Krzyżak Department of Computer Science and Software Engineering, Concordia University, Montreal,
Theories of Deep Learning
Theories of Deep Learning Lecture 02 Donoho, Monajemi, Papyan Department of Statistics Stanford Oct. 4, 2017 1 / 50 Stats 385 Fall 2017 2 / 50 Stats 285 Fall 2017 3 / 50 Course info Wed 3:00-4:20 PM in
Deep Learning: Self-Taught Learning and Deep vs. Shallow Architectures. Lecture 04
Deep Learning: Self-Taught Learning and Deep vs. Shallow Architectures Lecture 04 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Self-Taught Learning 1. Learn
Agenda. Digit Classification using CNN Digit Classification using SAE Visualization: Class models, filters and saliency 2 DCT
versus 1 Agenda Deep Learning: Motivation Learning: Backpropagation Deep architectures I: Convolutional Neural Networks (CNN) Deep architectures II: Stacked Auto Encoders (SAE) Caffe Deep Learning Toolbox:
Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error
Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error Chunhui Jiang, Guiying Li, Chao Qian, Ke Tang Anhui Province Key Lab of Big Data Analysis and Application, University
Loss Functions and Optimization. Lecture 3-1
Lecture 3: Loss Functions and Optimization Lecture 3-1 Administrative Assignment 1 is released: http://cs231n.github.io/assignments2017/assignment1/ Due Thursday April 20, 11:59pm on Canvas (Extending