Administrative Issues. CS5242 mirror site:
|
|
|
- Nelson Harrington
- 6 years ago
- Views:
Transcription
1 Administrative Issues CS5242 mirror site: 1
2 Assignments (10% * 4 = 40%) For each assignment It has multiple sub-tasks, including coding tasks You need to submit the answers, code and execution results 2
3 Assignments (10% * 4 = 40%) Quiz (20%) Final Project (40%) 3
4 Final Projects (40%) Projects will be held on Kaggle-in-class Register using your nus ( The project webpages will be announced once groups are finalized No data disclosure No additional data Each group <=2 students Each group will be randomly assigned to A computer vision project A natural language processing project Assessment is based on the report (15%) and ranking (25%) Top ranked groups of each project will be invited to do presentation at week 13. Will announce in IVLE once Kaggle is ready 4
5 Final Project (40%) Report template (15%) Problem definition Highlights (new algorithms, insights from the experiments) Dataset pre-processing description Training and testing procedure Experimental study (clarity, model understanding, and highlights are important for the assessment) Ranking -> score (25%) Rank top 10%, score ~25 (max score) Rank top 20%, score ~21 Rank top 40%, score ~17 Rank top 70%, score ~14 Rank others, score ~13 Prediction scores less than 10% better than random guess, score ~5 Final rank to grade distribution may be adjusted depending on overall class performance 5
6 Final projects timeline summary Date 27 August 18:00 04 November 18:00 Event Register your project IVLE 2 or less students per group Results submission deadline 05 November 18:00 11 November 18:00 Report deadline 12 November 18:00 12 November 18:00 Selection and notification of selected projects for presentation Selected project group send slides to cs5242@comp.nus.edu.sg Make appointment to meet Hwee Kuan to do presentation rehearsal 16 November 18:30 Project presentations by students 6
7 GPU Machine/Cluster NSCC (National Super-Computing Center) Free for NUS students 128 GPU nodes, each with a NVIDIA Tesla K40 (12 GB) Follow the manual and test program on IVLE workbin/misc/gpu to get yourself familiar with it. Google Cloud Platform 300 USD credit for new register NVIDIA Tesla K80 (2x12GB) is available for Region Taiwan Amazon EC2 (g2.8xlarge) 100 USD credit for students NVIDIA Grid GPU, 1536 CUDA cores, 4GB memory Available for Singapore region Tips: STOP/TERMINATE the instance immediately after your program terminates Check the usage status frequently to avoid high outstanding bills Use Amazon or Google cloud platform for debugging and use NSCC for actual training and tuning 7
8 Lecture 1: Introduction Lecture 2: Neural Network Basics Lecture 3: Training the Neural Network Lecture 4: Convolutional Neural Network Lecture 5: Convolutional Neural Network Lecture 6: Convolutional Neural Network Lecture 7: Recurrent Neural Network Lecture 8: Recurrent Neural Network Lecture 9: Recurrent Neural Network Lecture 10: Advanced Topic: Generative Adversarial Network Lecture 11: Advanced Topic: One-shot learning Lecture 12: Advanced Topic: Distributed training Lecture 13: Project demonstrations 8
9 Forward Propagation 9
10 The network and information flow feed in data into first layer transmitter receiver final layer presents the output of network transmitter receiver 10
11 Notation Let x 2 R d be the input space y 2 R y 2 N Let or be the label Let o 2 R be the output of the neural network 11
12 o x 1 x 2 x 2 x 1 12
13 x =(x 1,x 2 ) 2 R 2 o = (w 1 x 1 + w 2 x 2 + b) x 1 x 2 A o (x1,x2,o) A x 2 o (x1,x2) x 1 x 2 A (x1,x2) A = (x1, x2, o) x 1 13
14 x =(x 1,x 2 ) 2 R 2 o = (w 1 x 1 + w 2 x 2 + b) x 1 x 2 o x 2 A o A x 1 x 1 14
15 x =(x 1,x 2 ) 2 R 2 o = (w 1 x 1 + w 2 x 2 + b) x 1 x 2 o x 2 o x 1 x 1 x 2 15
16 x 2 o x 1 x 2 x 2 x 1 level sets form straight lines 16 x 1
17 Concept of level sets 17
18 Next to simplest x =(x 1,x 2 ) 2 R 2 z = w 1 x 1 + w 2 x 2 + b o = (w 1 x 1 + w 2 x 2 + b) x 1 x 2 o = (z) x 1 o x 2 lines of constant z x 1 x 2 18
19 o x 1 x 2 These three surfaces add to form a triangular decision boundary! 19
20 Adding functions f(x) function 1 + function 2 (f+g)(x) function 1 a h a+b 2h + A x = g(x) function 2 A x b h A x 20
21 h1 h2 h3 x 2 o x 1 x 2 x 2 x 1 21 x 1
22 h1 h2 h3 h 1 (x 1,x 2 ) h 2 (x 1,x 2 ) h 3 (x 1,x 2 ) + + x 1 x 2 x 1 x 2 x 1 x 2 w 1 h 1 (x 1,x 2 )+w 2 h 2 (x 1,x 2 )+w 3 h 3 (x 1,x 2 )+b x 2 22 x 1
23 If you add up enough step surfaces, are you able to form any functions? 23
24 neural network fingers activities 24
25 Activity one Take out a piece of paper, draw patterns as shown X X X X X o o o o o o Task: Cut (tear) with one straight line to completely separate the X and O 25
26 X X X X X Activity two X X X X X X o o o o X X X XX X X X X X X X o oo o oo oo o o o o X X X X X X X X X X X X X X XX X X X X X Cut (tear) with one straight line! 26
27 X X X X X Activity two X X X X X X o o o o X X X XX X X X X X X X o oo o oo oo o o o o X X X X X X X X X X X X X X XX X X X X X Cut (tear) with one straight line! 27
28 Activity three X X X X X X X X X X X X X X X X o o o o o o oo o oo oo o o o o o o o o o o o oo oo o o o o X X X X X X X X X X X X X X Cut (tear) with one straight line! 28
29 Activity three X X X X X X X X X X X X X X X X o o o o o o oo o oo oo o o o o o o o o o o o oo oo o o o o X X X X X X X X X X X X X X Cut (tear) with one straight line! 29
30 Activity four X o X X o X X X X X o oo o o o X X X X X X X X X X X o o o o o X X X X X X X X XX X X Cut (tear) with one straight line! 30
31 Manifold view of neural network feed in data into first layer final layer presents the output of network data hidden layer output 31
32 Function view of neural network h 1 (x 1,x 2 ) h 2 (x 1,x 2 ) h 3 (x 1,x 2 ) + + x 1 x 2 x 2 x 1 x 1 x 2 x 2 x 1 32
33 x 1 x 2 w (1) 1 w (1) 2 b (1) h (1) Follow the data point x 2 A (x1,x2) (x1,x2,h1) A x 2 A (x1,x2) x 1 x 1 x 2 w (2) 1 w (2) 2 b (2) h (2) x 1 (x1,x2,h2) A x 2 A (x1,x2) (x A 1,x A 2 ) 7! (h A 1,h A 2 ) x 1 33
34 x 1 w (2) 1 w (1) 1 w (1) 2 h (1) b (2) x w (2) 2 2 h (2) b (1) x 2 A (x A 1,x A 2 ) h 2 A (h A 1,h A 2 ) x 1 h 1 (x A 1,x A 2 ) 7! (h A 1,h A 2 ) 34
35 x 1 w (2) 1 w (1) 1 w (1) 2 h (1) b (2) w (2) x 2 2 h (2) b (1) x 2 A h 2 A x 1 Neighbourhood relationship is conserved h 1 35
36 continuous mapping x 2 h 2 x 1 h 1 (x A 1,x A 2 ) 7! (h A 1,h A 2 ) 36
37 x 1 h 1 h 2 x 2 h 3 x 2 A h 3 A (x A 1,x A 2 ) (h A 1,h A 2,h A 3 ) h 2 x 1 h 1 37
38 x 1 h 1 h 2 x 2 h 3 x 2 x 1 38
39 x 1 w (2) 1 w (1) 1 b (2) x 2 h (2) w (2) 2 w (1) 2 b (1) h (1) w (3) 1 w (3) 2 x 2 o x 1 h 2 h 1 h (1) = (w (1) 1 x 1 + w (1) 2 x 2 + b (1) ) h (2) = (w (2) 1 x 1 + w (2) 2 x 2 + b (2) ) o = (w (3) 1 h(1) + w (3) 2 h(2) + b (3) ) 39 o
40 Some real life examples courtesy of our TA Connie Kou source code will be available online 40
41 The Angle Data The XOR Data The Ring Data 41
42 The Angle Data - ReLu 42
43 The Angle Data - ReLu 43
44 The Angle Data - ReLu 44
45 The Angle Data - ReLu 45
46 The Angle Data - ReLu 46
47 The Angle Data - ReLu 47
48 The Angle Data - ReLu 48
49 The Angle Data - ReLu 49
50 The Angle Data - ReLu 50
51 The Angle Data - ReLu 51
52 The Angle Data - ReLu 52
53 The Angle Data - Sigmoid 53
54 The Angle Data - Sigmoid 54
55 The Angle Data - Sigmoid 55
56 The Angle Data - Sigmoid 56
57 The Angle Data - Sigmoid 57
58 The Angle Data - Sigmoid 3 hidden nodes 58
59 The Angle Data - Sigmoid 59
60 The Angle Data - Sigmoid 60
61 The XOR Data - ReLU 61
62 The XOR Data - ReLU 62
63 The XOR Data - ReLU 63
64 The XOR Data - ReLU 64
65 The XOR Data - ReLU 65
66 The XOR Data - ReLU 66
67 The XOR Data - ReLU 67
68 The XOR Data - Sigmoid 68
69 The XOR Data - Sigmoid 69
70 The XOR Data - Sigmoid 70
71 The Ring Data - Sigmoid how the manifold is being fold? 71
72 The Ring Data - Sigmoid 72
73 73
74 74
75 75
76 76
77 77
78 78
79 79
80 80
81 81
82 Spiral 82
83 How about this network? Fully connected 83
84 Feed backward 84
85 A general objective Tweak the output of neural network to be as similar to the label as possible. o y How to tweak? Adjust the weights 85
86 Given an example (x, y) x o o! y For this example, the neural network behaves well 86
87 After the first example, give another example (x 0,?) x 0 o 0 Now we say for a well behaved neural network o 0 = y 0 87
88 How good is the trained network? x 0 o 0 88
89 Square error Given data points (x i,y i ),i=1, n l(y i,o i )=ky i o i k 2 C = 1 n X l(y i,o i ) i o 1,y 1 o 2,y 2... x n,x n 1, x 2,x 1 o n,y n Adjust the weights until C = 0 (or close to zero) 89
90 Cross entropy cost function Two class example y i 2 {0, 1} C = 1 n P i y i log[o(x i )] + (1 y i ) log[1 o(x i )] Three class example y i 2 {(1, 0, 0), (0, 1, 0), (0, 0, 1)} m =exp(v 1 )+exp(v 2 )+exp(v 3 ) o i =( exp(v 1) m, exp(v 2) m, exp(v 3) m ) l(y i,o i )= y i log(o i )... output v 1 v2 v 3 C = 1 n X i l(y i,o i ) 90
91 How to adjust the weights? Conceptually, this will work: Try all possible weights combinations, for all weights combinations, calculate the cost. Then pick the weight combination that has the lowest cost. Practically, there are too many weights combinations to try. 91
92 Gradient descend x w1 w2 w2 w3 v1 v2 v3 v 1 = (z 1 )= (w 1 x + b 1 ) v 2 = (z 2 )= (w 2 v 1 + b 2 ) v 3 = (z 3 )= (w 3 v 2 + b j =0 92
93 j = 1 n = 2 n X i X i o i ) j (y i o i j We just j w j (t + 1) = w j 93
94 x w1 w2 w2 w3 v1 v2 v3 = o v 1 = (z 1 )= (w 1 x + b 1 ) v 2 = (z 2 )= (w 2 v 1 + b 2 ) v 3 = (z 3 )= (w 3 v 2 + b = 0 (z 3 )v = 0 (z 3 )w 2 = 0 (z 3 )w 3 0 (z 2 )v = 0 (z 3 )w 2 = 0 (z 3 )w 3 0 (z 2 )w 1 = 0 (z 3 )w 3 0 (z 2 )w 2 0 (z
95 x w1 w2 w2 w3 v1 v2 v3 = = 0 (z 3 )v = 0 (z 3 )w 2 = 0 (z 3 )w 3 0 (z 2 )v = 0 (z 3 )w 2 = 0 (z 3 )w 3 0 (z 2 )w 1 = 0 (z 3 )w 3 0 (z 2 )w 2 0 (z 1 1 problem!! : long mathematical expression leads to large computational time for deep network 95
96 Compute and store strategy x w1 w2 w2 w3 v1 v2 v3 = 3 = 0 (z w 3 0 (z w 2 0 (z 1 ) 96
97 Compute and store strategy x w1 w2 w2 w3 v1 v2 v3 = v v x j? homework question
98 w2 3 v1 v3 w35 w01 x w14 w02 v2 w23 w24 v4 w45 v5 = (z 4 )w 45 98
99 w2 3 v1 v3 w35 w01 x w14 w02 v2 w23 w24 v4 w45 v5 = (z 2 )w (z 2 )w 23 99
100 @v 3 w2 w13 v1 v3 w35 w01 x w14 w02 v2 w23 w24 v4 w45 v5 = 4 This is call the back propagation algorithm 100
101 101
102 102
103 103
104 104
105 105
106 106
107 107
108 108
109 Cost function is a surface 109
110 x 1 w1 b=0 o l(y i,o i )=ky i o i k 2 x 2 w2 C(w 1,w 2 )= 1 n X i l(y i,o i )+0.2(w w 2 2) x 2 cost x w1 w2
111 cost w2 w1 w j (t + 1) = w j 111
112 How to use data set? 1.Train a network 2.Keep it for future use 3.When previously unseen data comes in without labels, use the network to predict There is a problem here! How can we know when we keep the network at step (2), it is good enough to fulfil the task of step (3)? We got to find some ways to test it before we use the network 112
113 Training and Validation set Given an annotated data set, (x i,y i ),i=1, n randomly sample x% of it and keep aside. Train the network on the remaining (100-x)% Validate the network with the x% of the data that you have not yet show to the network Network is ready to use if validation results is good. Else, start troubleshoot. 113
Machine Learning for Large-Scale Data Analysis and Decision Making A. Neural Networks Week #6
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Neural Networks Week #6 Today Neural Networks A. Modeling B. Fitting C. Deep neural networks Today s material is (adapted)
Introduction to Neural Networks
Introduction to Neural Networks Philipp Koehn 3 October 207 Linear Models We used before weighted linear combination of feature values h j and weights λ j score(λ, d i ) = j λ j h j (d i ) Such models
Neural Networks. Intro to AI Bert Huang Virginia Tech
Neural Networks Intro to AI Bert Huang Virginia Tech Outline Biological inspiration for artificial neural networks Linear vs. nonlinear functions Learning with neural networks: back propagation https://en.wikipedia.org/wiki/neuron#/media/file:chemical_synapse_schema_cropped.jpg
Introduction to Neural Networks
Introduction to Neural Networks Philipp Koehn 4 April 205 Linear Models We used before weighted linear combination of feature values h j and weights λ j score(λ, d i ) = j λ j h j (d i ) Such models can
Machine Learning. Boris
Machine Learning Boris Nadion boris@astrails.com @borisnadion @borisnadion boris@astrails.com astrails http://astrails.com awesome web and mobile apps since 2005 terms AI (artificial intelligence)
COMP 551 Applied Machine Learning Lecture 14: Neural Networks
COMP 551 Applied Machine Learning Lecture 14: Neural Networks Instructor: Ryan Lowe (ryan.lowe@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted,
INF 5860 Machine learning for image classification. Lecture 5 : Introduction to TensorFlow Tollef Jahren February 14, 2018
INF 5860 Machine learning for image classification Lecture 5 : Introduction to TensorFlow Tollef Jahren February 14, 2018 OUTLINE Deep learning frameworks TensorFlow TensorFlow graphs TensorFlow session
Midterm: CS 6375 Spring 2015 Solutions
Midterm: CS 6375 Spring 2015 Solutions The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run out of room for an
ECE521 Lectures 9 Fully Connected Neural Networks
ECE521 Lectures 9 Fully Connected Neural Networks Outline Multi-class classification Learning multi-layer neural networks 2 Measuring distance in probability space We learnt that the squared L2 distance
Deep Learning Lab Course 2017 (Deep Learning Practical)
Deep Learning Lab Course 207 (Deep Learning Practical) Labs: (Computer Vision) Thomas Brox, (Robotics) Wolfram Burgard, (Machine Learning) Frank Hutter, (Neurorobotics) Joschka Boedecker University of
Classification goals: Make 1 guess about the label (Top-1 error) Make 5 guesses about the label (Top-5 error) No Bounding Box
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton Motivation Classification goals: Make 1 guess about the label (Top-1 error) Make 5 guesses
<Special Topics in VLSI> Learning for Deep Neural Networks (Back-propagation)
Learning for Deep Neural Networks (Back-propagation) Outline Summary of Previous Standford Lecture Universal Approximation Theorem Inference vs Training Gradient Descent Back-Propagation
Deep Feedforward Networks. Sargur N. Srihari
Deep Feedforward Networks Sargur N. srihari@cedar.buffalo.edu 1 Topics Overview 1. Example: Learning XOR 2. Gradient-Based Learning 3. Hidden Units 4. Architecture Design 5. Backpropagation and Other Differentiation
Artificial neural networks
Artificial neural networks B. Mehlig, Department of Physics, University of Gothenburg, Sweden FFR35/FIM70 Artificial Neural Networks Chalmers/Gothenburg University, 7.5 credits 3 Course home page Teachers
Deep Learning (CNNs)
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Deep Learning (CNNs) Deep Learning Readings: Murphy 28 Bishop - - HTF - - Mitchell
Deep Feedforward Networks
Deep Feedforward Networks Yongjin Park 1 Goal of Feedforward Networks Deep Feedforward Networks are also called as Feedforward neural networks or Multilayer Perceptrons Their Goal: approximate some function
MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October,
MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October, 23 2013 The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run
What Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1
What Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1 Multi-layer networks Steve Renals Machine Learning Practical MLP Lecture 3 7 October 2015 MLP Lecture 3 Multi-layer networks 2 What Do Single
Multilayer Perceptrons and Backpropagation
Multilayer Perceptrons and Backpropagation Informatics 1 CG: Lecture 7 Chris Lucas School of Informatics University of Edinburgh January 31, 2017 (Slides adapted from Mirella Lapata s.) 1 / 33 Reading:
CS 570: Machine Learning Seminar. Fall 2016
CS 570: Machine Learning Seminar Fall 2016 Class Information Class web page: http://web.cecs.pdx.edu/~mm/mlseminar2016-2017/fall2016/ Class mailing list: cs570@cs.pdx.edu My office hours: T,Th, 2-3pm or
ECE521 Lecture 7/8. Logistic Regression
ECE521 Lecture 7/8 Logistic Regression Outline Logistic regression (Continue) A single neuron Learning neural networks Multi-class classification 2 Logistic regression The output of a logistic regression
ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD
ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD WHAT IS A NEURAL NETWORK? The simplest definition of a neural network, more properly referred to as an 'artificial' neural network (ANN), is provided
Comments. Assignment 3 code released. Thought questions 3 due this week. Mini-project: hopefully you have started. implement classification algorithms
Neural networks Comments Assignment 3 code released implement classification algorithms use kernels for census dataset Thought questions 3 due this week Mini-project: hopefully you have started 2 Example:
Neural Networks 2. 2 Receptive fields and dealing with image inputs
CS 446 Machine Learning Fall 2016 Oct 04, 2016 Neural Networks 2 Professor: Dan Roth Scribe: C. Cheng, C. Cervantes Overview Convolutional Neural Networks Recurrent Neural Networks 1 Introduction There
Logistic Regression. COMP 527 Danushka Bollegala
Logistic Regression COMP 527 Danushka Bollegala Binary Classification Given an instance x we must classify it to either positive (1) or negative (0) class We can use {1,-1} instead of {1,0} but we will
CSC321 Lecture 16: ResNets and Attention
CSC321 Lecture 16: ResNets and Attention Roger Grosse Roger Grosse CSC321 Lecture 16: ResNets and Attention 1 / 24 Overview Two topics for today: Topic 1: Deep Residual Networks (ResNets) This is the state-of-the
Neural Networks. David Rosenberg. July 26, New York University. David Rosenberg (New York University) DS-GA 1003 July 26, / 35
Neural Networks David Rosenberg New York University July 26, 2017 David Rosenberg (New York University) DS-GA 1003 July 26, 2017 1 / 35 Neural Networks Overview Objectives What are neural networks? How
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Matrix Data: Prediction Instructor: Yizhou Sun yzsun@ccs.neu.edu September 21, 2015 Announcements TA Monisha s office hour has changed to Thursdays 10-12pm, 462WVH (the same
Welcome to Physics 161 Elements of Physics Fall 2018, Sept 4. Wim Kloet
Welcome to Physics 161 Elements of Physics Fall 2018, Sept 4 Wim Kloet 1 Lecture 1 TOPICS Administration - course web page - contact details Course materials - text book - iclicker - syllabus Course Components
Neural Networks (Part 1) Goals for the lecture
Neural Networks (Part ) Mark Craven and David Page Computer Sciences 760 Spring 208 www.biostat.wisc.edu/~craven/cs760/ Some of the slides in these lectures have been adapted/borrowed from materials developed
Convolutional Neural Networks
Convolutional Neural Networks Books» http://www.deeplearningbook.org/ Books http://neuralnetworksanddeeplearning.com/.org/ reviews» http://www.deeplearningbook.org/contents/linear_algebra.html» http://www.deeplearningbook.org/contents/prob.html»
Introduction to Natural Computation. Lecture 9. Multilayer Perceptrons and Backpropagation. Peter Lewis
Introduction to Natural Computation Lecture 9 Multilayer Perceptrons and Backpropagation Peter Lewis 1 / 25 Overview of the Lecture Why multilayer perceptrons? Some applications of multilayer perceptrons.
text classification 3: neural networks
text classification 3: neural networks CS 585, Fall 2018 Introduction to Natural Language Processing http://people.cs.umass.edu/~miyyer/cs585/ Mohit Iyyer College of Information and Computer Sciences University
Deep Neural Networks (1) Hidden layers; Back-propagation
Deep Neural Networs (1) Hidden layers; Bac-propagation Steve Renals Machine Learning Practical MLP Lecture 3 4 October 2017 / 9 October 2017 MLP Lecture 3 Deep Neural Networs (1) 1 Recap: Softmax single
Classification of Higgs Boson Tau-Tau decays using GPU accelerated Neural Networks
Classification of Higgs Boson Tau-Tau decays using GPU accelerated Neural Networks Mohit Shridhar Stanford University mohits@stanford.edu, mohit@u.nus.edu Abstract In particle physics, Higgs Boson to tau-tau
CS249: ADVANCED DATA MINING
CS249: ADVANCED DATA MINING Support Vector Machine and Neural Network Instructor: Yizhou Sun yzsun@cs.ucla.edu April 24, 2017 Homework 1 Announcements Due end of the day of this Friday (11:59pm) Reminder
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY 1 On-line Resources http://neuralnetworksanddeeplearning.com/index.html Online book by Michael Nielsen http://matlabtricks.com/post-5/3x3-convolution-kernelswith-online-demo
Machine Learning (CS 567) Lecture 2
Machine Learning (CS 567) Lecture 2 Time: T-Th 5:00pm - 6:20pm Location: GFS118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 6
Machine Learning: Chenhao Tan University of Colorado Boulder LECTURE 6 Slides adapted from Jordan Boyd-Graber, Chris Ketelsen Machine Learning: Chenhao Tan Boulder 1 of 39 HW1 turned in HW2 released Office
10. Artificial Neural Networks
Foundations of Machine Learning CentraleSupélec Fall 217 1. Artificial Neural Networks Chloé-Agathe Azencot Centre for Computational Biology, Mines ParisTech chloe-agathe.azencott@mines-paristech.fr Learning
Week 5: The Backpropagation Algorithm
Week 5: The Backpropagation Algorithm Hans Georg Schaathun 25th April 2017 Time Topic Reading 8.15- Recap of tutorials last week Lecture: From perceptron to back-propagation Marsland Section 4 9.00- Tutorial
Logistic Regression. Will Monroe CS 109. Lecture Notes #22 August 14, 2017
1 Will Monroe CS 109 Logistic Regression Lecture Notes #22 August 14, 2017 Based on a chapter by Chris Piech Logistic regression is a classification algorithm1 that works by trying to learn a function
Final Examination CS 540-2: Introduction to Artificial Intelligence
Final Examination CS 540-2: Introduction to Artificial Intelligence May 7, 2017 LAST NAME: SOLUTIONS FIRST NAME: Problem Score Max Score 1 14 2 10 3 6 4 10 5 11 6 9 7 8 9 10 8 12 12 8 Total 100 1 of 11
Deep Feedforward Networks. Han Shao, Hou Pong Chan, and Hongyi Zhang
Deep Feedforward Networks Han Shao, Hou Pong Chan, and Hongyi Zhang Deep Feedforward Networks Goal: approximate some function f e.g., a classifier, maps input to a class y = f (x) x y Defines a mapping
Neural Networks with Applications to Vision and Language. Feedforward Networks. Marco Kuhlmann
Neural Networks with Applications to Vision and Language Feedforward Networks Marco Kuhlmann Feedforward networks Linear separability x 2 x 2 0 1 0 1 0 0 x 1 1 0 x 1 linearly separable not linearly separable
Neural Networks Learning the network: Backprop , Fall 2018 Lecture 4
Neural Networks Learning the network: Backprop 11-785, Fall 2018 Lecture 4 1 Recap: The MLP can represent any function The MLP can be constructed to represent anything But how do we construct it? 2 Recap:
Introduction to Machine Learning Spring 2018 Note Neural Networks
CS 189 Introduction to Machine Learning Spring 2018 Note 14 1 Neural Networks Neural networks are a class of compositional function approximators. They come in a variety of shapes and sizes. In this class,
Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels
Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels (possibly selecting from a set, i.e. kernel learning)
Machine Learning Lecture 12
Machine Learning Lecture 12 Neural Networks 30.11.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory Probability
CSCI567 Machine Learning (Fall 2018)
CSCI567 Machine Learning (Fall 2018) Prof. Haipeng Luo U of Southern California Sep 12, 2018 September 12, 2018 1 / 49 Administration GitHub repos are setup (ask TA Chi Zhang for any issues) HW 1 is due
(Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann
(Feed-Forward) Neural Networks 2016-12-06 Dr. Hajira Jabeen, Prof. Jens Lehmann Outline In the previous lectures we have learned about tensors and factorization methods. RESCAL is a bilinear model for
CS 6501: Deep Learning for Computer Graphics. Basics of Neural Networks. Connelly Barnes
CS 6501: Deep Learning for Computer Graphics Basics of Neural Networks Connelly Barnes Overview Simple neural networks Perceptron Feedforward neural networks Multilayer perceptron and properties Autoencoders
Artificial Neuron (Perceptron)
9/6/208 Gradient Descent (GD) Hantao Zhang Deep Learning with Python Reading: https://en.wikipedia.org/wiki/gradient_descent Artificial Neuron (Perceptron) = w T = w 0 0 + + w 2 2 + + w d d where
Lecture 3 Feedforward Networks and Backpropagation
Lecture 3 Feedforward Networks and Backpropagation CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago April 3, 2017 Things we will look at today Recap of Logistic Regression
Neural Networks. Yan Shao Department of Linguistics and Philology, Uppsala University 7 December 2016
Neural Networks Yan Shao Department of Linguistics and Philology, Uppsala University 7 December 2016 Outline Part 1 Introduction Feedforward Neural Networks Stochastic Gradient Descent Computational Graph
Backpropagation Introduction to Machine Learning. Matt Gormley Lecture 12 Feb 23, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Backpropagation Matt Gormley Lecture 12 Feb 23, 2018 1 Neural Networks Outline
Machine Learning Lecture 5
Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory
Online Videos FERPA. Sign waiver or sit on the sides or in the back. Off camera question time before and after lecture. Questions?
Online Videos FERPA Sign waiver or sit on the sides or in the back Off camera question time before and after lecture Questions? Lecture 1, Slide 1 CS224d Deep NLP Lecture 4: Word Window Classification
Intelligent Systems Discriminative Learning, Neural Networks
Intelligent Systems Discriminative Learning, Neural Networks Carsten Rother, Dmitrij Schlesinger WS2014/2015, Outline 1. Discriminative learning 2. Neurons and linear classifiers: 1) Perceptron-Algorithm
Natural Language Processing with Deep Learning CS224N/Ling284
Natural Language Processing with Deep Learning CS224N/Ling284 Lecture 4: Word Window Classification and Neural Networks Richard Socher Organization Main midterm: Feb 13 Alternative midterm: Friday Feb
Artificial Intelligence
Artificial Intelligence Jeff Clune Assistant Professor Evolving Artificial Intelligence Laboratory Announcements Be making progress on your projects! Three Types of Learning Unsupervised Supervised Reinforcement
Deep Feedforward Networks. Lecture slides for Chapter 6 of Deep Learning Ian Goodfellow Last updated
Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org Ian Goodfellow Last updated 2016-10-04 Roadmap Example: Learning XOR Gradient-Based Learning Hidden Units
CS Machine Learning Qualifying Exam
CS Machine Learning Qualifying Exam Georgia Institute of Technology March 30, 2017 The exam is divided into four areas: Core, Statistical Methods and Models, Learning Theory, and Decision Processes. There
COGS Q250 Fall Homework 7: Learning in Neural Networks Due: 9:00am, Friday 2nd November.
COGS Q250 Fall 2012 Homework 7: Learning in Neural Networks Due: 9:00am, Friday 2nd November. For the first two questions of the homework you will need to understand the learning algorithm using the delta
CS230: Lecture 9 Deep Reinforcement Learning
CS230: Lecture 9 Deep Reinforcement Learning Kian Katanforoosh Menti code: 21 90 15 Today s outline I. Motivation II. Recycling is good: an introduction to RL III. Deep Q-Learning IV. Application of Deep
CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS
CS 179: LECTURE 16 MODEL COMPLEXITY, REGULARIZATION, AND CONVOLUTIONAL NETS LAST TIME Intro to cudnn Deep neural nets using cublas and cudnn TODAY Building a better model for image classification Overfitting
Need for Deep Networks Perceptron. Can only model linear functions. Kernel Machines. Non-linearity provided by kernels
Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels (possibly selecting from a set, i.e. kernel learning)
Neural Networks in Structured Prediction. November 17, 2015
Neural Networks in Structured Prediction November 17, 2015 HWs and Paper Last homework is going to be posted soon Neural net NER tagging model This is a new structured model Paper - Thursday after Thanksgiving
CSC321 Lecture 9 Recurrent neural nets
CSC321 Lecture 9 Recurrent neural nets Roger Grosse and Nitish Srivastava February 3, 2015 Roger Grosse and Nitish Srivastava CSC321 Lecture 9 Recurrent neural nets February 3, 2015 1 / 20 Overview You
6.034 Quiz 3 5 November 2018
6.034 Quiz 3 5 November 2018 Name Email For 1 extra credit point: Circle the TA whose recitations you attend so that we can more easily enter your score in our records and return your quiz to you promptly.
Neural Networks Lecturer: J. Matas Authors: J. Matas, B. Flach, O. Drbohlav
Neural Networks 30.11.2015 Lecturer: J. Matas Authors: J. Matas, B. Flach, O. Drbohlav 1 Talk Outline Perceptron Combining neurons to a network Neural network, processing input to an output Learning Cost
Neural Networks and Deep Learning
Neural Networks and Deep Learning Professor Ameet Talwalkar November 12, 2015 Professor Ameet Talwalkar Neural Networks and Deep Learning November 12, 2015 1 / 16 Outline 1 Review of last lecture AdaBoost
Ch.6 Deep Feedforward Networks (2/3)
Ch.6 Deep Feedforward Networks (2/3) 16. 10. 17. (Mon.) System Software Lab., Dept. of Mechanical & Information Eng. Woonggy Kim 1 Contents 6.3. Hidden Units 6.3.1. Rectified Linear Units and Their Generalizations
CSE446: Neural Networks Spring Many slides are adapted from Carlos Guestrin and Luke Zettlemoyer
CSE446: Neural Networks Spring 2017 Many slides are adapted from Carlos Guestrin and Luke Zettlemoyer Human Neurons Switching time ~ 0.001 second Number of neurons 10 10 Connections per neuron 10 4-5 Scene
Lecture 3 Feedforward Networks and Backpropagation
Lecture 3 Feedforward Networks and Backpropagation CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago April 3, 2017 Things we will look at today Recap of Logistic Regression
Deep Neural Networks (1) Hidden layers; Back-propagation
Deep Neural Networs (1) Hidden layers; Bac-propagation Steve Renals Machine Learning Practical MLP Lecture 3 2 October 2018 http://www.inf.ed.ac.u/teaching/courses/mlp/ MLP Lecture 3 / 2 October 2018 Deep
Neural Networks. Nicholas Ruozzi University of Texas at Dallas
Neural Networks Nicholas Ruozzi University of Texas at Dallas Handwritten Digit Recognition Given a collection of handwritten digits and their corresponding labels, we d like to be able to correctly classify
CS 4700: Foundations of Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman selman@cs.cornell.edu Machine Learning: Neural Networks R&N 18.7 Intro & perceptron learning 1 2 Neuron: How the brain works # neurons
Lecture 5 Neural models for NLP
CS546: Machine Learning in NLP (Spring 2018) http://courses.engr.illinois.edu/cs546/ Lecture 5 Neural models for NLP Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Office hours: Tue/Thu 2pm-3pm
Chemistry 330 Fall 2015 Organic Chemistry I
Chemistry 330 Fall 2015 Organic Chemistry I Instructor: John G. Kodet Contact Information: Office: Faraday Hall 335 Email: jkodet@niu.edu Office Hours: MW 2:00-3:00 pm, and by appointment Lecture: MWF
Deep Learning: a gentle introduction
Deep Learning: a gentle introduction Jamal Atif jamal.atif@dauphine.fr PSL, Université Paris-Dauphine, LAMSADE February 8, 206 Jamal Atif (Université Paris-Dauphine) Deep Learning February 8, 206 / Why
Machine Learning (CS 567) Lecture 3
Machine Learning (CS 567) Lecture 3 Time: T-Th 5:00pm - 6:20pm Location: GFS 118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol
CSC321 Lecture 5: Multilayer Perceptrons
CSC321 Lecture 5: Multilayer Perceptrons Roger Grosse Roger Grosse CSC321 Lecture 5: Multilayer Perceptrons 1 / 21 Overview Recall the simple neuron-like unit: y output output bias i'th weight w 1 w2 w3
8-1: Backpropagation Prof. J.C. Kao, UCLA. Backpropagation. Chain rule for the derivatives Backpropagation graphs Examples
8-1: Backpropagation Prof. J.C. Kao, UCLA Backpropagation Chain rule for the derivatives Backpropagation graphs Examples 8-2: Backpropagation Prof. J.C. Kao, UCLA Motivation for backpropagation To do gradient
Machine Learning, Fall 2011: Homework 5
10-601 Machine Learning, Fall 2011: Homework 5 Machine Learning Department Carnegie Mellon University Due:???????????, 5pm Instructions There are???? questions on this assignment. The?????? question involves
Artificial Neural Networks. Introduction to Computational Neuroscience Tambet Matiisen
Artificial Neural Networks Introduction to Computational Neuroscience Tambet Matiisen 2.04.2018 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
Topics in AI (CPSC 532L): Multimodal Learning with Vision, Language and Sound. Lecture 3: Introduction to Deep Learning (continued)
Topics in AI (CPSC 532L): Multimodal Learning with Vision, Language and Sound Lecture 3: Introduction to Deep Learning (continued) Course Logistics - Update on course registrations - 6 seats left now -
Long-Short Term Memory and Other Gated RNNs
Long-Short Term Memory and Other Gated RNNs Sargur Srihari srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Sequence Modeling
Lecture 4 Backpropagation
Lecture 4 Backpropagation CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago April 5, 2017 Things we will look at today More Backpropagation Still more backpropagation Quiz
Deep Learning. Convolutional Neural Networks Applications
Deep Learning Using a Convolutional Neural Network Dr. Ing. Morris Riedel Adjunct Associated Professor School of Engineering and Natural Sciences, University of Iceland Research Group Leader, Juelich Supercomputing
Backpropagation and Neural Networks part 1. Lecture 4-1
Lecture 4: Backpropagation and Neural Networks part 1 Lecture 4-1 Administrative A1 is due Jan 20 (Wednesday). ~150 hours left Warning: Jan 18 (Monday) is Holiday (no class/office hours) Also note: Lectures
ARTIFICIAL INTELLIGENCE. Artificial Neural Networks
INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Artificial Neural Networks Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html
Statistical NLP for the Web
Statistical NLP for the Web Neural Networks, Deep Belief Networks Sameer Maskey Week 8, October 24, 2012 *some slides from Andrew Rosenberg Announcements Please ask HW2 related questions in courseworks
Value Function Methods. CS : Deep Reinforcement Learning Sergey Levine
Value Function Methods CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 2 is due in one week 2. Remember to start forming final project groups and writing your proposal! Proposal
Decision Trees: Overfitting
Decision Trees: Overfitting Emily Fox University of Washington January 30, 2017 Decision tree recap Loan status: Root 22 18 poor 4 14 Credit? Income? excellent 9 0 3 years 0 4 Fair 9 4 Term? 5 years 9
From perceptrons to word embeddings. Simon Šuster University of Groningen
From perceptrons to word embeddings Simon Šuster University of Groningen Outline A basic computational unit Weighting some input to produce an output: classification Perceptron Classify tweets Written
Introduction Biologically Motivated Crude Model Backpropagation
Introduction Biologically Motivated Crude Model Backpropagation 1 McCulloch-Pitts Neurons In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published A logical calculus of the
Lecture 8: Introduction to Deep Learning: Part 2 (More on backpropagation, and ConvNets)
COS 402 Machine Learning and Artificial Intelligence Fall 2016 Lecture 8: Introduction to Deep Learning: Part 2 (More on backpropagation, and ConvNets) Sanjeev Arora Elad Hazan Recap: Structure of a deep
Grundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Neural networks Daniel Hennes 21.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Logistic regression Neural networks Perceptron