Deep Learning with Coherent Nanophotonic Circuits
|
|
|
- Nicholas Wells
- 5 years ago
- Views:
Transcription
1 Yichen Shen, Nicholas Harris, Dirk Englund, Marin Soljacic Massachusetts Institute of Berkeley, Oct
2 Neuromorphic Computing Biological Neural Networks Artificial Neural Networks 2
3 Artificial Neural Networks (ANN) Breakthroughs in deep learning: Natural Language Processing (NLP) Game Playing (Go, Atari) Autonomous Vehicles Control Ad Placement Researches (drug discovery, material study) Etc. 3
4 Basic Algorithm of ANN Matrix Multiplication: Nonlinear Activation: 4
5 Hardware and Data Enable Deep Learning 5
6 The Need for Speed More Data Bigger Models More need for Computation But Moore s Law is no-longer providing more computation The Market: On clouds: Millions of high power AI processors ($10,000 each) in data centers by 2020 Von Neumann ASIC/FPGA Optical Processing On premise: Billions of compact AI processors needed due to the rise of autonomouse driving, AR and IoT. 6
7 Optical AI Computing 7
8 In Deep Learning Key Operation is dense M x V In Optics, Matrix Multiplication is very common & (usually) consumes no energy! Convolution / FFT b i = W ij x a j Matrix Multiplication 8
9 Programmable Nanophotonic Processors a 100µm c Voltage 2 Transmission Waveguide Phase Shifter b O p t i c a l I n t e r f e r e n c e U n i t ( O I U ) i φ i MZI Detectors Input Modes S U ( 4 ) C o r e D M M C J. Mower et al, Physical Reviews A, 92, (2015) Carolan, Jacques, et al. "Universal linear optics." Science (2015):
10 ANN does NOT require high resolution Sze et al, arxiv: (2017) 10
11 Deep Learning Inference is Passive Once the Optical Neural Network is trained, no need to update the weights frequently 11
12 Deep Learning is very parallelizable Multiple wavelengths can be used to simultaneously execute batch of data 12
13 Coherent Optical Neural Networks (ONN) a Z (1) = W 0 X h (i ) = f (Z (i ) ) Y = W n h (n ) b Optical Input Optical Output X 1 h 1 (1) h 1 (i) h 1 (n) Y 1 X Layer 1 Layer i Layer n Y X 2 h 2 (1) h 2 (i) h 2 (n) Y 2 c X 3 h 3 (1) h 3 (i) h 3 (n) Y 3 x in V (n) 0 S (n) 0 U (n) 0 f NL 0 x out X 4 h 4 (1) h 4 (i) h 4 (n) Y 4 Waveguide Optical Interference Unit Optical Nonlinearity Unit Input Layer Hidden Layers Output Layer d f NL () f NL () Vowel X M (1) = U (1) S (1) V (1) M (2) (n 1) M M (n) Photonic Integrated Circuit 13
14 Optical Vowel Recognition (4d 4 classes) 14
15 Instance a Instance d V 2 Dq Simulated Optical Nonlinearity Input b OIU 1 OIU 2 CPU OIU 3 OIU 4 U 1 S 1 V 1 f SA ( I in ) U 1 S 1 V 1 Laser OIU Detectors Computer Output Transmission Dq 18µm Df Phase Shifter c Optical Interference Unit Directional Coupler 60µm SU(4) Core DMMC Y. Shen and N. Harris et al, Nature Photonics, 11, (2017) 15
16 Vowel Spoken Vowel Spoken Experimental Result Vowel Identified Vowel Identified Simulation Result: 165/180=91.7% Experiment Result: 138/180=76.7% 16
17 Fully Connected Neural Networks Recurrent Neural Networks Convolutional Neural Networks 17
18 Recurrent Neural Networks Commonly used for Speech Recognition and Language Processing 18
19 Convolution Neural Networks Scott Skirlo and Yichen Shen et al, Manuscript in Preparation 19
20 Optical Convolutional Neural Network Scott Skirlo and Yichen Shen et al, Manuscript in Preparation 20
21 Unified Buffer (local Storage SRAM) ADC arrays *modified block diagram from TPU architecture 21
22 Speed and Energy Efficiency Comparison with Electrical ANN NVIDIA TITAN X ONN (withthermal PS) Architecture Von Neumann Neuromorphic Power Consumption 1 kw 1-2 kw Operation Speed 10 TFLOP 10,000 TFLOP Y. Shen and N. Harris et al, Nature Photonics 11, 441 (2017) 22
23 23
24 Some History on Optical Neural Networks 24
25 Acknowledgement Nicholas Harris PhD, EECS MIT Scott Skirlo PhD, Physics MIT Li Jing PhD, Physics MIT Prof. Dirk Englund EECS, MIT Prof. Marin Soljacic Physics, MIT 25
26 Optical Convolutional Neural Network Scott Skirlo and Yichen Shen et al, Manuscript in Preparation 26
27 Nonlinearity For Deep Learning, the constraint on nonlinearity is weak I in Nonlinear I out =f(i in ) Photonic System Saturable Absorption Photodiode A. Selden, British Journal of Applied Physics 18, 743 (1967) M. Soljacic, Physical Review E 66, (2002) Z. Cheng et al, IEEE Journal of Selected Topics in Quantum Electronics 20.1 (2014):
Quantum Photonic Integrated Circuits
Quantum Photonic Integrated Circuits IHFG Hauptseminar: Nanooptik und Nanophotonik Supervisor: Prof. Dr. Peter Michler 14.07.2016 Motivation and Contents 1 Quantum Computer Basics and Materials Photon
Quantum Artificial Intelligence and Machine Learning: The Path to Enterprise Deployments. Randall Correll. +1 (703) Palo Alto, CA
Quantum Artificial Intelligence and Machine : The Path to Enterprise Deployments Randall Correll randall.correll@qcware.com +1 (703) 867-2395 Palo Alto, CA 1 Bundled software and services Professional
Post Von Neumann Computing
Post Von Neumann Computing Matthias Kaiserswerth Hasler Stiftung (formerly IBM Research) 1 2014 IBM Corporation Foundation Purpose Support information and communication technologies (ICT) to advance Switzerland
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) Human Brain Neurons Input-Output Transformation Input Spikes Output Spike Spike (= a brief pulse) (Excitatory Post-Synaptic Potential)
Synaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics
Synaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics Byung-Gook Park Inter-university Semiconductor Research Center & Department of Electrical and Computer Engineering Seoul National
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
AN INTRODUCTION TO NEURAL NETWORKS. Scott Kuindersma November 12, 2009
AN INTRODUCTION TO NEURAL NETWORKS Scott Kuindersma November 12, 2009 SUPERVISED LEARNING We are given some training data: We must learn a function If y is discrete, we call it classification If it is
Featured Articles Advanced Research into AI Ising Computer
156 Hitachi Review Vol. 65 (2016), No. 6 Featured Articles Advanced Research into AI Ising Computer Masanao Yamaoka, Ph.D. Chihiro Yoshimura Masato Hayashi Takuya Okuyama Hidetaka Aoki Hiroyuki Mizuno,
Arbitrary and reconfigurable optics - new opportunities for integrated photonics
Arbitrary and reconfigurable optics - new opportunities for integrated photonics David Miller, Stanford University For a copy of these slides, please e-mail dabm@ee.stanford.edu How to design any linear
Why DNN Works for Acoustic Modeling in Speech Recognition?
Why DNN Works for Acoustic Modeling in Speech Recognition? Prof. Hui Jiang Department of Computer Science and Engineering York University, Toronto, Ont. M3J 1P3, CANADA Joint work with Y. Bao, J. Pan,
Keywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm
Volume 4, Issue 5, May 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Huffman Encoding
Speech and Language Processing
Speech and Language Processing Lecture 5 Neural network based acoustic and language models Information and Communications Engineering Course Takahiro Shinoaki 08//6 Lecture Plan (Shinoaki s part) I gives
Machine Learning
Machine Learning 10-315 Maria Florina Balcan Machine Learning Department Carnegie Mellon University 03/29/2019 Today: Artificial neural networks Backpropagation Reading: Mitchell: Chapter 4 Bishop: Chapter
How can ideas from quantum computing improve or speed up neuromorphic models of computation?
Neuromorphic Computation: Architectures, Models, Applications Associative Memory Models with Adiabatic Quantum Optimization Kathleen Hamilton, Alexander McCaskey, Jonathan Schrock, Neena Imam and Travis
Artificial Neural Networks. Introduction to Computational Neuroscience Tambet Matiisen
Artificial Neural Networks Introduction to Computational Neuroscience Tambet Matiisen 2.04.2018 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
Approximate Q-Learning. Dan Weld / University of Washington
Approximate Q-Learning Dan Weld / University of Washington [Many slides taken from Dan Klein and Pieter Abbeel / CS188 Intro to AI at UC Berkeley materials available at http://ai.berkeley.edu.] Q Learning
Deep Learning for Gravitational Wave Analysis Results with LIGO Data
Link to these slides: http://tiny.cc/nips arxiv:1711.03121 Deep Learning for Gravitational Wave Analysis Results with LIGO Data Daniel George & E. A. Huerta NCSA Gravity Group - http://gravity.ncsa.illinois.edu/
Neural Network Based Density Measurement
Bulg. J. Phys. 31 (2004) 163 169 P. Neelamegam 1, A. Rajendran 2 1 PG and Research Department of Physics, AVVM Sri Pushpam College (Autonomous), Poondi, Thanjavur, Tamil Nadu-613 503, India 2 PG and Research
SP-CNN: A Scalable and Programmable CNN-based Accelerator. Dilan Manatunga Dr. Hyesoon Kim Dr. Saibal Mukhopadhyay
SP-CNN: A Scalable and Programmable CNN-based Accelerator Dilan Manatunga Dr. Hyesoon Kim Dr. Saibal Mukhopadhyay Motivation Power is a first-order design constraint, especially for embedded devices. Certain
On-Chip Quantum Nanophotonics: Challenges and Perspectives
On-Chip Quantum Nanophotonics: Challenges and Perspectives Vladimir M. Shalaev Purdue Quantum Center, Purdue University West Lafayette, IN, USA WHY QUANTUM PHOTONICS? Transformative impact: NEXT TECHNOLOGY
Optically-controlled controlled quantum dot spins for quantum computers
Optically-controlled controlled quantum dot spins for quantum computers David Press Yamamoto Group Applied Physics Department Ph.D. Oral Examination April 28, 2010 1 What could a Quantum Computer do? Simulating
Neuromorphic computing with Memristive devices. NCM group
Neuromorphic computing with Memristive devices NCM group Why neuromorphic? New needs for computing Recognition, Mining, Synthesis (Intel) Increase of Fault (nanoscale engineering) SEMICONDUCTOR TECHNOLOGY
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY 1 On-line Resources http://neuralnetworksanddeeplearning.com/index.html Online book by Michael Nielsen http://matlabtricks.com/post-5/3x3-convolution-kernelswith-online-demo
NEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES
NEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES KAUSHIK ROY ABHRONIL SENGUPTA, KARTHIK YOGENDRA, DELIANG FAN, SYED SARWAR, PRIYA PANDA, GOPAL SRINIVASAN, JASON
Quantum Effect or HPC without FLOPS. Lugano March 23, 2016
Quantum Effect or HPC without FLOPS Lugano March 23, 2016 Electronics April 19, 1965 2016 D-Wave Systems Inc. All Rights Reserved 2 Moore s Law 2016 D-Wave Systems Inc. All Rights Reserved 3 www.economist.com/technology-quarterly/2016-03-12/aftermoores-law
Hardware Acceleration of DNNs
Lecture 12: Hardware Acceleration of DNNs Visual omputing Systems Stanford S348V, Winter 2018 Hardware acceleration for DNNs Huawei Kirin NPU Google TPU: Apple Neural Engine Intel Lake rest Deep Learning
Lecture 14 Dispersion engineering part 1 - Introduction. EECS Winter 2006 Nanophotonics and Nano-scale Fabrication P.C.Ku
Lecture 14 Dispersion engineering part 1 - Introduction EEC 598-2 Winter 26 Nanophotonics and Nano-scale Fabrication P.C.Ku chedule for the rest of the semester Introduction to light-matter interaction
Introduction to Deep Learning CMPT 733. Steven Bergner
Introduction to Deep Learning CMPT 733 Steven Bergner Overview Renaissance of artificial neural networks Representation learning vs feature engineering Background Linear Algebra, Optimization Regularization
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning Lei Lei Ruoxuan Xiong December 16, 2017 1 Introduction Deep Neural Network
Deep Feedforward Networks. Sargur N. Srihari
Deep Feedforward Networks Sargur N. srihari@cedar.buffalo.edu 1 Topics Overview 1. Example: Learning XOR 2. Gradient-Based Learning 3. Hidden Units 4. Architecture Design 5. Backpropagation and Other Differentiation
Intelligent Systems: Reasoning and Recognition. Artificial Neural Networks
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley MOSIG M1 Winter Seester 2018 Lesson 7 1 March 2018 Outline Artificial Neural Networks Notation...2 Introduction...3 Key Equations... 3 Artificial
Artificial Neural Networks D B M G. Data Base and Data Mining Group of Politecnico di Torino. Elena Baralis. Politecnico di Torino
Artificial Neural Networks Data Base and Data Mining Group of Politecnico di Torino Elena Baralis Politecnico di Torino Artificial Neural Networks Inspired to the structure of the human brain Neurons as
Processor & SOC Architecture
Processor & SOC Architecture - Processor & Accelerator Design (Front-End) Neural Processor, DSP, GPU, CPU, MCU Design Accelerator, Multimedia Processor Design SOC architecture, C-modeling Memory system
A 68 Parallel Row Access Neuromorphic Core with 22K Multi-Level Synapses Based on Logic- Compatible Embedded Flash Memory Technology
A 68 Parallel Row Access Neuromorphic Core with 22K Multi-Level Synapses Based on Logic- Compatible Embedded Flash Memory Technology M. Kim 1, J. Kim 1, G. Park 1, L. Everson 1, H. Kim 1, S. Song 1,2,
Deep Learning. What Is Deep Learning? The Rise of Deep Learning. Long History (in Hind Sight)
CSCE 636 Neural Networks Instructor: Yoonsuck Choe Deep Learning What Is Deep Learning? Learning higher level abstractions/representations from data. Motivation: how the brain represents sensory information
Some Applications of Machine Learning to Astronomy. Eduardo Bezerra 20/fev/2018
Some Applications of Machine Learning to Astronomy Eduardo Bezerra ebezerra@cefet-rj.br 20/fev/2018 Overview 2 Introduction Definition Neural Nets Applications do Astronomy Ads: Machine Learning Course
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers Thomas E. Potok, Ph.D. Computational Data Analytics Group Oak Ridge National Laboratory ORNL is managed
Part 8: Neural Networks
METU Informatics Institute Min720 Pattern Classification ith Bio-Medical Applications Part 8: Neural Netors - INTRODUCTION: BIOLOGICAL VS. ARTIFICIAL Biological Neural Netors A Neuron: - A nerve cell as
COMP-4360 Machine Learning Neural Networks
COMP-4360 Machine Learning Neural Networks Jacky Baltes Autonomous Agents Lab University of Manitoba Winnipeg, Canada R3T 2N2 Email: jacky@cs.umanitoba.ca WWW: http://www.cs.umanitoba.ca/~jacky http://aalab.cs.umanitoba.ca
An Approximate Parallel Multiplier with Deterministic Errors for Ultra-High Speed Integrated Optical Circuits
An Approximate Parallel Multiplier with Deterministic Errors for Ultra-High Speed Integrated Optical Circuits Jun Shiomi 1, Tohru Ishihara 1, Hidetoshi Onodera 1, Akihiko Shinya 2, Masaya Notomi 2 1 Graduate
Neural Networks. Yan Shao Department of Linguistics and Philology, Uppsala University 7 December 2016
Neural Networks Yan Shao Department of Linguistics and Philology, Uppsala University 7 December 2016 Outline Part 1 Introduction Feedforward Neural Networks Stochastic Gradient Descent Computational Graph
<Special Topics in VLSI> Learning for Deep Neural Networks (Back-propagation)
Learning for Deep Neural Networks (Back-propagation) Outline Summary of Previous Standford Lecture Universal Approximation Theorem Inference vs Training Gradient Descent Back-Propagation
In silico generation of novel, drug-like chemical matter using the LSTM deep neural network
In silico generation of novel, drug-like chemical matter using the LSTM deep neural network Peter Ertl Novartis Institutes for BioMedical Research, Basel, CH September 2018 Neural networks in cheminformatics
Artificial Neural Networks
Artificial Neural Networks Threshold units Gradient descent Multilayer networks Backpropagation Hidden layer representations Example: Face Recognition Advanced topics 1 Connectionist Models Consider humans:
Introduction Biologically Motivated Crude Model Backpropagation
Introduction Biologically Motivated Crude Model Backpropagation 1 McCulloch-Pitts Neurons In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published A logical calculus of the
Deep Feedforward Networks
Deep Feedforward Networks Yongjin Park 1 Goal of Feedforward Networks Deep Feedforward Networks are also called as Feedforward neural networks or Multilayer Perceptrons Their Goal: approximate some function
Towards Fully-automated Driving
Towards Fully-automated Driving Challenges and Potential Solutions Dr. Gijs Dubbelman Mobile Perception Systems EE-SPS/VCA Mobile Perception Systems 6 PhDs, postdoc, project manager, software engineer,
CS 4700: Foundations of Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman selman@cs.cornell.edu Machine Learning: Neural Networks R&N 18.7 Intro & perceptron learning 1 2 Neuron: How the brain works # neurons
Deep learning on 3D geometries. Hope Yao Design Informatics Lab Department of Mechanical and Aerospace Engineering
Deep learning on 3D geometries Hope Yao Design Informatics Lab Department of Mechanical and Aerospace Engineering Overview Background Methods Numerical Result Future improvements Conclusion Background
Deep Recurrent Neural Networks
Deep Recurrent Neural Networks Artem Chernodub e-mail: a.chernodub@gmail.com web: http://zzphoto.me ZZ Photo IMMSP NASU 2 / 28 Neuroscience Biological-inspired models Machine Learning p x y = p y x p(x)/p(y)
CMSC 421: Neural Computation. Applications of Neural Networks
CMSC 42: Neural Computation definition synonyms neural networks artificial neural networks neural modeling connectionist models parallel distributed processing AI perspective Applications of Neural Networks
The N3XT Technology for. Brain-Inspired Computing
The N3XT Technology for Brain-Inspired Computing SystemX Alliance 27..8 Department of Electrical Engineering 25.4.5 2 25.4.5 Source: Google 3 25.4.5 Source: vrworld.com 4 25.4.5 Source: BDC Stanford Magazine
RE-ENGINEERING COMPUTING WITH NEURO- MIMETIC DEVICES, CIRCUITS, AND ALGORITHMS
RE-ENGINEERING COMPUTING WITH NEURO- MIMETIC DEVICES, CIRCUITS, AND ALGORITHMS Kaushik Roy Abhronil Sengupta, Gopal Srinivasan, Aayush Ankit, Priya Panda, Xuanyao Fong, Deliang Fan, Jason Allred School
Machine Learning for Large-Scale Data Analysis and Decision Making A. Neural Networks Week #6
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Neural Networks Week #6 Today Neural Networks A. Modeling B. Fitting C. Deep neural networks Today s material is (adapted)
Chapter 4 Dynamic Bayesian Networks Fall Jin Gu, Michael Zhang
Chapter 4 Dynamic Bayesian Networks 2016 Fall Jin Gu, Michael Zhang Reviews: BN Representation Basic steps for BN representations Define variables Define the preliminary relations between variables Check
Deep Reinforcement Learning SISL. Jeremy Morton (jmorton2) November 7, Stanford Intelligent Systems Laboratory
Deep Reinforcement Learning Jeremy Morton (jmorton2) November 7, 2016 SISL Stanford Intelligent Systems Laboratory Overview 2 1 Motivation 2 Neural Networks 3 Deep Reinforcement Learning 4 Deep Learning
self-driving car technology introduction
self-driving car technology introduction slide 1 Contents of this presentation 1. Motivation 2. Methods 2.1 road lane detection 2.2 collision avoidance 3. Summary 4. Future work slide 2 Motivation slide
IBM Systems for Cognitive Solutions
IBM Q Quantum Computing IBM Systems for Cognitive Solutions Ehningen 12 th of July 2017 Albert Frisch, PhD - albert.frisch@de.ibm.com 2017 IBM 1 st wave of Quantum Revolution lasers atomic clocks GPS sensors
Deep generative models for fast shower simulation in ATLAS
Deep generative models for fast shower simulation in ATLAS Dalila Salamani University of Geneva dalila.salamani@cern.ch On behalf of the ATLAS collaboration 14th e-science IEEE International Conference
Convolutional Neural Networks
Convolutional Neural Networks Books» http://www.deeplearningbook.org/ Books http://neuralnetworksanddeeplearning.com/.org/ reviews» http://www.deeplearningbook.org/contents/linear_algebra.html» http://www.deeplearningbook.org/contents/prob.html»
Hidden Markov Models Hamid R. Rabiee
Hidden Markov Models Hamid R. Rabiee 1 Hidden Markov Models (HMMs) In the previous slides, we have seen that in many cases the underlying behavior of nature could be modeled as a Markov process. However
The near-infrared spectra and distribution of excited states of electrodeless discharge rubidium vapour lamps
The near-infrared spectra and distribution of excited states of electrodeless discharge rubidium vapour lamps Sun Qin-Qing( ) a)b), Miao Xin-Yu( ) a), Sheng Rong-Wu( ) c), and Chen Jing-Biao( ) a)b) a)
NCU EE -- DSP VLSI Design. Tsung-Han Tsai 1
NCU EE -- DSP VLSI Design. Tsung-Han Tsai 1 Multi-processor vs. Multi-computer architecture µp vs. DSP RISC vs. DSP RISC Reduced-instruction-set Register-to-register operation Higher throughput by using
Artificial Neural Network and Fuzzy Logic
Artificial Neural Network and Fuzzy Logic 1 Syllabus 2 Syllabus 3 Books 1. Artificial Neural Networks by B. Yagnanarayan, PHI - (Cover Topologies part of unit 1 and All part of Unit 2) 2. Neural Networks
The Quantum Landscape
The Quantum Landscape Computational drug discovery employing machine learning and quantum computing Contact us! lucas@proteinqure.com Or visit our blog to learn more @ www.proteinqure.com 2 Applications
Recurrent Neural Networks. COMP-550 Oct 5, 2017
Recurrent Neural Networks COMP-550 Oct 5, 2017 Outline Introduction to neural networks and deep learning Feedforward neural networks Recurrent neural networks 2 Classification Review y = f( x) output label
Design and Development of a Smartphone Based Visible Spectrophotometer for Analytical Applications
Design and Development of a Smartphone Based Visible Spectrophotometer for Analytical Applications Bedanta Kr. Deka, D. Thakuria, H. Bora and S. Banerjee # Department of Physicis, B. Borooah College, Ulubari,
Improving L-BFGS Initialization for Trust-Region Methods in Deep Learning
Improving L-BFGS Initialization for Trust-Region Methods in Deep Learning Jacob Rafati http://rafati.net jrafatiheravi@ucmerced.edu Ph.D. Candidate, Electrical Engineering and Computer Science University
Long-Short Term Memory and Other Gated RNNs
Long-Short Term Memory and Other Gated RNNs Sargur Srihari srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Sequence Modeling
Address for Correspondence
Research Article APPLICATION OF ARTIFICIAL NEURAL NETWORK FOR INTERFERENCE STUDIES OF LOW-RISE BUILDINGS 1 Narayan K*, 2 Gairola A Address for Correspondence 1 Associate Professor, Department of Civil
Deep Learning. What Is Deep Learning? The Rise of Deep Learning. Long History (in Hind Sight)
CSCE 636 Neural Networks Instructor: Yoonsuck Choe Deep Learning What Is Deep Learning? Learning higher level abstractions/representations from data. Motivation: how the brain represents sensory information
Single-photon and two-photon absorption induced charge model calibration
Single-photon and two-photon absorption induced charge model calibration Vincent Pouget IES CNRS, University of Montpellier Motivation Follow-up of RADLAS2013 presentation on TPA modeling Many use cases
Putting quantum dot lasers to practical use
Enviroment Life Nanotechnology Information Science Reconstruction Others Science and and Technology andfrom Energy Communications andthe Materials for Great Society EastTechnology Japan Earthquake (FY2017
Machine Learning for Physicists Lecture 1
Machine Learning for Physicists Lecture 1 Summer 2017 University of Erlangen-Nuremberg Florian Marquardt (Image generated by a net with 20 hidden layers) OUTPUT INPUT (Picture: Wikimedia Commons) OUTPUT
Classification goals: Make 1 guess about the label (Top-1 error) Make 5 guesses about the label (Top-5 error) No Bounding Box
ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton Motivation Classification goals: Make 1 guess about the label (Top-1 error) Make 5 guesses
Addressing Challenges in Neuromorphic Computing with Memristive Synapses
Addressing Challenges in Neuromorphic Computing with Memristive Synapses Vishal Saxena 1, Xinyu Wu 1 and Maria Mitkova 2 1 Analog Mixed-Signal and Photonic IC (AMPIC) Lab 2 Nanoionic Materials and Devices
Deep Learning Architectures and Algorithms
Deep Learning Architectures and Algorithms In-Jung Kim 2016. 12. 2. Agenda Introduction to Deep Learning RBM and Auto-Encoders Convolutional Neural Networks Recurrent Neural Networks Reinforcement Learning
Recommended Courses by ECE Topic Area Graduate Students
Recommended s by ECE Topic Area Graduate Students The course recommendations below do not represent full plans of study. The courses listed under each heading represent appropriate courses to take if you
Artificial Neural Networks Examination, June 2004
Artificial Neural Networks Examination, June 2004 Instructions There are SIXTY questions (worth up to 60 marks). The exam mark (maximum 60) will be added to the mark obtained in the laborations (maximum
Pattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lessons 7 20 Dec 2017 Outline Artificial Neural networks Notation...2 Introduction...3 Key Equations... 3 Artificial
Multi-PMT Optical Module Designs for IceCube-Gen2
EPJ Web of Conferences 116, 01001 (2016) DOI: 10.1051/epjconf/201611601001 C Owned by the authors, published by EDP Sciences, 2016 Multi-PMT Optical Module Designs for IceCube-Gen2 Alexander Kappes a Erlangen
Artificial Neural Networks
Artificial Neural Networks Oliver Schulte - CMPT 310 Neural Networks Neural networks arise from attempts to model human/animal brains Many models, many claims of biological plausibility We will focus on
Convolutional networks. Sebastian Seung
Convolutional networks Sebastian Seung Convolutional network Neural network with spatial organization every neuron has a location usually on a grid Translation invariance synaptic strength depends on locations
Optics, Optoelectronics and Photonics
Optics, Optoelectronics and Photonics Engineering Principles and Applications Alan Billings Emeritus Professor, University of Western Australia New York London Toronto Sydney Tokyo Singapore v Contents
EEE 241: Linear Systems
EEE 4: Linear Systems Summary # 3: Introduction to artificial neural networks DISTRIBUTED REPRESENTATION An ANN consists of simple processing units communicating with each other. The basic elements of
We Live in Exciting Times. CSCI-567: Machine Learning (Spring 2019) Outline. Outline. ACM (an international computing research society) has named
We Live in Exciting Times ACM (an international computing research society) has named CSCI-567: Machine Learning (Spring 2019) Prof. Victor Adamchik U of Southern California Apr. 2, 2019 Yoshua Bengio,
TIME:A Training-in-memory Architecture for RRAM-based Deep Neural Networks
1 TIME:A Training-in-memory Architecture for -based Deep Neural Networks Ming Cheng, Lixue Xia, Student Member, IEEE, Zhenhua Zhu, Yi Cai, Yuan Xie, Fellow, IEEE, Yu Wang, Senior Member, IEEE, Huazhong
Lecture 7 Artificial neural networks: Supervised learning
Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in
Neuromorphic Network Based on Carbon Nanotube/Polymer Composites
Neuromorphic Network Based on Carbon Nanotube/Polymer Composites Andrew Tudor, Kyunghyun Kim, Alex Ming Shen, Chris Shaffer, Dongwon Lee, Cameron D. Danesh, and Yong Chen Department of Mechanical & Aerospace
Calibration and Temperature Retrieval of Improved Ground-based Atmospheric Microwave Sounder
PIERS ONLINE, VOL. 6, NO. 1, 2010 6 Calibration and Temperature Retrieval of Improved Ground-based Atmospheric Microwave Sounder Jie Ying He 1, 2, Yu Zhang 1, 2, and Sheng Wei Zhang 1 1 Center for Space
Neural Networks. Intro to AI Bert Huang Virginia Tech
Neural Networks Intro to AI Bert Huang Virginia Tech Outline Biological inspiration for artificial neural networks Linear vs. nonlinear functions Learning with neural networks: back propagation https://en.wikipedia.org/wiki/neuron#/media/file:chemical_synapse_schema_cropped.jpg
EXTENDED FUZZY COGNITIVE MAPS
EXTENDED FUZZY COGNITIVE MAPS Masafumi Hagiwara Department of Psychology Stanford University Stanford, CA 930 U.S.A. Abstract Fuzzy Cognitive Maps (FCMs) have been proposed to represent causal reasoning
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
T. BEN-NUN, T. HOEFLER Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis https://www.arxiv.org/abs/1802.09941 What is Deep Learning good for? Digit Recognition Object
(Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann
(Feed-Forward) Neural Networks 2016-12-06 Dr. Hajira Jabeen, Prof. Jens Lehmann Outline In the previous lectures we have learned about tensors and factorization methods. RESCAL is a bilinear model for
CMPE12 - Notes chapter 1. Digital Logic. (Textbook Chapter 3)
CMPE12 - Notes chapter 1 Digital Logic (Textbook Chapter 3) Transistor: Building Block of Computers Microprocessors contain TONS of transistors Intel Montecito (2005): 1.72 billion Intel Pentium 4 (2000):
The Changing Landscape of Land Administration
The Changing Landscape of Land Administration B r e n t J o n e s P E, PLS E s r i World s Largest Media Company No Journalists No Content Producers No Photographers World s Largest Hospitality Company
Naveed Anwar, AIT Solutions
1 Current Trends in Technologies How computing hardware and Software will impact the future Naveed Anwar, Ph.D. Executive Director, AIT Solutions 2 Computing and Technologies Software Hardware Programming
Apprentissage, réseaux de neurones et modèles graphiques (RCP209) Neural Networks and Deep Learning
Apprentissage, réseaux de neurones et modèles graphiques (RCP209) Neural Networks and Deep Learning Nicolas Thome Prenom.Nom@cnam.fr http://cedric.cnam.fr/vertigo/cours/ml2/ Département Informatique Conservatoire
CENG 783. Special topics in. Deep Learning. AlchemyAPI. Week 8. Sinan Kalkan
CENG 783 Special topics in Deep Learning AlchemyAPI Week 8 Sinan Kalkan Loss functions Many correct labels case: Binary prediction for each label, independently: L i = σ j max 0, 1 y ij f j y ij = +1 if