Juergen Gall. Analyzing Human Behavior in Video Sequences
|
|
|
- Ophelia Hancock
- 5 years ago
- Views:
Transcription
1 Juergen Gall Analyzing Human Behavior in Video Sequences
2 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 2 Analyzing Human Behavior Analyzing Human Behavior Human Pose Low level features, e.g., gradients, optical flow Videos
3 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 3 21 Actions from HMDB HMDB51 (Kuehne et al, ICCV 2011) 928 clips, frames
4 Puppet Annotation Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 4
5 Joint-annotated HMDB (JHMDB) [ H. Jhuang et al. Towards Understanding Action Recognition. ICCV 2013 ] [ ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 5
6 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 6 Study with Annotated Data (2013) Low Mid High baseline given puppet flow given puppet mask given joint positions baseline given flow given mask pose features GT + ~11% + ~9% + ~20% Large potential gain for pose feature Not with existing 2d human pose methods [ H. Jhuang et al. Towards Understanding Action Recognition. ICCV 2013 ] [ ]
7 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 7 CNNs for Pose Estimation Stack CNNs: [ S.-E. Wei et al. Convolutional Pose Machines. CVPR 2016 ]
8 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 16 Coupled Action Recognition and Pose Estimation [ U. Iqbal et al. Pose for Action Action for Pose. FG 2017 ]
9 Pose Estimation in Videos Video datasets for human pose in unconstrained videos does not exist. [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 18
10 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 19 Pose Estimation in Videos Video datasets for human pose in unconstrained videos does not exist. Unconstrained means Public available content from the Internet (e.g. Youtube) Multiple persons in a video (no assumption about position) Arbitrary number of visible joints (truncation and occlusion) Large scale variations (unknown scale) [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
11 Pose-Track Dataset [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 20
12 Joint-annotated HMDB (JHMDB) [ H. Jhuang et al. Towards Understanding Action Recognition. ICCV 2013 ] [ ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 22
13 Pose-Track Dataset [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 23
14 Pose-Track Dataset Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 24
15 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 25 Challenge ICCV 2017 [ ]
16 Pose Track: Simultaneous Pose Estimation and Tracking Estimate pose + person association over time: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 26
17 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 27 Pose Track: Simultaneous Pose Estimation and Tracking Estimate pose + person association over time: Predict body joints (CNN trained on MPII Pose)
18 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 28 Pose Track: Simultaneous Pose Estimation and Tracking Estimate pose + person association over time: Predict body joints (CNN trained on MPII Pose) Build a graph with temporal and spatial edges [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
19 Pose Track: Simultaneous Pose Estimation and Tracking f f f [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 29
20 Pose Track: Simultaneous Pose Estimation and Tracking f f f [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 30
21 Pose Track: Simultaneous Pose Estimation and Tracking Unaries: Confidences of detected joints [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 31
22 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 32 Pose Track: Simultaneous Pose Estimation and Tracking Spatial binaries: Extract quadratic bounding box around detection Two cases: Different joint type: Logistic regression based on distance and orientation [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
23 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 33 Pose Track: Simultaneous Pose Estimation and Tracking Spatial binaries: Extract quadratic bounding box around detection Two cases: Same joint type: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
![Estimation and Tracking. CVPR 2017 ] 09.](/docs-images/94/120550359/images/24-4.jpg "10. 2 01 7 Juer gen Gall Instit u t e")
24 Pose Track: Simultaneous Pose Estimation and Tracking [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 34
25 Pose Track: Simultaneous Pose Estimation and Tracking Temporal binaries: Compute optical flow (DeepMatching) [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 35
26 Pose Track: Simultaneous Pose Estimation and Tracking Temporal binaries: Compute optical flow (DeepMatching) Logistic regression: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 36 f f
27 Pose Track: Simultaneous Pose Estimation and Tracking Solve integer linear program: f f f [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 37
28 Pose Track: Simultaneous Pose Estimation and Tracking Solve integer linear program: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 38
29 Pose Track: Simultaneous Pose Estimation and Tracking Solve integer linear program: f f f [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 39
30 Pose Track: Simultaneous Pose Estimation and Tracking To obtain plausible pauses, constraints are added: Spatial transitivity: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 40
31 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 42 Pose Track: Simultaneous Pose Estimation and Tracking To obtain plausible pauses, constraints are added: Spatial transitivity: Temporal transitivity: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
32 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 43 Pose Track: Simultaneous Pose Estimation and Tracking To obtain plausible pauses, constraints are added: Spatial transitivity: Temporal transitivity: Spatio-temporal trans.: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
33 Pose Track: Simultaneous Pose Estimation and Tracking Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 44
34 Pose Track: Simultaneous Pose Estimation and Tracking To obtain plausible pauses, constraints are added: Spatio-temporal consistency: [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 45
35 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 46 Pose Track: Simultaneous Pose Estimation and Tracking Estimate pose + person association over time: Predict body joints (CNN trained on MPII Pose) Build a graph with temporal and spatial edges Partition spatio-temporal graph [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
36 Pose Track: Simultaneous Pose Estimation and Tracking Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 47
37 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 48 Pose Track: Evaluation Pose estimation accuracy (map) Person association (MOTA) [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
38 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 49 Pose Track: Evaluation Pose estimation accuracy (map) Person association (MOTA) [ U. Iqbal et al. Pose-Track: Joint Multi-Person Pose Estimation and Tracking. CVPR 2017 ]
39 Joint-annotated HMDB (JHMDB) [ H. Jhuang et al. Towards Understanding Action Recognition. ICCV 2013 ] [ ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 50
40 Video Analysis for Studying the Behavior of Mice Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 72
41 10/ 9 / Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 74 Recurrent Neural Networks Gated units (LSTM/GRU)
42 Weakly Supervised Learning Fully supervised: Weakly supervised (transcripts) [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 75
43 Weakly Supervised Learning Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 77
44 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 78 Weakly Supervised Learning Represent an activity a like spoon_powder by latent sub-activities s 1 (a),s 2 (a),s 3 (a), s (a) 1 s (a) 2 s (a) 3 s (a) 4 s (a) 5 s (a) 6 Optimal number of sub-activities is unknown: Many sub-activities for long activities Few sub-activities for short activities
45 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 79 Model RNN with Gated Recurrent Units (GRU)
46 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 80 Model Hidden Markov Model (HMM) enforce fixed order of sub-activities: s 1 (a),s 2 (a),s 3 (a), HMMs use probabilities of RNN as input
47 Model Hidden Markov Model (HMM) for each activity Juergen Gall Institute of Computer Science III Computer Vision Group 81
48 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 82 Model The transcripts define the order of activities:
49 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 83 Model The transcripts define the order of activities:
50 Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 84 Model The transcripts define the order of activities:
51 Weakly Supervised Learning [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 85
52 Weakly Supervised Learning [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 86
53 Weakly Supervised Learning [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 87
54 Weakly Supervised Learning [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 88
55 Weakly Supervised Learning [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 89
56 Results Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 90
57 Results Accuracy on unseen sequences (video without transcript) [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 91
58 Results Accuracy on unseen sequences (video without transcript) [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 92
59 Results Accuracy on unseen sequences (video with transcript) [ A. Richard et al. Weakly Supervised Action Learning with RNN based Fine-to-Coarse Modeling. CVPR 2017 ] Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 93
![Research Unit - Anticipating Human Behavior [ https://pages.iai.uni-bonn.de/for2535 ] 2 0.](/docs-images/94/120550359/images/60-1.jpg "0 9. 2 0 1 6 R e s e a r c h U n i t 2 5 3 5 - Antici p a t i n g H u m a n B e h a vi o r")
60 Research Unit - Anticipating Human Behavior [ ] R e s e a r c h U n i t Antici p a t i n g H u m a n B e h a vi o r 94
61 Research Unit - Anticipating Human Behavior R e s e a r c h U n i t Antici p a t i n g H u m a n B e h a vi o r 95
62 Thank you for your attention Juer gen Gall Instit u t e of Com puter Science III Com puter Vision Gr oup 96
Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity- Representativeness Reward
Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity- Representativeness Reward Kaiyang Zhou, Yu Qiao, Tao Xiang AAAI 2018 What is video summarization? Goal: to automatically
Dynamic Data Modeling, Recognition, and Synthesis. Rui Zhao Thesis Defense Advisor: Professor Qiang Ji
Dynamic Data Modeling, Recognition, and Synthesis Rui Zhao Thesis Defense Advisor: Professor Qiang Ji Contents Introduction Related Work Dynamic Data Modeling & Analysis Temporal localization Insufficient
Multimodal context analysis and prediction
Multimodal context analysis and prediction Valeria Tomaselli (valeria.tomaselli@st.com) Sebastiano Battiato Giovanni Maria Farinella Tiziana Rotondo (PhD student) Outline 2 Context analysis vs prediction
Recurrent Autoregressive Networks for Online Multi-Object Tracking. Presented By: Ishan Gupta
Recurrent Autoregressive Networks for Online Multi-Object Tracking Presented By: Ishan Gupta Outline Multi Object Tracking Recurrent Autoregressive Networks (RANs) RANs for Online Tracking Other State
LEARNING REPRESENTATIONS OF SEQUENCES IPAM GRADUATE SUMMER SCHOOL ON DEEP LEARNING
LEARNING REPRESENTATIONS OF SEQUENCES IPAM GRADUATE SUMMER SCHOOL ON DEEP LEARNING GRAHAM TAYLOR SCHOOL OF ENGINEERING UNIVERSITY OF GUELPH Papers and software available at: http://www.uoguelph.ca/~gwtaylor
Large-Scale Feature Learning with Spike-and-Slab Sparse Coding
Large-Scale Feature Learning with Spike-and-Slab Sparse Coding Ian J. Goodfellow, Aaron Courville, Yoshua Bengio ICML 2012 Presented by Xin Yuan January 17, 2013 1 Outline Contributions Spike-and-Slab
End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation
End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation Wei Yang, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang Department of Electronic Engineering,
Hidden CRFs for Human Activity Classification from RGBD Data
H Hidden s for Human from RGBD Data Avi Singh & Ankit Goyal IIT-Kanpur CS679: Machine Learning for Computer Vision April 13, 215 Overview H 1 2 3 H 4 5 6 7 Statement H Input An RGBD video with a human
Recurrent Neural Networks
Recurrent Neural Networks Datamining Seminar Kaspar Märtens Karl-Oskar Masing Today's Topics Modeling sequences: a brief overview Training RNNs with back propagation A toy example of training an RNN Why
Recurrent Neural Networks (Part - 2) Sumit Chopra Facebook
Recurrent Neural Networks (Part - 2) Sumit Chopra Facebook Recap Standard RNNs Training: Backpropagation Through Time (BPTT) Application to sequence modeling Language modeling Applications: Automatic speech
Spatial Transformation
Spatial Transformation Presented by Liqun Chen June 30, 2017 1 Overview 2 Spatial Transformer Networks 3 STN experiments 4 Recurrent Models of Visual Attention (RAM) 5 Recurrent Models of Visual Attention
Anticipating Visual Representations from Unlabeled Data. Carl Vondrick, Hamed Pirsiavash, Antonio Torralba
Anticipating Visual Representations from Unlabeled Data Carl Vondrick, Hamed Pirsiavash, Antonio Torralba Overview Problem Key Insight Methods Experiments Problem: Predict future actions and objects Image
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
Deep Generative Models. (Unsupervised Learning)
Deep Generative Models (Unsupervised Learning) CEng 783 Deep Learning Fall 2017 Emre Akbaş Reminders Next week: project progress demos in class Describe your problem/goal What you have done so far What
Deep Learning Srihari. Deep Belief Nets. Sargur N. Srihari
Deep Belief Nets Sargur N. Srihari srihari@cedar.buffalo.edu Topics 1. Boltzmann machines 2. Restricted Boltzmann machines 3. Deep Belief Networks 4. Deep Boltzmann machines 5. Boltzmann machines for continuous
Spatial Transformer Networks
BIL722 - Deep Learning for Computer Vision Spatial Transformer Networks Max Jaderberg Andrew Zisserman Karen Simonyan Koray Kavukcuoglu Contents Introduction to Spatial Transformers Related Works Spatial
Knowledge Extraction from DBNs for Images
Knowledge Extraction from DBNs for Images Son N. Tran and Artur d Avila Garcez Department of Computer Science City University London Contents 1 Introduction 2 Knowledge Extraction from DBNs 3 Experimental
Neural Networks. Intro to AI Bert Huang Virginia Tech
Neural Networks Intro to AI Bert Huang Virginia Tech Outline Biological inspiration for artificial neural networks Linear vs. nonlinear functions Learning with neural networks: back propagation https://en.wikipedia.org/wiki/neuron#/media/file:chemical_synapse_schema_cropped.jpg
Introduction to Convolutional Neural Networks 2018 / 02 / 23
Introduction to Convolutional Neural Networks 2018 / 02 / 23 Buzzword: CNN Convolutional neural networks (CNN, ConvNet) is a class of deep, feed-forward (not recurrent) artificial neural networks that
Two-Stream Bidirectional Long Short-Term Memory for Mitosis Event Detection and Stage Localization in Phase-Contrast Microscopy Images
Two-Stream Bidirectional Long Short-Term Memory for Mitosis Event Detection and Stage Localization in Phase-Contrast Microscopy Images Yunxiang Mao and Zhaozheng Yin (B) Computer Science, Missouri University
Unsupervised Learning
CS 3750 Advanced Machine Learning hkc6@pitt.edu Unsupervised Learning Data: Just data, no labels Goal: Learn some underlying hidden structure of the data P(, ) P( ) Principle Component Analysis (Dimensionality
CSC321 Lecture 9 Recurrent neural nets
CSC321 Lecture 9 Recurrent neural nets Roger Grosse and Nitish Srivastava February 3, 2015 Roger Grosse and Nitish Srivastava CSC321 Lecture 9 Recurrent neural nets February 3, 2015 1 / 20 Overview You
Deep Learning for Computer Vision
Deep Learning for Computer Vision Spring 2018 http://vllab.ee.ntu.edu.tw/dlcv.html (primary) https://ceiba.ntu.edu.tw/1062dlcv (grade, etc.) FB: DLCV Spring 2018 Yu-Chiang Frank Wang 王鈺強, Associate Professor
Neural Networks with Applications to Vision and Language. Feedforward Networks. Marco Kuhlmann
Neural Networks with Applications to Vision and Language Feedforward Networks Marco Kuhlmann Feedforward networks Linear separability x 2 x 2 0 1 0 1 0 0 x 1 1 0 x 1 linearly separable not linearly separable
Asaf Bar Zvi Adi Hayat. Semantic Segmentation
Asaf Bar Zvi Adi Hayat Semantic Segmentation Today s Topics Fully Convolutional Networks (FCN) (CVPR 2015) Conditional Random Fields as Recurrent Neural Networks (ICCV 2015) Gaussian Conditional random
Machine Learning for Signal Processing Neural Networks Continue. Instructor: Bhiksha Raj Slides by Najim Dehak 1 Dec 2016
Machine Learning for Signal Processing Neural Networks Continue Instructor: Bhiksha Raj Slides by Najim Dehak 1 Dec 2016 1 So what are neural networks?? Voice signal N.Net Transcription Image N.Net Text
Modelling Time Series with Neural Networks. Volker Tresp Summer 2017
Modelling Time Series with Neural Networks Volker Tresp Summer 2017 1 Modelling of Time Series The next figure shows a time series (DAX) Other interesting time-series: energy prize, energy consumption,
Lecture 5 Neural models for NLP
CS546: Machine Learning in NLP (Spring 2018) http://courses.engr.illinois.edu/cs546/ Lecture 5 Neural models for NLP Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Office hours: Tue/Thu 2pm-3pm
Recurrent Neural Networks. Jian Tang
Recurrent Neural Networks Jian Tang tangjianpku@gmail.com 1 RNN: Recurrent neural networks Neural networks for sequence modeling Summarize a sequence with fix-sized vector through recursively updating
Lecture 17: Neural Networks and Deep Learning
UVA CS 6316 / CS 4501-004 Machine Learning Fall 2016 Lecture 17: Neural Networks and Deep Learning Jack Lanchantin Dr. Yanjun Qi 1 Neurons 1-Layer Neural Network Multi-layer Neural Network Loss Functions
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
EE-559 Deep learning Recurrent Neural Networks
EE-559 Deep learning 11.1. Recurrent Neural Networks François Fleuret https://fleuret.org/ee559/ Sun Feb 24 20:33:31 UTC 2019 Inference from sequences François Fleuret EE-559 Deep learning / 11.1. Recurrent
Neural networks COMS 4771
Neural networks COMS 4771 1. Logistic regression Logistic regression Suppose X = R d and Y = {0, 1}. A logistic regression model is a statistical model where the conditional probability function has a
A graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
Global Behaviour Inference using Probabilistic Latent Semantic Analysis
Global Behaviour Inference using Probabilistic Latent Semantic Analysis Jian Li, Shaogang Gong, Tao Xiang Department of Computer Science Queen Mary College, University of London, London, E1 4NS, UK {jianli,
Recurrent Neural Networks (RNN) and Long-Short-Term-Memory (LSTM) Yuan YAO HKUST
1 Recurrent Neural Networks (RNN) and Long-Short-Term-Memory (LSTM) Yuan YAO HKUST Summary We have shown: Now First order optimization methods: GD (BP), SGD, Nesterov, Adagrad, ADAM, RMSPROP, etc. Second
Introduction to Neural Networks
CUONG TUAN NGUYEN SEIJI HOTTA MASAKI NAKAGAWA Tokyo University of Agriculture and Technology Copyright by Nguyen, Hotta and Nakagawa 1 Pattern classification Which category of an input? Example: Character
Random Field Models for Applications in Computer Vision
Random Field Models for Applications in Computer Vision Nazre Batool Post-doctorate Fellow, Team AYIN, INRIA Sophia Antipolis Outline Graphical Models Generative vs. Discriminative Classifiers Markov Random
arxiv: v2 [cs.sd] 7 Feb 2018
![arxiv: v2 [cs.sd] 7 Feb 2018](/thumbs/92/108747184.jpg "arxiv: v2 [cs.sd] 7 Feb 2018")
AUDIO SET CLASSIFICATION WITH ATTENTION MODEL: A PROBABILISTIC PERSPECTIVE Qiuqiang ong*, Yong Xu*, Wenwu Wang, Mark D. Plumbley Center for Vision, Speech and Signal Processing, University of Surrey, U
INF 5860 Machine learning for image classification. Lecture 14: Reinforcement learning May 9, 2018
Machine learning for image classification Lecture 14: Reinforcement learning May 9, 2018 Page 3 Outline Motivation Introduction to reinforcement learning (RL) Value function based methods (Q-learning)
Neural Architectures for Image, Language, and Speech Processing
Neural Architectures for Image, Language, and Speech Processing Karl Stratos June 26, 2018 1 / 31 Overview Feedforward Networks Need for Specialized Architectures Convolutional Neural Networks (CNNs) Recurrent
Introduction to Graphical Models
Introduction to Graphical Models The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 9 (Tue.) Yung-Kyun Noh GENERALIZATION FOR PREDICTION 2 Probabilistic
RECURRENT NETWORKS I. Philipp Krähenbühl
RECURRENT NETWORKS I Philipp Krähenbühl RECAP: CLASSIFICATION conv 1 conv 2 conv 3 conv 4 1 2 tu RECAP: SEGMENTATION conv 1 conv 2 conv 3 conv 4 RECAP: DETECTION conv 1 conv 2 conv 3 conv 4 RECAP: GENERATION
Modeling Complex Temporal Composition of Actionlets for Activity Prediction
Modeling Complex Temporal Composition of Actionlets for Activity Prediction ECCV 2012 Activity Recognition Reading Group Framework of activity prediction What is an Actionlet To segment a long sequence
Final Examination CS540-2: Introduction to Artificial Intelligence
Final Examination CS540-2: Introduction to Artificial Intelligence May 9, 2018 LAST NAME: SOLUTIONS FIRST NAME: Directions 1. This exam contains 33 questions worth a total of 100 points 2. Fill in your
Machine Learning. Classification, Discriminative learning. Marc Toussaint University of Stuttgart Summer 2015
Machine Learning Classification, Discriminative learning Structured output, structured input, discriminative function, joint input-output features, Likelihood Maximization, Logistic regression, binary
AN INTRODUCTION TO NEURAL NETWORKS. Scott Kuindersma November 12, 2009
AN INTRODUCTION TO NEURAL NETWORKS Scott Kuindersma November 12, 2009 SUPERVISED LEARNING We are given some training data: We must learn a function If y is discrete, we call it classification If it is
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
ECE521 Lectures 9 Fully Connected Neural Networks
ECE521 Lectures 9 Fully Connected Neural Networks Outline Multi-class classification Learning multi-layer neural networks 2 Measuring distance in probability space We learnt that the squared L2 distance
AUDIO SET CLASSIFICATION WITH ATTENTION MODEL: A PROBABILISTIC PERSPECTIVE. Qiuqiang Kong*, Yong Xu*, Wenwu Wang, Mark D. Plumbley
AUDIO SET CLASSIFICATION WITH ATTENTION MODEL: A PROBABILISTIC PERSPECTIVE Qiuqiang ong*, Yong Xu*, Wenwu Wang, Mark D. Plumbley Center for Vision, Speech and Signal Processing, University of Surrey, U
A Hierarchical Convolutional Neural Network for Mitosis Detection in Phase-Contrast Microscopy Images
A Hierarchical Convolutional Neural Network for Mitosis Detection in Phase-Contrast Microscopy Images Yunxiang Mao and Zhaozheng Yin (B) Department of Computer Science, Missouri University of Science and
UNSUPERVISED LEARNING
UNSUPERVISED LEARNING Topics Layer-wise (unsupervised) pre-training Restricted Boltzmann Machines Auto-encoders LAYER-WISE (UNSUPERVISED) PRE-TRAINING Breakthrough in 2006 Layer-wise (unsupervised) pre-training
Engineering Part IIB: Module 4F10 Statistical Pattern Processing Lecture 6: Multi-Layer Perceptrons I
Engineering Part IIB: Module 4F10 Statistical Pattern Processing Lecture 6: Multi-Layer Perceptrons I Phil Woodland: pcw@eng.cam.ac.uk Michaelmas 2012 Engineering Part IIB: Module 4F10 Introduction In
Lecture 12. Talk Announcement. Neural Networks. This Lecture: Advanced Machine Learning. Recap: Generalized Linear Discriminants
Advanced Machine Learning Lecture 2 Neural Networks 24..206 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Talk Announcement Yann LeCun (NYU & FaceBook AI) 28..
Introduction to Deep Neural Networks
Introduction to Deep Neural Networks Presenter: Chunyuan Li Pattern Classification and Recognition (ECE 681.01) Duke University April, 2016 Outline 1 Background and Preliminaries Why DNNs? Model: Logistic
Slide credit from Hung-Yi Lee & Richard Socher
Slide credit from Hung-Yi Lee & Richard Socher 1 Review Recurrent Neural Network 2 Recurrent Neural Network Idea: condition the neural network on all previous words and tie the weights at each time step
Neural Networks. David Rosenberg. July 26, New York University. David Rosenberg (New York University) DS-GA 1003 July 26, / 35
Neural Networks David Rosenberg New York University July 26, 2017 David Rosenberg (New York University) DS-GA 1003 July 26, 2017 1 / 35 Neural Networks Overview Objectives What are neural networks? How
Lecture 12. Neural Networks Bastian Leibe RWTH Aachen
Advanced Machine Learning Lecture 12 Neural Networks 10.12.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de This Lecture: Advanced Machine Learning Regression
A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
A Constraint Generation Approach to Learning Stable Linear Dynamical Systems Sajid M. Siddiqi Byron Boots Geoffrey J. Gordon Carnegie Mellon University NIPS 2007 poster W22 steam Application: Dynamic Textures
Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates
Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates Hiroaki Hayashi 1,* Jayanth Koushik 1,* Graham Neubig 1 arxiv:1611.01505v3 [cs.lg] 11 Jun 2018 Abstract Adaptive
COMP90051 Statistical Machine Learning
COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Trevor Cohn 24. Hidden Markov Models & message passing Looking back Representation of joint distributions Conditional/marginal independence
Lecture 12. Neural Networks Bastian Leibe RWTH Aachen
Advanced Machine Learning Lecture 12 Neural Networks 24.11.2016 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Talk Announcement Yann LeCun (NYU & FaceBook AI)
Text2Action: Generative Adversarial Synthesis from Language to Action
Text2Action: Generative Adversarial Synthesis from Language to Action Hyemin Ahn, Timothy Ha*, Yunho Choi*, Hwiyeon Yoo*, and Songhwai Oh Abstract In this paper, we propose a generative model which learns
Statistical Filters for Crowd Image Analysis
Statistical Filters for Crowd Image Analysis Ákos Utasi, Ákos Kiss and Tamás Szirányi Distributed Events Analysis Research Group, Computer and Automation Research Institute H-1111 Budapest, Kende utca
Unsupervised Learning of Hierarchical Models. in collaboration with Josh Susskind and Vlad Mnih
Unsupervised Learning of Hierarchical Models Marc'Aurelio Ranzato Geoff Hinton in collaboration with Josh Susskind and Vlad Mnih Advanced Machine Learning, 9 March 2011 Example: facial expression recognition
Machine Learning for Structured Prediction
Machine Learning for Structured Prediction Grzegorz Chrupa la National Centre for Language Technology School of Computing Dublin City University NCLT Seminar Grzegorz Chrupa la (DCU) Machine Learning for
Learning Semantic Prediction using Pretrained Deep Feedforward Networks
Learning Semantic Prediction using Pretrained Deep Feedforward Networks Jörg Wagner 1,2, Volker Fischer 1, Michael Herman 1 and Sven Behnke 2 1- Robert Bosch GmbH - 70442 Stuttgart - Germany 2- University
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting 卷积 LSTM 网络 : 利用机器学习预测短期降雨 施行健 香港科技大学 VALSE 2016/03/23
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting 卷积 LSTM 网络 : 利用机器学习预测短期降雨 施行健 香港科技大学 VALSE 2016/03/23 Content Quick Review of Recurrent Neural Network Introduction
AUTOMATIC Music Transcription (AMT) is a fundamental. An End-to-End Neural Network for Polyphonic Piano Music Transcription
1 An End-to-End Neural Network for Polyphonic Piano Music Transcription Siddharth Sigtia, Emmanouil Benetos, and Simon Dixon Abstract We present a supervised neural network model for polyphonic piano music
Artificial Neural Networks D B M G. Data Base and Data Mining Group of Politecnico di Torino. Elena Baralis. Politecnico di Torino
Artificial Neural Networks Data Base and Data Mining Group of Politecnico di Torino Elena Baralis Politecnico di Torino Artificial Neural Networks Inspired to the structure of the human brain Neurons as
Stephen Scott.
1 / 35 (Adapted from Vinod Variyam and Ian Goodfellow) sscott@cse.unl.edu 2 / 35 All our architectures so far work on fixed-sized inputs neural networks work on sequences of inputs E.g., text, biological
Conditional Language modeling with attention
Conditional Language modeling with attention 2017.08.25 Oxford Deep NLP 조수현 Review Conditional language model: assign probabilities to sequence of words given some conditioning context x What is the probability
Replicated Softmax: an Undirected Topic Model. Stephen Turner
Replicated Softmax: an Undirected Topic Model Stephen Turner 1. Introduction 2. Replicated Softmax: A Generative Model of Word Counts 3. Evaluating Replicated Softmax as a Generative Model 4. Experimental
Jakub Hajic Artificial Intelligence Seminar I
Jakub Hajic Artificial Intelligence Seminar I. 11. 11. 2014 Outline Key concepts Deep Belief Networks Convolutional Neural Networks A couple of questions Convolution Perceptron Feedforward Neural Network
KNOWLEDGE-GUIDED RECURRENT NEURAL NETWORK LEARNING FOR TASK-ORIENTED ACTION PREDICTION
KNOWLEDGE-GUIDED RECURRENT NEURAL NETWORK LEARNING FOR TASK-ORIENTED ACTION PREDICTION Liang Lin, Lili Huang, Tianshui Chen, Yukang Gan, and Hui Cheng School of Data and Computer Science, Sun Yat-Sen University,
ECE521 Lecture 19 HMM cont. Inference in HMM
ECE521 Lecture 19 HMM cont. Inference in HMM Outline Hidden Markov models Model definitions and notations Inference in HMMs Learning in HMMs 2 Formally, a hidden Markov model defines a generative process
Machine Learning for Computer Vision 8. Neural Networks and Deep Learning. Vladimir Golkov Technical University of Munich Computer Vision Group
Machine Learning for Computer Vision 8. Neural Networks and Deep Learning Vladimir Golkov Technical University of Munich Computer Vision Group INTRODUCTION Nonlinear Coordinate Transformation http://cs.stanford.edu/people/karpathy/convnetjs/
Introduction to Deep Learning
Introduction to Deep Learning A. G. Schwing & S. Fidler University of Toronto, 2015 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 1 / 39 Outline 1 Universality of Neural Networks
Recurrent Neural Network
Recurrent Neural Network Xiaogang Wang xgwang@ee..edu.hk March 2, 2017 Xiaogang Wang (linux) Recurrent Neural Network March 2, 2017 1 / 48 Outline 1 Recurrent neural networks Recurrent neural networks
Long-Short Term Memory and Other Gated RNNs
Long-Short Term Memory and Other Gated RNNs Sargur Srihari srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Sequence Modeling
Normalization Techniques in Training of Deep Neural Networks
Normalization Techniques in Training of Deep Neural Networks Lei Huang ( 黄雷 ) State Key Laboratory of Software Development Environment, Beihang University Mail:huanglei@nlsde.buaa.edu.cn August 17 th,
Machine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler
+ Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Prof. Alexander Ihler Linear Classifiers (Perceptrons) Linear Classifiers a linear classifier is a mapping which partitions
Lecture 16 Deep Neural Generative Models
Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed
Learning Deep Architectures
Learning Deep Architectures Yoshua Bengio, U. Montreal Microsoft Cambridge, U.K. July 7th, 2009, Montreal Thanks to: Aaron Courville, Pascal Vincent, Dumitru Erhan, Olivier Delalleau, Olivier Breuleux,
Based on slides by Richard Zemel
CSC 412/2506 Winter 2018 Probabilistic Learning and Reasoning Lecture 3: Directed Graphical Models and Latent Variables Based on slides by Richard Zemel Learning outcomes What aspects of a model can we
Information Extraction from Text
Information Extraction from Text Jing Jiang Chapter 2 from Mining Text Data (2012) Presented by Andrew Landgraf, September 13, 2013 1 What is Information Extraction? Goal is to discover structured information
Memory-Augmented Attention Model for Scene Text Recognition
Memory-Augmented Attention Model for Scene Text Recognition Cong Wang 1,2, Fei Yin 1,2, Cheng-Lin Liu 1,2,3 1 National Laboratory of Pattern Recognition Institute of Automation, Chinese Academy of Sciences
Backpropagation Introduction to Machine Learning. Matt Gormley Lecture 12 Feb 23, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Backpropagation Matt Gormley Lecture 12 Feb 23, 2018 1 Neural Networks Outline
Supervised Learning. George Konidaris
Supervised Learning George Konidaris gdk@cs.brown.edu Fall 2017 Machine Learning Subfield of AI concerned with learning from data. Broadly, using: Experience To Improve Performance On Some Task (Tom Mitchell,
Recurrent neural networks
12-1: Recurrent neural networks Prof. J.C. Kao, UCLA Recurrent neural networks Motivation Network unrollwing Backpropagation through time Vanishing and exploding gradients LSTMs GRUs 12-2: Recurrent neural
The Perceptron. Volker Tresp Summer 2016
The Perceptron Volker Tresp Summer 2016 1 Elements in Learning Tasks Collection, cleaning and preprocessing of training data Definition of a class of learning models. Often defined by the free model parameters
CSC321 Lecture 16: ResNets and Attention
CSC321 Lecture 16: ResNets and Attention Roger Grosse Roger Grosse CSC321 Lecture 16: ResNets and Attention 1 / 24 Overview Two topics for today: Topic 1: Deep Residual Networks (ResNets) This is the state-of-the
Deep Learning: Approximation of Functions by Composition
Deep Learning: Approximation of Functions by Composition Zuowei Shen Department of Mathematics National University of Singapore Outline 1 A brief introduction of approximation theory 2 Deep learning: approximation
Bayesian Networks (Part I)
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Bayesian Networks (Part I) Graphical Model Readings: Murphy 10 10.2.1 Bishop 8.1,
Competitive Learning for Deep Temporal Networks
Competitive Learning for Deep Temporal Networks Robert Gens Computer Science and Engineering University of Washington Seattle, WA 98195 rcg@cs.washington.edu Pedro Domingos Computer Science and Engineering
Machine Learning for Large-Scale Data Analysis and Decision Making A. Neural Networks Week #6
Machine Learning for Large-Scale Data Analysis and Decision Making 80-629-17A Neural Networks Week #6 Today Neural Networks A. Modeling B. Fitting C. Deep neural networks Today s material is (adapted)
Machine Learning Techniques for Computer Vision
Machine Learning Techniques for Computer Vision Part 2: Unsupervised Learning Microsoft Research Cambridge x 3 1 0.5 0.2 0 0.5 0.3 0 0.5 1 ECCV 2004, Prague x 2 x 1 Overview of Part 2 Mixture models EM
attention mechanisms and generative models
attention mechanisms and generative models Master's Deep Learning Sergey Nikolenko Harbour Space University, Barcelona, Spain November 20, 2017 attention in neural networks attention You re paying attention
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning
CS 229 Project Final Report: Reinforcement Learning for Neural Network Architecture Category : Theory & Reinforcement Learning Lei Lei Ruoxuan Xiong December 16, 2017 1 Introduction Deep Neural Network
De Novo molecular design with Deep Reinforcement Learning
De Novo molecular design with Deep Reinforcement Learning @olexandr Olexandr Isayev, Ph.D. University of North Carolina at Chapel Hill olexandr@unc.edu http://olexandrisayev.com About me Ph.D. in Chemistry