[6, 7], SVM, [7].,, SVM,, SVM,,,,,,, 0.01%,,,,.,,,, SVM,,,, [9], ( ) Soft Confidence-Weighted Learning (SCW)[8],, SCW[8], ( ),,, ( ),,,, , [

Size: px
Start display at page:

Download "[6, 7], SVM, [7].,, SVM,, SVM,,,,,,, 0.01%,,,,.,,,, SVM,,,, [9], ( ) Soft Confidence-Weighted Learning (SCW)[8],, SCW[8], ( ),,, ( ),,,, , ["

Transcription

1 SCW Predicting stock fluctuations using Two-level Mapping and SCW 1 Muhtar Fukuda Faculty of Environmental and Information Studies, Nagoya Sangyo University Abstract: Due to high uncertainty in the stock market, it is difficult to predict the future fluctuations of stock prices even if we use the state-of-the-art techniques of machine learning, such as Deep Learning. However, in some cases with choosing an appropriate machine learning algorithm, feature values and outputs for the prediction, we can have desirable predicted results, especially on short-term stock fluctuations about some market indices. Some initial reliable results have been achieved in our related work, by using Soft Confidence-Weighted (SCW) Leaning, which is one of online learning. In this paper, we propose a predicting method using two-level mapping and SCW. We will focus on feature transformations using the two-level mapping. The first one is based on the mathematical concept of the Singular Value Decomposition (SVD), to get strong convergence and higher accuracy. The second one is to make the predicted Fluctuation Strength (FS) more precisely, in which we use pre-learned outputs and do relearning. 1,,. ICT,,, SNS,,,,,,,, ) fukuda@nagoya-su.ac.jp (DL)[1] Deep Belief Network(DBN)[2] [3] [4],, DL,,,, (feature), Support Vector Machine (SVM)[5], SVM 89

2 [6, 7], SVM, [7].,, SVM,, SVM,,,,,,, 0.01%,,,,.,,,, SVM,,,, [9], ( ) Soft Confidence-Weighted Learning (SCW)[8],, SCW[8], ( ),,, ( ),,,, , [9], 6 1),, [10], ETF,,,,.,,,,., 2),,,, ( ),,,,,,,, SNS ( ), 1,,, 3) ( 1 (c)) ( 1 (b)) ( 1 (a)) 3600 ( 15 ),,. 4),,,, ( 1 (d)) ( 1 (e)) 90

3 5) ( 1 (a)) (feature), ( 1 (f)), 1: 6),,, ( (2)),, SCW, x t R d w R d, ŷ t = sgn(x t w), ŷ t {+1, 1} (1) x t, d, w,, ( ), FS (Fluctuation Strength) FS = x t w (2) w, FS, sgn(fs),,,, [9], DL,, 3, Web,,,,,., [9], ( 1 (f)) ( 1 (a)), 0, ( 1 (d)), 0 (+1), ( 1),,, 3.1, ETF,,,,.,, 91

4 ,., S, T, s S T s, s N s (T T s + N s ), T 0, N 0 Step0 s 0 S,, N s0 =0 Step1 S 1 = {s S T s T 0, N s N 0 }, s S 1, T = 3600, T 0 =0.99T = 3564, N 0 =1,, S Step2 S 1 P 1 %, S 2 s S 1 T, p t,v t,t=1, 2,...,T s AvgLiq(s) ( K ( )) 1 R 1 K+1 AvgLiq(s) = p t v t R k=0 t=1, K 0, 2 K+1 T, R = [ T 2 k ]. Step3 S 2 P 2 %, S 3 s S 2 T p t, t =1, 2,...,T,T +1 s AvgP C(s) ( K ( 1 R )) AvgP C(s) = (p t p t+1 ) 1 R p t+1 k=0 t=1, K 0, 2 K+1 T, R = [ T 2 k ]. K+1 Step4 S 3 {s0 } s 0 s P 3 %, s 0 S(s 0 ) Step s 0, s 0, S(s 0 ) = N, P 1 = P 2 = P 3, P 1 % P 2 % P 3 % S 1 = N P 1,P 2,P 3 N = 320, P 1 = P 2 = P 3 66., 320, s 0 s S(s 0 ) T + L p 0 t,p s t,t=1, 2,...,T,T+1,...,T+L, s 0 s AvgBeta(s 0 s) ( K AvgBeta(s 0 s) = α k β k β k α k = { p0 1 a 0 1 a 0, 1 β k = { ps 1 a s 1 a s, 1 a 0 t = 1 L L i=1 p 0 t+i, k=0 p 0 2 a 0 2 a 0,..., 2 p s 2 a s 2 a s,..., 2 a s t = 1 L ) 1 K+1 p 0 R a0 R a 0 } R p s R as R a s } R L i=1 p s t+i, K 0, 2 K+1 T, R = [ T 2 k ], L, a 0 t as t, t =1, 2,...,T L K(= 7), S(s 0 ), S(s 0 ) 30%, S 1 S (s 0 ) S(s 0 ), S(s 0 ),,. 3.2,,,, RSI (Relative Strength Index) [6], n [11] 92

5 , ( 1 (f)) ( ) t 0 p 0, L =3 2 stride + slack, p t, t =1, 2,..., L t 0 Step1 c t = (pt p0) p 0 100, t =1, 2,...,L Step2 t 1,t 2,t 3 {1, 2,...,L}, t, t {t 1,t 2,t 3 }, t t stride {c t1,c t2,c t3 },, stride Step3 {c t1,c t2,c t3 }, (l 1,l 2,l 3 ), 3 (h 1,h 2,h 3 ). stride =2, slack =2(L = 14), ( 1 (d), (e)) stride slack, s 0 S(s 0 )={s 1,s 2,...,s N },N= S(s 0 ), s S(s 0 ) t αt s =(lt1,l s t2,l s t3,h s s t1,h s t2,h s t3), x 0 t =(αt s1,αt s2,...,α s N t ), t =1, 2,...,T x 0 t, t = 1, 2,...,T X 0,, x 0 t,, 3.3, 2,,,, s 0,, ( 1 (a)) ( 1 (d)) PC(t) (Percentage change), p t t, M, p t 1,p t 2,,p t M, t =1, 2,...,T PC(t) = (a t p t ) 100, a t = 1 p t M, M p (t i) i=1 r 1,r 2,..., r [ T 3 ],..., r [ 2T 3 ],..., r T LC = r [ T 3 ] (Lower Criterion), UC = r [ 2T 3 ] (Upper Criterion) UC LC, { yt U +1 if PC(t) UC = 1 otherwise { yt D +1 if PC(t) LC = 1 otherwise t =1, 2,..., T, Y U =(y U 1,y U 2,...,y U T ), Y D =(y D 1,y D 2,...,y D T ) X = X 0 ( 3.2 ), {X, Y U } {X, Y D },, Y U Y D X Y = Y U ( Y D ),, [9], X 0 = UΣV (3), X = X 0 V, X Y SCW,, x 0 t, x t = x 0 t V, x t,. V SCW, x t R d w R d 93

6 , ŷ t = sgn(x t w), ŷ t {+1, 1} x t x t w 0,, w,,,, 1., 0, 3.3,,,, (2) FS, [9].,,, ( ),,,, SCW w =(w 1,w 2,,w n ), d =(d 1,d 2,,d T ), X 0 Vw = d (4), d f =(f 1,f 2,,f T ), f, d, (2) FS, (5) M X 0 VMw = d f f (5), W = w w, W = U 1 Σ 1 V 1, (3), M =Σ 1 U d f f wv 1 Σ 1 1 U 1 (6), M, x 0 t, t = 1 1, 2,...,T V M, x t 5 x t = x 0 t VM, t=1, 2,...,T (7) 2,, [10], ETF,, 100% 6%,,,.,,, ETF 3% 3.1 ETF, 1,2,, 5.1,, 1, 2, JASADQ, 94

7 3800, 3.1 Step1 1100, Step 320,, 370 Yahoo! [12] ,,,,,, %, ( ), (Yes) ( (Yes) ) ( A ), ( ) ( B ), B A 100 (%),,, ( 1 (d, e)) 3, (6 ), 3600, 1 1, 3%,, ( ),?? Yes Yes (%) : 2, Yahoo! 5.3, ( ) ( ), ( ) ( (2) ),, [9].,,,,,, 2 5.2?? : 2, 6 SCW[8],,, ( ) ( ), ( ) 95

8 1 3 ( ), ( ) 2,,, SCW [8],,, 0, ( ) 0,,,,,,,,,, [9],,, 30,,,, 320,, 2, [1] Hinton, G., Osindero, S. and Teh, Y. W. : A fast learning algorithm for deep belief nets, Neural Computation, Vol. 18, No. 7, pp (2006) [2] Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H.: Greedy Layer-Wise Training of Deep Networks, Advances in Neural Information Processing Systems 19 (NIPS 2006), accessed February 1, 2017, (2007) [3] Chao, J., Shen, F. and Zhao, J.: Forecasting exchange rate with deep belief networks, Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN), pp (2011) [4] Yeh, S., Wang, C. and Tsai, M.: Corporate Default Prediction via Deep Learning, In The 34th International Symposium on Forecasting (ISF 14), (2014) [5] Boser, B. E., Guyon, I. M., and Vapnik, V. N.: A training algorithm for optimal margin classifiers, Proceedings of the fifth annual workshop on Computational learning theory (COLT 92), pp (1992) [6], :,, KBSE , pp (2011) [7] Shen, S., Jiang, H., and Zhang, T.: Stock market forecasting using machine learning algorithms, CS229 (Machine Learning) at Stanford University, accessed February 1, 2017, Zhang-StockMarketForecastingusingMachine LearningAlgorithms.pdf (2012) [8] Wang, J., Zhao, P., and Hoi, S. C. H.: Exact Soft Confidence-Weighted Learning, Proceedings of the 29th International Conference on Machine Learning (ICML 2012),pp (2012) [9] : SCW,, SIG-FIN , (2016) [10], [11], : Deep Belief Network,, SIG-FIN , (2014) [12] Yahoo!, 96

Denoising Autoencoders

Denoising Autoencoders Denoising Autoencoders Oliver Worm, Daniel Leinfelder 20.11.2013 Oliver Worm, Daniel Leinfelder Denoising Autoencoders 20.11.2013 1 / 11 Introduction Poor initialisation can lead to local minima 1986 -

More information

Greedy Layer-Wise Training of Deep Networks

Greedy Layer-Wise Training of Deep Networks Greedy Layer-Wise Training of Deep Networks Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle NIPS 2007 Presented by Ahmed Hefny Story so far Deep neural nets are more expressive: Can learn

More information

Learning Deep Architectures

Learning Deep Architectures Learning Deep Architectures Yoshua Bengio, U. Montreal Microsoft Cambridge, U.K. July 7th, 2009, Montreal Thanks to: Aaron Courville, Pascal Vincent, Dumitru Erhan, Olivier Delalleau, Olivier Breuleux,

More information

arxiv: v3 [cs.lg] 18 Mar 2013

arxiv: v3 [cs.lg] 18 Mar 2013 Hierarchical Data Representation Model - Multi-layer NMF arxiv:1301.6316v3 [cs.lg] 18 Mar 2013 Hyun Ah Song Department of Electrical Engineering KAIST Daejeon, 305-701 hyunahsong@kaist.ac.kr Abstract Soo-Young

More information

Outline. Basic concepts: SVM and kernels SVM primal/dual problems. Chih-Jen Lin (National Taiwan Univ.) 1 / 22

Outline. Basic concepts: SVM and kernels SVM primal/dual problems. Chih-Jen Lin (National Taiwan Univ.) 1 / 22 Outline Basic concepts: SVM and kernels SVM primal/dual problems Chih-Jen Lin (National Taiwan Univ.) 1 / 22 Outline Basic concepts: SVM and kernels Basic concepts: SVM and kernels SVM primal/dual problems

More information

Deep Belief Networks are compact universal approximators

Deep Belief Networks are compact universal approximators 1 Deep Belief Networks are compact universal approximators Nicolas Le Roux 1, Yoshua Bengio 2 1 Microsoft Research Cambridge 2 University of Montreal Keywords: Deep Belief Networks, Universal Approximation

More information

FreezeOut: Accelerate Training by Progressively Freezing Layers

FreezeOut: Accelerate Training by Progressively Freezing Layers FreezeOut: Accelerate Training by Progressively Freezing Layers Andrew Brock, Theodore Lim, & J.M. Ritchie School of Engineering and Physical Sciences Heriot-Watt University Edinburgh, UK {ajb5, t.lim,

More information

Learning Deep Architectures

Learning Deep Architectures Learning Deep Architectures Yoshua Bengio, U. Montreal CIFAR NCAP Summer School 2009 August 6th, 2009, Montreal Main reference: Learning Deep Architectures for AI, Y. Bengio, to appear in Foundations and

More information

Nonlinear system modeling with deep neural networks and autoencoders algorithm

Nonlinear system modeling with deep neural networks and autoencoders algorithm Nonlinear system modeling with deep neural networks and autoencoders algorithm Erick De la Rosa, Wen Yu Departamento de Control Automatico CINVESTAV-IPN Mexico City, Mexico yuw@ctrl.cinvestav.mx Xiaoou

More information

WHY ARE DEEP NETS REVERSIBLE: A SIMPLE THEORY,

WHY ARE DEEP NETS REVERSIBLE: A SIMPLE THEORY, WHY ARE DEEP NETS REVERSIBLE: A SIMPLE THEORY, WITH IMPLICATIONS FOR TRAINING Sanjeev Arora, Yingyu Liang & Tengyu Ma Department of Computer Science Princeton University Princeton, NJ 08540, USA {arora,yingyul,tengyu}@cs.princeton.edu

More information

TUTORIAL PART 1 Unsupervised Learning

TUTORIAL PART 1 Unsupervised Learning TUTORIAL PART 1 Unsupervised Learning Marc'Aurelio Ranzato Department of Computer Science Univ. of Toronto ranzato@cs.toronto.edu Co-organizers: Honglak Lee, Yoshua Bengio, Geoff Hinton, Yann LeCun, Andrew

More information

Gaussian Cardinality Restricted Boltzmann Machines

Gaussian Cardinality Restricted Boltzmann Machines Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence Gaussian Cardinality Restricted Boltzmann Machines Cheng Wan, Xiaoming Jin, Guiguang Ding and Dou Shen School of Software, Tsinghua

More information

Knowledge Extraction from DBNs for Images

Knowledge Extraction from DBNs for Images Knowledge Extraction from DBNs for Images Son N. Tran and Artur d Avila Garcez Department of Computer Science City University London Contents 1 Introduction 2 Knowledge Extraction from DBNs 3 Experimental

More information

Active Learning with Support Vector Machines for Tornado Prediction

Active Learning with Support Vector Machines for Tornado Prediction International Conference on Computational Science (ICCS) 2007 Beijing, China May 27-30, 2007 Active Learning with Support Vector Machines for Tornado Prediction Theodore B. Trafalis 1, Indra Adrianto 1,

More information

Feature Design. Feature Design. Feature Design. & Deep Learning

Feature Design. Feature Design. Feature Design. & Deep Learning Artificial Intelligence and its applications Lecture 9 & Deep Learning Professor Daniel Yeung danyeung@ieee.org Dr. Patrick Chan patrickchan@ieee.org South China University of Technology, China Appropriately

More information

Neural Networks: A Very Brief Tutorial

Neural Networks: A Very Brief Tutorial Neural Networks: A Very Brief Tutorial Chloé-Agathe Azencott Machine Learning & Computational Biology MPIs for Developmental Biology & for Intelligent Systems Tübingen (Germany) cazencott@tue.mpg.de October

More information

Empirical Analysis of the Divergence of Gibbs Sampling Based Learning Algorithms for Restricted Boltzmann Machines

Empirical Analysis of the Divergence of Gibbs Sampling Based Learning Algorithms for Restricted Boltzmann Machines Empirical Analysis of the Divergence of Gibbs Sampling Based Learning Algorithms for Restricted Boltzmann Machines Asja Fischer and Christian Igel Institut für Neuroinformatik Ruhr-Universität Bochum,

More information

Low Bias Bagged Support Vector Machines

Low Bias Bagged Support Vector Machines Low Bias Bagged Support Vector Machines Giorgio Valentini Dipartimento di Scienze dell Informazione Università degli Studi di Milano, Italy valentini@dsi.unimi.it Thomas G. Dietterich Department of Computer

More information

arxiv: v2 [stat.ml] 18 Jun 2017

arxiv: v2 [stat.ml] 18 Jun 2017 FREEZEOUT: ACCELERATE TRAINING BY PROGRES- SIVELY FREEZING LAYERS Andrew Brock, Theodore Lim, & J.M. Ritchie School of Engineering and Physical Sciences Heriot-Watt University Edinburgh, UK {ajb5, t.lim,

More information

Introduction to Support Vector Machines

Introduction to Support Vector Machines Introduction to Support Vector Machines Andreas Maletti Technische Universität Dresden Fakultät Informatik June 15, 2006 1 The Problem 2 The Basics 3 The Proposed Solution Learning by Machines Learning

More information

STA141C: Big Data & High Performance Statistical Computing

STA141C: Big Data & High Performance Statistical Computing STA141C: Big Data & High Performance Statistical Computing Lecture 8: Optimization Cho-Jui Hsieh UC Davis May 9, 2017 Optimization Numerical Optimization Numerical Optimization: min X f (X ) Can be applied

More information

Deep Learning Basics Lecture 7: Factor Analysis. Princeton University COS 495 Instructor: Yingyu Liang

Deep Learning Basics Lecture 7: Factor Analysis. Princeton University COS 495 Instructor: Yingyu Liang Deep Learning Basics Lecture 7: Factor Analysis Princeton University COS 495 Instructor: Yingyu Liang Supervised v.s. Unsupervised Math formulation for supervised learning Given training data x i, y i

More information

A Hybrid Deep Learning Approach For Chaotic Time Series Prediction Based On Unsupervised Feature Learning

A Hybrid Deep Learning Approach For Chaotic Time Series Prediction Based On Unsupervised Feature Learning A Hybrid Deep Learning Approach For Chaotic Time Series Prediction Based On Unsupervised Feature Learning Norbert Ayine Agana Advisor: Abdollah Homaifar Autonomous Control & Information Technology Institute

More information

Learning Tetris. 1 Tetris. February 3, 2009

Learning Tetris. 1 Tetris. February 3, 2009 Learning Tetris Matt Zucker Andrew Maas February 3, 2009 1 Tetris The Tetris game has been used as a benchmark for Machine Learning tasks because its large state space (over 2 200 cell configurations are

More information

Deep Learning & Neural Networks Lecture 2

Deep Learning & Neural Networks Lecture 2 Deep Learning & Neural Networks Lecture 2 Kevin Duh Graduate School of Information Science Nara Institute of Science and Technology Jan 16, 2014 2/45 Today s Topics 1 General Ideas in Deep Learning Motivation

More information

STATE GENERALIZATION WITH SUPPORT VECTOR MACHINES IN REINFORCEMENT LEARNING. Ryo Goto, Toshihiro Matsui and Hiroshi Matsuo

STATE GENERALIZATION WITH SUPPORT VECTOR MACHINES IN REINFORCEMENT LEARNING. Ryo Goto, Toshihiro Matsui and Hiroshi Matsuo STATE GENERALIZATION WITH SUPPORT VECTOR MACHINES IN REINFORCEMENT LEARNING Ryo Goto, Toshihiro Matsui and Hiroshi Matsuo Department of Electrical and Computer Engineering, Nagoya Institute of Technology

More information

Deep Learning. Alexandre Allauzen, Michèle Sebag, Yann Ollivier CNRS & Université Paris-Sud

Deep Learning. Alexandre Allauzen, Michèle Sebag, Yann Ollivier CNRS & Université Paris-Sud Deep Learning Alexandre Allauzen, Michèle Sebag, Yann Ollivier CNRS & Université Paris-Sud Nov. 23rd, 2016 Credit for slides: Yoshua Bengio, Yann Le Cun, Nando de Freitas, Christian Perone, Honglak Lee

More information

Representational Power of Restricted Boltzmann Machines and Deep Belief Networks. Nicolas Le Roux and Yoshua Bengio Presented by Colin Graber

Representational Power of Restricted Boltzmann Machines and Deep Belief Networks. Nicolas Le Roux and Yoshua Bengio Presented by Colin Graber Representational Power of Restricted Boltzmann Machines and Deep Belief Networks Nicolas Le Roux and Yoshua Bengio Presented by Colin Graber Introduction Representational abilities of functions with some

More information

Au-delà de la Machine de Boltzmann Restreinte. Hugo Larochelle University of Toronto

Au-delà de la Machine de Boltzmann Restreinte. Hugo Larochelle University of Toronto Au-delà de la Machine de Boltzmann Restreinte Hugo Larochelle University of Toronto Introduction Restricted Boltzmann Machines (RBMs) are useful feature extractors They are mostly used to initialize deep

More information

Linear, threshold units. Linear Discriminant Functions and Support Vector Machines. Biometrics CSE 190 Lecture 11. X i : inputs W i : weights

Linear, threshold units. Linear Discriminant Functions and Support Vector Machines. Biometrics CSE 190 Lecture 11. X i : inputs W i : weights Linear Discriminant Functions and Support Vector Machines Linear, threshold units CSE19, Winter 11 Biometrics CSE 19 Lecture 11 1 X i : inputs W i : weights θ : threshold 3 4 5 1 6 7 Courtesy of University

More information

Linear classifiers Lecture 3

Linear classifiers Lecture 3 Linear classifiers Lecture 3 David Sontag New York University Slides adapted from Luke Zettlemoyer, Vibhav Gogate, and Carlos Guestrin ML Methodology Data: labeled instances, e.g. emails marked spam/ham

More information

ECS289: Scalable Machine Learning

ECS289: Scalable Machine Learning ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Sept 29, 2016 Outline Convex vs Nonconvex Functions Coordinate Descent Gradient Descent Newton s method Stochastic Gradient Descent Numerical Optimization

More information

TIME series forecasting is used for forecasting the future

TIME series forecasting is used for forecasting the future A Novel DBN Model for Time Series Forecasting Yongpan Ren, Jingli Mao, Yong Liu, Yingzhe Li Abstract Deep Belief Network (DBN) via stacking Restricted Boltzmann Machines (RBMs) has been successfully applied

More information

Neural network time series classification of changes in nuclear power plant processes

Neural network time series classification of changes in nuclear power plant processes 2009 Quality and Productivity Research Conference Neural network time series classification of changes in nuclear power plant processes Karel Kupka TriloByte Statistical Research, Center for Quality and

More information

Formulation with slack variables

Formulation with slack variables Formulation with slack variables Optimal margin classifier with slack variables and kernel functions described by Support Vector Machine (SVM). min (w,ξ) ½ w 2 + γσξ(i) subject to ξ(i) 0 i, d(i) (w T x(i)

More information

On the complexity of shallow and deep neural network classifiers

On the complexity of shallow and deep neural network classifiers On the complexity of shallow and deep neural network classifiers Monica Bianchini and Franco Scarselli Department of Information Engineering and Mathematics University of Siena Via Roma 56, I-53100, Siena,

More information

Final Overview. Introduction to ML. Marek Petrik 4/25/2017

Final Overview. Introduction to ML. Marek Petrik 4/25/2017 Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,

More information

RAGAV VENKATESAN VIJETHA GATUPALLI BAOXIN LI NEURAL DATASET GENERALITY

RAGAV VENKATESAN VIJETHA GATUPALLI BAOXIN LI NEURAL DATASET GENERALITY RAGAV VENKATESAN VIJETHA GATUPALLI BAOXIN LI NEURAL DATASET GENERALITY SIFT HOG ALL ABOUT THE FEATURES DAISY GABOR AlexNet GoogleNet CONVOLUTIONAL NEURAL NETWORKS VGG-19 ResNet FEATURES COMES FROM DATA

More information

Polyhedral Computation. Linear Classifiers & the SVM

Polyhedral Computation. Linear Classifiers & the SVM Polyhedral Computation Linear Classifiers & the SVM mcuturi@i.kyoto-u.ac.jp Nov 26 2010 1 Statistical Inference Statistical: useful to study random systems... Mutations, environmental changes etc. life

More information

Deep Learning of Invariant Spatiotemporal Features from Video. Bo Chen, Jo-Anne Ting, Ben Marlin, Nando de Freitas University of British Columbia

Deep Learning of Invariant Spatiotemporal Features from Video. Bo Chen, Jo-Anne Ting, Ben Marlin, Nando de Freitas University of British Columbia Deep Learning of Invariant Spatiotemporal Features from Video Bo Chen, Jo-Anne Ting, Ben Marlin, Nando de Freitas University of British Columbia Introduction Focus: Unsupervised feature extraction from

More information

NON-FIXED AND ASYMMETRICAL MARGIN APPROACH TO STOCK MARKET PREDICTION USING SUPPORT VECTOR REGRESSION. Haiqin Yang, Irwin King and Laiwan Chan

NON-FIXED AND ASYMMETRICAL MARGIN APPROACH TO STOCK MARKET PREDICTION USING SUPPORT VECTOR REGRESSION. Haiqin Yang, Irwin King and Laiwan Chan In The Proceedings of ICONIP 2002, Singapore, 2002. NON-FIXED AND ASYMMETRICAL MARGIN APPROACH TO STOCK MARKET PREDICTION USING SUPPORT VECTOR REGRESSION Haiqin Yang, Irwin King and Laiwan Chan Department

More information

Deep Convolutional Neural Networks for Pairwise Causality

Deep Convolutional Neural Networks for Pairwise Causality Deep Convolutional Neural Networks for Pairwise Causality Karamjit Singh, Garima Gupta, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal TCS Research, Delhi Tata Consultancy Services Ltd. {karamjit.singh,

More information

Perceptron Revisited: Linear Separators. Support Vector Machines

Perceptron Revisited: Linear Separators. Support Vector Machines Support Vector Machines Perceptron Revisited: Linear Separators Binary classification can be viewed as the task of separating classes in feature space: w T x + b > 0 w T x + b = 0 w T x + b < 0 Department

More information

Support Vector Machines. Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar

Support Vector Machines. Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar Data Mining Support Vector Machines Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar 02/03/2018 Introduction to Data Mining 1 Support Vector Machines Find a linear hyperplane

More information

Expressive Power and Approximation Errors of Restricted Boltzmann Machines

Expressive Power and Approximation Errors of Restricted Boltzmann Machines Expressive Power and Approximation Errors of Restricted Boltzmann Machines Guido F. Montúfar, Johannes Rauh, and Nihat Ay, Max Planck Institute for Mathematics in the Sciences, Inselstraße 0403 Leipzig,

More information

Support Vector Machine Regression for Volatile Stock Market Prediction

Support Vector Machine Regression for Volatile Stock Market Prediction Support Vector Machine Regression for Volatile Stock Market Prediction Haiqin Yang, Laiwan Chan, and Irwin King Department of Computer Science and Engineering The Chinese University of Hong Kong Shatin,

More information

Lecture 16 Deep Neural Generative Models

Lecture 16 Deep Neural Generative Models Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed

More information

Lecture Support Vector Machine (SVM) Classifiers

Lecture Support Vector Machine (SVM) Classifiers Introduction to Machine Learning Lecturer: Amir Globerson Lecture 6 Fall Semester Scribe: Yishay Mansour 6.1 Support Vector Machine (SVM) Classifiers Classification is one of the most important tasks in

More information

Sparse Support Vector Machines by Kernel Discriminant Analysis

Sparse Support Vector Machines by Kernel Discriminant Analysis Sparse Support Vector Machines by Kernel Discriminant Analysis Kazuki Iwamura and Shigeo Abe Kobe University - Graduate School of Engineering Kobe, Japan Abstract. We discuss sparse support vector machines

More information

Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes

Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes Ruslan Salakhutdinov and Geoffrey Hinton Department of Computer Science, University of Toronto 6 King s College Rd, M5S 3G4, Canada

More information

Deeply-Supervised Nets

Deeply-Supervised Nets Deeply-Supervised Nets Chen-Yu Lee Saining Xie Patrick W. Gallagher Dept. of ECE, UCSD Dept. of CSE and CogSci, UCSD Dept. of CogSci, UCSD chl60@ucsd.edu s9xie@ucsd.edu patrick.w.gallagher@gmail.com Zhengyou

More information

Very-short term solar power generation forecasting based on trend-additive and seasonal-multiplicative smoothing methodology

Very-short term solar power generation forecasting based on trend-additive and seasonal-multiplicative smoothing methodology Very-short term solar power generation forecasting based on trend-additive and seasonal-multiplicative smoothing methodology Stanislav Eroshenko 1, Alexandra Khalyasmaa 1,* and Rustam Valiev 1 1 Ural Federal

More information

Measuring the Usefulness of Hidden Units in Boltzmann Machines with Mutual Information

Measuring the Usefulness of Hidden Units in Boltzmann Machines with Mutual Information Measuring the Usefulness of Hidden Units in Boltzmann Machines with Mutual Information Mathias Berglund, Tapani Raiko, and KyungHyun Cho Department of Information and Computer Science Aalto University

More information

Scale-Invariance of Support Vector Machines based on the Triangular Kernel. Abstract

Scale-Invariance of Support Vector Machines based on the Triangular Kernel. Abstract Scale-Invariance of Support Vector Machines based on the Triangular Kernel François Fleuret Hichem Sahbi IMEDIA Research Group INRIA Domaine de Voluceau 78150 Le Chesnay, France Abstract This paper focuses

More information

From Lasso regression to Feature vector machine

From Lasso regression to Feature vector machine From Lasso regression to Feature vector machine Fan Li, Yiming Yang and Eric P. Xing,2 LTI and 2 CALD, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA USA 523 {hustlf,yiming,epxing}@cs.cmu.edu

More information

Support Vector Machines on General Confidence Functions

Support Vector Machines on General Confidence Functions Support Vector Machines on General Confidence Functions Yuhong Guo University of Alberta yuhong@cs.ualberta.ca Dale Schuurmans University of Alberta dale@cs.ualberta.ca Abstract We present a generalized

More information

Deep Belief Networks are Compact Universal Approximators

Deep Belief Networks are Compact Universal Approximators Deep Belief Networks are Compact Universal Approximators Franck Olivier Ndjakou Njeunje Applied Mathematics and Scientific Computation May 16, 2016 1 / 29 Outline 1 Introduction 2 Preliminaries Universal

More information

Outliers Treatment in Support Vector Regression for Financial Time Series Prediction

Outliers Treatment in Support Vector Regression for Financial Time Series Prediction Outliers Treatment in Support Vector Regression for Financial Time Series Prediction Haiqin Yang, Kaizhu Huang, Laiwan Chan, Irwin King, and Michael R. Lyu Department of Computer Science and Engineering

More information

Kai Yu NEC Laboratories America, Cupertino, California, USA

Kai Yu NEC Laboratories America, Cupertino, California, USA Kai Yu NEC Laboratories America, Cupertino, California, USA Joint work with Jinjun Wang, Fengjun Lv, Wei Xu, Yihong Gong Xi Zhou, Jianchao Yang, Thomas Huang, Tong Zhang Chen Wu NEC Laboratories America

More information

Predicting Time of Peak Foreign Exchange Rates. Charles Mulemi, Lucio Dery 0. ABSTRACT

Predicting Time of Peak Foreign Exchange Rates. Charles Mulemi, Lucio Dery 0. ABSTRACT Predicting Time of Peak Foreign Exchange Rates Charles Mulemi, Lucio Dery 0. ABSTRACT This paper explores various machine learning models of predicting the day foreign exchange rates peak in a given window.

More information

Calibrated Uncertainty in Deep Learning

Calibrated Uncertainty in Deep Learning Calibrated Uncertainty in Deep Learning U NCERTAINTY IN DEEP LEARNING W ORKSHOP @ UAI18 Volodymyr Kuleshov August 10, 2018 Estimating Uncertainty is Crucial in Many Applications Assessing uncertainty can

More information

An Introduction to Deep Learning

An Introduction to Deep Learning An Introduction to Deep Learning Ludovic Arnold 1,2, Sébastien Rebecchi 1, Sylvain Chevallier 1, Hélène Paugam-Moisy 1,3 1- Tao, INRIA-Saclay, LRI, UMR8623, Université Paris-Sud 11 F-91405 Orsay, France

More information

SVM TRADE-OFF BETWEEN MAXIMIZE THE MARGIN AND MINIMIZE THE VARIABLES USED FOR REGRESSION

SVM TRADE-OFF BETWEEN MAXIMIZE THE MARGIN AND MINIMIZE THE VARIABLES USED FOR REGRESSION International Journal of Pure and Applied Mathematics Volume 87 No. 6 2013, 741-750 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu doi: http://dx.doi.org/10.12732/ijpam.v87i6.2

More information

Probabilistic Energy Forecasting

Probabilistic Energy Forecasting Probabilistic Energy Forecasting Moritz Schmid Seminar Energieinformatik WS 2015/16 ^ KIT The Research University in the Helmholtz Association www.kit.edu Agenda Forecasting challenges Renewable energy

More information

Efficient and Principled Online Classification Algorithms for Lifelon

Efficient and Principled Online Classification Algorithms for Lifelon Efficient and Principled Online Classification Algorithms for Lifelong Learning Toyota Technological Institute at Chicago Chicago, IL USA Talk @ Lifelong Learning for Mobile Robotics Applications Workshop,

More information

Measuring Invariances in Deep Networks

Measuring Invariances in Deep Networks Measuring Invariances in Deep Networks Ian J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee, Andrew Y. Ng Computer Science Department Stanford University Stanford, CA 9435 {ia3n,quocle,asaxe,hllee,ang}@cs.stanford.edu

More information

Statistical Learning Reading Assignments

Statistical Learning Reading Assignments Statistical Learning Reading Assignments S. Gong et al. Dynamic Vision: From Images to Face Recognition, Imperial College Press, 2001 (Chapt. 3, hard copy). T. Evgeniou, M. Pontil, and T. Poggio, "Statistical

More information

Automatic Feature Decomposition for Single View Co-training

Automatic Feature Decomposition for Single View Co-training Automatic Feature Decomposition for Single View Co-training Minmin Chen, Kilian Weinberger, Yixin Chen Computer Science and Engineering Washington University in Saint Louis Minmin Chen, Kilian Weinberger,

More information

Support'Vector'Machines. Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan

Support'Vector'Machines. Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan Support'Vector'Machines Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan kasthuri.kannan@nyumc.org Overview Support Vector Machines for Classification Linear Discrimination Nonlinear Discrimination

More information

References. Lecture 7: Support Vector Machines. Optimum Margin Perceptron. Perceptron Learning Rule

References. Lecture 7: Support Vector Machines. Optimum Margin Perceptron. Perceptron Learning Rule References Lecture 7: Support Vector Machines Isabelle Guyon guyoni@inf.ethz.ch An training algorithm for optimal margin classifiers Boser-Guyon-Vapnik, COLT, 992 http://www.clopinet.com/isabelle/p apers/colt92.ps.z

More information

Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści

Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?

More information

A Revisit to Support Vector Data Description (SVDD)

A Revisit to Support Vector Data Description (SVDD) A Revisit to Support Vector Data Description (SVDD Wei-Cheng Chang b99902019@csie.ntu.edu.tw Ching-Pei Lee r00922098@csie.ntu.edu.tw Chih-Jen Lin cjlin@csie.ntu.edu.tw Department of Computer Science, National

More information

Basic Principles of Unsupervised and Unsupervised

Basic Principles of Unsupervised and Unsupervised Basic Principles of Unsupervised and Unsupervised Learning Toward Deep Learning Shun ichi Amari (RIKEN Brain Science Institute) collaborators: R. Karakida, M. Okada (U. Tokyo) Deep Learning Self Organization

More information

Machine Learning for Structured Prediction

Machine Learning for Structured Prediction Machine Learning for Structured Prediction Grzegorz Chrupa la National Centre for Language Technology School of Computing Dublin City University NCLT Seminar Grzegorz Chrupa la (DCU) Machine Learning for

More information

Bearing fault diagnosis based on TEO and SVM

Bearing fault diagnosis based on TEO and SVM Bearing fault diagnosis based on TEO and SVM Qingzhu Liu, Yujie Cheng 2 School of Reliability and Systems Engineering, Beihang University, Beijing 9, China Science and Technology on Reliability and Environmental

More information

Statistical Machine Learning from Data

Statistical Machine Learning from Data Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Ensembles Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique Fédérale de Lausanne

More information

Machine Learning (CS 567) Lecture 2

Machine Learning (CS 567) Lecture 2 Machine Learning (CS 567) Lecture 2 Time: T-Th 5:00pm - 6:20pm Location: GFS118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol

More information

Support Vector Machine II

Support Vector Machine II Support Vector Machine II Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative HW 1 due tonight HW 2 released. Online Scalable Learning Adaptive to Unknown Dynamics and Graphs Yanning

More information

Introduction to Machine Learning Midterm Exam

Introduction to Machine Learning Midterm Exam 10-701 Introduction to Machine Learning Midterm Exam Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes, but

More information

Support Vector Machines

Support Vector Machines Support Vector Machines Tobias Pohlen Selected Topics in Human Language Technology and Pattern Recognition February 10, 2014 Human Language Technology and Pattern Recognition Lehrstuhl für Informatik 6

More information

Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks

Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks Creighton Heaukulani and Zoubin Ghahramani University of Cambridge TU Denmark, June 2013 1 A Network Dynamic network data

More information

Final Examination CS540-2: Introduction to Artificial Intelligence

Final Examination CS540-2: Introduction to Artificial Intelligence Final Examination CS540-2: Introduction to Artificial Intelligence May 9, 2018 LAST NAME: SOLUTIONS FIRST NAME: Directions 1. This exam contains 33 questions worth a total of 100 points 2. Fill in your

More information

Neural Networks. Prof. Dr. Rudolf Kruse. Computational Intelligence Group Faculty for Computer Science

Neural Networks. Prof. Dr. Rudolf Kruse. Computational Intelligence Group Faculty for Computer Science Neural Networks Prof. Dr. Rudolf Kruse Computational Intelligence Group Faculty for Computer Science kruse@iws.cs.uni-magdeburg.de Rudolf Kruse Neural Networks 1 Supervised Learning / Support Vector Machines

More information

Support Vector Machines. Machine Learning Fall 2017

Support Vector Machines. Machine Learning Fall 2017 Support Vector Machines Machine Learning Fall 2017 1 Where are we? Learning algorithms Decision Trees Perceptron AdaBoost 2 Where are we? Learning algorithms Decision Trees Perceptron AdaBoost Produce

More information

Support Vector Machine for Classification and Regression

Support Vector Machine for Classification and Regression Support Vector Machine for Classification and Regression Ahlame Douzal AMA-LIG, Université Joseph Fourier Master 2R - MOSIG (2013) November 25, 2013 Loss function, Separating Hyperplanes, Canonical Hyperplan

More information

COMP 551 Applied Machine Learning Lecture 21: Bayesian optimisation

COMP 551 Applied Machine Learning Lecture 21: Bayesian optimisation COMP 55 Applied Machine Learning Lecture 2: Bayesian optimisation Associate Instructor: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp55 Unless otherwise noted, all material posted

More information

A graph contains a set of nodes (vertices) connected by links (edges or arcs)

A graph contains a set of nodes (vertices) connected by links (edges or arcs) BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,

More information

Introduction to Deep Learning

Introduction to Deep Learning Introduction to Deep Learning A. G. Schwing & S. Fidler University of Toronto, 2014 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2014 1 / 35 Outline 1 Universality of Neural Networks

More information

Machine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler

Machine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler + Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Prof. Alexander Ihler Linear Classifiers (Perceptrons) Linear Classifiers a linear classifier is a mapping which partitions

More information

Support Vector Machines

Support Vector Machines Support Vector Machines Hypothesis Space variable size deterministic continuous parameters Learning Algorithm linear and quadratic programming eager batch SVMs combine three important ideas Apply optimization

More information

Support Vector Machine (continued)

Support Vector Machine (continued) Support Vector Machine continued) Overlapping class distribution: In practice the class-conditional distributions may overlap, so that the training data points are no longer linearly separable. We need

More information

A Wavelet Neural Network Forecasting Model Based On ARIMA

A Wavelet Neural Network Forecasting Model Based On ARIMA A Wavelet Neural Network Forecasting Model Based On ARIMA Wang Bin*, Hao Wen-ning, Chen Gang, He Deng-chao, Feng Bo PLA University of Science &Technology Nanjing 210007, China e-mail:lgdwangbin@163.com

More information

Machine learning for pervasive systems Classification in high-dimensional spaces

Machine learning for pervasive systems Classification in high-dimensional spaces Machine learning for pervasive systems Classification in high-dimensional spaces Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Version

More information

Machine Learning Basics

Machine Learning Basics Security and Fairness of Deep Learning Machine Learning Basics Anupam Datta CMU Spring 2019 Image Classification Image Classification Image classification pipeline Input: A training set of N images, each

More information

Unsupervised Learning of Hierarchical Models. in collaboration with Josh Susskind and Vlad Mnih

Unsupervised Learning of Hierarchical Models. in collaboration with Josh Susskind and Vlad Mnih Unsupervised Learning of Hierarchical Models Marc'Aurelio Ranzato Geoff Hinton in collaboration with Josh Susskind and Vlad Mnih Advanced Machine Learning, 9 March 2011 Example: facial expression recognition

More information

Large-Scale Feature Learning with Spike-and-Slab Sparse Coding

Large-Scale Feature Learning with Spike-and-Slab Sparse Coding Large-Scale Feature Learning with Spike-and-Slab Sparse Coding Ian J. Goodfellow, Aaron Courville, Yoshua Bengio ICML 2012 Presented by Xin Yuan January 17, 2013 1 Outline Contributions Spike-and-Slab

More information

An Empirical Study of Building Compact Ensembles

An Empirical Study of Building Compact Ensembles An Empirical Study of Building Compact Ensembles Huan Liu, Amit Mandvikar, and Jigar Mody Computer Science & Engineering Arizona State University Tempe, AZ 85281 {huan.liu,amitm,jigar.mody}@asu.edu Abstract.

More information

UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2014

UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2014 UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2014 Exam policy: This exam allows two one-page, two-sided cheat sheets (i.e. 4 sides); No other materials. Time: 2 hours. Be sure to write

More information

Expressive Power and Approximation Errors of Restricted Boltzmann Machines

Expressive Power and Approximation Errors of Restricted Boltzmann Machines Expressive Power and Approximation Errors of Restricted Boltzmann Machines Guido F. Montúfar Johannes Rauh Nihat Ay SFI WORKING PAPER: 2011-09-041 SFI Working Papers contain accounts of scientific work

More information

MDP Preliminaries. Nan Jiang. February 10, 2019

MDP Preliminaries. Nan Jiang. February 10, 2019 MDP Preliminaries Nan Jiang February 10, 2019 1 Markov Decision Processes In reinforcement learning, the interactions between the agent and the environment are often described by a Markov Decision Process

More information