Capabilities and Prospects of Inductive Modeling Volodymyr STEPASHKO
|
|
|
- Shonda Perkins
- 6 years ago
- Views:
Transcription
1 Capabilities and Prospects of Inductive Modeling Volodmr STEPASHKO Prof., Dr. Sci., Head of Department INFORMATION TECHNOLOGIES FOR INDUCTIVE MODELING International Research and Training Centre of the Academ of Sciences of Ukraine 1
2 Laout 1. Historical aspects of IM 2. International events on IM 3. Attempt to define IM: what is it? 4. IM destination: what is this for? 5. IM explanation: basic algorithms and tools 6. Basic Theoretical Results 7. IM compared to ANN and CI 8. Real-world applications of IM 9. Main centers of IM research 10. IM development prospects 2
3 1. Historical aspects of IM 1968 First publication on GMDH: Iвахненко O.Г. Метод групового урахування аргументів конкурент методу стохастичної апроксимації // Автоматика С Terminolog evolution: heuristic self-organization of models (1970s) inductive method of model building (1980s) inductive learning algorithms for modeling (1992) inductive modeling (1998) GMDH: Group Method of Data Handling MGUA: Method of Group Using of Arguments 3
4 A.G.Ivakhnenko: GMDH originator 4
5 Main scientific results in inductive modelling theor: Foundations of cbernetic forecasting device construction Theor of models self-organization b experimental data Group method of data handling (GMDH) for automatic construction (self-organization) of model for complex sstems Method of control with optimization of forecast Principles of noise-immunit modelling from nois data Principles of polnomial networks construction Principle of neural networks construction with active neurons 5
6 Academician Ivakhnenko Originator of the scientific school of inductive modelling Author of 44 monographs and numerous articls Prepared more than 200 Cand. Sci (Ph.D.) and 27 Doct. Sci 6
7 2. International events on IM 2002 Lviv, Ukraine 1 st International Conference on Inductive Modelling ICIM Kiv, Ukraine 1 st International Workshop on Inductive Modelling IWIM Prague, Czech Republic 2 nd International Workshop on Inductive Modelling IWIM Kiv, Ukraine 2 nd International Conference on Inductive Modelling ICIM Krnica, Poland 3 rd International Workshop on Inductive Modelling IWIM Yevpatoria, Crimea, Ukraine 3 rd International Conference on Inductive Modelling ICIM 2009 Zhukn (near Kiv, Ukraine) Annual International Summer School on Inductive Modelling 7
8 3. Attempt to define IM: what is it? IM is MGUA / GMDH IM is a technique for model self-organization IM is a technolog for building models from nois data IM is the technolog of inductive transition from data to models under uncertaint conditions: small volume of nois data unknown character and level of noise inexact composition of relevant arguments (factors) unknown structure of relationships in an object 8
9 4. IM destination: what is this for? IM is used for solving the following problems: Modelling from experimental data Forecasting of complex processes Structure and parametric identification Classification and pattern recognition Data clasterization Machine learning Data Mining Knowledge Discover 9
10 5. IM explanation: algorithms and tools Basic Principles of GMDH as an Inductive Method Given: data sample of n observations after m input x 1, x 2,, x m and output variables Find: model f(x 1, x 2,, x m,θ) with minimum variance of prediction error GMDH Task: f arg minc( f ), C( f ) model qualit criterion, I f I set of models Basic principles of the GMDH as an inductive method: 1. generation of variants of the graduall complicated structures of models 2. successive selection of the best variants using the freedom of decisions choice 3. external addition (due to the sample division) as the selection criterion Sample Part А Generation of models f I Part В Calculation of criterion С( f ) f * C min 10
11 Main stages of the modeling process D A T A (s a m p le, a p rio r in fo rm a tio n ) C h o ic e o f a m o d e l c la s s S tru c tu re g e n e ra tio n P a ra m e te r e s tim a tio n C rite rio n m in im iz a tio n A d e q u a c a n a l s is F in is h in g th e p ro c e s s 11
12 GMDH features Model Classes: linear, polnomial, autoregressive, difference (dnamic), nonlinear of network tpe etc. Parameter estimation: Least Squares Method (LSM) Model structure generators: GMDH Generators Sorting-out Iterative Exhaustive search Directed search Multilaered Relaxative 12
13 13 Main generators of models structures 1. Combinatorial: 1 1, 1,, 1,, ) ( s s j s i s l s F l i m s x X θ ) ) 2. Combinatorial-selective: 3. Selective (multilaered iterative): , ; 1,, 0,1,...;, ) ( ) ( F r j r i r j r i l r j l r i l r l C l F j i r ϑ ϑ ϑ ϑ ϑ ),...,, ( ; 1,...,2, 2 1 m m v v v d d d d v X θ )
14 14 External Selection Criteria Given sample: W (X ), X [nxm], [nx1] Division into two subsamples: n n n X X X W W W B A B A B A B A + ; ; ;,,,, ) ( 1 B W A G X X X G T G G T G G θ ) Parameter estimation for a model Xθ: Regularit criterion: 2 B W A X W X CB θ θ ) ) Unbiasedness criterion: 2 A B B B X AR θ )
15 IM tools Information Technolog ASTRID (Kiv) KnowledgeMiner (Frank Lemke, Berlin) FAKE GAME (Pavel Kordik at al., Prague) GMDHshell (Oleksi Koshulko, Kiv) 15
16 6. Basic Theoretical Results f * arg min C( f ). f F F set of model structures С criterion of a model qualit Structure of a model: ) f ), θ ( X f Estimation of parameters: ) θ f arg Q criterion of the qualit of model parameters estimation f min θ R f m ) Q( θ f ). 16
17 Main concept: Self-organizing evolution of the model of optimal complexit under uncertaint conditions Main result: Complexit of the optimum forecasting model depends on the level of uncertaint in the data: the higher it is, the simpler (more robust) there must be the optimum model Main conclusion: GMDH is the method for construction of models with minimum variance of forecasting error 17
18 6 J(s σ 2 ) σ 2 2,0 σ 2 1,5 6 J(σ 2 s) s 4 s 3 s 2 5 σ 2 1,0 5 s s J b (s) J(s 0) σ 2 0, σ s σ 2 кр(2,3) σ 2 кр(1,2) σ 2 кр(0,1) σ ,5 1 1,5 2 2,5 Illustration to the GMDH theor 18
19 7. IM compared to ANN and CI Selective (multilaered) GMDH algorithm: x 1 f 1 g 1 x 2 f 2 g 2 f x 3 f 3 g 3 x 4 f 4 g 4 m C 2 m F C 2 F F 19
20 Optimal structure of the multilaered net x 1 f 1 x 2 g 2 f x 3 f 3 x 4 m f 4 C 2 m F g 4 C 2 F F 20
21 8. Real-world applications of IM 1. Prediction of tax revenues and inflation 2. Modelling of ecological processes activit of microorganisms in soil under influence of heav metals irrigation of trees b processed wastewaters water ecolog 3. Sstem prediction of power indicators 4. Integral evaluation of the state of the complex multidimensional sstems economic safet investment activit ecological state of water reservoirs power safet 5. Technolog of informative-analtical support of operative management decisions 21
22 9. Main centers of IM research IRTC ITS NANU, Kiv, Ukraine NTUU KPI, Kiv, Ukraine KnowledgeMiner, Berlin, German CTU in Prague, Czech Sichuan Universit, Chengdu, China 22
23 10. IM development prospects The most promising directions: 1. Theoretical investigations 2. Integration of best developments of IM, NN and CI 3. Paralleling 4. Preprocessing 5. Ensembling 6. Intellectual interface 7. Case studes 23
24 THANK YOU! Volodmr STEPASHKO Address: Prof. Volodmr Stepashko, International Centre of ITS, Akademik Glushkov Prospekt 40, Kiv, MSP, 03680, Ukraine. Phone: +38 (044) Fax: +38 (044) Web: 24
Robust Pareto Design of GMDH-type Neural Networks for Systems with Probabilistic Uncertainties
. Hybrid GMDH-type algorithms and neural networks Robust Pareto Design of GMDH-type eural etworks for Systems with Probabilistic Uncertainties. ariman-zadeh, F. Kalantary, A. Jamali, F. Ebrahimi Department
USING GMDH IN ECOLOGICAL AND SOCIO-ECONOMICAL MONITORING PROBLEMS
Systems Analysis Modelling Simulation Vol. 43, No. 10, October 2003, pp. 1409-1414 USING GMDH IN ECOLOGICAL AND SOCIO-ECONOMICAL MONITORING PROBLEMS LYUDMILA SARYCHEVA* Institute of Geoinformatics, National
Agent-based distributed time series forecasting system
Journal of Theoretical and Applied Computer Science Vol. 9, No. 1, 2015, pp. 17-27 ISSN 2299-2634 (printed), 2300-5653 (online) http://www.jtacs.org Agent-based distributed time series forecasting system
FEEDBACK GMDH-TYPE NEURAL NETWORK AND ITS APPLICATION TO MEDICAL IMAGE ANALYSIS OF LIVER CANCER. Tadashi Kondo and Junji Ueno
International Journal of Innovative Computing, Information and Control ICIC International c 2012 ISSN 1349-4198 Volume 8, Number 3(B), March 2012 pp. 2285 2300 FEEDBACK GMDH-TYPE NEURAL NETWORK AND ITS
GMDH-type Neural Networks with a Feedback Loop and their Application to the Identification of Large-spatial Air Pollution Patterns.
GMDH-type Neural Networks with a Feedback Loop and their Application to the Identification of Large-spatial Air Pollution Patterns. Tadashi Kondo 1 and Abhijit S.Pandya 2 1 School of Medical Sci.,The Univ.of
Classification of Ordinal Data Using Neural Networks
Classification of Ordinal Data Using Neural Networks Joaquim Pinto da Costa and Jaime S. Cardoso 2 Faculdade Ciências Universidade Porto, Porto, Portugal jpcosta@fc.up.pt 2 Faculdade Engenharia Universidade
A Priori and A Posteriori Machine Learning and Nonlinear Artificial Neural Networks
A Priori and A Posteriori Machine Learning and Nonlinear Artificial Neural Networks Jan Zelinka, Jan Romportl, and Luděk Müller The Department of Cybernetics, University of West Bohemia, Czech Republic
Predict Time Series with Multiple Artificial Neural Networks
, pp. 313-324 http://dx.doi.org/10.14257/ijhit.2016.9.7.28 Predict Time Series with Multiple Artificial Neural Networks Fei Li 1, Jin Liu 1 and Lei Kong 2,* 1 College of Information Engineering, Shanghai
1 History of statistical/machine learning. 2 Supervised learning. 3 Two approaches to supervised learning. 4 The general learning procedure
Overview Breiman L (2001). Statistical modeling: The two cultures Statistical learning reading group Aleander L 1 Histor of statistical/machine learning 2 Supervised learning 3 Two approaches to supervised
Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td
Data Mining Andrew Kusiak 2139 Seamans Center Iowa City, Iowa 52242-1527 Preamble: Control Application Goal: Maintain T ~Td Tel: 319-335 5934 Fax: 319-335 5669 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak
Lecture 13 - Handling Nonlinearity
Lecture 3 - Handling Nonlinearit Nonlinearit issues in control practice Setpoint scheduling/feedforward path planning repla - linear interpolation Nonlinear maps B-splines Multivariable interpolation:
BACKPROPAGATION. Neural network training optimization problem. Deriving backpropagation
BACKPROPAGATION Neural network training optimization problem min J(w) w The application of gradient descent to this problem is called backpropagation. Backpropagation is gradient descent applied to J(w)
Learning Tetris. 1 Tetris. February 3, 2009
Learning Tetris Matt Zucker Andrew Maas February 3, 2009 1 Tetris The Tetris game has been used as a benchmark for Machine Learning tasks because its large state space (over 2 200 cell configurations are
CS534 Machine Learning - Spring Final Exam
CS534 Machine Learning - Spring 2013 Final Exam Name: You have 110 minutes. There are 6 questions (8 pages including cover page). If you get stuck on one question, move on to others and come back to the
Data Mining und Maschinelles Lernen
Data Mining und Maschinelles Lernen Ensemble Methods Bias-Variance Trade-off Basic Idea of Ensembles Bagging Basic Algorithm Bagging with Costs Randomization Random Forests Boosting Stacking Error-Correcting
Some Fundamental Topics of Inductive Modeling
2-nd International Conference on Inductive Modelling { ICIM'2008 Some Fundamental Topics of Inductive Modeling Yuriy V. Dzyadyk International Center of Information Technologies and Systems, Academician
EM-algorithm for Training of State-space Models with Application to Time Series Prediction
EM-algorithm for Training of State-space Models with Application to Time Series Prediction Elia Liitiäinen, Nima Reyhani and Amaury Lendasse Helsinki University of Technology - Neural Networks Research
AdaBoost. Lecturer: Authors: Center for Machine Perception Czech Technical University, Prague
AdaBoost Lecturer: Jan Šochman Authors: Jan Šochman, Jiří Matas Center for Machine Perception Czech Technical University, Prague http://cmp.felk.cvut.cz Motivation Presentation 2/17 AdaBoost with trees
ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD
ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD WHAT IS A NEURAL NETWORK? The simplest definition of a neural network, more properly referred to as an 'artificial' neural network (ANN), is provided
IPAM Summer School Optimization methods for machine learning. Jorge Nocedal
IPAM Summer School 2012 Tutorial on Optimization methods for machine learning Jorge Nocedal Northwestern University Overview 1. We discuss some characteristics of optimization problems arising in deep
Statistical Learning Reading Assignments
Statistical Learning Reading Assignments S. Gong et al. Dynamic Vision: From Images to Face Recognition, Imperial College Press, 2001 (Chapt. 3, hard copy). T. Evgeniou, M. Pontil, and T. Poggio, "Statistical
Chap 1. Overview of Statistical Learning (HTF, , 2.9) Yongdai Kim Seoul National University
Chap 1. Overview of Statistical Learning (HTF, 2.1-2.6, 2.9) Yongdai Kim Seoul National University 0. Learning vs Statistical learning Learning procedure Construct a claim by observing data or using logics
The Perceptron. Volker Tresp Summer 2016
The Perceptron Volker Tresp Summer 2016 1 Elements in Learning Tasks Collection, cleaning and preprocessing of training data Definition of a class of learning models. Often defined by the free model parameters
The Perceptron. Volker Tresp Summer 2018
The Perceptron Volker Tresp Summer 2018 1 Elements in Learning Tasks Collection, cleaning and preprocessing of training data Definition of a class of learning models. Often defined by the free model parameters
Hierarchical Boosting and Filter Generation
January 29, 2007 Plan Combining Classifiers Boosting Neural Network Structure of AdaBoost Image processing Hierarchical Boosting Hierarchical Structure Filters Combining Classifiers Combining Classifiers
Development of a Data Mining Methodology using Robust Design
Development of a Data Mining Methodology using Robust Design Sangmun Shin, Myeonggil Choi, Youngsun Choi, Guo Yi Department of System Management Engineering, Inje University Gimhae, Kyung-Nam 61-749 South
Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata
Principles of Pattern Recognition C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata e-mail: murthy@isical.ac.in Pattern Recognition Measurement Space > Feature Space >Decision
Class Diagrams. CSC 440/540: Software Engineering Slide #1
Class Diagrams CSC 440/540: Software Engineering Slide # Topics. Design class diagrams (DCDs) 2. DCD development process 3. Associations and Attributes 4. Dependencies 5. Composition and Constraints 6.
Brief Introduction of Machine Learning Techniques for Content Analysis
1 Brief Introduction of Machine Learning Techniques for Content Analysis Wei-Ta Chu 2008/11/20 Outline 2 Overview Gaussian Mixture Model (GMM) Hidden Markov Model (HMM) Support Vector Machine (SVM) Overview
CHAPTER 6 CONCLUSION AND FUTURE SCOPE
CHAPTER 6 CONCLUSION AND FUTURE SCOPE 146 CHAPTER 6 CONCLUSION AND FUTURE SCOPE 6.1 SUMMARY The first chapter of the thesis highlighted the need of accurate wind forecasting models in order to transform
Statistics and learning: Big Data
Statistics and learning: Big Data Learning Decision Trees and an Introduction to Boosting Sébastien Gadat Toulouse School of Economics February 2017 S. Gadat (TSE) SAD 2013 1 / 30 Keywords Decision trees
COMS 4771 Introduction to Machine Learning. Nakul Verma
COMS 4771 Introduction to Machine Learning Nakul Verma Announcements HW1 due next lecture Project details are available decide on the group and topic by Thursday Last time Generative vs. Discriminative
A Statistical Input Pruning Method for Artificial Neural Networks Used in Environmental Modelling
A Statistical Input Pruning Method for Artificial Neural Networks Used in Environmental Modelling G. B. Kingston, H. R. Maier and M. F. Lambert Centre for Applied Modelling in Water Engineering, School
Artificial Neural Network
Artificial Neural Network Contents 2 What is ANN? Biological Neuron Structure of Neuron Types of Neuron Models of Neuron Analogy with human NN Perceptron OCR Multilayer Neural Network Back propagation
Machine Learning And Applications: Supervised Learning-SVM
Machine Learning And Applications: Supervised Learning-SVM Raphaël Bournhonesque École Normale Supérieure de Lyon, Lyon, France raphael.bournhonesque@ens-lyon.fr 1 Supervised vs unsupervised learning Machine
Machine Learning and Adaptive Systems. Lectures 3 & 4
ECE656- Lectures 3 & 4, Professor Department of Electrical and Computer Engineering Colorado State University Fall 2015 What is Learning? General Definition of Learning: Any change in the behavior or performance
Selection of the Appropriate Lag Structure of Foreign Exchange Rates Forecasting Based on Autocorrelation Coefficient
Selection of the Appropriate Lag Structure of Foreign Exchange Rates Forecasting Based on Autocorrelation Coefficient Wei Huang 1,2, Shouyang Wang 2, Hui Zhang 3,4, and Renbin Xiao 1 1 School of Management,
Multivariate Methods in Statistical Data Analysis
Multivariate Methods in Statistical Data Analysis Web-Site: http://tmva.sourceforge.net/ See also: "TMVA - Toolkit for Multivariate Data Analysis, A. Hoecker, P. Speckmayer, J. Stelzer, J. Therhaag, E.
A Hybrid Method of CART and Artificial Neural Network for Short-term term Load Forecasting in Power Systems
A Hybrid Method of CART and Artificial Neural Network for Short-term term Load Forecasting in Power Systems Hiroyuki Mori Dept. of Electrical & Electronics Engineering Meiji University Tama-ku, Kawasaki
ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92
ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 BIOLOGICAL INSPIRATIONS Some numbers The human brain contains about 10 billion nerve cells (neurons) Each neuron is connected to the others through 10000
Tutorial on Machine Learning for Advanced Electronics
Tutorial on Machine Learning for Advanced Electronics Maxim Raginsky March 2017 Part I (Some) Theory and Principles Machine Learning: estimation of dependencies from empirical data (V. Vapnik) enabling
SPATIAL-TEMPORAL TECHNIQUES FOR PREDICTION AND COMPRESSION OF SOIL FERTILITY DATA
SPATIAL-TEMPORAL TECHNIQUES FOR PREDICTION AND COMPRESSION OF SOIL FERTILITY DATA D. Pokrajac Center for Information Science and Technology Temple University Philadelphia, Pennsylvania A. Lazarevic Computer
Ultimate State. MEM 355 Performance Enhancement of Dynamical Systems
Ultimate State MEM 355 Performance Enhancement of Dnamical Sstems Harr G. Kwatn Department of Mechanical Engineering & Mechanics Drexel Universit Outline Design Criteria two step process Ultimate state
Decision Trees (Cont.)
Decision Trees (Cont.) R&N Chapter 18.2,18.3 Side example with discrete (categorical) attributes: Predicting age (3 values: less than 30, 30-45, more than 45 yrs old) from census data. Attributes (split
Optimization Methods for Machine Learning (OMML)
Optimization Methods for Machine Learning (OMML) 2nd lecture (2 slots) Prof. L. Palagi 16/10/2014 1 What is (not) Data Mining? By Namwar Rizvi - Ad Hoc Query: ad Hoc queries just examines the current data
Machine Learning on temporal data
Machine Learning on temporal data Classification rees for ime Series Ahlame Douzal (Ahlame.Douzal@imag.fr) AMA, LIG, Université Joseph Fourier Master 2R - MOSIG (2011) Plan ime Series classification approaches
Hierarchical models for the rainfall forecast DATA MINING APPROACH
Hierarchical models for the rainfall forecast DATA MINING APPROACH Thanh-Nghi Do dtnghi@cit.ctu.edu.vn June - 2014 Introduction Problem large scale GCM small scale models Aim Statistical downscaling local
Lecture 24: Other (Non-linear) Classifiers: Decision Tree Learning, Boosting, and Support Vector Classification Instructor: Prof. Ganesh Ramakrishnan
Lecture 24: Other (Non-linear) Classifiers: Decision Tree Learning, Boosting, and Support Vector Classification Instructor: Prof Ganesh Ramakrishnan October 20, 2016 1 / 25 Decision Trees: Cascade of step
Predicting Future Energy Consumption CS229 Project Report
Predicting Future Energy Consumption CS229 Project Report Adrien Boiron, Stephane Lo, Antoine Marot Abstract Load forecasting for electric utilities is a crucial step in planning and operations, especially
Artificial Neural Networks
Introduction ANN in Action Final Observations Application: Poverty Detection Artificial Neural Networks Alvaro J. Riascos Villegas University of los Andes and Quantil July 6 2018 Artificial Neural Networks
Lecture 4: Perceptrons and Multilayer Perceptrons
Lecture 4: Perceptrons and Multilayer Perceptrons Cognitive Systems II - Machine Learning SS 2005 Part I: Basic Approaches of Concept Learning Perceptrons, Artificial Neuronal Networks Lecture 4: Perceptrons
A FUZZY NEURAL NETWORK MODEL FOR FORECASTING STOCK PRICE
A FUZZY NEURAL NETWORK MODEL FOR FORECASTING STOCK PRICE Li Sheng Institute of intelligent information engineering Zheiang University Hangzhou, 3007, P. R. China ABSTRACT In this paper, a neural network-driven
CSE 417T: Introduction to Machine Learning. Final Review. Henry Chai 12/4/18
CSE 417T: Introduction to Machine Learning Final Review Henry Chai 12/4/18 Overfitting Overfitting is fitting the training data more than is warranted Fitting noise rather than signal 2 Estimating! "#$
Introduction to Machine Learning
10-701 Introduction to Machine Learning PCA Slides based on 18-661 Fall 2018 PCA Raw data can be Complex, High-dimensional To understand a phenomenon we measure various related quantities If we knew what
Influence of knn-based Load Forecasting Errors on Optimal Energy Production
Influence of knn-based Load Forecasting Errors on Optimal Energy Production Alicia Troncoso Lora 1, José C. Riquelme 1, José Luís Martínez Ramos 2, Jesús M. Riquelme Santos 2, and Antonio Gómez Expósito
An Evolution Strategy for the Induction of Fuzzy Finite-state Automata
Journal of Mathematics and Statistics 2 (2): 386-390, 2006 ISSN 1549-3644 Science Publications, 2006 An Evolution Strategy for the Induction of Fuzzy Finite-state Automata 1,2 Mozhiwen and 1 Wanmin 1 College
Security Analytics. Topic 6: Perceptron and Support Vector Machine
Security Analytics Topic 6: Perceptron and Support Vector Machine Purdue University Prof. Ninghui Li Based on slides by Prof. Jenifer Neville and Chris Clifton Readings Principle of Data Mining Chapter
CS6375: Machine Learning Gautam Kunapuli. Decision Trees
Gautam Kunapuli Example: Restaurant Recommendation Example: Develop a model to recommend restaurants to users depending on their past dining experiences. Here, the features are cost (x ) and the user s
Perceptron. (c) Marcin Sydow. Summary. Perceptron
Topics covered by this lecture: Neuron and its properties Mathematical model of neuron: as a classier ' Learning Rule (Delta Rule) Neuron Human neural system has been a natural source of inspiration for
Different Criteria for Active Learning in Neural Networks: A Comparative Study
Different Criteria for Active Learning in Neural Networks: A Comparative Study Jan Poland and Andreas Zell University of Tübingen, WSI - RA Sand 1, 72076 Tübingen, Germany Abstract. The field of active
Non-linear Measure Based Process Monitoring and Fault Diagnosis
Non-linear Measure Based Process Monitoring and Fault Diagnosis 12 th Annual AIChE Meeting, Reno, NV [275] Data Driven Approaches to Process Control 4:40 PM, Nov. 6, 2001 Sandeep Rajput Duane D. Bruns
Machine Learning Ensemble Learning I Hamid R. Rabiee Jafar Muhammadi, Alireza Ghasemi Spring /
Machine Learning Ensemble Learning I Hamid R. Rabiee Jafar Muhammadi, Alireza Ghasemi Spring 2015 http://ce.sharif.edu/courses/93-94/2/ce717-1 / Agenda Combining Classifiers Empirical view Theoretical
A Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation
A Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation Vu Malbasa and Slobodan Vucetic Abstract Resource-constrained data mining introduces many constraints when learning from
Learning with multiple models. Boosting.
CS 2750 Machine Learning Lecture 21 Learning with multiple models. Boosting. Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Learning with multiple models: Approach 2 Approach 2: use multiple models
Neural Networks: Optimization & Regularization
Neural Networks: Optimization & Regularization Shan-Hung Wu shwu@cs.nthu.edu.tw Department of Computer Science, National Tsing Hua University, Taiwan Machine Learning Shan-Hung Wu (CS, NTHU) NN Opt & Reg
Gradient-Based Learning. Sargur N. Srihari
Gradient-Based Learning Sargur N. srihari@cedar.buffalo.edu 1 Topics Overview 1. Example: Learning XOR 2. Gradient-Based Learning 3. Hidden Units 4. Architecture Design 5. Backpropagation and Other Differentiation
Neural Networks and Deep Learning
Neural Networks and Deep Learning Professor Ameet Talwalkar November 12, 2015 Professor Ameet Talwalkar Neural Networks and Deep Learning November 12, 2015 1 / 16 Outline 1 Review of last lecture AdaBoost
Data Mining Part 5. Prediction
Data Mining Part 5. Prediction 5.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline How the Brain Works Artificial Neural Networks Simple Computing Elements Feed-Forward Networks Perceptrons (Single-layer,
Statistical Machine Learning from Data
Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Ensembles Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique Fédérale de Lausanne
New Neural Architectures and New Adaptive Evaluation of Chaotic Time Series
New Neural Architectures and New Adaptive Evaluation of Chaotic Time Series Tutorial for the 2008-IEEE-ICAL Sunday, August 31, 2008 3 Hours Ivo Bukovsky, Jiri Bila, Madan M. Gupta, and Zeng-Guang Hou OUTLINES
Nonlinear Optimization Methods for Machine Learning
Nonlinear Optimization Methods for Machine Learning Jorge Nocedal Northwestern University University of California, Davis, Sept 2018 1 Introduction We don t really know, do we? a) Deep neural networks
Boosting. CAP5610: Machine Learning Instructor: Guo-Jun Qi
Boosting CAP5610: Machine Learning Instructor: Guo-Jun Qi Weak classifiers Weak classifiers Decision stump one layer decision tree Naive Bayes A classifier without feature correlations Linear classifier
Machine learning for pervasive systems Classification in high-dimensional spaces
Machine learning for pervasive systems Classification in high-dimensional spaces Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Version
Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning
Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function
A Wavelet Neural Network Forecasting Model Based On ARIMA
A Wavelet Neural Network Forecasting Model Based On ARIMA Wang Bin*, Hao Wen-ning, Chen Gang, He Deng-chao, Feng Bo PLA University of Science &Technology Nanjing 210007, China e-mail:lgdwangbin@163.com
Electric Load Forecasting Using Wavelet Transform and Extreme Learning Machine
Electric Load Forecasting Using Wavelet Transform and Extreme Learning Machine Song Li 1, Peng Wang 1 and Lalit Goel 1 1 School of Electrical and Electronic Engineering Nanyang Technological University
Multilayer Neural Networks
Pattern Recognition Multilaer Neural Networs Lecture 4 Prof. Daniel Yeung School of Computer Science and Engineering South China Universit of Technolog Outline Introduction (6.) Artificial Neural Networ
Estimating Gaussian Mixture Densities with EM A Tutorial
Estimating Gaussian Mixture Densities with EM A Tutorial Carlo Tomasi Due University Expectation Maximization (EM) [4, 3, 6] is a numerical algorithm for the maximization of functions of several variables
Data Mining: Concepts and Techniques. (3 rd ed.) Chapter 8. Chapter 8. Classification: Basic Concepts
Data Mining: Concepts and Techniques (3 rd ed.) Chapter 8 1 Chapter 8. Classification: Basic Concepts Classification: Basic Concepts Decision Tree Induction Bayes Classification Methods Rule-Based Classification
Ensemble Methods and Random Forests
Ensemble Methods and Random Forests Vaishnavi S May 2017 1 Introduction We have seen various analysis for classification and regression in the course. One of the common methods to reduce the generalization
CS60021: Scalable Data Mining. Large Scale Machine Learning
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, http://www.mmds.org 1 CS60021: Scalable Data Mining Large Scale Machine Learning Sourangshu Bhattacharya Example: Spam filtering Instance
Neural Nets in PR. Pattern Recognition XII. Michal Haindl. Outline. Neural Nets in PR 2
Neural Nets in PR NM P F Outline Motivation: Pattern Recognition XII human brain study complex cognitive tasks Michal Haindl Faculty of Information Technology, KTI Czech Technical University in Prague
Machine Learning for natural language processing
Machine Learning for natural language processing Classification: Maximum Entropy Models Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Summer 2016 1 / 24 Introduction Classification = supervised
Support Vector Machine. Industrial AI Lab. Prof. Seungchul Lee
Support Vector Machine Industrial AI Lab. Prof. Seungchul Lee Classification (Linear) Autonomously figure out which category (or class) an unknown item should be categorized into Number of categories /
Predicting Floods in North Central Province of Sri Lanka using Machine Learning and Data Mining Methods
Thilakarathne & Premachandra Predicting Floods in North Central Province of Sri Lanka using Machine Learning and Data Mining Methods H. Thilakarathne 1, K. Premachandra 2 1 Department of Physical Science,
Tools of AI. Marcin Sydow. Summary. Machine Learning
Machine Learning Outline of this Lecture Motivation for Data Mining and Machine Learning Idea of Machine Learning Decision Table: Cases and Attributes Supervised and Unsupervised Learning Classication
Learning Linear Detectors
Learning Linear Detectors Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Detection versus Classification Bayes Classifiers Linear Classifiers Examples of Detection 3 Learning: Detection
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
Artificial Neural Networks. MGS Lecture 2
Artificial Neural Networks MGS 2018 - Lecture 2 OVERVIEW Biological Neural Networks Cell Topology: Input, Output, and Hidden Layers Functional description Cost functions Training ANNs Back-Propagation
ARTIFICIAL NEURAL NETWORK WITH HYBRID TAGUCHI-GENETIC ALGORITHM FOR NONLINEAR MIMO MODEL OF MACHINING PROCESSES
International Journal of Innovative Computing, Information and Control ICIC International c 2013 ISSN 1349-4198 Volume 9, Number 4, April 2013 pp. 1455 1475 ARTIFICIAL NEURAL NETWORK WITH HYBRID TAGUCHI-GENETIC
ROBUST ESTIMATOR FOR MULTIPLE INLIER STRUCTURES
ROBUST ESTIMATOR FOR MULTIPLE INLIER STRUCTURES Xiang Yang (1) and Peter Meer (2) (1) Dept. of Mechanical and Aerospace Engineering (2) Dept. of Electrical and Computer Engineering Rutgers University,
Holdout and Cross-Validation Methods Overfitting Avoidance
Holdout and Cross-Validation Methods Overfitting Avoidance Decision Trees Reduce error pruning Cost-complexity pruning Neural Networks Early stopping Adjusting Regularizers via Cross-Validation Nearest
TDT4173 Machine Learning
TDT4173 Machine Learning Lecture 3 Bagging & Boosting + SVMs Norwegian University of Science and Technology Helge Langseth IT-VEST 310 helgel@idi.ntnu.no 1 TDT4173 Machine Learning Outline 1 Ensemble-methods
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2014
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2014 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
ADAPTIVE FILTER THEORY
ADAPTIVE FILTER THEORY Fourth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada Front ice Hall PRENTICE HALL Upper Saddle River, New Jersey 07458 Preface
Research Article Chaotic Attractor Generation via a Simple Linear Time-Varying System
Discrete Dnamics in Nature and Societ Volume, Article ID 836, 8 pages doi:.//836 Research Article Chaotic Attractor Generation via a Simple Linear Time-Varing Sstem Baiu Ou and Desheng Liu Department of
Statistical foundations
Statistical foundations Michael K. Tippett International Research Institute for Climate and Societ The Earth Institute, Columbia Universit ERFS Climate Predictabilit Tool Training Workshop Ma 4-9, 29 Ideas
Identification of Nonlinear Dynamic Systems with Multiple Inputs and Single Output using discrete-time Volterra Type Equations
Identification of Nonlinear Dnamic Sstems with Multiple Inputs and Single Output using discrete-time Volterra Tpe Equations Thomas Treichl, Stefan Hofmann, Dierk Schröder Institute for Electrical Drive
Optimal transfer function neural networks
Optimal transfer function neural networks Norbert Jankowski and Włodzisław Duch Department of Computer ethods Nicholas Copernicus University ul. Grudziądzka, 87 Toru ń, Poland, e-mail:{norbert,duch}@phys.uni.torun.pl
Choosing Variables with a Genetic Algorithm for Econometric models based on Neural Networks learning and adaptation.
Choosing Variables with a Genetic Algorithm for Econometric models based on Neural Networks learning and adaptation. Daniel Ramírez A., Israel Truijillo E. LINDA LAB, Computer Department, UNAM Facultad