Lecture 13 - Handling Nonlinearity

Size: px
Start display at page:

Download "Lecture 13 - Handling Nonlinearity"

Transcription

1 Lecture 3 - Handling Nonlinearit Nonlinearit issues in control practice Setpoint scheduling/feedforward path planning repla - linear interpolation Nonlinear maps B-splines Multivariable interpolation: polnomials/splines/rbf Neural Networks Fuzz logic Gain scheduling Local modeling EE39m - Winter 003 Control Engineering 3-

2 Nonlinearit in control practice Here are the nonlinearities we alread looked into Constraints - saturation in control anti-windup in PID control MPC handles the constraints Control program, path planning Static optimization Nonlinear dnamics dnamic inversion nonlinear IMC nonlinear MPC One additional nonlinearit in this lecture Controller gain scheduling EE39m - Winter 003 Control Engineering 3-

3 Dealing with nonlinear functions Analtical epressions models are given b analtical formulas, computable as required rarel sufficient in practice Models are computable off line pre-compute simple approimation on-line approimation Models contain data identified in the eperiments nonlinear maps interpolation or look-up tables Advanced approimation methods neural networks EE39m - Winter 003 Control Engineering 3-3

4 EE39m - Winter 003 Control Engineering 3-4 Path planning Real-time repla of a pre-computed reference traector d t or feedforward vt Reproduce a nonlinear function d t in a control sstem Path planner, data arras Y,Θ d t t d t Y t Y t Θ n n d n d d Y Y Y Y M M, Code:. Find, such that. Compute t

5 Linear interpolation vs. table look-up linear interpolation is more accurate requires less data storage simple computation EE39m - Winter 003 Control Engineering 3-5

6 Empirical models Aerospace - most developed nonlinear approaches automotive and process control have second place Aerodnamic tables Engine maps et turbines automotive Process maps, e.g., in semiconductor manufacturing Empirical map for a attenuation vs. temperature in an optical fiber Eample TEFTrailing Edge Flap EE39m - Winter 003 Control Engineering 3-6

7 Approimation Interpolation: compute function that will provide given values in the nodes not concerned with accurac in-between the nodes Approimation compute function that closel corresponds to given data, possibl with some error might provide better accurac throughout Y EE39m - Winter 003 Control Engineering 3-7

8 B-spline interpolation st-order look-up table, nearest neighbor nd-order linear interpolation d t Y B t n-th order: Piece-wise n-th order polnomials, matched n- derivatives zero outside a local support interval support interval etends to n nearest neighbors EE39m - Winter 003 Control Engineering 3-8

9 B-splines Accurate interpolation of smooth functions with relative few nodes For -D function the gain from using high-order B-splines is not worth an added compleit Introduced and developed in CAD for -D and 3-D curve and surface data Are used for defining multidimensional nonlinear maps EE39m - Winter 003 Control Engineering 3-9

10 Multivariable B-splines Regular grid in multiple variables Tensor product B-splines Used as a basis of finite-element models u, v w, k, kb u Bk v EE39m - Winter 003 Control Engineering 3-0

11 EE39m - Winter 003 Control Engineering 3- Linear regression for nonlinear map Linear regression Multidimensional B-splines Multivariate polnomials RBF - Radial Basis Functions T φ ϕ k n n k n,, K K ϕ K c a e c R ϕ n M

12 Linear regression approimation Nonlinear map data available at scattered nodal points Y [ N K ] K N Linear regression map Linear regression approimation Y [ N ] T φ K φ Φ T regularized least square estimate of the weight vector ˆ T T ΦΦ ri ΦY Works ust the same for vector-valued data! EE39m - Winter 003 Control Engineering 3-

13 Nonlinear map eample - Epi Epitaial growth semiconductor process process map for run-to-run control EE39m - Winter 003 Control Engineering 3-3

14 EE39m - Winter 003 Control Engineering 3-4 Linear regression for Epi map Linear regression model for epitaial grouth 0 p c p c w w w w p { { { { w c w c w c w c p c { { { { w c c v c v c v p c,,, T φ ϕ

15 Neural Networks An nonlinear approimator might be called a Neural Network RBF Neural Network Polnomial Neural Network Linear in parameters B-spline Neural Network Wavelet Neural Network MPL - Multilaered Perceptron Nonlinear in parameters Works for man inputs w,0 f w,, w,0 f w, f e f EE39m - Winter 003 Control Engineering 3-5

16 Multi-Laered Perceptrons Network parameter computation training data set parameter identification F ; Noninear LS problem V F Iterative NLS optimization Levenberg-Marquardt Backpropagation ; variation of a gradient descent min Y [ N K ] K N EE39m - Winter 003 Control Engineering 3-6

17 Fuzz Logic Function defined at nodes. Interpolation scheme Fuzzfication/de-fuzzfication interpolation Linear interpolation in -D µ µ µ Marketing communication and social value Computer science: emphasis on interaction with a user EE - emphasis on mathematical computations EE39m - Winter 003 Control Engineering 3-7

18 Neural Net application Internal Combustion Engine maps Eperimental map: data collected in a stead state regime for various combinations of parameters -D table NN map approimation of the eperimental map MLP was used in this eample works better for a smooth surface spark advance RPM EE39m - Winter 003 Control Engineering 3-8

19 Linear feedback in a nonlinear plant Simple eample f u k g u d u ff Linearizing propert of feedback for k >> d k g f g u ff d Eample: Control design requires varing k, u ff, d process These variables are gain scheduled on d u Controller Plant EE39m - Winter 003 Control Engineering 3-9

20 Gain scheduling Single out several regimes - model linearization or eperiments Design linear controllers in these regimes: setpoint, feedback, feedforward Approimate controller dependence on the regime parameters Nonlinear sstem Linear interpolation: Y Θ ϕ Θ EE39m - Winter 003 Control Engineering 3-0 Y Y vec A vec B vec C vec D

21 Gain scheduling - eample Flight control Flight envelope parameters are used for scheduling Shown Approimation nodes Evaluation points Ke assumption Attitude and Mach are changing much slower than time constant of the flight control loop EE39m - Winter 003 Control Engineering 3-

22 Local Modeling Based on Data Data mining in the loop Honewell product Multidimensional Data Cube Heat Loads Relational Database Heat demand Quer point What if? Outdoor temperature Time of da Forecasted variable Eplanator variables EE39m - Winter 003 Control Engineering 3-

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning

Lecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function

More information

What is Fuzzy Logic? Fuzzy logic is a tool for embedding human knowledge (experience, expertise, heuristics) Fuzzy Logic

What is Fuzzy Logic? Fuzzy logic is a tool for embedding human knowledge (experience, expertise, heuristics) Fuzzy Logic Fuzz Logic Andrew Kusiak 239 Seamans Center Iowa Cit, IA 52242 527 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak (Based on the material provided b Professor V. Kecman) What is Fuzz Logic?

More information

Chapter 4 Neural Networks in System Identification

Chapter 4 Neural Networks in System Identification Chapter 4 Neural Networks in System Identification Gábor HORVÁTH Department of Measurement and Information Systems Budapest University of Technology and Economics Magyar tudósok körútja 2, 52 Budapest,

More information

Introduction to Neural Networks: Structure and Training

Introduction to Neural Networks: Structure and Training Introduction to Neural Networks: Structure and Training Professor Q.J. Zhang Department of Electronics Carleton University, Ottawa, Canada www.doe.carleton.ca/~qjz, qjz@doe.carleton.ca A Quick Illustration

More information

Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter

Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter (Chair) STF - China Fellow francesco.dimaio@polimi.it

More information

Nonlinear System Identification Using MLP Dr.-Ing. Sudchai Boonto

Nonlinear System Identification Using MLP Dr.-Ing. Sudchai Boonto Dr-Ing Sudchai Boonto Department of Control System and Instrumentation Engineering King Mongkut s Unniversity of Technology Thonburi Thailand Nonlinear System Identification Given a data set Z N = {y(k),

More information

Neuro-Fuzzy Comp. Ch. 4 June 1, 2005

Neuro-Fuzzy Comp. Ch. 4 June 1, 2005 Neuro-Fuzz Comp Ch 4 June, 25 4 Feedforward Multilaer Neural Networks part I Feedforward multilaer neural networks (introduced in sec 7) with supervised error correcting learning are used to approximate

More information

MODELLING ENERGY DEMAND FORECASTING USING NEURAL NETWORKS WITH UNIVARIATE TIME SERIES

MODELLING ENERGY DEMAND FORECASTING USING NEURAL NETWORKS WITH UNIVARIATE TIME SERIES MODELLING ENERGY DEMAND FORECASTING USING NEURAL NETWORKS WITH UNIVARIATE TIME SERIES S. Cankurt 1, M. Yasin 2 1&2 Ishik University Erbil, Iraq 1 s.cankurt@ishik.edu.iq, 2 m.yasin@ishik.edu.iq doi:10.23918/iec2018.26

More information

Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011!

Artificial Neural Networks and Nonparametric Methods CMPSCI 383 Nov 17, 2011! Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011! 1 Todayʼs lecture" How the brain works (!)! Artificial neural networks! Perceptrons! Multilayer feed-forward networks! Error

More information

CS599 Lecture 2 Function Approximation in RL

CS599 Lecture 2 Function Approximation in RL CS599 Lecture 2 Function Approximation in RL Look at how experience with a limited part of the state set be used to produce good behavior over a much larger part. Overview of function approximation (FA)

More information

Neural Networks, Computation Graphs. CMSC 470 Marine Carpuat

Neural Networks, Computation Graphs. CMSC 470 Marine Carpuat Neural Networks, Computation Graphs CMSC 470 Marine Carpuat Binary Classification with a Multi-layer Perceptron φ A = 1 φ site = 1 φ located = 1 φ Maizuru = 1 φ, = 2 φ in = 1 φ Kyoto = 1 φ priest = 0 φ

More information

Using Neural Networks for Identification and Control of Systems

Using Neural Networks for Identification and Control of Systems Using Neural Networks for Identification and Control of Systems Jhonatam Cordeiro Department of Industrial and Systems Engineering North Carolina A&T State University, Greensboro, NC 27411 jcrodrig@aggies.ncat.edu

More information

Multilayer Perceptron = FeedForward Neural Network

Multilayer Perceptron = FeedForward Neural Network Multilayer Perceptron = FeedForward Neural Networ History Definition Classification = feedforward operation Learning = bacpropagation = local optimization in the space of weights Pattern Classification

More information

Deep Nonlinear Non-Gaussian Filtering for Dynamical Systems

Deep Nonlinear Non-Gaussian Filtering for Dynamical Systems Deep Nonlinear Non-Gaussian Filtering for Dnamical Sstems Arash Mehrjou Department of Empirical Inference Ma Planck Institute for Intelligent Sstems arash.mehrjou@tuebingen.mpg.de Bernhard Schölkopf Department

More information

Neural Networks Lecture 4: Radial Bases Function Networks

Neural Networks Lecture 4: Radial Bases Function Networks Neural Networks Lecture 4: Radial Bases Function Networks H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011. A. Talebi, Farzaneh Abdollahi

More information

Discrete-time models and control

Discrete-time models and control Discrete-time models and control Silvano Balemi University of Applied Sciences of Southern Switzerland Zürich, 2009-2010 Discrete-time signals 1 Step response of a sampled system Sample and hold 2 Sampling

More information

ARTIFICIAL INTELLIGENCE. Artificial Neural Networks

ARTIFICIAL INTELLIGENCE. Artificial Neural Networks INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Artificial Neural Networks Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html

More information

Feedforward Neural Networks

Feedforward Neural Networks Chapter 4 Feedforward Neural Networks 4. Motivation Let s start with our logistic regression model from before: P(k d) = softma k =k ( λ(k ) + w d λ(k, w) ). (4.) Recall that this model gives us a lot

More information

A NOVEL METHOD OF INTERPOLATION AND EXTRAPOLATION OF FUNCTIONS BY A LINEAR INITIAL VALUE PROBLEM

A NOVEL METHOD OF INTERPOLATION AND EXTRAPOLATION OF FUNCTIONS BY A LINEAR INITIAL VALUE PROBLEM A OVEL METHOD OF ITERPOLATIO AD EXTRAPOLATIO OF FUCTIOS BY A LIEAR IITIAL VALUE PROBLEM Michael Shatalov Sensor Science and Technolog o CSIR Manuacturing and Materials, P.O.Bo 395, Pretoria, CSIR and Department

More information

ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92

ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 BIOLOGICAL INSPIRATIONS Some numbers The human brain contains about 10 billion nerve cells (neurons) Each neuron is connected to the others through 10000

More information

CSTR CONTROL USING MULTIPLE MODELS

CSTR CONTROL USING MULTIPLE MODELS CSTR CONTROL USING MULTIPLE MODELS J. Novák, V. Bobál Univerzita Tomáše Bati, Fakulta aplikované informatiky Mostní 39, Zlín INTRODUCTION Almost every real process exhibits nonlinear behavior in a full

More information

AE/ME 339. K. M. Isaac Professor of Aerospace Engineering. December 21, 2001 topic13_grid_generation 1

AE/ME 339. K. M. Isaac Professor of Aerospace Engineering. December 21, 2001 topic13_grid_generation 1 AE/ME 339 Professor of Aerospace Engineering December 21, 2001 topic13_grid_generation 1 The basic idea behind grid generation is the creation of the transformation laws between the phsical space and the

More information

RANGE CONTROL MPC APPROACH FOR TWO-DIMENSIONAL SYSTEM 1

RANGE CONTROL MPC APPROACH FOR TWO-DIMENSIONAL SYSTEM 1 RANGE CONTROL MPC APPROACH FOR TWO-DIMENSIONAL SYSTEM Jirka Roubal Vladimír Havlena Department of Control Engineering, Facult of Electrical Engineering, Czech Technical Universit in Prague Karlovo náměstí

More information

Introduction to Neural Networks

Introduction to Neural Networks Introduction to Neural Networks What are (Artificial) Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning

More information

CSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?

CSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating? CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to

More information

Identification of Nonlinear Systems Using Neural Networks and Polynomial Models

Identification of Nonlinear Systems Using Neural Networks and Polynomial Models A.Janczak Identification of Nonlinear Systems Using Neural Networks and Polynomial Models A Block-Oriented Approach With 79 Figures and 22 Tables Springer Contents Symbols and notation XI 1 Introduction

More information

Chapter 8: Generalization and Function Approximation

Chapter 8: Generalization and Function Approximation Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to produce good behavior over a much larger part. Overview

More information

CSC242: Intro to AI. Lecture 21

CSC242: Intro to AI. Lecture 21 CSC242: Intro to AI Lecture 21 Administrivia Project 4 (homeworks 18 & 19) due Mon Apr 16 11:59PM Posters Apr 24 and 26 You need an idea! You need to present it nicely on 2-wide by 4-high landscape pages

More information

Lecture 4.2 Finite Difference Approximation

Lecture 4.2 Finite Difference Approximation Lecture 4. Finite Difference Approimation 1 Discretization As stated in Lecture 1.0, there are three steps in numerically solving the differential equations. They are: 1. Discretization of the domain by

More information

CSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?

CSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating? CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to

More information

Analysis of Fast Input Selection: Application in Time Series Prediction

Analysis of Fast Input Selection: Application in Time Series Prediction Analysis of Fast Input Selection: Application in Time Series Prediction Jarkko Tikka, Amaury Lendasse, and Jaakko Hollmén Helsinki University of Technology, Laboratory of Computer and Information Science,

More information

Radial Basis Function (RBF) Networks

Radial Basis Function (RBF) Networks CSE 5526: Introduction to Neural Networks Radial Basis Function (RBF) Networks 1 Function approximation We have been using MLPs as pattern classifiers But in general, they are function approximators Depending

More information

Unit III. A Survey of Neural Network Model

Unit III. A Survey of Neural Network Model Unit III A Survey of Neural Network Model 1 Single Layer Perceptron Perceptron the first adaptive network architecture was invented by Frank Rosenblatt in 1957. It can be used for the classification of

More information

Lecture 1: Course Introduction.

Lecture 1: Course Introduction. Lecture : Course Introduction. What is the Finite Element Method (FEM)? a numerical method for solving problems of engineering and mathematical physics. (Logan Pg. #). In MECH 40 we are concerned with

More information

Multilayer Perceptrons (MLPs)

Multilayer Perceptrons (MLPs) CSE 5526: Introduction to Neural Networks Multilayer Perceptrons (MLPs) 1 Motivation Multilayer networks are more powerful than singlelayer nets Example: XOR problem x 2 1 AND x o x 1 x 2 +1-1 o x x 1-1

More information

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur Module 12 Machine Learning Lesson 39 Neural Networks - III 12.4.4 Multi-Layer Perceptrons In contrast to perceptrons, multilayer networks can learn not only multiple decision boundaries, but the boundaries

More information

Intro to Neural Networks and Deep Learning

Intro to Neural Networks and Deep Learning Intro to Neural Networks and Deep Learning Jack Lanchantin Dr. Yanjun Qi UVA CS 6316 1 Neurons 1-Layer Neural Network Multi-layer Neural Network Loss Functions Backpropagation Nonlinearity Functions NNs

More information

Artificial Neuron (Perceptron)

Artificial Neuron (Perceptron) 9/6/208 Gradient Descent (GD) Hantao Zhang Deep Learning with Python Reading: https://en.wikipedia.org/wiki/gradient_descent Artificial Neuron (Perceptron) = w T = w 0 0 + + w 2 2 + + w d d where

More information

Wind Power Forecasting using Artificial Neural Networks

Wind Power Forecasting using Artificial Neural Networks Wind Power Forecasting using Artificial Neural Networks This paper aims at predicting the power output of wind turbines using artificial neural networks,two different algorithms and models were trained

More information

Seminar Course 392N Spring2011. ee392n - Spring 2011 Stanford University. Intelligent Energy Systems 1

Seminar Course 392N Spring2011. ee392n - Spring 2011 Stanford University. Intelligent Energy Systems 1 Seminar Course 392N Spring211 Lecture 3 Intelligent Energy Systems: Control and Monitoring Basics Dimitry Gorinevsky Intelligent Energy Systems 1 Traditional Grid Worlds Largest Machine! 33 utilities 15,

More information

Chapter 2 Review of Linear and Nonlinear Controller Designs

Chapter 2 Review of Linear and Nonlinear Controller Designs Chapter 2 Review of Linear and Nonlinear Controller Designs This Chapter reviews several flight controller designs for unmanned rotorcraft. 1 Flight control systems have been proposed and tested on a wide

More information

SPSS, University of Texas at Arlington. Topics in Machine Learning-EE 5359 Neural Networks

SPSS, University of Texas at Arlington. Topics in Machine Learning-EE 5359 Neural Networks Topics in Machine Learning-EE 5359 Neural Networks 1 The Perceptron Output: A perceptron is a function that maps D-dimensional vectors to real numbers. For notational convenience, we add a zero-th dimension

More information

CS545 Contents XVI. l Adaptive Control. l Reading Assignment for Next Class

CS545 Contents XVI. l Adaptive Control. l Reading Assignment for Next Class CS545 Contents XVI Adaptive Control Model Reference Adaptive Control Self-Tuning Regulators Linear Regression Recursive Least Squares Gradient Descent Feedback-Error Learning Reading Assignment for Next

More information

CS325 Artificial Intelligence Chs. 18 & 4 Supervised Machine Learning (cont)

CS325 Artificial Intelligence Chs. 18 & 4 Supervised Machine Learning (cont) CS325 Artificial Intelligence Cengiz Spring 2013 Model Complexity in Learning f(x) x Model Complexity in Learning f(x) x Let s start with the linear case... Linear Regression Linear Regression price =

More information

GT-POWER linearization and engine advanced control design applications

GT-POWER linearization and engine advanced control design applications GT-POWER linearization and engine advanced control design applications Kenny Follen Ali Borhan Ed Hodzen Cummins Inc. North American GT Conference 2016 November 14-15, 2016 Michigan, USA Outline Background

More information

A new method for short-term load forecasting based on chaotic time series and neural network

A new method for short-term load forecasting based on chaotic time series and neural network A new method for short-term load forecasting based on chaotic time series and neural network Sajjad Kouhi*, Navid Taghizadegan Electrical Engineering Department, Azarbaijan Shahid Madani University, Tabriz,

More information

Soft and hard models. Jiří Militky Computer assisted statistical modeling in the

Soft and hard models. Jiří Militky Computer assisted statistical modeling in the Soft and hard models Jiří Militky Computer assisted statistical s modeling in the applied research Monsters Giants Moore s law: processing capacity doubles every 18 months : CPU, cache, memory It s more

More information

Keywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm

Keywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm Volume 4, Issue 5, May 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Huffman Encoding

More information

Machine Learning (CSE 446): Neural Networks

Machine Learning (CSE 446): Neural Networks Machine Learning (CSE 446): Neural Networks Noah Smith c 2017 University of Washington nasmith@cs.washington.edu November 6, 2017 1 / 22 Admin No Wednesday office hours for Noah; no lecture Friday. 2 /

More information

6.034f Neural Net Notes October 28, 2010

6.034f Neural Net Notes October 28, 2010 6.034f Neural Net Notes October 28, 2010 These notes are a supplement to material presented in lecture. I lay out the mathematics more prettily and etend the analysis to handle multiple-neurons per layer.

More information

Multilayer Neural Networks

Multilayer Neural Networks Pattern Recognition Multilaer Neural Networs Lecture 4 Prof. Daniel Yeung School of Computer Science and Engineering South China Universit of Technolog Outline Introduction (6.) Artificial Neural Networ

More information

Introduction to System Identification and Adaptive Control

Introduction to System Identification and Adaptive Control Introduction to System Identification and Adaptive Control A. Khaki Sedigh Control Systems Group Faculty of Electrical and Computer Engineering K. N. Toosi University of Technology May 2009 Introduction

More information

Defining Feedforward Network Architecture. net = newff([pn],[s1 S2... SN],{TF1 TF2... TFN},BTF,LF,PF);

Defining Feedforward Network Architecture. net = newff([pn],[s1 S2... SN],{TF1 TF2... TFN},BTF,LF,PF); Appendix D MATLAB Programs for Neural Systems D.. Defining Feedforward Network Architecture Feedforward networks often have one or more hidden layers of sigmoid neurons followed by an output layer of linear

More information

NONLINEAR IDENTIFICATION ON BASED RBF NEURAL NETWORK

NONLINEAR IDENTIFICATION ON BASED RBF NEURAL NETWORK DAAAM INTERNATIONAL SCIENTIFIC BOOK 2011 pp. 547-554 CHAPTER 44 NONLINEAR IDENTIFICATION ON BASED RBF NEURAL NETWORK BURLAK, V. & PIVONKA, P. Abstract: This article is focused on the off-line identification

More information

7. Choice of approximating for the FE-method scalar problems

7. Choice of approximating for the FE-method scalar problems 7. Choice of approimating for the FE-method scalar problems Finite Element Method Differential Equation Weak Formulation Approimating Functions Weighted Residuals FEM - Formulation Weak form of heat flow

More information

COMS 4771 Introduction to Machine Learning. Nakul Verma

COMS 4771 Introduction to Machine Learning. Nakul Verma COMS 4771 Introduction to Machine Learning Nakul Verma Announcements HW1 due next lecture Project details are available decide on the group and topic by Thursday Last time Generative vs. Discriminative

More information

Radial Basis Function Networks. Ravi Kaushik Project 1 CSC Neural Networks and Pattern Recognition

Radial Basis Function Networks. Ravi Kaushik Project 1 CSC Neural Networks and Pattern Recognition Radial Basis Function Networks Ravi Kaushik Project 1 CSC 84010 Neural Networks and Pattern Recognition History Radial Basis Function (RBF) emerged in late 1980 s as a variant of artificial neural network.

More information

Generator Thermal Sensitivity Analysis with Support Vector Regression

Generator Thermal Sensitivity Analysis with Support Vector Regression 2 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July 2, 2 WeB5.3 Generator Thermal Sensitivity Analysis with Support Vector Regression Youliang Yang and Qing Zhao* Abstract

More information

8.1 Exponents and Roots

8.1 Exponents and Roots Section 8. Eponents and Roots 75 8. Eponents and Roots Before defining the net famil of functions, the eponential functions, we will need to discuss eponent notation in detail. As we shall see, eponents

More information

X/94 $ IEEE 1894

X/94 $ IEEE 1894 I Implementing Radial Basis Functions Using Bump-Resistor Networks John G. Harris University of Florida EE Dept., 436 CSE Bldg 42 Gainesville, FL 3261 1 harris@j upit er.ee.ufl.edu Abstract- Radial Basis

More information

Lecture 3 Feedforward Networks and Backpropagation

Lecture 3 Feedforward Networks and Backpropagation Lecture 3 Feedforward Networks and Backpropagation CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago April 3, 2017 Things we will look at today Recap of Logistic Regression

More information

Applications of Proper Orthogonal Decomposition for Inviscid Transonic Aerodynamics

Applications of Proper Orthogonal Decomposition for Inviscid Transonic Aerodynamics Applications of Proper Orthogonal Decomposition for Inviscid Transonic Aerodnamics Bui-Thanh Tan, Karen Willco and Murali Damodaran Abstract Two etensions to the proper orthogonal decomposition (POD) technique

More information

Artificial Neural Networks. Edward Gatt

Artificial Neural Networks. Edward Gatt Artificial Neural Networks Edward Gatt What are Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning Very

More information

Broken Extremals. Variational Methods & Optimal Control lecture 20. Broken extremals. Broken extremals

Broken Extremals. Variational Methods & Optimal Control lecture 20. Broken extremals. Broken extremals Broken etremals Variational Methods & Optimal Control lecture 2 Matthew Roughan Discipline of Applied Mathematics School of Mathematical Sciences Universit of Adelaide

More information

14.1 Systems of Linear Equations in Two Variables

14.1 Systems of Linear Equations in Two Variables 86 Chapter 1 Sstems of Equations and Matrices 1.1 Sstems of Linear Equations in Two Variables Use the method of substitution to solve sstems of equations in two variables. Use the method of elimination

More information

Presentation Topic 1: Feedback Control. Copyright 1998 DLMattern

Presentation Topic 1: Feedback Control. Copyright 1998 DLMattern Presentation Topic 1: Feedback Control Outline Feedback Terminology Purpose of Feedback Limitations of Feedback Linear Control Design Techniques Nonlinear Control Design Techniques Rapid Prototyping Environments

More information

More on Neural Networks

More on Neural Networks More on Neural Networks Yujia Yan Fall 2018 Outline Linear Regression y = Wx + b (1) Linear Regression y = Wx + b (1) Polynomial Regression y = Wφ(x) + b (2) where φ(x) gives the polynomial basis, e.g.,

More information

Simplified Fast Multipole Methods for Micromagnetic Modeling

Simplified Fast Multipole Methods for Micromagnetic Modeling implified Fast Multipole Methods for Micromagnetic Modeling D. M. Apalkov and P. B.Visscher Department of Phsics and Astronom and MIT Center The Universit of Alabama This project was supported b F grants

More information

Machine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler

Machine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler + Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Prof. Alexander Ihler Linear Classifiers (Perceptrons) Linear Classifiers a linear classifier is a mapping which partitions

More information

y(x n, w) t n 2. (1)

y(x n, w) t n 2. (1) Network training: Training a neural network involves determining the weight parameter vector w that minimizes a cost function. Given a training set comprising a set of input vector {x n }, n = 1,...N,

More information

Advanced Machine Learning

Advanced Machine Learning Advanced Machine Learning Lecture 4: Deep Learning Essentials Pierre Geurts, Gilles Louppe, Louis Wehenkel 1 / 52 Outline Goal: explain and motivate the basic constructs of neural networks. From linear

More information

ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD

ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD WHAT IS A NEURAL NETWORK? The simplest definition of a neural network, more properly referred to as an 'artificial' neural network (ANN), is provided

More information

Introduction to. Process Control. Ahmet Palazoglu. Second Edition. Jose A. Romagnoli. CRC Press. Taylor & Francis Group. Taylor & Francis Group,

Introduction to. Process Control. Ahmet Palazoglu. Second Edition. Jose A. Romagnoli. CRC Press. Taylor & Francis Group. Taylor & Francis Group, Introduction to Process Control Second Edition Jose A. Romagnoli Ahmet Palazoglu CRC Press Taylor & Francis Group Boca Raton London NewYork CRC Press is an imprint of the Taylor & Francis Group, an informa

More information

Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td

Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td Data Mining Andrew Kusiak 2139 Seamans Center Iowa City, Iowa 52242-1527 Preamble: Control Application Goal: Maintain T ~Td Tel: 319-335 5934 Fax: 319-335 5669 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak

More information

Revision: Neural Network

Revision: Neural Network Revision: Neural Network Exercise 1 Tell whether each of the following statements is true or false by checking the appropriate box. Statement True False a) A perceptron is guaranteed to perfectly learn

More information

NEAR-OPTIMAL FLIGHT LOAD SYNTHESIS USING NEURAL NETS

NEAR-OPTIMAL FLIGHT LOAD SYNTHESIS USING NEURAL NETS NEAR-OPTIMAL FLIGHT LOAD SYNTHESIS USING NEURAL NETS Michael T. Manry, Cheng-Hsiung Hsieh, and Hema Chandrasekaran Department of Electrical Engineering University of Texas at Arlington Arlington, Texas

More information

LECTURE # - NEURAL COMPUTATION, Feb 04, Linear Regression. x 1 θ 1 output... θ M x M. Assumes a functional form

LECTURE # - NEURAL COMPUTATION, Feb 04, Linear Regression. x 1 θ 1 output... θ M x M. Assumes a functional form LECTURE # - EURAL COPUTATIO, Feb 4, 4 Linear Regression Assumes a functional form f (, θ) = θ θ θ K θ (Eq) where = (,, ) are the attributes and θ = (θ, θ, θ ) are the function parameters Eample: f (, θ)

More information

DETERMINATION OF HORIZONTAL AXIS WIND TURBINE PERFORMANCE IN YAW BY USE OF SIMPLIFIED VORTEX THEORY

DETERMINATION OF HORIZONTAL AXIS WIND TURBINE PERFORMANCE IN YAW BY USE OF SIMPLIFIED VORTEX THEORY 5 th INTERNATIONAL MULTIDISCIPLINARY CONFERENCE DETERMINATION OF HORIZONTAL AXIS WIND TURBINE PERFORMANCE IN YAW BY USE OF SIMPLIFIED ORTEX THEORY Piotr Strzelczk, PhD Eng. Dept. Of Fluid Mechanics and

More information

VCAA Bulletin. VCE, VCAL and VET. Supplement 2

VCAA Bulletin. VCE, VCAL and VET. Supplement 2 VCE, VCAL and VET VCAA Bulletin Regulations and information about curriculum and assessment for the VCE, VCAL and VET No. 87 April 20 Victorian Certificate of Education Victorian Certificate of Applied

More information

A Multiresolution Approach for Modeling of Diffusion Phenomenon

A Multiresolution Approach for Modeling of Diffusion Phenomenon A Multiresolution Approach for Modeling of Diffusion Phenomenon Reza Madankan a Puneet Singla a Tarunraj Singh a Peter D. Scott b rm93@buffalo.edu psingla@buffalo.edu tsingh@buffalo.edu peter@buffalo.edu

More information

Three Tank System Control Using Neuro - Fuzzy Model Predictive Control

Three Tank System Control Using Neuro - Fuzzy Model Predictive Control Three Tank Sstem Control Using Neuro - Fuzz Model Predictive Control Adel Abdurahman Mechanical Engineering Department, Facult of Engineering, Sana a Universit ABSTRACT Three-tank (3T) sstem is the most

More information

In the Name of God. Lectures 15&16: Radial Basis Function Networks

In the Name of God. Lectures 15&16: Radial Basis Function Networks 1 In the Name of God Lectures 15&16: Radial Basis Function Networks Some Historical Notes Learning is equivalent to finding a surface in a multidimensional space that provides a best fit to the training

More information

Using Intercept Form

Using Intercept Form 8.5 Using Intercept Form Essential Question What are some of the characteristics of the graph of f () = a( p)( q)? Using Zeros to Write Functions Work with a partner. Each graph represents a function of

More information

Time Series Analysis for Quality Improvement: a Soft Computing Approach

Time Series Analysis for Quality Improvement: a Soft Computing Approach ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN -9337-4-8, pp. 19-114 ime Series Analsis or Qualit Improvement: a Sot Computing Approach

More information

CMSC 421: Neural Computation. Applications of Neural Networks

CMSC 421: Neural Computation. Applications of Neural Networks CMSC 42: Neural Computation definition synonyms neural networks artificial neural networks neural modeling connectionist models parallel distributed processing AI perspective Applications of Neural Networks

More information

PROPORTIONAL-Integral-Derivative (PID) controllers

PROPORTIONAL-Integral-Derivative (PID) controllers Multiple Model and Neural based Adaptive Multi-loop PID Controller for a CSTR Process R.Vinodha S. Abraham Lincoln and J. Prakash Abstract Multi-loop (De-centralized) Proportional-Integral- Derivative

More information

ADAPTIVE NEURO-FUZZY INFERENCE SYSTEMS

ADAPTIVE NEURO-FUZZY INFERENCE SYSTEMS ADAPTIVE NEURO-FUZZY INFERENCE SYSTEMS RBFN and TS systems Equivalent if the following hold: Both RBFN and TS use same aggregation method for output (weighted sum or weighted average) Number of basis functions

More information

Lecture 3 Feedforward Networks and Backpropagation

Lecture 3 Feedforward Networks and Backpropagation Lecture 3 Feedforward Networks and Backpropagation CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago April 3, 2017 Things we will look at today Recap of Logistic Regression

More information

ECE Introduction to Artificial Neural Network and Fuzzy Systems

ECE Introduction to Artificial Neural Network and Fuzzy Systems ECE 39 - Introduction to Artificial Neural Network and Fuzzy Systems Wavelet Neural Network control of two Continuous Stirred Tank Reactors in Series using MATLAB Tariq Ahamed Abstract. With the rapid

More information

Learning From Data Lecture 7 Approximation Versus Generalization

Learning From Data Lecture 7 Approximation Versus Generalization Learning From Data Lecture 7 Approimation Versus Generalization The VC Dimension Approimation Versus Generalization Bias and Variance The Learning Curve M. Magdon-Ismail CSCI 4100/6100 recap: The Vapnik-Chervonenkis

More information

Data Mining. 3.6 Regression Analysis. Fall Instructor: Dr. Masoud Yaghini. Numeric Prediction

Data Mining. 3.6 Regression Analysis. Fall Instructor: Dr. Masoud Yaghini. Numeric Prediction Data Mining 3.6 Regression Analysis Fall 2008 Instructor: Dr. Masoud Yaghini Outline Introduction Straight-Line Linear Regression Multiple Linear Regression Other Regression Models References Introduction

More information

Chapter 6 2D Elements Plate Elements

Chapter 6 2D Elements Plate Elements Institute of Structural Engineering Page 1 Chapter 6 2D Elements Plate Elements Method of Finite Elements I Institute of Structural Engineering Page 2 Continuum Elements Plane Stress Plane Strain Toda

More information

Speech and Language Processing

Speech and Language Processing Speech and Language Processing Lecture 5 Neural network based acoustic and language models Information and Communications Engineering Course Takahiro Shinoaki 08//6 Lecture Plan (Shinoaki s part) I gives

More information

Ch.8 Neural Networks

Ch.8 Neural Networks Ch.8 Neural Networks Hantao Zhang http://www.cs.uiowa.edu/ hzhang/c145 The University of Iowa Department of Computer Science Artificial Intelligence p.1/?? Brains as Computational Devices Motivation: Algorithms

More information

Neuro-Fuzzy Comp. Ch. 4 March 24, R p

Neuro-Fuzzy Comp. Ch. 4 March 24, R p 4 Feedforward Multilayer Neural Networks part I Feedforward multilayer neural networks (introduced in sec 17) with supervised error correcting learning are used to approximate (synthesise) a non-linear

More information

Ready To Go On? Skills Intervention 6-1 Polynomials

Ready To Go On? Skills Intervention 6-1 Polynomials 6A Read To Go On? Skills Intervention 6- Polnomials Find these vocabular words in Lesson 6- and the Multilingual Glossar. Vocabular monomial polnomial degree of a monomial degree of a polnomial leading

More information

L20: MLPs, RBFs and SPR Bayes discriminants and MLPs The role of MLP hidden units Bayes discriminants and RBFs Comparison between MLPs and RBFs

L20: MLPs, RBFs and SPR Bayes discriminants and MLPs The role of MLP hidden units Bayes discriminants and RBFs Comparison between MLPs and RBFs L0: MLPs, RBFs and SPR Bayes discriminants and MLPs The role of MLP hidden units Bayes discriminants and RBFs Comparison between MLPs and RBFs CSCE 666 Pattern Analysis Ricardo Gutierrez-Osuna CSE@TAMU

More information

Artificial Neural Networks 2

Artificial Neural Networks 2 CSC2515 Machine Learning Sam Roweis Artificial Neural s 2 We saw neural nets for classification. Same idea for regression. ANNs are just adaptive basis regression machines of the form: y k = j w kj σ(b

More information

Trajectory Planning, Setpoint Generation and Feedforward for Motion Systems

Trajectory Planning, Setpoint Generation and Feedforward for Motion Systems 2 Trajectory Planning, Setpoint Generation and Feedforward for Motion Systems Paul Lambrechts Digital Motion Control (4K4), 23 Faculty of Mechanical Engineering, Control Systems Technology Group /42 2

More information

Introduction to Natural Computation. Lecture 9. Multilayer Perceptrons and Backpropagation. Peter Lewis

Introduction to Natural Computation. Lecture 9. Multilayer Perceptrons and Backpropagation. Peter Lewis Introduction to Natural Computation Lecture 9 Multilayer Perceptrons and Backpropagation Peter Lewis 1 / 25 Overview of the Lecture Why multilayer perceptrons? Some applications of multilayer perceptrons.

More information